2024-09-10T23:16:56.815 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T23:16:56.836 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T23:16:56.929 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899790 branch: wip-hemanth-testing-2024-09-10-0723 description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} email: hyelloji@redhat.com first_in_suite: false job_id: '7899790' kernel: flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: client: pid file: '' mgr: client mount timeout: 30 debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6190 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 479/1024 suite: fs suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHiZMHMb7Bu+JC8d8aMhd1mPsXSyPurOEKbBVPuiRmYuZnzwYV7vYkNWTUVMnbRgkiQhTnz0J+nzYi5VJSCfabw= smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNKDdux8OkcgVO74ps3unEtezdTyoqM9GSd/vWinp22tgf5qlxMrAOcFUTbE0P5jRsi2OVR3js8SBLntcJU1YH8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_16:59:47 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T23:16:56.930 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-10T23:16:56.930 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-10T23:16:56.930 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T23:16:56.932 INFO:teuthology.task.internal:Checking packages... 2024-09-10T23:16:56.952 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-10T23:16:56.953 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T23:16:56.953 INFO:teuthology.packaging:ref: None 2024-09-10T23:16:56.953 INFO:teuthology.packaging:tag: None 2024-09-10T23:16:56.953 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-10T23:16:56.953 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:16:56.953 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-10T23:16:57.113 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854-g8f219c47-1jammy 2024-09-10T23:16:57.115 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T23:16:57.120 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T23:16:57.120 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T23:16:57.127 INFO:teuthology.task.internal:Saving configuration 2024-09-10T23:16:57.144 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T23:16:57.149 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T23:16:57.184 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899790', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 23:11:26.005896', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICRuDJjCUvhHs+/C4BhB7KUoN5hCKWol6QhhlzOFzNKI'} 2024-09-10T23:16:57.215 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899790', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 23:11:26.006850', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINZlvCrj6drELPmPOV1XSnqLTbmUjpKbPFXQOZn0GagF'} 2024-09-10T23:16:57.215 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T23:16:57.220 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-10T23:16:57.221 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-10T23:16:57.221 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T23:16:57.285 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f23c3b67be0>, signals=[15]) 2024-09-10T23:16:57.286 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T23:16:57.291 INFO:teuthology.task.internal:Opening connections... 2024-09-10T23:16:57.292 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:16:57.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:16:57.383 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:16:57.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:16:57.471 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T23:16:57.477 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-09-10T23:16:57.482 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-09-10T23:16:57.482 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-09-10T23:16:57.530 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-10T23:16:57.530 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="Ubuntu" 2024-09-10T23:16:57.530 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="22.04" 2024-09-10T23:16:57.531 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-10T23:16:57.531 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_CODENAME=jammy 2024-09-10T23:16:57.531 INFO:teuthology.orchestra.run.smithi094.stdout:ID=ubuntu 2024-09-10T23:16:57.531 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE=debian 2024-09-10T23:16:57.531 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-10T23:16:57.531 INFO:teuthology.orchestra.run.smithi094.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-10T23:16:57.531 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-10T23:16:57.531 INFO:teuthology.orchestra.run.smithi094.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-10T23:16:57.531 INFO:teuthology.orchestra.run.smithi094.stdout:UBUNTU_CODENAME=jammy 2024-09-10T23:16:57.532 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-09-10T23:16:57.560 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2024-09-10T23:16:57.569 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2024-09-10T23:16:57.569 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2024-09-10T23:16:57.618 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-10T23:16:57.618 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Ubuntu" 2024-09-10T23:16:57.618 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="22.04" 2024-09-10T23:16:57.618 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-10T23:16:57.618 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_CODENAME=jammy 2024-09-10T23:16:57.618 INFO:teuthology.orchestra.run.smithi181.stdout:ID=ubuntu 2024-09-10T23:16:57.618 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE=debian 2024-09-10T23:16:57.618 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-10T23:16:57.618 INFO:teuthology.orchestra.run.smithi181.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-10T23:16:57.619 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-10T23:16:57.619 INFO:teuthology.orchestra.run.smithi181.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-10T23:16:57.619 INFO:teuthology.orchestra.run.smithi181.stdout:UBUNTU_CODENAME=jammy 2024-09-10T23:16:57.619 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2024-09-10T23:16:57.652 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T23:16:57.660 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T23:16:57.665 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T23:16:57.665 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-09-10T23:16:57.667 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2024-09-10T23:16:57.671 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T23:16:57.677 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T23:16:57.677 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-09-10T23:16:57.716 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2024-09-10T23:16:57.731 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T23:16:57.802 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T23:16:57.816 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-10T23:16:57.816 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'mon.b': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}}, timeout 300 2024-09-10T23:16:57.816 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T23:16:57.817 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T23:16:57.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:16:57.821 DEBUG:teuthology.task.kernel:sha1 for mon.a is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T23:16:57.821 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T23:16:57.821 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-09-10T23:16:57.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:16:57.823 DEBUG:teuthology.task.kernel:sha1 for mon.b is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T23:16:57.823 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T23:16:57.823 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2024-09-10T23:16:57.870 INFO:teuthology.orchestra.run.smithi181.stdout:5.15.0-56-generic 2024-09-10T23:16:57.870 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T23:16:57.870 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T23:16:57.871 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T23:16:57.882 INFO:teuthology.orchestra.run.smithi094.stdout:5.15.0-56-generic 2024-09-10T23:16:57.883 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T23:16:57.883 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T23:16:57.883 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T23:16:58.023 INFO:teuthology.task.kernel:Installing linux-firmware on mon.b... 2024-09-10T23:16:58.023 DEBUG:teuthology.orchestra.run.smithi181:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-10T23:16:58.044 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-10T23:16:58.045 INFO:teuthology.task.kernel:Installing linux-firmware on mon.a... 2024-09-10T23:16:58.045 DEBUG:teuthology.orchestra.run.smithi094:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-10T23:16:58.065 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-10T23:16:58.182 INFO:teuthology.orchestra.run.smithi181.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-10T23:16:58.182 INFO:teuthology.orchestra.run.smithi181.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-10T23:16:58.182 INFO:teuthology.orchestra.run.smithi181.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-10T23:16:58.182 INFO:teuthology.orchestra.run.smithi181.stderr:hint: 2024-09-10T23:16:58.182 INFO:teuthology.orchestra.run.smithi181.stderr:hint: git config --global init.defaultBranch 2024-09-10T23:16:58.182 INFO:teuthology.orchestra.run.smithi181.stderr:hint: 2024-09-10T23:16:58.182 INFO:teuthology.orchestra.run.smithi181.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-10T23:16:58.182 INFO:teuthology.orchestra.run.smithi181.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-10T23:16:58.182 INFO:teuthology.orchestra.run.smithi181.stderr:hint: 2024-09-10T23:16:58.183 INFO:teuthology.orchestra.run.smithi181.stderr:hint: git branch -m 2024-09-10T23:16:58.183 INFO:teuthology.orchestra.run.smithi181.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-10T23:16:58.184 DEBUG:teuthology.orchestra.run.smithi181:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-10T23:16:58.186 INFO:teuthology.orchestra.run.smithi094.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-10T23:16:58.186 INFO:teuthology.orchestra.run.smithi094.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-10T23:16:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-10T23:16:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:hint: 2024-09-10T23:16:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:hint: git config --global init.defaultBranch 2024-09-10T23:16:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:hint: 2024-09-10T23:16:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-10T23:16:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-10T23:16:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:hint: 2024-09-10T23:16:58.187 INFO:teuthology.orchestra.run.smithi094.stderr:hint: git branch -m 2024-09-10T23:16:58.188 INFO:teuthology.orchestra.run.smithi094.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-10T23:16:58.190 DEBUG:teuthology.orchestra.run.smithi094:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-10T23:16:58.252 DEBUG:teuthology.orchestra.run.smithi181:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-10T23:16:58.267 DEBUG:teuthology.orchestra.run.smithi094:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-10T23:16:58.310 DEBUG:teuthology.orchestra.run.smithi181:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-10T23:16:58.324 DEBUG:teuthology.orchestra.run.smithi094:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-10T23:17:55.080 INFO:teuthology.orchestra.run.smithi094.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-10T23:17:55.081 INFO:teuthology.orchestra.run.smithi094.stderr: * [new branch] main -> origin/main 2024-09-10T23:17:55.116 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190312 -> 20190312 2024-09-10T23:17:55.117 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190416 -> 20190416 2024-09-10T23:17:55.117 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190514 -> 20190514 2024-09-10T23:17:55.117 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190618 -> 20190618 2024-09-10T23:17:55.117 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190717 -> 20190717 2024-09-10T23:17:55.117 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190815 -> 20190815 2024-09-10T23:17:55.117 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20190923 -> 20190923 2024-09-10T23:17:55.117 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20191022 -> 20191022 2024-09-10T23:17:55.117 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20191215 -> 20191215 2024-09-10T23:17:55.117 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200122 -> 20200122 2024-09-10T23:17:55.120 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200316 -> 20200316 2024-09-10T23:17:55.120 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200421 -> 20200421 2024-09-10T23:17:55.121 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200519 -> 20200519 2024-09-10T23:17:55.121 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200619 -> 20200619 2024-09-10T23:17:55.121 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200721 -> 20200721 2024-09-10T23:17:55.121 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200817 -> 20200817 2024-09-10T23:17:55.121 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20200918 -> 20200918 2024-09-10T23:17:55.121 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20201022 -> 20201022 2024-09-10T23:17:55.121 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20201118 -> 20201118 2024-09-10T23:17:55.126 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20201218 -> 20201218 2024-09-10T23:17:55.126 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210208 -> 20210208 2024-09-10T23:17:55.126 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210315 -> 20210315 2024-09-10T23:17:55.126 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210511 -> 20210511 2024-09-10T23:17:55.126 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210716 -> 20210716 2024-09-10T23:17:55.126 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210818 -> 20210818 2024-09-10T23:17:55.126 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20210919 -> 20210919 2024-09-10T23:17:55.126 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20211027 -> 20211027 2024-09-10T23:17:55.126 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20211216 -> 20211216 2024-09-10T23:17:55.126 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220209 -> 20220209 2024-09-10T23:17:55.126 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220310 -> 20220310 2024-09-10T23:17:55.126 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220411 -> 20220411 2024-09-10T23:17:55.127 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220509 -> 20220509 2024-09-10T23:17:55.127 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220610 -> 20220610 2024-09-10T23:17:55.127 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220708 -> 20220708 2024-09-10T23:17:55.127 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220815 -> 20220815 2024-09-10T23:17:55.127 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20220913 -> 20220913 2024-09-10T23:17:55.127 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20221012 -> 20221012 2024-09-10T23:17:55.127 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20221109 -> 20221109 2024-09-10T23:17:55.129 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20221214 -> 20221214 2024-09-10T23:17:55.130 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230117 -> 20230117 2024-09-10T23:17:55.130 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230210 -> 20230210 2024-09-10T23:17:55.130 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230310 -> 20230310 2024-09-10T23:17:55.130 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230404 -> 20230404 2024-09-10T23:17:55.130 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230515 -> 20230515 2024-09-10T23:17:55.130 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230625 -> 20230625 2024-09-10T23:17:55.130 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230804 -> 20230804 2024-09-10T23:17:55.130 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20230919 -> 20230919 2024-09-10T23:17:55.130 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20231030 -> 20231030 2024-09-10T23:17:55.130 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20231111 -> 20231111 2024-09-10T23:17:55.130 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20231211 -> 20231211 2024-09-10T23:17:55.130 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240115 -> 20240115 2024-09-10T23:17:55.130 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240220 -> 20240220 2024-09-10T23:17:55.130 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240312 -> 20240312 2024-09-10T23:17:55.131 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240410 -> 20240410 2024-09-10T23:17:55.131 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240513 -> 20240513 2024-09-10T23:17:55.131 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240610 -> 20240610 2024-09-10T23:17:55.131 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240709 -> 20240709 2024-09-10T23:17:55.131 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240811 -> 20240811 2024-09-10T23:17:55.131 INFO:teuthology.orchestra.run.smithi094.stderr: * [new tag] 20240909 -> 20240909 2024-09-10T23:17:57.118 INFO:teuthology.orchestra.run.smithi181.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-10T23:17:57.118 INFO:teuthology.orchestra.run.smithi181.stderr: * [new branch] main -> origin/main 2024-09-10T23:17:57.155 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20190312 -> 20190312 2024-09-10T23:17:57.155 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20190416 -> 20190416 2024-09-10T23:17:57.155 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20190514 -> 20190514 2024-09-10T23:17:57.155 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20190618 -> 20190618 2024-09-10T23:17:57.155 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20190717 -> 20190717 2024-09-10T23:17:57.155 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20190815 -> 20190815 2024-09-10T23:17:57.155 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20190923 -> 20190923 2024-09-10T23:17:57.157 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20191022 -> 20191022 2024-09-10T23:17:57.157 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20191215 -> 20191215 2024-09-10T23:17:57.158 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20200122 -> 20200122 2024-09-10T23:17:57.158 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20200316 -> 20200316 2024-09-10T23:17:57.158 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20200421 -> 20200421 2024-09-10T23:17:57.158 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20200519 -> 20200519 2024-09-10T23:17:57.158 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20200619 -> 20200619 2024-09-10T23:17:57.158 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20200721 -> 20200721 2024-09-10T23:17:57.158 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20200817 -> 20200817 2024-09-10T23:17:57.158 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20200918 -> 20200918 2024-09-10T23:17:57.158 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20201022 -> 20201022 2024-09-10T23:17:57.158 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20201118 -> 20201118 2024-09-10T23:17:57.158 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20201218 -> 20201218 2024-09-10T23:17:57.158 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20210208 -> 20210208 2024-09-10T23:17:57.158 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20210315 -> 20210315 2024-09-10T23:17:57.158 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20210511 -> 20210511 2024-09-10T23:17:57.161 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20210716 -> 20210716 2024-09-10T23:17:57.161 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20210818 -> 20210818 2024-09-10T23:17:57.161 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20210919 -> 20210919 2024-09-10T23:17:57.161 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20211027 -> 20211027 2024-09-10T23:17:57.161 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20211216 -> 20211216 2024-09-10T23:17:57.161 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20220209 -> 20220209 2024-09-10T23:17:57.161 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20220310 -> 20220310 2024-09-10T23:17:57.161 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20220411 -> 20220411 2024-09-10T23:17:57.161 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20220509 -> 20220509 2024-09-10T23:17:57.161 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20220610 -> 20220610 2024-09-10T23:17:57.162 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20220708 -> 20220708 2024-09-10T23:17:57.162 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20220815 -> 20220815 2024-09-10T23:17:57.162 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20220913 -> 20220913 2024-09-10T23:17:57.162 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20221012 -> 20221012 2024-09-10T23:17:57.162 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20221109 -> 20221109 2024-09-10T23:17:57.162 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20221214 -> 20221214 2024-09-10T23:17:57.162 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20230117 -> 20230117 2024-09-10T23:17:57.162 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20230210 -> 20230210 2024-09-10T23:17:57.162 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20230310 -> 20230310 2024-09-10T23:17:57.162 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20230404 -> 20230404 2024-09-10T23:17:57.162 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20230515 -> 20230515 2024-09-10T23:17:57.162 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20230625 -> 20230625 2024-09-10T23:17:57.164 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20230804 -> 20230804 2024-09-10T23:17:57.164 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20230919 -> 20230919 2024-09-10T23:17:57.164 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20231030 -> 20231030 2024-09-10T23:17:57.164 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20231111 -> 20231111 2024-09-10T23:17:57.164 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20231211 -> 20231211 2024-09-10T23:17:57.164 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20240115 -> 20240115 2024-09-10T23:17:57.164 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20240220 -> 20240220 2024-09-10T23:17:57.165 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20240312 -> 20240312 2024-09-10T23:17:57.165 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20240410 -> 20240410 2024-09-10T23:17:57.165 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20240513 -> 20240513 2024-09-10T23:17:57.165 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20240610 -> 20240610 2024-09-10T23:17:57.165 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20240709 -> 20240709 2024-09-10T23:17:57.165 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20240811 -> 20240811 2024-09-10T23:17:57.165 INFO:teuthology.orchestra.run.smithi181.stderr: * [new tag] 20240909 -> 20240909 2024-09-10T23:17:59.628 INFO:teuthology.orchestra.run.smithi094.stderr:Updating files: 44% (1563/3501) Updating files: 45% (1576/3501) Updating files: 46% (1611/3501) Updating files: 47% (1646/3501) Updating files: 48% (1681/3501) Updating files: 49% (1716/3501) Updating files: 50% (1751/3501) Updating files: 51% (1786/3501) Updating files: 52% (1821/3501) Updating files: 53% (1856/3501) Updating files: 54% (1891/3501) Updating files: 55% (1926/3501) Updating files: 56% (1961/3501) Updating files: 57% (1996/3501) Updating files: 58% (2031/3501) Updating files: 59% (2066/3501) Updating files: 59% (2069/3501) Updating files: 60% (2101/3501) Updating files: 61% (2136/3501) Updating files: 62% (2171/3501) Updating files: 63% (2206/3501) Updating files: 64% (2241/3501) Updating files: 65% (2276/3501) Updating files: 66% (2311/3501) Updating files: 67% (2346/3501) Updating files: 68% (2381/3501) Updating files: 69% (2416/3501) Updating files: 69% (2427/3501) Updating files: 70% (2451/3501) Updating files: 71% (2486/3501) Updating files: 72% (2521/3501) Updating files: 73% (2556/3501) Updating files: 74% (2591/3501) Updating files: 75% (2626/3501) Updating files: 76% (2661/3501) Updating files: 77% (2696/3501) Updating files: 78% (2731/3501) Updating files: 79% (2766/3501) Updating files: 80% (2801/3501) Updating files: 81% (2836/3501) Updating files: 82% (2871/3501) Updating files: 82% (2891/3501) Updating files: 83% (2906/3501) Updating files: 84% (2941/3501) Updating files: 85% (2976/3501) Updating files: 86% (3011/3501) Updating files: 87% (3046/3501) Updating files: 88% (3081/3501) Updating files: 89% (3116/3501) Updating files: 90% (3151/3501) Updating files: 91% (3186/3501) Updating files: 92% (3221/3501) Updating files: 93% (3256/3501) Updating files: 94% (3291/3501) Updating files: 95% (3326/3501) Updating files: 96% (3361/3501) Updating files: 97% (3396/3501) Updating files: 98% (3431/3501) Updating files: 99% (3466/3501) Updating files: 100% (3501/3501) Updating files: 100% (3501/3501), done. 2024-09-10T23:17:59.632 INFO:teuthology.orchestra.run.smithi094.stdout:HEAD is now at ce299c0f Merge branch 'rt1320' into 'main' 2024-09-10T23:17:59.652 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.a... 2024-09-10T23:17:59.653 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T23:17:59.802 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T23:17:59.802 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-10T23:17:59.805 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi094.front.sepia.ceph.com to complete... 2024-09-10T23:18:00.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10 23:18:00 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-10T23:18:00.781 INFO:teuthology.orchestra.run.smithi094.stderr:FINISHED --2024-09-10 23:18:00-- 2024-09-10T23:18:00.781 INFO:teuthology.orchestra.run.smithi094.stderr:Total wall clock time: 1.0s 2024-09-10T23:18:00.781 INFO:teuthology.orchestra.run.smithi094.stderr:Downloaded: 1 files, 43M in 0.8s (51.8 MB/s) 2024-09-10T23:18:00.785 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.a... 2024-09-10T23:18:00.785 DEBUG:teuthology.orchestra.run.smithi094:> sudo dpkg -i /tmp/linux-image.deb 2024-09-10T23:18:01.066 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-10T23:18:01.691 INFO:teuthology.orchestra.run.smithi181.stderr:Updating files: 37% (1298/3501) Updating files: 38% (1331/3501) Updating files: 39% (1366/3501) Updating files: 40% (1401/3501) Updating files: 41% (1436/3501) Updating files: 42% (1471/3501) Updating files: 43% (1506/3501) Updating files: 44% (1541/3501) Updating files: 45% (1576/3501) Updating files: 46% (1611/3501) Updating files: 47% (1646/3501) Updating files: 48% (1681/3501) Updating files: 49% (1716/3501) Updating files: 50% (1751/3501) Updating files: 51% (1786/3501) Updating files: 52% (1821/3501) Updating files: 53% (1856/3501) Updating files: 54% (1891/3501) Updating files: 55% (1926/3501) Updating files: 56% (1961/3501) Updating files: 57% (1996/3501) Updating files: 58% (2031/3501) Updating files: 58% (2063/3501) Updating files: 59% (2066/3501) Updating files: 60% (2101/3501) Updating files: 61% (2136/3501) Updating files: 62% (2171/3501) Updating files: 63% (2206/3501) Updating files: 64% (2241/3501) Updating files: 65% (2276/3501) Updating files: 66% (2311/3501) Updating files: 67% (2346/3501) Updating files: 68% (2381/3501) Updating files: 69% (2416/3501) Updating files: 69% (2423/3501) Updating files: 70% (2451/3501) Updating files: 71% (2486/3501) Updating files: 72% (2521/3501) Updating files: 73% (2556/3501) Updating files: 74% (2591/3501) Updating files: 75% (2626/3501) Updating files: 76% (2661/3501) Updating files: 77% (2696/3501) Updating files: 78% (2731/3501) Updating files: 79% (2766/3501) Updating files: 80% (2801/3501) Updating files: 81% (2836/3501) Updating files: 82% (2871/3501) Updating files: 82% (2891/3501) Updating files: 83% (2906/3501) Updating files: 84% (2941/3501) Updating files: 85% (2976/3501) Updating files: 86% (3011/3501) Updating files: 87% (3046/3501) Updating files: 88% (3081/3501) Updating files: 89% (3116/3501) Updating files: 90% (3151/3501) Updating files: 91% (3186/3501) Updating files: 92% (3221/3501) Updating files: 93% (3256/3501) Updating files: 94% (3291/3501) Updating files: 95% (3326/3501) Updating files: 96% (3361/3501) Updating files: 97% (3396/3501) Updating files: 98% (3431/3501) Updating files: 99% (3466/3501) Updating files: 100% (3501/3501) Updating files: 100% (3501/3501), done. 2024-09-10T23:18:01.696 INFO:teuthology.orchestra.run.smithi181.stdout:HEAD is now at ce299c0f Merge branch 'rt1320' into 'main' 2024-09-10T23:18:01.718 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.b... 2024-09-10T23:18:01.718 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T23:18:01.867 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T23:18:01.867 DEBUG:teuthology.orchestra.run.smithi181:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-10T23:18:01.869 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi181.front.sepia.ceph.com to complete... 2024-09-10T23:18:03.069 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10 23:18:02 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-10T23:18:03.070 INFO:teuthology.orchestra.run.smithi181.stderr:FINISHED --2024-09-10 23:18:02-- 2024-09-10T23:18:03.070 INFO:teuthology.orchestra.run.smithi181.stderr:Total wall clock time: 1.0s 2024-09-10T23:18:03.070 INFO:teuthology.orchestra.run.smithi181.stderr:Downloaded: 1 files, 43M in 0.9s (50.7 MB/s) 2024-09-10T23:18:03.074 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.b... 2024-09-10T23:18:03.074 DEBUG:teuthology.orchestra.run.smithi181:> sudo dpkg -i /tmp/linux-image.deb 2024-09-10T23:18:03.160 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-10T23:18:03.164 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-10T23:18:03.315 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-10T23:18:03.344 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T23:18:05.290 INFO:teuthology.orchestra.run.smithi181.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-10T23:18:05.294 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-10T23:18:05.421 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T23:18:07.527 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T23:18:07.630 INFO:teuthology.orchestra.run.smithi094.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:09.241 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T23:18:09.338 INFO:teuthology.orchestra.run.smithi181.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:31.012 INFO:teuthology.orchestra.run.smithi094.stderr:Sourcing file `/etc/default/grub' 2024-09-10T23:18:31.025 INFO:teuthology.orchestra.run.smithi094.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T23:18:31.052 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-09-10T23:18:31.373 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:31.397 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:31.467 INFO:teuthology.orchestra.run.smithi181.stderr:Sourcing file `/etc/default/grub' 2024-09-10T23:18:31.486 INFO:teuthology.orchestra.run.smithi181.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T23:18:31.507 INFO:teuthology.orchestra.run.smithi181.stderr:Generating grub configuration file ... 2024-09-10T23:18:31.742 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T23:18:31.752 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T23:18:31.799 INFO:teuthology.orchestra.run.smithi181.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:31.810 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T23:18:31.814 INFO:teuthology.orchestra.run.smithi181.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:31.815 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T23:18:32.011 INFO:teuthology.orchestra.run.smithi094.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T23:18:32.011 INFO:teuthology.orchestra.run.smithi094.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T23:18:32.011 INFO:teuthology.orchestra.run.smithi094.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T23:18:32.055 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-09-10T23:18:32.113 INFO:teuthology.orchestra.run.smithi181.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T23:18:32.124 INFO:teuthology.orchestra.run.smithi181.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T23:18:32.189 INFO:teuthology.orchestra.run.smithi181.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T23:18:32.194 INFO:teuthology.orchestra.run.smithi181.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T23:18:32.226 DEBUG:teuthology.orchestra.run.smithi094:> dpkg-deb -c /tmp/linux-image.deb 2024-09-10T23:18:32.234 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-10T23:18:32.234 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-10T23:18:32.234 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:32.252 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:32.252 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:32.272 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-10T23:18:32.272 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-10T23:18:32.272 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-10T23:18:32.272 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-10T23:18:32.273 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-10T23:18:32.273 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-10T23:18:32.273 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-10T23:18:32.273 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-10T23:18:32.273 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T23:18:32.273 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-10T23:18:32.273 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-10T23:18:32.273 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-10T23:18:32.273 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-10T23:18:32.273 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-10T23:18:32.273 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T23:18:32.273 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T23:18:32.273 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T23:18:32.273 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T23:18:32.274 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T23:18:32.274 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T23:18:32.276 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T23:18:32.276 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T23:18:32.276 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T23:18:32.276 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T23:18:32.276 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T23:18:32.276 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T23:18:32.276 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T23:18:32.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T23:18:32.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T23:18:32.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T23:18:32.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T23:18:32.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T23:18:32.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T23:18:32.277 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-10T23:18:32.277 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-10T23:18:32.277 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-10T23:18:32.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T23:18:32.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-10T23:18:32.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-10T23:18:32.277 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-10T23:18:32.277 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T23:18:32.278 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T23:18:32.278 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T23:18:32.291 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-10T23:18:32.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-10T23:18:32.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-10T23:18:32.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-10T23:18:32.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T23:18:32.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T23:18:32.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T23:18:32.292 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-10T23:18:32.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-10T23:18:32.292 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-10T23:18:32.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T23:18:32.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T23:18:32.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T23:18:32.292 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T23:18:32.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T23:18:32.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T23:18:32.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T23:18:32.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T23:18:32.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T23:18:32.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T23:18:32.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T23:18:32.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T23:18:32.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T23:18:32.293 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T23:18:32.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T23:18:32.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T23:18:32.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T23:18:32.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T23:18:32.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-10T23:18:32.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T23:18:32.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-10T23:18:32.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T23:18:32.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T23:18:32.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T23:18:32.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T23:18:32.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T23:18:32.294 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T23:18:32.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T23:18:32.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T23:18:32.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-10T23:18:32.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T23:18:32.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T23:18:32.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T23:18:32.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T23:18:32.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T23:18:32.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T23:18:32.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T23:18:32.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T23:18:32.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T23:18:32.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T23:18:32.295 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T23:18:32.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T23:18:32.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T23:18:32.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T23:18:32.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T23:18:32.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T23:18:32.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T23:18:32.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-10T23:18:32.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T23:18:32.296 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-10T23:18:32.296 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-10T23:18:32.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-10T23:18:32.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T23:18:32.296 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T23:18:32.296 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-10T23:18:32.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-10T23:18:32.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T23:18:32.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T23:18:32.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T23:18:32.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T23:18:32.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T23:18:32.297 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-10T23:18:32.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T23:18:32.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-10T23:18:32.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T23:18:32.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-10T23:18:32.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T23:18:32.297 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T23:18:32.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T23:18:32.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T23:18:32.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T23:18:32.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T23:18:32.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T23:18:32.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T23:18:32.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T23:18:32.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T23:18:32.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T23:18:32.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T23:18:32.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T23:18:32.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T23:18:32.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T23:18:32.298 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T23:18:32.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T23:18:32.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T23:18:32.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-10T23:18:32.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T23:18:32.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T23:18:32.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T23:18:32.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T23:18:32.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T23:18:32.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T23:18:32.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T23:18:32.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T23:18:32.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T23:18:32.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T23:18:32.299 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T23:18:32.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T23:18:32.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T23:18:32.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-10T23:18:32.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T23:18:32.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-10T23:18:32.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T23:18:32.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T23:18:32.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T23:18:32.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T23:18:32.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T23:18:32.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T23:18:32.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T23:18:32.300 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T23:18:32.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T23:18:32.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T23:18:32.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T23:18:32.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T23:18:32.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T23:18:32.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T23:18:32.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T23:18:32.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T23:18:32.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T23:18:32.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T23:18:32.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T23:18:32.301 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T23:18:32.301 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-10T23:18:32.302 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T23:18:32.302 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-10T23:18:32.302 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T23:18:32.302 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-10T23:18:32.302 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-10T23:18:32.302 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T23:18:32.302 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-10T23:18:32.302 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T23:18:32.302 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T23:18:32.302 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-10T23:18:32.302 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T23:18:32.347 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T23:18:32.347 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-10T23:18:32.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T23:18:32.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T23:18:32.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T23:18:32.348 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-10T23:18:32.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T23:18:32.348 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-10T23:18:32.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T23:18:32.348 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-10T23:18:32.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-10T23:18:32.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T23:18:32.348 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-10T23:18:32.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T23:18:32.348 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T23:18:32.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T23:18:32.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T23:18:32.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T23:18:32.349 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-10T23:18:32.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T23:18:32.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T23:18:32.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T23:18:32.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T23:18:32.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T23:18:32.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T23:18:32.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T23:18:32.349 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-10T23:18:32.349 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T23:18:32.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-10T23:18:32.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T23:18:32.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T23:18:32.350 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-10T23:18:32.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T23:18:32.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-10T23:18:32.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-10T23:18:32.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-10T23:18:32.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T23:18:32.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T23:18:32.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-10T23:18:32.350 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-10T23:18:32.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-10T23:18:32.350 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-10T23:18:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-10T23:18:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-10T23:18:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-10T23:18:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-10T23:18:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-10T23:18:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-10T23:18:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-10T23:18:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T23:18:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T23:18:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T23:18:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-10T23:18:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-10T23:18:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T23:18:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T23:18:32.352 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-10T23:18:32.352 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-10T23:18:32.352 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-10T23:18:32.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T23:18:32.352 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-10T23:18:32.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T23:18:32.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T23:18:32.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T23:18:32.352 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-10T23:18:32.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T23:18:32.352 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-10T23:18:32.352 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T23:18:32.352 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-10T23:18:32.353 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-10T23:18:32.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T23:18:32.353 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-10T23:18:32.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T23:18:32.353 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-10T23:18:32.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T23:18:32.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T23:18:32.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T23:18:32.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T23:18:32.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T23:18:32.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T23:18:32.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T23:18:32.353 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T23:18:32.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T23:18:32.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T23:18:32.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T23:18:32.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T23:18:32.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T23:18:32.354 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-10T23:18:32.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-10T23:18:32.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-10T23:18:32.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-10T23:18:32.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-10T23:18:32.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-10T23:18:32.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-10T23:18:32.354 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-10T23:18:32.354 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-10T23:18:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-10T23:18:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-10T23:18:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-10T23:18:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T23:18:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-10T23:18:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-10T23:18:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-10T23:18:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-10T23:18:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-10T23:18:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-10T23:18:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-10T23:18:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-10T23:18:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-10T23:18:32.355 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T23:18:32.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-10T23:18:32.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-10T23:18:32.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-10T23:18:32.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-10T23:18:32.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-10T23:18:32.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-10T23:18:32.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-10T23:18:32.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-10T23:18:32.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-10T23:18:32.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-10T23:18:32.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-10T23:18:32.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-10T23:18:32.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-10T23:18:32.356 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-10T23:18:32.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-10T23:18:32.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T23:18:32.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-10T23:18:32.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-10T23:18:32.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-10T23:18:32.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-10T23:18:32.357 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-10T23:18:32.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-10T23:18:32.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-10T23:18:32.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T23:18:32.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T23:18:32.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T23:18:32.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-10T23:18:32.357 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-10T23:18:32.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-10T23:18:32.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-10T23:18:32.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-10T23:18:32.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T23:18:32.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T23:18:32.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T23:18:32.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T23:18:32.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-10T23:18:32.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T23:18:32.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-10T23:18:32.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T23:18:32.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T23:18:32.358 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T23:18:32.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T23:18:32.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T23:18:32.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T23:18:32.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T23:18:32.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-10T23:18:32.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T23:18:32.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T23:18:32.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T23:18:32.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T23:18:32.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T23:18:32.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-10T23:18:32.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T23:18:32.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T23:18:32.359 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-10T23:18:32.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-10T23:18:32.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-10T23:18:32.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-10T23:18:32.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T23:18:32.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-10T23:18:32.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T23:18:32.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T23:18:32.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T23:18:32.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T23:18:32.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T23:18:32.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T23:18:32.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T23:18:32.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T23:18:32.360 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T23:18:32.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T23:18:32.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T23:18:32.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T23:18:32.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T23:18:32.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T23:18:32.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T23:18:32.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T23:18:32.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T23:18:32.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T23:18:32.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T23:18:32.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T23:18:32.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T23:18:32.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T23:18:32.361 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T23:18:32.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T23:18:32.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T23:18:32.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T23:18:32.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T23:18:32.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T23:18:32.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T23:18:32.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T23:18:32.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T23:18:32.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T23:18:32.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T23:18:32.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T23:18:32.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T23:18:32.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T23:18:32.362 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T23:18:32.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T23:18:32.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T23:18:32.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-10T23:18:32.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T23:18:32.363 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-10T23:18:32.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-10T23:18:32.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-10T23:18:32.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-10T23:18:32.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T23:18:32.363 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-10T23:18:32.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T23:18:32.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T23:18:32.363 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T23:18:32.363 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-10T23:18:32.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T23:18:32.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T23:18:32.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T23:18:32.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T23:18:32.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T23:18:32.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T23:18:32.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T23:18:32.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T23:18:32.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T23:18:32.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T23:18:32.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T23:18:32.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T23:18:32.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T23:18:32.364 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T23:18:32.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T23:18:32.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T23:18:32.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T23:18:32.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T23:18:32.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T23:18:32.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T23:18:32.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T23:18:32.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T23:18:32.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T23:18:32.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T23:18:32.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T23:18:32.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-10T23:18:32.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-10T23:18:32.365 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-10T23:18:32.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T23:18:32.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T23:18:32.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T23:18:32.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T23:18:32.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T23:18:32.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-10T23:18:32.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T23:18:32.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T23:18:32.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T23:18:32.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T23:18:32.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T23:18:32.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T23:18:32.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T23:18:32.366 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T23:18:32.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T23:18:32.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T23:18:32.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-10T23:18:32.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T23:18:32.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T23:18:32.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T23:18:32.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T23:18:32.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T23:18:32.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T23:18:32.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T23:18:32.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T23:18:32.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-10T23:18:32.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T23:18:32.367 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T23:18:32.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T23:18:32.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T23:18:32.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T23:18:32.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T23:18:32.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T23:18:32.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T23:18:32.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T23:18:32.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T23:18:32.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T23:18:32.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T23:18:32.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T23:18:32.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T23:18:32.368 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T23:18:32.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T23:18:32.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T23:18:32.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T23:18:32.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T23:18:32.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T23:18:32.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T23:18:32.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T23:18:32.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T23:18:32.369 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T23:18:32.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T23:18:32.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T23:18:32.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T23:18:32.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T23:18:32.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T23:18:32.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T23:18:32.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T23:18:32.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T23:18:32.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-10T23:18:32.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T23:18:32.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-10T23:18:32.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T23:18:32.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T23:18:32.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T23:18:32.370 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T23:18:32.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T23:18:32.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T23:18:32.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T23:18:32.371 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-10T23:18:32.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T23:18:32.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T23:18:32.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T23:18:32.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T23:18:32.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T23:18:32.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T23:18:32.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T23:18:32.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T23:18:32.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T23:18:32.371 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T23:18:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T23:18:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T23:18:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T23:18:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T23:18:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T23:18:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T23:18:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T23:18:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T23:18:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T23:18:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T23:18:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T23:18:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T23:18:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T23:18:32.372 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T23:18:32.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T23:18:32.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T23:18:32.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T23:18:32.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T23:18:32.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T23:18:32.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T23:18:32.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T23:18:32.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T23:18:32.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T23:18:32.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T23:18:32.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T23:18:32.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T23:18:32.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T23:18:32.373 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T23:18:32.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T23:18:32.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T23:18:32.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T23:18:32.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-10T23:18:32.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-10T23:18:32.374 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-10T23:18:32.374 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-10T23:18:32.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T23:18:32.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T23:18:32.374 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-10T23:18:32.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-10T23:18:32.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-10T23:18:32.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-10T23:18:32.374 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T23:18:32.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T23:18:32.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T23:18:32.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-10T23:18:32.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T23:18:32.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-10T23:18:32.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T23:18:32.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T23:18:32.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T23:18:32.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T23:18:32.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-10T23:18:32.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-10T23:18:32.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T23:18:32.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T23:18:32.375 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T23:18:32.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-10T23:18:32.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T23:18:32.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T23:18:32.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-10T23:18:32.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T23:18:32.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T23:18:32.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T23:18:32.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-10T23:18:32.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T23:18:32.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T23:18:32.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-10T23:18:32.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-10T23:18:32.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T23:18:32.376 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-10T23:18:32.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T23:18:32.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T23:18:32.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T23:18:32.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T23:18:32.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-10T23:18:32.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T23:18:32.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T23:18:32.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T23:18:32.377 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-10T23:18:32.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-10T23:18:32.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-10T23:18:32.377 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-10T23:18:32.377 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-10T23:18:32.377 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-10T23:18:32.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T23:18:32.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T23:18:32.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T23:18:32.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T23:18:32.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T23:18:32.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T23:18:32.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T23:18:32.378 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-10T23:18:32.378 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-10T23:18:32.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T23:18:32.378 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-10T23:18:32.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T23:18:32.378 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-10T23:18:32.378 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T23:18:32.379 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-10T23:18:32.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T23:18:32.379 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-10T23:18:32.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-10T23:18:32.379 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-10T23:18:32.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-10T23:18:32.379 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-10T23:18:32.379 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-10T23:18:32.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T23:18:32.379 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-10T23:18:32.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T23:18:32.379 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-10T23:18:32.379 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T23:18:32.379 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-10T23:18:32.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T23:18:32.380 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-10T23:18:32.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T23:18:32.380 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-10T23:18:32.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-10T23:18:32.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-10T23:18:32.380 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-10T23:18:32.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T23:18:32.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T23:18:32.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T23:18:32.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T23:18:32.380 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-10T23:18:32.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-10T23:18:32.380 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-10T23:18:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-10T23:18:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-10T23:18:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T23:18:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-10T23:18:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-10T23:18:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-10T23:18:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-10T23:18:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-10T23:18:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-10T23:18:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-10T23:18:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-10T23:18:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-10T23:18:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-10T23:18:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-10T23:18:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-10T23:18:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-10T23:18:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-10T23:18:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-10T23:18:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-10T23:18:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-10T23:18:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-10T23:18:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-10T23:18:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-10T23:18:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-10T23:18:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T23:18:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T23:18:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T23:18:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T23:18:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-10T23:18:32.383 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-10T23:18:32.383 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T23:18:32.383 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T23:18:32.383 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T23:18:32.383 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T23:18:32.383 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-10T23:18:32.383 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T23:18:32.383 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-10T23:18:32.383 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T23:18:32.383 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T23:18:32.383 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-10T23:18:32.383 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T23:18:32.383 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-10T23:18:32.384 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-10T23:18:32.384 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-10T23:18:32.384 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T23:18:32.384 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-10T23:18:32.384 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T23:18:32.384 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-10T23:18:32.384 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-10T23:18:32.384 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-10T23:18:32.384 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-10T23:18:32.384 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T23:18:32.384 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-10T23:18:32.384 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-10T23:18:32.384 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-10T23:18:32.384 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T23:18:32.385 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-10T23:18:32.385 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-10T23:18:32.385 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-10T23:18:32.385 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T23:18:32.385 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-10T23:18:32.385 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-10T23:18:32.385 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-10T23:18:32.385 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-10T23:18:32.385 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-10T23:18:32.385 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-10T23:18:32.385 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-10T23:18:32.385 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T23:18:32.385 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-10T23:18:32.386 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-10T23:18:32.386 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-10T23:18:32.386 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-10T23:18:32.386 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T23:18:32.386 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T23:18:32.386 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T23:18:32.386 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T23:18:32.386 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T23:18:32.386 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T23:18:32.386 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T23:18:32.386 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T23:18:32.386 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-10T23:18:32.386 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-10T23:18:32.386 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T23:18:32.387 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T23:18:32.387 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-10T23:18:32.387 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T23:18:32.387 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T23:18:32.387 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T23:18:32.387 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T23:18:32.387 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-10T23:18:32.387 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T23:18:32.387 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T23:18:32.387 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-10T23:18:32.387 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-10T23:18:32.387 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T23:18:32.387 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T23:18:32.387 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T23:18:32.388 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-10T23:18:32.388 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T23:18:32.388 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T23:18:32.388 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-10T23:18:32.388 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T23:18:32.388 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T23:18:32.388 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T23:18:32.388 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-10T23:18:32.388 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-10T23:18:32.388 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T23:18:32.388 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T23:18:32.388 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T23:18:32.388 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T23:18:32.388 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T23:18:32.389 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-10T23:18:32.389 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-10T23:18:32.389 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T23:18:32.389 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T23:18:32.389 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T23:18:32.389 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T23:18:32.389 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T23:18:32.389 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T23:18:32.390 INFO:teuthology.orchestra.run.smithi181.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T23:18:32.390 INFO:teuthology.orchestra.run.smithi181.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T23:18:32.390 INFO:teuthology.orchestra.run.smithi181.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T23:18:32.390 INFO:teuthology.orchestra.run.smithi181.stderr:done 2024-09-10T23:18:32.440 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T23:18:32.440 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-10T23:18:32.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-10T23:18:32.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-10T23:18:32.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-10T23:18:32.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-10T23:18:32.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-10T23:18:32.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-10T23:18:32.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-10T23:18:32.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-10T23:18:32.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-10T23:18:32.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-10T23:18:32.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-10T23:18:32.441 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-10T23:18:32.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-10T23:18:32.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T23:18:32.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T23:18:32.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-10T23:18:32.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-10T23:18:32.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-10T23:18:32.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-10T23:18:32.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-10T23:18:32.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-10T23:18:32.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-10T23:18:32.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-10T23:18:32.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-10T23:18:32.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-10T23:18:32.442 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T23:18:32.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-10T23:18:32.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-10T23:18:32.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T23:18:32.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T23:18:32.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T23:18:32.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-10T23:18:32.443 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-10T23:18:32.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-10T23:18:32.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-10T23:18:32.443 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-10T23:18:32.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T23:18:32.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T23:18:32.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T23:18:32.443 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T23:18:32.444 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-10T23:18:32.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-10T23:18:32.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-10T23:18:32.444 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-10T23:18:32.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T23:18:32.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-10T23:18:32.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-10T23:18:32.444 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-10T23:18:32.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-10T23:18:32.444 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-10T23:18:32.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T23:18:32.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T23:18:32.444 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T23:18:32.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T23:18:32.445 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-10T23:18:32.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-10T23:18:32.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-10T23:18:32.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T23:18:32.445 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-10T23:18:32.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-10T23:18:32.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-10T23:18:32.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T23:18:32.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T23:18:32.445 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-10T23:18:32.445 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T23:18:32.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T23:18:32.446 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-10T23:18:32.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T23:18:32.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T23:18:32.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T23:18:32.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-10T23:18:32.446 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-10T23:18:32.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-10T23:18:32.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T23:18:32.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T23:18:32.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T23:18:32.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T23:18:32.446 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-10T23:18:32.446 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T23:18:32.447 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-10T23:18:32.447 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-10T23:18:32.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T23:18:32.447 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-10T23:18:32.447 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-10T23:18:32.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T23:18:32.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T23:18:32.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T23:18:32.447 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-10T23:18:32.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T23:18:32.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T23:18:32.447 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T23:18:32.447 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-10T23:18:32.447 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-10T23:18:32.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T23:18:32.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T23:18:32.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T23:18:32.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T23:18:32.448 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-10T23:18:32.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T23:18:32.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T23:18:32.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T23:18:32.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T23:18:32.448 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-10T23:18:32.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T23:18:32.448 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-10T23:18:32.448 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T23:18:32.449 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-10T23:18:32.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T23:18:32.449 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-10T23:18:32.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T23:18:32.449 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-10T23:18:32.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T23:18:32.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T23:18:32.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T23:18:32.449 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-10T23:18:32.449 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-10T23:18:32.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T23:18:32.449 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-10T23:18:32.449 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T23:18:32.450 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-10T23:18:32.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T23:18:32.450 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-10T23:18:32.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T23:18:32.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T23:18:32.450 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-10T23:18:32.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T23:18:32.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T23:18:32.450 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-10T23:18:32.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T23:18:32.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T23:18:32.450 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-10T23:18:32.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T23:18:32.450 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T23:18:32.451 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-10T23:18:32.451 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-10T23:18:32.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T23:18:32.451 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-10T23:18:32.451 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-10T23:18:32.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T23:18:32.451 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-10T23:18:32.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T23:18:32.451 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-10T23:18:32.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T23:18:32.451 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-10T23:18:32.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T23:18:32.451 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-10T23:18:32.451 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T23:18:32.452 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-10T23:18:32.452 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-10T23:18:32.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T23:18:32.452 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-10T23:18:32.452 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-10T23:18:32.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T23:18:32.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T23:18:32.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T23:18:32.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T23:18:32.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T23:18:32.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T23:18:32.452 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-10T23:18:32.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T23:18:32.452 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T23:18:32.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T23:18:32.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-10T23:18:32.453 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-10T23:18:32.453 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-10T23:18:32.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T23:18:32.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T23:18:32.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T23:18:32.453 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-10T23:18:32.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-10T23:18:32.453 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-10T23:18:32.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T23:18:32.453 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-10T23:18:32.453 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T23:18:32.453 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-10T23:18:32.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T23:18:32.454 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-10T23:18:32.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T23:18:32.454 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-10T23:18:32.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T23:18:32.454 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-10T23:18:32.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T23:18:32.454 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-10T23:18:32.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T23:18:32.454 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-10T23:18:32.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T23:18:32.454 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-10T23:18:32.454 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T23:18:32.454 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-10T23:18:32.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T23:18:32.455 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-10T23:18:32.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T23:18:32.455 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-10T23:18:32.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T23:18:32.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T23:18:32.455 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-10T23:18:32.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T23:18:32.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T23:18:32.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T23:18:32.455 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-10T23:18:32.455 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-10T23:18:32.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T23:18:32.455 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T23:18:32.456 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-10T23:18:32.456 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-10T23:18:32.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T23:18:32.456 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-10T23:18:32.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-10T23:18:32.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-10T23:18:32.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-10T23:18:32.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T23:18:32.456 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-10T23:18:32.456 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-10T23:18:32.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T23:18:32.456 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-10T23:18:32.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T23:18:32.456 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T23:18:32.457 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-10T23:18:32.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T23:18:32.457 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-10T23:18:32.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T23:18:32.457 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-10T23:18:32.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T23:18:32.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T23:18:32.457 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-10T23:18:32.457 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-10T23:18:32.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T23:18:32.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T23:18:32.457 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-10T23:18:32.457 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T23:18:32.458 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-10T23:18:32.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T23:18:32.458 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-10T23:18:32.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T23:18:32.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T23:18:32.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T23:18:32.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T23:18:32.458 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-10T23:18:32.458 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-10T23:18:32.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-10T23:18:32.458 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-10T23:18:32.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T23:18:32.458 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-10T23:18:32.458 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T23:18:32.459 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-10T23:18:32.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T23:18:32.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T23:18:32.459 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-10T23:18:32.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T23:18:32.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T23:18:32.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T23:18:32.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T23:18:32.459 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-10T23:18:32.459 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-10T23:18:32.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-10T23:18:32.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T23:18:32.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-10T23:18:32.459 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T23:18:32.460 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-10T23:18:32.460 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T23:18:32.460 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T23:18:32.460 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T23:18:32.460 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T23:18:32.460 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-10T23:18:32.460 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T23:18:32.460 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-10T23:18:32.460 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T23:18:32.460 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-10T23:18:32.460 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T23:18:32.460 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T23:18:32.460 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-10T23:18:32.460 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T23:18:32.461 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T23:18:32.461 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-10T23:18:32.461 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T23:18:32.461 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-10T23:18:32.461 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-10T23:18:32.461 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-10T23:18:32.461 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-10T23:18:32.461 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T23:18:32.461 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-10T23:18:32.461 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T23:18:32.461 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-10T23:18:32.461 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T23:18:32.461 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T23:18:32.461 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-10T23:18:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T23:18:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T23:18:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T23:18:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-10T23:18:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T23:18:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T23:18:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-10T23:18:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T23:18:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T23:18:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T23:18:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T23:18:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T23:18:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-10T23:18:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T23:18:32.463 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T23:18:32.463 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-10T23:18:32.463 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T23:18:32.463 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T23:18:32.463 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T23:18:32.463 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T23:18:32.463 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T23:18:32.463 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T23:18:32.463 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T23:18:32.463 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T23:18:32.463 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T23:18:32.463 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T23:18:32.463 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T23:18:32.463 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T23:18:32.463 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T23:18:32.464 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T23:18:32.464 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T23:18:32.464 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T23:18:32.464 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T23:18:32.464 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T23:18:32.464 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T23:18:32.464 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-10T23:18:32.464 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-10T23:18:32.464 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-10T23:18:32.464 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T23:18:32.464 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T23:18:32.464 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T23:18:32.464 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T23:18:32.464 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T23:18:32.465 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T23:18:32.465 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T23:18:32.465 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T23:18:32.465 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-10T23:18:32.465 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T23:18:32.465 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T23:18:32.465 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T23:18:32.465 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-10T23:18:32.465 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T23:18:32.465 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T23:18:32.465 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T23:18:32.465 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T23:18:32.465 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T23:18:32.465 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T23:18:32.466 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T23:18:32.466 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-10T23:18:32.466 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T23:18:32.466 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-10T23:18:32.466 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T23:18:32.466 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-10T23:18:32.466 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T23:18:32.466 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-10T23:18:32.466 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T23:18:32.466 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-10T23:18:32.466 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-10T23:18:32.466 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T23:18:32.466 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T23:18:32.466 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-10T23:18:32.466 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T23:18:32.467 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T23:18:32.467 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T23:18:32.467 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T23:18:32.467 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-10T23:18:32.467 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-10T23:18:32.467 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T23:18:32.467 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-10T23:18:32.467 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-10T23:18:32.467 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-10T23:18:32.467 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T23:18:32.467 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-10T23:18:32.467 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T23:18:32.467 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-10T23:18:32.467 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-10T23:18:32.468 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-10T23:18:32.468 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T23:18:32.468 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T23:18:32.468 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-10T23:18:32.468 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-10T23:18:32.468 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-10T23:18:32.468 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-10T23:18:32.468 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-10T23:18:32.468 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-10T23:18:32.468 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-10T23:18:32.468 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T23:18:32.468 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-10T23:18:32.468 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-10T23:18:32.468 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-10T23:18:32.469 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-10T23:18:32.469 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-10T23:18:32.469 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-10T23:18:32.469 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-10T23:18:32.469 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-10T23:18:32.469 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-10T23:18:32.469 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-10T23:18:32.469 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T23:18:32.469 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T23:18:32.469 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T23:18:32.469 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T23:18:32.469 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T23:18:32.469 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T23:18:32.469 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T23:18:32.470 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T23:18:32.470 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T23:18:32.470 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T23:18:32.470 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T23:18:32.470 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T23:18:32.470 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-10T23:18:32.470 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-10T23:18:32.470 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T23:18:32.470 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-10T23:18:32.470 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T23:18:32.470 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T23:18:32.470 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T23:18:32.470 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T23:18:32.471 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T23:18:32.471 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T23:18:32.471 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T23:18:32.471 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T23:18:32.471 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T23:18:32.471 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T23:18:32.471 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T23:18:32.471 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-10T23:18:32.471 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-10T23:18:32.471 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-10T23:18:32.471 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-10T23:18:32.471 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-10T23:18:32.471 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-10T23:18:32.471 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-10T23:18:32.471 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-10T23:18:32.471 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-10T23:18:32.472 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-10T23:18:32.472 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-10T23:18:32.472 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-10T23:18:32.472 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-10T23:18:32.472 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-10T23:18:32.472 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-10T23:18:32.472 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-10T23:18:32.472 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-10T23:18:32.472 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-10T23:18:32.472 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-10T23:18:32.472 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-10T23:18:32.472 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-10T23:18:32.472 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-10T23:18:32.473 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-10T23:18:32.473 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-10T23:18:32.473 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-10T23:18:32.473 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-10T23:18:32.473 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-10T23:18:32.473 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-10T23:18:32.473 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-10T23:18:32.473 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-10T23:18:32.473 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T23:18:32.473 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T23:18:32.473 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-10T23:18:32.473 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-10T23:18:32.473 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T23:18:32.474 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T23:18:32.474 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T23:18:32.474 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T23:18:32.474 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-10T23:18:32.474 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T23:18:32.474 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T23:18:32.474 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-10T23:18:32.474 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-10T23:18:32.474 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-10T23:18:32.474 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T23:18:32.474 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T23:18:32.474 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-10T23:18:32.474 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-10T23:18:32.475 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-10T23:18:32.475 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T23:18:32.475 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-10T23:18:32.475 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-10T23:18:32.475 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-10T23:18:32.475 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T23:18:32.475 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T23:18:32.475 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T23:18:32.475 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T23:18:32.475 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T23:18:32.475 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T23:18:32.475 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T23:18:32.475 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T23:18:32.475 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T23:18:32.476 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T23:18:32.476 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T23:18:32.476 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T23:18:32.476 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T23:18:32.476 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-10T23:18:32.476 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T23:18:32.476 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T23:18:32.476 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T23:18:32.476 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-10T23:18:32.476 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T23:18:32.476 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-10T23:18:32.476 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-10T23:18:32.476 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-10T23:18:32.477 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-10T23:18:32.477 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-10T23:18:32.477 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T23:18:32.477 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-10T23:18:32.477 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T23:18:32.477 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-10T23:18:32.477 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T23:18:32.477 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T23:18:32.477 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T23:18:32.477 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T23:18:32.477 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-10T23:18:32.477 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T23:18:32.477 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T23:18:32.477 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T23:18:32.477 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T23:18:32.478 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T23:18:32.478 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-10T23:18:32.478 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T23:18:32.478 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-10T23:18:32.478 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-10T23:18:32.478 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-10T23:18:32.478 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-10T23:18:32.478 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T23:18:32.478 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-10T23:18:32.478 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T23:18:32.478 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T23:18:32.478 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T23:18:32.478 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T23:18:32.479 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T23:18:32.479 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-10T23:18:32.479 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T23:18:32.479 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T23:18:32.479 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-10T23:18:32.479 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T23:18:32.479 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T23:18:32.479 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-10T23:18:32.479 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-10T23:18:32.479 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T23:18:32.479 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-10T23:18:32.479 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T23:18:32.479 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T23:18:32.479 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-10T23:18:32.480 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T23:18:32.480 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-10T23:18:32.480 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T23:18:32.480 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-10T23:18:32.480 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T23:18:32.480 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-10T23:18:32.480 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T23:18:32.480 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T23:18:32.480 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-10T23:18:32.480 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T23:18:32.480 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-10T23:18:32.480 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-10T23:18:32.480 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T23:18:32.481 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-10T23:18:32.481 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T23:18:32.481 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T23:18:32.481 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T23:18:32.481 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T23:18:32.481 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-10T23:18:32.481 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T23:18:32.481 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T23:18:32.481 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-10T23:18:32.481 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T23:18:32.481 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T23:18:32.481 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-10T23:18:32.481 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-10T23:18:32.482 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T23:18:32.482 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T23:18:32.482 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T23:18:32.482 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T23:18:32.482 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-10T23:18:32.482 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T23:18:32.482 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T23:18:32.482 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T23:18:32.482 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-10T23:18:32.482 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T23:18:32.482 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T23:18:32.482 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T23:18:32.482 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-10T23:18:32.483 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T23:18:32.483 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T23:18:32.483 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T23:18:32.483 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-10T23:18:32.483 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T23:18:32.483 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-10T23:18:32.483 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T23:18:32.483 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-10T23:18:32.483 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T23:18:32.483 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T23:18:32.483 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T23:18:32.483 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T23:18:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-10T23:18:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T23:18:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-10T23:18:32.522 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T23:18:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T23:18:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-10T23:18:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-10T23:18:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-10T23:18:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-10T23:18:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-10T23:18:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-10T23:18:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T23:18:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T23:18:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-10T23:18:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T23:18:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-10T23:18:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T23:18:32.523 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T23:18:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-10T23:18:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T23:18:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-10T23:18:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T23:18:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T23:18:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T23:18:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T23:18:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T23:18:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T23:18:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T23:18:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T23:18:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T23:18:32.524 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T23:18:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-10T23:18:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T23:18:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T23:18:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T23:18:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T23:18:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-10T23:18:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-10T23:18:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T23:18:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-10T23:18:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-10T23:18:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T23:18:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-10T23:18:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T23:18:32.525 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T23:18:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T23:18:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T23:18:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T23:18:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T23:18:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-10T23:18:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T23:18:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-10T23:18:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-10T23:18:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-10T23:18:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T23:18:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T23:18:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T23:18:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T23:18:32.526 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T23:18:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T23:18:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T23:18:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T23:18:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T23:18:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T23:18:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T23:18:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T23:18:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T23:18:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T23:18:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T23:18:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-10T23:18:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T23:18:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T23:18:32.527 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T23:18:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-10T23:18:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-10T23:18:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T23:18:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-10T23:18:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-10T23:18:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T23:18:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T23:18:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T23:18:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-10T23:18:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-10T23:18:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-10T23:18:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T23:18:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-10T23:18:32.528 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-10T23:18:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T23:18:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-10T23:18:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-10T23:18:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T23:18:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-10T23:18:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-10T23:18:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-10T23:18:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-10T23:18:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-10T23:18:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T23:18:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-10T23:18:32.529 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T23:18:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-10T23:18:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-10T23:18:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T23:18:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T23:18:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T23:18:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-10T23:18:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-10T23:18:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-10T23:18:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T23:18:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-10T23:18:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T23:18:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-10T23:18:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-10T23:18:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-10T23:18:32.530 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-10T23:18:32.531 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-10T23:18:32.531 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-10T23:18:32.531 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-10T23:18:32.531 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-10T23:18:32.531 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-10T23:18:32.531 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-10T23:18:32.531 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T23:18:32.531 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T23:18:32.531 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T23:18:32.531 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T23:18:32.531 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-10T23:18:32.531 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-10T23:18:32.531 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-10T23:18:32.532 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-10T23:18:32.532 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-10T23:18:32.532 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-10T23:18:32.532 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-10T23:18:32.532 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-10T23:18:32.532 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-10T23:18:32.532 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-10T23:18:32.532 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-10T23:18:32.532 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-10T23:18:32.532 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-10T23:18:32.532 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-10T23:18:32.532 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-10T23:18:32.532 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-10T23:18:32.533 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-10T23:18:32.533 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-10T23:18:32.533 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-10T23:18:32.533 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-10T23:18:32.533 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-10T23:18:32.533 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-10T23:18:32.533 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-10T23:18:32.533 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-10T23:18:32.533 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-10T23:18:32.533 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-10T23:18:32.533 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-10T23:18:32.533 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-10T23:18:32.533 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-10T23:18:32.533 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-10T23:18:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-10T23:18:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-10T23:18:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-10T23:18:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-10T23:18:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-10T23:18:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-10T23:18:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-10T23:18:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-10T23:18:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-10T23:18:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-10T23:18:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-10T23:18:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-10T23:18:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-10T23:18:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-10T23:18:32.534 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-10T23:18:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-10T23:18:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-10T23:18:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-10T23:18:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-10T23:18:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-10T23:18:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-10T23:18:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-10T23:18:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-10T23:18:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-10T23:18:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-10T23:18:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-10T23:18:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-10T23:18:32.535 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-10T23:18:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-10T23:18:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-10T23:18:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-10T23:18:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-10T23:18:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-10T23:18:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T23:18:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-10T23:18:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-10T23:18:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-10T23:18:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-10T23:18:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T23:18:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T23:18:32.536 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-10T23:18:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T23:18:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T23:18:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-10T23:18:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-10T23:18:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-10T23:18:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T23:18:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T23:18:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T23:18:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T23:18:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T23:18:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T23:18:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T23:18:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T23:18:32.537 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-10T23:18:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-10T23:18:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T23:18:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T23:18:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T23:18:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T23:18:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T23:18:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-10T23:18:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T23:18:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T23:18:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-10T23:18:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-10T23:18:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-10T23:18:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-10T23:18:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-10T23:18:32.538 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T23:18:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-10T23:18:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-10T23:18:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-10T23:18:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-10T23:18:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T23:18:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T23:18:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T23:18:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T23:18:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T23:18:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T23:18:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T23:18:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T23:18:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T23:18:32.539 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-10T23:18:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T23:18:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T23:18:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T23:18:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T23:18:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T23:18:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T23:18:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T23:18:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T23:18:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T23:18:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T23:18:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T23:18:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T23:18:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T23:18:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-10T23:18:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T23:18:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T23:18:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-10T23:18:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T23:18:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T23:18:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T23:18:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T23:18:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T23:18:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T23:18:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T23:18:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T23:18:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T23:18:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T23:18:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T23:18:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T23:18:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T23:18:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T23:18:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T23:18:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T23:18:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T23:18:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T23:18:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-10T23:18:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T23:18:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T23:18:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-10T23:18:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T23:18:32.542 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-10T23:18:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T23:18:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T23:18:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T23:18:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T23:18:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T23:18:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T23:18:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T23:18:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T23:18:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T23:18:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T23:18:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T23:18:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T23:18:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T23:18:32.543 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T23:18:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T23:18:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-10T23:18:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T23:18:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T23:18:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T23:18:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-10T23:18:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-10T23:18:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T23:18:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T23:18:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T23:18:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T23:18:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-10T23:18:32.544 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T23:18:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T23:18:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T23:18:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T23:18:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-10T23:18:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-10T23:18:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-10T23:18:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-10T23:18:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-10T23:18:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-10T23:18:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-10T23:18:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T23:18:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-10T23:18:32.545 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-10T23:18:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-10T23:18:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-10T23:18:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-10T23:18:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-10T23:18:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T23:18:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-10T23:18:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-10T23:18:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-10T23:18:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-10T23:18:32.546 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-10T23:18:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-10T23:18:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T23:18:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-10T23:18:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-10T23:18:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-10T23:18:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T23:18:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T23:18:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-10T23:18:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-10T23:18:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-10T23:18:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T23:18:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T23:18:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T23:18:32.547 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T23:18:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T23:18:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T23:18:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T23:18:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T23:18:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T23:18:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T23:18:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T23:18:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T23:18:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-10T23:18:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-10T23:18:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-10T23:18:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T23:18:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T23:18:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-10T23:18:32.548 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-10T23:18:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-10T23:18:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-10T23:18:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-10T23:18:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-10T23:18:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T23:18:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T23:18:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T23:18:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T23:18:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T23:18:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T23:18:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T23:18:32.549 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T23:18:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T23:18:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T23:18:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-10T23:18:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T23:18:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-10T23:18:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-10T23:18:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T23:18:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-10T23:18:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-10T23:18:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T23:18:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T23:18:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-10T23:18:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-10T23:18:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-10T23:18:32.550 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T23:18:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-10T23:18:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T23:18:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-10T23:18:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T23:18:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T23:18:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-10T23:18:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T23:18:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T23:18:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T23:18:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T23:18:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-10T23:18:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T23:18:32.551 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-10T23:18:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-10T23:18:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T23:18:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-10T23:18:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T23:18:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T23:18:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T23:18:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-10T23:18:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T23:18:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T23:18:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-10T23:18:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-10T23:18:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T23:18:32.552 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-10T23:18:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-10T23:18:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T23:18:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-10T23:18:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-10T23:18:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-10T23:18:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T23:18:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-10T23:18:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-10T23:18:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-10T23:18:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-10T23:18:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-10T23:18:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T23:18:32.553 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-10T23:18:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-10T23:18:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T23:18:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-10T23:18:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T23:18:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-10T23:18:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T23:18:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-10T23:18:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T23:18:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-10T23:18:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T23:18:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-10T23:18:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-10T23:18:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-10T23:18:32.554 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T23:18:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-10T23:18:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T23:18:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-10T23:18:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-10T23:18:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-10T23:18:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T23:18:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-10T23:18:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T23:18:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-10T23:18:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-10T23:18:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-10T23:18:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T23:18:32.555 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-10T23:18:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T23:18:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-10T23:18:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T23:18:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-10T23:18:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T23:18:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-10T23:18:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-10T23:18:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-10T23:18:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T23:18:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-10T23:18:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T23:18:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-10T23:18:32.556 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T23:18:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-10T23:18:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T23:18:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-10T23:18:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T23:18:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-10T23:18:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-10T23:18:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T23:18:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-10T23:18:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T23:18:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-10T23:18:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T23:18:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T23:18:32.557 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-10T23:18:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T23:18:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T23:18:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-10T23:18:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T23:18:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T23:18:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-10T23:18:32.558 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T23:18:32.578 DEBUG:teuthology.orchestra.run.smithi181:> dpkg-deb -c /tmp/linux-image.deb 2024-09-10T23:18:32.595 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-10T23:18:32.595 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-10T23:18:32.595 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:32.611 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:32.611 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-10T23:18:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T23:18:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-10T23:18:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T23:18:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T23:18:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T23:18:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T23:18:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T23:18:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T23:18:32.612 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T23:18:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T23:18:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T23:18:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T23:18:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T23:18:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-10T23:18:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T23:18:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T23:18:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T23:18:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T23:18:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T23:18:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T23:18:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T23:18:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T23:18:32.613 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T23:18:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T23:18:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T23:18:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T23:18:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T23:18:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T23:18:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T23:18:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T23:18:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T23:18:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T23:18:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T23:18:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T23:18:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T23:18:32.614 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T23:18:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T23:18:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T23:18:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T23:18:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T23:18:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T23:18:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T23:18:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T23:18:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T23:18:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T23:18:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T23:18:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T23:18:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T23:18:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T23:18:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T23:18:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T23:18:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T23:18:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T23:18:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-10T23:18:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-10T23:18:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-10T23:18:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-10T23:18:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T23:18:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-10T23:18:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T23:18:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-10T23:18:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T23:18:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T23:18:32.616 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T23:18:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T23:18:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T23:18:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-10T23:18:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-10T23:18:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-10T23:18:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T23:18:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-10T23:18:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T23:18:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-10T23:18:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-10T23:18:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-10T23:18:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-10T23:18:32.617 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-10T23:18:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-10T23:18:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-10T23:18:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-10T23:18:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-10T23:18:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T23:18:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-10T23:18:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T23:18:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-10T23:18:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-10T23:18:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T23:18:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-10T23:18:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T23:18:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T23:18:32.618 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-10T23:18:32.619 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T23:18:32.619 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-10T23:18:32.619 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T23:18:32.619 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-10T23:18:32.619 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T23:18:32.619 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-10T23:18:32.619 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T23:18:32.619 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-10T23:18:32.619 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T23:18:32.619 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-10T23:18:32.619 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-10T23:18:32.619 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-10T23:18:32.619 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-10T23:18:32.619 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-10T23:18:32.620 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T23:18:32.620 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-10T23:18:32.620 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T23:18:32.620 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-10T23:18:32.620 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T23:18:32.620 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-10T23:18:32.620 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T23:18:32.620 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T23:18:32.620 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-10T23:18:32.620 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T23:18:32.620 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T23:18:32.620 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-10T23:18:32.621 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T23:18:32.621 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T23:18:32.621 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-10T23:18:32.621 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T23:18:32.621 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-10T23:18:32.621 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-10T23:18:32.621 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-10T23:18:32.621 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-10T23:18:32.621 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-10T23:18:32.621 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T23:18:32.621 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-10T23:18:32.621 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-10T23:18:32.621 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T23:18:32.622 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-10T23:18:32.622 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-10T23:18:32.622 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-10T23:18:32.622 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-10T23:18:32.622 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T23:18:32.622 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T23:18:32.622 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T23:18:32.622 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-10T23:18:32.622 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-10T23:18:32.622 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-10T23:18:32.622 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T23:18:32.622 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-10T23:18:32.622 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-10T23:18:32.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T23:18:32.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T23:18:32.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T23:18:32.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T23:18:32.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T23:18:32.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T23:18:32.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T23:18:32.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T23:18:32.623 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-10T23:18:32.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T23:18:32.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T23:18:32.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T23:18:32.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T23:18:32.623 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T23:18:32.624 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-10T23:18:32.624 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T23:18:32.624 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-10T23:18:32.624 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T23:18:32.624 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T23:18:32.624 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T23:18:32.624 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T23:18:32.624 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-10T23:18:32.624 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T23:18:32.624 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-10T23:18:32.624 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T23:18:32.624 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T23:18:32.624 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-10T23:18:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T23:18:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T23:18:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T23:18:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T23:18:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T23:18:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T23:18:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T23:18:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T23:18:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T23:18:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T23:18:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T23:18:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T23:18:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T23:18:32.625 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T23:18:32.626 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T23:18:32.626 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T23:18:32.626 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T23:18:32.626 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T23:18:32.626 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T23:18:32.626 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T23:18:32.626 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T23:18:32.626 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T23:18:32.626 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T23:18:32.626 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-10T23:18:32.626 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T23:18:32.626 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-10T23:18:32.626 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T23:18:32.626 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-10T23:18:32.627 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T23:18:32.627 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T23:18:32.627 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T23:18:32.627 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T23:18:32.627 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-10T23:18:32.627 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T23:18:32.627 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T23:18:32.627 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T23:18:32.627 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T23:18:32.627 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T23:18:32.627 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-10T23:18:32.627 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-10T23:18:32.627 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-10T23:18:32.627 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-10T23:18:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T23:18:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T23:18:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T23:18:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-10T23:18:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T23:18:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T23:18:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T23:18:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T23:18:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T23:18:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T23:18:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T23:18:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T23:18:32.629 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T23:18:32.629 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T23:18:32.629 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-10T23:18:32.629 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T23:18:32.629 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T23:18:32.629 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T23:18:32.629 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-10T23:18:32.629 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T23:18:32.629 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-10T23:18:32.629 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T23:18:32.629 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T23:18:32.629 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T23:18:32.629 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-10T23:18:32.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T23:18:32.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T23:18:32.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T23:18:32.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T23:18:32.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T23:18:32.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T23:18:32.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T23:18:32.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T23:18:32.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T23:18:32.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-10T23:18:32.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T23:18:32.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T23:18:32.630 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T23:18:32.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T23:18:32.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T23:18:32.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T23:18:32.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T23:18:32.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T23:18:32.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T23:18:32.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T23:18:32.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T23:18:32.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T23:18:32.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T23:18:32.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T23:18:32.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T23:18:32.631 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T23:18:32.632 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T23:18:32.632 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T23:18:32.632 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T23:18:32.632 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T23:18:32.632 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T23:18:32.632 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T23:18:32.632 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-10T23:18:32.632 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T23:18:32.632 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T23:18:32.632 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T23:18:32.632 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-10T23:18:32.632 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T23:18:32.632 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T23:18:32.633 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T23:18:32.633 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T23:18:32.633 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-10T23:18:32.633 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-10T23:18:32.633 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T23:18:32.633 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-10T23:18:32.633 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T23:18:32.633 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T23:18:32.633 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T23:18:32.633 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T23:18:32.633 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T23:18:32.633 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T23:18:32.634 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T23:18:32.634 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T23:18:32.634 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T23:18:32.634 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T23:18:32.634 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T23:18:32.634 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T23:18:32.634 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T23:18:32.634 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T23:18:32.634 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T23:18:32.634 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T23:18:32.634 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T23:18:32.634 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T23:18:32.634 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T23:18:32.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T23:18:32.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T23:18:32.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T23:18:32.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T23:18:32.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T23:18:32.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T23:18:32.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T23:18:32.635 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-10T23:18:32.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T23:18:32.635 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-10T23:18:32.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T23:18:32.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T23:18:32.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T23:18:32.635 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T23:18:32.636 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T23:18:32.636 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T23:18:32.636 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T23:18:32.636 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-10T23:18:32.636 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-10T23:18:32.636 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-10T23:18:32.636 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T23:18:32.636 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-10T23:18:32.636 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-10T23:18:32.636 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T23:18:32.636 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-10T23:18:32.636 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T23:18:32.636 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-10T23:18:32.637 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-10T23:18:32.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T23:18:32.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T23:18:32.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T23:18:32.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T23:18:32.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T23:18:32.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T23:18:32.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T23:18:32.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T23:18:32.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T23:18:32.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T23:18:32.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T23:18:32.637 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T23:18:32.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T23:18:32.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T23:18:32.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T23:18:32.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T23:18:32.638 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-10T23:18:32.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T23:18:32.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T23:18:32.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T23:18:32.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T23:18:32.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T23:18:32.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T23:18:32.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T23:18:32.638 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T23:18:32.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T23:18:32.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T23:18:32.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T23:18:32.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T23:18:32.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T23:18:32.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T23:18:32.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T23:18:32.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T23:18:32.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T23:18:32.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T23:18:32.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T23:18:32.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T23:18:32.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T23:18:32.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T23:18:32.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T23:18:32.639 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T23:18:32.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T23:18:32.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T23:18:32.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T23:18:32.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T23:18:32.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T23:18:32.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T23:18:32.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T23:18:32.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T23:18:32.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T23:18:32.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T23:18:32.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T23:18:32.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T23:18:32.640 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T23:18:32.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T23:18:32.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T23:18:32.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T23:18:32.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T23:18:32.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T23:18:32.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-10T23:18:32.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T23:18:32.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T23:18:32.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T23:18:32.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T23:18:32.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T23:18:32.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T23:18:32.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T23:18:32.641 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T23:18:32.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T23:18:32.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T23:18:32.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T23:18:32.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T23:18:32.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T23:18:32.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T23:18:32.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T23:18:32.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-10T23:18:32.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T23:18:32.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T23:18:32.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T23:18:32.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T23:18:32.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T23:18:32.642 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-10T23:18:32.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-10T23:18:32.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-10T23:18:32.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T23:18:32.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T23:18:32.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T23:18:32.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T23:18:32.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T23:18:32.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T23:18:32.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T23:18:32.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T23:18:32.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T23:18:32.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T23:18:32.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T23:18:32.643 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T23:18:32.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T23:18:32.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T23:18:32.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T23:18:32.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T23:18:32.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T23:18:32.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T23:18:32.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T23:18:32.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T23:18:32.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T23:18:32.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T23:18:32.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T23:18:32.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T23:18:32.644 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T23:18:32.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T23:18:32.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T23:18:32.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T23:18:32.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T23:18:32.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T23:18:32.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T23:18:32.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T23:18:32.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T23:18:32.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T23:18:32.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T23:18:32.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T23:18:32.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T23:18:32.645 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T23:18:32.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T23:18:32.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T23:18:32.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T23:18:32.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T23:18:32.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T23:18:32.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T23:18:32.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T23:18:32.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T23:18:32.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T23:18:32.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T23:18:32.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T23:18:32.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T23:18:32.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T23:18:32.646 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T23:18:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T23:18:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T23:18:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T23:18:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T23:18:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T23:18:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T23:18:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T23:18:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T23:18:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T23:18:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T23:18:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T23:18:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T23:18:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T23:18:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T23:18:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T23:18:32.648 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T23:18:32.648 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T23:18:32.648 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-10T23:18:32.648 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T23:18:32.648 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T23:18:32.648 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-10T23:18:32.648 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T23:18:32.648 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-10T23:18:32.648 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T23:18:32.648 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-10T23:18:32.648 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T23:18:32.648 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T23:18:32.648 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T23:18:32.648 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T23:18:32.649 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-10T23:18:32.649 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T23:18:32.649 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-10T23:18:32.649 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-10T23:18:32.649 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-10T23:18:32.649 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-10T23:18:32.649 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T23:18:32.649 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T23:18:32.649 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T23:18:32.649 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-10T23:18:32.649 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-10T23:18:32.649 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-10T23:18:32.649 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T23:18:32.649 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T23:18:32.649 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T23:18:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T23:18:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T23:18:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T23:18:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T23:18:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T23:18:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T23:18:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T23:18:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T23:18:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-10T23:18:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T23:18:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T23:18:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T23:18:32.650 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T23:18:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T23:18:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T23:18:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T23:18:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T23:18:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T23:18:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T23:18:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T23:18:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T23:18:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T23:18:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T23:18:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-10T23:18:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T23:18:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T23:18:32.651 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T23:18:32.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T23:18:32.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T23:18:32.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T23:18:32.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T23:18:32.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T23:18:32.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T23:18:32.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T23:18:32.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T23:18:32.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T23:18:32.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T23:18:32.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T23:18:32.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T23:18:32.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T23:18:32.652 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T23:18:32.653 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T23:18:32.653 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-10T23:18:32.653 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T23:18:32.653 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T23:18:32.653 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-10T23:18:32.653 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-10T23:18:32.653 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T23:18:32.653 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T23:18:32.653 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T23:18:32.653 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-10T23:18:32.653 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T23:18:32.653 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-10T23:18:32.653 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T23:18:32.654 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T23:18:32.654 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-10T23:18:32.654 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T23:18:32.654 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-10T23:18:32.654 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T23:18:32.654 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T23:18:32.654 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T23:18:32.654 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T23:18:32.654 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T23:18:32.654 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-10T23:18:32.654 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-10T23:18:32.654 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-10T23:18:32.654 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-10T23:18:32.654 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T23:18:32.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-10T23:18:32.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T23:18:32.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T23:18:32.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T23:18:32.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T23:18:32.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T23:18:32.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T23:18:32.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T23:18:32.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T23:18:32.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T23:18:32.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T23:18:32.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T23:18:32.655 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T23:18:32.656 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T23:18:32.656 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-10T23:18:32.656 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-10T23:18:32.656 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-10T23:18:32.656 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T23:18:32.656 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-10T23:18:32.656 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-10T23:18:32.661 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:32.662 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:32.662 DEBUG:teuthology.orchestra.run.smithi094:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-10T23:18:32.678 INFO:teuthology.orchestra.run.smithi094.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T23:18:32.678 INFO:teuthology.orchestra.run.smithi094.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T23:18:32.678 INFO:teuthology.orchestra.run.smithi094.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T23:18:32.679 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-10T23:18:32.679 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:32.679 DEBUG:teuthology.orchestra.run.smithi094:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-10T23:18:32.732 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi094.front.sepia.ceph.com to complete... 2024-09-10T23:18:32.737 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-10T23:18:32.737 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-10T23:18:32.737 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-10T23:18:32.737 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-10T23:18:32.737 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-10T23:18:32.737 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-10T23:18:32.737 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-10T23:18:32.737 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-10T23:18:32.737 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T23:18:32.737 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-10T23:18:32.738 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-10T23:18:32.738 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-10T23:18:32.738 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-10T23:18:32.738 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-10T23:18:32.738 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T23:18:32.738 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T23:18:32.738 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T23:18:32.738 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T23:18:32.738 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T23:18:32.738 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T23:18:32.738 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T23:18:32.738 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T23:18:32.738 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T23:18:32.738 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T23:18:32.739 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T23:18:32.739 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T23:18:32.739 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T23:18:32.739 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T23:18:32.739 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T23:18:32.739 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T23:18:32.739 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T23:18:32.739 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T23:18:32.739 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T23:18:32.739 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-10T23:18:32.739 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-10T23:18:32.739 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-10T23:18:32.739 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T23:18:32.740 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-10T23:18:32.740 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-10T23:18:32.740 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-10T23:18:32.740 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T23:18:32.740 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T23:18:32.740 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T23:18:32.740 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-10T23:18:32.740 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-10T23:18:32.740 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-10T23:18:32.740 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-10T23:18:32.740 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T23:18:32.740 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T23:18:32.740 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T23:18:32.741 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-10T23:18:32.741 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-10T23:18:32.741 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-10T23:18:32.741 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T23:18:32.741 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T23:18:32.741 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T23:18:32.741 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T23:18:32.741 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T23:18:32.741 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T23:18:32.741 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T23:18:32.741 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T23:18:32.741 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T23:18:32.741 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T23:18:32.741 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T23:18:32.742 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T23:18:32.742 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T23:18:32.742 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T23:18:32.742 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T23:18:32.742 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T23:18:32.742 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T23:18:32.742 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T23:18:32.742 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-10T23:18:32.742 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T23:18:32.742 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-10T23:18:32.742 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T23:18:32.742 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T23:18:32.743 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T23:18:32.743 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T23:18:32.743 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T23:18:32.743 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T23:18:32.743 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T23:18:32.743 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T23:18:32.743 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-10T23:18:32.743 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T23:18:32.743 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T23:18:32.743 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T23:18:32.743 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T23:18:32.743 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T23:18:32.743 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T23:18:32.743 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T23:18:32.744 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T23:18:32.744 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T23:18:32.744 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T23:18:32.744 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T23:18:32.744 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T23:18:32.744 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T23:18:32.744 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T23:18:32.744 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T23:18:32.744 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T23:18:32.744 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T23:18:32.744 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-10T23:18:32.744 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T23:18:32.744 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-10T23:18:32.744 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-10T23:18:32.745 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-10T23:18:32.745 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T23:18:32.745 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T23:18:32.745 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-10T23:18:32.745 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-10T23:18:32.745 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T23:18:32.745 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T23:18:32.745 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T23:18:32.745 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T23:18:32.745 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T23:18:32.745 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-10T23:18:32.745 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T23:18:32.745 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-10T23:18:32.745 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T23:18:32.746 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-10T23:18:32.746 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T23:18:32.746 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T23:18:32.746 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T23:18:32.746 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T23:18:32.746 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T23:18:32.746 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T23:18:32.746 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T23:18:32.746 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T23:18:32.746 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T23:18:32.746 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T23:18:32.746 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T23:18:32.746 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T23:18:32.746 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T23:18:32.747 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T23:18:32.747 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T23:18:32.747 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T23:18:32.747 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T23:18:32.747 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T23:18:32.747 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-10T23:18:32.747 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T23:18:32.747 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T23:18:32.747 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T23:18:32.747 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T23:18:32.747 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T23:18:32.747 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T23:18:32.747 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T23:18:32.747 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T23:18:32.748 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T23:18:32.748 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T23:18:32.748 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T23:18:32.748 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T23:18:32.748 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T23:18:32.748 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-10T23:18:32.748 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T23:18:32.748 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-10T23:18:32.748 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T23:18:32.748 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T23:18:32.748 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T23:18:32.748 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T23:18:32.748 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T23:18:32.749 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T23:18:32.749 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T23:18:32.749 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T23:18:32.749 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T23:18:32.749 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T23:18:32.749 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T23:18:32.749 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T23:18:32.749 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T23:18:32.749 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T23:18:32.749 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T23:18:32.749 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T23:18:32.749 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T23:18:32.749 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T23:18:32.749 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T23:18:32.750 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T23:18:32.750 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-10T23:18:32.750 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T23:18:32.750 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-10T23:18:32.750 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T23:18:32.750 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-10T23:18:32.750 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-10T23:18:32.750 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T23:18:32.750 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-10T23:18:32.750 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T23:18:32.750 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T23:18:32.750 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-10T23:18:32.750 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T23:18:32.751 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T23:18:32.751 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-10T23:18:32.751 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T23:18:32.751 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T23:18:32.751 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T23:18:32.751 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-10T23:18:32.751 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T23:18:32.751 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-10T23:18:32.751 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T23:18:32.751 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-10T23:18:32.751 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-10T23:18:32.751 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T23:18:32.751 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-10T23:18:32.751 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T23:18:32.751 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T23:18:32.752 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T23:18:32.752 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T23:18:32.752 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T23:18:32.752 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-10T23:18:32.752 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T23:18:32.752 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T23:18:32.752 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T23:18:32.752 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T23:18:32.752 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T23:18:32.752 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T23:18:32.752 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T23:18:32.752 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-10T23:18:32.752 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T23:18:32.753 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-10T23:18:32.753 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T23:18:32.753 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T23:18:32.753 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-10T23:18:32.753 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T23:18:32.753 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-10T23:18:32.753 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-10T23:18:32.753 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-10T23:18:32.753 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T23:18:32.753 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T23:18:32.753 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-10T23:18:32.753 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-10T23:18:32.753 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-10T23:18:32.753 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-10T23:18:32.754 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-10T23:18:32.754 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-10T23:18:32.754 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-10T23:18:32.754 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-10T23:18:32.754 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-10T23:18:32.754 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-10T23:18:32.754 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-10T23:18:32.754 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T23:18:32.754 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T23:18:32.754 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T23:18:32.754 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-10T23:18:32.754 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-10T23:18:32.754 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T23:18:32.754 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T23:18:32.755 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-10T23:18:32.755 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-10T23:18:32.755 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-10T23:18:32.755 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T23:18:32.755 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-10T23:18:32.755 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T23:18:32.755 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T23:18:32.755 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T23:18:32.755 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-10T23:18:32.755 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T23:18:32.755 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-10T23:18:32.755 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T23:18:32.755 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-10T23:18:32.755 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-10T23:18:32.756 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T23:18:32.756 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-10T23:18:32.756 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T23:18:32.756 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-10T23:18:32.756 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T23:18:32.756 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T23:18:32.756 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T23:18:32.756 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T23:18:32.756 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T23:18:32.756 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T23:18:32.756 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T23:18:32.756 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T23:18:32.756 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T23:18:32.756 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T23:18:32.757 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T23:18:32.757 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T23:18:32.757 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T23:18:32.757 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-10T23:18:32.757 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-10T23:18:32.757 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-10T23:18:32.757 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-10T23:18:32.757 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-10T23:18:32.757 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-10T23:18:32.757 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-10T23:18:32.757 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-10T23:18:32.757 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-10T23:18:32.757 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-10T23:18:32.757 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-10T23:18:32.758 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-10T23:18:32.758 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T23:18:32.758 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-10T23:18:32.758 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-10T23:18:32.758 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-10T23:18:32.758 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-10T23:18:32.758 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-10T23:18:32.758 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-10T23:18:32.758 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-10T23:18:32.758 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-10T23:18:32.758 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-10T23:18:32.758 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T23:18:32.758 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-10T23:18:32.758 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-10T23:18:32.759 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-10T23:18:32.759 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-10T23:18:32.759 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-10T23:18:32.759 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-10T23:18:32.759 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-10T23:18:32.759 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-10T23:18:32.759 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-10T23:18:32.759 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-10T23:18:32.759 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-10T23:18:32.759 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-10T23:18:32.759 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-10T23:18:32.759 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-10T23:18:32.759 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-10T23:18:32.759 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T23:18:32.760 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-10T23:18:32.760 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-10T23:18:32.760 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-10T23:18:32.760 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-10T23:18:32.760 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-10T23:18:32.760 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-10T23:18:32.760 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-10T23:18:32.760 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T23:18:32.760 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T23:18:32.760 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T23:18:32.760 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-10T23:18:32.760 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-10T23:18:32.761 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-10T23:18:32.761 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-10T23:18:32.761 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-10T23:18:32.761 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T23:18:32.761 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T23:18:32.761 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T23:18:32.761 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T23:18:32.761 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-10T23:18:32.761 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T23:18:32.761 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-10T23:18:32.761 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T23:18:32.761 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T23:18:32.761 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T23:18:32.761 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T23:18:32.762 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T23:18:32.762 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T23:18:32.762 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T23:18:32.762 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-10T23:18:32.762 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T23:18:32.762 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T23:18:32.762 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T23:18:32.762 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T23:18:32.762 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T23:18:32.762 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-10T23:18:32.762 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T23:18:32.762 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T23:18:32.762 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-10T23:18:32.763 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-10T23:18:32.763 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-10T23:18:32.763 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-10T23:18:32.763 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T23:18:32.763 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-10T23:18:32.763 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T23:18:32.763 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T23:18:32.763 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T23:18:32.763 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T23:18:32.763 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T23:18:32.763 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T23:18:32.763 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T23:18:32.763 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T23:18:32.763 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T23:18:32.763 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T23:18:32.764 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T23:18:32.764 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T23:18:32.764 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T23:18:32.764 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T23:18:32.764 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T23:18:32.764 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T23:18:32.764 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T23:18:32.764 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T23:18:32.764 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T23:18:32.764 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T23:18:32.764 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T23:18:32.764 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T23:18:32.764 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T23:18:32.764 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T23:18:32.765 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T23:18:32.765 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T23:18:32.765 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T23:18:32.765 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T23:18:32.765 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T23:18:32.765 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T23:18:32.765 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T23:18:32.765 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T23:18:32.765 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T23:18:32.765 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T23:18:32.765 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T23:18:32.765 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T23:18:32.765 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T23:18:32.765 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T23:18:32.766 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T23:18:32.766 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-10T23:18:32.766 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T23:18:32.766 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-10T23:18:32.766 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-10T23:18:32.766 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-10T23:18:32.766 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-10T23:18:32.766 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T23:18:32.766 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-10T23:18:32.766 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T23:18:32.766 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T23:18:32.766 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T23:18:32.766 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-10T23:18:32.766 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T23:18:32.767 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T23:18:32.767 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T23:18:32.767 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T23:18:32.767 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T23:18:32.767 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T23:18:32.767 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T23:18:32.767 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T23:18:32.767 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T23:18:32.767 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T23:18:32.767 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T23:18:32.767 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T23:18:32.767 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T23:18:32.767 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T23:18:32.767 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T23:18:32.768 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T23:18:32.768 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T23:18:32.768 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T23:18:32.768 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T23:18:32.768 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T23:18:32.768 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T23:18:32.768 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T23:18:32.768 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T23:18:32.768 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T23:18:32.768 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T23:18:32.768 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-10T23:18:32.768 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-10T23:18:32.768 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-10T23:18:32.768 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T23:18:32.769 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T23:18:32.769 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T23:18:32.769 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T23:18:32.769 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T23:18:32.769 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-10T23:18:32.769 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T23:18:32.769 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T23:18:32.769 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T23:18:32.769 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T23:18:32.769 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T23:18:32.769 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T23:18:32.769 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T23:18:32.769 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T23:18:32.769 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T23:18:32.769 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T23:18:32.770 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-10T23:18:32.770 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T23:18:32.770 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T23:18:32.770 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T23:18:32.770 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T23:18:32.770 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T23:18:32.770 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T23:18:32.770 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T23:18:32.770 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T23:18:32.770 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-10T23:18:32.770 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T23:18:32.770 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T23:18:32.770 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T23:18:32.771 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T23:18:32.771 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T23:18:32.771 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T23:18:32.771 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T23:18:32.771 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T23:18:32.771 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T23:18:32.771 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T23:18:32.771 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T23:18:32.771 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T23:18:32.771 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T23:18:32.771 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T23:18:32.771 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T23:18:32.771 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T23:18:32.771 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T23:18:32.771 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T23:18:32.772 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T23:18:32.772 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T23:18:32.772 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T23:18:32.772 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T23:18:32.772 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T23:18:32.772 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T23:18:32.772 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T23:18:32.772 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T23:18:32.772 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T23:18:32.772 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T23:18:32.772 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T23:18:32.772 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T23:18:32.772 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T23:18:32.773 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T23:18:32.773 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-10T23:18:32.773 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T23:18:32.773 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-10T23:18:32.773 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T23:18:32.773 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T23:18:32.773 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T23:18:32.773 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T23:18:32.773 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T23:18:32.773 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T23:18:32.773 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T23:18:32.773 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-10T23:18:32.773 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T23:18:32.774 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T23:18:32.774 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T23:18:32.774 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T23:18:32.774 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T23:18:32.774 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T23:18:32.774 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T23:18:32.774 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T23:18:32.774 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T23:18:32.774 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T23:18:32.774 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T23:18:32.774 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T23:18:32.774 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T23:18:32.774 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T23:18:32.774 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T23:18:32.774 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T23:18:32.775 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T23:18:32.775 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T23:18:32.775 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T23:18:32.775 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T23:18:32.775 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T23:18:32.775 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T23:18:32.775 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T23:18:32.775 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T23:18:32.775 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T23:18:32.775 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T23:18:32.775 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T23:18:32.775 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T23:18:32.775 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T23:18:32.775 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T23:18:32.776 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T23:18:32.776 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T23:18:32.776 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T23:18:32.776 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T23:18:32.776 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T23:18:32.776 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T23:18:32.776 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T23:18:32.776 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T23:18:32.776 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T23:18:32.776 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T23:18:32.776 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T23:18:32.776 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-10T23:18:32.776 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-10T23:18:32.776 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-10T23:18:32.777 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-10T23:18:32.777 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T23:18:32.777 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T23:18:32.777 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-10T23:18:32.777 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-10T23:18:32.777 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-10T23:18:32.777 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-10T23:18:32.777 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T23:18:32.777 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T23:18:32.777 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T23:18:32.777 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-10T23:18:32.777 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T23:18:32.777 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-10T23:18:32.777 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T23:18:32.778 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T23:18:32.778 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T23:18:32.778 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T23:18:32.778 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-10T23:18:32.778 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-10T23:18:32.778 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T23:18:32.778 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T23:18:32.778 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T23:18:32.778 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-10T23:18:32.778 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T23:18:32.778 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T23:18:32.778 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-10T23:18:32.778 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T23:18:32.779 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T23:18:32.779 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T23:18:32.779 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-10T23:18:32.779 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T23:18:32.779 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T23:18:32.779 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-10T23:18:32.779 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-10T23:18:32.779 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T23:18:32.779 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-10T23:18:32.779 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T23:18:32.779 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T23:18:32.779 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T23:18:32.779 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T23:18:32.779 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-10T23:18:32.780 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T23:18:32.780 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T23:18:32.780 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T23:18:32.780 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-10T23:18:32.780 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-10T23:18:32.780 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-10T23:18:32.780 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-10T23:18:32.780 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-10T23:18:32.780 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-10T23:18:32.780 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T23:18:32.780 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T23:18:32.780 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T23:18:32.780 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T23:18:32.780 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T23:18:32.781 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T23:18:32.781 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T23:18:32.781 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-10T23:18:32.781 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-10T23:18:32.781 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T23:18:32.781 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-10T23:18:32.781 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T23:18:32.781 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-10T23:18:32.781 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T23:18:32.781 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-10T23:18:32.781 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T23:18:32.781 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-10T23:18:32.781 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-10T23:18:32.781 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-10T23:18:32.781 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-10T23:18:32.782 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-10T23:18:32.782 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-10T23:18:32.782 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T23:18:32.782 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-10T23:18:32.782 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T23:18:32.782 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-10T23:18:32.782 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T23:18:32.782 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-10T23:18:32.782 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T23:18:32.782 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-10T23:18:32.782 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T23:18:32.782 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-10T23:18:32.782 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-10T23:18:32.782 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-10T23:18:32.783 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-10T23:18:32.783 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T23:18:32.783 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T23:18:32.783 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T23:18:32.783 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T23:18:32.783 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-10T23:18:32.783 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-10T23:18:32.783 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-10T23:18:32.783 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-10T23:18:32.783 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-10T23:18:32.783 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T23:18:32.783 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-10T23:18:32.783 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-10T23:18:32.783 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-10T23:18:32.784 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-10T23:18:32.784 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-10T23:18:32.784 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-10T23:18:32.784 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-10T23:18:32.784 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-10T23:18:32.784 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-10T23:18:32.784 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-10T23:18:32.784 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-10T23:18:32.784 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-10T23:18:32.784 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-10T23:18:32.784 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-10T23:18:32.784 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-10T23:18:32.784 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-10T23:18:32.784 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-10T23:18:32.785 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-10T23:18:32.785 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-10T23:18:32.785 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-10T23:18:32.785 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-10T23:18:32.785 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T23:18:32.785 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T23:18:32.785 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T23:18:32.785 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T23:18:32.785 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-10T23:18:32.785 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-10T23:18:32.785 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T23:18:32.785 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T23:18:32.785 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T23:18:32.785 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T23:18:32.786 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-10T23:18:32.786 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T23:18:32.786 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-10T23:18:32.786 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T23:18:32.786 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T23:18:32.786 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-10T23:18:32.786 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T23:18:32.786 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-10T23:18:32.786 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-10T23:18:32.786 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-10T23:18:32.786 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T23:18:32.786 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-10T23:18:32.786 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T23:18:32.787 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-10T23:18:32.787 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-10T23:18:32.787 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-10T23:18:32.787 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-10T23:18:32.787 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T23:18:32.787 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-10T23:18:32.787 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-10T23:18:32.787 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-10T23:18:32.787 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T23:18:32.787 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-10T23:18:32.787 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-10T23:18:32.787 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-10T23:18:32.787 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T23:18:32.787 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-10T23:18:32.788 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-10T23:18:32.788 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-10T23:18:32.788 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-10T23:18:32.788 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-10T23:18:32.788 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-10T23:18:32.788 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-10T23:18:32.788 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T23:18:32.788 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-10T23:18:32.788 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-10T23:18:32.788 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-10T23:18:32.788 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-10T23:18:32.788 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T23:18:32.788 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T23:18:32.788 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T23:18:32.789 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T23:18:32.789 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T23:18:32.789 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T23:18:32.789 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T23:18:32.789 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T23:18:32.789 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-10T23:18:32.789 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-10T23:18:32.789 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T23:18:32.789 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T23:18:32.789 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-10T23:18:32.789 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T23:18:32.789 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T23:18:32.789 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T23:18:32.790 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T23:18:32.790 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-10T23:18:32.790 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T23:18:32.790 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T23:18:32.790 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-10T23:18:32.790 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-10T23:18:32.790 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T23:18:32.790 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T23:18:32.790 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T23:18:32.790 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-10T23:18:32.790 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T23:18:32.790 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T23:18:32.790 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-10T23:18:32.790 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T23:18:32.791 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T23:18:32.791 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T23:18:32.791 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-10T23:18:32.791 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-10T23:18:32.791 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T23:18:32.791 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T23:18:32.791 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T23:18:32.791 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T23:18:32.791 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T23:18:32.791 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-10T23:18:32.791 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-10T23:18:32.791 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T23:18:32.791 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T23:18:32.791 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T23:18:32.792 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T23:18:32.792 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T23:18:32.792 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T23:18:32.792 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T23:18:32.792 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-10T23:18:32.792 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-10T23:18:32.792 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-10T23:18:32.792 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-10T23:18:32.792 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-10T23:18:32.792 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-10T23:18:32.792 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-10T23:18:32.792 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-10T23:18:32.792 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-10T23:18:32.793 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-10T23:18:32.793 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-10T23:18:32.793 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-10T23:18:32.793 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-10T23:18:32.793 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-10T23:18:32.793 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T23:18:32.793 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T23:18:32.793 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-10T23:18:32.793 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-10T23:18:32.793 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-10T23:18:32.793 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-10T23:18:32.793 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-10T23:18:32.793 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-10T23:18:32.794 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-10T23:18:32.794 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-10T23:18:32.794 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-10T23:18:32.794 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-10T23:18:32.794 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T23:18:32.794 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-10T23:18:32.794 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-10T23:18:32.794 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T23:18:32.794 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T23:18:32.794 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T23:18:32.794 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-10T23:18:32.794 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-10T23:18:32.794 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-10T23:18:32.794 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-10T23:18:32.795 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-10T23:18:32.795 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T23:18:32.795 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T23:18:32.795 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T23:18:32.795 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T23:18:32.795 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-10T23:18:32.795 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-10T23:18:32.795 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-10T23:18:32.795 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-10T23:18:32.795 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T23:18:32.795 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-10T23:18:32.795 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-10T23:18:32.795 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-10T23:18:32.795 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-10T23:18:32.796 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-10T23:18:32.796 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T23:18:32.796 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T23:18:32.796 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T23:18:32.796 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T23:18:32.796 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-10T23:18:32.796 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-10T23:18:32.796 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-10T23:18:32.796 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T23:18:32.796 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-10T23:18:32.796 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-10T23:18:32.796 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-10T23:18:32.796 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T23:18:32.796 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T23:18:32.797 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-10T23:18:32.797 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T23:18:32.797 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T23:18:32.797 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-10T23:18:32.797 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T23:18:32.797 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T23:18:32.797 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T23:18:32.797 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-10T23:18:32.797 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-10T23:18:32.797 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-10T23:18:32.797 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T23:18:32.797 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T23:18:32.797 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T23:18:32.798 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T23:18:32.798 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-10T23:18:32.798 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T23:18:32.798 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-10T23:18:32.798 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-10T23:18:32.798 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T23:18:32.798 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-10T23:18:32.798 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-10T23:18:32.798 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T23:18:32.798 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T23:18:32.798 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T23:18:32.798 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-10T23:18:32.798 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T23:18:32.798 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T23:18:32.799 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T23:18:32.799 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-10T23:18:32.799 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-10T23:18:32.799 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T23:18:32.799 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T23:18:32.799 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T23:18:32.799 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T23:18:32.799 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-10T23:18:32.799 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T23:18:32.799 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T23:18:32.799 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T23:18:32.799 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T23:18:32.799 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-10T23:18:32.800 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T23:18:32.800 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-10T23:18:32.800 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T23:18:32.800 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-10T23:18:32.800 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T23:18:32.800 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-10T23:18:32.800 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T23:18:32.800 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-10T23:18:32.800 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T23:18:32.800 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T23:18:32.800 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T23:18:32.800 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-10T23:18:32.800 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-10T23:18:32.800 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T23:18:32.801 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-10T23:18:32.801 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T23:18:32.801 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-10T23:18:32.801 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T23:18:32.801 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-10T23:18:32.801 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T23:18:32.801 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T23:18:32.801 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-10T23:18:32.801 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T23:18:32.801 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T23:18:32.801 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-10T23:18:32.801 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T23:18:32.801 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T23:18:32.801 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-10T23:18:32.802 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T23:18:32.802 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T23:18:32.802 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-10T23:18:32.802 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-10T23:18:32.802 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T23:18:32.802 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-10T23:18:32.802 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-10T23:18:32.802 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T23:18:32.802 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-10T23:18:32.802 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T23:18:32.802 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-10T23:18:32.802 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T23:18:32.802 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-10T23:18:32.802 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T23:18:32.803 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-10T23:18:32.803 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T23:18:32.803 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-10T23:18:32.803 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-10T23:18:32.803 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T23:18:32.803 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-10T23:18:32.803 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-10T23:18:32.803 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T23:18:32.803 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T23:18:32.803 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T23:18:32.803 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T23:18:32.803 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T23:18:32.803 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T23:18:32.803 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-10T23:18:32.804 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T23:18:32.804 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T23:18:32.804 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T23:18:32.804 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-10T23:18:32.804 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-10T23:18:32.804 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-10T23:18:32.804 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T23:18:32.804 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T23:18:32.804 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T23:18:32.804 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-10T23:18:32.804 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-10T23:18:32.804 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-10T23:18:32.804 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T23:18:32.805 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-10T23:18:32.805 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T23:18:32.805 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-10T23:18:32.805 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T23:18:32.805 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-10T23:18:32.805 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T23:18:32.805 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-10T23:18:32.805 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T23:18:32.805 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-10T23:18:32.805 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T23:18:32.805 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-10T23:18:32.805 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T23:18:32.805 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-10T23:18:32.806 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T23:18:32.806 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-10T23:18:32.806 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T23:18:32.806 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-10T23:18:32.806 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T23:18:32.806 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-10T23:18:32.806 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T23:18:32.806 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-10T23:18:32.806 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T23:18:32.806 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T23:18:32.806 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-10T23:18:32.806 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T23:18:32.806 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T23:18:32.806 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T23:18:32.807 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-10T23:18:32.807 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-10T23:18:32.807 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T23:18:32.807 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T23:18:32.807 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-10T23:18:32.807 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-10T23:18:32.807 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T23:18:32.807 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-10T23:18:32.807 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-10T23:18:32.807 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-10T23:18:32.807 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-10T23:18:32.807 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T23:18:32.807 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-10T23:18:32.808 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-10T23:18:32.808 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T23:18:32.808 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-10T23:18:32.808 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T23:18:32.808 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T23:18:32.808 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-10T23:18:32.808 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T23:18:32.808 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-10T23:18:32.808 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T23:18:32.808 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-10T23:18:32.808 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T23:18:32.808 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T23:18:32.808 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-10T23:18:32.809 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-10T23:18:32.809 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T23:18:32.809 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T23:18:32.812 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-10T23:18:32.812 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T23:18:32.812 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-10T23:18:32.812 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T23:18:32.812 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-10T23:18:32.812 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T23:18:32.812 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T23:18:32.812 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T23:18:32.812 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T23:18:32.813 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-10T23:18:32.813 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-10T23:18:32.813 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-10T23:18:32.813 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-10T23:18:32.813 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T23:18:32.813 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-10T23:18:32.813 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T23:18:32.813 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-10T23:18:32.813 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T23:18:32.813 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T23:18:32.813 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-10T23:18:32.813 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T23:18:32.813 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T23:18:32.813 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T23:18:32.814 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T23:18:32.814 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-10T23:18:32.814 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-10T23:18:32.814 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-10T23:18:32.814 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T23:18:32.814 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-10T23:18:32.814 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T23:18:32.814 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-10T23:18:32.814 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T23:18:32.814 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T23:18:32.814 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T23:18:32.814 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T23:18:32.814 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-10T23:18:32.815 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T23:18:32.815 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-10T23:18:32.815 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T23:18:32.815 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-10T23:18:32.815 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T23:18:32.815 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T23:18:32.815 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-10T23:18:32.815 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T23:18:32.815 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T23:18:32.815 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-10T23:18:32.815 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T23:18:32.815 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-10T23:18:32.815 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-10T23:18:32.816 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-10T23:18:32.816 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-10T23:18:32.816 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T23:18:32.816 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-10T23:18:32.816 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T23:18:32.888 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-10T23:18:32.888 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T23:18:32.889 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T23:18:32.889 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-10T23:18:32.889 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T23:18:32.889 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T23:18:32.889 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T23:18:32.889 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-10T23:18:32.889 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T23:18:32.889 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T23:18:32.889 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-10T23:18:32.889 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T23:18:32.889 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T23:18:32.890 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T23:18:32.890 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T23:18:32.890 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T23:18:32.890 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-10T23:18:32.890 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T23:18:32.890 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T23:18:32.890 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-10T23:18:32.890 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T23:18:32.890 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T23:18:32.890 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T23:18:32.890 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T23:18:32.890 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T23:18:32.890 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T23:18:32.890 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T23:18:32.891 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T23:18:32.891 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T23:18:32.891 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T23:18:32.891 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T23:18:32.891 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T23:18:32.891 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T23:18:32.891 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T23:18:32.891 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T23:18:32.891 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T23:18:32.891 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T23:18:32.891 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T23:18:32.891 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T23:18:32.891 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-10T23:18:32.892 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-10T23:18:32.892 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-10T23:18:32.892 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T23:18:32.892 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T23:18:32.892 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T23:18:32.892 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T23:18:32.892 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T23:18:32.892 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T23:18:32.892 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T23:18:32.892 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T23:18:32.892 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-10T23:18:32.892 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T23:18:32.893 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T23:18:32.893 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T23:18:32.893 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-10T23:18:32.893 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T23:18:32.893 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T23:18:32.893 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T23:18:32.893 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T23:18:32.893 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T23:18:32.893 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T23:18:32.893 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T23:18:32.893 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-10T23:18:32.893 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T23:18:32.893 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-10T23:18:32.894 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T23:18:32.894 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-10T23:18:32.894 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T23:18:32.894 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-10T23:18:32.894 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T23:18:32.894 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-10T23:18:32.894 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-10T23:18:32.894 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T23:18:32.894 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T23:18:32.894 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-10T23:18:32.894 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T23:18:32.894 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T23:18:32.894 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T23:18:32.894 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T23:18:32.895 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-10T23:18:32.895 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-10T23:18:32.895 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T23:18:32.895 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-10T23:18:32.895 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-10T23:18:32.895 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-10T23:18:32.895 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T23:18:32.895 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-10T23:18:32.895 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T23:18:32.895 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-10T23:18:32.895 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-10T23:18:32.895 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-10T23:18:32.895 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T23:18:32.896 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T23:18:32.896 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-10T23:18:32.896 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-10T23:18:32.896 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-10T23:18:32.896 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-10T23:18:32.896 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-10T23:18:32.896 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-10T23:18:32.896 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-10T23:18:32.896 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T23:18:32.896 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-10T23:18:32.896 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-10T23:18:32.896 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-10T23:18:32.896 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-10T23:18:32.897 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-10T23:18:32.897 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-10T23:18:32.897 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-10T23:18:32.897 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-10T23:18:32.897 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-10T23:18:32.897 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-10T23:18:32.897 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T23:18:32.897 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T23:18:32.897 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T23:18:32.897 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T23:18:32.897 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T23:18:32.897 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T23:18:32.897 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T23:18:32.898 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T23:18:32.898 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T23:18:32.898 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T23:18:32.898 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T23:18:32.898 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T23:18:32.898 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-10T23:18:32.898 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-10T23:18:32.898 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T23:18:32.898 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-10T23:18:32.898 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T23:18:32.898 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T23:18:32.898 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T23:18:32.899 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T23:18:32.899 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T23:18:32.899 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T23:18:32.899 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T23:18:32.899 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T23:18:32.899 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T23:18:32.899 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T23:18:32.899 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T23:18:32.899 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-10T23:18:32.899 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-10T23:18:32.899 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-10T23:18:32.899 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-10T23:18:32.899 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-10T23:18:32.900 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-10T23:18:32.900 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-10T23:18:32.900 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-10T23:18:32.900 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-10T23:18:32.900 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-10T23:18:32.900 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-10T23:18:32.900 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-10T23:18:32.900 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-10T23:18:32.900 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-10T23:18:32.900 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-10T23:18:32.900 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-10T23:18:32.900 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-10T23:18:32.900 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-10T23:18:32.900 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-10T23:18:32.901 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-10T23:18:32.901 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-10T23:18:32.901 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-10T23:18:32.901 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-10T23:18:32.901 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-10T23:18:32.901 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-10T23:18:32.901 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-10T23:18:32.901 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-10T23:18:32.901 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-10T23:18:32.901 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-10T23:18:32.901 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-10T23:18:32.901 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T23:18:32.901 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T23:18:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-10T23:18:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-10T23:18:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T23:18:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T23:18:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T23:18:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T23:18:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-10T23:18:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T23:18:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T23:18:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-10T23:18:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-10T23:18:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-10T23:18:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T23:18:32.903 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T23:18:32.903 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-10T23:18:32.903 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-10T23:18:32.903 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-10T23:18:32.903 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T23:18:32.903 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-10T23:18:32.903 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-10T23:18:32.903 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-10T23:18:32.903 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T23:18:32.903 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T23:18:32.903 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T23:18:32.903 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T23:18:32.903 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T23:18:32.904 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T23:18:32.904 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T23:18:32.904 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T23:18:32.904 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T23:18:32.904 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T23:18:32.904 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T23:18:32.904 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T23:18:32.904 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T23:18:32.904 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-10T23:18:32.904 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T23:18:32.904 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T23:18:32.904 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T23:18:32.904 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-10T23:18:32.905 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T23:18:32.905 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-10T23:18:32.905 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-10T23:18:32.905 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-10T23:18:32.905 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-10T23:18:32.905 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-10T23:18:32.905 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T23:18:32.905 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-10T23:18:32.905 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T23:18:32.905 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-10T23:18:32.905 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T23:18:32.905 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T23:18:32.906 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T23:18:32.906 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T23:18:32.906 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-10T23:18:32.906 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T23:18:32.906 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T23:18:32.906 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T23:18:32.906 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T23:18:32.906 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T23:18:32.906 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-10T23:18:32.906 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T23:18:32.906 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-10T23:18:32.906 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-10T23:18:32.906 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-10T23:18:32.906 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-10T23:18:32.907 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T23:18:32.907 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-10T23:18:32.907 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T23:18:32.907 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T23:18:32.907 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T23:18:32.907 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T23:18:32.907 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T23:18:32.907 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-10T23:18:32.907 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T23:18:32.907 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T23:18:32.907 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-10T23:18:32.907 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T23:18:32.907 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T23:18:32.907 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-10T23:18:32.907 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-10T23:18:32.908 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T23:18:32.908 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-10T23:18:32.908 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T23:18:32.908 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T23:18:32.908 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-10T23:18:32.908 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T23:18:32.908 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-10T23:18:32.908 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T23:18:32.908 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-10T23:18:32.908 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T23:18:32.908 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-10T23:18:32.908 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T23:18:32.908 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T23:18:32.909 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-10T23:18:32.909 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T23:18:32.909 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-10T23:18:32.909 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-10T23:18:32.909 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T23:18:32.909 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-10T23:18:32.909 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T23:18:32.909 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T23:18:32.909 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T23:18:32.909 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T23:18:32.909 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-10T23:18:32.909 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T23:18:32.909 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T23:18:32.910 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-10T23:18:32.910 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T23:18:32.910 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T23:18:32.910 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-10T23:18:32.910 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-10T23:18:32.910 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T23:18:32.910 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T23:18:32.910 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T23:18:32.910 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T23:18:32.910 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-10T23:18:32.910 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T23:18:32.910 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T23:18:32.910 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T23:18:32.910 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-10T23:18:32.911 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T23:18:32.911 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T23:18:32.911 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T23:18:32.911 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-10T23:18:32.911 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T23:18:32.911 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T23:18:32.911 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T23:18:32.911 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-10T23:18:32.911 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T23:18:32.911 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-10T23:18:32.911 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T23:18:32.911 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-10T23:18:32.911 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T23:18:32.911 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T23:18:32.912 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T23:18:32.912 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T23:18:32.912 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-10T23:18:32.912 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T23:18:32.912 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-10T23:18:32.912 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T23:18:32.912 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T23:18:32.912 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-10T23:18:32.912 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-10T23:18:32.912 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-10T23:18:32.912 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-10T23:18:32.912 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-10T23:18:32.912 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-10T23:18:32.912 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T23:18:32.913 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T23:18:32.913 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-10T23:18:32.913 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T23:18:32.913 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-10T23:18:32.913 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T23:18:32.913 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T23:18:32.913 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-10T23:18:32.913 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T23:18:32.913 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-10T23:18:32.913 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T23:18:32.913 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T23:18:32.913 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T23:18:32.913 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T23:18:32.913 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T23:18:32.914 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T23:18:32.914 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T23:18:32.914 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T23:18:32.914 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T23:18:32.914 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T23:18:32.914 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-10T23:18:32.914 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T23:18:32.914 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T23:18:32.914 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T23:18:32.914 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T23:18:32.914 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-10T23:18:32.914 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-10T23:18:32.915 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T23:18:32.915 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-10T23:18:32.915 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-10T23:18:32.915 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T23:18:32.915 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-10T23:18:32.915 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T23:18:32.915 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T23:18:32.915 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T23:18:32.915 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T23:18:32.915 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T23:18:32.915 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T23:18:32.915 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-10T23:18:32.915 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T23:18:32.915 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-10T23:18:32.916 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-10T23:18:32.916 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-10T23:18:32.916 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T23:18:32.916 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T23:18:32.916 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T23:18:32.916 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T23:18:32.916 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T23:18:32.916 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T23:18:32.916 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T23:18:32.916 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T23:18:32.916 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T23:18:32.916 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T23:18:32.916 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T23:18:32.917 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T23:18:32.917 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T23:18:32.917 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T23:18:32.917 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T23:18:32.917 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-10T23:18:32.917 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T23:18:32.917 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T23:18:32.917 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T23:18:32.917 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-10T23:18:32.917 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-10T23:18:32.917 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T23:18:32.917 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-10T23:18:32.917 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-10T23:18:32.917 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T23:18:32.918 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T23:18:32.918 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T23:18:32.918 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-10T23:18:32.918 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-10T23:18:32.918 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-10T23:18:32.918 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T23:18:32.918 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-10T23:18:32.918 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-10T23:18:32.918 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T23:18:32.918 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-10T23:18:32.918 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-10T23:18:32.918 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T23:18:32.918 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-10T23:18:32.919 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-10T23:18:32.919 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-10T23:18:32.919 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-10T23:18:32.919 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-10T23:18:32.919 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T23:18:32.919 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-10T23:18:32.919 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T23:18:32.919 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-10T23:18:32.919 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-10T23:18:32.919 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T23:18:32.919 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T23:18:32.919 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T23:18:32.919 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-10T23:18:32.919 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-10T23:18:32.920 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-10T23:18:32.920 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T23:18:32.920 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-10T23:18:32.920 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T23:18:32.920 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-10T23:18:32.920 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-10T23:18:32.920 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-10T23:18:32.920 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-10T23:18:32.920 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-10T23:18:32.920 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-10T23:18:32.920 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-10T23:18:32.920 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-10T23:18:32.920 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-10T23:18:32.921 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-10T23:18:32.921 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T23:18:32.921 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T23:18:32.921 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T23:18:32.921 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T23:18:32.921 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-10T23:18:32.921 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-10T23:18:32.921 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-10T23:18:32.921 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-10T23:18:32.921 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-10T23:18:32.921 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-10T23:18:32.921 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-10T23:18:32.921 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-10T23:18:32.922 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-10T23:18:32.922 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-10T23:18:32.922 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-10T23:18:32.922 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-10T23:18:32.922 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-10T23:18:32.922 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-10T23:18:32.922 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-10T23:18:32.922 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-10T23:18:32.922 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-10T23:18:32.922 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-10T23:18:32.922 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-10T23:18:32.922 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-10T23:18:32.922 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-10T23:18:32.923 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-10T23:18:32.923 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-10T23:18:32.923 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-10T23:18:32.923 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-10T23:18:32.923 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-10T23:18:32.923 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-10T23:18:32.923 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-10T23:18:32.923 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-10T23:18:32.923 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-10T23:18:32.923 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-10T23:18:32.923 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-10T23:18:32.923 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-10T23:18:32.923 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-10T23:18:32.923 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-10T23:18:32.924 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-10T23:18:32.924 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-10T23:18:32.924 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-10T23:18:32.924 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-10T23:18:32.924 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-10T23:18:32.924 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-10T23:18:32.924 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-10T23:18:32.924 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-10T23:18:32.924 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-10T23:18:32.924 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-10T23:18:32.924 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-10T23:18:32.924 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-10T23:18:32.924 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-10T23:18:32.924 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-10T23:18:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-10T23:18:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-10T23:18:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-10T23:18:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-10T23:18:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-10T23:18:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-10T23:18:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-10T23:18:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-10T23:18:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-10T23:18:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-10T23:18:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-10T23:18:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-10T23:18:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-10T23:18:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-10T23:18:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T23:18:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-10T23:18:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-10T23:18:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-10T23:18:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-10T23:18:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T23:18:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T23:18:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-10T23:18:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T23:18:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T23:18:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-10T23:18:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-10T23:18:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-10T23:18:32.927 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T23:18:32.927 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T23:18:32.927 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T23:18:32.927 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T23:18:32.927 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T23:18:32.927 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T23:18:32.927 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T23:18:32.927 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T23:18:32.927 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-10T23:18:32.927 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-10T23:18:32.927 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T23:18:32.927 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T23:18:32.927 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T23:18:32.927 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T23:18:32.928 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T23:18:32.928 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-10T23:18:32.928 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T23:18:32.928 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T23:18:32.928 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-10T23:18:32.928 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-10T23:18:32.928 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-10T23:18:32.928 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-10T23:18:32.928 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-10T23:18:32.928 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T23:18:32.928 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-10T23:18:32.928 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-10T23:18:32.928 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-10T23:18:32.928 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-10T23:18:32.929 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T23:18:32.929 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T23:18:32.929 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T23:18:32.929 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T23:18:32.929 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T23:18:32.929 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T23:18:32.929 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T23:18:32.929 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T23:18:32.929 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T23:18:32.929 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-10T23:18:32.929 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T23:18:32.929 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T23:18:32.929 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T23:18:32.930 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T23:18:32.930 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T23:18:32.930 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T23:18:32.930 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T23:18:32.930 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T23:18:32.930 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T23:18:32.930 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T23:18:32.930 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T23:18:32.930 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T23:18:32.930 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T23:18:32.930 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-10T23:18:32.930 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T23:18:32.930 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T23:18:32.930 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-10T23:18:32.931 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T23:18:32.931 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T23:18:32.931 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T23:18:32.931 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T23:18:32.931 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T23:18:32.931 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T23:18:32.931 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T23:18:32.931 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T23:18:32.931 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T23:18:32.931 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T23:18:32.931 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T23:18:32.931 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T23:18:32.931 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T23:18:32.931 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T23:18:32.932 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T23:18:32.932 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T23:18:32.932 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T23:18:32.932 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T23:18:32.932 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-10T23:18:32.932 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T23:18:32.932 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T23:18:32.932 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-10T23:18:32.932 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T23:18:32.932 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-10T23:18:32.932 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T23:18:32.932 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T23:18:32.932 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T23:18:32.932 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T23:18:32.933 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T23:18:32.933 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T23:18:32.933 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T23:18:32.933 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T23:18:32.933 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T23:18:32.933 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T23:18:32.933 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T23:18:32.933 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T23:18:32.933 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T23:18:32.933 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T23:18:32.933 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T23:18:32.934 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-10T23:18:32.934 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T23:18:32.934 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T23:18:32.934 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T23:18:32.934 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-10T23:18:32.934 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-10T23:18:32.934 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T23:18:32.934 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T23:18:32.934 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T23:18:32.934 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T23:18:32.934 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-10T23:18:32.934 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T23:18:32.934 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T23:18:32.935 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T23:18:32.935 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T23:18:32.935 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-10T23:18:32.935 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-10T23:18:32.935 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-10T23:18:32.935 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-10T23:18:32.935 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-10T23:18:32.935 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-10T23:18:32.935 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-10T23:18:32.935 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T23:18:32.935 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-10T23:18:32.935 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-10T23:18:32.935 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-10T23:18:32.935 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-10T23:18:32.936 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-10T23:18:32.936 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-10T23:18:32.936 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T23:18:32.936 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-10T23:18:32.936 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-10T23:18:32.936 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-10T23:18:32.936 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-10T23:18:32.936 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-10T23:18:32.936 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-10T23:18:32.936 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T23:18:32.936 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-10T23:18:32.936 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-10T23:18:32.936 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-10T23:18:32.936 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T23:18:32.937 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T23:18:32.937 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-10T23:18:32.937 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-10T23:18:32.937 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-10T23:18:32.937 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T23:18:32.937 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T23:18:32.937 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T23:18:32.937 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T23:18:32.937 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T23:18:32.937 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T23:18:32.937 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T23:18:32.937 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T23:18:32.937 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T23:18:32.937 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T23:18:32.938 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T23:18:32.938 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T23:18:32.938 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-10T23:18:32.938 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-10T23:18:32.938 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-10T23:18:32.938 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T23:18:32.938 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T23:18:32.938 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-10T23:18:32.938 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-10T23:18:32.938 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-10T23:18:32.938 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-10T23:18:32.938 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-10T23:18:32.938 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-10T23:18:32.938 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T23:18:32.938 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T23:18:32.939 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T23:18:32.939 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T23:18:32.939 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T23:18:32.939 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T23:18:32.939 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T23:18:32.939 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T23:18:32.939 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T23:18:32.939 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T23:18:32.939 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-10T23:18:32.939 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T23:18:32.939 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-10T23:18:32.939 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-10T23:18:32.939 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T23:18:32.939 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-10T23:18:32.939 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-10T23:18:32.940 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T23:18:32.940 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T23:18:32.940 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-10T23:18:32.940 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-10T23:18:32.940 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-10T23:18:32.940 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T23:18:32.940 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-10T23:18:32.940 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T23:18:32.940 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-10T23:18:32.940 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T23:18:32.940 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T23:18:32.940 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-10T23:18:32.940 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T23:18:32.940 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T23:18:32.940 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T23:18:32.941 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T23:18:32.941 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-10T23:18:32.941 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T23:18:32.941 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-10T23:18:32.941 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-10T23:18:32.941 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T23:18:32.941 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-10T23:18:32.941 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T23:18:32.941 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T23:18:32.941 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T23:18:32.941 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-10T23:18:32.941 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T23:18:32.941 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T23:18:32.941 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-10T23:18:32.942 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-10T23:18:32.942 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T23:18:32.942 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-10T23:18:32.942 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-10T23:18:32.942 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T23:18:32.942 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-10T23:18:32.942 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-10T23:18:32.942 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-10T23:18:32.942 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T23:18:32.942 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-10T23:18:32.942 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-10T23:18:32.942 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-10T23:18:32.942 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-10T23:18:32.943 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-10T23:18:32.943 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T23:18:32.943 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-10T23:18:32.943 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-10T23:18:32.943 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T23:18:32.943 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-10T23:18:32.943 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T23:18:32.943 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-10T23:18:32.943 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T23:18:32.943 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-10T23:18:32.943 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T23:18:32.943 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-10T23:18:32.943 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T23:18:32.943 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-10T23:18:32.943 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-10T23:18:32.944 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-10T23:18:32.944 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T23:18:32.944 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-10T23:18:32.944 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T23:18:32.944 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-10T23:18:32.944 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-10T23:18:32.944 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-10T23:18:32.944 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T23:18:32.944 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-10T23:18:32.944 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T23:18:32.944 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-10T23:18:32.944 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-10T23:18:32.945 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-10T23:18:32.945 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T23:18:32.945 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-10T23:18:32.945 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T23:18:32.945 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-10T23:18:32.945 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T23:18:32.945 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-10T23:18:32.945 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T23:18:32.945 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-10T23:18:32.945 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-10T23:18:32.945 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-10T23:18:32.945 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T23:18:32.945 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-10T23:18:32.946 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T23:18:32.946 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-10T23:18:32.946 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T23:18:32.946 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-10T23:18:32.946 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T23:18:32.946 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-10T23:18:32.946 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T23:18:32.946 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-10T23:18:32.946 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-10T23:18:32.946 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T23:18:32.946 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-10T23:18:32.946 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T23:18:32.946 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-10T23:18:32.947 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T23:18:32.947 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T23:18:32.947 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-10T23:18:32.947 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T23:18:32.947 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T23:18:32.947 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-10T23:18:32.947 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T23:18:32.947 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T23:18:32.947 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-10T23:18:32.947 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T23:18:32.947 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-10T23:18:32.947 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T23:18:32.947 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-10T23:18:32.947 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T23:18:32.948 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T23:18:32.948 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T23:18:32.948 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T23:18:32.948 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T23:18:32.948 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T23:18:32.948 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T23:18:32.948 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T23:18:32.948 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T23:18:32.948 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T23:18:32.948 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T23:18:32.948 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-10T23:18:32.948 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T23:18:32.948 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T23:18:32.948 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T23:18:32.949 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T23:18:32.949 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T23:18:32.949 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T23:18:32.949 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T23:18:32.949 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T23:18:32.949 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T23:18:32.949 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T23:18:32.949 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T23:18:32.949 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T23:18:32.949 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T23:18:32.949 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T23:18:32.949 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T23:18:32.949 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T23:18:32.950 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T23:18:32.950 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T23:18:32.950 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T23:18:32.950 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T23:18:32.950 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T23:18:32.950 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T23:18:32.950 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T23:18:32.950 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T23:18:32.950 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T23:18:32.950 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T23:18:32.950 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T23:18:32.950 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T23:18:32.950 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T23:18:32.950 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T23:18:32.950 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T23:18:32.951 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T23:18:32.951 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T23:18:32.951 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T23:18:32.951 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T23:18:32.951 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T23:18:32.951 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T23:18:32.951 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T23:18:32.951 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T23:18:32.951 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-10T23:18:32.951 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-10T23:18:32.951 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-10T23:18:32.951 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-10T23:18:32.951 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T23:18:32.951 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-10T23:18:32.951 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T23:18:32.952 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-10T23:18:32.952 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T23:18:32.952 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T23:18:32.952 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T23:18:32.952 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T23:18:32.952 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T23:18:32.952 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-10T23:18:32.952 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-10T23:18:32.952 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-10T23:18:32.952 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T23:18:32.952 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-10T23:18:32.952 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T23:18:32.952 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-10T23:18:32.953 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-10T23:18:32.953 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-10T23:18:32.953 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-10T23:18:32.953 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-10T23:18:32.953 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-10T23:18:32.953 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-10T23:18:32.953 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-10T23:18:32.953 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-10T23:18:32.953 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T23:18:32.953 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-10T23:18:32.953 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T23:18:32.953 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-10T23:18:32.953 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-10T23:18:32.953 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T23:18:32.954 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-10T23:18:32.954 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T23:18:32.954 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T23:18:32.954 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-10T23:18:32.954 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T23:18:32.954 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-10T23:18:32.954 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T23:18:32.954 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-10T23:18:32.954 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T23:18:32.954 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-10T23:18:32.954 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T23:18:32.954 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-10T23:18:32.954 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T23:18:32.954 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-10T23:18:32.955 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-10T23:18:32.955 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-10T23:18:32.955 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-10T23:18:32.955 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-10T23:18:32.955 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T23:18:32.955 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-10T23:18:32.955 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T23:18:32.955 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-10T23:18:32.955 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T23:18:32.955 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-10T23:18:32.955 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T23:18:32.955 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T23:18:32.955 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-10T23:18:32.955 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T23:18:32.956 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T23:18:32.956 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-10T23:18:32.956 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T23:18:32.956 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T23:18:32.956 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-10T23:18:32.956 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T23:18:32.956 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-10T23:18:32.956 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-10T23:18:32.956 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-10T23:18:32.956 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-10T23:18:32.956 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-10T23:18:32.956 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T23:18:32.956 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-10T23:18:32.956 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-10T23:18:32.957 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T23:18:32.957 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-10T23:18:32.957 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-10T23:18:32.957 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-10T23:18:32.957 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-10T23:18:32.957 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T23:18:32.957 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T23:18:32.957 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T23:18:32.957 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-10T23:18:32.957 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-10T23:18:32.957 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-10T23:18:32.957 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T23:18:32.957 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-10T23:18:32.957 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-10T23:18:32.958 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T23:18:32.958 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T23:18:32.958 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T23:18:32.958 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T23:18:32.958 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T23:18:32.958 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T23:18:32.958 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T23:18:32.958 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T23:18:32.958 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-10T23:18:32.958 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T23:18:32.958 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T23:18:32.958 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T23:18:32.958 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T23:18:32.959 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T23:18:32.959 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-10T23:18:32.959 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T23:18:32.959 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-10T23:18:32.959 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T23:18:32.959 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T23:18:32.959 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T23:18:32.959 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T23:18:32.959 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-10T23:18:32.959 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T23:18:32.959 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-10T23:18:32.959 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T23:18:32.959 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T23:18:32.993 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-10T23:18:32.993 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T23:18:32.993 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T23:18:32.993 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T23:18:32.993 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T23:18:32.994 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T23:18:32.994 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T23:18:32.994 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T23:18:32.994 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T23:18:32.994 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T23:18:32.994 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T23:18:32.994 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T23:18:32.994 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T23:18:32.994 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T23:18:32.994 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T23:18:32.994 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T23:18:32.994 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T23:18:32.994 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T23:18:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T23:18:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T23:18:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T23:18:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T23:18:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T23:18:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T23:18:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-10T23:18:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T23:18:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-10T23:18:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T23:18:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-10T23:18:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T23:18:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T23:18:32.995 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T23:18:32.996 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T23:18:32.996 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-10T23:18:32.996 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T23:18:32.996 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T23:18:32.996 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T23:18:32.996 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T23:18:32.996 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T23:18:32.996 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-10T23:18:32.996 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-10T23:18:32.996 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-10T23:18:32.996 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-10T23:18:32.996 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T23:18:32.996 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T23:18:32.997 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T23:18:32.997 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-10T23:18:32.997 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T23:18:32.997 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T23:18:32.997 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T23:18:32.997 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T23:18:32.997 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T23:18:32.997 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T23:18:32.997 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T23:18:32.997 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T23:18:32.997 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T23:18:32.997 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T23:18:32.997 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-10T23:18:32.997 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T23:18:32.998 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T23:18:32.998 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T23:18:32.998 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-10T23:18:32.998 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T23:18:32.998 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-10T23:18:32.998 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T23:18:32.998 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T23:18:32.998 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T23:18:32.998 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-10T23:18:32.998 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T23:18:32.998 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T23:18:32.998 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T23:18:32.998 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T23:18:32.998 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T23:18:32.999 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T23:18:32.999 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T23:18:32.999 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T23:18:32.999 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T23:18:32.999 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-10T23:18:32.999 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T23:18:32.999 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T23:18:32.999 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T23:18:32.999 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T23:18:32.999 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T23:18:32.999 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T23:18:32.999 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T23:18:32.999 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T23:18:32.999 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T23:18:33.000 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T23:18:33.000 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T23:18:33.000 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T23:18:33.000 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T23:18:33.000 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T23:18:33.000 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T23:18:33.000 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T23:18:33.000 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T23:18:33.000 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T23:18:33.000 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T23:18:33.000 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T23:18:33.000 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T23:18:33.000 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T23:18:33.000 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-10T23:18:33.000 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T23:18:33.001 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T23:18:33.001 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T23:18:33.001 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-10T23:18:33.001 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T23:18:33.001 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T23:18:33.001 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T23:18:33.001 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T23:18:33.001 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-10T23:18:33.001 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-10T23:18:33.001 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T23:18:33.001 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-10T23:18:33.001 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T23:18:33.001 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T23:18:33.001 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T23:18:33.002 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T23:18:33.002 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T23:18:33.002 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T23:18:33.002 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T23:18:33.002 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T23:18:33.002 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T23:18:33.002 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T23:18:33.002 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T23:18:33.002 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T23:18:33.002 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T23:18:33.002 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T23:18:33.002 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T23:18:33.002 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T23:18:33.003 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T23:18:33.003 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T23:18:33.003 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T23:18:33.003 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T23:18:33.003 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T23:18:33.003 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T23:18:33.003 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T23:18:33.003 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T23:18:33.003 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T23:18:33.003 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T23:18:33.003 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-10T23:18:33.003 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T23:18:33.003 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-10T23:18:33.003 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T23:18:33.003 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T23:18:33.004 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T23:18:33.004 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T23:18:33.004 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T23:18:33.004 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T23:18:33.004 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T23:18:33.004 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-10T23:18:33.004 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-10T23:18:33.004 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-10T23:18:33.004 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T23:18:33.004 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-10T23:18:33.004 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-10T23:18:33.004 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T23:18:33.005 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-10T23:18:33.005 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T23:18:33.005 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-10T23:18:33.005 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-10T23:18:33.005 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T23:18:33.005 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T23:18:33.005 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T23:18:33.005 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T23:18:33.005 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T23:18:33.005 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T23:18:33.005 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T23:18:33.005 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T23:18:33.005 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T23:18:33.005 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T23:18:33.006 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T23:18:33.006 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T23:18:33.006 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T23:18:33.006 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T23:18:33.006 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T23:18:33.006 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T23:18:33.006 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-10T23:18:33.006 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T23:18:33.006 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T23:18:33.006 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T23:18:33.006 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T23:18:33.006 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T23:18:33.006 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T23:18:33.006 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T23:18:33.006 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T23:18:33.007 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T23:18:33.007 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T23:18:33.007 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T23:18:33.007 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T23:18:33.007 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T23:18:33.007 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T23:18:33.007 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T23:18:33.007 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T23:18:33.007 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T23:18:33.007 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T23:18:33.007 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T23:18:33.007 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T23:18:33.007 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T23:18:33.007 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T23:18:33.008 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T23:18:33.008 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T23:18:33.008 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T23:18:33.008 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T23:18:33.008 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T23:18:33.008 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T23:18:33.008 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T23:18:33.008 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T23:18:33.008 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T23:18:33.008 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T23:18:33.008 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T23:18:33.008 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T23:18:33.009 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T23:18:33.009 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T23:18:33.009 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T23:18:33.009 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T23:18:33.009 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T23:18:33.009 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T23:18:33.009 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T23:18:33.009 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T23:18:33.009 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-10T23:18:33.009 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T23:18:33.009 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T23:18:33.009 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T23:18:33.009 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T23:18:33.009 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T23:18:33.009 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T23:18:33.010 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T23:18:33.010 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T23:18:33.010 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T23:18:33.010 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T23:18:33.010 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T23:18:33.010 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T23:18:33.010 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T23:18:33.010 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T23:18:33.010 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T23:18:33.010 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-10T23:18:33.010 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T23:18:33.010 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T23:18:33.010 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T23:18:33.010 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T23:18:33.011 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T23:18:33.011 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-10T23:18:33.011 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-10T23:18:33.011 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-10T23:18:33.011 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T23:18:33.011 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T23:18:33.011 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T23:18:33.011 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T23:18:33.011 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T23:18:33.011 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T23:18:33.011 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T23:18:33.011 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T23:18:33.011 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T23:18:33.011 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T23:18:33.012 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T23:18:33.012 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T23:18:33.012 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T23:18:33.012 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T23:18:33.012 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T23:18:33.012 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T23:18:33.012 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T23:18:33.012 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T23:18:33.012 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T23:18:33.012 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T23:18:33.012 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T23:18:33.012 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T23:18:33.012 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T23:18:33.013 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T23:18:33.013 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T23:18:33.013 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T23:18:33.013 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T23:18:33.013 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T23:18:33.013 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T23:18:33.013 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T23:18:33.013 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T23:18:33.013 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T23:18:33.013 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T23:18:33.013 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T23:18:33.013 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T23:18:33.013 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T23:18:33.014 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T23:18:33.014 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T23:18:33.014 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T23:18:33.014 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T23:18:33.014 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T23:18:33.014 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T23:18:33.014 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T23:18:33.014 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T23:18:33.014 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T23:18:33.014 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T23:18:33.014 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T23:18:33.014 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T23:18:33.014 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T23:18:33.014 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T23:18:33.015 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T23:18:33.015 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T23:18:33.015 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T23:18:33.015 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T23:18:33.015 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T23:18:33.015 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T23:18:33.015 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T23:18:33.015 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T23:18:33.015 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T23:18:33.015 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T23:18:33.015 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T23:18:33.015 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T23:18:33.015 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T23:18:33.015 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T23:18:33.015 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T23:18:33.016 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T23:18:33.016 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T23:18:33.016 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T23:18:33.016 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T23:18:33.016 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-10T23:18:33.016 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T23:18:33.016 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T23:18:33.016 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-10T23:18:33.016 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T23:18:33.016 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-10T23:18:33.016 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T23:18:33.016 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-10T23:18:33.016 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T23:18:33.016 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T23:18:33.017 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T23:18:33.017 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T23:18:33.017 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-10T23:18:33.017 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T23:18:33.017 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-10T23:18:33.017 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-10T23:18:33.017 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-10T23:18:33.017 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-10T23:18:33.017 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T23:18:33.017 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T23:18:33.017 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T23:18:33.017 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-10T23:18:33.017 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-10T23:18:33.017 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-10T23:18:33.018 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T23:18:33.018 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T23:18:33.018 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T23:18:33.018 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T23:18:33.018 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T23:18:33.018 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T23:18:33.018 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T23:18:33.018 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T23:18:33.018 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T23:18:33.018 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T23:18:33.018 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T23:18:33.018 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-10T23:18:33.018 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T23:18:33.018 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T23:18:33.019 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T23:18:33.019 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T23:18:33.019 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T23:18:33.019 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T23:18:33.019 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T23:18:33.019 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T23:18:33.019 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T23:18:33.019 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T23:18:33.019 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T23:18:33.019 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T23:18:33.019 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T23:18:33.019 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T23:18:33.019 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-10T23:18:33.019 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T23:18:33.020 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T23:18:33.020 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T23:18:33.020 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T23:18:33.020 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T23:18:33.020 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T23:18:33.020 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T23:18:33.020 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T23:18:33.020 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T23:18:33.020 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T23:18:33.020 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T23:18:33.020 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T23:18:33.020 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T23:18:33.020 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T23:18:33.020 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T23:18:33.021 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T23:18:33.021 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T23:18:33.021 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T23:18:33.021 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-10T23:18:33.021 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T23:18:33.021 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T23:18:33.021 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-10T23:18:33.021 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-10T23:18:33.021 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T23:18:33.021 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T23:18:33.021 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T23:18:33.021 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-10T23:18:33.021 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T23:18:33.021 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-10T23:18:33.022 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T23:18:33.022 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T23:18:33.022 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-10T23:18:33.022 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T23:18:33.022 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-10T23:18:33.022 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T23:18:33.022 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T23:18:33.022 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T23:18:33.022 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T23:18:33.022 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T23:18:33.022 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-10T23:18:33.022 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-10T23:18:33.022 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-10T23:18:33.022 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-10T23:18:33.022 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T23:18:33.023 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-10T23:18:33.023 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T23:18:33.023 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T23:18:33.023 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T23:18:33.023 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T23:18:33.023 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T23:18:33.023 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T23:18:33.023 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T23:18:33.023 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T23:18:33.023 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T23:18:33.023 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T23:18:33.023 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T23:18:33.023 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T23:18:33.023 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T23:18:33.024 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-10T23:18:33.024 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-10T23:18:33.024 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-10T23:18:33.024 INFO:teuthology.orchestra.run.smithi181.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T23:18:33.024 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-10T23:18:33.024 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-10T23:18:33.028 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:33.028 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:33.029 DEBUG:teuthology.orchestra.run.smithi181:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-10T23:18:33.035 INFO:teuthology.orchestra.run.smithi181.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T23:18:33.035 INFO:teuthology.orchestra.run.smithi181.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T23:18:33.035 INFO:teuthology.orchestra.run.smithi181.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T23:18:33.036 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-10T23:18:33.036 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:33.036 DEBUG:teuthology.orchestra.run.smithi181:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-10T23:18:33.078 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi181.front.sepia.ceph.com to complete... 2024-09-10T23:18:33.114 INFO:teuthology.orchestra.run.smithi094.stderr:Sourcing file `/etc/default/grub' 2024-09-10T23:18:33.115 INFO:teuthology.orchestra.run.smithi094.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T23:18:33.130 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-09-10T23:18:33.347 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:33.353 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:33.472 INFO:teuthology.orchestra.run.smithi181.stderr:Sourcing file `/etc/default/grub' 2024-09-10T23:18:33.472 INFO:teuthology.orchestra.run.smithi181.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T23:18:33.492 INFO:teuthology.orchestra.run.smithi181.stderr:Generating grub configuration file ... 2024-09-10T23:18:33.593 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T23:18:33.599 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T23:18:33.641 INFO:teuthology.orchestra.run.smithi094.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T23:18:33.646 INFO:teuthology.orchestra.run.smithi094.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T23:18:33.690 INFO:teuthology.orchestra.run.smithi181.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:33.696 INFO:teuthology.orchestra.run.smithi181.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:18:33.761 INFO:teuthology.orchestra.run.smithi094.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T23:18:33.761 INFO:teuthology.orchestra.run.smithi094.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T23:18:33.761 INFO:teuthology.orchestra.run.smithi094.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T23:18:33.771 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-09-10T23:18:33.902 INFO:teuthology.orchestra.run.smithi181.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T23:18:33.907 INFO:teuthology.orchestra.run.smithi181.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T23:18:33.943 INFO:teuthology.orchestra.run.smithi181.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T23:18:33.947 INFO:teuthology.orchestra.run.smithi181.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T23:18:34.064 INFO:teuthology.orchestra.run.smithi181.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T23:18:34.065 INFO:teuthology.orchestra.run.smithi181.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T23:18:34.065 INFO:teuthology.orchestra.run.smithi181.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T23:18:34.092 INFO:teuthology.orchestra.run.smithi181.stderr:done 2024-09-10T23:19:04.933 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T23:19:04.934 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T23:19:04.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:19:05.188 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T23:19:05.188 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-09-10T23:19:05.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:19:23.453 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-09-10T23:19:23.713 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.181 2024-09-10T23:19:32.461 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T23:19:32.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:19:32.714 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-09-10T23:19:32.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:19:35.553 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-09-10T23:19:35.777 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.181 2024-09-10T23:19:47.557 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T23:19:47.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:19:47.778 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-09-10T23:19:47.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:20:19.227 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-09-10T23:20:19.637 DEBUG:teuthology.orchestra.run.smithi181:> true 2024-09-10T23:20:19.944 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T23:20:19.944 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T23:20:19.945 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-09-10T23:20:19.996 INFO:teuthology.orchestra.run.smithi094.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:20:19.996 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:20:19.996 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T23:20:19.996 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-10T23:20:20.359 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-09-10T23:20:20.360 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T23:20:20.360 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2024-09-10T23:20:20.408 INFO:teuthology.orchestra.run.smithi181.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:20:20.408 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:20:20.408 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T23:20:20.408 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-10T23:20:20.997 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T23:20:20.998 DEBUG:teuthology.orchestra.run.smithi094:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T23:20:21.384 INFO:teuthology.orchestra.run.smithi094.stdout:ttyS1 2024-09-10T23:20:21.409 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T23:20:21.409 DEBUG:teuthology.orchestra.run.smithi181:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T23:20:21.454 DEBUG:teuthology.parallel:result is None 2024-09-10T23:20:21.488 INFO:teuthology.orchestra.run.smithi181.stdout:ttyS1 2024-09-10T23:20:21.534 DEBUG:teuthology.parallel:result is None 2024-09-10T23:20:21.534 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T23:20:21.540 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T23:20:21.540 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T23:20:21.544 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T23:20:21.552 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T23:20:21.559 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T23:20:21.565 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T23:20:21.566 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T23:20:21.595 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T23:20:21.615 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T23:20:21.621 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T23:20:21.621 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T23:20:21.655 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T23:20:21.681 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T23:20:21.694 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T23:20:21.709 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T23:20:21.723 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T23:20:21.725 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T23:20:21.732 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T23:20:21.732 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T23:20:21.736 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T23:20:21.765 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T23:20:21.774 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T23:20:21.774 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T23:20:21.803 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T23:20:21.817 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T23:20:21.864 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T23:20:21.920 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:20:21.920 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T23:20:21.985 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T23:20:21.995 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T23:20:22.045 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:20:22.045 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T23:20:22.118 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-09-10T23:20:22.121 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2024-09-10T23:20:22.271 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T23:20:22.278 INFO:teuthology.task.internal:Starting timer... 2024-09-10T23:20:22.279 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T23:20:22.288 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T23:20:22.296 DEBUG:teuthology.task.selinux:Excluding smithi094: OS 'ubuntu' does not support SELinux 2024-09-10T23:20:22.296 DEBUG:teuthology.task.selinux:Excluding smithi181: OS 'ubuntu' does not support SELinux 2024-09-10T23:20:22.296 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T23:20:22.296 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-10T23:20:22.296 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T23:20:22.297 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T23:20:22.305 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T23:20:22.316 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T23:20:22.317 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi094.front.sepia.ceph.com,smithi181.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T23:28:40.945 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com')] 2024-09-10T23:28:40.947 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T23:28:40.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:28:41.043 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-09-10T23:28:41.151 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T23:28:41.151 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-09-10T23:28:41.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:28:41.241 DEBUG:teuthology.orchestra.run.smithi181:> true 2024-09-10T23:28:41.359 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-09-10T23:28:41.359 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T23:28:41.371 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T23:28:41.371 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T23:28:41.371 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T23:28:41.375 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T23:28:41.375 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T23:28:41.410 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-10T23:28:41.410 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: Command line: ntpd -gq 2024-09-10T23:28:41.410 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: ---------------------------------------------------- 2024-09-10T23:28:41.410 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: ntp-4 is maintained by Network Time Foundation, 2024-09-10T23:28:41.410 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-10T23:28:41.410 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: corporation. Support and training for ntp-4 are 2024-09-10T23:28:41.411 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: available at https://www.nwtime.org/support 2024-09-10T23:28:41.411 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: ---------------------------------------------------- 2024-09-10T23:28:41.411 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: proto: precision = 0.044 usec (-24) 2024-09-10T23:28:41.411 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: basedate set to 2022-02-04 2024-09-10T23:28:41.411 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: gps base set to 2022-02-06 (week 2196) 2024-09-10T23:28:41.412 INFO:teuthology.orchestra.run.smithi094.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T23:28:41.412 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T23:28:41.412 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: restrict ::: KOD does nothing without LIMITED. 2024-09-10T23:28:41.412 INFO:teuthology.orchestra.run.smithi094.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-10T23:28:41.413 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: Listen and drop on 0 v6wildcard [::]:123 2024-09-10T23:28:41.413 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-10T23:28:41.413 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: Listen normally on 2 lo 127.0.0.1:123 2024-09-10T23:28:41.414 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: Listen normally on 3 enp3s0f1 172.21.15.94:123 2024-09-10T23:28:41.414 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: Listen normally on 4 lo [::1]:123 2024-09-10T23:28:41.414 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d763%5]:123 2024-09-10T23:28:41.414 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:41 ntpd[18491]: Listening on routing socket on fd #22 for interface updates 2024-09-10T23:28:41.442 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-10T23:28:41.442 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: Command line: ntpd -gq 2024-09-10T23:28:41.442 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: ---------------------------------------------------- 2024-09-10T23:28:41.442 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: ntp-4 is maintained by Network Time Foundation, 2024-09-10T23:28:41.442 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-10T23:28:41.443 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: corporation. Support and training for ntp-4 are 2024-09-10T23:28:41.443 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: available at https://www.nwtime.org/support 2024-09-10T23:28:41.443 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: ---------------------------------------------------- 2024-09-10T23:28:41.443 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: proto: precision = 0.042 usec (-24) 2024-09-10T23:28:41.443 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: basedate set to 2022-02-04 2024-09-10T23:28:41.444 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: gps base set to 2022-02-06 (week 2196) 2024-09-10T23:28:41.444 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T23:28:41.444 INFO:teuthology.orchestra.run.smithi181.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T23:28:41.444 INFO:teuthology.orchestra.run.smithi181.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-10T23:28:41.444 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: restrict ::: KOD does nothing without LIMITED. 2024-09-10T23:28:41.445 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: Listen and drop on 0 v6wildcard [::]:123 2024-09-10T23:28:41.445 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-10T23:28:41.445 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: Listen normally on 2 lo 127.0.0.1:123 2024-09-10T23:28:41.445 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: Listen normally on 3 enp3s0f1 172.21.15.181:123 2024-09-10T23:28:41.446 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: Listen normally on 4 lo [::1]:123 2024-09-10T23:28:41.446 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7b9d%5]:123 2024-09-10T23:28:41.446 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:41 ntpd[18431]: Listening on routing socket on fd #22 for interface updates 2024-09-10T23:28:42.411 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:42 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:42.411 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:42 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:42.411 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:42 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:42.411 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:42 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:42.412 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:42 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T23:28:42.412 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:42 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T23:28:42.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:42 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:42.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:42 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:42.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:42 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:42.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:42 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:42.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:42 ntpd[18431]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T23:28:42.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:42 ntpd[18431]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T23:28:43.410 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:43 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:43.411 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:43 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:43.442 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:43 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:43.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:43 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:44.411 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:44 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:44.411 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:44 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:44.411 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:44 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:44.411 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:44 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:44.411 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:44 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T23:28:44.411 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:44 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T23:28:44.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:44 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:44.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:44 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:44.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:44 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:44.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:44 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:44.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:44 ntpd[18431]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T23:28:44.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:44 ntpd[18431]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T23:28:45.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:45 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:45.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:45 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:45.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:45 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:45.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:45 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:45.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:45 ntpd[18431]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T23:28:45.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:45 ntpd[18431]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T23:28:46.412 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:46.412 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:46.412 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:46.412 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:46.412 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:46.412 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:46.413 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:46.413 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:46.413 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T23:28:46.413 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T23:28:46.413 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:46.413 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:46.413 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T23:28:46.413 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T23:28:46.413 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:46.413 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:46.413 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T23:28:46.413 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:46 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T23:28:46.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:46 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:46.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:46 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:46.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:46 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:46.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:46 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:46.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:46 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:46.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:46 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:46.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:46 ntpd[18431]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T23:28:46.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:46 ntpd[18431]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T23:28:46.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:46 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:46.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:46 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:46.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:46 ntpd[18431]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T23:28:46.445 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:46 ntpd[18431]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T23:28:47.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:47 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:47.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:47 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:47.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:47 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:47.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:47 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:47.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:47 ntpd[18431]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T23:28:47.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:47 ntpd[18431]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T23:28:48.411 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:48 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:48.411 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:48 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:48.411 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:48 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:48.411 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:48 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:48.412 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:48 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:48.412 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:48 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:48.412 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:48 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:48.412 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:48 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:48.412 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:48 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T23:28:48.412 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:48 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T23:28:48.412 INFO:teuthology.orchestra.run.smithi094.stdout:10 Sep 23:28:48 ntpd[18491]: ntpd: time slew -0.000282 s 2024-09-10T23:28:48.412 INFO:teuthology.orchestra.run.smithi094.stdout:ntpd: time slew -0.000282s 2024-09-10T23:28:48.413 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:48 ntpd[18491]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-10T23:28:48.413 INFO:teuthology.orchestra.run.smithi094.stderr:10 Sep 23:28:48 ntpd[18491]: can't open /var/log/ntpstats/loopstats.20240910: Permission denied 2024-09-10T23:28:48.443 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:48 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:48 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:48 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:48 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:48 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:48 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:48 ntpd[18431]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T23:28:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:48 ntpd[18431]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T23:28:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:48 ntpd[18431]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T23:28:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:48 ntpd[18431]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T23:28:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:48 ntpd[18431]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T23:28:48.444 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:48 ntpd[18431]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T23:28:48.445 INFO:teuthology.orchestra.run.smithi181.stdout:10 Sep 23:28:48 ntpd[18431]: ntpd: time slew -0.000819 s 2024-09-10T23:28:48.445 INFO:teuthology.orchestra.run.smithi181.stdout:ntpd: time slew -0.000819s 2024-09-10T23:28:48.445 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:48 ntpd[18431]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-10T23:28:48.445 INFO:teuthology.orchestra.run.smithi181.stderr:10 Sep 23:28:48 ntpd[18431]: can't open /var/log/ntpstats/loopstats.20240910: Permission denied 2024-09-10T23:28:48.522 INFO:teuthology.orchestra.run.smithi094.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T23:28:48.522 INFO:teuthology.orchestra.run.smithi094.stdout:============================================================================== 2024-09-10T23:28:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T23:28:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T23:28:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T23:28:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T23:28:48.529 INFO:teuthology.orchestra.run.smithi181.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T23:28:48.529 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================== 2024-09-10T23:28:48.529 INFO:teuthology.orchestra.run.smithi181.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T23:28:48.529 INFO:teuthology.orchestra.run.smithi181.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T23:28:48.529 INFO:teuthology.orchestra.run.smithi181.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T23:28:48.529 INFO:teuthology.orchestra.run.smithi181.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T23:28:48.530 INFO:teuthology.run_tasks:Running task install... 2024-09-10T23:28:48.539 DEBUG:teuthology.task.install:project ceph 2024-09-10T23:28:48.539 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-10T23:28:48.539 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-10T23:28:48.540 INFO:teuthology.task.install:Using flavor: default 2024-09-10T23:28:48.545 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T23:28:48.545 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T23:28:48.546 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-key list | grep Ceph 2024-09-10T23:28:48.546 DEBUG:teuthology.orchestra.run.smithi181:> sudo apt-key list | grep Ceph 2024-09-10T23:28:48.673 INFO:teuthology.orchestra.run.smithi094.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-10T23:28:48.678 INFO:teuthology.orchestra.run.smithi181.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-10T23:28:48.723 INFO:teuthology.orchestra.run.smithi094.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-10T23:28:48.724 INFO:teuthology.orchestra.run.smithi094.stdout:uid [ unknown] Ceph.com (release key) 2024-09-10T23:28:48.725 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-10T23:28:48.725 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-10T23:28:48.726 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:28:48.787 INFO:teuthology.orchestra.run.smithi181.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-10T23:28:48.787 INFO:teuthology.orchestra.run.smithi181.stdout:uid [ unknown] Ceph.com (release key) 2024-09-10T23:28:48.788 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-10T23:28:48.788 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-10T23:28:48.788 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:28:48.917 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default/ 2024-09-10T23:28:48.917 INFO:teuthology.task.install.deb:Package version is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T23:28:48.939 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default/ 2024-09-10T23:28:48.939 INFO:teuthology.task.install.deb:Package version is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T23:28:49.046 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:28:49.047 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-10T23:28:49.058 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:28:49.058 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-10T23:28:49.067 DEBUG:teuthology.orchestra.run.smithi094:> sudo apt-get update 2024-09-10T23:28:49.078 DEBUG:teuthology.orchestra.run.smithi181:> sudo apt-get update 2024-09-10T23:28:49.302 INFO:teuthology.orchestra.run.smithi094.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy InRelease 2024-09-10T23:28:49.311 INFO:teuthology.orchestra.run.smithi181.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy InRelease 2024-09-10T23:28:49.323 INFO:teuthology.orchestra.run.smithi094.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-10T23:28:49.333 INFO:teuthology.orchestra.run.smithi181.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-10T23:28:49.346 INFO:teuthology.orchestra.run.smithi094.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-10T23:28:49.354 INFO:teuthology.orchestra.run.smithi181.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-10T23:28:49.367 INFO:teuthology.orchestra.run.smithi094.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,412 B] 2024-09-10T23:28:49.376 INFO:teuthology.orchestra.run.smithi181.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-10T23:28:49.389 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-10T23:28:49.397 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-10T23:28:49.397 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-10T23:28:49.409 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-10T23:28:49.418 INFO:teuthology.orchestra.run.smithi181.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,412 B] 2024-09-10T23:28:49.463 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-10T23:28:49.544 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-10T23:28:49.991 INFO:teuthology.orchestra.run.smithi181.stdout:Fetched 29.3 kB in 1s (42.7 kB/s) 2024-09-10T23:28:51.022 INFO:teuthology.orchestra.run.smithi181.stdout:Reading package lists... 2024-09-10T23:28:51.037 DEBUG:teuthology.orchestra.run.smithi181:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4854-g8f219c47-1jammy cephadm=19.3.0-4854-g8f219c47-1jammy ceph-mds=19.3.0-4854-g8f219c47-1jammy ceph-mgr=19.3.0-4854-g8f219c47-1jammy ceph-common=19.3.0-4854-g8f219c47-1jammy ceph-fuse=19.3.0-4854-g8f219c47-1jammy ceph-test=19.3.0-4854-g8f219c47-1jammy ceph-volume=19.3.0-4854-g8f219c47-1jammy radosgw=19.3.0-4854-g8f219c47-1jammy python3-rados=19.3.0-4854-g8f219c47-1jammy python3-rgw=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy python3-rbd=19.3.0-4854-g8f219c47-1jammy libcephfs2=19.3.0-4854-g8f219c47-1jammy libcephfs-dev=19.3.0-4854-g8f219c47-1jammy librados2=19.3.0-4854-g8f219c47-1jammy librbd1=19.3.0-4854-g8f219c47-1jammy rbd-fuse=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy cephfs-shell=19.3.0-4854-g8f219c47-1jammy cephfs-top=19.3.0-4854-g8f219c47-1jammy cephfs-mirror=19.3.0-4854-g8f219c47-1jammy 2024-09-10T23:28:51.113 INFO:teuthology.orchestra.run.smithi181.stdout:Reading package lists... 2024-09-10T23:28:51.297 INFO:teuthology.orchestra.run.smithi181.stdout:Building dependency tree... 2024-09-10T23:28:51.298 INFO:teuthology.orchestra.run.smithi181.stdout:Reading state information... 2024-09-10T23:28:51.485 INFO:teuthology.orchestra.run.smithi181.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T23:28:51.485 INFO:teuthology.orchestra.run.smithi181.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T23:28:51.485 INFO:teuthology.orchestra.run.smithi181.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T23:28:51.485 INFO:teuthology.orchestra.run.smithi181.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T23:28:51.485 INFO:teuthology.orchestra.run.smithi181.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T23:28:51.485 INFO:teuthology.orchestra.run.smithi181.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T23:28:51.486 INFO:teuthology.orchestra.run.smithi181.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T23:28:51.486 INFO:teuthology.orchestra.run.smithi181.stdout:The following additional packages will be installed: 2024-09-10T23:28:51.486 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-10T23:28:51.487 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-10T23:28:51.487 INFO:teuthology.orchestra.run.smithi181.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T23:28:51.487 INFO:teuthology.orchestra.run.smithi181.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-10T23:28:51.487 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-10T23:28:51.488 INFO:teuthology.orchestra.run.smithi181.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-10T23:28:51.488 INFO:teuthology.orchestra.run.smithi181.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T23:28:51.488 INFO:teuthology.orchestra.run.smithi181.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T23:28:51.488 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-10T23:28:51.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-10T23:28:51.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-10T23:28:51.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-10T23:28:51.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-10T23:28:51.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-10T23:28:51.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-10T23:28:51.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-10T23:28:51.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-10T23:28:51.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-10T23:28:51.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-10T23:28:51.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-10T23:28:51.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-10T23:28:51.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-10T23:28:51.489 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-10T23:28:51.489 INFO:teuthology.orchestra.run.smithi181.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-10T23:28:51.490 INFO:teuthology.orchestra.run.smithi181.stdout:Suggested packages: 2024-09-10T23:28:51.490 INFO:teuthology.orchestra.run.smithi181.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-10T23:28:51.490 INFO:teuthology.orchestra.run.smithi181.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-10T23:28:51.490 INFO:teuthology.orchestra.run.smithi181.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-10T23:28:51.490 INFO:teuthology.orchestra.run.smithi181.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-10T23:28:51.490 INFO:teuthology.orchestra.run.smithi181.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-10T23:28:51.490 INFO:teuthology.orchestra.run.smithi181.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-10T23:28:51.490 INFO:teuthology.orchestra.run.smithi181.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-10T23:28:51.490 INFO:teuthology.orchestra.run.smithi181.stdout:Recommended packages: 2024-09-10T23:28:51.490 INFO:teuthology.orchestra.run.smithi181.stdout: btrfs-tools 2024-09-10T23:28:51.645 INFO:teuthology.orchestra.run.smithi181.stdout:The following NEW packages will be installed: 2024-09-10T23:28:51.646 INFO:teuthology.orchestra.run.smithi181.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-10T23:28:51.646 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-10T23:28:51.646 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-10T23:28:51.646 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-10T23:28:51.646 INFO:teuthology.orchestra.run.smithi181.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T23:28:51.646 INFO:teuthology.orchestra.run.smithi181.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-10T23:28:51.646 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-10T23:28:51.647 INFO:teuthology.orchestra.run.smithi181.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-10T23:28:51.647 INFO:teuthology.orchestra.run.smithi181.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T23:28:51.647 INFO:teuthology.orchestra.run.smithi181.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T23:28:51.648 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-10T23:28:51.648 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-10T23:28:51.648 INFO:teuthology.orchestra.run.smithi181.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-10T23:28:51.648 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-10T23:28:51.648 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-10T23:28:51.648 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-10T23:28:51.648 INFO:teuthology.orchestra.run.smithi181.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-10T23:28:51.648 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-10T23:28:51.648 INFO:teuthology.orchestra.run.smithi181.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-10T23:28:51.648 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-10T23:28:51.648 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-10T23:28:51.648 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-10T23:28:51.648 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-10T23:28:51.648 INFO:teuthology.orchestra.run.smithi181.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-10T23:28:51.649 INFO:teuthology.orchestra.run.smithi181.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-10T23:28:51.649 INFO:teuthology.orchestra.run.smithi181.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-10T23:28:51.649 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-10T23:28:51.649 INFO:teuthology.orchestra.run.smithi181.stdout: socat unzip xmlstarlet zip 2024-09-10T23:28:51.796 INFO:teuthology.orchestra.run.smithi181.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-10T23:28:51.796 INFO:teuthology.orchestra.run.smithi181.stdout:Need to get 251 MB of archives. 2024-09-10T23:28:51.796 INFO:teuthology.orchestra.run.smithi181.stdout:After this operation, 936 MB of additional disk space will be used. 2024-09-10T23:28:51.797 INFO:teuthology.orchestra.run.smithi181.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4854-g8f219c47-1jammy [3,633 kB] 2024-09-10T23:28:51.846 INFO:teuthology.orchestra.run.smithi181.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-10T23:28:51.926 INFO:teuthology.orchestra.run.smithi181.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-10T23:28:51.972 INFO:teuthology.orchestra.run.smithi181.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-10T23:28:51.977 INFO:teuthology.orchestra.run.smithi181.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4854-g8f219c47-1jammy [3,330 kB] 2024-09-10T23:28:52.025 INFO:teuthology.orchestra.run.smithi181.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4854-g8f219c47-1jammy [721 kB] 2024-09-10T23:28:52.035 INFO:teuthology.orchestra.run.smithi181.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4854-g8f219c47-1jammy [356 kB] 2024-09-10T23:28:52.041 INFO:teuthology.orchestra.run.smithi181.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4854-g8f219c47-1jammy [32.8 kB] 2024-09-10T23:28:52.041 INFO:teuthology.orchestra.run.smithi181.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4854-g8f219c47-1jammy [180 kB] 2024-09-10T23:28:52.044 INFO:teuthology.orchestra.run.smithi181.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4854-g8f219c47-1jammy [71.2 kB] 2024-09-10T23:28:52.045 INFO:teuthology.orchestra.run.smithi181.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4854-g8f219c47-1jammy [338 kB] 2024-09-10T23:28:52.051 INFO:teuthology.orchestra.run.smithi181.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4854-g8f219c47-1jammy [7,988 kB] 2024-09-10T23:28:52.169 INFO:teuthology.orchestra.run.smithi181.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-10T23:28:52.185 INFO:teuthology.orchestra.run.smithi181.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4854-g8f219c47-1jammy [112 kB] 2024-09-10T23:28:52.186 INFO:teuthology.orchestra.run.smithi181.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4854-g8f219c47-1jammy [473 kB] 2024-09-10T23:28:52.192 INFO:teuthology.orchestra.run.smithi181.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4854-g8f219c47-1jammy [28.3 MB] 2024-09-10T23:28:52.261 INFO:teuthology.orchestra.run.smithi181.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-10T23:28:52.274 INFO:teuthology.orchestra.run.smithi181.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-10T23:28:52.325 INFO:teuthology.orchestra.run.smithi181.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-10T23:28:52.501 INFO:teuthology.orchestra.run.smithi181.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-10T23:28:52.511 INFO:teuthology.orchestra.run.smithi181.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-10T23:28:52.550 INFO:teuthology.orchestra.run.smithi181.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-10T23:28:52.565 INFO:teuthology.orchestra.run.smithi181.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-10T23:28:52.568 INFO:teuthology.orchestra.run.smithi181.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-10T23:28:52.570 INFO:teuthology.orchestra.run.smithi181.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-10T23:28:52.601 INFO:teuthology.orchestra.run.smithi181.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-10T23:28:52.603 INFO:teuthology.orchestra.run.smithi181.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-10T23:28:52.610 INFO:teuthology.orchestra.run.smithi181.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-10T23:28:52.611 INFO:teuthology.orchestra.run.smithi181.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-10T23:28:52.663 INFO:teuthology.orchestra.run.smithi181.stdout:Get:30 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4854-g8f219c47-1jammy [5,206 kB] 2024-09-10T23:28:52.665 INFO:teuthology.orchestra.run.smithi181.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-10T23:28:52.668 INFO:teuthology.orchestra.run.smithi181.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-10T23:28:52.672 INFO:teuthology.orchestra.run.smithi181.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-10T23:28:52.674 INFO:teuthology.orchestra.run.smithi181.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-10T23:28:52.675 INFO:teuthology.orchestra.run.smithi181.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-10T23:28:52.742 INFO:teuthology.orchestra.run.smithi181.stdout:Get:36 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4854-g8f219c47-1jammy [251 kB] 2024-09-10T23:28:52.745 INFO:teuthology.orchestra.run.smithi181.stdout:Get:37 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4854-g8f219c47-1jammy [124 kB] 2024-09-10T23:28:52.746 INFO:teuthology.orchestra.run.smithi181.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-10T23:28:52.747 INFO:teuthology.orchestra.run.smithi181.stdout:Get:39 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4854-g8f219c47-1jammy [1,562 kB] 2024-09-10T23:28:52.748 INFO:teuthology.orchestra.run.smithi181.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-10T23:28:52.748 INFO:teuthology.orchestra.run.smithi181.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-10T23:28:52.749 INFO:teuthology.orchestra.run.smithi181.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-10T23:28:52.749 INFO:teuthology.orchestra.run.smithi181.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-10T23:28:52.774 INFO:teuthology.orchestra.run.smithi181.stdout:Get:44 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4854-g8f219c47-1jammy [6,308 kB] 2024-09-10T23:28:52.828 INFO:teuthology.orchestra.run.smithi181.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-10T23:28:52.828 INFO:teuthology.orchestra.run.smithi181.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-10T23:28:52.828 INFO:teuthology.orchestra.run.smithi181.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-10T23:28:52.831 INFO:teuthology.orchestra.run.smithi181.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-10T23:28:52.832 INFO:teuthology.orchestra.run.smithi181.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-10T23:28:52.878 INFO:teuthology.orchestra.run.smithi181.stdout:Get:50 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4854-g8f219c47-1jammy [23.1 MB] 2024-09-10T23:28:52.909 INFO:teuthology.orchestra.run.smithi181.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-10T23:28:52.910 INFO:teuthology.orchestra.run.smithi181.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-10T23:28:52.911 INFO:teuthology.orchestra.run.smithi181.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-10T23:28:52.911 INFO:teuthology.orchestra.run.smithi181.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-10T23:28:52.912 INFO:teuthology.orchestra.run.smithi181.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-10T23:28:52.992 INFO:teuthology.orchestra.run.smithi181.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-10T23:28:52.993 INFO:teuthology.orchestra.run.smithi181.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-10T23:28:52.994 INFO:teuthology.orchestra.run.smithi181.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-10T23:28:53.000 INFO:teuthology.orchestra.run.smithi181.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-10T23:28:53.000 INFO:teuthology.orchestra.run.smithi181.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-10T23:28:53.075 INFO:teuthology.orchestra.run.smithi181.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-10T23:28:53.075 INFO:teuthology.orchestra.run.smithi181.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-10T23:28:53.076 INFO:teuthology.orchestra.run.smithi181.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-10T23:28:53.079 INFO:teuthology.orchestra.run.smithi181.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-10T23:28:53.080 INFO:teuthology.orchestra.run.smithi181.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-10T23:28:53.156 INFO:teuthology.orchestra.run.smithi181.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-10T23:28:53.157 INFO:teuthology.orchestra.run.smithi181.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-10T23:28:53.160 INFO:teuthology.orchestra.run.smithi181.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-10T23:28:53.160 INFO:teuthology.orchestra.run.smithi181.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-10T23:28:53.161 INFO:teuthology.orchestra.run.smithi181.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-10T23:28:53.238 INFO:teuthology.orchestra.run.smithi181.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-10T23:28:53.262 INFO:teuthology.orchestra.run.smithi181.stdout:Get:72 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4854-g8f219c47-1jammy [14.0 kB] 2024-09-10T23:28:53.262 INFO:teuthology.orchestra.run.smithi181.stdout:Get:73 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4854-g8f219c47-1jammy [880 kB] 2024-09-10T23:28:53.270 INFO:teuthology.orchestra.run.smithi181.stdout:Get:74 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4854-g8f219c47-1jammy [2,454 kB] 2024-09-10T23:28:53.311 INFO:teuthology.orchestra.run.smithi181.stdout:Get:75 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4854-g8f219c47-1jammy [813 kB] 2024-09-10T23:28:53.323 INFO:teuthology.orchestra.run.smithi181.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4854-g8f219c47-1jammy [160 kB] 2024-09-10T23:28:53.326 INFO:teuthology.orchestra.run.smithi181.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4854-g8f219c47-1jammy [61.4 MB] 2024-09-10T23:28:53.358 INFO:teuthology.orchestra.run.smithi181.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-10T23:28:53.359 INFO:teuthology.orchestra.run.smithi181.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-10T23:28:53.360 INFO:teuthology.orchestra.run.smithi181.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-10T23:28:53.361 INFO:teuthology.orchestra.run.smithi181.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-10T23:28:53.362 INFO:teuthology.orchestra.run.smithi181.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-10T23:28:53.432 INFO:teuthology.orchestra.run.smithi181.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-10T23:28:53.434 INFO:teuthology.orchestra.run.smithi181.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-10T23:28:53.435 INFO:teuthology.orchestra.run.smithi181.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-10T23:28:53.499 INFO:teuthology.orchestra.run.smithi181.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-10T23:28:53.500 INFO:teuthology.orchestra.run.smithi181.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-10T23:28:53.524 INFO:teuthology.orchestra.run.smithi181.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-10T23:28:53.524 INFO:teuthology.orchestra.run.smithi181.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-10T23:28:53.525 INFO:teuthology.orchestra.run.smithi181.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-10T23:28:53.526 INFO:teuthology.orchestra.run.smithi181.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-10T23:28:53.526 INFO:teuthology.orchestra.run.smithi181.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-10T23:28:53.528 INFO:teuthology.orchestra.run.smithi181.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-10T23:28:53.531 INFO:teuthology.orchestra.run.smithi181.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-10T23:28:53.605 INFO:teuthology.orchestra.run.smithi181.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-10T23:28:53.606 INFO:teuthology.orchestra.run.smithi181.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-10T23:28:53.607 INFO:teuthology.orchestra.run.smithi181.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-10T23:28:53.687 INFO:teuthology.orchestra.run.smithi181.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-10T23:28:53.689 INFO:teuthology.orchestra.run.smithi181.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-10T23:28:53.690 INFO:teuthology.orchestra.run.smithi181.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-10T23:28:53.691 INFO:teuthology.orchestra.run.smithi181.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-10T23:28:53.692 INFO:teuthology.orchestra.run.smithi181.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-10T23:28:53.696 INFO:teuthology.orchestra.run.smithi181.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-10T23:28:53.697 INFO:teuthology.orchestra.run.smithi181.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-10T23:28:53.768 INFO:teuthology.orchestra.run.smithi181.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-10T23:28:53.768 INFO:teuthology.orchestra.run.smithi181.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-10T23:28:53.769 INFO:teuthology.orchestra.run.smithi181.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-10T23:28:53.849 INFO:teuthology.orchestra.run.smithi181.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-10T23:28:53.851 INFO:teuthology.orchestra.run.smithi181.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-10T23:28:53.852 INFO:teuthology.orchestra.run.smithi181.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-10T23:28:53.859 INFO:teuthology.orchestra.run.smithi181.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-10T23:28:53.860 INFO:teuthology.orchestra.run.smithi181.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-10T23:28:53.860 INFO:teuthology.orchestra.run.smithi181.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-10T23:28:53.862 INFO:teuthology.orchestra.run.smithi181.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-10T23:28:53.931 INFO:teuthology.orchestra.run.smithi181.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-10T23:28:54.353 INFO:teuthology.orchestra.run.smithi181.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4854-g8f219c47-1jammy [8,625 kB] 2024-09-10T23:28:54.479 INFO:teuthology.orchestra.run.smithi181.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4854-g8f219c47-1jammy [14.2 kB] 2024-09-10T23:28:54.479 INFO:teuthology.orchestra.run.smithi181.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4854-g8f219c47-1jammy [55.8 MB] 2024-09-10T23:28:54.570 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:7 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-10T23:28:54.591 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-10T23:28:54.612 INFO:teuthology.orchestra.run.smithi094.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-10T23:28:55.053 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 29.3 kB in 6s (5,085 B/s) 2024-09-10T23:28:55.407 INFO:teuthology.orchestra.run.smithi181.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4854-g8f219c47-1jammy [130 kB] 2024-09-10T23:28:55.408 INFO:teuthology.orchestra.run.smithi181.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4854-g8f219c47-1jammy [233 kB] 2024-09-10T23:28:55.410 INFO:teuthology.orchestra.run.smithi181.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4854-g8f219c47-1jammy [33.9 kB] 2024-09-10T23:28:55.410 INFO:teuthology.orchestra.run.smithi181.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4854-g8f219c47-1jammy [23.4 kB] 2024-09-10T23:28:55.411 INFO:teuthology.orchestra.run.smithi181.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4854-g8f219c47-1jammy [39.0 kB] 2024-09-10T23:28:55.411 INFO:teuthology.orchestra.run.smithi181.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4854-g8f219c47-1jammy [15.9 MB] 2024-09-10T23:28:55.663 INFO:teuthology.orchestra.run.smithi181.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4854-g8f219c47-1jammy [91.2 kB] 2024-09-10T23:28:56.087 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-09-10T23:28:56.102 DEBUG:teuthology.orchestra.run.smithi094:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4854-g8f219c47-1jammy cephadm=19.3.0-4854-g8f219c47-1jammy ceph-mds=19.3.0-4854-g8f219c47-1jammy ceph-mgr=19.3.0-4854-g8f219c47-1jammy ceph-common=19.3.0-4854-g8f219c47-1jammy ceph-fuse=19.3.0-4854-g8f219c47-1jammy ceph-test=19.3.0-4854-g8f219c47-1jammy ceph-volume=19.3.0-4854-g8f219c47-1jammy radosgw=19.3.0-4854-g8f219c47-1jammy python3-rados=19.3.0-4854-g8f219c47-1jammy python3-rgw=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy python3-rbd=19.3.0-4854-g8f219c47-1jammy libcephfs2=19.3.0-4854-g8f219c47-1jammy libcephfs-dev=19.3.0-4854-g8f219c47-1jammy librados2=19.3.0-4854-g8f219c47-1jammy librbd1=19.3.0-4854-g8f219c47-1jammy rbd-fuse=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy cephfs-shell=19.3.0-4854-g8f219c47-1jammy cephfs-top=19.3.0-4854-g8f219c47-1jammy cephfs-mirror=19.3.0-4854-g8f219c47-1jammy 2024-09-10T23:28:56.178 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-09-10T23:28:56.315 INFO:teuthology.orchestra.run.smithi181.stdout:Fetched 251 MB in 4s (62.8 MB/s) 2024-09-10T23:28:56.360 INFO:teuthology.orchestra.run.smithi094.stdout:Building dependency tree... 2024-09-10T23:28:56.360 INFO:teuthology.orchestra.run.smithi094.stdout:Reading state information... 2024-09-10T23:28:56.464 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-10T23:28:56.529 INFO:teuthology.orchestra.run.smithi181.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-10T23:28:56.532 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-10T23:28:56.549 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T23:28:56.549 INFO:teuthology.orchestra.run.smithi094.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T23:28:56.550 INFO:teuthology.orchestra.run.smithi094.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T23:28:56.550 INFO:teuthology.orchestra.run.smithi094.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T23:28:56.550 INFO:teuthology.orchestra.run.smithi094.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T23:28:56.550 INFO:teuthology.orchestra.run.smithi094.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T23:28:56.550 INFO:teuthology.orchestra.run.smithi094.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T23:28:56.551 INFO:teuthology.orchestra.run.smithi094.stdout:The following additional packages will be installed: 2024-09-10T23:28:56.551 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-10T23:28:56.551 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-10T23:28:56.551 INFO:teuthology.orchestra.run.smithi094.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T23:28:56.551 INFO:teuthology.orchestra.run.smithi094.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-10T23:28:56.552 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-10T23:28:56.553 INFO:teuthology.orchestra.run.smithi094.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-10T23:28:56.553 INFO:teuthology.orchestra.run.smithi094.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T23:28:56.553 INFO:teuthology.orchestra.run.smithi094.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T23:28:56.553 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-10T23:28:56.553 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-10T23:28:56.553 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-10T23:28:56.553 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-10T23:28:56.554 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-10T23:28:56.554 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-10T23:28:56.554 INFO:teuthology.orchestra.run.smithi094.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-10T23:28:56.554 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-10T23:28:56.554 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-10T23:28:56.554 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-10T23:28:56.554 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-10T23:28:56.554 INFO:teuthology.orchestra.run.smithi094.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-10T23:28:56.556 INFO:teuthology.orchestra.run.smithi094.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-10T23:28:56.556 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-10T23:28:56.556 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-10T23:28:56.556 INFO:teuthology.orchestra.run.smithi094.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-10T23:28:56.556 INFO:teuthology.orchestra.run.smithi094.stdout:Suggested packages: 2024-09-10T23:28:56.556 INFO:teuthology.orchestra.run.smithi094.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-10T23:28:56.556 INFO:teuthology.orchestra.run.smithi094.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-10T23:28:56.557 INFO:teuthology.orchestra.run.smithi094.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-10T23:28:56.557 INFO:teuthology.orchestra.run.smithi094.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-10T23:28:56.557 INFO:teuthology.orchestra.run.smithi094.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-10T23:28:56.557 INFO:teuthology.orchestra.run.smithi094.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-10T23:28:56.557 INFO:teuthology.orchestra.run.smithi094.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-10T23:28:56.557 INFO:teuthology.orchestra.run.smithi094.stdout:Recommended packages: 2024-09-10T23:28:56.557 INFO:teuthology.orchestra.run.smithi094.stdout: btrfs-tools 2024-09-10T23:28:56.588 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-10T23:28:56.710 INFO:teuthology.orchestra.run.smithi094.stdout:The following NEW packages will be installed: 2024-09-10T23:28:56.710 INFO:teuthology.orchestra.run.smithi094.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-10T23:28:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-10T23:28:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-10T23:28:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-10T23:28:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T23:28:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-10T23:28:56.711 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-10T23:28:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-10T23:28:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T23:28:56.712 INFO:teuthology.orchestra.run.smithi094.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T23:28:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-10T23:28:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-10T23:28:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-10T23:28:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-10T23:28:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-10T23:28:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-10T23:28:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-10T23:28:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-10T23:28:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-10T23:28:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-10T23:28:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-10T23:28:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-10T23:28:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-10T23:28:56.713 INFO:teuthology.orchestra.run.smithi094.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-10T23:28:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-10T23:28:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-10T23:28:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-10T23:28:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: socat unzip xmlstarlet zip 2024-09-10T23:28:56.863 INFO:teuthology.orchestra.run.smithi094.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-10T23:28:56.863 INFO:teuthology.orchestra.run.smithi094.stdout:Need to get 251 MB of archives. 2024-09-10T23:28:56.864 INFO:teuthology.orchestra.run.smithi094.stdout:After this operation, 936 MB of additional disk space will be used. 2024-09-10T23:28:56.864 INFO:teuthology.orchestra.run.smithi094.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4854-g8f219c47-1jammy [3,633 kB] 2024-09-10T23:28:56.890 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-10T23:28:56.904 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-10T23:28:56.949 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T23:28:57.056 INFO:teuthology.orchestra.run.smithi094.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4854-g8f219c47-1jammy [3,330 kB] 2024-09-10T23:28:57.103 INFO:teuthology.orchestra.run.smithi094.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4854-g8f219c47-1jammy [721 kB] 2024-09-10T23:28:57.113 INFO:teuthology.orchestra.run.smithi094.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4854-g8f219c47-1jammy [356 kB] 2024-09-10T23:28:57.119 INFO:teuthology.orchestra.run.smithi094.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4854-g8f219c47-1jammy [32.8 kB] 2024-09-10T23:28:57.120 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4854-g8f219c47-1jammy [180 kB] 2024-09-10T23:28:57.123 INFO:teuthology.orchestra.run.smithi094.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4854-g8f219c47-1jammy [71.2 kB] 2024-09-10T23:28:57.124 INFO:teuthology.orchestra.run.smithi094.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4854-g8f219c47-1jammy [338 kB] 2024-09-10T23:28:57.129 INFO:teuthology.orchestra.run.smithi094.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4854-g8f219c47-1jammy [7,988 kB] 2024-09-10T23:28:57.243 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-cmd2. 2024-09-10T23:28:57.257 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-10T23:28:57.263 INFO:teuthology.orchestra.run.smithi094.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4854-g8f219c47-1jammy [112 kB] 2024-09-10T23:28:57.265 INFO:teuthology.orchestra.run.smithi094.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4854-g8f219c47-1jammy [473 kB] 2024-09-10T23:28:57.270 INFO:teuthology.orchestra.run.smithi094.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4854-g8f219c47-1jammy [28.3 MB] 2024-09-10T23:28:57.293 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-10T23:28:57.621 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-10T23:28:57.635 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-10T23:28:57.721 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T23:28:57.742 INFO:teuthology.orchestra.run.smithi094.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4854-g8f219c47-1jammy [5,206 kB] 2024-09-10T23:28:57.821 INFO:teuthology.orchestra.run.smithi094.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4854-g8f219c47-1jammy [251 kB] 2024-09-10T23:28:57.823 INFO:teuthology.orchestra.run.smithi094.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4854-g8f219c47-1jammy [124 kB] 2024-09-10T23:28:57.824 INFO:teuthology.orchestra.run.smithi094.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4854-g8f219c47-1jammy [1,562 kB] 2024-09-10T23:28:57.851 INFO:teuthology.orchestra.run.smithi094.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4854-g8f219c47-1jammy [6,308 kB] 2024-09-10T23:28:57.956 INFO:teuthology.orchestra.run.smithi094.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4854-g8f219c47-1jammy [23.1 MB] 2024-09-10T23:28:58.091 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-10T23:28:58.105 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-10T23:28:58.150 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T23:28:58.340 INFO:teuthology.orchestra.run.smithi094.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4854-g8f219c47-1jammy [14.0 kB] 2024-09-10T23:28:58.340 INFO:teuthology.orchestra.run.smithi094.stdout:Get:20 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4854-g8f219c47-1jammy [880 kB] 2024-09-10T23:28:58.348 INFO:teuthology.orchestra.run.smithi094.stdout:Get:21 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4854-g8f219c47-1jammy [2,454 kB] 2024-09-10T23:28:58.389 INFO:teuthology.orchestra.run.smithi094.stdout:Get:22 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4854-g8f219c47-1jammy [813 kB] 2024-09-10T23:28:58.402 INFO:teuthology.orchestra.run.smithi094.stdout:Get:23 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4854-g8f219c47-1jammy [160 kB] 2024-09-10T23:28:58.404 INFO:teuthology.orchestra.run.smithi094.stdout:Get:24 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4854-g8f219c47-1jammy [61.4 MB] 2024-09-10T23:28:58.519 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-10T23:28:58.534 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-10T23:28:58.578 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T23:28:59.098 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-10T23:28:59.113 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T23:28:59.165 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T23:28:59.432 INFO:teuthology.orchestra.run.smithi094.stdout:Get:25 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4854-g8f219c47-1jammy [8,625 kB] 2024-09-10T23:28:59.556 INFO:teuthology.orchestra.run.smithi094.stdout:Get:26 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4854-g8f219c47-1jammy [14.2 kB] 2024-09-10T23:28:59.556 INFO:teuthology.orchestra.run.smithi094.stdout:Get:27 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4854-g8f219c47-1jammy [55.8 MB] 2024-09-10T23:28:59.609 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-10T23:28:59.624 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T23:28:59.668 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T23:29:00.004 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-10T23:29:00.020 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T23:29:00.062 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T23:29:00.227 INFO:teuthology.orchestra.run.smithi094.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-10T23:29:00.245 INFO:teuthology.orchestra.run.smithi094.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-10T23:29:00.254 INFO:teuthology.orchestra.run.smithi094.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-10T23:29:00.296 INFO:teuthology.orchestra.run.smithi094.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-10T23:29:00.318 INFO:teuthology.orchestra.run.smithi094.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-10T23:29:00.321 INFO:teuthology.orchestra.run.smithi094.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-10T23:29:00.332 INFO:teuthology.orchestra.run.smithi094.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-10T23:29:00.371 INFO:teuthology.orchestra.run.smithi094.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-10T23:29:00.373 INFO:teuthology.orchestra.run.smithi094.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-10T23:29:00.381 INFO:teuthology.orchestra.run.smithi094.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-10T23:29:00.383 INFO:teuthology.orchestra.run.smithi094.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-10T23:29:00.384 INFO:teuthology.orchestra.run.smithi094.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-10T23:29:00.385 INFO:teuthology.orchestra.run.smithi094.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-10T23:29:00.391 INFO:teuthology.orchestra.run.smithi094.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-10T23:29:00.393 INFO:teuthology.orchestra.run.smithi094.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-10T23:29:00.394 INFO:teuthology.orchestra.run.smithi094.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-10T23:29:00.396 INFO:teuthology.orchestra.run.smithi094.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-10T23:29:00.408 INFO:teuthology.orchestra.run.smithi094.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-10T23:29:00.410 INFO:teuthology.orchestra.run.smithi094.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-10T23:29:00.413 INFO:teuthology.orchestra.run.smithi094.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-10T23:29:00.414 INFO:teuthology.orchestra.run.smithi094.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-10T23:29:00.414 INFO:teuthology.orchestra.run.smithi094.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-10T23:29:00.426 INFO:teuthology.orchestra.run.smithi094.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-10T23:29:00.427 INFO:teuthology.orchestra.run.smithi094.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-10T23:29:00.428 INFO:teuthology.orchestra.run.smithi094.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-10T23:29:00.429 INFO:teuthology.orchestra.run.smithi094.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-10T23:29:00.429 INFO:teuthology.orchestra.run.smithi094.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-10T23:29:00.443 INFO:teuthology.orchestra.run.smithi094.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-10T23:29:00.444 INFO:teuthology.orchestra.run.smithi094.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-10T23:29:00.444 INFO:teuthology.orchestra.run.smithi094.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-10T23:29:00.446 INFO:teuthology.orchestra.run.smithi094.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-10T23:29:00.447 INFO:teuthology.orchestra.run.smithi094.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-10T23:29:00.449 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-10T23:29:00.460 INFO:teuthology.orchestra.run.smithi094.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-10T23:29:00.461 INFO:teuthology.orchestra.run.smithi094.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-10T23:29:00.461 INFO:teuthology.orchestra.run.smithi094.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-10T23:29:00.462 INFO:teuthology.orchestra.run.smithi094.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-10T23:29:00.462 INFO:teuthology.orchestra.run.smithi094.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-10T23:29:00.463 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-10T23:29:00.478 INFO:teuthology.orchestra.run.smithi094.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-10T23:29:00.478 INFO:teuthology.orchestra.run.smithi094.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-10T23:29:00.479 INFO:teuthology.orchestra.run.smithi094.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-10T23:29:00.484 INFO:teuthology.orchestra.run.smithi094.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-10T23:29:00.484 INFO:teuthology.orchestra.run.smithi094.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-10T23:29:00.495 INFO:teuthology.orchestra.run.smithi094.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-10T23:29:00.495 INFO:teuthology.orchestra.run.smithi094.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-10T23:29:00.497 INFO:teuthology.orchestra.run.smithi094.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-10T23:29:00.498 INFO:teuthology.orchestra.run.smithi094.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-10T23:29:00.499 INFO:teuthology.orchestra.run.smithi094.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-10T23:29:00.507 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T23:29:00.507 INFO:teuthology.orchestra.run.smithi094.stdout:Get:75 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4854-g8f219c47-1jammy [130 kB] 2024-09-10T23:29:00.508 INFO:teuthology.orchestra.run.smithi094.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4854-g8f219c47-1jammy [233 kB] 2024-09-10T23:29:00.511 INFO:teuthology.orchestra.run.smithi094.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4854-g8f219c47-1jammy [33.9 kB] 2024-09-10T23:29:00.512 INFO:teuthology.orchestra.run.smithi094.stdout:Get:78 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4854-g8f219c47-1jammy [23.4 kB] 2024-09-10T23:29:00.512 INFO:teuthology.orchestra.run.smithi094.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4854-g8f219c47-1jammy [39.0 kB] 2024-09-10T23:29:00.512 INFO:teuthology.orchestra.run.smithi094.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-10T23:29:00.512 INFO:teuthology.orchestra.run.smithi094.stdout:Get:81 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4854-g8f219c47-1jammy [15.9 MB] 2024-09-10T23:29:00.513 INFO:teuthology.orchestra.run.smithi094.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-10T23:29:00.515 INFO:teuthology.orchestra.run.smithi094.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-10T23:29:00.515 INFO:teuthology.orchestra.run.smithi094.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-10T23:29:00.516 INFO:teuthology.orchestra.run.smithi094.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-10T23:29:00.529 INFO:teuthology.orchestra.run.smithi094.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-10T23:29:00.575 INFO:teuthology.orchestra.run.smithi094.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-10T23:29:00.575 INFO:teuthology.orchestra.run.smithi094.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-10T23:29:00.577 INFO:teuthology.orchestra.run.smithi094.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-10T23:29:00.577 INFO:teuthology.orchestra.run.smithi094.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-10T23:29:00.578 INFO:teuthology.orchestra.run.smithi094.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-10T23:29:00.597 INFO:teuthology.orchestra.run.smithi094.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-10T23:29:00.599 INFO:teuthology.orchestra.run.smithi094.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-10T23:29:00.600 INFO:teuthology.orchestra.run.smithi094.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-10T23:29:00.617 INFO:teuthology.orchestra.run.smithi094.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-10T23:29:00.617 INFO:teuthology.orchestra.run.smithi094.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-10T23:29:00.618 INFO:teuthology.orchestra.run.smithi094.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-10T23:29:00.618 INFO:teuthology.orchestra.run.smithi094.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-10T23:29:00.619 INFO:teuthology.orchestra.run.smithi094.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-10T23:29:00.619 INFO:teuthology.orchestra.run.smithi094.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-10T23:29:00.628 INFO:teuthology.orchestra.run.smithi094.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-10T23:29:00.628 INFO:teuthology.orchestra.run.smithi094.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-10T23:29:00.632 INFO:teuthology.orchestra.run.smithi094.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-10T23:29:00.645 INFO:teuthology.orchestra.run.smithi094.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-10T23:29:00.646 INFO:teuthology.orchestra.run.smithi094.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-10T23:29:00.647 INFO:teuthology.orchestra.run.smithi094.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-10T23:29:00.650 INFO:teuthology.orchestra.run.smithi094.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-10T23:29:00.653 INFO:teuthology.orchestra.run.smithi094.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-10T23:29:00.654 INFO:teuthology.orchestra.run.smithi094.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-10T23:29:00.655 INFO:teuthology.orchestra.run.smithi094.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-10T23:29:00.662 INFO:teuthology.orchestra.run.smithi094.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-10T23:29:00.667 INFO:teuthology.orchestra.run.smithi094.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-10T23:29:00.667 INFO:teuthology.orchestra.run.smithi094.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-10T23:29:00.679 INFO:teuthology.orchestra.run.smithi094.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-10T23:29:00.679 INFO:teuthology.orchestra.run.smithi094.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-10T23:29:00.681 INFO:teuthology.orchestra.run.smithi094.stdout:Get:116 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-10T23:29:00.681 INFO:teuthology.orchestra.run.smithi094.stdout:Get:117 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-10T23:29:00.683 INFO:teuthology.orchestra.run.smithi094.stdout:Get:118 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-10T23:29:00.683 INFO:teuthology.orchestra.run.smithi094.stdout:Get:119 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-10T23:29:00.690 INFO:teuthology.orchestra.run.smithi094.stdout:Get:120 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-10T23:29:00.697 INFO:teuthology.orchestra.run.smithi094.stdout:Get:121 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-10T23:29:00.697 INFO:teuthology.orchestra.run.smithi094.stdout:Get:122 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-10T23:29:00.699 INFO:teuthology.orchestra.run.smithi094.stdout:Get:123 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-10T23:29:00.714 INFO:teuthology.orchestra.run.smithi094.stdout:Get:124 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-10T23:29:00.762 INFO:teuthology.orchestra.run.smithi094.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4854-g8f219c47-1jammy [91.2 kB] 2024-09-10T23:29:00.927 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package librados2. 2024-09-10T23:29:00.942 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../010-librados2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:00.993 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:01.415 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 251 MB in 4s (62.3 MB/s) 2024-09-10T23:29:01.542 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-10T23:29:01.546 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libnbd0. 2024-09-10T23:29:01.561 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-10T23:29:01.607 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-10T23:29:01.610 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-10T23:29:01.612 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-10T23:29:01.649 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-10T23:29:01.985 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-10T23:29:01.999 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package librbd1. 2024-09-10T23:29:01.999 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-10T23:29:02.014 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../012-librbd1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:02.044 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T23:29:02.057 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:02.363 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cmd2. 2024-09-10T23:29:02.377 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-10T23:29:02.421 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-10T23:29:02.544 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libcephfs2. 2024-09-10T23:29:02.559 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:02.611 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:02.766 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-10T23:29:02.781 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-10T23:29:02.867 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T23:29:02.954 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-rados. 2024-09-10T23:29:02.969 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:03.021 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:03.195 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-10T23:29:03.210 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-10T23:29:03.245 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T23:29:03.432 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-10T23:29:03.447 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:03.499 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:03.565 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-10T23:29:03.580 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-10T23:29:03.623 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T23:29:03.793 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-cephfs. 2024-09-10T23:29:03.808 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:03.852 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:04.069 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-10T23:29:04.084 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T23:29:04.310 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T23:29:04.471 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-10T23:29:04.486 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:04.537 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:04.689 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-10T23:29:04.704 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T23:29:04.739 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T23:29:04.949 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-prettytable. 2024-09-10T23:29:04.964 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-10T23:29:05.016 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-10T23:29:05.050 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-10T23:29:05.065 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T23:29:05.109 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T23:29:05.351 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-rbd. 2024-09-10T23:29:05.366 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:05.401 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:05.454 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-10T23:29:05.468 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-10T23:29:05.512 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T23:29:05.746 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-10T23:29:05.761 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-10T23:29:05.804 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T23:29:05.890 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librados2. 2024-09-10T23:29:05.905 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../010-librados2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:05.940 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:06.098 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-10T23:29:06.114 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-10T23:29:06.157 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T23:29:06.359 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libnbd0. 2024-09-10T23:29:06.374 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-10T23:29:06.417 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-10T23:29:06.468 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-10T23:29:06.483 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-10T23:29:06.535 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T23:29:06.729 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librbd1. 2024-09-10T23:29:06.744 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../012-librbd1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:06.788 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:06.838 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package lua5.1. 2024-09-10T23:29:06.853 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-10T23:29:06.888 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-10T23:29:07.233 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package lua-any. 2024-09-10T23:29:07.240 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libcephfs2. 2024-09-10T23:29:07.248 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-10T23:29:07.255 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:07.291 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-10T23:29:07.299 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:07.551 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rados. 2024-09-10T23:29:07.566 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:07.569 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package zip. 2024-09-10T23:29:07.584 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-10T23:29:07.610 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:07.627 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking zip (3.0-12build2) ... 2024-09-10T23:29:07.904 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-10T23:29:07.919 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:07.963 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:07.963 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package unzip. 2024-09-10T23:29:07.978 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-10T23:29:08.013 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-10T23:29:08.232 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cephfs. 2024-09-10T23:29:08.247 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:08.290 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:08.425 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package luarocks. 2024-09-10T23:29:08.441 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-10T23:29:08.484 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-10T23:29:08.560 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-10T23:29:08.575 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:08.610 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:08.891 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package librgw2. 2024-09-10T23:29:08.905 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-prettytable. 2024-09-10T23:29:08.906 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../028-librgw2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:08.920 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-10T23:29:08.950 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:08.954 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-10T23:29:09.207 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rbd. 2024-09-10T23:29:09.222 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:09.266 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:09.644 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-10T23:29:09.659 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-10T23:29:09.678 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-rgw. 2024-09-10T23:29:09.693 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:09.702 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T23:29:09.736 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:09.963 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-10T23:29:09.978 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-10T23:29:10.013 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T23:29:10.047 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-10T23:29:10.062 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-10T23:29:10.099 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T23:29:10.275 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-10T23:29:10.290 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-10T23:29:10.333 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T23:29:10.477 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libradosstriper1. 2024-09-10T23:29:10.492 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:10.535 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:10.678 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package lua5.1. 2024-09-10T23:29:10.693 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-10T23:29:10.736 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-10T23:29:10.938 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-common. 2024-09-10T23:29:10.954 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:10.988 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:11.065 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package lua-any. 2024-09-10T23:29:11.079 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-10T23:29:11.123 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-10T23:29:11.334 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package zip. 2024-09-10T23:29:11.349 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-10T23:29:11.392 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking zip (3.0-12build2) ... 2024-09-10T23:29:11.704 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package unzip. 2024-09-10T23:29:11.719 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-10T23:29:11.762 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-10T23:29:12.142 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package luarocks. 2024-09-10T23:29:12.156 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-10T23:29:12.191 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-10T23:29:12.293 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-base. 2024-09-10T23:29:12.362 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:12.362 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:12.533 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package librgw2. 2024-09-10T23:29:12.548 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../028-librgw2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:12.591 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:12.939 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-10T23:29:12.953 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-10T23:29:12.997 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-10T23:29:13.110 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rgw. 2024-09-10T23:29:13.125 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:13.168 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:13.317 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-cheroot. 2024-09-10T23:29:13.328 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-10T23:29:13.367 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T23:29:13.547 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-10T23:29:13.562 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-10T23:29:13.605 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T23:29:13.695 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-10T23:29:13.709 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-10T23:29:13.745 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-10T23:29:13.891 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libradosstriper1. 2024-09-10T23:29:13.906 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:13.941 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:14.023 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-10T23:29:14.036 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-10T23:29:14.089 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-10T23:29:14.401 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-10T23:29:14.413 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-10T23:29:14.450 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-10T23:29:14.579 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-common. 2024-09-10T23:29:14.595 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:14.661 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:14.778 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-tempora. 2024-09-10T23:29:14.791 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-10T23:29:14.828 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-10T23:29:15.115 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-portend. 2024-09-10T23:29:15.127 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-10T23:29:15.156 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-10T23:29:15.492 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-10T23:29:15.505 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-10T23:29:15.542 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-10T23:29:15.828 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-10T23:29:15.841 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-10T23:29:15.859 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-base. 2024-09-10T23:29:15.873 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:15.878 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-10T23:29:15.951 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:16.223 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-natsort. 2024-09-10T23:29:16.235 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-10T23:29:16.273 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-10T23:29:16.505 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-10T23:29:16.518 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-10T23:29:16.563 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-10T23:29:16.601 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-logutils. 2024-09-10T23:29:16.616 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-10T23:29:16.659 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-10T23:29:16.799 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cheroot. 2024-09-10T23:29:16.813 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-10T23:29:16.857 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T23:29:17.020 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-10T23:29:17.033 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-10T23:29:17.078 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-10T23:29:17.172 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-10T23:29:17.187 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-10T23:29:17.236 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-10T23:29:17.466 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-mako. 2024-09-10T23:29:17.478 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-10T23:29:17.523 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T23:29:17.547 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-10T23:29:17.562 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-10T23:29:17.606 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-10T23:29:17.877 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-10T23:29:17.883 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-10T23:29:17.890 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-10T23:29:17.898 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-10T23:29:17.933 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-10T23:29:17.934 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-10T23:29:18.237 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-tempora. 2024-09-10T23:29:18.237 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-10T23:29:18.251 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-10T23:29:18.251 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-10T23:29:18.295 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-10T23:29:18.312 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-10T23:29:18.615 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-webob. 2024-09-10T23:29:18.623 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-portend. 2024-09-10T23:29:18.628 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-10T23:29:18.638 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-10T23:29:18.673 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T23:29:18.681 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-10T23:29:18.968 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-10T23:29:18.983 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-10T23:29:19.017 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-10T23:29:19.085 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-waitress. 2024-09-10T23:29:19.098 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-10T23:29:19.137 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T23:29:19.296 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-10T23:29:19.311 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-10T23:29:19.354 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-10T23:29:19.479 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-tempita. 2024-09-10T23:29:19.493 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-10T23:29:19.537 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T23:29:19.699 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-natsort. 2024-09-10T23:29:19.717 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-10T23:29:19.766 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-10T23:29:19.799 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-paste. 2024-09-10T23:29:19.813 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-10T23:29:19.848 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T23:29:20.036 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-logutils. 2024-09-10T23:29:20.050 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-10T23:29:20.086 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-10T23:29:20.207 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-10T23:29:20.221 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-10T23:29:20.268 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T23:29:20.372 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-10T23:29:20.387 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-10T23:29:20.430 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-10T23:29:20.571 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-10T23:29:20.585 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-10T23:29:20.629 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-10T23:29:20.716 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-mako. 2024-09-10T23:29:20.731 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-10T23:29:20.775 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T23:29:20.924 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-webtest. 2024-09-10T23:29:20.938 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-10T23:29:20.974 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-10T23:29:21.128 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-10T23:29:21.143 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-10T23:29:21.195 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-10T23:29:21.260 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pecan. 2024-09-10T23:29:21.274 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-10T23:29:21.318 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T23:29:21.490 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-10T23:29:21.505 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-10T23:29:21.548 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-10T23:29:21.681 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-certifi. 2024-09-10T23:29:21.695 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-10T23:29:21.739 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-10T23:29:21.851 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-webob. 2024-09-10T23:29:21.866 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-10T23:29:21.901 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T23:29:22.109 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-idna. 2024-09-10T23:29:22.123 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-10T23:29:22.167 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T23:29:22.221 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-waitress. 2024-09-10T23:29:22.235 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-10T23:29:22.323 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T23:29:22.504 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-urllib3. 2024-09-10T23:29:22.517 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-10T23:29:22.579 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T23:29:22.665 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-tempita. 2024-09-10T23:29:22.680 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-10T23:29:22.723 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T23:29:22.907 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-requests. 2024-09-10T23:29:22.920 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-10T23:29:22.965 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T23:29:23.027 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-paste. 2024-09-10T23:29:23.041 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-10T23:29:23.084 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T23:29:23.243 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-10T23:29:23.256 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-10T23:29:23.292 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T23:29:23.430 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-10T23:29:23.444 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-10T23:29:23.488 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T23:29:23.613 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-10T23:29:23.623 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:23.662 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:23.783 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-10T23:29:23.796 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-10T23:29:23.841 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-10T23:29:24.084 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-10T23:29:24.092 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:24.134 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:24.144 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-webtest. 2024-09-10T23:29:24.158 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-10T23:29:24.202 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-10T23:29:24.523 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pecan. 2024-09-10T23:29:24.536 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-10T23:29:24.695 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T23:29:25.127 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-certifi. 2024-09-10T23:29:25.141 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-10T23:29:25.185 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-10T23:29:25.420 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-10T23:29:25.429 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-10T23:29:25.470 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-10T23:29:25.489 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-idna. 2024-09-10T23:29:25.503 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-10T23:29:25.547 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T23:29:25.773 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-openssl. 2024-09-10T23:29:25.781 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-10T23:29:25.822 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-10T23:29:25.859 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-urllib3. 2024-09-10T23:29:25.874 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-10T23:29:25.917 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T23:29:26.193 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr. 2024-09-10T23:29:26.201 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:26.242 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:26.261 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-requests. 2024-09-10T23:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-10T23:29:26.320 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T23:29:26.612 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mon. 2024-09-10T23:29:26.621 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:26.631 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-10T23:29:26.647 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-10T23:29:26.653 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:26.690 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T23:29:26.984 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-10T23:29:26.999 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:27.042 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:27.285 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-osd. 2024-09-10T23:29:27.290 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:27.365 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:27.475 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-10T23:29:27.493 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:27.533 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:27.844 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-10T23:29:27.859 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-10T23:29:27.903 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-10T23:29:28.169 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph. 2024-09-10T23:29:28.177 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../069-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:28.180 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-openssl. 2024-09-10T23:29:28.195 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-10T23:29:28.218 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:28.230 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-10T23:29:28.537 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-fuse. 2024-09-10T23:29:28.542 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr. 2024-09-10T23:29:28.546 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:28.556 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:28.587 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:28.600 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:28.965 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mds. 2024-09-10T23:29:28.974 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:28.995 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mon. 2024-09-10T23:29:29.009 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:29.015 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:29.053 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:29.435 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package cephadm. 2024-09-10T23:29:29.444 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../072-cephadm_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:29.485 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:29.574 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-osd. 2024-09-10T23:29:29.588 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:29.632 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:29.846 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-10T23:29:29.855 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-10T23:29:29.896 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-10T23:29:30.265 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-10T23:29:30.274 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T23:29:30.315 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T23:29:30.652 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph. 2024-09-10T23:29:30.668 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../069-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:31.286 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:31.395 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-babel. 2024-09-10T23:29:31.403 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T23:29:31.445 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T23:29:31.638 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-fuse. 2024-09-10T23:29:31.653 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:31.697 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:31.798 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-jinja2. 2024-09-10T23:29:31.806 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-10T23:29:31.848 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T23:29:32.042 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mds. 2024-09-10T23:29:32.057 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:32.091 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:32.217 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-10T23:29:32.226 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:32.267 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:32.436 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephadm. 2024-09-10T23:29:32.451 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../072-cephadm_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:32.495 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:32.621 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-10T23:29:32.633 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-10T23:29:32.670 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-10T23:29:32.839 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-10T23:29:32.854 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-10T23:29:32.897 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-10T23:29:32.998 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-routes. 2024-09-10T23:29:33.007 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-10T23:29:33.048 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T23:29:33.201 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-10T23:29:33.215 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T23:29:33.258 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T23:29:33.428 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-10T23:29:33.436 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:33.476 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:34.238 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-babel. 2024-09-10T23:29:34.252 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T23:29:34.297 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T23:29:34.617 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-jinja2. 2024-09-10T23:29:34.630 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-10T23:29:34.675 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T23:29:35.011 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-10T23:29:35.025 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:35.070 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:35.573 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-10T23:29:35.587 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-10T23:29:35.631 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-10T23:29:35.873 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-10T23:29:35.882 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-10T23:29:35.909 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-routes. 2024-09-10T23:29:35.922 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T23:29:35.923 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-10T23:29:35.968 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T23:29:36.296 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-10T23:29:36.310 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:36.354 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:36.451 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-joblib. 2024-09-10T23:29:36.459 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-10T23:29:36.594 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T23:29:36.998 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-10T23:29:37.007 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-10T23:29:37.048 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-10T23:29:37.392 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-sklearn. 2024-09-10T23:29:37.401 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-10T23:29:37.442 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T23:29:38.160 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-10T23:29:38.169 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:38.201 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:38.618 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-10T23:29:38.634 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-10T23:29:38.677 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T23:29:39.180 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-joblib. 2024-09-10T23:29:39.195 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-10T23:29:39.212 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-cachetools. 2024-09-10T23:29:39.223 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-10T23:29:39.262 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-10T23:29:39.366 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T23:29:39.548 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-10T23:29:39.558 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-10T23:29:39.598 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-10T23:29:39.729 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-10T23:29:39.743 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-10T23:29:39.787 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-10T23:29:39.968 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-10T23:29:39.978 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-10T23:29:40.018 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T23:29:40.073 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-sklearn. 2024-09-10T23:29:40.088 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-10T23:29:40.131 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T23:29:40.346 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-rsa. 2024-09-10T23:29:40.356 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-10T23:29:40.456 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-10T23:29:40.841 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-10T23:29:40.856 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:40.899 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:40.966 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-google-auth. 2024-09-10T23:29:40.976 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-10T23:29:41.009 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-10T23:29:41.311 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-10T23:29:41.321 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-10T23:29:41.353 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T23:29:41.656 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-websocket. 2024-09-10T23:29:41.666 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-10T23:29:41.705 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-10T23:29:41.786 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-cachetools. 2024-09-10T23:29:41.802 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-10T23:29:41.861 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-10T23:29:42.034 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-10T23:29:42.044 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-10T23:29:42.117 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T23:29:42.214 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-10T23:29:42.229 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-10T23:29:42.280 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-10T23:29:42.684 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-10T23:29:42.700 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-10T23:29:42.750 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T23:29:42.781 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-10T23:29:42.790 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:42.822 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:43.154 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-rsa. 2024-09-10T23:29:43.169 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-10T23:29:43.175 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-10T23:29:43.184 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-10T23:29:43.220 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-10T23:29:43.225 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T23:29:43.533 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-google-auth. 2024-09-10T23:29:43.720 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-10T23:29:43.720 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-10T23:29:43.720 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-10T23:29:43.720 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T23:29:43.720 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T23:29:43.853 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-10T23:29:43.868 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-10T23:29:43.880 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package jq. 2024-09-10T23:29:43.889 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T23:29:43.902 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T23:29:43.922 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-10T23:29:44.166 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package socat. 2024-09-10T23:29:44.175 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-10T23:29:44.206 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-websocket. 2024-09-10T23:29:44.216 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-10T23:29:44.221 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-10T23:29:44.264 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-10T23:29:44.536 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package xmlstarlet. 2024-09-10T23:29:44.545 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-10T23:29:44.586 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-10T23:29:44.618 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-10T23:29:44.633 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-10T23:29:44.676 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T23:29:44.915 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-test. 2024-09-10T23:29:44.924 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:44.998 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:45.532 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-10T23:29:45.547 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:45.590 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:46.068 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-10T23:29:46.082 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-10T23:29:46.126 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T23:29:46.413 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-10T23:29:46.427 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T23:29:46.462 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T23:29:46.698 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package jq. 2024-09-10T23:29:46.713 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T23:29:46.757 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-10T23:29:47.043 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package socat. 2024-09-10T23:29:47.058 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-10T23:29:47.101 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-10T23:29:47.336 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-volume. 2024-09-10T23:29:47.346 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:47.385 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:47.405 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package xmlstarlet. 2024-09-10T23:29:47.419 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-10T23:29:47.463 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-10T23:29:47.773 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-10T23:29:47.784 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:47.809 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-test. 2024-09-10T23:29:47.823 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:47.824 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:47.867 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:48.167 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-colorama. 2024-09-10T23:29:48.178 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-10T23:29:48.217 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-10T23:29:48.511 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package cephfs-shell. 2024-09-10T23:29:48.521 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:48.562 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:48.789 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package cephfs-top. 2024-09-10T23:29:48.799 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:48.830 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:49.100 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-10T23:29:49.110 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:49.149 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:49.487 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-10T23:29:49.498 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-10T23:29:49.537 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T23:29:49.858 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-10T23:29:49.869 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-10T23:29:49.925 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-10T23:29:50.014 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ceph-volume. 2024-09-10T23:29:50.030 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:50.072 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:50.304 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package nvme-cli. 2024-09-10T23:29:50.314 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-10T23:29:50.379 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T23:29:50.460 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-10T23:29:50.476 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:50.518 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:50.804 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package pkg-config. 2024-09-10T23:29:50.814 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-10T23:29:50.821 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-colorama. 2024-09-10T23:29:50.837 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-10T23:29:50.858 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T23:29:50.879 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-10T23:29:51.103 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-10T23:29:51.113 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-10T23:29:51.140 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephfs-shell. 2024-09-10T23:29:51.153 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-10T23:29:51.155 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:51.198 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:51.460 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package cephfs-top. 2024-09-10T23:29:51.475 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T23:29:51.518 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:51.581 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-10T23:29:51.590 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-10T23:29:51.631 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-10T23:29:51.771 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-10T23:29:51.786 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:51.829 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:51.909 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pastescript. 2024-09-10T23:29:51.918 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-10T23:29:51.958 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-10T23:29:52.140 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-10T23:29:52.156 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-10T23:29:52.198 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T23:29:52.287 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pluggy. 2024-09-10T23:29:52.297 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-10T23:29:52.336 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-10T23:29:52.570 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-10T23:29:52.586 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-10T23:29:52.628 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-10T23:29:52.647 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-psutil. 2024-09-10T23:29:52.658 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-10T23:29:52.722 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-10T23:29:52.974 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package nvme-cli. 2024-09-10T23:29:52.990 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-10T23:29:53.032 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T23:29:53.067 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-py. 2024-09-10T23:29:53.077 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-10T23:29:53.117 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-10T23:29:53.429 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package pkg-config. 2024-09-10T23:29:53.444 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-10T23:29:53.478 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pygments. 2024-09-10T23:29:53.487 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T23:29:53.489 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-10T23:29:53.529 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T23:29:53.783 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-10T23:29:53.799 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-10T23:29:53.841 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-10T23:29:53.999 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-10T23:29:54.009 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-10T23:29:54.049 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-10T23:29:54.252 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-10T23:29:54.267 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-10T23:29:54.310 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-10T23:29:54.368 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-toml. 2024-09-10T23:29:54.382 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-10T23:29:54.426 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-10T23:29:54.597 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pastescript. 2024-09-10T23:29:54.612 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-10T23:29:54.655 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-10T23:29:54.771 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pytest. 2024-09-10T23:29:54.783 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-10T23:29:54.821 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T23:29:55.008 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pluggy. 2024-09-10T23:29:55.024 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-10T23:29:55.066 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-10T23:29:55.200 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-simplejson. 2024-09-10T23:29:55.211 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-10T23:29:55.258 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-10T23:29:55.378 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-psutil. 2024-09-10T23:29:55.393 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-10T23:29:55.436 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-10T23:29:55.494 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-10T23:29:55.505 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-10T23:29:55.568 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-10T23:29:55.964 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-py. 2024-09-10T23:29:55.980 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-10T23:29:56.022 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-10T23:29:56.393 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pygments. 2024-09-10T23:29:56.408 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-10T23:29:56.451 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T23:29:56.539 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package radosgw. 2024-09-10T23:29:56.550 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../123-radosgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:56.614 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:56.897 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-10T23:29:56.911 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-10T23:29:56.955 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-10T23:29:57.241 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-toml. 2024-09-10T23:29:57.256 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-10T23:29:57.299 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-10T23:29:57.359 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package rbd-fuse. 2024-09-10T23:29:57.370 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:57.409 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:57.644 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-pytest. 2024-09-10T23:29:57.658 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-10T23:29:57.702 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T23:29:57.750 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-10T23:29:58.039 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T23:29:58.048 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package python3-simplejson. 2024-09-10T23:29:58.062 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-10T23:29:58.106 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-10T23:29:58.140 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T23:29:58.359 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-10T23:29:58.373 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-10T23:29:58.381 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-10T23:29:58.417 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-10T23:29:58.794 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-10T23:29:59.154 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package radosgw. 2024-09-10T23:29:59.169 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../123-radosgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:59.212 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:59.440 INFO:teuthology.orchestra.run.smithi181.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-10T23:29:59.519 INFO:teuthology.orchestra.run.smithi181.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-10T23:29:59.566 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:29:59.924 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package rbd-fuse. 2024-09-10T23:29:59.940 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T23:29:59.982 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:00.127 INFO:teuthology.orchestra.run.smithi181.stdout:Adding system user cephadm....done 2024-09-10T23:30:00.272 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T23:30:00.292 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-10T23:30:00.555 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T23:30:00.565 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-10T23:30:00.681 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T23:30:00.858 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-10T23:30:00.942 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-10T23:30:00.984 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-10T23:30:01.243 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-10T23:30:01.353 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-10T23:30:01.545 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T23:30:01.671 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-10T23:30:01.945 INFO:teuthology.orchestra.run.smithi094.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-10T23:30:01.964 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T23:30:01.983 INFO:teuthology.orchestra.run.smithi094.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-10T23:30:02.033 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:02.324 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-10T23:30:02.592 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-10T23:30:02.652 INFO:teuthology.orchestra.run.smithi094.stdout:Adding system user cephadm....done 2024-09-10T23:30:02.806 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T23:30:02.885 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-10T23:30:03.066 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-10T23:30:03.186 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-10T23:30:03.309 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-10T23:30:03.435 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-10T23:30:03.538 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-10T23:30:03.695 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-10T23:30:03.814 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-10T23:30:03.962 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T23:30:04.088 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-10T23:30:04.090 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-10T23:30:04.349 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:04.399 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T23:30:04.625 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T23:30:04.752 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-10T23:30:04.759 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-10T23:30:04.877 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T23:30:05.019 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-10T23:30:05.045 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-10T23:30:05.279 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-10T23:30:05.337 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T23:30:05.447 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-10T23:30:05.563 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-10T23:30:05.723 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-10T23:30:05.815 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-10T23:30:06.098 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-10T23:30:06.149 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-10T23:30:06.153 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-10T23:30:06.217 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T23:30:06.360 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-10T23:30:06.409 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-10T23:30:06.685 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:06.770 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T23:30:06.912 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-10T23:30:06.953 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T23:30:07.079 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-10T23:30:07.164 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T23:30:07.205 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T23:30:07.331 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-10T23:30:07.423 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-10T23:30:07.599 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T23:30:07.741 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up zip (3.0-12build2) ... 2024-09-10T23:30:07.752 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-10T23:30:07.892 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T23:30:08.001 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-10T23:30:08.413 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-10T23:30:08.449 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-10T23:30:08.513 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T23:30:08.585 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T23:30:08.630 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-10T23:30:08.870 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T23:30:08.979 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-10T23:30:09.024 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T23:30:09.080 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T23:30:09.141 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-10T23:30:09.381 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-10T23:30:09.392 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T23:30:09.670 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-10T23:30:09.782 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-10T23:30:09.988 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up zip (3.0-12build2) ... 2024-09-10T23:30:10.042 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T23:30:10.114 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T23:30:10.435 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T23:30:10.745 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-10T23:30:10.808 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T23:30:11.067 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T23:30:11.096 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T23:30:11.168 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-10T23:30:11.285 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T23:30:11.464 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-10T23:30:11.620 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-10T23:30:11.757 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-10T23:30:11.882 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:12.021 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-10T23:30:12.200 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T23:30:12.281 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T23:30:12.551 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T23:30:12.657 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T23:30:12.942 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-10T23:30:13.243 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T23:30:13.577 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-10T23:30:13.774 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T23:30:13.845 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-10T23:30:13.971 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:14.042 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T23:30:14.176 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-10T23:30:14.280 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T23:30:14.452 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T23:30:14.570 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-10T23:30:14.623 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T23:30:14.671 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T23:30:14.948 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-10T23:30:15.215 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-10T23:30:15.516 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T23:30:15.642 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-10T23:30:15.845 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T23:30:15.968 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-10T23:30:16.097 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T23:30:16.233 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-10T23:30:16.245 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-10T23:30:16.504 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-10T23:30:16.507 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T23:30:16.616 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-10T23:30:16.747 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-10T23:30:16.767 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T23:30:16.989 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-10T23:30:17.069 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-10T23:30:17.290 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T23:30:17.345 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-10T23:30:17.416 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-10T23:30:17.604 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T23:30:17.709 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T23:30:17.730 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-10T23:30:17.835 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T23:30:18.048 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-10T23:30:18.145 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T23:30:18.316 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-10T23:30:18.446 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-10T23:30:18.575 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-10T23:30:18.722 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T23:30:18.826 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-10T23:30:18.831 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-10T23:30:18.990 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T23:30:19.069 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-10T23:30:19.079 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-10T23:30:19.309 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T23:30:19.361 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T23:30:19.468 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T23:30:19.487 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-10T23:30:19.780 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T23:30:19.878 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-10T23:30:19.906 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T23:30:20.162 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-10T23:30:20.190 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T23:30:20.263 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-10T23:30:20.475 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-10T23:30:20.531 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T23:30:20.657 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T23:30:20.784 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T23:30:20.910 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-10T23:30:20.991 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T23:30:21.036 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T23:30:21.124 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-10T23:30:21.326 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T23:30:21.364 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T23:30:21.488 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T23:30:21.704 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-10T23:30:21.830 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-10T23:30:21.890 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-10T23:30:22.090 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-10T23:30:22.149 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-10T23:30:22.267 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-10T23:30:22.365 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T23:30:22.535 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T23:30:22.634 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-10T23:30:22.677 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T23:30:23.005 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T23:30:23.043 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-10T23:30:23.337 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T23:30:23.346 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T23:30:23.681 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T23:30:23.708 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-10T23:30:23.807 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:23.817 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-10T23:30:23.933 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:24.059 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-10T23:30:24.084 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-10T23:30:24.371 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T23:30:24.429 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:24.530 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:24.822 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-10T23:30:24.822 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-10T23:30:24.924 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:25.039 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-10T23:30:25.083 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:25.209 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:25.332 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T23:30:25.418 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T23:30:25.418 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T23:30:25.684 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T23:30:25.785 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:25.911 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:26.035 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:26.037 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-10T23:30:26.162 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:26.287 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:26.388 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:26.531 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T23:30:26.723 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:26.849 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:26.992 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-10T23:30:27.268 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:27.386 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:27.512 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:27.712 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T23:30:27.712 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T23:30:27.801 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:27.952 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:28.078 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:28.179 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:28.280 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:28.371 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:28.414 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:28.497 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:28.623 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:28.749 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:28.867 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T23:30:28.984 INFO:teuthology.orchestra.run.smithi181.stdout:Adding group ceph....done 2024-09-10T23:30:29.369 INFO:teuthology.orchestra.run.smithi181.stdout:Adding system user ceph....done 2024-09-10T23:30:29.512 INFO:teuthology.orchestra.run.smithi181.stdout:Setting system user ceph properties....done 2024-09-10T23:30:29.528 INFO:teuthology.orchestra.run.smithi181.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-10T23:30:29.671 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-10T23:30:30.057 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-10T23:30:30.129 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:30.255 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:30.380 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:30.506 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:30.632 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:30.750 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:30.765 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:30.973 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T23:30:30.973 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T23:30:31.269 INFO:teuthology.orchestra.run.smithi094.stdout:Adding group ceph....done 2024-09-10T23:30:31.616 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:31.662 INFO:teuthology.orchestra.run.smithi094.stdout:Adding system user ceph....done 2024-09-10T23:30:31.742 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:31.906 INFO:teuthology.orchestra.run.smithi094.stdout:Setting system user ceph properties....done 2024-09-10T23:30:31.924 INFO:teuthology.orchestra.run.smithi094.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-10T23:30:32.063 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-10T23:30:32.254 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T23:30:32.254 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T23:30:32.484 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-10T23:30:33.112 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:33.159 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:33.359 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T23:30:33.359 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T23:30:33.441 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-10T23:30:33.977 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:34.078 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:34.147 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:34.359 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T23:30:34.360 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T23:30:34.633 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T23:30:34.633 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T23:30:34.999 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:35.211 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T23:30:35.211 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T23:30:35.280 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:35.639 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-10T23:30:35.933 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:36.269 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T23:30:36.269 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T23:30:36.323 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:36.554 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T23:30:36.554 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T23:30:36.910 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:37.036 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:37.187 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:37.217 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:37.413 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T23:30:37.413 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T23:30:37.420 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T23:30:37.420 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T23:30:38.059 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:38.080 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:38.223 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:38.349 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:38.367 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T23:30:38.367 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T23:30:38.491 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:38.886 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T23:30:38.995 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:39.009 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T23:30:39.121 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:39.138 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T23:30:39.263 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:39.488 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T23:30:39.488 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T23:30:40.115 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:40.257 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:40.375 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:40.415 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T23:30:40.493 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T23:30:40.862 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T23:30:40.971 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T23:30:41.143 INFO:teuthology.orchestra.run.smithi181.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T23:30:41.149 DEBUG:teuthology.orchestra.run.smithi181:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-10T23:30:41.156 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T23:30:41.224 INFO:teuthology.orchestra.run.smithi181.stdout:Reading package lists... 2024-09-10T23:30:41.409 INFO:teuthology.orchestra.run.smithi181.stdout:Building dependency tree... 2024-09-10T23:30:41.409 INFO:teuthology.orchestra.run.smithi181.stdout:Reading state information... 2024-09-10T23:30:41.594 INFO:teuthology.orchestra.run.smithi181.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-10T23:30:41.594 INFO:teuthology.orchestra.run.smithi181.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-10T23:30:41.594 INFO:teuthology.orchestra.run.smithi181.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-10T23:30:41.594 INFO:teuthology.orchestra.run.smithi181.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T23:30:41.594 INFO:teuthology.orchestra.run.smithi181.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T23:30:41.594 INFO:teuthology.orchestra.run.smithi181.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T23:30:41.594 INFO:teuthology.orchestra.run.smithi181.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T23:30:41.595 INFO:teuthology.orchestra.run.smithi181.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T23:30:41.595 INFO:teuthology.orchestra.run.smithi181.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T23:30:41.595 INFO:teuthology.orchestra.run.smithi181.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T23:30:41.596 INFO:teuthology.orchestra.run.smithi181.stdout:The following additional packages will be installed: 2024-09-10T23:30:41.596 INFO:teuthology.orchestra.run.smithi181.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-10T23:30:41.596 INFO:teuthology.orchestra.run.smithi181.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-10T23:30:41.596 INFO:teuthology.orchestra.run.smithi181.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-10T23:30:41.597 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-10T23:30:41.597 INFO:teuthology.orchestra.run.smithi181.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T23:30:41.598 INFO:teuthology.orchestra.run.smithi181.stdout:Suggested packages: 2024-09-10T23:30:41.598 INFO:teuthology.orchestra.run.smithi181.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-10T23:30:41.598 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-10T23:30:41.598 INFO:teuthology.orchestra.run.smithi181.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-10T23:30:41.654 INFO:teuthology.orchestra.run.smithi181.stdout:The following NEW packages will be installed: 2024-09-10T23:30:41.654 INFO:teuthology.orchestra.run.smithi181.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-10T23:30:41.655 INFO:teuthology.orchestra.run.smithi181.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-10T23:30:41.655 INFO:teuthology.orchestra.run.smithi181.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-10T23:30:41.655 INFO:teuthology.orchestra.run.smithi181.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-10T23:30:41.655 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-10T23:30:41.655 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-10T23:30:41.655 INFO:teuthology.orchestra.run.smithi181.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T23:30:41.656 INFO:teuthology.orchestra.run.smithi181.stdout:The following packages will be upgraded: 2024-09-10T23:30:41.657 INFO:teuthology.orchestra.run.smithi181.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-10T23:30:41.841 INFO:teuthology.orchestra.run.smithi181.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-10T23:30:41.842 INFO:teuthology.orchestra.run.smithi181.stdout:Need to get 55.8 MB of archives. 2024-09-10T23:30:41.842 INFO:teuthology.orchestra.run.smithi181.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-10T23:30:41.842 INFO:teuthology.orchestra.run.smithi181.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-10T23:30:42.358 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T23:30:42.460 INFO:teuthology.orchestra.run.smithi181.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-10T23:30:42.476 INFO:teuthology.orchestra.run.smithi181.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-10T23:30:42.600 INFO:teuthology.orchestra.run.smithi181.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-10T23:30:42.601 INFO:teuthology.orchestra.run.smithi181.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-10T23:30:42.602 INFO:teuthology.orchestra.run.smithi181.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-10T23:30:42.630 INFO:teuthology.orchestra.run.smithi181.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-10T23:30:42.631 INFO:teuthology.orchestra.run.smithi181.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-10T23:30:42.637 INFO:teuthology.orchestra.run.smithi181.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-10T23:30:42.637 INFO:teuthology.orchestra.run.smithi181.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-10T23:30:42.638 INFO:teuthology.orchestra.run.smithi181.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-10T23:30:42.640 INFO:teuthology.orchestra.run.smithi181.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-10T23:30:42.704 INFO:teuthology.orchestra.run.smithi181.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-10T23:30:42.704 INFO:teuthology.orchestra.run.smithi181.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-10T23:30:42.704 INFO:teuthology.orchestra.run.smithi181.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-10T23:30:42.785 INFO:teuthology.orchestra.run.smithi181.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-10T23:30:43.045 INFO:teuthology.orchestra.run.smithi094.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T23:30:43.048 DEBUG:teuthology.orchestra.run.smithi094:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-10T23:30:43.124 INFO:teuthology.orchestra.run.smithi094.stdout:Reading package lists... 2024-09-10T23:30:43.307 INFO:teuthology.orchestra.run.smithi094.stdout:Building dependency tree... 2024-09-10T23:30:43.308 INFO:teuthology.orchestra.run.smithi094.stdout:Reading state information... 2024-09-10T23:30:43.464 INFO:teuthology.orchestra.run.smithi181.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-10T23:30:43.466 INFO:teuthology.orchestra.run.smithi181.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-10T23:30:43.466 INFO:teuthology.orchestra.run.smithi181.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-10T23:30:43.469 INFO:teuthology.orchestra.run.smithi181.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-10T23:30:43.469 INFO:teuthology.orchestra.run.smithi181.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-10T23:30:43.473 INFO:teuthology.orchestra.run.smithi181.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-10T23:30:43.475 INFO:teuthology.orchestra.run.smithi181.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-10T23:30:43.491 INFO:teuthology.orchestra.run.smithi094.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-10T23:30:43.491 INFO:teuthology.orchestra.run.smithi094.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-10T23:30:43.491 INFO:teuthology.orchestra.run.smithi094.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-10T23:30:43.491 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T23:30:43.491 INFO:teuthology.orchestra.run.smithi094.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T23:30:43.491 INFO:teuthology.orchestra.run.smithi094.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T23:30:43.491 INFO:teuthology.orchestra.run.smithi094.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T23:30:43.492 INFO:teuthology.orchestra.run.smithi094.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T23:30:43.492 INFO:teuthology.orchestra.run.smithi094.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T23:30:43.492 INFO:teuthology.orchestra.run.smithi094.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T23:30:43.493 INFO:teuthology.orchestra.run.smithi094.stdout:The following additional packages will be installed: 2024-09-10T23:30:43.493 INFO:teuthology.orchestra.run.smithi094.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-10T23:30:43.493 INFO:teuthology.orchestra.run.smithi094.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-10T23:30:43.494 INFO:teuthology.orchestra.run.smithi094.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-10T23:30:43.494 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-10T23:30:43.494 INFO:teuthology.orchestra.run.smithi094.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T23:30:43.495 INFO:teuthology.orchestra.run.smithi094.stdout:Suggested packages: 2024-09-10T23:30:43.495 INFO:teuthology.orchestra.run.smithi094.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-10T23:30:43.495 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-10T23:30:43.495 INFO:teuthology.orchestra.run.smithi094.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-10T23:30:43.526 INFO:teuthology.orchestra.run.smithi181.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-10T23:30:43.526 INFO:teuthology.orchestra.run.smithi181.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-10T23:30:43.553 INFO:teuthology.orchestra.run.smithi094.stdout:The following NEW packages will be installed: 2024-09-10T23:30:43.553 INFO:teuthology.orchestra.run.smithi094.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-10T23:30:43.553 INFO:teuthology.orchestra.run.smithi094.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-10T23:30:43.554 INFO:teuthology.orchestra.run.smithi094.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-10T23:30:43.554 INFO:teuthology.orchestra.run.smithi094.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-10T23:30:43.554 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-10T23:30:43.554 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-10T23:30:43.554 INFO:teuthology.orchestra.run.smithi094.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T23:30:43.555 INFO:teuthology.orchestra.run.smithi094.stdout:The following packages will be upgraded: 2024-09-10T23:30:43.556 INFO:teuthology.orchestra.run.smithi094.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-10T23:30:43.564 INFO:teuthology.orchestra.run.smithi181.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-10T23:30:43.609 INFO:teuthology.orchestra.run.smithi181.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-10T23:30:43.620 INFO:teuthology.orchestra.run.smithi181.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-10T23:30:43.696 INFO:teuthology.orchestra.run.smithi181.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-10T23:30:43.700 INFO:teuthology.orchestra.run.smithi181.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-10T23:30:43.700 INFO:teuthology.orchestra.run.smithi181.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-10T23:30:43.701 INFO:teuthology.orchestra.run.smithi181.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-10T23:30:43.701 INFO:teuthology.orchestra.run.smithi181.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-10T23:30:43.716 INFO:teuthology.orchestra.run.smithi181.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-10T23:30:43.718 INFO:teuthology.orchestra.run.smithi181.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-10T23:30:43.741 INFO:teuthology.orchestra.run.smithi094.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-10T23:30:43.742 INFO:teuthology.orchestra.run.smithi094.stdout:Need to get 55.8 MB of archives. 2024-09-10T23:30:43.742 INFO:teuthology.orchestra.run.smithi094.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-10T23:30:43.742 INFO:teuthology.orchestra.run.smithi094.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-10T23:30:44.205 INFO:teuthology.orchestra.run.smithi181.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-10T23:30:44.206 INFO:teuthology.orchestra.run.smithi181.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-10T23:30:44.206 INFO:teuthology.orchestra.run.smithi181.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-10T23:30:44.206 INFO:teuthology.orchestra.run.smithi181.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-10T23:30:44.207 INFO:teuthology.orchestra.run.smithi181.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-10T23:30:44.207 INFO:teuthology.orchestra.run.smithi181.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-10T23:30:44.266 INFO:teuthology.orchestra.run.smithi181.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-10T23:30:44.266 INFO:teuthology.orchestra.run.smithi181.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-10T23:30:44.362 INFO:teuthology.orchestra.run.smithi094.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-10T23:30:44.378 INFO:teuthology.orchestra.run.smithi094.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-10T23:30:44.502 INFO:teuthology.orchestra.run.smithi094.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-10T23:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-10T23:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-10T23:30:44.532 INFO:teuthology.orchestra.run.smithi094.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-10T23:30:44.533 INFO:teuthology.orchestra.run.smithi094.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-10T23:30:44.537 INFO:teuthology.orchestra.run.smithi094.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-10T23:30:44.539 INFO:teuthology.orchestra.run.smithi094.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-10T23:30:44.540 INFO:teuthology.orchestra.run.smithi094.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-10T23:30:44.541 INFO:teuthology.orchestra.run.smithi094.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-10T23:30:44.606 INFO:teuthology.orchestra.run.smithi094.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-10T23:30:44.606 INFO:teuthology.orchestra.run.smithi094.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-10T23:30:44.606 INFO:teuthology.orchestra.run.smithi094.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-10T23:30:44.650 INFO:teuthology.orchestra.run.smithi181.stdout:Preconfiguring packages ... 2024-09-10T23:30:44.687 INFO:teuthology.orchestra.run.smithi094.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-10T23:30:44.815 INFO:teuthology.orchestra.run.smithi181.stdout:Fetched 55.8 MB in 3s (21.4 MB/s) 2024-09-10T23:30:45.051 INFO:teuthology.orchestra.run.smithi181.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T23:30:45.055 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T23:30:45.190 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T23:30:45.369 INFO:teuthology.orchestra.run.smithi094.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-10T23:30:45.371 INFO:teuthology.orchestra.run.smithi094.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-10T23:30:45.371 INFO:teuthology.orchestra.run.smithi094.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-10T23:30:45.374 INFO:teuthology.orchestra.run.smithi094.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-10T23:30:45.374 INFO:teuthology.orchestra.run.smithi094.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-10T23:30:45.378 INFO:teuthology.orchestra.run.smithi094.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-10T23:30:45.380 INFO:teuthology.orchestra.run.smithi094.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-10T23:30:45.429 INFO:teuthology.orchestra.run.smithi094.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-10T23:30:45.429 INFO:teuthology.orchestra.run.smithi094.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-10T23:30:45.505 INFO:teuthology.orchestra.run.smithi094.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-10T23:30:45.511 INFO:teuthology.orchestra.run.smithi094.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-10T23:30:45.522 INFO:teuthology.orchestra.run.smithi094.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-10T23:30:45.598 INFO:teuthology.orchestra.run.smithi094.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-10T23:30:45.602 INFO:teuthology.orchestra.run.smithi094.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-10T23:30:45.603 INFO:teuthology.orchestra.run.smithi094.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-10T23:30:45.603 INFO:teuthology.orchestra.run.smithi094.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-10T23:30:45.604 INFO:teuthology.orchestra.run.smithi094.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-10T23:30:45.620 INFO:teuthology.orchestra.run.smithi094.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-10T23:30:45.622 INFO:teuthology.orchestra.run.smithi094.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-10T23:30:45.826 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-10T23:30:46.104 INFO:teuthology.orchestra.run.smithi094.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-10T23:30:46.104 INFO:teuthology.orchestra.run.smithi094.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-10T23:30:46.104 INFO:teuthology.orchestra.run.smithi094.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-10T23:30:46.104 INFO:teuthology.orchestra.run.smithi094.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-10T23:30:46.105 INFO:teuthology.orchestra.run.smithi094.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-10T23:30:46.105 INFO:teuthology.orchestra.run.smithi094.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-10T23:30:46.160 INFO:teuthology.orchestra.run.smithi094.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-10T23:30:46.161 INFO:teuthology.orchestra.run.smithi094.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-10T23:30:46.545 INFO:teuthology.orchestra.run.smithi094.stdout:Preconfiguring packages ... 2024-09-10T23:30:46.698 INFO:teuthology.orchestra.run.smithi094.stdout:Fetched 55.8 MB in 3s (21.5 MB/s) 2024-09-10T23:30:46.975 INFO:teuthology.orchestra.run.smithi181.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-10T23:30:46.975 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T23:30:46.979 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T23:30:47.221 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package flex. 2024-09-10T23:30:47.240 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T23:30:47.292 INFO:teuthology.orchestra.run.smithi181.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T23:30:47.295 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-10T23:30:47.345 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-10T23:30:47.898 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-10T23:30:48.008 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T23:30:48.177 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-10T23:30:48.583 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T23:30:49.013 INFO:teuthology.orchestra.run.smithi094.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-10T23:30:49.164 INFO:teuthology.orchestra.run.smithi181.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T23:30:49.168 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T23:30:49.260 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package flex. 2024-09-10T23:30:49.318 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T23:30:49.329 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T23:30:49.332 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-10T23:30:49.367 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-10T23:30:49.772 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T23:30:49.862 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T23:30:49.908 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T23:30:50.065 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-10T23:30:50.290 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T23:30:50.447 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T23:30:50.784 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package bison. 2024-09-10T23:30:50.860 INFO:teuthology.orchestra.run.smithi181.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T23:30:50.864 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-10T23:30:50.920 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T23:30:51.001 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T23:30:51.005 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T23:30:51.130 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T23:30:51.412 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-10T23:30:51.426 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-10T23:30:51.495 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T23:30:51.559 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T23:30:51.686 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T23:30:51.932 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package dump. 2024-09-10T23:30:51.948 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-10T23:30:52.009 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T23:30:52.015 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking dump (0.4b47-3) ... 2024-09-10T23:30:52.285 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package indent. 2024-09-10T23:30:52.301 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T23:30:52.343 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T23:30:52.420 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package bison. 2024-09-10T23:30:52.493 INFO:teuthology.orchestra.run.smithi094.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T23:30:52.496 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-10T23:30:52.530 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T23:30:52.664 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-10T23:30:52.680 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-10T23:30:52.714 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T23:30:52.940 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-10T23:30:52.954 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-10T23:30:52.990 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T23:30:53.059 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-10T23:30:53.075 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-10T23:30:53.118 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T23:30:53.301 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package dump. 2024-09-10T23:30:53.315 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-10T23:30:53.359 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking dump (0.4b47-3) ... 2024-09-10T23:30:53.412 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-10T23:30:53.429 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-10T23:30:53.470 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T23:30:53.638 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package indent. 2024-09-10T23:30:53.652 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T23:30:53.696 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T23:30:53.774 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-10T23:30:53.791 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-10T23:30:53.857 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T23:30:54.082 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-10T23:30:54.098 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-10T23:30:54.102 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-10T23:30:54.118 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-10T23:30:54.140 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T23:30:54.160 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T23:30:54.411 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-10T23:30:54.421 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libjson-perl. 2024-09-10T23:30:54.427 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-10T23:30:54.437 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-10T23:30:54.477 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T23:30:54.479 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-10T23:30:54.797 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-10T23:30:54.813 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-10T23:30:54.824 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-10T23:30:54.840 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-10T23:30:54.863 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T23:30:54.882 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T23:30:55.250 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-10T23:30:55.266 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-10T23:30:55.308 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T23:30:55.594 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-10T23:30:55.610 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-10T23:30:55.660 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T23:30:55.963 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libjson-perl. 2024-09-10T23:30:55.979 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-10T23:30:56.030 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-10T23:30:56.227 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-10T23:30:56.243 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T23:30:56.285 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T23:30:56.408 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-10T23:30:56.424 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-10T23:30:56.491 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T23:30:56.605 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-10T23:30:56.621 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T23:30:56.663 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T23:30:57.016 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-10T23:30:57.032 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T23:30:57.074 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T23:30:57.411 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-10T23:30:57.427 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-10T23:30:57.461 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T23:30:57.924 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-10T23:30:57.941 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T23:30:57.982 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T23:30:58.376 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-10T23:30:58.393 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T23:30:58.435 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T23:30:58.620 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-10T23:30:58.636 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T23:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T23:30:58.771 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-10T23:30:58.788 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T23:30:58.846 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T23:30:58.939 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-10T23:30:58.955 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T23:30:58.997 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T23:30:59.124 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-10T23:30:59.141 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T23:30:59.182 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T23:30:59.333 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-10T23:30:59.349 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T23:30:59.391 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T23:30:59.460 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-10T23:30:59.476 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T23:30:59.518 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T23:30:59.686 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-10T23:30:59.702 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-10T23:30:59.744 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T23:31:00.063 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-10T23:31:00.072 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-10T23:31:00.078 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-10T23:31:00.087 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T23:31:00.130 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T23:31:00.139 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T23:31:00.475 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-10T23:31:00.490 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T23:31:00.500 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package modemmanager. 2024-09-10T23:31:00.515 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T23:31:00.558 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T23:31:00.558 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T23:31:00.894 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-10T23:31:00.910 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T23:31:00.944 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T23:31:01.148 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package network-manager. 2024-09-10T23:31:01.164 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T23:31:01.205 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-10T23:31:01.221 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T23:31:01.231 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-10T23:31:01.263 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T23:31:01.516 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-10T23:31:01.531 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T23:31:01.574 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T23:31:01.956 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ppp. 2024-09-10T23:31:01.972 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-10T23:31:02.094 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-10T23:31:02.108 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-10T23:31:02.152 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T23:31:02.321 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T23:31:02.447 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package modemmanager. 2024-09-10T23:31:02.462 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T23:31:02.497 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T23:31:02.727 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package pptp-linux. 2024-09-10T23:31:02.742 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-10T23:31:02.794 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-10T23:31:03.261 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package network-manager. 2024-09-10T23:31:03.277 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T23:31:03.329 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-10T23:31:03.572 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-10T23:31:03.589 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-10T23:31:03.630 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-10T23:31:03.940 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package ppp. 2024-09-10T23:31:03.954 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-10T23:31:03.966 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-10T23:31:03.983 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-10T23:31:04.025 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking postgresql-client-common (238) ... 2024-09-10T23:31:04.334 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T23:31:04.354 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-10T23:31:04.370 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T23:31:04.412 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T23:31:04.679 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package pptp-linux. 2024-09-10T23:31:04.693 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-10T23:31:04.737 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-10T23:31:04.990 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-10T23:31:05.005 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-10T23:31:05.034 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package postgresql-common. 2024-09-10T23:31:05.049 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-10T23:31:05.050 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-10T23:31:05.110 INFO:teuthology.orchestra.run.smithi181.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-10T23:31:05.157 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking postgresql-common (238) ... 2024-09-10T23:31:05.344 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-10T23:31:05.358 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-10T23:31:05.402 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-client-common (238) ... 2024-09-10T23:31:05.732 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-10T23:31:05.745 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T23:31:05.790 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T23:31:05.825 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package postgresql-14. 2024-09-10T23:31:05.841 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T23:31:05.886 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T23:31:06.354 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-common. 2024-09-10T23:31:06.368 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-10T23:31:06.430 INFO:teuthology.orchestra.run.smithi094.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-10T23:31:06.453 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-common (238) ... 2024-09-10T23:31:06.887 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-14. 2024-09-10T23:31:06.902 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T23:31:06.973 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T23:31:07.011 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package postgresql. 2024-09-10T23:31:07.026 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-10T23:31:07.086 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking postgresql (14+238) ... 2024-09-10T23:31:07.321 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package postgresql-client. 2024-09-10T23:31:07.336 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-10T23:31:07.379 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking postgresql-client (14+238) ... 2024-09-10T23:31:07.649 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-10T23:31:07.664 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-10T23:31:07.707 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-10T23:31:08.001 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package tcl8.6. 2024-09-10T23:31:08.018 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-10T23:31:08.060 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T23:31:08.126 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql. 2024-09-10T23:31:08.140 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-10T23:31:08.184 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql (14+238) ... 2024-09-10T23:31:08.345 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package tcl. 2024-09-10T23:31:08.361 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-10T23:31:08.408 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-10T23:31:08.462 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-client. 2024-09-10T23:31:08.476 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-10T23:31:08.520 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-client (14+238) ... 2024-09-10T23:31:08.739 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-10T23:31:08.754 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-10T23:31:08.798 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-10T23:31:08.807 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package wpasupplicant. 2024-09-10T23:31:08.823 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-10T23:31:08.865 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T23:31:09.117 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package tcl8.6. 2024-09-10T23:31:09.132 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-10T23:31:09.176 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T23:31:09.219 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-10T23:31:09.236 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-10T23:31:09.286 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-10T23:31:09.454 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package tcl. 2024-09-10T23:31:09.469 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-10T23:31:09.533 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-10T23:31:09.689 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-10T23:31:09.705 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-10T23:31:09.756 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T23:31:09.868 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package wpasupplicant. 2024-09-10T23:31:09.881 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-10T23:31:09.924 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T23:31:10.075 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up postgresql-client-common (238) ... 2024-09-10T23:31:10.278 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-10T23:31:10.279 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-10T23:31:10.294 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-10T23:31:10.329 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-10T23:31:10.413 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T23:31:10.539 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T23:31:10.665 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T23:31:10.682 INFO:teuthology.orchestra.run.smithi094.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-10T23:31:10.697 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-10T23:31:10.791 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T23:31:10.799 INFO:teuthology.orchestra.run.smithi094.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T23:31:10.934 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T23:31:11.085 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-client-common (238) ... 2024-09-10T23:31:11.273 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-10T23:31:11.399 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T23:31:11.525 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T23:31:11.651 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T23:31:11.778 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T23:31:11.889 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-10T23:31:11.904 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T23:31:11.998 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T23:31:12.402 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T23:31:12.402 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T23:31:12.858 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-10T23:31:12.952 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T23:31:13.125 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T23:31:13.260 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T23:31:13.330 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T23:31:13.330 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T23:31:13.386 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T23:31:13.511 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-10T23:31:13.989 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T23:31:14.046 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T23:31:14.115 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T23:31:14.234 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T23:31:14.306 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T23:31:14.351 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-10T23:31:14.432 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T23:31:14.477 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T23:31:14.533 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T23:31:14.629 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T23:31:14.755 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T23:31:14.881 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T23:31:15.222 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T23:31:15.356 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up dump (0.4b47-3) ... 2024-09-10T23:31:15.451 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-10T23:31:15.524 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T23:31:15.596 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T23:31:15.667 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T23:31:15.722 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up dump (0.4b47-3) ... 2024-09-10T23:31:15.768 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-10T23:31:15.817 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-10T23:31:15.894 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-10T23:31:15.905 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T23:31:16.025 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T23:31:16.045 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T23:31:16.127 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-10T23:31:16.135 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-10T23:31:16.205 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T23:31:16.252 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-10T23:31:16.331 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T23:31:16.379 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T23:31:16.468 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-10T23:31:16.524 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T23:31:16.539 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T23:31:16.642 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T23:31:16.665 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T23:31:16.742 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-10T23:31:16.841 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T23:31:16.894 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up postgresql-client (14+238) ... 2024-09-10T23:31:16.959 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T23:31:17.045 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-10T23:31:17.069 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-10T23:31:17.262 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-client (14+238) ... 2024-09-10T23:31:17.388 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-10T23:31:17.640 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:17.641 INFO:teuthology.orchestra.run.smithi181.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-10T23:31:17.641 INFO:teuthology.orchestra.run.smithi181.stdout:which means they are currently configured by ifupdown: 2024-09-10T23:31:17.641 INFO:teuthology.orchestra.run.smithi181.stdout:- enp3s0f1 2024-09-10T23:31:17.641 INFO:teuthology.orchestra.run.smithi181.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-10T23:31:17.641 INFO:teuthology.orchestra.run.smithi181.stdout:remove their configuration from /etc/network/interfaces. 2024-09-10T23:31:17.641 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:17.773 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-10T23:31:17.982 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:17.983 INFO:teuthology.orchestra.run.smithi094.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-10T23:31:17.983 INFO:teuthology.orchestra.run.smithi094.stdout:which means they are currently configured by ifupdown: 2024-09-10T23:31:17.983 INFO:teuthology.orchestra.run.smithi094.stdout:- enp3s0f1 2024-09-10T23:31:17.983 INFO:teuthology.orchestra.run.smithi094.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-10T23:31:17.983 INFO:teuthology.orchestra.run.smithi094.stdout:remove their configuration from /etc/network/interfaces. 2024-09-10T23:31:17.983 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:18.115 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-10T23:31:18.190 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-10T23:31:18.500 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-10T23:31:18.578 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-10T23:31:18.907 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-10T23:31:19.331 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T23:31:19.482 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T23:31:19.608 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-10T23:31:19.625 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T23:31:19.767 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up postgresql-common (238) ... 2024-09-10T23:31:19.785 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T23:31:19.903 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-10T23:31:20.029 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-common (238) ... 2024-09-10T23:31:20.862 INFO:teuthology.orchestra.run.smithi181.stdout:Adding user postgres to group ssl-cert 2024-09-10T23:31:20.976 INFO:teuthology.orchestra.run.smithi094.stdout:Adding user postgres to group ssl-cert 2024-09-10T23:31:21.122 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:21.122 INFO:teuthology.orchestra.run.smithi181.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-10T23:31:21.230 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:21.230 INFO:teuthology.orchestra.run.smithi094.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-10T23:31:21.386 INFO:teuthology.orchestra.run.smithi181.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-10T23:31:21.386 INFO:teuthology.orchestra.run.smithi181.stdout:Removing obsolete dictionary files: 2024-09-10T23:31:21.469 INFO:teuthology.orchestra.run.smithi094.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-10T23:31:21.469 INFO:teuthology.orchestra.run.smithi094.stdout:Removing obsolete dictionary files: 2024-09-10T23:31:22.094 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-10T23:31:22.201 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-10T23:31:22.594 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T23:31:22.682 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T23:31:22.958 INFO:teuthology.orchestra.run.smithi181.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-10T23:31:22.972 INFO:teuthology.orchestra.run.smithi181.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-10T23:31:23.067 INFO:teuthology.orchestra.run.smithi181.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-10T23:31:23.067 INFO:teuthology.orchestra.run.smithi181.stdout:This user must also own the server process. 2024-09-10T23:31:23.067 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:23.067 INFO:teuthology.orchestra.run.smithi181.stdout:The database cluster will be initialized with locale "en_US". 2024-09-10T23:31:23.067 INFO:teuthology.orchestra.run.smithi181.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-10T23:31:23.067 INFO:teuthology.orchestra.run.smithi181.stdout:The default text search configuration will be set to "english". 2024-09-10T23:31:23.067 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:23.068 INFO:teuthology.orchestra.run.smithi181.stdout:Data page checksums are disabled. 2024-09-10T23:31:23.068 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:23.068 INFO:teuthology.orchestra.run.smithi181.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-10T23:31:23.069 INFO:teuthology.orchestra.run.smithi181.stdout:creating subdirectories ... ok 2024-09-10T23:31:23.069 INFO:teuthology.orchestra.run.smithi181.stdout:selecting dynamic shared memory implementation ... posix 2024-09-10T23:31:23.092 INFO:teuthology.orchestra.run.smithi181.stdout:selecting default max_connections ... 100 2024-09-10T23:31:23.126 INFO:teuthology.orchestra.run.smithi181.stdout:selecting default shared_buffers ... 128MB 2024-09-10T23:31:23.140 INFO:teuthology.orchestra.run.smithi181.stdout:selecting default time zone ... Etc/UTC 2024-09-10T23:31:23.141 INFO:teuthology.orchestra.run.smithi181.stdout:creating configuration files ... ok 2024-09-10T23:31:23.153 INFO:teuthology.orchestra.run.smithi094.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-10T23:31:23.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-10T23:31:23.233 INFO:teuthology.orchestra.run.smithi094.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-10T23:31:23.233 INFO:teuthology.orchestra.run.smithi094.stdout:This user must also own the server process. 2024-09-10T23:31:23.233 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:23.233 INFO:teuthology.orchestra.run.smithi094.stdout:The database cluster will be initialized with locale "en_US". 2024-09-10T23:31:23.233 INFO:teuthology.orchestra.run.smithi094.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-10T23:31:23.234 INFO:teuthology.orchestra.run.smithi094.stdout:The default text search configuration will be set to "english". 2024-09-10T23:31:23.234 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:23.234 INFO:teuthology.orchestra.run.smithi094.stdout:Data page checksums are disabled. 2024-09-10T23:31:23.234 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:23.234 INFO:teuthology.orchestra.run.smithi094.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-10T23:31:23.234 INFO:teuthology.orchestra.run.smithi094.stdout:creating subdirectories ... ok 2024-09-10T23:31:23.234 INFO:teuthology.orchestra.run.smithi094.stdout:selecting dynamic shared memory implementation ... posix 2024-09-10T23:31:23.259 INFO:teuthology.orchestra.run.smithi094.stdout:selecting default max_connections ... 100 2024-09-10T23:31:23.292 INFO:teuthology.orchestra.run.smithi094.stdout:selecting default shared_buffers ... 128MB 2024-09-10T23:31:23.304 INFO:teuthology.orchestra.run.smithi181.stdout:running bootstrap script ... ok 2024-09-10T23:31:23.306 INFO:teuthology.orchestra.run.smithi094.stdout:selecting default time zone ... Etc/UTC 2024-09-10T23:31:23.307 INFO:teuthology.orchestra.run.smithi094.stdout:creating configuration files ... ok 2024-09-10T23:31:23.470 INFO:teuthology.orchestra.run.smithi094.stdout:running bootstrap script ... ok 2024-09-10T23:31:24.089 INFO:teuthology.orchestra.run.smithi181.stdout:performing post-bootstrap initialization ... ok 2024-09-10T23:31:24.123 INFO:teuthology.orchestra.run.smithi094.stdout:performing post-bootstrap initialization ... ok 2024-09-10T23:31:24.645 INFO:teuthology.orchestra.run.smithi181.stdout:syncing data to disk ... ok 2024-09-10T23:31:24.681 INFO:teuthology.orchestra.run.smithi094.stdout:syncing data to disk ... ok 2024-09-10T23:31:25.172 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-10T23:31:25.181 INFO:teuthology.orchestra.run.smithi094.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-10T23:31:28.133 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T23:31:28.250 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-10T23:31:28.260 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T23:31:28.361 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-10T23:31:28.368 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up postgresql (14+238) ... 2024-09-10T23:31:28.462 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up postgresql (14+238) ... 2024-09-10T23:31:28.495 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T23:31:28.580 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T23:31:28.621 INFO:teuthology.orchestra.run.smithi094.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T23:31:28.730 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T23:31:28.991 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T23:31:28.991 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T23:31:29.306 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T23:31:29.306 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T23:31:29.750 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T23:31:29.842 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-10T23:31:30.186 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T23:31:30.215 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T23:31:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-10T23:31:30.337 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T23:31:30.610 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T23:31:30.779 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T23:31:31.681 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-10T23:31:31.781 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T23:31:32.175 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-10T23:31:32.275 INFO:teuthology.orchestra.run.smithi094.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T23:31:32.355 INFO:teuthology.orchestra.run.smithi181.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T23:31:32.362 DEBUG:teuthology.parallel:result is None 2024-09-10T23:31:32.830 INFO:teuthology.orchestra.run.smithi094.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T23:31:32.836 DEBUG:teuthology.parallel:result is None 2024-09-10T23:31:32.837 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:31:32.995 DEBUG:teuthology.orchestra.run.smithi094:> dpkg-query -W -f '${Version}' ceph 2024-09-10T23:31:33.016 INFO:teuthology.orchestra.run.smithi094.stdout:19.3.0-4854-g8f219c47-1jammy 2024-09-10T23:31:33.016 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T23:31:33.017 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854-g8f219c47-1jammy is installed. 2024-09-10T23:31:33.018 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:31:33.170 DEBUG:teuthology.orchestra.run.smithi181:> dpkg-query -W -f '${Version}' ceph 2024-09-10T23:31:33.193 INFO:teuthology.orchestra.run.smithi181.stdout:19.3.0-4854-g8f219c47-1jammy 2024-09-10T23:31:33.193 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T23:31:33.193 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854-g8f219c47-1jammy is installed. 2024-09-10T23:31:33.195 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T23:31:33.195 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:31:33.195 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T23:31:33.217 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:31:33.217 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T23:31:33.256 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T23:31:33.257 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:31:33.257 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T23:31:33.283 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T23:31:33.342 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:31:33.342 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T23:31:33.362 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T23:31:33.423 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T23:31:33.423 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:31:33.423 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T23:31:33.442 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T23:31:33.502 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:31:33.503 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T23:31:33.522 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T23:31:33.583 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T23:31:33.583 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:31:33.583 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T23:31:33.602 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T23:31:33.662 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:31:33.662 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T23:31:33.682 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T23:31:33.743 INFO:teuthology.run_tasks:Running task ceph... 2024-09-10T23:31:33.843 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-10T23:31:33.844 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /var/log/ceph 2024-09-10T23:31:33.847 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /var/log/ceph 2024-09-10T23:31:33.866 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-10T23:31:33.867 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T23:31:33.911 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T23:31:33.931 INFO:tasks.ceph:Creating extra log directories... 2024-09-10T23:31:33.931 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T23:31:33.971 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T23:31:34.004 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-10T23:31:34.005 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-10T23:31:34.005 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-10T23:31:34.005 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:31:34.005 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T23:31:34.040 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T23:31:34.125 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:31:34.125 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T23:31:34.134 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T23:31:34.283 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-10T23:31:34.283 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-10T23:31:34.283 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899790', 'branch': 'wip-hemanth-testing-2024-09-10-0723', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}}', 'email': 'hyelloji@redhat.com', 'first_in_suite': False, 'job_id': '7899790', 'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-hemanth-testing-2024-09-10-0723'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-hemanth-testing-2024-09-10-0723', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'owner': 'scheduled_hyelloji@teuthology', 'priority': 100, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6190, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'sleep_before_teardown': 0, 'subset': '479/1024', 'suite': 'fs', 'suite_branch': 'wip-hemanth-testing-2024-09-10-0723', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi094.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHiZMHMb7Bu+JC8d8aMhd1mPsXSyPurOEKbBVPuiRmYuZnzwYV7vYkNWTUVMnbRgkiQhTnz0J+nzYi5VJSCfabw=', 'smithi181.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNKDdux8OkcgVO74ps3unEtezdTyoqM9GSd/vWinp22tgf5qlxMrAOcFUTbE0P5jRsi2OVR3js8SBLntcJU1YH8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_16:59:47', 'tube': 'smithi', 'user': 'hyelloji', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-10T23:31:34.284 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T23:31:34.287 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T23:31:34.301 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T23:31:34.339 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T23:31:34.368 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:31:34.368 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T23:31:34.408 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T23:31:34.408 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-09-10T23:31:34.456 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T23:31:34.457 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:31:34.457 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 896 Links: 1 2024-09-10T23:31:34.457 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:31:34.457 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-10 23:24:48.600952001 +0000 2024-09-10T23:31:34.457 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-10 23:24:48.596952037 +0000 2024-09-10T23:31:34.457 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-10 23:24:48.596952037 +0000 2024-09-10T23:31:34.457 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-10 23:24:48.596952037 +0000 2024-09-10T23:31:34.457 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T23:31:34.517 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T23:31:34.517 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T23:31:34.517 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000228072 s, 2.2 MB/s 2024-09-10T23:31:34.519 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T23:31:34.570 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-09-10T23:31:34.625 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T23:31:34.625 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:31:34.625 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 915 Links: 1 2024-09-10T23:31:34.625 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:31:34.625 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-10 23:24:49.412944749 +0000 2024-09-10T23:31:34.625 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-10 23:24:49.132947248 +0000 2024-09-10T23:31:34.625 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-10 23:24:49.132947248 +0000 2024-09-10T23:31:34.625 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-10 23:24:49.132947248 +0000 2024-09-10T23:31:34.625 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T23:31:34.685 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T23:31:34.685 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T23:31:34.685 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000263556 s, 1.9 MB/s 2024-09-10T23:31:34.687 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T23:31:34.742 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-09-10T23:31:34.792 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T23:31:34.792 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:31:34.792 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 930 Links: 1 2024-09-10T23:31:34.792 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:31:34.793 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-10 23:24:49.992939568 +0000 2024-09-10T23:31:34.793 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-10 23:24:49.724941961 +0000 2024-09-10T23:31:34.793 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-10 23:24:49.724941961 +0000 2024-09-10T23:31:34.793 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-10 23:24:49.724941961 +0000 2024-09-10T23:31:34.793 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T23:31:34.849 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T23:31:34.849 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T23:31:34.849 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000256083 s, 2.0 MB/s 2024-09-10T23:31:34.851 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T23:31:34.902 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-09-10T23:31:34.952 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T23:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 948 Links: 1 2024-09-10T23:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-10 23:24:50.532934744 +0000 2024-09-10T23:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-10 23:24:50.260937173 +0000 2024-09-10T23:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-10 23:24:50.260937173 +0000 2024-09-10T23:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-10 23:24:50.260937173 +0000 2024-09-10T23:31:34.953 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T23:31:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T23:31:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T23:31:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000249154 s, 2.1 MB/s 2024-09-10T23:31:35.015 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T23:31:35.070 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T23:31:35.070 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:31:35.070 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T23:31:35.079 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T23:31:35.079 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2024-09-10T23:31:35.133 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T23:31:35.133 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:31:35.133 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 880 Links: 1 2024-09-10T23:31:35.133 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:31:35.133 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-09-10 23:24:55.463673654 +0000 2024-09-10T23:31:35.133 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-09-10 23:24:55.459673688 +0000 2024-09-10T23:31:35.134 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-09-10 23:24:55.459673688 +0000 2024-09-10T23:31:35.134 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-09-10 23:24:55.459673688 +0000 2024-09-10T23:31:35.134 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T23:31:35.195 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-09-10T23:31:35.195 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-09-10T23:31:35.195 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.00068271 s, 750 kB/s 2024-09-10T23:31:35.197 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T23:31:35.255 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2024-09-10T23:31:35.305 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T23:31:35.305 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:31:35.305 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 896 Links: 1 2024-09-10T23:31:35.305 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:31:35.305 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-09-10 23:24:56.323665972 +0000 2024-09-10T23:31:35.305 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-09-10 23:24:56.055668366 +0000 2024-09-10T23:31:35.305 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-09-10 23:24:56.055668366 +0000 2024-09-10T23:31:35.305 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-09-10 23:24:56.055668366 +0000 2024-09-10T23:31:35.306 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T23:31:35.366 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-09-10T23:31:35.366 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-09-10T23:31:35.367 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000404384 s, 1.3 MB/s 2024-09-10T23:31:35.368 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T23:31:35.422 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2024-09-10T23:31:35.473 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T23:31:35.473 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:31:35.473 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 913 Links: 1 2024-09-10T23:31:35.473 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:31:35.473 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-09-10 23:24:56.615663363 +0000 2024-09-10T23:31:35.473 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-09-10 23:24:56.611663399 +0000 2024-09-10T23:31:35.473 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-09-10 23:24:56.611663399 +0000 2024-09-10T23:31:35.473 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-09-10 23:24:56.611663399 +0000 2024-09-10T23:31:35.473 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T23:31:35.534 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-09-10T23:31:35.535 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-09-10T23:31:35.535 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000381611 s, 1.3 MB/s 2024-09-10T23:31:35.536 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T23:31:35.591 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2024-09-10T23:31:35.641 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T23:31:35.641 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:31:35.641 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 933 Links: 1 2024-09-10T23:31:35.641 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:31:35.641 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-09-10 23:24:57.439656004 +0000 2024-09-10T23:31:35.641 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-09-10 23:24:57.155658539 +0000 2024-09-10T23:31:35.641 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-09-10 23:24:57.155658539 +0000 2024-09-10T23:31:35.641 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-09-10 23:24:57.155658539 +0000 2024-09-10T23:31:35.642 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T23:31:35.706 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-09-10T23:31:35.707 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-09-10T23:31:35.707 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000383187 s, 1.3 MB/s 2024-09-10T23:31:35.709 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T23:31:35.762 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T23:31:35.763 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T23:31:35.763 INFO:tasks.ceph:Generating config... 2024-09-10T23:31:35.764 INFO:tasks.ceph:[client] pid file = 2024-09-10T23:31:35.764 INFO:tasks.ceph:[mgr] client mount timeout = 30 2024-09-10T23:31:35.764 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-10T23:31:35.764 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-09-10T23:31:35.765 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-10T23:31:35.765 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-10T23:31:35.765 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-10T23:31:35.765 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-10T23:31:35.765 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-10T23:31:35.765 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-10T23:31:35.765 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-10T23:31:35.765 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-10T23:31:35.765 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-10T23:31:35.765 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-10T23:31:35.765 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-10T23:31:35.766 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-10T23:31:35.766 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-10T23:31:35.766 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-10T23:31:35.766 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-10T23:31:35.766 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-10T23:31:35.766 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-10T23:31:35.766 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-10T23:31:35.766 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-10T23:31:35.766 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-10T23:31:35.766 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-10T23:31:35.767 INFO:tasks.ceph:Setting up mon.a... 2024-09-10T23:31:35.767 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-10T23:31:35.811 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.keyring 2024-09-10T23:31:35.827 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-10T23:31:35.915 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T23:31:35.979 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.94'), ('mon.b', '172.21.15.181'), ('mon.c', '[v2:172.21.15.181:3301,v1:172.21.15.181:6790]')] 2024-09-10T23:31:35.980 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.94,172.21.15.181,[v2:172.21.15.181:3301,v1:172.21.15.181:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-10T23:31:35.981 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:31:35.981 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T23:31:36.032 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.94 --add b 172.21.15.181 --addv c '[v2:172.21.15.181:3301,v1:172.21.15.181:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-10T23:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-10T23:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: generated fsid 1798cdb9-9937-4918-a569-86b41a92817e 2024-09-10T23:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout:setting min_mon_release = quincy 2024-09-10T23:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 0 2024-09-10T23:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout:fsid 1798cdb9-9937-4918-a569-86b41a92817e 2024-09-10T23:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout:last_changed 2024-09-10T23:31:36.125189+0000 2024-09-10T23:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-09-10T23:31:36.125189+0000 2024-09-10T23:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout:min_mon_release 17 (quincy) 2024-09-10T23:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout:election_strategy: 1 2024-09-10T23:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout:0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-09-10T23:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout:1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2024-09-10T23:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout:2: [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] mon.c 2024-09-10T23:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-10T23:31:36.126 DEBUG:teuthology.orchestra.run.smithi094:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T23:31:36.134 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1798cdb9-9937-4918-a569-86b41a92817e... 2024-09-10T23:31:36.135 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T23:31:36.183 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T23:31:36.221 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-09-10T23:31:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: chdir = "" 2024-09-10T23:31:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T23:31:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: auth supported = cephx 2024-09-10T23:31:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: filestore xattr use omap = true 2024-09-10T23:31:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: mon clock drift allowed = 1.000 2024-09-10T23:31:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: osd crush chooseleaf type = 0 2024-09-10T23:31:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: auth debug = true 2024-09-10T23:31:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on old message = true 2024-09-10T23:31:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on bug = true 2024-09-10T23:31:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T23:31:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: mon pg warn max object skew = 0 2024-09-10T23:31:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T23:31:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T23:31:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default size = 2 2024-09-10T23:31:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow primary affinity = true 2024-09-10T23:31:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow pg remap = true 2024-09-10T23:31:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on legacy crush tunables = false 2024-09-10T23:31:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on crush straw calc version zero = false 2024-09-10T23:31:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on no sortbitwise = false 2024-09-10T23:31:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on osd down out interval zero = false 2024-09-10T23:31:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on too few osds = false 2024-09-10T23:31:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T23:31:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T23:31:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: mon_allow_pool_size_one = true 2024-09-10T23:31:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T23:31:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: osd default data pool replay window = 5 2024-09-10T23:31:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: mon allow pool delete = true 2024-09-10T23:31:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: mon cluster log file level = debug 2024-09-10T23:31:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: debug asserts on shutdown = true 2024-09-10T23:31:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: mon health detail to clog = false 2024-09-10T23:31:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: mon host = "172.21.15.94,172.21.15.181,[v2:172.21.15.181:3301,v1:172.21.15.181:6790]" 2024-09-10T23:31:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = 1798cdb9-9937-4918-a569-86b41a92817e 2024-09-10T23:31:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.225 INFO:teuthology.orchestra.run.smithi094.stdout:[osd] 2024-09-10T23:31:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: osd journal size = 100 2024-09-10T23:31:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub load threshold = 5.0 2024-09-10T23:31:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub max interval = 600 2024-09-10T23:31:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: osd mclock profile = high_recovery_ops 2024-09-10T23:31:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: osd recover clone overlap = true 2024-09-10T23:31:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: osd recovery max chunk = 1048576 2024-09-10T23:31:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug shutdown = true 2024-09-10T23:31:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug op order = true 2024-09-10T23:31:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug verify stray on activate = true 2024-09-10T23:31:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug trim objects = true 2024-09-10T23:31:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: osd open classes on start = true 2024-09-10T23:31:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug pg log writeout = true 2024-09-10T23:31:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: osd deep scrub update digest min age = 30 2024-09-10T23:31:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: osd map max advance = 10 2024-09-10T23:31:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: journal zero on create = true 2024-09-10T23:31:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: filestore ondisk finisher threads = 3 2024-09-10T23:31:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: filestore apply finisher threads = 3 2024-09-10T23:31:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: bdev debug aio = true 2024-09-10T23:31:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug misdirected ops = true 2024-09-10T23:31:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: bdev async discard = True 2024-09-10T23:31:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: bdev enable discard = True 2024-09-10T23:31:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore allocator = bitmap 2024-09-10T23:31:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore block size = 96636764160 2024-09-10T23:31:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore fsck on mount = True 2024-09-10T23:31:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluefs = 1/20 2024-09-10T23:31:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluestore = 1/20 2024-09-10T23:31:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T23:31:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: debug osd = 20 2024-09-10T23:31:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: debug rocksdb = 4/10 2024-09-10T23:31:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T23:31:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd full ratio = 0.9 2024-09-10T23:31:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd nearfull ratio = 0.8 2024-09-10T23:31:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: osd failsafe full ratio = 0.95 2024-09-10T23:31:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: osd objectstore = bluestore 2024-09-10T23:31:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.229 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr] 2024-09-10T23:31:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T23:31:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: debug mgr = 20 2024-09-10T23:31:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-10T23:31:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-10T23:31:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-10T23:31:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-10T23:31:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: mgr/telemetry/nag = false 2024-09-10T23:31:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: client mount timeout = 30 2024-09-10T23:31:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: debug client = 20 2024-09-10T23:31:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: debug finisher = 20 2024-09-10T23:31:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.229 INFO:teuthology.orchestra.run.smithi094.stdout:[mon] 2024-09-10T23:31:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T23:31:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-10T23:31:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: debug paxos = 20 2024-09-10T23:31:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-10T23:31:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: mon data avail warn = 5 2024-09-10T23:31:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: mon mgr mkfs grace = 240 2024-09-10T23:31:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-10T23:31:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd reporter subtree level = osd 2024-09-10T23:31:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd prime pg temp = true 2024-09-10T23:31:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-10T23:31:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T23:31:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T23:31:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T23:31:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T23:31:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T23:31:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T23:31:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: # 1m isn't quite enough 2024-09-10T23:31:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: mon_down_mkfs_grace = 2m 2024-09-10T23:31:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_filestore_osds = false 2024-09-10T23:31:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: mon down mkfs grace = 300 2024-09-10T23:31:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:31:36.231 INFO:teuthology.orchestra.run.smithi094.stdout:[client] 2024-09-10T23:31:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: rgw cache enabled = true 2024-09-10T23:31:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable ops log = true 2024-09-10T23:31:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable usage log = true 2024-09-10T23:31:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T23:31:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T23:31:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = "" 2024-09-10T23:31:36.232 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.a] 2024-09-10T23:31:36.232 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.b] 2024-09-10T23:31:36.232 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.c] 2024-09-10T23:31:36.232 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2024-09-10T23:31:36.232 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2024-09-10T23:31:36.232 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2024-09-10T23:31:36.232 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2024-09-10T23:31:36.233 INFO:teuthology.orchestra.run.smithi181.stdout:[global] 2024-09-10T23:31:36.233 INFO:teuthology.orchestra.run.smithi181.stdout: chdir = "" 2024-09-10T23:31:36.233 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T23:31:36.233 INFO:teuthology.orchestra.run.smithi181.stdout: auth supported = cephx 2024-09-10T23:31:36.233 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.233 INFO:teuthology.orchestra.run.smithi181.stdout: filestore xattr use omap = true 2024-09-10T23:31:36.233 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.233 INFO:teuthology.orchestra.run.smithi181.stdout: mon clock drift allowed = 1.000 2024-09-10T23:31:36.233 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.233 INFO:teuthology.orchestra.run.smithi181.stdout: osd crush chooseleaf type = 0 2024-09-10T23:31:36.234 INFO:teuthology.orchestra.run.smithi181.stdout: auth debug = true 2024-09-10T23:31:36.234 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.234 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on old message = true 2024-09-10T23:31:36.234 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on bug = true 2024-09-10T23:31:36.234 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.234 INFO:teuthology.orchestra.run.smithi181.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T23:31:36.234 INFO:teuthology.orchestra.run.smithi181.stdout: mon pg warn max object skew = 0 2024-09-10T23:31:36.234 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.234 INFO:teuthology.orchestra.run.smithi181.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T23:31:36.234 INFO:teuthology.orchestra.run.smithi181.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T23:31:36.234 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.234 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default size = 2 2024-09-10T23:31:36.234 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.234 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow primary affinity = true 2024-09-10T23:31:36.235 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow pg remap = true 2024-09-10T23:31:36.235 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on legacy crush tunables = false 2024-09-10T23:31:36.235 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on crush straw calc version zero = false 2024-09-10T23:31:36.235 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on no sortbitwise = false 2024-09-10T23:31:36.235 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on osd down out interval zero = false 2024-09-10T23:31:36.235 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on too few osds = false 2024-09-10T23:31:36.235 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T23:31:36.235 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T23:31:36.235 INFO:teuthology.orchestra.run.smithi181.stdout: mon_allow_pool_size_one = true 2024-09-10T23:31:36.235 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.235 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T23:31:36.235 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.235 INFO:teuthology.orchestra.run.smithi181.stdout: osd default data pool replay window = 5 2024-09-10T23:31:36.236 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.236 INFO:teuthology.orchestra.run.smithi181.stdout: mon allow pool delete = true 2024-09-10T23:31:36.236 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.236 INFO:teuthology.orchestra.run.smithi181.stdout: mon cluster log file level = debug 2024-09-10T23:31:36.236 INFO:teuthology.orchestra.run.smithi181.stdout: debug asserts on shutdown = true 2024-09-10T23:31:36.236 INFO:teuthology.orchestra.run.smithi181.stdout: mon health detail to clog = false 2024-09-10T23:31:36.236 INFO:teuthology.orchestra.run.smithi181.stdout: mon host = "172.21.15.94,172.21.15.181,[v2:172.21.15.181:3301,v1:172.21.15.181:6790]" 2024-09-10T23:31:36.236 INFO:teuthology.orchestra.run.smithi181.stdout: fsid = 1798cdb9-9937-4918-a569-86b41a92817e 2024-09-10T23:31:36.236 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.236 INFO:teuthology.orchestra.run.smithi181.stdout:[osd] 2024-09-10T23:31:36.236 INFO:teuthology.orchestra.run.smithi181.stdout: osd journal size = 100 2024-09-10T23:31:36.236 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.236 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub load threshold = 5.0 2024-09-10T23:31:36.236 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub max interval = 600 2024-09-10T23:31:36.237 INFO:teuthology.orchestra.run.smithi181.stdout: osd mclock profile = high_recovery_ops 2024-09-10T23:31:36.237 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.237 INFO:teuthology.orchestra.run.smithi181.stdout: osd recover clone overlap = true 2024-09-10T23:31:36.237 INFO:teuthology.orchestra.run.smithi181.stdout: osd recovery max chunk = 1048576 2024-09-10T23:31:36.237 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.237 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug shutdown = true 2024-09-10T23:31:36.237 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug op order = true 2024-09-10T23:31:36.237 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug verify stray on activate = true 2024-09-10T23:31:36.237 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug trim objects = true 2024-09-10T23:31:36.237 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.237 INFO:teuthology.orchestra.run.smithi181.stdout: osd open classes on start = true 2024-09-10T23:31:36.237 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug pg log writeout = true 2024-09-10T23:31:36.237 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.237 INFO:teuthology.orchestra.run.smithi181.stdout: osd deep scrub update digest min age = 30 2024-09-10T23:31:36.238 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.238 INFO:teuthology.orchestra.run.smithi181.stdout: osd map max advance = 10 2024-09-10T23:31:36.238 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.238 INFO:teuthology.orchestra.run.smithi181.stdout: journal zero on create = true 2024-09-10T23:31:36.238 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.238 INFO:teuthology.orchestra.run.smithi181.stdout: filestore ondisk finisher threads = 3 2024-09-10T23:31:36.238 INFO:teuthology.orchestra.run.smithi181.stdout: filestore apply finisher threads = 3 2024-09-10T23:31:36.238 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.238 INFO:teuthology.orchestra.run.smithi181.stdout: bdev debug aio = true 2024-09-10T23:31:36.238 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug misdirected ops = true 2024-09-10T23:31:36.238 INFO:teuthology.orchestra.run.smithi181.stdout: bdev async discard = True 2024-09-10T23:31:36.238 INFO:teuthology.orchestra.run.smithi181.stdout: bdev enable discard = True 2024-09-10T23:31:36.238 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore allocator = bitmap 2024-09-10T23:31:36.238 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore block size = 96636764160 2024-09-10T23:31:36.238 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore fsck on mount = True 2024-09-10T23:31:36.238 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluefs = 1/20 2024-09-10T23:31:36.239 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluestore = 1/20 2024-09-10T23:31:36.239 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2024-09-10T23:31:36.239 INFO:teuthology.orchestra.run.smithi181.stdout: debug osd = 20 2024-09-10T23:31:36.239 INFO:teuthology.orchestra.run.smithi181.stdout: debug rocksdb = 4/10 2024-09-10T23:31:36.239 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T23:31:36.239 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd full ratio = 0.9 2024-09-10T23:31:36.239 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd nearfull ratio = 0.8 2024-09-10T23:31:36.239 INFO:teuthology.orchestra.run.smithi181.stdout: osd failsafe full ratio = 0.95 2024-09-10T23:31:36.239 INFO:teuthology.orchestra.run.smithi181.stdout: osd objectstore = bluestore 2024-09-10T23:31:36.239 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.239 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr] 2024-09-10T23:31:36.239 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2024-09-10T23:31:36.239 INFO:teuthology.orchestra.run.smithi181.stdout: debug mgr = 20 2024-09-10T23:31:36.239 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2024-09-10T23:31:36.239 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2024-09-10T23:31:36.239 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2024-09-10T23:31:36.240 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2024-09-10T23:31:36.240 INFO:teuthology.orchestra.run.smithi181.stdout: mgr/telemetry/nag = false 2024-09-10T23:31:36.240 INFO:teuthology.orchestra.run.smithi181.stdout: client mount timeout = 30 2024-09-10T23:31:36.240 INFO:teuthology.orchestra.run.smithi181.stdout: debug client = 20 2024-09-10T23:31:36.240 INFO:teuthology.orchestra.run.smithi181.stdout: debug finisher = 20 2024-09-10T23:31:36.240 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.240 INFO:teuthology.orchestra.run.smithi181.stdout:[mon] 2024-09-10T23:31:36.240 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2024-09-10T23:31:36.240 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2024-09-10T23:31:36.240 INFO:teuthology.orchestra.run.smithi181.stdout: debug paxos = 20 2024-09-10T23:31:36.240 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2024-09-10T23:31:36.240 INFO:teuthology.orchestra.run.smithi181.stdout: mon data avail warn = 5 2024-09-10T23:31:36.240 INFO:teuthology.orchestra.run.smithi181.stdout: mon mgr mkfs grace = 240 2024-09-10T23:31:36.240 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2024-09-10T23:31:36.240 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd reporter subtree level = osd 2024-09-10T23:31:36.241 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd prime pg temp = true 2024-09-10T23:31:36.241 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2024-09-10T23:31:36.241 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.241 INFO:teuthology.orchestra.run.smithi181.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T23:31:36.241 INFO:teuthology.orchestra.run.smithi181.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T23:31:36.241 INFO:teuthology.orchestra.run.smithi181.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T23:31:36.241 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.241 INFO:teuthology.orchestra.run.smithi181.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T23:31:36.241 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T23:31:36.241 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T23:31:36.241 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.241 INFO:teuthology.orchestra.run.smithi181.stdout: # 1m isn't quite enough 2024-09-10T23:31:36.241 INFO:teuthology.orchestra.run.smithi181.stdout: mon_down_mkfs_grace = 2m 2024-09-10T23:31:36.241 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.241 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_filestore_osds = false 2024-09-10T23:31:36.242 INFO:teuthology.orchestra.run.smithi181.stdout: mon down mkfs grace = 300 2024-09-10T23:31:36.242 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-10T23:31:36.242 INFO:teuthology.orchestra.run.smithi181.stdout:[client] 2024-09-10T23:31:36.242 INFO:teuthology.orchestra.run.smithi181.stdout: rgw cache enabled = true 2024-09-10T23:31:36.242 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable ops log = true 2024-09-10T23:31:36.242 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable usage log = true 2024-09-10T23:31:36.242 INFO:teuthology.orchestra.run.smithi181.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T23:31:36.242 INFO:teuthology.orchestra.run.smithi181.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T23:31:36.242 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = "" 2024-09-10T23:31:36.242 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.a] 2024-09-10T23:31:36.242 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.b] 2024-09-10T23:31:36.242 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.c] 2024-09-10T23:31:36.242 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.a] 2024-09-10T23:31:36.242 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.c] 2024-09-10T23:31:36.243 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.b] 2024-09-10T23:31:36.243 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.d] 2024-09-10T23:31:36.248 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-10T23:31:36.248 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-10T23:31:36.341 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-10T23:31:36.341 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:31:36.341 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-10T23:31:36.350 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:31:36.350 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-10T23:31:36.400 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:31:36.400 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:31:36.400 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T23:31:36.400 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T23:31:36.471 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:31:36.471 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T23:31:36.525 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:31:36.525 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:31:36.525 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T23:31:36.526 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T23:31:36.560 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:31:36.560 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T23:31:36.613 INFO:tasks.ceph:Setting up mon nodes... 2024-09-10T23:31:36.613 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-10T23:31:36.613 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T23:31:36.667 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T23:31:36.673 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T23:31:36.733 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T23:31:36.744 INFO:tasks.ceph:Setting up mds nodes... 2024-09-10T23:31:36.744 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T23:31:36.798 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T23:31:36.803 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-10T23:31:36.822 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T23:31:36.921 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T23:31:36.925 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-10T23:31:36.944 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T23:31:37.003 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T23:31:37.008 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-10T23:31:37.028 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T23:31:37.127 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T23:31:37.131 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-10T23:31:37.152 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-10T23:31:37.152 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T23:31:37.194 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-10T23:31:37.210 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-10T23:31:37.294 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-10T23:31:37.310 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T23:31:37.352 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-10T23:31:37.368 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-10T23:31:37.453 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-10T23:31:37.470 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-10T23:31:37.472 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T23:31:37.472 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-10T23:31:37.491 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T23:31:37.491 INFO:tasks.ceph:role: osd.0 2024-09-10T23:31:37.491 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:31:37.491 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T23:31:37.550 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T23:31:37.550 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T23:31:37.550 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T23:31:37.550 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T23:31:37.550 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T23:31:37.551 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-10T23:31:37.551 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T23:31:37.551 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T23:31:37.551 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T23:31:37.551 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T23:31:37.580 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-10T23:31:37.584 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-10T23:31:37.584 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-10T23:31:37.649 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-10T23:31:37.711 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T23:31:37.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:31:37.712 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-10T23:31:37.771 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T23:31:37.771 INFO:tasks.ceph:role: osd.1 2024-09-10T23:31:37.771 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:31:37.772 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T23:31:37.830 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T23:31:37.830 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T23:31:37.830 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T23:31:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T23:31:37.831 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T23:31:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-10T23:31:37.831 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T23:31:37.831 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T23:31:37.831 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T23:31:37.831 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T23:31:37.859 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-10T23:31:37.861 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-10T23:31:37.861 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-10T23:31:37.928 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-10T23:31:37.984 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T23:31:37.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:31:37.985 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-10T23:31:38.046 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T23:31:38.046 INFO:tasks.ceph:role: osd.2 2024-09-10T23:31:38.047 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:31:38.047 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T23:31:38.109 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T23:31:38.109 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T23:31:38.109 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T23:31:38.110 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T23:31:38.110 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T23:31:38.110 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-10T23:31:38.110 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T23:31:38.110 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T23:31:38.110 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T23:31:38.110 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T23:31:38.138 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-10T23:31:38.141 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-10T23:31:38.141 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-10T23:31:38.209 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-10T23:31:38.271 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T23:31:38.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:31:38.273 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-10T23:31:38.335 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T23:31:38.335 INFO:tasks.ceph:role: osd.3 2024-09-10T23:31:38.335 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:31:38.335 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T23:31:38.395 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T23:31:38.395 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T23:31:38.395 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T23:31:38.395 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T23:31:38.395 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T23:31:38.395 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-10T23:31:38.395 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T23:31:38.395 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T23:31:38.395 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T23:31:38.395 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T23:31:38.423 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-10T23:31:38.426 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-10T23:31:38.426 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-10T23:31:38.492 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-10T23:31:38.555 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T23:31:38.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:31:38.557 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T23:31:38.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:31:38.679+0000 7f401cad97c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-10T23:31:38.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:31:38.683+0000 7f401cad97c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-10T23:31:38.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:31:38.683+0000 7f401cad97c0 -1 bdev(0x55aaad23b800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-10T23:31:38.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:31:38.683+0000 7f401cad97c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-10T23:31:41.903 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T23:31:41.922 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T23:31:42.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:31:42.011+0000 7f93d0c2b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-10T23:31:42.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:31:42.011+0000 7f93d0c2b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-10T23:31:42.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:31:42.011+0000 7f93d0c2b7c0 -1 bdev(0x55a1e85e3800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-10T23:31:42.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:31:42.011+0000 7f93d0c2b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-10T23:31:45.255 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T23:31:45.276 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T23:31:45.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:31:45.363+0000 7f621b8c77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-10T23:31:45.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:31:45.363+0000 7f621b8c77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-10T23:31:45.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:31:45.363+0000 7f621b8c77c0 -1 bdev(0x561b19abd800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-10T23:31:45.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:31:45.363+0000 7f621b8c77c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-10T23:31:48.578 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T23:31:48.598 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T23:31:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:31:48.687+0000 7f4a003887c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-10T23:31:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:31:48.687+0000 7f4a003887c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-10T23:31:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:31:48.687+0000 7f4a003887c0 -1 bdev(0x55afcb953800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-10T23:31:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:31:48.687+0000 7f4a003887c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-10T23:31:51.935 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T23:31:51.956 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-10T23:31:51.978 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T23:31:51.978 INFO:tasks.ceph:role: osd.4 2024-09-10T23:31:51.978 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:31:51.978 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T23:31:52.048 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T23:31:52.048 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T23:31:52.048 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T23:31:52.048 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T23:31:52.049 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T23:31:52.049 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2024-09-10T23:31:52.049 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T23:31:52.049 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T23:31:52.049 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T23:31:52.049 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T23:31:52.451 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2024-09-10T23:31:52.454 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2024-09-10T23:31:52.455 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-10T23:31:52.546 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-10T23:31:52.563 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T23:31:52.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:31:52.564 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-10T23:31:52.632 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T23:31:52.632 INFO:tasks.ceph:role: osd.5 2024-09-10T23:31:52.632 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:31:52.632 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T23:31:52.708 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T23:31:52.708 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T23:31:52.708 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T23:31:52.708 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T23:31:52.708 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T23:31:52.708 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2024-09-10T23:31:52.708 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T23:31:52.709 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T23:31:52.709 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T23:31:52.709 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T23:31:53.104 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2024-09-10T23:31:53.107 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2024-09-10T23:31:53.107 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-10T23:31:53.247 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-10T23:31:53.265 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T23:31:53.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:31:53.266 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-10T23:31:53.329 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T23:31:53.329 INFO:tasks.ceph:role: osd.6 2024-09-10T23:31:53.329 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:31:53.329 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T23:31:53.400 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T23:31:53.400 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T23:31:53.401 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T23:31:53.401 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T23:31:53.401 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T23:31:53.401 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2024-09-10T23:31:53.401 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T23:31:53.401 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T23:31:53.401 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T23:31:53.401 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T23:31:53.824 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2024-09-10T23:31:53.827 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2024-09-10T23:31:53.827 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-10T23:31:53.964 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-10T23:31:53.981 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T23:31:53.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:31:53.982 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-10T23:31:54.043 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T23:31:54.044 INFO:tasks.ceph:role: osd.7 2024-09-10T23:31:54.044 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:31:54.044 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T23:31:54.114 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T23:31:54.115 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T23:31:54.115 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T23:31:54.115 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T23:31:54.115 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T23:31:54.115 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2024-09-10T23:31:54.115 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T23:31:54.115 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T23:31:54.115 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T23:31:54.115 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T23:31:54.480 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2024-09-10T23:31:54.483 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2024-09-10T23:31:54.483 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-10T23:31:54.560 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-10T23:31:54.620 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T23:31:54.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:31:54.621 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T23:31:54.773 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:31:54.773+0000 7f983cf297c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-10T23:31:54.775 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:31:54.773+0000 7f983cf297c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-10T23:31:54.775 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:31:54.773+0000 7f983cf297c0 -1 bdev(0x55fbc2cab800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-10T23:31:54.775 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:31:54.773+0000 7f983cf297c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-10T23:31:58.015 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T23:31:58.036 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T23:31:58.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:31:58.129+0000 7f3cf497e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-10T23:31:58.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:31:58.133+0000 7f3cf497e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-10T23:31:58.133 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:31:58.133+0000 7f3cf497e7c0 -1 bdev(0x5653cd22d800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-10T23:31:58.136 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:31:58.133+0000 7f3cf497e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-10T23:32:01.380 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T23:32:01.402 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T23:32:01.502 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:01.501+0000 7f1eebeb27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-10T23:32:01.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:01.501+0000 7f1eebeb27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-10T23:32:01.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:01.501+0000 7f1eebeb27c0 -1 bdev(0x55f07c4ad800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-10T23:32:01.503 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:01.501+0000 7f1eebeb27c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-10T23:32:04.285 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:32:04.289 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:32:04.742 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-10T23:32:04.761 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T23:32:04.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:04.857+0000 7fc82ed617c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-10T23:32:04.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:04.857+0000 7fc82ed617c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-10T23:32:04.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:04.857+0000 7fc82ed617c0 -1 bdev(0x555c7d327800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-10T23:32:04.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:04.857+0000 7fc82ed617c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-10T23:32:08.114 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T23:32:08.134 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-10T23:32:08.135 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:32:08.135 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-10T23:32:08.157 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:32:08.157 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-10T23:32:08.221 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:32:08.221 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-10T23:32:08.285 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:32:08.285 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-10T23:32:08.349 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:32:08.349 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-10T23:32:08.413 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:32:08.413 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-10T23:32:08.476 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:32:08.477 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-10T23:32:08.540 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:32:08.540 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-10T23:32:08.562 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:32:08.562 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-10T23:32:08.628 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:32:08.628 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-10T23:32:08.692 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:32:08.692 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-10T23:32:08.760 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:32:08.760 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-10T23:32:08.828 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:32:08.828 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-10T23:32:08.892 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:32:08.892 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-10T23:32:08.956 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:32:08.956 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-10T23:32:08.965 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:32:08.965 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-10T23:32:09.021 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:32:09.021 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-10T23:32:09.029 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:32:09.029 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-10T23:32:09.080 INFO:tasks.ceph:Adding keys to all mons... 2024-09-10T23:32:09.080 DEBUG:teuthology.orchestra.run.smithi094:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T23:32:09.083 DEBUG:teuthology.orchestra.run.smithi181:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T23:32:09.128 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.y] 2024-09-10T23:32:09.128 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDY1uBmhHvtJxAACe+79nYhv2ODwtmNhA45FA== 2024-09-10T23:32:09.128 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2024-09-10T23:32:09.128 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDY1uBm/2m3LxAAr0ErpLJg+0+I2r74zQya4w== 2024-09-10T23:32:09.128 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2024-09-10T23:32:09.128 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDY1uBmk98KNxAA44e9N4VHbw/94Lh1V9zugQ== 2024-09-10T23:32:09.128 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.0] 2024-09-10T23:32:09.128 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDa1uBmSAD3KBAAJTF1OEC44Ml599aL7iEuDw== 2024-09-10T23:32:09.128 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.1] 2024-09-10T23:32:09.128 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDe1uBma1L9ABAAfEjW1/Hb4fh4Kwv/ZUgFAQ== 2024-09-10T23:32:09.128 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.2] 2024-09-10T23:32:09.128 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDh1uBmtLUQFhAAweeFHj6BRQVC9BX+VLiWEQ== 2024-09-10T23:32:09.129 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.3] 2024-09-10T23:32:09.129 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDk1uBm1KFCKRAAev28iYcrEYOViX95uo6Kbw== 2024-09-10T23:32:09.129 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.x] 2024-09-10T23:32:09.129 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDY1uBm/EPVKxAA/Myv8Zjf7BejbzlkxyCC1w== 2024-09-10T23:32:09.129 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2024-09-10T23:32:09.129 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBmc8pYABAAw90zu8D+9IwK15h4HIlGBA== 2024-09-10T23:32:09.129 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2024-09-10T23:32:09.129 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBmVAKzBxAAJehOw6aphTF/g3ucoQfw9A== 2024-09-10T23:32:09.129 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.4] 2024-09-10T23:32:09.129 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDq1uBm/Hg9LhAAvB2D0rj4DeRPxTrGmXYIsA== 2024-09-10T23:32:09.129 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.5] 2024-09-10T23:32:09.129 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDu1uBm1fAKCBAAFHMjnGFAItLdaACF/UjE/w== 2024-09-10T23:32:09.129 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.6] 2024-09-10T23:32:09.129 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDx1uBmXgYfHhAA/6wwtM4mDRXsIpoTIHKJ0w== 2024-09-10T23:32:09.130 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.7] 2024-09-10T23:32:09.130 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQD01uBmw6hQMxAAk7oGAg+iIplNK5KrokGwNA== 2024-09-10T23:32:09.130 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-10T23:32:09.130 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g== 2024-09-10T23:32:09.130 INFO:teuthology.orchestra.run.smithi094.stdout:[client.2] 2024-09-10T23:32:09.130 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBmfvaqERAASpL0KtglFcAuRf4a/0z5JQ== 2024-09-10T23:32:09.130 INFO:teuthology.orchestra.run.smithi094.stdout:[client.1] 2024-09-10T23:32:09.130 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBmuSkhFRAA1R6WNXLImkGfhW9/XJzEAQ== 2024-09-10T23:32:09.130 INFO:teuthology.orchestra.run.smithi094.stdout:[client.3] 2024-09-10T23:32:09.130 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBm8xUtGxAARc0ywfAhMxBNd57TzBoV3g== 2024-09-10T23:32:09.142 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.y] 2024-09-10T23:32:09.142 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDY1uBmhHvtJxAACe+79nYhv2ODwtmNhA45FA== 2024-09-10T23:32:09.142 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.a] 2024-09-10T23:32:09.142 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDY1uBm/2m3LxAAr0ErpLJg+0+I2r74zQya4w== 2024-09-10T23:32:09.142 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.c] 2024-09-10T23:32:09.142 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDY1uBmk98KNxAA44e9N4VHbw/94Lh1V9zugQ== 2024-09-10T23:32:09.142 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.0] 2024-09-10T23:32:09.142 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDa1uBmSAD3KBAAJTF1OEC44Ml599aL7iEuDw== 2024-09-10T23:32:09.142 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.1] 2024-09-10T23:32:09.143 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDe1uBma1L9ABAAfEjW1/Hb4fh4Kwv/ZUgFAQ== 2024-09-10T23:32:09.143 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.2] 2024-09-10T23:32:09.143 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDh1uBmtLUQFhAAweeFHj6BRQVC9BX+VLiWEQ== 2024-09-10T23:32:09.143 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.3] 2024-09-10T23:32:09.143 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDk1uBm1KFCKRAAev28iYcrEYOViX95uo6Kbw== 2024-09-10T23:32:09.143 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.x] 2024-09-10T23:32:09.143 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDY1uBm/EPVKxAA/Myv8Zjf7BejbzlkxyCC1w== 2024-09-10T23:32:09.143 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.b] 2024-09-10T23:32:09.143 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDZ1uBmc8pYABAAw90zu8D+9IwK15h4HIlGBA== 2024-09-10T23:32:09.143 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.d] 2024-09-10T23:32:09.143 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDZ1uBmVAKzBxAAJehOw6aphTF/g3ucoQfw9A== 2024-09-10T23:32:09.143 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.4] 2024-09-10T23:32:09.143 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDq1uBm/Hg9LhAAvB2D0rj4DeRPxTrGmXYIsA== 2024-09-10T23:32:09.144 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.5] 2024-09-10T23:32:09.144 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDu1uBm1fAKCBAAFHMjnGFAItLdaACF/UjE/w== 2024-09-10T23:32:09.144 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.6] 2024-09-10T23:32:09.144 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDx1uBmXgYfHhAA/6wwtM4mDRXsIpoTIHKJ0w== 2024-09-10T23:32:09.144 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.7] 2024-09-10T23:32:09.144 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQD01uBmw6hQMxAAk7oGAg+iIplNK5KrokGwNA== 2024-09-10T23:32:09.144 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2024-09-10T23:32:09.144 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g== 2024-09-10T23:32:09.144 INFO:teuthology.orchestra.run.smithi181.stdout:[client.2] 2024-09-10T23:32:09.144 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDZ1uBmfvaqERAASpL0KtglFcAuRf4a/0z5JQ== 2024-09-10T23:32:09.144 INFO:teuthology.orchestra.run.smithi181.stdout:[client.1] 2024-09-10T23:32:09.144 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDZ1uBmuSkhFRAA1R6WNXLImkGfhW9/XJzEAQ== 2024-09-10T23:32:09.144 INFO:teuthology.orchestra.run.smithi181.stdout:[client.3] 2024-09-10T23:32:09.144 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDZ1uBm8xUtGxAARc0ywfAhMxBNd57TzBoV3g== 2024-09-10T23:32:09.145 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T23:32:09.180 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T23:32:09.246 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T23:32:09.249 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T23:32:09.308 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T23:32:09.312 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T23:32:09.368 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T23:32:09.371 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T23:32:09.424 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T23:32:09.427 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T23:32:09.480 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T23:32:09.483 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T23:32:09.538 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T23:32:09.541 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T23:32:09.594 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T23:32:09.597 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T23:32:09.651 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T23:32:09.654 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T23:32:09.710 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T23:32:09.713 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T23:32:09.769 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T23:32:09.773 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T23:32:09.825 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T23:32:09.828 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T23:32:09.881 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T23:32:09.884 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T23:32:09.937 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T23:32:09.940 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T23:32:09.993 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T23:32:09.996 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T23:32:10.052 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T23:32:10.055 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T23:32:10.112 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T23:32:10.115 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T23:32:10.171 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T23:32:10.174 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T23:32:10.231 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-10T23:32:10.231 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-10T23:32:10.250 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T23:32:10.358 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-10T23:32:10.377 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-10T23:32:10.396 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T23:32:10.509 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-10T23:32:10.528 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-10T23:32:10.590 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T23:32:10.697 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-10T23:32:10.716 DEBUG:teuthology.orchestra.run.smithi094:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T23:32:10.719 DEBUG:teuthology.orchestra.run.smithi181:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T23:32:10.771 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-10T23:32:10.771 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T23:32:10.772 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T23:32:10.774 INFO:tasks.ceph.mon.a:Started 2024-09-10T23:32:10.775 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T23:32:10.775 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T23:32:10.816 INFO:tasks.ceph.mon.b:Started 2024-09-10T23:32:10.816 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T23:32:10.816 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T23:32:10.819 INFO:tasks.ceph.mon.c:Started 2024-09-10T23:32:10.819 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-10T23:32:10.819 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-10T23:32:10.819 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-10T23:32:10.822 INFO:tasks.ceph.mgr.y:Started 2024-09-10T23:32:10.823 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-10T23:32:10.823 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-10T23:32:10.827 INFO:tasks.ceph.mgr.x:Started 2024-09-10T23:32:10.827 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-10T23:32:10.827 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-10T23:32:10.830 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-10T23:32:10.830 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-10T23:32:10.875 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-10T23:32:10.875 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-10T23:32:10.920 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-10T23:32:10.920 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-10T23:32:10.968 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-10T23:32:10.968 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-10T23:32:11.015 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-10T23:32:11.015 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-10T23:32:11.059 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-10T23:32:11.059 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-10T23:32:11.104 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-10T23:32:11.104 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-10T23:32:11.152 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-10T23:32:11.152 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-10T23:32:11.203 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-10T23:32:11.203 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-10T23:32:11.251 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-10T23:32:11.251 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-10T23:32:11.295 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-10T23:32:11.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-10T23:32:11.340 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-10T23:32:11.340 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-10T23:32:11.392 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-10T23:32:11.392 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-10T23:32:11.439 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-10T23:32:11.439 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-10T23:32:11.484 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-10T23:32:11.484 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-10T23:32:11.532 INFO:tasks.ceph:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-10T23:32:11.532 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set mon mon_warn_on_pool_no_app False 2024-09-10T23:32:11.580 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-10T23:32:11.581 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-10T23:32:11.628 DEBUG:tasks.ceph:set 18 configs 2024-09-10T23:32:11.628 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi094') 2024-09-10T23:32:16.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.390+0000 7f65f1600640 1 -- 172.21.15.94:0/3462558545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65dc0072d0 msgr2=0x7f65fc0b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:16.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.390+0000 7f65f1600640 1 --2- 172.21.15.94:0/3462558545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65dc0072d0 0x7f65fc0b6000 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f65ec099d00 tx=0x7f65e0005840 comp rx=0 tx=0).stop 2024-09-10T23:32:16.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.390+0000 7f65f1600640 1 -- 172.21.15.94:0/3462558545 shutdown_connections 2024-09-10T23:32:16.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.390+0000 7f65f1600640 1 --2- 172.21.15.94:0/3462558545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65fc0c4730 0x7f65fc0c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:16.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.390+0000 7f65f1600640 1 --2- 172.21.15.94:0/3462558545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e0004e70 0x7f65fc0b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:16.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.390+0000 7f65f1600640 1 --2- 172.21.15.94:0/3462558545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65dc0072d0 0x7f65fc0b6000 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:16.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.390+0000 7f65f1600640 1 -- 172.21.15.94:0/3462558545 >> 172.21.15.94:0/3462558545 conn(0x7f65fc0d0c00 msgr2=0x7f65fc15ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:16.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.390+0000 7f65f1600640 1 -- 172.21.15.94:0/3462558545 shutdown_connections 2024-09-10T23:32:16.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.390+0000 7f65f1600640 1 -- 172.21.15.94:0/3462558545 wait complete. 2024-09-10T23:32:16.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.402+0000 7fb878600640 1 -- 172.21.15.94:0/1839406779 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7fb8440050f0 con 0x7fb858004e70 2024-09-10T23:32:16.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.406+0000 7fb86d600640 1 -- 172.21.15.94:0/1839406779 <== mon.1 v2:172.21.15.181:3300/0 9 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7fb85c024c80 con 0x7fb858004e70 2024-09-10T23:32:16.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.406+0000 7fb86d600640 1 -- 172.21.15.94:0/1839406779 <== mon.1 v2:172.21.15.181:3300/0 10 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v12)=0 v12) ==== 114+0+0 (secure 0 0 0) 0x7fb85c01a450 con 0x7fb858004e70 2024-09-10T23:32:16.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.406+0000 7f9636a00640 1 -- 172.21.15.94:0/3638676232 <== mon.1 v2:172.21.15.181:3300/0 9 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7f9630003600 con 0x7f9624003ab0 2024-09-10T23:32:16.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.406+0000 7f9645c00640 1 -- 172.21.15.94:0/3638676232 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f96100050f0 con 0x7f9624003ab0 2024-09-10T23:32:16.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.406+0000 7fb878600640 1 -- 172.21.15.94:0/1839406779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb858004e70 msgr2=0x7fb8700cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:16.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.406+0000 7fb878600640 1 --2- 172.21.15.94:0/1839406779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb858004e70 0x7fb8700cb5b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb85c004fd0 tx=0x7fb85c00ea20 comp rx=0 tx=0).stop 2024-09-10T23:32:16.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.406+0000 7fb878600640 1 -- 172.21.15.94:0/1839406779 shutdown_connections 2024-09-10T23:32:16.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.406+0000 7fb878600640 1 --2- 172.21.15.94:0/1839406779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb858004e70 0x7fb8700cb5b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:16.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.406+0000 7fb878600640 1 --2- 172.21.15.94:0/1839406779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8700ce890 0x7fb87009d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:16.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.406+0000 7fb878600640 1 --2- 172.21.15.94:0/1839406779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb858003a60 0x7fb87015d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:16.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.406+0000 7fb878600640 1 -- 172.21.15.94:0/1839406779 >> 172.21.15.94:0/1839406779 conn(0x7fb8700d1c60 msgr2=0x7fb870071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:16.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.406+0000 7fb878600640 1 -- 172.21.15.94:0/1839406779 shutdown_connections 2024-09-10T23:32:16.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.406+0000 7fb878600640 1 -- 172.21.15.94:0/1839406779 wait complete. 2024-09-10T23:32:16.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.410+0000 7f9636a00640 1 -- 172.21.15.94:0/3638676232 <== mon.1 v2:172.21.15.181:3300/0 10 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v13)=0 v13) ==== 120+0+0 (secure 0 0 0) 0x7f9630003920 con 0x7f9624003ab0 2024-09-10T23:32:16.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.410+0000 7f9645c00640 1 -- 172.21.15.94:0/3638676232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9624003ab0 msgr2=0x7f96401f5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:16.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.410+0000 7f9645c00640 1 --2- 172.21.15.94:0/3638676232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9624003ab0 0x7f96401f5660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9630005f20 tx=0x7f9630013f10 comp rx=0 tx=0).stop 2024-09-10T23:32:16.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.414+0000 7f9645c00640 1 -- 172.21.15.94:0/3638676232 shutdown_connections 2024-09-10T23:32:16.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.414+0000 7f9645c00640 1 --2- 172.21.15.94:0/3638676232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9624005420 0x7f96400b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:16.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.414+0000 7f9645c00640 1 --2- 172.21.15.94:0/3638676232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9624004810 0x7f964014bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:16.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.414+0000 7f9645c00640 1 --2- 172.21.15.94:0/3638676232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9624003ab0 0x7f96401f5660 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:16.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.414+0000 7f9645c00640 1 -- 172.21.15.94:0/3638676232 >> 172.21.15.94:0/3638676232 conn(0x7f96400d1c60 msgr2=0x7f9640160440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:16.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.414+0000 7f9645c00640 1 -- 172.21.15.94:0/3638676232 shutdown_connections 2024-09-10T23:32:16.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.414+0000 7f9645c00640 1 -- 172.21.15.94:0/3638676232 wait complete. 2024-09-10T23:32:16.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd628e00640 1 -- 172.21.15.94:0/1949734975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6240ccfb0 msgr2=0x7fd60400b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:16.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd628e00640 1 --2- 172.21.15.94:0/1949734975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6240ccfb0 0x7fd60400b1c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd614006b30 tx=0x7fd61401d720 comp rx=0 tx=0).stop 2024-09-10T23:32:16.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd628e00640 1 -- 172.21.15.94:0/1949734975 shutdown_connections 2024-09-10T23:32:16.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd628e00640 1 --2- 172.21.15.94:0/1949734975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd604007d50 0x7fd6040081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:16.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd628e00640 1 --2- 172.21.15.94:0/1949734975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6240ccfb0 0x7fd60400b1c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:16.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd628e00640 1 -- 172.21.15.94:0/1949734975 >> 172.21.15.94:0/1949734975 conn(0x7fd6240d0c00 msgr2=0x7fd6240d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:16.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd628e00640 1 -- 172.21.15.94:0/1949734975 shutdown_connections 2024-09-10T23:32:16.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd628e00640 1 -- 172.21.15.94:0/1949734975 wait complete. 2024-09-10T23:32:16.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd628e00640 1 Processor -- start 2024-09-10T23:32:16.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd628e00640 1 -- start start 2024-09-10T23:32:16.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd628e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd604003b60 0x7fd62415b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:16.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd628e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd604007d50 0x7fd6240a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:16.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd628e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6240ccfb0 0x7fd624070400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:16.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd628e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd62406d430 con 0x7fd604007d50 2024-09-10T23:32:16.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd628e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd624162920 con 0x7fd604003b60 2024-09-10T23:32:16.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd628e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd62406d5b0 con 0x7fd6240ccfb0 2024-09-10T23:32:16.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd621400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd604007d50 0x7fd6240a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:16.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd621400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd604007d50 0x7fd6240a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56350/0 (socket says 172.21.15.94:56350) 2024-09-10T23:32:16.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd621400640 1 -- 172.21.15.94:0/3009323699 learned_addr learned my addr 172.21.15.94:0/3009323699 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:16.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd621e00640 1 --2- 172.21.15.94:0/3009323699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd604003b60 0x7fd62415b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:16.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd622800640 1 --2- 172.21.15.94:0/3009323699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6240ccfb0 0x7fd624070400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:16.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd621400640 1 -- 172.21.15.94:0/3009323699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6240ccfb0 msgr2=0x7fd624070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:16.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd621400640 1 --2- 172.21.15.94:0/3009323699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6240ccfb0 0x7fd624070400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:16.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd621400640 1 -- 172.21.15.94:0/3009323699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd604003b60 msgr2=0x7fd62415b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:16.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd621400640 1 --2- 172.21.15.94:0/3009323699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd604003b60 0x7fd62415b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:16.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.978+0000 7fd621400640 1 -- 172.21.15.94:0/3009323699 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6240b3f50 con 0x7fd604007d50 2024-09-10T23:32:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.982+0000 7fd621400640 1 --2- 172.21.15.94:0/3009323699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd604007d50 0x7fd6240a1730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd610011880 tx=0x7fd610011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.982+0000 7fd61a000640 1 -- 172.21.15.94:0/3009323699 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610010dc0 con 0x7fd604007d50 2024-09-10T23:32:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.982+0000 7fd61a000640 1 -- 172.21.15.94:0/3009323699 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7fd610022070 con 0x7fd604007d50 2024-09-10T23:32:16.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.982+0000 7fd628e00640 1 -- 172.21.15.94:0/3009323699 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6240c92e0 con 0x7fd604007d50 2024-09-10T23:32:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.982+0000 7fd61a000640 1 -- 172.21.15.94:0/3009323699 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61001d670 con 0x7fd604007d50 2024-09-10T23:32:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.982+0000 7fd628e00640 1 -- 172.21.15.94:0/3009323699 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6240c98a0 con 0x7fd604007d50 2024-09-10T23:32:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.982+0000 7fd61a000640 1 -- 172.21.15.94:0/3009323699 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fd61001d810 con 0x7fd604007d50 2024-09-10T23:32:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.982+0000 7fd61a000640 1 -- 172.21.15.94:0/3009323699 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd610027cb0 con 0x7fd604007d50 2024-09-10T23:32:16.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.982+0000 7fd628e00640 1 -- 172.21.15.94:0/3009323699 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd62406a060 con 0x7fd604007d50 2024-09-10T23:32:16.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:16.982+0000 7fd61a000640 1 -- 172.21.15.94:0/3009323699 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fd61001dcc0 con 0x7fd604007d50 2024-09-10T23:32:17.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.042+0000 7fd628e00640 1 -- 172.21.15.94:0/3009323699 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7fd62406da10 con 0x7fd604007d50 2024-09-10T23:32:17.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.046+0000 7fd61a000640 1 -- 172.21.15.94:0/3009323699 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7fd610027460 con 0x7fd604007d50 2024-09-10T23:32:17.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.046+0000 7fd61a000640 1 -- 172.21.15.94:0/3009323699 <== mon.0 v2:172.21.15.94:3300/0 8 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fd610027700 con 0x7fd604007d50 2024-09-10T23:32:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.046+0000 7fd628e00640 1 -- 172.21.15.94:0/3009323699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd604007d50 msgr2=0x7fd6240a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.046+0000 7fd628e00640 1 --2- 172.21.15.94:0/3009323699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd604007d50 0x7fd6240a1730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd610011880 tx=0x7fd610011d40 comp rx=0 tx=0).stop 2024-09-10T23:32:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.046+0000 7fd628e00640 1 -- 172.21.15.94:0/3009323699 shutdown_connections 2024-09-10T23:32:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.046+0000 7fd628e00640 1 --2- 172.21.15.94:0/3009323699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6240ccfb0 0x7fd624070400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.046+0000 7fd628e00640 1 --2- 172.21.15.94:0/3009323699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd604007d50 0x7fd6240a1730 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.046+0000 7fd628e00640 1 --2- 172.21.15.94:0/3009323699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd604003b60 0x7fd62415b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.046+0000 7fd628e00640 1 -- 172.21.15.94:0/3009323699 >> 172.21.15.94:0/3009323699 conn(0x7fd6240d0c00 msgr2=0x7fd6240d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.046+0000 7fd628e00640 1 -- 172.21.15.94:0/3009323699 shutdown_connections 2024-09-10T23:32:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.046+0000 7fd628e00640 1 -- 172.21.15.94:0/3009323699 wait complete. 2024-09-10T23:32:17.067 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.068 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.068 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.118+0000 7f360a200640 1 -- 172.21.15.94:0/543070698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35e4007d50 msgr2=0x7f35e4008200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:17.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.118+0000 7f360a200640 1 --2- 172.21.15.94:0/543070698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35e4007d50 0x7f35e4008200 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f36040767b0 tx=0x7f35fc01d8e0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.118+0000 7f360a200640 1 -- 172.21.15.94:0/543070698 shutdown_connections 2024-09-10T23:32:17.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.118+0000 7f360a200640 1 --2- 172.21.15.94:0/543070698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35e4007d50 0x7f35e4008200 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.118+0000 7f360a200640 1 --2- 172.21.15.94:0/543070698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35e4009400 0x7f35e400b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.118+0000 7f360a200640 1 -- 172.21.15.94:0/543070698 >> 172.21.15.94:0/543070698 conn(0x7f36040d0b90 msgr2=0x7f36040d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.118+0000 7f360a200640 1 -- 172.21.15.94:0/543070698 shutdown_connections 2024-09-10T23:32:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.118+0000 7f360a200640 1 -- 172.21.15.94:0/543070698 wait complete. 2024-09-10T23:32:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.118+0000 7f360a200640 1 Processor -- start 2024-09-10T23:32:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.118+0000 7f360a200640 1 -- start start 2024-09-10T23:32:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f360a200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35e4003b00 0x7f3604142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f360a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4007d50 0x7f3604145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f360a200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35e4009400 0x7f36041f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f360a200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f360406cfb0 con 0x7f35e4007d50 2024-09-10T23:32:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f360a200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f360406ce30 con 0x7f35e4003b00 2024-09-10T23:32:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f360a200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f360406d130 con 0x7f35e4009400 2024-09-10T23:32:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f3602a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4007d50 0x7f3604145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f3602a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4007d50 0x7f3604145bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56358/0 (socket says 172.21.15.94:56358) 2024-09-10T23:32:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f3602a00640 1 -- 172.21.15.94:0/4110503601 learned_addr learned my addr 172.21.15.94:0/4110503601 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f3603400640 1 --2- 172.21.15.94:0/4110503601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35e4003b00 0x7f3604142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f3603e00640 1 --2- 172.21.15.94:0/4110503601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35e4009400 0x7f36041f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f3603e00640 1 -- 172.21.15.94:0/4110503601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35e4009400 msgr2=0x7f36041f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T23:32:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f3603e00640 1 -- 172.21.15.94:0/4110503601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35e4009400 msgr2=0x7f36041f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T23:32:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f3603e00640 1 --2- 172.21.15.94:0/4110503601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35e4009400 0x7f36041f7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T23:32:17.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f3603e00640 1 --2- 172.21.15.94:0/4110503601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35e4009400 0x7f36041f7c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T23:32:17.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f3602a00640 1 -- 172.21.15.94:0/4110503601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35e4009400 msgr2=0x7f36041f7c50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:32:17.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f3602a00640 1 --2- 172.21.15.94:0/4110503601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35e4009400 0x7f36041f7c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f3602a00640 1 -- 172.21.15.94:0/4110503601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35e4003b00 msgr2=0x7f3604142ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:17.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f3602a00640 1 --2- 172.21.15.94:0/4110503601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35e4003b00 0x7f3604142ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f3602a00640 1 -- 172.21.15.94:0/4110503601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3604146130 con 0x7f35e4007d50 2024-09-10T23:32:17.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f3603400640 1 --2- 172.21.15.94:0/4110503601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35e4003b00 0x7f3604142ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:32:17.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f3602a00640 1 --2- 172.21.15.94:0/4110503601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4007d50 0x7f3604145bf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f35f0004d80 tx=0x7f35f0014830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:17.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f35fb400640 1 -- 172.21.15.94:0/4110503601 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0012860 con 0x7f35e4007d50 2024-09-10T23:32:17.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f35fb400640 1 -- 172.21.15.94:0/4110503601 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f35f0012a00 con 0x7f35e4007d50 2024-09-10T23:32:17.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f360a200640 1 -- 172.21.15.94:0/4110503601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36041fb290 con 0x7f35e4007d50 2024-09-10T23:32:17.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f35fb400640 1 -- 172.21.15.94:0/4110503601 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0012e30 con 0x7f35e4007d50 2024-09-10T23:32:17.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f360a200640 1 -- 172.21.15.94:0/4110503601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36041fb850 con 0x7f35e4007d50 2024-09-10T23:32:17.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f35fb400640 1 -- 172.21.15.94:0/4110503601 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f35f001b070 con 0x7f35e4007d50 2024-09-10T23:32:17.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f35fb400640 1 -- 172.21.15.94:0/4110503601 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f35f0026950 con 0x7f35e4007d50 2024-09-10T23:32:17.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.122+0000 7f360a200640 1 -- 172.21.15.94:0/4110503601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36040cf700 con 0x7f35e4007d50 2024-09-10T23:32:17.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.126+0000 7f35fb400640 1 -- 172.21.15.94:0/4110503601 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f35f002b080 con 0x7f35e4007d50 2024-09-10T23:32:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.186+0000 7f360a200640 1 -- 172.21.15.94:0/4110503601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f36040d1a00 con 0x7f35e4007d50 2024-09-10T23:32:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.186+0000 7f35fb400640 1 -- 172.21.15.94:0/4110503601 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v15)=0 v15) ==== 123+0+0 (secure 0 0 0) 0x7f35f0026ba0 con 0x7f35e4007d50 2024-09-10T23:32:17.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.190+0000 7f35fb400640 1 -- 172.21.15.94:0/4110503601 <== mon.0 v2:172.21.15.94:3300/0 8 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35f0026e40 con 0x7f35e4007d50 2024-09-10T23:32:17.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.190+0000 7f360a200640 1 -- 172.21.15.94:0/4110503601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4007d50 msgr2=0x7f3604145bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.190+0000 7f360a200640 1 --2- 172.21.15.94:0/4110503601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4007d50 0x7f3604145bf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f35f0004d80 tx=0x7f35f0014830 comp rx=0 tx=0).stop 2024-09-10T23:32:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.190+0000 7f360a200640 1 -- 172.21.15.94:0/4110503601 shutdown_connections 2024-09-10T23:32:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.190+0000 7f360a200640 1 --2- 172.21.15.94:0/4110503601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35e4009400 0x7f36041f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.190+0000 7f360a200640 1 --2- 172.21.15.94:0/4110503601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4007d50 0x7f3604145bf0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.190+0000 7f360a200640 1 --2- 172.21.15.94:0/4110503601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35e4003b00 0x7f3604142ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.190+0000 7f360a200640 1 -- 172.21.15.94:0/4110503601 >> 172.21.15.94:0/4110503601 conn(0x7f36040d0b90 msgr2=0x7f3604070130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.190+0000 7f360a200640 1 -- 172.21.15.94:0/4110503601 shutdown_connections 2024-09-10T23:32:17.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.190+0000 7f360a200640 1 -- 172.21.15.94:0/4110503601 wait complete. 2024-09-10T23:32:17.209 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.209 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.209 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.209 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.209 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.350+0000 7fcf10800640 1 -- 172.21.15.94:0/419637264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceec009380 msgr2=0x7fceec00b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:17.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.350+0000 7fcf10800640 1 --2- 172.21.15.94:0/419637264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceec009380 0x7fceec00b770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcefc005fd0 tx=0x7fcefc01e490 comp rx=0 tx=0).stop 2024-09-10T23:32:17.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.350+0000 7fcf10800640 1 -- 172.21.15.94:0/419637264 shutdown_connections 2024-09-10T23:32:17.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.350+0000 7fcf10800640 1 --2- 172.21.15.94:0/419637264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceec007dc0 0x7fceec008250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.350+0000 7fcf10800640 1 --2- 172.21.15.94:0/419637264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceec009380 0x7fceec00b770 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.350+0000 7fcf10800640 1 -- 172.21.15.94:0/419637264 >> 172.21.15.94:0/419637264 conn(0x7fcf080d1c60 msgr2=0x7fcf080d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:17.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.350+0000 7fcf10800640 1 -- 172.21.15.94:0/419637264 shutdown_connections 2024-09-10T23:32:17.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.350+0000 7fcf10800640 1 -- 172.21.15.94:0/419637264 wait complete. 2024-09-10T23:32:17.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.350+0000 7fcf10800640 1 Processor -- start 2024-09-10T23:32:17.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.350+0000 7fcf10800640 1 -- start start 2024-09-10T23:32:17.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf10800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceec003b60 0x7fcf0809f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:17.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf10800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceec007dc0 0x7fcf08157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:17.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf10800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceec009380 0x7fcf08158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:17.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf10800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf080d04c0 con 0x7fceec007dc0 2024-09-10T23:32:17.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf10800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcf080d0340 con 0x7fceec003b60 2024-09-10T23:32:17.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf10800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fcf080d0640 con 0x7fceec009380 2024-09-10T23:32:17.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf0d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceec007dc0 0x7fcf08157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:17.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf0d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceec007dc0 0x7fcf08157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56366/0 (socket says 172.21.15.94:56366) 2024-09-10T23:32:17.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf0d400640 1 -- 172.21.15.94:0/3871797602 learned_addr learned my addr 172.21.15.94:0/3871797602 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:17.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf0e800640 1 --2- 172.21.15.94:0/3871797602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceec009380 0x7fcf08158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:17.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf0de00640 1 --2- 172.21.15.94:0/3871797602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceec003b60 0x7fcf0809f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:17.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf0e800640 1 -- 172.21.15.94:0/3871797602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceec009380 msgr2=0x7fcf08158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T23:32:17.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf0e800640 1 -- 172.21.15.94:0/3871797602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceec009380 msgr2=0x7fcf08158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T23:32:17.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf0e800640 1 --2- 172.21.15.94:0/3871797602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceec009380 0x7fcf08158280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T23:32:17.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf0e800640 1 --2- 172.21.15.94:0/3871797602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceec009380 0x7fcf08158280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T23:32:17.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf0d400640 1 -- 172.21.15.94:0/3871797602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceec009380 msgr2=0x7fcf08158280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:32:17.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf0d400640 1 --2- 172.21.15.94:0/3871797602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceec009380 0x7fcf08158280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf0d400640 1 -- 172.21.15.94:0/3871797602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceec003b60 msgr2=0x7fcf0809f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:17.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf0d400640 1 --2- 172.21.15.94:0/3871797602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceec003b60 0x7fcf0809f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf0d400640 1 -- 172.21.15.94:0/3871797602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0814ce30 con 0x7fceec007dc0 2024-09-10T23:32:17.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf0d400640 1 --2- 172.21.15.94:0/3871797602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceec007dc0 0x7fcf08157d40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcef8000f70 tx=0x7fcef8014830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:17.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf02000640 1 -- 172.21.15.94:0/3871797602 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8012650 con 0x7fceec007dc0 2024-09-10T23:32:17.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf02000640 1 -- 172.21.15.94:0/3871797602 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcef80127f0 con 0x7fceec007dc0 2024-09-10T23:32:17.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf10800640 1 -- 172.21.15.94:0/3871797602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0814c070 con 0x7fceec007dc0 2024-09-10T23:32:17.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf02000640 1 -- 172.21.15.94:0/3871797602 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8012c20 con 0x7fceec007dc0 2024-09-10T23:32:17.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf10800640 1 -- 172.21.15.94:0/3871797602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf0814c630 con 0x7fceec007dc0 2024-09-10T23:32:17.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf02000640 1 -- 172.21.15.94:0/3871797602 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fcef801b070 con 0x7fceec007dc0 2024-09-10T23:32:17.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf02000640 1 -- 172.21.15.94:0/3871797602 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcef801d780 con 0x7fceec007dc0 2024-09-10T23:32:17.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.354+0000 7fcf10800640 1 -- 172.21.15.94:0/3871797602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf080c8b50 con 0x7fceec007dc0 2024-09-10T23:32:17.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.358+0000 7fcf02000640 1 -- 172.21.15.94:0/3871797602 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fcf080c8b50 con 0x7fceec007dc0 2024-09-10T23:32:17.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.418+0000 7fcf10800640 1 -- 172.21.15.94:0/3871797602 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fcf080d0ba0 con 0x7fceec007dc0 2024-09-10T23:32:17.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.418+0000 7fcf02000640 1 -- 172.21.15.94:0/3871797602 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v16)=0 v16) ==== 115+0+0 (secure 0 0 0) 0x7fcef8012dc0 con 0x7fceec007dc0 2024-09-10T23:32:17.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.422+0000 7fcf10800640 1 -- 172.21.15.94:0/3871797602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceec007dc0 msgr2=0x7fcf08157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:17.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.422+0000 7fcf10800640 1 --2- 172.21.15.94:0/3871797602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceec007dc0 0x7fcf08157d40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcef8000f70 tx=0x7fcef8014830 comp rx=0 tx=0).stop 2024-09-10T23:32:17.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.422+0000 7fcf10800640 1 -- 172.21.15.94:0/3871797602 shutdown_connections 2024-09-10T23:32:17.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.422+0000 7fcf10800640 1 --2- 172.21.15.94:0/3871797602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceec009380 0x7fcf08158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.422+0000 7fcf10800640 1 --2- 172.21.15.94:0/3871797602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceec007dc0 0x7fcf08157d40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.422+0000 7fcf10800640 1 --2- 172.21.15.94:0/3871797602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceec003b60 0x7fcf0809f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.422+0000 7fcf10800640 1 -- 172.21.15.94:0/3871797602 >> 172.21.15.94:0/3871797602 conn(0x7fcf080d1c60 msgr2=0x7fcf080b67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:17.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.422+0000 7fcf10800640 1 -- 172.21.15.94:0/3871797602 shutdown_connections 2024-09-10T23:32:17.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.422+0000 7fcf10800640 1 -- 172.21.15.94:0/3871797602 wait complete. 2024-09-10T23:32:17.949 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.949 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.949 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.486+0000 7f23e0c00640 1 -- 172.21.15.94:0/4277912012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8006200 msgr2=0x7f23b80086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:17.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.486+0000 7f23e0c00640 1 --2- 172.21.15.94:0/4277912012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8006200 0x7f23b80086a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f23b8008f20 tx=0x7f23cc01d7f0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.486+0000 7f23e0c00640 1 -- 172.21.15.94:0/4277912012 shutdown_connections 2024-09-10T23:32:17.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.486+0000 7f23e0c00640 1 --2- 172.21.15.94:0/4277912012 >> v2:172.21.15.181:3300/0 conn(0x7f23b8009730 0x7f23b800bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.486+0000 7f23e0c00640 1 --2- 172.21.15.94:0/4277912012 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23b8003b60 0x7f23b80091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.486+0000 7f23e0c00640 1 --2- 172.21.15.94:0/4277912012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8006200 0x7f23b80086a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.486+0000 7f23e0c00640 1 -- 172.21.15.94:0/4277912012 >> 172.21.15.94:0/4277912012 conn(0x7f23dc0d1c60 msgr2=0x7f23dc0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:17.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.486+0000 7f23e0c00640 1 -- 172.21.15.94:0/4277912012 shutdown_connections 2024-09-10T23:32:17.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.486+0000 7f23e0c00640 1 -- 172.21.15.94:0/4277912012 wait complete. 2024-09-10T23:32:17.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23e0c00640 1 Processor -- start 2024-09-10T23:32:17.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23e0c00640 1 -- start start 2024-09-10T23:32:17.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23e0c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23b8003b60 0x7f23dc15b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:17.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23e0c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23b8006200 0x7f23dc0afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:17.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23e0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8009730 0x7f23dc09b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:17.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23e0c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc06dd30 con 0x7f23b8009730 2024-09-10T23:32:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23e0c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc0b2c50 con 0x7f23b8003b60 2024-09-10T23:32:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23e0c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f23dc06deb0 con 0x7f23b8006200 2024-09-10T23:32:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23da800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8009730 0x7f23dc09b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23da800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8009730 0x7f23dc09b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56388/0 (socket says 172.21.15.94:56388) 2024-09-10T23:32:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23da800640 1 -- 172.21.15.94:0/272390191 learned_addr learned my addr 172.21.15.94:0/272390191 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23d3e00640 1 --2- 172.21.15.94:0/272390191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23b8006200 0x7f23dc0afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23d9e00640 1 --2- 172.21.15.94:0/272390191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23b8003b60 0x7f23dc15b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23d3e00640 1 -- 172.21.15.94:0/272390191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23b8006200 msgr2=0x7f23dc0afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T23:32:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23d3e00640 1 -- 172.21.15.94:0/272390191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23b8006200 msgr2=0x7f23dc0afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T23:32:17.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23d3e00640 1 --2- 172.21.15.94:0/272390191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23b8006200 0x7f23dc0afc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T23:32:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23d3e00640 1 --2- 172.21.15.94:0/272390191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23b8006200 0x7f23dc0afc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T23:32:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23da800640 1 -- 172.21.15.94:0/272390191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23b8006200 msgr2=0x7f23dc0afc20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:32:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23da800640 1 --2- 172.21.15.94:0/272390191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23b8006200 0x7f23dc0afc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23da800640 1 -- 172.21.15.94:0/272390191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23b8003b60 msgr2=0x7f23dc15b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23da800640 1 --2- 172.21.15.94:0/272390191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23b8003b60 0x7f23dc15b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23da800640 1 -- 172.21.15.94:0/272390191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc15e8a0 con 0x7f23b8009730 2024-09-10T23:32:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23da800640 1 --2- 172.21.15.94:0/272390191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8009730 0x7f23dc09b980 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0d2980 tx=0x7f23d4001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23d2000640 1 -- 172.21.15.94:0/272390191 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4011870 con 0x7f23b8009730 2024-09-10T23:32:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23d2000640 1 -- 172.21.15.94:0/272390191 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23d4017070 con 0x7f23b8009730 2024-09-10T23:32:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23d2000640 1 -- 172.21.15.94:0/272390191 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4011dc0 con 0x7f23b8009730 2024-09-10T23:32:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23e0c00640 1 -- 172.21.15.94:0/272390191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23dc09c2f0 con 0x7f23b8009730 2024-09-10T23:32:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.490+0000 7f23e0c00640 1 -- 172.21.15.94:0/272390191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23dc09c8b0 con 0x7f23b8009730 2024-09-10T23:32:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.494+0000 7f23d2000640 1 -- 172.21.15.94:0/272390191 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f23d401c460 con 0x7f23b8009730 2024-09-10T23:32:17.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.494+0000 7f23d2000640 1 -- 172.21.15.94:0/272390191 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f23d401b7b0 con 0x7f23b8009730 2024-09-10T23:32:17.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.494+0000 7f23e0c00640 1 -- 172.21.15.94:0/272390191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23dc0b5cf0 con 0x7f23b8009730 2024-09-10T23:32:17.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.494+0000 7f23d2000640 1 -- 172.21.15.94:0/272390191 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f23d401b9b0 con 0x7f23b8009730 2024-09-10T23:32:17.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.538+0000 7f4006c00640 1 -- 172.21.15.94:0/2306504137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f400015add0 msgr2=0x7f3fe000b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:17.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.538+0000 7f4006c00640 1 --2- 172.21.15.94:0/2306504137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f400015add0 0x7f3fe000b240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f40000cf9e0 tx=0x7f3fec01ed30 comp rx=0 tx=0).stop 2024-09-10T23:32:17.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.538+0000 7f4006c00640 1 -- 172.21.15.94:0/2306504137 shutdown_connections 2024-09-10T23:32:17.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.538+0000 7f4006c00640 1 --2- 172.21.15.94:0/2306504137 >> v2:172.21.15.181:3300/0 conn(0x7f3fe0003ae0 0x7f3fe000b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.538+0000 7f4006c00640 1 --2- 172.21.15.94:0/2306504137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f400015add0 0x7f3fe000b240 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.538+0000 7f4006c00640 1 --2- 172.21.15.94:0/2306504137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe0006190 0x7f3fe00085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.538+0000 7f4006c00640 1 -- 172.21.15.94:0/2306504137 >> 172.21.15.94:0/2306504137 conn(0x7f40000cfd20 msgr2=0x7f40000d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:17.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.538+0000 7f4006c00640 1 -- 172.21.15.94:0/2306504137 shutdown_connections 2024-09-10T23:32:17.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.538+0000 7f4006c00640 1 -- 172.21.15.94:0/2306504137 wait complete. 2024-09-10T23:32:17.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.542+0000 7f4006c00640 1 Processor -- start 2024-09-10T23:32:17.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.542+0000 7f4006c00640 1 -- start start 2024-09-10T23:32:17.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.542+0000 7f4006c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe0003ae0 0x7f40000c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:17.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.542+0000 7f4006c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe0006190 0x7f40000b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:17.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.542+0000 7f4006c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f400015add0 0x7f4000158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:17.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.542+0000 7f4006c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f400015a6c0 con 0x7f400015add0 2024-09-10T23:32:17.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.542+0000 7f4006c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f40000cdd80 con 0x7f3fe0003ae0 2024-09-10T23:32:17.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.542+0000 7f4006c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f400015a840 con 0x7f3fe0006190 2024-09-10T23:32:17.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4006200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f400015add0 0x7f4000158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:17.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4005800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe0003ae0 0x7f40000c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:17.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4004e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe0006190 0x7f40000b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:17.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4005800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe0003ae0 0x7f40000c9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:51070/0 (socket says 172.21.15.94:51070) 2024-09-10T23:32:17.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4005800640 1 -- 172.21.15.94:0/2612672180 learned_addr learned my addr 172.21.15.94:0/2612672180 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:17.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4004e00640 1 -- 172.21.15.94:0/2612672180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe0006190 msgr2=0x7f40000b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T23:32:17.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4004e00640 1 -- 172.21.15.94:0/2612672180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe0006190 msgr2=0x7f40000b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T23:32:17.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4004e00640 1 --2- 172.21.15.94:0/2612672180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe0006190 0x7f40000b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T23:32:17.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4004e00640 1 --2- 172.21.15.94:0/2612672180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe0006190 0x7f40000b3c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T23:32:17.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4006200640 1 -- 172.21.15.94:0/2612672180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe0006190 msgr2=0x7f40000b3c00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:32:17.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4006200640 1 --2- 172.21.15.94:0/2612672180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe0006190 0x7f40000b3c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4006200640 1 -- 172.21.15.94:0/2612672180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe0003ae0 msgr2=0x7f40000c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:17.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4006200640 1 --2- 172.21.15.94:0/2612672180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe0003ae0 0x7f40000c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4006200640 1 -- 172.21.15.94:0/2612672180 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40000b1680 con 0x7f400015add0 2024-09-10T23:32:17.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4006200640 1 --2- 172.21.15.94:0/2612672180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f400015add0 0x7f4000158b80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f40001511c0 tx=0x7f3ff8006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:17.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f3ffd600640 1 -- 172.21.15.94:0/2612672180 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8015070 con 0x7f400015add0 2024-09-10T23:32:17.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f3ffd600640 1 -- 172.21.15.94:0/2612672180 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ff80119f0 con 0x7f400015add0 2024-09-10T23:32:17.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f3ffd600640 1 -- 172.21.15.94:0/2612672180 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8011e60 con 0x7f400015add0 2024-09-10T23:32:17.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4006c00640 1 -- 172.21.15.94:0/2612672180 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4000151400 con 0x7f400015add0 2024-09-10T23:32:17.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4006c00640 1 -- 172.21.15.94:0/2612672180 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40001519c0 con 0x7f400015add0 2024-09-10T23:32:17.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f3ffd600640 1 -- 172.21.15.94:0/2612672180 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f3ff8010420 con 0x7f400015add0 2024-09-10T23:32:17.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f3ffd600640 1 -- 172.21.15.94:0/2612672180 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3ff8023b10 con 0x7f400015add0 2024-09-10T23:32:17.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.546+0000 7f4006c00640 1 -- 172.21.15.94:0/2612672180 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4000072800 con 0x7f400015add0 2024-09-10T23:32:17.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.550+0000 7f3ffd600640 1 -- 172.21.15.94:0/2612672180 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f3ff8023d10 con 0x7f400015add0 2024-09-10T23:32:17.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.558+0000 7f23e0c00640 1 -- 172.21.15.94:0/272390191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f23dc09d030 con 0x7f23b8009730 2024-09-10T23:32:17.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.562+0000 7f23d2000640 1 -- 172.21.15.94:0/272390191 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v17)=0 v17) ==== 119+0+0 (secure 0 0 0) 0x7f23d401bb90 con 0x7f23b8009730 2024-09-10T23:32:17.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.566+0000 7f23e0c00640 1 -- 172.21.15.94:0/272390191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8009730 msgr2=0x7f23dc09b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:17.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.566+0000 7f23e0c00640 1 --2- 172.21.15.94:0/272390191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8009730 0x7f23dc09b980 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0d2980 tx=0x7f23d4001a20 comp rx=0 tx=0).stop 2024-09-10T23:32:17.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.566+0000 7f23e0c00640 1 -- 172.21.15.94:0/272390191 shutdown_connections 2024-09-10T23:32:17.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.566+0000 7f23e0c00640 1 --2- 172.21.15.94:0/272390191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b8009730 0x7f23dc09b980 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.566+0000 7f23e0c00640 1 --2- 172.21.15.94:0/272390191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23b8006200 0x7f23dc0afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.566+0000 7f23e0c00640 1 --2- 172.21.15.94:0/272390191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23b8003b60 0x7f23dc15b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.566+0000 7f23e0c00640 1 -- 172.21.15.94:0/272390191 >> 172.21.15.94:0/272390191 conn(0x7f23dc0d1c60 msgr2=0x7f23dc071320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:17.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.566+0000 7f23e0c00640 1 -- 172.21.15.94:0/272390191 shutdown_connections 2024-09-10T23:32:17.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.566+0000 7f23e0c00640 1 -- 172.21.15.94:0/272390191 wait complete. 2024-09-10T23:32:17.964 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.614+0000 7f4006c00640 1 -- 172.21.15.94:0/2612672180 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f40000cfa10 con 0x7f400015add0 2024-09-10T23:32:17.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.614+0000 7f3ffd600640 1 -- 172.21.15.94:0/2612672180 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v18)=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f3ff8010880 con 0x7f400015add0 2024-09-10T23:32:17.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.618+0000 7f4006c00640 1 -- 172.21.15.94:0/2612672180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f400015add0 msgr2=0x7f4000158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:17.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.618+0000 7f4006c00640 1 --2- 172.21.15.94:0/2612672180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f400015add0 0x7f4000158b80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f40001511c0 tx=0x7f3ff8006430 comp rx=0 tx=0).stop 2024-09-10T23:32:17.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.618+0000 7f4006c00640 1 -- 172.21.15.94:0/2612672180 shutdown_connections 2024-09-10T23:32:17.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.618+0000 7f4006c00640 1 --2- 172.21.15.94:0/2612672180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f400015add0 0x7f4000158b80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.618+0000 7f4006c00640 1 --2- 172.21.15.94:0/2612672180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe0006190 0x7f40000b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.618+0000 7f4006c00640 1 --2- 172.21.15.94:0/2612672180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe0003ae0 0x7f40000c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.618+0000 7f4006c00640 1 -- 172.21.15.94:0/2612672180 >> 172.21.15.94:0/2612672180 conn(0x7f40000cfd20 msgr2=0x7f40000ca650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:17.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.618+0000 7f4006c00640 1 -- 172.21.15.94:0/2612672180 shutdown_connections 2024-09-10T23:32:17.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.618+0000 7f4006c00640 1 -- 172.21.15.94:0/2612672180 wait complete. 2024-09-10T23:32:17.967 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.967 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.967 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.678+0000 7ff42ca00640 1 -- 172.21.15.94:0/4114115937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c003b80 msgr2=0x7ff40c003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:17.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.678+0000 7ff42ca00640 1 --2- 172.21.15.94:0/4114115937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c003b80 0x7ff40c003f80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff418006b30 tx=0x7ff41801d820 comp rx=0 tx=0).stop 2024-09-10T23:32:17.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.678+0000 7ff426800640 1 -- 172.21.15.94:0/4114115937 reap_dead start 2024-09-10T23:32:17.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.678+0000 7ff42ca00640 1 -- 172.21.15.94:0/4114115937 shutdown_connections 2024-09-10T23:32:17.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.678+0000 7ff42ca00640 1 -- 172.21.15.94:0/4114115937 >> 172.21.15.94:0/4114115937 conn(0x7ff4280d0c00 msgr2=0x7ff4280d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:17.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.678+0000 7ff42ca00640 1 -- 172.21.15.94:0/4114115937 shutdown_connections 2024-09-10T23:32:17.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.678+0000 7ff42ca00640 1 -- 172.21.15.94:0/4114115937 wait complete. 2024-09-10T23:32:17.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff42ca00640 1 Processor -- start 2024-09-10T23:32:17.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff42ca00640 1 -- start start 2024-09-10T23:32:17.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff42ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c003b80 0x7ff4280c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:17.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff42ca00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4280c9d70 0x7ff4280ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:17.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff42ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4280b5a60 0x7ff4280b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:17.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff42ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff42815ef80 con 0x7ff40c003b80 2024-09-10T23:32:17.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff42ca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff42815f100 con 0x7ff4280b5a60 2024-09-10T23:32:17.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff42ca00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff42815f280 con 0x7ff4280c9d70 2024-09-10T23:32:17.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff425e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c003b80 0x7ff4280c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:17.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff425e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c003b80 0x7ff4280c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56404/0 (socket says 172.21.15.94:56404) 2024-09-10T23:32:17.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff425e00640 1 -- 172.21.15.94:0/3893275065 learned_addr learned my addr 172.21.15.94:0/3893275065 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:17.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff425400640 1 --2- 172.21.15.94:0/3893275065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4280c9d70 0x7ff4280ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:17.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff426800640 1 --2- 172.21.15.94:0/3893275065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4280b5a60 0x7ff4280b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:17.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff425400640 1 -- 172.21.15.94:0/3893275065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4280c9d70 msgr2=0x7ff4280ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T23:32:17.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff425400640 1 -- 172.21.15.94:0/3893275065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4280c9d70 msgr2=0x7ff4280ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T23:32:17.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff425400640 1 --2- 172.21.15.94:0/3893275065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4280c9d70 0x7ff4280ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T23:32:17.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff425400640 1 --2- 172.21.15.94:0/3893275065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4280c9d70 0x7ff4280ca1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T23:32:17.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff425e00640 1 -- 172.21.15.94:0/3893275065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4280c9d70 msgr2=0x7ff4280ca1d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:32:17.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff425e00640 1 --2- 172.21.15.94:0/3893275065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4280c9d70 0x7ff4280ca1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff425e00640 1 -- 172.21.15.94:0/3893275065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4280b5a60 msgr2=0x7ff4280b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:17.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff425e00640 1 --2- 172.21.15.94:0/3893275065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4280b5a60 0x7ff4280b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff425e00640 1 -- 172.21.15.94:0/3893275065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4280ca7c0 con 0x7ff40c003b80 2024-09-10T23:32:17.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff425e00640 1 --2- 172.21.15.94:0/3893275065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c003b80 0x7ff4280c7090 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff418006b50 tx=0x7ff4180067f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:17.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff41e000640 1 -- 172.21.15.94:0/3893275065 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4180ac040 con 0x7ff40c003b80 2024-09-10T23:32:17.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff41e000640 1 -- 172.21.15.94:0/3893275065 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4180a65c0 con 0x7ff40c003b80 2024-09-10T23:32:17.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff42ca00640 1 -- 172.21.15.94:0/3893275065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4280c8bb0 con 0x7ff40c003b80 2024-09-10T23:32:17.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff41e000640 1 -- 172.21.15.94:0/3893275065 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418006ba0 con 0x7ff40c003b80 2024-09-10T23:32:17.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.682+0000 7ff42ca00640 1 -- 172.21.15.94:0/3893275065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4280c9170 con 0x7ff40c003b80 2024-09-10T23:32:17.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.686+0000 7ff41e000640 1 -- 172.21.15.94:0/3893275065 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7ff4180b9070 con 0x7ff40c003b80 2024-09-10T23:32:17.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.686+0000 7ff41e000640 1 -- 172.21.15.94:0/3893275065 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff4180c36e0 con 0x7ff40c003b80 2024-09-10T23:32:17.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.686+0000 7ff42ca00640 1 -- 172.21.15.94:0/3893275065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42806cc20 con 0x7ff40c003b80 2024-09-10T23:32:17.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.690+0000 7ff41e000640 1 -- 172.21.15.94:0/3893275065 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7ff418017600 con 0x7ff40c003b80 2024-09-10T23:32:17.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.746+0000 7ff42ca00640 1 -- 172.21.15.94:0/3893275065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7ff42806cc20 con 0x7ff40c003b80 2024-09-10T23:32:17.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.750+0000 7ff41e000640 1 -- 172.21.15.94:0/3893275065 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7ff4180a6790 con 0x7ff40c003b80 2024-09-10T23:32:17.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.750+0000 7ff42ca00640 1 -- 172.21.15.94:0/3893275065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c003b80 msgr2=0x7ff4280c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:17.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.750+0000 7ff42ca00640 1 --2- 172.21.15.94:0/3893275065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c003b80 0x7ff4280c7090 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff418006b50 tx=0x7ff4180067f0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.750+0000 7ff42ca00640 1 -- 172.21.15.94:0/3893275065 shutdown_connections 2024-09-10T23:32:17.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.750+0000 7ff42ca00640 1 --2- 172.21.15.94:0/3893275065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4280b5a60 0x7ff4280b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.750+0000 7ff42ca00640 1 --2- 172.21.15.94:0/3893275065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4280c9d70 0x7ff4280ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.750+0000 7ff42ca00640 1 --2- 172.21.15.94:0/3893275065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40c003b80 0x7ff4280c7090 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:17.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.750+0000 7ff42ca00640 1 -- 172.21.15.94:0/3893275065 >> 172.21.15.94:0/3893275065 conn(0x7ff4280d0c00 msgr2=0x7ff4280b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:17.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.750+0000 7ff42ca00640 1 -- 172.21.15.94:0/3893275065 shutdown_connections 2024-09-10T23:32:17.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:17.750+0000 7ff42ca00640 1 -- 172.21.15.94:0/3893275065 wait complete. 2024-09-10T23:32:17.975 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.975 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi094') 2024-09-10T23:32:17.975 INFO:tasks.ceph:Setting crush tunables to default 2024-09-10T23:32:17.976 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd crush tunables default 2024-09-10T23:32:18.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.114+0000 7f0143000640 1 -- 172.21.15.94:0/3524129797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f013c06fb90 msgr2=0x7f013c15b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:18.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.114+0000 7f0143000640 1 --2- 172.21.15.94:0/3524129797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f013c06fb90 0x7f013c15b560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0134009bc0 tx=0x7f013401cba0 comp rx=0 tx=0).stop 2024-09-10T23:32:18.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.114+0000 7f0143000640 1 -- 172.21.15.94:0/3524129797 shutdown_connections 2024-09-10T23:32:18.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.114+0000 7f0143000640 1 --2- 172.21.15.94:0/3524129797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f013c06fb90 0x7f013c15b560 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:18.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.114+0000 7f0143000640 1 --2- 172.21.15.94:0/3524129797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f013c0cf590 0x7f013c06f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:18.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.114+0000 7f0143000640 1 -- 172.21.15.94:0/3524129797 >> 172.21.15.94:0/3524129797 conn(0x7f013c0b26e0 msgr2=0x7f013c0b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:18.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.114+0000 7f0143000640 1 -- 172.21.15.94:0/3524129797 shutdown_connections 2024-09-10T23:32:18.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.114+0000 7f0143000640 1 -- 172.21.15.94:0/3524129797 wait complete. 2024-09-10T23:32:18.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f0143000640 1 Processor -- start 2024-09-10T23:32:18.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f0143000640 1 -- start start 2024-09-10T23:32:18.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f0143000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f013c06fb90 0x7f013c14e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:18.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f0143000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f013c0cf1b0 0x7f013c1fb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:18.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f0143000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f013c0cf590 0x7f013c1fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:18.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f0143000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1684a0 con 0x7f013c06fb90 2024-09-10T23:32:18.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f0143000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f013c168320 con 0x7f013c0cf590 2024-09-10T23:32:18.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f0143000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f013c168620 con 0x7f013c0cf1b0 2024-09-10T23:32:18.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f013be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f013c06fb90 0x7f013c14e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:18.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f013be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f013c06fb90 0x7f013c14e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56434/0 (socket says 172.21.15.94:56434) 2024-09-10T23:32:18.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f013be00640 1 -- 172.21.15.94:0/3595874327 learned_addr learned my addr 172.21.15.94:0/3595874327 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:18.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f013b400640 1 --2- 172.21.15.94:0/3595874327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f013c0cf1b0 0x7f013c1fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:18.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f0141000640 1 --2- 172.21.15.94:0/3595874327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f013c0cf590 0x7f013c1fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:18.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f013b400640 1 -- 172.21.15.94:0/3595874327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f013c0cf1b0 msgr2=0x7f013c1fb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T23:32:18.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f013b400640 1 -- 172.21.15.94:0/3595874327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f013c0cf1b0 msgr2=0x7f013c1fb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T23:32:18.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f013b400640 1 --2- 172.21.15.94:0/3595874327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f013c0cf1b0 0x7f013c1fb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T23:32:18.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f013b400640 1 --2- 172.21.15.94:0/3595874327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f013c0cf1b0 0x7f013c1fb360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T23:32:18.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f013be00640 1 -- 172.21.15.94:0/3595874327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f013c0cf1b0 msgr2=0x7f013c1fb360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:32:18.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f013be00640 1 --2- 172.21.15.94:0/3595874327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f013c0cf1b0 0x7f013c1fb360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:18.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f013be00640 1 -- 172.21.15.94:0/3595874327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f013c0cf590 msgr2=0x7f013c1fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:18.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f013be00640 1 --2- 172.21.15.94:0/3595874327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f013c0cf590 0x7f013c1fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:18.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f013be00640 1 -- 172.21.15.94:0/3595874327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c157610 con 0x7f013c06fb90 2024-09-10T23:32:18.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f013be00640 1 --2- 172.21.15.94:0/3595874327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f013c06fb90 0x7f013c14e9c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f012c00c4b0 tx=0x7f012c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:18.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f0133e00640 1 -- 172.21.15.94:0/3595874327 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c016020 con 0x7f013c06fb90 2024-09-10T23:32:18.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f0133e00640 1 -- 172.21.15.94:0/3595874327 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f012c012920 con 0x7f013c06fb90 2024-09-10T23:32:18.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f0143000640 1 -- 172.21.15.94:0/3595874327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013c1578f0 con 0x7f013c06fb90 2024-09-10T23:32:18.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f0133e00640 1 -- 172.21.15.94:0/3595874327 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c012d90 con 0x7f013c06fb90 2024-09-10T23:32:18.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f0143000640 1 -- 172.21.15.94:0/3595874327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f013c0c7db0 con 0x7f013c06fb90 2024-09-10T23:32:18.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f0133e00640 1 -- 172.21.15.94:0/3595874327 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f012c01b790 con 0x7f013c06fb90 2024-09-10T23:32:18.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.118+0000 7f0133e00640 1 -- 172.21.15.94:0/3595874327 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f012c01ac10 con 0x7f013c06fb90 2024-09-10T23:32:18.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.122+0000 7f0143000640 1 -- 172.21.15.94:0/3595874327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f013c15b560 con 0x7f013c06fb90 2024-09-10T23:32:18.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.122+0000 7f0133e00640 1 -- 172.21.15.94:0/3595874327 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f012c020030 con 0x7f013c06fb90 2024-09-10T23:32:18.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.186+0000 7f0143000640 1 -- 172.21.15.94:0/3595874327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f013c0b6b50 con 0x7f013c06fb90 2024-09-10T23:32:18.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.190+0000 7f0133e00640 1 -- 172.21.15.94:0/3595874327 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f012c01ae10 con 0x7f013c06fb90 2024-09-10T23:32:18.189 INFO:teuthology.orchestra.run.smithi094.stderr:adjusted tunables profile to default 2024-09-10T23:32:18.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.190+0000 7f0143000640 1 -- 172.21.15.94:0/3595874327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f013c06fb90 msgr2=0x7f013c14e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:18.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.190+0000 7f0143000640 1 --2- 172.21.15.94:0/3595874327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f013c06fb90 0x7f013c14e9c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f012c00c4b0 tx=0x7f012c00c970 comp rx=0 tx=0).stop 2024-09-10T23:32:18.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.190+0000 7f0143000640 1 -- 172.21.15.94:0/3595874327 shutdown_connections 2024-09-10T23:32:18.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.190+0000 7f0143000640 1 --2- 172.21.15.94:0/3595874327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f013c0cf590 0x7f013c1fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:18.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.190+0000 7f0143000640 1 --2- 172.21.15.94:0/3595874327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f013c0cf1b0 0x7f013c1fb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:18.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.190+0000 7f0143000640 1 --2- 172.21.15.94:0/3595874327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f013c06fb90 0x7f013c14e9c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:18.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.190+0000 7f0143000640 1 -- 172.21.15.94:0/3595874327 >> 172.21.15.94:0/3595874327 conn(0x7f013c0b26e0 msgr2=0x7f013c159db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:18.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.190+0000 7f0143000640 1 -- 172.21.15.94:0/3595874327 shutdown_connections 2024-09-10T23:32:18.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:18.190+0000 7f0143000640 1 -- 172.21.15.94:0/3595874327 wait complete. 2024-09-10T23:32:18.216 INFO:tasks.ceph:check_enable_crimson: False 2024-09-10T23:32:18.216 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-10T23:32:18.216 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:32:18.216 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-10T23:32:18.236 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:32:18.236 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-10T23:32:18.296 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:32:18.296 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-10T23:32:18.357 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:32:18.357 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-10T23:32:18.420 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:32:18.421 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-10T23:32:18.441 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:32:18.441 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-10T23:32:18.463 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:32:18.463 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-10T23:32:18.524 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:32:18.524 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-10T23:32:18.584 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea 0 2024-09-10T23:32:18.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426f200640 1 -- 172.21.15.181:0/42151764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42700b2ac0 msgr2=0x7f42700b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:18.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426f200640 1 --2- 172.21.15.181:0/42151764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42700b2ac0 0x7f42700b1880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4270072eb0 tx=0x7f425c01c7d0 comp rx=0 tx=0).stop 2024-09-10T23:32:18.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426f200640 1 -- 172.21.15.181:0/42151764 shutdown_connections 2024-09-10T23:32:18.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426f200640 1 --2- 172.21.15.181:0/42151764 >> v2:172.21.15.94:3300/0 conn(0x7f42700af150 0x7f42700af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:18.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426f200640 1 --2- 172.21.15.181:0/42151764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42700b2ac0 0x7f42700b1880 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:18.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426f200640 1 -- 172.21.15.181:0/42151764 >> 172.21.15.181:0/42151764 conn(0x7f42700d1b00 msgr2=0x7f42700d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:18.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426f200640 1 -- 172.21.15.181:0/42151764 shutdown_connections 2024-09-10T23:32:18.766 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426f200640 1 -- 172.21.15.181:0/42151764 wait complete. 2024-09-10T23:32:18.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426f200640 1 Processor -- start 2024-09-10T23:32:18.767 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426f200640 1 -- start start 2024-09-10T23:32:18.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42700af150 0x7f4270166100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:18.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42700b2ac0 0x7f4270147fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:18.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426f200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427015f140 0x7f4270155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:18.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4270168ad0 con 0x7f42700b2ac0 2024-09-10T23:32:18.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426f200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4270168950 con 0x7f42700af150 2024-09-10T23:32:18.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426f200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f4270168c50 con 0x7f427015f140 2024-09-10T23:32:18.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42700af150 0x7f4270166100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:18.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426e800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427015f140 0x7f4270155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:18.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42700af150 0x7f4270166100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49050/0 (socket says 172.21.15.181:49050) 2024-09-10T23:32:18.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426de00640 1 -- 172.21.15.181:0/3849998491 learned_addr learned my addr 172.21.15.181:0/3849998491 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T23:32:18.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426d400640 1 --2- 172.21.15.181:0/3849998491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42700b2ac0 0x7f4270147fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:18.770 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426e800640 1 -- 172.21.15.181:0/3849998491 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427015f140 msgr2=0x7f4270155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T23:32:18.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426e800640 1 -- 172.21.15.181:0/3849998491 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427015f140 msgr2=0x7f4270155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T23:32:18.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.764+0000 7f426e800640 1 --2- 172.21.15.181:0/3849998491 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427015f140 0x7f4270155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T23:32:18.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.768+0000 7f426e800640 1 --2- 172.21.15.181:0/3849998491 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427015f140 0x7f4270155cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T23:32:18.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.768+0000 7f426de00640 1 -- 172.21.15.181:0/3849998491 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427015f140 msgr2=0x7f4270155cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:32:18.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.768+0000 7f426de00640 1 --2- 172.21.15.181:0/3849998491 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427015f140 0x7f4270155cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:18.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.768+0000 7f426de00640 1 -- 172.21.15.181:0/3849998491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42700b2ac0 msgr2=0x7f4270147fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:18.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.768+0000 7f426de00640 1 --2- 172.21.15.181:0/3849998491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42700b2ac0 0x7f4270147fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:18.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.768+0000 7f426de00640 1 -- 172.21.15.181:0/3849998491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4270153410 con 0x7f42700af150 2024-09-10T23:32:18.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.768+0000 7f426de00640 1 --2- 172.21.15.181:0/3849998491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42700af150 0x7f4270166100 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4260001ec0 tx=0x7f426000c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:18.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.768+0000 7f4266000640 1 -- 172.21.15.181:0/3849998491 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260017070 con 0x7f42700af150 2024-09-10T23:32:18.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.768+0000 7f4266000640 1 -- 172.21.15.181:0/3849998491 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4260003600 con 0x7f42700af150 2024-09-10T23:32:18.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.768+0000 7f426f200640 1 -- 172.21.15.181:0/3849998491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42701536f0 con 0x7f42700af150 2024-09-10T23:32:18.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.768+0000 7f4266000640 1 -- 172.21.15.181:0/3849998491 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260003a50 con 0x7f42700af150 2024-09-10T23:32:18.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.768+0000 7f426f200640 1 -- 172.21.15.181:0/3849998491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42701530c0 con 0x7f42700af150 2024-09-10T23:32:18.771 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.768+0000 7f4266000640 1 -- 172.21.15.181:0/3849998491 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f4260003bf0 con 0x7f42700af150 2024-09-10T23:32:18.772 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.768+0000 7f4266000640 1 -- 172.21.15.181:0/3849998491 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f4260012860 con 0x7f42700af150 2024-09-10T23:32:18.772 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.768+0000 7f426f200640 1 -- 172.21.15.181:0/3849998491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42700af5e0 con 0x7f42700af150 2024-09-10T23:32:18.772 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.768+0000 7f4266000640 1 -- 172.21.15.181:0/3849998491 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f4260021070 con 0x7f42700af150 2024-09-10T23:32:18.875 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.872+0000 7f426f200640 1 -- 172.21.15.181:0/3849998491 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea", "id": 0} v 0) -- 0x7f42701547b0 con 0x7f42700af150 2024-09-10T23:32:18.894 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.892+0000 7f4266000640 1 -- 172.21.15.181:0/3849998491 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f4260012a60 con 0x7f42700af150 2024-09-10T23:32:18.894 INFO:teuthology.orchestra.run.smithi181.stdout:0 2024-09-10T23:32:18.895 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.892+0000 7f426f200640 1 -- 172.21.15.181:0/3849998491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42700af150 msgr2=0x7f4270166100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:18.896 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.892+0000 7f426f200640 1 --2- 172.21.15.181:0/3849998491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42700af150 0x7f4270166100 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4260001ec0 tx=0x7f426000c890 comp rx=0 tx=0).stop 2024-09-10T23:32:18.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.896+0000 7f426f200640 1 -- 172.21.15.181:0/3849998491 shutdown_connections 2024-09-10T23:32:18.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.896+0000 7f426f200640 1 --2- 172.21.15.181:0/3849998491 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f427015f140 0x7f4270155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:18.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.896+0000 7f426f200640 1 --2- 172.21.15.181:0/3849998491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42700b2ac0 0x7f4270147fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:18.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.896+0000 7f426f200640 1 --2- 172.21.15.181:0/3849998491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42700af150 0x7f4270166100 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:18.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.896+0000 7f426f200640 1 -- 172.21.15.181:0/3849998491 >> 172.21.15.181:0/3849998491 conn(0x7f42700d1b00 msgr2=0x7f42700706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:18.897 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.896+0000 7f426f200640 1 -- 172.21.15.181:0/3849998491 shutdown_connections 2024-09-10T23:32:18.898 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:18.896+0000 7f426f200640 1 -- 172.21.15.181:0/3849998491 wait complete. 2024-09-10T23:32:18.924 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 870fb073-c560-4844-b8e7-594f8b27b3db 1 2024-09-10T23:32:19.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.064+0000 7fd072800640 1 -- 172.21.15.181:0/1256137652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06c0d0de0 msgr2=0x7fd06c06e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:19.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.064+0000 7fd072800640 1 --2- 172.21.15.181:0/1256137652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06c0d0de0 0x7fd06c06e270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0740d0 tx=0x7fd06401ccf0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.064+0000 7fd072800640 1 -- 172.21.15.181:0/1256137652 shutdown_connections 2024-09-10T23:32:19.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.064+0000 7fd072800640 1 --2- 172.21.15.181:0/1256137652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06c0d0de0 0x7fd06c06e270 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.064+0000 7fd072800640 1 --2- 172.21.15.181:0/1256137652 >> v2:172.21.15.94:3300/0 conn(0x7fd06c161600 0x7fd06c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.064+0000 7fd072800640 1 -- 172.21.15.181:0/1256137652 >> 172.21.15.181:0/1256137652 conn(0x7fd06c0d1c60 msgr2=0x7fd06c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:19.067 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.064+0000 7fd072800640 1 -- 172.21.15.181:0/1256137652 shutdown_connections 2024-09-10T23:32:19.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.064+0000 7fd072800640 1 -- 172.21.15.181:0/1256137652 wait complete. 2024-09-10T23:32:19.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.064+0000 7fd072800640 1 Processor -- start 2024-09-10T23:32:19.068 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.064+0000 7fd072800640 1 -- start start 2024-09-10T23:32:19.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd072800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd06c0b2960 0x7fd06c155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:19.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd072800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06c0d0de0 0x7fd06c1fa770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:19.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd072800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd06c161600 0x7fd06c15a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:19.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd072800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c16ad50 con 0x7fd06c0b2960 2024-09-10T23:32:19.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd072800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c0b7020 con 0x7fd06c0d0de0 2024-09-10T23:32:19.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd072800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd06c16aed0 con 0x7fd06c161600 2024-09-10T23:32:19.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd06be00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd06c161600 0x7fd06c15a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:19.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd06aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06c0d0de0 0x7fd06c1fa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:19.071 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd06b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd06c0b2960 0x7fd06c155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:19.072 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd06be00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd06c161600 0x7fd06c15a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:51888/0 (socket says 172.21.15.181:51888) 2024-09-10T23:32:19.072 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd06be00640 1 -- 172.21.15.181:0/1465223828 learned_addr learned my addr 172.21.15.181:0/1465223828 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T23:32:19.072 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd06be00640 1 -- 172.21.15.181:0/1465223828 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd06c161600 msgr2=0x7fd06c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T23:32:19.072 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd06be00640 1 -- 172.21.15.181:0/1465223828 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd06c161600 msgr2=0x7fd06c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T23:32:19.072 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd06be00640 1 --2- 172.21.15.181:0/1465223828 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd06c161600 0x7fd06c15a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T23:32:19.072 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd06be00640 1 --2- 172.21.15.181:0/1465223828 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd06c161600 0x7fd06c15a3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T23:32:19.072 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd06aa00640 1 -- 172.21.15.181:0/1465223828 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd06c161600 msgr2=0x7fd06c15a3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:32:19.072 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd06aa00640 1 --2- 172.21.15.181:0/1465223828 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd06c161600 0x7fd06c15a3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.072 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd06aa00640 1 -- 172.21.15.181:0/1465223828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd06c0b2960 msgr2=0x7fd06c155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:19.072 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd06aa00640 1 --2- 172.21.15.181:0/1465223828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd06c0b2960 0x7fd06c155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.072 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd06aa00640 1 -- 172.21.15.181:0/1465223828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06c1facb0 con 0x7fd06c0d0de0 2024-09-10T23:32:19.072 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd06aa00640 1 --2- 172.21.15.181:0/1465223828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06c0d0de0 0x7fd06c1fa770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd05800c3c0 tx=0x7fd05800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:19.072 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd063400640 1 -- 172.21.15.181:0/1465223828 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058017070 con 0x7fd06c0d0de0 2024-09-10T23:32:19.073 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd063400640 1 -- 172.21.15.181:0/1465223828 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0580053c0 con 0x7fd06c0d0de0 2024-09-10T23:32:19.073 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd072800640 1 -- 172.21.15.181:0/1465223828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06c14fbe0 con 0x7fd06c0d0de0 2024-09-10T23:32:19.073 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd063400640 1 -- 172.21.15.181:0/1465223828 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058003840 con 0x7fd06c0d0de0 2024-09-10T23:32:19.073 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd072800640 1 -- 172.21.15.181:0/1465223828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd06c1501a0 con 0x7fd06c0d0de0 2024-09-10T23:32:19.073 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd063400640 1 -- 172.21.15.181:0/1465223828 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fd0580039e0 con 0x7fd06c0d0de0 2024-09-10T23:32:19.073 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd063400640 1 -- 172.21.15.181:0/1465223828 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fd05801c480 con 0x7fd06c0d0de0 2024-09-10T23:32:19.073 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.068+0000 7fd072800640 1 -- 172.21.15.181:0/1465223828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd06c0b5cf0 con 0x7fd06c0d0de0 2024-09-10T23:32:19.073 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.072+0000 7fd063400640 1 -- 172.21.15.181:0/1465223828 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fd058021070 con 0x7fd06c0d0de0 2024-09-10T23:32:19.145 INFO:tasks.ceph.mgr.x.smithi181.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-10T23:32:19.145 INFO:tasks.ceph.mgr.x.smithi181.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-10T23:32:19.145 INFO:tasks.ceph.mgr.x.smithi181.stderr: from numpy import show_config as show_numpy_config 2024-09-10T23:32:19.146 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.144+0000 7fd072800640 1 -- 172.21.15.181:0/1465223828 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "870fb073-c560-4844-b8e7-594f8b27b3db", "id": 1} v 0) -- 0x7fd06c0c54b0 con 0x7fd06c0d0de0 2024-09-10T23:32:19.151 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.148+0000 7fd063400640 1 -- 172.21.15.181:0/1465223828 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "870fb073-c560-4844-b8e7-594f8b27b3db", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fd058011590 con 0x7fd06c0d0de0 2024-09-10T23:32:19.151 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-09-10T23:32:19.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.148+0000 7fd072800640 1 -- 172.21.15.181:0/1465223828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06c0d0de0 msgr2=0x7fd06c1fa770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:19.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.148+0000 7fd072800640 1 --2- 172.21.15.181:0/1465223828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06c0d0de0 0x7fd06c1fa770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd05800c3c0 tx=0x7fd05800c880 comp rx=0 tx=0).stop 2024-09-10T23:32:19.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.148+0000 7fd072800640 1 -- 172.21.15.181:0/1465223828 shutdown_connections 2024-09-10T23:32:19.152 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.148+0000 7fd072800640 1 --2- 172.21.15.181:0/1465223828 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd06c0d0de0 0x7fd06c1fa770 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.148+0000 7fd072800640 1 --2- 172.21.15.181:0/1465223828 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd06c161600 0x7fd06c15a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.148+0000 7fd072800640 1 --2- 172.21.15.181:0/1465223828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd06c0b2960 0x7fd06c155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.148+0000 7fd072800640 1 -- 172.21.15.181:0/1465223828 >> 172.21.15.181:0/1465223828 conn(0x7fd06c0d1c60 msgr2=0x7fd06c15fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:19.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.148+0000 7fd072800640 1 -- 172.21.15.181:0/1465223828 shutdown_connections 2024-09-10T23:32:19.153 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.148+0000 7fd072800640 1 -- 172.21.15.181:0/1465223828 wait complete. 2024-09-10T23:32:19.170 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 55862624-0f40-4cf4-8969-dc385ce085b3 2 2024-09-10T23:32:19.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.308+0000 7f61f9000640 1 -- 172.21.15.181:0/323251656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f40ce890 msgr2=0x7f61f40cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:19.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.308+0000 7f61f9000640 1 --2- 172.21.15.181:0/323251656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f40ce890 0x7f61f40cd710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009840 tx=0x7f61e401c790 comp rx=0 tx=0).stop 2024-09-10T23:32:19.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.308+0000 7f61f9000640 1 -- 172.21.15.181:0/323251656 shutdown_connections 2024-09-10T23:32:19.312 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.308+0000 7f61f9000640 1 --2- 172.21.15.181:0/323251656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f40ce890 0x7f61f40cd710 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.308+0000 7f61f9000640 1 -- 172.21.15.181:0/323251656 >> 172.21.15.181:0/323251656 conn(0x7f61f40d1c60 msgr2=0x7f61f40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:19.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.308+0000 7f61f9000640 1 -- 172.21.15.181:0/323251656 shutdown_connections 2024-09-10T23:32:19.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.308+0000 7f61f9000640 1 -- 172.21.15.181:0/323251656 wait complete. 2024-09-10T23:32:19.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f9000640 1 Processor -- start 2024-09-10T23:32:19.313 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f9000640 1 -- start start 2024-09-10T23:32:19.316 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f9000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61f40b1100 0x7f61f40c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:19.316 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f9000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f40ce1e0 0x7f61f414e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:19.316 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f40ce890 0x7f61f415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:19.316 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f9000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61f416a260 con 0x7f61f40ce890 2024-09-10T23:32:19.316 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f9000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f61f416a0e0 con 0x7f61f40ce1e0 2024-09-10T23:32:19.316 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f9000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f61f416a3e0 con 0x7f61f40b1100 2024-09-10T23:32:19.316 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f1e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61f40b1100 0x7f61f40c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:19.316 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61ebe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f40ce1e0 0x7f61f414e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:19.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f1e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61f40b1100 0x7f61f40c6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:51894/0 (socket says 172.21.15.181:51894) 2024-09-10T23:32:19.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f1e00640 1 -- 172.21.15.181:0/1482825396 learned_addr learned my addr 172.21.15.181:0/1482825396 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T23:32:19.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f2800640 1 --2- 172.21.15.181:0/1482825396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f40ce890 0x7f61f415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:19.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f1e00640 1 -- 172.21.15.181:0/1482825396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61f40b1100 msgr2=0x7f61f40c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T23:32:19.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f1e00640 1 -- 172.21.15.181:0/1482825396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61f40b1100 msgr2=0x7f61f40c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T23:32:19.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f1e00640 1 --2- 172.21.15.181:0/1482825396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61f40b1100 0x7f61f40c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T23:32:19.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f1e00640 1 --2- 172.21.15.181:0/1482825396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61f40b1100 0x7f61f40c6380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T23:32:19.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61ebe00640 1 -- 172.21.15.181:0/1482825396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61f40b1100 msgr2=0x7f61f40c6380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:32:19.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61ebe00640 1 --2- 172.21.15.181:0/1482825396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61f40b1100 0x7f61f40c6380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61ebe00640 1 -- 172.21.15.181:0/1482825396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f40ce890 msgr2=0x7f61f415dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:19.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61ebe00640 1 --2- 172.21.15.181:0/1482825396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f40ce890 0x7f61f415dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61ebe00640 1 -- 172.21.15.181:0/1482825396 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f40c55e0 con 0x7f61f40ce1e0 2024-09-10T23:32:19.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61ebe00640 1 --2- 172.21.15.181:0/1482825396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f40ce1e0 0x7f61f414e1b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00c410 tx=0x7f61dc00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:19.317 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61ea000640 1 -- 172.21.15.181:0/1482825396 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc017070 con 0x7f61f40ce1e0 2024-09-10T23:32:19.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61ea000640 1 -- 172.21.15.181:0/1482825396 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61dc003600 con 0x7f61f40ce1e0 2024-09-10T23:32:19.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f9000640 1 -- 172.21.15.181:0/1482825396 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f40c58a0 con 0x7f61f40ce1e0 2024-09-10T23:32:19.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61ea000640 1 -- 172.21.15.181:0/1482825396 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc003ad0 con 0x7f61f40ce1e0 2024-09-10T23:32:19.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61f9000640 1 -- 172.21.15.181:0/1482825396 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f415e100 con 0x7f61f40ce1e0 2024-09-10T23:32:19.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61ea000640 1 -- 172.21.15.181:0/1482825396 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f61dc003c70 con 0x7f61f40ce1e0 2024-09-10T23:32:19.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.312+0000 7f61ea000640 1 -- 172.21.15.181:0/1482825396 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f61dc01c860 con 0x7f61f40ce1e0 2024-09-10T23:32:19.318 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.316+0000 7f61f9000640 1 -- 172.21.15.181:0/1482825396 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f40cd710 con 0x7f61f40ce1e0 2024-09-10T23:32:19.319 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.316+0000 7f61ea000640 1 -- 172.21.15.181:0/1482825396 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f61dc021030 con 0x7f61f40ce1e0 2024-09-10T23:32:19.388 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.384+0000 7f61f9000640 1 -- 172.21.15.181:0/1482825396 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "55862624-0f40-4cf4-8969-dc385ce085b3", "id": 2} v 0) -- 0x7f61f40cc910 con 0x7f61f40ce1e0 2024-09-10T23:32:19.393 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.392+0000 7f61ea000640 1 -- 172.21.15.181:0/1482825396 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "55862624-0f40-4cf4-8969-dc385ce085b3", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f61dc01cb60 con 0x7f61f40ce1e0 2024-09-10T23:32:19.394 INFO:teuthology.orchestra.run.smithi181.stdout:2 2024-09-10T23:32:19.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.392+0000 7f61f9000640 1 -- 172.21.15.181:0/1482825396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f40ce1e0 msgr2=0x7f61f414e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:19.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.392+0000 7f61f9000640 1 --2- 172.21.15.181:0/1482825396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f40ce1e0 0x7f61f414e1b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00c410 tx=0x7f61dc00c8d0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.392+0000 7f61f9000640 1 -- 172.21.15.181:0/1482825396 shutdown_connections 2024-09-10T23:32:19.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.392+0000 7f61f9000640 1 --2- 172.21.15.181:0/1482825396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61f40ce1e0 0x7f61f414e1b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.392+0000 7f61f9000640 1 --2- 172.21.15.181:0/1482825396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f40ce890 0x7f61f415dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.392+0000 7f61f9000640 1 --2- 172.21.15.181:0/1482825396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61f40b1100 0x7f61f40c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.392+0000 7f61f9000640 1 -- 172.21.15.181:0/1482825396 >> 172.21.15.181:0/1482825396 conn(0x7f61f40d1c60 msgr2=0x7f61f4165cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:19.395 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.392+0000 7f61f9000640 1 -- 172.21.15.181:0/1482825396 shutdown_connections 2024-09-10T23:32:19.396 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.392+0000 7f61f9000640 1 -- 172.21.15.181:0/1482825396 wait complete. 2024-09-10T23:32:19.413 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 2a01b817-57fc-4068-ad96-19def0495f38 3 2024-09-10T23:32:19.555 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.552+0000 7fa4f5000640 1 -- 172.21.15.181:0/2344315594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4f0161450 msgr2=0x7fa4d4004cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:19.555 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.552+0000 7fa4f5000640 1 --2- 172.21.15.181:0/2344315594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4f0161450 0x7fa4d4004cb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000bbb0 tx=0x7fa4e001ce60 comp rx=0 tx=0).stop 2024-09-10T23:32:19.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.552+0000 7fa4ee800640 1 -- 172.21.15.181:0/2344315594 reap_dead start 2024-09-10T23:32:19.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.552+0000 7fa4f5000640 1 -- 172.21.15.181:0/2344315594 shutdown_connections 2024-09-10T23:32:19.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.552+0000 7fa4f5000640 1 -- 172.21.15.181:0/2344315594 >> 172.21.15.181:0/2344315594 conn(0x7fa4f009aa70 msgr2=0x7fa4f009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:19.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.552+0000 7fa4f5000640 1 -- 172.21.15.181:0/2344315594 shutdown_connections 2024-09-10T23:32:19.556 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.552+0000 7fa4f5000640 1 -- 172.21.15.181:0/2344315594 wait complete. 2024-09-10T23:32:19.557 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4f5000640 1 Processor -- start 2024-09-10T23:32:19.557 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4f5000640 1 -- start start 2024-09-10T23:32:19.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4f5000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4f0161450 0x7fa4f00cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:19.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4f5000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4f00b7630 0x7fa4f01531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:19.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4f5000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4f00c8d80 0x7fa4f00af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:19.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4f5000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f00cfe40 con 0x7fa4f0161450 2024-09-10T23:32:19.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4f5000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f006e520 con 0x7fa4f00b7630 2024-09-10T23:32:19.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4f5000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f00cffc0 con 0x7fa4f00c8d80 2024-09-10T23:32:19.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4ed400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4f00b7630 0x7fa4f01531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:19.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4ed400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4f00b7630 0x7fa4f01531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49104/0 (socket says 172.21.15.181:49104) 2024-09-10T23:32:19.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4ed400640 1 -- 172.21.15.181:0/3000315705 learned_addr learned my addr 172.21.15.181:0/3000315705 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T23:32:19.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4ee800640 1 --2- 172.21.15.181:0/3000315705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4f00c8d80 0x7fa4f00af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:19.560 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4ede00640 1 --2- 172.21.15.181:0/3000315705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4f0161450 0x7fa4f00cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:19.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4ee800640 1 -- 172.21.15.181:0/3000315705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4f00c8d80 msgr2=0x7fa4f00af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T23:32:19.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4ee800640 1 -- 172.21.15.181:0/3000315705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4f00c8d80 msgr2=0x7fa4f00af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T23:32:19.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4ee800640 1 --2- 172.21.15.181:0/3000315705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4f00c8d80 0x7fa4f00af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T23:32:19.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4ee800640 1 --2- 172.21.15.181:0/3000315705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4f00c8d80 0x7fa4f00af9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T23:32:19.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4ed400640 1 -- 172.21.15.181:0/3000315705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4f00c8d80 msgr2=0x7fa4f00af9d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:32:19.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4ed400640 1 --2- 172.21.15.181:0/3000315705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4f00c8d80 0x7fa4f00af9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4ed400640 1 -- 172.21.15.181:0/3000315705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4f0161450 msgr2=0x7fa4f00cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:19.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4ed400640 1 --2- 172.21.15.181:0/3000315705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4f0161450 0x7fa4f00cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4ed400640 1 -- 172.21.15.181:0/3000315705 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f00ca8f0 con 0x7fa4f00b7630 2024-09-10T23:32:19.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4ed400640 1 --2- 172.21.15.181:0/3000315705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4f00b7630 0x7fa4f01531a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc006fd0 tx=0x7fa4dc003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:19.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4e6000640 1 -- 172.21.15.181:0/3000315705 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc015070 con 0x7fa4f00b7630 2024-09-10T23:32:19.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4e6000640 1 -- 172.21.15.181:0/3000315705 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4dc00faa0 con 0x7fa4f00b7630 2024-09-10T23:32:19.561 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4e6000640 1 -- 172.21.15.181:0/3000315705 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc011430 con 0x7fa4f00b7630 2024-09-10T23:32:19.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4f5000640 1 -- 172.21.15.181:0/3000315705 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f00cabd0 con 0x7fa4f00b7630 2024-09-10T23:32:19.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4f5000640 1 -- 172.21.15.181:0/3000315705 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f00cb190 con 0x7fa4f00b7630 2024-09-10T23:32:19.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4e6000640 1 -- 172.21.15.181:0/3000315705 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fa4dc011960 con 0x7fa4f00b7630 2024-09-10T23:32:19.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4e6000640 1 -- 172.21.15.181:0/3000315705 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fa4dc011c80 con 0x7fa4f00b7630 2024-09-10T23:32:19.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.556+0000 7fa4f5000640 1 -- 172.21.15.181:0/3000315705 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f00b19f0 con 0x7fa4f00b7630 2024-09-10T23:32:19.562 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.560+0000 7fa4e6000640 1 -- 172.21.15.181:0/3000315705 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fa4dc010be0 con 0x7fa4f00b7630 2024-09-10T23:32:19.632 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.628+0000 7fa4f5000640 1 -- 172.21.15.181:0/3000315705 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2a01b817-57fc-4068-ad96-19def0495f38", "id": 3} v 0) -- 0x7fa4f0071680 con 0x7fa4f00b7630 2024-09-10T23:32:19.636 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.632+0000 7fa4e6000640 1 -- 172.21.15.181:0/3000315705 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2a01b817-57fc-4068-ad96-19def0495f38", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fa4dc010e00 con 0x7fa4f00b7630 2024-09-10T23:32:19.636 INFO:teuthology.orchestra.run.smithi181.stdout:3 2024-09-10T23:32:19.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.636+0000 7fa4f5000640 1 -- 172.21.15.181:0/3000315705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4f00b7630 msgr2=0x7fa4f01531a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:19.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.636+0000 7fa4f5000640 1 --2- 172.21.15.181:0/3000315705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4f00b7630 0x7fa4f01531a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc006fd0 tx=0x7fa4dc003a90 comp rx=0 tx=0).stop 2024-09-10T23:32:19.637 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.636+0000 7fa4f5000640 1 -- 172.21.15.181:0/3000315705 shutdown_connections 2024-09-10T23:32:19.638 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.636+0000 7fa4f5000640 1 --2- 172.21.15.181:0/3000315705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4f00b7630 0x7fa4f01531a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.638 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.636+0000 7fa4f5000640 1 --2- 172.21.15.181:0/3000315705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4f00c8d80 0x7fa4f00af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.638 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.636+0000 7fa4f5000640 1 --2- 172.21.15.181:0/3000315705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4f0161450 0x7fa4f00cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.638 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.636+0000 7fa4f5000640 1 -- 172.21.15.181:0/3000315705 >> 172.21.15.181:0/3000315705 conn(0x7fa4f009aa70 msgr2=0x7fa4f0071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:19.638 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.636+0000 7fa4f5000640 1 -- 172.21.15.181:0/3000315705 shutdown_connections 2024-09-10T23:32:19.638 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.636+0000 7fa4f5000640 1 -- 172.21.15.181:0/3000315705 wait complete. 2024-09-10T23:32:19.655 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 6362ed7b-8d59-474a-9e4c-6459c133b664 4 2024-09-10T23:32:19.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.792+0000 7f3ea3400640 1 -- 172.21.15.181:0/974284591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c073b70 msgr2=0x7f3e9c0cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:19.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.792+0000 7f3ea3400640 1 --2- 172.21.15.181:0/974284591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c073b70 0x7f3e9c0cbc10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009840 tx=0x7f3e8801c820 comp rx=0 tx=0).stop 2024-09-10T23:32:19.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.792+0000 7f3ea3400640 1 -- 172.21.15.181:0/974284591 shutdown_connections 2024-09-10T23:32:19.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.792+0000 7f3ea3400640 1 --2- 172.21.15.181:0/974284591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c073b70 0x7f3e9c0cbc10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.792+0000 7f3ea3400640 1 --2- 172.21.15.181:0/974284591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e9c15ab20 0x7f3e9c073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.792+0000 7f3ea3400640 1 -- 172.21.15.181:0/974284591 >> 172.21.15.181:0/974284591 conn(0x7f3e9c0d0c00 msgr2=0x7f3e9c0d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:19.797 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea3400640 1 -- 172.21.15.181:0/974284591 shutdown_connections 2024-09-10T23:32:19.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea3400640 1 -- 172.21.15.181:0/974284591 wait complete. 2024-09-10T23:32:19.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea3400640 1 Processor -- start 2024-09-10T23:32:19.798 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea3400640 1 -- start start 2024-09-10T23:32:19.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c073b70 0x7f3e9c152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:19.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea3400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e9c0cc150 0x7f3e9c1faac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:19.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e9c15ab20 0x7f3e9c1fb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:19.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea3400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c06cda0 con 0x7f3e9c073b70 2024-09-10T23:32:19.801 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea3400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c06cc20 con 0x7f3e9c15ab20 2024-09-10T23:32:19.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea3400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3e9c06cf20 con 0x7f3e9c0cc150 2024-09-10T23:32:19.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3e9be00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e9c0cc150 0x7f3e9c1faac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:19.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3e9be00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e9c0cc150 0x7f3e9c1faac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:51922/0 (socket says 172.21.15.181:51922) 2024-09-10T23:32:19.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3e9be00640 1 -- 172.21.15.181:0/300380513 learned_addr learned my addr 172.21.15.181:0/300380513 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T23:32:19.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e9c15ab20 0x7f3e9c1fb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:19.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea0a00640 1 --2- 172.21.15.181:0/300380513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c073b70 0x7f3e9c152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:19.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3e9be00640 1 -- 172.21.15.181:0/300380513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e9c0cc150 msgr2=0x7f3e9c1faac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T23:32:19.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3e9be00640 1 -- 172.21.15.181:0/300380513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e9c0cc150 msgr2=0x7f3e9c1faac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T23:32:19.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3e9be00640 1 --2- 172.21.15.181:0/300380513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e9c0cc150 0x7f3e9c1faac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T23:32:19.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3e9be00640 1 --2- 172.21.15.181:0/300380513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e9c0cc150 0x7f3e9c1faac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T23:32:19.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea1400640 1 -- 172.21.15.181:0/300380513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e9c0cc150 msgr2=0x7f3e9c1faac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:32:19.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea1400640 1 --2- 172.21.15.181:0/300380513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e9c0cc150 0x7f3e9c1faac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.802 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea1400640 1 -- 172.21.15.181:0/300380513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c073b70 msgr2=0x7f3e9c152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:19.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea1400640 1 --2- 172.21.15.181:0/300380513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c073b70 0x7f3e9c152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea1400640 1 -- 172.21.15.181:0/300380513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9c1fc040 con 0x7f3e9c15ab20 2024-09-10T23:32:19.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea0a00640 1 --2- 172.21.15.181:0/300380513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c073b70 0x7f3e9c152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:32:19.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3ea1400640 1 --2- 172.21.15.181:0/300380513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e9c15ab20 0x7f3e9c1fb200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400caa0 tx=0x7f3e9400cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:19.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.796+0000 7f3e98c00640 1 -- 172.21.15.181:0/300380513 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94016020 con 0x7f3e9c15ab20 2024-09-10T23:32:19.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.800+0000 7f3e98c00640 1 -- 172.21.15.181:0/300380513 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e9400a950 con 0x7f3e9c15ab20 2024-09-10T23:32:19.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.800+0000 7f3ea3400640 1 -- 172.21.15.181:0/300380513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e9c154f10 con 0x7f3e9c15ab20 2024-09-10T23:32:19.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.800+0000 7f3e98c00640 1 -- 172.21.15.181:0/300380513 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9400ada0 con 0x7f3e9c15ab20 2024-09-10T23:32:19.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.800+0000 7f3ea3400640 1 -- 172.21.15.181:0/300380513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e9c154be0 con 0x7f3e9c15ab20 2024-09-10T23:32:19.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.800+0000 7f3e98c00640 1 -- 172.21.15.181:0/300380513 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f3e940129c0 con 0x7f3e9c15ab20 2024-09-10T23:32:19.803 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.800+0000 7f3e98c00640 1 -- 172.21.15.181:0/300380513 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f3e9401bdb0 con 0x7f3e9c15ab20 2024-09-10T23:32:19.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.800+0000 7f3ea3400640 1 -- 172.21.15.181:0/300380513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e9c0cbc10 con 0x7f3e9c15ab20 2024-09-10T23:32:19.804 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.800+0000 7f3e98c00640 1 -- 172.21.15.181:0/300380513 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f3e94020020 con 0x7f3e9c15ab20 2024-09-10T23:32:19.872 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.868+0000 7f3ea3400640 1 -- 172.21.15.181:0/300380513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6362ed7b-8d59-474a-9e4c-6459c133b664", "id": 4} v 0) -- 0x7f3e9c0b6b80 con 0x7f3e9c15ab20 2024-09-10T23:32:19.877 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.876+0000 7f3e98c00640 1 -- 172.21.15.181:0/300380513 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6362ed7b-8d59-474a-9e4c-6459c133b664", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f3e94012db0 con 0x7f3e9c15ab20 2024-09-10T23:32:19.877 INFO:teuthology.orchestra.run.smithi181.stdout:4 2024-09-10T23:32:19.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.876+0000 7f3ea3400640 1 -- 172.21.15.181:0/300380513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e9c15ab20 msgr2=0x7f3e9c1fb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:19.878 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.876+0000 7f3ea3400640 1 --2- 172.21.15.181:0/300380513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e9c15ab20 0x7f3e9c1fb200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400caa0 tx=0x7f3e9400cb90 comp rx=0 tx=0).stop 2024-09-10T23:32:19.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.876+0000 7f3ea3400640 1 -- 172.21.15.181:0/300380513 shutdown_connections 2024-09-10T23:32:19.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.876+0000 7f3ea3400640 1 --2- 172.21.15.181:0/300380513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e9c15ab20 0x7f3e9c1fb200 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.876+0000 7f3ea3400640 1 --2- 172.21.15.181:0/300380513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e9c0cc150 0x7f3e9c1faac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.876+0000 7f3ea3400640 1 --2- 172.21.15.181:0/300380513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e9c073b70 0x7f3e9c152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:19.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.876+0000 7f3ea3400640 1 -- 172.21.15.181:0/300380513 >> 172.21.15.181:0/300380513 conn(0x7f3e9c0d0c00 msgr2=0x7f3e9c071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:19.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.876+0000 7f3ea3400640 1 -- 172.21.15.181:0/300380513 shutdown_connections 2024-09-10T23:32:19.879 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:19.876+0000 7f3ea3400640 1 -- 172.21.15.181:0/300380513 wait complete. 2024-09-10T23:32:19.897 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new bee62996-8fff-4730-9568-8f33784497c8 5 2024-09-10T23:32:19.995 INFO:tasks.ceph.mgr.y.smithi094.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-10T23:32:19.995 INFO:tasks.ceph.mgr.y.smithi094.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-10T23:32:19.995 INFO:tasks.ceph.mgr.y.smithi094.stderr: from numpy import show_config as show_numpy_config 2024-09-10T23:32:20.037 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.032+0000 7f2a6cc00640 1 -- 172.21.15.181:0/4162149354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a680d06e0 msgr2=0x7f2a6815e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:20.037 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.032+0000 7f2a6cc00640 1 --2- 172.21.15.181:0/4162149354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a680d06e0 0x7f2a6815e620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009900 tx=0x7f2a5001c8e0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.037 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.032+0000 7f2a6cc00640 1 -- 172.21.15.181:0/4162149354 shutdown_connections 2024-09-10T23:32:20.037 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.032+0000 7f2a6cc00640 1 --2- 172.21.15.181:0/4162149354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a68073630 0x7f2a68073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.037 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.032+0000 7f2a6cc00640 1 --2- 172.21.15.181:0/4162149354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a680d06e0 0x7f2a6815e620 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.037 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.032+0000 7f2a6cc00640 1 -- 172.21.15.181:0/4162149354 >> 172.21.15.181:0/4162149354 conn(0x7f2a680b27c0 msgr2=0x7f2a680b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:20.037 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.032+0000 7f2a6cc00640 1 -- 172.21.15.181:0/4162149354 shutdown_connections 2024-09-10T23:32:20.037 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.032+0000 7f2a6cc00640 1 -- 172.21.15.181:0/4162149354 wait complete. 2024-09-10T23:32:20.038 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a6cc00640 1 Processor -- start 2024-09-10T23:32:20.038 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a6cc00640 1 -- start start 2024-09-10T23:32:20.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a6cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a68073630 0x7f2a681fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:20.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a6cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a680d06e0 0x7f2a6815a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:20.040 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a6cc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a6815df60 0x7f2a681509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:20.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a6cc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a68167c50 con 0x7f2a68073630 2024-09-10T23:32:20.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a6cc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2a68167ad0 con 0x7f2a680d06e0 2024-09-10T23:32:20.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a6cc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2a68167dd0 con 0x7f2a6815df60 2024-09-10T23:32:20.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a5fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a680d06e0 0x7f2a6815a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:20.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a5fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a680d06e0 0x7f2a6815a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49138/0 (socket says 172.21.15.181:49138) 2024-09-10T23:32:20.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a5fe00640 1 -- 172.21.15.181:0/2928699887 learned_addr learned my addr 172.21.15.181:0/2928699887 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T23:32:20.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a66800640 1 --2- 172.21.15.181:0/2928699887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a6815df60 0x7f2a681509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:20.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a65e00640 1 --2- 172.21.15.181:0/2928699887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a68073630 0x7f2a681fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:20.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a66800640 1 -- 172.21.15.181:0/2928699887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a6815df60 msgr2=0x7f2a681509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T23:32:20.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a66800640 1 -- 172.21.15.181:0/2928699887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a6815df60 msgr2=0x7f2a681509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T23:32:20.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a66800640 1 --2- 172.21.15.181:0/2928699887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a6815df60 0x7f2a681509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T23:32:20.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a66800640 1 --2- 172.21.15.181:0/2928699887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a6815df60 0x7f2a681509f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T23:32:20.041 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a5fe00640 1 -- 172.21.15.181:0/2928699887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a6815df60 msgr2=0x7f2a681509f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:32:20.042 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a5fe00640 1 --2- 172.21.15.181:0/2928699887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a6815df60 0x7f2a681509f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.042 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a5fe00640 1 -- 172.21.15.181:0/2928699887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a68073630 msgr2=0x7f2a681fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:20.042 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a5fe00640 1 --2- 172.21.15.181:0/2928699887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a68073630 0x7f2a681fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.042 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a5fe00640 1 -- 172.21.15.181:0/2928699887 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a68151360 con 0x7f2a680d06e0 2024-09-10T23:32:20.042 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a65e00640 1 --2- 172.21.15.181:0/2928699887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a68073630 0x7f2a681fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:20.042 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a5fe00640 1 --2- 172.21.15.181:0/2928699887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a680d06e0 0x7f2a6815a960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009850 tx=0x7f2a50004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:20.042 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a5e000640 1 -- 172.21.15.181:0/2928699887 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a500b2040 con 0x7f2a680d06e0 2024-09-10T23:32:20.042 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a5e000640 1 -- 172.21.15.181:0/2928699887 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a50005030 con 0x7f2a680d06e0 2024-09-10T23:32:20.042 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a6cc00640 1 -- 172.21.15.181:0/2928699887 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a68151640 con 0x7f2a680d06e0 2024-09-10T23:32:20.042 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a5e000640 1 -- 172.21.15.181:0/2928699887 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a50005440 con 0x7f2a680d06e0 2024-09-10T23:32:20.042 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.036+0000 7f2a6cc00640 1 -- 172.21.15.181:0/2928699887 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a681fb380 con 0x7f2a680d06e0 2024-09-10T23:32:20.043 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.040+0000 7f2a5e000640 1 -- 172.21.15.181:0/2928699887 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f2a50004a60 con 0x7f2a680d06e0 2024-09-10T23:32:20.043 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.040+0000 7f2a5e000640 1 -- 172.21.15.181:0/2928699887 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f2a50003c40 con 0x7f2a680d06e0 2024-09-10T23:32:20.043 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.040+0000 7f2a6cc00640 1 -- 172.21.15.181:0/2928699887 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a68073a90 con 0x7f2a680d06e0 2024-09-10T23:32:20.043 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.040+0000 7f2a5e000640 1 -- 172.21.15.181:0/2928699887 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f2a500bc060 con 0x7f2a680d06e0 2024-09-10T23:32:20.116 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.112+0000 7f2a6cc00640 1 -- 172.21.15.181:0/2928699887 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bee62996-8fff-4730-9568-8f33784497c8", "id": 5} v 0) -- 0x7f2a68070dc0 con 0x7f2a680d06e0 2024-09-10T23:32:20.120 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.116+0000 7f2a5e000640 1 -- 172.21.15.181:0/2928699887 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bee62996-8fff-4730-9568-8f33784497c8", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f2a500164d0 con 0x7f2a680d06e0 2024-09-10T23:32:20.120 INFO:teuthology.orchestra.run.smithi181.stdout:5 2024-09-10T23:32:20.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.120+0000 7f2a6cc00640 1 -- 172.21.15.181:0/2928699887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a680d06e0 msgr2=0x7f2a6815a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:20.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.120+0000 7f2a6cc00640 1 --2- 172.21.15.181:0/2928699887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a680d06e0 0x7f2a6815a960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009850 tx=0x7f2a50004040 comp rx=0 tx=0).stop 2024-09-10T23:32:20.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.120+0000 7f2a6cc00640 1 -- 172.21.15.181:0/2928699887 shutdown_connections 2024-09-10T23:32:20.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.120+0000 7f2a6cc00640 1 --2- 172.21.15.181:0/2928699887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a680d06e0 0x7f2a6815a960 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.121 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.120+0000 7f2a6cc00640 1 --2- 172.21.15.181:0/2928699887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a6815df60 0x7f2a681509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.120+0000 7f2a6cc00640 1 --2- 172.21.15.181:0/2928699887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a68073630 0x7f2a681fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.120+0000 7f2a6cc00640 1 -- 172.21.15.181:0/2928699887 >> 172.21.15.181:0/2928699887 conn(0x7f2a680b27c0 msgr2=0x7f2a680ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:20.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.120+0000 7f2a6cc00640 1 -- 172.21.15.181:0/2928699887 shutdown_connections 2024-09-10T23:32:20.122 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.120+0000 7f2a6cc00640 1 -- 172.21.15.181:0/2928699887 wait complete. 2024-09-10T23:32:20.140 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 59d2093f-f326-45e0-b9a5-947eaf9ded46 6 2024-09-10T23:32:20.284 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.280+0000 7f9c84000640 1 -- 172.21.15.181:0/930850598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c64003bf0 msgr2=0x7f9c64003ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:20.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.280+0000 7f9c84000640 1 --2- 172.21.15.181:0/930850598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c64003bf0 0x7f9c64003ff0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c074f80 tx=0x7f9c7001c880 comp rx=0 tx=0).stop 2024-09-10T23:32:20.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.280+0000 7f9c82000640 1 -- 172.21.15.181:0/930850598 reap_dead start 2024-09-10T23:32:20.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.280+0000 7f9c84000640 1 -- 172.21.15.181:0/930850598 shutdown_connections 2024-09-10T23:32:20.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.280+0000 7f9c84000640 1 -- 172.21.15.181:0/930850598 >> 172.21.15.181:0/930850598 conn(0x7f9c7c0b27c0 msgr2=0x7f9c7c0b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:20.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.280+0000 7f9c84000640 1 -- 172.21.15.181:0/930850598 shutdown_connections 2024-09-10T23:32:20.285 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.280+0000 7f9c84000640 1 -- 172.21.15.181:0/930850598 wait complete. 2024-09-10T23:32:20.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.284+0000 7f9c84000640 1 Processor -- start 2024-09-10T23:32:20.287 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.284+0000 7f9c84000640 1 -- start start 2024-09-10T23:32:20.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.284+0000 7f9c84000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c64003bf0 0x7f9c7c1f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:20.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.284+0000 7f9c84000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c7c15bad0 0x7f9c7c1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:20.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.284+0000 7f9c84000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c7c1506f0 0x7f9c7c1f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:20.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.284+0000 7f9c84000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c070040 con 0x7f9c7c1506f0 2024-09-10T23:32:20.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.284+0000 7f9c84000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c1606d0 con 0x7f9c64003bf0 2024-09-10T23:32:20.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.284+0000 7f9c84000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9c7c0701c0 con 0x7f9c7c15bad0 2024-09-10T23:32:20.290 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c81600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c64003bf0 0x7f9c7c1f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c80c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c7c15bad0 0x7f9c7c1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c81600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c64003bf0 0x7f9c7c1f5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49164/0 (socket says 172.21.15.181:49164) 2024-09-10T23:32:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c81600640 1 -- 172.21.15.181:0/3710834777 learned_addr learned my addr 172.21.15.181:0/3710834777 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T23:32:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c82000640 1 --2- 172.21.15.181:0/3710834777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c7c1506f0 0x7f9c7c1f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c80c00640 1 -- 172.21.15.181:0/3710834777 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c7c15bad0 msgr2=0x7f9c7c1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T23:32:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c80c00640 1 -- 172.21.15.181:0/3710834777 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c7c15bad0 msgr2=0x7f9c7c1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T23:32:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c80c00640 1 --2- 172.21.15.181:0/3710834777 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c7c15bad0 0x7f9c7c1569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T23:32:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c80c00640 1 --2- 172.21.15.181:0/3710834777 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c7c15bad0 0x7f9c7c1569b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T23:32:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c81600640 1 -- 172.21.15.181:0/3710834777 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c7c15bad0 msgr2=0x7f9c7c1569b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:32:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c81600640 1 --2- 172.21.15.181:0/3710834777 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c7c15bad0 0x7f9c7c1569b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c81600640 1 -- 172.21.15.181:0/3710834777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c7c1506f0 msgr2=0x7f9c7c1f2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c81600640 1 --2- 172.21.15.181:0/3710834777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c7c1506f0 0x7f9c7c1f2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c81600640 1 -- 172.21.15.181:0/3710834777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7c1f6220 con 0x7f9c64003bf0 2024-09-10T23:32:20.291 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c81600640 1 --2- 172.21.15.181:0/3710834777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c64003bf0 0x7f9c7c1f5c20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9c70098440 tx=0x7f9c700a5ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c79600640 1 -- 172.21.15.181:0/3710834777 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c700014a0 con 0x7f9c64003bf0 2024-09-10T23:32:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c79600640 1 -- 172.21.15.181:0/3710834777 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c700038f0 con 0x7f9c64003bf0 2024-09-10T23:32:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c79600640 1 -- 172.21.15.181:0/3710834777 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c700068f0 con 0x7f9c64003bf0 2024-09-10T23:32:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c84000640 1 -- 172.21.15.181:0/3710834777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7c1f6500 con 0x7f9c64003bf0 2024-09-10T23:32:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c84000640 1 -- 172.21.15.181:0/3710834777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c7c1f6ac0 con 0x7f9c64003bf0 2024-09-10T23:32:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c79600640 1 -- 172.21.15.181:0/3710834777 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f9c700b9420 con 0x7f9c64003bf0 2024-09-10T23:32:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c79600640 1 -- 172.21.15.181:0/3710834777 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f9c700b9d30 con 0x7f9c64003bf0 2024-09-10T23:32:20.292 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.288+0000 7f9c84000640 1 -- 172.21.15.181:0/3710834777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c7c0cf060 con 0x7f9c64003bf0 2024-09-10T23:32:20.293 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.292+0000 7f9c79600640 1 -- 172.21.15.181:0/3710834777 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f9c70016630 con 0x7f9c64003bf0 2024-09-10T23:32:20.363 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.360+0000 7f9c84000640 1 -- 172.21.15.181:0/3710834777 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "59d2093f-f326-45e0-b9a5-947eaf9ded46", "id": 6} v 0) -- 0x7f9c7c15fd80 con 0x7f9c64003bf0 2024-09-10T23:32:20.367 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.364+0000 7f9c79600640 1 -- 172.21.15.181:0/3710834777 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "59d2093f-f326-45e0-b9a5-947eaf9ded46", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f9c700b9740 con 0x7f9c64003bf0 2024-09-10T23:32:20.367 INFO:teuthology.orchestra.run.smithi181.stdout:6 2024-09-10T23:32:20.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.364+0000 7f9c84000640 1 -- 172.21.15.181:0/3710834777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c64003bf0 msgr2=0x7f9c7c1f5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:20.368 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.364+0000 7f9c84000640 1 --2- 172.21.15.181:0/3710834777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c64003bf0 0x7f9c7c1f5c20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9c70098440 tx=0x7f9c700a5ce0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.364+0000 7f9c84000640 1 -- 172.21.15.181:0/3710834777 shutdown_connections 2024-09-10T23:32:20.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.364+0000 7f9c84000640 1 --2- 172.21.15.181:0/3710834777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c7c1506f0 0x7f9c7c1f2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.364+0000 7f9c84000640 1 --2- 172.21.15.181:0/3710834777 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c7c15bad0 0x7f9c7c1569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.364+0000 7f9c84000640 1 --2- 172.21.15.181:0/3710834777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c64003bf0 0x7f9c7c1f5c20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.364+0000 7f9c84000640 1 -- 172.21.15.181:0/3710834777 >> 172.21.15.181:0/3710834777 conn(0x7f9c7c0b27c0 msgr2=0x7f9c7c06cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:20.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.364+0000 7f9c84000640 1 -- 172.21.15.181:0/3710834777 shutdown_connections 2024-09-10T23:32:20.369 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.364+0000 7f9c84000640 1 -- 172.21.15.181:0/3710834777 wait complete. 2024-09-10T23:32:20.387 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 20c823a4-fe17-4261-8074-e8274fdb9274 7 2024-09-10T23:32:20.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.532+0000 7ff2d1200640 1 -- 172.21.15.181:0/3465068852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2b0004da0 msgr2=0x7ff2b0005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:20.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.532+0000 7ff2d1200640 1 --2- 172.21.15.181:0/3465068852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2b0004da0 0x7ff2b0005220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40093c0 tx=0x7ff2c401d110 comp rx=0 tx=0).stop 2024-09-10T23:32:20.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.532+0000 7ff2ca800640 1 -- 172.21.15.181:0/3465068852 reap_dead start 2024-09-10T23:32:20.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.532+0000 7ff2d1200640 1 -- 172.21.15.181:0/3465068852 shutdown_connections 2024-09-10T23:32:20.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.532+0000 7ff2d1200640 1 -- 172.21.15.181:0/3465068852 >> 172.21.15.181:0/3465068852 conn(0x7ff2cc0d1c60 msgr2=0x7ff2cc0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:20.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.532+0000 7ff2d1200640 1 -- 172.21.15.181:0/3465068852 shutdown_connections 2024-09-10T23:32:20.535 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.532+0000 7ff2d1200640 1 -- 172.21.15.181:0/3465068852 wait complete. 2024-09-10T23:32:20.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.532+0000 7ff2d1200640 1 Processor -- start 2024-09-10T23:32:20.536 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.532+0000 7ff2d1200640 1 -- start start 2024-09-10T23:32:20.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2d1200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2b0004da0 0x7ff2cc164740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:20.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2d1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2cc06e740 0x7ff2cc0b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:20.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2d1200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2cc0b64f0 0x7ff2cc06f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:20.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2d1200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc1683b0 con 0x7ff2b0004da0 2024-09-10T23:32:20.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2d1200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc168230 con 0x7ff2cc06e740 2024-09-10T23:32:20.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2d1200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff2cc168530 con 0x7ff2cc0b64f0 2024-09-10T23:32:20.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2c9400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2cc06e740 0x7ff2cc0b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:20.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2ca800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2cc0b64f0 0x7ff2cc06f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:20.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2c9400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2cc06e740 0x7ff2cc0b5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:49178/0 (socket says 172.21.15.181:49178) 2024-09-10T23:32:20.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2c9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2b0004da0 0x7ff2cc164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:20.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2ca800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2cc0b64f0 0x7ff2cc06f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:51988/0 (socket says 172.21.15.181:51988) 2024-09-10T23:32:20.539 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2c9400640 1 -- 172.21.15.181:0/2486734911 learned_addr learned my addr 172.21.15.181:0/2486734911 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-10T23:32:20.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2ca800640 1 -- 172.21.15.181:0/2486734911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2cc0b64f0 msgr2=0x7ff2cc06f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T23:32:20.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2ca800640 1 -- 172.21.15.181:0/2486734911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2cc0b64f0 msgr2=0x7ff2cc06f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T23:32:20.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2ca800640 1 --2- 172.21.15.181:0/2486734911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2cc0b64f0 0x7ff2cc06f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T23:32:20.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2ca800640 1 --2- 172.21.15.181:0/2486734911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2cc0b64f0 0x7ff2cc06f320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T23:32:20.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2c9400640 1 -- 172.21.15.181:0/2486734911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2cc0b64f0 msgr2=0x7ff2cc06f320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:32:20.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2c9400640 1 --2- 172.21.15.181:0/2486734911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2cc0b64f0 0x7ff2cc06f320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2c9400640 1 -- 172.21.15.181:0/2486734911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2b0004da0 msgr2=0x7ff2cc164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:20.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2c9400640 1 --2- 172.21.15.181:0/2486734911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2b0004da0 0x7ff2cc164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2c9400640 1 -- 172.21.15.181:0/2486734911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2cc164c80 con 0x7ff2cc06e740 2024-09-10T23:32:20.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2c9400640 1 --2- 172.21.15.181:0/2486734911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2cc06e740 0x7ff2cc0b5fb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8003f60 tx=0x7ff2b8003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:20.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2c2000640 1 -- 172.21.15.181:0/2486734911 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8015070 con 0x7ff2cc06e740 2024-09-10T23:32:20.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2d1200640 1 -- 172.21.15.181:0/2486734911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2cc1595c0 con 0x7ff2cc06e740 2024-09-10T23:32:20.540 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2c2000640 1 -- 172.21.15.181:0/2486734911 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2b8011900 con 0x7ff2cc06e740 2024-09-10T23:32:20.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2c2000640 1 -- 172.21.15.181:0/2486734911 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8011d50 con 0x7ff2cc06e740 2024-09-10T23:32:20.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2d1200640 1 -- 172.21.15.181:0/2486734911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2cc156300 con 0x7ff2cc06e740 2024-09-10T23:32:20.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2c2000640 1 -- 172.21.15.181:0/2486734911 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7ff2b800f7c0 con 0x7ff2cc06e740 2024-09-10T23:32:20.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2c2000640 1 -- 172.21.15.181:0/2486734911 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7ff2b80103c0 con 0x7ff2cc06e740 2024-09-10T23:32:20.541 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.536+0000 7ff2d1200640 1 -- 172.21.15.181:0/2486734911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2cc159ae0 con 0x7ff2cc06e740 2024-09-10T23:32:20.543 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.540+0000 7ff2c2000640 1 -- 172.21.15.181:0/2486734911 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7ff2cc159ae0 con 0x7ff2cc06e740 2024-09-10T23:32:20.612 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.608+0000 7ff2d1200640 1 -- 172.21.15.181:0/2486734911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "20c823a4-fe17-4261-8074-e8274fdb9274", "id": 7} v 0) -- 0x7ff2cc070900 con 0x7ff2cc06e740 2024-09-10T23:32:20.616 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.612+0000 7ff2c2000640 1 -- 172.21.15.181:0/2486734911 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "20c823a4-fe17-4261-8074-e8274fdb9274", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7ff2b800fae0 con 0x7ff2cc06e740 2024-09-10T23:32:20.616 INFO:teuthology.orchestra.run.smithi181.stdout:7 2024-09-10T23:32:20.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.616+0000 7ff2d1200640 1 -- 172.21.15.181:0/2486734911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2cc06e740 msgr2=0x7ff2cc0b5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:20.617 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.616+0000 7ff2d1200640 1 --2- 172.21.15.181:0/2486734911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2cc06e740 0x7ff2cc0b5fb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8003f60 tx=0x7ff2b8003f90 comp rx=0 tx=0).stop 2024-09-10T23:32:20.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.616+0000 7ff2d1200640 1 -- 172.21.15.181:0/2486734911 shutdown_connections 2024-09-10T23:32:20.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.616+0000 7ff2d1200640 1 --2- 172.21.15.181:0/2486734911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2cc06e740 0x7ff2cc0b5fb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.616+0000 7ff2d1200640 1 --2- 172.21.15.181:0/2486734911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2cc0b64f0 0x7ff2cc06f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.616+0000 7ff2d1200640 1 --2- 172.21.15.181:0/2486734911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2b0004da0 0x7ff2cc164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.616+0000 7ff2d1200640 1 -- 172.21.15.181:0/2486734911 >> 172.21.15.181:0/2486734911 conn(0x7ff2cc0d1c60 msgr2=0x7ff2cc0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:20.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.616+0000 7ff2d1200640 1 -- 172.21.15.181:0/2486734911 shutdown_connections 2024-09-10T23:32:20.618 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:32:20.616+0000 7ff2d1200640 1 -- 172.21.15.181:0/2486734911 wait complete. 2024-09-10T23:32:20.636 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T23:32:20.636 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-10T23:32:20.639 INFO:tasks.ceph.osd.0:Started 2024-09-10T23:32:20.639 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T23:32:20.639 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T23:32:20.642 INFO:tasks.ceph.osd.1:Started 2024-09-10T23:32:20.642 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T23:32:20.642 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T23:32:20.645 INFO:tasks.ceph.osd.2:Started 2024-09-10T23:32:20.645 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-10T23:32:20.645 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-10T23:32:20.649 INFO:tasks.ceph.osd.3:Started 2024-09-10T23:32:20.649 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-10T23:32:20.649 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-10T23:32:20.652 INFO:tasks.ceph.osd.4:Started 2024-09-10T23:32:20.652 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-10T23:32:20.652 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-10T23:32:20.655 INFO:tasks.ceph.osd.5:Started 2024-09-10T23:32:20.655 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-10T23:32:20.655 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-10T23:32:20.658 INFO:tasks.ceph.osd.6:Started 2024-09-10T23:32:20.658 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-10T23:32:20.658 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-10T23:32:20.662 INFO:tasks.ceph.osd.7:Started 2024-09-10T23:32:20.663 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:32:20.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.838+0000 7f6cd6c00640 1 -- 172.21.15.94:0/2551440461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb4003c70 msgr2=0x7f6cb4004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:20.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.838+0000 7f6cd6c00640 1 --2- 172.21.15.94:0/2551440461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb4003c70 0x7f6cb4004070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000baf0 tx=0x7f6cc001cd80 comp rx=0 tx=0).stop 2024-09-10T23:32:20.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.838+0000 7f6cd4c00640 1 -- 172.21.15.94:0/2551440461 reap_dead start 2024-09-10T23:32:20.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.838+0000 7f6cd6c00640 1 -- 172.21.15.94:0/2551440461 shutdown_connections 2024-09-10T23:32:20.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.838+0000 7f6cd6c00640 1 -- 172.21.15.94:0/2551440461 >> 172.21.15.94:0/2551440461 conn(0x7f6cd006cdf0 msgr2=0x7f6cd006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:20.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.838+0000 7f6cd6c00640 1 -- 172.21.15.94:0/2551440461 shutdown_connections 2024-09-10T23:32:20.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.838+0000 7f6cd6c00640 1 -- 172.21.15.94:0/2551440461 wait complete. 2024-09-10T23:32:20.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.838+0000 7f6cd6c00640 1 Processor -- start 2024-09-10T23:32:20.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.838+0000 7f6cd6c00640 1 -- start start 2024-09-10T23:32:20.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cd6c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cb4003c70 0x7f6cd0154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:20.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cd6c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd015a0c0 0x7f6cd00aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:20.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cd6c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cd0150510 0x7f6cd015d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:20.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cd6c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd0166e10 con 0x7f6cd015a0c0 2024-09-10T23:32:20.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cd6c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd0166c90 con 0x7f6cb4003c70 2024-09-10T23:32:20.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cd6c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd0166f90 con 0x7f6cd0150510 2024-09-10T23:32:20.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cc7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd015a0c0 0x7f6cd00aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:20.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cc7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd015a0c0 0x7f6cd00aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56478/0 (socket says 172.21.15.94:56478) 2024-09-10T23:32:20.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cc7e00640 1 -- 172.21.15.94:0/2804853849 learned_addr learned my addr 172.21.15.94:0/2804853849 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:20.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6ccfe00640 1 --2- 172.21.15.94:0/2804853849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cb4003c70 0x7f6cd0154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:20.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cd4c00640 1 --2- 172.21.15.94:0/2804853849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cd0150510 0x7f6cd015d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:20.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cc7e00640 1 -- 172.21.15.94:0/2804853849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cd0150510 msgr2=0x7f6cd015d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:20.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cc7e00640 1 --2- 172.21.15.94:0/2804853849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cd0150510 0x7f6cd015d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cc7e00640 1 -- 172.21.15.94:0/2804853849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cb4003c70 msgr2=0x7f6cd0154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:20.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cc7e00640 1 --2- 172.21.15.94:0/2804853849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cb4003c70 0x7f6cd0154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cc7e00640 1 -- 172.21.15.94:0/2804853849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd015a540 con 0x7f6cd015a0c0 2024-09-10T23:32:20.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cc7e00640 1 --2- 172.21.15.94:0/2804853849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd015a0c0 0x7f6cd00aee70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8003ed0 tx=0x7f6cb8001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:20.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cccc00640 1 -- 172.21.15.94:0/2804853849 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8011700 con 0x7f6cd015a0c0 2024-09-10T23:32:20.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cccc00640 1 -- 172.21.15.94:0/2804853849 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cb8016020 con 0x7f6cd015a0c0 2024-09-10T23:32:20.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cd6c00640 1 -- 172.21.15.94:0/2804853849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd00af3b0 con 0x7f6cd015a0c0 2024-09-10T23:32:20.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cccc00640 1 -- 172.21.15.94:0/2804853849 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8011b40 con 0x7f6cd015a0c0 2024-09-10T23:32:20.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cd6c00640 1 -- 172.21.15.94:0/2804853849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd015ad80 con 0x7f6cd015a0c0 2024-09-10T23:32:20.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cccc00640 1 -- 172.21.15.94:0/2804853849 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f6cb8011ce0 con 0x7f6cd015a0c0 2024-09-10T23:32:20.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cccc00640 1 -- 172.21.15.94:0/2804853849 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f6cb80103c0 con 0x7f6cd015a0c0 2024-09-10T23:32:20.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.842+0000 7f6cd6c00640 1 -- 172.21.15.94:0/2804853849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd00cc320 con 0x7f6cd015a0c0 2024-09-10T23:32:20.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.846+0000 7f6cccc00640 1 -- 172.21.15.94:0/2804853849 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f6cb8020020 con 0x7f6cd015a0c0 2024-09-10T23:32:20.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.906+0000 7f6cd6c00640 1 -- 172.21.15.94:0/2804853849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6cd0069850 con 0x7f6cd015a0c0 2024-09-10T23:32:20.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.906+0000 7f6cccc00640 1 -- 172.21.15.94:0/2804853849 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f6cd015a540 con 0x7f6cd015a0c0 2024-09-10T23:32:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:20.907 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":10,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:32:20.614769+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:32:20.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.906+0000 7f6cd6c00640 1 -- 172.21.15.94:0/2804853849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd015a0c0 msgr2=0x7f6cd00aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:20.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.906+0000 7f6cd6c00640 1 --2- 172.21.15.94:0/2804853849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd015a0c0 0x7f6cd00aee70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8003ed0 tx=0x7f6cb8001810 comp rx=0 tx=0).stop 2024-09-10T23:32:20.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.906+0000 7f6cd6c00640 1 -- 172.21.15.94:0/2804853849 shutdown_connections 2024-09-10T23:32:20.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.906+0000 7f6cd6c00640 1 --2- 172.21.15.94:0/2804853849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cd0150510 0x7f6cd015d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.906+0000 7f6cd6c00640 1 --2- 172.21.15.94:0/2804853849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cd015a0c0 0x7f6cd00aee70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.906+0000 7f6cd6c00640 1 --2- 172.21.15.94:0/2804853849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cb4003c70 0x7f6cd0154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:20.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.906+0000 7f6cd6c00640 1 -- 172.21.15.94:0/2804853849 >> 172.21.15.94:0/2804853849 conn(0x7f6cd006cdf0 msgr2=0x7f6cd0151220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:20.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.906+0000 7f6cd6c00640 1 -- 172.21.15.94:0/2804853849 shutdown_connections 2024-09-10T23:32:20.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:20.906+0000 7f6cd6c00640 1 -- 172.21.15.94:0/2804853849 wait complete. 2024-09-10T23:32:20.930 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-10T23:32:20.931 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-10T23:32:21.735 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T23:32:21.870 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79ec00640 1 -- 172.21.15.94:0/4236004767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7980d1ad0 msgr2=0x7fd7980d1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:21.870 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79ec00640 1 --2- 172.21.15.94:0/4236004767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7980d1ad0 0x7fd7980d1f20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd780009840 tx=0x7fd78001c7f0 comp rx=0 tx=0).stop 2024-09-10T23:32:21.870 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79ec00640 1 -- 172.21.15.94:0/4236004767 shutdown_connections 2024-09-10T23:32:21.870 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79ec00640 1 --2- 172.21.15.94:0/4236004767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7980cfef0 0x7fd7980d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:21.871 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79ec00640 1 --2- 172.21.15.94:0/4236004767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7980d1ad0 0x7fd7980d1f20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:21.871 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79ec00640 1 -- 172.21.15.94:0/4236004767 >> 172.21.15.94:0/4236004767 conn(0x7fd79809aa20 msgr2=0x7fd79809ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:21.871 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79ec00640 1 -- 172.21.15.94:0/4236004767 shutdown_connections 2024-09-10T23:32:21.871 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79ec00640 1 -- 172.21.15.94:0/4236004767 wait complete. 2024-09-10T23:32:21.871 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79ec00640 1 Processor -- start 2024-09-10T23:32:21.871 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79ec00640 1 -- start start 2024-09-10T23:32:21.873 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79ec00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7980cfef0 0x7fd798151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:21.873 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79ec00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7980d1ad0 0x7fd79815e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:21.873 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79ec00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd79815de10 0x7fd79815e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:21.873 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79ec00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd79816afe0 con 0x7fd7980d1ad0 2024-09-10T23:32:21.873 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79ec00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd79816ae60 con 0x7fd7980cfef0 2024-09-10T23:32:21.873 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79ec00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd79816b160 con 0x7fd79815de10 2024-09-10T23:32:21.873 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd78fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7980d1ad0 0x7fd79815e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:21.873 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd78fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7980d1ad0 0x7fd79815e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56496/0 (socket says 172.21.15.94:56496) 2024-09-10T23:32:21.873 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd78fe00640 1 -- 172.21.15.94:0/2680705155 learned_addr learned my addr 172.21.15.94:0/2680705155 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:21.874 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd79cc00640 1 --2- 172.21.15.94:0/2680705155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd79815de10 0x7fd79815e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:21.874 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd797e00640 1 --2- 172.21.15.94:0/2680705155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7980cfef0 0x7fd798151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:21.874 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd78fe00640 1 -- 172.21.15.94:0/2680705155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd79815de10 msgr2=0x7fd79815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:21.874 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd78fe00640 1 --2- 172.21.15.94:0/2680705155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd79815de10 0x7fd79815e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:21.874 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd78fe00640 1 -- 172.21.15.94:0/2680705155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7980cfef0 msgr2=0x7fd798151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:21.874 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd78fe00640 1 --2- 172.21.15.94:0/2680705155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7980cfef0 0x7fd798151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:21.874 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.870+0000 7fd78fe00640 1 -- 172.21.15.94:0/2680705155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7980b5bc0 con 0x7fd7980d1ad0 2024-09-10T23:32:21.875 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.874+0000 7fd78fe00640 1 --2- 172.21.15.94:0/2680705155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7980d1ad0 0x7fd79815e1f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd780009790 tx=0x7fd7800040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:21.875 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.874+0000 7fd794c00640 1 -- 172.21.15.94:0/2680705155 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7800b2040 con 0x7fd7980d1ad0 2024-09-10T23:32:21.875 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.874+0000 7fd794c00640 1 -- 172.21.15.94:0/2680705155 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd780005020 con 0x7fd7980d1ad0 2024-09-10T23:32:21.875 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.874+0000 7fd79ec00640 1 -- 172.21.15.94:0/2680705155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7980b2e80 con 0x7fd7980d1ad0 2024-09-10T23:32:21.875 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.874+0000 7fd794c00640 1 -- 172.21.15.94:0/2680705155 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd780005450 con 0x7fd7980d1ad0 2024-09-10T23:32:21.875 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.874+0000 7fd79ec00640 1 -- 172.21.15.94:0/2680705155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7980b3160 con 0x7fd7980d1ad0 2024-09-10T23:32:21.876 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.874+0000 7fd79ec00640 1 -- 172.21.15.94:0/2680705155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7980d0320 con 0x7fd7980d1ad0 2024-09-10T23:32:21.876 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.874+0000 7fd794c00640 1 -- 172.21.15.94:0/2680705155 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 2) ==== 48237+0+0 (secure 0 0 0) 0x7fd7800b5070 con 0x7fd7980d1ad0 2024-09-10T23:32:21.876 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.874+0000 7fd794c00640 1 -- 172.21.15.94:0/2680705155 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd7800e8040 con 0x7fd7980d1ad0 2024-09-10T23:32:21.878 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.878+0000 7fd794c00640 1 -- 172.21.15.94:0/2680705155 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fd780016410 con 0x7fd7980d1ad0 2024-09-10T23:32:21.936 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.934+0000 7fd79ec00640 1 -- 172.21.15.94:0/2680705155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd798076230 con 0x7fd7980d1ad0 2024-09-10T23:32:21.936 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.934+0000 7fd794c00640 1 -- 172.21.15.94:0/2680705155 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd780004d90 con 0x7fd7980d1ad0 2024-09-10T23:32:21.937 INFO:teuthology.misc.health.smithi094.stdout: 2024-09-10T23:32:21.937 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":10,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:32:20.614769+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:32:21.938 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.938+0000 7fd79ec00640 1 -- 172.21.15.94:0/2680705155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7980d1ad0 msgr2=0x7fd79815e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:21.938 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.938+0000 7fd79ec00640 1 --2- 172.21.15.94:0/2680705155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7980d1ad0 0x7fd79815e1f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd780009790 tx=0x7fd7800040c0 comp rx=0 tx=0).stop 2024-09-10T23:32:21.938 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.938+0000 7fd79ec00640 1 -- 172.21.15.94:0/2680705155 shutdown_connections 2024-09-10T23:32:21.938 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.938+0000 7fd79ec00640 1 --2- 172.21.15.94:0/2680705155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd79815de10 0x7fd79815e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:21.938 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.938+0000 7fd79ec00640 1 --2- 172.21.15.94:0/2680705155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7980d1ad0 0x7fd79815e1f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:21.938 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.938+0000 7fd79ec00640 1 --2- 172.21.15.94:0/2680705155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7980cfef0 0x7fd798151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:21.938 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.938+0000 7fd79ec00640 1 -- 172.21.15.94:0/2680705155 >> 172.21.15.94:0/2680705155 conn(0x7fd79809aa20 msgr2=0x7fd7980c4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:21.938 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.938+0000 7fd79ec00640 1 -- 172.21.15.94:0/2680705155 shutdown_connections 2024-09-10T23:32:21.938 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:21.938+0000 7fd79ec00640 1 -- 172.21.15.94:0/2680705155 wait complete. 2024-09-10T23:32:21.954 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T23:32:22.106 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-10T23:32:22.104+0000 7fd8117b67c0 -1 Falling back to public interface 2024-09-10T23:32:22.347 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:32:22.344+0000 7fb933400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T23:32:22.350 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:32:22.348+0000 7fb933400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T23:32:22.350 INFO:tasks.ceph.osd.4.smithi181.stderr:2024-09-10T23:32:22.348+0000 7f37c28ea7c0 -1 Falling back to public interface 2024-09-10T23:32:22.351 INFO:tasks.ceph.osd.7.smithi181.stderr:2024-09-10T23:32:22.348+0000 7fd7a96f77c0 -1 Falling back to public interface 2024-09-10T23:32:22.383 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-10T23:32:22.382+0000 7f7d6c1a97c0 -1 Falling back to public interface 2024-09-10T23:32:22.383 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-10T23:32:22.382+0000 7f2f0e2677c0 -1 Falling back to public interface 2024-09-10T23:32:22.410 INFO:tasks.ceph.osd.5.smithi181.stderr:2024-09-10T23:32:22.408+0000 7fe85602e7c0 -1 Falling back to public interface 2024-09-10T23:32:22.411 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-10T23:32:22.410+0000 7f81ab39d7c0 -1 Falling back to public interface 2024-09-10T23:32:22.615 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-10T23:32:22.614+0000 7f70b43437c0 -1 Falling back to public interface 2024-09-10T23:32:26.460 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-10T23:32:26.456+0000 7fd8117b67c0 -1 osd.6 0 log_to_monitors true 2024-09-10T23:32:26.562 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-10T23:32:26.562+0000 7f7d6c1a97c0 -1 osd.0 0 log_to_monitors true 2024-09-10T23:32:26.589 INFO:tasks.ceph.osd.7.smithi181.stderr:2024-09-10T23:32:26.588+0000 7fd7a96f77c0 -1 osd.7 0 log_to_monitors true 2024-09-10T23:32:26.618 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-10T23:32:26.618+0000 7f2f0e2677c0 -1 osd.1 0 log_to_monitors true 2024-09-10T23:32:26.760 INFO:tasks.ceph.osd.4.smithi181.stderr:2024-09-10T23:32:26.756+0000 7f37c28ea7c0 -1 osd.4 0 log_to_monitors true 2024-09-10T23:32:26.797 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-10T23:32:26.794+0000 7f81ab39d7c0 -1 osd.2 0 log_to_monitors true 2024-09-10T23:32:27.252 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-10T23:32:27.250+0000 7f70b43437c0 -1 osd.3 0 log_to_monitors true 2024-09-10T23:32:27.406 INFO:tasks.ceph.osd.5.smithi181.stderr:2024-09-10T23:32:27.404+0000 7fe85602e7c0 -1 osd.5 0 log_to_monitors true 2024-09-10T23:32:28.734 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-10T23:32:28.731+0000 7fd80d000640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T23:32:28.761 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T23:32:28.839 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-10T23:32:28.835+0000 7fd802600640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T23:32:28.951 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.950+0000 7fd4f5800640 1 -- 172.21.15.94:0/2975037498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4f015de20 msgr2=0x7fd4f0071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:28.951 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.950+0000 7fd4f5800640 1 --2- 172.21.15.94:0/2975037498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4f015de20 0x7fd4f0071bc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009900 tx=0x7fd4e001c7c0 comp rx=0 tx=0).stop 2024-09-10T23:32:28.951 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.950+0000 7fd4f5800640 1 -- 172.21.15.94:0/2975037498 shutdown_connections 2024-09-10T23:32:28.951 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.950+0000 7fd4f5800640 1 --2- 172.21.15.94:0/2975037498 >> v2:172.21.15.181:3300/0 conn(0x7fd4f00b3a90 0x7fd4f00d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:28.951 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.950+0000 7fd4f5800640 1 --2- 172.21.15.94:0/2975037498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4f015de20 0x7fd4f0071bc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:28.952 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.950+0000 7fd4f5800640 1 -- 172.21.15.94:0/2975037498 >> 172.21.15.94:0/2975037498 conn(0x7fd4f00d0b80 msgr2=0x7fd4f00d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:28.952 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.950+0000 7fd4f5800640 1 -- 172.21.15.94:0/2975037498 shutdown_connections 2024-09-10T23:32:28.952 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.950+0000 7fd4f5800640 1 -- 172.21.15.94:0/2975037498 wait complete. 2024-09-10T23:32:28.953 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.950+0000 7fd4f5800640 1 Processor -- start 2024-09-10T23:32:28.954 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4f5800640 1 -- start start 2024-09-10T23:32:28.956 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4f5800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4f006cc20 0x7fd4f00ce470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:28.956 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4f5800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4f00b3a90 0x7fd4f0156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:28.956 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4f5800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4f015de20 0x7fd4f01568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:28.956 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4f5800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f016b090 con 0x7fd4f006cc20 2024-09-10T23:32:28.956 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4f5800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f016af10 con 0x7fd4f015de20 2024-09-10T23:32:28.956 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4f5800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f016b210 con 0x7fd4f00b3a90 2024-09-10T23:32:28.956 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4ef400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4f015de20 0x7fd4f01568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:28.956 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4ee000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4f00b3a90 0x7fd4f0156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:28.957 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4ee000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4f00b3a90 0x7fd4f0156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:54424/0 (socket says 172.21.15.94:54424) 2024-09-10T23:32:28.957 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4ef400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4f015de20 0x7fd4f01568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:43418/0 (socket says 172.21.15.94:43418) 2024-09-10T23:32:28.957 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4ee000640 1 -- 172.21.15.94:0/2043931041 learned_addr learned my addr 172.21.15.94:0/2043931041 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:28.957 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4ee000640 1 -- 172.21.15.94:0/2043931041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4f015de20 msgr2=0x7fd4f01568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:28.957 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4ee000640 1 --2- 172.21.15.94:0/2043931041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4f015de20 0x7fd4f01568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:28.957 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4ee000640 1 -- 172.21.15.94:0/2043931041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4f006cc20 msgr2=0x7fd4f00ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:28.957 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4ee000640 1 --2- 172.21.15.94:0/2043931041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4f006cc20 0x7fd4f00ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:28.957 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4ee000640 1 -- 172.21.15.94:0/2043931041 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f01ff3d0 con 0x7fd4f00b3a90 2024-09-10T23:32:28.958 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4ee000640 1 --2- 172.21.15.94:0/2043931041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4f00b3a90 0x7fd4f0156390 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00e9f0 tx=0x7fd4dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:28.958 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4e6a00640 1 -- 172.21.15.94:0/2043931041 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00cd80 con 0x7fd4f00b3a90 2024-09-10T23:32:28.958 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4e6a00640 1 -- 172.21.15.94:0/2043931041 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4dc010430 con 0x7fd4f00b3a90 2024-09-10T23:32:28.959 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.954+0000 7fd4e6a00640 1 -- 172.21.15.94:0/2043931041 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc010860 con 0x7fd4f00b3a90 2024-09-10T23:32:28.959 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.958+0000 7fd4f5800640 1 -- 172.21.15.94:0/2043931041 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f01ff690 con 0x7fd4f00b3a90 2024-09-10T23:32:28.959 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.958+0000 7fd4f5800640 1 -- 172.21.15.94:0/2043931041 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f01ffc50 con 0x7fd4f00b3a90 2024-09-10T23:32:28.960 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.958+0000 7fd4e6a00640 1 -- 172.21.15.94:0/2043931041 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 4) ==== 96338+0+0 (secure 0 0 0) 0x7fd4dc010a00 con 0x7fd4f00b3a90 2024-09-10T23:32:28.962 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.962+0000 7fd4e6a00640 1 --2- 172.21.15.94:0/2043931041 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd4d0072550 0x7fd4d0074a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:28.962 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.962+0000 7fd4e6a00640 1 -- 172.21.15.94:0/2043931041 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7fd4dc092fa0 con 0x7fd4f00b3a90 2024-09-10T23:32:28.962 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.962+0000 7fd4eea00640 1 --2- 172.21.15.94:0/2043931041 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd4d0072550 0x7fd4d0074a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:28.962 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.962+0000 7fd4f5800640 1 -- 172.21.15.94:0/2043931041 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c80052a0 con 0x7fd4f00b3a90 2024-09-10T23:32:28.963 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.962+0000 7fd4eea00640 1 --2- 172.21.15.94:0/2043931041 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd4d0072550 0x7fd4d0074a10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00af000 tx=0x7fd4e0019560 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:28.969 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:28.966+0000 7fd4e6a00640 1 -- 172.21.15.94:0/2043931041 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd4dc05ef50 con 0x7fd4f00b3a90 2024-09-10T23:32:29.235 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:29.234+0000 7fd4f5800640 1 -- 172.21.15.94:0/2043931041 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4c8005590 con 0x7fd4f00b3a90 2024-09-10T23:32:29.236 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:29.234+0000 7fd4e6a00640 1 -- 172.21.15.94:0/2043931041 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7058 (secure 0 0 0) 0x7fd4dc063980 con 0x7fd4f00b3a90 2024-09-10T23:32:29.236 INFO:teuthology.misc.health.smithi094.stdout: 2024-09-10T23:32:29.236 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":13,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:32:28.528091+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:32:29.239 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:29.238+0000 7fd4f5800640 1 -- 172.21.15.94:0/2043931041 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd4d0072550 msgr2=0x7fd4d0074a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:29.239 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:29.238+0000 7fd4f5800640 1 --2- 172.21.15.94:0/2043931041 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd4d0072550 0x7fd4d0074a10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00af000 tx=0x7fd4e0019560 comp rx=0 tx=0).stop 2024-09-10T23:32:29.239 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:29.238+0000 7fd4f5800640 1 -- 172.21.15.94:0/2043931041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4f00b3a90 msgr2=0x7fd4f0156390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:29.240 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:29.238+0000 7fd4f5800640 1 --2- 172.21.15.94:0/2043931041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4f00b3a90 0x7fd4f0156390 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00e9f0 tx=0x7fd4dc00eeb0 comp rx=0 tx=0).stop 2024-09-10T23:32:29.240 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:29.238+0000 7fd4f5800640 1 -- 172.21.15.94:0/2043931041 shutdown_connections 2024-09-10T23:32:29.240 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:29.238+0000 7fd4f5800640 1 --2- 172.21.15.94:0/2043931041 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd4d0072550 0x7fd4d0074a10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:29.240 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:29.238+0000 7fd4f5800640 1 --2- 172.21.15.94:0/2043931041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4f015de20 0x7fd4f01568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:29.240 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:29.238+0000 7fd4f5800640 1 --2- 172.21.15.94:0/2043931041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4f00b3a90 0x7fd4f0156390 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:29.240 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:29.238+0000 7fd4f5800640 1 --2- 172.21.15.94:0/2043931041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4f006cc20 0x7fd4f00ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:29.240 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:29.238+0000 7fd4f5800640 1 -- 172.21.15.94:0/2043931041 >> 172.21.15.94:0/2043931041 conn(0x7fd4f00d0b80 msgr2=0x7fd4f00cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:29.240 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:29.238+0000 7fd4f5800640 1 -- 172.21.15.94:0/2043931041 shutdown_connections 2024-09-10T23:32:29.240 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:29.238+0000 7fd4f5800640 1 -- 172.21.15.94:0/2043931041 wait complete. 2024-09-10T23:32:29.260 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T23:32:30.290 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-10T23:32:30.290+0000 7f70af200640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T23:32:30.305 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-10T23:32:30.302+0000 7f81a6200640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T23:32:30.312 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-10T23:32:30.310+0000 7f7d67000640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T23:32:30.319 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-10T23:32:30.318+0000 7f70a5200640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T23:32:30.334 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-10T23:32:30.334+0000 7f7d5d000640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T23:32:30.342 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-10T23:32:30.342+0000 7f819c200640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T23:32:30.349 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-10T23:32:30.346+0000 7f2f09c00640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T23:32:30.361 INFO:tasks.ceph.osd.4.smithi181.stderr:2024-09-10T23:32:30.359+0000 7f37be200640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T23:32:30.363 INFO:tasks.ceph.osd.7.smithi181.stderr:2024-09-10T23:32:30.359+0000 7fd7a4600640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T23:32:30.378 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-10T23:32:30.378+0000 7f2eff200640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T23:32:30.385 INFO:tasks.ceph.osd.5.smithi181.stderr:2024-09-10T23:32:30.383+0000 7fe850000640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T23:32:30.388 INFO:tasks.ceph.osd.7.smithi181.stderr:2024-09-10T23:32:30.383+0000 7fd79a600640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T23:32:30.394 INFO:tasks.ceph.osd.4.smithi181.stderr:2024-09-10T23:32:30.391+0000 7f37b3800640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T23:32:30.406 INFO:tasks.ceph.osd.5.smithi181.stderr:2024-09-10T23:32:30.403+0000 7fe846000640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T23:32:34.373 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:32:34.377 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:32:36.065 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T23:32:36.206 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bf200640 1 -- 172.21.15.94:0/650500359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c004810 msgr2=0x7f519c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:36.207 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bf200640 1 --2- 172.21.15.94:0/650500359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c004810 0x7f519c004c90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f51a40016c0 tx=0x7f51a401c710 comp rx=0 tx=0).stop 2024-09-10T23:32:36.207 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bf200640 1 -- 172.21.15.94:0/650500359 shutdown_connections 2024-09-10T23:32:36.207 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bf200640 1 --2- 172.21.15.94:0/650500359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c004810 0x7f519c004c90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:36.207 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bf200640 1 -- 172.21.15.94:0/650500359 >> 172.21.15.94:0/650500359 conn(0x7f51b809aa20 msgr2=0x7f51b809ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:36.207 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bf200640 1 -- 172.21.15.94:0/650500359 shutdown_connections 2024-09-10T23:32:36.207 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bf200640 1 -- 172.21.15.94:0/650500359 wait complete. 2024-09-10T23:32:36.208 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bf200640 1 Processor -- start 2024-09-10T23:32:36.208 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bf200640 1 -- start start 2024-09-10T23:32:36.210 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bf200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c004810 0x7f51b81fa440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:36.210 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bf200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b80cf1a0 0x7f51b80ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:36.210 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bf200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51b81520a0 0x7f51b80ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:36.210 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bf200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51b80b2200 con 0x7f519c004810 2024-09-10T23:32:36.210 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bf200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f51b80b2080 con 0x7f51b80cf1a0 2024-09-10T23:32:36.210 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bf200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f51b80b2380 con 0x7f51b81520a0 2024-09-10T23:32:36.211 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51b7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c004810 0x7f51b81fa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:36.211 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51b7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c004810 0x7f51b81fa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57470/0 (socket says 172.21.15.94:57470) 2024-09-10T23:32:36.211 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51b7e00640 1 -- 172.21.15.94:0/2736066459 learned_addr learned my addr 172.21.15.94:0/2736066459 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:36.211 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51b7400640 1 --2- 172.21.15.94:0/2736066459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b80cf1a0 0x7f51b80ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:36.211 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bd200640 1 --2- 172.21.15.94:0/2736066459 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51b81520a0 0x7f51b80ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:36.211 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51b7e00640 1 -- 172.21.15.94:0/2736066459 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51b81520a0 msgr2=0x7f51b80ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:36.211 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51b7e00640 1 --2- 172.21.15.94:0/2736066459 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51b81520a0 0x7f51b80ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:36.211 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51b7e00640 1 -- 172.21.15.94:0/2736066459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b80cf1a0 msgr2=0x7f51b80ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:36.211 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51b7e00640 1 --2- 172.21.15.94:0/2736066459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b80cf1a0 0x7f51b80ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:36.211 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51b7e00640 1 -- 172.21.15.94:0/2736066459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b81f6f10 con 0x7f519c004810 2024-09-10T23:32:36.211 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.205+0000 7f51bd200640 1 --2- 172.21.15.94:0/2736066459 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51b81520a0 0x7f51b80ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:36.211 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.209+0000 7f51b7e00640 1 --2- 172.21.15.94:0/2736066459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c004810 0x7f51b81fa440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51a8011b40 tx=0x7f51a8009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:36.211 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.209+0000 7f51afe00640 1 -- 172.21.15.94:0/2736066459 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a800fd30 con 0x7f519c004810 2024-09-10T23:32:36.211 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.209+0000 7f51afe00640 1 -- 172.21.15.94:0/2736066459 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51a8004020 con 0x7f519c004810 2024-09-10T23:32:36.212 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.209+0000 7f51afe00640 1 -- 172.21.15.94:0/2736066459 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8004470 con 0x7f519c004810 2024-09-10T23:32:36.212 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.209+0000 7f51bf200640 1 -- 172.21.15.94:0/2736066459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b81f9960 con 0x7f519c004810 2024-09-10T23:32:36.212 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.209+0000 7f51bf200640 1 -- 172.21.15.94:0/2736066459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b81f71b0 con 0x7f519c004810 2024-09-10T23:32:36.212 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.209+0000 7f51bf200640 1 -- 172.21.15.94:0/2736066459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b806d6d0 con 0x7f519c004810 2024-09-10T23:32:36.212 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.209+0000 7f51afe00640 1 -- 172.21.15.94:0/2736066459 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f51a8003600 con 0x7f519c004810 2024-09-10T23:32:36.213 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.209+0000 7f51afe00640 1 --2- 172.21.15.94:0/2736066459 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f51980725e0 0x7f5198074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:36.213 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.209+0000 7f51afe00640 1 -- 172.21.15.94:0/2736066459 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f51a8093940 con 0x7f519c004810 2024-09-10T23:32:36.214 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.209+0000 7f51b7400640 1 --2- 172.21.15.94:0/2736066459 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f51980725e0 0x7f5198074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:36.214 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.213+0000 7f51b7400640 1 --2- 172.21.15.94:0/2736066459 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f51980725e0 0x7f5198074aa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f519c0057f0 tx=0x7f51a4019500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:36.218 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.217+0000 7f51afe00640 1 -- 172.21.15.94:0/2736066459 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f51a805ed10 con 0x7f519c004810 2024-09-10T23:32:36.385 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.381+0000 7f51bf200640 1 -- 172.21.15.94:0/2736066459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51b815c970 con 0x7f519c004810 2024-09-10T23:32:36.386 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.381+0000 7f51afe00640 1 -- 172.21.15.94:0/2736066459 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12884 (secure 0 0 0) 0x7f51a8063740 con 0x7f519c004810 2024-09-10T23:32:36.386 INFO:teuthology.misc.health.smithi094.stdout: 2024-09-10T23:32:36.386 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":18,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:32:34.328826+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:32:36.388 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.385+0000 7f51bf200640 1 -- 172.21.15.94:0/2736066459 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f51980725e0 msgr2=0x7f5198074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:36.388 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.385+0000 7f51bf200640 1 --2- 172.21.15.94:0/2736066459 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f51980725e0 0x7f5198074aa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f519c0057f0 tx=0x7f51a4019500 comp rx=0 tx=0).stop 2024-09-10T23:32:36.388 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.385+0000 7f51bf200640 1 -- 172.21.15.94:0/2736066459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c004810 msgr2=0x7f51b81fa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:36.388 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.385+0000 7f51bf200640 1 --2- 172.21.15.94:0/2736066459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c004810 0x7f51b81fa440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51a8011b40 tx=0x7f51a8009f20 comp rx=0 tx=0).stop 2024-09-10T23:32:36.388 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.385+0000 7f51bf200640 1 -- 172.21.15.94:0/2736066459 shutdown_connections 2024-09-10T23:32:36.388 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.385+0000 7f51bf200640 1 --2- 172.21.15.94:0/2736066459 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f51980725e0 0x7f5198074aa0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:36.389 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.385+0000 7f51bf200640 1 --2- 172.21.15.94:0/2736066459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b80cf1a0 0x7f51b80ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:36.389 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.385+0000 7f51bf200640 1 --2- 172.21.15.94:0/2736066459 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51b81520a0 0x7f51b80ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:36.389 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.385+0000 7f51bf200640 1 --2- 172.21.15.94:0/2736066459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f519c004810 0x7f51b81fa440 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:36.389 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.385+0000 7f51bf200640 1 -- 172.21.15.94:0/2736066459 >> 172.21.15.94:0/2736066459 conn(0x7f51b809aa20 msgr2=0x7f51b81528d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:36.389 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.385+0000 7f51bf200640 1 -- 172.21.15.94:0/2736066459 shutdown_connections 2024-09-10T23:32:36.389 INFO:teuthology.misc.health.smithi094.stderr:2024-09-10T23:32:36.385+0000 7f51bf200640 1 -- 172.21.15.94:0/2736066459 wait complete. 2024-09-10T23:32:36.405 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-10T23:32:36.405 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-10T23:32:36.406 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T23:32:36.406 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T23:32:36.409 INFO:tasks.ceph.mds.a:Started 2024-09-10T23:32:36.409 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T23:32:36.409 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T23:32:36.412 INFO:tasks.ceph.mds.c:Started 2024-09-10T23:32:36.412 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T23:32:36.412 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T23:32:36.416 INFO:tasks.ceph.mds.b:Started 2024-09-10T23:32:36.416 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T23:32:36.416 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T23:32:36.419 INFO:tasks.ceph.mds.d:Started 2024-09-10T23:32:36.419 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-10T23:32:36.420 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:32:36.508 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-09-10T23:32:36.509 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-09-10T23:32:36.528 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:32:36.529 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2024-09-10T23:32:36.530 INFO:tasks.ceph.mds.d.smithi181.stderr:2024-09-10T23:32:36.527+0000 7f13ba600640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-10T23:32:36.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.565+0000 7faf30200640 1 -- 172.21.15.94:0/1582883653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf280af310 msgr2=0x7faf280af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:36.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.565+0000 7faf30200640 1 --2- 172.21.15.94:0/1582883653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf280af310 0x7faf280af7a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faf2806d8b0 tx=0x7faf2001ca60 comp rx=0 tx=0).stop 2024-09-10T23:32:36.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.565+0000 7faf30200640 1 -- 172.21.15.94:0/1582883653 shutdown_connections 2024-09-10T23:32:36.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.565+0000 7faf30200640 1 --2- 172.21.15.94:0/1582883653 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf280af310 0x7faf280af7a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:36.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.565+0000 7faf30200640 1 --2- 172.21.15.94:0/1582883653 >> v2:172.21.15.181:3300/0 conn(0x7faf280b2bb0 0x7faf280d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:36.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.565+0000 7faf30200640 1 -- 172.21.15.94:0/1582883653 >> 172.21.15.94:0/1582883653 conn(0x7faf280d1c60 msgr2=0x7faf280d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:36.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.565+0000 7faf30200640 1 -- 172.21.15.94:0/1582883653 shutdown_connections 2024-09-10T23:32:36.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.565+0000 7faf30200640 1 -- 172.21.15.94:0/1582883653 wait complete. 2024-09-10T23:32:36.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf30200640 1 Processor -- start 2024-09-10T23:32:36.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf30200640 1 -- start start 2024-09-10T23:32:36.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf30200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf280af310 0x7faf280cede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:36.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf30200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf280b2bb0 0x7faf28154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:36.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf30200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2815f5c0 0x7faf2814b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:36.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf30200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf281690b0 con 0x7faf2815f5c0 2024-09-10T23:32:36.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf30200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faf28168f30 con 0x7faf280af310 2024-09-10T23:32:36.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf30200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7faf28169230 con 0x7faf280b2bb0 2024-09-10T23:32:36.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf2e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2815f5c0 0x7faf2814b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:36.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf2e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2815f5c0 0x7faf2814b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57520/0 (socket says 172.21.15.94:57520) 2024-09-10T23:32:36.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf2e200640 1 -- 172.21.15.94:0/2337931616 learned_addr learned my addr 172.21.15.94:0/2337931616 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:36.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf2ce00640 1 --2- 172.21.15.94:0/2337931616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf280b2bb0 0x7faf28154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:36.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf2d800640 1 --2- 172.21.15.94:0/2337931616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf280af310 0x7faf280cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:36.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf2e200640 1 -- 172.21.15.94:0/2337931616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf280b2bb0 msgr2=0x7faf28154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:36.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf2e200640 1 --2- 172.21.15.94:0/2337931616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf280b2bb0 0x7faf28154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:36.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf2e200640 1 -- 172.21.15.94:0/2337931616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf280af310 msgr2=0x7faf280cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:36.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf2e200640 1 --2- 172.21.15.94:0/2337931616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf280af310 0x7faf280cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:36.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf2e200640 1 -- 172.21.15.94:0/2337931616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf281f8cc0 con 0x7faf2815f5c0 2024-09-10T23:32:36.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf2d800640 1 --2- 172.21.15.94:0/2337931616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf280af310 0x7faf280cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:32:36.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf2e200640 1 --2- 172.21.15.94:0/2337931616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2815f5c0 0x7faf2814b7b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf20009b10 tx=0x7faf200a5860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:36.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf25600640 1 -- 172.21.15.94:0/2337931616 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf200bb070 con 0x7faf2815f5c0 2024-09-10T23:32:36.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf25600640 1 -- 172.21.15.94:0/2337931616 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf20004d40 con 0x7faf2815f5c0 2024-09-10T23:32:36.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf25600640 1 -- 172.21.15.94:0/2337931616 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf200b2040 con 0x7faf2815f5c0 2024-09-10T23:32:36.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf30200640 1 -- 172.21.15.94:0/2337931616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf281f8fa0 con 0x7faf2815f5c0 2024-09-10T23:32:36.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf30200640 1 -- 172.21.15.94:0/2337931616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf281f9560 con 0x7faf2815f5c0 2024-09-10T23:32:36.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.569+0000 7faf30200640 1 -- 172.21.15.94:0/2337931616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf280af7a0 con 0x7faf2815f5c0 2024-09-10T23:32:36.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.573+0000 7faf25600640 1 -- 172.21.15.94:0/2337931616 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faf200b5030 con 0x7faf2815f5c0 2024-09-10T23:32:36.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.573+0000 7faf25600640 1 --2- 172.21.15.94:0/2337931616 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faf08072610 0x7faf08074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:36.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.573+0000 7faf25600640 1 -- 172.21.15.94:0/2337931616 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7faf2012cb10 con 0x7faf2815f5c0 2024-09-10T23:32:36.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.573+0000 7faf2d800640 1 --2- 172.21.15.94:0/2337931616 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faf08072610 0x7faf08074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:36.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.573+0000 7faf2d800640 1 --2- 172.21.15.94:0/2337931616 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faf08072610 0x7faf08074ad0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faf1c000f70 tx=0x7faf1c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:36.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.577+0000 7faf25600640 1 -- 172.21.15.94:0/2337931616 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faf20016770 con 0x7faf2815f5c0 2024-09-10T23:32:36.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.773+0000 7faf30200640 1 -- 172.21.15.94:0/2337931616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf280749c0 con 0x7faf2815f5c0 2024-09-10T23:32:36.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.773+0000 7faf25600640 1 -- 172.21.15.94:0/2337931616 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7faf200f8530 con 0x7faf2815f5c0 2024-09-10T23:32:36.778 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:36.778 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":2,"btime":"2024-09-10T23:32:36:516407+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/2278166692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2278166692},{"type":"v1","addr":"172.21.15.94:6834","nonce":2278166692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T23:32:36.778 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 2 2024-09-10T23:32:36.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.777+0000 7faf30200640 1 -- 172.21.15.94:0/2337931616 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faf08072610 msgr2=0x7faf08074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:36.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.777+0000 7faf30200640 1 --2- 172.21.15.94:0/2337931616 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faf08072610 0x7faf08074ad0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faf1c000f70 tx=0x7faf1c009040 comp rx=0 tx=0).stop 2024-09-10T23:32:36.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.777+0000 7faf30200640 1 -- 172.21.15.94:0/2337931616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2815f5c0 msgr2=0x7faf2814b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:36.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.777+0000 7faf30200640 1 --2- 172.21.15.94:0/2337931616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2815f5c0 0x7faf2814b7b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf20009b10 tx=0x7faf200a5860 comp rx=0 tx=0).stop 2024-09-10T23:32:36.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.777+0000 7faf30200640 1 -- 172.21.15.94:0/2337931616 shutdown_connections 2024-09-10T23:32:36.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.777+0000 7faf30200640 1 --2- 172.21.15.94:0/2337931616 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faf08072610 0x7faf08074ad0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:36.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.777+0000 7faf30200640 1 --2- 172.21.15.94:0/2337931616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf2815f5c0 0x7faf2814b7b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:36.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.777+0000 7faf30200640 1 --2- 172.21.15.94:0/2337931616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf280b2bb0 0x7faf28154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:36.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.777+0000 7faf30200640 1 --2- 172.21.15.94:0/2337931616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf280af310 0x7faf280cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:36.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.777+0000 7faf30200640 1 -- 172.21.15.94:0/2337931616 >> 172.21.15.94:0/2337931616 conn(0x7faf280d1c60 msgr2=0x7faf2815d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:36.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.777+0000 7faf30200640 1 -- 172.21.15.94:0/2337931616 shutdown_connections 2024-09-10T23:32:36.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:36.777+0000 7faf30200640 1 -- 172.21.15.94:0/2337931616 wait complete. 2024-09-10T23:32:38.800 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:32:38.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.949+0000 7fafb0a00640 1 -- 172.21.15.94:0/2366567635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafac15f5c0 msgr2=0x7fafac0cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.949+0000 7fafb0a00640 1 --2- 172.21.15.94:0/2366567635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafac15f5c0 0x7fafac0cfb60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fafac06d8b0 tx=0x7faf9c01c7a0 comp rx=0 tx=0).stop 2024-09-10T23:32:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.949+0000 7fafb0a00640 1 -- 172.21.15.94:0/2366567635 shutdown_connections 2024-09-10T23:32:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.949+0000 7fafb0a00640 1 --2- 172.21.15.94:0/2366567635 >> v2:172.21.15.181:3300/0 conn(0x7fafac0b2bb0 0x7fafac0d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.949+0000 7fafb0a00640 1 --2- 172.21.15.94:0/2366567635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafac15f5c0 0x7fafac0cfb60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.949+0000 7fafb0a00640 1 -- 172.21.15.94:0/2366567635 >> 172.21.15.94:0/2366567635 conn(0x7fafac0d1c60 msgr2=0x7fafac0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.949+0000 7fafb0a00640 1 -- 172.21.15.94:0/2366567635 shutdown_connections 2024-09-10T23:32:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.949+0000 7fafb0a00640 1 -- 172.21.15.94:0/2366567635 wait complete. 2024-09-10T23:32:38.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafb0a00640 1 Processor -- start 2024-09-10T23:32:38.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafb0a00640 1 -- start start 2024-09-10T23:32:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafb0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafac0b2bb0 0x7fafac149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafb0a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fafac0d05e0 0x7fafac151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafb0a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafac15f5c0 0x7fafac152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:38.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafb0a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fafac1692b0 con 0x7fafac0b2bb0 2024-09-10T23:32:38.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafb0a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fafac169130 con 0x7fafac15f5c0 2024-09-10T23:32:38.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafb0a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fafac169430 con 0x7fafac0d05e0 2024-09-10T23:32:38.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafa9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafac0b2bb0 0x7fafac149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:38.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafa9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafac0b2bb0 0x7fafac149310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57526/0 (socket says 172.21.15.94:57526) 2024-09-10T23:32:38.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafa9e00640 1 -- 172.21.15.94:0/2161298326 learned_addr learned my addr 172.21.15.94:0/2161298326 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:38.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafaa800640 1 --2- 172.21.15.94:0/2161298326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafac15f5c0 0x7fafac152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:38.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafa9400640 1 --2- 172.21.15.94:0/2161298326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fafac0d05e0 0x7fafac151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:38.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafa9e00640 1 -- 172.21.15.94:0/2161298326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fafac0d05e0 msgr2=0x7fafac151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:38.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafa9e00640 1 --2- 172.21.15.94:0/2161298326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fafac0d05e0 0x7fafac151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:38.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafa9e00640 1 -- 172.21.15.94:0/2161298326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafac15f5c0 msgr2=0x7fafac152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:38.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafa9e00640 1 --2- 172.21.15.94:0/2161298326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafac15f5c0 0x7fafac152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:38.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafa9e00640 1 -- 172.21.15.94:0/2161298326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafac0ce470 con 0x7fafac0b2bb0 2024-09-10T23:32:38.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafaa800640 1 --2- 172.21.15.94:0/2161298326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafac15f5c0 0x7fafac152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafa9400640 1 --2- 172.21.15.94:0/2161298326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fafac0d05e0 0x7fafac151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafa9e00640 1 --2- 172.21.15.94:0/2161298326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafac0b2bb0 0x7fafac149310 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0b0040 tx=0x7faf9c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafa2000640 1 -- 172.21.15.94:0/2161298326 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0b2040 con 0x7fafac0b2bb0 2024-09-10T23:32:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafa2000640 1 -- 172.21.15.94:0/2161298326 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf9c0a5bd0 con 0x7fafac0b2bb0 2024-09-10T23:32:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafa2000640 1 -- 172.21.15.94:0/2161298326 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c005020 con 0x7fafac0b2bb0 2024-09-10T23:32:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafb0a00640 1 -- 172.21.15.94:0/2161298326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafac0ce750 con 0x7fafac0b2bb0 2024-09-10T23:32:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafb0a00640 1 -- 172.21.15.94:0/2161298326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafac0cf460 con 0x7fafac0b2bb0 2024-09-10T23:32:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.953+0000 7fafb0a00640 1 -- 172.21.15.94:0/2161298326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafac0b15c0 con 0x7fafac0b2bb0 2024-09-10T23:32:38.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.957+0000 7fafa2000640 1 -- 172.21.15.94:0/2161298326 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faf9c0051c0 con 0x7fafac0b2bb0 2024-09-10T23:32:38.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.957+0000 7fafa2000640 1 --2- 172.21.15.94:0/2161298326 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faf8c0725e0 0x7faf8c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:38.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.957+0000 7fafa2000640 1 -- 172.21.15.94:0/2161298326 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7faf9c12c830 con 0x7fafac0b2bb0 2024-09-10T23:32:38.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.957+0000 7fafa9400640 1 --2- 172.21.15.94:0/2161298326 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faf8c0725e0 0x7faf8c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:38.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.957+0000 7fafa9400640 1 --2- 172.21.15.94:0/2161298326 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faf8c0725e0 0x7faf8c074aa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faf980058d0 tx=0x7faf98005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:38.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:38.961+0000 7fafa2000640 1 -- 172.21.15.94:0/2161298326 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faf9c0164b0 con 0x7fafac0b2bb0 2024-09-10T23:32:39.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.161+0000 7fafb0a00640 1 -- 172.21.15.94:0/2161298326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafac0cbf30 con 0x7fafac0b2bb0 2024-09-10T23:32:39.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.161+0000 7fafa2000640 1 -- 172.21.15.94:0/2161298326 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3914 (secure 0 0 0) 0x7faf9c0f8250 con 0x7fafac0b2bb0 2024-09-10T23:32:39.163 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:39.163 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":3,"btime":"2024-09-10T23:32:37:517860+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/2278166692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2278166692},{"type":"v1","addr":"172.21.15.94:6834","nonce":2278166692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392173872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3392173872},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392173872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4380,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1894503548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1894503548},{"type":"v1","addr":"172.21.15.181:6837","nonce":1894503548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":194174,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1418253171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1418253171},{"type":"v1","addr":"172.21.15.181:6836","nonce":1418253171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T23:32:39.163 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 3 2024-09-10T23:32:39.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.161+0000 7fafb0a00640 1 -- 172.21.15.94:0/2161298326 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faf8c0725e0 msgr2=0x7faf8c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:39.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.161+0000 7fafb0a00640 1 --2- 172.21.15.94:0/2161298326 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faf8c0725e0 0x7faf8c074aa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faf980058d0 tx=0x7faf98005860 comp rx=0 tx=0).stop 2024-09-10T23:32:39.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.161+0000 7fafb0a00640 1 -- 172.21.15.94:0/2161298326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafac0b2bb0 msgr2=0x7fafac149310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:39.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.161+0000 7fafb0a00640 1 --2- 172.21.15.94:0/2161298326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafac0b2bb0 0x7fafac149310 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0b0040 tx=0x7faf9c004600 comp rx=0 tx=0).stop 2024-09-10T23:32:39.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.161+0000 7fafb0a00640 1 -- 172.21.15.94:0/2161298326 shutdown_connections 2024-09-10T23:32:39.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.161+0000 7fafb0a00640 1 --2- 172.21.15.94:0/2161298326 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faf8c0725e0 0x7faf8c074aa0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.161+0000 7fafb0a00640 1 --2- 172.21.15.94:0/2161298326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafac15f5c0 0x7fafac152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.161+0000 7fafb0a00640 1 --2- 172.21.15.94:0/2161298326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fafac0d05e0 0x7fafac151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.161+0000 7fafb0a00640 1 --2- 172.21.15.94:0/2161298326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafac0b2bb0 0x7fafac149310 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.161+0000 7fafb0a00640 1 -- 172.21.15.94:0/2161298326 >> 172.21.15.94:0/2161298326 conn(0x7fafac0d1c60 msgr2=0x7fafac06e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:39.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.161+0000 7fafb0a00640 1 -- 172.21.15.94:0/2161298326 shutdown_connections 2024-09-10T23:32:39.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.161+0000 7fafb0a00640 1 -- 172.21.15.94:0/2161298326 wait complete. 2024-09-10T23:32:39.184 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T23:32:39.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.321+0000 7f9988c00640 1 -- 172.21.15.94:0/385173648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998415f5c0 msgr2=0x7f99840cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:39.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.321+0000 7f9988c00640 1 --2- 172.21.15.94:0/385173648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998415f5c0 0x7f99840cfb60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9974009900 tx=0x7f997401c7c0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.321+0000 7f9988c00640 1 -- 172.21.15.94:0/385173648 shutdown_connections 2024-09-10T23:32:39.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.321+0000 7f9988c00640 1 --2- 172.21.15.94:0/385173648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99840af310 0x7f99840af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.321+0000 7f9988c00640 1 --2- 172.21.15.94:0/385173648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998415f5c0 0x7f99840cfb60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.321+0000 7f9988c00640 1 -- 172.21.15.94:0/385173648 >> 172.21.15.94:0/385173648 conn(0x7f99840d1c60 msgr2=0x7f99840d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:39.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.321+0000 7f9988c00640 1 -- 172.21.15.94:0/385173648 shutdown_connections 2024-09-10T23:32:39.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.321+0000 7f9988c00640 1 -- 172.21.15.94:0/385173648 wait complete. 2024-09-10T23:32:39.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9988c00640 1 Processor -- start 2024-09-10T23:32:39.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9988c00640 1 -- start start 2024-09-10T23:32:39.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9988c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99840af310 0x7f9984152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:39.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9988c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99840b2bb0 0x7f9984153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:39.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9988c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998415f5c0 0x7f998406dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:39.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9988c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9984168f50 con 0x7f998415f5c0 2024-09-10T23:32:39.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9988c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9984168dd0 con 0x7f99840af310 2024-09-10T23:32:39.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9988c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f99841690d0 con 0x7f99840b2bb0 2024-09-10T23:32:39.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9982800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998415f5c0 0x7f998406dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:39.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9982800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998415f5c0 0x7f998406dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57558/0 (socket says 172.21.15.94:57558) 2024-09-10T23:32:39.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9982800640 1 -- 172.21.15.94:0/4052013189 learned_addr learned my addr 172.21.15.94:0/4052013189 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:39.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9981e00640 1 --2- 172.21.15.94:0/4052013189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99840af310 0x7f9984152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:39.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9979400640 1 --2- 172.21.15.94:0/4052013189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99840b2bb0 0x7f9984153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:39.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9982800640 1 -- 172.21.15.94:0/4052013189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99840b2bb0 msgr2=0x7f9984153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:39.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9982800640 1 --2- 172.21.15.94:0/4052013189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99840b2bb0 0x7f9984153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9982800640 1 -- 172.21.15.94:0/4052013189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99840af310 msgr2=0x7f9984152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:39.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9982800640 1 --2- 172.21.15.94:0/4052013189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99840af310 0x7f9984152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9982800640 1 -- 172.21.15.94:0/4052013189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99840b22b0 con 0x7f998415f5c0 2024-09-10T23:32:39.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9981e00640 1 --2- 172.21.15.94:0/4052013189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99840af310 0x7f9984152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:32:39.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9979400640 1 --2- 172.21.15.94:0/4052013189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99840b2bb0 0x7f9984153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:32:39.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9982800640 1 --2- 172.21.15.94:0/4052013189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998415f5c0 0x7f998406dd30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f997c00eb20 tx=0x7f997c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:39.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f997aa00640 1 -- 172.21.15.94:0/4052013189 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c004b30 con 0x7f998415f5c0 2024-09-10T23:32:39.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f997aa00640 1 -- 172.21.15.94:0/4052013189 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f997c004cf0 con 0x7f998415f5c0 2024-09-10T23:32:39.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f997aa00640 1 -- 172.21.15.94:0/4052013189 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c005280 con 0x7f998415f5c0 2024-09-10T23:32:39.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9988c00640 1 -- 172.21.15.94:0/4052013189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998414f9c0 con 0x7f998415f5c0 2024-09-10T23:32:39.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.325+0000 7f9988c00640 1 -- 172.21.15.94:0/4052013189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f998414ff80 con 0x7f998415f5c0 2024-09-10T23:32:39.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.329+0000 7f9988c00640 1 -- 172.21.15.94:0/4052013189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99840af7a0 con 0x7f998415f5c0 2024-09-10T23:32:39.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.329+0000 7f997aa00640 1 -- 172.21.15.94:0/4052013189 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f997c005420 con 0x7f998415f5c0 2024-09-10T23:32:39.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.329+0000 7f997aa00640 1 --2- 172.21.15.94:0/4052013189 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f99600725e0 0x7f9960074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:39.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.329+0000 7f997aa00640 1 -- 172.21.15.94:0/4052013189 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f997c093d60 con 0x7f998415f5c0 2024-09-10T23:32:39.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.329+0000 7f9981e00640 1 --2- 172.21.15.94:0/4052013189 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f99600725e0 0x7f9960074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:39.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.329+0000 7f9981e00640 1 --2- 172.21.15.94:0/4052013189 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f99600725e0 0x7f9960074aa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f99740af000 tx=0x7f997401b000 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:39.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.333+0000 7f997aa00640 1 -- 172.21.15.94:0/4052013189 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f997c05f6f0 con 0x7f998415f5c0 2024-09-10T23:32:39.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.501+0000 7f9988c00640 1 -- 172.21.15.94:0/4052013189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f99840749c0 con 0x7f998415f5c0 2024-09-10T23:32:39.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.501+0000 7f997aa00640 1 -- 172.21.15.94:0/4052013189 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7f997c05eff0 con 0x7f998415f5c0 2024-09-10T23:32:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:39.503 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:32:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:32:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-09-10T23:32:39.504 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-09-10T23:32:39.504 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:32:39.504 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:32:39.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.501+0000 7f9988c00640 1 -- 172.21.15.94:0/4052013189 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f99600725e0 msgr2=0x7f9960074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:39.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.501+0000 7f9988c00640 1 --2- 172.21.15.94:0/4052013189 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f99600725e0 0x7f9960074aa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f99740af000 tx=0x7f997401b000 comp rx=0 tx=0).stop 2024-09-10T23:32:39.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.501+0000 7f9988c00640 1 -- 172.21.15.94:0/4052013189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998415f5c0 msgr2=0x7f998406dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:39.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.501+0000 7f9988c00640 1 --2- 172.21.15.94:0/4052013189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998415f5c0 0x7f998406dd30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f997c00eb20 tx=0x7f997c00c2d0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.501+0000 7f9988c00640 1 -- 172.21.15.94:0/4052013189 shutdown_connections 2024-09-10T23:32:39.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.501+0000 7f9988c00640 1 --2- 172.21.15.94:0/4052013189 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f99600725e0 0x7f9960074aa0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.501+0000 7f9988c00640 1 --2- 172.21.15.94:0/4052013189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f998415f5c0 0x7f998406dd30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.501+0000 7f9988c00640 1 --2- 172.21.15.94:0/4052013189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99840b2bb0 0x7f9984153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.501+0000 7f9988c00640 1 --2- 172.21.15.94:0/4052013189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99840af310 0x7f9984152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.501+0000 7f9988c00640 1 -- 172.21.15.94:0/4052013189 >> 172.21.15.94:0/4052013189 conn(0x7f99840d1c60 msgr2=0x7f99840d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:39.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.501+0000 7f9988c00640 1 -- 172.21.15.94:0/4052013189 shutdown_connections 2024-09-10T23:32:39.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.501+0000 7f9988c00640 1 -- 172.21.15.94:0/4052013189 wait complete. 2024-09-10T23:32:39.524 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T23:32:39.524 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T23:32:39.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.665+0000 7f77aa800640 1 -- 172.21.15.94:0/4227516423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a40cc2f0 msgr2=0x7f77a406f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:39.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.665+0000 7f77aa800640 1 --2- 172.21.15.94:0/4227516423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a40cc2f0 0x7f77a406f900 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f778c009840 tx=0x7f778c01c790 comp rx=0 tx=0).stop 2024-09-10T23:32:39.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.665+0000 7f77aa800640 1 -- 172.21.15.94:0/4227516423 shutdown_connections 2024-09-10T23:32:39.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.665+0000 7f77aa800640 1 --2- 172.21.15.94:0/4227516423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a40cc2f0 0x7f77a406f900 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.665+0000 7f77aa800640 1 -- 172.21.15.94:0/4227516423 >> 172.21.15.94:0/4227516423 conn(0x7f77a40d02d0 msgr2=0x7f77a40d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:39.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.665+0000 7f77aa800640 1 -- 172.21.15.94:0/4227516423 shutdown_connections 2024-09-10T23:32:39.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.665+0000 7f77aa800640 1 -- 172.21.15.94:0/4227516423 wait complete. 2024-09-10T23:32:39.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f77aa800640 1 Processor -- start 2024-09-10T23:32:39.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f77aa800640 1 -- start start 2024-09-10T23:32:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f77aa800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77a406fe40 0x7f77a4099a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f77aa800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a40cc2f0 0x7f77a40b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f77aa800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77a415f3f0 0x7f77a415b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f77aa800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4169970 con 0x7f77a40cc2f0 2024-09-10T23:32:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f77aa800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41697f0 con 0x7f77a415f3f0 2024-09-10T23:32:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f77aa800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f77a4169af0 con 0x7f77a406fe40 2024-09-10T23:32:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f779aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a40cc2f0 0x7f77a40b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f779aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a40cc2f0 0x7f77a40b3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57568/0 (socket says 172.21.15.94:57568) 2024-09-10T23:32:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f779aa00640 1 -- 172.21.15.94:0/1232764778 learned_addr learned my addr 172.21.15.94:0/1232764778 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f77a3e00640 1 --2- 172.21.15.94:0/1232764778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77a415f3f0 0x7f77a415b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f77a3400640 1 --2- 172.21.15.94:0/1232764778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77a406fe40 0x7f77a4099a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:39.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f779aa00640 1 -- 172.21.15.94:0/1232764778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77a406fe40 msgr2=0x7f77a4099a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:39.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f779aa00640 1 --2- 172.21.15.94:0/1232764778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77a406fe40 0x7f77a4099a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f779aa00640 1 -- 172.21.15.94:0/1232764778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77a415f3f0 msgr2=0x7f77a415b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:39.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f779aa00640 1 --2- 172.21.15.94:0/1232764778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77a415f3f0 0x7f77a415b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:39.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f779aa00640 1 -- 172.21.15.94:0/1232764778 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a40c3dd0 con 0x7f77a40cc2f0 2024-09-10T23:32:39.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f779aa00640 1 --2- 172.21.15.94:0/1232764778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a40cc2f0 0x7f77a40b3100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f778c009790 tx=0x7f778c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:39.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f779be00640 1 -- 172.21.15.94:0/1232764778 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c0bb070 con 0x7f77a40cc2f0 2024-09-10T23:32:39.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f779be00640 1 -- 172.21.15.94:0/1232764778 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f778c005030 con 0x7f77a40cc2f0 2024-09-10T23:32:39.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f779be00640 1 -- 172.21.15.94:0/1232764778 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c0b2040 con 0x7f77a40cc2f0 2024-09-10T23:32:39.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f77aa800640 1 -- 172.21.15.94:0/1232764778 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a40b0740 con 0x7f77a40cc2f0 2024-09-10T23:32:39.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.669+0000 7f77aa800640 1 -- 172.21.15.94:0/1232764778 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a40b0d00 con 0x7f77a40cc2f0 2024-09-10T23:32:39.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.673+0000 7f77aa800640 1 -- 172.21.15.94:0/1232764778 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a406f900 con 0x7f77a40cc2f0 2024-09-10T23:32:39.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.673+0000 7f779be00640 1 -- 172.21.15.94:0/1232764778 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f778c0b5030 con 0x7f77a40cc2f0 2024-09-10T23:32:39.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.673+0000 7f779be00640 1 --2- 172.21.15.94:0/1232764778 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f77800725e0 0x7f7780074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:39.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.673+0000 7f779be00640 1 -- 172.21.15.94:0/1232764778 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f778c12c6f0 con 0x7f77a40cc2f0 2024-09-10T23:32:39.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.673+0000 7f77a3400640 1 --2- 172.21.15.94:0/1232764778 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f77800725e0 0x7f7780074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:39.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.677+0000 7f77a3400640 1 --2- 172.21.15.94:0/1232764778 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f77800725e0 0x7f7780074aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f77940012f0 tx=0x7f7794009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:39.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.677+0000 7f779be00640 1 -- 172.21.15.94:0/1232764778 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f778c016410 con 0x7f77a40cc2f0 2024-09-10T23:32:39.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:39.841+0000 7f77aa800640 1 -- 172.21.15.94:0/1232764778 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f77a40b6250 con 0x7f77a40cc2f0 2024-09-10T23:32:40.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.533+0000 7f779be00640 1 -- 172.21.15.94:0/1232764778 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7f778c0f79a0 con 0x7f77a40cc2f0 2024-09-10T23:32:40.535 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-09-10T23:32:40.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.533+0000 7f77aa800640 1 -- 172.21.15.94:0/1232764778 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f77800725e0 msgr2=0x7f7780074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:40.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.533+0000 7f77aa800640 1 --2- 172.21.15.94:0/1232764778 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f77800725e0 0x7f7780074aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f77940012f0 tx=0x7f7794009040 comp rx=0 tx=0).stop 2024-09-10T23:32:40.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.533+0000 7f77aa800640 1 -- 172.21.15.94:0/1232764778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a40cc2f0 msgr2=0x7f77a40b3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:40.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.533+0000 7f77aa800640 1 --2- 172.21.15.94:0/1232764778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a40cc2f0 0x7f77a40b3100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f778c009790 tx=0x7f778c004040 comp rx=0 tx=0).stop 2024-09-10T23:32:40.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.533+0000 7f77aa800640 1 -- 172.21.15.94:0/1232764778 shutdown_connections 2024-09-10T23:32:40.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.533+0000 7f77aa800640 1 --2- 172.21.15.94:0/1232764778 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f77800725e0 0x7f7780074aa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:40.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.533+0000 7f77aa800640 1 --2- 172.21.15.94:0/1232764778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77a415f3f0 0x7f77a415b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:40.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.533+0000 7f77aa800640 1 --2- 172.21.15.94:0/1232764778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77a40cc2f0 0x7f77a40b3100 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:40.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.533+0000 7f77aa800640 1 --2- 172.21.15.94:0/1232764778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77a406fe40 0x7f77a4099a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:40.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.533+0000 7f77aa800640 1 -- 172.21.15.94:0/1232764778 >> 172.21.15.94:0/1232764778 conn(0x7f77a40d02d0 msgr2=0x7f77a40ad060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:40.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.533+0000 7f77aa800640 1 -- 172.21.15.94:0/1232764778 shutdown_connections 2024-09-10T23:32:40.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.533+0000 7f77aa800640 1 -- 172.21.15.94:0/1232764778 wait complete. 2024-09-10T23:32:40.566 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T23:32:40.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.713+0000 7f082a200640 1 -- 172.21.15.94:0/2164409921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08240ce890 msgr2=0x7f08240cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:40.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.713+0000 7f082a200640 1 --2- 172.21.15.94:0/2164409921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08240ce890 0x7f08240cd710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0814009840 tx=0x7f081401c760 comp rx=0 tx=0).stop 2024-09-10T23:32:40.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.713+0000 7f082a200640 1 -- 172.21.15.94:0/2164409921 shutdown_connections 2024-09-10T23:32:40.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.713+0000 7f082a200640 1 --2- 172.21.15.94:0/2164409921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08240ce1e0 0x7f08240cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:40.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.713+0000 7f082a200640 1 --2- 172.21.15.94:0/2164409921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08240ce890 0x7f08240cd710 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:40.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.713+0000 7f082a200640 1 -- 172.21.15.94:0/2164409921 >> 172.21.15.94:0/2164409921 conn(0x7f08240d1c60 msgr2=0x7f08240d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:40.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.713+0000 7f082a200640 1 -- 172.21.15.94:0/2164409921 shutdown_connections 2024-09-10T23:32:40.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.713+0000 7f082a200640 1 -- 172.21.15.94:0/2164409921 wait complete. 2024-09-10T23:32:40.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f082a200640 1 Processor -- start 2024-09-10T23:32:40.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f082a200640 1 -- start start 2024-09-10T23:32:40.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f082a200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08240b1100 0x7f08240b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:40.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f082a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240ce1e0 0x7f082414db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:40.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f082a200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08240ce890 0x7f082414e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:40.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f082a200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f08240d08e0 con 0x7f08240ce1e0 2024-09-10T23:32:40.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f082a200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f08240d0760 con 0x7f08240b1100 2024-09-10T23:32:40.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f082a200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f08240d0a60 con 0x7f08240ce890 2024-09-10T23:32:40.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f0822a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240ce1e0 0x7f082414db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:40.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f0823400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08240b1100 0x7f08240b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:40.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f0823e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08240ce890 0x7f082414e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:40.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f0822a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240ce1e0 0x7f082414db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57572/0 (socket says 172.21.15.94:57572) 2024-09-10T23:32:40.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f0822a00640 1 -- 172.21.15.94:0/2449971554 learned_addr learned my addr 172.21.15.94:0/2449971554 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:40.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f0822a00640 1 -- 172.21.15.94:0/2449971554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08240ce890 msgr2=0x7f082414e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:40.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f0822a00640 1 --2- 172.21.15.94:0/2449971554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08240ce890 0x7f082414e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:40.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f0822a00640 1 -- 172.21.15.94:0/2449971554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08240b1100 msgr2=0x7f08240b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:40.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f0822a00640 1 --2- 172.21.15.94:0/2449971554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08240b1100 0x7f08240b6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:40.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f0822a00640 1 -- 172.21.15.94:0/2449971554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082415e170 con 0x7f08240ce1e0 2024-09-10T23:32:40.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f0823e00640 1 --2- 172.21.15.94:0/2449971554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08240ce890 0x7f082414e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:32:40.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f0823400640 1 --2- 172.21.15.94:0/2449971554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08240b1100 0x7f08240b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:32:40.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f0822a00640 1 --2- 172.21.15.94:0/2449971554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240ce1e0 0x7f082414db20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f081000e9f0 tx=0x7f081000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:40.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f081b400640 1 -- 172.21.15.94:0/2449971554 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081000cde0 con 0x7f08240ce1e0 2024-09-10T23:32:40.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f081b400640 1 -- 172.21.15.94:0/2449971554 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0810010520 con 0x7f08240ce1e0 2024-09-10T23:32:40.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f081b400640 1 -- 172.21.15.94:0/2449971554 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810010970 con 0x7f08240ce1e0 2024-09-10T23:32:40.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.717+0000 7f082a200640 1 -- 172.21.15.94:0/2449971554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08240b5ac0 con 0x7f08240ce1e0 2024-09-10T23:32:40.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.721+0000 7f082a200640 1 -- 172.21.15.94:0/2449971554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08240b6080 con 0x7f08240ce1e0 2024-09-10T23:32:40.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.721+0000 7f082a200640 1 -- 172.21.15.94:0/2449971554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08240cd710 con 0x7f08240ce1e0 2024-09-10T23:32:40.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.721+0000 7f081b400640 1 -- 172.21.15.94:0/2449971554 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0810010b10 con 0x7f08240ce1e0 2024-09-10T23:32:40.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.721+0000 7f081b400640 1 --2- 172.21.15.94:0/2449971554 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f08040725e0 0x7f0804074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:40.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.721+0000 7f081b400640 1 -- 172.21.15.94:0/2449971554 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f0810093af0 con 0x7f08240ce1e0 2024-09-10T23:32:40.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.725+0000 7f0823400640 1 --2- 172.21.15.94:0/2449971554 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f08040725e0 0x7f0804074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:40.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.725+0000 7f0823400640 1 --2- 172.21.15.94:0/2449971554 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f08040725e0 0x7f0804074aa0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f08140af000 tx=0x7f081401afa0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:40.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.729+0000 7f081b400640 1 -- 172.21.15.94:0/2449971554 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f081005ed50 con 0x7f08240ce1e0 2024-09-10T23:32:40.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:40.893+0000 7f082a200640 1 -- 172.21.15.94:0/2449971554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f08240ceec0 con 0x7f08240ce1e0 2024-09-10T23:32:41.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.537+0000 7f081b400640 1 -- 172.21.15.94:0/2449971554 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) ==== 170+0+0 (secure 0 0 0) 0x7f0810063780 con 0x7f08240ce1e0 2024-09-10T23:32:41.542 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-09-10T23:32:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.545+0000 7f082a200640 1 -- 172.21.15.94:0/2449971554 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f08040725e0 msgr2=0x7f0804074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.545+0000 7f082a200640 1 --2- 172.21.15.94:0/2449971554 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f08040725e0 0x7f0804074aa0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f08140af000 tx=0x7f081401afa0 comp rx=0 tx=0).stop 2024-09-10T23:32:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.545+0000 7f082a200640 1 -- 172.21.15.94:0/2449971554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240ce1e0 msgr2=0x7f082414db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.545+0000 7f082a200640 1 --2- 172.21.15.94:0/2449971554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240ce1e0 0x7f082414db20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f081000e9f0 tx=0x7f081000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:32:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.545+0000 7f082a200640 1 -- 172.21.15.94:0/2449971554 shutdown_connections 2024-09-10T23:32:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.545+0000 7f082a200640 1 --2- 172.21.15.94:0/2449971554 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f08040725e0 0x7f0804074aa0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.545+0000 7f082a200640 1 --2- 172.21.15.94:0/2449971554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08240ce890 0x7f082414e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.545+0000 7f082a200640 1 --2- 172.21.15.94:0/2449971554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240ce1e0 0x7f082414db20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.545+0000 7f082a200640 1 --2- 172.21.15.94:0/2449971554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08240b1100 0x7f08240b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.545+0000 7f082a200640 1 -- 172.21.15.94:0/2449971554 >> 172.21.15.94:0/2449971554 conn(0x7f08240d1c60 msgr2=0x7f08240b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.545+0000 7f082a200640 1 -- 172.21.15.94:0/2449971554 shutdown_connections 2024-09-10T23:32:41.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.545+0000 7f082a200640 1 -- 172.21.15.94:0/2449971554 wait complete. 2024-09-10T23:32:41.589 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T23:32:41.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.733+0000 7f1d2ac00640 1 -- 172.21.15.94:0/1028314518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d240740c0 msgr2=0x7f1d24074540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:41.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.733+0000 7f1d2ac00640 1 --2- 172.21.15.94:0/1028314518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d240740c0 0x7f1d24074540 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009c80 tx=0x7f1d1c01cda0 comp rx=0 tx=0).stop 2024-09-10T23:32:41.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.733+0000 7f1d2ac00640 1 -- 172.21.15.94:0/1028314518 shutdown_connections 2024-09-10T23:32:41.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.733+0000 7f1d2ac00640 1 --2- 172.21.15.94:0/1028314518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d240740c0 0x7f1d24074540 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:41.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.733+0000 7f1d2ac00640 1 --2- 172.21.15.94:0/1028314518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d2415fba0 0x7f1d2406ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:41.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.733+0000 7f1d2ac00640 1 -- 172.21.15.94:0/1028314518 >> 172.21.15.94:0/1028314518 conn(0x7f1d240cff40 msgr2=0x7f1d240d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:41.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.733+0000 7f1d2ac00640 1 -- 172.21.15.94:0/1028314518 shutdown_connections 2024-09-10T23:32:41.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.733+0000 7f1d2ac00640 1 -- 172.21.15.94:0/1028314518 wait complete. 2024-09-10T23:32:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.733+0000 7f1d2ac00640 1 Processor -- start 2024-09-10T23:32:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d2ac00640 1 -- start start 2024-09-10T23:32:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d2ac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d240740c0 0x7f1d241559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d2ac00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d2415c0c0 0x7f1d24153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d2ac00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d2415fba0 0x7f1d241564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d2ac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1d24168e40 con 0x7f1d240740c0 2024-09-10T23:32:41.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d2ac00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1d24168cc0 con 0x7f1d2415fba0 2024-09-10T23:32:41.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d2ac00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1d24168fc0 con 0x7f1d2415c0c0 2024-09-10T23:32:41.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d23e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d240740c0 0x7f1d241559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:41.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d23e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d240740c0 0x7f1d241559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57584/0 (socket says 172.21.15.94:57584) 2024-09-10T23:32:41.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d23e00640 1 -- 172.21.15.94:0/2485902092 learned_addr learned my addr 172.21.15.94:0/2485902092 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:41.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d1be00640 1 --2- 172.21.15.94:0/2485902092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d2415c0c0 0x7f1d24153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:41.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d28c00640 1 --2- 172.21.15.94:0/2485902092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d2415fba0 0x7f1d241564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:41.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d23e00640 1 -- 172.21.15.94:0/2485902092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d2415c0c0 msgr2=0x7f1d24153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:41.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d23e00640 1 --2- 172.21.15.94:0/2485902092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d2415c0c0 0x7f1d24153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:41.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d23e00640 1 -- 172.21.15.94:0/2485902092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d2415fba0 msgr2=0x7f1d241564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:41.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d23e00640 1 --2- 172.21.15.94:0/2485902092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d2415fba0 0x7f1d241564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:41.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d23e00640 1 -- 172.21.15.94:0/2485902092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d240cf590 con 0x7f1d240740c0 2024-09-10T23:32:41.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d28c00640 1 --2- 172.21.15.94:0/2485902092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d2415fba0 0x7f1d241564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:41.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d23e00640 1 --2- 172.21.15.94:0/2485902092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d240740c0 0x7f1d241559f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400e970 tx=0x7f1d1400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:41.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d20c00640 1 -- 172.21.15.94:0/2485902092 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1400cd80 con 0x7f1d240740c0 2024-09-10T23:32:41.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d20c00640 1 -- 172.21.15.94:0/2485902092 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d14004020 con 0x7f1d240740c0 2024-09-10T23:32:41.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d2ac00640 1 -- 172.21.15.94:0/2485902092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d24159750 con 0x7f1d240740c0 2024-09-10T23:32:41.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d20c00640 1 -- 172.21.15.94:0/2485902092 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14004470 con 0x7f1d240740c0 2024-09-10T23:32:41.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d2ac00640 1 -- 172.21.15.94:0/2485902092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d241569f0 con 0x7f1d240740c0 2024-09-10T23:32:41.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.737+0000 7f1d2ac00640 1 -- 172.21.15.94:0/2485902092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d24159540 con 0x7f1d240740c0 2024-09-10T23:32:41.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.741+0000 7f1d20c00640 1 -- 172.21.15.94:0/2485902092 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1d14003600 con 0x7f1d240740c0 2024-09-10T23:32:41.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.741+0000 7f1d20c00640 1 --2- 172.21.15.94:0/2485902092 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1d00072610 0x7f1d00074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:41.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.741+0000 7f1d20c00640 1 -- 172.21.15.94:0/2485902092 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f1d14093b60 con 0x7f1d240740c0 2024-09-10T23:32:41.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.741+0000 7f1d1be00640 1 --2- 172.21.15.94:0/2485902092 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1d00072610 0x7f1d00074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:41.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.741+0000 7f1d1be00640 1 --2- 172.21.15.94:0/2485902092 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1d00072610 0x7f1d00074ad0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1d24165ee0 tx=0x7f1d0c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:41.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.745+0000 7f1d20c00640 1 -- 172.21.15.94:0/2485902092 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1d1405f200 con 0x7f1d240740c0 2024-09-10T23:32:41.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:41.945+0000 7f1d2ac00640 1 -- 172.21.15.94:0/2485902092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1d24074540 con 0x7f1d240740c0 2024-09-10T23:32:42.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.541+0000 7f1d20c00640 1 -- 172.21.15.94:0/2485902092 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f1d1405eb00 con 0x7f1d240740c0 2024-09-10T23:32:42.545 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T23:32:42.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.545+0000 7f1d2ac00640 1 -- 172.21.15.94:0/2485902092 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1d00072610 msgr2=0x7f1d00074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:42.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.545+0000 7f1d2ac00640 1 --2- 172.21.15.94:0/2485902092 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1d00072610 0x7f1d00074ad0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1d24165ee0 tx=0x7f1d0c009040 comp rx=0 tx=0).stop 2024-09-10T23:32:42.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.545+0000 7f1d2ac00640 1 -- 172.21.15.94:0/2485902092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d240740c0 msgr2=0x7f1d241559f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:42.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.545+0000 7f1d2ac00640 1 --2- 172.21.15.94:0/2485902092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d240740c0 0x7f1d241559f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400e970 tx=0x7f1d1400ee30 comp rx=0 tx=0).stop 2024-09-10T23:32:42.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.545+0000 7f1d2ac00640 1 -- 172.21.15.94:0/2485902092 shutdown_connections 2024-09-10T23:32:42.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.545+0000 7f1d2ac00640 1 --2- 172.21.15.94:0/2485902092 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1d00072610 0x7f1d00074ad0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:42.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.545+0000 7f1d2ac00640 1 --2- 172.21.15.94:0/2485902092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d2415fba0 0x7f1d241564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:42.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.545+0000 7f1d2ac00640 1 --2- 172.21.15.94:0/2485902092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d2415c0c0 0x7f1d24153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:42.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.545+0000 7f1d2ac00640 1 --2- 172.21.15.94:0/2485902092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d240740c0 0x7f1d241559f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:42.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.545+0000 7f1d2ac00640 1 -- 172.21.15.94:0/2485902092 >> 172.21.15.94:0/2485902092 conn(0x7f1d240cff40 msgr2=0x7f1d2415ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:42.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.545+0000 7f1d2ac00640 1 -- 172.21.15.94:0/2485902092 shutdown_connections 2024-09-10T23:32:42.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.545+0000 7f1d2ac00640 1 -- 172.21.15.94:0/2485902092 wait complete. 2024-09-10T23:32:42.596 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:32:42.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.733+0000 7fcb90e00640 1 -- 172.21.15.94:0/1035195677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb70004e70 msgr2=0x7fcb7000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:42.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.733+0000 7fcb90e00640 1 --2- 172.21.15.94:0/1035195677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb70004e70 0x7fcb7000b9d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcb84001df0 tx=0x7fcb8401cb10 comp rx=0 tx=0).stop 2024-09-10T23:32:42.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.733+0000 7fcb8a800640 1 -- 172.21.15.94:0/1035195677 reap_dead start 2024-09-10T23:32:42.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.733+0000 7fcb90e00640 1 -- 172.21.15.94:0/1035195677 shutdown_connections 2024-09-10T23:32:42.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.733+0000 7fcb90e00640 1 -- 172.21.15.94:0/1035195677 >> 172.21.15.94:0/1035195677 conn(0x7fcb8c0d1c60 msgr2=0x7fcb8c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:42.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.733+0000 7fcb90e00640 1 -- 172.21.15.94:0/1035195677 shutdown_connections 2024-09-10T23:32:42.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.733+0000 7fcb90e00640 1 -- 172.21.15.94:0/1035195677 wait complete. 2024-09-10T23:32:42.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb90e00640 1 Processor -- start 2024-09-10T23:32:42.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb90e00640 1 -- start start 2024-09-10T23:32:42.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb90e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb70004e70 0x7fcb8c1fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:42.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb90e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb8c15a650 0x7fcb8c1587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:42.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb90e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb8c0c64f0 0x7fcb8c155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:42.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb90e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c167ef0 con 0x7fcb70004e70 2024-09-10T23:32:42.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb90e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c167d70 con 0x7fcb8c15a650 2024-09-10T23:32:42.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb90e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fcb8c168070 con 0x7fcb8c0c64f0 2024-09-10T23:32:42.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb89e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb70004e70 0x7fcb8c1fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:42.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb89e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb70004e70 0x7fcb8c1fc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57604/0 (socket says 172.21.15.94:57604) 2024-09-10T23:32:42.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb89e00640 1 -- 172.21.15.94:0/3169720313 learned_addr learned my addr 172.21.15.94:0/3169720313 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:42.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb89400640 1 --2- 172.21.15.94:0/3169720313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb8c15a650 0x7fcb8c1587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:42.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb8a800640 1 --2- 172.21.15.94:0/3169720313 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb8c0c64f0 0x7fcb8c155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:42.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb89e00640 1 -- 172.21.15.94:0/3169720313 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb8c0c64f0 msgr2=0x7fcb8c155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:42.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb89e00640 1 --2- 172.21.15.94:0/3169720313 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb8c0c64f0 0x7fcb8c155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:42.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb89e00640 1 -- 172.21.15.94:0/3169720313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb8c15a650 msgr2=0x7fcb8c1587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:42.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb89e00640 1 --2- 172.21.15.94:0/3169720313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb8c15a650 0x7fcb8c1587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:42.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb89e00640 1 -- 172.21.15.94:0/3169720313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c1fc6e0 con 0x7fcb70004e70 2024-09-10T23:32:42.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb8a800640 1 --2- 172.21.15.94:0/3169720313 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb8c0c64f0 0x7fcb8c155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:42.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb89e00640 1 --2- 172.21.15.94:0/3169720313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb70004e70 0x7fcb8c1fc1a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c011ae0 tx=0x7fcb7c009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:42.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb82000640 1 -- 172.21.15.94:0/3169720313 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c01e070 con 0x7fcb70004e70 2024-09-10T23:32:42.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb82000640 1 -- 172.21.15.94:0/3169720313 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb7c00fe40 con 0x7fcb70004e70 2024-09-10T23:32:42.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb90e00640 1 -- 172.21.15.94:0/3169720313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8c15ab10 con 0x7fcb70004e70 2024-09-10T23:32:42.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb82000640 1 -- 172.21.15.94:0/3169720313 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c0042b0 con 0x7fcb70004e70 2024-09-10T23:32:42.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb90e00640 1 -- 172.21.15.94:0/3169720313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb8c15b0d0 con 0x7fcb70004e70 2024-09-10T23:32:42.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.737+0000 7fcb90e00640 1 -- 172.21.15.94:0/3169720313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8c0b6070 con 0x7fcb70004e70 2024-09-10T23:32:42.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.741+0000 7fcb82000640 1 -- 172.21.15.94:0/3169720313 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcb7c003600 con 0x7fcb70004e70 2024-09-10T23:32:42.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.741+0000 7fcb82000640 1 --2- 172.21.15.94:0/3169720313 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcb6c0725e0 0x7fcb6c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:42.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.741+0000 7fcb82000640 1 -- 172.21.15.94:0/3169720313 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcb7c093f50 con 0x7fcb70004e70 2024-09-10T23:32:42.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.741+0000 7fcb89400640 1 --2- 172.21.15.94:0/3169720313 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcb6c0725e0 0x7fcb6c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:42.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.741+0000 7fcb89400640 1 --2- 172.21.15.94:0/3169720313 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcb6c0725e0 0x7fcb6c074aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb78001870 tx=0x7fcb78006040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:42.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.745+0000 7fcb82000640 1 -- 172.21.15.94:0/3169720313 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcb8c1fc6e0 con 0x7fcb70004e70 2024-09-10T23:32:42.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.909+0000 7fcb90e00640 1 -- 172.21.15.94:0/3169720313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb8c0749c0 con 0x7fcb70004e70 2024-09-10T23:32:42.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.909+0000 7fcb82000640 1 -- 172.21.15.94:0/3169720313 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16670 (secure 0 0 0) 0x7fcb7c05f090 con 0x7fcb70004e70 2024-09-10T23:32:42.914 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:42.914 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":21,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:32:41.948942+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:32:39.855046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T23:32:40.904161+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:32:42.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.913+0000 7fcb90e00640 1 -- 172.21.15.94:0/3169720313 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcb6c0725e0 msgr2=0x7fcb6c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:42.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.913+0000 7fcb90e00640 1 --2- 172.21.15.94:0/3169720313 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcb6c0725e0 0x7fcb6c074aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb78001870 tx=0x7fcb78006040 comp rx=0 tx=0).stop 2024-09-10T23:32:42.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.913+0000 7fcb90e00640 1 -- 172.21.15.94:0/3169720313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb70004e70 msgr2=0x7fcb8c1fc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:42.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.913+0000 7fcb90e00640 1 --2- 172.21.15.94:0/3169720313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb70004e70 0x7fcb8c1fc1a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c011ae0 tx=0x7fcb7c009ea0 comp rx=0 tx=0).stop 2024-09-10T23:32:42.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.913+0000 7fcb90e00640 1 -- 172.21.15.94:0/3169720313 shutdown_connections 2024-09-10T23:32:42.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.913+0000 7fcb90e00640 1 --2- 172.21.15.94:0/3169720313 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcb6c0725e0 0x7fcb6c074aa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:42.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.913+0000 7fcb90e00640 1 --2- 172.21.15.94:0/3169720313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb8c15a650 0x7fcb8c1587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:42.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.913+0000 7fcb90e00640 1 --2- 172.21.15.94:0/3169720313 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb8c0c64f0 0x7fcb8c155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:42.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.913+0000 7fcb90e00640 1 --2- 172.21.15.94:0/3169720313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb70004e70 0x7fcb8c1fc1a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:42.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.913+0000 7fcb90e00640 1 -- 172.21.15.94:0/3169720313 >> 172.21.15.94:0/3169720313 conn(0x7fcb8c0d1c60 msgr2=0x7fcb8c0d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:42.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.913+0000 7fcb90e00640 1 -- 172.21.15.94:0/3169720313 shutdown_connections 2024-09-10T23:32:42.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:42.913+0000 7fcb90e00640 1 -- 172.21.15.94:0/3169720313 wait complete. 2024-09-10T23:32:42.935 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:32:43.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.073+0000 7fb437e00640 1 -- 172.21.15.94:0/243820140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4300d03c0 msgr2=0x7fb4300cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.073+0000 7fb437e00640 1 --2- 172.21.15.94:0/243820140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4300d03c0 0x7fb4300cddb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb428009bc0 tx=0x7fb42801cae0 comp rx=0 tx=0).stop 2024-09-10T23:32:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.073+0000 7fb437e00640 1 -- 172.21.15.94:0/243820140 shutdown_connections 2024-09-10T23:32:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.073+0000 7fb437e00640 1 --2- 172.21.15.94:0/243820140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4300d03c0 0x7fb4300cddb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.073+0000 7fb437e00640 1 --2- 172.21.15.94:0/243820140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb43015ba50 0x7fb4300b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.073+0000 7fb437e00640 1 -- 172.21.15.94:0/243820140 >> 172.21.15.94:0/243820140 conn(0x7fb43006cdf0 msgr2=0x7fb43006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.073+0000 7fb437e00640 1 -- 172.21.15.94:0/243820140 shutdown_connections 2024-09-10T23:32:43.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.073+0000 7fb437e00640 1 -- 172.21.15.94:0/243820140 wait complete. 2024-09-10T23:32:43.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.073+0000 7fb437e00640 1 Processor -- start 2024-09-10T23:32:43.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.073+0000 7fb437e00640 1 -- start start 2024-09-10T23:32:43.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.073+0000 7fb437e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4300b4c40 0x7fb430140630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:43.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.073+0000 7fb437e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4300d03c0 0x7fb43015c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:43.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.073+0000 7fb437e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb43015ba50 0x7fb43006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:43.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.073+0000 7fb437e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb430168be0 con 0x7fb43015ba50 2024-09-10T23:32:43.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.073+0000 7fb437e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb430168a60 con 0x7fb4300d03c0 2024-09-10T23:32:43.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.073+0000 7fb437e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb430168d60 con 0x7fb4300b4c40 2024-09-10T23:32:43.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb435e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb43015ba50 0x7fb43006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:43.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb435e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb43015ba50 0x7fb43006e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57620/0 (socket says 172.21.15.94:57620) 2024-09-10T23:32:43.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb435e00640 1 -- 172.21.15.94:0/3477783026 learned_addr learned my addr 172.21.15.94:0/3477783026 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:43.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb435400640 1 --2- 172.21.15.94:0/3477783026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4300b4c40 0x7fb430140630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:43.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb434a00640 1 --2- 172.21.15.94:0/3477783026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4300d03c0 0x7fb43015c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:43.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb435e00640 1 -- 172.21.15.94:0/3477783026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4300b4c40 msgr2=0x7fb430140630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:43.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb435e00640 1 --2- 172.21.15.94:0/3477783026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4300b4c40 0x7fb430140630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:43.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb435e00640 1 -- 172.21.15.94:0/3477783026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4300d03c0 msgr2=0x7fb43015c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:43.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb435e00640 1 --2- 172.21.15.94:0/3477783026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4300d03c0 0x7fb43015c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:43.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb435e00640 1 -- 172.21.15.94:0/3477783026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb430140b70 con 0x7fb43015ba50 2024-09-10T23:32:43.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb435400640 1 --2- 172.21.15.94:0/3477783026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4300b4c40 0x7fb430140630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:43.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb435e00640 1 --2- 172.21.15.94:0/3477783026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb43015ba50 0x7fb43006e4c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb428009b10 tx=0x7fb428003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:43.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb42d600640 1 -- 172.21.15.94:0/3477783026 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4280b2040 con 0x7fb43015ba50 2024-09-10T23:32:43.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb42d600640 1 -- 172.21.15.94:0/3477783026 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4280a5e50 con 0x7fb43015ba50 2024-09-10T23:32:43.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb42d600640 1 -- 172.21.15.94:0/3477783026 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428003fd0 con 0x7fb43015ba50 2024-09-10T23:32:43.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb437e00640 1 -- 172.21.15.94:0/3477783026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43006ef40 con 0x7fb43015ba50 2024-09-10T23:32:43.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb437e00640 1 -- 172.21.15.94:0/3477783026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb43006f500 con 0x7fb43015ba50 2024-09-10T23:32:43.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb437e00640 1 -- 172.21.15.94:0/3477783026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4300cddb0 con 0x7fb43015ba50 2024-09-10T23:32:43.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.077+0000 7fb42d600640 1 -- 172.21.15.94:0/3477783026 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb4280b5070 con 0x7fb43015ba50 2024-09-10T23:32:43.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.081+0000 7fb42d600640 1 --2- 172.21.15.94:0/3477783026 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4100725e0 0x7fb410074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:43.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.081+0000 7fb42d600640 1 -- 172.21.15.94:0/3477783026 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb42812ce30 con 0x7fb43015ba50 2024-09-10T23:32:43.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.081+0000 7fb435400640 1 --2- 172.21.15.94:0/3477783026 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4100725e0 0x7fb410074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:43.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.081+0000 7fb435400640 1 --2- 172.21.15.94:0/3477783026 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4100725e0 0x7fb410074aa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb424009b90 tx=0x7fb4240091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:43.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.085+0000 7fb42d600640 1 -- 172.21.15.94:0/3477783026 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb428016790 con 0x7fb43015ba50 2024-09-10T23:32:43.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.249+0000 7fb437e00640 1 -- 172.21.15.94:0/3477783026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb430151920 con 0x7fb43015ba50 2024-09-10T23:32:43.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.249+0000 7fb42d600640 1 -- 172.21.15.94:0/3477783026 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16670 (secure 0 0 0) 0x7fb4280f8470 con 0x7fb43015ba50 2024-09-10T23:32:43.254 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:43.254 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":21,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:32:41.948942+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:32:39.855046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T23:32:40.904161+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:32:43.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.253+0000 7fb437e00640 1 -- 172.21.15.94:0/3477783026 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4100725e0 msgr2=0x7fb410074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:43.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.253+0000 7fb437e00640 1 --2- 172.21.15.94:0/3477783026 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4100725e0 0x7fb410074aa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb424009b90 tx=0x7fb4240091e0 comp rx=0 tx=0).stop 2024-09-10T23:32:43.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.253+0000 7fb437e00640 1 -- 172.21.15.94:0/3477783026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb43015ba50 msgr2=0x7fb43006e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:43.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.253+0000 7fb437e00640 1 --2- 172.21.15.94:0/3477783026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb43015ba50 0x7fb43006e4c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb428009b10 tx=0x7fb428003f60 comp rx=0 tx=0).stop 2024-09-10T23:32:43.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.253+0000 7fb437e00640 1 -- 172.21.15.94:0/3477783026 shutdown_connections 2024-09-10T23:32:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.253+0000 7fb437e00640 1 --2- 172.21.15.94:0/3477783026 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4100725e0 0x7fb410074aa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.253+0000 7fb437e00640 1 --2- 172.21.15.94:0/3477783026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4300d03c0 0x7fb43015c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.253+0000 7fb437e00640 1 --2- 172.21.15.94:0/3477783026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb43015ba50 0x7fb43006e4c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.253+0000 7fb437e00640 1 --2- 172.21.15.94:0/3477783026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4300b4c40 0x7fb430140630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.253+0000 7fb437e00640 1 -- 172.21.15.94:0/3477783026 >> 172.21.15.94:0/3477783026 conn(0x7fb43006cdf0 msgr2=0x7fb43006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.253+0000 7fb437e00640 1 -- 172.21.15.94:0/3477783026 shutdown_connections 2024-09-10T23:32:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.253+0000 7fb437e00640 1 -- 172.21.15.94:0/3477783026 wait complete. 2024-09-10T23:32:43.275 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T23:32:43.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.417+0000 7fd5aae00640 1 -- 172.21.15.94:0/1884414896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5a415f140 msgr2=0x7fd5a40b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:43.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.417+0000 7fd5aae00640 1 --2- 172.21.15.94:0/1884414896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5a415f140 0x7fd5a40b1170 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd598009900 tx=0x7fd59801c8b0 comp rx=0 tx=0).stop 2024-09-10T23:32:43.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.417+0000 7fd5aae00640 1 -- 172.21.15.94:0/1884414896 shutdown_connections 2024-09-10T23:32:43.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.417+0000 7fd5aae00640 1 --2- 172.21.15.94:0/1884414896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5a40af150 0x7fd5a40af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:43.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.417+0000 7fd5aae00640 1 --2- 172.21.15.94:0/1884414896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5a415f140 0x7fd5a40b1170 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:43.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.417+0000 7fd5aae00640 1 -- 172.21.15.94:0/1884414896 >> 172.21.15.94:0/1884414896 conn(0x7fd5a40d1b00 msgr2=0x7fd5a40d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:43.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.417+0000 7fd5aae00640 1 -- 172.21.15.94:0/1884414896 shutdown_connections 2024-09-10T23:32:43.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.417+0000 7fd5aae00640 1 -- 172.21.15.94:0/1884414896 wait complete. 2024-09-10T23:32:43.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.417+0000 7fd5aae00640 1 Processor -- start 2024-09-10T23:32:43.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.417+0000 7fd5aae00640 1 -- start start 2024-09-10T23:32:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5aae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5a40af150 0x7fd5a41f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5aae00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5a40b2ac0 0x7fd5a4148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5aae00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5a415f140 0x7fd5a40c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5aae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a4168950 con 0x7fd5a40af150 2024-09-10T23:32:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5aae00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a40b1fd0 con 0x7fd5a415f140 2024-09-10T23:32:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5aae00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a4168ad0 con 0x7fd5a40b2ac0 2024-09-10T23:32:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5a9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5a40af150 0x7fd5a41f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5a9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5a40af150 0x7fd5a41f7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57624/0 (socket says 172.21.15.94:57624) 2024-09-10T23:32:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5a9a00640 1 -- 172.21.15.94:0/2951467304 learned_addr learned my addr 172.21.15.94:0/2951467304 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5aa400640 1 --2- 172.21.15.94:0/2951467304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5a415f140 0x7fd5a40c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5a9000640 1 --2- 172.21.15.94:0/2951467304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5a40b2ac0 0x7fd5a4148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5a9a00640 1 -- 172.21.15.94:0/2951467304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5a40b2ac0 msgr2=0x7fd5a4148620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5a9a00640 1 --2- 172.21.15.94:0/2951467304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5a40b2ac0 0x7fd5a4148620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5a9a00640 1 -- 172.21.15.94:0/2951467304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5a415f140 msgr2=0x7fd5a40c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5a9a00640 1 --2- 172.21.15.94:0/2951467304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5a415f140 0x7fd5a40c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5a9a00640 1 -- 172.21.15.94:0/2951467304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a40c7370 con 0x7fd5a40af150 2024-09-10T23:32:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5a9a00640 1 --2- 172.21.15.94:0/2951467304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5a40af150 0x7fd5a41f7f60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5980b0040 tx=0x7fd598005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5a1600640 1 -- 172.21.15.94:0/2951467304 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5980b2040 con 0x7fd5a40af150 2024-09-10T23:32:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5a1600640 1 -- 172.21.15.94:0/2951467304 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd598005320 con 0x7fd5a40af150 2024-09-10T23:32:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5a1600640 1 -- 172.21.15.94:0/2951467304 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5980046d0 con 0x7fd5a40af150 2024-09-10T23:32:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5aae00640 1 -- 172.21.15.94:0/2951467304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a41f7480 con 0x7fd5a40af150 2024-09-10T23:32:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5aae00640 1 -- 172.21.15.94:0/2951467304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a41f7a40 con 0x7fd5a40af150 2024-09-10T23:32:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5aae00640 1 -- 172.21.15.94:0/2951467304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a40af5c0 con 0x7fd5a40af150 2024-09-10T23:32:43.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.421+0000 7fd5a1600640 1 -- 172.21.15.94:0/2951467304 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd598004890 con 0x7fd5a40af150 2024-09-10T23:32:43.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.425+0000 7fd5a1600640 1 --2- 172.21.15.94:0/2951467304 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd584072610 0x7fd584074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:43.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.425+0000 7fd5a1600640 1 -- 172.21.15.94:0/2951467304 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd59812cfe0 con 0x7fd5a40af150 2024-09-10T23:32:43.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.425+0000 7fd5a9000640 1 --2- 172.21.15.94:0/2951467304 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd584072610 0x7fd584074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:43.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.425+0000 7fd5a9000640 1 --2- 172.21.15.94:0/2951467304 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd584072610 0x7fd584074ad0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4155c60 tx=0x7fd590009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:43.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.429+0000 7fd5a1600640 1 -- 172.21.15.94:0/2951467304 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd5980164d0 con 0x7fd5a40af150 2024-09-10T23:32:43.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:43.633+0000 7fd5aae00640 1 -- 172.21.15.94:0/2951467304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd5a40d0440 con 0x7fd5a40af150 2024-09-10T23:32:44.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.569+0000 7fd5a1600640 1 -- 172.21.15.94:0/2951467304 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fd5980f8680 con 0x7fd5a40af150 2024-09-10T23:32:44.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.573+0000 7fd5aae00640 1 -- 172.21.15.94:0/2951467304 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd584072610 msgr2=0x7fd584074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:44.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.573+0000 7fd5aae00640 1 --2- 172.21.15.94:0/2951467304 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd584072610 0x7fd584074ad0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4155c60 tx=0x7fd590009040 comp rx=0 tx=0).stop 2024-09-10T23:32:44.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.573+0000 7fd5aae00640 1 -- 172.21.15.94:0/2951467304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5a40af150 msgr2=0x7fd5a41f7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:44.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.573+0000 7fd5aae00640 1 --2- 172.21.15.94:0/2951467304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5a40af150 0x7fd5a41f7f60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5980b0040 tx=0x7fd598005de0 comp rx=0 tx=0).stop 2024-09-10T23:32:44.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.573+0000 7fd5aae00640 1 -- 172.21.15.94:0/2951467304 shutdown_connections 2024-09-10T23:32:44.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.573+0000 7fd5aae00640 1 --2- 172.21.15.94:0/2951467304 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd584072610 0x7fd584074ad0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:44.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.573+0000 7fd5aae00640 1 --2- 172.21.15.94:0/2951467304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5a415f140 0x7fd5a40c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:44.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.573+0000 7fd5aae00640 1 --2- 172.21.15.94:0/2951467304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5a40b2ac0 0x7fd5a4148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:44.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.573+0000 7fd5aae00640 1 --2- 172.21.15.94:0/2951467304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5a40af150 0x7fd5a41f7f60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:44.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.573+0000 7fd5aae00640 1 -- 172.21.15.94:0/2951467304 >> 172.21.15.94:0/2951467304 conn(0x7fd5a40d1b00 msgr2=0x7fd5a406ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:44.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.573+0000 7fd5aae00640 1 -- 172.21.15.94:0/2951467304 shutdown_connections 2024-09-10T23:32:44.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.573+0000 7fd5aae00640 1 -- 172.21.15.94:0/2951467304 wait complete. 2024-09-10T23:32:44.597 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T23:32:44.598 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T23:32:44.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.741+0000 7fae98800640 1 -- 172.21.15.94:0/4175029920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae78004d50 msgr2=0x7fae780051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:44.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.741+0000 7fae98800640 1 --2- 172.21.15.94:0/4175029920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae78004d50 0x7fae780051d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00c660 tx=0x7fae8c01fa40 comp rx=0 tx=0).stop 2024-09-10T23:32:44.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.741+0000 7fae96800640 1 -- 172.21.15.94:0/4175029920 reap_dead start 2024-09-10T23:32:44.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae98800640 1 -- 172.21.15.94:0/4175029920 shutdown_connections 2024-09-10T23:32:44.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae98800640 1 -- 172.21.15.94:0/4175029920 >> 172.21.15.94:0/4175029920 conn(0x7fae900d1c60 msgr2=0x7fae900d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:44.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae98800640 1 -- 172.21.15.94:0/4175029920 shutdown_connections 2024-09-10T23:32:44.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae98800640 1 -- 172.21.15.94:0/4175029920 wait complete. 2024-09-10T23:32:44.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae98800640 1 Processor -- start 2024-09-10T23:32:44.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae98800640 1 -- start start 2024-09-10T23:32:44.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae98800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae78004d50 0x7fae90159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:44.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae98800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae900b5ae0 0x7fae9015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:44.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae98800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae9015a800 0x7fae900cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:44.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae98800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fae900cdaa0 con 0x7fae9015a800 2024-09-10T23:32:44.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae98800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fae900cd920 con 0x7fae78004d50 2024-09-10T23:32:44.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae98800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fae901630e0 con 0x7fae900b5ae0 2024-09-10T23:32:44.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae96800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae9015a800 0x7fae900cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:44.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae96800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae9015a800 0x7fae900cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50804/0 (socket says 172.21.15.94:50804) 2024-09-10T23:32:44.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae96800640 1 -- 172.21.15.94:0/3998999397 learned_addr learned my addr 172.21.15.94:0/3998999397 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:44.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae95e00640 1 --2- 172.21.15.94:0/3998999397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae78004d50 0x7fae90159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:44.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae95400640 1 --2- 172.21.15.94:0/3998999397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae900b5ae0 0x7fae9015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:44.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae96800640 1 -- 172.21.15.94:0/3998999397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae900b5ae0 msgr2=0x7fae9015a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:44.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae96800640 1 --2- 172.21.15.94:0/3998999397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae900b5ae0 0x7fae9015a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:44.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae96800640 1 -- 172.21.15.94:0/3998999397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae78004d50 msgr2=0x7fae90159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:44.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae96800640 1 --2- 172.21.15.94:0/3998999397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae78004d50 0x7fae90159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:44.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae96800640 1 -- 172.21.15.94:0/3998999397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae900c5700 con 0x7fae9015a800 2024-09-10T23:32:44.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae96800640 1 --2- 172.21.15.94:0/3998999397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae9015a800 0x7fae900cb540 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fae8c000f90 tx=0x7fae8c001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:44.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae8a000640 1 -- 172.21.15.94:0/3998999397 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c00c280 con 0x7fae9015a800 2024-09-10T23:32:44.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae8a000640 1 -- 172.21.15.94:0/3998999397 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae8c004e40 con 0x7fae9015a800 2024-09-10T23:32:44.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae8a000640 1 -- 172.21.15.94:0/3998999397 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0020a0 con 0x7fae9015a800 2024-09-10T23:32:44.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.745+0000 7fae98800640 1 -- 172.21.15.94:0/3998999397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae900cafc0 con 0x7fae9015a800 2024-09-10T23:32:44.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.749+0000 7fae98800640 1 -- 172.21.15.94:0/3998999397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae9015a010 con 0x7fae9015a800 2024-09-10T23:32:44.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.749+0000 7fae98800640 1 -- 172.21.15.94:0/3998999397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae900b15b0 con 0x7fae9015a800 2024-09-10T23:32:44.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.749+0000 7fae8a000640 1 -- 172.21.15.94:0/3998999397 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fae8c003980 con 0x7fae9015a800 2024-09-10T23:32:44.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.749+0000 7fae8a000640 1 --2- 172.21.15.94:0/3998999397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fae740725e0 0x7fae74074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:44.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.749+0000 7fae8a000640 1 -- 172.21.15.94:0/3998999397 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fae8c12f480 con 0x7fae9015a800 2024-09-10T23:32:44.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.749+0000 7fae95e00640 1 --2- 172.21.15.94:0/3998999397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fae740725e0 0x7fae74074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:44.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.753+0000 7fae95e00640 1 --2- 172.21.15.94:0/3998999397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fae740725e0 0x7fae74074aa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fae901617a0 tx=0x7fae840011a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:44.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.753+0000 7fae8a000640 1 -- 172.21.15.94:0/3998999397 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fae8c0c2020 con 0x7fae9015a800 2024-09-10T23:32:44.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:44.953+0000 7fae98800640 1 -- 172.21.15.94:0/3998999397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fae900b2d90 con 0x7fae9015a800 2024-09-10T23:32:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.573+0000 7fae8a000640 1 -- 172.21.15.94:0/3998999397 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fae8c0197f0 con 0x7fae9015a800 2024-09-10T23:32:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.577+0000 7fae98800640 1 -- 172.21.15.94:0/3998999397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fae740725e0 msgr2=0x7fae74074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.577+0000 7fae98800640 1 --2- 172.21.15.94:0/3998999397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fae740725e0 0x7fae74074aa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fae901617a0 tx=0x7fae840011a0 comp rx=0 tx=0).stop 2024-09-10T23:32:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.577+0000 7fae98800640 1 -- 172.21.15.94:0/3998999397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae9015a800 msgr2=0x7fae900cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.577+0000 7fae98800640 1 --2- 172.21.15.94:0/3998999397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae9015a800 0x7fae900cb540 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fae8c000f90 tx=0x7fae8c001230 comp rx=0 tx=0).stop 2024-09-10T23:32:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.577+0000 7fae98800640 1 -- 172.21.15.94:0/3998999397 shutdown_connections 2024-09-10T23:32:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.577+0000 7fae98800640 1 --2- 172.21.15.94:0/3998999397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fae740725e0 0x7fae74074aa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:45.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.577+0000 7fae98800640 1 --2- 172.21.15.94:0/3998999397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae9015a800 0x7fae900cb540 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:45.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.577+0000 7fae98800640 1 --2- 172.21.15.94:0/3998999397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae900b5ae0 0x7fae9015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:45.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.577+0000 7fae98800640 1 --2- 172.21.15.94:0/3998999397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae78004d50 0x7fae90159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:45.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.577+0000 7fae98800640 1 -- 172.21.15.94:0/3998999397 >> 172.21.15.94:0/3998999397 conn(0x7fae900d1c60 msgr2=0x7fae900b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:45.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.577+0000 7fae98800640 1 -- 172.21.15.94:0/3998999397 shutdown_connections 2024-09-10T23:32:45.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.577+0000 7fae98800640 1 -- 172.21.15.94:0/3998999397 wait complete. 2024-09-10T23:32:45.599 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T23:32:45.600 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:32:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.737+0000 7f9cfd200640 1 -- 172.21.15.94:0/1905049911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cdc0040b0 msgr2=0x7f9cdc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.737+0000 7f9cfd200640 1 --2- 172.21.15.94:0/1905049911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cdc0040b0 0x7f9cdc004810 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40016c0 tx=0x7f9ce401c760 comp rx=0 tx=0).stop 2024-09-10T23:32:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.737+0000 7f9cf6800640 1 -- 172.21.15.94:0/1905049911 reap_dead start 2024-09-10T23:32:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.737+0000 7f9cfd200640 1 -- 172.21.15.94:0/1905049911 shutdown_connections 2024-09-10T23:32:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.737+0000 7f9cfd200640 1 -- 172.21.15.94:0/1905049911 >> 172.21.15.94:0/1905049911 conn(0x7f9cf806dd30 msgr2=0x7f9cf806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.737+0000 7f9cfd200640 1 -- 172.21.15.94:0/1905049911 shutdown_connections 2024-09-10T23:32:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.737+0000 7f9cfd200640 1 -- 172.21.15.94:0/1905049911 wait complete. 2024-09-10T23:32:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cfd200640 1 Processor -- start 2024-09-10T23:32:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cfd200640 1 -- start start 2024-09-10T23:32:45.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cfd200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cdc0040b0 0x7f9cf81f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:45.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cfd200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf8149730 0x7f9cf814e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cfd200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf81fc4c0 0x7f9cf814fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cfd200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf8167350 con 0x7f9cf8149730 2024-09-10T23:32:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cfd200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf81671d0 con 0x7f9cdc0040b0 2024-09-10T23:32:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cfd200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf81674d0 con 0x7f9cf81fc4c0 2024-09-10T23:32:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cf5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf8149730 0x7f9cf814e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cf5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf8149730 0x7f9cf814e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50828/0 (socket says 172.21.15.94:50828) 2024-09-10T23:32:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cf5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cdc0040b0 0x7f9cf81f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cf5400640 1 -- 172.21.15.94:0/3193556941 learned_addr learned my addr 172.21.15.94:0/3193556941 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cf6800640 1 --2- 172.21.15.94:0/3193556941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf81fc4c0 0x7f9cf814fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cf5400640 1 -- 172.21.15.94:0/3193556941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf81fc4c0 msgr2=0x7f9cf814fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cf5400640 1 --2- 172.21.15.94:0/3193556941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf81fc4c0 0x7f9cf814fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cf5400640 1 -- 172.21.15.94:0/3193556941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cdc0040b0 msgr2=0x7f9cf81f8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cf5400640 1 --2- 172.21.15.94:0/3193556941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cdc0040b0 0x7f9cf81f8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cf5400640 1 -- 172.21.15.94:0/3193556941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf81f81a0 con 0x7f9cf8149730 2024-09-10T23:32:45.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cf6800640 1 --2- 172.21.15.94:0/3193556941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf81fc4c0 0x7f9cf814fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:45.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cf5400640 1 --2- 172.21.15.94:0/3193556941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf8149730 0x7f9cf814e890 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ce401ae60 tx=0x7f9ce4003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:45.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cee000640 1 -- 172.21.15.94:0/3193556941 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4001310 con 0x7f9cf8149730 2024-09-10T23:32:45.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cee000640 1 -- 172.21.15.94:0/3193556941 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ce4006500 con 0x7f9cf8149730 2024-09-10T23:32:45.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cee000640 1 -- 172.21.15.94:0/3193556941 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4006930 con 0x7f9cf8149730 2024-09-10T23:32:45.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cfd200640 1 -- 172.21.15.94:0/3193556941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf81f9160 con 0x7f9cf8149730 2024-09-10T23:32:45.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.741+0000 7f9cfd200640 1 -- 172.21.15.94:0/3193556941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf814f460 con 0x7f9cf8149730 2024-09-10T23:32:45.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.745+0000 7f9cfd200640 1 -- 172.21.15.94:0/3193556941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf815e620 con 0x7f9cf8149730 2024-09-10T23:32:45.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.745+0000 7f9cee000640 1 -- 172.21.15.94:0/3193556941 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9ce40a64c0 con 0x7f9cf8149730 2024-09-10T23:32:45.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.745+0000 7f9cee000640 1 --2- 172.21.15.94:0/3193556941 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9cd80725e0 0x7f9cd8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:45.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.745+0000 7f9cee000640 1 -- 172.21.15.94:0/3193556941 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9ce412cf80 con 0x7f9cf8149730 2024-09-10T23:32:45.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.745+0000 7f9cf5e00640 1 --2- 172.21.15.94:0/3193556941 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9cd80725e0 0x7f9cd8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:45.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.745+0000 7f9cf5e00640 1 --2- 172.21.15.94:0/3193556941 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9cd80725e0 0x7f9cd8074aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009be0 tx=0x7f9ce80091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:45.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.749+0000 7f9cee000640 1 -- 172.21.15.94:0/3193556941 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9cf81f9160 con 0x7f9cf8149730 2024-09-10T23:32:45.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.945+0000 7f9cfd200640 1 -- 172.21.15.94:0/3193556941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cf8149b40 con 0x7f9cf8149730 2024-09-10T23:32:45.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.945+0000 7f9cee000640 1 -- 172.21.15.94:0/3193556941 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5233 (secure 0 0 0) 0x7f9cf81f81a0 con 0x7f9cf8149730 2024-09-10T23:32:45.948 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:45.949 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"btime":"2024-09-10T23:32:45:573319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/2278166692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2278166692},{"type":"v1","addr":"172.21.15.94:6834","nonce":2278166692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392173872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3392173872},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392173872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4380,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1894503548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1894503548},{"type":"v1","addr":"172.21.15.181:6837","nonce":1894503548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:32:41.948908+0000","modified":"2024-09-10T23:32:44.956870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194174},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194174":{"gid":194174,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1418253171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1418253171},{"type":"v1","addr":"172.21.15.181:6836","nonce":1418253171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194174,"qdb_cluster":[194174]},"id":1}]} 2024-09-10T23:32:45.949 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-09-10T23:32:45.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.949+0000 7f9cfd200640 1 -- 172.21.15.94:0/3193556941 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9cd80725e0 msgr2=0x7f9cd8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:45.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.949+0000 7f9cfd200640 1 --2- 172.21.15.94:0/3193556941 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9cd80725e0 0x7f9cd8074aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009be0 tx=0x7f9ce80091e0 comp rx=0 tx=0).stop 2024-09-10T23:32:45.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.949+0000 7f9cfd200640 1 -- 172.21.15.94:0/3193556941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf8149730 msgr2=0x7f9cf814e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:45.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.949+0000 7f9cfd200640 1 --2- 172.21.15.94:0/3193556941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf8149730 0x7f9cf814e890 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ce401ae60 tx=0x7f9ce4003bc0 comp rx=0 tx=0).stop 2024-09-10T23:32:45.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.949+0000 7f9cfd200640 1 -- 172.21.15.94:0/3193556941 shutdown_connections 2024-09-10T23:32:45.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.949+0000 7f9cfd200640 1 --2- 172.21.15.94:0/3193556941 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9cd80725e0 0x7f9cd8074aa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:45.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.949+0000 7f9cfd200640 1 --2- 172.21.15.94:0/3193556941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf81fc4c0 0x7f9cf814fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:45.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.949+0000 7f9cfd200640 1 --2- 172.21.15.94:0/3193556941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cf8149730 0x7f9cf814e890 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:45.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.949+0000 7f9cfd200640 1 --2- 172.21.15.94:0/3193556941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cdc0040b0 0x7f9cf81f8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:45.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.949+0000 7f9cfd200640 1 -- 172.21.15.94:0/3193556941 >> 172.21.15.94:0/3193556941 conn(0x7f9cf806dd30 msgr2=0x7f9cf80cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:45.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.949+0000 7f9cfd200640 1 -- 172.21.15.94:0/3193556941 shutdown_connections 2024-09-10T23:32:45.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:45.949+0000 7f9cfd200640 1 -- 172.21.15.94:0/3193556941 wait complete. 2024-09-10T23:32:45.971 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:32:46.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.109+0000 7fe891a00640 1 -- 172.21.15.94:0/3617574578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8700041d0 msgr2=0x7fe870004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:46.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.109+0000 7fe891a00640 1 --2- 172.21.15.94:0/3617574578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8700041d0 0x7fe870004810 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe874009b60 tx=0x7fe87401d240 comp rx=0 tx=0).stop 2024-09-10T23:32:46.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.109+0000 7fe88b400640 1 -- 172.21.15.94:0/3617574578 reap_dead start 2024-09-10T23:32:46.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.109+0000 7fe891a00640 1 -- 172.21.15.94:0/3617574578 shutdown_connections 2024-09-10T23:32:46.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.109+0000 7fe891a00640 1 -- 172.21.15.94:0/3617574578 >> 172.21.15.94:0/3617574578 conn(0x7fe88c09aa70 msgr2=0x7fe88c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:46.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.109+0000 7fe891a00640 1 -- 172.21.15.94:0/3617574578 shutdown_connections 2024-09-10T23:32:46.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.109+0000 7fe891a00640 1 -- 172.21.15.94:0/3617574578 wait complete. 2024-09-10T23:32:46.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.109+0000 7fe891a00640 1 Processor -- start 2024-09-10T23:32:46.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe891a00640 1 -- start start 2024-09-10T23:32:46.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe891a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8700041d0 0x7fe88c0b7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:46.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe891a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88c0cc430 0x7fe88c0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:46.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe891a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe88c0af2d0 0x7fe88c0b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:46.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe891a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c0d2550 con 0x7fe88c0cc430 2024-09-10T23:32:46.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe891a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c0d23d0 con 0x7fe8700041d0 2024-09-10T23:32:46.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe891a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe88c167b80 con 0x7fe88c0af2d0 2024-09-10T23:32:46.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe882000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88c0cc430 0x7fe88c0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:46.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe882000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88c0cc430 0x7fe88c0aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50838/0 (socket says 172.21.15.94:50838) 2024-09-10T23:32:46.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe882000640 1 -- 172.21.15.94:0/2807696131 learned_addr learned my addr 172.21.15.94:0/2807696131 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:46.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe88aa00640 1 --2- 172.21.15.94:0/2807696131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8700041d0 0x7fe88c0b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:46.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe88b400640 1 --2- 172.21.15.94:0/2807696131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe88c0af2d0 0x7fe88c0b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:46.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe882000640 1 -- 172.21.15.94:0/2807696131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe88c0af2d0 msgr2=0x7fe88c0b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:46.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe882000640 1 --2- 172.21.15.94:0/2807696131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe88c0af2d0 0x7fe88c0b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe882000640 1 -- 172.21.15.94:0/2807696131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8700041d0 msgr2=0x7fe88c0b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:46.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe882000640 1 --2- 172.21.15.94:0/2807696131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8700041d0 0x7fe88c0b7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe882000640 1 -- 172.21.15.94:0/2807696131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c0cc9d0 con 0x7fe88c0cc430 2024-09-10T23:32:46.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe882000640 1 --2- 172.21.15.94:0/2807696131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88c0cc430 0x7fe88c0aed90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe87401ccb0 tx=0x7fe874005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:46.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe883400640 1 -- 172.21.15.94:0/2807696131 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8740bd070 con 0x7fe88c0cc430 2024-09-10T23:32:46.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe883400640 1 -- 172.21.15.94:0/2807696131 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe874004030 con 0x7fe88c0cc430 2024-09-10T23:32:46.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe891a00640 1 -- 172.21.15.94:0/2807696131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88c0cccb0 con 0x7fe88c0cc430 2024-09-10T23:32:46.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe883400640 1 -- 172.21.15.94:0/2807696131 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8740b1040 con 0x7fe88c0cc430 2024-09-10T23:32:46.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe891a00640 1 -- 172.21.15.94:0/2807696131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88c0cd270 con 0x7fe88c0cc430 2024-09-10T23:32:46.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.113+0000 7fe891a00640 1 -- 172.21.15.94:0/2807696131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe88c0b1e70 con 0x7fe88c0cc430 2024-09-10T23:32:46.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.117+0000 7fe883400640 1 -- 172.21.15.94:0/2807696131 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe8740046d0 con 0x7fe88c0cc430 2024-09-10T23:32:46.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.117+0000 7fe883400640 1 --2- 172.21.15.94:0/2807696131 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe8680725e0 0x7fe868074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:46.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.117+0000 7fe883400640 1 -- 172.21.15.94:0/2807696131 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe87412d9f0 con 0x7fe88c0cc430 2024-09-10T23:32:46.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.117+0000 7fe88aa00640 1 --2- 172.21.15.94:0/2807696131 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe8680725e0 0x7fe868074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:46.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.117+0000 7fe88aa00640 1 --2- 172.21.15.94:0/2807696131 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe8680725e0 0x7fe868074aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe88c071630 tx=0x7fe87c006730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:46.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.121+0000 7fe883400640 1 -- 172.21.15.94:0/2807696131 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe8740a5e20 con 0x7fe88c0cc430 2024-09-10T23:32:46.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.285+0000 7fe891a00640 1 -- 172.21.15.94:0/2807696131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe88c0d02c0 con 0x7fe88c0cc430 2024-09-10T23:32:46.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.289+0000 7fe883400640 1 -- 172.21.15.94:0/2807696131 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16657 (secure 0 0 0) 0x7fe874016f80 con 0x7fe88c0cc430 2024-09-10T23:32:46.291 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:46.291 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":22,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:32:43.556132+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:32:39.855046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T23:32:40.904161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:32:46.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.289+0000 7fe891a00640 1 -- 172.21.15.94:0/2807696131 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe8680725e0 msgr2=0x7fe868074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:46.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.289+0000 7fe891a00640 1 --2- 172.21.15.94:0/2807696131 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe8680725e0 0x7fe868074aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe88c071630 tx=0x7fe87c006730 comp rx=0 tx=0).stop 2024-09-10T23:32:46.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.289+0000 7fe891a00640 1 -- 172.21.15.94:0/2807696131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88c0cc430 msgr2=0x7fe88c0aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:46.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.289+0000 7fe891a00640 1 --2- 172.21.15.94:0/2807696131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88c0cc430 0x7fe88c0aed90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe87401ccb0 tx=0x7fe874005520 comp rx=0 tx=0).stop 2024-09-10T23:32:46.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.289+0000 7fe891a00640 1 -- 172.21.15.94:0/2807696131 shutdown_connections 2024-09-10T23:32:46.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.289+0000 7fe891a00640 1 --2- 172.21.15.94:0/2807696131 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe8680725e0 0x7fe868074aa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.289+0000 7fe891a00640 1 --2- 172.21.15.94:0/2807696131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe88c0af2d0 0x7fe88c0b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.289+0000 7fe891a00640 1 --2- 172.21.15.94:0/2807696131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88c0cc430 0x7fe88c0aed90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.289+0000 7fe891a00640 1 --2- 172.21.15.94:0/2807696131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8700041d0 0x7fe88c0b7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.289+0000 7fe891a00640 1 -- 172.21.15.94:0/2807696131 >> 172.21.15.94:0/2807696131 conn(0x7fe88c09aa70 msgr2=0x7fe88c071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:46.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.289+0000 7fe891a00640 1 -- 172.21.15.94:0/2807696131 shutdown_connections 2024-09-10T23:32:46.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.289+0000 7fe891a00640 1 -- 172.21.15.94:0/2807696131 wait complete. 2024-09-10T23:32:46.312 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T23:32:46.313 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:32:46.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.449+0000 7f1c00c00640 1 -- 172.21.15.94:0/1840617505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bfc06e470 msgr2=0x7f1bfc0ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:46.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.449+0000 7f1c00c00640 1 --2- 172.21.15.94:0/1840617505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bfc06e470 0x7f1bfc0ce4e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1bec009c80 tx=0x7f1bec01cd10 comp rx=0 tx=0).stop 2024-09-10T23:32:46.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.449+0000 7f1c00c00640 1 -- 172.21.15.94:0/1840617505 shutdown_connections 2024-09-10T23:32:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.449+0000 7f1c00c00640 1 --2- 172.21.15.94:0/1840617505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bfc0d1490 0x7f1bfc0cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.449+0000 7f1c00c00640 1 --2- 172.21.15.94:0/1840617505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bfc06e470 0x7f1bfc0ce4e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.449+0000 7f1c00c00640 1 -- 172.21.15.94:0/1840617505 >> 172.21.15.94:0/1840617505 conn(0x7f1bfc0d1c60 msgr2=0x7f1bfc0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.449+0000 7f1c00c00640 1 -- 172.21.15.94:0/1840617505 shutdown_connections 2024-09-10T23:32:46.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.449+0000 7f1c00c00640 1 -- 172.21.15.94:0/1840617505 wait complete. 2024-09-10T23:32:46.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1c00c00640 1 Processor -- start 2024-09-10T23:32:46.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1c00c00640 1 -- start start 2024-09-10T23:32:46.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1c00c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bfc06dd30 0x7f1bfc155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:46.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1c00c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bfc06e470 0x7f1bfc1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:46.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1c00c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bfc0d1490 0x7f1bfc0b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:46.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1c00c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc16b590 con 0x7f1bfc06e470 2024-09-10T23:32:46.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1c00c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc16b410 con 0x7f1bfc06dd30 2024-09-10T23:32:46.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1c00c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1bfc16b710 con 0x7f1bfc0d1490 2024-09-10T23:32:46.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1bf1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bfc06e470 0x7f1bfc1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1bf1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bfc06e470 0x7f1bfc1596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50856/0 (socket says 172.21.15.94:50856) 2024-09-10T23:32:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1bf1400640 1 -- 172.21.15.94:0/3029863116 learned_addr learned my addr 172.21.15.94:0/3029863116 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1bf9e00640 1 --2- 172.21.15.94:0/3029863116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bfc06dd30 0x7f1bfc155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1bfa800640 1 --2- 172.21.15.94:0/3029863116 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bfc0d1490 0x7f1bfc0b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1bf1400640 1 -- 172.21.15.94:0/3029863116 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bfc0d1490 msgr2=0x7f1bfc0b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1bf1400640 1 --2- 172.21.15.94:0/3029863116 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bfc0d1490 0x7f1bfc0b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1bf1400640 1 -- 172.21.15.94:0/3029863116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bfc06dd30 msgr2=0x7f1bfc155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1bf1400640 1 --2- 172.21.15.94:0/3029863116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bfc06dd30 0x7f1bfc155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1bf1400640 1 -- 172.21.15.94:0/3029863116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bfc0cb8b0 con 0x7f1bfc06e470 2024-09-10T23:32:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1bfa800640 1 --2- 172.21.15.94:0/3029863116 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bfc0d1490 0x7f1bfc0b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1bf1400640 1 --2- 172.21.15.94:0/3029863116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bfc06e470 0x7f1bfc1596e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1be400c940 tx=0x7f1be400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1bf2a00640 1 -- 172.21.15.94:0/3029863116 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be4004220 con 0x7f1bfc06e470 2024-09-10T23:32:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1bf2a00640 1 -- 172.21.15.94:0/3029863116 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1be40079b0 con 0x7f1bfc06e470 2024-09-10T23:32:46.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1c00c00640 1 -- 172.21.15.94:0/3029863116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bfc0cbb90 con 0x7f1bfc06e470 2024-09-10T23:32:46.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1bf2a00640 1 -- 172.21.15.94:0/3029863116 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be40044b0 con 0x7f1bfc06e470 2024-09-10T23:32:46.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.453+0000 7f1c00c00640 1 -- 172.21.15.94:0/3029863116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bfc1593a0 con 0x7f1bfc06e470 2024-09-10T23:32:46.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.457+0000 7f1c00c00640 1 -- 172.21.15.94:0/3029863116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bfc161f30 con 0x7f1bfc06e470 2024-09-10T23:32:46.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.457+0000 7f1bf2a00640 1 -- 172.21.15.94:0/3029863116 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1be4003600 con 0x7f1bfc06e470 2024-09-10T23:32:46.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.457+0000 7f1bf2a00640 1 --2- 172.21.15.94:0/3029863116 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1bd80725e0 0x7f1bd8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:46.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.457+0000 7f1bf2a00640 1 -- 172.21.15.94:0/3029863116 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1be40936e0 con 0x7f1bfc06e470 2024-09-10T23:32:46.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.457+0000 7f1bf9e00640 1 --2- 172.21.15.94:0/3029863116 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1bd80725e0 0x7f1bd8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:46.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.457+0000 7f1bf9e00640 1 --2- 172.21.15.94:0/3029863116 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1bd80725e0 0x7f1bd8074aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0af000 tx=0x7f1bec01b550 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:46.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.461+0000 7f1bf2a00640 1 -- 172.21.15.94:0/3029863116 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1be4014030 con 0x7f1bfc06e470 2024-09-10T23:32:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.625+0000 7f1c00c00640 1 -- 172.21.15.94:0/3029863116 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1bfc161ca0 con 0x7f1bd80725e0 2024-09-10T23:32:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.645+0000 7f1bf2a00640 1 -- 172.21.15.94:0/3029863116 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242405 (secure 0 0 0) 0x7f1bfc161ca0 con 0x7f1bd80725e0 2024-09-10T23:32:46.649 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:32:46.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.645+0000 7f1c00c00640 1 -- 172.21.15.94:0/3029863116 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1bd80725e0 msgr2=0x7f1bd8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:46.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.645+0000 7f1c00c00640 1 --2- 172.21.15.94:0/3029863116 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1bd80725e0 0x7f1bd8074aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0af000 tx=0x7f1bec01b550 comp rx=0 tx=0).stop 2024-09-10T23:32:46.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.645+0000 7f1c00c00640 1 -- 172.21.15.94:0/3029863116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bfc06e470 msgr2=0x7f1bfc1596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:46.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.645+0000 7f1c00c00640 1 --2- 172.21.15.94:0/3029863116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bfc06e470 0x7f1bfc1596e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1be400c940 tx=0x7f1be400ce00 comp rx=0 tx=0).stop 2024-09-10T23:32:46.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.645+0000 7f1c00c00640 1 -- 172.21.15.94:0/3029863116 shutdown_connections 2024-09-10T23:32:46.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.645+0000 7f1c00c00640 1 --2- 172.21.15.94:0/3029863116 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1bd80725e0 0x7f1bd8074aa0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.645+0000 7f1c00c00640 1 --2- 172.21.15.94:0/3029863116 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bfc0d1490 0x7f1bfc0b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.645+0000 7f1c00c00640 1 --2- 172.21.15.94:0/3029863116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bfc06e470 0x7f1bfc1596e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.645+0000 7f1c00c00640 1 --2- 172.21.15.94:0/3029863116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bfc06dd30 0x7f1bfc155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.645+0000 7f1c00c00640 1 -- 172.21.15.94:0/3029863116 >> 172.21.15.94:0/3029863116 conn(0x7f1bfc0d1c60 msgr2=0x7f1bfc15e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.649+0000 7f1c00c00640 1 -- 172.21.15.94:0/3029863116 shutdown_connections 2024-09-10T23:32:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.649+0000 7f1c00c00640 1 -- 172.21.15.94:0/3029863116 wait complete. 2024-09-10T23:32:46.669 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-10T23:32:45.342130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":128,"acting":128,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":231292,"kb_used_data":2588,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754743428,"statfs":{"total":773094113280,"available":772857270272,"internally_reserved":0,"allocated":2650112,"data_stored":1630391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.804478"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.555557+0000","last_change":"2024-09-10T23:32:41.555557+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.555557+0000","last_undegraded":"2024-09-10T23:32:41.555557+0000","last_fullsized":"2024-09-10T23:32:41.555557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.552713+0000","last_change":"2024-09-10T23:32:41.552713+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.552713+0000","last_undegraded":"2024-09-10T23:32:41.552713+0000","last_fullsized":"2024-09-10T23:32:41.552713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.556029+0000","last_change":"2024-09-10T23:32:41.556029+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.556029+0000","last_undegraded":"2024-09-10T23:32:41.556029+0000","last_fullsized":"2024-09-10T23:32:41.556029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.551967+0000","last_change":"2024-09-10T23:32:41.551967+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.551967+0000","last_undegraded":"2024-09-10T23:32:41.551967+0000","last_fullsized":"2024-09-10T23:32:41.551967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.556000+0000","last_change":"2024-09-10T23:32:41.556000+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.556000+0000","last_undegraded":"2024-09-10T23:32:41.556000+0000","last_fullsized":"2024-09-10T23:32:41.556000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.560104+0000","last_change":"2024-09-10T23:32:41.560104+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.560104+0000","last_undegraded":"2024-09-10T23:32:41.560104+0000","last_fullsized":"2024-09-10T23:32:41.560104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.555506+0000","last_change":"2024-09-10T23:32:41.555506+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.555506+0000","last_undegraded":"2024-09-10T23:32:41.555506+0000","last_fullsized":"2024-09-10T23:32:41.555506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.555573+0000","last_change":"2024-09-10T23:32:41.555573+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.555573+0000","last_undegraded":"2024-09-10T23:32:41.555573+0000","last_fullsized":"2024-09-10T23:32:41.555573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.556388+0000","last_change":"2024-09-10T23:32:41.556388+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.556388+0000","last_undegraded":"2024-09-10T23:32:41.556388+0000","last_fullsized":"2024-09-10T23:32:41.556388+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.579129+0000","last_change":"2024-09-10T23:32:41.579129+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.579129+0000","last_undegraded":"2024-09-10T23:32:41.579129+0000","last_fullsized":"2024-09-10T23:32:41.579129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.578049+0000","last_change":"2024-09-10T23:32:41.578049+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.578049+0000","last_undegraded":"2024-09-10T23:32:41.578049+0000","last_fullsized":"2024-09-10T23:32:41.578049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.555463+0000","last_change":"2024-09-10T23:32:41.555463+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.555463+0000","last_undegraded":"2024-09-10T23:32:41.555463+0000","last_fullsized":"2024-09-10T23:32:41.555463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.559030+0000","last_change":"2024-09-10T23:32:41.559030+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.559030+0000","last_undegraded":"2024-09-10T23:32:41.559030+0000","last_fullsized":"2024-09-10T23:32:41.559030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.575570+0000","last_change":"2024-09-10T23:32:41.575570+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.575570+0000","last_undegraded":"2024-09-10T23:32:41.575570+0000","last_fullsized":"2024-09-10T23:32:41.575570+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.560780+0000","last_change":"2024-09-10T23:32:41.560780+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.560780+0000","last_undegraded":"2024-09-10T23:32:41.560780+0000","last_fullsized":"2024-09-10T23:32:41.560780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.575438+0000","last_change":"2024-09-10T23:32:41.575438+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.575438+0000","last_undegraded":"2024-09-10T23:32:41.575438+0000","last_fullsized":"2024-09-10T23:32:41.575438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.552836+0000","last_change":"2024-09-10T23:32:41.552836+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.552836+0000","last_undegraded":"2024-09-10T23:32:41.552836+0000","last_fullsized":"2024-09-10T23:32:41.552836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.558386+0000","last_change":"2024-09-10T23:32:41.558386+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.558386+0000","last_undegraded":"2024-09-10T23:32:41.558386+0000","last_fullsized":"2024-09-10T23:32:41.558386+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.546863+0000","last_change":"2024-09-10T23:32:40.546863+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.546863+0000","last_undegraded":"2024-09-10T23:32:40.546863+0000","last_fullsized":"2024-09-10T23:32:40.546863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T23:32:41.578717+0000","last_change":"2024-09-10T23:32:41.578635+0000","last_active":"2024-09-10T23:32:41.578717+0000","last_peered":"2024-09-10T23:32:41.578717+0000","last_clean":"2024-09-10T23:32:41.578717+0000","last_became_active":"2024-09-10T23:32:41.578346+0000","last_became_peered":"2024-09-10T23:32:41.578346+0000","last_unstale":"2024-09-10T23:32:41.578717+0000","last_undegraded":"2024-09-10T23:32:41.578717+0000","last_fullsized":"2024-09-10T23:32:41.578717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:31.006364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:40.529757+0000","last_change":"2024-09-10T23:32:40.529757+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.529757+0000","last_undegraded":"2024-09-10T23:32:40.529757+0000","last_fullsized":"2024-09-10T23:32:40.529757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.558201+0000","last_change":"2024-09-10T23:32:41.558201+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.558201+0000","last_undegraded":"2024-09-10T23:32:41.558201+0000","last_fullsized":"2024-09-10T23:32:41.558201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.559301+0000","last_change":"2024-09-10T23:32:41.559301+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.559301+0000","last_undegraded":"2024-09-10T23:32:41.559301+0000","last_fullsized":"2024-09-10T23:32:41.559301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.560253+0000","last_change":"2024-09-10T23:32:41.560253+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.560253+0000","last_undegraded":"2024-09-10T23:32:41.560253+0000","last_fullsized":"2024-09-10T23:32:41.560253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.546013+0000","last_change":"2024-09-10T23:32:41.546013+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.546013+0000","last_undegraded":"2024-09-10T23:32:41.546013+0000","last_fullsized":"2024-09-10T23:32:41.546013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.546146+0000","last_change":"2024-09-10T23:32:40.546146+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.546146+0000","last_undegraded":"2024-09-10T23:32:40.546146+0000","last_fullsized":"2024-09-10T23:32:40.546146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.576548+0000","last_change":"2024-09-10T23:32:41.576548+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.576548+0000","last_undegraded":"2024-09-10T23:32:41.576548+0000","last_fullsized":"2024-09-10T23:32:41.576548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T23:32:41.565370+0000","last_change":"2024-09-10T23:32:41.565270+0000","last_active":"2024-09-10T23:32:41.565370+0000","last_peered":"2024-09-10T23:32:41.565370+0000","last_clean":"2024-09-10T23:32:41.565370+0000","last_became_active":"2024-09-10T23:32:41.564864+0000","last_became_peered":"2024-09-10T23:32:41.564864+0000","last_unstale":"2024-09-10T23:32:41.565370+0000","last_undegraded":"2024-09-10T23:32:41.565370+0000","last_fullsized":"2024-09-10T23:32:41.565370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:41.753582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.554582+0000","last_change":"2024-09-10T23:32:41.554582+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.554582+0000","last_undegraded":"2024-09-10T23:32:41.554582+0000","last_fullsized":"2024-09-10T23:32:41.554582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.545784+0000","last_change":"2024-09-10T23:32:40.545784+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.545784+0000","last_undegraded":"2024-09-10T23:32:40.545784+0000","last_fullsized":"2024-09-10T23:32:40.545784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.559031+0000","last_change":"2024-09-10T23:32:41.559031+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.559031+0000","last_undegraded":"2024-09-10T23:32:41.559031+0000","last_fullsized":"2024-09-10T23:32:41.559031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.568964+0000","last_change":"2024-09-10T23:32:41.568964+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.568964+0000","last_undegraded":"2024-09-10T23:32:41.568964+0000","last_fullsized":"2024-09-10T23:32:41.568964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.545967+0000","last_change":"2024-09-10T23:32:41.545967+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.545967+0000","last_undegraded":"2024-09-10T23:32:41.545967+0000","last_fullsized":"2024-09-10T23:32:41.545967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.558320+0000","last_change":"2024-09-10T23:32:41.558320+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.558320+0000","last_undegraded":"2024-09-10T23:32:41.558320+0000","last_fullsized":"2024-09-10T23:32:41.558320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.541810+0000","last_change":"2024-09-10T23:32:40.541810+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.541810+0000","last_undegraded":"2024-09-10T23:32:40.541810+0000","last_fullsized":"2024-09-10T23:32:40.541810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.569261+0000","last_change":"2024-09-10T23:32:41.569261+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.569261+0000","last_undegraded":"2024-09-10T23:32:41.569261+0000","last_fullsized":"2024-09-10T23:32:41.569261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.546721+0000","last_change":"2024-09-10T23:32:41.546721+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.546721+0000","last_undegraded":"2024-09-10T23:32:41.546721+0000","last_fullsized":"2024-09-10T23:32:41.546721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.553226+0000","last_change":"2024-09-10T23:32:41.553226+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.553226+0000","last_undegraded":"2024-09-10T23:32:41.553226+0000","last_fullsized":"2024-09-10T23:32:41.553226+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.542501+0000","last_change":"2024-09-10T23:32:40.542501+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.542501+0000","last_undegraded":"2024-09-10T23:32:40.542501+0000","last_fullsized":"2024-09-10T23:32:40.542501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.553970+0000","last_change":"2024-09-10T23:32:41.553970+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.553970+0000","last_undegraded":"2024-09-10T23:32:41.553970+0000","last_fullsized":"2024-09-10T23:32:41.553970+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.546342+0000","last_change":"2024-09-10T23:32:40.546342+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.546342+0000","last_undegraded":"2024-09-10T23:32:40.546342+0000","last_fullsized":"2024-09-10T23:32:40.546342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.553272+0000","last_change":"2024-09-10T23:32:41.553272+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.553272+0000","last_undegraded":"2024-09-10T23:32:41.553272+0000","last_fullsized":"2024-09-10T23:32:41.553272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:40.529757+0000","last_change":"2024-09-10T23:32:40.529757+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.529757+0000","last_undegraded":"2024-09-10T23:32:40.529757+0000","last_fullsized":"2024-09-10T23:32:40.529757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.567080+0000","last_change":"2024-09-10T23:32:41.567080+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.567080+0000","last_undegraded":"2024-09-10T23:32:41.567080+0000","last_fullsized":"2024-09-10T23:32:41.567080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":107,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T23:32:41.562162+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:32:41.562162+0000","last_peered":"2024-09-10T23:32:41.562162+0000","last_clean":"2024-09-10T23:32:41.562162+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:32:41.562162+0000","last_undegraded":"2024-09-10T23:32:41.562162+0000","last_fullsized":"2024-09-10T23:32:41.562162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.567034+0000","last_change":"2024-09-10T23:32:41.567034+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.567034+0000","last_undegraded":"2024-09-10T23:32:41.567034+0000","last_fullsized":"2024-09-10T23:32:41.567034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T23:32:41.578399+0000","last_change":"2024-09-10T23:32:41.577467+0000","last_active":"2024-09-10T23:32:41.578399+0000","last_peered":"2024-09-10T23:32:41.578399+0000","last_clean":"2024-09-10T23:32:41.578399+0000","last_became_active":"2024-09-10T23:32:41.576702+0000","last_became_peered":"2024-09-10T23:32:41.576702+0000","last_unstale":"2024-09-10T23:32:41.578399+0000","last_undegraded":"2024-09-10T23:32:41.578399+0000","last_fullsized":"2024-09-10T23:32:41.578399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:05.600465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.545751+0000","last_change":"2024-09-10T23:32:40.545751+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.545751+0000","last_undegraded":"2024-09-10T23:32:40.545751+0000","last_fullsized":"2024-09-10T23:32:40.545751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T23:32:41.578263+0000","last_change":"2024-09-10T23:32:41.578157+0000","last_active":"2024-09-10T23:32:41.578263+0000","last_peered":"2024-09-10T23:32:41.578263+0000","last_clean":"2024-09-10T23:32:41.578263+0000","last_became_active":"2024-09-10T23:32:41.575442+0000","last_became_peered":"2024-09-10T23:32:41.575442+0000","last_unstale":"2024-09-10T23:32:41.578263+0000","last_undegraded":"2024-09-10T23:32:41.578263+0000","last_fullsized":"2024-09-10T23:32:41.578263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:40.438107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.551112+0000","last_change":"2024-09-10T23:32:41.551112+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.551112+0000","last_undegraded":"2024-09-10T23:32:41.551112+0000","last_fullsized":"2024-09-10T23:32:41.551112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.546815+0000","last_change":"2024-09-10T23:32:40.546815+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.546815+0000","last_undegraded":"2024-09-10T23:32:40.546815+0000","last_fullsized":"2024-09-10T23:32:40.546815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.555514+0000","last_change":"2024-09-10T23:32:41.555514+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.555514+0000","last_undegraded":"2024-09-10T23:32:41.555514+0000","last_fullsized":"2024-09-10T23:32:41.555514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.572501+0000","last_change":"2024-09-10T23:32:41.572501+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.572501+0000","last_undegraded":"2024-09-10T23:32:41.572501+0000","last_fullsized":"2024-09-10T23:32:41.572501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.540594+0000","last_change":"2024-09-10T23:32:40.540594+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.540594+0000","last_undegraded":"2024-09-10T23:32:40.540594+0000","last_fullsized":"2024-09-10T23:32:40.540594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T23:32:41.580152+0000","last_change":"2024-09-10T23:32:41.580054+0000","last_active":"2024-09-10T23:32:41.580152+0000","last_peered":"2024-09-10T23:32:41.580152+0000","last_clean":"2024-09-10T23:32:41.580152+0000","last_became_active":"2024-09-10T23:32:41.579663+0000","last_became_peered":"2024-09-10T23:32:41.579663+0000","last_unstale":"2024-09-10T23:32:41.580152+0000","last_undegraded":"2024-09-10T23:32:41.580152+0000","last_fullsized":"2024-09-10T23:32:41.580152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:58.170892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T23:32:41.562099+0000","last_change":"2024-09-10T23:32:41.562000+0000","last_active":"2024-09-10T23:32:41.562099+0000","last_peered":"2024-09-10T23:32:41.562099+0000","last_clean":"2024-09-10T23:32:41.562099+0000","last_became_active":"2024-09-10T23:32:41.561637+0000","last_became_peered":"2024-09-10T23:32:41.561637+0000","last_unstale":"2024-09-10T23:32:41.562099+0000","last_undegraded":"2024-09-10T23:32:41.562099+0000","last_fullsized":"2024-09-10T23:32:41.562099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:40.438107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.558369+0000","last_change":"2024-09-10T23:32:41.558369+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.558369+0000","last_undegraded":"2024-09-10T23:32:41.558369+0000","last_fullsized":"2024-09-10T23:32:41.558369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.541705+0000","last_change":"2024-09-10T23:32:40.541705+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.541705+0000","last_undegraded":"2024-09-10T23:32:40.541705+0000","last_fullsized":"2024-09-10T23:32:40.541705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:40.529757+0000","last_change":"2024-09-10T23:32:40.529757+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.529757+0000","last_undegraded":"2024-09-10T23:32:40.529757+0000","last_fullsized":"2024-09-10T23:32:40.529757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.563938+0000","last_change":"2024-09-10T23:32:41.563938+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.563938+0000","last_undegraded":"2024-09-10T23:32:41.563938+0000","last_fullsized":"2024-09-10T23:32:41.563938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.559861+0000","last_change":"2024-09-10T23:32:41.559861+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.559861+0000","last_undegraded":"2024-09-10T23:32:41.559861+0000","last_fullsized":"2024-09-10T23:32:41.559861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.575724+0000","last_change":"2024-09-10T23:32:41.575724+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.575724+0000","last_undegraded":"2024-09-10T23:32:41.575724+0000","last_fullsized":"2024-09-10T23:32:41.575724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.545970+0000","last_change":"2024-09-10T23:32:41.545970+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.545970+0000","last_undegraded":"2024-09-10T23:32:41.545970+0000","last_fullsized":"2024-09-10T23:32:41.545970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.559709+0000","last_change":"2024-09-10T23:32:41.559709+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.559709+0000","last_undegraded":"2024-09-10T23:32:41.559709+0000","last_fullsized":"2024-09-10T23:32:41.559709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.546029+0000","last_change":"2024-09-10T23:32:41.546029+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.546029+0000","last_undegraded":"2024-09-10T23:32:41.546029+0000","last_fullsized":"2024-09-10T23:32:41.546029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.547408+0000","last_change":"2024-09-10T23:32:40.547408+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.547408+0000","last_undegraded":"2024-09-10T23:32:40.547408+0000","last_fullsized":"2024-09-10T23:32:40.547408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.542309+0000","last_change":"2024-09-10T23:32:40.542309+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.542309+0000","last_undegraded":"2024-09-10T23:32:40.542309+0000","last_fullsized":"2024-09-10T23:32:40.542309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":68,"acting":68,"num_store_stats":4},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":58,"acting":58,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:32:46.673 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:32:46.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 -- 172.21.15.94:0/2273803817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2815ba30 msgr2=0x7f8b280b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:46.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 --2- 172.21.15.94:0/2273803817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2815ba30 0x7f8b280b1bd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8b280cc280 tx=0x7f8b1801c600 comp rx=0 tx=0).stop 2024-09-10T23:32:46.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 -- 172.21.15.94:0/2273803817 shutdown_connections 2024-09-10T23:32:46.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 --2- 172.21.15.94:0/2273803817 >> v2:172.21.15.181:3300/0 conn(0x7f8b280d02e0 0x7f8b2815f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 --2- 172.21.15.94:0/2273803817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b280b2110 0x7f8b280cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 --2- 172.21.15.94:0/2273803817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2815ba30 0x7f8b280b1bd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 -- 172.21.15.94:0/2273803817 >> 172.21.15.94:0/2273803817 conn(0x7f8b2809aa70 msgr2=0x7f8b2809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:46.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 -- 172.21.15.94:0/2273803817 shutdown_connections 2024-09-10T23:32:46.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 -- 172.21.15.94:0/2273803817 wait complete. 2024-09-10T23:32:46.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 Processor -- start 2024-09-10T23:32:46.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 -- start start 2024-09-10T23:32:46.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b280b2110 0x7f8b2814c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:46.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b280d02e0 0x7f8b280cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b2815ba30 0x7f8b281fad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8b280af410 con 0x7f8b280d02e0 2024-09-10T23:32:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8b280af290 con 0x7f8b2815ba30 2024-09-10T23:32:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b27200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8b280af590 con 0x7f8b280b2110 2024-09-10T23:32:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.849+0000 7f8b1d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b280d02e0 0x7f8b280cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b1d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b280d02e0 0x7f8b280cb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50872/0 (socket says 172.21.15.94:50872) 2024-09-10T23:32:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b1d400640 1 -- 172.21.15.94:0/2447907953 learned_addr learned my addr 172.21.15.94:0/2447907953 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b25e00640 1 --2- 172.21.15.94:0/2447907953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b280b2110 0x7f8b2814c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b26800640 1 --2- 172.21.15.94:0/2447907953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b2815ba30 0x7f8b281fad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b1d400640 1 -- 172.21.15.94:0/2447907953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b280b2110 msgr2=0x7f8b2814c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b1d400640 1 --2- 172.21.15.94:0/2447907953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b280b2110 0x7f8b2814c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b1d400640 1 -- 172.21.15.94:0/2447907953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b2815ba30 msgr2=0x7f8b281fad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b1d400640 1 --2- 172.21.15.94:0/2447907953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b2815ba30 0x7f8b281fad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b1d400640 1 -- 172.21.15.94:0/2447907953 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2806fd40 con 0x7f8b280d02e0 2024-09-10T23:32:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b1d400640 1 --2- 172.21.15.94:0/2447907953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b280d02e0 0x7f8b280cb3b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000d980 tx=0x7f8b1000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b1ea00640 1 -- 172.21.15.94:0/2447907953 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b10017070 con 0x7f8b280d02e0 2024-09-10T23:32:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b1ea00640 1 -- 172.21.15.94:0/2447907953 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b10009d60 con 0x7f8b280d02e0 2024-09-10T23:32:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b27200640 1 -- 172.21.15.94:0/2447907953 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b281fbad0 con 0x7f8b280d02e0 2024-09-10T23:32:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b1ea00640 1 -- 172.21.15.94:0/2447907953 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b10005280 con 0x7f8b280d02e0 2024-09-10T23:32:46.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b27200640 1 -- 172.21.15.94:0/2447907953 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2814feb0 con 0x7f8b280d02e0 2024-09-10T23:32:46.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b27200640 1 -- 172.21.15.94:0/2447907953 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2815f540 con 0x7f8b280d02e0 2024-09-10T23:32:46.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.853+0000 7f8b1ea00640 1 -- 172.21.15.94:0/2447907953 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8b10005420 con 0x7f8b280d02e0 2024-09-10T23:32:46.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.857+0000 7f8b1ea00640 1 --2- 172.21.15.94:0/2447907953 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b040725e0 0x7f8b04074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:46.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.857+0000 7f8b1ea00640 1 -- 172.21.15.94:0/2447907953 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8b10093410 con 0x7f8b280d02e0 2024-09-10T23:32:46.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.857+0000 7f8b25e00640 1 --2- 172.21.15.94:0/2447907953 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b040725e0 0x7f8b04074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:46.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.857+0000 7f8b25e00640 1 --2- 172.21.15.94:0/2447907953 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b040725e0 0x7f8b04074aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8b180af000 tx=0x7f8b1801ae40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:46.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:46.857+0000 7f8b1ea00640 1 -- 172.21.15.94:0/2447907953 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8b1005e400 con 0x7f8b280d02e0 2024-09-10T23:32:47.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.025+0000 7f8b27200640 1 -- 172.21.15.94:0/2447907953 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b280cb8f0 con 0x7f8b040725e0 2024-09-10T23:32:47.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.029+0000 7f8b1ea00640 1 -- 172.21.15.94:0/2447907953 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242405 (secure 0 0 0) 0x7f8b280cb8f0 con 0x7f8b040725e0 2024-09-10T23:32:47.033 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:47.034 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:32:47.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.033+0000 7f8b27200640 1 -- 172.21.15.94:0/2447907953 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b040725e0 msgr2=0x7f8b04074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:47.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.033+0000 7f8b27200640 1 --2- 172.21.15.94:0/2447907953 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b040725e0 0x7f8b04074aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8b180af000 tx=0x7f8b1801ae40 comp rx=0 tx=0).stop 2024-09-10T23:32:47.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.033+0000 7f8b27200640 1 -- 172.21.15.94:0/2447907953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b280d02e0 msgr2=0x7f8b280cb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:47.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.033+0000 7f8b27200640 1 --2- 172.21.15.94:0/2447907953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b280d02e0 0x7f8b280cb3b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000d980 tx=0x7f8b1000de40 comp rx=0 tx=0).stop 2024-09-10T23:32:47.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.033+0000 7f8b27200640 1 -- 172.21.15.94:0/2447907953 shutdown_connections 2024-09-10T23:32:47.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.033+0000 7f8b27200640 1 --2- 172.21.15.94:0/2447907953 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b040725e0 0x7f8b04074aa0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.033+0000 7f8b27200640 1 --2- 172.21.15.94:0/2447907953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b2815ba30 0x7f8b281fad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.033+0000 7f8b27200640 1 --2- 172.21.15.94:0/2447907953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b280d02e0 0x7f8b280cb3b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.033+0000 7f8b27200640 1 --2- 172.21.15.94:0/2447907953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b280b2110 0x7f8b2814c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.033+0000 7f8b27200640 1 -- 172.21.15.94:0/2447907953 >> 172.21.15.94:0/2447907953 conn(0x7f8b2809aa70 msgr2=0x7f8b280cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:47.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.033+0000 7f8b27200640 1 -- 172.21.15.94:0/2447907953 shutdown_connections 2024-09-10T23:32:47.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.033+0000 7f8b27200640 1 -- 172.21.15.94:0/2447907953 wait complete. 2024-09-10T23:32:47.053 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-10T23:32:45.342130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":128,"acting":128,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":231292,"kb_used_data":2588,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754743428,"statfs":{"total":773094113280,"available":772857270272,"internally_reserved":0,"allocated":2650112,"data_stored":1630391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.804478"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.555557+0000","last_change":"2024-09-10T23:32:41.555557+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.555557+0000","last_undegraded":"2024-09-10T23:32:41.555557+0000","last_fullsized":"2024-09-10T23:32:41.555557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.552713+0000","last_change":"2024-09-10T23:32:41.552713+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.552713+0000","last_undegraded":"2024-09-10T23:32:41.552713+0000","last_fullsized":"2024-09-10T23:32:41.552713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.556029+0000","last_change":"2024-09-10T23:32:41.556029+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.556029+0000","last_undegraded":"2024-09-10T23:32:41.556029+0000","last_fullsized":"2024-09-10T23:32:41.556029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.551967+0000","last_change":"2024-09-10T23:32:41.551967+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.551967+0000","last_undegraded":"2024-09-10T23:32:41.551967+0000","last_fullsized":"2024-09-10T23:32:41.551967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.556000+0000","last_change":"2024-09-10T23:32:41.556000+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.556000+0000","last_undegraded":"2024-09-10T23:32:41.556000+0000","last_fullsized":"2024-09-10T23:32:41.556000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.560104+0000","last_change":"2024-09-10T23:32:41.560104+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.560104+0000","last_undegraded":"2024-09-10T23:32:41.560104+0000","last_fullsized":"2024-09-10T23:32:41.560104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.555506+0000","last_change":"2024-09-10T23:32:41.555506+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.555506+0000","last_undegraded":"2024-09-10T23:32:41.555506+0000","last_fullsized":"2024-09-10T23:32:41.555506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.555573+0000","last_change":"2024-09-10T23:32:41.555573+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.555573+0000","last_undegraded":"2024-09-10T23:32:41.555573+0000","last_fullsized":"2024-09-10T23:32:41.555573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.556388+0000","last_change":"2024-09-10T23:32:41.556388+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.556388+0000","last_undegraded":"2024-09-10T23:32:41.556388+0000","last_fullsized":"2024-09-10T23:32:41.556388+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.579129+0000","last_change":"2024-09-10T23:32:41.579129+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.579129+0000","last_undegraded":"2024-09-10T23:32:41.579129+0000","last_fullsized":"2024-09-10T23:32:41.579129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.578049+0000","last_change":"2024-09-10T23:32:41.578049+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.578049+0000","last_undegraded":"2024-09-10T23:32:41.578049+0000","last_fullsized":"2024-09-10T23:32:41.578049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.555463+0000","last_change":"2024-09-10T23:32:41.555463+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.555463+0000","last_undegraded":"2024-09-10T23:32:41.555463+0000","last_fullsized":"2024-09-10T23:32:41.555463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.559030+0000","last_change":"2024-09-10T23:32:41.559030+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.559030+0000","last_undegraded":"2024-09-10T23:32:41.559030+0000","last_fullsized":"2024-09-10T23:32:41.559030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.575570+0000","last_change":"2024-09-10T23:32:41.575570+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.575570+0000","last_undegraded":"2024-09-10T23:32:41.575570+0000","last_fullsized":"2024-09-10T23:32:41.575570+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.560780+0000","last_change":"2024-09-10T23:32:41.560780+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.560780+0000","last_undegraded":"2024-09-10T23:32:41.560780+0000","last_fullsized":"2024-09-10T23:32:41.560780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.575438+0000","last_change":"2024-09-10T23:32:41.575438+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.575438+0000","last_undegraded":"2024-09-10T23:32:41.575438+0000","last_fullsized":"2024-09-10T23:32:41.575438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.552836+0000","last_change":"2024-09-10T23:32:41.552836+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.552836+0000","last_undegraded":"2024-09-10T23:32:41.552836+0000","last_fullsized":"2024-09-10T23:32:41.552836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.558386+0000","last_change":"2024-09-10T23:32:41.558386+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.558386+0000","last_undegraded":"2024-09-10T23:32:41.558386+0000","last_fullsized":"2024-09-10T23:32:41.558386+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.546863+0000","last_change":"2024-09-10T23:32:40.546863+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.546863+0000","last_undegraded":"2024-09-10T23:32:40.546863+0000","last_fullsized":"2024-09-10T23:32:40.546863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T23:32:41.578717+0000","last_change":"2024-09-10T23:32:41.578635+0000","last_active":"2024-09-10T23:32:41.578717+0000","last_peered":"2024-09-10T23:32:41.578717+0000","last_clean":"2024-09-10T23:32:41.578717+0000","last_became_active":"2024-09-10T23:32:41.578346+0000","last_became_peered":"2024-09-10T23:32:41.578346+0000","last_unstale":"2024-09-10T23:32:41.578717+0000","last_undegraded":"2024-09-10T23:32:41.578717+0000","last_fullsized":"2024-09-10T23:32:41.578717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:31.006364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:40.529757+0000","last_change":"2024-09-10T23:32:40.529757+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.529757+0000","last_undegraded":"2024-09-10T23:32:40.529757+0000","last_fullsized":"2024-09-10T23:32:40.529757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.558201+0000","last_change":"2024-09-10T23:32:41.558201+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.558201+0000","last_undegraded":"2024-09-10T23:32:41.558201+0000","last_fullsized":"2024-09-10T23:32:41.558201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.559301+0000","last_change":"2024-09-10T23:32:41.559301+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.559301+0000","last_undegraded":"2024-09-10T23:32:41.559301+0000","last_fullsized":"2024-09-10T23:32:41.559301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.560253+0000","last_change":"2024-09-10T23:32:41.560253+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.560253+0000","last_undegraded":"2024-09-10T23:32:41.560253+0000","last_fullsized":"2024-09-10T23:32:41.560253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.546013+0000","last_change":"2024-09-10T23:32:41.546013+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.546013+0000","last_undegraded":"2024-09-10T23:32:41.546013+0000","last_fullsized":"2024-09-10T23:32:41.546013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.546146+0000","last_change":"2024-09-10T23:32:40.546146+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.546146+0000","last_undegraded":"2024-09-10T23:32:40.546146+0000","last_fullsized":"2024-09-10T23:32:40.546146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.576548+0000","last_change":"2024-09-10T23:32:41.576548+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.576548+0000","last_undegraded":"2024-09-10T23:32:41.576548+0000","last_fullsized":"2024-09-10T23:32:41.576548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T23:32:41.565370+0000","last_change":"2024-09-10T23:32:41.565270+0000","last_active":"2024-09-10T23:32:41.565370+0000","last_peered":"2024-09-10T23:32:41.565370+0000","last_clean":"2024-09-10T23:32:41.565370+0000","last_became_active":"2024-09-10T23:32:41.564864+0000","last_became_peered":"2024-09-10T23:32:41.564864+0000","last_unstale":"2024-09-10T23:32:41.565370+0000","last_undegraded":"2024-09-10T23:32:41.565370+0000","last_fullsized":"2024-09-10T23:32:41.565370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:41.753582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.554582+0000","last_change":"2024-09-10T23:32:41.554582+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.554582+0000","last_undegraded":"2024-09-10T23:32:41.554582+0000","last_fullsized":"2024-09-10T23:32:41.554582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.545784+0000","last_change":"2024-09-10T23:32:40.545784+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.545784+0000","last_undegraded":"2024-09-10T23:32:40.545784+0000","last_fullsized":"2024-09-10T23:32:40.545784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.559031+0000","last_change":"2024-09-10T23:32:41.559031+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.559031+0000","last_undegraded":"2024-09-10T23:32:41.559031+0000","last_fullsized":"2024-09-10T23:32:41.559031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.568964+0000","last_change":"2024-09-10T23:32:41.568964+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.568964+0000","last_undegraded":"2024-09-10T23:32:41.568964+0000","last_fullsized":"2024-09-10T23:32:41.568964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.545967+0000","last_change":"2024-09-10T23:32:41.545967+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.545967+0000","last_undegraded":"2024-09-10T23:32:41.545967+0000","last_fullsized":"2024-09-10T23:32:41.545967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.558320+0000","last_change":"2024-09-10T23:32:41.558320+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.558320+0000","last_undegraded":"2024-09-10T23:32:41.558320+0000","last_fullsized":"2024-09-10T23:32:41.558320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.541810+0000","last_change":"2024-09-10T23:32:40.541810+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.541810+0000","last_undegraded":"2024-09-10T23:32:40.541810+0000","last_fullsized":"2024-09-10T23:32:40.541810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.569261+0000","last_change":"2024-09-10T23:32:41.569261+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.569261+0000","last_undegraded":"2024-09-10T23:32:41.569261+0000","last_fullsized":"2024-09-10T23:32:41.569261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.546721+0000","last_change":"2024-09-10T23:32:41.546721+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.546721+0000","last_undegraded":"2024-09-10T23:32:41.546721+0000","last_fullsized":"2024-09-10T23:32:41.546721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.553226+0000","last_change":"2024-09-10T23:32:41.553226+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.553226+0000","last_undegraded":"2024-09-10T23:32:41.553226+0000","last_fullsized":"2024-09-10T23:32:41.553226+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.542501+0000","last_change":"2024-09-10T23:32:40.542501+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.542501+0000","last_undegraded":"2024-09-10T23:32:40.542501+0000","last_fullsized":"2024-09-10T23:32:40.542501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.553970+0000","last_change":"2024-09-10T23:32:41.553970+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.553970+0000","last_undegraded":"2024-09-10T23:32:41.553970+0000","last_fullsized":"2024-09-10T23:32:41.553970+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.546342+0000","last_change":"2024-09-10T23:32:40.546342+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.546342+0000","last_undegraded":"2024-09-10T23:32:40.546342+0000","last_fullsized":"2024-09-10T23:32:40.546342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.553272+0000","last_change":"2024-09-10T23:32:41.553272+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.553272+0000","last_undegraded":"2024-09-10T23:32:41.553272+0000","last_fullsized":"2024-09-10T23:32:41.553272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:40.529757+0000","last_change":"2024-09-10T23:32:40.529757+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.529757+0000","last_undegraded":"2024-09-10T23:32:40.529757+0000","last_fullsized":"2024-09-10T23:32:40.529757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.567080+0000","last_change":"2024-09-10T23:32:41.567080+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.567080+0000","last_undegraded":"2024-09-10T23:32:41.567080+0000","last_fullsized":"2024-09-10T23:32:41.567080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":107,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T23:32:41.562162+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:32:41.562162+0000","last_peered":"2024-09-10T23:32:41.562162+0000","last_clean":"2024-09-10T23:32:41.562162+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:32:41.562162+0000","last_undegraded":"2024-09-10T23:32:41.562162+0000","last_fullsized":"2024-09-10T23:32:41.562162+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.567034+0000","last_change":"2024-09-10T23:32:41.567034+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.567034+0000","last_undegraded":"2024-09-10T23:32:41.567034+0000","last_fullsized":"2024-09-10T23:32:41.567034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T23:32:41.578399+0000","last_change":"2024-09-10T23:32:41.577467+0000","last_active":"2024-09-10T23:32:41.578399+0000","last_peered":"2024-09-10T23:32:41.578399+0000","last_clean":"2024-09-10T23:32:41.578399+0000","last_became_active":"2024-09-10T23:32:41.576702+0000","last_became_peered":"2024-09-10T23:32:41.576702+0000","last_unstale":"2024-09-10T23:32:41.578399+0000","last_undegraded":"2024-09-10T23:32:41.578399+0000","last_fullsized":"2024-09-10T23:32:41.578399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:05.600465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.545751+0000","last_change":"2024-09-10T23:32:40.545751+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.545751+0000","last_undegraded":"2024-09-10T23:32:40.545751+0000","last_fullsized":"2024-09-10T23:32:40.545751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T23:32:41.578263+0000","last_change":"2024-09-10T23:32:41.578157+0000","last_active":"2024-09-10T23:32:41.578263+0000","last_peered":"2024-09-10T23:32:41.578263+0000","last_clean":"2024-09-10T23:32:41.578263+0000","last_became_active":"2024-09-10T23:32:41.575442+0000","last_became_peered":"2024-09-10T23:32:41.575442+0000","last_unstale":"2024-09-10T23:32:41.578263+0000","last_undegraded":"2024-09-10T23:32:41.578263+0000","last_fullsized":"2024-09-10T23:32:41.578263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:40.438107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.551112+0000","last_change":"2024-09-10T23:32:41.551112+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.551112+0000","last_undegraded":"2024-09-10T23:32:41.551112+0000","last_fullsized":"2024-09-10T23:32:41.551112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.546815+0000","last_change":"2024-09-10T23:32:40.546815+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.546815+0000","last_undegraded":"2024-09-10T23:32:40.546815+0000","last_fullsized":"2024-09-10T23:32:40.546815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.555514+0000","last_change":"2024-09-10T23:32:41.555514+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.555514+0000","last_undegraded":"2024-09-10T23:32:41.555514+0000","last_fullsized":"2024-09-10T23:32:41.555514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.572501+0000","last_change":"2024-09-10T23:32:41.572501+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.572501+0000","last_undegraded":"2024-09-10T23:32:41.572501+0000","last_fullsized":"2024-09-10T23:32:41.572501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.540594+0000","last_change":"2024-09-10T23:32:40.540594+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.540594+0000","last_undegraded":"2024-09-10T23:32:40.540594+0000","last_fullsized":"2024-09-10T23:32:40.540594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T23:32:41.580152+0000","last_change":"2024-09-10T23:32:41.580054+0000","last_active":"2024-09-10T23:32:41.580152+0000","last_peered":"2024-09-10T23:32:41.580152+0000","last_clean":"2024-09-10T23:32:41.580152+0000","last_became_active":"2024-09-10T23:32:41.579663+0000","last_became_peered":"2024-09-10T23:32:41.579663+0000","last_unstale":"2024-09-10T23:32:41.580152+0000","last_undegraded":"2024-09-10T23:32:41.580152+0000","last_fullsized":"2024-09-10T23:32:41.580152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:58.170892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T23:32:41.562099+0000","last_change":"2024-09-10T23:32:41.562000+0000","last_active":"2024-09-10T23:32:41.562099+0000","last_peered":"2024-09-10T23:32:41.562099+0000","last_clean":"2024-09-10T23:32:41.562099+0000","last_became_active":"2024-09-10T23:32:41.561637+0000","last_became_peered":"2024-09-10T23:32:41.561637+0000","last_unstale":"2024-09-10T23:32:41.562099+0000","last_undegraded":"2024-09-10T23:32:41.562099+0000","last_fullsized":"2024-09-10T23:32:41.562099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:40.438107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.558369+0000","last_change":"2024-09-10T23:32:41.558369+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.558369+0000","last_undegraded":"2024-09-10T23:32:41.558369+0000","last_fullsized":"2024-09-10T23:32:41.558369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.541705+0000","last_change":"2024-09-10T23:32:40.541705+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.541705+0000","last_undegraded":"2024-09-10T23:32:40.541705+0000","last_fullsized":"2024-09-10T23:32:40.541705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:40.529757+0000","last_change":"2024-09-10T23:32:40.529757+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.529757+0000","last_undegraded":"2024-09-10T23:32:40.529757+0000","last_fullsized":"2024-09-10T23:32:40.529757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.563938+0000","last_change":"2024-09-10T23:32:41.563938+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.563938+0000","last_undegraded":"2024-09-10T23:32:41.563938+0000","last_fullsized":"2024-09-10T23:32:41.563938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.559861+0000","last_change":"2024-09-10T23:32:41.559861+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.559861+0000","last_undegraded":"2024-09-10T23:32:41.559861+0000","last_fullsized":"2024-09-10T23:32:41.559861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.575724+0000","last_change":"2024-09-10T23:32:41.575724+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.575724+0000","last_undegraded":"2024-09-10T23:32:41.575724+0000","last_fullsized":"2024-09-10T23:32:41.575724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.545970+0000","last_change":"2024-09-10T23:32:41.545970+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.545970+0000","last_undegraded":"2024-09-10T23:32:41.545970+0000","last_fullsized":"2024-09-10T23:32:41.545970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-10T23:32:41.559709+0000","last_change":"2024-09-10T23:32:41.559709+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.559709+0000","last_undegraded":"2024-09-10T23:32:41.559709+0000","last_fullsized":"2024-09-10T23:32:41.559709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":3,"reported_epoch":20,"state":"creating+activating","last_fresh":"2024-09-10T23:32:41.546029+0000","last_change":"2024-09-10T23:32:41.546029+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.546029+0000","last_undegraded":"2024-09-10T23:32:41.546029+0000","last_fullsized":"2024-09-10T23:32:41.546029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.547408+0000","last_change":"2024-09-10T23:32:40.547408+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.547408+0000","last_undegraded":"2024-09-10T23:32:40.547408+0000","last_fullsized":"2024-09-10T23:32:40.547408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T23:32:41.536785+0000","last_change":"2024-09-10T23:32:41.536785+0000","last_active":"2024-09-10T23:32:41.536785+0000","last_peered":"2024-09-10T23:32:41.536785+0000","last_clean":"2024-09-10T23:32:41.536785+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:41.536785+0000","last_undegraded":"2024-09-10T23:32:41.536785+0000","last_fullsized":"2024-09-10T23:32:41.536785+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-10T23:32:40.542309+0000","last_change":"2024-09-10T23:32:40.542309+0000","last_active":"2024-09-10T23:32:40.529757+0000","last_peered":"2024-09-10T23:32:40.529757+0000","last_clean":"2024-09-10T23:32:40.529757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:32:40.542309+0000","last_undegraded":"2024-09-10T23:32:40.542309+0000","last_fullsized":"2024-09-10T23:32:40.542309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":68,"acting":68,"num_store_stats":4},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":58,"acting":58,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:32:47.056 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:32:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.229+0000 7f6856200640 1 -- 172.21.15.94:0/1108263163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68500cfaf0 msgr2=0x7f6850071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.229+0000 7f6856200640 1 --2- 172.21.15.94:0/1108263163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68500cfaf0 0x7f6850071030 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6840009900 tx=0x7f684001c8b0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.229+0000 7f6856200640 1 -- 172.21.15.94:0/1108263163 shutdown_connections 2024-09-10T23:32:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.229+0000 7f6856200640 1 --2- 172.21.15.94:0/1108263163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f685015b100 0x7f6850070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.229+0000 7f6856200640 1 --2- 172.21.15.94:0/1108263163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68500cfaf0 0x7f6850071030 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.229+0000 7f6856200640 1 -- 172.21.15.94:0/1108263163 >> 172.21.15.94:0/1108263163 conn(0x7f68500d01e0 msgr2=0x7f68500d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.229+0000 7f6856200640 1 -- 172.21.15.94:0/1108263163 shutdown_connections 2024-09-10T23:32:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.229+0000 7f6856200640 1 -- 172.21.15.94:0/1108263163 wait complete. 2024-09-10T23:32:47.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6856200640 1 Processor -- start 2024-09-10T23:32:47.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6856200640 1 -- start start 2024-09-10T23:32:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6856200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6850070570 0x7f68500cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6856200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68500cfaf0 0x7f6850153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6856200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f685015b100 0x7f68500cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6856200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f685015e8f0 con 0x7f685015b100 2024-09-10T23:32:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6856200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f685015e770 con 0x7f6850070570 2024-09-10T23:32:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6856200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f685015ea70 con 0x7f68500cfaf0 2024-09-10T23:32:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6855800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f685015b100 0x7f68500cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6855800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f685015b100 0x7f68500cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50880/0 (socket says 172.21.15.94:50880) 2024-09-10T23:32:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6855800640 1 -- 172.21.15.94:0/2306178928 learned_addr learned my addr 172.21.15.94:0/2306178928 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6854e00640 1 --2- 172.21.15.94:0/2306178928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6850070570 0x7f68500cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6847e00640 1 --2- 172.21.15.94:0/2306178928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68500cfaf0 0x7f6850153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:47.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6855800640 1 -- 172.21.15.94:0/2306178928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68500cfaf0 msgr2=0x7f6850153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6855800640 1 --2- 172.21.15.94:0/2306178928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68500cfaf0 0x7f6850153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6855800640 1 -- 172.21.15.94:0/2306178928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6850070570 msgr2=0x7f68500cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6855800640 1 --2- 172.21.15.94:0/2306178928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6850070570 0x7f68500cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6855800640 1 -- 172.21.15.94:0/2306178928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68501fbb40 con 0x7f685015b100 2024-09-10T23:32:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6847e00640 1 --2- 172.21.15.94:0/2306178928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68500cfaf0 0x7f6850153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:32:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6855800640 1 --2- 172.21.15.94:0/2306178928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f685015b100 0x7f68500cc420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f684800c7c0 tx=0x7f684800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f684d600640 1 -- 172.21.15.94:0/2306178928 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848003b80 con 0x7f685015b100 2024-09-10T23:32:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f684d600640 1 -- 172.21.15.94:0/2306178928 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6848003d20 con 0x7f685015b100 2024-09-10T23:32:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f684d600640 1 -- 172.21.15.94:0/2306178928 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684800aab0 con 0x7f685015b100 2024-09-10T23:32:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.233+0000 7f6856200640 1 -- 172.21.15.94:0/2306178928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68501fbe20 con 0x7f685015b100 2024-09-10T23:32:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.237+0000 7f6856200640 1 -- 172.21.15.94:0/2306178928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68501fc3e0 con 0x7f685015b100 2024-09-10T23:32:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.237+0000 7f6856200640 1 -- 172.21.15.94:0/2306178928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6850071030 con 0x7f685015b100 2024-09-10T23:32:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.237+0000 7f684d600640 1 -- 172.21.15.94:0/2306178928 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f684800ac50 con 0x7f685015b100 2024-09-10T23:32:47.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.237+0000 7f684d600640 1 --2- 172.21.15.94:0/2306178928 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f682c0725e0 0x7f682c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:47.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.237+0000 7f684d600640 1 -- 172.21.15.94:0/2306178928 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f68480931b0 con 0x7f685015b100 2024-09-10T23:32:47.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.237+0000 7f6854e00640 1 --2- 172.21.15.94:0/2306178928 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f682c0725e0 0x7f682c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:47.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.241+0000 7f6854e00640 1 --2- 172.21.15.94:0/2306178928 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f682c0725e0 0x7f682c074aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f68400af000 tx=0x7f684001b0f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:47.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.241+0000 7f684d600640 1 -- 172.21.15.94:0/2306178928 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f684805e7f0 con 0x7f685015b100 2024-09-10T23:32:47.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.405+0000 7f6856200640 1 -- 172.21.15.94:0/2306178928 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f68500d0f40 con 0x7f682c0725e0 2024-09-10T23:32:47.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.413+0000 7f684d600640 1 -- 172.21.15.94:0/2306178928 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251933 (secure 0 0 0) 0x7f68400f9830 con 0x7f682c0725e0 2024-09-10T23:32:47.418 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:47.419 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:32:47.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.417+0000 7f6856200640 1 -- 172.21.15.94:0/2306178928 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f682c0725e0 msgr2=0x7f682c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:47.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.417+0000 7f6856200640 1 --2- 172.21.15.94:0/2306178928 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f682c0725e0 0x7f682c074aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f68400af000 tx=0x7f684001b0f0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.417+0000 7f6856200640 1 -- 172.21.15.94:0/2306178928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f685015b100 msgr2=0x7f68500cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:47.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.417+0000 7f6856200640 1 --2- 172.21.15.94:0/2306178928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f685015b100 0x7f68500cc420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f684800c7c0 tx=0x7f684800cc80 comp rx=0 tx=0).stop 2024-09-10T23:32:47.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.417+0000 7f6856200640 1 -- 172.21.15.94:0/2306178928 shutdown_connections 2024-09-10T23:32:47.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.417+0000 7f6856200640 1 --2- 172.21.15.94:0/2306178928 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f682c0725e0 0x7f682c074aa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.417+0000 7f6856200640 1 --2- 172.21.15.94:0/2306178928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f685015b100 0x7f68500cc420 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.417+0000 7f6856200640 1 --2- 172.21.15.94:0/2306178928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68500cfaf0 0x7f6850153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.417+0000 7f6856200640 1 --2- 172.21.15.94:0/2306178928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6850070570 0x7f68500cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.417+0000 7f6856200640 1 -- 172.21.15.94:0/2306178928 >> 172.21.15.94:0/2306178928 conn(0x7f68500d01e0 msgr2=0x7f6850151850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:47.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.417+0000 7f6856200640 1 -- 172.21.15.94:0/2306178928 shutdown_connections 2024-09-10T23:32:47.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.417+0000 7f6856200640 1 -- 172.21.15.94:0/2306178928 wait complete. 2024-09-10T23:32:47.438 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-10T23:32:47.342717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235848,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738872,"statfs":{"total":773094113280,"available":772852604928,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001381"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627741+0000","last_change":"2024-09-10T23:32:42.593279+0000","last_active":"2024-09-10T23:32:43.627741+0000","last_peered":"2024-09-10T23:32:43.627741+0000","last_clean":"2024-09-10T23:32:43.627741+0000","last_became_active":"2024-09-10T23:32:42.593056+0000","last_became_peered":"2024-09-10T23:32:42.593056+0000","last_unstale":"2024-09-10T23:32:43.627741+0000","last_undegraded":"2024-09-10T23:32:43.627741+0000","last_fullsized":"2024-09-10T23:32:43.627741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.112551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628801+0000","last_change":"2024-09-10T23:32:42.584101+0000","last_active":"2024-09-10T23:32:43.628801+0000","last_peered":"2024-09-10T23:32:43.628801+0000","last_clean":"2024-09-10T23:32:43.628801+0000","last_became_active":"2024-09-10T23:32:42.583796+0000","last_became_peered":"2024-09-10T23:32:42.583796+0000","last_unstale":"2024-09-10T23:32:43.628801+0000","last_undegraded":"2024-09-10T23:32:43.628801+0000","last_fullsized":"2024-09-10T23:32:43.628801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:01.463027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628392+0000","last_change":"2024-09-10T23:32:42.584302+0000","last_active":"2024-09-10T23:32:43.628392+0000","last_peered":"2024-09-10T23:32:43.628392+0000","last_clean":"2024-09-10T23:32:43.628392+0000","last_became_active":"2024-09-10T23:32:42.583989+0000","last_became_peered":"2024-09-10T23:32:42.583989+0000","last_unstale":"2024-09-10T23:32:43.628392+0000","last_undegraded":"2024-09-10T23:32:43.628392+0000","last_fullsized":"2024-09-10T23:32:43.628392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:53.622106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626948+0000","last_change":"2024-09-10T23:32:42.565401+0000","last_active":"2024-09-10T23:32:43.626948+0000","last_peered":"2024-09-10T23:32:43.626948+0000","last_clean":"2024-09-10T23:32:43.626948+0000","last_became_active":"2024-09-10T23:32:42.565106+0000","last_became_peered":"2024-09-10T23:32:42.565106+0000","last_unstale":"2024-09-10T23:32:43.626948+0000","last_undegraded":"2024-09-10T23:32:43.626948+0000","last_fullsized":"2024-09-10T23:32:43.626948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:20.388860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627066+0000","last_change":"2024-09-10T23:32:42.586253+0000","last_active":"2024-09-10T23:32:43.627066+0000","last_peered":"2024-09-10T23:32:43.627066+0000","last_clean":"2024-09-10T23:32:43.627066+0000","last_became_active":"2024-09-10T23:32:42.585961+0000","last_became_peered":"2024-09-10T23:32:42.585961+0000","last_unstale":"2024-09-10T23:32:43.627066+0000","last_undegraded":"2024-09-10T23:32:43.627066+0000","last_fullsized":"2024-09-10T23:32:43.627066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:19.891408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626891+0000","last_change":"2024-09-10T23:32:42.565746+0000","last_active":"2024-09-10T23:32:43.626891+0000","last_peered":"2024-09-10T23:32:43.626891+0000","last_clean":"2024-09-10T23:32:43.626891+0000","last_became_active":"2024-09-10T23:32:42.565398+0000","last_became_peered":"2024-09-10T23:32:42.565398+0000","last_unstale":"2024-09-10T23:32:43.626891+0000","last_undegraded":"2024-09-10T23:32:43.626891+0000","last_fullsized":"2024-09-10T23:32:43.626891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:28.431860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627187+0000","last_change":"2024-09-10T23:32:42.567960+0000","last_active":"2024-09-10T23:32:43.627187+0000","last_peered":"2024-09-10T23:32:43.627187+0000","last_clean":"2024-09-10T23:32:43.627187+0000","last_became_active":"2024-09-10T23:32:42.567653+0000","last_became_peered":"2024-09-10T23:32:42.567653+0000","last_unstale":"2024-09-10T23:32:43.627187+0000","last_undegraded":"2024-09-10T23:32:43.627187+0000","last_fullsized":"2024-09-10T23:32:43.627187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:26.192704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628554+0000","last_change":"2024-09-10T23:32:42.584639+0000","last_active":"2024-09-10T23:32:43.628554+0000","last_peered":"2024-09-10T23:32:43.628554+0000","last_clean":"2024-09-10T23:32:43.628554+0000","last_became_active":"2024-09-10T23:32:42.584396+0000","last_became_peered":"2024-09-10T23:32:42.584396+0000","last_unstale":"2024-09-10T23:32:43.628554+0000","last_undegraded":"2024-09-10T23:32:43.628554+0000","last_fullsized":"2024-09-10T23:32:43.628554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:27.237846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563757+0000","last_change":"2024-09-10T23:32:42.589096+0000","last_active":"2024-09-10T23:32:43.563757+0000","last_peered":"2024-09-10T23:32:43.563757+0000","last_clean":"2024-09-10T23:32:43.563757+0000","last_became_active":"2024-09-10T23:32:42.588695+0000","last_became_peered":"2024-09-10T23:32:42.588695+0000","last_unstale":"2024-09-10T23:32:43.563757+0000","last_undegraded":"2024-09-10T23:32:43.563757+0000","last_fullsized":"2024-09-10T23:32:43.563757+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:14.778902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627628+0000","last_change":"2024-09-10T23:32:42.572168+0000","last_active":"2024-09-10T23:32:43.627628+0000","last_peered":"2024-09-10T23:32:43.627628+0000","last_clean":"2024-09-10T23:32:43.627628+0000","last_became_active":"2024-09-10T23:32:42.571638+0000","last_became_peered":"2024-09-10T23:32:42.571638+0000","last_unstale":"2024-09-10T23:32:43.627628+0000","last_undegraded":"2024-09-10T23:32:43.627628+0000","last_fullsized":"2024-09-10T23:32:43.627628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:51.184952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627526+0000","last_change":"2024-09-10T23:32:42.580539+0000","last_active":"2024-09-10T23:32:43.627526+0000","last_peered":"2024-09-10T23:32:43.627526+0000","last_clean":"2024-09-10T23:32:43.627526+0000","last_became_active":"2024-09-10T23:32:42.580180+0000","last_became_peered":"2024-09-10T23:32:42.580180+0000","last_unstale":"2024-09-10T23:32:43.627526+0000","last_undegraded":"2024-09-10T23:32:43.627526+0000","last_fullsized":"2024-09-10T23:32:43.627526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:15.490925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627548+0000","last_change":"2024-09-10T23:32:42.568100+0000","last_active":"2024-09-10T23:32:43.627548+0000","last_peered":"2024-09-10T23:32:43.627548+0000","last_clean":"2024-09-10T23:32:43.627548+0000","last_became_active":"2024-09-10T23:32:42.567409+0000","last_became_peered":"2024-09-10T23:32:42.567409+0000","last_unstale":"2024-09-10T23:32:43.627548+0000","last_undegraded":"2024-09-10T23:32:43.627548+0000","last_fullsized":"2024-09-10T23:32:43.627548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.117710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569985+0000","last_change":"2024-09-10T23:32:42.569811+0000","last_active":"2024-09-10T23:32:43.569985+0000","last_peered":"2024-09-10T23:32:43.569985+0000","last_clean":"2024-09-10T23:32:43.569985+0000","last_became_active":"2024-09-10T23:32:42.569470+0000","last_became_peered":"2024-09-10T23:32:42.569470+0000","last_unstale":"2024-09-10T23:32:43.569985+0000","last_undegraded":"2024-09-10T23:32:43.569985+0000","last_fullsized":"2024-09-10T23:32:43.569985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:56.266045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570094+0000","last_change":"2024-09-10T23:32:42.578563+0000","last_active":"2024-09-10T23:32:43.570094+0000","last_peered":"2024-09-10T23:32:43.570094+0000","last_clean":"2024-09-10T23:32:43.570094+0000","last_became_active":"2024-09-10T23:32:42.578281+0000","last_became_peered":"2024-09-10T23:32:42.578281+0000","last_unstale":"2024-09-10T23:32:43.570094+0000","last_undegraded":"2024-09-10T23:32:43.570094+0000","last_fullsized":"2024-09-10T23:32:43.570094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:57.340004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628709+0000","last_change":"2024-09-10T23:32:42.564792+0000","last_active":"2024-09-10T23:32:43.628709+0000","last_peered":"2024-09-10T23:32:43.628709+0000","last_clean":"2024-09-10T23:32:43.628709+0000","last_became_active":"2024-09-10T23:32:42.564029+0000","last_became_peered":"2024-09-10T23:32:42.564029+0000","last_unstale":"2024-09-10T23:32:43.628709+0000","last_undegraded":"2024-09-10T23:32:43.628709+0000","last_fullsized":"2024-09-10T23:32:43.628709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:23.318586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628280+0000","last_change":"2024-09-10T23:32:42.583724+0000","last_active":"2024-09-10T23:32:43.628280+0000","last_peered":"2024-09-10T23:32:43.628280+0000","last_clean":"2024-09-10T23:32:43.628280+0000","last_became_active":"2024-09-10T23:32:42.583473+0000","last_became_peered":"2024-09-10T23:32:42.583473+0000","last_unstale":"2024-09-10T23:32:43.628280+0000","last_undegraded":"2024-09-10T23:32:43.628280+0000","last_fullsized":"2024-09-10T23:32:43.628280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:14.468774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629190+0000","last_change":"2024-09-10T23:32:42.588295+0000","last_active":"2024-09-10T23:32:43.629190+0000","last_peered":"2024-09-10T23:32:43.629190+0000","last_clean":"2024-09-10T23:32:43.629190+0000","last_became_active":"2024-09-10T23:32:42.587993+0000","last_became_peered":"2024-09-10T23:32:42.587993+0000","last_unstale":"2024-09-10T23:32:43.629190+0000","last_undegraded":"2024-09-10T23:32:43.629190+0000","last_fullsized":"2024-09-10T23:32:43.629190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:18:27.504742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629147+0000","last_change":"2024-09-10T23:32:42.579982+0000","last_active":"2024-09-10T23:32:43.629147+0000","last_peered":"2024-09-10T23:32:43.629147+0000","last_clean":"2024-09-10T23:32:43.629147+0000","last_became_active":"2024-09-10T23:32:42.575814+0000","last_became_peered":"2024-09-10T23:32:42.575814+0000","last_unstale":"2024-09-10T23:32:43.629147+0000","last_undegraded":"2024-09-10T23:32:43.629147+0000","last_fullsized":"2024-09-10T23:32:43.629147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:58.672853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629002+0000","last_change":"2024-09-10T23:32:42.585466+0000","last_active":"2024-09-10T23:32:43.629002+0000","last_peered":"2024-09-10T23:32:43.629002+0000","last_clean":"2024-09-10T23:32:43.629002+0000","last_became_active":"2024-09-10T23:32:42.582650+0000","last_became_peered":"2024-09-10T23:32:42.582650+0000","last_unstale":"2024-09-10T23:32:43.629002+0000","last_undegraded":"2024-09-10T23:32:43.629002+0000","last_fullsized":"2024-09-10T23:32:43.629002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:59.434883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627679+0000","last_change":"2024-09-10T23:32:42.573723+0000","last_active":"2024-09-10T23:32:43.627679+0000","last_peered":"2024-09-10T23:32:43.627679+0000","last_clean":"2024-09-10T23:32:43.627679+0000","last_became_active":"2024-09-10T23:32:42.573309+0000","last_became_peered":"2024-09-10T23:32:42.573309+0000","last_unstale":"2024-09-10T23:32:43.627679+0000","last_undegraded":"2024-09-10T23:32:43.627679+0000","last_fullsized":"2024-09-10T23:32:43.627679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:08.762419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569445+0000","last_change":"2024-09-10T23:32:42.558045+0000","last_active":"2024-09-10T23:32:43.569445+0000","last_peered":"2024-09-10T23:32:43.569445+0000","last_clean":"2024-09-10T23:32:43.569445+0000","last_became_active":"2024-09-10T23:32:42.557774+0000","last_became_peered":"2024-09-10T23:32:42.557774+0000","last_unstale":"2024-09-10T23:32:43.569445+0000","last_undegraded":"2024-09-10T23:32:43.569445+0000","last_fullsized":"2024-09-10T23:32:43.569445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:53.622106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563214+0000","last_change":"2024-09-10T23:32:42.556890+0000","last_active":"2024-09-10T23:32:43.563214+0000","last_peered":"2024-09-10T23:32:43.563214+0000","last_clean":"2024-09-10T23:32:43.563214+0000","last_became_active":"2024-09-10T23:32:42.556625+0000","last_became_peered":"2024-09-10T23:32:42.556625+0000","last_unstale":"2024-09-10T23:32:43.563214+0000","last_undegraded":"2024-09-10T23:32:43.563214+0000","last_fullsized":"2024-09-10T23:32:43.563214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:14.440912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628804+0000","last_change":"2024-09-10T23:32:42.575698+0000","last_active":"2024-09-10T23:32:43.628804+0000","last_peered":"2024-09-10T23:32:43.628804+0000","last_clean":"2024-09-10T23:32:43.628804+0000","last_became_active":"2024-09-10T23:32:42.575371+0000","last_became_peered":"2024-09-10T23:32:42.575371+0000","last_unstale":"2024-09-10T23:32:43.628804+0000","last_undegraded":"2024-09-10T23:32:43.628804+0000","last_fullsized":"2024-09-10T23:32:43.628804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:36.598389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628388+0000","last_change":"2024-09-10T23:32:42.591870+0000","last_active":"2024-09-10T23:32:43.628388+0000","last_peered":"2024-09-10T23:32:43.628388+0000","last_clean":"2024-09-10T23:32:43.628388+0000","last_became_active":"2024-09-10T23:32:42.590207+0000","last_became_peered":"2024-09-10T23:32:42.590207+0000","last_unstale":"2024-09-10T23:32:43.628388+0000","last_undegraded":"2024-09-10T23:32:43.628388+0000","last_fullsized":"2024-09-10T23:32:43.628388+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:40.104986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629051+0000","last_change":"2024-09-10T23:32:42.588702+0000","last_active":"2024-09-10T23:32:43.629051+0000","last_peered":"2024-09-10T23:32:43.629051+0000","last_clean":"2024-09-10T23:32:43.629051+0000","last_became_active":"2024-09-10T23:32:42.588444+0000","last_became_peered":"2024-09-10T23:32:42.588444+0000","last_unstale":"2024-09-10T23:32:43.629051+0000","last_undegraded":"2024-09-10T23:32:43.629051+0000","last_fullsized":"2024-09-10T23:32:43.629051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:43.173720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627479+0000","last_change":"2024-09-10T23:32:42.580821+0000","last_active":"2024-09-10T23:32:43.627479+0000","last_peered":"2024-09-10T23:32:43.627479+0000","last_clean":"2024-09-10T23:32:43.627479+0000","last_became_active":"2024-09-10T23:32:42.580319+0000","last_became_peered":"2024-09-10T23:32:42.580319+0000","last_unstale":"2024-09-10T23:32:43.627479+0000","last_undegraded":"2024-09-10T23:32:43.627479+0000","last_fullsized":"2024-09-10T23:32:43.627479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:26.735696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570454+0000","last_change":"2024-09-10T23:32:42.574625+0000","last_active":"2024-09-10T23:32:43.570454+0000","last_peered":"2024-09-10T23:32:43.570454+0000","last_clean":"2024-09-10T23:32:43.570454+0000","last_became_active":"2024-09-10T23:32:42.571847+0000","last_became_peered":"2024-09-10T23:32:42.571847+0000","last_unstale":"2024-09-10T23:32:43.570454+0000","last_undegraded":"2024-09-10T23:32:43.570454+0000","last_fullsized":"2024-09-10T23:32:43.570454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:20.906137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570331+0000","last_change":"2024-09-10T23:32:42.567479+0000","last_active":"2024-09-10T23:32:43.570331+0000","last_peered":"2024-09-10T23:32:43.570331+0000","last_clean":"2024-09-10T23:32:43.570331+0000","last_became_active":"2024-09-10T23:32:42.567172+0000","last_became_peered":"2024-09-10T23:32:42.567172+0000","last_unstale":"2024-09-10T23:32:43.570331+0000","last_undegraded":"2024-09-10T23:32:43.570331+0000","last_fullsized":"2024-09-10T23:32:43.570331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:15.280711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628395+0000","last_change":"2024-09-10T23:32:42.585087+0000","last_active":"2024-09-10T23:32:43.628395+0000","last_peered":"2024-09-10T23:32:43.628395+0000","last_clean":"2024-09-10T23:32:43.628395+0000","last_became_active":"2024-09-10T23:32:42.583712+0000","last_became_peered":"2024-09-10T23:32:42.583712+0000","last_unstale":"2024-09-10T23:32:43.628395+0000","last_undegraded":"2024-09-10T23:32:43.628395+0000","last_fullsized":"2024-09-10T23:32:43.628395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:25.952890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628168+0000","last_change":"2024-09-10T23:32:42.591882+0000","last_active":"2024-09-10T23:32:43.628168+0000","last_peered":"2024-09-10T23:32:43.628168+0000","last_clean":"2024-09-10T23:32:43.628168+0000","last_became_active":"2024-09-10T23:32:42.590231+0000","last_became_peered":"2024-09-10T23:32:42.590231+0000","last_unstale":"2024-09-10T23:32:43.628168+0000","last_undegraded":"2024-09-10T23:32:43.628168+0000","last_fullsized":"2024-09-10T23:32:43.628168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:33.100231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628038+0000","last_change":"2024-09-10T23:32:42.573851+0000","last_active":"2024-09-10T23:32:43.628038+0000","last_peered":"2024-09-10T23:32:43.628038+0000","last_clean":"2024-09-10T23:32:43.628038+0000","last_became_active":"2024-09-10T23:32:42.572836+0000","last_became_peered":"2024-09-10T23:32:42.572836+0000","last_unstale":"2024-09-10T23:32:43.628038+0000","last_undegraded":"2024-09-10T23:32:43.628038+0000","last_fullsized":"2024-09-10T23:32:43.628038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:54.816587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627085+0000","last_change":"2024-09-10T23:32:42.576085+0000","last_active":"2024-09-10T23:32:43.627085+0000","last_peered":"2024-09-10T23:32:43.627085+0000","last_clean":"2024-09-10T23:32:43.627085+0000","last_became_active":"2024-09-10T23:32:42.575374+0000","last_became_peered":"2024-09-10T23:32:42.575374+0000","last_unstale":"2024-09-10T23:32:43.627085+0000","last_undegraded":"2024-09-10T23:32:43.627085+0000","last_fullsized":"2024-09-10T23:32:43.627085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:14.778902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563267+0000","last_change":"2024-09-10T23:32:42.588842+0000","last_active":"2024-09-10T23:32:43.563267+0000","last_peered":"2024-09-10T23:32:43.563267+0000","last_clean":"2024-09-10T23:32:43.563267+0000","last_became_active":"2024-09-10T23:32:42.588582+0000","last_became_peered":"2024-09-10T23:32:42.588582+0000","last_unstale":"2024-09-10T23:32:43.563267+0000","last_undegraded":"2024-09-10T23:32:43.563267+0000","last_fullsized":"2024-09-10T23:32:43.563267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:28.431860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628414+0000","last_change":"2024-09-10T23:32:41.569293+0000","last_active":"2024-09-10T23:32:43.628414+0000","last_peered":"2024-09-10T23:32:43.628414+0000","last_clean":"2024-09-10T23:32:43.628414+0000","last_became_active":"2024-09-10T23:32:41.568848+0000","last_became_peered":"2024-09-10T23:32:41.568848+0000","last_unstale":"2024-09-10T23:32:43.628414+0000","last_undegraded":"2024-09-10T23:32:43.628414+0000","last_fullsized":"2024-09-10T23:32:43.628414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:17.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569497+0000","last_change":"2024-09-10T23:32:42.568214+0000","last_active":"2024-09-10T23:32:43.569497+0000","last_peered":"2024-09-10T23:32:43.569497+0000","last_clean":"2024-09-10T23:32:43.569497+0000","last_became_active":"2024-09-10T23:32:42.562093+0000","last_became_peered":"2024-09-10T23:32:42.562093+0000","last_unstale":"2024-09-10T23:32:43.569497+0000","last_undegraded":"2024-09-10T23:32:43.569497+0000","last_fullsized":"2024-09-10T23:32:43.569497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:02.454545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628550+0000","last_change":"2024-09-10T23:32:41.578635+0000","last_active":"2024-09-10T23:32:43.628550+0000","last_peered":"2024-09-10T23:32:43.628550+0000","last_clean":"2024-09-10T23:32:43.628550+0000","last_became_active":"2024-09-10T23:32:41.578346+0000","last_became_peered":"2024-09-10T23:32:41.578346+0000","last_unstale":"2024-09-10T23:32:43.628550+0000","last_undegraded":"2024-09-10T23:32:43.628550+0000","last_fullsized":"2024-09-10T23:32:43.628550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:25.728667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569793+0000","last_change":"2024-09-10T23:32:41.553674+0000","last_active":"2024-09-10T23:32:43.569793+0000","last_peered":"2024-09-10T23:32:43.569793+0000","last_clean":"2024-09-10T23:32:43.569793+0000","last_became_active":"2024-09-10T23:32:41.553244+0000","last_became_peered":"2024-09-10T23:32:41.553244+0000","last_unstale":"2024-09-10T23:32:43.569793+0000","last_undegraded":"2024-09-10T23:32:43.569793+0000","last_fullsized":"2024-09-10T23:32:43.569793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:38.098241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628828+0000","last_change":"2024-09-10T23:32:42.583587+0000","last_active":"2024-09-10T23:32:43.628828+0000","last_peered":"2024-09-10T23:32:43.628828+0000","last_clean":"2024-09-10T23:32:43.628828+0000","last_became_active":"2024-09-10T23:32:42.581831+0000","last_became_peered":"2024-09-10T23:32:42.581831+0000","last_unstale":"2024-09-10T23:32:43.628828+0000","last_undegraded":"2024-09-10T23:32:43.628828+0000","last_fullsized":"2024-09-10T23:32:43.628828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:23.493444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628532+0000","last_change":"2024-09-10T23:32:41.581810+0000","last_active":"2024-09-10T23:32:43.628532+0000","last_peered":"2024-09-10T23:32:43.628532+0000","last_clean":"2024-09-10T23:32:43.628532+0000","last_became_active":"2024-09-10T23:32:41.581531+0000","last_became_peered":"2024-09-10T23:32:41.581531+0000","last_unstale":"2024-09-10T23:32:43.628532+0000","last_undegraded":"2024-09-10T23:32:43.628532+0000","last_fullsized":"2024-09-10T23:32:43.628532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:57.665825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569188+0000","last_change":"2024-09-10T23:32:42.560663+0000","last_active":"2024-09-10T23:32:43.569188+0000","last_peered":"2024-09-10T23:32:43.569188+0000","last_clean":"2024-09-10T23:32:43.569188+0000","last_became_active":"2024-09-10T23:32:42.560293+0000","last_became_peered":"2024-09-10T23:32:42.560293+0000","last_unstale":"2024-09-10T23:32:43.569188+0000","last_undegraded":"2024-09-10T23:32:43.569188+0000","last_fullsized":"2024-09-10T23:32:43.569188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:39.387246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563757+0000","last_change":"2024-09-10T23:32:41.573958+0000","last_active":"2024-09-10T23:32:43.563757+0000","last_peered":"2024-09-10T23:32:43.563757+0000","last_clean":"2024-09-10T23:32:43.563757+0000","last_became_active":"2024-09-10T23:32:41.573639+0000","last_became_peered":"2024-09-10T23:32:41.573639+0000","last_unstale":"2024-09-10T23:32:43.563757+0000","last_undegraded":"2024-09-10T23:32:43.563757+0000","last_fullsized":"2024-09-10T23:32:43.563757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:27.548086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627506+0000","last_change":"2024-09-10T23:32:42.580586+0000","last_active":"2024-09-10T23:32:43.627506+0000","last_peered":"2024-09-10T23:32:43.627506+0000","last_clean":"2024-09-10T23:32:43.627506+0000","last_became_active":"2024-09-10T23:32:42.579960+0000","last_became_peered":"2024-09-10T23:32:42.579960+0000","last_unstale":"2024-09-10T23:32:43.627506+0000","last_undegraded":"2024-09-10T23:32:43.627506+0000","last_fullsized":"2024-09-10T23:32:43.627506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:09.640521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627910+0000","last_change":"2024-09-10T23:32:41.570662+0000","last_active":"2024-09-10T23:32:43.627910+0000","last_peered":"2024-09-10T23:32:43.627910+0000","last_clean":"2024-09-10T23:32:43.627910+0000","last_became_active":"2024-09-10T23:32:41.570330+0000","last_became_peered":"2024-09-10T23:32:41.570330+0000","last_unstale":"2024-09-10T23:32:43.627910+0000","last_undegraded":"2024-09-10T23:32:43.627910+0000","last_fullsized":"2024-09-10T23:32:43.627910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:53.809559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629328+0000","last_change":"2024-09-10T23:32:42.591662+0000","last_active":"2024-09-10T23:32:43.629328+0000","last_peered":"2024-09-10T23:32:43.629328+0000","last_clean":"2024-09-10T23:32:43.629328+0000","last_became_active":"2024-09-10T23:32:42.591315+0000","last_became_peered":"2024-09-10T23:32:42.591315+0000","last_unstale":"2024-09-10T23:32:43.629328+0000","last_undegraded":"2024-09-10T23:32:43.629328+0000","last_fullsized":"2024-09-10T23:32:43.629328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:34.259468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628947+0000","last_change":"2024-09-10T23:32:41.571940+0000","last_active":"2024-09-10T23:32:43.628947+0000","last_peered":"2024-09-10T23:32:43.628947+0000","last_clean":"2024-09-10T23:32:43.628947+0000","last_became_active":"2024-09-10T23:32:41.571537+0000","last_became_peered":"2024-09-10T23:32:41.571537+0000","last_unstale":"2024-09-10T23:32:43.628947+0000","last_undegraded":"2024-09-10T23:32:43.628947+0000","last_fullsized":"2024-09-10T23:32:43.628947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:49.898494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629065+0000","last_change":"2024-09-10T23:32:42.585332+0000","last_active":"2024-09-10T23:32:43.629065+0000","last_peered":"2024-09-10T23:32:43.629065+0000","last_clean":"2024-09-10T23:32:43.629065+0000","last_became_active":"2024-09-10T23:32:42.585046+0000","last_became_peered":"2024-09-10T23:32:42.585046+0000","last_unstale":"2024-09-10T23:32:43.629065+0000","last_undegraded":"2024-09-10T23:32:43.629065+0000","last_fullsized":"2024-09-10T23:32:43.629065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:24.681728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627051+0000","last_change":"2024-09-10T23:32:41.565270+0000","last_active":"2024-09-10T23:32:43.627051+0000","last_peered":"2024-09-10T23:32:43.627051+0000","last_clean":"2024-09-10T23:32:43.627051+0000","last_became_active":"2024-09-10T23:32:41.564864+0000","last_became_peered":"2024-09-10T23:32:41.564864+0000","last_unstale":"2024-09-10T23:32:43.627051+0000","last_undegraded":"2024-09-10T23:32:43.627051+0000","last_fullsized":"2024-09-10T23:32:43.627051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:55.242033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627514+0000","last_change":"2024-09-10T23:32:42.576998+0000","last_active":"2024-09-10T23:32:43.627514+0000","last_peered":"2024-09-10T23:32:43.627514+0000","last_clean":"2024-09-10T23:32:43.627514+0000","last_became_active":"2024-09-10T23:32:42.576695+0000","last_became_peered":"2024-09-10T23:32:42.576695+0000","last_unstale":"2024-09-10T23:32:43.627514+0000","last_undegraded":"2024-09-10T23:32:43.627514+0000","last_fullsized":"2024-09-10T23:32:43.627514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:09.931603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628218+0000","last_change":"2024-09-10T23:32:41.571048+0000","last_active":"2024-09-10T23:32:43.628218+0000","last_peered":"2024-09-10T23:32:43.628218+0000","last_clean":"2024-09-10T23:32:43.628218+0000","last_became_active":"2024-09-10T23:32:41.570753+0000","last_became_peered":"2024-09-10T23:32:41.570753+0000","last_unstale":"2024-09-10T23:32:43.628218+0000","last_undegraded":"2024-09-10T23:32:43.628218+0000","last_fullsized":"2024-09-10T23:32:43.628218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:55.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570370+0000","last_change":"2024-09-10T23:32:42.564300+0000","last_active":"2024-09-10T23:32:43.570370+0000","last_peered":"2024-09-10T23:32:43.570370+0000","last_clean":"2024-09-10T23:32:43.570370+0000","last_became_active":"2024-09-10T23:32:42.564047+0000","last_became_peered":"2024-09-10T23:32:42.564047+0000","last_unstale":"2024-09-10T23:32:43.570370+0000","last_undegraded":"2024-09-10T23:32:43.570370+0000","last_fullsized":"2024-09-10T23:32:43.570370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:22.724821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628645+0000","last_change":"2024-09-10T23:32:41.585012+0000","last_active":"2024-09-10T23:32:43.628645+0000","last_peered":"2024-09-10T23:32:43.628645+0000","last_clean":"2024-09-10T23:32:43.628645+0000","last_became_active":"2024-09-10T23:32:41.584732+0000","last_became_peered":"2024-09-10T23:32:41.584732+0000","last_unstale":"2024-09-10T23:32:43.628645+0000","last_undegraded":"2024-09-10T23:32:43.628645+0000","last_fullsized":"2024-09-10T23:32:43.628645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:39.041511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628794+0000","last_change":"2024-09-10T23:32:42.569000+0000","last_active":"2024-09-10T23:32:43.628794+0000","last_peered":"2024-09-10T23:32:43.628794+0000","last_clean":"2024-09-10T23:32:43.628794+0000","last_became_active":"2024-09-10T23:32:42.568721+0000","last_became_peered":"2024-09-10T23:32:42.568721+0000","last_unstale":"2024-09-10T23:32:43.628794+0000","last_undegraded":"2024-09-10T23:32:43.628794+0000","last_fullsized":"2024-09-10T23:32:43.628794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:37.891368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626833+0000","last_change":"2024-09-10T23:32:41.578661+0000","last_active":"2024-09-10T23:32:43.626833+0000","last_peered":"2024-09-10T23:32:43.626833+0000","last_clean":"2024-09-10T23:32:43.626833+0000","last_became_active":"2024-09-10T23:32:41.578343+0000","last_became_peered":"2024-09-10T23:32:41.578343+0000","last_unstale":"2024-09-10T23:32:43.626833+0000","last_undegraded":"2024-09-10T23:32:43.626833+0000","last_fullsized":"2024-09-10T23:32:43.626833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:47.815432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563134+0000","last_change":"2024-09-10T23:32:42.584934+0000","last_active":"2024-09-10T23:32:43.563134+0000","last_peered":"2024-09-10T23:32:43.563134+0000","last_clean":"2024-09-10T23:32:43.563134+0000","last_became_active":"2024-09-10T23:32:42.584709+0000","last_became_peered":"2024-09-10T23:32:42.584709+0000","last_unstale":"2024-09-10T23:32:43.563134+0000","last_undegraded":"2024-09-10T23:32:43.563134+0000","last_fullsized":"2024-09-10T23:32:43.563134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:30.543243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.568880+0000","last_change":"2024-09-10T23:32:41.566144+0000","last_active":"2024-09-10T23:32:43.568880+0000","last_peered":"2024-09-10T23:32:43.568880+0000","last_clean":"2024-09-10T23:32:43.568880+0000","last_became_active":"2024-09-10T23:32:41.565841+0000","last_became_peered":"2024-09-10T23:32:41.565841+0000","last_unstale":"2024-09-10T23:32:43.568880+0000","last_undegraded":"2024-09-10T23:32:43.568880+0000","last_fullsized":"2024-09-10T23:32:43.568880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:51.080022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628125+0000","last_change":"2024-09-10T23:32:42.586470+0000","last_active":"2024-09-10T23:32:43.628125+0000","last_peered":"2024-09-10T23:32:43.628125+0000","last_clean":"2024-09-10T23:32:43.628125+0000","last_became_active":"2024-09-10T23:32:42.579377+0000","last_became_peered":"2024-09-10T23:32:42.579377+0000","last_unstale":"2024-09-10T23:32:43.628125+0000","last_undegraded":"2024-09-10T23:32:43.628125+0000","last_fullsized":"2024-09-10T23:32:43.628125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.112551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628061+0000","last_change":"2024-09-10T23:32:41.567212+0000","last_active":"2024-09-10T23:32:43.628061+0000","last_peered":"2024-09-10T23:32:43.628061+0000","last_clean":"2024-09-10T23:32:43.628061+0000","last_became_active":"2024-09-10T23:32:41.566857+0000","last_became_peered":"2024-09-10T23:32:41.566857+0000","last_unstale":"2024-09-10T23:32:43.628061+0000","last_undegraded":"2024-09-10T23:32:43.628061+0000","last_fullsized":"2024-09-10T23:32:43.628061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:05.259433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627653+0000","last_change":"2024-09-10T23:32:42.558295+0000","last_active":"2024-09-10T23:32:43.627653+0000","last_peered":"2024-09-10T23:32:43.627653+0000","last_clean":"2024-09-10T23:32:43.627653+0000","last_became_active":"2024-09-10T23:32:42.558039+0000","last_became_peered":"2024-09-10T23:32:42.558039+0000","last_unstale":"2024-09-10T23:32:43.627653+0000","last_undegraded":"2024-09-10T23:32:43.627653+0000","last_fullsized":"2024-09-10T23:32:43.627653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:07.033834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569932+0000","last_change":"2024-09-10T23:32:41.568016+0000","last_active":"2024-09-10T23:32:43.569932+0000","last_peered":"2024-09-10T23:32:43.569932+0000","last_clean":"2024-09-10T23:32:43.569932+0000","last_became_active":"2024-09-10T23:32:41.566960+0000","last_became_peered":"2024-09-10T23:32:41.566960+0000","last_unstale":"2024-09-10T23:32:43.569932+0000","last_undegraded":"2024-09-10T23:32:43.569932+0000","last_fullsized":"2024-09-10T23:32:43.569932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:14.145496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628571+0000","last_change":"2024-09-10T23:32:42.562392+0000","last_active":"2024-09-10T23:32:43.628571+0000","last_peered":"2024-09-10T23:32:43.628571+0000","last_clean":"2024-09-10T23:32:43.628571+0000","last_became_active":"2024-09-10T23:32:42.562042+0000","last_became_peered":"2024-09-10T23:32:42.562042+0000","last_unstale":"2024-09-10T23:32:43.628571+0000","last_undegraded":"2024-09-10T23:32:43.628571+0000","last_fullsized":"2024-09-10T23:32:43.628571+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:33.100231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628521+0000","last_change":"2024-09-10T23:32:41.572986+0000","last_active":"2024-09-10T23:32:43.628521+0000","last_peered":"2024-09-10T23:32:43.628521+0000","last_clean":"2024-09-10T23:32:43.628521+0000","last_became_active":"2024-09-10T23:32:41.572722+0000","last_became_peered":"2024-09-10T23:32:41.572722+0000","last_unstale":"2024-09-10T23:32:43.628521+0000","last_undegraded":"2024-09-10T23:32:43.628521+0000","last_fullsized":"2024-09-10T23:32:43.628521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:49.898494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628742+0000","last_change":"2024-09-10T23:32:42.573643+0000","last_active":"2024-09-10T23:32:43.628742+0000","last_peered":"2024-09-10T23:32:43.628742+0000","last_clean":"2024-09-10T23:32:43.628742+0000","last_became_active":"2024-09-10T23:32:42.572873+0000","last_became_peered":"2024-09-10T23:32:42.572873+0000","last_unstale":"2024-09-10T23:32:43.628742+0000","last_undegraded":"2024-09-10T23:32:43.628742+0000","last_fullsized":"2024-09-10T23:32:43.628742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:14.468774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570459+0000","last_change":"2024-09-10T23:32:41.562365+0000","last_active":"2024-09-10T23:32:43.570459+0000","last_peered":"2024-09-10T23:32:43.570459+0000","last_clean":"2024-09-10T23:32:43.570459+0000","last_became_active":"2024-09-10T23:32:41.562021+0000","last_became_peered":"2024-09-10T23:32:41.562021+0000","last_unstale":"2024-09-10T23:32:43.570459+0000","last_undegraded":"2024-09-10T23:32:43.570459+0000","last_fullsized":"2024-09-10T23:32:43.570459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:08:28.878205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628732+0000","last_change":"2024-09-10T23:32:42.584713+0000","last_active":"2024-09-10T23:32:43.628732+0000","last_peered":"2024-09-10T23:32:43.628732+0000","last_clean":"2024-09-10T23:32:43.628732+0000","last_became_active":"2024-09-10T23:32:42.577789+0000","last_became_peered":"2024-09-10T23:32:42.577789+0000","last_unstale":"2024-09-10T23:32:43.628732+0000","last_undegraded":"2024-09-10T23:32:43.628732+0000","last_fullsized":"2024-09-10T23:32:43.628732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:33.835681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563126+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:32:43.563126+0000","last_peered":"2024-09-10T23:32:43.563126+0000","last_clean":"2024-09-10T23:32:43.563126+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:32:43.563126+0000","last_undegraded":"2024-09-10T23:32:43.563126+0000","last_fullsized":"2024-09-10T23:32:43.563126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629078+0000","last_change":"2024-09-10T23:32:41.581530+0000","last_active":"2024-09-10T23:32:43.629078+0000","last_peered":"2024-09-10T23:32:43.629078+0000","last_clean":"2024-09-10T23:32:43.629078+0000","last_became_active":"2024-09-10T23:32:41.581242+0000","last_became_peered":"2024-09-10T23:32:41.581242+0000","last_unstale":"2024-09-10T23:32:43.629078+0000","last_undegraded":"2024-09-10T23:32:43.629078+0000","last_fullsized":"2024-09-10T23:32:43.629078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:08.633493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569220+0000","last_change":"2024-09-10T23:32:42.558841+0000","last_active":"2024-09-10T23:32:43.569220+0000","last_peered":"2024-09-10T23:32:43.569220+0000","last_clean":"2024-09-10T23:32:43.569220+0000","last_became_active":"2024-09-10T23:32:42.558580+0000","last_became_peered":"2024-09-10T23:32:42.558580+0000","last_unstale":"2024-09-10T23:32:43.569220+0000","last_undegraded":"2024-09-10T23:32:43.569220+0000","last_fullsized":"2024-09-10T23:32:43.569220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:54.620648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628625+0000","last_change":"2024-09-10T23:32:41.577467+0000","last_active":"2024-09-10T23:32:43.628625+0000","last_peered":"2024-09-10T23:32:43.628625+0000","last_clean":"2024-09-10T23:32:43.628625+0000","last_became_active":"2024-09-10T23:32:41.576702+0000","last_became_peered":"2024-09-10T23:32:41.576702+0000","last_unstale":"2024-09-10T23:32:43.628625+0000","last_undegraded":"2024-09-10T23:32:43.628625+0000","last_fullsized":"2024-09-10T23:32:43.628625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:32.527218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569670+0000","last_change":"2024-09-10T23:32:42.565100+0000","last_active":"2024-09-10T23:32:43.569670+0000","last_peered":"2024-09-10T23:32:43.569670+0000","last_clean":"2024-09-10T23:32:43.569670+0000","last_became_active":"2024-09-10T23:32:42.564790+0000","last_became_peered":"2024-09-10T23:32:42.564790+0000","last_unstale":"2024-09-10T23:32:43.569670+0000","last_undegraded":"2024-09-10T23:32:43.569670+0000","last_fullsized":"2024-09-10T23:32:43.569670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:44.555042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628065+0000","last_change":"2024-09-10T23:32:41.571266+0000","last_active":"2024-09-10T23:32:43.628065+0000","last_peered":"2024-09-10T23:32:43.628065+0000","last_clean":"2024-09-10T23:32:43.628065+0000","last_became_active":"2024-09-10T23:32:41.570986+0000","last_became_peered":"2024-09-10T23:32:41.570986+0000","last_unstale":"2024-09-10T23:32:43.628065+0000","last_undegraded":"2024-09-10T23:32:43.628065+0000","last_fullsized":"2024-09-10T23:32:43.628065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:58.170892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628975+0000","last_change":"2024-09-10T23:32:42.584295+0000","last_active":"2024-09-10T23:32:43.628975+0000","last_peered":"2024-09-10T23:32:43.628975+0000","last_clean":"2024-09-10T23:32:43.628975+0000","last_became_active":"2024-09-10T23:32:42.584001+0000","last_became_peered":"2024-09-10T23:32:42.584001+0000","last_unstale":"2024-09-10T23:32:43.628975+0000","last_undegraded":"2024-09-10T23:32:43.628975+0000","last_fullsized":"2024-09-10T23:32:43.628975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:05.044760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628726+0000","last_change":"2024-09-10T23:32:41.578157+0000","last_active":"2024-09-10T23:32:43.628726+0000","last_peered":"2024-09-10T23:32:43.628726+0000","last_clean":"2024-09-10T23:32:43.628726+0000","last_became_active":"2024-09-10T23:32:41.575442+0000","last_became_peered":"2024-09-10T23:32:41.575442+0000","last_unstale":"2024-09-10T23:32:43.628726+0000","last_undegraded":"2024-09-10T23:32:43.628726+0000","last_fullsized":"2024-09-10T23:32:43.628726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:00.455999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627197+0000","last_change":"2024-09-10T23:32:42.571239+0000","last_active":"2024-09-10T23:32:43.627197+0000","last_peered":"2024-09-10T23:32:43.627197+0000","last_clean":"2024-09-10T23:32:43.627197+0000","last_became_active":"2024-09-10T23:32:42.570963+0000","last_became_peered":"2024-09-10T23:32:42.570963+0000","last_unstale":"2024-09-10T23:32:43.627197+0000","last_undegraded":"2024-09-10T23:32:43.627197+0000","last_fullsized":"2024-09-10T23:32:43.627197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:36.598389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628178+0000","last_change":"2024-09-10T23:32:41.569326+0000","last_active":"2024-09-10T23:32:43.628178+0000","last_peered":"2024-09-10T23:32:43.628178+0000","last_clean":"2024-09-10T23:32:43.628178+0000","last_became_active":"2024-09-10T23:32:41.569056+0000","last_became_peered":"2024-09-10T23:32:41.569056+0000","last_unstale":"2024-09-10T23:32:43.628178+0000","last_undegraded":"2024-09-10T23:32:43.628178+0000","last_fullsized":"2024-09-10T23:32:43.628178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:47.815432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626872+0000","last_change":"2024-09-10T23:32:42.574800+0000","last_active":"2024-09-10T23:32:43.626872+0000","last_peered":"2024-09-10T23:32:43.626872+0000","last_clean":"2024-09-10T23:32:43.626872+0000","last_became_active":"2024-09-10T23:32:42.574477+0000","last_became_peered":"2024-09-10T23:32:42.574477+0000","last_unstale":"2024-09-10T23:32:43.626872+0000","last_undegraded":"2024-09-10T23:32:43.626872+0000","last_fullsized":"2024-09-10T23:32:43.626872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:14.440912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629156+0000","last_change":"2024-09-10T23:32:41.582865+0000","last_active":"2024-09-10T23:32:43.629156+0000","last_peered":"2024-09-10T23:32:43.629156+0000","last_clean":"2024-09-10T23:32:43.629156+0000","last_became_active":"2024-09-10T23:32:41.582522+0000","last_became_peered":"2024-09-10T23:32:41.582522+0000","last_unstale":"2024-09-10T23:32:43.629156+0000","last_undegraded":"2024-09-10T23:32:43.629156+0000","last_fullsized":"2024-09-10T23:32:43.629156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:24.945862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570471+0000","last_change":"2024-09-10T23:32:42.587148+0000","last_active":"2024-09-10T23:32:43.570471+0000","last_peered":"2024-09-10T23:32:43.570471+0000","last_clean":"2024-09-10T23:32:43.570471+0000","last_became_active":"2024-09-10T23:32:42.586887+0000","last_became_peered":"2024-09-10T23:32:42.586887+0000","last_unstale":"2024-09-10T23:32:43.570471+0000","last_undegraded":"2024-09-10T23:32:43.570471+0000","last_fullsized":"2024-09-10T23:32:43.570471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:27.999337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570022+0000","last_change":"2024-09-10T23:32:41.553400+0000","last_active":"2024-09-10T23:32:43.570022+0000","last_peered":"2024-09-10T23:32:43.570022+0000","last_clean":"2024-09-10T23:32:43.570022+0000","last_became_active":"2024-09-10T23:32:41.553070+0000","last_became_peered":"2024-09-10T23:32:41.553070+0000","last_unstale":"2024-09-10T23:32:43.570022+0000","last_undegraded":"2024-09-10T23:32:43.570022+0000","last_fullsized":"2024-09-10T23:32:43.570022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:01.393799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569840+0000","last_change":"2024-09-10T23:32:42.580714+0000","last_active":"2024-09-10T23:32:43.569840+0000","last_peered":"2024-09-10T23:32:43.569840+0000","last_clean":"2024-09-10T23:32:43.569840+0000","last_became_active":"2024-09-10T23:32:42.579027+0000","last_became_peered":"2024-09-10T23:32:42.579027+0000","last_unstale":"2024-09-10T23:32:43.569840+0000","last_undegraded":"2024-09-10T23:32:43.569840+0000","last_fullsized":"2024-09-10T23:32:43.569840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:12.456589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629216+0000","last_change":"2024-09-10T23:32:41.580054+0000","last_active":"2024-09-10T23:32:43.629216+0000","last_peered":"2024-09-10T23:32:43.629216+0000","last_clean":"2024-09-10T23:32:43.629216+0000","last_became_active":"2024-09-10T23:32:41.579663+0000","last_became_peered":"2024-09-10T23:32:41.579663+0000","last_unstale":"2024-09-10T23:32:43.629216+0000","last_undegraded":"2024-09-10T23:32:43.629216+0000","last_fullsized":"2024-09-10T23:32:43.629216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:22.486416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569890+0000","last_change":"2024-09-10T23:32:42.574503+0000","last_active":"2024-09-10T23:32:43.569890+0000","last_peered":"2024-09-10T23:32:43.569890+0000","last_clean":"2024-09-10T23:32:43.569890+0000","last_became_active":"2024-09-10T23:32:42.574229+0000","last_became_peered":"2024-09-10T23:32:42.574229+0000","last_unstale":"2024-09-10T23:32:43.569890+0000","last_undegraded":"2024-09-10T23:32:43.569890+0000","last_fullsized":"2024-09-10T23:32:43.569890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:19.891408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626574+0000","last_change":"2024-09-10T23:32:41.562000+0000","last_active":"2024-09-10T23:32:43.626574+0000","last_peered":"2024-09-10T23:32:43.626574+0000","last_clean":"2024-09-10T23:32:43.626574+0000","last_became_active":"2024-09-10T23:32:41.561637+0000","last_became_peered":"2024-09-10T23:32:41.561637+0000","last_unstale":"2024-09-10T23:32:43.626574+0000","last_undegraded":"2024-09-10T23:32:43.626574+0000","last_fullsized":"2024-09-10T23:32:43.626574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:17.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628329+0000","last_change":"2024-09-10T23:32:42.583205+0000","last_active":"2024-09-10T23:32:43.628329+0000","last_peered":"2024-09-10T23:32:43.628329+0000","last_clean":"2024-09-10T23:32:43.628329+0000","last_became_active":"2024-09-10T23:32:42.582963+0000","last_became_peered":"2024-09-10T23:32:42.582963+0000","last_unstale":"2024-09-10T23:32:43.628329+0000","last_undegraded":"2024-09-10T23:32:43.628329+0000","last_fullsized":"2024-09-10T23:32:43.628329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.117710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569104+0000","last_change":"2024-09-10T23:32:41.557934+0000","last_active":"2024-09-10T23:32:43.569104+0000","last_peered":"2024-09-10T23:32:43.569104+0000","last_clean":"2024-09-10T23:32:43.569104+0000","last_became_active":"2024-09-10T23:32:41.557625+0000","last_became_peered":"2024-09-10T23:32:41.557625+0000","last_unstale":"2024-09-10T23:32:43.569104+0000","last_undegraded":"2024-09-10T23:32:43.569104+0000","last_fullsized":"2024-09-10T23:32:43.569104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:29.536215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569080+0000","last_change":"2024-09-10T23:32:42.567250+0000","last_active":"2024-09-10T23:32:43.569080+0000","last_peered":"2024-09-10T23:32:43.569080+0000","last_clean":"2024-09-10T23:32:43.569080+0000","last_became_active":"2024-09-10T23:32:42.566248+0000","last_became_peered":"2024-09-10T23:32:42.566248+0000","last_unstale":"2024-09-10T23:32:43.569080+0000","last_undegraded":"2024-09-10T23:32:43.569080+0000","last_fullsized":"2024-09-10T23:32:43.569080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:45.824956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570495+0000","last_change":"2024-09-10T23:32:41.566640+0000","last_active":"2024-09-10T23:32:43.570495+0000","last_peered":"2024-09-10T23:32:43.570495+0000","last_clean":"2024-09-10T23:32:43.570495+0000","last_became_active":"2024-09-10T23:32:41.566311+0000","last_became_peered":"2024-09-10T23:32:41.566311+0000","last_unstale":"2024-09-10T23:32:43.570495+0000","last_undegraded":"2024-09-10T23:32:43.570495+0000","last_fullsized":"2024-09-10T23:32:43.570495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:33.372580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628173+0000","last_change":"2024-09-10T23:32:42.580760+0000","last_active":"2024-09-10T23:32:43.628173+0000","last_peered":"2024-09-10T23:32:43.628173+0000","last_clean":"2024-09-10T23:32:43.628173+0000","last_became_active":"2024-09-10T23:32:42.580081+0000","last_became_peered":"2024-09-10T23:32:42.580081+0000","last_unstale":"2024-09-10T23:32:43.628173+0000","last_undegraded":"2024-09-10T23:32:43.628173+0000","last_fullsized":"2024-09-10T23:32:43.628173+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:06.266461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628952+0000","last_change":"2024-09-10T23:32:41.583976+0000","last_active":"2024-09-10T23:32:43.628952+0000","last_peered":"2024-09-10T23:32:43.628952+0000","last_clean":"2024-09-10T23:32:43.628952+0000","last_became_active":"2024-09-10T23:32:41.583639+0000","last_became_peered":"2024-09-10T23:32:41.583639+0000","last_unstale":"2024-09-10T23:32:43.628952+0000","last_undegraded":"2024-09-10T23:32:43.628952+0000","last_fullsized":"2024-09-10T23:32:43.628952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:26.669949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629220+0000","last_change":"2024-09-10T23:32:42.570056+0000","last_active":"2024-09-10T23:32:43.629220+0000","last_peered":"2024-09-10T23:32:43.629220+0000","last_clean":"2024-09-10T23:32:43.629220+0000","last_became_active":"2024-09-10T23:32:42.569794+0000","last_became_peered":"2024-09-10T23:32:42.569794+0000","last_unstale":"2024-09-10T23:32:43.629220+0000","last_undegraded":"2024-09-10T23:32:43.629220+0000","last_fullsized":"2024-09-10T23:32:43.629220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:36.453126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628526+0000","last_change":"2024-09-10T23:32:42.585151+0000","last_active":"2024-09-10T23:32:43.628526+0000","last_peered":"2024-09-10T23:32:43.628526+0000","last_clean":"2024-09-10T23:32:43.628526+0000","last_became_active":"2024-09-10T23:32:42.578092+0000","last_became_peered":"2024-09-10T23:32:42.578092+0000","last_unstale":"2024-09-10T23:32:43.628526+0000","last_undegraded":"2024-09-10T23:32:43.628526+0000","last_fullsized":"2024-09-10T23:32:43.628526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:39:56.735321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628891+0000","last_change":"2024-09-10T23:32:41.571659+0000","last_active":"2024-09-10T23:32:43.628891+0000","last_peered":"2024-09-10T23:32:43.628891+0000","last_clean":"2024-09-10T23:32:43.628891+0000","last_became_active":"2024-09-10T23:32:41.571418+0000","last_became_peered":"2024-09-10T23:32:41.571418+0000","last_unstale":"2024-09-10T23:32:43.628891+0000","last_undegraded":"2024-09-10T23:32:43.628891+0000","last_fullsized":"2024-09-10T23:32:43.628891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:57.665825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563423+0000","last_change":"2024-09-10T23:32:42.563064+0000","last_active":"2024-09-10T23:32:43.563423+0000","last_peered":"2024-09-10T23:32:43.563423+0000","last_clean":"2024-09-10T23:32:43.563423+0000","last_became_active":"2024-09-10T23:32:42.562808+0000","last_became_peered":"2024-09-10T23:32:42.562808+0000","last_unstale":"2024-09-10T23:32:43.563423+0000","last_undegraded":"2024-09-10T23:32:43.563423+0000","last_fullsized":"2024-09-10T23:32:43.563423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:20.388860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627661+0000","last_change":"2024-09-10T23:32:41.570670+0000","last_active":"2024-09-10T23:32:43.627661+0000","last_peered":"2024-09-10T23:32:43.627661+0000","last_clean":"2024-09-10T23:32:43.627661+0000","last_became_active":"2024-09-10T23:32:41.570291+0000","last_became_peered":"2024-09-10T23:32:41.570291+0000","last_unstale":"2024-09-10T23:32:43.627661+0000","last_undegraded":"2024-09-10T23:32:43.627661+0000","last_fullsized":"2024-09-10T23:32:43.627661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:33.183319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629201+0000","last_change":"2024-09-10T23:32:42.584736+0000","last_active":"2024-09-10T23:32:43.629201+0000","last_peered":"2024-09-10T23:32:43.629201+0000","last_clean":"2024-09-10T23:32:43.629201+0000","last_became_active":"2024-09-10T23:32:42.583535+0000","last_became_peered":"2024-09-10T23:32:42.583535+0000","last_unstale":"2024-09-10T23:32:43.629201+0000","last_undegraded":"2024-09-10T23:32:43.629201+0000","last_fullsized":"2024-09-10T23:32:43.629201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:00.689816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629197+0000","last_change":"2024-09-10T23:32:41.566445+0000","last_active":"2024-09-10T23:32:43.629197+0000","last_peered":"2024-09-10T23:32:43.629197+0000","last_clean":"2024-09-10T23:32:43.629197+0000","last_became_active":"2024-09-10T23:32:41.566158+0000","last_became_peered":"2024-09-10T23:32:41.566158+0000","last_unstale":"2024-09-10T23:32:43.629197+0000","last_undegraded":"2024-09-10T23:32:43.629197+0000","last_fullsized":"2024-09-10T23:32:43.629197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:55.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629299+0000","last_change":"2024-09-10T23:32:42.584147+0000","last_active":"2024-09-10T23:32:43.629299+0000","last_peered":"2024-09-10T23:32:43.629299+0000","last_clean":"2024-09-10T23:32:43.629299+0000","last_became_active":"2024-09-10T23:32:42.583766+0000","last_became_peered":"2024-09-10T23:32:42.583766+0000","last_unstale":"2024-09-10T23:32:43.629299+0000","last_undegraded":"2024-09-10T23:32:43.629299+0000","last_fullsized":"2024-09-10T23:32:43.629299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:19.288688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569913+0000","last_change":"2024-09-10T23:32:41.580965+0000","last_active":"2024-09-10T23:32:43.569913+0000","last_peered":"2024-09-10T23:32:43.569913+0000","last_clean":"2024-09-10T23:32:43.569913+0000","last_became_active":"2024-09-10T23:32:41.580712+0000","last_became_peered":"2024-09-10T23:32:41.580712+0000","last_unstale":"2024-09-10T23:32:43.569913+0000","last_undegraded":"2024-09-10T23:32:43.569913+0000","last_fullsized":"2024-09-10T23:32:43.569913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:13.433884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:32:47.442 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T23:32:47.531 INFO:tasks.ceph.mds.b.smithi181.stderr:2024-09-10T23:32:47.527+0000 7f50d7200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:32:47.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd542200640 1 -- 172.21.15.94:0/2413199585 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd53c0cddb0 msgr2=0x7fd53c0ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:47.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd542200640 1 --2- 172.21.15.94:0/2413199585 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd53c0cddb0 0x7fd53c0ce220 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd534009bc0 tx=0x7fd53401cb70 comp rx=0 tx=0).stop 2024-09-10T23:32:47.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd542200640 1 -- 172.21.15.94:0/2413199585 shutdown_connections 2024-09-10T23:32:47.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd542200640 1 --2- 172.21.15.94:0/2413199585 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd53c0cddb0 0x7fd53c0ce220 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd542200640 1 --2- 172.21.15.94:0/2413199585 >> v2:172.21.15.94:3300/0 conn(0x7fd53c0cfcb0 0x7fd53c0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd542200640 1 -- 172.21.15.94:0/2413199585 >> 172.21.15.94:0/2413199585 conn(0x7fd53c06cdf0 msgr2=0x7fd53c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:47.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd542200640 1 -- 172.21.15.94:0/2413199585 shutdown_connections 2024-09-10T23:32:47.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd542200640 1 -- 172.21.15.94:0/2413199585 wait complete. 2024-09-10T23:32:47.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd542200640 1 Processor -- start 2024-09-10T23:32:47.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd542200640 1 -- start start 2024-09-10T23:32:47.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd542200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd53c0cddb0 0x7fd53c142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:47.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd542200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd53c0cfcb0 0x7fd53c141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:47.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd542200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd53c15ba50 0x7fd53c147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:47.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd542200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c168be0 con 0x7fd53c15ba50 2024-09-10T23:32:47.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd542200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c168a60 con 0x7fd53c0cddb0 2024-09-10T23:32:47.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd542200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd53c168d60 con 0x7fd53c0cfcb0 2024-09-10T23:32:47.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd53be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd53c15ba50 0x7fd53c147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:47.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd53be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd53c15ba50 0x7fd53c147b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50898/0 (socket says 172.21.15.94:50898) 2024-09-10T23:32:47.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd53be00640 1 -- 172.21.15.94:0/1456057291 learned_addr learned my addr 172.21.15.94:0/1456057291 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:47.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd532a00640 1 --2- 172.21.15.94:0/1456057291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd53c0cfcb0 0x7fd53c141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:47.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd53b400640 1 --2- 172.21.15.94:0/1456057291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd53c0cddb0 0x7fd53c142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:47.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd53be00640 1 -- 172.21.15.94:0/1456057291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd53c0cfcb0 msgr2=0x7fd53c141e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:47.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd53be00640 1 --2- 172.21.15.94:0/1456057291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd53c0cfcb0 0x7fd53c141e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd53be00640 1 -- 172.21.15.94:0/1456057291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd53c0cddb0 msgr2=0x7fd53c142480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:47.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd53be00640 1 --2- 172.21.15.94:0/1456057291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd53c0cddb0 0x7fd53c142480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.625+0000 7fd53be00640 1 -- 172.21.15.94:0/1456057291 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53c163ea0 con 0x7fd53c15ba50 2024-09-10T23:32:47.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.629+0000 7fd53be00640 1 --2- 172.21.15.94:0/1456057291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd53c15ba50 0x7fd53c147b00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5340b0040 tx=0x7fd534005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:47.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.629+0000 7fd533e00640 1 -- 172.21.15.94:0/1456057291 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5340b2040 con 0x7fd53c15ba50 2024-09-10T23:32:47.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.629+0000 7fd533e00640 1 -- 172.21.15.94:0/1456057291 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd534004040 con 0x7fd53c15ba50 2024-09-10T23:32:47.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.629+0000 7fd533e00640 1 -- 172.21.15.94:0/1456057291 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534004470 con 0x7fd53c15ba50 2024-09-10T23:32:47.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.629+0000 7fd542200640 1 -- 172.21.15.94:0/1456057291 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53c164180 con 0x7fd53c15ba50 2024-09-10T23:32:47.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.629+0000 7fd542200640 1 -- 172.21.15.94:0/1456057291 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd53c165e50 con 0x7fd53c15ba50 2024-09-10T23:32:47.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.629+0000 7fd542200640 1 -- 172.21.15.94:0/1456057291 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd53c1659a0 con 0x7fd53c15ba50 2024-09-10T23:32:47.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.629+0000 7fd533e00640 1 -- 172.21.15.94:0/1456057291 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd5340b5070 con 0x7fd53c15ba50 2024-09-10T23:32:47.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.629+0000 7fd533e00640 1 --2- 172.21.15.94:0/1456057291 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd5180725e0 0x7fd518074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:47.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.629+0000 7fd533e00640 1 -- 172.21.15.94:0/1456057291 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd53412cf70 con 0x7fd53c15ba50 2024-09-10T23:32:47.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.629+0000 7fd53b400640 1 --2- 172.21.15.94:0/1456057291 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd5180725e0 0x7fd518074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:47.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.633+0000 7fd53b400640 1 --2- 172.21.15.94:0/1456057291 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd5180725e0 0x7fd518074aa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd52c001640 tx=0x7fd52c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:47.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.633+0000 7fd533e00640 1 -- 172.21.15.94:0/1456057291 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd534016790 con 0x7fd53c15ba50 2024-09-10T23:32:47.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.857+0000 7fd542200640 1 -- 172.21.15.94:0/1456057291 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd53c0ce220 con 0x7fd53c15ba50 2024-09-10T23:32:47.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.857+0000 7fd533e00640 1 -- 172.21.15.94:0/1456057291 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1055 (secure 0 0 0) 0x7fd5340f85c0 con 0x7fd53c15ba50 2024-09-10T23:32:47.861 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:47.861 INFO:teuthology.orchestra.run.smithi094.stdout:{"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":30,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":22,"num_osds":8,"num_up_osds":8,"osd_up_since":1726011151,"num_in_osds":8,"osd_in_since":1726011140,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":241508352,"bytes_avail":772852604928,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":8,"btime":"2024-09-10T23:32:45:573319+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":194174}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-10T23:32:16.096949+0000","services":{}},"progress_events":{}} 2024-09-10T23:32:47.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.861+0000 7fd542200640 1 -- 172.21.15.94:0/1456057291 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd5180725e0 msgr2=0x7fd518074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:47.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.861+0000 7fd542200640 1 --2- 172.21.15.94:0/1456057291 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd5180725e0 0x7fd518074aa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd52c001640 tx=0x7fd52c009040 comp rx=0 tx=0).stop 2024-09-10T23:32:47.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.861+0000 7fd542200640 1 -- 172.21.15.94:0/1456057291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd53c15ba50 msgr2=0x7fd53c147b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:47.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.861+0000 7fd542200640 1 --2- 172.21.15.94:0/1456057291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd53c15ba50 0x7fd53c147b00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5340b0040 tx=0x7fd534005a80 comp rx=0 tx=0).stop 2024-09-10T23:32:47.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.861+0000 7fd542200640 1 -- 172.21.15.94:0/1456057291 shutdown_connections 2024-09-10T23:32:47.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.861+0000 7fd542200640 1 --2- 172.21.15.94:0/1456057291 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd5180725e0 0x7fd518074aa0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.861+0000 7fd542200640 1 --2- 172.21.15.94:0/1456057291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd53c15ba50 0x7fd53c147b00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.861+0000 7fd542200640 1 --2- 172.21.15.94:0/1456057291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd53c0cfcb0 0x7fd53c141e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.861+0000 7fd542200640 1 --2- 172.21.15.94:0/1456057291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd53c0cddb0 0x7fd53c142480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:47.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.861+0000 7fd542200640 1 -- 172.21.15.94:0/1456057291 >> 172.21.15.94:0/1456057291 conn(0x7fd53c06cdf0 msgr2=0x7fd53c072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:47.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.861+0000 7fd542200640 1 -- 172.21.15.94:0/1456057291 shutdown_connections 2024-09-10T23:32:47.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:47.861+0000 7fd542200640 1 -- 172.21.15.94:0/1456057291 wait complete. 2024-09-10T23:32:47.882 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T23:32:47.882 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:32:48.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.021+0000 7f0bec400640 1 -- 172.21.15.94:0/3127618751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0be40d0780 msgr2=0x7f0be40d1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:48.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.021+0000 7f0bec400640 1 --2- 172.21.15.94:0/3127618751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0be40d0780 0x7f0be40d1b00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0be40704a0 tx=0x7f0bd001c600 comp rx=0 tx=0).stop 2024-09-10T23:32:48.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.021+0000 7f0bec400640 1 -- 172.21.15.94:0/3127618751 shutdown_connections 2024-09-10T23:32:48.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.021+0000 7f0bec400640 1 --2- 172.21.15.94:0/3127618751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0be40d2040 0x7f0be415f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:48.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.021+0000 7f0bec400640 1 --2- 172.21.15.94:0/3127618751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0be40d0780 0x7f0be40d1b00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:48.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.021+0000 7f0bec400640 1 --2- 172.21.15.94:0/3127618751 >> v2:172.21.15.181:3300/0 conn(0x7f0be415bdc0 0x7f0be40d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:48.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.021+0000 7f0bec400640 1 -- 172.21.15.94:0/3127618751 >> 172.21.15.94:0/3127618751 conn(0x7f0be409aa70 msgr2=0x7f0be409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:48.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.021+0000 7f0bec400640 1 -- 172.21.15.94:0/3127618751 shutdown_connections 2024-09-10T23:32:48.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.021+0000 7f0bec400640 1 -- 172.21.15.94:0/3127618751 wait complete. 2024-09-10T23:32:48.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.021+0000 7f0bec400640 1 Processor -- start 2024-09-10T23:32:48.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.021+0000 7f0bec400640 1 -- start start 2024-09-10T23:32:48.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0bec400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0be40d0780 0x7f0be4155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:48.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0bec400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0be40d2040 0x7f0be40c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:48.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0bec400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0be415bdc0 0x7f0be40cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:48.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0bec400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0be40aefc0 con 0x7f0be40d0780 2024-09-10T23:32:48.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0bec400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0be40aee40 con 0x7f0be40d2040 2024-09-10T23:32:48.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0bec400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0be40af140 con 0x7f0be415bdc0 2024-09-10T23:32:48.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0be9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0be40d0780 0x7f0be4155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:48.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0be9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0be40d0780 0x7f0be4155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50926/0 (socket says 172.21.15.94:50926) 2024-09-10T23:32:48.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0be9a00640 1 -- 172.21.15.94:0/3834904222 learned_addr learned my addr 172.21.15.94:0/3834904222 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:48.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0be9000640 1 --2- 172.21.15.94:0/3834904222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0be40d2040 0x7f0be40c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:48.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0bea400640 1 --2- 172.21.15.94:0/3834904222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0be415bdc0 0x7f0be40cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:48.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0be9a00640 1 -- 172.21.15.94:0/3834904222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0be415bdc0 msgr2=0x7f0be40cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:48.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0be9a00640 1 --2- 172.21.15.94:0/3834904222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0be415bdc0 0x7f0be40cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:48.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0be9a00640 1 -- 172.21.15.94:0/3834904222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0be40d2040 msgr2=0x7f0be40c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:48.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0be9a00640 1 --2- 172.21.15.94:0/3834904222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0be40d2040 0x7f0be40c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:48.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0be9a00640 1 -- 172.21.15.94:0/3834904222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be415ef70 con 0x7f0be40d0780 2024-09-10T23:32:48.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0bea400640 1 --2- 172.21.15.94:0/3834904222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0be415bdc0 0x7f0be40cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:32:48.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0be9a00640 1 --2- 172.21.15.94:0/3834904222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0be40d0780 0x7f0be4155150 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800ba50 tx=0x7f0bd800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:48.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0be1600640 1 -- 172.21.15.94:0/3834904222 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8019070 con 0x7f0be40d0780 2024-09-10T23:32:48.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0be1600640 1 -- 172.21.15.94:0/3834904222 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bd800ccf0 con 0x7f0be40d0780 2024-09-10T23:32:48.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0bec400640 1 -- 172.21.15.94:0/3834904222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be40cc810 con 0x7f0be40d0780 2024-09-10T23:32:48.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0be1600640 1 -- 172.21.15.94:0/3834904222 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80146a0 con 0x7f0be40d0780 2024-09-10T23:32:48.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0bec400640 1 -- 172.21.15.94:0/3834904222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be40ccdd0 con 0x7f0be40d0780 2024-09-10T23:32:48.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0bec400640 1 -- 172.21.15.94:0/3834904222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be4158000 con 0x7f0be40d0780 2024-09-10T23:32:48.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.025+0000 7f0be1600640 1 -- 172.21.15.94:0/3834904222 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0bd801d3b0 con 0x7f0be40d0780 2024-09-10T23:32:48.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.029+0000 7f0be1600640 1 --2- 172.21.15.94:0/3834904222 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0bc40725e0 0x7f0bc4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:48.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.029+0000 7f0be1600640 1 -- 172.21.15.94:0/3834904222 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0bd8093330 con 0x7f0be40d0780 2024-09-10T23:32:48.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.029+0000 7f0be9000640 1 --2- 172.21.15.94:0/3834904222 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0bc40725e0 0x7f0bc4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:48.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.029+0000 7f0be9000640 1 --2- 172.21.15.94:0/3834904222 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0bc40725e0 0x7f0bc4074aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00af000 tx=0x7f0bd001ae40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:48.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.033+0000 7f0be1600640 1 -- 172.21.15.94:0/3834904222 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0bd805ea00 con 0x7f0be40d0780 2024-09-10T23:32:48.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.193+0000 7f0bec400640 1 -- 172.21.15.94:0/3834904222 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0be40d2b10 con 0x7f0bc40725e0 2024-09-10T23:32:48.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.201+0000 7f0be1600640 1 -- 172.21.15.94:0/3834904222 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251933 (secure 0 0 0) 0x7f0be40d2b10 con 0x7f0bc40725e0 2024-09-10T23:32:48.204 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:48.206 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:32:48.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.201+0000 7f0bec400640 1 -- 172.21.15.94:0/3834904222 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0bc40725e0 msgr2=0x7f0bc4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:48.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.201+0000 7f0bec400640 1 --2- 172.21.15.94:0/3834904222 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0bc40725e0 0x7f0bc4074aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00af000 tx=0x7f0bd001ae40 comp rx=0 tx=0).stop 2024-09-10T23:32:48.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.201+0000 7f0bec400640 1 -- 172.21.15.94:0/3834904222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0be40d0780 msgr2=0x7f0be4155150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:48.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.201+0000 7f0bec400640 1 --2- 172.21.15.94:0/3834904222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0be40d0780 0x7f0be4155150 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800ba50 tx=0x7f0bd800bf10 comp rx=0 tx=0).stop 2024-09-10T23:32:48.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.201+0000 7f0bec400640 1 -- 172.21.15.94:0/3834904222 shutdown_connections 2024-09-10T23:32:48.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.201+0000 7f0bec400640 1 --2- 172.21.15.94:0/3834904222 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0bc40725e0 0x7f0bc4074aa0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:48.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.201+0000 7f0bec400640 1 --2- 172.21.15.94:0/3834904222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0be40d2040 0x7f0be40c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:48.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.201+0000 7f0bec400640 1 --2- 172.21.15.94:0/3834904222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0be415bdc0 0x7f0be40cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:48.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.201+0000 7f0bec400640 1 --2- 172.21.15.94:0/3834904222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0be40d0780 0x7f0be4155150 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:48.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.201+0000 7f0bec400640 1 -- 172.21.15.94:0/3834904222 >> 172.21.15.94:0/3834904222 conn(0x7f0be409aa70 msgr2=0x7f0be4071370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:48.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.201+0000 7f0bec400640 1 -- 172.21.15.94:0/3834904222 shutdown_connections 2024-09-10T23:32:48.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:48.201+0000 7f0bec400640 1 -- 172.21.15.94:0/3834904222 wait complete. 2024-09-10T23:32:48.225 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-10T23:32:47.342717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235848,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738872,"statfs":{"total":773094113280,"available":772852604928,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001381"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627741+0000","last_change":"2024-09-10T23:32:42.593279+0000","last_active":"2024-09-10T23:32:43.627741+0000","last_peered":"2024-09-10T23:32:43.627741+0000","last_clean":"2024-09-10T23:32:43.627741+0000","last_became_active":"2024-09-10T23:32:42.593056+0000","last_became_peered":"2024-09-10T23:32:42.593056+0000","last_unstale":"2024-09-10T23:32:43.627741+0000","last_undegraded":"2024-09-10T23:32:43.627741+0000","last_fullsized":"2024-09-10T23:32:43.627741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.112551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628801+0000","last_change":"2024-09-10T23:32:42.584101+0000","last_active":"2024-09-10T23:32:43.628801+0000","last_peered":"2024-09-10T23:32:43.628801+0000","last_clean":"2024-09-10T23:32:43.628801+0000","last_became_active":"2024-09-10T23:32:42.583796+0000","last_became_peered":"2024-09-10T23:32:42.583796+0000","last_unstale":"2024-09-10T23:32:43.628801+0000","last_undegraded":"2024-09-10T23:32:43.628801+0000","last_fullsized":"2024-09-10T23:32:43.628801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:01.463027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628392+0000","last_change":"2024-09-10T23:32:42.584302+0000","last_active":"2024-09-10T23:32:43.628392+0000","last_peered":"2024-09-10T23:32:43.628392+0000","last_clean":"2024-09-10T23:32:43.628392+0000","last_became_active":"2024-09-10T23:32:42.583989+0000","last_became_peered":"2024-09-10T23:32:42.583989+0000","last_unstale":"2024-09-10T23:32:43.628392+0000","last_undegraded":"2024-09-10T23:32:43.628392+0000","last_fullsized":"2024-09-10T23:32:43.628392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:53.622106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626948+0000","last_change":"2024-09-10T23:32:42.565401+0000","last_active":"2024-09-10T23:32:43.626948+0000","last_peered":"2024-09-10T23:32:43.626948+0000","last_clean":"2024-09-10T23:32:43.626948+0000","last_became_active":"2024-09-10T23:32:42.565106+0000","last_became_peered":"2024-09-10T23:32:42.565106+0000","last_unstale":"2024-09-10T23:32:43.626948+0000","last_undegraded":"2024-09-10T23:32:43.626948+0000","last_fullsized":"2024-09-10T23:32:43.626948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:20.388860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627066+0000","last_change":"2024-09-10T23:32:42.586253+0000","last_active":"2024-09-10T23:32:43.627066+0000","last_peered":"2024-09-10T23:32:43.627066+0000","last_clean":"2024-09-10T23:32:43.627066+0000","last_became_active":"2024-09-10T23:32:42.585961+0000","last_became_peered":"2024-09-10T23:32:42.585961+0000","last_unstale":"2024-09-10T23:32:43.627066+0000","last_undegraded":"2024-09-10T23:32:43.627066+0000","last_fullsized":"2024-09-10T23:32:43.627066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:19.891408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626891+0000","last_change":"2024-09-10T23:32:42.565746+0000","last_active":"2024-09-10T23:32:43.626891+0000","last_peered":"2024-09-10T23:32:43.626891+0000","last_clean":"2024-09-10T23:32:43.626891+0000","last_became_active":"2024-09-10T23:32:42.565398+0000","last_became_peered":"2024-09-10T23:32:42.565398+0000","last_unstale":"2024-09-10T23:32:43.626891+0000","last_undegraded":"2024-09-10T23:32:43.626891+0000","last_fullsized":"2024-09-10T23:32:43.626891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:28.431860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627187+0000","last_change":"2024-09-10T23:32:42.567960+0000","last_active":"2024-09-10T23:32:43.627187+0000","last_peered":"2024-09-10T23:32:43.627187+0000","last_clean":"2024-09-10T23:32:43.627187+0000","last_became_active":"2024-09-10T23:32:42.567653+0000","last_became_peered":"2024-09-10T23:32:42.567653+0000","last_unstale":"2024-09-10T23:32:43.627187+0000","last_undegraded":"2024-09-10T23:32:43.627187+0000","last_fullsized":"2024-09-10T23:32:43.627187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:26.192704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628554+0000","last_change":"2024-09-10T23:32:42.584639+0000","last_active":"2024-09-10T23:32:43.628554+0000","last_peered":"2024-09-10T23:32:43.628554+0000","last_clean":"2024-09-10T23:32:43.628554+0000","last_became_active":"2024-09-10T23:32:42.584396+0000","last_became_peered":"2024-09-10T23:32:42.584396+0000","last_unstale":"2024-09-10T23:32:43.628554+0000","last_undegraded":"2024-09-10T23:32:43.628554+0000","last_fullsized":"2024-09-10T23:32:43.628554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:27.237846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563757+0000","last_change":"2024-09-10T23:32:42.589096+0000","last_active":"2024-09-10T23:32:43.563757+0000","last_peered":"2024-09-10T23:32:43.563757+0000","last_clean":"2024-09-10T23:32:43.563757+0000","last_became_active":"2024-09-10T23:32:42.588695+0000","last_became_peered":"2024-09-10T23:32:42.588695+0000","last_unstale":"2024-09-10T23:32:43.563757+0000","last_undegraded":"2024-09-10T23:32:43.563757+0000","last_fullsized":"2024-09-10T23:32:43.563757+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:14.778902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627628+0000","last_change":"2024-09-10T23:32:42.572168+0000","last_active":"2024-09-10T23:32:43.627628+0000","last_peered":"2024-09-10T23:32:43.627628+0000","last_clean":"2024-09-10T23:32:43.627628+0000","last_became_active":"2024-09-10T23:32:42.571638+0000","last_became_peered":"2024-09-10T23:32:42.571638+0000","last_unstale":"2024-09-10T23:32:43.627628+0000","last_undegraded":"2024-09-10T23:32:43.627628+0000","last_fullsized":"2024-09-10T23:32:43.627628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:51.184952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627526+0000","last_change":"2024-09-10T23:32:42.580539+0000","last_active":"2024-09-10T23:32:43.627526+0000","last_peered":"2024-09-10T23:32:43.627526+0000","last_clean":"2024-09-10T23:32:43.627526+0000","last_became_active":"2024-09-10T23:32:42.580180+0000","last_became_peered":"2024-09-10T23:32:42.580180+0000","last_unstale":"2024-09-10T23:32:43.627526+0000","last_undegraded":"2024-09-10T23:32:43.627526+0000","last_fullsized":"2024-09-10T23:32:43.627526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:15.490925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627548+0000","last_change":"2024-09-10T23:32:42.568100+0000","last_active":"2024-09-10T23:32:43.627548+0000","last_peered":"2024-09-10T23:32:43.627548+0000","last_clean":"2024-09-10T23:32:43.627548+0000","last_became_active":"2024-09-10T23:32:42.567409+0000","last_became_peered":"2024-09-10T23:32:42.567409+0000","last_unstale":"2024-09-10T23:32:43.627548+0000","last_undegraded":"2024-09-10T23:32:43.627548+0000","last_fullsized":"2024-09-10T23:32:43.627548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.117710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569985+0000","last_change":"2024-09-10T23:32:42.569811+0000","last_active":"2024-09-10T23:32:43.569985+0000","last_peered":"2024-09-10T23:32:43.569985+0000","last_clean":"2024-09-10T23:32:43.569985+0000","last_became_active":"2024-09-10T23:32:42.569470+0000","last_became_peered":"2024-09-10T23:32:42.569470+0000","last_unstale":"2024-09-10T23:32:43.569985+0000","last_undegraded":"2024-09-10T23:32:43.569985+0000","last_fullsized":"2024-09-10T23:32:43.569985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:56.266045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570094+0000","last_change":"2024-09-10T23:32:42.578563+0000","last_active":"2024-09-10T23:32:43.570094+0000","last_peered":"2024-09-10T23:32:43.570094+0000","last_clean":"2024-09-10T23:32:43.570094+0000","last_became_active":"2024-09-10T23:32:42.578281+0000","last_became_peered":"2024-09-10T23:32:42.578281+0000","last_unstale":"2024-09-10T23:32:43.570094+0000","last_undegraded":"2024-09-10T23:32:43.570094+0000","last_fullsized":"2024-09-10T23:32:43.570094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:57.340004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628709+0000","last_change":"2024-09-10T23:32:42.564792+0000","last_active":"2024-09-10T23:32:43.628709+0000","last_peered":"2024-09-10T23:32:43.628709+0000","last_clean":"2024-09-10T23:32:43.628709+0000","last_became_active":"2024-09-10T23:32:42.564029+0000","last_became_peered":"2024-09-10T23:32:42.564029+0000","last_unstale":"2024-09-10T23:32:43.628709+0000","last_undegraded":"2024-09-10T23:32:43.628709+0000","last_fullsized":"2024-09-10T23:32:43.628709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:23.318586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628280+0000","last_change":"2024-09-10T23:32:42.583724+0000","last_active":"2024-09-10T23:32:43.628280+0000","last_peered":"2024-09-10T23:32:43.628280+0000","last_clean":"2024-09-10T23:32:43.628280+0000","last_became_active":"2024-09-10T23:32:42.583473+0000","last_became_peered":"2024-09-10T23:32:42.583473+0000","last_unstale":"2024-09-10T23:32:43.628280+0000","last_undegraded":"2024-09-10T23:32:43.628280+0000","last_fullsized":"2024-09-10T23:32:43.628280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:14.468774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629190+0000","last_change":"2024-09-10T23:32:42.588295+0000","last_active":"2024-09-10T23:32:43.629190+0000","last_peered":"2024-09-10T23:32:43.629190+0000","last_clean":"2024-09-10T23:32:43.629190+0000","last_became_active":"2024-09-10T23:32:42.587993+0000","last_became_peered":"2024-09-10T23:32:42.587993+0000","last_unstale":"2024-09-10T23:32:43.629190+0000","last_undegraded":"2024-09-10T23:32:43.629190+0000","last_fullsized":"2024-09-10T23:32:43.629190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:18:27.504742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629147+0000","last_change":"2024-09-10T23:32:42.579982+0000","last_active":"2024-09-10T23:32:43.629147+0000","last_peered":"2024-09-10T23:32:43.629147+0000","last_clean":"2024-09-10T23:32:43.629147+0000","last_became_active":"2024-09-10T23:32:42.575814+0000","last_became_peered":"2024-09-10T23:32:42.575814+0000","last_unstale":"2024-09-10T23:32:43.629147+0000","last_undegraded":"2024-09-10T23:32:43.629147+0000","last_fullsized":"2024-09-10T23:32:43.629147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:58.672853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629002+0000","last_change":"2024-09-10T23:32:42.585466+0000","last_active":"2024-09-10T23:32:43.629002+0000","last_peered":"2024-09-10T23:32:43.629002+0000","last_clean":"2024-09-10T23:32:43.629002+0000","last_became_active":"2024-09-10T23:32:42.582650+0000","last_became_peered":"2024-09-10T23:32:42.582650+0000","last_unstale":"2024-09-10T23:32:43.629002+0000","last_undegraded":"2024-09-10T23:32:43.629002+0000","last_fullsized":"2024-09-10T23:32:43.629002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:59.434883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627679+0000","last_change":"2024-09-10T23:32:42.573723+0000","last_active":"2024-09-10T23:32:43.627679+0000","last_peered":"2024-09-10T23:32:43.627679+0000","last_clean":"2024-09-10T23:32:43.627679+0000","last_became_active":"2024-09-10T23:32:42.573309+0000","last_became_peered":"2024-09-10T23:32:42.573309+0000","last_unstale":"2024-09-10T23:32:43.627679+0000","last_undegraded":"2024-09-10T23:32:43.627679+0000","last_fullsized":"2024-09-10T23:32:43.627679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:08.762419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569445+0000","last_change":"2024-09-10T23:32:42.558045+0000","last_active":"2024-09-10T23:32:43.569445+0000","last_peered":"2024-09-10T23:32:43.569445+0000","last_clean":"2024-09-10T23:32:43.569445+0000","last_became_active":"2024-09-10T23:32:42.557774+0000","last_became_peered":"2024-09-10T23:32:42.557774+0000","last_unstale":"2024-09-10T23:32:43.569445+0000","last_undegraded":"2024-09-10T23:32:43.569445+0000","last_fullsized":"2024-09-10T23:32:43.569445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:53.622106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563214+0000","last_change":"2024-09-10T23:32:42.556890+0000","last_active":"2024-09-10T23:32:43.563214+0000","last_peered":"2024-09-10T23:32:43.563214+0000","last_clean":"2024-09-10T23:32:43.563214+0000","last_became_active":"2024-09-10T23:32:42.556625+0000","last_became_peered":"2024-09-10T23:32:42.556625+0000","last_unstale":"2024-09-10T23:32:43.563214+0000","last_undegraded":"2024-09-10T23:32:43.563214+0000","last_fullsized":"2024-09-10T23:32:43.563214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:14.440912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628804+0000","last_change":"2024-09-10T23:32:42.575698+0000","last_active":"2024-09-10T23:32:43.628804+0000","last_peered":"2024-09-10T23:32:43.628804+0000","last_clean":"2024-09-10T23:32:43.628804+0000","last_became_active":"2024-09-10T23:32:42.575371+0000","last_became_peered":"2024-09-10T23:32:42.575371+0000","last_unstale":"2024-09-10T23:32:43.628804+0000","last_undegraded":"2024-09-10T23:32:43.628804+0000","last_fullsized":"2024-09-10T23:32:43.628804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:36.598389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628388+0000","last_change":"2024-09-10T23:32:42.591870+0000","last_active":"2024-09-10T23:32:43.628388+0000","last_peered":"2024-09-10T23:32:43.628388+0000","last_clean":"2024-09-10T23:32:43.628388+0000","last_became_active":"2024-09-10T23:32:42.590207+0000","last_became_peered":"2024-09-10T23:32:42.590207+0000","last_unstale":"2024-09-10T23:32:43.628388+0000","last_undegraded":"2024-09-10T23:32:43.628388+0000","last_fullsized":"2024-09-10T23:32:43.628388+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:40.104986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629051+0000","last_change":"2024-09-10T23:32:42.588702+0000","last_active":"2024-09-10T23:32:43.629051+0000","last_peered":"2024-09-10T23:32:43.629051+0000","last_clean":"2024-09-10T23:32:43.629051+0000","last_became_active":"2024-09-10T23:32:42.588444+0000","last_became_peered":"2024-09-10T23:32:42.588444+0000","last_unstale":"2024-09-10T23:32:43.629051+0000","last_undegraded":"2024-09-10T23:32:43.629051+0000","last_fullsized":"2024-09-10T23:32:43.629051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:43.173720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627479+0000","last_change":"2024-09-10T23:32:42.580821+0000","last_active":"2024-09-10T23:32:43.627479+0000","last_peered":"2024-09-10T23:32:43.627479+0000","last_clean":"2024-09-10T23:32:43.627479+0000","last_became_active":"2024-09-10T23:32:42.580319+0000","last_became_peered":"2024-09-10T23:32:42.580319+0000","last_unstale":"2024-09-10T23:32:43.627479+0000","last_undegraded":"2024-09-10T23:32:43.627479+0000","last_fullsized":"2024-09-10T23:32:43.627479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:26.735696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570454+0000","last_change":"2024-09-10T23:32:42.574625+0000","last_active":"2024-09-10T23:32:43.570454+0000","last_peered":"2024-09-10T23:32:43.570454+0000","last_clean":"2024-09-10T23:32:43.570454+0000","last_became_active":"2024-09-10T23:32:42.571847+0000","last_became_peered":"2024-09-10T23:32:42.571847+0000","last_unstale":"2024-09-10T23:32:43.570454+0000","last_undegraded":"2024-09-10T23:32:43.570454+0000","last_fullsized":"2024-09-10T23:32:43.570454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:20.906137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570331+0000","last_change":"2024-09-10T23:32:42.567479+0000","last_active":"2024-09-10T23:32:43.570331+0000","last_peered":"2024-09-10T23:32:43.570331+0000","last_clean":"2024-09-10T23:32:43.570331+0000","last_became_active":"2024-09-10T23:32:42.567172+0000","last_became_peered":"2024-09-10T23:32:42.567172+0000","last_unstale":"2024-09-10T23:32:43.570331+0000","last_undegraded":"2024-09-10T23:32:43.570331+0000","last_fullsized":"2024-09-10T23:32:43.570331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:15.280711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628395+0000","last_change":"2024-09-10T23:32:42.585087+0000","last_active":"2024-09-10T23:32:43.628395+0000","last_peered":"2024-09-10T23:32:43.628395+0000","last_clean":"2024-09-10T23:32:43.628395+0000","last_became_active":"2024-09-10T23:32:42.583712+0000","last_became_peered":"2024-09-10T23:32:42.583712+0000","last_unstale":"2024-09-10T23:32:43.628395+0000","last_undegraded":"2024-09-10T23:32:43.628395+0000","last_fullsized":"2024-09-10T23:32:43.628395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:25.952890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628168+0000","last_change":"2024-09-10T23:32:42.591882+0000","last_active":"2024-09-10T23:32:43.628168+0000","last_peered":"2024-09-10T23:32:43.628168+0000","last_clean":"2024-09-10T23:32:43.628168+0000","last_became_active":"2024-09-10T23:32:42.590231+0000","last_became_peered":"2024-09-10T23:32:42.590231+0000","last_unstale":"2024-09-10T23:32:43.628168+0000","last_undegraded":"2024-09-10T23:32:43.628168+0000","last_fullsized":"2024-09-10T23:32:43.628168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:33.100231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628038+0000","last_change":"2024-09-10T23:32:42.573851+0000","last_active":"2024-09-10T23:32:43.628038+0000","last_peered":"2024-09-10T23:32:43.628038+0000","last_clean":"2024-09-10T23:32:43.628038+0000","last_became_active":"2024-09-10T23:32:42.572836+0000","last_became_peered":"2024-09-10T23:32:42.572836+0000","last_unstale":"2024-09-10T23:32:43.628038+0000","last_undegraded":"2024-09-10T23:32:43.628038+0000","last_fullsized":"2024-09-10T23:32:43.628038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:54.816587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627085+0000","last_change":"2024-09-10T23:32:42.576085+0000","last_active":"2024-09-10T23:32:43.627085+0000","last_peered":"2024-09-10T23:32:43.627085+0000","last_clean":"2024-09-10T23:32:43.627085+0000","last_became_active":"2024-09-10T23:32:42.575374+0000","last_became_peered":"2024-09-10T23:32:42.575374+0000","last_unstale":"2024-09-10T23:32:43.627085+0000","last_undegraded":"2024-09-10T23:32:43.627085+0000","last_fullsized":"2024-09-10T23:32:43.627085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:14.778902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563267+0000","last_change":"2024-09-10T23:32:42.588842+0000","last_active":"2024-09-10T23:32:43.563267+0000","last_peered":"2024-09-10T23:32:43.563267+0000","last_clean":"2024-09-10T23:32:43.563267+0000","last_became_active":"2024-09-10T23:32:42.588582+0000","last_became_peered":"2024-09-10T23:32:42.588582+0000","last_unstale":"2024-09-10T23:32:43.563267+0000","last_undegraded":"2024-09-10T23:32:43.563267+0000","last_fullsized":"2024-09-10T23:32:43.563267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:28.431860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628414+0000","last_change":"2024-09-10T23:32:41.569293+0000","last_active":"2024-09-10T23:32:43.628414+0000","last_peered":"2024-09-10T23:32:43.628414+0000","last_clean":"2024-09-10T23:32:43.628414+0000","last_became_active":"2024-09-10T23:32:41.568848+0000","last_became_peered":"2024-09-10T23:32:41.568848+0000","last_unstale":"2024-09-10T23:32:43.628414+0000","last_undegraded":"2024-09-10T23:32:43.628414+0000","last_fullsized":"2024-09-10T23:32:43.628414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:17.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569497+0000","last_change":"2024-09-10T23:32:42.568214+0000","last_active":"2024-09-10T23:32:43.569497+0000","last_peered":"2024-09-10T23:32:43.569497+0000","last_clean":"2024-09-10T23:32:43.569497+0000","last_became_active":"2024-09-10T23:32:42.562093+0000","last_became_peered":"2024-09-10T23:32:42.562093+0000","last_unstale":"2024-09-10T23:32:43.569497+0000","last_undegraded":"2024-09-10T23:32:43.569497+0000","last_fullsized":"2024-09-10T23:32:43.569497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:02.454545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628550+0000","last_change":"2024-09-10T23:32:41.578635+0000","last_active":"2024-09-10T23:32:43.628550+0000","last_peered":"2024-09-10T23:32:43.628550+0000","last_clean":"2024-09-10T23:32:43.628550+0000","last_became_active":"2024-09-10T23:32:41.578346+0000","last_became_peered":"2024-09-10T23:32:41.578346+0000","last_unstale":"2024-09-10T23:32:43.628550+0000","last_undegraded":"2024-09-10T23:32:43.628550+0000","last_fullsized":"2024-09-10T23:32:43.628550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:25.728667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569793+0000","last_change":"2024-09-10T23:32:41.553674+0000","last_active":"2024-09-10T23:32:43.569793+0000","last_peered":"2024-09-10T23:32:43.569793+0000","last_clean":"2024-09-10T23:32:43.569793+0000","last_became_active":"2024-09-10T23:32:41.553244+0000","last_became_peered":"2024-09-10T23:32:41.553244+0000","last_unstale":"2024-09-10T23:32:43.569793+0000","last_undegraded":"2024-09-10T23:32:43.569793+0000","last_fullsized":"2024-09-10T23:32:43.569793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:38.098241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628828+0000","last_change":"2024-09-10T23:32:42.583587+0000","last_active":"2024-09-10T23:32:43.628828+0000","last_peered":"2024-09-10T23:32:43.628828+0000","last_clean":"2024-09-10T23:32:43.628828+0000","last_became_active":"2024-09-10T23:32:42.581831+0000","last_became_peered":"2024-09-10T23:32:42.581831+0000","last_unstale":"2024-09-10T23:32:43.628828+0000","last_undegraded":"2024-09-10T23:32:43.628828+0000","last_fullsized":"2024-09-10T23:32:43.628828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:23.493444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628532+0000","last_change":"2024-09-10T23:32:41.581810+0000","last_active":"2024-09-10T23:32:43.628532+0000","last_peered":"2024-09-10T23:32:43.628532+0000","last_clean":"2024-09-10T23:32:43.628532+0000","last_became_active":"2024-09-10T23:32:41.581531+0000","last_became_peered":"2024-09-10T23:32:41.581531+0000","last_unstale":"2024-09-10T23:32:43.628532+0000","last_undegraded":"2024-09-10T23:32:43.628532+0000","last_fullsized":"2024-09-10T23:32:43.628532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:57.665825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569188+0000","last_change":"2024-09-10T23:32:42.560663+0000","last_active":"2024-09-10T23:32:43.569188+0000","last_peered":"2024-09-10T23:32:43.569188+0000","last_clean":"2024-09-10T23:32:43.569188+0000","last_became_active":"2024-09-10T23:32:42.560293+0000","last_became_peered":"2024-09-10T23:32:42.560293+0000","last_unstale":"2024-09-10T23:32:43.569188+0000","last_undegraded":"2024-09-10T23:32:43.569188+0000","last_fullsized":"2024-09-10T23:32:43.569188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:39.387246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563757+0000","last_change":"2024-09-10T23:32:41.573958+0000","last_active":"2024-09-10T23:32:43.563757+0000","last_peered":"2024-09-10T23:32:43.563757+0000","last_clean":"2024-09-10T23:32:43.563757+0000","last_became_active":"2024-09-10T23:32:41.573639+0000","last_became_peered":"2024-09-10T23:32:41.573639+0000","last_unstale":"2024-09-10T23:32:43.563757+0000","last_undegraded":"2024-09-10T23:32:43.563757+0000","last_fullsized":"2024-09-10T23:32:43.563757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:27.548086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627506+0000","last_change":"2024-09-10T23:32:42.580586+0000","last_active":"2024-09-10T23:32:43.627506+0000","last_peered":"2024-09-10T23:32:43.627506+0000","last_clean":"2024-09-10T23:32:43.627506+0000","last_became_active":"2024-09-10T23:32:42.579960+0000","last_became_peered":"2024-09-10T23:32:42.579960+0000","last_unstale":"2024-09-10T23:32:43.627506+0000","last_undegraded":"2024-09-10T23:32:43.627506+0000","last_fullsized":"2024-09-10T23:32:43.627506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:09.640521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627910+0000","last_change":"2024-09-10T23:32:41.570662+0000","last_active":"2024-09-10T23:32:43.627910+0000","last_peered":"2024-09-10T23:32:43.627910+0000","last_clean":"2024-09-10T23:32:43.627910+0000","last_became_active":"2024-09-10T23:32:41.570330+0000","last_became_peered":"2024-09-10T23:32:41.570330+0000","last_unstale":"2024-09-10T23:32:43.627910+0000","last_undegraded":"2024-09-10T23:32:43.627910+0000","last_fullsized":"2024-09-10T23:32:43.627910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:53.809559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629328+0000","last_change":"2024-09-10T23:32:42.591662+0000","last_active":"2024-09-10T23:32:43.629328+0000","last_peered":"2024-09-10T23:32:43.629328+0000","last_clean":"2024-09-10T23:32:43.629328+0000","last_became_active":"2024-09-10T23:32:42.591315+0000","last_became_peered":"2024-09-10T23:32:42.591315+0000","last_unstale":"2024-09-10T23:32:43.629328+0000","last_undegraded":"2024-09-10T23:32:43.629328+0000","last_fullsized":"2024-09-10T23:32:43.629328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:34.259468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628947+0000","last_change":"2024-09-10T23:32:41.571940+0000","last_active":"2024-09-10T23:32:43.628947+0000","last_peered":"2024-09-10T23:32:43.628947+0000","last_clean":"2024-09-10T23:32:43.628947+0000","last_became_active":"2024-09-10T23:32:41.571537+0000","last_became_peered":"2024-09-10T23:32:41.571537+0000","last_unstale":"2024-09-10T23:32:43.628947+0000","last_undegraded":"2024-09-10T23:32:43.628947+0000","last_fullsized":"2024-09-10T23:32:43.628947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:49.898494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629065+0000","last_change":"2024-09-10T23:32:42.585332+0000","last_active":"2024-09-10T23:32:43.629065+0000","last_peered":"2024-09-10T23:32:43.629065+0000","last_clean":"2024-09-10T23:32:43.629065+0000","last_became_active":"2024-09-10T23:32:42.585046+0000","last_became_peered":"2024-09-10T23:32:42.585046+0000","last_unstale":"2024-09-10T23:32:43.629065+0000","last_undegraded":"2024-09-10T23:32:43.629065+0000","last_fullsized":"2024-09-10T23:32:43.629065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:24.681728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627051+0000","last_change":"2024-09-10T23:32:41.565270+0000","last_active":"2024-09-10T23:32:43.627051+0000","last_peered":"2024-09-10T23:32:43.627051+0000","last_clean":"2024-09-10T23:32:43.627051+0000","last_became_active":"2024-09-10T23:32:41.564864+0000","last_became_peered":"2024-09-10T23:32:41.564864+0000","last_unstale":"2024-09-10T23:32:43.627051+0000","last_undegraded":"2024-09-10T23:32:43.627051+0000","last_fullsized":"2024-09-10T23:32:43.627051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:55.242033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627514+0000","last_change":"2024-09-10T23:32:42.576998+0000","last_active":"2024-09-10T23:32:43.627514+0000","last_peered":"2024-09-10T23:32:43.627514+0000","last_clean":"2024-09-10T23:32:43.627514+0000","last_became_active":"2024-09-10T23:32:42.576695+0000","last_became_peered":"2024-09-10T23:32:42.576695+0000","last_unstale":"2024-09-10T23:32:43.627514+0000","last_undegraded":"2024-09-10T23:32:43.627514+0000","last_fullsized":"2024-09-10T23:32:43.627514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:09.931603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628218+0000","last_change":"2024-09-10T23:32:41.571048+0000","last_active":"2024-09-10T23:32:43.628218+0000","last_peered":"2024-09-10T23:32:43.628218+0000","last_clean":"2024-09-10T23:32:43.628218+0000","last_became_active":"2024-09-10T23:32:41.570753+0000","last_became_peered":"2024-09-10T23:32:41.570753+0000","last_unstale":"2024-09-10T23:32:43.628218+0000","last_undegraded":"2024-09-10T23:32:43.628218+0000","last_fullsized":"2024-09-10T23:32:43.628218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:55.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570370+0000","last_change":"2024-09-10T23:32:42.564300+0000","last_active":"2024-09-10T23:32:43.570370+0000","last_peered":"2024-09-10T23:32:43.570370+0000","last_clean":"2024-09-10T23:32:43.570370+0000","last_became_active":"2024-09-10T23:32:42.564047+0000","last_became_peered":"2024-09-10T23:32:42.564047+0000","last_unstale":"2024-09-10T23:32:43.570370+0000","last_undegraded":"2024-09-10T23:32:43.570370+0000","last_fullsized":"2024-09-10T23:32:43.570370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:22.724821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628645+0000","last_change":"2024-09-10T23:32:41.585012+0000","last_active":"2024-09-10T23:32:43.628645+0000","last_peered":"2024-09-10T23:32:43.628645+0000","last_clean":"2024-09-10T23:32:43.628645+0000","last_became_active":"2024-09-10T23:32:41.584732+0000","last_became_peered":"2024-09-10T23:32:41.584732+0000","last_unstale":"2024-09-10T23:32:43.628645+0000","last_undegraded":"2024-09-10T23:32:43.628645+0000","last_fullsized":"2024-09-10T23:32:43.628645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:39.041511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628794+0000","last_change":"2024-09-10T23:32:42.569000+0000","last_active":"2024-09-10T23:32:43.628794+0000","last_peered":"2024-09-10T23:32:43.628794+0000","last_clean":"2024-09-10T23:32:43.628794+0000","last_became_active":"2024-09-10T23:32:42.568721+0000","last_became_peered":"2024-09-10T23:32:42.568721+0000","last_unstale":"2024-09-10T23:32:43.628794+0000","last_undegraded":"2024-09-10T23:32:43.628794+0000","last_fullsized":"2024-09-10T23:32:43.628794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:37.891368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626833+0000","last_change":"2024-09-10T23:32:41.578661+0000","last_active":"2024-09-10T23:32:43.626833+0000","last_peered":"2024-09-10T23:32:43.626833+0000","last_clean":"2024-09-10T23:32:43.626833+0000","last_became_active":"2024-09-10T23:32:41.578343+0000","last_became_peered":"2024-09-10T23:32:41.578343+0000","last_unstale":"2024-09-10T23:32:43.626833+0000","last_undegraded":"2024-09-10T23:32:43.626833+0000","last_fullsized":"2024-09-10T23:32:43.626833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:47.815432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563134+0000","last_change":"2024-09-10T23:32:42.584934+0000","last_active":"2024-09-10T23:32:43.563134+0000","last_peered":"2024-09-10T23:32:43.563134+0000","last_clean":"2024-09-10T23:32:43.563134+0000","last_became_active":"2024-09-10T23:32:42.584709+0000","last_became_peered":"2024-09-10T23:32:42.584709+0000","last_unstale":"2024-09-10T23:32:43.563134+0000","last_undegraded":"2024-09-10T23:32:43.563134+0000","last_fullsized":"2024-09-10T23:32:43.563134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:30.543243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.568880+0000","last_change":"2024-09-10T23:32:41.566144+0000","last_active":"2024-09-10T23:32:43.568880+0000","last_peered":"2024-09-10T23:32:43.568880+0000","last_clean":"2024-09-10T23:32:43.568880+0000","last_became_active":"2024-09-10T23:32:41.565841+0000","last_became_peered":"2024-09-10T23:32:41.565841+0000","last_unstale":"2024-09-10T23:32:43.568880+0000","last_undegraded":"2024-09-10T23:32:43.568880+0000","last_fullsized":"2024-09-10T23:32:43.568880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:51.080022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628125+0000","last_change":"2024-09-10T23:32:42.586470+0000","last_active":"2024-09-10T23:32:43.628125+0000","last_peered":"2024-09-10T23:32:43.628125+0000","last_clean":"2024-09-10T23:32:43.628125+0000","last_became_active":"2024-09-10T23:32:42.579377+0000","last_became_peered":"2024-09-10T23:32:42.579377+0000","last_unstale":"2024-09-10T23:32:43.628125+0000","last_undegraded":"2024-09-10T23:32:43.628125+0000","last_fullsized":"2024-09-10T23:32:43.628125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.112551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628061+0000","last_change":"2024-09-10T23:32:41.567212+0000","last_active":"2024-09-10T23:32:43.628061+0000","last_peered":"2024-09-10T23:32:43.628061+0000","last_clean":"2024-09-10T23:32:43.628061+0000","last_became_active":"2024-09-10T23:32:41.566857+0000","last_became_peered":"2024-09-10T23:32:41.566857+0000","last_unstale":"2024-09-10T23:32:43.628061+0000","last_undegraded":"2024-09-10T23:32:43.628061+0000","last_fullsized":"2024-09-10T23:32:43.628061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:05.259433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627653+0000","last_change":"2024-09-10T23:32:42.558295+0000","last_active":"2024-09-10T23:32:43.627653+0000","last_peered":"2024-09-10T23:32:43.627653+0000","last_clean":"2024-09-10T23:32:43.627653+0000","last_became_active":"2024-09-10T23:32:42.558039+0000","last_became_peered":"2024-09-10T23:32:42.558039+0000","last_unstale":"2024-09-10T23:32:43.627653+0000","last_undegraded":"2024-09-10T23:32:43.627653+0000","last_fullsized":"2024-09-10T23:32:43.627653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:07.033834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569932+0000","last_change":"2024-09-10T23:32:41.568016+0000","last_active":"2024-09-10T23:32:43.569932+0000","last_peered":"2024-09-10T23:32:43.569932+0000","last_clean":"2024-09-10T23:32:43.569932+0000","last_became_active":"2024-09-10T23:32:41.566960+0000","last_became_peered":"2024-09-10T23:32:41.566960+0000","last_unstale":"2024-09-10T23:32:43.569932+0000","last_undegraded":"2024-09-10T23:32:43.569932+0000","last_fullsized":"2024-09-10T23:32:43.569932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:14.145496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628571+0000","last_change":"2024-09-10T23:32:42.562392+0000","last_active":"2024-09-10T23:32:43.628571+0000","last_peered":"2024-09-10T23:32:43.628571+0000","last_clean":"2024-09-10T23:32:43.628571+0000","last_became_active":"2024-09-10T23:32:42.562042+0000","last_became_peered":"2024-09-10T23:32:42.562042+0000","last_unstale":"2024-09-10T23:32:43.628571+0000","last_undegraded":"2024-09-10T23:32:43.628571+0000","last_fullsized":"2024-09-10T23:32:43.628571+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:33.100231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628521+0000","last_change":"2024-09-10T23:32:41.572986+0000","last_active":"2024-09-10T23:32:43.628521+0000","last_peered":"2024-09-10T23:32:43.628521+0000","last_clean":"2024-09-10T23:32:43.628521+0000","last_became_active":"2024-09-10T23:32:41.572722+0000","last_became_peered":"2024-09-10T23:32:41.572722+0000","last_unstale":"2024-09-10T23:32:43.628521+0000","last_undegraded":"2024-09-10T23:32:43.628521+0000","last_fullsized":"2024-09-10T23:32:43.628521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:49.898494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628742+0000","last_change":"2024-09-10T23:32:42.573643+0000","last_active":"2024-09-10T23:32:43.628742+0000","last_peered":"2024-09-10T23:32:43.628742+0000","last_clean":"2024-09-10T23:32:43.628742+0000","last_became_active":"2024-09-10T23:32:42.572873+0000","last_became_peered":"2024-09-10T23:32:42.572873+0000","last_unstale":"2024-09-10T23:32:43.628742+0000","last_undegraded":"2024-09-10T23:32:43.628742+0000","last_fullsized":"2024-09-10T23:32:43.628742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:14.468774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570459+0000","last_change":"2024-09-10T23:32:41.562365+0000","last_active":"2024-09-10T23:32:43.570459+0000","last_peered":"2024-09-10T23:32:43.570459+0000","last_clean":"2024-09-10T23:32:43.570459+0000","last_became_active":"2024-09-10T23:32:41.562021+0000","last_became_peered":"2024-09-10T23:32:41.562021+0000","last_unstale":"2024-09-10T23:32:43.570459+0000","last_undegraded":"2024-09-10T23:32:43.570459+0000","last_fullsized":"2024-09-10T23:32:43.570459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:08:28.878205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628732+0000","last_change":"2024-09-10T23:32:42.584713+0000","last_active":"2024-09-10T23:32:43.628732+0000","last_peered":"2024-09-10T23:32:43.628732+0000","last_clean":"2024-09-10T23:32:43.628732+0000","last_became_active":"2024-09-10T23:32:42.577789+0000","last_became_peered":"2024-09-10T23:32:42.577789+0000","last_unstale":"2024-09-10T23:32:43.628732+0000","last_undegraded":"2024-09-10T23:32:43.628732+0000","last_fullsized":"2024-09-10T23:32:43.628732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:33.835681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563126+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:32:43.563126+0000","last_peered":"2024-09-10T23:32:43.563126+0000","last_clean":"2024-09-10T23:32:43.563126+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:32:43.563126+0000","last_undegraded":"2024-09-10T23:32:43.563126+0000","last_fullsized":"2024-09-10T23:32:43.563126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629078+0000","last_change":"2024-09-10T23:32:41.581530+0000","last_active":"2024-09-10T23:32:43.629078+0000","last_peered":"2024-09-10T23:32:43.629078+0000","last_clean":"2024-09-10T23:32:43.629078+0000","last_became_active":"2024-09-10T23:32:41.581242+0000","last_became_peered":"2024-09-10T23:32:41.581242+0000","last_unstale":"2024-09-10T23:32:43.629078+0000","last_undegraded":"2024-09-10T23:32:43.629078+0000","last_fullsized":"2024-09-10T23:32:43.629078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:08.633493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569220+0000","last_change":"2024-09-10T23:32:42.558841+0000","last_active":"2024-09-10T23:32:43.569220+0000","last_peered":"2024-09-10T23:32:43.569220+0000","last_clean":"2024-09-10T23:32:43.569220+0000","last_became_active":"2024-09-10T23:32:42.558580+0000","last_became_peered":"2024-09-10T23:32:42.558580+0000","last_unstale":"2024-09-10T23:32:43.569220+0000","last_undegraded":"2024-09-10T23:32:43.569220+0000","last_fullsized":"2024-09-10T23:32:43.569220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:54.620648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628625+0000","last_change":"2024-09-10T23:32:41.577467+0000","last_active":"2024-09-10T23:32:43.628625+0000","last_peered":"2024-09-10T23:32:43.628625+0000","last_clean":"2024-09-10T23:32:43.628625+0000","last_became_active":"2024-09-10T23:32:41.576702+0000","last_became_peered":"2024-09-10T23:32:41.576702+0000","last_unstale":"2024-09-10T23:32:43.628625+0000","last_undegraded":"2024-09-10T23:32:43.628625+0000","last_fullsized":"2024-09-10T23:32:43.628625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:32.527218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569670+0000","last_change":"2024-09-10T23:32:42.565100+0000","last_active":"2024-09-10T23:32:43.569670+0000","last_peered":"2024-09-10T23:32:43.569670+0000","last_clean":"2024-09-10T23:32:43.569670+0000","last_became_active":"2024-09-10T23:32:42.564790+0000","last_became_peered":"2024-09-10T23:32:42.564790+0000","last_unstale":"2024-09-10T23:32:43.569670+0000","last_undegraded":"2024-09-10T23:32:43.569670+0000","last_fullsized":"2024-09-10T23:32:43.569670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:44.555042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628065+0000","last_change":"2024-09-10T23:32:41.571266+0000","last_active":"2024-09-10T23:32:43.628065+0000","last_peered":"2024-09-10T23:32:43.628065+0000","last_clean":"2024-09-10T23:32:43.628065+0000","last_became_active":"2024-09-10T23:32:41.570986+0000","last_became_peered":"2024-09-10T23:32:41.570986+0000","last_unstale":"2024-09-10T23:32:43.628065+0000","last_undegraded":"2024-09-10T23:32:43.628065+0000","last_fullsized":"2024-09-10T23:32:43.628065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:58.170892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628975+0000","last_change":"2024-09-10T23:32:42.584295+0000","last_active":"2024-09-10T23:32:43.628975+0000","last_peered":"2024-09-10T23:32:43.628975+0000","last_clean":"2024-09-10T23:32:43.628975+0000","last_became_active":"2024-09-10T23:32:42.584001+0000","last_became_peered":"2024-09-10T23:32:42.584001+0000","last_unstale":"2024-09-10T23:32:43.628975+0000","last_undegraded":"2024-09-10T23:32:43.628975+0000","last_fullsized":"2024-09-10T23:32:43.628975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:05.044760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628726+0000","last_change":"2024-09-10T23:32:41.578157+0000","last_active":"2024-09-10T23:32:43.628726+0000","last_peered":"2024-09-10T23:32:43.628726+0000","last_clean":"2024-09-10T23:32:43.628726+0000","last_became_active":"2024-09-10T23:32:41.575442+0000","last_became_peered":"2024-09-10T23:32:41.575442+0000","last_unstale":"2024-09-10T23:32:43.628726+0000","last_undegraded":"2024-09-10T23:32:43.628726+0000","last_fullsized":"2024-09-10T23:32:43.628726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:00.455999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627197+0000","last_change":"2024-09-10T23:32:42.571239+0000","last_active":"2024-09-10T23:32:43.627197+0000","last_peered":"2024-09-10T23:32:43.627197+0000","last_clean":"2024-09-10T23:32:43.627197+0000","last_became_active":"2024-09-10T23:32:42.570963+0000","last_became_peered":"2024-09-10T23:32:42.570963+0000","last_unstale":"2024-09-10T23:32:43.627197+0000","last_undegraded":"2024-09-10T23:32:43.627197+0000","last_fullsized":"2024-09-10T23:32:43.627197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:36.598389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628178+0000","last_change":"2024-09-10T23:32:41.569326+0000","last_active":"2024-09-10T23:32:43.628178+0000","last_peered":"2024-09-10T23:32:43.628178+0000","last_clean":"2024-09-10T23:32:43.628178+0000","last_became_active":"2024-09-10T23:32:41.569056+0000","last_became_peered":"2024-09-10T23:32:41.569056+0000","last_unstale":"2024-09-10T23:32:43.628178+0000","last_undegraded":"2024-09-10T23:32:43.628178+0000","last_fullsized":"2024-09-10T23:32:43.628178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:47.815432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626872+0000","last_change":"2024-09-10T23:32:42.574800+0000","last_active":"2024-09-10T23:32:43.626872+0000","last_peered":"2024-09-10T23:32:43.626872+0000","last_clean":"2024-09-10T23:32:43.626872+0000","last_became_active":"2024-09-10T23:32:42.574477+0000","last_became_peered":"2024-09-10T23:32:42.574477+0000","last_unstale":"2024-09-10T23:32:43.626872+0000","last_undegraded":"2024-09-10T23:32:43.626872+0000","last_fullsized":"2024-09-10T23:32:43.626872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:14.440912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629156+0000","last_change":"2024-09-10T23:32:41.582865+0000","last_active":"2024-09-10T23:32:43.629156+0000","last_peered":"2024-09-10T23:32:43.629156+0000","last_clean":"2024-09-10T23:32:43.629156+0000","last_became_active":"2024-09-10T23:32:41.582522+0000","last_became_peered":"2024-09-10T23:32:41.582522+0000","last_unstale":"2024-09-10T23:32:43.629156+0000","last_undegraded":"2024-09-10T23:32:43.629156+0000","last_fullsized":"2024-09-10T23:32:43.629156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:24.945862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570471+0000","last_change":"2024-09-10T23:32:42.587148+0000","last_active":"2024-09-10T23:32:43.570471+0000","last_peered":"2024-09-10T23:32:43.570471+0000","last_clean":"2024-09-10T23:32:43.570471+0000","last_became_active":"2024-09-10T23:32:42.586887+0000","last_became_peered":"2024-09-10T23:32:42.586887+0000","last_unstale":"2024-09-10T23:32:43.570471+0000","last_undegraded":"2024-09-10T23:32:43.570471+0000","last_fullsized":"2024-09-10T23:32:43.570471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:27.999337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570022+0000","last_change":"2024-09-10T23:32:41.553400+0000","last_active":"2024-09-10T23:32:43.570022+0000","last_peered":"2024-09-10T23:32:43.570022+0000","last_clean":"2024-09-10T23:32:43.570022+0000","last_became_active":"2024-09-10T23:32:41.553070+0000","last_became_peered":"2024-09-10T23:32:41.553070+0000","last_unstale":"2024-09-10T23:32:43.570022+0000","last_undegraded":"2024-09-10T23:32:43.570022+0000","last_fullsized":"2024-09-10T23:32:43.570022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:01.393799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569840+0000","last_change":"2024-09-10T23:32:42.580714+0000","last_active":"2024-09-10T23:32:43.569840+0000","last_peered":"2024-09-10T23:32:43.569840+0000","last_clean":"2024-09-10T23:32:43.569840+0000","last_became_active":"2024-09-10T23:32:42.579027+0000","last_became_peered":"2024-09-10T23:32:42.579027+0000","last_unstale":"2024-09-10T23:32:43.569840+0000","last_undegraded":"2024-09-10T23:32:43.569840+0000","last_fullsized":"2024-09-10T23:32:43.569840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:12.456589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629216+0000","last_change":"2024-09-10T23:32:41.580054+0000","last_active":"2024-09-10T23:32:43.629216+0000","last_peered":"2024-09-10T23:32:43.629216+0000","last_clean":"2024-09-10T23:32:43.629216+0000","last_became_active":"2024-09-10T23:32:41.579663+0000","last_became_peered":"2024-09-10T23:32:41.579663+0000","last_unstale":"2024-09-10T23:32:43.629216+0000","last_undegraded":"2024-09-10T23:32:43.629216+0000","last_fullsized":"2024-09-10T23:32:43.629216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:22.486416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569890+0000","last_change":"2024-09-10T23:32:42.574503+0000","last_active":"2024-09-10T23:32:43.569890+0000","last_peered":"2024-09-10T23:32:43.569890+0000","last_clean":"2024-09-10T23:32:43.569890+0000","last_became_active":"2024-09-10T23:32:42.574229+0000","last_became_peered":"2024-09-10T23:32:42.574229+0000","last_unstale":"2024-09-10T23:32:43.569890+0000","last_undegraded":"2024-09-10T23:32:43.569890+0000","last_fullsized":"2024-09-10T23:32:43.569890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:19.891408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626574+0000","last_change":"2024-09-10T23:32:41.562000+0000","last_active":"2024-09-10T23:32:43.626574+0000","last_peered":"2024-09-10T23:32:43.626574+0000","last_clean":"2024-09-10T23:32:43.626574+0000","last_became_active":"2024-09-10T23:32:41.561637+0000","last_became_peered":"2024-09-10T23:32:41.561637+0000","last_unstale":"2024-09-10T23:32:43.626574+0000","last_undegraded":"2024-09-10T23:32:43.626574+0000","last_fullsized":"2024-09-10T23:32:43.626574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:17.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628329+0000","last_change":"2024-09-10T23:32:42.583205+0000","last_active":"2024-09-10T23:32:43.628329+0000","last_peered":"2024-09-10T23:32:43.628329+0000","last_clean":"2024-09-10T23:32:43.628329+0000","last_became_active":"2024-09-10T23:32:42.582963+0000","last_became_peered":"2024-09-10T23:32:42.582963+0000","last_unstale":"2024-09-10T23:32:43.628329+0000","last_undegraded":"2024-09-10T23:32:43.628329+0000","last_fullsized":"2024-09-10T23:32:43.628329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.117710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569104+0000","last_change":"2024-09-10T23:32:41.557934+0000","last_active":"2024-09-10T23:32:43.569104+0000","last_peered":"2024-09-10T23:32:43.569104+0000","last_clean":"2024-09-10T23:32:43.569104+0000","last_became_active":"2024-09-10T23:32:41.557625+0000","last_became_peered":"2024-09-10T23:32:41.557625+0000","last_unstale":"2024-09-10T23:32:43.569104+0000","last_undegraded":"2024-09-10T23:32:43.569104+0000","last_fullsized":"2024-09-10T23:32:43.569104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:29.536215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569080+0000","last_change":"2024-09-10T23:32:42.567250+0000","last_active":"2024-09-10T23:32:43.569080+0000","last_peered":"2024-09-10T23:32:43.569080+0000","last_clean":"2024-09-10T23:32:43.569080+0000","last_became_active":"2024-09-10T23:32:42.566248+0000","last_became_peered":"2024-09-10T23:32:42.566248+0000","last_unstale":"2024-09-10T23:32:43.569080+0000","last_undegraded":"2024-09-10T23:32:43.569080+0000","last_fullsized":"2024-09-10T23:32:43.569080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:45.824956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570495+0000","last_change":"2024-09-10T23:32:41.566640+0000","last_active":"2024-09-10T23:32:43.570495+0000","last_peered":"2024-09-10T23:32:43.570495+0000","last_clean":"2024-09-10T23:32:43.570495+0000","last_became_active":"2024-09-10T23:32:41.566311+0000","last_became_peered":"2024-09-10T23:32:41.566311+0000","last_unstale":"2024-09-10T23:32:43.570495+0000","last_undegraded":"2024-09-10T23:32:43.570495+0000","last_fullsized":"2024-09-10T23:32:43.570495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:33.372580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628173+0000","last_change":"2024-09-10T23:32:42.580760+0000","last_active":"2024-09-10T23:32:43.628173+0000","last_peered":"2024-09-10T23:32:43.628173+0000","last_clean":"2024-09-10T23:32:43.628173+0000","last_became_active":"2024-09-10T23:32:42.580081+0000","last_became_peered":"2024-09-10T23:32:42.580081+0000","last_unstale":"2024-09-10T23:32:43.628173+0000","last_undegraded":"2024-09-10T23:32:43.628173+0000","last_fullsized":"2024-09-10T23:32:43.628173+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:06.266461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628952+0000","last_change":"2024-09-10T23:32:41.583976+0000","last_active":"2024-09-10T23:32:43.628952+0000","last_peered":"2024-09-10T23:32:43.628952+0000","last_clean":"2024-09-10T23:32:43.628952+0000","last_became_active":"2024-09-10T23:32:41.583639+0000","last_became_peered":"2024-09-10T23:32:41.583639+0000","last_unstale":"2024-09-10T23:32:43.628952+0000","last_undegraded":"2024-09-10T23:32:43.628952+0000","last_fullsized":"2024-09-10T23:32:43.628952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:26.669949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629220+0000","last_change":"2024-09-10T23:32:42.570056+0000","last_active":"2024-09-10T23:32:43.629220+0000","last_peered":"2024-09-10T23:32:43.629220+0000","last_clean":"2024-09-10T23:32:43.629220+0000","last_became_active":"2024-09-10T23:32:42.569794+0000","last_became_peered":"2024-09-10T23:32:42.569794+0000","last_unstale":"2024-09-10T23:32:43.629220+0000","last_undegraded":"2024-09-10T23:32:43.629220+0000","last_fullsized":"2024-09-10T23:32:43.629220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:36.453126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628526+0000","last_change":"2024-09-10T23:32:42.585151+0000","last_active":"2024-09-10T23:32:43.628526+0000","last_peered":"2024-09-10T23:32:43.628526+0000","last_clean":"2024-09-10T23:32:43.628526+0000","last_became_active":"2024-09-10T23:32:42.578092+0000","last_became_peered":"2024-09-10T23:32:42.578092+0000","last_unstale":"2024-09-10T23:32:43.628526+0000","last_undegraded":"2024-09-10T23:32:43.628526+0000","last_fullsized":"2024-09-10T23:32:43.628526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:39:56.735321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628891+0000","last_change":"2024-09-10T23:32:41.571659+0000","last_active":"2024-09-10T23:32:43.628891+0000","last_peered":"2024-09-10T23:32:43.628891+0000","last_clean":"2024-09-10T23:32:43.628891+0000","last_became_active":"2024-09-10T23:32:41.571418+0000","last_became_peered":"2024-09-10T23:32:41.571418+0000","last_unstale":"2024-09-10T23:32:43.628891+0000","last_undegraded":"2024-09-10T23:32:43.628891+0000","last_fullsized":"2024-09-10T23:32:43.628891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:57.665825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563423+0000","last_change":"2024-09-10T23:32:42.563064+0000","last_active":"2024-09-10T23:32:43.563423+0000","last_peered":"2024-09-10T23:32:43.563423+0000","last_clean":"2024-09-10T23:32:43.563423+0000","last_became_active":"2024-09-10T23:32:42.562808+0000","last_became_peered":"2024-09-10T23:32:42.562808+0000","last_unstale":"2024-09-10T23:32:43.563423+0000","last_undegraded":"2024-09-10T23:32:43.563423+0000","last_fullsized":"2024-09-10T23:32:43.563423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:20.388860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627661+0000","last_change":"2024-09-10T23:32:41.570670+0000","last_active":"2024-09-10T23:32:43.627661+0000","last_peered":"2024-09-10T23:32:43.627661+0000","last_clean":"2024-09-10T23:32:43.627661+0000","last_became_active":"2024-09-10T23:32:41.570291+0000","last_became_peered":"2024-09-10T23:32:41.570291+0000","last_unstale":"2024-09-10T23:32:43.627661+0000","last_undegraded":"2024-09-10T23:32:43.627661+0000","last_fullsized":"2024-09-10T23:32:43.627661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:33.183319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629201+0000","last_change":"2024-09-10T23:32:42.584736+0000","last_active":"2024-09-10T23:32:43.629201+0000","last_peered":"2024-09-10T23:32:43.629201+0000","last_clean":"2024-09-10T23:32:43.629201+0000","last_became_active":"2024-09-10T23:32:42.583535+0000","last_became_peered":"2024-09-10T23:32:42.583535+0000","last_unstale":"2024-09-10T23:32:43.629201+0000","last_undegraded":"2024-09-10T23:32:43.629201+0000","last_fullsized":"2024-09-10T23:32:43.629201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:00.689816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629197+0000","last_change":"2024-09-10T23:32:41.566445+0000","last_active":"2024-09-10T23:32:43.629197+0000","last_peered":"2024-09-10T23:32:43.629197+0000","last_clean":"2024-09-10T23:32:43.629197+0000","last_became_active":"2024-09-10T23:32:41.566158+0000","last_became_peered":"2024-09-10T23:32:41.566158+0000","last_unstale":"2024-09-10T23:32:43.629197+0000","last_undegraded":"2024-09-10T23:32:43.629197+0000","last_fullsized":"2024-09-10T23:32:43.629197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:55.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629299+0000","last_change":"2024-09-10T23:32:42.584147+0000","last_active":"2024-09-10T23:32:43.629299+0000","last_peered":"2024-09-10T23:32:43.629299+0000","last_clean":"2024-09-10T23:32:43.629299+0000","last_became_active":"2024-09-10T23:32:42.583766+0000","last_became_peered":"2024-09-10T23:32:42.583766+0000","last_unstale":"2024-09-10T23:32:43.629299+0000","last_undegraded":"2024-09-10T23:32:43.629299+0000","last_fullsized":"2024-09-10T23:32:43.629299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:19.288688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569913+0000","last_change":"2024-09-10T23:32:41.580965+0000","last_active":"2024-09-10T23:32:43.569913+0000","last_peered":"2024-09-10T23:32:43.569913+0000","last_clean":"2024-09-10T23:32:43.569913+0000","last_became_active":"2024-09-10T23:32:41.580712+0000","last_became_peered":"2024-09-10T23:32:41.580712+0000","last_unstale":"2024-09-10T23:32:43.569913+0000","last_undegraded":"2024-09-10T23:32:43.569913+0000","last_fullsized":"2024-09-10T23:32:43.569913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:13.433884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:32:51.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:32:51.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.377+0000 7f1f43a00640 1 -- 172.21.15.94:0/2731844794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c06f600 msgr2=0x7f1f3c0b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:51.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.377+0000 7f1f43a00640 1 --2- 172.21.15.94:0/2731844794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c06f600 0x7f1f3c0b35c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0b2760 tx=0x7f1f3401de20 comp rx=0 tx=0).stop 2024-09-10T23:32:51.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.377+0000 7f1f43a00640 1 -- 172.21.15.94:0/2731844794 shutdown_connections 2024-09-10T23:32:51.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.377+0000 7f1f43a00640 1 --2- 172.21.15.94:0/2731844794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c06f600 0x7f1f3c0b35c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.377+0000 7f1f43a00640 1 --2- 172.21.15.94:0/2731844794 >> v2:172.21.15.181:3300/0 conn(0x7f1f3c157c60 0x7f1f3c072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.377+0000 7f1f43a00640 1 -- 172.21.15.94:0/2731844794 >> 172.21.15.94:0/2731844794 conn(0x7f1f3c06cdf0 msgr2=0x7f1f3c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:51.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.377+0000 7f1f43a00640 1 -- 172.21.15.94:0/2731844794 shutdown_connections 2024-09-10T23:32:51.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.377+0000 7f1f43a00640 1 -- 172.21.15.94:0/2731844794 wait complete. 2024-09-10T23:32:51.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f43a00640 1 Processor -- start 2024-09-10T23:32:51.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f43a00640 1 -- start start 2024-09-10T23:32:51.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f43a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f3c06f600 0x7f1f3c13f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:51.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f43a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c070150 0x7f1f3c163230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:51.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f43a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f3c157c60 0x7f1f3c163770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:51.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f43a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c15b310 con 0x7f1f3c070150 2024-09-10T23:32:51.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f43a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c15b190 con 0x7f1f3c157c60 2024-09-10T23:32:51.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f43a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1f3c15b490 con 0x7f1f3c06f600 2024-09-10T23:32:51.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f33e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c070150 0x7f1f3c163230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:51.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f33e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c070150 0x7f1f3c163230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50940/0 (socket says 172.21.15.94:50940) 2024-09-10T23:32:51.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f33e00640 1 -- 172.21.15.94:0/2579430568 learned_addr learned my addr 172.21.15.94:0/2579430568 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:51.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f41a00640 1 --2- 172.21.15.94:0/2579430568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f3c157c60 0x7f1f3c163770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f41000640 1 --2- 172.21.15.94:0/2579430568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f3c06f600 0x7f1f3c13f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f33e00640 1 -- 172.21.15.94:0/2579430568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f3c06f600 msgr2=0x7f1f3c13f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f33e00640 1 --2- 172.21.15.94:0/2579430568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f3c06f600 0x7f1f3c13f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f33e00640 1 -- 172.21.15.94:0/2579430568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f3c157c60 msgr2=0x7f1f3c163770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f33e00640 1 --2- 172.21.15.94:0/2579430568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f3c157c60 0x7f1f3c163770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f33e00640 1 -- 172.21.15.94:0/2579430568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3c1f8580 con 0x7f1f3c070150 2024-09-10T23:32:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f41000640 1 --2- 172.21.15.94:0/2579430568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f3c06f600 0x7f1f3c13f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f41a00640 1 --2- 172.21.15.94:0/2579430568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f3c157c60 0x7f1f3c163770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f33e00640 1 --2- 172.21.15.94:0/2579430568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c070150 0x7f1f3c163230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400c420 tx=0x7f1f2400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f39600640 1 -- 172.21.15.94:0/2579430568 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24017070 con 0x7f1f3c070150 2024-09-10T23:32:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f39600640 1 -- 172.21.15.94:0/2579430568 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f24003600 con 0x7f1f3c070150 2024-09-10T23:32:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f43a00640 1 -- 172.21.15.94:0/2579430568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3c1521c0 con 0x7f1f3c070150 2024-09-10T23:32:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f39600640 1 -- 172.21.15.94:0/2579430568 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24003a90 con 0x7f1f3c070150 2024-09-10T23:32:51.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.381+0000 7f1f43a00640 1 -- 172.21.15.94:0/2579430568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f3c152780 con 0x7f1f3c070150 2024-09-10T23:32:51.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.385+0000 7f1f43a00640 1 -- 172.21.15.94:0/2579430568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f3c0b35c0 con 0x7f1f3c070150 2024-09-10T23:32:51.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.385+0000 7f1f39600640 1 -- 172.21.15.94:0/2579430568 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1f24003c30 con 0x7f1f3c070150 2024-09-10T23:32:51.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.385+0000 7f1f39600640 1 --2- 172.21.15.94:0/2579430568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1f18072610 0x7f1f18074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:51.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.385+0000 7f1f39600640 1 -- 172.21.15.94:0/2579430568 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1f24092800 con 0x7f1f3c070150 2024-09-10T23:32:51.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.385+0000 7f1f41000640 1 --2- 172.21.15.94:0/2579430568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1f18072610 0x7f1f18074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:51.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.385+0000 7f1f41000640 1 --2- 172.21.15.94:0/2579430568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1f18072610 0x7f1f18074ad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009b90 tx=0x7f1f2c0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:51.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.389+0000 7f1f39600640 1 -- 172.21.15.94:0/2579430568 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1f2405ded0 con 0x7f1f3c070150 2024-09-10T23:32:51.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.549+0000 7f1f43a00640 1 -- 172.21.15.94:0/2579430568 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1f3c06e7e0 con 0x7f1f18072610 2024-09-10T23:32:51.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.557+0000 7f1f39600640 1 -- 172.21.15.94:0/2579430568 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251921 (secure 0 0 0) 0x7f1f3c06e7e0 con 0x7f1f18072610 2024-09-10T23:32:51.561 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:51.562 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:32:51.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.557+0000 7f1f43a00640 1 -- 172.21.15.94:0/2579430568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1f18072610 msgr2=0x7f1f18074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:51.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.557+0000 7f1f43a00640 1 --2- 172.21.15.94:0/2579430568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1f18072610 0x7f1f18074ad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009b90 tx=0x7f1f2c0091e0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.557+0000 7f1f43a00640 1 -- 172.21.15.94:0/2579430568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c070150 msgr2=0x7f1f3c163230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.557+0000 7f1f43a00640 1 --2- 172.21.15.94:0/2579430568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c070150 0x7f1f3c163230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400c420 tx=0x7f1f2400c8e0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.561+0000 7f1f43a00640 1 -- 172.21.15.94:0/2579430568 shutdown_connections 2024-09-10T23:32:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.561+0000 7f1f43a00640 1 --2- 172.21.15.94:0/2579430568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1f18072610 0x7f1f18074ad0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.561+0000 7f1f43a00640 1 --2- 172.21.15.94:0/2579430568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f3c157c60 0x7f1f3c163770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.561+0000 7f1f43a00640 1 --2- 172.21.15.94:0/2579430568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f3c070150 0x7f1f3c163230 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.561+0000 7f1f43a00640 1 --2- 172.21.15.94:0/2579430568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f3c06f600 0x7f1f3c13f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.561+0000 7f1f43a00640 1 -- 172.21.15.94:0/2579430568 >> 172.21.15.94:0/2579430568 conn(0x7f1f3c06cdf0 msgr2=0x7f1f3c06eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.561+0000 7f1f43a00640 1 -- 172.21.15.94:0/2579430568 shutdown_connections 2024-09-10T23:32:51.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.561+0000 7f1f43a00640 1 -- 172.21.15.94:0/2579430568 wait complete. 2024-09-10T23:32:51.582 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-10T23:32:51.343724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235848,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738872,"statfs":{"total":773094113280,"available":772852604928,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.795946"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627741+0000","last_change":"2024-09-10T23:32:42.593279+0000","last_active":"2024-09-10T23:32:43.627741+0000","last_peered":"2024-09-10T23:32:43.627741+0000","last_clean":"2024-09-10T23:32:43.627741+0000","last_became_active":"2024-09-10T23:32:42.593056+0000","last_became_peered":"2024-09-10T23:32:42.593056+0000","last_unstale":"2024-09-10T23:32:43.627741+0000","last_undegraded":"2024-09-10T23:32:43.627741+0000","last_fullsized":"2024-09-10T23:32:43.627741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.112551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628801+0000","last_change":"2024-09-10T23:32:42.584101+0000","last_active":"2024-09-10T23:32:43.628801+0000","last_peered":"2024-09-10T23:32:43.628801+0000","last_clean":"2024-09-10T23:32:43.628801+0000","last_became_active":"2024-09-10T23:32:42.583796+0000","last_became_peered":"2024-09-10T23:32:42.583796+0000","last_unstale":"2024-09-10T23:32:43.628801+0000","last_undegraded":"2024-09-10T23:32:43.628801+0000","last_fullsized":"2024-09-10T23:32:43.628801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:01.463027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628392+0000","last_change":"2024-09-10T23:32:42.584302+0000","last_active":"2024-09-10T23:32:43.628392+0000","last_peered":"2024-09-10T23:32:43.628392+0000","last_clean":"2024-09-10T23:32:43.628392+0000","last_became_active":"2024-09-10T23:32:42.583989+0000","last_became_peered":"2024-09-10T23:32:42.583989+0000","last_unstale":"2024-09-10T23:32:43.628392+0000","last_undegraded":"2024-09-10T23:32:43.628392+0000","last_fullsized":"2024-09-10T23:32:43.628392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:53.622106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626948+0000","last_change":"2024-09-10T23:32:42.565401+0000","last_active":"2024-09-10T23:32:43.626948+0000","last_peered":"2024-09-10T23:32:43.626948+0000","last_clean":"2024-09-10T23:32:43.626948+0000","last_became_active":"2024-09-10T23:32:42.565106+0000","last_became_peered":"2024-09-10T23:32:42.565106+0000","last_unstale":"2024-09-10T23:32:43.626948+0000","last_undegraded":"2024-09-10T23:32:43.626948+0000","last_fullsized":"2024-09-10T23:32:43.626948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:20.388860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627066+0000","last_change":"2024-09-10T23:32:42.586253+0000","last_active":"2024-09-10T23:32:43.627066+0000","last_peered":"2024-09-10T23:32:43.627066+0000","last_clean":"2024-09-10T23:32:43.627066+0000","last_became_active":"2024-09-10T23:32:42.585961+0000","last_became_peered":"2024-09-10T23:32:42.585961+0000","last_unstale":"2024-09-10T23:32:43.627066+0000","last_undegraded":"2024-09-10T23:32:43.627066+0000","last_fullsized":"2024-09-10T23:32:43.627066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:19.891408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626891+0000","last_change":"2024-09-10T23:32:42.565746+0000","last_active":"2024-09-10T23:32:43.626891+0000","last_peered":"2024-09-10T23:32:43.626891+0000","last_clean":"2024-09-10T23:32:43.626891+0000","last_became_active":"2024-09-10T23:32:42.565398+0000","last_became_peered":"2024-09-10T23:32:42.565398+0000","last_unstale":"2024-09-10T23:32:43.626891+0000","last_undegraded":"2024-09-10T23:32:43.626891+0000","last_fullsized":"2024-09-10T23:32:43.626891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:28.431860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627187+0000","last_change":"2024-09-10T23:32:42.567960+0000","last_active":"2024-09-10T23:32:43.627187+0000","last_peered":"2024-09-10T23:32:43.627187+0000","last_clean":"2024-09-10T23:32:43.627187+0000","last_became_active":"2024-09-10T23:32:42.567653+0000","last_became_peered":"2024-09-10T23:32:42.567653+0000","last_unstale":"2024-09-10T23:32:43.627187+0000","last_undegraded":"2024-09-10T23:32:43.627187+0000","last_fullsized":"2024-09-10T23:32:43.627187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:26.192704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628554+0000","last_change":"2024-09-10T23:32:42.584639+0000","last_active":"2024-09-10T23:32:43.628554+0000","last_peered":"2024-09-10T23:32:43.628554+0000","last_clean":"2024-09-10T23:32:43.628554+0000","last_became_active":"2024-09-10T23:32:42.584396+0000","last_became_peered":"2024-09-10T23:32:42.584396+0000","last_unstale":"2024-09-10T23:32:43.628554+0000","last_undegraded":"2024-09-10T23:32:43.628554+0000","last_fullsized":"2024-09-10T23:32:43.628554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:27.237846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563757+0000","last_change":"2024-09-10T23:32:42.589096+0000","last_active":"2024-09-10T23:32:43.563757+0000","last_peered":"2024-09-10T23:32:43.563757+0000","last_clean":"2024-09-10T23:32:43.563757+0000","last_became_active":"2024-09-10T23:32:42.588695+0000","last_became_peered":"2024-09-10T23:32:42.588695+0000","last_unstale":"2024-09-10T23:32:43.563757+0000","last_undegraded":"2024-09-10T23:32:43.563757+0000","last_fullsized":"2024-09-10T23:32:43.563757+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:14.778902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627628+0000","last_change":"2024-09-10T23:32:42.572168+0000","last_active":"2024-09-10T23:32:43.627628+0000","last_peered":"2024-09-10T23:32:43.627628+0000","last_clean":"2024-09-10T23:32:43.627628+0000","last_became_active":"2024-09-10T23:32:42.571638+0000","last_became_peered":"2024-09-10T23:32:42.571638+0000","last_unstale":"2024-09-10T23:32:43.627628+0000","last_undegraded":"2024-09-10T23:32:43.627628+0000","last_fullsized":"2024-09-10T23:32:43.627628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:51.184952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627526+0000","last_change":"2024-09-10T23:32:42.580539+0000","last_active":"2024-09-10T23:32:43.627526+0000","last_peered":"2024-09-10T23:32:43.627526+0000","last_clean":"2024-09-10T23:32:43.627526+0000","last_became_active":"2024-09-10T23:32:42.580180+0000","last_became_peered":"2024-09-10T23:32:42.580180+0000","last_unstale":"2024-09-10T23:32:43.627526+0000","last_undegraded":"2024-09-10T23:32:43.627526+0000","last_fullsized":"2024-09-10T23:32:43.627526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:15.490925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627548+0000","last_change":"2024-09-10T23:32:42.568100+0000","last_active":"2024-09-10T23:32:43.627548+0000","last_peered":"2024-09-10T23:32:43.627548+0000","last_clean":"2024-09-10T23:32:43.627548+0000","last_became_active":"2024-09-10T23:32:42.567409+0000","last_became_peered":"2024-09-10T23:32:42.567409+0000","last_unstale":"2024-09-10T23:32:43.627548+0000","last_undegraded":"2024-09-10T23:32:43.627548+0000","last_fullsized":"2024-09-10T23:32:43.627548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.117710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569985+0000","last_change":"2024-09-10T23:32:42.569811+0000","last_active":"2024-09-10T23:32:43.569985+0000","last_peered":"2024-09-10T23:32:43.569985+0000","last_clean":"2024-09-10T23:32:43.569985+0000","last_became_active":"2024-09-10T23:32:42.569470+0000","last_became_peered":"2024-09-10T23:32:42.569470+0000","last_unstale":"2024-09-10T23:32:43.569985+0000","last_undegraded":"2024-09-10T23:32:43.569985+0000","last_fullsized":"2024-09-10T23:32:43.569985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:56.266045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570094+0000","last_change":"2024-09-10T23:32:42.578563+0000","last_active":"2024-09-10T23:32:43.570094+0000","last_peered":"2024-09-10T23:32:43.570094+0000","last_clean":"2024-09-10T23:32:43.570094+0000","last_became_active":"2024-09-10T23:32:42.578281+0000","last_became_peered":"2024-09-10T23:32:42.578281+0000","last_unstale":"2024-09-10T23:32:43.570094+0000","last_undegraded":"2024-09-10T23:32:43.570094+0000","last_fullsized":"2024-09-10T23:32:43.570094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:57.340004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628709+0000","last_change":"2024-09-10T23:32:42.564792+0000","last_active":"2024-09-10T23:32:43.628709+0000","last_peered":"2024-09-10T23:32:43.628709+0000","last_clean":"2024-09-10T23:32:43.628709+0000","last_became_active":"2024-09-10T23:32:42.564029+0000","last_became_peered":"2024-09-10T23:32:42.564029+0000","last_unstale":"2024-09-10T23:32:43.628709+0000","last_undegraded":"2024-09-10T23:32:43.628709+0000","last_fullsized":"2024-09-10T23:32:43.628709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:23.318586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628280+0000","last_change":"2024-09-10T23:32:42.583724+0000","last_active":"2024-09-10T23:32:43.628280+0000","last_peered":"2024-09-10T23:32:43.628280+0000","last_clean":"2024-09-10T23:32:43.628280+0000","last_became_active":"2024-09-10T23:32:42.583473+0000","last_became_peered":"2024-09-10T23:32:42.583473+0000","last_unstale":"2024-09-10T23:32:43.628280+0000","last_undegraded":"2024-09-10T23:32:43.628280+0000","last_fullsized":"2024-09-10T23:32:43.628280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:14.468774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629190+0000","last_change":"2024-09-10T23:32:42.588295+0000","last_active":"2024-09-10T23:32:43.629190+0000","last_peered":"2024-09-10T23:32:43.629190+0000","last_clean":"2024-09-10T23:32:43.629190+0000","last_became_active":"2024-09-10T23:32:42.587993+0000","last_became_peered":"2024-09-10T23:32:42.587993+0000","last_unstale":"2024-09-10T23:32:43.629190+0000","last_undegraded":"2024-09-10T23:32:43.629190+0000","last_fullsized":"2024-09-10T23:32:43.629190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:18:27.504742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629147+0000","last_change":"2024-09-10T23:32:42.579982+0000","last_active":"2024-09-10T23:32:43.629147+0000","last_peered":"2024-09-10T23:32:43.629147+0000","last_clean":"2024-09-10T23:32:43.629147+0000","last_became_active":"2024-09-10T23:32:42.575814+0000","last_became_peered":"2024-09-10T23:32:42.575814+0000","last_unstale":"2024-09-10T23:32:43.629147+0000","last_undegraded":"2024-09-10T23:32:43.629147+0000","last_fullsized":"2024-09-10T23:32:43.629147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:58.672853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629002+0000","last_change":"2024-09-10T23:32:42.585466+0000","last_active":"2024-09-10T23:32:43.629002+0000","last_peered":"2024-09-10T23:32:43.629002+0000","last_clean":"2024-09-10T23:32:43.629002+0000","last_became_active":"2024-09-10T23:32:42.582650+0000","last_became_peered":"2024-09-10T23:32:42.582650+0000","last_unstale":"2024-09-10T23:32:43.629002+0000","last_undegraded":"2024-09-10T23:32:43.629002+0000","last_fullsized":"2024-09-10T23:32:43.629002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:59.434883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627679+0000","last_change":"2024-09-10T23:32:42.573723+0000","last_active":"2024-09-10T23:32:43.627679+0000","last_peered":"2024-09-10T23:32:43.627679+0000","last_clean":"2024-09-10T23:32:43.627679+0000","last_became_active":"2024-09-10T23:32:42.573309+0000","last_became_peered":"2024-09-10T23:32:42.573309+0000","last_unstale":"2024-09-10T23:32:43.627679+0000","last_undegraded":"2024-09-10T23:32:43.627679+0000","last_fullsized":"2024-09-10T23:32:43.627679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:08.762419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569445+0000","last_change":"2024-09-10T23:32:42.558045+0000","last_active":"2024-09-10T23:32:43.569445+0000","last_peered":"2024-09-10T23:32:43.569445+0000","last_clean":"2024-09-10T23:32:43.569445+0000","last_became_active":"2024-09-10T23:32:42.557774+0000","last_became_peered":"2024-09-10T23:32:42.557774+0000","last_unstale":"2024-09-10T23:32:43.569445+0000","last_undegraded":"2024-09-10T23:32:43.569445+0000","last_fullsized":"2024-09-10T23:32:43.569445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:53.622106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563214+0000","last_change":"2024-09-10T23:32:42.556890+0000","last_active":"2024-09-10T23:32:43.563214+0000","last_peered":"2024-09-10T23:32:43.563214+0000","last_clean":"2024-09-10T23:32:43.563214+0000","last_became_active":"2024-09-10T23:32:42.556625+0000","last_became_peered":"2024-09-10T23:32:42.556625+0000","last_unstale":"2024-09-10T23:32:43.563214+0000","last_undegraded":"2024-09-10T23:32:43.563214+0000","last_fullsized":"2024-09-10T23:32:43.563214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:14.440912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628804+0000","last_change":"2024-09-10T23:32:42.575698+0000","last_active":"2024-09-10T23:32:43.628804+0000","last_peered":"2024-09-10T23:32:43.628804+0000","last_clean":"2024-09-10T23:32:43.628804+0000","last_became_active":"2024-09-10T23:32:42.575371+0000","last_became_peered":"2024-09-10T23:32:42.575371+0000","last_unstale":"2024-09-10T23:32:43.628804+0000","last_undegraded":"2024-09-10T23:32:43.628804+0000","last_fullsized":"2024-09-10T23:32:43.628804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:36.598389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628388+0000","last_change":"2024-09-10T23:32:42.591870+0000","last_active":"2024-09-10T23:32:43.628388+0000","last_peered":"2024-09-10T23:32:43.628388+0000","last_clean":"2024-09-10T23:32:43.628388+0000","last_became_active":"2024-09-10T23:32:42.590207+0000","last_became_peered":"2024-09-10T23:32:42.590207+0000","last_unstale":"2024-09-10T23:32:43.628388+0000","last_undegraded":"2024-09-10T23:32:43.628388+0000","last_fullsized":"2024-09-10T23:32:43.628388+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:40.104986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629051+0000","last_change":"2024-09-10T23:32:42.588702+0000","last_active":"2024-09-10T23:32:43.629051+0000","last_peered":"2024-09-10T23:32:43.629051+0000","last_clean":"2024-09-10T23:32:43.629051+0000","last_became_active":"2024-09-10T23:32:42.588444+0000","last_became_peered":"2024-09-10T23:32:42.588444+0000","last_unstale":"2024-09-10T23:32:43.629051+0000","last_undegraded":"2024-09-10T23:32:43.629051+0000","last_fullsized":"2024-09-10T23:32:43.629051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:43.173720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627479+0000","last_change":"2024-09-10T23:32:42.580821+0000","last_active":"2024-09-10T23:32:43.627479+0000","last_peered":"2024-09-10T23:32:43.627479+0000","last_clean":"2024-09-10T23:32:43.627479+0000","last_became_active":"2024-09-10T23:32:42.580319+0000","last_became_peered":"2024-09-10T23:32:42.580319+0000","last_unstale":"2024-09-10T23:32:43.627479+0000","last_undegraded":"2024-09-10T23:32:43.627479+0000","last_fullsized":"2024-09-10T23:32:43.627479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:26.735696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570454+0000","last_change":"2024-09-10T23:32:42.574625+0000","last_active":"2024-09-10T23:32:43.570454+0000","last_peered":"2024-09-10T23:32:43.570454+0000","last_clean":"2024-09-10T23:32:43.570454+0000","last_became_active":"2024-09-10T23:32:42.571847+0000","last_became_peered":"2024-09-10T23:32:42.571847+0000","last_unstale":"2024-09-10T23:32:43.570454+0000","last_undegraded":"2024-09-10T23:32:43.570454+0000","last_fullsized":"2024-09-10T23:32:43.570454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:20.906137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570331+0000","last_change":"2024-09-10T23:32:42.567479+0000","last_active":"2024-09-10T23:32:43.570331+0000","last_peered":"2024-09-10T23:32:43.570331+0000","last_clean":"2024-09-10T23:32:43.570331+0000","last_became_active":"2024-09-10T23:32:42.567172+0000","last_became_peered":"2024-09-10T23:32:42.567172+0000","last_unstale":"2024-09-10T23:32:43.570331+0000","last_undegraded":"2024-09-10T23:32:43.570331+0000","last_fullsized":"2024-09-10T23:32:43.570331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:15.280711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628395+0000","last_change":"2024-09-10T23:32:42.585087+0000","last_active":"2024-09-10T23:32:43.628395+0000","last_peered":"2024-09-10T23:32:43.628395+0000","last_clean":"2024-09-10T23:32:43.628395+0000","last_became_active":"2024-09-10T23:32:42.583712+0000","last_became_peered":"2024-09-10T23:32:42.583712+0000","last_unstale":"2024-09-10T23:32:43.628395+0000","last_undegraded":"2024-09-10T23:32:43.628395+0000","last_fullsized":"2024-09-10T23:32:43.628395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:25.952890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628168+0000","last_change":"2024-09-10T23:32:42.591882+0000","last_active":"2024-09-10T23:32:43.628168+0000","last_peered":"2024-09-10T23:32:43.628168+0000","last_clean":"2024-09-10T23:32:43.628168+0000","last_became_active":"2024-09-10T23:32:42.590231+0000","last_became_peered":"2024-09-10T23:32:42.590231+0000","last_unstale":"2024-09-10T23:32:43.628168+0000","last_undegraded":"2024-09-10T23:32:43.628168+0000","last_fullsized":"2024-09-10T23:32:43.628168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:33.100231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628038+0000","last_change":"2024-09-10T23:32:42.573851+0000","last_active":"2024-09-10T23:32:43.628038+0000","last_peered":"2024-09-10T23:32:43.628038+0000","last_clean":"2024-09-10T23:32:43.628038+0000","last_became_active":"2024-09-10T23:32:42.572836+0000","last_became_peered":"2024-09-10T23:32:42.572836+0000","last_unstale":"2024-09-10T23:32:43.628038+0000","last_undegraded":"2024-09-10T23:32:43.628038+0000","last_fullsized":"2024-09-10T23:32:43.628038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:54.816587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627085+0000","last_change":"2024-09-10T23:32:42.576085+0000","last_active":"2024-09-10T23:32:43.627085+0000","last_peered":"2024-09-10T23:32:43.627085+0000","last_clean":"2024-09-10T23:32:43.627085+0000","last_became_active":"2024-09-10T23:32:42.575374+0000","last_became_peered":"2024-09-10T23:32:42.575374+0000","last_unstale":"2024-09-10T23:32:43.627085+0000","last_undegraded":"2024-09-10T23:32:43.627085+0000","last_fullsized":"2024-09-10T23:32:43.627085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:14.778902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563267+0000","last_change":"2024-09-10T23:32:42.588842+0000","last_active":"2024-09-10T23:32:43.563267+0000","last_peered":"2024-09-10T23:32:43.563267+0000","last_clean":"2024-09-10T23:32:43.563267+0000","last_became_active":"2024-09-10T23:32:42.588582+0000","last_became_peered":"2024-09-10T23:32:42.588582+0000","last_unstale":"2024-09-10T23:32:43.563267+0000","last_undegraded":"2024-09-10T23:32:43.563267+0000","last_fullsized":"2024-09-10T23:32:43.563267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:28.431860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628414+0000","last_change":"2024-09-10T23:32:41.569293+0000","last_active":"2024-09-10T23:32:43.628414+0000","last_peered":"2024-09-10T23:32:43.628414+0000","last_clean":"2024-09-10T23:32:43.628414+0000","last_became_active":"2024-09-10T23:32:41.568848+0000","last_became_peered":"2024-09-10T23:32:41.568848+0000","last_unstale":"2024-09-10T23:32:43.628414+0000","last_undegraded":"2024-09-10T23:32:43.628414+0000","last_fullsized":"2024-09-10T23:32:43.628414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:17.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569497+0000","last_change":"2024-09-10T23:32:42.568214+0000","last_active":"2024-09-10T23:32:43.569497+0000","last_peered":"2024-09-10T23:32:43.569497+0000","last_clean":"2024-09-10T23:32:43.569497+0000","last_became_active":"2024-09-10T23:32:42.562093+0000","last_became_peered":"2024-09-10T23:32:42.562093+0000","last_unstale":"2024-09-10T23:32:43.569497+0000","last_undegraded":"2024-09-10T23:32:43.569497+0000","last_fullsized":"2024-09-10T23:32:43.569497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:02.454545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628550+0000","last_change":"2024-09-10T23:32:41.578635+0000","last_active":"2024-09-10T23:32:43.628550+0000","last_peered":"2024-09-10T23:32:43.628550+0000","last_clean":"2024-09-10T23:32:43.628550+0000","last_became_active":"2024-09-10T23:32:41.578346+0000","last_became_peered":"2024-09-10T23:32:41.578346+0000","last_unstale":"2024-09-10T23:32:43.628550+0000","last_undegraded":"2024-09-10T23:32:43.628550+0000","last_fullsized":"2024-09-10T23:32:43.628550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:25.728667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569793+0000","last_change":"2024-09-10T23:32:41.553674+0000","last_active":"2024-09-10T23:32:43.569793+0000","last_peered":"2024-09-10T23:32:43.569793+0000","last_clean":"2024-09-10T23:32:43.569793+0000","last_became_active":"2024-09-10T23:32:41.553244+0000","last_became_peered":"2024-09-10T23:32:41.553244+0000","last_unstale":"2024-09-10T23:32:43.569793+0000","last_undegraded":"2024-09-10T23:32:43.569793+0000","last_fullsized":"2024-09-10T23:32:43.569793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:38.098241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628828+0000","last_change":"2024-09-10T23:32:42.583587+0000","last_active":"2024-09-10T23:32:43.628828+0000","last_peered":"2024-09-10T23:32:43.628828+0000","last_clean":"2024-09-10T23:32:43.628828+0000","last_became_active":"2024-09-10T23:32:42.581831+0000","last_became_peered":"2024-09-10T23:32:42.581831+0000","last_unstale":"2024-09-10T23:32:43.628828+0000","last_undegraded":"2024-09-10T23:32:43.628828+0000","last_fullsized":"2024-09-10T23:32:43.628828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:23.493444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628532+0000","last_change":"2024-09-10T23:32:41.581810+0000","last_active":"2024-09-10T23:32:43.628532+0000","last_peered":"2024-09-10T23:32:43.628532+0000","last_clean":"2024-09-10T23:32:43.628532+0000","last_became_active":"2024-09-10T23:32:41.581531+0000","last_became_peered":"2024-09-10T23:32:41.581531+0000","last_unstale":"2024-09-10T23:32:43.628532+0000","last_undegraded":"2024-09-10T23:32:43.628532+0000","last_fullsized":"2024-09-10T23:32:43.628532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:57.665825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569188+0000","last_change":"2024-09-10T23:32:42.560663+0000","last_active":"2024-09-10T23:32:43.569188+0000","last_peered":"2024-09-10T23:32:43.569188+0000","last_clean":"2024-09-10T23:32:43.569188+0000","last_became_active":"2024-09-10T23:32:42.560293+0000","last_became_peered":"2024-09-10T23:32:42.560293+0000","last_unstale":"2024-09-10T23:32:43.569188+0000","last_undegraded":"2024-09-10T23:32:43.569188+0000","last_fullsized":"2024-09-10T23:32:43.569188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:39.387246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563757+0000","last_change":"2024-09-10T23:32:41.573958+0000","last_active":"2024-09-10T23:32:43.563757+0000","last_peered":"2024-09-10T23:32:43.563757+0000","last_clean":"2024-09-10T23:32:43.563757+0000","last_became_active":"2024-09-10T23:32:41.573639+0000","last_became_peered":"2024-09-10T23:32:41.573639+0000","last_unstale":"2024-09-10T23:32:43.563757+0000","last_undegraded":"2024-09-10T23:32:43.563757+0000","last_fullsized":"2024-09-10T23:32:43.563757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:27.548086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627506+0000","last_change":"2024-09-10T23:32:42.580586+0000","last_active":"2024-09-10T23:32:43.627506+0000","last_peered":"2024-09-10T23:32:43.627506+0000","last_clean":"2024-09-10T23:32:43.627506+0000","last_became_active":"2024-09-10T23:32:42.579960+0000","last_became_peered":"2024-09-10T23:32:42.579960+0000","last_unstale":"2024-09-10T23:32:43.627506+0000","last_undegraded":"2024-09-10T23:32:43.627506+0000","last_fullsized":"2024-09-10T23:32:43.627506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:09.640521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627910+0000","last_change":"2024-09-10T23:32:41.570662+0000","last_active":"2024-09-10T23:32:43.627910+0000","last_peered":"2024-09-10T23:32:43.627910+0000","last_clean":"2024-09-10T23:32:43.627910+0000","last_became_active":"2024-09-10T23:32:41.570330+0000","last_became_peered":"2024-09-10T23:32:41.570330+0000","last_unstale":"2024-09-10T23:32:43.627910+0000","last_undegraded":"2024-09-10T23:32:43.627910+0000","last_fullsized":"2024-09-10T23:32:43.627910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:53.809559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629328+0000","last_change":"2024-09-10T23:32:42.591662+0000","last_active":"2024-09-10T23:32:43.629328+0000","last_peered":"2024-09-10T23:32:43.629328+0000","last_clean":"2024-09-10T23:32:43.629328+0000","last_became_active":"2024-09-10T23:32:42.591315+0000","last_became_peered":"2024-09-10T23:32:42.591315+0000","last_unstale":"2024-09-10T23:32:43.629328+0000","last_undegraded":"2024-09-10T23:32:43.629328+0000","last_fullsized":"2024-09-10T23:32:43.629328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:34.259468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628947+0000","last_change":"2024-09-10T23:32:41.571940+0000","last_active":"2024-09-10T23:32:43.628947+0000","last_peered":"2024-09-10T23:32:43.628947+0000","last_clean":"2024-09-10T23:32:43.628947+0000","last_became_active":"2024-09-10T23:32:41.571537+0000","last_became_peered":"2024-09-10T23:32:41.571537+0000","last_unstale":"2024-09-10T23:32:43.628947+0000","last_undegraded":"2024-09-10T23:32:43.628947+0000","last_fullsized":"2024-09-10T23:32:43.628947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:49.898494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629065+0000","last_change":"2024-09-10T23:32:42.585332+0000","last_active":"2024-09-10T23:32:43.629065+0000","last_peered":"2024-09-10T23:32:43.629065+0000","last_clean":"2024-09-10T23:32:43.629065+0000","last_became_active":"2024-09-10T23:32:42.585046+0000","last_became_peered":"2024-09-10T23:32:42.585046+0000","last_unstale":"2024-09-10T23:32:43.629065+0000","last_undegraded":"2024-09-10T23:32:43.629065+0000","last_fullsized":"2024-09-10T23:32:43.629065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:24.681728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627051+0000","last_change":"2024-09-10T23:32:41.565270+0000","last_active":"2024-09-10T23:32:43.627051+0000","last_peered":"2024-09-10T23:32:43.627051+0000","last_clean":"2024-09-10T23:32:43.627051+0000","last_became_active":"2024-09-10T23:32:41.564864+0000","last_became_peered":"2024-09-10T23:32:41.564864+0000","last_unstale":"2024-09-10T23:32:43.627051+0000","last_undegraded":"2024-09-10T23:32:43.627051+0000","last_fullsized":"2024-09-10T23:32:43.627051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:55.242033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627514+0000","last_change":"2024-09-10T23:32:42.576998+0000","last_active":"2024-09-10T23:32:43.627514+0000","last_peered":"2024-09-10T23:32:43.627514+0000","last_clean":"2024-09-10T23:32:43.627514+0000","last_became_active":"2024-09-10T23:32:42.576695+0000","last_became_peered":"2024-09-10T23:32:42.576695+0000","last_unstale":"2024-09-10T23:32:43.627514+0000","last_undegraded":"2024-09-10T23:32:43.627514+0000","last_fullsized":"2024-09-10T23:32:43.627514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:09.931603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628218+0000","last_change":"2024-09-10T23:32:41.571048+0000","last_active":"2024-09-10T23:32:43.628218+0000","last_peered":"2024-09-10T23:32:43.628218+0000","last_clean":"2024-09-10T23:32:43.628218+0000","last_became_active":"2024-09-10T23:32:41.570753+0000","last_became_peered":"2024-09-10T23:32:41.570753+0000","last_unstale":"2024-09-10T23:32:43.628218+0000","last_undegraded":"2024-09-10T23:32:43.628218+0000","last_fullsized":"2024-09-10T23:32:43.628218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:55.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570370+0000","last_change":"2024-09-10T23:32:42.564300+0000","last_active":"2024-09-10T23:32:43.570370+0000","last_peered":"2024-09-10T23:32:43.570370+0000","last_clean":"2024-09-10T23:32:43.570370+0000","last_became_active":"2024-09-10T23:32:42.564047+0000","last_became_peered":"2024-09-10T23:32:42.564047+0000","last_unstale":"2024-09-10T23:32:43.570370+0000","last_undegraded":"2024-09-10T23:32:43.570370+0000","last_fullsized":"2024-09-10T23:32:43.570370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:22.724821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628645+0000","last_change":"2024-09-10T23:32:41.585012+0000","last_active":"2024-09-10T23:32:43.628645+0000","last_peered":"2024-09-10T23:32:43.628645+0000","last_clean":"2024-09-10T23:32:43.628645+0000","last_became_active":"2024-09-10T23:32:41.584732+0000","last_became_peered":"2024-09-10T23:32:41.584732+0000","last_unstale":"2024-09-10T23:32:43.628645+0000","last_undegraded":"2024-09-10T23:32:43.628645+0000","last_fullsized":"2024-09-10T23:32:43.628645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:39.041511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628794+0000","last_change":"2024-09-10T23:32:42.569000+0000","last_active":"2024-09-10T23:32:43.628794+0000","last_peered":"2024-09-10T23:32:43.628794+0000","last_clean":"2024-09-10T23:32:43.628794+0000","last_became_active":"2024-09-10T23:32:42.568721+0000","last_became_peered":"2024-09-10T23:32:42.568721+0000","last_unstale":"2024-09-10T23:32:43.628794+0000","last_undegraded":"2024-09-10T23:32:43.628794+0000","last_fullsized":"2024-09-10T23:32:43.628794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:37.891368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626833+0000","last_change":"2024-09-10T23:32:41.578661+0000","last_active":"2024-09-10T23:32:43.626833+0000","last_peered":"2024-09-10T23:32:43.626833+0000","last_clean":"2024-09-10T23:32:43.626833+0000","last_became_active":"2024-09-10T23:32:41.578343+0000","last_became_peered":"2024-09-10T23:32:41.578343+0000","last_unstale":"2024-09-10T23:32:43.626833+0000","last_undegraded":"2024-09-10T23:32:43.626833+0000","last_fullsized":"2024-09-10T23:32:43.626833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:47.815432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563134+0000","last_change":"2024-09-10T23:32:42.584934+0000","last_active":"2024-09-10T23:32:43.563134+0000","last_peered":"2024-09-10T23:32:43.563134+0000","last_clean":"2024-09-10T23:32:43.563134+0000","last_became_active":"2024-09-10T23:32:42.584709+0000","last_became_peered":"2024-09-10T23:32:42.584709+0000","last_unstale":"2024-09-10T23:32:43.563134+0000","last_undegraded":"2024-09-10T23:32:43.563134+0000","last_fullsized":"2024-09-10T23:32:43.563134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:30.543243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.568880+0000","last_change":"2024-09-10T23:32:41.566144+0000","last_active":"2024-09-10T23:32:43.568880+0000","last_peered":"2024-09-10T23:32:43.568880+0000","last_clean":"2024-09-10T23:32:43.568880+0000","last_became_active":"2024-09-10T23:32:41.565841+0000","last_became_peered":"2024-09-10T23:32:41.565841+0000","last_unstale":"2024-09-10T23:32:43.568880+0000","last_undegraded":"2024-09-10T23:32:43.568880+0000","last_fullsized":"2024-09-10T23:32:43.568880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:51.080022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628125+0000","last_change":"2024-09-10T23:32:42.586470+0000","last_active":"2024-09-10T23:32:43.628125+0000","last_peered":"2024-09-10T23:32:43.628125+0000","last_clean":"2024-09-10T23:32:43.628125+0000","last_became_active":"2024-09-10T23:32:42.579377+0000","last_became_peered":"2024-09-10T23:32:42.579377+0000","last_unstale":"2024-09-10T23:32:43.628125+0000","last_undegraded":"2024-09-10T23:32:43.628125+0000","last_fullsized":"2024-09-10T23:32:43.628125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.112551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628061+0000","last_change":"2024-09-10T23:32:41.567212+0000","last_active":"2024-09-10T23:32:43.628061+0000","last_peered":"2024-09-10T23:32:43.628061+0000","last_clean":"2024-09-10T23:32:43.628061+0000","last_became_active":"2024-09-10T23:32:41.566857+0000","last_became_peered":"2024-09-10T23:32:41.566857+0000","last_unstale":"2024-09-10T23:32:43.628061+0000","last_undegraded":"2024-09-10T23:32:43.628061+0000","last_fullsized":"2024-09-10T23:32:43.628061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:05.259433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627653+0000","last_change":"2024-09-10T23:32:42.558295+0000","last_active":"2024-09-10T23:32:43.627653+0000","last_peered":"2024-09-10T23:32:43.627653+0000","last_clean":"2024-09-10T23:32:43.627653+0000","last_became_active":"2024-09-10T23:32:42.558039+0000","last_became_peered":"2024-09-10T23:32:42.558039+0000","last_unstale":"2024-09-10T23:32:43.627653+0000","last_undegraded":"2024-09-10T23:32:43.627653+0000","last_fullsized":"2024-09-10T23:32:43.627653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:07.033834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569932+0000","last_change":"2024-09-10T23:32:41.568016+0000","last_active":"2024-09-10T23:32:43.569932+0000","last_peered":"2024-09-10T23:32:43.569932+0000","last_clean":"2024-09-10T23:32:43.569932+0000","last_became_active":"2024-09-10T23:32:41.566960+0000","last_became_peered":"2024-09-10T23:32:41.566960+0000","last_unstale":"2024-09-10T23:32:43.569932+0000","last_undegraded":"2024-09-10T23:32:43.569932+0000","last_fullsized":"2024-09-10T23:32:43.569932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:14.145496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628571+0000","last_change":"2024-09-10T23:32:42.562392+0000","last_active":"2024-09-10T23:32:43.628571+0000","last_peered":"2024-09-10T23:32:43.628571+0000","last_clean":"2024-09-10T23:32:43.628571+0000","last_became_active":"2024-09-10T23:32:42.562042+0000","last_became_peered":"2024-09-10T23:32:42.562042+0000","last_unstale":"2024-09-10T23:32:43.628571+0000","last_undegraded":"2024-09-10T23:32:43.628571+0000","last_fullsized":"2024-09-10T23:32:43.628571+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:33.100231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628521+0000","last_change":"2024-09-10T23:32:41.572986+0000","last_active":"2024-09-10T23:32:43.628521+0000","last_peered":"2024-09-10T23:32:43.628521+0000","last_clean":"2024-09-10T23:32:43.628521+0000","last_became_active":"2024-09-10T23:32:41.572722+0000","last_became_peered":"2024-09-10T23:32:41.572722+0000","last_unstale":"2024-09-10T23:32:43.628521+0000","last_undegraded":"2024-09-10T23:32:43.628521+0000","last_fullsized":"2024-09-10T23:32:43.628521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:49.898494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628742+0000","last_change":"2024-09-10T23:32:42.573643+0000","last_active":"2024-09-10T23:32:43.628742+0000","last_peered":"2024-09-10T23:32:43.628742+0000","last_clean":"2024-09-10T23:32:43.628742+0000","last_became_active":"2024-09-10T23:32:42.572873+0000","last_became_peered":"2024-09-10T23:32:42.572873+0000","last_unstale":"2024-09-10T23:32:43.628742+0000","last_undegraded":"2024-09-10T23:32:43.628742+0000","last_fullsized":"2024-09-10T23:32:43.628742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:14.468774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570459+0000","last_change":"2024-09-10T23:32:41.562365+0000","last_active":"2024-09-10T23:32:43.570459+0000","last_peered":"2024-09-10T23:32:43.570459+0000","last_clean":"2024-09-10T23:32:43.570459+0000","last_became_active":"2024-09-10T23:32:41.562021+0000","last_became_peered":"2024-09-10T23:32:41.562021+0000","last_unstale":"2024-09-10T23:32:43.570459+0000","last_undegraded":"2024-09-10T23:32:43.570459+0000","last_fullsized":"2024-09-10T23:32:43.570459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:08:28.878205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628732+0000","last_change":"2024-09-10T23:32:42.584713+0000","last_active":"2024-09-10T23:32:43.628732+0000","last_peered":"2024-09-10T23:32:43.628732+0000","last_clean":"2024-09-10T23:32:43.628732+0000","last_became_active":"2024-09-10T23:32:42.577789+0000","last_became_peered":"2024-09-10T23:32:42.577789+0000","last_unstale":"2024-09-10T23:32:43.628732+0000","last_undegraded":"2024-09-10T23:32:43.628732+0000","last_fullsized":"2024-09-10T23:32:43.628732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:33.835681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563126+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:32:43.563126+0000","last_peered":"2024-09-10T23:32:43.563126+0000","last_clean":"2024-09-10T23:32:43.563126+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:32:43.563126+0000","last_undegraded":"2024-09-10T23:32:43.563126+0000","last_fullsized":"2024-09-10T23:32:43.563126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629078+0000","last_change":"2024-09-10T23:32:41.581530+0000","last_active":"2024-09-10T23:32:43.629078+0000","last_peered":"2024-09-10T23:32:43.629078+0000","last_clean":"2024-09-10T23:32:43.629078+0000","last_became_active":"2024-09-10T23:32:41.581242+0000","last_became_peered":"2024-09-10T23:32:41.581242+0000","last_unstale":"2024-09-10T23:32:43.629078+0000","last_undegraded":"2024-09-10T23:32:43.629078+0000","last_fullsized":"2024-09-10T23:32:43.629078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:08.633493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569220+0000","last_change":"2024-09-10T23:32:42.558841+0000","last_active":"2024-09-10T23:32:43.569220+0000","last_peered":"2024-09-10T23:32:43.569220+0000","last_clean":"2024-09-10T23:32:43.569220+0000","last_became_active":"2024-09-10T23:32:42.558580+0000","last_became_peered":"2024-09-10T23:32:42.558580+0000","last_unstale":"2024-09-10T23:32:43.569220+0000","last_undegraded":"2024-09-10T23:32:43.569220+0000","last_fullsized":"2024-09-10T23:32:43.569220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:54.620648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628625+0000","last_change":"2024-09-10T23:32:41.577467+0000","last_active":"2024-09-10T23:32:43.628625+0000","last_peered":"2024-09-10T23:32:43.628625+0000","last_clean":"2024-09-10T23:32:43.628625+0000","last_became_active":"2024-09-10T23:32:41.576702+0000","last_became_peered":"2024-09-10T23:32:41.576702+0000","last_unstale":"2024-09-10T23:32:43.628625+0000","last_undegraded":"2024-09-10T23:32:43.628625+0000","last_fullsized":"2024-09-10T23:32:43.628625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:32.527218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569670+0000","last_change":"2024-09-10T23:32:42.565100+0000","last_active":"2024-09-10T23:32:43.569670+0000","last_peered":"2024-09-10T23:32:43.569670+0000","last_clean":"2024-09-10T23:32:43.569670+0000","last_became_active":"2024-09-10T23:32:42.564790+0000","last_became_peered":"2024-09-10T23:32:42.564790+0000","last_unstale":"2024-09-10T23:32:43.569670+0000","last_undegraded":"2024-09-10T23:32:43.569670+0000","last_fullsized":"2024-09-10T23:32:43.569670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:44.555042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628065+0000","last_change":"2024-09-10T23:32:41.571266+0000","last_active":"2024-09-10T23:32:43.628065+0000","last_peered":"2024-09-10T23:32:43.628065+0000","last_clean":"2024-09-10T23:32:43.628065+0000","last_became_active":"2024-09-10T23:32:41.570986+0000","last_became_peered":"2024-09-10T23:32:41.570986+0000","last_unstale":"2024-09-10T23:32:43.628065+0000","last_undegraded":"2024-09-10T23:32:43.628065+0000","last_fullsized":"2024-09-10T23:32:43.628065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:58.170892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628975+0000","last_change":"2024-09-10T23:32:42.584295+0000","last_active":"2024-09-10T23:32:43.628975+0000","last_peered":"2024-09-10T23:32:43.628975+0000","last_clean":"2024-09-10T23:32:43.628975+0000","last_became_active":"2024-09-10T23:32:42.584001+0000","last_became_peered":"2024-09-10T23:32:42.584001+0000","last_unstale":"2024-09-10T23:32:43.628975+0000","last_undegraded":"2024-09-10T23:32:43.628975+0000","last_fullsized":"2024-09-10T23:32:43.628975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:05.044760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628726+0000","last_change":"2024-09-10T23:32:41.578157+0000","last_active":"2024-09-10T23:32:43.628726+0000","last_peered":"2024-09-10T23:32:43.628726+0000","last_clean":"2024-09-10T23:32:43.628726+0000","last_became_active":"2024-09-10T23:32:41.575442+0000","last_became_peered":"2024-09-10T23:32:41.575442+0000","last_unstale":"2024-09-10T23:32:43.628726+0000","last_undegraded":"2024-09-10T23:32:43.628726+0000","last_fullsized":"2024-09-10T23:32:43.628726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:00.455999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627197+0000","last_change":"2024-09-10T23:32:42.571239+0000","last_active":"2024-09-10T23:32:43.627197+0000","last_peered":"2024-09-10T23:32:43.627197+0000","last_clean":"2024-09-10T23:32:43.627197+0000","last_became_active":"2024-09-10T23:32:42.570963+0000","last_became_peered":"2024-09-10T23:32:42.570963+0000","last_unstale":"2024-09-10T23:32:43.627197+0000","last_undegraded":"2024-09-10T23:32:43.627197+0000","last_fullsized":"2024-09-10T23:32:43.627197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:36.598389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628178+0000","last_change":"2024-09-10T23:32:41.569326+0000","last_active":"2024-09-10T23:32:43.628178+0000","last_peered":"2024-09-10T23:32:43.628178+0000","last_clean":"2024-09-10T23:32:43.628178+0000","last_became_active":"2024-09-10T23:32:41.569056+0000","last_became_peered":"2024-09-10T23:32:41.569056+0000","last_unstale":"2024-09-10T23:32:43.628178+0000","last_undegraded":"2024-09-10T23:32:43.628178+0000","last_fullsized":"2024-09-10T23:32:43.628178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:47.815432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626872+0000","last_change":"2024-09-10T23:32:42.574800+0000","last_active":"2024-09-10T23:32:43.626872+0000","last_peered":"2024-09-10T23:32:43.626872+0000","last_clean":"2024-09-10T23:32:43.626872+0000","last_became_active":"2024-09-10T23:32:42.574477+0000","last_became_peered":"2024-09-10T23:32:42.574477+0000","last_unstale":"2024-09-10T23:32:43.626872+0000","last_undegraded":"2024-09-10T23:32:43.626872+0000","last_fullsized":"2024-09-10T23:32:43.626872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:14.440912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629156+0000","last_change":"2024-09-10T23:32:41.582865+0000","last_active":"2024-09-10T23:32:43.629156+0000","last_peered":"2024-09-10T23:32:43.629156+0000","last_clean":"2024-09-10T23:32:43.629156+0000","last_became_active":"2024-09-10T23:32:41.582522+0000","last_became_peered":"2024-09-10T23:32:41.582522+0000","last_unstale":"2024-09-10T23:32:43.629156+0000","last_undegraded":"2024-09-10T23:32:43.629156+0000","last_fullsized":"2024-09-10T23:32:43.629156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:24.945862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570471+0000","last_change":"2024-09-10T23:32:42.587148+0000","last_active":"2024-09-10T23:32:43.570471+0000","last_peered":"2024-09-10T23:32:43.570471+0000","last_clean":"2024-09-10T23:32:43.570471+0000","last_became_active":"2024-09-10T23:32:42.586887+0000","last_became_peered":"2024-09-10T23:32:42.586887+0000","last_unstale":"2024-09-10T23:32:43.570471+0000","last_undegraded":"2024-09-10T23:32:43.570471+0000","last_fullsized":"2024-09-10T23:32:43.570471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:27.999337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570022+0000","last_change":"2024-09-10T23:32:41.553400+0000","last_active":"2024-09-10T23:32:43.570022+0000","last_peered":"2024-09-10T23:32:43.570022+0000","last_clean":"2024-09-10T23:32:43.570022+0000","last_became_active":"2024-09-10T23:32:41.553070+0000","last_became_peered":"2024-09-10T23:32:41.553070+0000","last_unstale":"2024-09-10T23:32:43.570022+0000","last_undegraded":"2024-09-10T23:32:43.570022+0000","last_fullsized":"2024-09-10T23:32:43.570022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:01.393799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569840+0000","last_change":"2024-09-10T23:32:42.580714+0000","last_active":"2024-09-10T23:32:43.569840+0000","last_peered":"2024-09-10T23:32:43.569840+0000","last_clean":"2024-09-10T23:32:43.569840+0000","last_became_active":"2024-09-10T23:32:42.579027+0000","last_became_peered":"2024-09-10T23:32:42.579027+0000","last_unstale":"2024-09-10T23:32:43.569840+0000","last_undegraded":"2024-09-10T23:32:43.569840+0000","last_fullsized":"2024-09-10T23:32:43.569840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:12.456589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629216+0000","last_change":"2024-09-10T23:32:41.580054+0000","last_active":"2024-09-10T23:32:43.629216+0000","last_peered":"2024-09-10T23:32:43.629216+0000","last_clean":"2024-09-10T23:32:43.629216+0000","last_became_active":"2024-09-10T23:32:41.579663+0000","last_became_peered":"2024-09-10T23:32:41.579663+0000","last_unstale":"2024-09-10T23:32:43.629216+0000","last_undegraded":"2024-09-10T23:32:43.629216+0000","last_fullsized":"2024-09-10T23:32:43.629216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:22.486416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569890+0000","last_change":"2024-09-10T23:32:42.574503+0000","last_active":"2024-09-10T23:32:43.569890+0000","last_peered":"2024-09-10T23:32:43.569890+0000","last_clean":"2024-09-10T23:32:43.569890+0000","last_became_active":"2024-09-10T23:32:42.574229+0000","last_became_peered":"2024-09-10T23:32:42.574229+0000","last_unstale":"2024-09-10T23:32:43.569890+0000","last_undegraded":"2024-09-10T23:32:43.569890+0000","last_fullsized":"2024-09-10T23:32:43.569890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:19.891408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626574+0000","last_change":"2024-09-10T23:32:41.562000+0000","last_active":"2024-09-10T23:32:43.626574+0000","last_peered":"2024-09-10T23:32:43.626574+0000","last_clean":"2024-09-10T23:32:43.626574+0000","last_became_active":"2024-09-10T23:32:41.561637+0000","last_became_peered":"2024-09-10T23:32:41.561637+0000","last_unstale":"2024-09-10T23:32:43.626574+0000","last_undegraded":"2024-09-10T23:32:43.626574+0000","last_fullsized":"2024-09-10T23:32:43.626574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:17.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628329+0000","last_change":"2024-09-10T23:32:42.583205+0000","last_active":"2024-09-10T23:32:43.628329+0000","last_peered":"2024-09-10T23:32:43.628329+0000","last_clean":"2024-09-10T23:32:43.628329+0000","last_became_active":"2024-09-10T23:32:42.582963+0000","last_became_peered":"2024-09-10T23:32:42.582963+0000","last_unstale":"2024-09-10T23:32:43.628329+0000","last_undegraded":"2024-09-10T23:32:43.628329+0000","last_fullsized":"2024-09-10T23:32:43.628329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.117710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569104+0000","last_change":"2024-09-10T23:32:41.557934+0000","last_active":"2024-09-10T23:32:43.569104+0000","last_peered":"2024-09-10T23:32:43.569104+0000","last_clean":"2024-09-10T23:32:43.569104+0000","last_became_active":"2024-09-10T23:32:41.557625+0000","last_became_peered":"2024-09-10T23:32:41.557625+0000","last_unstale":"2024-09-10T23:32:43.569104+0000","last_undegraded":"2024-09-10T23:32:43.569104+0000","last_fullsized":"2024-09-10T23:32:43.569104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:29.536215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569080+0000","last_change":"2024-09-10T23:32:42.567250+0000","last_active":"2024-09-10T23:32:43.569080+0000","last_peered":"2024-09-10T23:32:43.569080+0000","last_clean":"2024-09-10T23:32:43.569080+0000","last_became_active":"2024-09-10T23:32:42.566248+0000","last_became_peered":"2024-09-10T23:32:42.566248+0000","last_unstale":"2024-09-10T23:32:43.569080+0000","last_undegraded":"2024-09-10T23:32:43.569080+0000","last_fullsized":"2024-09-10T23:32:43.569080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:45.824956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570495+0000","last_change":"2024-09-10T23:32:41.566640+0000","last_active":"2024-09-10T23:32:43.570495+0000","last_peered":"2024-09-10T23:32:43.570495+0000","last_clean":"2024-09-10T23:32:43.570495+0000","last_became_active":"2024-09-10T23:32:41.566311+0000","last_became_peered":"2024-09-10T23:32:41.566311+0000","last_unstale":"2024-09-10T23:32:43.570495+0000","last_undegraded":"2024-09-10T23:32:43.570495+0000","last_fullsized":"2024-09-10T23:32:43.570495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:33.372580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628173+0000","last_change":"2024-09-10T23:32:42.580760+0000","last_active":"2024-09-10T23:32:43.628173+0000","last_peered":"2024-09-10T23:32:43.628173+0000","last_clean":"2024-09-10T23:32:43.628173+0000","last_became_active":"2024-09-10T23:32:42.580081+0000","last_became_peered":"2024-09-10T23:32:42.580081+0000","last_unstale":"2024-09-10T23:32:43.628173+0000","last_undegraded":"2024-09-10T23:32:43.628173+0000","last_fullsized":"2024-09-10T23:32:43.628173+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:06.266461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628952+0000","last_change":"2024-09-10T23:32:41.583976+0000","last_active":"2024-09-10T23:32:43.628952+0000","last_peered":"2024-09-10T23:32:43.628952+0000","last_clean":"2024-09-10T23:32:43.628952+0000","last_became_active":"2024-09-10T23:32:41.583639+0000","last_became_peered":"2024-09-10T23:32:41.583639+0000","last_unstale":"2024-09-10T23:32:43.628952+0000","last_undegraded":"2024-09-10T23:32:43.628952+0000","last_fullsized":"2024-09-10T23:32:43.628952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:26.669949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629220+0000","last_change":"2024-09-10T23:32:42.570056+0000","last_active":"2024-09-10T23:32:43.629220+0000","last_peered":"2024-09-10T23:32:43.629220+0000","last_clean":"2024-09-10T23:32:43.629220+0000","last_became_active":"2024-09-10T23:32:42.569794+0000","last_became_peered":"2024-09-10T23:32:42.569794+0000","last_unstale":"2024-09-10T23:32:43.629220+0000","last_undegraded":"2024-09-10T23:32:43.629220+0000","last_fullsized":"2024-09-10T23:32:43.629220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:36.453126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628526+0000","last_change":"2024-09-10T23:32:42.585151+0000","last_active":"2024-09-10T23:32:43.628526+0000","last_peered":"2024-09-10T23:32:43.628526+0000","last_clean":"2024-09-10T23:32:43.628526+0000","last_became_active":"2024-09-10T23:32:42.578092+0000","last_became_peered":"2024-09-10T23:32:42.578092+0000","last_unstale":"2024-09-10T23:32:43.628526+0000","last_undegraded":"2024-09-10T23:32:43.628526+0000","last_fullsized":"2024-09-10T23:32:43.628526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:39:56.735321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628891+0000","last_change":"2024-09-10T23:32:41.571659+0000","last_active":"2024-09-10T23:32:43.628891+0000","last_peered":"2024-09-10T23:32:43.628891+0000","last_clean":"2024-09-10T23:32:43.628891+0000","last_became_active":"2024-09-10T23:32:41.571418+0000","last_became_peered":"2024-09-10T23:32:41.571418+0000","last_unstale":"2024-09-10T23:32:43.628891+0000","last_undegraded":"2024-09-10T23:32:43.628891+0000","last_fullsized":"2024-09-10T23:32:43.628891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:57.665825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563423+0000","last_change":"2024-09-10T23:32:42.563064+0000","last_active":"2024-09-10T23:32:43.563423+0000","last_peered":"2024-09-10T23:32:43.563423+0000","last_clean":"2024-09-10T23:32:43.563423+0000","last_became_active":"2024-09-10T23:32:42.562808+0000","last_became_peered":"2024-09-10T23:32:42.562808+0000","last_unstale":"2024-09-10T23:32:43.563423+0000","last_undegraded":"2024-09-10T23:32:43.563423+0000","last_fullsized":"2024-09-10T23:32:43.563423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:20.388860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627661+0000","last_change":"2024-09-10T23:32:41.570670+0000","last_active":"2024-09-10T23:32:43.627661+0000","last_peered":"2024-09-10T23:32:43.627661+0000","last_clean":"2024-09-10T23:32:43.627661+0000","last_became_active":"2024-09-10T23:32:41.570291+0000","last_became_peered":"2024-09-10T23:32:41.570291+0000","last_unstale":"2024-09-10T23:32:43.627661+0000","last_undegraded":"2024-09-10T23:32:43.627661+0000","last_fullsized":"2024-09-10T23:32:43.627661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:33.183319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629201+0000","last_change":"2024-09-10T23:32:42.584736+0000","last_active":"2024-09-10T23:32:43.629201+0000","last_peered":"2024-09-10T23:32:43.629201+0000","last_clean":"2024-09-10T23:32:43.629201+0000","last_became_active":"2024-09-10T23:32:42.583535+0000","last_became_peered":"2024-09-10T23:32:42.583535+0000","last_unstale":"2024-09-10T23:32:43.629201+0000","last_undegraded":"2024-09-10T23:32:43.629201+0000","last_fullsized":"2024-09-10T23:32:43.629201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:00.689816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629197+0000","last_change":"2024-09-10T23:32:41.566445+0000","last_active":"2024-09-10T23:32:43.629197+0000","last_peered":"2024-09-10T23:32:43.629197+0000","last_clean":"2024-09-10T23:32:43.629197+0000","last_became_active":"2024-09-10T23:32:41.566158+0000","last_became_peered":"2024-09-10T23:32:41.566158+0000","last_unstale":"2024-09-10T23:32:43.629197+0000","last_undegraded":"2024-09-10T23:32:43.629197+0000","last_fullsized":"2024-09-10T23:32:43.629197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:55.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629299+0000","last_change":"2024-09-10T23:32:42.584147+0000","last_active":"2024-09-10T23:32:43.629299+0000","last_peered":"2024-09-10T23:32:43.629299+0000","last_clean":"2024-09-10T23:32:43.629299+0000","last_became_active":"2024-09-10T23:32:42.583766+0000","last_became_peered":"2024-09-10T23:32:42.583766+0000","last_unstale":"2024-09-10T23:32:43.629299+0000","last_undegraded":"2024-09-10T23:32:43.629299+0000","last_fullsized":"2024-09-10T23:32:43.629299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:19.288688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569913+0000","last_change":"2024-09-10T23:32:41.580965+0000","last_active":"2024-09-10T23:32:43.569913+0000","last_peered":"2024-09-10T23:32:43.569913+0000","last_clean":"2024-09-10T23:32:43.569913+0000","last_became_active":"2024-09-10T23:32:41.580712+0000","last_became_peered":"2024-09-10T23:32:41.580712+0000","last_unstale":"2024-09-10T23:32:43.569913+0000","last_undegraded":"2024-09-10T23:32:43.569913+0000","last_fullsized":"2024-09-10T23:32:43.569913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:13.433884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:32:51.585 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T23:32:51.586 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-10T23:32:51.586 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-10T23:32:51.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.765+0000 7fc014e00640 1 -- 172.21.15.94:0/2616508170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff4003a90 msgr2=0x7fbff4003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:51.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.765+0000 7fc014e00640 1 --2- 172.21.15.94:0/2616508170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff4003a90 0x7fbff4003eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc0000016f0 tx=0x7fc00001c7b0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.765+0000 7fc00e800640 1 -- 172.21.15.94:0/2616508170 reap_dead start 2024-09-10T23:32:51.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.765+0000 7fc014e00640 1 -- 172.21.15.94:0/2616508170 shutdown_connections 2024-09-10T23:32:51.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.765+0000 7fc014e00640 1 -- 172.21.15.94:0/2616508170 >> 172.21.15.94:0/2616508170 conn(0x7fc0100d1c90 msgr2=0x7fc0100d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:51.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.765+0000 7fc014e00640 1 -- 172.21.15.94:0/2616508170 shutdown_connections 2024-09-10T23:32:51.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.765+0000 7fc014e00640 1 -- 172.21.15.94:0/2616508170 wait complete. 2024-09-10T23:32:51.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.765+0000 7fc014e00640 1 Processor -- start 2024-09-10T23:32:51.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.765+0000 7fc014e00640 1 -- start start 2024-09-10T23:32:51.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc014e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff4003a90 0x7fc0100cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:51.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc014e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0100cef50 0x7fc0100c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:51.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc014e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0100b6ee0 0x7fc0100b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:51.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc014e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc01015f830 con 0x7fbff4003a90 2024-09-10T23:32:51.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc014e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc01015f6b0 con 0x7fc0100cef50 2024-09-10T23:32:51.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc014e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc0100b2790 con 0x7fc0100b6ee0 2024-09-10T23:32:51.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc00de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff4003a90 0x7fc0100cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:51.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc00de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff4003a90 0x7fc0100cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50968/0 (socket says 172.21.15.94:50968) 2024-09-10T23:32:51.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc00de00640 1 -- 172.21.15.94:0/564216016 learned_addr learned my addr 172.21.15.94:0/564216016 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:51.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc00e800640 1 --2- 172.21.15.94:0/564216016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0100b6ee0 0x7fc0100b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:51.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc005400640 1 --2- 172.21.15.94:0/564216016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0100cef50 0x7fc0100c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:51.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc00de00640 1 -- 172.21.15.94:0/564216016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0100b6ee0 msgr2=0x7fc0100b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:51.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc00de00640 1 --2- 172.21.15.94:0/564216016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0100b6ee0 0x7fc0100b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc00de00640 1 -- 172.21.15.94:0/564216016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0100cef50 msgr2=0x7fc0100c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:51.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc00de00640 1 --2- 172.21.15.94:0/564216016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0100cef50 0x7fc0100c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc00de00640 1 -- 172.21.15.94:0/564216016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0100ca810 con 0x7fbff4003a90 2024-09-10T23:32:51.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc00de00640 1 --2- 172.21.15.94:0/564216016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff4003a90 0x7fc0100cb8b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc000006040 tx=0x7fc0000013b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:51.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc006a00640 1 -- 172.21.15.94:0/564216016 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0000a5820 con 0x7fbff4003a90 2024-09-10T23:32:51.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc006a00640 1 -- 172.21.15.94:0/564216016 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0000a5ab0 con 0x7fbff4003a90 2024-09-10T23:32:51.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc014e00640 1 -- 172.21.15.94:0/564216016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0100caaf0 con 0x7fbff4003a90 2024-09-10T23:32:51.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc006a00640 1 -- 172.21.15.94:0/564216016 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000003600 con 0x7fbff4003a90 2024-09-10T23:32:51.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc014e00640 1 -- 172.21.15.94:0/564216016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0100cb0b0 con 0x7fbff4003a90 2024-09-10T23:32:51.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc014e00640 1 -- 172.21.15.94:0/564216016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0100cbf30 con 0x7fbff4003a90 2024-09-10T23:32:51.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.769+0000 7fc006a00640 1 -- 172.21.15.94:0/564216016 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc0000b6020 con 0x7fbff4003a90 2024-09-10T23:32:51.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.773+0000 7fc006a00640 1 --2- 172.21.15.94:0/564216016 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbfec0725e0 0x7fbfec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:51.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.773+0000 7fc006a00640 1 -- 172.21.15.94:0/564216016 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc00012cc10 con 0x7fbff4003a90 2024-09-10T23:32:51.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.773+0000 7fc005400640 1 --2- 172.21.15.94:0/564216016 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbfec0725e0 0x7fbfec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:51.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.773+0000 7fc005400640 1 --2- 172.21.15.94:0/564216016 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbfec0725e0 0x7fbfec074aa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc0100b7be0 tx=0x7fbff8000f60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:51.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.777+0000 7fc006a00640 1 -- 172.21.15.94:0/564216016 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc000016510 con 0x7fbff4003a90 2024-09-10T23:32:51.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.941+0000 7fc014e00640 1 -- 172.21.15.94:0/564216016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fc010072000 con 0x7fbff4003a90 2024-09-10T23:32:51.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.941+0000 7fc006a00640 1 -- 172.21.15.94:0/564216016 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v27) ==== 86+0+160 (secure 0 0 0) 0x7fc0000f7c10 con 0x7fbff4003a90 2024-09-10T23:32:51.946 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:51.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.945+0000 7fc014e00640 1 -- 172.21.15.94:0/564216016 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbfec0725e0 msgr2=0x7fbfec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:51.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.945+0000 7fc014e00640 1 --2- 172.21.15.94:0/564216016 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbfec0725e0 0x7fbfec074aa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc0100b7be0 tx=0x7fbff8000f60 comp rx=0 tx=0).stop 2024-09-10T23:32:51.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.945+0000 7fc014e00640 1 -- 172.21.15.94:0/564216016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff4003a90 msgr2=0x7fc0100cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:51.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.945+0000 7fc014e00640 1 --2- 172.21.15.94:0/564216016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff4003a90 0x7fc0100cb8b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc000006040 tx=0x7fc0000013b0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.945+0000 7fc014e00640 1 -- 172.21.15.94:0/564216016 shutdown_connections 2024-09-10T23:32:51.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.945+0000 7fc014e00640 1 --2- 172.21.15.94:0/564216016 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbfec0725e0 0x7fbfec074aa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.945+0000 7fc014e00640 1 --2- 172.21.15.94:0/564216016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0100cef50 0x7fc0100c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.945+0000 7fc014e00640 1 --2- 172.21.15.94:0/564216016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0100b6ee0 0x7fc0100b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.945+0000 7fc014e00640 1 --2- 172.21.15.94:0/564216016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff4003a90 0x7fc0100cb8b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:51.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.945+0000 7fc014e00640 1 -- 172.21.15.94:0/564216016 >> 172.21.15.94:0/564216016 conn(0x7fc0100d1c90 msgr2=0x7fc0101e6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:51.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.945+0000 7fc014e00640 1 -- 172.21.15.94:0/564216016 shutdown_connections 2024-09-10T23:32:51.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:51.945+0000 7fc014e00640 1 -- 172.21.15.94:0/564216016 wait complete. 2024-09-10T23:32:51.967 INFO:teuthology.orchestra.run.smithi094.stdout:[{"version":27,"timestamp":"2024-09-10T23:32:30.387339+0000","name":"","changes":[{"name":"osd.5/osd_mclock_max_capacity_iops_ssd","new_value":"4724.018776"}]}] 2024-09-10T23:32:51.967 INFO:tasks.ceph_manager:config epoch is 27 2024-09-10T23:32:51.967 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T23:32:51.968 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-10T23:32:51.968 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-10T23:32:52.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.105+0000 7f5ca7e00640 1 -- 172.21.15.94:0/696801454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ca815a650 msgr2=0x7f5ca815b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:52.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.105+0000 7f5ca7e00640 1 --2- 172.21.15.94:0/696801454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ca815a650 0x7f5ca815b880 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8072d80 tx=0x7f5c9801c850 comp rx=0 tx=0).stop 2024-09-10T23:32:52.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.105+0000 7f5ca7e00640 1 -- 172.21.15.94:0/696801454 shutdown_connections 2024-09-10T23:32:52.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.105+0000 7f5ca7e00640 1 --2- 172.21.15.94:0/696801454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ca815aa30 0x7f5ca80b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.105+0000 7f5ca7e00640 1 --2- 172.21.15.94:0/696801454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ca815a650 0x7f5ca815b880 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.105+0000 7f5ca7e00640 1 -- 172.21.15.94:0/696801454 >> 172.21.15.94:0/696801454 conn(0x7f5ca80d0110 msgr2=0x7f5ca80d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:52.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.105+0000 7f5ca7e00640 1 -- 172.21.15.94:0/696801454 shutdown_connections 2024-09-10T23:32:52.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.105+0000 7f5ca7e00640 1 -- 172.21.15.94:0/696801454 wait complete. 2024-09-10T23:32:52.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5ca7e00640 1 Processor -- start 2024-09-10T23:32:52.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5ca7e00640 1 -- start start 2024-09-10T23:32:52.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5ca7e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ca8074b70 0x7f5ca8159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:52.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5ca7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ca815a650 0x7f5ca814a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:52.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5ca7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ca815aa30 0x7f5ca814a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:52.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5ca7e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca81680e0 con 0x7f5ca815a650 2024-09-10T23:32:52.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5ca7e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca8167f60 con 0x7f5ca815aa30 2024-09-10T23:32:52.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5ca7e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca8168260 con 0x7f5ca8074b70 2024-09-10T23:32:52.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5c9e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ca815a650 0x7f5ca814a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:52.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5c9e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ca815a650 0x7f5ca814a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50980/0 (socket says 172.21.15.94:50980) 2024-09-10T23:32:52.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5c9e000640 1 -- 172.21.15.94:0/3435081360 learned_addr learned my addr 172.21.15.94:0/3435081360 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:52.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5ca6a00640 1 --2- 172.21.15.94:0/3435081360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ca8074b70 0x7f5ca8159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:52.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5ca7400640 1 --2- 172.21.15.94:0/3435081360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ca815aa30 0x7f5ca814a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:52.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5c9e000640 1 -- 172.21.15.94:0/3435081360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ca8074b70 msgr2=0x7f5ca8159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:52.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5c9e000640 1 --2- 172.21.15.94:0/3435081360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ca8074b70 0x7f5ca8159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5c9e000640 1 -- 172.21.15.94:0/3435081360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ca815aa30 msgr2=0x7f5ca814a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:52.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5c9e000640 1 --2- 172.21.15.94:0/3435081360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ca815aa30 0x7f5ca814a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5c9e000640 1 -- 172.21.15.94:0/3435081360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca80cd950 con 0x7f5ca815a650 2024-09-10T23:32:52.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5c9e000640 1 --2- 172.21.15.94:0/3435081360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ca815a650 0x7f5ca814a2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000ea20 tx=0x7f5c9000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:52.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5c9f400640 1 -- 172.21.15.94:0/3435081360 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9000cde0 con 0x7f5ca815a650 2024-09-10T23:32:52.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5c9f400640 1 -- 172.21.15.94:0/3435081360 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c90004c20 con 0x7f5ca815a650 2024-09-10T23:32:52.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5ca7e00640 1 -- 172.21.15.94:0/3435081360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca8164bb0 con 0x7f5ca815a650 2024-09-10T23:32:52.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5c9f400640 1 -- 172.21.15.94:0/3435081360 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90005020 con 0x7f5ca815a650 2024-09-10T23:32:52.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.109+0000 7f5ca7e00640 1 -- 172.21.15.94:0/3435081360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca814b050 con 0x7f5ca815a650 2024-09-10T23:32:52.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.113+0000 7f5ca7e00640 1 -- 172.21.15.94:0/3435081360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca815b880 con 0x7f5ca815a650 2024-09-10T23:32:52.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.113+0000 7f5c9f400640 1 -- 172.21.15.94:0/3435081360 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5c9000f040 con 0x7f5ca815a650 2024-09-10T23:32:52.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.113+0000 7f5c9f400640 1 --2- 172.21.15.94:0/3435081360 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c840725e0 0x7f5c84074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:52.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.113+0000 7f5c9f400640 1 -- 172.21.15.94:0/3435081360 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5c90093810 con 0x7f5ca815a650 2024-09-10T23:32:52.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.113+0000 7f5ca6a00640 1 --2- 172.21.15.94:0/3435081360 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c840725e0 0x7f5c84074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:52.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.113+0000 7f5ca6a00640 1 --2- 172.21.15.94:0/3435081360 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c840725e0 0x7f5c84074aa0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5c980af000 tx=0x7f5c9801b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:52.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.117+0000 7f5c9f400640 1 -- 172.21.15.94:0/3435081360 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5c90016030 con 0x7f5ca815a650 2024-09-10T23:32:52.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.325+0000 7f5ca7e00640 1 -- 172.21.15.94:0/3435081360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f5ca8068cc0 con 0x7f5ca815a650 2024-09-10T23:32:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.325+0000 7f5c9f400640 1 -- 172.21.15.94:0/3435081360 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+184992 (secure 0 0 0) 0x7f5c9005e890 con 0x7f5ca815a650 2024-09-10T23:32:52.332 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:52.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.329+0000 7f5ca7e00640 1 -- 172.21.15.94:0/3435081360 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c840725e0 msgr2=0x7f5c84074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.329+0000 7f5ca7e00640 1 --2- 172.21.15.94:0/3435081360 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c840725e0 0x7f5c84074aa0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5c980af000 tx=0x7f5c9801b090 comp rx=0 tx=0).stop 2024-09-10T23:32:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.329+0000 7f5ca7e00640 1 -- 172.21.15.94:0/3435081360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ca815a650 msgr2=0x7f5ca814a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.329+0000 7f5ca7e00640 1 --2- 172.21.15.94:0/3435081360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ca815a650 0x7f5ca814a2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000ea20 tx=0x7f5c9000eee0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.329+0000 7f5ca7e00640 1 -- 172.21.15.94:0/3435081360 shutdown_connections 2024-09-10T23:32:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.329+0000 7f5ca7e00640 1 --2- 172.21.15.94:0/3435081360 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c840725e0 0x7f5c84074aa0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.329+0000 7f5ca7e00640 1 --2- 172.21.15.94:0/3435081360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ca815aa30 0x7f5ca814a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.329+0000 7f5ca7e00640 1 --2- 172.21.15.94:0/3435081360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ca815a650 0x7f5ca814a2f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.329+0000 7f5ca7e00640 1 --2- 172.21.15.94:0/3435081360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ca8074b70 0x7f5ca8159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.329+0000 7f5ca7e00640 1 -- 172.21.15.94:0/3435081360 >> 172.21.15.94:0/3435081360 conn(0x7f5ca80d0110 msgr2=0x7f5ca814ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:52.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.329+0000 7f5ca7e00640 1 -- 172.21.15.94:0/3435081360 shutdown_connections 2024-09-10T23:32:52.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.329+0000 7f5ca7e00640 1 -- 172.21.15.94:0/3435081360 wait complete. 2024-09-10T23:32:52.352 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":5,"flags":0,"active_gid":4123,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3310396812},{"type":"v1","addr":"172.21.15.181:6801","nonce":3310396812}]},"active_addr":"172.21.15.181:6801/3310396812","active_change":"2024-09-10T23:32:21.329421+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4146,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":1584210907}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":1346366560}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2231303511}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2211863778}]}]} 2024-09-10T23:32:52.359 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-10T23:32:52.359 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T23:32:52.360 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:32:52.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.497+0000 7f8ba0200640 1 -- 172.21.15.94:0/3800986704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b80004d50 msgr2=0x7f8b800051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:52.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.497+0000 7f8ba0200640 1 --2- 172.21.15.94:0/3800986704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b80004d50 0x7f8b800051d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000c640 tx=0x7f8b9001d2a0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.497+0000 7f8b9e200640 1 -- 172.21.15.94:0/3800986704 reap_dead start 2024-09-10T23:32:52.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.497+0000 7f8ba0200640 1 -- 172.21.15.94:0/3800986704 shutdown_connections 2024-09-10T23:32:52.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.497+0000 7f8ba0200640 1 -- 172.21.15.94:0/3800986704 >> 172.21.15.94:0/3800986704 conn(0x7f8b980d1c60 msgr2=0x7f8b980d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:52.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8ba0200640 1 -- 172.21.15.94:0/3800986704 shutdown_connections 2024-09-10T23:32:52.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8ba0200640 1 -- 172.21.15.94:0/3800986704 wait complete. 2024-09-10T23:32:52.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8ba0200640 1 Processor -- start 2024-09-10T23:32:52.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8ba0200640 1 -- start start 2024-09-10T23:32:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8ba0200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b80004d50 0x7f8b980c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8ba0200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b980b4120 0x7f8b980b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8ba0200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b980c7b60 0x7f8b981f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8ba0200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98166ff0 con 0x7f8b80004d50 2024-09-10T23:32:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8ba0200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98166e70 con 0x7f8b980b4120 2024-09-10T23:32:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8ba0200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8b98167170 con 0x7f8b980c7b60 2024-09-10T23:32:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8b9d800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b80004d50 0x7f8b980c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8b9d800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b80004d50 0x7f8b980c65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51010/0 (socket says 172.21.15.94:51010) 2024-09-10T23:32:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8b9d800640 1 -- 172.21.15.94:0/2389153090 learned_addr learned my addr 172.21.15.94:0/2389153090 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8b9ce00640 1 --2- 172.21.15.94:0/2389153090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b980b4120 0x7f8b980b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8b9e200640 1 --2- 172.21.15.94:0/2389153090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b980c7b60 0x7f8b981f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8b9d800640 1 -- 172.21.15.94:0/2389153090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b980c7b60 msgr2=0x7f8b981f5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8b9d800640 1 --2- 172.21.15.94:0/2389153090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b980c7b60 0x7f8b981f5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8b9d800640 1 -- 172.21.15.94:0/2389153090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b980b4120 msgr2=0x7f8b980b6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8b9d800640 1 --2- 172.21.15.94:0/2389153090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b980b4120 0x7f8b980b6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8b9d800640 1 -- 172.21.15.94:0/2389153090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b980c5830 con 0x7f8b80004d50 2024-09-10T23:32:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8b9ce00640 1 --2- 172.21.15.94:0/2389153090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b980b4120 0x7f8b980b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:32:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8b9d800640 1 --2- 172.21.15.94:0/2389153090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b80004d50 0x7f8b980c65b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c011ae0 tx=0x7f8b8c009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8b95600640 1 -- 172.21.15.94:0/2389153090 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c01e070 con 0x7f8b80004d50 2024-09-10T23:32:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8b95600640 1 -- 172.21.15.94:0/2389153090 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b8c00fe40 con 0x7f8b80004d50 2024-09-10T23:32:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8b95600640 1 -- 172.21.15.94:0/2389153090 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0042b0 con 0x7f8b80004d50 2024-09-10T23:32:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.501+0000 7f8ba0200640 1 -- 172.21.15.94:0/2389153090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b980c5ad0 con 0x7f8b80004d50 2024-09-10T23:32:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.505+0000 7f8ba0200640 1 -- 172.21.15.94:0/2389153090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b980c6090 con 0x7f8b80004d50 2024-09-10T23:32:52.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.505+0000 7f8ba0200640 1 -- 172.21.15.94:0/2389153090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b980d0060 con 0x7f8b80004d50 2024-09-10T23:32:52.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.505+0000 7f8b95600640 1 -- 172.21.15.94:0/2389153090 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8b8c003600 con 0x7f8b80004d50 2024-09-10T23:32:52.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.505+0000 7f8b95600640 1 --2- 172.21.15.94:0/2389153090 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b780725e0 0x7f8b78074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:52.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.505+0000 7f8b95600640 1 -- 172.21.15.94:0/2389153090 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8b8c0940e0 con 0x7f8b80004d50 2024-09-10T23:32:52.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.505+0000 7f8b9ce00640 1 --2- 172.21.15.94:0/2389153090 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b780725e0 0x7f8b78074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:52.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.509+0000 7f8b9ce00640 1 --2- 172.21.15.94:0/2389153090 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b780725e0 0x7f8b78074aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8b98071fd0 tx=0x7f8b8400b260 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:52.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.509+0000 7f8b95600640 1 -- 172.21.15.94:0/2389153090 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8b8c05f190 con 0x7f8b80004d50 2024-09-10T23:32:52.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.681+0000 7f8ba0200640 1 -- 172.21.15.94:0/2389153090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b9815b790 con 0x7f8b80004d50 2024-09-10T23:32:52.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.681+0000 7f8b95600640 1 -- 172.21.15.94:0/2389153090 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16657 (secure 0 0 0) 0x7f8b8c063bc0 con 0x7f8b80004d50 2024-09-10T23:32:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:52.684 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":22,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:32:43.556132+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:32:39.855046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T23:32:40.904161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:32:52.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.681+0000 7f8ba0200640 1 -- 172.21.15.94:0/2389153090 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b780725e0 msgr2=0x7f8b78074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:52.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.681+0000 7f8ba0200640 1 --2- 172.21.15.94:0/2389153090 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b780725e0 0x7f8b78074aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8b98071fd0 tx=0x7f8b8400b260 comp rx=0 tx=0).stop 2024-09-10T23:32:52.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.681+0000 7f8ba0200640 1 -- 172.21.15.94:0/2389153090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b80004d50 msgr2=0x7f8b980c65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:52.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.681+0000 7f8ba0200640 1 --2- 172.21.15.94:0/2389153090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b80004d50 0x7f8b980c65b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c011ae0 tx=0x7f8b8c009ea0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.681+0000 7f8ba0200640 1 -- 172.21.15.94:0/2389153090 shutdown_connections 2024-09-10T23:32:52.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.681+0000 7f8ba0200640 1 --2- 172.21.15.94:0/2389153090 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b780725e0 0x7f8b78074aa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.681+0000 7f8ba0200640 1 --2- 172.21.15.94:0/2389153090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b980b4120 0x7f8b980b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.681+0000 7f8ba0200640 1 --2- 172.21.15.94:0/2389153090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b980c7b60 0x7f8b981f5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.681+0000 7f8ba0200640 1 --2- 172.21.15.94:0/2389153090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b80004d50 0x7f8b980c65b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.681+0000 7f8ba0200640 1 -- 172.21.15.94:0/2389153090 >> 172.21.15.94:0/2389153090 conn(0x7f8b980d1c60 msgr2=0x7f8b9815db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.681+0000 7f8ba0200640 1 -- 172.21.15.94:0/2389153090 shutdown_connections 2024-09-10T23:32:52.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.681+0000 7f8ba0200640 1 -- 172.21.15.94:0/2389153090 wait complete. 2024-09-10T23:32:52.705 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T23:32:52.705 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:32:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 -- 172.21.15.94:0/3199720135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bac15ecb0 msgr2=0x7f6bac0b35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 --2- 172.21.15.94:0/3199720135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bac15ecb0 0x7f6bac0b35f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0d0b80 tx=0x7f6ba401c890 comp rx=0 tx=0).stop 2024-09-10T23:32:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 -- 172.21.15.94:0/3199720135 shutdown_connections 2024-09-10T23:32:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 --2- 172.21.15.94:0/3199720135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bac15ecb0 0x7f6bac0b35f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 --2- 172.21.15.94:0/3199720135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6bac0d02e0 0x7f6bac15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 --2- 172.21.15.94:0/3199720135 >> v2:172.21.15.181:3300/0 conn(0x7f6bac073ff0 0x7f6bac0cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 -- 172.21.15.94:0/3199720135 >> 172.21.15.94:0/3199720135 conn(0x7f6bac06cdf0 msgr2=0x7f6bac06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:52.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 -- 172.21.15.94:0/3199720135 shutdown_connections 2024-09-10T23:32:52.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 -- 172.21.15.94:0/3199720135 wait complete. 2024-09-10T23:32:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 Processor -- start 2024-09-10T23:32:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 -- start start 2024-09-10T23:32:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6bac073ff0 0x7f6bac0af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bac0d02e0 0x7f6bac0b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bac15ecb0 0x7f6bac0b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac0729a0 con 0x7f6bac0d02e0 2024-09-10T23:32:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac072820 con 0x7f6bac15ecb0 2024-09-10T23:32:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6bb0a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6bac072b20 con 0x7f6bac073ff0 2024-09-10T23:32:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6ba1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bac0d02e0 0x7f6bac0b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6ba1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bac0d02e0 0x7f6bac0b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51028/0 (socket says 172.21.15.94:51028) 2024-09-10T23:32:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6ba1400640 1 -- 172.21.15.94:0/2066170704 learned_addr learned my addr 172.21.15.94:0/2066170704 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6ba9e00640 1 --2- 172.21.15.94:0/2066170704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6bac073ff0 0x7f6bac0af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6baa800640 1 --2- 172.21.15.94:0/2066170704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bac15ecb0 0x7f6bac0b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6ba1400640 1 -- 172.21.15.94:0/2066170704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6bac073ff0 msgr2=0x7f6bac0af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6ba1400640 1 --2- 172.21.15.94:0/2066170704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6bac073ff0 0x7f6bac0af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6ba1400640 1 -- 172.21.15.94:0/2066170704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bac15ecb0 msgr2=0x7f6bac0b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6ba1400640 1 --2- 172.21.15.94:0/2066170704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bac15ecb0 0x7f6bac0b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6ba1400640 1 -- 172.21.15.94:0/2066170704 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bac0b1b60 con 0x7f6bac0d02e0 2024-09-10T23:32:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.845+0000 7f6ba1400640 1 --2- 172.21.15.94:0/2066170704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bac0d02e0 0x7f6bac0b2e50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400d980 tx=0x7f6b9400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:52.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.849+0000 7f6ba2a00640 1 -- 172.21.15.94:0/2066170704 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b94017070 con 0x7f6bac0d02e0 2024-09-10T23:32:52.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.849+0000 7f6ba2a00640 1 -- 172.21.15.94:0/2066170704 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b94009d60 con 0x7f6bac0d02e0 2024-09-10T23:32:52.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.849+0000 7f6ba2a00640 1 -- 172.21.15.94:0/2066170704 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b94005280 con 0x7f6bac0d02e0 2024-09-10T23:32:52.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.849+0000 7f6bb0a00640 1 -- 172.21.15.94:0/2066170704 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bac0b1e40 con 0x7f6bac0d02e0 2024-09-10T23:32:52.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.849+0000 7f6bb0a00640 1 -- 172.21.15.94:0/2066170704 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bac0afb80 con 0x7f6bac0d02e0 2024-09-10T23:32:52.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.849+0000 7f6bb0a00640 1 -- 172.21.15.94:0/2066170704 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bac15e770 con 0x7f6bac0d02e0 2024-09-10T23:32:52.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.849+0000 7f6ba2a00640 1 -- 172.21.15.94:0/2066170704 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6b94005420 con 0x7f6bac0d02e0 2024-09-10T23:32:52.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.849+0000 7f6ba2a00640 1 --2- 172.21.15.94:0/2066170704 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6b880725e0 0x7f6b88074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:52.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.849+0000 7f6ba2a00640 1 -- 172.21.15.94:0/2066170704 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6b94093410 con 0x7f6bac0d02e0 2024-09-10T23:32:52.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.849+0000 7f6ba9e00640 1 --2- 172.21.15.94:0/2066170704 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6b880725e0 0x7f6b88074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:52.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.853+0000 7f6ba9e00640 1 --2- 172.21.15.94:0/2066170704 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6b880725e0 0x7f6b88074aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009f40 tx=0x7f6b9c009650 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:52.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:52.853+0000 7f6ba2a00640 1 -- 172.21.15.94:0/2066170704 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6b9405e400 con 0x7f6bac0d02e0 2024-09-10T23:32:53.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.021+0000 7f6bb0a00640 1 -- 172.21.15.94:0/2066170704 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6bac165560 con 0x7f6bac0d02e0 2024-09-10T23:32:53.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.021+0000 7f6ba2a00640 1 -- 172.21.15.94:0/2066170704 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16657 (secure 0 0 0) 0x7f6b94067170 con 0x7f6bac0d02e0 2024-09-10T23:32:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:53.024 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":22,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:32:43.556132+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:32:39.855046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T23:32:40.904161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:32:53.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.021+0000 7f6bb0a00640 1 -- 172.21.15.94:0/2066170704 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6b880725e0 msgr2=0x7f6b88074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.021+0000 7f6bb0a00640 1 --2- 172.21.15.94:0/2066170704 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6b880725e0 0x7f6b88074aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009f40 tx=0x7f6b9c009650 comp rx=0 tx=0).stop 2024-09-10T23:32:53.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.021+0000 7f6bb0a00640 1 -- 172.21.15.94:0/2066170704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bac0d02e0 msgr2=0x7f6bac0b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.021+0000 7f6bb0a00640 1 --2- 172.21.15.94:0/2066170704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bac0d02e0 0x7f6bac0b2e50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400d980 tx=0x7f6b9400de40 comp rx=0 tx=0).stop 2024-09-10T23:32:53.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.021+0000 7f6bb0a00640 1 -- 172.21.15.94:0/2066170704 shutdown_connections 2024-09-10T23:32:53.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.021+0000 7f6bb0a00640 1 --2- 172.21.15.94:0/2066170704 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6b880725e0 0x7f6b88074aa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.021+0000 7f6bb0a00640 1 --2- 172.21.15.94:0/2066170704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6bac15ecb0 0x7f6bac0b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.021+0000 7f6bb0a00640 1 --2- 172.21.15.94:0/2066170704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bac0d02e0 0x7f6bac0b2e50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.021+0000 7f6bb0a00640 1 --2- 172.21.15.94:0/2066170704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6bac073ff0 0x7f6bac0af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.021+0000 7f6bb0a00640 1 -- 172.21.15.94:0/2066170704 >> 172.21.15.94:0/2066170704 conn(0x7f6bac06cdf0 msgr2=0x7f6bac14ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.021+0000 7f6bb0a00640 1 -- 172.21.15.94:0/2066170704 shutdown_connections 2024-09-10T23:32:53.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.021+0000 7f6bb0a00640 1 -- 172.21.15.94:0/2066170704 wait complete. 2024-09-10T23:32:53.046 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T23:32:53.046 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T23:32:53.047 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T23:32:53.047 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-10T23:32:53.047 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-10T23:32:53.048 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-10T23:32:53.048 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-10T23:32:53.049 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-10T23:32:53.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.293+0000 7fe2b9200640 1 -- 172.21.15.94:0/1219106035 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2ac00a480 msgr2=0x7fe2ac00a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.293+0000 7fe2b9200640 1 --2- 172.21.15.94:0/1219106035 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2ac00a480 0x7fe2ac00a860 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80098f0 tx=0x7fe2a801c7e0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.293+0000 7f5aa8a00640 1 -- 172.21.15.94:0/652560216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5aa40cdc90 msgr2=0x7f5aa406f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.293+0000 7fe2b9200640 1 -- 172.21.15.94:0/1219106035 shutdown_connections 2024-09-10T23:32:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.293+0000 7fe2b9200640 1 --2- 172.21.15.94:0/1219106035 >> v2:172.21.15.94:3300/0 conn(0x7fe2ac00ba10 0x7fe2ac035ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.293+0000 7fe2b9200640 1 --2- 172.21.15.94:0/1219106035 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2ac00a480 0x7fe2ac00a860 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.293+0000 7fe2b9200640 1 -- 172.21.15.94:0/1219106035 >> 172.21.15.94:0/1219106035 conn(0x7fe2ac054570 msgr2=0x7fe2ac054980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.293+0000 7f5aa8a00640 1 --2- 172.21.15.94:0/652560216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5aa40cdc90 0x7f5aa406f4d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40753b0 tx=0x7f5a9c01cd50 comp rx=0 tx=0).stop 2024-09-10T23:32:53.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.293+0000 7f5aa8a00640 1 -- 172.21.15.94:0/652560216 shutdown_connections 2024-09-10T23:32:53.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.293+0000 7f5aa8a00640 1 --2- 172.21.15.94:0/652560216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5aa40cdc90 0x7f5aa406f4d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.293+0000 7f5aa8a00640 1 --2- 172.21.15.94:0/652560216 >> v2:172.21.15.181:3300/0 conn(0x7f5aa406ce80 0x7f5aa415f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.293+0000 7f5aa8a00640 1 -- 172.21.15.94:0/652560216 >> 172.21.15.94:0/652560216 conn(0x7f5aa40cf8d0 msgr2=0x7f5aa40cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.293+0000 7f5aa8a00640 1 -- 172.21.15.94:0/652560216 shutdown_connections 2024-09-10T23:32:53.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.293+0000 7f5aa8a00640 1 -- 172.21.15.94:0/652560216 wait complete. 2024-09-10T23:32:53.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.293+0000 7fe2b9200640 1 -- 172.21.15.94:0/1219106035 shutdown_connections 2024-09-10T23:32:53.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.297+0000 7fe2b9200640 1 -- 172.21.15.94:0/1219106035 wait complete. 2024-09-10T23:32:53.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.297+0000 7f5aa8a00640 1 Processor -- start 2024-09-10T23:32:53.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.297+0000 7f5aa8a00640 1 -- start start 2024-09-10T23:32:53.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5aa8a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5aa406ce80 0x7f5aa40b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5aa8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5aa406d260 0x7f5aa40c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5aa8a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5aa40cdc90 0x7f5aa4072f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7fe2b9200640 1 Processor -- start 2024-09-10T23:32:53.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7fe2b9200640 1 -- start start 2024-09-10T23:32:53.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5aa2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5aa406ce80 0x7f5aa40b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5aa2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5aa406ce80 0x7f5aa40b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:54522/0 (socket says 172.21.15.94:54522) 2024-09-10T23:32:53.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5aa2800640 1 -- 172.21.15.94:0/1468590985 learned_addr learned my addr 172.21.15.94:0/1468590985 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:53.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5aa8a00640 1 -- 172.21.15.94:0/1468590985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa4159ad0 con 0x7f5aa406d260 2024-09-10T23:32:53.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5aa8a00640 1 -- 172.21.15.94:0/1468590985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa4159950 con 0x7f5aa406ce80 2024-09-10T23:32:53.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5aa8a00640 1 -- 172.21.15.94:0/1468590985 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5aa4159c50 con 0x7f5aa40cdc90 2024-09-10T23:32:53.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5aa3200640 1 --2- 172.21.15.94:0/1468590985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5aa40cdc90 0x7f5aa4072f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5aa2800640 1 -- 172.21.15.94:0/1468590985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5aa40cdc90 msgr2=0x7f5aa4072f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5aa2800640 1 --2- 172.21.15.94:0/1468590985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5aa40cdc90 0x7f5aa4072f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5aa2800640 1 -- 172.21.15.94:0/1468590985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5aa406d260 msgr2=0x7f5aa40c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5aa2800640 1 --2- 172.21.15.94:0/1468590985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5aa406d260 0x7f5aa40c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5aa2800640 1 -- 172.21.15.94:0/1468590985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa415dc90 con 0x7f5aa406ce80 2024-09-10T23:32:53.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7fe2b9200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac00a480 0x7fe2ac0de690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5aa2800640 1 --2- 172.21.15.94:0/1468590985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5aa406ce80 0x7f5aa40b2ed0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400e9f0 tx=0x7f5a9400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5a9b400640 1 -- 172.21.15.94:0/1468590985 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9400cd80 con 0x7f5aa406ce80 2024-09-10T23:32:53.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5a9b400640 1 -- 172.21.15.94:0/1468590985 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a94010430 con 0x7f5aa406ce80 2024-09-10T23:32:53.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7fe2b9200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2ac00ba10 0x7fe2ac047810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7fe2b9200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2ac01c370 0x7fe2ac0da6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7fe2b9200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0eb4b0 con 0x7fe2ac00ba10 2024-09-10T23:32:53.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7fe2b9200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0eb330 con 0x7fe2ac00a480 2024-09-10T23:32:53.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7fe2b9200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac0eb630 con 0x7fe2ac01c370 2024-09-10T23:32:53.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7fe2b3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac00a480 0x7fe2ac0de690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7fe2b3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac00a480 0x7fe2ac0de690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:54542/0 (socket says 172.21.15.94:54542) 2024-09-10T23:32:53.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7fe2b3400640 1 -- 172.21.15.94:0/3838863128 learned_addr learned my addr 172.21.15.94:0/3838863128 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:53.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7fe2b3e00640 1 --2- 172.21.15.94:0/3838863128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2ac01c370 0x7fe2ac0da6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7fe2b2a00640 1 --2- 172.21.15.94:0/3838863128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2ac00ba10 0x7fe2ac047810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7fe2b3400640 1 -- 172.21.15.94:0/3838863128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2ac01c370 msgr2=0x7fe2ac0da6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7fe2b3400640 1 --2- 172.21.15.94:0/3838863128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2ac01c370 0x7fe2ac0da6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7fe2b3400640 1 -- 172.21.15.94:0/3838863128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2ac00ba10 msgr2=0x7fe2ac047810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.301+0000 7f5a9b400640 1 -- 172.21.15.94:0/1468590985 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94010860 con 0x7f5aa406ce80 2024-09-10T23:32:53.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fe2b3400640 1 --2- 172.21.15.94:0/3838863128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2ac00ba10 0x7fe2ac047810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fe2b3400640 1 -- 172.21.15.94:0/3838863128 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac0d9620 con 0x7fe2ac00a480 2024-09-10T23:32:53.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fe2b2a00640 1 --2- 172.21.15.94:0/3838863128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2ac00ba10 0x7fe2ac047810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:32:53.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fe2b3400640 1 --2- 172.21.15.94:0/3838863128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac00a480 0x7fe2ac0de690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8019450 tx=0x7fe2a80051a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7f0a85c00640 1 -- 172.21.15.94:0/3022145156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a78033470 msgr2=0x7f0a78054e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7f0a85c00640 1 --2- 172.21.15.94:0/3022145156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a78033470 0x7f0a78054e90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0a780e0fb0 tx=0x7f0a8009ac10 comp rx=0 tx=0).stop 2024-09-10T23:32:53.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7f0a85c00640 1 -- 172.21.15.94:0/3022145156 shutdown_connections 2024-09-10T23:32:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7f0a85c00640 1 --2- 172.21.15.94:0/3022145156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a78033470 0x7f0a78054e90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7f0a85c00640 1 -- 172.21.15.94:0/3022145156 >> 172.21.15.94:0/3022145156 conn(0x7f0a7801d780 msgr2=0x7f0a78054570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7f0a85c00640 1 -- 172.21.15.94:0/3022145156 shutdown_connections 2024-09-10T23:32:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7f0a85c00640 1 -- 172.21.15.94:0/3022145156 wait complete. 2024-09-10T23:32:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7f5aa8a00640 1 -- 172.21.15.94:0/1468590985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa4072430 con 0x7f5aa406ce80 2024-09-10T23:32:53.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fe2a7400640 1 -- 172.21.15.94:0/3838863128 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80b2040 con 0x7fe2ac00a480 2024-09-10T23:32:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fe2a7400640 1 -- 172.21.15.94:0/3838863128 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2a80055b0 con 0x7fe2ac00a480 2024-09-10T23:32:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fe2a7400640 1 -- 172.21.15.94:0/3838863128 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8004290 con 0x7fe2ac00a480 2024-09-10T23:32:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7f5aa8a00640 1 -- 172.21.15.94:0/1468590985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa40729f0 con 0x7f5aa406ce80 2024-09-10T23:32:53.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fe2b9200640 1 -- 172.21.15.94:0/3838863128 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ac0d9900 con 0x7fe2ac00a480 2024-09-10T23:32:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fc77a200640 1 -- 172.21.15.94:0/1712979625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc774154d80 msgr2=0x7fc7740c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fc77a200640 1 --2- 172.21.15.94:0/1712979625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc774154d80 0x7fc7740c53c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc77406e530 tx=0x7fc76401c800 comp rx=0 tx=0).stop 2024-09-10T23:32:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fc77a200640 1 -- 172.21.15.94:0/1712979625 shutdown_connections 2024-09-10T23:32:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fc77a200640 1 --2- 172.21.15.94:0/1712979625 >> v2:172.21.15.181:3300/0 conn(0x7fc774161a80 0x7fc774161ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fc77a200640 1 --2- 172.21.15.94:0/1712979625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc774154d80 0x7fc7740c53c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fc77a200640 1 -- 172.21.15.94:0/1712979625 >> 172.21.15.94:0/1712979625 conn(0x7fc7740cfaf0 msgr2=0x7fc7740cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7f5aa8a00640 1 -- 172.21.15.94:0/1468590985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5aa415f260 con 0x7f5aa406ce80 2024-09-10T23:32:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7f0a85c00640 1 Processor -- start 2024-09-10T23:32:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fe2b9200640 1 -- 172.21.15.94:0/3838863128 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ac0d9ec0 con 0x7fe2ac00a480 2024-09-10T23:32:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fc77a200640 1 -- 172.21.15.94:0/1712979625 shutdown_connections 2024-09-10T23:32:53.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7f0a85c00640 1 -- start start 2024-09-10T23:32:53.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7fe2b9200640 1 -- 172.21.15.94:0/3838863128 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe2ac00a860 con 0x7fe2ac00a480 2024-09-10T23:32:53.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f0a85c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a78033470 0x7f0a7804ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f0a85c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a78036700 0x7f0a78053100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f0a85c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a780e0a50 0x7f0a78053640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f0a85c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0a78006d40 con 0x7f0a78033470 2024-09-10T23:32:53.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f0a85c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0a78006bc0 con 0x7f0a78036700 2024-09-10T23:32:53.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f0a85c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0a78006ec0 con 0x7f0a780e0a50 2024-09-10T23:32:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.305+0000 7fc77a200640 1 -- 172.21.15.94:0/1712979625 wait complete. 2024-09-10T23:32:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f5a9b400640 1 -- 172.21.15.94:0/1468590985 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5a9403daa0 con 0x7f5aa406ce80 2024-09-10T23:32:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f0a7f400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a780e0a50 0x7f0a78053640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f0a7f400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a780e0a50 0x7f0a78053640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:50650/0 (socket says 172.21.15.94:50650) 2024-09-10T23:32:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f0a7f400640 1 -- 172.21.15.94:0/2212556058 learned_addr learned my addr 172.21.15.94:0/2212556058 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:53.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7fc77a200640 1 Processor -- start 2024-09-10T23:32:53.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f0a7ea00640 1 --2- 172.21.15.94:0/2212556058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a78033470 0x7f0a7804ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f0a7e000640 1 --2- 172.21.15.94:0/2212556058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a78036700 0x7f0a78053100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f5a9b400640 1 --2- 172.21.15.94:0/1468590985 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5a800725e0 0x7f5a80074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f0a7f400640 1 -- 172.21.15.94:0/2212556058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a78036700 msgr2=0x7f0a78053100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f0a7f400640 1 --2- 172.21.15.94:0/2212556058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a78036700 0x7f0a78053100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f0a7f400640 1 -- 172.21.15.94:0/2212556058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a78033470 msgr2=0x7f0a7804ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f0a7f400640 1 --2- 172.21.15.94:0/2212556058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a78033470 0x7f0a7804ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f0a7f400640 1 -- 172.21.15.94:0/2212556058 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7804cfa0 con 0x7f0a780e0a50 2024-09-10T23:32:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f29b4800640 1 -- 172.21.15.94:0/2703381389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29ac0cab50 msgr2=0x7f29ac0caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f29b4800640 1 --2- 172.21.15.94:0/2703381389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29ac0cab50 0x7f29ac0caf30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009840 tx=0x7f29a001c820 comp rx=0 tx=0).stop 2024-09-10T23:32:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f29b4800640 1 -- 172.21.15.94:0/2703381389 shutdown_connections 2024-09-10T23:32:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f29b4800640 1 --2- 172.21.15.94:0/2703381389 >> v2:172.21.15.181:3300/0 conn(0x7f29ac0cd9f0 0x7f29ac0cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f29b4800640 1 --2- 172.21.15.94:0/2703381389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29ac0cab50 0x7f29ac0caf30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f29b4800640 1 -- 172.21.15.94:0/2703381389 >> 172.21.15.94:0/2703381389 conn(0x7f29ac0cfaf0 msgr2=0x7f29ac0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f5a9b400640 1 -- 172.21.15.94:0/1468590985 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5a94094090 con 0x7f5aa406ce80 2024-09-10T23:32:53.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f5a9b400640 1 --2- 172.21.15.94:0/1468590985 >> [v2:172.21.15.94:6800/576048946,v1:172.21.15.94:6801/576048946] conn(0x7f5a8007a800 0x7f5a8007cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5a9b400640 1 -- 172.21.15.94:0/1468590985 --> [v2:172.21.15.94:6800/576048946,v1:172.21.15.94:6801/576048946] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a8007d2d0 con 0x7f5a8007a800 2024-09-10T23:32:53.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5a9b400640 1 -- 172.21.15.94:0/1468590985 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5a94094480 con 0x7f5aa406ce80 2024-09-10T23:32:53.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5a99e00640 1 --2- 172.21.15.94:0/1468590985 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5a800725e0 0x7f5a80074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5aa3200640 1 --2- 172.21.15.94:0/1468590985 >> [v2:172.21.15.94:6800/576048946,v1:172.21.15.94:6801/576048946] conn(0x7f5a8007a800 0x7f5a8007cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5aa3200640 1 --2- 172.21.15.94:0/1468590985 >> [v2:172.21.15.94:6800/576048946,v1:172.21.15.94:6801/576048946] conn(0x7f5a8007a800 0x7f5a8007cc20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7fe2a7400640 1 -- 172.21.15.94:0/3838863128 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe2a80b5070 con 0x7fe2ac00a480 2024-09-10T23:32:53.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f0a7f400640 1 --2- 172.21.15.94:0/2212556058 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a780e0a50 0x7f0a78053640 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0a8006c150 tx=0x7f0a80125ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f0a76a00640 1 -- 172.21.15.94:0/2212556058 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80125d60 con 0x7f0a780e0a50 2024-09-10T23:32:53.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f0a76a00640 1 -- 172.21.15.94:0/2212556058 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a80063640 con 0x7f0a780e0a50 2024-09-10T23:32:53.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5284000640 1 -- 172.21.15.94:0/122318482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f527c073010 msgr2=0x7f527c06ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5284000640 1 --2- 172.21.15.94:0/122318482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f527c073010 0x7f527c06ed80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5268009840 tx=0x7f526801c7d0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5284000640 1 -- 172.21.15.94:0/122318482 shutdown_connections 2024-09-10T23:32:53.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5284000640 1 --2- 172.21.15.94:0/122318482 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c159f10 0x7f527c0cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5284000640 1 --2- 172.21.15.94:0/122318482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f527c073010 0x7f527c06ed80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5284000640 1 -- 172.21.15.94:0/122318482 >> 172.21.15.94:0/122318482 conn(0x7f527c0cff50 msgr2=0x7f527c0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f5c13c00640 1 -- 172.21.15.94:0/492745390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c0c154d80 msgr2=0x7f5c0c0c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f5c13c00640 1 --2- 172.21.15.94:0/492745390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c0c154d80 0x7f5c0c0c53c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009c80 tx=0x7f5c0401cb70 comp rx=0 tx=0).stop 2024-09-10T23:32:53.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f5c13c00640 1 -- 172.21.15.94:0/492745390 shutdown_connections 2024-09-10T23:32:53.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f5c13c00640 1 --2- 172.21.15.94:0/492745390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c0c154d80 0x7f5c0c0c53c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f5c13c00640 1 -- 172.21.15.94:0/492745390 >> 172.21.15.94:0/492745390 conn(0x7f5c0c0cfaf0 msgr2=0x7f5c0c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5c13c00640 1 -- 172.21.15.94:0/492745390 shutdown_connections 2024-09-10T23:32:53.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f29b4800640 1 -- 172.21.15.94:0/2703381389 shutdown_connections 2024-09-10T23:32:53.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f0a76a00640 1 -- 172.21.15.94:0/2212556058 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a800639f0 con 0x7f0a780e0a50 2024-09-10T23:32:53.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f0a85c00640 1 -- 172.21.15.94:0/2212556058 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7817e650 con 0x7f0a780e0a50 2024-09-10T23:32:53.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f0a85c00640 1 -- 172.21.15.94:0/2212556058 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a780d9200 con 0x7f0a780e0a50 2024-09-10T23:32:53.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fe2a7400640 1 --2- 172.21.15.94:0/3838863128 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe2940726b0 0x7fe294074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fe2a7400640 1 -- 172.21.15.94:0/3838863128 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe2a812ccd0 con 0x7fe2ac00a480 2024-09-10T23:32:53.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fe2a7400640 1 --2- 172.21.15.94:0/3838863128 >> [v2:172.21.15.94:6803/3936794679,v1:172.21.15.94:6805/3936794679] conn(0x7fe29407a930 0x7fe29407cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fe2a7400640 1 -- 172.21.15.94:0/3838863128 --> [v2:172.21.15.94:6803/3936794679,v1:172.21.15.94:6805/3936794679] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe29407d400 con 0x7fe29407a930 2024-09-10T23:32:53.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fe2a7400640 1 -- 172.21.15.94:0/3838863128 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe2a812d0c0 con 0x7fe2ac00a480 2024-09-10T23:32:53.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5c13c00640 1 -- 172.21.15.94:0/492745390 wait complete. 2024-09-10T23:32:53.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc77a200640 1 -- start start 2024-09-10T23:32:53.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc77a200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7740cea00 0x7fc774200400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc77a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc774154d80 0x7fc77406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc77a200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc774161a80 0x7fc77406eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc77a200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc7740ae500 con 0x7fc774154d80 2024-09-10T23:32:53.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc77a200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc7740ae380 con 0x7fc774161a80 2024-09-10T23:32:53.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc77a200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc7740ae680 con 0x7fc7740cea00 2024-09-10T23:32:53.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5284000640 1 -- 172.21.15.94:0/122318482 shutdown_connections 2024-09-10T23:32:53.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5c13c00640 1 Processor -- start 2024-09-10T23:32:53.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.309+0000 7f29b4800640 1 -- 172.21.15.94:0/2703381389 wait complete. 2024-09-10T23:32:53.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f29b4800640 1 Processor -- start 2024-09-10T23:32:53.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc773e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc774161a80 0x7fc77406eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc773e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc774161a80 0x7fc77406eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:54566/0 (socket says 172.21.15.94:54566) 2024-09-10T23:32:53.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc773e00640 1 -- 172.21.15.94:0/205819740 learned_addr learned my addr 172.21.15.94:0/205819740 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:53.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fe2b2a00640 1 --2- 172.21.15.94:0/3838863128 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe2940726b0 0x7fe294074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fe2b3e00640 1 --2- 172.21.15.94:0/3838863128 >> [v2:172.21.15.94:6803/3936794679,v1:172.21.15.94:6805/3936794679] conn(0x7fe29407a930 0x7fe29407cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5a99e00640 1 --2- 172.21.15.94:0/1468590985 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5a800725e0 0x7f5a80074aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c007fd0 tx=0x7f5a8c0078e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f29b4800640 1 -- start start 2024-09-10T23:32:53.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f0a85c00640 1 -- 172.21.15.94:0/2212556058 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a780d9400 con 0x7f0a780e0a50 2024-09-10T23:32:53.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc772a00640 1 --2- 172.21.15.94:0/205819740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc774154d80 0x7fc77406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5c13c00640 1 -- start start 2024-09-10T23:32:53.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5a9b400640 1 -- 172.21.15.94:0/1468590985 <== osd.0 v2:172.21.15.94:6800/576048946 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a8007d2d0 con 0x7f5a8007a800 2024-09-10T23:32:53.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7fe2b3e00640 1 --2- 172.21.15.94:0/3838863128 >> [v2:172.21.15.94:6803/3936794679,v1:172.21.15.94:6805/3936794679] conn(0x7fe29407a930 0x7fe29407cd50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc773400640 1 --2- 172.21.15.94:0/205819740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7740cea00 0x7fc774200400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc773e00640 1 -- 172.21.15.94:0/205819740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7740cea00 msgr2=0x7fc774200400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc773e00640 1 --2- 172.21.15.94:0/205819740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7740cea00 0x7fc774200400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc773e00640 1 -- 172.21.15.94:0/205819740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc774154d80 msgr2=0x7fc77406e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc773e00640 1 --2- 172.21.15.94:0/205819740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc774154d80 0x7fc77406e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7fc773e00640 1 -- 172.21.15.94:0/205819740 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7741fc570 con 0x7fc774161a80 2024-09-10T23:32:53.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.313+0000 7f5284000640 1 -- 172.21.15.94:0/122318482 wait complete. 2024-09-10T23:32:53.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7fe2b2a00640 1 --2- 172.21.15.94:0/3838863128 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe2940726b0 0x7fe294074b70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac04b1a0 tx=0x7fe2a0008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b4800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29ac0cab50 0x7f29ac204ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b4800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29ac0cd9f0 0x7f29ac154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b4800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ac15b110 0x7f29ac1571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c13c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c0c0cea00 0x7f5c0c1517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c13c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c0c154d80 0x7f5c0c0adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c13c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c0c161a80 0x7f5c0c156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c13c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c1625e0 con 0x7f5c0c154d80 2024-09-10T23:32:53.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c13c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c155370 con 0x7f5c0c0cea00 2024-09-10T23:32:53.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c13c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5c0c162760 con 0x7f5c0c161a80 2024-09-10T23:32:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b4800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac167210 con 0x7f29ac15b110 2024-09-10T23:32:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b4800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac167090 con 0x7f29ac0cd9f0 2024-09-10T23:32:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b4800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f29ac167390 con 0x7f29ac0cab50 2024-09-10T23:32:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ac15b110 0x7f29ac1571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ac15b110 0x7f29ac1571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51088/0 (socket says 172.21.15.94:51088) 2024-09-10T23:32:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b2800640 1 -- 172.21.15.94:0/2399429926 learned_addr learned my addr 172.21.15.94:0/2399429926 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7fc773e00640 1 --2- 172.21.15.94:0/205819740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc774161a80 0x7fc77406eb20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00eb20 tx=0x7fc76c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7fc76b400640 1 -- 172.21.15.94:0/205819740 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c004b30 con 0x7fc774161a80 2024-09-10T23:32:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7fc76b400640 1 -- 172.21.15.94:0/205819740 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc76c004cf0 con 0x7fc774161a80 2024-09-10T23:32:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7fc76b400640 1 -- 172.21.15.94:0/205819740 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c005280 con 0x7fc774161a80 2024-09-10T23:32:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5284000640 1 Processor -- start 2024-09-10T23:32:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c11c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c0c161a80 0x7f5c0c156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c11c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c0c161a80 0x7f5c0c156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:50674/0 (socket says 172.21.15.94:50674) 2024-09-10T23:32:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c11c00640 1 -- 172.21.15.94:0/3944900568 learned_addr learned my addr 172.21.15.94:0/3944900568 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c0be00640 1 --2- 172.21.15.94:0/3944900568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c0c154d80 0x7f5c0c0adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c11200640 1 --2- 172.21.15.94:0/3944900568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c0c0cea00 0x7f5c0c1517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29ac0cd9f0 0x7f29ac154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b1e00640 1 --2- 172.21.15.94:0/2399429926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29ac0cab50 0x7f29ac204ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b2800640 1 -- 172.21.15.94:0/2399429926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29ac0cab50 msgr2=0x7f29ac204ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b2800640 1 --2- 172.21.15.94:0/2399429926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29ac0cab50 0x7f29ac204ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b2800640 1 -- 172.21.15.94:0/2399429926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29ac0cd9f0 msgr2=0x7f29ac154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b2800640 1 --2- 172.21.15.94:0/2399429926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29ac0cd9f0 0x7f29ac154f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b2800640 1 -- 172.21.15.94:0/2399429926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ac2050e0 con 0x7f29ac15b110 2024-09-10T23:32:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b1e00640 1 --2- 172.21.15.94:0/2399429926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29ac0cab50 0x7f29ac204ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f29b2800640 1 --2- 172.21.15.94:0/2399429926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ac15b110 0x7f29ac1571f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f29a800a660 tx=0x7f29a800ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c11c00640 1 -- 172.21.15.94:0/3944900568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c0c0cea00 msgr2=0x7f5c0c1517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c11c00640 1 --2- 172.21.15.94:0/3944900568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c0c0cea00 0x7f5c0c1517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c11c00640 1 -- 172.21.15.94:0/3944900568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c0c154d80 msgr2=0x7f5c0c0adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5284000640 1 -- start start 2024-09-10T23:32:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c11c00640 1 --2- 172.21.15.94:0/3944900568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c0c154d80 0x7f5c0c0adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c11c00640 1 -- 172.21.15.94:0/3944900568 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0c1fc590 con 0x7f5c0c161a80 2024-09-10T23:32:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f0a76a00640 1 -- 172.21.15.94:0/2212556058 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0a80063b90 con 0x7f0a780e0a50 2024-09-10T23:32:53.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7fe2a7400640 1 -- 172.21.15.94:0/3838863128 <== osd.1 v2:172.21.15.94:6803/3936794679 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe29407d400 con 0x7fe29407a930 2024-09-10T23:32:53.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.317+0000 7f5c11c00640 1 --2- 172.21.15.94:0/3944900568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c0c161a80 0x7f5c0c156840 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800c210 tx=0x7f5bf800c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5c08c00640 1 -- 172.21.15.94:0/3944900568 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8004020 con 0x7f5c0c161a80 2024-09-10T23:32:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7fc77a200640 1 -- 172.21.15.94:0/205819740 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7741fc810 con 0x7fc774161a80 2024-09-10T23:32:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7fc77a200640 1 -- 172.21.15.94:0/205819740 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7741fcdd0 con 0x7fc774161a80 2024-09-10T23:32:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5c08c00640 1 -- 172.21.15.94:0/3944900568 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bf80042b0 con 0x7f5c0c161a80 2024-09-10T23:32:53.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5c08c00640 1 -- 172.21.15.94:0/3944900568 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8003860 con 0x7f5c0c161a80 2024-09-10T23:32:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f0a76a00640 1 --2- 172.21.15.94:0/2212556058 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0a600726b0 0x7f0a60074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f0a76a00640 1 -- 172.21.15.94:0/2212556058 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0a801ac510 con 0x7f0a780e0a50 2024-09-10T23:32:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f0a76a00640 1 --2- 172.21.15.94:0/2212556058 >> [v2:172.21.15.94:6824/3187639367,v1:172.21.15.94:6825/3187639367] conn(0x7f0a6007a930 0x7f0a6007cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5284000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527c06a060 0x7f527c09db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5284000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f527c159f10 0x7f527c1f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5284000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c164310 0x7f527c152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5284000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f527c167e50 con 0x7f527c159f10 2024-09-10T23:32:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5284000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f527c167cd0 con 0x7f527c06a060 2024-09-10T23:32:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5284000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f527c167fd0 con 0x7f527c164310 2024-09-10T23:32:53.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f29a6000640 1 -- 172.21.15.94:0/2399429926 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a800c6f0 con 0x7f29ac15b110 2024-09-10T23:32:53.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f29a6000640 1 -- 172.21.15.94:0/2399429926 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29a800c890 con 0x7f29ac15b110 2024-09-10T23:32:53.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f29a6000640 1 -- 172.21.15.94:0/2399429926 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a800cce0 con 0x7f29ac15b110 2024-09-10T23:32:53.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f29b4800640 1 -- 172.21.15.94:0/2399429926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29ac205380 con 0x7f29ac15b110 2024-09-10T23:32:53.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5c13c00640 1 -- 172.21.15.94:0/3944900568 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c0c1fc830 con 0x7f5c0c161a80 2024-09-10T23:32:53.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f29b4800640 1 -- 172.21.15.94:0/2399429926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29ac205940 con 0x7f29ac15b110 2024-09-10T23:32:53.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f0a7ea00640 1 --2- 172.21.15.94:0/2212556058 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0a600726b0 0x7f0a60074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f0a7e000640 1 --2- 172.21.15.94:0/2212556058 >> [v2:172.21.15.94:6824/3187639367,v1:172.21.15.94:6825/3187639367] conn(0x7f0a6007a930 0x7f0a6007cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f0a76a00640 1 -- 172.21.15.94:0/2212556058 --> [v2:172.21.15.94:6824/3187639367,v1:172.21.15.94:6825/3187639367] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a80063640 con 0x7f0a6007a930 2024-09-10T23:32:53.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f0a76a00640 1 -- 172.21.15.94:0/2212556058 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f0a801ac900 con 0x7f0a780e0a50 2024-09-10T23:32:53.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5c13c00640 1 -- 172.21.15.94:0/3944900568 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c0c1fcdf0 con 0x7f5c0c161a80 2024-09-10T23:32:53.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5281600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527c06a060 0x7f527c09db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5282000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c164310 0x7f527c152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5282000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c164310 0x7f527c152950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:50688/0 (socket says 172.21.15.94:50688) 2024-09-10T23:32:53.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5281600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527c06a060 0x7f527c09db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:54598/0 (socket says 172.21.15.94:54598) 2024-09-10T23:32:53.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5282000640 1 -- 172.21.15.94:0/909551370 learned_addr learned my addr 172.21.15.94:0/909551370 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:53.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f0a7ea00640 1 --2- 172.21.15.94:0/2212556058 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0a600726b0 0x7f0a60074b70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a700011d0 tx=0x7f0a70009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5280c00640 1 --2- 172.21.15.94:0/909551370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f527c159f10 0x7f527c1f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5282000640 1 -- 172.21.15.94:0/909551370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527c06a060 msgr2=0x7f527c09db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5282000640 1 --2- 172.21.15.94:0/909551370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527c06a060 0x7f527c09db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5282000640 1 -- 172.21.15.94:0/909551370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f527c159f10 msgr2=0x7f527c1f6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5282000640 1 --2- 172.21.15.94:0/909551370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f527c159f10 0x7f527c1f6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5282000640 1 -- 172.21.15.94:0/909551370 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527c152e90 con 0x7f527c164310 2024-09-10T23:32:53.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7f5c13c00640 1 -- 172.21.15.94:0/3944900568 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5c0c156d80 con 0x7f5c0c161a80 2024-09-10T23:32:53.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7fc77a200640 1 -- 172.21.15.94:0/205819740 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc7740c53c0 con 0x7fc774161a80 2024-09-10T23:32:53.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.321+0000 7fc76b400640 1 -- 172.21.15.94:0/205819740 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc76c03a950 con 0x7fc774161a80 2024-09-10T23:32:53.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f29b4800640 1 -- 172.21.15.94:0/2399429926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2988001040 con 0x7f29ac15b110 2024-09-10T23:32:53.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f5282000640 1 --2- 172.21.15.94:0/909551370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c164310 0x7f527c152950 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f527400ef40 tx=0x7f5274010420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f0a7e000640 1 --2- 172.21.15.94:0/2212556058 >> [v2:172.21.15.94:6824/3187639367,v1:172.21.15.94:6825/3187639367] conn(0x7f0a6007a930 0x7f0a6007cd50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f5c08c00640 1 -- 172.21.15.94:0/3944900568 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5bf8003a00 con 0x7f5c0c161a80 2024-09-10T23:32:53.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f5279600640 1 -- 172.21.15.94:0/909551370 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527401d020 con 0x7f527c164310 2024-09-10T23:32:53.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7fc76b400640 1 --2- 172.21.15.94:0/205819740 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc7540725e0 0x7fc754074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7fc76b400640 1 -- 172.21.15.94:0/205819740 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc76c0941a0 con 0x7fc774161a80 2024-09-10T23:32:53.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f5279600640 1 -- 172.21.15.94:0/909551370 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f527400e470 con 0x7f527c164310 2024-09-10T23:32:53.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f5279600640 1 -- 172.21.15.94:0/909551370 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274019b10 con 0x7f527c164310 2024-09-10T23:32:53.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f5284000640 1 -- 172.21.15.94:0/909551370 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f527c1f5c90 con 0x7f527c164310 2024-09-10T23:32:53.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f5284000640 1 -- 172.21.15.94:0/909551370 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f527c09ba00 con 0x7f527c164310 2024-09-10T23:32:53.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7fc76b400640 1 --2- 172.21.15.94:0/205819740 >> [v2:172.21.15.94:6816/163947633,v1:172.21.15.94:6817/163947633] conn(0x7fc75407a7e0 0x7fc75407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f0a76a00640 1 -- 172.21.15.94:0/2212556058 <== osd.3 v2:172.21.15.94:6824/3187639367 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a80063640 con 0x7f0a6007a930 2024-09-10T23:32:53.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7fc773400640 1 --2- 172.21.15.94:0/205819740 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc7540725e0 0x7fc754074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7fc76b400640 1 -- 172.21.15.94:0/205819740 --> [v2:172.21.15.94:6816/163947633,v1:172.21.15.94:6817/163947633] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc75407d2b0 con 0x7fc75407a7e0 2024-09-10T23:32:53.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7fc76b400640 1 -- 172.21.15.94:0/205819740 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc76c094550 con 0x7fc774161a80 2024-09-10T23:32:53.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f5c08c00640 1 --2- 172.21.15.94:0/3944900568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5bec0725e0 0x7f5bec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f5c08c00640 1 -- 172.21.15.94:0/3944900568 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5bf8095ad0 con 0x7f5c0c161a80 2024-09-10T23:32:53.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f5c08c00640 1 --2- 172.21.15.94:0/3944900568 >> [v2:172.21.15.181:6826/1536008029,v1:172.21.15.181:6827/1536008029] conn(0x7f5bec07a800 0x7f5bec07cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f5c08c00640 1 -- 172.21.15.94:0/3944900568 --> [v2:172.21.15.181:6826/1536008029,v1:172.21.15.181:6827/1536008029] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5bf80044a0 con 0x7f5bec07a800 2024-09-10T23:32:53.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f5c08c00640 1 -- 172.21.15.94:0/3944900568 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5bf809a2a0 con 0x7f5c0c161a80 2024-09-10T23:32:53.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f5c11200640 1 --2- 172.21.15.94:0/3944900568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5bec0725e0 0x7f5bec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f5c0be00640 1 --2- 172.21.15.94:0/3944900568 >> [v2:172.21.15.181:6826/1536008029,v1:172.21.15.181:6827/1536008029] conn(0x7f5bec07a800 0x7f5bec07cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.325+0000 7f5284000640 1 -- 172.21.15.94:0/909551370 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f527c06ed80 con 0x7f527c164310 2024-09-10T23:32:53.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f5c11200640 1 --2- 172.21.15.94:0/3944900568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5bec0725e0 0x7f5bec074aa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c040af000 tx=0x7f5c0401b3b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7fa1da800640 1 -- 172.21.15.94:0/3795032633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1d406fc00 msgr2=0x7fa1d40c49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7fa1da800640 1 --2- 172.21.15.94:0/3795032633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1d406fc00 0x7fa1d40c49b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa1d415f840 tx=0x7fa1c001c8c0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f5c0be00640 1 --2- 172.21.15.94:0/3944900568 >> [v2:172.21.15.181:6826/1536008029,v1:172.21.15.181:6827/1536008029] conn(0x7f5bec07a800 0x7f5bec07cc20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7fc772a00640 1 --2- 172.21.15.94:0/205819740 >> [v2:172.21.15.94:6816/163947633,v1:172.21.15.94:6817/163947633] conn(0x7fc75407a7e0 0x7fc75407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f29a6000640 1 -- 172.21.15.94:0/2399429926 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f29a8013070 con 0x7f29ac15b110 2024-09-10T23:32:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7fc773400640 1 --2- 172.21.15.94:0/205819740 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc7540725e0 0x7fc754074aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7640af000 tx=0x7fc76401b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f5279600640 1 -- 172.21.15.94:0/909551370 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f527400c440 con 0x7f527c164310 2024-09-10T23:32:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7fa1da800640 1 -- 172.21.15.94:0/3795032633 shutdown_connections 2024-09-10T23:32:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7fa1da800640 1 --2- 172.21.15.94:0/3795032633 >> v2:172.21.15.181:3300/0 conn(0x7fa1d40cb4d0 0x7fa1d415b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7fa1da800640 1 --2- 172.21.15.94:0/3795032633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1d406fc00 0x7fa1d40c49b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f29a6000640 1 --2- 172.21.15.94:0/2399429926 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2990072630 0x7f2990074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f29a6000640 1 -- 172.21.15.94:0/2399429926 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f29a80938b0 con 0x7f29ac15b110 2024-09-10T23:32:53.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f5c08c00640 1 -- 172.21.15.94:0/3944900568 <== osd.5 v2:172.21.15.181:6826/1536008029 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bf80044a0 con 0x7f5bec07a800 2024-09-10T23:32:53.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7fa1da800640 1 -- 172.21.15.94:0/3795032633 >> 172.21.15.94:0/3795032633 conn(0x7fa1d40cfaf0 msgr2=0x7fa1d40cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f29a6000640 1 --2- 172.21.15.94:0/2399429926 >> [v2:172.21.15.181:6811/3568562877,v1:172.21.15.181:6813/3568562877] conn(0x7f299007a850 0x7f299007cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f29a6000640 1 -- 172.21.15.94:0/2399429926 --> [v2:172.21.15.181:6811/3568562877,v1:172.21.15.181:6813/3568562877] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f29a8022e80 con 0x7f299007a850 2024-09-10T23:32:53.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f29a6000640 1 -- 172.21.15.94:0/2399429926 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f29a8093d90 con 0x7f29ac15b110 2024-09-10T23:32:53.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f29b1e00640 1 --2- 172.21.15.94:0/2399429926 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2990072630 0x7f2990074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7fa1da800640 1 -- 172.21.15.94:0/3795032633 shutdown_connections 2024-09-10T23:32:53.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7fa1da800640 1 -- 172.21.15.94:0/3795032633 wait complete. 2024-09-10T23:32:53.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f29b1400640 1 --2- 172.21.15.94:0/2399429926 >> [v2:172.21.15.181:6811/3568562877,v1:172.21.15.181:6813/3568562877] conn(0x7f299007a850 0x7f299007cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7fc772a00640 1 --2- 172.21.15.94:0/205819740 >> [v2:172.21.15.94:6816/163947633,v1:172.21.15.94:6817/163947633] conn(0x7fc75407a7e0 0x7fc75407cc00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f5279600640 1 --2- 172.21.15.94:0/909551370 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f525c072660 0x7f525c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f5279600640 1 -- 172.21.15.94:0/909551370 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f527409cf10 con 0x7f527c164310 2024-09-10T23:32:53.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f5279600640 1 --2- 172.21.15.94:0/909551370 >> [v2:172.21.15.181:6810/955716519,v1:172.21.15.181:6812/955716519] conn(0x7f525c07a880 0x7f525c07cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f5279600640 1 -- 172.21.15.94:0/909551370 --> [v2:172.21.15.181:6810/955716519,v1:172.21.15.181:6812/955716519] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5274028e70 con 0x7f525c07a880 2024-09-10T23:32:53.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f5279600640 1 -- 172.21.15.94:0/909551370 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f527409d2c0 con 0x7f527c164310 2024-09-10T23:32:53.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f5281600640 1 --2- 172.21.15.94:0/909551370 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f525c072660 0x7f525c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f29b1400640 1 --2- 172.21.15.94:0/2399429926 >> [v2:172.21.15.181:6811/3568562877,v1:172.21.15.181:6813/3568562877] conn(0x7f299007a850 0x7f299007cc70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.329+0000 7f29b1e00640 1 --2- 172.21.15.94:0/2399429926 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2990072630 0x7f2990074af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f29a00ae000 tx=0x7f29a001b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.333+0000 7f5280c00640 1 --2- 172.21.15.94:0/909551370 >> [v2:172.21.15.181:6810/955716519,v1:172.21.15.181:6812/955716519] conn(0x7f525c07a880 0x7f525c07cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.333+0000 7fc76b400640 1 -- 172.21.15.94:0/205819740 <== osd.2 v2:172.21.15.94:6816/163947633 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc75407d2b0 con 0x7fc75407a7e0 2024-09-10T23:32:53.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.333+0000 7f5281600640 1 --2- 172.21.15.94:0/909551370 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f525c072660 0x7f525c074b20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5270000f70 tx=0x7f5270009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.333+0000 7f5280c00640 1 --2- 172.21.15.94:0/909551370 >> [v2:172.21.15.181:6810/955716519,v1:172.21.15.181:6812/955716519] conn(0x7f525c07a880 0x7f525c07cca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.333+0000 7f5279600640 1 -- 172.21.15.94:0/909551370 <== osd.4 v2:172.21.15.181:6810/955716519 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5274028e70 con 0x7f525c07a880 2024-09-10T23:32:53.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.333+0000 7fa1da800640 1 Processor -- start 2024-09-10T23:32:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.333+0000 7fa1da800640 1 -- start start 2024-09-10T23:32:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.333+0000 7fa1da800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1d406fc00 0x7fa1d40b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.333+0000 7fa1da800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d40cb4d0 0x7fa1d41ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.333+0000 7fa1da800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1d4158790 0x7fa1d4073a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.333+0000 7fa1da800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d4168010 con 0x7fa1d406fc00 2024-09-10T23:32:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.333+0000 7fa1da800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d4167e90 con 0x7fa1d40cb4d0 2024-09-10T23:32:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.333+0000 7fa1da800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d4168190 con 0x7fa1d4158790 2024-09-10T23:32:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.337+0000 7f29a6000640 1 -- 172.21.15.94:0/2399429926 <== osd.7 v2:172.21.15.181:6811/3568562877 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f29a8022e80 con 0x7f299007a850 2024-09-10T23:32:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.341+0000 7fa1d2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d40cb4d0 0x7fa1d41ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.341+0000 7fa1d3e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1d4158790 0x7fa1d4073a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.341+0000 7fa1d2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d40cb4d0 0x7fa1d41ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:54606/0 (socket says 172.21.15.94:54606) 2024-09-10T23:32:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.341+0000 7fa1d2a00640 1 -- 172.21.15.94:0/2907236980 learned_addr learned my addr 172.21.15.94:0/2907236980 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.341+0000 7fa1d3e00640 1 -- 172.21.15.94:0/2907236980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d40cb4d0 msgr2=0x7fa1d41ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.341+0000 7fa1d3e00640 1 --2- 172.21.15.94:0/2907236980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d40cb4d0 0x7fa1d41ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.341+0000 7fa1d3e00640 1 -- 172.21.15.94:0/2907236980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1d406fc00 msgr2=0x7fa1d40b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.341+0000 7fa1d3e00640 1 --2- 172.21.15.94:0/2907236980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1d406fc00 0x7fa1d40b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.341+0000 7fa1d3e00640 1 -- 172.21.15.94:0/2907236980 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d41fce40 con 0x7fa1d4158790 2024-09-10T23:32:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.341+0000 7fa1d3e00640 1 --2- 172.21.15.94:0/2907236980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1d4158790 0x7fa1d4073a80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00ea90 tx=0x7fa1cc00ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.341+0000 7fa1cb400640 1 -- 172.21.15.94:0/2907236980 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc00cea0 con 0x7fa1d4158790 2024-09-10T23:32:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.341+0000 7fa1cb400640 1 -- 172.21.15.94:0/2907236980 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1cc004c20 con 0x7fa1d4158790 2024-09-10T23:32:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.341+0000 7fa1cb400640 1 -- 172.21.15.94:0/2907236980 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc005020 con 0x7fa1d4158790 2024-09-10T23:32:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7fe2b9200640 1 -- 172.21.15.94:0/3838863128 --> [v2:172.21.15.94:6803/3936794679,v1:172.21.15.94:6805/3936794679] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe2ac02ab40 con 0x7fe29407a930 2024-09-10T23:32:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7fa1da800640 1 -- 172.21.15.94:0/2907236980 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d42003a0 con 0x7fa1d4158790 2024-09-10T23:32:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7fa1da800640 1 -- 172.21.15.94:0/2907236980 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d41fc5a0 con 0x7fa1d4158790 2024-09-10T23:32:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7fe2a7400640 1 -- 172.21.15.94:0/3838863128 <== osd.1 v2:172.21.15.94:6803/3936794679 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe2ac02ab40 con 0x7fe29407a930 2024-09-10T23:32:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7fa1da800640 1 -- 172.21.15.94:0/2907236980 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa1d40c49b0 con 0x7fa1d4158790 2024-09-10T23:32:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7fe2b9200640 1 -- 172.21.15.94:0/3838863128 >> [v2:172.21.15.94:6803/3936794679,v1:172.21.15.94:6805/3936794679] conn(0x7fe29407a930 msgr2=0x7fe29407cd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7f5aa8a00640 1 -- 172.21.15.94:0/1468590985 --> [v2:172.21.15.94:6800/576048946,v1:172.21.15.94:6801/576048946] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5aa41624b0 con 0x7f5a8007a800 2024-09-10T23:32:53.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7fe2b9200640 1 --2- 172.21.15.94:0/3838863128 >> [v2:172.21.15.94:6803/3936794679,v1:172.21.15.94:6805/3936794679] conn(0x7fe29407a930 0x7fe29407cd50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7fe2b9200640 1 -- 172.21.15.94:0/3838863128 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe2940726b0 msgr2=0x7fe294074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7fe2b9200640 1 --2- 172.21.15.94:0/3838863128 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe2940726b0 0x7fe294074b70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac04b1a0 tx=0x7fe2a0008040 comp rx=0 tx=0).stop 2024-09-10T23:32:53.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7fe2b9200640 1 -- 172.21.15.94:0/3838863128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac00a480 msgr2=0x7fe2ac0de690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7fe2b9200640 1 --2- 172.21.15.94:0/3838863128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2ac00a480 0x7fe2ac0de690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8019450 tx=0x7fe2a80051a0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7fe2b3e00640 1 -- 172.21.15.94:0/3838863128 reap_dead start 2024-09-10T23:32:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7fe2b9200640 1 -- 172.21.15.94:0/3838863128 shutdown_connections 2024-09-10T23:32:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7fe2b9200640 1 -- 172.21.15.94:0/3838863128 >> 172.21.15.94:0/3838863128 conn(0x7fe2ac054570 msgr2=0x7fe2ac0db420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7fe2b9200640 1 -- 172.21.15.94:0/3838863128 shutdown_connections 2024-09-10T23:32:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.345+0000 7fe2b9200640 1 -- 172.21.15.94:0/3838863128 wait complete. 2024-09-10T23:32:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7fa1cb400640 1 -- 172.21.15.94:0/2907236980 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa1cc020030 con 0x7fa1d4158790 2024-09-10T23:32:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7fa1cb400640 1 --2- 172.21.15.94:0/2907236980 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa1b4072630 0x7fa1b4074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7f5a9b400640 1 -- 172.21.15.94:0/1468590985 <== osd.0 v2:172.21.15.94:6800/576048946 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5aa41624b0 con 0x7f5a8007a800 2024-09-10T23:32:53.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7f5aa8a00640 1 -- 172.21.15.94:0/1468590985 >> [v2:172.21.15.94:6800/576048946,v1:172.21.15.94:6801/576048946] conn(0x7f5a8007a800 msgr2=0x7f5a8007cc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7f5aa8a00640 1 --2- 172.21.15.94:0/1468590985 >> [v2:172.21.15.94:6800/576048946,v1:172.21.15.94:6801/576048946] conn(0x7f5a8007a800 0x7f5a8007cc20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7f5aa8a00640 1 -- 172.21.15.94:0/1468590985 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5a800725e0 msgr2=0x7f5a80074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7f5aa8a00640 1 --2- 172.21.15.94:0/1468590985 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5a800725e0 0x7f5a80074aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c007fd0 tx=0x7f5a8c0078e0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7f5aa8a00640 1 -- 172.21.15.94:0/1468590985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5aa406ce80 msgr2=0x7f5aa40b2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7f5aa8a00640 1 --2- 172.21.15.94:0/1468590985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5aa406ce80 0x7f5aa40b2ed0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400e9f0 tx=0x7f5a9400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7f0a85c00640 1 -- 172.21.15.94:0/2212556058 --> [v2:172.21.15.94:6824/3187639367,v1:172.21.15.94:6825/3187639367] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0a78007320 con 0x7f0a6007a930 2024-09-10T23:32:53.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7fa1cb400640 1 -- 172.21.15.94:0/2907236980 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa1cc093c50 con 0x7fa1d4158790 2024-09-10T23:32:53.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7fa1d3400640 1 --2- 172.21.15.94:0/2907236980 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa1b4072630 0x7fa1b4074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7fa1cb400640 1 --2- 172.21.15.94:0/2907236980 >> [v2:172.21.15.181:6802/1977708519,v1:172.21.15.181:6803/1977708519] conn(0x7fa1b407a7d0 0x7fa1b407cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7fa1cb400640 1 -- 172.21.15.94:0/2907236980 --> [v2:172.21.15.181:6802/1977708519,v1:172.21.15.181:6803/1977708519] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa1cc01fe80 con 0x7fa1b407a7d0 2024-09-10T23:32:53.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7f5aa3200640 1 -- 172.21.15.94:0/1468590985 reap_dead start 2024-09-10T23:32:53.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7f5aa8a00640 1 -- 172.21.15.94:0/1468590985 shutdown_connections 2024-09-10T23:32:53.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7f5aa8a00640 1 -- 172.21.15.94:0/1468590985 >> 172.21.15.94:0/1468590985 conn(0x7f5aa40cf8d0 msgr2=0x7f5aa416c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7fa1cb400640 1 -- 172.21.15.94:0/2907236980 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa1cc05b360 con 0x7fa1d4158790 2024-09-10T23:32:53.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.349+0000 7fa1d2a00640 1 --2- 172.21.15.94:0/2907236980 >> [v2:172.21.15.181:6802/1977708519,v1:172.21.15.181:6803/1977708519] conn(0x7fa1b407a7d0 0x7fa1b407cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7fa1d3400640 1 --2- 172.21.15.94:0/2907236980 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa1b4072630 0x7fa1b4074af0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000f70 tx=0x7fa1c40079e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7f0a76a00640 1 -- 172.21.15.94:0/2212556058 <== osd.3 v2:172.21.15.94:6824/3187639367 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0a78007320 con 0x7f0a6007a930 2024-09-10T23:32:53.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7f5aa8a00640 1 -- 172.21.15.94:0/1468590985 shutdown_connections 2024-09-10T23:32:53.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7f5aa8a00640 1 -- 172.21.15.94:0/1468590985 wait complete. 2024-09-10T23:32:53.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7fa1d2a00640 1 --2- 172.21.15.94:0/2907236980 >> [v2:172.21.15.181:6802/1977708519,v1:172.21.15.181:6803/1977708519] conn(0x7fa1b407a7d0 0x7fa1b407cbf0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7f0a85c00640 1 -- 172.21.15.94:0/2212556058 >> [v2:172.21.15.94:6824/3187639367,v1:172.21.15.94:6825/3187639367] conn(0x7f0a6007a930 msgr2=0x7f0a6007cd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7f0a85c00640 1 --2- 172.21.15.94:0/2212556058 >> [v2:172.21.15.94:6824/3187639367,v1:172.21.15.94:6825/3187639367] conn(0x7f0a6007a930 0x7f0a6007cd50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7f0a85c00640 1 -- 172.21.15.94:0/2212556058 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0a600726b0 msgr2=0x7f0a60074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7f0a85c00640 1 --2- 172.21.15.94:0/2212556058 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0a600726b0 0x7f0a60074b70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a700011d0 tx=0x7f0a70009040 comp rx=0 tx=0).stop 2024-09-10T23:32:53.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7f0a85c00640 1 -- 172.21.15.94:0/2212556058 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a780e0a50 msgr2=0x7f0a78053640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7f0a85c00640 1 --2- 172.21.15.94:0/2212556058 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a780e0a50 0x7f0a78053640 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0a8006c150 tx=0x7f0a80125ab0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7f0a7f400640 1 -- 172.21.15.94:0/2212556058 reap_dead start 2024-09-10T23:32:53.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7fa1cb400640 1 -- 172.21.15.94:0/2907236980 <== osd.6 v2:172.21.15.181:6802/1977708519 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa1cc01fe80 con 0x7fa1b407a7d0 2024-09-10T23:32:53.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7f0a85c00640 1 -- 172.21.15.94:0/2212556058 shutdown_connections 2024-09-10T23:32:53.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7f0a85c00640 1 -- 172.21.15.94:0/2212556058 >> 172.21.15.94:0/2212556058 conn(0x7f0a7801d780 msgr2=0x7f0a780518f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7f0a85c00640 1 -- 172.21.15.94:0/2212556058 shutdown_connections 2024-09-10T23:32:53.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.353+0000 7f0a85c00640 1 -- 172.21.15.94:0/2212556058 wait complete. 2024-09-10T23:32:53.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.361+0000 7f5c13c00640 1 -- 172.21.15.94:0/3944900568 --> [v2:172.21.15.181:6826/1536008029,v1:172.21.15.181:6827/1536008029] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5c0c0b14c0 con 0x7f5bec07a800 2024-09-10T23:32:53.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.361+0000 7fc77a200640 1 -- 172.21.15.94:0/205819740 --> [v2:172.21.15.94:6816/163947633,v1:172.21.15.94:6817/163947633] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc7740cff70 con 0x7fc75407a7e0 2024-09-10T23:32:53.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.361+0000 7f5c08c00640 1 -- 172.21.15.94:0/3944900568 <== osd.5 v2:172.21.15.181:6826/1536008029 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5c0c0b14c0 con 0x7f5bec07a800 2024-09-10T23:32:53.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.361+0000 7fc76b400640 1 -- 172.21.15.94:0/205819740 <== osd.2 v2:172.21.15.94:6816/163947633 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc7740cff70 con 0x7fc75407a7e0 2024-09-10T23:32:53.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.361+0000 7f5c13c00640 1 -- 172.21.15.94:0/3944900568 >> [v2:172.21.15.181:6826/1536008029,v1:172.21.15.181:6827/1536008029] conn(0x7f5bec07a800 msgr2=0x7f5bec07cc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.361+0000 7f5c13c00640 1 --2- 172.21.15.94:0/3944900568 >> [v2:172.21.15.181:6826/1536008029,v1:172.21.15.181:6827/1536008029] conn(0x7f5bec07a800 0x7f5bec07cc20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.361+0000 7f5c13c00640 1 -- 172.21.15.94:0/3944900568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5bec0725e0 msgr2=0x7f5bec074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.361+0000 7f5c13c00640 1 --2- 172.21.15.94:0/3944900568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5bec0725e0 0x7f5bec074aa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c040af000 tx=0x7f5c0401b3b0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.361+0000 7f5c13c00640 1 -- 172.21.15.94:0/3944900568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c0c161a80 msgr2=0x7f5c0c156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.361+0000 7f5c13c00640 1 --2- 172.21.15.94:0/3944900568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c0c161a80 0x7f5c0c156840 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800c210 tx=0x7f5bf800c6d0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.361+0000 7f5c11c00640 1 -- 172.21.15.94:0/3944900568 reap_dead start 2024-09-10T23:32:53.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.361+0000 7f5c13c00640 1 -- 172.21.15.94:0/3944900568 shutdown_connections 2024-09-10T23:32:53.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.361+0000 7f5c13c00640 1 -- 172.21.15.94:0/3944900568 >> 172.21.15.94:0/3944900568 conn(0x7f5c0c0cfaf0 msgr2=0x7f5c0c15ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.361+0000 7f5284000640 1 -- 172.21.15.94:0/909551370 --> [v2:172.21.15.181:6810/955716519,v1:172.21.15.181:6812/955716519] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f527c0cc3d0 con 0x7f525c07a880 2024-09-10T23:32:53.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7f5c13c00640 1 -- 172.21.15.94:0/3944900568 shutdown_connections 2024-09-10T23:32:53.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7fc77a200640 1 -- 172.21.15.94:0/205819740 >> [v2:172.21.15.94:6816/163947633,v1:172.21.15.94:6817/163947633] conn(0x7fc75407a7e0 msgr2=0x7fc75407cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7fc77a200640 1 --2- 172.21.15.94:0/205819740 >> [v2:172.21.15.94:6816/163947633,v1:172.21.15.94:6817/163947633] conn(0x7fc75407a7e0 0x7fc75407cc00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7fc77a200640 1 -- 172.21.15.94:0/205819740 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc7540725e0 msgr2=0x7fc754074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7fc77a200640 1 --2- 172.21.15.94:0/205819740 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc7540725e0 0x7fc754074aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7640af000 tx=0x7fc76401b040 comp rx=0 tx=0).stop 2024-09-10T23:32:53.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7fc77a200640 1 -- 172.21.15.94:0/205819740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc774161a80 msgr2=0x7fc77406eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7fc77a200640 1 --2- 172.21.15.94:0/205819740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc774161a80 0x7fc77406eb20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00eb20 tx=0x7fc76c00c2d0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7f5279600640 1 -- 172.21.15.94:0/909551370 <== osd.4 v2:172.21.15.181:6810/955716519 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f527c0cc3d0 con 0x7f525c07a880 2024-09-10T23:32:53.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7f5c13c00640 1 -- 172.21.15.94:0/3944900568 wait complete. 2024-09-10T23:32:53.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7f5284000640 1 -- 172.21.15.94:0/909551370 >> [v2:172.21.15.181:6810/955716519,v1:172.21.15.181:6812/955716519] conn(0x7f525c07a880 msgr2=0x7f525c07cca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7f5284000640 1 --2- 172.21.15.94:0/909551370 >> [v2:172.21.15.181:6810/955716519,v1:172.21.15.181:6812/955716519] conn(0x7f525c07a880 0x7f525c07cca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7f5284000640 1 -- 172.21.15.94:0/909551370 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f525c072660 msgr2=0x7f525c074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7f5284000640 1 --2- 172.21.15.94:0/909551370 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f525c072660 0x7f525c074b20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5270000f70 tx=0x7f5270009040 comp rx=0 tx=0).stop 2024-09-10T23:32:53.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7f5284000640 1 -- 172.21.15.94:0/909551370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c164310 msgr2=0x7f527c152950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7f5284000640 1 --2- 172.21.15.94:0/909551370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527c164310 0x7f527c152950 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f527400ef40 tx=0x7f5274010420 comp rx=0 tx=0).stop 2024-09-10T23:32:53.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7fc773e00640 1 -- 172.21.15.94:0/205819740 reap_dead start 2024-09-10T23:32:53.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7fc77a200640 1 -- 172.21.15.94:0/205819740 shutdown_connections 2024-09-10T23:32:53.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7fc77a200640 1 -- 172.21.15.94:0/205819740 >> 172.21.15.94:0/205819740 conn(0x7fc7740cfaf0 msgr2=0x7fc7740c4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7f5282000640 1 -- 172.21.15.94:0/909551370 reap_dead start 2024-09-10T23:32:53.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7f5284000640 1 -- 172.21.15.94:0/909551370 shutdown_connections 2024-09-10T23:32:53.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7f5284000640 1 -- 172.21.15.94:0/909551370 >> 172.21.15.94:0/909551370 conn(0x7f527c0cff50 msgr2=0x7f527c155bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7fc77a200640 1 -- 172.21.15.94:0/205819740 shutdown_connections 2024-09-10T23:32:53.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7f29b4800640 1 -- 172.21.15.94:0/2399429926 --> [v2:172.21.15.181:6811/3568562877,v1:172.21.15.181:6813/3568562877] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2988002de0 con 0x7f299007a850 2024-09-10T23:32:53.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7fc77a200640 1 -- 172.21.15.94:0/205819740 wait complete. 2024-09-10T23:32:53.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7f5284000640 1 -- 172.21.15.94:0/909551370 shutdown_connections 2024-09-10T23:32:53.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7f5284000640 1 -- 172.21.15.94:0/909551370 wait complete. 2024-09-10T23:32:53.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.365+0000 7f29a6000640 1 -- 172.21.15.94:0/2399429926 <== osd.7 v2:172.21.15.181:6811/3568562877 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2988002de0 con 0x7f299007a850 2024-09-10T23:32:53.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.373+0000 7f29b4800640 1 -- 172.21.15.94:0/2399429926 >> [v2:172.21.15.181:6811/3568562877,v1:172.21.15.181:6813/3568562877] conn(0x7f299007a850 msgr2=0x7f299007cc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.373+0000 7f29b4800640 1 --2- 172.21.15.94:0/2399429926 >> [v2:172.21.15.181:6811/3568562877,v1:172.21.15.181:6813/3568562877] conn(0x7f299007a850 0x7f299007cc70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.373+0000 7f29b4800640 1 -- 172.21.15.94:0/2399429926 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2990072630 msgr2=0x7f2990074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.373+0000 7f29b4800640 1 --2- 172.21.15.94:0/2399429926 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2990072630 0x7f2990074af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f29a00ae000 tx=0x7f29a001b060 comp rx=0 tx=0).stop 2024-09-10T23:32:53.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.373+0000 7f29b4800640 1 -- 172.21.15.94:0/2399429926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ac15b110 msgr2=0x7f29ac1571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.373+0000 7f29b4800640 1 --2- 172.21.15.94:0/2399429926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ac15b110 0x7f29ac1571f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f29a800a660 tx=0x7f29a800ab20 comp rx=0 tx=0).stop 2024-09-10T23:32:53.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.373+0000 7f29b2800640 1 -- 172.21.15.94:0/2399429926 reap_dead start 2024-09-10T23:32:53.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.373+0000 7f29b4800640 1 -- 172.21.15.94:0/2399429926 shutdown_connections 2024-09-10T23:32:53.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.373+0000 7f29b4800640 1 -- 172.21.15.94:0/2399429926 >> 172.21.15.94:0/2399429926 conn(0x7f29ac0cfaf0 msgr2=0x7f29ac0c56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.373+0000 7f29b4800640 1 -- 172.21.15.94:0/2399429926 shutdown_connections 2024-09-10T23:32:53.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.373+0000 7f29b4800640 1 -- 172.21.15.94:0/2399429926 wait complete. 2024-09-10T23:32:53.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.381+0000 7fa1da800640 1 -- 172.21.15.94:0/2907236980 --> [v2:172.21.15.181:6802/1977708519,v1:172.21.15.181:6803/1977708519] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa1d40724d0 con 0x7fa1b407a7d0 2024-09-10T23:32:53.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.381+0000 7fa1cb400640 1 -- 172.21.15.94:0/2907236980 <== osd.6 v2:172.21.15.181:6802/1977708519 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa1d40724d0 con 0x7fa1b407a7d0 2024-09-10T23:32:53.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.385+0000 7fa1c8c00640 1 -- 172.21.15.94:0/2907236980 >> [v2:172.21.15.181:6802/1977708519,v1:172.21.15.181:6803/1977708519] conn(0x7fa1b407a7d0 msgr2=0x7fa1b407cbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.385+0000 7fa1c8c00640 1 --2- 172.21.15.94:0/2907236980 >> [v2:172.21.15.181:6802/1977708519,v1:172.21.15.181:6803/1977708519] conn(0x7fa1b407a7d0 0x7fa1b407cbf0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.385+0000 7fa1c8c00640 1 -- 172.21.15.94:0/2907236980 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa1b4072630 msgr2=0x7fa1b4074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.385+0000 7fa1c8c00640 1 --2- 172.21.15.94:0/2907236980 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa1b4072630 0x7fa1b4074af0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000f70 tx=0x7fa1c40079e0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.385+0000 7fa1c8c00640 1 -- 172.21.15.94:0/2907236980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1d4158790 msgr2=0x7fa1d4073a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.385+0000 7fa1c8c00640 1 --2- 172.21.15.94:0/2907236980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1d4158790 0x7fa1d4073a80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00ea90 tx=0x7fa1cc00ef50 comp rx=0 tx=0).stop 2024-09-10T23:32:53.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.385+0000 7fa1d3e00640 1 -- 172.21.15.94:0/2907236980 reap_dead start 2024-09-10T23:32:53.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.385+0000 7fa1c8c00640 1 -- 172.21.15.94:0/2907236980 shutdown_connections 2024-09-10T23:32:53.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.385+0000 7fa1c8c00640 1 -- 172.21.15.94:0/2907236980 >> 172.21.15.94:0/2907236980 conn(0x7fa1d40cfaf0 msgr2=0x7fa1d40aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.388 INFO:teuthology.orchestra.run.smithi094.stdout:64424509447 2024-09-10T23:32:53.389 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T23:32:53.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.385+0000 7fa1c8c00640 1 -- 172.21.15.94:0/2907236980 shutdown_connections 2024-09-10T23:32:53.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.385+0000 7fa1c8c00640 1 -- 172.21.15.94:0/2907236980 wait complete. 2024-09-10T23:32:53.390 INFO:teuthology.orchestra.run.smithi094.stdout:64424509447 2024-09-10T23:32:53.390 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T23:32:53.401 INFO:teuthology.orchestra.run.smithi094.stdout:64424509447 2024-09-10T23:32:53.401 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T23:32:53.410 INFO:teuthology.orchestra.run.smithi094.stdout:64424509448 2024-09-10T23:32:53.410 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T23:32:53.411 INFO:teuthology.orchestra.run.smithi094.stdout:64424509448 2024-09-10T23:32:53.411 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T23:32:53.414 INFO:teuthology.orchestra.run.smithi094.stdout:64424509446 2024-09-10T23:32:53.414 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T23:32:53.422 INFO:teuthology.orchestra.run.smithi094.stdout:64424509449 2024-09-10T23:32:53.422 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T23:32:53.428 INFO:teuthology.orchestra.run.smithi094.stdout:60129542152 2024-09-10T23:32:53.429 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T23:32:53.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.613+0000 7fe5f0a00640 1 -- 172.21.15.94:0/184003152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40e6d00 msgr2=0x7fe5e4006940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.613+0000 7fe5f0a00640 1 --2- 172.21.15.94:0/184003152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40e6d00 0x7fe5e4006940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec06c460 tx=0x7fe5ec098300 comp rx=0 tx=0).stop 2024-09-10T23:32:53.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.613+0000 7fe5f0a00640 1 -- 172.21.15.94:0/184003152 shutdown_connections 2024-09-10T23:32:53.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.613+0000 7fe5f0a00640 1 --2- 172.21.15.94:0/184003152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40e6d00 0x7fe5e4006940 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.613+0000 7fe5f0a00640 1 --2- 172.21.15.94:0/184003152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40578a0 0x7fe5e403bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.613+0000 7fe5f0a00640 1 -- 172.21.15.94:0/184003152 >> 172.21.15.94:0/184003152 conn(0x7fe5e4055360 msgr2=0x7fe5e4055770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.613+0000 7fe5f0a00640 1 -- 172.21.15.94:0/184003152 shutdown_connections 2024-09-10T23:32:53.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.613+0000 7fe5f0a00640 1 -- 172.21.15.94:0/184003152 wait complete. 2024-09-10T23:32:53.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.617+0000 7feb82c00640 1 -- 172.21.15.94:0/2544399360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb7c15baa0 msgr2=0x7feb7c1584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.617+0000 7feb82c00640 1 --2- 172.21.15.94:0/2544399360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb7c15baa0 0x7feb7c1584b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0d1a20 tx=0x7feb6801c8c0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.617+0000 7feb82c00640 1 -- 172.21.15.94:0/2544399360 shutdown_connections 2024-09-10T23:32:53.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.617+0000 7feb82c00640 1 --2- 172.21.15.94:0/2544399360 >> v2:172.21.15.181:3300/0 conn(0x7feb7c0d0920 0x7feb7c0d0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.617+0000 7feb82c00640 1 --2- 172.21.15.94:0/2544399360 >> v2:172.21.15.94:3300/0 conn(0x7feb7c0ce890 0x7feb7c073970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.617+0000 7feb82c00640 1 --2- 172.21.15.94:0/2544399360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb7c15baa0 0x7feb7c1584b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.617+0000 7fe5f0a00640 1 Processor -- start 2024-09-10T23:32:53.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.617+0000 7feb82c00640 1 -- 172.21.15.94:0/2544399360 >> 172.21.15.94:0/2544399360 conn(0x7feb7c06dd30 msgr2=0x7feb7c06e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.617+0000 7feb82c00640 1 -- 172.21.15.94:0/2544399360 shutdown_connections 2024-09-10T23:32:53.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.617+0000 7fe5f0a00640 1 -- start start 2024-09-10T23:32:53.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5f0a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40578a0 0x7fe5e40d7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5f0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e40e6920 0x7fe5e40ded40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7f5888a00640 1 -- 172.21.15.94:0/1609324520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f587c01c370 msgr2=0x7f587c035ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5f0a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40e6d00 0x7fe5e400b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5f0a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e40de760 con 0x7fe5e40e6920 2024-09-10T23:32:53.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5f0a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e40de5e0 con 0x7fe5e40578a0 2024-09-10T23:32:53.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5f0a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe5e40de8e0 con 0x7fe5e40e6d00 2024-09-10T23:32:53.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.617+0000 7feb82c00640 1 -- 172.21.15.94:0/2544399360 wait complete. 2024-09-10T23:32:53.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7f5888a00640 1 --2- 172.21.15.94:0/1609324520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f587c01c370 0x7f587c035ff0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5870009840 tx=0x7f587001c760 comp rx=0 tx=0).stop 2024-09-10T23:32:53.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5eb200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40e6d00 0x7fe5e400b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5eb200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40e6d00 0x7fe5e400b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:50738/0 (socket says 172.21.15.94:50738) 2024-09-10T23:32:53.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5eb200640 1 -- 172.21.15.94:0/898141537 learned_addr learned my addr 172.21.15.94:0/898141537 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:53.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5ea800640 1 --2- 172.21.15.94:0/898141537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40578a0 0x7fe5e40d7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5e9e00640 1 --2- 172.21.15.94:0/898141537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e40e6920 0x7fe5e40ded40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5eb200640 1 -- 172.21.15.94:0/898141537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40578a0 msgr2=0x7fe5e40d7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5eb200640 1 --2- 172.21.15.94:0/898141537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40578a0 0x7fe5e40d7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5eb200640 1 -- 172.21.15.94:0/898141537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e40e6920 msgr2=0x7fe5e40ded40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5eb200640 1 --2- 172.21.15.94:0/898141537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e40e6920 0x7fe5e40ded40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5eb200640 1 -- 172.21.15.94:0/898141537 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e400aa00 con 0x7fe5e40e6d00 2024-09-10T23:32:53.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5e9e00640 1 --2- 172.21.15.94:0/898141537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e40e6920 0x7fe5e40ded40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:53.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7f5888a00640 1 -- 172.21.15.94:0/1609324520 shutdown_connections 2024-09-10T23:32:53.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7feb82c00640 1 Processor -- start 2024-09-10T23:32:53.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7f5888a00640 1 --2- 172.21.15.94:0/1609324520 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f587c009aa0 0x7f587c036530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7feb82c00640 1 -- start start 2024-09-10T23:32:53.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5eb200640 1 --2- 172.21.15.94:0/898141537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40e6d00 0x7fe5e400b7c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec06b9b0 tx=0x7fe5ec063640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7f5888a00640 1 --2- 172.21.15.94:0/1609324520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f587c01c370 0x7f587c035ff0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7f5888a00640 1 -- 172.21.15.94:0/1609324520 >> 172.21.15.94:0/1609324520 conn(0x7f587c054570 msgr2=0x7f587c054980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5e2a00640 1 -- 172.21.15.94:0/898141537 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec12d040 con 0x7fe5e40e6d00 2024-09-10T23:32:53.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5e2a00640 1 -- 172.21.15.94:0/898141537 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5ec063a00 con 0x7fe5e40e6d00 2024-09-10T23:32:53.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5e2a00640 1 -- 172.21.15.94:0/898141537 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec063e50 con 0x7fe5e40e6d00 2024-09-10T23:32:53.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5f0a00640 1 -- 172.21.15.94:0/898141537 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e400ace0 con 0x7fe5e40e6d00 2024-09-10T23:32:53.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7fe5f0a00640 1 -- 172.21.15.94:0/898141537 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e400b2a0 con 0x7fe5e40e6d00 2024-09-10T23:32:53.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.621+0000 7feb82c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb7c0ce890 0x7feb7c154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb82c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb7c0d0920 0x7feb7c153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb82c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb7c15baa0 0x7feb7c14f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb82c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c15e610 con 0x7feb7c0d0920 2024-09-10T23:32:53.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb82c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c15e490 con 0x7feb7c15baa0 2024-09-10T23:32:53.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb82c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7feb7c15e790 con 0x7feb7c0ce890 2024-09-10T23:32:53.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7f5888a00640 1 -- 172.21.15.94:0/1609324520 shutdown_connections 2024-09-10T23:32:53.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb82200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb7c15baa0 0x7feb7c14f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb82200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb7c15baa0 0x7feb7c14f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:54662/0 (socket says 172.21.15.94:54662) 2024-09-10T23:32:53.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb82200640 1 -- 172.21.15.94:0/4115435700 learned_addr learned my addr 172.21.15.94:0/4115435700 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:53.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb80e00640 1 --2- 172.21.15.94:0/4115435700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb7c0d0920 0x7feb7c153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb82200640 1 -- 172.21.15.94:0/4115435700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb7c0ce890 msgr2=0x7feb7c154eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T23:32:53.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb82200640 1 --2- 172.21.15.94:0/4115435700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb7c0ce890 0x7feb7c154eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb82200640 1 -- 172.21.15.94:0/4115435700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb7c0d0920 msgr2=0x7feb7c153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb82200640 1 --2- 172.21.15.94:0/4115435700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb7c0d0920 0x7feb7c153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb82200640 1 -- 172.21.15.94:0/4115435700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7c0b1a40 con 0x7feb7c15baa0 2024-09-10T23:32:53.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb82200640 1 --2- 172.21.15.94:0/4115435700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb7c15baa0 0x7feb7c14f190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feb7000e8c0 tx=0x7feb7000ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb79600640 1 -- 172.21.15.94:0/4115435700 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7000ccc0 con 0x7feb7c15baa0 2024-09-10T23:32:53.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb79600640 1 -- 172.21.15.94:0/4115435700 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb7000ce60 con 0x7feb7c15baa0 2024-09-10T23:32:53.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb82c00640 1 -- 172.21.15.94:0/4115435700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7c153a10 con 0x7feb7c15baa0 2024-09-10T23:32:53.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb82c00640 1 -- 172.21.15.94:0/4115435700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb7c0b1fc0 con 0x7feb7c15baa0 2024-09-10T23:32:53.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7f5888a00640 1 -- 172.21.15.94:0/1609324520 wait complete. 2024-09-10T23:32:53.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7f5888a00640 1 Processor -- start 2024-09-10T23:32:53.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.625+0000 7feb79600640 1 -- 172.21.15.94:0/4115435700 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70005280 con 0x7feb7c15baa0 2024-09-10T23:32:53.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5888a00640 1 -- start start 2024-09-10T23:32:53.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7feb82c00640 1 -- 172.21.15.94:0/4115435700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb580052a0 con 0x7feb7c15baa0 2024-09-10T23:32:53.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7fe5f0a00640 1 -- 172.21.15.94:0/898141537 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e400b4a0 con 0x7fe5e40e6d00 2024-09-10T23:32:53.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7fe5e2a00640 1 -- 172.21.15.94:0/898141537 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe5ec128430 con 0x7fe5e40e6d00 2024-09-10T23:32:53.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5888a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c009aa0 0x7f587c04a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5888a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f587c00a480 0x7f587c01e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5888a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f587c01c370 0x7f587c01e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5888a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f587c037e70 con 0x7f587c009aa0 2024-09-10T23:32:53.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5888a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f587c037cf0 con 0x7f587c00a480 2024-09-10T23:32:53.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5888a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f587c037ff0 con 0x7f587c01c370 2024-09-10T23:32:53.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5883200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f587c01c370 0x7f587c01e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5882800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c009aa0 0x7f587c04a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5882800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c009aa0 0x7f587c04a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51134/0 (socket says 172.21.15.94:51134) 2024-09-10T23:32:53.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5882800640 1 -- 172.21.15.94:0/2398497356 learned_addr learned my addr 172.21.15.94:0/2398497356 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:53.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5882800640 1 -- 172.21.15.94:0/2398497356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f587c01c370 msgr2=0x7f587c01e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5882800640 1 --2- 172.21.15.94:0/2398497356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f587c01c370 0x7f587c01e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5882800640 1 -- 172.21.15.94:0/2398497356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f587c00a480 msgr2=0x7f587c01e1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:32:53.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5882800640 1 --2- 172.21.15.94:0/2398497356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f587c00a480 0x7f587c01e1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5882800640 1 -- 172.21.15.94:0/2398497356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c021390 con 0x7f587c009aa0 2024-09-10T23:32:53.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5882800640 1 --2- 172.21.15.94:0/2398497356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c009aa0 0x7f587c04a570 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f587400c4d0 tx=0x7f587400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7fe5e2a00640 1 --2- 172.21.15.94:0/898141537 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe5cc072630 0x7fe5cc074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7fe5e2a00640 1 -- 172.21.15.94:0/898141537 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe5ec1a8b00 con 0x7fe5e40e6d00 2024-09-10T23:32:53.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f587aa00640 1 -- 172.21.15.94:0/2398497356 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874016020 con 0x7f587c009aa0 2024-09-10T23:32:53.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.629+0000 7f5888a00640 1 -- 172.21.15.94:0/2398497356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587c021670 con 0x7f587c009aa0 2024-09-10T23:32:53.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.633+0000 7f5888a00640 1 -- 172.21.15.94:0/2398497356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587c0d97b0 con 0x7f587c009aa0 2024-09-10T23:32:53.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.633+0000 7feb79600640 1 -- 172.21.15.94:0/4115435700 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7feb70005420 con 0x7feb7c15baa0 2024-09-10T23:32:53.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.633+0000 7fe5ea800640 1 --2- 172.21.15.94:0/898141537 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe5cc072630 0x7fe5cc074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.633+0000 7f587aa00640 1 -- 172.21.15.94:0/2398497356 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5874005020 con 0x7f587c009aa0 2024-09-10T23:32:53.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.633+0000 7fe5ea800640 1 --2- 172.21.15.94:0/898141537 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe5cc072630 0x7fe5cc074af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0046c0 tx=0x7fe5dc004630 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.633+0000 7feb79600640 1 --2- 172.21.15.94:0/4115435700 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7feb5c072610 0x7feb5c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.633+0000 7feb79600640 1 -- 172.21.15.94:0/4115435700 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7feb70093f90 con 0x7feb7c15baa0 2024-09-10T23:32:53.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.633+0000 7f587aa00640 1 -- 172.21.15.94:0/2398497356 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58740053d0 con 0x7f587c009aa0 2024-09-10T23:32:53.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.633+0000 7feb81800640 1 --2- 172.21.15.94:0/4115435700 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7feb5c072610 0x7feb5c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.633+0000 7feb79600640 1 -- 172.21.15.94:0/4115435700 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feb7005f650 con 0x7feb7c15baa0 2024-09-10T23:32:53.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.633+0000 7f5888a00640 1 -- 172.21.15.94:0/2398497356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f585c0052a0 con 0x7f587c009aa0 2024-09-10T23:32:53.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.637+0000 7fe5e2a00640 1 -- 172.21.15.94:0/898141537 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe5e400b4a0 con 0x7fe5e40e6d00 2024-09-10T23:32:53.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.637+0000 7f587aa00640 1 -- 172.21.15.94:0/2398497356 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5874012830 con 0x7f587c009aa0 2024-09-10T23:32:53.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.637+0000 7f587aa00640 1 --2- 172.21.15.94:0/2398497356 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5864072780 0x7f5864074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.637+0000 7f587aa00640 1 -- 172.21.15.94:0/2398497356 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f58740051d0 con 0x7f587c009aa0 2024-09-10T23:32:53.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.641+0000 7f5881e00640 1 --2- 172.21.15.94:0/2398497356 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5864072780 0x7f5864074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.645+0000 7feb81800640 1 --2- 172.21.15.94:0/4115435700 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7feb5c072610 0x7feb5c074ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feb680af000 tx=0x7feb6801b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.641+0000 7f5881e00640 1 --2- 172.21.15.94:0/2398497356 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5864072780 0x7f5864074c40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f58700af000 tx=0x7f587001afa0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.645+0000 7f587aa00640 1 -- 172.21.15.94:0/2398497356 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f587405e190 con 0x7f587c009aa0 2024-09-10T23:32:53.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.665+0000 7fb9bcc00640 1 -- 172.21.15.94:0/3606557269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b00074f0 msgr2=0x7fb9b00e7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.665+0000 7fb9bcc00640 1 --2- 172.21.15.94:0/3606557269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b00074f0 0x7fb9b00e7500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009900 tx=0x7fb9a401c8f0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.665+0000 7fb9bcc00640 1 -- 172.21.15.94:0/3606557269 shutdown_connections 2024-09-10T23:32:53.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.665+0000 7fb9bcc00640 1 --2- 172.21.15.94:0/3606557269 >> v2:172.21.15.181:3300/0 conn(0x7fb9b00537e0 0x7fb9b0005dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.665+0000 7fb9bcc00640 1 --2- 172.21.15.94:0/3606557269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b00074f0 0x7fb9b00e7500 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.665+0000 7fb9bcc00640 1 -- 172.21.15.94:0/3606557269 >> 172.21.15.94:0/3606557269 conn(0x7fb9b0055360 msgr2=0x7fb9b0055770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.669+0000 7fb9bcc00640 1 -- 172.21.15.94:0/3606557269 shutdown_connections 2024-09-10T23:32:53.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.669+0000 7fb9bcc00640 1 -- 172.21.15.94:0/3606557269 wait complete. 2024-09-10T23:32:53.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.669+0000 7fb9bcc00640 1 Processor -- start 2024-09-10T23:32:53.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.669+0000 7fb9bcc00640 1 -- start start 2024-09-10T23:32:53.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.673+0000 7fb9bcc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b00074f0 0x7fb9b004ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.673+0000 7fb9bcc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b00537e0 0x7fb9b00d9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.673+0000 7fb9bcc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b00e8230 0x7fb9b00dd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.673+0000 7fb9bcc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b00f4040 con 0x7fb9b00537e0 2024-09-10T23:32:53.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.673+0000 7fb9bcc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b00f3ec0 con 0x7fb9b00074f0 2024-09-10T23:32:53.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.673+0000 7fb9bcc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b00f41c0 con 0x7fb9b00e8230 2024-09-10T23:32:53.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.673+0000 7fb9b6800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b00e8230 0x7fb9b00dd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.673+0000 7fb9b5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b00074f0 0x7fb9b004ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.673+0000 7fb9b6800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b00e8230 0x7fb9b00dd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:50784/0 (socket says 172.21.15.94:50784) 2024-09-10T23:32:53.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.673+0000 7fb9b6800640 1 -- 172.21.15.94:0/4226992709 learned_addr learned my addr 172.21.15.94:0/4226992709 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:53.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.673+0000 7fb9b6800640 1 -- 172.21.15.94:0/4226992709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b00074f0 msgr2=0x7fb9b004ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.673+0000 7fb9b6800640 1 --2- 172.21.15.94:0/4226992709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b00074f0 0x7fb9b004ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.677+0000 7fb9b6800640 1 -- 172.21.15.94:0/4226992709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b00537e0 msgr2=0x7fb9b00d9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.677+0000 7fb9b6800640 1 --2- 172.21.15.94:0/4226992709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b00537e0 0x7fb9b00d9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.677+0000 7fb9b6800640 1 -- 172.21.15.94:0/4226992709 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b00dc6e0 con 0x7fb9b00e8230 2024-09-10T23:32:53.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.677+0000 7fb9b6800640 1 --2- 172.21.15.94:0/4226992709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b00e8230 0x7fb9b00dd780 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb9b806c150 tx=0x7fb9b8069e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.677+0000 7fb9ae000640 1 -- 172.21.15.94:0/4226992709 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8064500 con 0x7fb9b00e8230 2024-09-10T23:32:53.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.677+0000 7fb9bcc00640 1 -- 172.21.15.94:0/4226992709 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b00dc9c0 con 0x7fb9b00e8230 2024-09-10T23:32:53.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.677+0000 7fb9bcc00640 1 -- 172.21.15.94:0/4226992709 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b00dcf80 con 0x7fb9b00e8230 2024-09-10T23:32:53.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.677+0000 7fb9ae000640 1 -- 172.21.15.94:0/4226992709 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9b8064500 con 0x7fb9b00e8230 2024-09-10T23:32:53.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.677+0000 7fb9ae000640 1 -- 172.21.15.94:0/4226992709 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8063640 con 0x7fb9b00e8230 2024-09-10T23:32:53.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.677+0000 7fb9bcc00640 1 -- 172.21.15.94:0/4226992709 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9900052a0 con 0x7fb9b00e8230 2024-09-10T23:32:53.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.681+0000 7f50d3c00640 1 -- 172.21.15.94:0/3325738307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50cc13ec20 msgr2=0x7f50cc13f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.681+0000 7f50d3c00640 1 --2- 172.21.15.94:0/3325738307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50cc13ec20 0x7f50cc13f400 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f50cc071e20 tx=0x7f50c001c8b0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.681+0000 7f50d3c00640 1 -- 172.21.15.94:0/3325738307 shutdown_connections 2024-09-10T23:32:53.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.681+0000 7f50d3c00640 1 --2- 172.21.15.94:0/3325738307 >> v2:172.21.15.94:3300/0 conn(0x7f50cc15c510 0x7f50cc15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.681+0000 7f50d3c00640 1 --2- 172.21.15.94:0/3325738307 >> v2:172.21.15.181:3300/0 conn(0x7f50cc072660 0x7f50cc072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.681+0000 7fb9ae000640 1 -- 172.21.15.94:0/4226992709 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb9b8092070 con 0x7fb9b00e8230 2024-09-10T23:32:53.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.681+0000 7f50d3c00640 1 --2- 172.21.15.94:0/3325738307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50cc13ec20 0x7f50cc13f400 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.681+0000 7f50d3c00640 1 -- 172.21.15.94:0/3325738307 >> 172.21.15.94:0/3325738307 conn(0x7f50cc0cfaf0 msgr2=0x7f50cc0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.681+0000 7f50d3c00640 1 -- 172.21.15.94:0/3325738307 shutdown_connections 2024-09-10T23:32:53.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.681+0000 7fb9ae000640 1 --2- 172.21.15.94:0/4226992709 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb998072810 0x7fb998074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.681+0000 7fb9ae000640 1 -- 172.21.15.94:0/4226992709 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb9b810ecd0 con 0x7fb9b00e8230 2024-09-10T23:32:53.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.681+0000 7fb9b5e00640 1 --2- 172.21.15.94:0/4226992709 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb998072810 0x7fb998074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.681+0000 7f50d3c00640 1 -- 172.21.15.94:0/3325738307 wait complete. 2024-09-10T23:32:53.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7fb9b5e00640 1 --2- 172.21.15.94:0/4226992709 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb998072810 0x7fb998074cd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8007fd0 tx=0x7fb9a8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d3c00640 1 Processor -- start 2024-09-10T23:32:53.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d3c00640 1 -- start start 2024-09-10T23:32:53.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d3c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50cc072660 0x7f50cc0b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d3c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc13ec20 0x7f50cc137360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d3c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc15c510 0x7f50cc13f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d3c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc0cf5b0 con 0x7f50cc13ec20 2024-09-10T23:32:53.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d3c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc0cf430 con 0x7f50cc15c510 2024-09-10T23:32:53.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d3c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f50cc141430 con 0x7f50cc072660 2024-09-10T23:32:53.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d1c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc15c510 0x7f50cc13f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d1c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc15c510 0x7f50cc13f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:54690/0 (socket says 172.21.15.94:54690) 2024-09-10T23:32:53.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d1c00640 1 -- 172.21.15.94:0/3619456539 learned_addr learned my addr 172.21.15.94:0/3619456539 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:53.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d1200640 1 --2- 172.21.15.94:0/3619456539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50cc072660 0x7f50cc0b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d1c00640 1 -- 172.21.15.94:0/3619456539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50cc072660 msgr2=0x7f50cc0b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d1c00640 1 --2- 172.21.15.94:0/3619456539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50cc072660 0x7f50cc0b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d1c00640 1 -- 172.21.15.94:0/3619456539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc13ec20 msgr2=0x7f50cc137360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T23:32:53.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7fb9ae000640 1 -- 172.21.15.94:0/4226992709 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb9b8114020 con 0x7fb9b00e8230 2024-09-10T23:32:53.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50cbe00640 1 --2- 172.21.15.94:0/3619456539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc13ec20 0x7f50cc137360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d1c00640 1 --2- 172.21.15.94:0/3619456539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc13ec20 0x7f50cc137360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d1c00640 1 -- 172.21.15.94:0/3619456539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50cc0b2cd0 con 0x7f50cc15c510 2024-09-10T23:32:53.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50cbe00640 1 --2- 172.21.15.94:0/3619456539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc13ec20 0x7f50cc137360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:32:53.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.685+0000 7f50d1c00640 1 --2- 172.21.15.94:0/3619456539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc15c510 0x7f50cc13f9f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f50c400daa0 tx=0x7f50c400df60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.689+0000 7f50c8c00640 1 -- 172.21.15.94:0/3619456539 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4017070 con 0x7f50cc15c510 2024-09-10T23:32:53.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.689+0000 7f50c8c00640 1 -- 172.21.15.94:0/3619456539 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50c4004b30 con 0x7f50cc15c510 2024-09-10T23:32:53.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.689+0000 7f50c8c00640 1 -- 172.21.15.94:0/3619456539 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4005020 con 0x7f50cc15c510 2024-09-10T23:32:53.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.689+0000 7f50d3c00640 1 -- 172.21.15.94:0/3619456539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50cc1378a0 con 0x7f50cc15c510 2024-09-10T23:32:53.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.689+0000 7f50d3c00640 1 -- 172.21.15.94:0/3619456539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50cc155bc0 con 0x7f50cc15c510 2024-09-10T23:32:53.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.693+0000 7f50c8c00640 1 -- 172.21.15.94:0/3619456539 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f50c40051c0 con 0x7f50cc15c510 2024-09-10T23:32:53.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.693+0000 7f50d3c00640 1 -- 172.21.15.94:0/3619456539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50cc13f400 con 0x7f50cc15c510 2024-09-10T23:32:53.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.693+0000 7f50c8c00640 1 --2- 172.21.15.94:0/3619456539 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f50ac072660 0x7f50ac074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.693+0000 7f50c8c00640 1 -- 172.21.15.94:0/3619456539 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f50c4093850 con 0x7f50cc15c510 2024-09-10T23:32:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.697+0000 7f50d1200640 1 --2- 172.21.15.94:0/3619456539 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f50ac072660 0x7f50ac074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.697+0000 7f50c8c00640 1 -- 172.21.15.94:0/3619456539 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f50c4020070 con 0x7f50cc15c510 2024-09-10T23:32:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.697+0000 7f50d1200640 1 --2- 172.21.15.94:0/3619456539 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f50ac072660 0x7f50ac074b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50c00af000 tx=0x7f50c001b0f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.729+0000 7f5c73400640 1 -- 172.21.15.94:0/60911634 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60009730 con 0x7f5c58004810 2024-09-10T23:32:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.729+0000 7f5c79e00640 1 -- 172.21.15.94:0/60911634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c58004810 msgr2=0x7f5c58004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.729+0000 7f5c79e00640 1 --2- 172.21.15.94:0/60911634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c58004810 0x7f5c58004c90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009b30 tx=0x7f5c6001d270 comp rx=0 tx=0).stop 2024-09-10T23:32:53.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.733+0000 7f308a000640 1 -- 172.21.15.94:0/1725797252 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30740016e0 con 0x7f308c13d530 2024-09-10T23:32:53.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.733+0000 7f5c79e00640 1 -- 172.21.15.94:0/60911634 shutdown_connections 2024-09-10T23:32:53.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.733+0000 7f5c79e00640 1 --2- 172.21.15.94:0/60911634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c58004810 0x7f5c58004c90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.733+0000 7f5c79e00640 1 -- 172.21.15.94:0/60911634 >> 172.21.15.94:0/60911634 conn(0x7f5c740cf970 msgr2=0x7f5c740cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.733+0000 7f5c79e00640 1 -- 172.21.15.94:0/60911634 shutdown_connections 2024-09-10T23:32:53.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.733+0000 7f5c79e00640 1 -- 172.21.15.94:0/60911634 wait complete. 2024-09-10T23:32:53.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.733+0000 7f3091e00640 1 -- 172.21.15.94:0/1725797252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f308c13d530 msgr2=0x7f308c0ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.733+0000 7f3091e00640 1 --2- 172.21.15.94:0/1725797252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f308c13d530 0x7f308c0ce720 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f307c00a950 tx=0x7f307c01da20 comp rx=0 tx=0).stop 2024-09-10T23:32:53.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.733+0000 7f3091e00640 1 -- 172.21.15.94:0/1725797252 shutdown_connections 2024-09-10T23:32:53.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.733+0000 7f3091e00640 1 --2- 172.21.15.94:0/1725797252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f308c13d530 0x7f308c0ce720 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.733+0000 7f3091e00640 1 -- 172.21.15.94:0/1725797252 >> 172.21.15.94:0/1725797252 conn(0x7f308c0cfaf0 msgr2=0x7f308c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.733+0000 7f3091e00640 1 -- 172.21.15.94:0/1725797252 shutdown_connections 2024-09-10T23:32:53.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.733+0000 7f3091e00640 1 -- 172.21.15.94:0/1725797252 wait complete. 2024-09-10T23:32:53.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f5c79e00640 1 Processor -- start 2024-09-10T23:32:53.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f5c79e00640 1 -- start start 2024-09-10T23:32:53.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f5c79e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c58004810 0x7f5c7415ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f5c79e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c74150920 0x7f5c74150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f5c79e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c74157290 0x7f5c74157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f5c79e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5c74073f90 con 0x7f5c74150920 2024-09-10T23:32:53.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f5c79e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5c74073e10 con 0x7f5c58004810 2024-09-10T23:32:53.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f5c79e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5c74072ce0 con 0x7f5c74157290 2024-09-10T23:32:53.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f5c73e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c74150920 0x7f5c74150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f5c73e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c74150920 0x7f5c74150d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51188/0 (socket says 172.21.15.94:51188) 2024-09-10T23:32:53.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f5c73e00640 1 -- 172.21.15.94:0/2315826233 learned_addr learned my addr 172.21.15.94:0/2315826233 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:53.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f3091e00640 1 Processor -- start 2024-09-10T23:32:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f3091e00640 1 -- start start 2024-09-10T23:32:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f3091e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f308c070670 0x7f308c150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f308aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f308c070670 0x7f308c150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f308aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f308c070670 0x7f308c150670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51194/0 (socket says 172.21.15.94:51194) 2024-09-10T23:32:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f308aa00640 1 -- 172.21.15.94:0/2425695115 learned_addr learned my addr 172.21.15.94:0/2425695115 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f3091e00640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f308c0cec60 0x7f308c13ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f3091e00640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f308c13d530 0x7f308c13efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f3091e00640 1 -- 172.21.15.94:0/2425695115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f308c168680 con 0x7f308c070670 2024-09-10T23:32:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f3091e00640 1 -- 172.21.15.94:0/2425695115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f308c168500 con 0x7f308c0cec60 2024-09-10T23:32:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f3091e00640 1 -- 172.21.15.94:0/2425695115 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f308c168800 con 0x7f308c13d530 2024-09-10T23:32:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f3082000640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f308c0cec60 0x7f308c13ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f308b400640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f308c13d530 0x7f308c13efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f3082000640 1 -- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f308c13d530 msgr2=0x7f308c13efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f3082000640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f308c13d530 0x7f308c13efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f3082000640 1 -- 172.21.15.94:0/2425695115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f308c070670 msgr2=0x7f308c150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f3082000640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f308c070670 0x7f308c150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f3082000640 1 -- 172.21.15.94:0/2425695115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c152ae0 con 0x7f308c0cec60 2024-09-10T23:32:53.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f5c73e00640 1 -- 172.21.15.94:0/2315826233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c74157290 msgr2=0x7f5c74157740 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T23:32:53.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f5c73e00640 1 --2- 172.21.15.94:0/2315826233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c74157290 0x7f5c74157740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f5c73e00640 1 -- 172.21.15.94:0/2315826233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c58004810 msgr2=0x7f5c7415ce10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T23:32:53.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f5c73e00640 1 --2- 172.21.15.94:0/2315826233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c58004810 0x7f5c7415ce10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.737+0000 7f5c73e00640 1 -- 172.21.15.94:0/2315826233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7414bb70 con 0x7f5c74150920 2024-09-10T23:32:53.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f5c73e00640 1 --2- 172.21.15.94:0/2315826233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c74150920 0x7f5c74150d60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c6001d0a0 tx=0x7f5c60005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f308aa00640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f308c070670 0x7f308c150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:53.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f5c70c00640 1 -- 172.21.15.94:0/2315826233 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c600b1070 con 0x7f5c74150920 2024-09-10T23:32:53.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f5c79e00640 1 -- 172.21.15.94:0/2315826233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7414be50 con 0x7f5c74150920 2024-09-10T23:32:53.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f5c79e00640 1 -- 172.21.15.94:0/2315826233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c741518b0 con 0x7f5c74150920 2024-09-10T23:32:53.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f5c70c00640 1 -- 172.21.15.94:0/2315826233 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c600bc070 con 0x7f5c74150920 2024-09-10T23:32:53.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f5c70c00640 1 -- 172.21.15.94:0/2315826233 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c600b1070 con 0x7f5c74150920 2024-09-10T23:32:53.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f3082000640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f308c0cec60 0x7f308c13ea60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f307c0ad000 tx=0x7f307c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f3083e00640 1 -- 172.21.15.94:0/2425695115 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c0bc070 con 0x7f308c0cec60 2024-09-10T23:32:53.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f3083e00640 1 -- 172.21.15.94:0/2425695115 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f307c0044e0 con 0x7f308c0cec60 2024-09-10T23:32:53.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.741+0000 7f3083e00640 1 -- 172.21.15.94:0/2425695115 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c0b3070 con 0x7f308c0cec60 2024-09-10T23:32:53.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.745+0000 7f3091e00640 1 -- 172.21.15.94:0/2425695115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308c152d80 con 0x7f308c0cec60 2024-09-10T23:32:53.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.745+0000 7f3091e00640 1 -- 172.21.15.94:0/2425695115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f308c150ce0 con 0x7f308c0cec60 2024-09-10T23:32:53.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.745+0000 7f3083e00640 1 -- 172.21.15.94:0/2425695115 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f308c15e280 con 0x7f308c0cec60 2024-09-10T23:32:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.745+0000 7f5c70c00640 1 -- 172.21.15.94:0/2315826233 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5c600a5ce0 con 0x7f5c74150920 2024-09-10T23:32:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.749+0000 7f5c6a000640 1 -- 172.21.15.94:0/2315826233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c74073160 con 0x7f5c74150920 2024-09-10T23:32:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.749+0000 7f5c70c00640 1 --2- 172.21.15.94:0/2315826233 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c54072810 0x7f5c54074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.749+0000 7f3083e00640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f30680725e0 0x7f3068074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.749+0000 7f5c78a00640 1 --2- 172.21.15.94:0/2315826233 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c54072810 0x7f5c54074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.749+0000 7f3083e00640 1 -- 172.21.15.94:0/2425695115 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f307c12cdc0 con 0x7f308c0cec60 2024-09-10T23:32:53.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.749+0000 7f308aa00640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f30680725e0 0x7f3068074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.749+0000 7f5c78a00640 1 --2- 172.21.15.94:0/2315826233 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c54072810 0x7f5c54074cd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5c64000c10 tx=0x7f5c64006b40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.749+0000 7f5c70c00640 1 -- 172.21.15.94:0/2315826233 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5c6012bfa0 con 0x7f5c74150920 2024-09-10T23:32:53.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.749+0000 7f3091e00640 1 -- 172.21.15.94:0/2425695115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f308c074a60 con 0x7f308c0cec60 2024-09-10T23:32:53.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.753+0000 7f308aa00640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f30680725e0 0x7f3068074aa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3074001850 tx=0x7f30740016c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.757+0000 7f3083e00640 1 -- 172.21.15.94:0/2425695115 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f307c0176a0 con 0x7f308c0cec60 2024-09-10T23:32:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.757+0000 7f5c70c00640 1 -- 172.21.15.94:0/2315826233 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5c60016f80 con 0x7f5c74150920 2024-09-10T23:32:53.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.781+0000 7efdab000640 1 -- 172.21.15.94:0/2050473753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd88004170 msgr2=0x7efd8800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.781+0000 7efdab000640 1 --2- 172.21.15.94:0/2050473753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd88004170 0x7efd8800b9d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00c470 tx=0x7efd9c01d180 comp rx=0 tx=0).stop 2024-09-10T23:32:53.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.781+0000 7efdab000640 1 -- 172.21.15.94:0/2050473753 shutdown_connections 2024-09-10T23:32:53.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.781+0000 7efdab000640 1 --2- 172.21.15.94:0/2050473753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd88004170 0x7efd8800b9d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.781+0000 7efdab000640 1 -- 172.21.15.94:0/2050473753 >> 172.21.15.94:0/2050473753 conn(0x7efda40cfaf0 msgr2=0x7efda40cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.781+0000 7efdab000640 1 -- 172.21.15.94:0/2050473753 shutdown_connections 2024-09-10T23:32:53.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.781+0000 7efdab000640 1 -- 172.21.15.94:0/2050473753 wait complete. 2024-09-10T23:32:53.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.785+0000 7efdab000640 1 Processor -- start 2024-09-10T23:32:53.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.785+0000 7efdab000640 1 -- start start 2024-09-10T23:32:53.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.785+0000 7efdab000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd88004170 0x7efda40b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.785+0000 7efdab000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efda414af90 0x7efda4208910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.789+0000 7efdab000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efda414e510 0x7efda4073600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.789+0000 7efda3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd88004170 0x7efda40b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.789+0000 7efda3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd88004170 0x7efda40b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:35948/0 (socket says 172.21.15.94:35948) 2024-09-10T23:32:53.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.789+0000 7efd9b400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efda414af90 0x7efda4208910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.789+0000 7efd9b400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efda414af90 0x7efda4208910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:50302/0 (socket says 172.21.15.94:50302) 2024-09-10T23:32:53.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.789+0000 7efda9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efda414e510 0x7efda4073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.789+0000 7efda9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efda414e510 0x7efda4073600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40180/0 (socket says 172.21.15.94:40180) 2024-09-10T23:32:53.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.789+0000 7efda3e00640 1 -- 172.21.15.94:0/3170470643 learned_addr learned my addr 172.21.15.94:0/3170470643 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:53.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.789+0000 7efdab000640 1 -- 172.21.15.94:0/3170470643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efda40cf1a0 con 0x7efda414e510 2024-09-10T23:32:53.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.789+0000 7efdab000640 1 -- 172.21.15.94:0/3170470643 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efda40cf020 con 0x7efd88004170 2024-09-10T23:32:53.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.789+0000 7efdab000640 1 -- 172.21.15.94:0/3170470643 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7efda40c53c0 con 0x7efda414af90 2024-09-10T23:32:53.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.793+0000 7efda9000640 1 -- 172.21.15.94:0/3170470643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efda414af90 msgr2=0x7efda4208910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.793+0000 7efda9000640 1 --2- 172.21.15.94:0/3170470643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efda414af90 0x7efda4208910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.793+0000 7efda9000640 1 -- 172.21.15.94:0/3170470643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd88004170 msgr2=0x7efda40b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.793+0000 7efda9000640 1 --2- 172.21.15.94:0/3170470643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd88004170 0x7efda40b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.793+0000 7efda9000640 1 -- 172.21.15.94:0/3170470643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda41571e0 con 0x7efda414e510 2024-09-10T23:32:53.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.793+0000 7efda9000640 1 --2- 172.21.15.94:0/3170470643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efda414e510 0x7efda4073600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efda4075030 tx=0x7efd9c006e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.793+0000 7efda3e00640 1 --2- 172.21.15.94:0/3170470643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd88004170 0x7efda40b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:32:53.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.793+0000 7efd9b400640 1 --2- 172.21.15.94:0/3170470643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efda414af90 0x7efda4208910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:32:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.793+0000 7efda0c00640 1 -- 172.21.15.94:0/3170470643 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c001a80 con 0x7efda414e510 2024-09-10T23:32:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.793+0000 7efda0c00640 1 -- 172.21.15.94:0/3170470643 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd9c003980 con 0x7efda414e510 2024-09-10T23:32:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.793+0000 7efda0c00640 1 -- 172.21.15.94:0/3170470643 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c003db0 con 0x7efda414e510 2024-09-10T23:32:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.793+0000 7efdab000640 1 -- 172.21.15.94:0/3170470643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda41554e0 con 0x7efda414e510 2024-09-10T23:32:53.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.793+0000 7efdab000640 1 -- 172.21.15.94:0/3170470643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda414e9a0 con 0x7efda414e510 2024-09-10T23:32:53.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.797+0000 7efda0c00640 1 -- 172.21.15.94:0/3170470643 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efd9c006070 con 0x7efda414e510 2024-09-10T23:32:53.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.801+0000 7efd99600640 1 -- 172.21.15.94:0/3170470643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd740052a0 con 0x7efda414e510 2024-09-10T23:32:53.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.801+0000 7efda0c00640 1 --2- 172.21.15.94:0/3170470643 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efd80072630 0x7efd80074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:53.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.801+0000 7efda0c00640 1 -- 172.21.15.94:0/3170470643 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7efd9c12d3b0 con 0x7efda414e510 2024-09-10T23:32:53.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.801+0000 7efda3e00640 1 --2- 172.21.15.94:0/3170470643 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efd80072630 0x7efd80074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:53.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.801+0000 7efda3e00640 1 --2- 172.21.15.94:0/3170470643 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efd80072630 0x7efd80074af0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd94001ec0 tx=0x7efd94009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:53.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.809+0000 7efda0c00640 1 -- 172.21.15.94:0/3170470643 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efd9c016f30 con 0x7efda414e510 2024-09-10T23:32:53.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.977+0000 7f5888a00640 1 -- 172.21.15.94:0/2398497356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f585c005590 con 0x7f587c009aa0 2024-09-10T23:32:53.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.985+0000 7f587aa00640 1 -- 172.21.15.94:0/2398497356 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f587405da90 con 0x7f587c009aa0 2024-09-10T23:32:53.988 INFO:teuthology.orchestra.run.smithi094.stdout:64424509446 2024-09-10T23:32:53.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.989+0000 7f5888a00640 1 -- 172.21.15.94:0/2398497356 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5864072780 msgr2=0x7f5864074c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.989+0000 7f5888a00640 1 --2- 172.21.15.94:0/2398497356 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5864072780 0x7f5864074c40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f58700af000 tx=0x7f587001afa0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.989+0000 7f5888a00640 1 -- 172.21.15.94:0/2398497356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c009aa0 msgr2=0x7f587c04a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:53.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.989+0000 7f5888a00640 1 --2- 172.21.15.94:0/2398497356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c009aa0 0x7f587c04a570 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f587400c4d0 tx=0x7f587400c990 comp rx=0 tx=0).stop 2024-09-10T23:32:53.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.989+0000 7f5888a00640 1 -- 172.21.15.94:0/2398497356 shutdown_connections 2024-09-10T23:32:53.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.989+0000 7f5888a00640 1 --2- 172.21.15.94:0/2398497356 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5864072780 0x7f5864074c40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.989+0000 7f5888a00640 1 --2- 172.21.15.94:0/2398497356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f587c00a480 0x7f587c01e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.989+0000 7f5888a00640 1 --2- 172.21.15.94:0/2398497356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f587c01c370 0x7f587c01e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.989+0000 7f5888a00640 1 --2- 172.21.15.94:0/2398497356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f587c009aa0 0x7f587c04a570 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:53.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.989+0000 7f5888a00640 1 -- 172.21.15.94:0/2398497356 >> 172.21.15.94:0/2398497356 conn(0x7f587c054570 msgr2=0x7f587c0df210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:53.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.989+0000 7f5888a00640 1 -- 172.21.15.94:0/2398497356 shutdown_connections 2024-09-10T23:32:53.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.989+0000 7f5888a00640 1 -- 172.21.15.94:0/2398497356 wait complete. 2024-09-10T23:32:54.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.997+0000 7fe5f0a00640 1 -- 172.21.15.94:0/898141537 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe5e403bcf0 con 0x7fe5e40e6d00 2024-09-10T23:32:54.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:53.997+0000 7fe5e2a00640 1 -- 172.21.15.94:0/898141537 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe5ec092140 con 0x7fe5e40e6d00 2024-09-10T23:32:54.002 INFO:teuthology.orchestra.run.smithi094.stdout:64424509446 2024-09-10T23:32:54.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.001+0000 7fe5f0a00640 1 -- 172.21.15.94:0/898141537 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe5cc072630 msgr2=0x7fe5cc074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:54.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.001+0000 7fe5f0a00640 1 --2- 172.21.15.94:0/898141537 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe5cc072630 0x7fe5cc074af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0046c0 tx=0x7fe5dc004630 comp rx=0 tx=0).stop 2024-09-10T23:32:54.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.001+0000 7fe5f0a00640 1 -- 172.21.15.94:0/898141537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40e6d00 msgr2=0x7fe5e400b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:54.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.001+0000 7fe5f0a00640 1 --2- 172.21.15.94:0/898141537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40e6d00 0x7fe5e400b7c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec06b9b0 tx=0x7fe5ec063640 comp rx=0 tx=0).stop 2024-09-10T23:32:54.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.001+0000 7fe5f0a00640 1 -- 172.21.15.94:0/898141537 shutdown_connections 2024-09-10T23:32:54.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.001+0000 7fe5f0a00640 1 --2- 172.21.15.94:0/898141537 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe5cc072630 0x7fe5cc074af0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.005+0000 7fe5f0a00640 1 --2- 172.21.15.94:0/898141537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40e6d00 0x7fe5e400b7c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.005+0000 7fe5f0a00640 1 --2- 172.21.15.94:0/898141537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e40e6920 0x7fe5e40ded40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.005+0000 7fe5f0a00640 1 --2- 172.21.15.94:0/898141537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40578a0 0x7fe5e40d7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.005+0000 7fe5f0a00640 1 -- 172.21.15.94:0/898141537 >> 172.21.15.94:0/898141537 conn(0x7fe5e4055360 msgr2=0x7fe5e40f0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:54.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.005+0000 7fe5f0a00640 1 -- 172.21.15.94:0/898141537 shutdown_connections 2024-09-10T23:32:54.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.005+0000 7fe5f0a00640 1 -- 172.21.15.94:0/898141537 wait complete. 2024-09-10T23:32:54.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.005+0000 7feb82c00640 1 -- 172.21.15.94:0/4115435700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7feb58005860 con 0x7feb7c15baa0 2024-09-10T23:32:54.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.009+0000 7feb79600640 1 -- 172.21.15.94:0/4115435700 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feb7005ef50 con 0x7feb7c15baa0 2024-09-10T23:32:54.012 INFO:teuthology.orchestra.run.smithi094.stdout:64424509446 2024-09-10T23:32:54.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.013+0000 7feb82c00640 1 -- 172.21.15.94:0/4115435700 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7feb5c072610 msgr2=0x7feb5c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:54.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.013+0000 7feb82c00640 1 --2- 172.21.15.94:0/4115435700 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7feb5c072610 0x7feb5c074ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feb680af000 tx=0x7feb6801b120 comp rx=0 tx=0).stop 2024-09-10T23:32:54.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.013+0000 7feb82c00640 1 -- 172.21.15.94:0/4115435700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb7c15baa0 msgr2=0x7feb7c14f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:54.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.013+0000 7feb82c00640 1 --2- 172.21.15.94:0/4115435700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb7c15baa0 0x7feb7c14f190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feb7000e8c0 tx=0x7feb7000ed80 comp rx=0 tx=0).stop 2024-09-10T23:32:54.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.013+0000 7feb82c00640 1 -- 172.21.15.94:0/4115435700 shutdown_connections 2024-09-10T23:32:54.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.013+0000 7feb82c00640 1 --2- 172.21.15.94:0/4115435700 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7feb5c072610 0x7feb5c074ad0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.013+0000 7feb82c00640 1 --2- 172.21.15.94:0/4115435700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb7c15baa0 0x7feb7c14f190 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.013+0000 7feb82c00640 1 --2- 172.21.15.94:0/4115435700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb7c0d0920 0x7feb7c153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.013+0000 7feb82c00640 1 --2- 172.21.15.94:0/4115435700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb7c0ce890 0x7feb7c154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.013+0000 7feb82c00640 1 -- 172.21.15.94:0/4115435700 >> 172.21.15.94:0/4115435700 conn(0x7feb7c06dd30 msgr2=0x7feb7c15af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:54.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.017+0000 7feb82c00640 1 -- 172.21.15.94:0/4115435700 shutdown_connections 2024-09-10T23:32:54.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.017+0000 7feb82c00640 1 -- 172.21.15.94:0/4115435700 wait complete. 2024-09-10T23:32:54.027 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.0 2024-09-10T23:32:54.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.1 2024-09-10T23:32:54.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.3 2024-09-10T23:32:54.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.049+0000 7fb9bcc00640 1 -- 172.21.15.94:0/4226992709 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb990005590 con 0x7fb9b00e8230 2024-09-10T23:32:54.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.049+0000 7fb9ae000640 1 -- 172.21.15.94:0/4226992709 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb9b80da3a0 con 0x7fb9b00e8230 2024-09-10T23:32:54.053 INFO:teuthology.orchestra.run.smithi094.stdout:64424509447 2024-09-10T23:32:54.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.053+0000 7fb9bcc00640 1 -- 172.21.15.94:0/4226992709 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb998072810 msgr2=0x7fb998074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:54.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.053+0000 7fb9bcc00640 1 --2- 172.21.15.94:0/4226992709 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb998072810 0x7fb998074cd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8007fd0 tx=0x7fb9a8009040 comp rx=0 tx=0).stop 2024-09-10T23:32:54.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.053+0000 7fb9bcc00640 1 -- 172.21.15.94:0/4226992709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b00e8230 msgr2=0x7fb9b00dd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:54.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.053+0000 7fb9bcc00640 1 --2- 172.21.15.94:0/4226992709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b00e8230 0x7fb9b00dd780 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb9b806c150 tx=0x7fb9b8069e30 comp rx=0 tx=0).stop 2024-09-10T23:32:54.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.053+0000 7fb9bcc00640 1 -- 172.21.15.94:0/4226992709 shutdown_connections 2024-09-10T23:32:54.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.053+0000 7fb9bcc00640 1 --2- 172.21.15.94:0/4226992709 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb998072810 0x7fb998074cd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.053+0000 7fb9bcc00640 1 --2- 172.21.15.94:0/4226992709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b00e8230 0x7fb9b00dd780 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.053+0000 7fb9bcc00640 1 --2- 172.21.15.94:0/4226992709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b00537e0 0x7fb9b00d9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.053+0000 7fb9bcc00640 1 --2- 172.21.15.94:0/4226992709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b00074f0 0x7fb9b004ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.053+0000 7fb9bcc00640 1 -- 172.21.15.94:0/4226992709 >> 172.21.15.94:0/4226992709 conn(0x7fb9b0055360 msgr2=0x7fb9b00db7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:54.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.053+0000 7fb9bcc00640 1 -- 172.21.15.94:0/4226992709 shutdown_connections 2024-09-10T23:32:54.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.053+0000 7fb9bcc00640 1 -- 172.21.15.94:0/4226992709 wait complete. 2024-09-10T23:32:54.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.065+0000 7f50d3c00640 1 -- 172.21.15.94:0/3619456539 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f50cc0d0750 con 0x7f50cc15c510 2024-09-10T23:32:54.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.065+0000 7f50c8c00640 1 -- 172.21.15.94:0/3619456539 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f50c405ef10 con 0x7f50cc15c510 2024-09-10T23:32:54.067 INFO:teuthology.orchestra.run.smithi094.stdout:64424509447 2024-09-10T23:32:54.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.069+0000 7f50d3c00640 1 -- 172.21.15.94:0/3619456539 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f50ac072660 msgr2=0x7f50ac074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:54.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.069+0000 7f50d3c00640 1 --2- 172.21.15.94:0/3619456539 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f50ac072660 0x7f50ac074b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50c00af000 tx=0x7f50c001b0f0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.069+0000 7f50d3c00640 1 -- 172.21.15.94:0/3619456539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc15c510 msgr2=0x7f50cc13f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:54.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.069+0000 7f50d3c00640 1 --2- 172.21.15.94:0/3619456539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc15c510 0x7f50cc13f9f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f50c400daa0 tx=0x7f50c400df60 comp rx=0 tx=0).stop 2024-09-10T23:32:54.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.069+0000 7f50d3c00640 1 -- 172.21.15.94:0/3619456539 shutdown_connections 2024-09-10T23:32:54.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.069+0000 7f50d3c00640 1 --2- 172.21.15.94:0/3619456539 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f50ac072660 0x7f50ac074b20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.069+0000 7f50d3c00640 1 --2- 172.21.15.94:0/3619456539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50cc15c510 0x7f50cc13f9f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.069+0000 7f50d3c00640 1 --2- 172.21.15.94:0/3619456539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50cc13ec20 0x7f50cc137360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.069+0000 7f50d3c00640 1 --2- 172.21.15.94:0/3619456539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50cc072660 0x7f50cc0b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.069+0000 7f50d3c00640 1 -- 172.21.15.94:0/3619456539 >> 172.21.15.94:0/3619456539 conn(0x7f50cc0cfaf0 msgr2=0x7f50cc152cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:54.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.069+0000 7f50d3c00640 1 -- 172.21.15.94:0/3619456539 shutdown_connections 2024-09-10T23:32:54.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.069+0000 7f50d3c00640 1 -- 172.21.15.94:0/3619456539 wait complete. 2024-09-10T23:32:54.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.073+0000 7f5c6a000640 1 -- 172.21.15.94:0/2315826233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f5c7414ec60 con 0x7f5c74150920 2024-09-10T23:32:54.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.073+0000 7f5c70c00640 1 -- 172.21.15.94:0/2315826233 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5c600f90d0 con 0x7f5c74150920 2024-09-10T23:32:54.076 INFO:teuthology.orchestra.run.smithi094.stdout:64424509448 2024-09-10T23:32:54.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.2 2024-09-10T23:32:54.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.077+0000 7f5c6a000640 1 -- 172.21.15.94:0/2315826233 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c54072810 msgr2=0x7f5c54074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:54.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.077+0000 7f5c6a000640 1 --2- 172.21.15.94:0/2315826233 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c54072810 0x7f5c54074cd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5c64000c10 tx=0x7f5c64006b40 comp rx=0 tx=0).stop 2024-09-10T23:32:54.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.077+0000 7f5c6a000640 1 -- 172.21.15.94:0/2315826233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c74150920 msgr2=0x7f5c74150d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:54.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.077+0000 7f5c6a000640 1 --2- 172.21.15.94:0/2315826233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c74150920 0x7f5c74150d60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c6001d0a0 tx=0x7f5c60005600 comp rx=0 tx=0).stop 2024-09-10T23:32:54.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.077+0000 7f5c6a000640 1 -- 172.21.15.94:0/2315826233 shutdown_connections 2024-09-10T23:32:54.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.077+0000 7f5c6a000640 1 --2- 172.21.15.94:0/2315826233 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c54072810 0x7f5c54074cd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.077+0000 7f5c6a000640 1 --2- 172.21.15.94:0/2315826233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c74157290 0x7f5c74157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.077+0000 7f5c6a000640 1 --2- 172.21.15.94:0/2315826233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c74150920 0x7f5c74150d60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.077+0000 7f5c6a000640 1 --2- 172.21.15.94:0/2315826233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c58004810 0x7f5c7415ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.077+0000 7f5c6a000640 1 -- 172.21.15.94:0/2315826233 >> 172.21.15.94:0/2315826233 conn(0x7f5c740cf970 msgr2=0x7f5c7414dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.077+0000 7f5c6a000640 1 -- 172.21.15.94:0/2315826233 shutdown_connections 2024-09-10T23:32:54.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.077+0000 7f5c6a000640 1 -- 172.21.15.94:0/2315826233 wait complete. 2024-09-10T23:32:54.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.085+0000 7f3091e00640 1 -- 172.21.15.94:0/2425695115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f308c152820 con 0x7f308c0cec60 2024-09-10T23:32:54.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.085+0000 7f3083e00640 1 -- 172.21.15.94:0/2425695115 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f307c0176a0 con 0x7f308c0cec60 2024-09-10T23:32:54.088 INFO:teuthology.orchestra.run.smithi094.stdout:64424509445 2024-09-10T23:32:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.089+0000 7f3091e00640 1 -- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f30680725e0 msgr2=0x7f3068074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.089+0000 7f3091e00640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f30680725e0 0x7f3068074aa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3074001850 tx=0x7f30740016c0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.089+0000 7f3091e00640 1 -- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f308c0cec60 msgr2=0x7f308c13ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.089+0000 7f3091e00640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f308c0cec60 0x7f308c13ea60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f307c0ad000 tx=0x7f307c0042f0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.089+0000 7f3091e00640 1 -- 172.21.15.94:0/2425695115 shutdown_connections 2024-09-10T23:32:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.089+0000 7f3091e00640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f30680725e0 0x7f3068074aa0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.089+0000 7f3091e00640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f308c0cec60 0x7f308c13ea60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.089+0000 7f3091e00640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f308c13d530 0x7f308c13efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.089+0000 7f3091e00640 1 --2- 172.21.15.94:0/2425695115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f308c070670 0x7f308c150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.089+0000 7f3091e00640 1 -- 172.21.15.94:0/2425695115 >> 172.21.15.94:0/2425695115 conn(0x7f308c0cfaf0 msgr2=0x7f308c06f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.089+0000 7f3091e00640 1 -- 172.21.15.94:0/2425695115 shutdown_connections 2024-09-10T23:32:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.089+0000 7f3091e00640 1 -- 172.21.15.94:0/2425695115 wait complete. 2024-09-10T23:32:54.103 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.7 2024-09-10T23:32:54.104 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.5 2024-09-10T23:32:54.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2024-09-10T23:32:54.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.129+0000 7efd99600640 1 -- 172.21.15.94:0/3170470643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7efd74005590 con 0x7efda414e510 2024-09-10T23:32:54.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.129+0000 7efda0c00640 1 -- 172.21.15.94:0/3170470643 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efd9c016f30 con 0x7efda414e510 2024-09-10T23:32:54.134 INFO:teuthology.orchestra.run.smithi094.stdout:60129542151 2024-09-10T23:32:54.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.133+0000 7efd99600640 1 -- 172.21.15.94:0/3170470643 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efd80072630 msgr2=0x7efd80074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:54.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.133+0000 7efd99600640 1 --2- 172.21.15.94:0/3170470643 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efd80072630 0x7efd80074af0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd94001ec0 tx=0x7efd94009040 comp rx=0 tx=0).stop 2024-09-10T23:32:54.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.133+0000 7efd99600640 1 -- 172.21.15.94:0/3170470643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efda414e510 msgr2=0x7efda4073600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:54.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.133+0000 7efd99600640 1 --2- 172.21.15.94:0/3170470643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efda414e510 0x7efda4073600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efda4075030 tx=0x7efd9c006e80 comp rx=0 tx=0).stop 2024-09-10T23:32:54.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.133+0000 7efd99600640 1 -- 172.21.15.94:0/3170470643 shutdown_connections 2024-09-10T23:32:54.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.133+0000 7efd99600640 1 --2- 172.21.15.94:0/3170470643 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efd80072630 0x7efd80074af0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.133+0000 7efd99600640 1 --2- 172.21.15.94:0/3170470643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efda414e510 0x7efda4073600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.133+0000 7efd99600640 1 --2- 172.21.15.94:0/3170470643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efda414af90 0x7efda4208910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.133+0000 7efd99600640 1 --2- 172.21.15.94:0/3170470643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd88004170 0x7efda40b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:54.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.133+0000 7efd99600640 1 -- 172.21.15.94:0/3170470643 >> 172.21.15.94:0/3170470643 conn(0x7efda40cfaf0 msgr2=0x7efda40cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:54.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.133+0000 7efd99600640 1 -- 172.21.15.94:0/3170470643 shutdown_connections 2024-09-10T23:32:54.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:54.133+0000 7efd99600640 1 -- 172.21.15.94:0/3170470643 wait complete. 2024-09-10T23:32:54.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.6 2024-09-10T23:32:55.028 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T23:32:55.039 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T23:32:55.045 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T23:32:55.078 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T23:32:55.104 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T23:32:55.105 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T23:32:55.112 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T23:32:55.157 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T23:32:55.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.229+0000 7fc066200640 1 -- 172.21.15.94:0/1819437725 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc06006ce80 msgr2=0x7fc06015f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.229+0000 7fc066200640 1 --2- 172.21.15.94:0/1819437725 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc06006ce80 0x7fc06015f260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc054009900 tx=0x7fc05401c8d0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.229+0000 7fc066200640 1 -- 172.21.15.94:0/1819437725 shutdown_connections 2024-09-10T23:32:55.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.229+0000 7fc066200640 1 --2- 172.21.15.94:0/1819437725 >> v2:172.21.15.181:3300/0 conn(0x7fc0600cdc90 0x7fc06006f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.229+0000 7fc066200640 1 --2- 172.21.15.94:0/1819437725 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc06006ce80 0x7fc06015f260 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.229+0000 7fc066200640 1 -- 172.21.15.94:0/1819437725 >> 172.21.15.94:0/1819437725 conn(0x7fc0600cf8d0 msgr2=0x7fc0600cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:55.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.233+0000 7fc066200640 1 -- 172.21.15.94:0/1819437725 shutdown_connections 2024-09-10T23:32:55.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.233+0000 7fc066200640 1 -- 172.21.15.94:0/1819437725 wait complete. 2024-09-10T23:32:55.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.233+0000 7fc066200640 1 Processor -- start 2024-09-10T23:32:55.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.233+0000 7fc066200640 1 -- start start 2024-09-10T23:32:55.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc066200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc06006ce80 0x7fc0600c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc066200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc06006d260 0x7fc0600b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc066200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0600cdc90 0x7fc0600ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc066200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc060159b40 con 0x7fc0600cdc90 2024-09-10T23:32:55.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc066200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc0601599c0 con 0x7fc06006ce80 2024-09-10T23:32:55.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc066200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc06016f240 con 0x7fc06006d260 2024-09-10T23:32:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc05fe00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc06006d260 0x7fc0600b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc05fe00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc06006d260 0x7fc0600b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:50330/0 (socket says 172.21.15.94:50330) 2024-09-10T23:32:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc05fe00640 1 -- 172.21.15.94:0/264539947 learned_addr learned my addr 172.21.15.94:0/264539947 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc064e00640 1 --2- 172.21.15.94:0/264539947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc06006ce80 0x7fc0600c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc065800640 1 --2- 172.21.15.94:0/264539947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0600cdc90 0x7fc0600ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc05fe00640 1 -- 172.21.15.94:0/264539947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc06006ce80 msgr2=0x7fc0600c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc05fe00640 1 --2- 172.21.15.94:0/264539947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc06006ce80 0x7fc0600c5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc05fe00640 1 -- 172.21.15.94:0/264539947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0600cdc90 msgr2=0x7fc0600ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc05fe00640 1 --2- 172.21.15.94:0/264539947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0600cdc90 0x7fc0600ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc05fe00640 1 -- 172.21.15.94:0/264539947 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0600b33a0 con 0x7fc06006d260 2024-09-10T23:32:55.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc065800640 1 --2- 172.21.15.94:0/264539947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0600cdc90 0x7fc0600ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:55.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc05fe00640 1 --2- 172.21.15.94:0/264539947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc06006d260 0x7fc0600b5000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00c440 tx=0x7fc04c00c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:55.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc05cc00640 1 -- 172.21.15.94:0/264539947 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c017070 con 0x7fc06006d260 2024-09-10T23:32:55.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc05cc00640 1 -- 172.21.15.94:0/264539947 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc04c003600 con 0x7fc06006d260 2024-09-10T23:32:55.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc05cc00640 1 -- 172.21.15.94:0/264539947 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c003a30 con 0x7fc06006d260 2024-09-10T23:32:55.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc066200640 1 -- 172.21.15.94:0/264539947 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0600b3640 con 0x7fc06006d260 2024-09-10T23:32:55.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.237+0000 7fc066200640 1 -- 172.21.15.94:0/264539947 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0600ca150 con 0x7fc06006d260 2024-09-10T23:32:55.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.241+0000 7fc066200640 1 -- 172.21.15.94:0/264539947 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc06015f260 con 0x7fc06006d260 2024-09-10T23:32:55.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.241+0000 7fc05cc00640 1 -- 172.21.15.94:0/264539947 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc04c023080 con 0x7fc06006d260 2024-09-10T23:32:55.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.245+0000 7fc05cc00640 1 --2- 172.21.15.94:0/264539947 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc0400725e0 0x7fc040074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.245+0000 7fc05cc00640 1 -- 172.21.15.94:0/264539947 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc04c092950 con 0x7fc06006d260 2024-09-10T23:32:55.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.245+0000 7fc064e00640 1 --2- 172.21.15.94:0/264539947 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc0400725e0 0x7fc040074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.245+0000 7fc05cc00640 1 -- 172.21.15.94:0/264539947 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc04c0c0de0 con 0x7fc06006d260 2024-09-10T23:32:55.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.245+0000 7fc064e00640 1 --2- 172.21.15.94:0/264539947 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc0400725e0 0x7fc040074aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc0540af000 tx=0x7fc05401b110 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:55.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.281+0000 7f02ba400640 1 -- 172.21.15.94:0/1925086301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02b40b1100 msgr2=0x7f02b40b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.281+0000 7f02ba400640 1 --2- 172.21.15.94:0/1925086301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02b40b1100 0x7f02b40b15b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f02b4073990 tx=0x7f02ac01cac0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.281+0000 7f02ba400640 1 -- 172.21.15.94:0/1925086301 shutdown_connections 2024-09-10T23:32:55.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.281+0000 7f02ba400640 1 --2- 172.21.15.94:0/1925086301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02b40b1100 0x7f02b40b15b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.281+0000 7f02ba400640 1 --2- 172.21.15.94:0/1925086301 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02b40ce1e0 0x7f02b40cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.281+0000 7f02ba400640 1 --2- 172.21.15.94:0/1925086301 >> v2:172.21.15.181:3300/0 conn(0x7f02b40ce890 0x7f02b40cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.281+0000 7f02ba400640 1 -- 172.21.15.94:0/1925086301 >> 172.21.15.94:0/1925086301 conn(0x7f02b40d1c60 msgr2=0x7f02b40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:55.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.281+0000 7f02ba400640 1 -- 172.21.15.94:0/1925086301 shutdown_connections 2024-09-10T23:32:55.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.281+0000 7f02ba400640 1 -- 172.21.15.94:0/1925086301 wait complete. 2024-09-10T23:32:55.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.285+0000 7f02ba400640 1 Processor -- start 2024-09-10T23:32:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.285+0000 7f02ba400640 1 -- start start 2024-09-10T23:32:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02ba400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02b40b1100 0x7f02b42075b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02ba400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b40ce1e0 0x7f02b40af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02ba400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02b40ce890 0x7f02b40afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02ba400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f02b40cfbc0 con 0x7f02b40b1100 2024-09-10T23:32:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02b2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b40ce1e0 0x7f02b40af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02ba400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02b40cfa40 con 0x7f02b40ce1e0 2024-09-10T23:32:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02ba400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f02b40cfd40 con 0x7f02b40ce890 2024-09-10T23:32:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02b2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b40ce1e0 0x7f02b40af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:35980/0 (socket says 172.21.15.94:35980) 2024-09-10T23:32:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02b2a00640 1 -- 172.21.15.94:0/2548457047 learned_addr learned my addr 172.21.15.94:0/2548457047 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02b3400640 1 --2- 172.21.15.94:0/2548457047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02b40b1100 0x7f02b42075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02b2a00640 1 -- 172.21.15.94:0/2548457047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02b40ce890 msgr2=0x7f02b40afe00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T23:32:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02b2a00640 1 --2- 172.21.15.94:0/2548457047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02b40ce890 0x7f02b40afe00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02b2a00640 1 -- 172.21.15.94:0/2548457047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02b40b1100 msgr2=0x7f02b42075b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02b2a00640 1 --2- 172.21.15.94:0/2548457047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02b40b1100 0x7f02b42075b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02b2a00640 1 -- 172.21.15.94:0/2548457047 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b4207af0 con 0x7f02b40ce1e0 2024-09-10T23:32:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02b2a00640 1 --2- 172.21.15.94:0/2548457047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b40ce1e0 0x7f02b40af8c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f02a000c940 tx=0x7f02a000ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02ab400640 1 -- 172.21.15.94:0/2548457047 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0004230 con 0x7f02b40ce1e0 2024-09-10T23:32:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02ab400640 1 -- 172.21.15.94:0/2548457047 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02a00044c0 con 0x7f02b40ce1e0 2024-09-10T23:32:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.289+0000 7f02ab400640 1 -- 172.21.15.94:0/2548457047 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0007c10 con 0x7f02b40ce1e0 2024-09-10T23:32:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.293+0000 7f02ba400640 1 -- 172.21.15.94:0/2548457047 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b40b73f0 con 0x7f02b40ce1e0 2024-09-10T23:32:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.293+0000 7f02ba400640 1 -- 172.21.15.94:0/2548457047 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b40b6950 con 0x7f02b40ce1e0 2024-09-10T23:32:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.293+0000 7f02ab400640 1 -- 172.21.15.94:0/2548457047 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f02a0020060 con 0x7f02b40ce1e0 2024-09-10T23:32:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.297+0000 7f02ab400640 1 --2- 172.21.15.94:0/2548457047 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f02940726e0 0x7f0294074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.297+0000 7f02ab400640 1 -- 172.21.15.94:0/2548457047 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f02a0092f80 con 0x7f02b40ce1e0 2024-09-10T23:32:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.297+0000 7f02b3400640 1 --2- 172.21.15.94:0/2548457047 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f02940726e0 0x7f0294074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.293+0000 7f02ba400640 1 -- 172.21.15.94:0/2548457047 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b40cec70 con 0x7f02b40ce1e0 2024-09-10T23:32:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.301+0000 7f02b3400640 1 --2- 172.21.15.94:0/2548457047 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f02940726e0 0x7f0294074ba0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f02a40058d0 tx=0x7f02a4005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.305+0000 7f02ab400640 1 -- 172.21.15.94:0/2548457047 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f02a0014070 con 0x7f02b40ce1e0 2024-09-10T23:32:55.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.305+0000 7fe8b7000640 1 -- 172.21.15.94:0/3789641584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe898004810 msgr2=0x7fe898004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.305+0000 7fe8b7000640 1 --2- 172.21.15.94:0/3789641584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe898004810 0x7fe898004c90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009b60 tx=0x7fe8a401d270 comp rx=0 tx=0).stop 2024-09-10T23:32:55.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.305+0000 7fe8b7000640 1 -- 172.21.15.94:0/3789641584 shutdown_connections 2024-09-10T23:32:55.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.305+0000 7fe8b7000640 1 --2- 172.21.15.94:0/3789641584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe898004810 0x7fe898004c90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.305+0000 7fe8b7000640 1 -- 172.21.15.94:0/3789641584 >> 172.21.15.94:0/3789641584 conn(0x7fe8b00d00e0 msgr2=0x7fe8b00d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:55.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.305+0000 7fe8b7000640 1 -- 172.21.15.94:0/3789641584 shutdown_connections 2024-09-10T23:32:55.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.309+0000 7fe8b7000640 1 -- 172.21.15.94:0/3789641584 wait complete. 2024-09-10T23:32:55.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.313+0000 7fe8b7000640 1 Processor -- start 2024-09-10T23:32:55.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.313+0000 7fe8b7000640 1 -- start start 2024-09-10T23:32:55.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b7000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe898004810 0x7fe8b014ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b7000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8b0148770 0x7fe8b01ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b7000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b0200110 0x7fe8b02005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b5c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe898004810 0x7fe8b014ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b7000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0161810 con 0x7fe898004810 2024-09-10T23:32:55.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b7000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0161690 con 0x7fe8b0200110 2024-09-10T23:32:55.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b7000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b0161990 con 0x7fe8b0148770 2024-09-10T23:32:55.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8b0148770 0x7fe8b01ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8b0148770 0x7fe8b01ffbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:50372/0 (socket says 172.21.15.94:50372) 2024-09-10T23:32:55.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b5200640 1 -- 172.21.15.94:0/127978845 learned_addr learned my addr 172.21.15.94:0/127978845 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:55.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b6600640 1 --2- 172.21.15.94:0/127978845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b0200110 0x7fe8b02005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b6600640 1 -- 172.21.15.94:0/127978845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8b0148770 msgr2=0x7fe8b01ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b6600640 1 --2- 172.21.15.94:0/127978845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8b0148770 0x7fe8b01ffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b6600640 1 -- 172.21.15.94:0/127978845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe898004810 msgr2=0x7fe8b014ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b6600640 1 --2- 172.21.15.94:0/127978845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe898004810 0x7fe8b014ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b6600640 1 -- 172.21.15.94:0/127978845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b01606f0 con 0x7fe8b0200110 2024-09-10T23:32:55.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b5200640 1 --2- 172.21.15.94:0/127978845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8b0148770 0x7fe8b01ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:32:55.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.317+0000 7fe8b5c00640 1 --2- 172.21.15.94:0/127978845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe898004810 0x7fe8b014ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:32:55.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.321+0000 7fe8b6600640 1 --2- 172.21.15.94:0/127978845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b0200110 0x7fe8b02005c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00709c0 tx=0x7fe8a8006430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:55.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.321+0000 7fe8ad600640 1 -- 172.21.15.94:0/127978845 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8015070 con 0x7fe8b0200110 2024-09-10T23:32:55.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.321+0000 7fe8ad600640 1 -- 172.21.15.94:0/127978845 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8a8011900 con 0x7fe8b0200110 2024-09-10T23:32:55.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.321+0000 7fe8ad600640 1 -- 172.21.15.94:0/127978845 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8011cf0 con 0x7fe8b0200110 2024-09-10T23:32:55.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.321+0000 7fe8b7000640 1 -- 172.21.15.94:0/127978845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b014b680 con 0x7fe8b0200110 2024-09-10T23:32:55.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.321+0000 7fe8b7000640 1 -- 172.21.15.94:0/127978845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b014b8f0 con 0x7fe8b0200110 2024-09-10T23:32:55.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.321+0000 7fe8b7000640 1 -- 172.21.15.94:0/127978845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b0161ef0 con 0x7fe8b0200110 2024-09-10T23:32:55.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.325+0000 7fe8ad600640 1 -- 172.21.15.94:0/127978845 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe8a801a450 con 0x7fe8b0200110 2024-09-10T23:32:55.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.325+0000 7fe8ad600640 1 --2- 172.21.15.94:0/127978845 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe894072610 0x7fe894074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.325+0000 7fe8ad600640 1 -- 172.21.15.94:0/127978845 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe8a8094220 con 0x7fe8b0200110 2024-09-10T23:32:55.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.325+0000 7fe8b5c00640 1 --2- 172.21.15.94:0/127978845 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe894072610 0x7fe894074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.329+0000 7fe8b5c00640 1 --2- 172.21.15.94:0/127978845 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe894072610 0x7fe894074ad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe8b006a000 tx=0x7fe8a0001090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:55.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.333+0000 7fe8ad600640 1 -- 172.21.15.94:0/127978845 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe8a805f220 con 0x7fe8b0200110 2024-09-10T23:32:55.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.365+0000 7fe1b6200640 1 -- 172.21.15.94:0/866664044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1a8055070 msgr2=0x7fe1a80337a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.365+0000 7fe1b6200640 1 --2- 172.21.15.94:0/866664044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1a8055070 0x7fe1a80337a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe1b006c460 tx=0x7fe1b0098c30 comp rx=0 tx=0).stop 2024-09-10T23:32:55.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.365+0000 7fe1b6200640 1 -- 172.21.15.94:0/866664044 shutdown_connections 2024-09-10T23:32:55.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.365+0000 7fe1b6200640 1 --2- 172.21.15.94:0/866664044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1a8055070 0x7fe1a80337a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.365+0000 7fe1b6200640 1 -- 172.21.15.94:0/866664044 >> 172.21.15.94:0/866664044 conn(0x7fe1a801d780 msgr2=0x7fe1a8054570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:55.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.365+0000 7fe1b6200640 1 -- 172.21.15.94:0/866664044 shutdown_connections 2024-09-10T23:32:55.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.369+0000 7fe1b6200640 1 -- 172.21.15.94:0/866664044 wait complete. 2024-09-10T23:32:55.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.369+0000 7fe1b6200640 1 Processor -- start 2024-09-10T23:32:55.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.369+0000 7fe1b6200640 1 -- start start 2024-09-10T23:32:55.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7f1aeb800640 1 -- 172.21.15.94:0/1885873482 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ae415df60 msgr2=0x7f1ae415e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7f1aeb800640 1 --2- 172.21.15.94:0/1885873482 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ae415df60 0x7f1ae415e620 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400dfc0 tx=0x7f1ad4025020 comp rx=0 tx=0).stop 2024-09-10T23:32:55.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7f1aeb800640 1 -- 172.21.15.94:0/1885873482 shutdown_connections 2024-09-10T23:32:55.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1b6200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1a8036250 0x7fe1a80de830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1b6200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1a8036630 0x7fe1a804eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1b6200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1a8055070 0x7fe1a804f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1b6200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a80e2820 con 0x7fe1a8036630 2024-09-10T23:32:55.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1b6200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a80e26a0 con 0x7fe1a8055070 2024-09-10T23:32:55.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1b6200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a80e29a0 con 0x7fe1a8036250 2024-09-10T23:32:55.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1b4e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1a8036250 0x7fe1a80de830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1b5800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1a8055070 0x7fe1a804f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1b4e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1a8036250 0x7fe1a80de830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:50392/0 (socket says 172.21.15.94:50392) 2024-09-10T23:32:55.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1b4e00640 1 -- 172.21.15.94:0/3629805302 learned_addr learned my addr 172.21.15.94:0/3629805302 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:55.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1afe00640 1 --2- 172.21.15.94:0/3629805302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1a8036630 0x7fe1a804eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1b5800640 1 -- 172.21.15.94:0/3629805302 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1a8036250 msgr2=0x7fe1a80de830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1b5800640 1 --2- 172.21.15.94:0/3629805302 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1a8036250 0x7fe1a80de830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1b5800640 1 -- 172.21.15.94:0/3629805302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1a8036630 msgr2=0x7fe1a804eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1b5800640 1 --2- 172.21.15.94:0/3629805302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1a8036630 0x7fe1a804eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1b5800640 1 -- 172.21.15.94:0/3629805302 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a80ded70 con 0x7fe1a8055070 2024-09-10T23:32:55.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7fe1b5800640 1 --2- 172.21.15.94:0/3629805302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1a8055070 0x7fe1a804f0c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1b006c150 tx=0x7fe1b0063a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:55.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7f1aeb800640 1 --2- 172.21.15.94:0/1885873482 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ae415df60 0x7f1ae415e620 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7f1aeb800640 1 -- 172.21.15.94:0/1885873482 >> 172.21.15.94:0/1885873482 conn(0x7f1ae40b27c0 msgr2=0x7f1ae40b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:55.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7f1aeb800640 1 -- 172.21.15.94:0/1885873482 shutdown_connections 2024-09-10T23:32:55.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.373+0000 7f1aeb800640 1 -- 172.21.15.94:0/1885873482 wait complete. 2024-09-10T23:32:55.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.377+0000 7fe1acc00640 1 -- 172.21.15.94:0/3629805302 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0063b40 con 0x7fe1a8055070 2024-09-10T23:32:55.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.377+0000 7fe1acc00640 1 -- 172.21.15.94:0/3629805302 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1b0063ce0 con 0x7fe1a8055070 2024-09-10T23:32:55.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.377+0000 7fe1b6200640 1 -- 172.21.15.94:0/3629805302 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a8053f40 con 0x7fe1a8055070 2024-09-10T23:32:55.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.377+0000 7fe1acc00640 1 -- 172.21.15.94:0/3629805302 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0131410 con 0x7fe1a8055070 2024-09-10T23:32:55.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.377+0000 7fe1b6200640 1 -- 172.21.15.94:0/3629805302 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a8054500 con 0x7fe1a8055070 2024-09-10T23:32:55.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.377+0000 7f1aeb800640 1 Processor -- start 2024-09-10T23:32:55.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.377+0000 7fe1a2000640 1 -- 172.21.15.94:0/3629805302 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a80e2e20 con 0x7fe1a8055070 2024-09-10T23:32:55.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.377+0000 7f16a3e00640 1 -- 172.21.15.94:0/863901155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16a415aa30 msgr2=0x7f16a415ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.377+0000 7f16a3e00640 1 --2- 172.21.15.94:0/863901155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16a415aa30 0x7f16a415ee20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1694009840 tx=0x7f169401c820 comp rx=0 tx=0).stop 2024-09-10T23:32:55.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.377+0000 7fe1acc00640 1 -- 172.21.15.94:0/3629805302 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe1b014eaf0 con 0x7fe1a8055070 2024-09-10T23:32:55.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f1aeb800640 1 -- start start 2024-09-10T23:32:55.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7fe1acc00640 1 --2- 172.21.15.94:0/3629805302 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe1900725e0 0x7fe190074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7fe1acc00640 1 -- 172.21.15.94:0/3629805302 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe1b01a72b0 con 0x7fe1a8055070 2024-09-10T23:32:55.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f16a3e00640 1 -- 172.21.15.94:0/863901155 shutdown_connections 2024-09-10T23:32:55.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f16a3e00640 1 --2- 172.21.15.94:0/863901155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16a40cd360 0x7f16a40cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7fe1b4e00640 1 --2- 172.21.15.94:0/3629805302 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe1900725e0 0x7fe190074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f16a3e00640 1 --2- 172.21.15.94:0/863901155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16a415aa30 0x7f16a415ee20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f16a3e00640 1 -- 172.21.15.94:0/863901155 >> 172.21.15.94:0/863901155 conn(0x7f16a40b2520 msgr2=0x7f16a40b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:55.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f16a3e00640 1 -- 172.21.15.94:0/863901155 shutdown_connections 2024-09-10T23:32:55.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f16a3e00640 1 -- 172.21.15.94:0/863901155 wait complete. 2024-09-10T23:32:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f1aeb800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ae4073630 0x7f1ae41f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f1aeb800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ae40d06e0 0x7f1ae41f7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f1aeb800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ae415df60 0x7f1ae4159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f1ae8e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ae4073630 0x7f1ae41f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f1ae8e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ae4073630 0x7f1ae41f54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:50396/0 (socket says 172.21.15.94:50396) 2024-09-10T23:32:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f1adbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ae40d06e0 0x7f1ae41f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f1adbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ae40d06e0 0x7f1ae41f7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:36026/0 (socket says 172.21.15.94:36026) 2024-09-10T23:32:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f1ae9800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ae415df60 0x7f1ae4159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f1ae9800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ae415df60 0x7f1ae4159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40256/0 (socket says 172.21.15.94:40256) 2024-09-10T23:32:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f1ae8e00640 1 -- 172.21.15.94:0/3643081335 learned_addr learned my addr 172.21.15.94:0/3643081335 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f1aeb800640 1 -- 172.21.15.94:0/3643081335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae4168940 con 0x7f1ae415df60 2024-09-10T23:32:55.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f1aeb800640 1 -- 172.21.15.94:0/3643081335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae41687c0 con 0x7f1ae40d06e0 2024-09-10T23:32:55.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.381+0000 7f1aeb800640 1 -- 172.21.15.94:0/3643081335 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae4168ac0 con 0x7f1ae4073630 2024-09-10T23:32:55.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f1ae8e00640 1 -- 172.21.15.94:0/3643081335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ae40d06e0 msgr2=0x7f1ae41f7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f16a3e00640 1 Processor -- start 2024-09-10T23:32:55.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f1ae8e00640 1 --2- 172.21.15.94:0/3643081335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ae40d06e0 0x7f1ae41f7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f1ae8e00640 1 -- 172.21.15.94:0/3643081335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ae415df60 msgr2=0x7f1ae4159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f1ae8e00640 1 --2- 172.21.15.94:0/3643081335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ae415df60 0x7f1ae4159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f1ae8e00640 1 -- 172.21.15.94:0/3643081335 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae415b0f0 con 0x7f1ae4073630 2024-09-10T23:32:55.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7fe1b4e00640 1 --2- 172.21.15.94:0/3629805302 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe1900725e0 0x7fe190074aa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4000fc0 tx=0x7fe1a4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:55.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f1ae8e00640 1 --2- 172.21.15.94:0/3643081335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ae4073630 0x7f1ae41f54f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4024e50 tx=0x7f1ad400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:55.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f16a3e00640 1 -- start start 2024-09-10T23:32:55.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7fe1acc00640 1 -- 172.21.15.94:0/3629805302 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe1b0092020 con 0x7fe1a8055070 2024-09-10T23:32:55.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f1ae1600640 1 -- 172.21.15.94:0/3643081335 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4003b80 con 0x7f1ae4073630 2024-09-10T23:32:55.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f1ae1600640 1 -- 172.21.15.94:0/3643081335 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ad4003e10 con 0x7f1ae4073630 2024-09-10T23:32:55.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f1ae1600640 1 -- 172.21.15.94:0/3643081335 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4004290 con 0x7f1ae4073630 2024-09-10T23:32:55.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f1aeb800640 1 -- 172.21.15.94:0/3643081335 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae41f76a0 con 0x7f1ae4073630 2024-09-10T23:32:55.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f1aeb800640 1 -- 172.21.15.94:0/3643081335 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae415b390 con 0x7f1ae4073630 2024-09-10T23:32:55.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f16a3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16a4073410 0x7f16a4148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f16a3e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16a40cd360 0x7f16a4156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f16a3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a415aa30 0x7f16a41597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f16a3e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16a4070b60 con 0x7f16a415aa30 2024-09-10T23:32:55.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f16a3e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f16a40709e0 con 0x7f16a4073410 2024-09-10T23:32:55.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f16a3e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f16a4070ce0 con 0x7f16a40cd360 2024-09-10T23:32:55.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f16a2000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16a40cd360 0x7f16a4156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f16a2000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16a40cd360 0x7f16a4156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:50404/0 (socket says 172.21.15.94:50404) 2024-09-10T23:32:55.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.385+0000 7f16a2000640 1 -- 172.21.15.94:0/3748576929 learned_addr learned my addr 172.21.15.94:0/3748576929 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:55.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.389+0000 7f16a2a00640 1 --2- 172.21.15.94:0/3748576929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16a4073410 0x7f16a4148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.389+0000 7f16a3400640 1 --2- 172.21.15.94:0/3748576929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a415aa30 0x7f16a41597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.389+0000 7f16a2000640 1 -- 172.21.15.94:0/3748576929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16a4073410 msgr2=0x7f16a4148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.389+0000 7f16a2000640 1 --2- 172.21.15.94:0/3748576929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16a4073410 0x7f16a4148cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.389+0000 7f16a2000640 1 -- 172.21.15.94:0/3748576929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a415aa30 msgr2=0x7f16a41597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.389+0000 7f16a2000640 1 --2- 172.21.15.94:0/3748576929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a415aa30 0x7f16a41597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.389+0000 7f16a2000640 1 -- 172.21.15.94:0/3748576929 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a41577f0 con 0x7f16a40cd360 2024-09-10T23:32:55.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.389+0000 7f16a2000640 1 --2- 172.21.15.94:0/3748576929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16a40cd360 0x7f16a4156d30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f169000e9c0 tx=0x7f169000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:55.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.389+0000 7f1aeb800640 1 -- 172.21.15.94:0/3643081335 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae415e620 con 0x7f1ae4073630 2024-09-10T23:32:55.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.389+0000 7f169aa00640 1 -- 172.21.15.94:0/3748576929 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169000cd80 con 0x7f16a40cd360 2024-09-10T23:32:55.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.389+0000 7f169aa00640 1 -- 172.21.15.94:0/3748576929 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1690004020 con 0x7f16a40cd360 2024-09-10T23:32:55.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.389+0000 7f169aa00640 1 -- 172.21.15.94:0/3748576929 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690004450 con 0x7f16a40cd360 2024-09-10T23:32:55.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.389+0000 7f16a3e00640 1 -- 172.21.15.94:0/3748576929 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a4157ad0 con 0x7f16a40cd360 2024-09-10T23:32:55.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.389+0000 7f1ae1600640 1 -- 172.21.15.94:0/3643081335 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1ad4004430 con 0x7f1ae4073630 2024-09-10T23:32:55.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.389+0000 7f16a3e00640 1 -- 172.21.15.94:0/3748576929 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a4149210 con 0x7f16a40cd360 2024-09-10T23:32:55.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.389+0000 7f1ae1600640 1 --2- 172.21.15.94:0/3643081335 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1ac00725e0 0x7f1ac0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.393+0000 7f1ae1600640 1 -- 172.21.15.94:0/3643081335 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1ad4135f40 con 0x7f1ae4073630 2024-09-10T23:32:55.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.393+0000 7f1adbe00640 1 --2- 172.21.15.94:0/3643081335 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1ac00725e0 0x7f1ac0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.393+0000 7f1adbe00640 1 --2- 172.21.15.94:0/3643081335 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1ac00725e0 0x7f1ac0074aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1ae415d7d0 tx=0x7f1acc0090e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.393+0000 7f1ae1600640 1 -- 172.21.15.94:0/3643081335 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1ad401ec10 con 0x7f1ae4073630 2024-09-10T23:32:55.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.397+0000 7f16a3e00640 1 -- 172.21.15.94:0/3748576929 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a40cd7a0 con 0x7f16a40cd360 2024-09-10T23:32:55.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.397+0000 7f169aa00640 1 -- 172.21.15.94:0/3748576929 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1690003600 con 0x7f16a40cd360 2024-09-10T23:32:55.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.397+0000 7f02ae400640 1 -- 172.21.15.94:0/716148674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02a80ccfb0 msgr2=0x7f02a8162460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.397+0000 7f02ae400640 1 --2- 172.21.15.94:0/716148674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02a80ccfb0 0x7f02a8162460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0298009840 tx=0x7f029801c820 comp rx=0 tx=0).stop 2024-09-10T23:32:55.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.397+0000 7f02ae400640 1 -- 172.21.15.94:0/716148674 shutdown_connections 2024-09-10T23:32:55.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.397+0000 7f02ae400640 1 --2- 172.21.15.94:0/716148674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02a80ccfb0 0x7f02a8162460 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.397+0000 7f02ae400640 1 --2- 172.21.15.94:0/716148674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02a8071cd0 0x7f02a80d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.397+0000 7f02ae400640 1 -- 172.21.15.94:0/716148674 >> 172.21.15.94:0/716148674 conn(0x7f02a80d0c00 msgr2=0x7f02a80d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:55.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.401+0000 7f02ae400640 1 -- 172.21.15.94:0/716148674 shutdown_connections 2024-09-10T23:32:55.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.401+0000 7f02ae400640 1 -- 172.21.15.94:0/716148674 wait complete. 2024-09-10T23:32:55.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.401+0000 7f169aa00640 1 --2- 172.21.15.94:0/3748576929 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1684072610 0x7f1684074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.401+0000 7f169aa00640 1 -- 172.21.15.94:0/3748576929 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1690093d20 con 0x7f16a40cd360 2024-09-10T23:32:55.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.401+0000 7f02ae400640 1 Processor -- start 2024-09-10T23:32:55.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.401+0000 7f16a2a00640 1 --2- 172.21.15.94:0/3748576929 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1684072610 0x7f1684074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.405+0000 7f169aa00640 1 -- 172.21.15.94:0/3748576929 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f169005f3f0 con 0x7f16a40cd360 2024-09-10T23:32:55.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.405+0000 7f02ae400640 1 -- start start 2024-09-10T23:32:55.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.405+0000 7f16a2a00640 1 --2- 172.21.15.94:0/3748576929 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1684072610 0x7f1684074ad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f16940af000 tx=0x7f169401b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.405+0000 7f02ae400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02a80718f0 0x7f02a8159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.405+0000 7f02ae400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a8071cd0 0x7f02a815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.405+0000 7f02ae400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02a80ccfb0 0x7f02a806fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.405+0000 7f02a7400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02a80718f0 0x7f02a8159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.405+0000 7f02a7400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02a80718f0 0x7f02a8159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:50420/0 (socket says 172.21.15.94:50420) 2024-09-10T23:32:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.405+0000 7f02a6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a8071cd0 0x7f02a815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.405+0000 7f02a7400640 1 -- 172.21.15.94:0/2246493762 learned_addr learned my addr 172.21.15.94:0/2246493762 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.409+0000 7f02ae400640 1 -- 172.21.15.94:0/2246493762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f02a816f710 con 0x7f02a8071cd0 2024-09-10T23:32:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.409+0000 7f02ae400640 1 -- 172.21.15.94:0/2246493762 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02a816f590 con 0x7f02a80ccfb0 2024-09-10T23:32:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.409+0000 7f02ae400640 1 -- 172.21.15.94:0/2246493762 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f02a816f890 con 0x7f02a80718f0 2024-09-10T23:32:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.409+0000 7f02a7e00640 1 --2- 172.21.15.94:0/2246493762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02a80ccfb0 0x7f02a806fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.409+0000 7f02a7400640 1 -- 172.21.15.94:0/2246493762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02a80ccfb0 msgr2=0x7f02a806fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.409+0000 7f02a7400640 1 --2- 172.21.15.94:0/2246493762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02a80ccfb0 0x7f02a806fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.409+0000 7f02a7400640 1 -- 172.21.15.94:0/2246493762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a8071cd0 msgr2=0x7f02a815c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.409+0000 7f02a7400640 1 --2- 172.21.15.94:0/2246493762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a8071cd0 0x7f02a815c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.409+0000 7f02a7400640 1 -- 172.21.15.94:0/2246493762 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a81600b0 con 0x7f02a80718f0 2024-09-10T23:32:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.409+0000 7f02a7400640 1 --2- 172.21.15.94:0/2246493762 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02a80718f0 0x7f02a8159ce0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f029400c420 tx=0x7f029400c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.409+0000 7f029f400640 1 -- 172.21.15.94:0/2246493762 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294017070 con 0x7f02a80718f0 2024-09-10T23:32:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.409+0000 7f029f400640 1 -- 172.21.15.94:0/2246493762 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0294003600 con 0x7f02a80718f0 2024-09-10T23:32:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.409+0000 7f029f400640 1 -- 172.21.15.94:0/2246493762 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294003a70 con 0x7f02a80718f0 2024-09-10T23:32:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.409+0000 7f02ae400640 1 -- 172.21.15.94:0/2246493762 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a816aff0 con 0x7f02a80718f0 2024-09-10T23:32:55.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.409+0000 7f02ae400640 1 -- 172.21.15.94:0/2246493762 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a806f450 con 0x7f02a80718f0 2024-09-10T23:32:55.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.413+0000 7f029f400640 1 -- 172.21.15.94:0/2246493762 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0294033ce0 con 0x7f02a80718f0 2024-09-10T23:32:55.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.417+0000 7f02ae400640 1 -- 172.21.15.94:0/2246493762 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a80d02a0 con 0x7f02a80718f0 2024-09-10T23:32:55.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.417+0000 7f029f400640 1 --2- 172.21.15.94:0/2246493762 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0288072630 0x7f0288074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.417+0000 7f029f400640 1 -- 172.21.15.94:0/2246493762 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0294092710 con 0x7f02a80718f0 2024-09-10T23:32:55.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.417+0000 7f02a6a00640 1 --2- 172.21.15.94:0/2246493762 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0288072630 0x7f0288074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.417+0000 7f02a6a00640 1 --2- 172.21.15.94:0/2246493762 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0288072630 0x7f0288074af0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f02a8070a70 tx=0x7f02a0008230 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:55.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.421+0000 7f029f400640 1 -- 172.21.15.94:0/2246493762 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f029405d7f0 con 0x7f02a80718f0 2024-09-10T23:32:55.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.457+0000 7f245ce00640 1 -- 172.21.15.94:0/3522657478 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2458157500 msgr2=0x7f245815e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.457+0000 7f245ce00640 1 --2- 172.21.15.94:0/3522657478 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2458157500 0x7f245815e330 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f24480074e0 tx=0x7f244801d560 comp rx=0 tx=0).stop 2024-09-10T23:32:55.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.457+0000 7f245ce00640 1 -- 172.21.15.94:0/3522657478 shutdown_connections 2024-09-10T23:32:55.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.457+0000 7f245ce00640 1 --2- 172.21.15.94:0/3522657478 >> v2:172.21.15.94:3300/0 conn(0x7f245806cc20 0x7f245806d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.457+0000 7f245ce00640 1 --2- 172.21.15.94:0/3522657478 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2458157500 0x7f245815e330 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.457+0000 7f245ce00640 1 -- 172.21.15.94:0/3522657478 >> 172.21.15.94:0/3522657478 conn(0x7f24580cf8d0 msgr2=0x7f24580cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:55.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.457+0000 7f245ce00640 1 -- 172.21.15.94:0/3522657478 shutdown_connections 2024-09-10T23:32:55.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.457+0000 7f245ce00640 1 -- 172.21.15.94:0/3522657478 wait complete. 2024-09-10T23:32:55.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f245ce00640 1 Processor -- start 2024-09-10T23:32:55.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f245ce00640 1 -- start start 2024-09-10T23:32:55.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f245ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f245806cc20 0x7f24580b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f245ce00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2458157500 0x7f245815a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f245ce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24581578e0 0x7f2458150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f245ce00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24580cd4a0 con 0x7f24581578e0 2024-09-10T23:32:55.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f245ce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2458073f60 con 0x7f245806cc20 2024-09-10T23:32:55.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f245ce00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f24580cd620 con 0x7f2458157500 2024-09-10T23:32:55.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f2456a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2458157500 0x7f245815a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f2456a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2458157500 0x7f245815a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:50438/0 (socket says 172.21.15.94:50438) 2024-09-10T23:32:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f2456a00640 1 -- 172.21.15.94:0/291461920 learned_addr learned my addr 172.21.15.94:0/291461920 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f2457e00640 1 --2- 172.21.15.94:0/291461920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24581578e0 0x7f2458150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f2457400640 1 --2- 172.21.15.94:0/291461920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f245806cc20 0x7f24580b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f2456a00640 1 -- 172.21.15.94:0/291461920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f245806cc20 msgr2=0x7f24580b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f2456a00640 1 --2- 172.21.15.94:0/291461920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f245806cc20 0x7f24580b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f2456a00640 1 -- 172.21.15.94:0/291461920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24581578e0 msgr2=0x7f2458150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f2456a00640 1 --2- 172.21.15.94:0/291461920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24581578e0 0x7f2458150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f2456a00640 1 -- 172.21.15.94:0/291461920 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24580b3110 con 0x7f2458157500 2024-09-10T23:32:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.461+0000 7f2457e00640 1 --2- 172.21.15.94:0/291461920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24581578e0 0x7f2458150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.465+0000 7f2457400640 1 --2- 172.21.15.94:0/291461920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f245806cc20 0x7f24580b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:32:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.465+0000 7f2456a00640 1 --2- 172.21.15.94:0/291461920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2458157500 0x7f245815a6f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f244400c3c0 tx=0x7f244400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.465+0000 7f244f400640 1 -- 172.21.15.94:0/291461920 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444017070 con 0x7f2458157500 2024-09-10T23:32:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.465+0000 7f244f400640 1 -- 172.21.15.94:0/291461920 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24440053c0 con 0x7f2458157500 2024-09-10T23:32:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.465+0000 7f244f400640 1 -- 172.21.15.94:0/291461920 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444003860 con 0x7f2458157500 2024-09-10T23:32:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.465+0000 7f245ce00640 1 -- 172.21.15.94:0/291461920 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24580b33f0 con 0x7f2458157500 2024-09-10T23:32:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.465+0000 7f245ce00640 1 -- 172.21.15.94:0/291461920 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f24580b39b0 con 0x7f2458157500 2024-09-10T23:32:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.465+0000 7f245ce00640 1 -- 172.21.15.94:0/291461920 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245806d090 con 0x7f2458157500 2024-09-10T23:32:55.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.469+0000 7f244f400640 1 -- 172.21.15.94:0/291461920 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2444003a00 con 0x7f2458157500 2024-09-10T23:32:55.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.469+0000 7f244f400640 1 --2- 172.21.15.94:0/291461920 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24380725e0 0x7f2438074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:55.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.469+0000 7f244f400640 1 -- 172.21.15.94:0/291461920 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2444090e40 con 0x7f2458157500 2024-09-10T23:32:55.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.473+0000 7f2457400640 1 --2- 172.21.15.94:0/291461920 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24380725e0 0x7f2438074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:55.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.473+0000 7f2457400640 1 --2- 172.21.15.94:0/291461920 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24380725e0 0x7f2438074aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f24480af000 tx=0x7f244801bda0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:55.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.477+0000 7f244f400640 1 -- 172.21.15.94:0/291461920 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f244405dee0 con 0x7f2458157500 2024-09-10T23:32:55.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.605+0000 7fc066200640 1 -- 172.21.15.94:0/264539947 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc0600cfe50 con 0x7fc06006d260 2024-09-10T23:32:55.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.605+0000 7fc05cc00640 1 -- 172.21.15.94:0/264539947 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc04c0010d0 con 0x7fc06006d260 2024-09-10T23:32:55.613 INFO:teuthology.orchestra.run.smithi094.stdout:64424509446 2024-09-10T23:32:55.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.609+0000 7fc066200640 1 -- 172.21.15.94:0/264539947 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc0400725e0 msgr2=0x7fc040074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.609+0000 7fc066200640 1 --2- 172.21.15.94:0/264539947 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc0400725e0 0x7fc040074aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc0540af000 tx=0x7fc05401b110 comp rx=0 tx=0).stop 2024-09-10T23:32:55.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.609+0000 7fc066200640 1 -- 172.21.15.94:0/264539947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc06006d260 msgr2=0x7fc0600b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.613+0000 7fc066200640 1 --2- 172.21.15.94:0/264539947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc06006d260 0x7fc0600b5000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00c440 tx=0x7fc04c00c900 comp rx=0 tx=0).stop 2024-09-10T23:32:55.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.613+0000 7fc066200640 1 -- 172.21.15.94:0/264539947 shutdown_connections 2024-09-10T23:32:55.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.613+0000 7fc066200640 1 --2- 172.21.15.94:0/264539947 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc0400725e0 0x7fc040074aa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.613+0000 7fc066200640 1 --2- 172.21.15.94:0/264539947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0600cdc90 0x7fc0600ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.613+0000 7fc066200640 1 --2- 172.21.15.94:0/264539947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc06006d260 0x7fc0600b5000 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.613+0000 7fc066200640 1 --2- 172.21.15.94:0/264539947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc06006ce80 0x7fc0600c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.613+0000 7fc066200640 1 -- 172.21.15.94:0/264539947 >> 172.21.15.94:0/264539947 conn(0x7fc0600cf8d0 msgr2=0x7fc060161e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:55.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.613+0000 7fc066200640 1 -- 172.21.15.94:0/264539947 shutdown_connections 2024-09-10T23:32:55.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.613+0000 7fc066200640 1 -- 172.21.15.94:0/264539947 wait complete. 2024-09-10T23:32:55.647 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.0 2024-09-10T23:32:55.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.669+0000 7f02ba400640 1 -- 172.21.15.94:0/2548457047 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f02b40b75c0 con 0x7f02b40ce1e0 2024-09-10T23:32:55.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.669+0000 7f02ab400640 1 -- 172.21.15.94:0/2548457047 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f02a005e6d0 con 0x7f02b40ce1e0 2024-09-10T23:32:55.674 INFO:teuthology.orchestra.run.smithi094.stdout:64424509447 2024-09-10T23:32:55.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.673+0000 7f02ba400640 1 -- 172.21.15.94:0/2548457047 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f02940726e0 msgr2=0x7f0294074ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.673+0000 7f02ba400640 1 --2- 172.21.15.94:0/2548457047 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f02940726e0 0x7f0294074ba0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f02a40058d0 tx=0x7f02a4005860 comp rx=0 tx=0).stop 2024-09-10T23:32:55.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.673+0000 7f02ba400640 1 -- 172.21.15.94:0/2548457047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b40ce1e0 msgr2=0x7f02b40af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.673+0000 7f02ba400640 1 --2- 172.21.15.94:0/2548457047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b40ce1e0 0x7f02b40af8c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f02a000c940 tx=0x7f02a000ce00 comp rx=0 tx=0).stop 2024-09-10T23:32:55.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.673+0000 7f02ba400640 1 -- 172.21.15.94:0/2548457047 shutdown_connections 2024-09-10T23:32:55.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.673+0000 7f02ba400640 1 --2- 172.21.15.94:0/2548457047 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f02940726e0 0x7f0294074ba0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.673+0000 7f02ba400640 1 --2- 172.21.15.94:0/2548457047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b40ce1e0 0x7f02b40af8c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.673+0000 7f02ba400640 1 --2- 172.21.15.94:0/2548457047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02b40ce890 0x7f02b40afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.673+0000 7f02ba400640 1 --2- 172.21.15.94:0/2548457047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02b40b1100 0x7f02b42075b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.673+0000 7f02ba400640 1 -- 172.21.15.94:0/2548457047 >> 172.21.15.94:0/2548457047 conn(0x7f02b40d1c60 msgr2=0x7f02b415bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:55.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.673+0000 7f02ba400640 1 -- 172.21.15.94:0/2548457047 shutdown_connections 2024-09-10T23:32:55.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.673+0000 7f02ba400640 1 -- 172.21.15.94:0/2548457047 wait complete. 2024-09-10T23:32:55.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.689+0000 7fe8b7000640 1 -- 172.21.15.94:0/127978845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe8b00d0fc0 con 0x7fe8b0200110 2024-09-10T23:32:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.693+0000 7fe8ad600640 1 -- 172.21.15.94:0/127978845 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe8a8063c50 con 0x7fe8b0200110 2024-09-10T23:32:55.694 INFO:teuthology.orchestra.run.smithi094.stdout:64424509447 2024-09-10T23:32:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.693+0000 7fe8b7000640 1 -- 172.21.15.94:0/127978845 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe894072610 msgr2=0x7fe894074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.693+0000 7fe8b7000640 1 --2- 172.21.15.94:0/127978845 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe894072610 0x7fe894074ad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe8b006a000 tx=0x7fe8a0001090 comp rx=0 tx=0).stop 2024-09-10T23:32:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.693+0000 7fe8b7000640 1 -- 172.21.15.94:0/127978845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b0200110 msgr2=0x7fe8b02005c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.693+0000 7fe8b7000640 1 --2- 172.21.15.94:0/127978845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b0200110 0x7fe8b02005c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00709c0 tx=0x7fe8a8006430 comp rx=0 tx=0).stop 2024-09-10T23:32:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.693+0000 7fe8b7000640 1 -- 172.21.15.94:0/127978845 shutdown_connections 2024-09-10T23:32:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.693+0000 7fe8b7000640 1 --2- 172.21.15.94:0/127978845 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe894072610 0x7fe894074ad0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.693+0000 7fe8b7000640 1 --2- 172.21.15.94:0/127978845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b0200110 0x7fe8b02005c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.693+0000 7fe8b7000640 1 --2- 172.21.15.94:0/127978845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8b0148770 0x7fe8b01ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.693+0000 7fe8b7000640 1 --2- 172.21.15.94:0/127978845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe898004810 0x7fe8b014ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.693+0000 7fe8b7000640 1 -- 172.21.15.94:0/127978845 >> 172.21.15.94:0/127978845 conn(0x7fe8b00d00e0 msgr2=0x7fe8b015ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.697+0000 7fe8b7000640 1 -- 172.21.15.94:0/127978845 shutdown_connections 2024-09-10T23:32:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.697+0000 7fe8b7000640 1 -- 172.21.15.94:0/127978845 wait complete. 2024-09-10T23:32:55.710 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2024-09-10T23:32:55.710 DEBUG:teuthology.parallel:result is None 2024-09-10T23:32:55.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.721+0000 7f02ae400640 1 -- 172.21.15.94:0/2246493762 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f02a80b6b80 con 0x7f02a80718f0 2024-09-10T23:32:55.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.721+0000 7f029f400640 1 -- 172.21.15.94:0/2246493762 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0294062220 con 0x7f02a80718f0 2024-09-10T23:32:55.725 INFO:teuthology.orchestra.run.smithi094.stdout:64424509446 2024-09-10T23:32:55.727 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.3 2024-09-10T23:32:55.727 DEBUG:teuthology.parallel:result is None 2024-09-10T23:32:55.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.725+0000 7f02ae400640 1 -- 172.21.15.94:0/2246493762 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0288072630 msgr2=0x7f0288074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.725+0000 7f02ae400640 1 --2- 172.21.15.94:0/2246493762 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0288072630 0x7f0288074af0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f02a8070a70 tx=0x7f02a0008230 comp rx=0 tx=0).stop 2024-09-10T23:32:55.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.725+0000 7f02ae400640 1 -- 172.21.15.94:0/2246493762 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02a80718f0 msgr2=0x7f02a8159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.725+0000 7f02ae400640 1 --2- 172.21.15.94:0/2246493762 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02a80718f0 0x7f02a8159ce0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f029400c420 tx=0x7f029400c8e0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.725+0000 7f02ae400640 1 -- 172.21.15.94:0/2246493762 shutdown_connections 2024-09-10T23:32:55.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.725+0000 7f02ae400640 1 --2- 172.21.15.94:0/2246493762 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0288072630 0x7f0288074af0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.725+0000 7f02ae400640 1 --2- 172.21.15.94:0/2246493762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02a80ccfb0 0x7f02a806fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.725+0000 7f02ae400640 1 --2- 172.21.15.94:0/2246493762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a8071cd0 0x7f02a815c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.725+0000 7f02ae400640 1 --2- 172.21.15.94:0/2246493762 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02a80718f0 0x7f02a8159ce0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.725+0000 7f02ae400640 1 -- 172.21.15.94:0/2246493762 >> 172.21.15.94:0/2246493762 conn(0x7f02a80d0c00 msgr2=0x7f02a81617d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:55.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.725+0000 7f02ae400640 1 -- 172.21.15.94:0/2246493762 shutdown_connections 2024-09-10T23:32:55.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.725+0000 7f02ae400640 1 -- 172.21.15.94:0/2246493762 wait complete. 2024-09-10T23:32:55.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.729+0000 7f1aeb800640 1 -- 172.21.15.94:0/3643081335 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1ae4166400 con 0x7f1ae4073630 2024-09-10T23:32:55.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.729+0000 7f1ae1600640 1 -- 172.21.15.94:0/3643081335 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ad4100f30 con 0x7f1ae4073630 2024-09-10T23:32:55.731 INFO:teuthology.orchestra.run.smithi094.stdout:64424509448 2024-09-10T23:32:55.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.729+0000 7f1aeb800640 1 -- 172.21.15.94:0/3643081335 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1ac00725e0 msgr2=0x7f1ac0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.729+0000 7f1aeb800640 1 --2- 172.21.15.94:0/3643081335 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1ac00725e0 0x7f1ac0074aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1ae415d7d0 tx=0x7f1acc0090e0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.729+0000 7f1aeb800640 1 -- 172.21.15.94:0/3643081335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ae4073630 msgr2=0x7f1ae41f54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.729+0000 7f1aeb800640 1 --2- 172.21.15.94:0/3643081335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ae4073630 0x7f1ae41f54f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4024e50 tx=0x7f1ad400bf90 comp rx=0 tx=0).stop 2024-09-10T23:32:55.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.729+0000 7f1aeb800640 1 -- 172.21.15.94:0/3643081335 shutdown_connections 2024-09-10T23:32:55.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.729+0000 7f1aeb800640 1 --2- 172.21.15.94:0/3643081335 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1ac00725e0 0x7f1ac0074aa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.729+0000 7f1aeb800640 1 --2- 172.21.15.94:0/3643081335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ae40d06e0 0x7f1ae41f7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.729+0000 7f1aeb800640 1 --2- 172.21.15.94:0/3643081335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ae415df60 0x7f1ae4159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.729+0000 7f1aeb800640 1 --2- 172.21.15.94:0/3643081335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ae4073630 0x7f1ae41f54f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.729+0000 7f1aeb800640 1 -- 172.21.15.94:0/3643081335 >> 172.21.15.94:0/3643081335 conn(0x7f1ae40b27c0 msgr2=0x7f1ae40b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:55.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.729+0000 7f1aeb800640 1 -- 172.21.15.94:0/3643081335 shutdown_connections 2024-09-10T23:32:55.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.729+0000 7f1aeb800640 1 -- 172.21.15.94:0/3643081335 wait complete. 2024-09-10T23:32:55.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.733+0000 7fe1a2000640 1 -- 172.21.15.94:0/3629805302 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe1a8005ba0 con 0x7fe1a8055070 2024-09-10T23:32:55.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.733+0000 7fe1acc00640 1 -- 172.21.15.94:0/3629805302 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe1b017b0a0 con 0x7fe1a8055070 2024-09-10T23:32:55.736 INFO:teuthology.orchestra.run.smithi094.stdout:64424509448 2024-09-10T23:32:55.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.733+0000 7fe1a2000640 1 -- 172.21.15.94:0/3629805302 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe1900725e0 msgr2=0x7fe190074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.733+0000 7fe1a2000640 1 --2- 172.21.15.94:0/3629805302 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe1900725e0 0x7fe190074aa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4000fc0 tx=0x7fe1a4009040 comp rx=0 tx=0).stop 2024-09-10T23:32:55.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.733+0000 7fe1a2000640 1 -- 172.21.15.94:0/3629805302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1a8055070 msgr2=0x7fe1a804f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.733+0000 7fe1a2000640 1 --2- 172.21.15.94:0/3629805302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1a8055070 0x7fe1a804f0c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1b006c150 tx=0x7fe1b0063a10 comp rx=0 tx=0).stop 2024-09-10T23:32:55.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.733+0000 7fe1a2000640 1 -- 172.21.15.94:0/3629805302 shutdown_connections 2024-09-10T23:32:55.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.733+0000 7fe1a2000640 1 --2- 172.21.15.94:0/3629805302 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe1900725e0 0x7fe190074aa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.733+0000 7fe1a2000640 1 --2- 172.21.15.94:0/3629805302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1a8055070 0x7fe1a804f0c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.733+0000 7fe1a2000640 1 --2- 172.21.15.94:0/3629805302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1a8036630 0x7fe1a804eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.733+0000 7fe1a2000640 1 --2- 172.21.15.94:0/3629805302 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1a8036250 0x7fe1a80de830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.733+0000 7fe1a2000640 1 -- 172.21.15.94:0/3629805302 >> 172.21.15.94:0/3629805302 conn(0x7fe1a801d780 msgr2=0x7fe1a8033080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:55.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.733+0000 7fe1a2000640 1 -- 172.21.15.94:0/3629805302 shutdown_connections 2024-09-10T23:32:55.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.733+0000 7fe1a2000640 1 -- 172.21.15.94:0/3629805302 wait complete. 2024-09-10T23:32:55.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.4 2024-09-10T23:32:55.755 DEBUG:teuthology.parallel:result is None 2024-09-10T23:32:55.759 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.5 2024-09-10T23:32:55.759 DEBUG:teuthology.parallel:result is None 2024-09-10T23:32:55.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.2 2024-09-10T23:32:55.763 DEBUG:teuthology.parallel:result is None 2024-09-10T23:32:55.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.781+0000 7f16a3e00640 1 -- 172.21.15.94:0/3748576929 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f16a40b0860 con 0x7f16a40cd360 2024-09-10T23:32:55.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.781+0000 7f169aa00640 1 -- 172.21.15.94:0/3748576929 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f169005ecf0 con 0x7f16a40cd360 2024-09-10T23:32:55.784 INFO:teuthology.orchestra.run.smithi094.stdout:64424509449 2024-09-10T23:32:55.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.781+0000 7f16a3e00640 1 -- 172.21.15.94:0/3748576929 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1684072610 msgr2=0x7f1684074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.781+0000 7f16a3e00640 1 --2- 172.21.15.94:0/3748576929 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1684072610 0x7f1684074ad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f16940af000 tx=0x7f169401b060 comp rx=0 tx=0).stop 2024-09-10T23:32:55.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.781+0000 7f16a3e00640 1 -- 172.21.15.94:0/3748576929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16a40cd360 msgr2=0x7f16a4156d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.781+0000 7f16a3e00640 1 --2- 172.21.15.94:0/3748576929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16a40cd360 0x7f16a4156d30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f169000e9c0 tx=0x7f169000ee80 comp rx=0 tx=0).stop 2024-09-10T23:32:55.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.781+0000 7f16a3e00640 1 -- 172.21.15.94:0/3748576929 shutdown_connections 2024-09-10T23:32:55.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.781+0000 7f16a3e00640 1 --2- 172.21.15.94:0/3748576929 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1684072610 0x7f1684074ad0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.781+0000 7f16a3e00640 1 --2- 172.21.15.94:0/3748576929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a415aa30 0x7f16a41597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.781+0000 7f16a3e00640 1 --2- 172.21.15.94:0/3748576929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16a40cd360 0x7f16a4156d30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.781+0000 7f16a3e00640 1 --2- 172.21.15.94:0/3748576929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16a4073410 0x7f16a4148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.781+0000 7f16a3e00640 1 -- 172.21.15.94:0/3748576929 >> 172.21.15.94:0/3748576929 conn(0x7f16a40b2520 msgr2=0x7f16a40b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:55.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.785+0000 7f16a3e00640 1 -- 172.21.15.94:0/3748576929 shutdown_connections 2024-09-10T23:32:55.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.785+0000 7f16a3e00640 1 -- 172.21.15.94:0/3748576929 wait complete. 2024-09-10T23:32:55.805 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.7 2024-09-10T23:32:55.806 DEBUG:teuthology.parallel:result is None 2024-09-10T23:32:55.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.805+0000 7f245ce00640 1 -- 172.21.15.94:0/291461920 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f24580741e0 con 0x7f2458157500 2024-09-10T23:32:55.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.809+0000 7f244f400640 1 -- 172.21.15.94:0/291461920 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f244405d7e0 con 0x7f2458157500 2024-09-10T23:32:55.811 INFO:teuthology.orchestra.run.smithi094.stdout:60129542152 2024-09-10T23:32:55.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.809+0000 7f245ce00640 1 -- 172.21.15.94:0/291461920 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24380725e0 msgr2=0x7f2438074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.809+0000 7f245ce00640 1 --2- 172.21.15.94:0/291461920 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24380725e0 0x7f2438074aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f24480af000 tx=0x7f244801bda0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.809+0000 7f245ce00640 1 -- 172.21.15.94:0/291461920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2458157500 msgr2=0x7f245815a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:55.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.809+0000 7f245ce00640 1 --2- 172.21.15.94:0/291461920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2458157500 0x7f245815a6f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f244400c3c0 tx=0x7f244400c880 comp rx=0 tx=0).stop 2024-09-10T23:32:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.809+0000 7f245ce00640 1 -- 172.21.15.94:0/291461920 shutdown_connections 2024-09-10T23:32:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.809+0000 7f245ce00640 1 --2- 172.21.15.94:0/291461920 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24380725e0 0x7f2438074aa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.809+0000 7f245ce00640 1 --2- 172.21.15.94:0/291461920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24581578e0 0x7f2458150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.809+0000 7f245ce00640 1 --2- 172.21.15.94:0/291461920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2458157500 0x7f245815a6f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.809+0000 7f245ce00640 1 --2- 172.21.15.94:0/291461920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f245806cc20 0x7f24580b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.809+0000 7f245ce00640 1 -- 172.21.15.94:0/291461920 >> 172.21.15.94:0/291461920 conn(0x7f24580cf8d0 msgr2=0x7f24580cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.809+0000 7f245ce00640 1 -- 172.21.15.94:0/291461920 shutdown_connections 2024-09-10T23:32:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:55.809+0000 7f245ce00640 1 -- 172.21.15.94:0/291461920 wait complete. 2024-09-10T23:32:55.832 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.6 2024-09-10T23:32:55.832 DEBUG:teuthology.parallel:result is None 2024-09-10T23:32:56.648 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T23:32:56.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.801+0000 7fea80800640 1 -- 172.21.15.94:0/783052226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea780b4c70 msgr2=0x7fea60003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.801+0000 7fea80800640 1 --2- 172.21.15.94:0/783052226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea780b4c70 0x7fea60003c20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea6c001800 tx=0x7fea6c01f040 comp rx=0 tx=0).stop 2024-09-10T23:32:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.801+0000 7fea7e800640 1 -- 172.21.15.94:0/783052226 reap_dead start 2024-09-10T23:32:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.801+0000 7fea80800640 1 -- 172.21.15.94:0/783052226 shutdown_connections 2024-09-10T23:32:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.801+0000 7fea80800640 1 -- 172.21.15.94:0/783052226 >> 172.21.15.94:0/783052226 conn(0x7fea780d0d80 msgr2=0x7fea780d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.801+0000 7fea80800640 1 -- 172.21.15.94:0/783052226 shutdown_connections 2024-09-10T23:32:56.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.801+0000 7fea80800640 1 -- 172.21.15.94:0/783052226 wait complete. 2024-09-10T23:32:56.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.801+0000 7fea80800640 1 Processor -- start 2024-09-10T23:32:56.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.801+0000 7fea80800640 1 -- start start 2024-09-10T23:32:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.801+0000 7fea80800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea780b4c70 0x7fea781561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.801+0000 7fea80800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea781556a0 0x7fea78152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.801+0000 7fea80800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea781f5550 0x7fea781f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.801+0000 7fea80800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea780ceab0 con 0x7fea781f5550 2024-09-10T23:32:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea80800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fea780b3d70 con 0x7fea780b4c70 2024-09-10T23:32:56.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea80800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fea780cec30 con 0x7fea781556a0 2024-09-10T23:32:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea7e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea781f5550 0x7fea781f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea7e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea781f5550 0x7fea781f59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40314/0 (socket says 172.21.15.94:40314) 2024-09-10T23:32:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea7e800640 1 -- 172.21.15.94:0/2123079746 learned_addr learned my addr 172.21.15.94:0/2123079746 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea7d400640 1 --2- 172.21.15.94:0/2123079746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea781556a0 0x7fea78152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea7de00640 1 --2- 172.21.15.94:0/2123079746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea780b4c70 0x7fea781561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea7e800640 1 -- 172.21.15.94:0/2123079746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea781556a0 msgr2=0x7fea78152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea7e800640 1 --2- 172.21.15.94:0/2123079746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea781556a0 0x7fea78152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea7e800640 1 -- 172.21.15.94:0/2123079746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea780b4c70 msgr2=0x7fea781561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea7e800640 1 --2- 172.21.15.94:0/2123079746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea780b4c70 0x7fea781561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea7e800640 1 -- 172.21.15.94:0/2123079746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea781566e0 con 0x7fea781f5550 2024-09-10T23:32:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea7e800640 1 --2- 172.21.15.94:0/2123079746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea781f5550 0x7fea781f59d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea78074a80 tx=0x7fea74011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea72000640 1 -- 172.21.15.94:0/2123079746 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68006be0 con 0x7fea781f5550 2024-09-10T23:32:56.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea72000640 1 -- 172.21.15.94:0/2123079746 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea74004b30 con 0x7fea781f5550 2024-09-10T23:32:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea80800640 1 -- 172.21.15.94:0/2123079746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea78153790 con 0x7fea781f5550 2024-09-10T23:32:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea72000640 1 -- 172.21.15.94:0/2123079746 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74003980 con 0x7fea781f5550 2024-09-10T23:32:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea80800640 1 -- 172.21.15.94:0/2123079746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea78153d50 con 0x7fea781f5550 2024-09-10T23:32:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea80800640 1 -- 172.21.15.94:0/2123079746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea780b4560 con 0x7fea781f5550 2024-09-10T23:32:56.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.805+0000 7fea72000640 1 -- 172.21.15.94:0/2123079746 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fea74022070 con 0x7fea781f5550 2024-09-10T23:32:56.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.809+0000 7fea72000640 1 --2- 172.21.15.94:0/2123079746 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea5c0725e0 0x7fea5c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:56.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.809+0000 7fea72000640 1 -- 172.21.15.94:0/2123079746 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fea74093d90 con 0x7fea781f5550 2024-09-10T23:32:56.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.809+0000 7fea7de00640 1 --2- 172.21.15.94:0/2123079746 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea5c0725e0 0x7fea5c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:56.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.809+0000 7fea7de00640 1 --2- 172.21.15.94:0/2123079746 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea5c0725e0 0x7fea5c074aa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0017d0 tx=0x7fea6c01be30 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:56.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.813+0000 7fea72000640 1 -- 172.21.15.94:0/2123079746 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fea7401f060 con 0x7fea781f5550 2024-09-10T23:32:56.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.977+0000 7fea80800640 1 -- 172.21.15.94:0/2123079746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fea780b2470 con 0x7fea781f5550 2024-09-10T23:32:56.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.981+0000 7fea72000640 1 -- 172.21.15.94:0/2123079746 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fea7405f470 con 0x7fea781f5550 2024-09-10T23:32:56.982 INFO:teuthology.orchestra.run.smithi094.stdout:64424509447 2024-09-10T23:32:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.981+0000 7fea80800640 1 -- 172.21.15.94:0/2123079746 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea5c0725e0 msgr2=0x7fea5c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.981+0000 7fea80800640 1 --2- 172.21.15.94:0/2123079746 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea5c0725e0 0x7fea5c074aa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0017d0 tx=0x7fea6c01be30 comp rx=0 tx=0).stop 2024-09-10T23:32:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.981+0000 7fea80800640 1 -- 172.21.15.94:0/2123079746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea781f5550 msgr2=0x7fea781f59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:56.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.981+0000 7fea80800640 1 --2- 172.21.15.94:0/2123079746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea781f5550 0x7fea781f59d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea78074a80 tx=0x7fea74011f70 comp rx=0 tx=0).stop 2024-09-10T23:32:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.981+0000 7fea80800640 1 -- 172.21.15.94:0/2123079746 shutdown_connections 2024-09-10T23:32:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.981+0000 7fea80800640 1 --2- 172.21.15.94:0/2123079746 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea5c0725e0 0x7fea5c074aa0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.981+0000 7fea80800640 1 --2- 172.21.15.94:0/2123079746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea781f5550 0x7fea781f59d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.981+0000 7fea80800640 1 --2- 172.21.15.94:0/2123079746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea781556a0 0x7fea78152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.981+0000 7fea80800640 1 --2- 172.21.15.94:0/2123079746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea780b4c70 0x7fea781561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.981+0000 7fea80800640 1 -- 172.21.15.94:0/2123079746 >> 172.21.15.94:0/2123079746 conn(0x7fea780d0d80 msgr2=0x7fea780710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.981+0000 7fea80800640 1 -- 172.21.15.94:0/2123079746 shutdown_connections 2024-09-10T23:32:56.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:56.981+0000 7fea80800640 1 -- 172.21.15.94:0/2123079746 wait complete. 2024-09-10T23:32:57.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.0 2024-09-10T23:32:57.003 DEBUG:teuthology.parallel:result is None 2024-09-10T23:32:57.003 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T23:32:57.003 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:32:57.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.141+0000 7f2451a00640 1 -- 172.21.15.94:0/3820964365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f244c15ba30 msgr2=0x7f244c0b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:57.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.141+0000 7f2451a00640 1 --2- 172.21.15.94:0/3820964365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f244c15ba30 0x7f244c0b1bd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f243c009900 tx=0x7f243c01c8b0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.141+0000 7f2451a00640 1 -- 172.21.15.94:0/3820964365 shutdown_connections 2024-09-10T23:32:57.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.141+0000 7f2451a00640 1 --2- 172.21.15.94:0/3820964365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f244c0d04b0 0x7f244c0aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.141+0000 7f2451a00640 1 --2- 172.21.15.94:0/3820964365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f244c15ba30 0x7f244c0b1bd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.141+0000 7f2451a00640 1 -- 172.21.15.94:0/3820964365 >> 172.21.15.94:0/3820964365 conn(0x7f244c09aa70 msgr2=0x7f244c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:57.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.141+0000 7f2451a00640 1 -- 172.21.15.94:0/3820964365 shutdown_connections 2024-09-10T23:32:57.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.141+0000 7f2451a00640 1 -- 172.21.15.94:0/3820964365 wait complete. 2024-09-10T23:32:57.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.141+0000 7f2451a00640 1 Processor -- start 2024-09-10T23:32:57.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.141+0000 7f2451a00640 1 -- start start 2024-09-10T23:32:57.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f244c0b2110 0x7f244c14f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:57.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f244c0d04b0 0x7f244c14fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:57.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f244c15ba30 0x7f244c15e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:57.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f244c168830 con 0x7f244c15ba30 2024-09-10T23:32:57.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f244c1686b0 con 0x7f244c0b2110 2024-09-10T23:32:57.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f244c1689b0 con 0x7f244c0d04b0 2024-09-10T23:32:57.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f244c15ba30 0x7f244c15e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:57.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f244c15ba30 0x7f244c15e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40338/0 (socket says 172.21.15.94:40338) 2024-09-10T23:32:57.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451000640 1 -- 172.21.15.94:0/1194252788 learned_addr learned my addr 172.21.15.94:0/1194252788 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:57.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f244be00640 1 --2- 172.21.15.94:0/1194252788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f244c0b2110 0x7f244c14f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:57.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2443e00640 1 --2- 172.21.15.94:0/1194252788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f244c0d04b0 0x7f244c14fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:57.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451000640 1 -- 172.21.15.94:0/1194252788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f244c0d04b0 msgr2=0x7f244c14fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:57.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451000640 1 --2- 172.21.15.94:0/1194252788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f244c0d04b0 0x7f244c14fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451000640 1 -- 172.21.15.94:0/1194252788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f244c0b2110 msgr2=0x7f244c14f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:57.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451000640 1 --2- 172.21.15.94:0/1194252788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f244c0b2110 0x7f244c14f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451000640 1 -- 172.21.15.94:0/1194252788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244c158ae0 con 0x7f244c15ba30 2024-09-10T23:32:57.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451000640 1 --2- 172.21.15.94:0/1194252788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f244c15ba30 0x7f244c15e490 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f244400e9b0 tx=0x7f244400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:57.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2448c00640 1 -- 172.21.15.94:0/1194252788 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444004d30 con 0x7f244c15ba30 2024-09-10T23:32:57.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2448c00640 1 -- 172.21.15.94:0/1194252788 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f244400cb20 con 0x7f244c15ba30 2024-09-10T23:32:57.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2448c00640 1 -- 172.21.15.94:0/1194252788 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444005020 con 0x7f244c15ba30 2024-09-10T23:32:57.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451a00640 1 -- 172.21.15.94:0/1194252788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244c1faaa0 con 0x7f244c15ba30 2024-09-10T23:32:57.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451a00640 1 -- 172.21.15.94:0/1194252788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244c1fb580 con 0x7f244c15ba30 2024-09-10T23:32:57.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.145+0000 7f2451a00640 1 -- 172.21.15.94:0/1194252788 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f244c1fa890 con 0x7f244c15ba30 2024-09-10T23:32:57.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.149+0000 7f2448c00640 1 -- 172.21.15.94:0/1194252788 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f244400f040 con 0x7f244c15ba30 2024-09-10T23:32:57.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.149+0000 7f2448c00640 1 --2- 172.21.15.94:0/1194252788 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24280725e0 0x7f2428074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:57.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.149+0000 7f2448c00640 1 -- 172.21.15.94:0/1194252788 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2444093840 con 0x7f244c15ba30 2024-09-10T23:32:57.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.149+0000 7f244be00640 1 --2- 172.21.15.94:0/1194252788 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24280725e0 0x7f2428074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:57.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.149+0000 7f244be00640 1 --2- 172.21.15.94:0/1194252788 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24280725e0 0x7f2428074aa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f243c0af000 tx=0x7f243c01b0f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:57.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.153+0000 7f2448c00640 1 -- 172.21.15.94:0/1194252788 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2444016030 con 0x7f244c15ba30 2024-09-10T23:32:57.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.313+0000 7f2451a00640 1 -- 172.21.15.94:0/1194252788 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f244c1fb250 con 0x7f24280725e0 2024-09-10T23:32:57.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.317+0000 7f2448c00640 1 -- 172.21.15.94:0/1194252788 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7f244c1fb250 con 0x7f24280725e0 2024-09-10T23:32:57.323 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:57.325 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:32:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.321+0000 7f2451a00640 1 -- 172.21.15.94:0/1194252788 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24280725e0 msgr2=0x7f2428074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.321+0000 7f2451a00640 1 --2- 172.21.15.94:0/1194252788 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24280725e0 0x7f2428074aa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f243c0af000 tx=0x7f243c01b0f0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.321+0000 7f2451a00640 1 -- 172.21.15.94:0/1194252788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f244c15ba30 msgr2=0x7f244c15e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.321+0000 7f2451a00640 1 --2- 172.21.15.94:0/1194252788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f244c15ba30 0x7f244c15e490 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f244400e9b0 tx=0x7f244400ee70 comp rx=0 tx=0).stop 2024-09-10T23:32:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.321+0000 7f2451a00640 1 -- 172.21.15.94:0/1194252788 shutdown_connections 2024-09-10T23:32:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.321+0000 7f2451a00640 1 --2- 172.21.15.94:0/1194252788 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24280725e0 0x7f2428074aa0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.321+0000 7f2451a00640 1 --2- 172.21.15.94:0/1194252788 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f244c15ba30 0x7f244c15e490 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.321+0000 7f2451a00640 1 --2- 172.21.15.94:0/1194252788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f244c0d04b0 0x7f244c14fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.321+0000 7f2451a00640 1 --2- 172.21.15.94:0/1194252788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f244c0b2110 0x7f244c14f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.321+0000 7f2451a00640 1 -- 172.21.15.94:0/1194252788 >> 172.21.15.94:0/1194252788 conn(0x7f244c09aa70 msgr2=0x7f244c153060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:57.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.321+0000 7f2451a00640 1 -- 172.21.15.94:0/1194252788 shutdown_connections 2024-09-10T23:32:57.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.321+0000 7f2451a00640 1 -- 172.21.15.94:0/1194252788 wait complete. 2024-09-10T23:32:57.343 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-10T23:32:55.344809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235848,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738872,"statfs":{"total":773094113280,"available":772852604928,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.778177"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627741+0000","last_change":"2024-09-10T23:32:42.593279+0000","last_active":"2024-09-10T23:32:43.627741+0000","last_peered":"2024-09-10T23:32:43.627741+0000","last_clean":"2024-09-10T23:32:43.627741+0000","last_became_active":"2024-09-10T23:32:42.593056+0000","last_became_peered":"2024-09-10T23:32:42.593056+0000","last_unstale":"2024-09-10T23:32:43.627741+0000","last_undegraded":"2024-09-10T23:32:43.627741+0000","last_fullsized":"2024-09-10T23:32:43.627741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.112551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628801+0000","last_change":"2024-09-10T23:32:42.584101+0000","last_active":"2024-09-10T23:32:43.628801+0000","last_peered":"2024-09-10T23:32:43.628801+0000","last_clean":"2024-09-10T23:32:43.628801+0000","last_became_active":"2024-09-10T23:32:42.583796+0000","last_became_peered":"2024-09-10T23:32:42.583796+0000","last_unstale":"2024-09-10T23:32:43.628801+0000","last_undegraded":"2024-09-10T23:32:43.628801+0000","last_fullsized":"2024-09-10T23:32:43.628801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:01.463027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628392+0000","last_change":"2024-09-10T23:32:42.584302+0000","last_active":"2024-09-10T23:32:43.628392+0000","last_peered":"2024-09-10T23:32:43.628392+0000","last_clean":"2024-09-10T23:32:43.628392+0000","last_became_active":"2024-09-10T23:32:42.583989+0000","last_became_peered":"2024-09-10T23:32:42.583989+0000","last_unstale":"2024-09-10T23:32:43.628392+0000","last_undegraded":"2024-09-10T23:32:43.628392+0000","last_fullsized":"2024-09-10T23:32:43.628392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:53.622106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626948+0000","last_change":"2024-09-10T23:32:42.565401+0000","last_active":"2024-09-10T23:32:43.626948+0000","last_peered":"2024-09-10T23:32:43.626948+0000","last_clean":"2024-09-10T23:32:43.626948+0000","last_became_active":"2024-09-10T23:32:42.565106+0000","last_became_peered":"2024-09-10T23:32:42.565106+0000","last_unstale":"2024-09-10T23:32:43.626948+0000","last_undegraded":"2024-09-10T23:32:43.626948+0000","last_fullsized":"2024-09-10T23:32:43.626948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:20.388860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627066+0000","last_change":"2024-09-10T23:32:42.586253+0000","last_active":"2024-09-10T23:32:43.627066+0000","last_peered":"2024-09-10T23:32:43.627066+0000","last_clean":"2024-09-10T23:32:43.627066+0000","last_became_active":"2024-09-10T23:32:42.585961+0000","last_became_peered":"2024-09-10T23:32:42.585961+0000","last_unstale":"2024-09-10T23:32:43.627066+0000","last_undegraded":"2024-09-10T23:32:43.627066+0000","last_fullsized":"2024-09-10T23:32:43.627066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:19.891408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626891+0000","last_change":"2024-09-10T23:32:42.565746+0000","last_active":"2024-09-10T23:32:43.626891+0000","last_peered":"2024-09-10T23:32:43.626891+0000","last_clean":"2024-09-10T23:32:43.626891+0000","last_became_active":"2024-09-10T23:32:42.565398+0000","last_became_peered":"2024-09-10T23:32:42.565398+0000","last_unstale":"2024-09-10T23:32:43.626891+0000","last_undegraded":"2024-09-10T23:32:43.626891+0000","last_fullsized":"2024-09-10T23:32:43.626891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:28.431860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627187+0000","last_change":"2024-09-10T23:32:42.567960+0000","last_active":"2024-09-10T23:32:43.627187+0000","last_peered":"2024-09-10T23:32:43.627187+0000","last_clean":"2024-09-10T23:32:43.627187+0000","last_became_active":"2024-09-10T23:32:42.567653+0000","last_became_peered":"2024-09-10T23:32:42.567653+0000","last_unstale":"2024-09-10T23:32:43.627187+0000","last_undegraded":"2024-09-10T23:32:43.627187+0000","last_fullsized":"2024-09-10T23:32:43.627187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:26.192704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628554+0000","last_change":"2024-09-10T23:32:42.584639+0000","last_active":"2024-09-10T23:32:43.628554+0000","last_peered":"2024-09-10T23:32:43.628554+0000","last_clean":"2024-09-10T23:32:43.628554+0000","last_became_active":"2024-09-10T23:32:42.584396+0000","last_became_peered":"2024-09-10T23:32:42.584396+0000","last_unstale":"2024-09-10T23:32:43.628554+0000","last_undegraded":"2024-09-10T23:32:43.628554+0000","last_fullsized":"2024-09-10T23:32:43.628554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:27.237846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563757+0000","last_change":"2024-09-10T23:32:42.589096+0000","last_active":"2024-09-10T23:32:43.563757+0000","last_peered":"2024-09-10T23:32:43.563757+0000","last_clean":"2024-09-10T23:32:43.563757+0000","last_became_active":"2024-09-10T23:32:42.588695+0000","last_became_peered":"2024-09-10T23:32:42.588695+0000","last_unstale":"2024-09-10T23:32:43.563757+0000","last_undegraded":"2024-09-10T23:32:43.563757+0000","last_fullsized":"2024-09-10T23:32:43.563757+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:14.778902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627628+0000","last_change":"2024-09-10T23:32:42.572168+0000","last_active":"2024-09-10T23:32:43.627628+0000","last_peered":"2024-09-10T23:32:43.627628+0000","last_clean":"2024-09-10T23:32:43.627628+0000","last_became_active":"2024-09-10T23:32:42.571638+0000","last_became_peered":"2024-09-10T23:32:42.571638+0000","last_unstale":"2024-09-10T23:32:43.627628+0000","last_undegraded":"2024-09-10T23:32:43.627628+0000","last_fullsized":"2024-09-10T23:32:43.627628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:51.184952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627526+0000","last_change":"2024-09-10T23:32:42.580539+0000","last_active":"2024-09-10T23:32:43.627526+0000","last_peered":"2024-09-10T23:32:43.627526+0000","last_clean":"2024-09-10T23:32:43.627526+0000","last_became_active":"2024-09-10T23:32:42.580180+0000","last_became_peered":"2024-09-10T23:32:42.580180+0000","last_unstale":"2024-09-10T23:32:43.627526+0000","last_undegraded":"2024-09-10T23:32:43.627526+0000","last_fullsized":"2024-09-10T23:32:43.627526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:15.490925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627548+0000","last_change":"2024-09-10T23:32:42.568100+0000","last_active":"2024-09-10T23:32:43.627548+0000","last_peered":"2024-09-10T23:32:43.627548+0000","last_clean":"2024-09-10T23:32:43.627548+0000","last_became_active":"2024-09-10T23:32:42.567409+0000","last_became_peered":"2024-09-10T23:32:42.567409+0000","last_unstale":"2024-09-10T23:32:43.627548+0000","last_undegraded":"2024-09-10T23:32:43.627548+0000","last_fullsized":"2024-09-10T23:32:43.627548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.117710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569985+0000","last_change":"2024-09-10T23:32:42.569811+0000","last_active":"2024-09-10T23:32:43.569985+0000","last_peered":"2024-09-10T23:32:43.569985+0000","last_clean":"2024-09-10T23:32:43.569985+0000","last_became_active":"2024-09-10T23:32:42.569470+0000","last_became_peered":"2024-09-10T23:32:42.569470+0000","last_unstale":"2024-09-10T23:32:43.569985+0000","last_undegraded":"2024-09-10T23:32:43.569985+0000","last_fullsized":"2024-09-10T23:32:43.569985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:56.266045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570094+0000","last_change":"2024-09-10T23:32:42.578563+0000","last_active":"2024-09-10T23:32:43.570094+0000","last_peered":"2024-09-10T23:32:43.570094+0000","last_clean":"2024-09-10T23:32:43.570094+0000","last_became_active":"2024-09-10T23:32:42.578281+0000","last_became_peered":"2024-09-10T23:32:42.578281+0000","last_unstale":"2024-09-10T23:32:43.570094+0000","last_undegraded":"2024-09-10T23:32:43.570094+0000","last_fullsized":"2024-09-10T23:32:43.570094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:57.340004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628709+0000","last_change":"2024-09-10T23:32:42.564792+0000","last_active":"2024-09-10T23:32:43.628709+0000","last_peered":"2024-09-10T23:32:43.628709+0000","last_clean":"2024-09-10T23:32:43.628709+0000","last_became_active":"2024-09-10T23:32:42.564029+0000","last_became_peered":"2024-09-10T23:32:42.564029+0000","last_unstale":"2024-09-10T23:32:43.628709+0000","last_undegraded":"2024-09-10T23:32:43.628709+0000","last_fullsized":"2024-09-10T23:32:43.628709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:23.318586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628280+0000","last_change":"2024-09-10T23:32:42.583724+0000","last_active":"2024-09-10T23:32:43.628280+0000","last_peered":"2024-09-10T23:32:43.628280+0000","last_clean":"2024-09-10T23:32:43.628280+0000","last_became_active":"2024-09-10T23:32:42.583473+0000","last_became_peered":"2024-09-10T23:32:42.583473+0000","last_unstale":"2024-09-10T23:32:43.628280+0000","last_undegraded":"2024-09-10T23:32:43.628280+0000","last_fullsized":"2024-09-10T23:32:43.628280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:14.468774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629190+0000","last_change":"2024-09-10T23:32:42.588295+0000","last_active":"2024-09-10T23:32:43.629190+0000","last_peered":"2024-09-10T23:32:43.629190+0000","last_clean":"2024-09-10T23:32:43.629190+0000","last_became_active":"2024-09-10T23:32:42.587993+0000","last_became_peered":"2024-09-10T23:32:42.587993+0000","last_unstale":"2024-09-10T23:32:43.629190+0000","last_undegraded":"2024-09-10T23:32:43.629190+0000","last_fullsized":"2024-09-10T23:32:43.629190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:18:27.504742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629147+0000","last_change":"2024-09-10T23:32:42.579982+0000","last_active":"2024-09-10T23:32:43.629147+0000","last_peered":"2024-09-10T23:32:43.629147+0000","last_clean":"2024-09-10T23:32:43.629147+0000","last_became_active":"2024-09-10T23:32:42.575814+0000","last_became_peered":"2024-09-10T23:32:42.575814+0000","last_unstale":"2024-09-10T23:32:43.629147+0000","last_undegraded":"2024-09-10T23:32:43.629147+0000","last_fullsized":"2024-09-10T23:32:43.629147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:58.672853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629002+0000","last_change":"2024-09-10T23:32:42.585466+0000","last_active":"2024-09-10T23:32:43.629002+0000","last_peered":"2024-09-10T23:32:43.629002+0000","last_clean":"2024-09-10T23:32:43.629002+0000","last_became_active":"2024-09-10T23:32:42.582650+0000","last_became_peered":"2024-09-10T23:32:42.582650+0000","last_unstale":"2024-09-10T23:32:43.629002+0000","last_undegraded":"2024-09-10T23:32:43.629002+0000","last_fullsized":"2024-09-10T23:32:43.629002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:59.434883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627679+0000","last_change":"2024-09-10T23:32:42.573723+0000","last_active":"2024-09-10T23:32:43.627679+0000","last_peered":"2024-09-10T23:32:43.627679+0000","last_clean":"2024-09-10T23:32:43.627679+0000","last_became_active":"2024-09-10T23:32:42.573309+0000","last_became_peered":"2024-09-10T23:32:42.573309+0000","last_unstale":"2024-09-10T23:32:43.627679+0000","last_undegraded":"2024-09-10T23:32:43.627679+0000","last_fullsized":"2024-09-10T23:32:43.627679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:08.762419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569445+0000","last_change":"2024-09-10T23:32:42.558045+0000","last_active":"2024-09-10T23:32:43.569445+0000","last_peered":"2024-09-10T23:32:43.569445+0000","last_clean":"2024-09-10T23:32:43.569445+0000","last_became_active":"2024-09-10T23:32:42.557774+0000","last_became_peered":"2024-09-10T23:32:42.557774+0000","last_unstale":"2024-09-10T23:32:43.569445+0000","last_undegraded":"2024-09-10T23:32:43.569445+0000","last_fullsized":"2024-09-10T23:32:43.569445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:53.622106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563214+0000","last_change":"2024-09-10T23:32:42.556890+0000","last_active":"2024-09-10T23:32:43.563214+0000","last_peered":"2024-09-10T23:32:43.563214+0000","last_clean":"2024-09-10T23:32:43.563214+0000","last_became_active":"2024-09-10T23:32:42.556625+0000","last_became_peered":"2024-09-10T23:32:42.556625+0000","last_unstale":"2024-09-10T23:32:43.563214+0000","last_undegraded":"2024-09-10T23:32:43.563214+0000","last_fullsized":"2024-09-10T23:32:43.563214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:14.440912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628804+0000","last_change":"2024-09-10T23:32:42.575698+0000","last_active":"2024-09-10T23:32:43.628804+0000","last_peered":"2024-09-10T23:32:43.628804+0000","last_clean":"2024-09-10T23:32:43.628804+0000","last_became_active":"2024-09-10T23:32:42.575371+0000","last_became_peered":"2024-09-10T23:32:42.575371+0000","last_unstale":"2024-09-10T23:32:43.628804+0000","last_undegraded":"2024-09-10T23:32:43.628804+0000","last_fullsized":"2024-09-10T23:32:43.628804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:36.598389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628388+0000","last_change":"2024-09-10T23:32:42.591870+0000","last_active":"2024-09-10T23:32:43.628388+0000","last_peered":"2024-09-10T23:32:43.628388+0000","last_clean":"2024-09-10T23:32:43.628388+0000","last_became_active":"2024-09-10T23:32:42.590207+0000","last_became_peered":"2024-09-10T23:32:42.590207+0000","last_unstale":"2024-09-10T23:32:43.628388+0000","last_undegraded":"2024-09-10T23:32:43.628388+0000","last_fullsized":"2024-09-10T23:32:43.628388+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:40.104986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629051+0000","last_change":"2024-09-10T23:32:42.588702+0000","last_active":"2024-09-10T23:32:43.629051+0000","last_peered":"2024-09-10T23:32:43.629051+0000","last_clean":"2024-09-10T23:32:43.629051+0000","last_became_active":"2024-09-10T23:32:42.588444+0000","last_became_peered":"2024-09-10T23:32:42.588444+0000","last_unstale":"2024-09-10T23:32:43.629051+0000","last_undegraded":"2024-09-10T23:32:43.629051+0000","last_fullsized":"2024-09-10T23:32:43.629051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:43.173720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627479+0000","last_change":"2024-09-10T23:32:42.580821+0000","last_active":"2024-09-10T23:32:43.627479+0000","last_peered":"2024-09-10T23:32:43.627479+0000","last_clean":"2024-09-10T23:32:43.627479+0000","last_became_active":"2024-09-10T23:32:42.580319+0000","last_became_peered":"2024-09-10T23:32:42.580319+0000","last_unstale":"2024-09-10T23:32:43.627479+0000","last_undegraded":"2024-09-10T23:32:43.627479+0000","last_fullsized":"2024-09-10T23:32:43.627479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:26.735696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570454+0000","last_change":"2024-09-10T23:32:42.574625+0000","last_active":"2024-09-10T23:32:43.570454+0000","last_peered":"2024-09-10T23:32:43.570454+0000","last_clean":"2024-09-10T23:32:43.570454+0000","last_became_active":"2024-09-10T23:32:42.571847+0000","last_became_peered":"2024-09-10T23:32:42.571847+0000","last_unstale":"2024-09-10T23:32:43.570454+0000","last_undegraded":"2024-09-10T23:32:43.570454+0000","last_fullsized":"2024-09-10T23:32:43.570454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:20.906137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570331+0000","last_change":"2024-09-10T23:32:42.567479+0000","last_active":"2024-09-10T23:32:43.570331+0000","last_peered":"2024-09-10T23:32:43.570331+0000","last_clean":"2024-09-10T23:32:43.570331+0000","last_became_active":"2024-09-10T23:32:42.567172+0000","last_became_peered":"2024-09-10T23:32:42.567172+0000","last_unstale":"2024-09-10T23:32:43.570331+0000","last_undegraded":"2024-09-10T23:32:43.570331+0000","last_fullsized":"2024-09-10T23:32:43.570331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:15.280711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628395+0000","last_change":"2024-09-10T23:32:42.585087+0000","last_active":"2024-09-10T23:32:43.628395+0000","last_peered":"2024-09-10T23:32:43.628395+0000","last_clean":"2024-09-10T23:32:43.628395+0000","last_became_active":"2024-09-10T23:32:42.583712+0000","last_became_peered":"2024-09-10T23:32:42.583712+0000","last_unstale":"2024-09-10T23:32:43.628395+0000","last_undegraded":"2024-09-10T23:32:43.628395+0000","last_fullsized":"2024-09-10T23:32:43.628395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:25.952890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628168+0000","last_change":"2024-09-10T23:32:42.591882+0000","last_active":"2024-09-10T23:32:43.628168+0000","last_peered":"2024-09-10T23:32:43.628168+0000","last_clean":"2024-09-10T23:32:43.628168+0000","last_became_active":"2024-09-10T23:32:42.590231+0000","last_became_peered":"2024-09-10T23:32:42.590231+0000","last_unstale":"2024-09-10T23:32:43.628168+0000","last_undegraded":"2024-09-10T23:32:43.628168+0000","last_fullsized":"2024-09-10T23:32:43.628168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:33.100231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628038+0000","last_change":"2024-09-10T23:32:42.573851+0000","last_active":"2024-09-10T23:32:43.628038+0000","last_peered":"2024-09-10T23:32:43.628038+0000","last_clean":"2024-09-10T23:32:43.628038+0000","last_became_active":"2024-09-10T23:32:42.572836+0000","last_became_peered":"2024-09-10T23:32:42.572836+0000","last_unstale":"2024-09-10T23:32:43.628038+0000","last_undegraded":"2024-09-10T23:32:43.628038+0000","last_fullsized":"2024-09-10T23:32:43.628038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:54.816587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627085+0000","last_change":"2024-09-10T23:32:42.576085+0000","last_active":"2024-09-10T23:32:43.627085+0000","last_peered":"2024-09-10T23:32:43.627085+0000","last_clean":"2024-09-10T23:32:43.627085+0000","last_became_active":"2024-09-10T23:32:42.575374+0000","last_became_peered":"2024-09-10T23:32:42.575374+0000","last_unstale":"2024-09-10T23:32:43.627085+0000","last_undegraded":"2024-09-10T23:32:43.627085+0000","last_fullsized":"2024-09-10T23:32:43.627085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:14.778902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563267+0000","last_change":"2024-09-10T23:32:42.588842+0000","last_active":"2024-09-10T23:32:43.563267+0000","last_peered":"2024-09-10T23:32:43.563267+0000","last_clean":"2024-09-10T23:32:43.563267+0000","last_became_active":"2024-09-10T23:32:42.588582+0000","last_became_peered":"2024-09-10T23:32:42.588582+0000","last_unstale":"2024-09-10T23:32:43.563267+0000","last_undegraded":"2024-09-10T23:32:43.563267+0000","last_fullsized":"2024-09-10T23:32:43.563267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:28.431860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628414+0000","last_change":"2024-09-10T23:32:41.569293+0000","last_active":"2024-09-10T23:32:43.628414+0000","last_peered":"2024-09-10T23:32:43.628414+0000","last_clean":"2024-09-10T23:32:43.628414+0000","last_became_active":"2024-09-10T23:32:41.568848+0000","last_became_peered":"2024-09-10T23:32:41.568848+0000","last_unstale":"2024-09-10T23:32:43.628414+0000","last_undegraded":"2024-09-10T23:32:43.628414+0000","last_fullsized":"2024-09-10T23:32:43.628414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:17.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569497+0000","last_change":"2024-09-10T23:32:42.568214+0000","last_active":"2024-09-10T23:32:43.569497+0000","last_peered":"2024-09-10T23:32:43.569497+0000","last_clean":"2024-09-10T23:32:43.569497+0000","last_became_active":"2024-09-10T23:32:42.562093+0000","last_became_peered":"2024-09-10T23:32:42.562093+0000","last_unstale":"2024-09-10T23:32:43.569497+0000","last_undegraded":"2024-09-10T23:32:43.569497+0000","last_fullsized":"2024-09-10T23:32:43.569497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:02.454545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628550+0000","last_change":"2024-09-10T23:32:41.578635+0000","last_active":"2024-09-10T23:32:43.628550+0000","last_peered":"2024-09-10T23:32:43.628550+0000","last_clean":"2024-09-10T23:32:43.628550+0000","last_became_active":"2024-09-10T23:32:41.578346+0000","last_became_peered":"2024-09-10T23:32:41.578346+0000","last_unstale":"2024-09-10T23:32:43.628550+0000","last_undegraded":"2024-09-10T23:32:43.628550+0000","last_fullsized":"2024-09-10T23:32:43.628550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:25.728667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569793+0000","last_change":"2024-09-10T23:32:41.553674+0000","last_active":"2024-09-10T23:32:43.569793+0000","last_peered":"2024-09-10T23:32:43.569793+0000","last_clean":"2024-09-10T23:32:43.569793+0000","last_became_active":"2024-09-10T23:32:41.553244+0000","last_became_peered":"2024-09-10T23:32:41.553244+0000","last_unstale":"2024-09-10T23:32:43.569793+0000","last_undegraded":"2024-09-10T23:32:43.569793+0000","last_fullsized":"2024-09-10T23:32:43.569793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:38.098241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628828+0000","last_change":"2024-09-10T23:32:42.583587+0000","last_active":"2024-09-10T23:32:43.628828+0000","last_peered":"2024-09-10T23:32:43.628828+0000","last_clean":"2024-09-10T23:32:43.628828+0000","last_became_active":"2024-09-10T23:32:42.581831+0000","last_became_peered":"2024-09-10T23:32:42.581831+0000","last_unstale":"2024-09-10T23:32:43.628828+0000","last_undegraded":"2024-09-10T23:32:43.628828+0000","last_fullsized":"2024-09-10T23:32:43.628828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:23.493444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628532+0000","last_change":"2024-09-10T23:32:41.581810+0000","last_active":"2024-09-10T23:32:43.628532+0000","last_peered":"2024-09-10T23:32:43.628532+0000","last_clean":"2024-09-10T23:32:43.628532+0000","last_became_active":"2024-09-10T23:32:41.581531+0000","last_became_peered":"2024-09-10T23:32:41.581531+0000","last_unstale":"2024-09-10T23:32:43.628532+0000","last_undegraded":"2024-09-10T23:32:43.628532+0000","last_fullsized":"2024-09-10T23:32:43.628532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:57.665825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569188+0000","last_change":"2024-09-10T23:32:42.560663+0000","last_active":"2024-09-10T23:32:43.569188+0000","last_peered":"2024-09-10T23:32:43.569188+0000","last_clean":"2024-09-10T23:32:43.569188+0000","last_became_active":"2024-09-10T23:32:42.560293+0000","last_became_peered":"2024-09-10T23:32:42.560293+0000","last_unstale":"2024-09-10T23:32:43.569188+0000","last_undegraded":"2024-09-10T23:32:43.569188+0000","last_fullsized":"2024-09-10T23:32:43.569188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:39.387246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563757+0000","last_change":"2024-09-10T23:32:41.573958+0000","last_active":"2024-09-10T23:32:43.563757+0000","last_peered":"2024-09-10T23:32:43.563757+0000","last_clean":"2024-09-10T23:32:43.563757+0000","last_became_active":"2024-09-10T23:32:41.573639+0000","last_became_peered":"2024-09-10T23:32:41.573639+0000","last_unstale":"2024-09-10T23:32:43.563757+0000","last_undegraded":"2024-09-10T23:32:43.563757+0000","last_fullsized":"2024-09-10T23:32:43.563757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:27.548086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627506+0000","last_change":"2024-09-10T23:32:42.580586+0000","last_active":"2024-09-10T23:32:43.627506+0000","last_peered":"2024-09-10T23:32:43.627506+0000","last_clean":"2024-09-10T23:32:43.627506+0000","last_became_active":"2024-09-10T23:32:42.579960+0000","last_became_peered":"2024-09-10T23:32:42.579960+0000","last_unstale":"2024-09-10T23:32:43.627506+0000","last_undegraded":"2024-09-10T23:32:43.627506+0000","last_fullsized":"2024-09-10T23:32:43.627506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:09.640521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627910+0000","last_change":"2024-09-10T23:32:41.570662+0000","last_active":"2024-09-10T23:32:43.627910+0000","last_peered":"2024-09-10T23:32:43.627910+0000","last_clean":"2024-09-10T23:32:43.627910+0000","last_became_active":"2024-09-10T23:32:41.570330+0000","last_became_peered":"2024-09-10T23:32:41.570330+0000","last_unstale":"2024-09-10T23:32:43.627910+0000","last_undegraded":"2024-09-10T23:32:43.627910+0000","last_fullsized":"2024-09-10T23:32:43.627910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:53.809559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629328+0000","last_change":"2024-09-10T23:32:42.591662+0000","last_active":"2024-09-10T23:32:43.629328+0000","last_peered":"2024-09-10T23:32:43.629328+0000","last_clean":"2024-09-10T23:32:43.629328+0000","last_became_active":"2024-09-10T23:32:42.591315+0000","last_became_peered":"2024-09-10T23:32:42.591315+0000","last_unstale":"2024-09-10T23:32:43.629328+0000","last_undegraded":"2024-09-10T23:32:43.629328+0000","last_fullsized":"2024-09-10T23:32:43.629328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:34.259468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628947+0000","last_change":"2024-09-10T23:32:41.571940+0000","last_active":"2024-09-10T23:32:43.628947+0000","last_peered":"2024-09-10T23:32:43.628947+0000","last_clean":"2024-09-10T23:32:43.628947+0000","last_became_active":"2024-09-10T23:32:41.571537+0000","last_became_peered":"2024-09-10T23:32:41.571537+0000","last_unstale":"2024-09-10T23:32:43.628947+0000","last_undegraded":"2024-09-10T23:32:43.628947+0000","last_fullsized":"2024-09-10T23:32:43.628947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:49.898494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629065+0000","last_change":"2024-09-10T23:32:42.585332+0000","last_active":"2024-09-10T23:32:43.629065+0000","last_peered":"2024-09-10T23:32:43.629065+0000","last_clean":"2024-09-10T23:32:43.629065+0000","last_became_active":"2024-09-10T23:32:42.585046+0000","last_became_peered":"2024-09-10T23:32:42.585046+0000","last_unstale":"2024-09-10T23:32:43.629065+0000","last_undegraded":"2024-09-10T23:32:43.629065+0000","last_fullsized":"2024-09-10T23:32:43.629065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:24.681728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627051+0000","last_change":"2024-09-10T23:32:41.565270+0000","last_active":"2024-09-10T23:32:43.627051+0000","last_peered":"2024-09-10T23:32:43.627051+0000","last_clean":"2024-09-10T23:32:43.627051+0000","last_became_active":"2024-09-10T23:32:41.564864+0000","last_became_peered":"2024-09-10T23:32:41.564864+0000","last_unstale":"2024-09-10T23:32:43.627051+0000","last_undegraded":"2024-09-10T23:32:43.627051+0000","last_fullsized":"2024-09-10T23:32:43.627051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:55.242033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627514+0000","last_change":"2024-09-10T23:32:42.576998+0000","last_active":"2024-09-10T23:32:43.627514+0000","last_peered":"2024-09-10T23:32:43.627514+0000","last_clean":"2024-09-10T23:32:43.627514+0000","last_became_active":"2024-09-10T23:32:42.576695+0000","last_became_peered":"2024-09-10T23:32:42.576695+0000","last_unstale":"2024-09-10T23:32:43.627514+0000","last_undegraded":"2024-09-10T23:32:43.627514+0000","last_fullsized":"2024-09-10T23:32:43.627514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:09.931603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628218+0000","last_change":"2024-09-10T23:32:41.571048+0000","last_active":"2024-09-10T23:32:43.628218+0000","last_peered":"2024-09-10T23:32:43.628218+0000","last_clean":"2024-09-10T23:32:43.628218+0000","last_became_active":"2024-09-10T23:32:41.570753+0000","last_became_peered":"2024-09-10T23:32:41.570753+0000","last_unstale":"2024-09-10T23:32:43.628218+0000","last_undegraded":"2024-09-10T23:32:43.628218+0000","last_fullsized":"2024-09-10T23:32:43.628218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:55.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570370+0000","last_change":"2024-09-10T23:32:42.564300+0000","last_active":"2024-09-10T23:32:43.570370+0000","last_peered":"2024-09-10T23:32:43.570370+0000","last_clean":"2024-09-10T23:32:43.570370+0000","last_became_active":"2024-09-10T23:32:42.564047+0000","last_became_peered":"2024-09-10T23:32:42.564047+0000","last_unstale":"2024-09-10T23:32:43.570370+0000","last_undegraded":"2024-09-10T23:32:43.570370+0000","last_fullsized":"2024-09-10T23:32:43.570370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:22.724821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628645+0000","last_change":"2024-09-10T23:32:41.585012+0000","last_active":"2024-09-10T23:32:43.628645+0000","last_peered":"2024-09-10T23:32:43.628645+0000","last_clean":"2024-09-10T23:32:43.628645+0000","last_became_active":"2024-09-10T23:32:41.584732+0000","last_became_peered":"2024-09-10T23:32:41.584732+0000","last_unstale":"2024-09-10T23:32:43.628645+0000","last_undegraded":"2024-09-10T23:32:43.628645+0000","last_fullsized":"2024-09-10T23:32:43.628645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:39.041511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628794+0000","last_change":"2024-09-10T23:32:42.569000+0000","last_active":"2024-09-10T23:32:43.628794+0000","last_peered":"2024-09-10T23:32:43.628794+0000","last_clean":"2024-09-10T23:32:43.628794+0000","last_became_active":"2024-09-10T23:32:42.568721+0000","last_became_peered":"2024-09-10T23:32:42.568721+0000","last_unstale":"2024-09-10T23:32:43.628794+0000","last_undegraded":"2024-09-10T23:32:43.628794+0000","last_fullsized":"2024-09-10T23:32:43.628794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:37.891368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626833+0000","last_change":"2024-09-10T23:32:41.578661+0000","last_active":"2024-09-10T23:32:43.626833+0000","last_peered":"2024-09-10T23:32:43.626833+0000","last_clean":"2024-09-10T23:32:43.626833+0000","last_became_active":"2024-09-10T23:32:41.578343+0000","last_became_peered":"2024-09-10T23:32:41.578343+0000","last_unstale":"2024-09-10T23:32:43.626833+0000","last_undegraded":"2024-09-10T23:32:43.626833+0000","last_fullsized":"2024-09-10T23:32:43.626833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:47.815432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563134+0000","last_change":"2024-09-10T23:32:42.584934+0000","last_active":"2024-09-10T23:32:43.563134+0000","last_peered":"2024-09-10T23:32:43.563134+0000","last_clean":"2024-09-10T23:32:43.563134+0000","last_became_active":"2024-09-10T23:32:42.584709+0000","last_became_peered":"2024-09-10T23:32:42.584709+0000","last_unstale":"2024-09-10T23:32:43.563134+0000","last_undegraded":"2024-09-10T23:32:43.563134+0000","last_fullsized":"2024-09-10T23:32:43.563134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:30.543243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.568880+0000","last_change":"2024-09-10T23:32:41.566144+0000","last_active":"2024-09-10T23:32:43.568880+0000","last_peered":"2024-09-10T23:32:43.568880+0000","last_clean":"2024-09-10T23:32:43.568880+0000","last_became_active":"2024-09-10T23:32:41.565841+0000","last_became_peered":"2024-09-10T23:32:41.565841+0000","last_unstale":"2024-09-10T23:32:43.568880+0000","last_undegraded":"2024-09-10T23:32:43.568880+0000","last_fullsized":"2024-09-10T23:32:43.568880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:51.080022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628125+0000","last_change":"2024-09-10T23:32:42.586470+0000","last_active":"2024-09-10T23:32:43.628125+0000","last_peered":"2024-09-10T23:32:43.628125+0000","last_clean":"2024-09-10T23:32:43.628125+0000","last_became_active":"2024-09-10T23:32:42.579377+0000","last_became_peered":"2024-09-10T23:32:42.579377+0000","last_unstale":"2024-09-10T23:32:43.628125+0000","last_undegraded":"2024-09-10T23:32:43.628125+0000","last_fullsized":"2024-09-10T23:32:43.628125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.112551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628061+0000","last_change":"2024-09-10T23:32:41.567212+0000","last_active":"2024-09-10T23:32:43.628061+0000","last_peered":"2024-09-10T23:32:43.628061+0000","last_clean":"2024-09-10T23:32:43.628061+0000","last_became_active":"2024-09-10T23:32:41.566857+0000","last_became_peered":"2024-09-10T23:32:41.566857+0000","last_unstale":"2024-09-10T23:32:43.628061+0000","last_undegraded":"2024-09-10T23:32:43.628061+0000","last_fullsized":"2024-09-10T23:32:43.628061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:05.259433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627653+0000","last_change":"2024-09-10T23:32:42.558295+0000","last_active":"2024-09-10T23:32:43.627653+0000","last_peered":"2024-09-10T23:32:43.627653+0000","last_clean":"2024-09-10T23:32:43.627653+0000","last_became_active":"2024-09-10T23:32:42.558039+0000","last_became_peered":"2024-09-10T23:32:42.558039+0000","last_unstale":"2024-09-10T23:32:43.627653+0000","last_undegraded":"2024-09-10T23:32:43.627653+0000","last_fullsized":"2024-09-10T23:32:43.627653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:07.033834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569932+0000","last_change":"2024-09-10T23:32:41.568016+0000","last_active":"2024-09-10T23:32:43.569932+0000","last_peered":"2024-09-10T23:32:43.569932+0000","last_clean":"2024-09-10T23:32:43.569932+0000","last_became_active":"2024-09-10T23:32:41.566960+0000","last_became_peered":"2024-09-10T23:32:41.566960+0000","last_unstale":"2024-09-10T23:32:43.569932+0000","last_undegraded":"2024-09-10T23:32:43.569932+0000","last_fullsized":"2024-09-10T23:32:43.569932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:14.145496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628571+0000","last_change":"2024-09-10T23:32:42.562392+0000","last_active":"2024-09-10T23:32:43.628571+0000","last_peered":"2024-09-10T23:32:43.628571+0000","last_clean":"2024-09-10T23:32:43.628571+0000","last_became_active":"2024-09-10T23:32:42.562042+0000","last_became_peered":"2024-09-10T23:32:42.562042+0000","last_unstale":"2024-09-10T23:32:43.628571+0000","last_undegraded":"2024-09-10T23:32:43.628571+0000","last_fullsized":"2024-09-10T23:32:43.628571+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:33.100231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628521+0000","last_change":"2024-09-10T23:32:41.572986+0000","last_active":"2024-09-10T23:32:43.628521+0000","last_peered":"2024-09-10T23:32:43.628521+0000","last_clean":"2024-09-10T23:32:43.628521+0000","last_became_active":"2024-09-10T23:32:41.572722+0000","last_became_peered":"2024-09-10T23:32:41.572722+0000","last_unstale":"2024-09-10T23:32:43.628521+0000","last_undegraded":"2024-09-10T23:32:43.628521+0000","last_fullsized":"2024-09-10T23:32:43.628521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:49.898494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628742+0000","last_change":"2024-09-10T23:32:42.573643+0000","last_active":"2024-09-10T23:32:43.628742+0000","last_peered":"2024-09-10T23:32:43.628742+0000","last_clean":"2024-09-10T23:32:43.628742+0000","last_became_active":"2024-09-10T23:32:42.572873+0000","last_became_peered":"2024-09-10T23:32:42.572873+0000","last_unstale":"2024-09-10T23:32:43.628742+0000","last_undegraded":"2024-09-10T23:32:43.628742+0000","last_fullsized":"2024-09-10T23:32:43.628742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:14.468774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570459+0000","last_change":"2024-09-10T23:32:41.562365+0000","last_active":"2024-09-10T23:32:43.570459+0000","last_peered":"2024-09-10T23:32:43.570459+0000","last_clean":"2024-09-10T23:32:43.570459+0000","last_became_active":"2024-09-10T23:32:41.562021+0000","last_became_peered":"2024-09-10T23:32:41.562021+0000","last_unstale":"2024-09-10T23:32:43.570459+0000","last_undegraded":"2024-09-10T23:32:43.570459+0000","last_fullsized":"2024-09-10T23:32:43.570459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:08:28.878205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628732+0000","last_change":"2024-09-10T23:32:42.584713+0000","last_active":"2024-09-10T23:32:43.628732+0000","last_peered":"2024-09-10T23:32:43.628732+0000","last_clean":"2024-09-10T23:32:43.628732+0000","last_became_active":"2024-09-10T23:32:42.577789+0000","last_became_peered":"2024-09-10T23:32:42.577789+0000","last_unstale":"2024-09-10T23:32:43.628732+0000","last_undegraded":"2024-09-10T23:32:43.628732+0000","last_fullsized":"2024-09-10T23:32:43.628732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:33.835681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563126+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:32:43.563126+0000","last_peered":"2024-09-10T23:32:43.563126+0000","last_clean":"2024-09-10T23:32:43.563126+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:32:43.563126+0000","last_undegraded":"2024-09-10T23:32:43.563126+0000","last_fullsized":"2024-09-10T23:32:43.563126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629078+0000","last_change":"2024-09-10T23:32:41.581530+0000","last_active":"2024-09-10T23:32:43.629078+0000","last_peered":"2024-09-10T23:32:43.629078+0000","last_clean":"2024-09-10T23:32:43.629078+0000","last_became_active":"2024-09-10T23:32:41.581242+0000","last_became_peered":"2024-09-10T23:32:41.581242+0000","last_unstale":"2024-09-10T23:32:43.629078+0000","last_undegraded":"2024-09-10T23:32:43.629078+0000","last_fullsized":"2024-09-10T23:32:43.629078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:08.633493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569220+0000","last_change":"2024-09-10T23:32:42.558841+0000","last_active":"2024-09-10T23:32:43.569220+0000","last_peered":"2024-09-10T23:32:43.569220+0000","last_clean":"2024-09-10T23:32:43.569220+0000","last_became_active":"2024-09-10T23:32:42.558580+0000","last_became_peered":"2024-09-10T23:32:42.558580+0000","last_unstale":"2024-09-10T23:32:43.569220+0000","last_undegraded":"2024-09-10T23:32:43.569220+0000","last_fullsized":"2024-09-10T23:32:43.569220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:54.620648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628625+0000","last_change":"2024-09-10T23:32:41.577467+0000","last_active":"2024-09-10T23:32:43.628625+0000","last_peered":"2024-09-10T23:32:43.628625+0000","last_clean":"2024-09-10T23:32:43.628625+0000","last_became_active":"2024-09-10T23:32:41.576702+0000","last_became_peered":"2024-09-10T23:32:41.576702+0000","last_unstale":"2024-09-10T23:32:43.628625+0000","last_undegraded":"2024-09-10T23:32:43.628625+0000","last_fullsized":"2024-09-10T23:32:43.628625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:32.527218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569670+0000","last_change":"2024-09-10T23:32:42.565100+0000","last_active":"2024-09-10T23:32:43.569670+0000","last_peered":"2024-09-10T23:32:43.569670+0000","last_clean":"2024-09-10T23:32:43.569670+0000","last_became_active":"2024-09-10T23:32:42.564790+0000","last_became_peered":"2024-09-10T23:32:42.564790+0000","last_unstale":"2024-09-10T23:32:43.569670+0000","last_undegraded":"2024-09-10T23:32:43.569670+0000","last_fullsized":"2024-09-10T23:32:43.569670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:44.555042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628065+0000","last_change":"2024-09-10T23:32:41.571266+0000","last_active":"2024-09-10T23:32:43.628065+0000","last_peered":"2024-09-10T23:32:43.628065+0000","last_clean":"2024-09-10T23:32:43.628065+0000","last_became_active":"2024-09-10T23:32:41.570986+0000","last_became_peered":"2024-09-10T23:32:41.570986+0000","last_unstale":"2024-09-10T23:32:43.628065+0000","last_undegraded":"2024-09-10T23:32:43.628065+0000","last_fullsized":"2024-09-10T23:32:43.628065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:58.170892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628975+0000","last_change":"2024-09-10T23:32:42.584295+0000","last_active":"2024-09-10T23:32:43.628975+0000","last_peered":"2024-09-10T23:32:43.628975+0000","last_clean":"2024-09-10T23:32:43.628975+0000","last_became_active":"2024-09-10T23:32:42.584001+0000","last_became_peered":"2024-09-10T23:32:42.584001+0000","last_unstale":"2024-09-10T23:32:43.628975+0000","last_undegraded":"2024-09-10T23:32:43.628975+0000","last_fullsized":"2024-09-10T23:32:43.628975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:05.044760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628726+0000","last_change":"2024-09-10T23:32:41.578157+0000","last_active":"2024-09-10T23:32:43.628726+0000","last_peered":"2024-09-10T23:32:43.628726+0000","last_clean":"2024-09-10T23:32:43.628726+0000","last_became_active":"2024-09-10T23:32:41.575442+0000","last_became_peered":"2024-09-10T23:32:41.575442+0000","last_unstale":"2024-09-10T23:32:43.628726+0000","last_undegraded":"2024-09-10T23:32:43.628726+0000","last_fullsized":"2024-09-10T23:32:43.628726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:00.455999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627197+0000","last_change":"2024-09-10T23:32:42.571239+0000","last_active":"2024-09-10T23:32:43.627197+0000","last_peered":"2024-09-10T23:32:43.627197+0000","last_clean":"2024-09-10T23:32:43.627197+0000","last_became_active":"2024-09-10T23:32:42.570963+0000","last_became_peered":"2024-09-10T23:32:42.570963+0000","last_unstale":"2024-09-10T23:32:43.627197+0000","last_undegraded":"2024-09-10T23:32:43.627197+0000","last_fullsized":"2024-09-10T23:32:43.627197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:36.598389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628178+0000","last_change":"2024-09-10T23:32:41.569326+0000","last_active":"2024-09-10T23:32:43.628178+0000","last_peered":"2024-09-10T23:32:43.628178+0000","last_clean":"2024-09-10T23:32:43.628178+0000","last_became_active":"2024-09-10T23:32:41.569056+0000","last_became_peered":"2024-09-10T23:32:41.569056+0000","last_unstale":"2024-09-10T23:32:43.628178+0000","last_undegraded":"2024-09-10T23:32:43.628178+0000","last_fullsized":"2024-09-10T23:32:43.628178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:47.815432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626872+0000","last_change":"2024-09-10T23:32:42.574800+0000","last_active":"2024-09-10T23:32:43.626872+0000","last_peered":"2024-09-10T23:32:43.626872+0000","last_clean":"2024-09-10T23:32:43.626872+0000","last_became_active":"2024-09-10T23:32:42.574477+0000","last_became_peered":"2024-09-10T23:32:42.574477+0000","last_unstale":"2024-09-10T23:32:43.626872+0000","last_undegraded":"2024-09-10T23:32:43.626872+0000","last_fullsized":"2024-09-10T23:32:43.626872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:14.440912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629156+0000","last_change":"2024-09-10T23:32:41.582865+0000","last_active":"2024-09-10T23:32:43.629156+0000","last_peered":"2024-09-10T23:32:43.629156+0000","last_clean":"2024-09-10T23:32:43.629156+0000","last_became_active":"2024-09-10T23:32:41.582522+0000","last_became_peered":"2024-09-10T23:32:41.582522+0000","last_unstale":"2024-09-10T23:32:43.629156+0000","last_undegraded":"2024-09-10T23:32:43.629156+0000","last_fullsized":"2024-09-10T23:32:43.629156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:24.945862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570471+0000","last_change":"2024-09-10T23:32:42.587148+0000","last_active":"2024-09-10T23:32:43.570471+0000","last_peered":"2024-09-10T23:32:43.570471+0000","last_clean":"2024-09-10T23:32:43.570471+0000","last_became_active":"2024-09-10T23:32:42.586887+0000","last_became_peered":"2024-09-10T23:32:42.586887+0000","last_unstale":"2024-09-10T23:32:43.570471+0000","last_undegraded":"2024-09-10T23:32:43.570471+0000","last_fullsized":"2024-09-10T23:32:43.570471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:27.999337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570022+0000","last_change":"2024-09-10T23:32:41.553400+0000","last_active":"2024-09-10T23:32:43.570022+0000","last_peered":"2024-09-10T23:32:43.570022+0000","last_clean":"2024-09-10T23:32:43.570022+0000","last_became_active":"2024-09-10T23:32:41.553070+0000","last_became_peered":"2024-09-10T23:32:41.553070+0000","last_unstale":"2024-09-10T23:32:43.570022+0000","last_undegraded":"2024-09-10T23:32:43.570022+0000","last_fullsized":"2024-09-10T23:32:43.570022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:01.393799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569840+0000","last_change":"2024-09-10T23:32:42.580714+0000","last_active":"2024-09-10T23:32:43.569840+0000","last_peered":"2024-09-10T23:32:43.569840+0000","last_clean":"2024-09-10T23:32:43.569840+0000","last_became_active":"2024-09-10T23:32:42.579027+0000","last_became_peered":"2024-09-10T23:32:42.579027+0000","last_unstale":"2024-09-10T23:32:43.569840+0000","last_undegraded":"2024-09-10T23:32:43.569840+0000","last_fullsized":"2024-09-10T23:32:43.569840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:12.456589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629216+0000","last_change":"2024-09-10T23:32:41.580054+0000","last_active":"2024-09-10T23:32:43.629216+0000","last_peered":"2024-09-10T23:32:43.629216+0000","last_clean":"2024-09-10T23:32:43.629216+0000","last_became_active":"2024-09-10T23:32:41.579663+0000","last_became_peered":"2024-09-10T23:32:41.579663+0000","last_unstale":"2024-09-10T23:32:43.629216+0000","last_undegraded":"2024-09-10T23:32:43.629216+0000","last_fullsized":"2024-09-10T23:32:43.629216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:22.486416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569890+0000","last_change":"2024-09-10T23:32:42.574503+0000","last_active":"2024-09-10T23:32:43.569890+0000","last_peered":"2024-09-10T23:32:43.569890+0000","last_clean":"2024-09-10T23:32:43.569890+0000","last_became_active":"2024-09-10T23:32:42.574229+0000","last_became_peered":"2024-09-10T23:32:42.574229+0000","last_unstale":"2024-09-10T23:32:43.569890+0000","last_undegraded":"2024-09-10T23:32:43.569890+0000","last_fullsized":"2024-09-10T23:32:43.569890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:19.891408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626574+0000","last_change":"2024-09-10T23:32:41.562000+0000","last_active":"2024-09-10T23:32:43.626574+0000","last_peered":"2024-09-10T23:32:43.626574+0000","last_clean":"2024-09-10T23:32:43.626574+0000","last_became_active":"2024-09-10T23:32:41.561637+0000","last_became_peered":"2024-09-10T23:32:41.561637+0000","last_unstale":"2024-09-10T23:32:43.626574+0000","last_undegraded":"2024-09-10T23:32:43.626574+0000","last_fullsized":"2024-09-10T23:32:43.626574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:17.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628329+0000","last_change":"2024-09-10T23:32:42.583205+0000","last_active":"2024-09-10T23:32:43.628329+0000","last_peered":"2024-09-10T23:32:43.628329+0000","last_clean":"2024-09-10T23:32:43.628329+0000","last_became_active":"2024-09-10T23:32:42.582963+0000","last_became_peered":"2024-09-10T23:32:42.582963+0000","last_unstale":"2024-09-10T23:32:43.628329+0000","last_undegraded":"2024-09-10T23:32:43.628329+0000","last_fullsized":"2024-09-10T23:32:43.628329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.117710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569104+0000","last_change":"2024-09-10T23:32:41.557934+0000","last_active":"2024-09-10T23:32:43.569104+0000","last_peered":"2024-09-10T23:32:43.569104+0000","last_clean":"2024-09-10T23:32:43.569104+0000","last_became_active":"2024-09-10T23:32:41.557625+0000","last_became_peered":"2024-09-10T23:32:41.557625+0000","last_unstale":"2024-09-10T23:32:43.569104+0000","last_undegraded":"2024-09-10T23:32:43.569104+0000","last_fullsized":"2024-09-10T23:32:43.569104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:29.536215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569080+0000","last_change":"2024-09-10T23:32:42.567250+0000","last_active":"2024-09-10T23:32:43.569080+0000","last_peered":"2024-09-10T23:32:43.569080+0000","last_clean":"2024-09-10T23:32:43.569080+0000","last_became_active":"2024-09-10T23:32:42.566248+0000","last_became_peered":"2024-09-10T23:32:42.566248+0000","last_unstale":"2024-09-10T23:32:43.569080+0000","last_undegraded":"2024-09-10T23:32:43.569080+0000","last_fullsized":"2024-09-10T23:32:43.569080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:45.824956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570495+0000","last_change":"2024-09-10T23:32:41.566640+0000","last_active":"2024-09-10T23:32:43.570495+0000","last_peered":"2024-09-10T23:32:43.570495+0000","last_clean":"2024-09-10T23:32:43.570495+0000","last_became_active":"2024-09-10T23:32:41.566311+0000","last_became_peered":"2024-09-10T23:32:41.566311+0000","last_unstale":"2024-09-10T23:32:43.570495+0000","last_undegraded":"2024-09-10T23:32:43.570495+0000","last_fullsized":"2024-09-10T23:32:43.570495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:33.372580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628173+0000","last_change":"2024-09-10T23:32:42.580760+0000","last_active":"2024-09-10T23:32:43.628173+0000","last_peered":"2024-09-10T23:32:43.628173+0000","last_clean":"2024-09-10T23:32:43.628173+0000","last_became_active":"2024-09-10T23:32:42.580081+0000","last_became_peered":"2024-09-10T23:32:42.580081+0000","last_unstale":"2024-09-10T23:32:43.628173+0000","last_undegraded":"2024-09-10T23:32:43.628173+0000","last_fullsized":"2024-09-10T23:32:43.628173+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:06.266461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628952+0000","last_change":"2024-09-10T23:32:41.583976+0000","last_active":"2024-09-10T23:32:43.628952+0000","last_peered":"2024-09-10T23:32:43.628952+0000","last_clean":"2024-09-10T23:32:43.628952+0000","last_became_active":"2024-09-10T23:32:41.583639+0000","last_became_peered":"2024-09-10T23:32:41.583639+0000","last_unstale":"2024-09-10T23:32:43.628952+0000","last_undegraded":"2024-09-10T23:32:43.628952+0000","last_fullsized":"2024-09-10T23:32:43.628952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:26.669949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629220+0000","last_change":"2024-09-10T23:32:42.570056+0000","last_active":"2024-09-10T23:32:43.629220+0000","last_peered":"2024-09-10T23:32:43.629220+0000","last_clean":"2024-09-10T23:32:43.629220+0000","last_became_active":"2024-09-10T23:32:42.569794+0000","last_became_peered":"2024-09-10T23:32:42.569794+0000","last_unstale":"2024-09-10T23:32:43.629220+0000","last_undegraded":"2024-09-10T23:32:43.629220+0000","last_fullsized":"2024-09-10T23:32:43.629220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:36.453126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628526+0000","last_change":"2024-09-10T23:32:42.585151+0000","last_active":"2024-09-10T23:32:43.628526+0000","last_peered":"2024-09-10T23:32:43.628526+0000","last_clean":"2024-09-10T23:32:43.628526+0000","last_became_active":"2024-09-10T23:32:42.578092+0000","last_became_peered":"2024-09-10T23:32:42.578092+0000","last_unstale":"2024-09-10T23:32:43.628526+0000","last_undegraded":"2024-09-10T23:32:43.628526+0000","last_fullsized":"2024-09-10T23:32:43.628526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:39:56.735321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628891+0000","last_change":"2024-09-10T23:32:41.571659+0000","last_active":"2024-09-10T23:32:43.628891+0000","last_peered":"2024-09-10T23:32:43.628891+0000","last_clean":"2024-09-10T23:32:43.628891+0000","last_became_active":"2024-09-10T23:32:41.571418+0000","last_became_peered":"2024-09-10T23:32:41.571418+0000","last_unstale":"2024-09-10T23:32:43.628891+0000","last_undegraded":"2024-09-10T23:32:43.628891+0000","last_fullsized":"2024-09-10T23:32:43.628891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:57.665825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563423+0000","last_change":"2024-09-10T23:32:42.563064+0000","last_active":"2024-09-10T23:32:43.563423+0000","last_peered":"2024-09-10T23:32:43.563423+0000","last_clean":"2024-09-10T23:32:43.563423+0000","last_became_active":"2024-09-10T23:32:42.562808+0000","last_became_peered":"2024-09-10T23:32:42.562808+0000","last_unstale":"2024-09-10T23:32:43.563423+0000","last_undegraded":"2024-09-10T23:32:43.563423+0000","last_fullsized":"2024-09-10T23:32:43.563423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:20.388860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627661+0000","last_change":"2024-09-10T23:32:41.570670+0000","last_active":"2024-09-10T23:32:43.627661+0000","last_peered":"2024-09-10T23:32:43.627661+0000","last_clean":"2024-09-10T23:32:43.627661+0000","last_became_active":"2024-09-10T23:32:41.570291+0000","last_became_peered":"2024-09-10T23:32:41.570291+0000","last_unstale":"2024-09-10T23:32:43.627661+0000","last_undegraded":"2024-09-10T23:32:43.627661+0000","last_fullsized":"2024-09-10T23:32:43.627661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:33.183319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629201+0000","last_change":"2024-09-10T23:32:42.584736+0000","last_active":"2024-09-10T23:32:43.629201+0000","last_peered":"2024-09-10T23:32:43.629201+0000","last_clean":"2024-09-10T23:32:43.629201+0000","last_became_active":"2024-09-10T23:32:42.583535+0000","last_became_peered":"2024-09-10T23:32:42.583535+0000","last_unstale":"2024-09-10T23:32:43.629201+0000","last_undegraded":"2024-09-10T23:32:43.629201+0000","last_fullsized":"2024-09-10T23:32:43.629201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:00.689816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629197+0000","last_change":"2024-09-10T23:32:41.566445+0000","last_active":"2024-09-10T23:32:43.629197+0000","last_peered":"2024-09-10T23:32:43.629197+0000","last_clean":"2024-09-10T23:32:43.629197+0000","last_became_active":"2024-09-10T23:32:41.566158+0000","last_became_peered":"2024-09-10T23:32:41.566158+0000","last_unstale":"2024-09-10T23:32:43.629197+0000","last_undegraded":"2024-09-10T23:32:43.629197+0000","last_fullsized":"2024-09-10T23:32:43.629197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:55.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629299+0000","last_change":"2024-09-10T23:32:42.584147+0000","last_active":"2024-09-10T23:32:43.629299+0000","last_peered":"2024-09-10T23:32:43.629299+0000","last_clean":"2024-09-10T23:32:43.629299+0000","last_became_active":"2024-09-10T23:32:42.583766+0000","last_became_peered":"2024-09-10T23:32:42.583766+0000","last_unstale":"2024-09-10T23:32:43.629299+0000","last_undegraded":"2024-09-10T23:32:43.629299+0000","last_fullsized":"2024-09-10T23:32:43.629299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:19.288688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569913+0000","last_change":"2024-09-10T23:32:41.580965+0000","last_active":"2024-09-10T23:32:43.569913+0000","last_peered":"2024-09-10T23:32:43.569913+0000","last_clean":"2024-09-10T23:32:43.569913+0000","last_became_active":"2024-09-10T23:32:41.580712+0000","last_became_peered":"2024-09-10T23:32:41.580712+0000","last_unstale":"2024-09-10T23:32:43.569913+0000","last_undegraded":"2024-09-10T23:32:43.569913+0000","last_fullsized":"2024-09-10T23:32:43.569913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:13.433884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:32:57.347 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:32:57.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.525+0000 7f9fa3200640 1 -- 172.21.15.94:0/1183952403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa40d22e0 msgr2=0x7f9fa40d2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:57.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.525+0000 7f9fa3200640 1 --2- 172.21.15.94:0/1183952403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa40d22e0 0x7f9fa40d2750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009840 tx=0x7f9f8c01c800 comp rx=0 tx=0).stop 2024-09-10T23:32:57.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.525+0000 7f9fa3200640 1 -- 172.21.15.94:0/1183952403 shutdown_connections 2024-09-10T23:32:57.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.525+0000 7f9fa3200640 1 --2- 172.21.15.94:0/1183952403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa40b3f30 0x7f9fa40b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.525+0000 7f9fa3200640 1 --2- 172.21.15.94:0/1183952403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa40d22e0 0x7f9fa40d2750 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.525+0000 7f9fa3200640 1 -- 172.21.15.94:0/1183952403 >> 172.21.15.94:0/1183952403 conn(0x7f9fa409d1c0 msgr2=0x7f9fa40d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:57.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa3200640 1 -- 172.21.15.94:0/1183952403 shutdown_connections 2024-09-10T23:32:57.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa3200640 1 -- 172.21.15.94:0/1183952403 wait complete. 2024-09-10T23:32:57.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa3200640 1 Processor -- start 2024-09-10T23:32:57.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa3200640 1 -- start start 2024-09-10T23:32:57.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa3200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa40b3f30 0x7f9fa4154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:57.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa3200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa40d1f00 0x7f9fa415c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:57.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa3200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fa40d22e0 0x7f9fa406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:57.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa3200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa4162940 con 0x7f9fa40b3f30 2024-09-10T23:32:57.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa3200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa41627c0 con 0x7f9fa40d22e0 2024-09-10T23:32:57.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa3200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa416aec0 con 0x7f9fa40d1f00 2024-09-10T23:32:57.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa40b3f30 0x7f9fa4154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:57.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa40b3f30 0x7f9fa4154bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40360/0 (socket says 172.21.15.94:40360) 2024-09-10T23:32:57.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa1e00640 1 -- 172.21.15.94:0/2382696759 learned_addr learned my addr 172.21.15.94:0/2382696759 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:57.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa1e00640 1 -- 172.21.15.94:0/2382696759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa40d1f00 msgr2=0x7f9fa415c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:57.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa2800640 1 --2- 172.21.15.94:0/2382696759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fa40d22e0 0x7f9fa406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:57.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9f99400640 1 --2- 172.21.15.94:0/2382696759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa40d1f00 0x7f9fa415c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:57.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa1e00640 1 --2- 172.21.15.94:0/2382696759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa40d1f00 0x7f9fa415c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa1e00640 1 -- 172.21.15.94:0/2382696759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fa40d22e0 msgr2=0x7f9fa406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:57.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa1e00640 1 --2- 172.21.15.94:0/2382696759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fa40d22e0 0x7f9fa406d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa1e00640 1 -- 172.21.15.94:0/2382696759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa40c95f0 con 0x7f9fa40b3f30 2024-09-10T23:32:57.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9f99400640 1 --2- 172.21.15.94:0/2382696759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa40d1f00 0x7f9fa415c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:32:57.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa2800640 1 --2- 172.21.15.94:0/2382696759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fa40d22e0 0x7f9fa406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:32:57.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa1e00640 1 --2- 172.21.15.94:0/2382696759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa40b3f30 0x7f9fa4154bc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400c530 tx=0x7f9f9400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:57.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9f9aa00640 1 -- 172.21.15.94:0/2382696759 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94017070 con 0x7f9fa40b3f30 2024-09-10T23:32:57.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9f9aa00640 1 -- 172.21.15.94:0/2382696759 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f94003600 con 0x7f9fa40b3f30 2024-09-10T23:32:57.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9f9aa00640 1 -- 172.21.15.94:0/2382696759 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94003a50 con 0x7f9fa40b3f30 2024-09-10T23:32:57.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.529+0000 7f9fa3200640 1 -- 172.21.15.94:0/2382696759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa40c98d0 con 0x7f9fa40b3f30 2024-09-10T23:32:57.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.533+0000 7f9fa3200640 1 -- 172.21.15.94:0/2382696759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa4155350 con 0x7f9fa40b3f30 2024-09-10T23:32:57.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.533+0000 7f9fa3200640 1 -- 172.21.15.94:0/2382696759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa40b43e0 con 0x7f9fa40b3f30 2024-09-10T23:32:57.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.533+0000 7f9f9aa00640 1 -- 172.21.15.94:0/2382696759 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9f94003bf0 con 0x7f9fa40b3f30 2024-09-10T23:32:57.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.533+0000 7f9f9aa00640 1 --2- 172.21.15.94:0/2382696759 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9f80072610 0x7f9f80074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:57.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.533+0000 7f9f9aa00640 1 -- 172.21.15.94:0/2382696759 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9f94092940 con 0x7f9fa40b3f30 2024-09-10T23:32:57.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.533+0000 7f9f99400640 1 --2- 172.21.15.94:0/2382696759 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9f80072610 0x7f9f80074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:57.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.537+0000 7f9f99400640 1 --2- 172.21.15.94:0/2382696759 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9f80072610 0x7f9f80074ad0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0af000 tx=0x7f9f8c01b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.537+0000 7f9f9aa00640 1 -- 172.21.15.94:0/2382696759 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9f9405e0d0 con 0x7f9fa40b3f30 2024-09-10T23:32:57.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.705+0000 7f9fa3200640 1 -- 172.21.15.94:0/2382696759 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9fa40c9df0 con 0x7f9f80072610 2024-09-10T23:32:57.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.709+0000 7f9f9aa00640 1 -- 172.21.15.94:0/2382696759 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7f9fa40c9df0 con 0x7f9f80072610 2024-09-10T23:32:57.716 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:57.717 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:32:57.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.713+0000 7f9fa3200640 1 -- 172.21.15.94:0/2382696759 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9f80072610 msgr2=0x7f9f80074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:57.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.713+0000 7f9fa3200640 1 --2- 172.21.15.94:0/2382696759 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9f80072610 0x7f9f80074ad0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0af000 tx=0x7f9f8c01b040 comp rx=0 tx=0).stop 2024-09-10T23:32:57.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.713+0000 7f9fa3200640 1 -- 172.21.15.94:0/2382696759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa40b3f30 msgr2=0x7f9fa4154bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:57.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.713+0000 7f9fa3200640 1 --2- 172.21.15.94:0/2382696759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa40b3f30 0x7f9fa4154bc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400c530 tx=0x7f9f9400c9f0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.713+0000 7f9fa3200640 1 -- 172.21.15.94:0/2382696759 shutdown_connections 2024-09-10T23:32:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.713+0000 7f9fa3200640 1 --2- 172.21.15.94:0/2382696759 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9f80072610 0x7f9f80074ad0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.713+0000 7f9fa3200640 1 --2- 172.21.15.94:0/2382696759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fa40d22e0 0x7f9fa406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.713+0000 7f9fa3200640 1 --2- 172.21.15.94:0/2382696759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa40d1f00 0x7f9fa415c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.713+0000 7f9fa3200640 1 --2- 172.21.15.94:0/2382696759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa40b3f30 0x7f9fa4154bc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.713+0000 7f9fa3200640 1 -- 172.21.15.94:0/2382696759 >> 172.21.15.94:0/2382696759 conn(0x7f9fa409d1c0 msgr2=0x7f9fa40ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.713+0000 7f9fa3200640 1 -- 172.21.15.94:0/2382696759 shutdown_connections 2024-09-10T23:32:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.713+0000 7f9fa3200640 1 -- 172.21.15.94:0/2382696759 wait complete. 2024-09-10T23:32:57.736 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-10T23:32:57.345361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235848,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738872,"statfs":{"total":773094113280,"available":772852604928,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003231"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627741+0000","last_change":"2024-09-10T23:32:42.593279+0000","last_active":"2024-09-10T23:32:43.627741+0000","last_peered":"2024-09-10T23:32:43.627741+0000","last_clean":"2024-09-10T23:32:43.627741+0000","last_became_active":"2024-09-10T23:32:42.593056+0000","last_became_peered":"2024-09-10T23:32:42.593056+0000","last_unstale":"2024-09-10T23:32:43.627741+0000","last_undegraded":"2024-09-10T23:32:43.627741+0000","last_fullsized":"2024-09-10T23:32:43.627741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.112551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628801+0000","last_change":"2024-09-10T23:32:42.584101+0000","last_active":"2024-09-10T23:32:43.628801+0000","last_peered":"2024-09-10T23:32:43.628801+0000","last_clean":"2024-09-10T23:32:43.628801+0000","last_became_active":"2024-09-10T23:32:42.583796+0000","last_became_peered":"2024-09-10T23:32:42.583796+0000","last_unstale":"2024-09-10T23:32:43.628801+0000","last_undegraded":"2024-09-10T23:32:43.628801+0000","last_fullsized":"2024-09-10T23:32:43.628801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:01.463027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628392+0000","last_change":"2024-09-10T23:32:42.584302+0000","last_active":"2024-09-10T23:32:43.628392+0000","last_peered":"2024-09-10T23:32:43.628392+0000","last_clean":"2024-09-10T23:32:43.628392+0000","last_became_active":"2024-09-10T23:32:42.583989+0000","last_became_peered":"2024-09-10T23:32:42.583989+0000","last_unstale":"2024-09-10T23:32:43.628392+0000","last_undegraded":"2024-09-10T23:32:43.628392+0000","last_fullsized":"2024-09-10T23:32:43.628392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:53.622106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626948+0000","last_change":"2024-09-10T23:32:42.565401+0000","last_active":"2024-09-10T23:32:43.626948+0000","last_peered":"2024-09-10T23:32:43.626948+0000","last_clean":"2024-09-10T23:32:43.626948+0000","last_became_active":"2024-09-10T23:32:42.565106+0000","last_became_peered":"2024-09-10T23:32:42.565106+0000","last_unstale":"2024-09-10T23:32:43.626948+0000","last_undegraded":"2024-09-10T23:32:43.626948+0000","last_fullsized":"2024-09-10T23:32:43.626948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:20.388860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627066+0000","last_change":"2024-09-10T23:32:42.586253+0000","last_active":"2024-09-10T23:32:43.627066+0000","last_peered":"2024-09-10T23:32:43.627066+0000","last_clean":"2024-09-10T23:32:43.627066+0000","last_became_active":"2024-09-10T23:32:42.585961+0000","last_became_peered":"2024-09-10T23:32:42.585961+0000","last_unstale":"2024-09-10T23:32:43.627066+0000","last_undegraded":"2024-09-10T23:32:43.627066+0000","last_fullsized":"2024-09-10T23:32:43.627066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:19.891408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626891+0000","last_change":"2024-09-10T23:32:42.565746+0000","last_active":"2024-09-10T23:32:43.626891+0000","last_peered":"2024-09-10T23:32:43.626891+0000","last_clean":"2024-09-10T23:32:43.626891+0000","last_became_active":"2024-09-10T23:32:42.565398+0000","last_became_peered":"2024-09-10T23:32:42.565398+0000","last_unstale":"2024-09-10T23:32:43.626891+0000","last_undegraded":"2024-09-10T23:32:43.626891+0000","last_fullsized":"2024-09-10T23:32:43.626891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:28.431860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627187+0000","last_change":"2024-09-10T23:32:42.567960+0000","last_active":"2024-09-10T23:32:43.627187+0000","last_peered":"2024-09-10T23:32:43.627187+0000","last_clean":"2024-09-10T23:32:43.627187+0000","last_became_active":"2024-09-10T23:32:42.567653+0000","last_became_peered":"2024-09-10T23:32:42.567653+0000","last_unstale":"2024-09-10T23:32:43.627187+0000","last_undegraded":"2024-09-10T23:32:43.627187+0000","last_fullsized":"2024-09-10T23:32:43.627187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:26.192704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628554+0000","last_change":"2024-09-10T23:32:42.584639+0000","last_active":"2024-09-10T23:32:43.628554+0000","last_peered":"2024-09-10T23:32:43.628554+0000","last_clean":"2024-09-10T23:32:43.628554+0000","last_became_active":"2024-09-10T23:32:42.584396+0000","last_became_peered":"2024-09-10T23:32:42.584396+0000","last_unstale":"2024-09-10T23:32:43.628554+0000","last_undegraded":"2024-09-10T23:32:43.628554+0000","last_fullsized":"2024-09-10T23:32:43.628554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:27.237846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563757+0000","last_change":"2024-09-10T23:32:42.589096+0000","last_active":"2024-09-10T23:32:43.563757+0000","last_peered":"2024-09-10T23:32:43.563757+0000","last_clean":"2024-09-10T23:32:43.563757+0000","last_became_active":"2024-09-10T23:32:42.588695+0000","last_became_peered":"2024-09-10T23:32:42.588695+0000","last_unstale":"2024-09-10T23:32:43.563757+0000","last_undegraded":"2024-09-10T23:32:43.563757+0000","last_fullsized":"2024-09-10T23:32:43.563757+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:14.778902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627628+0000","last_change":"2024-09-10T23:32:42.572168+0000","last_active":"2024-09-10T23:32:43.627628+0000","last_peered":"2024-09-10T23:32:43.627628+0000","last_clean":"2024-09-10T23:32:43.627628+0000","last_became_active":"2024-09-10T23:32:42.571638+0000","last_became_peered":"2024-09-10T23:32:42.571638+0000","last_unstale":"2024-09-10T23:32:43.627628+0000","last_undegraded":"2024-09-10T23:32:43.627628+0000","last_fullsized":"2024-09-10T23:32:43.627628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:51.184952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627526+0000","last_change":"2024-09-10T23:32:42.580539+0000","last_active":"2024-09-10T23:32:43.627526+0000","last_peered":"2024-09-10T23:32:43.627526+0000","last_clean":"2024-09-10T23:32:43.627526+0000","last_became_active":"2024-09-10T23:32:42.580180+0000","last_became_peered":"2024-09-10T23:32:42.580180+0000","last_unstale":"2024-09-10T23:32:43.627526+0000","last_undegraded":"2024-09-10T23:32:43.627526+0000","last_fullsized":"2024-09-10T23:32:43.627526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:15.490925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627548+0000","last_change":"2024-09-10T23:32:42.568100+0000","last_active":"2024-09-10T23:32:43.627548+0000","last_peered":"2024-09-10T23:32:43.627548+0000","last_clean":"2024-09-10T23:32:43.627548+0000","last_became_active":"2024-09-10T23:32:42.567409+0000","last_became_peered":"2024-09-10T23:32:42.567409+0000","last_unstale":"2024-09-10T23:32:43.627548+0000","last_undegraded":"2024-09-10T23:32:43.627548+0000","last_fullsized":"2024-09-10T23:32:43.627548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.117710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569985+0000","last_change":"2024-09-10T23:32:42.569811+0000","last_active":"2024-09-10T23:32:43.569985+0000","last_peered":"2024-09-10T23:32:43.569985+0000","last_clean":"2024-09-10T23:32:43.569985+0000","last_became_active":"2024-09-10T23:32:42.569470+0000","last_became_peered":"2024-09-10T23:32:42.569470+0000","last_unstale":"2024-09-10T23:32:43.569985+0000","last_undegraded":"2024-09-10T23:32:43.569985+0000","last_fullsized":"2024-09-10T23:32:43.569985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:56.266045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570094+0000","last_change":"2024-09-10T23:32:42.578563+0000","last_active":"2024-09-10T23:32:43.570094+0000","last_peered":"2024-09-10T23:32:43.570094+0000","last_clean":"2024-09-10T23:32:43.570094+0000","last_became_active":"2024-09-10T23:32:42.578281+0000","last_became_peered":"2024-09-10T23:32:42.578281+0000","last_unstale":"2024-09-10T23:32:43.570094+0000","last_undegraded":"2024-09-10T23:32:43.570094+0000","last_fullsized":"2024-09-10T23:32:43.570094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:57.340004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628709+0000","last_change":"2024-09-10T23:32:42.564792+0000","last_active":"2024-09-10T23:32:43.628709+0000","last_peered":"2024-09-10T23:32:43.628709+0000","last_clean":"2024-09-10T23:32:43.628709+0000","last_became_active":"2024-09-10T23:32:42.564029+0000","last_became_peered":"2024-09-10T23:32:42.564029+0000","last_unstale":"2024-09-10T23:32:43.628709+0000","last_undegraded":"2024-09-10T23:32:43.628709+0000","last_fullsized":"2024-09-10T23:32:43.628709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:23.318586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628280+0000","last_change":"2024-09-10T23:32:42.583724+0000","last_active":"2024-09-10T23:32:43.628280+0000","last_peered":"2024-09-10T23:32:43.628280+0000","last_clean":"2024-09-10T23:32:43.628280+0000","last_became_active":"2024-09-10T23:32:42.583473+0000","last_became_peered":"2024-09-10T23:32:42.583473+0000","last_unstale":"2024-09-10T23:32:43.628280+0000","last_undegraded":"2024-09-10T23:32:43.628280+0000","last_fullsized":"2024-09-10T23:32:43.628280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:14.468774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629190+0000","last_change":"2024-09-10T23:32:42.588295+0000","last_active":"2024-09-10T23:32:43.629190+0000","last_peered":"2024-09-10T23:32:43.629190+0000","last_clean":"2024-09-10T23:32:43.629190+0000","last_became_active":"2024-09-10T23:32:42.587993+0000","last_became_peered":"2024-09-10T23:32:42.587993+0000","last_unstale":"2024-09-10T23:32:43.629190+0000","last_undegraded":"2024-09-10T23:32:43.629190+0000","last_fullsized":"2024-09-10T23:32:43.629190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:18:27.504742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629147+0000","last_change":"2024-09-10T23:32:42.579982+0000","last_active":"2024-09-10T23:32:43.629147+0000","last_peered":"2024-09-10T23:32:43.629147+0000","last_clean":"2024-09-10T23:32:43.629147+0000","last_became_active":"2024-09-10T23:32:42.575814+0000","last_became_peered":"2024-09-10T23:32:42.575814+0000","last_unstale":"2024-09-10T23:32:43.629147+0000","last_undegraded":"2024-09-10T23:32:43.629147+0000","last_fullsized":"2024-09-10T23:32:43.629147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:58.672853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629002+0000","last_change":"2024-09-10T23:32:42.585466+0000","last_active":"2024-09-10T23:32:43.629002+0000","last_peered":"2024-09-10T23:32:43.629002+0000","last_clean":"2024-09-10T23:32:43.629002+0000","last_became_active":"2024-09-10T23:32:42.582650+0000","last_became_peered":"2024-09-10T23:32:42.582650+0000","last_unstale":"2024-09-10T23:32:43.629002+0000","last_undegraded":"2024-09-10T23:32:43.629002+0000","last_fullsized":"2024-09-10T23:32:43.629002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:59.434883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627679+0000","last_change":"2024-09-10T23:32:42.573723+0000","last_active":"2024-09-10T23:32:43.627679+0000","last_peered":"2024-09-10T23:32:43.627679+0000","last_clean":"2024-09-10T23:32:43.627679+0000","last_became_active":"2024-09-10T23:32:42.573309+0000","last_became_peered":"2024-09-10T23:32:42.573309+0000","last_unstale":"2024-09-10T23:32:43.627679+0000","last_undegraded":"2024-09-10T23:32:43.627679+0000","last_fullsized":"2024-09-10T23:32:43.627679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:08.762419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569445+0000","last_change":"2024-09-10T23:32:42.558045+0000","last_active":"2024-09-10T23:32:43.569445+0000","last_peered":"2024-09-10T23:32:43.569445+0000","last_clean":"2024-09-10T23:32:43.569445+0000","last_became_active":"2024-09-10T23:32:42.557774+0000","last_became_peered":"2024-09-10T23:32:42.557774+0000","last_unstale":"2024-09-10T23:32:43.569445+0000","last_undegraded":"2024-09-10T23:32:43.569445+0000","last_fullsized":"2024-09-10T23:32:43.569445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:53.622106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563214+0000","last_change":"2024-09-10T23:32:42.556890+0000","last_active":"2024-09-10T23:32:43.563214+0000","last_peered":"2024-09-10T23:32:43.563214+0000","last_clean":"2024-09-10T23:32:43.563214+0000","last_became_active":"2024-09-10T23:32:42.556625+0000","last_became_peered":"2024-09-10T23:32:42.556625+0000","last_unstale":"2024-09-10T23:32:43.563214+0000","last_undegraded":"2024-09-10T23:32:43.563214+0000","last_fullsized":"2024-09-10T23:32:43.563214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:14.440912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628804+0000","last_change":"2024-09-10T23:32:42.575698+0000","last_active":"2024-09-10T23:32:43.628804+0000","last_peered":"2024-09-10T23:32:43.628804+0000","last_clean":"2024-09-10T23:32:43.628804+0000","last_became_active":"2024-09-10T23:32:42.575371+0000","last_became_peered":"2024-09-10T23:32:42.575371+0000","last_unstale":"2024-09-10T23:32:43.628804+0000","last_undegraded":"2024-09-10T23:32:43.628804+0000","last_fullsized":"2024-09-10T23:32:43.628804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:36.598389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628388+0000","last_change":"2024-09-10T23:32:42.591870+0000","last_active":"2024-09-10T23:32:43.628388+0000","last_peered":"2024-09-10T23:32:43.628388+0000","last_clean":"2024-09-10T23:32:43.628388+0000","last_became_active":"2024-09-10T23:32:42.590207+0000","last_became_peered":"2024-09-10T23:32:42.590207+0000","last_unstale":"2024-09-10T23:32:43.628388+0000","last_undegraded":"2024-09-10T23:32:43.628388+0000","last_fullsized":"2024-09-10T23:32:43.628388+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:40.104986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629051+0000","last_change":"2024-09-10T23:32:42.588702+0000","last_active":"2024-09-10T23:32:43.629051+0000","last_peered":"2024-09-10T23:32:43.629051+0000","last_clean":"2024-09-10T23:32:43.629051+0000","last_became_active":"2024-09-10T23:32:42.588444+0000","last_became_peered":"2024-09-10T23:32:42.588444+0000","last_unstale":"2024-09-10T23:32:43.629051+0000","last_undegraded":"2024-09-10T23:32:43.629051+0000","last_fullsized":"2024-09-10T23:32:43.629051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:43.173720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627479+0000","last_change":"2024-09-10T23:32:42.580821+0000","last_active":"2024-09-10T23:32:43.627479+0000","last_peered":"2024-09-10T23:32:43.627479+0000","last_clean":"2024-09-10T23:32:43.627479+0000","last_became_active":"2024-09-10T23:32:42.580319+0000","last_became_peered":"2024-09-10T23:32:42.580319+0000","last_unstale":"2024-09-10T23:32:43.627479+0000","last_undegraded":"2024-09-10T23:32:43.627479+0000","last_fullsized":"2024-09-10T23:32:43.627479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:26.735696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570454+0000","last_change":"2024-09-10T23:32:42.574625+0000","last_active":"2024-09-10T23:32:43.570454+0000","last_peered":"2024-09-10T23:32:43.570454+0000","last_clean":"2024-09-10T23:32:43.570454+0000","last_became_active":"2024-09-10T23:32:42.571847+0000","last_became_peered":"2024-09-10T23:32:42.571847+0000","last_unstale":"2024-09-10T23:32:43.570454+0000","last_undegraded":"2024-09-10T23:32:43.570454+0000","last_fullsized":"2024-09-10T23:32:43.570454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:20.906137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570331+0000","last_change":"2024-09-10T23:32:42.567479+0000","last_active":"2024-09-10T23:32:43.570331+0000","last_peered":"2024-09-10T23:32:43.570331+0000","last_clean":"2024-09-10T23:32:43.570331+0000","last_became_active":"2024-09-10T23:32:42.567172+0000","last_became_peered":"2024-09-10T23:32:42.567172+0000","last_unstale":"2024-09-10T23:32:43.570331+0000","last_undegraded":"2024-09-10T23:32:43.570331+0000","last_fullsized":"2024-09-10T23:32:43.570331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:15.280711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628395+0000","last_change":"2024-09-10T23:32:42.585087+0000","last_active":"2024-09-10T23:32:43.628395+0000","last_peered":"2024-09-10T23:32:43.628395+0000","last_clean":"2024-09-10T23:32:43.628395+0000","last_became_active":"2024-09-10T23:32:42.583712+0000","last_became_peered":"2024-09-10T23:32:42.583712+0000","last_unstale":"2024-09-10T23:32:43.628395+0000","last_undegraded":"2024-09-10T23:32:43.628395+0000","last_fullsized":"2024-09-10T23:32:43.628395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:25.952890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628168+0000","last_change":"2024-09-10T23:32:42.591882+0000","last_active":"2024-09-10T23:32:43.628168+0000","last_peered":"2024-09-10T23:32:43.628168+0000","last_clean":"2024-09-10T23:32:43.628168+0000","last_became_active":"2024-09-10T23:32:42.590231+0000","last_became_peered":"2024-09-10T23:32:42.590231+0000","last_unstale":"2024-09-10T23:32:43.628168+0000","last_undegraded":"2024-09-10T23:32:43.628168+0000","last_fullsized":"2024-09-10T23:32:43.628168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:33.100231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628038+0000","last_change":"2024-09-10T23:32:42.573851+0000","last_active":"2024-09-10T23:32:43.628038+0000","last_peered":"2024-09-10T23:32:43.628038+0000","last_clean":"2024-09-10T23:32:43.628038+0000","last_became_active":"2024-09-10T23:32:42.572836+0000","last_became_peered":"2024-09-10T23:32:42.572836+0000","last_unstale":"2024-09-10T23:32:43.628038+0000","last_undegraded":"2024-09-10T23:32:43.628038+0000","last_fullsized":"2024-09-10T23:32:43.628038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:54.816587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627085+0000","last_change":"2024-09-10T23:32:42.576085+0000","last_active":"2024-09-10T23:32:43.627085+0000","last_peered":"2024-09-10T23:32:43.627085+0000","last_clean":"2024-09-10T23:32:43.627085+0000","last_became_active":"2024-09-10T23:32:42.575374+0000","last_became_peered":"2024-09-10T23:32:42.575374+0000","last_unstale":"2024-09-10T23:32:43.627085+0000","last_undegraded":"2024-09-10T23:32:43.627085+0000","last_fullsized":"2024-09-10T23:32:43.627085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:14.778902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563267+0000","last_change":"2024-09-10T23:32:42.588842+0000","last_active":"2024-09-10T23:32:43.563267+0000","last_peered":"2024-09-10T23:32:43.563267+0000","last_clean":"2024-09-10T23:32:43.563267+0000","last_became_active":"2024-09-10T23:32:42.588582+0000","last_became_peered":"2024-09-10T23:32:42.588582+0000","last_unstale":"2024-09-10T23:32:43.563267+0000","last_undegraded":"2024-09-10T23:32:43.563267+0000","last_fullsized":"2024-09-10T23:32:43.563267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:28.431860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628414+0000","last_change":"2024-09-10T23:32:41.569293+0000","last_active":"2024-09-10T23:32:43.628414+0000","last_peered":"2024-09-10T23:32:43.628414+0000","last_clean":"2024-09-10T23:32:43.628414+0000","last_became_active":"2024-09-10T23:32:41.568848+0000","last_became_peered":"2024-09-10T23:32:41.568848+0000","last_unstale":"2024-09-10T23:32:43.628414+0000","last_undegraded":"2024-09-10T23:32:43.628414+0000","last_fullsized":"2024-09-10T23:32:43.628414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:17.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569497+0000","last_change":"2024-09-10T23:32:42.568214+0000","last_active":"2024-09-10T23:32:43.569497+0000","last_peered":"2024-09-10T23:32:43.569497+0000","last_clean":"2024-09-10T23:32:43.569497+0000","last_became_active":"2024-09-10T23:32:42.562093+0000","last_became_peered":"2024-09-10T23:32:42.562093+0000","last_unstale":"2024-09-10T23:32:43.569497+0000","last_undegraded":"2024-09-10T23:32:43.569497+0000","last_fullsized":"2024-09-10T23:32:43.569497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:02.454545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628550+0000","last_change":"2024-09-10T23:32:41.578635+0000","last_active":"2024-09-10T23:32:43.628550+0000","last_peered":"2024-09-10T23:32:43.628550+0000","last_clean":"2024-09-10T23:32:43.628550+0000","last_became_active":"2024-09-10T23:32:41.578346+0000","last_became_peered":"2024-09-10T23:32:41.578346+0000","last_unstale":"2024-09-10T23:32:43.628550+0000","last_undegraded":"2024-09-10T23:32:43.628550+0000","last_fullsized":"2024-09-10T23:32:43.628550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:25.728667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569793+0000","last_change":"2024-09-10T23:32:41.553674+0000","last_active":"2024-09-10T23:32:43.569793+0000","last_peered":"2024-09-10T23:32:43.569793+0000","last_clean":"2024-09-10T23:32:43.569793+0000","last_became_active":"2024-09-10T23:32:41.553244+0000","last_became_peered":"2024-09-10T23:32:41.553244+0000","last_unstale":"2024-09-10T23:32:43.569793+0000","last_undegraded":"2024-09-10T23:32:43.569793+0000","last_fullsized":"2024-09-10T23:32:43.569793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:38.098241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628828+0000","last_change":"2024-09-10T23:32:42.583587+0000","last_active":"2024-09-10T23:32:43.628828+0000","last_peered":"2024-09-10T23:32:43.628828+0000","last_clean":"2024-09-10T23:32:43.628828+0000","last_became_active":"2024-09-10T23:32:42.581831+0000","last_became_peered":"2024-09-10T23:32:42.581831+0000","last_unstale":"2024-09-10T23:32:43.628828+0000","last_undegraded":"2024-09-10T23:32:43.628828+0000","last_fullsized":"2024-09-10T23:32:43.628828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:23.493444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628532+0000","last_change":"2024-09-10T23:32:41.581810+0000","last_active":"2024-09-10T23:32:43.628532+0000","last_peered":"2024-09-10T23:32:43.628532+0000","last_clean":"2024-09-10T23:32:43.628532+0000","last_became_active":"2024-09-10T23:32:41.581531+0000","last_became_peered":"2024-09-10T23:32:41.581531+0000","last_unstale":"2024-09-10T23:32:43.628532+0000","last_undegraded":"2024-09-10T23:32:43.628532+0000","last_fullsized":"2024-09-10T23:32:43.628532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:57.665825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569188+0000","last_change":"2024-09-10T23:32:42.560663+0000","last_active":"2024-09-10T23:32:43.569188+0000","last_peered":"2024-09-10T23:32:43.569188+0000","last_clean":"2024-09-10T23:32:43.569188+0000","last_became_active":"2024-09-10T23:32:42.560293+0000","last_became_peered":"2024-09-10T23:32:42.560293+0000","last_unstale":"2024-09-10T23:32:43.569188+0000","last_undegraded":"2024-09-10T23:32:43.569188+0000","last_fullsized":"2024-09-10T23:32:43.569188+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:39.387246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563757+0000","last_change":"2024-09-10T23:32:41.573958+0000","last_active":"2024-09-10T23:32:43.563757+0000","last_peered":"2024-09-10T23:32:43.563757+0000","last_clean":"2024-09-10T23:32:43.563757+0000","last_became_active":"2024-09-10T23:32:41.573639+0000","last_became_peered":"2024-09-10T23:32:41.573639+0000","last_unstale":"2024-09-10T23:32:43.563757+0000","last_undegraded":"2024-09-10T23:32:43.563757+0000","last_fullsized":"2024-09-10T23:32:43.563757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:27.548086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627506+0000","last_change":"2024-09-10T23:32:42.580586+0000","last_active":"2024-09-10T23:32:43.627506+0000","last_peered":"2024-09-10T23:32:43.627506+0000","last_clean":"2024-09-10T23:32:43.627506+0000","last_became_active":"2024-09-10T23:32:42.579960+0000","last_became_peered":"2024-09-10T23:32:42.579960+0000","last_unstale":"2024-09-10T23:32:43.627506+0000","last_undegraded":"2024-09-10T23:32:43.627506+0000","last_fullsized":"2024-09-10T23:32:43.627506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:09.640521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627910+0000","last_change":"2024-09-10T23:32:41.570662+0000","last_active":"2024-09-10T23:32:43.627910+0000","last_peered":"2024-09-10T23:32:43.627910+0000","last_clean":"2024-09-10T23:32:43.627910+0000","last_became_active":"2024-09-10T23:32:41.570330+0000","last_became_peered":"2024-09-10T23:32:41.570330+0000","last_unstale":"2024-09-10T23:32:43.627910+0000","last_undegraded":"2024-09-10T23:32:43.627910+0000","last_fullsized":"2024-09-10T23:32:43.627910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:53.809559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629328+0000","last_change":"2024-09-10T23:32:42.591662+0000","last_active":"2024-09-10T23:32:43.629328+0000","last_peered":"2024-09-10T23:32:43.629328+0000","last_clean":"2024-09-10T23:32:43.629328+0000","last_became_active":"2024-09-10T23:32:42.591315+0000","last_became_peered":"2024-09-10T23:32:42.591315+0000","last_unstale":"2024-09-10T23:32:43.629328+0000","last_undegraded":"2024-09-10T23:32:43.629328+0000","last_fullsized":"2024-09-10T23:32:43.629328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:34.259468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628947+0000","last_change":"2024-09-10T23:32:41.571940+0000","last_active":"2024-09-10T23:32:43.628947+0000","last_peered":"2024-09-10T23:32:43.628947+0000","last_clean":"2024-09-10T23:32:43.628947+0000","last_became_active":"2024-09-10T23:32:41.571537+0000","last_became_peered":"2024-09-10T23:32:41.571537+0000","last_unstale":"2024-09-10T23:32:43.628947+0000","last_undegraded":"2024-09-10T23:32:43.628947+0000","last_fullsized":"2024-09-10T23:32:43.628947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:49.898494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629065+0000","last_change":"2024-09-10T23:32:42.585332+0000","last_active":"2024-09-10T23:32:43.629065+0000","last_peered":"2024-09-10T23:32:43.629065+0000","last_clean":"2024-09-10T23:32:43.629065+0000","last_became_active":"2024-09-10T23:32:42.585046+0000","last_became_peered":"2024-09-10T23:32:42.585046+0000","last_unstale":"2024-09-10T23:32:43.629065+0000","last_undegraded":"2024-09-10T23:32:43.629065+0000","last_fullsized":"2024-09-10T23:32:43.629065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:24.681728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627051+0000","last_change":"2024-09-10T23:32:41.565270+0000","last_active":"2024-09-10T23:32:43.627051+0000","last_peered":"2024-09-10T23:32:43.627051+0000","last_clean":"2024-09-10T23:32:43.627051+0000","last_became_active":"2024-09-10T23:32:41.564864+0000","last_became_peered":"2024-09-10T23:32:41.564864+0000","last_unstale":"2024-09-10T23:32:43.627051+0000","last_undegraded":"2024-09-10T23:32:43.627051+0000","last_fullsized":"2024-09-10T23:32:43.627051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:55.242033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627514+0000","last_change":"2024-09-10T23:32:42.576998+0000","last_active":"2024-09-10T23:32:43.627514+0000","last_peered":"2024-09-10T23:32:43.627514+0000","last_clean":"2024-09-10T23:32:43.627514+0000","last_became_active":"2024-09-10T23:32:42.576695+0000","last_became_peered":"2024-09-10T23:32:42.576695+0000","last_unstale":"2024-09-10T23:32:43.627514+0000","last_undegraded":"2024-09-10T23:32:43.627514+0000","last_fullsized":"2024-09-10T23:32:43.627514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:09.931603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628218+0000","last_change":"2024-09-10T23:32:41.571048+0000","last_active":"2024-09-10T23:32:43.628218+0000","last_peered":"2024-09-10T23:32:43.628218+0000","last_clean":"2024-09-10T23:32:43.628218+0000","last_became_active":"2024-09-10T23:32:41.570753+0000","last_became_peered":"2024-09-10T23:32:41.570753+0000","last_unstale":"2024-09-10T23:32:43.628218+0000","last_undegraded":"2024-09-10T23:32:43.628218+0000","last_fullsized":"2024-09-10T23:32:43.628218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:55.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570370+0000","last_change":"2024-09-10T23:32:42.564300+0000","last_active":"2024-09-10T23:32:43.570370+0000","last_peered":"2024-09-10T23:32:43.570370+0000","last_clean":"2024-09-10T23:32:43.570370+0000","last_became_active":"2024-09-10T23:32:42.564047+0000","last_became_peered":"2024-09-10T23:32:42.564047+0000","last_unstale":"2024-09-10T23:32:43.570370+0000","last_undegraded":"2024-09-10T23:32:43.570370+0000","last_fullsized":"2024-09-10T23:32:43.570370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:22.724821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628645+0000","last_change":"2024-09-10T23:32:41.585012+0000","last_active":"2024-09-10T23:32:43.628645+0000","last_peered":"2024-09-10T23:32:43.628645+0000","last_clean":"2024-09-10T23:32:43.628645+0000","last_became_active":"2024-09-10T23:32:41.584732+0000","last_became_peered":"2024-09-10T23:32:41.584732+0000","last_unstale":"2024-09-10T23:32:43.628645+0000","last_undegraded":"2024-09-10T23:32:43.628645+0000","last_fullsized":"2024-09-10T23:32:43.628645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:39.041511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628794+0000","last_change":"2024-09-10T23:32:42.569000+0000","last_active":"2024-09-10T23:32:43.628794+0000","last_peered":"2024-09-10T23:32:43.628794+0000","last_clean":"2024-09-10T23:32:43.628794+0000","last_became_active":"2024-09-10T23:32:42.568721+0000","last_became_peered":"2024-09-10T23:32:42.568721+0000","last_unstale":"2024-09-10T23:32:43.628794+0000","last_undegraded":"2024-09-10T23:32:43.628794+0000","last_fullsized":"2024-09-10T23:32:43.628794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:37.891368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626833+0000","last_change":"2024-09-10T23:32:41.578661+0000","last_active":"2024-09-10T23:32:43.626833+0000","last_peered":"2024-09-10T23:32:43.626833+0000","last_clean":"2024-09-10T23:32:43.626833+0000","last_became_active":"2024-09-10T23:32:41.578343+0000","last_became_peered":"2024-09-10T23:32:41.578343+0000","last_unstale":"2024-09-10T23:32:43.626833+0000","last_undegraded":"2024-09-10T23:32:43.626833+0000","last_fullsized":"2024-09-10T23:32:43.626833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:47.815432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563134+0000","last_change":"2024-09-10T23:32:42.584934+0000","last_active":"2024-09-10T23:32:43.563134+0000","last_peered":"2024-09-10T23:32:43.563134+0000","last_clean":"2024-09-10T23:32:43.563134+0000","last_became_active":"2024-09-10T23:32:42.584709+0000","last_became_peered":"2024-09-10T23:32:42.584709+0000","last_unstale":"2024-09-10T23:32:43.563134+0000","last_undegraded":"2024-09-10T23:32:43.563134+0000","last_fullsized":"2024-09-10T23:32:43.563134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:30.543243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.568880+0000","last_change":"2024-09-10T23:32:41.566144+0000","last_active":"2024-09-10T23:32:43.568880+0000","last_peered":"2024-09-10T23:32:43.568880+0000","last_clean":"2024-09-10T23:32:43.568880+0000","last_became_active":"2024-09-10T23:32:41.565841+0000","last_became_peered":"2024-09-10T23:32:41.565841+0000","last_unstale":"2024-09-10T23:32:43.568880+0000","last_undegraded":"2024-09-10T23:32:43.568880+0000","last_fullsized":"2024-09-10T23:32:43.568880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:51.080022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628125+0000","last_change":"2024-09-10T23:32:42.586470+0000","last_active":"2024-09-10T23:32:43.628125+0000","last_peered":"2024-09-10T23:32:43.628125+0000","last_clean":"2024-09-10T23:32:43.628125+0000","last_became_active":"2024-09-10T23:32:42.579377+0000","last_became_peered":"2024-09-10T23:32:42.579377+0000","last_unstale":"2024-09-10T23:32:43.628125+0000","last_undegraded":"2024-09-10T23:32:43.628125+0000","last_fullsized":"2024-09-10T23:32:43.628125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:47.112551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628061+0000","last_change":"2024-09-10T23:32:41.567212+0000","last_active":"2024-09-10T23:32:43.628061+0000","last_peered":"2024-09-10T23:32:43.628061+0000","last_clean":"2024-09-10T23:32:43.628061+0000","last_became_active":"2024-09-10T23:32:41.566857+0000","last_became_peered":"2024-09-10T23:32:41.566857+0000","last_unstale":"2024-09-10T23:32:43.628061+0000","last_undegraded":"2024-09-10T23:32:43.628061+0000","last_fullsized":"2024-09-10T23:32:43.628061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:05.259433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627653+0000","last_change":"2024-09-10T23:32:42.558295+0000","last_active":"2024-09-10T23:32:43.627653+0000","last_peered":"2024-09-10T23:32:43.627653+0000","last_clean":"2024-09-10T23:32:43.627653+0000","last_became_active":"2024-09-10T23:32:42.558039+0000","last_became_peered":"2024-09-10T23:32:42.558039+0000","last_unstale":"2024-09-10T23:32:43.627653+0000","last_undegraded":"2024-09-10T23:32:43.627653+0000","last_fullsized":"2024-09-10T23:32:43.627653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:07.033834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569932+0000","last_change":"2024-09-10T23:32:41.568016+0000","last_active":"2024-09-10T23:32:43.569932+0000","last_peered":"2024-09-10T23:32:43.569932+0000","last_clean":"2024-09-10T23:32:43.569932+0000","last_became_active":"2024-09-10T23:32:41.566960+0000","last_became_peered":"2024-09-10T23:32:41.566960+0000","last_unstale":"2024-09-10T23:32:43.569932+0000","last_undegraded":"2024-09-10T23:32:43.569932+0000","last_fullsized":"2024-09-10T23:32:43.569932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:14.145496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628571+0000","last_change":"2024-09-10T23:32:42.562392+0000","last_active":"2024-09-10T23:32:43.628571+0000","last_peered":"2024-09-10T23:32:43.628571+0000","last_clean":"2024-09-10T23:32:43.628571+0000","last_became_active":"2024-09-10T23:32:42.562042+0000","last_became_peered":"2024-09-10T23:32:42.562042+0000","last_unstale":"2024-09-10T23:32:43.628571+0000","last_undegraded":"2024-09-10T23:32:43.628571+0000","last_fullsized":"2024-09-10T23:32:43.628571+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:33.100231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628521+0000","last_change":"2024-09-10T23:32:41.572986+0000","last_active":"2024-09-10T23:32:43.628521+0000","last_peered":"2024-09-10T23:32:43.628521+0000","last_clean":"2024-09-10T23:32:43.628521+0000","last_became_active":"2024-09-10T23:32:41.572722+0000","last_became_peered":"2024-09-10T23:32:41.572722+0000","last_unstale":"2024-09-10T23:32:43.628521+0000","last_undegraded":"2024-09-10T23:32:43.628521+0000","last_fullsized":"2024-09-10T23:32:43.628521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:49.898494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628742+0000","last_change":"2024-09-10T23:32:42.573643+0000","last_active":"2024-09-10T23:32:43.628742+0000","last_peered":"2024-09-10T23:32:43.628742+0000","last_clean":"2024-09-10T23:32:43.628742+0000","last_became_active":"2024-09-10T23:32:42.572873+0000","last_became_peered":"2024-09-10T23:32:42.572873+0000","last_unstale":"2024-09-10T23:32:43.628742+0000","last_undegraded":"2024-09-10T23:32:43.628742+0000","last_fullsized":"2024-09-10T23:32:43.628742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:14.468774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570459+0000","last_change":"2024-09-10T23:32:41.562365+0000","last_active":"2024-09-10T23:32:43.570459+0000","last_peered":"2024-09-10T23:32:43.570459+0000","last_clean":"2024-09-10T23:32:43.570459+0000","last_became_active":"2024-09-10T23:32:41.562021+0000","last_became_peered":"2024-09-10T23:32:41.562021+0000","last_unstale":"2024-09-10T23:32:43.570459+0000","last_undegraded":"2024-09-10T23:32:43.570459+0000","last_fullsized":"2024-09-10T23:32:43.570459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:08:28.878205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628732+0000","last_change":"2024-09-10T23:32:42.584713+0000","last_active":"2024-09-10T23:32:43.628732+0000","last_peered":"2024-09-10T23:32:43.628732+0000","last_clean":"2024-09-10T23:32:43.628732+0000","last_became_active":"2024-09-10T23:32:42.577789+0000","last_became_peered":"2024-09-10T23:32:42.577789+0000","last_unstale":"2024-09-10T23:32:43.628732+0000","last_undegraded":"2024-09-10T23:32:43.628732+0000","last_fullsized":"2024-09-10T23:32:43.628732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:33.835681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563126+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:32:43.563126+0000","last_peered":"2024-09-10T23:32:43.563126+0000","last_clean":"2024-09-10T23:32:43.563126+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:32:43.563126+0000","last_undegraded":"2024-09-10T23:32:43.563126+0000","last_fullsized":"2024-09-10T23:32:43.563126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629078+0000","last_change":"2024-09-10T23:32:41.581530+0000","last_active":"2024-09-10T23:32:43.629078+0000","last_peered":"2024-09-10T23:32:43.629078+0000","last_clean":"2024-09-10T23:32:43.629078+0000","last_became_active":"2024-09-10T23:32:41.581242+0000","last_became_peered":"2024-09-10T23:32:41.581242+0000","last_unstale":"2024-09-10T23:32:43.629078+0000","last_undegraded":"2024-09-10T23:32:43.629078+0000","last_fullsized":"2024-09-10T23:32:43.629078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:08.633493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569220+0000","last_change":"2024-09-10T23:32:42.558841+0000","last_active":"2024-09-10T23:32:43.569220+0000","last_peered":"2024-09-10T23:32:43.569220+0000","last_clean":"2024-09-10T23:32:43.569220+0000","last_became_active":"2024-09-10T23:32:42.558580+0000","last_became_peered":"2024-09-10T23:32:42.558580+0000","last_unstale":"2024-09-10T23:32:43.569220+0000","last_undegraded":"2024-09-10T23:32:43.569220+0000","last_fullsized":"2024-09-10T23:32:43.569220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:54.620648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628625+0000","last_change":"2024-09-10T23:32:41.577467+0000","last_active":"2024-09-10T23:32:43.628625+0000","last_peered":"2024-09-10T23:32:43.628625+0000","last_clean":"2024-09-10T23:32:43.628625+0000","last_became_active":"2024-09-10T23:32:41.576702+0000","last_became_peered":"2024-09-10T23:32:41.576702+0000","last_unstale":"2024-09-10T23:32:43.628625+0000","last_undegraded":"2024-09-10T23:32:43.628625+0000","last_fullsized":"2024-09-10T23:32:43.628625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:32.527218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569670+0000","last_change":"2024-09-10T23:32:42.565100+0000","last_active":"2024-09-10T23:32:43.569670+0000","last_peered":"2024-09-10T23:32:43.569670+0000","last_clean":"2024-09-10T23:32:43.569670+0000","last_became_active":"2024-09-10T23:32:42.564790+0000","last_became_peered":"2024-09-10T23:32:42.564790+0000","last_unstale":"2024-09-10T23:32:43.569670+0000","last_undegraded":"2024-09-10T23:32:43.569670+0000","last_fullsized":"2024-09-10T23:32:43.569670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:44.555042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628065+0000","last_change":"2024-09-10T23:32:41.571266+0000","last_active":"2024-09-10T23:32:43.628065+0000","last_peered":"2024-09-10T23:32:43.628065+0000","last_clean":"2024-09-10T23:32:43.628065+0000","last_became_active":"2024-09-10T23:32:41.570986+0000","last_became_peered":"2024-09-10T23:32:41.570986+0000","last_unstale":"2024-09-10T23:32:43.628065+0000","last_undegraded":"2024-09-10T23:32:43.628065+0000","last_fullsized":"2024-09-10T23:32:43.628065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:58.170892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628975+0000","last_change":"2024-09-10T23:32:42.584295+0000","last_active":"2024-09-10T23:32:43.628975+0000","last_peered":"2024-09-10T23:32:43.628975+0000","last_clean":"2024-09-10T23:32:43.628975+0000","last_became_active":"2024-09-10T23:32:42.584001+0000","last_became_peered":"2024-09-10T23:32:42.584001+0000","last_unstale":"2024-09-10T23:32:43.628975+0000","last_undegraded":"2024-09-10T23:32:43.628975+0000","last_fullsized":"2024-09-10T23:32:43.628975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:05.044760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628726+0000","last_change":"2024-09-10T23:32:41.578157+0000","last_active":"2024-09-10T23:32:43.628726+0000","last_peered":"2024-09-10T23:32:43.628726+0000","last_clean":"2024-09-10T23:32:43.628726+0000","last_became_active":"2024-09-10T23:32:41.575442+0000","last_became_peered":"2024-09-10T23:32:41.575442+0000","last_unstale":"2024-09-10T23:32:43.628726+0000","last_undegraded":"2024-09-10T23:32:43.628726+0000","last_fullsized":"2024-09-10T23:32:43.628726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:00.455999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627197+0000","last_change":"2024-09-10T23:32:42.571239+0000","last_active":"2024-09-10T23:32:43.627197+0000","last_peered":"2024-09-10T23:32:43.627197+0000","last_clean":"2024-09-10T23:32:43.627197+0000","last_became_active":"2024-09-10T23:32:42.570963+0000","last_became_peered":"2024-09-10T23:32:42.570963+0000","last_unstale":"2024-09-10T23:32:43.627197+0000","last_undegraded":"2024-09-10T23:32:43.627197+0000","last_fullsized":"2024-09-10T23:32:43.627197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:36.598389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628178+0000","last_change":"2024-09-10T23:32:41.569326+0000","last_active":"2024-09-10T23:32:43.628178+0000","last_peered":"2024-09-10T23:32:43.628178+0000","last_clean":"2024-09-10T23:32:43.628178+0000","last_became_active":"2024-09-10T23:32:41.569056+0000","last_became_peered":"2024-09-10T23:32:41.569056+0000","last_unstale":"2024-09-10T23:32:43.628178+0000","last_undegraded":"2024-09-10T23:32:43.628178+0000","last_fullsized":"2024-09-10T23:32:43.628178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:47.815432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626872+0000","last_change":"2024-09-10T23:32:42.574800+0000","last_active":"2024-09-10T23:32:43.626872+0000","last_peered":"2024-09-10T23:32:43.626872+0000","last_clean":"2024-09-10T23:32:43.626872+0000","last_became_active":"2024-09-10T23:32:42.574477+0000","last_became_peered":"2024-09-10T23:32:42.574477+0000","last_unstale":"2024-09-10T23:32:43.626872+0000","last_undegraded":"2024-09-10T23:32:43.626872+0000","last_fullsized":"2024-09-10T23:32:43.626872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:14.440912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629156+0000","last_change":"2024-09-10T23:32:41.582865+0000","last_active":"2024-09-10T23:32:43.629156+0000","last_peered":"2024-09-10T23:32:43.629156+0000","last_clean":"2024-09-10T23:32:43.629156+0000","last_became_active":"2024-09-10T23:32:41.582522+0000","last_became_peered":"2024-09-10T23:32:41.582522+0000","last_unstale":"2024-09-10T23:32:43.629156+0000","last_undegraded":"2024-09-10T23:32:43.629156+0000","last_fullsized":"2024-09-10T23:32:43.629156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:24.945862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570471+0000","last_change":"2024-09-10T23:32:42.587148+0000","last_active":"2024-09-10T23:32:43.570471+0000","last_peered":"2024-09-10T23:32:43.570471+0000","last_clean":"2024-09-10T23:32:43.570471+0000","last_became_active":"2024-09-10T23:32:42.586887+0000","last_became_peered":"2024-09-10T23:32:42.586887+0000","last_unstale":"2024-09-10T23:32:43.570471+0000","last_undegraded":"2024-09-10T23:32:43.570471+0000","last_fullsized":"2024-09-10T23:32:43.570471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:27.999337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570022+0000","last_change":"2024-09-10T23:32:41.553400+0000","last_active":"2024-09-10T23:32:43.570022+0000","last_peered":"2024-09-10T23:32:43.570022+0000","last_clean":"2024-09-10T23:32:43.570022+0000","last_became_active":"2024-09-10T23:32:41.553070+0000","last_became_peered":"2024-09-10T23:32:41.553070+0000","last_unstale":"2024-09-10T23:32:43.570022+0000","last_undegraded":"2024-09-10T23:32:43.570022+0000","last_fullsized":"2024-09-10T23:32:43.570022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:01.393799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569840+0000","last_change":"2024-09-10T23:32:42.580714+0000","last_active":"2024-09-10T23:32:43.569840+0000","last_peered":"2024-09-10T23:32:43.569840+0000","last_clean":"2024-09-10T23:32:43.569840+0000","last_became_active":"2024-09-10T23:32:42.579027+0000","last_became_peered":"2024-09-10T23:32:42.579027+0000","last_unstale":"2024-09-10T23:32:43.569840+0000","last_undegraded":"2024-09-10T23:32:43.569840+0000","last_fullsized":"2024-09-10T23:32:43.569840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:12.456589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629216+0000","last_change":"2024-09-10T23:32:41.580054+0000","last_active":"2024-09-10T23:32:43.629216+0000","last_peered":"2024-09-10T23:32:43.629216+0000","last_clean":"2024-09-10T23:32:43.629216+0000","last_became_active":"2024-09-10T23:32:41.579663+0000","last_became_peered":"2024-09-10T23:32:41.579663+0000","last_unstale":"2024-09-10T23:32:43.629216+0000","last_undegraded":"2024-09-10T23:32:43.629216+0000","last_fullsized":"2024-09-10T23:32:43.629216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:22.486416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569890+0000","last_change":"2024-09-10T23:32:42.574503+0000","last_active":"2024-09-10T23:32:43.569890+0000","last_peered":"2024-09-10T23:32:43.569890+0000","last_clean":"2024-09-10T23:32:43.569890+0000","last_became_active":"2024-09-10T23:32:42.574229+0000","last_became_peered":"2024-09-10T23:32:42.574229+0000","last_unstale":"2024-09-10T23:32:43.569890+0000","last_undegraded":"2024-09-10T23:32:43.569890+0000","last_fullsized":"2024-09-10T23:32:43.569890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:19.891408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.626574+0000","last_change":"2024-09-10T23:32:41.562000+0000","last_active":"2024-09-10T23:32:43.626574+0000","last_peered":"2024-09-10T23:32:43.626574+0000","last_clean":"2024-09-10T23:32:43.626574+0000","last_became_active":"2024-09-10T23:32:41.561637+0000","last_became_peered":"2024-09-10T23:32:41.561637+0000","last_unstale":"2024-09-10T23:32:43.626574+0000","last_undegraded":"2024-09-10T23:32:43.626574+0000","last_fullsized":"2024-09-10T23:32:43.626574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:17.031431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628329+0000","last_change":"2024-09-10T23:32:42.583205+0000","last_active":"2024-09-10T23:32:43.628329+0000","last_peered":"2024-09-10T23:32:43.628329+0000","last_clean":"2024-09-10T23:32:43.628329+0000","last_became_active":"2024-09-10T23:32:42.582963+0000","last_became_peered":"2024-09-10T23:32:42.582963+0000","last_unstale":"2024-09-10T23:32:43.628329+0000","last_undegraded":"2024-09-10T23:32:43.628329+0000","last_fullsized":"2024-09-10T23:32:43.628329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:57.117710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569104+0000","last_change":"2024-09-10T23:32:41.557934+0000","last_active":"2024-09-10T23:32:43.569104+0000","last_peered":"2024-09-10T23:32:43.569104+0000","last_clean":"2024-09-10T23:32:43.569104+0000","last_became_active":"2024-09-10T23:32:41.557625+0000","last_became_peered":"2024-09-10T23:32:41.557625+0000","last_unstale":"2024-09-10T23:32:43.569104+0000","last_undegraded":"2024-09-10T23:32:43.569104+0000","last_fullsized":"2024-09-10T23:32:43.569104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:29.536215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569080+0000","last_change":"2024-09-10T23:32:42.567250+0000","last_active":"2024-09-10T23:32:43.569080+0000","last_peered":"2024-09-10T23:32:43.569080+0000","last_clean":"2024-09-10T23:32:43.569080+0000","last_became_active":"2024-09-10T23:32:42.566248+0000","last_became_peered":"2024-09-10T23:32:42.566248+0000","last_unstale":"2024-09-10T23:32:43.569080+0000","last_undegraded":"2024-09-10T23:32:43.569080+0000","last_fullsized":"2024-09-10T23:32:43.569080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:45.824956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.570495+0000","last_change":"2024-09-10T23:32:41.566640+0000","last_active":"2024-09-10T23:32:43.570495+0000","last_peered":"2024-09-10T23:32:43.570495+0000","last_clean":"2024-09-10T23:32:43.570495+0000","last_became_active":"2024-09-10T23:32:41.566311+0000","last_became_peered":"2024-09-10T23:32:41.566311+0000","last_unstale":"2024-09-10T23:32:43.570495+0000","last_undegraded":"2024-09-10T23:32:43.570495+0000","last_fullsized":"2024-09-10T23:32:43.570495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:33.372580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628173+0000","last_change":"2024-09-10T23:32:42.580760+0000","last_active":"2024-09-10T23:32:43.628173+0000","last_peered":"2024-09-10T23:32:43.628173+0000","last_clean":"2024-09-10T23:32:43.628173+0000","last_became_active":"2024-09-10T23:32:42.580081+0000","last_became_peered":"2024-09-10T23:32:42.580081+0000","last_unstale":"2024-09-10T23:32:43.628173+0000","last_undegraded":"2024-09-10T23:32:43.628173+0000","last_fullsized":"2024-09-10T23:32:43.628173+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:06.266461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628952+0000","last_change":"2024-09-10T23:32:41.583976+0000","last_active":"2024-09-10T23:32:43.628952+0000","last_peered":"2024-09-10T23:32:43.628952+0000","last_clean":"2024-09-10T23:32:43.628952+0000","last_became_active":"2024-09-10T23:32:41.583639+0000","last_became_peered":"2024-09-10T23:32:41.583639+0000","last_unstale":"2024-09-10T23:32:43.628952+0000","last_undegraded":"2024-09-10T23:32:43.628952+0000","last_fullsized":"2024-09-10T23:32:43.628952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:26.669949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629220+0000","last_change":"2024-09-10T23:32:42.570056+0000","last_active":"2024-09-10T23:32:43.629220+0000","last_peered":"2024-09-10T23:32:43.629220+0000","last_clean":"2024-09-10T23:32:43.629220+0000","last_became_active":"2024-09-10T23:32:42.569794+0000","last_became_peered":"2024-09-10T23:32:42.569794+0000","last_unstale":"2024-09-10T23:32:43.629220+0000","last_undegraded":"2024-09-10T23:32:43.629220+0000","last_fullsized":"2024-09-10T23:32:43.629220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:36.453126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628526+0000","last_change":"2024-09-10T23:32:42.585151+0000","last_active":"2024-09-10T23:32:43.628526+0000","last_peered":"2024-09-10T23:32:43.628526+0000","last_clean":"2024-09-10T23:32:43.628526+0000","last_became_active":"2024-09-10T23:32:42.578092+0000","last_became_peered":"2024-09-10T23:32:42.578092+0000","last_unstale":"2024-09-10T23:32:43.628526+0000","last_undegraded":"2024-09-10T23:32:43.628526+0000","last_fullsized":"2024-09-10T23:32:43.628526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:39:56.735321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.628891+0000","last_change":"2024-09-10T23:32:41.571659+0000","last_active":"2024-09-10T23:32:43.628891+0000","last_peered":"2024-09-10T23:32:43.628891+0000","last_clean":"2024-09-10T23:32:43.628891+0000","last_became_active":"2024-09-10T23:32:41.571418+0000","last_became_peered":"2024-09-10T23:32:41.571418+0000","last_unstale":"2024-09-10T23:32:43.628891+0000","last_undegraded":"2024-09-10T23:32:43.628891+0000","last_fullsized":"2024-09-10T23:32:43.628891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:57.665825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.563423+0000","last_change":"2024-09-10T23:32:42.563064+0000","last_active":"2024-09-10T23:32:43.563423+0000","last_peered":"2024-09-10T23:32:43.563423+0000","last_clean":"2024-09-10T23:32:43.563423+0000","last_became_active":"2024-09-10T23:32:42.562808+0000","last_became_peered":"2024-09-10T23:32:42.562808+0000","last_unstale":"2024-09-10T23:32:43.563423+0000","last_undegraded":"2024-09-10T23:32:43.563423+0000","last_fullsized":"2024-09-10T23:32:43.563423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:20.388860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.627661+0000","last_change":"2024-09-10T23:32:41.570670+0000","last_active":"2024-09-10T23:32:43.627661+0000","last_peered":"2024-09-10T23:32:43.627661+0000","last_clean":"2024-09-10T23:32:43.627661+0000","last_became_active":"2024-09-10T23:32:41.570291+0000","last_became_peered":"2024-09-10T23:32:41.570291+0000","last_unstale":"2024-09-10T23:32:43.627661+0000","last_undegraded":"2024-09-10T23:32:43.627661+0000","last_fullsized":"2024-09-10T23:32:43.627661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:33.183319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629201+0000","last_change":"2024-09-10T23:32:42.584736+0000","last_active":"2024-09-10T23:32:43.629201+0000","last_peered":"2024-09-10T23:32:43.629201+0000","last_clean":"2024-09-10T23:32:43.629201+0000","last_became_active":"2024-09-10T23:32:42.583535+0000","last_became_peered":"2024-09-10T23:32:42.583535+0000","last_unstale":"2024-09-10T23:32:43.629201+0000","last_undegraded":"2024-09-10T23:32:43.629201+0000","last_fullsized":"2024-09-10T23:32:43.629201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:00.689816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629197+0000","last_change":"2024-09-10T23:32:41.566445+0000","last_active":"2024-09-10T23:32:43.629197+0000","last_peered":"2024-09-10T23:32:43.629197+0000","last_clean":"2024-09-10T23:32:43.629197+0000","last_became_active":"2024-09-10T23:32:41.566158+0000","last_became_peered":"2024-09-10T23:32:41.566158+0000","last_unstale":"2024-09-10T23:32:43.629197+0000","last_undegraded":"2024-09-10T23:32:43.629197+0000","last_fullsized":"2024-09-10T23:32:43.629197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:55.101886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.629299+0000","last_change":"2024-09-10T23:32:42.584147+0000","last_active":"2024-09-10T23:32:43.629299+0000","last_peered":"2024-09-10T23:32:43.629299+0000","last_clean":"2024-09-10T23:32:43.629299+0000","last_became_active":"2024-09-10T23:32:42.583766+0000","last_became_peered":"2024-09-10T23:32:42.583766+0000","last_unstale":"2024-09-10T23:32:43.629299+0000","last_undegraded":"2024-09-10T23:32:43.629299+0000","last_fullsized":"2024-09-10T23:32:43.629299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:41.536785+0000","last_clean_scrub_stamp":"2024-09-10T23:32:41.536785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:19.288688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-10T23:32:43.569913+0000","last_change":"2024-09-10T23:32:41.580965+0000","last_active":"2024-09-10T23:32:43.569913+0000","last_peered":"2024-09-10T23:32:43.569913+0000","last_clean":"2024-09-10T23:32:43.569913+0000","last_became_active":"2024-09-10T23:32:41.580712+0000","last_became_peered":"2024-09-10T23:32:41.580712+0000","last_unstale":"2024-09-10T23:32:43.569913+0000","last_undegraded":"2024-09-10T23:32:43.569913+0000","last_fullsized":"2024-09-10T23:32:43.569913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:40.529757+0000","last_clean_scrub_stamp":"2024-09-10T23:32:40.529757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:13.433884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:32:57.740 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T23:32:57.740 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T23:32:57.740 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-10T23:32:57.740 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T23:32:57.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.913+0000 7fb3a4800640 1 -- 172.21.15.94:0/2870735764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c15f5c0 msgr2=0x7fb39c0cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:57.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.913+0000 7fb3a4800640 1 --2- 172.21.15.94:0/2870735764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c15f5c0 0x7fb39c0cfb60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb390009900 tx=0x7fb39001c7c0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.913+0000 7fb3a4800640 1 -- 172.21.15.94:0/2870735764 shutdown_connections 2024-09-10T23:32:57.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.913+0000 7fb3a4800640 1 --2- 172.21.15.94:0/2870735764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb39c0b2bb0 0x7fb39c0d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.913+0000 7fb3a4800640 1 --2- 172.21.15.94:0/2870735764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c15f5c0 0x7fb39c0cfb60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.913+0000 7fb3a4800640 1 -- 172.21.15.94:0/2870735764 >> 172.21.15.94:0/2870735764 conn(0x7fb39c0d1c60 msgr2=0x7fb39c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:57.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.913+0000 7fb3a4800640 1 -- 172.21.15.94:0/2870735764 shutdown_connections 2024-09-10T23:32:57.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.913+0000 7fb3a4800640 1 -- 172.21.15.94:0/2870735764 wait complete. 2024-09-10T23:32:57.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a4800640 1 Processor -- start 2024-09-10T23:32:57.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a4800640 1 -- start start 2024-09-10T23:32:57.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a4800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c0b2bb0 0x7fb39c1500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:57.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a4800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb39c0d05e0 0x7fb39c1fb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:57.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a4800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb39c15f5c0 0x7fb39c1fbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:57.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a4800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c169340 con 0x7fb39c0b2bb0 2024-09-10T23:32:57.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a4800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c1691c0 con 0x7fb39c0d05e0 2024-09-10T23:32:57.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a4800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb39c1694c0 con 0x7fb39c15f5c0 2024-09-10T23:32:57.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c0b2bb0 0x7fb39c1500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:57.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c0b2bb0 0x7fb39c1500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40378/0 (socket says 172.21.15.94:40378) 2024-09-10T23:32:57.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a1e00640 1 -- 172.21.15.94:0/3398105813 learned_addr learned my addr 172.21.15.94:0/3398105813 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:57.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a1400640 1 --2- 172.21.15.94:0/3398105813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb39c0d05e0 0x7fb39c1fb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:57.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a2800640 1 --2- 172.21.15.94:0/3398105813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb39c15f5c0 0x7fb39c1fbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:57.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a1e00640 1 -- 172.21.15.94:0/3398105813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb39c15f5c0 msgr2=0x7fb39c1fbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:57.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a1e00640 1 --2- 172.21.15.94:0/3398105813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb39c15f5c0 0x7fb39c1fbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a1e00640 1 -- 172.21.15.94:0/3398105813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb39c0d05e0 msgr2=0x7fb39c1fb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:57.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a1e00640 1 --2- 172.21.15.94:0/3398105813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb39c0d05e0 0x7fb39c1fb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:57.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a1e00640 1 -- 172.21.15.94:0/3398105813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39c15d0b0 con 0x7fb39c0b2bb0 2024-09-10T23:32:57.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a1400640 1 --2- 172.21.15.94:0/3398105813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb39c0d05e0 0x7fb39c1fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:32:57.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a2800640 1 --2- 172.21.15.94:0/3398105813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb39c15f5c0 0x7fb39c1fbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:32:57.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a1e00640 1 --2- 172.21.15.94:0/3398105813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c0b2bb0 0x7fb39c1500b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb390009850 tx=0x7fb3900a5590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:57.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb396000640 1 -- 172.21.15.94:0/3398105813 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3900bb070 con 0x7fb39c0b2bb0 2024-09-10T23:32:57.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a4800640 1 -- 172.21.15.94:0/3398105813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39c15d390 con 0x7fb39c0b2bb0 2024-09-10T23:32:57.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb396000640 1 -- 172.21.15.94:0/3398105813 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3900a6c40 con 0x7fb39c0b2bb0 2024-09-10T23:32:57.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb396000640 1 -- 172.21.15.94:0/3398105813 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3900b2040 con 0x7fb39c0b2bb0 2024-09-10T23:32:57.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a4800640 1 -- 172.21.15.94:0/3398105813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb39c153330 con 0x7fb39c0b2bb0 2024-09-10T23:32:57.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.917+0000 7fb3a4800640 1 -- 172.21.15.94:0/3398105813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb39c0b15c0 con 0x7fb39c0b2bb0 2024-09-10T23:32:57.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.921+0000 7fb396000640 1 -- 172.21.15.94:0/3398105813 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb390003600 con 0x7fb39c0b2bb0 2024-09-10T23:32:57.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.921+0000 7fb396000640 1 --2- 172.21.15.94:0/3398105813 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3800725d0 0x7fb380074a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:57.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.921+0000 7fb396000640 1 -- 172.21.15.94:0/3398105813 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb39012cc90 con 0x7fb39c0b2bb0 2024-09-10T23:32:57.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.921+0000 7fb3a1400640 1 --2- 172.21.15.94:0/3398105813 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3800725d0 0x7fb380074a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:57.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.921+0000 7fb3a1400640 1 --2- 172.21.15.94:0/3398105813 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3800725d0 0x7fb380074a90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb38c005fd0 tx=0x7fb38c005dc0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:57.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:57.925+0000 7fb396000640 1 -- 172.21.15.94:0/3398105813 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb3900164d0 con 0x7fb39c0b2bb0 2024-09-10T23:32:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.121+0000 7fb3a4800640 1 -- 172.21.15.94:0/3398105813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb39c0afe90 con 0x7fb39c0b2bb0 2024-09-10T23:32:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.121+0000 7fb396000640 1 -- 172.21.15.94:0/3398105813 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb3900f8310 con 0x7fb39c0b2bb0 2024-09-10T23:32:58.125 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:58.125 INFO:teuthology.orchestra.run.smithi094.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T23:32:58.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.125+0000 7fb3a4800640 1 -- 172.21.15.94:0/3398105813 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3800725d0 msgr2=0x7fb380074a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.125+0000 7fb3a4800640 1 --2- 172.21.15.94:0/3398105813 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3800725d0 0x7fb380074a90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb38c005fd0 tx=0x7fb38c005dc0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.125+0000 7fb3a4800640 1 -- 172.21.15.94:0/3398105813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c0b2bb0 msgr2=0x7fb39c1500b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.125+0000 7fb3a4800640 1 --2- 172.21.15.94:0/3398105813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c0b2bb0 0x7fb39c1500b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb390009850 tx=0x7fb3900a5590 comp rx=0 tx=0).stop 2024-09-10T23:32:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.125+0000 7fb3a4800640 1 -- 172.21.15.94:0/3398105813 shutdown_connections 2024-09-10T23:32:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.125+0000 7fb3a4800640 1 --2- 172.21.15.94:0/3398105813 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3800725d0 0x7fb380074a90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.125+0000 7fb3a4800640 1 --2- 172.21.15.94:0/3398105813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb39c0d05e0 0x7fb39c1fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.125+0000 7fb3a4800640 1 --2- 172.21.15.94:0/3398105813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb39c15f5c0 0x7fb39c1fbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.125+0000 7fb3a4800640 1 --2- 172.21.15.94:0/3398105813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c0b2bb0 0x7fb39c1500b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.125+0000 7fb3a4800640 1 -- 172.21.15.94:0/3398105813 >> 172.21.15.94:0/3398105813 conn(0x7fb39c0d1c60 msgr2=0x7fb39c0ae900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.125+0000 7fb3a4800640 1 -- 172.21.15.94:0/3398105813 shutdown_connections 2024-09-10T23:32:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.125+0000 7fb3a4800640 1 -- 172.21.15.94:0/3398105813 wait complete. 2024-09-10T23:32:58.147 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-10T23:32:58.147 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:32:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.285+0000 7f7288c00640 1 -- 172.21.15.94:0/3522521788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72840b2bb0 msgr2=0x7f72840d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.285+0000 7f7288c00640 1 --2- 172.21.15.94:0/3522521788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72840b2bb0 0x7f72840d0270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7270009840 tx=0x7f727001c700 comp rx=0 tx=0).stop 2024-09-10T23:32:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.285+0000 7f7288c00640 1 -- 172.21.15.94:0/3522521788 shutdown_connections 2024-09-10T23:32:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.285+0000 7f7288c00640 1 --2- 172.21.15.94:0/3522521788 >> v2:172.21.15.181:3300/0 conn(0x7f72840af310 0x7f72840af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.285+0000 7f7288c00640 1 --2- 172.21.15.94:0/3522521788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72840b2bb0 0x7f72840d0270 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.285+0000 7f7288c00640 1 -- 172.21.15.94:0/3522521788 >> 172.21.15.94:0/3522521788 conn(0x7f72840d1c60 msgr2=0x7f72840d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.285+0000 7f7288c00640 1 -- 172.21.15.94:0/3522521788 shutdown_connections 2024-09-10T23:32:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.285+0000 7f7288c00640 1 -- 172.21.15.94:0/3522521788 wait complete. 2024-09-10T23:32:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7288c00640 1 Processor -- start 2024-09-10T23:32:58.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7288c00640 1 -- start start 2024-09-10T23:32:58.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7288c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72840af310 0x7f7284151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:58.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7288c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72840b2bb0 0x7f72840c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:58.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7288c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f728415f5c0 0x7f72840cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:58.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7288c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7284169000 con 0x7f72840af310 2024-09-10T23:32:58.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7288c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7284168e80 con 0x7f72840b2bb0 2024-09-10T23:32:58.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7288c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f7284169180 con 0x7f728415f5c0 2024-09-10T23:32:58.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7281e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72840af310 0x7f7284151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:58.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7281e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72840af310 0x7f7284151b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40386/0 (socket says 172.21.15.94:40386) 2024-09-10T23:32:58.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7281e00640 1 -- 172.21.15.94:0/486801686 learned_addr learned my addr 172.21.15.94:0/486801686 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:58.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7281400640 1 --2- 172.21.15.94:0/486801686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72840b2bb0 0x7f72840c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:58.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7282800640 1 --2- 172.21.15.94:0/486801686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f728415f5c0 0x7f72840cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7281e00640 1 -- 172.21.15.94:0/486801686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f728415f5c0 msgr2=0x7f72840cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7281e00640 1 --2- 172.21.15.94:0/486801686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f728415f5c0 0x7f72840cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7281e00640 1 -- 172.21.15.94:0/486801686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72840b2bb0 msgr2=0x7f72840c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7281e00640 1 --2- 172.21.15.94:0/486801686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72840b2bb0 0x7f72840c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7281e00640 1 -- 172.21.15.94:0/486801686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72840b4a90 con 0x7f72840af310 2024-09-10T23:32:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7282800640 1 --2- 172.21.15.94:0/486801686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f728415f5c0 0x7f72840cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7281e00640 1 --2- 172.21.15.94:0/486801686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72840af310 0x7f7284151b80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f727400c3c0 tx=0x7f727400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f727a000640 1 -- 172.21.15.94:0/486801686 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274011790 con 0x7f72840af310 2024-09-10T23:32:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f727a000640 1 -- 172.21.15.94:0/486801686 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7274017070 con 0x7f72840af310 2024-09-10T23:32:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7288c00640 1 -- 172.21.15.94:0/486801686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7284152620 con 0x7f72840af310 2024-09-10T23:32:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f727a000640 1 -- 172.21.15.94:0/486801686 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274011ce0 con 0x7f72840af310 2024-09-10T23:32:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7288c00640 1 -- 172.21.15.94:0/486801686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f728414eac0 con 0x7f72840af310 2024-09-10T23:32:58.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.289+0000 7f7288c00640 1 -- 172.21.15.94:0/486801686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72840af7c0 con 0x7f72840af310 2024-09-10T23:32:58.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.293+0000 7f727a000640 1 -- 172.21.15.94:0/486801686 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f727401c460 con 0x7f72840af310 2024-09-10T23:32:58.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.293+0000 7f727a000640 1 --2- 172.21.15.94:0/486801686 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f72640725e0 0x7f7264074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:58.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.293+0000 7f727a000640 1 -- 172.21.15.94:0/486801686 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7274093070 con 0x7f72840af310 2024-09-10T23:32:58.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.293+0000 7f7281400640 1 --2- 172.21.15.94:0/486801686 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f72640725e0 0x7f7264074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:58.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.293+0000 7f7281400640 1 --2- 172.21.15.94:0/486801686 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f72640725e0 0x7f7264074aa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f72700af000 tx=0x7f727001af40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:58.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.297+0000 7f727a000640 1 -- 172.21.15.94:0/486801686 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f727405e6b0 con 0x7f72840af310 2024-09-10T23:32:58.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.493+0000 7f7288c00640 1 -- 172.21.15.94:0/486801686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72840749c0 con 0x7f72840af310 2024-09-10T23:32:58.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.493+0000 7f727a000640 1 -- 172.21.15.94:0/486801686 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5233 (secure 0 0 0) 0x7f727405dfb0 con 0x7f72840af310 2024-09-10T23:32:58.498 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:58.498 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"btime":"2024-09-10T23:32:45:573319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/2278166692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2278166692},{"type":"v1","addr":"172.21.15.94:6834","nonce":2278166692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392173872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3392173872},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392173872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4380,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1894503548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1894503548},{"type":"v1","addr":"172.21.15.181:6837","nonce":1894503548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:32:41.948908+0000","modified":"2024-09-10T23:32:44.956870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194174},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194174":{"gid":194174,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1418253171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1418253171},{"type":"v1","addr":"172.21.15.181:6836","nonce":1418253171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194174,"qdb_cluster":[194174]},"id":1}]} 2024-09-10T23:32:58.498 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-09-10T23:32:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.497+0000 7f7288c00640 1 -- 172.21.15.94:0/486801686 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f72640725e0 msgr2=0x7f7264074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.497+0000 7f7288c00640 1 --2- 172.21.15.94:0/486801686 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f72640725e0 0x7f7264074aa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f72700af000 tx=0x7f727001af40 comp rx=0 tx=0).stop 2024-09-10T23:32:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.497+0000 7f7288c00640 1 -- 172.21.15.94:0/486801686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72840af310 msgr2=0x7f7284151b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.497+0000 7f7288c00640 1 --2- 172.21.15.94:0/486801686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72840af310 0x7f7284151b80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f727400c3c0 tx=0x7f727400c880 comp rx=0 tx=0).stop 2024-09-10T23:32:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.497+0000 7f7288c00640 1 -- 172.21.15.94:0/486801686 shutdown_connections 2024-09-10T23:32:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.497+0000 7f7288c00640 1 --2- 172.21.15.94:0/486801686 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f72640725e0 0x7f7264074aa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.497+0000 7f7288c00640 1 --2- 172.21.15.94:0/486801686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72840b2bb0 0x7f72840c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.497+0000 7f7288c00640 1 --2- 172.21.15.94:0/486801686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f728415f5c0 0x7f72840cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.497+0000 7f7288c00640 1 --2- 172.21.15.94:0/486801686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72840af310 0x7f7284151b80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.497+0000 7f7288c00640 1 -- 172.21.15.94:0/486801686 >> 172.21.15.94:0/486801686 conn(0x7f72840d1c60 msgr2=0x7f7284070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:58.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.497+0000 7f7288c00640 1 -- 172.21.15.94:0/486801686 shutdown_connections 2024-09-10T23:32:58.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.497+0000 7f7288c00640 1 -- 172.21.15.94:0/486801686 wait complete. 2024-09-10T23:32:58.518 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:32:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.661+0000 7f3e17e00640 1 -- 172.21.15.94:0/2373004418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dfc004d50 msgr2=0x7f3dfc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.661+0000 7f3e17e00640 1 --2- 172.21.15.94:0/2373004418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dfc004d50 0x7f3dfc0051d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009ee0 tx=0x7f3e1001d5f0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.661+0000 7f3e17400640 1 -- 172.21.15.94:0/2373004418 reap_dead start 2024-09-10T23:32:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.661+0000 7f3e17e00640 1 -- 172.21.15.94:0/2373004418 shutdown_connections 2024-09-10T23:32:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.661+0000 7f3e17e00640 1 -- 172.21.15.94:0/2373004418 >> 172.21.15.94:0/2373004418 conn(0x7f3e180b2520 msgr2=0x7f3e180b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.661+0000 7f3e17e00640 1 -- 172.21.15.94:0/2373004418 shutdown_connections 2024-09-10T23:32:58.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.661+0000 7f3e17e00640 1 -- 172.21.15.94:0/2373004418 wait complete. 2024-09-10T23:32:58.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e17e00640 1 Processor -- start 2024-09-10T23:32:58.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e17e00640 1 -- start start 2024-09-10T23:32:58.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e17e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dfc004d50 0x7f3e181f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:58.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e17e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e180b5830 0x7f3e181590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:58.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e17e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e180d0c60 0x7f3e180d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:58.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e17e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3e180afc60 con 0x7f3dfc004d50 2024-09-10T23:32:58.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e17e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3e180afae0 con 0x7f3e180d0c60 2024-09-10T23:32:58.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e17e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3e18070660 con 0x7f3e180b5830 2024-09-10T23:32:58.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e16a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dfc004d50 0x7f3e181f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:58.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e16a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dfc004d50 0x7f3e181f7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40402/0 (socket says 172.21.15.94:40402) 2024-09-10T23:32:58.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e16a00640 1 -- 172.21.15.94:0/2838794428 learned_addr learned my addr 172.21.15.94:0/2838794428 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:58.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e17400640 1 --2- 172.21.15.94:0/2838794428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e180d0c60 0x7f3e180d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:58.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e16000640 1 --2- 172.21.15.94:0/2838794428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e180b5830 0x7f3e181590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:58.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e16a00640 1 -- 172.21.15.94:0/2838794428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e180b5830 msgr2=0x7f3e181590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:58.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e16a00640 1 --2- 172.21.15.94:0/2838794428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e180b5830 0x7f3e181590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e16a00640 1 -- 172.21.15.94:0/2838794428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e180d0c60 msgr2=0x7f3e180d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:58.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e16a00640 1 --2- 172.21.15.94:0/2838794428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e180d0c60 0x7f3e180d10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e16a00640 1 -- 172.21.15.94:0/2838794428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e18159630 con 0x7f3dfc004d50 2024-09-10T23:32:58.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e16000640 1 --2- 172.21.15.94:0/2838794428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e180b5830 0x7f3e181590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:32:58.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e16a00640 1 --2- 172.21.15.94:0/2838794428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dfc004d50 0x7f3e181f7f80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3e08003680 tx=0x7f3e08003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:58.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e0ea00640 1 -- 172.21.15.94:0/2838794428 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0800f7a0 con 0x7f3dfc004d50 2024-09-10T23:32:58.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e0ea00640 1 -- 172.21.15.94:0/2838794428 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e08016020 con 0x7f3dfc004d50 2024-09-10T23:32:58.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e0ea00640 1 -- 172.21.15.94:0/2838794428 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0800fc00 con 0x7f3dfc004d50 2024-09-10T23:32:58.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e17e00640 1 -- 172.21.15.94:0/2838794428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e180b5fa0 con 0x7f3dfc004d50 2024-09-10T23:32:58.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e17e00640 1 -- 172.21.15.94:0/2838794428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e180b6560 con 0x7f3dfc004d50 2024-09-10T23:32:58.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.665+0000 7f3e17e00640 1 -- 172.21.15.94:0/2838794428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e180d1620 con 0x7f3dfc004d50 2024-09-10T23:32:58.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.669+0000 7f3e0ea00640 1 -- 172.21.15.94:0/2838794428 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3e08011430 con 0x7f3dfc004d50 2024-09-10T23:32:58.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.669+0000 7f3e0ea00640 1 --2- 172.21.15.94:0/2838794428 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3df8072610 0x7f3df8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:58.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.669+0000 7f3e0ea00640 1 -- 172.21.15.94:0/2838794428 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3e08093d10 con 0x7f3dfc004d50 2024-09-10T23:32:58.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.669+0000 7f3e16000640 1 --2- 172.21.15.94:0/2838794428 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3df8072610 0x7f3df8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:58.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.669+0000 7f3e16000640 1 --2- 172.21.15.94:0/2838794428 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3df8072610 0x7f3df8074ad0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3e180ae000 tx=0x7f3e040091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:58.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.673+0000 7f3e0ea00640 1 -- 172.21.15.94:0/2838794428 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3e08099020 con 0x7f3dfc004d50 2024-09-10T23:32:58.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.873+0000 7f3e17e00640 1 -- 172.21.15.94:0/2838794428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e1815ee20 con 0x7f3dfc004d50 2024-09-10T23:32:58.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.873+0000 7f3e0ea00640 1 -- 172.21.15.94:0/2838794428 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5233 (secure 0 0 0) 0x7f3e08020020 con 0x7f3dfc004d50 2024-09-10T23:32:58.875 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:58.875 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"btime":"2024-09-10T23:32:45:573319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/2278166692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2278166692},{"type":"v1","addr":"172.21.15.94:6834","nonce":2278166692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392173872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3392173872},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392173872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4380,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1894503548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1894503548},{"type":"v1","addr":"172.21.15.181:6837","nonce":1894503548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:32:41.948908+0000","modified":"2024-09-10T23:32:44.956870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194174},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194174":{"gid":194174,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1418253171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1418253171},{"type":"v1","addr":"172.21.15.181:6836","nonce":1418253171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194174,"qdb_cluster":[194174]},"id":1}]} 2024-09-10T23:32:58.875 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-09-10T23:32:58.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.873+0000 7f3e17e00640 1 -- 172.21.15.94:0/2838794428 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3df8072610 msgr2=0x7f3df8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:58.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.873+0000 7f3e17e00640 1 --2- 172.21.15.94:0/2838794428 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3df8072610 0x7f3df8074ad0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3e180ae000 tx=0x7f3e040091e0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.873+0000 7f3e17e00640 1 -- 172.21.15.94:0/2838794428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dfc004d50 msgr2=0x7f3e181f7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:58.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.873+0000 7f3e17e00640 1 --2- 172.21.15.94:0/2838794428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dfc004d50 0x7f3e181f7f80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3e08003680 tx=0x7f3e08003a80 comp rx=0 tx=0).stop 2024-09-10T23:32:58.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.873+0000 7f3e17e00640 1 -- 172.21.15.94:0/2838794428 shutdown_connections 2024-09-10T23:32:58.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.873+0000 7f3e17e00640 1 --2- 172.21.15.94:0/2838794428 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3df8072610 0x7f3df8074ad0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.873+0000 7f3e17e00640 1 --2- 172.21.15.94:0/2838794428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e180d0c60 0x7f3e180d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.873+0000 7f3e17e00640 1 --2- 172.21.15.94:0/2838794428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e180b5830 0x7f3e181590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.873+0000 7f3e17e00640 1 --2- 172.21.15.94:0/2838794428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dfc004d50 0x7f3e181f7f80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:58.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.873+0000 7f3e17e00640 1 -- 172.21.15.94:0/2838794428 >> 172.21.15.94:0/2838794428 conn(0x7f3e180b2520 msgr2=0x7f3e180cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:58.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.873+0000 7f3e17e00640 1 -- 172.21.15.94:0/2838794428 shutdown_connections 2024-09-10T23:32:58.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:58.873+0000 7f3e17e00640 1 -- 172.21.15.94:0/2838794428 wait complete. 2024-09-10T23:32:58.897 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:32:59.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.033+0000 7fc65f400640 1 -- 172.21.15.94:0/2400303004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c004d50 msgr2=0x7fc63c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:59.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.033+0000 7fc65f400640 1 --2- 172.21.15.94:0/2400303004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c004d50 0x7fc63c0051d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc65000c640 tx=0x7fc65001d320 comp rx=0 tx=0).stop 2024-09-10T23:32:59.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.033+0000 7fc65d400640 1 -- 172.21.15.94:0/2400303004 reap_dead start 2024-09-10T23:32:59.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65f400640 1 -- 172.21.15.94:0/2400303004 shutdown_connections 2024-09-10T23:32:59.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65f400640 1 -- 172.21.15.94:0/2400303004 >> 172.21.15.94:0/2400303004 conn(0x7fc6580d1c60 msgr2=0x7fc6580d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:59.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65f400640 1 -- 172.21.15.94:0/2400303004 shutdown_connections 2024-09-10T23:32:59.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65f400640 1 -- 172.21.15.94:0/2400303004 wait complete. 2024-09-10T23:32:59.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65f400640 1 Processor -- start 2024-09-10T23:32:59.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65f400640 1 -- start start 2024-09-10T23:32:59.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c004d50 0x7fc6581f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:59.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6581615e0 0x7fc65815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:59.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65f400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6580b6360 0x7fc6580b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:59.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65f400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc65806e9d0 con 0x7fc63c004d50 2024-09-10T23:32:59.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65f400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc65806e850 con 0x7fc6581615e0 2024-09-10T23:32:59.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65f400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc65806eb50 con 0x7fc6580b6360 2024-09-10T23:32:59.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c004d50 0x7fc6581f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:59.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c004d50 0x7fc6581f87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40418/0 (socket says 172.21.15.94:40418) 2024-09-10T23:32:59.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65ca00640 1 -- 172.21.15.94:0/2428960918 learned_addr learned my addr 172.21.15.94:0/2428960918 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:59.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc657e00640 1 --2- 172.21.15.94:0/2428960918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6581615e0 0x7fc65815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:59.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65d400640 1 --2- 172.21.15.94:0/2428960918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6580b6360 0x7fc6580b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:59.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65ca00640 1 -- 172.21.15.94:0/2428960918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6580b6360 msgr2=0x7fc6580b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:59.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65ca00640 1 --2- 172.21.15.94:0/2428960918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6580b6360 0x7fc6580b67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:59.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65ca00640 1 -- 172.21.15.94:0/2428960918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6581615e0 msgr2=0x7fc65815a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:59.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65ca00640 1 --2- 172.21.15.94:0/2428960918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6581615e0 0x7fc65815a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:59.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65ca00640 1 -- 172.21.15.94:0/2428960918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6580cbbf0 con 0x7fc63c004d50 2024-09-10T23:32:59.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.037+0000 7fc65ca00640 1 --2- 172.21.15.94:0/2428960918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c004d50 0x7fc6581f87f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc648011ae0 tx=0x7fc648009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:59.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.041+0000 7fc654c00640 1 -- 172.21.15.94:0/2428960918 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64801e070 con 0x7fc63c004d50 2024-09-10T23:32:59.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.041+0000 7fc654c00640 1 -- 172.21.15.94:0/2428960918 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc64800fe40 con 0x7fc63c004d50 2024-09-10T23:32:59.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.041+0000 7fc654c00640 1 -- 172.21.15.94:0/2428960918 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6480042b0 con 0x7fc63c004d50 2024-09-10T23:32:59.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.041+0000 7fc65f400640 1 -- 172.21.15.94:0/2428960918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6580cbed0 con 0x7fc63c004d50 2024-09-10T23:32:59.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.041+0000 7fc65f400640 1 -- 172.21.15.94:0/2428960918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6581f82d0 con 0x7fc63c004d50 2024-09-10T23:32:59.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.041+0000 7fc65f400640 1 -- 172.21.15.94:0/2428960918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6580ce7f0 con 0x7fc63c004d50 2024-09-10T23:32:59.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.041+0000 7fc654c00640 1 -- 172.21.15.94:0/2428960918 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc648003600 con 0x7fc63c004d50 2024-09-10T23:32:59.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.041+0000 7fc654c00640 1 --2- 172.21.15.94:0/2428960918 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc638072640 0x7fc638074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:59.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.041+0000 7fc654c00640 1 -- 172.21.15.94:0/2428960918 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc648093fc0 con 0x7fc63c004d50 2024-09-10T23:32:59.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.041+0000 7fc657e00640 1 --2- 172.21.15.94:0/2428960918 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc638072640 0x7fc638074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:59.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.045+0000 7fc657e00640 1 --2- 172.21.15.94:0/2428960918 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc638072640 0x7fc638074b00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc65815d630 tx=0x7fc6440091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:59.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.045+0000 7fc654c00640 1 -- 172.21.15.94:0/2428960918 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc64805f100 con 0x7fc63c004d50 2024-09-10T23:32:59.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.213+0000 7fc65f400640 1 -- 172.21.15.94:0/2428960918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6580cc0a0 con 0x7fc63c004d50 2024-09-10T23:32:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.213+0000 7fc654c00640 1 -- 172.21.15.94:0/2428960918 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16657 (secure 0 0 0) 0x7fc648066e20 con 0x7fc63c004d50 2024-09-10T23:32:59.216 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:59.216 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":22,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:32:43.556132+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:32:39.855046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T23:32:40.904161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:32:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.213+0000 7fc65f400640 1 -- 172.21.15.94:0/2428960918 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc638072640 msgr2=0x7fc638074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.213+0000 7fc65f400640 1 --2- 172.21.15.94:0/2428960918 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc638072640 0x7fc638074b00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc65815d630 tx=0x7fc6440091e0 comp rx=0 tx=0).stop 2024-09-10T23:32:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.213+0000 7fc65f400640 1 -- 172.21.15.94:0/2428960918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c004d50 msgr2=0x7fc6581f87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.213+0000 7fc65f400640 1 --2- 172.21.15.94:0/2428960918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c004d50 0x7fc6581f87f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc648011ae0 tx=0x7fc648009ea0 comp rx=0 tx=0).stop 2024-09-10T23:32:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.213+0000 7fc65f400640 1 -- 172.21.15.94:0/2428960918 shutdown_connections 2024-09-10T23:32:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.213+0000 7fc65f400640 1 --2- 172.21.15.94:0/2428960918 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc638072640 0x7fc638074b00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.213+0000 7fc65f400640 1 --2- 172.21.15.94:0/2428960918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6581615e0 0x7fc65815a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.213+0000 7fc65f400640 1 --2- 172.21.15.94:0/2428960918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6580b6360 0x7fc6580b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:59.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.213+0000 7fc65f400640 1 --2- 172.21.15.94:0/2428960918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc63c004d50 0x7fc6581f87f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:59.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.213+0000 7fc65f400640 1 -- 172.21.15.94:0/2428960918 >> 172.21.15.94:0/2428960918 conn(0x7fc6580d1c60 msgr2=0x7fc6580b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:59.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.213+0000 7fc65f400640 1 -- 172.21.15.94:0/2428960918 shutdown_connections 2024-09-10T23:32:59.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.213+0000 7fc65f400640 1 -- 172.21.15.94:0/2428960918 wait complete. 2024-09-10T23:32:59.237 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:32:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.373+0000 7f621d200640 1 -- 172.21.15.94:0/3471551946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180cfcb0 msgr2=0x7f62180d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.373+0000 7f621d200640 1 --2- 172.21.15.94:0/3471551946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180cfcb0 0x7f62180d00d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f62180cca90 tx=0x7f620401c890 comp rx=0 tx=0).stop 2024-09-10T23:32:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.373+0000 7f621d200640 1 -- 172.21.15.94:0/3471551946 shutdown_connections 2024-09-10T23:32:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.373+0000 7f621d200640 1 --2- 172.21.15.94:0/3471551946 >> v2:172.21.15.181:3300/0 conn(0x7f62180cddb0 0x7f62180ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.373+0000 7f621d200640 1 --2- 172.21.15.94:0/3471551946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180cfcb0 0x7f62180d00d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.373+0000 7f621d200640 1 -- 172.21.15.94:0/3471551946 >> 172.21.15.94:0/3471551946 conn(0x7f621806cdf0 msgr2=0x7f621806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.373+0000 7f621d200640 1 -- 172.21.15.94:0/3471551946 shutdown_connections 2024-09-10T23:32:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.373+0000 7f621d200640 1 -- 172.21.15.94:0/3471551946 wait complete. 2024-09-10T23:32:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f621d200640 1 Processor -- start 2024-09-10T23:32:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f621d200640 1 -- start start 2024-09-10T23:32:59.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f621d200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62180cddb0 0x7f62181460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:59.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f621d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180cfcb0 0x7f621806fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:59.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f621d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621815ba50 0x7f62181faf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:59.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f621d200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6218168be0 con 0x7f62180cfcb0 2024-09-10T23:32:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f621d200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6218168a60 con 0x7f621815ba50 2024-09-10T23:32:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f621d200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6218168d60 con 0x7f62180cddb0 2024-09-10T23:32:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f6215400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180cfcb0 0x7f621806fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f6215400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180cfcb0 0x7f621806fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40432/0 (socket says 172.21.15.94:40432) 2024-09-10T23:32:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f6215400640 1 -- 172.21.15.94:0/1303223497 learned_addr learned my addr 172.21.15.94:0/1303223497 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:32:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f6215e00640 1 --2- 172.21.15.94:0/1303223497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62180cddb0 0x7f62181460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f6216800640 1 --2- 172.21.15.94:0/1303223497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621815ba50 0x7f62181faf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f6215400640 1 -- 172.21.15.94:0/1303223497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62180cddb0 msgr2=0x7f62181460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f6215400640 1 --2- 172.21.15.94:0/1303223497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62180cddb0 0x7f62181460b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f6215400640 1 -- 172.21.15.94:0/1303223497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621815ba50 msgr2=0x7f62181faf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f6215400640 1 --2- 172.21.15.94:0/1303223497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621815ba50 0x7f62181faf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f6215400640 1 -- 172.21.15.94:0/1303223497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621806ede0 con 0x7f62180cfcb0 2024-09-10T23:32:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f6215400640 1 --2- 172.21.15.94:0/1303223497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180cfcb0 0x7f621806fe80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f62040194a0 tx=0x7f62040a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f620e000640 1 -- 172.21.15.94:0/1303223497 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62040b2040 con 0x7f62180cfcb0 2024-09-10T23:32:59.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f620e000640 1 -- 172.21.15.94:0/1303223497 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6204005230 con 0x7f62180cfcb0 2024-09-10T23:32:59.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f620e000640 1 -- 172.21.15.94:0/1303223497 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62040046d0 con 0x7f62180cfcb0 2024-09-10T23:32:59.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f621d200640 1 -- 172.21.15.94:0/1303223497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621806f0c0 con 0x7f62180cfcb0 2024-09-10T23:32:59.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.377+0000 7f621d200640 1 -- 172.21.15.94:0/1303223497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621806f680 con 0x7f62180cfcb0 2024-09-10T23:32:59.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.381+0000 7f621d200640 1 -- 172.21.15.94:0/1303223497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62180ce200 con 0x7f62180cfcb0 2024-09-10T23:32:59.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.381+0000 7f620e000640 1 -- 172.21.15.94:0/1303223497 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6204004870 con 0x7f62180cfcb0 2024-09-10T23:32:59.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.381+0000 7f620e000640 1 --2- 172.21.15.94:0/1303223497 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f61f8072610 0x7f61f8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:32:59.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.381+0000 7f620e000640 1 -- 172.21.15.94:0/1303223497 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f620412d350 con 0x7f62180cfcb0 2024-09-10T23:32:59.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.381+0000 7f6215e00640 1 --2- 172.21.15.94:0/1303223497 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f61f8072610 0x7f61f8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:32:59.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.381+0000 7f6215e00640 1 --2- 172.21.15.94:0/1303223497 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f61f8072610 0x7f61f8074ad0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f621806ecd0 tx=0x7f6208009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:32:59.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.385+0000 7f620e000640 1 -- 172.21.15.94:0/1303223497 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f62040164b0 con 0x7f62180cfcb0 2024-09-10T23:32:59.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.581+0000 7f621d200640 1 -- 172.21.15.94:0/1303223497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6218159b40 con 0x7f62180cfcb0 2024-09-10T23:32:59.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.581+0000 7f620e000640 1 -- 172.21.15.94:0/1303223497 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5233 (secure 0 0 0) 0x7f62040f8a20 con 0x7f62180cfcb0 2024-09-10T23:32:59.586 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:32:59.586 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"btime":"2024-09-10T23:32:45:573319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/2278166692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2278166692},{"type":"v1","addr":"172.21.15.94:6834","nonce":2278166692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392173872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3392173872},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392173872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4380,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1894503548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1894503548},{"type":"v1","addr":"172.21.15.181:6837","nonce":1894503548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:32:41.948908+0000","modified":"2024-09-10T23:32:44.956870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194174},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194174":{"gid":194174,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1418253171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1418253171},{"type":"v1","addr":"172.21.15.181:6836","nonce":1418253171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194174,"qdb_cluster":[194174]},"id":1}]} 2024-09-10T23:32:59.586 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-09-10T23:32:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.585+0000 7f621d200640 1 -- 172.21.15.94:0/1303223497 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f61f8072610 msgr2=0x7f61f8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.585+0000 7f621d200640 1 --2- 172.21.15.94:0/1303223497 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f61f8072610 0x7f61f8074ad0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f621806ecd0 tx=0x7f6208009040 comp rx=0 tx=0).stop 2024-09-10T23:32:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.585+0000 7f621d200640 1 -- 172.21.15.94:0/1303223497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180cfcb0 msgr2=0x7f621806fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:32:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.585+0000 7f621d200640 1 --2- 172.21.15.94:0/1303223497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180cfcb0 0x7f621806fe80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f62040194a0 tx=0x7f62040a5750 comp rx=0 tx=0).stop 2024-09-10T23:32:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.585+0000 7f621d200640 1 -- 172.21.15.94:0/1303223497 shutdown_connections 2024-09-10T23:32:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.585+0000 7f621d200640 1 --2- 172.21.15.94:0/1303223497 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f61f8072610 0x7f61f8074ad0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.585+0000 7f621d200640 1 --2- 172.21.15.94:0/1303223497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f621815ba50 0x7f62181faf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.585+0000 7f621d200640 1 --2- 172.21.15.94:0/1303223497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180cfcb0 0x7f621806fe80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.585+0000 7f621d200640 1 --2- 172.21.15.94:0/1303223497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62180cddb0 0x7f62181460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:32:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.585+0000 7f621d200640 1 -- 172.21.15.94:0/1303223497 >> 172.21.15.94:0/1303223497 conn(0x7f621806cdf0 msgr2=0x7f6218156c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:32:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.585+0000 7f621d200640 1 -- 172.21.15.94:0/1303223497 shutdown_connections 2024-09-10T23:32:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:32:59.585+0000 7f621d200640 1 -- 172.21.15.94:0/1303223497 wait complete. 2024-09-10T23:33:00.009 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T23:32:41.948908+0000', 'modified': '2024-09-10T23:32:44.956870+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 194174}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_194174': {'gid': 194174, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6836/1418253171', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 1418253171}, {'type': 'v1', 'addr': '172.21.15.181:6836', 'nonce': 1418253171}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 194174, 'qdb_cluster': [194174]} 2024-09-10T23:33:00.009 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T23:33:00.010 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T23:33:00.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.149+0000 7fea70800640 1 -- 172.21.15.94:0/3578336289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea680ce1e0 msgr2=0x7fea680cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:00.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.149+0000 7fea70800640 1 --2- 172.21.15.94:0/3578336289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea680ce1e0 0x7fea680cec70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fea58009840 tx=0x7fea5801c760 comp rx=0 tx=0).stop 2024-09-10T23:33:00.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.149+0000 7fea70800640 1 -- 172.21.15.94:0/3578336289 shutdown_connections 2024-09-10T23:33:00.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.149+0000 7fea70800640 1 --2- 172.21.15.94:0/3578336289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea680ce1e0 0x7fea680cec70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.149+0000 7fea70800640 1 --2- 172.21.15.94:0/3578336289 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea680ce890 0x7fea680cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.149+0000 7fea70800640 1 -- 172.21.15.94:0/3578336289 >> 172.21.15.94:0/3578336289 conn(0x7fea680d1c60 msgr2=0x7fea680d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:00.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.149+0000 7fea70800640 1 -- 172.21.15.94:0/3578336289 shutdown_connections 2024-09-10T23:33:00.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.149+0000 7fea70800640 1 -- 172.21.15.94:0/3578336289 wait complete. 2024-09-10T23:33:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea70800640 1 Processor -- start 2024-09-10T23:33:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea70800640 1 -- start start 2024-09-10T23:33:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea70800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea680b1100 0x7fea68157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea70800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea680ce1e0 0x7fea681fb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea70800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea680ce890 0x7fea680cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea70800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea680d08e0 con 0x7fea680ce890 2024-09-10T23:33:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea70800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fea680d0760 con 0x7fea680ce1e0 2024-09-10T23:33:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea70800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fea680d0a60 con 0x7fea680b1100 2024-09-10T23:33:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea6e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea680ce890 0x7fea680cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea6e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea680ce890 0x7fea680cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40442/0 (socket says 172.21.15.94:40442) 2024-09-10T23:33:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea6de00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea680b1100 0x7fea68157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea6e800640 1 -- 172.21.15.94:0/1498874383 learned_addr learned my addr 172.21.15.94:0/1498874383 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea6d400640 1 --2- 172.21.15.94:0/1498874383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea680ce1e0 0x7fea681fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea6d400640 1 -- 172.21.15.94:0/1498874383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea680b1100 msgr2=0x7fea68157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea6d400640 1 --2- 172.21.15.94:0/1498874383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea680b1100 0x7fea68157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea6d400640 1 -- 172.21.15.94:0/1498874383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea680ce890 msgr2=0x7fea680cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea6d400640 1 --2- 172.21.15.94:0/1498874383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea680ce890 0x7fea680cb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea6d400640 1 -- 172.21.15.94:0/1498874383 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea680ca580 con 0x7fea680ce1e0 2024-09-10T23:33:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea6e800640 1 --2- 172.21.15.94:0/1498874383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea680ce890 0x7fea680cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:33:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea6d400640 1 --2- 172.21.15.94:0/1498874383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea680ce1e0 0x7fea681fb890 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fea580ac000 tx=0x7fea580a5750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea62000640 1 -- 172.21.15.94:0/1498874383 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea580b2040 con 0x7fea680ce1e0 2024-09-10T23:33:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea62000640 1 -- 172.21.15.94:0/1498874383 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea58005020 con 0x7fea680ce1e0 2024-09-10T23:33:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea62000640 1 -- 172.21.15.94:0/1498874383 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea580044f0 con 0x7fea680ce1e0 2024-09-10T23:33:00.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.153+0000 7fea70800640 1 -- 172.21.15.94:0/1498874383 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea680ca860 con 0x7fea680ce1e0 2024-09-10T23:33:00.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.157+0000 7fea70800640 1 -- 172.21.15.94:0/1498874383 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea680b6ad0 con 0x7fea680ce1e0 2024-09-10T23:33:00.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.157+0000 7fea62000640 1 -- 172.21.15.94:0/1498874383 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fea580b5070 con 0x7fea680ce1e0 2024-09-10T23:33:00.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.157+0000 7fea73c5d000 1 Processor -- start 2024-09-10T23:33:00.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.157+0000 7fea73c5d000 1 -- start start 2024-09-10T23:33:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea62000640 1 --2- 172.21.15.94:0/1498874383 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea4c0725e0 0x7fea4c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fda59d0 0x55b07fda5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 0x55b07fd598d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fd5a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b07fc7a3f0 con 0x55b07fd59e10 2024-09-10T23:33:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b07fbb7b20 con 0x55b07fda62f0 2024-09-10T23:33:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x55b07fba6f50 con 0x55b07fda59d0 2024-09-10T23:33:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea62000640 1 -- 172.21.15.94:0/1498874383 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fea5812c4d0 con 0x7fea680ce1e0 2024-09-10T23:33:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea6e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fd5a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea6d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 0x55b07fd598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea6d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 0x55b07fd598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:36208/0 (socket says 172.21.15.94:36208) 2024-09-10T23:33:00.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea6d400640 1 -- 172.21.15.94:0/2143644121 learned_addr learned my addr 172.21.15.94:0/2143644121 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:00.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea6de00640 1 --2- 172.21.15.94:0/2143644121 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fda59d0 0x55b07fda5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea6e800640 1 --2- 172.21.15.94:0/1498874383 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea4c0725e0 0x7fea4c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea6e800640 1 -- 172.21.15.94:0/2143644121 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fda59d0 msgr2=0x55b07fda5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:33:00.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea6e800640 1 --2- 172.21.15.94:0/2143644121 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fda59d0 0x55b07fda5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea6e800640 1 -- 172.21.15.94:0/2143644121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 msgr2=0x55b07fd598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:33:00.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea6e800640 1 --2- 172.21.15.94:0/2143644121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 0x55b07fd598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea6e800640 1 -- 172.21.15.94:0/2143644121 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b07fb6e590 con 0x55b07fd59e10 2024-09-10T23:33:00.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea6d400640 1 --2- 172.21.15.94:0/2143644121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 0x55b07fd598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:33:00.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea6e800640 1 --2- 172.21.15.94:0/2143644121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fd5a1f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fea64018560 tx=0x7fea64018650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0436c701437f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:00.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea56a00640 1 -- 172.21.15.94:0/2143644121 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea64010040 con 0x55b07fd59e10 2024-09-10T23:33:00.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea56a00640 1 -- 172.21.15.94:0/2143644121 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea64011400 con 0x55b07fd59e10 2024-09-10T23:33:00.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea6e800640 1 --2- 172.21.15.94:0/1498874383 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea4c0725e0 0x7fea4c074aa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fea64018b50 tx=0x7fea64009000 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:00.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea56a00640 1 -- 172.21.15.94:0/2143644121 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea64004a10 con 0x55b07fd59e10 2024-09-10T23:33:00.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 -- 172.21.15.94:0/2143644121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 msgr2=0x55b07fd5a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:00.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 --2- 172.21.15.94:0/2143644121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fd5a1f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fea64018560 tx=0x7fea64018650 comp rx=0 tx=0).stop 2024-09-10T23:33:00.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 -- 172.21.15.94:0/2143644121 shutdown_connections 2024-09-10T23:33:00.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 --2- 172.21.15.94:0/2143644121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 0x55b07fd598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 --2- 172.21.15.94:0/2143644121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fd5a1f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 --2- 172.21.15.94:0/2143644121 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fda59d0 0x55b07fda5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 -- 172.21.15.94:0/2143644121 >> 172.21.15.94:0/2143644121 conn(0x55b07fc43130 msgr2=0x55b07fd88e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:00.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 -- 172.21.15.94:0/2143644121 shutdown_connections 2024-09-10T23:33:00.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 -- 172.21.15.94:0/2143644121 wait complete. 2024-09-10T23:33:00.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 -- start start 2024-09-10T23:33:00.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.161+0000 7fea73c5d000 1 Processor -- start 2024-09-10T23:33:00.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea73c5d000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fd59e10 0x55b07fd5b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea73c5d000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda59d0 0x55b07fd5b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea73c5d000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fda62f0 0x55b07fd5bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea73c5d000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b07fb415d0 con 0x55b07fda62f0 2024-09-10T23:33:00.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea73c5d000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b07fbaad60 con 0x55b07fda59d0 2024-09-10T23:33:00.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea73c5d000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x55b07fb4d500 con 0x55b07fd59e10 2024-09-10T23:33:00.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea73c5d000 10 client.0 _ll_register_callbacks cb 0x55b07fb51810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:33:00.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea6e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fda62f0 0x55b07fd5bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea6d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda59d0 0x55b07fd5b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea6de00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fd59e10 0x55b07fd5b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea6e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fda62f0 0x55b07fd5bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40464/0 (socket says 172.21.15.94:40464) 2024-09-10T23:33:00.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea6e800640 1 -- 172.21.15.94:0/1029864856 learned_addr learned my addr 172.21.15.94:0/1029864856 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:00.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea6d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda59d0 0x55b07fd5b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:36210/0 (socket says 172.21.15.94:36210) 2024-09-10T23:33:00.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea6e800640 1 -- 172.21.15.94:0/1029864856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fd59e10 msgr2=0x55b07fd5b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:33:00.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea6e800640 1 --2- 172.21.15.94:0/1029864856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fd59e10 0x55b07fd5b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea6e800640 1 -- 172.21.15.94:0/1029864856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda59d0 msgr2=0x55b07fd5b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:33:00.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea6e800640 1 --2- 172.21.15.94:0/1029864856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda59d0 0x55b07fd5b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea6e800640 1 -- 172.21.15.94:0/1029864856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b07fb45170 con 0x55b07fda62f0 2024-09-10T23:33:00.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea6e800640 1 --2- 172.21.15.94:0/1029864856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fda62f0 0x55b07fd5bcc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fea64026040 tx=0x7fea64011960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ee5fb1e5e1203f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:00.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea4b400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:33:00.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea4b400640 1 -- 172.21.15.94:0/1029864856 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6402f020 con 0x55b07fda62f0 2024-09-10T23:33:00.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea4b400640 1 -- 172.21.15.94:0/1029864856 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fea64005230 con 0x55b07fda62f0 2024-09-10T23:33:00.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea73c5d000 1 -- 172.21.15.94:0/1029864856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b07fb3f0f0 con 0x55b07fda62f0 2024-09-10T23:33:00.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea4b400640 1 -- 172.21.15.94:0/1029864856 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fea64039050 con 0x55b07fda62f0 2024-09-10T23:33:00.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea73c5d000 10 client.4695 fetch_fsmap learned FSMap version 8 2024-09-10T23:33:00.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea73c5d000 1 -- 172.21.15.94:0/1029864856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55b07fc2bef0 con 0x55b07fda62f0 2024-09-10T23:33:00.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea4b400640 1 -- 172.21.15.94:0/1029864856 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b07fb3f0f0 con 0x55b07fda62f0 2024-09-10T23:33:00.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea4b400640 1 -- 172.21.15.94:0/1029864856 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fea6403c020 con 0x55b07fda62f0 2024-09-10T23:33:00.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea73c5d000 10 client.4695 fetch_fsmap finished waiting for FSMap version 8 2024-09-10T23:33:00.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea73c5d000 10 client.4695 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T23:33:00.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea73c5d000 20 client.4695 populate_metadata read hostname 'smithi094' 2024-09-10T23:33:00.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea73c5d000 1 --2- 172.21.15.94:0/1029864856 >> [v2:172.21.15.181:6834/1418253171,v1:172.21.15.181:6836/1418253171] conn(0x55b07fdb4c70 0x55b07fdb7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea73c5d000 4 client.4695 mds_command: new command op to 194174 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:33:00.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea73c5d000 1 -- 172.21.15.94:0/1029864856 --> [v2:172.21.15.181:6834/1418253171,v1:172.21.15.181:6836/1418253171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b07fbb9a70 con 0x55b07fdb4c70 2024-09-10T23:33:00.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea6de00640 1 --2- 172.21.15.94:0/1029864856 >> [v2:172.21.15.181:6834/1418253171,v1:172.21.15.181:6836/1418253171] conn(0x55b07fdb4c70 0x55b07fdb7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea6de00640 1 --2- 172.21.15.94:0/1029864856 >> [v2:172.21.15.181:6834/1418253171,v1:172.21.15.181:6836/1418253171] conn(0x55b07fdb4c70 0x55b07fdb7060 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7e6294eefae27a9 server_cookie=69c9bb20cf136da6 in_seq=0 out_seq=0 2024-09-10T23:33:00.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.165+0000 7fea4b400640 10 client.4695 ms_handle_connect on v2:172.21.15.181:6834/1418253171 2024-09-10T23:33:00.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.169+0000 7fea4b400640 1 -- 172.21.15.94:0/1029864856 <== mds.0 v2:172.21.15.181:6834/1418253171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b07fbb9a70 con 0x55b07fdb4c70 2024-09-10T23:33:00.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.169+0000 7fea4b400640 10 client.4695 handle_command_reply: tid=0 2024-09-10T23:33:00.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.169+0000 7fea73c5d000 1 client.4695 shutdown 2024-09-10T23:33:00.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.169+0000 7fea73c5d000 1 -- 172.21.15.94:0/1029864856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fda62f0 msgr2=0x55b07fd5bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:00.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.169+0000 7fea73c5d000 1 --2- 172.21.15.94:0/1029864856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fda62f0 0x55b07fd5bcc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fea64026040 tx=0x7fea64011960 comp rx=0 tx=0).stop 2024-09-10T23:33:00.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.169+0000 7fea73c5d000 1 -- 172.21.15.94:0/1029864856 shutdown_connections 2024-09-10T23:33:00.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.169+0000 7fea73c5d000 1 --2- 172.21.15.94:0/1029864856 >> [v2:172.21.15.181:6834/1418253171,v1:172.21.15.181:6836/1418253171] conn(0x55b07fdb4c70 0x55b07fdb7060 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.169+0000 7fea73c5d000 1 --2- 172.21.15.94:0/1029864856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda59d0 0x55b07fd5b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.169+0000 7fea73c5d000 1 --2- 172.21.15.94:0/1029864856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fda62f0 0x55b07fd5bcc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.169+0000 7fea73c5d000 1 --2- 172.21.15.94:0/1029864856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fd59e10 0x55b07fd5b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.169+0000 7fea73c5d000 1 -- 172.21.15.94:0/1029864856 >> 172.21.15.94:0/1029864856 conn(0x55b07fc43130 msgr2=0x55b07fd58fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:00.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.169+0000 7fea4b400640 0 client.4695 ms_handle_reset on v2:172.21.15.181:6834/1418253171 2024-09-10T23:33:00.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.169+0000 7fea73c5d000 1 -- 172.21.15.94:0/1029864856 shutdown_connections 2024-09-10T23:33:00.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.169+0000 7fea73c5d000 1 -- 172.21.15.94:0/1029864856 wait complete. 2024-09-10T23:33:00.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.169+0000 7fea73c5d000 20 client.4695 trim_cache size 0 max 16384 2024-09-10T23:33:00.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.185+0000 7fea73c5d000 1 Processor -- start 2024-09-10T23:33:00.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.185+0000 7fea73c5d000 1 -- start start 2024-09-10T23:33:00.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.185+0000 7fea73c5d000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fe175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.185+0000 7fea73c5d000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fda59d0 0x55b07fe17b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.185+0000 7fea73c5d000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 0x55b07fe18050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.185+0000 7fea73c5d000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea681f78a0 con 0x55b07fd59e10 2024-09-10T23:33:00.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.185+0000 7fea73c5d000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fea681f7ef0 con 0x55b07fda62f0 2024-09-10T23:33:00.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.185+0000 7fea73c5d000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x55b07fb72ec0 con 0x55b07fda59d0 2024-09-10T23:33:00.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea6de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fe175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea6de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fe175d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40468/0 (socket says 172.21.15.94:40468) 2024-09-10T23:33:00.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea6d400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fda59d0 0x55b07fe17b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea6de00640 1 -- 172.21.15.94:0/1721400485 learned_addr learned my addr 172.21.15.94:0/1721400485 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:00.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea6e800640 1 --2- 172.21.15.94:0/1721400485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 0x55b07fe18050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea6de00640 1 -- 172.21.15.94:0/1721400485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fda59d0 msgr2=0x55b07fe17b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:33:00.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea6de00640 1 --2- 172.21.15.94:0/1721400485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fda59d0 0x55b07fe17b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea6de00640 1 -- 172.21.15.94:0/1721400485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 msgr2=0x55b07fe18050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:33:00.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea6de00640 1 --2- 172.21.15.94:0/1721400485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 0x55b07fe18050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea6de00640 1 -- 172.21.15.94:0/1721400485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b07fc02d90 con 0x55b07fd59e10 2024-09-10T23:33:00.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea6de00640 1 --2- 172.21.15.94:0/1721400485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fe175d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fea5c000c10 tx=0x7fea5c0091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3728febfecb862 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:00.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea56000640 1 -- 172.21.15.94:0/1721400485 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c003df0 con 0x55b07fd59e10 2024-09-10T23:33:00.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea56000640 1 -- 172.21.15.94:0/1721400485 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea5c010430 con 0x55b07fd59e10 2024-09-10T23:33:00.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea56000640 1 -- 172.21.15.94:0/1721400485 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c010880 con 0x55b07fd59e10 2024-09-10T23:33:00.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 -- 172.21.15.94:0/1721400485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 msgr2=0x55b07fe175d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:00.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 --2- 172.21.15.94:0/1721400485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fe175d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fea5c000c10 tx=0x7fea5c0091e0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 -- 172.21.15.94:0/1721400485 shutdown_connections 2024-09-10T23:33:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 --2- 172.21.15.94:0/1721400485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 0x55b07fe18050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 --2- 172.21.15.94:0/1721400485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fda59d0 0x55b07fe17b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 --2- 172.21.15.94:0/1721400485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fe175d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 -- 172.21.15.94:0/1721400485 >> 172.21.15.94:0/1721400485 conn(0x55b07fc43130 msgr2=0x55b07fd59470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 -- 172.21.15.94:0/1721400485 shutdown_connections 2024-09-10T23:33:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 -- 172.21.15.94:0/1721400485 wait complete. 2024-09-10T23:33:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 -- start start 2024-09-10T23:33:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 Processor -- start 2024-09-10T23:33:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fdbbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fda59d0 0x55b07fdd7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 0x55b07fdd7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b07fbb68f0 con 0x55b07fd59e10 2024-09-10T23:33:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b07fc66730 con 0x55b07fda62f0 2024-09-10T23:33:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x55b07fbaa070 con 0x55b07fda59d0 2024-09-10T23:33:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea73c5d000 10 client.0 _ll_register_callbacks cb 0x7fea5c009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:33:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.189+0000 7fea6de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fdbbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea6de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fdbbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40476/0 (socket says 172.21.15.94:40476) 2024-09-10T23:33:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea6de00640 1 -- 172.21.15.94:0/3882400831 learned_addr learned my addr 172.21.15.94:0/3882400831 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea6e800640 1 --2- 172.21.15.94:0/3882400831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 0x55b07fdd7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea6d400640 1 --2- 172.21.15.94:0/3882400831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fda59d0 0x55b07fdd7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea6de00640 1 -- 172.21.15.94:0/3882400831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fda59d0 msgr2=0x55b07fdd7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:33:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea6de00640 1 --2- 172.21.15.94:0/3882400831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fda59d0 0x55b07fdd7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea6de00640 1 -- 172.21.15.94:0/3882400831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 msgr2=0x55b07fdd7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:33:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea6de00640 1 --2- 172.21.15.94:0/3882400831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 0x55b07fdd7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea6de00640 1 -- 172.21.15.94:0/3882400831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b07fb6da90 con 0x55b07fd59e10 2024-09-10T23:33:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea6de00640 1 --2- 172.21.15.94:0/3882400831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fdbbc50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0095c0 tx=0x7fea5c009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad457824a2a24ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea4b400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:33:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea4b400640 1 -- 172.21.15.94:0/3882400831 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c003bf0 con 0x55b07fd59e10 2024-09-10T23:33:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea4b400640 1 -- 172.21.15.94:0/3882400831 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fea5c003d90 con 0x55b07fd59e10 2024-09-10T23:33:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 1 -- 172.21.15.94:0/3882400831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea5c010430 con 0x55b07fd59e10 2024-09-10T23:33:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea4b400640 1 -- 172.21.15.94:0/3882400831 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fea5c010620 con 0x55b07fd59e10 2024-09-10T23:33:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 10 client.4701 fetch_fsmap learned FSMap version 8 2024-09-10T23:33:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 1 -- 172.21.15.94:0/3882400831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55b07fb4c050 con 0x55b07fd59e10 2024-09-10T23:33:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea4b400640 1 -- 172.21.15.94:0/3882400831 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea5c010430 con 0x55b07fd59e10 2024-09-10T23:33:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea4b400640 1 -- 172.21.15.94:0/3882400831 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fea5c0107c0 con 0x55b07fd59e10 2024-09-10T23:33:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 10 client.4701 fetch_fsmap finished waiting for FSMap version 8 2024-09-10T23:33:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 10 client.4701 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T23:33:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 20 client.4701 populate_metadata read hostname 'smithi094' 2024-09-10T23:33:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 1 --2- 172.21.15.94:0/3882400831 >> [v2:172.21.15.181:6834/1418253171,v1:172.21.15.181:6836/1418253171] conn(0x55b07fdd8000 0x55b07fe1b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 4 client.4701 mds_command: new command op to 194174 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T23:33:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 1 -- 172.21.15.94:0/3882400831 --> [v2:172.21.15.181:6834/1418253171,v1:172.21.15.181:6836/1418253171] -- command(tid 0: {"prefix": "status"}) -- 0x55b07fbb5520 con 0x55b07fdd8000 2024-09-10T23:33:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea6d400640 1 --2- 172.21.15.94:0/3882400831 >> [v2:172.21.15.181:6834/1418253171,v1:172.21.15.181:6836/1418253171] conn(0x55b07fdd8000 0x55b07fe1b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea6d400640 1 --2- 172.21.15.94:0/3882400831 >> [v2:172.21.15.181:6834/1418253171,v1:172.21.15.181:6836/1418253171] conn(0x55b07fdd8000 0x55b07fe1b820 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e12a257ba7655f0 server_cookie=777cb34759ce3fd in_seq=0 out_seq=0 2024-09-10T23:33:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea4b400640 10 client.4701 ms_handle_connect on v2:172.21.15.181:6834/1418253171 2024-09-10T23:33:00.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea4b400640 1 -- 172.21.15.94:0/3882400831 <== mds.0 v2:172.21.15.181:6834/1418253171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b07fbb5520 con 0x55b07fdd8000 2024-09-10T23:33:00.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea4b400640 10 client.4701 handle_command_reply: tid=0 2024-09-10T23:33:00.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 1 client.4701 shutdown 2024-09-10T23:33:00.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 1 -- 172.21.15.94:0/3882400831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 msgr2=0x55b07fdbbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:00.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 1 --2- 172.21.15.94:0/3882400831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fdbbc50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0095c0 tx=0x7fea5c009710 comp rx=0 tx=0).stop 2024-09-10T23:33:00.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 1 -- 172.21.15.94:0/3882400831 shutdown_connections 2024-09-10T23:33:00.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 1 --2- 172.21.15.94:0/3882400831 >> [v2:172.21.15.181:6834/1418253171,v1:172.21.15.181:6836/1418253171] conn(0x55b07fdd8000 0x55b07fe1b820 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 1 --2- 172.21.15.94:0/3882400831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b07fda62f0 0x55b07fdd7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 1 --2- 172.21.15.94:0/3882400831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b07fda59d0 0x55b07fdd7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea4b400640 0 client.4701 ms_handle_reset on v2:172.21.15.181:6834/1418253171 2024-09-10T23:33:00.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 1 --2- 172.21.15.94:0/3882400831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b07fd59e10 0x55b07fdbbc50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 1 -- 172.21.15.94:0/3882400831 >> 172.21.15.94:0/3882400831 conn(0x55b07fc43130 msgr2=0x55b07fdb7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:00.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 1 -- 172.21.15.94:0/3882400831 shutdown_connections 2024-09-10T23:33:00.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 1 -- 172.21.15.94:0/3882400831 wait complete. 2024-09-10T23:33:00.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea73c5d000 20 client.4701 trim_cache size 0 max 16384 2024-09-10T23:33:00.202 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:33:00.202 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "1798cdb9-9937-4918-a569-86b41a92817e", 2024-09-10T23:33:00.202 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-09-10T23:33:00.202 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 194174, 2024-09-10T23:33:00.202 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-10T23:33:00.202 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-10T23:33:00.202 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-10T23:33:00.202 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 17.650823492000001, 2024-09-10T23:33:00.202 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 8, 2024-09-10T23:33:00.202 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 22, 2024-09-10T23:33:00.202 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 22, 2024-09-10T23:33:00.203 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 23.664134618999999 2024-09-10T23:33:00.203 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:33:00.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea70800640 1 -- 172.21.15.94:0/1498874383 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea4c0725e0 msgr2=0x7fea4c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:00.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea70800640 1 --2- 172.21.15.94:0/1498874383 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea4c0725e0 0x7fea4c074aa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fea64018b50 tx=0x7fea64009000 comp rx=0 tx=0).stop 2024-09-10T23:33:00.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea70800640 1 -- 172.21.15.94:0/1498874383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea680ce1e0 msgr2=0x7fea681fb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:00.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea70800640 1 --2- 172.21.15.94:0/1498874383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea680ce1e0 0x7fea681fb890 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fea580ac000 tx=0x7fea580a5750 comp rx=0 tx=0).stop 2024-09-10T23:33:00.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea70800640 1 -- 172.21.15.94:0/1498874383 shutdown_connections 2024-09-10T23:33:00.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.193+0000 7fea70800640 1 --2- 172.21.15.94:0/1498874383 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea4c0725e0 0x7fea4c074aa0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.197+0000 7fea70800640 1 --2- 172.21.15.94:0/1498874383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea680ce1e0 0x7fea681fb890 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.197+0000 7fea70800640 1 --2- 172.21.15.94:0/1498874383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea680ce890 0x7fea680cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.197+0000 7fea70800640 1 --2- 172.21.15.94:0/1498874383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea680b1100 0x7fea68157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.197+0000 7fea70800640 1 -- 172.21.15.94:0/1498874383 >> 172.21.15.94:0/1498874383 conn(0x7fea680d1c60 msgr2=0x7fea680b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:00.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.197+0000 7fea70800640 1 -- 172.21.15.94:0/1498874383 shutdown_connections 2024-09-10T23:33:00.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.197+0000 7fea70800640 1 -- 172.21.15.94:0/1498874383 wait complete. 2024-09-10T23:33:00.217 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-10T23:33:00.245 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-10T23:33:00.245 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-10T23:33:00.245 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-10T23:33:00.245 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T23:33:00.245 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:00.245 INFO:tasks.ceph_fuse:client.2 config is {} 2024-09-10T23:33:00.246 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T23:33:00.246 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T23:33:00.246 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-10T23:33:00.246 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T23:33:00.246 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:00.246 INFO:tasks.ceph_fuse:client.3 config is {} 2024-09-10T23:33:00.246 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T23:33:00.246 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T23:33:00.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:00.246 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T23:33:00.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:00.256 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-09-10T23:33:00.319 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2024-09-10T23:33:00.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:33:00.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:00.322 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-09-10T23:33:00.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:00.331 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2024-09-10T23:33:00.395 INFO:teuthology.orchestra.run.smithi181.stderr:Cannot find device "ceph-brx" 2024-09-10T23:33:00.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:33:00.398 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-10T23:33:00.398 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T23:33:00.398 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:33:00.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae8a00640 1 -- 172.21.15.94:0/3553671173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae415f370 msgr2=0x7fcae40ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:00.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae8a00640 1 --2- 172.21.15.94:0/3553671173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae415f370 0x7fcae40ce690 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009870 tx=0x7fcad401c860 comp rx=0 tx=0).stop 2024-09-10T23:33:00.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae8a00640 1 -- 172.21.15.94:0/3553671173 shutdown_connections 2024-09-10T23:33:00.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae8a00640 1 --2- 172.21.15.94:0/3553671173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae415f370 0x7fcae40ce690 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae8a00640 1 -- 172.21.15.94:0/3553671173 >> 172.21.15.94:0/3553671173 conn(0x7fcae40d1c80 msgr2=0x7fcae40d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:00.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae8a00640 1 -- 172.21.15.94:0/3553671173 shutdown_connections 2024-09-10T23:33:00.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae8a00640 1 -- 172.21.15.94:0/3553671173 wait complete. 2024-09-10T23:33:00.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae8a00640 1 Processor -- start 2024-09-10T23:33:00.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae8a00640 1 -- start start 2024-09-10T23:33:00.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae8a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae40b0ec0 0x7fcae415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40cebd0 0x7fcae415c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae8a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae415f370 0x7fcae40b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae8a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcae416a780 con 0x7fcae40cebd0 2024-09-10T23:33:00.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae8a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcae416a600 con 0x7fcae415f370 2024-09-10T23:33:00.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae8a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fcae416a900 con 0x7fcae40b0ec0 2024-09-10T23:33:00.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcad9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40cebd0 0x7fcae415c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcad9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40cebd0 0x7fcae415c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40490/0 (socket says 172.21.15.94:40490) 2024-09-10T23:33:00.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcad9400640 1 -- 172.21.15.94:0/392561107 learned_addr learned my addr 172.21.15.94:0/392561107 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:00.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae1e00640 1 --2- 172.21.15.94:0/392561107 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae40b0ec0 0x7fcae415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcae2800640 1 --2- 172.21.15.94:0/392561107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae415f370 0x7fcae40b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcad9400640 1 -- 172.21.15.94:0/392561107 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae40b0ec0 msgr2=0x7fcae415bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:00.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcad9400640 1 --2- 172.21.15.94:0/392561107 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae40b0ec0 0x7fcae415bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcad9400640 1 -- 172.21.15.94:0/392561107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae415f370 msgr2=0x7fcae40b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:00.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcad9400640 1 --2- 172.21.15.94:0/392561107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae415f370 0x7fcae40b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.537+0000 7fcad9400640 1 -- 172.21.15.94:0/392561107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae40b5430 con 0x7fcae40cebd0 2024-09-10T23:33:00.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.541+0000 7fcad9400640 1 --2- 172.21.15.94:0/392561107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40cebd0 0x7fcae415c350 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00c400 tx=0x7fcacc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:00.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.541+0000 7fcadaa00640 1 -- 172.21.15.94:0/392561107 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc017070 con 0x7fcae40cebd0 2024-09-10T23:33:00.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.541+0000 7fcadaa00640 1 -- 172.21.15.94:0/392561107 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcacc0053c0 con 0x7fcae40cebd0 2024-09-10T23:33:00.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.541+0000 7fcadaa00640 1 -- 172.21.15.94:0/392561107 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc003860 con 0x7fcae40cebd0 2024-09-10T23:33:00.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.541+0000 7fcae8a00640 1 -- 172.21.15.94:0/392561107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae40b39e0 con 0x7fcae40cebd0 2024-09-10T23:33:00.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.541+0000 7fcae8a00640 1 -- 172.21.15.94:0/392561107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae40b3f60 con 0x7fcae40cebd0 2024-09-10T23:33:00.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.541+0000 7fcae8a00640 1 -- 172.21.15.94:0/392561107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae40ce690 con 0x7fcae40cebd0 2024-09-10T23:33:00.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.541+0000 7fcadaa00640 1 -- 172.21.15.94:0/392561107 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcacc003a00 con 0x7fcae40cebd0 2024-09-10T23:33:00.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.541+0000 7fcadaa00640 1 --2- 172.21.15.94:0/392561107 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcac00725e0 0x7fcac0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:00.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.545+0000 7fcadaa00640 1 -- 172.21.15.94:0/392561107 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcacc092a40 con 0x7fcae40cebd0 2024-09-10T23:33:00.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.545+0000 7fcae1e00640 1 --2- 172.21.15.94:0/392561107 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcac00725e0 0x7fcac0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:00.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.545+0000 7fcae1e00640 1 --2- 172.21.15.94:0/392561107 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcac00725e0 0x7fcac0074aa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcad40af000 tx=0x7fcad401b0a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:00.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.549+0000 7fcadaa00640 1 -- 172.21.15.94:0/392561107 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcacc05da30 con 0x7fcae40cebd0 2024-09-10T23:33:00.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.749+0000 7fcae8a00640 1 -- 172.21.15.94:0/392561107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcae4157710 con 0x7fcae40cebd0 2024-09-10T23:33:00.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.749+0000 7fcadaa00640 1 -- 172.21.15.94:0/392561107 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fcacc062460 con 0x7fcae40cebd0 2024-09-10T23:33:00.754 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:33:00.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.753+0000 7fcae8a00640 1 -- 172.21.15.94:0/392561107 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcac00725e0 msgr2=0x7fcac0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:00.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.753+0000 7fcae8a00640 1 --2- 172.21.15.94:0/392561107 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcac00725e0 0x7fcac0074aa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcad40af000 tx=0x7fcad401b0a0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.753+0000 7fcae8a00640 1 -- 172.21.15.94:0/392561107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40cebd0 msgr2=0x7fcae415c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:00.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.753+0000 7fcae8a00640 1 --2- 172.21.15.94:0/392561107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40cebd0 0x7fcae415c350 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00c400 tx=0x7fcacc00c8c0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.753+0000 7fcae8a00640 1 -- 172.21.15.94:0/392561107 shutdown_connections 2024-09-10T23:33:00.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.753+0000 7fcae8a00640 1 --2- 172.21.15.94:0/392561107 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcac00725e0 0x7fcac0074aa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.753+0000 7fcae8a00640 1 --2- 172.21.15.94:0/392561107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcae415f370 0x7fcae40b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.753+0000 7fcae8a00640 1 --2- 172.21.15.94:0/392561107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcae40cebd0 0x7fcae415c350 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.753+0000 7fcae8a00640 1 --2- 172.21.15.94:0/392561107 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcae40b0ec0 0x7fcae415bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:00.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.753+0000 7fcae8a00640 1 -- 172.21.15.94:0/392561107 >> 172.21.15.94:0/392561107 conn(0x7fcae40d1c80 msgr2=0x7fcae40b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:00.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.753+0000 7fcae8a00640 1 -- 172.21.15.94:0/392561107 shutdown_connections 2024-09-10T23:33:00.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00.753+0000 7fcae8a00640 1 -- 172.21.15.94:0/392561107 wait complete. 2024-09-10T23:33:00.774 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:33:00.774 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:33:00.774 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T23:33:00.774 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T23:33:00.774 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:00.774 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:33:00.774 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:33:00.775 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:33:00.775 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:33:00.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:00.775 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T23:33:00.783 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:33:00.783 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:33:00.783 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:33:00.784 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:00.784 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T23:33:00.784 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:00.784 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:00.784 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:00.784 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T23:33:00.784 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:00.784 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:00.784 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T23:33:00.784 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:00.784 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:00.784 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:33:00.785 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:00.785 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:33:00.785 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42430sec preferred_lft 42430sec 2024-09-10T23:33:00.785 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-09-10T23:33:00.785 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:00.785 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:33:00.785 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:ed:7d:46:db brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:00.785 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:33:00.785 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:00.786 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T23:33:00.786 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:00.786 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:33:00.786 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2024-09-10T23:33:00.787 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2024-09-10T23:33:00.787 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2024-09-10T23:33:00.787 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T23:33:00.787 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:33:00.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:00.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:00.940 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:33:00.999 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T23:33:01.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:01.002 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T23:33:01.074 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T23:33:01.075 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:33:01.075 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:33:01.075 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:33:01.075 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:33:01.075 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:33:01.075 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:33:01.075 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:01.075 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:01.075 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:01.075 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:33:01.076 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:01.076 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:33:01.076 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:33:01.076 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:33:01.076 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:33:01.076 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:33:01.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:01.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:01.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:01.215 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T23:33:01.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:01.225 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-09-10T23:33:01.278 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:01.278 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:33:01.278 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:01.278 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T23:33:01.278 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:33:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:01.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:01.410 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T23:33:01.411 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:01.411 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:33:01.411 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T23:33:01.411 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T23:33:01.411 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T23:33:01.411 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T23:33:01.411 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T23:33:01.411 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:33:01.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:01.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:01.577 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:01.577 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:33:01.577 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-09-10T23:33:01.577 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T23:33:01.577 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:33:01.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:01.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:01.706 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T23:33:01.706 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:33:01.706 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:01.715 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:33:01.715 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:33:01.715 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:01.769 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:33:01.829 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:33:01.879 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:33:01.880 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:01.881 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:01.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:01.882 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:33:01.933 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:33:01.933 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T23:33:01.976 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:33:01.996 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:02.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:02.051 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:33:02.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:2024-09-10T23:33:02.065+0000 7f3120e79580 -1 init, newargv = 0x5637c5c31b80 newargc=15 2024-09-10T23:33:02.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[35504]: starting ceph client 2024-09-10T23:33:02.079 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[35504]: starting fuse 2024-09-10T23:33:03.180 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:33:03.205 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:33:03.260 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:33:03.260 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:33:03.260 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:33:03.260 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:33:03.260 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:33:03.260 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:33:03.260 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:33:03.260 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:33:03.261 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:33:03.261 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:33:03.261 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:33:03.261 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:33:03.261 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:33:03.261 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:33:03.261 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:33:03.261 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:33:03.261 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:33:03.261 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:33:03.261 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:33:03.261 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:33:03.261 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:33:03.261 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:33:03.261 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:33:03.262 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:33:03.262 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:33:03.262 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:33:03.262 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:33:03.262 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:33:03.262 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:33:03.262 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:03.262 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:03.262 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:03.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:03.263 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:33:03.314 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T23:33:03.314 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:33:03.314 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:33:03.314 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-09-10T23:33:03.314 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-09-10T23:33:03.314 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-09-10T23:33:03.315 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-09-10T23:33:03.315 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:33:03.315 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-09-10T23:33:03.315 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T23:33:03.315 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-09-10T23:33:03.315 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T23:33:03.315 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:33:03.315 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-09-10T23:33:03.315 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-09-10T23:33:03.315 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-09-10T23:33:03.315 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-09-10T23:33:03.315 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:33:03.315 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-09-10T23:33:03.315 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:33:03.316 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:33:03.316 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:33:03.316 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-09-10T23:33:03.316 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-09-10T23:33:03.316 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-09-10T23:33:03.316 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:33:03.316 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:33:03.316 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.0")) 2024-09-10T23:33:03.316 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-09-10T23:33:03.450 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.0.35504.asok 2024-09-10T23:33:03.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:03.462 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35504.asok 2024-09-10T23:33:03.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:03.463 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35504.asok status 2024-09-10T23:33:03.639 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:33:03.640 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-09-10T23:33:03.640 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:33:03.640 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:33:03.640 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "0", 2024-09-10T23:33:03.640 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-09-10T23:33:03.640 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T23:33:03.640 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "35504", 2024-09-10T23:33:03.640 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-09-10T23:33:03.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:03.640 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-09-10T23:33:03.640 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-09-10T23:33:03.641 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 4716, 2024-09-10T23:33:03.641 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-09-10T23:33:03.641 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-09-10T23:33:03.641 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-09-10T23:33:03.641 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 4716 2024-09-10T23:33:03.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:03.641 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:33:03.641 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:33:03.641 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:33:03.641 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1246197192 2024-09-10T23:33:03.641 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:03.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:03.641 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:33:03.641 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:33:03.642 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:33:03.642 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1246197192 2024-09-10T23:33:03.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:03.642 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.4716 192.168.0.1:0/1246197192", 2024-09-10T23:33:03.642 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/1246197192", 2024-09-10T23:33:03.642 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-09-10T23:33:03.642 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 8, 2024-09-10T23:33:03.642 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 22, 2024-09-10T23:33:03.642 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:33:03.642 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-09-10T23:33:03.642 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-09-10T23:33:03.642 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:33:03.654 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T23:33:03.654 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:33:03.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.793+0000 7f632c200640 1 -- 172.21.15.94:0/1443497511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63240b2bb0 msgr2=0x7f63240d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:03.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.793+0000 7f632c200640 1 --2- 172.21.15.94:0/1443497511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63240b2bb0 0x7f63240d0270 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f632415f9a0 tx=0x7f631001c7a0 comp rx=0 tx=0).stop 2024-09-10T23:33:03.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.793+0000 7f632c200640 1 -- 172.21.15.94:0/1443497511 shutdown_connections 2024-09-10T23:33:03.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.793+0000 7f632c200640 1 --2- 172.21.15.94:0/1443497511 >> v2:172.21.15.181:3300/0 conn(0x7f63240af310 0x7f63240af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:03.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.793+0000 7f632c200640 1 --2- 172.21.15.94:0/1443497511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63240b2bb0 0x7f63240d0270 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:03.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.793+0000 7f632c200640 1 -- 172.21.15.94:0/1443497511 >> 172.21.15.94:0/1443497511 conn(0x7f63240d1c60 msgr2=0x7f63240d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:03.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.793+0000 7f632c200640 1 -- 172.21.15.94:0/1443497511 shutdown_connections 2024-09-10T23:33:03.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.793+0000 7f632c200640 1 -- 172.21.15.94:0/1443497511 wait complete. 2024-09-10T23:33:03.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.793+0000 7f632c200640 1 Processor -- start 2024-09-10T23:33:03.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.793+0000 7f632c200640 1 -- start start 2024-09-10T23:33:03.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f632c200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63240af310 0x7f6324152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f632c200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63240b2bb0 0x7f63240c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f632c200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f632415f5c0 0x7f63240c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f632c200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f63241690b0 con 0x7f63240b2bb0 2024-09-10T23:33:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f632c200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6324168f30 con 0x7f63240af310 2024-09-10T23:33:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f632c200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6324169230 con 0x7f632415f5c0 2024-09-10T23:33:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f6328e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63240b2bb0 0x7f63240c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f6328e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63240b2bb0 0x7f63240c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38688/0 (socket says 172.21.15.94:38688) 2024-09-10T23:33:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f6328e00640 1 -- 172.21.15.94:0/2857936312 learned_addr learned my addr 172.21.15.94:0/2857936312 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f6328e00640 1 -- 172.21.15.94:0/2857936312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f632415f5c0 msgr2=0x7f63240c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f6328e00640 1 --2- 172.21.15.94:0/2857936312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f632415f5c0 0x7f63240c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f6328e00640 1 -- 172.21.15.94:0/2857936312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63240af310 msgr2=0x7f6324152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f6329800640 1 --2- 172.21.15.94:0/2857936312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63240af310 0x7f6324152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f6328e00640 1 --2- 172.21.15.94:0/2857936312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63240af310 0x7f6324152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:03.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f6328e00640 1 -- 172.21.15.94:0/2857936312 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63240c6400 con 0x7f63240b2bb0 2024-09-10T23:33:03.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f6329800640 1 --2- 172.21.15.94:0/2857936312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63240af310 0x7f6324152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:03.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f6328e00640 1 --2- 172.21.15.94:0/2857936312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63240b2bb0 0x7f63240c5820 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f63100b0040 tx=0x7f63100a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:03.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f6321600640 1 -- 172.21.15.94:0/2857936312 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63100b2040 con 0x7f63240b2bb0 2024-09-10T23:33:03.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f6321600640 1 -- 172.21.15.94:0/2857936312 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63100043e0 con 0x7f63240b2bb0 2024-09-10T23:33:03.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f6321600640 1 -- 172.21.15.94:0/2857936312 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310005280 con 0x7f63240b2bb0 2024-09-10T23:33:03.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f632c200640 1 -- 172.21.15.94:0/2857936312 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632415ae10 con 0x7f63240b2bb0 2024-09-10T23:33:03.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f632c200640 1 -- 172.21.15.94:0/2857936312 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f632415b3d0 con 0x7f63240b2bb0 2024-09-10T23:33:03.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f632c200640 1 -- 172.21.15.94:0/2857936312 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f632415b5d0 con 0x7f63240b2bb0 2024-09-10T23:33:03.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.797+0000 7f6321600640 1 -- 172.21.15.94:0/2857936312 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6310005420 con 0x7f63240b2bb0 2024-09-10T23:33:03.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.801+0000 7f6321600640 1 --2- 172.21.15.94:0/2857936312 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f63040725e0 0x7f6304074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:03.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.801+0000 7f6321600640 1 -- 172.21.15.94:0/2857936312 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f631012cfc0 con 0x7f63240b2bb0 2024-09-10T23:33:03.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.801+0000 7f6329800640 1 --2- 172.21.15.94:0/2857936312 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f63040725e0 0x7f6304074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:03.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.801+0000 7f6329800640 1 --2- 172.21.15.94:0/2857936312 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f63040725e0 0x7f6304074aa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f631800a830 tx=0x7f6318008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:03.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:03.805+0000 7f6321600640 1 -- 172.21.15.94:0/2857936312 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f632415b5d0 con 0x7f63240b2bb0 2024-09-10T23:33:04.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04.005+0000 7f632c200640 1 -- 172.21.15.94:0/2857936312 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f63240af7a0 con 0x7f63240b2bb0 2024-09-10T23:33:04.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04.005+0000 7f6321600640 1 -- 172.21.15.94:0/2857936312 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f63100164b0 con 0x7f63240b2bb0 2024-09-10T23:33:04.008 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:33:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04.005+0000 7f632c200640 1 -- 172.21.15.94:0/2857936312 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f63040725e0 msgr2=0x7f6304074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04.005+0000 7f632c200640 1 --2- 172.21.15.94:0/2857936312 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f63040725e0 0x7f6304074aa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f631800a830 tx=0x7f6318008040 comp rx=0 tx=0).stop 2024-09-10T23:33:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04.005+0000 7f632c200640 1 -- 172.21.15.94:0/2857936312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63240b2bb0 msgr2=0x7f63240c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04.005+0000 7f632c200640 1 --2- 172.21.15.94:0/2857936312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63240b2bb0 0x7f63240c5820 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f63100b0040 tx=0x7f63100a5690 comp rx=0 tx=0).stop 2024-09-10T23:33:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04.005+0000 7f632c200640 1 -- 172.21.15.94:0/2857936312 shutdown_connections 2024-09-10T23:33:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04.005+0000 7f632c200640 1 --2- 172.21.15.94:0/2857936312 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f63040725e0 0x7f6304074aa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04.005+0000 7f632c200640 1 --2- 172.21.15.94:0/2857936312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f632415f5c0 0x7f63240c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04.005+0000 7f632c200640 1 --2- 172.21.15.94:0/2857936312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63240b2bb0 0x7f63240c5820 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04.005+0000 7f632c200640 1 --2- 172.21.15.94:0/2857936312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63240af310 0x7f6324152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04.005+0000 7f632c200640 1 -- 172.21.15.94:0/2857936312 >> 172.21.15.94:0/2857936312 conn(0x7f63240d1c60 msgr2=0x7f632415e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04.009+0000 7f632c200640 1 -- 172.21.15.94:0/2857936312 shutdown_connections 2024-09-10T23:33:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04.009+0000 7f632c200640 1 -- 172.21.15.94:0/2857936312 wait complete. 2024-09-10T23:33:04.031 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:33:04.031 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:33:04.031 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T23:33:04.031 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-10T23:33:04.031 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T23:33:04.031 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:33:04.031 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:33:04.031 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:33:04.031 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-10T23:33:04.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:04.032 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T23:33:04.041 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:33:04.041 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:33:04.041 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:33:04.041 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:04.041 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T23:33:04.041 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:04.041 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:04.041 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:04.041 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T23:33:04.041 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:04.041 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:04.041 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T23:33:04.042 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:04.042 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:04.042 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:33:04.042 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:04.042 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:33:04.042 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42427sec preferred_lft 42427sec 2024-09-10T23:33:04.042 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-09-10T23:33:04.042 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:04.042 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:33:04.042 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:ed:7d:46:db brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:04.042 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:33:04.042 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:04.042 INFO:teuthology.orchestra.run.smithi094.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:33:04.042 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:04.043 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:33:04.043 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:04.043 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:33:04.043 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:04.043 INFO:teuthology.orchestra.run.smithi094.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:33:04.043 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:04.043 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:33:04.043 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:04.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:04.043 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:33:04.099 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T23:33:04.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:04.101 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T23:33:04.153 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T23:33:04.153 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:33:04.153 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:33:04.154 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:33:04.154 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:33:04.154 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:33:04.154 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:33:04.154 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:04.154 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:04.154 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:04.154 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:33:04.154 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:04.155 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:33:04.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:33:04.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:33:04.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:33:04.155 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:33:04.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:04.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:04.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:04.291 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T23:33:04.342 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T23:33:04.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:04.343 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-09-10T23:33:04.397 INFO:teuthology.orchestra.run.smithi094.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-10T23:33:04.398 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:04.398 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:33:04.398 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:33:04.398 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-10T23:33:04.398 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:33:04.425 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:33:04.446 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:33:04.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:04.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:04.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:04.526 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T23:33:04.593 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:33:04.594 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:33:04.594 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:33:04.594 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:04.594 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T23:33:04.594 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:04.594 INFO:teuthology.orchestra.run.smithi094.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:33:04.594 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 1a:1b:04:b9:8d:ed brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T23:33:04.594 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T23:33:04.594 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:04.594 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::181b:4ff:feb9:8ded/64 scope link 2024-09-10T23:33:04.594 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:04.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:04.596 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T23:33:04.661 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:33:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:33:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:33:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T23:33:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:04.662 INFO:teuthology.orchestra.run.smithi094.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:33:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 1a:1b:04:b9:8d:ed brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T23:33:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T23:33:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::181b:4ff:feb9:8ded/64 scope link 2024-09-10T23:33:04.662 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:04.663 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-10T23:33:04.664 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:04.664 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:33:04.664 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-10T23:33:04.664 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T23:33:04.664 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-10T23:33:04.664 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-10T23:33:04.664 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-10T23:33:04.664 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:33:04.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:04.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:04.846 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:04.846 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:33:04.846 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.1 up 2024-09-10T23:33:04.846 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T23:33:04.846 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:33:04.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:04.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:04.930 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-09-10T23:33:04.930 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:33:04.930 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-10T23:33:04.986 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-10T23:33:04.986 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:33:04.986 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-10T23:33:05.037 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:33:05.102 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:33:05.155 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:33:05.155 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:33:05.155 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:33:05.155 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:33:05.155 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:33:05.155 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:33:05.155 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:33:05.156 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:33:05.156 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:33:05.156 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:33:05.156 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:33:05.156 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:33:05.156 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:33:05.156 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:33:05.156 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:33:05.156 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:33:05.156 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:33:05.156 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:33:05.156 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:33:05.156 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:33:05.157 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:33:05.157 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:33:05.157 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:33:05.157 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:33:05.157 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:33:05.157 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:33:05.157 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:33:05.157 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:33:05.157 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:33:05.157 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:05.157 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:05.157 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:05.157 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:33:05.157 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:33:05.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:05.158 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:33:05.209 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T23:33:05.209 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [44] 2024-09-10T23:33:05.209 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-09-10T23:33:05.252 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:33:05.272 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:33:05.327 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:33:05.327 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:33:05.327 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:33:05.327 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:33:05.328 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:33:05.328 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:33:05.328 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:33:05.328 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:33:05.328 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:33:05.328 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:33:05.328 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:33:05.328 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:33:05.328 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:33:05.328 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:33:05.328 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:33:05.328 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:33:05.329 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:33:05.329 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:33:05.329 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:33:05.329 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:33:05.329 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:33:05.329 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:33:05.329 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:33:05.329 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:33:05.329 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:33:05.329 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:33:05.329 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:33:05.329 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:33:05.330 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:33:05.330 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:05.330 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:05.330 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:05.330 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:33:05.330 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:33:05.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:05.331 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:33:05.345 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi094.stderr:2024-09-10T23:33:05.341+0000 7f9075738580 -1 init, newargv = 0x55db2c425a30 newargc=15 2024-09-10T23:33:05.345 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi094.stderr:ceph-fuse[35651]: starting ceph client 2024-09-10T23:33:05.352 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi094.stderr:ceph-fuse[35651]: starting fuse 2024-09-10T23:33:05.354 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T23:33:05.354 INFO:teuthology.orchestra.run.smithi094.stdout:45 2024-09-10T23:33:05.354 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44, 45] 2024-09-10T23:33:05.354 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:33:05.354 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-09-10T23:33:05.354 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-09-10T23:33:05.354 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-09-10T23:33:05.354 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-09-10T23:33:05.354 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:33:05.354 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-09-10T23:33:05.354 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-09-10T23:33:05.355 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-09-10T23:33:05.355 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-09-10T23:33:05.355 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:33:05.355 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-09-10T23:33:05.355 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-09-10T23:33:05.355 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-09-10T23:33:05.355 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-09-10T23:33:05.355 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:33:05.355 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-09-10T23:33:05.355 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:33:05.355 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:33:05.355 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:33:05.355 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-09-10T23:33:05.355 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-09-10T23:33:05.356 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-09-10T23:33:05.356 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:33:05.356 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:33:05.356 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.2")) 2024-09-10T23:33:05.356 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-09-10T23:33:05.477 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.2.35651.asok 2024-09-10T23:33:05.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:05.488 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.35651.asok 2024-09-10T23:33:05.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:05.489 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.35651.asok status 2024-09-10T23:33:05.659 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:33:05.659 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-09-10T23:33:05.659 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:33:05.659 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:33:05.659 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "2", 2024-09-10T23:33:05.659 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-09-10T23:33:05.660 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-09-10T23:33:05.660 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "35651", 2024-09-10T23:33:05.660 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-09-10T23:33:05.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:05.660 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-09-10T23:33:05.660 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-09-10T23:33:05.660 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 4728, 2024-09-10T23:33:05.660 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-09-10T23:33:05.660 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-09-10T23:33:05.660 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-09-10T23:33:05.660 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 4728 2024-09-10T23:33:05.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:05.661 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:33:05.661 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:33:05.661 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.2:0", 2024-09-10T23:33:05.661 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2625711880 2024-09-10T23:33:05.661 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:05.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:05.661 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:33:05.662 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:33:05.662 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.2:0", 2024-09-10T23:33:05.662 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2625711880 2024-09-10T23:33:05.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:05.662 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.4728 192.168.0.2:0/2625711880", 2024-09-10T23:33:05.662 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.2:0/2625711880", 2024-09-10T23:33:05.662 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-09-10T23:33:05.662 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 8, 2024-09-10T23:33:05.662 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 22, 2024-09-10T23:33:05.662 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:33:05.662 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-09-10T23:33:05.662 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-09-10T23:33:05.663 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:33:05.674 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T23:33:05.674 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:33:05.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.813+0000 7f3bb0200640 1 -- 172.21.15.94:0/3860424155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba80d04b0 msgr2=0x7f3ba815e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:05.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.813+0000 7f3bb0200640 1 --2- 172.21.15.94:0/3860424155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba80d04b0 0x7f3ba815e770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3b94009900 tx=0x7f3b9401c8b0 comp rx=0 tx=0).stop 2024-09-10T23:33:05.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.813+0000 7f3bb0200640 1 -- 172.21.15.94:0/3860424155 shutdown_connections 2024-09-10T23:33:05.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.813+0000 7f3bb0200640 1 --2- 172.21.15.94:0/3860424155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ba815ecb0 0x7f3ba806f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:05.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.813+0000 7f3bb0200640 1 --2- 172.21.15.94:0/3860424155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba80d04b0 0x7f3ba815e770 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:05.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.813+0000 7f3bb0200640 1 -- 172.21.15.94:0/3860424155 >> 172.21.15.94:0/3860424155 conn(0x7f3ba806cdf0 msgr2=0x7f3ba806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:05.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.813+0000 7f3bb0200640 1 -- 172.21.15.94:0/3860424155 shutdown_connections 2024-09-10T23:33:05.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.813+0000 7f3bb0200640 1 -- 172.21.15.94:0/3860424155 wait complete. 2024-09-10T23:33:05.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.813+0000 7f3bb0200640 1 Processor -- start 2024-09-10T23:33:05.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.813+0000 7f3bb0200640 1 -- start start 2024-09-10T23:33:05.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.813+0000 7f3bb0200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba8073ff0 0x7f3ba814b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:05.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.813+0000 7f3bb0200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba80d04b0 0x7f3ba814a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:05.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.813+0000 7f3bb0200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ba815ecb0 0x7f3ba81f8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:05.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.813+0000 7f3bb0200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba8167930 con 0x7f3ba8073ff0 2024-09-10T23:33:05.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.813+0000 7f3bb0200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba81677b0 con 0x7f3ba80d04b0 2024-09-10T23:33:05.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.813+0000 7f3bb0200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba8167ab0 con 0x7f3ba815ecb0 2024-09-10T23:33:05.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3bad800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba8073ff0 0x7f3ba814b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:05.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3bad800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba8073ff0 0x7f3ba814b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38708/0 (socket says 172.21.15.94:38708) 2024-09-10T23:33:05.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3bad800640 1 -- 172.21.15.94:0/341507106 learned_addr learned my addr 172.21.15.94:0/341507106 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:05.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3bae200640 1 --2- 172.21.15.94:0/341507106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ba815ecb0 0x7f3ba81f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:05.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3bace00640 1 --2- 172.21.15.94:0/341507106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba80d04b0 0x7f3ba814a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:05.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3bad800640 1 -- 172.21.15.94:0/341507106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ba815ecb0 msgr2=0x7f3ba81f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:05.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3bad800640 1 --2- 172.21.15.94:0/341507106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ba815ecb0 0x7f3ba81f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:05.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3bad800640 1 -- 172.21.15.94:0/341507106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba80d04b0 msgr2=0x7f3ba814a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:05.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3bad800640 1 --2- 172.21.15.94:0/341507106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba80d04b0 0x7f3ba814a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:05.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3bad800640 1 -- 172.21.15.94:0/341507106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba81f5c90 con 0x7f3ba8073ff0 2024-09-10T23:33:05.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3bad800640 1 --2- 172.21.15.94:0/341507106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba8073ff0 0x7f3ba814b1c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00c500 tx=0x7f3b9c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:05.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3ba5600640 1 -- 172.21.15.94:0/341507106 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c017070 con 0x7f3ba8073ff0 2024-09-10T23:33:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3ba5600640 1 -- 172.21.15.94:0/341507106 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b9c0053c0 con 0x7f3ba8073ff0 2024-09-10T23:33:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3bb0200640 1 -- 172.21.15.94:0/341507106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba814e8c0 con 0x7f3ba8073ff0 2024-09-10T23:33:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3ba5600640 1 -- 172.21.15.94:0/341507106 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c012890 con 0x7f3ba8073ff0 2024-09-10T23:33:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3bb0200640 1 -- 172.21.15.94:0/341507106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba80af2c0 con 0x7f3ba8073ff0 2024-09-10T23:33:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3bb0200640 1 -- 172.21.15.94:0/341507106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba815e770 con 0x7f3ba8073ff0 2024-09-10T23:33:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.817+0000 7f3ba5600640 1 -- 172.21.15.94:0/341507106 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3b9c003600 con 0x7f3ba8073ff0 2024-09-10T23:33:05.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.821+0000 7f3ba5600640 1 --2- 172.21.15.94:0/341507106 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3b880725e0 0x7f3b88074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:05.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.821+0000 7f3ba5600640 1 -- 172.21.15.94:0/341507106 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3b9c092890 con 0x7f3ba8073ff0 2024-09-10T23:33:05.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.821+0000 7f3bace00640 1 --2- 172.21.15.94:0/341507106 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3b880725e0 0x7f3b88074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:05.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.821+0000 7f3bace00640 1 --2- 172.21.15.94:0/341507106 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3b880725e0 0x7f3b88074aa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3b940af000 tx=0x7f3b9401b0f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:05.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:05.825+0000 7f3ba5600640 1 -- 172.21.15.94:0/341507106 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3b9c05d890 con 0x7f3ba8073ff0 2024-09-10T23:33:06.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:06.021+0000 7f3bb0200640 1 -- 172.21.15.94:0/341507106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3ba8069850 con 0x7f3ba8073ff0 2024-09-10T23:33:06.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:06.025+0000 7f3ba5600640 1 -- 172.21.15.94:0/341507106 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f3b9c0622c0 con 0x7f3ba8073ff0 2024-09-10T23:33:06.027 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:33:06.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:06.025+0000 7f3bb0200640 1 -- 172.21.15.94:0/341507106 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3b880725e0 msgr2=0x7f3b88074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:06.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:06.025+0000 7f3bb0200640 1 --2- 172.21.15.94:0/341507106 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3b880725e0 0x7f3b88074aa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3b940af000 tx=0x7f3b9401b0f0 comp rx=0 tx=0).stop 2024-09-10T23:33:06.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:06.025+0000 7f3bb0200640 1 -- 172.21.15.94:0/341507106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba8073ff0 msgr2=0x7f3ba814b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:06.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:06.025+0000 7f3bb0200640 1 --2- 172.21.15.94:0/341507106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba8073ff0 0x7f3ba814b1c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00c500 tx=0x7f3b9c00c9c0 comp rx=0 tx=0).stop 2024-09-10T23:33:06.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:06.025+0000 7f3bb0200640 1 -- 172.21.15.94:0/341507106 shutdown_connections 2024-09-10T23:33:06.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:06.025+0000 7f3bb0200640 1 --2- 172.21.15.94:0/341507106 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3b880725e0 0x7f3b88074aa0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:06.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:06.025+0000 7f3bb0200640 1 --2- 172.21.15.94:0/341507106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ba80d04b0 0x7f3ba814a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:06.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:06.025+0000 7f3bb0200640 1 --2- 172.21.15.94:0/341507106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ba815ecb0 0x7f3ba81f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:06.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:06.025+0000 7f3bb0200640 1 --2- 172.21.15.94:0/341507106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ba8073ff0 0x7f3ba814b1c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:06.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:06.025+0000 7f3bb0200640 1 -- 172.21.15.94:0/341507106 >> 172.21.15.94:0/341507106 conn(0x7f3ba806cdf0 msgr2=0x7f3ba80af640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:06.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:06.025+0000 7f3bb0200640 1 -- 172.21.15.94:0/341507106 shutdown_connections 2024-09-10T23:33:06.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:06.025+0000 7f3bb0200640 1 -- 172.21.15.94:0/341507106 wait complete. 2024-09-10T23:33:06.048 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:33:06.048 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:33:06.048 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-09-10T23:33:06.048 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T23:33:06.048 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:06.048 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:33:06.048 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:33:06.048 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:33:06.048 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:33:06.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:06.049 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-09-10T23:33:06.058 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:33:06.059 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:33:06.059 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:33:06.059 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:06.059 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-09-10T23:33:06.059 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:06.059 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:06.059 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:06.059 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-09-10T23:33:06.059 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:06.059 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:06.059 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-09-10T23:33:06.059 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:06.060 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:06.060 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:33:06.060 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:06.060 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:33:06.060 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42429sec preferred_lft 42429sec 2024-09-10T23:33:06.060 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-09-10T23:33:06.060 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:06.060 INFO:teuthology.orchestra.run.smithi181.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:33:06.060 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 02:42:3b:2e:09:2d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:06.060 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:33:06.060 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:06.061 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T23:33:06.061 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:06.061 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:33:06.061 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add name ceph-brx type bridge 2024-09-10T23:33:06.061 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr flush dev ceph-brx 2024-09-10T23:33:06.062 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set ceph-brx up 2024-09-10T23:33:06.062 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T23:33:06.062 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:33:06.157 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:06.221 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:06.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:06.228 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:33:06.246 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-09-10T23:33:06.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:06.248 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-09-10T23:33:06.320 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-09-10T23:33:06.320 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:33:06.320 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:33:06.320 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:33:06.321 INFO:teuthology.orchestra.run.smithi181.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:33:06.321 INFO:teuthology.orchestra.run.smithi181.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:33:06.321 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:33:06.321 INFO:teuthology.orchestra.run.smithi181.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:06.321 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:06.321 INFO:teuthology.orchestra.run.smithi181.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:06.321 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:33:06.321 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:06.321 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:33:06.321 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:33:06.322 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:33:06.322 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:33:06.322 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:33:06.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:06.446 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:06.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:06.453 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-09-10T23:33:06.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:06.462 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list-id 2024-09-10T23:33:06.513 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:06.513 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:33:06.513 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:33:06.513 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-10T23:33:06.513 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:33:06.609 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:06.640 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:06.646 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-10T23:33:06.647 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:06.647 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:33:06.647 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-10T23:33:06.647 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T23:33:06.647 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-10T23:33:06.647 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-10T23:33:06.647 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-10T23:33:06.647 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:33:06.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:06.806 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:06.813 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:06.813 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:33:06.813 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-09-10T23:33:06.813 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T23:33:06.813 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:33:06.859 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:06.891 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:06.898 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T23:33:06.898 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:33:06.898 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:06.953 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:33:06.953 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:33:06.953 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:07.004 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:33:07.064 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:33:07.114 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:33:07.114 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:33:07.114 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:33:07.114 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:33:07.114 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:33:07.114 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:33:07.114 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:33:07.114 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:33:07.114 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:33:07.114 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:33:07.115 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:33:07.115 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:33:07.115 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:33:07.115 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:33:07.115 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:33:07.115 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:33:07.115 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:33:07.115 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:33:07.115 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:33:07.115 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:33:07.115 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:33:07.115 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:33:07.115 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:33:07.115 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:33:07.115 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:33:07.116 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:33:07.116 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:33:07.116 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:33:07.116 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:33:07.116 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:33:07.116 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:33:07.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:07.117 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:33:07.168 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:33:07.169 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T23:33:07.211 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:33:07.231 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:33:07.286 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:33:07.286 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:33:07.286 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:33:07.286 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:33:07.287 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:33:07.287 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:33:07.287 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:33:07.287 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:33:07.287 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:33:07.287 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:33:07.287 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:33:07.287 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:33:07.287 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:33:07.287 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:33:07.287 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:33:07.287 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:33:07.287 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:33:07.288 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:33:07.288 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:33:07.288 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:33:07.288 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:33:07.288 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:33:07.288 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:33:07.288 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:33:07.288 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:33:07.288 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:33:07.288 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:33:07.288 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:33:07.288 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:33:07.288 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:33:07.289 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:33:07.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:07.289 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:33:07.306 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:2024-09-10T23:33:07.306+0000 7fd6bab85580 -1 init, newargv = 0x55f7716ef240 newargc=15 2024-09-10T23:33:07.306 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[34131]: starting ceph client 2024-09-10T23:33:07.314 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[34131]: starting fuse 2024-09-10T23:33:07.315 INFO:teuthology.orchestra.run.smithi181.stdout:44 2024-09-10T23:33:07.315 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:33:07.316 DEBUG:teuthology.orchestra.run.smithi181:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:33:07.316 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2024-09-10T23:33:07.316 DEBUG:teuthology.orchestra.run.smithi181:> import re 2024-09-10T23:33:07.316 DEBUG:teuthology.orchestra.run.smithi181:> import os 2024-09-10T23:33:07.316 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2024-09-10T23:33:07.316 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:33:07.316 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2024-09-10T23:33:07.316 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T23:33:07.316 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2024-09-10T23:33:07.316 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T23:33:07.316 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:33:07.316 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2024-09-10T23:33:07.316 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2024-09-10T23:33:07.316 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2024-09-10T23:33:07.317 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2024-09-10T23:33:07.317 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:33:07.317 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2024-09-10T23:33:07.317 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:33:07.317 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:33:07.317 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:33:07.317 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2024-09-10T23:33:07.317 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2024-09-10T23:33:07.317 DEBUG:teuthology.orchestra.run.smithi181:> return f 2024-09-10T23:33:07.317 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:33:07.317 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:33:07.317 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.1")) 2024-09-10T23:33:07.317 DEBUG:teuthology.orchestra.run.smithi181:> ' 2024-09-10T23:33:07.441 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.1.34131.asok 2024-09-10T23:33:07.444 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:07.452 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34131.asok 2024-09-10T23:33:07.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:07.452 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34131.asok status 2024-09-10T23:33:07.624 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T23:33:07.625 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2024-09-10T23:33:07.625 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:33:07.625 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:33:07.625 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "1", 2024-09-10T23:33:07.625 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2024-09-10T23:33:07.625 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T23:33:07.625 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "34131", 2024-09-10T23:33:07.625 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2024-09-10T23:33:07.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:33:07.625 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2024-09-10T23:33:07.625 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2024-09-10T23:33:07.625 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 4717, 2024-09-10T23:33:07.625 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2024-09-10T23:33:07.626 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2024-09-10T23:33:07.626 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2024-09-10T23:33:07.626 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 4717 2024-09-10T23:33:07.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:33:07.626 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:33:07.626 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:33:07.626 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:33:07.626 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 2306828582 2024-09-10T23:33:07.626 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T23:33:07.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:33:07.626 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:33:07.626 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:33:07.626 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:33:07.626 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 2306828582 2024-09-10T23:33:07.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:33:07.627 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.4717 192.168.0.1:0/2306828582", 2024-09-10T23:33:07.627 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.1:0/2306828582", 2024-09-10T23:33:07.627 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2024-09-10T23:33:07.627 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 8, 2024-09-10T23:33:07.627 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 22, 2024-09-10T23:33:07.627 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:33:07.627 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2024-09-10T23:33:07.627 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2024-09-10T23:33:07.627 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T23:33:07.639 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T23:33:07.639 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:33:07.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.789+0000 7f2068800640 1 -- 172.21.15.94:0/2936317565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20600af310 msgr2=0x7f20600af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:07.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.789+0000 7f2068800640 1 --2- 172.21.15.94:0/2936317565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20600af310 0x7f20600af7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2060073820 tx=0x7f205c01cc60 comp rx=0 tx=0).stop 2024-09-10T23:33:07.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.789+0000 7f2068800640 1 -- 172.21.15.94:0/2936317565 shutdown_connections 2024-09-10T23:33:07.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.789+0000 7f2068800640 1 --2- 172.21.15.94:0/2936317565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20600af310 0x7f20600af7a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:07.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.789+0000 7f2068800640 1 --2- 172.21.15.94:0/2936317565 >> v2:172.21.15.181:3300/0 conn(0x7f206015f5c0 0x7f20600cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:07.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.789+0000 7f2068800640 1 -- 172.21.15.94:0/2936317565 >> 172.21.15.94:0/2936317565 conn(0x7f20600d1c60 msgr2=0x7f20600d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:07.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.789+0000 7f2068800640 1 -- 172.21.15.94:0/2936317565 shutdown_connections 2024-09-10T23:33:07.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.789+0000 7f2068800640 1 -- 172.21.15.94:0/2936317565 wait complete. 2024-09-10T23:33:07.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2068800640 1 Processor -- start 2024-09-10T23:33:07.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2068800640 1 -- start start 2024-09-10T23:33:07.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2068800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20600af310 0x7f206014cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:07.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2068800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20600b2bb0 0x7f20600cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:07.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2068800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f206015f5c0 0x7f20600c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:07.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2068800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20601690b0 con 0x7f20600af310 2024-09-10T23:33:07.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2068800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2060168f30 con 0x7f20600b2bb0 2024-09-10T23:33:07.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2068800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2060169230 con 0x7f206015f5c0 2024-09-10T23:33:07.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2065e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20600af310 0x7f206014cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:07.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2065e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20600af310 0x7f206014cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38732/0 (socket says 172.21.15.94:38732) 2024-09-10T23:33:07.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2065e00640 1 -- 172.21.15.94:0/1442198309 learned_addr learned my addr 172.21.15.94:0/1442198309 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:07.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2066800640 1 --2- 172.21.15.94:0/1442198309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f206015f5c0 0x7f20600c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:07.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2065400640 1 --2- 172.21.15.94:0/1442198309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20600b2bb0 0x7f20600cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:07.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2065e00640 1 -- 172.21.15.94:0/1442198309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f206015f5c0 msgr2=0x7f20600c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:07.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2065e00640 1 --2- 172.21.15.94:0/1442198309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f206015f5c0 0x7f20600c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:07.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2065e00640 1 -- 172.21.15.94:0/1442198309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20600b2bb0 msgr2=0x7f20600cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:07.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2065e00640 1 --2- 172.21.15.94:0/1442198309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20600b2bb0 0x7f20600cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:07.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2065e00640 1 -- 172.21.15.94:0/1442198309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20600ca3c0 con 0x7f20600af310 2024-09-10T23:33:07.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2065e00640 1 --2- 172.21.15.94:0/1442198309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20600af310 0x7f206014cd50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f205400d780 tx=0x7f205400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:07.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f205a000640 1 -- 172.21.15.94:0/1442198309 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054017070 con 0x7f20600af310 2024-09-10T23:33:07.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f205a000640 1 -- 172.21.15.94:0/1442198309 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f205400de90 con 0x7f20600af310 2024-09-10T23:33:07.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f205a000640 1 -- 172.21.15.94:0/1442198309 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20540099d0 con 0x7f20600af310 2024-09-10T23:33:07.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2068800640 1 -- 172.21.15.94:0/1442198309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20600ca6a0 con 0x7f20600af310 2024-09-10T23:33:07.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.793+0000 7f2068800640 1 -- 172.21.15.94:0/1442198309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20600cac60 con 0x7f20600af310 2024-09-10T23:33:07.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.797+0000 7f2068800640 1 -- 172.21.15.94:0/1442198309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20600af7a0 con 0x7f20600af310 2024-09-10T23:33:07.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.797+0000 7f205a000640 1 -- 172.21.15.94:0/1442198309 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2054009b70 con 0x7f20600af310 2024-09-10T23:33:07.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.797+0000 7f205a000640 1 --2- 172.21.15.94:0/1442198309 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f20440725e0 0x7f2044074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:07.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.797+0000 7f205a000640 1 -- 172.21.15.94:0/1442198309 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2054093540 con 0x7f20600af310 2024-09-10T23:33:07.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.797+0000 7f2065400640 1 --2- 172.21.15.94:0/1442198309 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f20440725e0 0x7f2044074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:07.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.797+0000 7f2065400640 1 --2- 172.21.15.94:0/1442198309 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f20440725e0 0x7f2044074aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f206015ccc0 tx=0x7f2050009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:07.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:07.801+0000 7f205a000640 1 -- 172.21.15.94:0/1442198309 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2054020020 con 0x7f20600af310 2024-09-10T23:33:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:08.005+0000 7f2068800640 1 -- 172.21.15.94:0/1442198309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f20600749c0 con 0x7f20600af310 2024-09-10T23:33:08.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:08.005+0000 7f205a000640 1 -- 172.21.15.94:0/1442198309 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f20540672a0 con 0x7f20600af310 2024-09-10T23:33:08.010 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:33:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:08.009+0000 7f2068800640 1 -- 172.21.15.94:0/1442198309 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f20440725e0 msgr2=0x7f2044074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:08.009+0000 7f2068800640 1 --2- 172.21.15.94:0/1442198309 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f20440725e0 0x7f2044074aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f206015ccc0 tx=0x7f2050009040 comp rx=0 tx=0).stop 2024-09-10T23:33:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:08.009+0000 7f2068800640 1 -- 172.21.15.94:0/1442198309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20600af310 msgr2=0x7f206014cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:08.009+0000 7f2068800640 1 --2- 172.21.15.94:0/1442198309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20600af310 0x7f206014cd50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f205400d780 tx=0x7f205400dc40 comp rx=0 tx=0).stop 2024-09-10T23:33:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:08.009+0000 7f2068800640 1 -- 172.21.15.94:0/1442198309 shutdown_connections 2024-09-10T23:33:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:08.009+0000 7f2068800640 1 --2- 172.21.15.94:0/1442198309 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f20440725e0 0x7f2044074aa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:08.009+0000 7f2068800640 1 --2- 172.21.15.94:0/1442198309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20600b2bb0 0x7f20600cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:08.009+0000 7f2068800640 1 --2- 172.21.15.94:0/1442198309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f206015f5c0 0x7f20600c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:08.009+0000 7f2068800640 1 --2- 172.21.15.94:0/1442198309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20600af310 0x7f206014cd50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:08.009+0000 7f2068800640 1 -- 172.21.15.94:0/1442198309 >> 172.21.15.94:0/1442198309 conn(0x7f20600d1c60 msgr2=0x7f206015cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:08.009+0000 7f2068800640 1 -- 172.21.15.94:0/1442198309 shutdown_connections 2024-09-10T23:33:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:08.009+0000 7f2068800640 1 -- 172.21.15.94:0/1442198309 wait complete. 2024-09-10T23:33:08.032 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:33:08.032 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:33:08.032 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-09-10T23:33:08.032 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-10T23:33:08.032 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T23:33:08.032 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:33:08.032 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:33:08.032 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:33:08.033 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-10T23:33:08.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:08.033 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-09-10T23:33:08.042 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:33:08.042 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:33:08.042 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:33:08.042 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:08.042 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-09-10T23:33:08.042 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:08.043 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:08.043 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:08.043 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-09-10T23:33:08.043 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:08.043 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:08.043 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-09-10T23:33:08.043 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:08.043 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:08.043 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:33:08.043 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:08.043 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:33:08.043 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42427sec preferred_lft 42427sec 2024-09-10T23:33:08.043 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-09-10T23:33:08.044 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:08.044 INFO:teuthology.orchestra.run.smithi181.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:33:08.044 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 02:42:3b:2e:09:2d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:08.044 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:33:08.044 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:08.044 INFO:teuthology.orchestra.run.smithi181.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:33:08.044 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:08.044 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:33:08.044 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:08.045 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2024-09-10T23:33:08.045 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:08.045 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:33:08.045 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:33:08.045 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-09-10T23:33:08.045 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:08.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:08.046 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:33:08.110 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-09-10T23:33:08.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:08.113 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-09-10T23:33:08.169 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-09-10T23:33:08.169 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:33:08.169 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:33:08.169 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:33:08.169 INFO:teuthology.orchestra.run.smithi181.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:33:08.169 INFO:teuthology.orchestra.run.smithi181.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:33:08.169 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:33:08.169 INFO:teuthology.orchestra.run.smithi181.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:08.169 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:08.170 INFO:teuthology.orchestra.run.smithi181.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:08.170 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:33:08.170 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:08.170 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:33:08.170 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:33:08.170 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:33:08.170 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:33:08.170 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:33:08.258 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:08.298 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:08.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:08.304 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-09-10T23:33:08.313 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T23:33:08.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:08.314 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list-id 2024-09-10T23:33:08.368 INFO:teuthology.orchestra.run.smithi181.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-10T23:33:08.369 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:08.369 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:33:08.369 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:33:08.369 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-10T23:33:08.369 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:33:08.458 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:08.488 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:08.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:08.495 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T23:33:08.514 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:33:08.514 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:33:08.515 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:33:08.515 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:08.515 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-09-10T23:33:08.515 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:08.515 INFO:teuthology.orchestra.run.smithi181.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:33:08.515 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7a:5b:fb:f5:da:13 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T23:33:08.515 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T23:33:08.515 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:08.515 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::785b:fbff:fef5:da13/64 scope link 2024-09-10T23:33:08.515 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:08.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:08.517 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T23:33:08.577 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:33:08.577 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:33:08.577 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:33:08.577 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:08.577 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-09-10T23:33:08.577 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:08.577 INFO:teuthology.orchestra.run.smithi181.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:33:08.577 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 7a:5b:fb:f5:da:13 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T23:33:08.577 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T23:33:08.577 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:08.577 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::785b:fbff:fef5:da13/64 scope link 2024-09-10T23:33:08.578 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:08.578 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-10T23:33:08.579 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:08.579 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:33:08.579 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-10T23:33:08.579 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T23:33:08.579 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-10T23:33:08.579 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-10T23:33:08.579 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-10T23:33:08.579 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:33:08.670 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:08.754 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:08.761 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:08.761 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:33:08.761 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.1 up 2024-09-10T23:33:08.761 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T23:33:08.761 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:33:08.851 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:08.880 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:08.887 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-09-10T23:33:08.887 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:33:08.887 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-10T23:33:08.895 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-10T23:33:08.896 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:33:08.896 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-10T23:33:08.948 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:33:09.012 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:33:09.066 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:33:09.066 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:33:09.066 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:33:09.066 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:33:09.066 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:33:09.066 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:33:09.066 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:33:09.066 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:33:09.066 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:33:09.066 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:33:09.066 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:33:09.067 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:33:09.067 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:33:09.067 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:33:09.067 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:33:09.067 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:33:09.067 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:33:09.067 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:33:09.067 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:33:09.067 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:33:09.067 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:33:09.067 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:33:09.067 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:33:09.067 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:33:09.067 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:33:09.068 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:33:09.068 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:33:09.068 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:33:09.068 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:33:09.068 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:33:09.068 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:33:09.068 INFO:teuthology.orchestra.run.smithi181.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:09.068 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:33:09.068 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:33:09.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:09.069 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:33:09.125 INFO:teuthology.orchestra.run.smithi181.stdout:44 2024-09-10T23:33:09.125 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [44] 2024-09-10T23:33:09.125 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-09-10T23:33:09.175 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:33:09.195 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:33:09.250 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:33:09.250 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:33:09.250 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:33:09.250 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:33:09.250 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:33:09.250 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:33:09.251 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:33:09.251 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:33:09.251 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:33:09.251 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:33:09.251 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:33:09.251 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:33:09.251 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:33:09.251 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:33:09.251 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:33:09.251 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:33:09.251 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:33:09.251 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:33:09.251 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:33:09.251 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:33:09.251 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:33:09.251 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:33:09.252 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:33:09.252 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:33:09.252 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:33:09.252 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:33:09.252 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:33:09.252 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:33:09.252 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:33:09.252 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:33:09.252 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:33:09.252 INFO:teuthology.orchestra.run.smithi181.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:09.252 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:33:09.252 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:33:09.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:09.253 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:33:09.268 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi181.stderr:2024-09-10T23:33:09.266+0000 7fb79a931580 -1 init, newargv = 0x55ffd566edc0 newargc=15 2024-09-10T23:33:09.269 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi181.stderr:ceph-fuse[34244]: starting ceph client 2024-09-10T23:33:09.275 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi181.stderr:ceph-fuse[34244]: starting fuse 2024-09-10T23:33:09.277 INFO:teuthology.orchestra.run.smithi181.stdout:44 2024-09-10T23:33:09.278 INFO:teuthology.orchestra.run.smithi181.stdout:45 2024-09-10T23:33:09.278 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44, 45] 2024-09-10T23:33:09.278 DEBUG:teuthology.orchestra.run.smithi181:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:33:09.278 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2024-09-10T23:33:09.278 DEBUG:teuthology.orchestra.run.smithi181:> import re 2024-09-10T23:33:09.278 DEBUG:teuthology.orchestra.run.smithi181:> import os 2024-09-10T23:33:09.278 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2024-09-10T23:33:09.278 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:33:09.278 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2024-09-10T23:33:09.278 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-09-10T23:33:09.278 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2024-09-10T23:33:09.278 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-09-10T23:33:09.279 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:33:09.279 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2024-09-10T23:33:09.279 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2024-09-10T23:33:09.279 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2024-09-10T23:33:09.279 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2024-09-10T23:33:09.279 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:33:09.279 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2024-09-10T23:33:09.279 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:33:09.279 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:33:09.279 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:33:09.279 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2024-09-10T23:33:09.279 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2024-09-10T23:33:09.279 DEBUG:teuthology.orchestra.run.smithi181:> return f 2024-09-10T23:33:09.279 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:33:09.279 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:33:09.280 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.3")) 2024-09-10T23:33:09.280 DEBUG:teuthology.orchestra.run.smithi181:> ' 2024-09-10T23:33:09.405 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.3.34244.asok 2024-09-10T23:33:09.408 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:09.416 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.34244.asok 2024-09-10T23:33:09.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:09.416 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.34244.asok status 2024-09-10T23:33:09.589 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T23:33:09.589 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2024-09-10T23:33:09.589 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:33:09.589 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:33:09.589 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "3", 2024-09-10T23:33:09.589 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2024-09-10T23:33:09.589 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-09-10T23:33:09.589 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "34244", 2024-09-10T23:33:09.590 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2024-09-10T23:33:09.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:33:09.590 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2024-09-10T23:33:09.590 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2024-09-10T23:33:09.590 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 194453, 2024-09-10T23:33:09.590 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2024-09-10T23:33:09.590 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2024-09-10T23:33:09.590 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2024-09-10T23:33:09.590 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 194453 2024-09-10T23:33:09.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:33:09.590 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:33:09.590 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:33:09.590 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.2:0", 2024-09-10T23:33:09.590 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 3308211503 2024-09-10T23:33:09.590 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T23:33:09.591 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:33:09.591 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:33:09.591 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:33:09.591 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.2:0", 2024-09-10T23:33:09.591 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 3308211503 2024-09-10T23:33:09.591 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:33:09.591 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.194453 192.168.0.2:0/3308211503", 2024-09-10T23:33:09.591 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.2:0/3308211503", 2024-09-10T23:33:09.591 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2024-09-10T23:33:09.591 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 8, 2024-09-10T23:33:09.591 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 22, 2024-09-10T23:33:09.591 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:33:09.591 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2024-09-10T23:33:09.591 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2024-09-10T23:33:09.591 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T23:33:09.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:09.604 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-09-10T23:33:09.604 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:09.614 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-09-10T23:33:09.614 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:09.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:09.614 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:09.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:09.682 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-09-10T23:33:09.682 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.2 2024-09-10T23:33:09.735 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-09-10T23:33:09.735 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-09-10T23:33:09.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:09.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-10T23:33:09.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:09.802 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2024-09-10T23:33:09.802 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:09.811 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2024-09-10T23:33:09.811 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:09.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:09.812 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:09.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:09.876 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2024-09-10T23:33:09.876 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.3 2024-09-10T23:33:09.925 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2024-09-10T23:33:09.925 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-09-10T23:33:09.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:09.925 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-10T23:33:09.987 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-10T23:33:10.072 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2024-09-10T23:33:10.072 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T23:33:10.073 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:33:10.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.221+0000 7f0219600640 1 -- 172.21.15.94:0/3082264337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f021415e3a0 msgr2=0x7f021406e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:10.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.221+0000 7f0219600640 1 --2- 172.21.15.94:0/3082264337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f021415e3a0 0x7f021406e5c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f020c009bf0 tx=0x7f020c01cb20 comp rx=0 tx=0).stop 2024-09-10T23:33:10.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.221+0000 7f0219600640 1 -- 172.21.15.94:0/3082264337 shutdown_connections 2024-09-10T23:33:10.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.221+0000 7f0219600640 1 --2- 172.21.15.94:0/3082264337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f021415e3a0 0x7f021406e5c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.221+0000 7f0219600640 1 -- 172.21.15.94:0/3082264337 >> 172.21.15.94:0/3082264337 conn(0x7f021406cdf0 msgr2=0x7f021406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:10.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.221+0000 7f0219600640 1 -- 172.21.15.94:0/3082264337 shutdown_connections 2024-09-10T23:33:10.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.221+0000 7f0219600640 1 -- 172.21.15.94:0/3082264337 wait complete. 2024-09-10T23:33:10.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f0219600640 1 Processor -- start 2024-09-10T23:33:10.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f0219600640 1 -- start start 2024-09-10T23:33:10.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f0219600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f021415a060 0x7f02141f7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:10.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f0219600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f021415e3a0 0x7f021415b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:10.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f0219600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f021415e8a0 0x7f021415bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:10.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f0219600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0214073760 con 0x7f021415e3a0 2024-09-10T23:33:10.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f0219600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02140735e0 con 0x7f021415e8a0 2024-09-10T23:33:10.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f0219600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f02140738e0 con 0x7f021415a060 2024-09-10T23:33:10.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f020be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f021415e3a0 0x7f021415b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:10.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f020be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f021415e3a0 0x7f021415b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38736/0 (socket says 172.21.15.94:38736) 2024-09-10T23:33:10.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f020be00640 1 -- 172.21.15.94:0/1009035969 learned_addr learned my addr 172.21.15.94:0/1009035969 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:10.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f0212a00640 1 --2- 172.21.15.94:0/1009035969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f021415a060 0x7f02141f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:10.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f0213400640 1 --2- 172.21.15.94:0/1009035969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f021415e8a0 0x7f021415bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:10.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f020be00640 1 -- 172.21.15.94:0/1009035969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f021415a060 msgr2=0x7f02141f7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:10.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f020be00640 1 --2- 172.21.15.94:0/1009035969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f021415a060 0x7f02141f7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f020be00640 1 -- 172.21.15.94:0/1009035969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f021415e8a0 msgr2=0x7f021415bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:10.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f020be00640 1 --2- 172.21.15.94:0/1009035969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f021415e8a0 0x7f021415bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f020be00640 1 -- 172.21.15.94:0/1009035969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021415a8f0 con 0x7f021415e3a0 2024-09-10T23:33:10.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f020be00640 1 --2- 172.21.15.94:0/1009035969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f021415e3a0 0x7f021415b990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00c430 tx=0x7f01fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:10.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f020aa00640 1 -- 172.21.15.94:0/1009035969 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc011850 con 0x7f021415e3a0 2024-09-10T23:33:10.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f020aa00640 1 -- 172.21.15.94:0/1009035969 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01fc017070 con 0x7f021415e3a0 2024-09-10T23:33:10.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f020aa00640 1 -- 172.21.15.94:0/1009035969 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc011da0 con 0x7f021415e3a0 2024-09-10T23:33:10.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f0219600640 1 -- 172.21.15.94:0/1009035969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021415abd0 con 0x7f021415e3a0 2024-09-10T23:33:10.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f0219600640 1 -- 172.21.15.94:0/1009035969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f021415b190 con 0x7f021415e3a0 2024-09-10T23:33:10.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.225+0000 7f0219600640 1 -- 172.21.15.94:0/1009035969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f021406e5c0 con 0x7f021415e3a0 2024-09-10T23:33:10.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.229+0000 7f020aa00640 1 -- 172.21.15.94:0/1009035969 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f01fc01c460 con 0x7f021415e3a0 2024-09-10T23:33:10.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.229+0000 7f020aa00640 1 --2- 172.21.15.94:0/1009035969 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f01f00725e0 0x7f01f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:10.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.229+0000 7f020aa00640 1 -- 172.21.15.94:0/1009035969 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f01fc092f80 con 0x7f021415e3a0 2024-09-10T23:33:10.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.229+0000 7f0212a00640 1 --2- 172.21.15.94:0/1009035969 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f01f00725e0 0x7f01f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:10.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.229+0000 7f0212a00640 1 --2- 172.21.15.94:0/1009035969 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f01f00725e0 0x7f01f0074aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0204000c10 tx=0x7f0204009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:10.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.233+0000 7f020aa00640 1 -- 172.21.15.94:0/1009035969 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f01fc05df70 con 0x7f021415e3a0 2024-09-10T23:33:10.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.397+0000 7f0219600640 1 -- 172.21.15.94:0/1009035969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02140cecf0 con 0x7f021415e3a0 2024-09-10T23:33:10.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.401+0000 7f020aa00640 1 -- 172.21.15.94:0/1009035969 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16657 (secure 0 0 0) 0x7f01fc0629a0 con 0x7f021415e3a0 2024-09-10T23:33:10.403 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:10.403 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":22,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:32:43.556132+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:32:39.855046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T23:32:40.904161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:33:10.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.401+0000 7f0219600640 1 -- 172.21.15.94:0/1009035969 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f01f00725e0 msgr2=0x7f01f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:10.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.401+0000 7f0219600640 1 --2- 172.21.15.94:0/1009035969 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f01f00725e0 0x7f01f0074aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0204000c10 tx=0x7f0204009040 comp rx=0 tx=0).stop 2024-09-10T23:33:10.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.401+0000 7f0219600640 1 -- 172.21.15.94:0/1009035969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f021415e3a0 msgr2=0x7f021415b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:10.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.401+0000 7f0219600640 1 --2- 172.21.15.94:0/1009035969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f021415e3a0 0x7f021415b990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00c430 tx=0x7f01fc00c8f0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.401+0000 7f0219600640 1 -- 172.21.15.94:0/1009035969 shutdown_connections 2024-09-10T23:33:10.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.401+0000 7f0219600640 1 --2- 172.21.15.94:0/1009035969 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f01f00725e0 0x7f01f0074aa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.401+0000 7f0219600640 1 --2- 172.21.15.94:0/1009035969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f021415e8a0 0x7f021415bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.401+0000 7f0219600640 1 --2- 172.21.15.94:0/1009035969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f021415e3a0 0x7f021415b990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.401+0000 7f0219600640 1 --2- 172.21.15.94:0/1009035969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f021415a060 0x7f02141f7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.401+0000 7f0219600640 1 -- 172.21.15.94:0/1009035969 >> 172.21.15.94:0/1009035969 conn(0x7f021406cdf0 msgr2=0x7f02140cd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:10.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.401+0000 7f0219600640 1 -- 172.21.15.94:0/1009035969 shutdown_connections 2024-09-10T23:33:10.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.401+0000 7f0219600640 1 -- 172.21.15.94:0/1009035969 wait complete. 2024-09-10T23:33:10.425 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T23:32:31.372521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T23:32:39.855046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T23:32:40.904161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T23:33:10.425 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T23:33:10.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4fb200640 1 -- 172.21.15.94:0/1415022278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d8003c70 msgr2=0x7ff4d8004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:10.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4fb200640 1 --2- 172.21.15.94:0/1415022278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d8003c70 0x7ff4d8004070 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400c640 tx=0x7ff4e401d220 comp rx=0 tx=0).stop 2024-09-10T23:33:10.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4f9200640 1 -- 172.21.15.94:0/1415022278 reap_dead start 2024-09-10T23:33:10.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4fb200640 1 -- 172.21.15.94:0/1415022278 shutdown_connections 2024-09-10T23:33:10.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4fb200640 1 -- 172.21.15.94:0/1415022278 >> 172.21.15.94:0/1415022278 conn(0x7ff4f40d1c60 msgr2=0x7ff4f40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:10.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4fb200640 1 -- 172.21.15.94:0/1415022278 shutdown_connections 2024-09-10T23:33:10.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4fb200640 1 -- 172.21.15.94:0/1415022278 wait complete. 2024-09-10T23:33:10.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4fb200640 1 Processor -- start 2024-09-10T23:33:10.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4fb200640 1 -- start start 2024-09-10T23:33:10.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4fb200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d8003c70 0x7ff4f415a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:10.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4fb200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4f415ff80 0x7ff4f41571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:10.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4fb200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4f415eec0 0x7ff4f415f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:10.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4fb200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f41683f0 con 0x7ff4d8003c70 2024-09-10T23:33:10.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4fb200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4168270 con 0x7ff4f415eec0 2024-09-10T23:33:10.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4fb200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f4168570 con 0x7ff4f415ff80 2024-09-10T23:33:10.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4f3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d8003c70 0x7ff4f415a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:10.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4f3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d8003c70 0x7ff4f415a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38758/0 (socket says 172.21.15.94:38758) 2024-09-10T23:33:10.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4f3e00640 1 -- 172.21.15.94:0/2852787728 learned_addr learned my addr 172.21.15.94:0/2852787728 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:10.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4eb400640 1 --2- 172.21.15.94:0/2852787728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4f415ff80 0x7ff4f41571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:10.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.569+0000 7ff4f9200640 1 --2- 172.21.15.94:0/2852787728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4f415eec0 0x7ff4f415f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:10.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.573+0000 7ff4f3e00640 1 -- 172.21.15.94:0/2852787728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4f415ff80 msgr2=0x7ff4f41571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:10.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.573+0000 7ff4f3e00640 1 --2- 172.21.15.94:0/2852787728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4f415ff80 0x7ff4f41571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.573+0000 7ff4f3e00640 1 -- 172.21.15.94:0/2852787728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4f415eec0 msgr2=0x7ff4f415f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:10.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.573+0000 7ff4f3e00640 1 --2- 172.21.15.94:0/2852787728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4f415eec0 0x7ff4f415f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.573+0000 7ff4f3e00640 1 -- 172.21.15.94:0/2852787728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f4160400 con 0x7ff4d8003c70 2024-09-10T23:33:10.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.573+0000 7ff4f9200640 1 --2- 172.21.15.94:0/2852787728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4f415eec0 0x7ff4f415f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:33:10.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.573+0000 7ff4f3e00640 1 --2- 172.21.15.94:0/2852787728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d8003c70 0x7ff4f415a5b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4098bd0 tx=0x7ff4e4006680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:10.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.573+0000 7ff4f0c00640 1 -- 172.21.15.94:0/2852787728 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4004e60 con 0x7ff4d8003c70 2024-09-10T23:33:10.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.573+0000 7ff4f0c00640 1 -- 172.21.15.94:0/2852787728 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4e40013b0 con 0x7ff4d8003c70 2024-09-10T23:33:10.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.573+0000 7ff4f0c00640 1 -- 172.21.15.94:0/2852787728 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4003980 con 0x7ff4d8003c70 2024-09-10T23:33:10.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.573+0000 7ff4fb200640 1 -- 172.21.15.94:0/2852787728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f41576f0 con 0x7ff4d8003c70 2024-09-10T23:33:10.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.573+0000 7ff4fb200640 1 -- 172.21.15.94:0/2852787728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f40c6500 con 0x7ff4d8003c70 2024-09-10T23:33:10.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.573+0000 7ff4fb200640 1 -- 172.21.15.94:0/2852787728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f406d7f0 con 0x7ff4d8003c70 2024-09-10T23:33:10.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.573+0000 7ff4f0c00640 1 -- 172.21.15.94:0/2852787728 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff4e4003b20 con 0x7ff4d8003c70 2024-09-10T23:33:10.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.573+0000 7ff4f0c00640 1 --2- 172.21.15.94:0/2852787728 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff4d00725e0 0x7ff4d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:10.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.573+0000 7ff4f0c00640 1 -- 172.21.15.94:0/2852787728 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff4e412db00 con 0x7ff4d8003c70 2024-09-10T23:33:10.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.577+0000 7ff4eb400640 1 --2- 172.21.15.94:0/2852787728 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff4d00725e0 0x7ff4d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:10.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.577+0000 7ff4eb400640 1 --2- 172.21.15.94:0/2852787728 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff4d00725e0 0x7ff4d0074aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40b75f0 tx=0x7ff4dc006b20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:10.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.577+0000 7ff4f0c00640 1 -- 172.21.15.94:0/2852787728 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff4e4016fd0 con 0x7ff4d8003c70 2024-09-10T23:33:10.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.745+0000 7ff4fb200640 1 -- 172.21.15.94:0/2852787728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff4f40b1100 con 0x7ff4d8003c70 2024-09-10T23:33:10.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.745+0000 7ff4f0c00640 1 -- 172.21.15.94:0/2852787728 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7ff4e412de20 con 0x7ff4d8003c70 2024-09-10T23:33:10.748 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-09-10T23:33:10.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.745+0000 7ff4fb200640 1 -- 172.21.15.94:0/2852787728 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff4d00725e0 msgr2=0x7ff4d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:10.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.745+0000 7ff4fb200640 1 --2- 172.21.15.94:0/2852787728 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff4d00725e0 0x7ff4d0074aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40b75f0 tx=0x7ff4dc006b20 comp rx=0 tx=0).stop 2024-09-10T23:33:10.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.745+0000 7ff4fb200640 1 -- 172.21.15.94:0/2852787728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d8003c70 msgr2=0x7ff4f415a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:10.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.745+0000 7ff4fb200640 1 --2- 172.21.15.94:0/2852787728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d8003c70 0x7ff4f415a5b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4098bd0 tx=0x7ff4e4006680 comp rx=0 tx=0).stop 2024-09-10T23:33:10.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.745+0000 7ff4fb200640 1 -- 172.21.15.94:0/2852787728 shutdown_connections 2024-09-10T23:33:10.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.745+0000 7ff4fb200640 1 --2- 172.21.15.94:0/2852787728 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff4d00725e0 0x7ff4d0074aa0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.745+0000 7ff4fb200640 1 --2- 172.21.15.94:0/2852787728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4f415eec0 0x7ff4f415f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.745+0000 7ff4fb200640 1 --2- 172.21.15.94:0/2852787728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4f415ff80 0x7ff4f41571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.745+0000 7ff4fb200640 1 --2- 172.21.15.94:0/2852787728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4d8003c70 0x7ff4f415a5b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.745+0000 7ff4fb200640 1 -- 172.21.15.94:0/2852787728 >> 172.21.15.94:0/2852787728 conn(0x7ff4f40d1c60 msgr2=0x7ff4f415b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:10.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.745+0000 7ff4fb200640 1 -- 172.21.15.94:0/2852787728 shutdown_connections 2024-09-10T23:33:10.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.745+0000 7ff4fb200640 1 -- 172.21.15.94:0/2852787728 wait complete. 2024-09-10T23:33:10.769 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T23:33:10.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.909+0000 7fc540c00640 1 -- 172.21.15.94:0/3522657692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc53c0cb4d0 msgr2=0x7fc53c15b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:10.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.909+0000 7fc540c00640 1 --2- 172.21.15.94:0/3522657692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc53c0cb4d0 0x7fc53c15b190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc534009c80 tx=0x7fc53401cda0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.909+0000 7fc540c00640 1 -- 172.21.15.94:0/3522657692 shutdown_connections 2024-09-10T23:33:10.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.909+0000 7fc540c00640 1 --2- 172.21.15.94:0/3522657692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc53c0cb4d0 0x7fc53c15b190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.909+0000 7fc540c00640 1 --2- 172.21.15.94:0/3522657692 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc53c06fc00 0x7fc53c0c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.909+0000 7fc540c00640 1 -- 172.21.15.94:0/3522657692 >> 172.21.15.94:0/3522657692 conn(0x7fc53c0cfaf0 msgr2=0x7fc53c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:10.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.909+0000 7fc540c00640 1 -- 172.21.15.94:0/3522657692 shutdown_connections 2024-09-10T23:33:10.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.909+0000 7fc540c00640 1 -- 172.21.15.94:0/3522657692 wait complete. 2024-09-10T23:33:10.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc540c00640 1 Processor -- start 2024-09-10T23:33:10.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc540c00640 1 -- start start 2024-09-10T23:33:10.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc540c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc53c06fc00 0x7fc53c1f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:10.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc540c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc53c0cb4d0 0x7fc53c145360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:10.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc540c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc53c158790 0x7fc53c1458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:10.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc540c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1680c0 con 0x7fc53c0cb4d0 2024-09-10T23:33:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc540c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c167f40 con 0x7fc53c158790 2024-09-10T23:33:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc540c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc53c168240 con 0x7fc53c06fc00 2024-09-10T23:33:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc531400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc53c0cb4d0 0x7fc53c145360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc531400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc53c0cb4d0 0x7fc53c145360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38774/0 (socket says 172.21.15.94:38774) 2024-09-10T23:33:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc531400640 1 -- 172.21.15.94:0/2270394597 learned_addr learned my addr 172.21.15.94:0/2270394597 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc531400640 1 -- 172.21.15.94:0/2270394597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc53c06fc00 msgr2=0x7fc53c1f76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc531400640 1 --2- 172.21.15.94:0/2270394597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc53c06fc00 0x7fc53c1f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc531400640 1 -- 172.21.15.94:0/2270394597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc53c158790 msgr2=0x7fc53c1458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc531400640 1 --2- 172.21.15.94:0/2270394597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc53c158790 0x7fc53c1458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc531400640 1 -- 172.21.15.94:0/2270394597 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c072fc0 con 0x7fc53c0cb4d0 2024-09-10T23:33:10.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc531400640 1 --2- 172.21.15.94:0/2270394597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc53c0cb4d0 0x7fc53c145360 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc52400d980 tx=0x7fc52400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:10.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc532a00640 1 -- 172.21.15.94:0/2270394597 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524017070 con 0x7fc53c0cb4d0 2024-09-10T23:33:10.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc532a00640 1 -- 172.21.15.94:0/2270394597 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc524009d20 con 0x7fc53c0cb4d0 2024-09-10T23:33:10.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc540c00640 1 -- 172.21.15.94:0/2270394597 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53c0732a0 con 0x7fc53c0cb4d0 2024-09-10T23:33:10.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc532a00640 1 -- 172.21.15.94:0/2270394597 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524004280 con 0x7fc53c0cb4d0 2024-09-10T23:33:10.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc540c00640 1 -- 172.21.15.94:0/2270394597 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53c073860 con 0x7fc53c0cb4d0 2024-09-10T23:33:10.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.913+0000 7fc540c00640 1 -- 172.21.15.94:0/2270394597 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc53c073cc0 con 0x7fc53c0cb4d0 2024-09-10T23:33:10.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.917+0000 7fc532a00640 1 -- 172.21.15.94:0/2270394597 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc524003600 con 0x7fc53c0cb4d0 2024-09-10T23:33:10.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.917+0000 7fc532a00640 1 --2- 172.21.15.94:0/2270394597 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc5180725e0 0x7fc518074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:10.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.917+0000 7fc532a00640 1 -- 172.21.15.94:0/2270394597 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc524093750 con 0x7fc53c0cb4d0 2024-09-10T23:33:10.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.917+0000 7fc539e00640 1 --2- 172.21.15.94:0/2270394597 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc5180725e0 0x7fc518074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:10.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.917+0000 7fc539e00640 1 --2- 172.21.15.94:0/2270394597 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc5180725e0 0x7fc518074aa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc52c000f70 tx=0x7fc52c0079e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:10.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:10.921+0000 7fc532a00640 1 -- 172.21.15.94:0/2270394597 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc52405ee20 con 0x7fc53c0cb4d0 2024-09-10T23:33:11.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.085+0000 7fc540c00640 1 -- 172.21.15.94:0/2270394597 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc53c0c49b0 con 0x7fc53c0cb4d0 2024-09-10T23:33:11.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.085+0000 7fc532a00640 1 -- 172.21.15.94:0/2270394597 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fc52405e720 con 0x7fc53c0cb4d0 2024-09-10T23:33:11.089 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-09-10T23:33:11.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.089+0000 7fc540c00640 1 -- 172.21.15.94:0/2270394597 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc5180725e0 msgr2=0x7fc518074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:11.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.089+0000 7fc540c00640 1 --2- 172.21.15.94:0/2270394597 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc5180725e0 0x7fc518074aa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc52c000f70 tx=0x7fc52c0079e0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.089+0000 7fc540c00640 1 -- 172.21.15.94:0/2270394597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc53c0cb4d0 msgr2=0x7fc53c145360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:11.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.089+0000 7fc540c00640 1 --2- 172.21.15.94:0/2270394597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc53c0cb4d0 0x7fc53c145360 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc52400d980 tx=0x7fc52400de40 comp rx=0 tx=0).stop 2024-09-10T23:33:11.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.089+0000 7fc540c00640 1 -- 172.21.15.94:0/2270394597 shutdown_connections 2024-09-10T23:33:11.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.089+0000 7fc540c00640 1 --2- 172.21.15.94:0/2270394597 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc5180725e0 0x7fc518074aa0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.089+0000 7fc540c00640 1 --2- 172.21.15.94:0/2270394597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc53c158790 0x7fc53c1458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.089+0000 7fc540c00640 1 --2- 172.21.15.94:0/2270394597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc53c0cb4d0 0x7fc53c145360 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.089+0000 7fc540c00640 1 --2- 172.21.15.94:0/2270394597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc53c06fc00 0x7fc53c1f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.089+0000 7fc540c00640 1 -- 172.21.15.94:0/2270394597 >> 172.21.15.94:0/2270394597 conn(0x7fc53c0cfaf0 msgr2=0x7fc53c0b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.089+0000 7fc540c00640 1 -- 172.21.15.94:0/2270394597 shutdown_connections 2024-09-10T23:33:11.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.089+0000 7fc540c00640 1 -- 172.21.15.94:0/2270394597 wait complete. 2024-09-10T23:33:11.111 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T23:33:11.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.253+0000 7f2b7aa00640 1 -- 172.21.15.94:0/2717020498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b740ce280 msgr2=0x7f2b740cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:11.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.253+0000 7f2b7aa00640 1 --2- 172.21.15.94:0/2717020498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b740ce280 0x7f2b740cf280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2b74072510 tx=0x7f2b6401dc70 comp rx=0 tx=0).stop 2024-09-10T23:33:11.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.253+0000 7f2b7aa00640 1 -- 172.21.15.94:0/2717020498 shutdown_connections 2024-09-10T23:33:11.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.253+0000 7f2b7aa00640 1 --2- 172.21.15.94:0/2717020498 >> v2:172.21.15.181:3300/0 conn(0x7f2b7406e7c0 0x7f2b7406ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.253+0000 7f2b7aa00640 1 --2- 172.21.15.94:0/2717020498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b740ce280 0x7f2b740cf280 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.253+0000 7f2b7aa00640 1 -- 172.21.15.94:0/2717020498 >> 172.21.15.94:0/2717020498 conn(0x7f2b740cfaf0 msgr2=0x7f2b740cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:11.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.253+0000 7f2b7aa00640 1 -- 172.21.15.94:0/2717020498 shutdown_connections 2024-09-10T23:33:11.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.253+0000 7f2b7aa00640 1 -- 172.21.15.94:0/2717020498 wait complete. 2024-09-10T23:33:11.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.253+0000 7f2b7aa00640 1 Processor -- start 2024-09-10T23:33:11.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.253+0000 7f2b7aa00640 1 -- start start 2024-09-10T23:33:11.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.253+0000 7f2b7aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b7406e7c0 0x7f2b741598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:11.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.253+0000 7f2b7aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b740c64f0 0x7f2b74153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:11.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.253+0000 7f2b7aa00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b740ce280 0x7f2b741f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:11.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.253+0000 7f2b7aa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7415b530 con 0x7f2b740c64f0 2024-09-10T23:33:11.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.253+0000 7f2b7aa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7415b3b0 con 0x7f2b7406e7c0 2024-09-10T23:33:11.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.253+0000 7f2b7aa00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7415b6b0 con 0x7f2b740ce280 2024-09-10T23:33:11.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b73400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b740c64f0 0x7f2b74153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:11.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b73400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b740c64f0 0x7f2b74153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38794/0 (socket says 172.21.15.94:38794) 2024-09-10T23:33:11.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b73400640 1 -- 172.21.15.94:0/2723369397 learned_addr learned my addr 172.21.15.94:0/2723369397 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:11.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b73e00640 1 --2- 172.21.15.94:0/2723369397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b7406e7c0 0x7f2b741598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:11.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b78a00640 1 --2- 172.21.15.94:0/2723369397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b740ce280 0x7f2b741f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:11.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b73400640 1 -- 172.21.15.94:0/2723369397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b740ce280 msgr2=0x7f2b741f7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:11.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b73400640 1 --2- 172.21.15.94:0/2723369397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b740ce280 0x7f2b741f7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b73400640 1 -- 172.21.15.94:0/2723369397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b7406e7c0 msgr2=0x7f2b741598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:11.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b73400640 1 --2- 172.21.15.94:0/2723369397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b7406e7c0 0x7f2b741598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b73400640 1 -- 172.21.15.94:0/2723369397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b74153ad0 con 0x7f2b740c64f0 2024-09-10T23:33:11.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b73400640 1 --2- 172.21.15.94:0/2723369397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b740c64f0 0x7f2b74153590 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000c500 tx=0x7f2b6000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:11.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b6be00640 1 -- 172.21.15.94:0/2723369397 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60017070 con 0x7f2b740c64f0 2024-09-10T23:33:11.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b6be00640 1 -- 172.21.15.94:0/2723369397 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b600053c0 con 0x7f2b740c64f0 2024-09-10T23:33:11.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b6be00640 1 -- 172.21.15.94:0/2723369397 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60003860 con 0x7f2b740c64f0 2024-09-10T23:33:11.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b7aa00640 1 -- 172.21.15.94:0/2723369397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7414ae70 con 0x7f2b740c64f0 2024-09-10T23:33:11.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b7aa00640 1 -- 172.21.15.94:0/2723369397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b74154050 con 0x7f2b740c64f0 2024-09-10T23:33:11.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b7aa00640 1 -- 172.21.15.94:0/2723369397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7406ebf0 con 0x7f2b740c64f0 2024-09-10T23:33:11.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.257+0000 7f2b6be00640 1 -- 172.21.15.94:0/2723369397 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2b60003a00 con 0x7f2b740c64f0 2024-09-10T23:33:11.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.261+0000 7f2b6be00640 1 --2- 172.21.15.94:0/2723369397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2b540725e0 0x7f2b54074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:11.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.261+0000 7f2b6be00640 1 -- 172.21.15.94:0/2723369397 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2b600927d0 con 0x7f2b740c64f0 2024-09-10T23:33:11.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.261+0000 7f2b73e00640 1 --2- 172.21.15.94:0/2723369397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2b540725e0 0x7f2b54074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:11.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.261+0000 7f2b73e00640 1 --2- 172.21.15.94:0/2723369397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2b540725e0 0x7f2b54074aa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2b640b0000 tx=0x7f2b6401c4b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:11.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.261+0000 7f2b6be00640 1 -- 172.21.15.94:0/2723369397 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2b6005d850 con 0x7f2b740c64f0 2024-09-10T23:33:11.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.429+0000 7f2b7aa00640 1 -- 172.21.15.94:0/2723369397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2b740d0850 con 0x7f2b740c64f0 2024-09-10T23:33:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.429+0000 7f2b6be00640 1 -- 172.21.15.94:0/2723369397 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f2b60062280 con 0x7f2b740c64f0 2024-09-10T23:33:11.431 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-09-10T23:33:11.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.429+0000 7f2b7aa00640 1 -- 172.21.15.94:0/2723369397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2b540725e0 msgr2=0x7f2b54074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:11.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.429+0000 7f2b7aa00640 1 --2- 172.21.15.94:0/2723369397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2b540725e0 0x7f2b54074aa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2b640b0000 tx=0x7f2b6401c4b0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.429+0000 7f2b7aa00640 1 -- 172.21.15.94:0/2723369397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b740c64f0 msgr2=0x7f2b74153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:11.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.429+0000 7f2b7aa00640 1 --2- 172.21.15.94:0/2723369397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b740c64f0 0x7f2b74153590 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000c500 tx=0x7f2b6000c900 comp rx=0 tx=0).stop 2024-09-10T23:33:11.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.429+0000 7f2b7aa00640 1 -- 172.21.15.94:0/2723369397 shutdown_connections 2024-09-10T23:33:11.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.429+0000 7f2b7aa00640 1 --2- 172.21.15.94:0/2723369397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2b540725e0 0x7f2b54074aa0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.429+0000 7f2b7aa00640 1 --2- 172.21.15.94:0/2723369397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b740ce280 0x7f2b741f7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.429+0000 7f2b7aa00640 1 --2- 172.21.15.94:0/2723369397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b740c64f0 0x7f2b74153590 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.429+0000 7f2b7aa00640 1 --2- 172.21.15.94:0/2723369397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b7406e7c0 0x7f2b741598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.429+0000 7f2b7aa00640 1 -- 172.21.15.94:0/2723369397 >> 172.21.15.94:0/2723369397 conn(0x7f2b740cfaf0 msgr2=0x7f2b740af640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:11.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.429+0000 7f2b7aa00640 1 -- 172.21.15.94:0/2723369397 shutdown_connections 2024-09-10T23:33:11.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.429+0000 7f2b7aa00640 1 -- 172.21.15.94:0/2723369397 wait complete. 2024-09-10T23:33:11.453 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-10T23:33:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ea00640 1 -- 172.21.15.94:0/2661794337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff288070150 msgr2=0x7ff2880b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ea00640 1 --2- 172.21.15.94:0/2661794337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff288070150 0x7ff2880b35c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff28000acd0 tx=0x7ff28001ddd0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ea00640 1 -- 172.21.15.94:0/2661794337 shutdown_connections 2024-09-10T23:33:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ea00640 1 --2- 172.21.15.94:0/2661794337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff288070150 0x7ff2880b35c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ea00640 1 --2- 172.21.15.94:0/2661794337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff28806f600 0x7ff288072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ea00640 1 -- 172.21.15.94:0/2661794337 >> 172.21.15.94:0/2661794337 conn(0x7ff28806cdf0 msgr2=0x7ff28806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ea00640 1 -- 172.21.15.94:0/2661794337 shutdown_connections 2024-09-10T23:33:11.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ea00640 1 -- 172.21.15.94:0/2661794337 wait complete. 2024-09-10T23:33:11.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ea00640 1 Processor -- start 2024-09-10T23:33:11.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ea00640 1 -- start start 2024-09-10T23:33:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff28806f600 0x7ff2880cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ea00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff288070150 0x7ff288163230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff288157c60 0x7ff288163770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ea00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff28815b3b0 con 0x7ff288157c60 2024-09-10T23:33:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ea00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff28815b230 con 0x7ff28806f600 2024-09-10T23:33:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ea00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff28815b530 con 0x7ff288070150 2024-09-10T23:33:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff288157c60 0x7ff288163770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff288157c60 0x7ff288163770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38810/0 (socket says 172.21.15.94:38810) 2024-09-10T23:33:11.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff28ca00640 1 -- 172.21.15.94:0/246078458 learned_addr learned my addr 172.21.15.94:0/246078458 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff287e00640 1 --2- 172.21.15.94:0/246078458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff28806f600 0x7ff2880cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.593+0000 7ff27f400640 1 --2- 172.21.15.94:0/246078458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff288070150 0x7ff288163230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.597+0000 7ff28ca00640 1 -- 172.21.15.94:0/246078458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff288070150 msgr2=0x7ff288163230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.597+0000 7ff28ca00640 1 --2- 172.21.15.94:0/246078458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff288070150 0x7ff288163230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.597+0000 7ff28ca00640 1 -- 172.21.15.94:0/246078458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff28806f600 msgr2=0x7ff2880cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.597+0000 7ff28ca00640 1 --2- 172.21.15.94:0/246078458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff28806f600 0x7ff2880cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.597+0000 7ff28ca00640 1 -- 172.21.15.94:0/246078458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28815a070 con 0x7ff288157c60 2024-09-10T23:33:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.597+0000 7ff28ca00640 1 --2- 172.21.15.94:0/246078458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff288157c60 0x7ff288163770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff28001ba00 tx=0x7ff28000a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.597+0000 7ff284c00640 1 -- 172.21.15.94:0/246078458 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2800bb020 con 0x7ff288157c60 2024-09-10T23:33:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.597+0000 7ff284c00640 1 -- 172.21.15.94:0/246078458 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff280004d40 con 0x7ff288157c60 2024-09-10T23:33:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.597+0000 7ff284c00640 1 -- 172.21.15.94:0/246078458 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2800b3040 con 0x7ff288157c60 2024-09-10T23:33:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.597+0000 7ff28ea00640 1 -- 172.21.15.94:0/246078458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2881fb670 con 0x7ff288157c60 2024-09-10T23:33:11.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.597+0000 7ff28ea00640 1 -- 172.21.15.94:0/246078458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2880cd4c0 con 0x7ff288157c60 2024-09-10T23:33:11.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.597+0000 7ff28ea00640 1 -- 172.21.15.94:0/246078458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff288072610 con 0x7ff288157c60 2024-09-10T23:33:11.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.597+0000 7ff284c00640 1 -- 172.21.15.94:0/246078458 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff2800b6070 con 0x7ff288157c60 2024-09-10T23:33:11.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.597+0000 7ff284c00640 1 --2- 172.21.15.94:0/246078458 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff264072610 0x7ff264074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:11.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.597+0000 7ff284c00640 1 -- 172.21.15.94:0/246078458 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff28012d550 con 0x7ff288157c60 2024-09-10T23:33:11.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.601+0000 7ff287e00640 1 --2- 172.21.15.94:0/246078458 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff264072610 0x7ff264074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:11.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.601+0000 7ff287e00640 1 --2- 172.21.15.94:0/246078458 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff264072610 0x7ff264074ad0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff288163fb0 tx=0x7ff278007ea0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:11.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.601+0000 7ff284c00640 1 -- 172.21.15.94:0/246078458 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff2800c5020 con 0x7ff288157c60 2024-09-10T23:33:11.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:11.829+0000 7ff28ea00640 1 -- 172.21.15.94:0/246078458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7ff28806e7e0 con 0x7ff288157c60 2024-09-10T23:33:12.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.689+0000 7ff284c00640 1 -- 172.21.15.94:0/246078458 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v51) ==== 161+0+0 (secure 0 0 0) 0x7ff280017ae0 con 0x7ff288157c60 2024-09-10T23:33:12.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.693+0000 7ff28ea00640 1 -- 172.21.15.94:0/246078458 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff264072610 msgr2=0x7ff264074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:12.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.693+0000 7ff28ea00640 1 --2- 172.21.15.94:0/246078458 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff264072610 0x7ff264074ad0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff288163fb0 tx=0x7ff278007ea0 comp rx=0 tx=0).stop 2024-09-10T23:33:12.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.693+0000 7ff28ea00640 1 -- 172.21.15.94:0/246078458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff288157c60 msgr2=0x7ff288163770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:12.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.693+0000 7ff28ea00640 1 --2- 172.21.15.94:0/246078458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff288157c60 0x7ff288163770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff28001ba00 tx=0x7ff28000a420 comp rx=0 tx=0).stop 2024-09-10T23:33:12.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.693+0000 7ff28ea00640 1 -- 172.21.15.94:0/246078458 shutdown_connections 2024-09-10T23:33:12.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.693+0000 7ff28ea00640 1 --2- 172.21.15.94:0/246078458 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff264072610 0x7ff264074ad0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:12.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.693+0000 7ff28ea00640 1 --2- 172.21.15.94:0/246078458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff288157c60 0x7ff288163770 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:12.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.693+0000 7ff28ea00640 1 --2- 172.21.15.94:0/246078458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff288070150 0x7ff288163230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:12.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.693+0000 7ff28ea00640 1 --2- 172.21.15.94:0/246078458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff28806f600 0x7ff2880cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:12.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.693+0000 7ff28ea00640 1 -- 172.21.15.94:0/246078458 >> 172.21.15.94:0/246078458 conn(0x7ff28806cdf0 msgr2=0x7ff288073b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:12.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.693+0000 7ff28ea00640 1 -- 172.21.15.94:0/246078458 shutdown_connections 2024-09-10T23:33:12.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.693+0000 7ff28ea00640 1 -- 172.21.15.94:0/246078458 wait complete. 2024-09-10T23:33:12.716 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T23:33:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.857+0000 7f0b67000640 1 -- 172.21.15.94:0/921324369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b600701d0 msgr2=0x7f0b60070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.857+0000 7f0b67000640 1 --2- 172.21.15.94:0/921324369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b600701d0 0x7f0b60070620 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0b6006f550 tx=0x7f0b5801cd80 comp rx=0 tx=0).stop 2024-09-10T23:33:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.857+0000 7f0b67000640 1 -- 172.21.15.94:0/921324369 shutdown_connections 2024-09-10T23:33:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.857+0000 7f0b67000640 1 --2- 172.21.15.94:0/921324369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b600701d0 0x7f0b60070620 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.857+0000 7f0b67000640 1 --2- 172.21.15.94:0/921324369 >> v2:172.21.15.181:3300/0 conn(0x7f0b600cda70 0x7f0b6015ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:12.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.857+0000 7f0b67000640 1 -- 172.21.15.94:0/921324369 >> 172.21.15.94:0/921324369 conn(0x7f0b600b2520 msgr2=0x7f0b600b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:12.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.857+0000 7f0b67000640 1 -- 172.21.15.94:0/921324369 shutdown_connections 2024-09-10T23:33:12.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.857+0000 7f0b67000640 1 -- 172.21.15.94:0/921324369 wait complete. 2024-09-10T23:33:12.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.857+0000 7f0b67000640 1 Processor -- start 2024-09-10T23:33:12.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.857+0000 7f0b67000640 1 -- start start 2024-09-10T23:33:12.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b67000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b600701d0 0x7f0b600c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:12.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b67000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b600cda70 0x7f0b601fbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:12.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b67000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b6015aa30 0x7f0b60163b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:12.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b67000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b60167ad0 con 0x7f0b6015aa30 2024-09-10T23:33:12.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b67000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0b60167950 con 0x7f0b600cda70 2024-09-10T23:33:12.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b67000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0b60167c50 con 0x7f0b600701d0 2024-09-10T23:33:12.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b66600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b6015aa30 0x7f0b60163b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:12.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b66600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b6015aa30 0x7f0b60163b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38822/0 (socket says 172.21.15.94:38822) 2024-09-10T23:33:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b66600640 1 -- 172.21.15.94:0/1084409733 learned_addr learned my addr 172.21.15.94:0/1084409733 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b65200640 1 --2- 172.21.15.94:0/1084409733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b600cda70 0x7f0b601fbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b65c00640 1 --2- 172.21.15.94:0/1084409733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b600701d0 0x7f0b600c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b66600640 1 -- 172.21.15.94:0/1084409733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b600701d0 msgr2=0x7f0b600c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b66600640 1 --2- 172.21.15.94:0/1084409733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b600701d0 0x7f0b600c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b66600640 1 -- 172.21.15.94:0/1084409733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b600cda70 msgr2=0x7f0b601fbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b66600640 1 --2- 172.21.15.94:0/1084409733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b600cda70 0x7f0b601fbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b66600640 1 -- 172.21.15.94:0/1084409733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b600c8b20 con 0x7f0b6015aa30 2024-09-10T23:33:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b65200640 1 --2- 172.21.15.94:0/1084409733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b600cda70 0x7f0b601fbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b66600640 1 --2- 172.21.15.94:0/1084409733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b6015aa30 0x7f0b60163b90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009bd0 tx=0x7f0b58003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b5d600640 1 -- 172.21.15.94:0/1084409733 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b580bb070 con 0x7f0b6015aa30 2024-09-10T23:33:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b5d600640 1 -- 172.21.15.94:0/1084409733 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b58004e00 con 0x7f0b6015aa30 2024-09-10T23:33:12.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b67000640 1 -- 172.21.15.94:0/1084409733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b600c82a0 con 0x7f0b6015aa30 2024-09-10T23:33:12.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b5d600640 1 -- 172.21.15.94:0/1084409733 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b580b2040 con 0x7f0b6015aa30 2024-09-10T23:33:12.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b67000640 1 -- 172.21.15.94:0/1084409733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b600c8840 con 0x7f0b6015aa30 2024-09-10T23:33:12.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b67000640 1 -- 172.21.15.94:0/1084409733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b60070620 con 0x7f0b6015aa30 2024-09-10T23:33:12.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.861+0000 7f0b5d600640 1 -- 172.21.15.94:0/1084409733 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0b580b5030 con 0x7f0b6015aa30 2024-09-10T23:33:12.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.865+0000 7f0b5d600640 1 --2- 172.21.15.94:0/1084409733 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b400725e0 0x7f0b40074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:12.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.865+0000 7f0b5d600640 1 -- 172.21.15.94:0/1084409733 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0b5812cf10 con 0x7f0b6015aa30 2024-09-10T23:33:12.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.865+0000 7f0b65c00640 1 --2- 172.21.15.94:0/1084409733 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b400725e0 0x7f0b40074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:12.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.865+0000 7f0b65c00640 1 --2- 172.21.15.94:0/1084409733 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b400725e0 0x7f0b40074aa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400b810 tx=0x7f0b54008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:12.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:12.869+0000 7f0b5d600640 1 -- 172.21.15.94:0/1084409733 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0b58016970 con 0x7f0b6015aa30 2024-09-10T23:33:13.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.037+0000 7f0b67000640 1 -- 172.21.15.94:0/1084409733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f0b600cd720 con 0x7f0b6015aa30 2024-09-10T23:33:13.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.037+0000 7f0b5d600640 1 -- 172.21.15.94:0/1084409733 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v27) ==== 59+0+1998 (secure 0 0 0) 0x7f0b580f8550 con 0x7f0b6015aa30 2024-09-10T23:33:13.041 INFO:teuthology.orchestra.run.smithi094.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T23:33:13.041 INFO:teuthology.orchestra.run.smithi094.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T23:33:13.041 INFO:teuthology.orchestra.run.smithi094.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T23:33:13.041 INFO:teuthology.orchestra.run.smithi094.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T23:33:13.041 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3944.074506 2024-09-10T23:33:13.041 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3865.464527 2024-09-10T23:33:13.041 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3898.861024 2024-09-10T23:33:13.041 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4010.048529 2024-09-10T23:33:13.041 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4650.562436 2024-09-10T23:33:13.042 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4724.018776 2024-09-10T23:33:13.042 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5255.753663 2024-09-10T23:33:13.042 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4718.009443 2024-09-10T23:33:13.042 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds 20/20 2024-09-10T23:33:13.042 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T23:33:13.042 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_ms 2/2 2024-09-10T23:33:13.042 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_frag true 2024-09-10T23:33:13.042 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_scatterstat true 2024-09-10T23:33:13.042 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T23:33:13.042 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_verify_scatter true 2024-09-10T23:33:13.042 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T23:33:13.042 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T23:33:13.042 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T23:33:13.042 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced client_mount_timeout 600 2024-09-10T23:33:13.043 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_client 20/20 2024-09-10T23:33:13.043 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_ms 1/1 2024-09-10T23:33:13.043 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T23:33:13.043 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T23:33:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.041+0000 7f0b67000640 1 -- 172.21.15.94:0/1084409733 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b400725e0 msgr2=0x7f0b40074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.041+0000 7f0b67000640 1 --2- 172.21.15.94:0/1084409733 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b400725e0 0x7f0b40074aa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400b810 tx=0x7f0b54008040 comp rx=0 tx=0).stop 2024-09-10T23:33:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.041+0000 7f0b67000640 1 -- 172.21.15.94:0/1084409733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b6015aa30 msgr2=0x7f0b60163b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.041+0000 7f0b67000640 1 --2- 172.21.15.94:0/1084409733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b6015aa30 0x7f0b60163b90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009bd0 tx=0x7f0b58003f60 comp rx=0 tx=0).stop 2024-09-10T23:33:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.041+0000 7f0b67000640 1 -- 172.21.15.94:0/1084409733 shutdown_connections 2024-09-10T23:33:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.041+0000 7f0b67000640 1 --2- 172.21.15.94:0/1084409733 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b400725e0 0x7f0b40074aa0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.041+0000 7f0b67000640 1 --2- 172.21.15.94:0/1084409733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b600cda70 0x7f0b601fbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.041+0000 7f0b67000640 1 --2- 172.21.15.94:0/1084409733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b6015aa30 0x7f0b60163b90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.041+0000 7f0b67000640 1 --2- 172.21.15.94:0/1084409733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b600701d0 0x7f0b600c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.041+0000 7f0b67000640 1 -- 172.21.15.94:0/1084409733 >> 172.21.15.94:0/1084409733 conn(0x7f0b600b2520 msgr2=0x7f0b600ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.041+0000 7f0b67000640 1 -- 172.21.15.94:0/1084409733 shutdown_connections 2024-09-10T23:33:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.041+0000 7f0b67000640 1 -- 172.21.15.94:0/1084409733 wait complete. 2024-09-10T23:33:13.061 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:33:13.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.201+0000 7f5e21800640 1 -- 172.21.15.94:0/694089237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e1c06e7c0 msgr2=0x7f5e1c06ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:13.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.201+0000 7f5e21800640 1 --2- 172.21.15.94:0/694089237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e1c06e7c0 0x7f5e1c06ec30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c069670 tx=0x7f5e0401c800 comp rx=0 tx=0).stop 2024-09-10T23:33:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e21800640 1 -- 172.21.15.94:0/694089237 shutdown_connections 2024-09-10T23:33:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e21800640 1 --2- 172.21.15.94:0/694089237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e1c06e7c0 0x7f5e1c06ec30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e21800640 1 --2- 172.21.15.94:0/694089237 >> v2:172.21.15.181:3300/0 conn(0x7f5e1c0c64f0 0x7f5e1c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e21800640 1 -- 172.21.15.94:0/694089237 >> 172.21.15.94:0/694089237 conn(0x7f5e1c0cfaf0 msgr2=0x7f5e1c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e21800640 1 -- 172.21.15.94:0/694089237 shutdown_connections 2024-09-10T23:33:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e21800640 1 -- 172.21.15.94:0/694089237 wait complete. 2024-09-10T23:33:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e21800640 1 Processor -- start 2024-09-10T23:33:13.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e21800640 1 -- start start 2024-09-10T23:33:13.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e21800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c06e7c0 0x7f5e1c14f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:13.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e21800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e1c0c64f0 0x7f5e1c1fb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:13.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e21800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e1c15af90 0x7f5e1c0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:13.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e21800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c0ce720 con 0x7f5e1c15af90 2024-09-10T23:33:13.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e21800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c0ce5a0 con 0x7f5e1c06e7c0 2024-09-10T23:33:13.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e21800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5e1c168400 con 0x7f5e1c0c64f0 2024-09-10T23:33:13.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e1b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e1c15af90 0x7f5e1c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:13.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e12000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e1c0c64f0 0x7f5e1c1fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:13.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e1aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c06e7c0 0x7f5e1c14f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:13.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e1b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e1c15af90 0x7f5e1c0b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38844/0 (socket says 172.21.15.94:38844) 2024-09-10T23:33:13.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e1aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c06e7c0 0x7f5e1c14f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:40462/0 (socket says 172.21.15.94:40462) 2024-09-10T23:33:13.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e1b400640 1 -- 172.21.15.94:0/2242572365 learned_addr learned my addr 172.21.15.94:0/2242572365 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:13.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e12000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e1c0c64f0 0x7f5e1c1fb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:39288/0 (socket says 172.21.15.94:39288) 2024-09-10T23:33:13.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e1b400640 1 -- 172.21.15.94:0/2242572365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e1c0c64f0 msgr2=0x7f5e1c1fb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:13.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e1b400640 1 --2- 172.21.15.94:0/2242572365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e1c0c64f0 0x7f5e1c1fb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e1b400640 1 -- 172.21.15.94:0/2242572365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c06e7c0 msgr2=0x7f5e1c14f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:13.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e1b400640 1 --2- 172.21.15.94:0/2242572365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c06e7c0 0x7f5e1c14f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e1b400640 1 -- 172.21.15.94:0/2242572365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1c14fba0 con 0x7f5e1c15af90 2024-09-10T23:33:13.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.205+0000 7f5e1b400640 1 --2- 172.21.15.94:0/2242572365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e1c15af90 0x7f5e1c0b0600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400c3c0 tx=0x7f5e1400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:13.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.209+0000 7f5e13400640 1 -- 172.21.15.94:0/2242572365 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e14017070 con 0x7f5e1c15af90 2024-09-10T23:33:13.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.209+0000 7f5e13400640 1 -- 172.21.15.94:0/2242572365 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e14003600 con 0x7f5e1c15af90 2024-09-10T23:33:13.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.209+0000 7f5e21800640 1 -- 172.21.15.94:0/2242572365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1c154fe0 con 0x7f5e1c15af90 2024-09-10T23:33:13.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.209+0000 7f5e13400640 1 -- 172.21.15.94:0/2242572365 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e14003a10 con 0x7f5e1c15af90 2024-09-10T23:33:13.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.209+0000 7f5e21800640 1 -- 172.21.15.94:0/2242572365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e1c1555a0 con 0x7f5e1c15af90 2024-09-10T23:33:13.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.209+0000 7f5e21800640 1 -- 172.21.15.94:0/2242572365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e1c06ec30 con 0x7f5e1c15af90 2024-09-10T23:33:13.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.209+0000 7f5e13400640 1 -- 172.21.15.94:0/2242572365 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5e14003bb0 con 0x7f5e1c15af90 2024-09-10T23:33:13.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.209+0000 7f5e13400640 1 --2- 172.21.15.94:0/2242572365 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5df80725e0 0x7f5df8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:13.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.209+0000 7f5e13400640 1 -- 172.21.15.94:0/2242572365 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5e14092780 con 0x7f5e1c15af90 2024-09-10T23:33:13.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.209+0000 7f5e1aa00640 1 --2- 172.21.15.94:0/2242572365 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5df80725e0 0x7f5df8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:13.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.213+0000 7f5e1aa00640 1 --2- 172.21.15.94:0/2242572365 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5df80725e0 0x7f5df8074aa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c009f60 tx=0x7f5e0c009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:13.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.213+0000 7f5e13400640 1 -- 172.21.15.94:0/2242572365 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5e1c14fba0 con 0x7f5e1c15af90 2024-09-10T23:33:13.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.381+0000 7f5e21800640 1 -- 172.21.15.94:0/2242572365 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5e1c0d0850 con 0x7f5e1c15af90 2024-09-10T23:33:13.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.385+0000 7f5e13400640 1 -- 172.21.15.94:0/2242572365 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f5e1405d800 con 0x7f5e1c15af90 2024-09-10T23:33:13.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.385+0000 7f5e21800640 1 -- 172.21.15.94:0/2242572365 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5df80725e0 msgr2=0x7f5df8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:13.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.385+0000 7f5e21800640 1 --2- 172.21.15.94:0/2242572365 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5df80725e0 0x7f5df8074aa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c009f60 tx=0x7f5e0c009500 comp rx=0 tx=0).stop 2024-09-10T23:33:13.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.385+0000 7f5e21800640 1 -- 172.21.15.94:0/2242572365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e1c15af90 msgr2=0x7f5e1c0b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:13.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.385+0000 7f5e21800640 1 --2- 172.21.15.94:0/2242572365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e1c15af90 0x7f5e1c0b0600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400c3c0 tx=0x7f5e1400c7c0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.385+0000 7f5e21800640 1 -- 172.21.15.94:0/2242572365 shutdown_connections 2024-09-10T23:33:13.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.385+0000 7f5e21800640 1 --2- 172.21.15.94:0/2242572365 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5df80725e0 0x7f5df8074aa0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.385+0000 7f5e21800640 1 --2- 172.21.15.94:0/2242572365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e1c15af90 0x7f5e1c0b0600 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.385+0000 7f5e21800640 1 --2- 172.21.15.94:0/2242572365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e1c0c64f0 0x7f5e1c1fb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.385+0000 7f5e21800640 1 --2- 172.21.15.94:0/2242572365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e1c06e7c0 0x7f5e1c14f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.385+0000 7f5e21800640 1 -- 172.21.15.94:0/2242572365 >> 172.21.15.94:0/2242572365 conn(0x7f5e1c0cfaf0 msgr2=0x7f5e1c0c5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:13.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.385+0000 7f5e21800640 1 -- 172.21.15.94:0/2242572365 shutdown_connections 2024-09-10T23:33:13.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.385+0000 7f5e21800640 1 -- 172.21.15.94:0/2242572365 wait complete. 2024-09-10T23:33:13.409 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:33:13.409 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:33:13.524 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:33:13.612 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:33:13.612 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:33:13.612 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:13.621 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:33:13.621 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:13.674 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:33:13.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.857+0000 7f1029e00640 1 -- 172.21.15.94:0/3926620383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10240b1d00 msgr2=0x7f1024163ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:13.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.857+0000 7f1029e00640 1 --2- 172.21.15.94:0/3926620383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10240b1d00 0x7f1024163ac0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f100c009900 tx=0x7f100c01f0d0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.857+0000 7f1029e00640 1 -- 172.21.15.94:0/3926620383 shutdown_connections 2024-09-10T23:33:13.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.857+0000 7f1029e00640 1 --2- 172.21.15.94:0/3926620383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10240b1d00 0x7f1024163ac0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.857+0000 7f1029e00640 1 -- 172.21.15.94:0/3926620383 >> 172.21.15.94:0/3926620383 conn(0x7f10240d1c90 msgr2=0x7f10240d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:13.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.857+0000 7f1029e00640 1 -- 172.21.15.94:0/3926620383 shutdown_connections 2024-09-10T23:33:13.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.857+0000 7f1029e00640 1 -- 172.21.15.94:0/3926620383 wait complete. 2024-09-10T23:33:13.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1029e00640 1 Processor -- start 2024-09-10T23:33:13.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1029e00640 1 -- start start 2024-09-10T23:33:13.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1029e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10240b1d00 0x7f10240ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:13.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1029e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10240cfc00 0x7f10240cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:13.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1029e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f102415f2d0 0x7f10241f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:13.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1029e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f102415f830 con 0x7f102415f2d0 2024-09-10T23:33:13.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1029e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f102415f6b0 con 0x7f10240cfc00 2024-09-10T23:33:13.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1029e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f10240b2790 con 0x7f10240b1d00 2024-09-10T23:33:13.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1023400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f102415f2d0 0x7f10241f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:13.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1023400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f102415f2d0 0x7f10241f2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38478/0 (socket says 172.21.15.94:38478) 2024-09-10T23:33:13.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1023400640 1 -- 172.21.15.94:0/2747896234 learned_addr learned my addr 172.21.15.94:0/2747896234 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:13.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1023400640 1 -- 172.21.15.94:0/2747896234 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10240b1d00 msgr2=0x7f10240ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:13.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1023400640 1 --2- 172.21.15.94:0/2747896234 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10240b1d00 0x7f10240ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1023400640 1 -- 172.21.15.94:0/2747896234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10240cfc00 msgr2=0x7f10240cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:13.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1023400640 1 --2- 172.21.15.94:0/2747896234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10240cfc00 0x7f10240cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:13.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1023400640 1 -- 172.21.15.94:0/2747896234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10241f6180 con 0x7f102415f2d0 2024-09-10T23:33:13.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1023400640 1 --2- 172.21.15.94:0/2747896234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f102415f2d0 0x7f10241f2c50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f101c00c730 tx=0x7f101c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:13.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f101be00640 1 -- 172.21.15.94:0/2747896234 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c003b80 con 0x7f102415f2d0 2024-09-10T23:33:13.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f101be00640 1 -- 172.21.15.94:0/2747896234 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f101c003d20 con 0x7f102415f2d0 2024-09-10T23:33:13.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f101be00640 1 -- 172.21.15.94:0/2747896234 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c00aab0 con 0x7f102415f2d0 2024-09-10T23:33:13.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1029e00640 1 -- 172.21.15.94:0/2747896234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10241f6460 con 0x7f102415f2d0 2024-09-10T23:33:13.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1029e00640 1 -- 172.21.15.94:0/2747896234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10241f6a20 con 0x7f102415f2d0 2024-09-10T23:33:13.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.861+0000 7f1029e00640 1 -- 172.21.15.94:0/2747896234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f102415aff0 con 0x7f102415f2d0 2024-09-10T23:33:13.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.865+0000 7f101be00640 1 -- 172.21.15.94:0/2747896234 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f101c00ac50 con 0x7f102415f2d0 2024-09-10T23:33:13.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.865+0000 7f101be00640 1 --2- 172.21.15.94:0/2747896234 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10000725e0 0x7f1000074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:13.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.865+0000 7f101be00640 1 -- 172.21.15.94:0/2747896234 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f101c093120 con 0x7f102415f2d0 2024-09-10T23:33:13.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.865+0000 7f1022a00640 1 --2- 172.21.15.94:0/2747896234 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10000725e0 0x7f1000074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:13.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.865+0000 7f1022a00640 1 --2- 172.21.15.94:0/2747896234 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10000725e0 0x7f1000074aa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1014009670 tx=0x7f1014006bd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:13.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:13.869+0000 7f101be00640 1 -- 172.21.15.94:0/2747896234 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f101c05e760 con 0x7f102415f2d0 2024-09-10T23:33:14.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.045+0000 7f1029e00640 1 -- 172.21.15.94:0/2747896234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f102415c190 con 0x7f102415f2d0 2024-09-10T23:33:14.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.045+0000 7f101be00640 1 -- 172.21.15.94:0/2747896234 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f101c05e060 con 0x7f102415f2d0 2024-09-10T23:33:14.048 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:33:14.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.045+0000 7f1029e00640 1 -- 172.21.15.94:0/2747896234 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10000725e0 msgr2=0x7f1000074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:14.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.045+0000 7f1029e00640 1 --2- 172.21.15.94:0/2747896234 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10000725e0 0x7f1000074aa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1014009670 tx=0x7f1014006bd0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.045+0000 7f1029e00640 1 -- 172.21.15.94:0/2747896234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f102415f2d0 msgr2=0x7f10241f2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:14.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.045+0000 7f1029e00640 1 --2- 172.21.15.94:0/2747896234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f102415f2d0 0x7f10241f2c50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f101c00c730 tx=0x7f101c00cbf0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.045+0000 7f1029e00640 1 -- 172.21.15.94:0/2747896234 shutdown_connections 2024-09-10T23:33:14.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.045+0000 7f1029e00640 1 --2- 172.21.15.94:0/2747896234 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10000725e0 0x7f1000074aa0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.045+0000 7f1029e00640 1 --2- 172.21.15.94:0/2747896234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10240cfc00 0x7f10240cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.045+0000 7f1029e00640 1 --2- 172.21.15.94:0/2747896234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f102415f2d0 0x7f10241f2c50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.045+0000 7f1029e00640 1 --2- 172.21.15.94:0/2747896234 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10240b1d00 0x7f10240ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.045+0000 7f1029e00640 1 -- 172.21.15.94:0/2747896234 >> 172.21.15.94:0/2747896234 conn(0x7f10240d1c90 msgr2=0x7f10240d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:14.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.045+0000 7f1029e00640 1 -- 172.21.15.94:0/2747896234 shutdown_connections 2024-09-10T23:33:14.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.045+0000 7f1029e00640 1 -- 172.21.15.94:0/2747896234 wait complete. 2024-09-10T23:33:14.069 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:33:14.069 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:14.078 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:33:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13dde00640 1 -- 172.21.15.94:0/2205150531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d806e7c0 msgr2=0x7f13d806ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13dde00640 1 --2- 172.21.15.94:0/2205150531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d806e7c0 0x7f13d806ec00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009900 tx=0x7f13c401c880 comp rx=0 tx=0).stop 2024-09-10T23:33:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13dde00640 1 -- 172.21.15.94:0/2205150531 shutdown_connections 2024-09-10T23:33:14.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13dde00640 1 --2- 172.21.15.94:0/2205150531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d806e7c0 0x7f13d806ec00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13dde00640 1 --2- 172.21.15.94:0/2205150531 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13d80c64f0 0x7f13d80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13dde00640 1 -- 172.21.15.94:0/2205150531 >> 172.21.15.94:0/2205150531 conn(0x7f13d80cfaf0 msgr2=0x7f13d80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:14.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13dde00640 1 -- 172.21.15.94:0/2205150531 shutdown_connections 2024-09-10T23:33:14.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13dde00640 1 -- 172.21.15.94:0/2205150531 wait complete. 2024-09-10T23:33:14.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13dde00640 1 Processor -- start 2024-09-10T23:33:14.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13dde00640 1 -- start start 2024-09-10T23:33:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13dde00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13d806e7c0 0x7f13d8156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13dde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d80c64f0 0x7f13d8151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13dde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13d815af90 0x7f13d8155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13dde00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f13d8168400 con 0x7f13d80c64f0 2024-09-10T23:33:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13dde00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f13d80cec10 con 0x7f13d815af90 2024-09-10T23:33:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13dde00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f13d8168580 con 0x7f13d806e7c0 2024-09-10T23:33:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13d6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d80c64f0 0x7f13d8151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13d6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d80c64f0 0x7f13d8151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38494/0 (socket says 172.21.15.94:38494) 2024-09-10T23:33:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13d6000640 1 -- 172.21.15.94:0/377025870 learned_addr learned my addr 172.21.15.94:0/377025870 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13d6a00640 1 --2- 172.21.15.94:0/377025870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13d806e7c0 0x7f13d8156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13d7400640 1 --2- 172.21.15.94:0/377025870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13d815af90 0x7f13d8155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13d6000640 1 -- 172.21.15.94:0/377025870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13d806e7c0 msgr2=0x7f13d8156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13d6000640 1 --2- 172.21.15.94:0/377025870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13d806e7c0 0x7f13d8156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13d6000640 1 -- 172.21.15.94:0/377025870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13d815af90 msgr2=0x7f13d8155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13d6000640 1 --2- 172.21.15.94:0/377025870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13d815af90 0x7f13d8155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.257+0000 7f13d6000640 1 -- 172.21.15.94:0/377025870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d815daa0 con 0x7f13d80c64f0 2024-09-10T23:33:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.261+0000 7f13d6000640 1 --2- 172.21.15.94:0/377025870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d80c64f0 0x7f13d8151720 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f13c40b0040 tx=0x7f13c40a58a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.261+0000 7f13cf400640 1 -- 172.21.15.94:0/377025870 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c40b2040 con 0x7f13d80c64f0 2024-09-10T23:33:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.261+0000 7f13cf400640 1 -- 172.21.15.94:0/377025870 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13c4004480 con 0x7f13d80c64f0 2024-09-10T23:33:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.261+0000 7f13dde00640 1 -- 172.21.15.94:0/377025870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d80b3d60 con 0x7f13d80c64f0 2024-09-10T23:33:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.261+0000 7f13cf400640 1 -- 172.21.15.94:0/377025870 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4005290 con 0x7f13d80c64f0 2024-09-10T23:33:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.261+0000 7f13dde00640 1 -- 172.21.15.94:0/377025870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d80b2ef0 con 0x7f13d80c64f0 2024-09-10T23:33:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.261+0000 7f13dde00640 1 -- 172.21.15.94:0/377025870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d806ec00 con 0x7f13d80c64f0 2024-09-10T23:33:14.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.261+0000 7f13cf400640 1 -- 172.21.15.94:0/377025870 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f13c4005430 con 0x7f13d80c64f0 2024-09-10T23:33:14.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.261+0000 7f13cf400640 1 --2- 172.21.15.94:0/377025870 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f13b80725e0 0x7f13b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:14.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.261+0000 7f13cf400640 1 -- 172.21.15.94:0/377025870 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f13c412d3c0 con 0x7f13d80c64f0 2024-09-10T23:33:14.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.261+0000 7f13d6a00640 1 --2- 172.21.15.94:0/377025870 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f13b80725e0 0x7f13b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:14.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.265+0000 7f13d6a00640 1 --2- 172.21.15.94:0/377025870 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f13b80725e0 0x7f13b8074aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f13c80058d0 tx=0x7f13c8005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:14.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.265+0000 7f13cf400640 1 -- 172.21.15.94:0/377025870 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f13c40c4020 con 0x7f13d80c64f0 2024-09-10T23:33:14.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.441+0000 7f13dde00640 1 -- 172.21.15.94:0/377025870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f13d80c56e0 con 0x7f13d80c64f0 2024-09-10T23:33:14.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.441+0000 7f13cf400640 1 -- 172.21.15.94:0/377025870 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f13c40164d0 con 0x7f13d80c64f0 2024-09-10T23:33:14.445 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:33:14.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.445+0000 7f13dde00640 1 -- 172.21.15.94:0/377025870 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f13b80725e0 msgr2=0x7f13b8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:14.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.445+0000 7f13dde00640 1 --2- 172.21.15.94:0/377025870 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f13b80725e0 0x7f13b8074aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f13c80058d0 tx=0x7f13c8005860 comp rx=0 tx=0).stop 2024-09-10T23:33:14.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.445+0000 7f13dde00640 1 -- 172.21.15.94:0/377025870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d80c64f0 msgr2=0x7f13d8151720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:14.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.445+0000 7f13dde00640 1 --2- 172.21.15.94:0/377025870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d80c64f0 0x7f13d8151720 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f13c40b0040 tx=0x7f13c40a58a0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.445+0000 7f13dde00640 1 -- 172.21.15.94:0/377025870 shutdown_connections 2024-09-10T23:33:14.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.445+0000 7f13dde00640 1 --2- 172.21.15.94:0/377025870 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f13b80725e0 0x7f13b8074aa0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.445+0000 7f13dde00640 1 --2- 172.21.15.94:0/377025870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13d815af90 0x7f13d8155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.445+0000 7f13dde00640 1 --2- 172.21.15.94:0/377025870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13d80c64f0 0x7f13d8151720 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.445+0000 7f13dde00640 1 --2- 172.21.15.94:0/377025870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13d806e7c0 0x7f13d8156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.445+0000 7f13dde00640 1 -- 172.21.15.94:0/377025870 >> 172.21.15.94:0/377025870 conn(0x7f13d80cfaf0 msgr2=0x7f13d81570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:14.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.445+0000 7f13dde00640 1 -- 172.21.15.94:0/377025870 shutdown_connections 2024-09-10T23:33:14.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.445+0000 7f13dde00640 1 -- 172.21.15.94:0/377025870 wait complete. 2024-09-10T23:33:14.465 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-09-10T23:33:14.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:14.466 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:14.495 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:33:14.495 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:33:14.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:14.496 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T23:33:14.554 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:33:14.554 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:14.564 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:33:14.565 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:14.618 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:33:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.757+0000 7fb3f2e00640 1 -- 172.21.15.94:0/1516264370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3ec158790 msgr2=0x7fb3ec06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.757+0000 7fb3f2e00640 1 --2- 172.21.15.94:0/1516264370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3ec158790 0x7fb3ec06fc00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009900 tx=0x7fb3dc01c8e0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.757+0000 7fb3f2e00640 1 -- 172.21.15.94:0/1516264370 shutdown_connections 2024-09-10T23:33:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.757+0000 7fb3f2e00640 1 --2- 172.21.15.94:0/1516264370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3ec0c49b0 0x7fb3ec0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.757+0000 7fb3f2e00640 1 --2- 172.21.15.94:0/1516264370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3ec158790 0x7fb3ec06fc00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.757+0000 7fb3f2e00640 1 -- 172.21.15.94:0/1516264370 >> 172.21.15.94:0/1516264370 conn(0x7fb3ec0cfaf0 msgr2=0x7fb3ec0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.757+0000 7fb3f2e00640 1 -- 172.21.15.94:0/1516264370 shutdown_connections 2024-09-10T23:33:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.757+0000 7fb3f2e00640 1 -- 172.21.15.94:0/1516264370 wait complete. 2024-09-10T23:33:14.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f2e00640 1 Processor -- start 2024-09-10T23:33:14.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f2e00640 1 -- start start 2024-09-10T23:33:14.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f2e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3ec0c49b0 0x7fb3ec149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f2e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3ec0cb300 0x7fb3ec15ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f2e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3ec158790 0x7fb3ec1f9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f2e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec168290 con 0x7fb3ec158790 2024-09-10T23:33:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f2e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec168110 con 0x7fb3ec0c49b0 2024-09-10T23:33:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f2e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb3ec168410 con 0x7fb3ec0cb300 2024-09-10T23:33:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f0e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3ec158790 0x7fb3ec1f9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f0e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3ec158790 0x7fb3ec1f9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38504/0 (socket says 172.21.15.94:38504) 2024-09-10T23:33:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f0e00640 1 -- 172.21.15.94:0/1519827792 learned_addr learned my addr 172.21.15.94:0/1519827792 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3e3400640 1 --2- 172.21.15.94:0/1519827792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3ec0cb300 0x7fb3ec15ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3ebe00640 1 --2- 172.21.15.94:0/1519827792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3ec0c49b0 0x7fb3ec149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f0e00640 1 -- 172.21.15.94:0/1519827792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3ec0cb300 msgr2=0x7fb3ec15ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f0e00640 1 --2- 172.21.15.94:0/1519827792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3ec0cb300 0x7fb3ec15ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f0e00640 1 -- 172.21.15.94:0/1519827792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3ec0c49b0 msgr2=0x7fb3ec149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f0e00640 1 --2- 172.21.15.94:0/1519827792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3ec0c49b0 0x7fb3ec149150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f0e00640 1 -- 172.21.15.94:0/1519827792 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ec15c870 con 0x7fb3ec158790 2024-09-10T23:33:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3ebe00640 1 --2- 172.21.15.94:0/1519827792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3ec0c49b0 0x7fb3ec149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f0e00640 1 --2- 172.21.15.94:0/1519827792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3ec158790 0x7fb3ec1f9e30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400c7b0 tx=0x7fb3e400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3e8c00640 1 -- 172.21.15.94:0/1519827792 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4003ba0 con 0x7fb3ec158790 2024-09-10T23:33:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3e8c00640 1 -- 172.21.15.94:0/1519827792 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3e4003d60 con 0x7fb3ec158790 2024-09-10T23:33:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3e8c00640 1 -- 172.21.15.94:0/1519827792 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e400aa50 con 0x7fb3ec158790 2024-09-10T23:33:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f2e00640 1 -- 172.21.15.94:0/1519827792 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3ec15cb10 con 0x7fb3ec158790 2024-09-10T23:33:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.761+0000 7fb3f2e00640 1 -- 172.21.15.94:0/1519827792 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3ec15d0d0 con 0x7fb3ec158790 2024-09-10T23:33:14.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.765+0000 7fb3f2e00640 1 -- 172.21.15.94:0/1519827792 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3ec15d2d0 con 0x7fb3ec158790 2024-09-10T23:33:14.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.765+0000 7fb3e8c00640 1 -- 172.21.15.94:0/1519827792 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb3e400abf0 con 0x7fb3ec158790 2024-09-10T23:33:14.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.765+0000 7fb3e8c00640 1 --2- 172.21.15.94:0/1519827792 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3c80725e0 0x7fb3c8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:14.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.765+0000 7fb3e8c00640 1 -- 172.21.15.94:0/1519827792 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb3e40930d0 con 0x7fb3ec158790 2024-09-10T23:33:14.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.765+0000 7fb3ebe00640 1 --2- 172.21.15.94:0/1519827792 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3c80725e0 0x7fb3c8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:14.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.765+0000 7fb3ebe00640 1 --2- 172.21.15.94:0/1519827792 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3c80725e0 0x7fb3c8074aa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0af000 tx=0x7fb3dc01b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:14.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.769+0000 7fb3e8c00640 1 -- 172.21.15.94:0/1519827792 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb3ec15d2d0 con 0x7fb3ec158790 2024-09-10T23:33:14.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.945+0000 7fb3f2e00640 1 -- 172.21.15.94:0/1519827792 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb3ec0c67a0 con 0x7fb3ec158790 2024-09-10T23:33:14.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.945+0000 7fb3e8c00640 1 -- 172.21.15.94:0/1519827792 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fb3e405e710 con 0x7fb3ec158790 2024-09-10T23:33:14.948 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:33:14.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.949+0000 7fb3f2e00640 1 -- 172.21.15.94:0/1519827792 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3c80725e0 msgr2=0x7fb3c8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.949+0000 7fb3f2e00640 1 --2- 172.21.15.94:0/1519827792 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3c80725e0 0x7fb3c8074aa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0af000 tx=0x7fb3dc01b120 comp rx=0 tx=0).stop 2024-09-10T23:33:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.949+0000 7fb3f2e00640 1 -- 172.21.15.94:0/1519827792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3ec158790 msgr2=0x7fb3ec1f9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.949+0000 7fb3f2e00640 1 --2- 172.21.15.94:0/1519827792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3ec158790 0x7fb3ec1f9e30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400c7b0 tx=0x7fb3e400cc70 comp rx=0 tx=0).stop 2024-09-10T23:33:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.949+0000 7fb3f2e00640 1 -- 172.21.15.94:0/1519827792 shutdown_connections 2024-09-10T23:33:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.949+0000 7fb3f2e00640 1 --2- 172.21.15.94:0/1519827792 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3c80725e0 0x7fb3c8074aa0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.949+0000 7fb3f2e00640 1 --2- 172.21.15.94:0/1519827792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3ec158790 0x7fb3ec1f9e30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.949+0000 7fb3f2e00640 1 --2- 172.21.15.94:0/1519827792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3ec0cb300 0x7fb3ec15ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.949+0000 7fb3f2e00640 1 --2- 172.21.15.94:0/1519827792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3ec0c49b0 0x7fb3ec149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.949+0000 7fb3f2e00640 1 -- 172.21.15.94:0/1519827792 >> 172.21.15.94:0/1519827792 conn(0x7fb3ec0cfaf0 msgr2=0x7fb3ec0b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.949+0000 7fb3f2e00640 1 -- 172.21.15.94:0/1519827792 shutdown_connections 2024-09-10T23:33:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:14.949+0000 7fb3f2e00640 1 -- 172.21.15.94:0/1519827792 wait complete. 2024-09-10T23:33:14.970 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:33:14.970 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:14.979 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:33:15.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89ea000640 1 -- 172.21.15.94:0/2054863203 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e415f560 msgr2=0x7f89e40cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:15.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89ea000640 1 --2- 172.21.15.94:0/2054863203 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e415f560 0x7f89e40cd0a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0098a0 tx=0x7f89cc01c840 comp rx=0 tx=0).stop 2024-09-10T23:33:15.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89ea000640 1 -- 172.21.15.94:0/2054863203 shutdown_connections 2024-09-10T23:33:15.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89ea000640 1 --2- 172.21.15.94:0/2054863203 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e415f560 0x7f89e40cd0a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:15.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89ea000640 1 -- 172.21.15.94:0/2054863203 >> 172.21.15.94:0/2054863203 conn(0x7f89e40d1cb0 msgr2=0x7f89e40d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:15.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89ea000640 1 -- 172.21.15.94:0/2054863203 shutdown_connections 2024-09-10T23:33:15.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89ea000640 1 -- 172.21.15.94:0/2054863203 wait complete. 2024-09-10T23:33:15.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89ea000640 1 Processor -- start 2024-09-10T23:33:15.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89ea000640 1 -- start start 2024-09-10T23:33:15.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89ea000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e40d01a0 0x7f89e4166650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:15.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89ea000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89e415f180 0x7f89e415b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:15.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89ea000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e415f560 0x7f89e40cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:15.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89ea000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f89e4169ea0 con 0x7f89e415f180 2024-09-10T23:33:15.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89ea000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f89e4169d20 con 0x7f89e415f560 2024-09-10T23:33:15.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89ea000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f89e416a020 con 0x7f89e40d01a0 2024-09-10T23:33:15.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89daa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89e415f180 0x7f89e415b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:15.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89daa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89e415f180 0x7f89e415b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38518/0 (socket says 172.21.15.94:38518) 2024-09-10T23:33:15.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89daa00640 1 -- 172.21.15.94:0/2797563472 learned_addr learned my addr 172.21.15.94:0/2797563472 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:15.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89e3e00640 1 --2- 172.21.15.94:0/2797563472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e415f560 0x7f89e40cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:15.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89e3400640 1 --2- 172.21.15.94:0/2797563472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e40d01a0 0x7f89e4166650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:15.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89daa00640 1 -- 172.21.15.94:0/2797563472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e40d01a0 msgr2=0x7f89e4166650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:15.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89daa00640 1 --2- 172.21.15.94:0/2797563472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e40d01a0 0x7f89e4166650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:15.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89daa00640 1 -- 172.21.15.94:0/2797563472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e415f560 msgr2=0x7f89e40cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:15.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89daa00640 1 --2- 172.21.15.94:0/2797563472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e415f560 0x7f89e40cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:15.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89daa00640 1 -- 172.21.15.94:0/2797563472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e41655b0 con 0x7f89e415f180 2024-09-10T23:33:15.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.121+0000 7f89e3e00640 1 --2- 172.21.15.94:0/2797563472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e415f560 0x7f89e40cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:15.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.125+0000 7f89daa00640 1 --2- 172.21.15.94:0/2797563472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89e415f180 0x7f89e415b490 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f89cc01a470 tx=0x7f89cc005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:15.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.125+0000 7f89dbe00640 1 -- 172.21.15.94:0/2797563472 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc0b2040 con 0x7f89e415f180 2024-09-10T23:33:15.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.125+0000 7f89ea000640 1 -- 172.21.15.94:0/2797563472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e4165890 con 0x7f89e415f180 2024-09-10T23:33:15.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.125+0000 7f89dbe00640 1 -- 172.21.15.94:0/2797563472 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89cc004320 con 0x7f89e415f180 2024-09-10T23:33:15.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.125+0000 7f89dbe00640 1 -- 172.21.15.94:0/2797563472 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc0052d0 con 0x7f89e415f180 2024-09-10T23:33:15.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.125+0000 7f89ea000640 1 -- 172.21.15.94:0/2797563472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e4165e50 con 0x7f89e415f180 2024-09-10T23:33:15.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.125+0000 7f89ea000640 1 -- 172.21.15.94:0/2797563472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89e40cd0a0 con 0x7f89e415f180 2024-09-10T23:33:15.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.125+0000 7f89dbe00640 1 -- 172.21.15.94:0/2797563472 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f89cc0b5070 con 0x7f89e415f180 2024-09-10T23:33:15.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.125+0000 7f89dbe00640 1 --2- 172.21.15.94:0/2797563472 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f89c00725e0 0x7f89c0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:15.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.125+0000 7f89dbe00640 1 -- 172.21.15.94:0/2797563472 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f89cc12cc70 con 0x7f89e415f180 2024-09-10T23:33:15.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.125+0000 7f89e3400640 1 --2- 172.21.15.94:0/2797563472 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f89c00725e0 0x7f89c0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:15.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.129+0000 7f89e3400640 1 --2- 172.21.15.94:0/2797563472 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f89c00725e0 0x7f89c0074aa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f89e40d00b0 tx=0x7f89d4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:15.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.129+0000 7f89dbe00640 1 -- 172.21.15.94:0/2797563472 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f89cc016430 con 0x7f89e415f180 2024-09-10T23:33:15.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.309+0000 7f89ea000640 1 -- 172.21.15.94:0/2797563472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f89e415b9d0 con 0x7f89e415f180 2024-09-10T23:33:15.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.309+0000 7f89dbe00640 1 -- 172.21.15.94:0/2797563472 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f89cc0f8230 con 0x7f89e415f180 2024-09-10T23:33:15.311 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:33:15.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.309+0000 7f89ea000640 1 -- 172.21.15.94:0/2797563472 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f89c00725e0 msgr2=0x7f89c0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:15.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.309+0000 7f89ea000640 1 --2- 172.21.15.94:0/2797563472 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f89c00725e0 0x7f89c0074aa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f89e40d00b0 tx=0x7f89d4009040 comp rx=0 tx=0).stop 2024-09-10T23:33:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.309+0000 7f89ea000640 1 -- 172.21.15.94:0/2797563472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89e415f180 msgr2=0x7f89e415b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.309+0000 7f89ea000640 1 --2- 172.21.15.94:0/2797563472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89e415f180 0x7f89e415b490 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f89cc01a470 tx=0x7f89cc005d20 comp rx=0 tx=0).stop 2024-09-10T23:33:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.309+0000 7f89ea000640 1 -- 172.21.15.94:0/2797563472 shutdown_connections 2024-09-10T23:33:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.309+0000 7f89ea000640 1 --2- 172.21.15.94:0/2797563472 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f89c00725e0 0x7f89c0074aa0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.309+0000 7f89ea000640 1 --2- 172.21.15.94:0/2797563472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e415f560 0x7f89e40cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.309+0000 7f89ea000640 1 --2- 172.21.15.94:0/2797563472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89e415f180 0x7f89e415b490 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.309+0000 7f89ea000640 1 --2- 172.21.15.94:0/2797563472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e40d01a0 0x7f89e4166650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.309+0000 7f89ea000640 1 -- 172.21.15.94:0/2797563472 >> 172.21.15.94:0/2797563472 conn(0x7f89e40d1cb0 msgr2=0x7f89e40d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.309+0000 7f89ea000640 1 -- 172.21.15.94:0/2797563472 shutdown_connections 2024-09-10T23:33:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.309+0000 7f89ea000640 1 -- 172.21.15.94:0/2797563472 wait complete. 2024-09-10T23:33:15.332 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi181.front.sepia.ceph.com... 2024-09-10T23:33:15.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:15.332 DEBUG:teuthology.orchestra.run.smithi181:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:15.369 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:33:15.369 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:33:15.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:15.370 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T23:33:15.448 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:33:15.448 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:15.457 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:33:15.457 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:15.506 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:33:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36c600640 1 -- 172.21.15.94:0/2227046477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff34c003a60 msgr2=0x7ff34c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36c600640 1 --2- 172.21.15.94:0/2227046477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff34c003a60 0x7ff34c003e80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff358006fc0 tx=0x7ff35801cf40 comp rx=0 tx=0).stop 2024-09-10T23:33:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36a600640 1 -- 172.21.15.94:0/2227046477 reap_dead start 2024-09-10T23:33:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36c600640 1 -- 172.21.15.94:0/2227046477 shutdown_connections 2024-09-10T23:33:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36c600640 1 -- 172.21.15.94:0/2227046477 >> 172.21.15.94:0/2227046477 conn(0x7ff3640d1c60 msgr2=0x7ff3640d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36c600640 1 -- 172.21.15.94:0/2227046477 shutdown_connections 2024-09-10T23:33:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36c600640 1 -- 172.21.15.94:0/2227046477 wait complete. 2024-09-10T23:33:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36c600640 1 Processor -- start 2024-09-10T23:33:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36c600640 1 -- start start 2024-09-10T23:33:15.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36c600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff34c003a60 0x7ff3641fb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:15.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36c600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3640cb8b0 0x7ff36406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:15.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36c600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364156b00 0x7ff364156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:15.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36c600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3640c7020 con 0x7ff34c003a60 2024-09-10T23:33:15.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36c600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff3640c6ea0 con 0x7ff364156b00 2024-09-10T23:33:15.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36c600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff364166970 con 0x7ff3640cb8b0 2024-09-10T23:33:15.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff369c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff34c003a60 0x7ff3641fb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:15.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff369c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff34c003a60 0x7ff3641fb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38542/0 (socket says 172.21.15.94:38542) 2024-09-10T23:33:15.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff369c00640 1 -- 172.21.15.94:0/3285827556 learned_addr learned my addr 172.21.15.94:0/3285827556 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff369200640 1 --2- 172.21.15.94:0/3285827556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3640cb8b0 0x7ff36406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36a600640 1 --2- 172.21.15.94:0/3285827556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364156b00 0x7ff364156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff369c00640 1 -- 172.21.15.94:0/3285827556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3640cb8b0 msgr2=0x7ff36406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff369c00640 1 --2- 172.21.15.94:0/3285827556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3640cb8b0 0x7ff36406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff369c00640 1 -- 172.21.15.94:0/3285827556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364156b00 msgr2=0x7ff364156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff369c00640 1 --2- 172.21.15.94:0/3285827556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364156b00 0x7ff364156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff369c00640 1 -- 172.21.15.94:0/3285827556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3640c9eb0 con 0x7ff34c003a60 2024-09-10T23:33:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.697+0000 7ff36a600640 1 --2- 172.21.15.94:0/3285827556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364156b00 0x7ff364156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.701+0000 7ff369c00640 1 --2- 172.21.15.94:0/3285827556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff34c003a60 0x7ff3641fb4f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff358098b10 tx=0x7ff3580019d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:15.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.701+0000 7ff361600640 1 -- 172.21.15.94:0/3285827556 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff358001c60 con 0x7ff34c003a60 2024-09-10T23:33:15.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.701+0000 7ff36c600640 1 -- 172.21.15.94:0/3285827556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3640ca190 con 0x7ff34c003a60 2024-09-10T23:33:15.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.701+0000 7ff361600640 1 -- 172.21.15.94:0/3285827556 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff358006380 con 0x7ff34c003a60 2024-09-10T23:33:15.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.701+0000 7ff361600640 1 -- 172.21.15.94:0/3285827556 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff358003860 con 0x7ff34c003a60 2024-09-10T23:33:15.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.701+0000 7ff36c600640 1 -- 172.21.15.94:0/3285827556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3640ca750 con 0x7ff34c003a60 2024-09-10T23:33:15.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.701+0000 7ff36c600640 1 -- 172.21.15.94:0/3285827556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3640cfce0 con 0x7ff34c003a60 2024-09-10T23:33:15.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.705+0000 7ff361600640 1 -- 172.21.15.94:0/3285827556 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff358003a00 con 0x7ff34c003a60 2024-09-10T23:33:15.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.705+0000 7ff361600640 1 --2- 172.21.15.94:0/3285827556 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff3440725e0 0x7ff344074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:15.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.705+0000 7ff361600640 1 -- 172.21.15.94:0/3285827556 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff35812daf0 con 0x7ff34c003a60 2024-09-10T23:33:15.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.705+0000 7ff369200640 1 --2- 172.21.15.94:0/3285827556 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff3440725e0 0x7ff344074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:15.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.705+0000 7ff369200640 1 --2- 172.21.15.94:0/3285827556 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff3440725e0 0x7ff344074aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff364072670 tx=0x7ff3500091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:15.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.705+0000 7ff361600640 1 -- 172.21.15.94:0/3285827556 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff358016cf0 con 0x7ff34c003a60 2024-09-10T23:33:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.885+0000 7ff36c600640 1 -- 172.21.15.94:0/3285827556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff3641574d0 con 0x7ff34c003a60 2024-09-10T23:33:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.885+0000 7ff361600640 1 -- 172.21.15.94:0/3285827556 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7ff3580f92d0 con 0x7ff34c003a60 2024-09-10T23:33:15.888 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:33:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.885+0000 7ff36c600640 1 -- 172.21.15.94:0/3285827556 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff3440725e0 msgr2=0x7ff344074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.885+0000 7ff36c600640 1 --2- 172.21.15.94:0/3285827556 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff3440725e0 0x7ff344074aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff364072670 tx=0x7ff3500091e0 comp rx=0 tx=0).stop 2024-09-10T23:33:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.885+0000 7ff36c600640 1 -- 172.21.15.94:0/3285827556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff34c003a60 msgr2=0x7ff3641fb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.885+0000 7ff36c600640 1 --2- 172.21.15.94:0/3285827556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff34c003a60 0x7ff3641fb4f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff358098b10 tx=0x7ff3580019d0 comp rx=0 tx=0).stop 2024-09-10T23:33:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.885+0000 7ff36c600640 1 -- 172.21.15.94:0/3285827556 shutdown_connections 2024-09-10T23:33:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.885+0000 7ff36c600640 1 --2- 172.21.15.94:0/3285827556 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff3440725e0 0x7ff344074aa0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.885+0000 7ff36c600640 1 --2- 172.21.15.94:0/3285827556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff364156b00 0x7ff364156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.885+0000 7ff36c600640 1 --2- 172.21.15.94:0/3285827556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3640cb8b0 0x7ff36406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.885+0000 7ff36c600640 1 --2- 172.21.15.94:0/3285827556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff34c003a60 0x7ff3641fb4f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.885+0000 7ff36c600640 1 -- 172.21.15.94:0/3285827556 >> 172.21.15.94:0/3285827556 conn(0x7ff3640d1c60 msgr2=0x7ff36415ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.889+0000 7ff36c600640 1 -- 172.21.15.94:0/3285827556 shutdown_connections 2024-09-10T23:33:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:15.889+0000 7ff36c600640 1 -- 172.21.15.94:0/3285827556 wait complete. 2024-09-10T23:33:15.909 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:33:15.909 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:15.918 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:33:16.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cdce00640 1 -- 172.21.15.94:0/1105040964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cbc0041d0 msgr2=0x7f3cbc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:16.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cdce00640 1 --2- 172.21.15.94:0/1105040964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cbc0041d0 0x7f3cbc004810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009b30 tx=0x7f3cc401d230 comp rx=0 tx=0).stop 2024-09-10T23:33:16.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cd7e00640 1 -- 172.21.15.94:0/1105040964 reap_dead start 2024-09-10T23:33:16.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cdce00640 1 -- 172.21.15.94:0/1105040964 shutdown_connections 2024-09-10T23:33:16.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cdce00640 1 -- 172.21.15.94:0/1105040964 >> 172.21.15.94:0/1105040964 conn(0x7f3cd80d1b00 msgr2=0x7f3cd80d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:16.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cdce00640 1 -- 172.21.15.94:0/1105040964 shutdown_connections 2024-09-10T23:33:16.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cdce00640 1 -- 172.21.15.94:0/1105040964 wait complete. 2024-09-10T23:33:16.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cdce00640 1 Processor -- start 2024-09-10T23:33:16.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cdce00640 1 -- start start 2024-09-10T23:33:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cdce00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cbc0041d0 0x7f3cd8151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cdce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cd80cc570 0x7f3cd8152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cdce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cd81531b0 0x7f3cd80ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cdce00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd80afe90 con 0x7f3cd81531b0 2024-09-10T23:33:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cdce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd80afd10 con 0x7f3cd80cc570 2024-09-10T23:33:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cdce00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd80b0010 con 0x7f3cbc0041d0 2024-09-10T23:33:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cd7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cd81531b0 0x7f3cd80ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cd7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cd81531b0 0x7f3cd80ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38554/0 (socket says 172.21.15.94:38554) 2024-09-10T23:33:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cd7e00640 1 -- 172.21.15.94:0/4002702298 learned_addr learned my addr 172.21.15.94:0/4002702298 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cd7e00640 1 -- 172.21.15.94:0/4002702298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cbc0041d0 msgr2=0x7f3cd8151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:16.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cd7e00640 1 --2- 172.21.15.94:0/4002702298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cbc0041d0 0x7f3cd8151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cd7e00640 1 -- 172.21.15.94:0/4002702298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cd80cc570 msgr2=0x7f3cd8152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cd7e00640 1 --2- 172.21.15.94:0/4002702298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cd80cc570 0x7f3cd8152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.105+0000 7f3cd7e00640 1 -- 172.21.15.94:0/4002702298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd8152850 con 0x7f3cd81531b0 2024-09-10T23:33:16.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.109+0000 7f3cd7e00640 1 --2- 172.21.15.94:0/4002702298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cd81531b0 0x7f3cd80ceaf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80b0bd0 tx=0x7f3cd000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:16.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.109+0000 7f3ccf400640 1 -- 172.21.15.94:0/4002702298 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0015070 con 0x7f3cd81531b0 2024-09-10T23:33:16.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.109+0000 7f3ccf400640 1 -- 172.21.15.94:0/4002702298 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cd000ca30 con 0x7f3cd81531b0 2024-09-10T23:33:16.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.109+0000 7f3cdce00640 1 -- 172.21.15.94:0/4002702298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd8152a60 con 0x7f3cd81531b0 2024-09-10T23:33:16.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.109+0000 7f3ccf400640 1 -- 172.21.15.94:0/4002702298 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd000ce80 con 0x7f3cd81531b0 2024-09-10T23:33:16.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.109+0000 7f3cdce00640 1 -- 172.21.15.94:0/4002702298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd8151830 con 0x7f3cd81531b0 2024-09-10T23:33:16.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.109+0000 7f3cdce00640 1 -- 172.21.15.94:0/4002702298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd8151650 con 0x7f3cd81531b0 2024-09-10T23:33:16.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.109+0000 7f3ccf400640 1 -- 172.21.15.94:0/4002702298 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3cd0011460 con 0x7f3cd81531b0 2024-09-10T23:33:16.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.109+0000 7f3ccf400640 1 --2- 172.21.15.94:0/4002702298 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3cb80725e0 0x7f3cb8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:16.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.109+0000 7f3ccf400640 1 -- 172.21.15.94:0/4002702298 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3cd0093920 con 0x7f3cd81531b0 2024-09-10T23:33:16.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.109+0000 7f3cd7400640 1 --2- 172.21.15.94:0/4002702298 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3cb80725e0 0x7f3cb8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:16.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.113+0000 7f3cd7400640 1 --2- 172.21.15.94:0/4002702298 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3cb80725e0 0x7f3cb8074aa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80b40e0 tx=0x7f3cc800aa20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:16.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.113+0000 7f3ccf400640 1 -- 172.21.15.94:0/4002702298 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3cd8151650 con 0x7f3cd81531b0 2024-09-10T23:33:16.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.289+0000 7f3cdce00640 1 -- 172.21.15.94:0/4002702298 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3cd80af600 con 0x7f3cd81531b0 2024-09-10T23:33:16.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.293+0000 7f3ccf400640 1 -- 172.21.15.94:0/4002702298 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f3cd005eff0 con 0x7f3cd81531b0 2024-09-10T23:33:16.294 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:33:16.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.293+0000 7f3cdce00640 1 -- 172.21.15.94:0/4002702298 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3cb80725e0 msgr2=0x7f3cb8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.293+0000 7f3cdce00640 1 --2- 172.21.15.94:0/4002702298 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3cb80725e0 0x7f3cb8074aa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80b40e0 tx=0x7f3cc800aa20 comp rx=0 tx=0).stop 2024-09-10T23:33:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.293+0000 7f3cdce00640 1 -- 172.21.15.94:0/4002702298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cd81531b0 msgr2=0x7f3cd80ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.293+0000 7f3cdce00640 1 --2- 172.21.15.94:0/4002702298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cd81531b0 0x7f3cd80ceaf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80b0bd0 tx=0x7f3cd000ed10 comp rx=0 tx=0).stop 2024-09-10T23:33:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.293+0000 7f3cdce00640 1 -- 172.21.15.94:0/4002702298 shutdown_connections 2024-09-10T23:33:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.293+0000 7f3cdce00640 1 --2- 172.21.15.94:0/4002702298 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3cb80725e0 0x7f3cb8074aa0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.293+0000 7f3cdce00640 1 --2- 172.21.15.94:0/4002702298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cd80cc570 0x7f3cd8152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.293+0000 7f3cdce00640 1 --2- 172.21.15.94:0/4002702298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cd81531b0 0x7f3cd80ceaf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.293+0000 7f3cdce00640 1 --2- 172.21.15.94:0/4002702298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cbc0041d0 0x7f3cd8151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.293+0000 7f3cdce00640 1 -- 172.21.15.94:0/4002702298 >> 172.21.15.94:0/4002702298 conn(0x7f3cd80d1b00 msgr2=0x7f3cd815cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.293+0000 7f3cdce00640 1 -- 172.21.15.94:0/4002702298 shutdown_connections 2024-09-10T23:33:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.293+0000 7f3cdce00640 1 -- 172.21.15.94:0/4002702298 wait complete. 2024-09-10T23:33:16.315 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-09-10T23:33:16.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:16.316 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-09-10T23:33:16.336 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:33:16.336 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:33:16.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:16.336 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T23:33:16.390 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:33:16.390 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:16.400 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:33:16.400 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:16.450 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:33:16.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.589+0000 7f84f9600640 1 -- 172.21.15.94:0/1142497343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f40ce890 msgr2=0x7f84f40cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:16.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.589+0000 7f84f9600640 1 --2- 172.21.15.94:0/1142497343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f40ce890 0x7f84f40cd710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009900 tx=0x7f84e401c850 comp rx=0 tx=0).stop 2024-09-10T23:33:16.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.589+0000 7f84f9600640 1 -- 172.21.15.94:0/1142497343 shutdown_connections 2024-09-10T23:33:16.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.589+0000 7f84f9600640 1 --2- 172.21.15.94:0/1142497343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84f40b1100 0x7f84f40b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.589+0000 7f84f9600640 1 --2- 172.21.15.94:0/1142497343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f40ce890 0x7f84f40cd710 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.589+0000 7f84f9600640 1 -- 172.21.15.94:0/1142497343 >> 172.21.15.94:0/1142497343 conn(0x7f84f40d1c60 msgr2=0x7f84f40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:16.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.589+0000 7f84f9600640 1 -- 172.21.15.94:0/1142497343 shutdown_connections 2024-09-10T23:33:16.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.589+0000 7f84f9600640 1 -- 172.21.15.94:0/1142497343 wait complete. 2024-09-10T23:33:16.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f9600640 1 Processor -- start 2024-09-10T23:33:16.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f9600640 1 -- start start 2024-09-10T23:33:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f9600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f40b1100 0x7f84f4152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f9600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84f40ce1e0 0x7f84f4158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f9600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84f40ce890 0x7f84f406dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f9600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f84f4169110 con 0x7f84f40b1100 2024-09-10T23:33:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f9600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f84f40d09d0 con 0x7f84f40ce1e0 2024-09-10T23:33:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f9600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f84f4169290 con 0x7f84f40ce890 2024-09-10T23:33:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f40b1100 0x7f84f4152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f40b1100 0x7f84f4152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38572/0 (socket says 172.21.15.94:38572) 2024-09-10T23:33:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f2a00640 1 -- 172.21.15.94:0/352265682 learned_addr learned my addr 172.21.15.94:0/352265682 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84ea000640 1 --2- 172.21.15.94:0/352265682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84f40ce1e0 0x7f84f4158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f3400640 1 --2- 172.21.15.94:0/352265682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84f40ce890 0x7f84f406dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f2a00640 1 -- 172.21.15.94:0/352265682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84f40ce890 msgr2=0x7f84f406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f2a00640 1 --2- 172.21.15.94:0/352265682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84f40ce890 0x7f84f406dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f2a00640 1 -- 172.21.15.94:0/352265682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84f40ce1e0 msgr2=0x7f84f4158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f2a00640 1 --2- 172.21.15.94:0/352265682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84f40ce1e0 0x7f84f4158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f2a00640 1 -- 172.21.15.94:0/352265682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f4165430 con 0x7f84f40b1100 2024-09-10T23:33:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f3400640 1 --2- 172.21.15.94:0/352265682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84f40ce890 0x7f84f406dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:16.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f2a00640 1 --2- 172.21.15.94:0/352265682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f40b1100 0x7f84f4152a80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f84e40b0040 tx=0x7f84e40a5810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84eb400640 1 -- 172.21.15.94:0/352265682 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e40ba020 con 0x7f84f40b1100 2024-09-10T23:33:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84eb400640 1 -- 172.21.15.94:0/352265682 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84e40044b0 con 0x7f84f40b1100 2024-09-10T23:33:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84eb400640 1 -- 172.21.15.94:0/352265682 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e40b2040 con 0x7f84f40b1100 2024-09-10T23:33:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f9600640 1 -- 172.21.15.94:0/352265682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f4165710 con 0x7f84f40b1100 2024-09-10T23:33:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.593+0000 7f84f9600640 1 -- 172.21.15.94:0/352265682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f4165cd0 con 0x7f84f40b1100 2024-09-10T23:33:16.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.597+0000 7f84f9600640 1 -- 172.21.15.94:0/352265682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f40b1590 con 0x7f84f40b1100 2024-09-10T23:33:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.597+0000 7f84eb400640 1 -- 172.21.15.94:0/352265682 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f84e4005270 con 0x7f84f40b1100 2024-09-10T23:33:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.597+0000 7f84eb400640 1 --2- 172.21.15.94:0/352265682 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f84d00725e0 0x7f84d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.597+0000 7f84eb400640 1 -- 172.21.15.94:0/352265682 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f84e412d3d0 con 0x7f84f40b1100 2024-09-10T23:33:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.597+0000 7f84ea000640 1 --2- 172.21.15.94:0/352265682 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f84d00725e0 0x7f84d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.597+0000 7f84ea000640 1 --2- 172.21.15.94:0/352265682 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f84d00725e0 0x7f84d0074aa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f84f41670d0 tx=0x7f84dc007e40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.601+0000 7f84eb400640 1 -- 172.21.15.94:0/352265682 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f84e40c4020 con 0x7f84f40b1100 2024-09-10T23:33:16.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.777+0000 7f84f9600640 1 -- 172.21.15.94:0/352265682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f84f40b0b00 con 0x7f84f40b1100 2024-09-10T23:33:16.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.781+0000 7f84eb400640 1 -- 172.21.15.94:0/352265682 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f84e40164d0 con 0x7f84f40b1100 2024-09-10T23:33:16.783 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:33:16.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.781+0000 7f84f9600640 1 -- 172.21.15.94:0/352265682 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f84d00725e0 msgr2=0x7f84d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:16.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.781+0000 7f84f9600640 1 --2- 172.21.15.94:0/352265682 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f84d00725e0 0x7f84d0074aa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f84f41670d0 tx=0x7f84dc007e40 comp rx=0 tx=0).stop 2024-09-10T23:33:16.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.781+0000 7f84f9600640 1 -- 172.21.15.94:0/352265682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f40b1100 msgr2=0x7f84f4152a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:16.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.781+0000 7f84f9600640 1 --2- 172.21.15.94:0/352265682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f40b1100 0x7f84f4152a80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f84e40b0040 tx=0x7f84e40a5810 comp rx=0 tx=0).stop 2024-09-10T23:33:16.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.781+0000 7f84f9600640 1 -- 172.21.15.94:0/352265682 shutdown_connections 2024-09-10T23:33:16.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.781+0000 7f84f9600640 1 --2- 172.21.15.94:0/352265682 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f84d00725e0 0x7f84d0074aa0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.781+0000 7f84f9600640 1 --2- 172.21.15.94:0/352265682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84f40ce1e0 0x7f84f4158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.781+0000 7f84f9600640 1 --2- 172.21.15.94:0/352265682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84f40ce890 0x7f84f406dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.781+0000 7f84f9600640 1 --2- 172.21.15.94:0/352265682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f40b1100 0x7f84f4152a80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.781+0000 7f84f9600640 1 -- 172.21.15.94:0/352265682 >> 172.21.15.94:0/352265682 conn(0x7f84f40d1c60 msgr2=0x7f84f40cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:16.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.781+0000 7f84f9600640 1 -- 172.21.15.94:0/352265682 shutdown_connections 2024-09-10T23:33:16.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.781+0000 7f84f9600640 1 -- 172.21.15.94:0/352265682 wait complete. 2024-09-10T23:33:16.804 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:33:16.804 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:16.814 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:33:16.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.953+0000 7f634ce00640 1 -- 172.21.15.94:0/2859661949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f634815efa0 msgr2=0x7f63480cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:16.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.953+0000 7f634ce00640 1 --2- 172.21.15.94:0/2859661949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f634815efa0 0x7f63480cf9e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6338009840 tx=0x7f633801d1c0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.953+0000 7f634ce00640 1 -- 172.21.15.94:0/2859661949 shutdown_connections 2024-09-10T23:33:16.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.953+0000 7f634ce00640 1 --2- 172.21.15.94:0/2859661949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f634815efa0 0x7f63480cf9e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.953+0000 7f634ce00640 1 -- 172.21.15.94:0/2859661949 >> 172.21.15.94:0/2859661949 conn(0x7f63480d1b70 msgr2=0x7f63480d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:16.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.953+0000 7f634ce00640 1 -- 172.21.15.94:0/2859661949 shutdown_connections 2024-09-10T23:33:16.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.953+0000 7f634ce00640 1 -- 172.21.15.94:0/2859661949 wait complete. 2024-09-10T23:33:16.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.953+0000 7f634ce00640 1 Processor -- start 2024-09-10T23:33:16.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.953+0000 7f634ce00640 1 -- start start 2024-09-10T23:33:16.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f634ce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63480d2660 0x7f6348158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:16.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f634ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f634815efa0 0x7f634815dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:16.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f634ce00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f634815f380 0x7f6348144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:16.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f634ce00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6348169a50 con 0x7f63480d2660 2024-09-10T23:33:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f634ce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f63481698d0 con 0x7f634815efa0 2024-09-10T23:33:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f634ce00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6348169bd0 con 0x7f634815f380 2024-09-10T23:33:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f6347400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63480d2660 0x7f6348158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f6347400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63480d2660 0x7f6348158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38596/0 (socket says 172.21.15.94:38596) 2024-09-10T23:33:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f6347400640 1 -- 172.21.15.94:0/2166686116 learned_addr learned my addr 172.21.15.94:0/2166686116 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f6347400640 1 -- 172.21.15.94:0/2166686116 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f634815f380 msgr2=0x7f6348144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f6347400640 1 --2- 172.21.15.94:0/2166686116 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f634815f380 0x7f6348144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f6347400640 1 -- 172.21.15.94:0/2166686116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f634815efa0 msgr2=0x7f634815dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f6347400640 1 --2- 172.21.15.94:0/2166686116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f634815efa0 0x7f634815dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f6347400640 1 -- 172.21.15.94:0/2166686116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634815a400 con 0x7f63480d2660 2024-09-10T23:33:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f6347400640 1 --2- 172.21.15.94:0/2166686116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63480d2660 0x7f6348158a10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6338009790 tx=0x7f63380a56c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f633fe00640 1 -- 172.21.15.94:0/2166686116 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63380b2040 con 0x7f63480d2660 2024-09-10T23:33:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f633fe00640 1 -- 172.21.15.94:0/2166686116 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6338004470 con 0x7f63480d2660 2024-09-10T23:33:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f633fe00640 1 -- 172.21.15.94:0/2166686116 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338005280 con 0x7f63480d2660 2024-09-10T23:33:16.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f634ce00640 1 -- 172.21.15.94:0/2166686116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63480b6eb0 con 0x7f63480d2660 2024-09-10T23:33:16.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f634ce00640 1 -- 172.21.15.94:0/2166686116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63480b7470 con 0x7f63480d2660 2024-09-10T23:33:16.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f634ce00640 1 -- 172.21.15.94:0/2166686116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63480af330 con 0x7f63480d2660 2024-09-10T23:33:16.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.957+0000 7f633fe00640 1 -- 172.21.15.94:0/2166686116 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f63380b5070 con 0x7f63480d2660 2024-09-10T23:33:16.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.961+0000 7f633fe00640 1 --2- 172.21.15.94:0/2166686116 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f63240725e0 0x7f6324074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:16.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.961+0000 7f633fe00640 1 -- 172.21.15.94:0/2166686116 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f633812caa0 con 0x7f63480d2660 2024-09-10T23:33:16.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.961+0000 7f633ea00640 1 --2- 172.21.15.94:0/2166686116 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f63240725e0 0x7f6324074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:16.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.961+0000 7f633ea00640 1 --2- 172.21.15.94:0/2166686116 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f63240725e0 0x7f6324074aa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6330007fd0 tx=0x7f63300078e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:16.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:16.961+0000 7f633fe00640 1 -- 172.21.15.94:0/2166686116 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f63380bc060 con 0x7f63480d2660 2024-09-10T23:33:17.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.137+0000 7f634ce00640 1 -- 172.21.15.94:0/2166686116 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f63480b3fc0 con 0x7f63480d2660 2024-09-10T23:33:17.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.137+0000 7f633fe00640 1 -- 172.21.15.94:0/2166686116 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f6338016410 con 0x7f63480d2660 2024-09-10T23:33:17.142 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:33:17.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.141+0000 7f634ce00640 1 -- 172.21.15.94:0/2166686116 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f63240725e0 msgr2=0x7f6324074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:17.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.141+0000 7f634ce00640 1 --2- 172.21.15.94:0/2166686116 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f63240725e0 0x7f6324074aa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6330007fd0 tx=0x7f63300078e0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.141+0000 7f634ce00640 1 -- 172.21.15.94:0/2166686116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63480d2660 msgr2=0x7f6348158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:17.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.141+0000 7f634ce00640 1 --2- 172.21.15.94:0/2166686116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63480d2660 0x7f6348158a10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6338009790 tx=0x7f63380a56c0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.141+0000 7f634ce00640 1 -- 172.21.15.94:0/2166686116 shutdown_connections 2024-09-10T23:33:17.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.141+0000 7f634ce00640 1 --2- 172.21.15.94:0/2166686116 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f63240725e0 0x7f6324074aa0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.141+0000 7f634ce00640 1 --2- 172.21.15.94:0/2166686116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f634815efa0 0x7f634815dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.141+0000 7f634ce00640 1 --2- 172.21.15.94:0/2166686116 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f634815f380 0x7f6348144750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.141+0000 7f634ce00640 1 --2- 172.21.15.94:0/2166686116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63480d2660 0x7f6348158a10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.141+0000 7f634ce00640 1 -- 172.21.15.94:0/2166686116 >> 172.21.15.94:0/2166686116 conn(0x7f63480d1b70 msgr2=0x7f6348071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:17.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.141+0000 7f634ce00640 1 -- 172.21.15.94:0/2166686116 shutdown_connections 2024-09-10T23:33:17.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.141+0000 7f634ce00640 1 -- 172.21.15.94:0/2166686116 wait complete. 2024-09-10T23:33:17.165 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi181.front.sepia.ceph.com... 2024-09-10T23:33:17.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:17.165 DEBUG:teuthology.orchestra.run.smithi181:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-09-10T23:33:17.186 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:33:17.186 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:33:17.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:17.186 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T23:33:17.239 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T23:33:17.239 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:33:17.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.377+0000 7f111c400640 1 -- 172.21.15.94:0/1562315111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10fc004d50 msgr2=0x7f10fc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:17.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.377+0000 7f111c400640 1 --2- 172.21.15.94:0/1562315111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10fc004d50 0x7f10fc0051d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f110c001d60 tx=0x7f110c01cb20 comp rx=0 tx=0).stop 2024-09-10T23:33:17.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.377+0000 7f111a400640 1 -- 172.21.15.94:0/1562315111 reap_dead start 2024-09-10T23:33:17.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.377+0000 7f111c400640 1 -- 172.21.15.94:0/1562315111 shutdown_connections 2024-09-10T23:33:17.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.377+0000 7f111c400640 1 -- 172.21.15.94:0/1562315111 >> 172.21.15.94:0/1562315111 conn(0x7f11140cc900 msgr2=0x7f11140ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:17.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.377+0000 7f111c400640 1 -- 172.21.15.94:0/1562315111 shutdown_connections 2024-09-10T23:33:17.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.377+0000 7f111c400640 1 -- 172.21.15.94:0/1562315111 wait complete. 2024-09-10T23:33:17.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f111c400640 1 Processor -- start 2024-09-10T23:33:17.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f111c400640 1 -- start start 2024-09-10T23:33:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f111c400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10fc004d50 0x7f111414a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f111c400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f111414af20 0x7f11140c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f111c400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11140c7500 0x7f11141f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f111c400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11140af1f0 con 0x7f10fc004d50 2024-09-10T23:33:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f111c400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f11140af070 con 0x7f11140c7500 2024-09-10T23:33:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f111c400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f111415de60 con 0x7f111414af20 2024-09-10T23:33:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f1119a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10fc004d50 0x7f111414a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f1119a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10fc004d50 0x7f111414a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38620/0 (socket says 172.21.15.94:38620) 2024-09-10T23:33:17.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f1119a00640 1 -- 172.21.15.94:0/1085908517 learned_addr learned my addr 172.21.15.94:0/1085908517 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f111a400640 1 --2- 172.21.15.94:0/1085908517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11140c7500 0x7f11141f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f1119000640 1 --2- 172.21.15.94:0/1085908517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f111414af20 0x7f11140c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f1119a00640 1 -- 172.21.15.94:0/1085908517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f111414af20 msgr2=0x7f11140c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f1119a00640 1 --2- 172.21.15.94:0/1085908517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f111414af20 0x7f11140c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f1119a00640 1 -- 172.21.15.94:0/1085908517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11140c7500 msgr2=0x7f11141f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f1119a00640 1 --2- 172.21.15.94:0/1085908517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11140c7500 0x7f11141f5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f1119a00640 1 -- 172.21.15.94:0/1085908517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11140c9ae0 con 0x7f10fc004d50 2024-09-10T23:33:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f111a400640 1 --2- 172.21.15.94:0/1085908517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11140c7500 0x7f11141f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f1119000640 1 --2- 172.21.15.94:0/1085908517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f111414af20 0x7f11140c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f1119a00640 1 --2- 172.21.15.94:0/1085908517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10fc004d50 0x7f111414a9e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1108011ae0 tx=0x7f1108009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f1111600640 1 -- 172.21.15.94:0/1085908517 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110801e070 con 0x7f10fc004d50 2024-09-10T23:33:17.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f1111600640 1 -- 172.21.15.94:0/1085908517 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f110800fe40 con 0x7f10fc004d50 2024-09-10T23:33:17.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f1111600640 1 -- 172.21.15.94:0/1085908517 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11080042b0 con 0x7f10fc004d50 2024-09-10T23:33:17.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f111c400640 1 -- 172.21.15.94:0/1085908517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11140c9dc0 con 0x7f10fc004d50 2024-09-10T23:33:17.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.381+0000 7f111c400640 1 -- 172.21.15.94:0/1085908517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11140c4c50 con 0x7f10fc004d50 2024-09-10T23:33:17.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.385+0000 7f111c400640 1 -- 172.21.15.94:0/1085908517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111415e3c0 con 0x7f10fc004d50 2024-09-10T23:33:17.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.385+0000 7f1111600640 1 -- 172.21.15.94:0/1085908517 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1108003600 con 0x7f10fc004d50 2024-09-10T23:33:17.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.385+0000 7f1111600640 1 --2- 172.21.15.94:0/1085908517 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10f40725e0 0x7f10f4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:17.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.385+0000 7f1111600640 1 -- 172.21.15.94:0/1085908517 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1108093ff0 con 0x7f10fc004d50 2024-09-10T23:33:17.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.385+0000 7f1119000640 1 --2- 172.21.15.94:0/1085908517 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10f40725e0 0x7f10f4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:17.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.385+0000 7f1119000640 1 --2- 172.21.15.94:0/1085908517 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10f40725e0 0x7f10f4074aa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1100000c10 tx=0x7f11000067b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:17.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.389+0000 7f1111600640 1 -- 172.21.15.94:0/1085908517 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f110805f190 con 0x7f10fc004d50 2024-09-10T23:33:17.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.588+0000 7f111c400640 1 -- 172.21.15.94:0/1085908517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11140cedf0 con 0x7f10fc004d50 2024-09-10T23:33:17.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.588+0000 7f1111600640 1 -- 172.21.15.94:0/1085908517 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5233 (secure 0 0 0) 0x7f1108063bc0 con 0x7f10fc004d50 2024-09-10T23:33:17.592 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:17.592 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"btime":"2024-09-10T23:32:45:573319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/2278166692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2278166692},{"type":"v1","addr":"172.21.15.94:6834","nonce":2278166692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392173872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3392173872},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392173872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4380,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1894503548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1894503548},{"type":"v1","addr":"172.21.15.181:6837","nonce":1894503548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:32:41.948908+0000","modified":"2024-09-10T23:32:44.956870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194174},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194174":{"gid":194174,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1418253171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1418253171},{"type":"v1","addr":"172.21.15.181:6836","nonce":1418253171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194174,"qdb_cluster":[194174]},"id":1}]} 2024-09-10T23:33:17.592 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-09-10T23:33:17.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.588+0000 7f111c400640 1 -- 172.21.15.94:0/1085908517 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10f40725e0 msgr2=0x7f10f4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:17.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.588+0000 7f111c400640 1 --2- 172.21.15.94:0/1085908517 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10f40725e0 0x7f10f4074aa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1100000c10 tx=0x7f11000067b0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.588+0000 7f111c400640 1 -- 172.21.15.94:0/1085908517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10fc004d50 msgr2=0x7f111414a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:17.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.588+0000 7f111c400640 1 --2- 172.21.15.94:0/1085908517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10fc004d50 0x7f111414a9e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1108011ae0 tx=0x7f1108009ea0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.588+0000 7f111c400640 1 -- 172.21.15.94:0/1085908517 shutdown_connections 2024-09-10T23:33:17.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.588+0000 7f111c400640 1 --2- 172.21.15.94:0/1085908517 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10f40725e0 0x7f10f4074aa0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.588+0000 7f111c400640 1 --2- 172.21.15.94:0/1085908517 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11140c7500 0x7f11141f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.588+0000 7f111c400640 1 --2- 172.21.15.94:0/1085908517 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f111414af20 0x7f11140c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.588+0000 7f111c400640 1 --2- 172.21.15.94:0/1085908517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10fc004d50 0x7f111414a9e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.588+0000 7f111c400640 1 -- 172.21.15.94:0/1085908517 >> 172.21.15.94:0/1085908517 conn(0x7f11140cc900 msgr2=0x7f1114159b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:17.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.588+0000 7f111c400640 1 -- 172.21.15.94:0/1085908517 shutdown_connections 2024-09-10T23:33:17.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.588+0000 7f111c400640 1 -- 172.21.15.94:0/1085908517 wait complete. 2024-09-10T23:33:17.614 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:33:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.752+0000 7f5cb5400640 1 -- 172.21.15.94:0/1367998584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c94003a40 msgr2=0x7f5c94003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.752+0000 7f5cb5400640 1 --2- 172.21.15.94:0/1367998584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c94003a40 0x7f5c94003e60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00016f0 tx=0x7f5ca001c7b0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.752+0000 7f5caf200640 1 -- 172.21.15.94:0/1367998584 reap_dead start 2024-09-10T23:33:17.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.752+0000 7f5cb5400640 1 -- 172.21.15.94:0/1367998584 shutdown_connections 2024-09-10T23:33:17.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.752+0000 7f5cb5400640 1 -- 172.21.15.94:0/1367998584 >> 172.21.15.94:0/1367998584 conn(0x7f5cb006cdf0 msgr2=0x7f5cb006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:17.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.752+0000 7f5cb5400640 1 -- 172.21.15.94:0/1367998584 shutdown_connections 2024-09-10T23:33:17.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.752+0000 7f5cb5400640 1 -- 172.21.15.94:0/1367998584 wait complete. 2024-09-10T23:33:17.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5cb5400640 1 Processor -- start 2024-09-10T23:33:17.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5cb5400640 1 -- start start 2024-09-10T23:33:17.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5cb5400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c94003a40 0x7f5cb01fbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:17.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5cb5400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb01f73a0 0x7f5cb0142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:17.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5cb5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb01431a0 0x7f5cb015c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:17.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5cb5400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb0157f00 con 0x7f5cb01431a0 2024-09-10T23:33:17.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5cb5400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb01581a0 con 0x7f5c94003a40 2024-09-10T23:33:17.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5cb5400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb015b260 con 0x7f5cb01f73a0 2024-09-10T23:33:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5caf200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb01431a0 0x7f5cb015c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5caf200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb01431a0 0x7f5cb015c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38630/0 (socket says 172.21.15.94:38630) 2024-09-10T23:33:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5caf200640 1 -- 172.21.15.94:0/530846619 learned_addr learned my addr 172.21.15.94:0/530846619 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5cae800640 1 --2- 172.21.15.94:0/530846619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c94003a40 0x7f5cb01fbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5cade00640 1 --2- 172.21.15.94:0/530846619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb01f73a0 0x7f5cb0142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5caf200640 1 -- 172.21.15.94:0/530846619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb01f73a0 msgr2=0x7f5cb0142c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5caf200640 1 --2- 172.21.15.94:0/530846619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb01f73a0 0x7f5cb0142c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5caf200640 1 -- 172.21.15.94:0/530846619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c94003a40 msgr2=0x7f5cb01fbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5caf200640 1 --2- 172.21.15.94:0/530846619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c94003a40 0x7f5cb01fbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5caf200640 1 -- 172.21.15.94:0/530846619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb0143700 con 0x7f5cb01431a0 2024-09-10T23:33:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5caf200640 1 --2- 172.21.15.94:0/530846619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb01431a0 0x7f5cb015c0f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00749f0 tx=0x7f5ca800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:17.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5ca6a00640 1 -- 172.21.15.94:0/530846619 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8015070 con 0x7f5cb01431a0 2024-09-10T23:33:17.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5ca6a00640 1 -- 172.21.15.94:0/530846619 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ca800cd10 con 0x7f5cb01431a0 2024-09-10T23:33:17.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5ca6a00640 1 -- 172.21.15.94:0/530846619 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80116c0 con 0x7f5cb01431a0 2024-09-10T23:33:17.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5cb5400640 1 -- 172.21.15.94:0/530846619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb01439e0 con 0x7f5cb01431a0 2024-09-10T23:33:17.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5cb5400640 1 -- 172.21.15.94:0/530846619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb0146ee0 con 0x7f5cb01431a0 2024-09-10T23:33:17.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.756+0000 7f5cb5400640 1 -- 172.21.15.94:0/530846619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb0143bb0 con 0x7f5cb01431a0 2024-09-10T23:33:17.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.760+0000 7f5ca6a00640 1 -- 172.21.15.94:0/530846619 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5ca8011860 con 0x7f5cb01431a0 2024-09-10T23:33:17.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.760+0000 7f5ca6a00640 1 --2- 172.21.15.94:0/530846619 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c90072610 0x7f5c90074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:17.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.760+0000 7f5ca6a00640 1 -- 172.21.15.94:0/530846619 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5ca8093930 con 0x7f5cb01431a0 2024-09-10T23:33:17.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.760+0000 7f5cae800640 1 --2- 172.21.15.94:0/530846619 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c90072610 0x7f5c90074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:17.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.760+0000 7f5cae800640 1 --2- 172.21.15.94:0/530846619 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c90072610 0x7f5c90074ad0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00016c0 tx=0x7f5ca00b1040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:17.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.764+0000 7f5ca6a00640 1 -- 172.21.15.94:0/530846619 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5cb0143bb0 con 0x7f5cb01431a0 2024-09-10T23:33:17.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.960+0000 7f5cb5400640 1 -- 172.21.15.94:0/530846619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cb01fb150 con 0x7f5cb01431a0 2024-09-10T23:33:17.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.960+0000 7f5ca6a00640 1 -- 172.21.15.94:0/530846619 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5233 (secure 0 0 0) 0x7f5ca805e9b0 con 0x7f5cb01431a0 2024-09-10T23:33:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:17.965 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"btime":"2024-09-10T23:32:45:573319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/2278166692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2278166692},{"type":"v1","addr":"172.21.15.94:6834","nonce":2278166692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392173872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3392173872},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392173872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4380,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1894503548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1894503548},{"type":"v1","addr":"172.21.15.181:6837","nonce":1894503548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:32:41.948908+0000","modified":"2024-09-10T23:32:44.956870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194174},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194174":{"gid":194174,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1418253171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1418253171},{"type":"v1","addr":"172.21.15.181:6836","nonce":1418253171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194174,"qdb_cluster":[194174]},"id":1}]} 2024-09-10T23:33:17.966 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-09-10T23:33:17.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.964+0000 7f5cb5400640 1 -- 172.21.15.94:0/530846619 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c90072610 msgr2=0x7f5c90074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:17.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.964+0000 7f5cb5400640 1 --2- 172.21.15.94:0/530846619 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c90072610 0x7f5c90074ad0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00016c0 tx=0x7f5ca00b1040 comp rx=0 tx=0).stop 2024-09-10T23:33:17.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.964+0000 7f5cb5400640 1 -- 172.21.15.94:0/530846619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb01431a0 msgr2=0x7f5cb015c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:17.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.964+0000 7f5cb5400640 1 --2- 172.21.15.94:0/530846619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb01431a0 0x7f5cb015c0f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00749f0 tx=0x7f5ca800ecf0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.964+0000 7f5cb5400640 1 -- 172.21.15.94:0/530846619 shutdown_connections 2024-09-10T23:33:17.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.964+0000 7f5cb5400640 1 --2- 172.21.15.94:0/530846619 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5c90072610 0x7f5c90074ad0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.964+0000 7f5cb5400640 1 --2- 172.21.15.94:0/530846619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb01431a0 0x7f5cb015c0f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.964+0000 7f5cb5400640 1 --2- 172.21.15.94:0/530846619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb01f73a0 0x7f5cb0142c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.964+0000 7f5cb5400640 1 --2- 172.21.15.94:0/530846619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c94003a40 0x7f5cb01fbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:17.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.964+0000 7f5cb5400640 1 -- 172.21.15.94:0/530846619 >> 172.21.15.94:0/530846619 conn(0x7f5cb006cdf0 msgr2=0x7f5cb0152780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:17.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.964+0000 7f5cb5400640 1 -- 172.21.15.94:0/530846619 shutdown_connections 2024-09-10T23:33:17.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:17.964+0000 7f5cb5400640 1 -- 172.21.15.94:0/530846619 wait complete. 2024-09-10T23:33:17.987 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:33:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.124+0000 7f9e3f600640 1 -- 172.21.15.94:0/3539575612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e380b0ec0 msgr2=0x7f9e380cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.124+0000 7f9e3f600640 1 --2- 172.21.15.94:0/3539575612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e380b0ec0 0x7f9e380cebd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9e380ce600 tx=0x7f9e2401c890 comp rx=0 tx=0).stop 2024-09-10T23:33:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.124+0000 7f9e3f600640 1 -- 172.21.15.94:0/3539575612 shutdown_connections 2024-09-10T23:33:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.124+0000 7f9e3f600640 1 --2- 172.21.15.94:0/3539575612 >> v2:172.21.15.181:3300/0 conn(0x7f9e380cc4e0 0x7f9e380cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.124+0000 7f9e3f600640 1 --2- 172.21.15.94:0/3539575612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e380b0ec0 0x7f9e380cebd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.124+0000 7f9e3f600640 1 --2- 172.21.15.94:0/3539575612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e3815f370 0x7f9e380ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.124+0000 7f9e3f600640 1 -- 172.21.15.94:0/3539575612 >> 172.21.15.94:0/3539575612 conn(0x7f9e380d1c80 msgr2=0x7f9e380d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:18.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.124+0000 7f9e3f600640 1 -- 172.21.15.94:0/3539575612 shutdown_connections 2024-09-10T23:33:18.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.124+0000 7f9e3f600640 1 -- 172.21.15.94:0/3539575612 wait complete. 2024-09-10T23:33:18.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.124+0000 7f9e3f600640 1 Processor -- start 2024-09-10T23:33:18.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.124+0000 7f9e3f600640 1 -- start start 2024-09-10T23:33:18.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e3f600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e380b0ec0 0x7f9e3815a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:18.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e3f600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e380cc4e0 0x7f9e381f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:18.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e3f600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e3815f370 0x7f9e381faa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:18.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e3f600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9e38070b80 con 0x7f9e380cc4e0 2024-09-10T23:33:18.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e3f600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9e38070a00 con 0x7f9e380b0ec0 2024-09-10T23:33:18.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e3f600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9e38070d00 con 0x7f9e3815f370 2024-09-10T23:33:18.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e37e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e380cc4e0 0x7f9e381f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:18.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e37e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e380cc4e0 0x7f9e381f6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38648/0 (socket says 172.21.15.94:38648) 2024-09-10T23:33:18.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e37e00640 1 -- 172.21.15.94:0/3113217064 learned_addr learned my addr 172.21.15.94:0/3113217064 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:18.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e3cc00640 1 --2- 172.21.15.94:0/3113217064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e380b0ec0 0x7f9e3815a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:18.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e3d600640 1 --2- 172.21.15.94:0/3113217064 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e3815f370 0x7f9e381faa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:18.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e37e00640 1 -- 172.21.15.94:0/3113217064 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e3815f370 msgr2=0x7f9e381faa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:18.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e37e00640 1 --2- 172.21.15.94:0/3113217064 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e3815f370 0x7f9e381faa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e37e00640 1 -- 172.21.15.94:0/3113217064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e380b0ec0 msgr2=0x7f9e3815a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:18.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e37e00640 1 --2- 172.21.15.94:0/3113217064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e380b0ec0 0x7f9e3815a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e37e00640 1 -- 172.21.15.94:0/3113217064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e38163ce0 con 0x7f9e380cc4e0 2024-09-10T23:33:18.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e3cc00640 1 --2- 172.21.15.94:0/3113217064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e380b0ec0 0x7f9e3815a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:18.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e3d600640 1 --2- 172.21.15.94:0/3113217064 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e3815f370 0x7f9e381faa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:18.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e37e00640 1 --2- 172.21.15.94:0/3113217064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e380cc4e0 0x7f9e381f6c00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9e240b0040 tx=0x7f9e24004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:18.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e34c00640 1 -- 172.21.15.94:0/3113217064 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e240b2040 con 0x7f9e380cc4e0 2024-09-10T23:33:18.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e34c00640 1 -- 172.21.15.94:0/3113217064 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e24005030 con 0x7f9e380cc4e0 2024-09-10T23:33:18.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e34c00640 1 -- 172.21.15.94:0/3113217064 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e24005480 con 0x7f9e380cc4e0 2024-09-10T23:33:18.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e3f600640 1 -- 172.21.15.94:0/3113217064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e3815ae70 con 0x7f9e380cc4e0 2024-09-10T23:33:18.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e3f600640 1 -- 172.21.15.94:0/3113217064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e3815b430 con 0x7f9e380cc4e0 2024-09-10T23:33:18.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e3f600640 1 -- 172.21.15.94:0/3113217064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e380cc990 con 0x7f9e380cc4e0 2024-09-10T23:33:18.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.128+0000 7f9e34c00640 1 -- 172.21.15.94:0/3113217064 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9e240bf060 con 0x7f9e380cc4e0 2024-09-10T23:33:18.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.132+0000 7f9e34c00640 1 --2- 172.21.15.94:0/3113217064 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9e180725e0 0x7f9e18074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:18.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.132+0000 7f9e34c00640 1 -- 172.21.15.94:0/3113217064 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9e2412c460 con 0x7f9e380cc4e0 2024-09-10T23:33:18.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.132+0000 7f9e3cc00640 1 --2- 172.21.15.94:0/3113217064 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9e180725e0 0x7f9e18074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:18.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.132+0000 7f9e3cc00640 1 --2- 172.21.15.94:0/3113217064 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9e180725e0 0x7f9e18074aa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9e381fa960 tx=0x7f9e2c005d50 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:18.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.136+0000 7f9e34c00640 1 -- 172.21.15.94:0/3113217064 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9e240164b0 con 0x7f9e380cc4e0 2024-09-10T23:33:18.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.300+0000 7f9e3f600640 1 -- 172.21.15.94:0/3113217064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e3815ab60 con 0x7f9e380cc4e0 2024-09-10T23:33:18.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.300+0000 7f9e34c00640 1 -- 172.21.15.94:0/3113217064 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16657 (secure 0 0 0) 0x7f9e240f7aa0 con 0x7f9e380cc4e0 2024-09-10T23:33:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:18.305 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":22,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:32:43.556132+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:32:39.855046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T23:32:40.904161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:33:18.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.304+0000 7f9e3f600640 1 -- 172.21.15.94:0/3113217064 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9e180725e0 msgr2=0x7f9e18074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:18.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.304+0000 7f9e3f600640 1 --2- 172.21.15.94:0/3113217064 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9e180725e0 0x7f9e18074aa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9e381fa960 tx=0x7f9e2c005d50 comp rx=0 tx=0).stop 2024-09-10T23:33:18.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.304+0000 7f9e3f600640 1 -- 172.21.15.94:0/3113217064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e380cc4e0 msgr2=0x7f9e381f6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:18.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.304+0000 7f9e3f600640 1 --2- 172.21.15.94:0/3113217064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e380cc4e0 0x7f9e381f6c00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9e240b0040 tx=0x7f9e24004040 comp rx=0 tx=0).stop 2024-09-10T23:33:18.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.304+0000 7f9e3f600640 1 -- 172.21.15.94:0/3113217064 shutdown_connections 2024-09-10T23:33:18.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.304+0000 7f9e3f600640 1 --2- 172.21.15.94:0/3113217064 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9e180725e0 0x7f9e18074aa0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.304+0000 7f9e3f600640 1 --2- 172.21.15.94:0/3113217064 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e3815f370 0x7f9e381faa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.304+0000 7f9e3f600640 1 --2- 172.21.15.94:0/3113217064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e380cc4e0 0x7f9e381f6c00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.304+0000 7f9e3f600640 1 --2- 172.21.15.94:0/3113217064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e380b0ec0 0x7f9e3815a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.304+0000 7f9e3f600640 1 -- 172.21.15.94:0/3113217064 >> 172.21.15.94:0/3113217064 conn(0x7f9e380d1c80 msgr2=0x7f9e380b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:18.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.304+0000 7f9e3f600640 1 -- 172.21.15.94:0/3113217064 shutdown_connections 2024-09-10T23:33:18.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.304+0000 7f9e3f600640 1 -- 172.21.15.94:0/3113217064 wait complete. 2024-09-10T23:33:18.376 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T23:33:18.376 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:33:18.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.512+0000 7f7341c00640 1 -- 172.21.15.94:0/443069268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f733c0d19b0 msgr2=0x7f733c0d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:18.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.512+0000 7f7341c00640 1 --2- 172.21.15.94:0/443069268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f733c0d19b0 0x7f733c0d0590 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7324009840 tx=0x7f732401f010 comp rx=0 tx=0).stop 2024-09-10T23:33:18.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.512+0000 7f7341c00640 1 -- 172.21.15.94:0/443069268 shutdown_connections 2024-09-10T23:33:18.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.512+0000 7f7341c00640 1 --2- 172.21.15.94:0/443069268 >> v2:172.21.15.181:3300/0 conn(0x7f733c0d4900 0x7f733c0d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.512+0000 7f7341c00640 1 --2- 172.21.15.94:0/443069268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f733c0d19b0 0x7f733c0d0590 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.512+0000 7f7341c00640 1 -- 172.21.15.94:0/443069268 >> 172.21.15.94:0/443069268 conn(0x7f733c0d3de0 msgr2=0x7f733c0d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:18.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.512+0000 7f7341c00640 1 -- 172.21.15.94:0/443069268 shutdown_connections 2024-09-10T23:33:18.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.512+0000 7f7341c00640 1 -- 172.21.15.94:0/443069268 wait complete. 2024-09-10T23:33:18.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f7341c00640 1 Processor -- start 2024-09-10T23:33:18.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f7341c00640 1 -- start start 2024-09-10T23:33:18.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f7341c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f733c0d19b0 0x7f733c0caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:18.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f7341c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f733c0d4900 0x7f733c1f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:18.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f7341c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c15dec0 0x7f733c15fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:18.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f7341c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f733c0b6db0 con 0x7f733c15dec0 2024-09-10T23:33:18.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f7341c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f733c0b6c30 con 0x7f733c0d4900 2024-09-10T23:33:18.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f7341c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f733c0b6f30 con 0x7f733c0d19b0 2024-09-10T23:33:18.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f733b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c15dec0 0x7f733c15fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:18.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f733b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c15dec0 0x7f733c15fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38652/0 (socket says 172.21.15.94:38652) 2024-09-10T23:33:18.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f733b400640 1 -- 172.21.15.94:0/1217450702 learned_addr learned my addr 172.21.15.94:0/1217450702 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:18.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f7332000640 1 --2- 172.21.15.94:0/1217450702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f733c0d4900 0x7f733c1f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:18.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f733aa00640 1 --2- 172.21.15.94:0/1217450702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f733c0d19b0 0x7f733c0caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:18.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f733b400640 1 -- 172.21.15.94:0/1217450702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f733c0d19b0 msgr2=0x7f733c0caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:18.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f733b400640 1 --2- 172.21.15.94:0/1217450702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f733c0d19b0 0x7f733c0caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f733b400640 1 -- 172.21.15.94:0/1217450702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f733c0d4900 msgr2=0x7f733c1f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:18.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f733b400640 1 --2- 172.21.15.94:0/1217450702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f733c0d4900 0x7f733c1f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f733b400640 1 -- 172.21.15.94:0/1217450702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733c1e85c0 con 0x7f733c15dec0 2024-09-10T23:33:18.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f733b400640 1 --2- 172.21.15.94:0/1217450702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c15dec0 0x7f733c15fd50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f733400e970 tx=0x7f733400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:18.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f7333400640 1 -- 172.21.15.94:0/1217450702 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733400ccc0 con 0x7f733c15dec0 2024-09-10T23:33:18.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f7333400640 1 -- 172.21.15.94:0/1217450702 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7334005020 con 0x7f733c15dec0 2024-09-10T23:33:18.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f7333400640 1 -- 172.21.15.94:0/1217450702 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733400ce60 con 0x7f733c15dec0 2024-09-10T23:33:18.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f7341c00640 1 -- 172.21.15.94:0/1217450702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733c1e88a0 con 0x7f733c15dec0 2024-09-10T23:33:18.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.516+0000 7f7341c00640 1 -- 172.21.15.94:0/1217450702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f733c1e8e60 con 0x7f733c15dec0 2024-09-10T23:33:18.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.520+0000 7f7341c00640 1 -- 172.21.15.94:0/1217450702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f733c0b7b80 con 0x7f733c15dec0 2024-09-10T23:33:18.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.520+0000 7f7333400640 1 -- 172.21.15.94:0/1217450702 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7334005450 con 0x7f733c15dec0 2024-09-10T23:33:18.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.520+0000 7f7333400640 1 --2- 172.21.15.94:0/1217450702 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f73180725e0 0x7f7318074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:18.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.520+0000 7f7333400640 1 -- 172.21.15.94:0/1217450702 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7334093df0 con 0x7f733c15dec0 2024-09-10T23:33:18.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.520+0000 7f733aa00640 1 --2- 172.21.15.94:0/1217450702 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f73180725e0 0x7f7318074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:18.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.520+0000 7f733aa00640 1 --2- 172.21.15.94:0/1217450702 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f73180725e0 0x7f7318074aa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f732c0015b0 tx=0x7f732c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:18.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.524+0000 7f7333400640 1 -- 172.21.15.94:0/1217450702 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f733405f430 con 0x7f733c15dec0 2024-09-10T23:33:18.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.724+0000 7f7341c00640 1 -- 172.21.15.94:0/1217450702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f733c0d1390 con 0x7f733c15dec0 2024-09-10T23:33:18.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.724+0000 7f7333400640 1 -- 172.21.15.94:0/1217450702 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5233 (secure 0 0 0) 0x7f733405ed30 con 0x7f733c15dec0 2024-09-10T23:33:18.728 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:18.728 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"btime":"2024-09-10T23:32:45:573319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/2278166692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2278166692},{"type":"v1","addr":"172.21.15.94:6834","nonce":2278166692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392173872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3392173872},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392173872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4380,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1894503548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1894503548},{"type":"v1","addr":"172.21.15.181:6837","nonce":1894503548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:32:41.948908+0000","modified":"2024-09-10T23:32:44.956870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194174},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194174":{"gid":194174,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1418253171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1418253171},{"type":"v1","addr":"172.21.15.181:6836","nonce":1418253171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194174,"qdb_cluster":[194174]},"id":1}]} 2024-09-10T23:33:18.728 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-09-10T23:33:18.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.724+0000 7f7341c00640 1 -- 172.21.15.94:0/1217450702 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f73180725e0 msgr2=0x7f7318074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:18.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.724+0000 7f7341c00640 1 --2- 172.21.15.94:0/1217450702 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f73180725e0 0x7f7318074aa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f732c0015b0 tx=0x7f732c009040 comp rx=0 tx=0).stop 2024-09-10T23:33:18.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.724+0000 7f7341c00640 1 -- 172.21.15.94:0/1217450702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c15dec0 msgr2=0x7f733c15fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:18.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.724+0000 7f7341c00640 1 --2- 172.21.15.94:0/1217450702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c15dec0 0x7f733c15fd50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f733400e970 tx=0x7f733400ee30 comp rx=0 tx=0).stop 2024-09-10T23:33:18.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.724+0000 7f7341c00640 1 -- 172.21.15.94:0/1217450702 shutdown_connections 2024-09-10T23:33:18.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.724+0000 7f7341c00640 1 --2- 172.21.15.94:0/1217450702 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f73180725e0 0x7f7318074aa0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.724+0000 7f7341c00640 1 --2- 172.21.15.94:0/1217450702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f733c0d4900 0x7f733c1f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.724+0000 7f7341c00640 1 --2- 172.21.15.94:0/1217450702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c15dec0 0x7f733c15fd50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.724+0000 7f7341c00640 1 --2- 172.21.15.94:0/1217450702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f733c0d19b0 0x7f733c0caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.724+0000 7f7341c00640 1 -- 172.21.15.94:0/1217450702 >> 172.21.15.94:0/1217450702 conn(0x7f733c0d3de0 msgr2=0x7f733c0b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:18.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.728+0000 7f7341c00640 1 -- 172.21.15.94:0/1217450702 shutdown_connections 2024-09-10T23:33:18.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.728+0000 7f7341c00640 1 -- 172.21.15.94:0/1217450702 wait complete. 2024-09-10T23:33:18.750 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:33:18.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.884+0000 7fe809400640 1 -- 172.21.15.94:0/3773925631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8040d2290 msgr2=0x7fe804166b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.884+0000 7fe809400640 1 --2- 172.21.15.94:0/3773925631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8040d2290 0x7fe804166b70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe80415c9a0 tx=0x7fe7fc01ca90 comp rx=0 tx=0).stop 2024-09-10T23:33:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe809400640 1 -- 172.21.15.94:0/3773925631 shutdown_connections 2024-09-10T23:33:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe809400640 1 --2- 172.21.15.94:0/3773925631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8040d2290 0x7fe804166b70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe809400640 1 -- 172.21.15.94:0/3773925631 >> 172.21.15.94:0/3773925631 conn(0x7fe80409aa70 msgr2=0x7fe80409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe809400640 1 -- 172.21.15.94:0/3773925631 shutdown_connections 2024-09-10T23:33:18.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe809400640 1 -- 172.21.15.94:0/3773925631 wait complete. 2024-09-10T23:33:18.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe809400640 1 Processor -- start 2024-09-10T23:33:18.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe809400640 1 -- start start 2024-09-10T23:33:18.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe809400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8040ae980 0x7fe8040cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:18.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe809400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8040d2290 0x7fe804155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:18.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe809400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80415bd30 0x7fe804070d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:18.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe809400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe80416a360 con 0x7fe8040d2290 2024-09-10T23:33:18.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe809400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe80416a1e0 con 0x7fe8040ae980 2024-09-10T23:33:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe809400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe80416a4e0 con 0x7fe80415bd30 2024-09-10T23:33:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe7f9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8040d2290 0x7fe804155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe7f9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8040d2290 0x7fe804155890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38654/0 (socket says 172.21.15.94:38654) 2024-09-10T23:33:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe7f9e00640 1 -- 172.21.15.94:0/2806775096 learned_addr learned my addr 172.21.15.94:0/2806775096 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe802800640 1 --2- 172.21.15.94:0/2806775096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8040ae980 0x7fe8040cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe803200640 1 --2- 172.21.15.94:0/2806775096 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80415bd30 0x7fe804070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe7f9e00640 1 -- 172.21.15.94:0/2806775096 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80415bd30 msgr2=0x7fe804070d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe7f9e00640 1 --2- 172.21.15.94:0/2806775096 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80415bd30 0x7fe804070d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe7f9e00640 1 -- 172.21.15.94:0/2806775096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8040ae980 msgr2=0x7fe8040cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe7f9e00640 1 --2- 172.21.15.94:0/2806775096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8040ae980 0x7fe8040cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe7f9e00640 1 -- 172.21.15.94:0/2806775096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8040c97d0 con 0x7fe8040d2290 2024-09-10T23:33:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe803200640 1 --2- 172.21.15.94:0/2806775096 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80415bd30 0x7fe804070d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:18.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe802800640 1 --2- 172.21.15.94:0/2806775096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8040ae980 0x7fe8040cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:18.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe7f9e00640 1 --2- 172.21.15.94:0/2806775096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8040d2290 0x7fe804155890 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c500 tx=0x7fe7ec00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:18.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.888+0000 7fe7fb400640 1 -- 172.21.15.94:0/2806775096 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec0052d0 con 0x7fe8040d2290 2024-09-10T23:33:18.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.892+0000 7fe7fb400640 1 -- 172.21.15.94:0/2806775096 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7ec005490 con 0x7fe8040d2290 2024-09-10T23:33:18.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.892+0000 7fe7fb400640 1 -- 172.21.15.94:0/2806775096 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec012850 con 0x7fe8040d2290 2024-09-10T23:33:18.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.892+0000 7fe809400640 1 -- 172.21.15.94:0/2806775096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8040c9a70 con 0x7fe8040d2290 2024-09-10T23:33:18.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.892+0000 7fe809400640 1 -- 172.21.15.94:0/2806775096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8040c92b0 con 0x7fe8040d2290 2024-09-10T23:33:18.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.892+0000 7fe809400640 1 -- 172.21.15.94:0/2806775096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8040d1b80 con 0x7fe8040d2290 2024-09-10T23:33:18.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.892+0000 7fe7fb400640 1 -- 172.21.15.94:0/2806775096 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe7ec0129f0 con 0x7fe8040d2290 2024-09-10T23:33:18.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.892+0000 7fe7fb400640 1 --2- 172.21.15.94:0/2806775096 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe7e00725e0 0x7fe7e0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:18.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.892+0000 7fe7fb400640 1 -- 172.21.15.94:0/2806775096 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe7ec092df0 con 0x7fe8040d2290 2024-09-10T23:33:18.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.892+0000 7fe802800640 1 --2- 172.21.15.94:0/2806775096 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe7e00725e0 0x7fe7e0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:18.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.896+0000 7fe802800640 1 --2- 172.21.15.94:0/2806775096 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe7e00725e0 0x7fe7e0074aa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4001230 tx=0x7fe7f4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:18.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:18.896+0000 7fe7fb400640 1 -- 172.21.15.94:0/2806775096 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe7ec05e560 con 0x7fe8040d2290 2024-09-10T23:33:19.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.092+0000 7fe809400640 1 -- 172.21.15.94:0/2806775096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8040d28a0 con 0x7fe8040d2290 2024-09-10T23:33:19.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.096+0000 7fe7fb400640 1 -- 172.21.15.94:0/2806775096 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5233 (secure 0 0 0) 0x7fe7ec05de60 con 0x7fe8040d2290 2024-09-10T23:33:19.098 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:19.099 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"btime":"2024-09-10T23:32:45:573319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/2278166692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2278166692},{"type":"v1","addr":"172.21.15.94:6834","nonce":2278166692}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392173872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3392173872},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392173872}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4380,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1894503548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1894503548},{"type":"v1","addr":"172.21.15.181:6837","nonce":1894503548}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:32:41.948908+0000","modified":"2024-09-10T23:32:44.956870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194174},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194174":{"gid":194174,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1418253171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1418253171},{"type":"v1","addr":"172.21.15.181:6836","nonce":1418253171}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194174,"qdb_cluster":[194174]},"id":1}]} 2024-09-10T23:33:19.099 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-09-10T23:33:19.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.096+0000 7fe809400640 1 -- 172.21.15.94:0/2806775096 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe7e00725e0 msgr2=0x7fe7e0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:19.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.096+0000 7fe809400640 1 --2- 172.21.15.94:0/2806775096 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe7e00725e0 0x7fe7e0074aa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4001230 tx=0x7fe7f4009040 comp rx=0 tx=0).stop 2024-09-10T23:33:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.096+0000 7fe809400640 1 -- 172.21.15.94:0/2806775096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8040d2290 msgr2=0x7fe804155890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.096+0000 7fe809400640 1 --2- 172.21.15.94:0/2806775096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8040d2290 0x7fe804155890 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c500 tx=0x7fe7ec00c9c0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.096+0000 7fe809400640 1 -- 172.21.15.94:0/2806775096 shutdown_connections 2024-09-10T23:33:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.096+0000 7fe809400640 1 --2- 172.21.15.94:0/2806775096 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe7e00725e0 0x7fe7e0074aa0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.096+0000 7fe809400640 1 --2- 172.21.15.94:0/2806775096 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe80415bd30 0x7fe804070d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.096+0000 7fe809400640 1 --2- 172.21.15.94:0/2806775096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8040d2290 0x7fe804155890 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.096+0000 7fe809400640 1 --2- 172.21.15.94:0/2806775096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8040ae980 0x7fe8040cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.096+0000 7fe809400640 1 -- 172.21.15.94:0/2806775096 >> 172.21.15.94:0/2806775096 conn(0x7fe80409aa70 msgr2=0x7fe8040b6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.096+0000 7fe809400640 1 -- 172.21.15.94:0/2806775096 shutdown_connections 2024-09-10T23:33:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.096+0000 7fe809400640 1 -- 172.21.15.94:0/2806775096 wait complete. 2024-09-10T23:33:19.120 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:33:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.256+0000 7f6628c00640 1 -- 172.21.15.94:0/634077001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6624073ff0 msgr2=0x7f66240cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.256+0000 7f6628c00640 1 --2- 172.21.15.94:0/634077001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6624073ff0 0x7f66240cfda0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f662406f240 tx=0x7f661401cd50 comp rx=0 tx=0).stop 2024-09-10T23:33:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.256+0000 7f6628c00640 1 -- 172.21.15.94:0/634077001 shutdown_connections 2024-09-10T23:33:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.256+0000 7f6628c00640 1 --2- 172.21.15.94:0/634077001 >> v2:172.21.15.181:3300/0 conn(0x7f66240d02e0 0x7f662415e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.256+0000 7f6628c00640 1 --2- 172.21.15.94:0/634077001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6624073ff0 0x7f66240cfda0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.256+0000 7f6628c00640 1 -- 172.21.15.94:0/634077001 >> 172.21.15.94:0/634077001 conn(0x7f662406cdf0 msgr2=0x7f662406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.256+0000 7f6628c00640 1 -- 172.21.15.94:0/634077001 shutdown_connections 2024-09-10T23:33:19.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.256+0000 7f6628c00640 1 -- 172.21.15.94:0/634077001 wait complete. 2024-09-10T23:33:19.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6628c00640 1 Processor -- start 2024-09-10T23:33:19.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6628c00640 1 -- start start 2024-09-10T23:33:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6628c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6624073ff0 0x7f66241f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6628c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66240d02e0 0x7f6624144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6628c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f662415ecb0 0x7f6624159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6628c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f66241679c0 con 0x7f662415ecb0 2024-09-10T23:33:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6628c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6624167b40 con 0x7f6624073ff0 2024-09-10T23:33:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6628c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6624167cc0 con 0x7f66240d02e0 2024-09-10T23:33:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6622800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f662415ecb0 0x7f6624159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6622800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f662415ecb0 0x7f6624159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38672/0 (socket says 172.21.15.94:38672) 2024-09-10T23:33:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6622800640 1 -- 172.21.15.94:0/1844771767 learned_addr learned my addr 172.21.15.94:0/1844771767 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6621400640 1 --2- 172.21.15.94:0/1844771767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66240d02e0 0x7f6624144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:19.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6621e00640 1 --2- 172.21.15.94:0/1844771767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6624073ff0 0x7f66241f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6622800640 1 -- 172.21.15.94:0/1844771767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66240d02e0 msgr2=0x7f6624144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6622800640 1 --2- 172.21.15.94:0/1844771767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66240d02e0 0x7f6624144550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6622800640 1 -- 172.21.15.94:0/1844771767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6624073ff0 msgr2=0x7f66241f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6622800640 1 --2- 172.21.15.94:0/1844771767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6624073ff0 0x7f66241f7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6622800640 1 -- 172.21.15.94:0/1844771767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6624158d80 con 0x7f662415ecb0 2024-09-10T23:33:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6622800640 1 --2- 172.21.15.94:0/1844771767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f662415ecb0 0x7f6624159e20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f661c00b410 tx=0x7f661c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f661a000640 1 -- 172.21.15.94:0/1844771767 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c015020 con 0x7f662415ecb0 2024-09-10T23:33:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f661a000640 1 -- 172.21.15.94:0/1844771767 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f661c011960 con 0x7f662415ecb0 2024-09-10T23:33:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f661a000640 1 -- 172.21.15.94:0/1844771767 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c011db0 con 0x7f662415ecb0 2024-09-10T23:33:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6628c00640 1 -- 172.21.15.94:0/1844771767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6624159060 con 0x7f662415ecb0 2024-09-10T23:33:19.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6628c00640 1 -- 172.21.15.94:0/1844771767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6624159620 con 0x7f662415ecb0 2024-09-10T23:33:19.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.260+0000 7f6628c00640 1 -- 172.21.15.94:0/1844771767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66240cfda0 con 0x7f662415ecb0 2024-09-10T23:33:19.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.264+0000 7f661a000640 1 -- 172.21.15.94:0/1844771767 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f661c0193b0 con 0x7f662415ecb0 2024-09-10T23:33:19.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.264+0000 7f661a000640 1 --2- 172.21.15.94:0/1844771767 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f66040725e0 0x7f6604074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:19.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.264+0000 7f661a000640 1 -- 172.21.15.94:0/1844771767 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f661c092ff0 con 0x7f662415ecb0 2024-09-10T23:33:19.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.264+0000 7f6621e00640 1 --2- 172.21.15.94:0/1844771767 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f66040725e0 0x7f6604074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:19.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.264+0000 7f6621e00640 1 --2- 172.21.15.94:0/1844771767 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f66040725e0 0x7f6604074aa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f66140af000 tx=0x7f661401b590 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:19.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.268+0000 7f661a000640 1 -- 172.21.15.94:0/1844771767 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f661c01f030 con 0x7f662415ecb0 2024-09-10T23:33:19.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.432+0000 7f6628c00640 1 -- 172.21.15.94:0/1844771767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66240b1410 con 0x7f662415ecb0 2024-09-10T23:33:19.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.436+0000 7f661a000640 1 -- 172.21.15.94:0/1844771767 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16657 (secure 0 0 0) 0x7f661c05e0b0 con 0x7f662415ecb0 2024-09-10T23:33:19.439 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:19.439 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":22,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:32:43.556132+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:32:39.855046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T23:32:40.904161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:33:19.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.436+0000 7f6628c00640 1 -- 172.21.15.94:0/1844771767 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f66040725e0 msgr2=0x7f6604074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:19.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.436+0000 7f6628c00640 1 --2- 172.21.15.94:0/1844771767 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f66040725e0 0x7f6604074aa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f66140af000 tx=0x7f661401b590 comp rx=0 tx=0).stop 2024-09-10T23:33:19.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.436+0000 7f6628c00640 1 -- 172.21.15.94:0/1844771767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f662415ecb0 msgr2=0x7f6624159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:19.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.436+0000 7f6628c00640 1 --2- 172.21.15.94:0/1844771767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f662415ecb0 0x7f6624159e20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f661c00b410 tx=0x7f661c00b8d0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.436+0000 7f6628c00640 1 -- 172.21.15.94:0/1844771767 shutdown_connections 2024-09-10T23:33:19.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.436+0000 7f6628c00640 1 --2- 172.21.15.94:0/1844771767 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f66040725e0 0x7f6604074aa0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.436+0000 7f6628c00640 1 --2- 172.21.15.94:0/1844771767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f662415ecb0 0x7f6624159e20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.436+0000 7f6628c00640 1 --2- 172.21.15.94:0/1844771767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66240d02e0 0x7f6624144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.436+0000 7f6628c00640 1 --2- 172.21.15.94:0/1844771767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6624073ff0 0x7f66241f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.436+0000 7f6628c00640 1 -- 172.21.15.94:0/1844771767 >> 172.21.15.94:0/1844771767 conn(0x7f662406cdf0 msgr2=0x7f66241510b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:19.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.436+0000 7f6628c00640 1 -- 172.21.15.94:0/1844771767 shutdown_connections 2024-09-10T23:33:19.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.436+0000 7f6628c00640 1 -- 172.21.15.94:0/1844771767 wait complete. 2024-09-10T23:33:19.461 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T23:33:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943c200640 1 -- 172.21.15.94:0/1461341611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94340cd9f0 msgr2=0x7f94340cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943c200640 1 --2- 172.21.15.94:0/1461341611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94340cd9f0 0x7f94340cde60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f942c009bc0 tx=0x7f942c01cba0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943c200640 1 -- 172.21.15.94:0/1461341611 shutdown_connections 2024-09-10T23:33:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943c200640 1 --2- 172.21.15.94:0/1461341611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94340cd9f0 0x7f94340cde60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943c200640 1 -- 172.21.15.94:0/1461341611 >> 172.21.15.94:0/1461341611 conn(0x7f94340cfaf0 msgr2=0x7f94340cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943c200640 1 -- 172.21.15.94:0/1461341611 shutdown_connections 2024-09-10T23:33:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943c200640 1 -- 172.21.15.94:0/1461341611 wait complete. 2024-09-10T23:33:19.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943c200640 1 Processor -- start 2024-09-10T23:33:19.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943c200640 1 -- start start 2024-09-10T23:33:19.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943c200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94340cd9f0 0x7f9434155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:19.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943c200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94340d0710 0x7f94341f9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:19.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943c200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94341455d0 0x7f94341f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:19.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943c200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f943416a460 con 0x7f94340cd9f0 2024-09-10T23:33:19.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943c200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f943416a2e0 con 0x7f94341455d0 2024-09-10T23:33:19.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943c200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f943416a5e0 con 0x7f94340d0710 2024-09-10T23:33:19.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f9439800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94340cd9f0 0x7f9434155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:19.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f9439800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94340cd9f0 0x7f9434155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38690/0 (socket says 172.21.15.94:38690) 2024-09-10T23:33:19.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f9439800640 1 -- 172.21.15.94:0/3280123779 learned_addr learned my addr 172.21.15.94:0/3280123779 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:19.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f9438e00640 1 --2- 172.21.15.94:0/3280123779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94340d0710 0x7f94341f9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:19.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943a200640 1 --2- 172.21.15.94:0/3280123779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94341455d0 0x7f94341f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:19.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f9439800640 1 -- 172.21.15.94:0/3280123779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94340d0710 msgr2=0x7f94341f9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:19.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f9439800640 1 --2- 172.21.15.94:0/3280123779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94340d0710 0x7f94341f9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f9439800640 1 -- 172.21.15.94:0/3280123779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94341455d0 msgr2=0x7f94341f5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:19.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f9439800640 1 --2- 172.21.15.94:0/3280123779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94341455d0 0x7f94341f5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f9439800640 1 -- 172.21.15.94:0/3280123779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94341f6cb0 con 0x7f94340cd9f0 2024-09-10T23:33:19.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f9438e00640 1 --2- 172.21.15.94:0/3280123779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94340d0710 0x7f94341f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:19.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.600+0000 7f943a200640 1 --2- 172.21.15.94:0/3280123779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94341455d0 0x7f94341f5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:19.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.604+0000 7f9439800640 1 --2- 172.21.15.94:0/3280123779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94340cd9f0 0x7f9434155d60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f942800c400 tx=0x7f942800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:19.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.604+0000 7f9431600640 1 -- 172.21.15.94:0/3280123779 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428011790 con 0x7f94340cd9f0 2024-09-10T23:33:19.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.604+0000 7f9431600640 1 -- 172.21.15.94:0/3280123779 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9428017070 con 0x7f94340cd9f0 2024-09-10T23:33:19.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.604+0000 7f943c200640 1 -- 172.21.15.94:0/3280123779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94341f66f0 con 0x7f94340cd9f0 2024-09-10T23:33:19.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.604+0000 7f9431600640 1 -- 172.21.15.94:0/3280123779 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428011cc0 con 0x7f94340cd9f0 2024-09-10T23:33:19.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.604+0000 7f943c200640 1 -- 172.21.15.94:0/3280123779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94341f93f0 con 0x7f94340cd9f0 2024-09-10T23:33:19.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.604+0000 7f943c200640 1 -- 172.21.15.94:0/3280123779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94340cde60 con 0x7f94340cd9f0 2024-09-10T23:33:19.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.604+0000 7f9431600640 1 -- 172.21.15.94:0/3280123779 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f942801c460 con 0x7f94340cd9f0 2024-09-10T23:33:19.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.604+0000 7f9431600640 1 --2- 172.21.15.94:0/3280123779 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f94140725e0 0x7f9414074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:19.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.604+0000 7f9431600640 1 -- 172.21.15.94:0/3280123779 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9428092fa0 con 0x7f94340cd9f0 2024-09-10T23:33:19.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.604+0000 7f9438e00640 1 --2- 172.21.15.94:0/3280123779 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f94140725e0 0x7f9414074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:19.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.608+0000 7f9438e00640 1 --2- 172.21.15.94:0/3280123779 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f94140725e0 0x7f9414074aa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9420003ec0 tx=0x7f9420009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:19.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.608+0000 7f9431600640 1 -- 172.21.15.94:0/3280123779 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f94340701c0 con 0x7f94340cd9f0 2024-09-10T23:33:19.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.808+0000 7f943c200640 1 -- 172.21.15.94:0/3280123779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f94340cf6e0 con 0x7f94340cd9f0 2024-09-10T23:33:19.813 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-10T23:33:19.808+0000 7f4045a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:33:19.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.812+0000 7f9431600640 1 -- 172.21.15.94:0/3280123779 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f942805e6f0 con 0x7f94340cd9f0 2024-09-10T23:33:19.815 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T23:33:19.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.812+0000 7f943c200640 1 -- 172.21.15.94:0/3280123779 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f94140725e0 msgr2=0x7f9414074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:19.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.812+0000 7f943c200640 1 --2- 172.21.15.94:0/3280123779 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f94140725e0 0x7f9414074aa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9420003ec0 tx=0x7f9420009040 comp rx=0 tx=0).stop 2024-09-10T23:33:19.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.812+0000 7f943c200640 1 -- 172.21.15.94:0/3280123779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94340cd9f0 msgr2=0x7f9434155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:19.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.812+0000 7f943c200640 1 --2- 172.21.15.94:0/3280123779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94340cd9f0 0x7f9434155d60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f942800c400 tx=0x7f942800c8c0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.812+0000 7f943c200640 1 -- 172.21.15.94:0/3280123779 shutdown_connections 2024-09-10T23:33:19.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.812+0000 7f943c200640 1 --2- 172.21.15.94:0/3280123779 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f94140725e0 0x7f9414074aa0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.812+0000 7f943c200640 1 --2- 172.21.15.94:0/3280123779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94341455d0 0x7f94341f5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.812+0000 7f943c200640 1 --2- 172.21.15.94:0/3280123779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94340d0710 0x7f94341f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.812+0000 7f943c200640 1 --2- 172.21.15.94:0/3280123779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94340cd9f0 0x7f9434155d60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.812+0000 7f943c200640 1 -- 172.21.15.94:0/3280123779 >> 172.21.15.94:0/3280123779 conn(0x7f94340cfaf0 msgr2=0x7f943415e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:19.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.812+0000 7f943c200640 1 -- 172.21.15.94:0/3280123779 shutdown_connections 2024-09-10T23:33:19.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.812+0000 7f943c200640 1 -- 172.21.15.94:0/3280123779 wait complete. 2024-09-10T23:33:19.826 INFO:tasks.ceph.mds.b.smithi181.stderr: -2365> 2024-09-10T23:32:47.527+0000 7f50d7200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:33:19.838 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T23:33:19.874 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:33:19.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.976+0000 7eff8c000640 1 -- 172.21.15.94:0/474272942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff8415f560 msgr2=0x7eff840cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:19.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.976+0000 7eff8c000640 1 --2- 172.21.15.94:0/474272942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff8415f560 0x7eff840cd0a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7eff840b7cc0 tx=0x7eff7001c8c0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.976+0000 7eff8c000640 1 -- 172.21.15.94:0/474272942 shutdown_connections 2024-09-10T23:33:19.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.976+0000 7eff8c000640 1 --2- 172.21.15.94:0/474272942 >> v2:172.21.15.181:3300/0 conn(0x7eff840d01a0 0x7eff840b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.976+0000 7eff8c000640 1 --2- 172.21.15.94:0/474272942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff8415f560 0x7eff840cd0a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.976+0000 7eff8c000640 1 -- 172.21.15.94:0/474272942 >> 172.21.15.94:0/474272942 conn(0x7eff840d1cb0 msgr2=0x7eff840d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:19.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.976+0000 7eff8c000640 1 -- 172.21.15.94:0/474272942 shutdown_connections 2024-09-10T23:33:19.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.976+0000 7eff8c000640 1 -- 172.21.15.94:0/474272942 wait complete. 2024-09-10T23:33:19.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff8c000640 1 Processor -- start 2024-09-10T23:33:19.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff8c000640 1 -- start start 2024-09-10T23:33:19.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff8c000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff840d01a0 0x7eff841654f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:19.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff8c000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff8415f180 0x7eff84164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:19.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff8c000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8415f560 0x7eff84157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:19.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff8c000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff84168c80 con 0x7eff840d01a0 2024-09-10T23:33:19.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff8c000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7eff84168b00 con 0x7eff8415f180 2024-09-10T23:33:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff8c000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7eff84168e00 con 0x7eff8415f560 2024-09-10T23:33:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff89600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff840d01a0 0x7eff841654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff89600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff840d01a0 0x7eff841654f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38698/0 (socket says 172.21.15.94:38698) 2024-09-10T23:33:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff89600640 1 -- 172.21.15.94:0/1319981691 learned_addr learned my addr 172.21.15.94:0/1319981691 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff88c00640 1 --2- 172.21.15.94:0/1319981691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff8415f180 0x7eff84164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff8a000640 1 --2- 172.21.15.94:0/1319981691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8415f560 0x7eff84157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff89600640 1 -- 172.21.15.94:0/1319981691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8415f560 msgr2=0x7eff84157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff89600640 1 --2- 172.21.15.94:0/1319981691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8415f560 0x7eff84157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff89600640 1 -- 172.21.15.94:0/1319981691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff8415f180 msgr2=0x7eff84164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff89600640 1 --2- 172.21.15.94:0/1319981691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff8415f180 0x7eff84164cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:19.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff89600640 1 -- 172.21.15.94:0/1319981691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff84165bb0 con 0x7eff840d01a0 2024-09-10T23:33:19.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff89600640 1 --2- 172.21.15.94:0/1319981691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff840d01a0 0x7eff841654f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7eff7800c500 tx=0x7eff7800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:19.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff81600640 1 -- 172.21.15.94:0/1319981691 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78017070 con 0x7eff840d01a0 2024-09-10T23:33:19.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff81600640 1 -- 172.21.15.94:0/1319981691 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff780053c0 con 0x7eff840d01a0 2024-09-10T23:33:19.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff8c000640 1 -- 172.21.15.94:0/1319981691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff84163ef0 con 0x7eff840d01a0 2024-09-10T23:33:19.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff81600640 1 -- 172.21.15.94:0/1319981691 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78012890 con 0x7eff840d01a0 2024-09-10T23:33:19.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.980+0000 7eff8c000640 1 -- 172.21.15.94:0/1319981691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff841644b0 con 0x7eff840d01a0 2024-09-10T23:33:19.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.984+0000 7eff8c000640 1 -- 172.21.15.94:0/1319981691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff840b16f0 con 0x7eff840d01a0 2024-09-10T23:33:19.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.984+0000 7eff81600640 1 -- 172.21.15.94:0/1319981691 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7eff78003600 con 0x7eff840d01a0 2024-09-10T23:33:19.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.984+0000 7eff81600640 1 --2- 172.21.15.94:0/1319981691 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7eff640725e0 0x7eff64074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:19.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.984+0000 7eff81600640 1 -- 172.21.15.94:0/1319981691 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5795+0+0 (secure 0 0 0) 0x7eff78092860 con 0x7eff840d01a0 2024-09-10T23:33:19.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.984+0000 7eff88c00640 1 --2- 172.21.15.94:0/1319981691 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7eff640725e0 0x7eff64074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:19.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.984+0000 7eff88c00640 1 --2- 172.21.15.94:0/1319981691 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7eff640725e0 0x7eff64074aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7eff700af000 tx=0x7eff700a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:19.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:19.988+0000 7eff81600640 1 -- 172.21.15.94:0/1319981691 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7eff7805d800 con 0x7eff840d01a0 2024-09-10T23:33:20.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.192+0000 7eff8c000640 1 -- 172.21.15.94:0/1319981691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7eff840c54c0 con 0x7eff840d01a0 2024-09-10T23:33:20.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.812+0000 7eff81600640 1 -- 172.21.15.94:0/1319981691 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7eff78062230 con 0x7eff840d01a0 2024-09-10T23:33:20.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.816+0000 7eff8c000640 1 -- 172.21.15.94:0/1319981691 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7eff640725e0 msgr2=0x7eff64074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:20.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.816+0000 7eff8c000640 1 --2- 172.21.15.94:0/1319981691 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7eff640725e0 0x7eff64074aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7eff700af000 tx=0x7eff700a7040 comp rx=0 tx=0).stop 2024-09-10T23:33:20.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.816+0000 7eff8c000640 1 -- 172.21.15.94:0/1319981691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff840d01a0 msgr2=0x7eff841654f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:20.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.816+0000 7eff8c000640 1 --2- 172.21.15.94:0/1319981691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff840d01a0 0x7eff841654f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7eff7800c500 tx=0x7eff7800c9c0 comp rx=0 tx=0).stop 2024-09-10T23:33:20.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.816+0000 7eff8c000640 1 -- 172.21.15.94:0/1319981691 shutdown_connections 2024-09-10T23:33:20.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.816+0000 7eff8c000640 1 --2- 172.21.15.94:0/1319981691 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7eff640725e0 0x7eff64074aa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:20.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.816+0000 7eff8c000640 1 --2- 172.21.15.94:0/1319981691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff8415f180 0x7eff84164cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:20.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.816+0000 7eff8c000640 1 --2- 172.21.15.94:0/1319981691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8415f560 0x7eff84157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:20.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.816+0000 7eff8c000640 1 --2- 172.21.15.94:0/1319981691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff840d01a0 0x7eff841654f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:20.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.816+0000 7eff8c000640 1 -- 172.21.15.94:0/1319981691 >> 172.21.15.94:0/1319981691 conn(0x7eff840d1cb0 msgr2=0x7eff8415c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:20.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.816+0000 7eff8c000640 1 -- 172.21.15.94:0/1319981691 shutdown_connections 2024-09-10T23:33:20.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.816+0000 7eff8c000640 1 -- 172.21.15.94:0/1319981691 wait complete. 2024-09-10T23:33:20.841 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T23:33:20.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f994a800640 1 -- 172.21.15.94:0/1814736949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9928003a60 msgr2=0x7f9928003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:20.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f994a800640 1 --2- 172.21.15.94:0/1814736949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9928003a60 0x7f9928003e80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f99340016c0 tx=0x7f993401c700 comp rx=0 tx=0).stop 2024-09-10T23:33:20.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f994a800640 1 -- 172.21.15.94:0/1814736949 shutdown_connections 2024-09-10T23:33:20.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f994a800640 1 --2- 172.21.15.94:0/1814736949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9928003a60 0x7f9928003e80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:20.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f994a800640 1 -- 172.21.15.94:0/1814736949 >> 172.21.15.94:0/1814736949 conn(0x7f994406cdf0 msgr2=0x7f994406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:20.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f994a800640 1 -- 172.21.15.94:0/1814736949 shutdown_connections 2024-09-10T23:33:20.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f994a800640 1 -- 172.21.15.94:0/1814736949 wait complete. 2024-09-10T23:33:20.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f994a800640 1 Processor -- start 2024-09-10T23:33:20.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f994a800640 1 -- start start 2024-09-10T23:33:20.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f994a800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9928003a60 0x7f9944140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:20.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f994a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9944144070 0x7f994413d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:20.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f994a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99440ce700 0x7f99440ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:20.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f994a800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f99440b4a90 con 0x7f99440ce700 2024-09-10T23:33:20.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f994a800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f99440b4910 con 0x7f9944144070 2024-09-10T23:33:20.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f994a800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f99440b4c10 con 0x7f9928003a60 2024-09-10T23:33:20.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f9943e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99440ce700 0x7f99440ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:20.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f9943e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99440ce700 0x7f99440ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38706/0 (socket says 172.21.15.94:38706) 2024-09-10T23:33:20.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f9943e00640 1 -- 172.21.15.94:0/433498041 learned_addr learned my addr 172.21.15.94:0/433498041 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:20.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f9943400640 1 --2- 172.21.15.94:0/433498041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9928003a60 0x7f9944140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:20.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f993aa00640 1 --2- 172.21.15.94:0/433498041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9944144070 0x7f994413d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:20.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f9943e00640 1 -- 172.21.15.94:0/433498041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9928003a60 msgr2=0x7f9944140970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:20.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f9943e00640 1 --2- 172.21.15.94:0/433498041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9928003a60 0x7f9944140970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:20.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f9943e00640 1 -- 172.21.15.94:0/433498041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9944144070 msgr2=0x7f994413d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:20.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f9943e00640 1 --2- 172.21.15.94:0/433498041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9944144070 0x7f994413d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:20.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.988+0000 7f9943e00640 1 -- 172.21.15.94:0/433498041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99441fabb0 con 0x7f99440ce700 2024-09-10T23:33:20.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.992+0000 7f9943e00640 1 --2- 172.21.15.94:0/433498041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99440ce700 0x7f99440ceb80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9944070a10 tx=0x7f993c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:20.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.992+0000 7f993be00640 1 -- 172.21.15.94:0/433498041 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c004b30 con 0x7f99440ce700 2024-09-10T23:33:20.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.992+0000 7f993be00640 1 -- 172.21.15.94:0/433498041 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f993c004cd0 con 0x7f99440ce700 2024-09-10T23:33:20.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.992+0000 7f993be00640 1 -- 172.21.15.94:0/433498041 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c003be0 con 0x7f99440ce700 2024-09-10T23:33:20.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.992+0000 7f994a800640 1 -- 172.21.15.94:0/433498041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9944140eb0 con 0x7f99440ce700 2024-09-10T23:33:20.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.992+0000 7f994a800640 1 -- 172.21.15.94:0/433498041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99441fb110 con 0x7f99440ce700 2024-09-10T23:33:20.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.992+0000 7f994a800640 1 -- 172.21.15.94:0/433498041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99440ce4f0 con 0x7f99440ce700 2024-09-10T23:33:20.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.992+0000 7f993be00640 1 -- 172.21.15.94:0/433498041 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f993c018070 con 0x7f99440ce700 2024-09-10T23:33:20.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.992+0000 7f993be00640 1 --2- 172.21.15.94:0/433498041 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f99200725e0 0x7f9920074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:20.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.996+0000 7f993be00640 1 -- 172.21.15.94:0/433498041 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5795+0+0 (secure 0 0 0) 0x7f993c093bf0 con 0x7f99440ce700 2024-09-10T23:33:20.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.996+0000 7f9943400640 1 --2- 172.21.15.94:0/433498041 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f99200725e0 0x7f9920074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:20.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:20.996+0000 7f9943400640 1 --2- 172.21.15.94:0/433498041 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f99200725e0 0x7f9920074aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f99280057f0 tx=0x7f99340b0040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:21.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:21.000+0000 7f993be00640 1 -- 172.21.15.94:0/433498041 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9944140eb0 con 0x7f99440ce700 2024-09-10T23:33:21.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:21.164+0000 7f994a800640 1 -- 172.21.15.94:0/433498041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f994415f490 con 0x7f99440ce700 2024-09-10T23:33:21.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:21.816+0000 7f993be00640 1 -- 172.21.15.94:0/433498041 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7f99441fabb0 con 0x7f99440ce700 2024-09-10T23:33:21.822 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-09-10T23:33:21.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:21.820+0000 7f994a800640 1 -- 172.21.15.94:0/433498041 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f99200725e0 msgr2=0x7f9920074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:21.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:21.820+0000 7f994a800640 1 --2- 172.21.15.94:0/433498041 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f99200725e0 0x7f9920074aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f99280057f0 tx=0x7f99340b0040 comp rx=0 tx=0).stop 2024-09-10T23:33:21.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:21.820+0000 7f994a800640 1 -- 172.21.15.94:0/433498041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99440ce700 msgr2=0x7f99440ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:21.820+0000 7f994a800640 1 --2- 172.21.15.94:0/433498041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99440ce700 0x7f99440ceb80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9944070a10 tx=0x7f993c00f3c0 comp rx=0 tx=0).stop 2024-09-10T23:33:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:21.820+0000 7f994a800640 1 -- 172.21.15.94:0/433498041 shutdown_connections 2024-09-10T23:33:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:21.820+0000 7f994a800640 1 --2- 172.21.15.94:0/433498041 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f99200725e0 0x7f9920074aa0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:21.820+0000 7f994a800640 1 --2- 172.21.15.94:0/433498041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9944144070 0x7f994413d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:21.820+0000 7f994a800640 1 --2- 172.21.15.94:0/433498041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99440ce700 0x7f99440ceb80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:21.820+0000 7f994a800640 1 --2- 172.21.15.94:0/433498041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9928003a60 0x7f9944140970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:21.820+0000 7f994a800640 1 -- 172.21.15.94:0/433498041 >> 172.21.15.94:0/433498041 conn(0x7f994406cdf0 msgr2=0x7f9944072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:21.820+0000 7f994a800640 1 -- 172.21.15.94:0/433498041 shutdown_connections 2024-09-10T23:33:21.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:21.820+0000 7f994a800640 1 -- 172.21.15.94:0/433498041 wait complete. 2024-09-10T23:33:21.855 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T23:33:22.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.000+0000 7f5ed0c00640 1 -- 172.21.15.94:0/2894660255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb0004810 msgr2=0x7f5eb0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:22.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.000+0000 7f5ed0c00640 1 --2- 172.21.15.94:0/2894660255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb0004810 0x7f5eb0004c90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80016f0 tx=0x7f5eb801c790 comp rx=0 tx=0).stop 2024-09-10T23:33:22.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.000+0000 7f5ed0c00640 1 -- 172.21.15.94:0/2894660255 shutdown_connections 2024-09-10T23:33:22.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.000+0000 7f5ed0c00640 1 --2- 172.21.15.94:0/2894660255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb0004810 0x7f5eb0004c90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:22.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.000+0000 7f5ed0c00640 1 -- 172.21.15.94:0/2894660255 >> 172.21.15.94:0/2894660255 conn(0x7f5ecc0d1c60 msgr2=0x7f5ecc0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:22.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.000+0000 7f5ed0c00640 1 -- 172.21.15.94:0/2894660255 shutdown_connections 2024-09-10T23:33:22.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.000+0000 7f5ed0c00640 1 -- 172.21.15.94:0/2894660255 wait complete. 2024-09-10T23:33:22.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.000+0000 7f5ed0c00640 1 Processor -- start 2024-09-10T23:33:22.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.000+0000 7f5ed0c00640 1 -- start start 2024-09-10T23:33:22.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5ed0c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb0004810 0x7f5ecc0b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:22.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5ed0c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ecc0b62f0 0x7f5ecc0c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:22.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5ed0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ecc0c7d40 0x7f5ecc06f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:22.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5ed0c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc0b1740 con 0x7f5ecc0c7d40 2024-09-10T23:33:22.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5ed0c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc0b15c0 con 0x7f5ecc0b62f0 2024-09-10T23:33:22.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5ed0c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5ecc0b18c0 con 0x7f5eb0004810 2024-09-10T23:33:22.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5eca800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ecc0c7d40 0x7f5ecc06f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:22.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5eca800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ecc0c7d40 0x7f5ecc06f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38738/0 (socket says 172.21.15.94:38738) 2024-09-10T23:33:22.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5eca800640 1 -- 172.21.15.94:0/2819159619 learned_addr learned my addr 172.21.15.94:0/2819159619 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:22.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5ec9400640 1 --2- 172.21.15.94:0/2819159619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ecc0b62f0 0x7f5ecc0c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:22.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5ec9e00640 1 --2- 172.21.15.94:0/2819159619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb0004810 0x7f5ecc0b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:22.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5eca800640 1 -- 172.21.15.94:0/2819159619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb0004810 msgr2=0x7f5ecc0b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:22.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5eca800640 1 --2- 172.21.15.94:0/2819159619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb0004810 0x7f5ecc0b5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:22.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5eca800640 1 -- 172.21.15.94:0/2819159619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ecc0b62f0 msgr2=0x7f5ecc0c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:22.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5eca800640 1 --2- 172.21.15.94:0/2819159619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ecc0b62f0 0x7f5ecc0c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:22.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5eca800640 1 -- 172.21.15.94:0/2819159619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc0c82c0 con 0x7f5ecc0c7d40 2024-09-10T23:33:22.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5eca800640 1 --2- 172.21.15.94:0/2819159619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ecc0c7d40 0x7f5ecc06f230 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc073640 tx=0x7f5ec400ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:22.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5ec2000640 1 -- 172.21.15.94:0/2819159619 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4010890 con 0x7f5ecc0c7d40 2024-09-10T23:33:22.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5ec2000640 1 -- 172.21.15.94:0/2819159619 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ec4017070 con 0x7f5ecc0c7d40 2024-09-10T23:33:22.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5ed0c00640 1 -- 172.21.15.94:0/2819159619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ecc0c85a0 con 0x7f5ecc0c7d40 2024-09-10T23:33:22.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5ec2000640 1 -- 172.21.15.94:0/2819159619 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4010dc0 con 0x7f5ecc0c7d40 2024-09-10T23:33:22.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5ed0c00640 1 -- 172.21.15.94:0/2819159619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ecc0b4e10 con 0x7f5ecc0c7d40 2024-09-10T23:33:22.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5ed0c00640 1 -- 172.21.15.94:0/2819159619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ecc0749c0 con 0x7f5ecc0c7d40 2024-09-10T23:33:22.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.004+0000 7f5ec2000640 1 -- 172.21.15.94:0/2819159619 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5ec401c460 con 0x7f5ecc0c7d40 2024-09-10T23:33:22.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.008+0000 7f5ec2000640 1 --2- 172.21.15.94:0/2819159619 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5eac0725e0 0x7f5eac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:22.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.008+0000 7f5ec2000640 1 -- 172.21.15.94:0/2819159619 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5386+0+0 (secure 0 0 0) 0x7f5ec4093450 con 0x7f5ecc0c7d40 2024-09-10T23:33:22.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.008+0000 7f5ec9e00640 1 --2- 172.21.15.94:0/2819159619 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5eac0725e0 0x7f5eac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:22.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.008+0000 7f5ec9e00640 1 --2- 172.21.15.94:0/2819159619 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5eac0725e0 0x7f5eac074aa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc06b7e0 tx=0x7f5ebc0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:22.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.012+0000 7f5ec2000640 1 -- 172.21.15.94:0/2819159619 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5ec405ebd0 con 0x7f5ecc0c7d40 2024-09-10T23:33:22.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.176+0000 7f5ed0c00640 1 -- 172.21.15.94:0/2819159619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5ecc070920 con 0x7f5ecc0c7d40 2024-09-10T23:33:22.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.820+0000 7f5ec2000640 1 -- 172.21.15.94:0/2819159619 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f5ec40672f0 con 0x7f5ecc0c7d40 2024-09-10T23:33:22.824 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-09-10T23:33:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.824+0000 7f5ed0c00640 1 -- 172.21.15.94:0/2819159619 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5eac0725e0 msgr2=0x7f5eac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.824+0000 7f5ed0c00640 1 --2- 172.21.15.94:0/2819159619 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5eac0725e0 0x7f5eac074aa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc06b7e0 tx=0x7f5ebc0091e0 comp rx=0 tx=0).stop 2024-09-10T23:33:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.824+0000 7f5ed0c00640 1 -- 172.21.15.94:0/2819159619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ecc0c7d40 msgr2=0x7f5ecc06f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.824+0000 7f5ed0c00640 1 --2- 172.21.15.94:0/2819159619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ecc0c7d40 0x7f5ecc06f230 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc073640 tx=0x7f5ec400ea20 comp rx=0 tx=0).stop 2024-09-10T23:33:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.824+0000 7f5ed0c00640 1 -- 172.21.15.94:0/2819159619 shutdown_connections 2024-09-10T23:33:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.824+0000 7f5ed0c00640 1 --2- 172.21.15.94:0/2819159619 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5eac0725e0 0x7f5eac074aa0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.824+0000 7f5ed0c00640 1 --2- 172.21.15.94:0/2819159619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ecc0b62f0 0x7f5ecc0c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.824+0000 7f5ed0c00640 1 --2- 172.21.15.94:0/2819159619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ecc0c7d40 0x7f5ecc06f230 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.824+0000 7f5ed0c00640 1 --2- 172.21.15.94:0/2819159619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb0004810 0x7f5ecc0b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:22.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.824+0000 7f5ed0c00640 1 -- 172.21.15.94:0/2819159619 >> 172.21.15.94:0/2819159619 conn(0x7f5ecc0d1c60 msgr2=0x7f5ecc06de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.824+0000 7f5ed0c00640 1 -- 172.21.15.94:0/2819159619 shutdown_connections 2024-09-10T23:33:22.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:22.824+0000 7f5ed0c00640 1 -- 172.21.15.94:0/2819159619 wait complete. 2024-09-10T23:33:22.851 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T23:33:22.851 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T23:33:22.851 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T23:33:22.851 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:33:22.851 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T23:33:22.851 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T23:33:22.852 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T23:33:22.852 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:33:22.852 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T23:33:22.852 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T23:33:22.852 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T23:33:22.852 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:33:22.852 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T23:33:22.852 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T23:33:22.853 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T23:33:22.853 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:33:22.863 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T23:33:22.863 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T23:33:22.864 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T23:33:22.864 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T23:33:22.865 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T23:33:22.866 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T23:33:22.866 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T23:33:22.867 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T23:33:22.909 INFO:tasks.ceph.mds.a:Started 2024-09-10T23:33:22.909 DEBUG:teuthology.parallel:result is None 2024-09-10T23:33:22.910 INFO:tasks.ceph.mds.c:Started 2024-09-10T23:33:22.910 DEBUG:teuthology.parallel:result is None 2024-09-10T23:33:22.916 INFO:tasks.ceph.mds.d:Started 2024-09-10T23:33:22.916 DEBUG:teuthology.parallel:result is None 2024-09-10T23:33:22.918 INFO:tasks.ceph.mds.b:Started 2024-09-10T23:33:22.918 DEBUG:teuthology.parallel:result is None 2024-09-10T23:33:22.918 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T23:33:22.994 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-09-10T23:33:22.995 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-09-10T23:33:23.005 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:33:23.006 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2024-09-10T23:33:23.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.064+0000 7fe055000640 1 -- 172.21.15.94:0/2733317390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0500ce280 msgr2=0x7fe0500cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:23.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.064+0000 7fe055000640 1 --2- 172.21.15.94:0/2733317390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0500ce280 0x7fe0500cf280 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe040009840 tx=0x7fe04001c820 comp rx=0 tx=0).stop 2024-09-10T23:33:23.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.064+0000 7fe055000640 1 -- 172.21.15.94:0/2733317390 shutdown_connections 2024-09-10T23:33:23.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.064+0000 7fe055000640 1 --2- 172.21.15.94:0/2733317390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0500c64f0 0x7fe0500ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:23.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.064+0000 7fe055000640 1 --2- 172.21.15.94:0/2733317390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0500ce280 0x7fe0500cf280 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:23.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.064+0000 7fe055000640 1 -- 172.21.15.94:0/2733317390 >> 172.21.15.94:0/2733317390 conn(0x7fe0500cfaf0 msgr2=0x7fe0500cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:23.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.064+0000 7fe055000640 1 -- 172.21.15.94:0/2733317390 shutdown_connections 2024-09-10T23:33:23.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.064+0000 7fe055000640 1 -- 172.21.15.94:0/2733317390 wait complete. 2024-09-10T23:33:23.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.064+0000 7fe055000640 1 Processor -- start 2024-09-10T23:33:23.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe055000640 1 -- start start 2024-09-10T23:33:23.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe055000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe05006e7c0 0x7fe0501fb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:23.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe055000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0500c64f0 0x7fe050142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:23.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe055000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0500ce280 0x7fe0501ee350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:23.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe055000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe050167680 con 0x7fe0500ce280 2024-09-10T23:33:23.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe055000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe050167500 con 0x7fe0500c64f0 2024-09-10T23:33:23.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe055000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe050167800 con 0x7fe05006e7c0 2024-09-10T23:33:23.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe04e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0500ce280 0x7fe0501ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:23.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe04de00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe05006e7c0 0x7fe0501fb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:23.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe04de00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe05006e7c0 0x7fe0501fb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:36368/0 (socket says 172.21.15.94:36368) 2024-09-10T23:33:23.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe04de00640 1 -- 172.21.15.94:0/4005961798 learned_addr learned my addr 172.21.15.94:0/4005961798 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:23.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe04d400640 1 --2- 172.21.15.94:0/4005961798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0500c64f0 0x7fe050142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:23.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe04de00640 1 -- 172.21.15.94:0/4005961798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0500c64f0 msgr2=0x7fe050142ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:23.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe04de00640 1 --2- 172.21.15.94:0/4005961798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0500c64f0 0x7fe050142ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:23.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe04de00640 1 -- 172.21.15.94:0/4005961798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0500ce280 msgr2=0x7fe0501ee350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:23.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe04de00640 1 --2- 172.21.15.94:0/4005961798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0500ce280 0x7fe0501ee350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:23.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe04de00640 1 -- 172.21.15.94:0/4005961798 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0501fbb80 con 0x7fe05006e7c0 2024-09-10T23:33:23.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe04de00640 1 --2- 172.21.15.94:0/4005961798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe05006e7c0 0x7fe0501fb5a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe04001aed0 tx=0x7fe0400a57d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:23.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe046000640 1 -- 172.21.15.94:0/4005961798 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0400bb070 con 0x7fe05006e7c0 2024-09-10T23:33:23.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe046000640 1 -- 172.21.15.94:0/4005961798 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe040004430 con 0x7fe05006e7c0 2024-09-10T23:33:23.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe046000640 1 -- 172.21.15.94:0/4005961798 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0400b2040 con 0x7fe05006e7c0 2024-09-10T23:33:23.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe055000640 1 -- 172.21.15.94:0/4005961798 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0501fbe60 con 0x7fe05006e7c0 2024-09-10T23:33:23.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe055000640 1 -- 172.21.15.94:0/4005961798 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0501ee890 con 0x7fe05006e7c0 2024-09-10T23:33:23.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.068+0000 7fe055000640 1 -- 172.21.15.94:0/4005961798 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0500ce660 con 0x7fe05006e7c0 2024-09-10T23:33:23.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.072+0000 7fe046000640 1 -- 172.21.15.94:0/4005961798 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe0400b5030 con 0x7fe05006e7c0 2024-09-10T23:33:23.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.072+0000 7fe046000640 1 --2- 172.21.15.94:0/4005961798 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe030072610 0x7fe030074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:23.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.072+0000 7fe046000640 1 -- 172.21.15.94:0/4005961798 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7fe04012c7e0 con 0x7fe05006e7c0 2024-09-10T23:33:23.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.072+0000 7fe04d400640 1 --2- 172.21.15.94:0/4005961798 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe030072610 0x7fe030074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:23.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.072+0000 7fe04d400640 1 --2- 172.21.15.94:0/4005961798 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe030072610 0x7fe030074ad0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe0500b5c50 tx=0x7fe03c010040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:23.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.076+0000 7fe046000640 1 -- 172.21.15.94:0/4005961798 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe0400c4020 con 0x7fe05006e7c0 2024-09-10T23:33:23.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.260+0000 7fe055000640 1 -- 172.21.15.94:0/4005961798 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe0500d0850 con 0x7fe05006e7c0 2024-09-10T23:33:23.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.852+0000 7fe046000640 1 -- 172.21.15.94:0/4005961798 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7fe040016410 con 0x7fe05006e7c0 2024-09-10T23:33:23.856 INFO:teuthology.orchestra.run.smithi094.stderr: removed all blocklist entries 2024-09-10T23:33:23.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.856+0000 7fe055000640 1 -- 172.21.15.94:0/4005961798 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe030072610 msgr2=0x7fe030074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:23.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.856+0000 7fe055000640 1 --2- 172.21.15.94:0/4005961798 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe030072610 0x7fe030074ad0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe0500b5c50 tx=0x7fe03c010040 comp rx=0 tx=0).stop 2024-09-10T23:33:23.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.856+0000 7fe055000640 1 -- 172.21.15.94:0/4005961798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe05006e7c0 msgr2=0x7fe0501fb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:23.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.856+0000 7fe055000640 1 --2- 172.21.15.94:0/4005961798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe05006e7c0 0x7fe0501fb5a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe04001aed0 tx=0x7fe0400a57d0 comp rx=0 tx=0).stop 2024-09-10T23:33:23.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.856+0000 7fe055000640 1 -- 172.21.15.94:0/4005961798 shutdown_connections 2024-09-10T23:33:23.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.856+0000 7fe055000640 1 --2- 172.21.15.94:0/4005961798 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe030072610 0x7fe030074ad0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:23.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.856+0000 7fe055000640 1 --2- 172.21.15.94:0/4005961798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0500c64f0 0x7fe050142ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:23.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.856+0000 7fe055000640 1 --2- 172.21.15.94:0/4005961798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0500ce280 0x7fe0501ee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:23.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.856+0000 7fe055000640 1 --2- 172.21.15.94:0/4005961798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe05006e7c0 0x7fe0501fb5a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:23.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.856+0000 7fe055000640 1 -- 172.21.15.94:0/4005961798 >> 172.21.15.94:0/4005961798 conn(0x7fe0500cfaf0 msgr2=0x7fe050073e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:23.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.856+0000 7fe055000640 1 -- 172.21.15.94:0/4005961798 shutdown_connections 2024-09-10T23:33:23.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:23.856+0000 7fe055000640 1 -- 172.21.15.94:0/4005961798 wait complete. 2024-09-10T23:33:23.879 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T23:33:24.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.016+0000 7fdcc4a00640 1 -- 172.21.15.94:0/2849166951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00b2bb0 msgr2=0x7fdcc00d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.016+0000 7fdcc4a00640 1 --2- 172.21.15.94:0/2849166951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00b2bb0 0x7fdcc00d00a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009900 tx=0x7fdcac01c7c0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.016+0000 7fdcc4a00640 1 -- 172.21.15.94:0/2849166951 shutdown_connections 2024-09-10T23:33:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.016+0000 7fdcc4a00640 1 --2- 172.21.15.94:0/2849166951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00b2bb0 0x7fdcc00d00a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.016+0000 7fdcc4a00640 1 --2- 172.21.15.94:0/2849166951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdcc015f5c0 0x7fdcc00cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.016+0000 7fdcc4a00640 1 -- 172.21.15.94:0/2849166951 >> 172.21.15.94:0/2849166951 conn(0x7fdcc00d1c60 msgr2=0x7fdcc00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.016+0000 7fdcc4a00640 1 -- 172.21.15.94:0/2849166951 shutdown_connections 2024-09-10T23:33:24.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.016+0000 7fdcc4a00640 1 -- 172.21.15.94:0/2849166951 wait complete. 2024-09-10T23:33:24.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.016+0000 7fdcc4a00640 1 Processor -- start 2024-09-10T23:33:24.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.016+0000 7fdcc4a00640 1 -- start start 2024-09-10T23:33:24.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcc4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00b2bb0 0x7fdcc014c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcc4a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdcc00d05e0 0x7fdcc0150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcc4a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcc015f5c0 0x7fdcc00cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcc4a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0169340 con 0x7fdcc00b2bb0 2024-09-10T23:33:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcc4a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01691c0 con 0x7fdcc015f5c0 2024-09-10T23:33:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcc4a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc01694c0 con 0x7fdcc00d05e0 2024-09-10T23:33:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcbde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00b2bb0 0x7fdcc014c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcbde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00b2bb0 0x7fdcc014c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60074/0 (socket says 172.21.15.94:60074) 2024-09-10T23:33:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcbde00640 1 -- 172.21.15.94:0/900484860 learned_addr learned my addr 172.21.15.94:0/900484860 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcbd400640 1 --2- 172.21.15.94:0/900484860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdcc00d05e0 0x7fdcc0150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcbe800640 1 --2- 172.21.15.94:0/900484860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcc015f5c0 0x7fdcc00cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcbde00640 1 -- 172.21.15.94:0/900484860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdcc00d05e0 msgr2=0x7fdcc0150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcbde00640 1 --2- 172.21.15.94:0/900484860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdcc00d05e0 0x7fdcc0150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcbde00640 1 -- 172.21.15.94:0/900484860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcc015f5c0 msgr2=0x7fdcc00cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcbde00640 1 --2- 172.21.15.94:0/900484860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcc015f5c0 0x7fdcc00cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcbde00640 1 -- 172.21.15.94:0/900484860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc00c7c90 con 0x7fdcc00b2bb0 2024-09-10T23:33:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcbe800640 1 --2- 172.21.15.94:0/900484860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcc015f5c0 0x7fdcc00cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcbde00640 1 --2- 172.21.15.94:0/900484860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00b2bb0 0x7fdcc014c7c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdcb00058d0 tx=0x7fdcb000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcb6000640 1 -- 172.21.15.94:0/900484860 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0017070 con 0x7fdcc00b2bb0 2024-09-10T23:33:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcb6000640 1 -- 172.21.15.94:0/900484860 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcb0009d20 con 0x7fdcc00b2bb0 2024-09-10T23:33:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcb6000640 1 -- 172.21.15.94:0/900484860 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00106c0 con 0x7fdcc00b2bb0 2024-09-10T23:33:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcc4a00640 1 -- 172.21.15.94:0/900484860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc014cd00 con 0x7fdcc00b2bb0 2024-09-10T23:33:24.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcc4a00640 1 -- 172.21.15.94:0/900484860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc00c7110 con 0x7fdcc00b2bb0 2024-09-10T23:33:24.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.020+0000 7fdcc4a00640 1 -- 172.21.15.94:0/900484860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc00b15c0 con 0x7fdcc00b2bb0 2024-09-10T23:33:24.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.024+0000 7fdcb6000640 1 -- 172.21.15.94:0/900484860 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdcb0010860 con 0x7fdcc00b2bb0 2024-09-10T23:33:24.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.024+0000 7fdcb6000640 1 --2- 172.21.15.94:0/900484860 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdca00725e0 0x7fdca0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:24.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.024+0000 7fdcb6000640 1 -- 172.21.15.94:0/900484860 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fdcb00932e0 con 0x7fdcc00b2bb0 2024-09-10T23:33:24.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.024+0000 7fdcbd400640 1 --2- 172.21.15.94:0/900484860 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdca00725e0 0x7fdca0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:24.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.024+0000 7fdcbd400640 1 --2- 172.21.15.94:0/900484860 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdca00725e0 0x7fdca0074aa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0af000 tx=0x7fdcac01b000 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:24.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.028+0000 7fdcb6000640 1 -- 172.21.15.94:0/900484860 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdcb005e6b0 con 0x7fdcc00b2bb0 2024-09-10T23:33:24.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.252+0000 7fdcc4a00640 1 -- 172.21.15.94:0/900484860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fdcc015e830 con 0x7fdcc00b2bb0 2024-09-10T23:33:24.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.252+0000 7fdcb6000640 1 -- 172.21.15.94:0/900484860 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+6645 (secure 0 0 0) 0x7fdcb00630e0 con 0x7fdcc00b2bb0 2024-09-10T23:33:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:24.256 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:33:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_dump": [ 2024-09-10T23:33:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.a", 2024-09-10T23:33:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/2m3LxAAr0ErpLJg+0+I2r74zQya4w==", 2024-09-10T23:33:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:33:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:33:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:33:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.b", 2024-09-10T23:33:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmc8pYABAAw90zu8D+9IwK15h4HIlGBA==", 2024-09-10T23:33:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:33:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:33:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:33:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.c", 2024-09-10T23:33:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmk98KNxAA44e9N4VHbw/94Lh1V9zugQ==", 2024-09-10T23:33:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:33:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:33:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:33:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.d", 2024-09-10T23:33:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmVAKzBxAAJehOw6aphTF/g3ucoQfw9A==", 2024-09-10T23:33:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:33:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:33:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:33:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.0", 2024-09-10T23:33:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDa1uBmSAD3KBAAJTF1OEC44Ml599aL7iEuDw==", 2024-09-10T23:33:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:33:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:33:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.1", 2024-09-10T23:33:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDe1uBma1L9ABAAfEjW1/Hb4fh4Kwv/ZUgFAQ==", 2024-09-10T23:33:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:33:24.259 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:33:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.2", 2024-09-10T23:33:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDh1uBmtLUQFhAAweeFHj6BRQVC9BX+VLiWEQ==", 2024-09-10T23:33:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:33:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:33:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.3", 2024-09-10T23:33:24.260 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDk1uBm1KFCKRAAev28iYcrEYOViX95uo6Kbw==", 2024-09-10T23:33:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:33:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:33:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.4", 2024-09-10T23:33:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDq1uBm/Hg9LhAAvB2D0rj4DeRPxTrGmXYIsA==", 2024-09-10T23:33:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:33:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:33:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.5", 2024-09-10T23:33:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDu1uBm1fAKCBAAFHMjnGFAItLdaACF/UjE/w==", 2024-09-10T23:33:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:33:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:33:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.6", 2024-09-10T23:33:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDx1uBmXgYfHhAA/6wwtM4mDRXsIpoTIHKJ0w==", 2024-09-10T23:33:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:33:24.262 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:33:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.7", 2024-09-10T23:33:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQD01uBmw6hQMxAAk7oGAg+iIplNK5KrokGwNA==", 2024-09-10T23:33:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:33:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:33:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.0", 2024-09-10T23:33:24.263 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g==", 2024-09-10T23:33:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:33:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:33:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:33:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:33:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.1", 2024-09-10T23:33:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmuSkhFRAA1R6WNXLImkGfhW9/XJzEAQ==", 2024-09-10T23:33:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:33:24.264 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:33:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:33:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:33:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.2", 2024-09-10T23:33:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmfvaqERAASpL0KtglFcAuRf4a/0z5JQ==", 2024-09-10T23:33:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:33:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:33:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:33:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:33:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.3", 2024-09-10T23:33:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBm8xUtGxAARc0ywfAhMxBNd57TzBoV3g==", 2024-09-10T23:33:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:33:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:33:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:33:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:33:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.admin", 2024-09-10T23:33:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmOpg1FBAA9NV6Jqt6doWMuAfcxztlOg==", 2024-09-10T23:33:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:33:24.266 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:33:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:33:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-mds", 2024-09-10T23:33:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQAA1+Bmle64BRAAOTPLqN+x0aa9uSxAMApbww==", 2024-09-10T23:33:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-mds" 2024-09-10T23:33:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-mgr", 2024-09-10T23:33:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQAA1+Bmzvm4BRAAEQkP5c+PsfeGG+gAjU73Zg==", 2024-09-10T23:33:24.267 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-10T23:33:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-osd", 2024-09-10T23:33:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQAA1+BmgwK5BRAAMQCtwxVBIQ2Z6HRmXXM3rg==", 2024-09-10T23:33:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-osd" 2024-09-10T23:33:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-rbd", 2024-09-10T23:33:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQAA1+Bm7Aq5BRAAPqHsMYw9fWAcBcNQ8xdhKA==", 2024-09-10T23:33:24.268 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-10T23:33:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-10T23:33:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQAA1+BmfhO5BRAAvT0zRY7JB4izUSOu6l7cYQ==", 2024-09-10T23:33:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-10T23:33:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.bootstrap-rgw", 2024-09-10T23:33:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQAA1+BmUhy5BRAA77YJD7KCTTPHGDM0BrbyoQ==", 2024-09-10T23:33:24.269 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-10T23:33:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.x", 2024-09-10T23:33:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/EPVKxAA/Myv8Zjf7BejbzlkxyCC1w==", 2024-09-10T23:33:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:33:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:33:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:24.270 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.y", 2024-09-10T23:33:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmhHvtJxAACe+79nYhv2ODwtmNhA45FA==", 2024-09-10T23:33:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:33:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:33:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:24.271 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:33:24.271 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:33:24.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.252+0000 7fdcc4a00640 1 -- 172.21.15.94:0/900484860 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdca00725e0 msgr2=0x7fdca0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:24.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.252+0000 7fdcc4a00640 1 --2- 172.21.15.94:0/900484860 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdca00725e0 0x7fdca0074aa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0af000 tx=0x7fdcac01b000 comp rx=0 tx=0).stop 2024-09-10T23:33:24.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.252+0000 7fdcc4a00640 1 -- 172.21.15.94:0/900484860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00b2bb0 msgr2=0x7fdcc014c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:24.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.252+0000 7fdcc4a00640 1 --2- 172.21.15.94:0/900484860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00b2bb0 0x7fdcc014c7c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdcb00058d0 tx=0x7fdcb000dde0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.252+0000 7fdcc4a00640 1 -- 172.21.15.94:0/900484860 shutdown_connections 2024-09-10T23:33:24.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.252+0000 7fdcc4a00640 1 --2- 172.21.15.94:0/900484860 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdca00725e0 0x7fdca0074aa0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.252+0000 7fdcc4a00640 1 --2- 172.21.15.94:0/900484860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdcc015f5c0 0x7fdcc00cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.252+0000 7fdcc4a00640 1 --2- 172.21.15.94:0/900484860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdcc00d05e0 0x7fdcc0150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.252+0000 7fdcc4a00640 1 --2- 172.21.15.94:0/900484860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcc00b2bb0 0x7fdcc014c7c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.252+0000 7fdcc4a00640 1 -- 172.21.15.94:0/900484860 >> 172.21.15.94:0/900484860 conn(0x7fdcc00d1c60 msgr2=0x7fdcc0152f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:24.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.252+0000 7fdcc4a00640 1 -- 172.21.15.94:0/900484860 shutdown_connections 2024-09-10T23:33:24.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.252+0000 7fdcc4a00640 1 -- 172.21.15.94:0/900484860 wait complete. 2024-09-10T23:33:24.277 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-10T23:33:24.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.416+0000 7f69f7800640 1 -- 172.21.15.94:0/2079612343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f015f560 msgr2=0x7f69f00cfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.416+0000 7f69f7800640 1 --2- 172.21.15.94:0/2079612343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f015f560 0x7f69f00cfa90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009870 tx=0x7f69d801c820 comp rx=0 tx=0).stop 2024-09-10T23:33:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.416+0000 7f69f7800640 1 -- 172.21.15.94:0/2079612343 shutdown_connections 2024-09-10T23:33:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.416+0000 7f69f7800640 1 --2- 172.21.15.94:0/2079612343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f015f560 0x7f69f00cfa90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.416+0000 7f69f7800640 1 --2- 172.21.15.94:0/2079612343 >> v2:172.21.15.181:3300/0 conn(0x7f69f015f180 0x7f69f00cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.416+0000 7f69f7800640 1 -- 172.21.15.94:0/2079612343 >> 172.21.15.94:0/2079612343 conn(0x7f69f00d1cb0 msgr2=0x7f69f00d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.416+0000 7f69f7800640 1 -- 172.21.15.94:0/2079612343 shutdown_connections 2024-09-10T23:33:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.416+0000 7f69f7800640 1 -- 172.21.15.94:0/2079612343 wait complete. 2024-09-10T23:33:24.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.416+0000 7f69f7800640 1 Processor -- start 2024-09-10T23:33:24.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.416+0000 7f69f7800640 1 -- start start 2024-09-10T23:33:24.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f7800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f00cffd0 0x7f69f006f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:24.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f7800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f015f180 0x7f69f01503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:24.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f7800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f015f560 0x7f69f006eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:24.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f7800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69f01693a0 con 0x7f69f00cffd0 2024-09-10T23:33:24.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f7800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0169220 con 0x7f69f015f180 2024-09-10T23:33:24.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f7800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f69f0169520 con 0x7f69f015f560 2024-09-10T23:33:24.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f4e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f00cffd0 0x7f69f006f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:24.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f4e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f00cffd0 0x7f69f006f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60088/0 (socket says 172.21.15.94:60088) 2024-09-10T23:33:24.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f4e00640 1 -- 172.21.15.94:0/3027129826 learned_addr learned my addr 172.21.15.94:0/3027129826 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:24.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69e7e00640 1 --2- 172.21.15.94:0/3027129826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f015f180 0x7f69f01503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:24.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f5800640 1 --2- 172.21.15.94:0/3027129826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f015f560 0x7f69f006eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:24.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f4e00640 1 -- 172.21.15.94:0/3027129826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f015f560 msgr2=0x7f69f006eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:24.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f4e00640 1 --2- 172.21.15.94:0/3027129826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f015f560 0x7f69f006eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f4e00640 1 -- 172.21.15.94:0/3027129826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f015f180 msgr2=0x7f69f01503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:24.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f4e00640 1 --2- 172.21.15.94:0/3027129826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f015f180 0x7f69f01503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f4e00640 1 -- 172.21.15.94:0/3027129826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f0150910 con 0x7f69f00cffd0 2024-09-10T23:33:24.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f5800640 1 --2- 172.21.15.94:0/3027129826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f015f560 0x7f69f006eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:24.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f4e00640 1 --2- 172.21.15.94:0/3027129826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f00cffd0 0x7f69f006f950 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f69e000e9f0 tx=0x7f69e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:24.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69ed600640 1 -- 172.21.15.94:0/3027129826 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e000cd80 con 0x7f69f00cffd0 2024-09-10T23:33:24.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69ed600640 1 -- 172.21.15.94:0/3027129826 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69e0010430 con 0x7f69f00cffd0 2024-09-10T23:33:24.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f7800640 1 -- 172.21.15.94:0/3027129826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f01f8bc0 con 0x7f69f00cffd0 2024-09-10T23:33:24.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69ed600640 1 -- 172.21.15.94:0/3027129826 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0010860 con 0x7f69f00cffd0 2024-09-10T23:33:24.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f7800640 1 -- 172.21.15.94:0/3027129826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f006e3e0 con 0x7f69f00cffd0 2024-09-10T23:33:24.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.420+0000 7f69f7800640 1 -- 172.21.15.94:0/3027129826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f00cd0a0 con 0x7f69f00cffd0 2024-09-10T23:33:24.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.424+0000 7f69ed600640 1 -- 172.21.15.94:0/3027129826 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f69e0010a00 con 0x7f69f00cffd0 2024-09-10T23:33:24.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.424+0000 7f69ed600640 1 --2- 172.21.15.94:0/3027129826 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f69cc0725e0 0x7f69cc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:24.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.424+0000 7f69ed600640 1 -- 172.21.15.94:0/3027129826 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f69e0093c40 con 0x7f69f00cffd0 2024-09-10T23:33:24.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.424+0000 7f69e7e00640 1 --2- 172.21.15.94:0/3027129826 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f69cc0725e0 0x7f69cc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:24.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.424+0000 7f69e7e00640 1 --2- 172.21.15.94:0/3027129826 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f69cc0725e0 0x7f69cc074aa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f69d80af000 tx=0x7f69d801b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:24.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.428+0000 7f69ed600640 1 -- 172.21.15.94:0/3027129826 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f69e005f5e0 con 0x7f69f00cffd0 2024-09-10T23:33:24.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.652+0000 7f69f7800640 1 -- 172.21.15.94:0/3027129826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f69f00ce840 con 0x7f69f00cffd0 2024-09-10T23:33:24.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.652+0000 7f69ed600640 1 -- 172.21.15.94:0/3027129826 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v22) ==== 90+0+0 (secure 0 0 0) 0x7f69e005eee0 con 0x7f69f00cffd0 2024-09-10T23:33:24.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.656+0000 7f69f7800640 1 -- 172.21.15.94:0/3027129826 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f69cc0725e0 msgr2=0x7f69cc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:24.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.656+0000 7f69f7800640 1 --2- 172.21.15.94:0/3027129826 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f69cc0725e0 0x7f69cc074aa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f69d80af000 tx=0x7f69d801b060 comp rx=0 tx=0).stop 2024-09-10T23:33:24.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.656+0000 7f69f7800640 1 -- 172.21.15.94:0/3027129826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f00cffd0 msgr2=0x7f69f006f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:24.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.656+0000 7f69f7800640 1 --2- 172.21.15.94:0/3027129826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f00cffd0 0x7f69f006f950 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f69e000e9f0 tx=0x7f69e000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.656+0000 7f69f7800640 1 -- 172.21.15.94:0/3027129826 shutdown_connections 2024-09-10T23:33:24.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.656+0000 7f69f7800640 1 --2- 172.21.15.94:0/3027129826 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f69cc0725e0 0x7f69cc074aa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.656+0000 7f69f7800640 1 --2- 172.21.15.94:0/3027129826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f015f180 0x7f69f01503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.656+0000 7f69f7800640 1 --2- 172.21.15.94:0/3027129826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f015f560 0x7f69f006eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.656+0000 7f69f7800640 1 --2- 172.21.15.94:0/3027129826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f00cffd0 0x7f69f006f950 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.656+0000 7f69f7800640 1 -- 172.21.15.94:0/3027129826 >> 172.21.15.94:0/3027129826 conn(0x7f69f00d1cb0 msgr2=0x7f69f00b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:24.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.656+0000 7f69f7800640 1 -- 172.21.15.94:0/3027129826 shutdown_connections 2024-09-10T23:33:24.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.656+0000 7f69f7800640 1 -- 172.21.15.94:0/3027129826 wait complete. 2024-09-10T23:33:24.680 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-10T23:33:24.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.816+0000 7f4cb1600640 1 -- 172.21.15.94:0/2840600457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cac0cea40 msgr2=0x7f4cac0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:24.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.816+0000 7f4cb1600640 1 --2- 172.21.15.94:0/2840600457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cac0cea40 0x7f4cac0b31a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009c80 tx=0x7f4ca401f540 comp rx=0 tx=0).stop 2024-09-10T23:33:24.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.816+0000 7f4cb1600640 1 -- 172.21.15.94:0/2840600457 shutdown_connections 2024-09-10T23:33:24.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.816+0000 7f4cb1600640 1 --2- 172.21.15.94:0/2840600457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cac0cea40 0x7f4cac0b31a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.816+0000 7f4cb1600640 1 --2- 172.21.15.94:0/2840600457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4cac15adc0 0x7f4cac0cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.816+0000 7f4cb1600640 1 -- 172.21.15.94:0/2840600457 >> 172.21.15.94:0/2840600457 conn(0x7f4cac074240 msgr2=0x7f4cac074620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:24.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.816+0000 7f4cb1600640 1 -- 172.21.15.94:0/2840600457 shutdown_connections 2024-09-10T23:33:24.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.816+0000 7f4cb1600640 1 -- 172.21.15.94:0/2840600457 wait complete. 2024-09-10T23:33:24.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4cb1600640 1 Processor -- start 2024-09-10T23:33:24.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4cb1600640 1 -- start start 2024-09-10T23:33:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4cb1600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cac0cab60 0x7f4cac0d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4cb1600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cac0cea40 0x7f4cac1ef990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4cb1600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4cac15adc0 0x7f4cac1f00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4cb1600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac0cd530 con 0x7f4cac0cab60 2024-09-10T23:33:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4cb1600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac0cd3b0 con 0x7f4cac0cea40 2024-09-10T23:33:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4cb1600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f4cac0cd6b0 con 0x7f4cac15adc0 2024-09-10T23:33:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4caaa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cac0cab60 0x7f4cac0d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:24.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4caaa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cac0cab60 0x7f4cac0d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60112/0 (socket says 172.21.15.94:60112) 2024-09-10T23:33:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4caaa00640 1 -- 172.21.15.94:0/2158675684 learned_addr learned my addr 172.21.15.94:0/2158675684 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4ca3e00640 1 --2- 172.21.15.94:0/2158675684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cac0cea40 0x7f4cac1ef990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4cab400640 1 --2- 172.21.15.94:0/2158675684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4cac15adc0 0x7f4cac1f00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4caaa00640 1 -- 172.21.15.94:0/2158675684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4cac15adc0 msgr2=0x7f4cac1f00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4caaa00640 1 --2- 172.21.15.94:0/2158675684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4cac15adc0 0x7f4cac1f00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4caaa00640 1 -- 172.21.15.94:0/2158675684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cac0cea40 msgr2=0x7f4cac1ef990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4caaa00640 1 --2- 172.21.15.94:0/2158675684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cac0cea40 0x7f4cac1ef990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4caaa00640 1 -- 172.21.15.94:0/2158675684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac1f4170 con 0x7f4cac0cab60 2024-09-10T23:33:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4caaa00640 1 --2- 172.21.15.94:0/2158675684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cac0cab60 0x7f4cac0d0b30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00d750 tx=0x7f4c9c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4ca2a00640 1 -- 172.21.15.94:0/2158675684 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c017070 con 0x7f4cac0cab60 2024-09-10T23:33:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4ca2a00640 1 -- 172.21.15.94:0/2158675684 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c9c009d20 con 0x7f4cac0cab60 2024-09-10T23:33:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4ca2a00640 1 -- 172.21.15.94:0/2158675684 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c005280 con 0x7f4cac0cab60 2024-09-10T23:33:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.820+0000 7f4cb1600640 1 -- 172.21.15.94:0/2158675684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cac1f4450 con 0x7f4cac0cab60 2024-09-10T23:33:24.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.824+0000 7f4cb1600640 1 -- 172.21.15.94:0/2158675684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cac159d10 con 0x7f4cac0cab60 2024-09-10T23:33:24.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.824+0000 7f4cb1600640 1 -- 172.21.15.94:0/2158675684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cac0b3980 con 0x7f4cac0cab60 2024-09-10T23:33:24.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.824+0000 7f4ca2a00640 1 -- 172.21.15.94:0/2158675684 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4c9c005420 con 0x7f4cac0cab60 2024-09-10T23:33:24.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.824+0000 7f4ca2a00640 1 --2- 172.21.15.94:0/2158675684 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4c880725e0 0x7f4c88074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:24.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.824+0000 7f4ca2a00640 1 -- 172.21.15.94:0/2158675684 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f4c9c092f30 con 0x7f4cac0cab60 2024-09-10T23:33:24.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.824+0000 7f4ca3e00640 1 --2- 172.21.15.94:0/2158675684 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4c880725e0 0x7f4c88074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:24.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.824+0000 7f4ca3e00640 1 --2- 172.21.15.94:0/2158675684 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4c880725e0 0x7f4c88074aa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4c94007fd0 tx=0x7f4c94009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:24.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:24.828+0000 7f4ca2a00640 1 -- 172.21.15.94:0/2158675684 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4c9c00de60 con 0x7f4cac0cab60 2024-09-10T23:33:25.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.052+0000 7f4cb1600640 1 -- 172.21.15.94:0/2158675684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f4cac0ce4b0 con 0x7f4cac0cab60 2024-09-10T23:33:25.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.056+0000 7f4ca2a00640 1 -- 172.21.15.94:0/2158675684 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v23) ==== 90+0+0 (secure 0 0 0) 0x7f4c9c067070 con 0x7f4cac0cab60 2024-09-10T23:33:25.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.060+0000 7f4cb1600640 1 -- 172.21.15.94:0/2158675684 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4c880725e0 msgr2=0x7f4c88074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:25.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.060+0000 7f4cb1600640 1 --2- 172.21.15.94:0/2158675684 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4c880725e0 0x7f4c88074aa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4c94007fd0 tx=0x7f4c94009040 comp rx=0 tx=0).stop 2024-09-10T23:33:25.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.060+0000 7f4cb1600640 1 -- 172.21.15.94:0/2158675684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cac0cab60 msgr2=0x7f4cac0d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:25.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.060+0000 7f4cb1600640 1 --2- 172.21.15.94:0/2158675684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cac0cab60 0x7f4cac0d0b30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00d750 tx=0x7f4c9c00dc10 comp rx=0 tx=0).stop 2024-09-10T23:33:25.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.060+0000 7f4cb1600640 1 -- 172.21.15.94:0/2158675684 shutdown_connections 2024-09-10T23:33:25.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.060+0000 7f4cb1600640 1 --2- 172.21.15.94:0/2158675684 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4c880725e0 0x7f4c88074aa0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.060+0000 7f4cb1600640 1 --2- 172.21.15.94:0/2158675684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cac0cea40 0x7f4cac1ef990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.060+0000 7f4cb1600640 1 --2- 172.21.15.94:0/2158675684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4cac15adc0 0x7f4cac1f00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.060+0000 7f4cb1600640 1 --2- 172.21.15.94:0/2158675684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cac0cab60 0x7f4cac0d0b30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.060+0000 7f4cb1600640 1 -- 172.21.15.94:0/2158675684 >> 172.21.15.94:0/2158675684 conn(0x7f4cac074240 msgr2=0x7f4cac06eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:25.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.060+0000 7f4cb1600640 1 -- 172.21.15.94:0/2158675684 shutdown_connections 2024-09-10T23:33:25.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.060+0000 7f4cb1600640 1 -- 172.21.15.94:0/2158675684 wait complete. 2024-09-10T23:33:25.082 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-10T23:33:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.220+0000 7feadbe00640 1 -- 172.21.15.94:0/3943375615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fead40ce1e0 msgr2=0x7fead40cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.220+0000 7feadbe00640 1 --2- 172.21.15.94:0/3943375615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fead40ce1e0 0x7fead40cd710 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7feac0009900 tx=0x7feac001c850 comp rx=0 tx=0).stop 2024-09-10T23:33:25.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.220+0000 7feadbe00640 1 -- 172.21.15.94:0/3943375615 shutdown_connections 2024-09-10T23:33:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.220+0000 7feadbe00640 1 --2- 172.21.15.94:0/3943375615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead40b1100 0x7fead40b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.220+0000 7feadbe00640 1 --2- 172.21.15.94:0/3943375615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fead40ce1e0 0x7fead40cd710 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.220+0000 7feadbe00640 1 -- 172.21.15.94:0/3943375615 >> 172.21.15.94:0/3943375615 conn(0x7fead40d1c60 msgr2=0x7fead40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.220+0000 7feadbe00640 1 -- 172.21.15.94:0/3943375615 shutdown_connections 2024-09-10T23:33:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.220+0000 7feadbe00640 1 -- 172.21.15.94:0/3943375615 wait complete. 2024-09-10T23:33:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7feadbe00640 1 Processor -- start 2024-09-10T23:33:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7feadbe00640 1 -- start start 2024-09-10T23:33:25.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7feadbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fead40b1100 0x7fead415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:25.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7feadbe00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead40ce1e0 0x7fead40af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:25.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7feadbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead40ce890 0x7fead40afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:25.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7feadbe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fead40d0b30 con 0x7fead40b1100 2024-09-10T23:33:25.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7feadbe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fead40d09b0 con 0x7fead40ce890 2024-09-10T23:33:25.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7feadbe00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fead4169110 con 0x7fead40ce1e0 2024-09-10T23:33:25.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7fead9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fead40b1100 0x7fead415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:25.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7fead9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fead40b1100 0x7fead415cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60116/0 (socket says 172.21.15.94:60116) 2024-09-10T23:33:25.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7fead9400640 1 -- 172.21.15.94:0/2187055651 learned_addr learned my addr 172.21.15.94:0/2187055651 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:25.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7fead8a00640 1 --2- 172.21.15.94:0/2187055651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead40ce1e0 0x7fead40af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:25.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7fead9e00640 1 --2- 172.21.15.94:0/2187055651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead40ce890 0x7fead40afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:25.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7fead9400640 1 -- 172.21.15.94:0/2187055651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead40ce1e0 msgr2=0x7fead40af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:25.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7fead9400640 1 --2- 172.21.15.94:0/2187055651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead40ce1e0 0x7fead40af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7fead9400640 1 -- 172.21.15.94:0/2187055651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead40ce890 msgr2=0x7fead40afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:25.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7fead9400640 1 --2- 172.21.15.94:0/2187055651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead40ce890 0x7fead40afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7fead9400640 1 -- 172.21.15.94:0/2187055651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead41f75d0 con 0x7fead40b1100 2024-09-10T23:33:25.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7fead9400640 1 --2- 172.21.15.94:0/2187055651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fead40b1100 0x7fead415cc30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feac800c3c0 tx=0x7feac800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:25.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7fead1600640 1 -- 172.21.15.94:0/2187055651 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8017070 con 0x7fead40b1100 2024-09-10T23:33:25.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7fead1600640 1 -- 172.21.15.94:0/2187055651 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feac80053c0 con 0x7fead40b1100 2024-09-10T23:33:25.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7fead1600640 1 -- 172.21.15.94:0/2187055651 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8003860 con 0x7fead40b1100 2024-09-10T23:33:25.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7feadbe00640 1 -- 172.21.15.94:0/2187055651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead40c8dd0 con 0x7fead40b1100 2024-09-10T23:33:25.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7feadbe00640 1 -- 172.21.15.94:0/2187055651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead40c9070 con 0x7fead40b1100 2024-09-10T23:33:25.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.224+0000 7feadbe00640 1 -- 172.21.15.94:0/2187055651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead40b1590 con 0x7fead40b1100 2024-09-10T23:33:25.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.228+0000 7fead1600640 1 -- 172.21.15.94:0/2187055651 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7feac8003a00 con 0x7fead40b1100 2024-09-10T23:33:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.228+0000 7fead1600640 1 --2- 172.21.15.94:0/2187055651 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7feab40725e0 0x7feab4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.228+0000 7fead1600640 1 -- 172.21.15.94:0/2187055651 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7feac8092600 con 0x7fead40b1100 2024-09-10T23:33:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.228+0000 7fead8a00640 1 --2- 172.21.15.94:0/2187055651 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7feab40725e0 0x7feab4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.228+0000 7fead8a00640 1 --2- 172.21.15.94:0/2187055651 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7feab40725e0 0x7feab4074aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feac00af000 tx=0x7feac001b090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:25.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.232+0000 7fead1600640 1 -- 172.21.15.94:0/2187055651 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fead41f75d0 con 0x7fead40b1100 2024-09-10T23:33:25.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.452+0000 7feadbe00640 1 -- 172.21.15.94:0/2187055651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fead40ceff0 con 0x7fead40b1100 2024-09-10T23:33:25.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.456+0000 7fead1600640 1 -- 172.21.15.94:0/2187055651 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v24) ==== 90+0+0 (secure 0 0 0) 0x7feac805df90 con 0x7fead40b1100 2024-09-10T23:33:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.460+0000 7feadbe00640 1 -- 172.21.15.94:0/2187055651 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7feab40725e0 msgr2=0x7feab4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.460+0000 7feadbe00640 1 --2- 172.21.15.94:0/2187055651 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7feab40725e0 0x7feab4074aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feac00af000 tx=0x7feac001b090 comp rx=0 tx=0).stop 2024-09-10T23:33:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.460+0000 7feadbe00640 1 -- 172.21.15.94:0/2187055651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fead40b1100 msgr2=0x7fead415cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.460+0000 7feadbe00640 1 --2- 172.21.15.94:0/2187055651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fead40b1100 0x7fead415cc30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feac800c3c0 tx=0x7feac800c880 comp rx=0 tx=0).stop 2024-09-10T23:33:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.460+0000 7feadbe00640 1 -- 172.21.15.94:0/2187055651 shutdown_connections 2024-09-10T23:33:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.460+0000 7feadbe00640 1 --2- 172.21.15.94:0/2187055651 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7feab40725e0 0x7feab4074aa0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.460+0000 7feadbe00640 1 --2- 172.21.15.94:0/2187055651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead40ce890 0x7fead40afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.460+0000 7feadbe00640 1 --2- 172.21.15.94:0/2187055651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead40ce1e0 0x7fead40af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.460+0000 7feadbe00640 1 --2- 172.21.15.94:0/2187055651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fead40b1100 0x7fead415cc30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.460+0000 7feadbe00640 1 -- 172.21.15.94:0/2187055651 >> 172.21.15.94:0/2187055651 conn(0x7fead40d1c60 msgr2=0x7fead40707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.460+0000 7feadbe00640 1 -- 172.21.15.94:0/2187055651 shutdown_connections 2024-09-10T23:33:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.460+0000 7feadbe00640 1 -- 172.21.15.94:0/2187055651 wait complete. 2024-09-10T23:33:25.483 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-10T23:33:25.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.620+0000 7f0200a00640 1 -- 172.21.15.94:0/4251253875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01e0003f80 msgr2=0x7f01e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:25.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.620+0000 7f0200a00640 1 --2- 172.21.15.94:0/4251253875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01e0003f80 0x7f01e0004810 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009b30 tx=0x7f01e801d230 comp rx=0 tx=0).stop 2024-09-10T23:33:25.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.620+0000 7f01fa800640 1 -- 172.21.15.94:0/4251253875 reap_dead start 2024-09-10T23:33:25.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.620+0000 7f0200a00640 1 -- 172.21.15.94:0/4251253875 shutdown_connections 2024-09-10T23:33:25.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.620+0000 7f0200a00640 1 -- 172.21.15.94:0/4251253875 >> 172.21.15.94:0/4251253875 conn(0x7f01fc0d1cb0 msgr2=0x7f01fc0d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:25.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.620+0000 7f0200a00640 1 -- 172.21.15.94:0/4251253875 shutdown_connections 2024-09-10T23:33:25.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.620+0000 7f0200a00640 1 -- 172.21.15.94:0/4251253875 wait complete. 2024-09-10T23:33:25.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f0200a00640 1 Processor -- start 2024-09-10T23:33:25.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f0200a00640 1 -- start start 2024-09-10T23:33:25.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f0200a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01e0003f80 0x7f01fc1f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:25.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f0200a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01fc1f3220 0x7f01fc0b69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:25.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f0200a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01fc071060 0x7f01fc071510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:25.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f0200a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc0cd430 con 0x7f01fc071060 2024-09-10T23:33:25.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f0200a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc0cd2b0 con 0x7f01fc1f3220 2024-09-10T23:33:25.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f0200a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f01fc0cd5b0 con 0x7f01e0003f80 2024-09-10T23:33:25.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f01fa800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01fc071060 0x7f01fc071510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:25.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f01fa800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01fc071060 0x7f01fc071510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60126/0 (socket says 172.21.15.94:60126) 2024-09-10T23:33:25.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f01fa800640 1 -- 172.21.15.94:0/1900511545 learned_addr learned my addr 172.21.15.94:0/1900511545 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:25.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f01f9400640 1 --2- 172.21.15.94:0/1900511545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01fc1f3220 0x7f01fc0b69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:25.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f01f9e00640 1 --2- 172.21.15.94:0/1900511545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01e0003f80 0x7f01fc1f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:25.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f01fa800640 1 -- 172.21.15.94:0/1900511545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01e0003f80 msgr2=0x7f01fc1f3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:25.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f01fa800640 1 --2- 172.21.15.94:0/1900511545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01e0003f80 0x7f01fc1f3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f01fa800640 1 -- 172.21.15.94:0/1900511545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01fc1f3220 msgr2=0x7f01fc0b69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:25.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f01fa800640 1 --2- 172.21.15.94:0/1900511545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01fc1f3220 0x7f01fc0b69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f01fa800640 1 -- 172.21.15.94:0/1900511545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01fc0c57e0 con 0x7f01fc071060 2024-09-10T23:33:25.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f01f9e00640 1 --2- 172.21.15.94:0/1900511545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01e0003f80 0x7f01fc1f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:25.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f01fa800640 1 --2- 172.21.15.94:0/1900511545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01fc071060 0x7f01fc071510 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f01f4006fd0 tx=0x7f01f4006420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:25.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f01f2000640 1 -- 172.21.15.94:0/1900511545 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4015020 con 0x7f01fc071060 2024-09-10T23:33:25.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f01f2000640 1 -- 172.21.15.94:0/1900511545 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01f4010a60 con 0x7f01fc071060 2024-09-10T23:33:25.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f01f2000640 1 -- 172.21.15.94:0/1900511545 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4011460 con 0x7f01fc071060 2024-09-10T23:33:25.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f0200a00640 1 -- 172.21.15.94:0/1900511545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01fc0c5aa0 con 0x7f01fc071060 2024-09-10T23:33:25.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f0200a00640 1 -- 172.21.15.94:0/1900511545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01fc0c6060 con 0x7f01fc071060 2024-09-10T23:33:25.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.624+0000 7f0200a00640 1 -- 172.21.15.94:0/1900511545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01fc0b16f0 con 0x7f01fc071060 2024-09-10T23:33:25.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.628+0000 7f01f2000640 1 -- 172.21.15.94:0/1900511545 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f01f4011600 con 0x7f01fc071060 2024-09-10T23:33:25.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.628+0000 7f01f2000640 1 --2- 172.21.15.94:0/1900511545 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f01dc072610 0x7f01dc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:25.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.628+0000 7f01f2000640 1 -- 172.21.15.94:0/1900511545 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f01f4093930 con 0x7f01fc071060 2024-09-10T23:33:25.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.628+0000 7f01f9e00640 1 --2- 172.21.15.94:0/1900511545 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f01dc072610 0x7f01dc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:25.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.628+0000 7f01f9e00640 1 --2- 172.21.15.94:0/1900511545 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f01dc072610 0x7f01dc074ad0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f01ec001bf0 tx=0x7f01ec001b80 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:25.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.632+0000 7f01f2000640 1 -- 172.21.15.94:0/1900511545 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f01f401f030 con 0x7f01fc071060 2024-09-10T23:33:25.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.856+0000 7f0200a00640 1 -- 172.21.15.94:0/1900511545 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f01fc0d11c0 con 0x7f01fc071060 2024-09-10T23:33:25.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.860+0000 7f01f2000640 1 -- 172.21.15.94:0/1900511545 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v25) ==== 90+0+0 (secure 0 0 0) 0x7f01f405f350 con 0x7f01fc071060 2024-09-10T23:33:25.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.864+0000 7f0200a00640 1 -- 172.21.15.94:0/1900511545 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f01dc072610 msgr2=0x7f01dc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:25.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.864+0000 7f0200a00640 1 --2- 172.21.15.94:0/1900511545 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f01dc072610 0x7f01dc074ad0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f01ec001bf0 tx=0x7f01ec001b80 comp rx=0 tx=0).stop 2024-09-10T23:33:25.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.864+0000 7f0200a00640 1 -- 172.21.15.94:0/1900511545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01fc071060 msgr2=0x7f01fc071510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:25.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.864+0000 7f0200a00640 1 --2- 172.21.15.94:0/1900511545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01fc071060 0x7f01fc071510 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f01f4006fd0 tx=0x7f01f4006420 comp rx=0 tx=0).stop 2024-09-10T23:33:25.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.864+0000 7f0200a00640 1 -- 172.21.15.94:0/1900511545 shutdown_connections 2024-09-10T23:33:25.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.864+0000 7f0200a00640 1 --2- 172.21.15.94:0/1900511545 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f01dc072610 0x7f01dc074ad0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.864+0000 7f0200a00640 1 --2- 172.21.15.94:0/1900511545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01fc1f3220 0x7f01fc0b69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.864+0000 7f0200a00640 1 --2- 172.21.15.94:0/1900511545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01fc071060 0x7f01fc071510 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.864+0000 7f0200a00640 1 --2- 172.21.15.94:0/1900511545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01e0003f80 0x7f01fc1f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:25.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.864+0000 7f0200a00640 1 -- 172.21.15.94:0/1900511545 >> 172.21.15.94:0/1900511545 conn(0x7f01fc0d1cb0 msgr2=0x7f01fc0cbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:25.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.864+0000 7f0200a00640 1 -- 172.21.15.94:0/1900511545 shutdown_connections 2024-09-10T23:33:25.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:25.864+0000 7f0200a00640 1 -- 172.21.15.94:0/1900511545 wait complete. 2024-09-10T23:33:25.886 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-10T23:33:26.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.024+0000 7f8adfc00640 1 -- 172.21.15.94:0/2606605941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8abc003a10 msgr2=0x7f8abc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:26.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.024+0000 7f8adfc00640 1 --2- 172.21.15.94:0/2606605941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8abc003a10 0x7f8abc003e30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009b60 tx=0x7f8ac801d220 comp rx=0 tx=0).stop 2024-09-10T23:33:26.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.024+0000 7f8addc00640 1 -- 172.21.15.94:0/2606605941 reap_dead start 2024-09-10T23:33:26.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.024+0000 7f8adfc00640 1 -- 172.21.15.94:0/2606605941 shutdown_connections 2024-09-10T23:33:26.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.024+0000 7f8adfc00640 1 -- 172.21.15.94:0/2606605941 >> 172.21.15.94:0/2606605941 conn(0x7f8ad80d3eb0 msgr2=0x7f8ad80d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:26.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.024+0000 7f8adfc00640 1 -- 172.21.15.94:0/2606605941 shutdown_connections 2024-09-10T23:33:26.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.024+0000 7f8adfc00640 1 -- 172.21.15.94:0/2606605941 wait complete. 2024-09-10T23:33:26.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8adfc00640 1 Processor -- start 2024-09-10T23:33:26.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8adfc00640 1 -- start start 2024-09-10T23:33:26.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8adfc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8abc003a10 0x7f8ad8149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:26.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8adfc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ad814a770 0x7f8ad815b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:26.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8adfc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ad815b5b0 0x7f8ad815ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:26.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8adfc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad80b85f0 con 0x7f8abc003a10 2024-09-10T23:33:26.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8adfc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad80b8470 con 0x7f8ad814a770 2024-09-10T23:33:26.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8adfc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad80b8770 con 0x7f8ad815b5b0 2024-09-10T23:33:26.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8add200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8abc003a10 0x7f8ad8149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:26.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8add200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8abc003a10 0x7f8ad8149f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60144/0 (socket says 172.21.15.94:60144) 2024-09-10T23:33:26.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8add200640 1 -- 172.21.15.94:0/2805297108 learned_addr learned my addr 172.21.15.94:0/2805297108 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:26.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8ad7e00640 1 --2- 172.21.15.94:0/2805297108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ad814a770 0x7f8ad815b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:26.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8addc00640 1 --2- 172.21.15.94:0/2805297108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ad815b5b0 0x7f8ad815ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:26.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8add200640 1 -- 172.21.15.94:0/2805297108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ad815b5b0 msgr2=0x7f8ad815ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:26.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8add200640 1 --2- 172.21.15.94:0/2805297108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ad815b5b0 0x7f8ad815ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8add200640 1 -- 172.21.15.94:0/2805297108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ad814a770 msgr2=0x7f8ad815b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:26.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8add200640 1 --2- 172.21.15.94:0/2805297108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ad814a770 0x7f8ad815b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8add200640 1 -- 172.21.15.94:0/2805297108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad80ce290 con 0x7f8abc003a10 2024-09-10T23:33:26.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8ad7e00640 1 --2- 172.21.15.94:0/2805297108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ad814a770 0x7f8ad815b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:26.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8addc00640 1 --2- 172.21.15.94:0/2805297108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ad815b5b0 0x7f8ad815ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:26.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8add200640 1 --2- 172.21.15.94:0/2805297108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8abc003a10 0x7f8ad8149f50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8ac801d050 tx=0x7f8ac8005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:26.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8ad4c00640 1 -- 172.21.15.94:0/2805297108 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac80b1040 con 0x7f8abc003a10 2024-09-10T23:33:26.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8ad4c00640 1 -- 172.21.15.94:0/2805297108 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ac80049c0 con 0x7f8abc003a10 2024-09-10T23:33:26.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8adfc00640 1 -- 172.21.15.94:0/2805297108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad815bf80 con 0x7f8abc003a10 2024-09-10T23:33:26.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8ad4c00640 1 -- 172.21.15.94:0/2805297108 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8004dd0 con 0x7f8abc003a10 2024-09-10T23:33:26.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.028+0000 7f8adfc00640 1 -- 172.21.15.94:0/2805297108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad80cdfb0 con 0x7f8abc003a10 2024-09-10T23:33:26.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.032+0000 7f8adfc00640 1 -- 172.21.15.94:0/2805297108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad80d49d0 con 0x7f8abc003a10 2024-09-10T23:33:26.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.032+0000 7f8ad4c00640 1 -- 172.21.15.94:0/2805297108 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8ac80b6410 con 0x7f8abc003a10 2024-09-10T23:33:26.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.032+0000 7f8ad4c00640 1 --2- 172.21.15.94:0/2805297108 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8ab80725e0 0x7f8ab8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:26.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.032+0000 7f8ad4c00640 1 -- 172.21.15.94:0/2805297108 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f8ac812dd60 con 0x7f8abc003a10 2024-09-10T23:33:26.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.032+0000 7f8ad7e00640 1 --2- 172.21.15.94:0/2805297108 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8ab80725e0 0x7f8ab8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:26.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.032+0000 7f8ad7e00640 1 --2- 172.21.15.94:0/2805297108 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8ab80725e0 0x7f8ab8074aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4001870 tx=0x7f8ac4001cc0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:26.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.036+0000 7f8ad4c00640 1 -- 172.21.15.94:0/2805297108 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8ad80ce290 con 0x7f8abc003a10 2024-09-10T23:33:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.264+0000 7f8adfc00640 1 -- 172.21.15.94:0/2805297108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f8ad815ceb0 con 0x7f8abc003a10 2024-09-10T23:33:26.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.268+0000 7f8ad4c00640 1 -- 172.21.15.94:0/2805297108 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v26) ==== 97+0+0 (secure 0 0 0) 0x7f8ac8016fc0 con 0x7f8abc003a10 2024-09-10T23:33:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.272+0000 7f8adfc00640 1 -- 172.21.15.94:0/2805297108 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8ab80725e0 msgr2=0x7f8ab8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.272+0000 7f8adfc00640 1 --2- 172.21.15.94:0/2805297108 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8ab80725e0 0x7f8ab8074aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4001870 tx=0x7f8ac4001cc0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.272+0000 7f8adfc00640 1 -- 172.21.15.94:0/2805297108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8abc003a10 msgr2=0x7f8ad8149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.272+0000 7f8adfc00640 1 --2- 172.21.15.94:0/2805297108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8abc003a10 0x7f8ad8149f50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8ac801d050 tx=0x7f8ac8005640 comp rx=0 tx=0).stop 2024-09-10T23:33:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.272+0000 7f8adfc00640 1 -- 172.21.15.94:0/2805297108 shutdown_connections 2024-09-10T23:33:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.272+0000 7f8adfc00640 1 --2- 172.21.15.94:0/2805297108 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8ab80725e0 0x7f8ab8074aa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.272+0000 7f8adfc00640 1 --2- 172.21.15.94:0/2805297108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ad814a770 0x7f8ad815b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.272+0000 7f8adfc00640 1 --2- 172.21.15.94:0/2805297108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ad815b5b0 0x7f8ad815ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.272+0000 7f8adfc00640 1 --2- 172.21.15.94:0/2805297108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8abc003a10 0x7f8ad8149f50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.272+0000 7f8adfc00640 1 -- 172.21.15.94:0/2805297108 >> 172.21.15.94:0/2805297108 conn(0x7f8ad80d3eb0 msgr2=0x7f8ad8159120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.272+0000 7f8adfc00640 1 -- 172.21.15.94:0/2805297108 shutdown_connections 2024-09-10T23:33:26.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.272+0000 7f8adfc00640 1 -- 172.21.15.94:0/2805297108 wait complete. 2024-09-10T23:33:26.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-10T23:33:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.432+0000 7fcd68200640 1 -- 172.21.15.94:0/2368705790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd48004010 msgr2=0x7fcd48004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.432+0000 7fcd68200640 1 --2- 172.21.15.94:0/2368705790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd48004010 0x7fcd48004810 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00c2e0 tx=0x7fcd4c01cf60 comp rx=0 tx=0).stop 2024-09-10T23:33:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.432+0000 7fcd66200640 1 -- 172.21.15.94:0/2368705790 reap_dead start 2024-09-10T23:33:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.432+0000 7fcd68200640 1 -- 172.21.15.94:0/2368705790 shutdown_connections 2024-09-10T23:33:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.432+0000 7fcd68200640 1 -- 172.21.15.94:0/2368705790 >> 172.21.15.94:0/2368705790 conn(0x7fcd600d1c60 msgr2=0x7fcd600d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.432+0000 7fcd68200640 1 -- 172.21.15.94:0/2368705790 shutdown_connections 2024-09-10T23:33:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.432+0000 7fcd68200640 1 -- 172.21.15.94:0/2368705790 wait complete. 2024-09-10T23:33:26.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd68200640 1 Processor -- start 2024-09-10T23:33:26.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd68200640 1 -- start start 2024-09-10T23:33:26.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd68200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd48004010 0x7fcd6014a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:26.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd68200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd600b6930 0x7fcd601592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:26.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd68200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd600b6db0 0x7fcd60159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:26.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd68200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcd601683b0 con 0x7fcd48004010 2024-09-10T23:33:26.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd68200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60168230 con 0x7fcd600b6930 2024-09-10T23:33:26.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd68200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fcd60168530 con 0x7fcd600b6db0 2024-09-10T23:33:26.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd65800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd48004010 0x7fcd6014a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:26.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd65800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd48004010 0x7fcd6014a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60164/0 (socket says 172.21.15.94:60164) 2024-09-10T23:33:26.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd65800640 1 -- 172.21.15.94:0/3549441453 learned_addr learned my addr 172.21.15.94:0/3549441453 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:26.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd64e00640 1 --2- 172.21.15.94:0/3549441453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd600b6930 0x7fcd601592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:26.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd66200640 1 --2- 172.21.15.94:0/3549441453 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd600b6db0 0x7fcd60159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:26.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd65800640 1 -- 172.21.15.94:0/3549441453 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd600b6db0 msgr2=0x7fcd60159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:26.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd65800640 1 --2- 172.21.15.94:0/3549441453 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd600b6db0 0x7fcd60159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd65800640 1 -- 172.21.15.94:0/3549441453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd600b6930 msgr2=0x7fcd601592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:26.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd65800640 1 --2- 172.21.15.94:0/3549441453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd600b6930 0x7fcd601592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd65800640 1 -- 172.21.15.94:0/3549441453 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd600b0da0 con 0x7fcd48004010 2024-09-10T23:33:26.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd65800640 1 --2- 172.21.15.94:0/3549441453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd48004010 0x7fcd6014a150 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcd54004f10 tx=0x7fcd54001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:26.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd5d600640 1 -- 172.21.15.94:0/3549441453 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54015020 con 0x7fcd48004010 2024-09-10T23:33:26.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd5d600640 1 -- 172.21.15.94:0/3549441453 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd54011960 con 0x7fcd48004010 2024-09-10T23:33:26.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd5d600640 1 -- 172.21.15.94:0/3549441453 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54011d90 con 0x7fcd48004010 2024-09-10T23:33:26.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd68200640 1 -- 172.21.15.94:0/3549441453 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd600cbf90 con 0x7fcd48004010 2024-09-10T23:33:26.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.436+0000 7fcd68200640 1 -- 172.21.15.94:0/3549441453 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd600b75f0 con 0x7fcd48004010 2024-09-10T23:33:26.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.440+0000 7fcd68200640 1 -- 172.21.15.94:0/3549441453 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd6006d7f0 con 0x7fcd48004010 2024-09-10T23:33:26.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.440+0000 7fcd5d600640 1 -- 172.21.15.94:0/3549441453 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcd5401a400 con 0x7fcd48004010 2024-09-10T23:33:26.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.440+0000 7fcd5d600640 1 --2- 172.21.15.94:0/3549441453 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcd400725e0 0x7fcd40074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:26.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.440+0000 7fcd5d600640 1 -- 172.21.15.94:0/3549441453 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fcd54093cd0 con 0x7fcd48004010 2024-09-10T23:33:26.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.440+0000 7fcd64e00640 1 --2- 172.21.15.94:0/3549441453 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcd400725e0 0x7fcd40074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:26.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.440+0000 7fcd64e00640 1 --2- 172.21.15.94:0/3549441453 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcd400725e0 0x7fcd40074aa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0ab000 tx=0x7fcd4c019da0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:26.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.444+0000 7fcd5d600640 1 -- 172.21.15.94:0/3549441453 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcd5401f020 con 0x7fcd48004010 2024-09-10T23:33:26.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.668+0000 7fcd68200640 1 -- 172.21.15.94:0/3549441453 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fcd600b1100 con 0x7fcd48004010 2024-09-10T23:33:26.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.668+0000 7fcd5d600640 1 -- 172.21.15.94:0/3549441453 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v27) ==== 90+0+0 (secure 0 0 0) 0x7fcd5405f6f0 con 0x7fcd48004010 2024-09-10T23:33:26.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.672+0000 7fcd68200640 1 -- 172.21.15.94:0/3549441453 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcd400725e0 msgr2=0x7fcd40074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:26.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.672+0000 7fcd68200640 1 --2- 172.21.15.94:0/3549441453 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcd400725e0 0x7fcd40074aa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0ab000 tx=0x7fcd4c019da0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.672+0000 7fcd68200640 1 -- 172.21.15.94:0/3549441453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd48004010 msgr2=0x7fcd6014a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:26.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.672+0000 7fcd68200640 1 --2- 172.21.15.94:0/3549441453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd48004010 0x7fcd6014a150 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcd54004f10 tx=0x7fcd54001640 comp rx=0 tx=0).stop 2024-09-10T23:33:26.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.672+0000 7fcd68200640 1 -- 172.21.15.94:0/3549441453 shutdown_connections 2024-09-10T23:33:26.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.672+0000 7fcd68200640 1 --2- 172.21.15.94:0/3549441453 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcd400725e0 0x7fcd40074aa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.672+0000 7fcd68200640 1 --2- 172.21.15.94:0/3549441453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd600b6930 0x7fcd601592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.672+0000 7fcd68200640 1 --2- 172.21.15.94:0/3549441453 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd600b6db0 0x7fcd60159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.672+0000 7fcd68200640 1 --2- 172.21.15.94:0/3549441453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd48004010 0x7fcd6014a150 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.672+0000 7fcd68200640 1 -- 172.21.15.94:0/3549441453 >> 172.21.15.94:0/3549441453 conn(0x7fcd600d1c60 msgr2=0x7fcd600717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:26.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.672+0000 7fcd68200640 1 -- 172.21.15.94:0/3549441453 shutdown_connections 2024-09-10T23:33:26.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.672+0000 7fcd68200640 1 -- 172.21.15.94:0/3549441453 wait complete. 2024-09-10T23:33:26.696 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T23:33:26.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.832+0000 7f690fa00640 1 -- 172.21.15.94:0/4075137839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69080d1690 msgr2=0x7f690815fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:26.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.832+0000 7f690fa00640 1 --2- 172.21.15.94:0/4075137839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69080d1690 0x7f690815fb50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f69080b6ca0 tx=0x7f690001ccd0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.832+0000 7f690fa00640 1 -- 172.21.15.94:0/4075137839 shutdown_connections 2024-09-10T23:33:26.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.832+0000 7f690fa00640 1 --2- 172.21.15.94:0/4075137839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69080d1690 0x7f690815fb50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.832+0000 7f690fa00640 1 --2- 172.21.15.94:0/4075137839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6908071a20 0x7f690815f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.832+0000 7f690fa00640 1 --2- 172.21.15.94:0/4075137839 >> v2:172.21.15.181:3300/0 conn(0x7f690806cc20 0x7f690806d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.832+0000 7f690fa00640 1 -- 172.21.15.94:0/4075137839 >> 172.21.15.94:0/4075137839 conn(0x7f69080d0b70 msgr2=0x7f69080d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:26.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.832+0000 7f690fa00640 1 -- 172.21.15.94:0/4075137839 shutdown_connections 2024-09-10T23:33:26.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.832+0000 7f690fa00640 1 -- 172.21.15.94:0/4075137839 wait complete. 2024-09-10T23:33:26.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.832+0000 7f690fa00640 1 Processor -- start 2024-09-10T23:33:26.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.832+0000 7f690fa00640 1 -- start start 2024-09-10T23:33:26.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f690fa00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f690806cc20 0x7f6908143e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:26.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f690fa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6908071a20 0x7f690815a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:26.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f690fa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69080d1690 0x7f690814e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:26.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f690fa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69080cffe0 con 0x7f6908071a20 2024-09-10T23:33:26.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f690fa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f69080cfe60 con 0x7f69080d1690 2024-09-10T23:33:26.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f690fa00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f69080d0160 con 0x7f690806cc20 2024-09-10T23:33:26.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f68ffe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6908071a20 0x7f690815a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:26.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f68ffe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6908071a20 0x7f690815a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60190/0 (socket says 172.21.15.94:60190) 2024-09-10T23:33:26.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f68ffe00640 1 -- 172.21.15.94:0/2680686516 learned_addr learned my addr 172.21.15.94:0/2680686516 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:26.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f690d000640 1 --2- 172.21.15.94:0/2680686516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f690806cc20 0x7f6908143e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:26.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f690da00640 1 --2- 172.21.15.94:0/2680686516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69080d1690 0x7f690814e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:26.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f68ffe00640 1 -- 172.21.15.94:0/2680686516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f690806cc20 msgr2=0x7f6908143e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:26.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f68ffe00640 1 --2- 172.21.15.94:0/2680686516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f690806cc20 0x7f6908143e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f68ffe00640 1 -- 172.21.15.94:0/2680686516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69080d1690 msgr2=0x7f690814e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:26.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f68ffe00640 1 --2- 172.21.15.94:0/2680686516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69080d1690 0x7f690814e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:26.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f68ffe00640 1 -- 172.21.15.94:0/2680686516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69081443d0 con 0x7f6908071a20 2024-09-10T23:33:26.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f690da00640 1 --2- 172.21.15.94:0/2680686516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69080d1690 0x7f690814e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:26.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f68ffe00640 1 --2- 172.21.15.94:0/2680686516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6908071a20 0x7f690815a550 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f68f000d980 tx=0x7f68f000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:26.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f6905600640 1 -- 172.21.15.94:0/2680686516 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0017070 con 0x7f6908071a20 2024-09-10T23:33:26.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f6905600640 1 -- 172.21.15.94:0/2680686516 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68f0009d60 con 0x7f6908071a20 2024-09-10T23:33:26.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f690fa00640 1 -- 172.21.15.94:0/2680686516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6908146c30 con 0x7f6908071a20 2024-09-10T23:33:26.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f6905600640 1 -- 172.21.15.94:0/2680686516 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0005280 con 0x7f6908071a20 2024-09-10T23:33:26.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f690fa00640 1 -- 172.21.15.94:0/2680686516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69081445f0 con 0x7f6908071a20 2024-09-10T23:33:26.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f690fa00640 1 -- 172.21.15.94:0/2680686516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f690806d000 con 0x7f6908071a20 2024-09-10T23:33:26.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.836+0000 7f6905600640 1 -- 172.21.15.94:0/2680686516 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f68f0005420 con 0x7f6908071a20 2024-09-10T23:33:26.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.840+0000 7f6905600640 1 --2- 172.21.15.94:0/2680686516 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f68e40725e0 0x7f68e4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:26.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.840+0000 7f6905600640 1 -- 172.21.15.94:0/2680686516 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f68f0093000 con 0x7f6908071a20 2024-09-10T23:33:26.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.840+0000 7f690d000640 1 --2- 172.21.15.94:0/2680686516 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f68e40725e0 0x7f68e4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:26.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.840+0000 7f690d000640 1 --2- 172.21.15.94:0/2680686516 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f68e40725e0 0x7f68e4074aa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f69081475f0 tx=0x7f68f800a2d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:26.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:26.844+0000 7f6905600640 1 -- 172.21.15.94:0/2680686516 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f68f005e3e0 con 0x7f6908071a20 2024-09-10T23:33:27.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.012+0000 7f690fa00640 1 -- 172.21.15.94:0/2680686516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6908148240 con 0x7f6908071a20 2024-09-10T23:33:27.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.012+0000 7f6905600640 1 -- 172.21.15.94:0/2680686516 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7f68f0067150 con 0x7f6908071a20 2024-09-10T23:33:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:27.016 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:33:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-09-10T23:33:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-09-10T23:33:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:27.017 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:33:27.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.012+0000 7f690fa00640 1 -- 172.21.15.94:0/2680686516 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f68e40725e0 msgr2=0x7f68e4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:27.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.012+0000 7f690fa00640 1 --2- 172.21.15.94:0/2680686516 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f68e40725e0 0x7f68e4074aa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f69081475f0 tx=0x7f68f800a2d0 comp rx=0 tx=0).stop 2024-09-10T23:33:27.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.012+0000 7f690fa00640 1 -- 172.21.15.94:0/2680686516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6908071a20 msgr2=0x7f690815a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:27.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.012+0000 7f690fa00640 1 --2- 172.21.15.94:0/2680686516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6908071a20 0x7f690815a550 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f68f000d980 tx=0x7f68f000de40 comp rx=0 tx=0).stop 2024-09-10T23:33:27.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.012+0000 7f690fa00640 1 -- 172.21.15.94:0/2680686516 shutdown_connections 2024-09-10T23:33:27.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.012+0000 7f690fa00640 1 --2- 172.21.15.94:0/2680686516 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f68e40725e0 0x7f68e4074aa0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:27.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.012+0000 7f690fa00640 1 --2- 172.21.15.94:0/2680686516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69080d1690 0x7f690814e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:27.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.012+0000 7f690fa00640 1 --2- 172.21.15.94:0/2680686516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6908071a20 0x7f690815a550 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:27.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.012+0000 7f690fa00640 1 --2- 172.21.15.94:0/2680686516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f690806cc20 0x7f6908143e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:27.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.012+0000 7f690fa00640 1 -- 172.21.15.94:0/2680686516 >> 172.21.15.94:0/2680686516 conn(0x7f69080d0b70 msgr2=0x7f69080d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:27.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.012+0000 7f690fa00640 1 -- 172.21.15.94:0/2680686516 shutdown_connections 2024-09-10T23:33:27.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.012+0000 7f690fa00640 1 -- 172.21.15.94:0/2680686516 wait complete. 2024-09-10T23:33:27.038 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T23:33:27.038 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T23:33:27.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.180+0000 7ff179000640 1 -- 172.21.15.94:0/1621477685 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1740d04b0 msgr2=0x7ff17415e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:27.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.180+0000 7ff179000640 1 --2- 172.21.15.94:0/1621477685 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1740d04b0 0x7ff17415e770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff160009900 tx=0x7ff16001c8b0 comp rx=0 tx=0).stop 2024-09-10T23:33:27.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.180+0000 7ff179000640 1 -- 172.21.15.94:0/1621477685 shutdown_connections 2024-09-10T23:33:27.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.180+0000 7ff179000640 1 --2- 172.21.15.94:0/1621477685 >> v2:172.21.15.181:3300/0 conn(0x7ff17415ecb0 0x7ff17406f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:27.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.180+0000 7ff179000640 1 --2- 172.21.15.94:0/1621477685 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1740d04b0 0x7ff17415e770 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:27.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.180+0000 7ff179000640 1 -- 172.21.15.94:0/1621477685 >> 172.21.15.94:0/1621477685 conn(0x7ff17406cdf0 msgr2=0x7ff17406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:27.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.180+0000 7ff179000640 1 -- 172.21.15.94:0/1621477685 shutdown_connections 2024-09-10T23:33:27.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.180+0000 7ff179000640 1 -- 172.21.15.94:0/1621477685 wait complete. 2024-09-10T23:33:27.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.180+0000 7ff179000640 1 Processor -- start 2024-09-10T23:33:27.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.180+0000 7ff179000640 1 -- start start 2024-09-10T23:33:27.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff179000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff174073ff0 0x7ff174148340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:27.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff179000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1740d04b0 0x7ff174136240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:27.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff179000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff17415ecb0 0x7ff174149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:27.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff179000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff174167930 con 0x7ff1740d04b0 2024-09-10T23:33:27.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff179000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff1741677b0 con 0x7ff174073ff0 2024-09-10T23:33:27.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff179000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff174167ab0 con 0x7ff17415ecb0 2024-09-10T23:33:27.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff171400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1740d04b0 0x7ff174136240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:27.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff171400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1740d04b0 0x7ff174136240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60204/0 (socket says 172.21.15.94:60204) 2024-09-10T23:33:27.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff171400640 1 -- 172.21.15.94:0/3878427835 learned_addr learned my addr 172.21.15.94:0/3878427835 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:27.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff171e00640 1 --2- 172.21.15.94:0/3878427835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff174073ff0 0x7ff174148340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:27.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff172800640 1 --2- 172.21.15.94:0/3878427835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff17415ecb0 0x7ff174149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:27.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff171400640 1 -- 172.21.15.94:0/3878427835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff17415ecb0 msgr2=0x7ff174149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:27.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff171400640 1 --2- 172.21.15.94:0/3878427835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff17415ecb0 0x7ff174149bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:27.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff171400640 1 -- 172.21.15.94:0/3878427835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff174073ff0 msgr2=0x7ff174148340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:27.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff171400640 1 --2- 172.21.15.94:0/3878427835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff174073ff0 0x7ff174148340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:27.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff171400640 1 -- 172.21.15.94:0/3878427835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff174148880 con 0x7ff1740d04b0 2024-09-10T23:33:27.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff171e00640 1 --2- 172.21.15.94:0/3878427835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff174073ff0 0x7ff174148340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:27.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff171400640 1 --2- 172.21.15.94:0/3878427835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1740d04b0 0x7ff174136240 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff160009850 tx=0x7ff160004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:27.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff16a000640 1 -- 172.21.15.94:0/3878427835 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1600b2040 con 0x7ff1740d04b0 2024-09-10T23:33:27.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff16a000640 1 -- 172.21.15.94:0/3878427835 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff160005030 con 0x7ff1740d04b0 2024-09-10T23:33:27.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff179000640 1 -- 172.21.15.94:0/3878427835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff174148b20 con 0x7ff1740d04b0 2024-09-10T23:33:27.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff16a000640 1 -- 172.21.15.94:0/3878427835 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160005460 con 0x7ff1740d04b0 2024-09-10T23:33:27.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff179000640 1 -- 172.21.15.94:0/3878427835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1741490e0 con 0x7ff1740d04b0 2024-09-10T23:33:27.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff179000640 1 -- 172.21.15.94:0/3878427835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff17415e770 con 0x7ff1740d04b0 2024-09-10T23:33:27.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.184+0000 7ff16a000640 1 -- 172.21.15.94:0/3878427835 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff1600046d0 con 0x7ff1740d04b0 2024-09-10T23:33:27.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.188+0000 7ff16a000640 1 --2- 172.21.15.94:0/3878427835 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff1540725e0 0x7ff154074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:27.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.188+0000 7ff16a000640 1 -- 172.21.15.94:0/3878427835 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7ff16012c810 con 0x7ff1740d04b0 2024-09-10T23:33:27.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.188+0000 7ff171e00640 1 --2- 172.21.15.94:0/3878427835 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff1540725e0 0x7ff154074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:27.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.188+0000 7ff171e00640 1 --2- 172.21.15.94:0/3878427835 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff1540725e0 0x7ff154074aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff164001e50 tx=0x7ff164009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:27.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.192+0000 7ff16a000640 1 -- 172.21.15.94:0/3878427835 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff1600bc060 con 0x7ff1740d04b0 2024-09-10T23:33:27.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.356+0000 7ff179000640 1 -- 172.21.15.94:0/3878427835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff174069850 con 0x7ff1740d04b0 2024-09-10T23:33:27.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.860+0000 7ff16a000640 1 -- 172.21.15.94:0/3878427835 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7ff1600164d0 con 0x7ff1740d04b0 2024-09-10T23:33:27.863 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-09-10T23:33:27.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.860+0000 7ff179000640 1 -- 172.21.15.94:0/3878427835 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff1540725e0 msgr2=0x7ff154074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:27.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.860+0000 7ff179000640 1 --2- 172.21.15.94:0/3878427835 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff1540725e0 0x7ff154074aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff164001e50 tx=0x7ff164009040 comp rx=0 tx=0).stop 2024-09-10T23:33:27.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.860+0000 7ff179000640 1 -- 172.21.15.94:0/3878427835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1740d04b0 msgr2=0x7ff174136240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:27.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.860+0000 7ff179000640 1 --2- 172.21.15.94:0/3878427835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1740d04b0 0x7ff174136240 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff160009850 tx=0x7ff160004040 comp rx=0 tx=0).stop 2024-09-10T23:33:27.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.860+0000 7ff179000640 1 -- 172.21.15.94:0/3878427835 shutdown_connections 2024-09-10T23:33:27.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.860+0000 7ff179000640 1 --2- 172.21.15.94:0/3878427835 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff1540725e0 0x7ff154074aa0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:27.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.860+0000 7ff179000640 1 --2- 172.21.15.94:0/3878427835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff17415ecb0 0x7ff174149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:27.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.860+0000 7ff179000640 1 --2- 172.21.15.94:0/3878427835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1740d04b0 0x7ff174136240 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:27.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.860+0000 7ff179000640 1 --2- 172.21.15.94:0/3878427835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff174073ff0 0x7ff174148340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:27.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.860+0000 7ff179000640 1 -- 172.21.15.94:0/3878427835 >> 172.21.15.94:0/3878427835 conn(0x7ff17406cdf0 msgr2=0x7ff1740b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:27.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.860+0000 7ff179000640 1 -- 172.21.15.94:0/3878427835 shutdown_connections 2024-09-10T23:33:27.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:27.860+0000 7ff179000640 1 -- 172.21.15.94:0/3878427835 wait complete. 2024-09-10T23:33:27.891 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T23:33:28.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.036+0000 7fe46e000640 1 -- 172.21.15.94:0/1645994772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4680b2700 msgr2=0x7fe4680cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:28.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.036+0000 7fe46e000640 1 --2- 172.21.15.94:0/1645994772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4680b2700 0x7fe4680cd0a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe460009c80 tx=0x7fe46001cda0 comp rx=0 tx=0).stop 2024-09-10T23:33:28.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.036+0000 7fe46e000640 1 -- 172.21.15.94:0/1645994772 shutdown_connections 2024-09-10T23:33:28.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.036+0000 7fe46e000640 1 --2- 172.21.15.94:0/1645994772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4680b2700 0x7fe4680cd0a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:28.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.036+0000 7fe46e000640 1 -- 172.21.15.94:0/1645994772 >> 172.21.15.94:0/1645994772 conn(0x7fe4680d1cb0 msgr2=0x7fe4680d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:28.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.036+0000 7fe46e000640 1 -- 172.21.15.94:0/1645994772 shutdown_connections 2024-09-10T23:33:28.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.036+0000 7fe46e000640 1 -- 172.21.15.94:0/1645994772 wait complete. 2024-09-10T23:33:28.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.036+0000 7fe46e000640 1 Processor -- start 2024-09-10T23:33:28.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.036+0000 7fe46e000640 1 -- start start 2024-09-10T23:33:28.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe46e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4680b2700 0x7fe4681fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:28.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe46e000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe46815f180 0x7fe468159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:28.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe46e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe46815f560 0x7fe4680aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:28.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe46e000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe46816a010 con 0x7fe46815f560 2024-09-10T23:33:28.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe46e000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe468169e90 con 0x7fe4680b2700 2024-09-10T23:33:28.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe46e000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe46816a190 con 0x7fe46815f180 2024-09-10T23:33:28.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe467e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe46815f560 0x7fe4680aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:28.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe467e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe46815f560 0x7fe4680aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60224/0 (socket says 172.21.15.94:60224) 2024-09-10T23:33:28.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe467e00640 1 -- 172.21.15.94:0/1110656244 learned_addr learned my addr 172.21.15.94:0/1110656244 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:28.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe467400640 1 --2- 172.21.15.94:0/1110656244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4680b2700 0x7fe4681fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:28.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe466a00640 1 --2- 172.21.15.94:0/1110656244 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe46815f180 0x7fe468159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:28.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe467e00640 1 -- 172.21.15.94:0/1110656244 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe46815f180 msgr2=0x7fe468159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:28.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe467e00640 1 --2- 172.21.15.94:0/1110656244 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe46815f180 0x7fe468159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:28.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe467e00640 1 -- 172.21.15.94:0/1110656244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4680b2700 msgr2=0x7fe4681fbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:28.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe467e00640 1 --2- 172.21.15.94:0/1110656244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4680b2700 0x7fe4681fbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:28.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe467e00640 1 -- 172.21.15.94:0/1110656244 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4681fb130 con 0x7fe46815f560 2024-09-10T23:33:28.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe467400640 1 --2- 172.21.15.94:0/1110656244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4680b2700 0x7fe4681fbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:28.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe466a00640 1 --2- 172.21.15.94:0/1110656244 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe46815f180 0x7fe468159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:28.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe467e00640 1 --2- 172.21.15.94:0/1110656244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe46815f560 0x7fe4680aecd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe46001b450 tx=0x7fe4600a5b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:28.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe45f400640 1 -- 172.21.15.94:0/1110656244 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4600bb070 con 0x7fe46815f560 2024-09-10T23:33:28.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe45f400640 1 -- 172.21.15.94:0/1110656244 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe460004d60 con 0x7fe46815f560 2024-09-10T23:33:28.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe46e000640 1 -- 172.21.15.94:0/1110656244 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4680b57d0 con 0x7fe46815f560 2024-09-10T23:33:28.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe45f400640 1 -- 172.21.15.94:0/1110656244 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4600b2040 con 0x7fe46815f560 2024-09-10T23:33:28.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe46e000640 1 -- 172.21.15.94:0/1110656244 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4681fb3d0 con 0x7fe46815f560 2024-09-10T23:33:28.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe46e000640 1 -- 172.21.15.94:0/1110656244 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4680cd0a0 con 0x7fe46815f560 2024-09-10T23:33:28.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.040+0000 7fe45f400640 1 -- 172.21.15.94:0/1110656244 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe4600b5030 con 0x7fe46815f560 2024-09-10T23:33:28.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.044+0000 7fe45f400640 1 --2- 172.21.15.94:0/1110656244 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe448072610 0x7fe448074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:28.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.044+0000 7fe45f400640 1 -- 172.21.15.94:0/1110656244 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7fe46012cda0 con 0x7fe46815f560 2024-09-10T23:33:28.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.044+0000 7fe467400640 1 --2- 172.21.15.94:0/1110656244 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe448072610 0x7fe448074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:28.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.044+0000 7fe467400640 1 --2- 172.21.15.94:0/1110656244 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe448072610 0x7fe448074ad0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe4580016e0 tx=0x7fe458009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:28.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.048+0000 7fe45f400640 1 -- 172.21.15.94:0/1110656244 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe460016990 con 0x7fe46815f560 2024-09-10T23:33:28.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.216+0000 7fe46e000640 1 -- 172.21.15.94:0/1110656244 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe46806c740 con 0x7fe46815f560 2024-09-10T23:33:28.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.864+0000 7fe45f400640 1 -- 172.21.15.94:0/1110656244 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7fe4600f8650 con 0x7fe46815f560 2024-09-10T23:33:28.870 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-09-10T23:33:28.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.868+0000 7fe46e000640 1 -- 172.21.15.94:0/1110656244 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe448072610 msgr2=0x7fe448074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:28.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.868+0000 7fe46e000640 1 --2- 172.21.15.94:0/1110656244 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe448072610 0x7fe448074ad0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe4580016e0 tx=0x7fe458009040 comp rx=0 tx=0).stop 2024-09-10T23:33:28.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.868+0000 7fe46e000640 1 -- 172.21.15.94:0/1110656244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe46815f560 msgr2=0x7fe4680aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:28.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.868+0000 7fe46e000640 1 --2- 172.21.15.94:0/1110656244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe46815f560 0x7fe4680aecd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe46001b450 tx=0x7fe4600a5b60 comp rx=0 tx=0).stop 2024-09-10T23:33:28.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.868+0000 7fe46e000640 1 -- 172.21.15.94:0/1110656244 shutdown_connections 2024-09-10T23:33:28.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.868+0000 7fe46e000640 1 --2- 172.21.15.94:0/1110656244 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe448072610 0x7fe448074ad0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:28.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.868+0000 7fe46e000640 1 --2- 172.21.15.94:0/1110656244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe46815f560 0x7fe4680aecd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:28.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.868+0000 7fe46e000640 1 --2- 172.21.15.94:0/1110656244 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe46815f180 0x7fe468159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:28.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.868+0000 7fe46e000640 1 --2- 172.21.15.94:0/1110656244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4680b2700 0x7fe4681fbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:28.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.868+0000 7fe46e000640 1 -- 172.21.15.94:0/1110656244 >> 172.21.15.94:0/1110656244 conn(0x7fe4680d1cb0 msgr2=0x7fe468071b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:28.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.872+0000 7fe46e000640 1 -- 172.21.15.94:0/1110656244 shutdown_connections 2024-09-10T23:33:28.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:28.872+0000 7fe46e000640 1 -- 172.21.15.94:0/1110656244 wait complete. 2024-09-10T23:33:28.907 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T23:33:29.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.048+0000 7f8663e00640 1 -- 172.21.15.94:0/4229164203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f865c0c64f0 msgr2=0x7f865c0ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:29.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.048+0000 7f8663e00640 1 --2- 172.21.15.94:0/4229164203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f865c0c64f0 0x7f865c0ce660 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f865c0cd580 tx=0x7f864801c8a0 comp rx=0 tx=0).stop 2024-09-10T23:33:29.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.048+0000 7f8663e00640 1 -- 172.21.15.94:0/4229164203 shutdown_connections 2024-09-10T23:33:29.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.048+0000 7f8663e00640 1 --2- 172.21.15.94:0/4229164203 >> v2:172.21.15.181:3300/0 conn(0x7f865c06e7c0 0x7f865c06ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:29.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.048+0000 7f8663e00640 1 --2- 172.21.15.94:0/4229164203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f865c0c64f0 0x7f865c0ce660 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:29.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.048+0000 7f8663e00640 1 --2- 172.21.15.94:0/4229164203 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f865c0ce280 0x7f865c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:29.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.048+0000 7f8663e00640 1 -- 172.21.15.94:0/4229164203 >> 172.21.15.94:0/4229164203 conn(0x7f865c0cfaf0 msgr2=0x7f865c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:29.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.048+0000 7f8663e00640 1 -- 172.21.15.94:0/4229164203 shutdown_connections 2024-09-10T23:33:29.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.048+0000 7f8663e00640 1 -- 172.21.15.94:0/4229164203 wait complete. 2024-09-10T23:33:29.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.048+0000 7f8663e00640 1 Processor -- start 2024-09-10T23:33:29.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.048+0000 7f8663e00640 1 -- start start 2024-09-10T23:33:29.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.048+0000 7f8663e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f865c06e7c0 0x7f865c0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:29.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.048+0000 7f8663e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f865c0c64f0 0x7f865c14b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:29.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.048+0000 7f8663e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f865c0ce280 0x7f865c0c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:29.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8663e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f865c15b400 con 0x7f865c0c64f0 2024-09-10T23:33:29.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8663e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f865c15b280 con 0x7f865c0ce280 2024-09-10T23:33:29.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8663e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f865c15b580 con 0x7f865c06e7c0 2024-09-10T23:33:29.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8660a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f865c0c64f0 0x7f865c14b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:29.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8660a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f865c0c64f0 0x7f865c14b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60244/0 (socket says 172.21.15.94:60244) 2024-09-10T23:33:29.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8660a00640 1 -- 172.21.15.94:0/3644374780 learned_addr learned my addr 172.21.15.94:0/3644374780 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:29.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8661e00640 1 --2- 172.21.15.94:0/3644374780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f865c0ce280 0x7f865c0c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:29.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8661400640 1 --2- 172.21.15.94:0/3644374780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f865c06e7c0 0x7f865c0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:29.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8660a00640 1 -- 172.21.15.94:0/3644374780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f865c06e7c0 msgr2=0x7f865c0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:29.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8660a00640 1 --2- 172.21.15.94:0/3644374780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f865c06e7c0 0x7f865c0b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:29.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8660a00640 1 -- 172.21.15.94:0/3644374780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f865c0ce280 msgr2=0x7f865c0c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:29.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8660a00640 1 --2- 172.21.15.94:0/3644374780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f865c0ce280 0x7f865c0c5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:29.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8660a00640 1 -- 172.21.15.94:0/3644374780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c1fb350 con 0x7f865c0c64f0 2024-09-10T23:33:29.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8661e00640 1 --2- 172.21.15.94:0/3644374780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f865c0ce280 0x7f865c0c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:29.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8661400640 1 --2- 172.21.15.94:0/3644374780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f865c06e7c0 0x7f865c0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:29.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8660a00640 1 --2- 172.21.15.94:0/3644374780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f865c0c64f0 0x7f865c14b470 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8648009850 tx=0x7f86480a5720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:29.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8659600640 1 -- 172.21.15.94:0/3644374780 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86480b2040 con 0x7f865c0c64f0 2024-09-10T23:33:29.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8659600640 1 -- 172.21.15.94:0/3644374780 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8648004470 con 0x7f865c0c64f0 2024-09-10T23:33:29.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8659600640 1 -- 172.21.15.94:0/3644374780 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86480052d0 con 0x7f865c0c64f0 2024-09-10T23:33:29.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8663e00640 1 -- 172.21.15.94:0/3644374780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865c1fb630 con 0x7f865c0c64f0 2024-09-10T23:33:29.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8663e00640 1 -- 172.21.15.94:0/3644374780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865c1fbbf0 con 0x7f865c0c64f0 2024-09-10T23:33:29.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8663e00640 1 -- 172.21.15.94:0/3644374780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f865c0ce660 con 0x7f865c0c64f0 2024-09-10T23:33:29.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.052+0000 7f8659600640 1 -- 172.21.15.94:0/3644374780 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8648005470 con 0x7f865c0c64f0 2024-09-10T23:33:29.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.056+0000 7f8659600640 1 --2- 172.21.15.94:0/3644374780 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f863c0725e0 0x7f863c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:29.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.056+0000 7f8659600640 1 -- 172.21.15.94:0/3644374780 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7f864812cb70 con 0x7f865c0c64f0 2024-09-10T23:33:29.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.056+0000 7f8661400640 1 --2- 172.21.15.94:0/3644374780 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f863c0725e0 0x7f863c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:29.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.056+0000 7f8661400640 1 --2- 172.21.15.94:0/3644374780 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f863c0725e0 0x7f863c074aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f865c158900 tx=0x7f865000a2f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:29.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.056+0000 7f8659600640 1 -- 172.21.15.94:0/3644374780 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8648016490 con 0x7f865c0c64f0 2024-09-10T23:33:29.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.256+0000 7f8663e00640 1 -- 172.21.15.94:0/3644374780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f865c0d0850 con 0x7f865c0c64f0 2024-09-10T23:33:29.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.880+0000 7f8659600640 1 -- 172.21.15.94:0/3644374780 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f86480f8340 con 0x7f865c0c64f0 2024-09-10T23:33:29.888 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 4 and data pool 5 2024-09-10T23:33:29.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.888+0000 7f8663e00640 1 -- 172.21.15.94:0/3644374780 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f863c0725e0 msgr2=0x7f863c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.888+0000 7f8663e00640 1 --2- 172.21.15.94:0/3644374780 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f863c0725e0 0x7f863c074aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f865c158900 tx=0x7f865000a2f0 comp rx=0 tx=0).stop 2024-09-10T23:33:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.888+0000 7f8663e00640 1 -- 172.21.15.94:0/3644374780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f865c0c64f0 msgr2=0x7f865c14b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.888+0000 7f8663e00640 1 --2- 172.21.15.94:0/3644374780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f865c0c64f0 0x7f865c14b470 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8648009850 tx=0x7f86480a5720 comp rx=0 tx=0).stop 2024-09-10T23:33:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.888+0000 7f8663e00640 1 -- 172.21.15.94:0/3644374780 shutdown_connections 2024-09-10T23:33:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.888+0000 7f8663e00640 1 --2- 172.21.15.94:0/3644374780 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f863c0725e0 0x7f863c074aa0 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f865c158900 tx=0x7f865000a2f0 comp rx=0 tx=0).stop 2024-09-10T23:33:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.888+0000 7f8663e00640 1 --2- 172.21.15.94:0/3644374780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f865c0ce280 0x7f865c0c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.888+0000 7f8663e00640 1 --2- 172.21.15.94:0/3644374780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f865c0c64f0 0x7f865c14b470 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:29.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.888+0000 7f8663e00640 1 --2- 172.21.15.94:0/3644374780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f865c06e7c0 0x7f865c0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.888+0000 7f8663e00640 1 -- 172.21.15.94:0/3644374780 >> 172.21.15.94:0/3644374780 conn(0x7f865c0cfaf0 msgr2=0x7f865c14c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.892+0000 7f8663e00640 1 -- 172.21.15.94:0/3644374780 shutdown_connections 2024-09-10T23:33:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:29.892+0000 7f8663e00640 1 -- 172.21.15.94:0/3644374780 wait complete. 2024-09-10T23:33:29.927 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:33:30.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.064+0000 7f3d67e00640 1 -- 172.21.15.94:0/2540515261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d6015ba50 msgr2=0x7f3d600b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:30.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.064+0000 7f3d67e00640 1 --2- 172.21.15.94:0/2540515261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d6015ba50 0x7f3d600b4700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3d600cd920 tx=0x7f3d5401c570 comp rx=0 tx=0).stop 2024-09-10T23:33:30.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.064+0000 7f3d67e00640 1 -- 172.21.15.94:0/2540515261 shutdown_connections 2024-09-10T23:33:30.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.064+0000 7f3d67e00640 1 --2- 172.21.15.94:0/2540515261 >> v2:172.21.15.181:3300/0 conn(0x7f3d600d01f0 0x7f3d6015f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.064+0000 7f3d67e00640 1 --2- 172.21.15.94:0/2540515261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d600b4c40 0x7f3d600cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.064+0000 7f3d67e00640 1 --2- 172.21.15.94:0/2540515261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d6015ba50 0x7f3d600b4700 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.064+0000 7f3d67e00640 1 -- 172.21.15.94:0/2540515261 >> 172.21.15.94:0/2540515261 conn(0x7f3d6006cdf0 msgr2=0x7f3d6006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:30.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.064+0000 7f3d67e00640 1 -- 172.21.15.94:0/2540515261 shutdown_connections 2024-09-10T23:33:30.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.064+0000 7f3d67e00640 1 -- 172.21.15.94:0/2540515261 wait complete. 2024-09-10T23:33:30.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.064+0000 7f3d67e00640 1 Processor -- start 2024-09-10T23:33:30.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d67e00640 1 -- start start 2024-09-10T23:33:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d67e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d600b4c40 0x7f3d60157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d67e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d600d01f0 0x7f3d601f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d67e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d6015ba50 0x7f3d6015a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d67e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d600ce3e0 con 0x7f3d6015ba50 2024-09-10T23:33:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d67e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3d600ce260 con 0x7f3d600d01f0 2024-09-10T23:33:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d67e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3d600ce560 con 0x7f3d600b4c40 2024-09-10T23:33:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d65e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d6015ba50 0x7f3d6015a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d65e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d6015ba50 0x7f3d6015a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60254/0 (socket says 172.21.15.94:60254) 2024-09-10T23:33:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d65e00640 1 -- 172.21.15.94:0/37893184 learned_addr learned my addr 172.21.15.94:0/37893184 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:30.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d65400640 1 --2- 172.21.15.94:0/37893184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d600b4c40 0x7f3d60157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:30.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d64a00640 1 --2- 172.21.15.94:0/37893184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d600d01f0 0x7f3d601f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:30.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d65e00640 1 -- 172.21.15.94:0/37893184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d600b4c40 msgr2=0x7f3d60157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:30.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d65e00640 1 --2- 172.21.15.94:0/37893184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d600b4c40 0x7f3d60157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d65e00640 1 -- 172.21.15.94:0/37893184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d600d01f0 msgr2=0x7f3d601f7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:30.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d65e00640 1 --2- 172.21.15.94:0/37893184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d600d01f0 0x7f3d601f7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d65e00640 1 -- 172.21.15.94:0/37893184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d60158460 con 0x7f3d6015ba50 2024-09-10T23:33:30.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d65e00640 1 --2- 172.21.15.94:0/37893184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d6015ba50 0x7f3d6015a3b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800ea20 tx=0x7f3d5800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:30.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d5d600640 1 -- 172.21.15.94:0/37893184 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5800cde0 con 0x7f3d6015ba50 2024-09-10T23:33:30.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d5d600640 1 -- 172.21.15.94:0/37893184 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d58004c40 con 0x7f3d6015ba50 2024-09-10T23:33:30.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d5d600640 1 -- 172.21.15.94:0/37893184 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58005280 con 0x7f3d6015ba50 2024-09-10T23:33:30.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d67e00640 1 -- 172.21.15.94:0/37893184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d60144130 con 0x7f3d6015ba50 2024-09-10T23:33:30.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d67e00640 1 -- 172.21.15.94:0/37893184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d601446f0 con 0x7f3d6015ba50 2024-09-10T23:33:30.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.068+0000 7f3d67e00640 1 -- 172.21.15.94:0/37893184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d600cfc50 con 0x7f3d6015ba50 2024-09-10T23:33:30.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.072+0000 7f3d5d600640 1 -- 172.21.15.94:0/37893184 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3d5800f040 con 0x7f3d6015ba50 2024-09-10T23:33:30.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.072+0000 7f3d5d600640 1 --2- 172.21.15.94:0/37893184 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3d40072610 0x7f3d40074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:30.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.072+0000 7f3d5d600640 1 -- 172.21.15.94:0/37893184 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f3d58093850 con 0x7f3d6015ba50 2024-09-10T23:33:30.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.072+0000 7f3d65400640 1 --2- 172.21.15.94:0/37893184 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3d40072610 0x7f3d40074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:30.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.072+0000 7f3d65400640 1 --2- 172.21.15.94:0/37893184 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3d40072610 0x7f3d40074ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3d540af000 tx=0x7f3d540b4040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:30.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.076+0000 7f3d5d600640 1 -- 172.21.15.94:0/37893184 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3d58016030 con 0x7f3d6015ba50 2024-09-10T23:33:30.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.244+0000 7f3d67e00640 1 -- 172.21.15.94:0/37893184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d600cff20 con 0x7f3d6015ba50 2024-09-10T23:33:30.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.244+0000 7f3d5d600640 1 -- 172.21.15.94:0/37893184 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16670 (secure 0 0 0) 0x7f3d5805e8d0 con 0x7f3d6015ba50 2024-09-10T23:33:30.249 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:30.249 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":32,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:33:29.261419+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:33:27.370128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T23:33:28.229333+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:33:30.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.248+0000 7f3d67e00640 1 -- 172.21.15.94:0/37893184 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3d40072610 msgr2=0x7f3d40074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:30.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.248+0000 7f3d67e00640 1 --2- 172.21.15.94:0/37893184 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3d40072610 0x7f3d40074ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3d540af000 tx=0x7f3d540b4040 comp rx=0 tx=0).stop 2024-09-10T23:33:30.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.248+0000 7f3d67e00640 1 -- 172.21.15.94:0/37893184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d6015ba50 msgr2=0x7f3d6015a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:30.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.248+0000 7f3d67e00640 1 --2- 172.21.15.94:0/37893184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d6015ba50 0x7f3d6015a3b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800ea20 tx=0x7f3d5800eee0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.248+0000 7f3d67e00640 1 -- 172.21.15.94:0/37893184 shutdown_connections 2024-09-10T23:33:30.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.248+0000 7f3d67e00640 1 --2- 172.21.15.94:0/37893184 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3d40072610 0x7f3d40074ad0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.248+0000 7f3d67e00640 1 --2- 172.21.15.94:0/37893184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d600d01f0 0x7f3d601f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.248+0000 7f3d67e00640 1 --2- 172.21.15.94:0/37893184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d6015ba50 0x7f3d6015a3b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.248+0000 7f3d67e00640 1 --2- 172.21.15.94:0/37893184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d600b4c40 0x7f3d60157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.248+0000 7f3d67e00640 1 -- 172.21.15.94:0/37893184 >> 172.21.15.94:0/37893184 conn(0x7f3d6006cdf0 msgr2=0x7f3d6015e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:30.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.248+0000 7f3d67e00640 1 -- 172.21.15.94:0/37893184 shutdown_connections 2024-09-10T23:33:30.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.248+0000 7f3d67e00640 1 -- 172.21.15.94:0/37893184 wait complete. 2024-09-10T23:33:30.270 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:33:30.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a5d200640 1 -- 172.21.15.94:0/2115020736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a58152c10 msgr2=0x7f1a580b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:30.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a5d200640 1 --2- 172.21.15.94:0/2115020736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a58152c10 0x7f1a580b29d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009bc0 tx=0x7f1a4801cb70 comp rx=0 tx=0).stop 2024-09-10T23:33:30.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a5d200640 1 -- 172.21.15.94:0/2115020736 shutdown_connections 2024-09-10T23:33:30.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a5d200640 1 --2- 172.21.15.94:0/2115020736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a58152c10 0x7f1a580b29d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a5d200640 1 -- 172.21.15.94:0/2115020736 >> 172.21.15.94:0/2115020736 conn(0x7f1a5806cdf0 msgr2=0x7f1a5806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:30.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a5d200640 1 -- 172.21.15.94:0/2115020736 shutdown_connections 2024-09-10T23:33:30.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a5d200640 1 -- 172.21.15.94:0/2115020736 wait complete. 2024-09-10T23:33:30.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a5d200640 1 Processor -- start 2024-09-10T23:33:30.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a5d200640 1 -- start start 2024-09-10T23:33:30.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a5d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a580cfd50 0x7f1a58137c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:30.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a5d200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a58152c10 0x7f1a5814d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:30.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a5d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a5815cbc0 0x7f1a5814a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:30.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a5d200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5816a5e0 con 0x7f1a580cfd50 2024-09-10T23:33:30.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a5d200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5816a460 con 0x7f1a5815cbc0 2024-09-10T23:33:30.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a5d200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1a5816a760 con 0x7f1a58152c10 2024-09-10T23:33:30.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a55e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a580cfd50 0x7f1a58137c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:30.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a55e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a580cfd50 0x7f1a58137c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60260/0 (socket says 172.21.15.94:60260) 2024-09-10T23:33:30.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a55e00640 1 -- 172.21.15.94:0/827251637 learned_addr learned my addr 172.21.15.94:0/827251637 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:30.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a56800640 1 --2- 172.21.15.94:0/827251637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a5815cbc0 0x7f1a5814a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:30.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a4d400640 1 --2- 172.21.15.94:0/827251637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a58152c10 0x7f1a5814d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:30.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a55e00640 1 -- 172.21.15.94:0/827251637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a58152c10 msgr2=0x7f1a5814d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:30.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a55e00640 1 --2- 172.21.15.94:0/827251637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a58152c10 0x7f1a5814d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a55e00640 1 -- 172.21.15.94:0/827251637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a5815cbc0 msgr2=0x7f1a5814a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:30.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a55e00640 1 --2- 172.21.15.94:0/827251637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a5815cbc0 0x7f1a5814a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a55e00640 1 -- 172.21.15.94:0/827251637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5814dac0 con 0x7f1a580cfd50 2024-09-10T23:33:30.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.408+0000 7f1a4d400640 1 --2- 172.21.15.94:0/827251637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a58152c10 0x7f1a5814d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:30.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.412+0000 7f1a55e00640 1 --2- 172.21.15.94:0/827251637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a580cfd50 0x7f1a58137c40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009b10 tx=0x7f1a480a5a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:30.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.412+0000 7f1a4ea00640 1 -- 172.21.15.94:0/827251637 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a480ba020 con 0x7f1a580cfd50 2024-09-10T23:33:30.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.412+0000 7f1a4ea00640 1 -- 172.21.15.94:0/827251637 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a48004d40 con 0x7f1a580cfd50 2024-09-10T23:33:30.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.412+0000 7f1a5d200640 1 -- 172.21.15.94:0/827251637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5814aeb0 con 0x7f1a580cfd50 2024-09-10T23:33:30.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.412+0000 7f1a4ea00640 1 -- 172.21.15.94:0/827251637 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a480b2040 con 0x7f1a580cfd50 2024-09-10T23:33:30.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.412+0000 7f1a5d200640 1 -- 172.21.15.94:0/827251637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a5814b470 con 0x7f1a580cfd50 2024-09-10T23:33:30.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.412+0000 7f1a5d200640 1 -- 172.21.15.94:0/827251637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a58072bd0 con 0x7f1a580cfd50 2024-09-10T23:33:30.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.412+0000 7f1a4ea00640 1 -- 172.21.15.94:0/827251637 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1a480b5070 con 0x7f1a580cfd50 2024-09-10T23:33:30.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.412+0000 7f1a4ea00640 1 --2- 172.21.15.94:0/827251637 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a34072610 0x7f1a34074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:30.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.412+0000 7f1a4ea00640 1 -- 172.21.15.94:0/827251637 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f1a4812cec0 con 0x7f1a580cfd50 2024-09-10T23:33:30.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.416+0000 7f1a4d400640 1 --2- 172.21.15.94:0/827251637 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a34072610 0x7f1a34074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:30.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.416+0000 7f1a4d400640 1 --2- 172.21.15.94:0/827251637 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a34072610 0x7f1a34074ad0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1a580cc730 tx=0x7f1a40009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:30.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.416+0000 7f1a4ea00640 1 -- 172.21.15.94:0/827251637 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1a48016790 con 0x7f1a580cfd50 2024-09-10T23:33:30.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.584+0000 7f1a5d200640 1 -- 172.21.15.94:0/827251637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a5806d370 con 0x7f1a580cfd50 2024-09-10T23:33:30.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.588+0000 7f1a4ea00640 1 -- 172.21.15.94:0/827251637 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16670 (secure 0 0 0) 0x7f1a480f7f40 con 0x7f1a580cfd50 2024-09-10T23:33:30.592 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:30.592 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":32,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:33:29.261419+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:33:27.370128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T23:33:28.229333+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:33:30.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.588+0000 7f1a5d200640 1 -- 172.21.15.94:0/827251637 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a34072610 msgr2=0x7f1a34074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:30.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.588+0000 7f1a5d200640 1 --2- 172.21.15.94:0/827251637 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a34072610 0x7f1a34074ad0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1a580cc730 tx=0x7f1a40009040 comp rx=0 tx=0).stop 2024-09-10T23:33:30.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.588+0000 7f1a5d200640 1 -- 172.21.15.94:0/827251637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a580cfd50 msgr2=0x7f1a58137c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:30.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.588+0000 7f1a5d200640 1 --2- 172.21.15.94:0/827251637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a580cfd50 0x7f1a58137c40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009b10 tx=0x7f1a480a5a60 comp rx=0 tx=0).stop 2024-09-10T23:33:30.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.588+0000 7f1a5d200640 1 -- 172.21.15.94:0/827251637 shutdown_connections 2024-09-10T23:33:30.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.588+0000 7f1a5d200640 1 --2- 172.21.15.94:0/827251637 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a34072610 0x7f1a34074ad0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.588+0000 7f1a5d200640 1 --2- 172.21.15.94:0/827251637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a5815cbc0 0x7f1a5814a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.588+0000 7f1a5d200640 1 --2- 172.21.15.94:0/827251637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a58152c10 0x7f1a5814d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.588+0000 7f1a5d200640 1 --2- 172.21.15.94:0/827251637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a580cfd50 0x7f1a58137c40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.588+0000 7f1a5d200640 1 -- 172.21.15.94:0/827251637 >> 172.21.15.94:0/827251637 conn(0x7f1a5806cdf0 msgr2=0x7f1a580b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:30.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.588+0000 7f1a5d200640 1 -- 172.21.15.94:0/827251637 shutdown_connections 2024-09-10T23:33:30.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.588+0000 7f1a5d200640 1 -- 172.21.15.94:0/827251637 wait complete. 2024-09-10T23:33:30.614 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T23:33:30.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.752+0000 7fa602000640 1 -- 172.21.15.94:0/2238850379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc0b3200 msgr2=0x7fa5fc0d49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:30.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.752+0000 7fa602000640 1 --2- 172.21.15.94:0/2238850379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc0b3200 0x7fa5fc0d49c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009900 tx=0x7fa5e401c8b0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.752+0000 7fa602000640 1 -- 172.21.15.94:0/2238850379 shutdown_connections 2024-09-10T23:33:30.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.752+0000 7fa602000640 1 --2- 172.21.15.94:0/2238850379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5fc0d0ea0 0x7fa5fc0b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.752+0000 7fa602000640 1 --2- 172.21.15.94:0/2238850379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc0b3200 0x7fa5fc0d49c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.752+0000 7fa602000640 1 -- 172.21.15.94:0/2238850379 >> 172.21.15.94:0/2238850379 conn(0x7fa5fc0d3ea0 msgr2=0x7fa5fc0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:30.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.752+0000 7fa602000640 1 -- 172.21.15.94:0/2238850379 shutdown_connections 2024-09-10T23:33:30.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.752+0000 7fa602000640 1 -- 172.21.15.94:0/2238850379 wait complete. 2024-09-10T23:33:30.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa602000640 1 Processor -- start 2024-09-10T23:33:30.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa602000640 1 -- start start 2024-09-10T23:33:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa602000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5fc0b3200 0x7fa5fc0cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa602000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc0d0ea0 0x7fa5fc0b9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa602000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5fc15df80 0x7fa5fc156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa602000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc16ac60 con 0x7fa5fc0d0ea0 2024-09-10T23:33:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa602000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc0b1ea0 con 0x7fa5fc0b3200 2024-09-10T23:33:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa602000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa5fc16ade0 con 0x7fa5fc15df80 2024-09-10T23:33:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa5f2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc0d0ea0 0x7fa5fc0b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa5f2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc0d0ea0 0x7fa5fc0b9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60266/0 (socket says 172.21.15.94:60266) 2024-09-10T23:33:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa5f2a00640 1 -- 172.21.15.94:0/3134373951 learned_addr learned my addr 172.21.15.94:0/3134373951 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa5fb400640 1 --2- 172.21.15.94:0/3134373951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5fc0b3200 0x7fa5fc0cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:30.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa5fbe00640 1 --2- 172.21.15.94:0/3134373951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5fc15df80 0x7fa5fc156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa5f2a00640 1 -- 172.21.15.94:0/3134373951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5fc15df80 msgr2=0x7fa5fc156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa5f2a00640 1 --2- 172.21.15.94:0/3134373951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5fc15df80 0x7fa5fc156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa5f2a00640 1 -- 172.21.15.94:0/3134373951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5fc0b3200 msgr2=0x7fa5fc0cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa5f2a00640 1 --2- 172.21.15.94:0/3134373951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5fc0b3200 0x7fa5fc0cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa5f2a00640 1 -- 172.21.15.94:0/3134373951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5fc0b9a60 con 0x7fa5fc0d0ea0 2024-09-10T23:33:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa5f2a00640 1 --2- 172.21.15.94:0/3134373951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc0d0ea0 0x7fa5fc0b9520 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009850 tx=0x7fa5e40040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa5f3e00640 1 -- 172.21.15.94:0/3134373951 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40b2040 con 0x7fa5fc0d0ea0 2024-09-10T23:33:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa5f3e00640 1 -- 172.21.15.94:0/3134373951 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5e4005030 con 0x7fa5fc0d0ea0 2024-09-10T23:33:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa5f3e00640 1 -- 172.21.15.94:0/3134373951 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4005460 con 0x7fa5fc0d0ea0 2024-09-10T23:33:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa602000640 1 -- 172.21.15.94:0/3134373951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5fc0b9ce0 con 0x7fa5fc0d0ea0 2024-09-10T23:33:30.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.756+0000 7fa602000640 1 -- 172.21.15.94:0/3134373951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5fc0b8bd0 con 0x7fa5fc0d0ea0 2024-09-10T23:33:30.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.760+0000 7fa602000640 1 -- 172.21.15.94:0/3134373951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5fc0d49c0 con 0x7fa5fc0d0ea0 2024-09-10T23:33:30.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.760+0000 7fa5f3e00640 1 -- 172.21.15.94:0/3134373951 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa5e40c0060 con 0x7fa5fc0d0ea0 2024-09-10T23:33:30.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.760+0000 7fa5f3e00640 1 --2- 172.21.15.94:0/3134373951 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa5d80725e0 0x7fa5d8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:30.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.760+0000 7fa5f3e00640 1 -- 172.21.15.94:0/3134373951 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa5e412cf40 con 0x7fa5fc0d0ea0 2024-09-10T23:33:30.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.760+0000 7fa5fb400640 1 --2- 172.21.15.94:0/3134373951 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa5d80725e0 0x7fa5d8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:30.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.760+0000 7fa5fb400640 1 --2- 172.21.15.94:0/3134373951 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa5d80725e0 0x7fa5d8074aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0b54c0 tx=0x7fa5ec009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:30.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.764+0000 7fa5f3e00640 1 -- 172.21.15.94:0/3134373951 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa5fc0b9a60 con 0x7fa5fc0d0ea0 2024-09-10T23:33:30.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:30.968+0000 7fa602000640 1 -- 172.21.15.94:0/3134373951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa5fc156fc0 con 0x7fa5fc0d0ea0 2024-09-10T23:33:31.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:31.896+0000 7fa5f3e00640 1 -- 172.21.15.94:0/3134373951 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fa5e40164d0 con 0x7fa5fc0d0ea0 2024-09-10T23:33:31.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:31.896+0000 7fa602000640 1 -- 172.21.15.94:0/3134373951 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa5d80725e0 msgr2=0x7fa5d8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:31.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:31.896+0000 7fa602000640 1 --2- 172.21.15.94:0/3134373951 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa5d80725e0 0x7fa5d8074aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0b54c0 tx=0x7fa5ec009040 comp rx=0 tx=0).stop 2024-09-10T23:33:31.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:31.896+0000 7fa602000640 1 -- 172.21.15.94:0/3134373951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc0d0ea0 msgr2=0x7fa5fc0b9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:31.896+0000 7fa602000640 1 --2- 172.21.15.94:0/3134373951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc0d0ea0 0x7fa5fc0b9520 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009850 tx=0x7fa5e40040e0 comp rx=0 tx=0).stop 2024-09-10T23:33:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:31.896+0000 7fa602000640 1 -- 172.21.15.94:0/3134373951 shutdown_connections 2024-09-10T23:33:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:31.896+0000 7fa602000640 1 --2- 172.21.15.94:0/3134373951 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa5d80725e0 0x7fa5d8074aa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:31.896+0000 7fa602000640 1 --2- 172.21.15.94:0/3134373951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5fc15df80 0x7fa5fc156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:31.896+0000 7fa602000640 1 --2- 172.21.15.94:0/3134373951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc0d0ea0 0x7fa5fc0b9520 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:31.896+0000 7fa602000640 1 --2- 172.21.15.94:0/3134373951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5fc0b3200 0x7fa5fc0cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:31.896+0000 7fa602000640 1 -- 172.21.15.94:0/3134373951 >> 172.21.15.94:0/3134373951 conn(0x7fa5fc0d3ea0 msgr2=0x7fa5fc06ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:31.900+0000 7fa602000640 1 -- 172.21.15.94:0/3134373951 shutdown_connections 2024-09-10T23:33:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:31.900+0000 7fa602000640 1 -- 172.21.15.94:0/3134373951 wait complete. 2024-09-10T23:33:31.925 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T23:33:31.926 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T23:33:32.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.068+0000 7f14e2e00640 1 -- 172.21.15.94:0/770049802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c0004d50 msgr2=0x7f14c00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:32.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.068+0000 7f14e2e00640 1 --2- 172.21.15.94:0/770049802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c0004d50 0x7f14c00051d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009ee0 tx=0x7f14d401d630 comp rx=0 tx=0).stop 2024-09-10T23:33:32.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.068+0000 7f14e0e00640 1 -- 172.21.15.94:0/770049802 reap_dead start 2024-09-10T23:33:32.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.068+0000 7f14e2e00640 1 -- 172.21.15.94:0/770049802 shutdown_connections 2024-09-10T23:33:32.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.068+0000 7f14e2e00640 1 -- 172.21.15.94:0/770049802 >> 172.21.15.94:0/770049802 conn(0x7f14dc0cfaf0 msgr2=0x7f14dc0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:32.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.068+0000 7f14e2e00640 1 -- 172.21.15.94:0/770049802 shutdown_connections 2024-09-10T23:33:32.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.068+0000 7f14e2e00640 1 -- 172.21.15.94:0/770049802 wait complete. 2024-09-10T23:33:32.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.068+0000 7f14e2e00640 1 Processor -- start 2024-09-10T23:33:32.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.068+0000 7f14e2e00640 1 -- start start 2024-09-10T23:33:32.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14e2e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c0004d50 0x7f14dc0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:32.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14e2e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14dc14bcf0 0x7f14dc14c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:32.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14e2e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14dc0b4270 0x7f14dc0b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:32.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14e2e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc0cf400 con 0x7f14c0004d50 2024-09-10T23:33:32.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14e2e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc0cf280 con 0x7f14dc0b4270 2024-09-10T23:33:32.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14e2e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f14dc0cf580 con 0x7f14dc14bcf0 2024-09-10T23:33:32.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14dbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c0004d50 0x7f14dc0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:32.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14dbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c0004d50 0x7f14dc0afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60284/0 (socket says 172.21.15.94:60284) 2024-09-10T23:33:32.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14dbe00640 1 -- 172.21.15.94:0/2625484688 learned_addr learned my addr 172.21.15.94:0/2625484688 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:32.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14e0e00640 1 --2- 172.21.15.94:0/2625484688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14dc0b4270 0x7f14dc0b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:32.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14d3400640 1 --2- 172.21.15.94:0/2625484688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14dc14bcf0 0x7f14dc14c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:32.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14dbe00640 1 -- 172.21.15.94:0/2625484688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14dc14bcf0 msgr2=0x7f14dc14c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:32.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14dbe00640 1 --2- 172.21.15.94:0/2625484688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14dc14bcf0 0x7f14dc14c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:32.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14dbe00640 1 -- 172.21.15.94:0/2625484688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14dc0b4270 msgr2=0x7f14dc0b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:32.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14dbe00640 1 --2- 172.21.15.94:0/2625484688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14dc0b4270 0x7f14dc0b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:32.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14dbe00640 1 -- 172.21.15.94:0/2625484688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc150640 con 0x7f14c0004d50 2024-09-10T23:33:32.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14d3400640 1 --2- 172.21.15.94:0/2625484688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14dc14bcf0 0x7f14dc14c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:32.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14e0e00640 1 --2- 172.21.15.94:0/2625484688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14dc0b4270 0x7f14dc0b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:32.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14dbe00640 1 --2- 172.21.15.94:0/2625484688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c0004d50 0x7f14dc0afed0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f14cc001870 tx=0x7f14cc00ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:32.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14d8c00640 1 -- 172.21.15.94:0/2625484688 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc015070 con 0x7f14c0004d50 2024-09-10T23:33:32.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14d8c00640 1 -- 172.21.15.94:0/2625484688 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14cc00ca60 con 0x7f14c0004d50 2024-09-10T23:33:32.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14d8c00640 1 -- 172.21.15.94:0/2625484688 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc011460 con 0x7f14c0004d50 2024-09-10T23:33:32.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14e2e00640 1 -- 172.21.15.94:0/2625484688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14dc150920 con 0x7f14c0004d50 2024-09-10T23:33:32.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14e2e00640 1 -- 172.21.15.94:0/2625484688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14dc150ee0 con 0x7f14c0004d50 2024-09-10T23:33:32.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.072+0000 7f14e2e00640 1 -- 172.21.15.94:0/2625484688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14dc1671e0 con 0x7f14c0004d50 2024-09-10T23:33:32.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.076+0000 7f14d8c00640 1 -- 172.21.15.94:0/2625484688 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f14cc011600 con 0x7f14c0004d50 2024-09-10T23:33:32.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.076+0000 7f14d8c00640 1 --2- 172.21.15.94:0/2625484688 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f14b80725e0 0x7f14b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:32.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.076+0000 7f14d8c00640 1 -- 172.21.15.94:0/2625484688 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f14cc093a90 con 0x7f14c0004d50 2024-09-10T23:33:32.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.076+0000 7f14d3400640 1 --2- 172.21.15.94:0/2625484688 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f14b80725e0 0x7f14b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:32.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.076+0000 7f14d3400640 1 --2- 172.21.15.94:0/2625484688 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f14b80725e0 0x7f14b8074aa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f14dc152ef0 tx=0x7f14c40091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:32.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.080+0000 7f14d8c00640 1 -- 172.21.15.94:0/2625484688 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f14cc05f0d0 con 0x7f14c0004d50 2024-09-10T23:33:32.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.280+0000 7f14e2e00640 1 -- 172.21.15.94:0/2625484688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f14dc0d0850 con 0x7f14c0004d50 2024-09-10T23:33:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.900+0000 7f14d8c00640 1 -- 172.21.15.94:0/2625484688 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f14cc05e9d0 con 0x7f14c0004d50 2024-09-10T23:33:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.900+0000 7f14e2e00640 1 -- 172.21.15.94:0/2625484688 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f14b80725e0 msgr2=0x7f14b8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.900+0000 7f14e2e00640 1 --2- 172.21.15.94:0/2625484688 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f14b80725e0 0x7f14b8074aa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f14dc152ef0 tx=0x7f14c40091e0 comp rx=0 tx=0).stop 2024-09-10T23:33:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.900+0000 7f14e2e00640 1 -- 172.21.15.94:0/2625484688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c0004d50 msgr2=0x7f14dc0afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.900+0000 7f14e2e00640 1 --2- 172.21.15.94:0/2625484688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c0004d50 0x7f14dc0afed0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f14cc001870 tx=0x7f14cc00ea90 comp rx=0 tx=0).stop 2024-09-10T23:33:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.900+0000 7f14e2e00640 1 -- 172.21.15.94:0/2625484688 shutdown_connections 2024-09-10T23:33:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.900+0000 7f14e2e00640 1 --2- 172.21.15.94:0/2625484688 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f14b80725e0 0x7f14b8074aa0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.900+0000 7f14e2e00640 1 --2- 172.21.15.94:0/2625484688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14dc0b4270 0x7f14dc0b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.900+0000 7f14e2e00640 1 --2- 172.21.15.94:0/2625484688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14dc14bcf0 0x7f14dc14c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.900+0000 7f14e2e00640 1 --2- 172.21.15.94:0/2625484688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14c0004d50 0x7f14dc0afed0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:32.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.900+0000 7f14e2e00640 1 -- 172.21.15.94:0/2625484688 >> 172.21.15.94:0/2625484688 conn(0x7f14dc0cfaf0 msgr2=0x7f14dc0ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:32.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.900+0000 7f14e2e00640 1 -- 172.21.15.94:0/2625484688 shutdown_connections 2024-09-10T23:33:32.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:32.900+0000 7f14e2e00640 1 -- 172.21.15.94:0/2625484688 wait complete. 2024-09-10T23:33:32.927 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T23:33:32.927 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:33:33.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.068+0000 7efd67200640 1 -- 172.21.15.94:0/1208224487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd6015e250 msgr2=0x7efd6015d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:33.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.068+0000 7efd67200640 1 --2- 172.21.15.94:0/1208224487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd6015e250 0x7efd6015d640 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efd600713b0 tx=0x7efd5001c7d0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.068+0000 7efd67200640 1 -- 172.21.15.94:0/1208224487 shutdown_connections 2024-09-10T23:33:33.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.068+0000 7efd67200640 1 --2- 172.21.15.94:0/1208224487 >> v2:172.21.15.181:3300/0 conn(0x7efd6015db80 0x7efd600b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.068+0000 7efd67200640 1 --2- 172.21.15.94:0/1208224487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd6015e250 0x7efd6015d640 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.068+0000 7efd67200640 1 -- 172.21.15.94:0/1208224487 >> 172.21.15.94:0/1208224487 conn(0x7efd600b1840 msgr2=0x7efd600b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:33.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.068+0000 7efd67200640 1 -- 172.21.15.94:0/1208224487 shutdown_connections 2024-09-10T23:33:33.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.068+0000 7efd67200640 1 -- 172.21.15.94:0/1208224487 wait complete. 2024-09-10T23:33:33.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.068+0000 7efd67200640 1 Processor -- start 2024-09-10T23:33:33.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.068+0000 7efd67200640 1 -- start start 2024-09-10T23:33:33.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd67200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd60159ad0 0x7efd60142860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:33.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd67200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd6015db80 0x7efd601fb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd67200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd6015e250 0x7efd600af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd67200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd60164940 con 0x7efd60159ad0 2024-09-10T23:33:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd67200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efd601647c0 con 0x7efd6015db80 2024-09-10T23:33:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd67200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7efd60164ac0 con 0x7efd6015e250 2024-09-10T23:33:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd65e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd60159ad0 0x7efd60142860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd65e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd60159ad0 0x7efd60142860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60312/0 (socket says 172.21.15.94:60312) 2024-09-10T23:33:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd65e00640 1 -- 172.21.15.94:0/3209041680 learned_addr learned my addr 172.21.15.94:0/3209041680 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd66800640 1 --2- 172.21.15.94:0/3209041680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd6015e250 0x7efd600af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd65400640 1 --2- 172.21.15.94:0/3209041680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd6015db80 0x7efd601fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd65e00640 1 -- 172.21.15.94:0/3209041680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd6015e250 msgr2=0x7efd600af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd65e00640 1 --2- 172.21.15.94:0/3209041680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd6015e250 0x7efd600af6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd65e00640 1 -- 172.21.15.94:0/3209041680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd6015db80 msgr2=0x7efd601fb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:33.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd65e00640 1 --2- 172.21.15.94:0/3209041680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd6015db80 0x7efd601fb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd65e00640 1 -- 172.21.15.94:0/3209041680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd601fb8c0 con 0x7efd60159ad0 2024-09-10T23:33:33.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd65400640 1 --2- 172.21.15.94:0/3209041680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd6015db80 0x7efd601fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:33.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd66800640 1 --2- 172.21.15.94:0/3209041680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd6015e250 0x7efd600af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:33.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd65e00640 1 --2- 172.21.15.94:0/3209041680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd60159ad0 0x7efd60142860 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efd5400c490 tx=0x7efd5400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:33.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd5a000640 1 -- 172.21.15.94:0/3209041680 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54016020 con 0x7efd60159ad0 2024-09-10T23:33:33.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd5a000640 1 -- 172.21.15.94:0/3209041680 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd54012940 con 0x7efd60159ad0 2024-09-10T23:33:33.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd67200640 1 -- 172.21.15.94:0/3209041680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd601f7690 con 0x7efd60159ad0 2024-09-10T23:33:33.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd5a000640 1 -- 172.21.15.94:0/3209041680 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54012d90 con 0x7efd60159ad0 2024-09-10T23:33:33.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd67200640 1 -- 172.21.15.94:0/3209041680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd601f7c50 con 0x7efd60159ad0 2024-09-10T23:33:33.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.072+0000 7efd67200640 1 -- 172.21.15.94:0/3209041680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd600b3340 con 0x7efd60159ad0 2024-09-10T23:33:33.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.076+0000 7efd5a000640 1 -- 172.21.15.94:0/3209041680 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efd5401b400 con 0x7efd60159ad0 2024-09-10T23:33:33.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.076+0000 7efd5a000640 1 --2- 172.21.15.94:0/3209041680 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efd440725e0 0x7efd44074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:33.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.076+0000 7efd5a000640 1 -- 172.21.15.94:0/3209041680 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7efd540935d0 con 0x7efd60159ad0 2024-09-10T23:33:33.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.076+0000 7efd65400640 1 --2- 172.21.15.94:0/3209041680 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efd440725e0 0x7efd44074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:33.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.076+0000 7efd65400640 1 --2- 172.21.15.94:0/3209041680 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efd440725e0 0x7efd44074aa0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efd500af000 tx=0x7efd5001b010 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:33.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.080+0000 7efd5a000640 1 -- 172.21.15.94:0/3209041680 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efd54020020 con 0x7efd60159ad0 2024-09-10T23:33:33.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.280+0000 7efd67200640 1 -- 172.21.15.94:0/3209041680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd600cf870 con 0x7efd60159ad0 2024-09-10T23:33:33.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.280+0000 7efd5a000640 1 -- 172.21.15.94:0/3209041680 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5238 (secure 0 0 0) 0x7efd601fb8c0 con 0x7efd60159ad0 2024-09-10T23:33:33.286 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:33.286 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":16,"btime":"2024-09-10T23:33:32:900364+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4900,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/765435890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":765435890},{"type":"v1","addr":"172.21.15.181:6837","nonce":765435890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3766049154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3766049154},{"type":"v1","addr":"172.21.15.94:6834","nonce":3766049154}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":194555,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2464857180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2464857180},{"type":"v1","addr":"172.21.15.94:6835","nonce":2464857180}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:29.261378+0000","modified":"2024-09-10T23:33:32.284558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194558},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194558":{"gid":194558,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1569801556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1569801556},{"type":"v1","addr":"172.21.15.181:6836","nonce":1569801556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194558,"qdb_cluster":[194558]},"id":2}]} 2024-09-10T23:33:33.286 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 16 2024-09-10T23:33:33.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.284+0000 7efd67200640 1 -- 172.21.15.94:0/3209041680 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efd440725e0 msgr2=0x7efd44074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:33.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.284+0000 7efd67200640 1 --2- 172.21.15.94:0/3209041680 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efd440725e0 0x7efd44074aa0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efd500af000 tx=0x7efd5001b010 comp rx=0 tx=0).stop 2024-09-10T23:33:33.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.284+0000 7efd67200640 1 -- 172.21.15.94:0/3209041680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd60159ad0 msgr2=0x7efd60142860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:33.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.284+0000 7efd67200640 1 --2- 172.21.15.94:0/3209041680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd60159ad0 0x7efd60142860 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efd5400c490 tx=0x7efd5400c950 comp rx=0 tx=0).stop 2024-09-10T23:33:33.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.284+0000 7efd67200640 1 -- 172.21.15.94:0/3209041680 shutdown_connections 2024-09-10T23:33:33.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.284+0000 7efd67200640 1 --2- 172.21.15.94:0/3209041680 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efd440725e0 0x7efd44074aa0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.284+0000 7efd67200640 1 --2- 172.21.15.94:0/3209041680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd6015db80 0x7efd601fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.284+0000 7efd67200640 1 --2- 172.21.15.94:0/3209041680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd6015e250 0x7efd600af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.284+0000 7efd67200640 1 --2- 172.21.15.94:0/3209041680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd60159ad0 0x7efd60142860 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.284+0000 7efd67200640 1 -- 172.21.15.94:0/3209041680 >> 172.21.15.94:0/3209041680 conn(0x7efd600b1840 msgr2=0x7efd601574b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:33.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.284+0000 7efd67200640 1 -- 172.21.15.94:0/3209041680 shutdown_connections 2024-09-10T23:33:33.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.284+0000 7efd67200640 1 -- 172.21.15.94:0/3209041680 wait complete. 2024-09-10T23:33:33.307 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:33:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.444+0000 7fcdede00640 1 -- 172.21.15.94:0/2335437454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcde80cfd50 msgr2=0x7fcde80d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.444+0000 7fcdede00640 1 --2- 172.21.15.94:0/2335437454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcde80cfd50 0x7fcde80d0200 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009840 tx=0x7fcde001c820 comp rx=0 tx=0).stop 2024-09-10T23:33:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.444+0000 7fcdede00640 1 -- 172.21.15.94:0/2335437454 shutdown_connections 2024-09-10T23:33:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.444+0000 7fcdede00640 1 --2- 172.21.15.94:0/2335437454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcde80cfd50 0x7fcde80d0200 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.444+0000 7fcdede00640 1 -- 172.21.15.94:0/2335437454 >> 172.21.15.94:0/2335437454 conn(0x7fcde806cdf0 msgr2=0x7fcde806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:33.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.444+0000 7fcdede00640 1 -- 172.21.15.94:0/2335437454 shutdown_connections 2024-09-10T23:33:33.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.444+0000 7fcdede00640 1 -- 172.21.15.94:0/2335437454 wait complete. 2024-09-10T23:33:33.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.444+0000 7fcdede00640 1 Processor -- start 2024-09-10T23:33:33.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.444+0000 7fcdede00640 1 -- start start 2024-09-10T23:33:33.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcdede00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcde80cfd50 0x7fcde814af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:33.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcdede00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcde8152c10 0x7fcde8148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:33.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcdede00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcde815cbc0 0x7fcde806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:33.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcdede00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcde8169dd0 con 0x7fcde815cbc0 2024-09-10T23:33:33.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcdede00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcde8169c50 con 0x7fcde80cfd50 2024-09-10T23:33:33.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcdede00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fcde8169f50 con 0x7fcde8152c10 2024-09-10T23:33:33.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcde7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcde815cbc0 0x7fcde806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:33.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcde7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcde815cbc0 0x7fcde806f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60340/0 (socket says 172.21.15.94:60340) 2024-09-10T23:33:33.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcde7400640 1 -- 172.21.15.94:0/2973851354 learned_addr learned my addr 172.21.15.94:0/2973851354 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:33.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcde6000640 1 --2- 172.21.15.94:0/2973851354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcde8152c10 0x7fcde8148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:33.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcde6a00640 1 --2- 172.21.15.94:0/2973851354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcde80cfd50 0x7fcde814af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:33.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcde7400640 1 -- 172.21.15.94:0/2973851354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcde8152c10 msgr2=0x7fcde8148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:33.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcde7400640 1 --2- 172.21.15.94:0/2973851354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcde8152c10 0x7fcde8148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcde7400640 1 -- 172.21.15.94:0/2973851354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcde80cfd50 msgr2=0x7fcde814af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:33.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcde7400640 1 --2- 172.21.15.94:0/2973851354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcde80cfd50 0x7fcde814af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcde7400640 1 -- 172.21.15.94:0/2973851354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde8145dd0 con 0x7fcde815cbc0 2024-09-10T23:33:33.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcde6000640 1 --2- 172.21.15.94:0/2973851354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcde8152c10 0x7fcde8148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:33.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcde7400640 1 --2- 172.21.15.94:0/2973851354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcde815cbc0 0x7fcde806f590 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009790 tx=0x7fcde0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:33.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcddf400640 1 -- 172.21.15.94:0/2973851354 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde00bb070 con 0x7fcde815cbc0 2024-09-10T23:33:33.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcddf400640 1 -- 172.21.15.94:0/2973851354 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcde0005030 con 0x7fcde815cbc0 2024-09-10T23:33:33.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcdede00640 1 -- 172.21.15.94:0/2973851354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde814b480 con 0x7fcde815cbc0 2024-09-10T23:33:33.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcddf400640 1 -- 172.21.15.94:0/2973851354 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde00b2040 con 0x7fcde815cbc0 2024-09-10T23:33:33.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcdede00640 1 -- 172.21.15.94:0/2973851354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde8145af0 con 0x7fcde815cbc0 2024-09-10T23:33:33.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcdede00640 1 -- 172.21.15.94:0/2973851354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde80d0200 con 0x7fcde815cbc0 2024-09-10T23:33:33.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.448+0000 7fcddf400640 1 -- 172.21.15.94:0/2973851354 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcde00b5030 con 0x7fcde815cbc0 2024-09-10T23:33:33.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.452+0000 7fcddf400640 1 --2- 172.21.15.94:0/2973851354 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcdc80725e0 0x7fcdc8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:33.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.452+0000 7fcddf400640 1 -- 172.21.15.94:0/2973851354 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fcde012cc30 con 0x7fcde815cbc0 2024-09-10T23:33:33.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.452+0000 7fcde6a00640 1 --2- 172.21.15.94:0/2973851354 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcdc80725e0 0x7fcdc8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:33.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.452+0000 7fcde6a00640 1 --2- 172.21.15.94:0/2973851354 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcdc80725e0 0x7fcdc8074aa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8007fd0 tx=0x7fcdd8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:33.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.456+0000 7fcddf400640 1 -- 172.21.15.94:0/2973851354 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcde0016410 con 0x7fcde815cbc0 2024-09-10T23:33:33.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.620+0000 7fcdede00640 1 -- 172.21.15.94:0/2973851354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcde80b46c0 con 0x7fcde815cbc0 2024-09-10T23:33:33.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.620+0000 7fcddf400640 1 -- 172.21.15.94:0/2973851354 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16657 (secure 0 0 0) 0x7fcde00f8300 con 0x7fcde815cbc0 2024-09-10T23:33:33.626 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:33.627 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":33,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:33:30.886007+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:33:27.370128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T23:33:28.229333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:33:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.624+0000 7fcdede00640 1 -- 172.21.15.94:0/2973851354 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcdc80725e0 msgr2=0x7fcdc8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.624+0000 7fcdede00640 1 --2- 172.21.15.94:0/2973851354 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcdc80725e0 0x7fcdc8074aa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8007fd0 tx=0x7fcdd8009040 comp rx=0 tx=0).stop 2024-09-10T23:33:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.624+0000 7fcdede00640 1 -- 172.21.15.94:0/2973851354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcde815cbc0 msgr2=0x7fcde806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.624+0000 7fcdede00640 1 --2- 172.21.15.94:0/2973851354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcde815cbc0 0x7fcde806f590 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009790 tx=0x7fcde0004040 comp rx=0 tx=0).stop 2024-09-10T23:33:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.624+0000 7fcdede00640 1 -- 172.21.15.94:0/2973851354 shutdown_connections 2024-09-10T23:33:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.624+0000 7fcdede00640 1 --2- 172.21.15.94:0/2973851354 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcdc80725e0 0x7fcdc8074aa0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.624+0000 7fcdede00640 1 --2- 172.21.15.94:0/2973851354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcde815cbc0 0x7fcde806f590 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.624+0000 7fcdede00640 1 --2- 172.21.15.94:0/2973851354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcde8152c10 0x7fcde8148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.624+0000 7fcdede00640 1 --2- 172.21.15.94:0/2973851354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcde80cfd50 0x7fcde814af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.624+0000 7fcdede00640 1 -- 172.21.15.94:0/2973851354 >> 172.21.15.94:0/2973851354 conn(0x7fcde806cdf0 msgr2=0x7fcde806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.624+0000 7fcdede00640 1 -- 172.21.15.94:0/2973851354 shutdown_connections 2024-09-10T23:33:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.624+0000 7fcdede00640 1 -- 172.21.15.94:0/2973851354 wait complete. 2024-09-10T23:33:33.647 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T23:33:33.648 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:33:33.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.784+0000 7fb44ca00640 1 -- 172.21.15.94:0/2978826166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb44806ce80 msgr2=0x7fb44806f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:33.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.784+0000 7fb44ca00640 1 --2- 172.21.15.94:0/2978826166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb44806ce80 0x7fb44806f430 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb438009840 tx=0x7fb43801c6d0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.784+0000 7fb44ca00640 1 -- 172.21.15.94:0/2978826166 shutdown_connections 2024-09-10T23:33:33.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.784+0000 7fb44ca00640 1 --2- 172.21.15.94:0/2978826166 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb44806d260 0x7fb448070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.784+0000 7fb44ca00640 1 --2- 172.21.15.94:0/2978826166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb44806ce80 0x7fb44806f430 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.784+0000 7fb44ca00640 1 -- 172.21.15.94:0/2978826166 >> 172.21.15.94:0/2978826166 conn(0x7fb4480b2520 msgr2=0x7fb4480b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:33.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.784+0000 7fb44ca00640 1 -- 172.21.15.94:0/2978826166 shutdown_connections 2024-09-10T23:33:33.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.784+0000 7fb44ca00640 1 -- 172.21.15.94:0/2978826166 wait complete. 2024-09-10T23:33:33.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.784+0000 7fb44ca00640 1 Processor -- start 2024-09-10T23:33:33.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.784+0000 7fb44ca00640 1 -- start start 2024-09-10T23:33:33.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb44ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb44806ce80 0x7fb4480cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:33.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb44ca00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb44806d260 0x7fb4481528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:33.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb44ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb448070fc0 0x7fb4481fafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:33.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb44ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb44815d9e0 con 0x7fb44806ce80 2024-09-10T23:33:33.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb44ca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb44815d860 con 0x7fb448070fc0 2024-09-10T23:33:33.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb44ca00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb44815db60 con 0x7fb44806d260 2024-09-10T23:33:33.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb447400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb44806ce80 0x7fb4480cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:33.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb447400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb44806ce80 0x7fb4480cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34034/0 (socket says 172.21.15.94:34034) 2024-09-10T23:33:33.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb447400640 1 -- 172.21.15.94:0/2857150097 learned_addr learned my addr 172.21.15.94:0/2857150097 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:33.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb447400640 1 -- 172.21.15.94:0/2857150097 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb44806d260 msgr2=0x7fb4481528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:33.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb447400640 1 --2- 172.21.15.94:0/2857150097 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb44806d260 0x7fb4481528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb447400640 1 -- 172.21.15.94:0/2857150097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb448070fc0 msgr2=0x7fb4481fafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:33.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb447400640 1 --2- 172.21.15.94:0/2857150097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb448070fc0 0x7fb4481fafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb447400640 1 -- 172.21.15.94:0/2857150097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4481532e0 con 0x7fb44806ce80 2024-09-10T23:33:33.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb447400640 1 --2- 172.21.15.94:0/2857150097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb44806ce80 0x7fb4480cd730 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb43801ad80 tx=0x7fb4380a5610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:33.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb43f400640 1 -- 172.21.15.94:0/2857150097 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4380ba020 con 0x7fb44806ce80 2024-09-10T23:33:33.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb43f400640 1 -- 172.21.15.94:0/2857150097 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4380043b0 con 0x7fb44806ce80 2024-09-10T23:33:33.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb44ca00640 1 -- 172.21.15.94:0/2857150097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4481535c0 con 0x7fb44806ce80 2024-09-10T23:33:33.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb43f400640 1 -- 172.21.15.94:0/2857150097 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4380b2040 con 0x7fb44806ce80 2024-09-10T23:33:33.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb44ca00640 1 -- 172.21.15.94:0/2857150097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb448153b80 con 0x7fb44806ce80 2024-09-10T23:33:33.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.788+0000 7fb44ca00640 1 -- 172.21.15.94:0/2857150097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb44815e0c0 con 0x7fb44806ce80 2024-09-10T23:33:33.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.792+0000 7fb43f400640 1 -- 172.21.15.94:0/2857150097 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb4380b5070 con 0x7fb44806ce80 2024-09-10T23:33:33.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.792+0000 7fb43f400640 1 --2- 172.21.15.94:0/2857150097 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4240725e0 0x7fb424074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:33.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.792+0000 7fb43f400640 1 -- 172.21.15.94:0/2857150097 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb43812d150 con 0x7fb44806ce80 2024-09-10T23:33:33.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.792+0000 7fb43fe00640 1 --2- 172.21.15.94:0/2857150097 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4240725e0 0x7fb424074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:33.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.792+0000 7fb43fe00640 1 --2- 172.21.15.94:0/2857150097 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4240725e0 0x7fb424074aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4300058d0 tx=0x7fb430005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:33.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.796+0000 7fb43f400640 1 -- 172.21.15.94:0/2857150097 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb4380c4020 con 0x7fb44806ce80 2024-09-10T23:33:33.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.960+0000 7fb44ca00640 1 -- 172.21.15.94:0/2857150097 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb4480b17d0 con 0x7fb4240725e0 2024-09-10T23:33:33.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.964+0000 7fb43f400640 1 -- 172.21.15.94:0/2857150097 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251942 (secure 0 0 0) 0x7fb4480b17d0 con 0x7fb4240725e0 2024-09-10T23:33:33.971 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:33.972 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:33:33.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.968+0000 7fb44ca00640 1 -- 172.21.15.94:0/2857150097 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4240725e0 msgr2=0x7fb424074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:33.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.968+0000 7fb44ca00640 1 --2- 172.21.15.94:0/2857150097 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4240725e0 0x7fb424074aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4300058d0 tx=0x7fb430005860 comp rx=0 tx=0).stop 2024-09-10T23:33:33.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.968+0000 7fb44ca00640 1 -- 172.21.15.94:0/2857150097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb44806ce80 msgr2=0x7fb4480cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:33.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.968+0000 7fb44ca00640 1 --2- 172.21.15.94:0/2857150097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb44806ce80 0x7fb4480cd730 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb43801ad80 tx=0x7fb4380a5610 comp rx=0 tx=0).stop 2024-09-10T23:33:33.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.968+0000 7fb44ca00640 1 -- 172.21.15.94:0/2857150097 shutdown_connections 2024-09-10T23:33:33.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.968+0000 7fb44ca00640 1 --2- 172.21.15.94:0/2857150097 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4240725e0 0x7fb424074aa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.968+0000 7fb44ca00640 1 --2- 172.21.15.94:0/2857150097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb448070fc0 0x7fb4481fafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.968+0000 7fb44ca00640 1 --2- 172.21.15.94:0/2857150097 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb44806d260 0x7fb4481528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.968+0000 7fb44ca00640 1 --2- 172.21.15.94:0/2857150097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb44806ce80 0x7fb4480cd730 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:33.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.968+0000 7fb44ca00640 1 -- 172.21.15.94:0/2857150097 >> 172.21.15.94:0/2857150097 conn(0x7fb4480b2520 msgr2=0x7fb4480b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:33.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.968+0000 7fb44ca00640 1 -- 172.21.15.94:0/2857150097 shutdown_connections 2024-09-10T23:33:33.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:33.968+0000 7fb44ca00640 1 -- 172.21.15.94:0/2857150097 wait complete. 2024-09-10T23:33:33.991 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-09-10T23:33:33.353379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236908,"kb_used_data":4024,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737812,"statfs":{"total":773094113280,"available":772851519488,"internally_reserved":0,"allocated":4120576,"data_stored":2276892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.487613"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968115+0000","last_change":"2024-09-10T23:33:29.891760+0000","last_active":"2024-09-10T23:33:30.968115+0000","last_peered":"2024-09-10T23:33:30.968115+0000","last_clean":"2024-09-10T23:33:30.968115+0000","last_became_active":"2024-09-10T23:33:29.891355+0000","last_became_peered":"2024-09-10T23:33:29.891355+0000","last_unstale":"2024-09-10T23:33:30.968115+0000","last_undegraded":"2024-09-10T23:33:30.968115+0000","last_fullsized":"2024-09-10T23:33:30.968115+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:35.604375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.893479+0000","last_change":"2024-09-10T23:33:29.901182+0000","last_active":"2024-09-10T23:33:30.893479+0000","last_peered":"2024-09-10T23:33:30.893479+0000","last_clean":"2024-09-10T23:33:30.893479+0000","last_became_active":"2024-09-10T23:33:29.900024+0000","last_became_peered":"2024-09-10T23:33:29.900024+0000","last_unstale":"2024-09-10T23:33:30.893479+0000","last_undegraded":"2024-09-10T23:33:30.893479+0000","last_fullsized":"2024-09-10T23:33:30.893479+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:17.739516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966272+0000","last_change":"2024-09-10T23:33:29.893113+0000","last_active":"2024-09-10T23:33:30.966272+0000","last_peered":"2024-09-10T23:33:30.966272+0000","last_clean":"2024-09-10T23:33:30.966272+0000","last_became_active":"2024-09-10T23:33:29.892877+0000","last_became_peered":"2024-09-10T23:33:29.892877+0000","last_unstale":"2024-09-10T23:33:30.966272+0000","last_undegraded":"2024-09-10T23:33:30.966272+0000","last_fullsized":"2024-09-10T23:33:30.966272+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:20.427642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967649+0000","last_change":"2024-09-10T23:33:29.897864+0000","last_active":"2024-09-10T23:33:30.967649+0000","last_peered":"2024-09-10T23:33:30.967649+0000","last_clean":"2024-09-10T23:33:30.967649+0000","last_became_active":"2024-09-10T23:33:29.896613+0000","last_became_peered":"2024-09-10T23:33:29.896613+0000","last_unstale":"2024-09-10T23:33:30.967649+0000","last_undegraded":"2024-09-10T23:33:30.967649+0000","last_fullsized":"2024-09-10T23:33:30.967649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:41.550538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.893317+0000","last_change":"2024-09-10T23:33:29.901649+0000","last_active":"2024-09-10T23:33:30.893317+0000","last_peered":"2024-09-10T23:33:30.893317+0000","last_clean":"2024-09-10T23:33:30.893317+0000","last_became_active":"2024-09-10T23:33:29.901232+0000","last_became_peered":"2024-09-10T23:33:29.901232+0000","last_unstale":"2024-09-10T23:33:30.893317+0000","last_undegraded":"2024-09-10T23:33:30.893317+0000","last_fullsized":"2024-09-10T23:33:30.893317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:00.564474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900616+0000","last_change":"2024-09-10T23:33:29.911689+0000","last_active":"2024-09-10T23:33:30.900616+0000","last_peered":"2024-09-10T23:33:30.900616+0000","last_clean":"2024-09-10T23:33:30.900616+0000","last_became_active":"2024-09-10T23:33:29.911343+0000","last_became_peered":"2024-09-10T23:33:29.911343+0000","last_unstale":"2024-09-10T23:33:30.900616+0000","last_undegraded":"2024-09-10T23:33:30.900616+0000","last_fullsized":"2024-09-10T23:33:30.900616+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:27.432397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967711+0000","last_change":"2024-09-10T23:33:29.892589+0000","last_active":"2024-09-10T23:33:30.967711+0000","last_peered":"2024-09-10T23:33:30.967711+0000","last_clean":"2024-09-10T23:33:30.967711+0000","last_became_active":"2024-09-10T23:33:29.892205+0000","last_became_peered":"2024-09-10T23:33:29.892205+0000","last_unstale":"2024-09-10T23:33:30.967711+0000","last_undegraded":"2024-09-10T23:33:30.967711+0000","last_fullsized":"2024-09-10T23:33:30.967711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:39.295916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966983+0000","last_change":"2024-09-10T23:33:29.894105+0000","last_active":"2024-09-10T23:33:30.966983+0000","last_peered":"2024-09-10T23:33:30.966983+0000","last_clean":"2024-09-10T23:33:30.966983+0000","last_became_active":"2024-09-10T23:33:29.893437+0000","last_became_peered":"2024-09-10T23:33:29.893437+0000","last_unstale":"2024-09-10T23:33:30.966983+0000","last_undegraded":"2024-09-10T23:33:30.966983+0000","last_fullsized":"2024-09-10T23:33:30.966983+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:46:08.526625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967149+0000","last_change":"2024-09-10T23:33:29.893307+0000","last_active":"2024-09-10T23:33:30.967149+0000","last_peered":"2024-09-10T23:33:30.967149+0000","last_clean":"2024-09-10T23:33:30.967149+0000","last_became_active":"2024-09-10T23:33:29.892708+0000","last_became_peered":"2024-09-10T23:33:29.892708+0000","last_unstale":"2024-09-10T23:33:30.967149+0000","last_undegraded":"2024-09-10T23:33:30.967149+0000","last_fullsized":"2024-09-10T23:33:30.967149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:44.062732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968096+0000","last_change":"2024-09-10T23:33:29.905924+0000","last_active":"2024-09-10T23:33:30.968096+0000","last_peered":"2024-09-10T23:33:30.968096+0000","last_clean":"2024-09-10T23:33:30.968096+0000","last_became_active":"2024-09-10T23:33:29.905611+0000","last_became_peered":"2024-09-10T23:33:29.905611+0000","last_unstale":"2024-09-10T23:33:30.968096+0000","last_undegraded":"2024-09-10T23:33:30.968096+0000","last_fullsized":"2024-09-10T23:33:30.968096+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:13.026516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891362+0000","last_change":"2024-09-10T23:33:29.897696+0000","last_active":"2024-09-10T23:33:30.891362+0000","last_peered":"2024-09-10T23:33:30.891362+0000","last_clean":"2024-09-10T23:33:30.891362+0000","last_became_active":"2024-09-10T23:33:29.896675+0000","last_became_peered":"2024-09-10T23:33:29.896675+0000","last_unstale":"2024-09-10T23:33:30.891362+0000","last_undegraded":"2024-09-10T23:33:30.891362+0000","last_fullsized":"2024-09-10T23:33:30.891362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:54.877966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966935+0000","last_change":"2024-09-10T23:33:29.900667+0000","last_active":"2024-09-10T23:33:30.966935+0000","last_peered":"2024-09-10T23:33:30.966935+0000","last_clean":"2024-09-10T23:33:30.966935+0000","last_became_active":"2024-09-10T23:33:29.900411+0000","last_became_peered":"2024-09-10T23:33:29.900411+0000","last_unstale":"2024-09-10T23:33:30.966935+0000","last_undegraded":"2024-09-10T23:33:30.966935+0000","last_fullsized":"2024-09-10T23:33:30.966935+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:27.432397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.907645+0000","last_change":"2024-09-10T23:33:29.902156+0000","last_active":"2024-09-10T23:33:29.907645+0000","last_peered":"2024-09-10T23:33:29.907645+0000","last_clean":"2024-09-10T23:33:29.907645+0000","last_became_active":"2024-09-10T23:33:29.901826+0000","last_became_peered":"2024-09-10T23:33:29.901826+0000","last_unstale":"2024-09-10T23:33:29.907645+0000","last_undegraded":"2024-09-10T23:33:29.907645+0000","last_fullsized":"2024-09-10T23:33:29.907645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:53.970887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967553+0000","last_change":"2024-09-10T23:33:29.908002+0000","last_active":"2024-09-10T23:33:30.967553+0000","last_peered":"2024-09-10T23:33:30.967553+0000","last_clean":"2024-09-10T23:33:30.967553+0000","last_became_active":"2024-09-10T23:33:29.905494+0000","last_became_peered":"2024-09-10T23:33:29.905494+0000","last_unstale":"2024-09-10T23:33:30.967553+0000","last_undegraded":"2024-09-10T23:33:30.967553+0000","last_fullsized":"2024-09-10T23:33:30.967553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:17:25.615539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966361+0000","last_change":"2024-09-10T23:33:29.898423+0000","last_active":"2024-09-10T23:33:30.966361+0000","last_peered":"2024-09-10T23:33:30.966361+0000","last_clean":"2024-09-10T23:33:30.966361+0000","last_became_active":"2024-09-10T23:33:29.897425+0000","last_became_peered":"2024-09-10T23:33:29.897425+0000","last_unstale":"2024-09-10T23:33:30.966361+0000","last_undegraded":"2024-09-10T23:33:30.966361+0000","last_fullsized":"2024-09-10T23:33:30.966361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:23.725141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966183+0000","last_change":"2024-09-10T23:33:29.900989+0000","last_active":"2024-09-10T23:33:30.966183+0000","last_peered":"2024-09-10T23:33:30.966183+0000","last_clean":"2024-09-10T23:33:30.966183+0000","last_became_active":"2024-09-10T23:33:29.900743+0000","last_became_peered":"2024-09-10T23:33:29.900743+0000","last_unstale":"2024-09-10T23:33:30.966183+0000","last_undegraded":"2024-09-10T23:33:30.966183+0000","last_fullsized":"2024-09-10T23:33:30.966183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:53.556368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891314+0000","last_change":"2024-09-10T23:33:29.895656+0000","last_active":"2024-09-10T23:33:30.891314+0000","last_peered":"2024-09-10T23:33:30.891314+0000","last_clean":"2024-09-10T23:33:30.891314+0000","last_became_active":"2024-09-10T23:33:29.889689+0000","last_became_peered":"2024-09-10T23:33:29.889689+0000","last_unstale":"2024-09-10T23:33:30.891314+0000","last_undegraded":"2024-09-10T23:33:30.891314+0000","last_fullsized":"2024-09-10T23:33:30.891314+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:15.868218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900863+0000","last_change":"2024-09-10T23:33:29.903174+0000","last_active":"2024-09-10T23:33:30.900863+0000","last_peered":"2024-09-10T23:33:30.900863+0000","last_clean":"2024-09-10T23:33:30.900863+0000","last_became_active":"2024-09-10T23:33:29.902925+0000","last_became_peered":"2024-09-10T23:33:29.902925+0000","last_unstale":"2024-09-10T23:33:30.900863+0000","last_undegraded":"2024-09-10T23:33:30.900863+0000","last_fullsized":"2024-09-10T23:33:30.900863+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:04.980472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900574+0000","last_change":"2024-09-10T23:33:29.900766+0000","last_active":"2024-09-10T23:33:30.900574+0000","last_peered":"2024-09-10T23:33:30.900574+0000","last_clean":"2024-09-10T23:33:30.900574+0000","last_became_active":"2024-09-10T23:33:29.900491+0000","last_became_peered":"2024-09-10T23:33:29.900491+0000","last_unstale":"2024-09-10T23:33:30.900574+0000","last_undegraded":"2024-09-10T23:33:30.900574+0000","last_fullsized":"2024-09-10T23:33:30.900574+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:44.445121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.899206+0000","last_change":"2024-09-10T23:33:29.896368+0000","last_active":"2024-09-10T23:33:29.899206+0000","last_peered":"2024-09-10T23:33:29.899206+0000","last_clean":"2024-09-10T23:33:29.899206+0000","last_became_active":"2024-09-10T23:33:29.896173+0000","last_became_peered":"2024-09-10T23:33:29.896173+0000","last_unstale":"2024-09-10T23:33:29.899206+0000","last_undegraded":"2024-09-10T23:33:29.899206+0000","last_fullsized":"2024-09-10T23:33:29.899206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:53.653730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966830+0000","last_change":"2024-09-10T23:33:29.887720+0000","last_active":"2024-09-10T23:33:30.966830+0000","last_peered":"2024-09-10T23:33:30.966830+0000","last_clean":"2024-09-10T23:33:30.966830+0000","last_became_active":"2024-09-10T23:33:29.887367+0000","last_became_peered":"2024-09-10T23:33:29.887367+0000","last_unstale":"2024-09-10T23:33:30.966830+0000","last_undegraded":"2024-09-10T23:33:30.966830+0000","last_fullsized":"2024-09-10T23:33:30.966830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:17.739516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967739+0000","last_change":"2024-09-10T23:33:29.887429+0000","last_active":"2024-09-10T23:33:30.967739+0000","last_peered":"2024-09-10T23:33:30.967739+0000","last_clean":"2024-09-10T23:33:30.967739+0000","last_became_active":"2024-09-10T23:33:29.887115+0000","last_became_peered":"2024-09-10T23:33:29.887115+0000","last_unstale":"2024-09-10T23:33:30.967739+0000","last_undegraded":"2024-09-10T23:33:30.967739+0000","last_fullsized":"2024-09-10T23:33:30.967739+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:35.273661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.905713+0000","last_change":"2024-09-10T23:33:29.899713+0000","last_active":"2024-09-10T23:33:29.905713+0000","last_peered":"2024-09-10T23:33:29.905713+0000","last_clean":"2024-09-10T23:33:29.905713+0000","last_became_active":"2024-09-10T23:33:29.899305+0000","last_became_peered":"2024-09-10T23:33:29.899305+0000","last_unstale":"2024-09-10T23:33:29.905713+0000","last_undegraded":"2024-09-10T23:33:29.905713+0000","last_fullsized":"2024-09-10T23:33:29.905713+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:47.866442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967173+0000","last_change":"2024-09-10T23:33:29.893860+0000","last_active":"2024-09-10T23:33:30.967173+0000","last_peered":"2024-09-10T23:33:30.967173+0000","last_clean":"2024-09-10T23:33:30.967173+0000","last_became_active":"2024-09-10T23:33:29.893091+0000","last_became_peered":"2024-09-10T23:33:29.893091+0000","last_unstale":"2024-09-10T23:33:30.967173+0000","last_undegraded":"2024-09-10T23:33:30.967173+0000","last_fullsized":"2024-09-10T23:33:30.967173+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:23.778319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968095+0000","last_change":"2024-09-10T23:33:29.908177+0000","last_active":"2024-09-10T23:33:30.968095+0000","last_peered":"2024-09-10T23:33:30.968095+0000","last_clean":"2024-09-10T23:33:30.968095+0000","last_became_active":"2024-09-10T23:33:29.899969+0000","last_became_peered":"2024-09-10T23:33:29.899969+0000","last_unstale":"2024-09-10T23:33:30.968095+0000","last_undegraded":"2024-09-10T23:33:30.968095+0000","last_fullsized":"2024-09-10T23:33:30.968095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:19.657339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968157+0000","last_change":"2024-09-10T23:33:29.889211+0000","last_active":"2024-09-10T23:33:30.968157+0000","last_peered":"2024-09-10T23:33:30.968157+0000","last_clean":"2024-09-10T23:33:30.968157+0000","last_became_active":"2024-09-10T23:33:29.888312+0000","last_became_peered":"2024-09-10T23:33:29.888312+0000","last_unstale":"2024-09-10T23:33:30.968157+0000","last_undegraded":"2024-09-10T23:33:30.968157+0000","last_fullsized":"2024-09-10T23:33:30.968157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:07.940639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.893528+0000","last_change":"2024-09-10T23:33:29.904832+0000","last_active":"2024-09-10T23:33:30.893528+0000","last_peered":"2024-09-10T23:33:30.893528+0000","last_clean":"2024-09-10T23:33:30.893528+0000","last_became_active":"2024-09-10T23:33:29.902828+0000","last_became_peered":"2024-09-10T23:33:29.902828+0000","last_unstale":"2024-09-10T23:33:30.893528+0000","last_undegraded":"2024-09-10T23:33:30.893528+0000","last_fullsized":"2024-09-10T23:33:30.893528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:22.440859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968109+0000","last_change":"2024-09-10T23:33:29.903254+0000","last_active":"2024-09-10T23:33:30.968109+0000","last_peered":"2024-09-10T23:33:30.968109+0000","last_clean":"2024-09-10T23:33:30.968109+0000","last_became_active":"2024-09-10T23:33:29.902942+0000","last_became_peered":"2024-09-10T23:33:29.902942+0000","last_unstale":"2024-09-10T23:33:30.968109+0000","last_undegraded":"2024-09-10T23:33:30.968109+0000","last_fullsized":"2024-09-10T23:33:30.968109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:21.241619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966885+0000","last_change":"2024-09-10T23:33:29.891471+0000","last_active":"2024-09-10T23:33:30.966885+0000","last_peered":"2024-09-10T23:33:30.966885+0000","last_clean":"2024-09-10T23:33:30.966885+0000","last_became_active":"2024-09-10T23:33:29.889128+0000","last_became_peered":"2024-09-10T23:33:29.889128+0000","last_unstale":"2024-09-10T23:33:30.966885+0000","last_undegraded":"2024-09-10T23:33:30.966885+0000","last_fullsized":"2024-09-10T23:33:30.966885+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:22.440859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966570+0000","last_change":"2024-09-10T23:33:29.888402+0000","last_active":"2024-09-10T23:33:30.966570+0000","last_peered":"2024-09-10T23:33:30.966570+0000","last_clean":"2024-09-10T23:33:30.966570+0000","last_became_active":"2024-09-10T23:33:29.888121+0000","last_became_peered":"2024-09-10T23:33:29.888121+0000","last_unstale":"2024-09-10T23:33:30.966570+0000","last_undegraded":"2024-09-10T23:33:30.966570+0000","last_fullsized":"2024-09-10T23:33:30.966570+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:16.077235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967692+0000","last_change":"2024-09-10T23:33:29.903252+0000","last_active":"2024-09-10T23:33:30.967692+0000","last_peered":"2024-09-10T23:33:30.967692+0000","last_clean":"2024-09-10T23:33:30.967692+0000","last_became_active":"2024-09-10T23:33:29.902822+0000","last_became_peered":"2024-09-10T23:33:29.902822+0000","last_unstale":"2024-09-10T23:33:30.967692+0000","last_undegraded":"2024-09-10T23:33:30.967692+0000","last_fullsized":"2024-09-10T23:33:30.967692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:59.841438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.890931+0000","last_change":"2024-09-10T23:33:29.890470+0000","last_active":"2024-09-10T23:33:30.890931+0000","last_peered":"2024-09-10T23:33:30.890931+0000","last_clean":"2024-09-10T23:33:30.890931+0000","last_became_active":"2024-09-10T23:33:29.890158+0000","last_became_peered":"2024-09-10T23:33:29.890158+0000","last_unstale":"2024-09-10T23:33:30.890931+0000","last_undegraded":"2024-09-10T23:33:30.890931+0000","last_fullsized":"2024-09-10T23:33:30.890931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:17.650619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.901171+0000","last_change":"2024-09-10T23:33:29.891670+0000","last_active":"2024-09-10T23:33:29.901171+0000","last_peered":"2024-09-10T23:33:29.901171+0000","last_clean":"2024-09-10T23:33:29.901171+0000","last_became_active":"2024-09-10T23:33:29.891300+0000","last_became_peered":"2024-09-10T23:33:29.891300+0000","last_unstale":"2024-09-10T23:33:29.901171+0000","last_undegraded":"2024-09-10T23:33:29.901171+0000","last_fullsized":"2024-09-10T23:33:29.901171+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:10.129515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891421+0000","last_change":"2024-09-10T23:33:28.899750+0000","last_active":"2024-09-10T23:33:30.891421+0000","last_peered":"2024-09-10T23:33:30.891421+0000","last_clean":"2024-09-10T23:33:30.891421+0000","last_became_active":"2024-09-10T23:33:28.899272+0000","last_became_peered":"2024-09-10T23:33:28.899272+0000","last_unstale":"2024-09-10T23:33:30.891421+0000","last_undegraded":"2024-09-10T23:33:30.891421+0000","last_fullsized":"2024-09-10T23:33:30.891421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:38.289889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968307+0000","last_change":"2024-09-10T23:33:29.905051+0000","last_active":"2024-09-10T23:33:30.968307+0000","last_peered":"2024-09-10T23:33:30.968307+0000","last_clean":"2024-09-10T23:33:30.968307+0000","last_became_active":"2024-09-10T23:33:29.904536+0000","last_became_peered":"2024-09-10T23:33:29.904536+0000","last_unstale":"2024-09-10T23:33:30.968307+0000","last_undegraded":"2024-09-10T23:33:30.968307+0000","last_fullsized":"2024-09-10T23:33:30.968307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:42.507561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968179+0000","last_change":"2024-09-10T23:33:28.897865+0000","last_active":"2024-09-10T23:33:30.968179+0000","last_peered":"2024-09-10T23:33:30.968179+0000","last_clean":"2024-09-10T23:33:30.968179+0000","last_became_active":"2024-09-10T23:33:28.897179+0000","last_became_peered":"2024-09-10T23:33:28.897179+0000","last_unstale":"2024-09-10T23:33:30.968179+0000","last_undegraded":"2024-09-10T23:33:30.968179+0000","last_fullsized":"2024-09-10T23:33:30.968179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:31:55.659032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968104+0000","last_change":"2024-09-10T23:33:29.889304+0000","last_active":"2024-09-10T23:33:30.968104+0000","last_peered":"2024-09-10T23:33:30.968104+0000","last_clean":"2024-09-10T23:33:30.968104+0000","last_became_active":"2024-09-10T23:33:29.889061+0000","last_became_peered":"2024-09-10T23:33:29.889061+0000","last_unstale":"2024-09-10T23:33:30.968104+0000","last_undegraded":"2024-09-10T23:33:30.968104+0000","last_fullsized":"2024-09-10T23:33:30.968104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:30.699198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900762+0000","last_change":"2024-09-10T23:33:28.895241+0000","last_active":"2024-09-10T23:33:30.900762+0000","last_peered":"2024-09-10T23:33:30.900762+0000","last_clean":"2024-09-10T23:33:30.900762+0000","last_became_active":"2024-09-10T23:33:28.894954+0000","last_became_peered":"2024-09-10T23:33:28.894954+0000","last_unstale":"2024-09-10T23:33:30.900762+0000","last_undegraded":"2024-09-10T23:33:30.900762+0000","last_fullsized":"2024-09-10T23:33:30.900762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:39.116940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968025+0000","last_change":"2024-09-10T23:33:29.903886+0000","last_active":"2024-09-10T23:33:30.968025+0000","last_peered":"2024-09-10T23:33:30.968025+0000","last_clean":"2024-09-10T23:33:30.968025+0000","last_became_active":"2024-09-10T23:33:29.903611+0000","last_became_peered":"2024-09-10T23:33:29.903611+0000","last_unstale":"2024-09-10T23:33:30.968025+0000","last_undegraded":"2024-09-10T23:33:30.968025+0000","last_fullsized":"2024-09-10T23:33:30.968025+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:10.777381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967884+0000","last_change":"2024-09-10T23:33:28.887047+0000","last_active":"2024-09-10T23:33:30.967884+0000","last_peered":"2024-09-10T23:33:30.967884+0000","last_clean":"2024-09-10T23:33:30.967884+0000","last_became_active":"2024-09-10T23:33:28.886032+0000","last_became_peered":"2024-09-10T23:33:28.886032+0000","last_unstale":"2024-09-10T23:33:30.967884+0000","last_undegraded":"2024-09-10T23:33:30.967884+0000","last_fullsized":"2024-09-10T23:33:30.967884+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:02.567377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966855+0000","last_change":"2024-09-10T23:33:29.892030+0000","last_active":"2024-09-10T23:33:30.966855+0000","last_peered":"2024-09-10T23:33:30.966855+0000","last_clean":"2024-09-10T23:33:30.966855+0000","last_became_active":"2024-09-10T23:33:29.891793+0000","last_became_peered":"2024-09-10T23:33:29.891793+0000","last_unstale":"2024-09-10T23:33:30.966855+0000","last_undegraded":"2024-09-10T23:33:30.966855+0000","last_fullsized":"2024-09-10T23:33:30.966855+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:44.445121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967953+0000","last_change":"2024-09-10T23:33:28.890106+0000","last_active":"2024-09-10T23:33:30.967953+0000","last_peered":"2024-09-10T23:33:30.967953+0000","last_clean":"2024-09-10T23:33:30.967953+0000","last_became_active":"2024-09-10T23:33:28.889916+0000","last_became_peered":"2024-09-10T23:33:28.889916+0000","last_unstale":"2024-09-10T23:33:30.967953+0000","last_undegraded":"2024-09-10T23:33:30.967953+0000","last_fullsized":"2024-09-10T23:33:30.967953+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:43.056706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968150+0000","last_change":"2024-09-10T23:33:29.911612+0000","last_active":"2024-09-10T23:33:30.968150+0000","last_peered":"2024-09-10T23:33:30.968150+0000","last_clean":"2024-09-10T23:33:30.968150+0000","last_became_active":"2024-09-10T23:33:29.911377+0000","last_became_peered":"2024-09-10T23:33:29.911377+0000","last_unstale":"2024-09-10T23:33:30.968150+0000","last_undegraded":"2024-09-10T23:33:30.968150+0000","last_fullsized":"2024-09-10T23:33:30.968150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:54.486020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967742+0000","last_change":"2024-09-10T23:33:28.886964+0000","last_active":"2024-09-10T23:33:30.967742+0000","last_peered":"2024-09-10T23:33:30.967742+0000","last_clean":"2024-09-10T23:33:30.967742+0000","last_became_active":"2024-09-10T23:33:28.886708+0000","last_became_peered":"2024-09-10T23:33:28.886708+0000","last_unstale":"2024-09-10T23:33:30.967742+0000","last_undegraded":"2024-09-10T23:33:30.967742+0000","last_fullsized":"2024-09-10T23:33:30.967742+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:19.421616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966256+0000","last_change":"2024-09-10T23:33:29.887895+0000","last_active":"2024-09-10T23:33:30.966256+0000","last_peered":"2024-09-10T23:33:30.966256+0000","last_clean":"2024-09-10T23:33:30.966256+0000","last_became_active":"2024-09-10T23:33:29.887679+0000","last_became_peered":"2024-09-10T23:33:29.887679+0000","last_unstale":"2024-09-10T23:33:30.966256+0000","last_undegraded":"2024-09-10T23:33:30.966256+0000","last_fullsized":"2024-09-10T23:33:30.966256+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:34.439962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900447+0000","last_change":"2024-09-10T23:33:28.891708+0000","last_active":"2024-09-10T23:33:30.900447+0000","last_peered":"2024-09-10T23:33:30.900447+0000","last_clean":"2024-09-10T23:33:30.900447+0000","last_became_active":"2024-09-10T23:33:28.891247+0000","last_became_peered":"2024-09-10T23:33:28.891247+0000","last_unstale":"2024-09-10T23:33:30.900447+0000","last_undegraded":"2024-09-10T23:33:30.900447+0000","last_fullsized":"2024-09-10T23:33:30.900447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:47.784412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967310+0000","last_change":"2024-09-10T23:33:29.895718+0000","last_active":"2024-09-10T23:33:30.967310+0000","last_peered":"2024-09-10T23:33:30.967310+0000","last_clean":"2024-09-10T23:33:30.967310+0000","last_became_active":"2024-09-10T23:33:29.895471+0000","last_became_peered":"2024-09-10T23:33:29.895471+0000","last_unstale":"2024-09-10T23:33:30.967310+0000","last_undegraded":"2024-09-10T23:33:30.967310+0000","last_fullsized":"2024-09-10T23:33:30.967310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:22.440859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967449+0000","last_change":"2024-09-10T23:33:28.895041+0000","last_active":"2024-09-10T23:33:30.967449+0000","last_peered":"2024-09-10T23:33:30.967449+0000","last_clean":"2024-09-10T23:33:30.967449+0000","last_became_active":"2024-09-10T23:33:28.893542+0000","last_became_peered":"2024-09-10T23:33:28.893542+0000","last_unstale":"2024-09-10T23:33:30.967449+0000","last_undegraded":"2024-09-10T23:33:30.967449+0000","last_fullsized":"2024-09-10T23:33:30.967449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:00.790159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891369+0000","last_change":"2024-09-10T23:33:29.890004+0000","last_active":"2024-09-10T23:33:30.891369+0000","last_peered":"2024-09-10T23:33:30.891369+0000","last_clean":"2024-09-10T23:33:30.891369+0000","last_became_active":"2024-09-10T23:33:29.889726+0000","last_became_peered":"2024-09-10T23:33:29.889726+0000","last_unstale":"2024-09-10T23:33:30.891369+0000","last_undegraded":"2024-09-10T23:33:30.891369+0000","last_fullsized":"2024-09-10T23:33:30.891369+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:33.225909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.893858+0000","last_change":"2024-09-10T23:33:28.889969+0000","last_active":"2024-09-10T23:33:30.893858+0000","last_peered":"2024-09-10T23:33:30.893858+0000","last_clean":"2024-09-10T23:33:30.893858+0000","last_became_active":"2024-09-10T23:33:28.889612+0000","last_became_peered":"2024-09-10T23:33:28.889612+0000","last_unstale":"2024-09-10T23:33:30.893858+0000","last_undegraded":"2024-09-10T23:33:30.893858+0000","last_fullsized":"2024-09-10T23:33:30.893858+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:22.772293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968063+0000","last_change":"2024-09-10T23:33:29.912019+0000","last_active":"2024-09-10T23:33:30.968063+0000","last_peered":"2024-09-10T23:33:30.968063+0000","last_clean":"2024-09-10T23:33:30.968063+0000","last_became_active":"2024-09-10T23:33:29.908348+0000","last_became_peered":"2024-09-10T23:33:29.908348+0000","last_unstale":"2024-09-10T23:33:30.968063+0000","last_undegraded":"2024-09-10T23:33:30.968063+0000","last_fullsized":"2024-09-10T23:33:30.968063+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:46.777181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.893567+0000","last_change":"2024-09-10T23:33:28.896131+0000","last_active":"2024-09-10T23:33:30.893567+0000","last_peered":"2024-09-10T23:33:30.893567+0000","last_clean":"2024-09-10T23:33:30.893567+0000","last_became_active":"2024-09-10T23:33:28.895323+0000","last_became_peered":"2024-09-10T23:33:28.895323+0000","last_unstale":"2024-09-10T23:33:30.893567+0000","last_undegraded":"2024-09-10T23:33:30.893567+0000","last_fullsized":"2024-09-10T23:33:30.893567+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:03.806158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967703+0000","last_change":"2024-09-10T23:33:29.898994+0000","last_active":"2024-09-10T23:33:30.967703+0000","last_peered":"2024-09-10T23:33:30.967703+0000","last_clean":"2024-09-10T23:33:30.967703+0000","last_became_active":"2024-09-10T23:33:29.896256+0000","last_became_peered":"2024-09-10T23:33:29.896256+0000","last_unstale":"2024-09-10T23:33:30.967703+0000","last_undegraded":"2024-09-10T23:33:30.967703+0000","last_fullsized":"2024-09-10T23:33:30.967703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:42.509513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891125+0000","last_change":"2024-09-10T23:33:28.898024+0000","last_active":"2024-09-10T23:33:30.891125+0000","last_peered":"2024-09-10T23:33:30.891125+0000","last_clean":"2024-09-10T23:33:30.891125+0000","last_became_active":"2024-09-10T23:33:28.897649+0000","last_became_peered":"2024-09-10T23:33:28.897649+0000","last_unstale":"2024-09-10T23:33:30.891125+0000","last_undegraded":"2024-09-10T23:33:30.891125+0000","last_fullsized":"2024-09-10T23:33:30.891125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:34.267635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891159+0000","last_change":"2024-09-10T23:33:29.897259+0000","last_active":"2024-09-10T23:33:30.891159+0000","last_peered":"2024-09-10T23:33:30.891159+0000","last_clean":"2024-09-10T23:33:30.891159+0000","last_became_active":"2024-09-10T23:33:29.897026+0000","last_became_peered":"2024-09-10T23:33:29.897026+0000","last_unstale":"2024-09-10T23:33:30.891159+0000","last_undegraded":"2024-09-10T23:33:30.891159+0000","last_fullsized":"2024-09-10T23:33:30.891159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:05.585379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.927174+0000","last_change":"2024-09-10T23:33:28.891535+0000","last_active":"2024-09-10T23:33:29.927174+0000","last_peered":"2024-09-10T23:33:29.927174+0000","last_clean":"2024-09-10T23:33:29.927174+0000","last_became_active":"2024-09-10T23:33:28.890670+0000","last_became_peered":"2024-09-10T23:33:28.890670+0000","last_unstale":"2024-09-10T23:33:29.927174+0000","last_undegraded":"2024-09-10T23:33:29.927174+0000","last_fullsized":"2024-09-10T23:33:29.927174+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:07.754026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.898580+0000","last_change":"2024-09-10T23:33:29.888620+0000","last_active":"2024-09-10T23:33:29.898580+0000","last_peered":"2024-09-10T23:33:29.898580+0000","last_clean":"2024-09-10T23:33:29.898580+0000","last_became_active":"2024-09-10T23:33:29.887885+0000","last_became_peered":"2024-09-10T23:33:29.887885+0000","last_unstale":"2024-09-10T23:33:29.898580+0000","last_undegraded":"2024-09-10T23:33:29.898580+0000","last_fullsized":"2024-09-10T23:33:29.898580+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:30.501329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968058+0000","last_change":"2024-09-10T23:33:28.890077+0000","last_active":"2024-09-10T23:33:30.968058+0000","last_peered":"2024-09-10T23:33:30.968058+0000","last_clean":"2024-09-10T23:33:30.968058+0000","last_became_active":"2024-09-10T23:33:28.889844+0000","last_became_peered":"2024-09-10T23:33:28.889844+0000","last_unstale":"2024-09-10T23:33:30.968058+0000","last_undegraded":"2024-09-10T23:33:30.968058+0000","last_fullsized":"2024-09-10T23:33:30.968058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:20.157066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967662+0000","last_change":"2024-09-10T23:33:29.895413+0000","last_active":"2024-09-10T23:33:30.967662+0000","last_peered":"2024-09-10T23:33:30.967662+0000","last_clean":"2024-09-10T23:33:30.967662+0000","last_became_active":"2024-09-10T23:33:29.894808+0000","last_became_peered":"2024-09-10T23:33:29.894808+0000","last_unstale":"2024-09-10T23:33:30.967662+0000","last_undegraded":"2024-09-10T23:33:30.967662+0000","last_fullsized":"2024-09-10T23:33:30.967662+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:54.486020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967343+0000","last_change":"2024-09-10T23:33:28.893119+0000","last_active":"2024-09-10T23:33:30.967343+0000","last_peered":"2024-09-10T23:33:30.967343+0000","last_clean":"2024-09-10T23:33:30.967343+0000","last_became_active":"2024-09-10T23:33:28.891724+0000","last_became_peered":"2024-09-10T23:33:28.891724+0000","last_unstale":"2024-09-10T23:33:30.967343+0000","last_undegraded":"2024-09-10T23:33:30.967343+0000","last_fullsized":"2024-09-10T23:33:30.967343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:40.817852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967377+0000","last_change":"2024-09-10T23:33:29.886176+0000","last_active":"2024-09-10T23:33:30.967377+0000","last_peered":"2024-09-10T23:33:30.967377+0000","last_clean":"2024-09-10T23:33:30.967377+0000","last_became_active":"2024-09-10T23:33:29.885799+0000","last_became_peered":"2024-09-10T23:33:29.885799+0000","last_unstale":"2024-09-10T23:33:30.967377+0000","last_undegraded":"2024-09-10T23:33:30.967377+0000","last_fullsized":"2024-09-10T23:33:30.967377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:04.209891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967250+0000","last_change":"2024-09-10T23:33:28.894213+0000","last_active":"2024-09-10T23:33:30.967250+0000","last_peered":"2024-09-10T23:33:30.967250+0000","last_clean":"2024-09-10T23:33:30.967250+0000","last_became_active":"2024-09-10T23:33:28.893938+0000","last_became_peered":"2024-09-10T23:33:28.893938+0000","last_unstale":"2024-09-10T23:33:30.967250+0000","last_undegraded":"2024-09-10T23:33:30.967250+0000","last_fullsized":"2024-09-10T23:33:30.967250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:42.220925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891375+0000","last_change":"2024-09-10T23:33:29.898823+0000","last_active":"2024-09-10T23:33:30.891375+0000","last_peered":"2024-09-10T23:33:30.891375+0000","last_clean":"2024-09-10T23:33:30.891375+0000","last_became_active":"2024-09-10T23:33:29.897711+0000","last_became_peered":"2024-09-10T23:33:29.897711+0000","last_unstale":"2024-09-10T23:33:30.891375+0000","last_undegraded":"2024-09-10T23:33:30.891375+0000","last_fullsized":"2024-09-10T23:33:30.891375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:25.157936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900303+0000","last_change":"2024-09-10T23:33:28.901196+0000","last_active":"2024-09-10T23:33:30.900303+0000","last_peered":"2024-09-10T23:33:30.900303+0000","last_clean":"2024-09-10T23:33:30.900303+0000","last_became_active":"2024-09-10T23:33:28.900972+0000","last_became_peered":"2024-09-10T23:33:28.900972+0000","last_unstale":"2024-09-10T23:33:30.900303+0000","last_undegraded":"2024-09-10T23:33:30.900303+0000","last_fullsized":"2024-09-10T23:33:30.900303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:09.639971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891525+0000","last_change":"2024-09-10T23:33:29.896971+0000","last_active":"2024-09-10T23:33:30.891525+0000","last_peered":"2024-09-10T23:33:30.891525+0000","last_clean":"2024-09-10T23:33:30.891525+0000","last_became_active":"2024-09-10T23:33:29.896650+0000","last_became_peered":"2024-09-10T23:33:29.896650+0000","last_unstale":"2024-09-10T23:33:30.891525+0000","last_undegraded":"2024-09-10T23:33:30.891525+0000","last_fullsized":"2024-09-10T23:33:30.891525+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:46.869254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900724+0000","last_change":"2024-09-10T23:33:28.895981+0000","last_active":"2024-09-10T23:33:30.900724+0000","last_peered":"2024-09-10T23:33:30.900724+0000","last_clean":"2024-09-10T23:33:30.900724+0000","last_became_active":"2024-09-10T23:33:28.894900+0000","last_became_peered":"2024-09-10T23:33:28.894900+0000","last_unstale":"2024-09-10T23:33:30.900724+0000","last_undegraded":"2024-09-10T23:33:30.900724+0000","last_fullsized":"2024-09-10T23:33:30.900724+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:09.814828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967368+0000","last_change":"2024-09-10T23:33:29.894203+0000","last_active":"2024-09-10T23:33:30.967368+0000","last_peered":"2024-09-10T23:33:30.967368+0000","last_clean":"2024-09-10T23:33:30.967368+0000","last_became_active":"2024-09-10T23:33:29.893930+0000","last_became_peered":"2024-09-10T23:33:29.893930+0000","last_unstale":"2024-09-10T23:33:30.967368+0000","last_undegraded":"2024-09-10T23:33:30.967368+0000","last_fullsized":"2024-09-10T23:33:30.967368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:21.163092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967844+0000","last_change":"2024-09-10T23:33:28.900157+0000","last_active":"2024-09-10T23:33:30.967844+0000","last_peered":"2024-09-10T23:33:30.967844+0000","last_clean":"2024-09-10T23:33:30.967844+0000","last_became_active":"2024-09-10T23:33:28.899917+0000","last_became_peered":"2024-09-10T23:33:28.899917+0000","last_unstale":"2024-09-10T23:33:30.967844+0000","last_undegraded":"2024-09-10T23:33:30.967844+0000","last_fullsized":"2024-09-10T23:33:30.967844+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:24.212753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.892871+0000","last_change":"2024-09-10T23:33:29.902485+0000","last_active":"2024-09-10T23:33:30.892871+0000","last_peered":"2024-09-10T23:33:30.892871+0000","last_clean":"2024-09-10T23:33:30.892871+0000","last_became_active":"2024-09-10T23:33:29.902156+0000","last_became_peered":"2024-09-10T23:33:29.902156+0000","last_unstale":"2024-09-10T23:33:30.892871+0000","last_undegraded":"2024-09-10T23:33:30.892871+0000","last_fullsized":"2024-09-10T23:33:30.892871+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:16.077235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891285+0000","last_change":"2024-09-10T23:33:28.889228+0000","last_active":"2024-09-10T23:33:30.891285+0000","last_peered":"2024-09-10T23:33:30.891285+0000","last_clean":"2024-09-10T23:33:30.891285+0000","last_became_active":"2024-09-10T23:33:28.888975+0000","last_became_peered":"2024-09-10T23:33:28.888975+0000","last_unstale":"2024-09-10T23:33:30.891285+0000","last_undegraded":"2024-09-10T23:33:30.891285+0000","last_fullsized":"2024-09-10T23:33:30.891285+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:29.693172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900634+0000","last_change":"2024-09-10T23:33:29.908232+0000","last_active":"2024-09-10T23:33:30.900634+0000","last_peered":"2024-09-10T23:33:30.900634+0000","last_clean":"2024-09-10T23:33:30.900634+0000","last_became_active":"2024-09-10T23:33:29.905885+0000","last_became_peered":"2024-09-10T23:33:29.905885+0000","last_unstale":"2024-09-10T23:33:30.900634+0000","last_undegraded":"2024-09-10T23:33:30.900634+0000","last_fullsized":"2024-09-10T23:33:30.900634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:06:57.528984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967197+0000","last_change":"2024-09-10T23:33:28.884526+0000","last_active":"2024-09-10T23:33:30.967197+0000","last_peered":"2024-09-10T23:33:30.967197+0000","last_clean":"2024-09-10T23:33:30.967197+0000","last_became_active":"2024-09-10T23:33:28.884235+0000","last_became_peered":"2024-09-10T23:33:28.884235+0000","last_unstale":"2024-09-10T23:33:30.967197+0000","last_undegraded":"2024-09-10T23:33:30.967197+0000","last_fullsized":"2024-09-10T23:33:30.967197+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:42.220925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.899157+0000","last_change":"2024-09-10T23:33:29.894914+0000","last_active":"2024-09-10T23:33:29.899157+0000","last_peered":"2024-09-10T23:33:29.899157+0000","last_clean":"2024-09-10T23:33:29.899157+0000","last_became_active":"2024-09-10T23:33:29.894598+0000","last_became_peered":"2024-09-10T23:33:29.894598+0000","last_unstale":"2024-09-10T23:33:29.899157+0000","last_undegraded":"2024-09-10T23:33:29.899157+0000","last_fullsized":"2024-09-10T23:33:29.899157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:10.328626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.893748+0000","last_change":"2024-09-10T23:33:28.896382+0000","last_active":"2024-09-10T23:33:30.893748+0000","last_peered":"2024-09-10T23:33:30.893748+0000","last_clean":"2024-09-10T23:33:30.893748+0000","last_became_active":"2024-09-10T23:33:28.895614+0000","last_became_peered":"2024-09-10T23:33:28.895614+0000","last_unstale":"2024-09-10T23:33:30.893748+0000","last_undegraded":"2024-09-10T23:33:30.893748+0000","last_fullsized":"2024-09-10T23:33:30.893748+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:19.167006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.893568+0000","last_change":"2024-09-10T23:33:29.901415+0000","last_active":"2024-09-10T23:33:30.893568+0000","last_peered":"2024-09-10T23:33:30.893568+0000","last_clean":"2024-09-10T23:33:30.893568+0000","last_became_active":"2024-09-10T23:33:29.900880+0000","last_became_peered":"2024-09-10T23:33:29.900880+0000","last_unstale":"2024-09-10T23:33:30.893568+0000","last_undegraded":"2024-09-10T23:33:30.893568+0000","last_fullsized":"2024-09-10T23:33:30.893568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:21.241619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967844+0000","last_change":"2024-09-10T23:33:28.899830+0000","last_active":"2024-09-10T23:33:30.967844+0000","last_peered":"2024-09-10T23:33:30.967844+0000","last_clean":"2024-09-10T23:33:30.967844+0000","last_became_active":"2024-09-10T23:33:28.899370+0000","last_became_peered":"2024-09-10T23:33:28.899370+0000","last_unstale":"2024-09-10T23:33:30.967844+0000","last_undegraded":"2024-09-10T23:33:30.967844+0000","last_fullsized":"2024-09-10T23:33:30.967844+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:43.439095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.899121+0000","last_change":"2024-09-10T23:33:29.891316+0000","last_active":"2024-09-10T23:33:29.899121+0000","last_peered":"2024-09-10T23:33:29.899121+0000","last_clean":"2024-09-10T23:33:29.899121+0000","last_became_active":"2024-09-10T23:33:29.890904+0000","last_became_peered":"2024-09-10T23:33:29.890904+0000","last_unstale":"2024-09-10T23:33:29.899121+0000","last_undegraded":"2024-09-10T23:33:29.899121+0000","last_fullsized":"2024-09-10T23:33:29.899121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:43.646370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966618+0000","last_change":"2024-09-10T23:33:28.892194+0000","last_active":"2024-09-10T23:33:30.966618+0000","last_peered":"2024-09-10T23:33:30.966618+0000","last_clean":"2024-09-10T23:33:30.966618+0000","last_became_active":"2024-09-10T23:33:28.891870+0000","last_became_peered":"2024-09-10T23:33:28.891870+0000","last_unstale":"2024-09-10T23:33:30.966618+0000","last_undegraded":"2024-09-10T23:33:30.966618+0000","last_fullsized":"2024-09-10T23:33:30.966618+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:58.777973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966521+0000","last_change":"2024-09-10T23:33:29.887363+0000","last_active":"2024-09-10T23:33:30.966521+0000","last_peered":"2024-09-10T23:33:30.966521+0000","last_clean":"2024-09-10T23:33:30.966521+0000","last_became_active":"2024-09-10T23:33:29.886873+0000","last_became_peered":"2024-09-10T23:33:29.886873+0000","last_unstale":"2024-09-10T23:33:30.966521+0000","last_undegraded":"2024-09-10T23:33:30.966521+0000","last_fullsized":"2024-09-10T23:33:30.966521+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:01.796185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.890827+0000","last_change":"2024-09-10T23:33:28.891920+0000","last_active":"2024-09-10T23:33:30.890827+0000","last_peered":"2024-09-10T23:33:30.890827+0000","last_clean":"2024-09-10T23:33:30.890827+0000","last_became_active":"2024-09-10T23:33:28.891616+0000","last_became_peered":"2024-09-10T23:33:28.891616+0000","last_unstale":"2024-09-10T23:33:30.890827+0000","last_undegraded":"2024-09-10T23:33:30.890827+0000","last_fullsized":"2024-09-10T23:33:30.890827+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:58.835412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900900+0000","last_change":"2024-09-10T23:33:29.905625+0000","last_active":"2024-09-10T23:33:30.900900+0000","last_peered":"2024-09-10T23:33:30.900900+0000","last_clean":"2024-09-10T23:33:30.900900+0000","last_became_active":"2024-09-10T23:33:29.905365+0000","last_became_peered":"2024-09-10T23:33:29.905365+0000","last_unstale":"2024-09-10T23:33:30.900900+0000","last_undegraded":"2024-09-10T23:33:30.900900+0000","last_fullsized":"2024-09-10T23:33:30.900900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:46.762294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.920409+0000","last_change":"2024-09-10T23:33:28.892990+0000","last_active":"2024-09-10T23:33:29.920409+0000","last_peered":"2024-09-10T23:33:29.920409+0000","last_clean":"2024-09-10T23:33:29.920409+0000","last_became_active":"2024-09-10T23:33:28.892645+0000","last_became_peered":"2024-09-10T23:33:28.892645+0000","last_unstale":"2024-09-10T23:33:29.920409+0000","last_undegraded":"2024-09-10T23:33:29.920409+0000","last_fullsized":"2024-09-10T23:33:29.920409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:51:33.842104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891323+0000","last_change":"2024-09-10T23:33:29.903694+0000","last_active":"2024-09-10T23:33:30.891323+0000","last_peered":"2024-09-10T23:33:30.891323+0000","last_clean":"2024-09-10T23:33:30.891323+0000","last_became_active":"2024-09-10T23:33:29.903459+0000","last_became_peered":"2024-09-10T23:33:29.903459+0000","last_unstale":"2024-09-10T23:33:30.891323+0000","last_undegraded":"2024-09-10T23:33:30.891323+0000","last_fullsized":"2024-09-10T23:33:30.891323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:48.137767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.899654+0000","last_change":"2024-09-10T23:33:28.894813+0000","last_active":"2024-09-10T23:33:30.899654+0000","last_peered":"2024-09-10T23:33:30.899654+0000","last_clean":"2024-09-10T23:33:30.899654+0000","last_became_active":"2024-09-10T23:33:28.894537+0000","last_became_peered":"2024-09-10T23:33:28.894537+0000","last_unstale":"2024-09-10T23:33:30.899654+0000","last_undegraded":"2024-09-10T23:33:30.899654+0000","last_fullsized":"2024-09-10T23:33:30.899654+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:38.526599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900872+0000","last_change":"2024-09-10T23:33:29.913940+0000","last_active":"2024-09-10T23:33:30.900872+0000","last_peered":"2024-09-10T23:33:30.900872+0000","last_clean":"2024-09-10T23:33:30.900872+0000","last_became_active":"2024-09-10T23:33:29.913669+0000","last_became_peered":"2024-09-10T23:33:29.913669+0000","last_unstale":"2024-09-10T23:33:30.900872+0000","last_undegraded":"2024-09-10T23:33:30.900872+0000","last_fullsized":"2024-09-10T23:33:30.900872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:21.163092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.886780+0000","last_change":"2024-09-10T23:33:28.885695+0000","last_active":"2024-09-10T23:33:29.886780+0000","last_peered":"2024-09-10T23:33:29.886780+0000","last_clean":"2024-09-10T23:33:29.886780+0000","last_became_active":"2024-09-10T23:33:28.885392+0000","last_became_peered":"2024-09-10T23:33:28.885392+0000","last_unstale":"2024-09-10T23:33:29.886780+0000","last_undegraded":"2024-09-10T23:33:29.886780+0000","last_fullsized":"2024-09-10T23:33:29.886780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:36.163987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966607+0000","last_change":"2024-09-10T23:33:29.899251+0000","last_active":"2024-09-10T23:33:30.966607+0000","last_peered":"2024-09-10T23:33:30.966607+0000","last_clean":"2024-09-10T23:33:30.966607+0000","last_became_active":"2024-09-10T23:33:29.898983+0000","last_became_peered":"2024-09-10T23:33:29.898983+0000","last_unstale":"2024-09-10T23:33:30.966607+0000","last_undegraded":"2024-09-10T23:33:30.966607+0000","last_fullsized":"2024-09-10T23:33:30.966607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:00.564474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.886543+0000","last_change":"2024-09-10T23:33:28.892866+0000","last_active":"2024-09-10T23:33:29.886543+0000","last_peered":"2024-09-10T23:33:29.886543+0000","last_clean":"2024-09-10T23:33:29.886543+0000","last_became_active":"2024-09-10T23:33:28.892590+0000","last_became_peered":"2024-09-10T23:33:28.892590+0000","last_unstale":"2024-09-10T23:33:29.886543+0000","last_undegraded":"2024-09-10T23:33:29.886543+0000","last_fullsized":"2024-09-10T23:33:29.886543+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:47.706256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967795+0000","last_change":"2024-09-10T23:33:29.895113+0000","last_active":"2024-09-10T23:33:30.967795+0000","last_peered":"2024-09-10T23:33:30.967795+0000","last_clean":"2024-09-10T23:33:30.967795+0000","last_became_active":"2024-09-10T23:33:29.894478+0000","last_became_peered":"2024-09-10T23:33:29.894478+0000","last_unstale":"2024-09-10T23:33:30.967795+0000","last_undegraded":"2024-09-10T23:33:30.967795+0000","last_fullsized":"2024-09-10T23:33:30.967795+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:21.216448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.885731+0000","last_change":"2024-09-10T23:33:28.898816+0000","last_active":"2024-09-10T23:33:29.885731+0000","last_peered":"2024-09-10T23:33:29.885731+0000","last_clean":"2024-09-10T23:33:29.885731+0000","last_became_active":"2024-09-10T23:33:28.898122+0000","last_became_peered":"2024-09-10T23:33:28.898122+0000","last_unstale":"2024-09-10T23:33:29.885731+0000","last_undegraded":"2024-09-10T23:33:29.885731+0000","last_fullsized":"2024-09-10T23:33:29.885731+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:13.526033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967326+0000","last_change":"2024-09-10T23:33:29.898566+0000","last_active":"2024-09-10T23:33:30.967326+0000","last_peered":"2024-09-10T23:33:30.967326+0000","last_clean":"2024-09-10T23:33:30.967326+0000","last_became_active":"2024-09-10T23:33:29.898272+0000","last_became_peered":"2024-09-10T23:33:29.898272+0000","last_unstale":"2024-09-10T23:33:30.967326+0000","last_undegraded":"2024-09-10T23:33:30.967326+0000","last_fullsized":"2024-09-10T23:33:30.967326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:21.241619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967789+0000","last_change":"2024-09-10T23:33:28.899677+0000","last_active":"2024-09-10T23:33:30.967789+0000","last_peered":"2024-09-10T23:33:30.967789+0000","last_clean":"2024-09-10T23:33:30.967789+0000","last_became_active":"2024-09-10T23:33:28.899342+0000","last_became_peered":"2024-09-10T23:33:28.899342+0000","last_unstale":"2024-09-10T23:33:30.967789+0000","last_undegraded":"2024-09-10T23:33:30.967789+0000","last_fullsized":"2024-09-10T23:33:30.967789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:26.426371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891257+0000","last_change":"2024-09-10T23:33:29.898700+0000","last_active":"2024-09-10T23:33:30.891257+0000","last_peered":"2024-09-10T23:33:30.891257+0000","last_clean":"2024-09-10T23:33:30.891257+0000","last_became_active":"2024-09-10T23:33:29.897992+0000","last_became_peered":"2024-09-10T23:33:29.897992+0000","last_unstale":"2024-09-10T23:33:30.891257+0000","last_undegraded":"2024-09-10T23:33:30.891257+0000","last_fullsized":"2024-09-10T23:33:30.891257+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:55.760343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967333+0000","last_change":"2024-09-10T23:33:28.888035+0000","last_active":"2024-09-10T23:33:30.967333+0000","last_peered":"2024-09-10T23:33:30.967333+0000","last_clean":"2024-09-10T23:33:30.967333+0000","last_became_active":"2024-09-10T23:33:28.887745+0000","last_became_peered":"2024-09-10T23:33:28.887745+0000","last_unstale":"2024-09-10T23:33:30.967333+0000","last_undegraded":"2024-09-10T23:33:30.967333+0000","last_fullsized":"2024-09-10T23:33:30.967333+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:51.076870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967142+0000","last_change":"2024-09-10T23:33:29.899193+0000","last_active":"2024-09-10T23:33:30.967142+0000","last_peered":"2024-09-10T23:33:30.967142+0000","last_clean":"2024-09-10T23:33:30.967142+0000","last_became_active":"2024-09-10T23:33:29.898941+0000","last_became_peered":"2024-09-10T23:33:29.898941+0000","last_unstale":"2024-09-10T23:33:30.967142+0000","last_undegraded":"2024-09-10T23:33:30.967142+0000","last_fullsized":"2024-09-10T23:33:30.967142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:21.241619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891379+0000","last_change":"2024-09-10T23:33:28.892699+0000","last_active":"2024-09-10T23:33:30.891379+0000","last_peered":"2024-09-10T23:33:30.891379+0000","last_clean":"2024-09-10T23:33:30.891379+0000","last_became_active":"2024-09-10T23:33:28.892461+0000","last_became_peered":"2024-09-10T23:33:28.892461+0000","last_unstale":"2024-09-10T23:33:30.891379+0000","last_undegraded":"2024-09-10T23:33:30.891379+0000","last_fullsized":"2024-09-10T23:33:30.891379+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:49.235674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":132,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966368+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:33:30.966368+0000","last_peered":"2024-09-10T23:33:30.966368+0000","last_clean":"2024-09-10T23:33:30.966368+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:33:30.966368+0000","last_undegraded":"2024-09-10T23:33:30.966368+0000","last_fullsized":"2024-09-10T23:33:30.966368+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":368640,"data_stored":139589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":950272,"data_stored":724217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":141201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":380928,"data_stored":140343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":368640,"data_stored":139601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":950272,"data_stored":723705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":356352,"data_stored":133357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":364544,"data_stored":134879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:33:33.995 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:33:34.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c7000640 1 -- 172.21.15.94:0/3880212958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c0157c60 msgr2=0x7ff7c0072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:34.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c7000640 1 --2- 172.21.15.94:0/3880212958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c0157c60 0x7ff7c0072610 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000acd0 tx=0x7ff7b001de60 comp rx=0 tx=0).stop 2024-09-10T23:33:34.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c7000640 1 -- 172.21.15.94:0/3880212958 shutdown_connections 2024-09-10T23:33:34.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c7000640 1 --2- 172.21.15.94:0/3880212958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c0157c60 0x7ff7c0072610 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:34.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c7000640 1 -- 172.21.15.94:0/3880212958 >> 172.21.15.94:0/3880212958 conn(0x7ff7c006cdf0 msgr2=0x7ff7c006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:34.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c7000640 1 -- 172.21.15.94:0/3880212958 shutdown_connections 2024-09-10T23:33:34.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c7000640 1 -- 172.21.15.94:0/3880212958 wait complete. 2024-09-10T23:33:34.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c7000640 1 Processor -- start 2024-09-10T23:33:34.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c7000640 1 -- start start 2024-09-10T23:33:34.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c7000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7c006f600 0x7ff7c00c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:34.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c7000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c0070150 0x7ff7c013f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:34.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c7000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7c0157c60 0x7ff7c00cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:34.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c7000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0168380 con 0x7ff7c0070150 2024-09-10T23:33:34.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c7000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0168200 con 0x7ff7c0157c60 2024-09-10T23:33:34.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c7000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c0168500 con 0x7ff7c006f600 2024-09-10T23:33:34.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7b7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c0070150 0x7ff7c013f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:34.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7b7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c0070150 0x7ff7c013f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34050/0 (socket says 172.21.15.94:34050) 2024-09-10T23:33:34.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7b7400640 1 -- 172.21.15.94:0/3873464130 learned_addr learned my addr 172.21.15.94:0/3873464130 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:34.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7bfe00640 1 --2- 172.21.15.94:0/3873464130 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7c006f600 0x7ff7c00c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:34.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7b7400640 1 -- 172.21.15.94:0/3873464130 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7c006f600 msgr2=0x7ff7c00c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:34.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7b7400640 1 --2- 172.21.15.94:0/3873464130 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7c006f600 0x7ff7c00c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:34.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7b7400640 1 -- 172.21.15.94:0/3873464130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7c0157c60 msgr2=0x7ff7c00cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:34.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c5000640 1 --2- 172.21.15.94:0/3873464130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7c0157c60 0x7ff7c00cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:34.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7b7400640 1 --2- 172.21.15.94:0/3873464130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7c0157c60 0x7ff7c00cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:34.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7b7400640 1 -- 172.21.15.94:0/3873464130 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c0145cf0 con 0x7ff7c0070150 2024-09-10T23:33:34.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7bfe00640 1 --2- 172.21.15.94:0/3873464130 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7c006f600 0x7ff7c00c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:34.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.172+0000 7ff7c5000640 1 --2- 172.21.15.94:0/3873464130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7c0157c60 0x7ff7c00cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:34.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.176+0000 7ff7b7400640 1 --2- 172.21.15.94:0/3873464130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c0070150 0x7ff7c013f980 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800c480 tx=0x7ff7a800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:34.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.176+0000 7ff7bcc00640 1 -- 172.21.15.94:0/3873464130 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8012830 con 0x7ff7c0070150 2024-09-10T23:33:34.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.176+0000 7ff7bcc00640 1 -- 172.21.15.94:0/3873464130 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7a8017070 con 0x7ff7c0070150 2024-09-10T23:33:34.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.176+0000 7ff7bcc00640 1 -- 172.21.15.94:0/3873464130 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8012d80 con 0x7ff7c0070150 2024-09-10T23:33:34.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.176+0000 7ff7c7000640 1 -- 172.21.15.94:0/3873464130 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c01657a0 con 0x7ff7c0070150 2024-09-10T23:33:34.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.176+0000 7ff7c7000640 1 -- 172.21.15.94:0/3873464130 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c00cd180 con 0x7ff7c0070150 2024-09-10T23:33:34.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.176+0000 7ff7c7000640 1 -- 172.21.15.94:0/3873464130 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c0165cb0 con 0x7ff7c0070150 2024-09-10T23:33:34.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.176+0000 7ff7bcc00640 1 -- 172.21.15.94:0/3873464130 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff7a8007020 con 0x7ff7c0070150 2024-09-10T23:33:34.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.176+0000 7ff7bcc00640 1 --2- 172.21.15.94:0/3873464130 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff79c0725e0 0x7ff79c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:34.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.176+0000 7ff7bcc00640 1 -- 172.21.15.94:0/3873464130 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff7a8093800 con 0x7ff7c0070150 2024-09-10T23:33:34.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.176+0000 7ff7bfe00640 1 --2- 172.21.15.94:0/3873464130 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff79c0725e0 0x7ff79c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:34.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.180+0000 7ff7bfe00640 1 --2- 172.21.15.94:0/3873464130 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff79c0725e0 0x7ff79c074aa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00b0000 tx=0x7ff7b001c6a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:34.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.180+0000 7ff7bcc00640 1 -- 172.21.15.94:0/3873464130 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff7a805ef50 con 0x7ff7c0070150 2024-09-10T23:33:34.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.344+0000 7ff7c7000640 1 -- 172.21.15.94:0/3873464130 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff7c006e7e0 con 0x7ff79c0725e0 2024-09-10T23:33:34.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.348+0000 7ff7bcc00640 1 -- 172.21.15.94:0/3873464130 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251942 (secure 0 0 0) 0x7ff7c006e7e0 con 0x7ff79c0725e0 2024-09-10T23:33:34.354 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:34.356 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:33:34.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.352+0000 7ff7c7000640 1 -- 172.21.15.94:0/3873464130 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff79c0725e0 msgr2=0x7ff79c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:34.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.352+0000 7ff7c7000640 1 --2- 172.21.15.94:0/3873464130 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff79c0725e0 0x7ff79c074aa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00b0000 tx=0x7ff7b001c6a0 comp rx=0 tx=0).stop 2024-09-10T23:33:34.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.352+0000 7ff7c7000640 1 -- 172.21.15.94:0/3873464130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c0070150 msgr2=0x7ff7c013f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:34.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.352+0000 7ff7c7000640 1 --2- 172.21.15.94:0/3873464130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c0070150 0x7ff7c013f980 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800c480 tx=0x7ff7a800c940 comp rx=0 tx=0).stop 2024-09-10T23:33:34.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.352+0000 7ff7c7000640 1 -- 172.21.15.94:0/3873464130 shutdown_connections 2024-09-10T23:33:34.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.352+0000 7ff7c7000640 1 --2- 172.21.15.94:0/3873464130 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff79c0725e0 0x7ff79c074aa0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:34.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.352+0000 7ff7c7000640 1 --2- 172.21.15.94:0/3873464130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7c0157c60 0x7ff7c00cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:34.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.352+0000 7ff7c7000640 1 --2- 172.21.15.94:0/3873464130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c0070150 0x7ff7c013f980 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:34.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.352+0000 7ff7c7000640 1 --2- 172.21.15.94:0/3873464130 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7c006f600 0x7ff7c00c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:34.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.352+0000 7ff7c7000640 1 -- 172.21.15.94:0/3873464130 >> 172.21.15.94:0/3873464130 conn(0x7ff7c006cdf0 msgr2=0x7ff7c01569b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:34.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.352+0000 7ff7c7000640 1 -- 172.21.15.94:0/3873464130 shutdown_connections 2024-09-10T23:33:34.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.352+0000 7ff7c7000640 1 -- 172.21.15.94:0/3873464130 wait complete. 2024-09-10T23:33:34.374 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-09-10T23:33:33.353379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236908,"kb_used_data":4024,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737812,"statfs":{"total":773094113280,"available":772851519488,"internally_reserved":0,"allocated":4120576,"data_stored":2276892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.487613"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968115+0000","last_change":"2024-09-10T23:33:29.891760+0000","last_active":"2024-09-10T23:33:30.968115+0000","last_peered":"2024-09-10T23:33:30.968115+0000","last_clean":"2024-09-10T23:33:30.968115+0000","last_became_active":"2024-09-10T23:33:29.891355+0000","last_became_peered":"2024-09-10T23:33:29.891355+0000","last_unstale":"2024-09-10T23:33:30.968115+0000","last_undegraded":"2024-09-10T23:33:30.968115+0000","last_fullsized":"2024-09-10T23:33:30.968115+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:35.604375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.893479+0000","last_change":"2024-09-10T23:33:29.901182+0000","last_active":"2024-09-10T23:33:30.893479+0000","last_peered":"2024-09-10T23:33:30.893479+0000","last_clean":"2024-09-10T23:33:30.893479+0000","last_became_active":"2024-09-10T23:33:29.900024+0000","last_became_peered":"2024-09-10T23:33:29.900024+0000","last_unstale":"2024-09-10T23:33:30.893479+0000","last_undegraded":"2024-09-10T23:33:30.893479+0000","last_fullsized":"2024-09-10T23:33:30.893479+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:17.739516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966272+0000","last_change":"2024-09-10T23:33:29.893113+0000","last_active":"2024-09-10T23:33:30.966272+0000","last_peered":"2024-09-10T23:33:30.966272+0000","last_clean":"2024-09-10T23:33:30.966272+0000","last_became_active":"2024-09-10T23:33:29.892877+0000","last_became_peered":"2024-09-10T23:33:29.892877+0000","last_unstale":"2024-09-10T23:33:30.966272+0000","last_undegraded":"2024-09-10T23:33:30.966272+0000","last_fullsized":"2024-09-10T23:33:30.966272+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:20.427642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967649+0000","last_change":"2024-09-10T23:33:29.897864+0000","last_active":"2024-09-10T23:33:30.967649+0000","last_peered":"2024-09-10T23:33:30.967649+0000","last_clean":"2024-09-10T23:33:30.967649+0000","last_became_active":"2024-09-10T23:33:29.896613+0000","last_became_peered":"2024-09-10T23:33:29.896613+0000","last_unstale":"2024-09-10T23:33:30.967649+0000","last_undegraded":"2024-09-10T23:33:30.967649+0000","last_fullsized":"2024-09-10T23:33:30.967649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:41.550538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.893317+0000","last_change":"2024-09-10T23:33:29.901649+0000","last_active":"2024-09-10T23:33:30.893317+0000","last_peered":"2024-09-10T23:33:30.893317+0000","last_clean":"2024-09-10T23:33:30.893317+0000","last_became_active":"2024-09-10T23:33:29.901232+0000","last_became_peered":"2024-09-10T23:33:29.901232+0000","last_unstale":"2024-09-10T23:33:30.893317+0000","last_undegraded":"2024-09-10T23:33:30.893317+0000","last_fullsized":"2024-09-10T23:33:30.893317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:00.564474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900616+0000","last_change":"2024-09-10T23:33:29.911689+0000","last_active":"2024-09-10T23:33:30.900616+0000","last_peered":"2024-09-10T23:33:30.900616+0000","last_clean":"2024-09-10T23:33:30.900616+0000","last_became_active":"2024-09-10T23:33:29.911343+0000","last_became_peered":"2024-09-10T23:33:29.911343+0000","last_unstale":"2024-09-10T23:33:30.900616+0000","last_undegraded":"2024-09-10T23:33:30.900616+0000","last_fullsized":"2024-09-10T23:33:30.900616+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:27.432397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967711+0000","last_change":"2024-09-10T23:33:29.892589+0000","last_active":"2024-09-10T23:33:30.967711+0000","last_peered":"2024-09-10T23:33:30.967711+0000","last_clean":"2024-09-10T23:33:30.967711+0000","last_became_active":"2024-09-10T23:33:29.892205+0000","last_became_peered":"2024-09-10T23:33:29.892205+0000","last_unstale":"2024-09-10T23:33:30.967711+0000","last_undegraded":"2024-09-10T23:33:30.967711+0000","last_fullsized":"2024-09-10T23:33:30.967711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:39.295916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966983+0000","last_change":"2024-09-10T23:33:29.894105+0000","last_active":"2024-09-10T23:33:30.966983+0000","last_peered":"2024-09-10T23:33:30.966983+0000","last_clean":"2024-09-10T23:33:30.966983+0000","last_became_active":"2024-09-10T23:33:29.893437+0000","last_became_peered":"2024-09-10T23:33:29.893437+0000","last_unstale":"2024-09-10T23:33:30.966983+0000","last_undegraded":"2024-09-10T23:33:30.966983+0000","last_fullsized":"2024-09-10T23:33:30.966983+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:46:08.526625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967149+0000","last_change":"2024-09-10T23:33:29.893307+0000","last_active":"2024-09-10T23:33:30.967149+0000","last_peered":"2024-09-10T23:33:30.967149+0000","last_clean":"2024-09-10T23:33:30.967149+0000","last_became_active":"2024-09-10T23:33:29.892708+0000","last_became_peered":"2024-09-10T23:33:29.892708+0000","last_unstale":"2024-09-10T23:33:30.967149+0000","last_undegraded":"2024-09-10T23:33:30.967149+0000","last_fullsized":"2024-09-10T23:33:30.967149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:44.062732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968096+0000","last_change":"2024-09-10T23:33:29.905924+0000","last_active":"2024-09-10T23:33:30.968096+0000","last_peered":"2024-09-10T23:33:30.968096+0000","last_clean":"2024-09-10T23:33:30.968096+0000","last_became_active":"2024-09-10T23:33:29.905611+0000","last_became_peered":"2024-09-10T23:33:29.905611+0000","last_unstale":"2024-09-10T23:33:30.968096+0000","last_undegraded":"2024-09-10T23:33:30.968096+0000","last_fullsized":"2024-09-10T23:33:30.968096+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:13.026516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891362+0000","last_change":"2024-09-10T23:33:29.897696+0000","last_active":"2024-09-10T23:33:30.891362+0000","last_peered":"2024-09-10T23:33:30.891362+0000","last_clean":"2024-09-10T23:33:30.891362+0000","last_became_active":"2024-09-10T23:33:29.896675+0000","last_became_peered":"2024-09-10T23:33:29.896675+0000","last_unstale":"2024-09-10T23:33:30.891362+0000","last_undegraded":"2024-09-10T23:33:30.891362+0000","last_fullsized":"2024-09-10T23:33:30.891362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:54.877966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966935+0000","last_change":"2024-09-10T23:33:29.900667+0000","last_active":"2024-09-10T23:33:30.966935+0000","last_peered":"2024-09-10T23:33:30.966935+0000","last_clean":"2024-09-10T23:33:30.966935+0000","last_became_active":"2024-09-10T23:33:29.900411+0000","last_became_peered":"2024-09-10T23:33:29.900411+0000","last_unstale":"2024-09-10T23:33:30.966935+0000","last_undegraded":"2024-09-10T23:33:30.966935+0000","last_fullsized":"2024-09-10T23:33:30.966935+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:27.432397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.907645+0000","last_change":"2024-09-10T23:33:29.902156+0000","last_active":"2024-09-10T23:33:29.907645+0000","last_peered":"2024-09-10T23:33:29.907645+0000","last_clean":"2024-09-10T23:33:29.907645+0000","last_became_active":"2024-09-10T23:33:29.901826+0000","last_became_peered":"2024-09-10T23:33:29.901826+0000","last_unstale":"2024-09-10T23:33:29.907645+0000","last_undegraded":"2024-09-10T23:33:29.907645+0000","last_fullsized":"2024-09-10T23:33:29.907645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:53.970887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967553+0000","last_change":"2024-09-10T23:33:29.908002+0000","last_active":"2024-09-10T23:33:30.967553+0000","last_peered":"2024-09-10T23:33:30.967553+0000","last_clean":"2024-09-10T23:33:30.967553+0000","last_became_active":"2024-09-10T23:33:29.905494+0000","last_became_peered":"2024-09-10T23:33:29.905494+0000","last_unstale":"2024-09-10T23:33:30.967553+0000","last_undegraded":"2024-09-10T23:33:30.967553+0000","last_fullsized":"2024-09-10T23:33:30.967553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:17:25.615539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966361+0000","last_change":"2024-09-10T23:33:29.898423+0000","last_active":"2024-09-10T23:33:30.966361+0000","last_peered":"2024-09-10T23:33:30.966361+0000","last_clean":"2024-09-10T23:33:30.966361+0000","last_became_active":"2024-09-10T23:33:29.897425+0000","last_became_peered":"2024-09-10T23:33:29.897425+0000","last_unstale":"2024-09-10T23:33:30.966361+0000","last_undegraded":"2024-09-10T23:33:30.966361+0000","last_fullsized":"2024-09-10T23:33:30.966361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:23.725141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966183+0000","last_change":"2024-09-10T23:33:29.900989+0000","last_active":"2024-09-10T23:33:30.966183+0000","last_peered":"2024-09-10T23:33:30.966183+0000","last_clean":"2024-09-10T23:33:30.966183+0000","last_became_active":"2024-09-10T23:33:29.900743+0000","last_became_peered":"2024-09-10T23:33:29.900743+0000","last_unstale":"2024-09-10T23:33:30.966183+0000","last_undegraded":"2024-09-10T23:33:30.966183+0000","last_fullsized":"2024-09-10T23:33:30.966183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:53.556368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891314+0000","last_change":"2024-09-10T23:33:29.895656+0000","last_active":"2024-09-10T23:33:30.891314+0000","last_peered":"2024-09-10T23:33:30.891314+0000","last_clean":"2024-09-10T23:33:30.891314+0000","last_became_active":"2024-09-10T23:33:29.889689+0000","last_became_peered":"2024-09-10T23:33:29.889689+0000","last_unstale":"2024-09-10T23:33:30.891314+0000","last_undegraded":"2024-09-10T23:33:30.891314+0000","last_fullsized":"2024-09-10T23:33:30.891314+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:15.868218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900863+0000","last_change":"2024-09-10T23:33:29.903174+0000","last_active":"2024-09-10T23:33:30.900863+0000","last_peered":"2024-09-10T23:33:30.900863+0000","last_clean":"2024-09-10T23:33:30.900863+0000","last_became_active":"2024-09-10T23:33:29.902925+0000","last_became_peered":"2024-09-10T23:33:29.902925+0000","last_unstale":"2024-09-10T23:33:30.900863+0000","last_undegraded":"2024-09-10T23:33:30.900863+0000","last_fullsized":"2024-09-10T23:33:30.900863+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:04.980472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900574+0000","last_change":"2024-09-10T23:33:29.900766+0000","last_active":"2024-09-10T23:33:30.900574+0000","last_peered":"2024-09-10T23:33:30.900574+0000","last_clean":"2024-09-10T23:33:30.900574+0000","last_became_active":"2024-09-10T23:33:29.900491+0000","last_became_peered":"2024-09-10T23:33:29.900491+0000","last_unstale":"2024-09-10T23:33:30.900574+0000","last_undegraded":"2024-09-10T23:33:30.900574+0000","last_fullsized":"2024-09-10T23:33:30.900574+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:44.445121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.899206+0000","last_change":"2024-09-10T23:33:29.896368+0000","last_active":"2024-09-10T23:33:29.899206+0000","last_peered":"2024-09-10T23:33:29.899206+0000","last_clean":"2024-09-10T23:33:29.899206+0000","last_became_active":"2024-09-10T23:33:29.896173+0000","last_became_peered":"2024-09-10T23:33:29.896173+0000","last_unstale":"2024-09-10T23:33:29.899206+0000","last_undegraded":"2024-09-10T23:33:29.899206+0000","last_fullsized":"2024-09-10T23:33:29.899206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:53.653730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966830+0000","last_change":"2024-09-10T23:33:29.887720+0000","last_active":"2024-09-10T23:33:30.966830+0000","last_peered":"2024-09-10T23:33:30.966830+0000","last_clean":"2024-09-10T23:33:30.966830+0000","last_became_active":"2024-09-10T23:33:29.887367+0000","last_became_peered":"2024-09-10T23:33:29.887367+0000","last_unstale":"2024-09-10T23:33:30.966830+0000","last_undegraded":"2024-09-10T23:33:30.966830+0000","last_fullsized":"2024-09-10T23:33:30.966830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:17.739516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967739+0000","last_change":"2024-09-10T23:33:29.887429+0000","last_active":"2024-09-10T23:33:30.967739+0000","last_peered":"2024-09-10T23:33:30.967739+0000","last_clean":"2024-09-10T23:33:30.967739+0000","last_became_active":"2024-09-10T23:33:29.887115+0000","last_became_peered":"2024-09-10T23:33:29.887115+0000","last_unstale":"2024-09-10T23:33:30.967739+0000","last_undegraded":"2024-09-10T23:33:30.967739+0000","last_fullsized":"2024-09-10T23:33:30.967739+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:35.273661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.905713+0000","last_change":"2024-09-10T23:33:29.899713+0000","last_active":"2024-09-10T23:33:29.905713+0000","last_peered":"2024-09-10T23:33:29.905713+0000","last_clean":"2024-09-10T23:33:29.905713+0000","last_became_active":"2024-09-10T23:33:29.899305+0000","last_became_peered":"2024-09-10T23:33:29.899305+0000","last_unstale":"2024-09-10T23:33:29.905713+0000","last_undegraded":"2024-09-10T23:33:29.905713+0000","last_fullsized":"2024-09-10T23:33:29.905713+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:47.866442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967173+0000","last_change":"2024-09-10T23:33:29.893860+0000","last_active":"2024-09-10T23:33:30.967173+0000","last_peered":"2024-09-10T23:33:30.967173+0000","last_clean":"2024-09-10T23:33:30.967173+0000","last_became_active":"2024-09-10T23:33:29.893091+0000","last_became_peered":"2024-09-10T23:33:29.893091+0000","last_unstale":"2024-09-10T23:33:30.967173+0000","last_undegraded":"2024-09-10T23:33:30.967173+0000","last_fullsized":"2024-09-10T23:33:30.967173+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:23.778319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968095+0000","last_change":"2024-09-10T23:33:29.908177+0000","last_active":"2024-09-10T23:33:30.968095+0000","last_peered":"2024-09-10T23:33:30.968095+0000","last_clean":"2024-09-10T23:33:30.968095+0000","last_became_active":"2024-09-10T23:33:29.899969+0000","last_became_peered":"2024-09-10T23:33:29.899969+0000","last_unstale":"2024-09-10T23:33:30.968095+0000","last_undegraded":"2024-09-10T23:33:30.968095+0000","last_fullsized":"2024-09-10T23:33:30.968095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:19.657339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968157+0000","last_change":"2024-09-10T23:33:29.889211+0000","last_active":"2024-09-10T23:33:30.968157+0000","last_peered":"2024-09-10T23:33:30.968157+0000","last_clean":"2024-09-10T23:33:30.968157+0000","last_became_active":"2024-09-10T23:33:29.888312+0000","last_became_peered":"2024-09-10T23:33:29.888312+0000","last_unstale":"2024-09-10T23:33:30.968157+0000","last_undegraded":"2024-09-10T23:33:30.968157+0000","last_fullsized":"2024-09-10T23:33:30.968157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:07.940639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.893528+0000","last_change":"2024-09-10T23:33:29.904832+0000","last_active":"2024-09-10T23:33:30.893528+0000","last_peered":"2024-09-10T23:33:30.893528+0000","last_clean":"2024-09-10T23:33:30.893528+0000","last_became_active":"2024-09-10T23:33:29.902828+0000","last_became_peered":"2024-09-10T23:33:29.902828+0000","last_unstale":"2024-09-10T23:33:30.893528+0000","last_undegraded":"2024-09-10T23:33:30.893528+0000","last_fullsized":"2024-09-10T23:33:30.893528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:22.440859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968109+0000","last_change":"2024-09-10T23:33:29.903254+0000","last_active":"2024-09-10T23:33:30.968109+0000","last_peered":"2024-09-10T23:33:30.968109+0000","last_clean":"2024-09-10T23:33:30.968109+0000","last_became_active":"2024-09-10T23:33:29.902942+0000","last_became_peered":"2024-09-10T23:33:29.902942+0000","last_unstale":"2024-09-10T23:33:30.968109+0000","last_undegraded":"2024-09-10T23:33:30.968109+0000","last_fullsized":"2024-09-10T23:33:30.968109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:21.241619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966885+0000","last_change":"2024-09-10T23:33:29.891471+0000","last_active":"2024-09-10T23:33:30.966885+0000","last_peered":"2024-09-10T23:33:30.966885+0000","last_clean":"2024-09-10T23:33:30.966885+0000","last_became_active":"2024-09-10T23:33:29.889128+0000","last_became_peered":"2024-09-10T23:33:29.889128+0000","last_unstale":"2024-09-10T23:33:30.966885+0000","last_undegraded":"2024-09-10T23:33:30.966885+0000","last_fullsized":"2024-09-10T23:33:30.966885+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:22.440859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966570+0000","last_change":"2024-09-10T23:33:29.888402+0000","last_active":"2024-09-10T23:33:30.966570+0000","last_peered":"2024-09-10T23:33:30.966570+0000","last_clean":"2024-09-10T23:33:30.966570+0000","last_became_active":"2024-09-10T23:33:29.888121+0000","last_became_peered":"2024-09-10T23:33:29.888121+0000","last_unstale":"2024-09-10T23:33:30.966570+0000","last_undegraded":"2024-09-10T23:33:30.966570+0000","last_fullsized":"2024-09-10T23:33:30.966570+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:16.077235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967692+0000","last_change":"2024-09-10T23:33:29.903252+0000","last_active":"2024-09-10T23:33:30.967692+0000","last_peered":"2024-09-10T23:33:30.967692+0000","last_clean":"2024-09-10T23:33:30.967692+0000","last_became_active":"2024-09-10T23:33:29.902822+0000","last_became_peered":"2024-09-10T23:33:29.902822+0000","last_unstale":"2024-09-10T23:33:30.967692+0000","last_undegraded":"2024-09-10T23:33:30.967692+0000","last_fullsized":"2024-09-10T23:33:30.967692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:59.841438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.890931+0000","last_change":"2024-09-10T23:33:29.890470+0000","last_active":"2024-09-10T23:33:30.890931+0000","last_peered":"2024-09-10T23:33:30.890931+0000","last_clean":"2024-09-10T23:33:30.890931+0000","last_became_active":"2024-09-10T23:33:29.890158+0000","last_became_peered":"2024-09-10T23:33:29.890158+0000","last_unstale":"2024-09-10T23:33:30.890931+0000","last_undegraded":"2024-09-10T23:33:30.890931+0000","last_fullsized":"2024-09-10T23:33:30.890931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:17.650619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.901171+0000","last_change":"2024-09-10T23:33:29.891670+0000","last_active":"2024-09-10T23:33:29.901171+0000","last_peered":"2024-09-10T23:33:29.901171+0000","last_clean":"2024-09-10T23:33:29.901171+0000","last_became_active":"2024-09-10T23:33:29.891300+0000","last_became_peered":"2024-09-10T23:33:29.891300+0000","last_unstale":"2024-09-10T23:33:29.901171+0000","last_undegraded":"2024-09-10T23:33:29.901171+0000","last_fullsized":"2024-09-10T23:33:29.901171+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:10.129515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891421+0000","last_change":"2024-09-10T23:33:28.899750+0000","last_active":"2024-09-10T23:33:30.891421+0000","last_peered":"2024-09-10T23:33:30.891421+0000","last_clean":"2024-09-10T23:33:30.891421+0000","last_became_active":"2024-09-10T23:33:28.899272+0000","last_became_peered":"2024-09-10T23:33:28.899272+0000","last_unstale":"2024-09-10T23:33:30.891421+0000","last_undegraded":"2024-09-10T23:33:30.891421+0000","last_fullsized":"2024-09-10T23:33:30.891421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:38.289889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968307+0000","last_change":"2024-09-10T23:33:29.905051+0000","last_active":"2024-09-10T23:33:30.968307+0000","last_peered":"2024-09-10T23:33:30.968307+0000","last_clean":"2024-09-10T23:33:30.968307+0000","last_became_active":"2024-09-10T23:33:29.904536+0000","last_became_peered":"2024-09-10T23:33:29.904536+0000","last_unstale":"2024-09-10T23:33:30.968307+0000","last_undegraded":"2024-09-10T23:33:30.968307+0000","last_fullsized":"2024-09-10T23:33:30.968307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:42.507561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968179+0000","last_change":"2024-09-10T23:33:28.897865+0000","last_active":"2024-09-10T23:33:30.968179+0000","last_peered":"2024-09-10T23:33:30.968179+0000","last_clean":"2024-09-10T23:33:30.968179+0000","last_became_active":"2024-09-10T23:33:28.897179+0000","last_became_peered":"2024-09-10T23:33:28.897179+0000","last_unstale":"2024-09-10T23:33:30.968179+0000","last_undegraded":"2024-09-10T23:33:30.968179+0000","last_fullsized":"2024-09-10T23:33:30.968179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:31:55.659032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968104+0000","last_change":"2024-09-10T23:33:29.889304+0000","last_active":"2024-09-10T23:33:30.968104+0000","last_peered":"2024-09-10T23:33:30.968104+0000","last_clean":"2024-09-10T23:33:30.968104+0000","last_became_active":"2024-09-10T23:33:29.889061+0000","last_became_peered":"2024-09-10T23:33:29.889061+0000","last_unstale":"2024-09-10T23:33:30.968104+0000","last_undegraded":"2024-09-10T23:33:30.968104+0000","last_fullsized":"2024-09-10T23:33:30.968104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:30.699198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900762+0000","last_change":"2024-09-10T23:33:28.895241+0000","last_active":"2024-09-10T23:33:30.900762+0000","last_peered":"2024-09-10T23:33:30.900762+0000","last_clean":"2024-09-10T23:33:30.900762+0000","last_became_active":"2024-09-10T23:33:28.894954+0000","last_became_peered":"2024-09-10T23:33:28.894954+0000","last_unstale":"2024-09-10T23:33:30.900762+0000","last_undegraded":"2024-09-10T23:33:30.900762+0000","last_fullsized":"2024-09-10T23:33:30.900762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:39.116940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968025+0000","last_change":"2024-09-10T23:33:29.903886+0000","last_active":"2024-09-10T23:33:30.968025+0000","last_peered":"2024-09-10T23:33:30.968025+0000","last_clean":"2024-09-10T23:33:30.968025+0000","last_became_active":"2024-09-10T23:33:29.903611+0000","last_became_peered":"2024-09-10T23:33:29.903611+0000","last_unstale":"2024-09-10T23:33:30.968025+0000","last_undegraded":"2024-09-10T23:33:30.968025+0000","last_fullsized":"2024-09-10T23:33:30.968025+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:10.777381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967884+0000","last_change":"2024-09-10T23:33:28.887047+0000","last_active":"2024-09-10T23:33:30.967884+0000","last_peered":"2024-09-10T23:33:30.967884+0000","last_clean":"2024-09-10T23:33:30.967884+0000","last_became_active":"2024-09-10T23:33:28.886032+0000","last_became_peered":"2024-09-10T23:33:28.886032+0000","last_unstale":"2024-09-10T23:33:30.967884+0000","last_undegraded":"2024-09-10T23:33:30.967884+0000","last_fullsized":"2024-09-10T23:33:30.967884+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:02.567377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966855+0000","last_change":"2024-09-10T23:33:29.892030+0000","last_active":"2024-09-10T23:33:30.966855+0000","last_peered":"2024-09-10T23:33:30.966855+0000","last_clean":"2024-09-10T23:33:30.966855+0000","last_became_active":"2024-09-10T23:33:29.891793+0000","last_became_peered":"2024-09-10T23:33:29.891793+0000","last_unstale":"2024-09-10T23:33:30.966855+0000","last_undegraded":"2024-09-10T23:33:30.966855+0000","last_fullsized":"2024-09-10T23:33:30.966855+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:44.445121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967953+0000","last_change":"2024-09-10T23:33:28.890106+0000","last_active":"2024-09-10T23:33:30.967953+0000","last_peered":"2024-09-10T23:33:30.967953+0000","last_clean":"2024-09-10T23:33:30.967953+0000","last_became_active":"2024-09-10T23:33:28.889916+0000","last_became_peered":"2024-09-10T23:33:28.889916+0000","last_unstale":"2024-09-10T23:33:30.967953+0000","last_undegraded":"2024-09-10T23:33:30.967953+0000","last_fullsized":"2024-09-10T23:33:30.967953+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:43.056706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968150+0000","last_change":"2024-09-10T23:33:29.911612+0000","last_active":"2024-09-10T23:33:30.968150+0000","last_peered":"2024-09-10T23:33:30.968150+0000","last_clean":"2024-09-10T23:33:30.968150+0000","last_became_active":"2024-09-10T23:33:29.911377+0000","last_became_peered":"2024-09-10T23:33:29.911377+0000","last_unstale":"2024-09-10T23:33:30.968150+0000","last_undegraded":"2024-09-10T23:33:30.968150+0000","last_fullsized":"2024-09-10T23:33:30.968150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:54.486020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967742+0000","last_change":"2024-09-10T23:33:28.886964+0000","last_active":"2024-09-10T23:33:30.967742+0000","last_peered":"2024-09-10T23:33:30.967742+0000","last_clean":"2024-09-10T23:33:30.967742+0000","last_became_active":"2024-09-10T23:33:28.886708+0000","last_became_peered":"2024-09-10T23:33:28.886708+0000","last_unstale":"2024-09-10T23:33:30.967742+0000","last_undegraded":"2024-09-10T23:33:30.967742+0000","last_fullsized":"2024-09-10T23:33:30.967742+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:19.421616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966256+0000","last_change":"2024-09-10T23:33:29.887895+0000","last_active":"2024-09-10T23:33:30.966256+0000","last_peered":"2024-09-10T23:33:30.966256+0000","last_clean":"2024-09-10T23:33:30.966256+0000","last_became_active":"2024-09-10T23:33:29.887679+0000","last_became_peered":"2024-09-10T23:33:29.887679+0000","last_unstale":"2024-09-10T23:33:30.966256+0000","last_undegraded":"2024-09-10T23:33:30.966256+0000","last_fullsized":"2024-09-10T23:33:30.966256+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:34.439962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900447+0000","last_change":"2024-09-10T23:33:28.891708+0000","last_active":"2024-09-10T23:33:30.900447+0000","last_peered":"2024-09-10T23:33:30.900447+0000","last_clean":"2024-09-10T23:33:30.900447+0000","last_became_active":"2024-09-10T23:33:28.891247+0000","last_became_peered":"2024-09-10T23:33:28.891247+0000","last_unstale":"2024-09-10T23:33:30.900447+0000","last_undegraded":"2024-09-10T23:33:30.900447+0000","last_fullsized":"2024-09-10T23:33:30.900447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:47.784412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967310+0000","last_change":"2024-09-10T23:33:29.895718+0000","last_active":"2024-09-10T23:33:30.967310+0000","last_peered":"2024-09-10T23:33:30.967310+0000","last_clean":"2024-09-10T23:33:30.967310+0000","last_became_active":"2024-09-10T23:33:29.895471+0000","last_became_peered":"2024-09-10T23:33:29.895471+0000","last_unstale":"2024-09-10T23:33:30.967310+0000","last_undegraded":"2024-09-10T23:33:30.967310+0000","last_fullsized":"2024-09-10T23:33:30.967310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:22.440859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967449+0000","last_change":"2024-09-10T23:33:28.895041+0000","last_active":"2024-09-10T23:33:30.967449+0000","last_peered":"2024-09-10T23:33:30.967449+0000","last_clean":"2024-09-10T23:33:30.967449+0000","last_became_active":"2024-09-10T23:33:28.893542+0000","last_became_peered":"2024-09-10T23:33:28.893542+0000","last_unstale":"2024-09-10T23:33:30.967449+0000","last_undegraded":"2024-09-10T23:33:30.967449+0000","last_fullsized":"2024-09-10T23:33:30.967449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:00.790159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891369+0000","last_change":"2024-09-10T23:33:29.890004+0000","last_active":"2024-09-10T23:33:30.891369+0000","last_peered":"2024-09-10T23:33:30.891369+0000","last_clean":"2024-09-10T23:33:30.891369+0000","last_became_active":"2024-09-10T23:33:29.889726+0000","last_became_peered":"2024-09-10T23:33:29.889726+0000","last_unstale":"2024-09-10T23:33:30.891369+0000","last_undegraded":"2024-09-10T23:33:30.891369+0000","last_fullsized":"2024-09-10T23:33:30.891369+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:33.225909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.893858+0000","last_change":"2024-09-10T23:33:28.889969+0000","last_active":"2024-09-10T23:33:30.893858+0000","last_peered":"2024-09-10T23:33:30.893858+0000","last_clean":"2024-09-10T23:33:30.893858+0000","last_became_active":"2024-09-10T23:33:28.889612+0000","last_became_peered":"2024-09-10T23:33:28.889612+0000","last_unstale":"2024-09-10T23:33:30.893858+0000","last_undegraded":"2024-09-10T23:33:30.893858+0000","last_fullsized":"2024-09-10T23:33:30.893858+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:22.772293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968063+0000","last_change":"2024-09-10T23:33:29.912019+0000","last_active":"2024-09-10T23:33:30.968063+0000","last_peered":"2024-09-10T23:33:30.968063+0000","last_clean":"2024-09-10T23:33:30.968063+0000","last_became_active":"2024-09-10T23:33:29.908348+0000","last_became_peered":"2024-09-10T23:33:29.908348+0000","last_unstale":"2024-09-10T23:33:30.968063+0000","last_undegraded":"2024-09-10T23:33:30.968063+0000","last_fullsized":"2024-09-10T23:33:30.968063+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:46.777181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.893567+0000","last_change":"2024-09-10T23:33:28.896131+0000","last_active":"2024-09-10T23:33:30.893567+0000","last_peered":"2024-09-10T23:33:30.893567+0000","last_clean":"2024-09-10T23:33:30.893567+0000","last_became_active":"2024-09-10T23:33:28.895323+0000","last_became_peered":"2024-09-10T23:33:28.895323+0000","last_unstale":"2024-09-10T23:33:30.893567+0000","last_undegraded":"2024-09-10T23:33:30.893567+0000","last_fullsized":"2024-09-10T23:33:30.893567+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:03.806158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967703+0000","last_change":"2024-09-10T23:33:29.898994+0000","last_active":"2024-09-10T23:33:30.967703+0000","last_peered":"2024-09-10T23:33:30.967703+0000","last_clean":"2024-09-10T23:33:30.967703+0000","last_became_active":"2024-09-10T23:33:29.896256+0000","last_became_peered":"2024-09-10T23:33:29.896256+0000","last_unstale":"2024-09-10T23:33:30.967703+0000","last_undegraded":"2024-09-10T23:33:30.967703+0000","last_fullsized":"2024-09-10T23:33:30.967703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:42.509513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891125+0000","last_change":"2024-09-10T23:33:28.898024+0000","last_active":"2024-09-10T23:33:30.891125+0000","last_peered":"2024-09-10T23:33:30.891125+0000","last_clean":"2024-09-10T23:33:30.891125+0000","last_became_active":"2024-09-10T23:33:28.897649+0000","last_became_peered":"2024-09-10T23:33:28.897649+0000","last_unstale":"2024-09-10T23:33:30.891125+0000","last_undegraded":"2024-09-10T23:33:30.891125+0000","last_fullsized":"2024-09-10T23:33:30.891125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:34.267635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891159+0000","last_change":"2024-09-10T23:33:29.897259+0000","last_active":"2024-09-10T23:33:30.891159+0000","last_peered":"2024-09-10T23:33:30.891159+0000","last_clean":"2024-09-10T23:33:30.891159+0000","last_became_active":"2024-09-10T23:33:29.897026+0000","last_became_peered":"2024-09-10T23:33:29.897026+0000","last_unstale":"2024-09-10T23:33:30.891159+0000","last_undegraded":"2024-09-10T23:33:30.891159+0000","last_fullsized":"2024-09-10T23:33:30.891159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:05.585379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.927174+0000","last_change":"2024-09-10T23:33:28.891535+0000","last_active":"2024-09-10T23:33:29.927174+0000","last_peered":"2024-09-10T23:33:29.927174+0000","last_clean":"2024-09-10T23:33:29.927174+0000","last_became_active":"2024-09-10T23:33:28.890670+0000","last_became_peered":"2024-09-10T23:33:28.890670+0000","last_unstale":"2024-09-10T23:33:29.927174+0000","last_undegraded":"2024-09-10T23:33:29.927174+0000","last_fullsized":"2024-09-10T23:33:29.927174+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:07.754026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.898580+0000","last_change":"2024-09-10T23:33:29.888620+0000","last_active":"2024-09-10T23:33:29.898580+0000","last_peered":"2024-09-10T23:33:29.898580+0000","last_clean":"2024-09-10T23:33:29.898580+0000","last_became_active":"2024-09-10T23:33:29.887885+0000","last_became_peered":"2024-09-10T23:33:29.887885+0000","last_unstale":"2024-09-10T23:33:29.898580+0000","last_undegraded":"2024-09-10T23:33:29.898580+0000","last_fullsized":"2024-09-10T23:33:29.898580+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:30.501329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.968058+0000","last_change":"2024-09-10T23:33:28.890077+0000","last_active":"2024-09-10T23:33:30.968058+0000","last_peered":"2024-09-10T23:33:30.968058+0000","last_clean":"2024-09-10T23:33:30.968058+0000","last_became_active":"2024-09-10T23:33:28.889844+0000","last_became_peered":"2024-09-10T23:33:28.889844+0000","last_unstale":"2024-09-10T23:33:30.968058+0000","last_undegraded":"2024-09-10T23:33:30.968058+0000","last_fullsized":"2024-09-10T23:33:30.968058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:20.157066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967662+0000","last_change":"2024-09-10T23:33:29.895413+0000","last_active":"2024-09-10T23:33:30.967662+0000","last_peered":"2024-09-10T23:33:30.967662+0000","last_clean":"2024-09-10T23:33:30.967662+0000","last_became_active":"2024-09-10T23:33:29.894808+0000","last_became_peered":"2024-09-10T23:33:29.894808+0000","last_unstale":"2024-09-10T23:33:30.967662+0000","last_undegraded":"2024-09-10T23:33:30.967662+0000","last_fullsized":"2024-09-10T23:33:30.967662+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:54.486020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967343+0000","last_change":"2024-09-10T23:33:28.893119+0000","last_active":"2024-09-10T23:33:30.967343+0000","last_peered":"2024-09-10T23:33:30.967343+0000","last_clean":"2024-09-10T23:33:30.967343+0000","last_became_active":"2024-09-10T23:33:28.891724+0000","last_became_peered":"2024-09-10T23:33:28.891724+0000","last_unstale":"2024-09-10T23:33:30.967343+0000","last_undegraded":"2024-09-10T23:33:30.967343+0000","last_fullsized":"2024-09-10T23:33:30.967343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:40.817852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967377+0000","last_change":"2024-09-10T23:33:29.886176+0000","last_active":"2024-09-10T23:33:30.967377+0000","last_peered":"2024-09-10T23:33:30.967377+0000","last_clean":"2024-09-10T23:33:30.967377+0000","last_became_active":"2024-09-10T23:33:29.885799+0000","last_became_peered":"2024-09-10T23:33:29.885799+0000","last_unstale":"2024-09-10T23:33:30.967377+0000","last_undegraded":"2024-09-10T23:33:30.967377+0000","last_fullsized":"2024-09-10T23:33:30.967377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:04.209891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967250+0000","last_change":"2024-09-10T23:33:28.894213+0000","last_active":"2024-09-10T23:33:30.967250+0000","last_peered":"2024-09-10T23:33:30.967250+0000","last_clean":"2024-09-10T23:33:30.967250+0000","last_became_active":"2024-09-10T23:33:28.893938+0000","last_became_peered":"2024-09-10T23:33:28.893938+0000","last_unstale":"2024-09-10T23:33:30.967250+0000","last_undegraded":"2024-09-10T23:33:30.967250+0000","last_fullsized":"2024-09-10T23:33:30.967250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:42.220925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891375+0000","last_change":"2024-09-10T23:33:29.898823+0000","last_active":"2024-09-10T23:33:30.891375+0000","last_peered":"2024-09-10T23:33:30.891375+0000","last_clean":"2024-09-10T23:33:30.891375+0000","last_became_active":"2024-09-10T23:33:29.897711+0000","last_became_peered":"2024-09-10T23:33:29.897711+0000","last_unstale":"2024-09-10T23:33:30.891375+0000","last_undegraded":"2024-09-10T23:33:30.891375+0000","last_fullsized":"2024-09-10T23:33:30.891375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:25.157936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900303+0000","last_change":"2024-09-10T23:33:28.901196+0000","last_active":"2024-09-10T23:33:30.900303+0000","last_peered":"2024-09-10T23:33:30.900303+0000","last_clean":"2024-09-10T23:33:30.900303+0000","last_became_active":"2024-09-10T23:33:28.900972+0000","last_became_peered":"2024-09-10T23:33:28.900972+0000","last_unstale":"2024-09-10T23:33:30.900303+0000","last_undegraded":"2024-09-10T23:33:30.900303+0000","last_fullsized":"2024-09-10T23:33:30.900303+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:09.639971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891525+0000","last_change":"2024-09-10T23:33:29.896971+0000","last_active":"2024-09-10T23:33:30.891525+0000","last_peered":"2024-09-10T23:33:30.891525+0000","last_clean":"2024-09-10T23:33:30.891525+0000","last_became_active":"2024-09-10T23:33:29.896650+0000","last_became_peered":"2024-09-10T23:33:29.896650+0000","last_unstale":"2024-09-10T23:33:30.891525+0000","last_undegraded":"2024-09-10T23:33:30.891525+0000","last_fullsized":"2024-09-10T23:33:30.891525+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:46.869254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900724+0000","last_change":"2024-09-10T23:33:28.895981+0000","last_active":"2024-09-10T23:33:30.900724+0000","last_peered":"2024-09-10T23:33:30.900724+0000","last_clean":"2024-09-10T23:33:30.900724+0000","last_became_active":"2024-09-10T23:33:28.894900+0000","last_became_peered":"2024-09-10T23:33:28.894900+0000","last_unstale":"2024-09-10T23:33:30.900724+0000","last_undegraded":"2024-09-10T23:33:30.900724+0000","last_fullsized":"2024-09-10T23:33:30.900724+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:09.814828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967368+0000","last_change":"2024-09-10T23:33:29.894203+0000","last_active":"2024-09-10T23:33:30.967368+0000","last_peered":"2024-09-10T23:33:30.967368+0000","last_clean":"2024-09-10T23:33:30.967368+0000","last_became_active":"2024-09-10T23:33:29.893930+0000","last_became_peered":"2024-09-10T23:33:29.893930+0000","last_unstale":"2024-09-10T23:33:30.967368+0000","last_undegraded":"2024-09-10T23:33:30.967368+0000","last_fullsized":"2024-09-10T23:33:30.967368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:21.163092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967844+0000","last_change":"2024-09-10T23:33:28.900157+0000","last_active":"2024-09-10T23:33:30.967844+0000","last_peered":"2024-09-10T23:33:30.967844+0000","last_clean":"2024-09-10T23:33:30.967844+0000","last_became_active":"2024-09-10T23:33:28.899917+0000","last_became_peered":"2024-09-10T23:33:28.899917+0000","last_unstale":"2024-09-10T23:33:30.967844+0000","last_undegraded":"2024-09-10T23:33:30.967844+0000","last_fullsized":"2024-09-10T23:33:30.967844+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:24.212753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.892871+0000","last_change":"2024-09-10T23:33:29.902485+0000","last_active":"2024-09-10T23:33:30.892871+0000","last_peered":"2024-09-10T23:33:30.892871+0000","last_clean":"2024-09-10T23:33:30.892871+0000","last_became_active":"2024-09-10T23:33:29.902156+0000","last_became_peered":"2024-09-10T23:33:29.902156+0000","last_unstale":"2024-09-10T23:33:30.892871+0000","last_undegraded":"2024-09-10T23:33:30.892871+0000","last_fullsized":"2024-09-10T23:33:30.892871+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:16.077235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891285+0000","last_change":"2024-09-10T23:33:28.889228+0000","last_active":"2024-09-10T23:33:30.891285+0000","last_peered":"2024-09-10T23:33:30.891285+0000","last_clean":"2024-09-10T23:33:30.891285+0000","last_became_active":"2024-09-10T23:33:28.888975+0000","last_became_peered":"2024-09-10T23:33:28.888975+0000","last_unstale":"2024-09-10T23:33:30.891285+0000","last_undegraded":"2024-09-10T23:33:30.891285+0000","last_fullsized":"2024-09-10T23:33:30.891285+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:29.693172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900634+0000","last_change":"2024-09-10T23:33:29.908232+0000","last_active":"2024-09-10T23:33:30.900634+0000","last_peered":"2024-09-10T23:33:30.900634+0000","last_clean":"2024-09-10T23:33:30.900634+0000","last_became_active":"2024-09-10T23:33:29.905885+0000","last_became_peered":"2024-09-10T23:33:29.905885+0000","last_unstale":"2024-09-10T23:33:30.900634+0000","last_undegraded":"2024-09-10T23:33:30.900634+0000","last_fullsized":"2024-09-10T23:33:30.900634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:06:57.528984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967197+0000","last_change":"2024-09-10T23:33:28.884526+0000","last_active":"2024-09-10T23:33:30.967197+0000","last_peered":"2024-09-10T23:33:30.967197+0000","last_clean":"2024-09-10T23:33:30.967197+0000","last_became_active":"2024-09-10T23:33:28.884235+0000","last_became_peered":"2024-09-10T23:33:28.884235+0000","last_unstale":"2024-09-10T23:33:30.967197+0000","last_undegraded":"2024-09-10T23:33:30.967197+0000","last_fullsized":"2024-09-10T23:33:30.967197+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:42.220925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.899157+0000","last_change":"2024-09-10T23:33:29.894914+0000","last_active":"2024-09-10T23:33:29.899157+0000","last_peered":"2024-09-10T23:33:29.899157+0000","last_clean":"2024-09-10T23:33:29.899157+0000","last_became_active":"2024-09-10T23:33:29.894598+0000","last_became_peered":"2024-09-10T23:33:29.894598+0000","last_unstale":"2024-09-10T23:33:29.899157+0000","last_undegraded":"2024-09-10T23:33:29.899157+0000","last_fullsized":"2024-09-10T23:33:29.899157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:10.328626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.893748+0000","last_change":"2024-09-10T23:33:28.896382+0000","last_active":"2024-09-10T23:33:30.893748+0000","last_peered":"2024-09-10T23:33:30.893748+0000","last_clean":"2024-09-10T23:33:30.893748+0000","last_became_active":"2024-09-10T23:33:28.895614+0000","last_became_peered":"2024-09-10T23:33:28.895614+0000","last_unstale":"2024-09-10T23:33:30.893748+0000","last_undegraded":"2024-09-10T23:33:30.893748+0000","last_fullsized":"2024-09-10T23:33:30.893748+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:19.167006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.893568+0000","last_change":"2024-09-10T23:33:29.901415+0000","last_active":"2024-09-10T23:33:30.893568+0000","last_peered":"2024-09-10T23:33:30.893568+0000","last_clean":"2024-09-10T23:33:30.893568+0000","last_became_active":"2024-09-10T23:33:29.900880+0000","last_became_peered":"2024-09-10T23:33:29.900880+0000","last_unstale":"2024-09-10T23:33:30.893568+0000","last_undegraded":"2024-09-10T23:33:30.893568+0000","last_fullsized":"2024-09-10T23:33:30.893568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:21.241619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967844+0000","last_change":"2024-09-10T23:33:28.899830+0000","last_active":"2024-09-10T23:33:30.967844+0000","last_peered":"2024-09-10T23:33:30.967844+0000","last_clean":"2024-09-10T23:33:30.967844+0000","last_became_active":"2024-09-10T23:33:28.899370+0000","last_became_peered":"2024-09-10T23:33:28.899370+0000","last_unstale":"2024-09-10T23:33:30.967844+0000","last_undegraded":"2024-09-10T23:33:30.967844+0000","last_fullsized":"2024-09-10T23:33:30.967844+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:43.439095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.899121+0000","last_change":"2024-09-10T23:33:29.891316+0000","last_active":"2024-09-10T23:33:29.899121+0000","last_peered":"2024-09-10T23:33:29.899121+0000","last_clean":"2024-09-10T23:33:29.899121+0000","last_became_active":"2024-09-10T23:33:29.890904+0000","last_became_peered":"2024-09-10T23:33:29.890904+0000","last_unstale":"2024-09-10T23:33:29.899121+0000","last_undegraded":"2024-09-10T23:33:29.899121+0000","last_fullsized":"2024-09-10T23:33:29.899121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:43.646370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966618+0000","last_change":"2024-09-10T23:33:28.892194+0000","last_active":"2024-09-10T23:33:30.966618+0000","last_peered":"2024-09-10T23:33:30.966618+0000","last_clean":"2024-09-10T23:33:30.966618+0000","last_became_active":"2024-09-10T23:33:28.891870+0000","last_became_peered":"2024-09-10T23:33:28.891870+0000","last_unstale":"2024-09-10T23:33:30.966618+0000","last_undegraded":"2024-09-10T23:33:30.966618+0000","last_fullsized":"2024-09-10T23:33:30.966618+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:58.777973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966521+0000","last_change":"2024-09-10T23:33:29.887363+0000","last_active":"2024-09-10T23:33:30.966521+0000","last_peered":"2024-09-10T23:33:30.966521+0000","last_clean":"2024-09-10T23:33:30.966521+0000","last_became_active":"2024-09-10T23:33:29.886873+0000","last_became_peered":"2024-09-10T23:33:29.886873+0000","last_unstale":"2024-09-10T23:33:30.966521+0000","last_undegraded":"2024-09-10T23:33:30.966521+0000","last_fullsized":"2024-09-10T23:33:30.966521+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:01.796185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.890827+0000","last_change":"2024-09-10T23:33:28.891920+0000","last_active":"2024-09-10T23:33:30.890827+0000","last_peered":"2024-09-10T23:33:30.890827+0000","last_clean":"2024-09-10T23:33:30.890827+0000","last_became_active":"2024-09-10T23:33:28.891616+0000","last_became_peered":"2024-09-10T23:33:28.891616+0000","last_unstale":"2024-09-10T23:33:30.890827+0000","last_undegraded":"2024-09-10T23:33:30.890827+0000","last_fullsized":"2024-09-10T23:33:30.890827+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:58.835412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900900+0000","last_change":"2024-09-10T23:33:29.905625+0000","last_active":"2024-09-10T23:33:30.900900+0000","last_peered":"2024-09-10T23:33:30.900900+0000","last_clean":"2024-09-10T23:33:30.900900+0000","last_became_active":"2024-09-10T23:33:29.905365+0000","last_became_peered":"2024-09-10T23:33:29.905365+0000","last_unstale":"2024-09-10T23:33:30.900900+0000","last_undegraded":"2024-09-10T23:33:30.900900+0000","last_fullsized":"2024-09-10T23:33:30.900900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:46.762294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.920409+0000","last_change":"2024-09-10T23:33:28.892990+0000","last_active":"2024-09-10T23:33:29.920409+0000","last_peered":"2024-09-10T23:33:29.920409+0000","last_clean":"2024-09-10T23:33:29.920409+0000","last_became_active":"2024-09-10T23:33:28.892645+0000","last_became_peered":"2024-09-10T23:33:28.892645+0000","last_unstale":"2024-09-10T23:33:29.920409+0000","last_undegraded":"2024-09-10T23:33:29.920409+0000","last_fullsized":"2024-09-10T23:33:29.920409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:51:33.842104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891323+0000","last_change":"2024-09-10T23:33:29.903694+0000","last_active":"2024-09-10T23:33:30.891323+0000","last_peered":"2024-09-10T23:33:30.891323+0000","last_clean":"2024-09-10T23:33:30.891323+0000","last_became_active":"2024-09-10T23:33:29.903459+0000","last_became_peered":"2024-09-10T23:33:29.903459+0000","last_unstale":"2024-09-10T23:33:30.891323+0000","last_undegraded":"2024-09-10T23:33:30.891323+0000","last_fullsized":"2024-09-10T23:33:30.891323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:48.137767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.899654+0000","last_change":"2024-09-10T23:33:28.894813+0000","last_active":"2024-09-10T23:33:30.899654+0000","last_peered":"2024-09-10T23:33:30.899654+0000","last_clean":"2024-09-10T23:33:30.899654+0000","last_became_active":"2024-09-10T23:33:28.894537+0000","last_became_peered":"2024-09-10T23:33:28.894537+0000","last_unstale":"2024-09-10T23:33:30.899654+0000","last_undegraded":"2024-09-10T23:33:30.899654+0000","last_fullsized":"2024-09-10T23:33:30.899654+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:38.526599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.900872+0000","last_change":"2024-09-10T23:33:29.913940+0000","last_active":"2024-09-10T23:33:30.900872+0000","last_peered":"2024-09-10T23:33:30.900872+0000","last_clean":"2024-09-10T23:33:30.900872+0000","last_became_active":"2024-09-10T23:33:29.913669+0000","last_became_peered":"2024-09-10T23:33:29.913669+0000","last_unstale":"2024-09-10T23:33:30.900872+0000","last_undegraded":"2024-09-10T23:33:30.900872+0000","last_fullsized":"2024-09-10T23:33:30.900872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:21.163092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.886780+0000","last_change":"2024-09-10T23:33:28.885695+0000","last_active":"2024-09-10T23:33:29.886780+0000","last_peered":"2024-09-10T23:33:29.886780+0000","last_clean":"2024-09-10T23:33:29.886780+0000","last_became_active":"2024-09-10T23:33:28.885392+0000","last_became_peered":"2024-09-10T23:33:28.885392+0000","last_unstale":"2024-09-10T23:33:29.886780+0000","last_undegraded":"2024-09-10T23:33:29.886780+0000","last_fullsized":"2024-09-10T23:33:29.886780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:36.163987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966607+0000","last_change":"2024-09-10T23:33:29.899251+0000","last_active":"2024-09-10T23:33:30.966607+0000","last_peered":"2024-09-10T23:33:30.966607+0000","last_clean":"2024-09-10T23:33:30.966607+0000","last_became_active":"2024-09-10T23:33:29.898983+0000","last_became_peered":"2024-09-10T23:33:29.898983+0000","last_unstale":"2024-09-10T23:33:30.966607+0000","last_undegraded":"2024-09-10T23:33:30.966607+0000","last_fullsized":"2024-09-10T23:33:30.966607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:00.564474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.886543+0000","last_change":"2024-09-10T23:33:28.892866+0000","last_active":"2024-09-10T23:33:29.886543+0000","last_peered":"2024-09-10T23:33:29.886543+0000","last_clean":"2024-09-10T23:33:29.886543+0000","last_became_active":"2024-09-10T23:33:28.892590+0000","last_became_peered":"2024-09-10T23:33:28.892590+0000","last_unstale":"2024-09-10T23:33:29.886543+0000","last_undegraded":"2024-09-10T23:33:29.886543+0000","last_fullsized":"2024-09-10T23:33:29.886543+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:47.706256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967795+0000","last_change":"2024-09-10T23:33:29.895113+0000","last_active":"2024-09-10T23:33:30.967795+0000","last_peered":"2024-09-10T23:33:30.967795+0000","last_clean":"2024-09-10T23:33:30.967795+0000","last_became_active":"2024-09-10T23:33:29.894478+0000","last_became_peered":"2024-09-10T23:33:29.894478+0000","last_unstale":"2024-09-10T23:33:30.967795+0000","last_undegraded":"2024-09-10T23:33:30.967795+0000","last_fullsized":"2024-09-10T23:33:30.967795+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:35:21.216448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T23:33:29.885731+0000","last_change":"2024-09-10T23:33:28.898816+0000","last_active":"2024-09-10T23:33:29.885731+0000","last_peered":"2024-09-10T23:33:29.885731+0000","last_clean":"2024-09-10T23:33:29.885731+0000","last_became_active":"2024-09-10T23:33:28.898122+0000","last_became_peered":"2024-09-10T23:33:28.898122+0000","last_unstale":"2024-09-10T23:33:29.885731+0000","last_undegraded":"2024-09-10T23:33:29.885731+0000","last_fullsized":"2024-09-10T23:33:29.885731+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:13.526033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967326+0000","last_change":"2024-09-10T23:33:29.898566+0000","last_active":"2024-09-10T23:33:30.967326+0000","last_peered":"2024-09-10T23:33:30.967326+0000","last_clean":"2024-09-10T23:33:30.967326+0000","last_became_active":"2024-09-10T23:33:29.898272+0000","last_became_peered":"2024-09-10T23:33:29.898272+0000","last_unstale":"2024-09-10T23:33:30.967326+0000","last_undegraded":"2024-09-10T23:33:30.967326+0000","last_fullsized":"2024-09-10T23:33:30.967326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:21.241619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967789+0000","last_change":"2024-09-10T23:33:28.899677+0000","last_active":"2024-09-10T23:33:30.967789+0000","last_peered":"2024-09-10T23:33:30.967789+0000","last_clean":"2024-09-10T23:33:30.967789+0000","last_became_active":"2024-09-10T23:33:28.899342+0000","last_became_peered":"2024-09-10T23:33:28.899342+0000","last_unstale":"2024-09-10T23:33:30.967789+0000","last_undegraded":"2024-09-10T23:33:30.967789+0000","last_fullsized":"2024-09-10T23:33:30.967789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:26.426371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891257+0000","last_change":"2024-09-10T23:33:29.898700+0000","last_active":"2024-09-10T23:33:30.891257+0000","last_peered":"2024-09-10T23:33:30.891257+0000","last_clean":"2024-09-10T23:33:30.891257+0000","last_became_active":"2024-09-10T23:33:29.897992+0000","last_became_peered":"2024-09-10T23:33:29.897992+0000","last_unstale":"2024-09-10T23:33:30.891257+0000","last_undegraded":"2024-09-10T23:33:30.891257+0000","last_fullsized":"2024-09-10T23:33:30.891257+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:55.760343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967333+0000","last_change":"2024-09-10T23:33:28.888035+0000","last_active":"2024-09-10T23:33:30.967333+0000","last_peered":"2024-09-10T23:33:30.967333+0000","last_clean":"2024-09-10T23:33:30.967333+0000","last_became_active":"2024-09-10T23:33:28.887745+0000","last_became_peered":"2024-09-10T23:33:28.887745+0000","last_unstale":"2024-09-10T23:33:30.967333+0000","last_undegraded":"2024-09-10T23:33:30.967333+0000","last_fullsized":"2024-09-10T23:33:30.967333+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:51.076870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.967142+0000","last_change":"2024-09-10T23:33:29.899193+0000","last_active":"2024-09-10T23:33:30.967142+0000","last_peered":"2024-09-10T23:33:30.967142+0000","last_clean":"2024-09-10T23:33:30.967142+0000","last_became_active":"2024-09-10T23:33:29.898941+0000","last_became_peered":"2024-09-10T23:33:29.898941+0000","last_unstale":"2024-09-10T23:33:30.967142+0000","last_undegraded":"2024-09-10T23:33:30.967142+0000","last_fullsized":"2024-09-10T23:33:30.967142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:28.864196+0000","last_clean_scrub_stamp":"2024-09-10T23:33:28.864196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:21.241619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.891379+0000","last_change":"2024-09-10T23:33:28.892699+0000","last_active":"2024-09-10T23:33:30.891379+0000","last_peered":"2024-09-10T23:33:30.891379+0000","last_clean":"2024-09-10T23:33:30.891379+0000","last_became_active":"2024-09-10T23:33:28.892461+0000","last_became_peered":"2024-09-10T23:33:28.892461+0000","last_unstale":"2024-09-10T23:33:30.891379+0000","last_undegraded":"2024-09-10T23:33:30.891379+0000","last_fullsized":"2024-09-10T23:33:30.891379+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:27.858170+0000","last_clean_scrub_stamp":"2024-09-10T23:33:27.858170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:49.235674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":132,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-10T23:33:30.966368+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:33:30.966368+0000","last_peered":"2024-09-10T23:33:30.966368+0000","last_clean":"2024-09-10T23:33:30.966368+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:33:30.966368+0000","last_undegraded":"2024-09-10T23:33:30.966368+0000","last_fullsized":"2024-09-10T23:33:30.966368+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":368640,"data_stored":139589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":950272,"data_stored":724217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":141201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":380928,"data_stored":140343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":368640,"data_stored":139601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":950272,"data_stored":723705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":356352,"data_stored":133357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":364544,"data_stored":134879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:33:34.378 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T23:33:34.378 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T23:33:34.496 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:33:34.500 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:33:34.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.564+0000 7f7306a00640 1 -- 172.21.15.94:0/398014501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73000c49b0 msgr2=0x7f7300165b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:34.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.564+0000 7f7306a00640 1 --2- 172.21.15.94:0/398014501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73000c49b0 0x7f7300165b10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009840 tx=0x7f72f801c820 comp rx=0 tx=0).stop 2024-09-10T23:33:34.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.564+0000 7f7306a00640 1 -- 172.21.15.94:0/398014501 shutdown_connections 2024-09-10T23:33:34.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.564+0000 7f7306a00640 1 --2- 172.21.15.94:0/398014501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73000c49b0 0x7f7300165b10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:34.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.564+0000 7f7306a00640 1 -- 172.21.15.94:0/398014501 >> 172.21.15.94:0/398014501 conn(0x7f73000cfaf0 msgr2=0x7f73000cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:34.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.564+0000 7f7306a00640 1 -- 172.21.15.94:0/398014501 shutdown_connections 2024-09-10T23:33:34.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.564+0000 7f7306a00640 1 -- 172.21.15.94:0/398014501 wait complete. 2024-09-10T23:33:34.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f7306a00640 1 Processor -- start 2024-09-10T23:33:34.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f7306a00640 1 -- start start 2024-09-10T23:33:34.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f7306a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f730006fc00 0x7f7300159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:34.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f7306a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73000c49b0 0x7f7300154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:34.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f7306a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7300158790 0x7f73000733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:34.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f7306a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7300168f30 con 0x7f73000c49b0 2024-09-10T23:33:34.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f7306a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7300168db0 con 0x7f730006fc00 2024-09-10T23:33:34.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f7306a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f73001690b0 con 0x7f7300158790 2024-09-10T23:33:34.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f72f7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73000c49b0 0x7f7300154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:34.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f72f7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73000c49b0 0x7f7300154fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34072/0 (socket says 172.21.15.94:34072) 2024-09-10T23:33:34.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f72f7400640 1 -- 172.21.15.94:0/3196258329 learned_addr learned my addr 172.21.15.94:0/3196258329 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:34.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f72ffe00640 1 --2- 172.21.15.94:0/3196258329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f730006fc00 0x7f7300159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:34.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f7304a00640 1 --2- 172.21.15.94:0/3196258329 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7300158790 0x7f73000733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:34.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f72f7400640 1 -- 172.21.15.94:0/3196258329 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7300158790 msgr2=0x7f73000733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:34.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f72f7400640 1 --2- 172.21.15.94:0/3196258329 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7300158790 0x7f73000733c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:34.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f72f7400640 1 -- 172.21.15.94:0/3196258329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f730006fc00 msgr2=0x7f7300159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:34.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f72f7400640 1 --2- 172.21.15.94:0/3196258329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f730006fc00 0x7f7300159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:34.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f72f7400640 1 -- 172.21.15.94:0/3196258329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7300155500 con 0x7f73000c49b0 2024-09-10T23:33:34.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f7304a00640 1 --2- 172.21.15.94:0/3196258329 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7300158790 0x7f73000733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:34.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f72f7400640 1 --2- 172.21.15.94:0/3196258329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73000c49b0 0x7f7300154fc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f72e800c450 tx=0x7f72e800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:34.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f72fcc00640 1 -- 172.21.15.94:0/3196258329 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8016020 con 0x7f73000c49b0 2024-09-10T23:33:34.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f72fcc00640 1 -- 172.21.15.94:0/3196258329 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72e8012940 con 0x7f73000c49b0 2024-09-10T23:33:34.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f7306a00640 1 -- 172.21.15.94:0/3196258329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7300072600 con 0x7f73000c49b0 2024-09-10T23:33:34.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f72fcc00640 1 -- 172.21.15.94:0/3196258329 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8012d90 con 0x7f73000c49b0 2024-09-10T23:33:34.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.568+0000 7f7306a00640 1 -- 172.21.15.94:0/3196258329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7300072bc0 con 0x7f73000c49b0 2024-09-10T23:33:34.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.572+0000 7f7306a00640 1 -- 172.21.15.94:0/3196258329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73000d0850 con 0x7f73000c49b0 2024-09-10T23:33:34.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.572+0000 7f72fcc00640 1 -- 172.21.15.94:0/3196258329 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f72e801a3b0 con 0x7f73000c49b0 2024-09-10T23:33:34.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.572+0000 7f72fcc00640 1 --2- 172.21.15.94:0/3196258329 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f72dc0725e0 0x7f72dc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:34.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.572+0000 7f72fcc00640 1 -- 172.21.15.94:0/3196258329 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f72e8093140 con 0x7f73000c49b0 2024-09-10T23:33:34.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.572+0000 7f72ffe00640 1 --2- 172.21.15.94:0/3196258329 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f72dc0725e0 0x7f72dc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:34.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.576+0000 7f72ffe00640 1 --2- 172.21.15.94:0/3196258329 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f72dc0725e0 0x7f72dc074aa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f73000c68d0 tx=0x7f72f000d040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:34.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.576+0000 7f72fcc00640 1 -- 172.21.15.94:0/3196258329 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f72e8020030 con 0x7f73000c49b0 2024-09-10T23:33:34.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.800+0000 7f7306a00640 1 -- 172.21.15.94:0/3196258329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f73000cada0 con 0x7f73000c49b0 2024-09-10T23:33:34.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.804+0000 7f72fcc00640 1 -- 172.21.15.94:0/3196258329 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v28) ==== 190+0+147 (secure 0 0 0) 0x7f72e805e820 con 0x7f73000c49b0 2024-09-10T23:33:34.807 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-10T23:33:34.807 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g== 2024-09-10T23:33:34.807 INFO:teuthology.orchestra.run.smithi094.stdout: caps mds = "allow" 2024-09-10T23:33:34.807 INFO:teuthology.orchestra.run.smithi094.stdout: caps mon = "allow r" 2024-09-10T23:33:34.807 INFO:teuthology.orchestra.run.smithi094.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T23:33:34.807 INFO:teuthology.orchestra.run.smithi094.stderr:updated caps for client.0 2024-09-10T23:33:34.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.804+0000 7f7306a00640 1 -- 172.21.15.94:0/3196258329 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f72dc0725e0 msgr2=0x7f72dc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:34.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.804+0000 7f7306a00640 1 --2- 172.21.15.94:0/3196258329 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f72dc0725e0 0x7f72dc074aa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f73000c68d0 tx=0x7f72f000d040 comp rx=0 tx=0).stop 2024-09-10T23:33:34.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.804+0000 7f7306a00640 1 -- 172.21.15.94:0/3196258329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73000c49b0 msgr2=0x7f7300154fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:34.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.804+0000 7f7306a00640 1 --2- 172.21.15.94:0/3196258329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73000c49b0 0x7f7300154fc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f72e800c450 tx=0x7f72e800c910 comp rx=0 tx=0).stop 2024-09-10T23:33:34.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.804+0000 7f7306a00640 1 -- 172.21.15.94:0/3196258329 shutdown_connections 2024-09-10T23:33:34.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.804+0000 7f7306a00640 1 --2- 172.21.15.94:0/3196258329 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f72dc0725e0 0x7f72dc074aa0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:34.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.804+0000 7f7306a00640 1 --2- 172.21.15.94:0/3196258329 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7300158790 0x7f73000733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:34.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.804+0000 7f7306a00640 1 --2- 172.21.15.94:0/3196258329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73000c49b0 0x7f7300154fc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:34.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.804+0000 7f7306a00640 1 --2- 172.21.15.94:0/3196258329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f730006fc00 0x7f7300159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:34.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.804+0000 7f7306a00640 1 -- 172.21.15.94:0/3196258329 >> 172.21.15.94:0/3196258329 conn(0x7f73000cfaf0 msgr2=0x7f73000af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:34.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.804+0000 7f7306a00640 1 -- 172.21.15.94:0/3196258329 shutdown_connections 2024-09-10T23:33:34.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:34.804+0000 7f7306a00640 1 -- 172.21.15.94:0/3196258329 wait complete. 2024-09-10T23:33:34.829 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:33:34.864 INFO:tasks.ceph.mds.b.smithi181.stderr:2024-09-10T23:33:34.861+0000 7f20d2c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:33:35.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.008+0000 7f1533e00640 1 -- 172.21.15.94:0/4202357681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1518003a10 msgr2=0x7f1518003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.008+0000 7f1533e00640 1 --2- 172.21.15.94:0/4202357681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1518003a10 0x7f1518003e30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f152400c290 tx=0x7f152401cfa0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.008+0000 7f1533400640 1 -- 172.21.15.94:0/4202357681 reap_dead start 2024-09-10T23:33:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.008+0000 7f1533e00640 1 -- 172.21.15.94:0/4202357681 shutdown_connections 2024-09-10T23:33:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.008+0000 7f1533e00640 1 -- 172.21.15.94:0/4202357681 >> 172.21.15.94:0/4202357681 conn(0x7f15340d0110 msgr2=0x7f15340d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.008+0000 7f1533e00640 1 -- 172.21.15.94:0/4202357681 shutdown_connections 2024-09-10T23:33:35.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.008+0000 7f1533e00640 1 -- 172.21.15.94:0/4202357681 wait complete. 2024-09-10T23:33:35.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.008+0000 7f1533e00640 1 Processor -- start 2024-09-10T23:33:35.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.008+0000 7f1533e00640 1 -- start start 2024-09-10T23:33:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1533e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1518003a10 0x7f1534155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1533e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1534162db0 0x7f1534161980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1533e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15341f77c0 0x7f1534161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1533e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1534074550 con 0x7f1518003a10 2024-09-10T23:33:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1533e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f15340743d0 con 0x7f15341f77c0 2024-09-10T23:33:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1533e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1534165780 con 0x7f1534162db0 2024-09-10T23:33:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1532a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1518003a10 0x7f1534155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1532a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1518003a10 0x7f1534155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34102/0 (socket says 172.21.15.94:34102) 2024-09-10T23:33:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1532a00640 1 -- 172.21.15.94:0/1091687013 learned_addr learned my addr 172.21.15.94:0/1091687013 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f152be00640 1 --2- 172.21.15.94:0/1091687013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1534162db0 0x7f1534161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1533400640 1 --2- 172.21.15.94:0/1091687013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15341f77c0 0x7f1534161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1532a00640 1 -- 172.21.15.94:0/1091687013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1534162db0 msgr2=0x7f1534161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1532a00640 1 --2- 172.21.15.94:0/1091687013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1534162db0 0x7f1534161980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1532a00640 1 -- 172.21.15.94:0/1091687013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15341f77c0 msgr2=0x7f1534161ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1532a00640 1 --2- 172.21.15.94:0/1091687013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15341f77c0 0x7f1534161ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1532a00640 1 -- 172.21.15.94:0/1091687013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15340cd6e0 con 0x7f1518003a10 2024-09-10T23:33:35.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1532a00640 1 --2- 172.21.15.94:0/1091687013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1518003a10 0x7f1534155e10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f152401ca60 tx=0x7f1524006d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:35.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f152aa00640 1 -- 172.21.15.94:0/1091687013 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1524006440 con 0x7f1518003a10 2024-09-10T23:33:35.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f152aa00640 1 -- 172.21.15.94:0/1091687013 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1524003600 con 0x7f1518003a10 2024-09-10T23:33:35.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1533e00640 1 -- 172.21.15.94:0/1091687013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15340cd9c0 con 0x7f1518003a10 2024-09-10T23:33:35.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f152aa00640 1 -- 172.21.15.94:0/1091687013 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1524003a10 con 0x7f1518003a10 2024-09-10T23:33:35.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1533e00640 1 -- 172.21.15.94:0/1091687013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15340cdf80 con 0x7f1518003a10 2024-09-10T23:33:35.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.012+0000 7f1533e00640 1 -- 172.21.15.94:0/1091687013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153415b880 con 0x7f1518003a10 2024-09-10T23:33:35.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.016+0000 7f152aa00640 1 -- 172.21.15.94:0/1091687013 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1524003bb0 con 0x7f1518003a10 2024-09-10T23:33:35.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.016+0000 7f152aa00640 1 --2- 172.21.15.94:0/1091687013 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f15100725e0 0x7f1510074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.016+0000 7f152aa00640 1 -- 172.21.15.94:0/1091687013 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f152412dc90 con 0x7f1518003a10 2024-09-10T23:33:35.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.016+0000 7f152be00640 1 --2- 172.21.15.94:0/1091687013 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f15100725e0 0x7f1510074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.016+0000 7f152be00640 1 --2- 172.21.15.94:0/1091687013 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f15100725e0 0x7f1510074aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1534074a60 tx=0x7f151c005f70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:35.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.020+0000 7f152aa00640 1 -- 172.21.15.94:0/1091687013 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f15240bf060 con 0x7f1518003a10 2024-09-10T23:33:35.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.216+0000 7f1533e00640 1 -- 172.21.15.94:0/1091687013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15340713a0 con 0x7f1518003a10 2024-09-10T23:33:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.220+0000 7f152aa00640 1 -- 172.21.15.94:0/1091687013 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5238 (secure 0 0 0) 0x7f1524016d50 con 0x7f1518003a10 2024-09-10T23:33:35.223 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:35.223 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":16,"btime":"2024-09-10T23:33:32:900364+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4900,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/765435890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":765435890},{"type":"v1","addr":"172.21.15.181:6837","nonce":765435890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3766049154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3766049154},{"type":"v1","addr":"172.21.15.94:6834","nonce":3766049154}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":194555,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2464857180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2464857180},{"type":"v1","addr":"172.21.15.94:6835","nonce":2464857180}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:29.261378+0000","modified":"2024-09-10T23:33:32.284558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194558},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194558":{"gid":194558,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1569801556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1569801556},{"type":"v1","addr":"172.21.15.181:6836","nonce":1569801556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194558,"qdb_cluster":[194558]},"id":2}]} 2024-09-10T23:33:35.223 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 16 2024-09-10T23:33:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.220+0000 7f1533e00640 1 -- 172.21.15.94:0/1091687013 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f15100725e0 msgr2=0x7f1510074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.220+0000 7f1533e00640 1 --2- 172.21.15.94:0/1091687013 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f15100725e0 0x7f1510074aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1534074a60 tx=0x7f151c005f70 comp rx=0 tx=0).stop 2024-09-10T23:33:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.220+0000 7f1533e00640 1 -- 172.21.15.94:0/1091687013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1518003a10 msgr2=0x7f1534155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.220+0000 7f1533e00640 1 --2- 172.21.15.94:0/1091687013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1518003a10 0x7f1534155e10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f152401ca60 tx=0x7f1524006d60 comp rx=0 tx=0).stop 2024-09-10T23:33:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.220+0000 7f1533e00640 1 -- 172.21.15.94:0/1091687013 shutdown_connections 2024-09-10T23:33:35.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.220+0000 7f1533e00640 1 --2- 172.21.15.94:0/1091687013 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f15100725e0 0x7f1510074aa0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.220+0000 7f1533e00640 1 --2- 172.21.15.94:0/1091687013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15341f77c0 0x7f1534161ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.220+0000 7f1533e00640 1 --2- 172.21.15.94:0/1091687013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1534162db0 0x7f1534161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.220+0000 7f1533e00640 1 --2- 172.21.15.94:0/1091687013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1518003a10 0x7f1534155e10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.220+0000 7f1533e00640 1 -- 172.21.15.94:0/1091687013 >> 172.21.15.94:0/1091687013 conn(0x7f15340d0110 msgr2=0x7f1534159190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:35.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.220+0000 7f1533e00640 1 -- 172.21.15.94:0/1091687013 shutdown_connections 2024-09-10T23:33:35.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.220+0000 7f1533e00640 1 -- 172.21.15.94:0/1091687013 wait complete. 2024-09-10T23:33:35.646 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T23:33:29.261378+0000', 'modified': '2024-09-10T23:33:32.284558+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 194558}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_194558': {'gid': 194558, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6836/1569801556', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 1569801556}, {'type': 'v1', 'addr': '172.21.15.181:6836', 'nonce': 1569801556}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 194558, 'qdb_cluster': [194558]} 2024-09-10T23:33:35.646 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T23:33:35.647 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T23:33:35.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.784+0000 7febf2800640 1 -- 172.21.15.94:0/3554215283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febec0ce280 msgr2=0x7febec0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.784+0000 7febf2800640 1 --2- 172.21.15.94:0/3554215283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febec0ce280 0x7febec0cf280 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7febdc00a950 tx=0x7febdc01db70 comp rx=0 tx=0).stop 2024-09-10T23:33:35.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.784+0000 7febf2800640 1 -- 172.21.15.94:0/3554215283 shutdown_connections 2024-09-10T23:33:35.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.784+0000 7febf2800640 1 --2- 172.21.15.94:0/3554215283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febec0ce280 0x7febec0cf280 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.784+0000 7febf2800640 1 -- 172.21.15.94:0/3554215283 >> 172.21.15.94:0/3554215283 conn(0x7febec0cfaf0 msgr2=0x7febec0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:35.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.784+0000 7febf2800640 1 -- 172.21.15.94:0/3554215283 shutdown_connections 2024-09-10T23:33:35.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.784+0000 7febf2800640 1 -- 172.21.15.94:0/3554215283 wait complete. 2024-09-10T23:33:35.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febf2800640 1 Processor -- start 2024-09-10T23:33:35.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febf2800640 1 -- start start 2024-09-10T23:33:35.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febf2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febec06e7c0 0x7febec0c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febf2800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febec0c64f0 0x7febec145660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febf2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febec0ce280 0x7febec0af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febf2800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7febec168850 con 0x7febec0ce280 2024-09-10T23:33:35.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febf2800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7febec1686d0 con 0x7febec06e7c0 2024-09-10T23:33:35.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febf2800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7febec1689d0 con 0x7febec0c64f0 2024-09-10T23:33:35.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febebe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febec0ce280 0x7febec0af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febebe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febec0ce280 0x7febec0af5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34122/0 (socket says 172.21.15.94:34122) 2024-09-10T23:33:35.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febebe00640 1 -- 172.21.15.94:0/1610758037 learned_addr learned my addr 172.21.15.94:0/1610758037 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:35.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febeb400640 1 --2- 172.21.15.94:0/1610758037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febec06e7c0 0x7febec0c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febe2a00640 1 --2- 172.21.15.94:0/1610758037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febec0c64f0 0x7febec145660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febebe00640 1 -- 172.21.15.94:0/1610758037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febec0c64f0 msgr2=0x7febec145660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febebe00640 1 --2- 172.21.15.94:0/1610758037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febec0c64f0 0x7febec145660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febebe00640 1 -- 172.21.15.94:0/1610758037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febec06e7c0 msgr2=0x7febec0c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febebe00640 1 --2- 172.21.15.94:0/1610758037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febec06e7c0 0x7febec0c6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febebe00640 1 -- 172.21.15.94:0/1610758037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec0c7b20 con 0x7febec0ce280 2024-09-10T23:33:35.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febebe00640 1 --2- 172.21.15.94:0/1610758037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febec0ce280 0x7febec0af5b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7febe400cbf0 tx=0x7febe400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:35.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febe3e00640 1 -- 172.21.15.94:0/1610758037 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4016020 con 0x7febec0ce280 2024-09-10T23:33:35.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febe3e00640 1 -- 172.21.15.94:0/1610758037 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febe400aae0 con 0x7febec0ce280 2024-09-10T23:33:35.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febe3e00640 1 -- 172.21.15.94:0/1610758037 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4012420 con 0x7febec0ce280 2024-09-10T23:33:35.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febf2800640 1 -- 172.21.15.94:0/1610758037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febec142f70 con 0x7febec0ce280 2024-09-10T23:33:35.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.788+0000 7febf2800640 1 -- 172.21.15.94:0/1610758037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febec0c7250 con 0x7febec0ce280 2024-09-10T23:33:35.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.792+0000 7febe3e00640 1 -- 172.21.15.94:0/1610758037 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7febe401b400 con 0x7febec0ce280 2024-09-10T23:33:35.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.792+0000 7febe3e00640 1 --2- 172.21.15.94:0/1610758037 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7febc80725e0 0x7febc8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.792+0000 7febf5b27000 1 Processor -- start 2024-09-10T23:33:35.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.792+0000 7febe3e00640 1 -- 172.21.15.94:0/1610758037 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7febe40936f0 con 0x7febec0ce280 2024-09-10T23:33:35.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.792+0000 7febf5b27000 1 -- start start 2024-09-10T23:33:35.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.792+0000 7febeb400640 1 --2- 172.21.15.94:0/1610758037 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7febc80725e0 0x7febc8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.792+0000 7febeb400640 1 --2- 172.21.15.94:0/1610758037 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7febc80725e0 0x7febc8074aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7febdc0b0000 tx=0x7febdc01c3b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:35.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.792+0000 7febf5b27000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e7959d0 0x56043e795db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.792+0000 7febf5b27000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7962f0 0x56043e7498d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.792+0000 7febf5b27000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e749e10 0x56043e74a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.792+0000 7febf5b27000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56043e66a3f0 con 0x56043e7962f0 2024-09-10T23:33:35.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.792+0000 7febf5b27000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56043e5a7b20 con 0x56043e749e10 2024-09-10T23:33:35.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.792+0000 7febf5b27000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x56043e596f50 con 0x56043e7959d0 2024-09-10T23:33:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.792+0000 7febe2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7962f0 0x56043e7498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.792+0000 7febe2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7962f0 0x56043e7498d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34124/0 (socket says 172.21.15.94:34124) 2024-09-10T23:33:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.792+0000 7febe2a00640 1 -- 172.21.15.94:0/2219652264 learned_addr learned my addr 172.21.15.94:0/2219652264 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febebe00640 1 --2- 172.21.15.94:0/2219652264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e749e10 0x56043e74a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febeb400640 1 --2- 172.21.15.94:0/2219652264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e7959d0 0x56043e795db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febe2a00640 1 -- 172.21.15.94:0/2219652264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e7959d0 msgr2=0x56043e795db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:33:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febe2a00640 1 --2- 172.21.15.94:0/2219652264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e7959d0 0x56043e795db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febe2a00640 1 -- 172.21.15.94:0/2219652264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e749e10 msgr2=0x56043e74a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:33:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febe2a00640 1 --2- 172.21.15.94:0/2219652264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e749e10 0x56043e74a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febe2a00640 1 -- 172.21.15.94:0/2219652264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56043e55e590 con 0x56043e7962f0 2024-09-10T23:33:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febe2a00640 1 --2- 172.21.15.94:0/2219652264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7962f0 0x56043e7498d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7febec1662a0 tx=0x7febd4007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cb945517c11d20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febdbe00640 1 -- 172.21.15.94:0/2219652264 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd40052e0 con 0x56043e7962f0 2024-09-10T23:33:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febdbe00640 1 -- 172.21.15.94:0/2219652264 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febd4018070 con 0x56043e7962f0 2024-09-10T23:33:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febdbe00640 1 -- 172.21.15.94:0/2219652264 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd400f9e0 con 0x56043e7962f0 2024-09-10T23:33:35.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 -- 172.21.15.94:0/2219652264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7962f0 msgr2=0x56043e7498d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 --2- 172.21.15.94:0/2219652264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7962f0 0x56043e7498d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7febec1662a0 tx=0x7febd4007e20 comp rx=0 tx=0).stop 2024-09-10T23:33:35.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 -- 172.21.15.94:0/2219652264 shutdown_connections 2024-09-10T23:33:35.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 --2- 172.21.15.94:0/2219652264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e749e10 0x56043e74a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 --2- 172.21.15.94:0/2219652264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7962f0 0x56043e7498d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 --2- 172.21.15.94:0/2219652264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e7959d0 0x56043e795db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 -- 172.21.15.94:0/2219652264 >> 172.21.15.94:0/2219652264 conn(0x56043e633130 msgr2=0x56043e778e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:35.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 -- 172.21.15.94:0/2219652264 shutdown_connections 2024-09-10T23:33:35.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 -- 172.21.15.94:0/2219652264 wait complete. 2024-09-10T23:33:35.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 -- start start 2024-09-10T23:33:35.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 Processor -- start 2024-09-10T23:33:35.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e749e10 0x56043e74b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7959d0 0x56043e74b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e7962f0 0x56043e74bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56043e5315d0 con 0x56043e7959d0 2024-09-10T23:33:35.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56043e59ad60 con 0x56043e7962f0 2024-09-10T23:33:35.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x56043e53d500 con 0x56043e749e10 2024-09-10T23:33:35.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febf5b27000 10 client.0 _ll_register_callbacks cb 0x56043e541810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:33:35.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febe2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7959d0 0x56043e74b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febe2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7959d0 0x56043e74b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34138/0 (socket says 172.21.15.94:34138) 2024-09-10T23:33:35.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febe2a00640 1 -- 172.21.15.94:0/3268857612 learned_addr learned my addr 172.21.15.94:0/3268857612 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:35.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febebe00640 1 --2- 172.21.15.94:0/3268857612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e7962f0 0x56043e74bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.796+0000 7febeb400640 1 --2- 172.21.15.94:0/3268857612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e749e10 0x56043e74b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febe2a00640 1 -- 172.21.15.94:0/3268857612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e749e10 msgr2=0x56043e74b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:33:35.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febe2a00640 1 --2- 172.21.15.94:0/3268857612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e749e10 0x56043e74b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febe2a00640 1 -- 172.21.15.94:0/3268857612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e7962f0 msgr2=0x56043e74bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:33:35.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febe2a00640 1 --2- 172.21.15.94:0/3268857612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e7962f0 0x56043e74bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febe2a00640 1 -- 172.21.15.94:0/3268857612 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56043e535170 con 0x56043e7959d0 2024-09-10T23:33:35.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febe2a00640 1 --2- 172.21.15.94:0/3268857612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7959d0 0x56043e74b780 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x56043e548130 tx=0x7febd400fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db09ba2b4b228b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:35.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:33:35.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febd8c00640 1 -- 172.21.15.94:0/3268857612 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4007230 con 0x56043e7959d0 2024-09-10T23:33:35.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 1 -- 172.21.15.94:0/3268857612 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56043e52f0f0 con 0x56043e7959d0 2024-09-10T23:33:35.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febd8c00640 1 -- 172.21.15.94:0/3268857612 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7febd40073d0 con 0x56043e7959d0 2024-09-10T23:33:35.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febd8c00640 1 -- 172.21.15.94:0/3268857612 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7febd402d050 con 0x56043e7959d0 2024-09-10T23:33:35.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 10 client.5103 fetch_fsmap learned FSMap version 16 2024-09-10T23:33:35.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 1 -- 172.21.15.94:0/3268857612 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x56043e61bef0 con 0x56043e7959d0 2024-09-10T23:33:35.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febd8c00640 1 -- 172.21.15.94:0/3268857612 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56043e52f0f0 con 0x56043e7959d0 2024-09-10T23:33:35.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febd8c00640 1 -- 172.21.15.94:0/3268857612 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7febd4030020 con 0x56043e7959d0 2024-09-10T23:33:35.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 10 client.5103 fetch_fsmap finished waiting for FSMap version 16 2024-09-10T23:33:35.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 10 client.5103 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T23:33:35.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 20 client.5103 populate_metadata read hostname 'smithi094' 2024-09-10T23:33:35.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 1 --2- 172.21.15.94:0/3268857612 >> [v2:172.21.15.181:6834/1569801556,v1:172.21.15.181:6836/1569801556] conn(0x56043e7a4c70 0x56043e7a7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 4 client.5103 mds_command: new command op to 194558 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:33:35.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 1 -- 172.21.15.94:0/3268857612 --> [v2:172.21.15.181:6834/1569801556,v1:172.21.15.181:6836/1569801556] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56043e5a9a70 con 0x56043e7a4c70 2024-09-10T23:33:35.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febeb400640 1 --2- 172.21.15.94:0/3268857612 >> [v2:172.21.15.181:6834/1569801556,v1:172.21.15.181:6836/1569801556] conn(0x56043e7a4c70 0x56043e7a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febeb400640 1 --2- 172.21.15.94:0/3268857612 >> [v2:172.21.15.181:6834/1569801556,v1:172.21.15.181:6836/1569801556] conn(0x56043e7a4c70 0x56043e7a7060 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34978a0b3f728312 server_cookie=9f1d082a59948a8a in_seq=0 out_seq=0 2024-09-10T23:33:35.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febd8c00640 10 client.5103 ms_handle_connect on v2:172.21.15.181:6834/1569801556 2024-09-10T23:33:35.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febd8c00640 1 -- 172.21.15.94:0/3268857612 <== mds.0 v2:172.21.15.181:6834/1569801556 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56043e5a9a70 con 0x56043e7a4c70 2024-09-10T23:33:35.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febd8c00640 10 client.5103 handle_command_reply: tid=0 2024-09-10T23:33:35.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 1 client.5103 shutdown 2024-09-10T23:33:35.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 1 -- 172.21.15.94:0/3268857612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7959d0 msgr2=0x56043e74b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 1 --2- 172.21.15.94:0/3268857612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7959d0 0x56043e74b780 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x56043e548130 tx=0x7febd400fd70 comp rx=0 tx=0).stop 2024-09-10T23:33:35.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 1 -- 172.21.15.94:0/3268857612 shutdown_connections 2024-09-10T23:33:35.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 1 --2- 172.21.15.94:0/3268857612 >> [v2:172.21.15.181:6834/1569801556,v1:172.21.15.181:6836/1569801556] conn(0x56043e7a4c70 0x56043e7a7060 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 1 --2- 172.21.15.94:0/3268857612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e7962f0 0x56043e74bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 1 --2- 172.21.15.94:0/3268857612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7959d0 0x56043e74b780 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 1 --2- 172.21.15.94:0/3268857612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e749e10 0x56043e74b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febd8c00640 0 client.5103 ms_handle_reset on v2:172.21.15.181:6834/1569801556 2024-09-10T23:33:35.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 1 -- 172.21.15.94:0/3268857612 >> 172.21.15.94:0/3268857612 conn(0x56043e633130 msgr2=0x56043e748fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:35.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.800+0000 7febf5b27000 1 -- 172.21.15.94:0/3268857612 shutdown_connections 2024-09-10T23:33:35.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.804+0000 7febf5b27000 1 -- 172.21.15.94:0/3268857612 wait complete. 2024-09-10T23:33:35.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.804+0000 7febf5b27000 20 client.5103 trim_cache size 0 max 16384 2024-09-10T23:33:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.820+0000 7febf5b27000 1 Processor -- start 2024-09-10T23:33:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.820+0000 7febf5b27000 1 -- start start 2024-09-10T23:33:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.820+0000 7febf5b27000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e749e10 0x56043e80fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.820+0000 7febf5b27000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e7959d0 0x56043e810110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.820+0000 7febf5b27000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e7962f0 0x56043e810650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.820+0000 7febf5b27000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7febec202e40 con 0x56043e749e10 2024-09-10T23:33:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.820+0000 7febf5b27000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7febec2032e0 con 0x56043e7962f0 2024-09-10T23:33:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.820+0000 7febf5b27000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x56043e59f390 con 0x56043e7959d0 2024-09-10T23:33:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.820+0000 7febeb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e749e10 0x56043e80fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.820+0000 7febeb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e749e10 0x56043e80fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34140/0 (socket says 172.21.15.94:34140) 2024-09-10T23:33:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.820+0000 7febeb400640 1 -- 172.21.15.94:0/3417322900 learned_addr learned my addr 172.21.15.94:0/3417322900 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.820+0000 7febeb400640 1 -- 172.21.15.94:0/3417322900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e7959d0 msgr2=0x56043e810110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:33:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.820+0000 7febeb400640 1 --2- 172.21.15.94:0/3417322900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e7959d0 0x56043e810110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.820+0000 7febeb400640 1 -- 172.21.15.94:0/3417322900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e7962f0 msgr2=0x56043e810650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:33:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.820+0000 7febeb400640 1 --2- 172.21.15.94:0/3417322900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e7962f0 0x56043e810650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.820+0000 7febeb400640 1 -- 172.21.15.94:0/3417322900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56043e5f2d90 con 0x56043e749e10 2024-09-10T23:33:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febeb400640 1 --2- 172.21.15.94:0/3417322900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e749e10 0x56043e80fbd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7febdc000c10 tx=0x7febdc0b6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c175f13d48c175 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febdb400640 1 -- 172.21.15.94:0/3417322900 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc0d4070 con 0x56043e749e10 2024-09-10T23:33:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febdb400640 1 -- 172.21.15.94:0/3417322900 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febdc0b4040 con 0x56043e749e10 2024-09-10T23:33:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febdb400640 1 -- 172.21.15.94:0/3417322900 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc0046d0 con 0x56043e749e10 2024-09-10T23:33:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 -- 172.21.15.94:0/3417322900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e749e10 msgr2=0x56043e80fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 --2- 172.21.15.94:0/3417322900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e749e10 0x56043e80fbd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7febdc000c10 tx=0x7febdc0b6000 comp rx=0 tx=0).stop 2024-09-10T23:33:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 -- 172.21.15.94:0/3417322900 shutdown_connections 2024-09-10T23:33:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 --2- 172.21.15.94:0/3417322900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e7962f0 0x56043e810650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 --2- 172.21.15.94:0/3417322900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e7959d0 0x56043e810110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 --2- 172.21.15.94:0/3417322900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e749e10 0x56043e80fbd0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 -- 172.21.15.94:0/3417322900 >> 172.21.15.94:0/3417322900 conn(0x56043e633130 msgr2=0x56043e7ace00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 -- 172.21.15.94:0/3417322900 shutdown_connections 2024-09-10T23:33:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 -- 172.21.15.94:0/3417322900 wait complete. 2024-09-10T23:33:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 -- start start 2024-09-10T23:33:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 Processor -- start 2024-09-10T23:33:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e749e10 0x56043e7aba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7959d0 0x56043e7cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e7962f0 0x56043e7d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56043e59a070 con 0x56043e7959d0 2024-09-10T23:33:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56043e5315d0 con 0x56043e7962f0 2024-09-10T23:33:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x56043e5a68f0 con 0x56043e749e10 2024-09-10T23:33:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febf5b27000 10 client.0 _ll_register_callbacks cb 0x7febe4001c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:33:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febe2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7959d0 0x56043e7cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febeb400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e749e10 0x56043e7aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febe2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7959d0 0x56043e7cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34146/0 (socket says 172.21.15.94:34146) 2024-09-10T23:33:35.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febe2a00640 1 -- 172.21.15.94:0/3505014043 learned_addr learned my addr 172.21.15.94:0/3505014043 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:35.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febebe00640 1 --2- 172.21.15.94:0/3505014043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e7962f0 0x56043e7d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febe2a00640 1 -- 172.21.15.94:0/3505014043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e749e10 msgr2=0x56043e7aba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:33:35.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febe2a00640 1 --2- 172.21.15.94:0/3505014043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e749e10 0x56043e7aba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febe2a00640 1 -- 172.21.15.94:0/3505014043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e7962f0 msgr2=0x56043e7d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:33:35.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febe2a00640 1 --2- 172.21.15.94:0/3505014043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e7962f0 0x56043e7d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febe2a00640 1 -- 172.21.15.94:0/3505014043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56043e55da90 con 0x56043e7959d0 2024-09-10T23:33:35.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febe2a00640 1 --2- 172.21.15.94:0/3505014043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7959d0 0x56043e7cfd90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7febd400fb10 tx=0x7febd4003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b990ddbec574c35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:35.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.824+0000 7febd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:33:35.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febd8c00640 1 -- 172.21.15.94:0/3505014043 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4026480 con 0x56043e7959d0 2024-09-10T23:33:35.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febd8c00640 1 -- 172.21.15.94:0/3505014043 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7febd4032aa0 con 0x56043e7959d0 2024-09-10T23:33:35.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 1 -- 172.21.15.94:0/3505014043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febdc0b4040 con 0x56043e7959d0 2024-09-10T23:33:35.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febd8c00640 1 -- 172.21.15.94:0/3505014043 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7febd4032ce0 con 0x56043e7959d0 2024-09-10T23:33:35.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 10 client.5109 fetch_fsmap learned FSMap version 16 2024-09-10T23:33:35.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 1 -- 172.21.15.94:0/3505014043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x56043e53c050 con 0x56043e7959d0 2024-09-10T23:33:35.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febd8c00640 1 -- 172.21.15.94:0/3505014043 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febdc0b4040 con 0x56043e7959d0 2024-09-10T23:33:35.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febd8c00640 1 -- 172.21.15.94:0/3505014043 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7febd4039030 con 0x56043e7959d0 2024-09-10T23:33:35.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 10 client.5109 fetch_fsmap finished waiting for FSMap version 16 2024-09-10T23:33:35.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 10 client.5109 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T23:33:35.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 20 client.5109 populate_metadata read hostname 'smithi094' 2024-09-10T23:33:35.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 1 --2- 172.21.15.94:0/3505014043 >> [v2:172.21.15.181:6834/1569801556,v1:172.21.15.181:6836/1569801556] conn(0x56043e7d0810 0x56043e813e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 4 client.5109 mds_command: new command op to 194558 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T23:33:35.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 1 -- 172.21.15.94:0/3505014043 --> [v2:172.21.15.181:6834/1569801556,v1:172.21.15.181:6836/1569801556] -- command(tid 0: {"prefix": "status"}) -- 0x56043e5a5520 con 0x56043e7d0810 2024-09-10T23:33:35.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febeb400640 1 --2- 172.21.15.94:0/3505014043 >> [v2:172.21.15.181:6834/1569801556,v1:172.21.15.181:6836/1569801556] conn(0x56043e7d0810 0x56043e813e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febd8c00640 10 client.5109 ms_handle_connect on v2:172.21.15.181:6834/1569801556 2024-09-10T23:33:35.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febeb400640 1 --2- 172.21.15.94:0/3505014043 >> [v2:172.21.15.181:6834/1569801556,v1:172.21.15.181:6836/1569801556] conn(0x56043e7d0810 0x56043e813e20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d0af2eb9f796c13 server_cookie=f236ef78a06faceb in_seq=0 out_seq=0 2024-09-10T23:33:35.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febd8c00640 1 -- 172.21.15.94:0/3505014043 <== mds.0 v2:172.21.15.181:6834/1569801556 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56043e5a5520 con 0x56043e7d0810 2024-09-10T23:33:35.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febd8c00640 10 client.5109 handle_command_reply: tid=0 2024-09-10T23:33:35.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 1 client.5109 shutdown 2024-09-10T23:33:35.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 1 -- 172.21.15.94:0/3505014043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7959d0 msgr2=0x56043e7cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 1 --2- 172.21.15.94:0/3505014043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7959d0 0x56043e7cfd90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7febd400fb10 tx=0x7febd4003b80 comp rx=0 tx=0).stop 2024-09-10T23:33:35.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 1 -- 172.21.15.94:0/3505014043 shutdown_connections 2024-09-10T23:33:35.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 1 --2- 172.21.15.94:0/3505014043 >> [v2:172.21.15.181:6834/1569801556,v1:172.21.15.181:6836/1569801556] conn(0x56043e7d0810 0x56043e813e20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 1 --2- 172.21.15.94:0/3505014043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56043e7962f0 0x56043e7d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 1 --2- 172.21.15.94:0/3505014043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56043e7959d0 0x56043e7cfd90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febd8c00640 0 client.5109 ms_handle_reset on v2:172.21.15.181:6834/1569801556 2024-09-10T23:33:35.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 1 --2- 172.21.15.94:0/3505014043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56043e749e10 0x56043e7aba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 1 -- 172.21.15.94:0/3505014043 >> 172.21.15.94:0/3505014043 conn(0x56043e633130 msgr2=0x56043e749470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:35.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 1 -- 172.21.15.94:0/3505014043 shutdown_connections 2024-09-10T23:33:35.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 1 -- 172.21.15.94:0/3505014043 wait complete. 2024-09-10T23:33:35.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf5b27000 20 client.5109 trim_cache size 0 max 16384 2024-09-10T23:33:35.837 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:33:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "1798cdb9-9937-4918-a569-86b41a92817e", 2024-09-10T23:33:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-09-10T23:33:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 194558, 2024-09-10T23:33:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-10T23:33:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-10T23:33:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-10T23:33:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 5.9435503289999998, 2024-09-10T23:33:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 16, 2024-09-10T23:33:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 33, 2024-09-10T23:33:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 33, 2024-09-10T23:33:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 12.823564525 2024-09-10T23:33:35.837 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:33:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf2800640 1 -- 172.21.15.94:0/1610758037 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7febc80725e0 msgr2=0x7febc8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf2800640 1 --2- 172.21.15.94:0/1610758037 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7febc80725e0 0x7febc8074aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7febdc0b0000 tx=0x7febdc01c3b0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf2800640 1 -- 172.21.15.94:0/1610758037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febec0ce280 msgr2=0x7febec0af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf2800640 1 --2- 172.21.15.94:0/1610758037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febec0ce280 0x7febec0af5b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7febe400cbf0 tx=0x7febe400cce0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf2800640 1 -- 172.21.15.94:0/1610758037 shutdown_connections 2024-09-10T23:33:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf2800640 1 --2- 172.21.15.94:0/1610758037 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7febc80725e0 0x7febc8074aa0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf2800640 1 --2- 172.21.15.94:0/1610758037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febec0ce280 0x7febec0af5b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf2800640 1 --2- 172.21.15.94:0/1610758037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febec0c64f0 0x7febec145660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf2800640 1 --2- 172.21.15.94:0/1610758037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febec06e7c0 0x7febec0c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf2800640 1 -- 172.21.15.94:0/1610758037 >> 172.21.15.94:0/1610758037 conn(0x7febec0cfaf0 msgr2=0x7febec0adbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf2800640 1 -- 172.21.15.94:0/1610758037 shutdown_connections 2024-09-10T23:33:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.828+0000 7febf2800640 1 -- 172.21.15.94:0/1610758037 wait complete. 2024-09-10T23:33:35.852 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:33:35.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.988+0000 7fb33f000640 1 -- 172.21.15.94:0/1543890578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3380d03c0 msgr2=0x7fb3380cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.988+0000 7fb33f000640 1 --2- 172.21.15.94:0/1543890578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3380d03c0 0x7fb3380cddb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb3380cd8f0 tx=0x7fb33001cc80 comp rx=0 tx=0).stop 2024-09-10T23:33:35.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb33f000640 1 -- 172.21.15.94:0/1543890578 shutdown_connections 2024-09-10T23:33:35.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb33f000640 1 --2- 172.21.15.94:0/1543890578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3380d03c0 0x7fb3380cddb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb33f000640 1 --2- 172.21.15.94:0/1543890578 >> v2:172.21.15.181:3300/0 conn(0x7fb33815ba50 0x7fb3380b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb33f000640 1 -- 172.21.15.94:0/1543890578 >> 172.21.15.94:0/1543890578 conn(0x7fb33806cdf0 msgr2=0x7fb33806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:35.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb33f000640 1 -- 172.21.15.94:0/1543890578 shutdown_connections 2024-09-10T23:33:35.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb33f000640 1 -- 172.21.15.94:0/1543890578 wait complete. 2024-09-10T23:33:35.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb33f000640 1 Processor -- start 2024-09-10T23:33:35.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb33f000640 1 -- start start 2024-09-10T23:33:35.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb33f000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3380b4c40 0x7fb3380701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb33f000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3380d03c0 0x7fb3381495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb33f000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb33815ba50 0x7fb33814c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:35.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb33f000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3381689d0 con 0x7fb3380d03c0 2024-09-10T23:33:35.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb33f000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb338168850 con 0x7fb33815ba50 2024-09-10T23:33:35.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb33f000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb338168b50 con 0x7fb3380b4c40 2024-09-10T23:33:35.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb32f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3380d03c0 0x7fb3381495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb32f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3380d03c0 0x7fb3381495c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34160/0 (socket says 172.21.15.94:34160) 2024-09-10T23:33:35.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb32f400640 1 -- 172.21.15.94:0/96833413 learned_addr learned my addr 172.21.15.94:0/96833413 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:35.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb33d000640 1 --2- 172.21.15.94:0/96833413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb33815ba50 0x7fb33814c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb337e00640 1 --2- 172.21.15.94:0/96833413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3380b4c40 0x7fb3380701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:35.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb32f400640 1 -- 172.21.15.94:0/96833413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3380b4c40 msgr2=0x7fb3380701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb32f400640 1 --2- 172.21.15.94:0/96833413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3380b4c40 0x7fb3380701b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb32f400640 1 -- 172.21.15.94:0/96833413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb33815ba50 msgr2=0x7fb33814c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:35.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb32f400640 1 --2- 172.21.15.94:0/96833413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb33815ba50 0x7fb33814c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:35.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb32f400640 1 -- 172.21.15.94:0/96833413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3381fabb0 con 0x7fb3380d03c0 2024-09-10T23:33:36.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb32f400640 1 --2- 172.21.15.94:0/96833413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3380d03c0 0x7fb3381495c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb32000c450 tx=0x7fb32000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:36.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.992+0000 7fb334c00640 1 -- 172.21.15.94:0/96833413 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb320012830 con 0x7fb3380d03c0 2024-09-10T23:33:36.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.996+0000 7fb334c00640 1 -- 172.21.15.94:0/96833413 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb320017070 con 0x7fb3380d03c0 2024-09-10T23:33:36.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.996+0000 7fb33f000640 1 -- 172.21.15.94:0/96833413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3381fae90 con 0x7fb3380d03c0 2024-09-10T23:33:36.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.996+0000 7fb334c00640 1 -- 172.21.15.94:0/96833413 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb320012d80 con 0x7fb3380d03c0 2024-09-10T23:33:36.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.996+0000 7fb33f000640 1 -- 172.21.15.94:0/96833413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3381fb450 con 0x7fb3380d03c0 2024-09-10T23:33:36.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.996+0000 7fb33f000640 1 -- 172.21.15.94:0/96833413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3380cddb0 con 0x7fb3380d03c0 2024-09-10T23:33:36.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.996+0000 7fb334c00640 1 -- 172.21.15.94:0/96833413 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb32001c460 con 0x7fb3380d03c0 2024-09-10T23:33:36.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.996+0000 7fb334c00640 1 --2- 172.21.15.94:0/96833413 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3140725e0 0x7fb314074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:36.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.996+0000 7fb334c00640 1 -- 172.21.15.94:0/96833413 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb320092e00 con 0x7fb3380d03c0 2024-09-10T23:33:36.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:35.996+0000 7fb337e00640 1 --2- 172.21.15.94:0/96833413 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3140725e0 0x7fb314074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:36.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36.000+0000 7fb337e00640 1 --2- 172.21.15.94:0/96833413 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3140725e0 0x7fb314074aa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb33815a490 tx=0x7fb32800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36.000+0000 7fb334c00640 1 -- 172.21.15.94:0/96833413 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb32005df20 con 0x7fb3380d03c0 2024-09-10T23:33:36.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36.200+0000 7fb33f000640 1 -- 172.21.15.94:0/96833413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb33814de40 con 0x7fb3380d03c0 2024-09-10T23:33:36.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36.204+0000 7fb334c00640 1 -- 172.21.15.94:0/96833413 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7fb320062950 con 0x7fb3380d03c0 2024-09-10T23:33:36.207 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:33:36.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36.204+0000 7fb33f000640 1 -- 172.21.15.94:0/96833413 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3140725e0 msgr2=0x7fb314074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:36.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36.204+0000 7fb33f000640 1 --2- 172.21.15.94:0/96833413 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3140725e0 0x7fb314074aa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb33815a490 tx=0x7fb32800b2e0 comp rx=0 tx=0).stop 2024-09-10T23:33:36.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36.204+0000 7fb33f000640 1 -- 172.21.15.94:0/96833413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3380d03c0 msgr2=0x7fb3381495c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:36.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36.204+0000 7fb33f000640 1 --2- 172.21.15.94:0/96833413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3380d03c0 0x7fb3381495c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb32000c450 tx=0x7fb32000c910 comp rx=0 tx=0).stop 2024-09-10T23:33:36.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36.204+0000 7fb33f000640 1 -- 172.21.15.94:0/96833413 shutdown_connections 2024-09-10T23:33:36.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36.204+0000 7fb33f000640 1 --2- 172.21.15.94:0/96833413 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3140725e0 0x7fb314074aa0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:36.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36.204+0000 7fb33f000640 1 --2- 172.21.15.94:0/96833413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb33815ba50 0x7fb33814c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:36.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36.204+0000 7fb33f000640 1 --2- 172.21.15.94:0/96833413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3380d03c0 0x7fb3381495c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:36.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36.204+0000 7fb33f000640 1 --2- 172.21.15.94:0/96833413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3380b4c40 0x7fb3380701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:36.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36.204+0000 7fb33f000640 1 -- 172.21.15.94:0/96833413 >> 172.21.15.94:0/96833413 conn(0x7fb33806cdf0 msgr2=0x7fb33815e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:36.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36.204+0000 7fb33f000640 1 -- 172.21.15.94:0/96833413 shutdown_connections 2024-09-10T23:33:36.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36.204+0000 7fb33f000640 1 -- 172.21.15.94:0/96833413 wait complete. 2024-09-10T23:33:36.228 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:33:36.228 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:33:36.228 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T23:33:36.228 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T23:33:36.228 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:36.228 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:33:36.228 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:33:36.228 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:33:36.228 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:33:36.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:36.229 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T23:33:36.238 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:33:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:33:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:33:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T23:33:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:36.238 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T23:33:36.239 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T23:33:36.239 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:36.239 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:33:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:33:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42395sec preferred_lft 42395sec 2024-09-10T23:33:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-09-10T23:33:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:36.239 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:33:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:ed:7d:46:db brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:33:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:36.240 INFO:teuthology.orchestra.run.smithi094.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:33:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:33:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:33:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:36.240 INFO:teuthology.orchestra.run.smithi094.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:33:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:33:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:36.240 INFO:teuthology.orchestra.run.smithi094.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:33:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:33:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:33:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:36.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:36.241 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:33:36.299 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T23:33:36.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:36.301 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T23:33:36.358 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T23:33:36.358 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:33:36.358 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:33:36.358 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:33:36.358 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:33:36.358 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:33:36.358 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:33:36.358 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:36.358 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:36.358 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:36.358 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:33:36.359 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:36.359 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:33:36.359 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:33:36.359 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:33:36.359 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:33:36.359 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:33:36.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:36.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:36.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:36.490 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T23:33:36.499 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T23:33:36.499 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T23:33:36.499 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:33:36.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:36.500 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-09-10T23:33:36.566 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T23:33:36.566 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:33:36.566 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:36.618 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:33:36.618 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:33:36.618 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:36.669 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:33:36.730 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:33:36.787 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:33:36.787 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:33:36.787 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:33:36.787 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:33:36.787 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:33:36.788 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:33:36.788 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:33:36.788 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:33:36.788 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:33:36.788 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:33:36.788 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:33:36.788 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:33:36.788 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:33:36.788 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:33:36.788 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:33:36.788 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:33:36.788 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:33:36.788 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:33:36.788 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:33:36.789 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:33:36.789 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:33:36.789 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:33:36.789 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:33:36.789 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:33:36.789 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:33:36.789 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:33:36.789 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:33:36.789 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:33:36.789 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:33:36.789 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:36.789 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:36.789 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:33:36.790 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:33:36.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:36.791 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:33:36.845 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:33:36.846 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T23:33:36.888 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:33:36.910 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:33:36.963 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:33:36.963 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:33:36.963 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:33:36.963 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:33:36.964 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:33:36.964 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:33:36.964 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:33:36.964 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:33:36.964 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:33:36.964 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:33:36.964 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:33:36.964 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:33:36.964 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:33:36.964 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:33:36.964 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:33:36.964 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:33:36.964 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:33:36.964 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:33:36.965 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:33:36.965 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:33:36.965 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:33:36.965 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:33:36.965 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:33:36.965 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:33:36.965 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:33:36.965 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:33:36.965 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:33:36.965 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:33:36.965 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:33:36.965 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:36.965 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:33:36.966 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:33:36.966 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:33:36.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:36.967 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:33:36.982 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:2024-09-10T23:33:36.976+0000 7fb3e7803580 -1 init, newargv = 0x55a3518d0da0 newargc=15 2024-09-10T23:33:36.982 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[37154]: starting ceph client 2024-09-10T23:33:36.989 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[37154]: starting fuse 2024-09-10T23:33:36.990 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T23:33:36.990 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:33:36.991 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:33:36.991 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-09-10T23:33:36.991 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-09-10T23:33:36.991 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-09-10T23:33:36.991 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-09-10T23:33:36.991 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:33:36.991 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-09-10T23:33:36.991 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T23:33:36.991 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-09-10T23:33:36.991 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T23:33:36.991 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:33:36.991 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-09-10T23:33:36.991 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-09-10T23:33:36.991 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-09-10T23:33:36.992 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-09-10T23:33:36.992 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:33:36.992 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-09-10T23:33:36.992 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:33:36.992 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:33:36.992 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:33:36.992 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-09-10T23:33:36.992 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-09-10T23:33:36.992 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-09-10T23:33:36.992 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:33:36.992 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:33:36.992 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.0")) 2024-09-10T23:33:36.992 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-09-10T23:33:37.113 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.0.37154.asok 2024-09-10T23:33:37.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:37.125 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.37154.asok 2024-09-10T23:33:37.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:37.125 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.37154.asok status 2024-09-10T23:33:37.292 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:33:37.292 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-09-10T23:33:37.292 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:33:37.292 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:33:37.292 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "0", 2024-09-10T23:33:37.292 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-09-10T23:33:37.292 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T23:33:37.292 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "37154", 2024-09-10T23:33:37.292 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-09-10T23:33:37.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:37.293 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-09-10T23:33:37.293 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-09-10T23:33:37.293 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 5124, 2024-09-10T23:33:37.293 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-09-10T23:33:37.293 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-09-10T23:33:37.293 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-09-10T23:33:37.293 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 5124 2024-09-10T23:33:37.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:37.293 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:33:37.293 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:33:37.293 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:33:37.293 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 159248905 2024-09-10T23:33:37.293 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:37.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:37.294 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:33:37.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:33:37.294 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:33:37.294 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 159248905 2024-09-10T23:33:37.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:37.294 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.5124 192.168.0.1:0/159248905", 2024-09-10T23:33:37.294 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/159248905", 2024-09-10T23:33:37.294 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-09-10T23:33:37.294 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 16, 2024-09-10T23:33:37.294 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 33, 2024-09-10T23:33:37.294 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:33:37.294 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-09-10T23:33:37.294 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-09-10T23:33:37.294 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:33:37.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:37.307 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-09-10T23:33:37.307 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:37.315 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-09-10T23:33:37.316 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:37.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:37.316 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:37.378 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:33:37.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8071600640 1 -- 172.21.15.94:0/3153736253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8050004d50 msgr2=0x7f80500051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:37.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8071600640 1 --2- 172.21.15.94:0/3153736253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8050004d50 0x7f80500051b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f806c0d0fa0 tx=0x7f806401fa00 comp rx=0 tx=0).stop 2024-09-10T23:33:37.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f806b400640 1 -- 172.21.15.94:0/3153736253 reap_dead start 2024-09-10T23:33:37.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8071600640 1 -- 172.21.15.94:0/3153736253 shutdown_connections 2024-09-10T23:33:37.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8071600640 1 -- 172.21.15.94:0/3153736253 >> 172.21.15.94:0/3153736253 conn(0x7f806c0d02d0 msgr2=0x7f806c0d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:37.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8071600640 1 -- 172.21.15.94:0/3153736253 shutdown_connections 2024-09-10T23:33:37.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8071600640 1 -- 172.21.15.94:0/3153736253 wait complete. 2024-09-10T23:33:37.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8071600640 1 Processor -- start 2024-09-10T23:33:37.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8071600640 1 -- start start 2024-09-10T23:33:37.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8071600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8050004d50 0x7f806c0b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:37.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8071600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0c5e90 0x7f806c0c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:37.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8071600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f806c15b490 0x7f806c0c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:37.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8071600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f806c06d8b0 con 0x7f806c0c5e90 2024-09-10T23:33:37.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8071600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f806c0cbbd0 con 0x7f8050004d50 2024-09-10T23:33:37.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8071600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f806c06da30 con 0x7f806c15b490 2024-09-10T23:33:37.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8062000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0c5e90 0x7f806c0c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:37.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8062000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0c5e90 0x7f806c0c7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34176/0 (socket says 172.21.15.94:34176) 2024-09-10T23:33:37.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f8062000640 1 -- 172.21.15.94:0/1071504059 learned_addr learned my addr 172.21.15.94:0/1071504059 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:37.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f806b400640 1 --2- 172.21.15.94:0/1071504059 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f806c15b490 0x7f806c0c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:37.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.564+0000 7f806aa00640 1 --2- 172.21.15.94:0/1071504059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8050004d50 0x7f806c0b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:37.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.568+0000 7f8062000640 1 -- 172.21.15.94:0/1071504059 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f806c15b490 msgr2=0x7f806c0c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:37.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.568+0000 7f8062000640 1 --2- 172.21.15.94:0/1071504059 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f806c15b490 0x7f806c0c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:37.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.568+0000 7f8062000640 1 -- 172.21.15.94:0/1071504059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8050004d50 msgr2=0x7f806c0b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:37.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.568+0000 7f8062000640 1 --2- 172.21.15.94:0/1071504059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8050004d50 0x7f806c0b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:37.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.568+0000 7f8062000640 1 -- 172.21.15.94:0/1071504059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806c0b54b0 con 0x7f806c0c5e90 2024-09-10T23:33:37.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.568+0000 7f806aa00640 1 --2- 172.21.15.94:0/1071504059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8050004d50 0x7f806c0b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:37.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.568+0000 7f8062000640 1 --2- 172.21.15.94:0/1071504059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0c5e90 0x7f806c0c7aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8054011b10 tx=0x7f8054009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:37.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.568+0000 7f8063400640 1 -- 172.21.15.94:0/1071504059 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805401e070 con 0x7f806c0c5e90 2024-09-10T23:33:37.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.568+0000 7f8063400640 1 -- 172.21.15.94:0/1071504059 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8054004460 con 0x7f806c0c5e90 2024-09-10T23:33:37.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.568+0000 7f8063400640 1 -- 172.21.15.94:0/1071504059 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805400fbb0 con 0x7f806c0c5e90 2024-09-10T23:33:37.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.568+0000 7f8071600640 1 -- 172.21.15.94:0/1071504059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806c0b4410 con 0x7f806c0c5e90 2024-09-10T23:33:37.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.568+0000 7f8071600640 1 -- 172.21.15.94:0/1071504059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f806c0b49d0 con 0x7f806c0c5e90 2024-09-10T23:33:37.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.568+0000 7f8071600640 1 -- 172.21.15.94:0/1071504059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f806c0ce830 con 0x7f806c0c5e90 2024-09-10T23:33:37.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.568+0000 7f8063400640 1 -- 172.21.15.94:0/1071504059 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8054003600 con 0x7f806c0c5e90 2024-09-10T23:33:37.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.568+0000 7f8063400640 1 --2- 172.21.15.94:0/1071504059 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8048072610 0x7f8048074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:37.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.568+0000 7f8063400640 1 -- 172.21.15.94:0/1071504059 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8054093db0 con 0x7f806c0c5e90 2024-09-10T23:33:37.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.572+0000 7f806aa00640 1 --2- 172.21.15.94:0/1071504059 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8048072610 0x7f8048074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:37.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.572+0000 7f806aa00640 1 --2- 172.21.15.94:0/1071504059 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8048072610 0x7f8048074ad0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f805c0066a0 tx=0x7f805c006c90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:37.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.572+0000 7f8063400640 1 -- 172.21.15.94:0/1071504059 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8054098070 con 0x7f806c0c5e90 2024-09-10T23:33:37.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.772+0000 7f8071600640 1 -- 172.21.15.94:0/1071504059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f806c0adef0 con 0x7f806c0c5e90 2024-09-10T23:33:37.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.772+0000 7f8063400640 1 -- 172.21.15.94:0/1071504059 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f805405ee30 con 0x7f806c0c5e90 2024-09-10T23:33:37.778 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:33:37.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.776+0000 7f8071600640 1 -- 172.21.15.94:0/1071504059 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8048072610 msgr2=0x7f8048074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:37.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.776+0000 7f8071600640 1 --2- 172.21.15.94:0/1071504059 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8048072610 0x7f8048074ad0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f805c0066a0 tx=0x7f805c006c90 comp rx=0 tx=0).stop 2024-09-10T23:33:37.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.776+0000 7f8071600640 1 -- 172.21.15.94:0/1071504059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0c5e90 msgr2=0x7f806c0c7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:37.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.776+0000 7f8071600640 1 --2- 172.21.15.94:0/1071504059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0c5e90 0x7f806c0c7aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8054011b10 tx=0x7f8054009e10 comp rx=0 tx=0).stop 2024-09-10T23:33:37.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.776+0000 7f8071600640 1 -- 172.21.15.94:0/1071504059 shutdown_connections 2024-09-10T23:33:37.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.776+0000 7f8071600640 1 --2- 172.21.15.94:0/1071504059 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8048072610 0x7f8048074ad0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:37.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.776+0000 7f8071600640 1 --2- 172.21.15.94:0/1071504059 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f806c15b490 0x7f806c0c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:37.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.776+0000 7f8071600640 1 --2- 172.21.15.94:0/1071504059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f806c0c5e90 0x7f806c0c7aa0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:37.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.776+0000 7f8071600640 1 --2- 172.21.15.94:0/1071504059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8050004d50 0x7f806c0b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:37.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.776+0000 7f8071600640 1 -- 172.21.15.94:0/1071504059 >> 172.21.15.94:0/1071504059 conn(0x7f806c0d02d0 msgr2=0x7f806c0ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:37.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.776+0000 7f8071600640 1 -- 172.21.15.94:0/1071504059 shutdown_connections 2024-09-10T23:33:37.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:37.776+0000 7f8071600640 1 -- 172.21.15.94:0/1071504059 wait complete. 2024-09-10T23:33:37.800 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:33:37.800 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:33:37.800 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-09-10T23:33:37.800 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T23:33:37.800 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:37.801 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:33:37.801 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:33:37.801 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:33:37.801 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:33:37.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:37.802 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-09-10T23:33:37.812 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:33:37.812 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:33:37.812 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:33:37.813 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:37.813 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-09-10T23:33:37.813 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:37.813 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:37.813 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:37.813 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-09-10T23:33:37.814 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:37.814 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:37.814 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-09-10T23:33:37.814 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:33:37.814 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:37.814 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:33:37.815 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:37.815 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:33:37.815 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42398sec preferred_lft 42398sec 2024-09-10T23:33:37.815 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-09-10T23:33:37.815 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:37.815 INFO:teuthology.orchestra.run.smithi181.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:33:37.816 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 02:42:3b:2e:09:2d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:37.816 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:33:37.816 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:37.816 INFO:teuthology.orchestra.run.smithi181.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:33:37.816 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:33:37.816 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:33:37.817 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:37.817 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:33:37.817 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:37.817 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:33:37.817 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:33:37.817 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:33:37.818 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:37.818 INFO:teuthology.orchestra.run.smithi181.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:33:37.818 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:33:37.818 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:33:37.818 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:33:37.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:37.820 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:33:37.881 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-09-10T23:33:37.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:37.884 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-09-10T23:33:37.942 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-09-10T23:33:37.943 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:33:37.943 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:33:37.943 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:33:37.943 INFO:teuthology.orchestra.run.smithi181.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:33:37.943 INFO:teuthology.orchestra.run.smithi181.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:33:37.943 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:33:37.943 INFO:teuthology.orchestra.run.smithi181.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:37.943 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:37.944 INFO:teuthology.orchestra.run.smithi181.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:33:37.944 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:33:37.944 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:33:37.944 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:33:37.944 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:33:37.944 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:33:37.945 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:33:37.945 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:33:38.036 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:33:38.078 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:38.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:38.085 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-09-10T23:33:38.093 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T23:33:38.093 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T23:33:38.094 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:33:38.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:38.094 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-09-10T23:33:38.161 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T23:33:38.161 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:33:38.161 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:38.209 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:33:38.210 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:33:38.210 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:38.264 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:33:38.329 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:33:38.383 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:33:38.383 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:33:38.383 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:33:38.383 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:33:38.383 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:33:38.383 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:33:38.383 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:33:38.383 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:33:38.383 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:33:38.384 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:33:38.384 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:33:38.384 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:33:38.384 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:33:38.384 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:33:38.384 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:33:38.384 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:33:38.384 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:33:38.384 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:33:38.384 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:33:38.384 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:33:38.384 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:33:38.384 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:33:38.385 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:33:38.385 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:33:38.385 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:33:38.385 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:33:38.385 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:33:38.385 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:33:38.385 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:33:38.385 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:33:38.385 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:33:38.385 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:33:38.385 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:33:38.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:38.386 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:33:38.442 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:33:38.442 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T23:33:38.488 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:33:38.508 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:33:38.559 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:33:38.559 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:33:38.559 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:33:38.559 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:33:38.559 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:33:38.559 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:33:38.559 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:33:38.559 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:33:38.559 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:33:38.560 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:33:38.560 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:33:38.560 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:33:38.560 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:33:38.560 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:33:38.560 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:33:38.560 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:33:38.560 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:33:38.560 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:33:38.560 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:33:38.560 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:33:38.560 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:33:38.560 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:33:38.561 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:33:38.561 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:33:38.561 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:33:38.561 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:33:38.561 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:33:38.561 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:33:38.561 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:33:38.561 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:33:38.561 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:33:38.561 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:33:38.561 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:33:38.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:38.562 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:33:38.580 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:2024-09-10T23:33:38.577+0000 7f0d9097c580 -1 init, newargv = 0x561009ad8ec0 newargc=15 2024-09-10T23:33:38.581 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[34425]: starting ceph client 2024-09-10T23:33:38.587 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[34425]: starting fuse 2024-09-10T23:33:38.589 INFO:teuthology.orchestra.run.smithi181.stdout:44 2024-09-10T23:33:38.589 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:33:38.589 DEBUG:teuthology.orchestra.run.smithi181:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:33:38.589 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2024-09-10T23:33:38.589 DEBUG:teuthology.orchestra.run.smithi181:> import re 2024-09-10T23:33:38.589 DEBUG:teuthology.orchestra.run.smithi181:> import os 2024-09-10T23:33:38.589 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2024-09-10T23:33:38.590 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:33:38.590 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2024-09-10T23:33:38.590 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T23:33:38.590 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2024-09-10T23:33:38.590 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T23:33:38.590 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:33:38.590 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2024-09-10T23:33:38.590 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2024-09-10T23:33:38.590 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2024-09-10T23:33:38.590 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2024-09-10T23:33:38.590 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:33:38.590 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2024-09-10T23:33:38.590 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:33:38.590 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:33:38.590 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:33:38.590 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2024-09-10T23:33:38.591 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2024-09-10T23:33:38.591 DEBUG:teuthology.orchestra.run.smithi181:> return f 2024-09-10T23:33:38.591 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:33:38.591 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:33:38.591 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.1")) 2024-09-10T23:33:38.591 DEBUG:teuthology.orchestra.run.smithi181:> ' 2024-09-10T23:33:38.712 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.1.34425.asok 2024-09-10T23:33:38.716 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:33:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:38.724 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34425.asok 2024-09-10T23:33:38.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:38.724 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34425.asok status 2024-09-10T23:33:38.893 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T23:33:38.893 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2024-09-10T23:33:38.893 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:33:38.893 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:33:38.893 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "1", 2024-09-10T23:33:38.893 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2024-09-10T23:33:38.893 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T23:33:38.893 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "34425", 2024-09-10T23:33:38.893 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2024-09-10T23:33:38.893 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:33:38.893 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2024-09-10T23:33:38.894 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2024-09-10T23:33:38.894 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 194684, 2024-09-10T23:33:38.894 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2024-09-10T23:33:38.894 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2024-09-10T23:33:38.894 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2024-09-10T23:33:38.894 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 194684 2024-09-10T23:33:38.894 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:33:38.894 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:33:38.894 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:33:38.894 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:33:38.894 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 3450675583 2024-09-10T23:33:38.894 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T23:33:38.895 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:33:38.895 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:33:38.895 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:33:38.895 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:33:38.895 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 3450675583 2024-09-10T23:33:38.895 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:33:38.895 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.194684 192.168.0.1:0/3450675583", 2024-09-10T23:33:38.895 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.1:0/3450675583", 2024-09-10T23:33:38.895 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2024-09-10T23:33:38.895 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 16, 2024-09-10T23:33:38.895 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 33, 2024-09-10T23:33:38.895 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:33:38.895 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2024-09-10T23:33:38.896 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2024-09-10T23:33:38.896 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T23:33:38.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:38.907 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2024-09-10T23:33:38.907 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:38.916 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2024-09-10T23:33:38.916 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:38.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:38.917 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:38.976 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T23:33:39.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.120+0000 7fc6a2800640 1 -- 172.21.15.94:0/276063834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc69c0b0320 msgr2=0x7fc69c166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:39.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.120+0000 7fc6a2800640 1 --2- 172.21.15.94:0/276063834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc69c0b0320 0x7fc69c166aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc694009870 tx=0x7fc69401c740 comp rx=0 tx=0).stop 2024-09-10T23:33:39.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.120+0000 7fc6a2800640 1 -- 172.21.15.94:0/276063834 shutdown_connections 2024-09-10T23:33:39.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.120+0000 7fc6a2800640 1 --2- 172.21.15.94:0/276063834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc69c0b0320 0x7fc69c166aa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:39.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.120+0000 7fc6a2800640 1 -- 172.21.15.94:0/276063834 >> 172.21.15.94:0/276063834 conn(0x7fc69c0d1c60 msgr2=0x7fc69c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:39.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.120+0000 7fc6a2800640 1 -- 172.21.15.94:0/276063834 shutdown_connections 2024-09-10T23:33:39.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.120+0000 7fc6a2800640 1 -- 172.21.15.94:0/276063834 wait complete. 2024-09-10T23:33:39.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc6a2800640 1 Processor -- start 2024-09-10T23:33:39.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc6a2800640 1 -- start start 2024-09-10T23:33:39.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc6a2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc69c0b0320 0x7fc69c150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:39.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc6a2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc69c0b2bb0 0x7fc69c150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:39.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc6a2800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc69c15f5c0 0x7fc69c14c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:39.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc6a2800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c169eb0 con 0x7fc69c0b2bb0 2024-09-10T23:33:39.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc6a2800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c169d30 con 0x7fc69c0b0320 2024-09-10T23:33:39.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc6a2800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c16a030 con 0x7fc69c15f5c0 2024-09-10T23:33:39.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc692a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc69c0b2bb0 0x7fc69c150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:39.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc692a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc69c0b2bb0 0x7fc69c150790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34180/0 (socket says 172.21.15.94:34180) 2024-09-10T23:33:39.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc692a00640 1 -- 172.21.15.94:0/3607585588 learned_addr learned my addr 172.21.15.94:0/3607585588 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:39.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc69be00640 1 --2- 172.21.15.94:0/3607585588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc69c15f5c0 0x7fc69c14c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:39.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc69b400640 1 --2- 172.21.15.94:0/3607585588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc69c0b0320 0x7fc69c150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:39.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc69be00640 1 -- 172.21.15.94:0/3607585588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc69c0b0320 msgr2=0x7fc69c150250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:39.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc69be00640 1 --2- 172.21.15.94:0/3607585588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc69c0b0320 0x7fc69c150250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:39.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc69be00640 1 -- 172.21.15.94:0/3607585588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc69c0b2bb0 msgr2=0x7fc69c150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:39.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc69be00640 1 --2- 172.21.15.94:0/3607585588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc69c0b2bb0 0x7fc69c150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:39.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc69be00640 1 -- 172.21.15.94:0/3607585588 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c0b6ac0 con 0x7fc69c15f5c0 2024-09-10T23:33:39.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc69be00640 1 --2- 172.21.15.94:0/3607585588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc69c15f5c0 0x7fc69c14c990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc6940ac000 tx=0x7fc694005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:39.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc693e00640 1 -- 172.21.15.94:0/3607585588 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6940be020 con 0x7fc69c15f5c0 2024-09-10T23:33:39.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc693e00640 1 -- 172.21.15.94:0/3607585588 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc694004130 con 0x7fc69c15f5c0 2024-09-10T23:33:39.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc693e00640 1 -- 172.21.15.94:0/3607585588 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc694003600 con 0x7fc69c15f5c0 2024-09-10T23:33:39.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc6a2800640 1 -- 172.21.15.94:0/3607585588 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69c1f7440 con 0x7fc69c15f5c0 2024-09-10T23:33:39.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.124+0000 7fc6a2800640 1 -- 172.21.15.94:0/3607585588 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69c0b6d40 con 0x7fc69c15f5c0 2024-09-10T23:33:39.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.128+0000 7fc6a2800640 1 -- 172.21.15.94:0/3607585588 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc69c0cfb60 con 0x7fc69c15f5c0 2024-09-10T23:33:39.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.128+0000 7fc693e00640 1 -- 172.21.15.94:0/3607585588 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc6940037a0 con 0x7fc69c15f5c0 2024-09-10T23:33:39.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.128+0000 7fc693e00640 1 --2- 172.21.15.94:0/3607585588 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc6780725e0 0x7fc678074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:39.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.128+0000 7fc693e00640 1 -- 172.21.15.94:0/3607585588 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc69412e510 con 0x7fc69c15f5c0 2024-09-10T23:33:39.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.128+0000 7fc69b400640 1 --2- 172.21.15.94:0/3607585588 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc6780725e0 0x7fc678074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:39.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.128+0000 7fc69b400640 1 --2- 172.21.15.94:0/3607585588 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc6780725e0 0x7fc678074aa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0017b0 tx=0x7fc68c001740 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:39.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.132+0000 7fc693e00640 1 -- 172.21.15.94:0/3607585588 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc6940c8030 con 0x7fc69c15f5c0 2024-09-10T23:33:39.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.332+0000 7fc6a2800640 1 -- 172.21.15.94:0/3607585588 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fc69c0749c0 con 0x7fc69c15f5c0 2024-09-10T23:33:39.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.932+0000 7fc693e00640 1 -- 172.21.15.94:0/3607585588 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v17) ==== 136+0+0 (secure 0 0 0) 0x7fc69c0b6ac0 con 0x7fc69c15f5c0 2024-09-10T23:33:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.932+0000 7fc6a2800640 1 -- 172.21.15.94:0/3607585588 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc6780725e0 msgr2=0x7fc678074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.932+0000 7fc6a2800640 1 --2- 172.21.15.94:0/3607585588 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc6780725e0 0x7fc678074aa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0017b0 tx=0x7fc68c001740 comp rx=0 tx=0).stop 2024-09-10T23:33:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.932+0000 7fc6a2800640 1 -- 172.21.15.94:0/3607585588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc69c15f5c0 msgr2=0x7fc69c14c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.932+0000 7fc6a2800640 1 --2- 172.21.15.94:0/3607585588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc69c15f5c0 0x7fc69c14c990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc6940ac000 tx=0x7fc694005770 comp rx=0 tx=0).stop 2024-09-10T23:33:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.932+0000 7fc6a2800640 1 -- 172.21.15.94:0/3607585588 shutdown_connections 2024-09-10T23:33:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.932+0000 7fc6a2800640 1 --2- 172.21.15.94:0/3607585588 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc6780725e0 0x7fc678074aa0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.932+0000 7fc6a2800640 1 --2- 172.21.15.94:0/3607585588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc69c15f5c0 0x7fc69c14c990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.932+0000 7fc6a2800640 1 --2- 172.21.15.94:0/3607585588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc69c0b2bb0 0x7fc69c150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.932+0000 7fc6a2800640 1 --2- 172.21.15.94:0/3607585588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc69c0b0320 0x7fc69c150250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.932+0000 7fc6a2800640 1 -- 172.21.15.94:0/3607585588 >> 172.21.15.94:0/3607585588 conn(0x7fc69c0d1c60 msgr2=0x7fc69c070780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:39.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.932+0000 7fc6a2800640 1 -- 172.21.15.94:0/3607585588 shutdown_connections 2024-09-10T23:33:39.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:39.932+0000 7fc6a2800640 1 -- 172.21.15.94:0/3607585588 wait complete. 2024-09-10T23:33:39.959 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T23:33:40.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.096+0000 7f9d69a00640 1 -- 172.21.15.94:0/699719011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d640c64f0 msgr2=0x7f9d640ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:40.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.096+0000 7f9d69a00640 1 --2- 172.21.15.94:0/699719011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d640c64f0 0x7f9d640ce660 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9d64069670 tx=0x7f9d4c01c7e0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.096+0000 7f9d69a00640 1 -- 172.21.15.94:0/699719011 shutdown_connections 2024-09-10T23:33:40.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.096+0000 7f9d69a00640 1 --2- 172.21.15.94:0/699719011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d640c64f0 0x7f9d640ce660 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.096+0000 7f9d69a00640 1 --2- 172.21.15.94:0/699719011 >> v2:172.21.15.181:3300/0 conn(0x7f9d640ce280 0x7f9d640cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.096+0000 7f9d69a00640 1 -- 172.21.15.94:0/699719011 >> 172.21.15.94:0/699719011 conn(0x7f9d640cfaf0 msgr2=0x7f9d640cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:40.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.096+0000 7f9d69a00640 1 -- 172.21.15.94:0/699719011 shutdown_connections 2024-09-10T23:33:40.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.096+0000 7f9d69a00640 1 -- 172.21.15.94:0/699719011 wait complete. 2024-09-10T23:33:40.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d69a00640 1 Processor -- start 2024-09-10T23:33:40.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d69a00640 1 -- start start 2024-09-10T23:33:40.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d69a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d6406e7c0 0x7f9d641f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:40.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d69a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d640c64f0 0x7f9d641524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:40.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d69a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d640ce280 0x7f9d640c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:40.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d69a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64165670 con 0x7f9d640c64f0 2024-09-10T23:33:40.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d69a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641654f0 con 0x7f9d640ce280 2024-09-10T23:33:40.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d69a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9d641657f0 con 0x7f9d6406e7c0 2024-09-10T23:33:40.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d5be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d640c64f0 0x7f9d641524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:40.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d5be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d640c64f0 0x7f9d641524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34186/0 (socket says 172.21.15.94:34186) 2024-09-10T23:33:40.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d5be00640 1 -- 172.21.15.94:0/374028281 learned_addr learned my addr 172.21.15.94:0/374028281 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:40.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d62a00640 1 --2- 172.21.15.94:0/374028281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d6406e7c0 0x7f9d641f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:40.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d63400640 1 --2- 172.21.15.94:0/374028281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d640ce280 0x7f9d640c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:40.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d5be00640 1 -- 172.21.15.94:0/374028281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d6406e7c0 msgr2=0x7f9d641f7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:40.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d5be00640 1 --2- 172.21.15.94:0/374028281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d6406e7c0 0x7f9d641f7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d5be00640 1 -- 172.21.15.94:0/374028281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d640ce280 msgr2=0x7f9d640c58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:40.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d5be00640 1 --2- 172.21.15.94:0/374028281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d640ce280 0x7f9d640c58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d5be00640 1 -- 172.21.15.94:0/374028281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d640adac0 con 0x7f9d640c64f0 2024-09-10T23:33:40.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d63400640 1 --2- 172.21.15.94:0/374028281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d640ce280 0x7f9d640c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:40.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d5be00640 1 --2- 172.21.15.94:0/374028281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d640c64f0 0x7f9d641524f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0193f0 tx=0x7f9d4c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:40.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d5aa00640 1 -- 172.21.15.94:0/374028281 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0bc070 con 0x7f9d640c64f0 2024-09-10T23:33:40.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d5aa00640 1 -- 172.21.15.94:0/374028281 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d4c003600 con 0x7f9d640c64f0 2024-09-10T23:33:40.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d69a00640 1 -- 172.21.15.94:0/374028281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d640adda0 con 0x7f9d640c64f0 2024-09-10T23:33:40.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d5aa00640 1 -- 172.21.15.94:0/374028281 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0b2040 con 0x7f9d640c64f0 2024-09-10T23:33:40.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.100+0000 7f9d69a00640 1 -- 172.21.15.94:0/374028281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d640ae880 con 0x7f9d640c64f0 2024-09-10T23:33:40.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.104+0000 7f9d69a00640 1 -- 172.21.15.94:0/374028281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d640ce660 con 0x7f9d640c64f0 2024-09-10T23:33:40.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.104+0000 7f9d5aa00640 1 -- 172.21.15.94:0/374028281 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9d4c0b5070 con 0x7f9d640c64f0 2024-09-10T23:33:40.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.104+0000 7f9d5aa00640 1 --2- 172.21.15.94:0/374028281 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9d400725e0 0x7f9d40074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:40.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.104+0000 7f9d5aa00640 1 -- 172.21.15.94:0/374028281 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9d4c12ca20 con 0x7f9d640c64f0 2024-09-10T23:33:40.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.104+0000 7f9d62a00640 1 --2- 172.21.15.94:0/374028281 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9d400725e0 0x7f9d40074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:40.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.108+0000 7f9d62a00640 1 --2- 172.21.15.94:0/374028281 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9d400725e0 0x7f9d40074aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9d540058d0 tx=0x7f9d54005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:40.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.108+0000 7f9d5aa00640 1 -- 172.21.15.94:0/374028281 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9d4c0163d0 con 0x7f9d640c64f0 2024-09-10T23:33:40.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.296+0000 7f9d69a00640 1 -- 172.21.15.94:0/374028281 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f9d64069e20 con 0x7f9d400725e0 2024-09-10T23:33:40.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.296+0000 7f9d5aa00640 1 -- 172.21.15.94:0/374028281 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f9d64069e20 con 0x7f9d400725e0 2024-09-10T23:33:40.301 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:33:40.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:40.302 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephfs" 2024-09-10T23:33:40.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:40.302 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:33:40.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.300+0000 7f9d69a00640 1 -- 172.21.15.94:0/374028281 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9d400725e0 msgr2=0x7f9d40074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:40.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.300+0000 7f9d69a00640 1 --2- 172.21.15.94:0/374028281 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9d400725e0 0x7f9d40074aa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9d540058d0 tx=0x7f9d54005860 comp rx=0 tx=0).stop 2024-09-10T23:33:40.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.300+0000 7f9d69a00640 1 -- 172.21.15.94:0/374028281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d640c64f0 msgr2=0x7f9d641524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:40.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.300+0000 7f9d69a00640 1 --2- 172.21.15.94:0/374028281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d640c64f0 0x7f9d641524f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0193f0 tx=0x7f9d4c004040 comp rx=0 tx=0).stop 2024-09-10T23:33:40.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.300+0000 7f9d69a00640 1 -- 172.21.15.94:0/374028281 shutdown_connections 2024-09-10T23:33:40.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.300+0000 7f9d69a00640 1 --2- 172.21.15.94:0/374028281 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9d400725e0 0x7f9d40074aa0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.300+0000 7f9d69a00640 1 --2- 172.21.15.94:0/374028281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d640ce280 0x7f9d640c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.300+0000 7f9d69a00640 1 --2- 172.21.15.94:0/374028281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d640c64f0 0x7f9d641524f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.300+0000 7f9d69a00640 1 --2- 172.21.15.94:0/374028281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d6406e7c0 0x7f9d641f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.300+0000 7f9d69a00640 1 -- 172.21.15.94:0/374028281 >> 172.21.15.94:0/374028281 conn(0x7f9d640cfaf0 msgr2=0x7f9d640c49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:40.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.300+0000 7f9d69a00640 1 -- 172.21.15.94:0/374028281 shutdown_connections 2024-09-10T23:33:40.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.300+0000 7f9d69a00640 1 -- 172.21.15.94:0/374028281 wait complete. 2024-09-10T23:33:40.324 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:33:40.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.464+0000 7fa6d5e00640 1 -- 172.21.15.94:0/999430450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d0073920 msgr2=0x7fa6d0073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:40.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.464+0000 7fa6d5e00640 1 --2- 172.21.15.94:0/999430450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d0073920 0x7fa6d0073d90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009840 tx=0x7fa6c801c7f0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.464+0000 7fa6d5e00640 1 -- 172.21.15.94:0/999430450 shutdown_connections 2024-09-10T23:33:40.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.464+0000 7fa6d5e00640 1 --2- 172.21.15.94:0/999430450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d0073920 0x7fa6d0073d90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.464+0000 7fa6d5e00640 1 -- 172.21.15.94:0/999430450 >> 172.21.15.94:0/999430450 conn(0x7fa6d00d0d80 msgr2=0x7fa6d00d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:40.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.464+0000 7fa6d5e00640 1 -- 172.21.15.94:0/999430450 shutdown_connections 2024-09-10T23:33:40.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.464+0000 7fa6d5e00640 1 -- 172.21.15.94:0/999430450 wait complete. 2024-09-10T23:33:40.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.464+0000 7fa6d5e00640 1 Processor -- start 2024-09-10T23:33:40.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.464+0000 7fa6d5e00640 1 -- start start 2024-09-10T23:33:40.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.464+0000 7fa6d5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d006ce80 0x7fa6d014ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:40.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.464+0000 7fa6d5e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6d006d260 0x7fa6d013e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:40.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.464+0000 7fa6d5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d0073920 0x7fa6d01551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:40.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.464+0000 7fa6d5e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d0169f60 con 0x7fa6d0073920 2024-09-10T23:33:40.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.464+0000 7fa6d5e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d0169de0 con 0x7fa6d006ce80 2024-09-10T23:33:40.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.464+0000 7fa6d5e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa6d016a0e0 con 0x7fa6d006d260 2024-09-10T23:33:40.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6cf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d0073920 0x7fa6d01551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:40.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6cf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d0073920 0x7fa6d01551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34212/0 (socket says 172.21.15.94:34212) 2024-09-10T23:33:40.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6cf400640 1 -- 172.21.15.94:0/3591809007 learned_addr learned my addr 172.21.15.94:0/3591809007 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:40.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6cea00640 1 --2- 172.21.15.94:0/3591809007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d006ce80 0x7fa6d014ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:40.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6ce000640 1 --2- 172.21.15.94:0/3591809007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6d006d260 0x7fa6d013e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:40.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6cf400640 1 -- 172.21.15.94:0/3591809007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6d006d260 msgr2=0x7fa6d013e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:40.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6cf400640 1 --2- 172.21.15.94:0/3591809007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6d006d260 0x7fa6d013e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6cf400640 1 -- 172.21.15.94:0/3591809007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d006ce80 msgr2=0x7fa6d014ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:40.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6cf400640 1 --2- 172.21.15.94:0/3591809007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d006ce80 0x7fa6d014ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6cf400640 1 -- 172.21.15.94:0/3591809007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d013d330 con 0x7fa6d0073920 2024-09-10T23:33:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6ce000640 1 --2- 172.21.15.94:0/3591809007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6d006d260 0x7fa6d013e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6cf400640 1 --2- 172.21.15.94:0/3591809007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d0073920 0x7fa6d01551e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa6c801a420 tx=0x7fa6c80a56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6c7400640 1 -- 172.21.15.94:0/3591809007 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c80bc070 con 0x7fa6d0073920 2024-09-10T23:33:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6c7400640 1 -- 172.21.15.94:0/3591809007 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6c8004430 con 0x7fa6d0073920 2024-09-10T23:33:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6c7400640 1 -- 172.21.15.94:0/3591809007 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c80b2040 con 0x7fa6d0073920 2024-09-10T23:33:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6d5e00640 1 -- 172.21.15.94:0/3591809007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d013d610 con 0x7fa6d0073920 2024-09-10T23:33:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6d5e00640 1 -- 172.21.15.94:0/3591809007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d013dbd0 con 0x7fa6d0073920 2024-09-10T23:33:40.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6d5e00640 1 -- 172.21.15.94:0/3591809007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d0073d90 con 0x7fa6d0073920 2024-09-10T23:33:40.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6c7400640 1 -- 172.21.15.94:0/3591809007 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa6c80b5070 con 0x7fa6d0073920 2024-09-10T23:33:40.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.468+0000 7fa6c7400640 1 --2- 172.21.15.94:0/3591809007 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa6b00725e0 0x7fa6b0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:40.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.472+0000 7fa6c7400640 1 -- 172.21.15.94:0/3591809007 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa6c812c8e0 con 0x7fa6d0073920 2024-09-10T23:33:40.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.472+0000 7fa6cea00640 1 --2- 172.21.15.94:0/3591809007 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa6b00725e0 0x7fa6b0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:40.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.472+0000 7fa6cea00640 1 --2- 172.21.15.94:0/3591809007 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa6b00725e0 0x7fa6b0074aa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0005680 tx=0x7fa6c0003dd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.472+0000 7fa6c7400640 1 -- 172.21.15.94:0/3591809007 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa6c8016410 con 0x7fa6d0073920 2024-09-10T23:33:40.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.640+0000 7fa6d5e00640 1 -- 172.21.15.94:0/3591809007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa6d0154e80 con 0x7fa6d0073920 2024-09-10T23:33:40.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.640+0000 7fa6c7400640 1 -- 172.21.15.94:0/3591809007 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7fa6c80f7960 con 0x7fa6d0073920 2024-09-10T23:33:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.644+0000 7fa6d5e00640 1 -- 172.21.15.94:0/3591809007 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa6b00725e0 msgr2=0x7fa6b0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.644+0000 7fa6d5e00640 1 --2- 172.21.15.94:0/3591809007 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa6b00725e0 0x7fa6b0074aa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0005680 tx=0x7fa6c0003dd0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.644+0000 7fa6d5e00640 1 -- 172.21.15.94:0/3591809007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d0073920 msgr2=0x7fa6d01551e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.644+0000 7fa6d5e00640 1 --2- 172.21.15.94:0/3591809007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d0073920 0x7fa6d01551e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa6c801a420 tx=0x7fa6c80a56d0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.644+0000 7fa6d5e00640 1 -- 172.21.15.94:0/3591809007 shutdown_connections 2024-09-10T23:33:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.644+0000 7fa6d5e00640 1 --2- 172.21.15.94:0/3591809007 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa6b00725e0 0x7fa6b0074aa0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.644+0000 7fa6d5e00640 1 --2- 172.21.15.94:0/3591809007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6d0073920 0x7fa6d01551e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.644+0000 7fa6d5e00640 1 --2- 172.21.15.94:0/3591809007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6d006d260 0x7fa6d013e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.644+0000 7fa6d5e00640 1 --2- 172.21.15.94:0/3591809007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d006ce80 0x7fa6d014ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:40.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.644+0000 7fa6d5e00640 1 -- 172.21.15.94:0/3591809007 >> 172.21.15.94:0/3591809007 conn(0x7fa6d00d0d80 msgr2=0x7fa6d0153350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:40.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.644+0000 7fa6d5e00640 1 -- 172.21.15.94:0/3591809007 shutdown_connections 2024-09-10T23:33:40.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:40.644+0000 7fa6d5e00640 1 -- 172.21.15.94:0/3591809007 wait complete. 2024-09-10T23:33:40.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:40.668 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp21720/subvol47840) 2024-09-10T23:33:40.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:40.700 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T23:33:40.769 INFO:teuthology.orchestra.run.smithi094.stdout:cephfs_data 2024-09-10T23:33:40.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:40.769 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp21720/subvol47840) 2024-09-10T23:33:40.836 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:33:40.837 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp21720/subvol47840/.meta 2024-09-10T23:33:40.905 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol47840 subvol_grp21720 2024-09-10T23:33:41.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.092+0000 7fe081600640 1 -- 172.21.15.94:0/2606361931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe07c0d2290 msgr2=0x7fe07c166b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:41.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.092+0000 7fe081600640 1 --2- 172.21.15.94:0/2606361931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe07c0d2290 0x7fe07c166b70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe07c15c9a0 tx=0x7fe07401c7f0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.092+0000 7fe081600640 1 -- 172.21.15.94:0/2606361931 shutdown_connections 2024-09-10T23:33:41.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.092+0000 7fe081600640 1 --2- 172.21.15.94:0/2606361931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe07c0d2290 0x7fe07c166b70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.092+0000 7fe081600640 1 -- 172.21.15.94:0/2606361931 >> 172.21.15.94:0/2606361931 conn(0x7fe07c09aa70 msgr2=0x7fe07c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:41.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.092+0000 7fe081600640 1 -- 172.21.15.94:0/2606361931 shutdown_connections 2024-09-10T23:33:41.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.092+0000 7fe081600640 1 -- 172.21.15.94:0/2606361931 wait complete. 2024-09-10T23:33:41.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.092+0000 7fe081600640 1 Processor -- start 2024-09-10T23:33:41.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.092+0000 7fe081600640 1 -- start start 2024-09-10T23:33:41.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.092+0000 7fe081600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe07c0ae980 0x7fe07c15a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:41.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.092+0000 7fe081600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe07c0d2290 0x7fe07c1fb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:41.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.092+0000 7fe081600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe07c15bd30 0x7fe07c1fb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.092+0000 7fe081600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c16a360 con 0x7fe07c0ae980 2024-09-10T23:33:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.092+0000 7fe081600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c16a1e0 con 0x7fe07c15bd30 2024-09-10T23:33:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.092+0000 7fe081600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe07c16a4e0 con 0x7fe07c0d2290 2024-09-10T23:33:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe07aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe07c0ae980 0x7fe07c15a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe07aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe07c0ae980 0x7fe07c15a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34216/0 (socket says 172.21.15.94:34216) 2024-09-10T23:33:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe07aa00640 1 -- 172.21.15.94:0/3452515209 learned_addr learned my addr 172.21.15.94:0/3452515209 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe072000640 1 --2- 172.21.15.94:0/3452515209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe07c0d2290 0x7fe07c1fb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe07b400640 1 --2- 172.21.15.94:0/3452515209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe07c15bd30 0x7fe07c1fb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe07aa00640 1 -- 172.21.15.94:0/3452515209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe07c0d2290 msgr2=0x7fe07c1fb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe07aa00640 1 --2- 172.21.15.94:0/3452515209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe07c0d2290 0x7fe07c1fb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe07aa00640 1 -- 172.21.15.94:0/3452515209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe07c15bd30 msgr2=0x7fe07c1fb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe07aa00640 1 --2- 172.21.15.94:0/3452515209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe07c15bd30 0x7fe07c1fb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe07aa00640 1 -- 172.21.15.94:0/3452515209 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07c15aaa0 con 0x7fe07c0ae980 2024-09-10T23:33:41.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe07b400640 1 --2- 172.21.15.94:0/3452515209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe07c15bd30 0x7fe07c1fb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe072000640 1 --2- 172.21.15.94:0/3452515209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe07c0d2290 0x7fe07c1fb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe07aa00640 1 --2- 172.21.15.94:0/3452515209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe07c0ae980 0x7fe07c15a0a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00c780 tx=0x7fe06c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe073400640 1 -- 172.21.15.94:0/3452515209 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c003b80 con 0x7fe07c0ae980 2024-09-10T23:33:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe073400640 1 -- 172.21.15.94:0/3452515209 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe06c003d20 con 0x7fe07c0ae980 2024-09-10T23:33:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe081600640 1 -- 172.21.15.94:0/3452515209 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe07c15ad80 con 0x7fe07c0ae980 2024-09-10T23:33:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe073400640 1 -- 172.21.15.94:0/3452515209 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c00aab0 con 0x7fe07c0ae980 2024-09-10T23:33:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe081600640 1 -- 172.21.15.94:0/3452515209 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe07c15b340 con 0x7fe07c0ae980 2024-09-10T23:33:41.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe081600640 1 -- 172.21.15.94:0/3452515209 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe07c0d1b80 con 0x7fe07c0ae980 2024-09-10T23:33:41.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.096+0000 7fe073400640 1 -- 172.21.15.94:0/3452515209 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe06c00ac50 con 0x7fe07c0ae980 2024-09-10T23:33:41.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.100+0000 7fe073400640 1 --2- 172.21.15.94:0/3452515209 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe0580725e0 0x7fe058074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:41.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.100+0000 7fe073400640 1 -- 172.21.15.94:0/3452515209 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe06c0931c0 con 0x7fe07c0ae980 2024-09-10T23:33:41.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.100+0000 7fe072000640 1 --2- 172.21.15.94:0/3452515209 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe0580725e0 0x7fe058074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:41.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.100+0000 7fe072000640 1 --2- 172.21.15.94:0/3452515209 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe0580725e0 0x7fe058074aa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe07c06ff10 tx=0x7fe064009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:41.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.104+0000 7fe073400640 1 -- 172.21.15.94:0/3452515209 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe06c05e780 con 0x7fe07c0ae980 2024-09-10T23:33:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.296+0000 7fe081600640 1 -- 172.21.15.94:0/3452515209 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol47840", "group_name": "subvol_grp21720", "target": ["mon-mgr", ""]}) -- 0x7fe07c15b6b0 con 0x7fe0580725e0 2024-09-10T23:33:41.306 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:33:41.305+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:33:41.306 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:33:41.305+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:33:41.306 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:33:41.305+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:33:41.306 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:33:41.305+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:33:41.306 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:33:41.305+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:33:41.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.456+0000 7fe073400640 1 -- 172.21.15.94:0/3452515209 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe07c15b6b0 con 0x7fe0580725e0 2024-09-10T23:33:41.462 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/subvol_grp21720/subvol47840 2024-09-10T23:33:41.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.460+0000 7fe081600640 1 -- 172.21.15.94:0/3452515209 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe0580725e0 msgr2=0x7fe058074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:41.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.460+0000 7fe081600640 1 --2- 172.21.15.94:0/3452515209 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe0580725e0 0x7fe058074aa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe07c06ff10 tx=0x7fe064009040 comp rx=0 tx=0).stop 2024-09-10T23:33:41.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.460+0000 7fe081600640 1 -- 172.21.15.94:0/3452515209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe07c0ae980 msgr2=0x7fe07c15a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:41.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.460+0000 7fe081600640 1 --2- 172.21.15.94:0/3452515209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe07c0ae980 0x7fe07c15a0a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00c780 tx=0x7fe06c00cc40 comp rx=0 tx=0).stop 2024-09-10T23:33:41.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.460+0000 7fe081600640 1 -- 172.21.15.94:0/3452515209 shutdown_connections 2024-09-10T23:33:41.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.460+0000 7fe081600640 1 --2- 172.21.15.94:0/3452515209 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe0580725e0 0x7fe058074aa0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.460+0000 7fe081600640 1 --2- 172.21.15.94:0/3452515209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe07c15bd30 0x7fe07c1fb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.460+0000 7fe081600640 1 --2- 172.21.15.94:0/3452515209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe07c0d2290 0x7fe07c1fb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.460+0000 7fe081600640 1 --2- 172.21.15.94:0/3452515209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe07c0ae980 0x7fe07c15a0a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.460+0000 7fe081600640 1 -- 172.21.15.94:0/3452515209 >> 172.21.15.94:0/3452515209 conn(0x7fe07c09aa70 msgr2=0x7fe07c070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:41.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.460+0000 7fe081600640 1 -- 172.21.15.94:0/3452515209 shutdown_connections 2024-09-10T23:33:41.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.460+0000 7fe081600640 1 -- 172.21.15.94:0/3452515209 wait complete. 2024-09-10T23:33:41.484 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol47840 subvol_grp21720 2024-09-10T23:33:41.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.624+0000 7f246be00640 1 -- 172.21.15.94:0/4055826185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f246c1560a0 msgr2=0x7f246c0cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:41.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.624+0000 7f246be00640 1 --2- 172.21.15.94:0/4055826185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f246c1560a0 0x7f246c0cfd50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f245c009c80 tx=0x7f245c01cd40 comp rx=0 tx=0).stop 2024-09-10T23:33:41.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.624+0000 7f246be00640 1 -- 172.21.15.94:0/4055826185 shutdown_connections 2024-09-10T23:33:41.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.624+0000 7f246be00640 1 --2- 172.21.15.94:0/4055826185 >> v2:172.21.15.181:3300/0 conn(0x7f246c06d600 0x7f246c073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.624+0000 7f246be00640 1 --2- 172.21.15.94:0/4055826185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f246c1560a0 0x7f246c0cfd50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.624+0000 7f246be00640 1 -- 172.21.15.94:0/4055826185 >> 172.21.15.94:0/4055826185 conn(0x7f246c0cd6b0 msgr2=0x7f246c0cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:41.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.624+0000 7f246be00640 1 -- 172.21.15.94:0/4055826185 shutdown_connections 2024-09-10T23:33:41.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.624+0000 7f246be00640 1 -- 172.21.15.94:0/4055826185 wait complete. 2024-09-10T23:33:41.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246be00640 1 Processor -- start 2024-09-10T23:33:41.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246be00640 1 -- start start 2024-09-10T23:33:41.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f246c06d600 0x7f246c1570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:41.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246be00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f246c0cc860 0x7f246c0c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:41.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f246c1560a0 0x7f246c0b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:41.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f246c168a00 con 0x7f246c06d600 2024-09-10T23:33:41.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246be00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f246c160140 con 0x7f246c1560a0 2024-09-10T23:33:41.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246be00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f246c168b80 con 0x7f246c0cc860 2024-09-10T23:33:41.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f246c06d600 0x7f246c1570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:41.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f246c06d600 0x7f246c1570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34220/0 (socket says 172.21.15.94:34220) 2024-09-10T23:33:41.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246aa00640 1 -- 172.21.15.94:0/2661021856 learned_addr learned my addr 172.21.15.94:0/2661021856 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:41.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246b400640 1 --2- 172.21.15.94:0/2661021856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f246c1560a0 0x7f246c0b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:41.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f2462000640 1 --2- 172.21.15.94:0/2661021856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f246c0cc860 0x7f246c0c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:41.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246aa00640 1 -- 172.21.15.94:0/2661021856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f246c0cc860 msgr2=0x7f246c0c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:41.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246aa00640 1 --2- 172.21.15.94:0/2661021856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f246c0cc860 0x7f246c0c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246aa00640 1 -- 172.21.15.94:0/2661021856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f246c1560a0 msgr2=0x7f246c0b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:41.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246aa00640 1 --2- 172.21.15.94:0/2661021856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f246c1560a0 0x7f246c0b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246aa00640 1 -- 172.21.15.94:0/2661021856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246c1577b0 con 0x7f246c06d600 2024-09-10T23:33:41.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246aa00640 1 --2- 172.21.15.94:0/2661021856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f246c06d600 0x7f246c1570d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f245c000f90 tx=0x7f245c005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:41.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f2463400640 1 -- 172.21.15.94:0/2661021856 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c0bb070 con 0x7f246c06d600 2024-09-10T23:33:41.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f2463400640 1 -- 172.21.15.94:0/2661021856 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f245c004b40 con 0x7f246c06d600 2024-09-10T23:33:41.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246be00640 1 -- 172.21.15.94:0/2661021856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f246c159a30 con 0x7f246c06d600 2024-09-10T23:33:41.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f2463400640 1 -- 172.21.15.94:0/2661021856 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c0b2040 con 0x7f246c06d600 2024-09-10T23:33:41.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246be00640 1 -- 172.21.15.94:0/2661021856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f246c0c9450 con 0x7f246c06d600 2024-09-10T23:33:41.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.628+0000 7f246be00640 1 -- 172.21.15.94:0/2661021856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f246c073e30 con 0x7f246c06d600 2024-09-10T23:33:41.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.632+0000 7f2463400640 1 -- 172.21.15.94:0/2661021856 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f245c0b5030 con 0x7f246c06d600 2024-09-10T23:33:41.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.632+0000 7f2463400640 1 --2- 172.21.15.94:0/2661021856 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24480725e0 0x7f2448074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:41.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.632+0000 7f2463400640 1 -- 172.21.15.94:0/2661021856 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f245c12cc40 con 0x7f246c06d600 2024-09-10T23:33:41.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.632+0000 7f2462000640 1 --2- 172.21.15.94:0/2661021856 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24480725e0 0x7f2448074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:41.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.632+0000 7f2462000640 1 --2- 172.21.15.94:0/2661021856 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24480725e0 0x7f2448074aa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f246c15f380 tx=0x7f24540091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:41.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.636+0000 7f2463400640 1 -- 172.21.15.94:0/2661021856 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f245c016990 con 0x7f246c06d600 2024-09-10T23:33:41.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.824+0000 7f246be00640 1 -- 172.21.15.94:0/2661021856 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol47840", "group_name": "subvol_grp21720", "target": ["mon-mgr", ""]}) -- 0x7f246c0ccff0 con 0x7f24480725e0 2024-09-10T23:33:41.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.832+0000 7f2463400640 1 -- 172.21.15.94:0/2661021856 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f246c0ccff0 con 0x7f24480725e0 2024-09-10T23:33:41.835 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/subvol_grp21720/subvol47840 2024-09-10T23:33:41.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.832+0000 7f246be00640 1 -- 172.21.15.94:0/2661021856 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24480725e0 msgr2=0x7f2448074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:41.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.832+0000 7f246be00640 1 --2- 172.21.15.94:0/2661021856 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24480725e0 0x7f2448074aa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f246c15f380 tx=0x7f24540091e0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.832+0000 7f246be00640 1 -- 172.21.15.94:0/2661021856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f246c06d600 msgr2=0x7f246c1570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:41.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.832+0000 7f246be00640 1 --2- 172.21.15.94:0/2661021856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f246c06d600 0x7f246c1570d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f245c000f90 tx=0x7f245c005ac0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.832+0000 7f246be00640 1 -- 172.21.15.94:0/2661021856 shutdown_connections 2024-09-10T23:33:41.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.832+0000 7f246be00640 1 --2- 172.21.15.94:0/2661021856 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24480725e0 0x7f2448074aa0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.832+0000 7f246be00640 1 --2- 172.21.15.94:0/2661021856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f246c1560a0 0x7f246c0b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.832+0000 7f246be00640 1 --2- 172.21.15.94:0/2661021856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f246c0cc860 0x7f246c0c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.832+0000 7f246be00640 1 --2- 172.21.15.94:0/2661021856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f246c06d600 0x7f246c1570d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:41.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.832+0000 7f246be00640 1 -- 172.21.15.94:0/2661021856 >> 172.21.15.94:0/2661021856 conn(0x7f246c0cd6b0 msgr2=0x7f246c06e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:41.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.832+0000 7f246be00640 1 -- 172.21.15.94:0/2661021856 shutdown_connections 2024-09-10T23:33:41.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:41.832+0000 7f246be00640 1 -- 172.21.15.94:0/2661021856 wait complete. 2024-09-10T23:33:41.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:41.857 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/6fc099154395cdf8226ee8c038ab674b.meta) 2024-09-10T23:33:43.019 INFO:teuthology.orchestra.run.smithi094.stdout:[GLOBAL] 2024-09-10T23:33:43.019 INFO:teuthology.orchestra.run.smithi094.stdout:version = 1 2024-09-10T23:33:43.019 INFO:teuthology.orchestra.run.smithi094.stdout:type = subvolume 2024-09-10T23:33:43.019 INFO:teuthology.orchestra.run.smithi094.stdout:path = /volumes/subvol_grp21720/subvol47840 2024-09-10T23:33:43.019 INFO:teuthology.orchestra.run.smithi094.stdout:state = complete 2024-09-10T23:33:43.019 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:43.021 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol47840 subvol_grp21720 2024-09-10T23:33:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.168+0000 7f33c0200640 1 -- 172.21.15.94:0/1033799754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b80d01f0 msgr2=0x7f33b815f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.168+0000 7f33c0200640 1 --2- 172.21.15.94:0/1033799754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b80d01f0 0x7f33b815f6c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f33b80d0890 tx=0x7f33b001c570 comp rx=0 tx=0).stop 2024-09-10T23:33:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.168+0000 7f33c0200640 1 -- 172.21.15.94:0/1033799754 shutdown_connections 2024-09-10T23:33:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.168+0000 7f33c0200640 1 --2- 172.21.15.94:0/1033799754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b80d01f0 0x7f33b815f6c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.168+0000 7f33c0200640 1 --2- 172.21.15.94:0/1033799754 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33b80b4c40 0x7f33b80cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.168+0000 7f33c0200640 1 --2- 172.21.15.94:0/1033799754 >> v2:172.21.15.181:3300/0 conn(0x7f33b815ba50 0x7f33b80b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.168+0000 7f33c0200640 1 -- 172.21.15.94:0/1033799754 >> 172.21.15.94:0/1033799754 conn(0x7f33b806cdf0 msgr2=0x7f33b806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.168+0000 7f33c0200640 1 -- 172.21.15.94:0/1033799754 shutdown_connections 2024-09-10T23:33:43.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.168+0000 7f33c0200640 1 -- 172.21.15.94:0/1033799754 wait complete. 2024-09-10T23:33:43.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.168+0000 7f33c0200640 1 Processor -- start 2024-09-10T23:33:43.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.168+0000 7f33c0200640 1 -- start start 2024-09-10T23:33:43.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33c0200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b80b4c40 0x7f33b814e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:43.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33c0200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33b80d01f0 0x7f33b8152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:43.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33c0200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b815ba50 0x7f33b8072870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:43.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33c0200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33b80ce4a0 con 0x7f33b80b4c40 2024-09-10T23:33:43.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33c0200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f33b80ce320 con 0x7f33b815ba50 2024-09-10T23:33:43.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33c0200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f33b80ce620 con 0x7f33b80d01f0 2024-09-10T23:33:43.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33bd800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b80b4c40 0x7f33b814e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:43.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33bd800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b80b4c40 0x7f33b814e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34232/0 (socket says 172.21.15.94:34232) 2024-09-10T23:33:43.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33bd800640 1 -- 172.21.15.94:0/4068746101 learned_addr learned my addr 172.21.15.94:0/4068746101 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:43.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33be200640 1 --2- 172.21.15.94:0/4068746101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b815ba50 0x7f33b8072870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:43.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33bce00640 1 --2- 172.21.15.94:0/4068746101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33b80d01f0 0x7f33b8152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:43.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33bd800640 1 -- 172.21.15.94:0/4068746101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33b80d01f0 msgr2=0x7f33b8152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:43.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33bd800640 1 --2- 172.21.15.94:0/4068746101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33b80d01f0 0x7f33b8152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33bd800640 1 -- 172.21.15.94:0/4068746101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b815ba50 msgr2=0x7f33b8072870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:43.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33bd800640 1 --2- 172.21.15.94:0/4068746101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b815ba50 0x7f33b8072870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33bd800640 1 -- 172.21.15.94:0/4068746101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b814e740 con 0x7f33b80b4c40 2024-09-10T23:33:43.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33bce00640 1 --2- 172.21.15.94:0/4068746101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33b80d01f0 0x7f33b8152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:43.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33bd800640 1 --2- 172.21.15.94:0/4068746101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b80b4c40 0x7f33b814e160 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00ccb0 tx=0x7f33ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:43.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33b5600640 1 -- 172.21.15.94:0/4068746101 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00f450 con 0x7f33b80b4c40 2024-09-10T23:33:43.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33b5600640 1 -- 172.21.15.94:0/4068746101 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33ac00f6e0 con 0x7f33b80b4c40 2024-09-10T23:33:43.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33c0200640 1 -- 172.21.15.94:0/4068746101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b8159600 con 0x7f33b80b4c40 2024-09-10T23:33:43.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33b5600640 1 -- 172.21.15.94:0/4068746101 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00fb30 con 0x7f33b80b4c40 2024-09-10T23:33:43.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33c0200640 1 -- 172.21.15.94:0/4068746101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b8159ea0 con 0x7f33b80b4c40 2024-09-10T23:33:43.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33c0200640 1 -- 172.21.15.94:0/4068746101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b815f6c0 con 0x7f33b80b4c40 2024-09-10T23:33:43.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.172+0000 7f33b5600640 1 -- 172.21.15.94:0/4068746101 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 6) ==== 96445+0+0 (secure 0 0 0) 0x7f33ac020030 con 0x7f33b80b4c40 2024-09-10T23:33:43.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.176+0000 7f33b5600640 1 --2- 172.21.15.94:0/4068746101 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f33980726a0 0x7f3398074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:43.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.176+0000 7f33b5600640 1 -- 172.21.15.94:0/4068746101 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f33ac093c20 con 0x7f33b80b4c40 2024-09-10T23:33:43.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.176+0000 7f33bce00640 1 --2- 172.21.15.94:0/4068746101 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f33980726a0 0x7f3398074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:43.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.176+0000 7f33bce00640 1 --2- 172.21.15.94:0/4068746101 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f33980726a0 0x7f3398074b60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f33a4005fd0 tx=0x7f33a4005f20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:43.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.180+0000 7f33b5600640 1 -- 172.21.15.94:0/4068746101 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f33ac014030 con 0x7f33b80b4c40 2024-09-10T23:33:43.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.368+0000 7f33c0200640 1 -- 172.21.15.94:0/4068746101 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol47840", "group_name": "subvol_grp21720", "target": ["mon-mgr", ""]}) -- 0x7f33b8165870 con 0x7f33980726a0 2024-09-10T23:33:43.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.384+0000 7f33b5600640 1 -- 172.21.15.94:0/4068746101 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f33b8165870 con 0x7f33980726a0 2024-09-10T23:33:43.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.384+0000 7f33c0200640 1 -- 172.21.15.94:0/4068746101 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f33980726a0 msgr2=0x7f3398074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:43.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.384+0000 7f33c0200640 1 --2- 172.21.15.94:0/4068746101 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f33980726a0 0x7f3398074b60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f33a4005fd0 tx=0x7f33a4005f20 comp rx=0 tx=0).stop 2024-09-10T23:33:43.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.384+0000 7f33c0200640 1 -- 172.21.15.94:0/4068746101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b80b4c40 msgr2=0x7f33b814e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:43.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.384+0000 7f33c0200640 1 --2- 172.21.15.94:0/4068746101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b80b4c40 0x7f33b814e160 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00ccb0 tx=0x7f33ac00a620 comp rx=0 tx=0).stop 2024-09-10T23:33:43.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.384+0000 7f33c0200640 1 -- 172.21.15.94:0/4068746101 shutdown_connections 2024-09-10T23:33:43.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.384+0000 7f33c0200640 1 --2- 172.21.15.94:0/4068746101 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f33980726a0 0x7f3398074b60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.384+0000 7f33c0200640 1 --2- 172.21.15.94:0/4068746101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33b815ba50 0x7f33b8072870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.384+0000 7f33c0200640 1 --2- 172.21.15.94:0/4068746101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33b80d01f0 0x7f33b8152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.384+0000 7f33c0200640 1 --2- 172.21.15.94:0/4068746101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33b80b4c40 0x7f33b814e160 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.384+0000 7f33c0200640 1 -- 172.21.15.94:0/4068746101 >> 172.21.15.94:0/4068746101 conn(0x7f33b806cdf0 msgr2=0x7f33b8166070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:43.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.384+0000 7f33c0200640 1 -- 172.21.15.94:0/4068746101 shutdown_connections 2024-09-10T23:33:43.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.384+0000 7f33c0200640 1 -- 172.21.15.94:0/4068746101 wait complete. 2024-09-10T23:33:43.393 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:33:43.389+0000 7fb90c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:33:43.393 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:33:43.389+0000 7fb90c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:33:43.393 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:33:43.389+0000 7fb90c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:33:43.393 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:33:43.389+0000 7fb90c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:33:43.393 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:33:43.389+0000 7fb90c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:33:43.409 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T23:33:43.411 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:33:43.409+0000 7fb90be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:33:43.411 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:33:43.409+0000 7fb90be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:33:43.411 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:33:43.409+0000 7fb90be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:33:43.411 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:33:43.409+0000 7fb90be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:33:43.411 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:33:43.409+0000 7fb90be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:33:43.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T23:33:43.464 INFO:teuthology.orchestra.run.smithi094.stdout:2 2024-09-10T23:33:43.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:33:43.471 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp21720 2024-09-10T23:33:43.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.612+0000 7f9b05c00640 1 -- 172.21.15.94:0/1363856239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b000cfce0 msgr2=0x7f9b000d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:43.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.612+0000 7f9b05c00640 1 --2- 172.21.15.94:0/1363856239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b000cfce0 0x7f9b000d0120 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9b000738c0 tx=0x7f9ae801c570 comp rx=0 tx=0).stop 2024-09-10T23:33:43.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.612+0000 7f9b05c00640 1 -- 172.21.15.94:0/1363856239 shutdown_connections 2024-09-10T23:33:43.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.612+0000 7f9b05c00640 1 --2- 172.21.15.94:0/1363856239 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b000d0660 0x7f9b000ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.612+0000 7f9b05c00640 1 --2- 172.21.15.94:0/1363856239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b000cfce0 0x7f9b000d0120 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.612+0000 7f9b05c00640 1 --2- 172.21.15.94:0/1363856239 >> v2:172.21.15.181:3300/0 conn(0x7f9b0015bf60 0x7f9b000d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.612+0000 7f9b05c00640 1 -- 172.21.15.94:0/1363856239 >> 172.21.15.94:0/1363856239 conn(0x7f9b000d1c60 msgr2=0x7f9b000d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:43.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.612+0000 7f9b05c00640 1 -- 172.21.15.94:0/1363856239 shutdown_connections 2024-09-10T23:33:43.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.612+0000 7f9b05c00640 1 -- 172.21.15.94:0/1363856239 wait complete. 2024-09-10T23:33:43.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.612+0000 7f9b05c00640 1 Processor -- start 2024-09-10T23:33:43.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.612+0000 7f9b05c00640 1 -- start start 2024-09-10T23:33:43.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9b05c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b000cfce0 0x7f9b00151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:43.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9b05c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b000d0660 0x7f9b000b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:43.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9b05c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b0015bf60 0x7f9b0015e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:43.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9b05c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b000c6a80 con 0x7f9b000d0660 2024-09-10T23:33:43.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9b05c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9b000c6900 con 0x7f9b000cfce0 2024-09-10T23:33:43.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9b05c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9b000c6c00 con 0x7f9b0015bf60 2024-09-10T23:33:43.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9af6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b000d0660 0x7f9b000b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:43.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9af6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b000d0660 0x7f9b000b6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34244/0 (socket says 172.21.15.94:34244) 2024-09-10T23:33:43.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9af6000640 1 -- 172.21.15.94:0/3501440469 learned_addr learned my addr 172.21.15.94:0/3501440469 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:43.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9af6000640 1 -- 172.21.15.94:0/3501440469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b0015bf60 msgr2=0x7f9b0015e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:43.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9af6000640 1 --2- 172.21.15.94:0/3501440469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b0015bf60 0x7f9b0015e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9af6000640 1 -- 172.21.15.94:0/3501440469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b000cfce0 msgr2=0x7f9b00151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:43.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9af6000640 1 --2- 172.21.15.94:0/3501440469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b000cfce0 0x7f9b00151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9af6000640 1 -- 172.21.15.94:0/3501440469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b000b4fb0 con 0x7f9b000d0660 2024-09-10T23:33:43.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9af6000640 1 --2- 172.21.15.94:0/3501440469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b000d0660 0x7f9b000b6050 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9ae801ca50 tx=0x7f9ae80a64c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:43.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9af7400640 1 -- 172.21.15.94:0/3501440469 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae80ac070 con 0x7f9b000d0660 2024-09-10T23:33:43.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9af7400640 1 -- 172.21.15.94:0/3501440469 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ae8005140 con 0x7f9b000d0660 2024-09-10T23:33:43.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9b05c00640 1 -- 172.21.15.94:0/3501440469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b000b5290 con 0x7f9b000d0660 2024-09-10T23:33:43.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9af7400640 1 -- 172.21.15.94:0/3501440469 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8005590 con 0x7f9b000d0660 2024-09-10T23:33:43.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9b05c00640 1 -- 172.21.15.94:0/3501440469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b000b5850 con 0x7f9b000d0660 2024-09-10T23:33:43.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9b05c00640 1 -- 172.21.15.94:0/3501440469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b000ce3f0 con 0x7f9b000d0660 2024-09-10T23:33:43.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.616+0000 7f9af7400640 1 -- 172.21.15.94:0/3501440469 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 6) ==== 96445+0+0 (secure 0 0 0) 0x7f9ae80a6040 con 0x7f9b000d0660 2024-09-10T23:33:43.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.620+0000 7f9af7400640 1 --2- 172.21.15.94:0/3501440469 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9adc072670 0x7f9adc074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:43.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.620+0000 7f9af7400640 1 -- 172.21.15.94:0/3501440469 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9ae812d650 con 0x7f9b000d0660 2024-09-10T23:33:43.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.620+0000 7f9afea00640 1 --2- 172.21.15.94:0/3501440469 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9adc072670 0x7f9adc074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:43.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.620+0000 7f9afea00640 1 --2- 172.21.15.94:0/3501440469 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9adc072670 0x7f9adc074b30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9af0005fd0 tx=0x7f9af0005f60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:43.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.624+0000 7f9af7400640 1 -- 172.21.15.94:0/3501440469 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9ae80164b0 con 0x7f9b000d0660 2024-09-10T23:33:43.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.808+0000 7f9b05c00640 1 -- 172.21.15.94:0/3501440469 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp21720", "target": ["mon-mgr", ""]}) -- 0x7f9b0006f1c0 con 0x7f9adc072670 2024-09-10T23:33:43.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.812+0000 7f9af7400640 1 -- 172.21.15.94:0/3501440469 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9b0006f1c0 con 0x7f9adc072670 2024-09-10T23:33:43.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.816+0000 7f9b05c00640 1 -- 172.21.15.94:0/3501440469 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9adc072670 msgr2=0x7f9adc074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:43.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.816+0000 7f9b05c00640 1 --2- 172.21.15.94:0/3501440469 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9adc072670 0x7f9adc074b30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9af0005fd0 tx=0x7f9af0005f60 comp rx=0 tx=0).stop 2024-09-10T23:33:43.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.816+0000 7f9b05c00640 1 -- 172.21.15.94:0/3501440469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b000d0660 msgr2=0x7f9b000b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:43.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.816+0000 7f9b05c00640 1 --2- 172.21.15.94:0/3501440469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b000d0660 0x7f9b000b6050 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9ae801ca50 tx=0x7f9ae80a64c0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.816+0000 7f9b05c00640 1 -- 172.21.15.94:0/3501440469 shutdown_connections 2024-09-10T23:33:43.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.816+0000 7f9b05c00640 1 --2- 172.21.15.94:0/3501440469 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9adc072670 0x7f9adc074b30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.816+0000 7f9b05c00640 1 --2- 172.21.15.94:0/3501440469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b0015bf60 0x7f9b0015e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.816+0000 7f9b05c00640 1 --2- 172.21.15.94:0/3501440469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b000d0660 0x7f9b000b6050 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.816+0000 7f9b05c00640 1 --2- 172.21.15.94:0/3501440469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b000cfce0 0x7f9b00151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:43.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.816+0000 7f9b05c00640 1 -- 172.21.15.94:0/3501440469 >> 172.21.15.94:0/3501440469 conn(0x7f9b000d1c60 msgr2=0x7f9b00158c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:43.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.816+0000 7f9b05c00640 1 -- 172.21.15.94:0/3501440469 shutdown_connections 2024-09-10T23:33:43.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:43.816+0000 7f9b05c00640 1 -- 172.21.15.94:0/3501440469 wait complete. 2024-09-10T23:33:43.839 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:33:43.840 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:33:43.929 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:33:44.045 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:33:44.045 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:33:44.045 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:33:44.045 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:44.054 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:33:44.054 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:44.102 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:33:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.280+0000 7ff276600640 1 -- 172.21.15.94:0/43931686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff27006d260 msgr2=0x7ff27015ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.280+0000 7ff276600640 1 --2- 172.21.15.94:0/43931686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff27006d260 0x7ff27015ca10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff2700d1c30 tx=0x7ff25c01c6b0 comp rx=0 tx=0).stop 2024-09-10T23:33:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.280+0000 7ff276600640 1 -- 172.21.15.94:0/43931686 shutdown_connections 2024-09-10T23:33:44.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.280+0000 7ff276600640 1 --2- 172.21.15.94:0/43931686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff27006d260 0x7ff27015ca10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.280+0000 7ff276600640 1 --2- 172.21.15.94:0/43931686 >> v2:172.21.15.181:3300/0 conn(0x7ff27006ce80 0x7ff27015f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.280+0000 7ff276600640 1 -- 172.21.15.94:0/43931686 >> 172.21.15.94:0/43931686 conn(0x7ff2700d0d80 msgr2=0x7ff2700d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.280+0000 7ff276600640 1 -- 172.21.15.94:0/43931686 shutdown_connections 2024-09-10T23:33:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.280+0000 7ff276600640 1 -- 172.21.15.94:0/43931686 wait complete. 2024-09-10T23:33:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff276600640 1 Processor -- start 2024-09-10T23:33:44.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff276600640 1 -- start start 2024-09-10T23:33:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff276600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff27006ce80 0x7ff270151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff276600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff27006d260 0x7ff2701519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff276600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2701fba70 0x7ff2700b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff276600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff270141860 con 0x7ff2701fba70 2024-09-10T23:33:44.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff276600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff2701416e0 con 0x7ff27006ce80 2024-09-10T23:33:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff276600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff2701419e0 con 0x7ff27006d260 2024-09-10T23:33:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff26fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2701fba70 0x7ff2700b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff26fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2701fba70 0x7ff2700b19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45526/0 (socket says 172.21.15.94:45526) 2024-09-10T23:33:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff26fe00640 1 -- 172.21.15.94:0/2049028260 learned_addr learned my addr 172.21.15.94:0/2049028260 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff26f400640 1 --2- 172.21.15.94:0/2049028260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff27006ce80 0x7ff270151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff26ea00640 1 --2- 172.21.15.94:0/2049028260 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff27006d260 0x7ff2701519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff26fe00640 1 -- 172.21.15.94:0/2049028260 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff27006d260 msgr2=0x7ff2701519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff26fe00640 1 --2- 172.21.15.94:0/2049028260 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff27006d260 0x7ff2701519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff26fe00640 1 -- 172.21.15.94:0/2049028260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff27006ce80 msgr2=0x7ff270151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff26fe00640 1 --2- 172.21.15.94:0/2049028260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff27006ce80 0x7ff270151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff26fe00640 1 -- 172.21.15.94:0/2049028260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff270151700 con 0x7ff2701fba70 2024-09-10T23:33:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff26ea00640 1 --2- 172.21.15.94:0/2049028260 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff27006d260 0x7ff2701519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:44.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff26fe00640 1 --2- 172.21.15.94:0/2049028260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2701fba70 0x7ff2700b19a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff26800be60 tx=0x7ff26800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:44.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff267400640 1 -- 172.21.15.94:0/2049028260 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268019070 con 0x7ff2701fba70 2024-09-10T23:33:44.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff267400640 1 -- 172.21.15.94:0/2049028260 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff268014440 con 0x7ff2701fba70 2024-09-10T23:33:44.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff276600640 1 -- 172.21.15.94:0/2049028260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2701faf90 con 0x7ff2701fba70 2024-09-10T23:33:44.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff267400640 1 -- 172.21.15.94:0/2049028260 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2680148d0 con 0x7ff2701fba70 2024-09-10T23:33:44.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff276600640 1 -- 172.21.15.94:0/2049028260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2701fb550 con 0x7ff2701fba70 2024-09-10T23:33:44.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.284+0000 7ff276600640 1 -- 172.21.15.94:0/2049028260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff270141f60 con 0x7ff2701fba70 2024-09-10T23:33:44.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.288+0000 7ff267400640 1 -- 172.21.15.94:0/2049028260 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 6) ==== 96445+0+0 (secure 0 0 0) 0x7ff268008030 con 0x7ff2701fba70 2024-09-10T23:33:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.288+0000 7ff267400640 1 --2- 172.21.15.94:0/2049028260 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff250072670 0x7ff250074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.288+0000 7ff267400640 1 -- 172.21.15.94:0/2049028260 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff268093320 con 0x7ff2701fba70 2024-09-10T23:33:44.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.288+0000 7ff26f400640 1 --2- 172.21.15.94:0/2049028260 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff250072670 0x7ff250074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:44.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.288+0000 7ff26f400640 1 --2- 172.21.15.94:0/2049028260 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff250072670 0x7ff250074b30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff260004630 tx=0x7ff26000b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:44.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.292+0000 7ff267400640 1 -- 172.21.15.94:0/2049028260 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff26805e240 con 0x7ff2701fba70 2024-09-10T23:33:44.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.468+0000 7ff276600640 1 -- 172.21.15.94:0/2049028260 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff270135a90 con 0x7ff2701fba70 2024-09-10T23:33:44.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.468+0000 7ff267400640 1 -- 172.21.15.94:0/2049028260 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7ff268062c70 con 0x7ff2701fba70 2024-09-10T23:33:44.472 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:33:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.468+0000 7ff276600640 1 -- 172.21.15.94:0/2049028260 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff250072670 msgr2=0x7ff250074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.468+0000 7ff276600640 1 --2- 172.21.15.94:0/2049028260 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff250072670 0x7ff250074b30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff260004630 tx=0x7ff26000b390 comp rx=0 tx=0).stop 2024-09-10T23:33:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.468+0000 7ff276600640 1 -- 172.21.15.94:0/2049028260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2701fba70 msgr2=0x7ff2700b19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.468+0000 7ff276600640 1 --2- 172.21.15.94:0/2049028260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2701fba70 0x7ff2700b19a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff26800be60 tx=0x7ff26800bf50 comp rx=0 tx=0).stop 2024-09-10T23:33:44.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.468+0000 7ff276600640 1 -- 172.21.15.94:0/2049028260 shutdown_connections 2024-09-10T23:33:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.468+0000 7ff276600640 1 --2- 172.21.15.94:0/2049028260 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff250072670 0x7ff250074b30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.468+0000 7ff276600640 1 --2- 172.21.15.94:0/2049028260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2701fba70 0x7ff2700b19a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.468+0000 7ff276600640 1 --2- 172.21.15.94:0/2049028260 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff27006d260 0x7ff2701519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.468+0000 7ff276600640 1 --2- 172.21.15.94:0/2049028260 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff27006ce80 0x7ff270151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.468+0000 7ff276600640 1 -- 172.21.15.94:0/2049028260 >> 172.21.15.94:0/2049028260 conn(0x7ff2700d0d80 msgr2=0x7ff2700d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.468+0000 7ff276600640 1 -- 172.21.15.94:0/2049028260 shutdown_connections 2024-09-10T23:33:44.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.468+0000 7ff276600640 1 -- 172.21.15.94:0/2049028260 wait complete. 2024-09-10T23:33:44.494 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-09-10T23:33:44.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:44.494 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T23:33:44.511 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[37154]: fuse finished with error 0 and tester_r 0 2024-09-10T23:33:44.514 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:33:44.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:44.514 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T23:33:44.686 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:33:44.686 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:33:44.686 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:44.695 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:33:44.695 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:44.746 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:33:44.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.884+0000 7fc1ce200640 1 -- 172.21.15.94:0/848020403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c815bf60 msgr2=0x7fc1c80d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:44.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.884+0000 7fc1ce200640 1 --2- 172.21.15.94:0/848020403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c815bf60 0x7fc1c80d2780 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009900 tx=0x7fc1b801c850 comp rx=0 tx=0).stop 2024-09-10T23:33:44.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.884+0000 7fc1ce200640 1 -- 172.21.15.94:0/848020403 shutdown_connections 2024-09-10T23:33:44.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.884+0000 7fc1ce200640 1 --2- 172.21.15.94:0/848020403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c80d0830 0x7fc1c80c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:44.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.884+0000 7fc1ce200640 1 --2- 172.21.15.94:0/848020403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c815bf60 0x7fc1c80d2780 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:44.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.884+0000 7fc1ce200640 1 -- 172.21.15.94:0/848020403 >> 172.21.15.94:0/848020403 conn(0x7fc1c80d1c60 msgr2=0x7fc1c80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:44.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.884+0000 7fc1ce200640 1 -- 172.21.15.94:0/848020403 shutdown_connections 2024-09-10T23:33:44.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.884+0000 7fc1ce200640 1 -- 172.21.15.94:0/848020403 wait complete. 2024-09-10T23:33:44.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.884+0000 7fc1ce200640 1 Processor -- start 2024-09-10T23:33:44.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.884+0000 7fc1ce200640 1 -- start start 2024-09-10T23:33:44.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1ce200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1c80cfce0 0x7fc1c8152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:44.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1ce200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c80d0830 0x7fc1c8152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:44.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1ce200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c815bf60 0x7fc1c80cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:44.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1ce200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c80c7070 con 0x7fc1c80cfce0 2024-09-10T23:33:44.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1ce200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c80c6ef0 con 0x7fc1c80d0830 2024-09-10T23:33:44.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1ce200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c8168980 con 0x7fc1c815bf60 2024-09-10T23:33:44.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1c7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1c80cfce0 0x7fc1c8152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:44.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1c7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1c80cfce0 0x7fc1c8152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45532/0 (socket says 172.21.15.94:45532) 2024-09-10T23:33:44.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1c7400640 1 -- 172.21.15.94:0/3227017437 learned_addr learned my addr 172.21.15.94:0/3227017437 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:44.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1bea00640 1 --2- 172.21.15.94:0/3227017437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c80d0830 0x7fc1c8152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:44.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1c7e00640 1 --2- 172.21.15.94:0/3227017437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c815bf60 0x7fc1c80cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:44.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1c7400640 1 -- 172.21.15.94:0/3227017437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c815bf60 msgr2=0x7fc1c80cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:44.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1c7400640 1 --2- 172.21.15.94:0/3227017437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c815bf60 0x7fc1c80cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:44.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1c7400640 1 -- 172.21.15.94:0/3227017437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c80d0830 msgr2=0x7fc1c8152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:44.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1c7400640 1 --2- 172.21.15.94:0/3227017437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c80d0830 0x7fc1c8152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:44.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1c7400640 1 -- 172.21.15.94:0/3227017437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c8156f30 con 0x7fc1c80cfce0 2024-09-10T23:33:44.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1c7400640 1 --2- 172.21.15.94:0/3227017437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1c80cfce0 0x7fc1c8152d00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009850 tx=0x7fc1b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:44.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1bfe00640 1 -- 172.21.15.94:0/3227017437 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80b2040 con 0x7fc1c80cfce0 2024-09-10T23:33:44.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1bfe00640 1 -- 172.21.15.94:0/3227017437 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1b80044e0 con 0x7fc1c80cfce0 2024-09-10T23:33:44.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1ce200640 1 -- 172.21.15.94:0/3227017437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c8157210 con 0x7fc1c80cfce0 2024-09-10T23:33:44.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1bfe00640 1 -- 172.21.15.94:0/3227017437 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80054b0 con 0x7fc1c80cfce0 2024-09-10T23:33:44.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1ce200640 1 -- 172.21.15.94:0/3227017437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c81577d0 con 0x7fc1c80cfce0 2024-09-10T23:33:44.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.888+0000 7fc1ce200640 1 -- 172.21.15.94:0/3227017437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c80c5e50 con 0x7fc1c80cfce0 2024-09-10T23:33:44.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.892+0000 7fc1bfe00640 1 -- 172.21.15.94:0/3227017437 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 6) ==== 96445+0+0 (secure 0 0 0) 0x7fc1b80046d0 con 0x7fc1c80cfce0 2024-09-10T23:33:44.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.892+0000 7fc1bfe00640 1 --2- 172.21.15.94:0/3227017437 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc1a4072670 0x7fc1a4074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:44.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.892+0000 7fc1bfe00640 1 -- 172.21.15.94:0/3227017437 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc1b812cfe0 con 0x7fc1c80cfce0 2024-09-10T23:33:44.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.892+0000 7fc1bea00640 1 --2- 172.21.15.94:0/3227017437 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc1a4072670 0x7fc1a4074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:44.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.892+0000 7fc1bea00640 1 --2- 172.21.15.94:0/3227017437 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc1a4072670 0x7fc1a4074b30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc1c815b690 tx=0x7fc1b0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:44.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:44.896+0000 7fc1bfe00640 1 -- 172.21.15.94:0/3227017437 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc1b80164d0 con 0x7fc1c80cfce0 2024-09-10T23:33:45.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.072+0000 7fc1ce200640 1 -- 172.21.15.94:0/3227017437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc1c80c7950 con 0x7fc1c80cfce0 2024-09-10T23:33:45.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.072+0000 7fc1bfe00640 1 -- 172.21.15.94:0/3227017437 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7fc1b80f7ed0 con 0x7fc1c80cfce0 2024-09-10T23:33:45.078 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:33:45.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.076+0000 7fc1ce200640 1 -- 172.21.15.94:0/3227017437 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc1a4072670 msgr2=0x7fc1a4074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:45.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.076+0000 7fc1ce200640 1 --2- 172.21.15.94:0/3227017437 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc1a4072670 0x7fc1a4074b30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc1c815b690 tx=0x7fc1b0009040 comp rx=0 tx=0).stop 2024-09-10T23:33:45.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.076+0000 7fc1ce200640 1 -- 172.21.15.94:0/3227017437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1c80cfce0 msgr2=0x7fc1c8152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:45.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.076+0000 7fc1ce200640 1 --2- 172.21.15.94:0/3227017437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1c80cfce0 0x7fc1c8152d00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009850 tx=0x7fc1b80042f0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.076+0000 7fc1ce200640 1 -- 172.21.15.94:0/3227017437 shutdown_connections 2024-09-10T23:33:45.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.076+0000 7fc1ce200640 1 --2- 172.21.15.94:0/3227017437 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc1a4072670 0x7fc1a4074b30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.076+0000 7fc1ce200640 1 --2- 172.21.15.94:0/3227017437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c80d0830 0x7fc1c8152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.076+0000 7fc1ce200640 1 --2- 172.21.15.94:0/3227017437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c815bf60 0x7fc1c80cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.076+0000 7fc1ce200640 1 --2- 172.21.15.94:0/3227017437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1c80cfce0 0x7fc1c8152d00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.076+0000 7fc1ce200640 1 -- 172.21.15.94:0/3227017437 >> 172.21.15.94:0/3227017437 conn(0x7fc1c80d1c60 msgr2=0x7fc1c8071880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:45.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.076+0000 7fc1ce200640 1 -- 172.21.15.94:0/3227017437 shutdown_connections 2024-09-10T23:33:45.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.076+0000 7fc1ce200640 1 -- 172.21.15.94:0/3227017437 wait complete. 2024-09-10T23:33:45.098 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi181.front.sepia.ceph.com... 2024-09-10T23:33:45.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:45.099 DEBUG:teuthology.orchestra.run.smithi181:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T23:33:45.119 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[34425]: fuse finished with error 0 and tester_r 0 2024-09-10T23:33:45.124 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:33:45.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:33:45.125 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T23:33:45.179 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:33:45.180 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:33:45.180 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:45.189 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:33:45.189 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:33:45.189 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:33:45.189 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:45.230 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:33:45.230 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:33:45.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.368+0000 7fbc0d400640 1 -- 172.21.15.94:0/1573578717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc080b2bb0 msgr2=0x7fbc080d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:45.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.368+0000 7fbc0d400640 1 --2- 172.21.15.94:0/1573578717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc080b2bb0 0x7fbc080d0270 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009840 tx=0x7fbbf001c700 comp rx=0 tx=0).stop 2024-09-10T23:33:45.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.368+0000 7fbc0d400640 1 -- 172.21.15.94:0/1573578717 shutdown_connections 2024-09-10T23:33:45.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.368+0000 7fbc0d400640 1 --2- 172.21.15.94:0/1573578717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc080b2bb0 0x7fbc080d0270 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.368+0000 7fbc0d400640 1 -- 172.21.15.94:0/1573578717 >> 172.21.15.94:0/1573578717 conn(0x7fbc080d1c60 msgr2=0x7fbc080d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:45.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.368+0000 7fbc0d400640 1 -- 172.21.15.94:0/1573578717 shutdown_connections 2024-09-10T23:33:45.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.368+0000 7fbc0d400640 1 -- 172.21.15.94:0/1573578717 wait complete. 2024-09-10T23:33:45.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.368+0000 7fbc0d400640 1 Processor -- start 2024-09-10T23:33:45.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.368+0000 7fbc0d400640 1 -- start start 2024-09-10T23:33:45.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.368+0000 7fbc0d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc080af310 0x7fbc0815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:45.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc0d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc080b2bb0 0x7fbc0815bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:45.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc0d400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc0815f5c0 0x7fbc0815c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:45.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc0d400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0816a0e0 con 0x7fbc080af310 2024-09-10T23:33:45.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc0d400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbc08169f60 con 0x7fbc080b2bb0 2024-09-10T23:33:45.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc0d400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fbc0816a260 con 0x7fbc0815f5c0 2024-09-10T23:33:45.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc06800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc080af310 0x7fbc0815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:45.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc06800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc080af310 0x7fbc0815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45558/0 (socket says 172.21.15.94:45558) 2024-09-10T23:33:45.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc06800640 1 -- 172.21.15.94:0/2558116978 learned_addr learned my addr 172.21.15.94:0/2558116978 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:45.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbbfde00640 1 --2- 172.21.15.94:0/2558116978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc080b2bb0 0x7fbc0815bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:45.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc06800640 1 -- 172.21.15.94:0/2558116978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc0815f5c0 msgr2=0x7fbc0815c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:45.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc07200640 1 --2- 172.21.15.94:0/2558116978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc0815f5c0 0x7fbc0815c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:45.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc06800640 1 --2- 172.21.15.94:0/2558116978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc0815f5c0 0x7fbc0815c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc06800640 1 -- 172.21.15.94:0/2558116978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc080b2bb0 msgr2=0x7fbc0815bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:45.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc06800640 1 --2- 172.21.15.94:0/2558116978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc080b2bb0 0x7fbc0815bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc06800640 1 -- 172.21.15.94:0/2558116978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc08153830 con 0x7fbc080af310 2024-09-10T23:33:45.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc07200640 1 --2- 172.21.15.94:0/2558116978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc0815f5c0 0x7fbc0815c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:45.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc06800640 1 --2- 172.21.15.94:0/2558116978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc080af310 0x7fbc0815b6a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800c7c0 tx=0x7fbbf800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:45.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbbff400640 1 -- 172.21.15.94:0/2558116978 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8003b80 con 0x7fbc080af310 2024-09-10T23:33:45.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbbff400640 1 -- 172.21.15.94:0/2558116978 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbf8003d20 con 0x7fbc080af310 2024-09-10T23:33:45.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc0d400640 1 -- 172.21.15.94:0/2558116978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc08153b10 con 0x7fbc080af310 2024-09-10T23:33:45.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbbff400640 1 -- 172.21.15.94:0/2558116978 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf800aa30 con 0x7fbc080af310 2024-09-10T23:33:45.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc0d400640 1 -- 172.21.15.94:0/2558116978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc08148510 con 0x7fbc080af310 2024-09-10T23:33:45.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbc0d400640 1 -- 172.21.15.94:0/2558116978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc080749c0 con 0x7fbc080af310 2024-09-10T23:33:45.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.372+0000 7fbbff400640 1 -- 172.21.15.94:0/2558116978 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 6) ==== 96445+0+0 (secure 0 0 0) 0x7fbbf800abd0 con 0x7fbc080af310 2024-09-10T23:33:45.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.376+0000 7fbbff400640 1 --2- 172.21.15.94:0/2558116978 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbbe40726a0 0x7fbbe4074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:45.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.376+0000 7fbbff400640 1 -- 172.21.15.94:0/2558116978 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fbbf8093260 con 0x7fbc080af310 2024-09-10T23:33:45.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.376+0000 7fbbfde00640 1 --2- 172.21.15.94:0/2558116978 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbbe40726a0 0x7fbbe4074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:45.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.376+0000 7fbbfde00640 1 --2- 172.21.15.94:0/2558116978 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbbe40726a0 0x7fbbe4074b60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00af000 tx=0x7fbbf001af40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:45.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.380+0000 7fbbff400640 1 -- 172.21.15.94:0/2558116978 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbbf805e740 con 0x7fbc080af310 2024-09-10T23:33:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.576+0000 7fbc0d400640 1 -- 172.21.15.94:0/2558116978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc08153ce0 con 0x7fbc080af310 2024-09-10T23:33:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.576+0000 7fbbff400640 1 -- 172.21.15.94:0/2558116978 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5238 (secure 0 0 0) 0x7fbbf805e040 con 0x7fbc080af310 2024-09-10T23:33:45.581 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:45.581 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":17,"btime":"2024-09-10T23:33:39:925925+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4900,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/765435890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":765435890},{"type":"v1","addr":"172.21.15.181:6837","nonce":765435890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3766049154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3766049154},{"type":"v1","addr":"172.21.15.94:6834","nonce":3766049154}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":194555,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2464857180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2464857180},{"type":"v1","addr":"172.21.15.94:6835","nonce":2464857180}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:29.261378+0000","modified":"2024-09-10T23:33:32.284558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194558},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194558":{"gid":194558,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1569801556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1569801556},{"type":"v1","addr":"172.21.15.181:6836","nonce":1569801556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194558,"qdb_cluster":[194558]},"id":2}]} 2024-09-10T23:33:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 17 2024-09-10T23:33:45.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.580+0000 7fbc0d400640 1 -- 172.21.15.94:0/2558116978 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbbe40726a0 msgr2=0x7fbbe4074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:45.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.580+0000 7fbc0d400640 1 --2- 172.21.15.94:0/2558116978 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbbe40726a0 0x7fbbe4074b60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00af000 tx=0x7fbbf001af40 comp rx=0 tx=0).stop 2024-09-10T23:33:45.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.580+0000 7fbc0d400640 1 -- 172.21.15.94:0/2558116978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc080af310 msgr2=0x7fbc0815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:45.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.580+0000 7fbc0d400640 1 --2- 172.21.15.94:0/2558116978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc080af310 0x7fbc0815b6a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800c7c0 tx=0x7fbbf800cc80 comp rx=0 tx=0).stop 2024-09-10T23:33:45.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.580+0000 7fbc0d400640 1 -- 172.21.15.94:0/2558116978 shutdown_connections 2024-09-10T23:33:45.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.580+0000 7fbc0d400640 1 --2- 172.21.15.94:0/2558116978 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbbe40726a0 0x7fbbe4074b60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.580+0000 7fbc0d400640 1 --2- 172.21.15.94:0/2558116978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc080b2bb0 0x7fbc0815bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.580+0000 7fbc0d400640 1 --2- 172.21.15.94:0/2558116978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc0815f5c0 0x7fbc0815c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.580+0000 7fbc0d400640 1 --2- 172.21.15.94:0/2558116978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc080af310 0x7fbc0815b6a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.580+0000 7fbc0d400640 1 -- 172.21.15.94:0/2558116978 >> 172.21.15.94:0/2558116978 conn(0x7fbc080d1c60 msgr2=0x7fbc081561a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:45.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.580+0000 7fbc0d400640 1 -- 172.21.15.94:0/2558116978 shutdown_connections 2024-09-10T23:33:45.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.580+0000 7fbc0d400640 1 -- 172.21.15.94:0/2558116978 wait complete. 2024-09-10T23:33:45.601 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:33:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.736+0000 7f34d6400640 1 -- 172.21.15.94:0/4155560164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d00ae980 msgr2=0x7f34d00d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.736+0000 7f34d6400640 1 --2- 172.21.15.94:0/4155560164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d00ae980 0x7f34d00d1b80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f34d00d2850 tx=0x7f34b801c830 comp rx=0 tx=0).stop 2024-09-10T23:33:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.736+0000 7f34d6400640 1 -- 172.21.15.94:0/4155560164 shutdown_connections 2024-09-10T23:33:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.736+0000 7f34d6400640 1 --2- 172.21.15.94:0/4155560164 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34d00b2ec0 0x7f34d00d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.736+0000 7f34d6400640 1 --2- 172.21.15.94:0/4155560164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d00ae980 0x7f34d00d1b80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.736+0000 7f34d6400640 1 --2- 172.21.15.94:0/4155560164 >> v2:172.21.15.181:3300/0 conn(0x7f34d015bd30 0x7f34d015c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.736+0000 7f34d6400640 1 -- 172.21.15.94:0/4155560164 >> 172.21.15.94:0/4155560164 conn(0x7f34d009aa70 msgr2=0x7f34d009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.736+0000 7f34d6400640 1 -- 172.21.15.94:0/4155560164 shutdown_connections 2024-09-10T23:33:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.736+0000 7f34d6400640 1 -- 172.21.15.94:0/4155560164 wait complete. 2024-09-10T23:33:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34d6400640 1 Processor -- start 2024-09-10T23:33:45.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34d6400640 1 -- start start 2024-09-10T23:33:45.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34d6400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d00ae980 0x7f34d0156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34d6400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34d00b2ec0 0x7f34d0071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34d6400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34d015bd30 0x7f34d00716d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34d6400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f34d00b1ff0 con 0x7f34d00ae980 2024-09-10T23:33:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34d6400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f34d00b1e70 con 0x7f34d00b2ec0 2024-09-10T23:33:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34d6400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f34d00b2170 con 0x7f34d015bd30 2024-09-10T23:33:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34cf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d00ae980 0x7f34d0156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34cf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d00ae980 0x7f34d0156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45566/0 (socket says 172.21.15.94:45566) 2024-09-10T23:33:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34cf400640 1 -- 172.21.15.94:0/758596789 learned_addr learned my addr 172.21.15.94:0/758596789 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34c6a00640 1 --2- 172.21.15.94:0/758596789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34d00b2ec0 0x7f34d0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34cfe00640 1 --2- 172.21.15.94:0/758596789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34d015bd30 0x7f34d00716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34cf400640 1 -- 172.21.15.94:0/758596789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34d015bd30 msgr2=0x7f34d00716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:45.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34cf400640 1 --2- 172.21.15.94:0/758596789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34d015bd30 0x7f34d00716d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34cf400640 1 -- 172.21.15.94:0/758596789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34d00b2ec0 msgr2=0x7f34d0071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:45.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34cf400640 1 --2- 172.21.15.94:0/758596789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34d00b2ec0 0x7f34d0071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34cf400640 1 -- 172.21.15.94:0/758596789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d0157480 con 0x7f34d00ae980 2024-09-10T23:33:45.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34cf400640 1 --2- 172.21.15.94:0/758596789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d00ae980 0x7f34d0156f40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f34c000d950 tx=0x7f34c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:45.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34c7e00640 1 -- 172.21.15.94:0/758596789 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0017070 con 0x7f34d00ae980 2024-09-10T23:33:45.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34c7e00640 1 -- 172.21.15.94:0/758596789 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34c0009d20 con 0x7f34d00ae980 2024-09-10T23:33:45.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34d6400640 1 -- 172.21.15.94:0/758596789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d00cc3c0 con 0x7f34d00ae980 2024-09-10T23:33:45.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34c7e00640 1 -- 172.21.15.94:0/758596789 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c00106c0 con 0x7f34d00ae980 2024-09-10T23:33:45.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.740+0000 7f34d6400640 1 -- 172.21.15.94:0/758596789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d006e580 con 0x7f34d00ae980 2024-09-10T23:33:45.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.744+0000 7f34d6400640 1 -- 172.21.15.94:0/758596789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d015c110 con 0x7f34d00ae980 2024-09-10T23:33:45.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.744+0000 7f34c7e00640 1 -- 172.21.15.94:0/758596789 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 6) ==== 96445+0+0 (secure 0 0 0) 0x7f34c0010860 con 0x7f34d00ae980 2024-09-10T23:33:45.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.744+0000 7f34c7e00640 1 --2- 172.21.15.94:0/758596789 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f34ac072670 0x7f34ac074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:45.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.744+0000 7f34c7e00640 1 -- 172.21.15.94:0/758596789 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f34c00937a0 con 0x7f34d00ae980 2024-09-10T23:33:45.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.744+0000 7f34c6a00640 1 --2- 172.21.15.94:0/758596789 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f34ac072670 0x7f34ac074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:45.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.748+0000 7f34c6a00640 1 --2- 172.21.15.94:0/758596789 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f34ac072670 0x7f34ac074b30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f34b80af000 tx=0x7f34b801b070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:45.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.748+0000 7f34c7e00640 1 -- 172.21.15.94:0/758596789 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f34c005e730 con 0x7f34d00ae980 2024-09-10T23:33:45.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.952+0000 7f34c7e00640 1 -- 172.21.15.94:0/758596789 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f34c005a9a0 con 0x7f34d00ae980 2024-09-10T23:33:45.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.956+0000 7f34d6400640 1 -- 172.21.15.94:0/758596789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34d00d28a0 con 0x7f34d00ae980 2024-09-10T23:33:45.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.956+0000 7f34c7e00640 1 -- 172.21.15.94:0/758596789 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5238 (secure 0 0 0) 0x7f34c0090650 con 0x7f34d00ae980 2024-09-10T23:33:45.962 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:45.962 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":17,"btime":"2024-09-10T23:33:39:925925+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4900,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/765435890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":765435890},{"type":"v1","addr":"172.21.15.181:6837","nonce":765435890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3766049154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3766049154},{"type":"v1","addr":"172.21.15.94:6834","nonce":3766049154}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":194555,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2464857180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2464857180},{"type":"v1","addr":"172.21.15.94:6835","nonce":2464857180}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:29.261378+0000","modified":"2024-09-10T23:33:32.284558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194558},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194558":{"gid":194558,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1569801556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1569801556},{"type":"v1","addr":"172.21.15.181:6836","nonce":1569801556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194558,"qdb_cluster":[194558]},"id":2}]} 2024-09-10T23:33:45.962 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 17 2024-09-10T23:33:45.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.960+0000 7f34d6400640 1 -- 172.21.15.94:0/758596789 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f34ac072670 msgr2=0x7f34ac074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.960+0000 7f34d6400640 1 --2- 172.21.15.94:0/758596789 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f34ac072670 0x7f34ac074b30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f34b80af000 tx=0x7f34b801b070 comp rx=0 tx=0).stop 2024-09-10T23:33:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.960+0000 7f34d6400640 1 -- 172.21.15.94:0/758596789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d00ae980 msgr2=0x7f34d0156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.960+0000 7f34d6400640 1 --2- 172.21.15.94:0/758596789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d00ae980 0x7f34d0156f40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f34c000d950 tx=0x7f34c000de10 comp rx=0 tx=0).stop 2024-09-10T23:33:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.960+0000 7f34d6400640 1 -- 172.21.15.94:0/758596789 shutdown_connections 2024-09-10T23:33:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.960+0000 7f34d6400640 1 --2- 172.21.15.94:0/758596789 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f34ac072670 0x7f34ac074b30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.960+0000 7f34d6400640 1 --2- 172.21.15.94:0/758596789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34d00b2ec0 0x7f34d0071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.960+0000 7f34d6400640 1 --2- 172.21.15.94:0/758596789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34d015bd30 0x7f34d00716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.960+0000 7f34d6400640 1 --2- 172.21.15.94:0/758596789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d00ae980 0x7f34d0156f40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.960+0000 7f34d6400640 1 -- 172.21.15.94:0/758596789 >> 172.21.15.94:0/758596789 conn(0x7f34d009aa70 msgr2=0x7f34d009b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.960+0000 7f34d6400640 1 -- 172.21.15.94:0/758596789 shutdown_connections 2024-09-10T23:33:45.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:45.960+0000 7f34d6400640 1 -- 172.21.15.94:0/758596789 wait complete. 2024-09-10T23:33:45.984 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:33:46.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.120+0000 7fe1d4600640 1 -- 172.21.15.94:0/783569315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc0d05e0 msgr2=0x7fe1cc0b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:46.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.120+0000 7fe1d4600640 1 --2- 172.21.15.94:0/783569315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc0d05e0 0x7fe1cc0b15c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc073820 tx=0x7fe1c401c9c0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.120+0000 7fe1d4600640 1 -- 172.21.15.94:0/783569315 shutdown_connections 2024-09-10T23:33:46.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.120+0000 7fe1d4600640 1 --2- 172.21.15.94:0/783569315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc0d05e0 0x7fe1cc0b15c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.120+0000 7fe1d4600640 1 --2- 172.21.15.94:0/783569315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1cc0b2bb0 0x7fe1cc0d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.120+0000 7fe1d4600640 1 --2- 172.21.15.94:0/783569315 >> v2:172.21.15.181:3300/0 conn(0x7fe1cc15f5c0 0x7fe1cc0cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.120+0000 7fe1d4600640 1 -- 172.21.15.94:0/783569315 >> 172.21.15.94:0/783569315 conn(0x7fe1cc0d1c60 msgr2=0x7fe1cc0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:46.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.120+0000 7fe1d4600640 1 -- 172.21.15.94:0/783569315 shutdown_connections 2024-09-10T23:33:46.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.120+0000 7fe1d4600640 1 -- 172.21.15.94:0/783569315 wait complete. 2024-09-10T23:33:46.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d4600640 1 Processor -- start 2024-09-10T23:33:46.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d4600640 1 -- start start 2024-09-10T23:33:46.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d4600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc0b2bb0 0x7fe1cc14fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:46.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d4600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1cc0d05e0 0x7fe1cc1590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:46.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d4600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1cc15f5c0 0x7fe1cc159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:46.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d4600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc0afc20 con 0x7fe1cc0b2bb0 2024-09-10T23:33:46.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d4600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc0afaa0 con 0x7fe1cc0d05e0 2024-09-10T23:33:46.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d4600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe1cc0afda0 con 0x7fe1cc15f5c0 2024-09-10T23:33:46.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d1c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc0b2bb0 0x7fe1cc14fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:46.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d1c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc0b2bb0 0x7fe1cc14fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45582/0 (socket says 172.21.15.94:45582) 2024-09-10T23:33:46.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d1c00640 1 -- 172.21.15.94:0/2982074510 learned_addr learned my addr 172.21.15.94:0/2982074510 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:46.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d2600640 1 --2- 172.21.15.94:0/2982074510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1cc15f5c0 0x7fe1cc159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:46.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d1200640 1 --2- 172.21.15.94:0/2982074510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1cc0d05e0 0x7fe1cc1590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:46.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d1c00640 1 -- 172.21.15.94:0/2982074510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1cc15f5c0 msgr2=0x7fe1cc159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:46.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d1c00640 1 --2- 172.21.15.94:0/2982074510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1cc15f5c0 0x7fe1cc159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d1c00640 1 -- 172.21.15.94:0/2982074510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1cc0d05e0 msgr2=0x7fe1cc1590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:46.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d1c00640 1 --2- 172.21.15.94:0/2982074510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1cc0d05e0 0x7fe1cc1590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d1c00640 1 -- 172.21.15.94:0/2982074510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc06fcf0 con 0x7fe1cc0b2bb0 2024-09-10T23:33:46.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d1c00640 1 --2- 172.21.15.94:0/2982074510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc0b2bb0 0x7fe1cc14fb10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000b440 tx=0x7fe1c000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:46.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1c9600640 1 -- 172.21.15.94:0/2982074510 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0015020 con 0x7fe1cc0b2bb0 2024-09-10T23:33:46.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1c9600640 1 -- 172.21.15.94:0/2982074510 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1c00119a0 con 0x7fe1cc0b2bb0 2024-09-10T23:33:46.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1c9600640 1 -- 172.21.15.94:0/2982074510 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0011df0 con 0x7fe1cc0b2bb0 2024-09-10T23:33:46.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d4600640 1 -- 172.21.15.94:0/2982074510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1cc06f450 con 0x7fe1cc0b2bb0 2024-09-10T23:33:46.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d4600640 1 -- 172.21.15.94:0/2982074510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1cc06fa10 con 0x7fe1cc0b2bb0 2024-09-10T23:33:46.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.124+0000 7fe1d4600640 1 -- 172.21.15.94:0/2982074510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1cc0b0350 con 0x7fe1cc0b2bb0 2024-09-10T23:33:46.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.128+0000 7fe1c9600640 1 -- 172.21.15.94:0/2982074510 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7fe1c001a400 con 0x7fe1cc0b2bb0 2024-09-10T23:33:46.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.128+0000 7fe1c9600640 1 --2- 172.21.15.94:0/2982074510 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe1ac072790 0x7fe1ac074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:46.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.128+0000 7fe1c9600640 1 -- 172.21.15.94:0/2982074510 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe1c0093b20 con 0x7fe1cc0b2bb0 2024-09-10T23:33:46.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.128+0000 7fe1d1200640 1 --2- 172.21.15.94:0/2982074510 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe1ac072790 0x7fe1ac074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:46.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.128+0000 7fe1d1200640 1 --2- 172.21.15.94:0/2982074510 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe1ac072790 0x7fe1ac074c50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8006fd0 tx=0x7fe1b8008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:46.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.132+0000 7fe1c9600640 1 -- 172.21.15.94:0/2982074510 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe1c001f020 con 0x7fe1cc0b2bb0 2024-09-10T23:33:46.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.300+0000 7fe1d4600640 1 -- 172.21.15.94:0/2982074510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1cc0749c0 con 0x7fe1cc0b2bb0 2024-09-10T23:33:46.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.300+0000 7fe1c9600640 1 -- 172.21.15.94:0/2982074510 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16657 (secure 0 0 0) 0x7fe1c000d040 con 0x7fe1cc0b2bb0 2024-09-10T23:33:46.305 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:46.305 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":33,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:33:30.886007+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:33:27.370128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T23:33:28.229333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:33:46.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.304+0000 7fe1d4600640 1 -- 172.21.15.94:0/2982074510 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe1ac072790 msgr2=0x7fe1ac074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:46.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.304+0000 7fe1d4600640 1 --2- 172.21.15.94:0/2982074510 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe1ac072790 0x7fe1ac074c50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8006fd0 tx=0x7fe1b8008040 comp rx=0 tx=0).stop 2024-09-10T23:33:46.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.304+0000 7fe1d4600640 1 -- 172.21.15.94:0/2982074510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc0b2bb0 msgr2=0x7fe1cc14fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:46.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.304+0000 7fe1d4600640 1 --2- 172.21.15.94:0/2982074510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc0b2bb0 0x7fe1cc14fb10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000b440 tx=0x7fe1c000b900 comp rx=0 tx=0).stop 2024-09-10T23:33:46.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.304+0000 7fe1d4600640 1 -- 172.21.15.94:0/2982074510 shutdown_connections 2024-09-10T23:33:46.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.304+0000 7fe1d4600640 1 --2- 172.21.15.94:0/2982074510 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe1ac072790 0x7fe1ac074c50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.304+0000 7fe1d4600640 1 --2- 172.21.15.94:0/2982074510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1cc0d05e0 0x7fe1cc1590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.304+0000 7fe1d4600640 1 --2- 172.21.15.94:0/2982074510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1cc15f5c0 0x7fe1cc159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.304+0000 7fe1d4600640 1 --2- 172.21.15.94:0/2982074510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc0b2bb0 0x7fe1cc14fb10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.304+0000 7fe1d4600640 1 -- 172.21.15.94:0/2982074510 >> 172.21.15.94:0/2982074510 conn(0x7fe1cc0d1c60 msgr2=0x7fe1cc15e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:46.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.304+0000 7fe1d4600640 1 -- 172.21.15.94:0/2982074510 shutdown_connections 2024-09-10T23:33:46.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.304+0000 7fe1d4600640 1 -- 172.21.15.94:0/2982074510 wait complete. 2024-09-10T23:33:46.327 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T23:33:46.327 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:33:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.464+0000 7f32dae00640 1 -- 172.21.15.94:0/403804466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d40b2bb0 msgr2=0x7f32d40d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.464+0000 7f32dae00640 1 --2- 172.21.15.94:0/403804466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d40b2bb0 0x7f32d40d0270 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009900 tx=0x7f32c001c7c0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.464+0000 7f32dae00640 1 -- 172.21.15.94:0/403804466 shutdown_connections 2024-09-10T23:33:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.464+0000 7f32dae00640 1 --2- 172.21.15.94:0/403804466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d40b2bb0 0x7f32d40d0270 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.464+0000 7f32dae00640 1 -- 172.21.15.94:0/403804466 >> 172.21.15.94:0/403804466 conn(0x7f32d40d1c60 msgr2=0x7f32d40d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.464+0000 7f32dae00640 1 -- 172.21.15.94:0/403804466 shutdown_connections 2024-09-10T23:33:46.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.464+0000 7f32dae00640 1 -- 172.21.15.94:0/403804466 wait complete. 2024-09-10T23:33:46.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.464+0000 7f32dae00640 1 Processor -- start 2024-09-10T23:33:46.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.464+0000 7f32dae00640 1 -- start start 2024-09-10T23:33:46.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32dae00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32d40af310 0x7f32d4148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:46.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32dae00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32d40b2bb0 0x7f32d4154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:46.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32dae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d415f5c0 0x7f32d40c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:46.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32dae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f32d416a130 con 0x7f32d415f5c0 2024-09-10T23:33:46.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32dae00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f32d4169fb0 con 0x7f32d40b2bb0 2024-09-10T23:33:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32dae00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f32d416a2b0 con 0x7f32d40af310 2024-09-10T23:33:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32d8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d415f5c0 0x7f32d40c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32d8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d415f5c0 0x7f32d40c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45602/0 (socket says 172.21.15.94:45602) 2024-09-10T23:33:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32d8e00640 1 -- 172.21.15.94:0/2366723277 learned_addr learned my addr 172.21.15.94:0/2366723277 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32d3400640 1 --2- 172.21.15.94:0/2366723277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32d40b2bb0 0x7f32d4154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32d3e00640 1 --2- 172.21.15.94:0/2366723277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32d40af310 0x7f32d4148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32d8e00640 1 -- 172.21.15.94:0/2366723277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32d40af310 msgr2=0x7f32d4148270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32d8e00640 1 --2- 172.21.15.94:0/2366723277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32d40af310 0x7f32d4148270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32d8e00640 1 -- 172.21.15.94:0/2366723277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32d40b2bb0 msgr2=0x7f32d4154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32d8e00640 1 --2- 172.21.15.94:0/2366723277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32d40b2bb0 0x7f32d4154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32d8e00640 1 -- 172.21.15.94:0/2366723277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d40c8590 con 0x7f32d415f5c0 2024-09-10T23:33:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32d3400640 1 --2- 172.21.15.94:0/2366723277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32d40b2bb0 0x7f32d4154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:46.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32d3e00640 1 --2- 172.21.15.94:0/2366723277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32d40af310 0x7f32d4148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:46.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32d8e00640 1 --2- 172.21.15.94:0/2366723277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d415f5c0 0x7f32d40c7650 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00c780 tx=0x7f32cc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:46.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32cbe00640 1 -- 172.21.15.94:0/2366723277 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc004b70 con 0x7f32d415f5c0 2024-09-10T23:33:46.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32cbe00640 1 -- 172.21.15.94:0/2366723277 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32cc004e00 con 0x7f32d415f5c0 2024-09-10T23:33:46.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32dae00640 1 -- 172.21.15.94:0/2366723277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d40c7fd0 con 0x7f32d415f5c0 2024-09-10T23:33:46.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32cbe00640 1 -- 172.21.15.94:0/2366723277 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc00aab0 con 0x7f32d415f5c0 2024-09-10T23:33:46.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32dae00640 1 -- 172.21.15.94:0/2366723277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d40c7130 con 0x7f32d415f5c0 2024-09-10T23:33:46.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32dae00640 1 -- 172.21.15.94:0/2366723277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d40af7a0 con 0x7f32d415f5c0 2024-09-10T23:33:46.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.468+0000 7f32cbe00640 1 -- 172.21.15.94:0/2366723277 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f32cc00ac50 con 0x7f32d415f5c0 2024-09-10T23:33:46.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.472+0000 7f32cbe00640 1 --2- 172.21.15.94:0/2366723277 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f32b40727c0 0x7f32b4074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:46.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.472+0000 7f32cbe00640 1 -- 172.21.15.94:0/2366723277 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f32cc0938e0 con 0x7f32d415f5c0 2024-09-10T23:33:46.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.472+0000 7f32d3e00640 1 --2- 172.21.15.94:0/2366723277 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f32b40727c0 0x7f32b4074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:46.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.472+0000 7f32d3e00640 1 --2- 172.21.15.94:0/2366723277 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f32b40727c0 0x7f32b4074c80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f32d415a4e0 tx=0x7f32c4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:46.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.476+0000 7f32cbe00640 1 -- 172.21.15.94:0/2366723277 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f32cc020070 con 0x7f32d415f5c0 2024-09-10T23:33:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.672+0000 7f32dae00640 1 -- 172.21.15.94:0/2366723277 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32d40749c0 con 0x7f32d415f5c0 2024-09-10T23:33:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.676+0000 7f32cbe00640 1 -- 172.21.15.94:0/2366723277 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5238 (secure 0 0 0) 0x7f32cc05ec30 con 0x7f32d415f5c0 2024-09-10T23:33:46.679 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:46.679 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":17,"btime":"2024-09-10T23:33:39:925925+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4900,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/765435890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":765435890},{"type":"v1","addr":"172.21.15.181:6837","nonce":765435890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3766049154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3766049154},{"type":"v1","addr":"172.21.15.94:6834","nonce":3766049154}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":194555,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2464857180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2464857180},{"type":"v1","addr":"172.21.15.94:6835","nonce":2464857180}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:29.261378+0000","modified":"2024-09-10T23:33:32.284558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194558},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194558":{"gid":194558,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1569801556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1569801556},{"type":"v1","addr":"172.21.15.181:6836","nonce":1569801556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194558,"qdb_cluster":[194558]},"id":2}]} 2024-09-10T23:33:46.679 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 17 2024-09-10T23:33:46.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.676+0000 7f32dae00640 1 -- 172.21.15.94:0/2366723277 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f32b40727c0 msgr2=0x7f32b4074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:46.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.676+0000 7f32dae00640 1 --2- 172.21.15.94:0/2366723277 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f32b40727c0 0x7f32b4074c80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f32d415a4e0 tx=0x7f32c4009040 comp rx=0 tx=0).stop 2024-09-10T23:33:46.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.676+0000 7f32dae00640 1 -- 172.21.15.94:0/2366723277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d415f5c0 msgr2=0x7f32d40c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:46.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.676+0000 7f32dae00640 1 --2- 172.21.15.94:0/2366723277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d415f5c0 0x7f32d40c7650 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00c780 tx=0x7f32cc00cc40 comp rx=0 tx=0).stop 2024-09-10T23:33:46.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.676+0000 7f32dae00640 1 -- 172.21.15.94:0/2366723277 shutdown_connections 2024-09-10T23:33:46.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.676+0000 7f32dae00640 1 --2- 172.21.15.94:0/2366723277 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f32b40727c0 0x7f32b4074c80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.676+0000 7f32dae00640 1 --2- 172.21.15.94:0/2366723277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32d40b2bb0 0x7f32d4154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.676+0000 7f32dae00640 1 --2- 172.21.15.94:0/2366723277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32d415f5c0 0x7f32d40c7650 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.676+0000 7f32dae00640 1 --2- 172.21.15.94:0/2366723277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32d40af310 0x7f32d4148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.676+0000 7f32dae00640 1 -- 172.21.15.94:0/2366723277 >> 172.21.15.94:0/2366723277 conn(0x7f32d40d1c60 msgr2=0x7f32d40b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:46.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.676+0000 7f32dae00640 1 -- 172.21.15.94:0/2366723277 shutdown_connections 2024-09-10T23:33:46.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.676+0000 7f32dae00640 1 -- 172.21.15.94:0/2366723277 wait complete. 2024-09-10T23:33:46.700 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:33:46.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.836+0000 7f005ae00640 1 -- 172.21.15.94:0/2947960815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054071a20 msgr2=0x7f005415f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.836+0000 7f005ae00640 1 --2- 172.21.15.94:0/2947960815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054071a20 0x7f005415f610 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f00540b6ca0 tx=0x7f004001c830 comp rx=0 tx=0).stop 2024-09-10T23:33:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.836+0000 7f005ae00640 1 -- 172.21.15.94:0/2947960815 shutdown_connections 2024-09-10T23:33:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.836+0000 7f005ae00640 1 --2- 172.21.15.94:0/2947960815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054071a20 0x7f005415f610 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.836+0000 7f005ae00640 1 --2- 172.21.15.94:0/2947960815 >> v2:172.21.15.181:3300/0 conn(0x7f005406cc20 0x7f005406d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.836+0000 7f005ae00640 1 -- 172.21.15.94:0/2947960815 >> 172.21.15.94:0/2947960815 conn(0x7f00540d0b70 msgr2=0x7f00540d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f005ae00640 1 -- 172.21.15.94:0/2947960815 shutdown_connections 2024-09-10T23:33:46.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f005ae00640 1 -- 172.21.15.94:0/2947960815 wait complete. 2024-09-10T23:33:46.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f005ae00640 1 Processor -- start 2024-09-10T23:33:46.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f005ae00640 1 -- start start 2024-09-10T23:33:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f005ae00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f005406cc20 0x7f005413e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f005ae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054071a20 0x7f00541f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f005ae00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00540d1690 0x7f005414b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f005ae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0054169130 con 0x7f0054071a20 2024-09-10T23:33:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f005ae00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0054168fb0 con 0x7f00540d1690 2024-09-10T23:33:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f005ae00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f00541692b0 con 0x7f005406cc20 2024-09-10T23:33:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f0053400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054071a20 0x7f00541f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f0053400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054071a20 0x7f00541f8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45616/0 (socket says 172.21.15.94:45616) 2024-09-10T23:33:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f0053400640 1 -- 172.21.15.94:0/364195568 learned_addr learned my addr 172.21.15.94:0/364195568 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:46.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f0053e00640 1 --2- 172.21.15.94:0/364195568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f005406cc20 0x7f005413e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:46.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f0058e00640 1 --2- 172.21.15.94:0/364195568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00540d1690 0x7f005414b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:46.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f0053400640 1 -- 172.21.15.94:0/364195568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f005406cc20 msgr2=0x7f005413e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:46.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f0053400640 1 --2- 172.21.15.94:0/364195568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f005406cc20 0x7f005413e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f0053400640 1 -- 172.21.15.94:0/364195568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00540d1690 msgr2=0x7f005414b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:46.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f0053400640 1 --2- 172.21.15.94:0/364195568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00540d1690 0x7f005414b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:46.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f0053400640 1 -- 172.21.15.94:0/364195568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00540b3c60 con 0x7f0054071a20 2024-09-10T23:33:46.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.840+0000 7f0053400640 1 --2- 172.21.15.94:0/364195568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054071a20 0x7f00541f8160 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f00400b0040 tx=0x7f00400a66c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:46.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.844+0000 7f004be00640 1 -- 172.21.15.94:0/364195568 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00400be070 con 0x7f0054071a20 2024-09-10T23:33:46.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.844+0000 7f004be00640 1 -- 172.21.15.94:0/364195568 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0040004290 con 0x7f0054071a20 2024-09-10T23:33:46.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.844+0000 7f004be00640 1 -- 172.21.15.94:0/364195568 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0040005030 con 0x7f0054071a20 2024-09-10T23:33:46.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.844+0000 7f005ae00640 1 -- 172.21.15.94:0/364195568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00540b3f40 con 0x7f0054071a20 2024-09-10T23:33:46.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.844+0000 7f005ae00640 1 -- 172.21.15.94:0/364195568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00541f7960 con 0x7f0054071a20 2024-09-10T23:33:46.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.844+0000 7f005ae00640 1 -- 172.21.15.94:0/364195568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f005406d000 con 0x7f0054071a20 2024-09-10T23:33:46.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.844+0000 7f004be00640 1 -- 172.21.15.94:0/364195568 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f00400051d0 con 0x7f0054071a20 2024-09-10T23:33:46.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.844+0000 7f004be00640 1 --2- 172.21.15.94:0/364195568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0034072790 0x7f0034074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:46.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.844+0000 7f004be00640 1 -- 172.21.15.94:0/364195568 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f004012d130 con 0x7f0054071a20 2024-09-10T23:33:46.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.844+0000 7f0053e00640 1 --2- 172.21.15.94:0/364195568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0034072790 0x7f0034074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:46.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.848+0000 7f0053e00640 1 --2- 172.21.15.94:0/364195568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0034072790 0x7f0034074c50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0044005d10 tx=0x7f004400b410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:46.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:46.848+0000 7f004be00640 1 -- 172.21.15.94:0/364195568 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f00400164b0 con 0x7f0054071a20 2024-09-10T23:33:47.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.056+0000 7f005ae00640 1 -- 172.21.15.94:0/364195568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f005414cc40 con 0x7f0054071a20 2024-09-10T23:33:47.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.056+0000 7f004be00640 1 -- 172.21.15.94:0/364195568 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5238 (secure 0 0 0) 0x7f00400f7e40 con 0x7f0054071a20 2024-09-10T23:33:47.061 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:47.061 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":17,"btime":"2024-09-10T23:33:39:925925+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4900,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/765435890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":765435890},{"type":"v1","addr":"172.21.15.181:6837","nonce":765435890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3766049154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3766049154},{"type":"v1","addr":"172.21.15.94:6834","nonce":3766049154}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":194555,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2464857180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2464857180},{"type":"v1","addr":"172.21.15.94:6835","nonce":2464857180}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:29.261378+0000","modified":"2024-09-10T23:33:32.284558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194558},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194558":{"gid":194558,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1569801556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1569801556},{"type":"v1","addr":"172.21.15.181:6836","nonce":1569801556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194558,"qdb_cluster":[194558]},"id":2}]} 2024-09-10T23:33:47.061 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 17 2024-09-10T23:33:47.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.056+0000 7f005ae00640 1 -- 172.21.15.94:0/364195568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0034072790 msgr2=0x7f0034074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:47.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.056+0000 7f005ae00640 1 --2- 172.21.15.94:0/364195568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0034072790 0x7f0034074c50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0044005d10 tx=0x7f004400b410 comp rx=0 tx=0).stop 2024-09-10T23:33:47.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.056+0000 7f005ae00640 1 -- 172.21.15.94:0/364195568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054071a20 msgr2=0x7f00541f8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:47.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.056+0000 7f005ae00640 1 --2- 172.21.15.94:0/364195568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054071a20 0x7f00541f8160 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f00400b0040 tx=0x7f00400a66c0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.056+0000 7f005ae00640 1 -- 172.21.15.94:0/364195568 shutdown_connections 2024-09-10T23:33:47.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.056+0000 7f005ae00640 1 --2- 172.21.15.94:0/364195568 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0034072790 0x7f0034074c50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.056+0000 7f005ae00640 1 --2- 172.21.15.94:0/364195568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00540d1690 0x7f005414b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.056+0000 7f005ae00640 1 --2- 172.21.15.94:0/364195568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054071a20 0x7f00541f8160 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.056+0000 7f005ae00640 1 --2- 172.21.15.94:0/364195568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f005406cc20 0x7f005413e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.056+0000 7f005ae00640 1 -- 172.21.15.94:0/364195568 >> 172.21.15.94:0/364195568 conn(0x7f00540d0b70 msgr2=0x7f00540d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:47.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.060+0000 7f005ae00640 1 -- 172.21.15.94:0/364195568 shutdown_connections 2024-09-10T23:33:47.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.060+0000 7f005ae00640 1 -- 172.21.15.94:0/364195568 wait complete. 2024-09-10T23:33:47.082 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:33:47.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.220+0000 7fdf24200640 1 -- 172.21.15.94:0/2678753979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf1c15bd30 msgr2=0x7fdf1c15c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:47.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.220+0000 7fdf24200640 1 --2- 172.21.15.94:0/2678753979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf1c15bd30 0x7fdf1c15c110 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009900 tx=0x7fdf1001c8e0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.220+0000 7fdf24200640 1 -- 172.21.15.94:0/2678753979 shutdown_connections 2024-09-10T23:33:47.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.220+0000 7fdf24200640 1 --2- 172.21.15.94:0/2678753979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf1c0d2290 0x7fdf1c0b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.220+0000 7fdf24200640 1 --2- 172.21.15.94:0/2678753979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf1c15bd30 0x7fdf1c15c110 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.220+0000 7fdf24200640 1 -- 172.21.15.94:0/2678753979 >> 172.21.15.94:0/2678753979 conn(0x7fdf1c09aa70 msgr2=0x7fdf1c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:47.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.220+0000 7fdf24200640 1 -- 172.21.15.94:0/2678753979 shutdown_connections 2024-09-10T23:33:47.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.220+0000 7fdf24200640 1 -- 172.21.15.94:0/2678753979 wait complete. 2024-09-10T23:33:47.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.220+0000 7fdf24200640 1 Processor -- start 2024-09-10T23:33:47.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.220+0000 7fdf24200640 1 -- start start 2024-09-10T23:33:47.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf24200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf1c0ae980 0x7fdf1c15df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:47.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf24200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf1c0d2290 0x7fdf1c1585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:47.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf24200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf1c15bd30 0x7fdf1c1596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:47.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf24200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1695a0 con 0x7fdf1c15bd30 2024-09-10T23:33:47.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf24200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c169420 con 0x7fdf1c0ae980 2024-09-10T23:33:47.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf24200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fdf1c169720 con 0x7fdf1c0d2290 2024-09-10T23:33:47.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf22200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf1c15bd30 0x7fdf1c1596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:47.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf22200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf1c15bd30 0x7fdf1c1596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45630/0 (socket says 172.21.15.94:45630) 2024-09-10T23:33:47.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf22200640 1 -- 172.21.15.94:0/631010404 learned_addr learned my addr 172.21.15.94:0/631010404 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:47.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf21800640 1 --2- 172.21.15.94:0/631010404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf1c0ae980 0x7fdf1c15df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:47.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf20e00640 1 --2- 172.21.15.94:0/631010404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf1c0d2290 0x7fdf1c1585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:47.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf22200640 1 -- 172.21.15.94:0/631010404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf1c0d2290 msgr2=0x7fdf1c1585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:47.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf22200640 1 --2- 172.21.15.94:0/631010404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf1c0d2290 0x7fdf1c1585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf22200640 1 -- 172.21.15.94:0/631010404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf1c0ae980 msgr2=0x7fdf1c15df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:47.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf22200640 1 --2- 172.21.15.94:0/631010404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf1c0ae980 0x7fdf1c15df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf22200640 1 -- 172.21.15.94:0/631010404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c159f50 con 0x7fdf1c15bd30 2024-09-10T23:33:47.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf22200640 1 --2- 172.21.15.94:0/631010404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf1c15bd30 0x7fdf1c1596f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400eaf0 tx=0x7fdf1400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:47.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf19600640 1 -- 172.21.15.94:0/631010404 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14004b30 con 0x7fdf1c15bd30 2024-09-10T23:33:47.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf19600640 1 -- 172.21.15.94:0/631010404 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf14004cf0 con 0x7fdf1c15bd30 2024-09-10T23:33:47.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf24200640 1 -- 172.21.15.94:0/631010404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c1577e0 con 0x7fdf1c15bd30 2024-09-10T23:33:47.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf19600640 1 -- 172.21.15.94:0/631010404 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14005280 con 0x7fdf1c15bd30 2024-09-10T23:33:47.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf24200640 1 -- 172.21.15.94:0/631010404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c157da0 con 0x7fdf1c15bd30 2024-09-10T23:33:47.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.224+0000 7fdf24200640 1 -- 172.21.15.94:0/631010404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1c0b1e70 con 0x7fdf1c15bd30 2024-09-10T23:33:47.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.228+0000 7fdf19600640 1 -- 172.21.15.94:0/631010404 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7fdf14005420 con 0x7fdf1c15bd30 2024-09-10T23:33:47.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.228+0000 7fdf19600640 1 --2- 172.21.15.94:0/631010404 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdefc0727c0 0x7fdefc074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:47.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.228+0000 7fdf19600640 1 -- 172.21.15.94:0/631010404 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdf140941d0 con 0x7fdf1c15bd30 2024-09-10T23:33:47.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.228+0000 7fdf21800640 1 --2- 172.21.15.94:0/631010404 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdefc0727c0 0x7fdefc074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:47.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.228+0000 7fdf21800640 1 --2- 172.21.15.94:0/631010404 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdefc0727c0 0x7fdefc074c80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdf100af000 tx=0x7fdf1001b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:47.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.232+0000 7fdf19600640 1 -- 172.21.15.94:0/631010404 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdf1405eec0 con 0x7fdf1c15bd30 2024-09-10T23:33:47.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.400+0000 7fdf24200640 1 -- 172.21.15.94:0/631010404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf1c0d0710 con 0x7fdf1c15bd30 2024-09-10T23:33:47.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.400+0000 7fdf19600640 1 -- 172.21.15.94:0/631010404 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16657 (secure 0 0 0) 0x7fdf140638f0 con 0x7fdf1c15bd30 2024-09-10T23:33:47.404 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:47.404 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":33,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:33:30.886007+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:33:27.370128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T23:33:28.229333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:33:47.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.400+0000 7fdf24200640 1 -- 172.21.15.94:0/631010404 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdefc0727c0 msgr2=0x7fdefc074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:47.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.400+0000 7fdf24200640 1 --2- 172.21.15.94:0/631010404 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdefc0727c0 0x7fdefc074c80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdf100af000 tx=0x7fdf1001b120 comp rx=0 tx=0).stop 2024-09-10T23:33:47.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.400+0000 7fdf24200640 1 -- 172.21.15.94:0/631010404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf1c15bd30 msgr2=0x7fdf1c1596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:47.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.400+0000 7fdf24200640 1 --2- 172.21.15.94:0/631010404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf1c15bd30 0x7fdf1c1596f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400eaf0 tx=0x7fdf1400c2d0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.404+0000 7fdf24200640 1 -- 172.21.15.94:0/631010404 shutdown_connections 2024-09-10T23:33:47.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.404+0000 7fdf24200640 1 --2- 172.21.15.94:0/631010404 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdefc0727c0 0x7fdefc074c80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.404+0000 7fdf24200640 1 --2- 172.21.15.94:0/631010404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf1c15bd30 0x7fdf1c1596f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.404+0000 7fdf24200640 1 --2- 172.21.15.94:0/631010404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf1c0d2290 0x7fdf1c1585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.404+0000 7fdf24200640 1 --2- 172.21.15.94:0/631010404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf1c0ae980 0x7fdf1c15df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.404+0000 7fdf24200640 1 -- 172.21.15.94:0/631010404 >> 172.21.15.94:0/631010404 conn(0x7fdf1c09aa70 msgr2=0x7fdf1c0af100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:47.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.404+0000 7fdf24200640 1 -- 172.21.15.94:0/631010404 shutdown_connections 2024-09-10T23:33:47.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.404+0000 7fdf24200640 1 -- 172.21.15.94:0/631010404 wait complete. 2024-09-10T23:33:47.427 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T23:33:47.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.564+0000 7f0826200640 1 -- 172.21.15.94:0/3959113719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0820070150 msgr2=0x7f0820070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:47.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.564+0000 7f0826200640 1 --2- 172.21.15.94:0/3959113719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0820070150 0x7f0820070530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0810009900 tx=0x7f081001c790 comp rx=0 tx=0).stop 2024-09-10T23:33:47.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.564+0000 7f0826200640 1 -- 172.21.15.94:0/3959113719 shutdown_connections 2024-09-10T23:33:47.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.564+0000 7f0826200640 1 --2- 172.21.15.94:0/3959113719 >> v2:172.21.15.181:3300/0 conn(0x7f08200b4070 0x7f082015a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.564+0000 7f0826200640 1 --2- 172.21.15.94:0/3959113719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0820070150 0x7f0820070530 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.564+0000 7f0826200640 1 -- 172.21.15.94:0/3959113719 >> 172.21.15.94:0/3959113719 conn(0x7f08200d0130 msgr2=0x7f08200d0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:47.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.564+0000 7f0826200640 1 -- 172.21.15.94:0/3959113719 shutdown_connections 2024-09-10T23:33:47.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.564+0000 7f0826200640 1 -- 172.21.15.94:0/3959113719 wait complete. 2024-09-10T23:33:47.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0826200640 1 Processor -- start 2024-09-10T23:33:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0826200640 1 -- start start 2024-09-10T23:33:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0826200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0820070150 0x7f082014f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0826200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0820071150 0x7f08201f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0826200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08200b4070 0x7f0820157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0826200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f082015e740 con 0x7f0820071150 2024-09-10T23:33:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0826200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f082015e5c0 con 0x7f08200b4070 2024-09-10T23:33:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0826200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f082015e8c0 con 0x7f0820070150 2024-09-10T23:33:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0817e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0820071150 0x7f08201f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0817e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0820071150 0x7f08201f7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45644/0 (socket says 172.21.15.94:45644) 2024-09-10T23:33:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0817e00640 1 -- 172.21.15.94:0/2018876663 learned_addr learned my addr 172.21.15.94:0/2018876663 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0824e00640 1 --2- 172.21.15.94:0/2018876663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0820070150 0x7f082014f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0825800640 1 --2- 172.21.15.94:0/2018876663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08200b4070 0x7f0820157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:47.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0817e00640 1 -- 172.21.15.94:0/2018876663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0820070150 msgr2=0x7f082014f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:47.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0817e00640 1 --2- 172.21.15.94:0/2018876663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0820070150 0x7f082014f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0817e00640 1 -- 172.21.15.94:0/2018876663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08200b4070 msgr2=0x7f0820157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:47.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0817e00640 1 --2- 172.21.15.94:0/2018876663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08200b4070 0x7f0820157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0817e00640 1 -- 172.21.15.94:0/2018876663 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08201fc2a0 con 0x7f0820071150 2024-09-10T23:33:47.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0817e00640 1 --2- 172.21.15.94:0/2018876663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0820071150 0x7f08201f7d60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f080800ea60 tx=0x7f080800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:47.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f081d600640 1 -- 172.21.15.94:0/2018876663 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080801b070 con 0x7f0820071150 2024-09-10T23:33:47.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f081d600640 1 -- 172.21.15.94:0/2018876663 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f080800ce30 con 0x7f0820071150 2024-09-10T23:33:47.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f081d600640 1 -- 172.21.15.94:0/2018876663 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808016650 con 0x7f0820071150 2024-09-10T23:33:47.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0826200640 1 -- 172.21.15.94:0/2018876663 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08201fb720 con 0x7f0820071150 2024-09-10T23:33:47.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.568+0000 7f0826200640 1 -- 172.21.15.94:0/2018876663 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08201fbce0 con 0x7f0820071150 2024-09-10T23:33:47.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.572+0000 7f0826200640 1 -- 172.21.15.94:0/2018876663 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0820070530 con 0x7f0820071150 2024-09-10T23:33:47.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.572+0000 7f081d600640 1 -- 172.21.15.94:0/2018876663 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f080800b040 con 0x7f0820071150 2024-09-10T23:33:47.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.572+0000 7f081d600640 1 --2- 172.21.15.94:0/2018876663 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f07fc072790 0x7f07fc074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:47.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.572+0000 7f081d600640 1 -- 172.21.15.94:0/2018876663 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0808093730 con 0x7f0820071150 2024-09-10T23:33:47.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.572+0000 7f0824e00640 1 --2- 172.21.15.94:0/2018876663 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f07fc072790 0x7f07fc074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:47.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.572+0000 7f0824e00640 1 --2- 172.21.15.94:0/2018876663 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f07fc072790 0x7f07fc074c50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f08100af000 tx=0x7f081001afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:47.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.576+0000 7f081d600640 1 -- 172.21.15.94:0/2018876663 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f080805e4d0 con 0x7f0820071150 2024-09-10T23:33:47.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.776+0000 7f0826200640 1 -- 172.21.15.94:0/2018876663 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f08200cc580 con 0x7f0820071150 2024-09-10T23:33:47.783 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-10T23:33:47.780+0000 7f4045a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:33:47.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.780+0000 7f081d600640 1 -- 172.21.15.94:0/2018876663 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f0808062f00 con 0x7f0820071150 2024-09-10T23:33:47.785 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T23:33:47.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.784+0000 7f0826200640 1 -- 172.21.15.94:0/2018876663 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f07fc072790 msgr2=0x7f07fc074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:47.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.784+0000 7f0826200640 1 --2- 172.21.15.94:0/2018876663 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f07fc072790 0x7f07fc074c50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f08100af000 tx=0x7f081001afd0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.784+0000 7f0826200640 1 -- 172.21.15.94:0/2018876663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0820071150 msgr2=0x7f08201f7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:47.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.784+0000 7f0826200640 1 --2- 172.21.15.94:0/2018876663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0820071150 0x7f08201f7d60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f080800ea60 tx=0x7f080800ef20 comp rx=0 tx=0).stop 2024-09-10T23:33:47.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.784+0000 7f0826200640 1 -- 172.21.15.94:0/2018876663 shutdown_connections 2024-09-10T23:33:47.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.784+0000 7f0826200640 1 --2- 172.21.15.94:0/2018876663 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f07fc072790 0x7f07fc074c50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.784+0000 7f0826200640 1 --2- 172.21.15.94:0/2018876663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08200b4070 0x7f0820157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.784+0000 7f0826200640 1 --2- 172.21.15.94:0/2018876663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0820071150 0x7f08201f7d60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.784+0000 7f0826200640 1 --2- 172.21.15.94:0/2018876663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0820070150 0x7f082014f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.784+0000 7f0826200640 1 -- 172.21.15.94:0/2018876663 >> 172.21.15.94:0/2018876663 conn(0x7f08200d0130 msgr2=0x7f08200cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:47.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.784+0000 7f0826200640 1 -- 172.21.15.94:0/2018876663 shutdown_connections 2024-09-10T23:33:47.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.784+0000 7f0826200640 1 -- 172.21.15.94:0/2018876663 wait complete. 2024-09-10T23:33:47.809 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T23:33:47.855 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:33:47.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.948+0000 7f0b96400640 1 -- 172.21.15.94:0/3454151228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006d260 msgr2=0x7f0b9015ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:47.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.948+0000 7f0b96400640 1 --2- 172.21.15.94:0/3454151228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006d260 0x7f0b9015ca10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0b900b0700 tx=0x7f0b7801c890 comp rx=0 tx=0).stop 2024-09-10T23:33:47.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.948+0000 7f0b96400640 1 -- 172.21.15.94:0/3454151228 shutdown_connections 2024-09-10T23:33:47.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.948+0000 7f0b96400640 1 --2- 172.21.15.94:0/3454151228 >> v2:172.21.15.181:3300/0 conn(0x7f0b9015cf50 0x7f0b90073920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.948+0000 7f0b96400640 1 --2- 172.21.15.94:0/3454151228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006d260 0x7f0b9015ca10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.948+0000 7f0b96400640 1 -- 172.21.15.94:0/3454151228 >> 172.21.15.94:0/3454151228 conn(0x7f0b900d0d80 msgr2=0x7f0b900d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:47.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.948+0000 7f0b96400640 1 -- 172.21.15.94:0/3454151228 shutdown_connections 2024-09-10T23:33:47.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.948+0000 7f0b96400640 1 -- 172.21.15.94:0/3454151228 wait complete. 2024-09-10T23:33:47.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.948+0000 7f0b96400640 1 Processor -- start 2024-09-10T23:33:47.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.948+0000 7f0b96400640 1 -- start start 2024-09-10T23:33:47.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b96400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006ce80 0x7f0b90153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:47.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b96400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b9006d260 0x7f0b90154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:47.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b96400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b90154b80 0x7f0b9013d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:47.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b96400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b90141890 con 0x7f0b9006ce80 2024-09-10T23:33:47.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b96400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0b90141710 con 0x7f0b9006d260 2024-09-10T23:33:47.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b96400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0b90141a10 con 0x7f0b90154b80 2024-09-10T23:33:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b8f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006ce80 0x7f0b90153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b8f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006ce80 0x7f0b90153ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45672/0 (socket says 172.21.15.94:45672) 2024-09-10T23:33:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b8f400640 1 -- 172.21.15.94:0/545616376 learned_addr learned my addr 172.21.15.94:0/545616376 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b86a00640 1 --2- 172.21.15.94:0/545616376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b9006d260 0x7f0b90154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b8fe00640 1 --2- 172.21.15.94:0/545616376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b90154b80 0x7f0b9013d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b8f400640 1 -- 172.21.15.94:0/545616376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b90154b80 msgr2=0x7f0b9013d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b8f400640 1 --2- 172.21.15.94:0/545616376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b90154b80 0x7f0b9013d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b8f400640 1 -- 172.21.15.94:0/545616376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b9006d260 msgr2=0x7f0b90154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b8f400640 1 --2- 172.21.15.94:0/545616376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b9006d260 0x7f0b90154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b8f400640 1 -- 172.21.15.94:0/545616376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b90154400 con 0x7f0b9006ce80 2024-09-10T23:33:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b8fe00640 1 --2- 172.21.15.94:0/545616376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b90154b80 0x7f0b9013d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b8f400640 1 --2- 172.21.15.94:0/545616376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006ce80 0x7f0b90153ec0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000c500 tx=0x7f0b8000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b87e00640 1 -- 172.21.15.94:0/545616376 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80017070 con 0x7f0b9006ce80 2024-09-10T23:33:47.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b87e00640 1 -- 172.21.15.94:0/545616376 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b800053c0 con 0x7f0b9006ce80 2024-09-10T23:33:47.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b96400640 1 -- 172.21.15.94:0/545616376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b901faea0 con 0x7f0b9006ce80 2024-09-10T23:33:47.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b87e00640 1 -- 172.21.15.94:0/545616376 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80012890 con 0x7f0b9006ce80 2024-09-10T23:33:47.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b96400640 1 -- 172.21.15.94:0/545616376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b901fb460 con 0x7f0b9006ce80 2024-09-10T23:33:47.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.952+0000 7f0b96400640 1 -- 172.21.15.94:0/545616376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b90141f30 con 0x7f0b9006ce80 2024-09-10T23:33:47.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.956+0000 7f0b87e00640 1 -- 172.21.15.94:0/545616376 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f0b80012a30 con 0x7f0b9006ce80 2024-09-10T23:33:47.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.956+0000 7f0b87e00640 1 --2- 172.21.15.94:0/545616376 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b6c072790 0x7f0b6c074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:47.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.956+0000 7f0b87e00640 1 -- 172.21.15.94:0/545616376 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f0b80092e40 con 0x7f0b9006ce80 2024-09-10T23:33:47.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.956+0000 7f0b86a00640 1 --2- 172.21.15.94:0/545616376 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b6c072790 0x7f0b6c074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:47.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.960+0000 7f0b86a00640 1 --2- 172.21.15.94:0/545616376 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b6c072790 0x7f0b6c074c50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0b780af000 tx=0x7f0b7801b0d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:47.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:47.960+0000 7f0b87e00640 1 -- 172.21.15.94:0/545616376 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0b80021080 con 0x7f0b9006ce80 2024-09-10T23:33:48.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.160+0000 7f0b96400640 1 -- 172.21.15.94:0/545616376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0b900736e0 con 0x7f0b9006ce80 2024-09-10T23:33:48.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.784+0000 7f0b87e00640 1 -- 172.21.15.94:0/545616376 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f0b9015c270 con 0x7f0b9006ce80 2024-09-10T23:33:48.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.784+0000 7f0b96400640 1 -- 172.21.15.94:0/545616376 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b6c072790 msgr2=0x7f0b6c074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:48.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.784+0000 7f0b96400640 1 --2- 172.21.15.94:0/545616376 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b6c072790 0x7f0b6c074c50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0b780af000 tx=0x7f0b7801b0d0 comp rx=0 tx=0).stop 2024-09-10T23:33:48.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.784+0000 7f0b96400640 1 -- 172.21.15.94:0/545616376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006ce80 msgr2=0x7f0b90153ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:48.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.784+0000 7f0b96400640 1 --2- 172.21.15.94:0/545616376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006ce80 0x7f0b90153ec0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000c500 tx=0x7f0b8000c9c0 comp rx=0 tx=0).stop 2024-09-10T23:33:48.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.784+0000 7f0b96400640 1 -- 172.21.15.94:0/545616376 shutdown_connections 2024-09-10T23:33:48.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.784+0000 7f0b96400640 1 --2- 172.21.15.94:0/545616376 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b6c072790 0x7f0b6c074c50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:48.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.784+0000 7f0b96400640 1 --2- 172.21.15.94:0/545616376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b9006d260 0x7f0b90154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:48.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.784+0000 7f0b96400640 1 --2- 172.21.15.94:0/545616376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b90154b80 0x7f0b9013d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:48.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.784+0000 7f0b96400640 1 --2- 172.21.15.94:0/545616376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006ce80 0x7f0b90153ec0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:48.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.784+0000 7f0b96400640 1 -- 172.21.15.94:0/545616376 >> 172.21.15.94:0/545616376 conn(0x7f0b900d0d80 msgr2=0x7f0b901405b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:48.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.784+0000 7f0b96400640 1 -- 172.21.15.94:0/545616376 shutdown_connections 2024-09-10T23:33:48.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.784+0000 7f0b96400640 1 -- 172.21.15.94:0/545616376 wait complete. 2024-09-10T23:33:48.811 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T23:33:48.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.952+0000 7f7682600640 1 -- 172.21.15.94:0/1006327264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f767c0b2bb0 msgr2=0x7f767c0d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:48.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.952+0000 7f7682600640 1 --2- 172.21.15.94:0/1006327264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f767c0b2bb0 0x7f767c0d0270 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7664009900 tx=0x7f766401c7c0 comp rx=0 tx=0).stop 2024-09-10T23:33:48.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.952+0000 7f7682600640 1 -- 172.21.15.94:0/1006327264 shutdown_connections 2024-09-10T23:33:48.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.952+0000 7f7682600640 1 --2- 172.21.15.94:0/1006327264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f767c0b2bb0 0x7f767c0d0270 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:48.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.952+0000 7f7682600640 1 -- 172.21.15.94:0/1006327264 >> 172.21.15.94:0/1006327264 conn(0x7f767c0d1c60 msgr2=0x7f767c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:48.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.952+0000 7f7682600640 1 -- 172.21.15.94:0/1006327264 shutdown_connections 2024-09-10T23:33:48.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.952+0000 7f7682600640 1 -- 172.21.15.94:0/1006327264 wait complete. 2024-09-10T23:33:48.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7682600640 1 Processor -- start 2024-09-10T23:33:48.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7682600640 1 -- start start 2024-09-10T23:33:48.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7682600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f767c0af310 0x7f767c153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:48.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7682600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f767c0b2bb0 0x7f767c15caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:48.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7682600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f767c15f5c0 0x7f767c15cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:48.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7682600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f767c16a130 con 0x7f767c0b2bb0 2024-09-10T23:33:48.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7682600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f767c169fb0 con 0x7f767c15f5c0 2024-09-10T23:33:48.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7682600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f767c16a2b0 con 0x7f767c0af310 2024-09-10T23:33:48.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7672a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f767c0b2bb0 0x7f767c15caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:48.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7672a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f767c0b2bb0 0x7f767c15caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45686/0 (socket says 172.21.15.94:45686) 2024-09-10T23:33:48.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7672a00640 1 -- 172.21.15.94:0/508325966 learned_addr learned my addr 172.21.15.94:0/508325966 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:48.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7672a00640 1 -- 172.21.15.94:0/508325966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f767c0af310 msgr2=0x7f767c153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:48.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f767be00640 1 --2- 172.21.15.94:0/508325966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f767c15f5c0 0x7f767c15cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:48.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f767b400640 1 --2- 172.21.15.94:0/508325966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f767c0af310 0x7f767c153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:48.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7672a00640 1 --2- 172.21.15.94:0/508325966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f767c0af310 0x7f767c153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:48.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7672a00640 1 -- 172.21.15.94:0/508325966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f767c15f5c0 msgr2=0x7f767c15cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:48.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7672a00640 1 --2- 172.21.15.94:0/508325966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f767c15f5c0 0x7f767c15cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:48.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7672a00640 1 -- 172.21.15.94:0/508325966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767c149ea0 con 0x7f767c0b2bb0 2024-09-10T23:33:48.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f767be00640 1 --2- 172.21.15.94:0/508325966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f767c15f5c0 0x7f767c15cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:48.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f767b400640 1 --2- 172.21.15.94:0/508325966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f767c0af310 0x7f767c153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:48.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7672a00640 1 --2- 172.21.15.94:0/508325966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f767c0b2bb0 0x7f767c15caa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f76640b0040 tx=0x7f76640a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:48.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7673e00640 1 -- 172.21.15.94:0/508325966 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76640b2040 con 0x7f767c0b2bb0 2024-09-10T23:33:48.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7673e00640 1 -- 172.21.15.94:0/508325966 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76640043c0 con 0x7f767c0b2bb0 2024-09-10T23:33:48.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7673e00640 1 -- 172.21.15.94:0/508325966 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664005290 con 0x7f767c0b2bb0 2024-09-10T23:33:48.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7682600640 1 -- 172.21.15.94:0/508325966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767c14a180 con 0x7f767c0b2bb0 2024-09-10T23:33:48.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7682600640 1 -- 172.21.15.94:0/508325966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f767c14a740 con 0x7f767c0b2bb0 2024-09-10T23:33:48.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.956+0000 7f7682600640 1 -- 172.21.15.94:0/508325966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f767c0af7a0 con 0x7f767c0b2bb0 2024-09-10T23:33:48.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.960+0000 7f7673e00640 1 -- 172.21.15.94:0/508325966 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f76640046d0 con 0x7f767c0b2bb0 2024-09-10T23:33:48.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.960+0000 7f7673e00640 1 --2- 172.21.15.94:0/508325966 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f76580727c0 0x7f7658074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:48.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.960+0000 7f7673e00640 1 -- 172.21.15.94:0/508325966 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f766412d830 con 0x7f767c0b2bb0 2024-09-10T23:33:48.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.960+0000 7f767b400640 1 --2- 172.21.15.94:0/508325966 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f76580727c0 0x7f7658074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:48.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.960+0000 7f767b400640 1 --2- 172.21.15.94:0/508325966 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f76580727c0 0x7f7658074c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f767c1fb530 tx=0x7f766c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:48.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:48.964+0000 7f7673e00640 1 -- 172.21.15.94:0/508325966 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f76640164d0 con 0x7f767c0b2bb0 2024-09-10T23:33:49.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:49.132+0000 7f7682600640 1 -- 172.21.15.94:0/508325966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f767c0749c0 con 0x7f767c0b2bb0 2024-09-10T23:33:49.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:49.972+0000 7f7673e00640 1 -- 172.21.15.94:0/508325966 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) ==== 181+0+0 (secure 0 0 0) 0x7f76640f8a80 con 0x7f767c0b2bb0 2024-09-10T23:33:49.976 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-09-10T23:33:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:49.972+0000 7f7682600640 1 -- 172.21.15.94:0/508325966 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f76580727c0 msgr2=0x7f7658074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:49.972+0000 7f7682600640 1 --2- 172.21.15.94:0/508325966 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f76580727c0 0x7f7658074c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f767c1fb530 tx=0x7f766c009040 comp rx=0 tx=0).stop 2024-09-10T23:33:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:49.972+0000 7f7682600640 1 -- 172.21.15.94:0/508325966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f767c0b2bb0 msgr2=0x7f767c15caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:49.972+0000 7f7682600640 1 --2- 172.21.15.94:0/508325966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f767c0b2bb0 0x7f767c15caa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f76640b0040 tx=0x7f76640a56f0 comp rx=0 tx=0).stop 2024-09-10T23:33:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:49.976+0000 7f7682600640 1 -- 172.21.15.94:0/508325966 shutdown_connections 2024-09-10T23:33:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:49.976+0000 7f7682600640 1 --2- 172.21.15.94:0/508325966 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f76580727c0 0x7f7658074c80 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f767c1fb530 tx=0x7f766c009040 comp rx=0 tx=0).stop 2024-09-10T23:33:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:49.976+0000 7f7682600640 1 --2- 172.21.15.94:0/508325966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f767c15f5c0 0x7f767c15cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:49.976+0000 7f7682600640 1 --2- 172.21.15.94:0/508325966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f767c0b2bb0 0x7f767c15caa0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:49.976+0000 7f7682600640 1 --2- 172.21.15.94:0/508325966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f767c0af310 0x7f767c153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:49.976+0000 7f7682600640 1 -- 172.21.15.94:0/508325966 >> 172.21.15.94:0/508325966 conn(0x7f767c0d1c60 msgr2=0x7f767c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:49.976+0000 7f7682600640 1 -- 172.21.15.94:0/508325966 shutdown_connections 2024-09-10T23:33:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:49.976+0000 7f7682600640 1 -- 172.21.15.94:0/508325966 wait complete. 2024-09-10T23:33:50.000 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T23:33:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.144+0000 7ff93a800640 1 -- 172.21.15.94:0/2813482838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340d4900 msgr2=0x7ff9340d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.144+0000 7ff93a800640 1 --2- 172.21.15.94:0/2813482838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340d4900 0x7ff9340d0ad0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009900 tx=0x7ff92c01f070 comp rx=0 tx=0).stop 2024-09-10T23:33:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.144+0000 7ff93a800640 1 -- 172.21.15.94:0/2813482838 shutdown_connections 2024-09-10T23:33:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.144+0000 7ff93a800640 1 --2- 172.21.15.94:0/2813482838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340d4900 0x7ff9340d0ad0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.144+0000 7ff93a800640 1 --2- 172.21.15.94:0/2813482838 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9340d19b0 0x7ff9340d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.144+0000 7ff93a800640 1 -- 172.21.15.94:0/2813482838 >> 172.21.15.94:0/2813482838 conn(0x7ff9340d3de0 msgr2=0x7ff9340d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:50.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.144+0000 7ff93a800640 1 -- 172.21.15.94:0/2813482838 shutdown_connections 2024-09-10T23:33:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.144+0000 7ff93a800640 1 -- 172.21.15.94:0/2813482838 wait complete. 2024-09-10T23:33:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff93a800640 1 Processor -- start 2024-09-10T23:33:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff93a800640 1 -- start start 2024-09-10T23:33:50.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff93a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340d19b0 0x7ff934158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:50.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff93a800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9340d4900 0x7ff9341498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:50.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff93a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff93415dec0 0x7ff93414a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:50.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff93a800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff9340b6db0 con 0x7ff9340d19b0 2024-09-10T23:33:50.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff93a800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff9340b6c30 con 0x7ff93415dec0 2024-09-10T23:33:50.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff93a800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff9340b6f30 con 0x7ff9340d4900 2024-09-10T23:33:50.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff933400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340d19b0 0x7ff934158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:50.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff933400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340d19b0 0x7ff934158c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45712/0 (socket says 172.21.15.94:45712) 2024-09-10T23:33:50.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff933400640 1 -- 172.21.15.94:0/2212703829 learned_addr learned my addr 172.21.15.94:0/2212703829 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:50.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff932a00640 1 --2- 172.21.15.94:0/2212703829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9340d4900 0x7ff9341498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:50.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff933e00640 1 --2- 172.21.15.94:0/2212703829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff93415dec0 0x7ff93414a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:50.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff933400640 1 -- 172.21.15.94:0/2212703829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9340d4900 msgr2=0x7ff9341498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:50.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff933400640 1 --2- 172.21.15.94:0/2212703829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9340d4900 0x7ff9341498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:50.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff933400640 1 -- 172.21.15.94:0/2212703829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff93415dec0 msgr2=0x7ff93414a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:50.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff933400640 1 --2- 172.21.15.94:0/2212703829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff93415dec0 0x7ff93414a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:50.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff933400640 1 -- 172.21.15.94:0/2212703829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9340cc9b0 con 0x7ff9340d19b0 2024-09-10T23:33:50.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff933e00640 1 --2- 172.21.15.94:0/2212703829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff93415dec0 0x7ff93414a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:50.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff933400640 1 --2- 172.21.15.94:0/2212703829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340d19b0 0x7ff934158c00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff92400c730 tx=0x7ff92400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:50.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff92b400640 1 -- 172.21.15.94:0/2212703829 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924003b80 con 0x7ff9340d19b0 2024-09-10T23:33:50.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff92b400640 1 -- 172.21.15.94:0/2212703829 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff924003d40 con 0x7ff9340d19b0 2024-09-10T23:33:50.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff93a800640 1 -- 172.21.15.94:0/2212703829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9341591e0 con 0x7ff9340d19b0 2024-09-10T23:33:50.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff92b400640 1 -- 172.21.15.94:0/2212703829 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92400aa50 con 0x7ff9340d19b0 2024-09-10T23:33:50.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.148+0000 7ff93a800640 1 -- 172.21.15.94:0/2212703829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff93415beb0 con 0x7ff9340d19b0 2024-09-10T23:33:50.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.152+0000 7ff93a800640 1 -- 172.21.15.94:0/2212703829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9340b73c0 con 0x7ff9340d19b0 2024-09-10T23:33:50.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.152+0000 7ff92b400640 1 -- 172.21.15.94:0/2212703829 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7ff92400abf0 con 0x7ff9340d19b0 2024-09-10T23:33:50.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.152+0000 7ff92b400640 1 --2- 172.21.15.94:0/2212703829 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff914072790 0x7ff914074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:50.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.152+0000 7ff92b400640 1 -- 172.21.15.94:0/2212703829 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5357+0+0 (secure 0 0 0) 0x7ff924093350 con 0x7ff9340d19b0 2024-09-10T23:33:50.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.152+0000 7ff932a00640 1 --2- 172.21.15.94:0/2212703829 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff914072790 0x7ff914074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:50.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.152+0000 7ff932a00640 1 --2- 172.21.15.94:0/2212703829 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff914072790 0x7ff914074c50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff9200058d0 tx=0x7ff920005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:50.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.156+0000 7ff92b400640 1 -- 172.21.15.94:0/2212703829 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff92405e7f0 con 0x7ff9340d19b0 2024-09-10T23:33:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.320+0000 7ff93a800640 1 -- 172.21.15.94:0/2212703829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff9340d1390 con 0x7ff9340d19b0 2024-09-10T23:33:50.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.980+0000 7ff92b400640 1 -- 172.21.15.94:0/2212703829 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) ==== 169+0+0 (secure 0 0 0) 0x7ff92405e0f0 con 0x7ff9340d19b0 2024-09-10T23:33:50.984 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-09-10T23:33:50.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.984+0000 7ff93a800640 1 -- 172.21.15.94:0/2212703829 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff914072790 msgr2=0x7ff914074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:50.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.984+0000 7ff93a800640 1 --2- 172.21.15.94:0/2212703829 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff914072790 0x7ff914074c50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff9200058d0 tx=0x7ff920005860 comp rx=0 tx=0).stop 2024-09-10T23:33:50.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.984+0000 7ff93a800640 1 -- 172.21.15.94:0/2212703829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340d19b0 msgr2=0x7ff934158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:50.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.984+0000 7ff93a800640 1 --2- 172.21.15.94:0/2212703829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340d19b0 0x7ff934158c00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff92400c730 tx=0x7ff92400cbf0 comp rx=0 tx=0).stop 2024-09-10T23:33:50.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.984+0000 7ff93a800640 1 -- 172.21.15.94:0/2212703829 shutdown_connections 2024-09-10T23:33:50.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.984+0000 7ff93a800640 1 --2- 172.21.15.94:0/2212703829 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff914072790 0x7ff914074c50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:50.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.984+0000 7ff93a800640 1 --2- 172.21.15.94:0/2212703829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff93415dec0 0x7ff93414a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:50.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.984+0000 7ff93a800640 1 --2- 172.21.15.94:0/2212703829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9340d4900 0x7ff9341498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:50.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.984+0000 7ff93a800640 1 --2- 172.21.15.94:0/2212703829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340d19b0 0x7ff934158c00 secure :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff92400c730 tx=0x7ff92400cbf0 comp rx=0 tx=0).stop 2024-09-10T23:33:50.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.984+0000 7ff93a800640 1 -- 172.21.15.94:0/2212703829 >> 172.21.15.94:0/2212703829 conn(0x7ff9340d3de0 msgr2=0x7ff9340b75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:50.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.984+0000 7ff93a800640 1 -- 172.21.15.94:0/2212703829 shutdown_connections 2024-09-10T23:33:50.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:50.984+0000 7ff93a800640 1 -- 172.21.15.94:0/2212703829 wait complete. 2024-09-10T23:33:51.009 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T23:33:51.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.156+0000 7f9efea00640 1 -- 172.21.15.94:0/528714795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9edc0052f0 msgr2=0x7f9edc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:51.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.156+0000 7f9efea00640 1 --2- 172.21.15.94:0/528714795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9edc0052f0 0x7f9edc00b9d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009eb0 tx=0x7f9ef001d5f0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.156+0000 7f9efea00640 1 -- 172.21.15.94:0/528714795 shutdown_connections 2024-09-10T23:33:51.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.156+0000 7f9efea00640 1 --2- 172.21.15.94:0/528714795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9edc0052f0 0x7f9edc00b9d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.156+0000 7f9efea00640 1 -- 172.21.15.94:0/528714795 >> 172.21.15.94:0/528714795 conn(0x7f9ef80d1c60 msgr2=0x7f9ef80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:51.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.156+0000 7f9efea00640 1 -- 172.21.15.94:0/528714795 shutdown_connections 2024-09-10T23:33:51.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.156+0000 7f9efea00640 1 -- 172.21.15.94:0/528714795 wait complete. 2024-09-10T23:33:51.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efea00640 1 Processor -- start 2024-09-10T23:33:51.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efea00640 1 -- start start 2024-09-10T23:33:51.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efea00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9edc0052f0 0x7f9ef8158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:51.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef81f8a50 0x7f9ef80b6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:51.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ef80b7340 0x7f9ef81e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:51.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efea00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef80cedf0 con 0x7f9ef80b7340 2024-09-10T23:33:51.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efea00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef80cec70 con 0x7f9ef81f8a50 2024-09-10T23:33:51.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efea00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef80cef70 con 0x7f9edc0052f0 2024-09-10T23:33:51.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ef80b7340 0x7f9ef81e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:51.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ef80b7340 0x7f9ef81e80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45730/0 (socket says 172.21.15.94:45730) 2024-09-10T23:33:51.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efca00640 1 -- 172.21.15.94:0/3776221101 learned_addr learned my addr 172.21.15.94:0/3776221101 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:51.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9eef400640 1 --2- 172.21.15.94:0/3776221101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef81f8a50 0x7f9ef80b6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:51.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9ef7e00640 1 --2- 172.21.15.94:0/3776221101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9edc0052f0 0x7f9ef8158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:51.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efca00640 1 -- 172.21.15.94:0/3776221101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9edc0052f0 msgr2=0x7f9ef8158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:51.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efca00640 1 --2- 172.21.15.94:0/3776221101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9edc0052f0 0x7f9ef8158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efca00640 1 -- 172.21.15.94:0/3776221101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef81f8a50 msgr2=0x7f9ef80b6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:51.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efca00640 1 --2- 172.21.15.94:0/3776221101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef81f8a50 0x7f9ef80b6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efca00640 1 -- 172.21.15.94:0/3776221101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef81f7fb0 con 0x7f9ef80b7340 2024-09-10T23:33:51.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efca00640 1 --2- 172.21.15.94:0/3776221101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ef80b7340 0x7f9ef81e80a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9ef001d420 tx=0x7f9ef0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:51.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9ef4c00640 1 -- 172.21.15.94:0/3776221101 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0009200 con 0x7f9ef80b7340 2024-09-10T23:33:51.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9ef4c00640 1 -- 172.21.15.94:0/3776221101 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ef00040c0 con 0x7f9ef80b7340 2024-09-10T23:33:51.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efea00640 1 -- 172.21.15.94:0/3776221101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef80b7880 con 0x7f9ef80b7340 2024-09-10T23:33:51.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9ef4c00640 1 -- 172.21.15.94:0/3776221101 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00042b0 con 0x7f9ef80b7340 2024-09-10T23:33:51.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.160+0000 7f9efea00640 1 -- 172.21.15.94:0/3776221101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef81f8530 con 0x7f9ef80b7340 2024-09-10T23:33:51.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.164+0000 7f9efea00640 1 -- 172.21.15.94:0/3776221101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef80b11e0 con 0x7f9ef80b7340 2024-09-10T23:33:51.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.164+0000 7f9ef4c00640 1 -- 172.21.15.94:0/3776221101 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f9ef0004450 con 0x7f9ef80b7340 2024-09-10T23:33:51.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.164+0000 7f9ef4c00640 1 --2- 172.21.15.94:0/3776221101 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9ed4072790 0x7f9ed4074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:51.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.164+0000 7f9ef4c00640 1 -- 172.21.15.94:0/3776221101 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f9ef012dfa0 con 0x7f9ef80b7340 2024-09-10T23:33:51.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.164+0000 7f9ef7e00640 1 --2- 172.21.15.94:0/3776221101 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9ed4072790 0x7f9ed4074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:51.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.168+0000 7f9ef7e00640 1 --2- 172.21.15.94:0/3776221101 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9ed4072790 0x7f9ed4074c50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8006fd0 tx=0x7f9ee80067b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:51.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.168+0000 7f9ef4c00640 1 -- 172.21.15.94:0/3776221101 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9ef0017300 con 0x7f9ef80b7340 2024-09-10T23:33:51.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.336+0000 7f9efea00640 1 -- 172.21.15.94:0/3776221101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f9ef80c8af0 con 0x7f9ef80b7340 2024-09-10T23:33:51.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.340+0000 7f9ef4c00640 1 -- 172.21.15.94:0/3776221101 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v29) ==== 71+0+0 (secure 0 0 0) 0x7f9ef00f8e70 con 0x7f9ef80b7340 2024-09-10T23:33:51.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.344+0000 7f9efea00640 1 -- 172.21.15.94:0/3776221101 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9ed4072790 msgr2=0x7f9ed4074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:51.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.344+0000 7f9efea00640 1 --2- 172.21.15.94:0/3776221101 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9ed4072790 0x7f9ed4074c50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8006fd0 tx=0x7f9ee80067b0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.344+0000 7f9efea00640 1 -- 172.21.15.94:0/3776221101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ef80b7340 msgr2=0x7f9ef81e80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:51.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.344+0000 7f9efea00640 1 --2- 172.21.15.94:0/3776221101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ef80b7340 0x7f9ef81e80a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9ef001d420 tx=0x7f9ef0003f60 comp rx=0 tx=0).stop 2024-09-10T23:33:51.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.344+0000 7f9efea00640 1 -- 172.21.15.94:0/3776221101 shutdown_connections 2024-09-10T23:33:51.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.344+0000 7f9efea00640 1 --2- 172.21.15.94:0/3776221101 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9ed4072790 0x7f9ed4074c50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.344+0000 7f9efea00640 1 --2- 172.21.15.94:0/3776221101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef81f8a50 0x7f9ef80b6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.344+0000 7f9efea00640 1 --2- 172.21.15.94:0/3776221101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ef80b7340 0x7f9ef81e80a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.344+0000 7f9efea00640 1 --2- 172.21.15.94:0/3776221101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9edc0052f0 0x7f9ef8158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.344+0000 7f9efea00640 1 -- 172.21.15.94:0/3776221101 >> 172.21.15.94:0/3776221101 conn(0x7f9ef80d1c60 msgr2=0x7f9ef8071650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:51.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.344+0000 7f9efea00640 1 -- 172.21.15.94:0/3776221101 shutdown_connections 2024-09-10T23:33:51.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.344+0000 7f9efea00640 1 -- 172.21.15.94:0/3776221101 wait complete. 2024-09-10T23:33:51.367 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-10T23:33:51.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.504+0000 7f5c15e00640 1 -- 172.21.15.94:0/3575940606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c1015bd30 msgr2=0x7f5c1015c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:51.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.504+0000 7f5c15e00640 1 --2- 172.21.15.94:0/3575940606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c1015bd30 0x7f5c1015c110 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5c100b7d20 tx=0x7f5c0001c8c0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.504+0000 7f5c15e00640 1 -- 172.21.15.94:0/3575940606 shutdown_connections 2024-09-10T23:33:51.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.504+0000 7f5c15e00640 1 --2- 172.21.15.94:0/3575940606 >> v2:172.21.15.181:3300/0 conn(0x7f5c100d2290 0x7f5c100b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.504+0000 7f5c15e00640 1 --2- 172.21.15.94:0/3575940606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c1015bd30 0x7f5c1015c110 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.504+0000 7f5c15e00640 1 -- 172.21.15.94:0/3575940606 >> 172.21.15.94:0/3575940606 conn(0x7f5c1009aa70 msgr2=0x7f5c1009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:51.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.504+0000 7f5c15e00640 1 -- 172.21.15.94:0/3575940606 shutdown_connections 2024-09-10T23:33:51.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.504+0000 7f5c15e00640 1 -- 172.21.15.94:0/3575940606 wait complete. 2024-09-10T23:33:51.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.504+0000 7f5c15e00640 1 Processor -- start 2024-09-10T23:33:51.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.504+0000 7f5c15e00640 1 -- start start 2024-09-10T23:33:51.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c15e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c100ae980 0x7f5c1014e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:51.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c15e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c100d2290 0x7f5c100c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c15e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c1015bd30 0x7f5c101565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c15e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5c10169550 con 0x7f5c100ae980 2024-09-10T23:33:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c15e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101693d0 con 0x7f5c100d2290 2024-09-10T23:33:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c15e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5c101696d0 con 0x7f5c1015bd30 2024-09-10T23:33:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c0ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c100ae980 0x7f5c1014e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c0ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c100ae980 0x7f5c1014e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45734/0 (socket says 172.21.15.94:45734) 2024-09-10T23:33:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c0ea00640 1 -- 172.21.15.94:0/3034077730 learned_addr learned my addr 172.21.15.94:0/3034077730 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c06000640 1 --2- 172.21.15.94:0/3034077730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c100d2290 0x7f5c100c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c0f400640 1 --2- 172.21.15.94:0/3034077730 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c1015bd30 0x7f5c101565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c0ea00640 1 -- 172.21.15.94:0/3034077730 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c1015bd30 msgr2=0x7f5c101565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c0ea00640 1 --2- 172.21.15.94:0/3034077730 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c1015bd30 0x7f5c101565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c0ea00640 1 -- 172.21.15.94:0/3034077730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c100d2290 msgr2=0x7f5c100c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:51.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c0ea00640 1 --2- 172.21.15.94:0/3034077730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c100d2290 0x7f5c100c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c0ea00640 1 -- 172.21.15.94:0/3034077730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c101fc4b0 con 0x7f5c100ae980 2024-09-10T23:33:51.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c0ea00640 1 --2- 172.21.15.94:0/3034077730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c100ae980 0x7f5c1014e460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5c000b0040 tx=0x7f5c00004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:51.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c07e00640 1 -- 172.21.15.94:0/3034077730 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c000b2040 con 0x7f5c100ae980 2024-09-10T23:33:51.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c07e00640 1 -- 172.21.15.94:0/3034077730 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c00005320 con 0x7f5c100ae980 2024-09-10T23:33:51.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c07e00640 1 -- 172.21.15.94:0/3034077730 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c000046d0 con 0x7f5c100ae980 2024-09-10T23:33:51.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c15e00640 1 -- 172.21.15.94:0/3034077730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c101fc790 con 0x7f5c100ae980 2024-09-10T23:33:51.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c15e00640 1 -- 172.21.15.94:0/3034077730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c100b4e40 con 0x7f5c100ae980 2024-09-10T23:33:51.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c15e00640 1 -- 172.21.15.94:0/3034077730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c100b1e70 con 0x7f5c100ae980 2024-09-10T23:33:51.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.508+0000 7f5c07e00640 1 -- 172.21.15.94:0/3034077730 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f5c00004870 con 0x7f5c100ae980 2024-09-10T23:33:51.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.512+0000 7f5c07e00640 1 --2- 172.21.15.94:0/3034077730 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5bec072790 0x7f5bec074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:51.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.512+0000 7f5c07e00640 1 -- 172.21.15.94:0/3034077730 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f5c0012cfe0 con 0x7f5c100ae980 2024-09-10T23:33:51.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.512+0000 7f5c06000640 1 --2- 172.21.15.94:0/3034077730 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5bec072790 0x7f5bec074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:51.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.512+0000 7f5c06000640 1 --2- 172.21.15.94:0/3034077730 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5bec072790 0x7f5bec074c50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5c1009fc80 tx=0x7f5bf8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:51.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.516+0000 7f5c07e00640 1 -- 172.21.15.94:0/3034077730 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5c000bc060 con 0x7f5c100ae980 2024-09-10T23:33:51.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.740+0000 7f5c15e00640 1 -- 172.21.15.94:0/3034077730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f5c1015a530 con 0x7f5c100ae980 2024-09-10T23:33:51.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.980+0000 7f5c07e00640 1 -- 172.21.15.94:0/3034077730 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v89) ==== 158+0+0 (secure 0 0 0) 0x7f5c000164b0 con 0x7f5c100ae980 2024-09-10T23:33:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.984+0000 7f5c15e00640 1 -- 172.21.15.94:0/3034077730 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5bec072790 msgr2=0x7f5bec074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.984+0000 7f5c15e00640 1 --2- 172.21.15.94:0/3034077730 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5bec072790 0x7f5bec074c50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5c1009fc80 tx=0x7f5bf8009040 comp rx=0 tx=0).stop 2024-09-10T23:33:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.984+0000 7f5c15e00640 1 -- 172.21.15.94:0/3034077730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c100ae980 msgr2=0x7f5c1014e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.984+0000 7f5c15e00640 1 --2- 172.21.15.94:0/3034077730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c100ae980 0x7f5c1014e460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5c000b0040 tx=0x7f5c00004600 comp rx=0 tx=0).stop 2024-09-10T23:33:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.984+0000 7f5c15e00640 1 -- 172.21.15.94:0/3034077730 shutdown_connections 2024-09-10T23:33:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.984+0000 7f5c15e00640 1 --2- 172.21.15.94:0/3034077730 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5bec072790 0x7f5bec074c50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.984+0000 7f5c15e00640 1 --2- 172.21.15.94:0/3034077730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c100d2290 0x7f5c100c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.984+0000 7f5c15e00640 1 --2- 172.21.15.94:0/3034077730 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c1015bd30 0x7f5c101565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.984+0000 7f5c15e00640 1 --2- 172.21.15.94:0/3034077730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c100ae980 0x7f5c1014e460 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.984+0000 7f5c15e00640 1 -- 172.21.15.94:0/3034077730 >> 172.21.15.94:0/3034077730 conn(0x7f5c1009aa70 msgr2=0x7f5c10071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:51.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.984+0000 7f5c15e00640 1 -- 172.21.15.94:0/3034077730 shutdown_connections 2024-09-10T23:33:51.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:51.984+0000 7f5c15e00640 1 -- 172.21.15.94:0/3034077730 wait complete. 2024-09-10T23:33:52.007 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2024-09-10T23:33:52.007 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T23:33:52.008 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:33:52.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.144+0000 7f4e54000640 1 -- 172.21.15.94:0/3685864366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e4c073950 msgr2=0x7f4e4c073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:52.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.144+0000 7f4e54000640 1 --2- 172.21.15.94:0/3685864366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e4c073950 0x7f4e4c073d50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400ad00 tx=0x7f4e4401df10 comp rx=0 tx=0).stop 2024-09-10T23:33:52.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.144+0000 7f4e54000640 1 -- 172.21.15.94:0/3685864366 shutdown_connections 2024-09-10T23:33:52.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.144+0000 7f4e54000640 1 --2- 172.21.15.94:0/3685864366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e4c073950 0x7f4e4c073d50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.144+0000 7f4e54000640 1 -- 172.21.15.94:0/3685864366 >> 172.21.15.94:0/3685864366 conn(0x7f4e4c06cdf0 msgr2=0x7f4e4c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:52.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.144+0000 7f4e54000640 1 -- 172.21.15.94:0/3685864366 shutdown_connections 2024-09-10T23:33:52.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.144+0000 7f4e54000640 1 -- 172.21.15.94:0/3685864366 wait complete. 2024-09-10T23:33:52.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e54000640 1 Processor -- start 2024-09-10T23:33:52.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e54000640 1 -- start start 2024-09-10T23:33:52.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e54000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e4c070700 0x7f4e4c13b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:52.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e54000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e4c073950 0x7f4e4c13c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:52.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e54000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e4c15db80 0x7f4e4c13c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:52.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e54000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c167ec0 con 0x7f4e4c070700 2024-09-10T23:33:52.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e54000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c167d40 con 0x7f4e4c15db80 2024-09-10T23:33:52.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e54000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f4e4c168040 con 0x7f4e4c073950 2024-09-10T23:33:52.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e51600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e4c070700 0x7f4e4c13b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:52.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e51600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e4c070700 0x7f4e4c13b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45736/0 (socket says 172.21.15.94:45736) 2024-09-10T23:33:52.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e51600640 1 -- 172.21.15.94:0/1628368195 learned_addr learned my addr 172.21.15.94:0/1628368195 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:52.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e52000640 1 --2- 172.21.15.94:0/1628368195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e4c15db80 0x7f4e4c13c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:52.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e50c00640 1 --2- 172.21.15.94:0/1628368195 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e4c073950 0x7f4e4c13c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:52.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e51600640 1 -- 172.21.15.94:0/1628368195 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e4c073950 msgr2=0x7f4e4c13c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:52.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e51600640 1 --2- 172.21.15.94:0/1628368195 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e4c073950 0x7f4e4c13c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e51600640 1 -- 172.21.15.94:0/1628368195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e4c15db80 msgr2=0x7f4e4c13c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:52.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e51600640 1 --2- 172.21.15.94:0/1628368195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e4c15db80 0x7f4e4c13c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e51600640 1 -- 172.21.15.94:0/1628368195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4c154e30 con 0x7f4e4c070700 2024-09-10T23:33:52.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e51600640 1 --2- 172.21.15.94:0/1628368195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e4c070700 0x7f4e4c13b9a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4e4000c440 tx=0x7f4e4000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:52.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e49600640 1 -- 172.21.15.94:0/1628368195 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40017070 con 0x7f4e4c070700 2024-09-10T23:33:52.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e49600640 1 -- 172.21.15.94:0/1628368195 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e40003600 con 0x7f4e4c070700 2024-09-10T23:33:52.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e54000640 1 -- 172.21.15.94:0/1628368195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4c155110 con 0x7f4e4c070700 2024-09-10T23:33:52.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e49600640 1 -- 172.21.15.94:0/1628368195 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40003a50 con 0x7f4e4c070700 2024-09-10T23:33:52.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.148+0000 7f4e54000640 1 -- 172.21.15.94:0/1628368195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e4c1556d0 con 0x7f4e4c070700 2024-09-10T23:33:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.152+0000 7f4e54000640 1 -- 172.21.15.94:0/1628368195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e4c073d50 con 0x7f4e4c070700 2024-09-10T23:33:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.152+0000 7f4e49600640 1 -- 172.21.15.94:0/1628368195 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f4e40003bf0 con 0x7f4e4c070700 2024-09-10T23:33:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.152+0000 7f4e49600640 1 --2- 172.21.15.94:0/1628368195 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4e2c072790 0x7f4e2c074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.152+0000 7f4e49600640 1 -- 172.21.15.94:0/1628368195 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f4e40092750 con 0x7f4e4c070700 2024-09-10T23:33:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.152+0000 7f4e50c00640 1 --2- 172.21.15.94:0/1628368195 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4e2c072790 0x7f4e2c074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:52.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.152+0000 7f4e50c00640 1 --2- 172.21.15.94:0/1628368195 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4e2c072790 0x7f4e2c074c50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0b1fb0 tx=0x7f4e38001440 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:52.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.156+0000 7f4e49600640 1 -- 172.21.15.94:0/1628368195 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4e4005d890 con 0x7f4e4c070700 2024-09-10T23:33:52.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.324+0000 7f4e54000640 1 -- 172.21.15.94:0/1628368195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e4c0cc0d0 con 0x7f4e4c070700 2024-09-10T23:33:52.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.324+0000 7f4e49600640 1 -- 172.21.15.94:0/1628368195 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+13022 (secure 0 0 0) 0x7f4e400622c0 con 0x7f4e4c070700 2024-09-10T23:33:52.328 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:52.328 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":36,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:33:50.976667+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6836/1569801556":"2024-09-11T23:33:47.783084+0000","172.21.15.181:6834/1569801556":"2024-09-11T23:33:47.783084+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:33:52.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.324+0000 7f4e54000640 1 -- 172.21.15.94:0/1628368195 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4e2c072790 msgr2=0x7f4e2c074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:52.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.324+0000 7f4e54000640 1 --2- 172.21.15.94:0/1628368195 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4e2c072790 0x7f4e2c074c50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0b1fb0 tx=0x7f4e38001440 comp rx=0 tx=0).stop 2024-09-10T23:33:52.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.324+0000 7f4e54000640 1 -- 172.21.15.94:0/1628368195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e4c070700 msgr2=0x7f4e4c13b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:52.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.324+0000 7f4e54000640 1 --2- 172.21.15.94:0/1628368195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e4c070700 0x7f4e4c13b9a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4e4000c440 tx=0x7f4e4000c900 comp rx=0 tx=0).stop 2024-09-10T23:33:52.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.324+0000 7f4e54000640 1 -- 172.21.15.94:0/1628368195 shutdown_connections 2024-09-10T23:33:52.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.324+0000 7f4e54000640 1 --2- 172.21.15.94:0/1628368195 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4e2c072790 0x7f4e2c074c50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.324+0000 7f4e54000640 1 --2- 172.21.15.94:0/1628368195 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e4c15db80 0x7f4e4c13c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.324+0000 7f4e54000640 1 --2- 172.21.15.94:0/1628368195 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e4c073950 0x7f4e4c13c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.324+0000 7f4e54000640 1 --2- 172.21.15.94:0/1628368195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e4c070700 0x7f4e4c13b9a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.324+0000 7f4e54000640 1 -- 172.21.15.94:0/1628368195 >> 172.21.15.94:0/1628368195 conn(0x7f4e4c06cdf0 msgr2=0x7f4e4c06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:52.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.324+0000 7f4e54000640 1 -- 172.21.15.94:0/1628368195 shutdown_connections 2024-09-10T23:33:52.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.324+0000 7f4e54000640 1 -- 172.21.15.94:0/1628368195 wait complete. 2024-09-10T23:33:52.351 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T23:32:31.372521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T23:33:52.351 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T23:33:52.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.492+0000 7f3583200640 1 -- 172.21.15.94:0/243056607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3560004b70 msgr2=0x7f3560004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:52.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.492+0000 7f3583200640 1 --2- 172.21.15.94:0/243056607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3560004b70 0x7f3560004f90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f356c0016f0 tx=0x7f356c01c750 comp rx=0 tx=0).stop 2024-09-10T23:33:52.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.492+0000 7f3583200640 1 -- 172.21.15.94:0/243056607 shutdown_connections 2024-09-10T23:33:52.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.492+0000 7f3583200640 1 --2- 172.21.15.94:0/243056607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3560004b70 0x7f3560004f90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.492+0000 7f3583200640 1 -- 172.21.15.94:0/243056607 >> 172.21.15.94:0/243056607 conn(0x7f357c0cfaf0 msgr2=0x7f357c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:52.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.492+0000 7f3583200640 1 -- 172.21.15.94:0/243056607 shutdown_connections 2024-09-10T23:33:52.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.492+0000 7f3583200640 1 -- 172.21.15.94:0/243056607 wait complete. 2024-09-10T23:33:52.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.492+0000 7f3583200640 1 Processor -- start 2024-09-10T23:33:52.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.492+0000 7f3583200640 1 -- start start 2024-09-10T23:33:52.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3583200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3560004b70 0x7f357c0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:52.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3583200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f357c1f6c60 0x7f357c1e7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:52.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3583200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f357c1e7560 0x7f357c1f95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:52.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3583200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f357c0ce870 con 0x7f357c1e7560 2024-09-10T23:33:52.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3583200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f357c158400 con 0x7f3560004b70 2024-09-10T23:33:52.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3583200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f357c0ce9f0 con 0x7f357c1f6c60 2024-09-10T23:33:52.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3581200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f357c1e7560 0x7f357c1f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:52.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3581200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f357c1e7560 0x7f357c1f95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45760/0 (socket says 172.21.15.94:45760) 2024-09-10T23:33:52.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3581200640 1 -- 172.21.15.94:0/2843835331 learned_addr learned my addr 172.21.15.94:0/2843835331 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:52.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f357be00640 1 --2- 172.21.15.94:0/2843835331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3560004b70 0x7f357c0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:52.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3573e00640 1 --2- 172.21.15.94:0/2843835331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f357c1f6c60 0x7f357c1e7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:52.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3581200640 1 -- 172.21.15.94:0/2843835331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f357c1f6c60 msgr2=0x7f357c1e7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:52.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3581200640 1 --2- 172.21.15.94:0/2843835331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f357c1f6c60 0x7f357c1e7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3581200640 1 -- 172.21.15.94:0/2843835331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3560004b70 msgr2=0x7f357c0b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:52.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3581200640 1 --2- 172.21.15.94:0/2843835331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3560004b70 0x7f357c0b50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3581200640 1 -- 172.21.15.94:0/2843835331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357c1f61a0 con 0x7f357c1e7560 2024-09-10T23:33:52.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3581200640 1 --2- 172.21.15.94:0/2843835331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f357c1e7560 0x7f357c1f95a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f357c069640 tx=0x7f357400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:52.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3578c00640 1 -- 172.21.15.94:0/2843835331 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357400ca20 con 0x7f357c1e7560 2024-09-10T23:33:52.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3578c00640 1 -- 172.21.15.94:0/2843835331 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3574017070 con 0x7f357c1e7560 2024-09-10T23:33:52.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3578c00640 1 -- 172.21.15.94:0/2843835331 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574011460 con 0x7f357c1e7560 2024-09-10T23:33:52.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3583200640 1 -- 172.21.15.94:0/2843835331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f357c1fa2d0 con 0x7f357c1e7560 2024-09-10T23:33:52.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3583200640 1 -- 172.21.15.94:0/2843835331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f357c1f6740 con 0x7f357c1e7560 2024-09-10T23:33:52.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3583200640 1 -- 172.21.15.94:0/2843835331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f357c1f6910 con 0x7f357c1e7560 2024-09-10T23:33:52.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.496+0000 7f3578c00640 1 -- 172.21.15.94:0/2843835331 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f3574011600 con 0x7f357c1e7560 2024-09-10T23:33:52.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.500+0000 7f3578c00640 1 --2- 172.21.15.94:0/2843835331 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3558072790 0x7f3558074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:52.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.500+0000 7f3578c00640 1 -- 172.21.15.94:0/2843835331 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f3574093740 con 0x7f357c1e7560 2024-09-10T23:33:52.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.500+0000 7f357be00640 1 --2- 172.21.15.94:0/2843835331 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3558072790 0x7f3558074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:52.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.500+0000 7f357be00640 1 --2- 172.21.15.94:0/2843835331 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3558072790 0x7f3558074c50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f356c0016c0 tx=0x7f356c019540 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:52.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.504+0000 7f3578c00640 1 -- 172.21.15.94:0/2843835331 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f357405e6a0 con 0x7f357c1e7560 2024-09-10T23:33:52.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.668+0000 7f3583200640 1 -- 172.21.15.94:0/2843835331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f357c0d0850 con 0x7f357c1e7560 2024-09-10T23:33:52.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.668+0000 7f3578c00640 1 -- 172.21.15.94:0/2843835331 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f3574067410 con 0x7f357c1e7560 2024-09-10T23:33:52.673 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-09-10T23:33:52.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.672+0000 7f3583200640 1 -- 172.21.15.94:0/2843835331 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3558072790 msgr2=0x7f3558074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:52.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.672+0000 7f3583200640 1 --2- 172.21.15.94:0/2843835331 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3558072790 0x7f3558074c50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f356c0016c0 tx=0x7f356c019540 comp rx=0 tx=0).stop 2024-09-10T23:33:52.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.672+0000 7f3583200640 1 -- 172.21.15.94:0/2843835331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f357c1e7560 msgr2=0x7f357c1f95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:52.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.672+0000 7f3583200640 1 --2- 172.21.15.94:0/2843835331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f357c1e7560 0x7f357c1f95a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f357c069640 tx=0x7f357400ecf0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.672+0000 7f3583200640 1 -- 172.21.15.94:0/2843835331 shutdown_connections 2024-09-10T23:33:52.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.672+0000 7f3583200640 1 --2- 172.21.15.94:0/2843835331 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3558072790 0x7f3558074c50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.672+0000 7f3583200640 1 --2- 172.21.15.94:0/2843835331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f357c1e7560 0x7f357c1f95a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.672+0000 7f3583200640 1 --2- 172.21.15.94:0/2843835331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f357c1f6c60 0x7f357c1e7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.672+0000 7f3583200640 1 --2- 172.21.15.94:0/2843835331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3560004b70 0x7f357c0b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.672+0000 7f3583200640 1 -- 172.21.15.94:0/2843835331 >> 172.21.15.94:0/2843835331 conn(0x7f357c0cfaf0 msgr2=0x7f357c0732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:52.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.672+0000 7f3583200640 1 -- 172.21.15.94:0/2843835331 shutdown_connections 2024-09-10T23:33:52.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.672+0000 7f3583200640 1 -- 172.21.15.94:0/2843835331 wait complete. 2024-09-10T23:33:52.695 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-10T23:33:52.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.832+0000 7fc41be00640 1 -- 172.21.15.94:0/4263139186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc41c0b30e0 msgr2=0x7fc41c0b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:52.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.832+0000 7fc41be00640 1 --2- 172.21.15.94:0/4263139186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc41c0b30e0 0x7fc41c0b1f40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc414009c80 tx=0x7fc41401cd70 comp rx=0 tx=0).stop 2024-09-10T23:33:52.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.832+0000 7fc41be00640 1 -- 172.21.15.94:0/4263139186 shutdown_connections 2024-09-10T23:33:52.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.832+0000 7fc41be00640 1 --2- 172.21.15.94:0/4263139186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc41c0b30e0 0x7fc41c0b1f40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.832+0000 7fc41be00640 1 --2- 172.21.15.94:0/4263139186 >> v2:172.21.15.181:3300/0 conn(0x7fc41c0d2170 0x7fc41c0b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.832+0000 7fc41be00640 1 -- 172.21.15.94:0/4263139186 >> 172.21.15.94:0/4263139186 conn(0x7fc41c09aa70 msgr2=0x7fc41c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:52.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.832+0000 7fc41be00640 1 -- 172.21.15.94:0/4263139186 shutdown_connections 2024-09-10T23:33:52.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.832+0000 7fc41be00640 1 -- 172.21.15.94:0/4263139186 wait complete. 2024-09-10T23:33:52.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41be00640 1 Processor -- start 2024-09-10T23:33:52.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41be00640 1 -- start start 2024-09-10T23:33:52.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41be00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc41c0b30e0 0x7fc41c165b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:52.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc41c0d2170 0x7fc41c1fafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:52.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc41c15bcd0 0x7fc41c1fb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:52.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c168ec0 con 0x7fc41c0d2170 2024-09-10T23:33:52.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41be00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c168d40 con 0x7fc41c15bcd0 2024-09-10T23:33:52.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41be00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc41c169040 con 0x7fc41c0b30e0 2024-09-10T23:33:52.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc41c0d2170 0x7fc41c1fafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:52.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc41c0d2170 0x7fc41c1fafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45774/0 (socket says 172.21.15.94:45774) 2024-09-10T23:33:52.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41a000640 1 -- 172.21.15.94:0/1955422230 learned_addr learned my addr 172.21.15.94:0/1955422230 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:52.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41b400640 1 --2- 172.21.15.94:0/1955422230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc41c15bcd0 0x7fc41c1fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:52.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41aa00640 1 --2- 172.21.15.94:0/1955422230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc41c0b30e0 0x7fc41c165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:52.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41a000640 1 -- 172.21.15.94:0/1955422230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc41c0b30e0 msgr2=0x7fc41c165b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:52.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41a000640 1 --2- 172.21.15.94:0/1955422230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc41c0b30e0 0x7fc41c165b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41a000640 1 -- 172.21.15.94:0/1955422230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc41c15bcd0 msgr2=0x7fc41c1fb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:52.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41a000640 1 --2- 172.21.15.94:0/1955422230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc41c15bcd0 0x7fc41c1fb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:52.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41a000640 1 -- 172.21.15.94:0/1955422230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41c06ddd0 con 0x7fc41c0d2170 2024-09-10T23:33:52.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41a000640 1 --2- 172.21.15.94:0/1955422230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc41c0d2170 0x7fc41c1fafe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc40800e9f0 tx=0x7fc40800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:52.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc412a00640 1 -- 172.21.15.94:0/1955422230 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40800cd80 con 0x7fc41c0d2170 2024-09-10T23:33:52.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc412a00640 1 -- 172.21.15.94:0/1955422230 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc408010430 con 0x7fc41c0d2170 2024-09-10T23:33:52.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc412a00640 1 -- 172.21.15.94:0/1955422230 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408010860 con 0x7fc41c0d2170 2024-09-10T23:33:52.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41be00640 1 -- 172.21.15.94:0/1955422230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41c14ffc0 con 0x7fc41c0d2170 2024-09-10T23:33:52.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41be00640 1 -- 172.21.15.94:0/1955422230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc41c150280 con 0x7fc41c0d2170 2024-09-10T23:33:52.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.836+0000 7fc41be00640 1 -- 172.21.15.94:0/1955422230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc41c0d1a60 con 0x7fc41c0d2170 2024-09-10T23:33:52.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.840+0000 7fc412a00640 1 -- 172.21.15.94:0/1955422230 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7fc408010a00 con 0x7fc41c0d2170 2024-09-10T23:33:52.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.840+0000 7fc412a00640 1 --2- 172.21.15.94:0/1955422230 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc3fc072790 0x7fc3fc074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:52.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.840+0000 7fc412a00640 1 -- 172.21.15.94:0/1955422230 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fc408093e80 con 0x7fc41c0d2170 2024-09-10T23:33:52.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.840+0000 7fc41aa00640 1 --2- 172.21.15.94:0/1955422230 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc3fc072790 0x7fc3fc074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.840+0000 7fc41aa00640 1 --2- 172.21.15.94:0/1955422230 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc3fc072790 0x7fc3fc074c50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc40c007fd0 tx=0x7fc40c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:52.844+0000 7fc412a00640 1 -- 172.21.15.94:0/1955422230 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc40805eff0 con 0x7fc41c0d2170 2024-09-10T23:33:53.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:53.072+0000 7fc41be00640 1 -- 172.21.15.94:0/1955422230 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7fc41c0c53a0 con 0x7fc41c0d2170 2024-09-10T23:33:53.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:53.988+0000 7fc412a00640 1 -- 172.21.15.94:0/1955422230 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v91) ==== 143+0+0 (secure 0 0 0) 0x7fc408063a20 con 0x7fc41c0d2170 2024-09-10T23:33:53.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:53.992+0000 7fc41be00640 1 -- 172.21.15.94:0/1955422230 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc3fc072790 msgr2=0x7fc3fc074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:53.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:53.992+0000 7fc41be00640 1 --2- 172.21.15.94:0/1955422230 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc3fc072790 0x7fc3fc074c50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc40c007fd0 tx=0x7fc40c009040 comp rx=0 tx=0).stop 2024-09-10T23:33:53.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:53.992+0000 7fc41be00640 1 -- 172.21.15.94:0/1955422230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc41c0d2170 msgr2=0x7fc41c1fafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:53.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:53.992+0000 7fc41be00640 1 --2- 172.21.15.94:0/1955422230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc41c0d2170 0x7fc41c1fafe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc40800e9f0 tx=0x7fc40800eeb0 comp rx=0 tx=0).stop 2024-09-10T23:33:53.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:53.992+0000 7fc41be00640 1 -- 172.21.15.94:0/1955422230 shutdown_connections 2024-09-10T23:33:53.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:53.992+0000 7fc41be00640 1 --2- 172.21.15.94:0/1955422230 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc3fc072790 0x7fc3fc074c50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:53.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:53.992+0000 7fc41be00640 1 --2- 172.21.15.94:0/1955422230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc41c15bcd0 0x7fc41c1fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:53.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:53.992+0000 7fc41be00640 1 --2- 172.21.15.94:0/1955422230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc41c0d2170 0x7fc41c1fafe0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:53.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:53.992+0000 7fc41be00640 1 --2- 172.21.15.94:0/1955422230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc41c0b30e0 0x7fc41c165b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:53.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:53.992+0000 7fc41be00640 1 -- 172.21.15.94:0/1955422230 >> 172.21.15.94:0/1955422230 conn(0x7fc41c09aa70 msgr2=0x7fc41c0ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:53.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:53.992+0000 7fc41be00640 1 -- 172.21.15.94:0/1955422230 shutdown_connections 2024-09-10T23:33:53.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:53.992+0000 7fc41be00640 1 -- 172.21.15.94:0/1955422230 wait complete. 2024-09-10T23:33:54.020 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T23:33:54.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.160+0000 7fb7edc00640 1 -- 172.21.15.94:0/2501071738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7cc0039f0 msgr2=0x7fb7cc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:54.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.160+0000 7fb7edc00640 1 --2- 172.21.15.94:0/2501071738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7cc0039f0 0x7fb7cc003dd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009b30 tx=0x7fb7d801d230 comp rx=0 tx=0).stop 2024-09-10T23:33:54.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.160+0000 7fb7ed200640 1 -- 172.21.15.94:0/2501071738 reap_dead start 2024-09-10T23:33:54.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.160+0000 7fb7edc00640 1 -- 172.21.15.94:0/2501071738 shutdown_connections 2024-09-10T23:33:54.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.160+0000 7fb7edc00640 1 -- 172.21.15.94:0/2501071738 >> 172.21.15.94:0/2501071738 conn(0x7fb7e809aa70 msgr2=0x7fb7e809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:54.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.160+0000 7fb7edc00640 1 -- 172.21.15.94:0/2501071738 shutdown_connections 2024-09-10T23:33:54.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.160+0000 7fb7edc00640 1 -- 172.21.15.94:0/2501071738 wait complete. 2024-09-10T23:33:54.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7edc00640 1 Processor -- start 2024-09-10T23:33:54.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7edc00640 1 -- start start 2024-09-10T23:33:54.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7edc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7cc0039f0 0x7fb7e8154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:54.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7edc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7e80c8440 0x7fb7e81f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:54.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7edc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7e815a0d0 0x7fb7e815a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:54.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7edc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e80b1f60 con 0x7fb7e815a0d0 2024-09-10T23:33:54.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7edc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e80b1de0 con 0x7fb7e80c8440 2024-09-10T23:33:54.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7edc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e80af540 con 0x7fb7cc0039f0 2024-09-10T23:33:54.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7ed200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7e815a0d0 0x7fb7e815a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:54.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7ed200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7e815a0d0 0x7fb7e815a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46852/0 (socket says 172.21.15.94:46852) 2024-09-10T23:33:54.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7ed200640 1 -- 172.21.15.94:0/769611370 learned_addr learned my addr 172.21.15.94:0/769611370 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:54.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7e7e00640 1 --2- 172.21.15.94:0/769611370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7cc0039f0 0x7fb7e8154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:54.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7df400640 1 --2- 172.21.15.94:0/769611370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7e80c8440 0x7fb7e81f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:54.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7ed200640 1 -- 172.21.15.94:0/769611370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7cc0039f0 msgr2=0x7fb7e8154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:54.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7ed200640 1 --2- 172.21.15.94:0/769611370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7cc0039f0 0x7fb7e8154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7ed200640 1 -- 172.21.15.94:0/769611370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7e80c8440 msgr2=0x7fb7e81f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:54.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7ed200640 1 --2- 172.21.15.94:0/769611370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7e80c8440 0x7fb7e81f2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7ed200640 1 -- 172.21.15.94:0/769611370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e80b4b90 con 0x7fb7e815a0d0 2024-09-10T23:33:54.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7ed200640 1 --2- 172.21.15.94:0/769611370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7e815a0d0 0x7fb7e815a550 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80c8d20 tx=0x7fb7e00064f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:54.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7e4c00640 1 -- 172.21.15.94:0/769611370 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0011900 con 0x7fb7e815a0d0 2024-09-10T23:33:54.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7e4c00640 1 -- 172.21.15.94:0/769611370 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7e0018070 con 0x7fb7e815a0d0 2024-09-10T23:33:54.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7edc00640 1 -- 172.21.15.94:0/769611370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e80b4e70 con 0x7fb7e815a0d0 2024-09-10T23:33:54.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7e4c00640 1 -- 172.21.15.94:0/769611370 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0011d80 con 0x7fb7e815a0d0 2024-09-10T23:33:54.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.164+0000 7fb7edc00640 1 -- 172.21.15.94:0/769611370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e80b5430 con 0x7fb7e815a0d0 2024-09-10T23:33:54.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.168+0000 7fb7edc00640 1 -- 172.21.15.94:0/769611370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e80cfda0 con 0x7fb7e815a0d0 2024-09-10T23:33:54.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.168+0000 7fb7e4c00640 1 -- 172.21.15.94:0/769611370 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7fb7e001d460 con 0x7fb7e815a0d0 2024-09-10T23:33:54.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.168+0000 7fb7e4c00640 1 --2- 172.21.15.94:0/769611370 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb7c40727c0 0x7fb7c4074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:54.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.168+0000 7fb7e4c00640 1 -- 172.21.15.94:0/769611370 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fb7e00939c0 con 0x7fb7e815a0d0 2024-09-10T23:33:54.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.168+0000 7fb7e7e00640 1 --2- 172.21.15.94:0/769611370 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb7c40727c0 0x7fb7c4074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:54.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.168+0000 7fb7e7e00640 1 --2- 172.21.15.94:0/769611370 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb7c40727c0 0x7fb7c4074c80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009b30 tx=0x7fb7d8019fd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:54.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.172+0000 7fb7e4c00640 1 -- 172.21.15.94:0/769611370 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb7e005e9b0 con 0x7fb7e815a0d0 2024-09-10T23:33:54.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.336+0000 7fb7edc00640 1 -- 172.21.15.94:0/769611370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fb7e80c8820 con 0x7fb7e815a0d0 2024-09-10T23:33:54.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.340+0000 7fb7e4c00640 1 -- 172.21.15.94:0/769611370 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v29) ==== 59+0+1998 (secure 0 0 0) 0x7fb7e00633e0 con 0x7fb7e815a0d0 2024-09-10T23:33:54.343 INFO:teuthology.orchestra.run.smithi094.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T23:33:54.343 INFO:teuthology.orchestra.run.smithi094.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T23:33:54.343 INFO:teuthology.orchestra.run.smithi094.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T23:33:54.343 INFO:teuthology.orchestra.run.smithi094.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T23:33:54.343 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3944.074506 2024-09-10T23:33:54.343 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3865.464527 2024-09-10T23:33:54.343 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3898.861024 2024-09-10T23:33:54.343 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4010.048529 2024-09-10T23:33:54.344 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4650.562436 2024-09-10T23:33:54.344 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4724.018776 2024-09-10T23:33:54.344 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5255.753663 2024-09-10T23:33:54.344 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4718.009443 2024-09-10T23:33:54.344 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds 20/20 2024-09-10T23:33:54.344 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T23:33:54.344 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_ms 2/2 2024-09-10T23:33:54.344 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_frag true 2024-09-10T23:33:54.344 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_scatterstat true 2024-09-10T23:33:54.344 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T23:33:54.344 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_verify_scatter true 2024-09-10T23:33:54.344 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T23:33:54.345 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T23:33:54.345 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T23:33:54.345 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced client_mount_timeout 600 2024-09-10T23:33:54.345 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_client 20/20 2024-09-10T23:33:54.345 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_ms 1/1 2024-09-10T23:33:54.345 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T23:33:54.345 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T23:33:54.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.340+0000 7fb7edc00640 1 -- 172.21.15.94:0/769611370 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb7c40727c0 msgr2=0x7fb7c4074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:54.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.340+0000 7fb7edc00640 1 --2- 172.21.15.94:0/769611370 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb7c40727c0 0x7fb7c4074c80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009b30 tx=0x7fb7d8019fd0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.340+0000 7fb7edc00640 1 -- 172.21.15.94:0/769611370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7e815a0d0 msgr2=0x7fb7e815a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:54.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.340+0000 7fb7edc00640 1 --2- 172.21.15.94:0/769611370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7e815a0d0 0x7fb7e815a550 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80c8d20 tx=0x7fb7e00064f0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.340+0000 7fb7edc00640 1 -- 172.21.15.94:0/769611370 shutdown_connections 2024-09-10T23:33:54.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.340+0000 7fb7edc00640 1 --2- 172.21.15.94:0/769611370 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb7c40727c0 0x7fb7c4074c80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.340+0000 7fb7edc00640 1 --2- 172.21.15.94:0/769611370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7e80c8440 0x7fb7e81f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.340+0000 7fb7edc00640 1 --2- 172.21.15.94:0/769611370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7e815a0d0 0x7fb7e815a550 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.340+0000 7fb7edc00640 1 --2- 172.21.15.94:0/769611370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7cc0039f0 0x7fb7e8154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.340+0000 7fb7edc00640 1 -- 172.21.15.94:0/769611370 >> 172.21.15.94:0/769611370 conn(0x7fb7e809aa70 msgr2=0x7fb7e80ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:54.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.340+0000 7fb7edc00640 1 -- 172.21.15.94:0/769611370 shutdown_connections 2024-09-10T23:33:54.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.340+0000 7fb7edc00640 1 -- 172.21.15.94:0/769611370 wait complete. 2024-09-10T23:33:54.364 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:33:54.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.508+0000 7f5129600640 1 -- 172.21.15.94:0/1250936772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51240c5a70 msgr2=0x7f51240cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:54.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.508+0000 7f5129600640 1 --2- 172.21.15.94:0/1250936772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51240c5a70 0x7f51240cd9f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f511c009c80 tx=0x7f511c01cd30 comp rx=0 tx=0).stop 2024-09-10T23:33:54.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.508+0000 7f5129600640 1 -- 172.21.15.94:0/1250936772 shutdown_connections 2024-09-10T23:33:54.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.508+0000 7f5129600640 1 --2- 172.21.15.94:0/1250936772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51240c5a70 0x7f51240cd9f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.508+0000 7f5129600640 1 --2- 172.21.15.94:0/1250936772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51240d0710 0x7f51240c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.508+0000 7f5129600640 1 --2- 172.21.15.94:0/1250936772 >> v2:172.21.15.181:3300/0 conn(0x7f51241455d0 0x7f51240c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.508+0000 7f5129600640 1 -- 172.21.15.94:0/1250936772 >> 172.21.15.94:0/1250936772 conn(0x7f51240cfaf0 msgr2=0x7f51240cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:54.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.508+0000 7f5129600640 1 -- 172.21.15.94:0/1250936772 shutdown_connections 2024-09-10T23:33:54.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.508+0000 7f5129600640 1 -- 172.21.15.94:0/1250936772 wait complete. 2024-09-10T23:33:54.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.508+0000 7f5129600640 1 Processor -- start 2024-09-10T23:33:54.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.508+0000 7f5129600640 1 -- start start 2024-09-10T23:33:54.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5129600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51240c5a70 0x7f512415a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:54.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5129600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51240d0710 0x7f51240b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:54.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5129600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51241455d0 0x7f51240b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:54.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5129600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f512415fe80 con 0x7f51240c5a70 2024-09-10T23:33:54.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5129600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f512415fd00 con 0x7f51241455d0 2024-09-10T23:33:54.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5129600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5124160000 con 0x7f51240d0710 2024-09-10T23:33:54.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5122a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51240c5a70 0x7f512415a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:54.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5122a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51240c5a70 0x7f512415a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46868/0 (socket says 172.21.15.94:46868) 2024-09-10T23:33:54.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5122a00640 1 -- 172.21.15.94:0/2403126309 learned_addr learned my addr 172.21.15.94:0/2403126309 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:54.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5123400640 1 --2- 172.21.15.94:0/2403126309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51241455d0 0x7f51240b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:54.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f511be00640 1 --2- 172.21.15.94:0/2403126309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51240d0710 0x7f51240b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:54.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5122a00640 1 -- 172.21.15.94:0/2403126309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51240d0710 msgr2=0x7f51240b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:54.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5122a00640 1 --2- 172.21.15.94:0/2403126309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51240d0710 0x7f51240b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5122a00640 1 -- 172.21.15.94:0/2403126309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51241455d0 msgr2=0x7f51240b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:54.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5122a00640 1 --2- 172.21.15.94:0/2403126309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51241455d0 0x7f51240b0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5122a00640 1 -- 172.21.15.94:0/2403126309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512415ad80 con 0x7f51240c5a70 2024-09-10T23:33:54.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5123400640 1 --2- 172.21.15.94:0/2403126309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51241455d0 0x7f51240b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:54.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f511be00640 1 --2- 172.21.15.94:0/2403126309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51240d0710 0x7f51240b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:33:54.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5122a00640 1 --2- 172.21.15.94:0/2403126309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51240c5a70 0x7f512415a840 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f511400c9f0 tx=0x7f511400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:54.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f511aa00640 1 -- 172.21.15.94:0/2403126309 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114007bb0 con 0x7f51240c5a70 2024-09-10T23:33:54.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f511aa00640 1 -- 172.21.15.94:0/2403126309 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5114007e40 con 0x7f51240c5a70 2024-09-10T23:33:54.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f511aa00640 1 -- 172.21.15.94:0/2403126309 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511400f690 con 0x7f51240c5a70 2024-09-10T23:33:54.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5129600640 1 -- 172.21.15.94:0/2403126309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5124142c70 con 0x7f51240c5a70 2024-09-10T23:33:54.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5129600640 1 -- 172.21.15.94:0/2403126309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51240c7ed0 con 0x7f51240c5a70 2024-09-10T23:33:54.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f5129600640 1 -- 172.21.15.94:0/2403126309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5124143060 con 0x7f51240c5a70 2024-09-10T23:33:54.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.512+0000 7f511aa00640 1 -- 172.21.15.94:0/2403126309 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f511400f830 con 0x7f51240c5a70 2024-09-10T23:33:54.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.516+0000 7f511aa00640 1 --2- 172.21.15.94:0/2403126309 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f51000727c0 0x7f5100074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:54.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.516+0000 7f511aa00640 1 -- 172.21.15.94:0/2403126309 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f5114093b70 con 0x7f51240c5a70 2024-09-10T23:33:54.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.516+0000 7f511be00640 1 --2- 172.21.15.94:0/2403126309 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f51000727c0 0x7f5100074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:54.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.516+0000 7f511be00640 1 --2- 172.21.15.94:0/2403126309 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f51000727c0 0x7f5100074c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f51240ae9a0 tx=0x7f510c005d50 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:54.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.520+0000 7f511aa00640 1 -- 172.21.15.94:0/2403126309 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5114014030 con 0x7f51240c5a70 2024-09-10T23:33:54.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.684+0000 7f5129600640 1 -- 172.21.15.94:0/2403126309 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f51241605b0 con 0x7f51240c5a70 2024-09-10T23:33:54.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.688+0000 7f511aa00640 1 -- 172.21.15.94:0/2403126309 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f511405f1b0 con 0x7f51240c5a70 2024-09-10T23:33:54.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.692+0000 7f5129600640 1 -- 172.21.15.94:0/2403126309 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f51000727c0 msgr2=0x7f5100074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:54.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.692+0000 7f5129600640 1 --2- 172.21.15.94:0/2403126309 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f51000727c0 0x7f5100074c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f51240ae9a0 tx=0x7f510c005d50 comp rx=0 tx=0).stop 2024-09-10T23:33:54.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.692+0000 7f5129600640 1 -- 172.21.15.94:0/2403126309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51240c5a70 msgr2=0x7f512415a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:54.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.692+0000 7f5129600640 1 --2- 172.21.15.94:0/2403126309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51240c5a70 0x7f512415a840 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f511400c9f0 tx=0x7f511400ceb0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.692+0000 7f5129600640 1 -- 172.21.15.94:0/2403126309 shutdown_connections 2024-09-10T23:33:54.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.692+0000 7f5129600640 1 --2- 172.21.15.94:0/2403126309 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f51000727c0 0x7f5100074c80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.692+0000 7f5129600640 1 --2- 172.21.15.94:0/2403126309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51241455d0 0x7f51240b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.692+0000 7f5129600640 1 --2- 172.21.15.94:0/2403126309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51240d0710 0x7f51240b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.692+0000 7f5129600640 1 --2- 172.21.15.94:0/2403126309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51240c5a70 0x7f512415a840 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:54.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.692+0000 7f5129600640 1 -- 172.21.15.94:0/2403126309 >> 172.21.15.94:0/2403126309 conn(0x7f51240cfaf0 msgr2=0x7f512415db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:54.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.692+0000 7f5129600640 1 -- 172.21.15.94:0/2403126309 shutdown_connections 2024-09-10T23:33:54.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:54.692+0000 7f5129600640 1 -- 172.21.15.94:0/2403126309 wait complete. 2024-09-10T23:33:54.715 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:33:54.715 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:33:54.797 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:33:54.897 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:33:54.931 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:33:54.931 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:54.939 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:33:54.939 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:33:54.939 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:54.949 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:33:54.949 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:33:54.949 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:54.994 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:33:54.994 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:33:54.994 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:33:55.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:33:55.003 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T23:33:55.003 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:33:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b9600640 1 -- 172.21.15.94:0/2058443972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12b415bd30 msgr2=0x7f12b415c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b9600640 1 --2- 172.21.15.94:0/2058443972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12b415bd30 0x7f12b415c110 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009c80 tx=0x7f12a401cd10 comp rx=0 tx=0).stop 2024-09-10T23:33:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b9600640 1 -- 172.21.15.94:0/2058443972 shutdown_connections 2024-09-10T23:33:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b9600640 1 --2- 172.21.15.94:0/2058443972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12b40ae980 0x7f12b40d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b9600640 1 --2- 172.21.15.94:0/2058443972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12b415bd30 0x7f12b415c110 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b9600640 1 -- 172.21.15.94:0/2058443972 >> 172.21.15.94:0/2058443972 conn(0x7f12b409aa70 msgr2=0x7f12b409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b9600640 1 -- 172.21.15.94:0/2058443972 shutdown_connections 2024-09-10T23:33:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b9600640 1 -- 172.21.15.94:0/2058443972 wait complete. 2024-09-10T23:33:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b9600640 1 Processor -- start 2024-09-10T23:33:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b9600640 1 -- start start 2024-09-10T23:33:55.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b9600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12b40ae980 0x7f12b41573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:55.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b9600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12b40b2ec0 0x7f12b4159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:55.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b9600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12b415bd30 0x7f12b406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:55.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b9600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f12b41692a0 con 0x7f12b40ae980 2024-09-10T23:33:55.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b9600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f12b4169120 con 0x7f12b415bd30 2024-09-10T23:33:55.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b9600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f12b4169420 con 0x7f12b40b2ec0 2024-09-10T23:33:55.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12b40ae980 0x7f12b41573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:55.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12b40ae980 0x7f12b41573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46892/0 (socket says 172.21.15.94:46892) 2024-09-10T23:33:55.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b2a00640 1 -- 172.21.15.94:0/846638354 learned_addr learned my addr 172.21.15.94:0/846638354 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:55.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12aa000640 1 --2- 172.21.15.94:0/846638354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12b40b2ec0 0x7f12b4159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:55.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.172+0000 7f12b3400640 1 --2- 172.21.15.94:0/846638354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12b415bd30 0x7f12b406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:55.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.176+0000 7f12b2a00640 1 -- 172.21.15.94:0/846638354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12b40b2ec0 msgr2=0x7f12b4159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:55.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.176+0000 7f12b2a00640 1 --2- 172.21.15.94:0/846638354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12b40b2ec0 0x7f12b4159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:55.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.176+0000 7f12b2a00640 1 -- 172.21.15.94:0/846638354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12b415bd30 msgr2=0x7f12b406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:55.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.176+0000 7f12b2a00640 1 --2- 172.21.15.94:0/846638354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12b415bd30 0x7f12b406ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:55.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.176+0000 7f12b2a00640 1 -- 172.21.15.94:0/846638354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b4157910 con 0x7f12b40ae980 2024-09-10T23:33:55.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.176+0000 7f12b2a00640 1 --2- 172.21.15.94:0/846638354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12b40ae980 0x7f12b41573d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009bd0 tx=0x7f12a40092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:55.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.176+0000 7f12ab400640 1 -- 172.21.15.94:0/846638354 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a40b2040 con 0x7f12b40ae980 2024-09-10T23:33:55.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.176+0000 7f12ab400640 1 -- 172.21.15.94:0/846638354 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12a4004200 con 0x7f12b40ae980 2024-09-10T23:33:55.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.176+0000 7f12ab400640 1 -- 172.21.15.94:0/846638354 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a40045d0 con 0x7f12b40ae980 2024-09-10T23:33:55.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.176+0000 7f12b9600640 1 -- 172.21.15.94:0/846638354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b41596a0 con 0x7f12b40ae980 2024-09-10T23:33:55.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.176+0000 7f12b9600640 1 -- 172.21.15.94:0/846638354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b41f9290 con 0x7f12b40ae980 2024-09-10T23:33:55.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.176+0000 7f12b9600640 1 -- 172.21.15.94:0/846638354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b40d1b80 con 0x7f12b40ae980 2024-09-10T23:33:55.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.176+0000 7f12ab400640 1 -- 172.21.15.94:0/846638354 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f12a40b5070 con 0x7f12b40ae980 2024-09-10T23:33:55.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.180+0000 7f12ab400640 1 --2- 172.21.15.94:0/846638354 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1290072790 0x7f1290074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:55.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.180+0000 7f12ab400640 1 -- 172.21.15.94:0/846638354 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f12a412d6a0 con 0x7f12b40ae980 2024-09-10T23:33:55.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.180+0000 7f12aa000640 1 --2- 172.21.15.94:0/846638354 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1290072790 0x7f1290074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:55.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.180+0000 7f12aa000640 1 --2- 172.21.15.94:0/846638354 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1290072790 0x7f1290074c50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f129c009be0 tx=0x7f129c0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:55.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.184+0000 7f12ab400640 1 -- 172.21.15.94:0/846638354 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f12b41596a0 con 0x7f12b40ae980 2024-09-10T23:33:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.380+0000 7f12b9600640 1 -- 172.21.15.94:0/846638354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12b4157b00 con 0x7f12b40ae980 2024-09-10T23:33:55.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.384+0000 7f12ab400640 1 -- 172.21.15.94:0/846638354 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3916 (secure 0 0 0) 0x7f12a4016990 con 0x7f12b40ae980 2024-09-10T23:33:55.387 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:55.387 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"btime":"2024-09-10T23:33:48:785555+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4900,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/765435890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":765435890},{"type":"v1","addr":"172.21.15.181:6837","nonce":765435890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3766049154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3766049154},{"type":"v1","addr":"172.21.15.94:6834","nonce":3766049154}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2397725399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2397725399},{"type":"v1","addr":"172.21.15.181:6836","nonce":2397725399}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":194555,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2464857180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2464857180},{"type":"v1","addr":"172.21.15.94:6835","nonce":2464857180}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-09-10T23:33:55.387 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 19 2024-09-10T23:33:55.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.384+0000 7f12b9600640 1 -- 172.21.15.94:0/846638354 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1290072790 msgr2=0x7f1290074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:55.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.384+0000 7f12b9600640 1 --2- 172.21.15.94:0/846638354 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1290072790 0x7f1290074c50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f129c009be0 tx=0x7f129c0091e0 comp rx=0 tx=0).stop 2024-09-10T23:33:55.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.384+0000 7f12b9600640 1 -- 172.21.15.94:0/846638354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12b40ae980 msgr2=0x7f12b41573d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:55.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.384+0000 7f12b9600640 1 --2- 172.21.15.94:0/846638354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12b40ae980 0x7f12b41573d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009bd0 tx=0x7f12a40092c0 comp rx=0 tx=0).stop 2024-09-10T23:33:55.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.384+0000 7f12b9600640 1 -- 172.21.15.94:0/846638354 shutdown_connections 2024-09-10T23:33:55.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.384+0000 7f12b9600640 1 --2- 172.21.15.94:0/846638354 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1290072790 0x7f1290074c50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:55.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.384+0000 7f12b9600640 1 --2- 172.21.15.94:0/846638354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12b415bd30 0x7f12b406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:55.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.384+0000 7f12b9600640 1 --2- 172.21.15.94:0/846638354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12b40b2ec0 0x7f12b4159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:55.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.384+0000 7f12b9600640 1 --2- 172.21.15.94:0/846638354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12b40ae980 0x7f12b41573d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:55.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.384+0000 7f12b9600640 1 -- 172.21.15.94:0/846638354 >> 172.21.15.94:0/846638354 conn(0x7f12b409aa70 msgr2=0x7f12b415b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:55.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.384+0000 7f12b9600640 1 -- 172.21.15.94:0/846638354 shutdown_connections 2024-09-10T23:33:55.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.384+0000 7f12b9600640 1 -- 172.21.15.94:0/846638354 wait complete. 2024-09-10T23:33:55.408 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T23:33:55.408 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T23:33:55.408 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T23:33:55.408 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:33:55.409 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T23:33:55.409 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T23:33:55.409 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T23:33:55.409 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:33:55.409 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T23:33:55.409 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T23:33:55.409 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T23:33:55.410 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:33:55.410 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T23:33:55.410 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T23:33:55.410 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T23:33:55.410 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:33:55.422 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T23:33:55.422 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T23:33:55.423 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T23:33:55.423 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T23:33:55.424 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T23:33:55.424 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T23:33:55.425 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T23:33:55.425 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T23:33:55.468 INFO:tasks.ceph.mds.b:Started 2024-09-10T23:33:55.469 DEBUG:teuthology.parallel:result is None 2024-09-10T23:33:55.469 INFO:tasks.ceph.mds.a:Started 2024-09-10T23:33:55.469 DEBUG:teuthology.parallel:result is None 2024-09-10T23:33:55.470 INFO:tasks.ceph.mds.d:Started 2024-09-10T23:33:55.470 DEBUG:teuthology.parallel:result is None 2024-09-10T23:33:55.470 INFO:tasks.ceph.mds.c:Started 2024-09-10T23:33:55.471 DEBUG:teuthology.parallel:result is None 2024-09-10T23:33:55.471 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T23:33:55.555 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-09-10T23:33:55.556 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-09-10T23:33:55.558 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:33:55.559 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2024-09-10T23:33:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 -- 172.21.15.94:0/3921675341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a00b1100 msgr2=0x7fc7a00b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 --2- 172.21.15.94:0/3921675341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a00b1100 0x7fc7a00b15b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0073df0 tx=0x7fc79801c800 comp rx=0 tx=0).stop 2024-09-10T23:33:55.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 -- 172.21.15.94:0/3921675341 shutdown_connections 2024-09-10T23:33:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 --2- 172.21.15.94:0/3921675341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a00b1100 0x7fc7a00b15b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 --2- 172.21.15.94:0/3921675341 >> v2:172.21.15.181:3300/0 conn(0x7fc7a00ce1e0 0x7fc7a00cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 --2- 172.21.15.94:0/3921675341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7a00ce890 0x7fc7a00cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 -- 172.21.15.94:0/3921675341 >> 172.21.15.94:0/3921675341 conn(0x7fc7a00d1c60 msgr2=0x7fc7a00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 -- 172.21.15.94:0/3921675341 shutdown_connections 2024-09-10T23:33:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 -- 172.21.15.94:0/3921675341 wait complete. 2024-09-10T23:33:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 Processor -- start 2024-09-10T23:33:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 -- start start 2024-09-10T23:33:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a00b1100 0x7fc7a0158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7a00ce1e0 0x7fc7a01fa110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7a00ce890 0x7fc7a01fa650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a00cfbc0 con 0x7fc7a00b1100 2024-09-10T23:33:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a00cfa40 con 0x7fc7a00ce890 2024-09-10T23:33:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc7a5800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a00cfd40 con 0x7fc7a00ce1e0 2024-09-10T23:33:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc79ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a00b1100 0x7fc7a0158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc79ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a00b1100 0x7fc7a0158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46936/0 (socket says 172.21.15.94:46936) 2024-09-10T23:33:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc79ea00640 1 -- 172.21.15.94:0/2725077854 learned_addr learned my addr 172.21.15.94:0/2725077854 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc796000640 1 --2- 172.21.15.94:0/2725077854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7a00ce1e0 0x7fc7a01fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc79f400640 1 --2- 172.21.15.94:0/2725077854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7a00ce890 0x7fc7a01fa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.616+0000 7fc79ea00640 1 -- 172.21.15.94:0/2725077854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7a00ce1e0 msgr2=0x7fc7a01fa110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.620+0000 7fc79ea00640 1 --2- 172.21.15.94:0/2725077854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7a00ce1e0 0x7fc7a01fa110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.620+0000 7fc79ea00640 1 -- 172.21.15.94:0/2725077854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7a00ce890 msgr2=0x7fc7a01fa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.620+0000 7fc79ea00640 1 --2- 172.21.15.94:0/2725077854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7a00ce890 0x7fc7a01fa650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.620+0000 7fc79ea00640 1 -- 172.21.15.94:0/2725077854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a0157a30 con 0x7fc7a00b1100 2024-09-10T23:33:55.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.620+0000 7fc79f400640 1 --2- 172.21.15.94:0/2725077854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7a00ce890 0x7fc7a01fa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:55.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.620+0000 7fc79ea00640 1 --2- 172.21.15.94:0/2725077854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a00b1100 0x7fc7a0158a20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc79000dac0 tx=0x7fc79000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:55.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.620+0000 7fc797400640 1 -- 172.21.15.94:0/2725077854 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790017070 con 0x7fc7a00b1100 2024-09-10T23:33:55.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.620+0000 7fc797400640 1 -- 172.21.15.94:0/2725077854 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc790004b30 con 0x7fc7a00b1100 2024-09-10T23:33:55.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.620+0000 7fc797400640 1 -- 172.21.15.94:0/2725077854 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790005020 con 0x7fc7a00b1100 2024-09-10T23:33:55.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.620+0000 7fc7a5800640 1 -- 172.21.15.94:0/2725077854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a01590e0 con 0x7fc7a00b1100 2024-09-10T23:33:55.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.620+0000 7fc7a5800640 1 -- 172.21.15.94:0/2725077854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a01596a0 con 0x7fc7a00b1100 2024-09-10T23:33:55.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.620+0000 7fc7a5800640 1 -- 172.21.15.94:0/2725077854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a00cec70 con 0x7fc7a00b1100 2024-09-10T23:33:55.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.620+0000 7fc797400640 1 -- 172.21.15.94:0/2725077854 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7fc7900051c0 con 0x7fc7a00b1100 2024-09-10T23:33:55.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.620+0000 7fc797400640 1 --2- 172.21.15.94:0/2725077854 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc77c0727c0 0x7fc77c074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:55.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.620+0000 7fc797400640 1 -- 172.21.15.94:0/2725077854 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fc790093800 con 0x7fc7a00b1100 2024-09-10T23:33:55.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.624+0000 7fc796000640 1 --2- 172.21.15.94:0/2725077854 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc77c0727c0 0x7fc77c074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:55.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.624+0000 7fc796000640 1 --2- 172.21.15.94:0/2725077854 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc77c0727c0 0x7fc77c074c80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc788005fd0 tx=0x7fc788005e60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:55.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.624+0000 7fc797400640 1 -- 172.21.15.94:0/2725077854 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc790020070 con 0x7fc7a00b1100 2024-09-10T23:33:55.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:55.804+0000 7fc7a5800640 1 -- 172.21.15.94:0/2725077854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc7a00b1610 con 0x7fc7a00b1100 2024-09-10T23:33:56.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.008+0000 7fc797400640 1 -- 172.21.15.94:0/2725077854 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v38) ==== 97+0+0 (secure 0 0 0) 0x7fc79005ee40 con 0x7fc7a00b1100 2024-09-10T23:33:56.014 INFO:teuthology.orchestra.run.smithi094.stderr: removed all blocklist entries 2024-09-10T23:33:56.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.012+0000 7fc7a5800640 1 -- 172.21.15.94:0/2725077854 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc77c0727c0 msgr2=0x7fc77c074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:56.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.012+0000 7fc7a5800640 1 --2- 172.21.15.94:0/2725077854 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc77c0727c0 0x7fc77c074c80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc788005fd0 tx=0x7fc788005e60 comp rx=0 tx=0).stop 2024-09-10T23:33:56.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.012+0000 7fc7a5800640 1 -- 172.21.15.94:0/2725077854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a00b1100 msgr2=0x7fc7a0158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:56.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.012+0000 7fc7a5800640 1 --2- 172.21.15.94:0/2725077854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a00b1100 0x7fc7a0158a20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc79000dac0 tx=0x7fc79000df80 comp rx=0 tx=0).stop 2024-09-10T23:33:56.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.012+0000 7fc7a5800640 1 -- 172.21.15.94:0/2725077854 shutdown_connections 2024-09-10T23:33:56.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.012+0000 7fc7a5800640 1 --2- 172.21.15.94:0/2725077854 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc77c0727c0 0x7fc77c074c80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.012+0000 7fc7a5800640 1 --2- 172.21.15.94:0/2725077854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7a00ce890 0x7fc7a01fa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.012+0000 7fc7a5800640 1 --2- 172.21.15.94:0/2725077854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7a00ce1e0 0x7fc7a01fa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.012+0000 7fc7a5800640 1 --2- 172.21.15.94:0/2725077854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a00b1100 0x7fc7a0158a20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.012+0000 7fc7a5800640 1 -- 172.21.15.94:0/2725077854 >> 172.21.15.94:0/2725077854 conn(0x7fc7a00d1c60 msgr2=0x7fc7a0070840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:56.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.012+0000 7fc7a5800640 1 -- 172.21.15.94:0/2725077854 shutdown_connections 2024-09-10T23:33:56.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.012+0000 7fc7a5800640 1 -- 172.21.15.94:0/2725077854 wait complete. 2024-09-10T23:33:56.037 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T23:33:56.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95b1600640 1 -- 172.21.15.94:0/3595884841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac0d2290 msgr2=0x7f95ac0b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:56.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95b1600640 1 --2- 172.21.15.94:0/3595884841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac0d2290 0x7f95ac0b1e70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0cf6b0 tx=0x7f95a401c8e0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95b1600640 1 -- 172.21.15.94:0/3595884841 shutdown_connections 2024-09-10T23:33:56.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95b1600640 1 --2- 172.21.15.94:0/3595884841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac0d2290 0x7f95ac0b1e70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95b1600640 1 --2- 172.21.15.94:0/3595884841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95ac15bd30 0x7f95ac15c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95b1600640 1 -- 172.21.15.94:0/3595884841 >> 172.21.15.94:0/3595884841 conn(0x7f95ac09aa70 msgr2=0x7f95ac09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:56.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95b1600640 1 -- 172.21.15.94:0/3595884841 shutdown_connections 2024-09-10T23:33:56.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95b1600640 1 -- 172.21.15.94:0/3595884841 wait complete. 2024-09-10T23:33:56.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95b1600640 1 Processor -- start 2024-09-10T23:33:56.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95b1600640 1 -- start start 2024-09-10T23:33:56.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95b1600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95ac0ae980 0x7f95ac15e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:56.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95b1600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95ac0d2290 0x7f95ac1655e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:56.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95b1600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac15bd30 0x7f95ac165b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:56.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95b1600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac169570 con 0x7f95ac15bd30 2024-09-10T23:33:56.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95b1600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1693f0 con 0x7f95ac0ae980 2024-09-10T23:33:56.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95b1600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f95ac1696f0 con 0x7f95ac0d2290 2024-09-10T23:33:56.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95ab400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac15bd30 0x7f95ac165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:56.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95ab400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac15bd30 0x7f95ac165b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46960/0 (socket says 172.21.15.94:46960) 2024-09-10T23:33:56.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95ab400640 1 -- 172.21.15.94:0/1129492636 learned_addr learned my addr 172.21.15.94:0/1129492636 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:56.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95a2000640 1 --2- 172.21.15.94:0/1129492636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95ac0d2290 0x7f95ac1655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:56.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.176+0000 7f95aaa00640 1 --2- 172.21.15.94:0/1129492636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95ac0ae980 0x7f95ac15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:56.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95ab400640 1 -- 172.21.15.94:0/1129492636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95ac0d2290 msgr2=0x7f95ac1655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:56.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95ab400640 1 --2- 172.21.15.94:0/1129492636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95ac0d2290 0x7f95ac1655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95ab400640 1 -- 172.21.15.94:0/1129492636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95ac0ae980 msgr2=0x7f95ac15e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:56.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95ab400640 1 --2- 172.21.15.94:0/1129492636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95ac0ae980 0x7f95ac15e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95ab400640 1 -- 172.21.15.94:0/1129492636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95ac1fb9c0 con 0x7f95ac15bd30 2024-09-10T23:33:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95a2000640 1 --2- 172.21.15.94:0/1129492636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95ac0d2290 0x7f95ac1655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95aaa00640 1 --2- 172.21.15.94:0/1129492636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95ac0ae980 0x7f95ac15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95ab400640 1 --2- 172.21.15.94:0/1129492636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac15bd30 0x7f95ac165b20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009850 tx=0x7f95a40a5740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95a3400640 1 -- 172.21.15.94:0/1129492636 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a40b2040 con 0x7f95ac15bd30 2024-09-10T23:33:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95a3400640 1 -- 172.21.15.94:0/1129492636 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95a4003990 con 0x7f95ac15bd30 2024-09-10T23:33:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95b1600640 1 -- 172.21.15.94:0/1129492636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95ac1fbf50 con 0x7f95ac15bd30 2024-09-10T23:33:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95a3400640 1 -- 172.21.15.94:0/1129492636 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4003de0 con 0x7f95ac15bd30 2024-09-10T23:33:56.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95b1600640 1 -- 172.21.15.94:0/1129492636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95ac1fc510 con 0x7f95ac15bd30 2024-09-10T23:33:56.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95b1600640 1 -- 172.21.15.94:0/1129492636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95ac0b1e70 con 0x7f95ac15bd30 2024-09-10T23:33:56.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95a3400640 1 -- 172.21.15.94:0/1129492636 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f95a40ad440 con 0x7f95ac15bd30 2024-09-10T23:33:56.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95a3400640 1 --2- 172.21.15.94:0/1129492636 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9588072790 0x7f9588074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:56.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95a3400640 1 -- 172.21.15.94:0/1129492636 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f95a412d6e0 con 0x7f95ac15bd30 2024-09-10T23:33:56.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.180+0000 7f95aaa00640 1 --2- 172.21.15.94:0/1129492636 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9588072790 0x7f9588074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:56.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.184+0000 7f95aaa00640 1 --2- 172.21.15.94:0/1129492636 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9588072790 0x7f9588074c50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f959c005fd0 tx=0x7f959c00a430 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:56.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.184+0000 7f95a3400640 1 -- 172.21.15.94:0/1129492636 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f95a40164d0 con 0x7f95ac15bd30 2024-09-10T23:33:56.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.408+0000 7f95b1600640 1 -- 172.21.15.94:0/1129492636 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f95ac1fc6e0 con 0x7f95ac15bd30 2024-09-10T23:33:56.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.408+0000 7f95a3400640 1 -- 172.21.15.94:0/1129492636 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v28) ==== 80+0+5304 (secure 0 0 0) 0x7f95a40f8d70 con 0x7f95ac15bd30 2024-09-10T23:33:56.414 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:56.414 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:33:56.414 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_dump": [ 2024-09-10T23:33:56.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.414 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.a", 2024-09-10T23:33:56.414 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/2m3LxAAr0ErpLJg+0+I2r74zQya4w==", 2024-09-10T23:33:56.414 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.415 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:33:56.415 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:33:56.415 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:33:56.415 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:56.415 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.415 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.b", 2024-09-10T23:33:56.415 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmc8pYABAAw90zu8D+9IwK15h4HIlGBA==", 2024-09-10T23:33:56.415 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.415 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:33:56.415 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:33:56.415 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:33:56.415 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:56.416 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.416 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.c", 2024-09-10T23:33:56.416 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmk98KNxAA44e9N4VHbw/94Lh1V9zugQ==", 2024-09-10T23:33:56.416 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.416 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:33:56.416 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:33:56.416 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:33:56.416 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:56.416 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.416 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.d", 2024-09-10T23:33:56.417 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmVAKzBxAAJehOw6aphTF/g3ucoQfw9A==", 2024-09-10T23:33:56.417 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.417 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:33:56.417 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:33:56.417 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:33:56.417 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:56.417 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.417 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.0", 2024-09-10T23:33:56.418 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDa1uBmSAD3KBAAJTF1OEC44Ml599aL7iEuDw==", 2024-09-10T23:33:56.418 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.418 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:33:56.418 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:33:56.418 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:56.418 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.418 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.1", 2024-09-10T23:33:56.418 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDe1uBma1L9ABAAfEjW1/Hb4fh4Kwv/ZUgFAQ==", 2024-09-10T23:33:56.418 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.419 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:33:56.419 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:33:56.419 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:56.419 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.419 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.2", 2024-09-10T23:33:56.419 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDh1uBmtLUQFhAAweeFHj6BRQVC9BX+VLiWEQ==", 2024-09-10T23:33:56.419 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.419 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:33:56.419 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:33:56.419 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:56.420 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.420 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.3", 2024-09-10T23:33:56.420 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDk1uBm1KFCKRAAev28iYcrEYOViX95uo6Kbw==", 2024-09-10T23:33:56.420 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.420 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:33:56.420 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:33:56.420 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:56.420 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.421 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.4", 2024-09-10T23:33:56.421 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDq1uBm/Hg9LhAAvB2D0rj4DeRPxTrGmXYIsA==", 2024-09-10T23:33:56.421 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.421 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:33:56.421 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:33:56.421 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:56.421 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.421 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.5", 2024-09-10T23:33:56.421 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDu1uBm1fAKCBAAFHMjnGFAItLdaACF/UjE/w==", 2024-09-10T23:33:56.421 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.421 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:33:56.421 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:33:56.422 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:56.422 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.422 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.6", 2024-09-10T23:33:56.422 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDx1uBmXgYfHhAA/6wwtM4mDRXsIpoTIHKJ0w==", 2024-09-10T23:33:56.422 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.422 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:33:56.422 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:33:56.422 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:56.422 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.422 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.7", 2024-09-10T23:33:56.423 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQD01uBmw6hQMxAAk7oGAg+iIplNK5KrokGwNA==", 2024-09-10T23:33:56.423 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.423 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:33:56.423 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:33:56.423 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:56.423 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.423 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.0", 2024-09-10T23:33:56.423 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g==", 2024-09-10T23:33:56.423 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.423 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:33:56.423 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow r", 2024-09-10T23:33:56.424 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T23:33:56.424 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.424 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.1", 2024-09-10T23:33:56.424 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmuSkhFRAA1R6WNXLImkGfhW9/XJzEAQ==", 2024-09-10T23:33:56.424 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.424 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:33:56.424 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:33:56.424 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:33:56.424 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:33:56.424 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.425 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.2", 2024-09-10T23:33:56.425 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmfvaqERAASpL0KtglFcAuRf4a/0z5JQ==", 2024-09-10T23:33:56.425 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.425 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:33:56.425 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:33:56.425 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:33:56.425 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:33:56.425 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.425 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.3", 2024-09-10T23:33:56.425 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBm8xUtGxAARc0ywfAhMxBNd57TzBoV3g==", 2024-09-10T23:33:56.425 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.425 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:33:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:33:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:33:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:33:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.admin", 2024-09-10T23:33:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmOpg1FBAA9NV6Jqt6doWMuAfcxztlOg==", 2024-09-10T23:33:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:33:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:33:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:33:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:56.426 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.427 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.x", 2024-09-10T23:33:56.427 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/EPVKxAA/Myv8Zjf7BejbzlkxyCC1w==", 2024-09-10T23:33:56.427 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.427 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:33:56.427 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:33:56.427 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:56.427 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:33:56.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.427 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.y", 2024-09-10T23:33:56.427 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmhHvtJxAACe+79nYhv2ODwtmNhA45FA==", 2024-09-10T23:33:56.428 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:33:56.428 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:33:56.428 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:33:56.428 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:33:56.428 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.428 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.428 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:33:56.428 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:33:56.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.412+0000 7f95b1600640 1 -- 172.21.15.94:0/1129492636 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9588072790 msgr2=0x7f9588074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:56.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.412+0000 7f95b1600640 1 --2- 172.21.15.94:0/1129492636 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9588072790 0x7f9588074c50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f959c005fd0 tx=0x7f959c00a430 comp rx=0 tx=0).stop 2024-09-10T23:33:56.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.412+0000 7f95b1600640 1 -- 172.21.15.94:0/1129492636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac15bd30 msgr2=0x7f95ac165b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:56.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.412+0000 7f95b1600640 1 --2- 172.21.15.94:0/1129492636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac15bd30 0x7f95ac165b20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009850 tx=0x7f95a40a5740 comp rx=0 tx=0).stop 2024-09-10T23:33:56.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.412+0000 7f95b1600640 1 -- 172.21.15.94:0/1129492636 shutdown_connections 2024-09-10T23:33:56.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.412+0000 7f95b1600640 1 --2- 172.21.15.94:0/1129492636 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9588072790 0x7f9588074c50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.412+0000 7f95b1600640 1 --2- 172.21.15.94:0/1129492636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95ac15bd30 0x7f95ac165b20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.412+0000 7f95b1600640 1 --2- 172.21.15.94:0/1129492636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95ac0d2290 0x7f95ac1655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.412+0000 7f95b1600640 1 --2- 172.21.15.94:0/1129492636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95ac0ae980 0x7f95ac15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.412+0000 7f95b1600640 1 -- 172.21.15.94:0/1129492636 >> 172.21.15.94:0/1129492636 conn(0x7f95ac09aa70 msgr2=0x7f95ac15d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.412+0000 7f95b1600640 1 -- 172.21.15.94:0/1129492636 shutdown_connections 2024-09-10T23:33:56.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.412+0000 7f95b1600640 1 -- 172.21.15.94:0/1129492636 wait complete. 2024-09-10T23:33:56.435 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T23:33:56.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.572+0000 7f4211c00640 1 -- 172.21.15.94:0/3061631354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f420c06cc20 msgr2=0x7f420c06d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:56.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.572+0000 7f4211c00640 1 --2- 172.21.15.94:0/3061631354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f420c06cc20 0x7f420c06d000 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f420c073620 tx=0x7f41fc01c830 comp rx=0 tx=0).stop 2024-09-10T23:33:56.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.572+0000 7f4211c00640 1 -- 172.21.15.94:0/3061631354 shutdown_connections 2024-09-10T23:33:56.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.572+0000 7f4211c00640 1 --2- 172.21.15.94:0/3061631354 >> v2:172.21.15.181:3300/0 conn(0x7f420c071a20 0x7f420c15f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.572+0000 7f4211c00640 1 --2- 172.21.15.94:0/3061631354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f420c06cc20 0x7f420c06d000 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.572+0000 7f4211c00640 1 -- 172.21.15.94:0/3061631354 >> 172.21.15.94:0/3061631354 conn(0x7f420c0d0b70 msgr2=0x7f420c0d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:56.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.572+0000 7f4211c00640 1 -- 172.21.15.94:0/3061631354 shutdown_connections 2024-09-10T23:33:56.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.572+0000 7f4211c00640 1 -- 172.21.15.94:0/3061631354 wait complete. 2024-09-10T23:33:56.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.572+0000 7f4211c00640 1 Processor -- start 2024-09-10T23:33:56.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.572+0000 7f4211c00640 1 -- start start 2024-09-10T23:33:56.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f4211c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f420c06cc20 0x7f420c15e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:56.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f4211c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f420c071a20 0x7f420c147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:56.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f4211c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f420c0d1690 0x7f420c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:56.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f4211c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f420c169130 con 0x7f420c071a20 2024-09-10T23:33:56.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f4211c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f420c168fb0 con 0x7f420c06cc20 2024-09-10T23:33:56.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f4211c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f420c1692b0 con 0x7f420c0d1690 2024-09-10T23:33:56.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f420a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f420c071a20 0x7f420c147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:56.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f420a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f420c071a20 0x7f420c147c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46988/0 (socket says 172.21.15.94:46988) 2024-09-10T23:33:56.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f420a000640 1 -- 172.21.15.94:0/1415545082 learned_addr learned my addr 172.21.15.94:0/1415545082 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:56.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f420b400640 1 --2- 172.21.15.94:0/1415545082 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f420c0d1690 0x7f420c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:56.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f420aa00640 1 --2- 172.21.15.94:0/1415545082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f420c06cc20 0x7f420c15e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:56.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f420a000640 1 -- 172.21.15.94:0/1415545082 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f420c0d1690 msgr2=0x7f420c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:56.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f420a000640 1 --2- 172.21.15.94:0/1415545082 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f420c0d1690 0x7f420c0b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f420a000640 1 -- 172.21.15.94:0/1415545082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f420c06cc20 msgr2=0x7f420c15e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:56.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f420a000640 1 --2- 172.21.15.94:0/1415545082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f420c06cc20 0x7f420c15e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f420a000640 1 -- 172.21.15.94:0/1415545082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420c148180 con 0x7f420c071a20 2024-09-10T23:33:56.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f420a000640 1 --2- 172.21.15.94:0/1415545082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f420c071a20 0x7f420c147c40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f41f800c970 tx=0x7f41f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:56.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f4202a00640 1 -- 172.21.15.94:0/1415545082 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8007c70 con 0x7f420c071a20 2024-09-10T23:33:56.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f4202a00640 1 -- 172.21.15.94:0/1415545082 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41f800f450 con 0x7f420c071a20 2024-09-10T23:33:56.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f4211c00640 1 -- 172.21.15.94:0/1415545082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420c148420 con 0x7f420c071a20 2024-09-10T23:33:56.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f4202a00640 1 -- 172.21.15.94:0/1415545082 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f800f8a0 con 0x7f420c071a20 2024-09-10T23:33:56.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f4211c00640 1 -- 172.21.15.94:0/1415545082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420c15d9f0 con 0x7f420c071a20 2024-09-10T23:33:56.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.576+0000 7f4211c00640 1 -- 172.21.15.94:0/1415545082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f420c06d000 con 0x7f420c071a20 2024-09-10T23:33:56.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.580+0000 7f4202a00640 1 -- 172.21.15.94:0/1415545082 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f41f8020030 con 0x7f420c071a20 2024-09-10T23:33:56.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.580+0000 7f4202a00640 1 --2- 172.21.15.94:0/1415545082 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f41ec0727c0 0x7f41ec074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:56.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.580+0000 7f4202a00640 1 -- 172.21.15.94:0/1415545082 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f41f80938c0 con 0x7f420c071a20 2024-09-10T23:33:56.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.580+0000 7f420aa00640 1 --2- 172.21.15.94:0/1415545082 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f41ec0727c0 0x7f41ec074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:56.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.580+0000 7f420aa00640 1 --2- 172.21.15.94:0/1415545082 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f41ec0727c0 0x7f41ec074c80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0af000 tx=0x7f41fc01b070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:56.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.584+0000 7f4202a00640 1 -- 172.21.15.94:0/1415545082 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f420c148180 con 0x7f420c071a20 2024-09-10T23:33:56.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.752+0000 7f4211c00640 1 -- 172.21.15.94:0/1415545082 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f420c072e90 con 0x7f420c071a20 2024-09-10T23:33:56.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.752+0000 7f4202a00640 1 -- 172.21.15.94:0/1415545082 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v40) ==== 84+0+65 (secure 0 0 0) 0x7f41f8014030 con 0x7f420c071a20 2024-09-10T23:33:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:33:56.755 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:33:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:33:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-09-10T23:33:56.756 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-09-10T23:33:56.756 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:33:56.756 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:33:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.752+0000 7f4211c00640 1 -- 172.21.15.94:0/1415545082 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f41ec0727c0 msgr2=0x7f41ec074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.752+0000 7f4211c00640 1 --2- 172.21.15.94:0/1415545082 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f41ec0727c0 0x7f41ec074c80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0af000 tx=0x7f41fc01b070 comp rx=0 tx=0).stop 2024-09-10T23:33:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.752+0000 7f4211c00640 1 -- 172.21.15.94:0/1415545082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f420c071a20 msgr2=0x7f420c147c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.752+0000 7f4211c00640 1 --2- 172.21.15.94:0/1415545082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f420c071a20 0x7f420c147c40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f41f800c970 tx=0x7f41f800ce30 comp rx=0 tx=0).stop 2024-09-10T23:33:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.752+0000 7f4211c00640 1 -- 172.21.15.94:0/1415545082 shutdown_connections 2024-09-10T23:33:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.752+0000 7f4211c00640 1 --2- 172.21.15.94:0/1415545082 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f41ec0727c0 0x7f41ec074c80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.752+0000 7f4211c00640 1 --2- 172.21.15.94:0/1415545082 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f420c0d1690 0x7f420c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.752+0000 7f4211c00640 1 --2- 172.21.15.94:0/1415545082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f420c071a20 0x7f420c147c40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.752+0000 7f4211c00640 1 --2- 172.21.15.94:0/1415545082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f420c06cc20 0x7f420c15e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.752+0000 7f4211c00640 1 -- 172.21.15.94:0/1415545082 >> 172.21.15.94:0/1415545082 conn(0x7f420c0d0b70 msgr2=0x7f420c06ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.752+0000 7f4211c00640 1 -- 172.21.15.94:0/1415545082 shutdown_connections 2024-09-10T23:33:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.752+0000 7f4211c00640 1 -- 172.21.15.94:0/1415545082 wait complete. 2024-09-10T23:33:56.777 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T23:33:56.777 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T23:33:56.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.920+0000 7f2051a00640 1 -- 172.21.15.94:0/1608386535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f204c15ba30 msgr2=0x7f204c0b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.920+0000 7f2051a00640 1 --2- 172.21.15.94:0/1608386535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f204c15ba30 0x7f204c0b1bd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f203c009840 tx=0x7f203c01c7f0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.920+0000 7f2051a00640 1 -- 172.21.15.94:0/1608386535 shutdown_connections 2024-09-10T23:33:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.920+0000 7f2051a00640 1 --2- 172.21.15.94:0/1608386535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f204c0d04b0 0x7f204c0aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.920+0000 7f2051a00640 1 --2- 172.21.15.94:0/1608386535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f204c15ba30 0x7f204c0b1bd0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.920+0000 7f2051a00640 1 -- 172.21.15.94:0/1608386535 >> 172.21.15.94:0/1608386535 conn(0x7f204c09aa70 msgr2=0x7f204c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.920+0000 7f2051a00640 1 -- 172.21.15.94:0/1608386535 shutdown_connections 2024-09-10T23:33:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.920+0000 7f2051a00640 1 -- 172.21.15.94:0/1608386535 wait complete. 2024-09-10T23:33:56.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.920+0000 7f2051a00640 1 Processor -- start 2024-09-10T23:33:56.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.920+0000 7f2051a00640 1 -- start start 2024-09-10T23:33:56.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f2051a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204c0b2110 0x7f204c1f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:56.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f2051a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f204c0d04b0 0x7f204c165e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f2051a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f204c15ba30 0x7f204c1fb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f2051a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f204c168830 con 0x7f204c0d04b0 2024-09-10T23:33:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f2051a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f204c1686b0 con 0x7f204c0b2110 2024-09-10T23:33:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f2051a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f204c1689b0 con 0x7f204c15ba30 2024-09-10T23:33:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f204b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f204c0d04b0 0x7f204c165e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f204b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f204c0d04b0 0x7f204c165e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47010/0 (socket says 172.21.15.94:47010) 2024-09-10T23:33:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f204b400640 1 -- 172.21.15.94:0/889817530 learned_addr learned my addr 172.21.15.94:0/889817530 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f204be00640 1 --2- 172.21.15.94:0/889817530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204c0b2110 0x7f204c1f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f2051000640 1 --2- 172.21.15.94:0/889817530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f204c15ba30 0x7f204c1fb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f204b400640 1 -- 172.21.15.94:0/889817530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f204c15ba30 msgr2=0x7f204c1fb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f204b400640 1 --2- 172.21.15.94:0/889817530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f204c15ba30 0x7f204c1fb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f204b400640 1 -- 172.21.15.94:0/889817530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204c0b2110 msgr2=0x7f204c1f84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:56.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f204b400640 1 --2- 172.21.15.94:0/889817530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204c0b2110 0x7f204c1f84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:56.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f204b400640 1 -- 172.21.15.94:0/889817530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204c1f7a40 con 0x7f204c0d04b0 2024-09-10T23:33:56.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f2051000640 1 --2- 172.21.15.94:0/889817530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f204c15ba30 0x7f204c1fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:56.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f204b400640 1 --2- 172.21.15.94:0/889817530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f204c0d04b0 0x7f204c165e10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f203800c450 tx=0x7f203800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:56.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f2043e00640 1 -- 172.21.15.94:0/889817530 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038016020 con 0x7f204c0d04b0 2024-09-10T23:33:56.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f2043e00640 1 -- 172.21.15.94:0/889817530 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2038012920 con 0x7f204c0d04b0 2024-09-10T23:33:56.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f2043e00640 1 -- 172.21.15.94:0/889817530 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038012d70 con 0x7f204c0d04b0 2024-09-10T23:33:56.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f2051a00640 1 -- 172.21.15.94:0/889817530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204c1fba60 con 0x7f204c0d04b0 2024-09-10T23:33:56.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f2051a00640 1 -- 172.21.15.94:0/889817530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f204c1f7cc0 con 0x7f204c0d04b0 2024-09-10T23:33:56.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f2051a00640 1 -- 172.21.15.94:0/889817530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f204c0aee30 con 0x7f204c0d04b0 2024-09-10T23:33:56.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.924+0000 7f2043e00640 1 -- 172.21.15.94:0/889817530 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f203801b400 con 0x7f204c0d04b0 2024-09-10T23:33:56.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.928+0000 7f2043e00640 1 --2- 172.21.15.94:0/889817530 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f202c072790 0x7f202c074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:56.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.928+0000 7f2043e00640 1 -- 172.21.15.94:0/889817530 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f2038093260 con 0x7f204c0d04b0 2024-09-10T23:33:56.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.928+0000 7f204be00640 1 --2- 172.21.15.94:0/889817530 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f202c072790 0x7f202c074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:56.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.928+0000 7f204be00640 1 --2- 172.21.15.94:0/889817530 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f202c072790 0x7f202c074c50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f203c0af000 tx=0x7f203c01b030 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:56.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:56.932+0000 7f2043e00640 1 -- 172.21.15.94:0/889817530 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2038020030 con 0x7f204c0d04b0 2024-09-10T23:33:57.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:57.100+0000 7f2051a00640 1 -- 172.21.15.94:0/889817530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f204c0b0e40 con 0x7f204c0d04b0 2024-09-10T23:33:58.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.016+0000 7f2043e00640 1 -- 172.21.15.94:0/889817530 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v41) ==== 138+0+0 (secure 0 0 0) 0x7f204c1f7a40 con 0x7f204c0d04b0 2024-09-10T23:33:58.019 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-09-10T23:33:58.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.016+0000 7f2051a00640 1 -- 172.21.15.94:0/889817530 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f202c072790 msgr2=0x7f202c074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:58.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.016+0000 7f2051a00640 1 --2- 172.21.15.94:0/889817530 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f202c072790 0x7f202c074c50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f203c0af000 tx=0x7f203c01b030 comp rx=0 tx=0).stop 2024-09-10T23:33:58.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.016+0000 7f2051a00640 1 -- 172.21.15.94:0/889817530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f204c0d04b0 msgr2=0x7f204c165e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:58.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.016+0000 7f2051a00640 1 --2- 172.21.15.94:0/889817530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f204c0d04b0 0x7f204c165e10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f203800c450 tx=0x7f203800c910 comp rx=0 tx=0).stop 2024-09-10T23:33:58.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.016+0000 7f2051a00640 1 -- 172.21.15.94:0/889817530 shutdown_connections 2024-09-10T23:33:58.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.016+0000 7f2051a00640 1 --2- 172.21.15.94:0/889817530 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f202c072790 0x7f202c074c50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:58.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.016+0000 7f2051a00640 1 --2- 172.21.15.94:0/889817530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f204c15ba30 0x7f204c1fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:58.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.016+0000 7f2051a00640 1 --2- 172.21.15.94:0/889817530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f204c0d04b0 0x7f204c165e10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:58.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.016+0000 7f2051a00640 1 --2- 172.21.15.94:0/889817530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f204c0b2110 0x7f204c1f84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:58.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.016+0000 7f2051a00640 1 -- 172.21.15.94:0/889817530 >> 172.21.15.94:0/889817530 conn(0x7f204c09aa70 msgr2=0x7f204c071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:58.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.016+0000 7f2051a00640 1 -- 172.21.15.94:0/889817530 shutdown_connections 2024-09-10T23:33:58.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.016+0000 7f2051a00640 1 -- 172.21.15.94:0/889817530 wait complete. 2024-09-10T23:33:58.048 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T23:33:58.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.192+0000 7fac4c600640 1 -- 172.21.15.94:0/4160119473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac4406fc00 msgr2=0x7fac440c49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:58.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.192+0000 7fac4c600640 1 --2- 172.21.15.94:0/4160119473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac4406fc00 0x7fac440c49b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fac30009900 tx=0x7fac3001c8e0 comp rx=0 tx=0).stop 2024-09-10T23:33:58.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.192+0000 7fac4c600640 1 -- 172.21.15.94:0/4160119473 shutdown_connections 2024-09-10T23:33:58.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.192+0000 7fac4c600640 1 --2- 172.21.15.94:0/4160119473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac440cb4d0 0x7fac4415b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:58.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.192+0000 7fac4c600640 1 --2- 172.21.15.94:0/4160119473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac4406fc00 0x7fac440c49b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:58.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.192+0000 7fac4c600640 1 -- 172.21.15.94:0/4160119473 >> 172.21.15.94:0/4160119473 conn(0x7fac440cfaf0 msgr2=0x7fac440cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:58.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.192+0000 7fac4c600640 1 -- 172.21.15.94:0/4160119473 shutdown_connections 2024-09-10T23:33:58.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.192+0000 7fac4c600640 1 -- 172.21.15.94:0/4160119473 wait complete. 2024-09-10T23:33:58.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac4c600640 1 Processor -- start 2024-09-10T23:33:58.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac4c600640 1 -- start start 2024-09-10T23:33:58.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac4c600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac4406fc00 0x7fac4413cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:58.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac4c600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac440cb4d0 0x7fac440b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:58.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac4c600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac44158790 0x7fac440b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:58.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac4c600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fac441680c0 con 0x7fac4406fc00 2024-09-10T23:33:58.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac4c600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fac44167f40 con 0x7fac440cb4d0 2024-09-10T23:33:58.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac4c600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fac44168240 con 0x7fac44158790 2024-09-10T23:33:58.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac49c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac4406fc00 0x7fac4413cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:58.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac49c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac4406fc00 0x7fac4413cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47038/0 (socket says 172.21.15.94:47038) 2024-09-10T23:33:58.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac49c00640 1 -- 172.21.15.94:0/1071982749 learned_addr learned my addr 172.21.15.94:0/1071982749 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:58.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac49200640 1 --2- 172.21.15.94:0/1071982749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac440cb4d0 0x7fac440b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:58.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac4a600640 1 --2- 172.21.15.94:0/1071982749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac44158790 0x7fac440b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:58.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac49c00640 1 -- 172.21.15.94:0/1071982749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac44158790 msgr2=0x7fac440b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:58.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac49c00640 1 --2- 172.21.15.94:0/1071982749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac44158790 0x7fac440b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:58.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac49c00640 1 -- 172.21.15.94:0/1071982749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac440cb4d0 msgr2=0x7fac440b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:58.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac49c00640 1 --2- 172.21.15.94:0/1071982749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac440cb4d0 0x7fac440b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:58.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac49c00640 1 -- 172.21.15.94:0/1071982749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4413f760 con 0x7fac4406fc00 2024-09-10T23:33:58.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac4a600640 1 --2- 172.21.15.94:0/1071982749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac44158790 0x7fac440b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:58.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac49c00640 1 --2- 172.21.15.94:0/1071982749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac4406fc00 0x7fac4413cc00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fac3800c3c0 tx=0x7fac3800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:58.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac41600640 1 -- 172.21.15.94:0/1071982749 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac380052d0 con 0x7fac4406fc00 2024-09-10T23:33:58.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac41600640 1 -- 172.21.15.94:0/1071982749 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac38007020 con 0x7fac4406fc00 2024-09-10T23:33:58.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac41600640 1 -- 172.21.15.94:0/1071982749 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38007470 con 0x7fac4406fc00 2024-09-10T23:33:58.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac4c600640 1 -- 172.21.15.94:0/1071982749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4413fa00 con 0x7fac4406fc00 2024-09-10T23:33:58.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.196+0000 7fac4c600640 1 -- 172.21.15.94:0/1071982749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac4413ffc0 con 0x7fac4406fc00 2024-09-10T23:33:58.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.200+0000 7fac4c600640 1 -- 172.21.15.94:0/1071982749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac440c49b0 con 0x7fac4406fc00 2024-09-10T23:33:58.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.200+0000 7fac41600640 1 -- 172.21.15.94:0/1071982749 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7fac380067e0 con 0x7fac4406fc00 2024-09-10T23:33:58.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.200+0000 7fac41600640 1 --2- 172.21.15.94:0/1071982749 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fac24072790 0x7fac24074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:58.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.200+0000 7fac41600640 1 -- 172.21.15.94:0/1071982749 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5227+0+0 (secure 0 0 0) 0x7fac38092840 con 0x7fac4406fc00 2024-09-10T23:33:58.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.200+0000 7fac49200640 1 --2- 172.21.15.94:0/1071982749 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fac24072790 0x7fac24074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:58.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.200+0000 7fac49200640 1 --2- 172.21.15.94:0/1071982749 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fac24072790 0x7fac24074c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fac300af000 tx=0x7fac3001b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:58.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.204+0000 7fac41600640 1 -- 172.21.15.94:0/1071982749 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fac38001090 con 0x7fac4406fc00 2024-09-10T23:33:58.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:58.372+0000 7fac4c600640 1 -- 172.21.15.94:0/1071982749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fac440d0850 con 0x7fac4406fc00 2024-09-10T23:33:59.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.020+0000 7fac41600640 1 -- 172.21.15.94:0/1071982749 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v42) ==== 170+0+0 (secure 0 0 0) 0x7fac3805d710 con 0x7fac4406fc00 2024-09-10T23:33:59.027 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-09-10T23:33:59.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.024+0000 7fac4c600640 1 -- 172.21.15.94:0/1071982749 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fac24072790 msgr2=0x7fac24074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:59.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.024+0000 7fac4c600640 1 --2- 172.21.15.94:0/1071982749 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fac24072790 0x7fac24074c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fac300af000 tx=0x7fac3001b120 comp rx=0 tx=0).stop 2024-09-10T23:33:59.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.024+0000 7fac4c600640 1 -- 172.21.15.94:0/1071982749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac4406fc00 msgr2=0x7fac4413cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:59.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.024+0000 7fac4c600640 1 --2- 172.21.15.94:0/1071982749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac4406fc00 0x7fac4413cc00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fac3800c3c0 tx=0x7fac3800c880 comp rx=0 tx=0).stop 2024-09-10T23:33:59.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.024+0000 7fac4c600640 1 -- 172.21.15.94:0/1071982749 shutdown_connections 2024-09-10T23:33:59.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.024+0000 7fac4c600640 1 --2- 172.21.15.94:0/1071982749 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fac24072790 0x7fac24074c50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:59.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.024+0000 7fac4c600640 1 --2- 172.21.15.94:0/1071982749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac440cb4d0 0x7fac440b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:59.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.024+0000 7fac4c600640 1 --2- 172.21.15.94:0/1071982749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac44158790 0x7fac440b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:59.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.024+0000 7fac4c600640 1 --2- 172.21.15.94:0/1071982749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac4406fc00 0x7fac4413cc00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:59.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.024+0000 7fac4c600640 1 -- 172.21.15.94:0/1071982749 >> 172.21.15.94:0/1071982749 conn(0x7fac440cfaf0 msgr2=0x7fac441571f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:59.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.024+0000 7fac4c600640 1 -- 172.21.15.94:0/1071982749 shutdown_connections 2024-09-10T23:33:59.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.028+0000 7fac4c600640 1 -- 172.21.15.94:0/1071982749 wait complete. 2024-09-10T23:33:59.070 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T23:33:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.212+0000 7f1edd800640 1 -- 172.21.15.94:0/4039519099 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ed8073920 msgr2=0x7f1ed8073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.212+0000 7f1edd800640 1 --2- 172.21.15.94:0/4039519099 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ed8073920 0x7f1ed8073d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009870 tx=0x7f1ed001c830 comp rx=0 tx=0).stop 2024-09-10T23:33:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.212+0000 7f1edd800640 1 -- 172.21.15.94:0/4039519099 shutdown_connections 2024-09-10T23:33:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.212+0000 7f1edd800640 1 --2- 172.21.15.94:0/4039519099 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ed8073920 0x7f1ed8073d90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.212+0000 7f1edd800640 1 -- 172.21.15.94:0/4039519099 >> 172.21.15.94:0/4039519099 conn(0x7f1ed80d0d80 msgr2=0x7f1ed80d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:33:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.212+0000 7f1edd800640 1 -- 172.21.15.94:0/4039519099 shutdown_connections 2024-09-10T23:33:59.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.212+0000 7f1edd800640 1 -- 172.21.15.94:0/4039519099 wait complete. 2024-09-10T23:33:59.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1edd800640 1 Processor -- start 2024-09-10T23:33:59.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1edd800640 1 -- start start 2024-09-10T23:33:59.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1edd800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ed806ce80 0x7f1ed80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:59.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1edd800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ed806d260 0x7f1ed814dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:59.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1edd800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ed8073920 0x7f1ed815ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:59.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1edd800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed8169f60 con 0x7f1ed8073920 2024-09-10T23:33:59.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1edd800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed8169de0 con 0x7f1ed806ce80 2024-09-10T23:33:59.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1edd800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed816a0e0 con 0x7f1ed806d260 2024-09-10T23:33:59.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1ed7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ed8073920 0x7f1ed815ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:59.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1ed7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ed8073920 0x7f1ed815ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47052/0 (socket says 172.21.15.94:47052) 2024-09-10T23:33:59.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1ed7400640 1 -- 172.21.15.94:0/2771250428 learned_addr learned my addr 172.21.15.94:0/2771250428 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:33:59.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1ed6000640 1 --2- 172.21.15.94:0/2771250428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ed806d260 0x7f1ed814dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:59.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1ed6a00640 1 --2- 172.21.15.94:0/2771250428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ed806ce80 0x7f1ed80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1ed7400640 1 -- 172.21.15.94:0/2771250428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ed806d260 msgr2=0x7f1ed814dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1ed7400640 1 --2- 172.21.15.94:0/2771250428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ed806d260 0x7f1ed814dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1ed7400640 1 -- 172.21.15.94:0/2771250428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ed806ce80 msgr2=0x7f1ed80b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:33:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1ed7400640 1 --2- 172.21.15.94:0/2771250428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ed806ce80 0x7f1ed80b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:33:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1ed7400640 1 -- 172.21.15.94:0/2771250428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed80b5290 con 0x7f1ed8073920 2024-09-10T23:33:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1ed6a00640 1 --2- 172.21.15.94:0/2771250428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ed806ce80 0x7f1ed80b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:33:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1ed7400640 1 --2- 172.21.15.94:0/2771250428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ed8073920 0x7f1ed815ad00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1ed001cd10 tx=0x7f1ed0005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1ecea00640 1 -- 172.21.15.94:0/2771250428 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00b2040 con 0x7f1ed8073920 2024-09-10T23:33:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1ecea00640 1 -- 172.21.15.94:0/2771250428 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ed00bb070 con 0x7f1ed8073920 2024-09-10T23:33:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1ecea00640 1 -- 172.21.15.94:0/2771250428 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0003600 con 0x7f1ed8073920 2024-09-10T23:33:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1edd800640 1 -- 172.21.15.94:0/2771250428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed814e2c0 con 0x7f1ed8073920 2024-09-10T23:33:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.216+0000 7f1edd800640 1 -- 172.21.15.94:0/2771250428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed81fb010 con 0x7f1ed8073920 2024-09-10T23:33:59.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.220+0000 7f1edd800640 1 -- 172.21.15.94:0/2771250428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed815f820 con 0x7f1ed8073920 2024-09-10T23:33:59.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.220+0000 7f1ecea00640 1 -- 172.21.15.94:0/2771250428 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f1ed00b5070 con 0x7f1ed8073920 2024-09-10T23:33:59.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.220+0000 7f1ecea00640 1 --2- 172.21.15.94:0/2771250428 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1eb8072790 0x7f1eb8074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:33:59.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.220+0000 7f1ecea00640 1 -- 172.21.15.94:0/2771250428 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5609+0+0 (secure 0 0 0) 0x7f1ed012e100 con 0x7f1ed8073920 2024-09-10T23:33:59.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.220+0000 7f1ed6a00640 1 --2- 172.21.15.94:0/2771250428 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1eb8072790 0x7f1eb8074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:33:59.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.224+0000 7f1ed6a00640 1 --2- 172.21.15.94:0/2771250428 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1eb8072790 0x7f1eb8074c50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8001a60 tx=0x7f1ec8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:33:59.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.224+0000 7f1ecea00640 1 -- 172.21.15.94:0/2771250428 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1ed0016450 con 0x7f1ed8073920 2024-09-10T23:33:59.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:33:59.424+0000 7f1edd800640 1 -- 172.21.15.94:0/2771250428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1ed8073700 con 0x7f1ed8073920 2024-09-10T23:34:00.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.036+0000 7f1ecea00640 1 -- 172.21.15.94:0/2771250428 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) ==== 172+0+0 (secure 0 0 0) 0x7f1ed00f95b0 con 0x7f1ed8073920 2024-09-10T23:34:00.044 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 6 and data pool 7 2024-09-10T23:34:00.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.048+0000 7f1edd800640 1 -- 172.21.15.94:0/2771250428 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1eb8072790 msgr2=0x7f1eb8074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:00.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.048+0000 7f1edd800640 1 --2- 172.21.15.94:0/2771250428 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1eb8072790 0x7f1eb8074c50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8001a60 tx=0x7f1ec8009040 comp rx=0 tx=0).stop 2024-09-10T23:34:00.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.048+0000 7f1edd800640 1 -- 172.21.15.94:0/2771250428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ed8073920 msgr2=0x7f1ed815ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:00.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.048+0000 7f1edd800640 1 --2- 172.21.15.94:0/2771250428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ed8073920 0x7f1ed815ad00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1ed001cd10 tx=0x7f1ed0005770 comp rx=0 tx=0).stop 2024-09-10T23:34:00.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.048+0000 7f1edd800640 1 -- 172.21.15.94:0/2771250428 shutdown_connections 2024-09-10T23:34:00.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.048+0000 7f1edd800640 1 --2- 172.21.15.94:0/2771250428 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1eb8072790 0x7f1eb8074c50 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8001a60 tx=0x7f1ec8009040 comp rx=0 tx=0).stop 2024-09-10T23:34:00.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.048+0000 7f1edd800640 1 --2- 172.21.15.94:0/2771250428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ed8073920 0x7f1ed815ad00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.048+0000 7f1edd800640 1 --2- 172.21.15.94:0/2771250428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ed806d260 0x7f1ed814dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.048+0000 7f1edd800640 1 --2- 172.21.15.94:0/2771250428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ed806ce80 0x7f1ed80b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.048+0000 7f1edd800640 1 -- 172.21.15.94:0/2771250428 >> 172.21.15.94:0/2771250428 conn(0x7f1ed80d0d80 msgr2=0x7f1ed80b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:00.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.056+0000 7f1edd800640 1 -- 172.21.15.94:0/2771250428 shutdown_connections 2024-09-10T23:34:00.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.056+0000 7f1edd800640 1 -- 172.21.15.94:0/2771250428 wait complete. 2024-09-10T23:34:00.086 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:34:00.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.224+0000 7f89be200640 1 -- 172.21.15.94:0/521068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89b815bc00 msgr2=0x7f89b80d15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:00.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.224+0000 7f89be200640 1 --2- 172.21.15.94:0/521068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89b815bc00 0x7f89b80d15a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f89a4009870 tx=0x7f89a401c7d0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.224+0000 7f89be200640 1 -- 172.21.15.94:0/521068 shutdown_connections 2024-09-10T23:34:00.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.224+0000 7f89be200640 1 --2- 172.21.15.94:0/521068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89b815bc00 0x7f89b80d15a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.224+0000 7f89be200640 1 -- 172.21.15.94:0/521068 >> 172.21.15.94:0/521068 conn(0x7f89b80d0a80 msgr2=0x7f89b80d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:00.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.224+0000 7f89be200640 1 -- 172.21.15.94:0/521068 shutdown_connections 2024-09-10T23:34:00.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.224+0000 7f89be200640 1 -- 172.21.15.94:0/521068 wait complete. 2024-09-10T23:34:00.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.224+0000 7f89be200640 1 Processor -- start 2024-09-10T23:34:00.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89be200640 1 -- start start 2024-09-10T23:34:00.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89be200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89b806cc20 0x7f89b80b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:00.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89be200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89b80affb0 0x7f89b81514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:00.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89be200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89b815bc00 0x7f89b80b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:00.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89be200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f89b816a1b0 con 0x7f89b806cc20 2024-09-10T23:34:00.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89be200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f89b816a030 con 0x7f89b815bc00 2024-09-10T23:34:00.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89be200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f89b816a330 con 0x7f89b80affb0 2024-09-10T23:34:00.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89bce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89b806cc20 0x7f89b80b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:00.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89bce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89b806cc20 0x7f89b80b20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47056/0 (socket says 172.21.15.94:47056) 2024-09-10T23:34:00.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89bce00640 1 -- 172.21.15.94:0/376992931 learned_addr learned my addr 172.21.15.94:0/376992931 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:00.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89b7e00640 1 --2- 172.21.15.94:0/376992931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89b80affb0 0x7f89b81514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:00.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89bd800640 1 --2- 172.21.15.94:0/376992931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89b815bc00 0x7f89b80b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:00.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89bce00640 1 -- 172.21.15.94:0/376992931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89b80affb0 msgr2=0x7f89b81514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:00.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89bce00640 1 --2- 172.21.15.94:0/376992931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89b80affb0 0x7f89b81514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89bce00640 1 -- 172.21.15.94:0/376992931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89b815bc00 msgr2=0x7f89b80b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:00.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89bce00640 1 --2- 172.21.15.94:0/376992931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89b815bc00 0x7f89b80b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89bce00640 1 -- 172.21.15.94:0/376992931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b80b4980 con 0x7f89b806cc20 2024-09-10T23:34:00.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89bce00640 1 --2- 172.21.15.94:0/376992931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89b806cc20 0x7f89b80b20d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00c490 tx=0x7f89ac00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:00.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89b4c00640 1 -- 172.21.15.94:0/376992931 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac0052d0 con 0x7f89b806cc20 2024-09-10T23:34:00.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89b4c00640 1 -- 172.21.15.94:0/376992931 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89ac005490 con 0x7f89b806cc20 2024-09-10T23:34:00.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89be200640 1 -- 172.21.15.94:0/376992931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b80b1310 con 0x7f89b806cc20 2024-09-10T23:34:00.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89b4c00640 1 -- 172.21.15.94:0/376992931 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac012870 con 0x7f89b806cc20 2024-09-10T23:34:00.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89be200640 1 -- 172.21.15.94:0/376992931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b80b18d0 con 0x7f89b806cc20 2024-09-10T23:34:00.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.228+0000 7f89be200640 1 -- 172.21.15.94:0/376992931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b80d15a0 con 0x7f89b806cc20 2024-09-10T23:34:00.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.232+0000 7f89b4c00640 1 -- 172.21.15.94:0/376992931 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f89ac012a10 con 0x7f89b806cc20 2024-09-10T23:34:00.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.232+0000 7f89b4c00640 1 --2- 172.21.15.94:0/376992931 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8998072790 0x7f8998074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:00.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.232+0000 7f89b4c00640 1 -- 172.21.15.94:0/376992931 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f89ac093090 con 0x7f89b806cc20 2024-09-10T23:34:00.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.232+0000 7f89b7e00640 1 --2- 172.21.15.94:0/376992931 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8998072790 0x7f8998074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:00.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.232+0000 7f89b7e00640 1 --2- 172.21.15.94:0/376992931 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8998072790 0x7f8998074c50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f89a40af000 tx=0x7f89a401b010 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:00.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.236+0000 7f89b4c00640 1 -- 172.21.15.94:0/376992931 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f89ac05dd80 con 0x7f89b806cc20 2024-09-10T23:34:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.420+0000 7f89be200640 1 -- 172.21.15.94:0/376992931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89b8153020 con 0x7f89b806cc20 2024-09-10T23:34:00.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.420+0000 7f89b4c00640 1 -- 172.21.15.94:0/376992931 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16670 (secure 0 0 0) 0x7f89ac0627b0 con 0x7f89b806cc20 2024-09-10T23:34:00.426 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:00.426 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":43,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:33:59.428719+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:33:57.113965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T23:33:58.384504+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:34:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.424+0000 7f89be200640 1 -- 172.21.15.94:0/376992931 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8998072790 msgr2=0x7f8998074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.424+0000 7f89be200640 1 --2- 172.21.15.94:0/376992931 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8998072790 0x7f8998074c50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f89a40af000 tx=0x7f89a401b010 comp rx=0 tx=0).stop 2024-09-10T23:34:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.424+0000 7f89be200640 1 -- 172.21.15.94:0/376992931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89b806cc20 msgr2=0x7f89b80b20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.424+0000 7f89be200640 1 --2- 172.21.15.94:0/376992931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89b806cc20 0x7f89b80b20d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00c490 tx=0x7f89ac00c950 comp rx=0 tx=0).stop 2024-09-10T23:34:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.424+0000 7f89be200640 1 -- 172.21.15.94:0/376992931 shutdown_connections 2024-09-10T23:34:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.424+0000 7f89be200640 1 --2- 172.21.15.94:0/376992931 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8998072790 0x7f8998074c50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.424+0000 7f89be200640 1 --2- 172.21.15.94:0/376992931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89b815bc00 0x7f89b80b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.424+0000 7f89be200640 1 --2- 172.21.15.94:0/376992931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89b80affb0 0x7f89b81514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.424+0000 7f89be200640 1 --2- 172.21.15.94:0/376992931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89b806cc20 0x7f89b80b20d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.424+0000 7f89be200640 1 -- 172.21.15.94:0/376992931 >> 172.21.15.94:0/376992931 conn(0x7f89b80d0a80 msgr2=0x7f89b80d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.424+0000 7f89be200640 1 -- 172.21.15.94:0/376992931 shutdown_connections 2024-09-10T23:34:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.424+0000 7f89be200640 1 -- 172.21.15.94:0/376992931 wait complete. 2024-09-10T23:34:00.448 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:34:00.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.584+0000 7f2cb4c00640 1 -- 172.21.15.94:0/1110618081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cb015cbc0 msgr2=0x7f2cb00cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:00.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.584+0000 7f2cb4c00640 1 --2- 172.21.15.94:0/1110618081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cb015cbc0 0x7f2cb00cde20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009840 tx=0x7f2c9801c7f0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.584+0000 7f2cb4c00640 1 -- 172.21.15.94:0/1110618081 shutdown_connections 2024-09-10T23:34:00.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.584+0000 7f2cb4c00640 1 --2- 172.21.15.94:0/1110618081 >> v2:172.21.15.181:3300/0 conn(0x7f2cb00cfd50 0x7f2cb00d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.584+0000 7f2cb4c00640 1 --2- 172.21.15.94:0/1110618081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cb015cbc0 0x7f2cb00cde20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.584+0000 7f2cb4c00640 1 -- 172.21.15.94:0/1110618081 >> 172.21.15.94:0/1110618081 conn(0x7f2cb006cdf0 msgr2=0x7f2cb006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.584+0000 7f2cb4c00640 1 -- 172.21.15.94:0/1110618081 shutdown_connections 2024-09-10T23:34:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.584+0000 7f2cb4c00640 1 -- 172.21.15.94:0/1110618081 wait complete. 2024-09-10T23:34:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cb4c00640 1 Processor -- start 2024-09-10T23:34:00.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cb4c00640 1 -- start start 2024-09-10T23:34:00.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cb4c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cb00cfd50 0x7f2cb013e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:00.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cb4c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cb0152c10 0x7f2cb01515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:00.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cb4c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cb015cbc0 0x7f2cb014d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:00.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cb4c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0168b50 con 0x7f2cb015cbc0 2024-09-10T23:34:00.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cb4c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb01689d0 con 0x7f2cb0152c10 2024-09-10T23:34:00.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cb4c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb0168cd0 con 0x7f2cb00cfd50 2024-09-10T23:34:00.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cae800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cb015cbc0 0x7f2cb014d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:00.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cae800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cb015cbc0 0x7f2cb014d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47070/0 (socket says 172.21.15.94:47070) 2024-09-10T23:34:00.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cae800640 1 -- 172.21.15.94:0/850409061 learned_addr learned my addr 172.21.15.94:0/850409061 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:00.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2ca7e00640 1 --2- 172.21.15.94:0/850409061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cb0152c10 0x7f2cb01515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:00.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cade00640 1 --2- 172.21.15.94:0/850409061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cb00cfd50 0x7f2cb013e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:00.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cae800640 1 -- 172.21.15.94:0/850409061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cb00cfd50 msgr2=0x7f2cb013e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:00.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cae800640 1 --2- 172.21.15.94:0/850409061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cb00cfd50 0x7f2cb013e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cae800640 1 -- 172.21.15.94:0/850409061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cb0152c10 msgr2=0x7f2cb01515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:00.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cae800640 1 --2- 172.21.15.94:0/850409061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cb0152c10 0x7f2cb01515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cae800640 1 -- 172.21.15.94:0/850409061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb0150580 con 0x7f2cb015cbc0 2024-09-10T23:34:00.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cae800640 1 --2- 172.21.15.94:0/850409061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cb015cbc0 0x7f2cb014d470 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800eb20 tx=0x7f2ca800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:00.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2ca6000640 1 -- 172.21.15.94:0/850409061 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8004b30 con 0x7f2cb015cbc0 2024-09-10T23:34:00.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2ca6000640 1 -- 172.21.15.94:0/850409061 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ca8004cf0 con 0x7f2cb015cbc0 2024-09-10T23:34:00.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2ca6000640 1 -- 172.21.15.94:0/850409061 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8005280 con 0x7f2cb015cbc0 2024-09-10T23:34:00.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cb4c00640 1 -- 172.21.15.94:0/850409061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb0150860 con 0x7f2cb015cbc0 2024-09-10T23:34:00.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cb4c00640 1 -- 172.21.15.94:0/850409061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb0150e20 con 0x7f2cb015cbc0 2024-09-10T23:34:00.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.588+0000 7f2cb4c00640 1 -- 172.21.15.94:0/850409061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb00d01b0 con 0x7f2cb015cbc0 2024-09-10T23:34:00.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.592+0000 7f2ca6000640 1 -- 172.21.15.94:0/850409061 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f2ca8005420 con 0x7f2cb015cbc0 2024-09-10T23:34:00.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.592+0000 7f2ca6000640 1 --2- 172.21.15.94:0/850409061 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2c8c072790 0x7f2c8c074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:00.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.592+0000 7f2ca6000640 1 -- 172.21.15.94:0/850409061 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f2ca80942d0 con 0x7f2cb015cbc0 2024-09-10T23:34:00.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.592+0000 7f2cade00640 1 --2- 172.21.15.94:0/850409061 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2c8c072790 0x7f2c8c074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:00.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.592+0000 7f2cade00640 1 --2- 172.21.15.94:0/850409061 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2c8c072790 0x7f2c8c074c50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0000f70 tx=0x7f2ca0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:00.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.596+0000 7f2ca6000640 1 -- 172.21.15.94:0/850409061 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2ca805efc0 con 0x7f2cb015cbc0 2024-09-10T23:34:00.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.760+0000 7f2cb4c00640 1 -- 172.21.15.94:0/850409061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2cb00b46c0 con 0x7f2cb015cbc0 2024-09-10T23:34:00.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.760+0000 7f2ca6000640 1 -- 172.21.15.94:0/850409061 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16670 (secure 0 0 0) 0x7f2ca80639f0 con 0x7f2cb015cbc0 2024-09-10T23:34:00.766 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:00.767 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":43,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:33:59.428719+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:33:57.113965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T23:33:58.384504+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:34:00.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.764+0000 7f2cb4c00640 1 -- 172.21.15.94:0/850409061 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2c8c072790 msgr2=0x7f2c8c074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:00.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.764+0000 7f2cb4c00640 1 --2- 172.21.15.94:0/850409061 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2c8c072790 0x7f2c8c074c50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0000f70 tx=0x7f2ca0009040 comp rx=0 tx=0).stop 2024-09-10T23:34:00.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.764+0000 7f2cb4c00640 1 -- 172.21.15.94:0/850409061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cb015cbc0 msgr2=0x7f2cb014d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:00.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.764+0000 7f2cb4c00640 1 --2- 172.21.15.94:0/850409061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cb015cbc0 0x7f2cb014d470 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800eb20 tx=0x7f2ca800c2d0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.764+0000 7f2cb4c00640 1 -- 172.21.15.94:0/850409061 shutdown_connections 2024-09-10T23:34:00.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.764+0000 7f2cb4c00640 1 --2- 172.21.15.94:0/850409061 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2c8c072790 0x7f2c8c074c50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.764+0000 7f2cb4c00640 1 --2- 172.21.15.94:0/850409061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cb0152c10 0x7f2cb01515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.764+0000 7f2cb4c00640 1 --2- 172.21.15.94:0/850409061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cb015cbc0 0x7f2cb014d470 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.764+0000 7f2cb4c00640 1 --2- 172.21.15.94:0/850409061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cb00cfd50 0x7f2cb013e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.764+0000 7f2cb4c00640 1 -- 172.21.15.94:0/850409061 >> 172.21.15.94:0/850409061 conn(0x7f2cb006cdf0 msgr2=0x7f2cb0070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:00.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.764+0000 7f2cb4c00640 1 -- 172.21.15.94:0/850409061 shutdown_connections 2024-09-10T23:34:00.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.764+0000 7f2cb4c00640 1 -- 172.21.15.94:0/850409061 wait complete. 2024-09-10T23:34:00.787 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T23:34:00.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.928+0000 7f1eac400640 1 -- 172.21.15.94:0/2601941341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ea40b1180 msgr2=0x7f1ea40ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:00.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.928+0000 7f1eac400640 1 --2- 172.21.15.94:0/2601941341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ea40b1180 0x7f1ea40ad060 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009c80 tx=0x7f1e9c01cda0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.928+0000 7f1eac400640 1 -- 172.21.15.94:0/2601941341 shutdown_connections 2024-09-10T23:34:00.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.928+0000 7f1eac400640 1 --2- 172.21.15.94:0/2601941341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ea40b1180 0x7f1ea40ad060 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.928+0000 7f1eac400640 1 --2- 172.21.15.94:0/2601941341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ea40affd0 0x7f1ea40b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.928+0000 7f1eac400640 1 -- 172.21.15.94:0/2601941341 >> 172.21.15.94:0/2601941341 conn(0x7f1ea4074240 msgr2=0x7f1ea4074620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.928+0000 7f1eac400640 1 -- 172.21.15.94:0/2601941341 shutdown_connections 2024-09-10T23:34:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.928+0000 7f1eac400640 1 -- 172.21.15.94:0/2601941341 wait complete. 2024-09-10T23:34:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.928+0000 7f1eac400640 1 Processor -- start 2024-09-10T23:34:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.928+0000 7f1eac400640 1 -- start start 2024-09-10T23:34:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.928+0000 7f1eac400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ea40b1180 0x7f1ea41fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1eac400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ea415bf80 0x7f1ea41f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1eac400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ea41541a0 0x7f1ea40cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1eac400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea4168f50 con 0x7f1ea40b1180 2024-09-10T23:34:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1eac400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea4168dd0 con 0x7f1ea41541a0 2024-09-10T23:34:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1eac400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea41690d0 con 0x7f1ea415bf80 2024-09-10T23:34:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1ea9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ea40b1180 0x7f1ea41fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1ea9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ea40b1180 0x7f1ea41fbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47098/0 (socket says 172.21.15.94:47098) 2024-09-10T23:34:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1ea9a00640 1 -- 172.21.15.94:0/3663308910 learned_addr learned my addr 172.21.15.94:0/3663308910 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1ea9000640 1 --2- 172.21.15.94:0/3663308910 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ea415bf80 0x7f1ea41f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1eaa400640 1 --2- 172.21.15.94:0/3663308910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ea41541a0 0x7f1ea40cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1ea9a00640 1 -- 172.21.15.94:0/3663308910 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ea415bf80 msgr2=0x7f1ea41f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1ea9a00640 1 --2- 172.21.15.94:0/3663308910 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ea415bf80 0x7f1ea41f7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1ea9a00640 1 -- 172.21.15.94:0/3663308910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ea41541a0 msgr2=0x7f1ea40cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1ea9a00640 1 --2- 172.21.15.94:0/3663308910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ea41541a0 0x7f1ea40cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1ea9a00640 1 -- 172.21.15.94:0/3663308910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea41f82e0 con 0x7f1ea40b1180 2024-09-10T23:34:00.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1ea9a00640 1 --2- 172.21.15.94:0/3663308910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ea40b1180 0x7f1ea41fbb70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800c3f0 tx=0x7f1e9800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:00.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1ea1600640 1 -- 172.21.15.94:0/3663308910 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e98017070 con 0x7f1ea40b1180 2024-09-10T23:34:00.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1ea1600640 1 -- 172.21.15.94:0/3663308910 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e980053c0 con 0x7f1ea40b1180 2024-09-10T23:34:00.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1ea1600640 1 -- 172.21.15.94:0/3663308910 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e98003860 con 0x7f1ea40b1180 2024-09-10T23:34:00.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1eac400640 1 -- 172.21.15.94:0/3663308910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea41f85c0 con 0x7f1ea40b1180 2024-09-10T23:34:00.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1eac400640 1 -- 172.21.15.94:0/3663308910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea4156e40 con 0x7f1ea40b1180 2024-09-10T23:34:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1eac400640 1 -- 172.21.15.94:0/3663308910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea40ad060 con 0x7f1ea40b1180 2024-09-10T23:34:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.932+0000 7f1ea1600640 1 -- 172.21.15.94:0/3663308910 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f1e98003a00 con 0x7f1ea40b1180 2024-09-10T23:34:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.936+0000 7f1ea1600640 1 --2- 172.21.15.94:0/3663308910 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1e84072790 0x7f1e84074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.936+0000 7f1ea1600640 1 -- 172.21.15.94:0/3663308910 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f1e98092c70 con 0x7f1ea40b1180 2024-09-10T23:34:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.936+0000 7f1ea9000640 1 --2- 172.21.15.94:0/3663308910 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1e84072790 0x7f1e84074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.936+0000 7f1ea9000640 1 --2- 172.21.15.94:0/3663308910 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1e84072790 0x7f1e84074c50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40c74d0 tx=0x7f1e90005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:00.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:00.940+0000 7f1ea1600640 1 -- 172.21.15.94:0/3663308910 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1e9805dfc0 con 0x7f1ea40b1180 2024-09-10T23:34:01.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:01.140+0000 7f1eac400640 1 -- 172.21.15.94:0/3663308910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1ea4159380 con 0x7f1ea40b1180 2024-09-10T23:34:02.021 INFO:tasks.ceph.mds.d.smithi181.stderr:2024-09-10T23:34:02.017+0000 7fd491200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:34:02.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.048+0000 7f1ea1600640 1 -- 172.21.15.94:0/3663308910 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) ==== 118+0+0 (secure 0 0 0) 0x7f1e9805d8c0 con 0x7f1ea40b1180 2024-09-10T23:34:02.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.048+0000 7f1eac400640 1 -- 172.21.15.94:0/3663308910 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1e84072790 msgr2=0x7f1e84074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:02.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.048+0000 7f1eac400640 1 --2- 172.21.15.94:0/3663308910 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1e84072790 0x7f1e84074c50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40c74d0 tx=0x7f1e90005860 comp rx=0 tx=0).stop 2024-09-10T23:34:02.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.048+0000 7f1eac400640 1 -- 172.21.15.94:0/3663308910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ea40b1180 msgr2=0x7f1ea41fbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:02.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.048+0000 7f1eac400640 1 --2- 172.21.15.94:0/3663308910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ea40b1180 0x7f1ea41fbb70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800c3f0 tx=0x7f1e9800c8b0 comp rx=0 tx=0).stop 2024-09-10T23:34:02.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.048+0000 7f1eac400640 1 -- 172.21.15.94:0/3663308910 shutdown_connections 2024-09-10T23:34:02.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.048+0000 7f1eac400640 1 --2- 172.21.15.94:0/3663308910 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1e84072790 0x7f1e84074c50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:02.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.048+0000 7f1eac400640 1 --2- 172.21.15.94:0/3663308910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ea41541a0 0x7f1ea40cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:02.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.048+0000 7f1eac400640 1 --2- 172.21.15.94:0/3663308910 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ea415bf80 0x7f1ea41f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:02.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.048+0000 7f1eac400640 1 --2- 172.21.15.94:0/3663308910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ea40b1180 0x7f1ea41fbb70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:02.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.048+0000 7f1eac400640 1 -- 172.21.15.94:0/3663308910 >> 172.21.15.94:0/3663308910 conn(0x7f1ea4074240 msgr2=0x7f1ea406f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:02.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.048+0000 7f1eac400640 1 -- 172.21.15.94:0/3663308910 shutdown_connections 2024-09-10T23:34:02.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.048+0000 7f1eac400640 1 -- 172.21.15.94:0/3663308910 wait complete. 2024-09-10T23:34:02.074 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T23:34:02.075 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T23:34:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0edda00640 1 -- 172.21.15.94:0/3015313291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed80cf280 msgr2=0x7f0ed815af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0edda00640 1 --2- 172.21.15.94:0/3015313291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed80cf280 0x7f0ed815af90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000a950 tx=0x7f0ed001db70 comp rx=0 tx=0).stop 2024-09-10T23:34:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0edda00640 1 -- 172.21.15.94:0/3015313291 shutdown_connections 2024-09-10T23:34:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0edda00640 1 --2- 172.21.15.94:0/3015313291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed80cf280 0x7f0ed815af90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0edda00640 1 -- 172.21.15.94:0/3015313291 >> 172.21.15.94:0/3015313291 conn(0x7f0ed80cfaf0 msgr2=0x7f0ed80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0edda00640 1 -- 172.21.15.94:0/3015313291 shutdown_connections 2024-09-10T23:34:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0edda00640 1 -- 172.21.15.94:0/3015313291 wait complete. 2024-09-10T23:34:02.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0edda00640 1 Processor -- start 2024-09-10T23:34:02.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0edda00640 1 -- start start 2024-09-10T23:34:02.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0edda00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed80c64f0 0x7f0ed814df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:02.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0edda00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ed80ce280 0x7f0ed80c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:02.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0edda00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ed80cf280 0x7f0ed814f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:02.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0edda00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed8168480 con 0x7f0ed80c64f0 2024-09-10T23:34:02.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0edda00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed8168300 con 0x7f0ed80cf280 2024-09-10T23:34:02.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0edda00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed8168600 con 0x7f0ed80ce280 2024-09-10T23:34:02.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0ed6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed80c64f0 0x7f0ed814df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:02.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0ed6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed80c64f0 0x7f0ed814df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47116/0 (socket says 172.21.15.94:47116) 2024-09-10T23:34:02.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0ed6a00640 1 -- 172.21.15.94:0/1021737786 learned_addr learned my addr 172.21.15.94:0/1021737786 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:02.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0ed7400640 1 --2- 172.21.15.94:0/1021737786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ed80cf280 0x7f0ed814f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:02.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0ed6000640 1 --2- 172.21.15.94:0/1021737786 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ed80ce280 0x7f0ed80c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:02.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0ed6a00640 1 -- 172.21.15.94:0/1021737786 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ed80ce280 msgr2=0x7f0ed80c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:02.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0ed6a00640 1 --2- 172.21.15.94:0/1021737786 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ed80ce280 0x7f0ed80c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:02.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0ed6a00640 1 -- 172.21.15.94:0/1021737786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ed80cf280 msgr2=0x7f0ed814f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:02.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0ed6a00640 1 --2- 172.21.15.94:0/1021737786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ed80cf280 0x7f0ed814f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:02.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0ed6a00640 1 -- 172.21.15.94:0/1021737786 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed80c7290 con 0x7f0ed80c64f0 2024-09-10T23:34:02.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.216+0000 7f0ed6a00640 1 --2- 172.21.15.94:0/1021737786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed80c64f0 0x7f0ed814df70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800cbe0 tx=0x7f0ec800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:02.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.220+0000 7f0ecea00640 1 -- 172.21.15.94:0/1021737786 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8016020 con 0x7f0ed80c64f0 2024-09-10T23:34:02.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.220+0000 7f0ecea00640 1 -- 172.21.15.94:0/1021737786 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ec800ab60 con 0x7f0ed80c64f0 2024-09-10T23:34:02.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.220+0000 7f0ecea00640 1 -- 172.21.15.94:0/1021737786 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8012420 con 0x7f0ed80c64f0 2024-09-10T23:34:02.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.220+0000 7f0edda00640 1 -- 172.21.15.94:0/1021737786 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed80c7570 con 0x7f0ed80c64f0 2024-09-10T23:34:02.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.220+0000 7f0edda00640 1 -- 172.21.15.94:0/1021737786 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed80c7b30 con 0x7f0ed80c64f0 2024-09-10T23:34:02.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.220+0000 7f0edda00640 1 -- 172.21.15.94:0/1021737786 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed815af90 con 0x7f0ed80c64f0 2024-09-10T23:34:02.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.220+0000 7f0ecea00640 1 -- 172.21.15.94:0/1021737786 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f0ec80125c0 con 0x7f0ed80c64f0 2024-09-10T23:34:02.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.220+0000 7f0ecea00640 1 --2- 172.21.15.94:0/1021737786 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0eb8072790 0x7f0eb8074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:02.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.220+0000 7f0ecea00640 1 -- 172.21.15.94:0/1021737786 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f0ec8093cd0 con 0x7f0ed80c64f0 2024-09-10T23:34:02.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.220+0000 7f0ed6000640 1 --2- 172.21.15.94:0/1021737786 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0eb8072790 0x7f0eb8074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:02.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.224+0000 7f0ed6000640 1 --2- 172.21.15.94:0/1021737786 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0eb8072790 0x7f0eb8074c50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80b57d0 tx=0x7f0ec4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:02.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.224+0000 7f0ecea00640 1 -- 172.21.15.94:0/1021737786 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0ec8020020 con 0x7f0ed80c64f0 2024-09-10T23:34:02.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:02.424+0000 7f0edda00640 1 -- 172.21.15.94:0/1021737786 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0ed80d0850 con 0x7f0ed80c64f0 2024-09-10T23:34:03.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.056+0000 7f0ecea00640 1 -- 172.21.15.94:0/1021737786 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) ==== 122+0+0 (secure 0 0 0) 0x7f0ec805ef80 con 0x7f0ed80c64f0 2024-09-10T23:34:03.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.056+0000 7f0edda00640 1 -- 172.21.15.94:0/1021737786 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0eb8072790 msgr2=0x7f0eb8074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:03.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.056+0000 7f0edda00640 1 --2- 172.21.15.94:0/1021737786 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0eb8072790 0x7f0eb8074c50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80b57d0 tx=0x7f0ec4009040 comp rx=0 tx=0).stop 2024-09-10T23:34:03.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.056+0000 7f0edda00640 1 -- 172.21.15.94:0/1021737786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed80c64f0 msgr2=0x7f0ed814df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:03.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.056+0000 7f0edda00640 1 --2- 172.21.15.94:0/1021737786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed80c64f0 0x7f0ed814df70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800cbe0 tx=0x7f0ec800ccd0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.056+0000 7f0edda00640 1 -- 172.21.15.94:0/1021737786 shutdown_connections 2024-09-10T23:34:03.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.056+0000 7f0edda00640 1 --2- 172.21.15.94:0/1021737786 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0eb8072790 0x7f0eb8074c50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.056+0000 7f0edda00640 1 --2- 172.21.15.94:0/1021737786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ed80cf280 0x7f0ed814f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.056+0000 7f0edda00640 1 --2- 172.21.15.94:0/1021737786 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ed80ce280 0x7f0ed80c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.056+0000 7f0edda00640 1 --2- 172.21.15.94:0/1021737786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed80c64f0 0x7f0ed814df70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.056+0000 7f0edda00640 1 -- 172.21.15.94:0/1021737786 >> 172.21.15.94:0/1021737786 conn(0x7f0ed80cfaf0 msgr2=0x7f0ed80ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:03.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.056+0000 7f0edda00640 1 -- 172.21.15.94:0/1021737786 shutdown_connections 2024-09-10T23:34:03.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.056+0000 7f0edda00640 1 -- 172.21.15.94:0/1021737786 wait complete. 2024-09-10T23:34:03.082 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T23:34:03.082 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:34:03.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.220+0000 7f3a24e00640 1 -- 172.21.15.94:0/2714233624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a2006df90 msgr2=0x7f3a200715c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:03.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.220+0000 7f3a24e00640 1 --2- 172.21.15.94:0/2714233624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a2006df90 0x7f3a200715c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009900 tx=0x7f3a1001c5e0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.220+0000 7f3a24e00640 1 -- 172.21.15.94:0/2714233624 shutdown_connections 2024-09-10T23:34:03.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.220+0000 7f3a24e00640 1 --2- 172.21.15.94:0/2714233624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a2006e8b0 0x7f3a200ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.220+0000 7f3a24e00640 1 --2- 172.21.15.94:0/2714233624 >> v2:172.21.15.181:3300/0 conn(0x7f3a200b1170 0x7f3a2006e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.220+0000 7f3a24e00640 1 --2- 172.21.15.94:0/2714233624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a2006df90 0x7f3a200715c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.220+0000 7f3a24e00640 1 -- 172.21.15.94:0/2714233624 >> 172.21.15.94:0/2714233624 conn(0x7f3a200d1c80 msgr2=0x7f3a200d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:03.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.220+0000 7f3a24e00640 1 -- 172.21.15.94:0/2714233624 shutdown_connections 2024-09-10T23:34:03.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.220+0000 7f3a24e00640 1 -- 172.21.15.94:0/2714233624 wait complete. 2024-09-10T23:34:03.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a24e00640 1 Processor -- start 2024-09-10T23:34:03.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a24e00640 1 -- start start 2024-09-10T23:34:03.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a24e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a2006df90 0x7f3a20158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:03.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a24e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a2006e8b0 0x7f3a200b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:03.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a24e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a200b1170 0x7f3a2015d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:03.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a24e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3a200d0030 con 0x7f3a2006e8b0 2024-09-10T23:34:03.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a24e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3a200cfeb0 con 0x7f3a200b1170 2024-09-10T23:34:03.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a24e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3a200d01b0 con 0x7f3a2006df90 2024-09-10T23:34:03.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a1d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a2006e8b0 0x7f3a200b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:03.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a1d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a2006e8b0 0x7f3a200b5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47138/0 (socket says 172.21.15.94:47138) 2024-09-10T23:34:03.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a1d400640 1 -- 172.21.15.94:0/2765019514 learned_addr learned my addr 172.21.15.94:0/2765019514 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:03.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a1d400640 1 -- 172.21.15.94:0/2765019514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a2006df90 msgr2=0x7f3a20158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:03.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a1d400640 1 --2- 172.21.15.94:0/2765019514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a2006df90 0x7f3a20158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a1d400640 1 -- 172.21.15.94:0/2765019514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a200b1170 msgr2=0x7f3a2015d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:03.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a1d400640 1 --2- 172.21.15.94:0/2765019514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a200b1170 0x7f3a2015d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a1d400640 1 -- 172.21.15.94:0/2765019514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a20158740 con 0x7f3a2006e8b0 2024-09-10T23:34:03.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a1d400640 1 --2- 172.21.15.94:0/2765019514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a2006e8b0 0x7f3a200b5f10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0058d0 tx=0x7f3a0c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:03.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a16000640 1 -- 172.21.15.94:0/2765019514 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c017070 con 0x7f3a2006e8b0 2024-09-10T23:34:03.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a16000640 1 -- 172.21.15.94:0/2765019514 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a0c009d20 con 0x7f3a2006e8b0 2024-09-10T23:34:03.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a24e00640 1 -- 172.21.15.94:0/2765019514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2015df40 con 0x7f3a2006e8b0 2024-09-10T23:34:03.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a16000640 1 -- 172.21.15.94:0/2765019514 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0106c0 con 0x7f3a2006e8b0 2024-09-10T23:34:03.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a24e00640 1 -- 172.21.15.94:0/2765019514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a200b5710 con 0x7f3a2006e8b0 2024-09-10T23:34:03.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.224+0000 7f3a24e00640 1 -- 172.21.15.94:0/2765019514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a200b5520 con 0x7f3a2006e8b0 2024-09-10T23:34:03.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.228+0000 7f3a16000640 1 -- 172.21.15.94:0/2765019514 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f3a0c010860 con 0x7f3a2006e8b0 2024-09-10T23:34:03.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.228+0000 7f3a16000640 1 --2- 172.21.15.94:0/2765019514 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3a00072790 0x7f3a00074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:03.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.228+0000 7f3a16000640 1 -- 172.21.15.94:0/2765019514 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f3a0c0939f0 con 0x7f3a2006e8b0 2024-09-10T23:34:03.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.228+0000 7f3a1de00640 1 --2- 172.21.15.94:0/2765019514 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3a00072790 0x7f3a00074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:03.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.228+0000 7f3a1de00640 1 --2- 172.21.15.94:0/2765019514 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3a00072790 0x7f3a00074c50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3a100ae000 tx=0x7f3a10019380 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:03.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.232+0000 7f3a16000640 1 -- 172.21.15.94:0/2765019514 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3a0c095e20 con 0x7f3a2006e8b0 2024-09-10T23:34:03.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.428+0000 7f3a24e00640 1 -- 172.21.15.94:0/2765019514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a200d07a0 con 0x7f3a2006e8b0 2024-09-10T23:34:03.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.428+0000 7f3a16000640 1 -- 172.21.15.94:0/2765019514 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5235 (secure 0 0 0) 0x7f3a0c05edb0 con 0x7f3a2006e8b0 2024-09-10T23:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"btime":"2024-09-10T23:34:03:053056+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:34:03.434 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-09-10T23:34:03.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.432+0000 7f3a24e00640 1 -- 172.21.15.94:0/2765019514 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3a00072790 msgr2=0x7f3a00074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:03.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.432+0000 7f3a24e00640 1 --2- 172.21.15.94:0/2765019514 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3a00072790 0x7f3a00074c50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3a100ae000 tx=0x7f3a10019380 comp rx=0 tx=0).stop 2024-09-10T23:34:03.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.432+0000 7f3a24e00640 1 -- 172.21.15.94:0/2765019514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a2006e8b0 msgr2=0x7f3a200b5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:03.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.432+0000 7f3a24e00640 1 --2- 172.21.15.94:0/2765019514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a2006e8b0 0x7f3a200b5f10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0058d0 tx=0x7f3a0c00dde0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.432+0000 7f3a24e00640 1 -- 172.21.15.94:0/2765019514 shutdown_connections 2024-09-10T23:34:03.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.432+0000 7f3a24e00640 1 --2- 172.21.15.94:0/2765019514 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3a00072790 0x7f3a00074c50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.432+0000 7f3a24e00640 1 --2- 172.21.15.94:0/2765019514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a200b1170 0x7f3a2015d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.432+0000 7f3a24e00640 1 --2- 172.21.15.94:0/2765019514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a2006e8b0 0x7f3a200b5f10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.432+0000 7f3a24e00640 1 --2- 172.21.15.94:0/2765019514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a2006df90 0x7f3a20158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.432+0000 7f3a24e00640 1 -- 172.21.15.94:0/2765019514 >> 172.21.15.94:0/2765019514 conn(0x7f3a200d1c80 msgr2=0x7f3a200b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:03.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.432+0000 7f3a24e00640 1 -- 172.21.15.94:0/2765019514 shutdown_connections 2024-09-10T23:34:03.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.432+0000 7f3a24e00640 1 -- 172.21.15.94:0/2765019514 wait complete. 2024-09-10T23:34:03.456 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:34:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.592+0000 7fa3dee00640 1 -- 172.21.15.94:0/556176680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d8070150 msgr2=0x7fa3d80b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.592+0000 7fa3dee00640 1 --2- 172.21.15.94:0/556176680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d8070150 0x7fa3d80b35c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009bc0 tx=0x7fa3d001ca80 comp rx=0 tx=0).stop 2024-09-10T23:34:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.592+0000 7fa3dee00640 1 -- 172.21.15.94:0/556176680 shutdown_connections 2024-09-10T23:34:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.592+0000 7fa3dee00640 1 --2- 172.21.15.94:0/556176680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d8070150 0x7fa3d80b35c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.592+0000 7fa3dee00640 1 --2- 172.21.15.94:0/556176680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3d806f600 0x7fa3d8072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.592+0000 7fa3dee00640 1 -- 172.21.15.94:0/556176680 >> 172.21.15.94:0/556176680 conn(0x7fa3d806cdf0 msgr2=0x7fa3d806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:03.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.592+0000 7fa3dee00640 1 -- 172.21.15.94:0/556176680 shutdown_connections 2024-09-10T23:34:03.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.592+0000 7fa3dee00640 1 -- 172.21.15.94:0/556176680 wait complete. 2024-09-10T23:34:03.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3dee00640 1 Processor -- start 2024-09-10T23:34:03.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3dee00640 1 -- start start 2024-09-10T23:34:03.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3dee00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d806f600 0x7fa3d81638e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:03.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3dee00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3d8070150 0x7fa3d8142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:03.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3dee00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3d8157c60 0x7fa3d81f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:03.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3dee00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d815b310 con 0x7fa3d806f600 2024-09-10T23:34:03.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3dee00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d815b190 con 0x7fa3d8070150 2024-09-10T23:34:03.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3dee00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d815b490 con 0x7fa3d8157c60 2024-09-10T23:34:03.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3d7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d806f600 0x7fa3d81638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:03.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3d7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d806f600 0x7fa3d81638e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47154/0 (socket says 172.21.15.94:47154) 2024-09-10T23:34:03.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3d7e00640 1 -- 172.21.15.94:0/2002980927 learned_addr learned my addr 172.21.15.94:0/2002980927 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:03.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3dce00640 1 --2- 172.21.15.94:0/2002980927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3d8157c60 0x7fa3d81f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:03.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3d7400640 1 --2- 172.21.15.94:0/2002980927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3d8070150 0x7fa3d8142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:03.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3d7e00640 1 -- 172.21.15.94:0/2002980927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3d8157c60 msgr2=0x7fa3d81f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:03.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3d7e00640 1 --2- 172.21.15.94:0/2002980927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3d8157c60 0x7fa3d81f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3d7e00640 1 -- 172.21.15.94:0/2002980927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3d8070150 msgr2=0x7fa3d8142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:03.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3d7e00640 1 --2- 172.21.15.94:0/2002980927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3d8070150 0x7fa3d8142530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3d7e00640 1 -- 172.21.15.94:0/2002980927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d80cd5e0 con 0x7fa3d806f600 2024-09-10T23:34:03.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3d7e00640 1 --2- 172.21.15.94:0/2002980927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d806f600 0x7fa3d81638e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8001ec0 tx=0x7fa3c800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:03.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3cfe00640 1 -- 172.21.15.94:0/2002980927 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8017070 con 0x7fa3d806f600 2024-09-10T23:34:03.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3cfe00640 1 -- 172.21.15.94:0/2002980927 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3c8003600 con 0x7fa3d806f600 2024-09-10T23:34:03.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3cfe00640 1 -- 172.21.15.94:0/2002980927 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8003a50 con 0x7fa3d806f600 2024-09-10T23:34:03.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3dee00640 1 -- 172.21.15.94:0/2002980927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d81f7dd0 con 0x7fa3d806f600 2024-09-10T23:34:03.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.596+0000 7fa3dee00640 1 -- 172.21.15.94:0/2002980927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d81f8390 con 0x7fa3d806f600 2024-09-10T23:34:03.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.600+0000 7fa3dee00640 1 -- 172.21.15.94:0/2002980927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d8072610 con 0x7fa3d806f600 2024-09-10T23:34:03.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.600+0000 7fa3cfe00640 1 -- 172.21.15.94:0/2002980927 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7fa3c8003bf0 con 0x7fa3d806f600 2024-09-10T23:34:03.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.600+0000 7fa3cfe00640 1 --2- 172.21.15.94:0/2002980927 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa3b8072790 0x7fa3b8074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:03.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.600+0000 7fa3cfe00640 1 -- 172.21.15.94:0/2002980927 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fa3c8092d40 con 0x7fa3d806f600 2024-09-10T23:34:03.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.600+0000 7fa3d7400640 1 --2- 172.21.15.94:0/2002980927 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa3b8072790 0x7fa3b8074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:03.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.604+0000 7fa3d7400640 1 --2- 172.21.15.94:0/2002980927 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa3b8072790 0x7fa3b8074c50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40058d0 tx=0x7fa3c4005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:03.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.604+0000 7fa3cfe00640 1 -- 172.21.15.94:0/2002980927 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa3c805dff0 con 0x7fa3d806f600 2024-09-10T23:34:03.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.768+0000 7fa3dee00640 1 -- 172.21.15.94:0/2002980927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3d806e7e0 con 0x7fa3d806f600 2024-09-10T23:34:03.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.772+0000 7fa3cfe00640 1 -- 172.21.15.94:0/2002980927 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16657 (secure 0 0 0) 0x7fa3c805d8f0 con 0x7fa3d806f600 2024-09-10T23:34:03.775 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:03.775 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":44,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:34:01.040569+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:33:57.113965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T23:33:58.384504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:34:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.772+0000 7fa3dee00640 1 -- 172.21.15.94:0/2002980927 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa3b8072790 msgr2=0x7fa3b8074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.772+0000 7fa3dee00640 1 --2- 172.21.15.94:0/2002980927 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa3b8072790 0x7fa3b8074c50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40058d0 tx=0x7fa3c4005860 comp rx=0 tx=0).stop 2024-09-10T23:34:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.772+0000 7fa3dee00640 1 -- 172.21.15.94:0/2002980927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d806f600 msgr2=0x7fa3d81638e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.772+0000 7fa3dee00640 1 --2- 172.21.15.94:0/2002980927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d806f600 0x7fa3d81638e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8001ec0 tx=0x7fa3c800c890 comp rx=0 tx=0).stop 2024-09-10T23:34:03.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.772+0000 7fa3dee00640 1 -- 172.21.15.94:0/2002980927 shutdown_connections 2024-09-10T23:34:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.772+0000 7fa3dee00640 1 --2- 172.21.15.94:0/2002980927 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa3b8072790 0x7fa3b8074c50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.772+0000 7fa3dee00640 1 --2- 172.21.15.94:0/2002980927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3d8070150 0x7fa3d8142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.772+0000 7fa3dee00640 1 --2- 172.21.15.94:0/2002980927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3d8157c60 0x7fa3d81f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.772+0000 7fa3dee00640 1 --2- 172.21.15.94:0/2002980927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d806f600 0x7fa3d81638e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.772+0000 7fa3dee00640 1 -- 172.21.15.94:0/2002980927 >> 172.21.15.94:0/2002980927 conn(0x7fa3d806cdf0 msgr2=0x7fa3d8156920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.772+0000 7fa3dee00640 1 -- 172.21.15.94:0/2002980927 shutdown_connections 2024-09-10T23:34:03.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.772+0000 7fa3dee00640 1 -- 172.21.15.94:0/2002980927 wait complete. 2024-09-10T23:34:03.798 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T23:34:03.798 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:34:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.932+0000 7f1447600640 1 -- 172.21.15.94:0/4158658835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f144015ba50 msgr2=0x7f14400b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.932+0000 7f1447600640 1 --2- 172.21.15.94:0/4158658835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f144015ba50 0x7f14400b4700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1430009840 tx=0x7f143001c790 comp rx=0 tx=0).stop 2024-09-10T23:34:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.932+0000 7f1447600640 1 -- 172.21.15.94:0/4158658835 shutdown_connections 2024-09-10T23:34:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.932+0000 7f1447600640 1 --2- 172.21.15.94:0/4158658835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14400b4c40 0x7f14400cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.932+0000 7f1447600640 1 --2- 172.21.15.94:0/4158658835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f144015ba50 0x7f14400b4700 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.932+0000 7f1447600640 1 -- 172.21.15.94:0/4158658835 >> 172.21.15.94:0/4158658835 conn(0x7f144006cdf0 msgr2=0x7f144006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1447600640 1 -- 172.21.15.94:0/4158658835 shutdown_connections 2024-09-10T23:34:03.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1447600640 1 -- 172.21.15.94:0/4158658835 wait complete. 2024-09-10T23:34:03.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1447600640 1 Processor -- start 2024-09-10T23:34:03.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1447600640 1 -- start start 2024-09-10T23:34:03.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1447600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14400b4c40 0x7f1440141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:03.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1447600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14400d01f0 0x7f144015d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:03.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1447600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f144015ba50 0x7f1440158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:03.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1447600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1440168be0 con 0x7f144015ba50 2024-09-10T23:34:03.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1447600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1440168a60 con 0x7f14400b4c40 2024-09-10T23:34:03.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1447600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1440168d60 con 0x7f14400d01f0 2024-09-10T23:34:03.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1445600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f144015ba50 0x7f1440158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:03.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1445600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f144015ba50 0x7f1440158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33912/0 (socket says 172.21.15.94:33912) 2024-09-10T23:34:03.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1445600640 1 -- 172.21.15.94:0/1256958032 learned_addr learned my addr 172.21.15.94:0/1256958032 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:03.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1444c00640 1 --2- 172.21.15.94:0/1256958032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14400b4c40 0x7f1440141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:03.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1437e00640 1 --2- 172.21.15.94:0/1256958032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14400d01f0 0x7f144015d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:03.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1445600640 1 -- 172.21.15.94:0/1256958032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14400d01f0 msgr2=0x7f144015d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:03.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1445600640 1 --2- 172.21.15.94:0/1256958032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14400d01f0 0x7f144015d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1445600640 1 -- 172.21.15.94:0/1256958032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14400b4c40 msgr2=0x7f1440141e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:03.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1445600640 1 --2- 172.21.15.94:0/1256958032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14400b4c40 0x7f1440141e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:03.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1445600640 1 -- 172.21.15.94:0/1256958032 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1440142380 con 0x7f144015ba50 2024-09-10T23:34:03.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1437e00640 1 --2- 172.21.15.94:0/1256958032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14400d01f0 0x7f144015d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:34:03.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1445600640 1 --2- 172.21.15.94:0/1256958032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f144015ba50 0x7f1440158120 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f143800c760 tx=0x7f143800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:03.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f143d600640 1 -- 172.21.15.94:0/1256958032 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1438003db0 con 0x7f144015ba50 2024-09-10T23:34:03.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f143d600640 1 -- 172.21.15.94:0/1256958032 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1438005070 con 0x7f144015ba50 2024-09-10T23:34:03.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f143d600640 1 -- 172.21.15.94:0/1256958032 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14380054c0 con 0x7f144015ba50 2024-09-10T23:34:03.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1447600640 1 -- 172.21.15.94:0/1256958032 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144015e110 con 0x7f144015ba50 2024-09-10T23:34:03.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.936+0000 7f1447600640 1 -- 172.21.15.94:0/1256958032 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14401fb090 con 0x7f144015ba50 2024-09-10T23:34:03.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.940+0000 7f1447600640 1 -- 172.21.15.94:0/1256958032 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14400cfcb0 con 0x7f144015ba50 2024-09-10T23:34:03.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.940+0000 7f143d600640 1 -- 172.21.15.94:0/1256958032 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f1438005660 con 0x7f144015ba50 2024-09-10T23:34:03.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.940+0000 7f143d600640 1 --2- 172.21.15.94:0/1256958032 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f141c0727c0 0x7f141c074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:03.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.940+0000 7f143d600640 1 -- 172.21.15.94:0/1256958032 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f14380934e0 con 0x7f144015ba50 2024-09-10T23:34:03.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.940+0000 7f1444c00640 1 --2- 172.21.15.94:0/1256958032 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f141c0727c0 0x7f141c074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:03.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.940+0000 7f1444c00640 1 --2- 172.21.15.94:0/1256958032 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f141c0727c0 0x7f141c074c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f14300ae000 tx=0x7f143001afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:03.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:03.944+0000 7f143d600640 1 -- 172.21.15.94:0/1256958032 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1440142380 con 0x7f144015ba50 2024-09-10T23:34:04.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.108+0000 7f1447600640 1 -- 172.21.15.94:0/1256958032 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1440151450 con 0x7f141c0727c0 2024-09-10T23:34:04.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.116+0000 7f143d600640 1 -- 172.21.15.94:0/1256958032 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7f1440151450 con 0x7f141c0727c0 2024-09-10T23:34:04.122 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:04.123 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:34:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.120+0000 7f1447600640 1 -- 172.21.15.94:0/1256958032 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f141c0727c0 msgr2=0x7f141c074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.120+0000 7f1447600640 1 --2- 172.21.15.94:0/1256958032 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f141c0727c0 0x7f141c074c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f14300ae000 tx=0x7f143001afd0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.120+0000 7f1447600640 1 -- 172.21.15.94:0/1256958032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f144015ba50 msgr2=0x7f1440158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.120+0000 7f1447600640 1 --2- 172.21.15.94:0/1256958032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f144015ba50 0x7f1440158120 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f143800c760 tx=0x7f143800cc20 comp rx=0 tx=0).stop 2024-09-10T23:34:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.120+0000 7f1447600640 1 -- 172.21.15.94:0/1256958032 shutdown_connections 2024-09-10T23:34:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.120+0000 7f1447600640 1 --2- 172.21.15.94:0/1256958032 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f141c0727c0 0x7f141c074c80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.120+0000 7f1447600640 1 --2- 172.21.15.94:0/1256958032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f144015ba50 0x7f1440158120 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.120+0000 7f1447600640 1 --2- 172.21.15.94:0/1256958032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14400d01f0 0x7f144015d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.120+0000 7f1447600640 1 --2- 172.21.15.94:0/1256958032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14400b4c40 0x7f1440141e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.120+0000 7f1447600640 1 -- 172.21.15.94:0/1256958032 >> 172.21.15.94:0/1256958032 conn(0x7f144006cdf0 msgr2=0x7f144015f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.120+0000 7f1447600640 1 -- 172.21.15.94:0/1256958032 shutdown_connections 2024-09-10T23:34:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.120+0000 7f1447600640 1 -- 172.21.15.94:0/1256958032 wait complete. 2024-09-10T23:34:04.142 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-09-10T23:34:03.360492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242160,"kb_used_data":5120,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732560,"statfs":{"total":773094113280,"available":772846141440,"internally_reserved":0,"allocated":5242880,"data_stored":2769892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.339417"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423181+0000","last_change":"2024-09-10T23:34:00.053526+0000","last_active":"2024-09-10T23:34:01.423181+0000","last_peered":"2024-09-10T23:34:01.423181+0000","last_clean":"2024-09-10T23:34:01.423181+0000","last_became_active":"2024-09-10T23:34:00.053281+0000","last_became_peered":"2024-09-10T23:34:00.053281+0000","last_unstale":"2024-09-10T23:34:01.423181+0000","last_undegraded":"2024-09-10T23:34:01.423181+0000","last_fullsized":"2024-09-10T23:34:01.423181+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:22.977489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.049182+0000","last_change":"2024-09-10T23:34:00.053927+0000","last_active":"2024-09-10T23:34:01.049182+0000","last_peered":"2024-09-10T23:34:01.049182+0000","last_clean":"2024-09-10T23:34:01.049182+0000","last_became_active":"2024-09-10T23:34:00.053417+0000","last_became_peered":"2024-09-10T23:34:00.053417+0000","last_unstale":"2024-09-10T23:34:01.049182+0000","last_undegraded":"2024-09-10T23:34:01.049182+0000","last_fullsized":"2024-09-10T23:34:01.049182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:18.510820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422786+0000","last_change":"2024-09-10T23:34:00.055977+0000","last_active":"2024-09-10T23:34:01.422786+0000","last_peered":"2024-09-10T23:34:01.422786+0000","last_clean":"2024-09-10T23:34:01.422786+0000","last_became_active":"2024-09-10T23:34:00.055710+0000","last_became_peered":"2024-09-10T23:34:00.055710+0000","last_unstale":"2024-09-10T23:34:01.422786+0000","last_undegraded":"2024-09-10T23:34:01.422786+0000","last_fullsized":"2024-09-10T23:34:01.422786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:05.760065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071293+0000","last_change":"2024-09-10T23:34:00.059234+0000","last_active":"2024-09-10T23:34:01.071293+0000","last_peered":"2024-09-10T23:34:01.071293+0000","last_clean":"2024-09-10T23:34:01.071293+0000","last_became_active":"2024-09-10T23:34:00.058896+0000","last_became_peered":"2024-09-10T23:34:00.058896+0000","last_unstale":"2024-09-10T23:34:01.071293+0000","last_undegraded":"2024-09-10T23:34:01.071293+0000","last_fullsized":"2024-09-10T23:34:01.071293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:44.401260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422656+0000","last_change":"2024-09-10T23:34:00.052621+0000","last_active":"2024-09-10T23:34:01.422656+0000","last_peered":"2024-09-10T23:34:01.422656+0000","last_clean":"2024-09-10T23:34:01.422656+0000","last_became_active":"2024-09-10T23:34:00.052321+0000","last_became_peered":"2024-09-10T23:34:00.052321+0000","last_unstale":"2024-09-10T23:34:01.422656+0000","last_undegraded":"2024-09-10T23:34:01.422656+0000","last_fullsized":"2024-09-10T23:34:01.422656+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:20.750358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.070157+0000","last_change":"2024-09-10T23:34:00.062842+0000","last_active":"2024-09-10T23:34:00.070157+0000","last_peered":"2024-09-10T23:34:00.070157+0000","last_clean":"2024-09-10T23:34:00.070157+0000","last_became_active":"2024-09-10T23:34:00.062137+0000","last_became_peered":"2024-09-10T23:34:00.062137+0000","last_unstale":"2024-09-10T23:34:00.070157+0000","last_undegraded":"2024-09-10T23:34:00.070157+0000","last_fullsized":"2024-09-10T23:34:00.070157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:18.362114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073565+0000","last_change":"2024-09-10T23:34:00.059881+0000","last_active":"2024-09-10T23:34:01.073565+0000","last_peered":"2024-09-10T23:34:01.073565+0000","last_clean":"2024-09-10T23:34:01.073565+0000","last_became_active":"2024-09-10T23:34:00.059617+0000","last_became_peered":"2024-09-10T23:34:00.059617+0000","last_unstale":"2024-09-10T23:34:01.073565+0000","last_undegraded":"2024-09-10T23:34:01.073565+0000","last_fullsized":"2024-09-10T23:34:01.073565+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:22.314467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.070670+0000","last_change":"2024-09-10T23:34:00.058585+0000","last_active":"2024-09-10T23:34:01.070670+0000","last_peered":"2024-09-10T23:34:01.070670+0000","last_clean":"2024-09-10T23:34:01.070670+0000","last_became_active":"2024-09-10T23:34:00.053995+0000","last_became_peered":"2024-09-10T23:34:00.053995+0000","last_unstale":"2024-09-10T23:34:01.070670+0000","last_undegraded":"2024-09-10T23:34:01.070670+0000","last_fullsized":"2024-09-10T23:34:01.070670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:26.445051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.067789+0000","last_change":"2024-09-10T23:34:00.062526+0000","last_active":"2024-09-10T23:34:00.067789+0000","last_peered":"2024-09-10T23:34:00.067789+0000","last_clean":"2024-09-10T23:34:00.067789+0000","last_became_active":"2024-09-10T23:34:00.062232+0000","last_became_peered":"2024-09-10T23:34:00.062232+0000","last_unstale":"2024-09-10T23:34:00.067789+0000","last_undegraded":"2024-09-10T23:34:00.067789+0000","last_fullsized":"2024-09-10T23:34:00.067789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:44.129265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050157+0000","last_change":"2024-09-10T23:34:00.055510+0000","last_active":"2024-09-10T23:34:01.050157+0000","last_peered":"2024-09-10T23:34:01.050157+0000","last_clean":"2024-09-10T23:34:01.050157+0000","last_became_active":"2024-09-10T23:34:00.055184+0000","last_became_peered":"2024-09-10T23:34:00.055184+0000","last_unstale":"2024-09-10T23:34:01.050157+0000","last_undegraded":"2024-09-10T23:34:01.050157+0000","last_fullsized":"2024-09-10T23:34:01.050157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:43.177224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071369+0000","last_change":"2024-09-10T23:34:00.060147+0000","last_active":"2024-09-10T23:34:01.071369+0000","last_peered":"2024-09-10T23:34:01.071369+0000","last_clean":"2024-09-10T23:34:01.071369+0000","last_became_active":"2024-09-10T23:34:00.059867+0000","last_became_peered":"2024-09-10T23:34:00.059867+0000","last_unstale":"2024-09-10T23:34:01.071369+0000","last_undegraded":"2024-09-10T23:34:01.071369+0000","last_fullsized":"2024-09-10T23:34:01.071369+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:08.146583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.048943+0000","last_change":"2024-09-10T23:34:00.057451+0000","last_active":"2024-09-10T23:34:01.048943+0000","last_peered":"2024-09-10T23:34:01.048943+0000","last_clean":"2024-09-10T23:34:01.048943+0000","last_became_active":"2024-09-10T23:34:00.057182+0000","last_became_peered":"2024-09-10T23:34:00.057182+0000","last_unstale":"2024-09-10T23:34:01.048943+0000","last_undegraded":"2024-09-10T23:34:01.048943+0000","last_fullsized":"2024-09-10T23:34:01.048943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:56.900603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.067967+0000","last_change":"2024-09-10T23:34:00.062646+0000","last_active":"2024-09-10T23:34:00.067967+0000","last_peered":"2024-09-10T23:34:00.067967+0000","last_clean":"2024-09-10T23:34:00.067967+0000","last_became_active":"2024-09-10T23:34:00.062350+0000","last_became_peered":"2024-09-10T23:34:00.062350+0000","last_unstale":"2024-09-10T23:34:00.067967+0000","last_undegraded":"2024-09-10T23:34:00.067967+0000","last_fullsized":"2024-09-10T23:34:00.067967+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:16.401795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071169+0000","last_change":"2024-09-10T23:34:00.064303+0000","last_active":"2024-09-10T23:34:01.071169+0000","last_peered":"2024-09-10T23:34:01.071169+0000","last_clean":"2024-09-10T23:34:01.071169+0000","last_became_active":"2024-09-10T23:34:00.063991+0000","last_became_peered":"2024-09-10T23:34:00.063991+0000","last_unstale":"2024-09-10T23:34:01.071169+0000","last_undegraded":"2024-09-10T23:34:01.071169+0000","last_fullsized":"2024-09-10T23:34:01.071169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:29.633890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071139+0000","last_change":"2024-09-10T23:34:00.046958+0000","last_active":"2024-09-10T23:34:01.071139+0000","last_peered":"2024-09-10T23:34:01.071139+0000","last_clean":"2024-09-10T23:34:01.071139+0000","last_became_active":"2024-09-10T23:34:00.046679+0000","last_became_peered":"2024-09-10T23:34:00.046679+0000","last_unstale":"2024-09-10T23:34:01.071139+0000","last_undegraded":"2024-09-10T23:34:01.071139+0000","last_fullsized":"2024-09-10T23:34:01.071139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:10.360339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422292+0000","last_change":"2024-09-10T23:34:00.062267+0000","last_active":"2024-09-10T23:34:01.422292+0000","last_peered":"2024-09-10T23:34:01.422292+0000","last_clean":"2024-09-10T23:34:01.422292+0000","last_became_active":"2024-09-10T23:34:00.062006+0000","last_became_peered":"2024-09-10T23:34:00.062006+0000","last_unstale":"2024-09-10T23:34:01.422292+0000","last_undegraded":"2024-09-10T23:34:01.422292+0000","last_fullsized":"2024-09-10T23:34:01.422292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:47.199893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.049245+0000","last_change":"2024-09-10T23:34:00.044662+0000","last_active":"2024-09-10T23:34:01.049245+0000","last_peered":"2024-09-10T23:34:01.049245+0000","last_clean":"2024-09-10T23:34:01.049245+0000","last_became_active":"2024-09-10T23:34:00.044423+0000","last_became_peered":"2024-09-10T23:34:00.044423+0000","last_unstale":"2024-09-10T23:34:01.049245+0000","last_undegraded":"2024-09-10T23:34:01.049245+0000","last_fullsized":"2024-09-10T23:34:01.049245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:13.449304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422951+0000","last_change":"2024-09-10T23:34:00.059149+0000","last_active":"2024-09-10T23:34:01.422951+0000","last_peered":"2024-09-10T23:34:01.422951+0000","last_clean":"2024-09-10T23:34:01.422951+0000","last_became_active":"2024-09-10T23:34:00.058747+0000","last_became_peered":"2024-09-10T23:34:00.058747+0000","last_unstale":"2024-09-10T23:34:01.422951+0000","last_undegraded":"2024-09-10T23:34:01.422951+0000","last_fullsized":"2024-09-10T23:34:01.422951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:12.663251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423150+0000","last_change":"2024-09-10T23:34:00.054238+0000","last_active":"2024-09-10T23:34:01.423150+0000","last_peered":"2024-09-10T23:34:01.423150+0000","last_clean":"2024-09-10T23:34:01.423150+0000","last_became_active":"2024-09-10T23:34:00.053957+0000","last_became_peered":"2024-09-10T23:34:00.053957+0000","last_unstale":"2024-09-10T23:34:01.423150+0000","last_undegraded":"2024-09-10T23:34:01.423150+0000","last_fullsized":"2024-09-10T23:34:01.423150+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:10.438658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050827+0000","last_change":"2024-09-10T23:34:00.055856+0000","last_active":"2024-09-10T23:34:01.050827+0000","last_peered":"2024-09-10T23:34:01.050827+0000","last_clean":"2024-09-10T23:34:01.050827+0000","last_became_active":"2024-09-10T23:34:00.055504+0000","last_became_peered":"2024-09-10T23:34:00.055504+0000","last_unstale":"2024-09-10T23:34:01.050827+0000","last_undegraded":"2024-09-10T23:34:01.050827+0000","last_fullsized":"2024-09-10T23:34:01.050827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:04.255156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071288+0000","last_change":"2024-09-10T23:34:00.053573+0000","last_active":"2024-09-10T23:34:01.071288+0000","last_peered":"2024-09-10T23:34:01.071288+0000","last_clean":"2024-09-10T23:34:01.071288+0000","last_became_active":"2024-09-10T23:34:00.053032+0000","last_became_peered":"2024-09-10T23:34:00.053032+0000","last_unstale":"2024-09-10T23:34:01.071288+0000","last_undegraded":"2024-09-10T23:34:01.071288+0000","last_fullsized":"2024-09-10T23:34:01.071288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:57.789272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423183+0000","last_change":"2024-09-10T23:34:00.054671+0000","last_active":"2024-09-10T23:34:01.423183+0000","last_peered":"2024-09-10T23:34:01.423183+0000","last_clean":"2024-09-10T23:34:01.423183+0000","last_became_active":"2024-09-10T23:34:00.054019+0000","last_became_peered":"2024-09-10T23:34:00.054019+0000","last_unstale":"2024-09-10T23:34:01.423183+0000","last_undegraded":"2024-09-10T23:34:01.423183+0000","last_fullsized":"2024-09-10T23:34:01.423183+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:11.706227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073964+0000","last_change":"2024-09-10T23:34:00.048488+0000","last_active":"2024-09-10T23:34:01.073964+0000","last_peered":"2024-09-10T23:34:01.073964+0000","last_clean":"2024-09-10T23:34:01.073964+0000","last_became_active":"2024-09-10T23:34:00.047950+0000","last_became_peered":"2024-09-10T23:34:00.047950+0000","last_unstale":"2024-09-10T23:34:01.073964+0000","last_undegraded":"2024-09-10T23:34:01.073964+0000","last_fullsized":"2024-09-10T23:34:01.073964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:26.190775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.048704+0000","last_change":"2024-09-10T23:34:00.061492+0000","last_active":"2024-09-10T23:34:01.048704+0000","last_peered":"2024-09-10T23:34:01.048704+0000","last_clean":"2024-09-10T23:34:01.048704+0000","last_became_active":"2024-09-10T23:34:00.061247+0000","last_became_peered":"2024-09-10T23:34:00.061247+0000","last_unstale":"2024-09-10T23:34:01.048704+0000","last_undegraded":"2024-09-10T23:34:01.048704+0000","last_fullsized":"2024-09-10T23:34:01.048704+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:29.286908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423253+0000","last_change":"2024-09-10T23:34:00.053262+0000","last_active":"2024-09-10T23:34:01.423253+0000","last_peered":"2024-09-10T23:34:01.423253+0000","last_clean":"2024-09-10T23:34:01.423253+0000","last_became_active":"2024-09-10T23:34:00.053022+0000","last_became_peered":"2024-09-10T23:34:00.053022+0000","last_unstale":"2024-09-10T23:34:01.423253+0000","last_undegraded":"2024-09-10T23:34:01.423253+0000","last_fullsized":"2024-09-10T23:34:01.423253+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:18.510820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071133+0000","last_change":"2024-09-10T23:34:00.060558+0000","last_active":"2024-09-10T23:34:01.071133+0000","last_peered":"2024-09-10T23:34:01.071133+0000","last_clean":"2024-09-10T23:34:01.071133+0000","last_became_active":"2024-09-10T23:34:00.060283+0000","last_became_peered":"2024-09-10T23:34:00.060283+0000","last_unstale":"2024-09-10T23:34:01.071133+0000","last_undegraded":"2024-09-10T23:34:01.071133+0000","last_fullsized":"2024-09-10T23:34:01.071133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:03.679840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073920+0000","last_change":"2024-09-10T23:34:00.043105+0000","last_active":"2024-09-10T23:34:01.073920+0000","last_peered":"2024-09-10T23:34:01.073920+0000","last_clean":"2024-09-10T23:34:01.073920+0000","last_became_active":"2024-09-10T23:34:00.042916+0000","last_became_peered":"2024-09-10T23:34:00.042916+0000","last_unstale":"2024-09-10T23:34:01.073920+0000","last_undegraded":"2024-09-10T23:34:01.073920+0000","last_fullsized":"2024-09-10T23:34:01.073920+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:50.495799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073852+0000","last_change":"2024-09-10T23:34:00.047179+0000","last_active":"2024-09-10T23:34:01.073852+0000","last_peered":"2024-09-10T23:34:01.073852+0000","last_clean":"2024-09-10T23:34:01.073852+0000","last_became_active":"2024-09-10T23:34:00.046939+0000","last_became_peered":"2024-09-10T23:34:00.046939+0000","last_unstale":"2024-09-10T23:34:01.073852+0000","last_undegraded":"2024-09-10T23:34:01.073852+0000","last_fullsized":"2024-09-10T23:34:01.073852+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:01.848161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422876+0000","last_change":"2024-09-10T23:34:00.050273+0000","last_active":"2024-09-10T23:34:01.422876+0000","last_peered":"2024-09-10T23:34:01.422876+0000","last_clean":"2024-09-10T23:34:01.422876+0000","last_became_active":"2024-09-10T23:34:00.049951+0000","last_became_peered":"2024-09-10T23:34:00.049951+0000","last_unstale":"2024-09-10T23:34:01.422876+0000","last_undegraded":"2024-09-10T23:34:01.422876+0000","last_fullsized":"2024-09-10T23:34:01.422876+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:56.900603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.048726+0000","last_change":"2024-09-10T23:34:00.045331+0000","last_active":"2024-09-10T23:34:01.048726+0000","last_peered":"2024-09-10T23:34:01.048726+0000","last_clean":"2024-09-10T23:34:01.048726+0000","last_became_active":"2024-09-10T23:34:00.045065+0000","last_became_peered":"2024-09-10T23:34:00.045065+0000","last_unstale":"2024-09-10T23:34:01.048726+0000","last_undegraded":"2024-09-10T23:34:01.048726+0000","last_fullsized":"2024-09-10T23:34:01.048726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:47.199893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.062628+0000","last_change":"2024-09-10T23:34:00.056612+0000","last_active":"2024-09-10T23:34:00.062628+0000","last_peered":"2024-09-10T23:34:00.062628+0000","last_clean":"2024-09-10T23:34:00.062628+0000","last_became_active":"2024-09-10T23:34:00.054312+0000","last_became_peered":"2024-09-10T23:34:00.054312+0000","last_unstale":"2024-09-10T23:34:00.062628+0000","last_undegraded":"2024-09-10T23:34:00.062628+0000","last_fullsized":"2024-09-10T23:34:00.062628+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:41:09.300122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.070777+0000","last_change":"2024-09-10T23:34:00.062416+0000","last_active":"2024-09-10T23:34:01.070777+0000","last_peered":"2024-09-10T23:34:01.070777+0000","last_clean":"2024-09-10T23:34:01.070777+0000","last_became_active":"2024-09-10T23:34:00.061864+0000","last_became_peered":"2024-09-10T23:34:00.061864+0000","last_unstale":"2024-09-10T23:34:01.070777+0000","last_undegraded":"2024-09-10T23:34:01.070777+0000","last_fullsized":"2024-09-10T23:34:01.070777+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:31.362214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423359+0000","last_change":"2024-09-10T23:34:00.068583+0000","last_active":"2024-09-10T23:34:01.423359+0000","last_peered":"2024-09-10T23:34:01.423359+0000","last_clean":"2024-09-10T23:34:01.423359+0000","last_became_active":"2024-09-10T23:34:00.068312+0000","last_became_peered":"2024-09-10T23:34:00.068312+0000","last_unstale":"2024-09-10T23:34:01.423359+0000","last_undegraded":"2024-09-10T23:34:01.423359+0000","last_fullsized":"2024-09-10T23:34:01.423359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:12.663251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.083066+0000","last_change":"2024-09-10T23:33:59.058670+0000","last_active":"2024-09-10T23:34:00.083066+0000","last_peered":"2024-09-10T23:34:00.083066+0000","last_clean":"2024-09-10T23:34:00.083066+0000","last_became_active":"2024-09-10T23:33:59.058416+0000","last_became_peered":"2024-09-10T23:33:59.058416+0000","last_unstale":"2024-09-10T23:34:00.083066+0000","last_undegraded":"2024-09-10T23:34:00.083066+0000","last_fullsized":"2024-09-10T23:34:00.083066+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:51.475453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423306+0000","last_change":"2024-09-10T23:34:00.054143+0000","last_active":"2024-09-10T23:34:01.423306+0000","last_peered":"2024-09-10T23:34:01.423306+0000","last_clean":"2024-09-10T23:34:01.423306+0000","last_became_active":"2024-09-10T23:34:00.053959+0000","last_became_peered":"2024-09-10T23:34:00.053959+0000","last_unstale":"2024-09-10T23:34:01.423306+0000","last_undegraded":"2024-09-10T23:34:01.423306+0000","last_fullsized":"2024-09-10T23:34:01.423306+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:13.449304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073773+0000","last_change":"2024-09-10T23:33:59.047976+0000","last_active":"2024-09-10T23:34:01.073773+0000","last_peered":"2024-09-10T23:34:01.073773+0000","last_clean":"2024-09-10T23:34:01.073773+0000","last_became_active":"2024-09-10T23:33:59.047659+0000","last_became_peered":"2024-09-10T23:33:59.047659+0000","last_unstale":"2024-09-10T23:34:01.073773+0000","last_undegraded":"2024-09-10T23:34:01.073773+0000","last_fullsized":"2024-09-10T23:34:01.073773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:14.745642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.049002+0000","last_change":"2024-09-10T23:34:00.056027+0000","last_active":"2024-09-10T23:34:01.049002+0000","last_peered":"2024-09-10T23:34:01.049002+0000","last_clean":"2024-09-10T23:34:01.049002+0000","last_became_active":"2024-09-10T23:34:00.055785+0000","last_became_peered":"2024-09-10T23:34:00.055785+0000","last_unstale":"2024-09-10T23:34:01.049002+0000","last_undegraded":"2024-09-10T23:34:01.049002+0000","last_fullsized":"2024-09-10T23:34:01.049002+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:10.438658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.082123+0000","last_change":"2024-09-10T23:33:59.058781+0000","last_active":"2024-09-10T23:34:00.082123+0000","last_peered":"2024-09-10T23:34:00.082123+0000","last_clean":"2024-09-10T23:34:00.082123+0000","last_became_active":"2024-09-10T23:33:59.058451+0000","last_became_peered":"2024-09-10T23:33:59.058451+0000","last_unstale":"2024-09-10T23:34:00.082123+0000","last_undegraded":"2024-09-10T23:34:00.082123+0000","last_fullsized":"2024-09-10T23:34:00.082123+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:58.333956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071247+0000","last_change":"2024-09-10T23:34:00.059235+0000","last_active":"2024-09-10T23:34:01.071247+0000","last_peered":"2024-09-10T23:34:01.071247+0000","last_clean":"2024-09-10T23:34:01.071247+0000","last_became_active":"2024-09-10T23:34:00.058953+0000","last_became_peered":"2024-09-10T23:34:00.058953+0000","last_unstale":"2024-09-10T23:34:01.071247+0000","last_undegraded":"2024-09-10T23:34:01.071247+0000","last_fullsized":"2024-09-10T23:34:01.071247+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:34.439228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073838+0000","last_change":"2024-09-10T23:33:59.045197+0000","last_active":"2024-09-10T23:34:01.073838+0000","last_peered":"2024-09-10T23:34:01.073838+0000","last_clean":"2024-09-10T23:34:01.073838+0000","last_became_active":"2024-09-10T23:33:59.044673+0000","last_became_peered":"2024-09-10T23:33:59.044673+0000","last_unstale":"2024-09-10T23:34:01.073838+0000","last_undegraded":"2024-09-10T23:34:01.073838+0000","last_fullsized":"2024-09-10T23:34:01.073838+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:05.832407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073675+0000","last_change":"2024-09-10T23:34:00.044065+0000","last_active":"2024-09-10T23:34:01.073675+0000","last_peered":"2024-09-10T23:34:01.073675+0000","last_clean":"2024-09-10T23:34:01.073675+0000","last_became_active":"2024-09-10T23:34:00.042623+0000","last_became_peered":"2024-09-10T23:34:00.042623+0000","last_unstale":"2024-09-10T23:34:01.073675+0000","last_undegraded":"2024-09-10T23:34:01.073675+0000","last_fullsized":"2024-09-10T23:34:01.073675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:40.285835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.082000+0000","last_change":"2024-09-10T23:33:59.050742+0000","last_active":"2024-09-10T23:34:00.082000+0000","last_peered":"2024-09-10T23:34:00.082000+0000","last_clean":"2024-09-10T23:34:00.082000+0000","last_became_active":"2024-09-10T23:33:59.050451+0000","last_became_peered":"2024-09-10T23:33:59.050451+0000","last_unstale":"2024-09-10T23:34:00.082000+0000","last_undegraded":"2024-09-10T23:34:00.082000+0000","last_fullsized":"2024-09-10T23:34:00.082000+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:43.648471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.049993+0000","last_change":"2024-09-10T23:34:00.066186+0000","last_active":"2024-09-10T23:34:01.049993+0000","last_peered":"2024-09-10T23:34:01.049993+0000","last_clean":"2024-09-10T23:34:01.049993+0000","last_became_active":"2024-09-10T23:34:00.065935+0000","last_became_peered":"2024-09-10T23:34:00.065935+0000","last_unstale":"2024-09-10T23:34:01.049993+0000","last_undegraded":"2024-09-10T23:34:01.049993+0000","last_fullsized":"2024-09-10T23:34:01.049993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:37.105867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073611+0000","last_change":"2024-09-10T23:33:59.053039+0000","last_active":"2024-09-10T23:34:01.073611+0000","last_peered":"2024-09-10T23:34:01.073611+0000","last_clean":"2024-09-10T23:34:01.073611+0000","last_became_active":"2024-09-10T23:33:59.052753+0000","last_became_peered":"2024-09-10T23:33:59.052753+0000","last_unstale":"2024-09-10T23:34:01.073611+0000","last_undegraded":"2024-09-10T23:34:01.073611+0000","last_fullsized":"2024-09-10T23:34:01.073611+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:38.777776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050550+0000","last_change":"2024-09-10T23:34:00.057716+0000","last_active":"2024-09-10T23:34:01.050550+0000","last_peered":"2024-09-10T23:34:01.050550+0000","last_clean":"2024-09-10T23:34:01.050550+0000","last_became_active":"2024-09-10T23:34:00.057328+0000","last_became_peered":"2024-09-10T23:34:00.057328+0000","last_unstale":"2024-09-10T23:34:01.050550+0000","last_undegraded":"2024-09-10T23:34:01.050550+0000","last_fullsized":"2024-09-10T23:34:01.050550+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:38.030151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422277+0000","last_change":"2024-09-10T23:33:59.049392+0000","last_active":"2024-09-10T23:34:01.422277+0000","last_peered":"2024-09-10T23:34:01.422277+0000","last_clean":"2024-09-10T23:34:01.422277+0000","last_became_active":"2024-09-10T23:33:59.049115+0000","last_became_peered":"2024-09-10T23:33:59.049115+0000","last_unstale":"2024-09-10T23:34:01.422277+0000","last_undegraded":"2024-09-10T23:34:01.422277+0000","last_fullsized":"2024-09-10T23:34:01.422277+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:23.329966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073580+0000","last_change":"2024-09-10T23:34:00.048785+0000","last_active":"2024-09-10T23:34:01.073580+0000","last_peered":"2024-09-10T23:34:01.073580+0000","last_clean":"2024-09-10T23:34:01.073580+0000","last_became_active":"2024-09-10T23:34:00.048260+0000","last_became_peered":"2024-09-10T23:34:00.048260+0000","last_unstale":"2024-09-10T23:34:01.073580+0000","last_undegraded":"2024-09-10T23:34:01.073580+0000","last_fullsized":"2024-09-10T23:34:01.073580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:25.014233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050604+0000","last_change":"2024-09-10T23:33:59.043494+0000","last_active":"2024-09-10T23:34:01.050604+0000","last_peered":"2024-09-10T23:34:01.050604+0000","last_clean":"2024-09-10T23:34:01.050604+0000","last_became_active":"2024-09-10T23:33:59.043195+0000","last_became_peered":"2024-09-10T23:33:59.043195+0000","last_unstale":"2024-09-10T23:34:01.050604+0000","last_undegraded":"2024-09-10T23:34:01.050604+0000","last_fullsized":"2024-09-10T23:34:01.050604+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:19.743920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422954+0000","last_change":"2024-09-10T23:34:00.056081+0000","last_active":"2024-09-10T23:34:01.422954+0000","last_peered":"2024-09-10T23:34:01.422954+0000","last_clean":"2024-09-10T23:34:01.422954+0000","last_became_active":"2024-09-10T23:34:00.055192+0000","last_became_peered":"2024-09-10T23:34:00.055192+0000","last_unstale":"2024-09-10T23:34:01.422954+0000","last_undegraded":"2024-09-10T23:34:01.422954+0000","last_fullsized":"2024-09-10T23:34:01.422954+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:45:06.264461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423030+0000","last_change":"2024-09-10T23:33:59.050927+0000","last_active":"2024-09-10T23:34:01.423030+0000","last_peered":"2024-09-10T23:34:01.423030+0000","last_clean":"2024-09-10T23:34:01.423030+0000","last_became_active":"2024-09-10T23:33:59.050620+0000","last_became_peered":"2024-09-10T23:33:59.050620+0000","last_unstale":"2024-09-10T23:34:01.423030+0000","last_undegraded":"2024-09-10T23:34:01.423030+0000","last_fullsized":"2024-09-10T23:34:01.423030+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:33.961437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422942+0000","last_change":"2024-09-10T23:34:00.050339+0000","last_active":"2024-09-10T23:34:01.422942+0000","last_peered":"2024-09-10T23:34:01.422942+0000","last_clean":"2024-09-10T23:34:01.422942+0000","last_became_active":"2024-09-10T23:34:00.047535+0000","last_became_peered":"2024-09-10T23:34:00.047535+0000","last_unstale":"2024-09-10T23:34:01.422942+0000","last_undegraded":"2024-09-10T23:34:01.422942+0000","last_fullsized":"2024-09-10T23:34:01.422942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:12.665202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050219+0000","last_change":"2024-09-10T23:33:59.049654+0000","last_active":"2024-09-10T23:34:01.050219+0000","last_peered":"2024-09-10T23:34:01.050219+0000","last_clean":"2024-09-10T23:34:01.050219+0000","last_became_active":"2024-09-10T23:33:59.049316+0000","last_became_peered":"2024-09-10T23:33:59.049316+0000","last_unstale":"2024-09-10T23:34:01.050219+0000","last_undegraded":"2024-09-10T23:34:01.050219+0000","last_fullsized":"2024-09-10T23:34:01.050219+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:17.504383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.070922+0000","last_change":"2024-09-10T23:34:00.064500+0000","last_active":"2024-09-10T23:34:01.070922+0000","last_peered":"2024-09-10T23:34:01.070922+0000","last_clean":"2024-09-10T23:34:01.070922+0000","last_became_active":"2024-09-10T23:34:00.064040+0000","last_became_peered":"2024-09-10T23:34:00.064040+0000","last_unstale":"2024-09-10T23:34:01.070922+0000","last_undegraded":"2024-09-10T23:34:01.070922+0000","last_fullsized":"2024-09-10T23:34:01.070922+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:22.977489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.048940+0000","last_change":"2024-09-10T23:33:59.052690+0000","last_active":"2024-09-10T23:34:01.048940+0000","last_peered":"2024-09-10T23:34:01.048940+0000","last_clean":"2024-09-10T23:34:01.048940+0000","last_became_active":"2024-09-10T23:33:59.052424+0000","last_became_peered":"2024-09-10T23:33:59.052424+0000","last_unstale":"2024-09-10T23:34:01.048940+0000","last_undegraded":"2024-09-10T23:34:01.048940+0000","last_fullsized":"2024-09-10T23:34:01.048940+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:23.329966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073988+0000","last_change":"2024-09-10T23:34:00.062569+0000","last_active":"2024-09-10T23:34:01.073988+0000","last_peered":"2024-09-10T23:34:01.073988+0000","last_clean":"2024-09-10T23:34:01.073988+0000","last_became_active":"2024-09-10T23:34:00.062326+0000","last_became_peered":"2024-09-10T23:34:00.062326+0000","last_unstale":"2024-09-10T23:34:01.073988+0000","last_undegraded":"2024-09-10T23:34:01.073988+0000","last_fullsized":"2024-09-10T23:34:01.073988+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:52.456744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":21,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050887+0000","last_change":"2024-09-10T23:33:59.043985+0000","last_active":"2024-09-10T23:34:01.050887+0000","last_peered":"2024-09-10T23:34:01.050887+0000","last_clean":"2024-09-10T23:34:01.050887+0000","last_became_active":"2024-09-10T23:33:59.043166+0000","last_became_peered":"2024-09-10T23:33:59.043166+0000","last_unstale":"2024-09-10T23:34:01.050887+0000","last_undegraded":"2024-09-10T23:34:01.050887+0000","last_fullsized":"2024-09-10T23:34:01.050887+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:12.442866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422606+0000","last_change":"2024-09-10T23:34:00.045511+0000","last_active":"2024-09-10T23:34:01.422606+0000","last_peered":"2024-09-10T23:34:01.422606+0000","last_clean":"2024-09-10T23:34:01.422606+0000","last_became_active":"2024-09-10T23:34:00.045251+0000","last_became_peered":"2024-09-10T23:34:00.045251+0000","last_unstale":"2024-09-10T23:34:01.422606+0000","last_undegraded":"2024-09-10T23:34:01.422606+0000","last_fullsized":"2024-09-10T23:34:01.422606+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:20.397390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423065+0000","last_change":"2024-09-10T23:33:59.047152+0000","last_active":"2024-09-10T23:34:01.423065+0000","last_peered":"2024-09-10T23:34:01.423065+0000","last_clean":"2024-09-10T23:34:01.423065+0000","last_became_active":"2024-09-10T23:33:59.046538+0000","last_became_peered":"2024-09-10T23:33:59.046538+0000","last_unstale":"2024-09-10T23:34:01.423065+0000","last_undegraded":"2024-09-10T23:34:01.423065+0000","last_fullsized":"2024-09-10T23:34:01.423065+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:32:24.262803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423174+0000","last_change":"2024-09-10T23:34:00.056643+0000","last_active":"2024-09-10T23:34:01.423174+0000","last_peered":"2024-09-10T23:34:01.423174+0000","last_clean":"2024-09-10T23:34:01.423174+0000","last_became_active":"2024-09-10T23:34:00.056427+0000","last_became_peered":"2024-09-10T23:34:00.056427+0000","last_unstale":"2024-09-10T23:34:01.423174+0000","last_undegraded":"2024-09-10T23:34:01.423174+0000","last_fullsized":"2024-09-10T23:34:01.423174+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:05.760065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422361+0000","last_change":"2024-09-10T23:33:59.050066+0000","last_active":"2024-09-10T23:34:01.422361+0000","last_peered":"2024-09-10T23:34:01.422361+0000","last_clean":"2024-09-10T23:34:01.422361+0000","last_became_active":"2024-09-10T23:33:59.049802+0000","last_became_peered":"2024-09-10T23:33:59.049802+0000","last_unstale":"2024-09-10T23:34:01.422361+0000","last_undegraded":"2024-09-10T23:34:01.422361+0000","last_fullsized":"2024-09-10T23:34:01.422361+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:24.302911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423113+0000","last_change":"2024-09-10T23:34:00.068659+0000","last_active":"2024-09-10T23:34:01.423113+0000","last_peered":"2024-09-10T23:34:01.423113+0000","last_clean":"2024-09-10T23:34:01.423113+0000","last_became_active":"2024-09-10T23:34:00.068373+0000","last_became_peered":"2024-09-10T23:34:00.068373+0000","last_unstale":"2024-09-10T23:34:01.423113+0000","last_undegraded":"2024-09-10T23:34:01.423113+0000","last_fullsized":"2024-09-10T23:34:01.423113+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:20.397390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.045996+0000","last_change":"2024-09-10T23:33:59.046524+0000","last_active":"2024-09-10T23:34:00.045996+0000","last_peered":"2024-09-10T23:34:00.045996+0000","last_clean":"2024-09-10T23:34:00.045996+0000","last_became_active":"2024-09-10T23:33:59.046257+0000","last_became_peered":"2024-09-10T23:33:59.046257+0000","last_unstale":"2024-09-10T23:34:00.045996+0000","last_undegraded":"2024-09-10T23:34:00.045996+0000","last_fullsized":"2024-09-10T23:34:00.045996+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:14.049866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.048712+0000","last_change":"2024-09-10T23:34:00.048486+0000","last_active":"2024-09-10T23:34:01.048712+0000","last_peered":"2024-09-10T23:34:01.048712+0000","last_clean":"2024-09-10T23:34:01.048712+0000","last_became_active":"2024-09-10T23:34:00.046456+0000","last_became_peered":"2024-09-10T23:34:00.046456+0000","last_unstale":"2024-09-10T23:34:01.048712+0000","last_undegraded":"2024-09-10T23:34:01.048712+0000","last_fullsized":"2024-09-10T23:34:01.048712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:26.445051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":23,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050653+0000","last_change":"2024-09-10T23:33:59.040987+0000","last_active":"2024-09-10T23:34:01.050653+0000","last_peered":"2024-09-10T23:34:01.050653+0000","last_clean":"2024-09-10T23:34:01.050653+0000","last_became_active":"2024-09-10T23:33:59.040648+0000","last_became_peered":"2024-09-10T23:33:59.040648+0000","last_unstale":"2024-09-10T23:34:01.050653+0000","last_undegraded":"2024-09-10T23:34:01.050653+0000","last_fullsized":"2024-09-10T23:34:01.050653+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:55.894165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423477+0000","last_change":"2024-09-10T23:34:00.054755+0000","last_active":"2024-09-10T23:34:01.423477+0000","last_peered":"2024-09-10T23:34:01.423477+0000","last_clean":"2024-09-10T23:34:01.423477+0000","last_became_active":"2024-09-10T23:34:00.054457+0000","last_became_peered":"2024-09-10T23:34:00.054457+0000","last_unstale":"2024-09-10T23:34:01.423477+0000","last_undegraded":"2024-09-10T23:34:01.423477+0000","last_fullsized":"2024-09-10T23:34:01.423477+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:04.985082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071056+0000","last_change":"2024-09-10T23:33:59.051243+0000","last_active":"2024-09-10T23:34:01.071056+0000","last_peered":"2024-09-10T23:34:01.071056+0000","last_clean":"2024-09-10T23:34:01.071056+0000","last_became_active":"2024-09-10T23:33:59.050869+0000","last_became_peered":"2024-09-10T23:33:59.050869+0000","last_unstale":"2024-09-10T23:34:01.071056+0000","last_undegraded":"2024-09-10T23:34:01.071056+0000","last_fullsized":"2024-09-10T23:34:01.071056+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:56:46.762024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423508+0000","last_change":"2024-09-10T23:34:00.075358+0000","last_active":"2024-09-10T23:34:01.423508+0000","last_peered":"2024-09-10T23:34:01.423508+0000","last_clean":"2024-09-10T23:34:01.423508+0000","last_became_active":"2024-09-10T23:34:00.072180+0000","last_became_peered":"2024-09-10T23:34:00.072180+0000","last_unstale":"2024-09-10T23:34:01.423508+0000","last_undegraded":"2024-09-10T23:34:01.423508+0000","last_fullsized":"2024-09-10T23:34:01.423508+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:45:06.264461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423040+0000","last_change":"2024-09-10T23:33:59.046310+0000","last_active":"2024-09-10T23:34:01.423040+0000","last_peered":"2024-09-10T23:34:01.423040+0000","last_clean":"2024-09-10T23:34:01.423040+0000","last_became_active":"2024-09-10T23:33:59.046022+0000","last_became_peered":"2024-09-10T23:33:59.046022+0000","last_unstale":"2024-09-10T23:34:01.423040+0000","last_undegraded":"2024-09-10T23:34:01.423040+0000","last_fullsized":"2024-09-10T23:34:01.423040+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:31:08.681878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.049130+0000","last_change":"2024-09-10T23:34:00.061374+0000","last_active":"2024-09-10T23:34:01.049130+0000","last_peered":"2024-09-10T23:34:01.049130+0000","last_clean":"2024-09-10T23:34:01.049130+0000","last_became_active":"2024-09-10T23:34:00.060549+0000","last_became_peered":"2024-09-10T23:34:00.060549+0000","last_unstale":"2024-09-10T23:34:01.049130+0000","last_undegraded":"2024-09-10T23:34:01.049130+0000","last_fullsized":"2024-09-10T23:34:01.049130+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:22.977489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422891+0000","last_change":"2024-09-10T23:33:59.054491+0000","last_active":"2024-09-10T23:34:01.422891+0000","last_peered":"2024-09-10T23:34:01.422891+0000","last_clean":"2024-09-10T23:34:01.422891+0000","last_became_active":"2024-09-10T23:33:59.054184+0000","last_became_peered":"2024-09-10T23:33:59.054184+0000","last_unstale":"2024-09-10T23:34:01.422891+0000","last_undegraded":"2024-09-10T23:34:01.422891+0000","last_fullsized":"2024-09-10T23:34:01.422891+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:36.722335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422809+0000","last_change":"2024-09-10T23:34:00.056435+0000","last_active":"2024-09-10T23:34:01.422809+0000","last_peered":"2024-09-10T23:34:01.422809+0000","last_clean":"2024-09-10T23:34:01.422809+0000","last_became_active":"2024-09-10T23:34:00.055481+0000","last_became_peered":"2024-09-10T23:34:00.055481+0000","last_unstale":"2024-09-10T23:34:01.422809+0000","last_undegraded":"2024-09-10T23:34:01.422809+0000","last_fullsized":"2024-09-10T23:34:01.422809+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:24.641710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.046180+0000","last_change":"2024-09-10T23:33:59.055487+0000","last_active":"2024-09-10T23:34:00.046180+0000","last_peered":"2024-09-10T23:34:00.046180+0000","last_clean":"2024-09-10T23:34:00.046180+0000","last_became_active":"2024-09-10T23:33:59.055219+0000","last_became_peered":"2024-09-10T23:33:59.055219+0000","last_unstale":"2024-09-10T23:34:00.046180+0000","last_undegraded":"2024-09-10T23:34:00.046180+0000","last_fullsized":"2024-09-10T23:34:00.046180+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:25.829230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050019+0000","last_change":"2024-09-10T23:34:00.068595+0000","last_active":"2024-09-10T23:34:01.050019+0000","last_peered":"2024-09-10T23:34:01.050019+0000","last_clean":"2024-09-10T23:34:01.050019+0000","last_became_active":"2024-09-10T23:34:00.068250+0000","last_became_peered":"2024-09-10T23:34:00.068250+0000","last_unstale":"2024-09-10T23:34:01.050019+0000","last_undegraded":"2024-09-10T23:34:01.050019+0000","last_fullsized":"2024-09-10T23:34:01.050019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:41.128499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050106+0000","last_change":"2024-09-10T23:33:59.044422+0000","last_active":"2024-09-10T23:34:01.050106+0000","last_peered":"2024-09-10T23:34:01.050106+0000","last_clean":"2024-09-10T23:34:01.050106+0000","last_became_active":"2024-09-10T23:33:59.044062+0000","last_became_peered":"2024-09-10T23:33:59.044062+0000","last_unstale":"2024-09-10T23:34:01.050106+0000","last_undegraded":"2024-09-10T23:34:01.050106+0000","last_fullsized":"2024-09-10T23:34:01.050106+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:21.971052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.053492+0000","last_change":"2024-09-10T23:34:00.047006+0000","last_active":"2024-09-10T23:34:00.053492+0000","last_peered":"2024-09-10T23:34:00.053492+0000","last_clean":"2024-09-10T23:34:00.053492+0000","last_became_active":"2024-09-10T23:34:00.046787+0000","last_became_peered":"2024-09-10T23:34:00.046787+0000","last_unstale":"2024-09-10T23:34:00.053492+0000","last_undegraded":"2024-09-10T23:34:00.053492+0000","last_fullsized":"2024-09-10T23:34:00.053492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:56.782316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422447+0000","last_change":"2024-09-10T23:33:59.055739+0000","last_active":"2024-09-10T23:34:01.422447+0000","last_peered":"2024-09-10T23:34:01.422447+0000","last_clean":"2024-09-10T23:34:01.422447+0000","last_became_active":"2024-09-10T23:33:59.054895+0000","last_became_peered":"2024-09-10T23:33:59.054895+0000","last_unstale":"2024-09-10T23:34:01.422447+0000","last_undegraded":"2024-09-10T23:34:01.422447+0000","last_fullsized":"2024-09-10T23:34:01.422447+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:43.182488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071027+0000","last_change":"2024-09-10T23:34:00.059283+0000","last_active":"2024-09-10T23:34:01.071027+0000","last_peered":"2024-09-10T23:34:01.071027+0000","last_clean":"2024-09-10T23:34:01.071027+0000","last_became_active":"2024-09-10T23:34:00.058940+0000","last_became_peered":"2024-09-10T23:34:00.058940+0000","last_unstale":"2024-09-10T23:34:01.071027+0000","last_undegraded":"2024-09-10T23:34:01.071027+0000","last_fullsized":"2024-09-10T23:34:01.071027+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:43.464171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.049837+0000","last_change":"2024-09-10T23:33:59.049904+0000","last_active":"2024-09-10T23:34:01.049837+0000","last_peered":"2024-09-10T23:34:01.049837+0000","last_clean":"2024-09-10T23:34:01.049837+0000","last_became_active":"2024-09-10T23:33:59.049325+0000","last_became_peered":"2024-09-10T23:33:59.049325+0000","last_unstale":"2024-09-10T23:34:01.049837+0000","last_undegraded":"2024-09-10T23:34:01.049837+0000","last_fullsized":"2024-09-10T23:34:01.049837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:09.432220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.070904+0000","last_change":"2024-09-10T23:34:00.064321+0000","last_active":"2024-09-10T23:34:01.070904+0000","last_peered":"2024-09-10T23:34:01.070904+0000","last_clean":"2024-09-10T23:34:01.070904+0000","last_became_active":"2024-09-10T23:34:00.063975+0000","last_became_peered":"2024-09-10T23:34:00.063975+0000","last_unstale":"2024-09-10T23:34:01.070904+0000","last_undegraded":"2024-09-10T23:34:01.070904+0000","last_fullsized":"2024-09-10T23:34:01.070904+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:34.191086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073436+0000","last_change":"2024-09-10T23:33:59.044716+0000","last_active":"2024-09-10T23:34:01.073436+0000","last_peered":"2024-09-10T23:34:01.073436+0000","last_clean":"2024-09-10T23:34:01.073436+0000","last_became_active":"2024-09-10T23:33:59.044365+0000","last_became_peered":"2024-09-10T23:33:59.044365+0000","last_unstale":"2024-09-10T23:34:01.073436+0000","last_undegraded":"2024-09-10T23:34:01.073436+0000","last_fullsized":"2024-09-10T23:34:01.073436+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:33.432790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073987+0000","last_change":"2024-09-10T23:34:00.051244+0000","last_active":"2024-09-10T23:34:01.073987+0000","last_peered":"2024-09-10T23:34:01.073987+0000","last_clean":"2024-09-10T23:34:01.073987+0000","last_became_active":"2024-09-10T23:34:00.050975+0000","last_became_peered":"2024-09-10T23:34:00.050975+0000","last_unstale":"2024-09-10T23:34:01.073987+0000","last_undegraded":"2024-09-10T23:34:01.073987+0000","last_fullsized":"2024-09-10T23:34:01.073987+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:09:53.083471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"44'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.072959+0000","last_change":"2024-09-10T23:33:59.052060+0000","last_active":"2024-09-10T23:34:01.072959+0000","last_peered":"2024-09-10T23:34:01.072959+0000","last_clean":"2024-09-10T23:34:01.072959+0000","last_became_active":"2024-09-10T23:33:59.051509+0000","last_became_peered":"2024-09-10T23:33:59.051509+0000","last_unstale":"2024-09-10T23:34:01.072959+0000","last_undegraded":"2024-09-10T23:34:01.072959+0000","last_fullsized":"2024-09-10T23:34:01.072959+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.926434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050942+0000","last_change":"2024-09-10T23:34:00.058061+0000","last_active":"2024-09-10T23:34:01.050942+0000","last_peered":"2024-09-10T23:34:01.050942+0000","last_clean":"2024-09-10T23:34:01.050942+0000","last_became_active":"2024-09-10T23:34:00.057723+0000","last_became_peered":"2024-09-10T23:34:00.057723+0000","last_unstale":"2024-09-10T23:34:01.050942+0000","last_undegraded":"2024-09-10T23:34:01.050942+0000","last_fullsized":"2024-09-10T23:34:01.050942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:46.755190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423062+0000","last_change":"2024-09-10T23:33:59.055246+0000","last_active":"2024-09-10T23:34:01.423062+0000","last_peered":"2024-09-10T23:34:01.423062+0000","last_clean":"2024-09-10T23:34:01.423062+0000","last_became_active":"2024-09-10T23:33:59.052293+0000","last_became_peered":"2024-09-10T23:33:59.052293+0000","last_unstale":"2024-09-10T23:34:01.423062+0000","last_undegraded":"2024-09-10T23:34:01.423062+0000","last_fullsized":"2024-09-10T23:34:01.423062+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:54.307188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.074107+0000","last_change":"2024-09-10T23:34:00.059855+0000","last_active":"2024-09-10T23:34:01.074107+0000","last_peered":"2024-09-10T23:34:01.074107+0000","last_clean":"2024-09-10T23:34:01.074107+0000","last_became_active":"2024-09-10T23:34:00.059589+0000","last_became_peered":"2024-09-10T23:34:00.059589+0000","last_unstale":"2024-09-10T23:34:01.074107+0000","last_undegraded":"2024-09-10T23:34:01.074107+0000","last_fullsized":"2024-09-10T23:34:01.074107+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:51.065061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423193+0000","last_change":"2024-09-10T23:33:59.044762+0000","last_active":"2024-09-10T23:34:01.423193+0000","last_peered":"2024-09-10T23:34:01.423193+0000","last_clean":"2024-09-10T23:34:01.423193+0000","last_became_active":"2024-09-10T23:33:59.044485+0000","last_became_peered":"2024-09-10T23:33:59.044485+0000","last_unstale":"2024-09-10T23:34:01.423193+0000","last_undegraded":"2024-09-10T23:34:01.423193+0000","last_fullsized":"2024-09-10T23:34:01.423193+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:28.285801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050189+0000","last_change":"2024-09-10T23:34:00.063321+0000","last_active":"2024-09-10T23:34:01.050189+0000","last_peered":"2024-09-10T23:34:01.050189+0000","last_clean":"2024-09-10T23:34:01.050189+0000","last_became_active":"2024-09-10T23:34:00.063046+0000","last_became_peered":"2024-09-10T23:34:00.063046+0000","last_unstale":"2024-09-10T23:34:01.050189+0000","last_undegraded":"2024-09-10T23:34:01.050189+0000","last_fullsized":"2024-09-10T23:34:01.050189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:57.628203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422766+0000","last_change":"2024-09-10T23:33:59.051537+0000","last_active":"2024-09-10T23:34:01.422766+0000","last_peered":"2024-09-10T23:34:01.422766+0000","last_clean":"2024-09-10T23:34:01.422766+0000","last_became_active":"2024-09-10T23:33:59.051244+0000","last_became_peered":"2024-09-10T23:33:59.051244+0000","last_unstale":"2024-09-10T23:34:01.422766+0000","last_undegraded":"2024-09-10T23:34:01.422766+0000","last_fullsized":"2024-09-10T23:34:01.422766+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:28.311294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.070910+0000","last_change":"2024-09-10T23:34:00.053297+0000","last_active":"2024-09-10T23:34:01.070910+0000","last_peered":"2024-09-10T23:34:01.070910+0000","last_clean":"2024-09-10T23:34:01.070910+0000","last_became_active":"2024-09-10T23:34:00.052705+0000","last_became_peered":"2024-09-10T23:34:00.052705+0000","last_unstale":"2024-09-10T23:34:01.070910+0000","last_undegraded":"2024-09-10T23:34:01.070910+0000","last_fullsized":"2024-09-10T23:34:01.070910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:10.438658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073764+0000","last_change":"2024-09-10T23:33:59.037368+0000","last_active":"2024-09-10T23:34:01.073764+0000","last_peered":"2024-09-10T23:34:01.073764+0000","last_clean":"2024-09-10T23:34:01.073764+0000","last_became_active":"2024-09-10T23:33:59.036540+0000","last_became_peered":"2024-09-10T23:33:59.036540+0000","last_unstale":"2024-09-10T23:34:01.073764+0000","last_undegraded":"2024-09-10T23:34:01.073764+0000","last_fullsized":"2024-09-10T23:34:01.073764+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:43.394823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422418+0000","last_change":"2024-09-10T23:34:00.050400+0000","last_active":"2024-09-10T23:34:01.422418+0000","last_peered":"2024-09-10T23:34:01.422418+0000","last_clean":"2024-09-10T23:34:01.422418+0000","last_became_active":"2024-09-10T23:34:00.050132+0000","last_became_peered":"2024-09-10T23:34:00.050132+0000","last_unstale":"2024-09-10T23:34:01.422418+0000","last_undegraded":"2024-09-10T23:34:01.422418+0000","last_fullsized":"2024-09-10T23:34:01.422418+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:00.854888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.046170+0000","last_change":"2024-09-10T23:33:59.053330+0000","last_active":"2024-09-10T23:34:00.046170+0000","last_peered":"2024-09-10T23:34:00.046170+0000","last_clean":"2024-09-10T23:34:00.046170+0000","last_became_active":"2024-09-10T23:33:59.053046+0000","last_became_peered":"2024-09-10T23:33:59.053046+0000","last_unstale":"2024-09-10T23:34:00.046170+0000","last_undegraded":"2024-09-10T23:34:00.046170+0000","last_fullsized":"2024-09-10T23:34:00.046170+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:55.206997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.048687+0000","last_change":"2024-09-10T23:34:00.048202+0000","last_active":"2024-09-10T23:34:01.048687+0000","last_peered":"2024-09-10T23:34:01.048687+0000","last_clean":"2024-09-10T23:34:01.048687+0000","last_became_active":"2024-09-10T23:34:00.047963+0000","last_became_peered":"2024-09-10T23:34:00.047963+0000","last_unstale":"2024-09-10T23:34:01.048687+0000","last_undegraded":"2024-09-10T23:34:01.048687+0000","last_fullsized":"2024-09-10T23:34:01.048687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:09.377196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423119+0000","last_change":"2024-09-10T23:33:59.042910+0000","last_active":"2024-09-10T23:34:01.423119+0000","last_peered":"2024-09-10T23:34:01.423119+0000","last_clean":"2024-09-10T23:34:01.423119+0000","last_became_active":"2024-09-10T23:33:59.042364+0000","last_became_peered":"2024-09-10T23:33:59.042364+0000","last_unstale":"2024-09-10T23:34:01.423119+0000","last_undegraded":"2024-09-10T23:34:01.423119+0000","last_fullsized":"2024-09-10T23:34:01.423119+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:06.893426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071262+0000","last_change":"2024-09-10T23:34:00.052373+0000","last_active":"2024-09-10T23:34:01.071262+0000","last_peered":"2024-09-10T23:34:01.071262+0000","last_clean":"2024-09-10T23:34:01.071262+0000","last_became_active":"2024-09-10T23:34:00.051533+0000","last_became_peered":"2024-09-10T23:34:00.051533+0000","last_unstale":"2024-09-10T23:34:01.071262+0000","last_undegraded":"2024-09-10T23:34:01.071262+0000","last_fullsized":"2024-09-10T23:34:01.071262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:55.647640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073978+0000","last_change":"2024-09-10T23:33:59.049000+0000","last_active":"2024-09-10T23:34:01.073978+0000","last_peered":"2024-09-10T23:34:01.073978+0000","last_clean":"2024-09-10T23:34:01.073978+0000","last_became_active":"2024-09-10T23:33:59.048734+0000","last_became_peered":"2024-09-10T23:33:59.048734+0000","last_unstale":"2024-09-10T23:34:01.073978+0000","last_undegraded":"2024-09-10T23:34:01.073978+0000","last_fullsized":"2024-09-10T23:34:01.073978+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:07.140145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":151,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422097+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:34:01.422097+0000","last_peered":"2024-09-10T23:34:01.422097+0000","last_clean":"2024-09-10T23:34:01.422097+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:34:01.422097+0000","last_undegraded":"2024-09-10T23:34:01.422097+0000","last_fullsized":"2024-09-10T23:34:01.422097+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":499712,"data_stored":196501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28100,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343740,"statfs":{"total":96636764160,"available":96607989760,"internally_reserved":0,"allocated":1093632,"data_stored":786857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":528384,"data_stored":204471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":507904,"data_stored":198079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":202857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1093632,"data_stored":786905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509461,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":503808,"data_stored":197139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542166,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":503808,"data_stored":197083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:34:04.147 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:34:04.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a41200640 1 -- 172.21.15.94:0/106436628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a3c0d2290 msgr2=0x7f1a3c0b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:04.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a41200640 1 --2- 172.21.15.94:0/106436628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a3c0d2290 0x7f1a3c0b1e70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0d2850 tx=0x7f1a3401cb80 comp rx=0 tx=0).stop 2024-09-10T23:34:04.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a41200640 1 -- 172.21.15.94:0/106436628 shutdown_connections 2024-09-10T23:34:04.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a41200640 1 --2- 172.21.15.94:0/106436628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a3c0d2290 0x7f1a3c0b1e70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a41200640 1 --2- 172.21.15.94:0/106436628 >> v2:172.21.15.181:3300/0 conn(0x7f1a3c15bd30 0x7f1a3c15c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a41200640 1 -- 172.21.15.94:0/106436628 >> 172.21.15.94:0/106436628 conn(0x7f1a3c09aa70 msgr2=0x7f1a3c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:04.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a41200640 1 -- 172.21.15.94:0/106436628 shutdown_connections 2024-09-10T23:34:04.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a41200640 1 -- 172.21.15.94:0/106436628 wait complete. 2024-09-10T23:34:04.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a41200640 1 Processor -- start 2024-09-10T23:34:04.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a41200640 1 -- start start 2024-09-10T23:34:04.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a41200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a3c0ae980 0x7f1a3c158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:04.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a41200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a3c0d2290 0x7f1a3c164a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:04.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a41200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a3c15bd30 0x7f1a3c1fb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:04.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a41200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c169550 con 0x7f1a3c0ae980 2024-09-10T23:34:04.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a41200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1693d0 con 0x7f1a3c0d2290 2024-09-10T23:34:04.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a41200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3c1696d0 con 0x7f1a3c15bd30 2024-09-10T23:34:04.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a39e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a3c0ae980 0x7f1a3c158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:04.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a39e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a3c0ae980 0x7f1a3c158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33934/0 (socket says 172.21.15.94:33934) 2024-09-10T23:34:04.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.324+0000 7f1a39e00640 1 -- 172.21.15.94:0/2227455246 learned_addr learned my addr 172.21.15.94:0/2227455246 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:04.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a31400640 1 --2- 172.21.15.94:0/2227455246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a3c0d2290 0x7f1a3c164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:04.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a3a800640 1 --2- 172.21.15.94:0/2227455246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a3c15bd30 0x7f1a3c1fb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:04.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a39e00640 1 -- 172.21.15.94:0/2227455246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a3c15bd30 msgr2=0x7f1a3c1fb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:04.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a39e00640 1 --2- 172.21.15.94:0/2227455246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a3c15bd30 0x7f1a3c1fb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a39e00640 1 -- 172.21.15.94:0/2227455246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a3c0d2290 msgr2=0x7f1a3c164a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:04.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a39e00640 1 --2- 172.21.15.94:0/2227455246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a3c0d2290 0x7f1a3c164a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a39e00640 1 -- 172.21.15.94:0/2227455246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3c1fbf40 con 0x7f1a3c0ae980 2024-09-10T23:34:04.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a3a800640 1 --2- 172.21.15.94:0/2227455246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a3c15bd30 0x7f1a3c1fb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:34:04.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a31400640 1 --2- 172.21.15.94:0/2227455246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a3c0d2290 0x7f1a3c164a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:34:04.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a39e00640 1 --2- 172.21.15.94:0/2227455246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a3c0ae980 0x7f1a3c158ea0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00e7a0 tx=0x7f1a2c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:04.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a32a00640 1 -- 172.21.15.94:0/2227455246 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c00ccc0 con 0x7f1a3c0ae980 2024-09-10T23:34:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a32a00640 1 -- 172.21.15.94:0/2227455246 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a2c010430 con 0x7f1a3c0ae980 2024-09-10T23:34:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a32a00640 1 -- 172.21.15.94:0/2227455246 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c010860 con 0x7f1a3c0ae980 2024-09-10T23:34:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a41200640 1 -- 172.21.15.94:0/2227455246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3c1fc220 con 0x7f1a3c0ae980 2024-09-10T23:34:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a41200640 1 -- 172.21.15.94:0/2227455246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a3c0c5c40 con 0x7f1a3c0ae980 2024-09-10T23:34:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a41200640 1 -- 172.21.15.94:0/2227455246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a3c0b1e70 con 0x7f1a3c0ae980 2024-09-10T23:34:04.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a32a00640 1 -- 172.21.15.94:0/2227455246 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f1a2c010a00 con 0x7f1a3c0ae980 2024-09-10T23:34:04.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a32a00640 1 --2- 172.21.15.94:0/2227455246 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a18072790 0x7f1a18074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:04.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.328+0000 7f1a32a00640 1 -- 172.21.15.94:0/2227455246 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f1a2c0940f0 con 0x7f1a3c0ae980 2024-09-10T23:34:04.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.332+0000 7f1a31400640 1 --2- 172.21.15.94:0/2227455246 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a18072790 0x7f1a18074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:04.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.332+0000 7f1a31400640 1 --2- 172.21.15.94:0/2227455246 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a18072790 0x7f1a18074c50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1a24001230 tx=0x7f1a24009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.336+0000 7f1a32a00640 1 -- 172.21.15.94:0/2227455246 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1a2c05f430 con 0x7f1a3c0ae980 2024-09-10T23:34:04.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.495+0000 7f1a41200640 1 -- 172.21.15.94:0/2227455246 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a3c0c57a0 con 0x7f1a18072790 2024-09-10T23:34:04.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.503+0000 7f1a32a00640 1 -- 172.21.15.94:0/2227455246 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7f1a24001d10 con 0x7f1a18072790 2024-09-10T23:34:04.507 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:04.508 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:34:04.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.503+0000 7f1a41200640 1 -- 172.21.15.94:0/2227455246 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a18072790 msgr2=0x7f1a18074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:04.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.503+0000 7f1a41200640 1 --2- 172.21.15.94:0/2227455246 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a18072790 0x7f1a18074c50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1a24001230 tx=0x7f1a24009040 comp rx=0 tx=0).stop 2024-09-10T23:34:04.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.503+0000 7f1a41200640 1 -- 172.21.15.94:0/2227455246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a3c0ae980 msgr2=0x7f1a3c158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.503+0000 7f1a41200640 1 --2- 172.21.15.94:0/2227455246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a3c0ae980 0x7f1a3c158ea0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00e7a0 tx=0x7f1a2c00ec60 comp rx=0 tx=0).stop 2024-09-10T23:34:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.503+0000 7f1a41200640 1 -- 172.21.15.94:0/2227455246 shutdown_connections 2024-09-10T23:34:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.503+0000 7f1a41200640 1 --2- 172.21.15.94:0/2227455246 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a18072790 0x7f1a18074c50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.503+0000 7f1a41200640 1 --2- 172.21.15.94:0/2227455246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a3c0d2290 0x7f1a3c164a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.503+0000 7f1a41200640 1 --2- 172.21.15.94:0/2227455246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a3c15bd30 0x7f1a3c1fb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.503+0000 7f1a41200640 1 --2- 172.21.15.94:0/2227455246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a3c0ae980 0x7f1a3c158ea0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.503+0000 7f1a41200640 1 -- 172.21.15.94:0/2227455246 >> 172.21.15.94:0/2227455246 conn(0x7f1a3c09aa70 msgr2=0x7f1a3c070d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.503+0000 7f1a41200640 1 -- 172.21.15.94:0/2227455246 shutdown_connections 2024-09-10T23:34:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.503+0000 7f1a41200640 1 -- 172.21.15.94:0/2227455246 wait complete. 2024-09-10T23:34:04.527 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-09-10T23:34:03.360492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242160,"kb_used_data":5120,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732560,"statfs":{"total":773094113280,"available":772846141440,"internally_reserved":0,"allocated":5242880,"data_stored":2769892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.339417"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423181+0000","last_change":"2024-09-10T23:34:00.053526+0000","last_active":"2024-09-10T23:34:01.423181+0000","last_peered":"2024-09-10T23:34:01.423181+0000","last_clean":"2024-09-10T23:34:01.423181+0000","last_became_active":"2024-09-10T23:34:00.053281+0000","last_became_peered":"2024-09-10T23:34:00.053281+0000","last_unstale":"2024-09-10T23:34:01.423181+0000","last_undegraded":"2024-09-10T23:34:01.423181+0000","last_fullsized":"2024-09-10T23:34:01.423181+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:22.977489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.049182+0000","last_change":"2024-09-10T23:34:00.053927+0000","last_active":"2024-09-10T23:34:01.049182+0000","last_peered":"2024-09-10T23:34:01.049182+0000","last_clean":"2024-09-10T23:34:01.049182+0000","last_became_active":"2024-09-10T23:34:00.053417+0000","last_became_peered":"2024-09-10T23:34:00.053417+0000","last_unstale":"2024-09-10T23:34:01.049182+0000","last_undegraded":"2024-09-10T23:34:01.049182+0000","last_fullsized":"2024-09-10T23:34:01.049182+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:18.510820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422786+0000","last_change":"2024-09-10T23:34:00.055977+0000","last_active":"2024-09-10T23:34:01.422786+0000","last_peered":"2024-09-10T23:34:01.422786+0000","last_clean":"2024-09-10T23:34:01.422786+0000","last_became_active":"2024-09-10T23:34:00.055710+0000","last_became_peered":"2024-09-10T23:34:00.055710+0000","last_unstale":"2024-09-10T23:34:01.422786+0000","last_undegraded":"2024-09-10T23:34:01.422786+0000","last_fullsized":"2024-09-10T23:34:01.422786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:05.760065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071293+0000","last_change":"2024-09-10T23:34:00.059234+0000","last_active":"2024-09-10T23:34:01.071293+0000","last_peered":"2024-09-10T23:34:01.071293+0000","last_clean":"2024-09-10T23:34:01.071293+0000","last_became_active":"2024-09-10T23:34:00.058896+0000","last_became_peered":"2024-09-10T23:34:00.058896+0000","last_unstale":"2024-09-10T23:34:01.071293+0000","last_undegraded":"2024-09-10T23:34:01.071293+0000","last_fullsized":"2024-09-10T23:34:01.071293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:44.401260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422656+0000","last_change":"2024-09-10T23:34:00.052621+0000","last_active":"2024-09-10T23:34:01.422656+0000","last_peered":"2024-09-10T23:34:01.422656+0000","last_clean":"2024-09-10T23:34:01.422656+0000","last_became_active":"2024-09-10T23:34:00.052321+0000","last_became_peered":"2024-09-10T23:34:00.052321+0000","last_unstale":"2024-09-10T23:34:01.422656+0000","last_undegraded":"2024-09-10T23:34:01.422656+0000","last_fullsized":"2024-09-10T23:34:01.422656+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:20.750358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.070157+0000","last_change":"2024-09-10T23:34:00.062842+0000","last_active":"2024-09-10T23:34:00.070157+0000","last_peered":"2024-09-10T23:34:00.070157+0000","last_clean":"2024-09-10T23:34:00.070157+0000","last_became_active":"2024-09-10T23:34:00.062137+0000","last_became_peered":"2024-09-10T23:34:00.062137+0000","last_unstale":"2024-09-10T23:34:00.070157+0000","last_undegraded":"2024-09-10T23:34:00.070157+0000","last_fullsized":"2024-09-10T23:34:00.070157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:18.362114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073565+0000","last_change":"2024-09-10T23:34:00.059881+0000","last_active":"2024-09-10T23:34:01.073565+0000","last_peered":"2024-09-10T23:34:01.073565+0000","last_clean":"2024-09-10T23:34:01.073565+0000","last_became_active":"2024-09-10T23:34:00.059617+0000","last_became_peered":"2024-09-10T23:34:00.059617+0000","last_unstale":"2024-09-10T23:34:01.073565+0000","last_undegraded":"2024-09-10T23:34:01.073565+0000","last_fullsized":"2024-09-10T23:34:01.073565+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:22.314467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.070670+0000","last_change":"2024-09-10T23:34:00.058585+0000","last_active":"2024-09-10T23:34:01.070670+0000","last_peered":"2024-09-10T23:34:01.070670+0000","last_clean":"2024-09-10T23:34:01.070670+0000","last_became_active":"2024-09-10T23:34:00.053995+0000","last_became_peered":"2024-09-10T23:34:00.053995+0000","last_unstale":"2024-09-10T23:34:01.070670+0000","last_undegraded":"2024-09-10T23:34:01.070670+0000","last_fullsized":"2024-09-10T23:34:01.070670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:26.445051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.067789+0000","last_change":"2024-09-10T23:34:00.062526+0000","last_active":"2024-09-10T23:34:00.067789+0000","last_peered":"2024-09-10T23:34:00.067789+0000","last_clean":"2024-09-10T23:34:00.067789+0000","last_became_active":"2024-09-10T23:34:00.062232+0000","last_became_peered":"2024-09-10T23:34:00.062232+0000","last_unstale":"2024-09-10T23:34:00.067789+0000","last_undegraded":"2024-09-10T23:34:00.067789+0000","last_fullsized":"2024-09-10T23:34:00.067789+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:44.129265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050157+0000","last_change":"2024-09-10T23:34:00.055510+0000","last_active":"2024-09-10T23:34:01.050157+0000","last_peered":"2024-09-10T23:34:01.050157+0000","last_clean":"2024-09-10T23:34:01.050157+0000","last_became_active":"2024-09-10T23:34:00.055184+0000","last_became_peered":"2024-09-10T23:34:00.055184+0000","last_unstale":"2024-09-10T23:34:01.050157+0000","last_undegraded":"2024-09-10T23:34:01.050157+0000","last_fullsized":"2024-09-10T23:34:01.050157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:43.177224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071369+0000","last_change":"2024-09-10T23:34:00.060147+0000","last_active":"2024-09-10T23:34:01.071369+0000","last_peered":"2024-09-10T23:34:01.071369+0000","last_clean":"2024-09-10T23:34:01.071369+0000","last_became_active":"2024-09-10T23:34:00.059867+0000","last_became_peered":"2024-09-10T23:34:00.059867+0000","last_unstale":"2024-09-10T23:34:01.071369+0000","last_undegraded":"2024-09-10T23:34:01.071369+0000","last_fullsized":"2024-09-10T23:34:01.071369+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:08.146583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.048943+0000","last_change":"2024-09-10T23:34:00.057451+0000","last_active":"2024-09-10T23:34:01.048943+0000","last_peered":"2024-09-10T23:34:01.048943+0000","last_clean":"2024-09-10T23:34:01.048943+0000","last_became_active":"2024-09-10T23:34:00.057182+0000","last_became_peered":"2024-09-10T23:34:00.057182+0000","last_unstale":"2024-09-10T23:34:01.048943+0000","last_undegraded":"2024-09-10T23:34:01.048943+0000","last_fullsized":"2024-09-10T23:34:01.048943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:56.900603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.067967+0000","last_change":"2024-09-10T23:34:00.062646+0000","last_active":"2024-09-10T23:34:00.067967+0000","last_peered":"2024-09-10T23:34:00.067967+0000","last_clean":"2024-09-10T23:34:00.067967+0000","last_became_active":"2024-09-10T23:34:00.062350+0000","last_became_peered":"2024-09-10T23:34:00.062350+0000","last_unstale":"2024-09-10T23:34:00.067967+0000","last_undegraded":"2024-09-10T23:34:00.067967+0000","last_fullsized":"2024-09-10T23:34:00.067967+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:16.401795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071169+0000","last_change":"2024-09-10T23:34:00.064303+0000","last_active":"2024-09-10T23:34:01.071169+0000","last_peered":"2024-09-10T23:34:01.071169+0000","last_clean":"2024-09-10T23:34:01.071169+0000","last_became_active":"2024-09-10T23:34:00.063991+0000","last_became_peered":"2024-09-10T23:34:00.063991+0000","last_unstale":"2024-09-10T23:34:01.071169+0000","last_undegraded":"2024-09-10T23:34:01.071169+0000","last_fullsized":"2024-09-10T23:34:01.071169+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:29.633890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071139+0000","last_change":"2024-09-10T23:34:00.046958+0000","last_active":"2024-09-10T23:34:01.071139+0000","last_peered":"2024-09-10T23:34:01.071139+0000","last_clean":"2024-09-10T23:34:01.071139+0000","last_became_active":"2024-09-10T23:34:00.046679+0000","last_became_peered":"2024-09-10T23:34:00.046679+0000","last_unstale":"2024-09-10T23:34:01.071139+0000","last_undegraded":"2024-09-10T23:34:01.071139+0000","last_fullsized":"2024-09-10T23:34:01.071139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:10.360339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422292+0000","last_change":"2024-09-10T23:34:00.062267+0000","last_active":"2024-09-10T23:34:01.422292+0000","last_peered":"2024-09-10T23:34:01.422292+0000","last_clean":"2024-09-10T23:34:01.422292+0000","last_became_active":"2024-09-10T23:34:00.062006+0000","last_became_peered":"2024-09-10T23:34:00.062006+0000","last_unstale":"2024-09-10T23:34:01.422292+0000","last_undegraded":"2024-09-10T23:34:01.422292+0000","last_fullsized":"2024-09-10T23:34:01.422292+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:47.199893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.049245+0000","last_change":"2024-09-10T23:34:00.044662+0000","last_active":"2024-09-10T23:34:01.049245+0000","last_peered":"2024-09-10T23:34:01.049245+0000","last_clean":"2024-09-10T23:34:01.049245+0000","last_became_active":"2024-09-10T23:34:00.044423+0000","last_became_peered":"2024-09-10T23:34:00.044423+0000","last_unstale":"2024-09-10T23:34:01.049245+0000","last_undegraded":"2024-09-10T23:34:01.049245+0000","last_fullsized":"2024-09-10T23:34:01.049245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:13.449304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422951+0000","last_change":"2024-09-10T23:34:00.059149+0000","last_active":"2024-09-10T23:34:01.422951+0000","last_peered":"2024-09-10T23:34:01.422951+0000","last_clean":"2024-09-10T23:34:01.422951+0000","last_became_active":"2024-09-10T23:34:00.058747+0000","last_became_peered":"2024-09-10T23:34:00.058747+0000","last_unstale":"2024-09-10T23:34:01.422951+0000","last_undegraded":"2024-09-10T23:34:01.422951+0000","last_fullsized":"2024-09-10T23:34:01.422951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:12.663251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423150+0000","last_change":"2024-09-10T23:34:00.054238+0000","last_active":"2024-09-10T23:34:01.423150+0000","last_peered":"2024-09-10T23:34:01.423150+0000","last_clean":"2024-09-10T23:34:01.423150+0000","last_became_active":"2024-09-10T23:34:00.053957+0000","last_became_peered":"2024-09-10T23:34:00.053957+0000","last_unstale":"2024-09-10T23:34:01.423150+0000","last_undegraded":"2024-09-10T23:34:01.423150+0000","last_fullsized":"2024-09-10T23:34:01.423150+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:10.438658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050827+0000","last_change":"2024-09-10T23:34:00.055856+0000","last_active":"2024-09-10T23:34:01.050827+0000","last_peered":"2024-09-10T23:34:01.050827+0000","last_clean":"2024-09-10T23:34:01.050827+0000","last_became_active":"2024-09-10T23:34:00.055504+0000","last_became_peered":"2024-09-10T23:34:00.055504+0000","last_unstale":"2024-09-10T23:34:01.050827+0000","last_undegraded":"2024-09-10T23:34:01.050827+0000","last_fullsized":"2024-09-10T23:34:01.050827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:04.255156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071288+0000","last_change":"2024-09-10T23:34:00.053573+0000","last_active":"2024-09-10T23:34:01.071288+0000","last_peered":"2024-09-10T23:34:01.071288+0000","last_clean":"2024-09-10T23:34:01.071288+0000","last_became_active":"2024-09-10T23:34:00.053032+0000","last_became_peered":"2024-09-10T23:34:00.053032+0000","last_unstale":"2024-09-10T23:34:01.071288+0000","last_undegraded":"2024-09-10T23:34:01.071288+0000","last_fullsized":"2024-09-10T23:34:01.071288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:57.789272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423183+0000","last_change":"2024-09-10T23:34:00.054671+0000","last_active":"2024-09-10T23:34:01.423183+0000","last_peered":"2024-09-10T23:34:01.423183+0000","last_clean":"2024-09-10T23:34:01.423183+0000","last_became_active":"2024-09-10T23:34:00.054019+0000","last_became_peered":"2024-09-10T23:34:00.054019+0000","last_unstale":"2024-09-10T23:34:01.423183+0000","last_undegraded":"2024-09-10T23:34:01.423183+0000","last_fullsized":"2024-09-10T23:34:01.423183+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:11.706227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073964+0000","last_change":"2024-09-10T23:34:00.048488+0000","last_active":"2024-09-10T23:34:01.073964+0000","last_peered":"2024-09-10T23:34:01.073964+0000","last_clean":"2024-09-10T23:34:01.073964+0000","last_became_active":"2024-09-10T23:34:00.047950+0000","last_became_peered":"2024-09-10T23:34:00.047950+0000","last_unstale":"2024-09-10T23:34:01.073964+0000","last_undegraded":"2024-09-10T23:34:01.073964+0000","last_fullsized":"2024-09-10T23:34:01.073964+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:26.190775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.048704+0000","last_change":"2024-09-10T23:34:00.061492+0000","last_active":"2024-09-10T23:34:01.048704+0000","last_peered":"2024-09-10T23:34:01.048704+0000","last_clean":"2024-09-10T23:34:01.048704+0000","last_became_active":"2024-09-10T23:34:00.061247+0000","last_became_peered":"2024-09-10T23:34:00.061247+0000","last_unstale":"2024-09-10T23:34:01.048704+0000","last_undegraded":"2024-09-10T23:34:01.048704+0000","last_fullsized":"2024-09-10T23:34:01.048704+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:29.286908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423253+0000","last_change":"2024-09-10T23:34:00.053262+0000","last_active":"2024-09-10T23:34:01.423253+0000","last_peered":"2024-09-10T23:34:01.423253+0000","last_clean":"2024-09-10T23:34:01.423253+0000","last_became_active":"2024-09-10T23:34:00.053022+0000","last_became_peered":"2024-09-10T23:34:00.053022+0000","last_unstale":"2024-09-10T23:34:01.423253+0000","last_undegraded":"2024-09-10T23:34:01.423253+0000","last_fullsized":"2024-09-10T23:34:01.423253+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:18.510820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071133+0000","last_change":"2024-09-10T23:34:00.060558+0000","last_active":"2024-09-10T23:34:01.071133+0000","last_peered":"2024-09-10T23:34:01.071133+0000","last_clean":"2024-09-10T23:34:01.071133+0000","last_became_active":"2024-09-10T23:34:00.060283+0000","last_became_peered":"2024-09-10T23:34:00.060283+0000","last_unstale":"2024-09-10T23:34:01.071133+0000","last_undegraded":"2024-09-10T23:34:01.071133+0000","last_fullsized":"2024-09-10T23:34:01.071133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:03.679840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073920+0000","last_change":"2024-09-10T23:34:00.043105+0000","last_active":"2024-09-10T23:34:01.073920+0000","last_peered":"2024-09-10T23:34:01.073920+0000","last_clean":"2024-09-10T23:34:01.073920+0000","last_became_active":"2024-09-10T23:34:00.042916+0000","last_became_peered":"2024-09-10T23:34:00.042916+0000","last_unstale":"2024-09-10T23:34:01.073920+0000","last_undegraded":"2024-09-10T23:34:01.073920+0000","last_fullsized":"2024-09-10T23:34:01.073920+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:50.495799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073852+0000","last_change":"2024-09-10T23:34:00.047179+0000","last_active":"2024-09-10T23:34:01.073852+0000","last_peered":"2024-09-10T23:34:01.073852+0000","last_clean":"2024-09-10T23:34:01.073852+0000","last_became_active":"2024-09-10T23:34:00.046939+0000","last_became_peered":"2024-09-10T23:34:00.046939+0000","last_unstale":"2024-09-10T23:34:01.073852+0000","last_undegraded":"2024-09-10T23:34:01.073852+0000","last_fullsized":"2024-09-10T23:34:01.073852+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:01.848161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422876+0000","last_change":"2024-09-10T23:34:00.050273+0000","last_active":"2024-09-10T23:34:01.422876+0000","last_peered":"2024-09-10T23:34:01.422876+0000","last_clean":"2024-09-10T23:34:01.422876+0000","last_became_active":"2024-09-10T23:34:00.049951+0000","last_became_peered":"2024-09-10T23:34:00.049951+0000","last_unstale":"2024-09-10T23:34:01.422876+0000","last_undegraded":"2024-09-10T23:34:01.422876+0000","last_fullsized":"2024-09-10T23:34:01.422876+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:56.900603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.048726+0000","last_change":"2024-09-10T23:34:00.045331+0000","last_active":"2024-09-10T23:34:01.048726+0000","last_peered":"2024-09-10T23:34:01.048726+0000","last_clean":"2024-09-10T23:34:01.048726+0000","last_became_active":"2024-09-10T23:34:00.045065+0000","last_became_peered":"2024-09-10T23:34:00.045065+0000","last_unstale":"2024-09-10T23:34:01.048726+0000","last_undegraded":"2024-09-10T23:34:01.048726+0000","last_fullsized":"2024-09-10T23:34:01.048726+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:47.199893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.062628+0000","last_change":"2024-09-10T23:34:00.056612+0000","last_active":"2024-09-10T23:34:00.062628+0000","last_peered":"2024-09-10T23:34:00.062628+0000","last_clean":"2024-09-10T23:34:00.062628+0000","last_became_active":"2024-09-10T23:34:00.054312+0000","last_became_peered":"2024-09-10T23:34:00.054312+0000","last_unstale":"2024-09-10T23:34:00.062628+0000","last_undegraded":"2024-09-10T23:34:00.062628+0000","last_fullsized":"2024-09-10T23:34:00.062628+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:41:09.300122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.070777+0000","last_change":"2024-09-10T23:34:00.062416+0000","last_active":"2024-09-10T23:34:01.070777+0000","last_peered":"2024-09-10T23:34:01.070777+0000","last_clean":"2024-09-10T23:34:01.070777+0000","last_became_active":"2024-09-10T23:34:00.061864+0000","last_became_peered":"2024-09-10T23:34:00.061864+0000","last_unstale":"2024-09-10T23:34:01.070777+0000","last_undegraded":"2024-09-10T23:34:01.070777+0000","last_fullsized":"2024-09-10T23:34:01.070777+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:31.362214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423359+0000","last_change":"2024-09-10T23:34:00.068583+0000","last_active":"2024-09-10T23:34:01.423359+0000","last_peered":"2024-09-10T23:34:01.423359+0000","last_clean":"2024-09-10T23:34:01.423359+0000","last_became_active":"2024-09-10T23:34:00.068312+0000","last_became_peered":"2024-09-10T23:34:00.068312+0000","last_unstale":"2024-09-10T23:34:01.423359+0000","last_undegraded":"2024-09-10T23:34:01.423359+0000","last_fullsized":"2024-09-10T23:34:01.423359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:12.663251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.083066+0000","last_change":"2024-09-10T23:33:59.058670+0000","last_active":"2024-09-10T23:34:00.083066+0000","last_peered":"2024-09-10T23:34:00.083066+0000","last_clean":"2024-09-10T23:34:00.083066+0000","last_became_active":"2024-09-10T23:33:59.058416+0000","last_became_peered":"2024-09-10T23:33:59.058416+0000","last_unstale":"2024-09-10T23:34:00.083066+0000","last_undegraded":"2024-09-10T23:34:00.083066+0000","last_fullsized":"2024-09-10T23:34:00.083066+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:51.475453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423306+0000","last_change":"2024-09-10T23:34:00.054143+0000","last_active":"2024-09-10T23:34:01.423306+0000","last_peered":"2024-09-10T23:34:01.423306+0000","last_clean":"2024-09-10T23:34:01.423306+0000","last_became_active":"2024-09-10T23:34:00.053959+0000","last_became_peered":"2024-09-10T23:34:00.053959+0000","last_unstale":"2024-09-10T23:34:01.423306+0000","last_undegraded":"2024-09-10T23:34:01.423306+0000","last_fullsized":"2024-09-10T23:34:01.423306+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:13.449304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073773+0000","last_change":"2024-09-10T23:33:59.047976+0000","last_active":"2024-09-10T23:34:01.073773+0000","last_peered":"2024-09-10T23:34:01.073773+0000","last_clean":"2024-09-10T23:34:01.073773+0000","last_became_active":"2024-09-10T23:33:59.047659+0000","last_became_peered":"2024-09-10T23:33:59.047659+0000","last_unstale":"2024-09-10T23:34:01.073773+0000","last_undegraded":"2024-09-10T23:34:01.073773+0000","last_fullsized":"2024-09-10T23:34:01.073773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:14.745642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.049002+0000","last_change":"2024-09-10T23:34:00.056027+0000","last_active":"2024-09-10T23:34:01.049002+0000","last_peered":"2024-09-10T23:34:01.049002+0000","last_clean":"2024-09-10T23:34:01.049002+0000","last_became_active":"2024-09-10T23:34:00.055785+0000","last_became_peered":"2024-09-10T23:34:00.055785+0000","last_unstale":"2024-09-10T23:34:01.049002+0000","last_undegraded":"2024-09-10T23:34:01.049002+0000","last_fullsized":"2024-09-10T23:34:01.049002+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:10.438658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.082123+0000","last_change":"2024-09-10T23:33:59.058781+0000","last_active":"2024-09-10T23:34:00.082123+0000","last_peered":"2024-09-10T23:34:00.082123+0000","last_clean":"2024-09-10T23:34:00.082123+0000","last_became_active":"2024-09-10T23:33:59.058451+0000","last_became_peered":"2024-09-10T23:33:59.058451+0000","last_unstale":"2024-09-10T23:34:00.082123+0000","last_undegraded":"2024-09-10T23:34:00.082123+0000","last_fullsized":"2024-09-10T23:34:00.082123+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:58.333956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071247+0000","last_change":"2024-09-10T23:34:00.059235+0000","last_active":"2024-09-10T23:34:01.071247+0000","last_peered":"2024-09-10T23:34:01.071247+0000","last_clean":"2024-09-10T23:34:01.071247+0000","last_became_active":"2024-09-10T23:34:00.058953+0000","last_became_peered":"2024-09-10T23:34:00.058953+0000","last_unstale":"2024-09-10T23:34:01.071247+0000","last_undegraded":"2024-09-10T23:34:01.071247+0000","last_fullsized":"2024-09-10T23:34:01.071247+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:34.439228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073838+0000","last_change":"2024-09-10T23:33:59.045197+0000","last_active":"2024-09-10T23:34:01.073838+0000","last_peered":"2024-09-10T23:34:01.073838+0000","last_clean":"2024-09-10T23:34:01.073838+0000","last_became_active":"2024-09-10T23:33:59.044673+0000","last_became_peered":"2024-09-10T23:33:59.044673+0000","last_unstale":"2024-09-10T23:34:01.073838+0000","last_undegraded":"2024-09-10T23:34:01.073838+0000","last_fullsized":"2024-09-10T23:34:01.073838+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:05.832407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073675+0000","last_change":"2024-09-10T23:34:00.044065+0000","last_active":"2024-09-10T23:34:01.073675+0000","last_peered":"2024-09-10T23:34:01.073675+0000","last_clean":"2024-09-10T23:34:01.073675+0000","last_became_active":"2024-09-10T23:34:00.042623+0000","last_became_peered":"2024-09-10T23:34:00.042623+0000","last_unstale":"2024-09-10T23:34:01.073675+0000","last_undegraded":"2024-09-10T23:34:01.073675+0000","last_fullsized":"2024-09-10T23:34:01.073675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:40.285835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.082000+0000","last_change":"2024-09-10T23:33:59.050742+0000","last_active":"2024-09-10T23:34:00.082000+0000","last_peered":"2024-09-10T23:34:00.082000+0000","last_clean":"2024-09-10T23:34:00.082000+0000","last_became_active":"2024-09-10T23:33:59.050451+0000","last_became_peered":"2024-09-10T23:33:59.050451+0000","last_unstale":"2024-09-10T23:34:00.082000+0000","last_undegraded":"2024-09-10T23:34:00.082000+0000","last_fullsized":"2024-09-10T23:34:00.082000+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:43.648471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.049993+0000","last_change":"2024-09-10T23:34:00.066186+0000","last_active":"2024-09-10T23:34:01.049993+0000","last_peered":"2024-09-10T23:34:01.049993+0000","last_clean":"2024-09-10T23:34:01.049993+0000","last_became_active":"2024-09-10T23:34:00.065935+0000","last_became_peered":"2024-09-10T23:34:00.065935+0000","last_unstale":"2024-09-10T23:34:01.049993+0000","last_undegraded":"2024-09-10T23:34:01.049993+0000","last_fullsized":"2024-09-10T23:34:01.049993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:37.105867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073611+0000","last_change":"2024-09-10T23:33:59.053039+0000","last_active":"2024-09-10T23:34:01.073611+0000","last_peered":"2024-09-10T23:34:01.073611+0000","last_clean":"2024-09-10T23:34:01.073611+0000","last_became_active":"2024-09-10T23:33:59.052753+0000","last_became_peered":"2024-09-10T23:33:59.052753+0000","last_unstale":"2024-09-10T23:34:01.073611+0000","last_undegraded":"2024-09-10T23:34:01.073611+0000","last_fullsized":"2024-09-10T23:34:01.073611+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:38.777776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050550+0000","last_change":"2024-09-10T23:34:00.057716+0000","last_active":"2024-09-10T23:34:01.050550+0000","last_peered":"2024-09-10T23:34:01.050550+0000","last_clean":"2024-09-10T23:34:01.050550+0000","last_became_active":"2024-09-10T23:34:00.057328+0000","last_became_peered":"2024-09-10T23:34:00.057328+0000","last_unstale":"2024-09-10T23:34:01.050550+0000","last_undegraded":"2024-09-10T23:34:01.050550+0000","last_fullsized":"2024-09-10T23:34:01.050550+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:38.030151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422277+0000","last_change":"2024-09-10T23:33:59.049392+0000","last_active":"2024-09-10T23:34:01.422277+0000","last_peered":"2024-09-10T23:34:01.422277+0000","last_clean":"2024-09-10T23:34:01.422277+0000","last_became_active":"2024-09-10T23:33:59.049115+0000","last_became_peered":"2024-09-10T23:33:59.049115+0000","last_unstale":"2024-09-10T23:34:01.422277+0000","last_undegraded":"2024-09-10T23:34:01.422277+0000","last_fullsized":"2024-09-10T23:34:01.422277+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:23.329966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073580+0000","last_change":"2024-09-10T23:34:00.048785+0000","last_active":"2024-09-10T23:34:01.073580+0000","last_peered":"2024-09-10T23:34:01.073580+0000","last_clean":"2024-09-10T23:34:01.073580+0000","last_became_active":"2024-09-10T23:34:00.048260+0000","last_became_peered":"2024-09-10T23:34:00.048260+0000","last_unstale":"2024-09-10T23:34:01.073580+0000","last_undegraded":"2024-09-10T23:34:01.073580+0000","last_fullsized":"2024-09-10T23:34:01.073580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:25.014233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050604+0000","last_change":"2024-09-10T23:33:59.043494+0000","last_active":"2024-09-10T23:34:01.050604+0000","last_peered":"2024-09-10T23:34:01.050604+0000","last_clean":"2024-09-10T23:34:01.050604+0000","last_became_active":"2024-09-10T23:33:59.043195+0000","last_became_peered":"2024-09-10T23:33:59.043195+0000","last_unstale":"2024-09-10T23:34:01.050604+0000","last_undegraded":"2024-09-10T23:34:01.050604+0000","last_fullsized":"2024-09-10T23:34:01.050604+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:19.743920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422954+0000","last_change":"2024-09-10T23:34:00.056081+0000","last_active":"2024-09-10T23:34:01.422954+0000","last_peered":"2024-09-10T23:34:01.422954+0000","last_clean":"2024-09-10T23:34:01.422954+0000","last_became_active":"2024-09-10T23:34:00.055192+0000","last_became_peered":"2024-09-10T23:34:00.055192+0000","last_unstale":"2024-09-10T23:34:01.422954+0000","last_undegraded":"2024-09-10T23:34:01.422954+0000","last_fullsized":"2024-09-10T23:34:01.422954+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:45:06.264461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423030+0000","last_change":"2024-09-10T23:33:59.050927+0000","last_active":"2024-09-10T23:34:01.423030+0000","last_peered":"2024-09-10T23:34:01.423030+0000","last_clean":"2024-09-10T23:34:01.423030+0000","last_became_active":"2024-09-10T23:33:59.050620+0000","last_became_peered":"2024-09-10T23:33:59.050620+0000","last_unstale":"2024-09-10T23:34:01.423030+0000","last_undegraded":"2024-09-10T23:34:01.423030+0000","last_fullsized":"2024-09-10T23:34:01.423030+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:33.961437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422942+0000","last_change":"2024-09-10T23:34:00.050339+0000","last_active":"2024-09-10T23:34:01.422942+0000","last_peered":"2024-09-10T23:34:01.422942+0000","last_clean":"2024-09-10T23:34:01.422942+0000","last_became_active":"2024-09-10T23:34:00.047535+0000","last_became_peered":"2024-09-10T23:34:00.047535+0000","last_unstale":"2024-09-10T23:34:01.422942+0000","last_undegraded":"2024-09-10T23:34:01.422942+0000","last_fullsized":"2024-09-10T23:34:01.422942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:12.665202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050219+0000","last_change":"2024-09-10T23:33:59.049654+0000","last_active":"2024-09-10T23:34:01.050219+0000","last_peered":"2024-09-10T23:34:01.050219+0000","last_clean":"2024-09-10T23:34:01.050219+0000","last_became_active":"2024-09-10T23:33:59.049316+0000","last_became_peered":"2024-09-10T23:33:59.049316+0000","last_unstale":"2024-09-10T23:34:01.050219+0000","last_undegraded":"2024-09-10T23:34:01.050219+0000","last_fullsized":"2024-09-10T23:34:01.050219+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:17.504383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.070922+0000","last_change":"2024-09-10T23:34:00.064500+0000","last_active":"2024-09-10T23:34:01.070922+0000","last_peered":"2024-09-10T23:34:01.070922+0000","last_clean":"2024-09-10T23:34:01.070922+0000","last_became_active":"2024-09-10T23:34:00.064040+0000","last_became_peered":"2024-09-10T23:34:00.064040+0000","last_unstale":"2024-09-10T23:34:01.070922+0000","last_undegraded":"2024-09-10T23:34:01.070922+0000","last_fullsized":"2024-09-10T23:34:01.070922+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:22.977489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.048940+0000","last_change":"2024-09-10T23:33:59.052690+0000","last_active":"2024-09-10T23:34:01.048940+0000","last_peered":"2024-09-10T23:34:01.048940+0000","last_clean":"2024-09-10T23:34:01.048940+0000","last_became_active":"2024-09-10T23:33:59.052424+0000","last_became_peered":"2024-09-10T23:33:59.052424+0000","last_unstale":"2024-09-10T23:34:01.048940+0000","last_undegraded":"2024-09-10T23:34:01.048940+0000","last_fullsized":"2024-09-10T23:34:01.048940+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:23.329966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073988+0000","last_change":"2024-09-10T23:34:00.062569+0000","last_active":"2024-09-10T23:34:01.073988+0000","last_peered":"2024-09-10T23:34:01.073988+0000","last_clean":"2024-09-10T23:34:01.073988+0000","last_became_active":"2024-09-10T23:34:00.062326+0000","last_became_peered":"2024-09-10T23:34:00.062326+0000","last_unstale":"2024-09-10T23:34:01.073988+0000","last_undegraded":"2024-09-10T23:34:01.073988+0000","last_fullsized":"2024-09-10T23:34:01.073988+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:52.456744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":21,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050887+0000","last_change":"2024-09-10T23:33:59.043985+0000","last_active":"2024-09-10T23:34:01.050887+0000","last_peered":"2024-09-10T23:34:01.050887+0000","last_clean":"2024-09-10T23:34:01.050887+0000","last_became_active":"2024-09-10T23:33:59.043166+0000","last_became_peered":"2024-09-10T23:33:59.043166+0000","last_unstale":"2024-09-10T23:34:01.050887+0000","last_undegraded":"2024-09-10T23:34:01.050887+0000","last_fullsized":"2024-09-10T23:34:01.050887+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:12.442866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422606+0000","last_change":"2024-09-10T23:34:00.045511+0000","last_active":"2024-09-10T23:34:01.422606+0000","last_peered":"2024-09-10T23:34:01.422606+0000","last_clean":"2024-09-10T23:34:01.422606+0000","last_became_active":"2024-09-10T23:34:00.045251+0000","last_became_peered":"2024-09-10T23:34:00.045251+0000","last_unstale":"2024-09-10T23:34:01.422606+0000","last_undegraded":"2024-09-10T23:34:01.422606+0000","last_fullsized":"2024-09-10T23:34:01.422606+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:20.397390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423065+0000","last_change":"2024-09-10T23:33:59.047152+0000","last_active":"2024-09-10T23:34:01.423065+0000","last_peered":"2024-09-10T23:34:01.423065+0000","last_clean":"2024-09-10T23:34:01.423065+0000","last_became_active":"2024-09-10T23:33:59.046538+0000","last_became_peered":"2024-09-10T23:33:59.046538+0000","last_unstale":"2024-09-10T23:34:01.423065+0000","last_undegraded":"2024-09-10T23:34:01.423065+0000","last_fullsized":"2024-09-10T23:34:01.423065+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:32:24.262803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423174+0000","last_change":"2024-09-10T23:34:00.056643+0000","last_active":"2024-09-10T23:34:01.423174+0000","last_peered":"2024-09-10T23:34:01.423174+0000","last_clean":"2024-09-10T23:34:01.423174+0000","last_became_active":"2024-09-10T23:34:00.056427+0000","last_became_peered":"2024-09-10T23:34:00.056427+0000","last_unstale":"2024-09-10T23:34:01.423174+0000","last_undegraded":"2024-09-10T23:34:01.423174+0000","last_fullsized":"2024-09-10T23:34:01.423174+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:05.760065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422361+0000","last_change":"2024-09-10T23:33:59.050066+0000","last_active":"2024-09-10T23:34:01.422361+0000","last_peered":"2024-09-10T23:34:01.422361+0000","last_clean":"2024-09-10T23:34:01.422361+0000","last_became_active":"2024-09-10T23:33:59.049802+0000","last_became_peered":"2024-09-10T23:33:59.049802+0000","last_unstale":"2024-09-10T23:34:01.422361+0000","last_undegraded":"2024-09-10T23:34:01.422361+0000","last_fullsized":"2024-09-10T23:34:01.422361+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:24.302911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423113+0000","last_change":"2024-09-10T23:34:00.068659+0000","last_active":"2024-09-10T23:34:01.423113+0000","last_peered":"2024-09-10T23:34:01.423113+0000","last_clean":"2024-09-10T23:34:01.423113+0000","last_became_active":"2024-09-10T23:34:00.068373+0000","last_became_peered":"2024-09-10T23:34:00.068373+0000","last_unstale":"2024-09-10T23:34:01.423113+0000","last_undegraded":"2024-09-10T23:34:01.423113+0000","last_fullsized":"2024-09-10T23:34:01.423113+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:20.397390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.045996+0000","last_change":"2024-09-10T23:33:59.046524+0000","last_active":"2024-09-10T23:34:00.045996+0000","last_peered":"2024-09-10T23:34:00.045996+0000","last_clean":"2024-09-10T23:34:00.045996+0000","last_became_active":"2024-09-10T23:33:59.046257+0000","last_became_peered":"2024-09-10T23:33:59.046257+0000","last_unstale":"2024-09-10T23:34:00.045996+0000","last_undegraded":"2024-09-10T23:34:00.045996+0000","last_fullsized":"2024-09-10T23:34:00.045996+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:14.049866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.048712+0000","last_change":"2024-09-10T23:34:00.048486+0000","last_active":"2024-09-10T23:34:01.048712+0000","last_peered":"2024-09-10T23:34:01.048712+0000","last_clean":"2024-09-10T23:34:01.048712+0000","last_became_active":"2024-09-10T23:34:00.046456+0000","last_became_peered":"2024-09-10T23:34:00.046456+0000","last_unstale":"2024-09-10T23:34:01.048712+0000","last_undegraded":"2024-09-10T23:34:01.048712+0000","last_fullsized":"2024-09-10T23:34:01.048712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:26.445051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":23,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050653+0000","last_change":"2024-09-10T23:33:59.040987+0000","last_active":"2024-09-10T23:34:01.050653+0000","last_peered":"2024-09-10T23:34:01.050653+0000","last_clean":"2024-09-10T23:34:01.050653+0000","last_became_active":"2024-09-10T23:33:59.040648+0000","last_became_peered":"2024-09-10T23:33:59.040648+0000","last_unstale":"2024-09-10T23:34:01.050653+0000","last_undegraded":"2024-09-10T23:34:01.050653+0000","last_fullsized":"2024-09-10T23:34:01.050653+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:37:55.894165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423477+0000","last_change":"2024-09-10T23:34:00.054755+0000","last_active":"2024-09-10T23:34:01.423477+0000","last_peered":"2024-09-10T23:34:01.423477+0000","last_clean":"2024-09-10T23:34:01.423477+0000","last_became_active":"2024-09-10T23:34:00.054457+0000","last_became_peered":"2024-09-10T23:34:00.054457+0000","last_unstale":"2024-09-10T23:34:01.423477+0000","last_undegraded":"2024-09-10T23:34:01.423477+0000","last_fullsized":"2024-09-10T23:34:01.423477+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:04.985082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071056+0000","last_change":"2024-09-10T23:33:59.051243+0000","last_active":"2024-09-10T23:34:01.071056+0000","last_peered":"2024-09-10T23:34:01.071056+0000","last_clean":"2024-09-10T23:34:01.071056+0000","last_became_active":"2024-09-10T23:33:59.050869+0000","last_became_peered":"2024-09-10T23:33:59.050869+0000","last_unstale":"2024-09-10T23:34:01.071056+0000","last_undegraded":"2024-09-10T23:34:01.071056+0000","last_fullsized":"2024-09-10T23:34:01.071056+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:56:46.762024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423508+0000","last_change":"2024-09-10T23:34:00.075358+0000","last_active":"2024-09-10T23:34:01.423508+0000","last_peered":"2024-09-10T23:34:01.423508+0000","last_clean":"2024-09-10T23:34:01.423508+0000","last_became_active":"2024-09-10T23:34:00.072180+0000","last_became_peered":"2024-09-10T23:34:00.072180+0000","last_unstale":"2024-09-10T23:34:01.423508+0000","last_undegraded":"2024-09-10T23:34:01.423508+0000","last_fullsized":"2024-09-10T23:34:01.423508+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:45:06.264461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423040+0000","last_change":"2024-09-10T23:33:59.046310+0000","last_active":"2024-09-10T23:34:01.423040+0000","last_peered":"2024-09-10T23:34:01.423040+0000","last_clean":"2024-09-10T23:34:01.423040+0000","last_became_active":"2024-09-10T23:33:59.046022+0000","last_became_peered":"2024-09-10T23:33:59.046022+0000","last_unstale":"2024-09-10T23:34:01.423040+0000","last_undegraded":"2024-09-10T23:34:01.423040+0000","last_fullsized":"2024-09-10T23:34:01.423040+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:31:08.681878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.049130+0000","last_change":"2024-09-10T23:34:00.061374+0000","last_active":"2024-09-10T23:34:01.049130+0000","last_peered":"2024-09-10T23:34:01.049130+0000","last_clean":"2024-09-10T23:34:01.049130+0000","last_became_active":"2024-09-10T23:34:00.060549+0000","last_became_peered":"2024-09-10T23:34:00.060549+0000","last_unstale":"2024-09-10T23:34:01.049130+0000","last_undegraded":"2024-09-10T23:34:01.049130+0000","last_fullsized":"2024-09-10T23:34:01.049130+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:22.977489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422891+0000","last_change":"2024-09-10T23:33:59.054491+0000","last_active":"2024-09-10T23:34:01.422891+0000","last_peered":"2024-09-10T23:34:01.422891+0000","last_clean":"2024-09-10T23:34:01.422891+0000","last_became_active":"2024-09-10T23:33:59.054184+0000","last_became_peered":"2024-09-10T23:33:59.054184+0000","last_unstale":"2024-09-10T23:34:01.422891+0000","last_undegraded":"2024-09-10T23:34:01.422891+0000","last_fullsized":"2024-09-10T23:34:01.422891+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:36.722335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422809+0000","last_change":"2024-09-10T23:34:00.056435+0000","last_active":"2024-09-10T23:34:01.422809+0000","last_peered":"2024-09-10T23:34:01.422809+0000","last_clean":"2024-09-10T23:34:01.422809+0000","last_became_active":"2024-09-10T23:34:00.055481+0000","last_became_peered":"2024-09-10T23:34:00.055481+0000","last_unstale":"2024-09-10T23:34:01.422809+0000","last_undegraded":"2024-09-10T23:34:01.422809+0000","last_fullsized":"2024-09-10T23:34:01.422809+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:24.641710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.046180+0000","last_change":"2024-09-10T23:33:59.055487+0000","last_active":"2024-09-10T23:34:00.046180+0000","last_peered":"2024-09-10T23:34:00.046180+0000","last_clean":"2024-09-10T23:34:00.046180+0000","last_became_active":"2024-09-10T23:33:59.055219+0000","last_became_peered":"2024-09-10T23:33:59.055219+0000","last_unstale":"2024-09-10T23:34:00.046180+0000","last_undegraded":"2024-09-10T23:34:00.046180+0000","last_fullsized":"2024-09-10T23:34:00.046180+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:25.829230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050019+0000","last_change":"2024-09-10T23:34:00.068595+0000","last_active":"2024-09-10T23:34:01.050019+0000","last_peered":"2024-09-10T23:34:01.050019+0000","last_clean":"2024-09-10T23:34:01.050019+0000","last_became_active":"2024-09-10T23:34:00.068250+0000","last_became_peered":"2024-09-10T23:34:00.068250+0000","last_unstale":"2024-09-10T23:34:01.050019+0000","last_undegraded":"2024-09-10T23:34:01.050019+0000","last_fullsized":"2024-09-10T23:34:01.050019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:41.128499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050106+0000","last_change":"2024-09-10T23:33:59.044422+0000","last_active":"2024-09-10T23:34:01.050106+0000","last_peered":"2024-09-10T23:34:01.050106+0000","last_clean":"2024-09-10T23:34:01.050106+0000","last_became_active":"2024-09-10T23:33:59.044062+0000","last_became_peered":"2024-09-10T23:33:59.044062+0000","last_unstale":"2024-09-10T23:34:01.050106+0000","last_undegraded":"2024-09-10T23:34:01.050106+0000","last_fullsized":"2024-09-10T23:34:01.050106+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:21.971052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.053492+0000","last_change":"2024-09-10T23:34:00.047006+0000","last_active":"2024-09-10T23:34:00.053492+0000","last_peered":"2024-09-10T23:34:00.053492+0000","last_clean":"2024-09-10T23:34:00.053492+0000","last_became_active":"2024-09-10T23:34:00.046787+0000","last_became_peered":"2024-09-10T23:34:00.046787+0000","last_unstale":"2024-09-10T23:34:00.053492+0000","last_undegraded":"2024-09-10T23:34:00.053492+0000","last_fullsized":"2024-09-10T23:34:00.053492+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:56.782316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422447+0000","last_change":"2024-09-10T23:33:59.055739+0000","last_active":"2024-09-10T23:34:01.422447+0000","last_peered":"2024-09-10T23:34:01.422447+0000","last_clean":"2024-09-10T23:34:01.422447+0000","last_became_active":"2024-09-10T23:33:59.054895+0000","last_became_peered":"2024-09-10T23:33:59.054895+0000","last_unstale":"2024-09-10T23:34:01.422447+0000","last_undegraded":"2024-09-10T23:34:01.422447+0000","last_fullsized":"2024-09-10T23:34:01.422447+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:43.182488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071027+0000","last_change":"2024-09-10T23:34:00.059283+0000","last_active":"2024-09-10T23:34:01.071027+0000","last_peered":"2024-09-10T23:34:01.071027+0000","last_clean":"2024-09-10T23:34:01.071027+0000","last_became_active":"2024-09-10T23:34:00.058940+0000","last_became_peered":"2024-09-10T23:34:00.058940+0000","last_unstale":"2024-09-10T23:34:01.071027+0000","last_undegraded":"2024-09-10T23:34:01.071027+0000","last_fullsized":"2024-09-10T23:34:01.071027+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:43.464171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.049837+0000","last_change":"2024-09-10T23:33:59.049904+0000","last_active":"2024-09-10T23:34:01.049837+0000","last_peered":"2024-09-10T23:34:01.049837+0000","last_clean":"2024-09-10T23:34:01.049837+0000","last_became_active":"2024-09-10T23:33:59.049325+0000","last_became_peered":"2024-09-10T23:33:59.049325+0000","last_unstale":"2024-09-10T23:34:01.049837+0000","last_undegraded":"2024-09-10T23:34:01.049837+0000","last_fullsized":"2024-09-10T23:34:01.049837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:09.432220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.070904+0000","last_change":"2024-09-10T23:34:00.064321+0000","last_active":"2024-09-10T23:34:01.070904+0000","last_peered":"2024-09-10T23:34:01.070904+0000","last_clean":"2024-09-10T23:34:01.070904+0000","last_became_active":"2024-09-10T23:34:00.063975+0000","last_became_peered":"2024-09-10T23:34:00.063975+0000","last_unstale":"2024-09-10T23:34:01.070904+0000","last_undegraded":"2024-09-10T23:34:01.070904+0000","last_fullsized":"2024-09-10T23:34:01.070904+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:34.191086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073436+0000","last_change":"2024-09-10T23:33:59.044716+0000","last_active":"2024-09-10T23:34:01.073436+0000","last_peered":"2024-09-10T23:34:01.073436+0000","last_clean":"2024-09-10T23:34:01.073436+0000","last_became_active":"2024-09-10T23:33:59.044365+0000","last_became_peered":"2024-09-10T23:33:59.044365+0000","last_unstale":"2024-09-10T23:34:01.073436+0000","last_undegraded":"2024-09-10T23:34:01.073436+0000","last_fullsized":"2024-09-10T23:34:01.073436+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:33.432790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073987+0000","last_change":"2024-09-10T23:34:00.051244+0000","last_active":"2024-09-10T23:34:01.073987+0000","last_peered":"2024-09-10T23:34:01.073987+0000","last_clean":"2024-09-10T23:34:01.073987+0000","last_became_active":"2024-09-10T23:34:00.050975+0000","last_became_peered":"2024-09-10T23:34:00.050975+0000","last_unstale":"2024-09-10T23:34:01.073987+0000","last_undegraded":"2024-09-10T23:34:01.073987+0000","last_fullsized":"2024-09-10T23:34:01.073987+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:09:53.083471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"44'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.072959+0000","last_change":"2024-09-10T23:33:59.052060+0000","last_active":"2024-09-10T23:34:01.072959+0000","last_peered":"2024-09-10T23:34:01.072959+0000","last_clean":"2024-09-10T23:34:01.072959+0000","last_became_active":"2024-09-10T23:33:59.051509+0000","last_became_peered":"2024-09-10T23:33:59.051509+0000","last_unstale":"2024-09-10T23:34:01.072959+0000","last_undegraded":"2024-09-10T23:34:01.072959+0000","last_fullsized":"2024-09-10T23:34:01.072959+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.926434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050942+0000","last_change":"2024-09-10T23:34:00.058061+0000","last_active":"2024-09-10T23:34:01.050942+0000","last_peered":"2024-09-10T23:34:01.050942+0000","last_clean":"2024-09-10T23:34:01.050942+0000","last_became_active":"2024-09-10T23:34:00.057723+0000","last_became_peered":"2024-09-10T23:34:00.057723+0000","last_unstale":"2024-09-10T23:34:01.050942+0000","last_undegraded":"2024-09-10T23:34:01.050942+0000","last_fullsized":"2024-09-10T23:34:01.050942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:46.755190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423062+0000","last_change":"2024-09-10T23:33:59.055246+0000","last_active":"2024-09-10T23:34:01.423062+0000","last_peered":"2024-09-10T23:34:01.423062+0000","last_clean":"2024-09-10T23:34:01.423062+0000","last_became_active":"2024-09-10T23:33:59.052293+0000","last_became_peered":"2024-09-10T23:33:59.052293+0000","last_unstale":"2024-09-10T23:34:01.423062+0000","last_undegraded":"2024-09-10T23:34:01.423062+0000","last_fullsized":"2024-09-10T23:34:01.423062+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:54.307188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.074107+0000","last_change":"2024-09-10T23:34:00.059855+0000","last_active":"2024-09-10T23:34:01.074107+0000","last_peered":"2024-09-10T23:34:01.074107+0000","last_clean":"2024-09-10T23:34:01.074107+0000","last_became_active":"2024-09-10T23:34:00.059589+0000","last_became_peered":"2024-09-10T23:34:00.059589+0000","last_unstale":"2024-09-10T23:34:01.074107+0000","last_undegraded":"2024-09-10T23:34:01.074107+0000","last_fullsized":"2024-09-10T23:34:01.074107+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:51.065061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423193+0000","last_change":"2024-09-10T23:33:59.044762+0000","last_active":"2024-09-10T23:34:01.423193+0000","last_peered":"2024-09-10T23:34:01.423193+0000","last_clean":"2024-09-10T23:34:01.423193+0000","last_became_active":"2024-09-10T23:33:59.044485+0000","last_became_peered":"2024-09-10T23:33:59.044485+0000","last_unstale":"2024-09-10T23:34:01.423193+0000","last_undegraded":"2024-09-10T23:34:01.423193+0000","last_fullsized":"2024-09-10T23:34:01.423193+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:28.285801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.050189+0000","last_change":"2024-09-10T23:34:00.063321+0000","last_active":"2024-09-10T23:34:01.050189+0000","last_peered":"2024-09-10T23:34:01.050189+0000","last_clean":"2024-09-10T23:34:01.050189+0000","last_became_active":"2024-09-10T23:34:00.063046+0000","last_became_peered":"2024-09-10T23:34:00.063046+0000","last_unstale":"2024-09-10T23:34:01.050189+0000","last_undegraded":"2024-09-10T23:34:01.050189+0000","last_fullsized":"2024-09-10T23:34:01.050189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:57.628203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422766+0000","last_change":"2024-09-10T23:33:59.051537+0000","last_active":"2024-09-10T23:34:01.422766+0000","last_peered":"2024-09-10T23:34:01.422766+0000","last_clean":"2024-09-10T23:34:01.422766+0000","last_became_active":"2024-09-10T23:33:59.051244+0000","last_became_peered":"2024-09-10T23:33:59.051244+0000","last_unstale":"2024-09-10T23:34:01.422766+0000","last_undegraded":"2024-09-10T23:34:01.422766+0000","last_fullsized":"2024-09-10T23:34:01.422766+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:28.311294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.070910+0000","last_change":"2024-09-10T23:34:00.053297+0000","last_active":"2024-09-10T23:34:01.070910+0000","last_peered":"2024-09-10T23:34:01.070910+0000","last_clean":"2024-09-10T23:34:01.070910+0000","last_became_active":"2024-09-10T23:34:00.052705+0000","last_became_peered":"2024-09-10T23:34:00.052705+0000","last_unstale":"2024-09-10T23:34:01.070910+0000","last_undegraded":"2024-09-10T23:34:01.070910+0000","last_fullsized":"2024-09-10T23:34:01.070910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:10.438658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073764+0000","last_change":"2024-09-10T23:33:59.037368+0000","last_active":"2024-09-10T23:34:01.073764+0000","last_peered":"2024-09-10T23:34:01.073764+0000","last_clean":"2024-09-10T23:34:01.073764+0000","last_became_active":"2024-09-10T23:33:59.036540+0000","last_became_peered":"2024-09-10T23:33:59.036540+0000","last_unstale":"2024-09-10T23:34:01.073764+0000","last_undegraded":"2024-09-10T23:34:01.073764+0000","last_fullsized":"2024-09-10T23:34:01.073764+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:43.394823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422418+0000","last_change":"2024-09-10T23:34:00.050400+0000","last_active":"2024-09-10T23:34:01.422418+0000","last_peered":"2024-09-10T23:34:01.422418+0000","last_clean":"2024-09-10T23:34:01.422418+0000","last_became_active":"2024-09-10T23:34:00.050132+0000","last_became_peered":"2024-09-10T23:34:00.050132+0000","last_unstale":"2024-09-10T23:34:01.422418+0000","last_undegraded":"2024-09-10T23:34:01.422418+0000","last_fullsized":"2024-09-10T23:34:01.422418+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:00.854888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T23:34:00.046170+0000","last_change":"2024-09-10T23:33:59.053330+0000","last_active":"2024-09-10T23:34:00.046170+0000","last_peered":"2024-09-10T23:34:00.046170+0000","last_clean":"2024-09-10T23:34:00.046170+0000","last_became_active":"2024-09-10T23:33:59.053046+0000","last_became_peered":"2024-09-10T23:33:59.053046+0000","last_unstale":"2024-09-10T23:34:00.046170+0000","last_undegraded":"2024-09-10T23:34:00.046170+0000","last_fullsized":"2024-09-10T23:34:00.046170+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:55.206997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.048687+0000","last_change":"2024-09-10T23:34:00.048202+0000","last_active":"2024-09-10T23:34:01.048687+0000","last_peered":"2024-09-10T23:34:01.048687+0000","last_clean":"2024-09-10T23:34:01.048687+0000","last_became_active":"2024-09-10T23:34:00.047963+0000","last_became_peered":"2024-09-10T23:34:00.047963+0000","last_unstale":"2024-09-10T23:34:01.048687+0000","last_undegraded":"2024-09-10T23:34:01.048687+0000","last_fullsized":"2024-09-10T23:34:01.048687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:09.377196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.423119+0000","last_change":"2024-09-10T23:33:59.042910+0000","last_active":"2024-09-10T23:34:01.423119+0000","last_peered":"2024-09-10T23:34:01.423119+0000","last_clean":"2024-09-10T23:34:01.423119+0000","last_became_active":"2024-09-10T23:33:59.042364+0000","last_became_peered":"2024-09-10T23:33:59.042364+0000","last_unstale":"2024-09-10T23:34:01.423119+0000","last_undegraded":"2024-09-10T23:34:01.423119+0000","last_fullsized":"2024-09-10T23:34:01.423119+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:06.893426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.071262+0000","last_change":"2024-09-10T23:34:00.052373+0000","last_active":"2024-09-10T23:34:01.071262+0000","last_peered":"2024-09-10T23:34:01.071262+0000","last_clean":"2024-09-10T23:34:01.071262+0000","last_became_active":"2024-09-10T23:34:00.051533+0000","last_became_peered":"2024-09-10T23:34:00.051533+0000","last_unstale":"2024-09-10T23:34:01.071262+0000","last_undegraded":"2024-09-10T23:34:01.071262+0000","last_fullsized":"2024-09-10T23:34:01.071262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:59.019886+0000","last_clean_scrub_stamp":"2024-09-10T23:33:59.019886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:55.647640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.073978+0000","last_change":"2024-09-10T23:33:59.049000+0000","last_active":"2024-09-10T23:34:01.073978+0000","last_peered":"2024-09-10T23:34:01.073978+0000","last_clean":"2024-09-10T23:34:01.073978+0000","last_became_active":"2024-09-10T23:33:59.048734+0000","last_became_peered":"2024-09-10T23:33:59.048734+0000","last_unstale":"2024-09-10T23:34:01.073978+0000","last_undegraded":"2024-09-10T23:34:01.073978+0000","last_fullsized":"2024-09-10T23:34:01.073978+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:33:58.013448+0000","last_clean_scrub_stamp":"2024-09-10T23:33:58.013448+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:07.140145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":151,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T23:34:01.422097+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:34:01.422097+0000","last_peered":"2024-09-10T23:34:01.422097+0000","last_clean":"2024-09-10T23:34:01.422097+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:34:01.422097+0000","last_undegraded":"2024-09-10T23:34:01.422097+0000","last_fullsized":"2024-09-10T23:34:01.422097+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":499712,"data_stored":196501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28100,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343740,"statfs":{"total":96636764160,"available":96607989760,"internally_reserved":0,"allocated":1093632,"data_stored":786857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":528384,"data_stored":204471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":507904,"data_stored":198079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":202857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1093632,"data_stored":786905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509461,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":503808,"data_stored":197139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542166,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":503808,"data_stored":197083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:34:04.531 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T23:34:04.531 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T23:34:04.553 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:34:04.578 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:34:04.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1f5800640 1 -- 172.21.15.94:0/3313381325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f00cb4d0 msgr2=0x7ff1f015b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:04.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1f5800640 1 --2- 172.21.15.94:0/3313381325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f00cb4d0 0x7ff1f015b190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009c80 tx=0x7ff1e801cda0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1f5800640 1 -- 172.21.15.94:0/3313381325 shutdown_connections 2024-09-10T23:34:04.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1f5800640 1 --2- 172.21.15.94:0/3313381325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f00cb4d0 0x7ff1f015b190 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1f5800640 1 --2- 172.21.15.94:0/3313381325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f006fc00 0x7ff1f00c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1f5800640 1 -- 172.21.15.94:0/3313381325 >> 172.21.15.94:0/3313381325 conn(0x7ff1f00cfaf0 msgr2=0x7ff1f00cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:04.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1f5800640 1 -- 172.21.15.94:0/3313381325 shutdown_connections 2024-09-10T23:34:04.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1f5800640 1 -- 172.21.15.94:0/3313381325 wait complete. 2024-09-10T23:34:04.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1f5800640 1 Processor -- start 2024-09-10T23:34:04.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1f5800640 1 -- start start 2024-09-10T23:34:04.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1f5800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f006fc00 0x7ff1f0151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:04.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1f5800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f00cb4d0 0x7ff1f00c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1f5800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f0158790 0x7ff1f01fbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1f5800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f01680c0 con 0x7ff1f00cb4d0 2024-09-10T23:34:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1f5800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0167f40 con 0x7ff1f006fc00 2024-09-10T23:34:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1f5800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f0168240 con 0x7ff1f0158790 2024-09-10T23:34:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1ee000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f00cb4d0 0x7ff1f00c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1ee000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f00cb4d0 0x7ff1f00c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33956/0 (socket says 172.21.15.94:33956) 2024-09-10T23:34:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1ee000640 1 -- 172.21.15.94:0/2834619500 learned_addr learned my addr 172.21.15.94:0/2834619500 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1eea00640 1 --2- 172.21.15.94:0/2834619500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f006fc00 0x7ff1f0151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1ef400640 1 --2- 172.21.15.94:0/2834619500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f0158790 0x7ff1f01fbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:04.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1ee000640 1 -- 172.21.15.94:0/2834619500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f0158790 msgr2=0x7ff1f01fbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:04.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1ee000640 1 --2- 172.21.15.94:0/2834619500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f0158790 0x7ff1f01fbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1ee000640 1 -- 172.21.15.94:0/2834619500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f006fc00 msgr2=0x7ff1f0151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:04.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1ee000640 1 --2- 172.21.15.94:0/2834619500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f006fc00 0x7ff1f0151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.719+0000 7ff1ee000640 1 -- 172.21.15.94:0/2834619500 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f01fc280 con 0x7ff1f00cb4d0 2024-09-10T23:34:04.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.723+0000 7ff1ee000640 1 --2- 172.21.15.94:0/2834619500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f00cb4d0 0x7ff1f00c7c10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00d980 tx=0x7ff1dc00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:04.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.723+0000 7ff1e6a00640 1 -- 172.21.15.94:0/2834619500 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc017070 con 0x7ff1f00cb4d0 2024-09-10T23:34:04.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.723+0000 7ff1e6a00640 1 -- 172.21.15.94:0/2834619500 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1dc009d60 con 0x7ff1f00cb4d0 2024-09-10T23:34:04.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.723+0000 7ff1e6a00640 1 -- 172.21.15.94:0/2834619500 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc005280 con 0x7ff1f00cb4d0 2024-09-10T23:34:04.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.723+0000 7ff1f5800640 1 -- 172.21.15.94:0/2834619500 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f013fb40 con 0x7ff1f00cb4d0 2024-09-10T23:34:04.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.723+0000 7ff1f5800640 1 -- 172.21.15.94:0/2834619500 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f0140100 con 0x7ff1f00cb4d0 2024-09-10T23:34:04.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.723+0000 7ff1f5800640 1 -- 172.21.15.94:0/2834619500 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f00c49b0 con 0x7ff1f00cb4d0 2024-09-10T23:34:04.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.723+0000 7ff1e6a00640 1 -- 172.21.15.94:0/2834619500 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7ff1dc005420 con 0x7ff1f00cb4d0 2024-09-10T23:34:04.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.723+0000 7ff1e6a00640 1 --2- 172.21.15.94:0/2834619500 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff1d0072790 0x7ff1d0074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:04.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.723+0000 7ff1e6a00640 1 -- 172.21.15.94:0/2834619500 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7ff1dc093730 con 0x7ff1f00cb4d0 2024-09-10T23:34:04.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.723+0000 7ff1eea00640 1 --2- 172.21.15.94:0/2834619500 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff1d0072790 0x7ff1d0074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:04.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.727+0000 7ff1eea00640 1 --2- 172.21.15.94:0/2834619500 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff1d0072790 0x7ff1d0074c50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0007fd0 tx=0x7ff1e0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:04.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.727+0000 7ff1e6a00640 1 -- 172.21.15.94:0/2834619500 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff1f01fc280 con 0x7ff1f00cb4d0 2024-09-10T23:34:04.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.951+0000 7ff1f5800640 1 -- 172.21.15.94:0/2834619500 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff1f00d0850 con 0x7ff1f00cb4d0 2024-09-10T23:34:04.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.955+0000 7ff1e6a00640 1 -- 172.21.15.94:0/2834619500 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v29) ==== 190+0+147 (secure 0 0 0) 0x7ff1dc067240 con 0x7ff1f00cb4d0 2024-09-10T23:34:04.960 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-10T23:34:04.961 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g== 2024-09-10T23:34:04.961 INFO:teuthology.orchestra.run.smithi094.stdout: caps mds = "allow" 2024-09-10T23:34:04.961 INFO:teuthology.orchestra.run.smithi094.stdout: caps mon = "allow r" 2024-09-10T23:34:04.961 INFO:teuthology.orchestra.run.smithi094.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T23:34:04.961 INFO:teuthology.orchestra.run.smithi094.stderr:updated caps for client.0 2024-09-10T23:34:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.955+0000 7ff1f5800640 1 -- 172.21.15.94:0/2834619500 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff1d0072790 msgr2=0x7ff1d0074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.955+0000 7ff1f5800640 1 --2- 172.21.15.94:0/2834619500 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff1d0072790 0x7ff1d0074c50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0007fd0 tx=0x7ff1e0009040 comp rx=0 tx=0).stop 2024-09-10T23:34:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.959+0000 7ff1f5800640 1 -- 172.21.15.94:0/2834619500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f00cb4d0 msgr2=0x7ff1f00c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.959+0000 7ff1f5800640 1 --2- 172.21.15.94:0/2834619500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f00cb4d0 0x7ff1f00c7c10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00d980 tx=0x7ff1dc00de40 comp rx=0 tx=0).stop 2024-09-10T23:34:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.959+0000 7ff1f5800640 1 -- 172.21.15.94:0/2834619500 shutdown_connections 2024-09-10T23:34:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.959+0000 7ff1f5800640 1 --2- 172.21.15.94:0/2834619500 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff1d0072790 0x7ff1d0074c50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.959+0000 7ff1f5800640 1 --2- 172.21.15.94:0/2834619500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f0158790 0x7ff1f01fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.959+0000 7ff1f5800640 1 --2- 172.21.15.94:0/2834619500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f00cb4d0 0x7ff1f00c7c10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.959+0000 7ff1f5800640 1 --2- 172.21.15.94:0/2834619500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f006fc00 0x7ff1f0151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.959+0000 7ff1f5800640 1 -- 172.21.15.94:0/2834619500 >> 172.21.15.94:0/2834619500 conn(0x7ff1f00cfaf0 msgr2=0x7ff1f00cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.959+0000 7ff1f5800640 1 -- 172.21.15.94:0/2834619500 shutdown_connections 2024-09-10T23:34:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:04.959+0000 7ff1f5800640 1 -- 172.21.15.94:0/2834619500 wait complete. 2024-09-10T23:34:04.982 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:34:05.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.119+0000 7f577f800640 1 -- 172.21.15.94:0/1034435421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57780cddb0 msgr2=0x7f57780ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:05.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.119+0000 7f577f800640 1 --2- 172.21.15.94:0/1034435421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57780cddb0 0x7f57780ce260 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5770009840 tx=0x7f577001c610 comp rx=0 tx=0).stop 2024-09-10T23:34:05.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.119+0000 7f577f800640 1 -- 172.21.15.94:0/1034435421 shutdown_connections 2024-09-10T23:34:05.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.119+0000 7f577f800640 1 --2- 172.21.15.94:0/1034435421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57780cddb0 0x7f57780ce260 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.119+0000 7f577f800640 1 --2- 172.21.15.94:0/1034435421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57780cfcb0 0x7f57780d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.119+0000 7f577f800640 1 -- 172.21.15.94:0/1034435421 >> 172.21.15.94:0/1034435421 conn(0x7f577806cdf0 msgr2=0x7f577806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:05.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.119+0000 7f577f800640 1 -- 172.21.15.94:0/1034435421 shutdown_connections 2024-09-10T23:34:05.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.119+0000 7f577f800640 1 -- 172.21.15.94:0/1034435421 wait complete. 2024-09-10T23:34:05.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.119+0000 7f577f800640 1 Processor -- start 2024-09-10T23:34:05.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.119+0000 7f577f800640 1 -- start start 2024-09-10T23:34:05.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.119+0000 7f577f800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57780cddb0 0x7f5778144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f577f800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57780cfcb0 0x7f577815ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f577f800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f577815ba50 0x7f57781f9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f577f800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5778168be0 con 0x7f57780cfcb0 2024-09-10T23:34:05.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f577f800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5778168a60 con 0x7f57780cddb0 2024-09-10T23:34:05.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f577f800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5778168d60 con 0x7f577815ba50 2024-09-10T23:34:05.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f5777e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57780cfcb0 0x7f577815ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f5777e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57780cfcb0 0x7f577815ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33974/0 (socket says 172.21.15.94:33974) 2024-09-10T23:34:05.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f5777e00640 1 -- 172.21.15.94:0/3810091144 learned_addr learned my addr 172.21.15.94:0/3810091144 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:05.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f577ce00640 1 --2- 172.21.15.94:0/3810091144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57780cddb0 0x7f5778144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f577d800640 1 --2- 172.21.15.94:0/3810091144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f577815ba50 0x7f57781f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f5777e00640 1 -- 172.21.15.94:0/3810091144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f577815ba50 msgr2=0x7f57781f9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:05.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f5777e00640 1 --2- 172.21.15.94:0/3810091144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f577815ba50 0x7f57781f9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f5777e00640 1 -- 172.21.15.94:0/3810091144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57780cddb0 msgr2=0x7f5778144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:05.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f5777e00640 1 --2- 172.21.15.94:0/3810091144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57780cddb0 0x7f5778144ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f5777e00640 1 -- 172.21.15.94:0/3810091144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57781f8cb0 con 0x7f57780cfcb0 2024-09-10T23:34:05.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f577ce00640 1 --2- 172.21.15.94:0/3810091144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57780cddb0 0x7f5778144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:34:05.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f577d800640 1 --2- 172.21.15.94:0/3810091144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f577815ba50 0x7f57781f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:34:05.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f5777e00640 1 --2- 172.21.15.94:0/3810091144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57780cfcb0 0x7f577815ea50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f576400d9b0 tx=0x7f576400de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:05.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f5774c00640 1 -- 172.21.15.94:0/3810091144 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764017070 con 0x7f57780cfcb0 2024-09-10T23:34:05.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f5774c00640 1 -- 172.21.15.94:0/3810091144 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5764004310 con 0x7f57780cfcb0 2024-09-10T23:34:05.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f577f800640 1 -- 172.21.15.94:0/3810091144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5778145b40 con 0x7f57780cfcb0 2024-09-10T23:34:05.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f5774c00640 1 -- 172.21.15.94:0/3810091144 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764009c30 con 0x7f57780cfcb0 2024-09-10T23:34:05.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f577f800640 1 -- 172.21.15.94:0/3810091144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5778145860 con 0x7f57780cfcb0 2024-09-10T23:34:05.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f577f800640 1 -- 172.21.15.94:0/3810091144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57780ce260 con 0x7f57780cfcb0 2024-09-10T23:34:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.123+0000 7f5774c00640 1 -- 172.21.15.94:0/3810091144 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f5764003600 con 0x7f57780cfcb0 2024-09-10T23:34:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.127+0000 7f5774c00640 1 --2- 172.21.15.94:0/3810091144 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f57580727c0 0x7f5758074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.127+0000 7f5774c00640 1 -- 172.21.15.94:0/3810091144 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f5764093760 con 0x7f57780cfcb0 2024-09-10T23:34:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.127+0000 7f577ce00640 1 --2- 172.21.15.94:0/3810091144 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f57580727c0 0x7f5758074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.127+0000 7f577ce00640 1 --2- 172.21.15.94:0/3810091144 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f57580727c0 0x7f5758074c80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f57680017b0 tx=0x7f5768009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:05.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.131+0000 7f5774c00640 1 -- 172.21.15.94:0/3810091144 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f576405eb40 con 0x7f57780cfcb0 2024-09-10T23:34:05.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.323+0000 7f577f800640 1 -- 172.21.15.94:0/3810091144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57781507b0 con 0x7f57780cfcb0 2024-09-10T23:34:05.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.327+0000 7f5774c00640 1 -- 172.21.15.94:0/3810091144 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5235 (secure 0 0 0) 0x7f576405e440 con 0x7f57780cfcb0 2024-09-10T23:34:05.331 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:05.331 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"btime":"2024-09-10T23:34:03:053056+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:34:05.331 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-09-10T23:34:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.327+0000 7f577f800640 1 -- 172.21.15.94:0/3810091144 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f57580727c0 msgr2=0x7f5758074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.327+0000 7f577f800640 1 --2- 172.21.15.94:0/3810091144 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f57580727c0 0x7f5758074c80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f57680017b0 tx=0x7f5768009040 comp rx=0 tx=0).stop 2024-09-10T23:34:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.327+0000 7f577f800640 1 -- 172.21.15.94:0/3810091144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57780cfcb0 msgr2=0x7f577815ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.327+0000 7f577f800640 1 --2- 172.21.15.94:0/3810091144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57780cfcb0 0x7f577815ea50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f576400d9b0 tx=0x7f576400de70 comp rx=0 tx=0).stop 2024-09-10T23:34:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.327+0000 7f577f800640 1 -- 172.21.15.94:0/3810091144 shutdown_connections 2024-09-10T23:34:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.327+0000 7f577f800640 1 --2- 172.21.15.94:0/3810091144 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f57580727c0 0x7f5758074c80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.327+0000 7f577f800640 1 --2- 172.21.15.94:0/3810091144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f577815ba50 0x7f57781f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.327+0000 7f577f800640 1 --2- 172.21.15.94:0/3810091144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57780cfcb0 0x7f577815ea50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.327+0000 7f577f800640 1 --2- 172.21.15.94:0/3810091144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57780cddb0 0x7f5778144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.327+0000 7f577f800640 1 -- 172.21.15.94:0/3810091144 >> 172.21.15.94:0/3810091144 conn(0x7f577806cdf0 msgr2=0x7f577806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.327+0000 7f577f800640 1 -- 172.21.15.94:0/3810091144 shutdown_connections 2024-09-10T23:34:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.327+0000 7f577f800640 1 -- 172.21.15.94:0/3810091144 wait complete. 2024-09-10T23:34:05.755 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T23:33:59.428685+0000', 'modified': '2024-09-10T23:34:02.428651+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 194816}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_194816': {'gid': 194816, 'name': 'd', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/1573303541', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 1573303541}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 1573303541}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 194816, 'qdb_cluster': [194816]} 2024-09-10T23:34:05.755 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T23:34:05.756 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T23:34:05.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.895+0000 7f5964e00640 1 -- 172.21.15.94:0/618896840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59600c49b0 msgr2=0x7f59600cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:05.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.895+0000 7f5964e00640 1 --2- 172.21.15.94:0/618896840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59600c49b0 0x7f59600cadc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5948009900 tx=0x7f594801c8e0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.895+0000 7f5964e00640 1 -- 172.21.15.94:0/618896840 shutdown_connections 2024-09-10T23:34:05.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.895+0000 7f5964e00640 1 --2- 172.21.15.94:0/618896840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59600c49b0 0x7f59600cadc0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.895+0000 7f5964e00640 1 --2- 172.21.15.94:0/618896840 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5960158790 0x7f596006fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.895+0000 7f5964e00640 1 -- 172.21.15.94:0/618896840 >> 172.21.15.94:0/618896840 conn(0x7f59600cfaf0 msgr2=0x7f59600cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:05.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.895+0000 7f5964e00640 1 -- 172.21.15.94:0/618896840 shutdown_connections 2024-09-10T23:34:05.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.895+0000 7f5964e00640 1 -- 172.21.15.94:0/618896840 wait complete. 2024-09-10T23:34:05.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.895+0000 7f5964e00640 1 Processor -- start 2024-09-10T23:34:05.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.895+0000 7f5964e00640 1 -- start start 2024-09-10T23:34:05.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f5964e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59600c49b0 0x7f5960150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f5964e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59600cb300 0x7f59601f58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f5964e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960158790 0x7f596015a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f5964e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5960168290 con 0x7f5960158790 2024-09-10T23:34:05.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f5964e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5960168110 con 0x7f59600cb300 2024-09-10T23:34:05.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f5964e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5960168410 con 0x7f59600c49b0 2024-09-10T23:34:05.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f595e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960158790 0x7f596015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f595de00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59600c49b0 0x7f5960150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f595e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960158790 0x7f596015a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33994/0 (socket says 172.21.15.94:33994) 2024-09-10T23:34:05.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f595e800640 1 -- 172.21.15.94:0/3516793766 learned_addr learned my addr 172.21.15.94:0/3516793766 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:05.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f5955400640 1 --2- 172.21.15.94:0/3516793766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59600cb300 0x7f59601f58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f595e800640 1 -- 172.21.15.94:0/3516793766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59600c49b0 msgr2=0x7f5960150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:05.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f595e800640 1 --2- 172.21.15.94:0/3516793766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59600c49b0 0x7f5960150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f595e800640 1 -- 172.21.15.94:0/3516793766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59600cb300 msgr2=0x7f59601f58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:05.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f595e800640 1 --2- 172.21.15.94:0/3516793766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59600cb300 0x7f59601f58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f595e800640 1 -- 172.21.15.94:0/3516793766 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5960151180 con 0x7f5960158790 2024-09-10T23:34:05.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f595e800640 1 --2- 172.21.15.94:0/3516793766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960158790 0x7f596015a730 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f595800c780 tx=0x7f595800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:05.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f5956a00640 1 -- 172.21.15.94:0/3516793766 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5958003ba0 con 0x7f5960158790 2024-09-10T23:34:05.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f5956a00640 1 -- 172.21.15.94:0/3516793766 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5958003d60 con 0x7f5960158790 2024-09-10T23:34:05.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f5964e00640 1 -- 172.21.15.94:0/3516793766 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5960151ce0 con 0x7f5960158790 2024-09-10T23:34:05.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f5956a00640 1 -- 172.21.15.94:0/3516793766 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595800aab0 con 0x7f5960158790 2024-09-10T23:34:05.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.899+0000 7f5964e00640 1 -- 172.21.15.94:0/3516793766 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59601522a0 con 0x7f5960158790 2024-09-10T23:34:05.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f5956a00640 1 -- 172.21.15.94:0/3516793766 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f595800ac50 con 0x7f5960158790 2024-09-10T23:34:05.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f5956a00640 1 --2- 172.21.15.94:0/3516793766 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f593c0727c0 0x7f593c074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f5968281000 1 Processor -- start 2024-09-10T23:34:05.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f5956a00640 1 -- 172.21.15.94:0/3516793766 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f5958093430 con 0x7f5960158790 2024-09-10T23:34:05.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f5968281000 1 -- start start 2024-09-10T23:34:05.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f595de00640 1 --2- 172.21.15.94:0/3516793766 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f593c0727c0 0x7f593c074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f5968281000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96b9d0 0x55cece96bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f5968281000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96c2f0 0x55cece91f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f5968281000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece9201f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f5968281000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cece8403f0 con 0x55cece91fe10 2024-09-10T23:34:05.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f5968281000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cece77db20 con 0x55cece96c2f0 2024-09-10T23:34:05.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f5968281000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x55cece76cf50 con 0x55cece96b9d0 2024-09-10T23:34:05.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f595de00640 1 --2- 172.21.15.94:0/3516793766 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f593c0727c0 0x7f593c074c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5960150930 tx=0x7f595000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:05.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f595e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece9201f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f595e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece9201f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33998/0 (socket says 172.21.15.94:33998) 2024-09-10T23:34:05.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f595e800640 1 -- 172.21.15.94:0/1510946541 learned_addr learned my addr 172.21.15.94:0/1510946541 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:05.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f595de00640 1 --2- 172.21.15.94:0/1510946541 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96b9d0 0x55cece96bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f5955400640 1 --2- 172.21.15.94:0/1510946541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96c2f0 0x55cece91f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f595e800640 1 -- 172.21.15.94:0/1510946541 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96b9d0 msgr2=0x55cece96bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:34:05.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f595e800640 1 --2- 172.21.15.94:0/1510946541 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96b9d0 0x55cece96bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f595e800640 1 -- 172.21.15.94:0/1510946541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96c2f0 msgr2=0x55cece91f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:34:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f595e800640 1 --2- 172.21.15.94:0/1510946541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96c2f0 0x55cece91f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f595e800640 1 -- 172.21.15.94:0/1510946541 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cece734590 con 0x55cece91fe10 2024-09-10T23:34:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f595de00640 1 --2- 172.21.15.94:0/1510946541 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96b9d0 0x55cece96bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:34:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f595e800640 1 --2- 172.21.15.94:0/1510946541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece9201f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f595805e020 tx=0x7f5958059ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd59d36ed32c4e37 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f594ea00640 1 -- 172.21.15.94:0/1510946541 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5958093270 con 0x55cece91fe10 2024-09-10T23:34:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f594ea00640 1 -- 172.21.15.94:0/1510946541 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f595805dba0 con 0x55cece91fe10 2024-09-10T23:34:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.903+0000 7f594ea00640 1 -- 172.21.15.94:0/1510946541 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595805e090 con 0x55cece91fe10 2024-09-10T23:34:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 -- 172.21.15.94:0/1510946541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 msgr2=0x55cece9201f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 --2- 172.21.15.94:0/1510946541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece9201f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f595805e020 tx=0x7f5958059ff0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 -- 172.21.15.94:0/1510946541 shutdown_connections 2024-09-10T23:34:05.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 --2- 172.21.15.94:0/1510946541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96c2f0 0x55cece91f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 --2- 172.21.15.94:0/1510946541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece9201f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 --2- 172.21.15.94:0/1510946541 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96b9d0 0x55cece96bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 -- 172.21.15.94:0/1510946541 >> 172.21.15.94:0/1510946541 conn(0x55cece809130 msgr2=0x55cece94ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 -- 172.21.15.94:0/1510946541 shutdown_connections 2024-09-10T23:34:05.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 -- 172.21.15.94:0/1510946541 wait complete. 2024-09-10T23:34:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 -- start start 2024-09-10T23:34:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 Processor -- start 2024-09-10T23:34:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece91fe10 0x55cece94c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece96b9d0 0x55cece94ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96c2f0 0x55cece94d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cece7075d0 con 0x55cece96b9d0 2024-09-10T23:34:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cece770d60 con 0x55cece96c2f0 2024-09-10T23:34:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x55cece713500 con 0x55cece91fe10 2024-09-10T23:34:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 10 client.0 _ll_register_callbacks cb 0x55cece717810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:34:05.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5955400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece96b9d0 0x55cece94ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5955400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece96b9d0 0x55cece94ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34004/0 (socket says 172.21.15.94:34004) 2024-09-10T23:34:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5955400640 1 -- 172.21.15.94:0/3002491496 learned_addr learned my addr 172.21.15.94:0/3002491496 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f595de00640 1 --2- 172.21.15.94:0/3002491496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece91fe10 0x55cece94c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f595e800640 1 --2- 172.21.15.94:0/3002491496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96c2f0 0x55cece94d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5955400640 1 -- 172.21.15.94:0/3002491496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece91fe10 msgr2=0x55cece94c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:34:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5955400640 1 --2- 172.21.15.94:0/3002491496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece91fe10 0x55cece94c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5955400640 1 -- 172.21.15.94:0/3002491496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96c2f0 msgr2=0x55cece94d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:34:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5955400640 1 --2- 172.21.15.94:0/3002491496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96c2f0 0x55cece94d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5955400640 1 -- 172.21.15.94:0/3002491496 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cece70b170 con 0x55cece96b9d0 2024-09-10T23:34:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5955400640 1 --2- 172.21.15.94:0/3002491496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece96b9d0 0x55cece94ce70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5948009850 tx=0x7f5948005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf9ef6e65c572e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5943400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:34:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5943400640 1 -- 172.21.15.94:0/3002491496 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59480ba020 con 0x55cece96b9d0 2024-09-10T23:34:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 -- 172.21.15.94:0/3002491496 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cece7050f0 con 0x55cece96b9d0 2024-09-10T23:34:05.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5943400640 1 -- 172.21.15.94:0/3002491496 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f5948005340 con 0x55cece96b9d0 2024-09-10T23:34:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5943400640 1 -- 172.21.15.94:0/3002491496 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f59480c4050 con 0x55cece96b9d0 2024-09-10T23:34:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 10 client.5472 fetch_fsmap learned FSMap version 25 2024-09-10T23:34:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.907+0000 7f5968281000 1 -- 172.21.15.94:0/3002491496 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x55cece7f1ef0 con 0x55cece96b9d0 2024-09-10T23:34:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5943400640 1 -- 172.21.15.94:0/3002491496 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55cece7050f0 con 0x55cece96b9d0 2024-09-10T23:34:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5943400640 1 -- 172.21.15.94:0/3002491496 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 3502+0+0 (secure 0 0 0) 0x7f59480c7020 con 0x55cece96b9d0 2024-09-10T23:34:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 10 client.5472 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T23:34:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 10 client.5472 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T23:34:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 20 client.5472 populate_metadata read hostname 'smithi094' 2024-09-10T23:34:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 1 --2- 172.21.15.94:0/3002491496 >> [v2:172.21.15.181:6836/1573303541,v1:172.21.15.181:6837/1573303541] conn(0x55cece94e0f0 0x55cece97dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 4 client.5472 mds_command: new command op to 194816 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:34:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 1 -- 172.21.15.94:0/3002491496 --> [v2:172.21.15.181:6836/1573303541,v1:172.21.15.181:6837/1573303541] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cece77fa70 con 0x55cece94e0f0 2024-09-10T23:34:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f595de00640 1 --2- 172.21.15.94:0/3002491496 >> [v2:172.21.15.181:6836/1573303541,v1:172.21.15.181:6837/1573303541] conn(0x55cece94e0f0 0x55cece97dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f595de00640 1 --2- 172.21.15.94:0/3002491496 >> [v2:172.21.15.181:6836/1573303541,v1:172.21.15.181:6837/1573303541] conn(0x55cece94e0f0 0x55cece97dc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7109f99ec7cb97c server_cookie=e6c99b6a60d1e981 in_seq=0 out_seq=0 2024-09-10T23:34:05.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5943400640 10 client.5472 ms_handle_connect on v2:172.21.15.181:6836/1573303541 2024-09-10T23:34:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5943400640 1 -- 172.21.15.94:0/3002491496 <== mds.0 v2:172.21.15.181:6836/1573303541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cece77fa70 con 0x55cece94e0f0 2024-09-10T23:34:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5943400640 10 client.5472 handle_command_reply: tid=0 2024-09-10T23:34:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 1 client.5472 shutdown 2024-09-10T23:34:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 1 -- 172.21.15.94:0/3002491496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece96b9d0 msgr2=0x55cece94ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 1 --2- 172.21.15.94:0/3002491496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece96b9d0 0x55cece94ce70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5948009850 tx=0x7f5948005dc0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 1 -- 172.21.15.94:0/3002491496 shutdown_connections 2024-09-10T23:34:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 1 --2- 172.21.15.94:0/3002491496 >> [v2:172.21.15.181:6836/1573303541,v1:172.21.15.181:6837/1573303541] conn(0x55cece94e0f0 0x55cece97dc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 1 --2- 172.21.15.94:0/3002491496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96c2f0 0x55cece94d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 1 --2- 172.21.15.94:0/3002491496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece96b9d0 0x55cece94ce70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5943400640 0 client.5472 ms_handle_reset on v2:172.21.15.181:6836/1573303541 2024-09-10T23:34:05.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 1 --2- 172.21.15.94:0/3002491496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece91fe10 0x55cece94c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 1 -- 172.21.15.94:0/3002491496 >> 172.21.15.94:0/3002491496 conn(0x55cece809130 msgr2=0x55cece91efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:05.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 1 -- 172.21.15.94:0/3002491496 shutdown_connections 2024-09-10T23:34:05.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 1 -- 172.21.15.94:0/3002491496 wait complete. 2024-09-10T23:34:05.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.911+0000 7f5968281000 20 client.5472 trim_cache size 0 max 16384 2024-09-10T23:34:05.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5968281000 1 Processor -- start 2024-09-10T23:34:05.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5968281000 1 -- start start 2024-09-10T23:34:05.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5968281000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece9ddbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5968281000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96b9d0 0x55cece9de100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5968281000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96c2f0 0x55cece9de640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5968281000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5960204410 con 0x55cece91fe10 2024-09-10T23:34:05.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5968281000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f59601f8dd0 con 0x55cece96b9d0 2024-09-10T23:34:05.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5968281000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5960155550 con 0x55cece96c2f0 2024-09-10T23:34:05.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f595de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece9ddbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f595de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece9ddbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34016/0 (socket says 172.21.15.94:34016) 2024-09-10T23:34:05.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f595de00640 1 -- 172.21.15.94:0/4056269102 learned_addr learned my addr 172.21.15.94:0/4056269102 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5955400640 1 --2- 172.21.15.94:0/4056269102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96b9d0 0x55cece9de100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f595e800640 1 --2- 172.21.15.94:0/4056269102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96c2f0 0x55cece9de640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f595de00640 1 -- 172.21.15.94:0/4056269102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96c2f0 msgr2=0x55cece9de640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:34:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f595de00640 1 --2- 172.21.15.94:0/4056269102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96c2f0 0x55cece9de640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f595de00640 1 -- 172.21.15.94:0/4056269102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96b9d0 msgr2=0x55cece9de100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:34:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f595de00640 1 --2- 172.21.15.94:0/4056269102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96b9d0 0x55cece9de100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f595de00640 1 -- 172.21.15.94:0/4056269102 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cece7c8d90 con 0x55cece91fe10 2024-09-10T23:34:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5955400640 1 --2- 172.21.15.94:0/4056269102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96b9d0 0x55cece9de100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:34:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f595e800640 1 --2- 172.21.15.94:0/4056269102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96c2f0 0x55cece9de640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:34:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f595de00640 1 --2- 172.21.15.94:0/4056269102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece9ddbc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x55cece733490 tx=0x7f5950013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b7aa0a511ad956 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f594e000640 1 -- 172.21.15.94:0/4056269102 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950011bd0 con 0x55cece91fe10 2024-09-10T23:34:05.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f594e000640 1 -- 172.21.15.94:0/4056269102 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5950011d70 con 0x55cece91fe10 2024-09-10T23:34:05.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f594e000640 1 -- 172.21.15.94:0/4056269102 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950005280 con 0x55cece91fe10 2024-09-10T23:34:05.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5968281000 1 -- 172.21.15.94:0/4056269102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 msgr2=0x55cece9ddbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:05.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5968281000 1 --2- 172.21.15.94:0/4056269102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece9ddbc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x55cece733490 tx=0x7f5950013000 comp rx=0 tx=0).stop 2024-09-10T23:34:05.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5968281000 1 -- 172.21.15.94:0/4056269102 shutdown_connections 2024-09-10T23:34:05.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5968281000 1 --2- 172.21.15.94:0/4056269102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96b9d0 0x55cece9de100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5968281000 1 --2- 172.21.15.94:0/4056269102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96c2f0 0x55cece9de640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5968281000 1 --2- 172.21.15.94:0/4056269102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece9ddbc0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.931+0000 7f5968281000 1 -- 172.21.15.94:0/4056269102 >> 172.21.15.94:0/4056269102 conn(0x55cece809130 msgr2=0x55cece978fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:05.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 1 -- 172.21.15.94:0/4056269102 shutdown_connections 2024-09-10T23:34:05.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 1 -- 172.21.15.94:0/4056269102 wait complete. 2024-09-10T23:34:05.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 1 -- start start 2024-09-10T23:34:05.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 1 Processor -- start 2024-09-10T23:34:05.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece97f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96b9d0 0x55cece97fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96c2f0 0x55cece980210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f59601f8780 con 0x55cece91fe10 2024-09-10T23:34:05.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cece982ce0 con 0x55cece96b9d0 2024-09-10T23:34:05.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x55cece770070 con 0x55cece96c2f0 2024-09-10T23:34:05.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 10 client.0 _ll_register_callbacks cb 0x7f5948009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:34:05.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f595de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece97f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f595de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece97f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34022/0 (socket says 172.21.15.94:34022) 2024-09-10T23:34:05.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f595de00640 1 -- 172.21.15.94:0/3514964614 learned_addr learned my addr 172.21.15.94:0/3514964614 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:05.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f595de00640 1 -- 172.21.15.94:0/3514964614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96c2f0 msgr2=0x55cece980210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:34:05.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5955400640 1 --2- 172.21.15.94:0/3514964614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96b9d0 0x55cece97fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f595e800640 1 --2- 172.21.15.94:0/3514964614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96c2f0 0x55cece980210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f595de00640 1 --2- 172.21.15.94:0/3514964614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96c2f0 0x55cece980210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f595de00640 1 -- 172.21.15.94:0/3514964614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96b9d0 msgr2=0x55cece97fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:34:05.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f595de00640 1 --2- 172.21.15.94:0/3514964614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96b9d0 0x55cece97fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f595de00640 1 -- 172.21.15.94:0/3514964614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cece733a90 con 0x55cece91fe10 2024-09-10T23:34:05.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f595e800640 1 --2- 172.21.15.94:0/3514964614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96c2f0 0x55cece980210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:34:05.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5955400640 1 --2- 172.21.15.94:0/3514964614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96b9d0 0x55cece97fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:34:05.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f595de00640 1 --2- 172.21.15.94:0/3514964614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece97f790 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5950004010 tx=0x7f595000bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3548f7f7beae48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:05.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5943400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:34:05.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5943400640 1 -- 172.21.15.94:0/3514964614 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950029420 con 0x55cece91fe10 2024-09-10T23:34:05.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5943400640 1 -- 172.21.15.94:0/3514964614 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f595003aad0 con 0x55cece91fe10 2024-09-10T23:34:05.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 1 -- 172.21.15.94:0/3514964614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5950011d70 con 0x55cece91fe10 2024-09-10T23:34:05.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5943400640 1 -- 172.21.15.94:0/3514964614 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f595003acd0 con 0x55cece91fe10 2024-09-10T23:34:05.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 10 client.5478 fetch_fsmap learned FSMap version 25 2024-09-10T23:34:05.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 1 -- 172.21.15.94:0/3514964614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x55cece712050 con 0x55cece91fe10 2024-09-10T23:34:05.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5943400640 1 -- 172.21.15.94:0/3514964614 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5950011d70 con 0x55cece91fe10 2024-09-10T23:34:05.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5943400640 1 -- 172.21.15.94:0/3514964614 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) ==== 3502+0+0 (secure 0 0 0) 0x7f59500295c0 con 0x55cece91fe10 2024-09-10T23:34:05.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 10 client.5478 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T23:34:05.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 10 client.5478 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T23:34:05.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 20 client.5478 populate_metadata read hostname 'smithi094' 2024-09-10T23:34:05.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 1 --2- 172.21.15.94:0/3514964614 >> [v2:172.21.15.181:6836/1573303541,v1:172.21.15.181:6837/1573303541] conn(0x55cece9e7c30 0x55cece9ea020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:05.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 4 client.5478 mds_command: new command op to 194816 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T23:34:05.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.935+0000 7f5968281000 1 -- 172.21.15.94:0/3514964614 --> [v2:172.21.15.181:6836/1573303541,v1:172.21.15.181:6837/1573303541] -- command(tid 0: {"prefix": "status"}) -- 0x55cece77b520 con 0x55cece9e7c30 2024-09-10T23:34:05.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5955400640 1 --2- 172.21.15.94:0/3514964614 >> [v2:172.21.15.181:6836/1573303541,v1:172.21.15.181:6837/1573303541] conn(0x55cece9e7c30 0x55cece9ea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:05.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5955400640 1 --2- 172.21.15.94:0/3514964614 >> [v2:172.21.15.181:6836/1573303541,v1:172.21.15.181:6837/1573303541] conn(0x55cece9e7c30 0x55cece9ea020 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a76b5d22d5b44c64 server_cookie=a9c9e1b9e2e0291e in_seq=0 out_seq=0 2024-09-10T23:34:05.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5943400640 10 client.5478 ms_handle_connect on v2:172.21.15.181:6836/1573303541 2024-09-10T23:34:05.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5943400640 1 -- 172.21.15.94:0/3514964614 <== mds.0 v2:172.21.15.181:6836/1573303541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55cece77b520 con 0x55cece9e7c30 2024-09-10T23:34:05.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5943400640 10 client.5478 handle_command_reply: tid=0 2024-09-10T23:34:05.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5968281000 1 client.5478 shutdown 2024-09-10T23:34:05.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5968281000 1 -- 172.21.15.94:0/3514964614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 msgr2=0x55cece97f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:05.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5968281000 1 --2- 172.21.15.94:0/3514964614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece97f790 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5950004010 tx=0x7f595000bc60 comp rx=0 tx=0).stop 2024-09-10T23:34:05.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5968281000 1 -- 172.21.15.94:0/3514964614 shutdown_connections 2024-09-10T23:34:05.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5968281000 1 --2- 172.21.15.94:0/3514964614 >> [v2:172.21.15.181:6836/1573303541,v1:172.21.15.181:6837/1573303541] conn(0x55cece9e7c30 0x55cece9ea020 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5968281000 1 --2- 172.21.15.94:0/3514964614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cece96b9d0 0x55cece97fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5968281000 1 --2- 172.21.15.94:0/3514964614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55cece96c2f0 0x55cece980210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5943400640 0 client.5478 ms_handle_reset on v2:172.21.15.181:6836/1573303541 2024-09-10T23:34:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5968281000 1 --2- 172.21.15.94:0/3514964614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cece91fe10 0x55cece97f790 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5968281000 1 -- 172.21.15.94:0/3514964614 >> 172.21.15.94:0/3514964614 conn(0x55cece809130 msgr2=0x55cece927a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5968281000 1 -- 172.21.15.94:0/3514964614 shutdown_connections 2024-09-10T23:34:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5968281000 1 -- 172.21.15.94:0/3514964614 wait complete. 2024-09-10T23:34:05.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5968281000 20 client.5478 trim_cache size 0 max 16384 2024-09-10T23:34:05.948 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:34:05.948 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "1798cdb9-9937-4918-a569-86b41a92817e", 2024-09-10T23:34:05.949 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-09-10T23:34:05.949 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 194816, 2024-09-10T23:34:05.949 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-10T23:34:05.949 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-10T23:34:05.949 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-10T23:34:05.949 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 5.8991323539999998, 2024-09-10T23:34:05.949 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 25, 2024-09-10T23:34:05.949 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 44, 2024-09-10T23:34:05.949 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 44, 2024-09-10T23:34:05.949 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 10.379408441000001 2024-09-10T23:34:05.949 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:34:05.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5964e00640 1 -- 172.21.15.94:0/3516793766 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f593c0727c0 msgr2=0x7f593c074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:05.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5964e00640 1 --2- 172.21.15.94:0/3516793766 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f593c0727c0 0x7f593c074c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5960150930 tx=0x7f595000b2e0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5964e00640 1 -- 172.21.15.94:0/3516793766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960158790 msgr2=0x7f596015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:05.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5964e00640 1 --2- 172.21.15.94:0/3516793766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960158790 0x7f596015a730 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f595800c780 tx=0x7f595800cc40 comp rx=0 tx=0).stop 2024-09-10T23:34:05.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5964e00640 1 -- 172.21.15.94:0/3516793766 shutdown_connections 2024-09-10T23:34:05.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5964e00640 1 --2- 172.21.15.94:0/3516793766 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f593c0727c0 0x7f593c074c80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5964e00640 1 --2- 172.21.15.94:0/3516793766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59600cb300 0x7f59601f58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5964e00640 1 --2- 172.21.15.94:0/3516793766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5960158790 0x7f596015a730 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5964e00640 1 --2- 172.21.15.94:0/3516793766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59600c49b0 0x7f5960150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:05.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5964e00640 1 -- 172.21.15.94:0/3516793766 >> 172.21.15.94:0/3516793766 conn(0x7f59600cfaf0 msgr2=0x7f59600b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:05.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5964e00640 1 -- 172.21.15.94:0/3516793766 shutdown_connections 2024-09-10T23:34:05.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:05.939+0000 7f5964e00640 1 -- 172.21.15.94:0/3516793766 wait complete. 2024-09-10T23:34:05.963 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:34:06.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.099+0000 7fd92b200640 1 -- 172.21.15.94:0/3005122583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd92c15ba30 msgr2=0x7fd92c0b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:06.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.099+0000 7fd92b200640 1 --2- 172.21.15.94:0/3005122583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd92c15ba30 0x7fd92c0b1bd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0c5320 tx=0x7fd91c01ca90 comp rx=0 tx=0).stop 2024-09-10T23:34:06.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.099+0000 7fd92b200640 1 -- 172.21.15.94:0/3005122583 shutdown_connections 2024-09-10T23:34:06.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.099+0000 7fd92b200640 1 --2- 172.21.15.94:0/3005122583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd92c0d02e0 0x7fd92c15f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:06.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.099+0000 7fd92b200640 1 --2- 172.21.15.94:0/3005122583 >> v2:172.21.15.181:3300/0 conn(0x7fd92c0b2110 0x7fd92c0cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:06.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.099+0000 7fd92b200640 1 --2- 172.21.15.94:0/3005122583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd92c15ba30 0x7fd92c0b1bd0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:06.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.099+0000 7fd92b200640 1 -- 172.21.15.94:0/3005122583 >> 172.21.15.94:0/3005122583 conn(0x7fd92c09aa70 msgr2=0x7fd92c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:06.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.099+0000 7fd92b200640 1 -- 172.21.15.94:0/3005122583 shutdown_connections 2024-09-10T23:34:06.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.099+0000 7fd92b200640 1 -- 172.21.15.94:0/3005122583 wait complete. 2024-09-10T23:34:06.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd92b200640 1 Processor -- start 2024-09-10T23:34:06.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd92b200640 1 -- start start 2024-09-10T23:34:06.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd92b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd92c0b2110 0x7fd92c157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:06.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd92b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92c0d02e0 0x7fd92c159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:06.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd92b200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd92c15ba30 0x7fd92c14c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:06.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd92b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c0af350 con 0x7fd92c0b2110 2024-09-10T23:34:06.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd92b200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c0af1d0 con 0x7fd92c0d02e0 2024-09-10T23:34:06.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd92b200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd92c0af4d0 con 0x7fd92c15ba30 2024-09-10T23:34:06.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd929e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd92c0b2110 0x7fd92c157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:06.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd929e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd92c0b2110 0x7fd92c157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34028/0 (socket says 172.21.15.94:34028) 2024-09-10T23:34:06.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd929e00640 1 -- 172.21.15.94:0/3308980664 learned_addr learned my addr 172.21.15.94:0/3308980664 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd921400640 1 --2- 172.21.15.94:0/3308980664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92c0d02e0 0x7fd92c159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd92a800640 1 --2- 172.21.15.94:0/3308980664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd92c15ba30 0x7fd92c14c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd929e00640 1 -- 172.21.15.94:0/3308980664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd92c15ba30 msgr2=0x7fd92c14c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd929e00640 1 --2- 172.21.15.94:0/3308980664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd92c15ba30 0x7fd92c14c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd929e00640 1 -- 172.21.15.94:0/3308980664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92c0d02e0 msgr2=0x7fd92c159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd929e00640 1 --2- 172.21.15.94:0/3308980664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92c0d02e0 0x7fd92c159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd929e00640 1 -- 172.21.15.94:0/3308980664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd92c158670 con 0x7fd92c0b2110 2024-09-10T23:34:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd92a800640 1 --2- 172.21.15.94:0/3308980664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd92c15ba30 0x7fd92c14c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:34:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd929e00640 1 --2- 172.21.15.94:0/3308980664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd92c0b2110 0x7fd92c157f00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd91c000f90 tx=0x7fd91c0a5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd922a00640 1 -- 172.21.15.94:0/3308980664 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0bb070 con 0x7fd92c0b2110 2024-09-10T23:34:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd922a00640 1 -- 172.21.15.94:0/3308980664 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd91c004e00 con 0x7fd92c0b2110 2024-09-10T23:34:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd92b200640 1 -- 172.21.15.94:0/3308980664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd92c158950 con 0x7fd92c0b2110 2024-09-10T23:34:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd922a00640 1 -- 172.21.15.94:0/3308980664 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0b2040 con 0x7fd92c0b2110 2024-09-10T23:34:06.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.103+0000 7fd92b200640 1 -- 172.21.15.94:0/3308980664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd92c158f10 con 0x7fd92c0b2110 2024-09-10T23:34:06.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.107+0000 7fd92b200640 1 -- 172.21.15.94:0/3308980664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd92c15f540 con 0x7fd92c0b2110 2024-09-10T23:34:06.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.107+0000 7fd922a00640 1 -- 172.21.15.94:0/3308980664 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7fd91c0b5070 con 0x7fd92c0b2110 2024-09-10T23:34:06.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.107+0000 7fd922a00640 1 --2- 172.21.15.94:0/3308980664 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd9080727c0 0x7fd908074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:06.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.107+0000 7fd922a00640 1 -- 172.21.15.94:0/3308980664 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fd91c12d4e0 con 0x7fd92c0b2110 2024-09-10T23:34:06.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.107+0000 7fd921400640 1 --2- 172.21.15.94:0/3308980664 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd9080727c0 0x7fd908074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:06.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.107+0000 7fd921400640 1 --2- 172.21.15.94:0/3308980664 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd9080727c0 0x7fd908074c80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd914005fd0 tx=0x7fd914005db0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:06.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.111+0000 7fd922a00640 1 -- 172.21.15.94:0/3308980664 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd91c016970 con 0x7fd92c0b2110 2024-09-10T23:34:06.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.311+0000 7fd92b200640 1 -- 172.21.15.94:0/3308980664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd92c0715b0 con 0x7fd92c0b2110 2024-09-10T23:34:06.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.311+0000 7fd922a00640 1 -- 172.21.15.94:0/3308980664 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7fd91c0f8820 con 0x7fd92c0b2110 2024-09-10T23:34:06.318 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:34:06.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.315+0000 7fd92b200640 1 -- 172.21.15.94:0/3308980664 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd9080727c0 msgr2=0x7fd908074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:06.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.315+0000 7fd92b200640 1 --2- 172.21.15.94:0/3308980664 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd9080727c0 0x7fd908074c80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd914005fd0 tx=0x7fd914005db0 comp rx=0 tx=0).stop 2024-09-10T23:34:06.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.315+0000 7fd92b200640 1 -- 172.21.15.94:0/3308980664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd92c0b2110 msgr2=0x7fd92c157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:06.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.315+0000 7fd92b200640 1 --2- 172.21.15.94:0/3308980664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd92c0b2110 0x7fd92c157f00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd91c000f90 tx=0x7fd91c0a5920 comp rx=0 tx=0).stop 2024-09-10T23:34:06.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.315+0000 7fd92b200640 1 -- 172.21.15.94:0/3308980664 shutdown_connections 2024-09-10T23:34:06.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.315+0000 7fd92b200640 1 --2- 172.21.15.94:0/3308980664 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd9080727c0 0x7fd908074c80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:06.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.315+0000 7fd92b200640 1 --2- 172.21.15.94:0/3308980664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92c0d02e0 0x7fd92c159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:06.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.315+0000 7fd92b200640 1 --2- 172.21.15.94:0/3308980664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd92c15ba30 0x7fd92c14c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:06.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.315+0000 7fd92b200640 1 --2- 172.21.15.94:0/3308980664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd92c0b2110 0x7fd92c157f00 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:06.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.315+0000 7fd92b200640 1 -- 172.21.15.94:0/3308980664 >> 172.21.15.94:0/3308980664 conn(0x7fd92c09aa70 msgr2=0x7fd92c070760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:06.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.315+0000 7fd92b200640 1 -- 172.21.15.94:0/3308980664 shutdown_connections 2024-09-10T23:34:06.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06.315+0000 7fd92b200640 1 -- 172.21.15.94:0/3308980664 wait complete. 2024-09-10T23:34:06.339 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:34:06.340 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:34:06.340 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T23:34:06.340 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T23:34:06.340 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T23:34:06.340 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:34:06.340 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:34:06.340 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:34:06.340 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:34:06.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:06.341 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T23:34:06.350 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:34:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:34:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:34:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:34:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T23:34:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:34:06.350 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:34:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:34:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T23:34:06.351 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:34:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:34:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T23:34:06.351 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:34:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:34:06.351 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:34:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:34:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:34:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42365sec preferred_lft 42365sec 2024-09-10T23:34:06.352 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-09-10T23:34:06.352 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:34:06.352 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:34:06.352 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:ed:7d:46:db brd ff:ff:ff:ff:ff:ff 2024-09-10T23:34:06.352 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:34:06.352 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:34:06.352 INFO:teuthology.orchestra.run.smithi094.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:34:06.352 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:34:06.352 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:34:06.352 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:34:06.352 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:34:06.352 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:34:06.353 INFO:teuthology.orchestra.run.smithi094.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:34:06.353 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:34:06.353 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:34:06.353 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:34:06.353 INFO:teuthology.orchestra.run.smithi094.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:34:06.353 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:34:06.353 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:34:06.353 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:34:06.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:06.355 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:34:06.415 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T23:34:06.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:06.418 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T23:34:06.474 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T23:34:06.475 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:34:06.475 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:34:06.475 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:34:06.475 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:34:06.475 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:34:06.475 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:34:06.475 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:34:06.475 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:34:06.475 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:34:06.475 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:34:06.476 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:34:06.476 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:34:06.476 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:34:06.476 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:34:06.476 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:34:06.476 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:34:06.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:34:06.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:34:06.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:06.611 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T23:34:06.620 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T23:34:06.621 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T23:34:06.621 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:34:06.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:06.621 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-09-10T23:34:06.681 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T23:34:06.681 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:34:06.681 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T23:34:06.734 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:34:06.734 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:34:06.734 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:34:06.785 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:34:06.845 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:34:06.895 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:34:06.895 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:34:06.896 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:34:06.896 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:34:06.896 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:34:06.896 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:34:06.896 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:34:06.896 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:34:06.896 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:34:06.896 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:34:06.896 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:34:06.896 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:34:06.896 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:34:06.897 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:34:06.897 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:34:06.897 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:34:06.897 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:34:06.897 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:34:06.897 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:34:06.897 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:34:06.897 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:34:06.897 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:34:06.897 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:34:06.897 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:34:06.897 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:34:06.897 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:34:06.898 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:34:06.898 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:34:06.898 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:34:06.898 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:34:06.898 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:34:06.898 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:34:06.898 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:34:06.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:06.899 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:34:06.949 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:34:06.950 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T23:34:06.992 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:34:07.013 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:34:07.067 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:34:07.068 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:34:07.068 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:34:07.068 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:34:07.068 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:34:07.068 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:34:07.068 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:34:07.068 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:34:07.068 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:34:07.068 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:34:07.068 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:34:07.068 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:34:07.068 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:34:07.069 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:34:07.069 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:34:07.069 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:34:07.069 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:34:07.069 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:34:07.069 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:34:07.069 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:34:07.069 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:34:07.069 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:34:07.069 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:34:07.069 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:34:07.069 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:34:07.069 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:34:07.069 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:34:07.070 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:34:07.070 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:34:07.070 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:34:07.070 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:34:07.070 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:34:07.070 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:34:07.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:07.071 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:34:07.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:2024-09-10T23:34:07.083+0000 7fe28f328580 -1 init, newargv = 0x55961044cc50 newargc=15 2024-09-10T23:34:07.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[38541]: starting ceph client 2024-09-10T23:34:07.095 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[38541]: starting fuse 2024-09-10T23:34:07.096 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T23:34:07.096 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:34:07.096 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:34:07.096 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-09-10T23:34:07.096 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-09-10T23:34:07.097 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-09-10T23:34:07.097 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-09-10T23:34:07.097 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:34:07.097 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-09-10T23:34:07.097 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T23:34:07.097 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-09-10T23:34:07.097 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T23:34:07.097 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:34:07.097 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-09-10T23:34:07.097 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-09-10T23:34:07.097 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-09-10T23:34:07.097 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-09-10T23:34:07.097 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:34:07.097 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-09-10T23:34:07.098 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:34:07.098 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:34:07.098 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:34:07.098 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-09-10T23:34:07.098 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-09-10T23:34:07.098 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-09-10T23:34:07.098 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:34:07.098 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:34:07.098 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.0")) 2024-09-10T23:34:07.098 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-09-10T23:34:07.217 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.0.38541.asok 2024-09-10T23:34:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:34:07.229 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38541.asok 2024-09-10T23:34:07.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:07.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38541.asok status 2024-09-10T23:34:07.398 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:34:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-09-10T23:34:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:34:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:34:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "0", 2024-09-10T23:34:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-09-10T23:34:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T23:34:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "38541", 2024-09-10T23:34:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-09-10T23:34:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:34:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-09-10T23:34:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-09-10T23:34:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 5493, 2024-09-10T23:34:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-09-10T23:34:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-09-10T23:34:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-09-10T23:34:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 5493 2024-09-10T23:34:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:34:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:34:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:34:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:34:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1626264990 2024-09-10T23:34:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:34:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:34:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:34:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:34:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:34:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1626264990 2024-09-10T23:34:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:34:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.5493 192.168.0.1:0/1626264990", 2024-09-10T23:34:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/1626264990", 2024-09-10T23:34:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-09-10T23:34:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 25, 2024-09-10T23:34:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 44, 2024-09-10T23:34:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:34:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-09-10T23:34:07.401 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-09-10T23:34:07.401 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:34:07.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:07.413 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-09-10T23:34:07.413 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T23:34:07.422 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-09-10T23:34:07.422 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T23:34:07.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:07.422 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:34:07.490 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:34:07.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.671+0000 7f04f8c00640 1 -- 172.21.15.94:0/216455991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f40b2ec0 msgr2=0x7f04f40d20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:07.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.671+0000 7f04f8c00640 1 --2- 172.21.15.94:0/216455991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f40b2ec0 0x7f04f40d20c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f04f4073fc0 tx=0x7f04ec01c800 comp rx=0 tx=0).stop 2024-09-10T23:34:07.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.671+0000 7f04f8c00640 1 -- 172.21.15.94:0/216455991 shutdown_connections 2024-09-10T23:34:07.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.671+0000 7f04f8c00640 1 --2- 172.21.15.94:0/216455991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f40b2ec0 0x7f04f40d20c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:07.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.671+0000 7f04f8c00640 1 --2- 172.21.15.94:0/216455991 >> v2:172.21.15.181:3300/0 conn(0x7f04f40ae980 0x7f04f40d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:07.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.671+0000 7f04f8c00640 1 -- 172.21.15.94:0/216455991 >> 172.21.15.94:0/216455991 conn(0x7f04f409aa70 msgr2=0x7f04f409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:07.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.671+0000 7f04f8c00640 1 -- 172.21.15.94:0/216455991 shutdown_connections 2024-09-10T23:34:07.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.671+0000 7f04f8c00640 1 -- 172.21.15.94:0/216455991 wait complete. 2024-09-10T23:34:07.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f8c00640 1 Processor -- start 2024-09-10T23:34:07.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f8c00640 1 -- start start 2024-09-10T23:34:07.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f8c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04f40ae980 0x7f04f41f9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:07.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f8c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04f40b2ec0 0x7f04f4158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:07.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f8c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f415bd30 0x7f04f4159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:07.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f8c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04f4168d00 con 0x7f04f415bd30 2024-09-10T23:34:07.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f8c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04f4168b80 con 0x7f04f40ae980 2024-09-10T23:34:07.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f8c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f04f4168e80 con 0x7f04f40b2ec0 2024-09-10T23:34:07.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f415bd30 0x7f04f4159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:07.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f415bd30 0x7f04f4159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34044/0 (socket says 172.21.15.94:34044) 2024-09-10T23:34:07.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f2800640 1 -- 172.21.15.94:0/43276763 learned_addr learned my addr 172.21.15.94:0/43276763 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:07.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04e9400640 1 --2- 172.21.15.94:0/43276763 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04f40b2ec0 0x7f04f4158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:07.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f1e00640 1 --2- 172.21.15.94:0/43276763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04f40ae980 0x7f04f41f9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:07.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f2800640 1 -- 172.21.15.94:0/43276763 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04f40b2ec0 msgr2=0x7f04f4158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:07.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f2800640 1 --2- 172.21.15.94:0/43276763 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04f40b2ec0 0x7f04f4158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:07.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f2800640 1 -- 172.21.15.94:0/43276763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04f40ae980 msgr2=0x7f04f41f9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:07.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f2800640 1 --2- 172.21.15.94:0/43276763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04f40ae980 0x7f04f41f9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:07.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f2800640 1 -- 172.21.15.94:0/43276763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f40c9f60 con 0x7f04f415bd30 2024-09-10T23:34:07.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f1e00640 1 --2- 172.21.15.94:0/43276763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04f40ae980 0x7f04f41f9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:34:07.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f2800640 1 --2- 172.21.15.94:0/43276763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f415bd30 0x7f04f4159070 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009850 tx=0x7f04ec004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:07.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04eaa00640 1 -- 172.21.15.94:0/43276763 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec0bd070 con 0x7f04f415bd30 2024-09-10T23:34:07.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04eaa00640 1 -- 172.21.15.94:0/43276763 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04ec003600 con 0x7f04f415bd30 2024-09-10T23:34:07.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04eaa00640 1 -- 172.21.15.94:0/43276763 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec0b2040 con 0x7f04f415bd30 2024-09-10T23:34:07.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f8c00640 1 -- 172.21.15.94:0/43276763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f40ca240 con 0x7f04f415bd30 2024-09-10T23:34:07.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.675+0000 7f04f8c00640 1 -- 172.21.15.94:0/43276763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f40ca800 con 0x7f04f415bd30 2024-09-10T23:34:07.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.679+0000 7f04f8c00640 1 -- 172.21.15.94:0/43276763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f415c110 con 0x7f04f415bd30 2024-09-10T23:34:07.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.679+0000 7f04eaa00640 1 -- 172.21.15.94:0/43276763 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f04ec003a50 con 0x7f04f415bd30 2024-09-10T23:34:07.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.679+0000 7f04eaa00640 1 --2- 172.21.15.94:0/43276763 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f04d0072790 0x7f04d0074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:07.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.679+0000 7f04eaa00640 1 -- 172.21.15.94:0/43276763 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f04ec12db10 con 0x7f04f415bd30 2024-09-10T23:34:07.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.679+0000 7f04f1e00640 1 --2- 172.21.15.94:0/43276763 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f04d0072790 0x7f04d0074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:07.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.683+0000 7f04f1e00640 1 --2- 172.21.15.94:0/43276763 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f04d0072790 0x7f04d0074c50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f04e4007fd0 tx=0x7f04e4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:07.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.683+0000 7f04eaa00640 1 -- 172.21.15.94:0/43276763 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f04ec0164b0 con 0x7f04f415bd30 2024-09-10T23:34:07.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.883+0000 7f04f8c00640 1 -- 172.21.15.94:0/43276763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f04f40d28a0 con 0x7f04f415bd30 2024-09-10T23:34:07.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.883+0000 7f04eaa00640 1 -- 172.21.15.94:0/43276763 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f04ec0f8dc0 con 0x7f04f415bd30 2024-09-10T23:34:07.889 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:34:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.887+0000 7f04f8c00640 1 -- 172.21.15.94:0/43276763 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f04d0072790 msgr2=0x7f04d0074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.887+0000 7f04f8c00640 1 --2- 172.21.15.94:0/43276763 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f04d0072790 0x7f04d0074c50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f04e4007fd0 tx=0x7f04e4009040 comp rx=0 tx=0).stop 2024-09-10T23:34:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.887+0000 7f04f8c00640 1 -- 172.21.15.94:0/43276763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f415bd30 msgr2=0x7f04f4159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:07.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.887+0000 7f04f8c00640 1 --2- 172.21.15.94:0/43276763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f415bd30 0x7f04f4159070 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009850 tx=0x7f04ec004040 comp rx=0 tx=0).stop 2024-09-10T23:34:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.887+0000 7f04f8c00640 1 -- 172.21.15.94:0/43276763 shutdown_connections 2024-09-10T23:34:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.887+0000 7f04f8c00640 1 --2- 172.21.15.94:0/43276763 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f04d0072790 0x7f04d0074c50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.887+0000 7f04f8c00640 1 --2- 172.21.15.94:0/43276763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f415bd30 0x7f04f4159070 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.887+0000 7f04f8c00640 1 --2- 172.21.15.94:0/43276763 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04f40b2ec0 0x7f04f4158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.887+0000 7f04f8c00640 1 --2- 172.21.15.94:0/43276763 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04f40ae980 0x7f04f41f9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.887+0000 7f04f8c00640 1 -- 172.21.15.94:0/43276763 >> 172.21.15.94:0/43276763 conn(0x7f04f409aa70 msgr2=0x7f04f4072120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.887+0000 7f04f8c00640 1 -- 172.21.15.94:0/43276763 shutdown_connections 2024-09-10T23:34:07.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:07.887+0000 7f04f8c00640 1 -- 172.21.15.94:0/43276763 wait complete. 2024-09-10T23:34:07.910 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:34:07.911 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:34:07.911 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-09-10T23:34:07.911 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T23:34:07.911 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T23:34:07.911 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:34:07.911 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:34:07.911 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:34:07.911 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:34:07.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:07.911 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-09-10T23:34:07.921 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:34:07.922 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:34:07.922 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:34:07.922 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:34:07.922 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-09-10T23:34:07.922 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:34:07.922 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:34:07.922 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:34:07.922 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-09-10T23:34:07.922 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:34:07.922 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:34:07.922 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-09-10T23:34:07.922 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:34:07.923 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:34:07.923 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:34:07.923 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:34:07.923 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:34:07.923 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42367sec preferred_lft 42367sec 2024-09-10T23:34:07.923 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-09-10T23:34:07.923 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:34:07.923 INFO:teuthology.orchestra.run.smithi181.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:34:07.923 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 02:42:3b:2e:09:2d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:34:07.923 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:34:07.923 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:34:07.923 INFO:teuthology.orchestra.run.smithi181.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:34:07.923 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:34:07.923 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:34:07.924 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:34:07.924 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:34:07.924 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:34:07.924 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:34:07.924 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:34:07.924 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:34:07.924 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:34:07.924 INFO:teuthology.orchestra.run.smithi181.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:34:07.924 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:34:07.924 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:34:07.924 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:34:07.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:07.926 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:34:07.989 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-09-10T23:34:07.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:07.992 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-09-10T23:34:08.046 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-09-10T23:34:08.046 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:34:08.046 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:34:08.046 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:34:08.046 INFO:teuthology.orchestra.run.smithi181.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:34:08.046 INFO:teuthology.orchestra.run.smithi181.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:34:08.046 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:34:08.046 INFO:teuthology.orchestra.run.smithi181.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:34:08.046 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:34:08.046 INFO:teuthology.orchestra.run.smithi181.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:34:08.046 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:34:08.047 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:34:08.048 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:34:08.048 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:34:08.048 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:34:08.048 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:34:08.048 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:34:08.140 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:34:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:34:08.182 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:34:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:34:08.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:08.188 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-09-10T23:34:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T23:34:08.197 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T23:34:08.198 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:34:08.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:08.198 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-09-10T23:34:08.257 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T23:34:08.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:34:08.257 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T23:34:08.309 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:34:08.310 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:34:08.310 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:34:08.365 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:34:08.426 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:34:08.479 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:34:08.479 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:34:08.479 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:34:08.479 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:34:08.479 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:34:08.479 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:34:08.480 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:34:08.480 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:34:08.480 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:34:08.480 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:34:08.480 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:34:08.480 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:34:08.480 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:34:08.480 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:34:08.480 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:34:08.480 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:34:08.480 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:34:08.480 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:34:08.480 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:34:08.481 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:34:08.481 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:34:08.481 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:34:08.481 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:34:08.481 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:34:08.481 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:34:08.481 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:34:08.481 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:34:08.481 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:34:08.481 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:34:08.481 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:34:08.481 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:34:08.481 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:34:08.482 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:34:08.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:08.483 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:34:08.533 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:34:08.533 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T23:34:08.580 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:34:08.600 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:34:08.651 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:34:08.651 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:34:08.651 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:34:08.651 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:34:08.651 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:34:08.651 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:34:08.652 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:34:08.652 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:34:08.652 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:34:08.652 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:34:08.652 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:34:08.652 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:34:08.652 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:34:08.652 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:34:08.652 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:34:08.652 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:34:08.652 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:34:08.652 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:34:08.652 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:34:08.653 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:34:08.653 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:34:08.653 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:34:08.653 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:34:08.653 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:34:08.653 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:34:08.653 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:34:08.653 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:34:08.653 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:34:08.653 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:34:08.653 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:34:08.653 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:34:08.653 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:34:08.653 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:34:08.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:08.655 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:34:08.673 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:2024-09-10T23:34:08.668+0000 7f183fa49580 -1 init, newargv = 0x55a10c5fcec0 newargc=15 2024-09-10T23:34:08.673 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[34643]: starting ceph client 2024-09-10T23:34:08.680 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[34643]: starting fuse 2024-09-10T23:34:08.681 INFO:teuthology.orchestra.run.smithi181.stdout:44 2024-09-10T23:34:08.682 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:34:08.682 DEBUG:teuthology.orchestra.run.smithi181:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:34:08.682 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2024-09-10T23:34:08.682 DEBUG:teuthology.orchestra.run.smithi181:> import re 2024-09-10T23:34:08.682 DEBUG:teuthology.orchestra.run.smithi181:> import os 2024-09-10T23:34:08.682 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2024-09-10T23:34:08.682 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:34:08.682 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2024-09-10T23:34:08.682 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T23:34:08.683 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2024-09-10T23:34:08.683 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T23:34:08.683 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:34:08.683 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2024-09-10T23:34:08.683 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2024-09-10T23:34:08.683 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2024-09-10T23:34:08.683 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2024-09-10T23:34:08.683 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:34:08.683 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2024-09-10T23:34:08.683 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:34:08.683 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:34:08.683 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:34:08.683 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2024-09-10T23:34:08.683 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2024-09-10T23:34:08.683 DEBUG:teuthology.orchestra.run.smithi181:> return f 2024-09-10T23:34:08.684 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:34:08.684 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:34:08.684 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.1")) 2024-09-10T23:34:08.684 DEBUG:teuthology.orchestra.run.smithi181:> ' 2024-09-10T23:34:08.809 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.1.34643.asok 2024-09-10T23:34:08.812 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:34:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:34:08.820 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34643.asok 2024-09-10T23:34:08.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:08.820 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34643.asok status 2024-09-10T23:34:08.989 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T23:34:08.989 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2024-09-10T23:34:08.989 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:34:08.989 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:34:08.989 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "1", 2024-09-10T23:34:08.989 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2024-09-10T23:34:08.989 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T23:34:08.989 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "34643", 2024-09-10T23:34:08.989 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2024-09-10T23:34:08.989 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:34:08.990 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2024-09-10T23:34:08.990 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2024-09-10T23:34:08.990 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 194918, 2024-09-10T23:34:08.990 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2024-09-10T23:34:08.990 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2024-09-10T23:34:08.990 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2024-09-10T23:34:08.990 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 194918 2024-09-10T23:34:08.990 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:34:08.990 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:34:08.990 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:34:08.990 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:34:08.990 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 3149135162 2024-09-10T23:34:08.990 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T23:34:08.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:34:08.991 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:34:08.991 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:34:08.991 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:34:08.991 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 3149135162 2024-09-10T23:34:08.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:34:08.991 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.194918 192.168.0.1:0/3149135162", 2024-09-10T23:34:08.991 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.1:0/3149135162", 2024-09-10T23:34:08.991 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2024-09-10T23:34:08.991 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 25, 2024-09-10T23:34:08.991 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 44, 2024-09-10T23:34:08.991 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:34:08.991 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2024-09-10T23:34:08.992 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2024-09-10T23:34:08.992 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T23:34:09.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:09.004 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2024-09-10T23:34:09.004 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T23:34:09.013 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2024-09-10T23:34:09.014 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T23:34:09.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:09.014 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:34:09.076 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T23:34:09.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.223+0000 7f4757400640 1 -- 172.21.15.94:0/3666252276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4750073ff0 msgr2=0x7f47500cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.223+0000 7f4757400640 1 --2- 172.21.15.94:0/3666252276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4750073ff0 0x7f47500cfda0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4740009900 tx=0x7f474001c8e0 comp rx=0 tx=0).stop 2024-09-10T23:34:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.223+0000 7f4757400640 1 -- 172.21.15.94:0/3666252276 shutdown_connections 2024-09-10T23:34:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.223+0000 7f4757400640 1 --2- 172.21.15.94:0/3666252276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4750073ff0 0x7f47500cfda0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.223+0000 7f4757400640 1 -- 172.21.15.94:0/3666252276 >> 172.21.15.94:0/3666252276 conn(0x7f475006cdf0 msgr2=0x7f475006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.223+0000 7f4757400640 1 -- 172.21.15.94:0/3666252276 shutdown_connections 2024-09-10T23:34:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.223+0000 7f4757400640 1 -- 172.21.15.94:0/3666252276 wait complete. 2024-09-10T23:34:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4757400640 1 Processor -- start 2024-09-10T23:34:09.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4757400640 1 -- start start 2024-09-10T23:34:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4757400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4750073ff0 0x7f47501516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4757400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47500d02e0 0x7f4750158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:09.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4757400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f475015ee80 0x7f4750159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:09.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4757400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4750168df0 con 0x7f475015ee80 2024-09-10T23:34:09.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4757400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4750168c70 con 0x7f4750073ff0 2024-09-10T23:34:09.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4757400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f4750168f70 con 0x7f47500d02e0 2024-09-10T23:34:09.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4755400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f475015ee80 0x7f4750159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:09.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4754a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4750073ff0 0x7f47501516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:09.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4755400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f475015ee80 0x7f4750159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34064/0 (socket says 172.21.15.94:34064) 2024-09-10T23:34:09.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4755400640 1 -- 172.21.15.94:0/4221988578 learned_addr learned my addr 172.21.15.94:0/4221988578 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:09.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4747e00640 1 --2- 172.21.15.94:0/4221988578 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47500d02e0 0x7f4750158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:09.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4755400640 1 -- 172.21.15.94:0/4221988578 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47500d02e0 msgr2=0x7f4750158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:09.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4755400640 1 --2- 172.21.15.94:0/4221988578 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47500d02e0 0x7f4750158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:09.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4755400640 1 -- 172.21.15.94:0/4221988578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4750073ff0 msgr2=0x7f47501516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:09.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4755400640 1 --2- 172.21.15.94:0/4221988578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4750073ff0 0x7f47501516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:09.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4755400640 1 -- 172.21.15.94:0/4221988578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4750158c00 con 0x7f475015ee80 2024-09-10T23:34:09.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4755400640 1 --2- 172.21.15.94:0/4221988578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f475015ee80 0x7f4750159ca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f474800c710 tx=0x7f474800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f474d600640 1 -- 172.21.15.94:0/4221988578 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748003b80 con 0x7f475015ee80 2024-09-10T23:34:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f474d600640 1 -- 172.21.15.94:0/4221988578 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4748003d20 con 0x7f475015ee80 2024-09-10T23:34:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4757400640 1 -- 172.21.15.94:0/4221988578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4750158ee0 con 0x7f475015ee80 2024-09-10T23:34:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f474d600640 1 -- 172.21.15.94:0/4221988578 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474800aab0 con 0x7f475015ee80 2024-09-10T23:34:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.227+0000 7f4757400640 1 -- 172.21.15.94:0/4221988578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47501594a0 con 0x7f475015ee80 2024-09-10T23:34:09.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.231+0000 7f4757400640 1 -- 172.21.15.94:0/4221988578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47500cfda0 con 0x7f475015ee80 2024-09-10T23:34:09.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.231+0000 7f474d600640 1 -- 172.21.15.94:0/4221988578 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f474800ac50 con 0x7f475015ee80 2024-09-10T23:34:09.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.231+0000 7f474d600640 1 --2- 172.21.15.94:0/4221988578 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f472c0727c0 0x7f472c074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:09.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.231+0000 7f474d600640 1 -- 172.21.15.94:0/4221988578 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f47480932d0 con 0x7f475015ee80 2024-09-10T23:34:09.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.231+0000 7f4754a00640 1 --2- 172.21.15.94:0/4221988578 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f472c0727c0 0x7f472c074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:09.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.231+0000 7f4754a00640 1 --2- 172.21.15.94:0/4221988578 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f472c0727c0 0x7f472c074c80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f47400af000 tx=0x7f474001b120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:09.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.235+0000 7f474d600640 1 -- 172.21.15.94:0/4221988578 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f474800ce20 con 0x7f475015ee80 2024-09-10T23:34:09.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:09.435+0000 7f4757400640 1 -- 172.21.15.94:0/4221988578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f47500b4140 con 0x7f475015ee80 2024-09-10T23:34:10.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.079+0000 7f474d600640 1 -- 172.21.15.94:0/4221988578 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v26) ==== 136+0+0 (secure 0 0 0) 0x7f474805dfc0 con 0x7f475015ee80 2024-09-10T23:34:10.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.083+0000 7f4757400640 1 -- 172.21.15.94:0/4221988578 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f472c0727c0 msgr2=0x7f472c074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:10.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.083+0000 7f4757400640 1 --2- 172.21.15.94:0/4221988578 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f472c0727c0 0x7f472c074c80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f47400af000 tx=0x7f474001b120 comp rx=0 tx=0).stop 2024-09-10T23:34:10.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.083+0000 7f4757400640 1 -- 172.21.15.94:0/4221988578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f475015ee80 msgr2=0x7f4750159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:10.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.083+0000 7f4757400640 1 --2- 172.21.15.94:0/4221988578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f475015ee80 0x7f4750159ca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f474800c710 tx=0x7f474800cbd0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.083+0000 7f4757400640 1 -- 172.21.15.94:0/4221988578 shutdown_connections 2024-09-10T23:34:10.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.083+0000 7f4757400640 1 --2- 172.21.15.94:0/4221988578 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f472c0727c0 0x7f472c074c80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.083+0000 7f4757400640 1 --2- 172.21.15.94:0/4221988578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f475015ee80 0x7f4750159ca0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.083+0000 7f4757400640 1 --2- 172.21.15.94:0/4221988578 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47500d02e0 0x7f4750158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.083+0000 7f4757400640 1 --2- 172.21.15.94:0/4221988578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4750073ff0 0x7f47501516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.083+0000 7f4757400640 1 -- 172.21.15.94:0/4221988578 >> 172.21.15.94:0/4221988578 conn(0x7f475006cdf0 msgr2=0x7f47500b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:10.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.083+0000 7f4757400640 1 -- 172.21.15.94:0/4221988578 shutdown_connections 2024-09-10T23:34:10.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.083+0000 7f4757400640 1 -- 172.21.15.94:0/4221988578 wait complete. 2024-09-10T23:34:10.109 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T23:34:10.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.247+0000 7f3ae2e00640 1 -- 172.21.15.94:0/214329390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3adc15c0c0 msgr2=0x7f3adc15ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:10.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.247+0000 7f3ae2e00640 1 --2- 172.21.15.94:0/214329390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3adc15c0c0 0x7f3adc15ff80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4009840 tx=0x7f3ac401c820 comp rx=0 tx=0).stop 2024-09-10T23:34:10.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.247+0000 7f3ae2e00640 1 -- 172.21.15.94:0/214329390 shutdown_connections 2024-09-10T23:34:10.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.247+0000 7f3ae2e00640 1 --2- 172.21.15.94:0/214329390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3adc15c0c0 0x7f3adc15ff80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.247+0000 7f3ae2e00640 1 --2- 172.21.15.94:0/214329390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3adc15fba0 0x7f3adc06ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.247+0000 7f3ae2e00640 1 -- 172.21.15.94:0/214329390 >> 172.21.15.94:0/214329390 conn(0x7f3adc0cff40 msgr2=0x7f3adc0d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:10.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.247+0000 7f3ae2e00640 1 -- 172.21.15.94:0/214329390 shutdown_connections 2024-09-10T23:34:10.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.247+0000 7f3ae2e00640 1 -- 172.21.15.94:0/214329390 wait complete. 2024-09-10T23:34:10.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ae2e00640 1 Processor -- start 2024-09-10T23:34:10.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ae2e00640 1 -- start start 2024-09-10T23:34:10.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ae2e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3adc0740c0 0x7f3adc15e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:10.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ae2e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3adc15c0c0 0x7f3adc15eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:10.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ae2e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3adc15fba0 0x7f3adc137ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:10.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ae2e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc169440 con 0x7f3adc15c0c0 2024-09-10T23:34:10.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ae2e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc1692c0 con 0x7f3adc15fba0 2024-09-10T23:34:10.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ae2e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3adc1695c0 con 0x7f3adc0740c0 2024-09-10T23:34:10.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ad3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3adc15c0c0 0x7f3adc15eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:10.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ad3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3adc15c0c0 0x7f3adc15eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34070/0 (socket says 172.21.15.94:34070) 2024-09-10T23:34:10.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ad3400640 1 -- 172.21.15.94:0/3264172934 learned_addr learned my addr 172.21.15.94:0/3264172934 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:10.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3adbe00640 1 --2- 172.21.15.94:0/3264172934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3adc0740c0 0x7f3adc15e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:10.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ae0e00640 1 --2- 172.21.15.94:0/3264172934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3adc15fba0 0x7f3adc137ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:10.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ad3400640 1 -- 172.21.15.94:0/3264172934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3adc0740c0 msgr2=0x7f3adc15e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:10.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ad3400640 1 --2- 172.21.15.94:0/3264172934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3adc0740c0 0x7f3adc15e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ad3400640 1 -- 172.21.15.94:0/3264172934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3adc15fba0 msgr2=0x7f3adc137ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:10.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ad3400640 1 --2- 172.21.15.94:0/3264172934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3adc15fba0 0x7f3adc137ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ad3400640 1 -- 172.21.15.94:0/3264172934 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3adc0ccad0 con 0x7f3adc15c0c0 2024-09-10T23:34:10.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ad3400640 1 --2- 172.21.15.94:0/3264172934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3adc15c0c0 0x7f3adc15eb20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4000c10 tx=0x7f3ac4005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:10.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ad8c00640 1 -- 172.21.15.94:0/3264172934 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac40b2040 con 0x7f3adc15c0c0 2024-09-10T23:34:10.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ad8c00640 1 -- 172.21.15.94:0/3264172934 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ac4004330 con 0x7f3adc15c0c0 2024-09-10T23:34:10.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ad8c00640 1 -- 172.21.15.94:0/3264172934 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4005220 con 0x7f3adc15c0c0 2024-09-10T23:34:10.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ae2e00640 1 -- 172.21.15.94:0/3264172934 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3adc0ccdd0 con 0x7f3adc15c0c0 2024-09-10T23:34:10.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ae2e00640 1 -- 172.21.15.94:0/3264172934 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3adc09d2b0 con 0x7f3adc15c0c0 2024-09-10T23:34:10.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.251+0000 7f3ae2e00640 1 -- 172.21.15.94:0/3264172934 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3adc15ff80 con 0x7f3adc15c0c0 2024-09-10T23:34:10.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.255+0000 7f3ad8c00640 1 -- 172.21.15.94:0/3264172934 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f3ac40b5070 con 0x7f3adc15c0c0 2024-09-10T23:34:10.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.255+0000 7f3ad8c00640 1 --2- 172.21.15.94:0/3264172934 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3ab8072790 0x7f3ab8074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:10.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.255+0000 7f3ad8c00640 1 -- 172.21.15.94:0/3264172934 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f3ac412c700 con 0x7f3adc15c0c0 2024-09-10T23:34:10.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.255+0000 7f3adbe00640 1 --2- 172.21.15.94:0/3264172934 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3ab8072790 0x7f3ab8074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:10.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.255+0000 7f3adbe00640 1 --2- 172.21.15.94:0/3264172934 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3ab8072790 0x7f3ab8074c50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3acc005c50 tx=0x7f3acc005be0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:10.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.259+0000 7f3ad8c00640 1 -- 172.21.15.94:0/3264172934 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3adc0ccdd0 con 0x7f3adc15c0c0 2024-09-10T23:34:10.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.447+0000 7f3ae2e00640 1 -- 172.21.15.94:0/3264172934 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f3adc070e40 con 0x7f3ab8072790 2024-09-10T23:34:10.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.447+0000 7f3ad8c00640 1 -- 172.21.15.94:0/3264172934 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f3adc070e40 con 0x7f3ab8072790 2024-09-10T23:34:10.453 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:34:10.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:34:10.453 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephfs" 2024-09-10T23:34:10.453 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:34:10.453 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:34:10.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.451+0000 7f3ae2e00640 1 -- 172.21.15.94:0/3264172934 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3ab8072790 msgr2=0x7f3ab8074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:10.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.451+0000 7f3ae2e00640 1 --2- 172.21.15.94:0/3264172934 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3ab8072790 0x7f3ab8074c50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3acc005c50 tx=0x7f3acc005be0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.451+0000 7f3ae2e00640 1 -- 172.21.15.94:0/3264172934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3adc15c0c0 msgr2=0x7f3adc15eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:10.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.451+0000 7f3ae2e00640 1 --2- 172.21.15.94:0/3264172934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3adc15c0c0 0x7f3adc15eb20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4000c10 tx=0x7f3ac4005db0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.451+0000 7f3ae2e00640 1 -- 172.21.15.94:0/3264172934 shutdown_connections 2024-09-10T23:34:10.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.451+0000 7f3ae2e00640 1 --2- 172.21.15.94:0/3264172934 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3ab8072790 0x7f3ab8074c50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.451+0000 7f3ae2e00640 1 --2- 172.21.15.94:0/3264172934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3adc15fba0 0x7f3adc137ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.451+0000 7f3ae2e00640 1 --2- 172.21.15.94:0/3264172934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3adc15c0c0 0x7f3adc15eb20 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.451+0000 7f3ae2e00640 1 --2- 172.21.15.94:0/3264172934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3adc0740c0 0x7f3adc15e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.451+0000 7f3ae2e00640 1 -- 172.21.15.94:0/3264172934 >> 172.21.15.94:0/3264172934 conn(0x7f3adc0cff40 msgr2=0x7f3adc14f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:10.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.451+0000 7f3ae2e00640 1 -- 172.21.15.94:0/3264172934 shutdown_connections 2024-09-10T23:34:10.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.451+0000 7f3ae2e00640 1 -- 172.21.15.94:0/3264172934 wait complete. 2024-09-10T23:34:10.475 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:34:10.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.615+0000 7f7e55000640 1 -- 172.21.15.94:0/2519183711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e50158790 msgr2=0x7f7e5006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.615+0000 7f7e55000640 1 --2- 172.21.15.94:0/2519183711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e50158790 0x7f7e5006fc00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7e50069640 tx=0x7f7e4001c8c0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.615+0000 7f7e55000640 1 -- 172.21.15.94:0/2519183711 shutdown_connections 2024-09-10T23:34:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.615+0000 7f7e55000640 1 --2- 172.21.15.94:0/2519183711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e500cb300 0x7f7e5015e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.615+0000 7f7e55000640 1 --2- 172.21.15.94:0/2519183711 >> v2:172.21.15.181:3300/0 conn(0x7f7e500c49b0 0x7f7e500cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.615+0000 7f7e55000640 1 --2- 172.21.15.94:0/2519183711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e50158790 0x7f7e5006fc00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.615+0000 7f7e55000640 1 -- 172.21.15.94:0/2519183711 >> 172.21.15.94:0/2519183711 conn(0x7f7e500cfaf0 msgr2=0x7f7e500cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.615+0000 7f7e55000640 1 -- 172.21.15.94:0/2519183711 shutdown_connections 2024-09-10T23:34:10.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.615+0000 7f7e55000640 1 -- 172.21.15.94:0/2519183711 wait complete. 2024-09-10T23:34:10.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.615+0000 7f7e55000640 1 Processor -- start 2024-09-10T23:34:10.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.615+0000 7f7e55000640 1 -- start start 2024-09-10T23:34:10.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e55000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e500c49b0 0x7f7e5013fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:10.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e55000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e500cb300 0x7f7e500c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:10.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e55000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e50158790 0x7f7e50142570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:10.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e55000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5015b8d0 con 0x7f7e500c49b0 2024-09-10T23:34:10.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e55000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5015b750 con 0x7f7e500cb300 2024-09-10T23:34:10.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e55000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5015ba50 con 0x7f7e50158790 2024-09-10T23:34:10.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e4de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e500c49b0 0x7f7e5013fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:10.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e4de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e500c49b0 0x7f7e5013fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34088/0 (socket says 172.21.15.94:34088) 2024-09-10T23:34:10.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e4de00640 1 -- 172.21.15.94:0/86242860 learned_addr learned my addr 172.21.15.94:0/86242860 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:10.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e4e800640 1 --2- 172.21.15.94:0/86242860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e50158790 0x7f7e50142570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:10.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e45400640 1 --2- 172.21.15.94:0/86242860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e500cb300 0x7f7e500c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:10.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e4de00640 1 -- 172.21.15.94:0/86242860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e50158790 msgr2=0x7f7e50142570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:10.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e4de00640 1 --2- 172.21.15.94:0/86242860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e50158790 0x7f7e50142570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e4de00640 1 -- 172.21.15.94:0/86242860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e500cb300 msgr2=0x7f7e500c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:10.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e4de00640 1 --2- 172.21.15.94:0/86242860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e500cb300 0x7f7e500c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e4de00640 1 -- 172.21.15.94:0/86242860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e500c9070 con 0x7f7e500c49b0 2024-09-10T23:34:10.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e4de00640 1 --2- 172.21.15.94:0/86242860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e500c49b0 0x7f7e5013fe90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7e400b0040 tx=0x7f7e40005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:10.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e46a00640 1 -- 172.21.15.94:0/86242860 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e400b2040 con 0x7f7e500c49b0 2024-09-10T23:34:10.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e46a00640 1 -- 172.21.15.94:0/86242860 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e40004330 con 0x7f7e500c49b0 2024-09-10T23:34:10.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e46a00640 1 -- 172.21.15.94:0/86242860 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40005290 con 0x7f7e500c49b0 2024-09-10T23:34:10.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e55000640 1 -- 172.21.15.94:0/86242860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e500c92f0 con 0x7f7e500c49b0 2024-09-10T23:34:10.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e55000640 1 -- 172.21.15.94:0/86242860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e500c9870 con 0x7f7e500c49b0 2024-09-10T23:34:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e55000640 1 -- 172.21.15.94:0/86242860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e5015bf60 con 0x7f7e500c49b0 2024-09-10T23:34:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.619+0000 7f7e46a00640 1 -- 172.21.15.94:0/86242860 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f7e40005430 con 0x7f7e500c49b0 2024-09-10T23:34:10.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.623+0000 7f7e46a00640 1 --2- 172.21.15.94:0/86242860 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f7e2c072790 0x7f7e2c074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:10.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.623+0000 7f7e46a00640 1 -- 172.21.15.94:0/86242860 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f7e4012d120 con 0x7f7e500c49b0 2024-09-10T23:34:10.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.623+0000 7f7e45400640 1 --2- 172.21.15.94:0/86242860 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f7e2c072790 0x7f7e2c074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:10.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.623+0000 7f7e45400640 1 --2- 172.21.15.94:0/86242860 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f7e2c072790 0x7f7e2c074c50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7e38005e10 tx=0x7f7e38005c60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:10.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.627+0000 7f7e46a00640 1 -- 172.21.15.94:0/86242860 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7e500c9070 con 0x7f7e500c49b0 2024-09-10T23:34:10.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.791+0000 7f7e55000640 1 -- 172.21.15.94:0/86242860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f7e5015eaa0 con 0x7f7e500c49b0 2024-09-10T23:34:10.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.791+0000 7f7e46a00640 1 -- 172.21.15.94:0/86242860 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f7e400164b0 con 0x7f7e500c49b0 2024-09-10T23:34:10.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.791+0000 7f7e55000640 1 -- 172.21.15.94:0/86242860 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f7e2c072790 msgr2=0x7f7e2c074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:10.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.791+0000 7f7e55000640 1 --2- 172.21.15.94:0/86242860 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f7e2c072790 0x7f7e2c074c50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7e38005e10 tx=0x7f7e38005c60 comp rx=0 tx=0).stop 2024-09-10T23:34:10.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.791+0000 7f7e55000640 1 -- 172.21.15.94:0/86242860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e500c49b0 msgr2=0x7f7e5013fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:10.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.791+0000 7f7e55000640 1 --2- 172.21.15.94:0/86242860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e500c49b0 0x7f7e5013fe90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7e400b0040 tx=0x7f7e40005de0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.791+0000 7f7e55000640 1 -- 172.21.15.94:0/86242860 shutdown_connections 2024-09-10T23:34:10.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.791+0000 7f7e55000640 1 --2- 172.21.15.94:0/86242860 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f7e2c072790 0x7f7e2c074c50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.791+0000 7f7e55000640 1 --2- 172.21.15.94:0/86242860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e500cb300 0x7f7e500c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.791+0000 7f7e55000640 1 --2- 172.21.15.94:0/86242860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e50158790 0x7f7e50142570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.791+0000 7f7e55000640 1 --2- 172.21.15.94:0/86242860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e500c49b0 0x7f7e5013fe90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:10.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.791+0000 7f7e55000640 1 -- 172.21.15.94:0/86242860 >> 172.21.15.94:0/86242860 conn(0x7f7e500cfaf0 msgr2=0x7f7e500b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:10.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.791+0000 7f7e55000640 1 -- 172.21.15.94:0/86242860 shutdown_connections 2024-09-10T23:34:10.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:10.791+0000 7f7e55000640 1 -- 172.21.15.94:0/86242860 wait complete. 2024-09-10T23:34:10.818 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:34:10.818 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:34:10.827 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:34:11.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.011+0000 7f621f600640 1 -- 172.21.15.94:0/2462297411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62180ce890 msgr2=0x7f62180cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:11.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.011+0000 7f621f600640 1 --2- 172.21.15.94:0/2462297411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62180ce890 0x7f62180cd710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f620c009870 tx=0x7f620c01c7d0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.011+0000 7f621f600640 1 -- 172.21.15.94:0/2462297411 shutdown_connections 2024-09-10T23:34:11.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.011+0000 7f621f600640 1 --2- 172.21.15.94:0/2462297411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62180ce890 0x7f62180cd710 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.011+0000 7f621f600640 1 -- 172.21.15.94:0/2462297411 >> 172.21.15.94:0/2462297411 conn(0x7f62180d1c60 msgr2=0x7f62180d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:11.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.011+0000 7f621f600640 1 -- 172.21.15.94:0/2462297411 shutdown_connections 2024-09-10T23:34:11.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.011+0000 7f621f600640 1 -- 172.21.15.94:0/2462297411 wait complete. 2024-09-10T23:34:11.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621f600640 1 Processor -- start 2024-09-10T23:34:11.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621f600640 1 -- start start 2024-09-10T23:34:11.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621f600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180b1100 0x7f621815b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:11.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621f600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62180ce1e0 0x7f621806ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:11.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621f600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62180ce890 0x7f621806f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:11.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621f600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f621816a260 con 0x7f62180b1100 2024-09-10T23:34:11.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621f600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f621816a0e0 con 0x7f62180ce890 2024-09-10T23:34:11.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621f600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f621816a3e0 con 0x7f62180ce1e0 2024-09-10T23:34:11.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180b1100 0x7f621815b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:11.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180b1100 0x7f621815b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34092/0 (socket says 172.21.15.94:34092) 2024-09-10T23:34:11.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621cc00640 1 -- 172.21.15.94:0/2196012344 learned_addr learned my addr 172.21.15.94:0/2196012344 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:11.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621cc00640 1 -- 172.21.15.94:0/2196012344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62180ce1e0 msgr2=0x7f621806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:11.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621cc00640 1 --2- 172.21.15.94:0/2196012344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62180ce1e0 0x7f621806ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621cc00640 1 -- 172.21.15.94:0/2196012344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62180ce890 msgr2=0x7f621806f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:11.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621cc00640 1 --2- 172.21.15.94:0/2196012344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62180ce890 0x7f621806f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621cc00640 1 -- 172.21.15.94:0/2196012344 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621815ba70 con 0x7f62180b1100 2024-09-10T23:34:11.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621cc00640 1 --2- 172.21.15.94:0/2196012344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180b1100 0x7f621815b530 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f620c0193e0 tx=0x7f620c0a6530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:11.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f6214c00640 1 -- 172.21.15.94:0/2196012344 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c005250 con 0x7f62180b1100 2024-09-10T23:34:11.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f6214c00640 1 -- 172.21.15.94:0/2196012344 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f620c0053f0 con 0x7f62180b1100 2024-09-10T23:34:11.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f6214c00640 1 -- 172.21.15.94:0/2196012344 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c0046d0 con 0x7f62180b1100 2024-09-10T23:34:11.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621f600640 1 -- 172.21.15.94:0/2196012344 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62180b0070 con 0x7f62180b1100 2024-09-10T23:34:11.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621f600640 1 -- 172.21.15.94:0/2196012344 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62180ae910 con 0x7f62180b1100 2024-09-10T23:34:11.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.015+0000 7f621f600640 1 -- 172.21.15.94:0/2196012344 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62180cd710 con 0x7f62180b1100 2024-09-10T23:34:11.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.019+0000 7f6214c00640 1 -- 172.21.15.94:0/2196012344 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f620c0a6720 con 0x7f62180b1100 2024-09-10T23:34:11.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.019+0000 7f6214c00640 1 --2- 172.21.15.94:0/2196012344 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f61fc072790 0x7f61fc074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:11.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.019+0000 7f6214c00640 1 -- 172.21.15.94:0/2196012344 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f620c12ddd0 con 0x7f62180b1100 2024-09-10T23:34:11.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.019+0000 7f6217e00640 1 --2- 172.21.15.94:0/2196012344 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f61fc072790 0x7f61fc074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:11.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.019+0000 7f6217e00640 1 --2- 172.21.15.94:0/2196012344 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f61fc072790 0x7f61fc074c50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6208007fd0 tx=0x7f6208007900 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:11.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.023+0000 7f6214c00640 1 -- 172.21.15.94:0/2196012344 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f620c0bf060 con 0x7f62180b1100 2024-09-10T23:34:11.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.203+0000 7f621f600640 1 -- 172.21.15.94:0/2196012344 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f62180ceec0 con 0x7f62180b1100 2024-09-10T23:34:11.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.203+0000 7f6214c00640 1 -- 172.21.15.94:0/2196012344 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v44) ==== 80+0+0 (secure 0 0 0) 0x7f621815ba70 con 0x7f62180b1100 2024-09-10T23:34:11.209 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:34:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.207+0000 7f621f600640 1 -- 172.21.15.94:0/2196012344 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f61fc072790 msgr2=0x7f61fc074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.207+0000 7f621f600640 1 --2- 172.21.15.94:0/2196012344 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f61fc072790 0x7f61fc074c50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6208007fd0 tx=0x7f6208007900 comp rx=0 tx=0).stop 2024-09-10T23:34:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.207+0000 7f621f600640 1 -- 172.21.15.94:0/2196012344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180b1100 msgr2=0x7f621815b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.207+0000 7f621f600640 1 --2- 172.21.15.94:0/2196012344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180b1100 0x7f621815b530 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f620c0193e0 tx=0x7f620c0a6530 comp rx=0 tx=0).stop 2024-09-10T23:34:11.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.207+0000 7f621f600640 1 -- 172.21.15.94:0/2196012344 shutdown_connections 2024-09-10T23:34:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.207+0000 7f621f600640 1 --2- 172.21.15.94:0/2196012344 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f61fc072790 0x7f61fc074c50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.207+0000 7f621f600640 1 --2- 172.21.15.94:0/2196012344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62180ce890 0x7f621806f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.207+0000 7f621f600640 1 --2- 172.21.15.94:0/2196012344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62180ce1e0 0x7f621806ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.207+0000 7f621f600640 1 --2- 172.21.15.94:0/2196012344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62180b1100 0x7f621815b530 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.207+0000 7f621f600640 1 -- 172.21.15.94:0/2196012344 >> 172.21.15.94:0/2196012344 conn(0x7f62180d1c60 msgr2=0x7f6218071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.207+0000 7f621f600640 1 -- 172.21.15.94:0/2196012344 shutdown_connections 2024-09-10T23:34:11.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.207+0000 7f621f600640 1 -- 172.21.15.94:0/2196012344 wait complete. 2024-09-10T23:34:11.230 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:34:11.230 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:34:11.239 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:34:11.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 -- 172.21.15.94:0/1502896441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f015df80 msgr2=0x7fb9f00d49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:11.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 --2- 172.21.15.94:0/1502896441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f015df80 0x7fb9f00d49c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb9f008e620 tx=0x7fb9e401c890 comp rx=0 tx=0).stop 2024-09-10T23:34:11.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 -- 172.21.15.94:0/1502896441 shutdown_connections 2024-09-10T23:34:11.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 --2- 172.21.15.94:0/1502896441 >> v2:172.21.15.181:3300/0 conn(0x7fb9f00d0cd0 0x7fb9f00d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 --2- 172.21.15.94:0/1502896441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9f00b3200 0x7fb9f00d0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 --2- 172.21.15.94:0/1502896441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f015df80 0x7fb9f00d49c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 -- 172.21.15.94:0/1502896441 >> 172.21.15.94:0/1502896441 conn(0x7fb9f00d3ea0 msgr2=0x7fb9f00d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:11.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 -- 172.21.15.94:0/1502896441 shutdown_connections 2024-09-10T23:34:11.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 -- 172.21.15.94:0/1502896441 wait complete. 2024-09-10T23:34:11.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 Processor -- start 2024-09-10T23:34:11.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 -- start start 2024-09-10T23:34:11.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9f00b3200 0x7fb9f0155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:11.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f00d0cd0 0x7fb9f00c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9f015df80 0x7fb9f015f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f00b15f0 con 0x7fb9f00d0cd0 2024-09-10T23:34:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f00b1470 con 0x7fb9f015df80 2024-09-10T23:34:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f7600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb9f00b1770 con 0x7fb9f00b3200 2024-09-10T23:34:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f00d0cd0 0x7fb9f00c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f00d0cd0 0x7fb9f00c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34126/0 (socket says 172.21.15.94:34126) 2024-09-10T23:34:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9efe00640 1 -- 172.21.15.94:0/2707456719 learned_addr learned my addr 172.21.15.94:0/2707456719 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.423+0000 7fb9f4c00640 1 --2- 172.21.15.94:0/2707456719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9f00b3200 0x7fb9f0155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.427+0000 7fb9f5600640 1 --2- 172.21.15.94:0/2707456719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9f015df80 0x7fb9f015f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.427+0000 7fb9efe00640 1 -- 172.21.15.94:0/2707456719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9f00b3200 msgr2=0x7fb9f0155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.427+0000 7fb9efe00640 1 --2- 172.21.15.94:0/2707456719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9f00b3200 0x7fb9f0155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.427+0000 7fb9efe00640 1 -- 172.21.15.94:0/2707456719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9f015df80 msgr2=0x7fb9f015f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:11.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.427+0000 7fb9efe00640 1 --2- 172.21.15.94:0/2707456719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9f015df80 0x7fb9f015f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.427+0000 7fb9efe00640 1 -- 172.21.15.94:0/2707456719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f0161170 con 0x7fb9f00d0cd0 2024-09-10T23:34:11.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.427+0000 7fb9efe00640 1 --2- 172.21.15.94:0/2707456719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f00d0cd0 0x7fb9f00c8150 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00d980 tx=0x7fb9dc00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:11.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.427+0000 7fb9ecc00640 1 -- 172.21.15.94:0/2707456719 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc017070 con 0x7fb9f00d0cd0 2024-09-10T23:34:11.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.427+0000 7fb9ecc00640 1 -- 172.21.15.94:0/2707456719 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9dc009d20 con 0x7fb9f00d0cd0 2024-09-10T23:34:11.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.427+0000 7fb9ecc00640 1 -- 172.21.15.94:0/2707456719 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc0106c0 con 0x7fb9f00d0cd0 2024-09-10T23:34:11.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.427+0000 7fb9f7600640 1 -- 172.21.15.94:0/2707456719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f0160bb0 con 0x7fb9f00d0cd0 2024-09-10T23:34:11.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.427+0000 7fb9f7600640 1 -- 172.21.15.94:0/2707456719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f0156310 con 0x7fb9f00d0cd0 2024-09-10T23:34:11.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.427+0000 7fb9f7600640 1 -- 172.21.15.94:0/2707456719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9f00b1c90 con 0x7fb9f00d0cd0 2024-09-10T23:34:11.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.427+0000 7fb9ecc00640 1 -- 172.21.15.94:0/2707456719 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7fb9dc010860 con 0x7fb9f00d0cd0 2024-09-10T23:34:11.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.427+0000 7fb9ecc00640 1 --2- 172.21.15.94:0/2707456719 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb9d0072790 0x7fb9d0074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:11.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.431+0000 7fb9ecc00640 1 -- 172.21.15.94:0/2707456719 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fb9dc093a30 con 0x7fb9f00d0cd0 2024-09-10T23:34:11.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.431+0000 7fb9f4c00640 1 --2- 172.21.15.94:0/2707456719 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb9d0072790 0x7fb9d0074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:11.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.431+0000 7fb9f4c00640 1 --2- 172.21.15.94:0/2707456719 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb9d0072790 0x7fb9d0074c50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40af000 tx=0x7fb9e40a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:11.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.431+0000 7fb9ecc00640 1 -- 172.21.15.94:0/2707456719 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb9f0161170 con 0x7fb9f00d0cd0 2024-09-10T23:34:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.611+0000 7fb9f7600640 1 -- 172.21.15.94:0/2707456719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb9f015c280 con 0x7fb9f00d0cd0 2024-09-10T23:34:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.611+0000 7fb9ecc00640 1 -- 172.21.15.94:0/2707456719 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v44) ==== 80+0+0 (secure 0 0 0) 0x7fb9dc05e7d0 con 0x7fb9f00d0cd0 2024-09-10T23:34:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:34:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.611+0000 7fb9f7600640 1 -- 172.21.15.94:0/2707456719 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb9d0072790 msgr2=0x7fb9d0074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.611+0000 7fb9f7600640 1 --2- 172.21.15.94:0/2707456719 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb9d0072790 0x7fb9d0074c50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40af000 tx=0x7fb9e40a7040 comp rx=0 tx=0).stop 2024-09-10T23:34:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.611+0000 7fb9f7600640 1 -- 172.21.15.94:0/2707456719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f00d0cd0 msgr2=0x7fb9f00c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.611+0000 7fb9f7600640 1 --2- 172.21.15.94:0/2707456719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f00d0cd0 0x7fb9f00c8150 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00d980 tx=0x7fb9dc00de40 comp rx=0 tx=0).stop 2024-09-10T23:34:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.611+0000 7fb9f7600640 1 -- 172.21.15.94:0/2707456719 shutdown_connections 2024-09-10T23:34:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.611+0000 7fb9f7600640 1 --2- 172.21.15.94:0/2707456719 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb9d0072790 0x7fb9d0074c50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.611+0000 7fb9f7600640 1 --2- 172.21.15.94:0/2707456719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9f015df80 0x7fb9f015f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.611+0000 7fb9f7600640 1 --2- 172.21.15.94:0/2707456719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f00d0cd0 0x7fb9f00c8150 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.611+0000 7fb9f7600640 1 --2- 172.21.15.94:0/2707456719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9f00b3200 0x7fb9f0155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.611+0000 7fb9f7600640 1 -- 172.21.15.94:0/2707456719 >> 172.21.15.94:0/2707456719 conn(0x7fb9f00d3ea0 msgr2=0x7fb9f00b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.611+0000 7fb9f7600640 1 -- 172.21.15.94:0/2707456719 shutdown_connections 2024-09-10T23:34:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.611+0000 7fb9f7600640 1 -- 172.21.15.94:0/2707456719 wait complete. 2024-09-10T23:34:11.637 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-09-10T23:34:11.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:11.638 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T23:34:11.654 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[38541]: fuse finished with error 0 and tester_r 0 2024-09-10T23:34:11.658 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:34:11.673 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:34:11.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:11.674 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T23:34:11.723 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:34:11.723 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:34:11.732 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:34:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.903+0000 7f24db000640 1 -- 172.21.15.94:0/3975762681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24d415f180 msgr2=0x7f24d40cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.903+0000 7f24db000640 1 --2- 172.21.15.94:0/3975762681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24d415f180 0x7f24d40cd0a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f24d40d1c80 tx=0x7f24c401c800 comp rx=0 tx=0).stop 2024-09-10T23:34:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.903+0000 7f24db000640 1 -- 172.21.15.94:0/3975762681 shutdown_connections 2024-09-10T23:34:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.903+0000 7f24db000640 1 --2- 172.21.15.94:0/3975762681 >> v2:172.21.15.181:3300/0 conn(0x7f24d415f560 0x7f24d40cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.903+0000 7f24db000640 1 --2- 172.21.15.94:0/3975762681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24d415f180 0x7f24d40cd0a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.903+0000 7f24db000640 1 -- 172.21.15.94:0/3975762681 >> 172.21.15.94:0/3975762681 conn(0x7f24d40d1cb0 msgr2=0x7f24d40d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.903+0000 7f24db000640 1 -- 172.21.15.94:0/3975762681 shutdown_connections 2024-09-10T23:34:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.903+0000 7f24db000640 1 -- 172.21.15.94:0/3975762681 wait complete. 2024-09-10T23:34:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.903+0000 7f24db000640 1 Processor -- start 2024-09-10T23:34:11.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.903+0000 7f24db000640 1 -- start start 2024-09-10T23:34:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.903+0000 7f24db000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24d40cffd0 0x7f24d414d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.903+0000 7f24db000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24d415f180 0x7f24d4157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:11.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.903+0000 7f24db000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24d415f560 0x7f24d4157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:11.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.903+0000 7f24db000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41692a0 con 0x7f24d415f560 2024-09-10T23:34:11.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.903+0000 7f24db000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4169120 con 0x7f24d415f180 2024-09-10T23:34:11.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.903+0000 7f24db000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f24d4169420 con 0x7f24d40cffd0 2024-09-10T23:34:11.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24d9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24d415f560 0x7f24d4157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:11.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24d9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24d415f560 0x7f24d4157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34134/0 (socket says 172.21.15.94:34134) 2024-09-10T23:34:11.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24d9000640 1 -- 172.21.15.94:0/1371154587 learned_addr learned my addr 172.21.15.94:0/1371154587 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:11.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24d3e00640 1 --2- 172.21.15.94:0/1371154587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24d40cffd0 0x7f24d414d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:11.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24cbe00640 1 --2- 172.21.15.94:0/1371154587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24d415f180 0x7f24d4157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:11.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24d9000640 1 -- 172.21.15.94:0/1371154587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24d40cffd0 msgr2=0x7f24d414d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:11.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24d9000640 1 --2- 172.21.15.94:0/1371154587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24d40cffd0 0x7f24d414d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24d9000640 1 -- 172.21.15.94:0/1371154587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24d415f180 msgr2=0x7f24d4157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:11.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24d9000640 1 --2- 172.21.15.94:0/1371154587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24d415f180 0x7f24d4157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:11.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24d9000640 1 -- 172.21.15.94:0/1371154587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d414e5d0 con 0x7f24d415f560 2024-09-10T23:34:11.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24d9000640 1 --2- 172.21.15.94:0/1371154587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24d415f560 0x7f24d4157b40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00cbc0 tx=0x7f24cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:11.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24d0c00640 1 -- 172.21.15.94:0/1371154587 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc016020 con 0x7f24d415f560 2024-09-10T23:34:11.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24d0c00640 1 -- 172.21.15.94:0/1371154587 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24cc00ab40 con 0x7f24d415f560 2024-09-10T23:34:11.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24d0c00640 1 -- 172.21.15.94:0/1371154587 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc012420 con 0x7f24d415f560 2024-09-10T23:34:11.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24db000640 1 -- 172.21.15.94:0/1371154587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d414e870 con 0x7f24d415f560 2024-09-10T23:34:11.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24db000640 1 -- 172.21.15.94:0/1371154587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d414ee30 con 0x7f24d415f560 2024-09-10T23:34:11.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24db000640 1 -- 172.21.15.94:0/1371154587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d40cd0a0 con 0x7f24d415f560 2024-09-10T23:34:11.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.907+0000 7f24d0c00640 1 -- 172.21.15.94:0/1371154587 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f24cc01a3b0 con 0x7f24d415f560 2024-09-10T23:34:11.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.911+0000 7f24d0c00640 1 --2- 172.21.15.94:0/1371154587 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24b0072790 0x7f24b0074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:11.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.911+0000 7f24d0c00640 1 -- 172.21.15.94:0/1371154587 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f24cc093790 con 0x7f24d415f560 2024-09-10T23:34:11.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.911+0000 7f24d3e00640 1 --2- 172.21.15.94:0/1371154587 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24b0072790 0x7f24b0074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:11.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.911+0000 7f24d3e00640 1 --2- 172.21.15.94:0/1371154587 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24b0072790 0x7f24b0074c50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f24c40af000 tx=0x7f24c401b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:11.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:11.915+0000 7f24d0c00640 1 -- 172.21.15.94:0/1371154587 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f24cc020030 con 0x7f24d415f560 2024-09-10T23:34:12.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.091+0000 7f24db000640 1 -- 172.21.15.94:0/1371154587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f24d415c310 con 0x7f24d415f560 2024-09-10T23:34:12.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.091+0000 7f24d0c00640 1 -- 172.21.15.94:0/1371154587 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v44) ==== 80+0+0 (secure 0 0 0) 0x7f24cc0671f0 con 0x7f24d415f560 2024-09-10T23:34:12.096 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:34:12.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.091+0000 7f24db000640 1 -- 172.21.15.94:0/1371154587 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24b0072790 msgr2=0x7f24b0074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:12.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.091+0000 7f24db000640 1 --2- 172.21.15.94:0/1371154587 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24b0072790 0x7f24b0074c50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f24c40af000 tx=0x7f24c401b040 comp rx=0 tx=0).stop 2024-09-10T23:34:12.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.091+0000 7f24db000640 1 -- 172.21.15.94:0/1371154587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24d415f560 msgr2=0x7f24d4157b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:12.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.091+0000 7f24db000640 1 --2- 172.21.15.94:0/1371154587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24d415f560 0x7f24d4157b40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00cbc0 tx=0x7f24cc00ccb0 comp rx=0 tx=0).stop 2024-09-10T23:34:12.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.095+0000 7f24db000640 1 -- 172.21.15.94:0/1371154587 shutdown_connections 2024-09-10T23:34:12.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.095+0000 7f24db000640 1 --2- 172.21.15.94:0/1371154587 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f24b0072790 0x7f24b0074c50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:12.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.095+0000 7f24db000640 1 --2- 172.21.15.94:0/1371154587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24d415f180 0x7f24d4157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:12.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.095+0000 7f24db000640 1 --2- 172.21.15.94:0/1371154587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24d415f560 0x7f24d4157b40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:12.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.095+0000 7f24db000640 1 --2- 172.21.15.94:0/1371154587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24d40cffd0 0x7f24d414d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:12.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.095+0000 7f24db000640 1 -- 172.21.15.94:0/1371154587 >> 172.21.15.94:0/1371154587 conn(0x7f24d40d1cb0 msgr2=0x7f24d40b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:12.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.095+0000 7f24db000640 1 -- 172.21.15.94:0/1371154587 shutdown_connections 2024-09-10T23:34:12.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.095+0000 7f24db000640 1 -- 172.21.15.94:0/1371154587 wait complete. 2024-09-10T23:34:12.118 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:34:12.118 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:34:12.128 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:34:12.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d56e00640 1 -- 172.21.15.94:0/1229639470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d34004d50 msgr2=0x7f2d340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:12.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d56e00640 1 --2- 172.21.15.94:0/1229639470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d34004d50 0x7f2d340051d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009b30 tx=0x7f2d4801d230 comp rx=0 tx=0).stop 2024-09-10T23:34:12.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d54e00640 1 -- 172.21.15.94:0/1229639470 reap_dead start 2024-09-10T23:34:12.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d56e00640 1 -- 172.21.15.94:0/1229639470 shutdown_connections 2024-09-10T23:34:12.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d56e00640 1 -- 172.21.15.94:0/1229639470 >> 172.21.15.94:0/1229639470 conn(0x7f2d500cfaf0 msgr2=0x7f2d500cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:12.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d56e00640 1 -- 172.21.15.94:0/1229639470 shutdown_connections 2024-09-10T23:34:12.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d56e00640 1 -- 172.21.15.94:0/1229639470 wait complete. 2024-09-10T23:34:12.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d56e00640 1 Processor -- start 2024-09-10T23:34:12.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d56e00640 1 -- start start 2024-09-10T23:34:12.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d56e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d34004d50 0x7f2d500c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:12.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d56e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d500af130 0x7f2d500b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:12.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d56e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d5014dbe0 0x7f2d500b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:12.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d56e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d50163c80 con 0x7f2d5014dbe0 2024-09-10T23:34:12.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d56e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2d50163b00 con 0x7f2d500af130 2024-09-10T23:34:12.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d56e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2d50163e00 con 0x7f2d34004d50 2024-09-10T23:34:12.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d54e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d5014dbe0 0x7f2d500b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:12.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d54e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d5014dbe0 0x7f2d500b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34152/0 (socket says 172.21.15.94:34152) 2024-09-10T23:34:12.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d54e00640 1 -- 172.21.15.94:0/2914192591 learned_addr learned my addr 172.21.15.94:0/2914192591 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:12.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d4fe00640 1 --2- 172.21.15.94:0/2914192591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d34004d50 0x7f2d500c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:12.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d54e00640 1 -- 172.21.15.94:0/2914192591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d34004d50 msgr2=0x7f2d500c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:12.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d54e00640 1 --2- 172.21.15.94:0/2914192591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d34004d50 0x7f2d500c55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:12.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d54e00640 1 -- 172.21.15.94:0/2914192591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d500af130 msgr2=0x7f2d500b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:12.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d54e00640 1 --2- 172.21.15.94:0/2914192591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d500af130 0x7f2d500b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:12.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.267+0000 7f2d54e00640 1 -- 172.21.15.94:0/2914192591 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d50074490 con 0x7f2d5014dbe0 2024-09-10T23:34:12.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.271+0000 7f2d54e00640 1 --2- 172.21.15.94:0/2914192591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d5014dbe0 0x7f2d500b2f80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009b30 tx=0x7f2d480add80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:12.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.271+0000 7f2d47e00640 1 -- 172.21.15.94:0/2914192591 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d480b1040 con 0x7f2d5014dbe0 2024-09-10T23:34:12.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.271+0000 7f2d47e00640 1 -- 172.21.15.94:0/2914192591 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d480bb070 con 0x7f2d5014dbe0 2024-09-10T23:34:12.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.271+0000 7f2d47e00640 1 -- 172.21.15.94:0/2914192591 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c001410 con 0x7f2d5014dbe0 2024-09-10T23:34:12.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.271+0000 7f2d56e00640 1 -- 172.21.15.94:0/2914192591 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d501f7ed0 con 0x7f2d5014dbe0 2024-09-10T23:34:12.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.271+0000 7f2d56e00640 1 -- 172.21.15.94:0/2914192591 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d501f8490 con 0x7f2d5014dbe0 2024-09-10T23:34:12.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.271+0000 7f2d56e00640 1 -- 172.21.15.94:0/2914192591 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d50069e20 con 0x7f2d5014dbe0 2024-09-10T23:34:12.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.271+0000 7f2d47e00640 1 -- 172.21.15.94:0/2914192591 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f2d48004030 con 0x7f2d5014dbe0 2024-09-10T23:34:12.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.271+0000 7f2d47e00640 1 --2- 172.21.15.94:0/2914192591 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2d30072790 0x7f2d30074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:12.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.271+0000 7f2d47e00640 1 -- 172.21.15.94:0/2914192591 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f2d4812dfa0 con 0x7f2d5014dbe0 2024-09-10T23:34:12.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.275+0000 7f2d4fe00640 1 --2- 172.21.15.94:0/2914192591 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2d30072790 0x7f2d30074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:12.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.275+0000 7f2d4fe00640 1 --2- 172.21.15.94:0/2914192591 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2d30072790 0x7f2d30074c50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2d40002240 tx=0x7f2d40006ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:12.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.279+0000 7f2d47e00640 1 -- 172.21.15.94:0/2914192591 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2d50069e20 con 0x7f2d5014dbe0 2024-09-10T23:34:12.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.451+0000 7f2d56e00640 1 -- 172.21.15.94:0/2914192591 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2d500d0850 con 0x7f2d5014dbe0 2024-09-10T23:34:12.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.451+0000 7f2d47e00640 1 -- 172.21.15.94:0/2914192591 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v44) ==== 80+0+0 (secure 0 0 0) 0x7f2d480c5020 con 0x7f2d5014dbe0 2024-09-10T23:34:12.458 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:34:12.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.455+0000 7f2d56e00640 1 -- 172.21.15.94:0/2914192591 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2d30072790 msgr2=0x7f2d30074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:12.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.455+0000 7f2d56e00640 1 --2- 172.21.15.94:0/2914192591 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2d30072790 0x7f2d30074c50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2d40002240 tx=0x7f2d40006ab0 comp rx=0 tx=0).stop 2024-09-10T23:34:12.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.455+0000 7f2d56e00640 1 -- 172.21.15.94:0/2914192591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d5014dbe0 msgr2=0x7f2d500b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:12.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.455+0000 7f2d56e00640 1 --2- 172.21.15.94:0/2914192591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d5014dbe0 0x7f2d500b2f80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009b30 tx=0x7f2d480add80 comp rx=0 tx=0).stop 2024-09-10T23:34:12.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.455+0000 7f2d56e00640 1 -- 172.21.15.94:0/2914192591 shutdown_connections 2024-09-10T23:34:12.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.455+0000 7f2d56e00640 1 --2- 172.21.15.94:0/2914192591 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2d30072790 0x7f2d30074c50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:12.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.455+0000 7f2d56e00640 1 --2- 172.21.15.94:0/2914192591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d500af130 0x7f2d500b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:12.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.455+0000 7f2d56e00640 1 --2- 172.21.15.94:0/2914192591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d5014dbe0 0x7f2d500b2f80 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:12.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.455+0000 7f2d56e00640 1 --2- 172.21.15.94:0/2914192591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d34004d50 0x7f2d500c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:12.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.455+0000 7f2d56e00640 1 -- 172.21.15.94:0/2914192591 >> 172.21.15.94:0/2914192591 conn(0x7f2d500cfaf0 msgr2=0x7f2d500ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:12.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.455+0000 7f2d56e00640 1 -- 172.21.15.94:0/2914192591 shutdown_connections 2024-09-10T23:34:12.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:12.455+0000 7f2d56e00640 1 -- 172.21.15.94:0/2914192591 wait complete. 2024-09-10T23:34:12.479 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi181.front.sepia.ceph.com... 2024-09-10T23:34:12.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:12.480 DEBUG:teuthology.orchestra.run.smithi181:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T23:34:12.500 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[34643]: fuse finished with error 0 and tester_r 0 2024-09-10T23:34:12.503 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:34:18.604 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:34:18.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:34:18.605 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T23:34:18.615 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:34:18.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b3600640 1 -- 172.21.15.94:0/4186847787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31ac0cfd50 msgr2=0x7f31ac0d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:18.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b3600640 1 --2- 172.21.15.94:0/4186847787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31ac0cfd50 0x7f31ac0d0200 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0d08b0 tx=0x7f31a401cb80 comp rx=0 tx=0).stop 2024-09-10T23:34:18.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b3600640 1 -- 172.21.15.94:0/4186847787 shutdown_connections 2024-09-10T23:34:18.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b3600640 1 --2- 172.21.15.94:0/4186847787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31ac0cfd50 0x7f31ac0d0200 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:18.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b3600640 1 --2- 172.21.15.94:0/4186847787 >> v2:172.21.15.181:3300/0 conn(0x7f31ac152c10 0x7f31ac0b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:18.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b3600640 1 -- 172.21.15.94:0/4186847787 >> 172.21.15.94:0/4186847787 conn(0x7f31ac06cdf0 msgr2=0x7f31ac06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:18.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b3600640 1 -- 172.21.15.94:0/4186847787 shutdown_connections 2024-09-10T23:34:18.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b3600640 1 -- 172.21.15.94:0/4186847787 wait complete. 2024-09-10T23:34:18.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b3600640 1 Processor -- start 2024-09-10T23:34:18.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b3600640 1 -- start start 2024-09-10T23:34:18.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b3600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31ac0cfd50 0x7f31ac13d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:18.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b3600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31ac152c10 0x7f31ac151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:18.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b3600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31ac15cbc0 0x7f31ac14d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:18.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b3600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac168dc0 con 0x7f31ac0cfd50 2024-09-10T23:34:18.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b3600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac168c40 con 0x7f31ac152c10 2024-09-10T23:34:18.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b3600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f31ac168f40 con 0x7f31ac15cbc0 2024-09-10T23:34:18.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.763+0000 7f31b0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31ac0cfd50 0x7f31ac13d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:18.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31b0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31ac0cfd50 0x7f31ac13d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37454/0 (socket says 172.21.15.94:37454) 2024-09-10T23:34:18.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31b0c00640 1 -- 172.21.15.94:0/1693204087 learned_addr learned my addr 172.21.15.94:0/1693204087 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:18.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31a3e00640 1 --2- 172.21.15.94:0/1693204087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31ac152c10 0x7f31ac151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:18.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31b1600640 1 --2- 172.21.15.94:0/1693204087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31ac15cbc0 0x7f31ac14d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:18.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31b0c00640 1 -- 172.21.15.94:0/1693204087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31ac15cbc0 msgr2=0x7f31ac14d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:18.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31b0c00640 1 --2- 172.21.15.94:0/1693204087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31ac15cbc0 0x7f31ac14d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:18.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31b0c00640 1 -- 172.21.15.94:0/1693204087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31ac152c10 msgr2=0x7f31ac151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:18.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31b0c00640 1 --2- 172.21.15.94:0/1693204087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31ac152c10 0x7f31ac151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:18.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31b0c00640 1 -- 172.21.15.94:0/1693204087 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ac13da40 con 0x7f31ac0cfd50 2024-09-10T23:34:18.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31a3e00640 1 --2- 172.21.15.94:0/1693204087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31ac152c10 0x7f31ac151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:34:18.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31b0c00640 1 --2- 172.21.15.94:0/1693204087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31ac0cfd50 0x7f31ac13d500 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f319c00d950 tx=0x7f319c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:18.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31a9600640 1 -- 172.21.15.94:0/1693204087 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c017070 con 0x7f31ac0cfd50 2024-09-10T23:34:18.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31a9600640 1 -- 172.21.15.94:0/1693204087 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f319c009d20 con 0x7f31ac0cfd50 2024-09-10T23:34:18.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31a9600640 1 -- 172.21.15.94:0/1693204087 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c0106c0 con 0x7f31ac0cfd50 2024-09-10T23:34:18.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31b3600640 1 -- 172.21.15.94:0/1693204087 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31ac14a8d0 con 0x7f31ac0cfd50 2024-09-10T23:34:18.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31b3600640 1 -- 172.21.15.94:0/1693204087 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31ac1512c0 con 0x7f31ac0cfd50 2024-09-10T23:34:18.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31b3600640 1 -- 172.21.15.94:0/1693204087 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31ac0d0200 con 0x7f31ac0cfd50 2024-09-10T23:34:18.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.767+0000 7f31a9600640 1 -- 172.21.15.94:0/1693204087 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7f319c010860 con 0x7f31ac0cfd50 2024-09-10T23:34:18.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.771+0000 7f31a9600640 1 --2- 172.21.15.94:0/1693204087 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f31880727c0 0x7f3188074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:18.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.771+0000 7f31a9600640 1 -- 172.21.15.94:0/1693204087 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f319c093be0 con 0x7f31ac0cfd50 2024-09-10T23:34:18.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.771+0000 7f31a3e00640 1 --2- 172.21.15.94:0/1693204087 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f31880727c0 0x7f3188074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:18.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.771+0000 7f31a3e00640 1 --2- 172.21.15.94:0/1693204087 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f31880727c0 0x7f3188074c80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f31ac14bc70 tx=0x7f3194009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:18.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.775+0000 7f31a9600640 1 -- 172.21.15.94:0/1693204087 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f319c05ee00 con 0x7f31ac0cfd50 2024-09-10T23:34:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.971+0000 7f31b3600640 1 -- 172.21.15.94:0/1693204087 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31ac15c1a0 con 0x7f31ac0cfd50 2024-09-10T23:34:18.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.971+0000 7f31a9600640 1 -- 172.21.15.94:0/1693204087 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7f319c05e700 con 0x7f31ac0cfd50 2024-09-10T23:34:18.977 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:18.977 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:34:18.977 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:34:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.975+0000 7f31b3600640 1 -- 172.21.15.94:0/1693204087 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f31880727c0 msgr2=0x7f3188074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.975+0000 7f31b3600640 1 --2- 172.21.15.94:0/1693204087 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f31880727c0 0x7f3188074c80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f31ac14bc70 tx=0x7f3194009040 comp rx=0 tx=0).stop 2024-09-10T23:34:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.975+0000 7f31b3600640 1 -- 172.21.15.94:0/1693204087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31ac0cfd50 msgr2=0x7f31ac13d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.975+0000 7f31b3600640 1 --2- 172.21.15.94:0/1693204087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31ac0cfd50 0x7f31ac13d500 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f319c00d950 tx=0x7f319c00de10 comp rx=0 tx=0).stop 2024-09-10T23:34:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.975+0000 7f31b3600640 1 -- 172.21.15.94:0/1693204087 shutdown_connections 2024-09-10T23:34:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.975+0000 7f31b3600640 1 --2- 172.21.15.94:0/1693204087 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f31880727c0 0x7f3188074c80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.975+0000 7f31b3600640 1 --2- 172.21.15.94:0/1693204087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31ac152c10 0x7f31ac151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.975+0000 7f31b3600640 1 --2- 172.21.15.94:0/1693204087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31ac15cbc0 0x7f31ac14d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.975+0000 7f31b3600640 1 --2- 172.21.15.94:0/1693204087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31ac0cfd50 0x7f31ac13d500 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.975+0000 7f31b3600640 1 -- 172.21.15.94:0/1693204087 >> 172.21.15.94:0/1693204087 conn(0x7f31ac06cdf0 msgr2=0x7f31ac14b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.975+0000 7f31b3600640 1 -- 172.21.15.94:0/1693204087 shutdown_connections 2024-09-10T23:34:18.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:18.975+0000 7f31b3600640 1 -- 172.21.15.94:0/1693204087 wait complete. 2024-09-10T23:34:18.998 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:34:19.169 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2024-09-10T23:34:19.170 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-10T23:34:19.170 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol76610 2024-09-10T23:34:19.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc909800640 1 -- 172.21.15.94:0/1645126493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9040b80b0 msgr2=0x7fc904168ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:19.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc909800640 1 --2- 172.21.15.94:0/1645126493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9040b80b0 0x7fc904168ab0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009bc0 tx=0x7fc8fc01f3b0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc909800640 1 -- 172.21.15.94:0/1645126493 shutdown_connections 2024-09-10T23:34:19.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc909800640 1 --2- 172.21.15.94:0/1645126493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9040b80b0 0x7fc904168ab0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc909800640 1 -- 172.21.15.94:0/1645126493 >> 172.21.15.94:0/1645126493 conn(0x7fc9040d3d00 msgr2=0x7fc9040d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:19.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc909800640 1 -- 172.21.15.94:0/1645126493 shutdown_connections 2024-09-10T23:34:19.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc909800640 1 -- 172.21.15.94:0/1645126493 wait complete. 2024-09-10T23:34:19.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc909800640 1 Processor -- start 2024-09-10T23:34:19.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc909800640 1 -- start start 2024-09-10T23:34:19.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc909800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9040b80b0 0x7fc904156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:19.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc909800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9041611f0 0x7fc904151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:19.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc909800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9041615d0 0x7fc904152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:19.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc909800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc904168ff0 con 0x7fc9041615d0 2024-09-10T23:34:19.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc909800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc9040d4a30 con 0x7fc9040b80b0 2024-09-10T23:34:19.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc909800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc904169170 con 0x7fc9041611f0 2024-09-10T23:34:19.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc908e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9041615d0 0x7fc904152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:19.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc908e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9041615d0 0x7fc904152190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37474/0 (socket says 172.21.15.94:37474) 2024-09-10T23:34:19.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc908e00640 1 -- 172.21.15.94:0/464663000 learned_addr learned my addr 172.21.15.94:0/464663000 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:19.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc903e00640 1 --2- 172.21.15.94:0/464663000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9040b80b0 0x7fc904156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:19.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.311+0000 7fc8fbe00640 1 --2- 172.21.15.94:0/464663000 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9041611f0 0x7fc904151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:19.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.315+0000 7fc908e00640 1 -- 172.21.15.94:0/464663000 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9041611f0 msgr2=0x7fc904151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:19.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.315+0000 7fc908e00640 1 --2- 172.21.15.94:0/464663000 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9041611f0 0x7fc904151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.315+0000 7fc908e00640 1 -- 172.21.15.94:0/464663000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9040b80b0 msgr2=0x7fc904156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:19.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.315+0000 7fc908e00640 1 --2- 172.21.15.94:0/464663000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9040b80b0 0x7fc904156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.315+0000 7fc908e00640 1 -- 172.21.15.94:0/464663000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9041f8f60 con 0x7fc9041615d0 2024-09-10T23:34:19.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.315+0000 7fc908e00640 1 --2- 172.21.15.94:0/464663000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9041615d0 0x7fc904152190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc01cfe0 tx=0x7fc8fc0af560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:19.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.315+0000 7fc900c00640 1 -- 172.21.15.94:0/464663000 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc0b4040 con 0x7fc9041615d0 2024-09-10T23:34:19.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.315+0000 7fc900c00640 1 -- 172.21.15.94:0/464663000 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8fc0c1070 con 0x7fc9041615d0 2024-09-10T23:34:19.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.315+0000 7fc900c00640 1 -- 172.21.15.94:0/464663000 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc004000 con 0x7fc9041615d0 2024-09-10T23:34:19.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.315+0000 7fc909800640 1 -- 172.21.15.94:0/464663000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9041f4f00 con 0x7fc9041615d0 2024-09-10T23:34:19.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.315+0000 7fc909800640 1 -- 172.21.15.94:0/464663000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9041f54c0 con 0x7fc9041615d0 2024-09-10T23:34:19.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.315+0000 7fc909800640 1 -- 172.21.15.94:0/464663000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9041f4cf0 con 0x7fc9041615d0 2024-09-10T23:34:19.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.315+0000 7fc900c00640 1 -- 172.21.15.94:0/464663000 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7fc8fc0041a0 con 0x7fc9041615d0 2024-09-10T23:34:19.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.315+0000 7fc900c00640 1 --2- 172.21.15.94:0/464663000 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc8e00727c0 0x7fc8e0074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:19.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.315+0000 7fc900c00640 1 -- 172.21.15.94:0/464663000 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fc8fc131fe0 con 0x7fc9041615d0 2024-09-10T23:34:19.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.319+0000 7fc903e00640 1 --2- 172.21.15.94:0/464663000 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc8e00727c0 0x7fc8e0074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:19.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.319+0000 7fc903e00640 1 --2- 172.21.15.94:0/464663000 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc8e00727c0 0x7fc8e0074c80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc9041f2850 tx=0x7fc8f4007e40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:19.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.319+0000 7fc900c00640 1 -- 172.21.15.94:0/464663000 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc8fc018f10 con 0x7fc9041615d0 2024-09-10T23:34:19.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.507+0000 7fc909800640 1 -- 172.21.15.94:0/464663000 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol76610", "target": ["mon-mgr", ""]}) -- 0x7fc9040d1d80 con 0x7fc8e00727c0 2024-09-10T23:34:19.520 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:34:19.516+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:34:19.520 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:34:19.516+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:34:19.520 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:34:19.516+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:34:19.520 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:34:19.516+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:34:19.520 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:34:19.516+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:34:19.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.543+0000 7fc900c00640 1 -- 172.21.15.94:0/464663000 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc9040d1d80 con 0x7fc8e00727c0 2024-09-10T23:34:19.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.547+0000 7fc909800640 1 -- 172.21.15.94:0/464663000 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc8e00727c0 msgr2=0x7fc8e0074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:19.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.547+0000 7fc909800640 1 --2- 172.21.15.94:0/464663000 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc8e00727c0 0x7fc8e0074c80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc9041f2850 tx=0x7fc8f4007e40 comp rx=0 tx=0).stop 2024-09-10T23:34:19.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.547+0000 7fc909800640 1 -- 172.21.15.94:0/464663000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9041615d0 msgr2=0x7fc904152190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:19.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.547+0000 7fc909800640 1 --2- 172.21.15.94:0/464663000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9041615d0 0x7fc904152190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc01cfe0 tx=0x7fc8fc0af560 comp rx=0 tx=0).stop 2024-09-10T23:34:19.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.547+0000 7fc909800640 1 -- 172.21.15.94:0/464663000 shutdown_connections 2024-09-10T23:34:19.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.547+0000 7fc909800640 1 --2- 172.21.15.94:0/464663000 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc8e00727c0 0x7fc8e0074c80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.547+0000 7fc909800640 1 --2- 172.21.15.94:0/464663000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9041615d0 0x7fc904152190 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.547+0000 7fc909800640 1 --2- 172.21.15.94:0/464663000 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9041611f0 0x7fc904151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.547+0000 7fc909800640 1 --2- 172.21.15.94:0/464663000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9040b80b0 0x7fc904156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.547+0000 7fc909800640 1 -- 172.21.15.94:0/464663000 >> 172.21.15.94:0/464663000 conn(0x7fc9040d3d00 msgr2=0x7fc9040d40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:19.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.547+0000 7fc909800640 1 -- 172.21.15.94:0/464663000 shutdown_connections 2024-09-10T23:34:19.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.547+0000 7fc909800640 1 -- 172.21.15.94:0/464663000 wait complete. 2024-09-10T23:34:19.572 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:34:19.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.707+0000 7ff4bba00640 1 -- 172.21.15.94:0/2980821792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b415e3a0 msgr2=0x7ff4b406e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.707+0000 7ff4bba00640 1 --2- 172.21.15.94:0/2980821792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b415e3a0 0x7ff4b406e5c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00a950 tx=0x7ff4ac01dab0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.707+0000 7ff4bba00640 1 -- 172.21.15.94:0/2980821792 shutdown_connections 2024-09-10T23:34:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.707+0000 7ff4bba00640 1 --2- 172.21.15.94:0/2980821792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b415e3a0 0x7ff4b406e5c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.707+0000 7ff4bba00640 1 -- 172.21.15.94:0/2980821792 >> 172.21.15.94:0/2980821792 conn(0x7ff4b406cdf0 msgr2=0x7ff4b406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.707+0000 7ff4bba00640 1 -- 172.21.15.94:0/2980821792 shutdown_connections 2024-09-10T23:34:19.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.707+0000 7ff4bba00640 1 -- 172.21.15.94:0/2980821792 wait complete. 2024-09-10T23:34:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4bba00640 1 Processor -- start 2024-09-10T23:34:19.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4bba00640 1 -- start start 2024-09-10T23:34:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4bba00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b415a060 0x7ff4b4146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4bba00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b415e3a0 0x7ff4b415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4bba00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b415e8a0 0x7ff4b40cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4bba00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b4073790 con 0x7ff4b415e3a0 2024-09-10T23:34:19.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4bba00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b4073610 con 0x7ff4b415e8a0 2024-09-10T23:34:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4bba00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff4b4073910 con 0x7ff4b415a060 2024-09-10T23:34:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4b3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b415e3a0 0x7ff4b415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4b3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b415e3a0 0x7ff4b415a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37490/0 (socket says 172.21.15.94:37490) 2024-09-10T23:34:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4b3e00640 1 -- 172.21.15.94:0/2899796866 learned_addr learned my addr 172.21.15.94:0/2899796866 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4b9000640 1 --2- 172.21.15.94:0/2899796866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b415a060 0x7ff4b4146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4b9a00640 1 --2- 172.21.15.94:0/2899796866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b415e8a0 0x7ff4b40cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4b3e00640 1 -- 172.21.15.94:0/2899796866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b415a060 msgr2=0x7ff4b4146470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4b3e00640 1 --2- 172.21.15.94:0/2899796866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b415a060 0x7ff4b4146470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4b3e00640 1 -- 172.21.15.94:0/2899796866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b415e8a0 msgr2=0x7ff4b40cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4b3e00640 1 --2- 172.21.15.94:0/2899796866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b415e8a0 0x7ff4b40cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4b3e00640 1 -- 172.21.15.94:0/2899796866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b415b1d0 con 0x7ff4b415e3a0 2024-09-10T23:34:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4b3e00640 1 --2- 172.21.15.94:0/2899796866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b415e3a0 0x7ff4b415a710 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000c450 tx=0x7ff4a000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:19.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4b0c00640 1 -- 172.21.15.94:0/2899796866 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00052d0 con 0x7ff4b415e3a0 2024-09-10T23:34:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4b0c00640 1 -- 172.21.15.94:0/2899796866 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4a0005470 con 0x7ff4b415e3a0 2024-09-10T23:34:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4b0c00640 1 -- 172.21.15.94:0/2899796866 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0012870 con 0x7ff4b415e3a0 2024-09-10T23:34:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4bba00640 1 -- 172.21.15.94:0/2899796866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b415b4b0 con 0x7ff4b415e3a0 2024-09-10T23:34:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.711+0000 7ff4bba00640 1 -- 172.21.15.94:0/2899796866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b415c720 con 0x7ff4b415e3a0 2024-09-10T23:34:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.715+0000 7ff4bba00640 1 -- 172.21.15.94:0/2899796866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b406e5c0 con 0x7ff4b415e3a0 2024-09-10T23:34:19.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.715+0000 7ff4b0c00640 1 -- 172.21.15.94:0/2899796866 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7ff4a0003600 con 0x7ff4b415e3a0 2024-09-10T23:34:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.715+0000 7ff4b0c00640 1 --2- 172.21.15.94:0/2899796866 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff494072790 0x7ff494074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.715+0000 7ff4b0c00640 1 -- 172.21.15.94:0/2899796866 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7ff4a0093090 con 0x7ff4b415e3a0 2024-09-10T23:34:19.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.715+0000 7ff4b9000640 1 --2- 172.21.15.94:0/2899796866 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff494072790 0x7ff494074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.715+0000 7ff4b9000640 1 --2- 172.21.15.94:0/2899796866 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff494072790 0x7ff494074c50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4004a40 tx=0x7ff4a4001ae0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:19.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.719+0000 7ff4b0c00640 1 -- 172.21.15.94:0/2899796866 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff4a005dcf0 con 0x7ff4b415e3a0 2024-09-10T23:34:19.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.919+0000 7ff4bba00640 1 -- 172.21.15.94:0/2899796866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4b415b680 con 0x7ff4b415e3a0 2024-09-10T23:34:19.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.919+0000 7ff4b0c00640 1 -- 172.21.15.94:0/2899796866 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7ff4a0062720 con 0x7ff4b415e3a0 2024-09-10T23:34:19.925 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:19.926 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:34:19.926 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:34:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.923+0000 7ff4bba00640 1 -- 172.21.15.94:0/2899796866 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff494072790 msgr2=0x7ff494074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.923+0000 7ff4bba00640 1 --2- 172.21.15.94:0/2899796866 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff494072790 0x7ff494074c50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4004a40 tx=0x7ff4a4001ae0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.923+0000 7ff4bba00640 1 -- 172.21.15.94:0/2899796866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b415e3a0 msgr2=0x7ff4b415a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.923+0000 7ff4bba00640 1 --2- 172.21.15.94:0/2899796866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b415e3a0 0x7ff4b415a710 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000c450 tx=0x7ff4a000c910 comp rx=0 tx=0).stop 2024-09-10T23:34:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.923+0000 7ff4bba00640 1 -- 172.21.15.94:0/2899796866 shutdown_connections 2024-09-10T23:34:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.923+0000 7ff4bba00640 1 --2- 172.21.15.94:0/2899796866 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff494072790 0x7ff494074c50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.923+0000 7ff4bba00640 1 --2- 172.21.15.94:0/2899796866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b415e8a0 0x7ff4b40cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.923+0000 7ff4bba00640 1 --2- 172.21.15.94:0/2899796866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4b415e3a0 0x7ff4b415a710 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.923+0000 7ff4bba00640 1 --2- 172.21.15.94:0/2899796866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b415a060 0x7ff4b4146470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.923+0000 7ff4bba00640 1 -- 172.21.15.94:0/2899796866 >> 172.21.15.94:0/2899796866 conn(0x7ff4b406cdf0 msgr2=0x7ff4b4157db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.923+0000 7ff4bba00640 1 -- 172.21.15.94:0/2899796866 shutdown_connections 2024-09-10T23:34:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:19.923+0000 7ff4bba00640 1 -- 172.21.15.94:0/2899796866 wait complete. 2024-09-10T23:34:19.947 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:34:20.110 INFO:teuthology.orchestra.run.smithi181.stdout:[{"id":5377,"entity":{"name":{"type":"client","num":5377},"addr":{"type":"any","addr":"172.21.15.181:0","nonce":1309871564}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.57198068099999999,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.971434140159754,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5377 172.21.15.181:0/1309871564","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8f219c476cb4f26fc5719c661c0323719a45cb30","ceph_version":"ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)","entity_id":"x","hostname":"smithi181","pid":"28823","root":"/"}}] 2024-09-10T23:34:20.111 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi181", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28823", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.181:0", "nonce": 1309871564, "type": "any" }, "name": { "num": 5377, "type": "client" } }, "id": 5377, "inst": "client.5377 172.21.15.181:0/1309871564", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.971434140159754 }, "state": "open", "uptime": 0.571980681 } ] 2024-09-10T23:34:20.111 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:34:20.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.247+0000 7fb9a1c00640 1 -- 172.21.15.94:0/293936306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb99c0cddb0 msgr2=0x7fb99c0ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:20.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.247+0000 7fb9a1c00640 1 --2- 172.21.15.94:0/293936306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb99c0cddb0 0x7fb99c0ce260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb994009840 tx=0x7fb99401c610 comp rx=0 tx=0).stop 2024-09-10T23:34:20.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.247+0000 7fb9a1c00640 1 -- 172.21.15.94:0/293936306 shutdown_connections 2024-09-10T23:34:20.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.247+0000 7fb9a1c00640 1 --2- 172.21.15.94:0/293936306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb99c0cddb0 0x7fb99c0ce260 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:20.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.247+0000 7fb9a1c00640 1 --2- 172.21.15.94:0/293936306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb99c0cfcb0 0x7fb99c0d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:20.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.247+0000 7fb9a1c00640 1 -- 172.21.15.94:0/293936306 >> 172.21.15.94:0/293936306 conn(0x7fb99c06cdf0 msgr2=0x7fb99c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:20.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.247+0000 7fb9a1c00640 1 -- 172.21.15.94:0/293936306 shutdown_connections 2024-09-10T23:34:20.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.247+0000 7fb9a1c00640 1 -- 172.21.15.94:0/293936306 wait complete. 2024-09-10T23:34:20.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb9a1c00640 1 Processor -- start 2024-09-10T23:34:20.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb9a1c00640 1 -- start start 2024-09-10T23:34:20.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb9a1c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb99c0cddb0 0x7fb99c15d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:20.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb9a1c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb99c0cfcb0 0x7fb99c151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:20.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb9a1c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb99c15ba50 0x7fb99c15c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:20.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb9a1c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb99c168be0 con 0x7fb99c0cfcb0 2024-09-10T23:34:20.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb9a1c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb99c168a60 con 0x7fb99c0cddb0 2024-09-10T23:34:20.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb9a1c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb99c168d60 con 0x7fb99c15ba50 2024-09-10T23:34:20.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb992000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb99c0cfcb0 0x7fb99c151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:20.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb99aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb99c0cddb0 0x7fb99c15d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:20.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb992000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb99c0cfcb0 0x7fb99c151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37498/0 (socket says 172.21.15.94:37498) 2024-09-10T23:34:20.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb992000640 1 -- 172.21.15.94:0/1479051961 learned_addr learned my addr 172.21.15.94:0/1479051961 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:20.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb99aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb99c0cddb0 0x7fb99c15d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:40518/0 (socket says 172.21.15.94:40518) 2024-09-10T23:34:20.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb99b400640 1 --2- 172.21.15.94:0/1479051961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb99c15ba50 0x7fb99c15c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:20.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb992000640 1 -- 172.21.15.94:0/1479051961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb99c15ba50 msgr2=0x7fb99c15c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:20.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb992000640 1 --2- 172.21.15.94:0/1479051961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb99c15ba50 0x7fb99c15c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:20.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb992000640 1 -- 172.21.15.94:0/1479051961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb99c0cddb0 msgr2=0x7fb99c15d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:20.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb992000640 1 --2- 172.21.15.94:0/1479051961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb99c0cddb0 0x7fb99c15d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:20.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb992000640 1 -- 172.21.15.94:0/1479051961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99c150a20 con 0x7fb99c0cfcb0 2024-09-10T23:34:20.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb992000640 1 --2- 172.21.15.94:0/1479051961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb99c0cfcb0 0x7fb99c151ac0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb98400e9f0 tx=0x7fb98400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:20.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb993400640 1 -- 172.21.15.94:0/1479051961 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98400cde0 con 0x7fb99c0cfcb0 2024-09-10T23:34:20.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb993400640 1 -- 172.21.15.94:0/1479051961 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb984010430 con 0x7fb99c0cfcb0 2024-09-10T23:34:20.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.251+0000 7fb9a1c00640 1 -- 172.21.15.94:0/1479051961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb99c150d00 con 0x7fb99c0cfcb0 2024-09-10T23:34:20.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.255+0000 7fb993400640 1 -- 172.21.15.94:0/1479051961 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984010880 con 0x7fb99c0cfcb0 2024-09-10T23:34:20.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.255+0000 7fb9a1c00640 1 -- 172.21.15.94:0/1479051961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb99c1512c0 con 0x7fb99c0cfcb0 2024-09-10T23:34:20.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.255+0000 7fb9a1c00640 1 -- 172.21.15.94:0/1479051961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb99c0ce260 con 0x7fb99c0cfcb0 2024-09-10T23:34:20.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.255+0000 7fb993400640 1 -- 172.21.15.94:0/1479051961 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) ==== 96547+0+0 (secure 0 0 0) 0x7fb984010a20 con 0x7fb99c0cfcb0 2024-09-10T23:34:20.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.255+0000 7fb993400640 1 --2- 172.21.15.94:0/1479051961 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb978072790 0x7fb978074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:20.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.255+0000 7fb993400640 1 -- 172.21.15.94:0/1479051961 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fb9840945d0 con 0x7fb99c0cfcb0 2024-09-10T23:34:20.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.255+0000 7fb99aa00640 1 --2- 172.21.15.94:0/1479051961 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb978072790 0x7fb978074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:20.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.259+0000 7fb99aa00640 1 --2- 172.21.15.94:0/1479051961 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb978072790 0x7fb978074c50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb99c159b80 tx=0x7fb98c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:20.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.259+0000 7fb993400640 1 -- 172.21.15.94:0/1479051961 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb984026020 con 0x7fb99c0cfcb0 2024-09-10T23:34:20.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.459+0000 7fb9a1c00640 1 -- 172.21.15.94:0/1479051961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb99c14cba0 con 0x7fb99c0cfcb0 2024-09-10T23:34:20.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.459+0000 7fb993400640 1 -- 172.21.15.94:0/1479051961 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7fb98405f2b0 con 0x7fb99c0cfcb0 2024-09-10T23:34:20.466 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:20.466 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:34:20.466 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:34:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.463+0000 7fb9a1c00640 1 -- 172.21.15.94:0/1479051961 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb978072790 msgr2=0x7fb978074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.463+0000 7fb9a1c00640 1 --2- 172.21.15.94:0/1479051961 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb978072790 0x7fb978074c50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb99c159b80 tx=0x7fb98c009040 comp rx=0 tx=0).stop 2024-09-10T23:34:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.463+0000 7fb9a1c00640 1 -- 172.21.15.94:0/1479051961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb99c0cfcb0 msgr2=0x7fb99c151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.463+0000 7fb9a1c00640 1 --2- 172.21.15.94:0/1479051961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb99c0cfcb0 0x7fb99c151ac0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb98400e9f0 tx=0x7fb98400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:34:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.463+0000 7fb9a1c00640 1 -- 172.21.15.94:0/1479051961 shutdown_connections 2024-09-10T23:34:20.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.463+0000 7fb9a1c00640 1 --2- 172.21.15.94:0/1479051961 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb978072790 0x7fb978074c50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.463+0000 7fb9a1c00640 1 --2- 172.21.15.94:0/1479051961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb99c15ba50 0x7fb99c15c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.463+0000 7fb9a1c00640 1 --2- 172.21.15.94:0/1479051961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb99c0cfcb0 0x7fb99c151ac0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.463+0000 7fb9a1c00640 1 --2- 172.21.15.94:0/1479051961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb99c0cddb0 0x7fb99c15d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.463+0000 7fb9a1c00640 1 -- 172.21.15.94:0/1479051961 >> 172.21.15.94:0/1479051961 conn(0x7fb99c06cdf0 msgr2=0x7fb99c072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.463+0000 7fb9a1c00640 1 -- 172.21.15.94:0/1479051961 shutdown_connections 2024-09-10T23:34:20.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:20.463+0000 7fb9a1c00640 1 -- 172.21.15.94:0/1479051961 wait complete. 2024-09-10T23:34:20.487 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:34:20.647 INFO:teuthology.orchestra.run.smithi181.stdout:[{"id":5377,"entity":{"name":{"type":"client","num":5377},"addr":{"type":"any","addr":"172.21.15.181:0","nonce":1309871564}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":1.1079630300000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.944241948286731,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5377 172.21.15.181:0/1309871564","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8f219c476cb4f26fc5719c661c0323719a45cb30","ceph_version":"ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)","entity_id":"x","hostname":"smithi181","pid":"28823","root":"/"}}] 2024-09-10T23:34:20.648 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi181", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28823", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.181:0", "nonce": 1309871564, "type": "any" }, "name": { "num": 5377, "type": "client" } }, "id": 5377, "inst": "client.5377 172.21.15.181:0/1309871564", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.94424194828673 }, "state": "open", "uptime": 1.10796303 } ] 2024-09-10T23:34:20.648 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T23:34:25.649 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:34:25.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.795+0000 7f10ca800640 1 -- 172.21.15.94:0/1199306531 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c415f5c0 msgr2=0x7f10c40cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:25.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.795+0000 7f10ca800640 1 --2- 172.21.15.94:0/1199306531 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c415f5c0 0x7f10c40cfb60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009bc0 tx=0x7f10b401ca80 comp rx=0 tx=0).stop 2024-09-10T23:34:25.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.795+0000 7f10ca800640 1 -- 172.21.15.94:0/1199306531 shutdown_connections 2024-09-10T23:34:25.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.795+0000 7f10ca800640 1 --2- 172.21.15.94:0/1199306531 >> v2:172.21.15.181:3300/0 conn(0x7f10c40b2bb0 0x7f10c40d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:25.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.795+0000 7f10ca800640 1 --2- 172.21.15.94:0/1199306531 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c415f5c0 0x7f10c40cfb60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:25.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.795+0000 7f10ca800640 1 -- 172.21.15.94:0/1199306531 >> 172.21.15.94:0/1199306531 conn(0x7f10c40d1c60 msgr2=0x7f10c40d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:25.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.795+0000 7f10ca800640 1 -- 172.21.15.94:0/1199306531 shutdown_connections 2024-09-10T23:34:25.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.795+0000 7f10ca800640 1 -- 172.21.15.94:0/1199306531 wait complete. 2024-09-10T23:34:25.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10ca800640 1 Processor -- start 2024-09-10T23:34:25.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10ca800640 1 -- start start 2024-09-10T23:34:25.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10ca800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c40b2bb0 0x7f10c40c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:25.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10ca800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c40d05e0 0x7f10c40c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:25.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10ca800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10c415f5c0 0x7f10c4154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:25.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10ca800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41691b0 con 0x7f10c415f5c0 2024-09-10T23:34:25.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10ca800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f10c4169030 con 0x7f10c40b2bb0 2024-09-10T23:34:25.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10ca800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f10c4169330 con 0x7f10c40d05e0 2024-09-10T23:34:25.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10c3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10c415f5c0 0x7f10c4154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:25.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10c3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10c415f5c0 0x7f10c4154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59102/0 (socket says 172.21.15.94:59102) 2024-09-10T23:34:25.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10c3e00640 1 -- 172.21.15.94:0/1073841985 learned_addr learned my addr 172.21.15.94:0/1073841985 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:25.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10baa00640 1 --2- 172.21.15.94:0/1073841985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c40d05e0 0x7f10c40c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:25.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10c3400640 1 --2- 172.21.15.94:0/1073841985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c40b2bb0 0x7f10c40c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:25.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10c3e00640 1 -- 172.21.15.94:0/1073841985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c40d05e0 msgr2=0x7f10c40c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:25.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10c3e00640 1 --2- 172.21.15.94:0/1073841985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c40d05e0 0x7f10c40c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:25.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10c3e00640 1 -- 172.21.15.94:0/1073841985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c40b2bb0 msgr2=0x7f10c40c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:25.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10c3e00640 1 --2- 172.21.15.94:0/1073841985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c40b2bb0 0x7f10c40c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:25.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10c3e00640 1 -- 172.21.15.94:0/1073841985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c40c78b0 con 0x7f10c415f5c0 2024-09-10T23:34:25.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10c3e00640 1 --2- 172.21.15.94:0/1073841985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10c415f5c0 0x7f10c4154ae0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f10bc001ec0 tx=0x7f10bc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:25.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10bbe00640 1 -- 172.21.15.94:0/1073841985 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc017070 con 0x7f10c415f5c0 2024-09-10T23:34:25.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10bbe00640 1 -- 172.21.15.94:0/1073841985 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10bc0053c0 con 0x7f10c415f5c0 2024-09-10T23:34:25.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10bbe00640 1 -- 172.21.15.94:0/1073841985 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc003860 con 0x7f10c415f5c0 2024-09-10T23:34:25.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10ca800640 1 -- 172.21.15.94:0/1073841985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c40c5970 con 0x7f10c415f5c0 2024-09-10T23:34:25.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.799+0000 7f10ca800640 1 -- 172.21.15.94:0/1073841985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c40c5f30 con 0x7f10c415f5c0 2024-09-10T23:34:25.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.803+0000 7f10ca800640 1 -- 172.21.15.94:0/1073841985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c40c6130 con 0x7f10c415f5c0 2024-09-10T23:34:25.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.803+0000 7f10bbe00640 1 -- 172.21.15.94:0/1073841985 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 96598+0+0 (secure 0 0 0) 0x7f10bc003a00 con 0x7f10c415f5c0 2024-09-10T23:34:25.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.803+0000 7f10bbe00640 1 --2- 172.21.15.94:0/1073841985 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10a0072850 0x7f10a0074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:25.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.803+0000 7f10bbe00640 1 -- 172.21.15.94:0/1073841985 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7f10bc092d40 con 0x7f10c415f5c0 2024-09-10T23:34:25.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.803+0000 7f10c3400640 1 --2- 172.21.15.94:0/1073841985 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10a0072850 0x7f10a0074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:25.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.803+0000 7f10c3400640 1 --2- 172.21.15.94:0/1073841985 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10a0072850 0x7f10a0074d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f10b40af000 tx=0x7f10b401b2c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:25.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:25.807+0000 7f10bbe00640 1 -- 172.21.15.94:0/1073841985 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f10c40c6130 con 0x7f10c415f5c0 2024-09-10T23:34:26.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:26.003+0000 7f10ca800640 1 -- 172.21.15.94:0/1073841985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10c40b15c0 con 0x7f10c415f5c0 2024-09-10T23:34:26.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:26.003+0000 7f10bbe00640 1 -- 172.21.15.94:0/1073841985 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7f10bc05d990 con 0x7f10c415f5c0 2024-09-10T23:34:26.009 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:26.009 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:34:26.009 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:34:26.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:26.007+0000 7f10ca800640 1 -- 172.21.15.94:0/1073841985 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10a0072850 msgr2=0x7f10a0074d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:26.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:26.007+0000 7f10ca800640 1 --2- 172.21.15.94:0/1073841985 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10a0072850 0x7f10a0074d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f10b40af000 tx=0x7f10b401b2c0 comp rx=0 tx=0).stop 2024-09-10T23:34:26.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:26.007+0000 7f10ca800640 1 -- 172.21.15.94:0/1073841985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10c415f5c0 msgr2=0x7f10c4154ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:26.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:26.007+0000 7f10ca800640 1 --2- 172.21.15.94:0/1073841985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10c415f5c0 0x7f10c4154ae0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f10bc001ec0 tx=0x7f10bc00c890 comp rx=0 tx=0).stop 2024-09-10T23:34:26.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:26.007+0000 7f10ca800640 1 -- 172.21.15.94:0/1073841985 shutdown_connections 2024-09-10T23:34:26.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:26.007+0000 7f10ca800640 1 --2- 172.21.15.94:0/1073841985 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f10a0072850 0x7f10a0074d10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:26.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:26.007+0000 7f10ca800640 1 --2- 172.21.15.94:0/1073841985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10c415f5c0 0x7f10c4154ae0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:26.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:26.007+0000 7f10ca800640 1 --2- 172.21.15.94:0/1073841985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c40d05e0 0x7f10c40c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:26.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:26.007+0000 7f10ca800640 1 --2- 172.21.15.94:0/1073841985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c40b2bb0 0x7f10c40c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:26.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:26.007+0000 7f10ca800640 1 -- 172.21.15.94:0/1073841985 >> 172.21.15.94:0/1073841985 conn(0x7f10c40d1c60 msgr2=0x7f10c40d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:26.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:26.007+0000 7f10ca800640 1 -- 172.21.15.94:0/1073841985 shutdown_connections 2024-09-10T23:34:26.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:26.007+0000 7f10ca800640 1 -- 172.21.15.94:0/1073841985 wait complete. 2024-09-10T23:34:26.031 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:34:26.204 INFO:teuthology.orchestra.run.smithi181.stdout:[{"id":5377,"entity":{"name":{"type":"client","num":5377},"addr":{"type":"any","addr":"172.21.15.181:0","nonce":1309871564}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":6.6637927860000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.664350745657465,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5377 172.21.15.181:0/1309871564","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8f219c476cb4f26fc5719c661c0323719a45cb30","ceph_version":"ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)","entity_id":"x","hostname":"smithi181","pid":"28823","root":"/"}}] 2024-09-10T23:34:26.204 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi181", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28823", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.181:0", "nonce": 1309871564, "type": "any" }, "name": { "num": 5377, "type": "client" } }, "id": 5377, "inst": "client.5377 172.21.15.181:0/1309871564", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.664350745657465 }, "state": "open", "uptime": 6.663792786 } ] 2024-09-10T23:34:26.204 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T23:34:31.205 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:34:31.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.351+0000 7f915b200640 1 -- 172.21.15.94:0/2668615289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915c06d060 msgr2=0x7f915c0d1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:31.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.351+0000 7f915b200640 1 --2- 172.21.15.94:0/2668615289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915c06d060 0x7f915c0d1020 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9154009c80 tx=0x7f915401cda0 comp rx=0 tx=0).stop 2024-09-10T23:34:31.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.351+0000 7f915b200640 1 -- 172.21.15.94:0/2668615289 shutdown_connections 2024-09-10T23:34:31.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.351+0000 7f915b200640 1 --2- 172.21.15.94:0/2668615289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915c06d060 0x7f915c0d1020 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:31.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.351+0000 7f915b200640 1 --2- 172.21.15.94:0/2668615289 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f915c0b0010 0x7f915c15fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:31.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.351+0000 7f915b200640 1 -- 172.21.15.94:0/2668615289 >> 172.21.15.94:0/2668615289 conn(0x7f915c06dd30 msgr2=0x7f915c06e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:31.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.351+0000 7f915b200640 1 -- 172.21.15.94:0/2668615289 shutdown_connections 2024-09-10T23:34:31.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.351+0000 7f915b200640 1 -- 172.21.15.94:0/2668615289 wait complete. 2024-09-10T23:34:31.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.351+0000 7f915b200640 1 Processor -- start 2024-09-10T23:34:31.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.351+0000 7f915b200640 1 -- start start 2024-09-10T23:34:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f915b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f915c06cc20 0x7f915c0b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f915b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915c06d060 0x7f915c148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f915b200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f915c0b0010 0x7f915c1648a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f915b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f915c15cf70 con 0x7f915c06d060 2024-09-10T23:34:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f915b200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f915c15cdf0 con 0x7f915c06cc20 2024-09-10T23:34:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f915b200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f915c0d1560 con 0x7f915c0b0010 2024-09-10T23:34:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f9159400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915c06d060 0x7f915c148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f9159400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915c06d060 0x7f915c148510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59126/0 (socket says 172.21.15.94:59126) 2024-09-10T23:34:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f9159400640 1 -- 172.21.15.94:0/4230971122 learned_addr learned my addr 172.21.15.94:0/4230971122 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f9159e00640 1 --2- 172.21.15.94:0/4230971122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f915c06cc20 0x7f915c0b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f915a800640 1 --2- 172.21.15.94:0/4230971122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f915c0b0010 0x7f915c1648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f9159400640 1 -- 172.21.15.94:0/4230971122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f915c0b0010 msgr2=0x7f915c1648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f9159400640 1 --2- 172.21.15.94:0/4230971122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f915c0b0010 0x7f915c1648a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f9159400640 1 -- 172.21.15.94:0/4230971122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f915c06cc20 msgr2=0x7f915c0b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f9159400640 1 --2- 172.21.15.94:0/4230971122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f915c06cc20 0x7f915c0b3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f9159400640 1 -- 172.21.15.94:0/4230971122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915c15b970 con 0x7f915c06d060 2024-09-10T23:34:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f9159400640 1 --2- 172.21.15.94:0/4230971122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915c06d060 0x7f915c148510 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f914800c530 tx=0x7f914800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f9152000640 1 -- 172.21.15.94:0/4230971122 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9148017070 con 0x7f915c06d060 2024-09-10T23:34:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f9152000640 1 -- 172.21.15.94:0/4230971122 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91480053c0 con 0x7f915c06d060 2024-09-10T23:34:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f9152000640 1 -- 172.21.15.94:0/4230971122 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9148012890 con 0x7f915c06d060 2024-09-10T23:34:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f915b200640 1 -- 172.21.15.94:0/4230971122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f915c164de0 con 0x7f915c06d060 2024-09-10T23:34:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f915b200640 1 -- 172.21.15.94:0/4230971122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f915c0b1570 con 0x7f915c06d060 2024-09-10T23:34:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.355+0000 7f915b200640 1 -- 172.21.15.94:0/4230971122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f915c15fe60 con 0x7f915c06d060 2024-09-10T23:34:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.359+0000 7f9152000640 1 -- 172.21.15.94:0/4230971122 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 96598+0+0 (secure 0 0 0) 0x7f9148003600 con 0x7f915c06d060 2024-09-10T23:34:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.359+0000 7f9152000640 1 --2- 172.21.15.94:0/4230971122 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f913c072820 0x7f913c074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.359+0000 7f9152000640 1 -- 172.21.15.94:0/4230971122 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7f9148092da0 con 0x7f915c06d060 2024-09-10T23:34:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.359+0000 7f9159e00640 1 --2- 172.21.15.94:0/4230971122 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f913c072820 0x7f913c074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.359+0000 7f9159e00640 1 --2- 172.21.15.94:0/4230971122 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f913c072820 0x7f913c074ce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f914c005fd0 tx=0x7f914c00b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:31.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.363+0000 7f9152000640 1 -- 172.21.15.94:0/4230971122 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f915c0ceb70 con 0x7f915c06d060 2024-09-10T23:34:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.559+0000 7f915b200640 1 -- 172.21.15.94:0/4230971122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f915c15ed10 con 0x7f915c06d060 2024-09-10T23:34:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.559+0000 7f9152000640 1 -- 172.21.15.94:0/4230971122 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7f914805d890 con 0x7f915c06d060 2024-09-10T23:34:31.565 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:31.565 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:34:31.565 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:34:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.563+0000 7f915b200640 1 -- 172.21.15.94:0/4230971122 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f913c072820 msgr2=0x7f913c074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.563+0000 7f915b200640 1 --2- 172.21.15.94:0/4230971122 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f913c072820 0x7f913c074ce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f914c005fd0 tx=0x7f914c00b390 comp rx=0 tx=0).stop 2024-09-10T23:34:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.563+0000 7f915b200640 1 -- 172.21.15.94:0/4230971122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915c06d060 msgr2=0x7f915c148510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.563+0000 7f915b200640 1 --2- 172.21.15.94:0/4230971122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915c06d060 0x7f915c148510 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f914800c530 tx=0x7f914800c9f0 comp rx=0 tx=0).stop 2024-09-10T23:34:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.563+0000 7f915b200640 1 -- 172.21.15.94:0/4230971122 shutdown_connections 2024-09-10T23:34:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.563+0000 7f915b200640 1 --2- 172.21.15.94:0/4230971122 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f913c072820 0x7f913c074ce0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.563+0000 7f915b200640 1 --2- 172.21.15.94:0/4230971122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f915c0b0010 0x7f915c1648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.563+0000 7f915b200640 1 --2- 172.21.15.94:0/4230971122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915c06d060 0x7f915c148510 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.563+0000 7f915b200640 1 --2- 172.21.15.94:0/4230971122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f915c06cc20 0x7f915c0b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.563+0000 7f915b200640 1 -- 172.21.15.94:0/4230971122 >> 172.21.15.94:0/4230971122 conn(0x7f915c06dd30 msgr2=0x7f915c1510d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.563+0000 7f915b200640 1 -- 172.21.15.94:0/4230971122 shutdown_connections 2024-09-10T23:34:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:31.563+0000 7f915b200640 1 -- 172.21.15.94:0/4230971122 wait complete. 2024-09-10T23:34:31.587 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:34:31.757 INFO:teuthology.orchestra.run.smithi181.stdout:[{"id":5377,"entity":{"name":{"type":"client","num":5377},"addr":{"type":"any","addr":"172.21.15.181:0","nonce":1309871564}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":12.215642069999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.388226161595412,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5377 172.21.15.181:0/1309871564","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8f219c476cb4f26fc5719c661c0323719a45cb30","ceph_version":"ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)","entity_id":"x","hostname":"smithi181","pid":"28823","root":"/"}}] 2024-09-10T23:34:31.757 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi181", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28823", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.181:0", "nonce": 1309871564, "type": "any" }, "name": { "num": 5377, "type": "client" } }, "id": 5377, "inst": "client.5377 172.21.15.181:0/1309871564", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.388226161595412 }, "state": "open", "uptime": 12.21564207 } ] 2024-09-10T23:34:31.757 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T23:34:34.643 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:34:34.646 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:34:36.758 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:34:36.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.899+0000 7f29b9000640 1 -- 172.21.15.94:0/718261401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29b415ee80 msgr2=0x7f29b406f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:36.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.899+0000 7f29b9000640 1 --2- 172.21.15.94:0/718261401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29b415ee80 0x7f29b406f4b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009840 tx=0x7f29ac01c820 comp rx=0 tx=0).stop 2024-09-10T23:34:36.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.899+0000 7f29b9000640 1 -- 172.21.15.94:0/718261401 shutdown_connections 2024-09-10T23:34:36.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.899+0000 7f29b9000640 1 --2- 172.21.15.94:0/718261401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29b415ee80 0x7f29b406f4b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:36.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.899+0000 7f29b9000640 1 --2- 172.21.15.94:0/718261401 >> v2:172.21.15.181:3300/0 conn(0x7f29b4073ff0 0x7f29b40cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:36.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.899+0000 7f29b9000640 1 -- 172.21.15.94:0/718261401 >> 172.21.15.94:0/718261401 conn(0x7f29b406cdf0 msgr2=0x7f29b406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:36.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.899+0000 7f29b9000640 1 -- 172.21.15.94:0/718261401 shutdown_connections 2024-09-10T23:34:36.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.899+0000 7f29b9000640 1 -- 172.21.15.94:0/718261401 wait complete. 2024-09-10T23:34:36.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.899+0000 7f29b9000640 1 Processor -- start 2024-09-10T23:34:36.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29b9000640 1 -- start start 2024-09-10T23:34:36.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29b9000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29b4073ff0 0x7f29b40b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29b9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29b40d02e0 0x7f29b40b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29b9000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29b415ee80 0x7f29b415def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29b9000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f29b4167b40 con 0x7f29b40d02e0 2024-09-10T23:34:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29b9000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f29b41679c0 con 0x7f29b415ee80 2024-09-10T23:34:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29b9000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f29b4167cc0 con 0x7f29b4073ff0 2024-09-10T23:34:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29a9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29b40d02e0 0x7f29b40b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29a9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29b40d02e0 0x7f29b40b12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59284/0 (socket says 172.21.15.94:59284) 2024-09-10T23:34:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29a9400640 1 -- 172.21.15.94:0/3890489436 learned_addr learned my addr 172.21.15.94:0/3890489436 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29b1e00640 1 --2- 172.21.15.94:0/3890489436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29b4073ff0 0x7f29b40b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29b2800640 1 --2- 172.21.15.94:0/3890489436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29b415ee80 0x7f29b415def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29a9400640 1 -- 172.21.15.94:0/3890489436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29b4073ff0 msgr2=0x7f29b40b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:36.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29a9400640 1 --2- 172.21.15.94:0/3890489436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29b4073ff0 0x7f29b40b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29a9400640 1 -- 172.21.15.94:0/3890489436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29b415ee80 msgr2=0x7f29b415def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29a9400640 1 --2- 172.21.15.94:0/3890489436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29b415ee80 0x7f29b415def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29a9400640 1 -- 172.21.15.94:0/3890489436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b40b17e0 con 0x7f29b40d02e0 2024-09-10T23:34:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29b1e00640 1 --2- 172.21.15.94:0/3890489436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29b4073ff0 0x7f29b40b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:34:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29a9400640 1 --2- 172.21.15.94:0/3890489436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29b40d02e0 0x7f29b40b12a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f299c00c530 tx=0x7f299c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29aaa00640 1 -- 172.21.15.94:0/3890489436 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c017070 con 0x7f29b40d02e0 2024-09-10T23:34:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29aaa00640 1 -- 172.21.15.94:0/3890489436 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f299c003600 con 0x7f29b40d02e0 2024-09-10T23:34:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29aaa00640 1 -- 172.21.15.94:0/3890489436 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c003a50 con 0x7f29b40d02e0 2024-09-10T23:34:36.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29b9000640 1 -- 172.21.15.94:0/3890489436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b4154d30 con 0x7f29b40d02e0 2024-09-10T23:34:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29b9000640 1 -- 172.21.15.94:0/3890489436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b40b0aa0 con 0x7f29b40d02e0 2024-09-10T23:34:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.903+0000 7f29b9000640 1 -- 172.21.15.94:0/3890489436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b40cfda0 con 0x7f29b40d02e0 2024-09-10T23:34:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.907+0000 7f29aaa00640 1 -- 172.21.15.94:0/3890489436 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 96598+0+0 (secure 0 0 0) 0x7f299c003bf0 con 0x7f29b40d02e0 2024-09-10T23:34:36.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.907+0000 7f29aaa00640 1 --2- 172.21.15.94:0/3890489436 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2990072820 0x7f2990074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:36.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.907+0000 7f29aaa00640 1 -- 172.21.15.94:0/3890489436 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7f299c092e80 con 0x7f29b40d02e0 2024-09-10T23:34:36.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.907+0000 7f29b1e00640 1 --2- 172.21.15.94:0/3890489436 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2990072820 0x7f2990074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:36.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.907+0000 7f29b1e00640 1 --2- 172.21.15.94:0/3890489436 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2990072820 0x7f2990074ce0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009980 tx=0x7f29a4009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:36.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:36.911+0000 7f29aaa00640 1 -- 172.21.15.94:0/3890489436 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f299c05dac0 con 0x7f29b40d02e0 2024-09-10T23:34:37.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:37.107+0000 7f29b9000640 1 -- 172.21.15.94:0/3890489436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29b4069850 con 0x7f29b40d02e0 2024-09-10T23:34:37.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:37.107+0000 7f29aaa00640 1 -- 172.21.15.94:0/3890489436 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7f299c0624f0 con 0x7f29b40d02e0 2024-09-10T23:34:37.113 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:37.114 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:34:37.114 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:34:37.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:37.111+0000 7f29b9000640 1 -- 172.21.15.94:0/3890489436 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2990072820 msgr2=0x7f2990074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:37.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:37.111+0000 7f29b9000640 1 --2- 172.21.15.94:0/3890489436 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2990072820 0x7f2990074ce0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009980 tx=0x7f29a4009500 comp rx=0 tx=0).stop 2024-09-10T23:34:37.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:37.111+0000 7f29b9000640 1 -- 172.21.15.94:0/3890489436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29b40d02e0 msgr2=0x7f29b40b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:37.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:37.111+0000 7f29b9000640 1 --2- 172.21.15.94:0/3890489436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29b40d02e0 0x7f29b40b12a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f299c00c530 tx=0x7f299c00c9f0 comp rx=0 tx=0).stop 2024-09-10T23:34:37.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:37.111+0000 7f29b9000640 1 -- 172.21.15.94:0/3890489436 shutdown_connections 2024-09-10T23:34:37.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:37.111+0000 7f29b9000640 1 --2- 172.21.15.94:0/3890489436 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2990072820 0x7f2990074ce0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:37.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:37.111+0000 7f29b9000640 1 --2- 172.21.15.94:0/3890489436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29b415ee80 0x7f29b415def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:37.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:37.111+0000 7f29b9000640 1 --2- 172.21.15.94:0/3890489436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29b40d02e0 0x7f29b40b12a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:37.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:37.111+0000 7f29b9000640 1 --2- 172.21.15.94:0/3890489436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29b4073ff0 0x7f29b40b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:37.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:37.111+0000 7f29b9000640 1 -- 172.21.15.94:0/3890489436 >> 172.21.15.94:0/3890489436 conn(0x7f29b406cdf0 msgr2=0x7f29b406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:37.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:37.111+0000 7f29b9000640 1 -- 172.21.15.94:0/3890489436 shutdown_connections 2024-09-10T23:34:37.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:37.111+0000 7f29b9000640 1 -- 172.21.15.94:0/3890489436 wait complete. 2024-09-10T23:34:37.135 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:34:37.305 INFO:teuthology.orchestra.run.smithi181.stdout:[{"id":5377,"entity":{"name":{"type":"client","num":5377},"addr":{"type":"any","addr":"172.21.15.181:0","nonce":1309871564}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":17.763507280999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.115815317465231,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5377 172.21.15.181:0/1309871564","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8f219c476cb4f26fc5719c661c0323719a45cb30","ceph_version":"ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)","entity_id":"x","hostname":"smithi181","pid":"28823","root":"/"}}] 2024-09-10T23:34:37.306 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi181", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28823", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.181:0", "nonce": 1309871564, "type": "any" }, "name": { "num": 5377, "type": "client" } }, "id": 5377, "inst": "client.5377 172.21.15.181:0/1309871564", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.11581531746523 }, "state": "open", "uptime": 17.763507281 } ] 2024-09-10T23:34:37.306 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T23:34:42.306 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:34:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.455+0000 7fccc2e00640 1 -- 172.21.15.94:0/2300957232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccbc0ccaa0 msgr2=0x7fccbc15c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.455+0000 7fccc2e00640 1 --2- 172.21.15.94:0/2300957232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccbc0ccaa0 0x7fccbc15c5f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009840 tx=0x7fcca801c790 comp rx=0 tx=0).stop 2024-09-10T23:34:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.455+0000 7fccc2e00640 1 -- 172.21.15.94:0/2300957232 shutdown_connections 2024-09-10T23:34:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.455+0000 7fccc2e00640 1 --2- 172.21.15.94:0/2300957232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccbc0ccaa0 0x7fccbc15c5f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.455+0000 7fccc2e00640 1 --2- 172.21.15.94:0/2300957232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccbc15c210 0x7fccbc068990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.455+0000 7fccc2e00640 1 -- 172.21.15.94:0/2300957232 >> 172.21.15.94:0/2300957232 conn(0x7fccbc0d0120 msgr2=0x7fccbc0d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.455+0000 7fccc2e00640 1 -- 172.21.15.94:0/2300957232 shutdown_connections 2024-09-10T23:34:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.455+0000 7fccc2e00640 1 -- 172.21.15.94:0/2300957232 wait complete. 2024-09-10T23:34:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.455+0000 7fccc2e00640 1 Processor -- start 2024-09-10T23:34:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.455+0000 7fccc2e00640 1 -- start start 2024-09-10T23:34:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc2e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccbc0ccaa0 0x7fccbc14c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc2e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccbc15c210 0x7fccbc158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc2e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccbc15cb30 0x7fccbc158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc2e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc0711b0 con 0x7fccbc15c210 2024-09-10T23:34:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc2e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc071030 con 0x7fccbc15cb30 2024-09-10T23:34:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc2e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fccbc071330 con 0x7fccbc0ccaa0 2024-09-10T23:34:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc1000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccbc15c210 0x7fccbc158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc1000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccbc15c210 0x7fccbc158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59312/0 (socket says 172.21.15.94:59312) 2024-09-10T23:34:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc1000640 1 -- 172.21.15.94:0/185324839 learned_addr learned my addr 172.21.15.94:0/185324839 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc1a00640 1 --2- 172.21.15.94:0/185324839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccbc0ccaa0 0x7fccbc14c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc2400640 1 --2- 172.21.15.94:0/185324839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccbc15cb30 0x7fccbc158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc1000640 1 -- 172.21.15.94:0/185324839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccbc0ccaa0 msgr2=0x7fccbc14c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc1000640 1 --2- 172.21.15.94:0/185324839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccbc0ccaa0 0x7fccbc14c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc1000640 1 -- 172.21.15.94:0/185324839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccbc15cb30 msgr2=0x7fccbc158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc1000640 1 --2- 172.21.15.94:0/185324839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccbc15cb30 0x7fccbc158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc1000640 1 -- 172.21.15.94:0/185324839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccbc0af600 con 0x7fccbc15c210 2024-09-10T23:34:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc1000640 1 --2- 172.21.15.94:0/185324839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccbc15c210 0x7fccbc158a00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009790 tx=0x7fcca80a5540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccb9600640 1 -- 172.21.15.94:0/185324839 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca80b2040 con 0x7fccbc15c210 2024-09-10T23:34:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccb9600640 1 -- 172.21.15.94:0/185324839 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcca80043b0 con 0x7fccbc15c210 2024-09-10T23:34:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccb9600640 1 -- 172.21.15.94:0/185324839 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8005280 con 0x7fccbc15c210 2024-09-10T23:34:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc2e00640 1 -- 172.21.15.94:0/185324839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccbc15add0 con 0x7fccbc15c210 2024-09-10T23:34:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc2e00640 1 -- 172.21.15.94:0/185324839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccbc15b670 con 0x7fccbc15c210 2024-09-10T23:34:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.459+0000 7fccc2e00640 1 -- 172.21.15.94:0/185324839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccbc15c5f0 con 0x7fccbc15c210 2024-09-10T23:34:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.463+0000 7fccb9600640 1 -- 172.21.15.94:0/185324839 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 96598+0+0 (secure 0 0 0) 0x7fcca80b5070 con 0x7fccbc15c210 2024-09-10T23:34:42.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.463+0000 7fccb9600640 1 --2- 172.21.15.94:0/185324839 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcc9c072820 0x7fcc9c074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:42.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.463+0000 7fccb9600640 1 -- 172.21.15.94:0/185324839 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7fcca812cf70 con 0x7fccbc15c210 2024-09-10T23:34:42.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.463+0000 7fccc1a00640 1 --2- 172.21.15.94:0/185324839 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcc9c072820 0x7fcc9c074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.463+0000 7fccc1a00640 1 --2- 172.21.15.94:0/185324839 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcc9c072820 0x7fcc9c074ce0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009be0 tx=0x7fccb00091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:42.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.467+0000 7fccb9600640 1 -- 172.21.15.94:0/185324839 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fccbc0af600 con 0x7fccbc15c210 2024-09-10T23:34:42.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.663+0000 7fccc2e00640 1 -- 172.21.15.94:0/185324839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccbc073d90 con 0x7fccbc15c210 2024-09-10T23:34:42.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.663+0000 7fccb9600640 1 -- 172.21.15.94:0/185324839 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7fcca8016410 con 0x7fccbc15c210 2024-09-10T23:34:42.670 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:42.671 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:34:42.671 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:34:42.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.667+0000 7fccc2e00640 1 -- 172.21.15.94:0/185324839 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcc9c072820 msgr2=0x7fcc9c074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:42.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.667+0000 7fccc2e00640 1 --2- 172.21.15.94:0/185324839 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcc9c072820 0x7fcc9c074ce0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009be0 tx=0x7fccb00091e0 comp rx=0 tx=0).stop 2024-09-10T23:34:42.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.667+0000 7fccc2e00640 1 -- 172.21.15.94:0/185324839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccbc15c210 msgr2=0x7fccbc158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:42.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.667+0000 7fccc2e00640 1 --2- 172.21.15.94:0/185324839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccbc15c210 0x7fccbc158a00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009790 tx=0x7fcca80a5540 comp rx=0 tx=0).stop 2024-09-10T23:34:42.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.667+0000 7fccc2e00640 1 -- 172.21.15.94:0/185324839 shutdown_connections 2024-09-10T23:34:42.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.667+0000 7fccc2e00640 1 --2- 172.21.15.94:0/185324839 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fcc9c072820 0x7fcc9c074ce0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:42.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.667+0000 7fccc2e00640 1 --2- 172.21.15.94:0/185324839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccbc15cb30 0x7fccbc158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:42.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.667+0000 7fccc2e00640 1 --2- 172.21.15.94:0/185324839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccbc15c210 0x7fccbc158a00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:42.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.667+0000 7fccc2e00640 1 --2- 172.21.15.94:0/185324839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccbc0ccaa0 0x7fccbc14c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:42.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.667+0000 7fccc2e00640 1 -- 172.21.15.94:0/185324839 >> 172.21.15.94:0/185324839 conn(0x7fccbc0d0120 msgr2=0x7fccbc14ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:42.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.667+0000 7fccc2e00640 1 -- 172.21.15.94:0/185324839 shutdown_connections 2024-09-10T23:34:42.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:42.667+0000 7fccc2e00640 1 -- 172.21.15.94:0/185324839 wait complete. 2024-09-10T23:34:42.691 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:34:42.862 INFO:teuthology.orchestra.run.smithi181.stdout:[{"id":5377,"entity":{"name":{"type":"client","num":5377},"addr":{"type":"any","addr":"172.21.15.181:0","nonce":1309871564}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":23.319386529999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.84648802561113,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5377 172.21.15.181:0/1309871564","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8f219c476cb4f26fc5719c661c0323719a45cb30","ceph_version":"ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)","entity_id":"x","hostname":"smithi181","pid":"28823","root":"/"}}] 2024-09-10T23:34:42.862 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi181", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28823", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.181:0", "nonce": 1309871564, "type": "any" }, "name": { "num": 5377, "type": "client" } }, "id": 5377, "inst": "client.5377 172.21.15.181:0/1309871564", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.84648802561113 }, "state": "open", "uptime": 23.31938653 } ] 2024-09-10T23:34:42.862 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T23:34:47.863 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:34:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.011+0000 7f8025e00640 1 -- 172.21.15.94:0/1378665304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8004003a60 msgr2=0x7f8004003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.011+0000 7f8025e00640 1 --2- 172.21.15.94:0/1378665304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8004003a60 0x7f8004003e80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8010009ee0 tx=0x7f801001d5f0 comp rx=0 tx=0).stop 2024-09-10T23:34:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.011+0000 7f8025e00640 1 -- 172.21.15.94:0/1378665304 shutdown_connections 2024-09-10T23:34:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.011+0000 7f8025e00640 1 --2- 172.21.15.94:0/1378665304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8004003a60 0x7f8004003e80 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.011+0000 7f8025e00640 1 -- 172.21.15.94:0/1378665304 >> 172.21.15.94:0/1378665304 conn(0x7f802006cdf0 msgr2=0x7f802006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.011+0000 7f8025e00640 1 -- 172.21.15.94:0/1378665304 shutdown_connections 2024-09-10T23:34:48.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.011+0000 7f8025e00640 1 -- 172.21.15.94:0/1378665304 wait complete. 2024-09-10T23:34:48.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.011+0000 7f8025e00640 1 Processor -- start 2024-09-10T23:34:48.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.011+0000 7f8025e00640 1 -- start start 2024-09-10T23:34:48.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f8025e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8004003a60 0x7f80200aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:48.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f8025e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80201fa1a0 0x7f80201f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:48.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f8025e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200cb250 0x7f80200cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:48.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f8025e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f80200b4190 con 0x7f80200cb250 2024-09-10T23:34:48.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f8025e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f80200b4010 con 0x7f8004003a60 2024-09-10T23:34:48.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f8025e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f80200b4310 con 0x7f80201fa1a0 2024-09-10T23:34:48.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f801f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200cb250 0x7f80200cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:48.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f801f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200cb250 0x7f80200cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35372/0 (socket says 172.21.15.94:35372) 2024-09-10T23:34:48.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f801f400640 1 -- 172.21.15.94:0/1491215871 learned_addr learned my addr 172.21.15.94:0/1491215871 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:48.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f801e000640 1 --2- 172.21.15.94:0/1491215871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80201fa1a0 0x7f80201f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:48.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f801ea00640 1 --2- 172.21.15.94:0/1491215871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8004003a60 0x7f80200aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:48.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f801f400640 1 -- 172.21.15.94:0/1491215871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80201fa1a0 msgr2=0x7f80201f97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:48.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f801f400640 1 --2- 172.21.15.94:0/1491215871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80201fa1a0 0x7f80201f97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:48.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f801f400640 1 -- 172.21.15.94:0/1491215871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8004003a60 msgr2=0x7f80200aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:48.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f801f400640 1 --2- 172.21.15.94:0/1491215871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8004003a60 0x7f80200aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:48.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f801f400640 1 -- 172.21.15.94:0/1491215871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80200ac530 con 0x7f80200cb250 2024-09-10T23:34:48.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f801e000640 1 --2- 172.21.15.94:0/1491215871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80201fa1a0 0x7f80201f97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:34:48.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f801f400640 1 --2- 172.21.15.94:0/1491215871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200cb250 0x7f80200cb6c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f802006e720 tx=0x7f8018001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:48.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f8017400640 1 -- 172.21.15.94:0/1491215871 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018011630 con 0x7f80200cb250 2024-09-10T23:34:48.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f8017400640 1 -- 172.21.15.94:0/1491215871 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8018018070 con 0x7f80200cb250 2024-09-10T23:34:48.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f8017400640 1 -- 172.21.15.94:0/1491215871 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018011a70 con 0x7f80200cb250 2024-09-10T23:34:48.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f8025e00640 1 -- 172.21.15.94:0/1491215871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80200ac7d0 con 0x7f80200cb250 2024-09-10T23:34:48.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f8025e00640 1 -- 172.21.15.94:0/1491215871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80200acd90 con 0x7f80200cb250 2024-09-10T23:34:48.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.015+0000 7f8025e00640 1 -- 172.21.15.94:0/1491215871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f802006f690 con 0x7f80200cb250 2024-09-10T23:34:48.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.019+0000 7f8017400640 1 -- 172.21.15.94:0/1491215871 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 96598+0+0 (secure 0 0 0) 0x7f8018011c10 con 0x7f80200cb250 2024-09-10T23:34:48.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.019+0000 7f8017400640 1 --2- 172.21.15.94:0/1491215871 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8000072820 0x7f8000074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:48.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.019+0000 7f8017400640 1 -- 172.21.15.94:0/1491215871 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7f8018093ed0 con 0x7f80200cb250 2024-09-10T23:34:48.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.019+0000 7f801ea00640 1 --2- 172.21.15.94:0/1491215871 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8000072820 0x7f8000074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:48.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.019+0000 7f801ea00640 1 --2- 172.21.15.94:0/1491215871 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8000072820 0x7f8000074ce0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f80100b0000 tx=0x7f801001a390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:48.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.023+0000 7f8017400640 1 -- 172.21.15.94:0/1491215871 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f801805f010 con 0x7f80200cb250 2024-09-10T23:34:48.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.223+0000 7f8025e00640 1 -- 172.21.15.94:0/1491215871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80200cd360 con 0x7f80200cb250 2024-09-10T23:34:48.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.227+0000 7f8017400640 1 -- 172.21.15.94:0/1491215871 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7f801805e910 con 0x7f80200cb250 2024-09-10T23:34:48.231 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:48.231 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:34:48.231 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:34:48.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.227+0000 7f8025e00640 1 -- 172.21.15.94:0/1491215871 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8000072820 msgr2=0x7f8000074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:48.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.227+0000 7f8025e00640 1 --2- 172.21.15.94:0/1491215871 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8000072820 0x7f8000074ce0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f80100b0000 tx=0x7f801001a390 comp rx=0 tx=0).stop 2024-09-10T23:34:48.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.227+0000 7f8025e00640 1 -- 172.21.15.94:0/1491215871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200cb250 msgr2=0x7f80200cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:48.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.227+0000 7f8025e00640 1 --2- 172.21.15.94:0/1491215871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200cb250 0x7f80200cb6c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f802006e720 tx=0x7f8018001640 comp rx=0 tx=0).stop 2024-09-10T23:34:48.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.227+0000 7f8025e00640 1 -- 172.21.15.94:0/1491215871 shutdown_connections 2024-09-10T23:34:48.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.227+0000 7f8025e00640 1 --2- 172.21.15.94:0/1491215871 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8000072820 0x7f8000074ce0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:48.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.227+0000 7f8025e00640 1 --2- 172.21.15.94:0/1491215871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200cb250 0x7f80200cb6c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:48.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.227+0000 7f8025e00640 1 --2- 172.21.15.94:0/1491215871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80201fa1a0 0x7f80201f97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:48.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.227+0000 7f8025e00640 1 --2- 172.21.15.94:0/1491215871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8004003a60 0x7f80200aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:48.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.227+0000 7f8025e00640 1 -- 172.21.15.94:0/1491215871 >> 172.21.15.94:0/1491215871 conn(0x7f802006cdf0 msgr2=0x7f80200b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:48.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.227+0000 7f8025e00640 1 -- 172.21.15.94:0/1491215871 shutdown_connections 2024-09-10T23:34:48.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:48.227+0000 7f8025e00640 1 -- 172.21.15.94:0/1491215871 wait complete. 2024-09-10T23:34:48.252 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:34:48.421 INFO:teuthology.orchestra.run.smithi181.stdout:[{"id":5377,"entity":{"name":{"type":"client","num":5377},"addr":{"type":"any","addr":"172.21.15.181:0","nonce":1309871564}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":10,"uptime":28.879276926999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.580405206139719,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5377 172.21.15.181:0/1309871564","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8f219c476cb4f26fc5719c661c0323719a45cb30","ceph_version":"ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)","entity_id":"x","hostname":"smithi181","pid":"28823","root":"/"}}] 2024-09-10T23:34:48.422 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi181", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28823", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.181:0", "nonce": 1309871564, "type": "any" }, "name": { "num": 5377, "type": "client" } }, "id": 5377, "inst": "client.5377 172.21.15.181:0/1309871564", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.58040520613972 }, "state": "open", "uptime": 28.879276927 } ] 2024-09-10T23:34:48.422 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T23:34:53.422 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:34:53.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.571+0000 7fe2c1200640 1 -- 172.21.15.94:0/2855197394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc15eca0 msgr2=0x7fe2bc06f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:53.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.571+0000 7fe2c1200640 1 --2- 172.21.15.94:0/2855197394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc15eca0 0x7fe2bc06f790 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0d0890 tx=0x7fe2b401c6e0 comp rx=0 tx=0).stop 2024-09-10T23:34:53.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.571+0000 7fe2c1200640 1 -- 172.21.15.94:0/2855197394 shutdown_connections 2024-09-10T23:34:53.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.571+0000 7fe2c1200640 1 --2- 172.21.15.94:0/2855197394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc15eca0 0x7fe2bc06f790 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:53.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.571+0000 7fe2c1200640 1 --2- 172.21.15.94:0/2855197394 >> v2:172.21.15.181:3300/0 conn(0x7fe2bc0cf3c0 0x7fe2bc0cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:53.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.571+0000 7fe2c1200640 1 -- 172.21.15.94:0/2855197394 >> 172.21.15.94:0/2855197394 conn(0x7fe2bc06cdf0 msgr2=0x7fe2bc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.571+0000 7fe2c1200640 1 -- 172.21.15.94:0/2855197394 shutdown_connections 2024-09-10T23:34:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.571+0000 7fe2c1200640 1 -- 172.21.15.94:0/2855197394 wait complete. 2024-09-10T23:34:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.571+0000 7fe2c1200640 1 Processor -- start 2024-09-10T23:34:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.571+0000 7fe2c1200640 1 -- start start 2024-09-10T23:34:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.571+0000 7fe2c1200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2bc0cf3c0 0x7fe2bc1421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.571+0000 7fe2c1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2bc0d01f0 0x7fe2bc1429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.571+0000 7fe2c1200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc15eca0 0x7fe2bc14cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.571+0000 7fe2c1200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1678a0 con 0x7fe2bc15eca0 2024-09-10T23:34:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.571+0000 7fe2c1200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc167720 con 0x7fe2bc0d01f0 2024-09-10T23:34:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.571+0000 7fe2c1200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe2bc167a20 con 0x7fe2bc0cf3c0 2024-09-10T23:34:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2ba800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc15eca0 0x7fe2bc14cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2ba800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc15eca0 0x7fe2bc14cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35398/0 (socket says 172.21.15.94:35398) 2024-09-10T23:34:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2ba800640 1 -- 172.21.15.94:0/297285184 learned_addr learned my addr 172.21.15.94:0/297285184 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2b9e00640 1 --2- 172.21.15.94:0/297285184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2bc0cf3c0 0x7fe2bc1421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2b9400640 1 --2- 172.21.15.94:0/297285184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2bc0d01f0 0x7fe2bc1429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2ba800640 1 -- 172.21.15.94:0/297285184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2bc0cf3c0 msgr2=0x7fe2bc1421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2ba800640 1 --2- 172.21.15.94:0/297285184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2bc0cf3c0 0x7fe2bc1421d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2ba800640 1 -- 172.21.15.94:0/297285184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2bc0d01f0 msgr2=0x7fe2bc1429a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2ba800640 1 --2- 172.21.15.94:0/297285184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2bc0d01f0 0x7fe2bc1429a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2ba800640 1 -- 172.21.15.94:0/297285184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2bc1433b0 con 0x7fe2bc15eca0 2024-09-10T23:34:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2ba800640 1 --2- 172.21.15.94:0/297285184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc15eca0 0x7fe2bc14cab0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40192f0 tx=0x7fe2b40a6690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2b2000640 1 -- 172.21.15.94:0/297285184 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4004030 con 0x7fe2bc15eca0 2024-09-10T23:34:53.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2b2000640 1 -- 172.21.15.94:0/297285184 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2b40041d0 con 0x7fe2bc15eca0 2024-09-10T23:34:53.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2b2000640 1 -- 172.21.15.94:0/297285184 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4005070 con 0x7fe2bc15eca0 2024-09-10T23:34:53.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2c1200640 1 -- 172.21.15.94:0/297285184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2bc143690 con 0x7fe2bc15eca0 2024-09-10T23:34:53.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2c1200640 1 -- 172.21.15.94:0/297285184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2bc152f10 con 0x7fe2bc15eca0 2024-09-10T23:34:53.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2c1200640 1 -- 172.21.15.94:0/297285184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2bc0cfcb0 con 0x7fe2bc15eca0 2024-09-10T23:34:53.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.575+0000 7fe2b2000640 1 -- 172.21.15.94:0/297285184 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 96598+0+0 (secure 0 0 0) 0x7fe2b4005210 con 0x7fe2bc15eca0 2024-09-10T23:34:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.579+0000 7fe2b2000640 1 --2- 172.21.15.94:0/297285184 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe29c072820 0x7fe29c074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.579+0000 7fe2b2000640 1 -- 172.21.15.94:0/297285184 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7fe2b412d340 con 0x7fe2bc15eca0 2024-09-10T23:34:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.579+0000 7fe2b9e00640 1 --2- 172.21.15.94:0/297285184 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe29c072820 0x7fe29c074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.579+0000 7fe2b9e00640 1 --2- 172.21.15.94:0/297285184 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe29c072820 0x7fe29c074ce0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009f60 tx=0x7fe2ac009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:53.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.583+0000 7fe2b2000640 1 -- 172.21.15.94:0/297285184 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe2b40bf070 con 0x7fe2bc15eca0 2024-09-10T23:34:53.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.783+0000 7fe2c1200640 1 -- 172.21.15.94:0/297285184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2bc15ea60 con 0x7fe2bc15eca0 2024-09-10T23:34:53.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.783+0000 7fe2b2000640 1 -- 172.21.15.94:0/297285184 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7fe2b40ac030 con 0x7fe2bc15eca0 2024-09-10T23:34:53.789 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:53.789 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:34:53.789 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:34:53.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.787+0000 7fe2c1200640 1 -- 172.21.15.94:0/297285184 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe29c072820 msgr2=0x7fe29c074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:53.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.787+0000 7fe2c1200640 1 --2- 172.21.15.94:0/297285184 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe29c072820 0x7fe29c074ce0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009f60 tx=0x7fe2ac009500 comp rx=0 tx=0).stop 2024-09-10T23:34:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.787+0000 7fe2c1200640 1 -- 172.21.15.94:0/297285184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc15eca0 msgr2=0x7fe2bc14cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.787+0000 7fe2c1200640 1 --2- 172.21.15.94:0/297285184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc15eca0 0x7fe2bc14cab0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40192f0 tx=0x7fe2b40a6690 comp rx=0 tx=0).stop 2024-09-10T23:34:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.787+0000 7fe2c1200640 1 -- 172.21.15.94:0/297285184 shutdown_connections 2024-09-10T23:34:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.787+0000 7fe2c1200640 1 --2- 172.21.15.94:0/297285184 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe29c072820 0x7fe29c074ce0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.787+0000 7fe2c1200640 1 --2- 172.21.15.94:0/297285184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2bc0d01f0 0x7fe2bc1429a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.787+0000 7fe2c1200640 1 --2- 172.21.15.94:0/297285184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc15eca0 0x7fe2bc14cab0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.787+0000 7fe2c1200640 1 --2- 172.21.15.94:0/297285184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2bc0cf3c0 0x7fe2bc1421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.787+0000 7fe2c1200640 1 -- 172.21.15.94:0/297285184 >> 172.21.15.94:0/297285184 conn(0x7fe2bc06cdf0 msgr2=0x7fe2bc1464f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.787+0000 7fe2c1200640 1 -- 172.21.15.94:0/297285184 shutdown_connections 2024-09-10T23:34:53.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:53.787+0000 7fe2c1200640 1 -- 172.21.15.94:0/297285184 wait complete. 2024-09-10T23:34:53.811 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:34:53.979 INFO:teuthology.orchestra.run.smithi181.stdout:[{"id":5377,"entity":{"name":{"type":"client","num":5377},"addr":{"type":"any","addr":"172.21.15.181:0","nonce":1309871564}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":10,"uptime":34.439177571999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.317718164311064,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5377 172.21.15.181:0/1309871564","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8f219c476cb4f26fc5719c661c0323719a45cb30","ceph_version":"ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)","entity_id":"x","hostname":"smithi181","pid":"28823","root":"/"}}] 2024-09-10T23:34:53.980 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi181", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28823", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.181:0", "nonce": 1309871564, "type": "any" }, "name": { "num": 5377, "type": "client" } }, "id": 5377, "inst": "client.5377 172.21.15.181:0/1309871564", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.317718164311064 }, "state": "open", "uptime": 34.439177572 } ] 2024-09-10T23:34:53.980 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T23:34:58.981 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:34:59.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.127+0000 7f17d3c00640 1 -- 172.21.15.94:0/206034681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17cc15bd30 msgr2=0x7f17cc15c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:59.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.127+0000 7f17d3c00640 1 --2- 172.21.15.94:0/206034681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17cc15bd30 0x7f17cc15c110 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009840 tx=0x7f17c001c820 comp rx=0 tx=0).stop 2024-09-10T23:34:59.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.127+0000 7f17d3c00640 1 -- 172.21.15.94:0/206034681 shutdown_connections 2024-09-10T23:34:59.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.127+0000 7f17d3c00640 1 --2- 172.21.15.94:0/206034681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17cc0d2290 0x7f17cc0b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:59.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.127+0000 7f17d3c00640 1 --2- 172.21.15.94:0/206034681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17cc15bd30 0x7f17cc15c110 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:59.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.127+0000 7f17d3c00640 1 -- 172.21.15.94:0/206034681 >> 172.21.15.94:0/206034681 conn(0x7f17cc09aa70 msgr2=0x7f17cc09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:59.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.127+0000 7f17d3c00640 1 -- 172.21.15.94:0/206034681 shutdown_connections 2024-09-10T23:34:59.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.127+0000 7f17d3c00640 1 -- 172.21.15.94:0/206034681 wait complete. 2024-09-10T23:34:59.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d3c00640 1 Processor -- start 2024-09-10T23:34:59.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d3c00640 1 -- start start 2024-09-10T23:34:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d3c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17cc0ae980 0x7f17cc072240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d3c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17cc0d2290 0x7f17cc0ced60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d3c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17cc15bd30 0x7f17cc06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d3c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc169500 con 0x7f17cc15bd30 2024-09-10T23:34:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d3c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc169380 con 0x7f17cc0ae980 2024-09-10T23:34:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d3c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f17cc169680 con 0x7f17cc0d2290 2024-09-10T23:34:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d1c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17cc15bd30 0x7f17cc06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d1c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17cc15bd30 0x7f17cc06dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40270/0 (socket says 172.21.15.94:40270) 2024-09-10T23:34:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d1c00640 1 -- 172.21.15.94:0/2022674871 learned_addr learned my addr 172.21.15.94:0/2022674871 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:34:59.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d1200640 1 --2- 172.21.15.94:0/2022674871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17cc0ae980 0x7f17cc072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:59.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17cbe00640 1 --2- 172.21.15.94:0/2022674871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17cc0d2290 0x7f17cc0ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:59.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d1c00640 1 -- 172.21.15.94:0/2022674871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17cc0d2290 msgr2=0x7f17cc0ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:59.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d1c00640 1 --2- 172.21.15.94:0/2022674871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17cc0d2290 0x7f17cc0ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:59.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d1c00640 1 -- 172.21.15.94:0/2022674871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17cc0ae980 msgr2=0x7f17cc072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:59.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d1c00640 1 --2- 172.21.15.94:0/2022674871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17cc0ae980 0x7f17cc072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:59.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d1c00640 1 -- 172.21.15.94:0/2022674871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc0cf2a0 con 0x7f17cc15bd30 2024-09-10T23:34:59.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d1200640 1 --2- 172.21.15.94:0/2022674871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17cc0ae980 0x7f17cc072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:34:59.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d1c00640 1 --2- 172.21.15.94:0/2022674871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17cc15bd30 0x7f17cc06dd30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f17c400cce0 tx=0x7f17c400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:59.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17c8c00640 1 -- 172.21.15.94:0/2022674871 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c400f450 con 0x7f17cc15bd30 2024-09-10T23:34:59.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17c8c00640 1 -- 172.21.15.94:0/2022674871 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17c400f6e0 con 0x7f17cc15bd30 2024-09-10T23:34:59.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d3c00640 1 -- 172.21.15.94:0/2022674871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc1fb500 con 0x7f17cc15bd30 2024-09-10T23:34:59.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17c8c00640 1 -- 172.21.15.94:0/2022674871 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c400fb70 con 0x7f17cc15bd30 2024-09-10T23:34:59.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d3c00640 1 -- 172.21.15.94:0/2022674871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc0ccb70 con 0x7f17cc15bd30 2024-09-10T23:34:59.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.131+0000 7f17d3c00640 1 -- 172.21.15.94:0/2022674871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17cc0b1e70 con 0x7f17cc15bd30 2024-09-10T23:34:59.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.135+0000 7f17c8c00640 1 -- 172.21.15.94:0/2022674871 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 96598+0+0 (secure 0 0 0) 0x7f17c40213f0 con 0x7f17cc15bd30 2024-09-10T23:34:59.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.135+0000 7f17c8c00640 1 --2- 172.21.15.94:0/2022674871 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f17ac072850 0x7f17ac074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:34:59.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.135+0000 7f17c8c00640 1 -- 172.21.15.94:0/2022674871 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7f17c40943d0 con 0x7f17cc15bd30 2024-09-10T23:34:59.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.135+0000 7f17d1200640 1 --2- 172.21.15.94:0/2022674871 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f17ac072850 0x7f17ac074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:34:59.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.135+0000 7f17d1200640 1 --2- 172.21.15.94:0/2022674871 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f17ac072850 0x7f17ac074d10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f17c00af000 tx=0x7f17c001b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:34:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.139+0000 7f17c8c00640 1 -- 172.21.15.94:0/2022674871 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f17cc0cf2a0 con 0x7f17cc15bd30 2024-09-10T23:34:59.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.335+0000 7f17d3c00640 1 -- 172.21.15.94:0/2022674871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17cc1fb6d0 con 0x7f17cc15bd30 2024-09-10T23:34:59.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.335+0000 7f17c8c00640 1 -- 172.21.15.94:0/2022674871 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7f17c4014030 con 0x7f17cc15bd30 2024-09-10T23:34:59.341 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:34:59.341 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:34:59.342 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:34:59.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.339+0000 7f17d3c00640 1 -- 172.21.15.94:0/2022674871 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f17ac072850 msgr2=0x7f17ac074d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:59.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.339+0000 7f17d3c00640 1 --2- 172.21.15.94:0/2022674871 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f17ac072850 0x7f17ac074d10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f17c00af000 tx=0x7f17c001b060 comp rx=0 tx=0).stop 2024-09-10T23:34:59.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.339+0000 7f17d3c00640 1 -- 172.21.15.94:0/2022674871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17cc15bd30 msgr2=0x7f17cc06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:34:59.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.339+0000 7f17d3c00640 1 --2- 172.21.15.94:0/2022674871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17cc15bd30 0x7f17cc06dd30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f17c400cce0 tx=0x7f17c400a620 comp rx=0 tx=0).stop 2024-09-10T23:34:59.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.339+0000 7f17d3c00640 1 -- 172.21.15.94:0/2022674871 shutdown_connections 2024-09-10T23:34:59.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.339+0000 7f17d3c00640 1 --2- 172.21.15.94:0/2022674871 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f17ac072850 0x7f17ac074d10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:59.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.339+0000 7f17d3c00640 1 --2- 172.21.15.94:0/2022674871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17cc15bd30 0x7f17cc06dd30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:59.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.339+0000 7f17d3c00640 1 --2- 172.21.15.94:0/2022674871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17cc0d2290 0x7f17cc0ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:59.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.339+0000 7f17d3c00640 1 --2- 172.21.15.94:0/2022674871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17cc0ae980 0x7f17cc072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:34:59.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.339+0000 7f17d3c00640 1 -- 172.21.15.94:0/2022674871 >> 172.21.15.94:0/2022674871 conn(0x7f17cc09aa70 msgr2=0x7f17cc0afa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:34:59.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.339+0000 7f17d3c00640 1 -- 172.21.15.94:0/2022674871 shutdown_connections 2024-09-10T23:34:59.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:34:59.339+0000 7f17d3c00640 1 -- 172.21.15.94:0/2022674871 wait complete. 2024-09-10T23:34:59.362 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:34:59.533 INFO:teuthology.orchestra.run.smithi181.stdout:[{"id":5377,"entity":{"name":{"type":"client","num":5377},"addr":{"type":"any","addr":"172.21.15.181:0","nonce":1309871564}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":39.991086066999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.058754441202993,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5377 172.21.15.181:0/1309871564","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8f219c476cb4f26fc5719c661c0323719a45cb30","ceph_version":"ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)","entity_id":"x","hostname":"smithi181","pid":"28823","root":"/"}}] 2024-09-10T23:34:59.534 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi181", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28823", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.181:0", "nonce": 1309871564, "type": "any" }, "name": { "num": 5377, "type": "client" } }, "id": 5377, "inst": "client.5377 172.21.15.181:0/1309871564", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.058754441202993 }, "state": "open", "uptime": 39.991086067 } ] 2024-09-10T23:34:59.534 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T23:35:04.535 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:35:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.691+0000 7f6c7be00640 1 -- 172.21.15.94:0/4223208485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0cc860 msgr2=0x7f6c7c0ccd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.691+0000 7f6c7be00640 1 --2- 172.21.15.94:0/4223208485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0cc860 0x7f6c7c0ccd10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009c80 tx=0x7f6c7401cd40 comp rx=0 tx=0).stop 2024-09-10T23:35:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.691+0000 7f6c7be00640 1 -- 172.21.15.94:0/4223208485 shutdown_connections 2024-09-10T23:35:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.691+0000 7f6c7be00640 1 --2- 172.21.15.94:0/4223208485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0cc860 0x7f6c7c0ccd10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.691+0000 7f6c7be00640 1 --2- 172.21.15.94:0/4223208485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c7c0cfd50 0x7f6c7c06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.691+0000 7f6c7be00640 1 -- 172.21.15.94:0/4223208485 >> 172.21.15.94:0/4223208485 conn(0x7f6c7c0cd6b0 msgr2=0x7f6c7c0cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.691+0000 7f6c7be00640 1 -- 172.21.15.94:0/4223208485 shutdown_connections 2024-09-10T23:35:04.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.691+0000 7f6c7be00640 1 -- 172.21.15.94:0/4223208485 wait complete. 2024-09-10T23:35:04.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.691+0000 7f6c7be00640 1 Processor -- start 2024-09-10T23:35:04.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.691+0000 7f6c7be00640 1 -- start start 2024-09-10T23:35:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.691+0000 7f6c7be00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c7c0cc860 0x7f6c7c15e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.691+0000 7f6c7be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0cfd50 0x7f6c7c157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.691+0000 7f6c7be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c7c1560a0 0x7f6c7c0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.691+0000 7f6c7be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c168b80 con 0x7f6c7c0cfd50 2024-09-10T23:35:04.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.691+0000 7f6c7be00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c168a00 con 0x7f6c7c1560a0 2024-09-10T23:35:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.691+0000 7f6c7be00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6c7c168d00 con 0x7f6c7c0cc860 2024-09-10T23:35:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c73e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0cfd50 0x7f6c7c157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c73e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0cfd50 0x7f6c7c157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47260/0 (socket says 172.21.15.94:47260) 2024-09-10T23:35:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c73e00640 1 -- 172.21.15.94:0/1240330583 learned_addr learned my addr 172.21.15.94:0/1240330583 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c7aa00640 1 --2- 172.21.15.94:0/1240330583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c7c0cc860 0x7f6c7c15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c7b400640 1 --2- 172.21.15.94:0/1240330583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c7c1560a0 0x7f6c7c0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c73e00640 1 -- 172.21.15.94:0/1240330583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c7c0cc860 msgr2=0x7f6c7c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c73e00640 1 --2- 172.21.15.94:0/1240330583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c7c0cc860 0x7f6c7c15e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c73e00640 1 -- 172.21.15.94:0/1240330583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c7c1560a0 msgr2=0x7f6c7c0ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c73e00640 1 --2- 172.21.15.94:0/1240330583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c7c1560a0 0x7f6c7c0ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c73e00640 1 -- 172.21.15.94:0/1240330583 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c15ee50 con 0x7f6c7c0cfd50 2024-09-10T23:35:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c7b400640 1 --2- 172.21.15.94:0/1240330583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c7c1560a0 0x7f6c7c0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:35:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c73e00640 1 --2- 172.21.15.94:0/1240330583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0cfd50 0x7f6c7c157330 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400e9f0 tx=0x7f6c6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:04.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c72a00640 1 -- 172.21.15.94:0/1240330583 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6400cd80 con 0x7f6c7c0cfd50 2024-09-10T23:35:04.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c72a00640 1 -- 172.21.15.94:0/1240330583 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c64004020 con 0x7f6c7c0cfd50 2024-09-10T23:35:04.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c7be00640 1 -- 172.21.15.94:0/1240330583 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7c1fa950 con 0x7f6c7c0cfd50 2024-09-10T23:35:04.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c72a00640 1 -- 172.21.15.94:0/1240330583 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64004450 con 0x7f6c7c0cfd50 2024-09-10T23:35:04.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c7be00640 1 -- 172.21.15.94:0/1240330583 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7c1faf10 con 0x7f6c7c0cfd50 2024-09-10T23:35:04.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c7be00640 1 -- 172.21.15.94:0/1240330583 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c7c0ccd10 con 0x7f6c7c0cfd50 2024-09-10T23:35:04.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.695+0000 7f6c72a00640 1 -- 172.21.15.94:0/1240330583 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 96598+0+0 (secure 0 0 0) 0x7f6c64003600 con 0x7f6c7c0cfd50 2024-09-10T23:35:04.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.699+0000 7f6c72a00640 1 --2- 172.21.15.94:0/1240330583 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6c58072820 0x7f6c58074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:04.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.699+0000 7f6c72a00640 1 -- 172.21.15.94:0/1240330583 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7f6c64094250 con 0x7f6c7c0cfd50 2024-09-10T23:35:04.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.699+0000 7f6c7aa00640 1 --2- 172.21.15.94:0/1240330583 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6c58072820 0x7f6c58074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:04.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.699+0000 7f6c7aa00640 1 --2- 172.21.15.94:0/1240330583 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6c58072820 0x7f6c58074ce0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c15e530 tx=0x7f6c6c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:04.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.703+0000 7f6c72a00640 1 -- 172.21.15.94:0/1240330583 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6c6405f4e0 con 0x7f6c7c0cfd50 2024-09-10T23:35:04.707 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:35:04.711 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:35:04.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.911+0000 7f6c7be00640 1 -- 172.21.15.94:0/1240330583 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c7c15f0a0 con 0x7f6c7c0cfd50 2024-09-10T23:35:04.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.911+0000 7f6c72a00640 1 -- 172.21.15.94:0/1240330583 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7f6c6405ede0 con 0x7f6c7c0cfd50 2024-09-10T23:35:04.916 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:04.916 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:35:04.917 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:35:04.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.915+0000 7f6c7be00640 1 -- 172.21.15.94:0/1240330583 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6c58072820 msgr2=0x7f6c58074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.915+0000 7f6c7be00640 1 --2- 172.21.15.94:0/1240330583 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6c58072820 0x7f6c58074ce0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c15e530 tx=0x7f6c6c009040 comp rx=0 tx=0).stop 2024-09-10T23:35:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.915+0000 7f6c7be00640 1 -- 172.21.15.94:0/1240330583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0cfd50 msgr2=0x7f6c7c157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.915+0000 7f6c7be00640 1 --2- 172.21.15.94:0/1240330583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0cfd50 0x7f6c7c157330 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400e9f0 tx=0x7f6c6400eeb0 comp rx=0 tx=0).stop 2024-09-10T23:35:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.915+0000 7f6c7be00640 1 -- 172.21.15.94:0/1240330583 shutdown_connections 2024-09-10T23:35:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.915+0000 7f6c7be00640 1 --2- 172.21.15.94:0/1240330583 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6c58072820 0x7f6c58074ce0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.915+0000 7f6c7be00640 1 --2- 172.21.15.94:0/1240330583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c7c1560a0 0x7f6c7c0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.915+0000 7f6c7be00640 1 --2- 172.21.15.94:0/1240330583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0cfd50 0x7f6c7c157330 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.915+0000 7f6c7be00640 1 --2- 172.21.15.94:0/1240330583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c7c0cc860 0x7f6c7c15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.915+0000 7f6c7be00640 1 -- 172.21.15.94:0/1240330583 >> 172.21.15.94:0/1240330583 conn(0x7f6c7c0cd6b0 msgr2=0x7f6c7c06dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.915+0000 7f6c7be00640 1 -- 172.21.15.94:0/1240330583 shutdown_connections 2024-09-10T23:35:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:04.915+0000 7f6c7be00640 1 -- 172.21.15.94:0/1240330583 wait complete. 2024-09-10T23:35:04.939 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:35:05.101 INFO:teuthology.orchestra.run.smithi181.stdout:[{"id":5377,"entity":{"name":{"type":"client","num":5377},"addr":{"type":"any","addr":"172.21.15.181:0","nonce":1309871564}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":45.559000505999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.802359058219011,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5377 172.21.15.181:0/1309871564","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8f219c476cb4f26fc5719c661c0323719a45cb30","ceph_version":"ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)","entity_id":"x","hostname":"smithi181","pid":"28823","root":"/"}}] 2024-09-10T23:35:05.102 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi181", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28823", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.181:0", "nonce": 1309871564, "type": "any" }, "name": { "num": 5377, "type": "client" } }, "id": 5377, "inst": "client.5377 172.21.15.181:0/1309871564", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.80235905821901 }, "state": "open", "uptime": 45.559000506 } ] 2024-09-10T23:35:05.102 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T23:35:10.103 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:35:10.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.255+0000 7fb3e4800640 1 -- 172.21.15.94:0/3357175034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3dc068c30 msgr2=0x7fb3dc0ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:10.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.255+0000 7fb3e4800640 1 --2- 172.21.15.94:0/3357175034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3dc068c30 0x7fb3dc0ad5a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc071b30 tx=0x7fb3cc01c770 comp rx=0 tx=0).stop 2024-09-10T23:35:10.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.255+0000 7fb3e4800640 1 -- 172.21.15.94:0/3357175034 shutdown_connections 2024-09-10T23:35:10.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.255+0000 7fb3e4800640 1 --2- 172.21.15.94:0/3357175034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3dc068c30 0x7fb3dc0ad5a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:10.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.255+0000 7fb3e4800640 1 --2- 172.21.15.94:0/3357175034 >> v2:172.21.15.181:3300/0 conn(0x7fb3dc15bcc0 0x7fb3dc0ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:10.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.255+0000 7fb3e4800640 1 -- 172.21.15.94:0/3357175034 >> 172.21.15.94:0/3357175034 conn(0x7fb3dc0d02d0 msgr2=0x7fb3dc0d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:10.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.255+0000 7fb3e4800640 1 -- 172.21.15.94:0/3357175034 shutdown_connections 2024-09-10T23:35:10.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.255+0000 7fb3e4800640 1 -- 172.21.15.94:0/3357175034 wait complete. 2024-09-10T23:35:10.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.255+0000 7fb3e4800640 1 Processor -- start 2024-09-10T23:35:10.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.255+0000 7fb3e4800640 1 -- start start 2024-09-10T23:35:10.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e4800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3dc068c30 0x7fb3dc097c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:10.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e4800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3dc0cf690 0x7fb3dc0c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:10.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e4800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3dc15bcc0 0x7fb3dc099020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:10.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e4800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc1694c0 con 0x7fb3dc15bcc0 2024-09-10T23:35:10.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e4800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc169340 con 0x7fb3dc0cf690 2024-09-10T23:35:10.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e4800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb3dc169640 con 0x7fb3dc068c30 2024-09-10T23:35:10.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3dc15bcc0 0x7fb3dc099020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:10.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3dc15bcc0 0x7fb3dc099020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47280/0 (socket says 172.21.15.94:47280) 2024-09-10T23:35:10.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e2800640 1 -- 172.21.15.94:0/3596258601 learned_addr learned my addr 172.21.15.94:0/3596258601 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:10.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e1e00640 1 --2- 172.21.15.94:0/3596258601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3dc068c30 0x7fb3dc097c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:10.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e1400640 1 --2- 172.21.15.94:0/3596258601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3dc0cf690 0x7fb3dc0c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:10.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e2800640 1 -- 172.21.15.94:0/3596258601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3dc068c30 msgr2=0x7fb3dc097c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:10.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e2800640 1 --2- 172.21.15.94:0/3596258601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3dc068c30 0x7fb3dc097c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:10.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e2800640 1 -- 172.21.15.94:0/3596258601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3dc0cf690 msgr2=0x7fb3dc0c3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:10.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e2800640 1 --2- 172.21.15.94:0/3596258601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3dc0cf690 0x7fb3dc0c3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:10.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e2800640 1 -- 172.21.15.94:0/3596258601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3dc098260 con 0x7fb3dc15bcc0 2024-09-10T23:35:10.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e2800640 1 --2- 172.21.15.94:0/3596258601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3dc15bcc0 0x7fb3dc099020 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800cbc0 tx=0x7fb3d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:10.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3d6000640 1 -- 172.21.15.94:0/3596258601 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8016020 con 0x7fb3dc15bcc0 2024-09-10T23:35:10.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3d6000640 1 -- 172.21.15.94:0/3596258601 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3d800ab00 con 0x7fb3dc15bcc0 2024-09-10T23:35:10.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3d6000640 1 -- 172.21.15.94:0/3596258601 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8012420 con 0x7fb3dc15bcc0 2024-09-10T23:35:10.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e4800640 1 -- 172.21.15.94:0/3596258601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3dc098540 con 0x7fb3dc15bcc0 2024-09-10T23:35:10.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e4800640 1 -- 172.21.15.94:0/3596258601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3dc0c4340 con 0x7fb3dc15bcc0 2024-09-10T23:35:10.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3e4800640 1 -- 172.21.15.94:0/3596258601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3dc0c37c0 con 0x7fb3dc15bcc0 2024-09-10T23:35:10.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.259+0000 7fb3d6000640 1 -- 172.21.15.94:0/3596258601 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 96598+0+0 (secure 0 0 0) 0x7fb3d80125c0 con 0x7fb3dc15bcc0 2024-09-10T23:35:10.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.263+0000 7fb3d6000640 1 --2- 172.21.15.94:0/3596258601 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3c0072850 0x7fb3c0074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:10.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.263+0000 7fb3d6000640 1 -- 172.21.15.94:0/3596258601 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7fb3d8093d60 con 0x7fb3dc15bcc0 2024-09-10T23:35:10.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.263+0000 7fb3e1e00640 1 --2- 172.21.15.94:0/3596258601 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3c0072850 0x7fb3c0074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:10.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.263+0000 7fb3e1e00640 1 --2- 172.21.15.94:0/3596258601 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3c0072850 0x7fb3c0074d10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000ab40 tx=0x7fb3d000a250 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:10.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.267+0000 7fb3d6000640 1 -- 172.21.15.94:0/3596258601 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb3d8020020 con 0x7fb3dc15bcc0 2024-09-10T23:35:10.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.463+0000 7fb3e4800640 1 -- 172.21.15.94:0/3596258601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3dc0cdea0 con 0x7fb3dc15bcc0 2024-09-10T23:35:10.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.463+0000 7fb3d6000640 1 -- 172.21.15.94:0/3596258601 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7fb3d805e850 con 0x7fb3dc15bcc0 2024-09-10T23:35:10.465 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:10.466 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:35:10.466 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:35:10.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.467+0000 7fb3e4800640 1 -- 172.21.15.94:0/3596258601 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3c0072850 msgr2=0x7fb3c0074d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:10.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.467+0000 7fb3e4800640 1 --2- 172.21.15.94:0/3596258601 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3c0072850 0x7fb3c0074d10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000ab40 tx=0x7fb3d000a250 comp rx=0 tx=0).stop 2024-09-10T23:35:10.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.467+0000 7fb3e4800640 1 -- 172.21.15.94:0/3596258601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3dc15bcc0 msgr2=0x7fb3dc099020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:10.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.467+0000 7fb3e4800640 1 --2- 172.21.15.94:0/3596258601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3dc15bcc0 0x7fb3dc099020 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb3d800cbc0 tx=0x7fb3d800ccb0 comp rx=0 tx=0).stop 2024-09-10T23:35:10.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.467+0000 7fb3e4800640 1 -- 172.21.15.94:0/3596258601 shutdown_connections 2024-09-10T23:35:10.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.467+0000 7fb3e4800640 1 --2- 172.21.15.94:0/3596258601 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb3c0072850 0x7fb3c0074d10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:10.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.467+0000 7fb3e4800640 1 --2- 172.21.15.94:0/3596258601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3dc0cf690 0x7fb3dc0c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:10.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.467+0000 7fb3e4800640 1 --2- 172.21.15.94:0/3596258601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3dc15bcc0 0x7fb3dc099020 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:10.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.467+0000 7fb3e4800640 1 --2- 172.21.15.94:0/3596258601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3dc068c30 0x7fb3dc097c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:10.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.467+0000 7fb3e4800640 1 -- 172.21.15.94:0/3596258601 >> 172.21.15.94:0/3596258601 conn(0x7fb3dc0d02d0 msgr2=0x7fb3dc15a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:10.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.467+0000 7fb3e4800640 1 -- 172.21.15.94:0/3596258601 shutdown_connections 2024-09-10T23:35:10.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:10.467+0000 7fb3e4800640 1 -- 172.21.15.94:0/3596258601 wait complete. 2024-09-10T23:35:10.487 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:35:10.658 INFO:teuthology.orchestra.run.smithi181.stdout:[{"id":5377,"entity":{"name":{"type":"client","num":5377},"addr":{"type":"any","addr":"172.21.15.181:0","nonce":1309871564}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":51.118922052999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.54960200603146,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5377 172.21.15.181:0/1309871564","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8f219c476cb4f26fc5719c661c0323719a45cb30","ceph_version":"ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)","entity_id":"x","hostname":"smithi181","pid":"28823","root":"/"}}] 2024-09-10T23:35:10.659 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi181", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28823", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.181:0", "nonce": 1309871564, "type": "any" }, "name": { "num": 5377, "type": "client" } }, "id": 5377, "inst": "client.5377 172.21.15.181:0/1309871564", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.54960200603146 }, "state": "open", "uptime": 51.118922053 } ] 2024-09-10T23:35:10.659 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T23:35:15.659 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:35:15.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.811+0000 7fdad4e00640 1 -- 172.21.15.94:0/4140575824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad00cfda0 msgr2=0x7fdad00ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:15.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.811+0000 7fdad4e00640 1 --2- 172.21.15.94:0/4140575824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad00cfda0 0x7fdad00ad5a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdad0071b30 tx=0x7fdac801c770 comp rx=0 tx=0).stop 2024-09-10T23:35:15.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.811+0000 7fdad4e00640 1 -- 172.21.15.94:0/4140575824 shutdown_connections 2024-09-10T23:35:15.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.811+0000 7fdad4e00640 1 --2- 172.21.15.94:0/4140575824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad00cfda0 0x7fdad00ad5a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:15.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.811+0000 7fdad4e00640 1 --2- 172.21.15.94:0/4140575824 >> v2:172.21.15.181:3300/0 conn(0x7fdad015bcc0 0x7fdad00ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:15.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.811+0000 7fdad4e00640 1 -- 172.21.15.94:0/4140575824 >> 172.21.15.94:0/4140575824 conn(0x7fdad00d02d0 msgr2=0x7fdad00d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:15.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.811+0000 7fdad4e00640 1 -- 172.21.15.94:0/4140575824 shutdown_connections 2024-09-10T23:35:15.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.811+0000 7fdad4e00640 1 -- 172.21.15.94:0/4140575824 wait complete. 2024-09-10T23:35:15.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.811+0000 7fdad4e00640 1 Processor -- start 2024-09-10T23:35:15.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.811+0000 7fdad4e00640 1 -- start start 2024-09-10T23:35:15.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.811+0000 7fdad4e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdad0068c30 0x7fdad00c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:15.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.811+0000 7fdad4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad00cfda0 0x7fdad0099390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:15.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.811+0000 7fdad4e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad015bcc0 0x7fdad00998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:15.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.811+0000 7fdad4e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdad0168b30 con 0x7fdad015bcc0 2024-09-10T23:35:15.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.811+0000 7fdad4e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdad00ce920 con 0x7fdad00cfda0 2024-09-10T23:35:15.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.811+0000 7fdad4e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fdad0168cb0 con 0x7fdad0068c30 2024-09-10T23:35:15.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdace800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad015bcc0 0x7fdad00998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:15.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdace800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad015bcc0 0x7fdad00998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56522/0 (socket says 172.21.15.94:56522) 2024-09-10T23:35:15.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdace800640 1 -- 172.21.15.94:0/499086716 learned_addr learned my addr 172.21.15.94:0/499086716 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:15.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdac5400640 1 --2- 172.21.15.94:0/499086716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad00cfda0 0x7fdad0099390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:15.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdacde00640 1 --2- 172.21.15.94:0/499086716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdad0068c30 0x7fdad00c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:15.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdace800640 1 -- 172.21.15.94:0/499086716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdad0068c30 msgr2=0x7fdad00c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:15.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdace800640 1 --2- 172.21.15.94:0/499086716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdad0068c30 0x7fdad00c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:15.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdace800640 1 -- 172.21.15.94:0/499086716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad00cfda0 msgr2=0x7fdad0099390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:15.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdace800640 1 --2- 172.21.15.94:0/499086716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad00cfda0 0x7fdad0099390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:15.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdace800640 1 -- 172.21.15.94:0/499086716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad00afed0 con 0x7fdad015bcc0 2024-09-10T23:35:15.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdac5400640 1 --2- 172.21.15.94:0/499086716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad00cfda0 0x7fdad0099390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:35:15.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdacde00640 1 --2- 172.21.15.94:0/499086716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdad0068c30 0x7fdad00c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:35:15.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdace800640 1 --2- 172.21.15.94:0/499086716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad015bcc0 0x7fdad00998d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdac801cc50 tx=0x7fdac8005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:15.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdac6a00640 1 -- 172.21.15.94:0/499086716 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac80b2040 con 0x7fdad015bcc0 2024-09-10T23:35:15.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdac6a00640 1 -- 172.21.15.94:0/499086716 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdac80bb070 con 0x7fdad015bcc0 2024-09-10T23:35:15.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdad4e00640 1 -- 172.21.15.94:0/499086716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad00b01b0 con 0x7fdad015bcc0 2024-09-10T23:35:15.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdac6a00640 1 -- 172.21.15.94:0/499086716 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8003600 con 0x7fdad015bcc0 2024-09-10T23:35:15.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdad4e00640 1 -- 172.21.15.94:0/499086716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad00c40e0 con 0x7fdad015bcc0 2024-09-10T23:35:15.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdad4e00640 1 -- 172.21.15.94:0/499086716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad00cf690 con 0x7fdad015bcc0 2024-09-10T23:35:15.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.815+0000 7fdac6a00640 1 -- 172.21.15.94:0/499086716 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 96598+0+0 (secure 0 0 0) 0x7fdac80b5070 con 0x7fdad015bcc0 2024-09-10T23:35:15.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.819+0000 7fdac6a00640 1 --2- 172.21.15.94:0/499086716 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdaac072850 0x7fdaac074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:15.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.819+0000 7fdac6a00640 1 -- 172.21.15.94:0/499086716 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7fdac812d740 con 0x7fdad015bcc0 2024-09-10T23:35:15.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.819+0000 7fdacde00640 1 --2- 172.21.15.94:0/499086716 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdaac072850 0x7fdaac074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:15.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.819+0000 7fdacde00640 1 --2- 172.21.15.94:0/499086716 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdaac072850 0x7fdaac074d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009d90 tx=0x7fdac0009320 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:15.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:15.823+0000 7fdac6a00640 1 -- 172.21.15.94:0/499086716 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdac80163f0 con 0x7fdad015bcc0 2024-09-10T23:35:16.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:16.023+0000 7fdad4e00640 1 -- 172.21.15.94:0/499086716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdad006f9e0 con 0x7fdad015bcc0 2024-09-10T23:35:16.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:16.023+0000 7fdac6a00640 1 -- 172.21.15.94:0/499086716 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7fdac80f8880 con 0x7fdad015bcc0 2024-09-10T23:35:16.023 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:16.023 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:35:16.023 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:35:16.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:16.023+0000 7fdad4e00640 1 -- 172.21.15.94:0/499086716 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdaac072850 msgr2=0x7fdaac074d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:16.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:16.023+0000 7fdad4e00640 1 --2- 172.21.15.94:0/499086716 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdaac072850 0x7fdaac074d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009d90 tx=0x7fdac0009320 comp rx=0 tx=0).stop 2024-09-10T23:35:16.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:16.023+0000 7fdad4e00640 1 -- 172.21.15.94:0/499086716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad015bcc0 msgr2=0x7fdad00998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:16.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:16.023+0000 7fdad4e00640 1 --2- 172.21.15.94:0/499086716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad015bcc0 0x7fdad00998d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdac801cc50 tx=0x7fdac8005790 comp rx=0 tx=0).stop 2024-09-10T23:35:16.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:16.023+0000 7fdad4e00640 1 -- 172.21.15.94:0/499086716 shutdown_connections 2024-09-10T23:35:16.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:16.023+0000 7fdad4e00640 1 --2- 172.21.15.94:0/499086716 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdaac072850 0x7fdaac074d10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:16.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:16.023+0000 7fdad4e00640 1 --2- 172.21.15.94:0/499086716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad00cfda0 0x7fdad0099390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:16.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:16.023+0000 7fdad4e00640 1 --2- 172.21.15.94:0/499086716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad015bcc0 0x7fdad00998d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:16.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:16.023+0000 7fdad4e00640 1 --2- 172.21.15.94:0/499086716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdad0068c30 0x7fdad00c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:16.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:16.023+0000 7fdad4e00640 1 -- 172.21.15.94:0/499086716 >> 172.21.15.94:0/499086716 conn(0x7fdad00d02d0 msgr2=0x7fdad015a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:16.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:16.023+0000 7fdad4e00640 1 -- 172.21.15.94:0/499086716 shutdown_connections 2024-09-10T23:35:16.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:16.023+0000 7fdad4e00640 1 -- 172.21.15.94:0/499086716 wait complete. 2024-09-10T23:35:16.045 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:35:16.213 INFO:teuthology.orchestra.run.smithi181.stdout:[{"id":5377,"entity":{"name":{"type":"client","num":5377},"addr":{"type":"any","addr":"172.21.15.181:0","nonce":1309871564}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":56.670847881,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.300427695010633,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5377 172.21.15.181:0/1309871564","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8f219c476cb4f26fc5719c661c0323719a45cb30","ceph_version":"ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)","entity_id":"x","hostname":"smithi181","pid":"28823","root":"/"}}] 2024-09-10T23:35:16.214 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi181", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28823", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.181:0", "nonce": 1309871564, "type": "any" }, "name": { "num": 5377, "type": "client" } }, "id": 5377, "inst": "client.5377 172.21.15.181:0/1309871564", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.300427695010633 }, "state": "open", "uptime": 56.670847881 } ] 2024-09-10T23:35:16.214 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-10T23:35:21.214 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:35:21.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.367+0000 7f560e400640 1 -- 172.21.15.94:0/504298979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5608073ff0 msgr2=0x7f56080cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.367+0000 7f560e400640 1 --2- 172.21.15.94:0/504298979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5608073ff0 0x7f56080cfda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009840 tx=0x7f55f801c7f0 comp rx=0 tx=0).stop 2024-09-10T23:35:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.367+0000 7f560e400640 1 -- 172.21.15.94:0/504298979 shutdown_connections 2024-09-10T23:35:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.367+0000 7f560e400640 1 --2- 172.21.15.94:0/504298979 >> v2:172.21.15.181:3300/0 conn(0x7f56080d02e0 0x7f560815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.367+0000 7f560e400640 1 --2- 172.21.15.94:0/504298979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5608073ff0 0x7f56080cfda0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.367+0000 7f560e400640 1 -- 172.21.15.94:0/504298979 >> 172.21.15.94:0/504298979 conn(0x7f560806cdf0 msgr2=0x7f560806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.367+0000 7f560e400640 1 -- 172.21.15.94:0/504298979 shutdown_connections 2024-09-10T23:35:21.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.367+0000 7f560e400640 1 -- 172.21.15.94:0/504298979 wait complete. 2024-09-10T23:35:21.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.367+0000 7f560e400640 1 Processor -- start 2024-09-10T23:35:21.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.367+0000 7f560e400640 1 -- start start 2024-09-10T23:35:21.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f560e400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5608073ff0 0x7f5608163db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:21.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f560e400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56080d02e0 0x7f56081443b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:21.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f560e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f560815ecb0 0x7f5608159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:21.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f560e400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5608167930 con 0x7f560815ecb0 2024-09-10T23:35:21.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f560e400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f56081677b0 con 0x7f56080d02e0 2024-09-10T23:35:21.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f560e400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5608167ab0 con 0x7f5608073ff0 2024-09-10T23:35:21.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f5607e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f560815ecb0 0x7f5608159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:21.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f5607400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5608073ff0 0x7f5608163db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:21.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f5607e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f560815ecb0 0x7f5608159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56534/0 (socket says 172.21.15.94:56534) 2024-09-10T23:35:21.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f55fea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56080d02e0 0x7f56081443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:21.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f5607e00640 1 -- 172.21.15.94:0/4244049095 learned_addr learned my addr 172.21.15.94:0/4244049095 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:21.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f5607400640 1 -- 172.21.15.94:0/4244049095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56080d02e0 msgr2=0x7f56081443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:21.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f5607400640 1 --2- 172.21.15.94:0/4244049095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56080d02e0 0x7f56081443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:21.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f5607400640 1 -- 172.21.15.94:0/4244049095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f560815ecb0 msgr2=0x7f5608159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:21.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f5607400640 1 --2- 172.21.15.94:0/4244049095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f560815ecb0 0x7f5608159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:21.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f5607400640 1 -- 172.21.15.94:0/4244049095 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5608162a30 con 0x7f5608073ff0 2024-09-10T23:35:21.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f5607e00640 1 --2- 172.21.15.94:0/4244049095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f560815ecb0 0x7f5608159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:35:21.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f5607400640 1 --2- 172.21.15.94:0/4244049095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5608073ff0 0x7f5608163db0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f55f80ac000 tx=0x7f55f80a5730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:21.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f55ffe00640 1 -- 172.21.15.94:0/4244049095 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f80b2040 con 0x7f5608073ff0 2024-09-10T23:35:21.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f55ffe00640 1 -- 172.21.15.94:0/4244049095 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55f80043b0 con 0x7f5608073ff0 2024-09-10T23:35:21.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f55ffe00640 1 -- 172.21.15.94:0/4244049095 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8003860 con 0x7f5608073ff0 2024-09-10T23:35:21.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f560e400640 1 -- 172.21.15.94:0/4244049095 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5608162ff0 con 0x7f5608073ff0 2024-09-10T23:35:21.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f560e400640 1 -- 172.21.15.94:0/4244049095 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f56081635b0 con 0x7f5608073ff0 2024-09-10T23:35:21.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.371+0000 7f560e400640 1 -- 172.21.15.94:0/4244049095 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56080cfda0 con 0x7f5608073ff0 2024-09-10T23:35:21.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.375+0000 7f55ffe00640 1 -- 172.21.15.94:0/4244049095 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 8) ==== 96598+0+0 (secure 0 0 0) 0x7f55f80b5070 con 0x7f5608073ff0 2024-09-10T23:35:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.375+0000 7f55ffe00640 1 --2- 172.21.15.94:0/4244049095 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f55e4072850 0x7f55e4074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.375+0000 7f55ffe00640 1 -- 172.21.15.94:0/4244049095 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7f55f812cd90 con 0x7f5608073ff0 2024-09-10T23:35:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.375+0000 7f55fea00640 1 --2- 172.21.15.94:0/4244049095 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f55e4072850 0x7f55e4074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.375+0000 7f55fea00640 1 --2- 172.21.15.94:0/4244049095 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f55e4072850 0x7f55e4074d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f55f00096c0 tx=0x7f55f0009260 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:21.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.379+0000 7f55ffe00640 1 -- 172.21.15.94:0/4244049095 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f55f8016410 con 0x7f5608073ff0 2024-09-10T23:35:21.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.611+0000 7f560e400640 1 -- 172.21.15.94:0/4244049095 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56080b13c0 con 0x7f5608073ff0 2024-09-10T23:35:21.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.611+0000 7f55ffe00640 1 -- 172.21.15.94:0/4244049095 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7f55f80f7910 con 0x7f5608073ff0 2024-09-10T23:35:21.611 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:21.612 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:35:21.612 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:35:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.611+0000 7f560e400640 1 -- 172.21.15.94:0/4244049095 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f55e4072850 msgr2=0x7f55e4074d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.611+0000 7f560e400640 1 --2- 172.21.15.94:0/4244049095 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f55e4072850 0x7f55e4074d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f55f00096c0 tx=0x7f55f0009260 comp rx=0 tx=0).stop 2024-09-10T23:35:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.611+0000 7f560e400640 1 -- 172.21.15.94:0/4244049095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5608073ff0 msgr2=0x7f5608163db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.611+0000 7f560e400640 1 --2- 172.21.15.94:0/4244049095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5608073ff0 0x7f5608163db0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f55f80ac000 tx=0x7f55f80a5730 comp rx=0 tx=0).stop 2024-09-10T23:35:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.611+0000 7f560e400640 1 -- 172.21.15.94:0/4244049095 shutdown_connections 2024-09-10T23:35:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.611+0000 7f560e400640 1 --2- 172.21.15.94:0/4244049095 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f55e4072850 0x7f55e4074d10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.611+0000 7f560e400640 1 --2- 172.21.15.94:0/4244049095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56080d02e0 0x7f56081443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.611+0000 7f560e400640 1 --2- 172.21.15.94:0/4244049095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f560815ecb0 0x7f5608159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.611+0000 7f560e400640 1 --2- 172.21.15.94:0/4244049095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5608073ff0 0x7f5608163db0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:21.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.611+0000 7f560e400640 1 -- 172.21.15.94:0/4244049095 >> 172.21.15.94:0/4244049095 conn(0x7f560806cdf0 msgr2=0x7f56080b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:21.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.611+0000 7f560e400640 1 -- 172.21.15.94:0/4244049095 shutdown_connections 2024-09-10T23:35:21.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:21.611+0000 7f560e400640 1 -- 172.21.15.94:0/4244049095 wait complete. 2024-09-10T23:35:21.633 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:35:21.795 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2024-09-10T23:35:21.796 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-10T23:35:21.796 DEBUG:tasks.ceph_test_case:wait_until_true: success in 55s and 0 retries 2024-09-10T23:35:21.796 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:35:21.796 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:35:21.893 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:35:21.981 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:35:21.981 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:35:21.981 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:35:21.981 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:35:21.990 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:35:21.990 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:35:21.990 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:35:21.990 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:35:21.998 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:35:21.998 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:35:21.998 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:35:21.999 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:35:22.041 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:35:22.042 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:35:22.042 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:35:22.042 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:35:22.050 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:35:22.050 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:35:22.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.231+0000 7f9f9ca00640 1 -- 172.21.15.94:0/949987484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f9806f600 msgr2=0x7f9f98072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:22.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.231+0000 7f9f9ca00640 1 --2- 172.21.15.94:0/949987484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f9806f600 0x7f9f98072610 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009840 tx=0x7f9f8001c790 comp rx=0 tx=0).stop 2024-09-10T23:35:22.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.231+0000 7f9f9ca00640 1 -- 172.21.15.94:0/949987484 shutdown_connections 2024-09-10T23:35:22.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.231+0000 7f9f9ca00640 1 --2- 172.21.15.94:0/949987484 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f98070150 0x7f9f980b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.231+0000 7f9f9ca00640 1 --2- 172.21.15.94:0/949987484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f9806f600 0x7f9f98072610 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.231+0000 7f9f9ca00640 1 -- 172.21.15.94:0/949987484 >> 172.21.15.94:0/949987484 conn(0x7f9f9806cdf0 msgr2=0x7f9f9806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:22.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.231+0000 7f9f9ca00640 1 -- 172.21.15.94:0/949987484 shutdown_connections 2024-09-10T23:35:22.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.231+0000 7f9f9ca00640 1 -- 172.21.15.94:0/949987484 wait complete. 2024-09-10T23:35:22.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.231+0000 7f9f9ca00640 1 Processor -- start 2024-09-10T23:35:22.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.231+0000 7f9f9ca00640 1 -- start start 2024-09-10T23:35:22.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f9ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f9806f600 0x7f9f980ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:22.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f9ca00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f98070150 0x7f9f98072e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:22.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f9ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f98157c60 0x7f9f9813f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:22.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f9ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9815b310 con 0x7f9f9806f600 2024-09-10T23:35:22.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f9ca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9815b190 con 0x7f9f98157c60 2024-09-10T23:35:22.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f9ca00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9f9815b490 con 0x7f9f98070150 2024-09-10T23:35:22.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f95e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f9806f600 0x7f9f980ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:22.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f96800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f98157c60 0x7f9f9813f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:22.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f8d400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f98070150 0x7f9f98072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:22.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f95e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f9806f600 0x7f9f980ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56536/0 (socket says 172.21.15.94:56536) 2024-09-10T23:35:22.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f95e00640 1 -- 172.21.15.94:0/3521511808 learned_addr learned my addr 172.21.15.94:0/3521511808 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:22.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f95e00640 1 -- 172.21.15.94:0/3521511808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f98070150 msgr2=0x7f9f98072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:22.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f95e00640 1 --2- 172.21.15.94:0/3521511808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f98070150 0x7f9f98072e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f95e00640 1 -- 172.21.15.94:0/3521511808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f98157c60 msgr2=0x7f9f9813f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:22.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f95e00640 1 --2- 172.21.15.94:0/3521511808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f98157c60 0x7f9f9813f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f95e00640 1 -- 172.21.15.94:0/3521511808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f98163e60 con 0x7f9f9806f600 2024-09-10T23:35:22.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f95e00640 1 --2- 172.21.15.94:0/3521511808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f9806f600 0x7f9f980ce0c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800cbc0 tx=0x7f9f8800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:22.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f8ea00640 1 -- 172.21.15.94:0/3521511808 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88016020 con 0x7f9f9806f600 2024-09-10T23:35:22.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f8ea00640 1 -- 172.21.15.94:0/3521511808 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f8800aae0 con 0x7f9f9806f600 2024-09-10T23:35:22.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f9ca00640 1 -- 172.21.15.94:0/3521511808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f981403e0 con 0x7f9f9806f600 2024-09-10T23:35:22.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f8ea00640 1 -- 172.21.15.94:0/3521511808 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8800ce40 con 0x7f9f9806f600 2024-09-10T23:35:22.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f9ca00640 1 -- 172.21.15.94:0/3521511808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f980cd5e0 con 0x7f9f9806f600 2024-09-10T23:35:22.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f9ca00640 1 -- 172.21.15.94:0/3521511808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f98072610 con 0x7f9f9806f600 2024-09-10T23:35:22.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.235+0000 7f9f8ea00640 1 -- 172.21.15.94:0/3521511808 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 96598+0+0 (secure 0 0 0) 0x7f9f8801a3b0 con 0x7f9f9806f600 2024-09-10T23:35:22.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.239+0000 7f9f8ea00640 1 --2- 172.21.15.94:0/3521511808 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9f74072850 0x7f9f74074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:22.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.239+0000 7f9f8ea00640 1 -- 172.21.15.94:0/3521511808 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7f9f880938f0 con 0x7f9f9806f600 2024-09-10T23:35:22.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.239+0000 7f9f8d400640 1 --2- 172.21.15.94:0/3521511808 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9f74072850 0x7f9f74074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:22.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.239+0000 7f9f8d400640 1 --2- 172.21.15.94:0/3521511808 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9f74072850 0x7f9f74074d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9f800af000 tx=0x7f9f8001afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:22.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.243+0000 7f9f8ea00640 1 -- 172.21.15.94:0/3521511808 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9f88020030 con 0x7f9f9806f600 2024-09-10T23:35:22.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.439+0000 7f9f9ca00640 1 -- 172.21.15.94:0/3521511808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f9806e7e0 con 0x7f9f9806f600 2024-09-10T23:35:22.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.439+0000 7f9f8ea00640 1 -- 172.21.15.94:0/3521511808 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7f9f8805e470 con 0x7f9f9806f600 2024-09-10T23:35:22.442 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:22.442 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:35:22.442 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:35:22.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.443+0000 7f9f9ca00640 1 -- 172.21.15.94:0/3521511808 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9f74072850 msgr2=0x7f9f74074d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:22.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.443+0000 7f9f9ca00640 1 --2- 172.21.15.94:0/3521511808 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9f74072850 0x7f9f74074d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9f800af000 tx=0x7f9f8001afd0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.443+0000 7f9f9ca00640 1 -- 172.21.15.94:0/3521511808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f9806f600 msgr2=0x7f9f980ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:22.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.443+0000 7f9f9ca00640 1 --2- 172.21.15.94:0/3521511808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f9806f600 0x7f9f980ce0c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800cbc0 tx=0x7f9f8800cbf0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.443+0000 7f9f9ca00640 1 -- 172.21.15.94:0/3521511808 shutdown_connections 2024-09-10T23:35:22.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.443+0000 7f9f9ca00640 1 --2- 172.21.15.94:0/3521511808 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9f74072850 0x7f9f74074d10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.443+0000 7f9f9ca00640 1 --2- 172.21.15.94:0/3521511808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f98157c60 0x7f9f9813f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.443+0000 7f9f9ca00640 1 --2- 172.21.15.94:0/3521511808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f98070150 0x7f9f98072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.443+0000 7f9f9ca00640 1 --2- 172.21.15.94:0/3521511808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f9806f600 0x7f9f980ce0c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.443+0000 7f9f9ca00640 1 -- 172.21.15.94:0/3521511808 >> 172.21.15.94:0/3521511808 conn(0x7f9f9806cdf0 msgr2=0x7f9f9806eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:22.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.443+0000 7f9f9ca00640 1 -- 172.21.15.94:0/3521511808 shutdown_connections 2024-09-10T23:35:22.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.443+0000 7f9f9ca00640 1 -- 172.21.15.94:0/3521511808 wait complete. 2024-09-10T23:35:22.463 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:35:22.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.603+0000 7f947f200640 1 -- 172.21.15.94:0/3364385272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f947815ee80 msgr2=0x7f947806f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:22.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.603+0000 7f947f200640 1 --2- 172.21.15.94:0/3364385272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f947815ee80 0x7f947806f4b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9470009c80 tx=0x7f947001cda0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947f200640 1 -- 172.21.15.94:0/3364385272 shutdown_connections 2024-09-10T23:35:22.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947f200640 1 --2- 172.21.15.94:0/3364385272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f947815ee80 0x7f947806f4b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947f200640 1 --2- 172.21.15.94:0/3364385272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9478073ff0 0x7f94780cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947f200640 1 -- 172.21.15.94:0/3364385272 >> 172.21.15.94:0/3364385272 conn(0x7f947806cdf0 msgr2=0x7f947806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:22.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947f200640 1 -- 172.21.15.94:0/3364385272 shutdown_connections 2024-09-10T23:35:22.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947f200640 1 -- 172.21.15.94:0/3364385272 wait complete. 2024-09-10T23:35:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947f200640 1 Processor -- start 2024-09-10T23:35:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947f200640 1 -- start start 2024-09-10T23:35:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947f200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9478073ff0 0x7f94781544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94780d02e0 0x7f947815d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f947815ee80 0x7f9478072610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9478167970 con 0x7f947815ee80 2024-09-10T23:35:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947f200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f94781677f0 con 0x7f94780d02e0 2024-09-10T23:35:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947f200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9478167af0 con 0x7f9478073ff0 2024-09-10T23:35:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f947815ee80 0x7f9478072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:22.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f946f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94780d02e0 0x7f947815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f9477e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9478073ff0 0x7f94781544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f946f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94780d02e0 0x7f947815d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:51370/0 (socket says 172.21.15.94:51370) 2024-09-10T23:35:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f946f400640 1 -- 172.21.15.94:0/747537654 learned_addr learned my addr 172.21.15.94:0/747537654 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947d200640 1 -- 172.21.15.94:0/747537654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9478073ff0 msgr2=0x7f94781544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947d200640 1 --2- 172.21.15.94:0/747537654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9478073ff0 0x7f94781544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947d200640 1 -- 172.21.15.94:0/747537654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94780d02e0 msgr2=0x7f947815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947d200640 1 --2- 172.21.15.94:0/747537654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94780d02e0 0x7f947815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f947d200640 1 -- 172.21.15.94:0/747537654 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947815c860 con 0x7f947815ee80 2024-09-10T23:35:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.607+0000 7f946f400640 1 --2- 172.21.15.94:0/747537654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94780d02e0 0x7f947815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:35:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.611+0000 7f947d200640 1 --2- 172.21.15.94:0/747537654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f947815ee80 0x7f9478072610 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f947806ed90 tx=0x7f94700a5ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.611+0000 7f9474c00640 1 -- 172.21.15.94:0/747537654 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94700bb070 con 0x7f947815ee80 2024-09-10T23:35:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.611+0000 7f9474c00640 1 -- 172.21.15.94:0/747537654 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9470004e00 con 0x7f947815ee80 2024-09-10T23:35:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.611+0000 7f947f200640 1 -- 172.21.15.94:0/747537654 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94781509d0 con 0x7f947815ee80 2024-09-10T23:35:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.611+0000 7f9474c00640 1 -- 172.21.15.94:0/747537654 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94700b2040 con 0x7f947815ee80 2024-09-10T23:35:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.611+0000 7f947f200640 1 -- 172.21.15.94:0/747537654 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94781f83e0 con 0x7f947815ee80 2024-09-10T23:35:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.611+0000 7f947f200640 1 -- 172.21.15.94:0/747537654 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94780cfda0 con 0x7f947815ee80 2024-09-10T23:35:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.611+0000 7f9474c00640 1 -- 172.21.15.94:0/747537654 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 96598+0+0 (secure 0 0 0) 0x7f94700b5030 con 0x7f947815ee80 2024-09-10T23:35:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.611+0000 7f9474c00640 1 --2- 172.21.15.94:0/747537654 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9454072820 0x7f9454074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.611+0000 7f9474c00640 1 -- 172.21.15.94:0/747537654 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7f947012d3f0 con 0x7f947815ee80 2024-09-10T23:35:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.611+0000 7f9477e00640 1 --2- 172.21.15.94:0/747537654 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9454072820 0x7f9454074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.615+0000 7f9477e00640 1 --2- 172.21.15.94:0/747537654 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9454072820 0x7f9454074ce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f946800abe0 tx=0x7f946800a2d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.615+0000 7f9474c00640 1 -- 172.21.15.94:0/747537654 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9470016990 con 0x7f947815ee80 2024-09-10T23:35:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.815+0000 7f947f200640 1 -- 172.21.15.94:0/747537654 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94781515a0 con 0x7f947815ee80 2024-09-10T23:35:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.815+0000 7f9474c00640 1 -- 172.21.15.94:0/747537654 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7f94700f7ee0 con 0x7f947815ee80 2024-09-10T23:35:22.818 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:22.818 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:35:22.820 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:35:22.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.819+0000 7f947f200640 1 -- 172.21.15.94:0/747537654 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9454072820 msgr2=0x7f9454074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:22.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.819+0000 7f947f200640 1 --2- 172.21.15.94:0/747537654 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9454072820 0x7f9454074ce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f946800abe0 tx=0x7f946800a2d0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.819+0000 7f947f200640 1 -- 172.21.15.94:0/747537654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f947815ee80 msgr2=0x7f9478072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.819+0000 7f947f200640 1 --2- 172.21.15.94:0/747537654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f947815ee80 0x7f9478072610 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f947806ed90 tx=0x7f94700a5ae0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.819+0000 7f947f200640 1 -- 172.21.15.94:0/747537654 shutdown_connections 2024-09-10T23:35:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.819+0000 7f947f200640 1 --2- 172.21.15.94:0/747537654 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9454072820 0x7f9454074ce0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.819+0000 7f947f200640 1 --2- 172.21.15.94:0/747537654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94780d02e0 0x7f947815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.819+0000 7f947f200640 1 --2- 172.21.15.94:0/747537654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f947815ee80 0x7f9478072610 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.819+0000 7f947f200640 1 --2- 172.21.15.94:0/747537654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9478073ff0 0x7f94781544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.819+0000 7f947f200640 1 -- 172.21.15.94:0/747537654 >> 172.21.15.94:0/747537654 conn(0x7f947806cdf0 msgr2=0x7f94780ce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:22.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.819+0000 7f947f200640 1 -- 172.21.15.94:0/747537654 shutdown_connections 2024-09-10T23:35:22.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.819+0000 7f947f200640 1 -- 172.21.15.94:0/747537654 wait complete. 2024-09-10T23:35:22.838 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:35:22.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.979+0000 7f73f1400640 1 -- 172.21.15.94:0/1407809902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73d0003c70 msgr2=0x7f73d0004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:22.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.979+0000 7f73f1400640 1 --2- 172.21.15.94:0/1407809902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73d0003c70 0x7f73d0004070 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f73dc006f20 tx=0x7f73dc01cea0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.979+0000 7f73f0a00640 1 -- 172.21.15.94:0/1407809902 reap_dead start 2024-09-10T23:35:22.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.979+0000 7f73f1400640 1 -- 172.21.15.94:0/1407809902 shutdown_connections 2024-09-10T23:35:22.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.979+0000 7f73f1400640 1 -- 172.21.15.94:0/1407809902 >> 172.21.15.94:0/1407809902 conn(0x7f73ec0b2520 msgr2=0x7f73ec0b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:22.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.979+0000 7f73f1400640 1 -- 172.21.15.94:0/1407809902 shutdown_connections 2024-09-10T23:35:22.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.979+0000 7f73f1400640 1 -- 172.21.15.94:0/1407809902 wait complete. 2024-09-10T23:35:22.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f1400640 1 Processor -- start 2024-09-10T23:35:22.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f1400640 1 -- start start 2024-09-10T23:35:22.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73d0003c70 0x7f73ec0af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:22.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f1400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec14c0d0 0x7f73ec15c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:22.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0d08a0 0x7f73ec0d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:22.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f1400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec167a00 con 0x7f73ec0d08a0 2024-09-10T23:35:22.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f1400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec167880 con 0x7f73d0003c70 2024-09-10T23:35:22.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f1400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f73ec167b80 con 0x7f73ec14c0d0 2024-09-10T23:35:22.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0d08a0 0x7f73ec0d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:22.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0d08a0 0x7f73ec0d0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56580/0 (socket says 172.21.15.94:56580) 2024-09-10T23:35:22.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f0a00640 1 -- 172.21.15.94:0/73046521 learned_addr learned my addr 172.21.15.94:0/73046521 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:22.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73ebe00640 1 --2- 172.21.15.94:0/73046521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73d0003c70 0x7f73ec0af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:22.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73e3400640 1 --2- 172.21.15.94:0/73046521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec14c0d0 0x7f73ec15c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:22.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f0a00640 1 -- 172.21.15.94:0/73046521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec14c0d0 msgr2=0x7f73ec15c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:22.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f0a00640 1 --2- 172.21.15.94:0/73046521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec14c0d0 0x7f73ec15c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f0a00640 1 -- 172.21.15.94:0/73046521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73d0003c70 msgr2=0x7f73ec0af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:22.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f0a00640 1 --2- 172.21.15.94:0/73046521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73d0003c70 0x7f73ec0af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:22.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f0a00640 1 -- 172.21.15.94:0/73046521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec0af610 con 0x7f73ec0d08a0 2024-09-10T23:35:22.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73e3400640 1 --2- 172.21.15.94:0/73046521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec14c0d0 0x7f73ec15c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:35:22.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f0a00640 1 --2- 172.21.15.94:0/73046521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0d08a0 0x7f73ec0d0d20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0748b0 tx=0x7f73e400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:22.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73e8c00640 1 -- 172.21.15.94:0/73046521 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4017070 con 0x7f73ec0d08a0 2024-09-10T23:35:22.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73e8c00640 1 -- 172.21.15.94:0/73046521 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73e4015070 con 0x7f73ec0d08a0 2024-09-10T23:35:22.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f1400640 1 -- 172.21.15.94:0/73046521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec149630 con 0x7f73ec0d08a0 2024-09-10T23:35:22.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73e8c00640 1 -- 172.21.15.94:0/73046521 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e400eea0 con 0x7f73ec0d08a0 2024-09-10T23:35:22.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f1400640 1 -- 172.21.15.94:0/73046521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec149bf0 con 0x7f73ec0d08a0 2024-09-10T23:35:22.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.983+0000 7f73f1400640 1 -- 172.21.15.94:0/73046521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ec06cc20 con 0x7f73ec0d08a0 2024-09-10T23:35:22.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.987+0000 7f73e8c00640 1 -- 172.21.15.94:0/73046521 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 96598+0+0 (secure 0 0 0) 0x7f73e4011440 con 0x7f73ec0d08a0 2024-09-10T23:35:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.987+0000 7f73e8c00640 1 --2- 172.21.15.94:0/73046521 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f73c8072820 0x7f73c8074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.987+0000 7f73e8c00640 1 -- 172.21.15.94:0/73046521 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7f73e4093f10 con 0x7f73ec0d08a0 2024-09-10T23:35:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.987+0000 7f73ebe00640 1 --2- 172.21.15.94:0/73046521 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f73c8072820 0x7f73c8074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.987+0000 7f73ebe00640 1 --2- 172.21.15.94:0/73046521 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f73c8072820 0x7f73c8074ce0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0ac000 tx=0x7f73dc005040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:22.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:22.991+0000 7f73e8c00640 1 -- 172.21.15.94:0/73046521 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f73e405ea00 con 0x7f73ec0d08a0 2024-09-10T23:35:23.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.155+0000 7f73f1400640 1 -- 172.21.15.94:0/73046521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73ec159990 con 0x7f73ec0d08a0 2024-09-10T23:35:23.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.159+0000 7f73e8c00640 1 -- 172.21.15.94:0/73046521 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16950 (secure 0 0 0) 0x7f73e4063430 con 0x7f73ec0d08a0 2024-09-10T23:35:23.159 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:23.159 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":46,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:34:23.122764+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:33:57.113965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T23:33:58.384504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.14","mappings":[{"from":5,"to":7}]},{"pgid":"7.1b","mappings":[{"from":5,"to":7}]},{"pgid":"7.20","mappings":[{"from":5,"to":0}]},{"pgid":"7.32","mappings":[{"from":5,"to":7}]},{"pgid":"7.33","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:35:23.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.159+0000 7f73f1400640 1 -- 172.21.15.94:0/73046521 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f73c8072820 msgr2=0x7f73c8074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:23.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.159+0000 7f73f1400640 1 --2- 172.21.15.94:0/73046521 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f73c8072820 0x7f73c8074ce0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0ac000 tx=0x7f73dc005040 comp rx=0 tx=0).stop 2024-09-10T23:35:23.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.159+0000 7f73f1400640 1 -- 172.21.15.94:0/73046521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0d08a0 msgr2=0x7f73ec0d0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:23.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.159+0000 7f73f1400640 1 --2- 172.21.15.94:0/73046521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0d08a0 0x7f73ec0d0d20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0748b0 tx=0x7f73e400e720 comp rx=0 tx=0).stop 2024-09-10T23:35:23.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.159+0000 7f73f1400640 1 -- 172.21.15.94:0/73046521 shutdown_connections 2024-09-10T23:35:23.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.159+0000 7f73f1400640 1 --2- 172.21.15.94:0/73046521 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f73c8072820 0x7f73c8074ce0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.159+0000 7f73f1400640 1 --2- 172.21.15.94:0/73046521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec0d08a0 0x7f73ec0d0d20 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.159+0000 7f73f1400640 1 --2- 172.21.15.94:0/73046521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec14c0d0 0x7f73ec15c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.159+0000 7f73f1400640 1 --2- 172.21.15.94:0/73046521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73d0003c70 0x7f73ec0af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.159+0000 7f73f1400640 1 -- 172.21.15.94:0/73046521 >> 172.21.15.94:0/73046521 conn(0x7f73ec0b2520 msgr2=0x7f73ec0ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:23.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.159+0000 7f73f1400640 1 -- 172.21.15.94:0/73046521 shutdown_connections 2024-09-10T23:35:23.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.159+0000 7f73f1400640 1 -- 172.21.15.94:0/73046521 wait complete. 2024-09-10T23:35:23.181 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T23:35:23.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:35:23.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.319+0000 7f0d6ba00640 1 -- 172.21.15.94:0/3736919424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d6406ce80 msgr2=0x7f0d640d1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:23.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.319+0000 7f0d6ba00640 1 --2- 172.21.15.94:0/3736919424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d6406ce80 0x7f0d640d1690 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0d640b4ff0 tx=0x7f0d5401c770 comp rx=0 tx=0).stop 2024-09-10T23:35:23.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.319+0000 7f0d6ba00640 1 -- 172.21.15.94:0/3736919424 shutdown_connections 2024-09-10T23:35:23.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.319+0000 7f0d6ba00640 1 --2- 172.21.15.94:0/3736919424 >> v2:172.21.15.181:3300/0 conn(0x7f0d640afec0 0x7f0d64162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.319+0000 7f0d6ba00640 1 --2- 172.21.15.94:0/3736919424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d6406ce80 0x7f0d640d1690 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.319+0000 7f0d6ba00640 1 -- 172.21.15.94:0/3736919424 >> 172.21.15.94:0/3736919424 conn(0x7f0d640d0b70 msgr2=0x7f0d640d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:23.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.319+0000 7f0d6ba00640 1 -- 172.21.15.94:0/3736919424 shutdown_connections 2024-09-10T23:35:23.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.319+0000 7f0d6ba00640 1 -- 172.21.15.94:0/3736919424 wait complete. 2024-09-10T23:35:23.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d6ba00640 1 Processor -- start 2024-09-10T23:35:23.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d6ba00640 1 -- start start 2024-09-10T23:35:23.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d6ba00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d6406ce80 0x7f0d64157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:23.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d6ba00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d640afec0 0x7f0d6414fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:23.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d6ba00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64150330 0x7f0d6413c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:23.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d6ba00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6416b190 con 0x7f0d640afec0 2024-09-10T23:35:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d6ba00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6416b010 con 0x7f0d64150330 2024-09-10T23:35:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d6ba00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0d6416b310 con 0x7f0d6406ce80 2024-09-10T23:35:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d69000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d6406ce80 0x7f0d64157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d5be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d640afec0 0x7f0d6414fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d69000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d6406ce80 0x7f0d64157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:48054/0 (socket says 172.21.15.94:48054) 2024-09-10T23:35:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d69000640 1 -- 172.21.15.94:0/212946051 learned_addr learned my addr 172.21.15.94:0/212946051 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d69a00640 1 --2- 172.21.15.94:0/212946051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64150330 0x7f0d6413c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d69000640 1 -- 172.21.15.94:0/212946051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64150330 msgr2=0x7f0d6413c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d69000640 1 --2- 172.21.15.94:0/212946051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64150330 0x7f0d6413c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d69000640 1 -- 172.21.15.94:0/212946051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d640afec0 msgr2=0x7f0d6414fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d69000640 1 --2- 172.21.15.94:0/212946051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d640afec0 0x7f0d6414fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d69000640 1 -- 172.21.15.94:0/212946051 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6414daf0 con 0x7f0d6406ce80 2024-09-10T23:35:23.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d5be00640 1 --2- 172.21.15.94:0/212946051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d640afec0 0x7f0d6414fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:35:23.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d69000640 1 --2- 172.21.15.94:0/212946051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d6406ce80 0x7f0d64157150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0d540b0040 tx=0x7f0d54004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:23.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d61600640 1 -- 172.21.15.94:0/212946051 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d540b2040 con 0x7f0d6406ce80 2024-09-10T23:35:23.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d61600640 1 -- 172.21.15.94:0/212946051 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d54005030 con 0x7f0d6406ce80 2024-09-10T23:35:23.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d61600640 1 -- 172.21.15.94:0/212946051 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54005480 con 0x7f0d6406ce80 2024-09-10T23:35:23.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d6ba00640 1 -- 172.21.15.94:0/212946051 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6414dd90 con 0x7f0d6406ce80 2024-09-10T23:35:23.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.323+0000 7f0d6ba00640 1 -- 172.21.15.94:0/212946051 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d6414e630 con 0x7f0d6406ce80 2024-09-10T23:35:23.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.327+0000 7f0d6ba00640 1 -- 172.21.15.94:0/212946051 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d640d1690 con 0x7f0d6406ce80 2024-09-10T23:35:23.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.327+0000 7f0d61600640 1 -- 172.21.15.94:0/212946051 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 8) ==== 96598+0+0 (secure 0 0 0) 0x7f0d540bf060 con 0x7f0d6406ce80 2024-09-10T23:35:23.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.327+0000 7f0d61600640 1 --2- 172.21.15.94:0/212946051 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0d40072820 0x7f0d40074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:23.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.327+0000 7f0d61600640 1 -- 172.21.15.94:0/212946051 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7f0d5412cb00 con 0x7f0d6406ce80 2024-09-10T23:35:23.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.327+0000 7f0d5be00640 1 --2- 172.21.15.94:0/212946051 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0d40072820 0x7f0d40074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:23.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.327+0000 7f0d5be00640 1 --2- 172.21.15.94:0/212946051 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0d40072820 0x7f0d40074ce0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0d640d05f0 tx=0x7f0d4c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:23.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.331+0000 7f0d61600640 1 -- 172.21.15.94:0/212946051 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0d540164b0 con 0x7f0d6406ce80 2024-09-10T23:35:23.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.531+0000 7f0d6ba00640 1 -- 172.21.15.94:0/212946051 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d64071dc0 con 0x7f0d6406ce80 2024-09-10T23:35:23.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.531+0000 7f0d61600640 1 -- 172.21.15.94:0/212946051 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7f0d540f74e0 con 0x7f0d6406ce80 2024-09-10T23:35:23.532 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:23.532 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:35:23.532 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:35:23.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.531+0000 7f0d6ba00640 1 -- 172.21.15.94:0/212946051 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0d40072820 msgr2=0x7f0d40074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:23.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.531+0000 7f0d6ba00640 1 --2- 172.21.15.94:0/212946051 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0d40072820 0x7f0d40074ce0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0d640d05f0 tx=0x7f0d4c009040 comp rx=0 tx=0).stop 2024-09-10T23:35:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.531+0000 7f0d6ba00640 1 -- 172.21.15.94:0/212946051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d6406ce80 msgr2=0x7f0d64157150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.531+0000 7f0d6ba00640 1 --2- 172.21.15.94:0/212946051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d6406ce80 0x7f0d64157150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0d540b0040 tx=0x7f0d54004040 comp rx=0 tx=0).stop 2024-09-10T23:35:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.535+0000 7f0d6ba00640 1 -- 172.21.15.94:0/212946051 shutdown_connections 2024-09-10T23:35:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.535+0000 7f0d6ba00640 1 --2- 172.21.15.94:0/212946051 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0d40072820 0x7f0d40074ce0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.535+0000 7f0d6ba00640 1 --2- 172.21.15.94:0/212946051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d64150330 0x7f0d6413c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.535+0000 7f0d6ba00640 1 --2- 172.21.15.94:0/212946051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d640afec0 0x7f0d6414fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.535+0000 7f0d6ba00640 1 --2- 172.21.15.94:0/212946051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d6406ce80 0x7f0d64157150 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.535+0000 7f0d6ba00640 1 -- 172.21.15.94:0/212946051 >> 172.21.15.94:0/212946051 conn(0x7f0d640d0b70 msgr2=0x7f0d64151c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.535+0000 7f0d6ba00640 1 -- 172.21.15.94:0/212946051 shutdown_connections 2024-09-10T23:35:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.535+0000 7f0d6ba00640 1 -- 172.21.15.94:0/212946051 wait complete. 2024-09-10T23:35:23.556 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:35:23.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabe6000640 1 -- 172.21.15.94:0/820917611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe0070150 msgr2=0x7fabe00b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:23.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabe6000640 1 --2- 172.21.15.94:0/820917611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe0070150 0x7fabe00b35c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fabe00cc090 tx=0x7fabd801de20 comp rx=0 tx=0).stop 2024-09-10T23:35:23.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabe6000640 1 -- 172.21.15.94:0/820917611 shutdown_connections 2024-09-10T23:35:23.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabe6000640 1 --2- 172.21.15.94:0/820917611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe0070150 0x7fabe00b35c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabe6000640 1 --2- 172.21.15.94:0/820917611 >> v2:172.21.15.181:3300/0 conn(0x7fabe006f600 0x7fabe0072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabe6000640 1 -- 172.21.15.94:0/820917611 >> 172.21.15.94:0/820917611 conn(0x7fabe006cdf0 msgr2=0x7fabe006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:23.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabe6000640 1 -- 172.21.15.94:0/820917611 shutdown_connections 2024-09-10T23:35:23.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabe6000640 1 -- 172.21.15.94:0/820917611 wait complete. 2024-09-10T23:35:23.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabe6000640 1 Processor -- start 2024-09-10T23:35:23.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabe6000640 1 -- start start 2024-09-10T23:35:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabe6000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe006f600 0x7fabe013f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabe6000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe0070150 0x7fabe00c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabe6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe0157c60 0x7fabe00c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabe6000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fabe015b360 con 0x7fabe0157c60 2024-09-10T23:35:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabe6000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fabe015b1e0 con 0x7fabe0070150 2024-09-10T23:35:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabe6000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fabe015b4e0 con 0x7fabe006f600 2024-09-10T23:35:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabdfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe0157c60 0x7fabe00c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabdfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe0157c60 0x7fabe00c4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56622/0 (socket says 172.21.15.94:56622) 2024-09-10T23:35:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabdfe00640 1 -- 172.21.15.94:0/953528502 learned_addr learned my addr 172.21.15.94:0/953528502 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabd6a00640 1 --2- 172.21.15.94:0/953528502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe0070150 0x7fabe00c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabdf400640 1 --2- 172.21.15.94:0/953528502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe006f600 0x7fabe013f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabdfe00640 1 -- 172.21.15.94:0/953528502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe006f600 msgr2=0x7fabe013f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabdfe00640 1 --2- 172.21.15.94:0/953528502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe006f600 0x7fabe013f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabdfe00640 1 -- 172.21.15.94:0/953528502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe0070150 msgr2=0x7fabe00c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabdfe00640 1 --2- 172.21.15.94:0/953528502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe0070150 0x7fabe00c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabdfe00640 1 -- 172.21.15.94:0/953528502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe015cf20 con 0x7fabe0157c60 2024-09-10T23:35:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabd6a00640 1 --2- 172.21.15.94:0/953528502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe0070150 0x7fabe00c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:35:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabdf400640 1 --2- 172.21.15.94:0/953528502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe006f600 0x7fabe013f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:35:23.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.699+0000 7fabdfe00640 1 --2- 172.21.15.94:0/953528502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe0157c60 0x7fabe00c4e80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fabd800ac20 tx=0x7fabd800a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.703+0000 7fabd7e00640 1 -- 172.21.15.94:0/953528502 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd80bd070 con 0x7fabe0157c60 2024-09-10T23:35:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.703+0000 7fabd7e00640 1 -- 172.21.15.94:0/953528502 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabd8004d40 con 0x7fabe0157c60 2024-09-10T23:35:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.703+0000 7fabd7e00640 1 -- 172.21.15.94:0/953528502 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd80b3040 con 0x7fabe0157c60 2024-09-10T23:35:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.703+0000 7fabe6000640 1 -- 172.21.15.94:0/953528502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe013fb20 con 0x7fabe0157c60 2024-09-10T23:35:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.703+0000 7fabe6000640 1 -- 172.21.15.94:0/953528502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe01400e0 con 0x7fabe0157c60 2024-09-10T23:35:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.703+0000 7fabe6000640 1 -- 172.21.15.94:0/953528502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe0072610 con 0x7fabe0157c60 2024-09-10T23:35:23.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.703+0000 7fabd7e00640 1 -- 172.21.15.94:0/953528502 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7fabd800b040 con 0x7fabe0157c60 2024-09-10T23:35:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.703+0000 7fabd7e00640 1 --2- 172.21.15.94:0/953528502 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fabbc0727c0 0x7fabbc074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.703+0000 7fabd7e00640 1 -- 172.21.15.94:0/953528502 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7fabd812d530 con 0x7fabe0157c60 2024-09-10T23:35:23.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.703+0000 7fabdf400640 1 --2- 172.21.15.94:0/953528502 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fabbc0727c0 0x7fabbc074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:23.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.707+0000 7fabdf400640 1 --2- 172.21.15.94:0/953528502 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fabbc0727c0 0x7fabbc074c80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fabd0001230 tx=0x7fabd0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:23.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.707+0000 7fabd7e00640 1 -- 172.21.15.94:0/953528502 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fabd8017aa0 con 0x7fabe0157c60 2024-09-10T23:35:23.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.907+0000 7fabe6000640 1 -- 172.21.15.94:0/953528502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabe01402b0 con 0x7fabe0157c60 2024-09-10T23:35:23.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.911+0000 7fabd7e00640 1 -- 172.21.15.94:0/953528502 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5235 (secure 0 0 0) 0x7fabd80ffe20 con 0x7fabe0157c60 2024-09-10T23:35:23.911 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:23.911 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"btime":"2024-09-10T23:34:10:080486+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:33:59.428685+0000","modified":"2024-09-10T23:34:02.428651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":194816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_194816":{"gid":194816,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1573303541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1573303541},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573303541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":194816,"qdb_cluster":[194816]},"id":3}]} 2024-09-10T23:35:23.911 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-09-10T23:35:23.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.911+0000 7fabe6000640 1 -- 172.21.15.94:0/953528502 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fabbc0727c0 msgr2=0x7fabbc074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.911+0000 7fabe6000640 1 --2- 172.21.15.94:0/953528502 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fabbc0727c0 0x7fabbc074c80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fabd0001230 tx=0x7fabd0009040 comp rx=0 tx=0).stop 2024-09-10T23:35:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.911+0000 7fabe6000640 1 -- 172.21.15.94:0/953528502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe0157c60 msgr2=0x7fabe00c4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.911+0000 7fabe6000640 1 --2- 172.21.15.94:0/953528502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe0157c60 0x7fabe00c4e80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fabd800ac20 tx=0x7fabd800a410 comp rx=0 tx=0).stop 2024-09-10T23:35:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.911+0000 7fabe6000640 1 -- 172.21.15.94:0/953528502 shutdown_connections 2024-09-10T23:35:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.911+0000 7fabe6000640 1 --2- 172.21.15.94:0/953528502 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fabbc0727c0 0x7fabbc074c80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.911+0000 7fabe6000640 1 --2- 172.21.15.94:0/953528502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe0070150 0x7fabe00c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.911+0000 7fabe6000640 1 --2- 172.21.15.94:0/953528502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe0157c60 0x7fabe00c4e80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.911+0000 7fabe6000640 1 --2- 172.21.15.94:0/953528502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe006f600 0x7fabe013f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.911+0000 7fabe6000640 1 -- 172.21.15.94:0/953528502 >> 172.21.15.94:0/953528502 conn(0x7fabe006cdf0 msgr2=0x7fabe0151870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.911+0000 7fabe6000640 1 -- 172.21.15.94:0/953528502 shutdown_connections 2024-09-10T23:35:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:23.911+0000 7fabe6000640 1 -- 172.21.15.94:0/953528502 wait complete. 2024-09-10T23:35:23.932 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:35:24.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.071+0000 7f0bfac00640 1 -- 172.21.15.94:0/1206028807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bf40af310 msgr2=0x7f0bf40af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:24.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.071+0000 7f0bfac00640 1 --2- 172.21.15.94:0/1206028807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bf40af310 0x7f0bf40af7a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009bc0 tx=0x7f0bec01c9a0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.071+0000 7f0bfac00640 1 -- 172.21.15.94:0/1206028807 shutdown_connections 2024-09-10T23:35:24.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.071+0000 7f0bfac00640 1 --2- 172.21.15.94:0/1206028807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bf40af310 0x7f0bf40af7a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.071+0000 7f0bfac00640 1 --2- 172.21.15.94:0/1206028807 >> v2:172.21.15.181:3300/0 conn(0x7f0bf40b2bb0 0x7f0bf40d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.071+0000 7f0bfac00640 1 -- 172.21.15.94:0/1206028807 >> 172.21.15.94:0/1206028807 conn(0x7f0bf40d1c60 msgr2=0x7f0bf40d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:24.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.071+0000 7f0bfac00640 1 -- 172.21.15.94:0/1206028807 shutdown_connections 2024-09-10T23:35:24.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.071+0000 7f0bfac00640 1 -- 172.21.15.94:0/1206028807 wait complete. 2024-09-10T23:35:24.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bfac00640 1 Processor -- start 2024-09-10T23:35:24.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bfac00640 1 -- start start 2024-09-10T23:35:24.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bfac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bf40af310 0x7f0bf4145be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:24.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bfac00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bf40b2bb0 0x7f0bf40cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:24.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bfac00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bf415f5c0 0x7f0bf40c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:24.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bfac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4168fa0 con 0x7f0bf40af310 2024-09-10T23:35:24.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bfac00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4168e20 con 0x7f0bf40b2bb0 2024-09-10T23:35:24.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bfac00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf4169120 con 0x7f0bf415f5c0 2024-09-10T23:35:24.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bf3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bf40af310 0x7f0bf4145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:24.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bf3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bf40af310 0x7f0bf4145be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60270/0 (socket says 172.21.15.94:60270) 2024-09-10T23:35:24.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bf3e00640 1 -- 172.21.15.94:0/2052235691 learned_addr learned my addr 172.21.15.94:0/2052235691 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:24.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bf8c00640 1 --2- 172.21.15.94:0/2052235691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bf415f5c0 0x7f0bf40c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:24.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bf3400640 1 --2- 172.21.15.94:0/2052235691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bf40b2bb0 0x7f0bf40cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:24.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bf3e00640 1 -- 172.21.15.94:0/2052235691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bf415f5c0 msgr2=0x7f0bf40c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:24.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bf3e00640 1 --2- 172.21.15.94:0/2052235691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bf415f5c0 0x7f0bf40c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bf3e00640 1 -- 172.21.15.94:0/2052235691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bf40b2bb0 msgr2=0x7f0bf40cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bf3e00640 1 --2- 172.21.15.94:0/2052235691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bf40b2bb0 0x7f0bf40cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bf3e00640 1 -- 172.21.15.94:0/2052235691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf40b0c10 con 0x7f0bf40af310 2024-09-10T23:35:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bf3400640 1 --2- 172.21.15.94:0/2052235691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bf40b2bb0 0x7f0bf40cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:35:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bf3e00640 1 --2- 172.21.15.94:0/2052235691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bf40af310 0x7f0bf4145be0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0be400c500 tx=0x7f0be400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bebe00640 1 -- 172.21.15.94:0/2052235691 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4016020 con 0x7f0bf40af310 2024-09-10T23:35:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bebe00640 1 -- 172.21.15.94:0/2052235691 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0be4012940 con 0x7f0bf40af310 2024-09-10T23:35:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bebe00640 1 -- 172.21.15.94:0/2052235691 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4012d90 con 0x7f0bf40af310 2024-09-10T23:35:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bfac00640 1 -- 172.21.15.94:0/2052235691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf414f830 con 0x7f0bf40af310 2024-09-10T23:35:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bfac00640 1 -- 172.21.15.94:0/2052235691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf414fdb0 con 0x7f0bf40af310 2024-09-10T23:35:24.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.075+0000 7f0bfac00640 1 -- 172.21.15.94:0/2052235691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf40af7a0 con 0x7f0bf40af310 2024-09-10T23:35:24.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.079+0000 7f0bebe00640 1 -- 172.21.15.94:0/2052235691 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f0be401b400 con 0x7f0bf40af310 2024-09-10T23:35:24.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.079+0000 7f0bebe00640 1 --2- 172.21.15.94:0/2052235691 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0bd4072790 0x7f0bd4074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:24.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.079+0000 7f0bebe00640 1 -- 172.21.15.94:0/2052235691 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7f0be4093b90 con 0x7f0bf40af310 2024-09-10T23:35:24.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.079+0000 7f0bf3400640 1 --2- 172.21.15.94:0/2052235691 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0bd4072790 0x7f0bd4074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:24.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.079+0000 7f0bf3400640 1 --2- 172.21.15.94:0/2052235691 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0bd4072790 0x7f0bd4074c50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40cbf70 tx=0x7f0be00091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:24.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.083+0000 7f0bebe00640 1 -- 172.21.15.94:0/2052235691 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0be4020020 con 0x7f0bf40af310 2024-09-10T23:35:24.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.247+0000 7f0bfac00640 1 -- 172.21.15.94:0/2052235691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0bf40749c0 con 0x7f0bf40af310 2024-09-10T23:35:24.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.251+0000 7f0bebe00640 1 -- 172.21.15.94:0/2052235691 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16950 (secure 0 0 0) 0x7f0be405edc0 con 0x7f0bf40af310 2024-09-10T23:35:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:24.251 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":46,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:34:23.122764+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:33:57.113965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T23:33:58.384504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.14","mappings":[{"from":5,"to":7}]},{"pgid":"7.1b","mappings":[{"from":5,"to":7}]},{"pgid":"7.20","mappings":[{"from":5,"to":0}]},{"pgid":"7.32","mappings":[{"from":5,"to":7}]},{"pgid":"7.33","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:35:24.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.251+0000 7f0bfac00640 1 -- 172.21.15.94:0/2052235691 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0bd4072790 msgr2=0x7f0bd4074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:24.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.251+0000 7f0bfac00640 1 --2- 172.21.15.94:0/2052235691 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0bd4072790 0x7f0bd4074c50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40cbf70 tx=0x7f0be00091e0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.251+0000 7f0bfac00640 1 -- 172.21.15.94:0/2052235691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bf40af310 msgr2=0x7f0bf4145be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:24.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.251+0000 7f0bfac00640 1 --2- 172.21.15.94:0/2052235691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bf40af310 0x7f0bf4145be0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0be400c500 tx=0x7f0be400c9c0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.251+0000 7f0bfac00640 1 -- 172.21.15.94:0/2052235691 shutdown_connections 2024-09-10T23:35:24.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.251+0000 7f0bfac00640 1 --2- 172.21.15.94:0/2052235691 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0bd4072790 0x7f0bd4074c50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.251+0000 7f0bfac00640 1 --2- 172.21.15.94:0/2052235691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bf40b2bb0 0x7f0bf40cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.251+0000 7f0bfac00640 1 --2- 172.21.15.94:0/2052235691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bf415f5c0 0x7f0bf40c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.251+0000 7f0bfac00640 1 --2- 172.21.15.94:0/2052235691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bf40af310 0x7f0bf4145be0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.251+0000 7f0bfac00640 1 -- 172.21.15.94:0/2052235691 >> 172.21.15.94:0/2052235691 conn(0x7f0bf40d1c60 msgr2=0x7f0bf415cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:24.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.251+0000 7f0bfac00640 1 -- 172.21.15.94:0/2052235691 shutdown_connections 2024-09-10T23:35:24.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.251+0000 7f0bfac00640 1 -- 172.21.15.94:0/2052235691 wait complete. 2024-09-10T23:35:24.273 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T23:35:24.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.415+0000 7f8c99000640 1 -- 172.21.15.94:0/1012734470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c940ce280 msgr2=0x7f8c940cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:24.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.415+0000 7f8c99000640 1 --2- 172.21.15.94:0/1012734470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c940ce280 0x7f8c940cf280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400a980 tx=0x7f8c8401db90 comp rx=0 tx=0).stop 2024-09-10T23:35:24.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.415+0000 7f8c99000640 1 -- 172.21.15.94:0/1012734470 shutdown_connections 2024-09-10T23:35:24.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.415+0000 7f8c99000640 1 --2- 172.21.15.94:0/1012734470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c940ce280 0x7f8c940cf280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.415+0000 7f8c99000640 1 -- 172.21.15.94:0/1012734470 >> 172.21.15.94:0/1012734470 conn(0x7f8c940cfaf0 msgr2=0x7f8c940cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:24.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.415+0000 7f8c99000640 1 -- 172.21.15.94:0/1012734470 shutdown_connections 2024-09-10T23:35:24.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.415+0000 7f8c99000640 1 -- 172.21.15.94:0/1012734470 wait complete. 2024-09-10T23:35:24.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.415+0000 7f8c99000640 1 Processor -- start 2024-09-10T23:35:24.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.415+0000 7f8c99000640 1 -- start start 2024-09-10T23:35:24.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c99000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c9406e7c0 0x7f8c940c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:24.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c99000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c940c64f0 0x7f8c9414ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:24.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c99000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c940ce280 0x7f8c940b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:24.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c99000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8c94168850 con 0x7f8c940ce280 2024-09-10T23:35:24.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c99000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8c941686d0 con 0x7f8c940c64f0 2024-09-10T23:35:24.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c99000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8c941689d0 con 0x7f8c9406e7c0 2024-09-10T23:35:24.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c92800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c940ce280 0x7f8c940b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:24.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c92800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c940ce280 0x7f8c940b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60272/0 (socket says 172.21.15.94:60272) 2024-09-10T23:35:24.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c92800640 1 -- 172.21.15.94:0/2469909378 learned_addr learned my addr 172.21.15.94:0/2469909378 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:24.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c91400640 1 --2- 172.21.15.94:0/2469909378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c940c64f0 0x7f8c9414ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:24.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c91e00640 1 --2- 172.21.15.94:0/2469909378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c9406e7c0 0x7f8c940c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c92800640 1 -- 172.21.15.94:0/2469909378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c9406e7c0 msgr2=0x7f8c940c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c92800640 1 --2- 172.21.15.94:0/2469909378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c9406e7c0 0x7f8c940c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c92800640 1 -- 172.21.15.94:0/2469909378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c940c64f0 msgr2=0x7f8c9414ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c92800640 1 --2- 172.21.15.94:0/2469909378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c940c64f0 0x7f8c9414ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c92800640 1 -- 172.21.15.94:0/2469909378 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c940c7fd0 con 0x7f8c940ce280 2024-09-10T23:35:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c92800640 1 --2- 172.21.15.94:0/2469909378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c940ce280 0x7f8c940b0610 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00c7d0 tx=0x7f8c8c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c8a000640 1 -- 172.21.15.94:0/2469909378 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c003b80 con 0x7f8c940ce280 2024-09-10T23:35:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c8a000640 1 -- 172.21.15.94:0/2469909378 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c8c003d40 con 0x7f8c940ce280 2024-09-10T23:35:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c99000640 1 -- 172.21.15.94:0/2469909378 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c940c82b0 con 0x7f8c940ce280 2024-09-10T23:35:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c8a000640 1 -- 172.21.15.94:0/2469909378 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00aab0 con 0x7f8c940ce280 2024-09-10T23:35:24.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c99000640 1 -- 172.21.15.94:0/2469909378 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c940c8870 con 0x7f8c940ce280 2024-09-10T23:35:24.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c99000640 1 -- 172.21.15.94:0/2469909378 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c940b4a90 con 0x7f8c940ce280 2024-09-10T23:35:24.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.419+0000 7f8c8a000640 1 -- 172.21.15.94:0/2469909378 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f8c8c00ac50 con 0x7f8c940ce280 2024-09-10T23:35:24.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.423+0000 7f8c8a000640 1 --2- 172.21.15.94:0/2469909378 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8c74072790 0x7f8c74074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:24.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.423+0000 7f8c8a000640 1 -- 172.21.15.94:0/2469909378 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5854+0+0 (secure 0 0 0) 0x7f8c8c093450 con 0x7f8c940ce280 2024-09-10T23:35:24.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.423+0000 7f8c91e00640 1 --2- 172.21.15.94:0/2469909378 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8c74072790 0x7f8c74074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:24.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.423+0000 7f8c91e00640 1 --2- 172.21.15.94:0/2469909378 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8c74072790 0x7f8c74074c50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8c840b0000 tx=0x7f8c8401c3d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:24.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.427+0000 7f8c8a000640 1 -- 172.21.15.94:0/2469909378 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8c8c05e0b0 con 0x7f8c940ce280 2024-09-10T23:35:24.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.631+0000 7f8c99000640 1 -- 172.21.15.94:0/2469909378 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8c940d0850 con 0x7f8c940ce280 2024-09-10T23:35:24.632 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-10T23:35:24.631+0000 7f4045a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:35:24.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.635+0000 7f8c8a000640 1 -- 172.21.15.94:0/2469909378 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f8c8c062ae0 con 0x7f8c940ce280 2024-09-10T23:35:24.634 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T23:35:24.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.635+0000 7f8c99000640 1 -- 172.21.15.94:0/2469909378 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8c74072790 msgr2=0x7f8c74074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:24.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.635+0000 7f8c99000640 1 --2- 172.21.15.94:0/2469909378 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8c74072790 0x7f8c74074c50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8c840b0000 tx=0x7f8c8401c3d0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.635+0000 7f8c99000640 1 -- 172.21.15.94:0/2469909378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c940ce280 msgr2=0x7f8c940b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:24.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.635+0000 7f8c99000640 1 --2- 172.21.15.94:0/2469909378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c940ce280 0x7f8c940b0610 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00c7d0 tx=0x7f8c8c00cc90 comp rx=0 tx=0).stop 2024-09-10T23:35:24.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.635+0000 7f8c99000640 1 -- 172.21.15.94:0/2469909378 shutdown_connections 2024-09-10T23:35:24.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.635+0000 7f8c99000640 1 --2- 172.21.15.94:0/2469909378 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8c74072790 0x7f8c74074c50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.635+0000 7f8c99000640 1 --2- 172.21.15.94:0/2469909378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c940c64f0 0x7f8c9414ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.635+0000 7f8c99000640 1 --2- 172.21.15.94:0/2469909378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c940ce280 0x7f8c940b0610 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.635+0000 7f8c99000640 1 --2- 172.21.15.94:0/2469909378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c9406e7c0 0x7f8c940c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.635+0000 7f8c99000640 1 -- 172.21.15.94:0/2469909378 >> 172.21.15.94:0/2469909378 conn(0x7f8c940cfaf0 msgr2=0x7f8c94151a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:24.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.635+0000 7f8c99000640 1 -- 172.21.15.94:0/2469909378 shutdown_connections 2024-09-10T23:35:24.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.635+0000 7f8c99000640 1 -- 172.21.15.94:0/2469909378 wait complete. 2024-09-10T23:35:24.649 INFO:tasks.ceph.mds.d.smithi181.stderr: -9663> 2024-09-10T23:34:02.017+0000 7fd491200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:35:24.657 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T23:35:24.701 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2024-09-10T23:35:24.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.799+0000 7fc5f6600640 1 -- 172.21.15.94:0/2537850135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d40051d0 msgr2=0x7fc5d400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:24.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.799+0000 7fc5f6600640 1 --2- 172.21.15.94:0/2537850135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d40051d0 0x7fc5d400b9d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009eb0 tx=0x7fc5e801d5f0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.799+0000 7fc5f6600640 1 -- 172.21.15.94:0/2537850135 shutdown_connections 2024-09-10T23:35:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.799+0000 7fc5f6600640 1 --2- 172.21.15.94:0/2537850135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d40051d0 0x7fc5d400b9d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.799+0000 7fc5f6600640 1 -- 172.21.15.94:0/2537850135 >> 172.21.15.94:0/2537850135 conn(0x7fc5f00d1c90 msgr2=0x7fc5f00d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.799+0000 7fc5f6600640 1 -- 172.21.15.94:0/2537850135 shutdown_connections 2024-09-10T23:35:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.799+0000 7fc5f6600640 1 -- 172.21.15.94:0/2537850135 wait complete. 2024-09-10T23:35:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5f6600640 1 Processor -- start 2024-09-10T23:35:24.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5f6600640 1 -- start start 2024-09-10T23:35:24.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5f6600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d40051d0 0x7fc5f015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:24.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5f6600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5f00c57c0 0x7fc5f00c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:24.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5f6600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f00c7b10 0x7fc5f00c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:24.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5f6600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f00d27b0 con 0x7fc5f00c57c0 2024-09-10T23:35:24.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5f6600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f00b25b0 con 0x7fc5f00c7b10 2024-09-10T23:35:24.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5f6600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc5f00d2930 con 0x7fc5d40051d0 2024-09-10T23:35:24.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5eea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5f00c57c0 0x7fc5f00c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:24.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5ef400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d40051d0 0x7fc5f015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:24.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5ef400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d40051d0 0x7fc5f015c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:36106/0 (socket says 172.21.15.94:36106) 2024-09-10T23:35:24.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5ef400640 1 -- 172.21.15.94:0/3447363534 learned_addr learned my addr 172.21.15.94:0/3447363534 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:24.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5efe00640 1 --2- 172.21.15.94:0/3447363534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f00c7b10 0x7fc5f00c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:24.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5eea00640 1 -- 172.21.15.94:0/3447363534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d40051d0 msgr2=0x7fc5f015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:24.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5eea00640 1 --2- 172.21.15.94:0/3447363534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d40051d0 0x7fc5f015c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5eea00640 1 -- 172.21.15.94:0/3447363534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f00c7b10 msgr2=0x7fc5f00c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:24.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5eea00640 1 --2- 172.21.15.94:0/3447363534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f00c7b10 0x7fc5f00c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:24.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5eea00640 1 -- 172.21.15.94:0/3447363534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f015ccf0 con 0x7fc5f00c57c0 2024-09-10T23:35:24.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5ef400640 1 --2- 172.21.15.94:0/3447363534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d40051d0 0x7fc5f015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:35:24.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5eea00640 1 --2- 172.21.15.94:0/3447363534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5f00c57c0 0x7fc5f00c5c40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc003d90 tx=0x7fc5dc001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:24.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5e7400640 1 -- 172.21.15.94:0/3447363534 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc001d50 con 0x7fc5f00c57c0 2024-09-10T23:35:24.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5e7400640 1 -- 172.21.15.94:0/3447363534 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5dc016020 con 0x7fc5f00c57c0 2024-09-10T23:35:24.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5f6600640 1 -- 172.21.15.94:0/3447363534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f015cf90 con 0x7fc5f00c57c0 2024-09-10T23:35:24.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5e7400640 1 -- 172.21.15.94:0/3447363534 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc011880 con 0x7fc5f00c57c0 2024-09-10T23:35:24.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.803+0000 7fc5f6600640 1 -- 172.21.15.94:0/3447363534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f015d550 con 0x7fc5f00c57c0 2024-09-10T23:35:24.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.807+0000 7fc5f6600640 1 -- 172.21.15.94:0/3447363534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f00b2a10 con 0x7fc5f00c57c0 2024-09-10T23:35:24.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.807+0000 7fc5e7400640 1 -- 172.21.15.94:0/3447363534 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7fc5dc011a20 con 0x7fc5f00c57c0 2024-09-10T23:35:24.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.807+0000 7fc5e7400640 1 --2- 172.21.15.94:0/3447363534 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc5d00727c0 0x7fc5d0074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:24.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.807+0000 7fc5e7400640 1 -- 172.21.15.94:0/3447363534 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5940+0+0 (secure 0 0 0) 0x7fc5dc094130 con 0x7fc5f00c57c0 2024-09-10T23:35:24.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.807+0000 7fc5ef400640 1 --2- 172.21.15.94:0/3447363534 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc5d00727c0 0x7fc5d0074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:24.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.807+0000 7fc5ef400640 1 --2- 172.21.15.94:0/3447363534 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc5d00727c0 0x7fc5d0074c80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0001d90 tx=0x7fc5e00067b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:24.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:24.811+0000 7fc5e7400640 1 -- 172.21.15.94:0/3447363534 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc5dc020020 con 0x7fc5f00c57c0 2024-09-10T23:35:25.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.011+0000 7fc5f6600640 1 -- 172.21.15.94:0/3447363534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc5f0070b90 con 0x7fc5f00c57c0 2024-09-10T23:35:25.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.639+0000 7fc5e7400640 1 -- 172.21.15.94:0/3447363534 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7fc5dc05eca0 con 0x7fc5f00c57c0 2024-09-10T23:35:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.643+0000 7fc5f6600640 1 -- 172.21.15.94:0/3447363534 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc5d00727c0 msgr2=0x7fc5d0074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.643+0000 7fc5f6600640 1 --2- 172.21.15.94:0/3447363534 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc5d00727c0 0x7fc5d0074c80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0001d90 tx=0x7fc5e00067b0 comp rx=0 tx=0).stop 2024-09-10T23:35:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.643+0000 7fc5f6600640 1 -- 172.21.15.94:0/3447363534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5f00c57c0 msgr2=0x7fc5f00c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.643+0000 7fc5f6600640 1 --2- 172.21.15.94:0/3447363534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5f00c57c0 0x7fc5f00c5c40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc003d90 tx=0x7fc5dc001640 comp rx=0 tx=0).stop 2024-09-10T23:35:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.643+0000 7fc5f6600640 1 -- 172.21.15.94:0/3447363534 shutdown_connections 2024-09-10T23:35:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.643+0000 7fc5f6600640 1 --2- 172.21.15.94:0/3447363534 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc5d00727c0 0x7fc5d0074c80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.643+0000 7fc5f6600640 1 --2- 172.21.15.94:0/3447363534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f00c7b10 0x7fc5f00c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.643+0000 7fc5f6600640 1 --2- 172.21.15.94:0/3447363534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5f00c57c0 0x7fc5f00c5c40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.643+0000 7fc5f6600640 1 --2- 172.21.15.94:0/3447363534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d40051d0 0x7fc5f015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.643+0000 7fc5f6600640 1 -- 172.21.15.94:0/3447363534 >> 172.21.15.94:0/3447363534 conn(0x7fc5f00d1c90 msgr2=0x7fc5f00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.643+0000 7fc5f6600640 1 -- 172.21.15.94:0/3447363534 shutdown_connections 2024-09-10T23:35:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.643+0000 7fc5f6600640 1 -- 172.21.15.94:0/3447363534 wait complete. 2024-09-10T23:35:25.663 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T23:35:25.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.807+0000 7fa49a600640 1 -- 172.21.15.94:0/3368947204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa478003c20 msgr2=0x7fa478004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:25.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.807+0000 7fa49a600640 1 --2- 172.21.15.94:0/3368947204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa478003c20 0x7fa478004040 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa48400c560 tx=0x7fa48401d230 comp rx=0 tx=0).stop 2024-09-10T23:35:25.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.807+0000 7fa493e00640 1 -- 172.21.15.94:0/3368947204 reap_dead start 2024-09-10T23:35:25.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.807+0000 7fa49a600640 1 -- 172.21.15.94:0/3368947204 shutdown_connections 2024-09-10T23:35:25.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.807+0000 7fa49a600640 1 -- 172.21.15.94:0/3368947204 >> 172.21.15.94:0/3368947204 conn(0x7fa49406cdf0 msgr2=0x7fa49406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:25.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.807+0000 7fa49a600640 1 -- 172.21.15.94:0/3368947204 shutdown_connections 2024-09-10T23:35:25.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.807+0000 7fa49a600640 1 -- 172.21.15.94:0/3368947204 wait complete. 2024-09-10T23:35:25.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa49a600640 1 Processor -- start 2024-09-10T23:35:25.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa49a600640 1 -- start start 2024-09-10T23:35:25.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa49a600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa478003c20 0x7fa494165850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:25.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa49a600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4941f6160 0x7fa4941661e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:25.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa49a600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4941f65c0 0x7fa494166720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:25.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa49a600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa494167be0 con 0x7fa4941f6160 2024-09-10T23:35:25.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa49a600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa494167a60 con 0x7fa478003c20 2024-09-10T23:35:25.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa49a600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa494167d60 con 0x7fa4941f65c0 2024-09-10T23:35:25.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa48aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4941f6160 0x7fa4941661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:25.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa48aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4941f6160 0x7fa4941661e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60316/0 (socket says 172.21.15.94:60316) 2024-09-10T23:35:25.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa48aa00640 1 -- 172.21.15.94:0/54257668 learned_addr learned my addr 172.21.15.94:0/54257668 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:25.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa493e00640 1 --2- 172.21.15.94:0/54257668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4941f65c0 0x7fa494166720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:25.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa493400640 1 --2- 172.21.15.94:0/54257668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa478003c20 0x7fa494165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:25.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa48aa00640 1 -- 172.21.15.94:0/54257668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4941f65c0 msgr2=0x7fa494166720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:25.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa48aa00640 1 --2- 172.21.15.94:0/54257668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4941f65c0 0x7fa494166720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:25.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa48aa00640 1 -- 172.21.15.94:0/54257668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa478003c20 msgr2=0x7fa494165850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:25.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa48aa00640 1 --2- 172.21.15.94:0/54257668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa478003c20 0x7fa494165850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:25.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa48aa00640 1 -- 172.21.15.94:0/54257668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa494154630 con 0x7fa4941f6160 2024-09-10T23:35:25.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa493e00640 1 --2- 172.21.15.94:0/54257668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4941f65c0 0x7fa494166720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:35:25.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa48aa00640 1 --2- 172.21.15.94:0/54257668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4941f6160 0x7fa4941661e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa47c011ae0 tx=0x7fa47c009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:25.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa48be00640 1 -- 172.21.15.94:0/54257668 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c01e070 con 0x7fa4941f6160 2024-09-10T23:35:25.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa48be00640 1 -- 172.21.15.94:0/54257668 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa47c00fc10 con 0x7fa4941f6160 2024-09-10T23:35:25.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa48be00640 1 -- 172.21.15.94:0/54257668 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c019410 con 0x7fa4941f6160 2024-09-10T23:35:25.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa49a600640 1 -- 172.21.15.94:0/54257668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa494154070 con 0x7fa4941f6160 2024-09-10T23:35:25.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa49a600640 1 -- 172.21.15.94:0/54257668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49414aca0 con 0x7fa4941f6160 2024-09-10T23:35:25.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.811+0000 7fa49a600640 1 -- 172.21.15.94:0/54257668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4940ce260 con 0x7fa4941f6160 2024-09-10T23:35:25.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.815+0000 7fa48be00640 1 -- 172.21.15.94:0/54257668 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7fa47c0195b0 con 0x7fa4941f6160 2024-09-10T23:35:25.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.815+0000 7fa48be00640 1 --2- 172.21.15.94:0/54257668 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa470072790 0x7fa470074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:25.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.815+0000 7fa48be00640 1 -- 172.21.15.94:0/54257668 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5940+0+0 (secure 0 0 0) 0x7fa47c094230 con 0x7fa4941f6160 2024-09-10T23:35:25.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.815+0000 7fa493400640 1 --2- 172.21.15.94:0/54257668 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa470072790 0x7fa470074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:25.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.815+0000 7fa493400640 1 --2- 172.21.15.94:0/54257668 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa470072790 0x7fa470074c50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa4840ac000 tx=0x7fa48401a070 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:25.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.819+0000 7fa48be00640 1 -- 172.21.15.94:0/54257668 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa47c05ee50 con 0x7fa4941f6160 2024-09-10T23:35:25.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:25.987+0000 7fa49a600640 1 -- 172.21.15.94:0/54257668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa494159b40 con 0x7fa4941f6160 2024-09-10T23:35:26.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.639+0000 7fa48be00640 1 -- 172.21.15.94:0/54257668 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v48) ==== 181+0+0 (secure 0 0 0) 0x7fa47c063880 con 0x7fa4941f6160 2024-09-10T23:35:26.642 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-09-10T23:35:26.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.643+0000 7fa49a600640 1 -- 172.21.15.94:0/54257668 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa470072790 msgr2=0x7fa470074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:26.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.643+0000 7fa49a600640 1 --2- 172.21.15.94:0/54257668 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa470072790 0x7fa470074c50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa4840ac000 tx=0x7fa48401a070 comp rx=0 tx=0).stop 2024-09-10T23:35:26.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.643+0000 7fa49a600640 1 -- 172.21.15.94:0/54257668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4941f6160 msgr2=0x7fa4941661e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:26.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.643+0000 7fa49a600640 1 --2- 172.21.15.94:0/54257668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4941f6160 0x7fa4941661e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa47c011ae0 tx=0x7fa47c009e70 comp rx=0 tx=0).stop 2024-09-10T23:35:26.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.643+0000 7fa49a600640 1 -- 172.21.15.94:0/54257668 shutdown_connections 2024-09-10T23:35:26.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.643+0000 7fa49a600640 1 --2- 172.21.15.94:0/54257668 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa470072790 0x7fa470074c50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:26.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.643+0000 7fa49a600640 1 --2- 172.21.15.94:0/54257668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4941f65c0 0x7fa494166720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:26.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.643+0000 7fa49a600640 1 --2- 172.21.15.94:0/54257668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa4941f6160 0x7fa4941661e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:26.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.643+0000 7fa49a600640 1 --2- 172.21.15.94:0/54257668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa478003c20 0x7fa494165850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:26.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.643+0000 7fa49a600640 1 -- 172.21.15.94:0/54257668 >> 172.21.15.94:0/54257668 conn(0x7fa49406cdf0 msgr2=0x7fa494154870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:26.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.643+0000 7fa49a600640 1 -- 172.21.15.94:0/54257668 shutdown_connections 2024-09-10T23:35:26.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.643+0000 7fa49a600640 1 -- 172.21.15.94:0/54257668 wait complete. 2024-09-10T23:35:26.669 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T23:35:26.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.819+0000 7efce4600640 1 -- 172.21.15.94:0/990966032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcdc15ba50 msgr2=0x7efcdc0b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:26.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.819+0000 7efce4600640 1 --2- 172.21.15.94:0/990966032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcdc15ba50 0x7efcdc0b4700 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009840 tx=0x7efcd001c760 comp rx=0 tx=0).stop 2024-09-10T23:35:26.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.819+0000 7efce4600640 1 -- 172.21.15.94:0/990966032 shutdown_connections 2024-09-10T23:35:26.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.819+0000 7efce4600640 1 --2- 172.21.15.94:0/990966032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efcdc0d03c0 0x7efcdc0cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:26.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.819+0000 7efce4600640 1 --2- 172.21.15.94:0/990966032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcdc15ba50 0x7efcdc0b4700 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:26.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.819+0000 7efce4600640 1 -- 172.21.15.94:0/990966032 >> 172.21.15.94:0/990966032 conn(0x7efcdc06cdf0 msgr2=0x7efcdc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:26.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.819+0000 7efce4600640 1 -- 172.21.15.94:0/990966032 shutdown_connections 2024-09-10T23:35:26.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.819+0000 7efce4600640 1 -- 172.21.15.94:0/990966032 wait complete. 2024-09-10T23:35:26.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.819+0000 7efce4600640 1 Processor -- start 2024-09-10T23:35:26.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.819+0000 7efce4600640 1 -- start start 2024-09-10T23:35:26.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce4600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efcdc0b4c40 0x7efcdc1f8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:26.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce4600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcdc0d03c0 0x7efcdc14c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:26.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce4600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcdc15ba50 0x7efcdc14c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:26.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce4600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc1689d0 con 0x7efcdc15ba50 2024-09-10T23:35:26.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce4600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc168850 con 0x7efcdc0d03c0 2024-09-10T23:35:26.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce4600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7efcdc168b50 con 0x7efcdc0b4c40 2024-09-10T23:35:26.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce2600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcdc15ba50 0x7efcdc14c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:26.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce1c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efcdc0b4c40 0x7efcdc1f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:26.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce1c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efcdc0b4c40 0x7efcdc1f8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:36160/0 (socket says 172.21.15.94:36160) 2024-09-10T23:35:26.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce1c00640 1 -- 172.21.15.94:0/2920710207 learned_addr learned my addr 172.21.15.94:0/2920710207 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:26.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce2600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcdc15ba50 0x7efcdc14c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60326/0 (socket says 172.21.15.94:60326) 2024-09-10T23:35:26.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce1200640 1 --2- 172.21.15.94:0/2920710207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcdc0d03c0 0x7efcdc14c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:26.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce2600640 1 -- 172.21.15.94:0/2920710207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efcdc0b4c40 msgr2=0x7efcdc1f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:26.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce2600640 1 --2- 172.21.15.94:0/2920710207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efcdc0b4c40 0x7efcdc1f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:26.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce2600640 1 -- 172.21.15.94:0/2920710207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcdc0d03c0 msgr2=0x7efcdc14c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:26.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce2600640 1 --2- 172.21.15.94:0/2920710207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcdc0d03c0 0x7efcdc14c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:26.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce2600640 1 -- 172.21.15.94:0/2920710207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcdc13a270 con 0x7efcdc15ba50 2024-09-10T23:35:26.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce1c00640 1 --2- 172.21.15.94:0/2920710207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efcdc0b4c40 0x7efcdc1f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:35:26.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce2600640 1 --2- 172.21.15.94:0/2920710207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcdc15ba50 0x7efcdc14c8d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efcd400ea60 tx=0x7efcd400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:26.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efcd9600640 1 -- 172.21.15.94:0/2920710207 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd400ce40 con 0x7efcdc15ba50 2024-09-10T23:35:26.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efcd9600640 1 -- 172.21.15.94:0/2920710207 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efcd4004c20 con 0x7efcdc15ba50 2024-09-10T23:35:26.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efcd9600640 1 -- 172.21.15.94:0/2920710207 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd4005020 con 0x7efcdc15ba50 2024-09-10T23:35:26.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce4600640 1 -- 172.21.15.94:0/2920710207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcdc13a550 con 0x7efcdc15ba50 2024-09-10T23:35:26.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce4600640 1 -- 172.21.15.94:0/2920710207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcdc1f86b0 con 0x7efcdc15ba50 2024-09-10T23:35:26.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efce4600640 1 -- 172.21.15.94:0/2920710207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcdc0cddb0 con 0x7efcdc15ba50 2024-09-10T23:35:26.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.823+0000 7efcd9600640 1 -- 172.21.15.94:0/2920710207 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7efcd40051c0 con 0x7efcdc15ba50 2024-09-10T23:35:26.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.827+0000 7efcd9600640 1 --2- 172.21.15.94:0/2920710207 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efcbc072790 0x7efcbc074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:26.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.827+0000 7efcd9600640 1 -- 172.21.15.94:0/2920710207 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5502+0+0 (secure 0 0 0) 0x7efcd4094320 con 0x7efcdc15ba50 2024-09-10T23:35:26.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.827+0000 7efce1c00640 1 --2- 172.21.15.94:0/2920710207 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efcbc072790 0x7efcbc074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:26.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.827+0000 7efce1c00640 1 --2- 172.21.15.94:0/2920710207 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efcbc072790 0x7efcbc074c50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009840 tx=0x7efcd001afa0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:26.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.831+0000 7efcd9600640 1 -- 172.21.15.94:0/2920710207 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efcd405f050 con 0x7efcdc15ba50 2024-09-10T23:35:26.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:26.995+0000 7efce4600640 1 -- 172.21.15.94:0/2920710207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7efcdc166620 con 0x7efcdc15ba50 2024-09-10T23:35:27.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.647+0000 7efcd9600640 1 -- 172.21.15.94:0/2920710207 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v49) ==== 169+0+0 (secure 0 0 0) 0x7efcd4063a80 con 0x7efcdc15ba50 2024-09-10T23:35:27.647 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-09-10T23:35:27.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.651+0000 7efce4600640 1 -- 172.21.15.94:0/2920710207 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efcbc072790 msgr2=0x7efcbc074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:27.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.651+0000 7efce4600640 1 --2- 172.21.15.94:0/2920710207 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efcbc072790 0x7efcbc074c50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009840 tx=0x7efcd001afa0 comp rx=0 tx=0).stop 2024-09-10T23:35:27.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.651+0000 7efce4600640 1 -- 172.21.15.94:0/2920710207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcdc15ba50 msgr2=0x7efcdc14c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:27.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.651+0000 7efce4600640 1 --2- 172.21.15.94:0/2920710207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcdc15ba50 0x7efcdc14c8d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efcd400ea60 tx=0x7efcd400ef20 comp rx=0 tx=0).stop 2024-09-10T23:35:27.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.651+0000 7efce4600640 1 -- 172.21.15.94:0/2920710207 shutdown_connections 2024-09-10T23:35:27.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.651+0000 7efce4600640 1 --2- 172.21.15.94:0/2920710207 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efcbc072790 0x7efcbc074c50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:27.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.651+0000 7efce4600640 1 --2- 172.21.15.94:0/2920710207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efcdc0d03c0 0x7efcdc14c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:27.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.651+0000 7efce4600640 1 --2- 172.21.15.94:0/2920710207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcdc15ba50 0x7efcdc14c8d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:27.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.651+0000 7efce4600640 1 --2- 172.21.15.94:0/2920710207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efcdc0b4c40 0x7efcdc1f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:27.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.651+0000 7efce4600640 1 -- 172.21.15.94:0/2920710207 >> 172.21.15.94:0/2920710207 conn(0x7efcdc06cdf0 msgr2=0x7efcdc06e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:27.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.651+0000 7efce4600640 1 -- 172.21.15.94:0/2920710207 shutdown_connections 2024-09-10T23:35:27.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.651+0000 7efce4600640 1 -- 172.21.15.94:0/2920710207 wait complete. 2024-09-10T23:35:27.674 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T23:35:27.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.815+0000 7f18a9800640 1 -- 172.21.15.94:0/1811994837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18a4073630 msgr2=0x7f18a4070840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:27.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.815+0000 7f18a9800640 1 --2- 172.21.15.94:0/1811994837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18a4073630 0x7f18a4070840 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f18a40d1b70 tx=0x7f188c01c770 comp rx=0 tx=0).stop 2024-09-10T23:35:27.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.815+0000 7f18a9800640 1 -- 172.21.15.94:0/1811994837 shutdown_connections 2024-09-10T23:35:27.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.815+0000 7f18a9800640 1 --2- 172.21.15.94:0/1811994837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18a4073630 0x7f18a4070840 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:27.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.815+0000 7f18a9800640 1 --2- 172.21.15.94:0/1811994837 >> v2:172.21.15.181:3300/0 conn(0x7f18a41455f0 0x7f18a40cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:27.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.815+0000 7f18a9800640 1 -- 172.21.15.94:0/1811994837 >> 172.21.15.94:0/1811994837 conn(0x7f18a40d0d80 msgr2=0x7f18a40d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:27.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.815+0000 7f18a9800640 1 -- 172.21.15.94:0/1811994837 shutdown_connections 2024-09-10T23:35:27.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.815+0000 7f18a9800640 1 -- 172.21.15.94:0/1811994837 wait complete. 2024-09-10T23:35:27.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a9800640 1 Processor -- start 2024-09-10T23:35:27.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a9800640 1 -- start start 2024-09-10T23:35:27.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a9800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18a4070d80 0x7f18a4140f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:27.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a9800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18a4073630 0x7f18a40b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:27.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a9800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18a41455f0 0x7f18a40b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:27.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a9800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f18a4168ab0 con 0x7f18a4070d80 2024-09-10T23:35:27.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a9800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f18a40cfb40 con 0x7f18a4073630 2024-09-10T23:35:27.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a9800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f18a4168c30 con 0x7f18a41455f0 2024-09-10T23:35:27.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18a4070d80 0x7f18a4140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:27.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18a4070d80 0x7f18a4140f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60338/0 (socket says 172.21.15.94:60338) 2024-09-10T23:35:27.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a2a00640 1 -- 172.21.15.94:0/2873085013 learned_addr learned my addr 172.21.15.94:0/2873085013 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f189a000640 1 --2- 172.21.15.94:0/2873085013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18a4073630 0x7f18a40b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a2a00640 1 -- 172.21.15.94:0/2873085013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18a41455f0 msgr2=0x7f18a40b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a3400640 1 --2- 172.21.15.94:0/2873085013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18a41455f0 0x7f18a40b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a2a00640 1 --2- 172.21.15.94:0/2873085013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18a41455f0 0x7f18a40b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a2a00640 1 -- 172.21.15.94:0/2873085013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18a4073630 msgr2=0x7f18a40b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a2a00640 1 --2- 172.21.15.94:0/2873085013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18a4073630 0x7f18a40b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a2a00640 1 -- 172.21.15.94:0/2873085013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a40af200 con 0x7f18a4070d80 2024-09-10T23:35:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f189a000640 1 --2- 172.21.15.94:0/2873085013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18a4073630 0x7f18a40b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:35:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a3400640 1 --2- 172.21.15.94:0/2873085013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18a41455f0 0x7f18a40b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:35:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a2a00640 1 --2- 172.21.15.94:0/2873085013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18a4070d80 0x7f18a4140f50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f189400b410 tx=0x7f189400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f189b400640 1 -- 172.21.15.94:0/2873085013 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894015020 con 0x7f18a4070d80 2024-09-10T23:35:27.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f189b400640 1 -- 172.21.15.94:0/2873085013 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1894011920 con 0x7f18a4070d80 2024-09-10T23:35:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f189b400640 1 -- 172.21.15.94:0/2873085013 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894011d70 con 0x7f18a4070d80 2024-09-10T23:35:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a9800640 1 -- 172.21.15.94:0/2873085013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a40af4e0 con 0x7f18a4070d80 2024-09-10T23:35:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a9800640 1 -- 172.21.15.94:0/2873085013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a40afaa0 con 0x7f18a4070d80 2024-09-10T23:35:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.819+0000 7f18a9800640 1 -- 172.21.15.94:0/2873085013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a4070840 con 0x7f18a4070d80 2024-09-10T23:35:27.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.823+0000 7f189b400640 1 -- 172.21.15.94:0/2873085013 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f189401a400 con 0x7f18a4070d80 2024-09-10T23:35:27.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.823+0000 7f189b400640 1 --2- 172.21.15.94:0/2873085013 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1880072790 0x7f1880074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:27.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.823+0000 7f189b400640 1 -- 172.21.15.94:0/2873085013 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f18940936e0 con 0x7f18a4070d80 2024-09-10T23:35:27.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.823+0000 7f189a000640 1 --2- 172.21.15.94:0/2873085013 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1880072790 0x7f1880074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:27.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.823+0000 7f189a000640 1 --2- 172.21.15.94:0/2873085013 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1880072790 0x7f1880074c50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f188c0af000 tx=0x7f188c01afb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:27.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.827+0000 7f189b400640 1 -- 172.21.15.94:0/2873085013 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f189401f020 con 0x7f18a4070d80 2024-09-10T23:35:27.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.991+0000 7f18a9800640 1 -- 172.21.15.94:0/2873085013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f18a40d1370 con 0x7f18a4070d80 2024-09-10T23:35:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.995+0000 7f189b400640 1 -- 172.21.15.94:0/2873085013 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v31) ==== 71+0+0 (secure 0 0 0) 0x7f189405ec90 con 0x7f18a4070d80 2024-09-10T23:35:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.999+0000 7f18a9800640 1 -- 172.21.15.94:0/2873085013 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1880072790 msgr2=0x7f1880074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.999+0000 7f18a9800640 1 --2- 172.21.15.94:0/2873085013 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1880072790 0x7f1880074c50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f188c0af000 tx=0x7f188c01afb0 comp rx=0 tx=0).stop 2024-09-10T23:35:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.999+0000 7f18a9800640 1 -- 172.21.15.94:0/2873085013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18a4070d80 msgr2=0x7f18a4140f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.999+0000 7f18a9800640 1 --2- 172.21.15.94:0/2873085013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18a4070d80 0x7f18a4140f50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f189400b410 tx=0x7f189400b8d0 comp rx=0 tx=0).stop 2024-09-10T23:35:27.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.999+0000 7f18a9800640 1 -- 172.21.15.94:0/2873085013 shutdown_connections 2024-09-10T23:35:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.999+0000 7f18a9800640 1 --2- 172.21.15.94:0/2873085013 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1880072790 0x7f1880074c50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.999+0000 7f18a9800640 1 --2- 172.21.15.94:0/2873085013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18a4073630 0x7f18a40b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.999+0000 7f18a9800640 1 --2- 172.21.15.94:0/2873085013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18a41455f0 0x7f18a40b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.999+0000 7f18a9800640 1 --2- 172.21.15.94:0/2873085013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18a4070d80 0x7f18a4140f50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.999+0000 7f18a9800640 1 -- 172.21.15.94:0/2873085013 >> 172.21.15.94:0/2873085013 conn(0x7f18a40d0d80 msgr2=0x7f18a4142a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.999+0000 7f18a9800640 1 -- 172.21.15.94:0/2873085013 shutdown_connections 2024-09-10T23:35:28.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:27.999+0000 7f18a9800640 1 -- 172.21.15.94:0/2873085013 wait complete. 2024-09-10T23:35:28.018 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-10T23:35:28.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.159+0000 7ff7bfe00640 1 -- 172.21.15.94:0/1192996786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c015bb30 msgr2=0x7ff7c00d19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:28.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.159+0000 7ff7bfe00640 1 --2- 172.21.15.94:0/1192996786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c015bb30 0x7ff7c00d19c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009900 tx=0x7ff7b001c700 comp rx=0 tx=0).stop 2024-09-10T23:35:28.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.159+0000 7ff7bfe00640 1 -- 172.21.15.94:0/1192996786 shutdown_connections 2024-09-10T23:35:28.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.159+0000 7ff7bfe00640 1 --2- 172.21.15.94:0/1192996786 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7c00b3020 0x7ff7c00d1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.159+0000 7ff7bfe00640 1 --2- 172.21.15.94:0/1192996786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c015bb30 0x7ff7c00d19c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.159+0000 7ff7bfe00640 1 -- 172.21.15.94:0/1192996786 >> 172.21.15.94:0/1192996786 conn(0x7ff7c009aa70 msgr2=0x7ff7c009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:28.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.159+0000 7ff7bfe00640 1 -- 172.21.15.94:0/1192996786 shutdown_connections 2024-09-10T23:35:28.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.159+0000 7ff7bfe00640 1 -- 172.21.15.94:0/1192996786 wait complete. 2024-09-10T23:35:28.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7bfe00640 1 Processor -- start 2024-09-10T23:35:28.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7bfe00640 1 -- start start 2024-09-10T23:35:28.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7bfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7c00b1980 0x7ff7c01fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:28.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7bfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00b3020 0x7ff7c006dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:28.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7bfe00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7c015bb30 0x7ff7c00cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:28.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7bfe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c00b0ec0 con 0x7ff7c00b3020 2024-09-10T23:35:28.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7bfe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c00d0240 con 0x7ff7c00b1980 2024-09-10T23:35:28.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7bfe00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c0168f10 con 0x7ff7c015bb30 2024-09-10T23:35:28.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7b6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00b3020 0x7ff7c006dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:28.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7b6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00b3020 0x7ff7c006dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60352/0 (socket says 172.21.15.94:60352) 2024-09-10T23:35:28.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7b6000640 1 -- 172.21.15.94:0/2378475119 learned_addr learned my addr 172.21.15.94:0/2378475119 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:28.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7bea00640 1 --2- 172.21.15.94:0/2378475119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7c00b1980 0x7ff7c01fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:28.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7bf400640 1 --2- 172.21.15.94:0/2378475119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7c015bb30 0x7ff7c00cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:28.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7b6000640 1 -- 172.21.15.94:0/2378475119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7c015bb30 msgr2=0x7ff7c00cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:28.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7b6000640 1 --2- 172.21.15.94:0/2378475119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7c015bb30 0x7ff7c00cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7b6000640 1 -- 172.21.15.94:0/2378475119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7c00b1980 msgr2=0x7ff7c01fb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:28.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7b6000640 1 --2- 172.21.15.94:0/2378475119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7c00b1980 0x7ff7c01fb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7b6000640 1 -- 172.21.15.94:0/2378475119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c00cb020 con 0x7ff7c00b3020 2024-09-10T23:35:28.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7b6000640 1 --2- 172.21.15.94:0/2378475119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00b3020 0x7ff7c006dd30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800c9a0 tx=0x7ff7a800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:28.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7b7e00640 1 -- 172.21.15.94:0/2378475119 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8007c70 con 0x7ff7c00b3020 2024-09-10T23:35:28.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7b7e00640 1 -- 172.21.15.94:0/2378475119 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7a800f450 con 0x7ff7c00b3020 2024-09-10T23:35:28.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7bfe00640 1 -- 172.21.15.94:0/2378475119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c009f4f0 con 0x7ff7c00b3020 2024-09-10T23:35:28.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7b7e00640 1 -- 172.21.15.94:0/2378475119 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a800f8a0 con 0x7ff7c00b3020 2024-09-10T23:35:28.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7bfe00640 1 -- 172.21.15.94:0/2378475119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c00c98d0 con 0x7ff7c00b3020 2024-09-10T23:35:28.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.163+0000 7ff7bfe00640 1 -- 172.21.15.94:0/2378475119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c00b1e30 con 0x7ff7c00b3020 2024-09-10T23:35:28.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.167+0000 7ff7b7e00640 1 -- 172.21.15.94:0/2378475119 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7ff7a8020030 con 0x7ff7c00b3020 2024-09-10T23:35:28.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.167+0000 7ff7b7e00640 1 --2- 172.21.15.94:0/2378475119 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff79c072790 0x7ff79c074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:28.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.167+0000 7ff7b7e00640 1 -- 172.21.15.94:0/2378475119 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7ff7a8093880 con 0x7ff7c00b3020 2024-09-10T23:35:28.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.167+0000 7ff7bea00640 1 --2- 172.21.15.94:0/2378475119 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff79c072790 0x7ff79c074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:28.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.167+0000 7ff7bea00640 1 --2- 172.21.15.94:0/2378475119 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff79c072790 0x7ff79c074c50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00ae000 tx=0x7ff7b001af40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:28.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.171+0000 7ff7b7e00640 1 -- 172.21.15.94:0/2378475119 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff7c00cb020 con 0x7ff7c00b3020 2024-09-10T23:35:28.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.395+0000 7ff7bfe00640 1 -- 172.21.15.94:0/2378475119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7ff7c00b0660 con 0x7ff7c00b3020 2024-09-10T23:35:28.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.647+0000 7ff7b7e00640 1 -- 172.21.15.94:0/2378475119 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v159) ==== 140+0+0 (secure 0 0 0) 0x7ff7a8014030 con 0x7ff7c00b3020 2024-09-10T23:35:28.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.651+0000 7ff7bfe00640 1 -- 172.21.15.94:0/2378475119 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff79c072790 msgr2=0x7ff79c074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:28.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.651+0000 7ff7bfe00640 1 --2- 172.21.15.94:0/2378475119 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff79c072790 0x7ff79c074c50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00ae000 tx=0x7ff7b001af40 comp rx=0 tx=0).stop 2024-09-10T23:35:28.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.651+0000 7ff7bfe00640 1 -- 172.21.15.94:0/2378475119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00b3020 msgr2=0x7ff7c006dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:28.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.651+0000 7ff7bfe00640 1 --2- 172.21.15.94:0/2378475119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00b3020 0x7ff7c006dd30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800c9a0 tx=0x7ff7a800ce60 comp rx=0 tx=0).stop 2024-09-10T23:35:28.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.651+0000 7ff7bfe00640 1 -- 172.21.15.94:0/2378475119 shutdown_connections 2024-09-10T23:35:28.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.651+0000 7ff7bfe00640 1 --2- 172.21.15.94:0/2378475119 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff79c072790 0x7ff79c074c50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.651+0000 7ff7bfe00640 1 --2- 172.21.15.94:0/2378475119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7c015bb30 0x7ff7c00cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.651+0000 7ff7bfe00640 1 --2- 172.21.15.94:0/2378475119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c00b3020 0x7ff7c006dd30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.651+0000 7ff7bfe00640 1 --2- 172.21.15.94:0/2378475119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7c00b1980 0x7ff7c01fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.651+0000 7ff7bfe00640 1 -- 172.21.15.94:0/2378475119 >> 172.21.15.94:0/2378475119 conn(0x7ff7c009aa70 msgr2=0x7ff7c00cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:28.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.651+0000 7ff7bfe00640 1 -- 172.21.15.94:0/2378475119 shutdown_connections 2024-09-10T23:35:28.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.651+0000 7ff7bfe00640 1 -- 172.21.15.94:0/2378475119 wait complete. 2024-09-10T23:35:28.672 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-10T23:35:28.673 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2024-09-10T23:35:28.673 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T23:35:28.673 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:35:28.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 -- 172.21.15.94:0/1259755820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07140d02e0 msgr2=0x7f071415e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:28.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 --2- 172.21.15.94:0/1259755820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07140d02e0 0x7f071415e770 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f07140d0b80 tx=0x7f070001c890 comp rx=0 tx=0).stop 2024-09-10T23:35:28.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 -- 172.21.15.94:0/1259755820 shutdown_connections 2024-09-10T23:35:28.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 --2- 172.21.15.94:0/1259755820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f071415ecb0 0x7f07140b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 --2- 172.21.15.94:0/1259755820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07140d02e0 0x7f071415e770 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 --2- 172.21.15.94:0/1259755820 >> v2:172.21.15.181:3300/0 conn(0x7f0714073ff0 0x7f07140cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 -- 172.21.15.94:0/1259755820 >> 172.21.15.94:0/1259755820 conn(0x7f071406cdf0 msgr2=0x7f071406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:28.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 -- 172.21.15.94:0/1259755820 shutdown_connections 2024-09-10T23:35:28.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 -- 172.21.15.94:0/1259755820 wait complete. 2024-09-10T23:35:28.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 Processor -- start 2024-09-10T23:35:28.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 -- start start 2024-09-10T23:35:28.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0714073ff0 0x7f0714164180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:28.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07140d02e0 0x7f0714162360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:28.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f071415ecb0 0x7f0714138200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:28.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f071406f840 con 0x7f07140d02e0 2024-09-10T23:35:28.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f071406f6c0 con 0x7f0714073ff0 2024-09-10T23:35:28.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0718a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f071406f9c0 con 0x7f071415ecb0 2024-09-10T23:35:28.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0711400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07140d02e0 0x7f0714162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:28.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0711400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07140d02e0 0x7f0714162360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60366/0 (socket says 172.21.15.94:60366) 2024-09-10T23:35:28.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0711400640 1 -- 172.21.15.94:0/934797697 learned_addr learned my addr 172.21.15.94:0/934797697 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:28.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0712800640 1 --2- 172.21.15.94:0/934797697 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f071415ecb0 0x7f0714138200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:28.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0711e00640 1 --2- 172.21.15.94:0/934797697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0714073ff0 0x7f0714164180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:28.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0711400640 1 -- 172.21.15.94:0/934797697 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f071415ecb0 msgr2=0x7f0714138200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:28.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0711400640 1 --2- 172.21.15.94:0/934797697 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f071415ecb0 0x7f0714138200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0711400640 1 -- 172.21.15.94:0/934797697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0714073ff0 msgr2=0x7f0714164180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:28.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0711400640 1 --2- 172.21.15.94:0/934797697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0714073ff0 0x7f0714164180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.815+0000 7f0711400640 1 -- 172.21.15.94:0/934797697 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071414dd40 con 0x7f07140d02e0 2024-09-10T23:35:28.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.819+0000 7f0711400640 1 --2- 172.21.15.94:0/934797697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07140d02e0 0x7f0714162360 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f070001af40 tx=0x7f0700004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:28.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.819+0000 7f070a000640 1 -- 172.21.15.94:0/934797697 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07000ba020 con 0x7f07140d02e0 2024-09-10T23:35:28.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.819+0000 7f070a000640 1 -- 172.21.15.94:0/934797697 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0700005030 con 0x7f07140d02e0 2024-09-10T23:35:28.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.819+0000 7f070a000640 1 -- 172.21.15.94:0/934797697 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07000b2040 con 0x7f07140d02e0 2024-09-10T23:35:28.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.819+0000 7f0718a00640 1 -- 172.21.15.94:0/934797697 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071414e020 con 0x7f07140d02e0 2024-09-10T23:35:28.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.819+0000 7f0718a00640 1 -- 172.21.15.94:0/934797697 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f071414e5e0 con 0x7f07140d02e0 2024-09-10T23:35:28.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.819+0000 7f0718a00640 1 -- 172.21.15.94:0/934797697 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f071415e770 con 0x7f07140d02e0 2024-09-10T23:35:28.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.819+0000 7f070a000640 1 -- 172.21.15.94:0/934797697 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f0700005480 con 0x7f07140d02e0 2024-09-10T23:35:28.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.819+0000 7f070a000640 1 --2- 172.21.15.94:0/934797697 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f06f4072790 0x7f06f4074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:28.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.819+0000 7f070a000640 1 -- 172.21.15.94:0/934797697 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f070012d3d0 con 0x7f07140d02e0 2024-09-10T23:35:28.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.819+0000 7f0711e00640 1 --2- 172.21.15.94:0/934797697 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f06f4072790 0x7f06f4074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:28.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.823+0000 7f0711e00640 1 --2- 172.21.15.94:0/934797697 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f06f4072790 0x7f06f4074c50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0704005950 tx=0x7f070400a4e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:28.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.823+0000 7f070a000640 1 -- 172.21.15.94:0/934797697 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f07000c4060 con 0x7f07140d02e0 2024-09-10T23:35:28.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.987+0000 7f0718a00640 1 -- 172.21.15.94:0/934797697 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f071415d350 con 0x7f07140d02e0 2024-09-10T23:35:28.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.991+0000 7f070a000640 1 -- 172.21.15.94:0/934797697 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+13022 (secure 0 0 0) 0x7f07000164b0 con 0x7f07140d02e0 2024-09-10T23:35:28.991 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:28.991 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":49,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:35:27.643116+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6837/1573303541":"2024-09-11T23:35:24.632789+0000","172.21.15.181:6836/1573303541":"2024-09-11T23:35:24.632789+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:35:28.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.991+0000 7f0718a00640 1 -- 172.21.15.94:0/934797697 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f06f4072790 msgr2=0x7f06f4074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:28.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.991+0000 7f0718a00640 1 --2- 172.21.15.94:0/934797697 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f06f4072790 0x7f06f4074c50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0704005950 tx=0x7f070400a4e0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.991+0000 7f0718a00640 1 -- 172.21.15.94:0/934797697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07140d02e0 msgr2=0x7f0714162360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:28.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.991+0000 7f0718a00640 1 --2- 172.21.15.94:0/934797697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07140d02e0 0x7f0714162360 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f070001af40 tx=0x7f0700004040 comp rx=0 tx=0).stop 2024-09-10T23:35:28.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.991+0000 7f0718a00640 1 -- 172.21.15.94:0/934797697 shutdown_connections 2024-09-10T23:35:28.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.991+0000 7f0718a00640 1 --2- 172.21.15.94:0/934797697 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f06f4072790 0x7f06f4074c50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.991+0000 7f0718a00640 1 --2- 172.21.15.94:0/934797697 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f071415ecb0 0x7f0714138200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.991+0000 7f0718a00640 1 --2- 172.21.15.94:0/934797697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07140d02e0 0x7f0714162360 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.991+0000 7f0718a00640 1 --2- 172.21.15.94:0/934797697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0714073ff0 0x7f0714164180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:28.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.991+0000 7f0718a00640 1 -- 172.21.15.94:0/934797697 >> 172.21.15.94:0/934797697 conn(0x7f071406cdf0 msgr2=0x7f0714072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:28.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.991+0000 7f0718a00640 1 -- 172.21.15.94:0/934797697 shutdown_connections 2024-09-10T23:35:28.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:28.991+0000 7f0718a00640 1 -- 172.21.15.94:0/934797697 wait complete. 2024-09-10T23:35:29.012 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T23:32:31.372521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T23:35:29.012 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T23:35:29.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.163+0000 7efee0000640 1 -- 172.21.15.94:0/2749964916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed815bf60 msgr2=0x7efed80d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:29.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.163+0000 7efee0000640 1 --2- 172.21.15.94:0/2749964916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed815bf60 0x7efed80d2780 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efecc009bf0 tx=0x7efecc01cb10 comp rx=0 tx=0).stop 2024-09-10T23:35:29.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.163+0000 7efee0000640 1 -- 172.21.15.94:0/2749964916 shutdown_connections 2024-09-10T23:35:29.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.163+0000 7efee0000640 1 --2- 172.21.15.94:0/2749964916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed815bf60 0x7efed80d2780 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:29.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.163+0000 7efee0000640 1 -- 172.21.15.94:0/2749964916 >> 172.21.15.94:0/2749964916 conn(0x7efed80d1c60 msgr2=0x7efed80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:29.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.163+0000 7efee0000640 1 -- 172.21.15.94:0/2749964916 shutdown_connections 2024-09-10T23:35:29.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.163+0000 7efee0000640 1 -- 172.21.15.94:0/2749964916 wait complete. 2024-09-10T23:35:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efee0000640 1 Processor -- start 2024-09-10T23:35:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efee0000640 1 -- start start 2024-09-10T23:35:29.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efee0000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed80cfce0 0x7efed81584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:29.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efee0000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed80d0830 0x7efed809e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:29.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efee0000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed815bf60 0x7efed809e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:29.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efee0000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efed8169990 con 0x7efed80d0830 2024-09-10T23:35:29.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efee0000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efed8169b10 con 0x7efed815bf60 2024-09-10T23:35:29.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efee0000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7efed8169c90 con 0x7efed80cfce0 2024-09-10T23:35:29.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efedcc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed80d0830 0x7efed809e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:29.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efedcc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed80d0830 0x7efed809e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60368/0 (socket says 172.21.15.94:60368) 2024-09-10T23:35:29.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efedcc00640 1 -- 172.21.15.94:0/145081728 learned_addr learned my addr 172.21.15.94:0/145081728 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:29.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efedd600640 1 --2- 172.21.15.94:0/145081728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed80cfce0 0x7efed81584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:29.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efede000640 1 --2- 172.21.15.94:0/145081728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed815bf60 0x7efed809e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:29.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efedcc00640 1 -- 172.21.15.94:0/145081728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed80cfce0 msgr2=0x7efed81584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:29.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efedcc00640 1 --2- 172.21.15.94:0/145081728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed80cfce0 0x7efed81584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:29.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efedcc00640 1 -- 172.21.15.94:0/145081728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed815bf60 msgr2=0x7efed809e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:29.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efedcc00640 1 --2- 172.21.15.94:0/145081728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed815bf60 0x7efed809e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:29.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efedcc00640 1 -- 172.21.15.94:0/145081728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed8158a20 con 0x7efed80d0830 2024-09-10T23:35:29.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efede000640 1 --2- 172.21.15.94:0/145081728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed815bf60 0x7efed809e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:35:29.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efedcc00640 1 --2- 172.21.15.94:0/145081728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed80d0830 0x7efed809e0a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7efec400c4d0 tx=0x7efec400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:29.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efed5600640 1 -- 172.21.15.94:0/145081728 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4016020 con 0x7efed80d0830 2024-09-10T23:35:29.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efed5600640 1 -- 172.21.15.94:0/145081728 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efec4012920 con 0x7efed80d0830 2024-09-10T23:35:29.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efee0000640 1 -- 172.21.15.94:0/145081728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed80affc0 con 0x7efed80d0830 2024-09-10T23:35:29.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efed5600640 1 -- 172.21.15.94:0/145081728 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4012d70 con 0x7efed80d0830 2024-09-10T23:35:29.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efee0000640 1 -- 172.21.15.94:0/145081728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed809eef0 con 0x7efed80d0830 2024-09-10T23:35:29.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.167+0000 7efee0000640 1 -- 172.21.15.94:0/145081728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed80d2780 con 0x7efed80d0830 2024-09-10T23:35:29.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.171+0000 7efed5600640 1 -- 172.21.15.94:0/145081728 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7efec401b400 con 0x7efed80d0830 2024-09-10T23:35:29.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.171+0000 7efed5600640 1 --2- 172.21.15.94:0/145081728 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efeb8072790 0x7efeb8074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:29.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.171+0000 7efed5600640 1 -- 172.21.15.94:0/145081728 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7efec40936e0 con 0x7efed80d0830 2024-09-10T23:35:29.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.171+0000 7efedd600640 1 --2- 172.21.15.94:0/145081728 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efeb8072790 0x7efeb8074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:29.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.171+0000 7efedd600640 1 --2- 172.21.15.94:0/145081728 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efeb8072790 0x7efeb8074c50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efecc0af000 tx=0x7efecc01b350 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:29.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.175+0000 7efed5600640 1 -- 172.21.15.94:0/145081728 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efec4020020 con 0x7efed80d0830 2024-09-10T23:35:29.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.339+0000 7efee0000640 1 -- 172.21.15.94:0/145081728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efed80cdfa0 con 0x7efed80d0830 2024-09-10T23:35:29.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.339+0000 7efed5600640 1 -- 172.21.15.94:0/145081728 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) ==== 93+0+10 (secure 0 0 0) 0x7efec405eca0 con 0x7efed80d0830 2024-09-10T23:35:29.341 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-09-10T23:35:29.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.343+0000 7efee0000640 1 -- 172.21.15.94:0/145081728 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efeb8072790 msgr2=0x7efeb8074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:29.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.343+0000 7efee0000640 1 --2- 172.21.15.94:0/145081728 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efeb8072790 0x7efeb8074c50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efecc0af000 tx=0x7efecc01b350 comp rx=0 tx=0).stop 2024-09-10T23:35:29.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.343+0000 7efee0000640 1 -- 172.21.15.94:0/145081728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed80d0830 msgr2=0x7efed809e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:29.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.343+0000 7efee0000640 1 --2- 172.21.15.94:0/145081728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed80d0830 0x7efed809e0a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7efec400c4d0 tx=0x7efec400c990 comp rx=0 tx=0).stop 2024-09-10T23:35:29.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.343+0000 7efee0000640 1 -- 172.21.15.94:0/145081728 shutdown_connections 2024-09-10T23:35:29.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.343+0000 7efee0000640 1 --2- 172.21.15.94:0/145081728 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efeb8072790 0x7efeb8074c50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:29.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.343+0000 7efee0000640 1 --2- 172.21.15.94:0/145081728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed815bf60 0x7efed809e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:29.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.343+0000 7efee0000640 1 --2- 172.21.15.94:0/145081728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed80d0830 0x7efed809e0a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:29.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.343+0000 7efee0000640 1 --2- 172.21.15.94:0/145081728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed80cfce0 0x7efed81584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:29.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.343+0000 7efee0000640 1 -- 172.21.15.94:0/145081728 >> 172.21.15.94:0/145081728 conn(0x7efed80d1c60 msgr2=0x7efed80d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:29.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.343+0000 7efee0000640 1 -- 172.21.15.94:0/145081728 shutdown_connections 2024-09-10T23:35:29.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.343+0000 7efee0000640 1 -- 172.21.15.94:0/145081728 wait complete. 2024-09-10T23:35:29.364 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-10T23:35:29.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.503+0000 7f5a85a00640 1 -- 172.21.15.94:0/1851893933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a64003fe0 msgr2=0x7f5a64004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:29.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.503+0000 7f5a85a00640 1 --2- 172.21.15.94:0/1851893933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a64003fe0 0x7f5a64004810 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5a8006d8b0 tx=0x7f5a6801d1d0 comp rx=0 tx=0).stop 2024-09-10T23:35:29.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.503+0000 7f5a7f400640 1 -- 172.21.15.94:0/1851893933 reap_dead start 2024-09-10T23:35:29.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a85a00640 1 -- 172.21.15.94:0/1851893933 shutdown_connections 2024-09-10T23:35:29.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a85a00640 1 -- 172.21.15.94:0/1851893933 >> 172.21.15.94:0/1851893933 conn(0x7f5a800d1c60 msgr2=0x7f5a800d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:29.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a85a00640 1 -- 172.21.15.94:0/1851893933 shutdown_connections 2024-09-10T23:35:29.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a85a00640 1 -- 172.21.15.94:0/1851893933 wait complete. 2024-09-10T23:35:29.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a85a00640 1 Processor -- start 2024-09-10T23:35:29.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a85a00640 1 -- start start 2024-09-10T23:35:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a85a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a64003fe0 0x7f5a801f8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a85a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a80153940 0x7f5a801622b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a85a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a8015b2c0 0x7f5a8015b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a85a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a800b05c0 con 0x7f5a64003fe0 2024-09-10T23:35:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a85a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5a800b0440 con 0x7f5a80153940 2024-09-10T23:35:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a85a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5a80166aa0 con 0x7f5a8015b2c0 2024-09-10T23:35:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a7ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a64003fe0 0x7f5a801f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a7ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a64003fe0 0x7f5a801f8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60384/0 (socket says 172.21.15.94:60384) 2024-09-10T23:35:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a7ea00640 1 -- 172.21.15.94:0/1296723080 learned_addr learned my addr 172.21.15.94:0/1296723080 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a76000640 1 --2- 172.21.15.94:0/1296723080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a80153940 0x7f5a801622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a7f400640 1 --2- 172.21.15.94:0/1296723080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a8015b2c0 0x7f5a8015b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a7ea00640 1 -- 172.21.15.94:0/1296723080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a8015b2c0 msgr2=0x7f5a8015b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a7ea00640 1 --2- 172.21.15.94:0/1296723080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a8015b2c0 0x7f5a8015b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a7ea00640 1 -- 172.21.15.94:0/1296723080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a80153940 msgr2=0x7f5a801622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a7ea00640 1 --2- 172.21.15.94:0/1296723080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a80153940 0x7f5a801622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a7ea00640 1 -- 172.21.15.94:0/1296723080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a801f8ab0 con 0x7f5a64003fe0 2024-09-10T23:35:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a7ea00640 1 --2- 172.21.15.94:0/1296723080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a64003fe0 0x7f5a801f8570 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5a70006600 tx=0x7f5a700039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a77400640 1 -- 172.21.15.94:0/1296723080 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70015070 con 0x7f5a64003fe0 2024-09-10T23:35:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a77400640 1 -- 172.21.15.94:0/1296723080 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a70011920 con 0x7f5a64003fe0 2024-09-10T23:35:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a77400640 1 -- 172.21.15.94:0/1296723080 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70011d70 con 0x7f5a64003fe0 2024-09-10T23:35:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a85a00640 1 -- 172.21.15.94:0/1296723080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a80153e50 con 0x7f5a64003fe0 2024-09-10T23:35:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.507+0000 7f5a85a00640 1 -- 172.21.15.94:0/1296723080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a801f9270 con 0x7f5a64003fe0 2024-09-10T23:35:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.511+0000 7f5a85a00640 1 -- 172.21.15.94:0/1296723080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a800af7c0 con 0x7f5a64003fe0 2024-09-10T23:35:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.511+0000 7f5a77400640 1 -- 172.21.15.94:0/1296723080 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f5a700103c0 con 0x7f5a64003fe0 2024-09-10T23:35:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.511+0000 7f5a77400640 1 --2- 172.21.15.94:0/1296723080 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5a5c072790 0x7f5a5c074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:29.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.511+0000 7f5a77400640 1 -- 172.21.15.94:0/1296723080 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f5a70094470 con 0x7f5a64003fe0 2024-09-10T23:35:29.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.511+0000 7f5a76000640 1 --2- 172.21.15.94:0/1296723080 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5a5c072790 0x7f5a5c074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:29.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.511+0000 7f5a76000640 1 --2- 172.21.15.94:0/1296723080 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5a5c072790 0x7f5a5c074c50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5a680af000 tx=0x7f5a680a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.515+0000 7f5a77400640 1 -- 172.21.15.94:0/1296723080 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5a7005fab0 con 0x7f5a64003fe0 2024-09-10T23:35:29.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:29.743+0000 7f5a85a00640 1 -- 172.21.15.94:0/1296723080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f5a800749c0 con 0x7f5a64003fe0 2024-09-10T23:35:30.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.655+0000 7f5a77400640 1 -- 172.21.15.94:0/1296723080 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v161) ==== 164+0+0 (secure 0 0 0) 0x7f5a7005f3b0 con 0x7f5a64003fe0 2024-09-10T23:35:30.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.659+0000 7f5a85a00640 1 -- 172.21.15.94:0/1296723080 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5a5c072790 msgr2=0x7f5a5c074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:30.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.659+0000 7f5a85a00640 1 --2- 172.21.15.94:0/1296723080 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5a5c072790 0x7f5a5c074c50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5a680af000 tx=0x7f5a680a7040 comp rx=0 tx=0).stop 2024-09-10T23:35:30.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.659+0000 7f5a85a00640 1 -- 172.21.15.94:0/1296723080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a64003fe0 msgr2=0x7f5a801f8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:30.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.659+0000 7f5a85a00640 1 --2- 172.21.15.94:0/1296723080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a64003fe0 0x7f5a801f8570 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5a70006600 tx=0x7f5a700039d0 comp rx=0 tx=0).stop 2024-09-10T23:35:30.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.659+0000 7f5a85a00640 1 -- 172.21.15.94:0/1296723080 shutdown_connections 2024-09-10T23:35:30.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.659+0000 7f5a85a00640 1 --2- 172.21.15.94:0/1296723080 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5a5c072790 0x7f5a5c074c50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:30.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.659+0000 7f5a85a00640 1 --2- 172.21.15.94:0/1296723080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a80153940 0x7f5a801622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:30.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.659+0000 7f5a85a00640 1 --2- 172.21.15.94:0/1296723080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a8015b2c0 0x7f5a8015b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:30.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.659+0000 7f5a85a00640 1 --2- 172.21.15.94:0/1296723080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a64003fe0 0x7f5a801f8570 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:30.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.659+0000 7f5a85a00640 1 -- 172.21.15.94:0/1296723080 >> 172.21.15.94:0/1296723080 conn(0x7f5a800d1c60 msgr2=0x7f5a80071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:30.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.659+0000 7f5a85a00640 1 -- 172.21.15.94:0/1296723080 shutdown_connections 2024-09-10T23:35:30.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.659+0000 7f5a85a00640 1 -- 172.21.15.94:0/1296723080 wait complete. 2024-09-10T23:35:30.682 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T23:35:30.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.823+0000 7f55dda00640 1 -- 172.21.15.94:0/2931322047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55d80b0320 msgr2=0x7f55d8166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:30.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.823+0000 7f55dda00640 1 --2- 172.21.15.94:0/2931322047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55d80b0320 0x7f55d8166aa0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009840 tx=0x7f55d001c700 comp rx=0 tx=0).stop 2024-09-10T23:35:30.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.823+0000 7f55dda00640 1 -- 172.21.15.94:0/2931322047 shutdown_connections 2024-09-10T23:35:30.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.823+0000 7f55dda00640 1 --2- 172.21.15.94:0/2931322047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55d80b0320 0x7f55d8166aa0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:30.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.823+0000 7f55dda00640 1 -- 172.21.15.94:0/2931322047 >> 172.21.15.94:0/2931322047 conn(0x7f55d80d1c60 msgr2=0x7f55d80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:30.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.823+0000 7f55dda00640 1 -- 172.21.15.94:0/2931322047 shutdown_connections 2024-09-10T23:35:30.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.823+0000 7f55dda00640 1 -- 172.21.15.94:0/2931322047 wait complete. 2024-09-10T23:35:30.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55dda00640 1 Processor -- start 2024-09-10T23:35:30.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55dda00640 1 -- start start 2024-09-10T23:35:30.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55dda00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d80b0320 0x7f55d814bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:30.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55dda00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55d80b2bb0 0x7f55d815a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:30.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55dda00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55d815f5c0 0x7f55d81fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:30.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55dda00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8169eb0 con 0x7f55d80b2bb0 2024-09-10T23:35:30.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55dda00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8169d30 con 0x7f55d80b0320 2024-09-10T23:35:30.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55dda00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f55d816a030 con 0x7f55d815f5c0 2024-09-10T23:35:30.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55d6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55d80b2bb0 0x7f55d815a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:30.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55d6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55d80b2bb0 0x7f55d815a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60398/0 (socket says 172.21.15.94:60398) 2024-09-10T23:35:30.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55d6000640 1 -- 172.21.15.94:0/3251340435 learned_addr learned my addr 172.21.15.94:0/3251340435 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:30.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55d6a00640 1 --2- 172.21.15.94:0/3251340435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d80b0320 0x7f55d814bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55d7400640 1 --2- 172.21.15.94:0/3251340435 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55d815f5c0 0x7f55d81fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55d6000640 1 -- 172.21.15.94:0/3251340435 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55d815f5c0 msgr2=0x7f55d81fbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55d6000640 1 --2- 172.21.15.94:0/3251340435 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55d815f5c0 0x7f55d81fbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55d6000640 1 -- 172.21.15.94:0/3251340435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d80b0320 msgr2=0x7f55d814bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55d6000640 1 --2- 172.21.15.94:0/3251340435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d80b0320 0x7f55d814bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55d6000640 1 -- 172.21.15.94:0/3251340435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d814b620 con 0x7f55d80b2bb0 2024-09-10T23:35:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55d6000640 1 --2- 172.21.15.94:0/3251340435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55d80b2bb0 0x7f55d815a650 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f55c400c450 tx=0x7f55c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55cea00640 1 -- 172.21.15.94:0/3251340435 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4016020 con 0x7f55d80b2bb0 2024-09-10T23:35:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55cea00640 1 -- 172.21.15.94:0/3251340435 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55c4012920 con 0x7f55d80b2bb0 2024-09-10T23:35:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55cea00640 1 -- 172.21.15.94:0/3251340435 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4012d70 con 0x7f55d80b2bb0 2024-09-10T23:35:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.827+0000 7f55dda00640 1 -- 172.21.15.94:0/3251340435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d81523e0 con 0x7f55d80b2bb0 2024-09-10T23:35:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.831+0000 7f55dda00640 1 -- 172.21.15.94:0/3251340435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d81529a0 con 0x7f55d80b2bb0 2024-09-10T23:35:30.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.831+0000 7f55dda00640 1 -- 172.21.15.94:0/3251340435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d80cfb60 con 0x7f55d80b2bb0 2024-09-10T23:35:30.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.831+0000 7f55cea00640 1 -- 172.21.15.94:0/3251340435 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f55c401b400 con 0x7f55d80b2bb0 2024-09-10T23:35:30.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.831+0000 7f55cea00640 1 --2- 172.21.15.94:0/3251340435 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f55b8072790 0x7f55b8074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:30.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.831+0000 7f55cea00640 1 -- 172.21.15.94:0/3251340435 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f55c4093640 con 0x7f55d80b2bb0 2024-09-10T23:35:30.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.831+0000 7f55d6a00640 1 --2- 172.21.15.94:0/3251340435 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f55b8072790 0x7f55b8074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:30.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.835+0000 7f55d6a00640 1 --2- 172.21.15.94:0/3251340435 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f55b8072790 0x7f55b8074c50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009f60 tx=0x7f55c80094f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:30.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:30.835+0000 7f55cea00640 1 -- 172.21.15.94:0/3251340435 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f55c4020020 con 0x7f55d80b2bb0 2024-09-10T23:35:31.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.003+0000 7f55dda00640 1 -- 172.21.15.94:0/3251340435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f55d80749c0 con 0x7f55d80b2bb0 2024-09-10T23:35:31.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.003+0000 7f55cea00640 1 -- 172.21.15.94:0/3251340435 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+1998 (secure 0 0 0) 0x7f55c405ec90 con 0x7f55d80b2bb0 2024-09-10T23:35:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T23:35:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T23:35:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T23:35:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T23:35:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3944.074506 2024-09-10T23:35:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3865.464527 2024-09-10T23:35:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3898.861024 2024-09-10T23:35:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4010.048529 2024-09-10T23:35:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4650.562436 2024-09-10T23:35:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4724.018776 2024-09-10T23:35:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5255.753663 2024-09-10T23:35:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4718.009443 2024-09-10T23:35:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds 20/20 2024-09-10T23:35:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T23:35:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_ms 2/2 2024-09-10T23:35:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_frag true 2024-09-10T23:35:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_scatterstat true 2024-09-10T23:35:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T23:35:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_verify_scatter true 2024-09-10T23:35:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T23:35:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T23:35:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T23:35:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced client_mount_timeout 600 2024-09-10T23:35:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_client 20/20 2024-09-10T23:35:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_ms 1/1 2024-09-10T23:35:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T23:35:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T23:35:31.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.003+0000 7f55dda00640 1 -- 172.21.15.94:0/3251340435 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f55b8072790 msgr2=0x7f55b8074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:31.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.003+0000 7f55dda00640 1 --2- 172.21.15.94:0/3251340435 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f55b8072790 0x7f55b8074c50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009f60 tx=0x7f55c80094f0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.003+0000 7f55dda00640 1 -- 172.21.15.94:0/3251340435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55d80b2bb0 msgr2=0x7f55d815a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:31.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.003+0000 7f55dda00640 1 --2- 172.21.15.94:0/3251340435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55d80b2bb0 0x7f55d815a650 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f55c400c450 tx=0x7f55c400c910 comp rx=0 tx=0).stop 2024-09-10T23:35:31.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.003+0000 7f55dda00640 1 -- 172.21.15.94:0/3251340435 shutdown_connections 2024-09-10T23:35:31.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.003+0000 7f55dda00640 1 --2- 172.21.15.94:0/3251340435 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f55b8072790 0x7f55b8074c50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.003+0000 7f55dda00640 1 --2- 172.21.15.94:0/3251340435 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55d815f5c0 0x7f55d81fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.003+0000 7f55dda00640 1 --2- 172.21.15.94:0/3251340435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55d80b2bb0 0x7f55d815a650 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.003+0000 7f55dda00640 1 --2- 172.21.15.94:0/3251340435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d80b0320 0x7f55d814bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.003+0000 7f55dda00640 1 -- 172.21.15.94:0/3251340435 >> 172.21.15.94:0/3251340435 conn(0x7f55d80d1c60 msgr2=0x7f55d80b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:31.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.003+0000 7f55dda00640 1 -- 172.21.15.94:0/3251340435 shutdown_connections 2024-09-10T23:35:31.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.003+0000 7f55dda00640 1 -- 172.21.15.94:0/3251340435 wait complete. 2024-09-10T23:35:31.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:35:31.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.167+0000 7f1c23600640 1 -- 172.21.15.94:0/3733634497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0ce1e0 msgr2=0x7f1c1c0cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:31.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.167+0000 7f1c23600640 1 --2- 172.21.15.94:0/3733634497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0ce1e0 0x7f1c1c0cec70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0b3070 tx=0x7f1c0801c800 comp rx=0 tx=0).stop 2024-09-10T23:35:31.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.167+0000 7f1c23600640 1 -- 172.21.15.94:0/3733634497 shutdown_connections 2024-09-10T23:35:31.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.167+0000 7f1c23600640 1 --2- 172.21.15.94:0/3733634497 >> v2:172.21.15.181:3300/0 conn(0x7f1c1c0b1100 0x7f1c1c0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.167+0000 7f1c23600640 1 --2- 172.21.15.94:0/3733634497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0ce1e0 0x7f1c1c0cec70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.167+0000 7f1c23600640 1 --2- 172.21.15.94:0/3733634497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c1c0ce890 0x7f1c1c0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.167+0000 7f1c23600640 1 -- 172.21.15.94:0/3733634497 >> 172.21.15.94:0/3733634497 conn(0x7f1c1c0d1c60 msgr2=0x7f1c1c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:31.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.167+0000 7f1c23600640 1 -- 172.21.15.94:0/3733634497 shutdown_connections 2024-09-10T23:35:31.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.167+0000 7f1c23600640 1 -- 172.21.15.94:0/3733634497 wait complete. 2024-09-10T23:35:31.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c23600640 1 Processor -- start 2024-09-10T23:35:31.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c23600640 1 -- start start 2024-09-10T23:35:31.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c23600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c1c0b1100 0x7f1c1c0b7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:31.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c23600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c1c0ce1e0 0x7f1c1c1f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:31.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c23600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0ce890 0x7f1c1c158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:31.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c23600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c0cfbc0 con 0x7f1c1c0ce890 2024-09-10T23:35:31.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c23600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c0cfa40 con 0x7f1c1c0ce1e0 2024-09-10T23:35:31.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c23600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1c1c0cfd40 con 0x7f1c1c0b1100 2024-09-10T23:35:31.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c21600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0ce890 0x7f1c1c158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:31.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c20c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c1c0b1100 0x7f1c1c0b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:31.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c21600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0ce890 0x7f1c1c158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60404/0 (socket says 172.21.15.94:60404) 2024-09-10T23:35:31.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c21600640 1 -- 172.21.15.94:0/2140852127 learned_addr learned my addr 172.21.15.94:0/2140852127 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:31.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c1be00640 1 --2- 172.21.15.94:0/2140852127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c1c0ce1e0 0x7f1c1c1f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:31.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c21600640 1 -- 172.21.15.94:0/2140852127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c1c0b1100 msgr2=0x7f1c1c0b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:31.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c21600640 1 --2- 172.21.15.94:0/2140852127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c1c0b1100 0x7f1c1c0b7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c21600640 1 -- 172.21.15.94:0/2140852127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c1c0ce1e0 msgr2=0x7f1c1c1f85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:31.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c21600640 1 --2- 172.21.15.94:0/2140852127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c1c0ce1e0 0x7f1c1c1f85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c21600640 1 -- 172.21.15.94:0/2140852127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c15ed80 con 0x7f1c1c0ce890 2024-09-10T23:35:31.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c21600640 1 --2- 172.21.15.94:0/2140852127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0ce890 0x7f1c1c158fa0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400e970 tx=0x7f1c1400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:31.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c18c00640 1 -- 172.21.15.94:0/2140852127 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1400ccc0 con 0x7f1c1c0ce890 2024-09-10T23:35:31.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c18c00640 1 -- 172.21.15.94:0/2140852127 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c1400ce60 con 0x7f1c1c0ce890 2024-09-10T23:35:31.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c18c00640 1 -- 172.21.15.94:0/2140852127 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14005280 con 0x7f1c1c0ce890 2024-09-10T23:35:31.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c23600640 1 -- 172.21.15.94:0/2140852127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c15f060 con 0x7f1c1c0ce890 2024-09-10T23:35:31.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c23600640 1 -- 172.21.15.94:0/2140852127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c15d690 con 0x7f1c1c0ce890 2024-09-10T23:35:31.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.171+0000 7f1c23600640 1 -- 172.21.15.94:0/2140852127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c1c0cec70 con 0x7f1c1c0ce890 2024-09-10T23:35:31.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.175+0000 7f1c18c00640 1 -- 172.21.15.94:0/2140852127 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f1c14005420 con 0x7f1c1c0ce890 2024-09-10T23:35:31.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.175+0000 7f1c18c00640 1 --2- 172.21.15.94:0/2140852127 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1bfc072790 0x7f1bfc074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:31.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.175+0000 7f1c18c00640 1 -- 172.21.15.94:0/2140852127 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f1c14093cc0 con 0x7f1c1c0ce890 2024-09-10T23:35:31.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.175+0000 7f1c20c00640 1 --2- 172.21.15.94:0/2140852127 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1bfc072790 0x7f1bfc074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:31.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.175+0000 7f1c20c00640 1 --2- 172.21.15.94:0/2140852127 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1bfc072790 0x7f1bfc074c50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c15d340 tx=0x7f1c0c009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:31.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.179+0000 7f1c18c00640 1 -- 172.21.15.94:0/2140852127 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1c1409a080 con 0x7f1c1c0ce890 2024-09-10T23:35:31.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.343+0000 7f1c23600640 1 -- 172.21.15.94:0/2140852127 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1c1c0c8af0 con 0x7f1c1c0ce890 2024-09-10T23:35:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.347+0000 7f1c18c00640 1 -- 172.21.15.94:0/2140852127 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f1c1405ecb0 con 0x7f1c1c0ce890 2024-09-10T23:35:31.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.351+0000 7f1c23600640 1 -- 172.21.15.94:0/2140852127 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1bfc072790 msgr2=0x7f1bfc074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:31.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.351+0000 7f1c23600640 1 --2- 172.21.15.94:0/2140852127 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1bfc072790 0x7f1bfc074c50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c15d340 tx=0x7f1c0c009500 comp rx=0 tx=0).stop 2024-09-10T23:35:31.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.351+0000 7f1c23600640 1 -- 172.21.15.94:0/2140852127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0ce890 msgr2=0x7f1c1c158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:31.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.351+0000 7f1c23600640 1 --2- 172.21.15.94:0/2140852127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0ce890 0x7f1c1c158fa0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400e970 tx=0x7f1c1400ee30 comp rx=0 tx=0).stop 2024-09-10T23:35:31.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.351+0000 7f1c23600640 1 -- 172.21.15.94:0/2140852127 shutdown_connections 2024-09-10T23:35:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.351+0000 7f1c23600640 1 --2- 172.21.15.94:0/2140852127 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1bfc072790 0x7f1bfc074c50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.351+0000 7f1c23600640 1 --2- 172.21.15.94:0/2140852127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c1c0ce1e0 0x7f1c1c1f85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.351+0000 7f1c23600640 1 --2- 172.21.15.94:0/2140852127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c1c0ce890 0x7f1c1c158fa0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.351+0000 7f1c23600640 1 --2- 172.21.15.94:0/2140852127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c1c0b1100 0x7f1c1c0b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.351+0000 7f1c23600640 1 -- 172.21.15.94:0/2140852127 >> 172.21.15.94:0/2140852127 conn(0x7f1c1c0d1c60 msgr2=0x7f1c1c071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.351+0000 7f1c23600640 1 -- 172.21.15.94:0/2140852127 shutdown_connections 2024-09-10T23:35:31.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.351+0000 7f1c23600640 1 -- 172.21.15.94:0/2140852127 wait complete. 2024-09-10T23:35:31.371 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:35:31.371 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:35:31.448 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:35:31.529 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:35:31.529 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:35:31.529 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:35:31.538 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:35:31.538 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:35:31.538 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:35:31.547 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:35:31.547 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:35:31.547 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:35:31.594 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:35:31.594 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:35:31.594 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:35:31.604 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:35:31.604 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T23:35:31.604 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:35:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea59600640 1 -- 172.21.15.94:0/3640631891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea540cfcb0 msgr2=0x7fea540d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea59600640 1 --2- 172.21.15.94:0/3640631891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea540cfcb0 0x7fea540d00d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009900 tx=0x7fea3c01c8b0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea59600640 1 -- 172.21.15.94:0/3640631891 shutdown_connections 2024-09-10T23:35:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea59600640 1 --2- 172.21.15.94:0/3640631891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea540cddb0 0x7fea540ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea59600640 1 --2- 172.21.15.94:0/3640631891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea540cfcb0 0x7fea540d00d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea59600640 1 -- 172.21.15.94:0/3640631891 >> 172.21.15.94:0/3640631891 conn(0x7fea5406cdf0 msgr2=0x7fea5406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea59600640 1 -- 172.21.15.94:0/3640631891 shutdown_connections 2024-09-10T23:35:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea59600640 1 -- 172.21.15.94:0/3640631891 wait complete. 2024-09-10T23:35:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea59600640 1 Processor -- start 2024-09-10T23:35:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea59600640 1 -- start start 2024-09-10T23:35:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea59600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea540cddb0 0x7fea541f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea59600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea540cfcb0 0x7fea5415c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea59600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea5415ba50 0x7fea5415d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea59600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea54168be0 con 0x7fea540cfcb0 2024-09-10T23:35:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea59600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fea54168a60 con 0x7fea5415ba50 2024-09-10T23:35:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea59600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fea54168d60 con 0x7fea540cddb0 2024-09-10T23:35:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea4a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea540cfcb0 0x7fea5415c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea4a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea540cfcb0 0x7fea5415c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60422/0 (socket says 172.21.15.94:60422) 2024-09-10T23:35:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea4a000640 1 -- 172.21.15.94:0/1414092776 learned_addr learned my addr 172.21.15.94:0/1414092776 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea52a00640 1 --2- 172.21.15.94:0/1414092776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea540cddb0 0x7fea541f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea53400640 1 --2- 172.21.15.94:0/1414092776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea5415ba50 0x7fea5415d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea4a000640 1 -- 172.21.15.94:0/1414092776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea540cddb0 msgr2=0x7fea541f7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.779+0000 7fea4a000640 1 --2- 172.21.15.94:0/1414092776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea540cddb0 0x7fea541f7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.783+0000 7fea4a000640 1 -- 172.21.15.94:0/1414092776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea5415ba50 msgr2=0x7fea5415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.783+0000 7fea4a000640 1 --2- 172.21.15.94:0/1414092776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea5415ba50 0x7fea5415d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.783+0000 7fea4a000640 1 -- 172.21.15.94:0/1414092776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5415d070 con 0x7fea540cfcb0 2024-09-10T23:35:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.783+0000 7fea4a000640 1 --2- 172.21.15.94:0/1414092776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea540cfcb0 0x7fea5415c140 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009850 tx=0x7fea3c004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:31.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.783+0000 7fea4b400640 1 -- 172.21.15.94:0/1414092776 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0b2040 con 0x7fea540cfcb0 2024-09-10T23:35:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.783+0000 7fea4b400640 1 -- 172.21.15.94:0/1414092776 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea3c005110 con 0x7fea540cfcb0 2024-09-10T23:35:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.783+0000 7fea59600640 1 -- 172.21.15.94:0/1414092776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5415c7d0 con 0x7fea540cfcb0 2024-09-10T23:35:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.783+0000 7fea4b400640 1 -- 172.21.15.94:0/1414092776 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c005540 con 0x7fea540cfcb0 2024-09-10T23:35:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.783+0000 7fea59600640 1 -- 172.21.15.94:0/1414092776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5415cd90 con 0x7fea540cfcb0 2024-09-10T23:35:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.783+0000 7fea59600640 1 -- 172.21.15.94:0/1414092776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea540ce220 con 0x7fea540cfcb0 2024-09-10T23:35:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.783+0000 7fea4b400640 1 -- 172.21.15.94:0/1414092776 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7fea3c0046d0 con 0x7fea540cfcb0 2024-09-10T23:35:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.787+0000 7fea4b400640 1 --2- 172.21.15.94:0/1414092776 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea30072790 0x7fea30074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.787+0000 7fea4b400640 1 -- 172.21.15.94:0/1414092776 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7fea3c12cdd0 con 0x7fea540cfcb0 2024-09-10T23:35:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.787+0000 7fea52a00640 1 --2- 172.21.15.94:0/1414092776 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea30072790 0x7fea30074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:31.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.787+0000 7fea52a00640 1 --2- 172.21.15.94:0/1414092776 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea30072790 0x7fea30074c50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fea54149df0 tx=0x7fea44009060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.791+0000 7fea4b400640 1 -- 172.21.15.94:0/1414092776 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fea3c0bd070 con 0x7fea540cfcb0 2024-09-10T23:35:31.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.991+0000 7fea59600640 1 -- 172.21.15.94:0/1414092776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea5415ab70 con 0x7fea540cfcb0 2024-09-10T23:35:31.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.991+0000 7fea4b400640 1 -- 172.21.15.94:0/1414092776 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3915 (secure 0 0 0) 0x7fea3c0164d0 con 0x7fea540cfcb0 2024-09-10T23:35:31.991 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:31.991 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":28,"btime":"2024-09-10T23:35:25:635652+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5343,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/919392312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":919392312},{"type":"v1","addr":"172.21.15.94:6834","nonce":919392312}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5346,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/964334868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":964334868},{"type":"v1","addr":"172.21.15.94:6835","nonce":964334868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":194813,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/4051613809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4051613809},{"type":"v1","addr":"172.21.15.181:6835","nonce":4051613809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":195053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3943842082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3943842082},{"type":"v1","addr":"172.21.15.181:6837","nonce":3943842082}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-09-10T23:35:31.991 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 28 2024-09-10T23:35:31.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.991+0000 7fea59600640 1 -- 172.21.15.94:0/1414092776 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea30072790 msgr2=0x7fea30074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:31.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.991+0000 7fea59600640 1 --2- 172.21.15.94:0/1414092776 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea30072790 0x7fea30074c50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fea54149df0 tx=0x7fea44009060 comp rx=0 tx=0).stop 2024-09-10T23:35:31.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.991+0000 7fea59600640 1 -- 172.21.15.94:0/1414092776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea540cfcb0 msgr2=0x7fea5415c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.991+0000 7fea59600640 1 --2- 172.21.15.94:0/1414092776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea540cfcb0 0x7fea5415c140 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009850 tx=0x7fea3c004220 comp rx=0 tx=0).stop 2024-09-10T23:35:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.991+0000 7fea59600640 1 -- 172.21.15.94:0/1414092776 shutdown_connections 2024-09-10T23:35:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.991+0000 7fea59600640 1 --2- 172.21.15.94:0/1414092776 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fea30072790 0x7fea30074c50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.991+0000 7fea59600640 1 --2- 172.21.15.94:0/1414092776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea5415ba50 0x7fea5415d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.991+0000 7fea59600640 1 --2- 172.21.15.94:0/1414092776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea540cfcb0 0x7fea5415c140 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.991+0000 7fea59600640 1 --2- 172.21.15.94:0/1414092776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea540cddb0 0x7fea541f7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.991+0000 7fea59600640 1 -- 172.21.15.94:0/1414092776 >> 172.21.15.94:0/1414092776 conn(0x7fea5406cdf0 msgr2=0x7fea5414a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.991+0000 7fea59600640 1 -- 172.21.15.94:0/1414092776 shutdown_connections 2024-09-10T23:35:31.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:31.991+0000 7fea59600640 1 -- 172.21.15.94:0/1414092776 wait complete. 2024-09-10T23:35:32.014 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T23:35:32.014 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T23:35:32.014 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T23:35:32.014 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:35:32.014 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T23:35:32.014 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T23:35:32.015 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T23:35:32.015 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:35:32.015 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T23:35:32.015 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T23:35:32.015 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T23:35:32.015 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:35:32.016 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T23:35:32.016 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T23:35:32.016 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T23:35:32.016 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:35:32.027 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T23:35:32.028 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T23:35:32.028 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T23:35:32.028 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T23:35:32.033 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T23:35:32.033 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T23:35:32.034 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T23:35:32.034 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T23:35:32.072 INFO:tasks.ceph.mds.a:Started 2024-09-10T23:35:32.073 DEBUG:teuthology.parallel:result is None 2024-09-10T23:35:32.074 INFO:tasks.ceph.mds.c:Started 2024-09-10T23:35:32.074 DEBUG:teuthology.parallel:result is None 2024-09-10T23:35:32.080 INFO:tasks.ceph.mds.d:Started 2024-09-10T23:35:32.081 DEBUG:teuthology.parallel:result is None 2024-09-10T23:35:32.082 INFO:tasks.ceph.mds.b:Started 2024-09-10T23:35:32.082 DEBUG:teuthology.parallel:result is None 2024-09-10T23:35:32.083 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T23:35:32.169 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2024-09-10T23:35:32.170 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:35:32.171 INFO:tasks.ceph.mds.d.smithi181.stderr:2024-09-10T23:35:32.167+0000 7f8ea7200640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-10T23:35:32.208 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-09-10T23:35:32.208 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-09-10T23:35:32.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.231+0000 7f2b91000640 1 -- 172.21.15.94:0/1095110155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b8c0ce1e0 msgr2=0x7f2b8c0cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:32.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.231+0000 7f2b91000640 1 --2- 172.21.15.94:0/1095110155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b8c0ce1e0 0x7f2b8c0cec70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c073990 tx=0x7f2b7801c800 comp rx=0 tx=0).stop 2024-09-10T23:35:32.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.231+0000 7f2b91000640 1 -- 172.21.15.94:0/1095110155 shutdown_connections 2024-09-10T23:35:32.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.231+0000 7f2b91000640 1 --2- 172.21.15.94:0/1095110155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b8c0b1100 0x7f2b8c0b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.231+0000 7f2b91000640 1 --2- 172.21.15.94:0/1095110155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b8c0ce1e0 0x7f2b8c0cec70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.231+0000 7f2b91000640 1 --2- 172.21.15.94:0/1095110155 >> v2:172.21.15.181:3300/0 conn(0x7f2b8c0ce890 0x7f2b8c0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.231+0000 7f2b91000640 1 -- 172.21.15.94:0/1095110155 >> 172.21.15.94:0/1095110155 conn(0x7f2b8c0d1c60 msgr2=0x7f2b8c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.231+0000 7f2b91000640 1 -- 172.21.15.94:0/1095110155 shutdown_connections 2024-09-10T23:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.231+0000 7f2b91000640 1 -- 172.21.15.94:0/1095110155 wait complete. 2024-09-10T23:35:32.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b91000640 1 Processor -- start 2024-09-10T23:35:32.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b91000640 1 -- start start 2024-09-10T23:35:32.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b91000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b8c0b1100 0x7f2b8c1f60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:32.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b91000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b8c0ce1e0 0x7f2b8c155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:32.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b91000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b8c0ce890 0x7f2b8c0afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:32.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b91000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c0cfbc0 con 0x7f2b8c0b1100 2024-09-10T23:35:32.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b91000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c0cfa40 con 0x7f2b8c0ce890 2024-09-10T23:35:32.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b91000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2b8c0cfd40 con 0x7f2b8c0ce1e0 2024-09-10T23:35:32.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b89e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b8c0b1100 0x7f2b8c1f60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:32.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b89e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b8c0b1100 0x7f2b8c1f60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60472/0 (socket says 172.21.15.94:60472) 2024-09-10T23:35:32.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b89e00640 1 -- 172.21.15.94:0/2616903233 learned_addr learned my addr 172.21.15.94:0/2616903233 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:32.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b89400640 1 --2- 172.21.15.94:0/2616903233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b8c0ce1e0 0x7f2b8c155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:32.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b8a800640 1 --2- 172.21.15.94:0/2616903233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b8c0ce890 0x7f2b8c0afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b89e00640 1 -- 172.21.15.94:0/2616903233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b8c0ce1e0 msgr2=0x7f2b8c155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b89e00640 1 --2- 172.21.15.94:0/2616903233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b8c0ce1e0 0x7f2b8c155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b89e00640 1 -- 172.21.15.94:0/2616903233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b8c0ce890 msgr2=0x7f2b8c0afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b89e00640 1 --2- 172.21.15.94:0/2616903233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b8c0ce890 0x7f2b8c0afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b89e00640 1 -- 172.21.15.94:0/2616903233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c1f6630 con 0x7f2b8c0b1100 2024-09-10T23:35:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b8a800640 1 --2- 172.21.15.94:0/2616903233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b8c0ce890 0x7f2b8c0afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:35:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b89e00640 1 --2- 172.21.15.94:0/2616903233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b8c0b1100 0x7f2b8c1f60f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00b440 tx=0x7f2b7c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b82000640 1 -- 172.21.15.94:0/2616903233 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c015020 con 0x7f2b8c0b1100 2024-09-10T23:35:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b82000640 1 -- 172.21.15.94:0/2616903233 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b7c0119a0 con 0x7f2b8c0b1100 2024-09-10T23:35:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b91000640 1 -- 172.21.15.94:0/2616903233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8c0b60c0 con 0x7f2b8c0b1100 2024-09-10T23:35:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b82000640 1 -- 172.21.15.94:0/2616903233 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c011df0 con 0x7f2b8c0b1100 2024-09-10T23:35:32.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.235+0000 7f2b91000640 1 -- 172.21.15.94:0/2616903233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8c0b6680 con 0x7f2b8c0b1100 2024-09-10T23:35:32.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.239+0000 7f2b91000640 1 -- 172.21.15.94:0/2616903233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b8c0cec70 con 0x7f2b8c0b1100 2024-09-10T23:35:32.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.239+0000 7f2b82000640 1 -- 172.21.15.94:0/2616903233 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f2b7c01a400 con 0x7f2b8c0b1100 2024-09-10T23:35:32.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.239+0000 7f2b82000640 1 --2- 172.21.15.94:0/2616903233 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2b6c072790 0x7f2b6c074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:32.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.239+0000 7f2b82000640 1 -- 172.21.15.94:0/2616903233 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f2b7c0936f0 con 0x7f2b8c0b1100 2024-09-10T23:35:32.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.239+0000 7f2b89400640 1 --2- 172.21.15.94:0/2616903233 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2b6c072790 0x7f2b6c074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:32.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.239+0000 7f2b89400640 1 --2- 172.21.15.94:0/2616903233 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2b6c072790 0x7f2b6c074c50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2b780af000 tx=0x7f2b7801b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:32.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.243+0000 7f2b82000640 1 -- 172.21.15.94:0/2616903233 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2b7c01f020 con 0x7f2b8c0b1100 2024-09-10T23:35:32.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:32.427+0000 7f2b91000640 1 -- 172.21.15.94:0/2616903233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2b8c0cd720 con 0x7f2b8c0b1100 2024-09-10T23:35:33.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.366+0000 7f2b82000640 1 -- 172.21.15.94:0/2616903233 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) ==== 97+0+0 (secure 0 0 0) 0x7f2b8c1f6630 con 0x7f2b8c0b1100 2024-09-10T23:35:33.370 INFO:teuthology.orchestra.run.smithi094.stderr: removed all blocklist entries 2024-09-10T23:35:33.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.370+0000 7f2b91000640 1 -- 172.21.15.94:0/2616903233 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2b6c072790 msgr2=0x7f2b6c074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:33.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.370+0000 7f2b91000640 1 --2- 172.21.15.94:0/2616903233 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2b6c072790 0x7f2b6c074c50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2b780af000 tx=0x7f2b7801b040 comp rx=0 tx=0).stop 2024-09-10T23:35:33.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.370+0000 7f2b91000640 1 -- 172.21.15.94:0/2616903233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b8c0b1100 msgr2=0x7f2b8c1f60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:33.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.370+0000 7f2b91000640 1 --2- 172.21.15.94:0/2616903233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b8c0b1100 0x7f2b8c1f60f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00b440 tx=0x7f2b7c00b900 comp rx=0 tx=0).stop 2024-09-10T23:35:33.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.370+0000 7f2b91000640 1 -- 172.21.15.94:0/2616903233 shutdown_connections 2024-09-10T23:35:33.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.370+0000 7f2b91000640 1 --2- 172.21.15.94:0/2616903233 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2b6c072790 0x7f2b6c074c50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:33.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.370+0000 7f2b91000640 1 --2- 172.21.15.94:0/2616903233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b8c0ce890 0x7f2b8c0afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:33.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.370+0000 7f2b91000640 1 --2- 172.21.15.94:0/2616903233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b8c0ce1e0 0x7f2b8c155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:33.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.370+0000 7f2b91000640 1 --2- 172.21.15.94:0/2616903233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b8c0b1100 0x7f2b8c1f60f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:33.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.370+0000 7f2b91000640 1 -- 172.21.15.94:0/2616903233 >> 172.21.15.94:0/2616903233 conn(0x7f2b8c0d1c60 msgr2=0x7f2b8c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:33.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.370+0000 7f2b91000640 1 -- 172.21.15.94:0/2616903233 shutdown_connections 2024-09-10T23:35:33.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.370+0000 7f2b91000640 1 -- 172.21.15.94:0/2616903233 wait complete. 2024-09-10T23:35:33.395 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T23:35:33.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.538+0000 7f36ad400640 1 -- 172.21.15.94:0/1568440919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36a815ba30 msgr2=0x7f36a80b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:33.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.538+0000 7f36ad400640 1 --2- 172.21.15.94:0/1568440919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36a815ba30 0x7f36a80b1bd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3698009840 tx=0x7f369801c7f0 comp rx=0 tx=0).stop 2024-09-10T23:35:33.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.538+0000 7f36ad400640 1 -- 172.21.15.94:0/1568440919 shutdown_connections 2024-09-10T23:35:33.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.538+0000 7f36ad400640 1 --2- 172.21.15.94:0/1568440919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36a815ba30 0x7f36a80b1bd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:33.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.538+0000 7f36ad400640 1 -- 172.21.15.94:0/1568440919 >> 172.21.15.94:0/1568440919 conn(0x7f36a809aa70 msgr2=0x7f36a809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:33.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.538+0000 7f36ad400640 1 -- 172.21.15.94:0/1568440919 shutdown_connections 2024-09-10T23:35:33.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.538+0000 7f36ad400640 1 -- 172.21.15.94:0/1568440919 wait complete. 2024-09-10T23:35:33.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.538+0000 7f36ad400640 1 Processor -- start 2024-09-10T23:35:33.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.538+0000 7f36ad400640 1 -- start start 2024-09-10T23:35:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36ad400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36a80b2110 0x7f36a814afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36ad400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a80d04b0 0x7f36a8157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36ad400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36a815ba30 0x7f36a81396d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:33.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36ad400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f36a8169a50 con 0x7f36a80d04b0 2024-09-10T23:35:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36ad400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81698d0 con 0x7f36a80b2110 2024-09-10T23:35:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36ad400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f36a8169bd0 con 0x7f36a815ba30 2024-09-10T23:35:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36a7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a80d04b0 0x7f36a8157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36a7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a80d04b0 0x7f36a8157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60484/0 (socket says 172.21.15.94:60484) 2024-09-10T23:35:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36a7400640 1 -- 172.21.15.94:0/2505774989 learned_addr learned my addr 172.21.15.94:0/2505774989 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36a7e00640 1 --2- 172.21.15.94:0/2505774989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36a80b2110 0x7f36a814afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36aca00640 1 --2- 172.21.15.94:0/2505774989 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36a815ba30 0x7f36a81396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36a7400640 1 -- 172.21.15.94:0/2505774989 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36a815ba30 msgr2=0x7f36a81396d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36a7400640 1 --2- 172.21.15.94:0/2505774989 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36a815ba30 0x7f36a81396d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36a7400640 1 -- 172.21.15.94:0/2505774989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36a80b2110 msgr2=0x7f36a814afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36a7400640 1 --2- 172.21.15.94:0/2505774989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36a80b2110 0x7f36a814afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36a7400640 1 -- 172.21.15.94:0/2505774989 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a815a310 con 0x7f36a80d04b0 2024-09-10T23:35:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36a7e00640 1 --2- 172.21.15.94:0/2505774989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36a80b2110 0x7f36a814afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:35:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36a7400640 1 --2- 172.21.15.94:0/2505774989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a80d04b0 0x7f36a8157f00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f369400c4b0 tx=0x7f369400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f369fe00640 1 -- 172.21.15.94:0/2505774989 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36940052d0 con 0x7f36a80d04b0 2024-09-10T23:35:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f369fe00640 1 -- 172.21.15.94:0/2505774989 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3694005470 con 0x7f36a80d04b0 2024-09-10T23:35:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f369fe00640 1 -- 172.21.15.94:0/2505774989 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694012870 con 0x7f36a80d04b0 2024-09-10T23:35:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36ad400640 1 -- 172.21.15.94:0/2505774989 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a8157760 con 0x7f36a80d04b0 2024-09-10T23:35:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36ad400640 1 -- 172.21.15.94:0/2505774989 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a815a860 con 0x7f36a80d04b0 2024-09-10T23:35:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.542+0000 7f36ad400640 1 -- 172.21.15.94:0/2505774989 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a8072ef0 con 0x7f36a80d04b0 2024-09-10T23:35:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.546+0000 7f369fe00640 1 -- 172.21.15.94:0/2505774989 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f3694003600 con 0x7f36a80d04b0 2024-09-10T23:35:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.546+0000 7f369fe00640 1 --2- 172.21.15.94:0/2505774989 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3688072790 0x7f3688074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.546+0000 7f369fe00640 1 -- 172.21.15.94:0/2505774989 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f3694092e70 con 0x7f36a80d04b0 2024-09-10T23:35:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.546+0000 7f36a7e00640 1 --2- 172.21.15.94:0/2505774989 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3688072790 0x7f3688074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:33.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.546+0000 7f36a7e00640 1 --2- 172.21.15.94:0/2505774989 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3688072790 0x7f3688074c50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f36980af000 tx=0x7f369801b030 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:33.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.550+0000 7f369fe00640 1 -- 172.21.15.94:0/2505774989 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f369405de20 con 0x7f36a80d04b0 2024-09-10T23:35:33.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.786+0000 7f36ad400640 1 -- 172.21.15.94:0/2505774989 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f36a80d1de0 con 0x7f36a80d04b0 2024-09-10T23:35:33.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.786+0000 7f369fe00640 1 -- 172.21.15.94:0/2505774989 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v29) ==== 80+0+5304 (secure 0 0 0) 0x7f3694062850 con 0x7f36a80d04b0 2024-09-10T23:35:33.789 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:33.789 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:35:33.789 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_dump": [ 2024-09-10T23:35:33.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.789 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.a", 2024-09-10T23:35:33.789 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/2m3LxAAr0ErpLJg+0+I2r74zQya4w==", 2024-09-10T23:35:33.790 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.790 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:35:33.790 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:35:33.790 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:35:33.790 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:35:33.791 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.791 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.b", 2024-09-10T23:35:33.791 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmc8pYABAAw90zu8D+9IwK15h4HIlGBA==", 2024-09-10T23:35:33.791 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.791 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:35:33.791 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:35:33.791 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:35:33.792 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:35:33.792 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.792 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.c", 2024-09-10T23:35:33.792 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmk98KNxAA44e9N4VHbw/94Lh1V9zugQ==", 2024-09-10T23:35:33.792 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.792 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:35:33.792 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:35:33.792 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:35:33.792 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:35:33.793 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.793 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.d", 2024-09-10T23:35:33.793 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmVAKzBxAAJehOw6aphTF/g3ucoQfw9A==", 2024-09-10T23:35:33.793 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.793 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:35:33.793 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:35:33.793 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:35:33.793 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:35:33.793 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.794 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.0", 2024-09-10T23:35:33.795 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDa1uBmSAD3KBAAJTF1OEC44Ml599aL7iEuDw==", 2024-09-10T23:35:33.795 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.795 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:35:33.795 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:35:33.795 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:35:33.795 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.795 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.1", 2024-09-10T23:35:33.796 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDe1uBma1L9ABAAfEjW1/Hb4fh4Kwv/ZUgFAQ==", 2024-09-10T23:35:33.797 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.798 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:35:33.798 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:35:33.798 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:35:33.798 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.798 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.2", 2024-09-10T23:35:33.798 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDh1uBmtLUQFhAAweeFHj6BRQVC9BX+VLiWEQ==", 2024-09-10T23:35:33.798 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.798 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:35:33.798 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:35:33.798 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:35:33.798 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.799 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.3", 2024-09-10T23:35:33.799 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDk1uBm1KFCKRAAev28iYcrEYOViX95uo6Kbw==", 2024-09-10T23:35:33.799 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.799 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:35:33.799 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:35:33.799 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:35:33.799 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.799 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.4", 2024-09-10T23:35:33.799 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDq1uBm/Hg9LhAAvB2D0rj4DeRPxTrGmXYIsA==", 2024-09-10T23:35:33.799 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.800 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:35:33.800 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:35:33.800 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:35:33.800 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.800 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.5", 2024-09-10T23:35:33.800 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDu1uBm1fAKCBAAFHMjnGFAItLdaACF/UjE/w==", 2024-09-10T23:35:33.800 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.800 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:35:33.800 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:35:33.800 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:35:33.800 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.801 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.6", 2024-09-10T23:35:33.801 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDx1uBmXgYfHhAA/6wwtM4mDRXsIpoTIHKJ0w==", 2024-09-10T23:35:33.801 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.801 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:35:33.801 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:35:33.801 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:35:33.801 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.801 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.7", 2024-09-10T23:35:33.801 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQD01uBmw6hQMxAAk7oGAg+iIplNK5KrokGwNA==", 2024-09-10T23:35:33.802 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.802 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:35:33.802 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:35:33.802 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:35:33.802 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.802 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.0", 2024-09-10T23:35:33.802 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g==", 2024-09-10T23:35:33.802 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.802 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:35:33.802 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow r", 2024-09-10T23:35:33.802 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T23:35:33.802 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.803 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.1", 2024-09-10T23:35:33.803 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmuSkhFRAA1R6WNXLImkGfhW9/XJzEAQ==", 2024-09-10T23:35:33.803 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.803 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:35:33.803 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:35:33.803 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:35:33.803 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:35:33.803 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.803 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.2", 2024-09-10T23:35:33.804 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmfvaqERAASpL0KtglFcAuRf4a/0z5JQ==", 2024-09-10T23:35:33.804 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.804 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:35:33.804 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:35:33.804 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:35:33.804 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:35:33.804 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.804 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.3", 2024-09-10T23:35:33.804 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBm8xUtGxAARc0ywfAhMxBNd57TzBoV3g==", 2024-09-10T23:35:33.804 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.804 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:35:33.805 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:35:33.805 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:35:33.805 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:35:33.805 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.805 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.admin", 2024-09-10T23:35:33.805 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmOpg1FBAA9NV6Jqt6doWMuAfcxztlOg==", 2024-09-10T23:35:33.805 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.805 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:35:33.805 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:35:33.805 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:35:33.805 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:35:33.806 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.806 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.x", 2024-09-10T23:35:33.806 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/EPVKxAA/Myv8Zjf7BejbzlkxyCC1w==", 2024-09-10T23:35:33.806 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.806 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:35:33.806 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:35:33.806 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:35:33.806 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:33.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:33.806 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.y", 2024-09-10T23:35:33.806 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmhHvtJxAACe+79nYhv2ODwtmNhA45FA==", 2024-09-10T23:35:33.807 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:35:33.807 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:35:33.807 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:35:33.807 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:35:33.807 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.807 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:33.807 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:35:33.807 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:35:33.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.790+0000 7f36ad400640 1 -- 172.21.15.94:0/2505774989 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3688072790 msgr2=0x7f3688074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:33.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.790+0000 7f36ad400640 1 --2- 172.21.15.94:0/2505774989 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3688072790 0x7f3688074c50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f36980af000 tx=0x7f369801b030 comp rx=0 tx=0).stop 2024-09-10T23:35:33.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.790+0000 7f36ad400640 1 -- 172.21.15.94:0/2505774989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a80d04b0 msgr2=0x7f36a8157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:33.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.790+0000 7f36ad400640 1 --2- 172.21.15.94:0/2505774989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a80d04b0 0x7f36a8157f00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f369400c4b0 tx=0x7f369400c970 comp rx=0 tx=0).stop 2024-09-10T23:35:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.790+0000 7f36ad400640 1 -- 172.21.15.94:0/2505774989 shutdown_connections 2024-09-10T23:35:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.790+0000 7f36ad400640 1 --2- 172.21.15.94:0/2505774989 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3688072790 0x7f3688074c50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.790+0000 7f36ad400640 1 --2- 172.21.15.94:0/2505774989 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36a815ba30 0x7f36a81396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.790+0000 7f36ad400640 1 --2- 172.21.15.94:0/2505774989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36a80d04b0 0x7f36a8157f00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.790+0000 7f36ad400640 1 --2- 172.21.15.94:0/2505774989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36a80b2110 0x7f36a814afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.790+0000 7f36ad400640 1 -- 172.21.15.94:0/2505774989 >> 172.21.15.94:0/2505774989 conn(0x7f36a809aa70 msgr2=0x7f36a80b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.790+0000 7f36ad400640 1 -- 172.21.15.94:0/2505774989 shutdown_connections 2024-09-10T23:35:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.790+0000 7f36ad400640 1 -- 172.21.15.94:0/2505774989 wait complete. 2024-09-10T23:35:33.810 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T23:35:33.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.950+0000 7fbdf9c00640 1 -- 172.21.15.94:0/791417657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdf415bd30 msgr2=0x7fbdf415c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:33.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.950+0000 7fbdf9c00640 1 --2- 172.21.15.94:0/791417657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdf415bd30 0x7fbdf415c110 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40b7d20 tx=0x7fbde401cb80 comp rx=0 tx=0).stop 2024-09-10T23:35:33.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.950+0000 7fbdf9c00640 1 -- 172.21.15.94:0/791417657 shutdown_connections 2024-09-10T23:35:33.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.950+0000 7fbdf9c00640 1 --2- 172.21.15.94:0/791417657 >> v2:172.21.15.181:3300/0 conn(0x7fbdf40d2290 0x7fbdf40b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:33.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.950+0000 7fbdf9c00640 1 --2- 172.21.15.94:0/791417657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdf415bd30 0x7fbdf415c110 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:33.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.950+0000 7fbdf9c00640 1 -- 172.21.15.94:0/791417657 >> 172.21.15.94:0/791417657 conn(0x7fbdf409aa70 msgr2=0x7fbdf409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:33.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.950+0000 7fbdf9c00640 1 -- 172.21.15.94:0/791417657 shutdown_connections 2024-09-10T23:35:33.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.950+0000 7fbdf9c00640 1 -- 172.21.15.94:0/791417657 wait complete. 2024-09-10T23:35:33.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.950+0000 7fbdf9c00640 1 Processor -- start 2024-09-10T23:35:33.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.950+0000 7fbdf9c00640 1 -- start start 2024-09-10T23:35:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf9c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf40ae980 0x7fbdf4155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf9c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdf40d2290 0x7fbdf40c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf9c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdf415bd30 0x7fbdf40b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf9c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf4169550 con 0x7fbdf415bd30 2024-09-10T23:35:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf9c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41693d0 con 0x7fbdf40ae980 2024-09-10T23:35:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf9c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf41696d0 con 0x7fbdf40d2290 2024-09-10T23:35:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdf415bd30 0x7fbdf40b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf40ae980 0x7fbdf4155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdf415bd30 0x7fbdf40b60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42352/0 (socket says 172.21.15.94:42352) 2024-09-10T23:35:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdea000640 1 --2- 172.21.15.94:0/423296202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdf40d2290 0x7fbdf40c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf3400640 1 -- 172.21.15.94:0/423296202 learned_addr learned my addr 172.21.15.94:0/423296202 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf3400640 1 -- 172.21.15.94:0/423296202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdf40d2290 msgr2=0x7fbdf40c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf3400640 1 --2- 172.21.15.94:0/423296202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdf40d2290 0x7fbdf40c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf3400640 1 -- 172.21.15.94:0/423296202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf40ae980 msgr2=0x7fbdf4155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf3400640 1 --2- 172.21.15.94:0/423296202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf40ae980 0x7fbdf4155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf3400640 1 -- 172.21.15.94:0/423296202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf4156480 con 0x7fbdf415bd30 2024-09-10T23:35:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf2a00640 1 --2- 172.21.15.94:0/423296202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf40ae980 0x7fbdf4155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:35:33.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf3400640 1 --2- 172.21.15.94:0/423296202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdf415bd30 0x7fbdf40b60d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00b410 tx=0x7fbdec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:33.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdeb400640 1 -- 172.21.15.94:0/423296202 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec015020 con 0x7fbdf415bd30 2024-09-10T23:35:33.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdeb400640 1 -- 172.21.15.94:0/423296202 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdec011940 con 0x7fbdf415bd30 2024-09-10T23:35:33.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdeb400640 1 -- 172.21.15.94:0/423296202 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec011d90 con 0x7fbdf415bd30 2024-09-10T23:35:33.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf9c00640 1 -- 172.21.15.94:0/423296202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf4156760 con 0x7fbdf415bd30 2024-09-10T23:35:33.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf9c00640 1 -- 172.21.15.94:0/423296202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf4156d20 con 0x7fbdf415bd30 2024-09-10T23:35:33.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdf9c00640 1 -- 172.21.15.94:0/423296202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf40b1e70 con 0x7fbdf415bd30 2024-09-10T23:35:33.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.954+0000 7fbdeb400640 1 -- 172.21.15.94:0/423296202 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7fbdec01a400 con 0x7fbdf415bd30 2024-09-10T23:35:33.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.958+0000 7fbdeb400640 1 --2- 172.21.15.94:0/423296202 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbdd0072790 0x7fbdd0074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:33.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.958+0000 7fbdeb400640 1 -- 172.21.15.94:0/423296202 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7fbdec093570 con 0x7fbdf415bd30 2024-09-10T23:35:33.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.958+0000 7fbdf2a00640 1 --2- 172.21.15.94:0/423296202 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbdd0072790 0x7fbdd0074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:33.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.958+0000 7fbdf2a00640 1 --2- 172.21.15.94:0/423296202 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbdd0072790 0x7fbdd0074c50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbde40af000 tx=0x7fbde401b3c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:33.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:33.962+0000 7fbdeb400640 1 -- 172.21.15.94:0/423296202 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbdec01f020 con 0x7fbdf415bd30 2024-09-10T23:35:34.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.126+0000 7fbdf9c00640 1 -- 172.21.15.94:0/423296202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbdf40d28a0 con 0x7fbdf415bd30 2024-09-10T23:35:34.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.126+0000 7fbdeb400640 1 -- 172.21.15.94:0/423296202 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) ==== 84+0+65 (secure 0 0 0) 0x7fbdec05e520 con 0x7fbdf415bd30 2024-09-10T23:35:34.130 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:34.130 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:35:34.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:34.130 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-09-10T23:35:34.130 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-09-10T23:35:34.130 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:34.130 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:35:34.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.130+0000 7fbdf9c00640 1 -- 172.21.15.94:0/423296202 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbdd0072790 msgr2=0x7fbdd0074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:34.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.130+0000 7fbdf9c00640 1 --2- 172.21.15.94:0/423296202 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbdd0072790 0x7fbdd0074c50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbde40af000 tx=0x7fbde401b3c0 comp rx=0 tx=0).stop 2024-09-10T23:35:34.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.130+0000 7fbdf9c00640 1 -- 172.21.15.94:0/423296202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdf415bd30 msgr2=0x7fbdf40b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:34.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.130+0000 7fbdf9c00640 1 --2- 172.21.15.94:0/423296202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdf415bd30 0x7fbdf40b60d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00b410 tx=0x7fbdec00b8d0 comp rx=0 tx=0).stop 2024-09-10T23:35:34.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.130+0000 7fbdf9c00640 1 -- 172.21.15.94:0/423296202 shutdown_connections 2024-09-10T23:35:34.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.130+0000 7fbdf9c00640 1 --2- 172.21.15.94:0/423296202 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fbdd0072790 0x7fbdd0074c50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:34.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.130+0000 7fbdf9c00640 1 --2- 172.21.15.94:0/423296202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdf415bd30 0x7fbdf40b60d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:34.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.130+0000 7fbdf9c00640 1 --2- 172.21.15.94:0/423296202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdf40d2290 0x7fbdf40c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:34.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.130+0000 7fbdf9c00640 1 --2- 172.21.15.94:0/423296202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf40ae980 0x7fbdf4155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:34.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.130+0000 7fbdf9c00640 1 -- 172.21.15.94:0/423296202 >> 172.21.15.94:0/423296202 conn(0x7fbdf409aa70 msgr2=0x7fbdf4070960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:34.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.130+0000 7fbdf9c00640 1 -- 172.21.15.94:0/423296202 shutdown_connections 2024-09-10T23:35:34.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.130+0000 7fbdf9c00640 1 -- 172.21.15.94:0/423296202 wait complete. 2024-09-10T23:35:34.150 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T23:35:34.150 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T23:35:34.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.294+0000 7fab5ac00640 1 -- 172.21.15.94:0/2489129051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab540cddb0 msgr2=0x7fab540ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:34.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.294+0000 7fab5ac00640 1 --2- 172.21.15.94:0/2489129051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab540cddb0 0x7fab540ce200 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fab540b3f30 tx=0x7fab4c01cd50 comp rx=0 tx=0).stop 2024-09-10T23:35:34.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.294+0000 7fab5ac00640 1 -- 172.21.15.94:0/2489129051 shutdown_connections 2024-09-10T23:35:34.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.294+0000 7fab5ac00640 1 --2- 172.21.15.94:0/2489129051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab540cddb0 0x7fab540ce200 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:34.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.294+0000 7fab5ac00640 1 --2- 172.21.15.94:0/2489129051 >> v2:172.21.15.181:3300/0 conn(0x7fab540cfcb0 0x7fab540d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:34.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.294+0000 7fab5ac00640 1 -- 172.21.15.94:0/2489129051 >> 172.21.15.94:0/2489129051 conn(0x7fab5406cdf0 msgr2=0x7fab5406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:34.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.294+0000 7fab5ac00640 1 -- 172.21.15.94:0/2489129051 shutdown_connections 2024-09-10T23:35:34.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.294+0000 7fab5ac00640 1 -- 172.21.15.94:0/2489129051 wait complete. 2024-09-10T23:35:34.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.294+0000 7fab5ac00640 1 Processor -- start 2024-09-10T23:35:34.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.294+0000 7fab5ac00640 1 -- start start 2024-09-10T23:35:34.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab5ac00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab540cddb0 0x7fab541fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:34.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab5ac00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab540cfcb0 0x7fab5415e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:34.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab5ac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab5415ba50 0x7fab541f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:34.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab5ac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab54168be0 con 0x7fab5415ba50 2024-09-10T23:35:34.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab5ac00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fab54168a60 con 0x7fab540cddb0 2024-09-10T23:35:34.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab5ac00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fab54168d60 con 0x7fab540cfcb0 2024-09-10T23:35:34.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab58c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab5415ba50 0x7fab541f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:34.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab58c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab5415ba50 0x7fab541f6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42366/0 (socket says 172.21.15.94:42366) 2024-09-10T23:35:34.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab58c00640 1 -- 172.21.15.94:0/2120420606 learned_addr learned my addr 172.21.15.94:0/2120420606 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:34.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab4be00640 1 --2- 172.21.15.94:0/2120420606 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab540cfcb0 0x7fab5415e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:34.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab53e00640 1 --2- 172.21.15.94:0/2120420606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab540cddb0 0x7fab541fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:34.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab58c00640 1 -- 172.21.15.94:0/2120420606 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab540cfcb0 msgr2=0x7fab5415e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:34.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab58c00640 1 --2- 172.21.15.94:0/2120420606 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab540cfcb0 0x7fab5415e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:34.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab58c00640 1 -- 172.21.15.94:0/2120420606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab540cddb0 msgr2=0x7fab541fbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:34.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab58c00640 1 --2- 172.21.15.94:0/2120420606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab540cddb0 0x7fab541fbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:34.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab58c00640 1 -- 172.21.15.94:0/2120420606 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab541fc1d0 con 0x7fab5415ba50 2024-09-10T23:35:34.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab58c00640 1 --2- 172.21.15.94:0/2120420606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab5415ba50 0x7fab541f6fd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fab4c019960 tx=0x7fab4c009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:34.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab50c00640 1 -- 172.21.15.94:0/2120420606 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c0bb070 con 0x7fab5415ba50 2024-09-10T23:35:34.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab50c00640 1 -- 172.21.15.94:0/2120420606 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab4c004c50 con 0x7fab5415ba50 2024-09-10T23:35:34.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab50c00640 1 -- 172.21.15.94:0/2120420606 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c0b2040 con 0x7fab5415ba50 2024-09-10T23:35:34.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab5ac00640 1 -- 172.21.15.94:0/2120420606 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5406f360 con 0x7fab5415ba50 2024-09-10T23:35:34.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab5ac00640 1 -- 172.21.15.94:0/2120420606 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab5406f920 con 0x7fab5415ba50 2024-09-10T23:35:34.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab5ac00640 1 -- 172.21.15.94:0/2120420606 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab540ce200 con 0x7fab5415ba50 2024-09-10T23:35:34.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.298+0000 7fab50c00640 1 -- 172.21.15.94:0/2120420606 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7fab4c0b5030 con 0x7fab5415ba50 2024-09-10T23:35:34.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.302+0000 7fab50c00640 1 --2- 172.21.15.94:0/2120420606 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fab300727c0 0x7fab30074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:34.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.302+0000 7fab50c00640 1 -- 172.21.15.94:0/2120420606 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7fab4c12cdc0 con 0x7fab5415ba50 2024-09-10T23:35:34.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.302+0000 7fab53e00640 1 --2- 172.21.15.94:0/2120420606 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fab300727c0 0x7fab30074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:34.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.302+0000 7fab53e00640 1 --2- 172.21.15.94:0/2120420606 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fab300727c0 0x7fab30074c80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fab44001650 tx=0x7fab44009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:34.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.306+0000 7fab50c00640 1 -- 172.21.15.94:0/2120420606 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fab4c016970 con 0x7fab5415ba50 2024-09-10T23:35:34.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:34.474+0000 7fab5ac00640 1 -- 172.21.15.94:0/2120420606 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fab5414e910 con 0x7fab5415ba50 2024-09-10T23:35:34.763 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:35:34.767 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:35:35.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.370+0000 7fab50c00640 1 -- 172.21.15.94:0/2120420606 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) ==== 138+0+0 (secure 0 0 0) 0x7fab4c0f7d70 con 0x7fab5415ba50 2024-09-10T23:35:35.374 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-09-10T23:35:35.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.374+0000 7fab5ac00640 1 -- 172.21.15.94:0/2120420606 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fab300727c0 msgr2=0x7fab30074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:35.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.374+0000 7fab5ac00640 1 --2- 172.21.15.94:0/2120420606 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fab300727c0 0x7fab30074c80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fab44001650 tx=0x7fab44009040 comp rx=0 tx=0).stop 2024-09-10T23:35:35.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.374+0000 7fab5ac00640 1 -- 172.21.15.94:0/2120420606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab5415ba50 msgr2=0x7fab541f6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:35.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.374+0000 7fab5ac00640 1 --2- 172.21.15.94:0/2120420606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab5415ba50 0x7fab541f6fd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fab4c019960 tx=0x7fab4c009300 comp rx=0 tx=0).stop 2024-09-10T23:35:35.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.374+0000 7fab5ac00640 1 -- 172.21.15.94:0/2120420606 shutdown_connections 2024-09-10T23:35:35.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.374+0000 7fab5ac00640 1 --2- 172.21.15.94:0/2120420606 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fab300727c0 0x7fab30074c80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:35.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.374+0000 7fab5ac00640 1 --2- 172.21.15.94:0/2120420606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab5415ba50 0x7fab541f6fd0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:35.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.374+0000 7fab5ac00640 1 --2- 172.21.15.94:0/2120420606 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab540cfcb0 0x7fab5415e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:35.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.374+0000 7fab5ac00640 1 --2- 172.21.15.94:0/2120420606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab540cddb0 0x7fab541fbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:35.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.374+0000 7fab5ac00640 1 -- 172.21.15.94:0/2120420606 >> 172.21.15.94:0/2120420606 conn(0x7fab5406cdf0 msgr2=0x7fab5414d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:35.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.374+0000 7fab5ac00640 1 -- 172.21.15.94:0/2120420606 shutdown_connections 2024-09-10T23:35:35.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.374+0000 7fab5ac00640 1 -- 172.21.15.94:0/2120420606 wait complete. 2024-09-10T23:35:35.403 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T23:35:35.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.550+0000 7fc6d9200640 1 -- 172.21.15.94:0/3036525098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6d406e7c0 msgr2=0x7fc6d406ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:35.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.550+0000 7fc6d9200640 1 --2- 172.21.15.94:0/3036525098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6d406e7c0 0x7fc6d406ec30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4069670 tx=0x7fc6cc01dbf0 comp rx=0 tx=0).stop 2024-09-10T23:35:35.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.550+0000 7fc6d9200640 1 -- 172.21.15.94:0/3036525098 shutdown_connections 2024-09-10T23:35:35.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.550+0000 7fc6d9200640 1 --2- 172.21.15.94:0/3036525098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6d406e7c0 0x7fc6d406ec30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:35.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.550+0000 7fc6d9200640 1 --2- 172.21.15.94:0/3036525098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6d40c64f0 0x7fc6d40ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:35.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.550+0000 7fc6d9200640 1 --2- 172.21.15.94:0/3036525098 >> v2:172.21.15.181:3300/0 conn(0x7fc6d40ce280 0x7fc6d40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:35.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.550+0000 7fc6d9200640 1 -- 172.21.15.94:0/3036525098 >> 172.21.15.94:0/3036525098 conn(0x7fc6d40cfaf0 msgr2=0x7fc6d40cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:35.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.550+0000 7fc6d9200640 1 -- 172.21.15.94:0/3036525098 shutdown_connections 2024-09-10T23:35:35.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6d9200640 1 -- 172.21.15.94:0/3036525098 wait complete. 2024-09-10T23:35:35.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6d9200640 1 Processor -- start 2024-09-10T23:35:35.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6d9200640 1 -- start start 2024-09-10T23:35:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6d9200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6d406e7c0 0x7fc6d41f8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6d9200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6d40c64f0 0x7fc6d4147ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6d9200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6d40ce280 0x7fc6d4158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:35.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6d9200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d415b400 con 0x7fc6d40c64f0 2024-09-10T23:35:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6d9200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d415b280 con 0x7fc6d406e7c0 2024-09-10T23:35:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6d9200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc6d415b580 con 0x7fc6d40ce280 2024-09-10T23:35:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6c9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6d40c64f0 0x7fc6d4147ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6c9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6d40c64f0 0x7fc6d4147ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42384/0 (socket says 172.21.15.94:42384) 2024-09-10T23:35:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6c9400640 1 -- 172.21.15.94:0/420637722 learned_addr learned my addr 172.21.15.94:0/420637722 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6d1e00640 1 --2- 172.21.15.94:0/420637722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6d406e7c0 0x7fc6d41f8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6d2800640 1 --2- 172.21.15.94:0/420637722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6d40ce280 0x7fc6d4158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6c9400640 1 -- 172.21.15.94:0/420637722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6d40ce280 msgr2=0x7fc6d4158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6c9400640 1 --2- 172.21.15.94:0/420637722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6d40ce280 0x7fc6d4158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6c9400640 1 -- 172.21.15.94:0/420637722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6d406e7c0 msgr2=0x7fc6d41f8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6c9400640 1 --2- 172.21.15.94:0/420637722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6d406e7c0 0x7fc6d41f8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6c9400640 1 -- 172.21.15.94:0/420637722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d4158ac0 con 0x7fc6d40c64f0 2024-09-10T23:35:35.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6c9400640 1 --2- 172.21.15.94:0/420637722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6d40c64f0 0x7fc6d4147ca0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc00c940 tx=0x7fc6bc00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:35.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6caa00640 1 -- 172.21.15.94:0/420637722 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc007bb0 con 0x7fc6d40c64f0 2024-09-10T23:35:35.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6caa00640 1 -- 172.21.15.94:0/420637722 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6bc00f450 con 0x7fc6d40c64f0 2024-09-10T23:35:35.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6caa00640 1 -- 172.21.15.94:0/420637722 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc007e60 con 0x7fc6d40c64f0 2024-09-10T23:35:35.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.554+0000 7fc6d9200640 1 -- 172.21.15.94:0/420637722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d4148420 con 0x7fc6d40c64f0 2024-09-10T23:35:35.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.558+0000 7fc6d9200640 1 -- 172.21.15.94:0/420637722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d41489e0 con 0x7fc6d40c64f0 2024-09-10T23:35:35.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.558+0000 7fc6d9200640 1 -- 172.21.15.94:0/420637722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d40ce660 con 0x7fc6d40c64f0 2024-09-10T23:35:35.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.558+0000 7fc6caa00640 1 -- 172.21.15.94:0/420637722 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7fc6bc005020 con 0x7fc6d40c64f0 2024-09-10T23:35:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.558+0000 7fc6caa00640 1 --2- 172.21.15.94:0/420637722 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc6b0072790 0x7fc6b0074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:35.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.558+0000 7fc6caa00640 1 -- 172.21.15.94:0/420637722 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5227+0+0 (secure 0 0 0) 0x7fc6bc093bf0 con 0x7fc6d40c64f0 2024-09-10T23:35:35.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.562+0000 7fc6d1e00640 1 --2- 172.21.15.94:0/420637722 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc6b0072790 0x7fc6b0074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:35.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.562+0000 7fc6d1e00640 1 --2- 172.21.15.94:0/420637722 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc6b0072790 0x7fc6b0074c50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4005fd0 tx=0x7fc6c400a590 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:35.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.562+0000 7fc6caa00640 1 -- 172.21.15.94:0/420637722 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc6bc014030 con 0x7fc6d40c64f0 2024-09-10T23:35:35.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:35.726+0000 7fc6d9200640 1 -- 172.21.15.94:0/420637722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc6d40d0850 con 0x7fc6d40c64f0 2024-09-10T23:35:36.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.378+0000 7fc6caa00640 1 -- 172.21.15.94:0/420637722 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) ==== 170+0+0 (secure 0 0 0) 0x7fc6bc05ec40 con 0x7fc6d40c64f0 2024-09-10T23:35:36.381 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-09-10T23:35:36.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.382+0000 7fc6d9200640 1 -- 172.21.15.94:0/420637722 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc6b0072790 msgr2=0x7fc6b0074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:36.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.382+0000 7fc6d9200640 1 --2- 172.21.15.94:0/420637722 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc6b0072790 0x7fc6b0074c50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4005fd0 tx=0x7fc6c400a590 comp rx=0 tx=0).stop 2024-09-10T23:35:36.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.382+0000 7fc6d9200640 1 -- 172.21.15.94:0/420637722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6d40c64f0 msgr2=0x7fc6d4147ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:36.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.382+0000 7fc6d9200640 1 --2- 172.21.15.94:0/420637722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6d40c64f0 0x7fc6d4147ca0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc00c940 tx=0x7fc6bc00ce00 comp rx=0 tx=0).stop 2024-09-10T23:35:36.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.382+0000 7fc6d9200640 1 -- 172.21.15.94:0/420637722 shutdown_connections 2024-09-10T23:35:36.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.382+0000 7fc6d9200640 1 --2- 172.21.15.94:0/420637722 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc6b0072790 0x7fc6b0074c50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:36.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.382+0000 7fc6d9200640 1 --2- 172.21.15.94:0/420637722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6d40ce280 0x7fc6d4158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:36.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.382+0000 7fc6d9200640 1 --2- 172.21.15.94:0/420637722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6d40c64f0 0x7fc6d4147ca0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:36.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.382+0000 7fc6d9200640 1 --2- 172.21.15.94:0/420637722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6d406e7c0 0x7fc6d41f8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:36.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.382+0000 7fc6d9200640 1 -- 172.21.15.94:0/420637722 >> 172.21.15.94:0/420637722 conn(0x7fc6d40cfaf0 msgr2=0x7fc6d40af720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:36.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.382+0000 7fc6d9200640 1 -- 172.21.15.94:0/420637722 shutdown_connections 2024-09-10T23:35:36.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.382+0000 7fc6d9200640 1 -- 172.21.15.94:0/420637722 wait complete. 2024-09-10T23:35:36.418 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T23:35:36.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc0d600640 1 -- 172.21.15.94:0/3108290784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc0806e7c0 msgr2=0x7fdc0806ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:36.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc0d600640 1 --2- 172.21.15.94:0/3108290784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc0806e7c0 0x7fdc0806ebf0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdc08069670 tx=0x7fdc0001cd80 comp rx=0 tx=0).stop 2024-09-10T23:35:36.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc0d600640 1 -- 172.21.15.94:0/3108290784 shutdown_connections 2024-09-10T23:35:36.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc0d600640 1 --2- 172.21.15.94:0/3108290784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc0806e7c0 0x7fdc0806ebf0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:36.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc0d600640 1 --2- 172.21.15.94:0/3108290784 >> v2:172.21.15.181:3300/0 conn(0x7fdc080ce280 0x7fdc080cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:36.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc0d600640 1 -- 172.21.15.94:0/3108290784 >> 172.21.15.94:0/3108290784 conn(0x7fdc080cfaf0 msgr2=0x7fdc080cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:36.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc0d600640 1 -- 172.21.15.94:0/3108290784 shutdown_connections 2024-09-10T23:35:36.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc0d600640 1 -- 172.21.15.94:0/3108290784 wait complete. 2024-09-10T23:35:36.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc0d600640 1 Processor -- start 2024-09-10T23:35:36.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc0d600640 1 -- start start 2024-09-10T23:35:36.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc0d600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0806e7c0 0x7fdc081430f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:36.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc0d600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc080c64f0 0x7fdc080b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:36.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc0d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc080ce280 0x7fdc081458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:36.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc0d600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0815b530 con 0x7fdc080ce280 2024-09-10T23:35:36.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc0d600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0815b3b0 con 0x7fdc0806e7c0 2024-09-10T23:35:36.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc0d600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fdc0815b6b0 con 0x7fdc080c64f0 2024-09-10T23:35:36.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc07400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc080ce280 0x7fdc081458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:36.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc07400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc080ce280 0x7fdc081458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42408/0 (socket says 172.21.15.94:42408) 2024-09-10T23:35:36.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc07400640 1 -- 172.21.15.94:0/3788417168 learned_addr learned my addr 172.21.15.94:0/3788417168 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:36.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc06000640 1 --2- 172.21.15.94:0/3788417168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc080c64f0 0x7fdc080b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:36.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.562+0000 7fdc06a00640 1 --2- 172.21.15.94:0/3788417168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0806e7c0 0x7fdc081430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:36.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.566+0000 7fdc07400640 1 -- 172.21.15.94:0/3788417168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc080c64f0 msgr2=0x7fdc080b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:36.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.566+0000 7fdc07400640 1 --2- 172.21.15.94:0/3788417168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc080c64f0 0x7fdc080b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:36.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.566+0000 7fdc07400640 1 -- 172.21.15.94:0/3788417168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0806e7c0 msgr2=0x7fdc081430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:36.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.566+0000 7fdc07400640 1 --2- 172.21.15.94:0/3788417168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0806e7c0 0x7fdc081430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:36.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.566+0000 7fdc07400640 1 -- 172.21.15.94:0/3788417168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc08146350 con 0x7fdc080ce280 2024-09-10T23:35:36.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.566+0000 7fdc07400640 1 --2- 172.21.15.94:0/3788417168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc080ce280 0x7fdc081458b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdc00019990 tx=0x7fdc00003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:36.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.566+0000 7fdbfea00640 1 -- 172.21.15.94:0/3788417168 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc000bb070 con 0x7fdc080ce280 2024-09-10T23:35:36.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.566+0000 7fdbfea00640 1 -- 172.21.15.94:0/3788417168 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc00004e00 con 0x7fdc080ce280 2024-09-10T23:35:36.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.566+0000 7fdbfea00640 1 -- 172.21.15.94:0/3788417168 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc000b2040 con 0x7fdc080ce280 2024-09-10T23:35:36.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.566+0000 7fdc0d600640 1 -- 172.21.15.94:0/3788417168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc08146630 con 0x7fdc080ce280 2024-09-10T23:35:36.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.566+0000 7fdc0d600640 1 -- 172.21.15.94:0/3788417168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc080b4dc0 con 0x7fdc080ce280 2024-09-10T23:35:36.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.566+0000 7fdc0d600640 1 -- 172.21.15.94:0/3788417168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc0806ebf0 con 0x7fdc080ce280 2024-09-10T23:35:36.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.566+0000 7fdbfea00640 1 -- 172.21.15.94:0/3788417168 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7fdc000b5030 con 0x7fdc080ce280 2024-09-10T23:35:36.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.570+0000 7fdbfea00640 1 --2- 172.21.15.94:0/3788417168 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdbe8072790 0x7fdbe8074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:36.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.570+0000 7fdbfea00640 1 -- 172.21.15.94:0/3788417168 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5609+0+0 (secure 0 0 0) 0x7fdc0012d1b0 con 0x7fdc080ce280 2024-09-10T23:35:36.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.570+0000 7fdc06a00640 1 --2- 172.21.15.94:0/3788417168 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdbe8072790 0x7fdbe8074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:36.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.570+0000 7fdc06a00640 1 --2- 172.21.15.94:0/3788417168 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdbe8072790 0x7fdbe8074c50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdc081fae50 tx=0x7fdbf800b3e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:36.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.574+0000 7fdbfea00640 1 -- 172.21.15.94:0/3788417168 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdc00016970 con 0x7fdc080ce280 2024-09-10T23:35:36.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:36.774+0000 7fdc0d600640 1 -- 172.21.15.94:0/3788417168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fdc08069e20 con 0x7fdc080ce280 2024-09-10T23:35:37.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.382+0000 7fdbfea00640 1 -- 172.21.15.94:0/3788417168 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v30) ==== 172+0+0 (secure 0 0 0) 0x7fdc08069e20 con 0x7fdc080ce280 2024-09-10T23:35:37.383 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 8 and data pool 9 2024-09-10T23:35:37.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.386+0000 7fdc0d600640 1 -- 172.21.15.94:0/3788417168 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdbe8072790 msgr2=0x7fdbe8074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:37.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.386+0000 7fdc0d600640 1 --2- 172.21.15.94:0/3788417168 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdbe8072790 0x7fdbe8074c50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdc081fae50 tx=0x7fdbf800b3e0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.386+0000 7fdc0d600640 1 -- 172.21.15.94:0/3788417168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc080ce280 msgr2=0x7fdc081458b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:37.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.386+0000 7fdc0d600640 1 --2- 172.21.15.94:0/3788417168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc080ce280 0x7fdc081458b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdc00019990 tx=0x7fdc00003f60 comp rx=0 tx=0).stop 2024-09-10T23:35:37.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.386+0000 7fdc0d600640 1 -- 172.21.15.94:0/3788417168 shutdown_connections 2024-09-10T23:35:37.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.386+0000 7fdc0d600640 1 --2- 172.21.15.94:0/3788417168 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdbe8072790 0x7fdbe8074c50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.386+0000 7fdc0d600640 1 --2- 172.21.15.94:0/3788417168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc080ce280 0x7fdc081458b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.386+0000 7fdc0d600640 1 --2- 172.21.15.94:0/3788417168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc080c64f0 0x7fdc080b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.386+0000 7fdc0d600640 1 --2- 172.21.15.94:0/3788417168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0806e7c0 0x7fdc081430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.386+0000 7fdc0d600640 1 -- 172.21.15.94:0/3788417168 >> 172.21.15.94:0/3788417168 conn(0x7fdc080cfaf0 msgr2=0x7fdc080afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.386+0000 7fdc0d600640 1 -- 172.21.15.94:0/3788417168 shutdown_connections 2024-09-10T23:35:37.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.386+0000 7fdc0d600640 1 -- 172.21.15.94:0/3788417168 wait complete. 2024-09-10T23:35:37.436 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:35:37.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.578+0000 7f04eb600640 1 -- 172.21.15.94:0/1289168851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04e40cddb0 msgr2=0x7f04e40ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:37.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.578+0000 7f04eb600640 1 --2- 172.21.15.94:0/1289168851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04e40cddb0 0x7f04e40ce200 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f04e40b3f30 tx=0x7f04dc01cd50 comp rx=0 tx=0).stop 2024-09-10T23:35:37.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.578+0000 7f04eb600640 1 -- 172.21.15.94:0/1289168851 shutdown_connections 2024-09-10T23:35:37.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.578+0000 7f04eb600640 1 --2- 172.21.15.94:0/1289168851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04e40cddb0 0x7f04e40ce200 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.578+0000 7f04eb600640 1 --2- 172.21.15.94:0/1289168851 >> v2:172.21.15.181:3300/0 conn(0x7f04e40cfcb0 0x7f04e40d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.578+0000 7f04eb600640 1 -- 172.21.15.94:0/1289168851 >> 172.21.15.94:0/1289168851 conn(0x7f04e406cdf0 msgr2=0x7f04e406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:37.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.578+0000 7f04eb600640 1 -- 172.21.15.94:0/1289168851 shutdown_connections 2024-09-10T23:35:37.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.578+0000 7f04eb600640 1 -- 172.21.15.94:0/1289168851 wait complete. 2024-09-10T23:35:37.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.578+0000 7f04eb600640 1 Processor -- start 2024-09-10T23:35:37.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.578+0000 7f04eb600640 1 -- start start 2024-09-10T23:35:37.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.578+0000 7f04eb600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04e40cddb0 0x7f04e4145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:37.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.578+0000 7f04eb600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04e40cfcb0 0x7f04e4150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:37.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.578+0000 7f04eb600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04e415ba50 0x7f04e4146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:37.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.578+0000 7f04eb600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4168be0 con 0x7f04e40cddb0 2024-09-10T23:35:37.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.578+0000 7f04eb600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4168a60 con 0x7f04e40cfcb0 2024-09-10T23:35:37.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.578+0000 7f04eb600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f04e4168d60 con 0x7f04e415ba50 2024-09-10T23:35:37.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04e8c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04e40cddb0 0x7f04e4145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:37.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04e8c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04e40cddb0 0x7f04e4145d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42426/0 (socket says 172.21.15.94:42426) 2024-09-10T23:35:37.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04e8c00640 1 -- 172.21.15.94:0/1052318215 learned_addr learned my addr 172.21.15.94:0/1052318215 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:37.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04e9600640 1 --2- 172.21.15.94:0/1052318215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04e415ba50 0x7f04e4146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:37.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04dbe00640 1 --2- 172.21.15.94:0/1052318215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04e40cfcb0 0x7f04e4150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:37.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04e8c00640 1 -- 172.21.15.94:0/1052318215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04e415ba50 msgr2=0x7f04e4146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:37.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04e8c00640 1 --2- 172.21.15.94:0/1052318215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04e415ba50 0x7f04e4146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04e8c00640 1 -- 172.21.15.94:0/1052318215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04e40cfcb0 msgr2=0x7f04e4150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:37.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04e8c00640 1 --2- 172.21.15.94:0/1052318215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04e40cfcb0 0x7f04e4150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04e8c00640 1 -- 172.21.15.94:0/1052318215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e41f9330 con 0x7f04e40cddb0 2024-09-10T23:35:37.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04e8c00640 1 --2- 172.21.15.94:0/1052318215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04e40cddb0 0x7f04e4145d20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f04d400b410 tx=0x7f04d400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:37.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04e1600640 1 -- 172.21.15.94:0/1052318215 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4015030 con 0x7f04e40cddb0 2024-09-10T23:35:37.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04e1600640 1 -- 172.21.15.94:0/1052318215 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04d4004020 con 0x7f04e40cddb0 2024-09-10T23:35:37.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04e1600640 1 -- 172.21.15.94:0/1052318215 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d40044b0 con 0x7f04e40cddb0 2024-09-10T23:35:37.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04eb600640 1 -- 172.21.15.94:0/1052318215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e41f9610 con 0x7f04e40cddb0 2024-09-10T23:35:37.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04eb600640 1 -- 172.21.15.94:0/1052318215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e41f9bd0 con 0x7f04e40cddb0 2024-09-10T23:35:37.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04eb600640 1 -- 172.21.15.94:0/1052318215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e41f9e10 con 0x7f04e40cddb0 2024-09-10T23:35:37.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.582+0000 7f04e1600640 1 -- 172.21.15.94:0/1052318215 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f04d4022060 con 0x7f04e40cddb0 2024-09-10T23:35:37.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.586+0000 7f04e1600640 1 --2- 172.21.15.94:0/1052318215 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f04c0072790 0x7f04c0074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:37.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.586+0000 7f04e1600640 1 -- 172.21.15.94:0/1052318215 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f04d4093080 con 0x7f04e40cddb0 2024-09-10T23:35:37.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.586+0000 7f04dbe00640 1 --2- 172.21.15.94:0/1052318215 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f04c0072790 0x7f04c0074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:37.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.586+0000 7f04dbe00640 1 --2- 172.21.15.94:0/1052318215 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f04c0072790 0x7f04c0074c50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f04e4155a80 tx=0x7f04cc00b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:37.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.590+0000 7f04e1600640 1 -- 172.21.15.94:0/1052318215 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f04d405dd70 con 0x7f04e40cddb0 2024-09-10T23:35:37.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.754+0000 7f04eb600640 1 -- 172.21.15.94:0/1052318215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04e40ce200 con 0x7f04e40cddb0 2024-09-10T23:35:37.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.754+0000 7f04e1600640 1 -- 172.21.15.94:0/1052318215 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16726 (secure 0 0 0) 0x7f04d40627a0 con 0x7f04e40cddb0 2024-09-10T23:35:37.757 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:37.757 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":57,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:35:36.778201+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:35:34.484637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T23:35:35.740496+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:35:37.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.758+0000 7f04eb600640 1 -- 172.21.15.94:0/1052318215 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f04c0072790 msgr2=0x7f04c0074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:37.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.758+0000 7f04eb600640 1 --2- 172.21.15.94:0/1052318215 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f04c0072790 0x7f04c0074c50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f04e4155a80 tx=0x7f04cc00b390 comp rx=0 tx=0).stop 2024-09-10T23:35:37.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.758+0000 7f04eb600640 1 -- 172.21.15.94:0/1052318215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04e40cddb0 msgr2=0x7f04e4145d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:37.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.758+0000 7f04eb600640 1 --2- 172.21.15.94:0/1052318215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04e40cddb0 0x7f04e4145d20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f04d400b410 tx=0x7f04d400b8d0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.758+0000 7f04eb600640 1 -- 172.21.15.94:0/1052318215 shutdown_connections 2024-09-10T23:35:37.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.758+0000 7f04eb600640 1 --2- 172.21.15.94:0/1052318215 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f04c0072790 0x7f04c0074c50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.758+0000 7f04eb600640 1 --2- 172.21.15.94:0/1052318215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04e40cfcb0 0x7f04e4150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.758+0000 7f04eb600640 1 --2- 172.21.15.94:0/1052318215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04e415ba50 0x7f04e4146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.758+0000 7f04eb600640 1 --2- 172.21.15.94:0/1052318215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04e40cddb0 0x7f04e4145d20 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.758+0000 7f04eb600640 1 -- 172.21.15.94:0/1052318215 >> 172.21.15.94:0/1052318215 conn(0x7f04e406cdf0 msgr2=0x7f04e4148190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:37.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.758+0000 7f04eb600640 1 -- 172.21.15.94:0/1052318215 shutdown_connections 2024-09-10T23:35:37.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.758+0000 7f04eb600640 1 -- 172.21.15.94:0/1052318215 wait complete. 2024-09-10T23:35:37.779 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:35:37.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.918+0000 7f483f200640 1 -- 172.21.15.94:0/306251293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f484015ba30 msgr2=0x7f48400b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:37.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.918+0000 7f483f200640 1 --2- 172.21.15.94:0/306251293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f484015ba30 0x7f48400b1bd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f48400c5320 tx=0x7f483001c890 comp rx=0 tx=0).stop 2024-09-10T23:35:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.918+0000 7f483f200640 1 -- 172.21.15.94:0/306251293 shutdown_connections 2024-09-10T23:35:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.918+0000 7f483f200640 1 --2- 172.21.15.94:0/306251293 >> v2:172.21.15.181:3300/0 conn(0x7f48400b2110 0x7f48400cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.918+0000 7f483f200640 1 --2- 172.21.15.94:0/306251293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f484015ba30 0x7f48400b1bd0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.918+0000 7f483f200640 1 -- 172.21.15.94:0/306251293 >> 172.21.15.94:0/306251293 conn(0x7f484009aa70 msgr2=0x7f484009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.918+0000 7f483f200640 1 -- 172.21.15.94:0/306251293 shutdown_connections 2024-09-10T23:35:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.918+0000 7f483f200640 1 -- 172.21.15.94:0/306251293 wait complete. 2024-09-10T23:35:37.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.918+0000 7f483f200640 1 Processor -- start 2024-09-10T23:35:37.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.918+0000 7f483f200640 1 -- start start 2024-09-10T23:35:37.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48400b2110 0x7f48400b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:37.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48400d02e0 0x7f484015a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:37.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483f200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f484015ba30 0x7f48400ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:37.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4840168b50 con 0x7f48400d02e0 2024-09-10T23:35:37.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483f200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f48401689d0 con 0x7f48400b2110 2024-09-10T23:35:37.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483f200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f4840168cd0 con 0x7f484015ba30 2024-09-10T23:35:37.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48400d02e0 0x7f484015a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:37.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48400d02e0 0x7f484015a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42442/0 (socket says 172.21.15.94:42442) 2024-09-10T23:35:37.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483d400640 1 -- 172.21.15.94:0/2977010616 learned_addr learned my addr 172.21.15.94:0/2977010616 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:37.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483de00640 1 --2- 172.21.15.94:0/2977010616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48400b2110 0x7f48400b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:37.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483e800640 1 --2- 172.21.15.94:0/2977010616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f484015ba30 0x7f48400ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:37.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483d400640 1 -- 172.21.15.94:0/2977010616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f484015ba30 msgr2=0x7f48400ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:37.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483d400640 1 --2- 172.21.15.94:0/2977010616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f484015ba30 0x7f48400ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483d400640 1 -- 172.21.15.94:0/2977010616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48400b2110 msgr2=0x7f48400b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:37.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483d400640 1 --2- 172.21.15.94:0/2977010616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48400b2110 0x7f48400b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:37.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483d400640 1 -- 172.21.15.94:0/2977010616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4840158c80 con 0x7f48400d02e0 2024-09-10T23:35:37.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483d400640 1 --2- 172.21.15.94:0/2977010616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48400d02e0 0x7f484015a5b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f482c00d980 tx=0x7f482c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:37.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f4836000640 1 -- 172.21.15.94:0/2977010616 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c017030 con 0x7f48400d02e0 2024-09-10T23:35:37.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f4836000640 1 -- 172.21.15.94:0/2977010616 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f482c009b20 con 0x7f48400d02e0 2024-09-10T23:35:37.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483f200640 1 -- 172.21.15.94:0/2977010616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4840158f60 con 0x7f48400d02e0 2024-09-10T23:35:37.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f4836000640 1 -- 172.21.15.94:0/2977010616 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c004020 con 0x7f48400d02e0 2024-09-10T23:35:37.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483f200640 1 -- 172.21.15.94:0/2977010616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f484015aaf0 con 0x7f48400d02e0 2024-09-10T23:35:37.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.922+0000 7f483f200640 1 -- 172.21.15.94:0/2977010616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48400cfda0 con 0x7f48400d02e0 2024-09-10T23:35:37.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.926+0000 7f4836000640 1 -- 172.21.15.94:0/2977010616 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f482c024060 con 0x7f48400d02e0 2024-09-10T23:35:37.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.926+0000 7f4836000640 1 --2- 172.21.15.94:0/2977010616 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4820072790 0x7f4820074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:37.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.926+0000 7f4836000640 1 -- 172.21.15.94:0/2977010616 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f482c0940f0 con 0x7f48400d02e0 2024-09-10T23:35:37.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.926+0000 7f483de00640 1 --2- 172.21.15.94:0/2977010616 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4820072790 0x7f4820074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:37.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.926+0000 7f483de00640 1 --2- 172.21.15.94:0/2977010616 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4820072790 0x7f4820074c50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f48300af000 tx=0x7f48300a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:37.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:37.930+0000 7f4836000640 1 -- 172.21.15.94:0/2977010616 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f482c05edd0 con 0x7f48400d02e0 2024-09-10T23:35:38.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.094+0000 7f483f200640 1 -- 172.21.15.94:0/2977010616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48400d1b90 con 0x7f48400d02e0 2024-09-10T23:35:38.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.094+0000 7f4836000640 1 -- 172.21.15.94:0/2977010616 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16726 (secure 0 0 0) 0x7f482c063800 con 0x7f48400d02e0 2024-09-10T23:35:38.097 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:38.098 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":57,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:35:36.778201+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:35:34.484637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T23:35:35.740496+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:35:38.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.098+0000 7f483f200640 1 -- 172.21.15.94:0/2977010616 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4820072790 msgr2=0x7f4820074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:38.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.098+0000 7f483f200640 1 --2- 172.21.15.94:0/2977010616 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4820072790 0x7f4820074c50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f48300af000 tx=0x7f48300a7040 comp rx=0 tx=0).stop 2024-09-10T23:35:38.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.098+0000 7f483f200640 1 -- 172.21.15.94:0/2977010616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48400d02e0 msgr2=0x7f484015a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:38.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.098+0000 7f483f200640 1 --2- 172.21.15.94:0/2977010616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48400d02e0 0x7f484015a5b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f482c00d980 tx=0x7f482c00de40 comp rx=0 tx=0).stop 2024-09-10T23:35:38.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.098+0000 7f483f200640 1 -- 172.21.15.94:0/2977010616 shutdown_connections 2024-09-10T23:35:38.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.098+0000 7f483f200640 1 --2- 172.21.15.94:0/2977010616 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4820072790 0x7f4820074c50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:38.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.098+0000 7f483f200640 1 --2- 172.21.15.94:0/2977010616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f484015ba30 0x7f48400ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:38.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.098+0000 7f483f200640 1 --2- 172.21.15.94:0/2977010616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48400d02e0 0x7f484015a5b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:38.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.098+0000 7f483f200640 1 --2- 172.21.15.94:0/2977010616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48400b2110 0x7f48400b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:38.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.098+0000 7f483f200640 1 -- 172.21.15.94:0/2977010616 >> 172.21.15.94:0/2977010616 conn(0x7f484009aa70 msgr2=0x7f48400cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:38.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.098+0000 7f483f200640 1 -- 172.21.15.94:0/2977010616 shutdown_connections 2024-09-10T23:35:38.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.098+0000 7f483f200640 1 -- 172.21.15.94:0/2977010616 wait complete. 2024-09-10T23:35:38.119 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T23:35:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.262+0000 7f6658800640 1 -- 172.21.15.94:0/3198983852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66500b1100 msgr2=0x7f66500b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.262+0000 7f6658800640 1 --2- 172.21.15.94:0/3198983852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66500b1100 0x7f66500b1590 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6650073990 tx=0x7f664c01caf0 comp rx=0 tx=0).stop 2024-09-10T23:35:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.262+0000 7f6658800640 1 -- 172.21.15.94:0/3198983852 shutdown_connections 2024-09-10T23:35:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.262+0000 7f6658800640 1 --2- 172.21.15.94:0/3198983852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66500b1100 0x7f66500b1590 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.262+0000 7f6658800640 1 --2- 172.21.15.94:0/3198983852 >> v2:172.21.15.181:3300/0 conn(0x7f66500ce890 0x7f66500cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.262+0000 7f6658800640 1 -- 172.21.15.94:0/3198983852 >> 172.21.15.94:0/3198983852 conn(0x7f66500d1c60 msgr2=0x7f66500d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.262+0000 7f6658800640 1 -- 172.21.15.94:0/3198983852 shutdown_connections 2024-09-10T23:35:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.262+0000 7f6658800640 1 -- 172.21.15.94:0/3198983852 wait complete. 2024-09-10T23:35:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6658800640 1 Processor -- start 2024-09-10T23:35:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6658800640 1 -- start start 2024-09-10T23:35:38.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6658800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66500b1100 0x7f665014fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:38.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6658800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66500ce1e0 0x7f66500af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:38.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6658800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66500ce890 0x7f66500afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:38.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6658800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6650169110 con 0x7f66500b1100 2024-09-10T23:35:38.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6658800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f66500d0a30 con 0x7f66500ce1e0 2024-09-10T23:35:38.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6658800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6650169290 con 0x7f66500ce890 2024-09-10T23:35:38.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6655e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66500b1100 0x7f665014fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:38.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6655e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66500b1100 0x7f665014fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42464/0 (socket says 172.21.15.94:42464) 2024-09-10T23:35:38.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6655e00640 1 -- 172.21.15.94:0/2208702466 learned_addr learned my addr 172.21.15.94:0/2208702466 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:38.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6655400640 1 --2- 172.21.15.94:0/2208702466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66500ce1e0 0x7f66500af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:38.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6656800640 1 --2- 172.21.15.94:0/2208702466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66500ce890 0x7f66500afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:38.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6655e00640 1 -- 172.21.15.94:0/2208702466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66500ce890 msgr2=0x7f66500afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:38.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6655e00640 1 --2- 172.21.15.94:0/2208702466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66500ce890 0x7f66500afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:38.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6655e00640 1 -- 172.21.15.94:0/2208702466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66500ce1e0 msgr2=0x7f66500af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:38.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6655e00640 1 --2- 172.21.15.94:0/2208702466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66500ce1e0 0x7f66500af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:38.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6655e00640 1 -- 172.21.15.94:0/2208702466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6650153ba0 con 0x7f66500b1100 2024-09-10T23:35:38.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6655e00640 1 --2- 172.21.15.94:0/2208702466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66500b1100 0x7f665014fbd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f664400d950 tx=0x7f664400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:38.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f664a000640 1 -- 172.21.15.94:0/2208702466 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644017070 con 0x7f66500b1100 2024-09-10T23:35:38.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f664a000640 1 -- 172.21.15.94:0/2208702466 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6644009d20 con 0x7f66500b1100 2024-09-10T23:35:38.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f664a000640 1 -- 172.21.15.94:0/2208702466 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66440106c0 con 0x7f66500b1100 2024-09-10T23:35:38.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6658800640 1 -- 172.21.15.94:0/2208702466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6650153e80 con 0x7f66500b1100 2024-09-10T23:35:38.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6658800640 1 -- 172.21.15.94:0/2208702466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6650154440 con 0x7f66500b1100 2024-09-10T23:35:38.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.266+0000 7f6658800640 1 -- 172.21.15.94:0/2208702466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66500b1590 con 0x7f66500b1100 2024-09-10T23:35:38.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.270+0000 7f664a000640 1 -- 172.21.15.94:0/2208702466 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f6644010860 con 0x7f66500b1100 2024-09-10T23:35:38.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.270+0000 7f664a000640 1 --2- 172.21.15.94:0/2208702466 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6634072790 0x7f6634074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:38.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.270+0000 7f664a000640 1 -- 172.21.15.94:0/2208702466 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f6644093b10 con 0x7f66500b1100 2024-09-10T23:35:38.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.270+0000 7f6655400640 1 --2- 172.21.15.94:0/2208702466 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6634072790 0x7f6634074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:38.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.270+0000 7f6655400640 1 --2- 172.21.15.94:0/2208702466 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6634072790 0x7f6634074c50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f66500b78c0 tx=0x7f6640009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:38.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.274+0000 7f664a000640 1 -- 172.21.15.94:0/2208702466 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f664405ee50 con 0x7f66500b1100 2024-09-10T23:35:38.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:38.478+0000 7f6658800640 1 -- 172.21.15.94:0/2208702466 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f66500ccec0 con 0x7f66500b1100 2024-09-10T23:35:39.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.402+0000 7f664a000640 1 -- 172.21.15.94:0/2208702466 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) ==== 118+0+0 (secure 0 0 0) 0x7f664405e750 con 0x7f66500b1100 2024-09-10T23:35:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.406+0000 7f6658800640 1 -- 172.21.15.94:0/2208702466 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6634072790 msgr2=0x7f6634074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.406+0000 7f6658800640 1 --2- 172.21.15.94:0/2208702466 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6634072790 0x7f6634074c50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f66500b78c0 tx=0x7f6640009040 comp rx=0 tx=0).stop 2024-09-10T23:35:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.406+0000 7f6658800640 1 -- 172.21.15.94:0/2208702466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66500b1100 msgr2=0x7f665014fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.406+0000 7f6658800640 1 --2- 172.21.15.94:0/2208702466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66500b1100 0x7f665014fbd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f664400d950 tx=0x7f664400de10 comp rx=0 tx=0).stop 2024-09-10T23:35:39.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.406+0000 7f6658800640 1 -- 172.21.15.94:0/2208702466 shutdown_connections 2024-09-10T23:35:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.406+0000 7f6658800640 1 --2- 172.21.15.94:0/2208702466 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6634072790 0x7f6634074c50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.406+0000 7f6658800640 1 --2- 172.21.15.94:0/2208702466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66500ce1e0 0x7f66500af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.406+0000 7f6658800640 1 --2- 172.21.15.94:0/2208702466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66500ce890 0x7f66500afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.406+0000 7f6658800640 1 --2- 172.21.15.94:0/2208702466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66500b1100 0x7f665014fbd0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.406+0000 7f6658800640 1 -- 172.21.15.94:0/2208702466 >> 172.21.15.94:0/2208702466 conn(0x7f66500d1c60 msgr2=0x7f66500720a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.406+0000 7f6658800640 1 -- 172.21.15.94:0/2208702466 shutdown_connections 2024-09-10T23:35:39.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.406+0000 7f6658800640 1 -- 172.21.15.94:0/2208702466 wait complete. 2024-09-10T23:35:39.428 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T23:35:39.428 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T23:35:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.570+0000 7fab41a00640 1 -- 172.21.15.94:0/740099481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab20003a10 msgr2=0x7fab20003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.570+0000 7fab41a00640 1 --2- 172.21.15.94:0/740099481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab20003a10 0x7fab20003e30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fab2c005fd0 tx=0x7fab2c01cda0 comp rx=0 tx=0).stop 2024-09-10T23:35:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.570+0000 7fab3b400640 1 -- 172.21.15.94:0/740099481 reap_dead start 2024-09-10T23:35:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.570+0000 7fab41a00640 1 -- 172.21.15.94:0/740099481 shutdown_connections 2024-09-10T23:35:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.570+0000 7fab41a00640 1 -- 172.21.15.94:0/740099481 >> 172.21.15.94:0/740099481 conn(0x7fab3c0cfaf0 msgr2=0x7fab3c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.570+0000 7fab41a00640 1 -- 172.21.15.94:0/740099481 shutdown_connections 2024-09-10T23:35:39.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.570+0000 7fab41a00640 1 -- 172.21.15.94:0/740099481 wait complete. 2024-09-10T23:35:39.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab41a00640 1 Processor -- start 2024-09-10T23:35:39.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab41a00640 1 -- start start 2024-09-10T23:35:39.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab41a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab20003a10 0x7fab3c0b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:39.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab41a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab3c14e980 0x7fab3c0c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:39.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab41a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab3c150ef0 0x7fab3c0c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:39.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab41a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c06ef60 con 0x7fab3c14e980 2024-09-10T23:35:39.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab41a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c06ede0 con 0x7fab20003a10 2024-09-10T23:35:39.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab41a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fab3c1652e0 con 0x7fab3c150ef0 2024-09-10T23:35:39.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab3a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab3c14e980 0x7fab3c0c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:39.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab3a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab3c14e980 0x7fab3c0c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42496/0 (socket says 172.21.15.94:42496) 2024-09-10T23:35:39.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab3a000640 1 -- 172.21.15.94:0/1956447077 learned_addr learned my addr 172.21.15.94:0/1956447077 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:39.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab3aa00640 1 --2- 172.21.15.94:0/1956447077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab20003a10 0x7fab3c0b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:39.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab3b400640 1 --2- 172.21.15.94:0/1956447077 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab3c150ef0 0x7fab3c0c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:39.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab3a000640 1 -- 172.21.15.94:0/1956447077 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab3c150ef0 msgr2=0x7fab3c0c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:39.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab3a000640 1 --2- 172.21.15.94:0/1956447077 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab3c150ef0 0x7fab3c0c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:39.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab3a000640 1 -- 172.21.15.94:0/1956447077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab20003a10 msgr2=0x7fab3c0b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:39.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab3a000640 1 --2- 172.21.15.94:0/1956447077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab20003a10 0x7fab3c0b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:39.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab3a000640 1 -- 172.21.15.94:0/1956447077 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3c0b1560 con 0x7fab3c14e980 2024-09-10T23:35:39.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab3a000640 1 --2- 172.21.15.94:0/1956447077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab3c14e980 0x7fab3c0c5230 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fab2800f9d0 tx=0x7fab2800fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:39.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab32a00640 1 -- 172.21.15.94:0/1956447077 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2800bcb0 con 0x7fab3c14e980 2024-09-10T23:35:39.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab32a00640 1 -- 172.21.15.94:0/1956447077 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab28011440 con 0x7fab3c14e980 2024-09-10T23:35:39.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab41a00640 1 -- 172.21.15.94:0/1956447077 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3c1f93d0 con 0x7fab3c14e980 2024-09-10T23:35:39.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab32a00640 1 -- 172.21.15.94:0/1956447077 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28011850 con 0x7fab3c14e980 2024-09-10T23:35:39.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.574+0000 7fab41a00640 1 -- 172.21.15.94:0/1956447077 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab3c1f9990 con 0x7fab3c14e980 2024-09-10T23:35:39.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.578+0000 7fab41a00640 1 -- 172.21.15.94:0/1956447077 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab3c160fa0 con 0x7fab3c14e980 2024-09-10T23:35:39.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.578+0000 7fab32a00640 1 -- 172.21.15.94:0/1956447077 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7fab28023030 con 0x7fab3c14e980 2024-09-10T23:35:39.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.578+0000 7fab32a00640 1 --2- 172.21.15.94:0/1956447077 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fab1c0727c0 0x7fab1c074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:39.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.578+0000 7fab32a00640 1 -- 172.21.15.94:0/1956447077 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fab28093e00 con 0x7fab3c14e980 2024-09-10T23:35:39.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.578+0000 7fab3aa00640 1 --2- 172.21.15.94:0/1956447077 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fab1c0727c0 0x7fab1c074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:39.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.578+0000 7fab3aa00640 1 --2- 172.21.15.94:0/1956447077 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fab1c0727c0 0x7fab1c074c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0b0000 tx=0x7fab2c0a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.582+0000 7fab32a00640 1 -- 172.21.15.94:0/1956447077 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fab3c0b1560 con 0x7fab3c14e980 2024-09-10T23:35:39.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:39.782+0000 7fab41a00640 1 -- 172.21.15.94:0/1956447077 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fab3c0d0850 con 0x7fab3c14e980 2024-09-10T23:35:40.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.410+0000 7fab32a00640 1 -- 172.21.15.94:0/1956447077 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fab2805eba0 con 0x7fab3c14e980 2024-09-10T23:35:40.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.410+0000 7fab41a00640 1 -- 172.21.15.94:0/1956447077 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fab1c0727c0 msgr2=0x7fab1c074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.410+0000 7fab41a00640 1 --2- 172.21.15.94:0/1956447077 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fab1c0727c0 0x7fab1c074c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0b0000 tx=0x7fab2c0a7040 comp rx=0 tx=0).stop 2024-09-10T23:35:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.410+0000 7fab41a00640 1 -- 172.21.15.94:0/1956447077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab3c14e980 msgr2=0x7fab3c0c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.410+0000 7fab41a00640 1 --2- 172.21.15.94:0/1956447077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab3c14e980 0x7fab3c0c5230 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fab2800f9d0 tx=0x7fab2800fe90 comp rx=0 tx=0).stop 2024-09-10T23:35:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.410+0000 7fab41a00640 1 -- 172.21.15.94:0/1956447077 shutdown_connections 2024-09-10T23:35:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.410+0000 7fab41a00640 1 --2- 172.21.15.94:0/1956447077 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fab1c0727c0 0x7fab1c074c80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.410+0000 7fab41a00640 1 --2- 172.21.15.94:0/1956447077 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab3c150ef0 0x7fab3c0c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.410+0000 7fab41a00640 1 --2- 172.21.15.94:0/1956447077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab3c14e980 0x7fab3c0c5230 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.410+0000 7fab41a00640 1 --2- 172.21.15.94:0/1956447077 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab20003a10 0x7fab3c0b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.410+0000 7fab41a00640 1 -- 172.21.15.94:0/1956447077 >> 172.21.15.94:0/1956447077 conn(0x7fab3c0cfaf0 msgr2=0x7fab3c0ca550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.410+0000 7fab41a00640 1 -- 172.21.15.94:0/1956447077 shutdown_connections 2024-09-10T23:35:40.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.410+0000 7fab41a00640 1 -- 172.21.15.94:0/1956447077 wait complete. 2024-09-10T23:35:40.435 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T23:35:40.435 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:35:40.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.578+0000 7f1d90800640 1 -- 172.21.15.94:0/2421559620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d70004810 msgr2=0x7f1d70004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.578+0000 7f1d90800640 1 --2- 172.21.15.94:0/2421559620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d70004810 0x7f1d70004c90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009b30 tx=0x7f1d7801d270 comp rx=0 tx=0).stop 2024-09-10T23:35:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.578+0000 7f1d8e800640 1 -- 172.21.15.94:0/2421559620 reap_dead start 2024-09-10T23:35:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.578+0000 7f1d90800640 1 -- 172.21.15.94:0/2421559620 shutdown_connections 2024-09-10T23:35:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.578+0000 7f1d90800640 1 -- 172.21.15.94:0/2421559620 >> 172.21.15.94:0/2421559620 conn(0x7f1d8806cdf0 msgr2=0x7f1d8806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.578+0000 7f1d90800640 1 -- 172.21.15.94:0/2421559620 shutdown_connections 2024-09-10T23:35:40.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.578+0000 7f1d90800640 1 -- 172.21.15.94:0/2421559620 wait complete. 2024-09-10T23:35:40.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.578+0000 7f1d90800640 1 Processor -- start 2024-09-10T23:35:40.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d90800640 1 -- start start 2024-09-10T23:35:40.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d90800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d70004810 0x7f1d881f9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d90800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d88159d00 0x7f1d8814d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d90800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d881fb580 0x7f1d88152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d90800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8806f6a0 con 0x7f1d70004810 2024-09-10T23:35:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d90800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8806f520 con 0x7f1d88159d00 2024-09-10T23:35:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d90800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1d8806f820 con 0x7f1d881fb580 2024-09-10T23:35:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d8de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d70004810 0x7f1d881f9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d8de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d70004810 0x7f1d881f9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42522/0 (socket says 172.21.15.94:42522) 2024-09-10T23:35:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d8de00640 1 -- 172.21.15.94:0/2852994318 learned_addr learned my addr 172.21.15.94:0/2852994318 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d8de00640 1 -- 172.21.15.94:0/2852994318 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d881fb580 msgr2=0x7f1d88152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d8de00640 1 --2- 172.21.15.94:0/2852994318 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d881fb580 0x7f1d88152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d8de00640 1 -- 172.21.15.94:0/2852994318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d88159d00 msgr2=0x7f1d8814d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d8de00640 1 --2- 172.21.15.94:0/2852994318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d88159d00 0x7f1d8814d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:40.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d8de00640 1 -- 172.21.15.94:0/2852994318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d881fbb40 con 0x7f1d70004810 2024-09-10T23:35:40.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d8de00640 1 --2- 172.21.15.94:0/2852994318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d70004810 0x7f1d881f9150 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00fda0 tx=0x7f1d7c00d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:40.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d82000640 1 -- 172.21.15.94:0/2852994318 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c005220 con 0x7f1d70004810 2024-09-10T23:35:40.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d82000640 1 -- 172.21.15.94:0/2852994318 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d7c0054b0 con 0x7f1d70004810 2024-09-10T23:35:40.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d82000640 1 -- 172.21.15.94:0/2852994318 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c004a10 con 0x7f1d70004810 2024-09-10T23:35:40.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d90800640 1 -- 172.21.15.94:0/2852994318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d881fbe20 con 0x7f1d70004810 2024-09-10T23:35:40.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d90800640 1 -- 172.21.15.94:0/2852994318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d881fc3e0 con 0x7f1d70004810 2024-09-10T23:35:40.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.582+0000 7f1d90800640 1 -- 172.21.15.94:0/2852994318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d880b35c0 con 0x7f1d70004810 2024-09-10T23:35:40.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.586+0000 7f1d82000640 1 -- 172.21.15.94:0/2852994318 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f1d7c00b040 con 0x7f1d70004810 2024-09-10T23:35:40.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.586+0000 7f1d82000640 1 --2- 172.21.15.94:0/2852994318 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1d6c072790 0x7f1d6c074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:40.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.586+0000 7f1d82000640 1 -- 172.21.15.94:0/2852994318 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f1d7c0941a0 con 0x7f1d70004810 2024-09-10T23:35:40.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.586+0000 7f1d8d400640 1 --2- 172.21.15.94:0/2852994318 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1d6c072790 0x7f1d6c074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:40.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.586+0000 7f1d8d400640 1 --2- 172.21.15.94:0/2852994318 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1d6c072790 0x7f1d6c074c50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1d780ae000 tx=0x7f1d70005840 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:40.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.590+0000 7f1d82000640 1 -- 172.21.15.94:0/2852994318 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1d7c05f450 con 0x7f1d70004810 2024-09-10T23:35:40.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.786+0000 7f1d90800640 1 -- 172.21.15.94:0/2852994318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d8806e7e0 con 0x7f1d70004810 2024-09-10T23:35:40.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.790+0000 7f1d82000640 1 -- 172.21.15.94:0/2852994318 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5243 (secure 0 0 0) 0x7f1d7c0056a0 con 0x7f1d70004810 2024-09-10T23:35:40.790 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:40.790 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":34,"btime":"2024-09-10T23:35:40:410237+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5814,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3613711739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3613711739},{"type":"v1","addr":"172.21.15.94:6834","nonce":3613711739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":195107,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1388563408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1388563408},{"type":"v1","addr":"172.21.15.181:6836","nonce":1388563408}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":195110,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1882902083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1882902083},{"type":"v1","addr":"172.21.15.181:6837","nonce":1882902083}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:35:36.778166+0000","modified":"2024-09-10T23:35:39.784176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195113":{"gid":195113,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/2905667261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2905667261},{"type":"v1","addr":"172.21.15.94:6835","nonce":2905667261}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195113,"qdb_cluster":[195113]},"id":4}]} 2024-09-10T23:35:40.790 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 34 2024-09-10T23:35:40.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.790+0000 7f1d90800640 1 -- 172.21.15.94:0/2852994318 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1d6c072790 msgr2=0x7f1d6c074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:40.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.790+0000 7f1d90800640 1 --2- 172.21.15.94:0/2852994318 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1d6c072790 0x7f1d6c074c50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1d780ae000 tx=0x7f1d70005840 comp rx=0 tx=0).stop 2024-09-10T23:35:40.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.790+0000 7f1d90800640 1 -- 172.21.15.94:0/2852994318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d70004810 msgr2=0x7f1d881f9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.790+0000 7f1d90800640 1 --2- 172.21.15.94:0/2852994318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d70004810 0x7f1d881f9150 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00fda0 tx=0x7f1d7c00d7b0 comp rx=0 tx=0).stop 2024-09-10T23:35:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.790+0000 7f1d90800640 1 -- 172.21.15.94:0/2852994318 shutdown_connections 2024-09-10T23:35:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.790+0000 7f1d90800640 1 --2- 172.21.15.94:0/2852994318 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1d6c072790 0x7f1d6c074c50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.790+0000 7f1d90800640 1 --2- 172.21.15.94:0/2852994318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d88159d00 0x7f1d8814d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.790+0000 7f1d90800640 1 --2- 172.21.15.94:0/2852994318 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d881fb580 0x7f1d88152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.790+0000 7f1d90800640 1 --2- 172.21.15.94:0/2852994318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d70004810 0x7f1d881f9150 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.790+0000 7f1d90800640 1 -- 172.21.15.94:0/2852994318 >> 172.21.15.94:0/2852994318 conn(0x7f1d8806cdf0 msgr2=0x7f1d88072c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.790+0000 7f1d90800640 1 -- 172.21.15.94:0/2852994318 shutdown_connections 2024-09-10T23:35:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.790+0000 7f1d90800640 1 -- 172.21.15.94:0/2852994318 wait complete. 2024-09-10T23:35:40.812 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:35:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.950+0000 7f6d4a000640 1 -- 172.21.15.94:0/1907053703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d28003ae0 msgr2=0x7f6d28003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.950+0000 7f6d4a000640 1 --2- 172.21.15.94:0/1907053703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d28003ae0 0x7f6d28003f00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6d4406a310 tx=0x7f6d3401cc60 comp rx=0 tx=0).stop 2024-09-10T23:35:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.950+0000 7f6d43e00640 1 -- 172.21.15.94:0/1907053703 reap_dead start 2024-09-10T23:35:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.950+0000 7f6d4a000640 1 -- 172.21.15.94:0/1907053703 shutdown_connections 2024-09-10T23:35:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.950+0000 7f6d4a000640 1 -- 172.21.15.94:0/1907053703 >> 172.21.15.94:0/1907053703 conn(0x7f6d440d1c60 msgr2=0x7f6d440d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:40.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.950+0000 7f6d4a000640 1 -- 172.21.15.94:0/1907053703 shutdown_connections 2024-09-10T23:35:40.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.950+0000 7f6d4a000640 1 -- 172.21.15.94:0/1907053703 wait complete. 2024-09-10T23:35:40.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d4a000640 1 Processor -- start 2024-09-10T23:35:40.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d4a000640 1 -- start start 2024-09-10T23:35:40.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d4a000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d28003ae0 0x7f6d440b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:40.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d4a000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d441fb2f0 0x7f6d441fb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:40.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d4a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d440c7580 0x7f6d440c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:40.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d4a000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44167ef0 con 0x7f6d440c7580 2024-09-10T23:35:40.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d4a000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44167d70 con 0x7f6d28003ae0 2024-09-10T23:35:40.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d4a000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6d44168070 con 0x7f6d441fb2f0 2024-09-10T23:35:40.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d43e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d440c7580 0x7f6d440c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:40.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d43e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d440c7580 0x7f6d440c7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42536/0 (socket says 172.21.15.94:42536) 2024-09-10T23:35:40.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d43e00640 1 -- 172.21.15.94:0/402300056 learned_addr learned my addr 172.21.15.94:0/402300056 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:40.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d42a00640 1 --2- 172.21.15.94:0/402300056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d441fb2f0 0x7f6d441fb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:40.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d43400640 1 --2- 172.21.15.94:0/402300056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d28003ae0 0x7f6d440b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:40.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d43e00640 1 -- 172.21.15.94:0/402300056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d441fb2f0 msgr2=0x7f6d441fb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:40.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d43e00640 1 --2- 172.21.15.94:0/402300056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d441fb2f0 0x7f6d441fb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:40.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d43e00640 1 -- 172.21.15.94:0/402300056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d28003ae0 msgr2=0x7f6d440b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:40.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d43e00640 1 --2- 172.21.15.94:0/402300056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d28003ae0 0x7f6d440b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:40.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d43e00640 1 -- 172.21.15.94:0/402300056 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d440c67e0 con 0x7f6d440c7580 2024-09-10T23:35:40.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d43e00640 1 --2- 172.21.15.94:0/402300056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d440c7580 0x7f6d440c7a00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6d4409fc50 tx=0x7f6d3c006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:40.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d3b400640 1 -- 172.21.15.94:0/402300056 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c00fa80 con 0x7f6d440c7580 2024-09-10T23:35:40.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d3b400640 1 -- 172.21.15.94:0/402300056 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d3c00fc20 con 0x7f6d440c7580 2024-09-10T23:35:40.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d4a000640 1 -- 172.21.15.94:0/402300056 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4406fc40 con 0x7f6d440c7580 2024-09-10T23:35:40.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d3b400640 1 -- 172.21.15.94:0/402300056 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c011430 con 0x7f6d440c7580 2024-09-10T23:35:40.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.954+0000 7f6d4a000640 1 -- 172.21.15.94:0/402300056 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d440c6d80 con 0x7f6d440c7580 2024-09-10T23:35:40.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.958+0000 7f6d4a000640 1 -- 172.21.15.94:0/402300056 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d440af7c0 con 0x7f6d440c7580 2024-09-10T23:35:40.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.958+0000 7f6d3b400640 1 -- 172.21.15.94:0/402300056 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f6d3c0115d0 con 0x7f6d440c7580 2024-09-10T23:35:40.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.958+0000 7f6d3b400640 1 --2- 172.21.15.94:0/402300056 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6d240727c0 0x7f6d24074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:40.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.958+0000 7f6d3b400640 1 -- 172.21.15.94:0/402300056 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f6d3c093e90 con 0x7f6d440c7580 2024-09-10T23:35:40.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.958+0000 7f6d43400640 1 --2- 172.21.15.94:0/402300056 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6d240727c0 0x7f6d24074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:40.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.958+0000 7f6d43400640 1 --2- 172.21.15.94:0/402300056 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6d240727c0 0x7f6d24074c80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6d340b0000 tx=0x7f6d340a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:40.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:40.962+0000 7f6d3b400640 1 -- 172.21.15.94:0/402300056 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6d3c05f1c0 con 0x7f6d440c7580 2024-09-10T23:35:41.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.130+0000 7f6d4a000640 1 -- 172.21.15.94:0/402300056 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d440749c0 con 0x7f6d440c7580 2024-09-10T23:35:41.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.130+0000 7f6d3b400640 1 -- 172.21.15.94:0/402300056 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16713 (secure 0 0 0) 0x7f6d3c095030 con 0x7f6d440c7580 2024-09-10T23:35:41.131 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:41.132 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":58,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:35:38.384032+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:35:34.484637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T23:35:35.740496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:35:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.130+0000 7f6d4a000640 1 -- 172.21.15.94:0/402300056 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6d240727c0 msgr2=0x7f6d24074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.130+0000 7f6d4a000640 1 --2- 172.21.15.94:0/402300056 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6d240727c0 0x7f6d24074c80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6d340b0000 tx=0x7f6d340a7040 comp rx=0 tx=0).stop 2024-09-10T23:35:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.130+0000 7f6d4a000640 1 -- 172.21.15.94:0/402300056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d440c7580 msgr2=0x7f6d440c7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.130+0000 7f6d4a000640 1 --2- 172.21.15.94:0/402300056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d440c7580 0x7f6d440c7a00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6d4409fc50 tx=0x7f6d3c006660 comp rx=0 tx=0).stop 2024-09-10T23:35:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.130+0000 7f6d4a000640 1 -- 172.21.15.94:0/402300056 shutdown_connections 2024-09-10T23:35:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.130+0000 7f6d4a000640 1 --2- 172.21.15.94:0/402300056 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6d240727c0 0x7f6d24074c80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.130+0000 7f6d4a000640 1 --2- 172.21.15.94:0/402300056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d440c7580 0x7f6d440c7a00 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.130+0000 7f6d4a000640 1 --2- 172.21.15.94:0/402300056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d441fb2f0 0x7f6d441fb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.130+0000 7f6d4a000640 1 --2- 172.21.15.94:0/402300056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d28003ae0 0x7f6d440b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.130+0000 7f6d4a000640 1 -- 172.21.15.94:0/402300056 >> 172.21.15.94:0/402300056 conn(0x7f6d440d1c60 msgr2=0x7f6d4415a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.130+0000 7f6d4a000640 1 -- 172.21.15.94:0/402300056 shutdown_connections 2024-09-10T23:35:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.130+0000 7f6d4a000640 1 -- 172.21.15.94:0/402300056 wait complete. 2024-09-10T23:35:41.154 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T23:35:41.154 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:35:41.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f648c000640 1 -- 172.21.15.94:0/1268458053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64840cbbe0 msgr2=0x7f64840cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:41.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f648c000640 1 --2- 172.21.15.94:0/1268458053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64840cbbe0 0x7f64840cfe90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f647800ad90 tx=0x7f647801dc80 comp rx=0 tx=0).stop 2024-09-10T23:35:41.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f648c000640 1 -- 172.21.15.94:0/1268458053 shutdown_connections 2024-09-10T23:35:41.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f648c000640 1 --2- 172.21.15.94:0/1268458053 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64840cbfc0 0x7f64840d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f648c000640 1 --2- 172.21.15.94:0/1268458053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64840cbbe0 0x7f64840cfe90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f648c000640 1 -- 172.21.15.94:0/1268458053 >> 172.21.15.94:0/1268458053 conn(0x7f648406cdf0 msgr2=0x7f648406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:41.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f648c000640 1 -- 172.21.15.94:0/1268458053 shutdown_connections 2024-09-10T23:35:41.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f648c000640 1 -- 172.21.15.94:0/1268458053 wait complete. 2024-09-10T23:35:41.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f648c000640 1 Processor -- start 2024-09-10T23:35:41.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f648c000640 1 -- start start 2024-09-10T23:35:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f648c000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64840cbbe0 0x7f648413b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f648c000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64840cbfc0 0x7f64840b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f648c000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f648415db80 0x7f64840aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f648c000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f648413f710 con 0x7f64840cbfc0 2024-09-10T23:35:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f648c000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f648413f590 con 0x7f64840cbbe0 2024-09-10T23:35:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f648c000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f648413f890 con 0x7f648415db80 2024-09-10T23:35:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f6488c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64840cbfc0 0x7f64840b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f6488c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64840cbfc0 0x7f64840b0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42564/0 (socket says 172.21.15.94:42564) 2024-09-10T23:35:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.294+0000 7f6488c00640 1 -- 172.21.15.94:0/632042235 learned_addr learned my addr 172.21.15.94:0/632042235 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.298+0000 7f6489600640 1 --2- 172.21.15.94:0/632042235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64840cbbe0 0x7f648413b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:41.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.298+0000 7f648a000640 1 --2- 172.21.15.94:0/632042235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f648415db80 0x7f64840aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:41.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.298+0000 7f6488c00640 1 -- 172.21.15.94:0/632042235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f648415db80 msgr2=0x7f64840aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:41.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.298+0000 7f6488c00640 1 --2- 172.21.15.94:0/632042235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f648415db80 0x7f64840aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.298+0000 7f6488c00640 1 -- 172.21.15.94:0/632042235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64840cbbe0 msgr2=0x7f648413b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:41.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.298+0000 7f6488c00640 1 --2- 172.21.15.94:0/632042235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64840cbbe0 0x7f648413b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.298+0000 7f6488c00640 1 -- 172.21.15.94:0/632042235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64840ada80 con 0x7f64840cbfc0 2024-09-10T23:35:41.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.298+0000 7f6488c00640 1 --2- 172.21.15.94:0/632042235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64840cbfc0 0x7f64840b0380 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f647000d980 tx=0x7f647000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:41.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.298+0000 7f6481600640 1 -- 172.21.15.94:0/632042235 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470017070 con 0x7f64840cbfc0 2024-09-10T23:35:41.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.298+0000 7f6481600640 1 -- 172.21.15.94:0/632042235 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6470009d20 con 0x7f64840cbfc0 2024-09-10T23:35:41.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.298+0000 7f6481600640 1 -- 172.21.15.94:0/632042235 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470004280 con 0x7f64840cbfc0 2024-09-10T23:35:41.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.298+0000 7f648c000640 1 -- 172.21.15.94:0/632042235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64840add60 con 0x7f64840cbfc0 2024-09-10T23:35:41.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.298+0000 7f648c000640 1 -- 172.21.15.94:0/632042235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64840ae320 con 0x7f64840cbfc0 2024-09-10T23:35:41.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.298+0000 7f648c000640 1 -- 172.21.15.94:0/632042235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f648413fe40 con 0x7f64840cbfc0 2024-09-10T23:35:41.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.298+0000 7f6481600640 1 -- 172.21.15.94:0/632042235 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f6470003600 con 0x7f64840cbfc0 2024-09-10T23:35:41.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.302+0000 7f6481600640 1 --2- 172.21.15.94:0/632042235 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6464072790 0x7f6464074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:41.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.302+0000 7f6481600640 1 -- 172.21.15.94:0/632042235 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f6470093670 con 0x7f64840cbfc0 2024-09-10T23:35:41.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.302+0000 7f6489600640 1 --2- 172.21.15.94:0/632042235 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6464072790 0x7f6464074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:41.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.302+0000 7f6489600640 1 --2- 172.21.15.94:0/632042235 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6464072790 0x7f6464074c50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f64780b0000 tx=0x7f647801c4c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:41.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.306+0000 7f6481600640 1 -- 172.21.15.94:0/632042235 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f647005e410 con 0x7f64840cbfc0 2024-09-10T23:35:41.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.470+0000 7f648c000640 1 -- 172.21.15.94:0/632042235 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f64840b2c80 con 0x7f6464072790 2024-09-10T23:35:41.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.474+0000 7f6481600640 1 -- 172.21.15.94:0/632042235 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247559 (secure 0 0 0) 0x7f64840b2c80 con 0x7f6464072790 2024-09-10T23:35:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:41.481 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:35:41.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.478+0000 7f648c000640 1 -- 172.21.15.94:0/632042235 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6464072790 msgr2=0x7f6464074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:41.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.478+0000 7f648c000640 1 --2- 172.21.15.94:0/632042235 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6464072790 0x7f6464074c50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f64780b0000 tx=0x7f647801c4c0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.478+0000 7f648c000640 1 -- 172.21.15.94:0/632042235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64840cbfc0 msgr2=0x7f64840b0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:41.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.478+0000 7f648c000640 1 --2- 172.21.15.94:0/632042235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64840cbfc0 0x7f64840b0380 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f647000d980 tx=0x7f647000de40 comp rx=0 tx=0).stop 2024-09-10T23:35:41.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.478+0000 7f648c000640 1 -- 172.21.15.94:0/632042235 shutdown_connections 2024-09-10T23:35:41.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.478+0000 7f648c000640 1 --2- 172.21.15.94:0/632042235 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6464072790 0x7f6464074c50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.478+0000 7f648c000640 1 --2- 172.21.15.94:0/632042235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f648415db80 0x7f64840aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.478+0000 7f648c000640 1 --2- 172.21.15.94:0/632042235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64840cbfc0 0x7f64840b0380 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.478+0000 7f648c000640 1 --2- 172.21.15.94:0/632042235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64840cbbe0 0x7f648413b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.478+0000 7f648c000640 1 -- 172.21.15.94:0/632042235 >> 172.21.15.94:0/632042235 conn(0x7f648406cdf0 msgr2=0x7f648415c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:41.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.478+0000 7f648c000640 1 -- 172.21.15.94:0/632042235 shutdown_connections 2024-09-10T23:35:41.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.478+0000 7f648c000640 1 -- 172.21.15.94:0/632042235 wait complete. 2024-09-10T23:35:41.499 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-10T23:35:41.388360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247508,"kb_used_data":6312,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727212,"statfs":{"total":773094113280,"available":772840665088,"internally_reserved":0,"allocated":6463488,"data_stored":3329394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001510"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391197+0000","last_change":"2024-09-10T23:35:37.411838+0000","last_active":"2024-09-10T23:35:38.391197+0000","last_peered":"2024-09-10T23:35:38.391197+0000","last_clean":"2024-09-10T23:35:38.391197+0000","last_became_active":"2024-09-10T23:35:37.411168+0000","last_became_peered":"2024-09-10T23:35:37.411168+0000","last_unstale":"2024-09-10T23:35:38.391197+0000","last_undegraded":"2024-09-10T23:35:38.391197+0000","last_fullsized":"2024-09-10T23:35:38.391197+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:27.635555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388506+0000","last_change":"2024-09-10T23:35:36.388506+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388506+0000","last_undegraded":"2024-09-10T23:35:36.388506+0000","last_fullsized":"2024-09-10T23:35:36.388506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.390335+0000","last_change":"2024-09-10T23:35:36.390335+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.390335+0000","last_undegraded":"2024-09-10T23:35:36.390335+0000","last_fullsized":"2024-09-10T23:35:36.390335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392121+0000","last_change":"2024-09-10T23:35:36.392121+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392121+0000","last_undegraded":"2024-09-10T23:35:36.392121+0000","last_fullsized":"2024-09-10T23:35:36.392121+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390631+0000","last_change":"2024-09-10T23:35:37.422878+0000","last_active":"2024-09-10T23:35:38.390631+0000","last_peered":"2024-09-10T23:35:38.390631+0000","last_clean":"2024-09-10T23:35:38.390631+0000","last_became_active":"2024-09-10T23:35:37.422602+0000","last_became_peered":"2024-09-10T23:35:37.422602+0000","last_unstale":"2024-09-10T23:35:38.390631+0000","last_undegraded":"2024-09-10T23:35:38.390631+0000","last_fullsized":"2024-09-10T23:35:38.390631+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:17.965755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390045+0000","last_change":"2024-09-10T23:35:37.401854+0000","last_active":"2024-09-10T23:35:38.390045+0000","last_peered":"2024-09-10T23:35:38.390045+0000","last_clean":"2024-09-10T23:35:38.390045+0000","last_became_active":"2024-09-10T23:35:37.400521+0000","last_became_peered":"2024-09-10T23:35:37.400521+0000","last_unstale":"2024-09-10T23:35:38.390045+0000","last_undegraded":"2024-09-10T23:35:38.390045+0000","last_fullsized":"2024-09-10T23:35:38.390045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:48.537421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394892+0000","last_change":"2024-09-10T23:35:36.394892+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394892+0000","last_undegraded":"2024-09-10T23:35:36.394892+0000","last_fullsized":"2024-09-10T23:35:36.394892+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389814+0000","last_change":"2024-09-10T23:35:36.389814+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389814+0000","last_undegraded":"2024-09-10T23:35:36.389814+0000","last_fullsized":"2024-09-10T23:35:36.389814+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389006+0000","last_change":"2024-09-10T23:35:36.389006+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389006+0000","last_undegraded":"2024-09-10T23:35:36.389006+0000","last_fullsized":"2024-09-10T23:35:36.389006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390598+0000","last_change":"2024-09-10T23:35:37.405331+0000","last_active":"2024-09-10T23:35:38.390598+0000","last_peered":"2024-09-10T23:35:38.390598+0000","last_clean":"2024-09-10T23:35:38.390598+0000","last_became_active":"2024-09-10T23:35:37.404010+0000","last_became_peered":"2024-09-10T23:35:37.404010+0000","last_unstale":"2024-09-10T23:35:38.390598+0000","last_undegraded":"2024-09-10T23:35:38.390598+0000","last_fullsized":"2024-09-10T23:35:38.390598+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:30.787726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394567+0000","last_change":"2024-09-10T23:35:36.394567+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394567+0000","last_undegraded":"2024-09-10T23:35:36.394567+0000","last_fullsized":"2024-09-10T23:35:36.394567+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391006+0000","last_change":"2024-09-10T23:35:36.391006+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391006+0000","last_undegraded":"2024-09-10T23:35:36.391006+0000","last_fullsized":"2024-09-10T23:35:36.391006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.397258+0000","last_change":"2024-09-10T23:35:36.397258+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.397258+0000","last_undegraded":"2024-09-10T23:35:36.397258+0000","last_fullsized":"2024-09-10T23:35:36.397258+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389045+0000","last_change":"2024-09-10T23:35:36.389045+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389045+0000","last_undegraded":"2024-09-10T23:35:36.389045+0000","last_fullsized":"2024-09-10T23:35:36.389045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394449+0000","last_change":"2024-09-10T23:35:36.394449+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394449+0000","last_undegraded":"2024-09-10T23:35:36.394449+0000","last_fullsized":"2024-09-10T23:35:36.394449+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389559+0000","last_change":"2024-09-10T23:35:37.421193+0000","last_active":"2024-09-10T23:35:38.389559+0000","last_peered":"2024-09-10T23:35:38.389559+0000","last_clean":"2024-09-10T23:35:38.389559+0000","last_became_active":"2024-09-10T23:35:37.420909+0000","last_became_peered":"2024-09-10T23:35:37.420909+0000","last_unstale":"2024-09-10T23:35:38.389559+0000","last_undegraded":"2024-09-10T23:35:38.389559+0000","last_fullsized":"2024-09-10T23:35:38.389559+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:31.257242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391217+0000","last_change":"2024-09-10T23:35:37.404324+0000","last_active":"2024-09-10T23:35:38.391217+0000","last_peered":"2024-09-10T23:35:38.391217+0000","last_clean":"2024-09-10T23:35:38.391217+0000","last_became_active":"2024-09-10T23:35:37.404042+0000","last_became_peered":"2024-09-10T23:35:37.404042+0000","last_unstale":"2024-09-10T23:35:38.391217+0000","last_undegraded":"2024-09-10T23:35:38.391217+0000","last_fullsized":"2024-09-10T23:35:38.391217+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:31.250579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392062+0000","last_change":"2024-09-10T23:35:36.392062+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392062+0000","last_undegraded":"2024-09-10T23:35:36.392062+0000","last_fullsized":"2024-09-10T23:35:36.392062+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389708+0000","last_change":"2024-09-10T23:35:36.389708+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389708+0000","last_undegraded":"2024-09-10T23:35:36.389708+0000","last_fullsized":"2024-09-10T23:35:36.389708+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394443+0000","last_change":"2024-09-10T23:35:36.394443+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394443+0000","last_undegraded":"2024-09-10T23:35:36.394443+0000","last_fullsized":"2024-09-10T23:35:36.394443+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.396891+0000","last_change":"2024-09-10T23:35:36.396891+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.396891+0000","last_undegraded":"2024-09-10T23:35:36.396891+0000","last_fullsized":"2024-09-10T23:35:36.396891+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.396024+0000","last_change":"2024-09-10T23:35:36.396024+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.396024+0000","last_undegraded":"2024-09-10T23:35:36.396024+0000","last_fullsized":"2024-09-10T23:35:36.396024+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392968+0000","last_change":"2024-09-10T23:35:36.392968+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392968+0000","last_undegraded":"2024-09-10T23:35:36.392968+0000","last_fullsized":"2024-09-10T23:35:36.392968+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.390266+0000","last_change":"2024-09-10T23:35:36.390266+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.390266+0000","last_undegraded":"2024-09-10T23:35:36.390266+0000","last_fullsized":"2024-09-10T23:35:36.390266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389015+0000","last_change":"2024-09-10T23:35:36.389015+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389015+0000","last_undegraded":"2024-09-10T23:35:36.389015+0000","last_fullsized":"2024-09-10T23:35:36.389015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.395481+0000","last_change":"2024-09-10T23:35:36.395481+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.395481+0000","last_undegraded":"2024-09-10T23:35:36.395481+0000","last_fullsized":"2024-09-10T23:35:36.395481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389120+0000","last_change":"2024-09-10T23:35:36.389120+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389120+0000","last_undegraded":"2024-09-10T23:35:36.389120+0000","last_fullsized":"2024-09-10T23:35:36.389120+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392340+0000","last_change":"2024-09-10T23:35:36.392340+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392340+0000","last_undegraded":"2024-09-10T23:35:36.392340+0000","last_fullsized":"2024-09-10T23:35:36.392340+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388535+0000","last_change":"2024-09-10T23:35:36.388535+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388535+0000","last_undegraded":"2024-09-10T23:35:36.388535+0000","last_fullsized":"2024-09-10T23:35:36.388535+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391729+0000","last_change":"2024-09-10T23:35:36.391729+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391729+0000","last_undegraded":"2024-09-10T23:35:36.391729+0000","last_fullsized":"2024-09-10T23:35:36.391729+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388251+0000","last_change":"2024-09-10T23:35:36.388251+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388251+0000","last_undegraded":"2024-09-10T23:35:36.388251+0000","last_fullsized":"2024-09-10T23:35:36.388251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394059+0000","last_change":"2024-09-10T23:35:36.394059+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394059+0000","last_undegraded":"2024-09-10T23:35:36.394059+0000","last_fullsized":"2024-09-10T23:35:36.394059+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391188+0000","last_change":"2024-09-10T23:35:37.405530+0000","last_active":"2024-09-10T23:35:38.391188+0000","last_peered":"2024-09-10T23:35:38.391188+0000","last_clean":"2024-09-10T23:35:38.391188+0000","last_became_active":"2024-09-10T23:35:37.402795+0000","last_became_peered":"2024-09-10T23:35:37.402795+0000","last_unstale":"2024-09-10T23:35:38.391188+0000","last_undegraded":"2024-09-10T23:35:38.391188+0000","last_fullsized":"2024-09-10T23:35:38.391188+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:15.871163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.404962+0000","last_change":"2024-09-10T23:35:36.401576+0000","last_active":"2024-09-10T23:35:36.404962+0000","last_peered":"2024-09-10T23:35:36.404962+0000","last_clean":"2024-09-10T23:35:36.404962+0000","last_became_active":"2024-09-10T23:35:36.400780+0000","last_became_peered":"2024-09-10T23:35:36.400780+0000","last_unstale":"2024-09-10T23:35:36.404962+0000","last_undegraded":"2024-09-10T23:35:36.404962+0000","last_fullsized":"2024-09-10T23:35:36.404962+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:15.011008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391401+0000","last_change":"2024-09-10T23:35:37.411543+0000","last_active":"2024-09-10T23:35:38.391401+0000","last_peered":"2024-09-10T23:35:38.391401+0000","last_clean":"2024-09-10T23:35:38.391401+0000","last_became_active":"2024-09-10T23:35:37.410807+0000","last_became_peered":"2024-09-10T23:35:37.410807+0000","last_unstale":"2024-09-10T23:35:38.391401+0000","last_undegraded":"2024-09-10T23:35:38.391401+0000","last_fullsized":"2024-09-10T23:35:38.391401+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:52.749748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.410357+0000","last_change":"2024-09-10T23:35:36.406745+0000","last_active":"2024-09-10T23:35:36.410357+0000","last_peered":"2024-09-10T23:35:36.410357+0000","last_clean":"2024-09-10T23:35:36.410357+0000","last_became_active":"2024-09-10T23:35:36.406460+0000","last_became_peered":"2024-09-10T23:35:36.406460+0000","last_unstale":"2024-09-10T23:35:36.410357+0000","last_undegraded":"2024-09-10T23:35:36.410357+0000","last_fullsized":"2024-09-10T23:35:36.410357+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:58.569452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391940+0000","last_change":"2024-09-10T23:35:36.391940+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391940+0000","last_undegraded":"2024-09-10T23:35:36.391940+0000","last_fullsized":"2024-09-10T23:35:36.391940+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390807+0000","last_change":"2024-09-10T23:35:36.409095+0000","last_active":"2024-09-10T23:35:38.390807+0000","last_peered":"2024-09-10T23:35:38.390807+0000","last_clean":"2024-09-10T23:35:38.390807+0000","last_became_active":"2024-09-10T23:35:36.408811+0000","last_became_peered":"2024-09-10T23:35:36.408811+0000","last_unstale":"2024-09-10T23:35:38.390807+0000","last_undegraded":"2024-09-10T23:35:38.390807+0000","last_fullsized":"2024-09-10T23:35:38.390807+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:47.477496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390766+0000","last_change":"2024-09-10T23:35:37.413035+0000","last_active":"2024-09-10T23:35:38.390766+0000","last_peered":"2024-09-10T23:35:38.390766+0000","last_clean":"2024-09-10T23:35:38.390766+0000","last_became_active":"2024-09-10T23:35:37.412781+0000","last_became_peered":"2024-09-10T23:35:37.412781+0000","last_unstale":"2024-09-10T23:35:38.390766+0000","last_undegraded":"2024-09-10T23:35:38.390766+0000","last_fullsized":"2024-09-10T23:35:38.390766+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:11.135478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.404617+0000","last_change":"2024-09-10T23:35:36.396384+0000","last_active":"2024-09-10T23:35:36.404617+0000","last_peered":"2024-09-10T23:35:36.404617+0000","last_clean":"2024-09-10T23:35:36.404617+0000","last_became_active":"2024-09-10T23:35:36.396085+0000","last_became_peered":"2024-09-10T23:35:36.396085+0000","last_unstale":"2024-09-10T23:35:36.404617+0000","last_undegraded":"2024-09-10T23:35:36.404617+0000","last_fullsized":"2024-09-10T23:35:36.404617+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:34.607437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.387665+0000","last_change":"2024-09-10T23:35:36.387665+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.387665+0000","last_undegraded":"2024-09-10T23:35:36.387665+0000","last_fullsized":"2024-09-10T23:35:36.387665+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407850+0000","last_change":"2024-09-10T23:35:36.404943+0000","last_active":"2024-09-10T23:35:36.407850+0000","last_peered":"2024-09-10T23:35:36.407850+0000","last_clean":"2024-09-10T23:35:36.407850+0000","last_became_active":"2024-09-10T23:35:36.404615+0000","last_became_peered":"2024-09-10T23:35:36.404615+0000","last_unstale":"2024-09-10T23:35:36.407850+0000","last_undegraded":"2024-09-10T23:35:36.407850+0000","last_fullsized":"2024-09-10T23:35:36.407850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:58.666819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.396962+0000","last_change":"2024-09-10T23:35:36.396962+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.396962+0000","last_undegraded":"2024-09-10T23:35:36.396962+0000","last_fullsized":"2024-09-10T23:35:36.396962+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390698+0000","last_change":"2024-09-10T23:35:36.408437+0000","last_active":"2024-09-10T23:35:38.390698+0000","last_peered":"2024-09-10T23:35:38.390698+0000","last_clean":"2024-09-10T23:35:38.390698+0000","last_became_active":"2024-09-10T23:35:36.408156+0000","last_became_peered":"2024-09-10T23:35:36.408156+0000","last_unstale":"2024-09-10T23:35:38.390698+0000","last_undegraded":"2024-09-10T23:35:38.390698+0000","last_fullsized":"2024-09-10T23:35:38.390698+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:01.998360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389936+0000","last_change":"2024-09-10T23:35:36.389936+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389936+0000","last_undegraded":"2024-09-10T23:35:36.389936+0000","last_fullsized":"2024-09-10T23:35:36.389936+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407639+0000","last_change":"2024-09-10T23:35:36.401028+0000","last_active":"2024-09-10T23:35:36.407639+0000","last_peered":"2024-09-10T23:35:36.407639+0000","last_clean":"2024-09-10T23:35:36.407639+0000","last_became_active":"2024-09-10T23:35:36.400743+0000","last_became_peered":"2024-09-10T23:35:36.400743+0000","last_unstale":"2024-09-10T23:35:36.407639+0000","last_undegraded":"2024-09-10T23:35:36.407639+0000","last_fullsized":"2024-09-10T23:35:36.407639+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:39:33.252954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389654+0000","last_change":"2024-09-10T23:35:37.422945+0000","last_active":"2024-09-10T23:35:38.389654+0000","last_peered":"2024-09-10T23:35:38.389654+0000","last_clean":"2024-09-10T23:35:38.389654+0000","last_became_active":"2024-09-10T23:35:37.422648+0000","last_became_peered":"2024-09-10T23:35:37.422648+0000","last_unstale":"2024-09-10T23:35:38.389654+0000","last_undegraded":"2024-09-10T23:35:38.389654+0000","last_fullsized":"2024-09-10T23:35:38.389654+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:10.705976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406280+0000","last_change":"2024-09-10T23:35:36.401434+0000","last_active":"2024-09-10T23:35:36.406280+0000","last_peered":"2024-09-10T23:35:36.406280+0000","last_clean":"2024-09-10T23:35:36.406280+0000","last_became_active":"2024-09-10T23:35:36.401155+0000","last_became_peered":"2024-09-10T23:35:36.401155+0000","last_unstale":"2024-09-10T23:35:36.406280+0000","last_undegraded":"2024-09-10T23:35:36.406280+0000","last_fullsized":"2024-09-10T23:35:36.406280+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:37.536960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394992+0000","last_change":"2024-09-10T23:35:36.394992+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394992+0000","last_undegraded":"2024-09-10T23:35:36.394992+0000","last_fullsized":"2024-09-10T23:35:36.394992+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407810+0000","last_change":"2024-09-10T23:35:36.405013+0000","last_active":"2024-09-10T23:35:36.407810+0000","last_peered":"2024-09-10T23:35:36.407810+0000","last_clean":"2024-09-10T23:35:36.407810+0000","last_became_active":"2024-09-10T23:35:36.404715+0000","last_became_peered":"2024-09-10T23:35:36.404715+0000","last_unstale":"2024-09-10T23:35:36.407810+0000","last_undegraded":"2024-09-10T23:35:36.407810+0000","last_fullsized":"2024-09-10T23:35:36.407810+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:11:46.791009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394500+0000","last_change":"2024-09-10T23:35:36.394500+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394500+0000","last_undegraded":"2024-09-10T23:35:36.394500+0000","last_fullsized":"2024-09-10T23:35:36.394500+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407797+0000","last_change":"2024-09-10T23:35:36.402529+0000","last_active":"2024-09-10T23:35:36.407797+0000","last_peered":"2024-09-10T23:35:36.407797+0000","last_clean":"2024-09-10T23:35:36.407797+0000","last_became_active":"2024-09-10T23:35:36.402270+0000","last_became_peered":"2024-09-10T23:35:36.402270+0000","last_unstale":"2024-09-10T23:35:36.407797+0000","last_undegraded":"2024-09-10T23:35:36.407797+0000","last_fullsized":"2024-09-10T23:35:36.407797+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:28.809096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389483+0000","last_change":"2024-09-10T23:35:36.389483+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389483+0000","last_undegraded":"2024-09-10T23:35:36.389483+0000","last_fullsized":"2024-09-10T23:35:36.389483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406578+0000","last_change":"2024-09-10T23:35:36.402631+0000","last_active":"2024-09-10T23:35:36.406578+0000","last_peered":"2024-09-10T23:35:36.406578+0000","last_clean":"2024-09-10T23:35:36.406578+0000","last_became_active":"2024-09-10T23:35:36.399947+0000","last_became_peered":"2024-09-10T23:35:36.399947+0000","last_unstale":"2024-09-10T23:35:36.406578+0000","last_undegraded":"2024-09-10T23:35:36.406578+0000","last_fullsized":"2024-09-10T23:35:36.406578+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:53.498796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391900+0000","last_change":"2024-09-10T23:35:36.391900+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391900+0000","last_undegraded":"2024-09-10T23:35:36.391900+0000","last_fullsized":"2024-09-10T23:35:36.391900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.408350+0000","last_change":"2024-09-10T23:35:36.405711+0000","last_active":"2024-09-10T23:35:36.408350+0000","last_peered":"2024-09-10T23:35:36.408350+0000","last_clean":"2024-09-10T23:35:36.408350+0000","last_became_active":"2024-09-10T23:35:36.405455+0000","last_became_peered":"2024-09-10T23:35:36.405455+0000","last_unstale":"2024-09-10T23:35:36.408350+0000","last_undegraded":"2024-09-10T23:35:36.408350+0000","last_fullsized":"2024-09-10T23:35:36.408350+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:56.422393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392350+0000","last_change":"2024-09-10T23:35:36.392350+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392350+0000","last_undegraded":"2024-09-10T23:35:36.392350+0000","last_fullsized":"2024-09-10T23:35:36.392350+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407863+0000","last_change":"2024-09-10T23:35:36.404848+0000","last_active":"2024-09-10T23:35:36.407863+0000","last_peered":"2024-09-10T23:35:36.407863+0000","last_clean":"2024-09-10T23:35:36.407863+0000","last_became_active":"2024-09-10T23:35:36.404567+0000","last_became_peered":"2024-09-10T23:35:36.404567+0000","last_unstale":"2024-09-10T23:35:36.407863+0000","last_undegraded":"2024-09-10T23:35:36.407863+0000","last_fullsized":"2024-09-10T23:35:36.407863+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:33.274868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388987+0000","last_change":"2024-09-10T23:35:36.388987+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388987+0000","last_undegraded":"2024-09-10T23:35:36.388987+0000","last_fullsized":"2024-09-10T23:35:36.388987+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407454+0000","last_change":"2024-09-10T23:35:36.402546+0000","last_active":"2024-09-10T23:35:36.407454+0000","last_peered":"2024-09-10T23:35:36.407454+0000","last_clean":"2024-09-10T23:35:36.407454+0000","last_became_active":"2024-09-10T23:35:36.402187+0000","last_became_peered":"2024-09-10T23:35:36.402187+0000","last_unstale":"2024-09-10T23:35:36.407454+0000","last_undegraded":"2024-09-10T23:35:36.407454+0000","last_fullsized":"2024-09-10T23:35:36.407454+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:18.611164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389852+0000","last_change":"2024-09-10T23:35:36.389852+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389852+0000","last_undegraded":"2024-09-10T23:35:36.389852+0000","last_fullsized":"2024-09-10T23:35:36.389852+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406227+0000","last_change":"2024-09-10T23:35:36.401288+0000","last_active":"2024-09-10T23:35:36.406227+0000","last_peered":"2024-09-10T23:35:36.406227+0000","last_clean":"2024-09-10T23:35:36.406227+0000","last_became_active":"2024-09-10T23:35:36.400889+0000","last_became_peered":"2024-09-10T23:35:36.400889+0000","last_unstale":"2024-09-10T23:35:36.406227+0000","last_undegraded":"2024-09-10T23:35:36.406227+0000","last_fullsized":"2024-09-10T23:35:36.406227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:05.132993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.390490+0000","last_change":"2024-09-10T23:35:36.390490+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.390490+0000","last_undegraded":"2024-09-10T23:35:36.390490+0000","last_fullsized":"2024-09-10T23:35:36.390490+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406140+0000","last_change":"2024-09-10T23:35:36.398356+0000","last_active":"2024-09-10T23:35:36.406140+0000","last_peered":"2024-09-10T23:35:36.406140+0000","last_clean":"2024-09-10T23:35:36.406140+0000","last_became_active":"2024-09-10T23:35:36.398077+0000","last_became_peered":"2024-09-10T23:35:36.398077+0000","last_unstale":"2024-09-10T23:35:36.406140+0000","last_undegraded":"2024-09-10T23:35:36.406140+0000","last_fullsized":"2024-09-10T23:35:36.406140+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:11.718995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391693+0000","last_change":"2024-09-10T23:35:36.391693+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391693+0000","last_undegraded":"2024-09-10T23:35:36.391693+0000","last_fullsized":"2024-09-10T23:35:36.391693+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.411637+0000","last_change":"2024-09-10T23:35:36.409579+0000","last_active":"2024-09-10T23:35:36.411637+0000","last_peered":"2024-09-10T23:35:36.411637+0000","last_clean":"2024-09-10T23:35:36.411637+0000","last_became_active":"2024-09-10T23:35:36.409321+0000","last_became_peered":"2024-09-10T23:35:36.409321+0000","last_unstale":"2024-09-10T23:35:36.411637+0000","last_undegraded":"2024-09-10T23:35:36.411637+0000","last_fullsized":"2024-09-10T23:35:36.411637+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:19.816523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394047+0000","last_change":"2024-09-10T23:35:36.394047+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394047+0000","last_undegraded":"2024-09-10T23:35:36.394047+0000","last_fullsized":"2024-09-10T23:35:36.394047+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405719+0000","last_change":"2024-09-10T23:35:36.400916+0000","last_active":"2024-09-10T23:35:36.405719+0000","last_peered":"2024-09-10T23:35:36.405719+0000","last_clean":"2024-09-10T23:35:36.405719+0000","last_became_active":"2024-09-10T23:35:36.400639+0000","last_became_peered":"2024-09-10T23:35:36.400639+0000","last_unstale":"2024-09-10T23:35:36.405719+0000","last_undegraded":"2024-09-10T23:35:36.405719+0000","last_fullsized":"2024-09-10T23:35:36.405719+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:47.327210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389265+0000","last_change":"2024-09-10T23:35:36.389265+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389265+0000","last_undegraded":"2024-09-10T23:35:36.389265+0000","last_fullsized":"2024-09-10T23:35:36.389265+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405406+0000","last_change":"2024-09-10T23:35:36.398766+0000","last_active":"2024-09-10T23:35:36.405406+0000","last_peered":"2024-09-10T23:35:36.405406+0000","last_clean":"2024-09-10T23:35:36.405406+0000","last_became_active":"2024-09-10T23:35:36.398473+0000","last_became_peered":"2024-09-10T23:35:36.398473+0000","last_unstale":"2024-09-10T23:35:36.405406+0000","last_undegraded":"2024-09-10T23:35:36.405406+0000","last_fullsized":"2024-09-10T23:35:36.405406+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:37.536960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388202+0000","last_change":"2024-09-10T23:35:36.388202+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388202+0000","last_undegraded":"2024-09-10T23:35:36.388202+0000","last_fullsized":"2024-09-10T23:35:36.388202+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.414301+0000","last_change":"2024-09-10T23:35:36.409772+0000","last_active":"2024-09-10T23:35:36.414301+0000","last_peered":"2024-09-10T23:35:36.414301+0000","last_clean":"2024-09-10T23:35:36.414301+0000","last_became_active":"2024-09-10T23:35:36.409432+0000","last_became_peered":"2024-09-10T23:35:36.409432+0000","last_unstale":"2024-09-10T23:35:36.414301+0000","last_undegraded":"2024-09-10T23:35:36.414301+0000","last_fullsized":"2024-09-10T23:35:36.414301+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:09.775738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391962+0000","last_change":"2024-09-10T23:35:36.391962+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391962+0000","last_undegraded":"2024-09-10T23:35:36.391962+0000","last_fullsized":"2024-09-10T23:35:36.391962+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.409418+0000","last_change":"2024-09-10T23:35:36.405162+0000","last_active":"2024-09-10T23:35:36.409418+0000","last_peered":"2024-09-10T23:35:36.409418+0000","last_clean":"2024-09-10T23:35:36.409418+0000","last_became_active":"2024-09-10T23:35:36.404891+0000","last_became_peered":"2024-09-10T23:35:36.404891+0000","last_unstale":"2024-09-10T23:35:36.409418+0000","last_undegraded":"2024-09-10T23:35:36.409418+0000","last_fullsized":"2024-09-10T23:35:36.409418+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:25.179787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388064+0000","last_change":"2024-09-10T23:35:36.388064+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388064+0000","last_undegraded":"2024-09-10T23:35:36.388064+0000","last_fullsized":"2024-09-10T23:35:36.388064+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406216+0000","last_change":"2024-09-10T23:35:36.401149+0000","last_active":"2024-09-10T23:35:36.406216+0000","last_peered":"2024-09-10T23:35:36.406216+0000","last_clean":"2024-09-10T23:35:36.406216+0000","last_became_active":"2024-09-10T23:35:36.400884+0000","last_became_peered":"2024-09-10T23:35:36.400884+0000","last_unstale":"2024-09-10T23:35:36.406216+0000","last_undegraded":"2024-09-10T23:35:36.406216+0000","last_fullsized":"2024-09-10T23:35:36.406216+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:43.580364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388430+0000","last_change":"2024-09-10T23:35:36.388430+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388430+0000","last_undegraded":"2024-09-10T23:35:36.388430+0000","last_fullsized":"2024-09-10T23:35:36.388430+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407985+0000","last_change":"2024-09-10T23:35:36.406290+0000","last_active":"2024-09-10T23:35:36.407985+0000","last_peered":"2024-09-10T23:35:36.407985+0000","last_clean":"2024-09-10T23:35:36.407985+0000","last_became_active":"2024-09-10T23:35:36.406056+0000","last_became_peered":"2024-09-10T23:35:36.406056+0000","last_unstale":"2024-09-10T23:35:36.407985+0000","last_undegraded":"2024-09-10T23:35:36.407985+0000","last_fullsized":"2024-09-10T23:35:36.407985+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:59.329841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391377+0000","last_change":"2024-09-10T23:35:36.391377+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391377+0000","last_undegraded":"2024-09-10T23:35:36.391377+0000","last_fullsized":"2024-09-10T23:35:36.391377+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.410432+0000","last_change":"2024-09-10T23:35:36.406893+0000","last_active":"2024-09-10T23:35:36.410432+0000","last_peered":"2024-09-10T23:35:36.410432+0000","last_clean":"2024-09-10T23:35:36.410432+0000","last_became_active":"2024-09-10T23:35:36.406516+0000","last_became_peered":"2024-09-10T23:35:36.406516+0000","last_unstale":"2024-09-10T23:35:36.410432+0000","last_undegraded":"2024-09-10T23:35:36.410432+0000","last_fullsized":"2024-09-10T23:35:36.410432+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:13.080452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.393643+0000","last_change":"2024-09-10T23:35:36.393643+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.393643+0000","last_undegraded":"2024-09-10T23:35:36.393643+0000","last_fullsized":"2024-09-10T23:35:36.393643+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391082+0000","last_change":"2024-09-10T23:35:36.409344+0000","last_active":"2024-09-10T23:35:38.391082+0000","last_peered":"2024-09-10T23:35:38.391082+0000","last_clean":"2024-09-10T23:35:38.391082+0000","last_became_active":"2024-09-10T23:35:36.408975+0000","last_became_peered":"2024-09-10T23:35:36.408975+0000","last_unstale":"2024-09-10T23:35:38.391082+0000","last_undegraded":"2024-09-10T23:35:38.391082+0000","last_fullsized":"2024-09-10T23:35:38.391082+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:00.834693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388544+0000","last_change":"2024-09-10T23:35:36.388544+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388544+0000","last_undegraded":"2024-09-10T23:35:36.388544+0000","last_fullsized":"2024-09-10T23:35:36.388544+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405535+0000","last_change":"2024-09-10T23:35:36.400467+0000","last_active":"2024-09-10T23:35:36.405535+0000","last_peered":"2024-09-10T23:35:36.405535+0000","last_clean":"2024-09-10T23:35:36.405535+0000","last_became_active":"2024-09-10T23:35:36.400164+0000","last_became_peered":"2024-09-10T23:35:36.400164+0000","last_unstale":"2024-09-10T23:35:36.405535+0000","last_undegraded":"2024-09-10T23:35:36.405535+0000","last_fullsized":"2024-09-10T23:35:36.405535+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:57.102709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391111+0000","last_change":"2024-09-10T23:35:36.391111+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391111+0000","last_undegraded":"2024-09-10T23:35:36.391111+0000","last_fullsized":"2024-09-10T23:35:36.391111+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407753+0000","last_change":"2024-09-10T23:35:36.399341+0000","last_active":"2024-09-10T23:35:36.407753+0000","last_peered":"2024-09-10T23:35:36.407753+0000","last_clean":"2024-09-10T23:35:36.407753+0000","last_became_active":"2024-09-10T23:35:36.399064+0000","last_became_peered":"2024-09-10T23:35:36.399064+0000","last_unstale":"2024-09-10T23:35:36.407753+0000","last_undegraded":"2024-09-10T23:35:36.407753+0000","last_fullsized":"2024-09-10T23:35:36.407753+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:34.124846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390616+0000","last_change":"2024-09-10T23:35:37.403404+0000","last_active":"2024-09-10T23:35:38.390616+0000","last_peered":"2024-09-10T23:35:38.390616+0000","last_clean":"2024-09-10T23:35:38.390616+0000","last_became_active":"2024-09-10T23:35:37.400175+0000","last_became_peered":"2024-09-10T23:35:37.400175+0000","last_unstale":"2024-09-10T23:35:38.390616+0000","last_undegraded":"2024-09-10T23:35:38.390616+0000","last_fullsized":"2024-09-10T23:35:38.390616+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:54.308424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.414180+0000","last_change":"2024-09-10T23:35:36.410996+0000","last_active":"2024-09-10T23:35:36.414180+0000","last_peered":"2024-09-10T23:35:36.414180+0000","last_clean":"2024-09-10T23:35:36.414180+0000","last_became_active":"2024-09-10T23:35:36.410559+0000","last_became_peered":"2024-09-10T23:35:36.410559+0000","last_unstale":"2024-09-10T23:35:36.414180+0000","last_undegraded":"2024-09-10T23:35:36.414180+0000","last_fullsized":"2024-09-10T23:35:36.414180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:40.032192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.395931+0000","last_change":"2024-09-10T23:35:36.395931+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.395931+0000","last_undegraded":"2024-09-10T23:35:36.395931+0000","last_fullsized":"2024-09-10T23:35:36.395931+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405469+0000","last_change":"2024-09-10T23:35:36.399492+0000","last_active":"2024-09-10T23:35:36.405469+0000","last_peered":"2024-09-10T23:35:36.405469+0000","last_clean":"2024-09-10T23:35:36.405469+0000","last_became_active":"2024-09-10T23:35:36.397982+0000","last_became_peered":"2024-09-10T23:35:36.397982+0000","last_unstale":"2024-09-10T23:35:36.405469+0000","last_undegraded":"2024-09-10T23:35:36.405469+0000","last_fullsized":"2024-09-10T23:35:36.405469+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:56.422393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389339+0000","last_change":"2024-09-10T23:35:36.389339+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389339+0000","last_undegraded":"2024-09-10T23:35:36.389339+0000","last_fullsized":"2024-09-10T23:35:36.389339+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389803+0000","last_change":"2024-09-10T23:35:36.406500+0000","last_active":"2024-09-10T23:35:38.389803+0000","last_peered":"2024-09-10T23:35:38.389803+0000","last_clean":"2024-09-10T23:35:38.389803+0000","last_became_active":"2024-09-10T23:35:36.406222+0000","last_became_peered":"2024-09-10T23:35:36.406222+0000","last_unstale":"2024-09-10T23:35:38.389803+0000","last_undegraded":"2024-09-10T23:35:38.389803+0000","last_fullsized":"2024-09-10T23:35:38.389803+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:36.140228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.387662+0000","last_change":"2024-09-10T23:35:36.387662+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.387662+0000","last_undegraded":"2024-09-10T23:35:36.387662+0000","last_fullsized":"2024-09-10T23:35:36.387662+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405662+0000","last_change":"2024-09-10T23:35:36.398978+0000","last_active":"2024-09-10T23:35:36.405662+0000","last_peered":"2024-09-10T23:35:36.405662+0000","last_clean":"2024-09-10T23:35:36.405662+0000","last_became_active":"2024-09-10T23:35:36.398751+0000","last_became_peered":"2024-09-10T23:35:36.398751+0000","last_unstale":"2024-09-10T23:35:36.405662+0000","last_undegraded":"2024-09-10T23:35:36.405662+0000","last_fullsized":"2024-09-10T23:35:36.405662+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:24.457146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.387645+0000","last_change":"2024-09-10T23:35:36.387645+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.387645+0000","last_undegraded":"2024-09-10T23:35:36.387645+0000","last_fullsized":"2024-09-10T23:35:36.387645+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407690+0000","last_change":"2024-09-10T23:35:36.403531+0000","last_active":"2024-09-10T23:35:36.407690+0000","last_peered":"2024-09-10T23:35:36.407690+0000","last_clean":"2024-09-10T23:35:36.407690+0000","last_became_active":"2024-09-10T23:35:36.403263+0000","last_became_peered":"2024-09-10T23:35:36.403263+0000","last_unstale":"2024-09-10T23:35:36.407690+0000","last_undegraded":"2024-09-10T23:35:36.407690+0000","last_fullsized":"2024-09-10T23:35:36.407690+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:34.124846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":173,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.390482+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:35:36.390482+0000","last_peered":"2024-09-10T23:35:36.390482+0000","last_clean":"2024-09-10T23:35:36.390482+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:35:36.390482+0000","last_undegraded":"2024-09-10T23:35:36.390482+0000","last_fullsized":"2024-09-10T23:35:36.390482+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":651264,"data_stored":264592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28244,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343596,"statfs":{"total":96636764160,"available":96607842304,"internally_reserved":0,"allocated":1245184,"data_stored":854960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":264592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":270867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":264592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":854960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":270867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":692224,"data_stored":283964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:35:41.504 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:35:41.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa2a5600640 1 -- 172.21.15.94:0/1516860690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a00cfcb0 msgr2=0x7fa2a00d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:41.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa2a5600640 1 --2- 172.21.15.94:0/1516860690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a00cfcb0 0x7fa2a00d00d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa288009840 tx=0x7fa28801c710 comp rx=0 tx=0).stop 2024-09-10T23:35:41.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa2a5600640 1 -- 172.21.15.94:0/1516860690 shutdown_connections 2024-09-10T23:35:41.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa2a5600640 1 --2- 172.21.15.94:0/1516860690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a00cfcb0 0x7fa2a00d00d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa2a5600640 1 -- 172.21.15.94:0/1516860690 >> 172.21.15.94:0/1516860690 conn(0x7fa2a006cdf0 msgr2=0x7fa2a006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:41.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa2a5600640 1 -- 172.21.15.94:0/1516860690 shutdown_connections 2024-09-10T23:35:41.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa2a5600640 1 -- 172.21.15.94:0/1516860690 wait complete. 2024-09-10T23:35:41.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa2a5600640 1 Processor -- start 2024-09-10T23:35:41.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa2a5600640 1 -- start start 2024-09-10T23:35:41.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa2a5600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a00cddb0 0x7fa2a0144340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:41.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa2a5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a00cfcb0 0x7fa2a00af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:41.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa2a5600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a015ba50 0x7fa2a00afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:41.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa2a5600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0169bf0 con 0x7fa2a00cfcb0 2024-09-10T23:35:41.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa2a5600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0169a70 con 0x7fa2a00cddb0 2024-09-10T23:35:41.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa2a5600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a0169d70 con 0x7fa2a015ba50 2024-09-10T23:35:41.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa296000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a00cfcb0 0x7fa2a00af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:41.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa296000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a00cfcb0 0x7fa2a00af6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42566/0 (socket says 172.21.15.94:42566) 2024-09-10T23:35:41.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa296000640 1 -- 172.21.15.94:0/1747957010 learned_addr learned my addr 172.21.15.94:0/1747957010 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:41.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa29ea00640 1 --2- 172.21.15.94:0/1747957010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a00cddb0 0x7fa2a0144340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:41.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa29f400640 1 --2- 172.21.15.94:0/1747957010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a015ba50 0x7fa2a00afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:41.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa296000640 1 -- 172.21.15.94:0/1747957010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a015ba50 msgr2=0x7fa2a00afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:41.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa296000640 1 --2- 172.21.15.94:0/1747957010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a015ba50 0x7fa2a00afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa296000640 1 -- 172.21.15.94:0/1747957010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a00cddb0 msgr2=0x7fa2a0144340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:41.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa296000640 1 --2- 172.21.15.94:0/1747957010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a00cddb0 0x7fa2a0144340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa296000640 1 -- 172.21.15.94:0/1747957010 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a01547a0 con 0x7fa2a00cfcb0 2024-09-10T23:35:41.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa296000640 1 --2- 172.21.15.94:0/1747957010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a00cfcb0 0x7fa2a00af6d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa288009790 tx=0x7fa288005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:41.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.686+0000 7fa297400640 1 -- 172.21.15.94:0/1747957010 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2880b2040 con 0x7fa2a00cfcb0 2024-09-10T23:35:41.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.690+0000 7fa297400640 1 -- 172.21.15.94:0/1747957010 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2880a5d20 con 0x7fa2a00cfcb0 2024-09-10T23:35:41.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.690+0000 7fa297400640 1 -- 172.21.15.94:0/1747957010 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288005280 con 0x7fa2a00cfcb0 2024-09-10T23:35:41.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.690+0000 7fa2a5600640 1 -- 172.21.15.94:0/1747957010 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a0154a80 con 0x7fa2a00cfcb0 2024-09-10T23:35:41.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.690+0000 7fa2a5600640 1 -- 172.21.15.94:0/1747957010 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a00b0e10 con 0x7fa2a00cfcb0 2024-09-10T23:35:41.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.690+0000 7fa2a5600640 1 -- 172.21.15.94:0/1747957010 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a00ce200 con 0x7fa2a00cfcb0 2024-09-10T23:35:41.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.690+0000 7fa297400640 1 -- 172.21.15.94:0/1747957010 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7fa2880b5070 con 0x7fa2a00cfcb0 2024-09-10T23:35:41.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.690+0000 7fa297400640 1 --2- 172.21.15.94:0/1747957010 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa27c0727c0 0x7fa27c074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:41.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.690+0000 7fa297400640 1 -- 172.21.15.94:0/1747957010 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa28812ccb0 con 0x7fa2a00cfcb0 2024-09-10T23:35:41.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.690+0000 7fa29ea00640 1 --2- 172.21.15.94:0/1747957010 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa27c0727c0 0x7fa27c074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:41.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.694+0000 7fa29ea00640 1 --2- 172.21.15.94:0/1747957010 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa27c0727c0 0x7fa27c074c80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa290007fd0 tx=0x7fa290009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:41.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.694+0000 7fa297400640 1 -- 172.21.15.94:0/1747957010 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa2880bc060 con 0x7fa2a00cfcb0 2024-09-10T23:35:41.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.854+0000 7fa2a5600640 1 -- 172.21.15.94:0/1747957010 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa2a015a620 con 0x7fa27c0727c0 2024-09-10T23:35:41.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.862+0000 7fa297400640 1 -- 172.21.15.94:0/1747957010 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247559 (secure 0 0 0) 0x7fa2a015a620 con 0x7fa27c0727c0 2024-09-10T23:35:41.865 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:41.866 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:35:41.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.862+0000 7fa2a5600640 1 -- 172.21.15.94:0/1747957010 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa27c0727c0 msgr2=0x7fa27c074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:41.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.862+0000 7fa2a5600640 1 --2- 172.21.15.94:0/1747957010 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa27c0727c0 0x7fa27c074c80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa290007fd0 tx=0x7fa290009040 comp rx=0 tx=0).stop 2024-09-10T23:35:41.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.862+0000 7fa2a5600640 1 -- 172.21.15.94:0/1747957010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a00cfcb0 msgr2=0x7fa2a00af6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:41.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.862+0000 7fa2a5600640 1 --2- 172.21.15.94:0/1747957010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a00cfcb0 0x7fa2a00af6d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa288009790 tx=0x7fa288005db0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.862+0000 7fa2a5600640 1 -- 172.21.15.94:0/1747957010 shutdown_connections 2024-09-10T23:35:41.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.862+0000 7fa2a5600640 1 --2- 172.21.15.94:0/1747957010 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa27c0727c0 0x7fa27c074c80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.862+0000 7fa2a5600640 1 --2- 172.21.15.94:0/1747957010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a015ba50 0x7fa2a00afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.862+0000 7fa2a5600640 1 --2- 172.21.15.94:0/1747957010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a00cfcb0 0x7fa2a00af6d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.862+0000 7fa2a5600640 1 --2- 172.21.15.94:0/1747957010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a00cddb0 0x7fa2a0144340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:41.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.862+0000 7fa2a5600640 1 -- 172.21.15.94:0/1747957010 >> 172.21.15.94:0/1747957010 conn(0x7fa2a006cdf0 msgr2=0x7fa2a0072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:41.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.862+0000 7fa2a5600640 1 -- 172.21.15.94:0/1747957010 shutdown_connections 2024-09-10T23:35:41.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:41.866+0000 7fa2a5600640 1 -- 172.21.15.94:0/1747957010 wait complete. 2024-09-10T23:35:41.885 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-10T23:35:41.388360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247508,"kb_used_data":6312,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727212,"statfs":{"total":773094113280,"available":772840665088,"internally_reserved":0,"allocated":6463488,"data_stored":3329394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001510"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391197+0000","last_change":"2024-09-10T23:35:37.411838+0000","last_active":"2024-09-10T23:35:38.391197+0000","last_peered":"2024-09-10T23:35:38.391197+0000","last_clean":"2024-09-10T23:35:38.391197+0000","last_became_active":"2024-09-10T23:35:37.411168+0000","last_became_peered":"2024-09-10T23:35:37.411168+0000","last_unstale":"2024-09-10T23:35:38.391197+0000","last_undegraded":"2024-09-10T23:35:38.391197+0000","last_fullsized":"2024-09-10T23:35:38.391197+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:27.635555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388506+0000","last_change":"2024-09-10T23:35:36.388506+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388506+0000","last_undegraded":"2024-09-10T23:35:36.388506+0000","last_fullsized":"2024-09-10T23:35:36.388506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.390335+0000","last_change":"2024-09-10T23:35:36.390335+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.390335+0000","last_undegraded":"2024-09-10T23:35:36.390335+0000","last_fullsized":"2024-09-10T23:35:36.390335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392121+0000","last_change":"2024-09-10T23:35:36.392121+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392121+0000","last_undegraded":"2024-09-10T23:35:36.392121+0000","last_fullsized":"2024-09-10T23:35:36.392121+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390631+0000","last_change":"2024-09-10T23:35:37.422878+0000","last_active":"2024-09-10T23:35:38.390631+0000","last_peered":"2024-09-10T23:35:38.390631+0000","last_clean":"2024-09-10T23:35:38.390631+0000","last_became_active":"2024-09-10T23:35:37.422602+0000","last_became_peered":"2024-09-10T23:35:37.422602+0000","last_unstale":"2024-09-10T23:35:38.390631+0000","last_undegraded":"2024-09-10T23:35:38.390631+0000","last_fullsized":"2024-09-10T23:35:38.390631+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:17.965755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390045+0000","last_change":"2024-09-10T23:35:37.401854+0000","last_active":"2024-09-10T23:35:38.390045+0000","last_peered":"2024-09-10T23:35:38.390045+0000","last_clean":"2024-09-10T23:35:38.390045+0000","last_became_active":"2024-09-10T23:35:37.400521+0000","last_became_peered":"2024-09-10T23:35:37.400521+0000","last_unstale":"2024-09-10T23:35:38.390045+0000","last_undegraded":"2024-09-10T23:35:38.390045+0000","last_fullsized":"2024-09-10T23:35:38.390045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:48.537421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394892+0000","last_change":"2024-09-10T23:35:36.394892+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394892+0000","last_undegraded":"2024-09-10T23:35:36.394892+0000","last_fullsized":"2024-09-10T23:35:36.394892+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389814+0000","last_change":"2024-09-10T23:35:36.389814+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389814+0000","last_undegraded":"2024-09-10T23:35:36.389814+0000","last_fullsized":"2024-09-10T23:35:36.389814+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389006+0000","last_change":"2024-09-10T23:35:36.389006+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389006+0000","last_undegraded":"2024-09-10T23:35:36.389006+0000","last_fullsized":"2024-09-10T23:35:36.389006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390598+0000","last_change":"2024-09-10T23:35:37.405331+0000","last_active":"2024-09-10T23:35:38.390598+0000","last_peered":"2024-09-10T23:35:38.390598+0000","last_clean":"2024-09-10T23:35:38.390598+0000","last_became_active":"2024-09-10T23:35:37.404010+0000","last_became_peered":"2024-09-10T23:35:37.404010+0000","last_unstale":"2024-09-10T23:35:38.390598+0000","last_undegraded":"2024-09-10T23:35:38.390598+0000","last_fullsized":"2024-09-10T23:35:38.390598+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:30.787726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394567+0000","last_change":"2024-09-10T23:35:36.394567+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394567+0000","last_undegraded":"2024-09-10T23:35:36.394567+0000","last_fullsized":"2024-09-10T23:35:36.394567+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391006+0000","last_change":"2024-09-10T23:35:36.391006+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391006+0000","last_undegraded":"2024-09-10T23:35:36.391006+0000","last_fullsized":"2024-09-10T23:35:36.391006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.397258+0000","last_change":"2024-09-10T23:35:36.397258+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.397258+0000","last_undegraded":"2024-09-10T23:35:36.397258+0000","last_fullsized":"2024-09-10T23:35:36.397258+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389045+0000","last_change":"2024-09-10T23:35:36.389045+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389045+0000","last_undegraded":"2024-09-10T23:35:36.389045+0000","last_fullsized":"2024-09-10T23:35:36.389045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394449+0000","last_change":"2024-09-10T23:35:36.394449+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394449+0000","last_undegraded":"2024-09-10T23:35:36.394449+0000","last_fullsized":"2024-09-10T23:35:36.394449+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389559+0000","last_change":"2024-09-10T23:35:37.421193+0000","last_active":"2024-09-10T23:35:38.389559+0000","last_peered":"2024-09-10T23:35:38.389559+0000","last_clean":"2024-09-10T23:35:38.389559+0000","last_became_active":"2024-09-10T23:35:37.420909+0000","last_became_peered":"2024-09-10T23:35:37.420909+0000","last_unstale":"2024-09-10T23:35:38.389559+0000","last_undegraded":"2024-09-10T23:35:38.389559+0000","last_fullsized":"2024-09-10T23:35:38.389559+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:31.257242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391217+0000","last_change":"2024-09-10T23:35:37.404324+0000","last_active":"2024-09-10T23:35:38.391217+0000","last_peered":"2024-09-10T23:35:38.391217+0000","last_clean":"2024-09-10T23:35:38.391217+0000","last_became_active":"2024-09-10T23:35:37.404042+0000","last_became_peered":"2024-09-10T23:35:37.404042+0000","last_unstale":"2024-09-10T23:35:38.391217+0000","last_undegraded":"2024-09-10T23:35:38.391217+0000","last_fullsized":"2024-09-10T23:35:38.391217+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:31.250579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392062+0000","last_change":"2024-09-10T23:35:36.392062+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392062+0000","last_undegraded":"2024-09-10T23:35:36.392062+0000","last_fullsized":"2024-09-10T23:35:36.392062+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389708+0000","last_change":"2024-09-10T23:35:36.389708+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389708+0000","last_undegraded":"2024-09-10T23:35:36.389708+0000","last_fullsized":"2024-09-10T23:35:36.389708+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394443+0000","last_change":"2024-09-10T23:35:36.394443+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394443+0000","last_undegraded":"2024-09-10T23:35:36.394443+0000","last_fullsized":"2024-09-10T23:35:36.394443+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.396891+0000","last_change":"2024-09-10T23:35:36.396891+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.396891+0000","last_undegraded":"2024-09-10T23:35:36.396891+0000","last_fullsized":"2024-09-10T23:35:36.396891+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.396024+0000","last_change":"2024-09-10T23:35:36.396024+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.396024+0000","last_undegraded":"2024-09-10T23:35:36.396024+0000","last_fullsized":"2024-09-10T23:35:36.396024+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392968+0000","last_change":"2024-09-10T23:35:36.392968+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392968+0000","last_undegraded":"2024-09-10T23:35:36.392968+0000","last_fullsized":"2024-09-10T23:35:36.392968+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.390266+0000","last_change":"2024-09-10T23:35:36.390266+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.390266+0000","last_undegraded":"2024-09-10T23:35:36.390266+0000","last_fullsized":"2024-09-10T23:35:36.390266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389015+0000","last_change":"2024-09-10T23:35:36.389015+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389015+0000","last_undegraded":"2024-09-10T23:35:36.389015+0000","last_fullsized":"2024-09-10T23:35:36.389015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.395481+0000","last_change":"2024-09-10T23:35:36.395481+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.395481+0000","last_undegraded":"2024-09-10T23:35:36.395481+0000","last_fullsized":"2024-09-10T23:35:36.395481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389120+0000","last_change":"2024-09-10T23:35:36.389120+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389120+0000","last_undegraded":"2024-09-10T23:35:36.389120+0000","last_fullsized":"2024-09-10T23:35:36.389120+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392340+0000","last_change":"2024-09-10T23:35:36.392340+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392340+0000","last_undegraded":"2024-09-10T23:35:36.392340+0000","last_fullsized":"2024-09-10T23:35:36.392340+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388535+0000","last_change":"2024-09-10T23:35:36.388535+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388535+0000","last_undegraded":"2024-09-10T23:35:36.388535+0000","last_fullsized":"2024-09-10T23:35:36.388535+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391729+0000","last_change":"2024-09-10T23:35:36.391729+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391729+0000","last_undegraded":"2024-09-10T23:35:36.391729+0000","last_fullsized":"2024-09-10T23:35:36.391729+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388251+0000","last_change":"2024-09-10T23:35:36.388251+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388251+0000","last_undegraded":"2024-09-10T23:35:36.388251+0000","last_fullsized":"2024-09-10T23:35:36.388251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394059+0000","last_change":"2024-09-10T23:35:36.394059+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394059+0000","last_undegraded":"2024-09-10T23:35:36.394059+0000","last_fullsized":"2024-09-10T23:35:36.394059+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391188+0000","last_change":"2024-09-10T23:35:37.405530+0000","last_active":"2024-09-10T23:35:38.391188+0000","last_peered":"2024-09-10T23:35:38.391188+0000","last_clean":"2024-09-10T23:35:38.391188+0000","last_became_active":"2024-09-10T23:35:37.402795+0000","last_became_peered":"2024-09-10T23:35:37.402795+0000","last_unstale":"2024-09-10T23:35:38.391188+0000","last_undegraded":"2024-09-10T23:35:38.391188+0000","last_fullsized":"2024-09-10T23:35:38.391188+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:15.871163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.404962+0000","last_change":"2024-09-10T23:35:36.401576+0000","last_active":"2024-09-10T23:35:36.404962+0000","last_peered":"2024-09-10T23:35:36.404962+0000","last_clean":"2024-09-10T23:35:36.404962+0000","last_became_active":"2024-09-10T23:35:36.400780+0000","last_became_peered":"2024-09-10T23:35:36.400780+0000","last_unstale":"2024-09-10T23:35:36.404962+0000","last_undegraded":"2024-09-10T23:35:36.404962+0000","last_fullsized":"2024-09-10T23:35:36.404962+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:15.011008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391401+0000","last_change":"2024-09-10T23:35:37.411543+0000","last_active":"2024-09-10T23:35:38.391401+0000","last_peered":"2024-09-10T23:35:38.391401+0000","last_clean":"2024-09-10T23:35:38.391401+0000","last_became_active":"2024-09-10T23:35:37.410807+0000","last_became_peered":"2024-09-10T23:35:37.410807+0000","last_unstale":"2024-09-10T23:35:38.391401+0000","last_undegraded":"2024-09-10T23:35:38.391401+0000","last_fullsized":"2024-09-10T23:35:38.391401+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:52.749748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.410357+0000","last_change":"2024-09-10T23:35:36.406745+0000","last_active":"2024-09-10T23:35:36.410357+0000","last_peered":"2024-09-10T23:35:36.410357+0000","last_clean":"2024-09-10T23:35:36.410357+0000","last_became_active":"2024-09-10T23:35:36.406460+0000","last_became_peered":"2024-09-10T23:35:36.406460+0000","last_unstale":"2024-09-10T23:35:36.410357+0000","last_undegraded":"2024-09-10T23:35:36.410357+0000","last_fullsized":"2024-09-10T23:35:36.410357+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:58.569452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391940+0000","last_change":"2024-09-10T23:35:36.391940+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391940+0000","last_undegraded":"2024-09-10T23:35:36.391940+0000","last_fullsized":"2024-09-10T23:35:36.391940+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390807+0000","last_change":"2024-09-10T23:35:36.409095+0000","last_active":"2024-09-10T23:35:38.390807+0000","last_peered":"2024-09-10T23:35:38.390807+0000","last_clean":"2024-09-10T23:35:38.390807+0000","last_became_active":"2024-09-10T23:35:36.408811+0000","last_became_peered":"2024-09-10T23:35:36.408811+0000","last_unstale":"2024-09-10T23:35:38.390807+0000","last_undegraded":"2024-09-10T23:35:38.390807+0000","last_fullsized":"2024-09-10T23:35:38.390807+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:47.477496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390766+0000","last_change":"2024-09-10T23:35:37.413035+0000","last_active":"2024-09-10T23:35:38.390766+0000","last_peered":"2024-09-10T23:35:38.390766+0000","last_clean":"2024-09-10T23:35:38.390766+0000","last_became_active":"2024-09-10T23:35:37.412781+0000","last_became_peered":"2024-09-10T23:35:37.412781+0000","last_unstale":"2024-09-10T23:35:38.390766+0000","last_undegraded":"2024-09-10T23:35:38.390766+0000","last_fullsized":"2024-09-10T23:35:38.390766+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:11.135478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.404617+0000","last_change":"2024-09-10T23:35:36.396384+0000","last_active":"2024-09-10T23:35:36.404617+0000","last_peered":"2024-09-10T23:35:36.404617+0000","last_clean":"2024-09-10T23:35:36.404617+0000","last_became_active":"2024-09-10T23:35:36.396085+0000","last_became_peered":"2024-09-10T23:35:36.396085+0000","last_unstale":"2024-09-10T23:35:36.404617+0000","last_undegraded":"2024-09-10T23:35:36.404617+0000","last_fullsized":"2024-09-10T23:35:36.404617+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:34.607437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.387665+0000","last_change":"2024-09-10T23:35:36.387665+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.387665+0000","last_undegraded":"2024-09-10T23:35:36.387665+0000","last_fullsized":"2024-09-10T23:35:36.387665+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407850+0000","last_change":"2024-09-10T23:35:36.404943+0000","last_active":"2024-09-10T23:35:36.407850+0000","last_peered":"2024-09-10T23:35:36.407850+0000","last_clean":"2024-09-10T23:35:36.407850+0000","last_became_active":"2024-09-10T23:35:36.404615+0000","last_became_peered":"2024-09-10T23:35:36.404615+0000","last_unstale":"2024-09-10T23:35:36.407850+0000","last_undegraded":"2024-09-10T23:35:36.407850+0000","last_fullsized":"2024-09-10T23:35:36.407850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:58.666819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.396962+0000","last_change":"2024-09-10T23:35:36.396962+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.396962+0000","last_undegraded":"2024-09-10T23:35:36.396962+0000","last_fullsized":"2024-09-10T23:35:36.396962+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390698+0000","last_change":"2024-09-10T23:35:36.408437+0000","last_active":"2024-09-10T23:35:38.390698+0000","last_peered":"2024-09-10T23:35:38.390698+0000","last_clean":"2024-09-10T23:35:38.390698+0000","last_became_active":"2024-09-10T23:35:36.408156+0000","last_became_peered":"2024-09-10T23:35:36.408156+0000","last_unstale":"2024-09-10T23:35:38.390698+0000","last_undegraded":"2024-09-10T23:35:38.390698+0000","last_fullsized":"2024-09-10T23:35:38.390698+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:01.998360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389936+0000","last_change":"2024-09-10T23:35:36.389936+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389936+0000","last_undegraded":"2024-09-10T23:35:36.389936+0000","last_fullsized":"2024-09-10T23:35:36.389936+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407639+0000","last_change":"2024-09-10T23:35:36.401028+0000","last_active":"2024-09-10T23:35:36.407639+0000","last_peered":"2024-09-10T23:35:36.407639+0000","last_clean":"2024-09-10T23:35:36.407639+0000","last_became_active":"2024-09-10T23:35:36.400743+0000","last_became_peered":"2024-09-10T23:35:36.400743+0000","last_unstale":"2024-09-10T23:35:36.407639+0000","last_undegraded":"2024-09-10T23:35:36.407639+0000","last_fullsized":"2024-09-10T23:35:36.407639+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:39:33.252954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389654+0000","last_change":"2024-09-10T23:35:37.422945+0000","last_active":"2024-09-10T23:35:38.389654+0000","last_peered":"2024-09-10T23:35:38.389654+0000","last_clean":"2024-09-10T23:35:38.389654+0000","last_became_active":"2024-09-10T23:35:37.422648+0000","last_became_peered":"2024-09-10T23:35:37.422648+0000","last_unstale":"2024-09-10T23:35:38.389654+0000","last_undegraded":"2024-09-10T23:35:38.389654+0000","last_fullsized":"2024-09-10T23:35:38.389654+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:10.705976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406280+0000","last_change":"2024-09-10T23:35:36.401434+0000","last_active":"2024-09-10T23:35:36.406280+0000","last_peered":"2024-09-10T23:35:36.406280+0000","last_clean":"2024-09-10T23:35:36.406280+0000","last_became_active":"2024-09-10T23:35:36.401155+0000","last_became_peered":"2024-09-10T23:35:36.401155+0000","last_unstale":"2024-09-10T23:35:36.406280+0000","last_undegraded":"2024-09-10T23:35:36.406280+0000","last_fullsized":"2024-09-10T23:35:36.406280+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:37.536960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394992+0000","last_change":"2024-09-10T23:35:36.394992+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394992+0000","last_undegraded":"2024-09-10T23:35:36.394992+0000","last_fullsized":"2024-09-10T23:35:36.394992+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407810+0000","last_change":"2024-09-10T23:35:36.405013+0000","last_active":"2024-09-10T23:35:36.407810+0000","last_peered":"2024-09-10T23:35:36.407810+0000","last_clean":"2024-09-10T23:35:36.407810+0000","last_became_active":"2024-09-10T23:35:36.404715+0000","last_became_peered":"2024-09-10T23:35:36.404715+0000","last_unstale":"2024-09-10T23:35:36.407810+0000","last_undegraded":"2024-09-10T23:35:36.407810+0000","last_fullsized":"2024-09-10T23:35:36.407810+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:11:46.791009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394500+0000","last_change":"2024-09-10T23:35:36.394500+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394500+0000","last_undegraded":"2024-09-10T23:35:36.394500+0000","last_fullsized":"2024-09-10T23:35:36.394500+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407797+0000","last_change":"2024-09-10T23:35:36.402529+0000","last_active":"2024-09-10T23:35:36.407797+0000","last_peered":"2024-09-10T23:35:36.407797+0000","last_clean":"2024-09-10T23:35:36.407797+0000","last_became_active":"2024-09-10T23:35:36.402270+0000","last_became_peered":"2024-09-10T23:35:36.402270+0000","last_unstale":"2024-09-10T23:35:36.407797+0000","last_undegraded":"2024-09-10T23:35:36.407797+0000","last_fullsized":"2024-09-10T23:35:36.407797+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:28.809096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389483+0000","last_change":"2024-09-10T23:35:36.389483+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389483+0000","last_undegraded":"2024-09-10T23:35:36.389483+0000","last_fullsized":"2024-09-10T23:35:36.389483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406578+0000","last_change":"2024-09-10T23:35:36.402631+0000","last_active":"2024-09-10T23:35:36.406578+0000","last_peered":"2024-09-10T23:35:36.406578+0000","last_clean":"2024-09-10T23:35:36.406578+0000","last_became_active":"2024-09-10T23:35:36.399947+0000","last_became_peered":"2024-09-10T23:35:36.399947+0000","last_unstale":"2024-09-10T23:35:36.406578+0000","last_undegraded":"2024-09-10T23:35:36.406578+0000","last_fullsized":"2024-09-10T23:35:36.406578+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:53.498796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391900+0000","last_change":"2024-09-10T23:35:36.391900+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391900+0000","last_undegraded":"2024-09-10T23:35:36.391900+0000","last_fullsized":"2024-09-10T23:35:36.391900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.408350+0000","last_change":"2024-09-10T23:35:36.405711+0000","last_active":"2024-09-10T23:35:36.408350+0000","last_peered":"2024-09-10T23:35:36.408350+0000","last_clean":"2024-09-10T23:35:36.408350+0000","last_became_active":"2024-09-10T23:35:36.405455+0000","last_became_peered":"2024-09-10T23:35:36.405455+0000","last_unstale":"2024-09-10T23:35:36.408350+0000","last_undegraded":"2024-09-10T23:35:36.408350+0000","last_fullsized":"2024-09-10T23:35:36.408350+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:56.422393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392350+0000","last_change":"2024-09-10T23:35:36.392350+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392350+0000","last_undegraded":"2024-09-10T23:35:36.392350+0000","last_fullsized":"2024-09-10T23:35:36.392350+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407863+0000","last_change":"2024-09-10T23:35:36.404848+0000","last_active":"2024-09-10T23:35:36.407863+0000","last_peered":"2024-09-10T23:35:36.407863+0000","last_clean":"2024-09-10T23:35:36.407863+0000","last_became_active":"2024-09-10T23:35:36.404567+0000","last_became_peered":"2024-09-10T23:35:36.404567+0000","last_unstale":"2024-09-10T23:35:36.407863+0000","last_undegraded":"2024-09-10T23:35:36.407863+0000","last_fullsized":"2024-09-10T23:35:36.407863+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:33.274868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388987+0000","last_change":"2024-09-10T23:35:36.388987+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388987+0000","last_undegraded":"2024-09-10T23:35:36.388987+0000","last_fullsized":"2024-09-10T23:35:36.388987+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407454+0000","last_change":"2024-09-10T23:35:36.402546+0000","last_active":"2024-09-10T23:35:36.407454+0000","last_peered":"2024-09-10T23:35:36.407454+0000","last_clean":"2024-09-10T23:35:36.407454+0000","last_became_active":"2024-09-10T23:35:36.402187+0000","last_became_peered":"2024-09-10T23:35:36.402187+0000","last_unstale":"2024-09-10T23:35:36.407454+0000","last_undegraded":"2024-09-10T23:35:36.407454+0000","last_fullsized":"2024-09-10T23:35:36.407454+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:18.611164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389852+0000","last_change":"2024-09-10T23:35:36.389852+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389852+0000","last_undegraded":"2024-09-10T23:35:36.389852+0000","last_fullsized":"2024-09-10T23:35:36.389852+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406227+0000","last_change":"2024-09-10T23:35:36.401288+0000","last_active":"2024-09-10T23:35:36.406227+0000","last_peered":"2024-09-10T23:35:36.406227+0000","last_clean":"2024-09-10T23:35:36.406227+0000","last_became_active":"2024-09-10T23:35:36.400889+0000","last_became_peered":"2024-09-10T23:35:36.400889+0000","last_unstale":"2024-09-10T23:35:36.406227+0000","last_undegraded":"2024-09-10T23:35:36.406227+0000","last_fullsized":"2024-09-10T23:35:36.406227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:05.132993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.390490+0000","last_change":"2024-09-10T23:35:36.390490+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.390490+0000","last_undegraded":"2024-09-10T23:35:36.390490+0000","last_fullsized":"2024-09-10T23:35:36.390490+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406140+0000","last_change":"2024-09-10T23:35:36.398356+0000","last_active":"2024-09-10T23:35:36.406140+0000","last_peered":"2024-09-10T23:35:36.406140+0000","last_clean":"2024-09-10T23:35:36.406140+0000","last_became_active":"2024-09-10T23:35:36.398077+0000","last_became_peered":"2024-09-10T23:35:36.398077+0000","last_unstale":"2024-09-10T23:35:36.406140+0000","last_undegraded":"2024-09-10T23:35:36.406140+0000","last_fullsized":"2024-09-10T23:35:36.406140+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:11.718995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391693+0000","last_change":"2024-09-10T23:35:36.391693+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391693+0000","last_undegraded":"2024-09-10T23:35:36.391693+0000","last_fullsized":"2024-09-10T23:35:36.391693+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.411637+0000","last_change":"2024-09-10T23:35:36.409579+0000","last_active":"2024-09-10T23:35:36.411637+0000","last_peered":"2024-09-10T23:35:36.411637+0000","last_clean":"2024-09-10T23:35:36.411637+0000","last_became_active":"2024-09-10T23:35:36.409321+0000","last_became_peered":"2024-09-10T23:35:36.409321+0000","last_unstale":"2024-09-10T23:35:36.411637+0000","last_undegraded":"2024-09-10T23:35:36.411637+0000","last_fullsized":"2024-09-10T23:35:36.411637+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:19.816523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394047+0000","last_change":"2024-09-10T23:35:36.394047+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394047+0000","last_undegraded":"2024-09-10T23:35:36.394047+0000","last_fullsized":"2024-09-10T23:35:36.394047+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405719+0000","last_change":"2024-09-10T23:35:36.400916+0000","last_active":"2024-09-10T23:35:36.405719+0000","last_peered":"2024-09-10T23:35:36.405719+0000","last_clean":"2024-09-10T23:35:36.405719+0000","last_became_active":"2024-09-10T23:35:36.400639+0000","last_became_peered":"2024-09-10T23:35:36.400639+0000","last_unstale":"2024-09-10T23:35:36.405719+0000","last_undegraded":"2024-09-10T23:35:36.405719+0000","last_fullsized":"2024-09-10T23:35:36.405719+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:47.327210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389265+0000","last_change":"2024-09-10T23:35:36.389265+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389265+0000","last_undegraded":"2024-09-10T23:35:36.389265+0000","last_fullsized":"2024-09-10T23:35:36.389265+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405406+0000","last_change":"2024-09-10T23:35:36.398766+0000","last_active":"2024-09-10T23:35:36.405406+0000","last_peered":"2024-09-10T23:35:36.405406+0000","last_clean":"2024-09-10T23:35:36.405406+0000","last_became_active":"2024-09-10T23:35:36.398473+0000","last_became_peered":"2024-09-10T23:35:36.398473+0000","last_unstale":"2024-09-10T23:35:36.405406+0000","last_undegraded":"2024-09-10T23:35:36.405406+0000","last_fullsized":"2024-09-10T23:35:36.405406+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:37.536960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388202+0000","last_change":"2024-09-10T23:35:36.388202+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388202+0000","last_undegraded":"2024-09-10T23:35:36.388202+0000","last_fullsized":"2024-09-10T23:35:36.388202+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.414301+0000","last_change":"2024-09-10T23:35:36.409772+0000","last_active":"2024-09-10T23:35:36.414301+0000","last_peered":"2024-09-10T23:35:36.414301+0000","last_clean":"2024-09-10T23:35:36.414301+0000","last_became_active":"2024-09-10T23:35:36.409432+0000","last_became_peered":"2024-09-10T23:35:36.409432+0000","last_unstale":"2024-09-10T23:35:36.414301+0000","last_undegraded":"2024-09-10T23:35:36.414301+0000","last_fullsized":"2024-09-10T23:35:36.414301+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:09.775738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391962+0000","last_change":"2024-09-10T23:35:36.391962+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391962+0000","last_undegraded":"2024-09-10T23:35:36.391962+0000","last_fullsized":"2024-09-10T23:35:36.391962+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.409418+0000","last_change":"2024-09-10T23:35:36.405162+0000","last_active":"2024-09-10T23:35:36.409418+0000","last_peered":"2024-09-10T23:35:36.409418+0000","last_clean":"2024-09-10T23:35:36.409418+0000","last_became_active":"2024-09-10T23:35:36.404891+0000","last_became_peered":"2024-09-10T23:35:36.404891+0000","last_unstale":"2024-09-10T23:35:36.409418+0000","last_undegraded":"2024-09-10T23:35:36.409418+0000","last_fullsized":"2024-09-10T23:35:36.409418+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:25.179787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388064+0000","last_change":"2024-09-10T23:35:36.388064+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388064+0000","last_undegraded":"2024-09-10T23:35:36.388064+0000","last_fullsized":"2024-09-10T23:35:36.388064+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406216+0000","last_change":"2024-09-10T23:35:36.401149+0000","last_active":"2024-09-10T23:35:36.406216+0000","last_peered":"2024-09-10T23:35:36.406216+0000","last_clean":"2024-09-10T23:35:36.406216+0000","last_became_active":"2024-09-10T23:35:36.400884+0000","last_became_peered":"2024-09-10T23:35:36.400884+0000","last_unstale":"2024-09-10T23:35:36.406216+0000","last_undegraded":"2024-09-10T23:35:36.406216+0000","last_fullsized":"2024-09-10T23:35:36.406216+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:43.580364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388430+0000","last_change":"2024-09-10T23:35:36.388430+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388430+0000","last_undegraded":"2024-09-10T23:35:36.388430+0000","last_fullsized":"2024-09-10T23:35:36.388430+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407985+0000","last_change":"2024-09-10T23:35:36.406290+0000","last_active":"2024-09-10T23:35:36.407985+0000","last_peered":"2024-09-10T23:35:36.407985+0000","last_clean":"2024-09-10T23:35:36.407985+0000","last_became_active":"2024-09-10T23:35:36.406056+0000","last_became_peered":"2024-09-10T23:35:36.406056+0000","last_unstale":"2024-09-10T23:35:36.407985+0000","last_undegraded":"2024-09-10T23:35:36.407985+0000","last_fullsized":"2024-09-10T23:35:36.407985+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:59.329841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391377+0000","last_change":"2024-09-10T23:35:36.391377+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391377+0000","last_undegraded":"2024-09-10T23:35:36.391377+0000","last_fullsized":"2024-09-10T23:35:36.391377+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.410432+0000","last_change":"2024-09-10T23:35:36.406893+0000","last_active":"2024-09-10T23:35:36.410432+0000","last_peered":"2024-09-10T23:35:36.410432+0000","last_clean":"2024-09-10T23:35:36.410432+0000","last_became_active":"2024-09-10T23:35:36.406516+0000","last_became_peered":"2024-09-10T23:35:36.406516+0000","last_unstale":"2024-09-10T23:35:36.410432+0000","last_undegraded":"2024-09-10T23:35:36.410432+0000","last_fullsized":"2024-09-10T23:35:36.410432+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:13.080452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.393643+0000","last_change":"2024-09-10T23:35:36.393643+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.393643+0000","last_undegraded":"2024-09-10T23:35:36.393643+0000","last_fullsized":"2024-09-10T23:35:36.393643+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391082+0000","last_change":"2024-09-10T23:35:36.409344+0000","last_active":"2024-09-10T23:35:38.391082+0000","last_peered":"2024-09-10T23:35:38.391082+0000","last_clean":"2024-09-10T23:35:38.391082+0000","last_became_active":"2024-09-10T23:35:36.408975+0000","last_became_peered":"2024-09-10T23:35:36.408975+0000","last_unstale":"2024-09-10T23:35:38.391082+0000","last_undegraded":"2024-09-10T23:35:38.391082+0000","last_fullsized":"2024-09-10T23:35:38.391082+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:00.834693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388544+0000","last_change":"2024-09-10T23:35:36.388544+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388544+0000","last_undegraded":"2024-09-10T23:35:36.388544+0000","last_fullsized":"2024-09-10T23:35:36.388544+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405535+0000","last_change":"2024-09-10T23:35:36.400467+0000","last_active":"2024-09-10T23:35:36.405535+0000","last_peered":"2024-09-10T23:35:36.405535+0000","last_clean":"2024-09-10T23:35:36.405535+0000","last_became_active":"2024-09-10T23:35:36.400164+0000","last_became_peered":"2024-09-10T23:35:36.400164+0000","last_unstale":"2024-09-10T23:35:36.405535+0000","last_undegraded":"2024-09-10T23:35:36.405535+0000","last_fullsized":"2024-09-10T23:35:36.405535+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:57.102709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391111+0000","last_change":"2024-09-10T23:35:36.391111+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391111+0000","last_undegraded":"2024-09-10T23:35:36.391111+0000","last_fullsized":"2024-09-10T23:35:36.391111+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407753+0000","last_change":"2024-09-10T23:35:36.399341+0000","last_active":"2024-09-10T23:35:36.407753+0000","last_peered":"2024-09-10T23:35:36.407753+0000","last_clean":"2024-09-10T23:35:36.407753+0000","last_became_active":"2024-09-10T23:35:36.399064+0000","last_became_peered":"2024-09-10T23:35:36.399064+0000","last_unstale":"2024-09-10T23:35:36.407753+0000","last_undegraded":"2024-09-10T23:35:36.407753+0000","last_fullsized":"2024-09-10T23:35:36.407753+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:34.124846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390616+0000","last_change":"2024-09-10T23:35:37.403404+0000","last_active":"2024-09-10T23:35:38.390616+0000","last_peered":"2024-09-10T23:35:38.390616+0000","last_clean":"2024-09-10T23:35:38.390616+0000","last_became_active":"2024-09-10T23:35:37.400175+0000","last_became_peered":"2024-09-10T23:35:37.400175+0000","last_unstale":"2024-09-10T23:35:38.390616+0000","last_undegraded":"2024-09-10T23:35:38.390616+0000","last_fullsized":"2024-09-10T23:35:38.390616+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:54.308424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.414180+0000","last_change":"2024-09-10T23:35:36.410996+0000","last_active":"2024-09-10T23:35:36.414180+0000","last_peered":"2024-09-10T23:35:36.414180+0000","last_clean":"2024-09-10T23:35:36.414180+0000","last_became_active":"2024-09-10T23:35:36.410559+0000","last_became_peered":"2024-09-10T23:35:36.410559+0000","last_unstale":"2024-09-10T23:35:36.414180+0000","last_undegraded":"2024-09-10T23:35:36.414180+0000","last_fullsized":"2024-09-10T23:35:36.414180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:40.032192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.395931+0000","last_change":"2024-09-10T23:35:36.395931+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.395931+0000","last_undegraded":"2024-09-10T23:35:36.395931+0000","last_fullsized":"2024-09-10T23:35:36.395931+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405469+0000","last_change":"2024-09-10T23:35:36.399492+0000","last_active":"2024-09-10T23:35:36.405469+0000","last_peered":"2024-09-10T23:35:36.405469+0000","last_clean":"2024-09-10T23:35:36.405469+0000","last_became_active":"2024-09-10T23:35:36.397982+0000","last_became_peered":"2024-09-10T23:35:36.397982+0000","last_unstale":"2024-09-10T23:35:36.405469+0000","last_undegraded":"2024-09-10T23:35:36.405469+0000","last_fullsized":"2024-09-10T23:35:36.405469+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:56.422393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389339+0000","last_change":"2024-09-10T23:35:36.389339+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389339+0000","last_undegraded":"2024-09-10T23:35:36.389339+0000","last_fullsized":"2024-09-10T23:35:36.389339+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389803+0000","last_change":"2024-09-10T23:35:36.406500+0000","last_active":"2024-09-10T23:35:38.389803+0000","last_peered":"2024-09-10T23:35:38.389803+0000","last_clean":"2024-09-10T23:35:38.389803+0000","last_became_active":"2024-09-10T23:35:36.406222+0000","last_became_peered":"2024-09-10T23:35:36.406222+0000","last_unstale":"2024-09-10T23:35:38.389803+0000","last_undegraded":"2024-09-10T23:35:38.389803+0000","last_fullsized":"2024-09-10T23:35:38.389803+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:36.140228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.387662+0000","last_change":"2024-09-10T23:35:36.387662+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.387662+0000","last_undegraded":"2024-09-10T23:35:36.387662+0000","last_fullsized":"2024-09-10T23:35:36.387662+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405662+0000","last_change":"2024-09-10T23:35:36.398978+0000","last_active":"2024-09-10T23:35:36.405662+0000","last_peered":"2024-09-10T23:35:36.405662+0000","last_clean":"2024-09-10T23:35:36.405662+0000","last_became_active":"2024-09-10T23:35:36.398751+0000","last_became_peered":"2024-09-10T23:35:36.398751+0000","last_unstale":"2024-09-10T23:35:36.405662+0000","last_undegraded":"2024-09-10T23:35:36.405662+0000","last_fullsized":"2024-09-10T23:35:36.405662+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:24.457146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.387645+0000","last_change":"2024-09-10T23:35:36.387645+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.387645+0000","last_undegraded":"2024-09-10T23:35:36.387645+0000","last_fullsized":"2024-09-10T23:35:36.387645+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407690+0000","last_change":"2024-09-10T23:35:36.403531+0000","last_active":"2024-09-10T23:35:36.407690+0000","last_peered":"2024-09-10T23:35:36.407690+0000","last_clean":"2024-09-10T23:35:36.407690+0000","last_became_active":"2024-09-10T23:35:36.403263+0000","last_became_peered":"2024-09-10T23:35:36.403263+0000","last_unstale":"2024-09-10T23:35:36.407690+0000","last_undegraded":"2024-09-10T23:35:36.407690+0000","last_fullsized":"2024-09-10T23:35:36.407690+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:34.124846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":173,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.390482+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:35:36.390482+0000","last_peered":"2024-09-10T23:35:36.390482+0000","last_clean":"2024-09-10T23:35:36.390482+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:35:36.390482+0000","last_undegraded":"2024-09-10T23:35:36.390482+0000","last_fullsized":"2024-09-10T23:35:36.390482+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":651264,"data_stored":264592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28244,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343596,"statfs":{"total":96636764160,"available":96607842304,"internally_reserved":0,"allocated":1245184,"data_stored":854960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":264592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":270867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":264592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":854960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":270867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":692224,"data_stored":283964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:35:41.890 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:35:42.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.070+0000 7f5765000640 1 -- 172.21.15.94:0/2882805598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5744004d50 msgr2=0x7f57440051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:42.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.070+0000 7f5765000640 1 --2- 172.21.15.94:0/2882805598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5744004d50 0x7f57440051d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f575800abf0 tx=0x7f575801cd90 comp rx=0 tx=0).stop 2024-09-10T23:35:42.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.070+0000 7f575e800640 1 -- 172.21.15.94:0/2882805598 reap_dead start 2024-09-10T23:35:42.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.070+0000 7f5765000640 1 -- 172.21.15.94:0/2882805598 shutdown_connections 2024-09-10T23:35:42.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.070+0000 7f5765000640 1 -- 172.21.15.94:0/2882805598 >> 172.21.15.94:0/2882805598 conn(0x7f576006cdf0 msgr2=0x7f576006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:42.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.070+0000 7f5765000640 1 -- 172.21.15.94:0/2882805598 shutdown_connections 2024-09-10T23:35:42.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.070+0000 7f5765000640 1 -- 172.21.15.94:0/2882805598 wait complete. 2024-09-10T23:35:42.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.070+0000 7f5765000640 1 Processor -- start 2024-09-10T23:35:42.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.070+0000 7f5765000640 1 -- start start 2024-09-10T23:35:42.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f5765000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5744004d50 0x7f5760145710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:42.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f5765000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57600c8d50 0x7f576015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:42.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f5765000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57601ea2f0 0x7f57601ea770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:42.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f5765000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f57600b35c0 con 0x7f5744004d50 2024-09-10T23:35:42.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f5765000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5760072a50 con 0x7f57601ea2f0 2024-09-10T23:35:42.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f5765000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f57600b3740 con 0x7f57600c8d50 2024-09-10T23:35:42.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f575de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5744004d50 0x7f5760145710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:42.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f575de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5744004d50 0x7f5760145710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42578/0 (socket says 172.21.15.94:42578) 2024-09-10T23:35:42.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f575de00640 1 -- 172.21.15.94:0/791037633 learned_addr learned my addr 172.21.15.94:0/791037633 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:42.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f575e800640 1 --2- 172.21.15.94:0/791037633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57601ea2f0 0x7f57601ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:42.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f5755400640 1 --2- 172.21.15.94:0/791037633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57600c8d50 0x7f576015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:42.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f575de00640 1 -- 172.21.15.94:0/791037633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57600c8d50 msgr2=0x7f576015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:42.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f575de00640 1 --2- 172.21.15.94:0/791037633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57600c8d50 0x7f576015d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f575de00640 1 -- 172.21.15.94:0/791037633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57601ea2f0 msgr2=0x7f57601ea770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:42.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f575de00640 1 --2- 172.21.15.94:0/791037633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57601ea2f0 0x7f57601ea770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f575de00640 1 -- 172.21.15.94:0/791037633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57600c92a0 con 0x7f5744004d50 2024-09-10T23:35:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f5755400640 1 --2- 172.21.15.94:0/791037633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57600c8d50 0x7f576015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:35:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f575e800640 1 --2- 172.21.15.94:0/791037633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57601ea2f0 0x7f57601ea770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:35:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f575de00640 1 --2- 172.21.15.94:0/791037633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5744004d50 0x7f5760145710 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f57500063c0 tx=0x7f5750006880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f5756a00640 1 -- 172.21.15.94:0/791037633 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5750003b90 con 0x7f5744004d50 2024-09-10T23:35:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f5756a00640 1 -- 172.21.15.94:0/791037633 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5750018020 con 0x7f5744004d50 2024-09-10T23:35:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f5756a00640 1 -- 172.21.15.94:0/791037633 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57500119b0 con 0x7f5744004d50 2024-09-10T23:35:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f5765000640 1 -- 172.21.15.94:0/791037633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57600c9580 con 0x7f5744004d50 2024-09-10T23:35:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f5765000640 1 -- 172.21.15.94:0/791037633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57600c9b40 con 0x7f5744004d50 2024-09-10T23:35:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f5765000640 1 -- 172.21.15.94:0/791037633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f576006e7e0 con 0x7f5744004d50 2024-09-10T23:35:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.074+0000 7f5756a00640 1 -- 172.21.15.94:0/791037633 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f5750010040 con 0x7f5744004d50 2024-09-10T23:35:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.078+0000 7f5756a00640 1 --2- 172.21.15.94:0/791037633 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f573c0727c0 0x7f573c074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.078+0000 7f5756a00640 1 -- 172.21.15.94:0/791037633 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f57500973a0 con 0x7f5744004d50 2024-09-10T23:35:42.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.078+0000 7f5755400640 1 --2- 172.21.15.94:0/791037633 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f573c0727c0 0x7f573c074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:42.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.078+0000 7f5755400640 1 --2- 172.21.15.94:0/791037633 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f573c0727c0 0x7f573c074c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5748001890 tx=0x7f5748006730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:42.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.082+0000 7f5756a00640 1 -- 172.21.15.94:0/791037633 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5750022020 con 0x7f5744004d50 2024-09-10T23:35:42.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.246+0000 7f5765000640 1 -- 172.21.15.94:0/791037633 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5760073660 con 0x7f573c0727c0 2024-09-10T23:35:42.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.250+0000 7f5756a00640 1 -- 172.21.15.94:0/791037633 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247559 (secure 0 0 0) 0x7f5760073660 con 0x7f573c0727c0 2024-09-10T23:35:42.255 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:42.256 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:35:42.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.254+0000 7f5765000640 1 -- 172.21.15.94:0/791037633 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f573c0727c0 msgr2=0x7f573c074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:42.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.254+0000 7f5765000640 1 --2- 172.21.15.94:0/791037633 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f573c0727c0 0x7f573c074c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5748001890 tx=0x7f5748006730 comp rx=0 tx=0).stop 2024-09-10T23:35:42.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.254+0000 7f5765000640 1 -- 172.21.15.94:0/791037633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5744004d50 msgr2=0x7f5760145710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:42.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.254+0000 7f5765000640 1 --2- 172.21.15.94:0/791037633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5744004d50 0x7f5760145710 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f57500063c0 tx=0x7f5750006880 comp rx=0 tx=0).stop 2024-09-10T23:35:42.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.254+0000 7f5765000640 1 -- 172.21.15.94:0/791037633 shutdown_connections 2024-09-10T23:35:42.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.254+0000 7f5765000640 1 --2- 172.21.15.94:0/791037633 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f573c0727c0 0x7f573c074c80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.254+0000 7f5765000640 1 --2- 172.21.15.94:0/791037633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57601ea2f0 0x7f57601ea770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.254+0000 7f5765000640 1 --2- 172.21.15.94:0/791037633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57600c8d50 0x7f576015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.254+0000 7f5765000640 1 --2- 172.21.15.94:0/791037633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5744004d50 0x7f5760145710 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.254+0000 7f5765000640 1 -- 172.21.15.94:0/791037633 >> 172.21.15.94:0/791037633 conn(0x7f576006cdf0 msgr2=0x7f57601560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:42.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.254+0000 7f5765000640 1 -- 172.21.15.94:0/791037633 shutdown_connections 2024-09-10T23:35:42.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.254+0000 7f5765000640 1 -- 172.21.15.94:0/791037633 wait complete. 2024-09-10T23:35:42.275 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-10T23:35:41.388360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247508,"kb_used_data":6312,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727212,"statfs":{"total":773094113280,"available":772840665088,"internally_reserved":0,"allocated":6463488,"data_stored":3329394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001510"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391197+0000","last_change":"2024-09-10T23:35:37.411838+0000","last_active":"2024-09-10T23:35:38.391197+0000","last_peered":"2024-09-10T23:35:38.391197+0000","last_clean":"2024-09-10T23:35:38.391197+0000","last_became_active":"2024-09-10T23:35:37.411168+0000","last_became_peered":"2024-09-10T23:35:37.411168+0000","last_unstale":"2024-09-10T23:35:38.391197+0000","last_undegraded":"2024-09-10T23:35:38.391197+0000","last_fullsized":"2024-09-10T23:35:38.391197+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:27.635555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388506+0000","last_change":"2024-09-10T23:35:36.388506+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388506+0000","last_undegraded":"2024-09-10T23:35:36.388506+0000","last_fullsized":"2024-09-10T23:35:36.388506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.390335+0000","last_change":"2024-09-10T23:35:36.390335+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.390335+0000","last_undegraded":"2024-09-10T23:35:36.390335+0000","last_fullsized":"2024-09-10T23:35:36.390335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392121+0000","last_change":"2024-09-10T23:35:36.392121+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392121+0000","last_undegraded":"2024-09-10T23:35:36.392121+0000","last_fullsized":"2024-09-10T23:35:36.392121+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390631+0000","last_change":"2024-09-10T23:35:37.422878+0000","last_active":"2024-09-10T23:35:38.390631+0000","last_peered":"2024-09-10T23:35:38.390631+0000","last_clean":"2024-09-10T23:35:38.390631+0000","last_became_active":"2024-09-10T23:35:37.422602+0000","last_became_peered":"2024-09-10T23:35:37.422602+0000","last_unstale":"2024-09-10T23:35:38.390631+0000","last_undegraded":"2024-09-10T23:35:38.390631+0000","last_fullsized":"2024-09-10T23:35:38.390631+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:17.965755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390045+0000","last_change":"2024-09-10T23:35:37.401854+0000","last_active":"2024-09-10T23:35:38.390045+0000","last_peered":"2024-09-10T23:35:38.390045+0000","last_clean":"2024-09-10T23:35:38.390045+0000","last_became_active":"2024-09-10T23:35:37.400521+0000","last_became_peered":"2024-09-10T23:35:37.400521+0000","last_unstale":"2024-09-10T23:35:38.390045+0000","last_undegraded":"2024-09-10T23:35:38.390045+0000","last_fullsized":"2024-09-10T23:35:38.390045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:48.537421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394892+0000","last_change":"2024-09-10T23:35:36.394892+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394892+0000","last_undegraded":"2024-09-10T23:35:36.394892+0000","last_fullsized":"2024-09-10T23:35:36.394892+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389814+0000","last_change":"2024-09-10T23:35:36.389814+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389814+0000","last_undegraded":"2024-09-10T23:35:36.389814+0000","last_fullsized":"2024-09-10T23:35:36.389814+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389006+0000","last_change":"2024-09-10T23:35:36.389006+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389006+0000","last_undegraded":"2024-09-10T23:35:36.389006+0000","last_fullsized":"2024-09-10T23:35:36.389006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390598+0000","last_change":"2024-09-10T23:35:37.405331+0000","last_active":"2024-09-10T23:35:38.390598+0000","last_peered":"2024-09-10T23:35:38.390598+0000","last_clean":"2024-09-10T23:35:38.390598+0000","last_became_active":"2024-09-10T23:35:37.404010+0000","last_became_peered":"2024-09-10T23:35:37.404010+0000","last_unstale":"2024-09-10T23:35:38.390598+0000","last_undegraded":"2024-09-10T23:35:38.390598+0000","last_fullsized":"2024-09-10T23:35:38.390598+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:30.787726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394567+0000","last_change":"2024-09-10T23:35:36.394567+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394567+0000","last_undegraded":"2024-09-10T23:35:36.394567+0000","last_fullsized":"2024-09-10T23:35:36.394567+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391006+0000","last_change":"2024-09-10T23:35:36.391006+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391006+0000","last_undegraded":"2024-09-10T23:35:36.391006+0000","last_fullsized":"2024-09-10T23:35:36.391006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.397258+0000","last_change":"2024-09-10T23:35:36.397258+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.397258+0000","last_undegraded":"2024-09-10T23:35:36.397258+0000","last_fullsized":"2024-09-10T23:35:36.397258+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389045+0000","last_change":"2024-09-10T23:35:36.389045+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389045+0000","last_undegraded":"2024-09-10T23:35:36.389045+0000","last_fullsized":"2024-09-10T23:35:36.389045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394449+0000","last_change":"2024-09-10T23:35:36.394449+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394449+0000","last_undegraded":"2024-09-10T23:35:36.394449+0000","last_fullsized":"2024-09-10T23:35:36.394449+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389559+0000","last_change":"2024-09-10T23:35:37.421193+0000","last_active":"2024-09-10T23:35:38.389559+0000","last_peered":"2024-09-10T23:35:38.389559+0000","last_clean":"2024-09-10T23:35:38.389559+0000","last_became_active":"2024-09-10T23:35:37.420909+0000","last_became_peered":"2024-09-10T23:35:37.420909+0000","last_unstale":"2024-09-10T23:35:38.389559+0000","last_undegraded":"2024-09-10T23:35:38.389559+0000","last_fullsized":"2024-09-10T23:35:38.389559+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:31.257242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391217+0000","last_change":"2024-09-10T23:35:37.404324+0000","last_active":"2024-09-10T23:35:38.391217+0000","last_peered":"2024-09-10T23:35:38.391217+0000","last_clean":"2024-09-10T23:35:38.391217+0000","last_became_active":"2024-09-10T23:35:37.404042+0000","last_became_peered":"2024-09-10T23:35:37.404042+0000","last_unstale":"2024-09-10T23:35:38.391217+0000","last_undegraded":"2024-09-10T23:35:38.391217+0000","last_fullsized":"2024-09-10T23:35:38.391217+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:31.250579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392062+0000","last_change":"2024-09-10T23:35:36.392062+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392062+0000","last_undegraded":"2024-09-10T23:35:36.392062+0000","last_fullsized":"2024-09-10T23:35:36.392062+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389708+0000","last_change":"2024-09-10T23:35:36.389708+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389708+0000","last_undegraded":"2024-09-10T23:35:36.389708+0000","last_fullsized":"2024-09-10T23:35:36.389708+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394443+0000","last_change":"2024-09-10T23:35:36.394443+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394443+0000","last_undegraded":"2024-09-10T23:35:36.394443+0000","last_fullsized":"2024-09-10T23:35:36.394443+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.396891+0000","last_change":"2024-09-10T23:35:36.396891+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.396891+0000","last_undegraded":"2024-09-10T23:35:36.396891+0000","last_fullsized":"2024-09-10T23:35:36.396891+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.396024+0000","last_change":"2024-09-10T23:35:36.396024+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.396024+0000","last_undegraded":"2024-09-10T23:35:36.396024+0000","last_fullsized":"2024-09-10T23:35:36.396024+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392968+0000","last_change":"2024-09-10T23:35:36.392968+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392968+0000","last_undegraded":"2024-09-10T23:35:36.392968+0000","last_fullsized":"2024-09-10T23:35:36.392968+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.390266+0000","last_change":"2024-09-10T23:35:36.390266+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.390266+0000","last_undegraded":"2024-09-10T23:35:36.390266+0000","last_fullsized":"2024-09-10T23:35:36.390266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389015+0000","last_change":"2024-09-10T23:35:36.389015+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389015+0000","last_undegraded":"2024-09-10T23:35:36.389015+0000","last_fullsized":"2024-09-10T23:35:36.389015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.395481+0000","last_change":"2024-09-10T23:35:36.395481+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.395481+0000","last_undegraded":"2024-09-10T23:35:36.395481+0000","last_fullsized":"2024-09-10T23:35:36.395481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389120+0000","last_change":"2024-09-10T23:35:36.389120+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389120+0000","last_undegraded":"2024-09-10T23:35:36.389120+0000","last_fullsized":"2024-09-10T23:35:36.389120+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392340+0000","last_change":"2024-09-10T23:35:36.392340+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392340+0000","last_undegraded":"2024-09-10T23:35:36.392340+0000","last_fullsized":"2024-09-10T23:35:36.392340+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388535+0000","last_change":"2024-09-10T23:35:36.388535+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388535+0000","last_undegraded":"2024-09-10T23:35:36.388535+0000","last_fullsized":"2024-09-10T23:35:36.388535+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391729+0000","last_change":"2024-09-10T23:35:36.391729+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391729+0000","last_undegraded":"2024-09-10T23:35:36.391729+0000","last_fullsized":"2024-09-10T23:35:36.391729+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388251+0000","last_change":"2024-09-10T23:35:36.388251+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388251+0000","last_undegraded":"2024-09-10T23:35:36.388251+0000","last_fullsized":"2024-09-10T23:35:36.388251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394059+0000","last_change":"2024-09-10T23:35:36.394059+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394059+0000","last_undegraded":"2024-09-10T23:35:36.394059+0000","last_fullsized":"2024-09-10T23:35:36.394059+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391188+0000","last_change":"2024-09-10T23:35:37.405530+0000","last_active":"2024-09-10T23:35:38.391188+0000","last_peered":"2024-09-10T23:35:38.391188+0000","last_clean":"2024-09-10T23:35:38.391188+0000","last_became_active":"2024-09-10T23:35:37.402795+0000","last_became_peered":"2024-09-10T23:35:37.402795+0000","last_unstale":"2024-09-10T23:35:38.391188+0000","last_undegraded":"2024-09-10T23:35:38.391188+0000","last_fullsized":"2024-09-10T23:35:38.391188+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:15.871163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.404962+0000","last_change":"2024-09-10T23:35:36.401576+0000","last_active":"2024-09-10T23:35:36.404962+0000","last_peered":"2024-09-10T23:35:36.404962+0000","last_clean":"2024-09-10T23:35:36.404962+0000","last_became_active":"2024-09-10T23:35:36.400780+0000","last_became_peered":"2024-09-10T23:35:36.400780+0000","last_unstale":"2024-09-10T23:35:36.404962+0000","last_undegraded":"2024-09-10T23:35:36.404962+0000","last_fullsized":"2024-09-10T23:35:36.404962+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:15.011008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391401+0000","last_change":"2024-09-10T23:35:37.411543+0000","last_active":"2024-09-10T23:35:38.391401+0000","last_peered":"2024-09-10T23:35:38.391401+0000","last_clean":"2024-09-10T23:35:38.391401+0000","last_became_active":"2024-09-10T23:35:37.410807+0000","last_became_peered":"2024-09-10T23:35:37.410807+0000","last_unstale":"2024-09-10T23:35:38.391401+0000","last_undegraded":"2024-09-10T23:35:38.391401+0000","last_fullsized":"2024-09-10T23:35:38.391401+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:52.749748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.410357+0000","last_change":"2024-09-10T23:35:36.406745+0000","last_active":"2024-09-10T23:35:36.410357+0000","last_peered":"2024-09-10T23:35:36.410357+0000","last_clean":"2024-09-10T23:35:36.410357+0000","last_became_active":"2024-09-10T23:35:36.406460+0000","last_became_peered":"2024-09-10T23:35:36.406460+0000","last_unstale":"2024-09-10T23:35:36.410357+0000","last_undegraded":"2024-09-10T23:35:36.410357+0000","last_fullsized":"2024-09-10T23:35:36.410357+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:58.569452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391940+0000","last_change":"2024-09-10T23:35:36.391940+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391940+0000","last_undegraded":"2024-09-10T23:35:36.391940+0000","last_fullsized":"2024-09-10T23:35:36.391940+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390807+0000","last_change":"2024-09-10T23:35:36.409095+0000","last_active":"2024-09-10T23:35:38.390807+0000","last_peered":"2024-09-10T23:35:38.390807+0000","last_clean":"2024-09-10T23:35:38.390807+0000","last_became_active":"2024-09-10T23:35:36.408811+0000","last_became_peered":"2024-09-10T23:35:36.408811+0000","last_unstale":"2024-09-10T23:35:38.390807+0000","last_undegraded":"2024-09-10T23:35:38.390807+0000","last_fullsized":"2024-09-10T23:35:38.390807+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:47.477496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390766+0000","last_change":"2024-09-10T23:35:37.413035+0000","last_active":"2024-09-10T23:35:38.390766+0000","last_peered":"2024-09-10T23:35:38.390766+0000","last_clean":"2024-09-10T23:35:38.390766+0000","last_became_active":"2024-09-10T23:35:37.412781+0000","last_became_peered":"2024-09-10T23:35:37.412781+0000","last_unstale":"2024-09-10T23:35:38.390766+0000","last_undegraded":"2024-09-10T23:35:38.390766+0000","last_fullsized":"2024-09-10T23:35:38.390766+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:11.135478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.404617+0000","last_change":"2024-09-10T23:35:36.396384+0000","last_active":"2024-09-10T23:35:36.404617+0000","last_peered":"2024-09-10T23:35:36.404617+0000","last_clean":"2024-09-10T23:35:36.404617+0000","last_became_active":"2024-09-10T23:35:36.396085+0000","last_became_peered":"2024-09-10T23:35:36.396085+0000","last_unstale":"2024-09-10T23:35:36.404617+0000","last_undegraded":"2024-09-10T23:35:36.404617+0000","last_fullsized":"2024-09-10T23:35:36.404617+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:34.607437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.387665+0000","last_change":"2024-09-10T23:35:36.387665+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.387665+0000","last_undegraded":"2024-09-10T23:35:36.387665+0000","last_fullsized":"2024-09-10T23:35:36.387665+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407850+0000","last_change":"2024-09-10T23:35:36.404943+0000","last_active":"2024-09-10T23:35:36.407850+0000","last_peered":"2024-09-10T23:35:36.407850+0000","last_clean":"2024-09-10T23:35:36.407850+0000","last_became_active":"2024-09-10T23:35:36.404615+0000","last_became_peered":"2024-09-10T23:35:36.404615+0000","last_unstale":"2024-09-10T23:35:36.407850+0000","last_undegraded":"2024-09-10T23:35:36.407850+0000","last_fullsized":"2024-09-10T23:35:36.407850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:58.666819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.396962+0000","last_change":"2024-09-10T23:35:36.396962+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.396962+0000","last_undegraded":"2024-09-10T23:35:36.396962+0000","last_fullsized":"2024-09-10T23:35:36.396962+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390698+0000","last_change":"2024-09-10T23:35:36.408437+0000","last_active":"2024-09-10T23:35:38.390698+0000","last_peered":"2024-09-10T23:35:38.390698+0000","last_clean":"2024-09-10T23:35:38.390698+0000","last_became_active":"2024-09-10T23:35:36.408156+0000","last_became_peered":"2024-09-10T23:35:36.408156+0000","last_unstale":"2024-09-10T23:35:38.390698+0000","last_undegraded":"2024-09-10T23:35:38.390698+0000","last_fullsized":"2024-09-10T23:35:38.390698+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:01.998360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389936+0000","last_change":"2024-09-10T23:35:36.389936+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389936+0000","last_undegraded":"2024-09-10T23:35:36.389936+0000","last_fullsized":"2024-09-10T23:35:36.389936+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407639+0000","last_change":"2024-09-10T23:35:36.401028+0000","last_active":"2024-09-10T23:35:36.407639+0000","last_peered":"2024-09-10T23:35:36.407639+0000","last_clean":"2024-09-10T23:35:36.407639+0000","last_became_active":"2024-09-10T23:35:36.400743+0000","last_became_peered":"2024-09-10T23:35:36.400743+0000","last_unstale":"2024-09-10T23:35:36.407639+0000","last_undegraded":"2024-09-10T23:35:36.407639+0000","last_fullsized":"2024-09-10T23:35:36.407639+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:39:33.252954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389654+0000","last_change":"2024-09-10T23:35:37.422945+0000","last_active":"2024-09-10T23:35:38.389654+0000","last_peered":"2024-09-10T23:35:38.389654+0000","last_clean":"2024-09-10T23:35:38.389654+0000","last_became_active":"2024-09-10T23:35:37.422648+0000","last_became_peered":"2024-09-10T23:35:37.422648+0000","last_unstale":"2024-09-10T23:35:38.389654+0000","last_undegraded":"2024-09-10T23:35:38.389654+0000","last_fullsized":"2024-09-10T23:35:38.389654+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:10.705976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406280+0000","last_change":"2024-09-10T23:35:36.401434+0000","last_active":"2024-09-10T23:35:36.406280+0000","last_peered":"2024-09-10T23:35:36.406280+0000","last_clean":"2024-09-10T23:35:36.406280+0000","last_became_active":"2024-09-10T23:35:36.401155+0000","last_became_peered":"2024-09-10T23:35:36.401155+0000","last_unstale":"2024-09-10T23:35:36.406280+0000","last_undegraded":"2024-09-10T23:35:36.406280+0000","last_fullsized":"2024-09-10T23:35:36.406280+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:37.536960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394992+0000","last_change":"2024-09-10T23:35:36.394992+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394992+0000","last_undegraded":"2024-09-10T23:35:36.394992+0000","last_fullsized":"2024-09-10T23:35:36.394992+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407810+0000","last_change":"2024-09-10T23:35:36.405013+0000","last_active":"2024-09-10T23:35:36.407810+0000","last_peered":"2024-09-10T23:35:36.407810+0000","last_clean":"2024-09-10T23:35:36.407810+0000","last_became_active":"2024-09-10T23:35:36.404715+0000","last_became_peered":"2024-09-10T23:35:36.404715+0000","last_unstale":"2024-09-10T23:35:36.407810+0000","last_undegraded":"2024-09-10T23:35:36.407810+0000","last_fullsized":"2024-09-10T23:35:36.407810+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:11:46.791009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394500+0000","last_change":"2024-09-10T23:35:36.394500+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394500+0000","last_undegraded":"2024-09-10T23:35:36.394500+0000","last_fullsized":"2024-09-10T23:35:36.394500+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407797+0000","last_change":"2024-09-10T23:35:36.402529+0000","last_active":"2024-09-10T23:35:36.407797+0000","last_peered":"2024-09-10T23:35:36.407797+0000","last_clean":"2024-09-10T23:35:36.407797+0000","last_became_active":"2024-09-10T23:35:36.402270+0000","last_became_peered":"2024-09-10T23:35:36.402270+0000","last_unstale":"2024-09-10T23:35:36.407797+0000","last_undegraded":"2024-09-10T23:35:36.407797+0000","last_fullsized":"2024-09-10T23:35:36.407797+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:28.809096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389483+0000","last_change":"2024-09-10T23:35:36.389483+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389483+0000","last_undegraded":"2024-09-10T23:35:36.389483+0000","last_fullsized":"2024-09-10T23:35:36.389483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406578+0000","last_change":"2024-09-10T23:35:36.402631+0000","last_active":"2024-09-10T23:35:36.406578+0000","last_peered":"2024-09-10T23:35:36.406578+0000","last_clean":"2024-09-10T23:35:36.406578+0000","last_became_active":"2024-09-10T23:35:36.399947+0000","last_became_peered":"2024-09-10T23:35:36.399947+0000","last_unstale":"2024-09-10T23:35:36.406578+0000","last_undegraded":"2024-09-10T23:35:36.406578+0000","last_fullsized":"2024-09-10T23:35:36.406578+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:53.498796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391900+0000","last_change":"2024-09-10T23:35:36.391900+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391900+0000","last_undegraded":"2024-09-10T23:35:36.391900+0000","last_fullsized":"2024-09-10T23:35:36.391900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.408350+0000","last_change":"2024-09-10T23:35:36.405711+0000","last_active":"2024-09-10T23:35:36.408350+0000","last_peered":"2024-09-10T23:35:36.408350+0000","last_clean":"2024-09-10T23:35:36.408350+0000","last_became_active":"2024-09-10T23:35:36.405455+0000","last_became_peered":"2024-09-10T23:35:36.405455+0000","last_unstale":"2024-09-10T23:35:36.408350+0000","last_undegraded":"2024-09-10T23:35:36.408350+0000","last_fullsized":"2024-09-10T23:35:36.408350+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:56.422393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392350+0000","last_change":"2024-09-10T23:35:36.392350+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392350+0000","last_undegraded":"2024-09-10T23:35:36.392350+0000","last_fullsized":"2024-09-10T23:35:36.392350+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407863+0000","last_change":"2024-09-10T23:35:36.404848+0000","last_active":"2024-09-10T23:35:36.407863+0000","last_peered":"2024-09-10T23:35:36.407863+0000","last_clean":"2024-09-10T23:35:36.407863+0000","last_became_active":"2024-09-10T23:35:36.404567+0000","last_became_peered":"2024-09-10T23:35:36.404567+0000","last_unstale":"2024-09-10T23:35:36.407863+0000","last_undegraded":"2024-09-10T23:35:36.407863+0000","last_fullsized":"2024-09-10T23:35:36.407863+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:33.274868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388987+0000","last_change":"2024-09-10T23:35:36.388987+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388987+0000","last_undegraded":"2024-09-10T23:35:36.388987+0000","last_fullsized":"2024-09-10T23:35:36.388987+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407454+0000","last_change":"2024-09-10T23:35:36.402546+0000","last_active":"2024-09-10T23:35:36.407454+0000","last_peered":"2024-09-10T23:35:36.407454+0000","last_clean":"2024-09-10T23:35:36.407454+0000","last_became_active":"2024-09-10T23:35:36.402187+0000","last_became_peered":"2024-09-10T23:35:36.402187+0000","last_unstale":"2024-09-10T23:35:36.407454+0000","last_undegraded":"2024-09-10T23:35:36.407454+0000","last_fullsized":"2024-09-10T23:35:36.407454+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:18.611164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389852+0000","last_change":"2024-09-10T23:35:36.389852+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389852+0000","last_undegraded":"2024-09-10T23:35:36.389852+0000","last_fullsized":"2024-09-10T23:35:36.389852+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406227+0000","last_change":"2024-09-10T23:35:36.401288+0000","last_active":"2024-09-10T23:35:36.406227+0000","last_peered":"2024-09-10T23:35:36.406227+0000","last_clean":"2024-09-10T23:35:36.406227+0000","last_became_active":"2024-09-10T23:35:36.400889+0000","last_became_peered":"2024-09-10T23:35:36.400889+0000","last_unstale":"2024-09-10T23:35:36.406227+0000","last_undegraded":"2024-09-10T23:35:36.406227+0000","last_fullsized":"2024-09-10T23:35:36.406227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:05.132993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.390490+0000","last_change":"2024-09-10T23:35:36.390490+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.390490+0000","last_undegraded":"2024-09-10T23:35:36.390490+0000","last_fullsized":"2024-09-10T23:35:36.390490+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406140+0000","last_change":"2024-09-10T23:35:36.398356+0000","last_active":"2024-09-10T23:35:36.406140+0000","last_peered":"2024-09-10T23:35:36.406140+0000","last_clean":"2024-09-10T23:35:36.406140+0000","last_became_active":"2024-09-10T23:35:36.398077+0000","last_became_peered":"2024-09-10T23:35:36.398077+0000","last_unstale":"2024-09-10T23:35:36.406140+0000","last_undegraded":"2024-09-10T23:35:36.406140+0000","last_fullsized":"2024-09-10T23:35:36.406140+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:11.718995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391693+0000","last_change":"2024-09-10T23:35:36.391693+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391693+0000","last_undegraded":"2024-09-10T23:35:36.391693+0000","last_fullsized":"2024-09-10T23:35:36.391693+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.411637+0000","last_change":"2024-09-10T23:35:36.409579+0000","last_active":"2024-09-10T23:35:36.411637+0000","last_peered":"2024-09-10T23:35:36.411637+0000","last_clean":"2024-09-10T23:35:36.411637+0000","last_became_active":"2024-09-10T23:35:36.409321+0000","last_became_peered":"2024-09-10T23:35:36.409321+0000","last_unstale":"2024-09-10T23:35:36.411637+0000","last_undegraded":"2024-09-10T23:35:36.411637+0000","last_fullsized":"2024-09-10T23:35:36.411637+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:19.816523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394047+0000","last_change":"2024-09-10T23:35:36.394047+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394047+0000","last_undegraded":"2024-09-10T23:35:36.394047+0000","last_fullsized":"2024-09-10T23:35:36.394047+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405719+0000","last_change":"2024-09-10T23:35:36.400916+0000","last_active":"2024-09-10T23:35:36.405719+0000","last_peered":"2024-09-10T23:35:36.405719+0000","last_clean":"2024-09-10T23:35:36.405719+0000","last_became_active":"2024-09-10T23:35:36.400639+0000","last_became_peered":"2024-09-10T23:35:36.400639+0000","last_unstale":"2024-09-10T23:35:36.405719+0000","last_undegraded":"2024-09-10T23:35:36.405719+0000","last_fullsized":"2024-09-10T23:35:36.405719+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:47.327210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389265+0000","last_change":"2024-09-10T23:35:36.389265+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389265+0000","last_undegraded":"2024-09-10T23:35:36.389265+0000","last_fullsized":"2024-09-10T23:35:36.389265+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405406+0000","last_change":"2024-09-10T23:35:36.398766+0000","last_active":"2024-09-10T23:35:36.405406+0000","last_peered":"2024-09-10T23:35:36.405406+0000","last_clean":"2024-09-10T23:35:36.405406+0000","last_became_active":"2024-09-10T23:35:36.398473+0000","last_became_peered":"2024-09-10T23:35:36.398473+0000","last_unstale":"2024-09-10T23:35:36.405406+0000","last_undegraded":"2024-09-10T23:35:36.405406+0000","last_fullsized":"2024-09-10T23:35:36.405406+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:37.536960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388202+0000","last_change":"2024-09-10T23:35:36.388202+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388202+0000","last_undegraded":"2024-09-10T23:35:36.388202+0000","last_fullsized":"2024-09-10T23:35:36.388202+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.414301+0000","last_change":"2024-09-10T23:35:36.409772+0000","last_active":"2024-09-10T23:35:36.414301+0000","last_peered":"2024-09-10T23:35:36.414301+0000","last_clean":"2024-09-10T23:35:36.414301+0000","last_became_active":"2024-09-10T23:35:36.409432+0000","last_became_peered":"2024-09-10T23:35:36.409432+0000","last_unstale":"2024-09-10T23:35:36.414301+0000","last_undegraded":"2024-09-10T23:35:36.414301+0000","last_fullsized":"2024-09-10T23:35:36.414301+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:09.775738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391962+0000","last_change":"2024-09-10T23:35:36.391962+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391962+0000","last_undegraded":"2024-09-10T23:35:36.391962+0000","last_fullsized":"2024-09-10T23:35:36.391962+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.409418+0000","last_change":"2024-09-10T23:35:36.405162+0000","last_active":"2024-09-10T23:35:36.409418+0000","last_peered":"2024-09-10T23:35:36.409418+0000","last_clean":"2024-09-10T23:35:36.409418+0000","last_became_active":"2024-09-10T23:35:36.404891+0000","last_became_peered":"2024-09-10T23:35:36.404891+0000","last_unstale":"2024-09-10T23:35:36.409418+0000","last_undegraded":"2024-09-10T23:35:36.409418+0000","last_fullsized":"2024-09-10T23:35:36.409418+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:25.179787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388064+0000","last_change":"2024-09-10T23:35:36.388064+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388064+0000","last_undegraded":"2024-09-10T23:35:36.388064+0000","last_fullsized":"2024-09-10T23:35:36.388064+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406216+0000","last_change":"2024-09-10T23:35:36.401149+0000","last_active":"2024-09-10T23:35:36.406216+0000","last_peered":"2024-09-10T23:35:36.406216+0000","last_clean":"2024-09-10T23:35:36.406216+0000","last_became_active":"2024-09-10T23:35:36.400884+0000","last_became_peered":"2024-09-10T23:35:36.400884+0000","last_unstale":"2024-09-10T23:35:36.406216+0000","last_undegraded":"2024-09-10T23:35:36.406216+0000","last_fullsized":"2024-09-10T23:35:36.406216+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:43.580364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388430+0000","last_change":"2024-09-10T23:35:36.388430+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388430+0000","last_undegraded":"2024-09-10T23:35:36.388430+0000","last_fullsized":"2024-09-10T23:35:36.388430+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407985+0000","last_change":"2024-09-10T23:35:36.406290+0000","last_active":"2024-09-10T23:35:36.407985+0000","last_peered":"2024-09-10T23:35:36.407985+0000","last_clean":"2024-09-10T23:35:36.407985+0000","last_became_active":"2024-09-10T23:35:36.406056+0000","last_became_peered":"2024-09-10T23:35:36.406056+0000","last_unstale":"2024-09-10T23:35:36.407985+0000","last_undegraded":"2024-09-10T23:35:36.407985+0000","last_fullsized":"2024-09-10T23:35:36.407985+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:59.329841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391377+0000","last_change":"2024-09-10T23:35:36.391377+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391377+0000","last_undegraded":"2024-09-10T23:35:36.391377+0000","last_fullsized":"2024-09-10T23:35:36.391377+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.410432+0000","last_change":"2024-09-10T23:35:36.406893+0000","last_active":"2024-09-10T23:35:36.410432+0000","last_peered":"2024-09-10T23:35:36.410432+0000","last_clean":"2024-09-10T23:35:36.410432+0000","last_became_active":"2024-09-10T23:35:36.406516+0000","last_became_peered":"2024-09-10T23:35:36.406516+0000","last_unstale":"2024-09-10T23:35:36.410432+0000","last_undegraded":"2024-09-10T23:35:36.410432+0000","last_fullsized":"2024-09-10T23:35:36.410432+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:13.080452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.393643+0000","last_change":"2024-09-10T23:35:36.393643+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.393643+0000","last_undegraded":"2024-09-10T23:35:36.393643+0000","last_fullsized":"2024-09-10T23:35:36.393643+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391082+0000","last_change":"2024-09-10T23:35:36.409344+0000","last_active":"2024-09-10T23:35:38.391082+0000","last_peered":"2024-09-10T23:35:38.391082+0000","last_clean":"2024-09-10T23:35:38.391082+0000","last_became_active":"2024-09-10T23:35:36.408975+0000","last_became_peered":"2024-09-10T23:35:36.408975+0000","last_unstale":"2024-09-10T23:35:38.391082+0000","last_undegraded":"2024-09-10T23:35:38.391082+0000","last_fullsized":"2024-09-10T23:35:38.391082+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:00.834693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388544+0000","last_change":"2024-09-10T23:35:36.388544+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388544+0000","last_undegraded":"2024-09-10T23:35:36.388544+0000","last_fullsized":"2024-09-10T23:35:36.388544+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405535+0000","last_change":"2024-09-10T23:35:36.400467+0000","last_active":"2024-09-10T23:35:36.405535+0000","last_peered":"2024-09-10T23:35:36.405535+0000","last_clean":"2024-09-10T23:35:36.405535+0000","last_became_active":"2024-09-10T23:35:36.400164+0000","last_became_peered":"2024-09-10T23:35:36.400164+0000","last_unstale":"2024-09-10T23:35:36.405535+0000","last_undegraded":"2024-09-10T23:35:36.405535+0000","last_fullsized":"2024-09-10T23:35:36.405535+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:57.102709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391111+0000","last_change":"2024-09-10T23:35:36.391111+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391111+0000","last_undegraded":"2024-09-10T23:35:36.391111+0000","last_fullsized":"2024-09-10T23:35:36.391111+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407753+0000","last_change":"2024-09-10T23:35:36.399341+0000","last_active":"2024-09-10T23:35:36.407753+0000","last_peered":"2024-09-10T23:35:36.407753+0000","last_clean":"2024-09-10T23:35:36.407753+0000","last_became_active":"2024-09-10T23:35:36.399064+0000","last_became_peered":"2024-09-10T23:35:36.399064+0000","last_unstale":"2024-09-10T23:35:36.407753+0000","last_undegraded":"2024-09-10T23:35:36.407753+0000","last_fullsized":"2024-09-10T23:35:36.407753+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:34.124846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390616+0000","last_change":"2024-09-10T23:35:37.403404+0000","last_active":"2024-09-10T23:35:38.390616+0000","last_peered":"2024-09-10T23:35:38.390616+0000","last_clean":"2024-09-10T23:35:38.390616+0000","last_became_active":"2024-09-10T23:35:37.400175+0000","last_became_peered":"2024-09-10T23:35:37.400175+0000","last_unstale":"2024-09-10T23:35:38.390616+0000","last_undegraded":"2024-09-10T23:35:38.390616+0000","last_fullsized":"2024-09-10T23:35:38.390616+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:54.308424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.414180+0000","last_change":"2024-09-10T23:35:36.410996+0000","last_active":"2024-09-10T23:35:36.414180+0000","last_peered":"2024-09-10T23:35:36.414180+0000","last_clean":"2024-09-10T23:35:36.414180+0000","last_became_active":"2024-09-10T23:35:36.410559+0000","last_became_peered":"2024-09-10T23:35:36.410559+0000","last_unstale":"2024-09-10T23:35:36.414180+0000","last_undegraded":"2024-09-10T23:35:36.414180+0000","last_fullsized":"2024-09-10T23:35:36.414180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:40.032192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.395931+0000","last_change":"2024-09-10T23:35:36.395931+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.395931+0000","last_undegraded":"2024-09-10T23:35:36.395931+0000","last_fullsized":"2024-09-10T23:35:36.395931+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405469+0000","last_change":"2024-09-10T23:35:36.399492+0000","last_active":"2024-09-10T23:35:36.405469+0000","last_peered":"2024-09-10T23:35:36.405469+0000","last_clean":"2024-09-10T23:35:36.405469+0000","last_became_active":"2024-09-10T23:35:36.397982+0000","last_became_peered":"2024-09-10T23:35:36.397982+0000","last_unstale":"2024-09-10T23:35:36.405469+0000","last_undegraded":"2024-09-10T23:35:36.405469+0000","last_fullsized":"2024-09-10T23:35:36.405469+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:56.422393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389339+0000","last_change":"2024-09-10T23:35:36.389339+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389339+0000","last_undegraded":"2024-09-10T23:35:36.389339+0000","last_fullsized":"2024-09-10T23:35:36.389339+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389803+0000","last_change":"2024-09-10T23:35:36.406500+0000","last_active":"2024-09-10T23:35:38.389803+0000","last_peered":"2024-09-10T23:35:38.389803+0000","last_clean":"2024-09-10T23:35:38.389803+0000","last_became_active":"2024-09-10T23:35:36.406222+0000","last_became_peered":"2024-09-10T23:35:36.406222+0000","last_unstale":"2024-09-10T23:35:38.389803+0000","last_undegraded":"2024-09-10T23:35:38.389803+0000","last_fullsized":"2024-09-10T23:35:38.389803+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:36.140228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.387662+0000","last_change":"2024-09-10T23:35:36.387662+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.387662+0000","last_undegraded":"2024-09-10T23:35:36.387662+0000","last_fullsized":"2024-09-10T23:35:36.387662+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405662+0000","last_change":"2024-09-10T23:35:36.398978+0000","last_active":"2024-09-10T23:35:36.405662+0000","last_peered":"2024-09-10T23:35:36.405662+0000","last_clean":"2024-09-10T23:35:36.405662+0000","last_became_active":"2024-09-10T23:35:36.398751+0000","last_became_peered":"2024-09-10T23:35:36.398751+0000","last_unstale":"2024-09-10T23:35:36.405662+0000","last_undegraded":"2024-09-10T23:35:36.405662+0000","last_fullsized":"2024-09-10T23:35:36.405662+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:24.457146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.387645+0000","last_change":"2024-09-10T23:35:36.387645+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.387645+0000","last_undegraded":"2024-09-10T23:35:36.387645+0000","last_fullsized":"2024-09-10T23:35:36.387645+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407690+0000","last_change":"2024-09-10T23:35:36.403531+0000","last_active":"2024-09-10T23:35:36.407690+0000","last_peered":"2024-09-10T23:35:36.407690+0000","last_clean":"2024-09-10T23:35:36.407690+0000","last_became_active":"2024-09-10T23:35:36.403263+0000","last_became_peered":"2024-09-10T23:35:36.403263+0000","last_unstale":"2024-09-10T23:35:36.407690+0000","last_undegraded":"2024-09-10T23:35:36.407690+0000","last_fullsized":"2024-09-10T23:35:36.407690+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:34.124846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":173,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.390482+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:35:36.390482+0000","last_peered":"2024-09-10T23:35:36.390482+0000","last_clean":"2024-09-10T23:35:36.390482+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:35:36.390482+0000","last_undegraded":"2024-09-10T23:35:36.390482+0000","last_fullsized":"2024-09-10T23:35:36.390482+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":651264,"data_stored":264592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28244,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343596,"statfs":{"total":96636764160,"available":96607842304,"internally_reserved":0,"allocated":1245184,"data_stored":854960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":264592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":270867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":264592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":854960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":270867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":692224,"data_stored":283964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:35:42.279 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3e is not active+clean 2024-09-10T23:35:42.279 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.388506+0000', 'last_change': '2024-09-10T23:35:36.388506+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.388506+0000', 'last_undegraded': '2024-09-10T23:35:36.388506+0000', 'last_fullsized': '2024-09-10T23:35:36.388506+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:35:42.279 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3d is not active+clean 2024-09-10T23:35:42.279 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.390335+0000', 'last_change': '2024-09-10T23:35:36.390335+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.390335+0000', 'last_undegraded': '2024-09-10T23:35:36.390335+0000', 'last_fullsized': '2024-09-10T23:35:36.390335+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T23:35:42.279 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3c is not active+clean 2024-09-10T23:35:42.279 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.392121+0000', 'last_change': '2024-09-10T23:35:36.392121+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.392121+0000', 'last_undegraded': '2024-09-10T23:35:36.392121+0000', 'last_fullsized': '2024-09-10T23:35:36.392121+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T23:35:42.279 INFO:tasks.ceph.ceph_manager.ceph:PG 9.39 is not active+clean 2024-09-10T23:35:42.279 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.394892+0000', 'last_change': '2024-09-10T23:35:36.394892+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.394892+0000', 'last_undegraded': '2024-09-10T23:35:36.394892+0000', 'last_fullsized': '2024-09-10T23:35:36.394892+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T23:35:42.279 INFO:tasks.ceph.ceph_manager.ceph:PG 9.38 is not active+clean 2024-09-10T23:35:42.279 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.389814+0000', 'last_change': '2024-09-10T23:35:36.389814+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.389814+0000', 'last_undegraded': '2024-09-10T23:35:36.389814+0000', 'last_fullsized': '2024-09-10T23:35:36.389814+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:35:42.280 INFO:tasks.ceph.ceph_manager.ceph:PG 9.37 is not active+clean 2024-09-10T23:35:42.280 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.389006+0000', 'last_change': '2024-09-10T23:35:36.389006+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.389006+0000', 'last_undegraded': '2024-09-10T23:35:36.389006+0000', 'last_fullsized': '2024-09-10T23:35:36.389006+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T23:35:42.280 INFO:tasks.ceph.ceph_manager.ceph:PG 9.35 is not active+clean 2024-09-10T23:35:42.280 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.394567+0000', 'last_change': '2024-09-10T23:35:36.394567+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.394567+0000', 'last_undegraded': '2024-09-10T23:35:36.394567+0000', 'last_fullsized': '2024-09-10T23:35:36.394567+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T23:35:42.280 INFO:tasks.ceph.ceph_manager.ceph:PG 9.34 is not active+clean 2024-09-10T23:35:42.280 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.391006+0000', 'last_change': '2024-09-10T23:35:36.391006+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.391006+0000', 'last_undegraded': '2024-09-10T23:35:36.391006+0000', 'last_fullsized': '2024-09-10T23:35:36.391006+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T23:35:42.280 INFO:tasks.ceph.ceph_manager.ceph:PG 9.33 is not active+clean 2024-09-10T23:35:42.280 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.397258+0000', 'last_change': '2024-09-10T23:35:36.397258+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.397258+0000', 'last_undegraded': '2024-09-10T23:35:36.397258+0000', 'last_fullsized': '2024-09-10T23:35:36.397258+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T23:35:42.280 INFO:tasks.ceph.ceph_manager.ceph:PG 9.32 is not active+clean 2024-09-10T23:35:42.281 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.389045+0000', 'last_change': '2024-09-10T23:35:36.389045+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.389045+0000', 'last_undegraded': '2024-09-10T23:35:36.389045+0000', 'last_fullsized': '2024-09-10T23:35:36.389045+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T23:35:42.281 INFO:tasks.ceph.ceph_manager.ceph:PG 9.31 is not active+clean 2024-09-10T23:35:42.281 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.394449+0000', 'last_change': '2024-09-10T23:35:36.394449+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.394449+0000', 'last_undegraded': '2024-09-10T23:35:36.394449+0000', 'last_fullsized': '2024-09-10T23:35:36.394449+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T23:35:42.281 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2e is not active+clean 2024-09-10T23:35:42.281 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.392062+0000', 'last_change': '2024-09-10T23:35:36.392062+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.392062+0000', 'last_undegraded': '2024-09-10T23:35:36.392062+0000', 'last_fullsized': '2024-09-10T23:35:36.392062+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T23:35:42.281 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2d is not active+clean 2024-09-10T23:35:42.281 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.389708+0000', 'last_change': '2024-09-10T23:35:36.389708+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.389708+0000', 'last_undegraded': '2024-09-10T23:35:36.389708+0000', 'last_fullsized': '2024-09-10T23:35:36.389708+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:35:42.281 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2c is not active+clean 2024-09-10T23:35:42.281 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.394443+0000', 'last_change': '2024-09-10T23:35:36.394443+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.394443+0000', 'last_undegraded': '2024-09-10T23:35:36.394443+0000', 'last_fullsized': '2024-09-10T23:35:36.394443+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T23:35:42.281 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2b is not active+clean 2024-09-10T23:35:42.282 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.396891+0000', 'last_change': '2024-09-10T23:35:36.396891+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.396891+0000', 'last_undegraded': '2024-09-10T23:35:36.396891+0000', 'last_fullsized': '2024-09-10T23:35:36.396891+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T23:35:42.282 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2a is not active+clean 2024-09-10T23:35:42.282 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.396024+0000', 'last_change': '2024-09-10T23:35:36.396024+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.396024+0000', 'last_undegraded': '2024-09-10T23:35:36.396024+0000', 'last_fullsized': '2024-09-10T23:35:36.396024+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T23:35:42.282 INFO:tasks.ceph.ceph_manager.ceph:PG 9.29 is not active+clean 2024-09-10T23:35:42.282 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.392968+0000', 'last_change': '2024-09-10T23:35:36.392968+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.392968+0000', 'last_undegraded': '2024-09-10T23:35:36.392968+0000', 'last_fullsized': '2024-09-10T23:35:36.392968+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T23:35:42.282 INFO:tasks.ceph.ceph_manager.ceph:PG 9.28 is not active+clean 2024-09-10T23:35:42.282 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.390266+0000', 'last_change': '2024-09-10T23:35:36.390266+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.390266+0000', 'last_undegraded': '2024-09-10T23:35:36.390266+0000', 'last_fullsized': '2024-09-10T23:35:36.390266+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T23:35:42.282 INFO:tasks.ceph.ceph_manager.ceph:PG 9.27 is not active+clean 2024-09-10T23:35:42.282 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.389015+0000', 'last_change': '2024-09-10T23:35:36.389015+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.389015+0000', 'last_undegraded': '2024-09-10T23:35:36.389015+0000', 'last_fullsized': '2024-09-10T23:35:36.389015+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:35:42.282 INFO:tasks.ceph.ceph_manager.ceph:PG 9.26 is not active+clean 2024-09-10T23:35:42.282 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.395481+0000', 'last_change': '2024-09-10T23:35:36.395481+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.395481+0000', 'last_undegraded': '2024-09-10T23:35:36.395481+0000', 'last_fullsized': '2024-09-10T23:35:36.395481+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T23:35:42.283 INFO:tasks.ceph.ceph_manager.ceph:PG 9.25 is not active+clean 2024-09-10T23:35:42.283 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.389120+0000', 'last_change': '2024-09-10T23:35:36.389120+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.389120+0000', 'last_undegraded': '2024-09-10T23:35:36.389120+0000', 'last_fullsized': '2024-09-10T23:35:36.389120+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:35:42.283 INFO:tasks.ceph.ceph_manager.ceph:PG 9.24 is not active+clean 2024-09-10T23:35:42.283 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.392340+0000', 'last_change': '2024-09-10T23:35:36.392340+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.392340+0000', 'last_undegraded': '2024-09-10T23:35:36.392340+0000', 'last_fullsized': '2024-09-10T23:35:36.392340+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:35:42.283 INFO:tasks.ceph.ceph_manager.ceph:PG 9.23 is not active+clean 2024-09-10T23:35:42.283 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.388535+0000', 'last_change': '2024-09-10T23:35:36.388535+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.388535+0000', 'last_undegraded': '2024-09-10T23:35:36.388535+0000', 'last_fullsized': '2024-09-10T23:35:36.388535+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:35:42.283 INFO:tasks.ceph.ceph_manager.ceph:PG 9.22 is not active+clean 2024-09-10T23:35:42.283 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.391729+0000', 'last_change': '2024-09-10T23:35:36.391729+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.391729+0000', 'last_undegraded': '2024-09-10T23:35:36.391729+0000', 'last_fullsized': '2024-09-10T23:35:36.391729+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T23:35:42.283 INFO:tasks.ceph.ceph_manager.ceph:PG 9.21 is not active+clean 2024-09-10T23:35:42.283 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.388251+0000', 'last_change': '2024-09-10T23:35:36.388251+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.388251+0000', 'last_undegraded': '2024-09-10T23:35:36.388251+0000', 'last_fullsized': '2024-09-10T23:35:36.388251+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:35:42.283 INFO:tasks.ceph.ceph_manager.ceph:PG 9.20 is not active+clean 2024-09-10T23:35:42.284 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.394059+0000', 'last_change': '2024-09-10T23:35:36.394059+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.394059+0000', 'last_undegraded': '2024-09-10T23:35:36.394059+0000', 'last_fullsized': '2024-09-10T23:35:36.394059+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T23:35:42.284 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1c is not active+clean 2024-09-10T23:35:42.284 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.391940+0000', 'last_change': '2024-09-10T23:35:36.391940+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.391940+0000', 'last_undegraded': '2024-09-10T23:35:36.391940+0000', 'last_fullsized': '2024-09-10T23:35:36.391940+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:35:42.284 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1a is not active+clean 2024-09-10T23:35:42.284 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.387665+0000', 'last_change': '2024-09-10T23:35:36.387665+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.387665+0000', 'last_undegraded': '2024-09-10T23:35:36.387665+0000', 'last_fullsized': '2024-09-10T23:35:36.387665+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:35:42.284 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1b is not active+clean 2024-09-10T23:35:42.284 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.396962+0000', 'last_change': '2024-09-10T23:35:36.396962+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.396962+0000', 'last_undegraded': '2024-09-10T23:35:36.396962+0000', 'last_fullsized': '2024-09-10T23:35:36.396962+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T23:35:42.284 INFO:tasks.ceph.ceph_manager.ceph:PG 9.18 is not active+clean 2024-09-10T23:35:42.284 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.389936+0000', 'last_change': '2024-09-10T23:35:36.389936+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.389936+0000', 'last_undegraded': '2024-09-10T23:35:36.389936+0000', 'last_fullsized': '2024-09-10T23:35:36.389936+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T23:35:42.284 INFO:tasks.ceph.ceph_manager.ceph:PG 9.16 is not active+clean 2024-09-10T23:35:42.284 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.394992+0000', 'last_change': '2024-09-10T23:35:36.394992+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.394992+0000', 'last_undegraded': '2024-09-10T23:35:36.394992+0000', 'last_fullsized': '2024-09-10T23:35:36.394992+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T23:35:42.285 INFO:tasks.ceph.ceph_manager.ceph:PG 9.17 is not active+clean 2024-09-10T23:35:42.285 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.394500+0000', 'last_change': '2024-09-10T23:35:36.394500+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.394500+0000', 'last_undegraded': '2024-09-10T23:35:36.394500+0000', 'last_fullsized': '2024-09-10T23:35:36.394500+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T23:35:42.285 INFO:tasks.ceph.ceph_manager.ceph:PG 9.14 is not active+clean 2024-09-10T23:35:42.285 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.389483+0000', 'last_change': '2024-09-10T23:35:36.389483+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.389483+0000', 'last_undegraded': '2024-09-10T23:35:36.389483+0000', 'last_fullsized': '2024-09-10T23:35:36.389483+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:35:42.285 INFO:tasks.ceph.ceph_manager.ceph:PG 9.15 is not active+clean 2024-09-10T23:35:42.285 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.391900+0000', 'last_change': '2024-09-10T23:35:36.391900+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.391900+0000', 'last_undegraded': '2024-09-10T23:35:36.391900+0000', 'last_fullsized': '2024-09-10T23:35:36.391900+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T23:35:42.285 INFO:tasks.ceph.ceph_manager.ceph:PG 9.12 is not active+clean 2024-09-10T23:35:42.285 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.392350+0000', 'last_change': '2024-09-10T23:35:36.392350+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.392350+0000', 'last_undegraded': '2024-09-10T23:35:36.392350+0000', 'last_fullsized': '2024-09-10T23:35:36.392350+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T23:35:42.285 INFO:tasks.ceph.ceph_manager.ceph:PG 9.13 is not active+clean 2024-09-10T23:35:42.285 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.388987+0000', 'last_change': '2024-09-10T23:35:36.388987+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.388987+0000', 'last_undegraded': '2024-09-10T23:35:36.388987+0000', 'last_fullsized': '2024-09-10T23:35:36.388987+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T23:35:42.286 INFO:tasks.ceph.ceph_manager.ceph:PG 9.10 is not active+clean 2024-09-10T23:35:42.286 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.389852+0000', 'last_change': '2024-09-10T23:35:36.389852+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.389852+0000', 'last_undegraded': '2024-09-10T23:35:36.389852+0000', 'last_fullsized': '2024-09-10T23:35:36.389852+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T23:35:42.286 INFO:tasks.ceph.ceph_manager.ceph:PG 9.11 is not active+clean 2024-09-10T23:35:42.286 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.390490+0000', 'last_change': '2024-09-10T23:35:36.390490+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.390490+0000', 'last_undegraded': '2024-09-10T23:35:36.390490+0000', 'last_fullsized': '2024-09-10T23:35:36.390490+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T23:35:42.286 INFO:tasks.ceph.ceph_manager.ceph:PG 9.e is not active+clean 2024-09-10T23:35:42.286 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.391693+0000', 'last_change': '2024-09-10T23:35:36.391693+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.391693+0000', 'last_undegraded': '2024-09-10T23:35:36.391693+0000', 'last_fullsized': '2024-09-10T23:35:36.391693+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T23:35:42.286 INFO:tasks.ceph.ceph_manager.ceph:PG 9.f is not active+clean 2024-09-10T23:35:42.286 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.394047+0000', 'last_change': '2024-09-10T23:35:36.394047+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.394047+0000', 'last_undegraded': '2024-09-10T23:35:36.394047+0000', 'last_fullsized': '2024-09-10T23:35:36.394047+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T23:35:42.286 INFO:tasks.ceph.ceph_manager.ceph:PG 9.c is not active+clean 2024-09-10T23:35:42.286 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.389265+0000', 'last_change': '2024-09-10T23:35:36.389265+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.389265+0000', 'last_undegraded': '2024-09-10T23:35:36.389265+0000', 'last_fullsized': '2024-09-10T23:35:36.389265+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:35:42.286 INFO:tasks.ceph.ceph_manager.ceph:PG 9.d is not active+clean 2024-09-10T23:35:42.286 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.388202+0000', 'last_change': '2024-09-10T23:35:36.388202+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.388202+0000', 'last_undegraded': '2024-09-10T23:35:36.388202+0000', 'last_fullsized': '2024-09-10T23:35:36.388202+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T23:35:42.287 INFO:tasks.ceph.ceph_manager.ceph:PG 9.a is not active+clean 2024-09-10T23:35:42.287 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.391962+0000', 'last_change': '2024-09-10T23:35:36.391962+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.391962+0000', 'last_undegraded': '2024-09-10T23:35:36.391962+0000', 'last_fullsized': '2024-09-10T23:35:36.391962+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T23:35:42.287 INFO:tasks.ceph.ceph_manager.ceph:PG 9.b is not active+clean 2024-09-10T23:35:42.287 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.388064+0000', 'last_change': '2024-09-10T23:35:36.388064+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.388064+0000', 'last_undegraded': '2024-09-10T23:35:36.388064+0000', 'last_fullsized': '2024-09-10T23:35:36.388064+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:35:42.287 INFO:tasks.ceph.ceph_manager.ceph:PG 9.9 is not active+clean 2024-09-10T23:35:42.287 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.388430+0000', 'last_change': '2024-09-10T23:35:36.388430+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.388430+0000', 'last_undegraded': '2024-09-10T23:35:36.388430+0000', 'last_fullsized': '2024-09-10T23:35:36.388430+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:35:42.287 INFO:tasks.ceph.ceph_manager.ceph:PG 9.6 is not active+clean 2024-09-10T23:35:42.287 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.391377+0000', 'last_change': '2024-09-10T23:35:36.391377+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.391377+0000', 'last_undegraded': '2024-09-10T23:35:36.391377+0000', 'last_fullsized': '2024-09-10T23:35:36.391377+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T23:35:42.287 INFO:tasks.ceph.ceph_manager.ceph:PG 9.7 is not active+clean 2024-09-10T23:35:42.287 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.393643+0000', 'last_change': '2024-09-10T23:35:36.393643+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.393643+0000', 'last_undegraded': '2024-09-10T23:35:36.393643+0000', 'last_fullsized': '2024-09-10T23:35:36.393643+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T23:35:42.288 INFO:tasks.ceph.ceph_manager.ceph:PG 9.4 is not active+clean 2024-09-10T23:35:42.288 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.388544+0000', 'last_change': '2024-09-10T23:35:36.388544+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.388544+0000', 'last_undegraded': '2024-09-10T23:35:36.388544+0000', 'last_fullsized': '2024-09-10T23:35:36.388544+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:35:42.288 INFO:tasks.ceph.ceph_manager.ceph:PG 9.5 is not active+clean 2024-09-10T23:35:42.288 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.391111+0000', 'last_change': '2024-09-10T23:35:36.391111+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.391111+0000', 'last_undegraded': '2024-09-10T23:35:36.391111+0000', 'last_fullsized': '2024-09-10T23:35:36.391111+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:35:42.288 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3 is not active+clean 2024-09-10T23:35:42.288 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.395931+0000', 'last_change': '2024-09-10T23:35:36.395931+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.395931+0000', 'last_undegraded': '2024-09-10T23:35:36.395931+0000', 'last_fullsized': '2024-09-10T23:35:36.395931+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T23:35:42.288 INFO:tasks.ceph.ceph_manager.ceph:PG 9.0 is not active+clean 2024-09-10T23:35:42.288 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.389339+0000', 'last_change': '2024-09-10T23:35:36.389339+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.389339+0000', 'last_undegraded': '2024-09-10T23:35:36.389339+0000', 'last_fullsized': '2024-09-10T23:35:36.389339+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:35:42.288 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1 is not active+clean 2024-09-10T23:35:42.288 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.387662+0000', 'last_change': '2024-09-10T23:35:36.387662+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.387662+0000', 'last_undegraded': '2024-09-10T23:35:36.387662+0000', 'last_fullsized': '2024-09-10T23:35:36.387662+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:35:42.288 INFO:tasks.ceph.ceph_manager.ceph:PG 9.8 is not active+clean 2024-09-10T23:35:42.288 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:35:36.387645+0000', 'last_change': '2024-09-10T23:35:36.387645+0000', 'last_active': '2024-09-10T23:35:36.375658+0000', 'last_peered': '2024-09-10T23:35:36.375658+0000', 'last_clean': '2024-09-10T23:35:36.375658+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:35:36.387645+0000', 'last_undegraded': '2024-09-10T23:35:36.387645+0000', 'last_fullsized': '2024-09-10T23:35:36.387645+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'last_clean_scrub_stamp': '2024-09-10T23:35:36.375658+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:35:42.289 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T23:35:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.458+0000 7f3dbfe00640 1 -- 172.21.15.94:0/3134081130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dc00d1fa0 msgr2=0x7f3dc00b2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.458+0000 7f3dbfe00640 1 --2- 172.21.15.94:0/3134081130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dc00d1fa0 0x7f3dc00b2a10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009900 tx=0x7f3db801c8e0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.458+0000 7f3dbfe00640 1 -- 172.21.15.94:0/3134081130 shutdown_connections 2024-09-10T23:35:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.458+0000 7f3dbfe00640 1 --2- 172.21.15.94:0/3134081130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dc00d1fa0 0x7f3dc00b2a10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.458+0000 7f3dbfe00640 1 --2- 172.21.15.94:0/3134081130 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3dc015bcd0 0x7f3dc00d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.458+0000 7f3dbfe00640 1 -- 172.21.15.94:0/3134081130 >> 172.21.15.94:0/3134081130 conn(0x7f3dc009aa70 msgr2=0x7f3dc009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.458+0000 7f3dbfe00640 1 -- 172.21.15.94:0/3134081130 shutdown_connections 2024-09-10T23:35:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.458+0000 7f3dbfe00640 1 -- 172.21.15.94:0/3134081130 wait complete. 2024-09-10T23:35:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.458+0000 7f3dbfe00640 1 Processor -- start 2024-09-10T23:35:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.458+0000 7f3dbfe00640 1 -- start start 2024-09-10T23:35:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc00b30e0 0x7f3dc01576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbfe00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3dc00d1fa0 0x7f3dc0154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dc015bcd0 0x7f3dc00b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbfe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc0168870 con 0x7f3dc00b30e0 2024-09-10T23:35:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbfe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc01686f0 con 0x7f3dc015bcd0 2024-09-10T23:35:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbfe00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc01689f0 con 0x7f3dc00d1fa0 2024-09-10T23:35:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc00b30e0 0x7f3dc01576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc00b30e0 0x7f3dc01576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42594/0 (socket says 172.21.15.94:42594) 2024-09-10T23:35:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbea00640 1 -- 172.21.15.94:0/1615354073 learned_addr learned my addr 172.21.15.94:0/1615354073 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbe000640 1 --2- 172.21.15.94:0/1615354073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3dc00d1fa0 0x7f3dc0154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbf400640 1 --2- 172.21.15.94:0/1615354073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dc015bcd0 0x7f3dc00b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbea00640 1 -- 172.21.15.94:0/1615354073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3dc00d1fa0 msgr2=0x7f3dc0154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbea00640 1 --2- 172.21.15.94:0/1615354073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3dc00d1fa0 0x7f3dc0154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbea00640 1 -- 172.21.15.94:0/1615354073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dc015bcd0 msgr2=0x7f3dc00b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbea00640 1 --2- 172.21.15.94:0/1615354073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dc015bcd0 0x7f3dc00b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbea00640 1 -- 172.21.15.94:0/1615354073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc01571a0 con 0x7f3dc00b30e0 2024-09-10T23:35:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbea00640 1 --2- 172.21.15.94:0/1615354073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc00b30e0 0x7f3dc01576a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3db000ed00 tx=0x7f3db000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3db6a00640 1 -- 172.21.15.94:0/1615354073 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0003b80 con 0x7f3dc00b30e0 2024-09-10T23:35:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3db6a00640 1 -- 172.21.15.94:0/1615354073 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3db0003d20 con 0x7f3dc00b30e0 2024-09-10T23:35:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbfe00640 1 -- 172.21.15.94:0/1615354073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc00c5a70 con 0x7f3dc00b30e0 2024-09-10T23:35:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3db6a00640 1 -- 172.21.15.94:0/1615354073 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0004280 con 0x7f3dc00b30e0 2024-09-10T23:35:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbfe00640 1 -- 172.21.15.94:0/1615354073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc015a5f0 con 0x7f3dc00b30e0 2024-09-10T23:35:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3dbfe00640 1 -- 172.21.15.94:0/1615354073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc00d1a60 con 0x7f3dc00b30e0 2024-09-10T23:35:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.462+0000 7f3db6a00640 1 -- 172.21.15.94:0/1615354073 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f3db0004420 con 0x7f3dc00b30e0 2024-09-10T23:35:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.466+0000 7f3db6a00640 1 --2- 172.21.15.94:0/1615354073 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3da0072790 0x7f3da0074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.466+0000 7f3db6a00640 1 -- 172.21.15.94:0/1615354073 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f3db0094340 con 0x7f3dc00b30e0 2024-09-10T23:35:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.466+0000 7f3dbe000640 1 --2- 172.21.15.94:0/1615354073 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3da0072790 0x7f3da0074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.466+0000 7f3dbe000640 1 --2- 172.21.15.94:0/1615354073 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3da0072790 0x7f3da0074c50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3dc01552a0 tx=0x7f3dac009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.470+0000 7f3db6a00640 1 -- 172.21.15.94:0/1615354073 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3db005efc0 con 0x7f3dc00b30e0 2024-09-10T23:35:42.678 INFO:tasks.ceph.mds.c.smithi094.stderr:2024-09-10T23:35:42.678+0000 7fa9ec200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:35:42.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.694+0000 7f3dbfe00640 1 -- 172.21.15.94:0/1615354073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3dc0158750 con 0x7f3dc00b30e0 2024-09-10T23:35:42.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.694+0000 7f3db6a00640 1 -- 172.21.15.94:0/1615354073 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1139 (secure 0 0 0) 0x7f3db00639f0 con 0x7f3dc00b30e0 2024-09-10T23:35:42.732 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:42.732 INFO:teuthology.orchestra.run.smithi094.stdout:{"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":205,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":8,"num_up_osds":8,"osd_up_since":1726011151,"num_in_osds":8,"osd_in_since":1726011140,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":53},{"state_name":"active+clean","count":44}],"num_pgs":97,"num_pools":3,"num_objects":4,"data_bytes":590368,"bytes_used":253448192,"bytes_avail":772840665088,"bytes_total":773094113280,"inactive_pgs_ratio":0.54639172554016113,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":34,"btime":"2024-09-10T23:35:40:410237+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":195113}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T23:35:07.379006+0000","services":{}},"progress_events":{}} 2024-09-10T23:35:42.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.694+0000 7f3dbfe00640 1 -- 172.21.15.94:0/1615354073 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3da0072790 msgr2=0x7f3da0074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:42.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.694+0000 7f3dbfe00640 1 --2- 172.21.15.94:0/1615354073 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3da0072790 0x7f3da0074c50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3dc01552a0 tx=0x7f3dac009040 comp rx=0 tx=0).stop 2024-09-10T23:35:42.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.694+0000 7f3dbfe00640 1 -- 172.21.15.94:0/1615354073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc00b30e0 msgr2=0x7f3dc01576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:42.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.694+0000 7f3dbfe00640 1 --2- 172.21.15.94:0/1615354073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc00b30e0 0x7f3dc01576a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3db000ed00 tx=0x7f3db000c6a0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.694+0000 7f3dbfe00640 1 -- 172.21.15.94:0/1615354073 shutdown_connections 2024-09-10T23:35:42.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.694+0000 7f3dbfe00640 1 --2- 172.21.15.94:0/1615354073 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3da0072790 0x7f3da0074c50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.694+0000 7f3dbfe00640 1 --2- 172.21.15.94:0/1615354073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dc015bcd0 0x7f3dc00b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.694+0000 7f3dbfe00640 1 --2- 172.21.15.94:0/1615354073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3dc00d1fa0 0x7f3dc0154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.694+0000 7f3dbfe00640 1 --2- 172.21.15.94:0/1615354073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3dc00b30e0 0x7f3dc01576a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.694+0000 7f3dbfe00640 1 -- 172.21.15.94:0/1615354073 >> 172.21.15.94:0/1615354073 conn(0x7f3dc009aa70 msgr2=0x7f3dc00ce720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:42.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.694+0000 7f3dbfe00640 1 -- 172.21.15.94:0/1615354073 shutdown_connections 2024-09-10T23:35:42.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.694+0000 7f3dbfe00640 1 -- 172.21.15.94:0/1615354073 wait complete. 2024-09-10T23:35:42.735 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T23:35:42.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:35:42.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.874+0000 7f78df800640 1 -- 172.21.15.94:0/2433386417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78bc004810 msgr2=0x7f78bc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:42.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.874+0000 7f78df800640 1 --2- 172.21.15.94:0/2433386417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78bc004810 0x7f78bc004c90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f78d80b3ce0 tx=0x7f78c401cef0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.874+0000 7f78df800640 1 -- 172.21.15.94:0/2433386417 shutdown_connections 2024-09-10T23:35:42.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.874+0000 7f78df800640 1 --2- 172.21.15.94:0/2433386417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78bc004810 0x7f78bc004c90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.874+0000 7f78df800640 1 -- 172.21.15.94:0/2433386417 >> 172.21.15.94:0/2433386417 conn(0x7f78d806cdf0 msgr2=0x7f78d806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:42.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.874+0000 7f78df800640 1 -- 172.21.15.94:0/2433386417 shutdown_connections 2024-09-10T23:35:42.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.874+0000 7f78df800640 1 -- 172.21.15.94:0/2433386417 wait complete. 2024-09-10T23:35:42.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.874+0000 7f78df800640 1 Processor -- start 2024-09-10T23:35:42.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.874+0000 7f78df800640 1 -- start start 2024-09-10T23:35:42.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78df800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78bc004810 0x7f78d814fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:42.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78df800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78d81401a0 0x7f78d8140600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:42.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78df800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78d81592d0 0x7f78d8159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:42.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78df800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f78d80d0040 con 0x7f78bc004810 2024-09-10T23:35:42.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78df800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f78d80cfec0 con 0x7f78d81592d0 2024-09-10T23:35:42.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78df800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f78d80d01c0 con 0x7f78d81401a0 2024-09-10T23:35:42.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78dce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78bc004810 0x7f78d814fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:42.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78dce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78bc004810 0x7f78d814fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42616/0 (socket says 172.21.15.94:42616) 2024-09-10T23:35:42.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78dce00640 1 -- 172.21.15.94:0/806351162 learned_addr learned my addr 172.21.15.94:0/806351162 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:42.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78dce00640 1 -- 172.21.15.94:0/806351162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78d81401a0 msgr2=0x7f78d8140600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:42.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78dce00640 1 --2- 172.21.15.94:0/806351162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78d81401a0 0x7f78d8140600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78dce00640 1 -- 172.21.15.94:0/806351162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78d81592d0 msgr2=0x7f78d8159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:42.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78dce00640 1 --2- 172.21.15.94:0/806351162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78d81592d0 0x7f78d8159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:42.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78dce00640 1 -- 172.21.15.94:0/806351162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d8140b40 con 0x7f78bc004810 2024-09-10T23:35:42.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78dce00640 1 --2- 172.21.15.94:0/806351162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78bc004810 0x7f78d814fcd0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f78c8011cc0 tx=0x7f78c800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:42.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78d4c00640 1 -- 172.21.15.94:0/806351162 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8003b80 con 0x7f78bc004810 2024-09-10T23:35:42.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78d4c00640 1 -- 172.21.15.94:0/806351162 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78c8003d20 con 0x7f78bc004810 2024-09-10T23:35:42.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78d4c00640 1 -- 172.21.15.94:0/806351162 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8004270 con 0x7f78bc004810 2024-09-10T23:35:42.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78df800640 1 -- 172.21.15.94:0/806351162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d8159cc0 con 0x7f78bc004810 2024-09-10T23:35:42.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78df800640 1 -- 172.21.15.94:0/806351162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d815a240 con 0x7f78bc004810 2024-09-10T23:35:42.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.878+0000 7f78df800640 1 -- 172.21.15.94:0/806351162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d80ce7e0 con 0x7f78bc004810 2024-09-10T23:35:42.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.882+0000 7f78d4c00640 1 -- 172.21.15.94:0/806351162 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f78c8022030 con 0x7f78bc004810 2024-09-10T23:35:42.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.882+0000 7f78d4c00640 1 --2- 172.21.15.94:0/806351162 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f78b8072790 0x7f78b8074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:42.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.882+0000 7f78d4c00640 1 -- 172.21.15.94:0/806351162 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f78c8094920 con 0x7f78bc004810 2024-09-10T23:35:42.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.882+0000 7f78d7e00640 1 --2- 172.21.15.94:0/806351162 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f78b8072790 0x7f78b8074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:42.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.882+0000 7f78d7e00640 1 --2- 172.21.15.94:0/806351162 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f78b8072790 0x7f78b8074c50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f78c40ac000 tx=0x7f78bc005840 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:42.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:42.886+0000 7f78d4c00640 1 -- 172.21.15.94:0/806351162 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f78c805fc60 con 0x7f78bc004810 2024-09-10T23:35:43.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:43.046+0000 7f78df800640 1 -- 172.21.15.94:0/806351162 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78d8155f30 con 0x7f78b8072790 2024-09-10T23:35:43.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:43.050+0000 7f78d4c00640 1 -- 172.21.15.94:0/806351162 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247559 (secure 0 0 0) 0x7f78d8155f30 con 0x7f78b8072790 2024-09-10T23:35:43.056 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:43.057 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:35:43.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:43.054+0000 7f78df800640 1 -- 172.21.15.94:0/806351162 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f78b8072790 msgr2=0x7f78b8074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:43.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:43.054+0000 7f78df800640 1 --2- 172.21.15.94:0/806351162 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f78b8072790 0x7f78b8074c50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f78c40ac000 tx=0x7f78bc005840 comp rx=0 tx=0).stop 2024-09-10T23:35:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:43.054+0000 7f78df800640 1 -- 172.21.15.94:0/806351162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78bc004810 msgr2=0x7f78d814fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:43.054+0000 7f78df800640 1 --2- 172.21.15.94:0/806351162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78bc004810 0x7f78d814fcd0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f78c8011cc0 tx=0x7f78c800f790 comp rx=0 tx=0).stop 2024-09-10T23:35:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:43.054+0000 7f78df800640 1 -- 172.21.15.94:0/806351162 shutdown_connections 2024-09-10T23:35:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:43.054+0000 7f78df800640 1 --2- 172.21.15.94:0/806351162 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f78b8072790 0x7f78b8074c50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:43.054+0000 7f78df800640 1 --2- 172.21.15.94:0/806351162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78d81592d0 0x7f78d8159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:43.054+0000 7f78df800640 1 --2- 172.21.15.94:0/806351162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78d81401a0 0x7f78d8140600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:43.054+0000 7f78df800640 1 --2- 172.21.15.94:0/806351162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78bc004810 0x7f78d814fcd0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:43.054+0000 7f78df800640 1 -- 172.21.15.94:0/806351162 >> 172.21.15.94:0/806351162 conn(0x7f78d806cdf0 msgr2=0x7f78d815c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:43.054+0000 7f78df800640 1 -- 172.21.15.94:0/806351162 shutdown_connections 2024-09-10T23:35:43.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:43.054+0000 7f78df800640 1 -- 172.21.15.94:0/806351162 wait complete. 2024-09-10T23:35:43.075 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-10T23:35:41.388360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247508,"kb_used_data":6312,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727212,"statfs":{"total":773094113280,"available":772840665088,"internally_reserved":0,"allocated":6463488,"data_stored":3329394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001510"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391197+0000","last_change":"2024-09-10T23:35:37.411838+0000","last_active":"2024-09-10T23:35:38.391197+0000","last_peered":"2024-09-10T23:35:38.391197+0000","last_clean":"2024-09-10T23:35:38.391197+0000","last_became_active":"2024-09-10T23:35:37.411168+0000","last_became_peered":"2024-09-10T23:35:37.411168+0000","last_unstale":"2024-09-10T23:35:38.391197+0000","last_undegraded":"2024-09-10T23:35:38.391197+0000","last_fullsized":"2024-09-10T23:35:38.391197+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:27.635555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388506+0000","last_change":"2024-09-10T23:35:36.388506+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388506+0000","last_undegraded":"2024-09-10T23:35:36.388506+0000","last_fullsized":"2024-09-10T23:35:36.388506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.390335+0000","last_change":"2024-09-10T23:35:36.390335+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.390335+0000","last_undegraded":"2024-09-10T23:35:36.390335+0000","last_fullsized":"2024-09-10T23:35:36.390335+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392121+0000","last_change":"2024-09-10T23:35:36.392121+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392121+0000","last_undegraded":"2024-09-10T23:35:36.392121+0000","last_fullsized":"2024-09-10T23:35:36.392121+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390631+0000","last_change":"2024-09-10T23:35:37.422878+0000","last_active":"2024-09-10T23:35:38.390631+0000","last_peered":"2024-09-10T23:35:38.390631+0000","last_clean":"2024-09-10T23:35:38.390631+0000","last_became_active":"2024-09-10T23:35:37.422602+0000","last_became_peered":"2024-09-10T23:35:37.422602+0000","last_unstale":"2024-09-10T23:35:38.390631+0000","last_undegraded":"2024-09-10T23:35:38.390631+0000","last_fullsized":"2024-09-10T23:35:38.390631+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:17.965755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390045+0000","last_change":"2024-09-10T23:35:37.401854+0000","last_active":"2024-09-10T23:35:38.390045+0000","last_peered":"2024-09-10T23:35:38.390045+0000","last_clean":"2024-09-10T23:35:38.390045+0000","last_became_active":"2024-09-10T23:35:37.400521+0000","last_became_peered":"2024-09-10T23:35:37.400521+0000","last_unstale":"2024-09-10T23:35:38.390045+0000","last_undegraded":"2024-09-10T23:35:38.390045+0000","last_fullsized":"2024-09-10T23:35:38.390045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:48.537421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394892+0000","last_change":"2024-09-10T23:35:36.394892+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394892+0000","last_undegraded":"2024-09-10T23:35:36.394892+0000","last_fullsized":"2024-09-10T23:35:36.394892+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389814+0000","last_change":"2024-09-10T23:35:36.389814+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389814+0000","last_undegraded":"2024-09-10T23:35:36.389814+0000","last_fullsized":"2024-09-10T23:35:36.389814+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389006+0000","last_change":"2024-09-10T23:35:36.389006+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389006+0000","last_undegraded":"2024-09-10T23:35:36.389006+0000","last_fullsized":"2024-09-10T23:35:36.389006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390598+0000","last_change":"2024-09-10T23:35:37.405331+0000","last_active":"2024-09-10T23:35:38.390598+0000","last_peered":"2024-09-10T23:35:38.390598+0000","last_clean":"2024-09-10T23:35:38.390598+0000","last_became_active":"2024-09-10T23:35:37.404010+0000","last_became_peered":"2024-09-10T23:35:37.404010+0000","last_unstale":"2024-09-10T23:35:38.390598+0000","last_undegraded":"2024-09-10T23:35:38.390598+0000","last_fullsized":"2024-09-10T23:35:38.390598+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:30.787726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394567+0000","last_change":"2024-09-10T23:35:36.394567+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394567+0000","last_undegraded":"2024-09-10T23:35:36.394567+0000","last_fullsized":"2024-09-10T23:35:36.394567+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391006+0000","last_change":"2024-09-10T23:35:36.391006+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391006+0000","last_undegraded":"2024-09-10T23:35:36.391006+0000","last_fullsized":"2024-09-10T23:35:36.391006+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.397258+0000","last_change":"2024-09-10T23:35:36.397258+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.397258+0000","last_undegraded":"2024-09-10T23:35:36.397258+0000","last_fullsized":"2024-09-10T23:35:36.397258+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389045+0000","last_change":"2024-09-10T23:35:36.389045+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389045+0000","last_undegraded":"2024-09-10T23:35:36.389045+0000","last_fullsized":"2024-09-10T23:35:36.389045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394449+0000","last_change":"2024-09-10T23:35:36.394449+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394449+0000","last_undegraded":"2024-09-10T23:35:36.394449+0000","last_fullsized":"2024-09-10T23:35:36.394449+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389559+0000","last_change":"2024-09-10T23:35:37.421193+0000","last_active":"2024-09-10T23:35:38.389559+0000","last_peered":"2024-09-10T23:35:38.389559+0000","last_clean":"2024-09-10T23:35:38.389559+0000","last_became_active":"2024-09-10T23:35:37.420909+0000","last_became_peered":"2024-09-10T23:35:37.420909+0000","last_unstale":"2024-09-10T23:35:38.389559+0000","last_undegraded":"2024-09-10T23:35:38.389559+0000","last_fullsized":"2024-09-10T23:35:38.389559+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:31.257242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391217+0000","last_change":"2024-09-10T23:35:37.404324+0000","last_active":"2024-09-10T23:35:38.391217+0000","last_peered":"2024-09-10T23:35:38.391217+0000","last_clean":"2024-09-10T23:35:38.391217+0000","last_became_active":"2024-09-10T23:35:37.404042+0000","last_became_peered":"2024-09-10T23:35:37.404042+0000","last_unstale":"2024-09-10T23:35:38.391217+0000","last_undegraded":"2024-09-10T23:35:38.391217+0000","last_fullsized":"2024-09-10T23:35:38.391217+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:31.250579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392062+0000","last_change":"2024-09-10T23:35:36.392062+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392062+0000","last_undegraded":"2024-09-10T23:35:36.392062+0000","last_fullsized":"2024-09-10T23:35:36.392062+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389708+0000","last_change":"2024-09-10T23:35:36.389708+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389708+0000","last_undegraded":"2024-09-10T23:35:36.389708+0000","last_fullsized":"2024-09-10T23:35:36.389708+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394443+0000","last_change":"2024-09-10T23:35:36.394443+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394443+0000","last_undegraded":"2024-09-10T23:35:36.394443+0000","last_fullsized":"2024-09-10T23:35:36.394443+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.396891+0000","last_change":"2024-09-10T23:35:36.396891+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.396891+0000","last_undegraded":"2024-09-10T23:35:36.396891+0000","last_fullsized":"2024-09-10T23:35:36.396891+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.396024+0000","last_change":"2024-09-10T23:35:36.396024+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.396024+0000","last_undegraded":"2024-09-10T23:35:36.396024+0000","last_fullsized":"2024-09-10T23:35:36.396024+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392968+0000","last_change":"2024-09-10T23:35:36.392968+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392968+0000","last_undegraded":"2024-09-10T23:35:36.392968+0000","last_fullsized":"2024-09-10T23:35:36.392968+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.390266+0000","last_change":"2024-09-10T23:35:36.390266+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.390266+0000","last_undegraded":"2024-09-10T23:35:36.390266+0000","last_fullsized":"2024-09-10T23:35:36.390266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389015+0000","last_change":"2024-09-10T23:35:36.389015+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389015+0000","last_undegraded":"2024-09-10T23:35:36.389015+0000","last_fullsized":"2024-09-10T23:35:36.389015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.395481+0000","last_change":"2024-09-10T23:35:36.395481+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.395481+0000","last_undegraded":"2024-09-10T23:35:36.395481+0000","last_fullsized":"2024-09-10T23:35:36.395481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389120+0000","last_change":"2024-09-10T23:35:36.389120+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389120+0000","last_undegraded":"2024-09-10T23:35:36.389120+0000","last_fullsized":"2024-09-10T23:35:36.389120+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392340+0000","last_change":"2024-09-10T23:35:36.392340+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392340+0000","last_undegraded":"2024-09-10T23:35:36.392340+0000","last_fullsized":"2024-09-10T23:35:36.392340+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388535+0000","last_change":"2024-09-10T23:35:36.388535+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388535+0000","last_undegraded":"2024-09-10T23:35:36.388535+0000","last_fullsized":"2024-09-10T23:35:36.388535+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391729+0000","last_change":"2024-09-10T23:35:36.391729+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391729+0000","last_undegraded":"2024-09-10T23:35:36.391729+0000","last_fullsized":"2024-09-10T23:35:36.391729+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388251+0000","last_change":"2024-09-10T23:35:36.388251+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388251+0000","last_undegraded":"2024-09-10T23:35:36.388251+0000","last_fullsized":"2024-09-10T23:35:36.388251+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394059+0000","last_change":"2024-09-10T23:35:36.394059+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394059+0000","last_undegraded":"2024-09-10T23:35:36.394059+0000","last_fullsized":"2024-09-10T23:35:36.394059+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391188+0000","last_change":"2024-09-10T23:35:37.405530+0000","last_active":"2024-09-10T23:35:38.391188+0000","last_peered":"2024-09-10T23:35:38.391188+0000","last_clean":"2024-09-10T23:35:38.391188+0000","last_became_active":"2024-09-10T23:35:37.402795+0000","last_became_peered":"2024-09-10T23:35:37.402795+0000","last_unstale":"2024-09-10T23:35:38.391188+0000","last_undegraded":"2024-09-10T23:35:38.391188+0000","last_fullsized":"2024-09-10T23:35:38.391188+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:15.871163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.404962+0000","last_change":"2024-09-10T23:35:36.401576+0000","last_active":"2024-09-10T23:35:36.404962+0000","last_peered":"2024-09-10T23:35:36.404962+0000","last_clean":"2024-09-10T23:35:36.404962+0000","last_became_active":"2024-09-10T23:35:36.400780+0000","last_became_peered":"2024-09-10T23:35:36.400780+0000","last_unstale":"2024-09-10T23:35:36.404962+0000","last_undegraded":"2024-09-10T23:35:36.404962+0000","last_fullsized":"2024-09-10T23:35:36.404962+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:15.011008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391401+0000","last_change":"2024-09-10T23:35:37.411543+0000","last_active":"2024-09-10T23:35:38.391401+0000","last_peered":"2024-09-10T23:35:38.391401+0000","last_clean":"2024-09-10T23:35:38.391401+0000","last_became_active":"2024-09-10T23:35:37.410807+0000","last_became_peered":"2024-09-10T23:35:37.410807+0000","last_unstale":"2024-09-10T23:35:38.391401+0000","last_undegraded":"2024-09-10T23:35:38.391401+0000","last_fullsized":"2024-09-10T23:35:38.391401+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:52.749748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.410357+0000","last_change":"2024-09-10T23:35:36.406745+0000","last_active":"2024-09-10T23:35:36.410357+0000","last_peered":"2024-09-10T23:35:36.410357+0000","last_clean":"2024-09-10T23:35:36.410357+0000","last_became_active":"2024-09-10T23:35:36.406460+0000","last_became_peered":"2024-09-10T23:35:36.406460+0000","last_unstale":"2024-09-10T23:35:36.410357+0000","last_undegraded":"2024-09-10T23:35:36.410357+0000","last_fullsized":"2024-09-10T23:35:36.410357+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:58.569452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391940+0000","last_change":"2024-09-10T23:35:36.391940+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391940+0000","last_undegraded":"2024-09-10T23:35:36.391940+0000","last_fullsized":"2024-09-10T23:35:36.391940+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390807+0000","last_change":"2024-09-10T23:35:36.409095+0000","last_active":"2024-09-10T23:35:38.390807+0000","last_peered":"2024-09-10T23:35:38.390807+0000","last_clean":"2024-09-10T23:35:38.390807+0000","last_became_active":"2024-09-10T23:35:36.408811+0000","last_became_peered":"2024-09-10T23:35:36.408811+0000","last_unstale":"2024-09-10T23:35:38.390807+0000","last_undegraded":"2024-09-10T23:35:38.390807+0000","last_fullsized":"2024-09-10T23:35:38.390807+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:47.477496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390766+0000","last_change":"2024-09-10T23:35:37.413035+0000","last_active":"2024-09-10T23:35:38.390766+0000","last_peered":"2024-09-10T23:35:38.390766+0000","last_clean":"2024-09-10T23:35:38.390766+0000","last_became_active":"2024-09-10T23:35:37.412781+0000","last_became_peered":"2024-09-10T23:35:37.412781+0000","last_unstale":"2024-09-10T23:35:38.390766+0000","last_undegraded":"2024-09-10T23:35:38.390766+0000","last_fullsized":"2024-09-10T23:35:38.390766+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:11.135478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.404617+0000","last_change":"2024-09-10T23:35:36.396384+0000","last_active":"2024-09-10T23:35:36.404617+0000","last_peered":"2024-09-10T23:35:36.404617+0000","last_clean":"2024-09-10T23:35:36.404617+0000","last_became_active":"2024-09-10T23:35:36.396085+0000","last_became_peered":"2024-09-10T23:35:36.396085+0000","last_unstale":"2024-09-10T23:35:36.404617+0000","last_undegraded":"2024-09-10T23:35:36.404617+0000","last_fullsized":"2024-09-10T23:35:36.404617+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:34.607437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.387665+0000","last_change":"2024-09-10T23:35:36.387665+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.387665+0000","last_undegraded":"2024-09-10T23:35:36.387665+0000","last_fullsized":"2024-09-10T23:35:36.387665+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407850+0000","last_change":"2024-09-10T23:35:36.404943+0000","last_active":"2024-09-10T23:35:36.407850+0000","last_peered":"2024-09-10T23:35:36.407850+0000","last_clean":"2024-09-10T23:35:36.407850+0000","last_became_active":"2024-09-10T23:35:36.404615+0000","last_became_peered":"2024-09-10T23:35:36.404615+0000","last_unstale":"2024-09-10T23:35:36.407850+0000","last_undegraded":"2024-09-10T23:35:36.407850+0000","last_fullsized":"2024-09-10T23:35:36.407850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:58.666819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.396962+0000","last_change":"2024-09-10T23:35:36.396962+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.396962+0000","last_undegraded":"2024-09-10T23:35:36.396962+0000","last_fullsized":"2024-09-10T23:35:36.396962+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390698+0000","last_change":"2024-09-10T23:35:36.408437+0000","last_active":"2024-09-10T23:35:38.390698+0000","last_peered":"2024-09-10T23:35:38.390698+0000","last_clean":"2024-09-10T23:35:38.390698+0000","last_became_active":"2024-09-10T23:35:36.408156+0000","last_became_peered":"2024-09-10T23:35:36.408156+0000","last_unstale":"2024-09-10T23:35:38.390698+0000","last_undegraded":"2024-09-10T23:35:38.390698+0000","last_fullsized":"2024-09-10T23:35:38.390698+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:01.998360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389936+0000","last_change":"2024-09-10T23:35:36.389936+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389936+0000","last_undegraded":"2024-09-10T23:35:36.389936+0000","last_fullsized":"2024-09-10T23:35:36.389936+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407639+0000","last_change":"2024-09-10T23:35:36.401028+0000","last_active":"2024-09-10T23:35:36.407639+0000","last_peered":"2024-09-10T23:35:36.407639+0000","last_clean":"2024-09-10T23:35:36.407639+0000","last_became_active":"2024-09-10T23:35:36.400743+0000","last_became_peered":"2024-09-10T23:35:36.400743+0000","last_unstale":"2024-09-10T23:35:36.407639+0000","last_undegraded":"2024-09-10T23:35:36.407639+0000","last_fullsized":"2024-09-10T23:35:36.407639+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:39:33.252954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389654+0000","last_change":"2024-09-10T23:35:37.422945+0000","last_active":"2024-09-10T23:35:38.389654+0000","last_peered":"2024-09-10T23:35:38.389654+0000","last_clean":"2024-09-10T23:35:38.389654+0000","last_became_active":"2024-09-10T23:35:37.422648+0000","last_became_peered":"2024-09-10T23:35:37.422648+0000","last_unstale":"2024-09-10T23:35:38.389654+0000","last_undegraded":"2024-09-10T23:35:38.389654+0000","last_fullsized":"2024-09-10T23:35:38.389654+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:10.705976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406280+0000","last_change":"2024-09-10T23:35:36.401434+0000","last_active":"2024-09-10T23:35:36.406280+0000","last_peered":"2024-09-10T23:35:36.406280+0000","last_clean":"2024-09-10T23:35:36.406280+0000","last_became_active":"2024-09-10T23:35:36.401155+0000","last_became_peered":"2024-09-10T23:35:36.401155+0000","last_unstale":"2024-09-10T23:35:36.406280+0000","last_undegraded":"2024-09-10T23:35:36.406280+0000","last_fullsized":"2024-09-10T23:35:36.406280+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:37.536960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394992+0000","last_change":"2024-09-10T23:35:36.394992+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394992+0000","last_undegraded":"2024-09-10T23:35:36.394992+0000","last_fullsized":"2024-09-10T23:35:36.394992+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407810+0000","last_change":"2024-09-10T23:35:36.405013+0000","last_active":"2024-09-10T23:35:36.407810+0000","last_peered":"2024-09-10T23:35:36.407810+0000","last_clean":"2024-09-10T23:35:36.407810+0000","last_became_active":"2024-09-10T23:35:36.404715+0000","last_became_peered":"2024-09-10T23:35:36.404715+0000","last_unstale":"2024-09-10T23:35:36.407810+0000","last_undegraded":"2024-09-10T23:35:36.407810+0000","last_fullsized":"2024-09-10T23:35:36.407810+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:11:46.791009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394500+0000","last_change":"2024-09-10T23:35:36.394500+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394500+0000","last_undegraded":"2024-09-10T23:35:36.394500+0000","last_fullsized":"2024-09-10T23:35:36.394500+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407797+0000","last_change":"2024-09-10T23:35:36.402529+0000","last_active":"2024-09-10T23:35:36.407797+0000","last_peered":"2024-09-10T23:35:36.407797+0000","last_clean":"2024-09-10T23:35:36.407797+0000","last_became_active":"2024-09-10T23:35:36.402270+0000","last_became_peered":"2024-09-10T23:35:36.402270+0000","last_unstale":"2024-09-10T23:35:36.407797+0000","last_undegraded":"2024-09-10T23:35:36.407797+0000","last_fullsized":"2024-09-10T23:35:36.407797+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:28.809096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389483+0000","last_change":"2024-09-10T23:35:36.389483+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389483+0000","last_undegraded":"2024-09-10T23:35:36.389483+0000","last_fullsized":"2024-09-10T23:35:36.389483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406578+0000","last_change":"2024-09-10T23:35:36.402631+0000","last_active":"2024-09-10T23:35:36.406578+0000","last_peered":"2024-09-10T23:35:36.406578+0000","last_clean":"2024-09-10T23:35:36.406578+0000","last_became_active":"2024-09-10T23:35:36.399947+0000","last_became_peered":"2024-09-10T23:35:36.399947+0000","last_unstale":"2024-09-10T23:35:36.406578+0000","last_undegraded":"2024-09-10T23:35:36.406578+0000","last_fullsized":"2024-09-10T23:35:36.406578+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:53.498796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391900+0000","last_change":"2024-09-10T23:35:36.391900+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391900+0000","last_undegraded":"2024-09-10T23:35:36.391900+0000","last_fullsized":"2024-09-10T23:35:36.391900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.408350+0000","last_change":"2024-09-10T23:35:36.405711+0000","last_active":"2024-09-10T23:35:36.408350+0000","last_peered":"2024-09-10T23:35:36.408350+0000","last_clean":"2024-09-10T23:35:36.408350+0000","last_became_active":"2024-09-10T23:35:36.405455+0000","last_became_peered":"2024-09-10T23:35:36.405455+0000","last_unstale":"2024-09-10T23:35:36.408350+0000","last_undegraded":"2024-09-10T23:35:36.408350+0000","last_fullsized":"2024-09-10T23:35:36.408350+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:56.422393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.392350+0000","last_change":"2024-09-10T23:35:36.392350+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.392350+0000","last_undegraded":"2024-09-10T23:35:36.392350+0000","last_fullsized":"2024-09-10T23:35:36.392350+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407863+0000","last_change":"2024-09-10T23:35:36.404848+0000","last_active":"2024-09-10T23:35:36.407863+0000","last_peered":"2024-09-10T23:35:36.407863+0000","last_clean":"2024-09-10T23:35:36.407863+0000","last_became_active":"2024-09-10T23:35:36.404567+0000","last_became_peered":"2024-09-10T23:35:36.404567+0000","last_unstale":"2024-09-10T23:35:36.407863+0000","last_undegraded":"2024-09-10T23:35:36.407863+0000","last_fullsized":"2024-09-10T23:35:36.407863+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:33.274868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388987+0000","last_change":"2024-09-10T23:35:36.388987+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388987+0000","last_undegraded":"2024-09-10T23:35:36.388987+0000","last_fullsized":"2024-09-10T23:35:36.388987+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407454+0000","last_change":"2024-09-10T23:35:36.402546+0000","last_active":"2024-09-10T23:35:36.407454+0000","last_peered":"2024-09-10T23:35:36.407454+0000","last_clean":"2024-09-10T23:35:36.407454+0000","last_became_active":"2024-09-10T23:35:36.402187+0000","last_became_peered":"2024-09-10T23:35:36.402187+0000","last_unstale":"2024-09-10T23:35:36.407454+0000","last_undegraded":"2024-09-10T23:35:36.407454+0000","last_fullsized":"2024-09-10T23:35:36.407454+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:18.611164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389852+0000","last_change":"2024-09-10T23:35:36.389852+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389852+0000","last_undegraded":"2024-09-10T23:35:36.389852+0000","last_fullsized":"2024-09-10T23:35:36.389852+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406227+0000","last_change":"2024-09-10T23:35:36.401288+0000","last_active":"2024-09-10T23:35:36.406227+0000","last_peered":"2024-09-10T23:35:36.406227+0000","last_clean":"2024-09-10T23:35:36.406227+0000","last_became_active":"2024-09-10T23:35:36.400889+0000","last_became_peered":"2024-09-10T23:35:36.400889+0000","last_unstale":"2024-09-10T23:35:36.406227+0000","last_undegraded":"2024-09-10T23:35:36.406227+0000","last_fullsized":"2024-09-10T23:35:36.406227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:05.132993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.390490+0000","last_change":"2024-09-10T23:35:36.390490+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.390490+0000","last_undegraded":"2024-09-10T23:35:36.390490+0000","last_fullsized":"2024-09-10T23:35:36.390490+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406140+0000","last_change":"2024-09-10T23:35:36.398356+0000","last_active":"2024-09-10T23:35:36.406140+0000","last_peered":"2024-09-10T23:35:36.406140+0000","last_clean":"2024-09-10T23:35:36.406140+0000","last_became_active":"2024-09-10T23:35:36.398077+0000","last_became_peered":"2024-09-10T23:35:36.398077+0000","last_unstale":"2024-09-10T23:35:36.406140+0000","last_undegraded":"2024-09-10T23:35:36.406140+0000","last_fullsized":"2024-09-10T23:35:36.406140+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:11.718995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391693+0000","last_change":"2024-09-10T23:35:36.391693+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391693+0000","last_undegraded":"2024-09-10T23:35:36.391693+0000","last_fullsized":"2024-09-10T23:35:36.391693+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.411637+0000","last_change":"2024-09-10T23:35:36.409579+0000","last_active":"2024-09-10T23:35:36.411637+0000","last_peered":"2024-09-10T23:35:36.411637+0000","last_clean":"2024-09-10T23:35:36.411637+0000","last_became_active":"2024-09-10T23:35:36.409321+0000","last_became_peered":"2024-09-10T23:35:36.409321+0000","last_unstale":"2024-09-10T23:35:36.411637+0000","last_undegraded":"2024-09-10T23:35:36.411637+0000","last_fullsized":"2024-09-10T23:35:36.411637+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:19.816523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.394047+0000","last_change":"2024-09-10T23:35:36.394047+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.394047+0000","last_undegraded":"2024-09-10T23:35:36.394047+0000","last_fullsized":"2024-09-10T23:35:36.394047+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405719+0000","last_change":"2024-09-10T23:35:36.400916+0000","last_active":"2024-09-10T23:35:36.405719+0000","last_peered":"2024-09-10T23:35:36.405719+0000","last_clean":"2024-09-10T23:35:36.405719+0000","last_became_active":"2024-09-10T23:35:36.400639+0000","last_became_peered":"2024-09-10T23:35:36.400639+0000","last_unstale":"2024-09-10T23:35:36.405719+0000","last_undegraded":"2024-09-10T23:35:36.405719+0000","last_fullsized":"2024-09-10T23:35:36.405719+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:47.327210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389265+0000","last_change":"2024-09-10T23:35:36.389265+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389265+0000","last_undegraded":"2024-09-10T23:35:36.389265+0000","last_fullsized":"2024-09-10T23:35:36.389265+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405406+0000","last_change":"2024-09-10T23:35:36.398766+0000","last_active":"2024-09-10T23:35:36.405406+0000","last_peered":"2024-09-10T23:35:36.405406+0000","last_clean":"2024-09-10T23:35:36.405406+0000","last_became_active":"2024-09-10T23:35:36.398473+0000","last_became_peered":"2024-09-10T23:35:36.398473+0000","last_unstale":"2024-09-10T23:35:36.405406+0000","last_undegraded":"2024-09-10T23:35:36.405406+0000","last_fullsized":"2024-09-10T23:35:36.405406+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:37.536960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388202+0000","last_change":"2024-09-10T23:35:36.388202+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388202+0000","last_undegraded":"2024-09-10T23:35:36.388202+0000","last_fullsized":"2024-09-10T23:35:36.388202+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.414301+0000","last_change":"2024-09-10T23:35:36.409772+0000","last_active":"2024-09-10T23:35:36.414301+0000","last_peered":"2024-09-10T23:35:36.414301+0000","last_clean":"2024-09-10T23:35:36.414301+0000","last_became_active":"2024-09-10T23:35:36.409432+0000","last_became_peered":"2024-09-10T23:35:36.409432+0000","last_unstale":"2024-09-10T23:35:36.414301+0000","last_undegraded":"2024-09-10T23:35:36.414301+0000","last_fullsized":"2024-09-10T23:35:36.414301+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:09.775738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391962+0000","last_change":"2024-09-10T23:35:36.391962+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391962+0000","last_undegraded":"2024-09-10T23:35:36.391962+0000","last_fullsized":"2024-09-10T23:35:36.391962+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.409418+0000","last_change":"2024-09-10T23:35:36.405162+0000","last_active":"2024-09-10T23:35:36.409418+0000","last_peered":"2024-09-10T23:35:36.409418+0000","last_clean":"2024-09-10T23:35:36.409418+0000","last_became_active":"2024-09-10T23:35:36.404891+0000","last_became_peered":"2024-09-10T23:35:36.404891+0000","last_unstale":"2024-09-10T23:35:36.409418+0000","last_undegraded":"2024-09-10T23:35:36.409418+0000","last_fullsized":"2024-09-10T23:35:36.409418+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:25.179787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388064+0000","last_change":"2024-09-10T23:35:36.388064+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388064+0000","last_undegraded":"2024-09-10T23:35:36.388064+0000","last_fullsized":"2024-09-10T23:35:36.388064+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.406216+0000","last_change":"2024-09-10T23:35:36.401149+0000","last_active":"2024-09-10T23:35:36.406216+0000","last_peered":"2024-09-10T23:35:36.406216+0000","last_clean":"2024-09-10T23:35:36.406216+0000","last_became_active":"2024-09-10T23:35:36.400884+0000","last_became_peered":"2024-09-10T23:35:36.400884+0000","last_unstale":"2024-09-10T23:35:36.406216+0000","last_undegraded":"2024-09-10T23:35:36.406216+0000","last_fullsized":"2024-09-10T23:35:36.406216+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:43.580364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388430+0000","last_change":"2024-09-10T23:35:36.388430+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388430+0000","last_undegraded":"2024-09-10T23:35:36.388430+0000","last_fullsized":"2024-09-10T23:35:36.388430+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407985+0000","last_change":"2024-09-10T23:35:36.406290+0000","last_active":"2024-09-10T23:35:36.407985+0000","last_peered":"2024-09-10T23:35:36.407985+0000","last_clean":"2024-09-10T23:35:36.407985+0000","last_became_active":"2024-09-10T23:35:36.406056+0000","last_became_peered":"2024-09-10T23:35:36.406056+0000","last_unstale":"2024-09-10T23:35:36.407985+0000","last_undegraded":"2024-09-10T23:35:36.407985+0000","last_fullsized":"2024-09-10T23:35:36.407985+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:59.329841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391377+0000","last_change":"2024-09-10T23:35:36.391377+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391377+0000","last_undegraded":"2024-09-10T23:35:36.391377+0000","last_fullsized":"2024-09-10T23:35:36.391377+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.410432+0000","last_change":"2024-09-10T23:35:36.406893+0000","last_active":"2024-09-10T23:35:36.410432+0000","last_peered":"2024-09-10T23:35:36.410432+0000","last_clean":"2024-09-10T23:35:36.410432+0000","last_became_active":"2024-09-10T23:35:36.406516+0000","last_became_peered":"2024-09-10T23:35:36.406516+0000","last_unstale":"2024-09-10T23:35:36.410432+0000","last_undegraded":"2024-09-10T23:35:36.410432+0000","last_fullsized":"2024-09-10T23:35:36.410432+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:13.080452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.393643+0000","last_change":"2024-09-10T23:35:36.393643+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.393643+0000","last_undegraded":"2024-09-10T23:35:36.393643+0000","last_fullsized":"2024-09-10T23:35:36.393643+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391082+0000","last_change":"2024-09-10T23:35:36.409344+0000","last_active":"2024-09-10T23:35:38.391082+0000","last_peered":"2024-09-10T23:35:38.391082+0000","last_clean":"2024-09-10T23:35:38.391082+0000","last_became_active":"2024-09-10T23:35:36.408975+0000","last_became_peered":"2024-09-10T23:35:36.408975+0000","last_unstale":"2024-09-10T23:35:38.391082+0000","last_undegraded":"2024-09-10T23:35:38.391082+0000","last_fullsized":"2024-09-10T23:35:38.391082+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:00.834693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.388544+0000","last_change":"2024-09-10T23:35:36.388544+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.388544+0000","last_undegraded":"2024-09-10T23:35:36.388544+0000","last_fullsized":"2024-09-10T23:35:36.388544+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405535+0000","last_change":"2024-09-10T23:35:36.400467+0000","last_active":"2024-09-10T23:35:36.405535+0000","last_peered":"2024-09-10T23:35:36.405535+0000","last_clean":"2024-09-10T23:35:36.405535+0000","last_became_active":"2024-09-10T23:35:36.400164+0000","last_became_peered":"2024-09-10T23:35:36.400164+0000","last_unstale":"2024-09-10T23:35:36.405535+0000","last_undegraded":"2024-09-10T23:35:36.405535+0000","last_fullsized":"2024-09-10T23:35:36.405535+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:57.102709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.391111+0000","last_change":"2024-09-10T23:35:36.391111+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.391111+0000","last_undegraded":"2024-09-10T23:35:36.391111+0000","last_fullsized":"2024-09-10T23:35:36.391111+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407753+0000","last_change":"2024-09-10T23:35:36.399341+0000","last_active":"2024-09-10T23:35:36.407753+0000","last_peered":"2024-09-10T23:35:36.407753+0000","last_clean":"2024-09-10T23:35:36.407753+0000","last_became_active":"2024-09-10T23:35:36.399064+0000","last_became_peered":"2024-09-10T23:35:36.399064+0000","last_unstale":"2024-09-10T23:35:36.407753+0000","last_undegraded":"2024-09-10T23:35:36.407753+0000","last_fullsized":"2024-09-10T23:35:36.407753+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:34.124846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390616+0000","last_change":"2024-09-10T23:35:37.403404+0000","last_active":"2024-09-10T23:35:38.390616+0000","last_peered":"2024-09-10T23:35:38.390616+0000","last_clean":"2024-09-10T23:35:38.390616+0000","last_became_active":"2024-09-10T23:35:37.400175+0000","last_became_peered":"2024-09-10T23:35:37.400175+0000","last_unstale":"2024-09-10T23:35:38.390616+0000","last_undegraded":"2024-09-10T23:35:38.390616+0000","last_fullsized":"2024-09-10T23:35:38.390616+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:54.308424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.414180+0000","last_change":"2024-09-10T23:35:36.410996+0000","last_active":"2024-09-10T23:35:36.414180+0000","last_peered":"2024-09-10T23:35:36.414180+0000","last_clean":"2024-09-10T23:35:36.414180+0000","last_became_active":"2024-09-10T23:35:36.410559+0000","last_became_peered":"2024-09-10T23:35:36.410559+0000","last_unstale":"2024-09-10T23:35:36.414180+0000","last_undegraded":"2024-09-10T23:35:36.414180+0000","last_fullsized":"2024-09-10T23:35:36.414180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:40.032192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.395931+0000","last_change":"2024-09-10T23:35:36.395931+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.395931+0000","last_undegraded":"2024-09-10T23:35:36.395931+0000","last_fullsized":"2024-09-10T23:35:36.395931+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405469+0000","last_change":"2024-09-10T23:35:36.399492+0000","last_active":"2024-09-10T23:35:36.405469+0000","last_peered":"2024-09-10T23:35:36.405469+0000","last_clean":"2024-09-10T23:35:36.405469+0000","last_became_active":"2024-09-10T23:35:36.397982+0000","last_became_peered":"2024-09-10T23:35:36.397982+0000","last_unstale":"2024-09-10T23:35:36.405469+0000","last_undegraded":"2024-09-10T23:35:36.405469+0000","last_fullsized":"2024-09-10T23:35:36.405469+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:56.422393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.389339+0000","last_change":"2024-09-10T23:35:36.389339+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.389339+0000","last_undegraded":"2024-09-10T23:35:36.389339+0000","last_fullsized":"2024-09-10T23:35:36.389339+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389803+0000","last_change":"2024-09-10T23:35:36.406500+0000","last_active":"2024-09-10T23:35:38.389803+0000","last_peered":"2024-09-10T23:35:38.389803+0000","last_clean":"2024-09-10T23:35:38.389803+0000","last_became_active":"2024-09-10T23:35:36.406222+0000","last_became_peered":"2024-09-10T23:35:36.406222+0000","last_unstale":"2024-09-10T23:35:38.389803+0000","last_undegraded":"2024-09-10T23:35:38.389803+0000","last_fullsized":"2024-09-10T23:35:38.389803+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:36.140228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.387662+0000","last_change":"2024-09-10T23:35:36.387662+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.387662+0000","last_undegraded":"2024-09-10T23:35:36.387662+0000","last_fullsized":"2024-09-10T23:35:36.387662+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.405662+0000","last_change":"2024-09-10T23:35:36.398978+0000","last_active":"2024-09-10T23:35:36.405662+0000","last_peered":"2024-09-10T23:35:36.405662+0000","last_clean":"2024-09-10T23:35:36.405662+0000","last_became_active":"2024-09-10T23:35:36.398751+0000","last_became_peered":"2024-09-10T23:35:36.398751+0000","last_unstale":"2024-09-10T23:35:36.405662+0000","last_undegraded":"2024-09-10T23:35:36.405662+0000","last_fullsized":"2024-09-10T23:35:36.405662+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:24.457146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-09-10T23:35:36.387645+0000","last_change":"2024-09-10T23:35:36.387645+0000","last_active":"2024-09-10T23:35:36.375658+0000","last_peered":"2024-09-10T23:35:36.375658+0000","last_clean":"2024-09-10T23:35:36.375658+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:35:36.387645+0000","last_undegraded":"2024-09-10T23:35:36.387645+0000","last_fullsized":"2024-09-10T23:35:36.387645+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.407690+0000","last_change":"2024-09-10T23:35:36.403531+0000","last_active":"2024-09-10T23:35:36.407690+0000","last_peered":"2024-09-10T23:35:36.407690+0000","last_clean":"2024-09-10T23:35:36.407690+0000","last_became_active":"2024-09-10T23:35:36.403263+0000","last_became_peered":"2024-09-10T23:35:36.403263+0000","last_unstale":"2024-09-10T23:35:36.407690+0000","last_undegraded":"2024-09-10T23:35:36.407690+0000","last_fullsized":"2024-09-10T23:35:36.407690+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:34.124846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":173,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T23:35:36.390482+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:35:36.390482+0000","last_peered":"2024-09-10T23:35:36.390482+0000","last_clean":"2024-09-10T23:35:36.390482+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:35:36.390482+0000","last_undegraded":"2024-09-10T23:35:36.390482+0000","last_fullsized":"2024-09-10T23:35:36.390482+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":651264,"data_stored":264592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28244,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343596,"statfs":{"total":96636764160,"available":96607842304,"internally_reserved":0,"allocated":1245184,"data_stored":854960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":264592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":270867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":264592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":854960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":270867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":692224,"data_stored":283964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:35:46.079 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:35:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.230+0000 7ff305800640 1 -- 172.21.15.94:0/1647801371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3000b2bb0 msgr2=0x7ff3000d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.230+0000 7ff305800640 1 --2- 172.21.15.94:0/1647801371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3000b2bb0 0x7ff3000d00a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009840 tx=0x7ff2ec01c700 comp rx=0 tx=0).stop 2024-09-10T23:35:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.230+0000 7ff305800640 1 -- 172.21.15.94:0/1647801371 shutdown_connections 2024-09-10T23:35:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.230+0000 7ff305800640 1 --2- 172.21.15.94:0/1647801371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3000b2bb0 0x7ff3000d00a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.230+0000 7ff305800640 1 --2- 172.21.15.94:0/1647801371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff30015f5c0 0x7ff3000cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.230+0000 7ff305800640 1 -- 172.21.15.94:0/1647801371 >> 172.21.15.94:0/1647801371 conn(0x7ff3000d1c60 msgr2=0x7ff3000d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.230+0000 7ff305800640 1 -- 172.21.15.94:0/1647801371 shutdown_connections 2024-09-10T23:35:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.230+0000 7ff305800640 1 -- 172.21.15.94:0/1647801371 wait complete. 2024-09-10T23:35:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff305800640 1 Processor -- start 2024-09-10T23:35:46.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff305800640 1 -- start start 2024-09-10T23:35:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff305800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3000b2bb0 0x7ff3000c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff305800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3000d05e0 0x7ff3001f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff305800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30015f5c0 0x7ff3000b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:46.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff305800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff300169250 con 0x7ff3000d05e0 2024-09-10T23:35:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff305800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff3001690d0 con 0x7ff30015f5c0 2024-09-10T23:35:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff305800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff3001693d0 con 0x7ff3000b2bb0 2024-09-10T23:35:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff2fe000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3000d05e0 0x7ff3001f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff2fe000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3000d05e0 0x7ff3001f64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44170/0 (socket says 172.21.15.94:44170) 2024-09-10T23:35:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff2fe000640 1 -- 172.21.15.94:0/2726011397 learned_addr learned my addr 172.21.15.94:0/2726011397 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff2fea00640 1 --2- 172.21.15.94:0/2726011397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3000b2bb0 0x7ff3000c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff2ff400640 1 --2- 172.21.15.94:0/2726011397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30015f5c0 0x7ff3000b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff2fe000640 1 -- 172.21.15.94:0/2726011397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3000b2bb0 msgr2=0x7ff3000c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff2fe000640 1 --2- 172.21.15.94:0/2726011397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3000b2bb0 0x7ff3000c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff2fe000640 1 -- 172.21.15.94:0/2726011397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30015f5c0 msgr2=0x7ff3000b6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff2fe000640 1 --2- 172.21.15.94:0/2726011397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30015f5c0 0x7ff3000b6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff2fe000640 1 -- 172.21.15.94:0/2726011397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff300154f40 con 0x7ff3000d05e0 2024-09-10T23:35:46.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff2ff400640 1 --2- 172.21.15.94:0/2726011397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30015f5c0 0x7ff3000b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:35:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff2fe000640 1 --2- 172.21.15.94:0/2726011397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3000d05e0 0x7ff3001f64c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec01adb0 tx=0x7ff2ec0a65d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff2f6a00640 1 -- 172.21.15.94:0/2726011397 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec004230 con 0x7ff3000d05e0 2024-09-10T23:35:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff2f6a00640 1 -- 172.21.15.94:0/2726011397 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2ec0bd070 con 0x7ff3000d05e0 2024-09-10T23:35:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff2f6a00640 1 -- 172.21.15.94:0/2726011397 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec0052d0 con 0x7ff3000d05e0 2024-09-10T23:35:46.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff305800640 1 -- 172.21.15.94:0/2726011397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3000c8570 con 0x7ff3000d05e0 2024-09-10T23:35:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff305800640 1 -- 172.21.15.94:0/2726011397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3000b5110 con 0x7ff3000d05e0 2024-09-10T23:35:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.234+0000 7ff305800640 1 -- 172.21.15.94:0/2726011397 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3000b15c0 con 0x7ff3000d05e0 2024-09-10T23:35:46.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.238+0000 7ff2f6a00640 1 -- 172.21.15.94:0/2726011397 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7ff2ec005470 con 0x7ff3000d05e0 2024-09-10T23:35:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.238+0000 7ff2f6a00640 1 --2- 172.21.15.94:0/2726011397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff2e0072790 0x7ff2e0074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.238+0000 7ff2f6a00640 1 -- 172.21.15.94:0/2726011397 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff2ec12e580 con 0x7ff3000d05e0 2024-09-10T23:35:46.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.238+0000 7ff2fea00640 1 --2- 172.21.15.94:0/2726011397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff2e0072790 0x7ff2e0074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:46.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.238+0000 7ff2fea00640 1 --2- 172.21.15.94:0/2726011397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff2e0072790 0x7ff2e0074c50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009e50 tx=0x7ff2f00095e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:46.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.242+0000 7ff2f6a00640 1 -- 172.21.15.94:0/2726011397 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff2ec016410 con 0x7ff3000d05e0 2024-09-10T23:35:46.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.402+0000 7ff305800640 1 -- 172.21.15.94:0/2726011397 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3000b5a40 con 0x7ff2e0072790 2024-09-10T23:35:46.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.410+0000 7ff2f6a00640 1 -- 172.21.15.94:0/2726011397 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7ff3000b5a40 con 0x7ff2e0072790 2024-09-10T23:35:46.413 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:35:46.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.414+0000 7ff305800640 1 -- 172.21.15.94:0/2726011397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff2e0072790 msgr2=0x7ff2e0074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.414+0000 7ff305800640 1 --2- 172.21.15.94:0/2726011397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff2e0072790 0x7ff2e0074c50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009e50 tx=0x7ff2f00095e0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.414+0000 7ff305800640 1 -- 172.21.15.94:0/2726011397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3000d05e0 msgr2=0x7ff3001f64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.414+0000 7ff305800640 1 --2- 172.21.15.94:0/2726011397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3000d05e0 0x7ff3001f64c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec01adb0 tx=0x7ff2ec0a65d0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.414+0000 7ff305800640 1 -- 172.21.15.94:0/2726011397 shutdown_connections 2024-09-10T23:35:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.414+0000 7ff305800640 1 --2- 172.21.15.94:0/2726011397 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff2e0072790 0x7ff2e0074c50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.414+0000 7ff305800640 1 --2- 172.21.15.94:0/2726011397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff30015f5c0 0x7ff3000b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.414+0000 7ff305800640 1 --2- 172.21.15.94:0/2726011397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3000d05e0 0x7ff3001f64c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.414+0000 7ff305800640 1 --2- 172.21.15.94:0/2726011397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3000b2bb0 0x7ff3000c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.414+0000 7ff305800640 1 -- 172.21.15.94:0/2726011397 >> 172.21.15.94:0/2726011397 conn(0x7ff3000d1c60 msgr2=0x7ff30015b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.414+0000 7ff305800640 1 -- 172.21.15.94:0/2726011397 shutdown_connections 2024-09-10T23:35:46.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.414+0000 7ff305800640 1 -- 172.21.15.94:0/2726011397 wait complete. 2024-09-10T23:35:46.434 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-09-10T23:35:45.389685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251968,"kb_used_data":6600,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754722752,"statfs":{"total":773094113280,"available":772836098048,"internally_reserved":0,"allocated":6758400,"data_stored":3457092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003984"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391197+0000","last_change":"2024-09-10T23:35:37.411838+0000","last_active":"2024-09-10T23:35:38.391197+0000","last_peered":"2024-09-10T23:35:38.391197+0000","last_clean":"2024-09-10T23:35:38.391197+0000","last_became_active":"2024-09-10T23:35:37.411168+0000","last_became_peered":"2024-09-10T23:35:37.411168+0000","last_unstale":"2024-09-10T23:35:38.391197+0000","last_undegraded":"2024-09-10T23:35:38.391197+0000","last_fullsized":"2024-09-10T23:35:38.391197+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:27.635555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.728793+0000","last_change":"2024-09-10T23:35:37.416849+0000","last_active":"2024-09-10T23:35:38.728793+0000","last_peered":"2024-09-10T23:35:38.728793+0000","last_clean":"2024-09-10T23:35:38.728793+0000","last_became_active":"2024-09-10T23:35:37.416360+0000","last_became_peered":"2024-09-10T23:35:37.416360+0000","last_unstale":"2024-09-10T23:35:38.728793+0000","last_undegraded":"2024-09-10T23:35:38.728793+0000","last_fullsized":"2024-09-10T23:35:38.728793+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:32.845136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.396277+0000","last_change":"2024-09-10T23:35:37.403344+0000","last_active":"2024-09-10T23:35:39.396277+0000","last_peered":"2024-09-10T23:35:39.396277+0000","last_clean":"2024-09-10T23:35:39.396277+0000","last_became_active":"2024-09-10T23:35:37.403011+0000","last_became_peered":"2024-09-10T23:35:37.403011+0000","last_unstale":"2024-09-10T23:35:39.396277+0000","last_undegraded":"2024-09-10T23:35:39.396277+0000","last_fullsized":"2024-09-10T23:35:39.396277+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:25.168116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.382950+0000","last_change":"2024-09-10T23:35:37.419954+0000","last_active":"2024-09-10T23:35:39.382950+0000","last_peered":"2024-09-10T23:35:39.382950+0000","last_clean":"2024-09-10T23:35:39.382950+0000","last_became_active":"2024-09-10T23:35:37.419704+0000","last_became_peered":"2024-09-10T23:35:37.419704+0000","last_unstale":"2024-09-10T23:35:39.382950+0000","last_undegraded":"2024-09-10T23:35:39.382950+0000","last_fullsized":"2024-09-10T23:35:39.382950+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:48.969870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390631+0000","last_change":"2024-09-10T23:35:37.422878+0000","last_active":"2024-09-10T23:35:38.390631+0000","last_peered":"2024-09-10T23:35:38.390631+0000","last_clean":"2024-09-10T23:35:38.390631+0000","last_became_active":"2024-09-10T23:35:37.422602+0000","last_became_peered":"2024-09-10T23:35:37.422602+0000","last_unstale":"2024-09-10T23:35:38.390631+0000","last_undegraded":"2024-09-10T23:35:38.390631+0000","last_fullsized":"2024-09-10T23:35:38.390631+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:17.965755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390045+0000","last_change":"2024-09-10T23:35:37.401854+0000","last_active":"2024-09-10T23:35:38.390045+0000","last_peered":"2024-09-10T23:35:38.390045+0000","last_clean":"2024-09-10T23:35:38.390045+0000","last_became_active":"2024-09-10T23:35:37.400521+0000","last_became_peered":"2024-09-10T23:35:37.400521+0000","last_unstale":"2024-09-10T23:35:38.390045+0000","last_undegraded":"2024-09-10T23:35:38.390045+0000","last_fullsized":"2024-09-10T23:35:38.390045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:48.537421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.382644+0000","last_change":"2024-09-10T23:35:37.417972+0000","last_active":"2024-09-10T23:35:39.382644+0000","last_peered":"2024-09-10T23:35:39.382644+0000","last_clean":"2024-09-10T23:35:39.382644+0000","last_became_active":"2024-09-10T23:35:37.417539+0000","last_became_peered":"2024-09-10T23:35:37.417539+0000","last_unstale":"2024-09-10T23:35:39.382644+0000","last_undegraded":"2024-09-10T23:35:39.382644+0000","last_fullsized":"2024-09-10T23:35:39.382644+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:59.461964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.838702+0000","last_change":"2024-09-10T23:35:37.402644+0000","last_active":"2024-09-10T23:35:38.838702+0000","last_peered":"2024-09-10T23:35:38.838702+0000","last_clean":"2024-09-10T23:35:38.838702+0000","last_became_active":"2024-09-10T23:35:37.402392+0000","last_became_peered":"2024-09-10T23:35:37.402392+0000","last_unstale":"2024-09-10T23:35:38.838702+0000","last_undegraded":"2024-09-10T23:35:38.838702+0000","last_fullsized":"2024-09-10T23:35:38.838702+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:54.502217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.388487+0000","last_change":"2024-09-10T23:35:37.414390+0000","last_active":"2024-09-10T23:35:38.388487+0000","last_peered":"2024-09-10T23:35:38.388487+0000","last_clean":"2024-09-10T23:35:38.388487+0000","last_became_active":"2024-09-10T23:35:37.414097+0000","last_became_peered":"2024-09-10T23:35:37.414097+0000","last_unstale":"2024-09-10T23:35:38.388487+0000","last_undegraded":"2024-09-10T23:35:38.388487+0000","last_fullsized":"2024-09-10T23:35:38.388487+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:01.101863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390598+0000","last_change":"2024-09-10T23:35:37.405331+0000","last_active":"2024-09-10T23:35:38.390598+0000","last_peered":"2024-09-10T23:35:38.390598+0000","last_clean":"2024-09-10T23:35:38.390598+0000","last_became_active":"2024-09-10T23:35:37.404010+0000","last_became_peered":"2024-09-10T23:35:37.404010+0000","last_unstale":"2024-09-10T23:35:38.390598+0000","last_undegraded":"2024-09-10T23:35:38.390598+0000","last_fullsized":"2024-09-10T23:35:38.390598+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:30.787726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390324+0000","last_change":"2024-09-10T23:35:37.411543+0000","last_active":"2024-09-10T23:35:38.390324+0000","last_peered":"2024-09-10T23:35:38.390324+0000","last_clean":"2024-09-10T23:35:38.390324+0000","last_became_active":"2024-09-10T23:35:37.411231+0000","last_became_peered":"2024-09-10T23:35:37.411231+0000","last_unstale":"2024-09-10T23:35:38.390324+0000","last_undegraded":"2024-09-10T23:35:38.390324+0000","last_fullsized":"2024-09-10T23:35:38.390324+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:18.604994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.388185+0000","last_change":"2024-09-10T23:35:37.407602+0000","last_active":"2024-09-10T23:35:38.388185+0000","last_peered":"2024-09-10T23:35:38.388185+0000","last_clean":"2024-09-10T23:35:38.388185+0000","last_became_active":"2024-09-10T23:35:37.406369+0000","last_became_peered":"2024-09-10T23:35:37.406369+0000","last_unstale":"2024-09-10T23:35:38.388185+0000","last_undegraded":"2024-09-10T23:35:38.388185+0000","last_fullsized":"2024-09-10T23:35:38.388185+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:39.343527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.096123+0000","last_change":"2024-09-10T23:35:37.410344+0000","last_active":"2024-09-10T23:35:39.096123+0000","last_peered":"2024-09-10T23:35:39.096123+0000","last_clean":"2024-09-10T23:35:39.096123+0000","last_became_active":"2024-09-10T23:35:37.410086+0000","last_became_peered":"2024-09-10T23:35:37.410086+0000","last_unstale":"2024-09-10T23:35:39.096123+0000","last_undegraded":"2024-09-10T23:35:39.096123+0000","last_fullsized":"2024-09-10T23:35:39.096123+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:40.740634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.395794+0000","last_change":"2024-09-10T23:35:37.407274+0000","last_active":"2024-09-10T23:35:39.395794+0000","last_peered":"2024-09-10T23:35:39.395794+0000","last_clean":"2024-09-10T23:35:39.395794+0000","last_became_active":"2024-09-10T23:35:37.406978+0000","last_became_peered":"2024-09-10T23:35:37.406978+0000","last_unstale":"2024-09-10T23:35:39.395794+0000","last_undegraded":"2024-09-10T23:35:39.395794+0000","last_fullsized":"2024-09-10T23:35:39.395794+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:27.646388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.095828+0000","last_change":"2024-09-10T23:35:37.392893+0000","last_active":"2024-09-10T23:35:39.095828+0000","last_peered":"2024-09-10T23:35:39.095828+0000","last_clean":"2024-09-10T23:35:39.095828+0000","last_became_active":"2024-09-10T23:35:37.392589+0000","last_became_peered":"2024-09-10T23:35:37.392589+0000","last_unstale":"2024-09-10T23:35:39.095828+0000","last_undegraded":"2024-09-10T23:35:39.095828+0000","last_fullsized":"2024-09-10T23:35:39.095828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:38.447923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389559+0000","last_change":"2024-09-10T23:35:37.421193+0000","last_active":"2024-09-10T23:35:38.389559+0000","last_peered":"2024-09-10T23:35:38.389559+0000","last_clean":"2024-09-10T23:35:38.389559+0000","last_became_active":"2024-09-10T23:35:37.420909+0000","last_became_peered":"2024-09-10T23:35:37.420909+0000","last_unstale":"2024-09-10T23:35:38.389559+0000","last_undegraded":"2024-09-10T23:35:38.389559+0000","last_fullsized":"2024-09-10T23:35:38.389559+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:31.257242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391217+0000","last_change":"2024-09-10T23:35:37.404324+0000","last_active":"2024-09-10T23:35:38.391217+0000","last_peered":"2024-09-10T23:35:38.391217+0000","last_clean":"2024-09-10T23:35:38.391217+0000","last_became_active":"2024-09-10T23:35:37.404042+0000","last_became_peered":"2024-09-10T23:35:37.404042+0000","last_unstale":"2024-09-10T23:35:38.391217+0000","last_undegraded":"2024-09-10T23:35:38.391217+0000","last_fullsized":"2024-09-10T23:35:38.391217+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:31.250579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.396175+0000","last_change":"2024-09-10T23:35:37.417116+0000","last_active":"2024-09-10T23:35:39.396175+0000","last_peered":"2024-09-10T23:35:39.396175+0000","last_clean":"2024-09-10T23:35:39.396175+0000","last_became_active":"2024-09-10T23:35:37.416859+0000","last_became_peered":"2024-09-10T23:35:37.416859+0000","last_unstale":"2024-09-10T23:35:39.396175+0000","last_undegraded":"2024-09-10T23:35:39.396175+0000","last_fullsized":"2024-09-10T23:35:39.396175+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:09.157870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.728609+0000","last_change":"2024-09-10T23:35:37.416905+0000","last_active":"2024-09-10T23:35:38.728609+0000","last_peered":"2024-09-10T23:35:38.728609+0000","last_clean":"2024-09-10T23:35:38.728609+0000","last_became_active":"2024-09-10T23:35:37.416244+0000","last_became_peered":"2024-09-10T23:35:37.416244+0000","last_unstale":"2024-09-10T23:35:38.728609+0000","last_undegraded":"2024-09-10T23:35:38.728609+0000","last_fullsized":"2024-09-10T23:35:38.728609+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:23.267604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390344+0000","last_change":"2024-09-10T23:35:37.410895+0000","last_active":"2024-09-10T23:35:38.390344+0000","last_peered":"2024-09-10T23:35:38.390344+0000","last_clean":"2024-09-10T23:35:38.390344+0000","last_became_active":"2024-09-10T23:35:37.410549+0000","last_became_peered":"2024-09-10T23:35:37.410549+0000","last_unstale":"2024-09-10T23:35:38.390344+0000","last_undegraded":"2024-09-10T23:35:38.390344+0000","last_fullsized":"2024-09-10T23:35:38.390344+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:49.342750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.095984+0000","last_change":"2024-09-10T23:35:37.404765+0000","last_active":"2024-09-10T23:35:39.095984+0000","last_peered":"2024-09-10T23:35:39.095984+0000","last_clean":"2024-09-10T23:35:39.095984+0000","last_became_active":"2024-09-10T23:35:37.402548+0000","last_became_peered":"2024-09-10T23:35:37.402548+0000","last_unstale":"2024-09-10T23:35:39.095984+0000","last_undegraded":"2024-09-10T23:35:39.095984+0000","last_fullsized":"2024-09-10T23:35:39.095984+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:04:16.256444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389860+0000","last_change":"2024-09-10T23:35:37.419510+0000","last_active":"2024-09-10T23:35:38.389860+0000","last_peered":"2024-09-10T23:35:38.389860+0000","last_clean":"2024-09-10T23:35:38.389860+0000","last_became_active":"2024-09-10T23:35:37.419248+0000","last_became_peered":"2024-09-10T23:35:37.419248+0000","last_unstale":"2024-09-10T23:35:38.389860+0000","last_undegraded":"2024-09-10T23:35:38.389860+0000","last_fullsized":"2024-09-10T23:35:38.389860+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:08.841891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.382835+0000","last_change":"2024-09-10T23:35:37.421898+0000","last_active":"2024-09-10T23:35:39.382835+0000","last_peered":"2024-09-10T23:35:39.382835+0000","last_clean":"2024-09-10T23:35:39.382835+0000","last_became_active":"2024-09-10T23:35:37.421494+0000","last_became_peered":"2024-09-10T23:35:37.421494+0000","last_unstale":"2024-09-10T23:35:39.382835+0000","last_undegraded":"2024-09-10T23:35:39.382835+0000","last_fullsized":"2024-09-10T23:35:39.382835+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:55.660236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.387859+0000","last_change":"2024-09-10T23:35:37.421815+0000","last_active":"2024-09-10T23:35:38.387859+0000","last_peered":"2024-09-10T23:35:38.387859+0000","last_clean":"2024-09-10T23:35:38.387859+0000","last_became_active":"2024-09-10T23:35:37.420483+0000","last_became_peered":"2024-09-10T23:35:37.420483+0000","last_unstale":"2024-09-10T23:35:38.387859+0000","last_undegraded":"2024-09-10T23:35:38.387859+0000","last_fullsized":"2024-09-10T23:35:38.387859+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:02.641647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.728850+0000","last_change":"2024-09-10T23:35:37.422351+0000","last_active":"2024-09-10T23:35:38.728850+0000","last_peered":"2024-09-10T23:35:38.728850+0000","last_clean":"2024-09-10T23:35:38.728850+0000","last_became_active":"2024-09-10T23:35:37.422098+0000","last_became_peered":"2024-09-10T23:35:37.422098+0000","last_unstale":"2024-09-10T23:35:38.728850+0000","last_undegraded":"2024-09-10T23:35:38.728850+0000","last_fullsized":"2024-09-10T23:35:38.728850+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:40.434547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.096285+0000","last_change":"2024-09-10T23:35:37.420340+0000","last_active":"2024-09-10T23:35:39.096285+0000","last_peered":"2024-09-10T23:35:39.096285+0000","last_clean":"2024-09-10T23:35:39.096285+0000","last_became_active":"2024-09-10T23:35:37.420049+0000","last_became_peered":"2024-09-10T23:35:37.420049+0000","last_unstale":"2024-09-10T23:35:39.096285+0000","last_undegraded":"2024-09-10T23:35:39.096285+0000","last_fullsized":"2024-09-10T23:35:39.096285+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:10.126773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.728837+0000","last_change":"2024-09-10T23:35:37.424291+0000","last_active":"2024-09-10T23:35:38.728837+0000","last_peered":"2024-09-10T23:35:38.728837+0000","last_clean":"2024-09-10T23:35:38.728837+0000","last_became_active":"2024-09-10T23:35:37.424000+0000","last_became_peered":"2024-09-10T23:35:37.424000+0000","last_unstale":"2024-09-10T23:35:38.728837+0000","last_undegraded":"2024-09-10T23:35:38.728837+0000","last_fullsized":"2024-09-10T23:35:38.728837+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:47.751380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.838882+0000","last_change":"2024-09-10T23:35:37.408828+0000","last_active":"2024-09-10T23:35:38.838882+0000","last_peered":"2024-09-10T23:35:38.838882+0000","last_clean":"2024-09-10T23:35:38.838882+0000","last_became_active":"2024-09-10T23:35:37.408529+0000","last_became_peered":"2024-09-10T23:35:37.408529+0000","last_unstale":"2024-09-10T23:35:38.838882+0000","last_undegraded":"2024-09-10T23:35:38.838882+0000","last_fullsized":"2024-09-10T23:35:38.838882+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:16.293480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.838785+0000","last_change":"2024-09-10T23:35:37.414976+0000","last_active":"2024-09-10T23:35:38.838785+0000","last_peered":"2024-09-10T23:35:38.838785+0000","last_clean":"2024-09-10T23:35:38.838785+0000","last_became_active":"2024-09-10T23:35:37.414428+0000","last_became_peered":"2024-09-10T23:35:37.414428+0000","last_unstale":"2024-09-10T23:35:38.838785+0000","last_undegraded":"2024-09-10T23:35:38.838785+0000","last_fullsized":"2024-09-10T23:35:38.838785+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:54.887009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.382722+0000","last_change":"2024-09-10T23:35:37.420486+0000","last_active":"2024-09-10T23:35:39.382722+0000","last_peered":"2024-09-10T23:35:39.382722+0000","last_clean":"2024-09-10T23:35:39.382722+0000","last_became_active":"2024-09-10T23:35:37.418668+0000","last_became_peered":"2024-09-10T23:35:37.418668+0000","last_unstale":"2024-09-10T23:35:39.382722+0000","last_undegraded":"2024-09-10T23:35:39.382722+0000","last_fullsized":"2024-09-10T23:35:39.382722+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:31.979808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.728535+0000","last_change":"2024-09-10T23:35:37.409042+0000","last_active":"2024-09-10T23:35:38.728535+0000","last_peered":"2024-09-10T23:35:38.728535+0000","last_clean":"2024-09-10T23:35:38.728535+0000","last_became_active":"2024-09-10T23:35:37.408783+0000","last_became_peered":"2024-09-10T23:35:37.408783+0000","last_unstale":"2024-09-10T23:35:38.728535+0000","last_undegraded":"2024-09-10T23:35:38.728535+0000","last_fullsized":"2024-09-10T23:35:38.728535+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:05.761817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389578+0000","last_change":"2024-09-10T23:35:37.410535+0000","last_active":"2024-09-10T23:35:38.389578+0000","last_peered":"2024-09-10T23:35:38.389578+0000","last_clean":"2024-09-10T23:35:38.389578+0000","last_became_active":"2024-09-10T23:35:37.408303+0000","last_became_peered":"2024-09-10T23:35:37.408303+0000","last_unstale":"2024-09-10T23:35:38.389578+0000","last_undegraded":"2024-09-10T23:35:38.389578+0000","last_fullsized":"2024-09-10T23:35:38.389578+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:57.997246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391188+0000","last_change":"2024-09-10T23:35:37.405530+0000","last_active":"2024-09-10T23:35:38.391188+0000","last_peered":"2024-09-10T23:35:38.391188+0000","last_clean":"2024-09-10T23:35:38.391188+0000","last_became_active":"2024-09-10T23:35:37.402795+0000","last_became_peered":"2024-09-10T23:35:37.402795+0000","last_unstale":"2024-09-10T23:35:38.391188+0000","last_undegraded":"2024-09-10T23:35:38.391188+0000","last_fullsized":"2024-09-10T23:35:38.391188+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:15.871163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":19,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390848+0000","last_change":"2024-09-10T23:35:36.401576+0000","last_active":"2024-09-10T23:35:38.390848+0000","last_peered":"2024-09-10T23:35:38.390848+0000","last_clean":"2024-09-10T23:35:38.390848+0000","last_became_active":"2024-09-10T23:35:36.400780+0000","last_became_peered":"2024-09-10T23:35:36.400780+0000","last_unstale":"2024-09-10T23:35:38.390848+0000","last_undegraded":"2024-09-10T23:35:38.390848+0000","last_fullsized":"2024-09-10T23:35:38.390848+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:36:02.665179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391401+0000","last_change":"2024-09-10T23:35:37.411543+0000","last_active":"2024-09-10T23:35:38.391401+0000","last_peered":"2024-09-10T23:35:38.391401+0000","last_clean":"2024-09-10T23:35:38.391401+0000","last_became_active":"2024-09-10T23:35:37.410807+0000","last_became_peered":"2024-09-10T23:35:37.410807+0000","last_unstale":"2024-09-10T23:35:38.391401+0000","last_undegraded":"2024-09-10T23:35:38.391401+0000","last_fullsized":"2024-09-10T23:35:38.391401+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:52.749748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.728759+0000","last_change":"2024-09-10T23:35:36.406745+0000","last_active":"2024-09-10T23:35:38.728759+0000","last_peered":"2024-09-10T23:35:38.728759+0000","last_clean":"2024-09-10T23:35:38.728759+0000","last_became_active":"2024-09-10T23:35:36.406460+0000","last_became_peered":"2024-09-10T23:35:36.406460+0000","last_unstale":"2024-09-10T23:35:38.728759+0000","last_undegraded":"2024-09-10T23:35:38.728759+0000","last_fullsized":"2024-09-10T23:35:38.728759+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:20.099481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.838932+0000","last_change":"2024-09-10T23:35:37.413235+0000","last_active":"2024-09-10T23:35:38.838932+0000","last_peered":"2024-09-10T23:35:38.838932+0000","last_clean":"2024-09-10T23:35:38.838932+0000","last_became_active":"2024-09-10T23:35:37.412948+0000","last_became_peered":"2024-09-10T23:35:37.412948+0000","last_unstale":"2024-09-10T23:35:38.838932+0000","last_undegraded":"2024-09-10T23:35:38.838932+0000","last_fullsized":"2024-09-10T23:35:38.838932+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:35:11.974402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390807+0000","last_change":"2024-09-10T23:35:36.409095+0000","last_active":"2024-09-10T23:35:38.390807+0000","last_peered":"2024-09-10T23:35:38.390807+0000","last_clean":"2024-09-10T23:35:38.390807+0000","last_became_active":"2024-09-10T23:35:36.408811+0000","last_became_peered":"2024-09-10T23:35:36.408811+0000","last_unstale":"2024-09-10T23:35:38.390807+0000","last_undegraded":"2024-09-10T23:35:38.390807+0000","last_fullsized":"2024-09-10T23:35:38.390807+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:47.477496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390766+0000","last_change":"2024-09-10T23:35:37.413035+0000","last_active":"2024-09-10T23:35:38.390766+0000","last_peered":"2024-09-10T23:35:38.390766+0000","last_clean":"2024-09-10T23:35:38.390766+0000","last_became_active":"2024-09-10T23:35:37.412781+0000","last_became_peered":"2024-09-10T23:35:37.412781+0000","last_unstale":"2024-09-10T23:35:38.390766+0000","last_undegraded":"2024-09-10T23:35:38.390766+0000","last_fullsized":"2024-09-10T23:35:38.390766+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:11.135478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.383090+0000","last_change":"2024-09-10T23:35:36.396384+0000","last_active":"2024-09-10T23:35:39.383090+0000","last_peered":"2024-09-10T23:35:39.383090+0000","last_clean":"2024-09-10T23:35:39.383090+0000","last_became_active":"2024-09-10T23:35:36.396085+0000","last_became_peered":"2024-09-10T23:35:36.396085+0000","last_unstale":"2024-09-10T23:35:39.383090+0000","last_undegraded":"2024-09-10T23:35:39.383090+0000","last_fullsized":"2024-09-10T23:35:39.383090+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:47.163552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.838725+0000","last_change":"2024-09-10T23:35:37.408896+0000","last_active":"2024-09-10T23:35:38.838725+0000","last_peered":"2024-09-10T23:35:38.838725+0000","last_clean":"2024-09-10T23:35:38.838725+0000","last_became_active":"2024-09-10T23:35:37.408005+0000","last_became_peered":"2024-09-10T23:35:37.408005+0000","last_unstale":"2024-09-10T23:35:38.838725+0000","last_undegraded":"2024-09-10T23:35:38.838725+0000","last_fullsized":"2024-09-10T23:35:38.838725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:48.166972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.395930+0000","last_change":"2024-09-10T23:35:36.404943+0000","last_active":"2024-09-10T23:35:39.395930+0000","last_peered":"2024-09-10T23:35:39.395930+0000","last_clean":"2024-09-10T23:35:39.395930+0000","last_became_active":"2024-09-10T23:35:36.404615+0000","last_became_peered":"2024-09-10T23:35:36.404615+0000","last_unstale":"2024-09-10T23:35:39.395930+0000","last_undegraded":"2024-09-10T23:35:39.395930+0000","last_fullsized":"2024-09-10T23:35:39.395930+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:26.848151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389848+0000","last_change":"2024-09-10T23:35:37.417426+0000","last_active":"2024-09-10T23:35:38.389848+0000","last_peered":"2024-09-10T23:35:38.389848+0000","last_clean":"2024-09-10T23:35:38.389848+0000","last_became_active":"2024-09-10T23:35:37.417139+0000","last_became_peered":"2024-09-10T23:35:37.417139+0000","last_unstale":"2024-09-10T23:35:38.389848+0000","last_undegraded":"2024-09-10T23:35:38.389848+0000","last_fullsized":"2024-09-10T23:35:38.389848+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:57.376554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390698+0000","last_change":"2024-09-10T23:35:36.408437+0000","last_active":"2024-09-10T23:35:38.390698+0000","last_peered":"2024-09-10T23:35:38.390698+0000","last_clean":"2024-09-10T23:35:38.390698+0000","last_became_active":"2024-09-10T23:35:36.408156+0000","last_became_peered":"2024-09-10T23:35:36.408156+0000","last_unstale":"2024-09-10T23:35:38.390698+0000","last_undegraded":"2024-09-10T23:35:38.390698+0000","last_fullsized":"2024-09-10T23:35:38.390698+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:01.998360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.395766+0000","last_change":"2024-09-10T23:35:37.419914+0000","last_active":"2024-09-10T23:35:39.395766+0000","last_peered":"2024-09-10T23:35:39.395766+0000","last_clean":"2024-09-10T23:35:39.395766+0000","last_became_active":"2024-09-10T23:35:37.419585+0000","last_became_peered":"2024-09-10T23:35:37.419585+0000","last_unstale":"2024-09-10T23:35:39.395766+0000","last_undegraded":"2024-09-10T23:35:39.395766+0000","last_fullsized":"2024-09-10T23:35:39.395766+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:31.787781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.387902+0000","last_change":"2024-09-10T23:35:36.401028+0000","last_active":"2024-09-10T23:35:38.387902+0000","last_peered":"2024-09-10T23:35:38.387902+0000","last_clean":"2024-09-10T23:35:38.387902+0000","last_became_active":"2024-09-10T23:35:36.400743+0000","last_became_peered":"2024-09-10T23:35:36.400743+0000","last_unstale":"2024-09-10T23:35:38.387902+0000","last_undegraded":"2024-09-10T23:35:38.387902+0000","last_fullsized":"2024-09-10T23:35:38.387902+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:54.863172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389654+0000","last_change":"2024-09-10T23:35:37.422945+0000","last_active":"2024-09-10T23:35:38.389654+0000","last_peered":"2024-09-10T23:35:38.389654+0000","last_clean":"2024-09-10T23:35:38.389654+0000","last_became_active":"2024-09-10T23:35:37.422648+0000","last_became_peered":"2024-09-10T23:35:37.422648+0000","last_unstale":"2024-09-10T23:35:38.389654+0000","last_undegraded":"2024-09-10T23:35:38.389654+0000","last_fullsized":"2024-09-10T23:35:38.389654+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:10.705976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.095767+0000","last_change":"2024-09-10T23:35:36.401434+0000","last_active":"2024-09-10T23:35:39.095767+0000","last_peered":"2024-09-10T23:35:39.095767+0000","last_clean":"2024-09-10T23:35:39.095767+0000","last_became_active":"2024-09-10T23:35:36.401155+0000","last_became_peered":"2024-09-10T23:35:36.401155+0000","last_unstale":"2024-09-10T23:35:39.095767+0000","last_undegraded":"2024-09-10T23:35:39.095767+0000","last_fullsized":"2024-09-10T23:35:39.095767+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:58.666819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.096288+0000","last_change":"2024-09-10T23:35:37.406681+0000","last_active":"2024-09-10T23:35:39.096288+0000","last_peered":"2024-09-10T23:35:39.096288+0000","last_clean":"2024-09-10T23:35:39.096288+0000","last_became_active":"2024-09-10T23:35:37.406349+0000","last_became_peered":"2024-09-10T23:35:37.406349+0000","last_unstale":"2024-09-10T23:35:39.096288+0000","last_undegraded":"2024-09-10T23:35:39.096288+0000","last_fullsized":"2024-09-10T23:35:39.096288+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:31.176909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.388521+0000","last_change":"2024-09-10T23:35:36.405013+0000","last_active":"2024-09-10T23:35:38.388521+0000","last_peered":"2024-09-10T23:35:38.388521+0000","last_clean":"2024-09-10T23:35:38.388521+0000","last_became_active":"2024-09-10T23:35:36.404715+0000","last_became_peered":"2024-09-10T23:35:36.404715+0000","last_unstale":"2024-09-10T23:35:38.388521+0000","last_undegraded":"2024-09-10T23:35:38.388521+0000","last_fullsized":"2024-09-10T23:35:38.388521+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:19.816523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.396224+0000","last_change":"2024-09-10T23:35:37.411032+0000","last_active":"2024-09-10T23:35:39.396224+0000","last_peered":"2024-09-10T23:35:39.396224+0000","last_clean":"2024-09-10T23:35:39.396224+0000","last_became_active":"2024-09-10T23:35:37.410538+0000","last_became_peered":"2024-09-10T23:35:37.410538+0000","last_unstale":"2024-09-10T23:35:39.396224+0000","last_undegraded":"2024-09-10T23:35:39.396224+0000","last_fullsized":"2024-09-10T23:35:39.396224+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:57.997246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.838946+0000","last_change":"2024-09-10T23:35:36.402529+0000","last_active":"2024-09-10T23:35:38.838946+0000","last_peered":"2024-09-10T23:35:38.838946+0000","last_clean":"2024-09-10T23:35:38.838946+0000","last_became_active":"2024-09-10T23:35:36.402270+0000","last_became_peered":"2024-09-10T23:35:36.402270+0000","last_unstale":"2024-09-10T23:35:38.838946+0000","last_undegraded":"2024-09-10T23:35:38.838946+0000","last_fullsized":"2024-09-10T23:35:38.838946+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:37.531017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.839206+0000","last_change":"2024-09-10T23:35:37.412127+0000","last_active":"2024-09-10T23:35:38.839206+0000","last_peered":"2024-09-10T23:35:38.839206+0000","last_clean":"2024-09-10T23:35:38.839206+0000","last_became_active":"2024-09-10T23:35:37.409186+0000","last_became_peered":"2024-09-10T23:35:37.409186+0000","last_unstale":"2024-09-10T23:35:38.839206+0000","last_undegraded":"2024-09-10T23:35:38.839206+0000","last_fullsized":"2024-09-10T23:35:38.839206+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:13.333799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.383246+0000","last_change":"2024-09-10T23:35:36.402631+0000","last_active":"2024-09-10T23:35:39.383246+0000","last_peered":"2024-09-10T23:35:39.383246+0000","last_clean":"2024-09-10T23:35:39.383246+0000","last_became_active":"2024-09-10T23:35:36.399947+0000","last_became_peered":"2024-09-10T23:35:36.399947+0000","last_unstale":"2024-09-10T23:35:39.383246+0000","last_undegraded":"2024-09-10T23:35:39.383246+0000","last_fullsized":"2024-09-10T23:35:39.383246+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:34.521774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.383111+0000","last_change":"2024-09-10T23:35:37.417391+0000","last_active":"2024-09-10T23:35:39.383111+0000","last_peered":"2024-09-10T23:35:39.383111+0000","last_clean":"2024-09-10T23:35:39.383111+0000","last_became_active":"2024-09-10T23:35:37.417079+0000","last_became_peered":"2024-09-10T23:35:37.417079+0000","last_unstale":"2024-09-10T23:35:39.383111+0000","last_undegraded":"2024-09-10T23:35:39.383111+0000","last_fullsized":"2024-09-10T23:35:39.383111+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:18.044044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":21,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.096054+0000","last_change":"2024-09-10T23:35:36.405711+0000","last_active":"2024-09-10T23:35:39.096054+0000","last_peered":"2024-09-10T23:35:39.096054+0000","last_clean":"2024-09-10T23:35:39.096054+0000","last_became_active":"2024-09-10T23:35:36.405455+0000","last_became_peered":"2024-09-10T23:35:36.405455+0000","last_unstale":"2024-09-10T23:35:39.096054+0000","last_undegraded":"2024-09-10T23:35:39.096054+0000","last_fullsized":"2024-09-10T23:35:39.096054+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:01.366584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.387954+0000","last_change":"2024-09-10T23:35:37.412855+0000","last_active":"2024-09-10T23:35:38.387954+0000","last_peered":"2024-09-10T23:35:38.387954+0000","last_clean":"2024-09-10T23:35:38.387954+0000","last_became_active":"2024-09-10T23:35:37.412598+0000","last_became_peered":"2024-09-10T23:35:37.412598+0000","last_unstale":"2024-09-10T23:35:38.387954+0000","last_undegraded":"2024-09-10T23:35:38.387954+0000","last_fullsized":"2024-09-10T23:35:38.387954+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:18.413713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.839172+0000","last_change":"2024-09-10T23:35:36.404848+0000","last_active":"2024-09-10T23:35:38.839172+0000","last_peered":"2024-09-10T23:35:38.839172+0000","last_clean":"2024-09-10T23:35:38.839172+0000","last_became_active":"2024-09-10T23:35:36.404567+0000","last_became_peered":"2024-09-10T23:35:36.404567+0000","last_unstale":"2024-09-10T23:35:38.839172+0000","last_undegraded":"2024-09-10T23:35:38.839172+0000","last_fullsized":"2024-09-10T23:35:38.839172+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:17:01.417788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.395883+0000","last_change":"2024-09-10T23:35:37.411173+0000","last_active":"2024-09-10T23:35:39.395883+0000","last_peered":"2024-09-10T23:35:39.395883+0000","last_clean":"2024-09-10T23:35:39.395883+0000","last_became_active":"2024-09-10T23:35:37.410890+0000","last_became_peered":"2024-09-10T23:35:37.410890+0000","last_unstale":"2024-09-10T23:35:39.395883+0000","last_undegraded":"2024-09-10T23:35:39.395883+0000","last_fullsized":"2024-09-10T23:35:39.395883+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:42.593307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.382645+0000","last_change":"2024-09-10T23:35:36.402546+0000","last_active":"2024-09-10T23:35:39.382645+0000","last_peered":"2024-09-10T23:35:39.382645+0000","last_clean":"2024-09-10T23:35:39.382645+0000","last_became_active":"2024-09-10T23:35:36.402187+0000","last_became_peered":"2024-09-10T23:35:36.402187+0000","last_unstale":"2024-09-10T23:35:39.382645+0000","last_undegraded":"2024-09-10T23:35:39.382645+0000","last_fullsized":"2024-09-10T23:35:39.382645+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:35:10.970981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.387920+0000","last_change":"2024-09-10T23:35:37.408117+0000","last_active":"2024-09-10T23:35:38.387920+0000","last_peered":"2024-09-10T23:35:38.387920+0000","last_clean":"2024-09-10T23:35:38.387920+0000","last_became_active":"2024-09-10T23:35:37.407826+0000","last_became_peered":"2024-09-10T23:35:37.407826+0000","last_unstale":"2024-09-10T23:35:38.387920+0000","last_undegraded":"2024-09-10T23:35:38.387920+0000","last_fullsized":"2024-09-10T23:35:38.387920+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:40.214723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.383202+0000","last_change":"2024-09-10T23:35:36.401288+0000","last_active":"2024-09-10T23:35:39.383202+0000","last_peered":"2024-09-10T23:35:39.383202+0000","last_clean":"2024-09-10T23:35:39.383202+0000","last_became_active":"2024-09-10T23:35:36.400889+0000","last_became_peered":"2024-09-10T23:35:36.400889+0000","last_unstale":"2024-09-10T23:35:39.383202+0000","last_undegraded":"2024-09-10T23:35:39.383202+0000","last_fullsized":"2024-09-10T23:35:39.383202+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:15.290059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.382948+0000","last_change":"2024-09-10T23:35:37.418485+0000","last_active":"2024-09-10T23:35:39.382948+0000","last_peered":"2024-09-10T23:35:39.382948+0000","last_clean":"2024-09-10T23:35:39.382948+0000","last_became_active":"2024-09-10T23:35:37.417242+0000","last_became_peered":"2024-09-10T23:35:37.417242+0000","last_unstale":"2024-09-10T23:35:39.382948+0000","last_undegraded":"2024-09-10T23:35:39.382948+0000","last_fullsized":"2024-09-10T23:35:39.382948+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:59:59.519922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":23,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.382689+0000","last_change":"2024-09-10T23:35:36.398356+0000","last_active":"2024-09-10T23:35:39.382689+0000","last_peered":"2024-09-10T23:35:39.382689+0000","last_clean":"2024-09-10T23:35:39.382689+0000","last_became_active":"2024-09-10T23:35:36.398077+0000","last_became_peered":"2024-09-10T23:35:36.398077+0000","last_unstale":"2024-09-10T23:35:39.382689+0000","last_undegraded":"2024-09-10T23:35:39.382689+0000","last_fullsized":"2024-09-10T23:35:39.382689+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:58.471465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.396258+0000","last_change":"2024-09-10T23:35:37.411565+0000","last_active":"2024-09-10T23:35:39.396258+0000","last_peered":"2024-09-10T23:35:39.396258+0000","last_clean":"2024-09-10T23:35:39.396258+0000","last_became_active":"2024-09-10T23:35:37.408778+0000","last_became_peered":"2024-09-10T23:35:37.408778+0000","last_unstale":"2024-09-10T23:35:39.396258+0000","last_undegraded":"2024-09-10T23:35:39.396258+0000","last_fullsized":"2024-09-10T23:35:39.396258+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:57.376554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.728858+0000","last_change":"2024-09-10T23:35:36.409579+0000","last_active":"2024-09-10T23:35:38.728858+0000","last_peered":"2024-09-10T23:35:38.728858+0000","last_clean":"2024-09-10T23:35:38.728858+0000","last_became_active":"2024-09-10T23:35:36.409321+0000","last_became_peered":"2024-09-10T23:35:36.409321+0000","last_unstale":"2024-09-10T23:35:38.728858+0000","last_undegraded":"2024-09-10T23:35:38.728858+0000","last_fullsized":"2024-09-10T23:35:38.728858+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:05.986241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.396462+0000","last_change":"2024-09-10T23:35:37.409059+0000","last_active":"2024-09-10T23:35:39.396462+0000","last_peered":"2024-09-10T23:35:39.396462+0000","last_clean":"2024-09-10T23:35:39.396462+0000","last_became_active":"2024-09-10T23:35:37.408767+0000","last_became_peered":"2024-09-10T23:35:37.408767+0000","last_unstale":"2024-09-10T23:35:39.396462+0000","last_undegraded":"2024-09-10T23:35:39.396462+0000","last_fullsized":"2024-09-10T23:35:39.396462+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:49.342750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.839230+0000","last_change":"2024-09-10T23:35:36.400916+0000","last_active":"2024-09-10T23:35:38.839230+0000","last_peered":"2024-09-10T23:35:38.839230+0000","last_clean":"2024-09-10T23:35:38.839230+0000","last_became_active":"2024-09-10T23:35:36.400639+0000","last_became_peered":"2024-09-10T23:35:36.400639+0000","last_unstale":"2024-09-10T23:35:38.839230+0000","last_undegraded":"2024-09-10T23:35:38.839230+0000","last_fullsized":"2024-09-10T23:35:38.839230+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:56.778708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.728428+0000","last_change":"2024-09-10T23:35:37.416025+0000","last_active":"2024-09-10T23:35:38.728428+0000","last_peered":"2024-09-10T23:35:38.728428+0000","last_clean":"2024-09-10T23:35:38.728428+0000","last_became_active":"2024-09-10T23:35:37.415749+0000","last_became_peered":"2024-09-10T23:35:37.415749+0000","last_unstale":"2024-09-10T23:35:38.728428+0000","last_undegraded":"2024-09-10T23:35:38.728428+0000","last_fullsized":"2024-09-10T23:35:38.728428+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:23.787833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.396504+0000","last_change":"2024-09-10T23:35:36.398766+0000","last_active":"2024-09-10T23:35:39.396504+0000","last_peered":"2024-09-10T23:35:39.396504+0000","last_clean":"2024-09-10T23:35:39.396504+0000","last_became_active":"2024-09-10T23:35:36.398473+0000","last_became_peered":"2024-09-10T23:35:36.398473+0000","last_unstale":"2024-09-10T23:35:39.396504+0000","last_undegraded":"2024-09-10T23:35:39.396504+0000","last_fullsized":"2024-09-10T23:35:39.396504+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:38.200513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.383029+0000","last_change":"2024-09-10T23:35:37.420431+0000","last_active":"2024-09-10T23:35:39.383029+0000","last_peered":"2024-09-10T23:35:39.383029+0000","last_clean":"2024-09-10T23:35:39.383029+0000","last_became_active":"2024-09-10T23:35:37.418839+0000","last_became_peered":"2024-09-10T23:35:37.418839+0000","last_unstale":"2024-09-10T23:35:39.383029+0000","last_undegraded":"2024-09-10T23:35:39.383029+0000","last_fullsized":"2024-09-10T23:35:39.383029+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:56.510699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.839187+0000","last_change":"2024-09-10T23:35:36.409772+0000","last_active":"2024-09-10T23:35:38.839187+0000","last_peered":"2024-09-10T23:35:38.839187+0000","last_clean":"2024-09-10T23:35:38.839187+0000","last_became_active":"2024-09-10T23:35:36.409432+0000","last_became_peered":"2024-09-10T23:35:36.409432+0000","last_unstale":"2024-09-10T23:35:38.839187+0000","last_undegraded":"2024-09-10T23:35:38.839187+0000","last_fullsized":"2024-09-10T23:35:38.839187+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:30.173489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.388169+0000","last_change":"2024-09-10T23:35:37.412598+0000","last_active":"2024-09-10T23:35:38.388169+0000","last_peered":"2024-09-10T23:35:38.388169+0000","last_clean":"2024-09-10T23:35:38.388169+0000","last_became_active":"2024-09-10T23:35:37.412324+0000","last_became_peered":"2024-09-10T23:35:37.412324+0000","last_unstale":"2024-09-10T23:35:38.388169+0000","last_undegraded":"2024-09-10T23:35:38.388169+0000","last_fullsized":"2024-09-10T23:35:38.388169+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:46.402727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390258+0000","last_change":"2024-09-10T23:35:36.405162+0000","last_active":"2024-09-10T23:35:38.390258+0000","last_peered":"2024-09-10T23:35:38.390258+0000","last_clean":"2024-09-10T23:35:38.390258+0000","last_became_active":"2024-09-10T23:35:36.404891+0000","last_became_peered":"2024-09-10T23:35:36.404891+0000","last_unstale":"2024-09-10T23:35:38.390258+0000","last_undegraded":"2024-09-10T23:35:38.390258+0000","last_fullsized":"2024-09-10T23:35:38.390258+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:09.123353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.839151+0000","last_change":"2024-09-10T23:35:37.420023+0000","last_active":"2024-09-10T23:35:38.839151+0000","last_peered":"2024-09-10T23:35:38.839151+0000","last_clean":"2024-09-10T23:35:38.839151+0000","last_became_active":"2024-09-10T23:35:37.419672+0000","last_became_peered":"2024-09-10T23:35:37.419672+0000","last_unstale":"2024-09-10T23:35:38.839151+0000","last_undegraded":"2024-09-10T23:35:38.839151+0000","last_fullsized":"2024-09-10T23:35:38.839151+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:50.788788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.838683+0000","last_change":"2024-09-10T23:35:36.401149+0000","last_active":"2024-09-10T23:35:38.838683+0000","last_peered":"2024-09-10T23:35:38.838683+0000","last_clean":"2024-09-10T23:35:38.838683+0000","last_became_active":"2024-09-10T23:35:36.400884+0000","last_became_peered":"2024-09-10T23:35:36.400884+0000","last_unstale":"2024-09-10T23:35:38.838683+0000","last_undegraded":"2024-09-10T23:35:38.838683+0000","last_fullsized":"2024-09-10T23:35:38.838683+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:39.737213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.838680+0000","last_change":"2024-09-10T23:35:37.408527+0000","last_active":"2024-09-10T23:35:38.838680+0000","last_peered":"2024-09-10T23:35:38.838680+0000","last_clean":"2024-09-10T23:35:38.838680+0000","last_became_active":"2024-09-10T23:35:37.408180+0000","last_became_peered":"2024-09-10T23:35:37.408180+0000","last_unstale":"2024-09-10T23:35:38.838680+0000","last_undegraded":"2024-09-10T23:35:38.838680+0000","last_fullsized":"2024-09-10T23:35:38.838680+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:19.614584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.388030+0000","last_change":"2024-09-10T23:35:36.406290+0000","last_active":"2024-09-10T23:35:38.388030+0000","last_peered":"2024-09-10T23:35:38.388030+0000","last_clean":"2024-09-10T23:35:38.388030+0000","last_became_active":"2024-09-10T23:35:36.406056+0000","last_became_peered":"2024-09-10T23:35:36.406056+0000","last_unstale":"2024-09-10T23:35:38.388030+0000","last_undegraded":"2024-09-10T23:35:38.388030+0000","last_fullsized":"2024-09-10T23:35:38.388030+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:40.032192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.396363+0000","last_change":"2024-09-10T23:35:37.410659+0000","last_active":"2024-09-10T23:35:39.396363+0000","last_peered":"2024-09-10T23:35:39.396363+0000","last_clean":"2024-09-10T23:35:39.396363+0000","last_became_active":"2024-09-10T23:35:37.410361+0000","last_became_peered":"2024-09-10T23:35:37.410361+0000","last_unstale":"2024-09-10T23:35:39.396363+0000","last_undegraded":"2024-09-10T23:35:39.396363+0000","last_fullsized":"2024-09-10T23:35:39.396363+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:18.604994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.728987+0000","last_change":"2024-09-10T23:35:36.406893+0000","last_active":"2024-09-10T23:35:38.728987+0000","last_peered":"2024-09-10T23:35:38.728987+0000","last_clean":"2024-09-10T23:35:38.728987+0000","last_became_active":"2024-09-10T23:35:36.406516+0000","last_became_peered":"2024-09-10T23:35:36.406516+0000","last_unstale":"2024-09-10T23:35:38.728987+0000","last_undegraded":"2024-09-10T23:35:38.728987+0000","last_fullsized":"2024-09-10T23:35:38.728987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:10.791579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.396735+0000","last_change":"2024-09-10T23:35:37.419950+0000","last_active":"2024-09-10T23:35:39.396735+0000","last_peered":"2024-09-10T23:35:39.396735+0000","last_clean":"2024-09-10T23:35:39.396735+0000","last_became_active":"2024-09-10T23:35:37.419675+0000","last_became_peered":"2024-09-10T23:35:37.419675+0000","last_unstale":"2024-09-10T23:35:39.396735+0000","last_undegraded":"2024-09-10T23:35:39.396735+0000","last_fullsized":"2024-09-10T23:35:39.396735+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:35.610857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.391082+0000","last_change":"2024-09-10T23:35:36.409344+0000","last_active":"2024-09-10T23:35:38.391082+0000","last_peered":"2024-09-10T23:35:38.391082+0000","last_clean":"2024-09-10T23:35:38.391082+0000","last_became_active":"2024-09-10T23:35:36.408975+0000","last_became_peered":"2024-09-10T23:35:36.408975+0000","last_unstale":"2024-09-10T23:35:38.391082+0000","last_undegraded":"2024-09-10T23:35:38.391082+0000","last_fullsized":"2024-09-10T23:35:38.391082+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:00.834693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.728454+0000","last_change":"2024-09-10T23:35:37.421434+0000","last_active":"2024-09-10T23:35:38.728454+0000","last_peered":"2024-09-10T23:35:38.728454+0000","last_clean":"2024-09-10T23:35:38.728454+0000","last_became_active":"2024-09-10T23:35:37.421194+0000","last_became_peered":"2024-09-10T23:35:37.421194+0000","last_unstale":"2024-09-10T23:35:38.728454+0000","last_undegraded":"2024-09-10T23:35:38.728454+0000","last_fullsized":"2024-09-10T23:35:38.728454+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:05.742251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.388175+0000","last_change":"2024-09-10T23:35:36.400467+0000","last_active":"2024-09-10T23:35:38.388175+0000","last_peered":"2024-09-10T23:35:38.388175+0000","last_clean":"2024-09-10T23:35:38.388175+0000","last_became_active":"2024-09-10T23:35:36.400164+0000","last_became_peered":"2024-09-10T23:35:36.400164+0000","last_unstale":"2024-09-10T23:35:38.388175+0000","last_undegraded":"2024-09-10T23:35:38.388175+0000","last_fullsized":"2024-09-10T23:35:38.388175+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:20.099481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.839445+0000","last_change":"2024-09-10T23:35:37.412297+0000","last_active":"2024-09-10T23:35:38.839445+0000","last_peered":"2024-09-10T23:35:38.839445+0000","last_clean":"2024-09-10T23:35:38.839445+0000","last_became_active":"2024-09-10T23:35:37.412016+0000","last_became_peered":"2024-09-10T23:35:37.412016+0000","last_unstale":"2024-09-10T23:35:38.839445+0000","last_undegraded":"2024-09-10T23:35:38.839445+0000","last_fullsized":"2024-09-10T23:35:38.839445+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:26.854147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.396021+0000","last_change":"2024-09-10T23:35:36.399341+0000","last_active":"2024-09-10T23:35:39.396021+0000","last_peered":"2024-09-10T23:35:39.396021+0000","last_clean":"2024-09-10T23:35:39.396021+0000","last_became_active":"2024-09-10T23:35:36.399064+0000","last_became_peered":"2024-09-10T23:35:36.399064+0000","last_unstale":"2024-09-10T23:35:39.396021+0000","last_undegraded":"2024-09-10T23:35:39.396021+0000","last_fullsized":"2024-09-10T23:35:39.396021+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:16.638187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.390616+0000","last_change":"2024-09-10T23:35:37.403404+0000","last_active":"2024-09-10T23:35:38.390616+0000","last_peered":"2024-09-10T23:35:38.390616+0000","last_clean":"2024-09-10T23:35:38.390616+0000","last_became_active":"2024-09-10T23:35:37.400175+0000","last_became_peered":"2024-09-10T23:35:37.400175+0000","last_unstale":"2024-09-10T23:35:38.390616+0000","last_undegraded":"2024-09-10T23:35:38.390616+0000","last_fullsized":"2024-09-10T23:35:38.390616+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:03:54.308424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.728333+0000","last_change":"2024-09-10T23:35:36.410996+0000","last_active":"2024-09-10T23:35:38.728333+0000","last_peered":"2024-09-10T23:35:38.728333+0000","last_clean":"2024-09-10T23:35:38.728333+0000","last_became_active":"2024-09-10T23:35:36.410559+0000","last_became_peered":"2024-09-10T23:35:36.410559+0000","last_unstale":"2024-09-10T23:35:38.728333+0000","last_undegraded":"2024-09-10T23:35:38.728333+0000","last_fullsized":"2024-09-10T23:35:38.728333+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:16.136565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.096236+0000","last_change":"2024-09-10T23:35:37.415546+0000","last_active":"2024-09-10T23:35:39.096236+0000","last_peered":"2024-09-10T23:35:39.096236+0000","last_clean":"2024-09-10T23:35:39.096236+0000","last_became_active":"2024-09-10T23:35:37.414983+0000","last_became_peered":"2024-09-10T23:35:37.414983+0000","last_unstale":"2024-09-10T23:35:39.096236+0000","last_undegraded":"2024-09-10T23:35:39.096236+0000","last_fullsized":"2024-09-10T23:35:39.096236+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:28.347909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.395751+0000","last_change":"2024-09-10T23:35:36.399492+0000","last_active":"2024-09-10T23:35:39.395751+0000","last_peered":"2024-09-10T23:35:39.395751+0000","last_clean":"2024-09-10T23:35:39.395751+0000","last_became_active":"2024-09-10T23:35:36.397982+0000","last_became_peered":"2024-09-10T23:35:36.397982+0000","last_unstale":"2024-09-10T23:35:39.395751+0000","last_undegraded":"2024-09-10T23:35:39.395751+0000","last_fullsized":"2024-09-10T23:35:39.395751+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:01.366584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.838754+0000","last_change":"2024-09-10T23:35:37.405232+0000","last_active":"2024-09-10T23:35:38.838754+0000","last_peered":"2024-09-10T23:35:38.838754+0000","last_clean":"2024-09-10T23:35:38.838754+0000","last_became_active":"2024-09-10T23:35:37.404933+0000","last_became_peered":"2024-09-10T23:35:37.404933+0000","last_unstale":"2024-09-10T23:35:38.838754+0000","last_undegraded":"2024-09-10T23:35:38.838754+0000","last_fullsized":"2024-09-10T23:35:38.838754+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:59.474886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.389803+0000","last_change":"2024-09-10T23:35:36.406500+0000","last_active":"2024-09-10T23:35:38.389803+0000","last_peered":"2024-09-10T23:35:38.389803+0000","last_clean":"2024-09-10T23:35:38.389803+0000","last_became_active":"2024-09-10T23:35:36.406222+0000","last_became_peered":"2024-09-10T23:35:36.406222+0000","last_unstale":"2024-09-10T23:35:38.389803+0000","last_undegraded":"2024-09-10T23:35:38.389803+0000","last_fullsized":"2024-09-10T23:35:38.389803+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:36.140228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.728462+0000","last_change":"2024-09-10T23:35:37.420954+0000","last_active":"2024-09-10T23:35:38.728462+0000","last_peered":"2024-09-10T23:35:38.728462+0000","last_clean":"2024-09-10T23:35:38.728462+0000","last_became_active":"2024-09-10T23:35:37.417771+0000","last_became_peered":"2024-09-10T23:35:37.417771+0000","last_unstale":"2024-09-10T23:35:38.728462+0000","last_undegraded":"2024-09-10T23:35:38.728462+0000","last_fullsized":"2024-09-10T23:35:38.728462+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:24.110962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.838957+0000","last_change":"2024-09-10T23:35:36.398978+0000","last_active":"2024-09-10T23:35:38.838957+0000","last_peered":"2024-09-10T23:35:38.838957+0000","last_clean":"2024-09-10T23:35:38.838957+0000","last_became_active":"2024-09-10T23:35:36.398751+0000","last_became_peered":"2024-09-10T23:35:36.398751+0000","last_unstale":"2024-09-10T23:35:38.838957+0000","last_undegraded":"2024-09-10T23:35:38.838957+0000","last_fullsized":"2024-09-10T23:35:38.838957+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:04:15.253024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.839173+0000","last_change":"2024-09-10T23:35:37.420689+0000","last_active":"2024-09-10T23:35:38.839173+0000","last_peered":"2024-09-10T23:35:38.839173+0000","last_clean":"2024-09-10T23:35:38.839173+0000","last_became_active":"2024-09-10T23:35:37.420397+0000","last_became_peered":"2024-09-10T23:35:37.420397+0000","last_unstale":"2024-09-10T23:35:38.839173+0000","last_undegraded":"2024-09-10T23:35:38.839173+0000","last_fullsized":"2024-09-10T23:35:38.839173+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:36.375658+0000","last_clean_scrub_stamp":"2024-09-10T23:35:36.375658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:55:22.388448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:39.095914+0000","last_change":"2024-09-10T23:35:36.403531+0000","last_active":"2024-09-10T23:35:39.095914+0000","last_peered":"2024-09-10T23:35:39.095914+0000","last_clean":"2024-09-10T23:35:39.095914+0000","last_became_active":"2024-09-10T23:35:36.403263+0000","last_became_peered":"2024-09-10T23:35:36.403263+0000","last_unstale":"2024-09-10T23:35:39.095914+0000","last_undegraded":"2024-09-10T23:35:39.095914+0000","last_fullsized":"2024-09-10T23:35:39.095914+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:35:35.372237+0000","last_clean_scrub_stamp":"2024-09-10T23:35:35.372237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:16.638187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":177,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T23:35:38.388080+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:35:38.388080+0000","last_peered":"2024-09-10T23:35:38.388080+0000","last_clean":"2024-09-10T23:35:38.388080+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:35:38.388080+0000","last_undegraded":"2024-09-10T23:35:38.388080+0000","last_fullsized":"2024-09-10T23:35:38.388080+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":696320,"data_stored":284034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509482,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":874344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":696320,"data_stored":284558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":692224,"data_stored":284032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":708608,"data_stored":285714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509481,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":874892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":700416,"data_stored":285554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":692224,"data_stored":283964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:35:46.438 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T23:35:46.439 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T23:35:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27f200640 1 -- 172.21.15.94:0/2370927212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2800d2670 msgr2=0x7fe2800d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27f200640 1 --2- 172.21.15.94:0/2370927212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2800d2670 0x7fe2800d0440 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009900 tx=0x7fe26c01c8e0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27f200640 1 -- 172.21.15.94:0/2370927212 shutdown_connections 2024-09-10T23:35:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27f200640 1 --2- 172.21.15.94:0/2370927212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2800d2670 0x7fe2800d0440 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27f200640 1 --2- 172.21.15.94:0/2370927212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe28015bcc0 0x7fe2800cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27f200640 1 -- 172.21.15.94:0/2370927212 >> 172.21.15.94:0/2370927212 conn(0x7fe2800d1b50 msgr2=0x7fe2800d1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27f200640 1 -- 172.21.15.94:0/2370927212 shutdown_connections 2024-09-10T23:35:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27f200640 1 -- 172.21.15.94:0/2370927212 wait complete. 2024-09-10T23:35:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27f200640 1 Processor -- start 2024-09-10T23:35:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27f200640 1 -- start start 2024-09-10T23:35:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2800ce560 0x7fe28006f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2800d2670 0x7fe28014d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27f200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe28015bcc0 0x7fe2801511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe2801693a0 con 0x7fe2800d2670 2024-09-10T23:35:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27f200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe280169220 con 0x7fe2800ce560 2024-09-10T23:35:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27f200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe280169520 con 0x7fe28015bcc0 2024-09-10T23:35:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2800d2670 0x7fe28014d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2800d2670 0x7fe28014d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44182/0 (socket says 172.21.15.94:44182) 2024-09-10T23:35:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27d400640 1 -- 172.21.15.94:0/3112497552 learned_addr learned my addr 172.21.15.94:0/3112497552 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27de00640 1 --2- 172.21.15.94:0/3112497552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2800ce560 0x7fe28006f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27e800640 1 --2- 172.21.15.94:0/3112497552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe28015bcc0 0x7fe2801511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27d400640 1 -- 172.21.15.94:0/3112497552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe28015bcc0 msgr2=0x7fe2801511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27d400640 1 --2- 172.21.15.94:0/3112497552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe28015bcc0 0x7fe2801511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27d400640 1 -- 172.21.15.94:0/3112497552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2800ce560 msgr2=0x7fe28006f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27d400640 1 --2- 172.21.15.94:0/3112497552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2800ce560 0x7fe28006f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27d400640 1 -- 172.21.15.94:0/3112497552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2801542d0 con 0x7fe2800d2670 2024-09-10T23:35:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.626+0000 7fe27d400640 1 --2- 172.21.15.94:0/3112497552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2800d2670 0x7fe28014d120 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009850 tx=0x7fe26c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.630+0000 7fe276000640 1 -- 172.21.15.94:0/3112497552 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c0b2040 con 0x7fe2800d2670 2024-09-10T23:35:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.630+0000 7fe276000640 1 -- 172.21.15.94:0/3112497552 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe26c005030 con 0x7fe2800d2670 2024-09-10T23:35:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.630+0000 7fe276000640 1 -- 172.21.15.94:0/3112497552 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c005440 con 0x7fe2800d2670 2024-09-10T23:35:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.630+0000 7fe27f200640 1 -- 172.21.15.94:0/3112497552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2801545b0 con 0x7fe2800d2670 2024-09-10T23:35:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.630+0000 7fe27f200640 1 -- 172.21.15.94:0/3112497552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe280154b70 con 0x7fe2800d2670 2024-09-10T23:35:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.630+0000 7fe27f200640 1 -- 172.21.15.94:0/3112497552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2800ce9e0 con 0x7fe2800d2670 2024-09-10T23:35:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.630+0000 7fe276000640 1 -- 172.21.15.94:0/3112497552 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7fe26c0046d0 con 0x7fe2800d2670 2024-09-10T23:35:46.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.630+0000 7fe276000640 1 --2- 172.21.15.94:0/3112497552 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe260072790 0x7fe260074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:46.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.630+0000 7fe276000640 1 -- 172.21.15.94:0/3112497552 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fe26c12ce40 con 0x7fe2800d2670 2024-09-10T23:35:46.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.630+0000 7fe27de00640 1 --2- 172.21.15.94:0/3112497552 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe260072790 0x7fe260074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:46.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.634+0000 7fe27de00640 1 --2- 172.21.15.94:0/3112497552 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe260072790 0x7fe260074c50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe270005fd0 tx=0x7fe270005d10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:46.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.634+0000 7fe276000640 1 -- 172.21.15.94:0/3112497552 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe26c0164d0 con 0x7fe2800d2670 2024-09-10T23:35:46.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.858+0000 7fe27f200640 1 -- 172.21.15.94:0/3112497552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe28006fd00 con 0x7fe2800d2670 2024-09-10T23:35:46.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.862+0000 7fe276000640 1 -- 172.21.15.94:0/3112497552 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v30) ==== 190+0+147 (secure 0 0 0) 0x7fe26c0f80f0 con 0x7fe2800d2670 2024-09-10T23:35:46.863 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-10T23:35:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g== 2024-09-10T23:35:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: caps mds = "allow" 2024-09-10T23:35:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: caps mon = "allow r" 2024-09-10T23:35:46.864 INFO:teuthology.orchestra.run.smithi094.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T23:35:46.864 INFO:teuthology.orchestra.run.smithi094.stderr:updated caps for client.0 2024-09-10T23:35:46.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.862+0000 7fe27f200640 1 -- 172.21.15.94:0/3112497552 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe260072790 msgr2=0x7fe260074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:46.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.862+0000 7fe27f200640 1 --2- 172.21.15.94:0/3112497552 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe260072790 0x7fe260074c50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe270005fd0 tx=0x7fe270005d10 comp rx=0 tx=0).stop 2024-09-10T23:35:46.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.862+0000 7fe27f200640 1 -- 172.21.15.94:0/3112497552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2800d2670 msgr2=0x7fe28014d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:46.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.862+0000 7fe27f200640 1 --2- 172.21.15.94:0/3112497552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2800d2670 0x7fe28014d120 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009850 tx=0x7fe26c004040 comp rx=0 tx=0).stop 2024-09-10T23:35:46.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.862+0000 7fe27f200640 1 -- 172.21.15.94:0/3112497552 shutdown_connections 2024-09-10T23:35:46.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.862+0000 7fe27f200640 1 --2- 172.21.15.94:0/3112497552 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe260072790 0x7fe260074c50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.862+0000 7fe27f200640 1 --2- 172.21.15.94:0/3112497552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe28015bcc0 0x7fe2801511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.862+0000 7fe27f200640 1 --2- 172.21.15.94:0/3112497552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2800d2670 0x7fe28014d120 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.862+0000 7fe27f200640 1 --2- 172.21.15.94:0/3112497552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2800ce560 0x7fe28006f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:46.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.862+0000 7fe27f200640 1 -- 172.21.15.94:0/3112497552 >> 172.21.15.94:0/3112497552 conn(0x7fe2800d1b50 msgr2=0x7fe2800b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:46.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.862+0000 7fe27f200640 1 -- 172.21.15.94:0/3112497552 shutdown_connections 2024-09-10T23:35:46.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:46.862+0000 7fe27f200640 1 -- 172.21.15.94:0/3112497552 wait complete. 2024-09-10T23:35:46.886 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:35:47.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.026+0000 7f4a6d600640 1 -- 172.21.15.94:0/2074092602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a68157c60 msgr2=0x7f4a680b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:47.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.026+0000 7f4a6d600640 1 --2- 172.21.15.94:0/2074092602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a68157c60 0x7f4a680b35c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4a680b2760 tx=0x7f4a5001c830 comp rx=0 tx=0).stop 2024-09-10T23:35:47.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.026+0000 7f4a6d600640 1 -- 172.21.15.94:0/2074092602 shutdown_connections 2024-09-10T23:35:47.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.026+0000 7f4a6d600640 1 --2- 172.21.15.94:0/2074092602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a68157c60 0x7f4a680b35c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.026+0000 7f4a6d600640 1 --2- 172.21.15.94:0/2074092602 >> v2:172.21.15.181:3300/0 conn(0x7f4a68070150 0x7f4a68072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.026+0000 7f4a6d600640 1 -- 172.21.15.94:0/2074092602 >> 172.21.15.94:0/2074092602 conn(0x7f4a6806cdf0 msgr2=0x7f4a6806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:47.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.026+0000 7f4a6d600640 1 -- 172.21.15.94:0/2074092602 shutdown_connections 2024-09-10T23:35:47.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.026+0000 7f4a6d600640 1 -- 172.21.15.94:0/2074092602 wait complete. 2024-09-10T23:35:47.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.026+0000 7f4a6d600640 1 Processor -- start 2024-09-10T23:35:47.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.026+0000 7f4a6d600640 1 -- start start 2024-09-10T23:35:47.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a6d600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a68070150 0x7f4a68152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a6d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680cfd30 0x7f4a68159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a6d600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a68157c60 0x7f4a6814d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a6d600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6815b3e0 con 0x7f4a680cfd30 2024-09-10T23:35:47.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a6d600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6815b260 con 0x7f4a68070150 2024-09-10T23:35:47.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a6d600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f4a6815b560 con 0x7f4a68157c60 2024-09-10T23:35:47.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a5e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680cfd30 0x7f4a68159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a5e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680cfd30 0x7f4a68159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44196/0 (socket says 172.21.15.94:44196) 2024-09-10T23:35:47.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a5e000640 1 -- 172.21.15.94:0/2663935431 learned_addr learned my addr 172.21.15.94:0/2663935431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:47.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a66a00640 1 --2- 172.21.15.94:0/2663935431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a68070150 0x7f4a68152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a67400640 1 --2- 172.21.15.94:0/2663935431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a68157c60 0x7f4a6814d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a5e000640 1 -- 172.21.15.94:0/2663935431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a68157c60 msgr2=0x7f4a6814d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:47.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a5e000640 1 --2- 172.21.15.94:0/2663935431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a68157c60 0x7f4a6814d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a5e000640 1 -- 172.21.15.94:0/2663935431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a68070150 msgr2=0x7f4a68152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:47.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a5e000640 1 --2- 172.21.15.94:0/2663935431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a68070150 0x7f4a68152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a5e000640 1 -- 172.21.15.94:0/2663935431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a68163a60 con 0x7f4a680cfd30 2024-09-10T23:35:47.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a5e000640 1 --2- 172.21.15.94:0/2663935431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680cfd30 0x7f4a68159c50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4a500b0040 tx=0x7f4a500a56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:47.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a5f400640 1 -- 172.21.15.94:0/2663935431 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a500b2040 con 0x7f4a680cfd30 2024-09-10T23:35:47.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a5f400640 1 -- 172.21.15.94:0/2663935431 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a50004430 con 0x7f4a680cfd30 2024-09-10T23:35:47.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a6d600640 1 -- 172.21.15.94:0/2663935431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a681634a0 con 0x7f4a680cfd30 2024-09-10T23:35:47.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a5f400640 1 -- 172.21.15.94:0/2663935431 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50003860 con 0x7f4a680cfd30 2024-09-10T23:35:47.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a6d600640 1 -- 172.21.15.94:0/2663935431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a681584c0 con 0x7f4a680cfd30 2024-09-10T23:35:47.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a6d600640 1 -- 172.21.15.94:0/2663935431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a68072610 con 0x7f4a680cfd30 2024-09-10T23:35:47.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.030+0000 7f4a5f400640 1 -- 172.21.15.94:0/2663935431 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f4a50003a00 con 0x7f4a680cfd30 2024-09-10T23:35:47.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.034+0000 7f4a5f400640 1 --2- 172.21.15.94:0/2663935431 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4a44072790 0x7f4a44074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.034+0000 7f4a5f400640 1 -- 172.21.15.94:0/2663935431 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f4a5012d300 con 0x7f4a680cfd30 2024-09-10T23:35:47.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.034+0000 7f4a66a00640 1 --2- 172.21.15.94:0/2663935431 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4a44072790 0x7f4a44074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.034+0000 7f4a66a00640 1 --2- 172.21.15.94:0/2663935431 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4a44072790 0x7f4a44074c50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4a58006fd0 tx=0x7f4a58008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:47.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.038+0000 7f4a5f400640 1 -- 172.21.15.94:0/2663935431 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4a500bd060 con 0x7f4a680cfd30 2024-09-10T23:35:47.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.234+0000 7f4a6d600640 1 -- 172.21.15.94:0/2663935431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a68158690 con 0x7f4a680cfd30 2024-09-10T23:35:47.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.234+0000 7f4a5f400640 1 -- 172.21.15.94:0/2663935431 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5243 (secure 0 0 0) 0x7f4a68163a60 con 0x7f4a680cfd30 2024-09-10T23:35:47.237 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:35:47.237 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":34,"btime":"2024-09-10T23:35:40:410237+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5814,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3613711739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3613711739},{"type":"v1","addr":"172.21.15.94:6834","nonce":3613711739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":195107,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1388563408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1388563408},{"type":"v1","addr":"172.21.15.181:6836","nonce":1388563408}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":195110,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1882902083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1882902083},{"type":"v1","addr":"172.21.15.181:6837","nonce":1882902083}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:35:36.778166+0000","modified":"2024-09-10T23:35:39.784176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195113":{"gid":195113,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/2905667261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2905667261},{"type":"v1","addr":"172.21.15.94:6835","nonce":2905667261}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195113,"qdb_cluster":[195113]},"id":4}]} 2024-09-10T23:35:47.237 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 34 2024-09-10T23:35:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.238+0000 7f4a6d600640 1 -- 172.21.15.94:0/2663935431 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4a44072790 msgr2=0x7f4a44074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.238+0000 7f4a6d600640 1 --2- 172.21.15.94:0/2663935431 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4a44072790 0x7f4a44074c50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4a58006fd0 tx=0x7f4a58008040 comp rx=0 tx=0).stop 2024-09-10T23:35:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.238+0000 7f4a6d600640 1 -- 172.21.15.94:0/2663935431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680cfd30 msgr2=0x7f4a68159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.238+0000 7f4a6d600640 1 --2- 172.21.15.94:0/2663935431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680cfd30 0x7f4a68159c50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4a500b0040 tx=0x7f4a500a56d0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.238+0000 7f4a6d600640 1 -- 172.21.15.94:0/2663935431 shutdown_connections 2024-09-10T23:35:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.238+0000 7f4a6d600640 1 --2- 172.21.15.94:0/2663935431 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4a44072790 0x7f4a44074c50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.238+0000 7f4a6d600640 1 --2- 172.21.15.94:0/2663935431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a68157c60 0x7f4a6814d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.238+0000 7f4a6d600640 1 --2- 172.21.15.94:0/2663935431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680cfd30 0x7f4a68159c50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.238+0000 7f4a6d600640 1 --2- 172.21.15.94:0/2663935431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a68070150 0x7f4a68152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.238+0000 7f4a6d600640 1 -- 172.21.15.94:0/2663935431 >> 172.21.15.94:0/2663935431 conn(0x7f4a6806cdf0 msgr2=0x7f4a68154ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.238+0000 7f4a6d600640 1 -- 172.21.15.94:0/2663935431 shutdown_connections 2024-09-10T23:35:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.238+0000 7f4a6d600640 1 -- 172.21.15.94:0/2663935431 wait complete. 2024-09-10T23:35:47.663 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T23:35:36.778166+0000', 'modified': '2024-09-10T23:35:39.784176+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 195113}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_195113': {'gid': 195113, 'name': 'c', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.94:6835/2905667261', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6833', 'nonce': 2905667261}, {'type': 'v1', 'addr': '172.21.15.94:6835', 'nonce': 2905667261}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 195113, 'qdb_cluster': [195113]} 2024-09-10T23:35:47.663 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T23:35:47.663 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T23:35:47.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.802+0000 7fa2b2600640 1 -- 172.21.15.94:0/1407555229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2ac15bf60 msgr2=0x7fa2ac0d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:47.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.802+0000 7fa2b2600640 1 --2- 172.21.15.94:0/1407555229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2ac15bf60 0x7fa2ac0d2780 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0731c0 tx=0x7fa29c01ccf0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2b2600640 1 -- 172.21.15.94:0/1407555229 shutdown_connections 2024-09-10T23:35:47.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2b2600640 1 --2- 172.21.15.94:0/1407555229 >> v2:172.21.15.181:3300/0 conn(0x7fa2ac0cfce0 0x7fa2ac0d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2b2600640 1 --2- 172.21.15.94:0/1407555229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2ac15bf60 0x7fa2ac0d2780 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2b2600640 1 -- 172.21.15.94:0/1407555229 >> 172.21.15.94:0/1407555229 conn(0x7fa2ac0d1c60 msgr2=0x7fa2ac0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:47.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2b2600640 1 -- 172.21.15.94:0/1407555229 shutdown_connections 2024-09-10T23:35:47.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2b2600640 1 -- 172.21.15.94:0/1407555229 wait complete. 2024-09-10T23:35:47.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2b2600640 1 Processor -- start 2024-09-10T23:35:47.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2b2600640 1 -- start start 2024-09-10T23:35:47.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2b2600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2ac0cfce0 0x7fa2ac143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2b2600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2ac0d0660 0x7fa2ac144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2b2600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2ac15bf60 0x7fa2ac1585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2b2600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac168dc0 con 0x7fa2ac0d0660 2024-09-10T23:35:47.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2b2600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac168c40 con 0x7fa2ac15bf60 2024-09-10T23:35:47.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2b2600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa2ac168f40 con 0x7fa2ac0cfce0 2024-09-10T23:35:47.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2aaa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2ac0d0660 0x7fa2ac144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2aaa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2ac0d0660 0x7fa2ac144030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44214/0 (socket says 172.21.15.94:44214) 2024-09-10T23:35:47.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2aaa00640 1 -- 172.21.15.94:0/1679797883 learned_addr learned my addr 172.21.15.94:0/1679797883 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:47.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2abe00640 1 --2- 172.21.15.94:0/1679797883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2ac15bf60 0x7fa2ac1585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2ab400640 1 --2- 172.21.15.94:0/1679797883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2ac0cfce0 0x7fa2ac143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2aaa00640 1 -- 172.21.15.94:0/1679797883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2ac0cfce0 msgr2=0x7fa2ac143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:47.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2aaa00640 1 --2- 172.21.15.94:0/1679797883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2ac0cfce0 0x7fa2ac143af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2aaa00640 1 -- 172.21.15.94:0/1679797883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2ac15bf60 msgr2=0x7fa2ac1585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:47.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2aaa00640 1 --2- 172.21.15.94:0/1679797883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2ac15bf60 0x7fa2ac1585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2aaa00640 1 -- 172.21.15.94:0/1679797883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ac0cb820 con 0x7fa2ac0d0660 2024-09-10T23:35:47.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2ab400640 1 --2- 172.21.15.94:0/1679797883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2ac0cfce0 0x7fa2ac143af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:35:47.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2aaa00640 1 --2- 172.21.15.94:0/1679797883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2ac0d0660 0x7fa2ac144030 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa29800b580 tx=0x7fa29800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:47.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2a3400640 1 -- 172.21.15.94:0/1679797883 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298014070 con 0x7fa2ac0d0660 2024-09-10T23:35:47.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2a3400640 1 -- 172.21.15.94:0/1679797883 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa298010980 con 0x7fa2ac0d0660 2024-09-10T23:35:47.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2b2600640 1 -- 172.21.15.94:0/1679797883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ac0b4b50 con 0x7fa2ac0d0660 2024-09-10T23:35:47.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.806+0000 7fa2a3400640 1 -- 172.21.15.94:0/1679797883 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298010dd0 con 0x7fa2ac0d0660 2024-09-10T23:35:47.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.810+0000 7fa2b2600640 1 -- 172.21.15.94:0/1679797883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ac0b5110 con 0x7fa2ac0d0660 2024-09-10T23:35:47.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.810+0000 7fa2a3400640 1 -- 172.21.15.94:0/1679797883 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7fa29800f3c0 con 0x7fa2ac0d0660 2024-09-10T23:35:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.810+0000 7fa2a3400640 1 --2- 172.21.15.94:0/1679797883 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa28c0727c0 0x7fa28c074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.810+0000 7fa2a3400640 1 -- 172.21.15.94:0/1679797883 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa2980937c0 con 0x7fa2ac0d0660 2024-09-10T23:35:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.810+0000 7fa2b59b3000 1 Processor -- start 2024-09-10T23:35:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.810+0000 7fa2b59b3000 1 -- start start 2024-09-10T23:35:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.810+0000 7fa2ab400640 1 --2- 172.21.15.94:0/1679797883 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa28c0727c0 0x7fa28c074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b401da9d0 0x555b401dadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401db2f0 0x555b4018e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2ab400640 1 --2- 172.21.15.94:0/1679797883 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa28c0727c0 0x7fa28c074c80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0af000 tx=0x7fa29c01b530 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b4018ee10 0x555b4018f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555b400af3f0 con 0x555b401da9d0 2024-09-10T23:35:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555b3ffecb20 con 0x555b401db2f0 2024-09-10T23:35:47.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x555b3ffdbf50 con 0x555b4018ee10 2024-09-10T23:35:47.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2ab400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b401da9d0 0x555b401dadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2ab400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b401da9d0 0x555b401dadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44224/0 (socket says 172.21.15.94:44224) 2024-09-10T23:35:47.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2ab400640 1 -- 172.21.15.94:0/1529813399 learned_addr learned my addr 172.21.15.94:0/1529813399 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:47.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2aaa00640 1 --2- 172.21.15.94:0/1529813399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401db2f0 0x555b4018e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2abe00640 1 --2- 172.21.15.94:0/1529813399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b4018ee10 0x555b4018f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2ab400640 1 -- 172.21.15.94:0/1529813399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b4018ee10 msgr2=0x555b4018f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:35:47.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2ab400640 1 --2- 172.21.15.94:0/1529813399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b4018ee10 0x555b4018f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2ab400640 1 -- 172.21.15.94:0/1529813399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401db2f0 msgr2=0x555b4018e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:35:47.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2ab400640 1 --2- 172.21.15.94:0/1529813399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401db2f0 0x555b4018e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2ab400640 1 -- 172.21.15.94:0/1529813399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b3ffa3590 con 0x555b401da9d0 2024-09-10T23:35:47.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2ab400640 1 --2- 172.21.15.94:0/1529813399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b401da9d0 0x555b401dadb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa29c098900 tx=0x7fa29c0b5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21479539400df2e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa297e00640 1 -- 172.21.15.94:0/1529813399 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c0bf040 con 0x555b401da9d0 2024-09-10T23:35:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa297e00640 1 -- 172.21.15.94:0/1529813399 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa29c0c8070 con 0x555b401da9d0 2024-09-10T23:35:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa297e00640 1 -- 172.21.15.94:0/1529813399 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c0046b0 con 0x555b401da9d0 2024-09-10T23:35:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 -- 172.21.15.94:0/1529813399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b401da9d0 msgr2=0x555b401dadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/1529813399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b401da9d0 0x555b401dadb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa29c098900 tx=0x7fa29c0b5000 comp rx=0 tx=0).stop 2024-09-10T23:35:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 -- 172.21.15.94:0/1529813399 shutdown_connections 2024-09-10T23:35:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/1529813399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401db2f0 0x555b4018e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/1529813399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b4018ee10 0x555b4018f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/1529813399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b401da9d0 0x555b401dadb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 -- 172.21.15.94:0/1529813399 >> 172.21.15.94:0/1529813399 conn(0x555b40078130 msgr2=0x555b401bde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:47.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 -- 172.21.15.94:0/1529813399 shutdown_connections 2024-09-10T23:35:47.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 -- 172.21.15.94:0/1529813399 wait complete. 2024-09-10T23:35:47.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 -- start start 2024-09-10T23:35:47.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.814+0000 7fa2b59b3000 1 Processor -- start 2024-09-10T23:35:47.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b4018ee10 0x555b401bb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b401da9d0 0x555b401bbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401db2f0 0x555b401bc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555b3ff765d0 con 0x555b401da9d0 2024-09-10T23:35:47.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555b3ffdfd60 con 0x555b401db2f0 2024-09-10T23:35:47.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x555b3ff82500 con 0x555b4018ee10 2024-09-10T23:35:47.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 10 client.0 _ll_register_callbacks cb 0x555b3ff86810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:35:47.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2aaa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b401da9d0 0x555b401bbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2aaa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b401da9d0 0x555b401bbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44240/0 (socket says 172.21.15.94:44240) 2024-09-10T23:35:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2aaa00640 1 -- 172.21.15.94:0/1537688476 learned_addr learned my addr 172.21.15.94:0/1537688476 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2abe00640 1 --2- 172.21.15.94:0/1537688476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401db2f0 0x555b401bc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2ab400640 1 --2- 172.21.15.94:0/1537688476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b4018ee10 0x555b401bb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2aaa00640 1 -- 172.21.15.94:0/1537688476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b4018ee10 msgr2=0x555b401bb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:35:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2aaa00640 1 --2- 172.21.15.94:0/1537688476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b4018ee10 0x555b401bb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2aaa00640 1 -- 172.21.15.94:0/1537688476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401db2f0 msgr2=0x555b401bc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:35:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2aaa00640 1 --2- 172.21.15.94:0/1537688476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401db2f0 0x555b401bc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2aaa00640 1 -- 172.21.15.94:0/1537688476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b3ff7a170 con 0x555b401da9d0 2024-09-10T23:35:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2aaa00640 1 --2- 172.21.15.94:0/1537688476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b401da9d0 0x555b401bbe70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa29800b550 tx=0x7fa29805e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52609f33418efd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa294c00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:35:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa294c00640 1 -- 172.21.15.94:0/1537688476 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29801e040 con 0x555b401da9d0 2024-09-10T23:35:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa294c00640 1 -- 172.21.15.94:0/1537688476 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa298059cd0 con 0x555b401da9d0 2024-09-10T23:35:47.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 -- 172.21.15.94:0/1537688476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555b3ff740f0 con 0x555b401da9d0 2024-09-10T23:35:47.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa294c00640 1 -- 172.21.15.94:0/1537688476 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa29805de90 con 0x555b401da9d0 2024-09-10T23:35:47.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 10 client.5967 fetch_fsmap learned FSMap version 34 2024-09-10T23:35:47.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 -- 172.21.15.94:0/1537688476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x555b40060ef0 con 0x555b401da9d0 2024-09-10T23:35:47.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa294c00640 1 -- 172.21.15.94:0/1537688476 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555b3ff740f0 con 0x555b401da9d0 2024-09-10T23:35:47.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa294c00640 1 -- 172.21.15.94:0/1537688476 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 3502+0+0 (secure 0 0 0) 0x7fa298059ed0 con 0x555b401da9d0 2024-09-10T23:35:47.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 10 client.5967 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T23:35:47.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 10 client.5967 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T23:35:47.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 20 client.5967 populate_metadata read hostname 'smithi094' 2024-09-10T23:35:47.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/1537688476 >> [v2:172.21.15.94:6833/2905667261,v1:172.21.15.94:6835/2905667261] conn(0x555b401bd0f0 0x555b401ecc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 4 client.5967 mds_command: new command op to 195113 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:35:47.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 -- 172.21.15.94:0/1537688476 --> [v2:172.21.15.94:6833/2905667261,v1:172.21.15.94:6835/2905667261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b3ffeea70 con 0x555b401bd0f0 2024-09-10T23:35:47.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2ab400640 1 --2- 172.21.15.94:0/1537688476 >> [v2:172.21.15.94:6833/2905667261,v1:172.21.15.94:6835/2905667261] conn(0x555b401bd0f0 0x555b401ecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2ab400640 1 --2- 172.21.15.94:0/1537688476 >> [v2:172.21.15.94:6833/2905667261,v1:172.21.15.94:6835/2905667261] conn(0x555b401bd0f0 0x555b401ecc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93a56a2cd3281b00 server_cookie=40e347060f0be2e4 in_seq=0 out_seq=0 2024-09-10T23:35:47.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa294c00640 10 client.5967 ms_handle_connect on v2:172.21.15.94:6833/2905667261 2024-09-10T23:35:47.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa294c00640 1 -- 172.21.15.94:0/1537688476 <== mds.0 v2:172.21.15.94:6833/2905667261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b3ffeea70 con 0x555b401bd0f0 2024-09-10T23:35:47.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa294c00640 10 client.5967 handle_command_reply: tid=0 2024-09-10T23:35:47.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 client.5967 shutdown 2024-09-10T23:35:47.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 -- 172.21.15.94:0/1537688476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b401da9d0 msgr2=0x555b401bbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:47.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/1537688476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b401da9d0 0x555b401bbe70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa29800b550 tx=0x7fa29805e9a0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 -- 172.21.15.94:0/1537688476 shutdown_connections 2024-09-10T23:35:47.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/1537688476 >> [v2:172.21.15.94:6833/2905667261,v1:172.21.15.94:6835/2905667261] conn(0x555b401bd0f0 0x555b401ecc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/1537688476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401db2f0 0x555b401bc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/1537688476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b401da9d0 0x555b401bbe70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/1537688476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b4018ee10 0x555b401bb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa294c00640 0 client.5967 ms_handle_reset on v2:172.21.15.94:6833/2905667261 2024-09-10T23:35:47.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.818+0000 7fa2b59b3000 1 -- 172.21.15.94:0/1537688476 >> 172.21.15.94:0/1537688476 conn(0x555b40078130 msgr2=0x555b4018dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:47.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.822+0000 7fa2b59b3000 1 -- 172.21.15.94:0/1537688476 shutdown_connections 2024-09-10T23:35:47.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.822+0000 7fa2b59b3000 1 -- 172.21.15.94:0/1537688476 wait complete. 2024-09-10T23:35:47.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.822+0000 7fa2b59b3000 20 client.5967 trim_cache size 0 max 16384 2024-09-10T23:35:47.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.838+0000 7fa2b59b3000 1 Processor -- start 2024-09-10T23:35:47.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.838+0000 7fa2b59b3000 1 -- start start 2024-09-10T23:35:47.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.838+0000 7fa2b59b3000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b4018ee10 0x555b4024cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.838+0000 7fa2b59b3000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401da9d0 0x555b4024d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.838+0000 7fa2b59b3000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b401db2f0 0x555b4024da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.838+0000 7fa2b59b3000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac2040f0 con 0x555b4018ee10 2024-09-10T23:35:47.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.838+0000 7fa2b59b3000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac0b6ba0 con 0x555b401da9d0 2024-09-10T23:35:47.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.838+0000 7fa2b59b3000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa2ac0cb460 con 0x555b401db2f0 2024-09-10T23:35:47.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.838+0000 7fa2ab400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b4018ee10 0x555b4024cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.838+0000 7fa2ab400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b4018ee10 0x555b4024cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44256/0 (socket says 172.21.15.94:44256) 2024-09-10T23:35:47.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.838+0000 7fa2ab400640 1 -- 172.21.15.94:0/3790512885 learned_addr learned my addr 172.21.15.94:0/3790512885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:47.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2aaa00640 1 --2- 172.21.15.94:0/3790512885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401da9d0 0x555b4024d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2abe00640 1 --2- 172.21.15.94:0/3790512885 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b401db2f0 0x555b4024da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2ab400640 1 -- 172.21.15.94:0/3790512885 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b401db2f0 msgr2=0x555b4024da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:35:47.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2ab400640 1 --2- 172.21.15.94:0/3790512885 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b401db2f0 0x555b4024da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2ab400640 1 -- 172.21.15.94:0/3790512885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401da9d0 msgr2=0x555b4024d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:35:47.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2ab400640 1 --2- 172.21.15.94:0/3790512885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401da9d0 0x555b4024d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2ab400640 1 -- 172.21.15.94:0/3790512885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b40037d90 con 0x555b4018ee10 2024-09-10T23:35:47.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2ab400640 1 --2- 172.21.15.94:0/3790512885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b4018ee10 0x555b4024cfe0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa29c01c760 tx=0x7fa29c0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b78e56b8e5a1a78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:47.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa297400640 1 -- 172.21.15.94:0/3790512885 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c004100 con 0x555b4018ee10 2024-09-10T23:35:47.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa297400640 1 -- 172.21.15.94:0/3790512885 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa29c0c8040 con 0x555b4018ee10 2024-09-10T23:35:47.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa297400640 1 -- 172.21.15.94:0/3790512885 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c004680 con 0x555b4018ee10 2024-09-10T23:35:47.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 -- 172.21.15.94:0/3790512885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b4018ee10 msgr2=0x555b4024cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:47.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/3790512885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b4018ee10 0x555b4024cfe0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa29c01c760 tx=0x7fa29c0bb040 comp rx=0 tx=0).stop 2024-09-10T23:35:47.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 -- 172.21.15.94:0/3790512885 shutdown_connections 2024-09-10T23:35:47.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/3790512885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401da9d0 0x555b4024d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/3790512885 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b401db2f0 0x555b4024da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/3790512885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b4018ee10 0x555b4024cfe0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 -- 172.21.15.94:0/3790512885 >> 172.21.15.94:0/3790512885 conn(0x555b40078130 msgr2=0x555b40196b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:47.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 -- 172.21.15.94:0/3790512885 shutdown_connections 2024-09-10T23:35:47.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 -- 172.21.15.94:0/3790512885 wait complete. 2024-09-10T23:35:47.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 -- start start 2024-09-10T23:35:47.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 Processor -- start 2024-09-10T23:35:47.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b4018ee10 0x555b40250e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b401da9d0 0x555b40251350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401db2f0 0x555b40251890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555b401f1ce0 con 0x555b4018ee10 2024-09-10T23:35:47.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555b3ffe4390 con 0x555b401db2f0 2024-09-10T23:35:47.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x555b3ffeb8f0 con 0x555b401da9d0 2024-09-10T23:35:47.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2b59b3000 10 client.0 _ll_register_callbacks cb 0x7fa29805e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:35:47.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2ab400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b4018ee10 0x555b40250e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2ab400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b4018ee10 0x555b40250e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44258/0 (socket says 172.21.15.94:44258) 2024-09-10T23:35:47.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2ab400640 1 -- 172.21.15.94:0/831743672 learned_addr learned my addr 172.21.15.94:0/831743672 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:47.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2aaa00640 1 --2- 172.21.15.94:0/831743672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b401da9d0 0x555b40251350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2abe00640 1 --2- 172.21.15.94:0/831743672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401db2f0 0x555b40251890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2ab400640 1 -- 172.21.15.94:0/831743672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b401da9d0 msgr2=0x555b40251350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:35:47.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2ab400640 1 --2- 172.21.15.94:0/831743672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b401da9d0 0x555b40251350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2ab400640 1 -- 172.21.15.94:0/831743672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401db2f0 msgr2=0x555b40251890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:35:47.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2ab400640 1 --2- 172.21.15.94:0/831743672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401db2f0 0x555b40251890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2ab400640 1 -- 172.21.15.94:0/831743672 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b3ffa2a90 con 0x555b4018ee10 2024-09-10T23:35:47.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2aaa00640 1 --2- 172.21.15.94:0/831743672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b401da9d0 0x555b40251350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:35:47.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.842+0000 7fa2abe00640 1 --2- 172.21.15.94:0/831743672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401db2f0 0x555b40251890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:35:47.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2ab400640 1 --2- 172.21.15.94:0/831743672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b4018ee10 0x555b40250e10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa29c01c760 tx=0x7fa29c004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6ff8d46eb35cd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:47.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa294c00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:35:47.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa294c00640 1 -- 172.21.15.94:0/831743672 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c0e3060 con 0x555b4018ee10 2024-09-10T23:35:47.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa294c00640 1 -- 172.21.15.94:0/831743672 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa29c004620 con 0x555b4018ee10 2024-09-10T23:35:47.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 1 -- 172.21.15.94:0/831743672 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa29c0c8040 con 0x555b4018ee10 2024-09-10T23:35:47.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa294c00640 1 -- 172.21.15.94:0/831743672 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa29c0e9050 con 0x555b4018ee10 2024-09-10T23:35:47.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 10 client.5973 fetch_fsmap learned FSMap version 34 2024-09-10T23:35:47.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 1 -- 172.21.15.94:0/831743672 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x555b3ff81050 con 0x555b4018ee10 2024-09-10T23:35:47.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa294c00640 1 -- 172.21.15.94:0/831743672 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa29c0c8040 con 0x555b4018ee10 2024-09-10T23:35:47.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa294c00640 1 -- 172.21.15.94:0/831743672 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 34) ==== 3502+0+0 (secure 0 0 0) 0x7fa29c0ec020 con 0x555b4018ee10 2024-09-10T23:35:47.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 10 client.5973 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T23:35:47.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 10 client.5973 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T23:35:47.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 20 client.5973 populate_metadata read hostname 'smithi094' 2024-09-10T23:35:47.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/831743672 >> [v2:172.21.15.94:6833/2905667261,v1:172.21.15.94:6835/2905667261] conn(0x555b401ef450 0x555b401ef830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:47.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 4 client.5973 mds_command: new command op to 195113 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T23:35:47.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 1 -- 172.21.15.94:0/831743672 --> [v2:172.21.15.94:6833/2905667261,v1:172.21.15.94:6835/2905667261] -- command(tid 0: {"prefix": "status"}) -- 0x555b3ffea520 con 0x555b401ef450 2024-09-10T23:35:47.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2aaa00640 1 --2- 172.21.15.94:0/831743672 >> [v2:172.21.15.94:6833/2905667261,v1:172.21.15.94:6835/2905667261] conn(0x555b401ef450 0x555b401ef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:47.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa294c00640 10 client.5973 ms_handle_connect on v2:172.21.15.94:6833/2905667261 2024-09-10T23:35:47.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2aaa00640 1 --2- 172.21.15.94:0/831743672 >> [v2:172.21.15.94:6833/2905667261,v1:172.21.15.94:6835/2905667261] conn(0x555b401ef450 0x555b401ef830 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c84ae9383b027020 server_cookie=c64abbce0fb80f6f in_seq=0 out_seq=0 2024-09-10T23:35:47.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa294c00640 1 -- 172.21.15.94:0/831743672 <== mds.0 v2:172.21.15.94:6833/2905667261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x555b3ffea520 con 0x555b401ef450 2024-09-10T23:35:47.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa294c00640 10 client.5973 handle_command_reply: tid=0 2024-09-10T23:35:47.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 1 client.5973 shutdown 2024-09-10T23:35:47.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 1 -- 172.21.15.94:0/831743672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b4018ee10 msgr2=0x555b40250e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:47.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/831743672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b4018ee10 0x555b40250e10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa29c01c760 tx=0x7fa29c004180 comp rx=0 tx=0).stop 2024-09-10T23:35:47.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 1 -- 172.21.15.94:0/831743672 shutdown_connections 2024-09-10T23:35:47.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/831743672 >> [v2:172.21.15.94:6833/2905667261,v1:172.21.15.94:6835/2905667261] conn(0x555b401ef450 0x555b401ef830 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/831743672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555b401db2f0 0x555b40251890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/831743672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555b401da9d0 0x555b40251350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 1 --2- 172.21.15.94:0/831743672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555b4018ee10 0x555b40250e10 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa294c00640 0 client.5973 ms_handle_reset on v2:172.21.15.94:6833/2905667261 2024-09-10T23:35:47.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 1 -- 172.21.15.94:0/831743672 >> 172.21.15.94:0/831743672 conn(0x555b40078130 msgr2=0x555b401d95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:47.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 1 -- 172.21.15.94:0/831743672 shutdown_connections 2024-09-10T23:35:47.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 1 -- 172.21.15.94:0/831743672 wait complete. 2024-09-10T23:35:47.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b59b3000 20 client.5973 trim_cache size 0 max 16384 2024-09-10T23:35:47.855 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:35:47.855 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "1798cdb9-9937-4918-a569-86b41a92817e", 2024-09-10T23:35:47.855 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-09-10T23:35:47.855 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 195113, 2024-09-10T23:35:47.855 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-10T23:35:47.855 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-10T23:35:47.855 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-10T23:35:47.855 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 10.463374935999999, 2024-09-10T23:35:47.856 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 34, 2024-09-10T23:35:47.856 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 58, 2024-09-10T23:35:47.856 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 58, 2024-09-10T23:35:47.856 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 15.63718364 2024-09-10T23:35:47.856 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:35:47.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b2600640 1 -- 172.21.15.94:0/1679797883 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa28c0727c0 msgr2=0x7fa28c074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:47.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b2600640 1 --2- 172.21.15.94:0/1679797883 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa28c0727c0 0x7fa28c074c80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0af000 tx=0x7fa29c01b530 comp rx=0 tx=0).stop 2024-09-10T23:35:47.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b2600640 1 -- 172.21.15.94:0/1679797883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2ac0d0660 msgr2=0x7fa2ac144030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:47.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b2600640 1 --2- 172.21.15.94:0/1679797883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2ac0d0660 0x7fa2ac144030 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa29800b580 tx=0x7fa29800ba40 comp rx=0 tx=0).stop 2024-09-10T23:35:47.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b2600640 1 -- 172.21.15.94:0/1679797883 shutdown_connections 2024-09-10T23:35:47.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b2600640 1 --2- 172.21.15.94:0/1679797883 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa28c0727c0 0x7fa28c074c80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b2600640 1 --2- 172.21.15.94:0/1679797883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2ac15bf60 0x7fa2ac1585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b2600640 1 --2- 172.21.15.94:0/1679797883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2ac0d0660 0x7fa2ac144030 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b2600640 1 --2- 172.21.15.94:0/1679797883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2ac0cfce0 0x7fa2ac143af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:47.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b2600640 1 -- 172.21.15.94:0/1679797883 >> 172.21.15.94:0/1679797883 conn(0x7fa2ac0d1c60 msgr2=0x7fa2ac0707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:47.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b2600640 1 -- 172.21.15.94:0/1679797883 shutdown_connections 2024-09-10T23:35:47.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:47.846+0000 7fa2b2600640 1 -- 172.21.15.94:0/1679797883 wait complete. 2024-09-10T23:35:47.869 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:35:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.006+0000 7f0a1f000640 1 -- 172.21.15.94:0/1452915524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a18070090 msgr2=0x7f0a180cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.006+0000 7f0a1f000640 1 --2- 172.21.15.94:0/1452915524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a18070090 0x7f0a180cd2a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0a18069ce0 tx=0x7f0a0401c8c0 comp rx=0 tx=0).stop 2024-09-10T23:35:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.006+0000 7f0a1f000640 1 -- 172.21.15.94:0/1452915524 shutdown_connections 2024-09-10T23:35:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.006+0000 7f0a1f000640 1 --2- 172.21.15.94:0/1452915524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a18070090 0x7f0a180cd2a0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.006+0000 7f0a1f000640 1 --2- 172.21.15.94:0/1452915524 >> v2:172.21.15.181:3300/0 conn(0x7f0a1815aab0 0x7f0a180cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.006+0000 7f0a1f000640 1 -- 172.21.15.94:0/1452915524 >> 172.21.15.94:0/1452915524 conn(0x7f0a180b2490 msgr2=0x7f0a180b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.006+0000 7f0a1f000640 1 -- 172.21.15.94:0/1452915524 shutdown_connections 2024-09-10T23:35:48.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.006+0000 7f0a1f000640 1 -- 172.21.15.94:0/1452915524 wait complete. 2024-09-10T23:35:48.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1f000640 1 Processor -- start 2024-09-10T23:35:48.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1f000640 1 -- start start 2024-09-10T23:35:48.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1f000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a180cf540 0x7f0a180b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:48.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1f000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a1815aab0 0x7f0a180ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:48.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1f000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a180b3cd0 0x7f0a180ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:48.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1f000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0a18167f80 con 0x7f0a180cf540 2024-09-10T23:35:48.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1f000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0a18167e00 con 0x7f0a1815aab0 2024-09-10T23:35:48.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1f000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0a18168100 con 0x7f0a180b3cd0 2024-09-10T23:35:48.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1dc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a180cf540 0x7f0a180b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:48.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1dc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a180cf540 0x7f0a180b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44270/0 (socket says 172.21.15.94:44270) 2024-09-10T23:35:48.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1dc00640 1 -- 172.21.15.94:0/4130095722 learned_addr learned my addr 172.21.15.94:0/4130095722 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:48.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1e600640 1 --2- 172.21.15.94:0/4130095722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a180b3cd0 0x7f0a180ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1d200640 1 --2- 172.21.15.94:0/4130095722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a1815aab0 0x7f0a180ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1dc00640 1 -- 172.21.15.94:0/4130095722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a180b3cd0 msgr2=0x7f0a180ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1dc00640 1 --2- 172.21.15.94:0/4130095722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a180b3cd0 0x7f0a180ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1dc00640 1 -- 172.21.15.94:0/4130095722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a1815aab0 msgr2=0x7f0a180ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1dc00640 1 --2- 172.21.15.94:0/4130095722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a1815aab0 0x7f0a180ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1dc00640 1 -- 172.21.15.94:0/4130095722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a180c9cd0 con 0x7f0a180cf540 2024-09-10T23:35:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1d200640 1 --2- 172.21.15.94:0/4130095722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a1815aab0 0x7f0a180ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:35:48.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1dc00640 1 --2- 172.21.15.94:0/4130095722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a180cf540 0x7f0a180b3790 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00ba80 tx=0x7f0a0c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:48.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a15600640 1 -- 172.21.15.94:0/4130095722 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c019070 con 0x7f0a180cf540 2024-09-10T23:35:48.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a15600640 1 -- 172.21.15.94:0/4130095722 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a0c00cd10 con 0x7f0a180cf540 2024-09-10T23:35:48.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a15600640 1 -- 172.21.15.94:0/4130095722 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c0146a0 con 0x7f0a180cf540 2024-09-10T23:35:48.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1f000640 1 -- 172.21.15.94:0/4130095722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a180b5d80 con 0x7f0a180cf540 2024-09-10T23:35:48.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1f000640 1 -- 172.21.15.94:0/4130095722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a180b6340 con 0x7f0a180cf540 2024-09-10T23:35:48.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.010+0000 7f0a1f000640 1 -- 172.21.15.94:0/4130095722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a180c9ea0 con 0x7f0a180cf540 2024-09-10T23:35:48.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.014+0000 7f0a15600640 1 -- 172.21.15.94:0/4130095722 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f0a0c01d3b0 con 0x7f0a180cf540 2024-09-10T23:35:48.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.014+0000 7f0a15600640 1 --2- 172.21.15.94:0/4130095722 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f09f80727c0 0x7f09f8074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.014+0000 7f0a15600640 1 -- 172.21.15.94:0/4130095722 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f0a0c093830 con 0x7f0a180cf540 2024-09-10T23:35:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.014+0000 7f0a1d200640 1 --2- 172.21.15.94:0/4130095722 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f09f80727c0 0x7f09f8074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:48.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.014+0000 7f0a1d200640 1 --2- 172.21.15.94:0/4130095722 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f09f80727c0 0x7f09f8074c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0a040af000 tx=0x7f0a0401b100 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:48.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.018+0000 7f0a15600640 1 -- 172.21.15.94:0/4130095722 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0a0c05eae0 con 0x7f0a180cf540 2024-09-10T23:35:48.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.218+0000 7f0a1f000640 1 -- 172.21.15.94:0/4130095722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0a18159ee0 con 0x7f0a180cf540 2024-09-10T23:35:48.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.218+0000 7f0a15600640 1 -- 172.21.15.94:0/4130095722 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) ==== 53+0+73 (secure 0 0 0) 0x7f0a0c067030 con 0x7f0a180cf540 2024-09-10T23:35:48.222 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:35:48.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.222+0000 7f0a1f000640 1 -- 172.21.15.94:0/4130095722 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f09f80727c0 msgr2=0x7f09f8074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:48.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.222+0000 7f0a1f000640 1 --2- 172.21.15.94:0/4130095722 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f09f80727c0 0x7f09f8074c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0a040af000 tx=0x7f0a0401b100 comp rx=0 tx=0).stop 2024-09-10T23:35:48.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.222+0000 7f0a1f000640 1 -- 172.21.15.94:0/4130095722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a180cf540 msgr2=0x7f0a180b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:48.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.222+0000 7f0a1f000640 1 --2- 172.21.15.94:0/4130095722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a180cf540 0x7f0a180b3790 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00ba80 tx=0x7f0a0c00bf40 comp rx=0 tx=0).stop 2024-09-10T23:35:48.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.222+0000 7f0a1f000640 1 -- 172.21.15.94:0/4130095722 shutdown_connections 2024-09-10T23:35:48.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.222+0000 7f0a1f000640 1 --2- 172.21.15.94:0/4130095722 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f09f80727c0 0x7f09f8074c80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:48.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.222+0000 7f0a1f000640 1 --2- 172.21.15.94:0/4130095722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a1815aab0 0x7f0a180ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:48.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.222+0000 7f0a1f000640 1 --2- 172.21.15.94:0/4130095722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a180b3cd0 0x7f0a180ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:48.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.222+0000 7f0a1f000640 1 --2- 172.21.15.94:0/4130095722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a180cf540 0x7f0a180b3790 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:48.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.222+0000 7f0a1f000640 1 -- 172.21.15.94:0/4130095722 >> 172.21.15.94:0/4130095722 conn(0x7f0a180b2490 msgr2=0x7f0a1806cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:48.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.222+0000 7f0a1f000640 1 -- 172.21.15.94:0/4130095722 shutdown_connections 2024-09-10T23:35:48.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48.222+0000 7f0a1f000640 1 -- 172.21.15.94:0/4130095722 wait complete. 2024-09-10T23:35:48.243 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:35:48.244 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:35:48.244 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T23:35:48.244 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T23:35:48.244 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T23:35:48.244 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:35:48.244 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:35:48.244 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:35:48.244 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:35:48.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:48.244 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T23:35:48.254 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:35:48.254 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:35:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:35:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:35:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T23:35:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:35:48.255 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:35:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:35:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T23:35:48.255 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:35:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:35:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T23:35:48.255 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:35:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:35:48.256 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:35:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:35:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:35:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42263sec preferred_lft 42263sec 2024-09-10T23:35:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-09-10T23:35:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:35:48.256 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:35:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:ed:7d:46:db brd ff:ff:ff:ff:ff:ff 2024-09-10T23:35:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:35:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:35:48.256 INFO:teuthology.orchestra.run.smithi094.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:35:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:35:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:35:48.257 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:35:48.257 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:35:48.257 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:35:48.257 INFO:teuthology.orchestra.run.smithi094.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:35:48.257 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:35:48.257 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:35:48.257 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:35:48.257 INFO:teuthology.orchestra.run.smithi094.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:35:48.257 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:35:48.257 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:35:48.257 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:35:48.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:48.258 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:35:48.315 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T23:35:48.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:48.317 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T23:35:48.369 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T23:35:48.370 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:35:48.370 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:35:48.370 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:35:48.370 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:35:48.370 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:35:48.370 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:35:48.370 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:35:48.370 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:35:48.370 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:35:48.370 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:35:48.371 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:35:48.371 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:35:48.371 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:35:48.371 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:35:48.371 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:35:48.371 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:35:48.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:35:48.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:35:48.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:48.506 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T23:35:48.514 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T23:35:48.514 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T23:35:48.515 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:35:48.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:48.515 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-09-10T23:35:48.577 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T23:35:48.577 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:35:48.577 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T23:35:48.625 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:35:48.626 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:35:48.626 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:35:48.677 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:35:48.737 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:35:48.791 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:35:48.791 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:35:48.791 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:35:48.791 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:35:48.791 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:35:48.791 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:35:48.791 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:35:48.791 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:35:48.791 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:35:48.791 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:35:48.792 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:35:48.792 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:35:48.792 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:35:48.792 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:35:48.792 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:35:48.792 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:35:48.792 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:35:48.792 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:35:48.792 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:35:48.792 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:35:48.792 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:35:48.792 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:35:48.792 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:35:48.792 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:35:48.793 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:35:48.793 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:35:48.793 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:35:48.793 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:35:48.793 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:35:48.793 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:35:48.793 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:35:48.793 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:35:48.793 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:35:48.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:48.794 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:35:48.845 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:35:48.845 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T23:35:48.888 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:35:48.909 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:35:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:35:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:35:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:35:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:35:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:35:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:35:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:35:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:35:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:35:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:35:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:35:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:35:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:35:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:35:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:35:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:35:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:35:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:35:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:35:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:35:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:35:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:35:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:35:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:35:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:35:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:35:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:35:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:35:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:35:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:35:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:35:48.970 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:35:48.970 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:35:48.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:48.971 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:35:48.983 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:2024-09-10T23:35:48.982+0000 7f151bbb4580 -1 init, newargv = 0x564cf098ff30 newargc=15 2024-09-10T23:35:48.983 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[40379]: starting ceph client 2024-09-10T23:35:48.989 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[40379]: starting fuse 2024-09-10T23:35:48.992 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T23:35:48.993 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:35:48.993 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:35:48.993 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-09-10T23:35:48.993 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-09-10T23:35:48.993 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-09-10T23:35:48.993 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-09-10T23:35:48.993 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:35:48.993 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-09-10T23:35:48.993 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T23:35:48.994 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-09-10T23:35:48.994 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T23:35:48.994 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:35:48.994 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-09-10T23:35:48.994 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-09-10T23:35:48.994 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-09-10T23:35:48.994 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-09-10T23:35:48.994 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:35:48.994 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-09-10T23:35:48.994 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:35:48.994 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:35:48.995 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:35:48.995 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-09-10T23:35:48.995 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-09-10T23:35:48.995 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-09-10T23:35:48.995 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:35:48.995 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:35:48.995 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.0")) 2024-09-10T23:35:48.995 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-09-10T23:35:49.113 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.0.40379.asok 2024-09-10T23:35:49.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:35:49.124 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40379.asok 2024-09-10T23:35:49.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:49.124 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.40379.asok status 2024-09-10T23:35:49.296 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:35:49.296 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-09-10T23:35:49.296 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:35:49.296 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:35:49.296 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "0", 2024-09-10T23:35:49.296 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-09-10T23:35:49.296 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T23:35:49.296 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "40379", 2024-09-10T23:35:49.297 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-09-10T23:35:49.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:49.297 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-09-10T23:35:49.297 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-09-10T23:35:49.297 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 5985, 2024-09-10T23:35:49.297 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-09-10T23:35:49.297 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-09-10T23:35:49.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-09-10T23:35:49.297 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 5985 2024-09-10T23:35:49.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:49.297 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:35:49.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:35:49.297 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:35:49.298 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 266394999 2024-09-10T23:35:49.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:49.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:49.298 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:35:49.298 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:35:49.298 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:35:49.298 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 266394999 2024-09-10T23:35:49.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:35:49.298 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.5985 192.168.0.1:0/266394999", 2024-09-10T23:35:49.298 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/266394999", 2024-09-10T23:35:49.298 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-09-10T23:35:49.298 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 34, 2024-09-10T23:35:49.298 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 58, 2024-09-10T23:35:49.298 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:35:49.299 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-09-10T23:35:49.299 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-09-10T23:35:49.299 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:35:49.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:49.312 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-09-10T23:35:49.312 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T23:35:49.321 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-09-10T23:35:49.321 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T23:35:49.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:49.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:35:49.381 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:35:49.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.566+0000 7fb176a00640 1 -- 172.21.15.94:0/2302166179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154004810 msgr2=0x7fb154004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.566+0000 7fb176a00640 1 --2- 172.21.15.94:0/2302166179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154004810 0x7fb154004c90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb158001830 tx=0x7fb15801f010 comp rx=0 tx=0).stop 2024-09-10T23:35:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.566+0000 7fb174a00640 1 -- 172.21.15.94:0/2302166179 reap_dead start 2024-09-10T23:35:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.566+0000 7fb176a00640 1 -- 172.21.15.94:0/2302166179 shutdown_connections 2024-09-10T23:35:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.566+0000 7fb176a00640 1 -- 172.21.15.94:0/2302166179 >> 172.21.15.94:0/2302166179 conn(0x7fb17009aa70 msgr2=0x7fb17009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.566+0000 7fb176a00640 1 -- 172.21.15.94:0/2302166179 shutdown_connections 2024-09-10T23:35:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.566+0000 7fb176a00640 1 -- 172.21.15.94:0/2302166179 wait complete. 2024-09-10T23:35:49.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.566+0000 7fb176a00640 1 Processor -- start 2024-09-10T23:35:49.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb176a00640 1 -- start start 2024-09-10T23:35:49.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb176a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154004810 0x7fb1700b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:49.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb176a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb17015adc0 0x7fb17015b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:49.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb176a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1701e86a0 0x7fb1701e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:49.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb176a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb17015c4e0 con 0x7fb154004810 2024-09-10T23:35:49.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb176a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb17015c360 con 0x7fb1701e86a0 2024-09-10T23:35:49.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb176a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb17015c660 con 0x7fb17015adc0 2024-09-10T23:35:49.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb16fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154004810 0x7fb1700b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:49.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb16fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154004810 0x7fb1700b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44290/0 (socket says 172.21.15.94:44290) 2024-09-10T23:35:49.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb16fe00640 1 -- 172.21.15.94:0/597498479 learned_addr learned my addr 172.21.15.94:0/597498479 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:49.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb167400640 1 --2- 172.21.15.94:0/597498479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb17015adc0 0x7fb17015b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:49.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb174a00640 1 --2- 172.21.15.94:0/597498479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1701e86a0 0x7fb1701e8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:49.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb16fe00640 1 -- 172.21.15.94:0/597498479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb17015adc0 msgr2=0x7fb17015b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:49.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb16fe00640 1 --2- 172.21.15.94:0/597498479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb17015adc0 0x7fb17015b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:49.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb16fe00640 1 -- 172.21.15.94:0/597498479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1701e86a0 msgr2=0x7fb1701e8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:49.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb16fe00640 1 --2- 172.21.15.94:0/597498479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1701e86a0 0x7fb1701e8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:49.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb16fe00640 1 -- 172.21.15.94:0/597498479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17014ced0 con 0x7fb154004810 2024-09-10T23:35:49.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb16fe00640 1 --2- 172.21.15.94:0/597498479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154004810 0x7fb1700b5e20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb160009f90 tx=0x7fb16000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:49.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb16cc00640 1 -- 172.21.15.94:0/597498479 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160017070 con 0x7fb154004810 2024-09-10T23:35:49.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb16cc00640 1 -- 172.21.15.94:0/597498479 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb160015070 con 0x7fb154004810 2024-09-10T23:35:49.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb16cc00640 1 -- 172.21.15.94:0/597498479 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16001c9b0 con 0x7fb154004810 2024-09-10T23:35:49.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb176a00640 1 -- 172.21.15.94:0/597498479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1700b5340 con 0x7fb154004810 2024-09-10T23:35:49.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb176a00640 1 -- 172.21.15.94:0/597498479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1700b5900 con 0x7fb154004810 2024-09-10T23:35:49.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.570+0000 7fb176a00640 1 -- 172.21.15.94:0/597498479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1700b1e70 con 0x7fb154004810 2024-09-10T23:35:49.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.574+0000 7fb16cc00640 1 -- 172.21.15.94:0/597498479 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7fb16001cb50 con 0x7fb154004810 2024-09-10T23:35:49.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.574+0000 7fb16cc00640 1 --2- 172.21.15.94:0/597498479 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb14c0727c0 0x7fb14c074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:49.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.574+0000 7fb16cc00640 1 -- 172.21.15.94:0/597498479 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb160094630 con 0x7fb154004810 2024-09-10T23:35:49.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.574+0000 7fb167400640 1 --2- 172.21.15.94:0/597498479 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb14c0727c0 0x7fb14c074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:49.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.574+0000 7fb167400640 1 --2- 172.21.15.94:0/597498479 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb14c0727c0 0x7fb14c074c80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb1700aed60 tx=0x7fb1580a8f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:49.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.578+0000 7fb16cc00640 1 -- 172.21.15.94:0/597498479 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb16005f290 con 0x7fb154004810 2024-09-10T23:35:49.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.778+0000 7fb176a00640 1 -- 172.21.15.94:0/597498479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb1700d0710 con 0x7fb154004810 2024-09-10T23:35:49.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.778+0000 7fb16cc00640 1 -- 172.21.15.94:0/597498479 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) ==== 53+0+73 (secure 0 0 0) 0x7fb160063cc0 con 0x7fb154004810 2024-09-10T23:35:49.782 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:35:49.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.782+0000 7fb176a00640 1 -- 172.21.15.94:0/597498479 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb14c0727c0 msgr2=0x7fb14c074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:49.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.782+0000 7fb176a00640 1 --2- 172.21.15.94:0/597498479 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb14c0727c0 0x7fb14c074c80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb1700aed60 tx=0x7fb1580a8f90 comp rx=0 tx=0).stop 2024-09-10T23:35:49.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.782+0000 7fb176a00640 1 -- 172.21.15.94:0/597498479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154004810 msgr2=0x7fb1700b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:49.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.782+0000 7fb176a00640 1 --2- 172.21.15.94:0/597498479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154004810 0x7fb1700b5e20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb160009f90 tx=0x7fb16000e720 comp rx=0 tx=0).stop 2024-09-10T23:35:49.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.782+0000 7fb176a00640 1 -- 172.21.15.94:0/597498479 shutdown_connections 2024-09-10T23:35:49.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.782+0000 7fb176a00640 1 --2- 172.21.15.94:0/597498479 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb14c0727c0 0x7fb14c074c80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:49.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.782+0000 7fb176a00640 1 --2- 172.21.15.94:0/597498479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1701e86a0 0x7fb1701e8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:49.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.782+0000 7fb176a00640 1 --2- 172.21.15.94:0/597498479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb17015adc0 0x7fb17015b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:49.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.782+0000 7fb176a00640 1 --2- 172.21.15.94:0/597498479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb154004810 0x7fb1700b5e20 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:49.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.782+0000 7fb176a00640 1 -- 172.21.15.94:0/597498479 >> 172.21.15.94:0/597498479 conn(0x7fb17009aa70 msgr2=0x7fb17015d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:49.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.782+0000 7fb176a00640 1 -- 172.21.15.94:0/597498479 shutdown_connections 2024-09-10T23:35:49.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:49.782+0000 7fb176a00640 1 -- 172.21.15.94:0/597498479 wait complete. 2024-09-10T23:35:49.804 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:35:49.804 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:35:49.804 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-09-10T23:35:49.804 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T23:35:49.804 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T23:35:49.804 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:35:49.805 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:35:49.805 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:35:49.805 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:35:49.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:49.805 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-09-10T23:35:49.817 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:35:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:35:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:35:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:35:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-09-10T23:35:49.817 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:35:49.818 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:35:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:35:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-09-10T23:35:49.818 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:35:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:35:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-09-10T23:35:49.818 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:35:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:35:49.818 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:35:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:35:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:35:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42266sec preferred_lft 42266sec 2024-09-10T23:35:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-09-10T23:35:49.818 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:35:49.818 INFO:teuthology.orchestra.run.smithi181.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:35:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 02:42:3b:2e:09:2d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:35:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:35:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:35:49.819 INFO:teuthology.orchestra.run.smithi181.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:35:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:35:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:35:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:35:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:35:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:35:49.819 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:35:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:35:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:35:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:35:49.819 INFO:teuthology.orchestra.run.smithi181.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:35:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:35:49.819 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:35:49.820 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:35:49.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:49.820 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:35:49.883 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-09-10T23:35:49.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:49.886 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-09-10T23:35:49.942 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-09-10T23:35:49.942 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:35:49.943 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:35:49.943 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:35:49.943 INFO:teuthology.orchestra.run.smithi181.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:35:49.943 INFO:teuthology.orchestra.run.smithi181.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:35:49.943 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:35:49.943 INFO:teuthology.orchestra.run.smithi181.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:35:49.943 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:35:49.943 INFO:teuthology.orchestra.run.smithi181.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:35:49.943 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:35:49.943 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:35:49.943 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:35:49.944 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:35:49.944 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:35:49.944 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:35:49.944 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:35:50.036 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:35:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:35:50.078 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:35:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:35:50.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:50.085 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-09-10T23:35:50.093 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T23:35:50.094 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T23:35:50.095 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:35:50.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:50.095 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-09-10T23:35:50.161 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T23:35:50.161 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:35:50.161 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T23:35:50.209 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:35:50.209 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:35:50.210 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:35:50.261 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:35:50.322 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:35:50.375 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:35:50.375 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:35:50.375 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:35:50.375 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:35:50.375 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:35:50.375 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:35:50.375 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:35:50.375 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:35:50.375 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:35:50.375 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:35:50.376 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:35:50.376 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:35:50.376 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:35:50.376 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:35:50.376 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:35:50.376 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:35:50.376 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:35:50.376 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:35:50.376 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:35:50.376 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:35:50.376 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:35:50.376 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:35:50.376 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:35:50.376 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:35:50.377 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:35:50.377 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:35:50.377 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:35:50.377 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:35:50.377 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:35:50.377 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:35:50.377 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:35:50.377 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:35:50.377 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:35:50.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:50.378 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:35:50.429 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:35:50.429 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T23:35:50.472 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:35:50.492 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:35:50.543 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:35:50.543 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:35:50.543 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:35:50.543 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:35:50.543 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:35:50.543 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:35:50.544 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:35:50.544 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:35:50.544 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:35:50.544 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:35:50.544 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:35:50.544 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:35:50.544 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:35:50.544 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:35:50.544 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:35:50.544 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:35:50.544 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:35:50.544 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:35:50.545 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:35:50.545 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:35:50.545 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:35:50.545 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:35:50.545 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:35:50.545 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:35:50.545 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:35:50.545 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:35:50.545 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:35:50.545 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:35:50.545 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:35:50.545 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:35:50.545 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:35:50.546 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:35:50.546 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:35:50.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:50.547 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:35:50.566 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:2024-09-10T23:35:50.559+0000 7f9cbe8dc580 -1 init, newargv = 0x5616cea45100 newargc=15 2024-09-10T23:35:50.566 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[34889]: starting ceph client 2024-09-10T23:35:50.574 INFO:teuthology.orchestra.run.smithi181.stdout:44 2024-09-10T23:35:50.574 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:35:50.575 DEBUG:teuthology.orchestra.run.smithi181:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:35:50.575 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2024-09-10T23:35:50.575 DEBUG:teuthology.orchestra.run.smithi181:> import re 2024-09-10T23:35:50.575 DEBUG:teuthology.orchestra.run.smithi181:> import os 2024-09-10T23:35:50.575 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2024-09-10T23:35:50.575 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:35:50.575 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2024-09-10T23:35:50.575 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T23:35:50.575 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2024-09-10T23:35:50.575 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T23:35:50.575 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:35:50.575 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2024-09-10T23:35:50.576 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2024-09-10T23:35:50.576 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2024-09-10T23:35:50.576 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2024-09-10T23:35:50.576 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:35:50.576 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2024-09-10T23:35:50.576 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:35:50.576 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:35:50.576 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:35:50.576 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2024-09-10T23:35:50.576 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2024-09-10T23:35:50.576 DEBUG:teuthology.orchestra.run.smithi181:> return f 2024-09-10T23:35:50.576 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:35:50.576 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:35:50.576 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.1")) 2024-09-10T23:35:50.576 DEBUG:teuthology.orchestra.run.smithi181:> ' 2024-09-10T23:35:50.577 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[34889]: starting fuse 2024-09-10T23:35:50.697 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.1.34889.asok 2024-09-10T23:35:50.700 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:35:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:35:50.708 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34889.asok 2024-09-10T23:35:50.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:50.708 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34889.asok status 2024-09-10T23:35:50.875 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T23:35:50.875 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2024-09-10T23:35:50.875 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:35:50.875 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:35:50.875 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "1", 2024-09-10T23:35:50.875 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2024-09-10T23:35:50.875 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T23:35:50.875 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "34889", 2024-09-10T23:35:50.875 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2024-09-10T23:35:50.875 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:35:50.876 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2024-09-10T23:35:50.876 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2024-09-10T23:35:50.876 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 5713, 2024-09-10T23:35:50.876 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2024-09-10T23:35:50.876 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2024-09-10T23:35:50.876 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2024-09-10T23:35:50.876 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 5713 2024-09-10T23:35:50.876 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:35:50.876 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:35:50.876 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:35:50.876 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:35:50.876 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 2391054318 2024-09-10T23:35:50.876 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T23:35:50.877 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:35:50.877 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:35:50.877 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:35:50.877 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:35:50.877 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 2391054318 2024-09-10T23:35:50.877 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:35:50.877 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.5713 192.168.0.1:0/2391054318", 2024-09-10T23:35:50.877 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.1:0/2391054318", 2024-09-10T23:35:50.877 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2024-09-10T23:35:50.877 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 34, 2024-09-10T23:35:50.877 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 58, 2024-09-10T23:35:50.877 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:35:50.877 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2024-09-10T23:35:50.877 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2024-09-10T23:35:50.878 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T23:35:50.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:50.892 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2024-09-10T23:35:50.892 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T23:35:50.901 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2024-09-10T23:35:50.901 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T23:35:50.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:50.901 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:35:50.964 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T23:35:51.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.118+0000 7fedab400640 1 -- 172.21.15.94:0/2399785448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda4152c10 msgr2=0x7feda40b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.118+0000 7fedab400640 1 --2- 172.21.15.94:0/2399785448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda4152c10 0x7feda40b29d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fed94009840 tx=0x7fed9401c7f0 comp rx=0 tx=0).stop 2024-09-10T23:35:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.118+0000 7fedab400640 1 -- 172.21.15.94:0/2399785448 shutdown_connections 2024-09-10T23:35:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.118+0000 7fedab400640 1 --2- 172.21.15.94:0/2399785448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda4152c10 0x7feda40b29d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.118+0000 7fedab400640 1 -- 172.21.15.94:0/2399785448 >> 172.21.15.94:0/2399785448 conn(0x7feda406cdf0 msgr2=0x7feda406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.118+0000 7fedab400640 1 -- 172.21.15.94:0/2399785448 shutdown_connections 2024-09-10T23:35:51.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.118+0000 7fedab400640 1 -- 172.21.15.94:0/2399785448 wait complete. 2024-09-10T23:35:51.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7fedab400640 1 Processor -- start 2024-09-10T23:35:51.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7fedab400640 1 -- start start 2024-09-10T23:35:51.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7fedab400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda40cfd50 0x7feda4144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:51.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7fedab400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feda4152c10 0x7feda4155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:51.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7fedab400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feda415cbc0 0x7feda414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:51.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7fedab400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feda416a5e0 con 0x7feda40cfd50 2024-09-10T23:35:51.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7fedab400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feda416a460 con 0x7feda415cbc0 2024-09-10T23:35:51.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7fedab400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7feda416a760 con 0x7feda4152c10 2024-09-10T23:35:51.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7feda8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda40cfd50 0x7feda4144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:51.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7feda8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda40cfd50 0x7feda4144900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44306/0 (socket says 172.21.15.94:44306) 2024-09-10T23:35:51.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7feda8a00640 1 -- 172.21.15.94:0/552192787 learned_addr learned my addr 172.21.15.94:0/552192787 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:51.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7feda8a00640 1 -- 172.21.15.94:0/552192787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feda4152c10 msgr2=0x7feda4155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:51.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7feda8a00640 1 --2- 172.21.15.94:0/552192787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feda4152c10 0x7feda4155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:51.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7feda8a00640 1 -- 172.21.15.94:0/552192787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feda415cbc0 msgr2=0x7feda414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:51.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7feda8a00640 1 --2- 172.21.15.94:0/552192787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feda415cbc0 0x7feda414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:51.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7feda8a00640 1 -- 172.21.15.94:0/552192787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda4156120 con 0x7feda40cfd50 2024-09-10T23:35:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7feda8a00640 1 --2- 172.21.15.94:0/552192787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda40cfd50 0x7feda4144900 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fed94019400 tx=0x7fed940a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7feda0c00640 1 -- 172.21.15.94:0/552192787 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed940b2040 con 0x7feda40cfd50 2024-09-10T23:35:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7feda0c00640 1 -- 172.21.15.94:0/552192787 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed94004380 con 0x7feda40cfd50 2024-09-10T23:35:51.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7feda0c00640 1 -- 172.21.15.94:0/552192787 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94005240 con 0x7feda40cfd50 2024-09-10T23:35:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7fedab400640 1 -- 172.21.15.94:0/552192787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda4153320 con 0x7feda40cfd50 2024-09-10T23:35:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7fedab400640 1 -- 172.21.15.94:0/552192787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda4152090 con 0x7feda40cfd50 2024-09-10T23:35:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.122+0000 7fedab400640 1 -- 172.21.15.94:0/552192787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda4072bd0 con 0x7feda40cfd50 2024-09-10T23:35:51.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.126+0000 7feda0c00640 1 -- 172.21.15.94:0/552192787 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7fed940b5070 con 0x7feda40cfd50 2024-09-10T23:35:51.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.126+0000 7feda0c00640 1 --2- 172.21.15.94:0/552192787 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fed84072790 0x7fed84074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:51.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.126+0000 7feda0c00640 1 -- 172.21.15.94:0/552192787 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fed9412ce90 con 0x7feda40cfd50 2024-09-10T23:35:51.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.126+0000 7feda3e00640 1 --2- 172.21.15.94:0/552192787 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fed84072790 0x7fed84074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:51.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.126+0000 7feda3e00640 1 --2- 172.21.15.94:0/552192787 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fed84072790 0x7fed84074c50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fed90007fd0 tx=0x7fed90007940 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:51.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.130+0000 7feda0c00640 1 -- 172.21.15.94:0/552192787 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fed94016410 con 0x7feda40cfd50 2024-09-10T23:35:51.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.326+0000 7fedab400640 1 -- 172.21.15.94:0/552192787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7feda40b46c0 con 0x7feda40cfd50 2024-09-10T23:35:51.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.878+0000 7feda0c00640 1 -- 172.21.15.94:0/552192787 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v35) ==== 136+0+0 (secure 0 0 0) 0x7fed940f8150 con 0x7feda40cfd50 2024-09-10T23:35:51.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.882+0000 7fedab400640 1 -- 172.21.15.94:0/552192787 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fed84072790 msgr2=0x7fed84074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:51.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.882+0000 7fedab400640 1 --2- 172.21.15.94:0/552192787 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fed84072790 0x7fed84074c50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fed90007fd0 tx=0x7fed90007940 comp rx=0 tx=0).stop 2024-09-10T23:35:51.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.882+0000 7fedab400640 1 -- 172.21.15.94:0/552192787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda40cfd50 msgr2=0x7feda4144900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:51.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.882+0000 7fedab400640 1 --2- 172.21.15.94:0/552192787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda40cfd50 0x7feda4144900 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fed94019400 tx=0x7fed940a5750 comp rx=0 tx=0).stop 2024-09-10T23:35:51.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.882+0000 7fedab400640 1 -- 172.21.15.94:0/552192787 shutdown_connections 2024-09-10T23:35:51.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.882+0000 7fedab400640 1 --2- 172.21.15.94:0/552192787 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fed84072790 0x7fed84074c50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:51.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.882+0000 7fedab400640 1 --2- 172.21.15.94:0/552192787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feda415cbc0 0x7feda414fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:51.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.882+0000 7fedab400640 1 --2- 172.21.15.94:0/552192787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feda4152c10 0x7feda4155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:51.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.882+0000 7fedab400640 1 --2- 172.21.15.94:0/552192787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feda40cfd50 0x7feda4144900 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:51.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.882+0000 7fedab400640 1 -- 172.21.15.94:0/552192787 >> 172.21.15.94:0/552192787 conn(0x7feda406cdf0 msgr2=0x7feda415d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:51.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.882+0000 7fedab400640 1 -- 172.21.15.94:0/552192787 shutdown_connections 2024-09-10T23:35:51.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:51.882+0000 7fedab400640 1 -- 172.21.15.94:0/552192787 wait complete. 2024-09-10T23:35:51.906 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T23:35:52.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.046+0000 7f9e54400640 1 -- 172.21.15.94:0/1187237329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e4c06d260 msgr2=0x7f9e4c15ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:52.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.046+0000 7f9e54400640 1 --2- 172.21.15.94:0/1187237329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e4c06d260 0x7f9e4c15ca10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009840 tx=0x7f9e3801c7f0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.046+0000 7f9e54400640 1 -- 172.21.15.94:0/1187237329 shutdown_connections 2024-09-10T23:35:52.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.046+0000 7f9e54400640 1 --2- 172.21.15.94:0/1187237329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e4c06d260 0x7f9e4c15ca10 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.046+0000 7f9e54400640 1 -- 172.21.15.94:0/1187237329 >> 172.21.15.94:0/1187237329 conn(0x7f9e4c0d0d80 msgr2=0x7f9e4c0d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:52.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.046+0000 7f9e54400640 1 -- 172.21.15.94:0/1187237329 shutdown_connections 2024-09-10T23:35:52.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.046+0000 7f9e54400640 1 -- 172.21.15.94:0/1187237329 wait complete. 2024-09-10T23:35:52.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e54400640 1 Processor -- start 2024-09-10T23:35:52.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e54400640 1 -- start start 2024-09-10T23:35:52.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e54400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e4c06ce80 0x7f9e4c0b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:52.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e54400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e4c06d260 0x7f9e4c14fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:52.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e54400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e4c13c9a0 0x7f9e4c155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:52.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e54400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c169fb0 con 0x7f9e4c13c9a0 2024-09-10T23:35:52.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e54400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c169e30 con 0x7f9e4c06ce80 2024-09-10T23:35:52.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e54400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4c16a130 con 0x7f9e4c06d260 2024-09-10T23:35:52.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e52400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e4c13c9a0 0x7f9e4c155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:52.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e51a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e4c06ce80 0x7f9e4c0b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:52.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e52400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e4c13c9a0 0x7f9e4c155760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44320/0 (socket says 172.21.15.94:44320) 2024-09-10T23:35:52.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e52400640 1 -- 172.21.15.94:0/3709411288 learned_addr learned my addr 172.21.15.94:0/3709411288 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:52.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e51000640 1 --2- 172.21.15.94:0/3709411288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e4c06d260 0x7f9e4c14fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:52.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e52400640 1 -- 172.21.15.94:0/3709411288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e4c06d260 msgr2=0x7f9e4c14fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:52.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e52400640 1 --2- 172.21.15.94:0/3709411288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e4c06d260 0x7f9e4c14fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e52400640 1 -- 172.21.15.94:0/3709411288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e4c06ce80 msgr2=0x7f9e4c0b5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:52.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e52400640 1 --2- 172.21.15.94:0/3709411288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e4c06ce80 0x7f9e4c0b5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e52400640 1 -- 172.21.15.94:0/3709411288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c129ed0 con 0x7f9e4c13c9a0 2024-09-10T23:35:52.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e52400640 1 --2- 172.21.15.94:0/3709411288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e4c13c9a0 0x7f9e4c155760 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400b490 tx=0x7f9e4400b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:52.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e49600640 1 -- 172.21.15.94:0/3709411288 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44015020 con 0x7f9e4c13c9a0 2024-09-10T23:35:52.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e49600640 1 -- 172.21.15.94:0/3709411288 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e44011920 con 0x7f9e4c13c9a0 2024-09-10T23:35:52.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e49600640 1 -- 172.21.15.94:0/3709411288 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44011db0 con 0x7f9e4c13c9a0 2024-09-10T23:35:52.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.050+0000 7f9e54400640 1 -- 172.21.15.94:0/3709411288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4c12a1b0 con 0x7f9e4c13c9a0 2024-09-10T23:35:52.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.054+0000 7f9e54400640 1 -- 172.21.15.94:0/3709411288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e4c12a770 con 0x7f9e4c13c9a0 2024-09-10T23:35:52.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.054+0000 7f9e54400640 1 -- 172.21.15.94:0/3709411288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e4c15ca10 con 0x7f9e4c13c9a0 2024-09-10T23:35:52.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.054+0000 7f9e49600640 1 -- 172.21.15.94:0/3709411288 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7f9e4401a400 con 0x7f9e4c13c9a0 2024-09-10T23:35:52.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.054+0000 7f9e49600640 1 --2- 172.21.15.94:0/3709411288 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9e2c0727c0 0x7f9e2c074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:52.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.054+0000 7f9e49600640 1 -- 172.21.15.94:0/3709411288 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f9e44093400 con 0x7f9e4c13c9a0 2024-09-10T23:35:52.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.054+0000 7f9e51a00640 1 --2- 172.21.15.94:0/3709411288 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9e2c0727c0 0x7f9e2c074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:52.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.058+0000 7f9e51a00640 1 --2- 172.21.15.94:0/3709411288 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9e2c0727c0 0x7f9e2c074c80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9e40001780 tx=0x7f9e40001f20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:52.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.058+0000 7f9e49600640 1 -- 172.21.15.94:0/3709411288 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9e4401f030 con 0x7f9e4c13c9a0 2024-09-10T23:35:52.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.246+0000 7f9e54400640 1 -- 172.21.15.94:0/3709411288 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f9e4c074560 con 0x7f9e2c0727c0 2024-09-10T23:35:52.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.250+0000 7f9e49600640 1 -- 172.21.15.94:0/3709411288 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f9e4c074560 con 0x7f9e2c0727c0 2024-09-10T23:35:52.252 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:35:52.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:35:52.252 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephfs" 2024-09-10T23:35:52.252 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:35:52.252 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:35:52.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.250+0000 7f9e54400640 1 -- 172.21.15.94:0/3709411288 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9e2c0727c0 msgr2=0x7f9e2c074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:52.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.250+0000 7f9e54400640 1 --2- 172.21.15.94:0/3709411288 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9e2c0727c0 0x7f9e2c074c80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9e40001780 tx=0x7f9e40001f20 comp rx=0 tx=0).stop 2024-09-10T23:35:52.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.250+0000 7f9e54400640 1 -- 172.21.15.94:0/3709411288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e4c13c9a0 msgr2=0x7f9e4c155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:52.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.250+0000 7f9e54400640 1 --2- 172.21.15.94:0/3709411288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e4c13c9a0 0x7f9e4c155760 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400b490 tx=0x7f9e4400b950 comp rx=0 tx=0).stop 2024-09-10T23:35:52.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.254+0000 7f9e54400640 1 -- 172.21.15.94:0/3709411288 shutdown_connections 2024-09-10T23:35:52.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.254+0000 7f9e54400640 1 --2- 172.21.15.94:0/3709411288 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9e2c0727c0 0x7f9e2c074c80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.254+0000 7f9e54400640 1 --2- 172.21.15.94:0/3709411288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e4c13c9a0 0x7f9e4c155760 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.254+0000 7f9e54400640 1 --2- 172.21.15.94:0/3709411288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e4c06d260 0x7f9e4c14fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.254+0000 7f9e54400640 1 --2- 172.21.15.94:0/3709411288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e4c06ce80 0x7f9e4c0b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.254+0000 7f9e54400640 1 -- 172.21.15.94:0/3709411288 >> 172.21.15.94:0/3709411288 conn(0x7f9e4c0d0d80 msgr2=0x7f9e4c0b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:52.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.254+0000 7f9e54400640 1 -- 172.21.15.94:0/3709411288 shutdown_connections 2024-09-10T23:35:52.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.254+0000 7f9e54400640 1 -- 172.21.15.94:0/3709411288 wait complete. 2024-09-10T23:35:52.273 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:35:52.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.418+0000 7fb4ae000640 1 -- 172.21.15.94:0/646769238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb48c004810 msgr2=0x7fb48c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:52.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.418+0000 7fb4ae000640 1 --2- 172.21.15.94:0/646769238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb48c004810 0x7fb48c004c90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb4a806dbf0 tx=0x7fb49401d210 comp rx=0 tx=0).stop 2024-09-10T23:35:52.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.418+0000 7fb4a7e00640 1 -- 172.21.15.94:0/646769238 reap_dead start 2024-09-10T23:35:52.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.418+0000 7fb4ae000640 1 -- 172.21.15.94:0/646769238 shutdown_connections 2024-09-10T23:35:52.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.418+0000 7fb4ae000640 1 -- 172.21.15.94:0/646769238 >> 172.21.15.94:0/646769238 conn(0x7fb4a80d1c90 msgr2=0x7fb4a80d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:52.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.418+0000 7fb4ae000640 1 -- 172.21.15.94:0/646769238 shutdown_connections 2024-09-10T23:35:52.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.418+0000 7fb4ae000640 1 -- 172.21.15.94:0/646769238 wait complete. 2024-09-10T23:35:52.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.418+0000 7fb4ae000640 1 Processor -- start 2024-09-10T23:35:52.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.418+0000 7fb4ae000640 1 -- start start 2024-09-10T23:35:52.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4ae000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb48c004810 0x7fb4a81f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4ae000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4a81523d0 0x7fb4a80cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4ae000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4a80af9d0 0x7fb4a80afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4ae000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a815f830 con 0x7fb48c004810 2024-09-10T23:35:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4ae000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a815f6b0 con 0x7fb4a80af9d0 2024-09-10T23:35:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4ae000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a80b2790 con 0x7fb4a81523d0 2024-09-10T23:35:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4a7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb48c004810 0x7fb4a81f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4a7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb48c004810 0x7fb4a81f62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44346/0 (socket says 172.21.15.94:44346) 2024-09-10T23:35:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4a7400640 1 -- 172.21.15.94:0/1426741391 learned_addr learned my addr 172.21.15.94:0/1426741391 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4a7e00640 1 --2- 172.21.15.94:0/1426741391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4a80af9d0 0x7fb4a80afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4a6a00640 1 --2- 172.21.15.94:0/1426741391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4a81523d0 0x7fb4a80cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4a7400640 1 -- 172.21.15.94:0/1426741391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4a81523d0 msgr2=0x7fb4a80cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4a7400640 1 --2- 172.21.15.94:0/1426741391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4a81523d0 0x7fb4a80cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4a7400640 1 -- 172.21.15.94:0/1426741391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4a80af9d0 msgr2=0x7fb4a80afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4a7400640 1 --2- 172.21.15.94:0/1426741391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4a80af9d0 0x7fb4a80afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4a7400640 1 -- 172.21.15.94:0/1426741391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a8152850 con 0x7fb48c004810 2024-09-10T23:35:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4a7e00640 1 --2- 172.21.15.94:0/1426741391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4a80af9d0 0x7fb4a80afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:35:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4a6a00640 1 --2- 172.21.15.94:0/1426741391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4a81523d0 0x7fb4a80cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:35:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4a7400640 1 --2- 172.21.15.94:0/1426741391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb48c004810 0x7fb4a81f62a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb498003f20 tx=0x7fb49800f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb49f400640 1 -- 172.21.15.94:0/1426741391 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498015070 con 0x7fb48c004810 2024-09-10T23:35:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb49f400640 1 -- 172.21.15.94:0/1426741391 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb498011840 con 0x7fb48c004810 2024-09-10T23:35:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb49f400640 1 -- 172.21.15.94:0/1426741391 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498011c70 con 0x7fb48c004810 2024-09-10T23:35:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4ae000640 1 -- 172.21.15.94:0/1426741391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a80b6440 con 0x7fb48c004810 2024-09-10T23:35:52.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4ae000640 1 -- 172.21.15.94:0/1426741391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a80b5960 con 0x7fb48c004810 2024-09-10T23:35:52.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb4ae000640 1 -- 172.21.15.94:0/1426741391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a80b0390 con 0x7fb48c004810 2024-09-10T23:35:52.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.422+0000 7fb49f400640 1 -- 172.21.15.94:0/1426741391 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7fb4980103c0 con 0x7fb48c004810 2024-09-10T23:35:52.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.426+0000 7fb49f400640 1 --2- 172.21.15.94:0/1426741391 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4880727c0 0x7fb488074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:52.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.426+0000 7fb49f400640 1 -- 172.21.15.94:0/1426741391 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb498094260 con 0x7fb48c004810 2024-09-10T23:35:52.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.426+0000 7fb4a6a00640 1 --2- 172.21.15.94:0/1426741391 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4880727c0 0x7fb488074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:52.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.426+0000 7fb4a6a00640 1 --2- 172.21.15.94:0/1426741391 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4880727c0 0x7fb488074c80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb4940ae000 tx=0x7fb494019fb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:52.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.430+0000 7fb49f400640 1 -- 172.21.15.94:0/1426741391 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb49805f510 con 0x7fb48c004810 2024-09-10T23:35:52.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.594+0000 7fb4ae000640 1 -- 172.21.15.94:0/1426741391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb4a8071660 con 0x7fb48c004810 2024-09-10T23:35:52.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.594+0000 7fb49f400640 1 -- 172.21.15.94:0/1426741391 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7fb49805ee10 con 0x7fb48c004810 2024-09-10T23:35:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.598+0000 7fb4ae000640 1 -- 172.21.15.94:0/1426741391 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4880727c0 msgr2=0x7fb488074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.598+0000 7fb4ae000640 1 --2- 172.21.15.94:0/1426741391 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4880727c0 0x7fb488074c80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb4940ae000 tx=0x7fb494019fb0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.598+0000 7fb4ae000640 1 -- 172.21.15.94:0/1426741391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb48c004810 msgr2=0x7fb4a81f62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.598+0000 7fb4ae000640 1 --2- 172.21.15.94:0/1426741391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb48c004810 0x7fb4a81f62a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb498003f20 tx=0x7fb49800f7e0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.598+0000 7fb4ae000640 1 -- 172.21.15.94:0/1426741391 shutdown_connections 2024-09-10T23:35:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.598+0000 7fb4ae000640 1 --2- 172.21.15.94:0/1426741391 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4880727c0 0x7fb488074c80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.598+0000 7fb4ae000640 1 --2- 172.21.15.94:0/1426741391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4a80af9d0 0x7fb4a80afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.598+0000 7fb4ae000640 1 --2- 172.21.15.94:0/1426741391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4a81523d0 0x7fb4a80cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.598+0000 7fb4ae000640 1 --2- 172.21.15.94:0/1426741391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb48c004810 0x7fb4a81f62a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.598+0000 7fb4ae000640 1 -- 172.21.15.94:0/1426741391 >> 172.21.15.94:0/1426741391 conn(0x7fb4a80d1c90 msgr2=0x7fb4a8071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.598+0000 7fb4ae000640 1 -- 172.21.15.94:0/1426741391 shutdown_connections 2024-09-10T23:35:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.598+0000 7fb4ae000640 1 -- 172.21.15.94:0/1426741391 wait complete. 2024-09-10T23:35:52.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:52.618 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol98180) 2024-09-10T23:35:52.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:52.649 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T23:35:52.704 INFO:teuthology.orchestra.run.smithi094.stdout:cephfs_data 2024-09-10T23:35:52.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:52.705 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvol98180) 2024-09-10T23:35:52.774 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol74911 2024-09-10T23:35:52.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25a00640 1 -- 172.21.15.94:0/1984156681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae04004810 msgr2=0x7fae04004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:52.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25a00640 1 --2- 172.21.15.94:0/1984156681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae04004810 0x7fae04004c90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fae08006f90 tx=0x7fae0801f630 comp rx=0 tx=0).stop 2024-09-10T23:35:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25a00640 1 -- 172.21.15.94:0/1984156681 shutdown_connections 2024-09-10T23:35:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25a00640 1 --2- 172.21.15.94:0/1984156681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae04004810 0x7fae04004c90 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25a00640 1 -- 172.21.15.94:0/1984156681 >> 172.21.15.94:0/1984156681 conn(0x7fae200d1b00 msgr2=0x7fae200d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25a00640 1 -- 172.21.15.94:0/1984156681 shutdown_connections 2024-09-10T23:35:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25a00640 1 -- 172.21.15.94:0/1984156681 wait complete. 2024-09-10T23:35:52.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25a00640 1 Processor -- start 2024-09-10T23:35:52.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25a00640 1 -- start start 2024-09-10T23:35:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae04004810 0x7fae201f6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae200cb080 0x7fae2015dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae2015e510 0x7fae200cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:52.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fae200af740 con 0x7fae2015e510 2024-09-10T23:35:52.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fae200af5c0 con 0x7fae04004810 2024-09-10T23:35:52.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fae200af8c0 con 0x7fae200cb080 2024-09-10T23:35:52.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae2015e510 0x7fae200cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:52.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae1fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae04004810 0x7fae201f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:52.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae17400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae200cb080 0x7fae2015dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:52.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae2015e510 0x7fae200cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44358/0 (socket says 172.21.15.94:44358) 2024-09-10T23:35:52.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25000640 1 -- 172.21.15.94:0/4076681876 learned_addr learned my addr 172.21.15.94:0/4076681876 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:52.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25000640 1 -- 172.21.15.94:0/4076681876 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae200cb080 msgr2=0x7fae2015dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:52.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25000640 1 --2- 172.21.15.94:0/4076681876 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae200cb080 0x7fae2015dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25000640 1 -- 172.21.15.94:0/4076681876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae04004810 msgr2=0x7fae201f6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:52.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25000640 1 --2- 172.21.15.94:0/4076681876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae04004810 0x7fae201f6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:52.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.962+0000 7fae25000640 1 -- 172.21.15.94:0/4076681876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae201f2ae0 con 0x7fae2015e510 2024-09-10T23:35:52.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.966+0000 7fae25000640 1 --2- 172.21.15.94:0/4076681876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae2015e510 0x7fae200cd5c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fae200b23f0 tx=0x7fae18001680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:52.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.966+0000 7fae1cc00640 1 -- 172.21.15.94:0/4076681876 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180107d0 con 0x7fae2015e510 2024-09-10T23:35:52.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.966+0000 7fae1cc00640 1 -- 172.21.15.94:0/4076681876 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae18016070 con 0x7fae2015e510 2024-09-10T23:35:52.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.966+0000 7fae25a00640 1 -- 172.21.15.94:0/4076681876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae201f2dc0 con 0x7fae2015e510 2024-09-10T23:35:52.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.966+0000 7fae1cc00640 1 -- 172.21.15.94:0/4076681876 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18010d20 con 0x7fae2015e510 2024-09-10T23:35:52.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.966+0000 7fae25a00640 1 -- 172.21.15.94:0/4076681876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae201f3380 con 0x7fae2015e510 2024-09-10T23:35:52.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.966+0000 7fae25a00640 1 -- 172.21.15.94:0/4076681876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae2015e990 con 0x7fae2015e510 2024-09-10T23:35:52.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.966+0000 7fae1cc00640 1 -- 172.21.15.94:0/4076681876 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 96547+0+0 (secure 0 0 0) 0x7fae1801b460 con 0x7fae2015e510 2024-09-10T23:35:52.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.966+0000 7fae1cc00640 1 --2- 172.21.15.94:0/4076681876 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fadfc0727c0 0x7fadfc074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:52.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.966+0000 7fae1cc00640 1 -- 172.21.15.94:0/4076681876 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fae18093d00 con 0x7fae2015e510 2024-09-10T23:35:52.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.966+0000 7fae1fe00640 1 --2- 172.21.15.94:0/4076681876 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fadfc0727c0 0x7fadfc074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:52.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.970+0000 7fae1fe00640 1 --2- 172.21.15.94:0/4076681876 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fadfc0727c0 0x7fadfc074c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fae10003a90 tx=0x7fae10009510 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:52.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:52.970+0000 7fae1cc00640 1 -- 172.21.15.94:0/4076681876 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fae1805e960 con 0x7fae2015e510 2024-09-10T23:35:53.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:53.158+0000 7fae25a00640 1 -- 172.21.15.94:0/4076681876 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol74911", "target": ["mon-mgr", ""]}) -- 0x7fae200afdd0 con 0x7fadfc0727c0 2024-09-10T23:35:53.168 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:35:53.167+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:35:53.168 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:35:53.167+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:35:53.168 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:35:53.167+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:35:53.168 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:35:53.167+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:35:53.168 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:35:53.167+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:35:53.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:53.202+0000 7fae1cc00640 1 -- 172.21.15.94:0/4076681876 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fae200afdd0 con 0x7fadfc0727c0 2024-09-10T23:35:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:53.206+0000 7fae25a00640 1 -- 172.21.15.94:0/4076681876 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fadfc0727c0 msgr2=0x7fadfc074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:53.206+0000 7fae25a00640 1 --2- 172.21.15.94:0/4076681876 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fadfc0727c0 0x7fadfc074c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fae10003a90 tx=0x7fae10009510 comp rx=0 tx=0).stop 2024-09-10T23:35:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:53.206+0000 7fae25a00640 1 -- 172.21.15.94:0/4076681876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae2015e510 msgr2=0x7fae200cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:53.206+0000 7fae25a00640 1 --2- 172.21.15.94:0/4076681876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae2015e510 0x7fae200cd5c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fae200b23f0 tx=0x7fae18001680 comp rx=0 tx=0).stop 2024-09-10T23:35:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:53.206+0000 7fae25a00640 1 -- 172.21.15.94:0/4076681876 shutdown_connections 2024-09-10T23:35:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:53.206+0000 7fae25a00640 1 --2- 172.21.15.94:0/4076681876 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fadfc0727c0 0x7fadfc074c80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:53.206+0000 7fae25a00640 1 --2- 172.21.15.94:0/4076681876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae2015e510 0x7fae200cd5c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:53.206+0000 7fae25a00640 1 --2- 172.21.15.94:0/4076681876 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae200cb080 0x7fae2015dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:53.206+0000 7fae25a00640 1 --2- 172.21.15.94:0/4076681876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae04004810 0x7fae201f6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:53.206+0000 7fae25a00640 1 -- 172.21.15.94:0/4076681876 >> 172.21.15.94:0/4076681876 conn(0x7fae200d1b00 msgr2=0x7fae2015bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:53.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:53.206+0000 7fae25a00640 1 -- 172.21.15.94:0/4076681876 shutdown_connections 2024-09-10T23:35:53.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:53.206+0000 7fae25a00640 1 -- 172.21.15.94:0/4076681876 wait complete. 2024-09-10T23:35:53.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:53.226 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvol74911/.meta ./volumes/_nogroup/subvol98180) 2024-09-10T23:35:57.232 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol98180 2024-09-10T23:35:57.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.382+0000 7fad7cc00640 1 -- 172.21.15.94:0/667449992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad5c005220 msgr2=0x7fad5c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:57.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.382+0000 7fad7cc00640 1 --2- 172.21.15.94:0/667449992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad5c005220 0x7fad5c00b9d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fad7000c700 tx=0x7fad7001c6c0 comp rx=0 tx=0).stop 2024-09-10T23:35:57.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.382+0000 7fad7cc00640 1 -- 172.21.15.94:0/667449992 shutdown_connections 2024-09-10T23:35:57.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.382+0000 7fad7cc00640 1 --2- 172.21.15.94:0/667449992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad5c005220 0x7fad5c00b9d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:57.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.382+0000 7fad7cc00640 1 -- 172.21.15.94:0/667449992 >> 172.21.15.94:0/667449992 conn(0x7fad780cfaf0 msgr2=0x7fad780cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:57.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.382+0000 7fad7cc00640 1 -- 172.21.15.94:0/667449992 shutdown_connections 2024-09-10T23:35:57.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.382+0000 7fad7cc00640 1 -- 172.21.15.94:0/667449992 wait complete. 2024-09-10T23:35:57.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad7cc00640 1 Processor -- start 2024-09-10T23:35:57.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad7cc00640 1 -- start start 2024-09-10T23:35:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad7cc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad5c005220 0x7fad7815ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad7cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad78152a80 0x7fad781517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad7cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad780af6b0 0x7fad780afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad7cc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad780c49b0 con 0x7fad78152a80 2024-09-10T23:35:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad7cc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fad780c4b30 con 0x7fad780af6b0 2024-09-10T23:35:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad7cc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fad780c4cb0 con 0x7fad5c005220 2024-09-10T23:35:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad6d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad78152a80 0x7fad781517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad6d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad78152a80 0x7fad781517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36162/0 (socket says 172.21.15.94:36162) 2024-09-10T23:35:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad6d400640 1 -- 172.21.15.94:0/4058471635 learned_addr learned my addr 172.21.15.94:0/4058471635 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad75e00640 1 --2- 172.21.15.94:0/4058471635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad5c005220 0x7fad7815ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad76800640 1 --2- 172.21.15.94:0/4058471635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad780af6b0 0x7fad780afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad6d400640 1 -- 172.21.15.94:0/4058471635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad5c005220 msgr2=0x7fad7815ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:57.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad6d400640 1 --2- 172.21.15.94:0/4058471635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad5c005220 0x7fad7815ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad6d400640 1 -- 172.21.15.94:0/4058471635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad780af6b0 msgr2=0x7fad780afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad6d400640 1 --2- 172.21.15.94:0/4058471635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad780af6b0 0x7fad780afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad6d400640 1 -- 172.21.15.94:0/4058471635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad781545a0 con 0x7fad78152a80 2024-09-10T23:35:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad76800640 1 --2- 172.21.15.94:0/4058471635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad780af6b0 0x7fad780afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:35:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad6d400640 1 --2- 172.21.15.94:0/4058471635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad78152a80 0x7fad781517b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fad60010a10 tx=0x7fad60010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad6ea00640 1 -- 172.21.15.94:0/4058471635 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6000ee00 con 0x7fad78152a80 2024-09-10T23:35:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad6ea00640 1 -- 172.21.15.94:0/4058471635 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad60012450 con 0x7fad78152a80 2024-09-10T23:35:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad6ea00640 1 -- 172.21.15.94:0/4058471635 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60012880 con 0x7fad78152a80 2024-09-10T23:35:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad7cc00640 1 -- 172.21.15.94:0/4058471635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad78154880 con 0x7fad78152a80 2024-09-10T23:35:57.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.386+0000 7fad7cc00640 1 -- 172.21.15.94:0/4058471635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad78154e40 con 0x7fad78152a80 2024-09-10T23:35:57.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.390+0000 7fad7cc00640 1 -- 172.21.15.94:0/4058471635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad780caea0 con 0x7fad78152a80 2024-09-10T23:35:57.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.390+0000 7fad6ea00640 1 -- 172.21.15.94:0/4058471635 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 10) ==== 96598+0+0 (secure 0 0 0) 0x7fad6000f040 con 0x7fad78152a80 2024-09-10T23:35:57.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.390+0000 7fad6ea00640 1 --2- 172.21.15.94:0/4058471635 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fad54072820 0x7fad54074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:57.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.390+0000 7fad6ea00640 1 -- 172.21.15.94:0/4058471635 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fad600943c0 con 0x7fad78152a80 2024-09-10T23:35:57.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.390+0000 7fad75e00640 1 --2- 172.21.15.94:0/4058471635 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fad54072820 0x7fad54074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:57.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.390+0000 7fad75e00640 1 --2- 172.21.15.94:0/4058471635 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fad54072820 0x7fad54074ce0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fad68001060 tx=0x7fad680067b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:57.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.394+0000 7fad6ea00640 1 -- 172.21.15.94:0/4058471635 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fad6005efd0 con 0x7fad78152a80 2024-09-10T23:35:57.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.586+0000 7fad7cc00640 1 -- 172.21.15.94:0/4058471635 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol98180", "target": ["mon-mgr", ""]}) -- 0x7fad780d0850 con 0x7fad54072820 2024-09-10T23:35:57.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.622+0000 7fad6ea00640 1 -- 172.21.15.94:0/4058471635 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7fad780d0850 con 0x7fad54072820 2024-09-10T23:35:57.626 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/_nogroup/subvol98180 2024-09-10T23:35:57.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.626+0000 7fad7cc00640 1 -- 172.21.15.94:0/4058471635 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fad54072820 msgr2=0x7fad54074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:57.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.626+0000 7fad7cc00640 1 --2- 172.21.15.94:0/4058471635 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fad54072820 0x7fad54074ce0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fad68001060 tx=0x7fad680067b0 comp rx=0 tx=0).stop 2024-09-10T23:35:57.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.626+0000 7fad7cc00640 1 -- 172.21.15.94:0/4058471635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad78152a80 msgr2=0x7fad781517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:57.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.626+0000 7fad7cc00640 1 --2- 172.21.15.94:0/4058471635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad78152a80 0x7fad781517b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fad60010a10 tx=0x7fad60010ed0 comp rx=0 tx=0).stop 2024-09-10T23:35:57.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.626+0000 7fad7cc00640 1 -- 172.21.15.94:0/4058471635 shutdown_connections 2024-09-10T23:35:57.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.626+0000 7fad7cc00640 1 --2- 172.21.15.94:0/4058471635 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fad54072820 0x7fad54074ce0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:57.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.626+0000 7fad7cc00640 1 --2- 172.21.15.94:0/4058471635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad780af6b0 0x7fad780afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:57.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.626+0000 7fad7cc00640 1 --2- 172.21.15.94:0/4058471635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad78152a80 0x7fad781517b0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:57.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.626+0000 7fad7cc00640 1 --2- 172.21.15.94:0/4058471635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad5c005220 0x7fad7815ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:57.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.626+0000 7fad7cc00640 1 -- 172.21.15.94:0/4058471635 >> 172.21.15.94:0/4058471635 conn(0x7fad780cfaf0 msgr2=0x7fad78072d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:57.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.626+0000 7fad7cc00640 1 -- 172.21.15.94:0/4058471635 shutdown_connections 2024-09-10T23:35:57.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.626+0000 7fad7cc00640 1 -- 172.21.15.94:0/4058471635 wait complete. 2024-09-10T23:35:57.647 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol98180 2024-09-10T23:35:57.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.790+0000 7f218ae00640 1 -- 172.21.15.94:0/1676273262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21840ce890 msgr2=0x7f21840cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:57.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.790+0000 7f218ae00640 1 --2- 172.21.15.94:0/1676273262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21840ce890 0x7f21840cd710 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2174009840 tx=0x7f217401c790 comp rx=0 tx=0).stop 2024-09-10T23:35:57.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.790+0000 7f218ae00640 1 -- 172.21.15.94:0/1676273262 shutdown_connections 2024-09-10T23:35:57.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.790+0000 7f218ae00640 1 --2- 172.21.15.94:0/1676273262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21840ce890 0x7f21840cd710 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:57.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.790+0000 7f218ae00640 1 -- 172.21.15.94:0/1676273262 >> 172.21.15.94:0/1676273262 conn(0x7f21840d1c60 msgr2=0x7f21840d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:57.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.790+0000 7f218ae00640 1 -- 172.21.15.94:0/1676273262 shutdown_connections 2024-09-10T23:35:57.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.790+0000 7f218ae00640 1 -- 172.21.15.94:0/1676273262 wait complete. 2024-09-10T23:35:57.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.790+0000 7f218ae00640 1 Processor -- start 2024-09-10T23:35:57.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.790+0000 7f218ae00640 1 -- start start 2024-09-10T23:35:57.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.790+0000 7f218ae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21840b1100 0x7f21840b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:57.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.790+0000 7f218ae00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21840ce1e0 0x7f218415c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:57.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f218ae00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21840ce890 0x7f218415bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:57.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f218ae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f218416a300 con 0x7f21840b1100 2024-09-10T23:35:57.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f218ae00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f218416a180 con 0x7f21840ce890 2024-09-10T23:35:57.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f218ae00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f218416a480 con 0x7f21840ce1e0 2024-09-10T23:35:57.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f2183e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21840b1100 0x7f21840b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f2183e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21840b1100 0x7f21840b6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36178/0 (socket says 172.21.15.94:36178) 2024-09-10T23:35:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f2183e00640 1 -- 172.21.15.94:0/4050904751 learned_addr learned my addr 172.21.15.94:0/4050904751 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:35:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f2183e00640 1 -- 172.21.15.94:0/4050904751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21840ce1e0 msgr2=0x7f218415c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f2183e00640 1 --2- 172.21.15.94:0/4050904751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21840ce1e0 0x7f218415c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f2183e00640 1 -- 172.21.15.94:0/4050904751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21840ce890 msgr2=0x7f218415bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f2183e00640 1 --2- 172.21.15.94:0/4050904751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21840ce890 0x7f218415bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f2183e00640 1 -- 172.21.15.94:0/4050904751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21841f7d20 con 0x7f21840b1100 2024-09-10T23:35:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f2183e00640 1 --2- 172.21.15.94:0/4050904751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21840b1100 0x7f21840b6ac0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2174000c10 tx=0x7f21740a65a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f217be00640 1 -- 172.21.15.94:0/4050904751 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174005240 con 0x7f21840b1100 2024-09-10T23:35:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f217be00640 1 -- 172.21.15.94:0/4050904751 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21740053e0 con 0x7f21840b1100 2024-09-10T23:35:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f217be00640 1 -- 172.21.15.94:0/4050904751 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21740046d0 con 0x7f21840b1100 2024-09-10T23:35:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f218ae00640 1 -- 172.21.15.94:0/4050904751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21840b70a0 con 0x7f21840b1100 2024-09-10T23:35:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f218ae00640 1 -- 172.21.15.94:0/4050904751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f218415c050 con 0x7f21840b1100 2024-09-10T23:35:57.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f218ae00640 1 -- 172.21.15.94:0/4050904751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21840cd710 con 0x7f21840b1100 2024-09-10T23:35:57.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f217be00640 1 -- 172.21.15.94:0/4050904751 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 10) ==== 96598+0+0 (secure 0 0 0) 0x7f2174004870 con 0x7f21840b1100 2024-09-10T23:35:57.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.794+0000 7f217be00640 1 --2- 172.21.15.94:0/4050904751 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2164072820 0x7f2164074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:35:57.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.798+0000 7f217be00640 1 -- 172.21.15.94:0/4050904751 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f217412d910 con 0x7f21840b1100 2024-09-10T23:35:57.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.798+0000 7f2183400640 1 --2- 172.21.15.94:0/4050904751 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2164072820 0x7f2164074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:35:57.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.798+0000 7f2183400640 1 --2- 172.21.15.94:0/4050904751 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2164072820 0x7f2164074ce0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f21700046c0 tx=0x7f2170007910 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:35:57.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.798+0000 7f217be00640 1 -- 172.21.15.94:0/4050904751 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f21740bf030 con 0x7f21840b1100 2024-09-10T23:35:57.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.990+0000 7f218ae00640 1 -- 172.21.15.94:0/4050904751 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol98180", "target": ["mon-mgr", ""]}) -- 0x7f21840b4640 con 0x7f2164072820 2024-09-10T23:35:57.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.994+0000 7f217be00640 1 -- 172.21.15.94:0/4050904751 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f21840b4640 con 0x7f2164072820 2024-09-10T23:35:57.998 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/_nogroup/subvol98180 2024-09-10T23:35:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.998+0000 7f218ae00640 1 -- 172.21.15.94:0/4050904751 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2164072820 msgr2=0x7f2164074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.998+0000 7f218ae00640 1 --2- 172.21.15.94:0/4050904751 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2164072820 0x7f2164074ce0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f21700046c0 tx=0x7f2170007910 comp rx=0 tx=0).stop 2024-09-10T23:35:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.998+0000 7f218ae00640 1 -- 172.21.15.94:0/4050904751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21840b1100 msgr2=0x7f21840b6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:35:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.998+0000 7f218ae00640 1 --2- 172.21.15.94:0/4050904751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21840b1100 0x7f21840b6ac0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2174000c10 tx=0x7f21740a65a0 comp rx=0 tx=0).stop 2024-09-10T23:35:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.998+0000 7f218ae00640 1 -- 172.21.15.94:0/4050904751 shutdown_connections 2024-09-10T23:35:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.998+0000 7f218ae00640 1 --2- 172.21.15.94:0/4050904751 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2164072820 0x7f2164074ce0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:58.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.998+0000 7f218ae00640 1 --2- 172.21.15.94:0/4050904751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21840ce890 0x7f218415bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:58.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.998+0000 7f218ae00640 1 --2- 172.21.15.94:0/4050904751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21840ce1e0 0x7f218415c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:58.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.998+0000 7f218ae00640 1 --2- 172.21.15.94:0/4050904751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21840b1100 0x7f21840b6ac0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:35:58.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.998+0000 7f218ae00640 1 -- 172.21.15.94:0/4050904751 >> 172.21.15.94:0/4050904751 conn(0x7f21840d1c60 msgr2=0x7f2184071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:35:58.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.998+0000 7f218ae00640 1 -- 172.21.15.94:0/4050904751 shutdown_connections 2024-09-10T23:35:58.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:35:57.998+0000 7f218ae00640 1 -- 172.21.15.94:0/4050904751 wait complete. 2024-09-10T23:35:58.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:35:58.019 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/f2d51a7c588daac4646352a178b76546.meta) 2024-09-10T23:36:02.223 INFO:teuthology.orchestra.run.smithi094.stdout:[GLOBAL] 2024-09-10T23:36:02.223 INFO:teuthology.orchestra.run.smithi094.stdout:version = 1 2024-09-10T23:36:02.223 INFO:teuthology.orchestra.run.smithi094.stdout:type = subvolume 2024-09-10T23:36:02.223 INFO:teuthology.orchestra.run.smithi094.stdout:path = /volumes/_nogroup/subvol98180 2024-09-10T23:36:02.223 INFO:teuthology.orchestra.run.smithi094.stdout:state = complete 2024-09-10T23:36:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:02.226 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvol98180 alice 2024-09-10T23:36:02.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.378+0000 7fe1bf400640 1 -- 172.21.15.94:0/1937446963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe19c0051d0 msgr2=0x7fe19c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.378+0000 7fe1bf400640 1 --2- 172.21.15.94:0/1937446963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe19c0051d0 0x7fe19c00b9d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80d0130 tx=0x7fe1b001cfe0 comp rx=0 tx=0).stop 2024-09-10T23:36:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.378+0000 7fe1bd400640 1 -- 172.21.15.94:0/1937446963 reap_dead start 2024-09-10T23:36:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.378+0000 7fe1bf400640 1 -- 172.21.15.94:0/1937446963 shutdown_connections 2024-09-10T23:36:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.378+0000 7fe1bf400640 1 -- 172.21.15.94:0/1937446963 >> 172.21.15.94:0/1937446963 conn(0x7fe1b806cdf0 msgr2=0x7fe1b806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.378+0000 7fe1bf400640 1 -- 172.21.15.94:0/1937446963 shutdown_connections 2024-09-10T23:36:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.378+0000 7fe1bf400640 1 -- 172.21.15.94:0/1937446963 wait complete. 2024-09-10T23:36:02.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bf400640 1 Processor -- start 2024-09-10T23:36:02.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bf400640 1 -- start start 2024-09-10T23:36:02.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bf400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe19c0051d0 0x7fe1b8150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bf400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1b8151200 0x7fe1b80c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1b80c8790 0x7fe1b813a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bf400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b8072bd0 con 0x7fe1b80c8790 2024-09-10T23:36:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bf400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b80d06b0 con 0x7fe19c0051d0 2024-09-10T23:36:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bf400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b8072d50 con 0x7fe1b8151200 2024-09-10T23:36:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bd400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1b80c8790 0x7fe1b813a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bd400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1b80c8790 0x7fe1b813a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36202/0 (socket says 172.21.15.94:36202) 2024-09-10T23:36:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bd400640 1 -- 172.21.15.94:0/4109218154 learned_addr learned my addr 172.21.15.94:0/4109218154 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bca00640 1 --2- 172.21.15.94:0/4109218154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe19c0051d0 0x7fe1b8150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1afe00640 1 --2- 172.21.15.94:0/4109218154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1b8151200 0x7fe1b80c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:02.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bd400640 1 -- 172.21.15.94:0/4109218154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1b8151200 msgr2=0x7fe1b80c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:02.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bd400640 1 --2- 172.21.15.94:0/4109218154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1b8151200 0x7fe1b80c8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:02.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bd400640 1 -- 172.21.15.94:0/4109218154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe19c0051d0 msgr2=0x7fe1b8150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:02.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bd400640 1 --2- 172.21.15.94:0/4109218154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe19c0051d0 0x7fe1b8150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:02.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bd400640 1 -- 172.21.15.94:0/4109218154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b81f2280 con 0x7fe1b80c8790 2024-09-10T23:36:02.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bca00640 1 --2- 172.21.15.94:0/4109218154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe19c0051d0 0x7fe1b8150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:02.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bd400640 1 --2- 172.21.15.94:0/4109218154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1b80c8790 0x7fe1b813a060 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0098b10 tx=0x7fe1b00afb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:02.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1b5600640 1 -- 172.21.15.94:0/4109218154 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b00afb90 con 0x7fe1b80c8790 2024-09-10T23:36:02.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1b5600640 1 -- 172.21.15.94:0/4109218154 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1b00afd30 con 0x7fe1b80c8790 2024-09-10T23:36:02.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1b5600640 1 -- 172.21.15.94:0/4109218154 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0004a50 con 0x7fe1b80c8790 2024-09-10T23:36:02.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bf400640 1 -- 172.21.15.94:0/4109218154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b81f2520 con 0x7fe1b80c8790 2024-09-10T23:36:02.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bf400640 1 -- 172.21.15.94:0/4109218154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b81f2ae0 con 0x7fe1b80c8790 2024-09-10T23:36:02.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.382+0000 7fe1bf400640 1 -- 172.21.15.94:0/4109218154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b80b3140 con 0x7fe1b80c8790 2024-09-10T23:36:02.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.386+0000 7fe1b5600640 1 -- 172.21.15.94:0/4109218154 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 10) ==== 96598+0+0 (secure 0 0 0) 0x7fe1b0004bf0 con 0x7fe1b80c8790 2024-09-10T23:36:02.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.386+0000 7fe1b5600640 1 --2- 172.21.15.94:0/4109218154 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe194072850 0x7fe194074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:02.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.386+0000 7fe1b5600640 1 -- 172.21.15.94:0/4109218154 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fe1b0002040 con 0x7fe1b80c8790 2024-09-10T23:36:02.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.386+0000 7fe1bca00640 1 --2- 172.21.15.94:0/4109218154 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe194072850 0x7fe194074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:02.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.386+0000 7fe1bca00640 1 --2- 172.21.15.94:0/4109218154 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe194072850 0x7fe194074d10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8001870 tx=0x7fe1a8000f60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:02.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.390+0000 7fe1b5600640 1 -- 172.21.15.94:0/4109218154 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe1b0016cf0 con 0x7fe1b80c8790 2024-09-10T23:36:02.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.578+0000 7fe1bf400640 1 -- 172.21.15.94:0/4109218154 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvol98180", "auth_id": "alice", "target": ["mon-mgr", ""]}) -- 0x7fe1b80b46c0 con 0x7fe194072850 2024-09-10T23:36:02.594 INFO:tasks.ceph.mgr.x.smithi181.stderr:Creating meta for ID alice with tenant None 2024-09-10T23:36:02.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.626+0000 7fe1b5600640 1 -- 172.21.15.94:0/4109218154 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe1b80b46c0 con 0x7fe194072850 2024-09-10T23:36:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.630+0000 7fe1bf400640 1 -- 172.21.15.94:0/4109218154 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe194072850 msgr2=0x7fe194074d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.630+0000 7fe1bf400640 1 --2- 172.21.15.94:0/4109218154 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe194072850 0x7fe194074d10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8001870 tx=0x7fe1a8000f60 comp rx=0 tx=0).stop 2024-09-10T23:36:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.630+0000 7fe1bf400640 1 -- 172.21.15.94:0/4109218154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1b80c8790 msgr2=0x7fe1b813a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.630+0000 7fe1bf400640 1 --2- 172.21.15.94:0/4109218154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1b80c8790 0x7fe1b813a060 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0098b10 tx=0x7fe1b00afb20 comp rx=0 tx=0).stop 2024-09-10T23:36:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.630+0000 7fe1bf400640 1 -- 172.21.15.94:0/4109218154 shutdown_connections 2024-09-10T23:36:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.630+0000 7fe1bf400640 1 --2- 172.21.15.94:0/4109218154 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe194072850 0x7fe194074d10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.630+0000 7fe1bf400640 1 --2- 172.21.15.94:0/4109218154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1b80c8790 0x7fe1b813a060 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.630+0000 7fe1bf400640 1 --2- 172.21.15.94:0/4109218154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1b8151200 0x7fe1b80c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.630+0000 7fe1bf400640 1 --2- 172.21.15.94:0/4109218154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe19c0051d0 0x7fe1b8150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.630+0000 7fe1bf400640 1 -- 172.21.15.94:0/4109218154 >> 172.21.15.94:0/4109218154 conn(0x7fe1b806cdf0 msgr2=0x7fe1b814f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.630+0000 7fe1bf400640 1 -- 172.21.15.94:0/4109218154 shutdown_connections 2024-09-10T23:36:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.630+0000 7fe1bf400640 1 -- 172.21.15.94:0/4109218154 wait complete. 2024-09-10T23:36:02.651 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2024-09-10T23:36:02.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f326ee00640 1 -- 172.21.15.94:0/1602951251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32680c64f0 msgr2=0x7f32680cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:02.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f326ee00640 1 --2- 172.21.15.94:0/1602951251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32680c64f0 0x7f32680cf280 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3258009840 tx=0x7f325801c820 comp rx=0 tx=0).stop 2024-09-10T23:36:02.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f326ee00640 1 -- 172.21.15.94:0/1602951251 shutdown_connections 2024-09-10T23:36:02.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f326ee00640 1 --2- 172.21.15.94:0/1602951251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32680c64f0 0x7f32680cf280 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:02.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f326ee00640 1 -- 172.21.15.94:0/1602951251 >> 172.21.15.94:0/1602951251 conn(0x7f32680cfaf0 msgr2=0x7f32680cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:02.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f326ee00640 1 -- 172.21.15.94:0/1602951251 shutdown_connections 2024-09-10T23:36:02.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f326ee00640 1 -- 172.21.15.94:0/1602951251 wait complete. 2024-09-10T23:36:02.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f326ee00640 1 Processor -- start 2024-09-10T23:36:02.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f326ee00640 1 -- start start 2024-09-10T23:36:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f326ee00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f326806e7c0 0x7f32680aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f326ee00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32680c64f0 0x7f32680c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f326ee00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f326815b6a0 0x7f32680c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f326ee00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3268169590 con 0x7f32680c64f0 2024-09-10T23:36:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f326ee00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3268169410 con 0x7f326815b6a0 2024-09-10T23:36:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f326ee00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3268169710 con 0x7f326806e7c0 2024-09-10T23:36:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f325f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32680c64f0 0x7f32680c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f325f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32680c64f0 0x7f32680c7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36224/0 (socket says 172.21.15.94:36224) 2024-09-10T23:36:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f325f400640 1 -- 172.21.15.94:0/1366049100 learned_addr learned my addr 172.21.15.94:0/1366049100 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f3267e00640 1 --2- 172.21.15.94:0/1366049100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f326806e7c0 0x7f32680aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f326ce00640 1 --2- 172.21.15.94:0/1366049100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f326815b6a0 0x7f32680c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f325f400640 1 -- 172.21.15.94:0/1366049100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f326806e7c0 msgr2=0x7f32680aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f325f400640 1 --2- 172.21.15.94:0/1366049100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f326806e7c0 0x7f32680aef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f325f400640 1 -- 172.21.15.94:0/1366049100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f326815b6a0 msgr2=0x7f32680c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f325f400640 1 --2- 172.21.15.94:0/1366049100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f326815b6a0 0x7f32680c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f325f400640 1 -- 172.21.15.94:0/1366049100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3268155c30 con 0x7f32680c64f0 2024-09-10T23:36:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f325f400640 1 --2- 172.21.15.94:0/1366049100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32680c64f0 0x7f32680c7410 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f325000c480 tx=0x7f325000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.794+0000 7f3264c00640 1 -- 172.21.15.94:0/1366049100 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3250016020 con 0x7f32680c64f0 2024-09-10T23:36:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.798+0000 7f3264c00640 1 -- 172.21.15.94:0/1366049100 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3250012940 con 0x7f32680c64f0 2024-09-10T23:36:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.798+0000 7f3264c00640 1 -- 172.21.15.94:0/1366049100 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3250012d90 con 0x7f32680c64f0 2024-09-10T23:36:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.798+0000 7f326ee00640 1 -- 172.21.15.94:0/1366049100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32680ad980 con 0x7f32680c64f0 2024-09-10T23:36:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.798+0000 7f326ee00640 1 -- 172.21.15.94:0/1366049100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32681fbbd0 con 0x7f32680c64f0 2024-09-10T23:36:02.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.798+0000 7f326ee00640 1 -- 172.21.15.94:0/1366049100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32680d0850 con 0x7f32680c64f0 2024-09-10T23:36:02.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.798+0000 7f3264c00640 1 -- 172.21.15.94:0/1366049100 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 10) ==== 96598+0+0 (secure 0 0 0) 0x7f325001a3b0 con 0x7f32680c64f0 2024-09-10T23:36:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.798+0000 7f3264c00640 1 --2- 172.21.15.94:0/1366049100 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3244072820 0x7f3244074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.798+0000 7f3264c00640 1 -- 172.21.15.94:0/1366049100 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f3250093690 con 0x7f32680c64f0 2024-09-10T23:36:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.798+0000 7f3267e00640 1 --2- 172.21.15.94:0/1366049100 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3244072820 0x7f3244074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.802+0000 7f3267e00640 1 --2- 172.21.15.94:0/1366049100 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3244072820 0x7f3244074ce0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f32580af000 tx=0x7f325801b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:02.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:02.802+0000 7f3264c00640 1 -- 172.21.15.94:0/1366049100 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3250020030 con 0x7f32680c64f0 2024-09-10T23:36:03.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.026+0000 7f326ee00640 1 -- 172.21.15.94:0/1366049100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) -- 0x7f32680adb50 con 0x7f32680c64f0 2024-09-10T23:36:03.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.026+0000 7f3264c00640 1 -- 172.21.15.94:0/1366049100 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v31) ==== 107+0+280 (secure 0 0 0) 0x7f3268155c30 con 0x7f32680c64f0 2024-09-10T23:36:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:03.028 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:36:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.alice", 2024-09-10T23:36:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDi1+BmAfZDJBAAOx1kuV3ZnOhMI/UiYmnyGQ==", 2024-09-10T23:36:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol98180", 2024-09-10T23:36:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow r", 2024-09-10T23:36:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-10T23:36:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:03.028 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:36:03.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.026+0000 7f326ee00640 1 -- 172.21.15.94:0/1366049100 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3244072820 msgr2=0x7f3244074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:03.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.026+0000 7f326ee00640 1 --2- 172.21.15.94:0/1366049100 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3244072820 0x7f3244074ce0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f32580af000 tx=0x7f325801b060 comp rx=0 tx=0).stop 2024-09-10T23:36:03.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.026+0000 7f326ee00640 1 -- 172.21.15.94:0/1366049100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32680c64f0 msgr2=0x7f32680c7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:03.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.026+0000 7f326ee00640 1 --2- 172.21.15.94:0/1366049100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32680c64f0 0x7f32680c7410 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f325000c480 tx=0x7f325000c940 comp rx=0 tx=0).stop 2024-09-10T23:36:03.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.026+0000 7f326ee00640 1 -- 172.21.15.94:0/1366049100 shutdown_connections 2024-09-10T23:36:03.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.026+0000 7f326ee00640 1 --2- 172.21.15.94:0/1366049100 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3244072820 0x7f3244074ce0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.026+0000 7f326ee00640 1 --2- 172.21.15.94:0/1366049100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f326815b6a0 0x7f32680c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.026+0000 7f326ee00640 1 --2- 172.21.15.94:0/1366049100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32680c64f0 0x7f32680c7410 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.026+0000 7f326ee00640 1 --2- 172.21.15.94:0/1366049100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f326806e7c0 0x7f32680aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.026+0000 7f326ee00640 1 -- 172.21.15.94:0/1366049100 >> 172.21.15.94:0/1366049100 conn(0x7f32680cfaf0 msgr2=0x7f32680b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:03.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.026+0000 7f326ee00640 1 -- 172.21.15.94:0/1366049100 shutdown_connections 2024-09-10T23:36:03.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.026+0000 7f326ee00640 1 -- 172.21.15.94:0/1366049100 wait complete. 2024-09-10T23:36:03.049 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol98180 2024-09-10T23:36:03.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.190+0000 7f1fe6a00640 1 -- 172.21.15.94:0/1593410425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe00b2ac0 msgr2=0x7f1fe00b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:03.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.190+0000 7f1fe6a00640 1 --2- 172.21.15.94:0/1593410425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe00b2ac0 0x7f1fe00b16b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0072ee0 tx=0x7f1fcc01c890 comp rx=0 tx=0).stop 2024-09-10T23:36:03.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.190+0000 7f1fe6a00640 1 -- 172.21.15.94:0/1593410425 shutdown_connections 2024-09-10T23:36:03.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.190+0000 7f1fe6a00640 1 --2- 172.21.15.94:0/1593410425 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fe00b1bf0 0x7f1fe00cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.190+0000 7f1fe6a00640 1 --2- 172.21.15.94:0/1593410425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe00b2ac0 0x7f1fe00b16b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.190+0000 7f1fe6a00640 1 --2- 172.21.15.94:0/1593410425 >> v2:172.21.15.181:3300/0 conn(0x7f1fe015f140 0x7f1fe00b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.190+0000 7f1fe6a00640 1 -- 172.21.15.94:0/1593410425 >> 172.21.15.94:0/1593410425 conn(0x7f1fe00d1b00 msgr2=0x7f1fe00d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:03.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.190+0000 7f1fe6a00640 1 -- 172.21.15.94:0/1593410425 shutdown_connections 2024-09-10T23:36:03.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.190+0000 7f1fe6a00640 1 -- 172.21.15.94:0/1593410425 wait complete. 2024-09-10T23:36:03.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe6a00640 1 Processor -- start 2024-09-10T23:36:03.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe6a00640 1 -- start start 2024-09-10T23:36:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe00b1bf0 0x7f1fe014db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fe00b2ac0 0x7f1fe00c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:03.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe6a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fe015f140 0x7f1fe014d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe6a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe00af4c0 con 0x7f1fe00b1bf0 2024-09-10T23:36:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe6a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe00af340 con 0x7f1fe00b2ac0 2024-09-10T23:36:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe6a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe00af640 con 0x7f1fe015f140 2024-09-10T23:36:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe00b1bf0 0x7f1fe014db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe00b1bf0 0x7f1fe014db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36230/0 (socket says 172.21.15.94:36230) 2024-09-10T23:36:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe5600640 1 -- 172.21.15.94:0/2227837977 learned_addr learned my addr 172.21.15.94:0/2227837977 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe4c00640 1 --2- 172.21.15.94:0/2227837977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fe00b2ac0 0x7f1fe00c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe6000640 1 --2- 172.21.15.94:0/2227837977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fe015f140 0x7f1fe014d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe5600640 1 -- 172.21.15.94:0/2227837977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fe015f140 msgr2=0x7f1fe014d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe5600640 1 --2- 172.21.15.94:0/2227837977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fe015f140 0x7f1fe014d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe5600640 1 -- 172.21.15.94:0/2227837977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fe00b2ac0 msgr2=0x7f1fe00c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe5600640 1 --2- 172.21.15.94:0/2227837977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fe00b2ac0 0x7f1fe00c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe5600640 1 -- 172.21.15.94:0/2227837977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe0152110 con 0x7f1fe00b1bf0 2024-09-10T23:36:03.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe5600640 1 --2- 172.21.15.94:0/2227837977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe00b1bf0 0x7f1fe014db10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400d950 tx=0x7f1fd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:03.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fdd600640 1 -- 172.21.15.94:0/2227837977 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4017070 con 0x7f1fe00b1bf0 2024-09-10T23:36:03.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fdd600640 1 -- 172.21.15.94:0/2227837977 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fd4009d20 con 0x7f1fe00b1bf0 2024-09-10T23:36:03.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe6a00640 1 -- 172.21.15.94:0/2227837977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe01523f0 con 0x7f1fe00b1bf0 2024-09-10T23:36:03.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fdd600640 1 -- 172.21.15.94:0/2227837977 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd40106c0 con 0x7f1fe00b1bf0 2024-09-10T23:36:03.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.194+0000 7f1fe6a00640 1 -- 172.21.15.94:0/2227837977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe0152cb0 con 0x7f1fe00b1bf0 2024-09-10T23:36:03.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.198+0000 7f1fe6a00640 1 -- 172.21.15.94:0/2227837977 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe00cfa00 con 0x7f1fe00b1bf0 2024-09-10T23:36:03.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.198+0000 7f1fdd600640 1 -- 172.21.15.94:0/2227837977 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 10) ==== 96598+0+0 (secure 0 0 0) 0x7f1fd4010860 con 0x7f1fe00b1bf0 2024-09-10T23:36:03.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.198+0000 7f1fdd600640 1 --2- 172.21.15.94:0/2227837977 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1fc0072820 0x7f1fc0074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:03.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.198+0000 7f1fdd600640 1 -- 172.21.15.94:0/2227837977 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f1fd4093c30 con 0x7f1fe00b1bf0 2024-09-10T23:36:03.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.198+0000 7f1fe4c00640 1 --2- 172.21.15.94:0/2227837977 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1fc0072820 0x7f1fc0074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:03.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.198+0000 7f1fe4c00640 1 --2- 172.21.15.94:0/2227837977 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1fc0072820 0x7f1fc0074ce0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0af000 tx=0x7f1fcc01b0d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:03.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.202+0000 7f1fdd600640 1 -- 172.21.15.94:0/2227837977 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1fd405e7c0 con 0x7f1fe00b1bf0 2024-09-10T23:36:03.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.394+0000 7f1fe6a00640 1 -- 172.21.15.94:0/2227837977 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol98180", "target": ["mon-mgr", ""]}) -- 0x7f1fe0154d90 con 0x7f1fc0072820 2024-09-10T23:36:03.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.410+0000 7f1fdd600640 1 -- 172.21.15.94:0/2227837977 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1fe0154d90 con 0x7f1fc0072820 2024-09-10T23:36:03.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.414+0000 7f1fe6a00640 1 -- 172.21.15.94:0/2227837977 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1fc0072820 msgr2=0x7f1fc0074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:03.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.414+0000 7f1fe6a00640 1 --2- 172.21.15.94:0/2227837977 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1fc0072820 0x7f1fc0074ce0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0af000 tx=0x7f1fcc01b0d0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.414+0000 7f1fe6a00640 1 -- 172.21.15.94:0/2227837977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe00b1bf0 msgr2=0x7f1fe014db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:03.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.414+0000 7f1fe6a00640 1 --2- 172.21.15.94:0/2227837977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe00b1bf0 0x7f1fe014db10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400d950 tx=0x7f1fd400de10 comp rx=0 tx=0).stop 2024-09-10T23:36:03.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.414+0000 7f1fe6a00640 1 -- 172.21.15.94:0/2227837977 shutdown_connections 2024-09-10T23:36:03.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.414+0000 7f1fe6a00640 1 --2- 172.21.15.94:0/2227837977 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1fc0072820 0x7f1fc0074ce0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.414+0000 7f1fe6a00640 1 --2- 172.21.15.94:0/2227837977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1fe00b2ac0 0x7f1fe00c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.414+0000 7f1fe6a00640 1 --2- 172.21.15.94:0/2227837977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1fe015f140 0x7f1fe014d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.414+0000 7f1fe6a00640 1 --2- 172.21.15.94:0/2227837977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fe00b1bf0 0x7f1fe014db10 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.414+0000 7f1fe6a00640 1 -- 172.21.15.94:0/2227837977 >> 172.21.15.94:0/2227837977 conn(0x7f1fe00d1b00 msgr2=0x7f1fe00d1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:03.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.414+0000 7f1fe6a00640 1 -- 172.21.15.94:0/2227837977 shutdown_connections 2024-09-10T23:36:03.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.414+0000 7f1fe6a00640 1 -- 172.21.15.94:0/2227837977 wait complete. 2024-09-10T23:36:03.418 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:36:03.418+0000 7fb90be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:03.418 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:36:03.418+0000 7fb90be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:03.418 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:36:03.418+0000 7fb90be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:03.418 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:36:03.418+0000 7fb90be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:03.418 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:36:03.418+0000 7fb90be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:03.436 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:36:03.434+0000 7fb90aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:03.436 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:36:03.434+0000 7fb90aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:03.436 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:36:03.434+0000 7fb90aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:03.436 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:36:03.434+0000 7fb90aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:03.436 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:36:03.434+0000 7fb90aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:03.437 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol74911 2024-09-10T23:36:03.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.582+0000 7f0cb6200640 1 -- 172.21.15.94:0/4028719183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c94004d50 msgr2=0x7f0c940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:03.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.582+0000 7f0cb6200640 1 --2- 172.21.15.94:0/4028719183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c94004d50 0x7f0c940051d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800c640 tx=0x7f0ca801d320 comp rx=0 tx=0).stop 2024-09-10T23:36:03.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.582+0000 7f0cafe00640 1 -- 172.21.15.94:0/4028719183 reap_dead start 2024-09-10T23:36:03.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.582+0000 7f0cb6200640 1 -- 172.21.15.94:0/4028719183 shutdown_connections 2024-09-10T23:36:03.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.582+0000 7f0cb6200640 1 -- 172.21.15.94:0/4028719183 >> 172.21.15.94:0/4028719183 conn(0x7f0cb00cfaf0 msgr2=0x7f0cb00cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:03.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.582+0000 7f0cb6200640 1 -- 172.21.15.94:0/4028719183 shutdown_connections 2024-09-10T23:36:03.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.582+0000 7f0cb6200640 1 -- 172.21.15.94:0/4028719183 wait complete. 2024-09-10T23:36:03.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0cb6200640 1 Processor -- start 2024-09-10T23:36:03.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0cb6200640 1 -- start start 2024-09-10T23:36:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0cb6200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c94004d50 0x7f0cb0158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0cb6200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb00c99b0 0x7f0cb014ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0cb6200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cb00c5770 0x7f0cb00c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0cb6200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb015b6a0 con 0x7f0cb00c99b0 2024-09-10T23:36:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0cb6200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb0070940 con 0x7f0c94004d50 2024-09-10T23:36:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0cb6200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb015b820 con 0x7f0cb00c5770 2024-09-10T23:36:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0ca6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb00c99b0 0x7f0cb014ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0ca6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb00c99b0 0x7f0cb014ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36256/0 (socket says 172.21.15.94:36256) 2024-09-10T23:36:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0ca6a00640 1 -- 172.21.15.94:0/2227391651 learned_addr learned my addr 172.21.15.94:0/2227391651 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0caf400640 1 --2- 172.21.15.94:0/2227391651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c94004d50 0x7f0cb0158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0cafe00640 1 --2- 172.21.15.94:0/2227391651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cb00c5770 0x7f0cb00c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0ca6a00640 1 -- 172.21.15.94:0/2227391651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cb00c5770 msgr2=0x7f0cb00c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0ca6a00640 1 --2- 172.21.15.94:0/2227391651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cb00c5770 0x7f0cb00c5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0ca6a00640 1 -- 172.21.15.94:0/2227391651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c94004d50 msgr2=0x7f0cb0158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:03.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0ca6a00640 1 --2- 172.21.15.94:0/2227391651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c94004d50 0x7f0cb0158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0ca6a00640 1 -- 172.21.15.94:0/2227391651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb006f300 con 0x7f0cb00c99b0 2024-09-10T23:36:03.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0cafe00640 1 --2- 172.21.15.94:0/2227391651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cb00c5770 0x7f0cb00c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:03.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0ca6a00640 1 --2- 172.21.15.94:0/2227391651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb00c99b0 0x7f0cb014ed20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800ea70 tx=0x7f0c9800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:03.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0ca7e00640 1 -- 172.21.15.94:0/2227391651 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800dc30 con 0x7f0cb00c99b0 2024-09-10T23:36:03.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0ca7e00640 1 -- 172.21.15.94:0/2227391651 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c9800ddd0 con 0x7f0cb00c99b0 2024-09-10T23:36:03.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0cb6200640 1 -- 172.21.15.94:0/2227391651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb006f740 con 0x7f0cb00c99b0 2024-09-10T23:36:03.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0ca7e00640 1 -- 172.21.15.94:0/2227391651 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c980042b0 con 0x7f0cb00c99b0 2024-09-10T23:36:03.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.586+0000 7f0cb6200640 1 -- 172.21.15.94:0/2227391651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb01f9130 con 0x7f0cb00c99b0 2024-09-10T23:36:03.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.590+0000 7f0cb6200640 1 -- 172.21.15.94:0/2227391651 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb00ce7d0 con 0x7f0cb00c99b0 2024-09-10T23:36:03.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.590+0000 7f0ca7e00640 1 -- 172.21.15.94:0/2227391651 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 10) ==== 96598+0+0 (secure 0 0 0) 0x7f0c9800b040 con 0x7f0cb00c99b0 2024-09-10T23:36:03.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.590+0000 7f0ca7e00640 1 --2- 172.21.15.94:0/2227391651 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0c8c072820 0x7f0c8c074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:03.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.590+0000 7f0ca7e00640 1 -- 172.21.15.94:0/2227391651 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f0c98094110 con 0x7f0cb00c99b0 2024-09-10T23:36:03.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.590+0000 7f0caf400640 1 --2- 172.21.15.94:0/2227391651 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0c8c072820 0x7f0c8c074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:03.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.590+0000 7f0caf400640 1 --2- 172.21.15.94:0/2227391651 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0c8c072820 0x7f0c8c074ce0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009ce0 tx=0x7f0ca00091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:03.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.594+0000 7f0ca7e00640 1 -- 172.21.15.94:0/2227391651 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0c9805f2e0 con 0x7f0cb00c99b0 2024-09-10T23:36:03.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.786+0000 7f0cb6200640 1 -- 172.21.15.94:0/2227391651 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol74911", "target": ["mon-mgr", ""]}) -- 0x7f0cb00d0850 con 0x7f0c8c072820 2024-09-10T23:36:03.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.806+0000 7f0ca7e00640 1 -- 172.21.15.94:0/2227391651 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0cb00d0850 con 0x7f0c8c072820 2024-09-10T23:36:03.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.810+0000 7f0cb6200640 1 -- 172.21.15.94:0/2227391651 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0c8c072820 msgr2=0x7f0c8c074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:03.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.810+0000 7f0cb6200640 1 --2- 172.21.15.94:0/2227391651 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0c8c072820 0x7f0c8c074ce0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009ce0 tx=0x7f0ca00091e0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.810+0000 7f0cb6200640 1 -- 172.21.15.94:0/2227391651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb00c99b0 msgr2=0x7f0cb014ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:03.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.810+0000 7f0cb6200640 1 --2- 172.21.15.94:0/2227391651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb00c99b0 0x7f0cb014ed20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800ea70 tx=0x7f0c9800ee70 comp rx=0 tx=0).stop 2024-09-10T23:36:03.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.810+0000 7f0cb6200640 1 -- 172.21.15.94:0/2227391651 shutdown_connections 2024-09-10T23:36:03.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.810+0000 7f0cb6200640 1 --2- 172.21.15.94:0/2227391651 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0c8c072820 0x7f0c8c074ce0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.810+0000 7f0cb6200640 1 --2- 172.21.15.94:0/2227391651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0cb00c5770 0x7f0cb00c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.810+0000 7f0cb6200640 1 --2- 172.21.15.94:0/2227391651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb00c99b0 0x7f0cb014ed20 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.810+0000 7f0cb6200640 1 --2- 172.21.15.94:0/2227391651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c94004d50 0x7f0cb0158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:03.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.810+0000 7f0cb6200640 1 -- 172.21.15.94:0/2227391651 >> 172.21.15.94:0/2227391651 conn(0x7f0cb00cfaf0 msgr2=0x7f0cb01ea790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:03.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.810+0000 7f0cb6200640 1 -- 172.21.15.94:0/2227391651 shutdown_connections 2024-09-10T23:36:03.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03.810+0000 7f0cb6200640 1 -- 172.21.15.94:0/2227391651 wait complete. 2024-09-10T23:36:03.835 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T23:36:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T23:36:03.892 INFO:teuthology.orchestra.run.smithi094.stdout:2 2024-09-10T23:36:03.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:36:03.898 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:36:03.899 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:36:03.989 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:36:04.069 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:36:04.069 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:36:04.069 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:36:04.069 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:04.078 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:36:04.078 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:04.133 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:36:04.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.314+0000 7f6f79c00640 1 -- 172.21.15.94:0/889511300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58003a10 msgr2=0x7f6f58003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:04.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.314+0000 7f6f79c00640 1 --2- 172.21.15.94:0/889511300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58003a10 0x7f6f58003e30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009b60 tx=0x7f6f6401d270 comp rx=0 tx=0).stop 2024-09-10T23:36:04.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.314+0000 7f6f73400640 1 -- 172.21.15.94:0/889511300 reap_dead start 2024-09-10T23:36:04.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.314+0000 7f6f79c00640 1 -- 172.21.15.94:0/889511300 shutdown_connections 2024-09-10T23:36:04.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.314+0000 7f6f79c00640 1 -- 172.21.15.94:0/889511300 >> 172.21.15.94:0/889511300 conn(0x7f6f740cfaf0 msgr2=0x7f6f740cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.314+0000 7f6f79c00640 1 -- 172.21.15.94:0/889511300 shutdown_connections 2024-09-10T23:36:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.314+0000 7f6f79c00640 1 -- 172.21.15.94:0/889511300 wait complete. 2024-09-10T23:36:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.314+0000 7f6f79c00640 1 Processor -- start 2024-09-10T23:36:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.314+0000 7f6f79c00640 1 -- start start 2024-09-10T23:36:04.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f79c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58003a10 0x7f6f741f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:04.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f79c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f740b2ca0 0x7f6f74142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:04.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f79c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f740b3120 0x7f6f741431a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:04.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f79c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74165ea0 con 0x7f6f58003a10 2024-09-10T23:36:04.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f79c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6f74165d20 con 0x7f6f740b3120 2024-09-10T23:36:04.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f79c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6f74166020 con 0x7f6f740b2ca0 2024-09-10T23:36:04.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f72a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58003a10 0x7f6f741f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:04.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f72a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58003a10 0x7f6f741f9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59234/0 (socket says 172.21.15.94:59234) 2024-09-10T23:36:04.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f72a00640 1 -- 172.21.15.94:0/1405734749 learned_addr learned my addr 172.21.15.94:0/1405734749 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:04.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f73400640 1 --2- 172.21.15.94:0/1405734749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f740b3120 0x7f6f741431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:04.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f6a000640 1 --2- 172.21.15.94:0/1405734749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f740b2ca0 0x7f6f74142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:04.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f72a00640 1 -- 172.21.15.94:0/1405734749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f740b2ca0 msgr2=0x7f6f74142c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:04.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f72a00640 1 --2- 172.21.15.94:0/1405734749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f740b2ca0 0x7f6f74142c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:04.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f72a00640 1 -- 172.21.15.94:0/1405734749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f740b3120 msgr2=0x7f6f741431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:04.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f72a00640 1 --2- 172.21.15.94:0/1405734749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f740b3120 0x7f6f741431a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:04.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f72a00640 1 -- 172.21.15.94:0/1405734749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f741f6d80 con 0x7f6f58003a10 2024-09-10T23:36:04.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f73400640 1 --2- 172.21.15.94:0/1405734749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f740b3120 0x7f6f741431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:36:04.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f72a00640 1 --2- 172.21.15.94:0/1405734749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58003a10 0x7f6f741f9db0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6f640af040 tx=0x7f6f64005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:04.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f6b400640 1 -- 172.21.15.94:0/1405734749 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f640ba070 con 0x7f6f58003a10 2024-09-10T23:36:04.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f6b400640 1 -- 172.21.15.94:0/1405734749 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f640ac8a0 con 0x7f6f58003a10 2024-09-10T23:36:04.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f79c00640 1 -- 172.21.15.94:0/1405734749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f741fa2f0 con 0x7f6f58003a10 2024-09-10T23:36:04.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f6b400640 1 -- 172.21.15.94:0/1405734749 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f640b1040 con 0x7f6f58003a10 2024-09-10T23:36:04.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f79c00640 1 -- 172.21.15.94:0/1405734749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f741f64e0 con 0x7f6f58003a10 2024-09-10T23:36:04.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.318+0000 7f6f79c00640 1 -- 172.21.15.94:0/1405734749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f7415b190 con 0x7f6f58003a10 2024-09-10T23:36:04.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.322+0000 7f6f6b400640 1 -- 172.21.15.94:0/1405734749 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 10) ==== 96598+0+0 (secure 0 0 0) 0x7f6f640046d0 con 0x7f6f58003a10 2024-09-10T23:36:04.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.322+0000 7f6f6b400640 1 --2- 172.21.15.94:0/1405734749 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f50072820 0x7f6f50074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:04.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.322+0000 7f6f6b400640 1 -- 172.21.15.94:0/1405734749 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f6f6412e610 con 0x7f6f58003a10 2024-09-10T23:36:04.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.322+0000 7f6f6a000640 1 --2- 172.21.15.94:0/1405734749 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f50072820 0x7f6f50074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:04.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.322+0000 7f6f6a000640 1 --2- 172.21.15.94:0/1405734749 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f50072820 0x7f6f50074ce0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c001c10 tx=0x7f6f5c002080 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:04.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.326+0000 7f6f6b400640 1 -- 172.21.15.94:0/1405734749 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6f640c4020 con 0x7f6f58003a10 2024-09-10T23:36:04.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.502+0000 7f6f79c00640 1 -- 172.21.15.94:0/1405734749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6f740d0850 con 0x7f6f58003a10 2024-09-10T23:36:04.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.502+0000 7f6f6b400640 1 -- 172.21.15.94:0/1405734749 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) ==== 80+0+0 (secure 0 0 0) 0x7f6f741fa2f0 con 0x7f6f58003a10 2024-09-10T23:36:04.505 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:36:04.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.506+0000 7f6f79c00640 1 -- 172.21.15.94:0/1405734749 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f50072820 msgr2=0x7f6f50074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:04.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.506+0000 7f6f79c00640 1 --2- 172.21.15.94:0/1405734749 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f50072820 0x7f6f50074ce0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c001c10 tx=0x7f6f5c002080 comp rx=0 tx=0).stop 2024-09-10T23:36:04.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.506+0000 7f6f79c00640 1 -- 172.21.15.94:0/1405734749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58003a10 msgr2=0x7f6f741f9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:04.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.506+0000 7f6f79c00640 1 --2- 172.21.15.94:0/1405734749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58003a10 0x7f6f741f9db0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6f640af040 tx=0x7f6f64005620 comp rx=0 tx=0).stop 2024-09-10T23:36:04.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.506+0000 7f6f79c00640 1 -- 172.21.15.94:0/1405734749 shutdown_connections 2024-09-10T23:36:04.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.506+0000 7f6f79c00640 1 --2- 172.21.15.94:0/1405734749 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f50072820 0x7f6f50074ce0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:04.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.506+0000 7f6f79c00640 1 --2- 172.21.15.94:0/1405734749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f740b3120 0x7f6f741431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:04.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.506+0000 7f6f79c00640 1 --2- 172.21.15.94:0/1405734749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f740b2ca0 0x7f6f74142c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:04.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.506+0000 7f6f79c00640 1 --2- 172.21.15.94:0/1405734749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58003a10 0x7f6f741f9db0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:04.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.506+0000 7f6f79c00640 1 -- 172.21.15.94:0/1405734749 >> 172.21.15.94:0/1405734749 conn(0x7f6f740cfaf0 msgr2=0x7f6f74072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:04.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.506+0000 7f6f79c00640 1 -- 172.21.15.94:0/1405734749 shutdown_connections 2024-09-10T23:36:04.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.506+0000 7f6f79c00640 1 -- 172.21.15.94:0/1405734749 wait complete. 2024-09-10T23:36:04.526 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-09-10T23:36:04.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:04.527 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T23:36:04.544 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[40379]: fuse finished with error 0 and tester_r 0 2024-09-10T23:36:04.547 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:36:04.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:04.547 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T23:36:04.712 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:36:04.713 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:36:04.713 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:04.722 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:36:04.722 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:04.774 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:36:04.819 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:36:04.823 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:36:04.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.918+0000 7fa0f1400640 1 -- 172.21.15.94:0/2796634820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0ec06e7c0 msgr2=0x7fa0ec06ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.918+0000 7fa0f1400640 1 --2- 172.21.15.94:0/2796634820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0ec06e7c0 0x7fa0ec06ebf0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0cd580 tx=0x7fa0e401c800 comp rx=0 tx=0).stop 2024-09-10T23:36:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.918+0000 7fa0f1400640 1 -- 172.21.15.94:0/2796634820 shutdown_connections 2024-09-10T23:36:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.918+0000 7fa0f1400640 1 --2- 172.21.15.94:0/2796634820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0ec06e7c0 0x7fa0ec06ebf0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.918+0000 7fa0f1400640 1 --2- 172.21.15.94:0/2796634820 >> v2:172.21.15.181:3300/0 conn(0x7fa0ec0c64f0 0x7fa0ec0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.918+0000 7fa0f1400640 1 -- 172.21.15.94:0/2796634820 >> 172.21.15.94:0/2796634820 conn(0x7fa0ec0cfaf0 msgr2=0x7fa0ec0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.918+0000 7fa0f1400640 1 -- 172.21.15.94:0/2796634820 shutdown_connections 2024-09-10T23:36:04.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.918+0000 7fa0f1400640 1 -- 172.21.15.94:0/2796634820 wait complete. 2024-09-10T23:36:04.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.918+0000 7fa0f1400640 1 Processor -- start 2024-09-10T23:36:04.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.918+0000 7fa0f1400640 1 -- start start 2024-09-10T23:36:04.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0f1400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0ec06e7c0 0x7fa0ec14d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:04.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0f1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0ec0c64f0 0x7fa0ec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:04.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0f1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0ec0ce280 0x7fa0ec1529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:04.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0f1400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec15b530 con 0x7fa0ec0ce280 2024-09-10T23:36:04.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0f1400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec15b3b0 con 0x7fa0ec0c64f0 2024-09-10T23:36:04.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0f1400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa0ec15b6b0 con 0x7fa0ec06e7c0 2024-09-10T23:36:04.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0eb200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0ec0ce280 0x7fa0ec1529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:04.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0eb200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0ec0ce280 0x7fa0ec1529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59252/0 (socket says 172.21.15.94:59252) 2024-09-10T23:36:04.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0eb200640 1 -- 172.21.15.94:0/1967718257 learned_addr learned my addr 172.21.15.94:0/1967718257 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:04.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0ea800640 1 --2- 172.21.15.94:0/1967718257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0ec06e7c0 0x7fa0ec14d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:04.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0e1e00640 1 --2- 172.21.15.94:0/1967718257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0ec0c64f0 0x7fa0ec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:04.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0eb200640 1 -- 172.21.15.94:0/1967718257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0ec06e7c0 msgr2=0x7fa0ec14d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:04.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0eb200640 1 --2- 172.21.15.94:0/1967718257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0ec06e7c0 0x7fa0ec14d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:04.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0eb200640 1 -- 172.21.15.94:0/1967718257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0ec0c64f0 msgr2=0x7fa0ec0b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:04.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0eb200640 1 --2- 172.21.15.94:0/1967718257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0ec0c64f0 0x7fa0ec0b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:04.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0eb200640 1 -- 172.21.15.94:0/1967718257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec0b5640 con 0x7fa0ec0ce280 2024-09-10T23:36:04.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0ea800640 1 --2- 172.21.15.94:0/1967718257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0ec06e7c0 0x7fa0ec14d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:04.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0eb200640 1 --2- 172.21.15.94:0/1967718257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0ec0ce280 0x7fa0ec1529a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40b0040 tx=0x7fa0e40041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:04.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0e3400640 1 -- 172.21.15.94:0/1967718257 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40bc070 con 0x7fa0ec0ce280 2024-09-10T23:36:04.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0e3400640 1 -- 172.21.15.94:0/1967718257 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0e4003600 con 0x7fa0ec0ce280 2024-09-10T23:36:04.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0f1400640 1 -- 172.21.15.94:0/1967718257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ec1f7ba0 con 0x7fa0ec0ce280 2024-09-10T23:36:04.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0e3400640 1 -- 172.21.15.94:0/1967718257 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40b2040 con 0x7fa0ec0ce280 2024-09-10T23:36:04.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0f1400640 1 -- 172.21.15.94:0/1967718257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ec1f8160 con 0x7fa0ec0ce280 2024-09-10T23:36:04.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0f1400640 1 -- 172.21.15.94:0/1967718257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ec06ebf0 con 0x7fa0ec0ce280 2024-09-10T23:36:04.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.922+0000 7fa0e3400640 1 -- 172.21.15.94:0/1967718257 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 10) ==== 96598+0+0 (secure 0 0 0) 0x7fa0e40b5070 con 0x7fa0ec0ce280 2024-09-10T23:36:04.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.926+0000 7fa0e3400640 1 --2- 172.21.15.94:0/1967718257 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa0c8072820 0x7fa0c8074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:04.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.926+0000 7fa0e3400640 1 -- 172.21.15.94:0/1967718257 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa0e412ce60 con 0x7fa0ec0ce280 2024-09-10T23:36:04.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.926+0000 7fa0ea800640 1 --2- 172.21.15.94:0/1967718257 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa0c8072820 0x7fa0c8074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:04.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.926+0000 7fa0ea800640 1 --2- 172.21.15.94:0/1967718257 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa0c8072820 0x7fa0c8074ce0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc0012f0 tx=0x7fa0dc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:04.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:04.930+0000 7fa0e3400640 1 -- 172.21.15.94:0/1967718257 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa0e40163f0 con 0x7fa0ec0ce280 2024-09-10T23:36:05.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.106+0000 7fa0f1400640 1 -- 172.21.15.94:0/1967718257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa0ec069e20 con 0x7fa0ec0ce280 2024-09-10T23:36:05.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.106+0000 7fa0e3400640 1 -- 172.21.15.94:0/1967718257 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) ==== 80+0+0 (secure 0 0 0) 0x7fa0ec069e20 con 0x7fa0ec0ce280 2024-09-10T23:36:05.108 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:36:05.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.106+0000 7fa0f1400640 1 -- 172.21.15.94:0/1967718257 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa0c8072820 msgr2=0x7fa0c8074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:05.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.106+0000 7fa0f1400640 1 --2- 172.21.15.94:0/1967718257 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa0c8072820 0x7fa0c8074ce0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc0012f0 tx=0x7fa0dc009040 comp rx=0 tx=0).stop 2024-09-10T23:36:05.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.106+0000 7fa0f1400640 1 -- 172.21.15.94:0/1967718257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0ec0ce280 msgr2=0x7fa0ec1529a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:05.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.106+0000 7fa0f1400640 1 --2- 172.21.15.94:0/1967718257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0ec0ce280 0x7fa0ec1529a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40b0040 tx=0x7fa0e40041a0 comp rx=0 tx=0).stop 2024-09-10T23:36:05.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.106+0000 7fa0f1400640 1 -- 172.21.15.94:0/1967718257 shutdown_connections 2024-09-10T23:36:05.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.106+0000 7fa0f1400640 1 --2- 172.21.15.94:0/1967718257 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa0c8072820 0x7fa0c8074ce0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:05.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.106+0000 7fa0f1400640 1 --2- 172.21.15.94:0/1967718257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0ec0c64f0 0x7fa0ec0b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:05.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.106+0000 7fa0f1400640 1 --2- 172.21.15.94:0/1967718257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0ec0ce280 0x7fa0ec1529a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:05.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.106+0000 7fa0f1400640 1 --2- 172.21.15.94:0/1967718257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0ec06e7c0 0x7fa0ec14d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:05.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.106+0000 7fa0f1400640 1 -- 172.21.15.94:0/1967718257 >> 172.21.15.94:0/1967718257 conn(0x7fa0ec0cfaf0 msgr2=0x7fa0ec0b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:05.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.106+0000 7fa0f1400640 1 -- 172.21.15.94:0/1967718257 shutdown_connections 2024-09-10T23:36:05.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.106+0000 7fa0f1400640 1 -- 172.21.15.94:0/1967718257 wait complete. 2024-09-10T23:36:05.129 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi181.front.sepia.ceph.com... 2024-09-10T23:36:05.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:05.129 DEBUG:teuthology.orchestra.run.smithi181:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T23:36:05.148 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[34889]: fuse finished with error 0 and tester_r 0 2024-09-10T23:36:05.150 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:36:05.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:05.150 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T23:36:05.218 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:36:05.218 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:36:05.218 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:05.227 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:36:05.227 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:36:05.227 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:36:05.227 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:05.269 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:36:05.270 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:05.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.410+0000 7f0088a00640 1 -- 172.21.15.94:0/3761004086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f008415aa30 msgr2=0x7f008415ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:05.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.410+0000 7f0088a00640 1 --2- 172.21.15.94:0/3761004086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f008415aa30 0x7f008415ee20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0074009840 tx=0x7f007401c820 comp rx=0 tx=0).stop 2024-09-10T23:36:05.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.410+0000 7f0088a00640 1 -- 172.21.15.94:0/3761004086 shutdown_connections 2024-09-10T23:36:05.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.410+0000 7f0088a00640 1 --2- 172.21.15.94:0/3761004086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f008415aa30 0x7f008415ee20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:05.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.410+0000 7f0088a00640 1 -- 172.21.15.94:0/3761004086 >> 172.21.15.94:0/3761004086 conn(0x7f00840b2520 msgr2=0x7f00840b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:05.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.410+0000 7f0088a00640 1 -- 172.21.15.94:0/3761004086 shutdown_connections 2024-09-10T23:36:05.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.410+0000 7f0088a00640 1 -- 172.21.15.94:0/3761004086 wait complete. 2024-09-10T23:36:05.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0088a00640 1 Processor -- start 2024-09-10T23:36:05.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0088a00640 1 -- start start 2024-09-10T23:36:05.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0088a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00840cd360 0x7f008414d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:05.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0088a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00840cd7a0 0x7f008414daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:05.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0088a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f008415aa30 0x7f00841f8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:05.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0088a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00840afc60 con 0x7f00840cd360 2024-09-10T23:36:05.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0088a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f00840afae0 con 0x7f00840cd7a0 2024-09-10T23:36:05.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0088a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0084168da0 con 0x7f008415aa30 2024-09-10T23:36:05.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0083400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00840cd360 0x7f008414d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:05.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0083400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00840cd360 0x7f008414d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59256/0 (socket says 172.21.15.94:59256) 2024-09-10T23:36:05.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0083400640 1 -- 172.21.15.94:0/1814165465 learned_addr learned my addr 172.21.15.94:0/1814165465 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:05.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0082a00640 1 --2- 172.21.15.94:0/1814165465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00840cd7a0 0x7f008414daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:05.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0083e00640 1 --2- 172.21.15.94:0/1814165465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f008415aa30 0x7f00841f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:05.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0083400640 1 -- 172.21.15.94:0/1814165465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f008415aa30 msgr2=0x7f00841f8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:05.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0083400640 1 --2- 172.21.15.94:0/1814165465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f008415aa30 0x7f00841f8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:05.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0083400640 1 -- 172.21.15.94:0/1814165465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00840cd7a0 msgr2=0x7f008414daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:05.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0083400640 1 --2- 172.21.15.94:0/1814165465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00840cd7a0 0x7f008414daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:05.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0083400640 1 -- 172.21.15.94:0/1814165465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008414e030 con 0x7f00840cd360 2024-09-10T23:36:05.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0083e00640 1 --2- 172.21.15.94:0/1814165465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f008415aa30 0x7f00841f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:05.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0083400640 1 --2- 172.21.15.94:0/1814165465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00840cd360 0x7f008414d5b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0074009790 tx=0x7f00740a6570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:05.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f007b400640 1 -- 172.21.15.94:0/1814165465 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00740bb020 con 0x7f00840cd360 2024-09-10T23:36:05.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f007b400640 1 -- 172.21.15.94:0/1814165465 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0074004130 con 0x7f00840cd360 2024-09-10T23:36:05.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f007b400640 1 -- 172.21.15.94:0/1814165465 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074005030 con 0x7f00840cd360 2024-09-10T23:36:05.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0088a00640 1 -- 172.21.15.94:0/1814165465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00841f93a0 con 0x7f00840cd360 2024-09-10T23:36:05.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.414+0000 7f0088a00640 1 -- 172.21.15.94:0/1814165465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00840c4950 con 0x7f00840cd360 2024-09-10T23:36:05.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.418+0000 7f0088a00640 1 -- 172.21.15.94:0/1814165465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f008415ee20 con 0x7f00840cd360 2024-09-10T23:36:05.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.418+0000 7f007b400640 1 -- 172.21.15.94:0/1814165465 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 10) ==== 96598+0+0 (secure 0 0 0) 0x7f00740051d0 con 0x7f00840cd360 2024-09-10T23:36:05.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.418+0000 7f007b400640 1 --2- 172.21.15.94:0/1814165465 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0064072820 0x7f0064074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:05.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.418+0000 7f007b400640 1 -- 172.21.15.94:0/1814165465 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f007412dd00 con 0x7f00840cd360 2024-09-10T23:36:05.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.418+0000 7f0082a00640 1 --2- 172.21.15.94:0/1814165465 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0064072820 0x7f0064074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:05.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.418+0000 7f0082a00640 1 --2- 172.21.15.94:0/1814165465 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0064072820 0x7f0064074ce0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0084153f80 tx=0x7f0070009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:05.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.422+0000 7f007b400640 1 -- 172.21.15.94:0/1814165465 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0074016410 con 0x7f00840cd360 2024-09-10T23:36:05.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.618+0000 7f0088a00640 1 -- 172.21.15.94:0/1814165465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f008414d1d0 con 0x7f00840cd360 2024-09-10T23:36:05.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.622+0000 7f007b400640 1 -- 172.21.15.94:0/1814165465 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5243 (secure 0 0 0) 0x7f00740f8ee0 con 0x7f00840cd360 2024-09-10T23:36:05.622 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:05.623 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":35,"btime":"2024-09-10T23:35:51:877544+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5814,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3613711739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3613711739},{"type":"v1","addr":"172.21.15.94:6834","nonce":3613711739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":195107,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1388563408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1388563408},{"type":"v1","addr":"172.21.15.181:6836","nonce":1388563408}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":195110,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1882902083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1882902083},{"type":"v1","addr":"172.21.15.181:6837","nonce":1882902083}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:35:36.778166+0000","modified":"2024-09-10T23:35:39.784176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195113":{"gid":195113,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/2905667261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2905667261},{"type":"v1","addr":"172.21.15.94:6835","nonce":2905667261}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195113,"qdb_cluster":[195113]},"id":4}]} 2024-09-10T23:36:05.623 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 35 2024-09-10T23:36:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.622+0000 7f0088a00640 1 -- 172.21.15.94:0/1814165465 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0064072820 msgr2=0x7f0064074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.622+0000 7f0088a00640 1 --2- 172.21.15.94:0/1814165465 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0064072820 0x7f0064074ce0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0084153f80 tx=0x7f0070009040 comp rx=0 tx=0).stop 2024-09-10T23:36:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.622+0000 7f0088a00640 1 -- 172.21.15.94:0/1814165465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00840cd360 msgr2=0x7f008414d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.622+0000 7f0088a00640 1 --2- 172.21.15.94:0/1814165465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00840cd360 0x7f008414d5b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0074009790 tx=0x7f00740a6570 comp rx=0 tx=0).stop 2024-09-10T23:36:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.622+0000 7f0088a00640 1 -- 172.21.15.94:0/1814165465 shutdown_connections 2024-09-10T23:36:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.622+0000 7f0088a00640 1 --2- 172.21.15.94:0/1814165465 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0064072820 0x7f0064074ce0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.622+0000 7f0088a00640 1 --2- 172.21.15.94:0/1814165465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00840cd7a0 0x7f008414daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.622+0000 7f0088a00640 1 --2- 172.21.15.94:0/1814165465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f008415aa30 0x7f00841f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:05.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.622+0000 7f0088a00640 1 --2- 172.21.15.94:0/1814165465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00840cd360 0x7f008414d5b0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:05.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.622+0000 7f0088a00640 1 -- 172.21.15.94:0/1814165465 >> 172.21.15.94:0/1814165465 conn(0x7f00840b2520 msgr2=0x7f00840cbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:05.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.622+0000 7f0088a00640 1 -- 172.21.15.94:0/1814165465 shutdown_connections 2024-09-10T23:36:05.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.622+0000 7f0088a00640 1 -- 172.21.15.94:0/1814165465 wait complete. 2024-09-10T23:36:05.644 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:05.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.786+0000 7f6fc2800640 1 -- 172.21.15.94:0/3466769632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa4004810 msgr2=0x7f6fa4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:05.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.786+0000 7f6fc2800640 1 --2- 172.21.15.94:0/3466769632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa4004810 0x7f6fa4004c90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009d30 tx=0x7f6fa801d410 comp rx=0 tx=0).stop 2024-09-10T23:36:05.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.786+0000 7f6fc2800640 1 -- 172.21.15.94:0/3466769632 shutdown_connections 2024-09-10T23:36:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.786+0000 7f6fc2800640 1 --2- 172.21.15.94:0/3466769632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa4004810 0x7f6fa4004c90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.786+0000 7f6fc2800640 1 -- 172.21.15.94:0/3466769632 >> 172.21.15.94:0/3466769632 conn(0x7f6fbc09aa70 msgr2=0x7f6fbc09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.786+0000 7f6fc2800640 1 -- 172.21.15.94:0/3466769632 shutdown_connections 2024-09-10T23:36:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.786+0000 7f6fc2800640 1 -- 172.21.15.94:0/3466769632 wait complete. 2024-09-10T23:36:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.786+0000 7f6fc2800640 1 Processor -- start 2024-09-10T23:36:05.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.786+0000 7f6fc2800640 1 -- start start 2024-09-10T23:36:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.786+0000 7f6fc2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa4004810 0x7f6fbc1f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.786+0000 7f6fc2800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc0c7f00 0x7f6fbc14b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.786+0000 7f6fc2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fbc06dd30 0x7f6fbc1f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.786+0000 7f6fc2800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc0d01e0 con 0x7f6fa4004810 2024-09-10T23:36:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.786+0000 7f6fc2800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc0ae960 con 0x7f6fbc06dd30 2024-09-10T23:36:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.786+0000 7f6fc2800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6fbc0d0360 con 0x7f6fbc0c7f00 2024-09-10T23:36:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fc1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa4004810 0x7f6fbc1f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fc1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa4004810 0x7f6fbc1f1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59268/0 (socket says 172.21.15.94:59268) 2024-09-10T23:36:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fc1400640 1 -- 172.21.15.94:0/3117412322 learned_addr learned my addr 172.21.15.94:0/3117412322 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:05.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fc0a00640 1 --2- 172.21.15.94:0/3117412322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc0c7f00 0x7f6fbc14b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:05.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fc1e00640 1 --2- 172.21.15.94:0/3117412322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fbc06dd30 0x7f6fbc1f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:05.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fc1400640 1 -- 172.21.15.94:0/3117412322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc0c7f00 msgr2=0x7f6fbc14b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:05.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fc1400640 1 --2- 172.21.15.94:0/3117412322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc0c7f00 0x7f6fbc14b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:05.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fc1400640 1 -- 172.21.15.94:0/3117412322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fbc06dd30 msgr2=0x7f6fbc1f7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:05.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fc1400640 1 --2- 172.21.15.94:0/3117412322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fbc06dd30 0x7f6fbc1f7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:05.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fc1400640 1 -- 172.21.15.94:0/3117412322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fbc1f1cd0 con 0x7f6fa4004810 2024-09-10T23:36:05.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fc1400640 1 --2- 172.21.15.94:0/3117412322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa4004810 0x7f6fbc1f1790 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0003f30 tx=0x7f6fb0001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:05.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fb9600640 1 -- 172.21.15.94:0/3117412322 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0011640 con 0x7f6fa4004810 2024-09-10T23:36:05.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fb9600640 1 -- 172.21.15.94:0/3117412322 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fb0016020 con 0x7f6fa4004810 2024-09-10T23:36:05.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fb9600640 1 -- 172.21.15.94:0/3117412322 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0011a40 con 0x7f6fa4004810 2024-09-10T23:36:05.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fc2800640 1 -- 172.21.15.94:0/3117412322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fbc14bcc0 con 0x7f6fa4004810 2024-09-10T23:36:05.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fc2800640 1 -- 172.21.15.94:0/3117412322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fbc0b63b0 con 0x7f6fa4004810 2024-09-10T23:36:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fc2800640 1 -- 172.21.15.94:0/3117412322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fbc0b1c20 con 0x7f6fa4004810 2024-09-10T23:36:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.790+0000 7f6fb9600640 1 -- 172.21.15.94:0/3117412322 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 10) ==== 96598+0+0 (secure 0 0 0) 0x7f6fb0011be0 con 0x7f6fa4004810 2024-09-10T23:36:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.794+0000 7f6fb9600640 1 --2- 172.21.15.94:0/3117412322 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f9c072820 0x7f6f9c074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.794+0000 7f6fb9600640 1 -- 172.21.15.94:0/3117412322 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f6fb0094380 con 0x7f6fa4004810 2024-09-10T23:36:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.794+0000 7f6fc0a00640 1 --2- 172.21.15.94:0/3117412322 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f9c072820 0x7f6f9c074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:05.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.794+0000 7f6fc0a00640 1 --2- 172.21.15.94:0/3117412322 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f9c072820 0x7f6f9c074ce0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80af000 tx=0x7f6fa801a1b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:05.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.798+0000 7f6fb9600640 1 -- 172.21.15.94:0/3117412322 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6fb0020020 con 0x7f6fa4004810 2024-09-10T23:36:05.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.926+0000 7f6fb9600640 1 -- 172.21.15.94:0/3117412322 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f6fb005b240 con 0x7f6fa4004810 2024-09-10T23:36:05.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.998+0000 7f6fc2800640 1 -- 172.21.15.94:0/3117412322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fbc09aff0 con 0x7f6fa4004810 2024-09-10T23:36:06.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:05.998+0000 7f6fb9600640 1 -- 172.21.15.94:0/3117412322 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5243 (secure 0 0 0) 0x7f6fbc1f1cd0 con 0x7f6fa4004810 2024-09-10T23:36:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:06.000 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":35,"btime":"2024-09-10T23:35:51:877544+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5814,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3613711739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3613711739},{"type":"v1","addr":"172.21.15.94:6834","nonce":3613711739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":195107,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1388563408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1388563408},{"type":"v1","addr":"172.21.15.181:6836","nonce":1388563408}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":195110,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1882902083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1882902083},{"type":"v1","addr":"172.21.15.181:6837","nonce":1882902083}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:35:36.778166+0000","modified":"2024-09-10T23:35:39.784176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195113":{"gid":195113,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/2905667261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2905667261},{"type":"v1","addr":"172.21.15.94:6835","nonce":2905667261}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195113,"qdb_cluster":[195113]},"id":4}]} 2024-09-10T23:36:06.001 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 35 2024-09-10T23:36:06.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.002+0000 7f6fc2800640 1 -- 172.21.15.94:0/3117412322 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f9c072820 msgr2=0x7f6f9c074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:06.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.002+0000 7f6fc2800640 1 --2- 172.21.15.94:0/3117412322 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f9c072820 0x7f6f9c074ce0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80af000 tx=0x7f6fa801a1b0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.002+0000 7f6fc2800640 1 -- 172.21.15.94:0/3117412322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa4004810 msgr2=0x7f6fbc1f1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:06.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.002+0000 7f6fc2800640 1 --2- 172.21.15.94:0/3117412322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa4004810 0x7f6fbc1f1790 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0003f30 tx=0x7f6fb0001700 comp rx=0 tx=0).stop 2024-09-10T23:36:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.002+0000 7f6fc2800640 1 -- 172.21.15.94:0/3117412322 shutdown_connections 2024-09-10T23:36:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.002+0000 7f6fc2800640 1 --2- 172.21.15.94:0/3117412322 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f9c072820 0x7f6f9c074ce0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.002+0000 7f6fc2800640 1 --2- 172.21.15.94:0/3117412322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fbc06dd30 0x7f6fbc1f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.002+0000 7f6fc2800640 1 --2- 172.21.15.94:0/3117412322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fbc0c7f00 0x7f6fbc14b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.002+0000 7f6fc2800640 1 --2- 172.21.15.94:0/3117412322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fa4004810 0x7f6fbc1f1790 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.002+0000 7f6fc2800640 1 -- 172.21.15.94:0/3117412322 >> 172.21.15.94:0/3117412322 conn(0x7f6fbc09aa70 msgr2=0x7f6fbc0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.002+0000 7f6fc2800640 1 -- 172.21.15.94:0/3117412322 shutdown_connections 2024-09-10T23:36:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.002+0000 7f6fc2800640 1 -- 172.21.15.94:0/3117412322 wait complete. 2024-09-10T23:36:06.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:36:06.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.162+0000 7f8462800640 1 -- 172.21.15.94:0/2436675338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8440003a40 msgr2=0x7f8440003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:06.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.162+0000 7f8462800640 1 --2- 172.21.15.94:0/2436675338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8440003a40 0x7f8440003e60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f844c00c260 tx=0x7f844c01cf10 comp rx=0 tx=0).stop 2024-09-10T23:36:06.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.162+0000 7f845be00640 1 -- 172.21.15.94:0/2436675338 reap_dead start 2024-09-10T23:36:06.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.162+0000 7f8462800640 1 -- 172.21.15.94:0/2436675338 shutdown_connections 2024-09-10T23:36:06.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.162+0000 7f8462800640 1 -- 172.21.15.94:0/2436675338 >> 172.21.15.94:0/2436675338 conn(0x7f845c09aa70 msgr2=0x7f845c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:06.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.162+0000 7f8462800640 1 -- 172.21.15.94:0/2436675338 shutdown_connections 2024-09-10T23:36:06.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.162+0000 7f8462800640 1 -- 172.21.15.94:0/2436675338 wait complete. 2024-09-10T23:36:06.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.162+0000 7f8462800640 1 Processor -- start 2024-09-10T23:36:06.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.162+0000 7f8462800640 1 -- start start 2024-09-10T23:36:06.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f8462800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8440003a40 0x7f845c15e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:06.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f8462800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f845c1f9050 0x7f845c1f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:06.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f8462800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f845c155e70 0x7f845c070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:06.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f8462800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f845c167b80 con 0x7f845c1f9050 2024-09-10T23:36:06.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f8462800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f845c0d2400 con 0x7f8440003a40 2024-09-10T23:36:06.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f8462800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f845c167d00 con 0x7f845c155e70 2024-09-10T23:36:06.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f845aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f845c1f9050 0x7f845c1f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:06.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f845aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f845c1f9050 0x7f845c1f5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59288/0 (socket says 172.21.15.94:59288) 2024-09-10T23:36:06.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f845aa00640 1 -- 172.21.15.94:0/1261357869 learned_addr learned my addr 172.21.15.94:0/1261357869 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:06.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f845b400640 1 --2- 172.21.15.94:0/1261357869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8440003a40 0x7f845c15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:06.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f845be00640 1 --2- 172.21.15.94:0/1261357869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f845c155e70 0x7f845c070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f845aa00640 1 -- 172.21.15.94:0/1261357869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f845c155e70 msgr2=0x7f845c070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f845aa00640 1 --2- 172.21.15.94:0/1261357869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f845c155e70 0x7f845c070b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f845aa00640 1 -- 172.21.15.94:0/1261357869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8440003a40 msgr2=0x7f845c15e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f845aa00640 1 --2- 172.21.15.94:0/1261357869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8440003a40 0x7f845c15e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f845aa00640 1 -- 172.21.15.94:0/1261357869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845c15eed0 con 0x7f845c1f9050 2024-09-10T23:36:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f845aa00640 1 --2- 172.21.15.94:0/1261357869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f845c1f9050 0x7f845c1f5a10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8448011ae0 tx=0x7f8448009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f8453400640 1 -- 172.21.15.94:0/1261357869 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844801e070 con 0x7f845c1f9050 2024-09-10T23:36:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f8453400640 1 -- 172.21.15.94:0/1261357869 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f844800fe40 con 0x7f845c1f9050 2024-09-10T23:36:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f8462800640 1 -- 172.21.15.94:0/1261357869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845c1f6240 con 0x7f845c1f9050 2024-09-10T23:36:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f8453400640 1 -- 172.21.15.94:0/1261357869 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84480042b0 con 0x7f845c1f9050 2024-09-10T23:36:06.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f8462800640 1 -- 172.21.15.94:0/1261357869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f845c1f6800 con 0x7f845c1f9050 2024-09-10T23:36:06.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f8462800640 1 -- 172.21.15.94:0/1261357869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f845c1617e0 con 0x7f845c1f9050 2024-09-10T23:36:06.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.166+0000 7f8453400640 1 -- 172.21.15.94:0/1261357869 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f844802b080 con 0x7f845c1f9050 2024-09-10T23:36:06.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.170+0000 7f8453400640 1 --2- 172.21.15.94:0/1261357869 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f843c072940 0x7f843c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:06.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.170+0000 7f8453400640 1 -- 172.21.15.94:0/1261357869 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8448094ad0 con 0x7f845c1f9050 2024-09-10T23:36:06.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.170+0000 7f845b400640 1 --2- 172.21.15.94:0/1261357869 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f843c072940 0x7f843c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:06.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.170+0000 7f845b400640 1 --2- 172.21.15.94:0/1261357869 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f843c072940 0x7f843c074e00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f844c0ab000 tx=0x7f844c019d50 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:06.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.170+0000 7f8453400640 1 -- 172.21.15.94:0/1261357869 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f844805f6d0 con 0x7f845c1f9050 2024-09-10T23:36:06.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.338+0000 7f8462800640 1 -- 172.21.15.94:0/1261357869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f845c0b1e70 con 0x7f845c1f9050 2024-09-10T23:36:06.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.338+0000 7f8453400640 1 -- 172.21.15.94:0/1261357869 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16713 (secure 0 0 0) 0x7f8448064100 con 0x7f845c1f9050 2024-09-10T23:36:06.340 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:06.340 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":58,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:35:38.384032+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:35:34.484637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T23:35:35.740496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:36:06.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.342+0000 7f8462800640 1 -- 172.21.15.94:0/1261357869 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f843c072940 msgr2=0x7f843c074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:06.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.342+0000 7f8462800640 1 --2- 172.21.15.94:0/1261357869 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f843c072940 0x7f843c074e00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f844c0ab000 tx=0x7f844c019d50 comp rx=0 tx=0).stop 2024-09-10T23:36:06.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.342+0000 7f8462800640 1 -- 172.21.15.94:0/1261357869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f845c1f9050 msgr2=0x7f845c1f5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:06.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.342+0000 7f8462800640 1 --2- 172.21.15.94:0/1261357869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f845c1f9050 0x7f845c1f5a10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8448011ae0 tx=0x7f8448009ea0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.342+0000 7f8462800640 1 -- 172.21.15.94:0/1261357869 shutdown_connections 2024-09-10T23:36:06.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.342+0000 7f8462800640 1 --2- 172.21.15.94:0/1261357869 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f843c072940 0x7f843c074e00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.342+0000 7f8462800640 1 --2- 172.21.15.94:0/1261357869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f845c155e70 0x7f845c070b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.342+0000 7f8462800640 1 --2- 172.21.15.94:0/1261357869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f845c1f9050 0x7f845c1f5a10 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.342+0000 7f8462800640 1 --2- 172.21.15.94:0/1261357869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8440003a40 0x7f845c15e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.342+0000 7f8462800640 1 -- 172.21.15.94:0/1261357869 >> 172.21.15.94:0/1261357869 conn(0x7f845c09aa70 msgr2=0x7f845c15d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:06.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.342+0000 7f8462800640 1 -- 172.21.15.94:0/1261357869 shutdown_connections 2024-09-10T23:36:06.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.342+0000 7f8462800640 1 -- 172.21.15.94:0/1261357869 wait complete. 2024-09-10T23:36:06.362 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T23:36:06.362 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:06.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.502+0000 7f8135a00640 1 -- 172.21.15.94:0/1943689384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81300b2bb0 msgr2=0x7f81300d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:06.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.502+0000 7f8135a00640 1 --2- 172.21.15.94:0/1943689384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81300b2bb0 0x7f81300d00a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8130073820 tx=0x7f811c01c4e0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.502+0000 7f8135a00640 1 -- 172.21.15.94:0/1943689384 shutdown_connections 2024-09-10T23:36:06.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.502+0000 7f8135a00640 1 --2- 172.21.15.94:0/1943689384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81300d05e0 0x7f81300b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.502+0000 7f8135a00640 1 --2- 172.21.15.94:0/1943689384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81300b2bb0 0x7f81300d00a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.502+0000 7f8135a00640 1 --2- 172.21.15.94:0/1943689384 >> v2:172.21.15.181:3300/0 conn(0x7f813015f5c0 0x7f81300cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.502+0000 7f8135a00640 1 -- 172.21.15.94:0/1943689384 >> 172.21.15.94:0/1943689384 conn(0x7f81300d1c60 msgr2=0x7f81300d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:06.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.502+0000 7f8135a00640 1 -- 172.21.15.94:0/1943689384 shutdown_connections 2024-09-10T23:36:06.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.502+0000 7f8135a00640 1 -- 172.21.15.94:0/1943689384 wait complete. 2024-09-10T23:36:06.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f8135a00640 1 Processor -- start 2024-09-10T23:36:06.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f8135a00640 1 -- start start 2024-09-10T23:36:06.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f8135a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81300b2bb0 0x7f8130147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:06.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f8135a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81300d05e0 0x7f81300c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:06.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f8135a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f813015f5c0 0x7f81300b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:06.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f8135a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f81300afa90 con 0x7f81300d05e0 2024-09-10T23:36:06.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f8135a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f81300af910 con 0x7f813015f5c0 2024-09-10T23:36:06.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f8135a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f81300afc10 con 0x7f81300b2bb0 2024-09-10T23:36:06.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f812e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81300d05e0 0x7f81300c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:06.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f812e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81300d05e0 0x7f81300c5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59300/0 (socket says 172.21.15.94:59300) 2024-09-10T23:36:06.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f812e000640 1 -- 172.21.15.94:0/3721965017 learned_addr learned my addr 172.21.15.94:0/3721965017 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:06.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f812ea00640 1 --2- 172.21.15.94:0/3721965017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81300b2bb0 0x7f8130147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:06.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f812f400640 1 --2- 172.21.15.94:0/3721965017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f813015f5c0 0x7f81300b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:06.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f812e000640 1 -- 172.21.15.94:0/3721965017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81300b2bb0 msgr2=0x7f8130147e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:06.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f812e000640 1 --2- 172.21.15.94:0/3721965017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81300b2bb0 0x7f8130147e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f812e000640 1 -- 172.21.15.94:0/3721965017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f813015f5c0 msgr2=0x7f81300b6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:06.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f812e000640 1 --2- 172.21.15.94:0/3721965017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f813015f5c0 0x7f81300b6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f812e000640 1 -- 172.21.15.94:0/3721965017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81300c5fc0 con 0x7f81300d05e0 2024-09-10T23:36:06.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f812ea00640 1 --2- 172.21.15.94:0/3721965017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81300b2bb0 0x7f8130147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:36:06.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f812f400640 1 --2- 172.21.15.94:0/3721965017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f813015f5c0 0x7f81300b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:36:06.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f812e000640 1 --2- 172.21.15.94:0/3721965017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81300d05e0 0x7f81300c5790 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f811c009230 tx=0x7f811c0051b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:06.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f8126a00640 1 -- 172.21.15.94:0/3721965017 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c0bc070 con 0x7f81300d05e0 2024-09-10T23:36:06.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f8126a00640 1 -- 172.21.15.94:0/3721965017 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f811c005490 con 0x7f81300d05e0 2024-09-10T23:36:06.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f8135a00640 1 -- 172.21.15.94:0/3721965017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81300c62a0 con 0x7f81300d05e0 2024-09-10T23:36:06.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f8126a00640 1 -- 172.21.15.94:0/3721965017 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c0b1040 con 0x7f81300d05e0 2024-09-10T23:36:06.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.506+0000 7f8135a00640 1 -- 172.21.15.94:0/3721965017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81300c6f80 con 0x7f81300d05e0 2024-09-10T23:36:06.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.510+0000 7f8135a00640 1 -- 172.21.15.94:0/3721965017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81300b15c0 con 0x7f81300d05e0 2024-09-10T23:36:06.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.510+0000 7f8126a00640 1 -- 172.21.15.94:0/3721965017 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f811c0ac8f0 con 0x7f81300d05e0 2024-09-10T23:36:06.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.510+0000 7f8126a00640 1 --2- 172.21.15.94:0/3721965017 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8110072940 0x7f8110074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:06.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.510+0000 7f8126a00640 1 -- 172.21.15.94:0/3721965017 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f811c12d670 con 0x7f81300d05e0 2024-09-10T23:36:06.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.510+0000 7f812ea00640 1 --2- 172.21.15.94:0/3721965017 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8110072940 0x7f8110074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:06.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.510+0000 7f812ea00640 1 --2- 172.21.15.94:0/3721965017 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8110072940 0x7f8110074e00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8120009d90 tx=0x7f8120009320 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.514+0000 7f8126a00640 1 -- 172.21.15.94:0/3721965017 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f811c0164b0 con 0x7f81300d05e0 2024-09-10T23:36:06.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.714+0000 7f8135a00640 1 -- 172.21.15.94:0/3721965017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81300749c0 con 0x7f81300d05e0 2024-09-10T23:36:06.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.714+0000 7f8126a00640 1 -- 172.21.15.94:0/3721965017 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5243 (secure 0 0 0) 0x7f811c0f86f0 con 0x7f81300d05e0 2024-09-10T23:36:06.715 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:06.715 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":35,"btime":"2024-09-10T23:35:51:877544+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5814,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3613711739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3613711739},{"type":"v1","addr":"172.21.15.94:6834","nonce":3613711739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":195107,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1388563408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1388563408},{"type":"v1","addr":"172.21.15.181:6836","nonce":1388563408}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":195110,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1882902083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1882902083},{"type":"v1","addr":"172.21.15.181:6837","nonce":1882902083}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:35:36.778166+0000","modified":"2024-09-10T23:35:39.784176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195113":{"gid":195113,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/2905667261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2905667261},{"type":"v1","addr":"172.21.15.94:6835","nonce":2905667261}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195113,"qdb_cluster":[195113]},"id":4}]} 2024-09-10T23:36:06.716 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 35 2024-09-10T23:36:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.714+0000 7f8135a00640 1 -- 172.21.15.94:0/3721965017 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8110072940 msgr2=0x7f8110074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.714+0000 7f8135a00640 1 --2- 172.21.15.94:0/3721965017 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8110072940 0x7f8110074e00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8120009d90 tx=0x7f8120009320 comp rx=0 tx=0).stop 2024-09-10T23:36:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.714+0000 7f8135a00640 1 -- 172.21.15.94:0/3721965017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81300d05e0 msgr2=0x7f81300c5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.714+0000 7f8135a00640 1 --2- 172.21.15.94:0/3721965017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81300d05e0 0x7f81300c5790 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f811c009230 tx=0x7f811c0051b0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.714+0000 7f8135a00640 1 -- 172.21.15.94:0/3721965017 shutdown_connections 2024-09-10T23:36:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.714+0000 7f8135a00640 1 --2- 172.21.15.94:0/3721965017 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8110072940 0x7f8110074e00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.714+0000 7f8135a00640 1 --2- 172.21.15.94:0/3721965017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f813015f5c0 0x7f81300b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.714+0000 7f8135a00640 1 --2- 172.21.15.94:0/3721965017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81300d05e0 0x7f81300c5790 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.714+0000 7f8135a00640 1 --2- 172.21.15.94:0/3721965017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81300b2bb0 0x7f8130147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.714+0000 7f8135a00640 1 -- 172.21.15.94:0/3721965017 >> 172.21.15.94:0/3721965017 conn(0x7f81300d1c60 msgr2=0x7f813015dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.714+0000 7f8135a00640 1 -- 172.21.15.94:0/3721965017 shutdown_connections 2024-09-10T23:36:06.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.714+0000 7f8135a00640 1 -- 172.21.15.94:0/3721965017 wait complete. 2024-09-10T23:36:06.737 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.874+0000 7f23d9c00640 1 -- 172.21.15.94:0/1508542693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23d406e7d0 msgr2=0x7f23d40ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.874+0000 7f23d9c00640 1 --2- 172.21.15.94:0/1508542693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23d406e7d0 0x7f23d40ce680 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f23d40b47d0 tx=0x7f23cc01ca90 comp rx=0 tx=0).stop 2024-09-10T23:36:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.874+0000 7f23d9c00640 1 -- 172.21.15.94:0/1508542693 shutdown_connections 2024-09-10T23:36:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.874+0000 7f23d9c00640 1 --2- 172.21.15.94:0/1508542693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23d406e7d0 0x7f23d40ce680 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.874+0000 7f23d9c00640 1 --2- 172.21.15.94:0/1508542693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23d4162560 0x7f23d406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.874+0000 7f23d9c00640 1 --2- 172.21.15.94:0/1508542693 >> v2:172.21.15.181:3300/0 conn(0x7f23d415dd50 0x7f23d40b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.874+0000 7f23d9c00640 1 -- 172.21.15.94:0/1508542693 >> 172.21.15.94:0/1508542693 conn(0x7f23d409aa70 msgr2=0x7f23d409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.874+0000 7f23d9c00640 1 -- 172.21.15.94:0/1508542693 shutdown_connections 2024-09-10T23:36:06.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.874+0000 7f23d9c00640 1 -- 172.21.15.94:0/1508542693 wait complete. 2024-09-10T23:36:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d9c00640 1 Processor -- start 2024-09-10T23:36:06.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d9c00640 1 -- start start 2024-09-10T23:36:06.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d9c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23d406e7d0 0x7f23d414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:06.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d9c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d415dd50 0x7f23d40ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d9c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23d4162560 0x7f23d40ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d9c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23d40cc6a0 con 0x7f23d406e7d0 2024-09-10T23:36:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d9c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f23d40cc520 con 0x7f23d415dd50 2024-09-10T23:36:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d9c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f23d40cc820 con 0x7f23d4162560 2024-09-10T23:36:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23d406e7d0 0x7f23d414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23d406e7d0 0x7f23d414e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59316/0 (socket says 172.21.15.94:59316) 2024-09-10T23:36:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d2a00640 1 -- 172.21.15.94:0/3155990066 learned_addr learned my addr 172.21.15.94:0/3155990066 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d3400640 1 --2- 172.21.15.94:0/3155990066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23d4162560 0x7f23d40ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23ca000640 1 --2- 172.21.15.94:0/3155990066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d415dd50 0x7f23d40ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d2a00640 1 -- 172.21.15.94:0/3155990066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23d4162560 msgr2=0x7f23d40ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d2a00640 1 --2- 172.21.15.94:0/3155990066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23d4162560 0x7f23d40ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d2a00640 1 -- 172.21.15.94:0/3155990066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d415dd50 msgr2=0x7f23d40ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d2a00640 1 --2- 172.21.15.94:0/3155990066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d415dd50 0x7f23d40ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:06.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d2a00640 1 -- 172.21.15.94:0/3155990066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d40cb710 con 0x7f23d406e7d0 2024-09-10T23:36:06.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d2a00640 1 --2- 172.21.15.94:0/3155990066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23d406e7d0 0x7f23d414e2d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f23c400d7a0 tx=0x7f23c400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:06.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23cb400640 1 -- 172.21.15.94:0/3155990066 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4017070 con 0x7f23d406e7d0 2024-09-10T23:36:06.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23cb400640 1 -- 172.21.15.94:0/3155990066 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23c4009d20 con 0x7f23d406e7d0 2024-09-10T23:36:06.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d9c00640 1 -- 172.21.15.94:0/3155990066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d40c97f0 con 0x7f23d406e7d0 2024-09-10T23:36:06.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23cb400640 1 -- 172.21.15.94:0/3155990066 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c40106c0 con 0x7f23d406e7d0 2024-09-10T23:36:06.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.878+0000 7f23d9c00640 1 -- 172.21.15.94:0/3155990066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d40c9db0 con 0x7f23d406e7d0 2024-09-10T23:36:06.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.882+0000 7f23d9c00640 1 -- 172.21.15.94:0/3155990066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d40b1b80 con 0x7f23d406e7d0 2024-09-10T23:36:06.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.882+0000 7f23cb400640 1 -- 172.21.15.94:0/3155990066 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f23c4010860 con 0x7f23d406e7d0 2024-09-10T23:36:06.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.882+0000 7f23cb400640 1 --2- 172.21.15.94:0/3155990066 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f23b0072940 0x7f23b0074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:06.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.882+0000 7f23cb400640 1 -- 172.21.15.94:0/3155990066 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f23c4093d60 con 0x7f23d406e7d0 2024-09-10T23:36:06.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.882+0000 7f23ca000640 1 --2- 172.21.15.94:0/3155990066 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f23b0072940 0x7f23b0074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:06.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.882+0000 7f23ca000640 1 --2- 172.21.15.94:0/3155990066 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f23b0072940 0x7f23b0074e00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f23d40cbd70 tx=0x7f23bc00b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:06.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:06.886+0000 7f23cb400640 1 -- 172.21.15.94:0/3155990066 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f23c405e820 con 0x7f23d406e7d0 2024-09-10T23:36:07.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.086+0000 7f23d9c00640 1 -- 172.21.15.94:0/3155990066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23d40d29c0 con 0x7f23d406e7d0 2024-09-10T23:36:07.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.086+0000 7f23cb400640 1 -- 172.21.15.94:0/3155990066 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5243 (secure 0 0 0) 0x7f23c4063250 con 0x7f23d406e7d0 2024-09-10T23:36:07.087 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:07.087 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":35,"btime":"2024-09-10T23:35:51:877544+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5814,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3613711739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3613711739},{"type":"v1","addr":"172.21.15.94:6834","nonce":3613711739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":195107,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1388563408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1388563408},{"type":"v1","addr":"172.21.15.181:6836","nonce":1388563408}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":195110,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1882902083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1882902083},{"type":"v1","addr":"172.21.15.181:6837","nonce":1882902083}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:35:36.778166+0000","modified":"2024-09-10T23:35:39.784176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195113},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195113":{"gid":195113,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/2905667261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":2905667261},{"type":"v1","addr":"172.21.15.94:6835","nonce":2905667261}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195113,"qdb_cluster":[195113]},"id":4}]} 2024-09-10T23:36:07.087 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 35 2024-09-10T23:36:07.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.086+0000 7f23d9c00640 1 -- 172.21.15.94:0/3155990066 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f23b0072940 msgr2=0x7f23b0074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:07.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.086+0000 7f23d9c00640 1 --2- 172.21.15.94:0/3155990066 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f23b0072940 0x7f23b0074e00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f23d40cbd70 tx=0x7f23bc00b2e0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.086+0000 7f23d9c00640 1 -- 172.21.15.94:0/3155990066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23d406e7d0 msgr2=0x7f23d414e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:07.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.086+0000 7f23d9c00640 1 --2- 172.21.15.94:0/3155990066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23d406e7d0 0x7f23d414e2d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f23c400d7a0 tx=0x7f23c400dc60 comp rx=0 tx=0).stop 2024-09-10T23:36:07.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.086+0000 7f23d9c00640 1 -- 172.21.15.94:0/3155990066 shutdown_connections 2024-09-10T23:36:07.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.086+0000 7f23d9c00640 1 --2- 172.21.15.94:0/3155990066 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f23b0072940 0x7f23b0074e00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.086+0000 7f23d9c00640 1 --2- 172.21.15.94:0/3155990066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d415dd50 0x7f23d40ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.086+0000 7f23d9c00640 1 --2- 172.21.15.94:0/3155990066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23d4162560 0x7f23d40ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.086+0000 7f23d9c00640 1 --2- 172.21.15.94:0/3155990066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23d406e7d0 0x7f23d414e2d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.086+0000 7f23d9c00640 1 -- 172.21.15.94:0/3155990066 >> 172.21.15.94:0/3155990066 conn(0x7f23d409aa70 msgr2=0x7f23d40cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:07.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.086+0000 7f23d9c00640 1 -- 172.21.15.94:0/3155990066 shutdown_connections 2024-09-10T23:36:07.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.086+0000 7f23d9c00640 1 -- 172.21.15.94:0/3155990066 wait complete. 2024-09-10T23:36:07.109 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:36:07.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.250+0000 7f6f4fa00640 1 -- 172.21.15.94:0/2196094512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f4806f600 msgr2=0x7f6f480b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:07.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.250+0000 7f6f4fa00640 1 --2- 172.21.15.94:0/2196094512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f4806f600 0x7f6f480b35c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000ad90 tx=0x7f6f4001dc20 comp rx=0 tx=0).stop 2024-09-10T23:36:07.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.250+0000 7f6f4fa00640 1 -- 172.21.15.94:0/2196094512 shutdown_connections 2024-09-10T23:36:07.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.250+0000 7f6f4fa00640 1 --2- 172.21.15.94:0/2196094512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f4806f600 0x7f6f480b35c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.250+0000 7f6f4fa00640 1 --2- 172.21.15.94:0/2196094512 >> v2:172.21.15.181:3300/0 conn(0x7f6f48157c60 0x7f6f48072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.250+0000 7f6f4fa00640 1 -- 172.21.15.94:0/2196094512 >> 172.21.15.94:0/2196094512 conn(0x7f6f4806cdf0 msgr2=0x7f6f4806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:07.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.250+0000 7f6f4fa00640 1 -- 172.21.15.94:0/2196094512 shutdown_connections 2024-09-10T23:36:07.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.250+0000 7f6f4fa00640 1 -- 172.21.15.94:0/2196094512 wait complete. 2024-09-10T23:36:07.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.250+0000 7f6f4fa00640 1 Processor -- start 2024-09-10T23:36:07.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.250+0000 7f6f4fa00640 1 -- start start 2024-09-10T23:36:07.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f4fa00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f4806f600 0x7f6f481508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:07.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f4fa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f48070150 0x7f6f48158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:07.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f4fa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f48157c60 0x7f6f48142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:07.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f4fa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4815b360 con 0x7f6f48070150 2024-09-10T23:36:07.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f4fa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4815b1e0 con 0x7f6f48157c60 2024-09-10T23:36:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f4fa00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4815b4e0 con 0x7f6f4806f600 2024-09-10T23:36:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f3fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f48070150 0x7f6f48158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f3fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f48070150 0x7f6f48158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59328/0 (socket says 172.21.15.94:59328) 2024-09-10T23:36:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f3fe00640 1 -- 172.21.15.94:0/93819918 learned_addr learned my addr 172.21.15.94:0/93819918 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f4d000640 1 --2- 172.21.15.94:0/93819918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f4806f600 0x7f6f481508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f4da00640 1 --2- 172.21.15.94:0/93819918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f48157c60 0x7f6f48142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:07.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f3fe00640 1 -- 172.21.15.94:0/93819918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f4806f600 msgr2=0x7f6f481508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:07.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f3fe00640 1 --2- 172.21.15.94:0/93819918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f4806f600 0x7f6f481508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f3fe00640 1 -- 172.21.15.94:0/93819918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f48157c60 msgr2=0x7f6f48142070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:07.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f3fe00640 1 --2- 172.21.15.94:0/93819918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f48157c60 0x7f6f48142070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f3fe00640 1 -- 172.21.15.94:0/93819918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f48159c30 con 0x7f6f48070150 2024-09-10T23:36:07.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f3fe00640 1 --2- 172.21.15.94:0/93819918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f48070150 0x7f6f48158c70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000c500 tx=0x7f6f3000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:07.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f45600640 1 -- 172.21.15.94:0/93819918 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f300052d0 con 0x7f6f48070150 2024-09-10T23:36:07.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f45600640 1 -- 172.21.15.94:0/93819918 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f30005470 con 0x7f6f48070150 2024-09-10T23:36:07.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f4fa00640 1 -- 172.21.15.94:0/93819918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f48159f10 con 0x7f6f48070150 2024-09-10T23:36:07.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f45600640 1 -- 172.21.15.94:0/93819918 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30012890 con 0x7f6f48070150 2024-09-10T23:36:07.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f4fa00640 1 -- 172.21.15.94:0/93819918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f48141870 con 0x7f6f48070150 2024-09-10T23:36:07.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f4fa00640 1 -- 172.21.15.94:0/93819918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f480b35c0 con 0x7f6f48070150 2024-09-10T23:36:07.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.254+0000 7f6f45600640 1 -- 172.21.15.94:0/93819918 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f6f30003600 con 0x7f6f48070150 2024-09-10T23:36:07.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.258+0000 7f6f45600640 1 --2- 172.21.15.94:0/93819918 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f24072940 0x7f6f24074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:07.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.258+0000 7f6f45600640 1 -- 172.21.15.94:0/93819918 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f6f30093440 con 0x7f6f48070150 2024-09-10T23:36:07.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.258+0000 7f6f4d000640 1 --2- 172.21.15.94:0/93819918 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f24072940 0x7f6f24074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:07.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.258+0000 7f6f4d000640 1 --2- 172.21.15.94:0/93819918 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f24072940 0x7f6f24074e00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009b40 tx=0x7f6f380091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:07.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.262+0000 7f6f45600640 1 -- 172.21.15.94:0/93819918 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6f3005de80 con 0x7f6f48070150 2024-09-10T23:36:07.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.426+0000 7f6f4fa00640 1 -- 172.21.15.94:0/93819918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f48073ae0 con 0x7f6f48070150 2024-09-10T23:36:07.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.430+0000 7f6f45600640 1 -- 172.21.15.94:0/93819918 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16713 (secure 0 0 0) 0x7f6f300628b0 con 0x7f6f48070150 2024-09-10T23:36:07.431 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:07.431 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":58,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:35:38.384032+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:35:34.484637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T23:35:35.740496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:36:07.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.430+0000 7f6f4fa00640 1 -- 172.21.15.94:0/93819918 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f24072940 msgr2=0x7f6f24074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:07.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.430+0000 7f6f4fa00640 1 --2- 172.21.15.94:0/93819918 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f24072940 0x7f6f24074e00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009b40 tx=0x7f6f380091e0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.430+0000 7f6f4fa00640 1 -- 172.21.15.94:0/93819918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f48070150 msgr2=0x7f6f48158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:07.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.430+0000 7f6f4fa00640 1 --2- 172.21.15.94:0/93819918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f48070150 0x7f6f48158c70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000c500 tx=0x7f6f3000c9c0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.430+0000 7f6f4fa00640 1 -- 172.21.15.94:0/93819918 shutdown_connections 2024-09-10T23:36:07.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.430+0000 7f6f4fa00640 1 --2- 172.21.15.94:0/93819918 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6f24072940 0x7f6f24074e00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.430+0000 7f6f4fa00640 1 --2- 172.21.15.94:0/93819918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f48157c60 0x7f6f48142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.430+0000 7f6f4fa00640 1 --2- 172.21.15.94:0/93819918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f48070150 0x7f6f48158c70 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.430+0000 7f6f4fa00640 1 --2- 172.21.15.94:0/93819918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f4806f600 0x7f6f481508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.430+0000 7f6f4fa00640 1 -- 172.21.15.94:0/93819918 >> 172.21.15.94:0/93819918 conn(0x7f6f4806cdf0 msgr2=0x7f6f4806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:07.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.430+0000 7f6f4fa00640 1 -- 172.21.15.94:0/93819918 shutdown_connections 2024-09-10T23:36:07.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.430+0000 7f6f4fa00640 1 -- 172.21.15.94:0/93819918 wait complete. 2024-09-10T23:36:07.453 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T23:36:07.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.590+0000 7fb51e400640 1 -- 172.21.15.94:0/2061672917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5180ce280 msgr2=0x7fb5180cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:07.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.590+0000 7fb51e400640 1 --2- 172.21.15.94:0/2061672917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5180ce280 0x7fb5180cf280 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb50800acd0 tx=0x7fb50801def0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.590+0000 7fb51e400640 1 -- 172.21.15.94:0/2061672917 shutdown_connections 2024-09-10T23:36:07.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.590+0000 7fb51e400640 1 --2- 172.21.15.94:0/2061672917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5180ce280 0x7fb5180cf280 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.590+0000 7fb51e400640 1 -- 172.21.15.94:0/2061672917 >> 172.21.15.94:0/2061672917 conn(0x7fb5180cfaf0 msgr2=0x7fb5180cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:07.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.590+0000 7fb51e400640 1 -- 172.21.15.94:0/2061672917 shutdown_connections 2024-09-10T23:36:07.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.590+0000 7fb51e400640 1 -- 172.21.15.94:0/2061672917 wait complete. 2024-09-10T23:36:07.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb51e400640 1 Processor -- start 2024-09-10T23:36:07.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb51e400640 1 -- start start 2024-09-10T23:36:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb51e400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51806e7c0 0x7fb518149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb51e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5180c64f0 0x7fb5180ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb51e400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5180ce280 0x7fb5180c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb51e400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb518168850 con 0x7fb5180c64f0 2024-09-10T23:36:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb51e400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181686d0 con 0x7fb51806e7c0 2024-09-10T23:36:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb51e400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb5181689d0 con 0x7fb5180ce280 2024-09-10T23:36:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb50ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5180c64f0 0x7fb5180ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb50ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5180c64f0 0x7fb5180ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59346/0 (socket says 172.21.15.94:59346) 2024-09-10T23:36:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb50ea00640 1 -- 172.21.15.94:0/856632211 learned_addr learned my addr 172.21.15.94:0/856632211 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:07.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb517400640 1 --2- 172.21.15.94:0/856632211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51806e7c0 0x7fb518149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb517e00640 1 --2- 172.21.15.94:0/856632211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5180ce280 0x7fb5180c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb50ea00640 1 -- 172.21.15.94:0/856632211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5180ce280 msgr2=0x7fb5180c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb50ea00640 1 --2- 172.21.15.94:0/856632211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5180ce280 0x7fb5180c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb50ea00640 1 -- 172.21.15.94:0/856632211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51806e7c0 msgr2=0x7fb518149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb50ea00640 1 --2- 172.21.15.94:0/856632211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51806e7c0 0x7fb518149d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb50ea00640 1 -- 172.21.15.94:0/856632211 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5180ada80 con 0x7fb5180c64f0 2024-09-10T23:36:07.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb50ea00640 1 --2- 172.21.15.94:0/856632211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5180c64f0 0x7fb5180ae840 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb50000c530 tx=0x7fb50000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb50fe00640 1 -- 172.21.15.94:0/856632211 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500017070 con 0x7fb5180c64f0 2024-09-10T23:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb50fe00640 1 -- 172.21.15.94:0/856632211 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb500003600 con 0x7fb5180c64f0 2024-09-10T23:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb50fe00640 1 -- 172.21.15.94:0/856632211 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500003a50 con 0x7fb5180c64f0 2024-09-10T23:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.594+0000 7fb51e400640 1 -- 172.21.15.94:0/856632211 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5180add60 con 0x7fb5180c64f0 2024-09-10T23:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.598+0000 7fb51e400640 1 -- 172.21.15.94:0/856632211 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51814a250 con 0x7fb5180c64f0 2024-09-10T23:36:07.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.598+0000 7fb51e400640 1 -- 172.21.15.94:0/856632211 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5180d0850 con 0x7fb5180c64f0 2024-09-10T23:36:07.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.598+0000 7fb50fe00640 1 -- 172.21.15.94:0/856632211 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7fb500003bf0 con 0x7fb5180c64f0 2024-09-10T23:36:07.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.598+0000 7fb50fe00640 1 --2- 172.21.15.94:0/856632211 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4f4072940 0x7fb4f4074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:07.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.598+0000 7fb50fe00640 1 -- 172.21.15.94:0/856632211 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb500092ea0 con 0x7fb5180c64f0 2024-09-10T23:36:07.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.598+0000 7fb517400640 1 --2- 172.21.15.94:0/856632211 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4f4072940 0x7fb4f4074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:07.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.602+0000 7fb517400640 1 --2- 172.21.15.94:0/856632211 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4f4072940 0x7fb4f4074e00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb5080b0000 tx=0x7fb50801c730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:07.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.602+0000 7fb50fe00640 1 -- 172.21.15.94:0/856632211 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb50005db20 con 0x7fb5180c64f0 2024-09-10T23:36:07.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.802+0000 7fb51e400640 1 -- 172.21.15.94:0/856632211 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb518157630 con 0x7fb5180c64f0 2024-09-10T23:36:07.804 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-10T23:36:07.802+0000 7f4045a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:36:07.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.806+0000 7fb50fe00640 1 -- 172.21.15.94:0/856632211 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7fb500062550 con 0x7fb5180c64f0 2024-09-10T23:36:07.806 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T23:36:07.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.806+0000 7fb51e400640 1 -- 172.21.15.94:0/856632211 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4f4072940 msgr2=0x7fb4f4074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:07.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.806+0000 7fb51e400640 1 --2- 172.21.15.94:0/856632211 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4f4072940 0x7fb4f4074e00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb5080b0000 tx=0x7fb50801c730 comp rx=0 tx=0).stop 2024-09-10T23:36:07.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.806+0000 7fb51e400640 1 -- 172.21.15.94:0/856632211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5180c64f0 msgr2=0x7fb5180ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:07.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.806+0000 7fb51e400640 1 --2- 172.21.15.94:0/856632211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5180c64f0 0x7fb5180ae840 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb50000c530 tx=0x7fb50000c9f0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.806+0000 7fb51e400640 1 -- 172.21.15.94:0/856632211 shutdown_connections 2024-09-10T23:36:07.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.806+0000 7fb51e400640 1 --2- 172.21.15.94:0/856632211 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb4f4072940 0x7fb4f4074e00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.806+0000 7fb51e400640 1 --2- 172.21.15.94:0/856632211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5180ce280 0x7fb5180c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.806+0000 7fb51e400640 1 --2- 172.21.15.94:0/856632211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5180c64f0 0x7fb5180ae840 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.806+0000 7fb51e400640 1 --2- 172.21.15.94:0/856632211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51806e7c0 0x7fb518149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.806+0000 7fb51e400640 1 -- 172.21.15.94:0/856632211 >> 172.21.15.94:0/856632211 conn(0x7fb5180cfaf0 msgr2=0x7fb5180ca550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:07.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.806+0000 7fb51e400640 1 -- 172.21.15.94:0/856632211 shutdown_connections 2024-09-10T23:36:07.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.806+0000 7fb51e400640 1 -- 172.21.15.94:0/856632211 wait complete. 2024-09-10T23:36:07.830 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T23:36:07.874 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-09-10T23:36:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9edae00640 1 -- 172.21.15.94:0/63692630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ed415f180 msgr2=0x7f9ed40cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9edae00640 1 --2- 172.21.15.94:0/63692630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ed415f180 0x7f9ed40cd0a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009900 tx=0x7f9ec401c8e0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9edae00640 1 -- 172.21.15.94:0/63692630 shutdown_connections 2024-09-10T23:36:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9edae00640 1 --2- 172.21.15.94:0/63692630 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ed40d01a0 0x7f9ed40b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9edae00640 1 --2- 172.21.15.94:0/63692630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ed415f180 0x7f9ed40cd0a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9edae00640 1 -- 172.21.15.94:0/63692630 >> 172.21.15.94:0/63692630 conn(0x7f9ed40d1cb0 msgr2=0x7f9ed40d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9edae00640 1 -- 172.21.15.94:0/63692630 shutdown_connections 2024-09-10T23:36:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9edae00640 1 -- 172.21.15.94:0/63692630 wait complete. 2024-09-10T23:36:07.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9edae00640 1 Processor -- start 2024-09-10T23:36:07.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9edae00640 1 -- start start 2024-09-10T23:36:07.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9edae00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ed40d01a0 0x7f9ed4157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:07.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9edae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ed415f180 0x7f9ed40afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:07.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9edae00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ed415f560 0x7f9ed4158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:07.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9edae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed4168c80 con 0x7f9ed415f180 2024-09-10T23:36:07.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9edae00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed4168b00 con 0x7f9ed40d01a0 2024-09-10T23:36:07.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9edae00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed4168e00 con 0x7f9ed415f560 2024-09-10T23:36:07.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9ed3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ed40d01a0 0x7f9ed4157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:07.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9ecb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ed415f180 0x7f9ed40afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:07.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9ed3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ed40d01a0 0x7f9ed4157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:53054/0 (socket says 172.21.15.94:53054) 2024-09-10T23:36:07.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9ed3e00640 1 -- 172.21.15.94:0/1409921421 learned_addr learned my addr 172.21.15.94:0/1409921421 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:07.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.978+0000 7f9ed8e00640 1 --2- 172.21.15.94:0/1409921421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ed415f560 0x7f9ed4158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:07.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.982+0000 7f9ed3e00640 1 -- 172.21.15.94:0/1409921421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ed415f560 msgr2=0x7f9ed4158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:07.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.982+0000 7f9ed3e00640 1 --2- 172.21.15.94:0/1409921421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ed415f560 0x7f9ed4158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.982+0000 7f9ed3e00640 1 -- 172.21.15.94:0/1409921421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ed415f180 msgr2=0x7f9ed40afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:07.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.982+0000 7f9ed3e00640 1 --2- 172.21.15.94:0/1409921421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ed415f180 0x7f9ed40afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:07.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.982+0000 7f9ed3e00640 1 -- 172.21.15.94:0/1409921421 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed41fad70 con 0x7f9ed40d01a0 2024-09-10T23:36:07.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.982+0000 7f9ecb400640 1 --2- 172.21.15.94:0/1409921421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ed415f180 0x7f9ed40afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:36:07.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.982+0000 7f9ed3e00640 1 --2- 172.21.15.94:0/1409921421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ed40d01a0 0x7f9ed4157d50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009850 tx=0x7f9ec40a57c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:07.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.982+0000 7f9ed0c00640 1 -- 172.21.15.94:0/1409921421 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec40b2040 con 0x7f9ed40d01a0 2024-09-10T23:36:07.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.982+0000 7f9ed0c00640 1 -- 172.21.15.94:0/1409921421 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ec4004430 con 0x7f9ed40d01a0 2024-09-10T23:36:07.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.982+0000 7f9ed0c00640 1 -- 172.21.15.94:0/1409921421 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4005290 con 0x7f9ed40d01a0 2024-09-10T23:36:07.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.982+0000 7f9edae00640 1 -- 172.21.15.94:0/1409921421 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed40b0790 con 0x7f9ed40d01a0 2024-09-10T23:36:07.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.982+0000 7f9edae00640 1 -- 172.21.15.94:0/1409921421 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed4158290 con 0x7f9ed40d01a0 2024-09-10T23:36:07.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.982+0000 7f9edae00640 1 -- 172.21.15.94:0/1409921421 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed41faf40 con 0x7f9ed40d01a0 2024-09-10T23:36:07.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.982+0000 7f9ed0c00640 1 -- 172.21.15.94:0/1409921421 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f9ec4005430 con 0x7f9ed40d01a0 2024-09-10T23:36:07.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.982+0000 7f9ed0c00640 1 --2- 172.21.15.94:0/1409921421 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9eb0072940 0x7f9eb0074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:07.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.986+0000 7f9ed0c00640 1 -- 172.21.15.94:0/1409921421 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5795+0+0 (secure 0 0 0) 0x7f9ec412cf50 con 0x7f9ed40d01a0 2024-09-10T23:36:07.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.986+0000 7f9ecb400640 1 --2- 172.21.15.94:0/1409921421 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9eb0072940 0x7f9eb0074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:07.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.986+0000 7f9ecb400640 1 --2- 172.21.15.94:0/1409921421 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9eb0072940 0x7f9eb0074e00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9ed406e670 tx=0x7f9ebc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:07.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:07.990+0000 7f9ed0c00640 1 -- 172.21.15.94:0/1409921421 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9ec40164d0 con 0x7f9ed40d01a0 2024-09-10T23:36:08.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:08.190+0000 7f9edae00640 1 -- 172.21.15.94:0/1409921421 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9ed40b16f0 con 0x7f9ed40d01a0 2024-09-10T23:36:08.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:08.810+0000 7f9ed0c00640 1 -- 172.21.15.94:0/1409921421 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7f9ec40f8070 con 0x7f9ed40d01a0 2024-09-10T23:36:08.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:08.810+0000 7f9edae00640 1 -- 172.21.15.94:0/1409921421 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9eb0072940 msgr2=0x7f9eb0074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:08.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:08.810+0000 7f9edae00640 1 --2- 172.21.15.94:0/1409921421 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9eb0072940 0x7f9eb0074e00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9ed406e670 tx=0x7f9ebc009040 comp rx=0 tx=0).stop 2024-09-10T23:36:08.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:08.810+0000 7f9edae00640 1 -- 172.21.15.94:0/1409921421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ed40d01a0 msgr2=0x7f9ed4157d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:08.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:08.810+0000 7f9edae00640 1 --2- 172.21.15.94:0/1409921421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ed40d01a0 0x7f9ed4157d50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009850 tx=0x7f9ec40a57c0 comp rx=0 tx=0).stop 2024-09-10T23:36:08.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:08.810+0000 7f9edae00640 1 -- 172.21.15.94:0/1409921421 shutdown_connections 2024-09-10T23:36:08.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:08.810+0000 7f9edae00640 1 --2- 172.21.15.94:0/1409921421 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9eb0072940 0x7f9eb0074e00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:08.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:08.810+0000 7f9edae00640 1 --2- 172.21.15.94:0/1409921421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ed415f560 0x7f9ed4158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:08.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:08.810+0000 7f9edae00640 1 --2- 172.21.15.94:0/1409921421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ed415f180 0x7f9ed40afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:08.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:08.810+0000 7f9edae00640 1 --2- 172.21.15.94:0/1409921421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ed40d01a0 0x7f9ed4157d50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:08.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:08.810+0000 7f9edae00640 1 -- 172.21.15.94:0/1409921421 >> 172.21.15.94:0/1409921421 conn(0x7f9ed40d1cb0 msgr2=0x7f9ed415d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:08.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:08.810+0000 7f9edae00640 1 -- 172.21.15.94:0/1409921421 shutdown_connections 2024-09-10T23:36:08.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:08.810+0000 7f9edae00640 1 -- 172.21.15.94:0/1409921421 wait complete. 2024-09-10T23:36:08.834 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T23:36:09.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.022+0000 7fe388400640 1 -- 172.21.15.94:0/3283673915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe368003fe0 msgr2=0x7fe368004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:09.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.022+0000 7fe388400640 1 --2- 172.21.15.94:0/3283673915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe368003fe0 0x7fe368004810 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009b30 tx=0x7fe36c01d230 comp rx=0 tx=0).stop 2024-09-10T23:36:09.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.022+0000 7fe386400640 1 -- 172.21.15.94:0/3283673915 reap_dead start 2024-09-10T23:36:09.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.022+0000 7fe388400640 1 -- 172.21.15.94:0/3283673915 shutdown_connections 2024-09-10T23:36:09.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.022+0000 7fe388400640 1 -- 172.21.15.94:0/3283673915 >> 172.21.15.94:0/3283673915 conn(0x7fe38006cdf0 msgr2=0x7fe38006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:09.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.022+0000 7fe388400640 1 -- 172.21.15.94:0/3283673915 shutdown_connections 2024-09-10T23:36:09.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.022+0000 7fe388400640 1 -- 172.21.15.94:0/3283673915 wait complete. 2024-09-10T23:36:09.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.022+0000 7fe388400640 1 Processor -- start 2024-09-10T23:36:09.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.022+0000 7fe388400640 1 -- start start 2024-09-10T23:36:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe388400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe368003fe0 0x7fe380144570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe388400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3801f8cf0 0x7fe3800cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe388400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3801f8000 0x7fe3801f8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe388400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe3800b3950 con 0x7fe3801f8cf0 2024-09-10T23:36:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe388400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe3800b37d0 con 0x7fe368003fe0 2024-09-10T23:36:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe388400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe38006f330 con 0x7fe3801f8000 2024-09-10T23:36:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe385000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3801f8cf0 0x7fe3800cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe385000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3801f8cf0 0x7fe3800cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59412/0 (socket says 172.21.15.94:59412) 2024-09-10T23:36:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe385000640 1 -- 172.21.15.94:0/2561779028 learned_addr learned my addr 172.21.15.94:0/2561779028 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe385a00640 1 --2- 172.21.15.94:0/2561779028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe368003fe0 0x7fe380144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe386400640 1 --2- 172.21.15.94:0/2561779028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3801f8000 0x7fe3801f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe385000640 1 -- 172.21.15.94:0/2561779028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3801f8000 msgr2=0x7fe3801f8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe385000640 1 --2- 172.21.15.94:0/2561779028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3801f8000 0x7fe3801f8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe385000640 1 -- 172.21.15.94:0/2561779028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe368003fe0 msgr2=0x7fe380144570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe385000640 1 --2- 172.21.15.94:0/2561779028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe368003fe0 0x7fe380144570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe385000640 1 -- 172.21.15.94:0/2561779028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3800ca0d0 con 0x7fe3801f8cf0 2024-09-10T23:36:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe385000640 1 --2- 172.21.15.94:0/2561779028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3801f8cf0 0x7fe3800cb800 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009b30 tx=0x7fe36c0acd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:09.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe37d600640 1 -- 172.21.15.94:0/2561779028 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0bd020 con 0x7fe3801f8cf0 2024-09-10T23:36:09.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe37d600640 1 -- 172.21.15.94:0/2561779028 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe36c005020 con 0x7fe3801f8cf0 2024-09-10T23:36:09.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe37d600640 1 -- 172.21.15.94:0/2561779028 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0b1040 con 0x7fe3801f8cf0 2024-09-10T23:36:09.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe388400640 1 -- 172.21.15.94:0/2561779028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3801e4ad0 con 0x7fe3801f8cf0 2024-09-10T23:36:09.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe388400640 1 -- 172.21.15.94:0/2561779028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3801e5090 con 0x7fe3801f8cf0 2024-09-10T23:36:09.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.026+0000 7fe388400640 1 -- 172.21.15.94:0/2561779028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe380072610 con 0x7fe3801f8cf0 2024-09-10T23:36:09.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.030+0000 7fe37d600640 1 -- 172.21.15.94:0/2561779028 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7fe36c005450 con 0x7fe3801f8cf0 2024-09-10T23:36:09.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.030+0000 7fe37d600640 1 --2- 172.21.15.94:0/2561779028 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe360072940 0x7fe360074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:09.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.030+0000 7fe37d600640 1 -- 172.21.15.94:0/2561779028 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5795+0+0 (secure 0 0 0) 0x7fe36c12e7b0 con 0x7fe3801f8cf0 2024-09-10T23:36:09.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.030+0000 7fe385a00640 1 --2- 172.21.15.94:0/2561779028 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe360072940 0x7fe360074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:09.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.030+0000 7fe385a00640 1 --2- 172.21.15.94:0/2561779028 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe360072940 0x7fe360074e00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe37400ae30 tx=0x7fe37400a2f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:09.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.034+0000 7fe37d600640 1 -- 172.21.15.94:0/2561779028 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe36c016f40 con 0x7fe3801f8cf0 2024-09-10T23:36:09.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.198+0000 7fe388400640 1 -- 172.21.15.94:0/2561779028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe38006e7e0 con 0x7fe3801f8cf0 2024-09-10T23:36:09.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.942+0000 7fe37d600640 1 -- 172.21.15.94:0/2561779028 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) ==== 181+0+0 (secure 0 0 0) 0x7fe36c0f98f0 con 0x7fe3801f8cf0 2024-09-10T23:36:09.942 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-09-10T23:36:09.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.942+0000 7fe388400640 1 -- 172.21.15.94:0/2561779028 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe360072940 msgr2=0x7fe360074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:09.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.942+0000 7fe388400640 1 --2- 172.21.15.94:0/2561779028 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe360072940 0x7fe360074e00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe37400ae30 tx=0x7fe37400a2f0 comp rx=0 tx=0).stop 2024-09-10T23:36:09.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.942+0000 7fe388400640 1 -- 172.21.15.94:0/2561779028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3801f8cf0 msgr2=0x7fe3800cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:09.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.942+0000 7fe388400640 1 --2- 172.21.15.94:0/2561779028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3801f8cf0 0x7fe3800cb800 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009b30 tx=0x7fe36c0acd20 comp rx=0 tx=0).stop 2024-09-10T23:36:09.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.942+0000 7fe388400640 1 -- 172.21.15.94:0/2561779028 shutdown_connections 2024-09-10T23:36:09.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.942+0000 7fe388400640 1 --2- 172.21.15.94:0/2561779028 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe360072940 0x7fe360074e00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:09.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.942+0000 7fe388400640 1 --2- 172.21.15.94:0/2561779028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3801f8000 0x7fe3801f8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:09.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.942+0000 7fe388400640 1 --2- 172.21.15.94:0/2561779028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3801f8cf0 0x7fe3800cb800 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:09.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.942+0000 7fe388400640 1 --2- 172.21.15.94:0/2561779028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe368003fe0 0x7fe380144570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:09.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.942+0000 7fe388400640 1 -- 172.21.15.94:0/2561779028 >> 172.21.15.94:0/2561779028 conn(0x7fe38006cdf0 msgr2=0x7fe380156a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:09.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.946+0000 7fe388400640 1 -- 172.21.15.94:0/2561779028 shutdown_connections 2024-09-10T23:36:09.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:09.946+0000 7fe388400640 1 -- 172.21.15.94:0/2561779028 wait complete. 2024-09-10T23:36:09.968 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T23:36:10.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.114+0000 7f904ce00640 1 -- 172.21.15.94:0/1069065172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9048157c60 msgr2=0x7f9048072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:10.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.114+0000 7f904ce00640 1 --2- 172.21.15.94:0/1069065172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9048157c60 0x7f9048072610 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f903800a950 tx=0x7f903801dae0 comp rx=0 tx=0).stop 2024-09-10T23:36:10.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.114+0000 7f904ce00640 1 -- 172.21.15.94:0/1069065172 shutdown_connections 2024-09-10T23:36:10.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.114+0000 7f904ce00640 1 --2- 172.21.15.94:0/1069065172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f904806f600 0x7f90480b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:10.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.114+0000 7f904ce00640 1 --2- 172.21.15.94:0/1069065172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9048157c60 0x7f9048072610 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:10.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.114+0000 7f904ce00640 1 -- 172.21.15.94:0/1069065172 >> 172.21.15.94:0/1069065172 conn(0x7f904806cdf0 msgr2=0x7f904806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:10.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.114+0000 7f904ce00640 1 -- 172.21.15.94:0/1069065172 shutdown_connections 2024-09-10T23:36:10.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.114+0000 7f904ce00640 1 -- 172.21.15.94:0/1069065172 wait complete. 2024-09-10T23:36:10.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.114+0000 7f904ce00640 1 Processor -- start 2024-09-10T23:36:10.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.114+0000 7f904ce00640 1 -- start start 2024-09-10T23:36:10.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f904ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904806f600 0x7f904814a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:10.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f904ce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9048070150 0x7f9048158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:10.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f904ce00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9048157c60 0x7f90481f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:10.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f904ce00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f904815b360 con 0x7f9048070150 2024-09-10T23:36:10.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f904ce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f904815b1e0 con 0x7f904806f600 2024-09-10T23:36:10.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f904ce00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f904815b4e0 con 0x7f9048157c60 2024-09-10T23:36:10.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f9045400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9048070150 0x7f9048158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:10.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f9045400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9048070150 0x7f9048158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59434/0 (socket says 172.21.15.94:59434) 2024-09-10T23:36:10.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f9045400640 1 -- 172.21.15.94:0/3108077491 learned_addr learned my addr 172.21.15.94:0/3108077491 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:10.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f9045e00640 1 --2- 172.21.15.94:0/3108077491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904806f600 0x7f904814a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:10.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f9046800640 1 --2- 172.21.15.94:0/3108077491 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9048157c60 0x7f90481f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:10.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f9045400640 1 -- 172.21.15.94:0/3108077491 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9048157c60 msgr2=0x7f90481f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:10.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f9045400640 1 --2- 172.21.15.94:0/3108077491 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9048157c60 0x7f90481f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:10.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f9045400640 1 -- 172.21.15.94:0/3108077491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904806f600 msgr2=0x7f904814a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:10.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f9045400640 1 --2- 172.21.15.94:0/3108077491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904806f600 0x7f904814a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:10.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f9045400640 1 -- 172.21.15.94:0/3108077491 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904814ac10 con 0x7f9048070150 2024-09-10T23:36:10.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f9045e00640 1 --2- 172.21.15.94:0/3108077491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904806f600 0x7f904814a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:36:10.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f9046800640 1 --2- 172.21.15.94:0/3108077491 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9048157c60 0x7f90481f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:36:10.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f9045400640 1 --2- 172.21.15.94:0/3108077491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9048070150 0x7f9048158f60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f903400c480 tx=0x7f903400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:10.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f903e000640 1 -- 172.21.15.94:0/3108077491 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034016020 con 0x7f9048070150 2024-09-10T23:36:10.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f903e000640 1 -- 172.21.15.94:0/3108077491 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9034012940 con 0x7f9048070150 2024-09-10T23:36:10.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f903e000640 1 -- 172.21.15.94:0/3108077491 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034012d90 con 0x7f9048070150 2024-09-10T23:36:10.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f904ce00640 1 -- 172.21.15.94:0/3108077491 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90481584a0 con 0x7f9048070150 2024-09-10T23:36:10.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f904ce00640 1 -- 172.21.15.94:0/3108077491 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9048158760 con 0x7f9048070150 2024-09-10T23:36:10.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f904ce00640 1 -- 172.21.15.94:0/3108077491 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90480b35c0 con 0x7f9048070150 2024-09-10T23:36:10.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.118+0000 7f903e000640 1 -- 172.21.15.94:0/3108077491 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f903401a3b0 con 0x7f9048070150 2024-09-10T23:36:10.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.122+0000 7f903e000640 1 --2- 172.21.15.94:0/3108077491 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9028072940 0x7f9028074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:10.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.122+0000 7f903e000640 1 -- 172.21.15.94:0/3108077491 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5357+0+0 (secure 0 0 0) 0x7f9034093720 con 0x7f9048070150 2024-09-10T23:36:10.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.122+0000 7f9045e00640 1 --2- 172.21.15.94:0/3108077491 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9028072940 0x7f9028074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:10.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.122+0000 7f9045e00640 1 --2- 172.21.15.94:0/3108077491 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9028072940 0x7f9028074e00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f90380b0000 tx=0x7f903801c320 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:10.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.126+0000 7f903e000640 1 -- 172.21.15.94:0/3108077491 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9034020030 con 0x7f9048070150 2024-09-10T23:36:10.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.290+0000 7f904ce00640 1 -- 172.21.15.94:0/3108077491 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f904806e7e0 con 0x7f9048070150 2024-09-10T23:36:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.942+0000 7f903e000640 1 -- 172.21.15.94:0/3108077491 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) ==== 169+0+0 (secure 0 0 0) 0x7f903405ea90 con 0x7f9048070150 2024-09-10T23:36:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-09-10T23:36:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.946+0000 7f904ce00640 1 -- 172.21.15.94:0/3108077491 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9028072940 msgr2=0x7f9028074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.946+0000 7f904ce00640 1 --2- 172.21.15.94:0/3108077491 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9028072940 0x7f9028074e00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f90380b0000 tx=0x7f903801c320 comp rx=0 tx=0).stop 2024-09-10T23:36:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.946+0000 7f904ce00640 1 -- 172.21.15.94:0/3108077491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9048070150 msgr2=0x7f9048158f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.946+0000 7f904ce00640 1 --2- 172.21.15.94:0/3108077491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9048070150 0x7f9048158f60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f903400c480 tx=0x7f903400c940 comp rx=0 tx=0).stop 2024-09-10T23:36:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.946+0000 7f904ce00640 1 -- 172.21.15.94:0/3108077491 shutdown_connections 2024-09-10T23:36:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.946+0000 7f904ce00640 1 --2- 172.21.15.94:0/3108077491 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f9028072940 0x7f9028074e00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.946+0000 7f904ce00640 1 --2- 172.21.15.94:0/3108077491 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9048157c60 0x7f90481f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.946+0000 7f904ce00640 1 --2- 172.21.15.94:0/3108077491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9048070150 0x7f9048158f60 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.946+0000 7f904ce00640 1 --2- 172.21.15.94:0/3108077491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904806f600 0x7f904814a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.946+0000 7f904ce00640 1 -- 172.21.15.94:0/3108077491 >> 172.21.15.94:0/3108077491 conn(0x7f904806cdf0 msgr2=0x7f9048164020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.946+0000 7f904ce00640 1 -- 172.21.15.94:0/3108077491 shutdown_connections 2024-09-10T23:36:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:10.946+0000 7f904ce00640 1 -- 172.21.15.94:0/3108077491 wait complete. 2024-09-10T23:36:10.977 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T23:36:11.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.126+0000 7ffa11400640 1 -- 172.21.15.94:0/4083029147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c0c64f0 msgr2=0x7ffa0c0ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.126+0000 7ffa11400640 1 --2- 172.21.15.94:0/4083029147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c0c64f0 0x7ffa0c0ce660 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c069670 tx=0x7ff9f401db30 comp rx=0 tx=0).stop 2024-09-10T23:36:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.126+0000 7ffa11400640 1 -- 172.21.15.94:0/4083029147 shutdown_connections 2024-09-10T23:36:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.126+0000 7ffa11400640 1 --2- 172.21.15.94:0/4083029147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c0c64f0 0x7ffa0c0ce660 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.126+0000 7ffa11400640 1 --2- 172.21.15.94:0/4083029147 >> v2:172.21.15.181:3300/0 conn(0x7ffa0c0ce280 0x7ffa0c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.126+0000 7ffa11400640 1 -- 172.21.15.94:0/4083029147 >> 172.21.15.94:0/4083029147 conn(0x7ffa0c0cfaf0 msgr2=0x7ffa0c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.126+0000 7ffa11400640 1 -- 172.21.15.94:0/4083029147 shutdown_connections 2024-09-10T23:36:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.126+0000 7ffa11400640 1 -- 172.21.15.94:0/4083029147 wait complete. 2024-09-10T23:36:11.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa11400640 1 Processor -- start 2024-09-10T23:36:11.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa11400640 1 -- start start 2024-09-10T23:36:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa11400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c06e7c0 0x7ffa0c1fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa11400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c0c64f0 0x7ffa0c14ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa11400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa0c0ce280 0x7ffa0c157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa11400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c167680 con 0x7ffa0c06e7c0 2024-09-10T23:36:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa11400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0c167500 con 0x7ffa0c0c64f0 2024-09-10T23:36:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa11400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0c167800 con 0x7ffa0c0ce280 2024-09-10T23:36:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa0a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c06e7c0 0x7ffa0c1fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa0a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c06e7c0 0x7ffa0c1fbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59460/0 (socket says 172.21.15.94:59460) 2024-09-10T23:36:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa0a800640 1 -- 172.21.15.94:0/130913899 learned_addr learned my addr 172.21.15.94:0/130913899 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa01e00640 1 --2- 172.21.15.94:0/130913899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c0c64f0 0x7ffa0c14ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa0b200640 1 --2- 172.21.15.94:0/130913899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa0c0ce280 0x7ffa0c157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa0a800640 1 -- 172.21.15.94:0/130913899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa0c0ce280 msgr2=0x7ffa0c157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa0a800640 1 --2- 172.21.15.94:0/130913899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa0c0ce280 0x7ffa0c157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa0a800640 1 -- 172.21.15.94:0/130913899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c0c64f0 msgr2=0x7ffa0c14ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa0a800640 1 --2- 172.21.15.94:0/130913899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c0c64f0 0x7ffa0c14ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa0a800640 1 -- 172.21.15.94:0/130913899 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0c1f8340 con 0x7ffa0c06e7c0 2024-09-10T23:36:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa0a800640 1 --2- 172.21.15.94:0/130913899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c06e7c0 0x7ffa0c1fbe40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0058d0 tx=0x7ff9fc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa03400640 1 -- 172.21.15.94:0/130913899 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc017070 con 0x7ffa0c06e7c0 2024-09-10T23:36:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa03400640 1 -- 172.21.15.94:0/130913899 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9fc009d20 con 0x7ffa0c06e7c0 2024-09-10T23:36:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa11400640 1 -- 172.21.15.94:0/130913899 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa0c1f8620 con 0x7ffa0c06e7c0 2024-09-10T23:36:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa03400640 1 -- 172.21.15.94:0/130913899 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0106c0 con 0x7ffa0c06e7c0 2024-09-10T23:36:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa11400640 1 -- 172.21.15.94:0/130913899 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa0c1f8be0 con 0x7ffa0c06e7c0 2024-09-10T23:36:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.130+0000 7ffa11400640 1 -- 172.21.15.94:0/130913899 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa0c0ce660 con 0x7ffa0c06e7c0 2024-09-10T23:36:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.134+0000 7ffa03400640 1 -- 172.21.15.94:0/130913899 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7ff9fc010860 con 0x7ffa0c06e7c0 2024-09-10T23:36:11.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.134+0000 7ffa03400640 1 --2- 172.21.15.94:0/130913899 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff9e8072940 0x7ff9e8074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:11.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.134+0000 7ffa03400640 1 -- 172.21.15.94:0/130913899 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7ff9fc093b10 con 0x7ffa0c06e7c0 2024-09-10T23:36:11.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.134+0000 7ffa01e00640 1 --2- 172.21.15.94:0/130913899 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff9e8072940 0x7ff9e8074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:11.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.134+0000 7ffa01e00640 1 --2- 172.21.15.94:0/130913899 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff9e8072940 0x7ff9e8074e00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40b0000 tx=0x7ff9f401c370 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:11.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.138+0000 7ffa03400640 1 -- 172.21.15.94:0/130913899 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff9fc05e7c0 con 0x7ffa0c06e7c0 2024-09-10T23:36:11.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.302+0000 7ffa11400640 1 -- 172.21.15.94:0/130913899 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7ffa0c0d0850 con 0x7ffa0c06e7c0 2024-09-10T23:36:11.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.306+0000 7ffa03400640 1 -- 172.21.15.94:0/130913899 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7ff9fc0631f0 con 0x7ffa0c06e7c0 2024-09-10T23:36:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.310+0000 7ffa11400640 1 -- 172.21.15.94:0/130913899 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff9e8072940 msgr2=0x7ff9e8074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.310+0000 7ffa11400640 1 --2- 172.21.15.94:0/130913899 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff9e8072940 0x7ff9e8074e00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40b0000 tx=0x7ff9f401c370 comp rx=0 tx=0).stop 2024-09-10T23:36:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.310+0000 7ffa11400640 1 -- 172.21.15.94:0/130913899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c06e7c0 msgr2=0x7ffa0c1fbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.310+0000 7ffa11400640 1 --2- 172.21.15.94:0/130913899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c06e7c0 0x7ffa0c1fbe40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0058d0 tx=0x7ff9fc00dde0 comp rx=0 tx=0).stop 2024-09-10T23:36:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.310+0000 7ffa11400640 1 -- 172.21.15.94:0/130913899 shutdown_connections 2024-09-10T23:36:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.310+0000 7ffa11400640 1 --2- 172.21.15.94:0/130913899 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff9e8072940 0x7ff9e8074e00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.310+0000 7ffa11400640 1 --2- 172.21.15.94:0/130913899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa0c0c64f0 0x7ffa0c14ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.310+0000 7ffa11400640 1 --2- 172.21.15.94:0/130913899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa0c0ce280 0x7ffa0c157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.310+0000 7ffa11400640 1 --2- 172.21.15.94:0/130913899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa0c06e7c0 0x7ffa0c1fbe40 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.310+0000 7ffa11400640 1 -- 172.21.15.94:0/130913899 >> 172.21.15.94:0/130913899 conn(0x7ffa0c0cfaf0 msgr2=0x7ffa0c0aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.310+0000 7ffa11400640 1 -- 172.21.15.94:0/130913899 shutdown_connections 2024-09-10T23:36:11.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.310+0000 7ffa11400640 1 -- 172.21.15.94:0/130913899 wait complete. 2024-09-10T23:36:11.332 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-10T23:36:11.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f3400640 1 -- 172.21.15.94:0/1016896534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23d0003fe0 msgr2=0x7f23d0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:11.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f3400640 1 --2- 172.21.15.94:0/1016896534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23d0003fe0 0x7f23d0004810 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009b30 tx=0x7f23d401d230 comp rx=0 tx=0).stop 2024-09-10T23:36:11.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f1400640 1 -- 172.21.15.94:0/1016896534 reap_dead start 2024-09-10T23:36:11.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f3400640 1 -- 172.21.15.94:0/1016896534 shutdown_connections 2024-09-10T23:36:11.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f3400640 1 -- 172.21.15.94:0/1016896534 >> 172.21.15.94:0/1016896534 conn(0x7f23ec0d02d0 msgr2=0x7f23ec0d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:11.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f3400640 1 -- 172.21.15.94:0/1016896534 shutdown_connections 2024-09-10T23:36:11.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f3400640 1 -- 172.21.15.94:0/1016896534 wait complete. 2024-09-10T23:36:11.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f3400640 1 Processor -- start 2024-09-10T23:36:11.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f3400640 1 -- start start 2024-09-10T23:36:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d0003fe0 0x7f23ec0c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f3400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23ec06f080 0x7f23ec06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0d0c80 0x7f23ec06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f3400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec0cfe50 con 0x7f23ec0d0c80 2024-09-10T23:36:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f3400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec0cfcd0 con 0x7f23d0003fe0 2024-09-10T23:36:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f3400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f23ec0cffd0 con 0x7f23ec06f080 2024-09-10T23:36:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0d0c80 0x7f23ec06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0d0c80 0x7f23ec06fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59486/0 (socket says 172.21.15.94:59486) 2024-09-10T23:36:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f1400640 1 -- 172.21.15.94:0/1408812287 learned_addr learned my addr 172.21.15.94:0/1408812287 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f0a00640 1 --2- 172.21.15.94:0/1408812287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d0003fe0 0x7f23ec0c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23e3e00640 1 --2- 172.21.15.94:0/1408812287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23ec06f080 0x7f23ec06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f1400640 1 -- 172.21.15.94:0/1408812287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23ec06f080 msgr2=0x7f23ec06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f1400640 1 --2- 172.21.15.94:0/1408812287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23ec06f080 0x7f23ec06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f1400640 1 -- 172.21.15.94:0/1408812287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d0003fe0 msgr2=0x7f23ec0c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f1400640 1 --2- 172.21.15.94:0/1408812287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d0003fe0 0x7f23ec0c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f1400640 1 -- 172.21.15.94:0/1408812287 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ec15a0f0 con 0x7f23ec0d0c80 2024-09-10T23:36:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23e3e00640 1 --2- 172.21.15.94:0/1408812287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23ec06f080 0x7f23ec06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.478+0000 7f23f1400640 1 --2- 172.21.15.94:0/1408812287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0d0c80 0x7f23ec06fb10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0c7170 tx=0x7f23e400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.482+0000 7f23e9600640 1 -- 172.21.15.94:0/1408812287 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4015020 con 0x7f23ec0d0c80 2024-09-10T23:36:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.482+0000 7f23e9600640 1 -- 172.21.15.94:0/1408812287 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23e400ca90 con 0x7f23ec0d0c80 2024-09-10T23:36:11.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.482+0000 7f23f3400640 1 -- 172.21.15.94:0/1408812287 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ec1558f0 con 0x7f23ec0d0c80 2024-09-10T23:36:11.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.482+0000 7f23e9600640 1 -- 172.21.15.94:0/1408812287 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4011460 con 0x7f23ec0d0c80 2024-09-10T23:36:11.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.482+0000 7f23f3400640 1 -- 172.21.15.94:0/1408812287 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ec159d80 con 0x7f23ec0d0c80 2024-09-10T23:36:11.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.482+0000 7f23f3400640 1 -- 172.21.15.94:0/1408812287 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23ec0afb10 con 0x7f23ec0d0c80 2024-09-10T23:36:11.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.482+0000 7f23e9600640 1 -- 172.21.15.94:0/1408812287 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f23e4011600 con 0x7f23ec0d0c80 2024-09-10T23:36:11.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.482+0000 7f23e9600640 1 --2- 172.21.15.94:0/1408812287 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f23c8072970 0x7f23c8074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:11.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.482+0000 7f23e9600640 1 -- 172.21.15.94:0/1408812287 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f23e4093db0 con 0x7f23ec0d0c80 2024-09-10T23:36:11.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.482+0000 7f23f0a00640 1 --2- 172.21.15.94:0/1408812287 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f23c8072970 0x7f23c8074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:11.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.486+0000 7f23f0a00640 1 --2- 172.21.15.94:0/1408812287 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f23c8072970 0x7f23c8074e30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009be0 tx=0x7f23dc0091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:11.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.486+0000 7f23e9600640 1 -- 172.21.15.94:0/1408812287 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f23e4098070 con 0x7f23ec0d0c80 2024-09-10T23:36:11.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.718+0000 7f23f3400640 1 -- 172.21.15.94:0/1408812287 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f23ec0adc50 con 0x7f23ec0d0c80 2024-09-10T23:36:11.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.954+0000 7f23e9600640 1 -- 172.21.15.94:0/1408812287 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v196) ==== 161+0+0 (secure 0 0 0) 0x7f23e401f020 con 0x7f23ec0d0c80 2024-09-10T23:36:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.958+0000 7f23f3400640 1 -- 172.21.15.94:0/1408812287 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f23c8072970 msgr2=0x7f23c8074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.958+0000 7f23f3400640 1 --2- 172.21.15.94:0/1408812287 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f23c8072970 0x7f23c8074e30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009be0 tx=0x7f23dc0091e0 comp rx=0 tx=0).stop 2024-09-10T23:36:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.958+0000 7f23f3400640 1 -- 172.21.15.94:0/1408812287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0d0c80 msgr2=0x7f23ec06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.958+0000 7f23f3400640 1 --2- 172.21.15.94:0/1408812287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0d0c80 0x7f23ec06fb10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0c7170 tx=0x7f23e400ed10 comp rx=0 tx=0).stop 2024-09-10T23:36:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.958+0000 7f23f3400640 1 -- 172.21.15.94:0/1408812287 shutdown_connections 2024-09-10T23:36:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.958+0000 7f23f3400640 1 --2- 172.21.15.94:0/1408812287 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f23c8072970 0x7f23c8074e30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.958+0000 7f23f3400640 1 --2- 172.21.15.94:0/1408812287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23ec0d0c80 0x7f23ec06fb10 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.958+0000 7f23f3400640 1 --2- 172.21.15.94:0/1408812287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23ec06f080 0x7f23ec06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.958+0000 7f23f3400640 1 --2- 172.21.15.94:0/1408812287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23d0003fe0 0x7f23ec0c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.958+0000 7f23f3400640 1 -- 172.21.15.94:0/1408812287 >> 172.21.15.94:0/1408812287 conn(0x7f23ec0d02d0 msgr2=0x7f23ec1550b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.958+0000 7f23f3400640 1 -- 172.21.15.94:0/1408812287 shutdown_connections 2024-09-10T23:36:11.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:11.958+0000 7f23f3400640 1 -- 172.21.15.94:0/1408812287 wait complete. 2024-09-10T23:36:11.980 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2024-09-10T23:36:11.980 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T23:36:11.980 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:36:12.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb487200640 1 -- 172.21.15.94:0/2338126739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb480073ff0 msgr2=0x7fb4800cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:12.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb487200640 1 --2- 172.21.15.94:0/2338126739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb480073ff0 0x7fb4800cfda0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb4800b2df0 tx=0x7fb47001c8c0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb487200640 1 -- 172.21.15.94:0/2338126739 shutdown_connections 2024-09-10T23:36:12.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb487200640 1 --2- 172.21.15.94:0/2338126739 >> v2:172.21.15.181:3300/0 conn(0x7fb48015ee80 0x7fb48006f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb487200640 1 --2- 172.21.15.94:0/2338126739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb480073ff0 0x7fb4800cfda0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb487200640 1 -- 172.21.15.94:0/2338126739 >> 172.21.15.94:0/2338126739 conn(0x7fb48006cdf0 msgr2=0x7fb48006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:12.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb487200640 1 -- 172.21.15.94:0/2338126739 shutdown_connections 2024-09-10T23:36:12.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb487200640 1 -- 172.21.15.94:0/2338126739 wait complete. 2024-09-10T23:36:12.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb487200640 1 Processor -- start 2024-09-10T23:36:12.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb487200640 1 -- start start 2024-09-10T23:36:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb487200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb480073ff0 0x7fb48015e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb487200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4800d02e0 0x7fb48014b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb487200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb48015ee80 0x7fb480163b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb487200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb480167930 con 0x7fb4800d02e0 2024-09-10T23:36:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb487200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801677b0 con 0x7fb48015ee80 2024-09-10T23:36:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb487200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb480167ab0 con 0x7fb480073ff0 2024-09-10T23:36:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb477400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4800d02e0 0x7fb48014b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb477400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4800d02e0 0x7fb48014b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59496/0 (socket says 172.21.15.94:59496) 2024-09-10T23:36:12.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb477400640 1 -- 172.21.15.94:0/3100336581 learned_addr learned my addr 172.21.15.94:0/3100336581 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb485200640 1 --2- 172.21.15.94:0/3100336581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb48015ee80 0x7fb480163b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.122+0000 7fb47fe00640 1 --2- 172.21.15.94:0/3100336581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb480073ff0 0x7fb48015e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.126+0000 7fb477400640 1 -- 172.21.15.94:0/3100336581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb480073ff0 msgr2=0x7fb48015e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.126+0000 7fb477400640 1 --2- 172.21.15.94:0/3100336581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb480073ff0 0x7fb48015e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.126+0000 7fb477400640 1 -- 172.21.15.94:0/3100336581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb48015ee80 msgr2=0x7fb480163b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.126+0000 7fb477400640 1 --2- 172.21.15.94:0/3100336581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb48015ee80 0x7fb480163b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.126+0000 7fb477400640 1 -- 172.21.15.94:0/3100336581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48014b620 con 0x7fb4800d02e0 2024-09-10T23:36:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.126+0000 7fb477400640 1 --2- 172.21.15.94:0/3100336581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4800d02e0 0x7fb48014b0e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb46800c7c0 tx=0x7fb46800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.126+0000 7fb47cc00640 1 -- 172.21.15.94:0/3100336581 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468016020 con 0x7fb4800d02e0 2024-09-10T23:36:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.126+0000 7fb47cc00640 1 -- 172.21.15.94:0/3100336581 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb46800ab40 con 0x7fb4800d02e0 2024-09-10T23:36:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.126+0000 7fb487200640 1 -- 172.21.15.94:0/3100336581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4801536a0 con 0x7fb4800d02e0 2024-09-10T23:36:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.126+0000 7fb47cc00640 1 -- 172.21.15.94:0/3100336581 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468012420 con 0x7fb4800d02e0 2024-09-10T23:36:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.126+0000 7fb487200640 1 -- 172.21.15.94:0/3100336581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb480153c20 con 0x7fb4800d02e0 2024-09-10T23:36:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.126+0000 7fb487200640 1 -- 172.21.15.94:0/3100336581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4800b1490 con 0x7fb4800d02e0 2024-09-10T23:36:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.126+0000 7fb47cc00640 1 -- 172.21.15.94:0/3100336581 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7fb46801b400 con 0x7fb4800d02e0 2024-09-10T23:36:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.126+0000 7fb47cc00640 1 --2- 172.21.15.94:0/3100336581 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb45c072940 0x7fb45c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.126+0000 7fb47cc00640 1 -- 172.21.15.94:0/3100336581 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fb4680937d0 con 0x7fb4800d02e0 2024-09-10T23:36:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.130+0000 7fb47fe00640 1 --2- 172.21.15.94:0/3100336581 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb45c072940 0x7fb45c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.130+0000 7fb47fe00640 1 --2- 172.21.15.94:0/3100336581 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb45c072940 0x7fb45c074e00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb4700af000 tx=0x7fb47001b100 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:12.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.130+0000 7fb47cc00640 1 -- 172.21.15.94:0/3100336581 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb468020030 con 0x7fb4800d02e0 2024-09-10T23:36:12.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.298+0000 7fb487200640 1 -- 172.21.15.94:0/3100336581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4800cfda0 con 0x7fb4800d02e0 2024-09-10T23:36:12.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.302+0000 7fb47cc00640 1 -- 172.21.15.94:0/3100336581 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+13020 (secure 0 0 0) 0x7fb46805e640 con 0x7fb4800d02e0 2024-09-10T23:36:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:12.302 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":61,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:36:10.943489+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6835/2905667261":"2024-09-11T23:36:07.805365+0000","172.21.15.94:6833/2905667261":"2024-09-11T23:36:07.805365+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:36:12.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.302+0000 7fb487200640 1 -- 172.21.15.94:0/3100336581 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb45c072940 msgr2=0x7fb45c074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:12.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.302+0000 7fb487200640 1 --2- 172.21.15.94:0/3100336581 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb45c072940 0x7fb45c074e00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb4700af000 tx=0x7fb47001b100 comp rx=0 tx=0).stop 2024-09-10T23:36:12.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.302+0000 7fb487200640 1 -- 172.21.15.94:0/3100336581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4800d02e0 msgr2=0x7fb48014b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:12.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.302+0000 7fb487200640 1 --2- 172.21.15.94:0/3100336581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4800d02e0 0x7fb48014b0e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb46800c7c0 tx=0x7fb46800cc80 comp rx=0 tx=0).stop 2024-09-10T23:36:12.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.302+0000 7fb487200640 1 -- 172.21.15.94:0/3100336581 shutdown_connections 2024-09-10T23:36:12.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.302+0000 7fb487200640 1 --2- 172.21.15.94:0/3100336581 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb45c072940 0x7fb45c074e00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.302+0000 7fb487200640 1 --2- 172.21.15.94:0/3100336581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb48015ee80 0x7fb480163b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.302+0000 7fb487200640 1 --2- 172.21.15.94:0/3100336581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4800d02e0 0x7fb48014b0e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.302+0000 7fb487200640 1 --2- 172.21.15.94:0/3100336581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb480073ff0 0x7fb48015e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.302+0000 7fb487200640 1 -- 172.21.15.94:0/3100336581 >> 172.21.15.94:0/3100336581 conn(0x7fb48006cdf0 msgr2=0x7fb4800af660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:12.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.302+0000 7fb487200640 1 -- 172.21.15.94:0/3100336581 shutdown_connections 2024-09-10T23:36:12.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.302+0000 7fb487200640 1 -- 172.21.15.94:0/3100336581 wait complete. 2024-09-10T23:36:12.323 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T23:32:31.372521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T23:36:12.324 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T23:36:12.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.466+0000 7f1a6d000640 1 -- 172.21.15.94:0/2009793480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a680d2780 msgr2=0x7f1a680d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:12.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.466+0000 7f1a6d000640 1 --2- 172.21.15.94:0/2009793480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a680d2780 0x7f1a680d03f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009840 tx=0x7f1a5001c790 comp rx=0 tx=0).stop 2024-09-10T23:36:12.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.466+0000 7f1a6d000640 1 -- 172.21.15.94:0/2009793480 shutdown_connections 2024-09-10T23:36:12.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.466+0000 7f1a6d000640 1 --2- 172.21.15.94:0/2009793480 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a680c5e50 0x7f1a680c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.466+0000 7f1a6d000640 1 --2- 172.21.15.94:0/2009793480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a680d2780 0x7f1a680d03f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.466+0000 7f1a6d000640 1 -- 172.21.15.94:0/2009793480 >> 172.21.15.94:0/2009793480 conn(0x7f1a680d1c60 msgr2=0x7f1a680d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:12.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.466+0000 7f1a6d000640 1 -- 172.21.15.94:0/2009793480 shutdown_connections 2024-09-10T23:36:12.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.466+0000 7f1a6d000640 1 -- 172.21.15.94:0/2009793480 wait complete. 2024-09-10T23:36:12.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a6d000640 1 Processor -- start 2024-09-10T23:36:12.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a6d000640 1 -- start start 2024-09-10T23:36:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a6d000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a680c5e50 0x7f1a68144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a6d000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a680d2780 0x7f1a6815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a6d000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a6815bf60 0x7f1a6815ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a6d000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1a68168f30 con 0x7f1a680c5e50 2024-09-10T23:36:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a6d000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1a68168db0 con 0x7f1a6815bf60 2024-09-10T23:36:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a6d000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1a681690b0 con 0x7f1a680d2780 2024-09-10T23:36:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a65e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a680c5e50 0x7f1a68144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a65e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a680c5e50 0x7f1a68144f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59498/0 (socket says 172.21.15.94:59498) 2024-09-10T23:36:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a65e00640 1 -- 172.21.15.94:0/3199034289 learned_addr learned my addr 172.21.15.94:0/3199034289 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a66800640 1 --2- 172.21.15.94:0/3199034289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a6815bf60 0x7f1a6815ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a5d400640 1 --2- 172.21.15.94:0/3199034289 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a680d2780 0x7f1a6815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a65e00640 1 -- 172.21.15.94:0/3199034289 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a680d2780 msgr2=0x7f1a6815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a65e00640 1 --2- 172.21.15.94:0/3199034289 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a680d2780 0x7f1a6815a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a65e00640 1 -- 172.21.15.94:0/3199034289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a6815bf60 msgr2=0x7f1a6815ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a65e00640 1 --2- 172.21.15.94:0/3199034289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a6815bf60 0x7f1a6815ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a65e00640 1 -- 172.21.15.94:0/3199034289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6806e7e0 con 0x7f1a680c5e50 2024-09-10T23:36:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a5d400640 1 --2- 172.21.15.94:0/3199034289 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a680d2780 0x7f1a6815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a65e00640 1 --2- 172.21.15.94:0/3199034289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a680c5e50 0x7f1a68144f90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800c3f0 tx=0x7f1a5800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a5ea00640 1 -- 172.21.15.94:0/3199034289 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58011790 con 0x7f1a680c5e50 2024-09-10T23:36:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a5ea00640 1 -- 172.21.15.94:0/3199034289 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a58017070 con 0x7f1a680c5e50 2024-09-10T23:36:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a5ea00640 1 -- 172.21.15.94:0/3199034289 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58011ce0 con 0x7f1a680c5e50 2024-09-10T23:36:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a6d000640 1 -- 172.21.15.94:0/3199034289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6806eac0 con 0x7f1a680c5e50 2024-09-10T23:36:12.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a6d000640 1 -- 172.21.15.94:0/3199034289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6806f080 con 0x7f1a680c5e50 2024-09-10T23:36:12.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.470+0000 7f1a6d000640 1 -- 172.21.15.94:0/3199034289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a680c62e0 con 0x7f1a680c5e50 2024-09-10T23:36:12.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.474+0000 7f1a5ea00640 1 -- 172.21.15.94:0/3199034289 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f1a5801c460 con 0x7f1a680c5e50 2024-09-10T23:36:12.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.474+0000 7f1a5ea00640 1 --2- 172.21.15.94:0/3199034289 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a44072940 0x7f1a44074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:12.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.474+0000 7f1a5ea00640 1 -- 172.21.15.94:0/3199034289 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f1a580932e0 con 0x7f1a680c5e50 2024-09-10T23:36:12.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.474+0000 7f1a5d400640 1 --2- 172.21.15.94:0/3199034289 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a44072940 0x7f1a44074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:12.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.474+0000 7f1a5d400640 1 --2- 172.21.15.94:0/3199034289 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a44072940 0x7f1a44074e00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1a500af000 tx=0x7f1a5001afd0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:12.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.478+0000 7f1a5ea00640 1 -- 172.21.15.94:0/3199034289 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1a5805e7f0 con 0x7f1a680c5e50 2024-09-10T23:36:12.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.642+0000 7f1a6d000640 1 -- 172.21.15.94:0/3199034289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1a68152ce0 con 0x7f1a680c5e50 2024-09-10T23:36:12.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.642+0000 7f1a5ea00640 1 -- 172.21.15.94:0/3199034289 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) ==== 93+0+10 (secure 0 0 0) 0x7f1a580650f0 con 0x7f1a680c5e50 2024-09-10T23:36:12.646 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-09-10T23:36:12.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.646+0000 7f1a6d000640 1 -- 172.21.15.94:0/3199034289 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a44072940 msgr2=0x7f1a44074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:12.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.646+0000 7f1a6d000640 1 --2- 172.21.15.94:0/3199034289 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a44072940 0x7f1a44074e00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1a500af000 tx=0x7f1a5001afd0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.646+0000 7f1a6d000640 1 -- 172.21.15.94:0/3199034289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a680c5e50 msgr2=0x7f1a68144f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:12.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.646+0000 7f1a6d000640 1 --2- 172.21.15.94:0/3199034289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a680c5e50 0x7f1a68144f90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800c3f0 tx=0x7f1a5800c8b0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.646+0000 7f1a6d000640 1 -- 172.21.15.94:0/3199034289 shutdown_connections 2024-09-10T23:36:12.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.646+0000 7f1a6d000640 1 --2- 172.21.15.94:0/3199034289 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f1a44072940 0x7f1a44074e00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.646+0000 7f1a6d000640 1 --2- 172.21.15.94:0/3199034289 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a6815bf60 0x7f1a6815ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.646+0000 7f1a6d000640 1 --2- 172.21.15.94:0/3199034289 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a680d2780 0x7f1a6815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.646+0000 7f1a6d000640 1 --2- 172.21.15.94:0/3199034289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a680c5e50 0x7f1a68144f90 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.646+0000 7f1a6d000640 1 -- 172.21.15.94:0/3199034289 >> 172.21.15.94:0/3199034289 conn(0x7f1a680d1c60 msgr2=0x7f1a681587d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:12.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.646+0000 7f1a6d000640 1 -- 172.21.15.94:0/3199034289 shutdown_connections 2024-09-10T23:36:12.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.646+0000 7f1a6d000640 1 -- 172.21.15.94:0/3199034289 wait complete. 2024-09-10T23:36:12.666 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-10T23:36:12.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.806+0000 7f362d400640 1 -- 172.21.15.94:0/643633143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36280d03c0 msgr2=0x7f36280cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:12.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.806+0000 7f362d400640 1 --2- 172.21.15.94:0/643633143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36280d03c0 0x7f36280cddb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f36280d0890 tx=0x7f362001ccc0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.806+0000 7f362d400640 1 -- 172.21.15.94:0/643633143 shutdown_connections 2024-09-10T23:36:12.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.806+0000 7f362d400640 1 --2- 172.21.15.94:0/643633143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36280d03c0 0x7f36280cddb0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.806+0000 7f362d400640 1 --2- 172.21.15.94:0/643633143 >> v2:172.21.15.181:3300/0 conn(0x7f362815ba50 0x7f36280b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.806+0000 7f362d400640 1 -- 172.21.15.94:0/643633143 >> 172.21.15.94:0/643633143 conn(0x7f362806cdf0 msgr2=0x7f362806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:12.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.806+0000 7f362d400640 1 -- 172.21.15.94:0/643633143 shutdown_connections 2024-09-10T23:36:12.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.806+0000 7f362d400640 1 -- 172.21.15.94:0/643633143 wait complete. 2024-09-10T23:36:12.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f362d400640 1 Processor -- start 2024-09-10T23:36:12.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f362d400640 1 -- start start 2024-09-10T23:36:12.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f362d400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36280b4c40 0x7f362814ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:12.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f362d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36280d03c0 0x7f36281490e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:12.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f362d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f362815ba50 0x7f36281f7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:12.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f362d400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f36281689d0 con 0x7f362815ba50 2024-09-10T23:36:12.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f362d400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3628168850 con 0x7f36280d03c0 2024-09-10T23:36:12.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f362d400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3628168b50 con 0x7f36280b4c40 2024-09-10T23:36:12.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f3627200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f362815ba50 0x7f36281f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:12.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f3626800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36280b4c40 0x7f362814ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:12.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f3625e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36280d03c0 0x7f36281490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:12.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f3627200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f362815ba50 0x7f36281f7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59522/0 (socket says 172.21.15.94:59522) 2024-09-10T23:36:12.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f3627200640 1 -- 172.21.15.94:0/1366468703 learned_addr learned my addr 172.21.15.94:0/1366468703 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:12.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f3626800640 1 --2- 172.21.15.94:0/1366468703 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36280b4c40 0x7f362814ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:58988/0 (socket says 172.21.15.94:58988) 2024-09-10T23:36:12.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f3627200640 1 -- 172.21.15.94:0/1366468703 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36280b4c40 msgr2=0x7f362814ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:12.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f3627200640 1 --2- 172.21.15.94:0/1366468703 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36280b4c40 0x7f362814ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f3627200640 1 -- 172.21.15.94:0/1366468703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36280d03c0 msgr2=0x7f36281490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:12.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f3627200640 1 --2- 172.21.15.94:0/1366468703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36280d03c0 0x7f36281490e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:12.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f3627200640 1 -- 172.21.15.94:0/1366468703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3628149b70 con 0x7f362815ba50 2024-09-10T23:36:12.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f3626800640 1 --2- 172.21.15.94:0/1366468703 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36280b4c40 0x7f362814ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:36:12.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f3625e00640 1 --2- 172.21.15.94:0/1366468703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36280d03c0 0x7f36281490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:36:12.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f3627200640 1 --2- 172.21.15.94:0/1366468703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f362815ba50 0x7f36281f7460 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f36200198d0 tx=0x7f3620009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:12.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f361ea00640 1 -- 172.21.15.94:0/1366468703 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36200bb070 con 0x7f362815ba50 2024-09-10T23:36:12.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f361ea00640 1 -- 172.21.15.94:0/1366468703 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3620004d40 con 0x7f362815ba50 2024-09-10T23:36:12.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f361ea00640 1 -- 172.21.15.94:0/1366468703 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36200b2040 con 0x7f362815ba50 2024-09-10T23:36:12.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f362d400640 1 -- 172.21.15.94:0/1366468703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3628149620 con 0x7f362815ba50 2024-09-10T23:36:12.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f362d400640 1 -- 172.21.15.94:0/1366468703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3628150890 con 0x7f362815ba50 2024-09-10T23:36:12.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.810+0000 7f362d400640 1 -- 172.21.15.94:0/1366468703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36280cddb0 con 0x7f362815ba50 2024-09-10T23:36:12.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.814+0000 7f361ea00640 1 -- 172.21.15.94:0/1366468703 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f36200b5030 con 0x7f362815ba50 2024-09-10T23:36:12.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.814+0000 7f361ea00640 1 --2- 172.21.15.94:0/1366468703 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3608072970 0x7f3608074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:12.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.814+0000 7f361ea00640 1 -- 172.21.15.94:0/1366468703 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f362012d110 con 0x7f362815ba50 2024-09-10T23:36:12.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.814+0000 7f3626800640 1 --2- 172.21.15.94:0/1366468703 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3608072970 0x7f3608074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:12.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.814+0000 7f3626800640 1 --2- 172.21.15.94:0/1366468703 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3608072970 0x7f3608074e30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3618009f30 tx=0x7f36180091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:12.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:12.818+0000 7f361ea00640 1 -- 172.21.15.94:0/1366468703 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3620016970 con 0x7f362815ba50 2024-09-10T23:36:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:13.042+0000 7f362d400640 1 -- 172.21.15.94:0/1366468703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7f362815ae80 con 0x7f362815ba50 2024-09-10T23:36:13.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:13.966+0000 7f361ea00640 1 -- 172.21.15.94:0/1366468703 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v198) ==== 134+0+0 (secure 0 0 0) 0x7f36200f7fd0 con 0x7f362815ba50 2024-09-10T23:36:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:13.966+0000 7f362d400640 1 -- 172.21.15.94:0/1366468703 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3608072970 msgr2=0x7f3608074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:13.966+0000 7f362d400640 1 --2- 172.21.15.94:0/1366468703 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3608072970 0x7f3608074e30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3618009f30 tx=0x7f36180091e0 comp rx=0 tx=0).stop 2024-09-10T23:36:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:13.966+0000 7f362d400640 1 -- 172.21.15.94:0/1366468703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f362815ba50 msgr2=0x7f36281f7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:13.966+0000 7f362d400640 1 --2- 172.21.15.94:0/1366468703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f362815ba50 0x7f36281f7460 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f36200198d0 tx=0x7f3620009300 comp rx=0 tx=0).stop 2024-09-10T23:36:13.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:13.966+0000 7f362d400640 1 -- 172.21.15.94:0/1366468703 shutdown_connections 2024-09-10T23:36:13.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:13.966+0000 7f362d400640 1 --2- 172.21.15.94:0/1366468703 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3608072970 0x7f3608074e30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:13.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:13.966+0000 7f362d400640 1 --2- 172.21.15.94:0/1366468703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36280d03c0 0x7f36281490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:13.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:13.966+0000 7f362d400640 1 --2- 172.21.15.94:0/1366468703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f362815ba50 0x7f36281f7460 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:13.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:13.966+0000 7f362d400640 1 --2- 172.21.15.94:0/1366468703 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36280b4c40 0x7f362814ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:13.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:13.966+0000 7f362d400640 1 -- 172.21.15.94:0/1366468703 >> 172.21.15.94:0/1366468703 conn(0x7f362806cdf0 msgr2=0x7f3628075950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:13.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:13.966+0000 7f362d400640 1 -- 172.21.15.94:0/1366468703 shutdown_connections 2024-09-10T23:36:13.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:13.966+0000 7f362d400640 1 -- 172.21.15.94:0/1366468703 wait complete. 2024-09-10T23:36:13.993 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T23:36:14.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f203b200640 1 -- 172.21.15.94:0/3880371187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f203415ecb0 msgr2=0x7f203406f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:14.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f203b200640 1 --2- 172.21.15.94:0/3880371187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f203415ecb0 0x7f203406f4b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f203406f240 tx=0x7f202c01c7d0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f203b200640 1 -- 172.21.15.94:0/3880371187 shutdown_connections 2024-09-10T23:36:14.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f203b200640 1 --2- 172.21.15.94:0/3880371187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f203415ecb0 0x7f203406f4b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f203b200640 1 --2- 172.21.15.94:0/3880371187 >> v2:172.21.15.181:3300/0 conn(0x7f20340d04b0 0x7f203415e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f203b200640 1 -- 172.21.15.94:0/3880371187 >> 172.21.15.94:0/3880371187 conn(0x7f203406cdf0 msgr2=0x7f203406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:14.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f203b200640 1 -- 172.21.15.94:0/3880371187 shutdown_connections 2024-09-10T23:36:14.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f203b200640 1 -- 172.21.15.94:0/3880371187 wait complete. 2024-09-10T23:36:14.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f203b200640 1 Processor -- start 2024-09-10T23:36:14.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f203b200640 1 -- start start 2024-09-10T23:36:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f203b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2034073ff0 0x7f2034152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f203b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20340d04b0 0x7f203414fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f203b200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f203415ecb0 0x7f2034160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f203b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2034167930 con 0x7f20340d04b0 2024-09-10T23:36:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f203b200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f20341677b0 con 0x7f2034073ff0 2024-09-10T23:36:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f203b200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2034167ab0 con 0x7f203415ecb0 2024-09-10T23:36:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f202b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20340d04b0 0x7f203414fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f202b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20340d04b0 0x7f203414fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54904/0 (socket says 172.21.15.94:54904) 2024-09-10T23:36:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f202b400640 1 -- 172.21.15.94:0/4079732269 learned_addr learned my addr 172.21.15.94:0/4079732269 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f2033e00640 1 --2- 172.21.15.94:0/4079732269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2034073ff0 0x7f2034152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:14.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f2039200640 1 --2- 172.21.15.94:0/4079732269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f203415ecb0 0x7f2034160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:14.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f202b400640 1 -- 172.21.15.94:0/4079732269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f203415ecb0 msgr2=0x7f2034160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:14.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f202b400640 1 --2- 172.21.15.94:0/4079732269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f203415ecb0 0x7f2034160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f202b400640 1 -- 172.21.15.94:0/4079732269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2034073ff0 msgr2=0x7f2034152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:14.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f202b400640 1 --2- 172.21.15.94:0/4079732269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2034073ff0 0x7f2034152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.138+0000 7f202b400640 1 -- 172.21.15.94:0/4079732269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2034152850 con 0x7f20340d04b0 2024-09-10T23:36:14.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.142+0000 7f202b400640 1 --2- 172.21.15.94:0/4079732269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20340d04b0 0x7f203414fbe0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f201c00d980 tx=0x7f201c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:14.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.142+0000 7f2030c00640 1 -- 172.21.15.94:0/4079732269 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c017070 con 0x7f20340d04b0 2024-09-10T23:36:14.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.142+0000 7f2030c00640 1 -- 172.21.15.94:0/4079732269 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f201c009d20 con 0x7f20340d04b0 2024-09-10T23:36:14.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.142+0000 7f203b200640 1 -- 172.21.15.94:0/4079732269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2034152b30 con 0x7f20340d04b0 2024-09-10T23:36:14.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.142+0000 7f2030c00640 1 -- 172.21.15.94:0/4079732269 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c0106c0 con 0x7f20340d04b0 2024-09-10T23:36:14.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.142+0000 7f203b200640 1 -- 172.21.15.94:0/4079732269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2034153880 con 0x7f20340d04b0 2024-09-10T23:36:14.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.142+0000 7f203b200640 1 -- 172.21.15.94:0/4079732269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f203415e770 con 0x7f20340d04b0 2024-09-10T23:36:14.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.142+0000 7f2030c00640 1 -- 172.21.15.94:0/4079732269 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f201c010860 con 0x7f20340d04b0 2024-09-10T23:36:14.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.142+0000 7f2030c00640 1 --2- 172.21.15.94:0/4079732269 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2010072970 0x7f2010074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:14.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.142+0000 7f2030c00640 1 -- 172.21.15.94:0/4079732269 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f201c093920 con 0x7f20340d04b0 2024-09-10T23:36:14.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.146+0000 7f2033e00640 1 --2- 172.21.15.94:0/4079732269 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2010072970 0x7f2010074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:14.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.146+0000 7f2033e00640 1 --2- 172.21.15.94:0/4079732269 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2010072970 0x7f2010074e30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f203415faa0 tx=0x7f2024001aa0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:14.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.146+0000 7f2030c00640 1 -- 172.21.15.94:0/4079732269 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f201c05e6e0 con 0x7f20340d04b0 2024-09-10T23:36:14.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.314+0000 7f203b200640 1 -- 172.21.15.94:0/4079732269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f2034160020 con 0x7f20340d04b0 2024-09-10T23:36:14.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.314+0000 7f2030c00640 1 -- 172.21.15.94:0/4079732269 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+1998 (secure 0 0 0) 0x7f201c063110 con 0x7f20340d04b0 2024-09-10T23:36:14.317 INFO:teuthology.orchestra.run.smithi094.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T23:36:14.317 INFO:teuthology.orchestra.run.smithi094.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T23:36:14.317 INFO:teuthology.orchestra.run.smithi094.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T23:36:14.317 INFO:teuthology.orchestra.run.smithi094.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T23:36:14.317 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3944.074506 2024-09-10T23:36:14.317 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3865.464527 2024-09-10T23:36:14.317 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3898.861024 2024-09-10T23:36:14.317 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4010.048529 2024-09-10T23:36:14.317 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4650.562436 2024-09-10T23:36:14.318 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4724.018776 2024-09-10T23:36:14.318 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5255.753663 2024-09-10T23:36:14.318 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4718.009443 2024-09-10T23:36:14.318 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds 20/20 2024-09-10T23:36:14.318 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T23:36:14.318 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_ms 2/2 2024-09-10T23:36:14.318 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_frag true 2024-09-10T23:36:14.318 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_scatterstat true 2024-09-10T23:36:14.318 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T23:36:14.318 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_verify_scatter true 2024-09-10T23:36:14.318 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T23:36:14.318 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T23:36:14.318 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T23:36:14.318 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced client_mount_timeout 600 2024-09-10T23:36:14.318 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_client 20/20 2024-09-10T23:36:14.319 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_ms 1/1 2024-09-10T23:36:14.319 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T23:36:14.319 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T23:36:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.318+0000 7f203b200640 1 -- 172.21.15.94:0/4079732269 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2010072970 msgr2=0x7f2010074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.318+0000 7f203b200640 1 --2- 172.21.15.94:0/4079732269 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2010072970 0x7f2010074e30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f203415faa0 tx=0x7f2024001aa0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.318+0000 7f203b200640 1 -- 172.21.15.94:0/4079732269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20340d04b0 msgr2=0x7f203414fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.318+0000 7f203b200640 1 --2- 172.21.15.94:0/4079732269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20340d04b0 0x7f203414fbe0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f201c00d980 tx=0x7f201c00de40 comp rx=0 tx=0).stop 2024-09-10T23:36:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.318+0000 7f203b200640 1 -- 172.21.15.94:0/4079732269 shutdown_connections 2024-09-10T23:36:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.318+0000 7f203b200640 1 --2- 172.21.15.94:0/4079732269 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2010072970 0x7f2010074e30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.318+0000 7f203b200640 1 --2- 172.21.15.94:0/4079732269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f203415ecb0 0x7f2034160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.318+0000 7f203b200640 1 --2- 172.21.15.94:0/4079732269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20340d04b0 0x7f203414fbe0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.318+0000 7f203b200640 1 --2- 172.21.15.94:0/4079732269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2034073ff0 0x7f2034152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.318+0000 7f203b200640 1 -- 172.21.15.94:0/4079732269 >> 172.21.15.94:0/4079732269 conn(0x7f203406cdf0 msgr2=0x7f20340b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.318+0000 7f203b200640 1 -- 172.21.15.94:0/4079732269 shutdown_connections 2024-09-10T23:36:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.318+0000 7f203b200640 1 -- 172.21.15.94:0/4079732269 wait complete. 2024-09-10T23:36:14.338 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:36:14.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.482+0000 7faa4d600640 1 -- 172.21.15.94:0/4260817230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa480cb4d0 msgr2=0x7faa4815b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.482+0000 7faa4d600640 1 --2- 172.21.15.94:0/4260817230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa480cb4d0 0x7faa4815b190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faa40009bc0 tx=0x7faa4001cba0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.482+0000 7faa4d600640 1 -- 172.21.15.94:0/4260817230 shutdown_connections 2024-09-10T23:36:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.482+0000 7faa4d600640 1 --2- 172.21.15.94:0/4260817230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa480cb4d0 0x7faa4815b190 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.482+0000 7faa4d600640 1 --2- 172.21.15.94:0/4260817230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa48158790 0x7faa4806fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.482+0000 7faa4d600640 1 -- 172.21.15.94:0/4260817230 >> 172.21.15.94:0/4260817230 conn(0x7faa480cfaf0 msgr2=0x7faa480cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.482+0000 7faa4d600640 1 -- 172.21.15.94:0/4260817230 shutdown_connections 2024-09-10T23:36:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.482+0000 7faa4d600640 1 -- 172.21.15.94:0/4260817230 wait complete. 2024-09-10T23:36:14.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.482+0000 7faa4d600640 1 Processor -- start 2024-09-10T23:36:14.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.482+0000 7faa4d600640 1 -- start start 2024-09-10T23:36:14.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa4d600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa480c49b0 0x7faa481496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:14.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa4d600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa480cb4d0 0x7faa4813b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:14.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa4d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48158790 0x7faa48072580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:14.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa4d600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa48167f60 con 0x7faa48158790 2024-09-10T23:36:14.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa4d600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faa48167de0 con 0x7faa480c49b0 2024-09-10T23:36:14.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa4d600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7faa481680e0 con 0x7faa480cb4d0 2024-09-10T23:36:14.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa47400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48158790 0x7faa48072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:14.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa47400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48158790 0x7faa48072580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54914/0 (socket says 172.21.15.94:54914) 2024-09-10T23:36:14.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa47400640 1 -- 172.21.15.94:0/756052508 learned_addr learned my addr 172.21.15.94:0/756052508 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:14.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa46a00640 1 --2- 172.21.15.94:0/756052508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa480c49b0 0x7faa481496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:14.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa46000640 1 --2- 172.21.15.94:0/756052508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa480cb4d0 0x7faa4813b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:14.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa47400640 1 -- 172.21.15.94:0/756052508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa480cb4d0 msgr2=0x7faa4813b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:14.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa47400640 1 --2- 172.21.15.94:0/756052508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa480cb4d0 0x7faa4813b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa47400640 1 -- 172.21.15.94:0/756052508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa480c49b0 msgr2=0x7faa481496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:14.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa47400640 1 --2- 172.21.15.94:0/756052508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa480c49b0 0x7faa481496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa47400640 1 -- 172.21.15.94:0/756052508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa48072ac0 con 0x7faa48158790 2024-09-10T23:36:14.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa47400640 1 --2- 172.21.15.94:0/756052508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48158790 0x7faa48072580 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7faa4001a7d0 tx=0x7faa40009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:14.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa3ea00640 1 -- 172.21.15.94:0/756052508 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa400b2040 con 0x7faa48158790 2024-09-10T23:36:14.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa3ea00640 1 -- 172.21.15.94:0/756052508 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa40004b40 con 0x7faa48158790 2024-09-10T23:36:14.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa4d600640 1 -- 172.21.15.94:0/756052508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa48072d60 con 0x7faa48158790 2024-09-10T23:36:14.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa3ea00640 1 -- 172.21.15.94:0/756052508 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40004000 con 0x7faa48158790 2024-09-10T23:36:14.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa4d600640 1 -- 172.21.15.94:0/756052508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa48073320 con 0x7faa48158790 2024-09-10T23:36:14.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa4d600640 1 -- 172.21.15.94:0/756052508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa4815b190 con 0x7faa48158790 2024-09-10T23:36:14.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.486+0000 7faa3ea00640 1 -- 172.21.15.94:0/756052508 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7faa400b5070 con 0x7faa48158790 2024-09-10T23:36:14.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.490+0000 7faa3ea00640 1 --2- 172.21.15.94:0/756052508 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faa28072970 0x7faa28074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:14.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.490+0000 7faa3ea00640 1 -- 172.21.15.94:0/756052508 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7faa4012ced0 con 0x7faa48158790 2024-09-10T23:36:14.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.490+0000 7faa46a00640 1 --2- 172.21.15.94:0/756052508 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faa28072970 0x7faa28074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:14.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.490+0000 7faa46a00640 1 --2- 172.21.15.94:0/756052508 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faa28072970 0x7faa28074e30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faa4815d290 tx=0x7faa38009260 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:14.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.494+0000 7faa3ea00640 1 -- 172.21.15.94:0/756052508 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faa40016790 con 0x7faa48158790 2024-09-10T23:36:14.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.654+0000 7faa4d600640 1 -- 172.21.15.94:0/756052508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7faa480d0850 con 0x7faa48158790 2024-09-10T23:36:14.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.658+0000 7faa3ea00640 1 -- 172.21.15.94:0/756052508 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7faa400f7d40 con 0x7faa48158790 2024-09-10T23:36:14.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.662+0000 7faa4d600640 1 -- 172.21.15.94:0/756052508 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faa28072970 msgr2=0x7faa28074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:14.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.662+0000 7faa4d600640 1 --2- 172.21.15.94:0/756052508 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faa28072970 0x7faa28074e30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faa4815d290 tx=0x7faa38009260 comp rx=0 tx=0).stop 2024-09-10T23:36:14.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.662+0000 7faa4d600640 1 -- 172.21.15.94:0/756052508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48158790 msgr2=0x7faa48072580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:14.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.662+0000 7faa4d600640 1 --2- 172.21.15.94:0/756052508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48158790 0x7faa48072580 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7faa4001a7d0 tx=0x7faa40009300 comp rx=0 tx=0).stop 2024-09-10T23:36:14.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.662+0000 7faa4d600640 1 -- 172.21.15.94:0/756052508 shutdown_connections 2024-09-10T23:36:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.662+0000 7faa4d600640 1 --2- 172.21.15.94:0/756052508 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faa28072970 0x7faa28074e30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.662+0000 7faa4d600640 1 --2- 172.21.15.94:0/756052508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa48158790 0x7faa48072580 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.662+0000 7faa4d600640 1 --2- 172.21.15.94:0/756052508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa480cb4d0 0x7faa4813b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.662+0000 7faa4d600640 1 --2- 172.21.15.94:0/756052508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa480c49b0 0x7faa481496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.662+0000 7faa4d600640 1 -- 172.21.15.94:0/756052508 >> 172.21.15.94:0/756052508 conn(0x7faa480cfaf0 msgr2=0x7faa480ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.662+0000 7faa4d600640 1 -- 172.21.15.94:0/756052508 shutdown_connections 2024-09-10T23:36:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:14.662+0000 7faa4d600640 1 -- 172.21.15.94:0/756052508 wait complete. 2024-09-10T23:36:14.684 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:36:14.684 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:36:14.760 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:36:14.852 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:36:14.852 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:36:14.852 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:14.861 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:36:14.861 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:36:14.861 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:14.870 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:36:14.870 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:36:14.870 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:14.909 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:36:14.909 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:36:14.910 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:14.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:36:14.921 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T23:36:14.922 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:15.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e9000640 1 -- 172.21.15.94:0/1395911390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91c8004d50 msgr2=0x7f91c80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:15.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e9000640 1 --2- 172.21.15.94:0/1395911390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91c8004d50 0x7f91c80051d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f91dc009b60 tx=0x7f91dc01d2b0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e2800640 1 -- 172.21.15.94:0/1395911390 reap_dead start 2024-09-10T23:36:15.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e9000640 1 -- 172.21.15.94:0/1395911390 shutdown_connections 2024-09-10T23:36:15.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e9000640 1 -- 172.21.15.94:0/1395911390 >> 172.21.15.94:0/1395911390 conn(0x7f91e406cdf0 msgr2=0x7f91e406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:15.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e9000640 1 -- 172.21.15.94:0/1395911390 shutdown_connections 2024-09-10T23:36:15.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e9000640 1 -- 172.21.15.94:0/1395911390 wait complete. 2024-09-10T23:36:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e9000640 1 Processor -- start 2024-09-10T23:36:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e9000640 1 -- start start 2024-09-10T23:36:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e9000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91c8004d50 0x7f91e41f7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:15.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e40af640 0x7f91e41e54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:15.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e9000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91e40afa80 0x7f91e41e59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:15.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e9000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91e40cff20 con 0x7f91e40af640 2024-09-10T23:36:15.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e9000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f91e40cfda0 con 0x7f91c8004d50 2024-09-10T23:36:15.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e9000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f91e40d00a0 con 0x7f91e40afa80 2024-09-10T23:36:15.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91d9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e40af640 0x7f91e41e54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:15.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91d9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e40af640 0x7f91e41e54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54938/0 (socket says 172.21.15.94:54938) 2024-09-10T23:36:15.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91d9400640 1 -- 172.21.15.94:0/2330999095 learned_addr learned my addr 172.21.15.94:0/2330999095 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:15.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e1e00640 1 --2- 172.21.15.94:0/2330999095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91c8004d50 0x7f91e41f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:15.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e2800640 1 --2- 172.21.15.94:0/2330999095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91e40afa80 0x7f91e41e59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:15.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91d9400640 1 -- 172.21.15.94:0/2330999095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91e40afa80 msgr2=0x7f91e41e59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:15.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91d9400640 1 --2- 172.21.15.94:0/2330999095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91e40afa80 0x7f91e41e59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91d9400640 1 -- 172.21.15.94:0/2330999095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91c8004d50 msgr2=0x7f91e41f7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91d9400640 1 --2- 172.21.15.94:0/2330999095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91c8004d50 0x7f91e41f7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91d9400640 1 -- 172.21.15.94:0/2330999095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e41f77c0 con 0x7f91e40af640 2024-09-10T23:36:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.094+0000 7f91e1e00640 1 --2- 172.21.15.94:0/2330999095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91c8004d50 0x7f91e41f7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:36:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.098+0000 7f91d9400640 1 --2- 172.21.15.94:0/2330999095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e40af640 0x7f91e41e54b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0066b0 tx=0x7f91cc0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.098+0000 7f91daa00640 1 -- 172.21.15.94:0/2330999095 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc015070 con 0x7f91e40af640 2024-09-10T23:36:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.098+0000 7f91daa00640 1 -- 172.21.15.94:0/2330999095 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91cc011920 con 0x7f91e40af640 2024-09-10T23:36:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.098+0000 7f91daa00640 1 -- 172.21.15.94:0/2330999095 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc011d70 con 0x7f91e40af640 2024-09-10T23:36:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.098+0000 7f91e9000640 1 -- 172.21.15.94:0/2330999095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e41f4670 con 0x7f91e40af640 2024-09-10T23:36:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.098+0000 7f91e9000640 1 -- 172.21.15.94:0/2330999095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e41f4c30 con 0x7f91e40af640 2024-09-10T23:36:15.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.098+0000 7f91e9000640 1 -- 172.21.15.94:0/2330999095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e406f4b0 con 0x7f91e40af640 2024-09-10T23:36:15.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.098+0000 7f91daa00640 1 -- 172.21.15.94:0/2330999095 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f91cc010420 con 0x7f91e40af640 2024-09-10T23:36:15.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.098+0000 7f91daa00640 1 --2- 172.21.15.94:0/2330999095 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f91c0072940 0x7f91c0074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:15.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.098+0000 7f91daa00640 1 -- 172.21.15.94:0/2330999095 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f91cc0942d0 con 0x7f91e40af640 2024-09-10T23:36:15.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.098+0000 7f91e1e00640 1 --2- 172.21.15.94:0/2330999095 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f91c0072940 0x7f91c0074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:15.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.102+0000 7f91e1e00640 1 --2- 172.21.15.94:0/2330999095 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f91c0072940 0x7f91c0074e00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009fd0 tx=0x7f91d4009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:15.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.102+0000 7f91daa00640 1 -- 172.21.15.94:0/2330999095 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f91cc05f000 con 0x7f91e40af640 2024-09-10T23:36:15.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.302+0000 7f91e9000640 1 -- 172.21.15.94:0/2330999095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91e41ea8d0 con 0x7f91e40af640 2024-09-10T23:36:15.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.306+0000 7f91daa00640 1 -- 172.21.15.94:0/2330999095 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+3921 (secure 0 0 0) 0x7f91cc063a30 con 0x7f91e40af640 2024-09-10T23:36:15.306 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:15.306 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":37,"btime":"2024-09-10T23:36:08:807640+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5814,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3613711739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3613711739},{"type":"v1","addr":"172.21.15.94:6834","nonce":3613711739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6150,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3459298371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3459298371},{"type":"v1","addr":"172.21.15.94:6835","nonce":3459298371}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":195107,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1388563408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1388563408},{"type":"v1","addr":"172.21.15.181:6836","nonce":1388563408}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":195110,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1882902083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1882902083},{"type":"v1","addr":"172.21.15.181:6837","nonce":1882902083}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-09-10T23:36:15.306 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 37 2024-09-10T23:36:15.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.306+0000 7f91e9000640 1 -- 172.21.15.94:0/2330999095 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f91c0072940 msgr2=0x7f91c0074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:15.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.306+0000 7f91e9000640 1 --2- 172.21.15.94:0/2330999095 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f91c0072940 0x7f91c0074e00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009fd0 tx=0x7f91d4009500 comp rx=0 tx=0).stop 2024-09-10T23:36:15.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.306+0000 7f91e9000640 1 -- 172.21.15.94:0/2330999095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e40af640 msgr2=0x7f91e41e54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:15.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.306+0000 7f91e9000640 1 --2- 172.21.15.94:0/2330999095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e40af640 0x7f91e41e54b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0066b0 tx=0x7f91cc0039d0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.306+0000 7f91e9000640 1 -- 172.21.15.94:0/2330999095 shutdown_connections 2024-09-10T23:36:15.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.306+0000 7f91e9000640 1 --2- 172.21.15.94:0/2330999095 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f91c0072940 0x7f91c0074e00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.306+0000 7f91e9000640 1 --2- 172.21.15.94:0/2330999095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91e40afa80 0x7f91e41e59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.306+0000 7f91e9000640 1 --2- 172.21.15.94:0/2330999095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e40af640 0x7f91e41e54b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.306+0000 7f91e9000640 1 --2- 172.21.15.94:0/2330999095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91c8004d50 0x7f91e41f7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.306+0000 7f91e9000640 1 -- 172.21.15.94:0/2330999095 >> 172.21.15.94:0/2330999095 conn(0x7f91e406cdf0 msgr2=0x7f91e40cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:15.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.306+0000 7f91e9000640 1 -- 172.21.15.94:0/2330999095 shutdown_connections 2024-09-10T23:36:15.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.306+0000 7f91e9000640 1 -- 172.21.15.94:0/2330999095 wait complete. 2024-09-10T23:36:15.327 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T23:36:15.327 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T23:36:15.328 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T23:36:15.328 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:36:15.328 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T23:36:15.328 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T23:36:15.328 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T23:36:15.328 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:36:15.329 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T23:36:15.329 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T23:36:15.329 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T23:36:15.329 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:36:15.329 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T23:36:15.329 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T23:36:15.329 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T23:36:15.329 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:36:15.339 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T23:36:15.339 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T23:36:15.340 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T23:36:15.340 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T23:36:15.343 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T23:36:15.343 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T23:36:15.344 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T23:36:15.344 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T23:36:15.387 INFO:tasks.ceph.mds.c:Started 2024-09-10T23:36:15.388 DEBUG:teuthology.parallel:result is None 2024-09-10T23:36:15.388 INFO:tasks.ceph.mds.b:Started 2024-09-10T23:36:15.389 DEBUG:teuthology.parallel:result is None 2024-09-10T23:36:15.393 INFO:tasks.ceph.mds.a:Started 2024-09-10T23:36:15.393 DEBUG:teuthology.parallel:result is None 2024-09-10T23:36:15.394 INFO:tasks.ceph.mds.d:Started 2024-09-10T23:36:15.394 DEBUG:teuthology.parallel:result is None 2024-09-10T23:36:15.394 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T23:36:15.474 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-09-10T23:36:15.475 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-09-10T23:36:15.481 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:36:15.481 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2024-09-10T23:36:15.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb7600640 1 -- 172.21.15.94:0/978206476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cb006ce80 msgr2=0x7f6cb0071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:15.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb7600640 1 --2- 172.21.15.94:0/978206476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cb006ce80 0x7f6cb0071930 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009900 tx=0x7f6ca001f0d0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb7600640 1 -- 172.21.15.94:0/978206476 shutdown_connections 2024-09-10T23:36:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb7600640 1 --2- 172.21.15.94:0/978206476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cb015b6f0 0x7f6cb015bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb7600640 1 --2- 172.21.15.94:0/978206476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cb006ce80 0x7f6cb0071930 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb7600640 1 -- 172.21.15.94:0/978206476 >> 172.21.15.94:0/978206476 conn(0x7f6cb00d0c70 msgr2=0x7f6cb00d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb7600640 1 -- 172.21.15.94:0/978206476 shutdown_connections 2024-09-10T23:36:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb7600640 1 -- 172.21.15.94:0/978206476 wait complete. 2024-09-10T23:36:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb7600640 1 Processor -- start 2024-09-10T23:36:15.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb7600640 1 -- start start 2024-09-10T23:36:15.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb7600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cb006ce80 0x7f6cb00c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:15.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb7600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb0074060 0x7f6cb01f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb7600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cb015b6f0 0x7f6cb00a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb7600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb00cb650 con 0x7f6cb0074060 2024-09-10T23:36:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb7600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb00cb4d0 con 0x7f6cb006ce80 2024-09-10T23:36:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb7600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb00cb7d0 con 0x7f6cb015b6f0 2024-09-10T23:36:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6ca7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb0074060 0x7f6cb01f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6ca7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb0074060 0x7f6cb01f3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54978/0 (socket says 172.21.15.94:54978) 2024-09-10T23:36:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6ca7e00640 1 -- 172.21.15.94:0/3539866629 learned_addr learned my addr 172.21.15.94:0/3539866629 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb4c00640 1 --2- 172.21.15.94:0/3539866629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cb006ce80 0x7f6cb00c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb5600640 1 --2- 172.21.15.94:0/3539866629 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cb015b6f0 0x7f6cb00a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb4c00640 1 -- 172.21.15.94:0/3539866629 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cb015b6f0 msgr2=0x7f6cb00a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb4c00640 1 --2- 172.21.15.94:0/3539866629 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cb015b6f0 0x7f6cb00a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb4c00640 1 -- 172.21.15.94:0/3539866629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb0074060 msgr2=0x7f6cb01f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:15.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb4c00640 1 --2- 172.21.15.94:0/3539866629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb0074060 0x7f6cb01f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6cb4c00640 1 -- 172.21.15.94:0/3539866629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb00c8cc0 con 0x7f6cb006ce80 2024-09-10T23:36:15.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.546+0000 7f6ca7e00640 1 --2- 172.21.15.94:0/3539866629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb0074060 0x7f6cb01f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:36:15.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.550+0000 7f6cb4c00640 1 --2- 172.21.15.94:0/3539866629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cb006ce80 0x7f6cb00c7d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00ce700 tx=0x7f6ca0005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:15.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.550+0000 7f6cad600640 1 -- 172.21.15.94:0/3539866629 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca00b5040 con 0x7f6cb006ce80 2024-09-10T23:36:15.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.550+0000 7f6cad600640 1 -- 172.21.15.94:0/3539866629 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ca0005330 con 0x7f6cb006ce80 2024-09-10T23:36:15.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.550+0000 7f6cad600640 1 -- 172.21.15.94:0/3539866629 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca00046d0 con 0x7f6cb006ce80 2024-09-10T23:36:15.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.550+0000 7f6cb7600640 1 -- 172.21.15.94:0/3539866629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb00c8700 con 0x7f6cb006ce80 2024-09-10T23:36:15.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.550+0000 7f6cb7600640 1 -- 172.21.15.94:0/3539866629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb015c350 con 0x7f6cb006ce80 2024-09-10T23:36:15.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.550+0000 7f6cb7600640 1 -- 172.21.15.94:0/3539866629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb0071930 con 0x7f6cb006ce80 2024-09-10T23:36:15.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.550+0000 7f6cad600640 1 -- 172.21.15.94:0/3539866629 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f6ca00c2060 con 0x7f6cb006ce80 2024-09-10T23:36:15.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.550+0000 7f6cad600640 1 --2- 172.21.15.94:0/3539866629 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6c8c072930 0x7f6c8c074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:15.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.550+0000 7f6cad600640 1 -- 172.21.15.94:0/3539866629 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f6ca0130470 con 0x7f6cb006ce80 2024-09-10T23:36:15.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.550+0000 7f6ca7e00640 1 --2- 172.21.15.94:0/3539866629 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6c8c072930 0x7f6c8c074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:15.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.554+0000 7f6ca7e00640 1 --2- 172.21.15.94:0/3539866629 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6c8c072930 0x7f6c8c074df0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00d07d0 tx=0x7f6c98009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:15.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.554+0000 7f6cad600640 1 -- 172.21.15.94:0/3539866629 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6ca0018c30 con 0x7f6cb006ce80 2024-09-10T23:36:15.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.730+0000 7f6cb7600640 1 -- 172.21.15.94:0/3539866629 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6cb00d0520 con 0x7f6cb006ce80 2024-09-10T23:36:15.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.978+0000 7f6cad600640 1 -- 172.21.15.94:0/3539866629 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v63) ==== 97+0+0 (secure 0 0 0) 0x7f6ca00fb2b0 con 0x7f6cb006ce80 2024-09-10T23:36:15.981 INFO:teuthology.orchestra.run.smithi094.stderr: removed all blocklist entries 2024-09-10T23:36:15.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.982+0000 7f6cb7600640 1 -- 172.21.15.94:0/3539866629 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6c8c072930 msgr2=0x7f6c8c074df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:15.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.982+0000 7f6cb7600640 1 --2- 172.21.15.94:0/3539866629 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6c8c072930 0x7f6c8c074df0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00d07d0 tx=0x7f6c98009040 comp rx=0 tx=0).stop 2024-09-10T23:36:15.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.982+0000 7f6cb7600640 1 -- 172.21.15.94:0/3539866629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cb006ce80 msgr2=0x7f6cb00c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:15.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.982+0000 7f6cb7600640 1 --2- 172.21.15.94:0/3539866629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cb006ce80 0x7f6cb00c7d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00ce700 tx=0x7f6ca0005de0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.982+0000 7f6cb7600640 1 -- 172.21.15.94:0/3539866629 shutdown_connections 2024-09-10T23:36:15.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.982+0000 7f6cb7600640 1 --2- 172.21.15.94:0/3539866629 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f6c8c072930 0x7f6c8c074df0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.982+0000 7f6cb7600640 1 --2- 172.21.15.94:0/3539866629 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cb015b6f0 0x7f6cb00a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.982+0000 7f6cb7600640 1 --2- 172.21.15.94:0/3539866629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb0074060 0x7f6cb01f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.982+0000 7f6cb7600640 1 --2- 172.21.15.94:0/3539866629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cb006ce80 0x7f6cb00c7d40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:15.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.982+0000 7f6cb7600640 1 -- 172.21.15.94:0/3539866629 >> 172.21.15.94:0/3539866629 conn(0x7f6cb00d0c70 msgr2=0x7f6cb00b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:15.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.982+0000 7f6cb7600640 1 -- 172.21.15.94:0/3539866629 shutdown_connections 2024-09-10T23:36:15.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:15.982+0000 7f6cb7600640 1 -- 172.21.15.94:0/3539866629 wait complete. 2024-09-10T23:36:16.005 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T23:36:16.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.146+0000 7ff0ef600640 1 -- 172.21.15.94:0/3311325486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0e806e9a0 msgr2=0x7ff0e80cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:16.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.146+0000 7ff0ef600640 1 --2- 172.21.15.94:0/3311325486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0e806e9a0 0x7ff0e80cc2c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009bc0 tx=0x7ff0e001cb70 comp rx=0 tx=0).stop 2024-09-10T23:36:16.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.146+0000 7ff0ef600640 1 -- 172.21.15.94:0/3311325486 shutdown_connections 2024-09-10T23:36:16.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.146+0000 7ff0ef600640 1 --2- 172.21.15.94:0/3311325486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0e806e9a0 0x7ff0e80cc2c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.146+0000 7ff0ef600640 1 --2- 172.21.15.94:0/3311325486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0e815dd50 0x7ff0e80b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.146+0000 7ff0ef600640 1 -- 172.21.15.94:0/3311325486 >> 172.21.15.94:0/3311325486 conn(0x7ff0e809aa70 msgr2=0x7ff0e809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:16.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.146+0000 7ff0ef600640 1 -- 172.21.15.94:0/3311325486 shutdown_connections 2024-09-10T23:36:16.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.146+0000 7ff0ef600640 1 -- 172.21.15.94:0/3311325486 wait complete. 2024-09-10T23:36:16.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.146+0000 7ff0ef600640 1 Processor -- start 2024-09-10T23:36:16.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.146+0000 7ff0ef600640 1 -- start start 2024-09-10T23:36:16.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ef600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0e806e9a0 0x7ff0e8159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:16.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ef600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0e815dd50 0x7ff0e80c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:16.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ef600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0e8162560 0x7ff0e80ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:16.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ef600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e816ae70 con 0x7ff0e806e9a0 2024-09-10T23:36:16.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ef600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e816acf0 con 0x7ff0e8162560 2024-09-10T23:36:16.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ef600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e816aff0 con 0x7ff0e815dd50 2024-09-10T23:36:16.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ecc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0e806e9a0 0x7ff0e8159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:16.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ecc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0e806e9a0 0x7ff0e8159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54980/0 (socket says 172.21.15.94:54980) 2024-09-10T23:36:16.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ecc00640 1 -- 172.21.15.94:0/2497911061 learned_addr learned my addr 172.21.15.94:0/2497911061 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:16.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0e7e00640 1 --2- 172.21.15.94:0/2497911061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0e815dd50 0x7ff0e80c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:16.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ed600640 1 --2- 172.21.15.94:0/2497911061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0e8162560 0x7ff0e80ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:16.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ecc00640 1 -- 172.21.15.94:0/2497911061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0e815dd50 msgr2=0x7ff0e80c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:16.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ecc00640 1 --2- 172.21.15.94:0/2497911061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0e815dd50 0x7ff0e80c9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ecc00640 1 -- 172.21.15.94:0/2497911061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0e8162560 msgr2=0x7ff0e80ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:16.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ecc00640 1 --2- 172.21.15.94:0/2497911061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0e8162560 0x7ff0e80ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ecc00640 1 -- 172.21.15.94:0/2497911061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e80c92e0 con 0x7ff0e806e9a0 2024-09-10T23:36:16.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ed600640 1 --2- 172.21.15.94:0/2497911061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0e8162560 0x7ff0e80ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:36:16.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ecc00640 1 --2- 172.21.15.94:0/2497911061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0e806e9a0 0x7ff0e8159320 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00e9d0 tx=0x7ff0dc00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:16.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0e4c00640 1 -- 172.21.15.94:0/2497911061 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc00cde0 con 0x7ff0e806e9a0 2024-09-10T23:36:16.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0e4c00640 1 -- 172.21.15.94:0/2497911061 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0dc004020 con 0x7ff0e806e9a0 2024-09-10T23:36:16.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0e4c00640 1 -- 172.21.15.94:0/2497911061 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc004450 con 0x7ff0e806e9a0 2024-09-10T23:36:16.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ef600640 1 -- 172.21.15.94:0/2497911061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e8159a70 con 0x7ff0e806e9a0 2024-09-10T23:36:16.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ef600640 1 -- 172.21.15.94:0/2497911061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e815a030 con 0x7ff0e806e9a0 2024-09-10T23:36:16.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0ef600640 1 -- 172.21.15.94:0/2497911061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0e806c7d0 con 0x7ff0e806e9a0 2024-09-10T23:36:16.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.150+0000 7ff0e4c00640 1 -- 172.21.15.94:0/2497911061 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7ff0dc003600 con 0x7ff0e806e9a0 2024-09-10T23:36:16.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.154+0000 7ff0e4c00640 1 --2- 172.21.15.94:0/2497911061 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff0c8072940 0x7ff0c8074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:16.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.154+0000 7ff0e4c00640 1 -- 172.21.15.94:0/2497911061 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7ff0dc0941f0 con 0x7ff0e806e9a0 2024-09-10T23:36:16.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.154+0000 7ff0e7e00640 1 --2- 172.21.15.94:0/2497911061 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff0c8072940 0x7ff0c8074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:16.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.154+0000 7ff0e7e00640 1 --2- 172.21.15.94:0/2497911061 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff0c8072940 0x7ff0c8074e00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff0e81fb120 tx=0x7ff0d4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:16.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.158+0000 7ff0e4c00640 1 -- 172.21.15.94:0/2497911061 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff0dc05f540 con 0x7ff0e806e9a0 2024-09-10T23:36:16.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.382+0000 7ff0ef600640 1 -- 172.21.15.94:0/2497911061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ff0e80cc2c0 con 0x7ff0e806e9a0 2024-09-10T23:36:16.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.386+0000 7ff0e4c00640 1 -- 172.21.15.94:0/2497911061 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) ==== 80+0+5617 (secure 0 0 0) 0x7ff0dc05ee40 con 0x7ff0e806e9a0 2024-09-10T23:36:16.386 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:16.386 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:36:16.386 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_dump": [ 2024-09-10T23:36:16.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.386 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.a", 2024-09-10T23:36:16.386 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/2m3LxAAr0ErpLJg+0+I2r74zQya4w==", 2024-09-10T23:36:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:36:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:36:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:36:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.b", 2024-09-10T23:36:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmc8pYABAAw90zu8D+9IwK15h4HIlGBA==", 2024-09-10T23:36:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:36:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:36:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:36:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.c", 2024-09-10T23:36:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmk98KNxAA44e9N4VHbw/94Lh1V9zugQ==", 2024-09-10T23:36:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:36:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:36:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:36:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.d", 2024-09-10T23:36:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmVAKzBxAAJehOw6aphTF/g3ucoQfw9A==", 2024-09-10T23:36:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:36:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:36:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:36:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.0", 2024-09-10T23:36:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDa1uBmSAD3KBAAJTF1OEC44Ml599aL7iEuDw==", 2024-09-10T23:36:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:36:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:36:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.1", 2024-09-10T23:36:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDe1uBma1L9ABAAfEjW1/Hb4fh4Kwv/ZUgFAQ==", 2024-09-10T23:36:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:36:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:36:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.2", 2024-09-10T23:36:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDh1uBmtLUQFhAAweeFHj6BRQVC9BX+VLiWEQ==", 2024-09-10T23:36:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:36:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:36:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.3", 2024-09-10T23:36:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDk1uBm1KFCKRAAev28iYcrEYOViX95uo6Kbw==", 2024-09-10T23:36:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:36:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:36:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.4", 2024-09-10T23:36:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDq1uBm/Hg9LhAAvB2D0rj4DeRPxTrGmXYIsA==", 2024-09-10T23:36:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:36:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:36:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.5", 2024-09-10T23:36:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDu1uBm1fAKCBAAFHMjnGFAItLdaACF/UjE/w==", 2024-09-10T23:36:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:36:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:36:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.6", 2024-09-10T23:36:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDx1uBmXgYfHhAA/6wwtM4mDRXsIpoTIHKJ0w==", 2024-09-10T23:36:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:36:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:36:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.7", 2024-09-10T23:36:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQD01uBmw6hQMxAAk7oGAg+iIplNK5KrokGwNA==", 2024-09-10T23:36:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:36:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:36:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.0", 2024-09-10T23:36:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g==", 2024-09-10T23:36:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:36:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow r", 2024-09-10T23:36:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T23:36:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.1", 2024-09-10T23:36:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmuSkhFRAA1R6WNXLImkGfhW9/XJzEAQ==", 2024-09-10T23:36:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:36:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:36:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:36:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:36:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.2", 2024-09-10T23:36:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmfvaqERAASpL0KtglFcAuRf4a/0z5JQ==", 2024-09-10T23:36:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:36:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:36:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:36:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:36:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.3", 2024-09-10T23:36:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBm8xUtGxAARc0ywfAhMxBNd57TzBoV3g==", 2024-09-10T23:36:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:36:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:36:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:36:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:36:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.admin", 2024-09-10T23:36:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmOpg1FBAA9NV6Jqt6doWMuAfcxztlOg==", 2024-09-10T23:36:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:36:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:36:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:36:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.alice", 2024-09-10T23:36:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDi1+BmAfZDJBAAOx1kuV3ZnOhMI/UiYmnyGQ==", 2024-09-10T23:36:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol98180", 2024-09-10T23:36:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow r", 2024-09-10T23:36:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-10T23:36:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.x", 2024-09-10T23:36:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/EPVKxAA/Myv8Zjf7BejbzlkxyCC1w==", 2024-09-10T23:36:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:36:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:36:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.y", 2024-09-10T23:36:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmhHvtJxAACe+79nYhv2ODwtmNhA45FA==", 2024-09-10T23:36:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:36:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:36:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:36:16.400 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:36:16.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.386+0000 7ff0ef600640 1 -- 172.21.15.94:0/2497911061 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff0c8072940 msgr2=0x7ff0c8074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:16.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.386+0000 7ff0ef600640 1 --2- 172.21.15.94:0/2497911061 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff0c8072940 0x7ff0c8074e00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff0e81fb120 tx=0x7ff0d4009040 comp rx=0 tx=0).stop 2024-09-10T23:36:16.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.386+0000 7ff0ef600640 1 -- 172.21.15.94:0/2497911061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0e806e9a0 msgr2=0x7ff0e8159320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:16.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.386+0000 7ff0ef600640 1 --2- 172.21.15.94:0/2497911061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0e806e9a0 0x7ff0e8159320 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00e9d0 tx=0x7ff0dc00ee90 comp rx=0 tx=0).stop 2024-09-10T23:36:16.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.386+0000 7ff0ef600640 1 -- 172.21.15.94:0/2497911061 shutdown_connections 2024-09-10T23:36:16.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.386+0000 7ff0ef600640 1 --2- 172.21.15.94:0/2497911061 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff0c8072940 0x7ff0c8074e00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.386+0000 7ff0ef600640 1 --2- 172.21.15.94:0/2497911061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0e8162560 0x7ff0e80ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.386+0000 7ff0ef600640 1 --2- 172.21.15.94:0/2497911061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0e815dd50 0x7ff0e80c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.386+0000 7ff0ef600640 1 --2- 172.21.15.94:0/2497911061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0e806e9a0 0x7ff0e8159320 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.386+0000 7ff0ef600640 1 -- 172.21.15.94:0/2497911061 >> 172.21.15.94:0/2497911061 conn(0x7ff0e809aa70 msgr2=0x7ff0e80cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:16.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.386+0000 7ff0ef600640 1 -- 172.21.15.94:0/2497911061 shutdown_connections 2024-09-10T23:36:16.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.386+0000 7ff0ef600640 1 -- 172.21.15.94:0/2497911061 wait complete. 2024-09-10T23:36:16.408 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2024-09-10T23:36:16.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.546+0000 7f3ff3e00640 1 -- 172.21.15.94:0/2977245719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fec0c49b0 msgr2=0x7f3fec0cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.546+0000 7f3ff3e00640 1 --2- 172.21.15.94:0/2977245719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fec0c49b0 0x7f3fec0cadc0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3fec06b000 tx=0x7f3fd801c8e0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.546+0000 7f3ff3e00640 1 -- 172.21.15.94:0/2977245719 shutdown_connections 2024-09-10T23:36:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.546+0000 7f3ff3e00640 1 --2- 172.21.15.94:0/2977245719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fec0cb300 0x7f3fec15e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.546+0000 7f3ff3e00640 1 --2- 172.21.15.94:0/2977245719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fec0c49b0 0x7f3fec0cadc0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.546+0000 7f3ff3e00640 1 --2- 172.21.15.94:0/2977245719 >> v2:172.21.15.181:3300/0 conn(0x7f3fec158790 0x7f3fec06fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.546+0000 7f3ff3e00640 1 -- 172.21.15.94:0/2977245719 >> 172.21.15.94:0/2977245719 conn(0x7f3fec0cfaf0 msgr2=0x7f3fec0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.546+0000 7f3ff3e00640 1 -- 172.21.15.94:0/2977245719 shutdown_connections 2024-09-10T23:36:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.546+0000 7f3ff3e00640 1 -- 172.21.15.94:0/2977245719 wait complete. 2024-09-10T23:36:16.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff3e00640 1 Processor -- start 2024-09-10T23:36:16.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff3e00640 1 -- start start 2024-09-10T23:36:16.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff3e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fec0c49b0 0x7f3fec146700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:16.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fec0cb300 0x7f3fec15c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:16.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fec158790 0x7f3fec073e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:16.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff3e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec15b930 con 0x7f3fec0cb300 2024-09-10T23:36:16.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff3e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec15b7b0 con 0x7f3fec158790 2024-09-10T23:36:16.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff3e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3fec15bab0 con 0x7f3fec0c49b0 2024-09-10T23:36:16.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fec0cb300 0x7f3fec15c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:16.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fec0cb300 0x7f3fec15c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55002/0 (socket says 172.21.15.94:55002) 2024-09-10T23:36:16.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff0a00640 1 -- 172.21.15.94:0/912033141 learned_addr learned my addr 172.21.15.94:0/912033141 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:16.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff0a00640 1 -- 172.21.15.94:0/912033141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fec0c49b0 msgr2=0x7f3fec146700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:16.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff0a00640 1 --2- 172.21.15.94:0/912033141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fec0c49b0 0x7f3fec146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff0a00640 1 -- 172.21.15.94:0/912033141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fec158790 msgr2=0x7f3fec073e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:16.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff1e00640 1 --2- 172.21.15.94:0/912033141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fec158790 0x7f3fec073e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:16.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff0a00640 1 --2- 172.21.15.94:0/912033141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fec158790 0x7f3fec073e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff0a00640 1 -- 172.21.15.94:0/912033141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fec0ce740 con 0x7f3fec0cb300 2024-09-10T23:36:16.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff1e00640 1 --2- 172.21.15.94:0/912033141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fec158790 0x7f3fec073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:36:16.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff0a00640 1 --2- 172.21.15.94:0/912033141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fec0cb300 0x7f3fec15c1f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3fd801af90 tx=0x7f3fd80a55e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:16.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3fe9600640 1 -- 172.21.15.94:0/912033141 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd80b2040 con 0x7f3fec0cb300 2024-09-10T23:36:16.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3fe9600640 1 -- 172.21.15.94:0/912033141 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fd8004490 con 0x7f3fec0cb300 2024-09-10T23:36:16.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3fe9600640 1 -- 172.21.15.94:0/912033141 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8005290 con 0x7f3fec0cb300 2024-09-10T23:36:16.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff3e00640 1 -- 172.21.15.94:0/912033141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fec0cea20 con 0x7f3fec0cb300 2024-09-10T23:36:16.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.550+0000 7f3ff3e00640 1 -- 172.21.15.94:0/912033141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fec073910 con 0x7f3fec0cb300 2024-09-10T23:36:16.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.554+0000 7f3ff3e00640 1 -- 172.21.15.94:0/912033141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fec15bfe0 con 0x7f3fec0cb300 2024-09-10T23:36:16.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.554+0000 7f3fe9600640 1 -- 172.21.15.94:0/912033141 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f3fd8005430 con 0x7f3fec0cb300 2024-09-10T23:36:16.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.554+0000 7f3fe9600640 1 --2- 172.21.15.94:0/912033141 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3fcc072940 0x7f3fcc074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:16.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.554+0000 7f3fe9600640 1 -- 172.21.15.94:0/912033141 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f3fd812cc70 con 0x7f3fec0cb300 2024-09-10T23:36:16.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.554+0000 7f3ff1400640 1 --2- 172.21.15.94:0/912033141 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3fcc072940 0x7f3fcc074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:16.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.554+0000 7f3ff1400640 1 --2- 172.21.15.94:0/912033141 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3fcc072940 0x7f3fcc074e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009650 tx=0x7f3fe0006bb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:16.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.558+0000 7f3fe9600640 1 -- 172.21.15.94:0/912033141 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3fd80164d0 con 0x7f3fec0cb300 2024-09-10T23:36:16.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.782+0000 7f3ff3e00640 1 -- 172.21.15.94:0/912033141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) -- 0x7f3fec0d0850 con 0x7f3fec0cb300 2024-09-10T23:36:16.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.782+0000 7f3fe9600640 1 -- 172.21.15.94:0/912033141 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v32) ==== 82+0+0 (secure 0 0 0) 0x7f3fd80f8150 con 0x7f3fec0cb300 2024-09-10T23:36:16.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.786+0000 7f3ff3e00640 1 -- 172.21.15.94:0/912033141 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3fcc072940 msgr2=0x7f3fcc074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:16.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.786+0000 7f3ff3e00640 1 --2- 172.21.15.94:0/912033141 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3fcc072940 0x7f3fcc074e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009650 tx=0x7f3fe0006bb0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.786+0000 7f3ff3e00640 1 -- 172.21.15.94:0/912033141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fec0cb300 msgr2=0x7f3fec15c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:16.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.786+0000 7f3ff3e00640 1 --2- 172.21.15.94:0/912033141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fec0cb300 0x7f3fec15c1f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3fd801af90 tx=0x7f3fd80a55e0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.786+0000 7f3ff3e00640 1 -- 172.21.15.94:0/912033141 shutdown_connections 2024-09-10T23:36:16.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.786+0000 7f3ff3e00640 1 --2- 172.21.15.94:0/912033141 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3fcc072940 0x7f3fcc074e00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.786+0000 7f3ff3e00640 1 --2- 172.21.15.94:0/912033141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fec158790 0x7f3fec073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.786+0000 7f3ff3e00640 1 --2- 172.21.15.94:0/912033141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fec0cb300 0x7f3fec15c1f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.786+0000 7f3ff3e00640 1 --2- 172.21.15.94:0/912033141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fec0c49b0 0x7f3fec146700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.786+0000 7f3ff3e00640 1 -- 172.21.15.94:0/912033141 >> 172.21.15.94:0/912033141 conn(0x7f3fec0cfaf0 msgr2=0x7f3fec06f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:16.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.786+0000 7f3ff3e00640 1 -- 172.21.15.94:0/912033141 shutdown_connections 2024-09-10T23:36:16.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.786+0000 7f3ff3e00640 1 -- 172.21.15.94:0/912033141 wait complete. 2024-09-10T23:36:16.808 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T23:36:16.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.946+0000 7f4eeea00640 1 -- 172.21.15.94:0/1358259425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ee815f5c0 msgr2=0x7f4ee80cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:16.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.946+0000 7f4eeea00640 1 --2- 172.21.15.94:0/1358259425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ee815f5c0 0x7f4ee80cfb60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009900 tx=0x7f4ed801c7c0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.946+0000 7f4eeea00640 1 -- 172.21.15.94:0/1358259425 shutdown_connections 2024-09-10T23:36:16.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.946+0000 7f4eeea00640 1 --2- 172.21.15.94:0/1358259425 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee80af310 0x7f4ee80af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.946+0000 7f4eeea00640 1 --2- 172.21.15.94:0/1358259425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ee815f5c0 0x7f4ee80cfb60 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.946+0000 7f4eeea00640 1 -- 172.21.15.94:0/1358259425 >> 172.21.15.94:0/1358259425 conn(0x7f4ee80d1c60 msgr2=0x7f4ee80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:16.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.946+0000 7f4eeea00640 1 -- 172.21.15.94:0/1358259425 shutdown_connections 2024-09-10T23:36:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.946+0000 7f4eeea00640 1 -- 172.21.15.94:0/1358259425 wait complete. 2024-09-10T23:36:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4eeea00640 1 Processor -- start 2024-09-10T23:36:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4eeea00640 1 -- start start 2024-09-10T23:36:16.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4eeea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ee80af310 0x7f4ee8146e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:16.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4eeea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee80b2bb0 0x7f4ee80b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:16.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4eeea00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee815f5c0 0x7f4ee80c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:16.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4eeea00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8169000 con 0x7f4ee80af310 2024-09-10T23:36:16.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4eeea00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8168e80 con 0x7f4ee80b2bb0 2024-09-10T23:36:16.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4eeea00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee8169180 con 0x7f4ee815f5c0 2024-09-10T23:36:16.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4ee7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ee80af310 0x7f4ee8146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:16.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4ee7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ee80af310 0x7f4ee8146e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55018/0 (socket says 172.21.15.94:55018) 2024-09-10T23:36:16.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4ee7e00640 1 -- 172.21.15.94:0/3497598850 learned_addr learned my addr 172.21.15.94:0/3497598850 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:16.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4eeca00640 1 --2- 172.21.15.94:0/3497598850 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee815f5c0 0x7f4ee80c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:16.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4edf400640 1 --2- 172.21.15.94:0/3497598850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee80b2bb0 0x7f4ee80b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:16.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4ee7e00640 1 -- 172.21.15.94:0/3497598850 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee815f5c0 msgr2=0x7f4ee80c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:16.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4ee7e00640 1 --2- 172.21.15.94:0/3497598850 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee815f5c0 0x7f4ee80c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4ee7e00640 1 -- 172.21.15.94:0/3497598850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee80b2bb0 msgr2=0x7f4ee80b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:16.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4ee7e00640 1 --2- 172.21.15.94:0/3497598850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee80b2bb0 0x7f4ee80b7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:16.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4ee7e00640 1 -- 172.21.15.94:0/3497598850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee80c7d40 con 0x7f4ee80af310 2024-09-10T23:36:16.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4ee7e00640 1 --2- 172.21.15.94:0/3497598850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ee80af310 0x7f4ee8146e80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009850 tx=0x7f4ed80a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:16.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4ee4c00640 1 -- 172.21.15.94:0/3497598850 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed80ba020 con 0x7f4ee80af310 2024-09-10T23:36:16.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4ee4c00640 1 -- 172.21.15.94:0/3497598850 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ed8004430 con 0x7f4ee80af310 2024-09-10T23:36:16.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4ee4c00640 1 -- 172.21.15.94:0/3497598850 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed80b2040 con 0x7f4ee80af310 2024-09-10T23:36:16.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4eeea00640 1 -- 172.21.15.94:0/3497598850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee80c8020 con 0x7f4ee80af310 2024-09-10T23:36:16.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4eeea00640 1 -- 172.21.15.94:0/3497598850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee80b7790 con 0x7f4ee80af310 2024-09-10T23:36:16.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.950+0000 7f4eeea00640 1 -- 172.21.15.94:0/3497598850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee80af7a0 con 0x7f4ee80af310 2024-09-10T23:36:16.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.954+0000 7f4ee4c00640 1 -- 172.21.15.94:0/3497598850 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f4ed8003860 con 0x7f4ee80af310 2024-09-10T23:36:16.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.954+0000 7f4ee4c00640 1 --2- 172.21.15.94:0/3497598850 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4ec4072940 0x7f4ec4074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:16.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.954+0000 7f4ee4c00640 1 -- 172.21.15.94:0/3497598850 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f4ed812d540 con 0x7f4ee80af310 2024-09-10T23:36:16.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.954+0000 7f4edf400640 1 --2- 172.21.15.94:0/3497598850 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4ec4072940 0x7f4ec4074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:16.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.954+0000 7f4edf400640 1 --2- 172.21.15.94:0/3497598850 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4ec4072940 0x7f4ec4074e00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8150780 tx=0x7f4ed0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:16.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:16.958+0000 7f4ee4c00640 1 -- 172.21.15.94:0/3497598850 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4ed80c4060 con 0x7f4ee80af310 2024-09-10T23:36:17.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.126+0000 7f4eeea00640 1 -- 172.21.15.94:0/3497598850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4ee80749c0 con 0x7f4ee80af310 2024-09-10T23:36:17.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.126+0000 7f4ee4c00640 1 -- 172.21.15.94:0/3497598850 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v65) ==== 84+0+65 (secure 0 0 0) 0x7f4ed80164d0 con 0x7f4ee80af310 2024-09-10T23:36:17.130 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:17.130 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:36:17.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:17.130 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-09-10T23:36:17.130 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-09-10T23:36:17.130 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:17.130 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:36:17.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.130+0000 7f4eeea00640 1 -- 172.21.15.94:0/3497598850 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4ec4072940 msgr2=0x7f4ec4074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:17.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.130+0000 7f4eeea00640 1 --2- 172.21.15.94:0/3497598850 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4ec4072940 0x7f4ec4074e00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8150780 tx=0x7f4ed0009040 comp rx=0 tx=0).stop 2024-09-10T23:36:17.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.130+0000 7f4eeea00640 1 -- 172.21.15.94:0/3497598850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ee80af310 msgr2=0x7f4ee8146e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:17.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.130+0000 7f4eeea00640 1 --2- 172.21.15.94:0/3497598850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ee80af310 0x7f4ee8146e80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009850 tx=0x7f4ed80a5620 comp rx=0 tx=0).stop 2024-09-10T23:36:17.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.130+0000 7f4eeea00640 1 -- 172.21.15.94:0/3497598850 shutdown_connections 2024-09-10T23:36:17.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.130+0000 7f4eeea00640 1 --2- 172.21.15.94:0/3497598850 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4ec4072940 0x7f4ec4074e00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:17.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.130+0000 7f4eeea00640 1 --2- 172.21.15.94:0/3497598850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ee80b2bb0 0x7f4ee80b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:17.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.130+0000 7f4eeea00640 1 --2- 172.21.15.94:0/3497598850 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ee815f5c0 0x7f4ee80c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:17.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.130+0000 7f4eeea00640 1 --2- 172.21.15.94:0/3497598850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ee80af310 0x7f4ee8146e80 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:17.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.130+0000 7f4eeea00640 1 -- 172.21.15.94:0/3497598850 >> 172.21.15.94:0/3497598850 conn(0x7f4ee80d1c60 msgr2=0x7f4ee815e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:17.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.130+0000 7f4eeea00640 1 -- 172.21.15.94:0/3497598850 shutdown_connections 2024-09-10T23:36:17.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.130+0000 7f4eeea00640 1 -- 172.21.15.94:0/3497598850 wait complete. 2024-09-10T23:36:17.151 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T23:36:17.151 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T23:36:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.294+0000 7f3023200640 1 -- 172.21.15.94:0/2371908819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f302415ba30 msgr2=0x7f30240b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.294+0000 7f3023200640 1 --2- 172.21.15.94:0/2371908819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f302415ba30 0x7f30240b1bd0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3014009bc0 tx=0x7f301401cb70 comp rx=0 tx=0).stop 2024-09-10T23:36:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.294+0000 7f3023200640 1 -- 172.21.15.94:0/2371908819 shutdown_connections 2024-09-10T23:36:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.294+0000 7f3023200640 1 --2- 172.21.15.94:0/2371908819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f302415ba30 0x7f30240b1bd0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.294+0000 7f3023200640 1 -- 172.21.15.94:0/2371908819 >> 172.21.15.94:0/2371908819 conn(0x7f302409aa70 msgr2=0x7f302409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.294+0000 7f3023200640 1 -- 172.21.15.94:0/2371908819 shutdown_connections 2024-09-10T23:36:17.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.294+0000 7f3023200640 1 -- 172.21.15.94:0/2371908819 wait complete. 2024-09-10T23:36:17.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.294+0000 7f3023200640 1 Processor -- start 2024-09-10T23:36:17.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.294+0000 7f3023200640 1 -- start start 2024-09-10T23:36:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3023200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30240b2110 0x7f3024154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3023200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30240d04b0 0x7f30241f8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3023200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f302415ba30 0x7f30241f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3023200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3024169a50 con 0x7f302415ba30 2024-09-10T23:36:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3023200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f30241698d0 con 0x7f30240b2110 2024-09-10T23:36:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3023200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3024169bd0 con 0x7f30240d04b0 2024-09-10T23:36:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3022800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f302415ba30 0x7f30241f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3022800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f302415ba30 0x7f30241f8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55042/0 (socket says 172.21.15.94:55042) 2024-09-10T23:36:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3022800640 1 -- 172.21.15.94:0/136700053 learned_addr learned my addr 172.21.15.94:0/136700053 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:17.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3021e00640 1 --2- 172.21.15.94:0/136700053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30240b2110 0x7f3024154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:17.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3019400640 1 --2- 172.21.15.94:0/136700053 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30240d04b0 0x7f30241f8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:17.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3022800640 1 -- 172.21.15.94:0/136700053 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30240d04b0 msgr2=0x7f30241f8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:17.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3022800640 1 --2- 172.21.15.94:0/136700053 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30240d04b0 0x7f30241f8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:17.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3022800640 1 -- 172.21.15.94:0/136700053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30240b2110 msgr2=0x7f3024154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:17.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3022800640 1 --2- 172.21.15.94:0/136700053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30240b2110 0x7f3024154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:17.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3022800640 1 -- 172.21.15.94:0/136700053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30240c35d0 con 0x7f302415ba30 2024-09-10T23:36:17.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3019400640 1 --2- 172.21.15.94:0/136700053 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30240d04b0 0x7f30241f8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:17.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3022800640 1 --2- 172.21.15.94:0/136700053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f302415ba30 0x7f30241f8e70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f301c00c400 tx=0x7f301c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:17.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f301aa00640 1 -- 172.21.15.94:0/136700053 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c011790 con 0x7f302415ba30 2024-09-10T23:36:17.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f301aa00640 1 -- 172.21.15.94:0/136700053 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f301c017070 con 0x7f302415ba30 2024-09-10T23:36:17.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f301aa00640 1 -- 172.21.15.94:0/136700053 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c011cc0 con 0x7f302415ba30 2024-09-10T23:36:17.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3023200640 1 -- 172.21.15.94:0/136700053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30240c38b0 con 0x7f302415ba30 2024-09-10T23:36:17.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3023200640 1 -- 172.21.15.94:0/136700053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30240c3e70 con 0x7f302415ba30 2024-09-10T23:36:17.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f3023200640 1 -- 172.21.15.94:0/136700053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30240b0f20 con 0x7f302415ba30 2024-09-10T23:36:17.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.298+0000 7f301aa00640 1 -- 172.21.15.94:0/136700053 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f301c01c460 con 0x7f302415ba30 2024-09-10T23:36:17.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.302+0000 7f301aa00640 1 --2- 172.21.15.94:0/136700053 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3000072970 0x7f3000074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:17.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.302+0000 7f301aa00640 1 -- 172.21.15.94:0/136700053 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f301c093230 con 0x7f302415ba30 2024-09-10T23:36:17.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.302+0000 7f3021e00640 1 --2- 172.21.15.94:0/136700053 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3000072970 0x7f3000074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:17.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.302+0000 7f3021e00640 1 --2- 172.21.15.94:0/136700053 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3000072970 0x7f3000074e30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f30140af000 tx=0x7f301401b3b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:17.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.306+0000 7f301aa00640 1 -- 172.21.15.94:0/136700053 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f301c05e790 con 0x7f302415ba30 2024-09-10T23:36:17.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.470+0000 7f3023200640 1 -- 172.21.15.94:0/136700053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f30240b0960 con 0x7f302415ba30 2024-09-10T23:36:17.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.990+0000 7f301aa00640 1 -- 172.21.15.94:0/136700053 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v66) ==== 138+0+0 (secure 0 0 0) 0x7f301c05e090 con 0x7f302415ba30 2024-09-10T23:36:17.990 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-09-10T23:36:17.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.990+0000 7f3023200640 1 -- 172.21.15.94:0/136700053 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3000072970 msgr2=0x7f3000074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:17.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.990+0000 7f3023200640 1 --2- 172.21.15.94:0/136700053 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3000072970 0x7f3000074e30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f30140af000 tx=0x7f301401b3b0 comp rx=0 tx=0).stop 2024-09-10T23:36:17.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.990+0000 7f3023200640 1 -- 172.21.15.94:0/136700053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f302415ba30 msgr2=0x7f30241f8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:17.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.990+0000 7f3023200640 1 --2- 172.21.15.94:0/136700053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f302415ba30 0x7f30241f8e70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f301c00c400 tx=0x7f301c00c8c0 comp rx=0 tx=0).stop 2024-09-10T23:36:17.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.990+0000 7f3023200640 1 -- 172.21.15.94:0/136700053 shutdown_connections 2024-09-10T23:36:17.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.990+0000 7f3023200640 1 --2- 172.21.15.94:0/136700053 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3000072970 0x7f3000074e30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:17.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.990+0000 7f3023200640 1 --2- 172.21.15.94:0/136700053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f302415ba30 0x7f30241f8e70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:17.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.990+0000 7f3023200640 1 --2- 172.21.15.94:0/136700053 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30240d04b0 0x7f30241f8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:17.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.990+0000 7f3023200640 1 --2- 172.21.15.94:0/136700053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30240b2110 0x7f3024154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:17.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.990+0000 7f3023200640 1 -- 172.21.15.94:0/136700053 >> 172.21.15.94:0/136700053 conn(0x7f302409aa70 msgr2=0x7f30240cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:17.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.990+0000 7f3023200640 1 -- 172.21.15.94:0/136700053 shutdown_connections 2024-09-10T23:36:17.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:17.990+0000 7f3023200640 1 -- 172.21.15.94:0/136700053 wait complete. 2024-09-10T23:36:18.017 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T23:36:18.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faacd000640 1 -- 172.21.15.94:0/3324710661 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac80afe60 msgr2=0x7faac80d1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:18.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faacd000640 1 --2- 172.21.15.94:0/3324710661 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac80afe60 0x7faac80d1560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faac0009bf0 tx=0x7faac001cb50 comp rx=0 tx=0).stop 2024-09-10T23:36:18.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faacd000640 1 -- 172.21.15.94:0/3324710661 shutdown_connections 2024-09-10T23:36:18.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faacd000640 1 --2- 172.21.15.94:0/3324710661 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac80afe60 0x7faac80d1560 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:18.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faacd000640 1 -- 172.21.15.94:0/3324710661 >> 172.21.15.94:0/3324710661 conn(0x7faac80d0a40 msgr2=0x7faac80d0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:18.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faacd000640 1 -- 172.21.15.94:0/3324710661 shutdown_connections 2024-09-10T23:36:18.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faacd000640 1 -- 172.21.15.94:0/3324710661 wait complete. 2024-09-10T23:36:18.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faacd000640 1 Processor -- start 2024-09-10T23:36:18.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faacd000640 1 -- start start 2024-09-10T23:36:18.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faacd000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac8071a10 0x7faac81f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:18.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faacd000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac80afe60 0x7faac80c4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:18.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faacd000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faac80ccf30 0x7faac80aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:18.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faacd000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faac8169be0 con 0x7faac8071a10 2024-09-10T23:36:18.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faacd000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faac8169a60 con 0x7faac80ccf30 2024-09-10T23:36:18.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faacd000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7faac8169d60 con 0x7faac80afe60 2024-09-10T23:36:18.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faac7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac8071a10 0x7faac81f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:18.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faac7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac8071a10 0x7faac81f73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55050/0 (socket says 172.21.15.94:55050) 2024-09-10T23:36:18.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faac7400640 1 -- 172.21.15.94:0/1592666587 learned_addr learned my addr 172.21.15.94:0/1592666587 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:18.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faac7e00640 1 --2- 172.21.15.94:0/1592666587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faac80ccf30 0x7faac80aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:18.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faabea00640 1 --2- 172.21.15.94:0/1592666587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac80afe60 0x7faac80c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:18.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faac7400640 1 -- 172.21.15.94:0/1592666587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac80afe60 msgr2=0x7faac80c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:18.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faac7400640 1 --2- 172.21.15.94:0/1592666587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac80afe60 0x7faac80c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:18.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faac7400640 1 -- 172.21.15.94:0/1592666587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faac80ccf30 msgr2=0x7faac80aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:18.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faac7400640 1 --2- 172.21.15.94:0/1592666587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faac80ccf30 0x7faac80aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:18.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faac7400640 1 -- 172.21.15.94:0/1592666587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac80cfa40 con 0x7faac8071a10 2024-09-10T23:36:18.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faac7e00640 1 --2- 172.21.15.94:0/1592666587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faac80ccf30 0x7faac80aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:18.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.166+0000 7faabea00640 1 --2- 172.21.15.94:0/1592666587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac80afe60 0x7faac80c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:18.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.170+0000 7faac7400640 1 --2- 172.21.15.94:0/1592666587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac8071a10 0x7faac81f73e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7faab800c730 tx=0x7faab800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:18.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.170+0000 7faabfe00640 1 -- 172.21.15.94:0/1592666587 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8016020 con 0x7faac8071a10 2024-09-10T23:36:18.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.170+0000 7faabfe00640 1 -- 172.21.15.94:0/1592666587 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faab800a930 con 0x7faac8071a10 2024-09-10T23:36:18.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.170+0000 7faabfe00640 1 -- 172.21.15.94:0/1592666587 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab800ae00 con 0x7faac8071a10 2024-09-10T23:36:18.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.170+0000 7faacd000640 1 -- 172.21.15.94:0/1592666587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac80cab40 con 0x7faac8071a10 2024-09-10T23:36:18.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.170+0000 7faacd000640 1 -- 172.21.15.94:0/1592666587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac80cea90 con 0x7faac8071a10 2024-09-10T23:36:18.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.170+0000 7faacd000640 1 -- 172.21.15.94:0/1592666587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac80d1560 con 0x7faac8071a10 2024-09-10T23:36:18.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.170+0000 7faabfe00640 1 -- 172.21.15.94:0/1592666587 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7faab8025480 con 0x7faac8071a10 2024-09-10T23:36:18.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.170+0000 7faabfe00640 1 --2- 172.21.15.94:0/1592666587 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faaa4072940 0x7faaa4074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:18.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.174+0000 7faabfe00640 1 -- 172.21.15.94:0/1592666587 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5227+0+0 (secure 0 0 0) 0x7faab8094310 con 0x7faac8071a10 2024-09-10T23:36:18.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.174+0000 7faabea00640 1 --2- 172.21.15.94:0/1592666587 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faaa4072940 0x7faaa4074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:18.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.174+0000 7faabea00640 1 --2- 172.21.15.94:0/1592666587 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faaa4072940 0x7faaa4074e00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faab0007b10 tx=0x7faab0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:18.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.174+0000 7faabfe00640 1 -- 172.21.15.94:0/1592666587 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faab8020020 con 0x7faac8071a10 2024-09-10T23:36:18.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.342+0000 7faacd000640 1 -- 172.21.15.94:0/1592666587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faac815d4b0 con 0x7faac8071a10 2024-09-10T23:36:18.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:18.998+0000 7faabfe00640 1 -- 172.21.15.94:0/1592666587 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v67) ==== 170+0+0 (secure 0 0 0) 0x7faac80ccbe0 con 0x7faac8071a10 2024-09-10T23:36:18.999 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-09-10T23:36:19.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.002+0000 7faacd000640 1 -- 172.21.15.94:0/1592666587 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faaa4072940 msgr2=0x7faaa4074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:19.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.002+0000 7faacd000640 1 --2- 172.21.15.94:0/1592666587 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faaa4072940 0x7faaa4074e00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faab0007b10 tx=0x7faab0009040 comp rx=0 tx=0).stop 2024-09-10T23:36:19.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.002+0000 7faacd000640 1 -- 172.21.15.94:0/1592666587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac8071a10 msgr2=0x7faac81f73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:19.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.002+0000 7faacd000640 1 --2- 172.21.15.94:0/1592666587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac8071a10 0x7faac81f73e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7faab800c730 tx=0x7faab800cbf0 comp rx=0 tx=0).stop 2024-09-10T23:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.002+0000 7faacd000640 1 -- 172.21.15.94:0/1592666587 shutdown_connections 2024-09-10T23:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.002+0000 7faacd000640 1 --2- 172.21.15.94:0/1592666587 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7faaa4072940 0x7faaa4074e00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.002+0000 7faacd000640 1 --2- 172.21.15.94:0/1592666587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faac80ccf30 0x7faac80aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.002+0000 7faacd000640 1 --2- 172.21.15.94:0/1592666587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac80afe60 0x7faac80c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.002+0000 7faacd000640 1 --2- 172.21.15.94:0/1592666587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac8071a10 0x7faac81f73e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.002+0000 7faacd000640 1 -- 172.21.15.94:0/1592666587 >> 172.21.15.94:0/1592666587 conn(0x7faac80d0a40 msgr2=0x7faac80cbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.002+0000 7faacd000640 1 -- 172.21.15.94:0/1592666587 shutdown_connections 2024-09-10T23:36:19.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.002+0000 7faacd000640 1 -- 172.21.15.94:0/1592666587 wait complete. 2024-09-10T23:36:19.045 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T23:36:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.190+0000 7f4d47400640 1 -- 172.21.15.94:0/426923576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d400d01a0 msgr2=0x7f4d400b16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.190+0000 7f4d47400640 1 --2- 172.21.15.94:0/426923576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d400d01a0 0x7f4d400b16f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009bf0 tx=0x7f4d3801cba0 comp rx=0 tx=0).stop 2024-09-10T23:36:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.190+0000 7f4d47400640 1 -- 172.21.15.94:0/426923576 shutdown_connections 2024-09-10T23:36:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.190+0000 7f4d47400640 1 --2- 172.21.15.94:0/426923576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d400d01a0 0x7f4d400b16f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.190+0000 7f4d47400640 1 --2- 172.21.15.94:0/426923576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d4015f180 0x7f4d400cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.190+0000 7f4d47400640 1 -- 172.21.15.94:0/426923576 >> 172.21.15.94:0/426923576 conn(0x7f4d400d1cb0 msgr2=0x7f4d400d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.190+0000 7f4d47400640 1 -- 172.21.15.94:0/426923576 shutdown_connections 2024-09-10T23:36:19.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.190+0000 7f4d47400640 1 -- 172.21.15.94:0/426923576 wait complete. 2024-09-10T23:36:19.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.190+0000 7f4d47400640 1 Processor -- start 2024-09-10T23:36:19.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.190+0000 7f4d47400640 1 -- start start 2024-09-10T23:36:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d47400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d400d01a0 0x7f4d400b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d47400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d4015f180 0x7f4d400b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d47400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d4015f560 0x7f4d4014c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d47400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d40168e90 con 0x7f4d4015f180 2024-09-10T23:36:19.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d47400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4d40168d10 con 0x7f4d400d01a0 2024-09-10T23:36:19.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d47400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f4d40169010 con 0x7f4d4015f560 2024-09-10T23:36:19.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d37e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d4015f180 0x7f4d400b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:19.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d37e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d4015f180 0x7f4d400b67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55072/0 (socket says 172.21.15.94:55072) 2024-09-10T23:36:19.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d37e00640 1 -- 172.21.15.94:0/1374566785 learned_addr learned my addr 172.21.15.94:0/1374566785 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:19.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d44a00640 1 --2- 172.21.15.94:0/1374566785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d400d01a0 0x7f4d400b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:19.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d45400640 1 --2- 172.21.15.94:0/1374566785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d4015f560 0x7f4d4014c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:19.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d37e00640 1 -- 172.21.15.94:0/1374566785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d4015f560 msgr2=0x7f4d4014c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:19.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d37e00640 1 --2- 172.21.15.94:0/1374566785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d4015f560 0x7f4d4014c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:19.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d37e00640 1 -- 172.21.15.94:0/1374566785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d400d01a0 msgr2=0x7f4d400b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:19.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d37e00640 1 --2- 172.21.15.94:0/1374566785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d400d01a0 0x7f4d400b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:19.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d37e00640 1 -- 172.21.15.94:0/1374566785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4015c3d0 con 0x7f4d4015f180 2024-09-10T23:36:19.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d37e00640 1 --2- 172.21.15.94:0/1374566785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d4015f180 0x7f4d400b67e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800c430 tx=0x7f4d2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:19.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d3d600640 1 -- 172.21.15.94:0/1374566785 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28011790 con 0x7f4d4015f180 2024-09-10T23:36:19.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d3d600640 1 -- 172.21.15.94:0/1374566785 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d28017070 con 0x7f4d4015f180 2024-09-10T23:36:19.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d47400640 1 -- 172.21.15.94:0/1374566785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4015c6b0 con 0x7f4d4015f180 2024-09-10T23:36:19.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d3d600640 1 -- 172.21.15.94:0/1374566785 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28011d00 con 0x7f4d4015f180 2024-09-10T23:36:19.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d47400640 1 -- 172.21.15.94:0/1374566785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d400b7480 con 0x7f4d4015f180 2024-09-10T23:36:19.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.194+0000 7f4d47400640 1 -- 172.21.15.94:0/1374566785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d400b16f0 con 0x7f4d4015f180 2024-09-10T23:36:19.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.198+0000 7f4d3d600640 1 -- 172.21.15.94:0/1374566785 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f4d2801c460 con 0x7f4d4015f180 2024-09-10T23:36:19.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.198+0000 7f4d3d600640 1 --2- 172.21.15.94:0/1374566785 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4d1c072940 0x7f4d1c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:19.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.198+0000 7f4d3d600640 1 -- 172.21.15.94:0/1374566785 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5609+0+0 (secure 0 0 0) 0x7f4d28093f40 con 0x7f4d4015f180 2024-09-10T23:36:19.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.198+0000 7f4d44a00640 1 --2- 172.21.15.94:0/1374566785 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4d1c072940 0x7f4d1c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:19.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.198+0000 7f4d44a00640 1 --2- 172.21.15.94:0/1374566785 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4d1c072940 0x7f4d1c074e00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000bc00 tx=0x7f4d3000b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:19.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.202+0000 7f4d3d600640 1 -- 172.21.15.94:0/1374566785 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4d2805eb50 con 0x7f4d4015f180 2024-09-10T23:36:19.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:19.402+0000 7f4d47400640 1 -- 172.21.15.94:0/1374566785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4d400b4300 con 0x7f4d4015f180 2024-09-10T23:36:20.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.022+0000 7f4d3d600640 1 -- 172.21.15.94:0/1374566785 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v40) ==== 174+0+0 (secure 0 0 0) 0x7f4d28063580 con 0x7f4d4015f180 2024-09-10T23:36:20.032 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 10 and data pool 11 2024-09-10T23:36:20.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.034+0000 7f4d47400640 1 -- 172.21.15.94:0/1374566785 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4d1c072940 msgr2=0x7f4d1c074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:20.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.034+0000 7f4d47400640 1 --2- 172.21.15.94:0/1374566785 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4d1c072940 0x7f4d1c074e00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000bc00 tx=0x7f4d3000b390 comp rx=0 tx=0).stop 2024-09-10T23:36:20.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.034+0000 7f4d47400640 1 -- 172.21.15.94:0/1374566785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d4015f180 msgr2=0x7f4d400b67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:20.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.034+0000 7f4d47400640 1 --2- 172.21.15.94:0/1374566785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d4015f180 0x7f4d400b67e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800c430 tx=0x7f4d2800c8f0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.034+0000 7f4d47400640 1 -- 172.21.15.94:0/1374566785 shutdown_connections 2024-09-10T23:36:20.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.034+0000 7f4d47400640 1 --2- 172.21.15.94:0/1374566785 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4d1c072940 0x7f4d1c074e00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.034+0000 7f4d47400640 1 --2- 172.21.15.94:0/1374566785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d4015f560 0x7f4d4014c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.034+0000 7f4d47400640 1 --2- 172.21.15.94:0/1374566785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d4015f180 0x7f4d400b67e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.034+0000 7f4d47400640 1 --2- 172.21.15.94:0/1374566785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d400d01a0 0x7f4d400b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.034+0000 7f4d47400640 1 -- 172.21.15.94:0/1374566785 >> 172.21.15.94:0/1374566785 conn(0x7f4d400d1cb0 msgr2=0x7f4d4014eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:20.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.034+0000 7f4d47400640 1 -- 172.21.15.94:0/1374566785 shutdown_connections 2024-09-10T23:36:20.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.034+0000 7f4d47400640 1 -- 172.21.15.94:0/1374566785 wait complete. 2024-09-10T23:36:20.064 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:36:20.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.206+0000 7f14fee00640 1 -- 172.21.15.94:0/2619496597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14dc004d50 msgr2=0x7f14dc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:20.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.206+0000 7f14fee00640 1 --2- 172.21.15.94:0/2619496597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14dc004d50 0x7f14dc0051d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f14f0001d60 tx=0x7f14f001cb20 comp rx=0 tx=0).stop 2024-09-10T23:36:20.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.206+0000 7f14fce00640 1 -- 172.21.15.94:0/2619496597 reap_dead start 2024-09-10T23:36:20.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.206+0000 7f14fee00640 1 -- 172.21.15.94:0/2619496597 shutdown_connections 2024-09-10T23:36:20.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.206+0000 7f14fee00640 1 -- 172.21.15.94:0/2619496597 >> 172.21.15.94:0/2619496597 conn(0x7f14f809aa70 msgr2=0x7f14f809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:20.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.206+0000 7f14fee00640 1 -- 172.21.15.94:0/2619496597 shutdown_connections 2024-09-10T23:36:20.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.206+0000 7f14fee00640 1 -- 172.21.15.94:0/2619496597 wait complete. 2024-09-10T23:36:20.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.206+0000 7f14fee00640 1 Processor -- start 2024-09-10T23:36:20.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fee00640 1 -- start start 2024-09-10T23:36:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fee00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14dc004d50 0x7f14f80c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fee00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14f8156dd0 0x7f14f8157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fee00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14f8146170 0x7f14f81465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fee00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f14f80b2aa0 con 0x7f14f8146170 2024-09-10T23:36:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fee00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f14f80b2920 con 0x7f14dc004d50 2024-09-10T23:36:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fee00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f14f80b2c20 con 0x7f14f8156dd0 2024-09-10T23:36:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14f8146170 0x7f14f81465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14f8146170 0x7f14f81465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55090/0 (socket says 172.21.15.94:55090) 2024-09-10T23:36:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fce00640 1 -- 172.21.15.94:0/2009029014 learned_addr learned my addr 172.21.15.94:0/2009029014 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14f7400640 1 --2- 172.21.15.94:0/2009029014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14f8156dd0 0x7f14f8157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14f7e00640 1 --2- 172.21.15.94:0/2009029014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14dc004d50 0x7f14f80c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fce00640 1 -- 172.21.15.94:0/2009029014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14f8156dd0 msgr2=0x7f14f8157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:20.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fce00640 1 --2- 172.21.15.94:0/2009029014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14f8156dd0 0x7f14f8157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fce00640 1 -- 172.21.15.94:0/2009029014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14dc004d50 msgr2=0x7f14f80c5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:20.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fce00640 1 --2- 172.21.15.94:0/2009029014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14dc004d50 0x7f14f80c5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fce00640 1 -- 172.21.15.94:0/2009029014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f8159c30 con 0x7f14f8146170 2024-09-10T23:36:20.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fce00640 1 --2- 172.21.15.94:0/2009029014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14f8146170 0x7f14f81465f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f14f0001d60 tx=0x7f14f0003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:20.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14efe00640 1 -- 172.21.15.94:0/2009029014 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f00a5b30 con 0x7f14f8146170 2024-09-10T23:36:20.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14efe00640 1 -- 172.21.15.94:0/2009029014 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14f00a5dc0 con 0x7f14f8146170 2024-09-10T23:36:20.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14efe00640 1 -- 172.21.15.94:0/2009029014 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f0006330 con 0x7f14f8146170 2024-09-10T23:36:20.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fee00640 1 -- 172.21.15.94:0/2009029014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f8159f10 con 0x7f14f8146170 2024-09-10T23:36:20.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fee00640 1 -- 172.21.15.94:0/2009029014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f815a4d0 con 0x7f14f8146170 2024-09-10T23:36:20.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.210+0000 7f14fee00640 1 -- 172.21.15.94:0/2009029014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f80b1e70 con 0x7f14f8146170 2024-09-10T23:36:20.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.214+0000 7f14efe00640 1 -- 172.21.15.94:0/2009029014 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f14f00064d0 con 0x7f14f8146170 2024-09-10T23:36:20.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.214+0000 7f14efe00640 1 --2- 172.21.15.94:0/2009029014 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f14d8072940 0x7f14d8074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:20.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.214+0000 7f14efe00640 1 -- 172.21.15.94:0/2009029014 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f14f012cb20 con 0x7f14f8146170 2024-09-10T23:36:20.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.214+0000 7f14f7e00640 1 --2- 172.21.15.94:0/2009029014 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f14d8072940 0x7f14d8074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:20.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.214+0000 7f14f7e00640 1 --2- 172.21.15.94:0/2009029014 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f14d8072940 0x7f14d8074e00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f14e8003fb0 tx=0x7f14e80067b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:20.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.218+0000 7f14efe00640 1 -- 172.21.15.94:0/2009029014 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f14f00b4070 con 0x7f14f8146170 2024-09-10T23:36:20.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.386+0000 7f14fee00640 1 -- 172.21.15.94:0/2009029014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14f815a6a0 con 0x7f14f8146170 2024-09-10T23:36:20.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.386+0000 7f14efe00640 1 -- 172.21.15.94:0/2009029014 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16669 (secure 0 0 0) 0x7f14f0016830 con 0x7f14f8146170 2024-09-10T23:36:20.388 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:20.388 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":68,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:36:19.404280+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:36:17.484207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T23:36:18.356053+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:36:20.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.386+0000 7f14fee00640 1 -- 172.21.15.94:0/2009029014 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f14d8072940 msgr2=0x7f14d8074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:20.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.386+0000 7f14fee00640 1 --2- 172.21.15.94:0/2009029014 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f14d8072940 0x7f14d8074e00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f14e8003fb0 tx=0x7f14e80067b0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.386+0000 7f14fee00640 1 -- 172.21.15.94:0/2009029014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14f8146170 msgr2=0x7f14f81465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:20.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.386+0000 7f14fee00640 1 --2- 172.21.15.94:0/2009029014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14f8146170 0x7f14f81465f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f14f0001d60 tx=0x7f14f0003f80 comp rx=0 tx=0).stop 2024-09-10T23:36:20.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.386+0000 7f14fee00640 1 -- 172.21.15.94:0/2009029014 shutdown_connections 2024-09-10T23:36:20.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.386+0000 7f14fee00640 1 --2- 172.21.15.94:0/2009029014 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f14d8072940 0x7f14d8074e00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.386+0000 7f14fee00640 1 --2- 172.21.15.94:0/2009029014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14f8146170 0x7f14f81465f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.386+0000 7f14fee00640 1 --2- 172.21.15.94:0/2009029014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14f8156dd0 0x7f14f8157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.386+0000 7f14fee00640 1 --2- 172.21.15.94:0/2009029014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14dc004d50 0x7f14f80c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.390+0000 7f14fee00640 1 -- 172.21.15.94:0/2009029014 >> 172.21.15.94:0/2009029014 conn(0x7f14f809aa70 msgr2=0x7f14f815d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:20.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.390+0000 7f14fee00640 1 -- 172.21.15.94:0/2009029014 shutdown_connections 2024-09-10T23:36:20.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.390+0000 7f14fee00640 1 -- 172.21.15.94:0/2009029014 wait complete. 2024-09-10T23:36:20.409 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:36:20.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccee000640 1 -- 172.21.15.94:0/2956243247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce80d03c0 msgr2=0x7fcce815e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:20.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccee000640 1 --2- 172.21.15.94:0/2956243247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce80d03c0 0x7fcce815e590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009840 tx=0x7fccd001c760 comp rx=0 tx=0).stop 2024-09-10T23:36:20.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccee000640 1 -- 172.21.15.94:0/2956243247 shutdown_connections 2024-09-10T23:36:20.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccee000640 1 --2- 172.21.15.94:0/2956243247 >> v2:172.21.15.181:3300/0 conn(0x7fcce815ead0 0x7fcce806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccee000640 1 --2- 172.21.15.94:0/2956243247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce80d03c0 0x7fcce815e590 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccee000640 1 -- 172.21.15.94:0/2956243247 >> 172.21.15.94:0/2956243247 conn(0x7fcce806cdf0 msgr2=0x7fcce806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:20.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccee000640 1 -- 172.21.15.94:0/2956243247 shutdown_connections 2024-09-10T23:36:20.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccee000640 1 -- 172.21.15.94:0/2956243247 wait complete. 2024-09-10T23:36:20.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccee000640 1 Processor -- start 2024-09-10T23:36:20.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccee000640 1 -- start start 2024-09-10T23:36:20.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccee000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce80cf3c0 0x7fcce8142e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:20.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccee000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce80d03c0 0x7fcce8158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:20.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccee000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce815ead0 0x7fcce8155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccee000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8167720 con 0x7fcce80d03c0 2024-09-10T23:36:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccee000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcce80706f0 con 0x7fcce815ead0 2024-09-10T23:36:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccee000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fcce81678a0 con 0x7fcce80cf3c0 2024-09-10T23:36:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccdea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce80d03c0 0x7fcce8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccdea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce80d03c0 0x7fcce8158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55098/0 (socket says 172.21.15.94:55098) 2024-09-10T23:36:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccdea00640 1 -- 172.21.15.94:0/3489909073 learned_addr learned my addr 172.21.15.94:0/3489909073 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fcce7e00640 1 --2- 172.21.15.94:0/3489909073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce815ead0 0x7fcce8155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fcce7400640 1 --2- 172.21.15.94:0/3489909073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce80cf3c0 0x7fcce8142e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccdea00640 1 -- 172.21.15.94:0/3489909073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce80cf3c0 msgr2=0x7fcce8142e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.550+0000 7fccdea00640 1 --2- 172.21.15.94:0/3489909073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce80cf3c0 0x7fcce8142e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.554+0000 7fccdea00640 1 -- 172.21.15.94:0/3489909073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce815ead0 msgr2=0x7fcce8155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.554+0000 7fccdea00640 1 --2- 172.21.15.94:0/3489909073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce815ead0 0x7fcce8155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.554+0000 7fccdea00640 1 -- 172.21.15.94:0/3489909073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce8157070 con 0x7fcce80d03c0 2024-09-10T23:36:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.554+0000 7fcce7e00640 1 --2- 172.21.15.94:0/3489909073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce815ead0 0x7fcce8155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.554+0000 7fccdea00640 1 --2- 172.21.15.94:0/3489909073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce80d03c0 0x7fcce8158110 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fccd001ae10 tx=0x7fccd00a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.554+0000 7fccdfe00640 1 -- 172.21.15.94:0/3489909073 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd00b2040 con 0x7fcce80d03c0 2024-09-10T23:36:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.554+0000 7fccdfe00640 1 -- 172.21.15.94:0/3489909073 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccd0004430 con 0x7fcce80d03c0 2024-09-10T23:36:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.554+0000 7fccdfe00640 1 -- 172.21.15.94:0/3489909073 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0005280 con 0x7fcce80d03c0 2024-09-10T23:36:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.554+0000 7fccee000640 1 -- 172.21.15.94:0/3489909073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce8157350 con 0x7fcce80d03c0 2024-09-10T23:36:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.554+0000 7fccee000640 1 -- 172.21.15.94:0/3489909073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce8157910 con 0x7fcce80d03c0 2024-09-10T23:36:20.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.554+0000 7fccee000640 1 -- 172.21.15.94:0/3489909073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce814d8d0 con 0x7fcce80d03c0 2024-09-10T23:36:20.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.554+0000 7fccdfe00640 1 -- 172.21.15.94:0/3489909073 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7fccd00b5070 con 0x7fcce80d03c0 2024-09-10T23:36:20.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.554+0000 7fccdfe00640 1 --2- 172.21.15.94:0/3489909073 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fccc4072940 0x7fccc4074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:20.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.554+0000 7fccdfe00640 1 -- 172.21.15.94:0/3489909073 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fccd012d110 con 0x7fcce80d03c0 2024-09-10T23:36:20.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.554+0000 7fcce7400640 1 --2- 172.21.15.94:0/3489909073 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fccc4072940 0x7fccc4074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:20.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.558+0000 7fcce7400640 1 --2- 172.21.15.94:0/3489909073 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fccc4072940 0x7fccc4074e00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcce8158ab0 tx=0x7fccd8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:20.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.558+0000 7fccdfe00640 1 -- 172.21.15.94:0/3489909073 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fccd0016410 con 0x7fcce80d03c0 2024-09-10T23:36:20.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.726+0000 7fccee000640 1 -- 172.21.15.94:0/3489909073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcce8069850 con 0x7fcce80d03c0 2024-09-10T23:36:20.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.726+0000 7fccdfe00640 1 -- 172.21.15.94:0/3489909073 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16669 (secure 0 0 0) 0x7fccd00f8290 con 0x7fcce80d03c0 2024-09-10T23:36:20.729 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:20.729 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":68,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:36:19.404280+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:36:17.484207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T23:36:18.356053+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:36:20.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.730+0000 7fccee000640 1 -- 172.21.15.94:0/3489909073 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fccc4072940 msgr2=0x7fccc4074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:20.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.730+0000 7fccee000640 1 --2- 172.21.15.94:0/3489909073 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fccc4072940 0x7fccc4074e00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcce8158ab0 tx=0x7fccd8009040 comp rx=0 tx=0).stop 2024-09-10T23:36:20.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.730+0000 7fccee000640 1 -- 172.21.15.94:0/3489909073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce80d03c0 msgr2=0x7fcce8158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:20.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.730+0000 7fccee000640 1 --2- 172.21.15.94:0/3489909073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce80d03c0 0x7fcce8158110 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fccd001ae10 tx=0x7fccd00a5730 comp rx=0 tx=0).stop 2024-09-10T23:36:20.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.730+0000 7fccee000640 1 -- 172.21.15.94:0/3489909073 shutdown_connections 2024-09-10T23:36:20.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.730+0000 7fccee000640 1 --2- 172.21.15.94:0/3489909073 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fccc4072940 0x7fccc4074e00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.730+0000 7fccee000640 1 --2- 172.21.15.94:0/3489909073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce815ead0 0x7fcce8155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.730+0000 7fccee000640 1 --2- 172.21.15.94:0/3489909073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcce80d03c0 0x7fcce8158110 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.730+0000 7fccee000640 1 --2- 172.21.15.94:0/3489909073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce80cf3c0 0x7fcce8142e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.730+0000 7fccee000640 1 -- 172.21.15.94:0/3489909073 >> 172.21.15.94:0/3489909073 conn(0x7fcce806cdf0 msgr2=0x7fcce815a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:20.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.730+0000 7fccee000640 1 -- 172.21.15.94:0/3489909073 shutdown_connections 2024-09-10T23:36:20.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.730+0000 7fccee000640 1 -- 172.21.15.94:0/3489909073 wait complete. 2024-09-10T23:36:20.751 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T23:36:20.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.894+0000 7f389fe00640 1 -- 172.21.15.94:0/2534651569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a015f140 msgr2=0x7f38a00b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:20.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.894+0000 7f389fe00640 1 --2- 172.21.15.94:0/2534651569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a015f140 0x7f38a00b1170 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3890009840 tx=0x7f389001c7f0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.894+0000 7f389fe00640 1 -- 172.21.15.94:0/2534651569 shutdown_connections 2024-09-10T23:36:20.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.894+0000 7f389fe00640 1 --2- 172.21.15.94:0/2534651569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38a00b2ac0 0x7f38a00b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.894+0000 7f389fe00640 1 --2- 172.21.15.94:0/2534651569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a015f140 0x7f38a00b1170 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.894+0000 7f389fe00640 1 -- 172.21.15.94:0/2534651569 >> 172.21.15.94:0/2534651569 conn(0x7f38a00d1b00 msgr2=0x7f38a00d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:20.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.894+0000 7f389fe00640 1 -- 172.21.15.94:0/2534651569 shutdown_connections 2024-09-10T23:36:20.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.894+0000 7f389fe00640 1 -- 172.21.15.94:0/2534651569 wait complete. 2024-09-10T23:36:20.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389fe00640 1 Processor -- start 2024-09-10T23:36:20.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389fe00640 1 -- start start 2024-09-10T23:36:20.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389fe00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38a00b1bf0 0x7f38a015d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:20.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38a00b2ac0 0x7f38a00c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:20.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a015f140 0x7f38a014e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0168ad0 con 0x7f38a015f140 2024-09-10T23:36:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389fe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0168950 con 0x7f38a00b2ac0 2024-09-10T23:36:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389fe00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f38a0168c50 con 0x7f38a00b1bf0 2024-09-10T23:36:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a015f140 0x7f38a014e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a015f140 0x7f38a014e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55114/0 (socket says 172.21.15.94:55114) 2024-09-10T23:36:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389f400640 1 -- 172.21.15.94:0/4060629253 learned_addr learned my addr 172.21.15.94:0/4060629253 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389ea00640 1 --2- 172.21.15.94:0/4060629253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38a00b1bf0 0x7f38a015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389f400640 1 -- 172.21.15.94:0/4060629253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38a00b1bf0 msgr2=0x7f38a015d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f3896000640 1 --2- 172.21.15.94:0/4060629253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38a00b2ac0 0x7f38a00c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389f400640 1 --2- 172.21.15.94:0/4060629253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38a00b1bf0 0x7f38a015d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389f400640 1 -- 172.21.15.94:0/4060629253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38a00b2ac0 msgr2=0x7f38a00c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389f400640 1 --2- 172.21.15.94:0/4060629253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38a00b2ac0 0x7f38a00c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:20.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389f400640 1 -- 172.21.15.94:0/4060629253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a00c5d50 con 0x7f38a015f140 2024-09-10T23:36:20.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389ea00640 1 --2- 172.21.15.94:0/4060629253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38a00b1bf0 0x7f38a015d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:36:20.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f3896000640 1 --2- 172.21.15.94:0/4060629253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38a00b2ac0 0x7f38a00c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:36:20.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389f400640 1 --2- 172.21.15.94:0/4060629253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a015f140 0x7f38a014e8f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f389800c7f0 tx=0x7f389800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:20.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f3897400640 1 -- 172.21.15.94:0/4060629253 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898017070 con 0x7f38a015f140 2024-09-10T23:36:20.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f3897400640 1 -- 172.21.15.94:0/4060629253 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3898005070 con 0x7f38a015f140 2024-09-10T23:36:20.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f3897400640 1 -- 172.21.15.94:0/4060629253 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38980054e0 con 0x7f38a015f140 2024-09-10T23:36:20.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389fe00640 1 -- 172.21.15.94:0/4060629253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a00af7e0 con 0x7f38a015f140 2024-09-10T23:36:20.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389fe00640 1 -- 172.21.15.94:0/4060629253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a015c900 con 0x7f38a015f140 2024-09-10T23:36:20.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.898+0000 7f389fe00640 1 -- 172.21.15.94:0/4060629253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a00cfa00 con 0x7f38a015f140 2024-09-10T23:36:20.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.902+0000 7f3897400640 1 -- 172.21.15.94:0/4060629253 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f3898005680 con 0x7f38a015f140 2024-09-10T23:36:20.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.902+0000 7f3897400640 1 --2- 172.21.15.94:0/4060629253 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f387c072940 0x7f387c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:20.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.902+0000 7f3897400640 1 -- 172.21.15.94:0/4060629253 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f3898093260 con 0x7f38a015f140 2024-09-10T23:36:20.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.902+0000 7f389ea00640 1 --2- 172.21.15.94:0/4060629253 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f387c072940 0x7f387c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:20.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.902+0000 7f389ea00640 1 --2- 172.21.15.94:0/4060629253 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f387c072940 0x7f387c074e00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f38900af000 tx=0x7f389001b030 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:20.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:20.906+0000 7f3897400640 1 -- 172.21.15.94:0/4060629253 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f389805e3e0 con 0x7f38a015f140 2024-09-10T23:36:21.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:21.122+0000 7f389fe00640 1 -- 172.21.15.94:0/4060629253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f38a0153400 con 0x7f38a015f140 2024-09-10T23:36:21.988 INFO:tasks.ceph.mds.d.smithi181.stderr:2024-09-10T23:36:21.986+0000 7fca07a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:36:22.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.022+0000 7f3897400640 1 -- 172.21.15.94:0/4060629253 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) ==== 118+0+0 (secure 0 0 0) 0x7f389805dce0 con 0x7f38a015f140 2024-09-10T23:36:22.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.026+0000 7f389fe00640 1 -- 172.21.15.94:0/4060629253 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f387c072940 msgr2=0x7f387c074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:22.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.026+0000 7f389fe00640 1 --2- 172.21.15.94:0/4060629253 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f387c072940 0x7f387c074e00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f38900af000 tx=0x7f389001b030 comp rx=0 tx=0).stop 2024-09-10T23:36:22.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.026+0000 7f389fe00640 1 -- 172.21.15.94:0/4060629253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a015f140 msgr2=0x7f38a014e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:22.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.026+0000 7f389fe00640 1 --2- 172.21.15.94:0/4060629253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a015f140 0x7f38a014e8f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f389800c7f0 tx=0x7f389800ccb0 comp rx=0 tx=0).stop 2024-09-10T23:36:22.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.026+0000 7f389fe00640 1 -- 172.21.15.94:0/4060629253 shutdown_connections 2024-09-10T23:36:22.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.026+0000 7f389fe00640 1 --2- 172.21.15.94:0/4060629253 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f387c072940 0x7f387c074e00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:22.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.026+0000 7f389fe00640 1 --2- 172.21.15.94:0/4060629253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38a00b2ac0 0x7f38a00c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:22.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.026+0000 7f389fe00640 1 --2- 172.21.15.94:0/4060629253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a015f140 0x7f38a014e8f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:22.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.026+0000 7f389fe00640 1 --2- 172.21.15.94:0/4060629253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38a00b1bf0 0x7f38a015d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:22.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.026+0000 7f389fe00640 1 -- 172.21.15.94:0/4060629253 >> 172.21.15.94:0/4060629253 conn(0x7f38a00d1b00 msgr2=0x7f38a01519b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:22.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.026+0000 7f389fe00640 1 -- 172.21.15.94:0/4060629253 shutdown_connections 2024-09-10T23:36:22.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.026+0000 7f389fe00640 1 -- 172.21.15.94:0/4060629253 wait complete. 2024-09-10T23:36:22.047 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T23:36:22.048 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T23:36:22.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe88b400640 1 -- 172.21.15.94:0/2285291734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe868004010 msgr2=0x7fe868004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:22.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe88b400640 1 --2- 172.21.15.94:0/2285291734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe868004010 0x7fe868004810 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00c2e0 tx=0x7fe86c01f6e0 comp rx=0 tx=0).stop 2024-09-10T23:36:22.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe889400640 1 -- 172.21.15.94:0/2285291734 reap_dead start 2024-09-10T23:36:22.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe88b400640 1 -- 172.21.15.94:0/2285291734 shutdown_connections 2024-09-10T23:36:22.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe88b400640 1 -- 172.21.15.94:0/2285291734 >> 172.21.15.94:0/2285291734 conn(0x7fe8840d3ea0 msgr2=0x7fe8840d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:22.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe88b400640 1 -- 172.21.15.94:0/2285291734 shutdown_connections 2024-09-10T23:36:22.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe88b400640 1 -- 172.21.15.94:0/2285291734 wait complete. 2024-09-10T23:36:22.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe88b400640 1 Processor -- start 2024-09-10T23:36:22.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe88b400640 1 -- start start 2024-09-10T23:36:22.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe88b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe868004010 0x7fe884154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:22.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe88b400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe884160500 0x7fe88415e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:22.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe88b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88415cec0 0x7fe88415d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:22.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe88b400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe884166c40 con 0x7fe88415cec0 2024-09-10T23:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe88b400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe8840d4d10 con 0x7fe868004010 2024-09-10T23:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe88b400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe884166dc0 con 0x7fe884160500 2024-09-10T23:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe889400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88415cec0 0x7fe88415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe889400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88415cec0 0x7fe88415d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55124/0 (socket says 172.21.15.94:55124) 2024-09-10T23:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe889400640 1 -- 172.21.15.94:0/955811131 learned_addr learned my addr 172.21.15.94:0/955811131 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe888a00640 1 --2- 172.21.15.94:0/955811131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe868004010 0x7fe884154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.194+0000 7fe87be00640 1 --2- 172.21.15.94:0/955811131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe884160500 0x7fe88415e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.198+0000 7fe889400640 1 -- 172.21.15.94:0/955811131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe884160500 msgr2=0x7fe88415e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.198+0000 7fe889400640 1 --2- 172.21.15.94:0/955811131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe884160500 0x7fe88415e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.198+0000 7fe889400640 1 -- 172.21.15.94:0/955811131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe868004010 msgr2=0x7fe884154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.198+0000 7fe889400640 1 --2- 172.21.15.94:0/955811131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe868004010 0x7fe884154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.198+0000 7fe889400640 1 -- 172.21.15.94:0/955811131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88415ef20 con 0x7fe88415cec0 2024-09-10T23:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.198+0000 7fe889400640 1 --2- 172.21.15.94:0/955811131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88415cec0 0x7fe88415d300 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe8840b20e0 tx=0x7fe87c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:22.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.198+0000 7fe881600640 1 -- 172.21.15.94:0/955811131 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c004b30 con 0x7fe88415cec0 2024-09-10T23:36:22.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.198+0000 7fe881600640 1 -- 172.21.15.94:0/955811131 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe87c004cd0 con 0x7fe88415cec0 2024-09-10T23:36:22.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.198+0000 7fe88b400640 1 -- 172.21.15.94:0/955811131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88415f200 con 0x7fe88415cec0 2024-09-10T23:36:22.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.198+0000 7fe881600640 1 -- 172.21.15.94:0/955811131 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c003be0 con 0x7fe88415cec0 2024-09-10T23:36:22.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.198+0000 7fe88b400640 1 -- 172.21.15.94:0/955811131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe884155200 con 0x7fe88415cec0 2024-09-10T23:36:22.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.198+0000 7fe88b400640 1 -- 172.21.15.94:0/955811131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe884155d00 con 0x7fe88415cec0 2024-09-10T23:36:22.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.198+0000 7fe881600640 1 -- 172.21.15.94:0/955811131 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7fe87c018070 con 0x7fe88415cec0 2024-09-10T23:36:22.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.198+0000 7fe881600640 1 --2- 172.21.15.94:0/955811131 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe860072940 0x7fe860074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:22.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.198+0000 7fe881600640 1 -- 172.21.15.94:0/955811131 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fe87c093ff0 con 0x7fe88415cec0 2024-09-10T23:36:22.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.202+0000 7fe888a00640 1 --2- 172.21.15.94:0/955811131 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe860072940 0x7fe860074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:22.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.202+0000 7fe888a00640 1 --2- 172.21.15.94:0/955811131 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe860072940 0x7fe860074e00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe874000c10 tx=0x7fe874006b80 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:22.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.202+0000 7fe881600640 1 -- 172.21.15.94:0/955811131 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe87c05f100 con 0x7fe88415cec0 2024-09-10T23:36:22.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:22.402+0000 7fe88b400640 1 -- 172.21.15.94:0/955811131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe8840d0790 con 0x7fe88415cec0 2024-09-10T23:36:23.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.026+0000 7fe881600640 1 -- 172.21.15.94:0/955811131 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) ==== 122+0+0 (secure 0 0 0) 0x7fe87c05ea00 con 0x7fe88415cec0 2024-09-10T23:36:23.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.026+0000 7fe88b400640 1 -- 172.21.15.94:0/955811131 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe860072940 msgr2=0x7fe860074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:23.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.026+0000 7fe88b400640 1 --2- 172.21.15.94:0/955811131 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe860072940 0x7fe860074e00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe874000c10 tx=0x7fe874006b80 comp rx=0 tx=0).stop 2024-09-10T23:36:23.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.026+0000 7fe88b400640 1 -- 172.21.15.94:0/955811131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88415cec0 msgr2=0x7fe88415d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:23.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.026+0000 7fe88b400640 1 --2- 172.21.15.94:0/955811131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88415cec0 0x7fe88415d300 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe8840b20e0 tx=0x7fe87c00f3c0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.026+0000 7fe88b400640 1 -- 172.21.15.94:0/955811131 shutdown_connections 2024-09-10T23:36:23.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.026+0000 7fe88b400640 1 --2- 172.21.15.94:0/955811131 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe860072940 0x7fe860074e00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.026+0000 7fe88b400640 1 --2- 172.21.15.94:0/955811131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe88415cec0 0x7fe88415d300 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.026+0000 7fe88b400640 1 --2- 172.21.15.94:0/955811131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe884160500 0x7fe88415e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.026+0000 7fe88b400640 1 --2- 172.21.15.94:0/955811131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe868004010 0x7fe884154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.026+0000 7fe88b400640 1 -- 172.21.15.94:0/955811131 >> 172.21.15.94:0/955811131 conn(0x7fe8840d3ea0 msgr2=0x7fe88415b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:23.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.026+0000 7fe88b400640 1 -- 172.21.15.94:0/955811131 shutdown_connections 2024-09-10T23:36:23.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.026+0000 7fe88b400640 1 -- 172.21.15.94:0/955811131 wait complete. 2024-09-10T23:36:23.051 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T23:36:23.051 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:23.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.194+0000 7efc56c00640 1 -- 172.21.15.94:0/1005843917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc340041d0 msgr2=0x7efc34004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:23.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.194+0000 7efc56c00640 1 --2- 172.21.15.94:0/1005843917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc340041d0 0x7efc34004810 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7efc3800abc0 tx=0x7efc3801cd70 comp rx=0 tx=0).stop 2024-09-10T23:36:23.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.194+0000 7efc54c00640 1 -- 172.21.15.94:0/1005843917 reap_dead start 2024-09-10T23:36:23.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.194+0000 7efc56c00640 1 -- 172.21.15.94:0/1005843917 shutdown_connections 2024-09-10T23:36:23.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.194+0000 7efc56c00640 1 -- 172.21.15.94:0/1005843917 >> 172.21.15.94:0/1005843917 conn(0x7efc5006cdf0 msgr2=0x7efc5006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:23.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.194+0000 7efc56c00640 1 -- 172.21.15.94:0/1005843917 shutdown_connections 2024-09-10T23:36:23.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.194+0000 7efc56c00640 1 -- 172.21.15.94:0/1005843917 wait complete. 2024-09-10T23:36:23.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.194+0000 7efc56c00640 1 Processor -- start 2024-09-10T23:36:23.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.194+0000 7efc56c00640 1 -- start start 2024-09-10T23:36:23.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc56c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc340041d0 0x7efc501f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:23.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc56c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc500cb6e0 0x7efc50159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:23.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc56c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc501598b0 0x7efc501fb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:23.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc56c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc5015b6e0 con 0x7efc340041d0 2024-09-10T23:36:23.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc56c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efc5015b560 con 0x7efc500cb6e0 2024-09-10T23:36:23.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc56c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7efc50165e80 con 0x7efc501598b0 2024-09-10T23:36:23.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc4fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc340041d0 0x7efc501f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:23.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc4fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc340041d0 0x7efc501f8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55136/0 (socket says 172.21.15.94:55136) 2024-09-10T23:36:23.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc4fe00640 1 -- 172.21.15.94:0/3636681976 learned_addr learned my addr 172.21.15.94:0/3636681976 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:23.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc47400640 1 --2- 172.21.15.94:0/3636681976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc500cb6e0 0x7efc50159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:23.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc54c00640 1 --2- 172.21.15.94:0/3636681976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc501598b0 0x7efc501fb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:23.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc4fe00640 1 -- 172.21.15.94:0/3636681976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc501598b0 msgr2=0x7efc501fb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:23.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc4fe00640 1 --2- 172.21.15.94:0/3636681976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc501598b0 0x7efc501fb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc4fe00640 1 -- 172.21.15.94:0/3636681976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc500cb6e0 msgr2=0x7efc50159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:23.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc4fe00640 1 --2- 172.21.15.94:0/3636681976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc500cb6e0 0x7efc50159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc4fe00640 1 -- 172.21.15.94:0/3636681976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5015a460 con 0x7efc340041d0 2024-09-10T23:36:23.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc4fe00640 1 --2- 172.21.15.94:0/3636681976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc340041d0 0x7efc501f8ba0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7efc40001170 tx=0x7efc40001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:23.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc4cc00640 1 -- 172.21.15.94:0/3636681976 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40015020 con 0x7efc340041d0 2024-09-10T23:36:23.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc4cc00640 1 -- 172.21.15.94:0/3636681976 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc40011980 con 0x7efc340041d0 2024-09-10T23:36:23.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc56c00640 1 -- 172.21.15.94:0/3636681976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc501522c0 con 0x7efc340041d0 2024-09-10T23:36:23.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc4cc00640 1 -- 172.21.15.94:0/3636681976 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40011dd0 con 0x7efc340041d0 2024-09-10T23:36:23.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc56c00640 1 -- 172.21.15.94:0/3636681976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc50152880 con 0x7efc340041d0 2024-09-10T23:36:23.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc56c00640 1 -- 172.21.15.94:0/3636681976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc50159cc0 con 0x7efc340041d0 2024-09-10T23:36:23.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.198+0000 7efc4cc00640 1 -- 172.21.15.94:0/3636681976 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7efc4001a400 con 0x7efc340041d0 2024-09-10T23:36:23.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.202+0000 7efc4cc00640 1 --2- 172.21.15.94:0/3636681976 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efc2c072970 0x7efc2c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:23.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.202+0000 7efc4cc00640 1 -- 172.21.15.94:0/3636681976 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7efc40094540 con 0x7efc340041d0 2024-09-10T23:36:23.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.202+0000 7efc47400640 1 --2- 172.21.15.94:0/3636681976 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efc2c072970 0x7efc2c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:23.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.202+0000 7efc47400640 1 --2- 172.21.15.94:0/3636681976 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efc2c072970 0x7efc2c074e30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efc380af000 tx=0x7efc38019b10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:23.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.206+0000 7efc4cc00640 1 -- 172.21.15.94:0/3636681976 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efc50159cc0 con 0x7efc340041d0 2024-09-10T23:36:23.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.406+0000 7efc56c00640 1 -- 172.21.15.94:0/3636681976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc5015aa50 con 0x7efc340041d0 2024-09-10T23:36:23.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.406+0000 7efc4cc00640 1 -- 172.21.15.94:0/3636681976 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5238 (secure 0 0 0) 0x7efc40066e20 con 0x7efc340041d0 2024-09-10T23:36:23.407 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:23.407 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":43,"btime":"2024-09-10T23:36:23:025336+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3320339747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3320339747},{"type":"v1","addr":"172.21.15.94:6834","nonce":3320339747}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3123300514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3123300514},{"type":"v1","addr":"172.21.15.94:6835","nonce":3123300514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/339523200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":339523200},{"type":"v1","addr":"172.21.15.181:6837","nonce":339523200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:36:19.404249+0000","modified":"2024-09-10T23:36:22.404999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195389":{"gid":195389,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/2999113739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2999113739},{"type":"v1","addr":"172.21.15.181:6836","nonce":2999113739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195389,"qdb_cluster":[195389]},"id":5}]} 2024-09-10T23:36:23.408 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 43 2024-09-10T23:36:23.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.406+0000 7efc56c00640 1 -- 172.21.15.94:0/3636681976 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efc2c072970 msgr2=0x7efc2c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:23.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.406+0000 7efc56c00640 1 --2- 172.21.15.94:0/3636681976 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efc2c072970 0x7efc2c074e30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efc380af000 tx=0x7efc38019b10 comp rx=0 tx=0).stop 2024-09-10T23:36:23.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.406+0000 7efc56c00640 1 -- 172.21.15.94:0/3636681976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc340041d0 msgr2=0x7efc501f8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:23.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.406+0000 7efc56c00640 1 --2- 172.21.15.94:0/3636681976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc340041d0 0x7efc501f8ba0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7efc40001170 tx=0x7efc40001810 comp rx=0 tx=0).stop 2024-09-10T23:36:23.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.406+0000 7efc56c00640 1 -- 172.21.15.94:0/3636681976 shutdown_connections 2024-09-10T23:36:23.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.406+0000 7efc56c00640 1 --2- 172.21.15.94:0/3636681976 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7efc2c072970 0x7efc2c074e30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.406+0000 7efc56c00640 1 --2- 172.21.15.94:0/3636681976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc500cb6e0 0x7efc50159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.406+0000 7efc56c00640 1 --2- 172.21.15.94:0/3636681976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc501598b0 0x7efc501fb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.406+0000 7efc56c00640 1 --2- 172.21.15.94:0/3636681976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc340041d0 0x7efc501f8ba0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.406+0000 7efc56c00640 1 -- 172.21.15.94:0/3636681976 >> 172.21.15.94:0/3636681976 conn(0x7efc5006cdf0 msgr2=0x7efc50072c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:23.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.406+0000 7efc56c00640 1 -- 172.21.15.94:0/3636681976 shutdown_connections 2024-09-10T23:36:23.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.406+0000 7efc56c00640 1 -- 172.21.15.94:0/3636681976 wait complete. 2024-09-10T23:36:23.428 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:36:23.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdad4a00640 1 -- 172.21.15.94:0/3123271275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad00d0010 msgr2=0x7fdad015fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:23.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdad4a00640 1 --2- 172.21.15.94:0/3123271275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad00d0010 0x7fdad015fbe0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fdad006d820 tx=0x7fdab801eff0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdad4a00640 1 -- 172.21.15.94:0/3123271275 shutdown_connections 2024-09-10T23:36:23.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdad4a00640 1 --2- 172.21.15.94:0/3123271275 >> v2:172.21.15.181:3300/0 conn(0x7fdad0160120 0x7fdad0160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdad4a00640 1 --2- 172.21.15.94:0/3123271275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad00d0010 0x7fdad015fbe0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdad4a00640 1 -- 172.21.15.94:0/3123271275 >> 172.21.15.94:0/3123271275 conn(0x7fdad00d1bf0 msgr2=0x7fdad00d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:23.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdad4a00640 1 -- 172.21.15.94:0/3123271275 shutdown_connections 2024-09-10T23:36:23.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdad4a00640 1 -- 172.21.15.94:0/3123271275 wait complete. 2024-09-10T23:36:23.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdad4a00640 1 Processor -- start 2024-09-10T23:36:23.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdad4a00640 1 -- start start 2024-09-10T23:36:23.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdad4a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad00cfc30 0x7fdad015df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:23.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdad4a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdad00d0010 0x7fdad0150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:23.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdad4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad0160120 0x7fdad00af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:23.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdad4a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdad00b1c10 con 0x7fdad0160120 2024-09-10T23:36:23.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdad4a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdad00b1a90 con 0x7fdad00cfc30 2024-09-10T23:36:23.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdad4a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fdad00b1d90 con 0x7fdad00d0010 2024-09-10T23:36:23.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.570+0000 7fdace800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad0160120 0x7fdad00af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:23.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdace800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad0160120 0x7fdad00af950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55158/0 (socket says 172.21.15.94:55158) 2024-09-10T23:36:23.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdace800640 1 -- 172.21.15.94:0/2059827814 learned_addr learned my addr 172.21.15.94:0/2059827814 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:23.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdac5400640 1 --2- 172.21.15.94:0/2059827814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdad00d0010 0x7fdad0150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:23.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdacde00640 1 --2- 172.21.15.94:0/2059827814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad00cfc30 0x7fdad015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:23.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdace800640 1 -- 172.21.15.94:0/2059827814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdad00d0010 msgr2=0x7fdad0150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:23.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdace800640 1 --2- 172.21.15.94:0/2059827814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdad00d0010 0x7fdad0150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdace800640 1 -- 172.21.15.94:0/2059827814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad00cfc30 msgr2=0x7fdad015df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:23.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdace800640 1 --2- 172.21.15.94:0/2059827814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad00cfc30 0x7fdad015df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdace800640 1 -- 172.21.15.94:0/2059827814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad01505a0 con 0x7fdad0160120 2024-09-10T23:36:23.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdace800640 1 --2- 172.21.15.94:0/2059827814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad0160120 0x7fdad00af950 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fdac800e970 tx=0x7fdac800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:23.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdac6a00640 1 -- 172.21.15.94:0/2059827814 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8004d30 con 0x7fdad0160120 2024-09-10T23:36:23.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdac6a00640 1 -- 172.21.15.94:0/2059827814 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdac800cb20 con 0x7fdad0160120 2024-09-10T23:36:23.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdad4a00640 1 -- 172.21.15.94:0/2059827814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad01f54f0 con 0x7fdad0160120 2024-09-10T23:36:23.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdac6a00640 1 -- 172.21.15.94:0/2059827814 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8005020 con 0x7fdad0160120 2024-09-10T23:36:23.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdad4a00640 1 -- 172.21.15.94:0/2059827814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad01f4730 con 0x7fdad0160120 2024-09-10T23:36:23.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdad4a00640 1 -- 172.21.15.94:0/2059827814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad015fbe0 con 0x7fdad0160120 2024-09-10T23:36:23.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdac6a00640 1 -- 172.21.15.94:0/2059827814 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7fdac800f040 con 0x7fdad0160120 2024-09-10T23:36:23.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdac6a00640 1 --2- 172.21.15.94:0/2059827814 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdaac072940 0x7fdaac074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:23.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.574+0000 7fdac6a00640 1 -- 172.21.15.94:0/2059827814 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fdac8093ff0 con 0x7fdad0160120 2024-09-10T23:36:23.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.578+0000 7fdacde00640 1 --2- 172.21.15.94:0/2059827814 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdaac072940 0x7fdaac074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:23.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.578+0000 7fdacde00640 1 --2- 172.21.15.94:0/2059827814 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdaac072940 0x7fdaac074e00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdad00b1100 tx=0x7fdac0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:23.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.578+0000 7fdac6a00640 1 -- 172.21.15.94:0/2059827814 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdac8016030 con 0x7fdad0160120 2024-09-10T23:36:23.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.746+0000 7fdad4a00640 1 -- 172.21.15.94:0/2059827814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdad00d2480 con 0x7fdad0160120 2024-09-10T23:36:23.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.746+0000 7fdac6a00640 1 -- 172.21.15.94:0/2059827814 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16656 (secure 0 0 0) 0x7fdac805f100 con 0x7fdad0160120 2024-09-10T23:36:23.749 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:23.749 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":69,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:36:21.002358+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:36:17.484207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T23:36:18.356053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:36:23.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.750+0000 7fdad4a00640 1 -- 172.21.15.94:0/2059827814 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdaac072940 msgr2=0x7fdaac074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:23.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.750+0000 7fdad4a00640 1 --2- 172.21.15.94:0/2059827814 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdaac072940 0x7fdaac074e00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdad00b1100 tx=0x7fdac0009040 comp rx=0 tx=0).stop 2024-09-10T23:36:23.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.750+0000 7fdad4a00640 1 -- 172.21.15.94:0/2059827814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad0160120 msgr2=0x7fdad00af950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:23.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.750+0000 7fdad4a00640 1 --2- 172.21.15.94:0/2059827814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad0160120 0x7fdad00af950 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fdac800e970 tx=0x7fdac800ee30 comp rx=0 tx=0).stop 2024-09-10T23:36:23.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.750+0000 7fdad4a00640 1 -- 172.21.15.94:0/2059827814 shutdown_connections 2024-09-10T23:36:23.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.750+0000 7fdad4a00640 1 --2- 172.21.15.94:0/2059827814 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fdaac072940 0x7fdaac074e00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.750+0000 7fdad4a00640 1 --2- 172.21.15.94:0/2059827814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdad0160120 0x7fdad00af950 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.750+0000 7fdad4a00640 1 --2- 172.21.15.94:0/2059827814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdad00d0010 0x7fdad0150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.750+0000 7fdad4a00640 1 --2- 172.21.15.94:0/2059827814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdad00cfc30 0x7fdad015df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.750+0000 7fdad4a00640 1 -- 172.21.15.94:0/2059827814 >> 172.21.15.94:0/2059827814 conn(0x7fdad00d1bf0 msgr2=0x7fdad006dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:23.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.750+0000 7fdad4a00640 1 -- 172.21.15.94:0/2059827814 shutdown_connections 2024-09-10T23:36:23.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.750+0000 7fdad4a00640 1 -- 172.21.15.94:0/2059827814 wait complete. 2024-09-10T23:36:23.771 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T23:36:23.772 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:36:23.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.910+0000 7fed3ea00640 1 -- 172.21.15.94:0/1911847503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed3815a060 msgr2=0x7fed3806e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.910+0000 7fed3ea00640 1 --2- 172.21.15.94:0/1911847503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed3815a060 0x7fed3806e5c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fed2800aa10 tx=0x7fed2801db70 comp rx=0 tx=0).stop 2024-09-10T23:36:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.910+0000 7fed3ea00640 1 -- 172.21.15.94:0/1911847503 shutdown_connections 2024-09-10T23:36:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.910+0000 7fed3ea00640 1 --2- 172.21.15.94:0/1911847503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed3815e3a0 0x7fed38072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.910+0000 7fed3ea00640 1 --2- 172.21.15.94:0/1911847503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed3815a060 0x7fed3806e5c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.910+0000 7fed3ea00640 1 -- 172.21.15.94:0/1911847503 >> 172.21.15.94:0/1911847503 conn(0x7fed3806cdf0 msgr2=0x7fed3806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.910+0000 7fed3ea00640 1 -- 172.21.15.94:0/1911847503 shutdown_connections 2024-09-10T23:36:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.910+0000 7fed3ea00640 1 -- 172.21.15.94:0/1911847503 wait complete. 2024-09-10T23:36:23.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ea00640 1 Processor -- start 2024-09-10T23:36:23.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ea00640 1 -- start start 2024-09-10T23:36:23.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3815a060 0x7fed3813ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:23.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ea00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed3815e3a0 0x7fed3815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:23.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed3815e8a0 0x7fed3815b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:23.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ea00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fed380735c0 con 0x7fed3815e8a0 2024-09-10T23:36:23.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ea00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fed38073440 con 0x7fed3815a060 2024-09-10T23:36:23.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ea00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fed38073740 con 0x7fed3815e3a0 2024-09-10T23:36:23.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed3815e8a0 0x7fed3815b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:23.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed3815e8a0 0x7fed3815b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51014/0 (socket says 172.21.15.94:51014) 2024-09-10T23:36:23.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ca00640 1 -- 172.21.15.94:0/339312995 learned_addr learned my addr 172.21.15.94:0/339312995 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:23.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed37e00640 1 --2- 172.21.15.94:0/339312995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3815a060 0x7fed3813ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:23.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed37400640 1 --2- 172.21.15.94:0/339312995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed3815e3a0 0x7fed3815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:23.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ca00640 1 -- 172.21.15.94:0/339312995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed3815e3a0 msgr2=0x7fed3815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:23.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ca00640 1 --2- 172.21.15.94:0/339312995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed3815e3a0 0x7fed3815aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ca00640 1 -- 172.21.15.94:0/339312995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3815a060 msgr2=0x7fed3813ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:23.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ca00640 1 --2- 172.21.15.94:0/339312995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3815a060 0x7fed3813ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:23.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ca00640 1 -- 172.21.15.94:0/339312995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed38140920 con 0x7fed3815e8a0 2024-09-10T23:36:23.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed37e00640 1 --2- 172.21.15.94:0/339312995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3815a060 0x7fed3813ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:23.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ca00640 1 --2- 172.21.15.94:0/339312995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed3815e8a0 0x7fed3815b020 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fed3000da40 tx=0x7fed3000df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:23.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed2fe00640 1 -- 172.21.15.94:0/339312995 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30017070 con 0x7fed3815e8a0 2024-09-10T23:36:23.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed2fe00640 1 -- 172.21.15.94:0/339312995 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed30004b30 con 0x7fed3815e8a0 2024-09-10T23:36:23.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed2fe00640 1 -- 172.21.15.94:0/339312995 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30003980 con 0x7fed3815e8a0 2024-09-10T23:36:23.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ea00640 1 -- 172.21.15.94:0/339312995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed38140c00 con 0x7fed3815e8a0 2024-09-10T23:36:23.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ea00640 1 -- 172.21.15.94:0/339312995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed381416e0 con 0x7fed3815e8a0 2024-09-10T23:36:23.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.914+0000 7fed3ea00640 1 -- 172.21.15.94:0/339312995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed3806e5c0 con 0x7fed3815e8a0 2024-09-10T23:36:23.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.918+0000 7fed2fe00640 1 -- 172.21.15.94:0/339312995 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7fed30003b20 con 0x7fed3815e8a0 2024-09-10T23:36:23.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.918+0000 7fed2fe00640 1 --2- 172.21.15.94:0/339312995 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fed18072940 0x7fed18074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:23.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.918+0000 7fed2fe00640 1 -- 172.21.15.94:0/339312995 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fed30093ff0 con 0x7fed3815e8a0 2024-09-10T23:36:23.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.918+0000 7fed37e00640 1 --2- 172.21.15.94:0/339312995 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fed18072940 0x7fed18074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:23.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.918+0000 7fed37e00640 1 --2- 172.21.15.94:0/339312995 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fed18072940 0x7fed18074e00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fed280b0000 tx=0x7fed2801c3b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:23.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:23.922+0000 7fed2fe00640 1 -- 172.21.15.94:0/339312995 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fed30020020 con 0x7fed3815e8a0 2024-09-10T23:36:24.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.086+0000 7fed3ea00640 1 -- 172.21.15.94:0/339312995 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed3806d420 con 0x7fed18072940 2024-09-10T23:36:24.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.090+0000 7fed2fe00640 1 -- 172.21.15.94:0/339312995 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252059 (secure 0 0 0) 0x7fed280fa8b0 con 0x7fed18072940 2024-09-10T23:36:24.094 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:24.096 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:36:24.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.094+0000 7fed3ea00640 1 -- 172.21.15.94:0/339312995 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fed18072940 msgr2=0x7fed18074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:24.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.094+0000 7fed3ea00640 1 --2- 172.21.15.94:0/339312995 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fed18072940 0x7fed18074e00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fed280b0000 tx=0x7fed2801c3b0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.094+0000 7fed3ea00640 1 -- 172.21.15.94:0/339312995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed3815e8a0 msgr2=0x7fed3815b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:24.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.094+0000 7fed3ea00640 1 --2- 172.21.15.94:0/339312995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed3815e8a0 0x7fed3815b020 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fed3000da40 tx=0x7fed3000df00 comp rx=0 tx=0).stop 2024-09-10T23:36:24.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.094+0000 7fed3ea00640 1 -- 172.21.15.94:0/339312995 shutdown_connections 2024-09-10T23:36:24.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.094+0000 7fed3ea00640 1 --2- 172.21.15.94:0/339312995 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fed18072940 0x7fed18074e00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.094+0000 7fed3ea00640 1 --2- 172.21.15.94:0/339312995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed3815e8a0 0x7fed3815b020 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.094+0000 7fed3ea00640 1 --2- 172.21.15.94:0/339312995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed3815e3a0 0x7fed3815aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.094+0000 7fed3ea00640 1 --2- 172.21.15.94:0/339312995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed3815a060 0x7fed3813ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.094+0000 7fed3ea00640 1 -- 172.21.15.94:0/339312995 >> 172.21.15.94:0/339312995 conn(0x7fed3806cdf0 msgr2=0x7fed3806eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:24.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.094+0000 7fed3ea00640 1 -- 172.21.15.94:0/339312995 shutdown_connections 2024-09-10T23:36:24.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.094+0000 7fed3ea00640 1 -- 172.21.15.94:0/339312995 wait complete. 2024-09-10T23:36:24.114 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-10T23:36:23.400297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257216,"kb_used_data":7660,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717504,"statfs":{"total":773094113280,"available":772830724096,"internally_reserved":0,"allocated":7843840,"data_stored":3931128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.407919"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030891+0000","last_change":"2024-09-10T23:36:20.020146+0000","last_active":"2024-09-10T23:36:21.030891+0000","last_peered":"2024-09-10T23:36:21.030891+0000","last_clean":"2024-09-10T23:36:21.030891+0000","last_became_active":"2024-09-10T23:36:20.019811+0000","last_became_peered":"2024-09-10T23:36:20.019811+0000","last_unstale":"2024-09-10T23:36:21.030891+0000","last_undegraded":"2024-09-10T23:36:21.030891+0000","last_fullsized":"2024-09-10T23:36:21.030891+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:33.686403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287535+0000","last_change":"2024-09-10T23:36:20.021782+0000","last_active":"2024-09-10T23:36:21.287535+0000","last_peered":"2024-09-10T23:36:21.287535+0000","last_clean":"2024-09-10T23:36:21.287535+0000","last_became_active":"2024-09-10T23:36:20.019103+0000","last_became_peered":"2024-09-10T23:36:20.019103+0000","last_unstale":"2024-09-10T23:36:21.287535+0000","last_undegraded":"2024-09-10T23:36:21.287535+0000","last_fullsized":"2024-09-10T23:36:21.287535+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:05.271339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289254+0000","last_change":"2024-09-10T23:36:20.033946+0000","last_active":"2024-09-10T23:36:21.289254+0000","last_peered":"2024-09-10T23:36:21.289254+0000","last_clean":"2024-09-10T23:36:21.289254+0000","last_became_active":"2024-09-10T23:36:20.032470+0000","last_became_peered":"2024-09-10T23:36:20.032470+0000","last_unstale":"2024-09-10T23:36:21.289254+0000","last_undegraded":"2024-09-10T23:36:21.289254+0000","last_fullsized":"2024-09-10T23:36:21.289254+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:55.338717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288952+0000","last_change":"2024-09-10T23:36:20.032023+0000","last_active":"2024-09-10T23:36:21.288952+0000","last_peered":"2024-09-10T23:36:21.288952+0000","last_clean":"2024-09-10T23:36:21.288952+0000","last_became_active":"2024-09-10T23:36:20.031742+0000","last_became_peered":"2024-09-10T23:36:20.031742+0000","last_unstale":"2024-09-10T23:36:21.288952+0000","last_undegraded":"2024-09-10T23:36:21.288952+0000","last_fullsized":"2024-09-10T23:36:21.288952+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:01.221296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288936+0000","last_change":"2024-09-10T23:36:20.028753+0000","last_active":"2024-09-10T23:36:21.288936+0000","last_peered":"2024-09-10T23:36:21.288936+0000","last_clean":"2024-09-10T23:36:21.288936+0000","last_became_active":"2024-09-10T23:36:20.026131+0000","last_became_peered":"2024-09-10T23:36:20.026131+0000","last_unstale":"2024-09-10T23:36:21.288936+0000","last_undegraded":"2024-09-10T23:36:21.288936+0000","last_fullsized":"2024-09-10T23:36:21.288936+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:22.475012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.007208+0000","last_change":"2024-09-10T23:36:20.024515+0000","last_active":"2024-09-10T23:36:21.007208+0000","last_peered":"2024-09-10T23:36:21.007208+0000","last_clean":"2024-09-10T23:36:21.007208+0000","last_became_active":"2024-09-10T23:36:20.023422+0000","last_became_peered":"2024-09-10T23:36:20.023422+0000","last_unstale":"2024-09-10T23:36:21.007208+0000","last_undegraded":"2024-09-10T23:36:21.007208+0000","last_fullsized":"2024-09-10T23:36:21.007208+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:50.318502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033198+0000","last_change":"2024-09-10T23:36:20.020717+0000","last_active":"2024-09-10T23:36:21.033198+0000","last_peered":"2024-09-10T23:36:21.033198+0000","last_clean":"2024-09-10T23:36:21.033198+0000","last_became_active":"2024-09-10T23:36:20.020371+0000","last_became_peered":"2024-09-10T23:36:20.020371+0000","last_unstale":"2024-09-10T23:36:21.033198+0000","last_undegraded":"2024-09-10T23:36:21.033198+0000","last_fullsized":"2024-09-10T23:36:21.033198+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:43.823658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030220+0000","last_change":"2024-09-10T23:36:20.021691+0000","last_active":"2024-09-10T23:36:21.030220+0000","last_peered":"2024-09-10T23:36:21.030220+0000","last_clean":"2024-09-10T23:36:21.030220+0000","last_became_active":"2024-09-10T23:36:20.021345+0000","last_became_peered":"2024-09-10T23:36:20.021345+0000","last_unstale":"2024-09-10T23:36:21.030220+0000","last_undegraded":"2024-09-10T23:36:21.030220+0000","last_fullsized":"2024-09-10T23:36:21.030220+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:49.604517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288899+0000","last_change":"2024-09-10T23:36:20.037904+0000","last_active":"2024-09-10T23:36:21.288899+0000","last_peered":"2024-09-10T23:36:21.288899+0000","last_clean":"2024-09-10T23:36:21.288899+0000","last_became_active":"2024-09-10T23:36:20.037650+0000","last_became_peered":"2024-09-10T23:36:20.037650+0000","last_unstale":"2024-09-10T23:36:21.288899+0000","last_undegraded":"2024-09-10T23:36:21.288899+0000","last_fullsized":"2024-09-10T23:36:21.288899+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:51.458193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033626+0000","last_change":"2024-09-10T23:36:20.022961+0000","last_active":"2024-09-10T23:36:21.033626+0000","last_peered":"2024-09-10T23:36:21.033626+0000","last_clean":"2024-09-10T23:36:21.033626+0000","last_became_active":"2024-09-10T23:36:20.022524+0000","last_became_peered":"2024-09-10T23:36:20.022524+0000","last_unstale":"2024-09-10T23:36:21.033626+0000","last_undegraded":"2024-09-10T23:36:21.033626+0000","last_fullsized":"2024-09-10T23:36:21.033626+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:16.045657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030757+0000","last_change":"2024-09-10T23:36:20.029568+0000","last_active":"2024-09-10T23:36:21.030757+0000","last_peered":"2024-09-10T23:36:21.030757+0000","last_clean":"2024-09-10T23:36:21.030757+0000","last_became_active":"2024-09-10T23:36:20.029269+0000","last_became_peered":"2024-09-10T23:36:20.029269+0000","last_unstale":"2024-09-10T23:36:21.030757+0000","last_undegraded":"2024-09-10T23:36:21.030757+0000","last_fullsized":"2024-09-10T23:36:21.030757+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:25:46.983237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.009377+0000","last_change":"2024-09-10T23:36:20.029394+0000","last_active":"2024-09-10T23:36:21.009377+0000","last_peered":"2024-09-10T23:36:21.009377+0000","last_clean":"2024-09-10T23:36:21.009377+0000","last_became_active":"2024-09-10T23:36:20.029079+0000","last_became_peered":"2024-09-10T23:36:20.029079+0000","last_unstale":"2024-09-10T23:36:21.009377+0000","last_undegraded":"2024-09-10T23:36:21.009377+0000","last_fullsized":"2024-09-10T23:36:21.009377+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:23.949351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288680+0000","last_change":"2024-09-10T23:36:20.036241+0000","last_active":"2024-09-10T23:36:21.288680+0000","last_peered":"2024-09-10T23:36:21.288680+0000","last_clean":"2024-09-10T23:36:21.288680+0000","last_became_active":"2024-09-10T23:36:20.035965+0000","last_became_peered":"2024-09-10T23:36:20.035965+0000","last_unstale":"2024-09-10T23:36:21.288680+0000","last_undegraded":"2024-09-10T23:36:21.288680+0000","last_fullsized":"2024-09-10T23:36:21.288680+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:40.613548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.009053+0000","last_change":"2024-09-10T23:36:20.025057+0000","last_active":"2024-09-10T23:36:21.009053+0000","last_peered":"2024-09-10T23:36:21.009053+0000","last_clean":"2024-09-10T23:36:21.009053+0000","last_became_active":"2024-09-10T23:36:20.024588+0000","last_became_peered":"2024-09-10T23:36:20.024588+0000","last_unstale":"2024-09-10T23:36:21.009053+0000","last_undegraded":"2024-09-10T23:36:21.009053+0000","last_fullsized":"2024-09-10T23:36:21.009053+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:05.271339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.008184+0000","last_change":"2024-09-10T23:36:20.019283+0000","last_active":"2024-09-10T23:36:21.008184+0000","last_peered":"2024-09-10T23:36:21.008184+0000","last_clean":"2024-09-10T23:36:21.008184+0000","last_became_active":"2024-09-10T23:36:20.018869+0000","last_became_peered":"2024-09-10T23:36:20.018869+0000","last_unstale":"2024-09-10T23:36:21.008184+0000","last_undegraded":"2024-09-10T23:36:21.008184+0000","last_fullsized":"2024-09-10T23:36:21.008184+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:22.038276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030189+0000","last_change":"2024-09-10T23:36:20.030690+0000","last_active":"2024-09-10T23:36:21.030189+0000","last_peered":"2024-09-10T23:36:21.030189+0000","last_clean":"2024-09-10T23:36:21.030189+0000","last_became_active":"2024-09-10T23:36:20.030391+0000","last_became_peered":"2024-09-10T23:36:20.030391+0000","last_unstale":"2024-09-10T23:36:21.030189+0000","last_undegraded":"2024-09-10T23:36:21.030189+0000","last_fullsized":"2024-09-10T23:36:21.030189+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:20.396257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289342+0000","last_change":"2024-09-10T23:36:20.039019+0000","last_active":"2024-09-10T23:36:21.289342+0000","last_peered":"2024-09-10T23:36:21.289342+0000","last_clean":"2024-09-10T23:36:21.289342+0000","last_became_active":"2024-09-10T23:36:20.038402+0000","last_became_peered":"2024-09-10T23:36:20.038402+0000","last_unstale":"2024-09-10T23:36:21.289342+0000","last_undegraded":"2024-09-10T23:36:21.289342+0000","last_fullsized":"2024-09-10T23:36:21.289342+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:26.191671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289584+0000","last_change":"2024-09-10T23:36:20.027947+0000","last_active":"2024-09-10T23:36:21.289584+0000","last_peered":"2024-09-10T23:36:21.289584+0000","last_clean":"2024-09-10T23:36:21.289584+0000","last_became_active":"2024-09-10T23:36:20.027680+0000","last_became_peered":"2024-09-10T23:36:20.027680+0000","last_unstale":"2024-09-10T23:36:21.289584+0000","last_undegraded":"2024-09-10T23:36:21.289584+0000","last_fullsized":"2024-09-10T23:36:21.289584+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:09.479231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030644+0000","last_change":"2024-09-10T23:36:20.038789+0000","last_active":"2024-09-10T23:36:21.030644+0000","last_peered":"2024-09-10T23:36:21.030644+0000","last_clean":"2024-09-10T23:36:21.030644+0000","last_became_active":"2024-09-10T23:36:20.038398+0000","last_became_peered":"2024-09-10T23:36:20.038398+0000","last_unstale":"2024-09-10T23:36:21.030644+0000","last_undegraded":"2024-09-10T23:36:21.030644+0000","last_fullsized":"2024-09-10T23:36:21.030644+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:19.927593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288656+0000","last_change":"2024-09-10T23:36:20.029210+0000","last_active":"2024-09-10T23:36:21.288656+0000","last_peered":"2024-09-10T23:36:21.288656+0000","last_clean":"2024-09-10T23:36:21.288656+0000","last_became_active":"2024-09-10T23:36:20.028830+0000","last_became_peered":"2024-09-10T23:36:20.028830+0000","last_unstale":"2024-09-10T23:36:21.288656+0000","last_undegraded":"2024-09-10T23:36:21.288656+0000","last_fullsized":"2024-09-10T23:36:21.288656+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:43.055413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287405+0000","last_change":"2024-09-10T23:36:20.027209+0000","last_active":"2024-09-10T23:36:21.287405+0000","last_peered":"2024-09-10T23:36:21.287405+0000","last_clean":"2024-09-10T23:36:21.287405+0000","last_became_active":"2024-09-10T23:36:20.026847+0000","last_became_peered":"2024-09-10T23:36:20.026847+0000","last_unstale":"2024-09-10T23:36:21.287405+0000","last_undegraded":"2024-09-10T23:36:21.287405+0000","last_fullsized":"2024-09-10T23:36:21.287405+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:27:38.276538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.006968+0000","last_change":"2024-09-10T23:36:20.025152+0000","last_active":"2024-09-10T23:36:21.006968+0000","last_peered":"2024-09-10T23:36:21.006968+0000","last_clean":"2024-09-10T23:36:21.006968+0000","last_became_active":"2024-09-10T23:36:20.024258+0000","last_became_peered":"2024-09-10T23:36:20.024258+0000","last_unstale":"2024-09-10T23:36:21.006968+0000","last_undegraded":"2024-09-10T23:36:21.006968+0000","last_fullsized":"2024-09-10T23:36:21.006968+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:00.660346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033143+0000","last_change":"2024-09-10T23:36:20.016548+0000","last_active":"2024-09-10T23:36:21.033143+0000","last_peered":"2024-09-10T23:36:21.033143+0000","last_clean":"2024-09-10T23:36:21.033143+0000","last_became_active":"2024-09-10T23:36:20.016257+0000","last_became_peered":"2024-09-10T23:36:20.016257+0000","last_unstale":"2024-09-10T23:36:21.033143+0000","last_undegraded":"2024-09-10T23:36:21.033143+0000","last_fullsized":"2024-09-10T23:36:21.033143+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:34.056422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030362+0000","last_change":"2024-09-10T23:36:20.030646+0000","last_active":"2024-09-10T23:36:21.030362+0000","last_peered":"2024-09-10T23:36:21.030362+0000","last_clean":"2024-09-10T23:36:21.030362+0000","last_became_active":"2024-09-10T23:36:20.030051+0000","last_became_peered":"2024-09-10T23:36:20.030051+0000","last_unstale":"2024-09-10T23:36:21.030362+0000","last_undegraded":"2024-09-10T23:36:21.030362+0000","last_fullsized":"2024-09-10T23:36:21.030362+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:24.939026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033630+0000","last_change":"2024-09-10T23:36:20.016660+0000","last_active":"2024-09-10T23:36:21.033630+0000","last_peered":"2024-09-10T23:36:21.033630+0000","last_clean":"2024-09-10T23:36:21.033630+0000","last_became_active":"2024-09-10T23:36:20.016352+0000","last_became_peered":"2024-09-10T23:36:20.016352+0000","last_unstale":"2024-09-10T23:36:21.033630+0000","last_undegraded":"2024-09-10T23:36:21.033630+0000","last_fullsized":"2024-09-10T23:36:21.033630+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:48:37.118518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030712+0000","last_change":"2024-09-10T23:36:20.035385+0000","last_active":"2024-09-10T23:36:21.030712+0000","last_peered":"2024-09-10T23:36:21.030712+0000","last_clean":"2024-09-10T23:36:21.030712+0000","last_became_active":"2024-09-10T23:36:20.035012+0000","last_became_peered":"2024-09-10T23:36:20.035012+0000","last_unstale":"2024-09-10T23:36:21.030712+0000","last_undegraded":"2024-09-10T23:36:21.030712+0000","last_fullsized":"2024-09-10T23:36:21.030712+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:49.694876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289026+0000","last_change":"2024-09-10T23:36:20.025247+0000","last_active":"2024-09-10T23:36:21.289026+0000","last_peered":"2024-09-10T23:36:21.289026+0000","last_clean":"2024-09-10T23:36:21.289026+0000","last_became_active":"2024-09-10T23:36:20.024632+0000","last_became_peered":"2024-09-10T23:36:20.024632+0000","last_unstale":"2024-09-10T23:36:21.289026+0000","last_undegraded":"2024-09-10T23:36:21.289026+0000","last_fullsized":"2024-09-10T23:36:21.289026+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:30.604978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287415+0000","last_change":"2024-09-10T23:36:20.029466+0000","last_active":"2024-09-10T23:36:21.287415+0000","last_peered":"2024-09-10T23:36:21.287415+0000","last_clean":"2024-09-10T23:36:21.287415+0000","last_became_active":"2024-09-10T23:36:20.027018+0000","last_became_peered":"2024-09-10T23:36:20.027018+0000","last_unstale":"2024-09-10T23:36:21.287415+0000","last_undegraded":"2024-09-10T23:36:21.287415+0000","last_fullsized":"2024-09-10T23:36:21.287415+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:00.660346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288824+0000","last_change":"2024-09-10T23:36:20.036469+0000","last_active":"2024-09-10T23:36:21.288824+0000","last_peered":"2024-09-10T23:36:21.288824+0000","last_clean":"2024-09-10T23:36:21.288824+0000","last_became_active":"2024-09-10T23:36:20.032983+0000","last_became_peered":"2024-09-10T23:36:20.032983+0000","last_unstale":"2024-09-10T23:36:21.288824+0000","last_undegraded":"2024-09-10T23:36:21.288824+0000","last_fullsized":"2024-09-10T23:36:21.288824+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:39.992856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030336+0000","last_change":"2024-09-10T23:36:20.027753+0000","last_active":"2024-09-10T23:36:21.030336+0000","last_peered":"2024-09-10T23:36:21.030336+0000","last_clean":"2024-09-10T23:36:21.030336+0000","last_became_active":"2024-09-10T23:36:20.026478+0000","last_became_peered":"2024-09-10T23:36:20.026478+0000","last_unstale":"2024-09-10T23:36:21.030336+0000","last_undegraded":"2024-09-10T23:36:21.030336+0000","last_fullsized":"2024-09-10T23:36:21.030336+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:17.739314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287388+0000","last_change":"2024-09-10T23:36:20.027567+0000","last_active":"2024-09-10T23:36:21.287388+0000","last_peered":"2024-09-10T23:36:21.287388+0000","last_clean":"2024-09-10T23:36:21.287388+0000","last_became_active":"2024-09-10T23:36:20.027272+0000","last_became_peered":"2024-09-10T23:36:20.027272+0000","last_unstale":"2024-09-10T23:36:21.287388+0000","last_undegraded":"2024-09-10T23:36:21.287388+0000","last_fullsized":"2024-09-10T23:36:21.287388+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:43.377507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287272+0000","last_change":"2024-09-10T23:36:20.027894+0000","last_active":"2024-09-10T23:36:21.287272+0000","last_peered":"2024-09-10T23:36:21.287272+0000","last_clean":"2024-09-10T23:36:21.287272+0000","last_became_active":"2024-09-10T23:36:20.027426+0000","last_became_peered":"2024-09-10T23:36:20.027426+0000","last_unstale":"2024-09-10T23:36:21.287272+0000","last_undegraded":"2024-09-10T23:36:21.287272+0000","last_fullsized":"2024-09-10T23:36:21.287272+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:42.136223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030801+0000","last_change":"2024-09-10T23:36:20.024322+0000","last_active":"2024-09-10T23:36:21.030801+0000","last_peered":"2024-09-10T23:36:21.030801+0000","last_clean":"2024-09-10T23:36:21.030801+0000","last_became_active":"2024-09-10T23:36:20.024019+0000","last_became_peered":"2024-09-10T23:36:20.024019+0000","last_unstale":"2024-09-10T23:36:21.030801+0000","last_undegraded":"2024-09-10T23:36:21.030801+0000","last_fullsized":"2024-09-10T23:36:21.030801+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:25.060145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.006986+0000","last_change":"2024-09-10T23:36:19.020218+0000","last_active":"2024-09-10T23:36:21.006986+0000","last_peered":"2024-09-10T23:36:21.006986+0000","last_clean":"2024-09-10T23:36:21.006986+0000","last_became_active":"2024-09-10T23:36:19.019884+0000","last_became_peered":"2024-09-10T23:36:19.019884+0000","last_unstale":"2024-09-10T23:36:21.006986+0000","last_undegraded":"2024-09-10T23:36:21.006986+0000","last_fullsized":"2024-09-10T23:36:21.006986+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:25:20.502178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289123+0000","last_change":"2024-09-10T23:36:20.042051+0000","last_active":"2024-09-10T23:36:21.289123+0000","last_peered":"2024-09-10T23:36:21.289123+0000","last_clean":"2024-09-10T23:36:21.289123+0000","last_became_active":"2024-09-10T23:36:20.041741+0000","last_became_peered":"2024-09-10T23:36:20.041741+0000","last_unstale":"2024-09-10T23:36:21.289123+0000","last_undegraded":"2024-09-10T23:36:21.289123+0000","last_fullsized":"2024-09-10T23:36:21.289123+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:18.227159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289696+0000","last_change":"2024-09-10T23:36:19.022835+0000","last_active":"2024-09-10T23:36:21.289696+0000","last_peered":"2024-09-10T23:36:21.289696+0000","last_clean":"2024-09-10T23:36:21.289696+0000","last_became_active":"2024-09-10T23:36:19.022544+0000","last_became_peered":"2024-09-10T23:36:19.022544+0000","last_unstale":"2024-09-10T23:36:21.289696+0000","last_undegraded":"2024-09-10T23:36:21.289696+0000","last_fullsized":"2024-09-10T23:36:21.289696+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:45.433150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.007338+0000","last_change":"2024-09-10T23:36:20.024221+0000","last_active":"2024-09-10T23:36:21.007338+0000","last_peered":"2024-09-10T23:36:21.007338+0000","last_clean":"2024-09-10T23:36:21.007338+0000","last_became_active":"2024-09-10T23:36:20.023960+0000","last_became_peered":"2024-09-10T23:36:20.023960+0000","last_unstale":"2024-09-10T23:36:21.007338+0000","last_undegraded":"2024-09-10T23:36:21.007338+0000","last_fullsized":"2024-09-10T23:36:21.007338+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:05.271339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030768+0000","last_change":"2024-09-10T23:36:19.028889+0000","last_active":"2024-09-10T23:36:21.030768+0000","last_peered":"2024-09-10T23:36:21.030768+0000","last_clean":"2024-09-10T23:36:21.030768+0000","last_became_active":"2024-09-10T23:36:19.028611+0000","last_became_peered":"2024-09-10T23:36:19.028611+0000","last_unstale":"2024-09-10T23:36:21.030768+0000","last_undegraded":"2024-09-10T23:36:21.030768+0000","last_fullsized":"2024-09-10T23:36:21.030768+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:29.598577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033281+0000","last_change":"2024-09-10T23:36:20.015665+0000","last_active":"2024-09-10T23:36:21.033281+0000","last_peered":"2024-09-10T23:36:21.033281+0000","last_clean":"2024-09-10T23:36:21.033281+0000","last_became_active":"2024-09-10T23:36:20.015366+0000","last_became_peered":"2024-09-10T23:36:20.015366+0000","last_unstale":"2024-09-10T23:36:21.033281+0000","last_undegraded":"2024-09-10T23:36:21.033281+0000","last_fullsized":"2024-09-10T23:36:21.033281+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:18.227159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.007289+0000","last_change":"2024-09-10T23:36:19.015875+0000","last_active":"2024-09-10T23:36:21.007289+0000","last_peered":"2024-09-10T23:36:21.007289+0000","last_clean":"2024-09-10T23:36:21.007289+0000","last_became_active":"2024-09-10T23:36:19.015619+0000","last_became_peered":"2024-09-10T23:36:19.015619+0000","last_unstale":"2024-09-10T23:36:21.007289+0000","last_undegraded":"2024-09-10T23:36:21.007289+0000","last_fullsized":"2024-09-10T23:36:21.007289+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:11.311110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030333+0000","last_change":"2024-09-10T23:36:20.020951+0000","last_active":"2024-09-10T23:36:21.030333+0000","last_peered":"2024-09-10T23:36:21.030333+0000","last_clean":"2024-09-10T23:36:21.030333+0000","last_became_active":"2024-09-10T23:36:20.020639+0000","last_became_peered":"2024-09-10T23:36:20.020639+0000","last_unstale":"2024-09-10T23:36:21.030333+0000","last_undegraded":"2024-09-10T23:36:21.030333+0000","last_fullsized":"2024-09-10T23:36:21.030333+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:16:15.280849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.009019+0000","last_change":"2024-09-10T23:36:19.020177+0000","last_active":"2024-09-10T23:36:21.009019+0000","last_peered":"2024-09-10T23:36:21.009019+0000","last_clean":"2024-09-10T23:36:21.009019+0000","last_became_active":"2024-09-10T23:36:19.019826+0000","last_became_peered":"2024-09-10T23:36:19.019826+0000","last_unstale":"2024-09-10T23:36:21.009019+0000","last_undegraded":"2024-09-10T23:36:21.009019+0000","last_fullsized":"2024-09-10T23:36:21.009019+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:41.071865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.007040+0000","last_change":"2024-09-10T23:36:20.022555+0000","last_active":"2024-09-10T23:36:21.007040+0000","last_peered":"2024-09-10T23:36:21.007040+0000","last_clean":"2024-09-10T23:36:21.007040+0000","last_became_active":"2024-09-10T23:36:20.022266+0000","last_became_peered":"2024-09-10T23:36:20.022266+0000","last_unstale":"2024-09-10T23:36:21.007040+0000","last_undegraded":"2024-09-10T23:36:21.007040+0000","last_fullsized":"2024-09-10T23:36:21.007040+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:22.038276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033072+0000","last_change":"2024-09-10T23:36:19.025715+0000","last_active":"2024-09-10T23:36:21.033072+0000","last_peered":"2024-09-10T23:36:21.033072+0000","last_clean":"2024-09-10T23:36:21.033072+0000","last_became_active":"2024-09-10T23:36:19.020534+0000","last_became_peered":"2024-09-10T23:36:19.020534+0000","last_unstale":"2024-09-10T23:36:21.033072+0000","last_undegraded":"2024-09-10T23:36:21.033072+0000","last_fullsized":"2024-09-10T23:36:21.033072+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:52.389059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T23:36:20.042361+0000","last_change":"2024-09-10T23:36:20.028676+0000","last_active":"2024-09-10T23:36:20.042361+0000","last_peered":"2024-09-10T23:36:20.042361+0000","last_clean":"2024-09-10T23:36:20.042361+0000","last_became_active":"2024-09-10T23:36:20.028395+0000","last_became_peered":"2024-09-10T23:36:20.028395+0000","last_unstale":"2024-09-10T23:36:20.042361+0000","last_undegraded":"2024-09-10T23:36:20.042361+0000","last_fullsized":"2024-09-10T23:36:20.042361+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:42.456259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T23:36:20.013339+0000","last_change":"2024-09-10T23:36:19.020666+0000","last_active":"2024-09-10T23:36:20.013339+0000","last_peered":"2024-09-10T23:36:20.013339+0000","last_clean":"2024-09-10T23:36:20.013339+0000","last_became_active":"2024-09-10T23:36:19.020316+0000","last_became_peered":"2024-09-10T23:36:19.020316+0000","last_unstale":"2024-09-10T23:36:20.013339+0000","last_undegraded":"2024-09-10T23:36:20.013339+0000","last_fullsized":"2024-09-10T23:36:20.013339+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:04.135022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.008390+0000","last_change":"2024-09-10T23:36:20.021927+0000","last_active":"2024-09-10T23:36:21.008390+0000","last_peered":"2024-09-10T23:36:21.008390+0000","last_clean":"2024-09-10T23:36:21.008390+0000","last_became_active":"2024-09-10T23:36:20.021694+0000","last_became_peered":"2024-09-10T23:36:20.021694+0000","last_unstale":"2024-09-10T23:36:21.008390+0000","last_undegraded":"2024-09-10T23:36:21.008390+0000","last_fullsized":"2024-09-10T23:36:21.008390+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:58.056854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.007889+0000","last_change":"2024-09-10T23:36:19.018425+0000","last_active":"2024-09-10T23:36:21.007889+0000","last_peered":"2024-09-10T23:36:21.007889+0000","last_clean":"2024-09-10T23:36:21.007889+0000","last_became_active":"2024-09-10T23:36:19.018091+0000","last_became_peered":"2024-09-10T23:36:19.018091+0000","last_unstale":"2024-09-10T23:36:21.007889+0000","last_undegraded":"2024-09-10T23:36:21.007889+0000","last_fullsized":"2024-09-10T23:36:21.007889+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:56.848956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033591+0000","last_change":"2024-09-10T23:36:20.017610+0000","last_active":"2024-09-10T23:36:21.033591+0000","last_peered":"2024-09-10T23:36:21.033591+0000","last_clean":"2024-09-10T23:36:21.033591+0000","last_became_active":"2024-09-10T23:36:20.017324+0000","last_became_peered":"2024-09-10T23:36:20.017324+0000","last_unstale":"2024-09-10T23:36:21.033591+0000","last_undegraded":"2024-09-10T23:36:21.033591+0000","last_fullsized":"2024-09-10T23:36:21.033591+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:45.146267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030631+0000","last_change":"2024-09-10T23:36:19.024545+0000","last_active":"2024-09-10T23:36:21.030631+0000","last_peered":"2024-09-10T23:36:21.030631+0000","last_clean":"2024-09-10T23:36:21.030631+0000","last_became_active":"2024-09-10T23:36:19.024289+0000","last_became_peered":"2024-09-10T23:36:19.024289+0000","last_unstale":"2024-09-10T23:36:21.030631+0000","last_undegraded":"2024-09-10T23:36:21.030631+0000","last_fullsized":"2024-09-10T23:36:21.030631+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:42.049011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287432+0000","last_change":"2024-09-10T23:36:20.025694+0000","last_active":"2024-09-10T23:36:21.287432+0000","last_peered":"2024-09-10T23:36:21.287432+0000","last_clean":"2024-09-10T23:36:21.287432+0000","last_became_active":"2024-09-10T23:36:20.025391+0000","last_became_peered":"2024-09-10T23:36:20.025391+0000","last_unstale":"2024-09-10T23:36:21.287432+0000","last_undegraded":"2024-09-10T23:36:21.287432+0000","last_fullsized":"2024-09-10T23:36:21.287432+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:22.038276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287570+0000","last_change":"2024-09-10T23:36:19.024336+0000","last_active":"2024-09-10T23:36:21.287570+0000","last_peered":"2024-09-10T23:36:21.287570+0000","last_clean":"2024-09-10T23:36:21.287570+0000","last_became_active":"2024-09-10T23:36:19.023540+0000","last_became_peered":"2024-09-10T23:36:19.023540+0000","last_unstale":"2024-09-10T23:36:21.287570+0000","last_undegraded":"2024-09-10T23:36:21.287570+0000","last_fullsized":"2024-09-10T23:36:21.287570+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:34.975621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030683+0000","last_change":"2024-09-10T23:36:20.029664+0000","last_active":"2024-09-10T23:36:21.030683+0000","last_peered":"2024-09-10T23:36:21.030683+0000","last_clean":"2024-09-10T23:36:21.030683+0000","last_became_active":"2024-09-10T23:36:20.029384+0000","last_became_peered":"2024-09-10T23:36:20.029384+0000","last_unstale":"2024-09-10T23:36:21.030683+0000","last_undegraded":"2024-09-10T23:36:21.030683+0000","last_fullsized":"2024-09-10T23:36:21.030683+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:13.253664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.009327+0000","last_change":"2024-09-10T23:36:19.017076+0000","last_active":"2024-09-10T23:36:21.009327+0000","last_peered":"2024-09-10T23:36:21.009327+0000","last_clean":"2024-09-10T23:36:21.009327+0000","last_became_active":"2024-09-10T23:36:19.016745+0000","last_became_peered":"2024-09-10T23:36:19.016745+0000","last_unstale":"2024-09-10T23:36:21.009327+0000","last_undegraded":"2024-09-10T23:36:21.009327+0000","last_fullsized":"2024-09-10T23:36:21.009327+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:46.227666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289275+0000","last_change":"2024-09-10T23:36:20.025547+0000","last_active":"2024-09-10T23:36:21.289275+0000","last_peered":"2024-09-10T23:36:21.289275+0000","last_clean":"2024-09-10T23:36:21.289275+0000","last_became_active":"2024-09-10T23:36:20.024978+0000","last_became_peered":"2024-09-10T23:36:20.024978+0000","last_unstale":"2024-09-10T23:36:21.289275+0000","last_undegraded":"2024-09-10T23:36:21.289275+0000","last_fullsized":"2024-09-10T23:36:21.289275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:22.933224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":19,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T23:36:20.048372+0000","last_change":"2024-09-10T23:36:19.022284+0000","last_active":"2024-09-10T23:36:20.048372+0000","last_peered":"2024-09-10T23:36:20.048372+0000","last_clean":"2024-09-10T23:36:20.048372+0000","last_became_active":"2024-09-10T23:36:19.021141+0000","last_became_peered":"2024-09-10T23:36:19.021141+0000","last_unstale":"2024-09-10T23:36:20.048372+0000","last_undegraded":"2024-09-10T23:36:20.048372+0000","last_fullsized":"2024-09-10T23:36:20.048372+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:53:30.820182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.007175+0000","last_change":"2024-09-10T23:36:20.025665+0000","last_active":"2024-09-10T23:36:21.007175+0000","last_peered":"2024-09-10T23:36:21.007175+0000","last_clean":"2024-09-10T23:36:21.007175+0000","last_became_active":"2024-09-10T23:36:20.023011+0000","last_became_peered":"2024-09-10T23:36:20.023011+0000","last_unstale":"2024-09-10T23:36:21.007175+0000","last_undegraded":"2024-09-10T23:36:21.007175+0000","last_fullsized":"2024-09-10T23:36:21.007175+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:58.056854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T23:36:20.045501+0000","last_change":"2024-09-10T23:36:19.020496+0000","last_active":"2024-09-10T23:36:20.045501+0000","last_peered":"2024-09-10T23:36:20.045501+0000","last_clean":"2024-09-10T23:36:20.045501+0000","last_became_active":"2024-09-10T23:36:19.020267+0000","last_became_peered":"2024-09-10T23:36:19.020267+0000","last_unstale":"2024-09-10T23:36:20.045501+0000","last_undegraded":"2024-09-10T23:36:20.045501+0000","last_fullsized":"2024-09-10T23:36:20.045501+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:28.089952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033278+0000","last_change":"2024-09-10T23:36:20.023520+0000","last_active":"2024-09-10T23:36:21.033278+0000","last_peered":"2024-09-10T23:36:21.033278+0000","last_clean":"2024-09-10T23:36:21.033278+0000","last_became_active":"2024-09-10T23:36:20.020356+0000","last_became_peered":"2024-09-10T23:36:20.020356+0000","last_unstale":"2024-09-10T23:36:21.033278+0000","last_undegraded":"2024-09-10T23:36:21.033278+0000","last_fullsized":"2024-09-10T23:36:21.033278+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:53.266472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033362+0000","last_change":"2024-09-10T23:36:19.019162+0000","last_active":"2024-09-10T23:36:21.033362+0000","last_peered":"2024-09-10T23:36:21.033362+0000","last_clean":"2024-09-10T23:36:21.033362+0000","last_became_active":"2024-09-10T23:36:19.018864+0000","last_became_peered":"2024-09-10T23:36:19.018864+0000","last_unstale":"2024-09-10T23:36:21.033362+0000","last_undegraded":"2024-09-10T23:36:21.033362+0000","last_fullsized":"2024-09-10T23:36:21.033362+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:15.888189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288762+0000","last_change":"2024-09-10T23:36:20.024099+0000","last_active":"2024-09-10T23:36:21.288762+0000","last_peered":"2024-09-10T23:36:21.288762+0000","last_clean":"2024-09-10T23:36:21.288762+0000","last_became_active":"2024-09-10T23:36:20.023846+0000","last_became_peered":"2024-09-10T23:36:20.023846+0000","last_unstale":"2024-09-10T23:36:21.288762+0000","last_undegraded":"2024-09-10T23:36:21.288762+0000","last_fullsized":"2024-09-10T23:36:21.288762+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:47.186028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033261+0000","last_change":"2024-09-10T23:36:19.016343+0000","last_active":"2024-09-10T23:36:21.033261+0000","last_peered":"2024-09-10T23:36:21.033261+0000","last_clean":"2024-09-10T23:36:21.033261+0000","last_became_active":"2024-09-10T23:36:19.015988+0000","last_became_peered":"2024-09-10T23:36:19.015988+0000","last_unstale":"2024-09-10T23:36:21.033261+0000","last_undegraded":"2024-09-10T23:36:21.033261+0000","last_fullsized":"2024-09-10T23:36:21.033261+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:04:57.866345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288470+0000","last_change":"2024-09-10T23:36:20.042708+0000","last_active":"2024-09-10T23:36:21.288470+0000","last_peered":"2024-09-10T23:36:21.288470+0000","last_clean":"2024-09-10T23:36:21.288470+0000","last_became_active":"2024-09-10T23:36:20.041834+0000","last_became_peered":"2024-09-10T23:36:20.041834+0000","last_unstale":"2024-09-10T23:36:21.288470+0000","last_undegraded":"2024-09-10T23:36:21.288470+0000","last_fullsized":"2024-09-10T23:36:21.288470+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:51.774172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":23,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.032884+0000","last_change":"2024-09-10T23:36:19.017661+0000","last_active":"2024-09-10T23:36:21.032884+0000","last_peered":"2024-09-10T23:36:21.032884+0000","last_clean":"2024-09-10T23:36:21.032884+0000","last_became_active":"2024-09-10T23:36:19.017084+0000","last_became_peered":"2024-09-10T23:36:19.017084+0000","last_unstale":"2024-09-10T23:36:21.032884+0000","last_undegraded":"2024-09-10T23:36:21.032884+0000","last_fullsized":"2024-09-10T23:36:21.032884+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:22.350535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287466+0000","last_change":"2024-09-10T23:36:20.022114+0000","last_active":"2024-09-10T23:36:21.287466+0000","last_peered":"2024-09-10T23:36:21.287466+0000","last_clean":"2024-09-10T23:36:21.287466+0000","last_became_active":"2024-09-10T23:36:20.021447+0000","last_became_peered":"2024-09-10T23:36:20.021447+0000","last_unstale":"2024-09-10T23:36:21.287466+0000","last_undegraded":"2024-09-10T23:36:21.287466+0000","last_fullsized":"2024-09-10T23:36:21.287466+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:58.056854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030741+0000","last_change":"2024-09-10T23:36:19.028083+0000","last_active":"2024-09-10T23:36:21.030741+0000","last_peered":"2024-09-10T23:36:21.030741+0000","last_clean":"2024-09-10T23:36:21.030741+0000","last_became_active":"2024-09-10T23:36:19.025982+0000","last_became_peered":"2024-09-10T23:36:19.025982+0000","last_unstale":"2024-09-10T23:36:21.030741+0000","last_undegraded":"2024-09-10T23:36:21.030741+0000","last_fullsized":"2024-09-10T23:36:21.030741+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:46.179627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287495+0000","last_change":"2024-09-10T23:36:20.017954+0000","last_active":"2024-09-10T23:36:21.287495+0000","last_peered":"2024-09-10T23:36:21.287495+0000","last_clean":"2024-09-10T23:36:21.287495+0000","last_became_active":"2024-09-10T23:36:20.017623+0000","last_became_peered":"2024-09-10T23:36:20.017623+0000","last_unstale":"2024-09-10T23:36:21.287495+0000","last_undegraded":"2024-09-10T23:36:21.287495+0000","last_fullsized":"2024-09-10T23:36:21.287495+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:50.318502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287591+0000","last_change":"2024-09-10T23:36:19.019143+0000","last_active":"2024-09-10T23:36:21.287591+0000","last_peered":"2024-09-10T23:36:21.287591+0000","last_clean":"2024-09-10T23:36:21.287591+0000","last_became_active":"2024-09-10T23:36:19.018868+0000","last_became_peered":"2024-09-10T23:36:19.018868+0000","last_unstale":"2024-09-10T23:36:21.287591+0000","last_undegraded":"2024-09-10T23:36:21.287591+0000","last_fullsized":"2024-09-10T23:36:21.287591+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:14.976336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289094+0000","last_change":"2024-09-10T23:36:20.035368+0000","last_active":"2024-09-10T23:36:21.289094+0000","last_peered":"2024-09-10T23:36:21.289094+0000","last_clean":"2024-09-10T23:36:21.289094+0000","last_became_active":"2024-09-10T23:36:20.035120+0000","last_became_peered":"2024-09-10T23:36:20.035120+0000","last_unstale":"2024-09-10T23:36:21.289094+0000","last_undegraded":"2024-09-10T23:36:21.289094+0000","last_fullsized":"2024-09-10T23:36:21.289094+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:31.959052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287151+0000","last_change":"2024-09-10T23:36:19.021077+0000","last_active":"2024-09-10T23:36:21.287151+0000","last_peered":"2024-09-10T23:36:21.287151+0000","last_clean":"2024-09-10T23:36:21.287151+0000","last_became_active":"2024-09-10T23:36:19.020683+0000","last_became_peered":"2024-09-10T23:36:19.020683+0000","last_unstale":"2024-09-10T23:36:21.287151+0000","last_undegraded":"2024-09-10T23:36:21.287151+0000","last_fullsized":"2024-09-10T23:36:21.287151+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:15.039256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289279+0000","last_change":"2024-09-10T23:36:20.032886+0000","last_active":"2024-09-10T23:36:21.289279+0000","last_peered":"2024-09-10T23:36:21.289279+0000","last_clean":"2024-09-10T23:36:21.289279+0000","last_became_active":"2024-09-10T23:36:20.032635+0000","last_became_peered":"2024-09-10T23:36:20.032635+0000","last_unstale":"2024-09-10T23:36:21.289279+0000","last_undegraded":"2024-09-10T23:36:21.289279+0000","last_fullsized":"2024-09-10T23:36:21.289279+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:48.752715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.007385+0000","last_change":"2024-09-10T23:36:19.023593+0000","last_active":"2024-09-10T23:36:21.007385+0000","last_peered":"2024-09-10T23:36:21.007385+0000","last_clean":"2024-09-10T23:36:21.007385+0000","last_became_active":"2024-09-10T23:36:19.023298+0000","last_became_peered":"2024-09-10T23:36:19.023298+0000","last_unstale":"2024-09-10T23:36:21.007385+0000","last_undegraded":"2024-09-10T23:36:21.007385+0000","last_fullsized":"2024-09-10T23:36:21.007385+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:53:30.350802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T23:36:20.042445+0000","last_change":"2024-09-10T23:36:20.029501+0000","last_active":"2024-09-10T23:36:20.042445+0000","last_peered":"2024-09-10T23:36:20.042445+0000","last_clean":"2024-09-10T23:36:20.042445+0000","last_became_active":"2024-09-10T23:36:20.026891+0000","last_became_peered":"2024-09-10T23:36:20.026891+0000","last_unstale":"2024-09-10T23:36:20.042445+0000","last_undegraded":"2024-09-10T23:36:20.042445+0000","last_fullsized":"2024-09-10T23:36:20.042445+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:37.093491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.007265+0000","last_change":"2024-09-10T23:36:19.019734+0000","last_active":"2024-09-10T23:36:21.007265+0000","last_peered":"2024-09-10T23:36:21.007265+0000","last_clean":"2024-09-10T23:36:21.007265+0000","last_became_active":"2024-09-10T23:36:19.019447+0000","last_became_peered":"2024-09-10T23:36:19.019447+0000","last_unstale":"2024-09-10T23:36:21.007265+0000","last_undegraded":"2024-09-10T23:36:21.007265+0000","last_fullsized":"2024-09-10T23:36:21.007265+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:52.241631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T23:36:20.039019+0000","last_change":"2024-09-10T23:36:20.019309+0000","last_active":"2024-09-10T23:36:20.039019+0000","last_peered":"2024-09-10T23:36:20.039019+0000","last_clean":"2024-09-10T23:36:20.039019+0000","last_became_active":"2024-09-10T23:36:20.019017+0000","last_became_peered":"2024-09-10T23:36:20.019017+0000","last_unstale":"2024-09-10T23:36:20.039019+0000","last_undegraded":"2024-09-10T23:36:20.039019+0000","last_fullsized":"2024-09-10T23:36:20.039019+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:31.161121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287309+0000","last_change":"2024-09-10T23:36:19.021026+0000","last_active":"2024-09-10T23:36:21.287309+0000","last_peered":"2024-09-10T23:36:21.287309+0000","last_clean":"2024-09-10T23:36:21.287309+0000","last_became_active":"2024-09-10T23:36:19.018580+0000","last_became_peered":"2024-09-10T23:36:19.018580+0000","last_unstale":"2024-09-10T23:36:21.287309+0000","last_undegraded":"2024-09-10T23:36:21.287309+0000","last_fullsized":"2024-09-10T23:36:21.287309+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:21.047623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.008555+0000","last_change":"2024-09-10T23:36:20.028309+0000","last_active":"2024-09-10T23:36:21.008555+0000","last_peered":"2024-09-10T23:36:21.008555+0000","last_clean":"2024-09-10T23:36:21.008555+0000","last_became_active":"2024-09-10T23:36:20.027822+0000","last_became_peered":"2024-09-10T23:36:20.027822+0000","last_unstale":"2024-09-10T23:36:21.008555+0000","last_undegraded":"2024-09-10T23:36:21.008555+0000","last_fullsized":"2024-09-10T23:36:21.008555+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:50.318502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288466+0000","last_change":"2024-09-10T23:36:19.021589+0000","last_active":"2024-09-10T23:36:21.288466+0000","last_peered":"2024-09-10T23:36:21.288466+0000","last_clean":"2024-09-10T23:36:21.288466+0000","last_became_active":"2024-09-10T23:36:19.021325+0000","last_became_peered":"2024-09-10T23:36:19.021325+0000","last_unstale":"2024-09-10T23:36:21.288466+0000","last_undegraded":"2024-09-10T23:36:21.288466+0000","last_fullsized":"2024-09-10T23:36:21.288466+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:19.736122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289585+0000","last_change":"2024-09-10T23:36:20.025380+0000","last_active":"2024-09-10T23:36:21.289585+0000","last_peered":"2024-09-10T23:36:21.289585+0000","last_clean":"2024-09-10T23:36:21.289585+0000","last_became_active":"2024-09-10T23:36:20.025113+0000","last_became_peered":"2024-09-10T23:36:20.025113+0000","last_unstale":"2024-09-10T23:36:21.289585+0000","last_undegraded":"2024-09-10T23:36:21.289585+0000","last_fullsized":"2024-09-10T23:36:21.289585+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:37.050818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030842+0000","last_change":"2024-09-10T23:36:19.026256+0000","last_active":"2024-09-10T23:36:21.030842+0000","last_peered":"2024-09-10T23:36:21.030842+0000","last_clean":"2024-09-10T23:36:21.030842+0000","last_became_active":"2024-09-10T23:36:19.025653+0000","last_became_peered":"2024-09-10T23:36:19.025653+0000","last_unstale":"2024-09-10T23:36:21.030842+0000","last_undegraded":"2024-09-10T23:36:21.030842+0000","last_fullsized":"2024-09-10T23:36:21.030842+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:51.432935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033678+0000","last_change":"2024-09-10T23:36:20.027159+0000","last_active":"2024-09-10T23:36:21.033678+0000","last_peered":"2024-09-10T23:36:21.033678+0000","last_clean":"2024-09-10T23:36:21.033678+0000","last_became_active":"2024-09-10T23:36:20.026190+0000","last_became_peered":"2024-09-10T23:36:20.026190+0000","last_unstale":"2024-09-10T23:36:21.033678+0000","last_undegraded":"2024-09-10T23:36:21.033678+0000","last_fullsized":"2024-09-10T23:36:21.033678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:15.982737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.032363+0000","last_change":"2024-09-10T23:36:19.021521+0000","last_active":"2024-09-10T23:36:21.032363+0000","last_peered":"2024-09-10T23:36:21.032363+0000","last_clean":"2024-09-10T23:36:21.032363+0000","last_became_active":"2024-09-10T23:36:19.021239+0000","last_became_peered":"2024-09-10T23:36:19.021239+0000","last_unstale":"2024-09-10T23:36:21.032363+0000","last_undegraded":"2024-09-10T23:36:21.032363+0000","last_fullsized":"2024-09-10T23:36:21.032363+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:21.468611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287277+0000","last_change":"2024-09-10T23:36:20.022366+0000","last_active":"2024-09-10T23:36:21.287277+0000","last_peered":"2024-09-10T23:36:21.287277+0000","last_clean":"2024-09-10T23:36:21.287277+0000","last_became_active":"2024-09-10T23:36:20.021872+0000","last_became_peered":"2024-09-10T23:36:20.021872+0000","last_unstale":"2024-09-10T23:36:21.287277+0000","last_undegraded":"2024-09-10T23:36:21.287277+0000","last_fullsized":"2024-09-10T23:36:21.287277+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:31.586172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.009391+0000","last_change":"2024-09-10T23:36:19.014145+0000","last_active":"2024-09-10T23:36:21.009391+0000","last_peered":"2024-09-10T23:36:21.009391+0000","last_clean":"2024-09-10T23:36:21.009391+0000","last_became_active":"2024-09-10T23:36:19.013594+0000","last_became_peered":"2024-09-10T23:36:19.013594+0000","last_unstale":"2024-09-10T23:36:21.009391+0000","last_undegraded":"2024-09-10T23:36:21.009391+0000","last_fullsized":"2024-09-10T23:36:21.009391+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:51.476811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033479+0000","last_change":"2024-09-10T23:36:20.019497+0000","last_active":"2024-09-10T23:36:21.033479+0000","last_peered":"2024-09-10T23:36:21.033479+0000","last_clean":"2024-09-10T23:36:21.033479+0000","last_became_active":"2024-09-10T23:36:20.019211+0000","last_became_peered":"2024-09-10T23:36:20.019211+0000","last_unstale":"2024-09-10T23:36:21.033479+0000","last_undegraded":"2024-09-10T23:36:21.033479+0000","last_fullsized":"2024-09-10T23:36:21.033479+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:57.865930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.009330+0000","last_change":"2024-09-10T23:36:19.017410+0000","last_active":"2024-09-10T23:36:21.009330+0000","last_peered":"2024-09-10T23:36:21.009330+0000","last_clean":"2024-09-10T23:36:21.009330+0000","last_became_active":"2024-09-10T23:36:19.016975+0000","last_became_peered":"2024-09-10T23:36:19.016975+0000","last_unstale":"2024-09-10T23:36:21.009330+0000","last_undegraded":"2024-09-10T23:36:21.009330+0000","last_fullsized":"2024-09-10T23:36:21.009330+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:21.431140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289529+0000","last_change":"2024-09-10T23:36:20.030079+0000","last_active":"2024-09-10T23:36:21.289529+0000","last_peered":"2024-09-10T23:36:21.289529+0000","last_clean":"2024-09-10T23:36:21.289529+0000","last_became_active":"2024-09-10T23:36:20.029105+0000","last_became_peered":"2024-09-10T23:36:20.029105+0000","last_unstale":"2024-09-10T23:36:21.289529+0000","last_undegraded":"2024-09-10T23:36:21.289529+0000","last_fullsized":"2024-09-10T23:36:21.289529+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:56.299901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.009132+0000","last_change":"2024-09-10T23:36:19.020315+0000","last_active":"2024-09-10T23:36:21.009132+0000","last_peered":"2024-09-10T23:36:21.009132+0000","last_clean":"2024-09-10T23:36:21.009132+0000","last_became_active":"2024-09-10T23:36:19.019928+0000","last_became_peered":"2024-09-10T23:36:19.019928+0000","last_unstale":"2024-09-10T23:36:21.009132+0000","last_undegraded":"2024-09-10T23:36:21.009132+0000","last_fullsized":"2024-09-10T23:36:21.009132+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:13.589708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030499+0000","last_change":"2024-09-10T23:36:20.025120+0000","last_active":"2024-09-10T23:36:21.030499+0000","last_peered":"2024-09-10T23:36:21.030499+0000","last_clean":"2024-09-10T23:36:21.030499+0000","last_became_active":"2024-09-10T23:36:20.023938+0000","last_became_peered":"2024-09-10T23:36:20.023938+0000","last_unstale":"2024-09-10T23:36:21.030499+0000","last_undegraded":"2024-09-10T23:36:21.030499+0000","last_fullsized":"2024-09-10T23:36:21.030499+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:58.570190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289481+0000","last_change":"2024-09-10T23:36:19.027466+0000","last_active":"2024-09-10T23:36:21.289481+0000","last_peered":"2024-09-10T23:36:21.289481+0000","last_clean":"2024-09-10T23:36:21.289481+0000","last_became_active":"2024-09-10T23:36:19.027220+0000","last_became_peered":"2024-09-10T23:36:19.027220+0000","last_unstale":"2024-09-10T23:36:21.289481+0000","last_undegraded":"2024-09-10T23:36:21.289481+0000","last_fullsized":"2024-09-10T23:36:21.289481+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:42.637323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T23:36:20.042387+0000","last_change":"2024-09-10T23:36:20.032463+0000","last_active":"2024-09-10T23:36:20.042387+0000","last_peered":"2024-09-10T23:36:20.042387+0000","last_clean":"2024-09-10T23:36:20.042387+0000","last_became_active":"2024-09-10T23:36:20.032170+0000","last_became_peered":"2024-09-10T23:36:20.032170+0000","last_unstale":"2024-09-10T23:36:20.042387+0000","last_undegraded":"2024-09-10T23:36:20.042387+0000","last_fullsized":"2024-09-10T23:36:20.042387+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:05:02.417345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287404+0000","last_change":"2024-09-10T23:36:19.027020+0000","last_active":"2024-09-10T23:36:21.287404+0000","last_peered":"2024-09-10T23:36:21.287404+0000","last_clean":"2024-09-10T23:36:21.287404+0000","last_became_active":"2024-09-10T23:36:19.026549+0000","last_became_peered":"2024-09-10T23:36:19.026549+0000","last_unstale":"2024-09-10T23:36:21.287404+0000","last_undegraded":"2024-09-10T23:36:21.287404+0000","last_fullsized":"2024-09-10T23:36:21.287404+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:36.496910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287121+0000","last_change":"2024-09-10T23:36:20.021902+0000","last_active":"2024-09-10T23:36:21.287121+0000","last_peered":"2024-09-10T23:36:21.287121+0000","last_clean":"2024-09-10T23:36:21.287121+0000","last_became_active":"2024-09-10T23:36:20.019325+0000","last_became_peered":"2024-09-10T23:36:20.019325+0000","last_unstale":"2024-09-10T23:36:21.287121+0000","last_undegraded":"2024-09-10T23:36:21.287121+0000","last_fullsized":"2024-09-10T23:36:21.287121+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:14.596110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.006725+0000","last_change":"2024-09-10T23:36:19.018292+0000","last_active":"2024-09-10T23:36:21.006725+0000","last_peered":"2024-09-10T23:36:21.006725+0000","last_clean":"2024-09-10T23:36:21.006725+0000","last_became_active":"2024-09-10T23:36:19.017520+0000","last_became_peered":"2024-09-10T23:36:19.017520+0000","last_unstale":"2024-09-10T23:36:21.006725+0000","last_undegraded":"2024-09-10T23:36:21.006725+0000","last_fullsized":"2024-09-10T23:36:21.006725+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:08.464048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288833+0000","last_change":"2024-09-10T23:36:20.033011+0000","last_active":"2024-09-10T23:36:21.288833+0000","last_peered":"2024-09-10T23:36:21.288833+0000","last_clean":"2024-09-10T23:36:21.288833+0000","last_became_active":"2024-09-10T23:36:20.032004+0000","last_became_peered":"2024-09-10T23:36:20.032004+0000","last_unstale":"2024-09-10T23:36:21.288833+0000","last_undegraded":"2024-09-10T23:36:21.288833+0000","last_fullsized":"2024-09-10T23:36:21.288833+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:52.439336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033314+0000","last_change":"2024-09-10T23:36:19.020456+0000","last_active":"2024-09-10T23:36:21.033314+0000","last_peered":"2024-09-10T23:36:21.033314+0000","last_clean":"2024-09-10T23:36:21.033314+0000","last_became_active":"2024-09-10T23:36:19.018692+0000","last_became_peered":"2024-09-10T23:36:19.018692+0000","last_unstale":"2024-09-10T23:36:21.033314+0000","last_undegraded":"2024-09-10T23:36:21.033314+0000","last_fullsized":"2024-09-10T23:36:21.033314+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:20.057823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":198,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.032903+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:36:21.032903+0000","last_peered":"2024-09-10T23:36:21.032903+0000","last_clean":"2024-09-10T23:36:21.032903+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:36:21.032903+0000","last_undegraded":"2024-09-10T23:36:21.032903+0000","last_fullsized":"2024-09-10T23:36:21.032903+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509491,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":831488,"data_stored":342446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509490,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1429504,"data_stored":937592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509488,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":843776,"data_stored":347828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":831488,"data_stored":340960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509490,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":823296,"data_stored":341416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509489,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1425408,"data_stored":937502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509489,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":823296,"data_stored":340904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542194,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":835584,"data_stored":342480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:36:24.119 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:36:24.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c8600640 1 -- 172.21.15.94:0/3360335119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c0164830 msgr2=0x7fc3c0166c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:24.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c8600640 1 --2- 172.21.15.94:0/3360335119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c0164830 0x7fc3c0166c20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009bc0 tx=0x7fc3b801cae0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c8600640 1 -- 172.21.15.94:0/3360335119 shutdown_connections 2024-09-10T23:36:24.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c8600640 1 --2- 172.21.15.94:0/3360335119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c0164830 0x7fc3c0166c20 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c8600640 1 -- 172.21.15.94:0/3360335119 >> 172.21.15.94:0/3360335119 conn(0x7fc3c006cdf0 msgr2=0x7fc3c006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:24.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c8600640 1 -- 172.21.15.94:0/3360335119 shutdown_connections 2024-09-10T23:36:24.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c8600640 1 -- 172.21.15.94:0/3360335119 wait complete. 2024-09-10T23:36:24.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c8600640 1 Processor -- start 2024-09-10T23:36:24.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c8600640 1 -- start start 2024-09-10T23:36:24.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c8600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00cfcb0 0x7fc3c013fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:24.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c8600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3c015ba50 0x7fc3c01662d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:24.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c8600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3c0164830 0x7fc3c0149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:24.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c8600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c0169b00 con 0x7fc3c00cfcb0 2024-09-10T23:36:24.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c8600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c0169980 con 0x7fc3c015ba50 2024-09-10T23:36:24.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c8600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c0169c80 con 0x7fc3c0164830 2024-09-10T23:36:24.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c5c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00cfcb0 0x7fc3c013fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:24.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c5c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00cfcb0 0x7fc3c013fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51038/0 (socket says 172.21.15.94:51038) 2024-09-10T23:36:24.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c5c00640 1 -- 172.21.15.94:0/3692012094 learned_addr learned my addr 172.21.15.94:0/3692012094 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:24.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c5200640 1 --2- 172.21.15.94:0/3692012094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3c015ba50 0x7fc3c01662d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:24.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c6600640 1 --2- 172.21.15.94:0/3692012094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3c0164830 0x7fc3c0149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:24.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c5c00640 1 -- 172.21.15.94:0/3692012094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3c0164830 msgr2=0x7fc3c0149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:24.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c5c00640 1 --2- 172.21.15.94:0/3692012094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3c0164830 0x7fc3c0149bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c5c00640 1 -- 172.21.15.94:0/3692012094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3c015ba50 msgr2=0x7fc3c01662d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:24.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c5c00640 1 --2- 172.21.15.94:0/3692012094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3c015ba50 0x7fc3c01662d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c5c00640 1 -- 172.21.15.94:0/3692012094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c0166d20 con 0x7fc3c00cfcb0 2024-09-10T23:36:24.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c6600640 1 --2- 172.21.15.94:0/3692012094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3c0164830 0x7fc3c0149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:36:24.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.302+0000 7fc3c5c00640 1 --2- 172.21.15.94:0/3692012094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00cfcb0 0x7fc3c013fe20 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400c490 tx=0x7fc3b400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:24.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.306+0000 7fc3bd600640 1 -- 172.21.15.94:0/3692012094 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4016020 con 0x7fc3c00cfcb0 2024-09-10T23:36:24.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.306+0000 7fc3bd600640 1 -- 172.21.15.94:0/3692012094 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3b4012940 con 0x7fc3c00cfcb0 2024-09-10T23:36:24.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.306+0000 7fc3bd600640 1 -- 172.21.15.94:0/3692012094 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4012d90 con 0x7fc3c00cfcb0 2024-09-10T23:36:24.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.306+0000 7fc3c8600640 1 -- 172.21.15.94:0/3692012094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c0157000 con 0x7fc3c00cfcb0 2024-09-10T23:36:24.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.306+0000 7fc3c8600640 1 -- 172.21.15.94:0/3692012094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c01f7530 con 0x7fc3c00cfcb0 2024-09-10T23:36:24.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.306+0000 7fc3c8600640 1 -- 172.21.15.94:0/3692012094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c00d00d0 con 0x7fc3c00cfcb0 2024-09-10T23:36:24.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.306+0000 7fc3bd600640 1 -- 172.21.15.94:0/3692012094 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7fc3b401b400 con 0x7fc3c00cfcb0 2024-09-10T23:36:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.306+0000 7fc3bd600640 1 --2- 172.21.15.94:0/3692012094 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc3a0072970 0x7fc3a0074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.306+0000 7fc3bd600640 1 -- 172.21.15.94:0/3692012094 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fc3b4093b80 con 0x7fc3c00cfcb0 2024-09-10T23:36:24.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.306+0000 7fc3c5200640 1 --2- 172.21.15.94:0/3692012094 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc3a0072970 0x7fc3a0074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:24.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.310+0000 7fc3c5200640 1 --2- 172.21.15.94:0/3692012094 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc3a0072970 0x7fc3a0074e30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0016f0 tx=0x7fc3ac009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:24.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.310+0000 7fc3bd600640 1 -- 172.21.15.94:0/3692012094 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc3b4020020 con 0x7fc3c00cfcb0 2024-09-10T23:36:24.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.474+0000 7fc3c8600640 1 -- 172.21.15.94:0/3692012094 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc3c0157ac0 con 0x7fc3a0072970 2024-09-10T23:36:24.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.478+0000 7fc3bd600640 1 -- 172.21.15.94:0/3692012094 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252059 (secure 0 0 0) 0x7fc3c0157ac0 con 0x7fc3a0072970 2024-09-10T23:36:24.482 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:24.484 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:36:24.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.482+0000 7fc3c8600640 1 -- 172.21.15.94:0/3692012094 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc3a0072970 msgr2=0x7fc3a0074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:24.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.482+0000 7fc3c8600640 1 --2- 172.21.15.94:0/3692012094 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc3a0072970 0x7fc3a0074e30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0016f0 tx=0x7fc3ac009040 comp rx=0 tx=0).stop 2024-09-10T23:36:24.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.482+0000 7fc3c8600640 1 -- 172.21.15.94:0/3692012094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00cfcb0 msgr2=0x7fc3c013fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:24.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.482+0000 7fc3c8600640 1 --2- 172.21.15.94:0/3692012094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00cfcb0 0x7fc3c013fe20 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400c490 tx=0x7fc3b400c950 comp rx=0 tx=0).stop 2024-09-10T23:36:24.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.482+0000 7fc3c8600640 1 -- 172.21.15.94:0/3692012094 shutdown_connections 2024-09-10T23:36:24.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.482+0000 7fc3c8600640 1 --2- 172.21.15.94:0/3692012094 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fc3a0072970 0x7fc3a0074e30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.482+0000 7fc3c8600640 1 --2- 172.21.15.94:0/3692012094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3c015ba50 0x7fc3c01662d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.482+0000 7fc3c8600640 1 --2- 172.21.15.94:0/3692012094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3c0164830 0x7fc3c0149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.482+0000 7fc3c8600640 1 --2- 172.21.15.94:0/3692012094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00cfcb0 0x7fc3c013fe20 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.482+0000 7fc3c8600640 1 -- 172.21.15.94:0/3692012094 >> 172.21.15.94:0/3692012094 conn(0x7fc3c006cdf0 msgr2=0x7fc3c01588c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:24.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.482+0000 7fc3c8600640 1 -- 172.21.15.94:0/3692012094 shutdown_connections 2024-09-10T23:36:24.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.482+0000 7fc3c8600640 1 -- 172.21.15.94:0/3692012094 wait complete. 2024-09-10T23:36:24.503 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-10T23:36:23.400297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257216,"kb_used_data":7660,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717504,"statfs":{"total":773094113280,"available":772830724096,"internally_reserved":0,"allocated":7843840,"data_stored":3931128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.407919"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030891+0000","last_change":"2024-09-10T23:36:20.020146+0000","last_active":"2024-09-10T23:36:21.030891+0000","last_peered":"2024-09-10T23:36:21.030891+0000","last_clean":"2024-09-10T23:36:21.030891+0000","last_became_active":"2024-09-10T23:36:20.019811+0000","last_became_peered":"2024-09-10T23:36:20.019811+0000","last_unstale":"2024-09-10T23:36:21.030891+0000","last_undegraded":"2024-09-10T23:36:21.030891+0000","last_fullsized":"2024-09-10T23:36:21.030891+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:33.686403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287535+0000","last_change":"2024-09-10T23:36:20.021782+0000","last_active":"2024-09-10T23:36:21.287535+0000","last_peered":"2024-09-10T23:36:21.287535+0000","last_clean":"2024-09-10T23:36:21.287535+0000","last_became_active":"2024-09-10T23:36:20.019103+0000","last_became_peered":"2024-09-10T23:36:20.019103+0000","last_unstale":"2024-09-10T23:36:21.287535+0000","last_undegraded":"2024-09-10T23:36:21.287535+0000","last_fullsized":"2024-09-10T23:36:21.287535+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:05.271339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289254+0000","last_change":"2024-09-10T23:36:20.033946+0000","last_active":"2024-09-10T23:36:21.289254+0000","last_peered":"2024-09-10T23:36:21.289254+0000","last_clean":"2024-09-10T23:36:21.289254+0000","last_became_active":"2024-09-10T23:36:20.032470+0000","last_became_peered":"2024-09-10T23:36:20.032470+0000","last_unstale":"2024-09-10T23:36:21.289254+0000","last_undegraded":"2024-09-10T23:36:21.289254+0000","last_fullsized":"2024-09-10T23:36:21.289254+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:55.338717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288952+0000","last_change":"2024-09-10T23:36:20.032023+0000","last_active":"2024-09-10T23:36:21.288952+0000","last_peered":"2024-09-10T23:36:21.288952+0000","last_clean":"2024-09-10T23:36:21.288952+0000","last_became_active":"2024-09-10T23:36:20.031742+0000","last_became_peered":"2024-09-10T23:36:20.031742+0000","last_unstale":"2024-09-10T23:36:21.288952+0000","last_undegraded":"2024-09-10T23:36:21.288952+0000","last_fullsized":"2024-09-10T23:36:21.288952+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:01.221296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288936+0000","last_change":"2024-09-10T23:36:20.028753+0000","last_active":"2024-09-10T23:36:21.288936+0000","last_peered":"2024-09-10T23:36:21.288936+0000","last_clean":"2024-09-10T23:36:21.288936+0000","last_became_active":"2024-09-10T23:36:20.026131+0000","last_became_peered":"2024-09-10T23:36:20.026131+0000","last_unstale":"2024-09-10T23:36:21.288936+0000","last_undegraded":"2024-09-10T23:36:21.288936+0000","last_fullsized":"2024-09-10T23:36:21.288936+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:22.475012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.007208+0000","last_change":"2024-09-10T23:36:20.024515+0000","last_active":"2024-09-10T23:36:21.007208+0000","last_peered":"2024-09-10T23:36:21.007208+0000","last_clean":"2024-09-10T23:36:21.007208+0000","last_became_active":"2024-09-10T23:36:20.023422+0000","last_became_peered":"2024-09-10T23:36:20.023422+0000","last_unstale":"2024-09-10T23:36:21.007208+0000","last_undegraded":"2024-09-10T23:36:21.007208+0000","last_fullsized":"2024-09-10T23:36:21.007208+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:50.318502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033198+0000","last_change":"2024-09-10T23:36:20.020717+0000","last_active":"2024-09-10T23:36:21.033198+0000","last_peered":"2024-09-10T23:36:21.033198+0000","last_clean":"2024-09-10T23:36:21.033198+0000","last_became_active":"2024-09-10T23:36:20.020371+0000","last_became_peered":"2024-09-10T23:36:20.020371+0000","last_unstale":"2024-09-10T23:36:21.033198+0000","last_undegraded":"2024-09-10T23:36:21.033198+0000","last_fullsized":"2024-09-10T23:36:21.033198+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:43.823658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030220+0000","last_change":"2024-09-10T23:36:20.021691+0000","last_active":"2024-09-10T23:36:21.030220+0000","last_peered":"2024-09-10T23:36:21.030220+0000","last_clean":"2024-09-10T23:36:21.030220+0000","last_became_active":"2024-09-10T23:36:20.021345+0000","last_became_peered":"2024-09-10T23:36:20.021345+0000","last_unstale":"2024-09-10T23:36:21.030220+0000","last_undegraded":"2024-09-10T23:36:21.030220+0000","last_fullsized":"2024-09-10T23:36:21.030220+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:49.604517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288899+0000","last_change":"2024-09-10T23:36:20.037904+0000","last_active":"2024-09-10T23:36:21.288899+0000","last_peered":"2024-09-10T23:36:21.288899+0000","last_clean":"2024-09-10T23:36:21.288899+0000","last_became_active":"2024-09-10T23:36:20.037650+0000","last_became_peered":"2024-09-10T23:36:20.037650+0000","last_unstale":"2024-09-10T23:36:21.288899+0000","last_undegraded":"2024-09-10T23:36:21.288899+0000","last_fullsized":"2024-09-10T23:36:21.288899+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:51.458193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033626+0000","last_change":"2024-09-10T23:36:20.022961+0000","last_active":"2024-09-10T23:36:21.033626+0000","last_peered":"2024-09-10T23:36:21.033626+0000","last_clean":"2024-09-10T23:36:21.033626+0000","last_became_active":"2024-09-10T23:36:20.022524+0000","last_became_peered":"2024-09-10T23:36:20.022524+0000","last_unstale":"2024-09-10T23:36:21.033626+0000","last_undegraded":"2024-09-10T23:36:21.033626+0000","last_fullsized":"2024-09-10T23:36:21.033626+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:16.045657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030757+0000","last_change":"2024-09-10T23:36:20.029568+0000","last_active":"2024-09-10T23:36:21.030757+0000","last_peered":"2024-09-10T23:36:21.030757+0000","last_clean":"2024-09-10T23:36:21.030757+0000","last_became_active":"2024-09-10T23:36:20.029269+0000","last_became_peered":"2024-09-10T23:36:20.029269+0000","last_unstale":"2024-09-10T23:36:21.030757+0000","last_undegraded":"2024-09-10T23:36:21.030757+0000","last_fullsized":"2024-09-10T23:36:21.030757+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:25:46.983237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.009377+0000","last_change":"2024-09-10T23:36:20.029394+0000","last_active":"2024-09-10T23:36:21.009377+0000","last_peered":"2024-09-10T23:36:21.009377+0000","last_clean":"2024-09-10T23:36:21.009377+0000","last_became_active":"2024-09-10T23:36:20.029079+0000","last_became_peered":"2024-09-10T23:36:20.029079+0000","last_unstale":"2024-09-10T23:36:21.009377+0000","last_undegraded":"2024-09-10T23:36:21.009377+0000","last_fullsized":"2024-09-10T23:36:21.009377+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:23.949351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288680+0000","last_change":"2024-09-10T23:36:20.036241+0000","last_active":"2024-09-10T23:36:21.288680+0000","last_peered":"2024-09-10T23:36:21.288680+0000","last_clean":"2024-09-10T23:36:21.288680+0000","last_became_active":"2024-09-10T23:36:20.035965+0000","last_became_peered":"2024-09-10T23:36:20.035965+0000","last_unstale":"2024-09-10T23:36:21.288680+0000","last_undegraded":"2024-09-10T23:36:21.288680+0000","last_fullsized":"2024-09-10T23:36:21.288680+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:40.613548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.009053+0000","last_change":"2024-09-10T23:36:20.025057+0000","last_active":"2024-09-10T23:36:21.009053+0000","last_peered":"2024-09-10T23:36:21.009053+0000","last_clean":"2024-09-10T23:36:21.009053+0000","last_became_active":"2024-09-10T23:36:20.024588+0000","last_became_peered":"2024-09-10T23:36:20.024588+0000","last_unstale":"2024-09-10T23:36:21.009053+0000","last_undegraded":"2024-09-10T23:36:21.009053+0000","last_fullsized":"2024-09-10T23:36:21.009053+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:05.271339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.008184+0000","last_change":"2024-09-10T23:36:20.019283+0000","last_active":"2024-09-10T23:36:21.008184+0000","last_peered":"2024-09-10T23:36:21.008184+0000","last_clean":"2024-09-10T23:36:21.008184+0000","last_became_active":"2024-09-10T23:36:20.018869+0000","last_became_peered":"2024-09-10T23:36:20.018869+0000","last_unstale":"2024-09-10T23:36:21.008184+0000","last_undegraded":"2024-09-10T23:36:21.008184+0000","last_fullsized":"2024-09-10T23:36:21.008184+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:22.038276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030189+0000","last_change":"2024-09-10T23:36:20.030690+0000","last_active":"2024-09-10T23:36:21.030189+0000","last_peered":"2024-09-10T23:36:21.030189+0000","last_clean":"2024-09-10T23:36:21.030189+0000","last_became_active":"2024-09-10T23:36:20.030391+0000","last_became_peered":"2024-09-10T23:36:20.030391+0000","last_unstale":"2024-09-10T23:36:21.030189+0000","last_undegraded":"2024-09-10T23:36:21.030189+0000","last_fullsized":"2024-09-10T23:36:21.030189+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:20.396257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289342+0000","last_change":"2024-09-10T23:36:20.039019+0000","last_active":"2024-09-10T23:36:21.289342+0000","last_peered":"2024-09-10T23:36:21.289342+0000","last_clean":"2024-09-10T23:36:21.289342+0000","last_became_active":"2024-09-10T23:36:20.038402+0000","last_became_peered":"2024-09-10T23:36:20.038402+0000","last_unstale":"2024-09-10T23:36:21.289342+0000","last_undegraded":"2024-09-10T23:36:21.289342+0000","last_fullsized":"2024-09-10T23:36:21.289342+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:26.191671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289584+0000","last_change":"2024-09-10T23:36:20.027947+0000","last_active":"2024-09-10T23:36:21.289584+0000","last_peered":"2024-09-10T23:36:21.289584+0000","last_clean":"2024-09-10T23:36:21.289584+0000","last_became_active":"2024-09-10T23:36:20.027680+0000","last_became_peered":"2024-09-10T23:36:20.027680+0000","last_unstale":"2024-09-10T23:36:21.289584+0000","last_undegraded":"2024-09-10T23:36:21.289584+0000","last_fullsized":"2024-09-10T23:36:21.289584+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:09.479231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030644+0000","last_change":"2024-09-10T23:36:20.038789+0000","last_active":"2024-09-10T23:36:21.030644+0000","last_peered":"2024-09-10T23:36:21.030644+0000","last_clean":"2024-09-10T23:36:21.030644+0000","last_became_active":"2024-09-10T23:36:20.038398+0000","last_became_peered":"2024-09-10T23:36:20.038398+0000","last_unstale":"2024-09-10T23:36:21.030644+0000","last_undegraded":"2024-09-10T23:36:21.030644+0000","last_fullsized":"2024-09-10T23:36:21.030644+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:19.927593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288656+0000","last_change":"2024-09-10T23:36:20.029210+0000","last_active":"2024-09-10T23:36:21.288656+0000","last_peered":"2024-09-10T23:36:21.288656+0000","last_clean":"2024-09-10T23:36:21.288656+0000","last_became_active":"2024-09-10T23:36:20.028830+0000","last_became_peered":"2024-09-10T23:36:20.028830+0000","last_unstale":"2024-09-10T23:36:21.288656+0000","last_undegraded":"2024-09-10T23:36:21.288656+0000","last_fullsized":"2024-09-10T23:36:21.288656+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:43.055413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287405+0000","last_change":"2024-09-10T23:36:20.027209+0000","last_active":"2024-09-10T23:36:21.287405+0000","last_peered":"2024-09-10T23:36:21.287405+0000","last_clean":"2024-09-10T23:36:21.287405+0000","last_became_active":"2024-09-10T23:36:20.026847+0000","last_became_peered":"2024-09-10T23:36:20.026847+0000","last_unstale":"2024-09-10T23:36:21.287405+0000","last_undegraded":"2024-09-10T23:36:21.287405+0000","last_fullsized":"2024-09-10T23:36:21.287405+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:27:38.276538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.006968+0000","last_change":"2024-09-10T23:36:20.025152+0000","last_active":"2024-09-10T23:36:21.006968+0000","last_peered":"2024-09-10T23:36:21.006968+0000","last_clean":"2024-09-10T23:36:21.006968+0000","last_became_active":"2024-09-10T23:36:20.024258+0000","last_became_peered":"2024-09-10T23:36:20.024258+0000","last_unstale":"2024-09-10T23:36:21.006968+0000","last_undegraded":"2024-09-10T23:36:21.006968+0000","last_fullsized":"2024-09-10T23:36:21.006968+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:00.660346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033143+0000","last_change":"2024-09-10T23:36:20.016548+0000","last_active":"2024-09-10T23:36:21.033143+0000","last_peered":"2024-09-10T23:36:21.033143+0000","last_clean":"2024-09-10T23:36:21.033143+0000","last_became_active":"2024-09-10T23:36:20.016257+0000","last_became_peered":"2024-09-10T23:36:20.016257+0000","last_unstale":"2024-09-10T23:36:21.033143+0000","last_undegraded":"2024-09-10T23:36:21.033143+0000","last_fullsized":"2024-09-10T23:36:21.033143+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:34.056422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030362+0000","last_change":"2024-09-10T23:36:20.030646+0000","last_active":"2024-09-10T23:36:21.030362+0000","last_peered":"2024-09-10T23:36:21.030362+0000","last_clean":"2024-09-10T23:36:21.030362+0000","last_became_active":"2024-09-10T23:36:20.030051+0000","last_became_peered":"2024-09-10T23:36:20.030051+0000","last_unstale":"2024-09-10T23:36:21.030362+0000","last_undegraded":"2024-09-10T23:36:21.030362+0000","last_fullsized":"2024-09-10T23:36:21.030362+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:24.939026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033630+0000","last_change":"2024-09-10T23:36:20.016660+0000","last_active":"2024-09-10T23:36:21.033630+0000","last_peered":"2024-09-10T23:36:21.033630+0000","last_clean":"2024-09-10T23:36:21.033630+0000","last_became_active":"2024-09-10T23:36:20.016352+0000","last_became_peered":"2024-09-10T23:36:20.016352+0000","last_unstale":"2024-09-10T23:36:21.033630+0000","last_undegraded":"2024-09-10T23:36:21.033630+0000","last_fullsized":"2024-09-10T23:36:21.033630+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:48:37.118518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030712+0000","last_change":"2024-09-10T23:36:20.035385+0000","last_active":"2024-09-10T23:36:21.030712+0000","last_peered":"2024-09-10T23:36:21.030712+0000","last_clean":"2024-09-10T23:36:21.030712+0000","last_became_active":"2024-09-10T23:36:20.035012+0000","last_became_peered":"2024-09-10T23:36:20.035012+0000","last_unstale":"2024-09-10T23:36:21.030712+0000","last_undegraded":"2024-09-10T23:36:21.030712+0000","last_fullsized":"2024-09-10T23:36:21.030712+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:49.694876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289026+0000","last_change":"2024-09-10T23:36:20.025247+0000","last_active":"2024-09-10T23:36:21.289026+0000","last_peered":"2024-09-10T23:36:21.289026+0000","last_clean":"2024-09-10T23:36:21.289026+0000","last_became_active":"2024-09-10T23:36:20.024632+0000","last_became_peered":"2024-09-10T23:36:20.024632+0000","last_unstale":"2024-09-10T23:36:21.289026+0000","last_undegraded":"2024-09-10T23:36:21.289026+0000","last_fullsized":"2024-09-10T23:36:21.289026+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:30.604978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287415+0000","last_change":"2024-09-10T23:36:20.029466+0000","last_active":"2024-09-10T23:36:21.287415+0000","last_peered":"2024-09-10T23:36:21.287415+0000","last_clean":"2024-09-10T23:36:21.287415+0000","last_became_active":"2024-09-10T23:36:20.027018+0000","last_became_peered":"2024-09-10T23:36:20.027018+0000","last_unstale":"2024-09-10T23:36:21.287415+0000","last_undegraded":"2024-09-10T23:36:21.287415+0000","last_fullsized":"2024-09-10T23:36:21.287415+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:00.660346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288824+0000","last_change":"2024-09-10T23:36:20.036469+0000","last_active":"2024-09-10T23:36:21.288824+0000","last_peered":"2024-09-10T23:36:21.288824+0000","last_clean":"2024-09-10T23:36:21.288824+0000","last_became_active":"2024-09-10T23:36:20.032983+0000","last_became_peered":"2024-09-10T23:36:20.032983+0000","last_unstale":"2024-09-10T23:36:21.288824+0000","last_undegraded":"2024-09-10T23:36:21.288824+0000","last_fullsized":"2024-09-10T23:36:21.288824+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:39.992856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030336+0000","last_change":"2024-09-10T23:36:20.027753+0000","last_active":"2024-09-10T23:36:21.030336+0000","last_peered":"2024-09-10T23:36:21.030336+0000","last_clean":"2024-09-10T23:36:21.030336+0000","last_became_active":"2024-09-10T23:36:20.026478+0000","last_became_peered":"2024-09-10T23:36:20.026478+0000","last_unstale":"2024-09-10T23:36:21.030336+0000","last_undegraded":"2024-09-10T23:36:21.030336+0000","last_fullsized":"2024-09-10T23:36:21.030336+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:17.739314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287388+0000","last_change":"2024-09-10T23:36:20.027567+0000","last_active":"2024-09-10T23:36:21.287388+0000","last_peered":"2024-09-10T23:36:21.287388+0000","last_clean":"2024-09-10T23:36:21.287388+0000","last_became_active":"2024-09-10T23:36:20.027272+0000","last_became_peered":"2024-09-10T23:36:20.027272+0000","last_unstale":"2024-09-10T23:36:21.287388+0000","last_undegraded":"2024-09-10T23:36:21.287388+0000","last_fullsized":"2024-09-10T23:36:21.287388+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:43.377507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287272+0000","last_change":"2024-09-10T23:36:20.027894+0000","last_active":"2024-09-10T23:36:21.287272+0000","last_peered":"2024-09-10T23:36:21.287272+0000","last_clean":"2024-09-10T23:36:21.287272+0000","last_became_active":"2024-09-10T23:36:20.027426+0000","last_became_peered":"2024-09-10T23:36:20.027426+0000","last_unstale":"2024-09-10T23:36:21.287272+0000","last_undegraded":"2024-09-10T23:36:21.287272+0000","last_fullsized":"2024-09-10T23:36:21.287272+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:42.136223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030801+0000","last_change":"2024-09-10T23:36:20.024322+0000","last_active":"2024-09-10T23:36:21.030801+0000","last_peered":"2024-09-10T23:36:21.030801+0000","last_clean":"2024-09-10T23:36:21.030801+0000","last_became_active":"2024-09-10T23:36:20.024019+0000","last_became_peered":"2024-09-10T23:36:20.024019+0000","last_unstale":"2024-09-10T23:36:21.030801+0000","last_undegraded":"2024-09-10T23:36:21.030801+0000","last_fullsized":"2024-09-10T23:36:21.030801+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:25.060145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.006986+0000","last_change":"2024-09-10T23:36:19.020218+0000","last_active":"2024-09-10T23:36:21.006986+0000","last_peered":"2024-09-10T23:36:21.006986+0000","last_clean":"2024-09-10T23:36:21.006986+0000","last_became_active":"2024-09-10T23:36:19.019884+0000","last_became_peered":"2024-09-10T23:36:19.019884+0000","last_unstale":"2024-09-10T23:36:21.006986+0000","last_undegraded":"2024-09-10T23:36:21.006986+0000","last_fullsized":"2024-09-10T23:36:21.006986+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:25:20.502178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289123+0000","last_change":"2024-09-10T23:36:20.042051+0000","last_active":"2024-09-10T23:36:21.289123+0000","last_peered":"2024-09-10T23:36:21.289123+0000","last_clean":"2024-09-10T23:36:21.289123+0000","last_became_active":"2024-09-10T23:36:20.041741+0000","last_became_peered":"2024-09-10T23:36:20.041741+0000","last_unstale":"2024-09-10T23:36:21.289123+0000","last_undegraded":"2024-09-10T23:36:21.289123+0000","last_fullsized":"2024-09-10T23:36:21.289123+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:18.227159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289696+0000","last_change":"2024-09-10T23:36:19.022835+0000","last_active":"2024-09-10T23:36:21.289696+0000","last_peered":"2024-09-10T23:36:21.289696+0000","last_clean":"2024-09-10T23:36:21.289696+0000","last_became_active":"2024-09-10T23:36:19.022544+0000","last_became_peered":"2024-09-10T23:36:19.022544+0000","last_unstale":"2024-09-10T23:36:21.289696+0000","last_undegraded":"2024-09-10T23:36:21.289696+0000","last_fullsized":"2024-09-10T23:36:21.289696+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:45.433150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.007338+0000","last_change":"2024-09-10T23:36:20.024221+0000","last_active":"2024-09-10T23:36:21.007338+0000","last_peered":"2024-09-10T23:36:21.007338+0000","last_clean":"2024-09-10T23:36:21.007338+0000","last_became_active":"2024-09-10T23:36:20.023960+0000","last_became_peered":"2024-09-10T23:36:20.023960+0000","last_unstale":"2024-09-10T23:36:21.007338+0000","last_undegraded":"2024-09-10T23:36:21.007338+0000","last_fullsized":"2024-09-10T23:36:21.007338+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:05.271339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030768+0000","last_change":"2024-09-10T23:36:19.028889+0000","last_active":"2024-09-10T23:36:21.030768+0000","last_peered":"2024-09-10T23:36:21.030768+0000","last_clean":"2024-09-10T23:36:21.030768+0000","last_became_active":"2024-09-10T23:36:19.028611+0000","last_became_peered":"2024-09-10T23:36:19.028611+0000","last_unstale":"2024-09-10T23:36:21.030768+0000","last_undegraded":"2024-09-10T23:36:21.030768+0000","last_fullsized":"2024-09-10T23:36:21.030768+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:29.598577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033281+0000","last_change":"2024-09-10T23:36:20.015665+0000","last_active":"2024-09-10T23:36:21.033281+0000","last_peered":"2024-09-10T23:36:21.033281+0000","last_clean":"2024-09-10T23:36:21.033281+0000","last_became_active":"2024-09-10T23:36:20.015366+0000","last_became_peered":"2024-09-10T23:36:20.015366+0000","last_unstale":"2024-09-10T23:36:21.033281+0000","last_undegraded":"2024-09-10T23:36:21.033281+0000","last_fullsized":"2024-09-10T23:36:21.033281+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:18.227159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.007289+0000","last_change":"2024-09-10T23:36:19.015875+0000","last_active":"2024-09-10T23:36:21.007289+0000","last_peered":"2024-09-10T23:36:21.007289+0000","last_clean":"2024-09-10T23:36:21.007289+0000","last_became_active":"2024-09-10T23:36:19.015619+0000","last_became_peered":"2024-09-10T23:36:19.015619+0000","last_unstale":"2024-09-10T23:36:21.007289+0000","last_undegraded":"2024-09-10T23:36:21.007289+0000","last_fullsized":"2024-09-10T23:36:21.007289+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:11.311110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030333+0000","last_change":"2024-09-10T23:36:20.020951+0000","last_active":"2024-09-10T23:36:21.030333+0000","last_peered":"2024-09-10T23:36:21.030333+0000","last_clean":"2024-09-10T23:36:21.030333+0000","last_became_active":"2024-09-10T23:36:20.020639+0000","last_became_peered":"2024-09-10T23:36:20.020639+0000","last_unstale":"2024-09-10T23:36:21.030333+0000","last_undegraded":"2024-09-10T23:36:21.030333+0000","last_fullsized":"2024-09-10T23:36:21.030333+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:16:15.280849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.009019+0000","last_change":"2024-09-10T23:36:19.020177+0000","last_active":"2024-09-10T23:36:21.009019+0000","last_peered":"2024-09-10T23:36:21.009019+0000","last_clean":"2024-09-10T23:36:21.009019+0000","last_became_active":"2024-09-10T23:36:19.019826+0000","last_became_peered":"2024-09-10T23:36:19.019826+0000","last_unstale":"2024-09-10T23:36:21.009019+0000","last_undegraded":"2024-09-10T23:36:21.009019+0000","last_fullsized":"2024-09-10T23:36:21.009019+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:41.071865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.007040+0000","last_change":"2024-09-10T23:36:20.022555+0000","last_active":"2024-09-10T23:36:21.007040+0000","last_peered":"2024-09-10T23:36:21.007040+0000","last_clean":"2024-09-10T23:36:21.007040+0000","last_became_active":"2024-09-10T23:36:20.022266+0000","last_became_peered":"2024-09-10T23:36:20.022266+0000","last_unstale":"2024-09-10T23:36:21.007040+0000","last_undegraded":"2024-09-10T23:36:21.007040+0000","last_fullsized":"2024-09-10T23:36:21.007040+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:22.038276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033072+0000","last_change":"2024-09-10T23:36:19.025715+0000","last_active":"2024-09-10T23:36:21.033072+0000","last_peered":"2024-09-10T23:36:21.033072+0000","last_clean":"2024-09-10T23:36:21.033072+0000","last_became_active":"2024-09-10T23:36:19.020534+0000","last_became_peered":"2024-09-10T23:36:19.020534+0000","last_unstale":"2024-09-10T23:36:21.033072+0000","last_undegraded":"2024-09-10T23:36:21.033072+0000","last_fullsized":"2024-09-10T23:36:21.033072+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:52.389059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T23:36:20.042361+0000","last_change":"2024-09-10T23:36:20.028676+0000","last_active":"2024-09-10T23:36:20.042361+0000","last_peered":"2024-09-10T23:36:20.042361+0000","last_clean":"2024-09-10T23:36:20.042361+0000","last_became_active":"2024-09-10T23:36:20.028395+0000","last_became_peered":"2024-09-10T23:36:20.028395+0000","last_unstale":"2024-09-10T23:36:20.042361+0000","last_undegraded":"2024-09-10T23:36:20.042361+0000","last_fullsized":"2024-09-10T23:36:20.042361+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:29:42.456259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T23:36:20.013339+0000","last_change":"2024-09-10T23:36:19.020666+0000","last_active":"2024-09-10T23:36:20.013339+0000","last_peered":"2024-09-10T23:36:20.013339+0000","last_clean":"2024-09-10T23:36:20.013339+0000","last_became_active":"2024-09-10T23:36:19.020316+0000","last_became_peered":"2024-09-10T23:36:19.020316+0000","last_unstale":"2024-09-10T23:36:20.013339+0000","last_undegraded":"2024-09-10T23:36:20.013339+0000","last_fullsized":"2024-09-10T23:36:20.013339+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:04.135022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.008390+0000","last_change":"2024-09-10T23:36:20.021927+0000","last_active":"2024-09-10T23:36:21.008390+0000","last_peered":"2024-09-10T23:36:21.008390+0000","last_clean":"2024-09-10T23:36:21.008390+0000","last_became_active":"2024-09-10T23:36:20.021694+0000","last_became_peered":"2024-09-10T23:36:20.021694+0000","last_unstale":"2024-09-10T23:36:21.008390+0000","last_undegraded":"2024-09-10T23:36:21.008390+0000","last_fullsized":"2024-09-10T23:36:21.008390+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:58.056854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.007889+0000","last_change":"2024-09-10T23:36:19.018425+0000","last_active":"2024-09-10T23:36:21.007889+0000","last_peered":"2024-09-10T23:36:21.007889+0000","last_clean":"2024-09-10T23:36:21.007889+0000","last_became_active":"2024-09-10T23:36:19.018091+0000","last_became_peered":"2024-09-10T23:36:19.018091+0000","last_unstale":"2024-09-10T23:36:21.007889+0000","last_undegraded":"2024-09-10T23:36:21.007889+0000","last_fullsized":"2024-09-10T23:36:21.007889+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:56.848956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033591+0000","last_change":"2024-09-10T23:36:20.017610+0000","last_active":"2024-09-10T23:36:21.033591+0000","last_peered":"2024-09-10T23:36:21.033591+0000","last_clean":"2024-09-10T23:36:21.033591+0000","last_became_active":"2024-09-10T23:36:20.017324+0000","last_became_peered":"2024-09-10T23:36:20.017324+0000","last_unstale":"2024-09-10T23:36:21.033591+0000","last_undegraded":"2024-09-10T23:36:21.033591+0000","last_fullsized":"2024-09-10T23:36:21.033591+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:45.146267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030631+0000","last_change":"2024-09-10T23:36:19.024545+0000","last_active":"2024-09-10T23:36:21.030631+0000","last_peered":"2024-09-10T23:36:21.030631+0000","last_clean":"2024-09-10T23:36:21.030631+0000","last_became_active":"2024-09-10T23:36:19.024289+0000","last_became_peered":"2024-09-10T23:36:19.024289+0000","last_unstale":"2024-09-10T23:36:21.030631+0000","last_undegraded":"2024-09-10T23:36:21.030631+0000","last_fullsized":"2024-09-10T23:36:21.030631+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:42.049011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287432+0000","last_change":"2024-09-10T23:36:20.025694+0000","last_active":"2024-09-10T23:36:21.287432+0000","last_peered":"2024-09-10T23:36:21.287432+0000","last_clean":"2024-09-10T23:36:21.287432+0000","last_became_active":"2024-09-10T23:36:20.025391+0000","last_became_peered":"2024-09-10T23:36:20.025391+0000","last_unstale":"2024-09-10T23:36:21.287432+0000","last_undegraded":"2024-09-10T23:36:21.287432+0000","last_fullsized":"2024-09-10T23:36:21.287432+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:22.038276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287570+0000","last_change":"2024-09-10T23:36:19.024336+0000","last_active":"2024-09-10T23:36:21.287570+0000","last_peered":"2024-09-10T23:36:21.287570+0000","last_clean":"2024-09-10T23:36:21.287570+0000","last_became_active":"2024-09-10T23:36:19.023540+0000","last_became_peered":"2024-09-10T23:36:19.023540+0000","last_unstale":"2024-09-10T23:36:21.287570+0000","last_undegraded":"2024-09-10T23:36:21.287570+0000","last_fullsized":"2024-09-10T23:36:21.287570+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:34.975621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030683+0000","last_change":"2024-09-10T23:36:20.029664+0000","last_active":"2024-09-10T23:36:21.030683+0000","last_peered":"2024-09-10T23:36:21.030683+0000","last_clean":"2024-09-10T23:36:21.030683+0000","last_became_active":"2024-09-10T23:36:20.029384+0000","last_became_peered":"2024-09-10T23:36:20.029384+0000","last_unstale":"2024-09-10T23:36:21.030683+0000","last_undegraded":"2024-09-10T23:36:21.030683+0000","last_fullsized":"2024-09-10T23:36:21.030683+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:13.253664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.009327+0000","last_change":"2024-09-10T23:36:19.017076+0000","last_active":"2024-09-10T23:36:21.009327+0000","last_peered":"2024-09-10T23:36:21.009327+0000","last_clean":"2024-09-10T23:36:21.009327+0000","last_became_active":"2024-09-10T23:36:19.016745+0000","last_became_peered":"2024-09-10T23:36:19.016745+0000","last_unstale":"2024-09-10T23:36:21.009327+0000","last_undegraded":"2024-09-10T23:36:21.009327+0000","last_fullsized":"2024-09-10T23:36:21.009327+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:46.227666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289275+0000","last_change":"2024-09-10T23:36:20.025547+0000","last_active":"2024-09-10T23:36:21.289275+0000","last_peered":"2024-09-10T23:36:21.289275+0000","last_clean":"2024-09-10T23:36:21.289275+0000","last_became_active":"2024-09-10T23:36:20.024978+0000","last_became_peered":"2024-09-10T23:36:20.024978+0000","last_unstale":"2024-09-10T23:36:21.289275+0000","last_undegraded":"2024-09-10T23:36:21.289275+0000","last_fullsized":"2024-09-10T23:36:21.289275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:22.933224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":19,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T23:36:20.048372+0000","last_change":"2024-09-10T23:36:19.022284+0000","last_active":"2024-09-10T23:36:20.048372+0000","last_peered":"2024-09-10T23:36:20.048372+0000","last_clean":"2024-09-10T23:36:20.048372+0000","last_became_active":"2024-09-10T23:36:19.021141+0000","last_became_peered":"2024-09-10T23:36:19.021141+0000","last_unstale":"2024-09-10T23:36:20.048372+0000","last_undegraded":"2024-09-10T23:36:20.048372+0000","last_fullsized":"2024-09-10T23:36:20.048372+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:53:30.820182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.007175+0000","last_change":"2024-09-10T23:36:20.025665+0000","last_active":"2024-09-10T23:36:21.007175+0000","last_peered":"2024-09-10T23:36:21.007175+0000","last_clean":"2024-09-10T23:36:21.007175+0000","last_became_active":"2024-09-10T23:36:20.023011+0000","last_became_peered":"2024-09-10T23:36:20.023011+0000","last_unstale":"2024-09-10T23:36:21.007175+0000","last_undegraded":"2024-09-10T23:36:21.007175+0000","last_fullsized":"2024-09-10T23:36:21.007175+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:58.056854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T23:36:20.045501+0000","last_change":"2024-09-10T23:36:19.020496+0000","last_active":"2024-09-10T23:36:20.045501+0000","last_peered":"2024-09-10T23:36:20.045501+0000","last_clean":"2024-09-10T23:36:20.045501+0000","last_became_active":"2024-09-10T23:36:19.020267+0000","last_became_peered":"2024-09-10T23:36:19.020267+0000","last_unstale":"2024-09-10T23:36:20.045501+0000","last_undegraded":"2024-09-10T23:36:20.045501+0000","last_fullsized":"2024-09-10T23:36:20.045501+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:28.089952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033278+0000","last_change":"2024-09-10T23:36:20.023520+0000","last_active":"2024-09-10T23:36:21.033278+0000","last_peered":"2024-09-10T23:36:21.033278+0000","last_clean":"2024-09-10T23:36:21.033278+0000","last_became_active":"2024-09-10T23:36:20.020356+0000","last_became_peered":"2024-09-10T23:36:20.020356+0000","last_unstale":"2024-09-10T23:36:21.033278+0000","last_undegraded":"2024-09-10T23:36:21.033278+0000","last_fullsized":"2024-09-10T23:36:21.033278+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:53.266472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033362+0000","last_change":"2024-09-10T23:36:19.019162+0000","last_active":"2024-09-10T23:36:21.033362+0000","last_peered":"2024-09-10T23:36:21.033362+0000","last_clean":"2024-09-10T23:36:21.033362+0000","last_became_active":"2024-09-10T23:36:19.018864+0000","last_became_peered":"2024-09-10T23:36:19.018864+0000","last_unstale":"2024-09-10T23:36:21.033362+0000","last_undegraded":"2024-09-10T23:36:21.033362+0000","last_fullsized":"2024-09-10T23:36:21.033362+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:15.888189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288762+0000","last_change":"2024-09-10T23:36:20.024099+0000","last_active":"2024-09-10T23:36:21.288762+0000","last_peered":"2024-09-10T23:36:21.288762+0000","last_clean":"2024-09-10T23:36:21.288762+0000","last_became_active":"2024-09-10T23:36:20.023846+0000","last_became_peered":"2024-09-10T23:36:20.023846+0000","last_unstale":"2024-09-10T23:36:21.288762+0000","last_undegraded":"2024-09-10T23:36:21.288762+0000","last_fullsized":"2024-09-10T23:36:21.288762+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:47.186028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033261+0000","last_change":"2024-09-10T23:36:19.016343+0000","last_active":"2024-09-10T23:36:21.033261+0000","last_peered":"2024-09-10T23:36:21.033261+0000","last_clean":"2024-09-10T23:36:21.033261+0000","last_became_active":"2024-09-10T23:36:19.015988+0000","last_became_peered":"2024-09-10T23:36:19.015988+0000","last_unstale":"2024-09-10T23:36:21.033261+0000","last_undegraded":"2024-09-10T23:36:21.033261+0000","last_fullsized":"2024-09-10T23:36:21.033261+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:04:57.866345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288470+0000","last_change":"2024-09-10T23:36:20.042708+0000","last_active":"2024-09-10T23:36:21.288470+0000","last_peered":"2024-09-10T23:36:21.288470+0000","last_clean":"2024-09-10T23:36:21.288470+0000","last_became_active":"2024-09-10T23:36:20.041834+0000","last_became_peered":"2024-09-10T23:36:20.041834+0000","last_unstale":"2024-09-10T23:36:21.288470+0000","last_undegraded":"2024-09-10T23:36:21.288470+0000","last_fullsized":"2024-09-10T23:36:21.288470+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:51.774172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":23,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.032884+0000","last_change":"2024-09-10T23:36:19.017661+0000","last_active":"2024-09-10T23:36:21.032884+0000","last_peered":"2024-09-10T23:36:21.032884+0000","last_clean":"2024-09-10T23:36:21.032884+0000","last_became_active":"2024-09-10T23:36:19.017084+0000","last_became_peered":"2024-09-10T23:36:19.017084+0000","last_unstale":"2024-09-10T23:36:21.032884+0000","last_undegraded":"2024-09-10T23:36:21.032884+0000","last_fullsized":"2024-09-10T23:36:21.032884+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:22.350535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287466+0000","last_change":"2024-09-10T23:36:20.022114+0000","last_active":"2024-09-10T23:36:21.287466+0000","last_peered":"2024-09-10T23:36:21.287466+0000","last_clean":"2024-09-10T23:36:21.287466+0000","last_became_active":"2024-09-10T23:36:20.021447+0000","last_became_peered":"2024-09-10T23:36:20.021447+0000","last_unstale":"2024-09-10T23:36:21.287466+0000","last_undegraded":"2024-09-10T23:36:21.287466+0000","last_fullsized":"2024-09-10T23:36:21.287466+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:58.056854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030741+0000","last_change":"2024-09-10T23:36:19.028083+0000","last_active":"2024-09-10T23:36:21.030741+0000","last_peered":"2024-09-10T23:36:21.030741+0000","last_clean":"2024-09-10T23:36:21.030741+0000","last_became_active":"2024-09-10T23:36:19.025982+0000","last_became_peered":"2024-09-10T23:36:19.025982+0000","last_unstale":"2024-09-10T23:36:21.030741+0000","last_undegraded":"2024-09-10T23:36:21.030741+0000","last_fullsized":"2024-09-10T23:36:21.030741+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:46.179627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287495+0000","last_change":"2024-09-10T23:36:20.017954+0000","last_active":"2024-09-10T23:36:21.287495+0000","last_peered":"2024-09-10T23:36:21.287495+0000","last_clean":"2024-09-10T23:36:21.287495+0000","last_became_active":"2024-09-10T23:36:20.017623+0000","last_became_peered":"2024-09-10T23:36:20.017623+0000","last_unstale":"2024-09-10T23:36:21.287495+0000","last_undegraded":"2024-09-10T23:36:21.287495+0000","last_fullsized":"2024-09-10T23:36:21.287495+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:50.318502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287591+0000","last_change":"2024-09-10T23:36:19.019143+0000","last_active":"2024-09-10T23:36:21.287591+0000","last_peered":"2024-09-10T23:36:21.287591+0000","last_clean":"2024-09-10T23:36:21.287591+0000","last_became_active":"2024-09-10T23:36:19.018868+0000","last_became_peered":"2024-09-10T23:36:19.018868+0000","last_unstale":"2024-09-10T23:36:21.287591+0000","last_undegraded":"2024-09-10T23:36:21.287591+0000","last_fullsized":"2024-09-10T23:36:21.287591+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:14.976336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289094+0000","last_change":"2024-09-10T23:36:20.035368+0000","last_active":"2024-09-10T23:36:21.289094+0000","last_peered":"2024-09-10T23:36:21.289094+0000","last_clean":"2024-09-10T23:36:21.289094+0000","last_became_active":"2024-09-10T23:36:20.035120+0000","last_became_peered":"2024-09-10T23:36:20.035120+0000","last_unstale":"2024-09-10T23:36:21.289094+0000","last_undegraded":"2024-09-10T23:36:21.289094+0000","last_fullsized":"2024-09-10T23:36:21.289094+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:31.959052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287151+0000","last_change":"2024-09-10T23:36:19.021077+0000","last_active":"2024-09-10T23:36:21.287151+0000","last_peered":"2024-09-10T23:36:21.287151+0000","last_clean":"2024-09-10T23:36:21.287151+0000","last_became_active":"2024-09-10T23:36:19.020683+0000","last_became_peered":"2024-09-10T23:36:19.020683+0000","last_unstale":"2024-09-10T23:36:21.287151+0000","last_undegraded":"2024-09-10T23:36:21.287151+0000","last_fullsized":"2024-09-10T23:36:21.287151+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:15.039256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289279+0000","last_change":"2024-09-10T23:36:20.032886+0000","last_active":"2024-09-10T23:36:21.289279+0000","last_peered":"2024-09-10T23:36:21.289279+0000","last_clean":"2024-09-10T23:36:21.289279+0000","last_became_active":"2024-09-10T23:36:20.032635+0000","last_became_peered":"2024-09-10T23:36:20.032635+0000","last_unstale":"2024-09-10T23:36:21.289279+0000","last_undegraded":"2024-09-10T23:36:21.289279+0000","last_fullsized":"2024-09-10T23:36:21.289279+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:48.752715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.007385+0000","last_change":"2024-09-10T23:36:19.023593+0000","last_active":"2024-09-10T23:36:21.007385+0000","last_peered":"2024-09-10T23:36:21.007385+0000","last_clean":"2024-09-10T23:36:21.007385+0000","last_became_active":"2024-09-10T23:36:19.023298+0000","last_became_peered":"2024-09-10T23:36:19.023298+0000","last_unstale":"2024-09-10T23:36:21.007385+0000","last_undegraded":"2024-09-10T23:36:21.007385+0000","last_fullsized":"2024-09-10T23:36:21.007385+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:53:30.350802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T23:36:20.042445+0000","last_change":"2024-09-10T23:36:20.029501+0000","last_active":"2024-09-10T23:36:20.042445+0000","last_peered":"2024-09-10T23:36:20.042445+0000","last_clean":"2024-09-10T23:36:20.042445+0000","last_became_active":"2024-09-10T23:36:20.026891+0000","last_became_peered":"2024-09-10T23:36:20.026891+0000","last_unstale":"2024-09-10T23:36:20.042445+0000","last_undegraded":"2024-09-10T23:36:20.042445+0000","last_fullsized":"2024-09-10T23:36:20.042445+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:37.093491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.007265+0000","last_change":"2024-09-10T23:36:19.019734+0000","last_active":"2024-09-10T23:36:21.007265+0000","last_peered":"2024-09-10T23:36:21.007265+0000","last_clean":"2024-09-10T23:36:21.007265+0000","last_became_active":"2024-09-10T23:36:19.019447+0000","last_became_peered":"2024-09-10T23:36:19.019447+0000","last_unstale":"2024-09-10T23:36:21.007265+0000","last_undegraded":"2024-09-10T23:36:21.007265+0000","last_fullsized":"2024-09-10T23:36:21.007265+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:52.241631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T23:36:20.039019+0000","last_change":"2024-09-10T23:36:20.019309+0000","last_active":"2024-09-10T23:36:20.039019+0000","last_peered":"2024-09-10T23:36:20.039019+0000","last_clean":"2024-09-10T23:36:20.039019+0000","last_became_active":"2024-09-10T23:36:20.019017+0000","last_became_peered":"2024-09-10T23:36:20.019017+0000","last_unstale":"2024-09-10T23:36:20.039019+0000","last_undegraded":"2024-09-10T23:36:20.039019+0000","last_fullsized":"2024-09-10T23:36:20.039019+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:31.161121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287309+0000","last_change":"2024-09-10T23:36:19.021026+0000","last_active":"2024-09-10T23:36:21.287309+0000","last_peered":"2024-09-10T23:36:21.287309+0000","last_clean":"2024-09-10T23:36:21.287309+0000","last_became_active":"2024-09-10T23:36:19.018580+0000","last_became_peered":"2024-09-10T23:36:19.018580+0000","last_unstale":"2024-09-10T23:36:21.287309+0000","last_undegraded":"2024-09-10T23:36:21.287309+0000","last_fullsized":"2024-09-10T23:36:21.287309+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:21.047623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.008555+0000","last_change":"2024-09-10T23:36:20.028309+0000","last_active":"2024-09-10T23:36:21.008555+0000","last_peered":"2024-09-10T23:36:21.008555+0000","last_clean":"2024-09-10T23:36:21.008555+0000","last_became_active":"2024-09-10T23:36:20.027822+0000","last_became_peered":"2024-09-10T23:36:20.027822+0000","last_unstale":"2024-09-10T23:36:21.008555+0000","last_undegraded":"2024-09-10T23:36:21.008555+0000","last_fullsized":"2024-09-10T23:36:21.008555+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:50.318502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288466+0000","last_change":"2024-09-10T23:36:19.021589+0000","last_active":"2024-09-10T23:36:21.288466+0000","last_peered":"2024-09-10T23:36:21.288466+0000","last_clean":"2024-09-10T23:36:21.288466+0000","last_became_active":"2024-09-10T23:36:19.021325+0000","last_became_peered":"2024-09-10T23:36:19.021325+0000","last_unstale":"2024-09-10T23:36:21.288466+0000","last_undegraded":"2024-09-10T23:36:21.288466+0000","last_fullsized":"2024-09-10T23:36:21.288466+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:19.736122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289585+0000","last_change":"2024-09-10T23:36:20.025380+0000","last_active":"2024-09-10T23:36:21.289585+0000","last_peered":"2024-09-10T23:36:21.289585+0000","last_clean":"2024-09-10T23:36:21.289585+0000","last_became_active":"2024-09-10T23:36:20.025113+0000","last_became_peered":"2024-09-10T23:36:20.025113+0000","last_unstale":"2024-09-10T23:36:21.289585+0000","last_undegraded":"2024-09-10T23:36:21.289585+0000","last_fullsized":"2024-09-10T23:36:21.289585+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:37.050818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030842+0000","last_change":"2024-09-10T23:36:19.026256+0000","last_active":"2024-09-10T23:36:21.030842+0000","last_peered":"2024-09-10T23:36:21.030842+0000","last_clean":"2024-09-10T23:36:21.030842+0000","last_became_active":"2024-09-10T23:36:19.025653+0000","last_became_peered":"2024-09-10T23:36:19.025653+0000","last_unstale":"2024-09-10T23:36:21.030842+0000","last_undegraded":"2024-09-10T23:36:21.030842+0000","last_fullsized":"2024-09-10T23:36:21.030842+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:51.432935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033678+0000","last_change":"2024-09-10T23:36:20.027159+0000","last_active":"2024-09-10T23:36:21.033678+0000","last_peered":"2024-09-10T23:36:21.033678+0000","last_clean":"2024-09-10T23:36:21.033678+0000","last_became_active":"2024-09-10T23:36:20.026190+0000","last_became_peered":"2024-09-10T23:36:20.026190+0000","last_unstale":"2024-09-10T23:36:21.033678+0000","last_undegraded":"2024-09-10T23:36:21.033678+0000","last_fullsized":"2024-09-10T23:36:21.033678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:15.982737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.032363+0000","last_change":"2024-09-10T23:36:19.021521+0000","last_active":"2024-09-10T23:36:21.032363+0000","last_peered":"2024-09-10T23:36:21.032363+0000","last_clean":"2024-09-10T23:36:21.032363+0000","last_became_active":"2024-09-10T23:36:19.021239+0000","last_became_peered":"2024-09-10T23:36:19.021239+0000","last_unstale":"2024-09-10T23:36:21.032363+0000","last_undegraded":"2024-09-10T23:36:21.032363+0000","last_fullsized":"2024-09-10T23:36:21.032363+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:21.468611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287277+0000","last_change":"2024-09-10T23:36:20.022366+0000","last_active":"2024-09-10T23:36:21.287277+0000","last_peered":"2024-09-10T23:36:21.287277+0000","last_clean":"2024-09-10T23:36:21.287277+0000","last_became_active":"2024-09-10T23:36:20.021872+0000","last_became_peered":"2024-09-10T23:36:20.021872+0000","last_unstale":"2024-09-10T23:36:21.287277+0000","last_undegraded":"2024-09-10T23:36:21.287277+0000","last_fullsized":"2024-09-10T23:36:21.287277+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:31.586172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.009391+0000","last_change":"2024-09-10T23:36:19.014145+0000","last_active":"2024-09-10T23:36:21.009391+0000","last_peered":"2024-09-10T23:36:21.009391+0000","last_clean":"2024-09-10T23:36:21.009391+0000","last_became_active":"2024-09-10T23:36:19.013594+0000","last_became_peered":"2024-09-10T23:36:19.013594+0000","last_unstale":"2024-09-10T23:36:21.009391+0000","last_undegraded":"2024-09-10T23:36:21.009391+0000","last_fullsized":"2024-09-10T23:36:21.009391+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:51.476811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033479+0000","last_change":"2024-09-10T23:36:20.019497+0000","last_active":"2024-09-10T23:36:21.033479+0000","last_peered":"2024-09-10T23:36:21.033479+0000","last_clean":"2024-09-10T23:36:21.033479+0000","last_became_active":"2024-09-10T23:36:20.019211+0000","last_became_peered":"2024-09-10T23:36:20.019211+0000","last_unstale":"2024-09-10T23:36:21.033479+0000","last_undegraded":"2024-09-10T23:36:21.033479+0000","last_fullsized":"2024-09-10T23:36:21.033479+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:57.865930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.009330+0000","last_change":"2024-09-10T23:36:19.017410+0000","last_active":"2024-09-10T23:36:21.009330+0000","last_peered":"2024-09-10T23:36:21.009330+0000","last_clean":"2024-09-10T23:36:21.009330+0000","last_became_active":"2024-09-10T23:36:19.016975+0000","last_became_peered":"2024-09-10T23:36:19.016975+0000","last_unstale":"2024-09-10T23:36:21.009330+0000","last_undegraded":"2024-09-10T23:36:21.009330+0000","last_fullsized":"2024-09-10T23:36:21.009330+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:21.431140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289529+0000","last_change":"2024-09-10T23:36:20.030079+0000","last_active":"2024-09-10T23:36:21.289529+0000","last_peered":"2024-09-10T23:36:21.289529+0000","last_clean":"2024-09-10T23:36:21.289529+0000","last_became_active":"2024-09-10T23:36:20.029105+0000","last_became_peered":"2024-09-10T23:36:20.029105+0000","last_unstale":"2024-09-10T23:36:21.289529+0000","last_undegraded":"2024-09-10T23:36:21.289529+0000","last_fullsized":"2024-09-10T23:36:21.289529+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:56.299901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.009132+0000","last_change":"2024-09-10T23:36:19.020315+0000","last_active":"2024-09-10T23:36:21.009132+0000","last_peered":"2024-09-10T23:36:21.009132+0000","last_clean":"2024-09-10T23:36:21.009132+0000","last_became_active":"2024-09-10T23:36:19.019928+0000","last_became_peered":"2024-09-10T23:36:19.019928+0000","last_unstale":"2024-09-10T23:36:21.009132+0000","last_undegraded":"2024-09-10T23:36:21.009132+0000","last_fullsized":"2024-09-10T23:36:21.009132+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:13.589708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.030499+0000","last_change":"2024-09-10T23:36:20.025120+0000","last_active":"2024-09-10T23:36:21.030499+0000","last_peered":"2024-09-10T23:36:21.030499+0000","last_clean":"2024-09-10T23:36:21.030499+0000","last_became_active":"2024-09-10T23:36:20.023938+0000","last_became_peered":"2024-09-10T23:36:20.023938+0000","last_unstale":"2024-09-10T23:36:21.030499+0000","last_undegraded":"2024-09-10T23:36:21.030499+0000","last_fullsized":"2024-09-10T23:36:21.030499+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:58.570190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.289481+0000","last_change":"2024-09-10T23:36:19.027466+0000","last_active":"2024-09-10T23:36:21.289481+0000","last_peered":"2024-09-10T23:36:21.289481+0000","last_clean":"2024-09-10T23:36:21.289481+0000","last_became_active":"2024-09-10T23:36:19.027220+0000","last_became_peered":"2024-09-10T23:36:19.027220+0000","last_unstale":"2024-09-10T23:36:21.289481+0000","last_undegraded":"2024-09-10T23:36:21.289481+0000","last_fullsized":"2024-09-10T23:36:21.289481+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:42.637323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-10T23:36:20.042387+0000","last_change":"2024-09-10T23:36:20.032463+0000","last_active":"2024-09-10T23:36:20.042387+0000","last_peered":"2024-09-10T23:36:20.042387+0000","last_clean":"2024-09-10T23:36:20.042387+0000","last_became_active":"2024-09-10T23:36:20.032170+0000","last_became_peered":"2024-09-10T23:36:20.032170+0000","last_unstale":"2024-09-10T23:36:20.042387+0000","last_undegraded":"2024-09-10T23:36:20.042387+0000","last_fullsized":"2024-09-10T23:36:20.042387+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:05:02.417345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287404+0000","last_change":"2024-09-10T23:36:19.027020+0000","last_active":"2024-09-10T23:36:21.287404+0000","last_peered":"2024-09-10T23:36:21.287404+0000","last_clean":"2024-09-10T23:36:21.287404+0000","last_became_active":"2024-09-10T23:36:19.026549+0000","last_became_peered":"2024-09-10T23:36:19.026549+0000","last_unstale":"2024-09-10T23:36:21.287404+0000","last_undegraded":"2024-09-10T23:36:21.287404+0000","last_fullsized":"2024-09-10T23:36:21.287404+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:36.496910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.287121+0000","last_change":"2024-09-10T23:36:20.021902+0000","last_active":"2024-09-10T23:36:21.287121+0000","last_peered":"2024-09-10T23:36:21.287121+0000","last_clean":"2024-09-10T23:36:21.287121+0000","last_became_active":"2024-09-10T23:36:20.019325+0000","last_became_peered":"2024-09-10T23:36:20.019325+0000","last_unstale":"2024-09-10T23:36:21.287121+0000","last_undegraded":"2024-09-10T23:36:21.287121+0000","last_fullsized":"2024-09-10T23:36:21.287121+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:14.596110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.006725+0000","last_change":"2024-09-10T23:36:19.018292+0000","last_active":"2024-09-10T23:36:21.006725+0000","last_peered":"2024-09-10T23:36:21.006725+0000","last_clean":"2024-09-10T23:36:21.006725+0000","last_became_active":"2024-09-10T23:36:19.017520+0000","last_became_peered":"2024-09-10T23:36:19.017520+0000","last_unstale":"2024-09-10T23:36:21.006725+0000","last_undegraded":"2024-09-10T23:36:21.006725+0000","last_fullsized":"2024-09-10T23:36:21.006725+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:08.464048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.288833+0000","last_change":"2024-09-10T23:36:20.033011+0000","last_active":"2024-09-10T23:36:21.288833+0000","last_peered":"2024-09-10T23:36:21.288833+0000","last_clean":"2024-09-10T23:36:21.288833+0000","last_became_active":"2024-09-10T23:36:20.032004+0000","last_became_peered":"2024-09-10T23:36:20.032004+0000","last_unstale":"2024-09-10T23:36:21.288833+0000","last_undegraded":"2024-09-10T23:36:21.288833+0000","last_fullsized":"2024-09-10T23:36:21.288833+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:18.991959+0000","last_clean_scrub_stamp":"2024-09-10T23:36:18.991959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:52.439336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.033314+0000","last_change":"2024-09-10T23:36:19.020456+0000","last_active":"2024-09-10T23:36:21.033314+0000","last_peered":"2024-09-10T23:36:21.033314+0000","last_clean":"2024-09-10T23:36:21.033314+0000","last_became_active":"2024-09-10T23:36:19.018692+0000","last_became_peered":"2024-09-10T23:36:19.018692+0000","last_unstale":"2024-09-10T23:36:21.033314+0000","last_undegraded":"2024-09-10T23:36:21.033314+0000","last_fullsized":"2024-09-10T23:36:21.033314+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:17.985558+0000","last_clean_scrub_stamp":"2024-09-10T23:36:17.985558+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:20.057823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":198,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T23:36:21.032903+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:36:21.032903+0000","last_peered":"2024-09-10T23:36:21.032903+0000","last_clean":"2024-09-10T23:36:21.032903+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:36:21.032903+0000","last_undegraded":"2024-09-10T23:36:21.032903+0000","last_fullsized":"2024-09-10T23:36:21.032903+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509491,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":831488,"data_stored":342446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509490,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1429504,"data_stored":937592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509488,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":843776,"data_stored":347828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":831488,"data_stored":340960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509490,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":823296,"data_stored":341416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509489,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1425408,"data_stored":937502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509489,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":823296,"data_stored":340904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542194,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":835584,"data_stored":342480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:36:24.507 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T23:36:24.508 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T23:36:24.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.690+0000 7ffb7c000640 1 -- 172.21.15.94:0/2413176355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb5c004d50 msgr2=0x7ffb5c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:24.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.690+0000 7ffb7c000640 1 --2- 172.21.15.94:0/2413176355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb5c004d50 0x7ffb5c0051d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009ee0 tx=0x7ffb6c01d5f0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.690+0000 7ffb7a000640 1 -- 172.21.15.94:0/2413176355 reap_dead start 2024-09-10T23:36:24.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.690+0000 7ffb7c000640 1 -- 172.21.15.94:0/2413176355 shutdown_connections 2024-09-10T23:36:24.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.690+0000 7ffb7c000640 1 -- 172.21.15.94:0/2413176355 >> 172.21.15.94:0/2413176355 conn(0x7ffb740d1c60 msgr2=0x7ffb740d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:24.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.690+0000 7ffb7c000640 1 -- 172.21.15.94:0/2413176355 shutdown_connections 2024-09-10T23:36:24.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb7c000640 1 -- 172.21.15.94:0/2413176355 wait complete. 2024-09-10T23:36:24.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb7c000640 1 Processor -- start 2024-09-10T23:36:24.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb7c000640 1 -- start start 2024-09-10T23:36:24.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb7c000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb5c004d50 0x7ffb7415f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:24.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb7c000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb740cb520 0x7ffb740b7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:24.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb7c000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb740b3750 0x7ffb740b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:24.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb7c000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb740b1ca0 con 0x7ffb740cb520 2024-09-10T23:36:24.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb7c000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffb740b1b20 con 0x7ffb5c004d50 2024-09-10T23:36:24.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb7c000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ffb741f89d0 con 0x7ffb740b3750 2024-09-10T23:36:24.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb78c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb740cb520 0x7ffb740b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:24.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb78c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb740cb520 0x7ffb740b7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51050/0 (socket says 172.21.15.94:51050) 2024-09-10T23:36:24.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb78c00640 1 -- 172.21.15.94:0/3399416856 learned_addr learned my addr 172.21.15.94:0/3399416856 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:24.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb79600640 1 --2- 172.21.15.94:0/3399416856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb5c004d50 0x7ffb7415f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:24.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb7a000640 1 --2- 172.21.15.94:0/3399416856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb740b3750 0x7ffb740b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:24.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb78c00640 1 -- 172.21.15.94:0/3399416856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb740b3750 msgr2=0x7ffb740b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:24.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb78c00640 1 --2- 172.21.15.94:0/3399416856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb740b3750 0x7ffb740b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb78c00640 1 -- 172.21.15.94:0/3399416856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb5c004d50 msgr2=0x7ffb7415f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:24.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb78c00640 1 --2- 172.21.15.94:0/3399416856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb5c004d50 0x7ffb7415f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb78c00640 1 -- 172.21.15.94:0/3399416856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb741f7c10 con 0x7ffb740cb520 2024-09-10T23:36:24.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb79600640 1 --2- 172.21.15.94:0/3399416856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb5c004d50 0x7ffb7415f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:24.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb78c00640 1 --2- 172.21.15.94:0/3399416856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb740cb520 0x7ffb740b7200 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ffb60011ae0 tx=0x7ffb60009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:24.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb71600640 1 -- 172.21.15.94:0/3399416856 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6000fcd0 con 0x7ffb740cb520 2024-09-10T23:36:24.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb71600640 1 -- 172.21.15.94:0/3399416856 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb6000fe70 con 0x7ffb740cb520 2024-09-10T23:36:24.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb7c000640 1 -- 172.21.15.94:0/3399416856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb741f7ef0 con 0x7ffb740cb520 2024-09-10T23:36:24.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb71600640 1 -- 172.21.15.94:0/3399416856 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60004260 con 0x7ffb740cb520 2024-09-10T23:36:24.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.694+0000 7ffb7c000640 1 -- 172.21.15.94:0/3399416856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb741f84b0 con 0x7ffb740cb520 2024-09-10T23:36:24.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.698+0000 7ffb7c000640 1 -- 172.21.15.94:0/3399416856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb740cd710 con 0x7ffb740cb520 2024-09-10T23:36:24.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.698+0000 7ffb71600640 1 -- 172.21.15.94:0/3399416856 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7ffb60003600 con 0x7ffb740cb520 2024-09-10T23:36:24.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.698+0000 7ffb71600640 1 --2- 172.21.15.94:0/3399416856 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ffb54072940 0x7ffb54074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:24.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.698+0000 7ffb71600640 1 -- 172.21.15.94:0/3399416856 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7ffb60094320 con 0x7ffb740cb520 2024-09-10T23:36:24.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.698+0000 7ffb79600640 1 --2- 172.21.15.94:0/3399416856 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ffb54072940 0x7ffb54074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:24.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.698+0000 7ffb79600640 1 --2- 172.21.15.94:0/3399416856 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ffb54072940 0x7ffb54074e00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffb680046e0 tx=0x7ffb68004610 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:24.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.702+0000 7ffb71600640 1 -- 172.21.15.94:0/3399416856 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ffb6005f530 con 0x7ffb740cb520 2024-09-10T23:36:24.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.926+0000 7ffb7c000640 1 -- 172.21.15.94:0/3399416856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ffb7406fd20 con 0x7ffb740cb520 2024-09-10T23:36:24.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.926+0000 7ffb71600640 1 -- 172.21.15.94:0/3399416856 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v33) ==== 190+0+147 (secure 0 0 0) 0x7ffb6005ee30 con 0x7ffb740cb520 2024-09-10T23:36:24.929 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-10T23:36:24.930 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g== 2024-09-10T23:36:24.930 INFO:teuthology.orchestra.run.smithi094.stdout: caps mds = "allow" 2024-09-10T23:36:24.930 INFO:teuthology.orchestra.run.smithi094.stdout: caps mon = "allow r" 2024-09-10T23:36:24.930 INFO:teuthology.orchestra.run.smithi094.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T23:36:24.930 INFO:teuthology.orchestra.run.smithi094.stderr:updated caps for client.0 2024-09-10T23:36:24.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.930+0000 7ffb7c000640 1 -- 172.21.15.94:0/3399416856 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ffb54072940 msgr2=0x7ffb54074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:24.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.930+0000 7ffb7c000640 1 --2- 172.21.15.94:0/3399416856 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ffb54072940 0x7ffb54074e00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffb680046e0 tx=0x7ffb68004610 comp rx=0 tx=0).stop 2024-09-10T23:36:24.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.930+0000 7ffb7c000640 1 -- 172.21.15.94:0/3399416856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb740cb520 msgr2=0x7ffb740b7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:24.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.930+0000 7ffb7c000640 1 --2- 172.21.15.94:0/3399416856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb740cb520 0x7ffb740b7200 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ffb60011ae0 tx=0x7ffb60009f20 comp rx=0 tx=0).stop 2024-09-10T23:36:24.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.930+0000 7ffb7c000640 1 -- 172.21.15.94:0/3399416856 shutdown_connections 2024-09-10T23:36:24.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.930+0000 7ffb7c000640 1 --2- 172.21.15.94:0/3399416856 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ffb54072940 0x7ffb54074e00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.930+0000 7ffb7c000640 1 --2- 172.21.15.94:0/3399416856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb740b3750 0x7ffb740b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.930+0000 7ffb7c000640 1 --2- 172.21.15.94:0/3399416856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb740cb520 0x7ffb740b7200 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.930+0000 7ffb7c000640 1 --2- 172.21.15.94:0/3399416856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb5c004d50 0x7ffb7415f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:24.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.930+0000 7ffb7c000640 1 -- 172.21.15.94:0/3399416856 >> 172.21.15.94:0/3399416856 conn(0x7ffb740d1c60 msgr2=0x7ffb74070760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:24.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.930+0000 7ffb7c000640 1 -- 172.21.15.94:0/3399416856 shutdown_connections 2024-09-10T23:36:24.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:24.930+0000 7ffb7c000640 1 -- 172.21.15.94:0/3399416856 wait complete. 2024-09-10T23:36:24.953 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:25.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.094+0000 7f83a8a00640 1 -- 172.21.15.94:0/787085244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8388004d50 msgr2=0x7f83880051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:25.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.094+0000 7f83a8a00640 1 --2- 172.21.15.94:0/787085244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8388004d50 0x7f83880051d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f839c009ee0 tx=0x7f839c01d5f0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.094+0000 7f83a3200640 1 -- 172.21.15.94:0/787085244 reap_dead start 2024-09-10T23:36:25.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.094+0000 7f83a8a00640 1 -- 172.21.15.94:0/787085244 shutdown_connections 2024-09-10T23:36:25.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.094+0000 7f83a8a00640 1 -- 172.21.15.94:0/787085244 >> 172.21.15.94:0/787085244 conn(0x7f83a40d0170 msgr2=0x7f83a40d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:25.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.094+0000 7f83a8a00640 1 -- 172.21.15.94:0/787085244 shutdown_connections 2024-09-10T23:36:25.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.094+0000 7f83a8a00640 1 -- 172.21.15.94:0/787085244 wait complete. 2024-09-10T23:36:25.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a8a00640 1 Processor -- start 2024-09-10T23:36:25.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a8a00640 1 -- start start 2024-09-10T23:36:25.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a8a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8388004d50 0x7f83a4144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a8a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83a41e2800 0x7f83a4141270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83a40cea20 0x7f83a40ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a8a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83a40707c0 con 0x7f83a40cea20 2024-09-10T23:36:25.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a8a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f83a4070640 con 0x7f8388004d50 2024-09-10T23:36:25.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a8a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f83a4068990 con 0x7f83a41e2800 2024-09-10T23:36:25.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a3200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83a40cea20 0x7f83a40ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a3200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83a40cea20 0x7f83a40ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51066/0 (socket says 172.21.15.94:51066) 2024-09-10T23:36:25.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a3200640 1 -- 172.21.15.94:0/3988220133 learned_addr learned my addr 172.21.15.94:0/3988220133 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:25.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a1e00640 1 --2- 172.21.15.94:0/3988220133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83a41e2800 0x7f83a4141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a2800640 1 --2- 172.21.15.94:0/3988220133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8388004d50 0x7f83a4144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a3200640 1 -- 172.21.15.94:0/3988220133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83a41e2800 msgr2=0x7f83a4141270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:25.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a3200640 1 --2- 172.21.15.94:0/3988220133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83a41e2800 0x7f83a4141270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a3200640 1 -- 172.21.15.94:0/3988220133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8388004d50 msgr2=0x7f83a4144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:25.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a3200640 1 --2- 172.21.15.94:0/3988220133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8388004d50 0x7f83a4144690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a3200640 1 -- 172.21.15.94:0/3988220133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a41417b0 con 0x7f83a40cea20 2024-09-10T23:36:25.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a1e00640 1 --2- 172.21.15.94:0/3988220133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83a41e2800 0x7f83a4141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:25.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a3200640 1 --2- 172.21.15.94:0/3988220133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83a40cea20 0x7f83a40ceea0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f839c009ee0 tx=0x7f839c004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:25.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f839aa00640 1 -- 172.21.15.94:0/3988220133 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c0ad050 con 0x7f83a40cea20 2024-09-10T23:36:25.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f839aa00640 1 -- 172.21.15.94:0/3988220133 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f839c0a7a90 con 0x7f83a40cea20 2024-09-10T23:36:25.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f839aa00640 1 -- 172.21.15.94:0/3988220133 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c0a7e40 con 0x7f83a40cea20 2024-09-10T23:36:25.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a8a00640 1 -- 172.21.15.94:0/3988220133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a41e2f70 con 0x7f83a40cea20 2024-09-10T23:36:25.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a8a00640 1 -- 172.21.15.94:0/3988220133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a41e3530 con 0x7f83a40cea20 2024-09-10T23:36:25.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.098+0000 7f83a8a00640 1 -- 172.21.15.94:0/3988220133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a40cf3e0 con 0x7f83a40cea20 2024-09-10T23:36:25.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.102+0000 7f839aa00640 1 -- 172.21.15.94:0/3988220133 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f839c003980 con 0x7f83a40cea20 2024-09-10T23:36:25.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.102+0000 7f839aa00640 1 --2- 172.21.15.94:0/3988220133 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8384072970 0x7f8384074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.102+0000 7f839aa00640 1 -- 172.21.15.94:0/3988220133 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f839c12e9a0 con 0x7f83a40cea20 2024-09-10T23:36:25.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.102+0000 7f83a2800640 1 --2- 172.21.15.94:0/3988220133 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8384072970 0x7f8384074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.102+0000 7f83a2800640 1 --2- 172.21.15.94:0/3988220133 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8384072970 0x7f8384074e30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8394000fe0 tx=0x7f8394001e20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:25.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.106+0000 7f839aa00640 1 -- 172.21.15.94:0/3988220133 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f839c017300 con 0x7f83a40cea20 2024-09-10T23:36:25.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.302+0000 7f83a8a00640 1 -- 172.21.15.94:0/3988220133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83a4068ea0 con 0x7f83a40cea20 2024-09-10T23:36:25.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.306+0000 7f839aa00640 1 -- 172.21.15.94:0/3988220133 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5238 (secure 0 0 0) 0x7f839c0f9ab0 con 0x7f83a40cea20 2024-09-10T23:36:25.306 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:25.306 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":43,"btime":"2024-09-10T23:36:23:025336+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3320339747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3320339747},{"type":"v1","addr":"172.21.15.94:6834","nonce":3320339747}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3123300514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3123300514},{"type":"v1","addr":"172.21.15.94:6835","nonce":3123300514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/339523200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":339523200},{"type":"v1","addr":"172.21.15.181:6837","nonce":339523200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:36:19.404249+0000","modified":"2024-09-10T23:36:22.404999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195389":{"gid":195389,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/2999113739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2999113739},{"type":"v1","addr":"172.21.15.181:6836","nonce":2999113739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195389,"qdb_cluster":[195389]},"id":5}]} 2024-09-10T23:36:25.307 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 43 2024-09-10T23:36:25.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.306+0000 7f83a8a00640 1 -- 172.21.15.94:0/3988220133 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8384072970 msgr2=0x7f8384074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:25.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.306+0000 7f83a8a00640 1 --2- 172.21.15.94:0/3988220133 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8384072970 0x7f8384074e30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8394000fe0 tx=0x7f8394001e20 comp rx=0 tx=0).stop 2024-09-10T23:36:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.306+0000 7f83a8a00640 1 -- 172.21.15.94:0/3988220133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83a40cea20 msgr2=0x7f83a40ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.306+0000 7f83a8a00640 1 --2- 172.21.15.94:0/3988220133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83a40cea20 0x7f83a40ceea0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f839c009ee0 tx=0x7f839c004f50 comp rx=0 tx=0).stop 2024-09-10T23:36:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.306+0000 7f83a8a00640 1 -- 172.21.15.94:0/3988220133 shutdown_connections 2024-09-10T23:36:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.306+0000 7f83a8a00640 1 --2- 172.21.15.94:0/3988220133 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8384072970 0x7f8384074e30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.306+0000 7f83a8a00640 1 --2- 172.21.15.94:0/3988220133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83a40cea20 0x7f83a40ceea0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.306+0000 7f83a8a00640 1 --2- 172.21.15.94:0/3988220133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83a41e2800 0x7f83a4141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.306+0000 7f83a8a00640 1 --2- 172.21.15.94:0/3988220133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8388004d50 0x7f83a4144690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.306+0000 7f83a8a00640 1 -- 172.21.15.94:0/3988220133 >> 172.21.15.94:0/3988220133 conn(0x7f83a40d0170 msgr2=0x7f83a415bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.306+0000 7f83a8a00640 1 -- 172.21.15.94:0/3988220133 shutdown_connections 2024-09-10T23:36:25.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.306+0000 7f83a8a00640 1 -- 172.21.15.94:0/3988220133 wait complete. 2024-09-10T23:36:25.730 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T23:36:19.404249+0000', 'modified': '2024-09-10T23:36:22.404999+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 195389}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_195389': {'gid': 195389, 'name': 'd', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6836/2999113739', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 2999113739}, {'type': 'v1', 'addr': '172.21.15.181:6836', 'nonce': 2999113739}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 195389, 'qdb_cluster': [195389]} 2024-09-10T23:36:25.730 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T23:36:25.730 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T23:36:25.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f55a5c00640 1 -- 172.21.15.94:0/1722739627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5584003c20 msgr2=0x7f5584004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:25.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f55a5c00640 1 --2- 172.21.15.94:0/1722739627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5584003c20 0x7f5584004020 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5590006f90 tx=0x7f559001cee0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f559f400640 1 -- 172.21.15.94:0/1722739627 reap_dead start 2024-09-10T23:36:25.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f55a5c00640 1 -- 172.21.15.94:0/1722739627 shutdown_connections 2024-09-10T23:36:25.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f55a5c00640 1 -- 172.21.15.94:0/1722739627 >> 172.21.15.94:0/1722739627 conn(0x7f55a00cfaf0 msgr2=0x7f55a00cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:25.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f55a5c00640 1 -- 172.21.15.94:0/1722739627 shutdown_connections 2024-09-10T23:36:25.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f55a5c00640 1 -- 172.21.15.94:0/1722739627 wait complete. 2024-09-10T23:36:25.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f55a5c00640 1 Processor -- start 2024-09-10T23:36:25.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f55a5c00640 1 -- start start 2024-09-10T23:36:25.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f55a5c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5584003c20 0x7f55a01fb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f55a5c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55a01516b0 0x7f55a00af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f55a5c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55a0151b30 0x7f55a00afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f55a5c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55a0166a30 con 0x7f5584003c20 2024-09-10T23:36:25.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f55a5c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01668b0 con 0x7f55a0151b30 2024-09-10T23:36:25.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f55a5c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f55a0166bb0 con 0x7f55a01516b0 2024-09-10T23:36:25.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f559ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5584003c20 0x7f55a01fb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f559ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5584003c20 0x7f55a01fb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51092/0 (socket says 172.21.15.94:51092) 2024-09-10T23:36:25.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f559ea00640 1 -- 172.21.15.94:0/3400339632 learned_addr learned my addr 172.21.15.94:0/3400339632 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:25.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f559f400640 1 --2- 172.21.15.94:0/3400339632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55a0151b30 0x7f55a00afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f5596000640 1 --2- 172.21.15.94:0/3400339632 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55a01516b0 0x7f55a00af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f559ea00640 1 -- 172.21.15.94:0/3400339632 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55a01516b0 msgr2=0x7f55a00af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:25.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f559ea00640 1 --2- 172.21.15.94:0/3400339632 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55a01516b0 0x7f55a00af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f559ea00640 1 -- 172.21.15.94:0/3400339632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55a0151b30 msgr2=0x7f55a00afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:25.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f559ea00640 1 --2- 172.21.15.94:0/3400339632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55a0151b30 0x7f55a00afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.874+0000 7f559ea00640 1 -- 172.21.15.94:0/3400339632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a01fbe50 con 0x7f5584003c20 2024-09-10T23:36:25.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.878+0000 7f559ea00640 1 --2- 172.21.15.94:0/3400339632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5584003c20 0x7f55a01fb910 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5590098890 tx=0x7f55900011b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:25.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.878+0000 7f5597400640 1 -- 172.21.15.94:0/3400339632 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900a5b70 con 0x7f5584003c20 2024-09-10T23:36:25.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.878+0000 7f5597400640 1 -- 172.21.15.94:0/3400339632 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55900a5d10 con 0x7f5584003c20 2024-09-10T23:36:25.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.878+0000 7f55a5c00640 1 -- 172.21.15.94:0/3400339632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a0072ca0 con 0x7f5584003c20 2024-09-10T23:36:25.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.878+0000 7f5597400640 1 -- 172.21.15.94:0/3400339632 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900062b0 con 0x7f5584003c20 2024-09-10T23:36:25.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.878+0000 7f55a5c00640 1 -- 172.21.15.94:0/3400339632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a0073260 con 0x7f5584003c20 2024-09-10T23:36:25.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.878+0000 7f5597400640 1 -- 172.21.15.94:0/3400339632 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f5590003620 con 0x7f5584003c20 2024-09-10T23:36:25.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.878+0000 7f5597400640 1 --2- 172.21.15.94:0/3400339632 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f557c072940 0x7f557c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.878+0000 7f5597400640 1 -- 172.21.15.94:0/3400339632 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f559012d4d0 con 0x7f5584003c20 2024-09-10T23:36:25.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.878+0000 7f55a9016000 1 Processor -- start 2024-09-10T23:36:25.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.878+0000 7f55a9016000 1 -- start start 2024-09-10T23:36:25.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f5596000640 1 --2- 172.21.15.94:0/3400339632 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f557c072940 0x7f557c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f5596000640 1 --2- 172.21.15.94:0/3400339632 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f557c072940 0x7f557c074e00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f55a00b0550 tx=0x7f5588006b20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:25.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca65909d0 0x560ca6590db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca65912f0 0x560ca65448d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca6544e10 0x560ca65451f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560ca64653f0 con 0x560ca65909d0 2024-09-10T23:36:25.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560ca63a2b20 con 0x560ca6544e10 2024-09-10T23:36:25.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x560ca6391f50 con 0x560ca65912f0 2024-09-10T23:36:25.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f559ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca65909d0 0x560ca6590db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f559ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca65909d0 0x560ca6590db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51094/0 (socket says 172.21.15.94:51094) 2024-09-10T23:36:25.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f559ea00640 1 -- 172.21.15.94:0/1512022824 learned_addr learned my addr 172.21.15.94:0/1512022824 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:25.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f559f400640 1 --2- 172.21.15.94:0/1512022824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca6544e10 0x560ca65451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f5596000640 1 --2- 172.21.15.94:0/1512022824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca65912f0 0x560ca65448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f559ea00640 1 -- 172.21.15.94:0/1512022824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca65912f0 msgr2=0x560ca65448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:36:25.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f559ea00640 1 --2- 172.21.15.94:0/1512022824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca65912f0 0x560ca65448d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f559ea00640 1 -- 172.21.15.94:0/1512022824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca6544e10 msgr2=0x560ca65451f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:36:25.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f559ea00640 1 --2- 172.21.15.94:0/1512022824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca6544e10 0x560ca65451f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f559ea00640 1 -- 172.21.15.94:0/1512022824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ca6359590 con 0x560ca65909d0 2024-09-10T23:36:25.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f559ea00640 1 --2- 172.21.15.94:0/1512022824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca65909d0 0x560ca6590db0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f55900f7be0 tx=0x7f55900f7650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a869375c872cf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:25.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f558f400640 1 -- 172.21.15.94:0/1512022824 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900c0070 con 0x560ca65909d0 2024-09-10T23:36:25.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f558f400640 1 -- 172.21.15.94:0/1512022824 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55900f85e0 con 0x560ca65909d0 2024-09-10T23:36:25.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f558f400640 1 -- 172.21.15.94:0/1512022824 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900f7ee0 con 0x560ca65909d0 2024-09-10T23:36:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 -- 172.21.15.94:0/1512022824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca65909d0 msgr2=0x560ca6590db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 --2- 172.21.15.94:0/1512022824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca65909d0 0x560ca6590db0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f55900f7be0 tx=0x7f55900f7650 comp rx=0 tx=0).stop 2024-09-10T23:36:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 -- 172.21.15.94:0/1512022824 shutdown_connections 2024-09-10T23:36:25.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 --2- 172.21.15.94:0/1512022824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca6544e10 0x560ca65451f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 --2- 172.21.15.94:0/1512022824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca65912f0 0x560ca65448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 --2- 172.21.15.94:0/1512022824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca65909d0 0x560ca6590db0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 -- 172.21.15.94:0/1512022824 >> 172.21.15.94:0/1512022824 conn(0x560ca642e130 msgr2=0x560ca6573e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 -- 172.21.15.94:0/1512022824 shutdown_connections 2024-09-10T23:36:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 -- 172.21.15.94:0/1512022824 wait complete. 2024-09-10T23:36:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 -- start start 2024-09-10T23:36:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.882+0000 7f55a9016000 1 Processor -- start 2024-09-10T23:36:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f55a9016000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca6544e10 0x560ca65716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f55a9016000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca65909d0 0x560ca6571e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f55a9016000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca65912f0 0x560ca65723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f55a9016000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560ca632c5d0 con 0x560ca6544e10 2024-09-10T23:36:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f55a9016000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560ca6395d60 con 0x560ca65912f0 2024-09-10T23:36:25.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f55a9016000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x560ca6338500 con 0x560ca65909d0 2024-09-10T23:36:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f55a9016000 10 client.0 _ll_register_callbacks cb 0x560ca633c810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:36:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f559ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca6544e10 0x560ca65716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f559ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca6544e10 0x560ca65716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51104/0 (socket says 172.21.15.94:51104) 2024-09-10T23:36:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f559ea00640 1 -- 172.21.15.94:0/533273552 learned_addr learned my addr 172.21.15.94:0/533273552 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f5596000640 1 --2- 172.21.15.94:0/533273552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca65909d0 0x560ca6571e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f559f400640 1 --2- 172.21.15.94:0/533273552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca65912f0 0x560ca65723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f559ea00640 1 -- 172.21.15.94:0/533273552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca65909d0 msgr2=0x560ca6571e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:36:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f559ea00640 1 --2- 172.21.15.94:0/533273552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca65909d0 0x560ca6571e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f559ea00640 1 -- 172.21.15.94:0/533273552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca65912f0 msgr2=0x560ca65723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:36:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f559ea00640 1 --2- 172.21.15.94:0/533273552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca65912f0 0x560ca65723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f559ea00640 1 -- 172.21.15.94:0/533273552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ca6330170 con 0x560ca6544e10 2024-09-10T23:36:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f559ea00640 1 --2- 172.21.15.94:0/533273552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca6544e10 0x560ca65716a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f55900aa040 tx=0x7f55900a5f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d705c50d9b844b73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f5583e00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:36:25.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f5583e00640 1 -- 172.21.15.94:0/533273552 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900c0030 con 0x560ca6544e10 2024-09-10T23:36:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f5583e00640 1 -- 172.21.15.94:0/533273552 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f559012d2c0 con 0x560ca6544e10 2024-09-10T23:36:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f55a9016000 1 -- 172.21.15.94:0/533273552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560ca632a0f0 con 0x560ca6544e10 2024-09-10T23:36:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f5583e00640 1 -- 172.21.15.94:0/533273552 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f55900fc9c0 con 0x560ca6544e10 2024-09-10T23:36:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f55a9016000 10 client.6366 fetch_fsmap learned FSMap version 43 2024-09-10T23:36:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f55a9016000 1 -- 172.21.15.94:0/533273552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x560ca6416ef0 con 0x560ca6544e10 2024-09-10T23:36:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f5583e00640 1 -- 172.21.15.94:0/533273552 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x560ca632a0f0 con 0x560ca6544e10 2024-09-10T23:36:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f5583e00640 1 -- 172.21.15.94:0/533273552 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 43) ==== 3502+0+0 (secure 0 0 0) 0x7f55900fdb50 con 0x560ca6544e10 2024-09-10T23:36:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f55a9016000 10 client.6366 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T23:36:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f55a9016000 10 client.6366 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T23:36:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f55a9016000 20 client.6366 populate_metadata read hostname 'smithi094' 2024-09-10T23:36:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f55a9016000 1 --2- 172.21.15.94:0/533273552 >> [v2:172.21.15.181:6834/2999113739,v1:172.21.15.181:6836/2999113739] conn(0x560ca65730f0 0x560ca65a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f55a9016000 4 client.6366 mds_command: new command op to 195389 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:36:25.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f55a9016000 1 -- 172.21.15.94:0/533273552 --> [v2:172.21.15.181:6834/2999113739,v1:172.21.15.181:6836/2999113739] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ca63a4a70 con 0x560ca65730f0 2024-09-10T23:36:25.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f5596000640 1 --2- 172.21.15.94:0/533273552 >> [v2:172.21.15.181:6834/2999113739,v1:172.21.15.181:6836/2999113739] conn(0x560ca65730f0 0x560ca65a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f5596000640 1 --2- 172.21.15.94:0/533273552 >> [v2:172.21.15.181:6834/2999113739,v1:172.21.15.181:6836/2999113739] conn(0x560ca65730f0 0x560ca65a2c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5a982ba40e90789 server_cookie=81f462da9f3cb381 in_seq=0 out_seq=0 2024-09-10T23:36:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.886+0000 7f5583e00640 10 client.6366 ms_handle_connect on v2:172.21.15.181:6834/2999113739 2024-09-10T23:36:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.890+0000 7f5583e00640 1 -- 172.21.15.94:0/533273552 <== mds.0 v2:172.21.15.181:6834/2999113739 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560ca63a4a70 con 0x560ca65730f0 2024-09-10T23:36:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.890+0000 7f5583e00640 10 client.6366 handle_command_reply: tid=0 2024-09-10T23:36:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.890+0000 7f55a9016000 1 client.6366 shutdown 2024-09-10T23:36:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.890+0000 7f55a9016000 1 -- 172.21.15.94:0/533273552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca6544e10 msgr2=0x560ca65716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.890+0000 7f55a9016000 1 --2- 172.21.15.94:0/533273552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca6544e10 0x560ca65716a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f55900aa040 tx=0x7f55900a5f30 comp rx=0 tx=0).stop 2024-09-10T23:36:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.890+0000 7f55a9016000 1 -- 172.21.15.94:0/533273552 shutdown_connections 2024-09-10T23:36:25.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.890+0000 7f55a9016000 1 --2- 172.21.15.94:0/533273552 >> [v2:172.21.15.181:6834/2999113739,v1:172.21.15.181:6836/2999113739] conn(0x560ca65730f0 0x560ca65a2c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.890+0000 7f55a9016000 1 --2- 172.21.15.94:0/533273552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca65912f0 0x560ca65723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.890+0000 7f55a9016000 1 --2- 172.21.15.94:0/533273552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca65909d0 0x560ca6571e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.890+0000 7f55a9016000 1 --2- 172.21.15.94:0/533273552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca6544e10 0x560ca65716a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.890+0000 7f5583e00640 0 client.6366 ms_handle_reset on v2:172.21.15.181:6834/2999113739 2024-09-10T23:36:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.890+0000 7f55a9016000 1 -- 172.21.15.94:0/533273552 >> 172.21.15.94:0/533273552 conn(0x560ca642e130 msgr2=0x560ca6543fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.890+0000 7f55a9016000 1 -- 172.21.15.94:0/533273552 shutdown_connections 2024-09-10T23:36:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.890+0000 7f55a9016000 1 -- 172.21.15.94:0/533273552 wait complete. 2024-09-10T23:36:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.890+0000 7f55a9016000 20 client.6366 trim_cache size 0 max 16384 2024-09-10T23:36:25.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.906+0000 7f55a9016000 1 Processor -- start 2024-09-10T23:36:25.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.906+0000 7f55a9016000 1 -- start start 2024-09-10T23:36:25.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca6544e10 0x560ca658fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca65909d0 0x560ca6590310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca65912f0 0x560ca65a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55a02046e0 con 0x560ca65909d0 2024-09-10T23:36:25.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f55a0203710 con 0x560ca65912f0 2024-09-10T23:36:25.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f55a015a740 con 0x560ca6544e10 2024-09-10T23:36:25.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f5596000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca65909d0 0x560ca6590310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f5596000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca65909d0 0x560ca6590310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51110/0 (socket says 172.21.15.94:51110) 2024-09-10T23:36:25.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f5596000640 1 -- 172.21.15.94:0/593998084 learned_addr learned my addr 172.21.15.94:0/593998084 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:25.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f559ea00640 1 --2- 172.21.15.94:0/593998084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca6544e10 0x560ca658fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f559f400640 1 --2- 172.21.15.94:0/593998084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca65912f0 0x560ca65a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f5596000640 1 -- 172.21.15.94:0/593998084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca6544e10 msgr2=0x560ca658fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:36:25.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f5596000640 1 --2- 172.21.15.94:0/593998084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca6544e10 0x560ca658fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f5596000640 1 -- 172.21.15.94:0/593998084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca65912f0 msgr2=0x560ca65a2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:36:25.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f5596000640 1 --2- 172.21.15.94:0/593998084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca65912f0 0x560ca65a2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f5596000640 1 -- 172.21.15.94:0/593998084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ca63edd90 con 0x560ca65909d0 2024-09-10T23:36:25.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f5596000640 1 --2- 172.21.15.94:0/593998084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca65909d0 0x560ca6590310 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5588006150 tx=0x7f5588014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04df2c519c6621a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:25.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f558ea00640 1 -- 172.21.15.94:0/593998084 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558802d070 con 0x560ca65909d0 2024-09-10T23:36:25.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f558ea00640 1 -- 172.21.15.94:0/593998084 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5588016070 con 0x560ca65909d0 2024-09-10T23:36:25.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f558ea00640 1 -- 172.21.15.94:0/593998084 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558801f430 con 0x560ca65909d0 2024-09-10T23:36:25.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 -- 172.21.15.94:0/593998084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca65909d0 msgr2=0x560ca6590310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:25.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 --2- 172.21.15.94:0/593998084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca65909d0 0x560ca6590310 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5588006150 tx=0x7f5588014000 comp rx=0 tx=0).stop 2024-09-10T23:36:25.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 -- 172.21.15.94:0/593998084 shutdown_connections 2024-09-10T23:36:25.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 --2- 172.21.15.94:0/593998084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca65912f0 0x560ca65a2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 --2- 172.21.15.94:0/593998084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca65909d0 0x560ca6590310 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 --2- 172.21.15.94:0/593998084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca6544e10 0x560ca658fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 -- 172.21.15.94:0/593998084 >> 172.21.15.94:0/593998084 conn(0x560ca642e130 msgr2=0x560ca659dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:25.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 -- 172.21.15.94:0/593998084 shutdown_connections 2024-09-10T23:36:25.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 -- 172.21.15.94:0/593998084 wait complete. 2024-09-10T23:36:25.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 -- start start 2024-09-10T23:36:25.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 Processor -- start 2024-09-10T23:36:25.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca6544e10 0x560ca6606220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca65909d0 0x560ca6606760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca65912f0 0x560ca6606ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560ca6338500 con 0x560ca6544e10 2024-09-10T23:36:25.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f55a015ad90 con 0x560ca65912f0 2024-09-10T23:36:25.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x560ca632c5d0 con 0x560ca65909d0 2024-09-10T23:36:25.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f55a9016000 10 client.0 _ll_register_callbacks cb 0x7f55900f7d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:36:25.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f559ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca6544e10 0x560ca6606220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f559ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca6544e10 0x560ca6606220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51118/0 (socket says 172.21.15.94:51118) 2024-09-10T23:36:25.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.910+0000 7f559ea00640 1 -- 172.21.15.94:0/1679461737 learned_addr learned my addr 172.21.15.94:0/1679461737 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:25.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f559f400640 1 --2- 172.21.15.94:0/1679461737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca65912f0 0x560ca6606ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f5596000640 1 --2- 172.21.15.94:0/1679461737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca65909d0 0x560ca6606760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f559ea00640 1 -- 172.21.15.94:0/1679461737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca65909d0 msgr2=0x560ca6606760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:36:25.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f559ea00640 1 --2- 172.21.15.94:0/1679461737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca65909d0 0x560ca6606760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f559ea00640 1 -- 172.21.15.94:0/1679461737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca65912f0 msgr2=0x560ca6606ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:36:25.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f559ea00640 1 --2- 172.21.15.94:0/1679461737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca65912f0 0x560ca6606ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f559ea00640 1 -- 172.21.15.94:0/1679461737 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ca6358a90 con 0x560ca6544e10 2024-09-10T23:36:25.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f5596000640 1 --2- 172.21.15.94:0/1679461737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca65909d0 0x560ca6606760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f559ea00640 1 --2- 172.21.15.94:0/1679461737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca6544e10 0x560ca6606220 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f559001d780 tx=0x7f55900a5ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c071ae41cfa095 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f5583e00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:36:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f5583e00640 1 -- 172.21.15.94:0/1679461737 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900c0070 con 0x560ca6544e10 2024-09-10T23:36:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f5583e00640 1 -- 172.21.15.94:0/1679461737 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f5590125c80 con 0x560ca6544e10 2024-09-10T23:36:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 1 -- 172.21.15.94:0/1679461737 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5588016070 con 0x560ca6544e10 2024-09-10T23:36:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f5583e00640 1 -- 172.21.15.94:0/1679461737 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f55900f4530 con 0x560ca6544e10 2024-09-10T23:36:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 10 client.6372 fetch_fsmap learned FSMap version 43 2024-09-10T23:36:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 1 -- 172.21.15.94:0/1679461737 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x560ca6337050 con 0x560ca6544e10 2024-09-10T23:36:25.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f5583e00640 1 -- 172.21.15.94:0/1679461737 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5588016070 con 0x560ca6544e10 2024-09-10T23:36:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f5583e00640 1 -- 172.21.15.94:0/1679461737 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 43) ==== 3502+0+0 (secure 0 0 0) 0x7f55900f3830 con 0x560ca6544e10 2024-09-10T23:36:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 10 client.6372 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T23:36:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 10 client.6372 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T23:36:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 20 client.6372 populate_metadata read hostname 'smithi094' 2024-09-10T23:36:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 1 --2- 172.21.15.94:0/1679461737 >> [v2:172.21.15.181:6834/2999113739,v1:172.21.15.181:6836/2999113739] conn(0x560ca65a5960 0x560ca65a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 4 client.6372 mds_command: new command op to 195389 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T23:36:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 1 -- 172.21.15.94:0/1679461737 --> [v2:172.21.15.181:6834/2999113739,v1:172.21.15.181:6836/2999113739] -- command(tid 0: {"prefix": "status"}) -- 0x560ca63a0520 con 0x560ca65a5960 2024-09-10T23:36:25.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f5596000640 1 --2- 172.21.15.94:0/1679461737 >> [v2:172.21.15.181:6834/2999113739,v1:172.21.15.181:6836/2999113739] conn(0x560ca65a5960 0x560ca65a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f5583e00640 10 client.6372 ms_handle_connect on v2:172.21.15.181:6834/2999113739 2024-09-10T23:36:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f5596000640 1 --2- 172.21.15.94:0/1679461737 >> [v2:172.21.15.181:6834/2999113739,v1:172.21.15.181:6836/2999113739] conn(0x560ca65a5960 0x560ca65a5d40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcfde406cd16f232 server_cookie=2978bb3644b36d72 in_seq=0 out_seq=0 2024-09-10T23:36:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f5583e00640 1 -- 172.21.15.94:0/1679461737 <== mds.0 v2:172.21.15.181:6834/2999113739 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560ca63a0520 con 0x560ca65a5960 2024-09-10T23:36:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f5583e00640 10 client.6372 handle_command_reply: tid=0 2024-09-10T23:36:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 1 client.6372 shutdown 2024-09-10T23:36:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 1 -- 172.21.15.94:0/1679461737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca6544e10 msgr2=0x560ca6606220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 1 --2- 172.21.15.94:0/1679461737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca6544e10 0x560ca6606220 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f559001d780 tx=0x7f55900a5ec0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 1 -- 172.21.15.94:0/1679461737 shutdown_connections 2024-09-10T23:36:25.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 1 --2- 172.21.15.94:0/1679461737 >> [v2:172.21.15.181:6834/2999113739,v1:172.21.15.181:6836/2999113739] conn(0x560ca65a5960 0x560ca65a5d40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 1 --2- 172.21.15.94:0/1679461737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560ca65912f0 0x560ca6606ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 1 --2- 172.21.15.94:0/1679461737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560ca65909d0 0x560ca6606760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 1 --2- 172.21.15.94:0/1679461737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560ca6544e10 0x560ca6606220 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 1 -- 172.21.15.94:0/1679461737 >> 172.21.15.94:0/1679461737 conn(0x560ca642e130 msgr2=0x560ca658fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:25.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f5583e00640 0 client.6372 ms_handle_reset on v2:172.21.15.181:6834/2999113739 2024-09-10T23:36:25.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 1 -- 172.21.15.94:0/1679461737 shutdown_connections 2024-09-10T23:36:25.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 1 -- 172.21.15.94:0/1679461737 wait complete. 2024-09-10T23:36:25.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a9016000 20 client.6372 trim_cache size 0 max 16384 2024-09-10T23:36:25.927 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:36:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "1798cdb9-9937-4918-a569-86b41a92817e", 2024-09-10T23:36:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-09-10T23:36:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 195389, 2024-09-10T23:36:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-10T23:36:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-10T23:36:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-10T23:36:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 5.8961611339999997, 2024-09-10T23:36:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 43, 2024-09-10T23:36:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 69, 2024-09-10T23:36:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 69, 2024-09-10T23:36:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 10.432252231 2024-09-10T23:36:25.928 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:36:25.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a5c00640 1 -- 172.21.15.94:0/3400339632 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f557c072940 msgr2=0x7f557c074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:25.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a5c00640 1 --2- 172.21.15.94:0/3400339632 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f557c072940 0x7f557c074e00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f55a00b0550 tx=0x7f5588006b20 comp rx=0 tx=0).stop 2024-09-10T23:36:25.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a5c00640 1 -- 172.21.15.94:0/3400339632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5584003c20 msgr2=0x7f55a01fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:25.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a5c00640 1 --2- 172.21.15.94:0/3400339632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5584003c20 0x7f55a01fb910 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5590098890 tx=0x7f55900011b0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a5c00640 1 -- 172.21.15.94:0/3400339632 shutdown_connections 2024-09-10T23:36:25.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a5c00640 1 --2- 172.21.15.94:0/3400339632 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f557c072940 0x7f557c074e00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a5c00640 1 --2- 172.21.15.94:0/3400339632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55a0151b30 0x7f55a00afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a5c00640 1 --2- 172.21.15.94:0/3400339632 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55a01516b0 0x7f55a00af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a5c00640 1 --2- 172.21.15.94:0/3400339632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5584003c20 0x7f55a01fb910 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:25.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.914+0000 7f55a5c00640 1 -- 172.21.15.94:0/3400339632 >> 172.21.15.94:0/3400339632 conn(0x7f55a00cfaf0 msgr2=0x7f55a00ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:25.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.918+0000 7f55a5c00640 1 -- 172.21.15.94:0/3400339632 shutdown_connections 2024-09-10T23:36:25.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:25.918+0000 7f55a5c00640 1 -- 172.21.15.94:0/3400339632 wait complete. 2024-09-10T23:36:25.936 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:36:26.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.082+0000 7fe9a4a00640 1 -- 172.21.15.94:0/1096487313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9840051d0 msgr2=0x7fe98400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:26.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.082+0000 7fe9a4a00640 1 --2- 172.21.15.94:0/1096487313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9840051d0 0x7fe98400b9d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe998009ee0 tx=0x7fe99801d630 comp rx=0 tx=0).stop 2024-09-10T23:36:26.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.082+0000 7fe9a4a00640 1 -- 172.21.15.94:0/1096487313 shutdown_connections 2024-09-10T23:36:26.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.082+0000 7fe9a4a00640 1 --2- 172.21.15.94:0/1096487313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9840051d0 0x7fe98400b9d0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:26.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.082+0000 7fe9a4a00640 1 -- 172.21.15.94:0/1096487313 >> 172.21.15.94:0/1096487313 conn(0x7fe9a006cdf0 msgr2=0x7fe9a006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:26.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.082+0000 7fe9a4a00640 1 -- 172.21.15.94:0/1096487313 shutdown_connections 2024-09-10T23:36:26.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.082+0000 7fe9a4a00640 1 -- 172.21.15.94:0/1096487313 wait complete. 2024-09-10T23:36:26.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.082+0000 7fe9a4a00640 1 Processor -- start 2024-09-10T23:36:26.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.082+0000 7fe9a4a00640 1 -- start start 2024-09-10T23:36:26.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.082+0000 7fe9a4a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9840051d0 0x7fe9a00ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:26.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.082+0000 7fe9a4a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9a00cb550 0x7fe9a00b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:26.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.082+0000 7fe9a4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9a01f4ec0 0x7fe9a00c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:26.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe9a4a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a00cfda0 con 0x7fe9a01f4ec0 2024-09-10T23:36:26.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe9a4a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a00cff20 con 0x7fe9a00cb550 2024-09-10T23:36:26.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe9a4a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a00d00a0 con 0x7fe9840051d0 2024-09-10T23:36:26.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe99e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9a01f4ec0 0x7fe9a00c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:26.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe99e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9a01f4ec0 0x7fe9a00c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51124/0 (socket says 172.21.15.94:51124) 2024-09-10T23:36:26.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe99e800640 1 -- 172.21.15.94:0/4103670745 learned_addr learned my addr 172.21.15.94:0/4103670745 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:26.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe997e00640 1 --2- 172.21.15.94:0/4103670745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9a00cb550 0x7fe9a00b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:26.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe99de00640 1 --2- 172.21.15.94:0/4103670745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9840051d0 0x7fe9a00ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe99e800640 1 -- 172.21.15.94:0/4103670745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9840051d0 msgr2=0x7fe9a00ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe99e800640 1 --2- 172.21.15.94:0/4103670745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9840051d0 0x7fe9a00ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe99e800640 1 -- 172.21.15.94:0/4103670745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9a00cb550 msgr2=0x7fe9a00b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe99e800640 1 --2- 172.21.15.94:0/4103670745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9a00cb550 0x7fe9a00b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe99e800640 1 -- 172.21.15.94:0/4103670745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a00ca7b0 con 0x7fe9a01f4ec0 2024-09-10T23:36:26.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe99e800640 1 --2- 172.21.15.94:0/4103670745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9a01f4ec0 0x7fe9a00c8d50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe99801ded0 tx=0x7fe9980a7ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe996000640 1 -- 172.21.15.94:0/4103670745 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998009200 con 0x7fe9a01f4ec0 2024-09-10T23:36:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe996000640 1 -- 172.21.15.94:0/4103670745 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe998004b40 con 0x7fe9a01f4ec0 2024-09-10T23:36:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe9a4a00640 1 -- 172.21.15.94:0/4103670745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a00caa70 con 0x7fe9a01f4ec0 2024-09-10T23:36:26.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe996000640 1 -- 172.21.15.94:0/4103670745 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9980a7ed0 con 0x7fe9a01f4ec0 2024-09-10T23:36:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe9a4a00640 1 -- 172.21.15.94:0/4103670745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a00cea20 con 0x7fe9a01f4ec0 2024-09-10T23:36:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe9a4a00640 1 -- 172.21.15.94:0/4103670745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a015e7c0 con 0x7fe9a01f4ec0 2024-09-10T23:36:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.086+0000 7fe996000640 1 -- 172.21.15.94:0/4103670745 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7fe998005050 con 0x7fe9a01f4ec0 2024-09-10T23:36:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.090+0000 7fe996000640 1 --2- 172.21.15.94:0/4103670745 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe97c072970 0x7fe97c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.090+0000 7fe996000640 1 -- 172.21.15.94:0/4103670745 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fe99812dc60 con 0x7fe9a01f4ec0 2024-09-10T23:36:26.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.090+0000 7fe99de00640 1 --2- 172.21.15.94:0/4103670745 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe97c072970 0x7fe97c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.090+0000 7fe99de00640 1 --2- 172.21.15.94:0/4103670745 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe97c072970 0x7fe97c074e30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe990001870 tx=0x7fe990006040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.094+0000 7fe996000640 1 -- 172.21.15.94:0/4103670745 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe9980ba060 con 0x7fe9a01f4ec0 2024-09-10T23:36:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.294+0000 7fe9a4a00640 1 -- 172.21.15.94:0/4103670745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe9a00b2410 con 0x7fe9a01f4ec0 2024-09-10T23:36:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.294+0000 7fe996000640 1 -- 172.21.15.94:0/4103670745 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) ==== 53+0+73 (secure 0 0 0) 0x7fe998017340 con 0x7fe9a01f4ec0 2024-09-10T23:36:26.298 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:36:26.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.298+0000 7fe9a4a00640 1 -- 172.21.15.94:0/4103670745 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe97c072970 msgr2=0x7fe97c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:26.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.298+0000 7fe9a4a00640 1 --2- 172.21.15.94:0/4103670745 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe97c072970 0x7fe97c074e30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe990001870 tx=0x7fe990006040 comp rx=0 tx=0).stop 2024-09-10T23:36:26.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.298+0000 7fe9a4a00640 1 -- 172.21.15.94:0/4103670745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9a01f4ec0 msgr2=0x7fe9a00c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:26.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.298+0000 7fe9a4a00640 1 --2- 172.21.15.94:0/4103670745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9a01f4ec0 0x7fe9a00c8d50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe99801ded0 tx=0x7fe9980a7ad0 comp rx=0 tx=0).stop 2024-09-10T23:36:26.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.298+0000 7fe9a4a00640 1 -- 172.21.15.94:0/4103670745 shutdown_connections 2024-09-10T23:36:26.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.298+0000 7fe9a4a00640 1 --2- 172.21.15.94:0/4103670745 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fe97c072970 0x7fe97c074e30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:26.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.298+0000 7fe9a4a00640 1 --2- 172.21.15.94:0/4103670745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9a00cb550 0x7fe9a00b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:26.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.298+0000 7fe9a4a00640 1 --2- 172.21.15.94:0/4103670745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9a01f4ec0 0x7fe9a00c8d50 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:26.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.298+0000 7fe9a4a00640 1 --2- 172.21.15.94:0/4103670745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9840051d0 0x7fe9a00ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:26.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.298+0000 7fe9a4a00640 1 -- 172.21.15.94:0/4103670745 >> 172.21.15.94:0/4103670745 conn(0x7fe9a006cdf0 msgr2=0x7fe9a00cbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:26.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.298+0000 7fe9a4a00640 1 -- 172.21.15.94:0/4103670745 shutdown_connections 2024-09-10T23:36:26.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26.298+0000 7fe9a4a00640 1 -- 172.21.15.94:0/4103670745 wait complete. 2024-09-10T23:36:26.319 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:36:26.320 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:36:26.320 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T23:36:26.320 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T23:36:26.320 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T23:36:26.320 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:36:26.320 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:36:26.320 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:36:26.320 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:36:26.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:26.320 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T23:36:26.330 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:36:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:36:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:36:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T23:36:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:26.330 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:36:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T23:36:26.331 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:36:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T23:36:26.331 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:36:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:26.331 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:36:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:36:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42225sec preferred_lft 42225sec 2024-09-10T23:36:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-09-10T23:36:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:26.332 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:36:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:ed:7d:46:db brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:36:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:26.332 INFO:teuthology.orchestra.run.smithi094.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:36:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:36:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:36:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:26.332 INFO:teuthology.orchestra.run.smithi094.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:36:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:36:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:36:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:26.333 INFO:teuthology.orchestra.run.smithi094.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:36:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:36:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:36:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:26.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:26.333 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:36:26.395 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T23:36:26.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:26.397 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T23:36:26.452 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T23:36:26.452 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:36:26.452 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:36:26.452 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:36:26.452 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:36:26.453 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:36:26.453 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:36:26.453 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:36:26.453 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:36:26.453 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:36:26.453 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:36:26.453 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:36:26.453 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:36:26.453 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:36:26.453 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:36:26.453 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:36:26.454 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:36:26.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:36:26.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:36:26.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:26.590 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T23:36:26.599 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T23:36:26.599 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T23:36:26.600 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:36:26.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:26.600 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-09-10T23:36:26.665 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T23:36:26.665 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:36:26.665 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T23:36:26.713 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:36:26.713 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:36:26.713 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:36:26.765 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:36:26.830 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:36:26.886 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:36:26.887 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:36:26.887 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:36:26.887 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:36:26.887 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:36:26.887 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:36:26.887 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:36:26.887 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:36:26.887 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:36:26.887 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:36:26.887 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:36:26.887 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:36:26.887 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:36:26.888 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:36:26.888 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:36:26.888 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:36:26.888 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:36:26.888 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:36:26.888 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:36:26.888 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:36:26.888 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:36:26.888 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:36:26.888 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:36:26.888 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:36:26.889 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:36:26.889 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:36:26.889 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:36:26.889 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:36:26.889 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:36:26.889 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:36:26.889 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:36:26.889 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:36:26.889 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:36:26.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:26.890 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:36:26.941 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:36:26.941 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T23:36:26.989 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:36:27.009 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:36:27.058 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:36:27.059 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:36:27.059 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:36:27.059 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:36:27.059 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:36:27.059 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:36:27.059 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:36:27.059 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:36:27.059 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:36:27.059 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:36:27.059 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:36:27.060 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:36:27.060 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:36:27.060 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:36:27.060 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:36:27.060 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:36:27.060 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:36:27.060 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:36:27.060 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:36:27.060 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:36:27.060 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:36:27.060 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:36:27.060 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:36:27.060 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:36:27.061 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:36:27.061 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:36:27.061 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:36:27.061 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:36:27.061 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:36:27.061 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:36:27.061 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:36:27.061 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:36:27.061 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:36:27.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:27.062 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:36:27.083 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:2024-09-10T23:36:27.082+0000 7fe1f9509580 -1 init, newargv = 0x561bd2518b10 newargc=15 2024-09-10T23:36:27.083 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[41891]: starting ceph client 2024-09-10T23:36:27.091 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[41891]: starting fuse 2024-09-10T23:36:28.193 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:36:28.217 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:36:28.271 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:36:28.271 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:36:28.272 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:36:28.272 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:36:28.272 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:36:28.272 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:36:28.272 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:36:28.272 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:36:28.272 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:36:28.272 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:36:28.272 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:36:28.272 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:36:28.272 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:36:28.272 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:36:28.273 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:36:28.273 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:36:28.273 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:36:28.273 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:36:28.273 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:36:28.273 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:36:28.273 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:36:28.273 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:36:28.273 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:36:28.273 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:36:28.273 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:36:28.273 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:36:28.273 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:36:28.274 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:36:28.274 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:36:28.274 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:36:28.274 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:36:28.274 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:36:28.274 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:36:28.274 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-10T23:36:28.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:28.275 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:36:28.326 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T23:36:28.326 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:36:28.327 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:36:28.327 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-09-10T23:36:28.327 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-09-10T23:36:28.327 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-09-10T23:36:28.327 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-09-10T23:36:28.327 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:36:28.328 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-09-10T23:36:28.328 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T23:36:28.328 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-09-10T23:36:28.328 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T23:36:28.328 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:36:28.328 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-09-10T23:36:28.329 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-09-10T23:36:28.329 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-09-10T23:36:28.329 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-09-10T23:36:28.329 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:36:28.329 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-09-10T23:36:28.329 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:36:28.329 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:36:28.329 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:36:28.329 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-09-10T23:36:28.329 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-09-10T23:36:28.329 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-09-10T23:36:28.329 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:36:28.329 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:36:28.329 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.0")) 2024-09-10T23:36:28.330 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-09-10T23:36:28.457 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.0.41891.asok 2024-09-10T23:36:28.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:36:28.469 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41891.asok 2024-09-10T23:36:28.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:28.469 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41891.asok status 2024-09-10T23:36:28.641 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:36:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-09-10T23:36:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:36:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:36:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "0", 2024-09-10T23:36:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-09-10T23:36:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T23:36:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "41891", 2024-09-10T23:36:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-09-10T23:36:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:28.641 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-09-10T23:36:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-09-10T23:36:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 6387, 2024-09-10T23:36:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-09-10T23:36:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-09-10T23:36:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-09-10T23:36:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 6387 2024-09-10T23:36:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:36:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:36:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:36:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2816221820 2024-09-10T23:36:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:28.642 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:36:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:36:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:36:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2816221820 2024-09-10T23:36:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.6387 192.168.0.1:0/2816221820", 2024-09-10T23:36:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/2816221820", 2024-09-10T23:36:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-09-10T23:36:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 43, 2024-09-10T23:36:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 69, 2024-09-10T23:36:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:36:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-09-10T23:36:28.643 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-09-10T23:36:28.643 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:36:28.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:28.655 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-09-10T23:36:28.655 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T23:36:28.664 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-09-10T23:36:28.664 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T23:36:28.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:28.665 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:36:28.725 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:36:28.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f0293000640 1 -- 172.21.15.94:0/4066123709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0270004810 msgr2=0x7f0270004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:28.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f0293000640 1 --2- 172.21.15.94:0/4066123709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0270004810 0x7f0270004c90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f028c06d8b0 tx=0x7f027801d210 comp rx=0 tx=0).stop 2024-09-10T23:36:28.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f0291000640 1 -- 172.21.15.94:0/4066123709 reap_dead start 2024-09-10T23:36:28.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f0293000640 1 -- 172.21.15.94:0/4066123709 shutdown_connections 2024-09-10T23:36:28.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f0293000640 1 -- 172.21.15.94:0/4066123709 >> 172.21.15.94:0/4066123709 conn(0x7f028c0d1c60 msgr2=0x7f028c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:28.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f0293000640 1 -- 172.21.15.94:0/4066123709 shutdown_connections 2024-09-10T23:36:28.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f0293000640 1 -- 172.21.15.94:0/4066123709 wait complete. 2024-09-10T23:36:28.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f0293000640 1 Processor -- start 2024-09-10T23:36:28.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f0293000640 1 -- start start 2024-09-10T23:36:28.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f0293000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0270004810 0x7f028c1573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:28.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f0293000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c0c7f10 0x7f028c06e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:28.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f0293000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f028c0af480 0x7f028c0af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:28.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f0293000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f028c166ff0 con 0x7f0270004810 2024-09-10T23:36:28.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f0293000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f028c166e70 con 0x7f028c0c7f10 2024-09-10T23:36:28.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f0293000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f028c167170 con 0x7f028c0af480 2024-09-10T23:36:28.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f028be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0270004810 0x7f028c1573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:28.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f028be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0270004810 0x7f028c1573f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51142/0 (socket says 172.21.15.94:51142) 2024-09-10T23:36:28.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f028be00640 1 -- 172.21.15.94:0/2531810111 learned_addr learned my addr 172.21.15.94:0/2531810111 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:28.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f028b400640 1 --2- 172.21.15.94:0/2531810111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c0c7f10 0x7f028c06e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:28.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f0291000640 1 --2- 172.21.15.94:0/2531810111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f028c0af480 0x7f028c0af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:28.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f028be00640 1 -- 172.21.15.94:0/2531810111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f028c0af480 msgr2=0x7f028c0af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:28.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f028be00640 1 --2- 172.21.15.94:0/2531810111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f028c0af480 0x7f028c0af900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:28.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f028be00640 1 -- 172.21.15.94:0/2531810111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c0c7f10 msgr2=0x7f028c06e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:28.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f028be00640 1 --2- 172.21.15.94:0/2531810111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c0c7f10 0x7f028c06e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:28.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f028be00640 1 -- 172.21.15.94:0/2531810111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028c157930 con 0x7f0270004810 2024-09-10T23:36:28.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f028b400640 1 --2- 172.21.15.94:0/2531810111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c0c7f10 0x7f028c06e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:28.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.910+0000 7f028be00640 1 --2- 172.21.15.94:0/2531810111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0270004810 0x7f028c1573f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f027c011ab0 tx=0x7f027c011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:28.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.914+0000 7f0283e00640 1 -- 172.21.15.94:0/2531810111 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c01e070 con 0x7f0270004810 2024-09-10T23:36:28.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.914+0000 7f0283e00640 1 -- 172.21.15.94:0/2531810111 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f027c004460 con 0x7f0270004810 2024-09-10T23:36:28.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.914+0000 7f0293000640 1 -- 172.21.15.94:0/2531810111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028c0c6e70 con 0x7f0270004810 2024-09-10T23:36:28.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.914+0000 7f0283e00640 1 -- 172.21.15.94:0/2531810111 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c00fc60 con 0x7f0270004810 2024-09-10T23:36:28.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.914+0000 7f0293000640 1 -- 172.21.15.94:0/2531810111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f028c0c7430 con 0x7f0270004810 2024-09-10T23:36:28.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.914+0000 7f0293000640 1 -- 172.21.15.94:0/2531810111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f028c0d00a0 con 0x7f0270004810 2024-09-10T23:36:28.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.914+0000 7f0283e00640 1 -- 172.21.15.94:0/2531810111 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f027c003600 con 0x7f0270004810 2024-09-10T23:36:28.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.914+0000 7f0283e00640 1 --2- 172.21.15.94:0/2531810111 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f026c072970 0x7f026c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:28.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.914+0000 7f0283e00640 1 -- 172.21.15.94:0/2531810111 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f027c094550 con 0x7f0270004810 2024-09-10T23:36:28.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.918+0000 7f028b400640 1 --2- 172.21.15.94:0/2531810111 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f026c072970 0x7f026c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:28.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.918+0000 7f028b400640 1 --2- 172.21.15.94:0/2531810111 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f026c072970 0x7f026c074e30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f02780ae000 tx=0x7f0278019fb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:28.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:28.918+0000 7f0283e00640 1 -- 172.21.15.94:0/2531810111 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f027c05ef80 con 0x7f0270004810 2024-09-10T23:36:29.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:29.122+0000 7f0293000640 1 -- 172.21.15.94:0/2531810111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f028c0749c0 con 0x7f0270004810 2024-09-10T23:36:29.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:29.122+0000 7f0283e00640 1 -- 172.21.15.94:0/2531810111 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) ==== 53+0+73 (secure 0 0 0) 0x7f027c0639b0 con 0x7f0270004810 2024-09-10T23:36:29.122 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:36:29.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:29.122+0000 7f0293000640 1 -- 172.21.15.94:0/2531810111 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f026c072970 msgr2=0x7f026c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:29.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:29.122+0000 7f0293000640 1 --2- 172.21.15.94:0/2531810111 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f026c072970 0x7f026c074e30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f02780ae000 tx=0x7f0278019fb0 comp rx=0 tx=0).stop 2024-09-10T23:36:29.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:29.122+0000 7f0293000640 1 -- 172.21.15.94:0/2531810111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0270004810 msgr2=0x7f028c1573f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:29.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:29.122+0000 7f0293000640 1 --2- 172.21.15.94:0/2531810111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0270004810 0x7f028c1573f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f027c011ab0 tx=0x7f027c011f70 comp rx=0 tx=0).stop 2024-09-10T23:36:29.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:29.122+0000 7f0293000640 1 -- 172.21.15.94:0/2531810111 shutdown_connections 2024-09-10T23:36:29.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:29.122+0000 7f0293000640 1 --2- 172.21.15.94:0/2531810111 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f026c072970 0x7f026c074e30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:29.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:29.122+0000 7f0293000640 1 --2- 172.21.15.94:0/2531810111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f028c0c7f10 0x7f028c06e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:29.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:29.122+0000 7f0293000640 1 --2- 172.21.15.94:0/2531810111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f028c0af480 0x7f028c0af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:29.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:29.122+0000 7f0293000640 1 --2- 172.21.15.94:0/2531810111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0270004810 0x7f028c1573f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:29.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:29.122+0000 7f0293000640 1 -- 172.21.15.94:0/2531810111 >> 172.21.15.94:0/2531810111 conn(0x7f028c0d1c60 msgr2=0x7f028c070780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:29.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:29.122+0000 7f0293000640 1 -- 172.21.15.94:0/2531810111 shutdown_connections 2024-09-10T23:36:29.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:29.122+0000 7f0293000640 1 -- 172.21.15.94:0/2531810111 wait complete. 2024-09-10T23:36:29.144 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:36:29.144 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:36:29.144 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-09-10T23:36:29.144 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T23:36:29.144 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T23:36:29.144 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:36:29.144 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:36:29.145 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:36:29.145 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:36:29.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:29.145 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-09-10T23:36:29.155 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:36:29.155 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:36:29.155 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:36:29.155 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:29.155 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-09-10T23:36:29.155 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:29.155 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:36:29.155 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:29.155 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-09-10T23:36:29.155 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:36:29.155 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:29.155 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-09-10T23:36:29.156 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:36:29.156 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:29.156 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:36:29.156 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:29.156 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:36:29.156 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42226sec preferred_lft 42226sec 2024-09-10T23:36:29.156 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-09-10T23:36:29.156 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:29.156 INFO:teuthology.orchestra.run.smithi181.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:36:29.156 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 02:42:3b:2e:09:2d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:29.156 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:36:29.156 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:29.156 INFO:teuthology.orchestra.run.smithi181.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:36:29.157 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:29.157 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:36:29.157 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:29.157 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:36:29.157 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:29.157 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:36:29.157 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:36:29.157 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:36:29.157 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:29.157 INFO:teuthology.orchestra.run.smithi181.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:36:29.157 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:36:29.157 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:36:29.157 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:29.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:29.158 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:36:29.222 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-09-10T23:36:29.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:29.224 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-09-10T23:36:29.277 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-09-10T23:36:29.277 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:36:29.277 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:36:29.277 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:36:29.277 INFO:teuthology.orchestra.run.smithi181.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:36:29.277 INFO:teuthology.orchestra.run.smithi181.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:36:29.278 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:36:29.278 INFO:teuthology.orchestra.run.smithi181.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:36:29.278 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:36:29.278 INFO:teuthology.orchestra.run.smithi181.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:36:29.278 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:36:29.278 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:36:29.278 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:36:29.278 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:36:29.278 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:36:29.278 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:36:29.278 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:36:29.377 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:36:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:36:29.421 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:36:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:36:29.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:29.428 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-09-10T23:36:29.437 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T23:36:29.438 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T23:36:29.438 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:36:29.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:29.439 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-09-10T23:36:29.504 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T23:36:29.505 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:36:29.505 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T23:36:29.553 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:36:29.553 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:36:29.553 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:36:29.604 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:36:29.665 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:36:29.718 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:36:29.719 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:36:29.719 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:36:29.719 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:36:29.719 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:36:29.719 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:36:29.719 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:36:29.719 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:36:29.719 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:36:29.719 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:36:29.719 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:36:29.719 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:36:29.719 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:36:29.720 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:36:29.720 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:36:29.720 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:36:29.720 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:36:29.720 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:36:29.720 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:36:29.720 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:36:29.720 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:36:29.720 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:36:29.720 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:36:29.721 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:36:29.721 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:36:29.721 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:36:29.721 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:36:29.721 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:36:29.721 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:36:29.721 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:36:29.721 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:36:29.721 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:36:29.721 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:36:29.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:29.722 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:36:29.776 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:36:29.777 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T23:36:29.819 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:36:29.840 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:36:29.895 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:36:29.895 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:36:29.895 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:36:29.895 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:36:29.895 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:36:29.896 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:36:29.896 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:36:29.896 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:36:29.896 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:36:29.896 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:36:29.896 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:36:29.896 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:36:29.896 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:36:29.896 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:36:29.896 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:36:29.896 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:36:29.896 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:36:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:36:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:36:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:36:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:36:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:36:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:36:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:36:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:36:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:36:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:36:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:36:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:36:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:36:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:36:29.897 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:36:29.898 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:36:29.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:29.899 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:36:29.913 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:2024-09-10T23:36:29.910+0000 7fcd48a2a580 -1 init, newargv = 0x5564eb54efc0 newargc=15 2024-09-10T23:36:29.914 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[35085]: starting ceph client 2024-09-10T23:36:29.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[35085]: starting fuse 2024-09-10T23:36:29.923 INFO:teuthology.orchestra.run.smithi181.stdout:44 2024-09-10T23:36:29.923 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:36:29.923 DEBUG:teuthology.orchestra.run.smithi181:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:36:29.923 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2024-09-10T23:36:29.923 DEBUG:teuthology.orchestra.run.smithi181:> import re 2024-09-10T23:36:29.923 DEBUG:teuthology.orchestra.run.smithi181:> import os 2024-09-10T23:36:29.924 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2024-09-10T23:36:29.924 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:36:29.924 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2024-09-10T23:36:29.924 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T23:36:29.924 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2024-09-10T23:36:29.924 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T23:36:29.924 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:36:29.924 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2024-09-10T23:36:29.924 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2024-09-10T23:36:29.924 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2024-09-10T23:36:29.924 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2024-09-10T23:36:29.924 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:36:29.924 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2024-09-10T23:36:29.925 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:36:29.925 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:36:29.925 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:36:29.925 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2024-09-10T23:36:29.925 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2024-09-10T23:36:29.925 DEBUG:teuthology.orchestra.run.smithi181:> return f 2024-09-10T23:36:29.925 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:36:29.925 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:36:29.925 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.1")) 2024-09-10T23:36:29.925 DEBUG:teuthology.orchestra.run.smithi181:> ' 2024-09-10T23:36:30.045 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.1.35085.asok 2024-09-10T23:36:30.048 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:36:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:36:30.057 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35085.asok 2024-09-10T23:36:30.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:30.057 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35085.asok status 2024-09-10T23:36:30.225 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T23:36:30.226 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2024-09-10T23:36:30.226 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:36:30.226 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:36:30.226 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "1", 2024-09-10T23:36:30.226 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2024-09-10T23:36:30.226 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T23:36:30.226 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "35085", 2024-09-10T23:36:30.226 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2024-09-10T23:36:30.227 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:36:30.227 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2024-09-10T23:36:30.227 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2024-09-10T23:36:30.227 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 6061, 2024-09-10T23:36:30.227 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2024-09-10T23:36:30.227 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2024-09-10T23:36:30.227 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2024-09-10T23:36:30.227 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 6061 2024-09-10T23:36:30.227 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:36:30.227 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:36:30.227 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:36:30.227 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:36:30.227 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 3004280156 2024-09-10T23:36:30.227 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T23:36:30.227 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:36:30.227 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:36:30.228 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:36:30.228 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:36:30.228 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 3004280156 2024-09-10T23:36:30.228 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:36:30.228 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.6061 192.168.0.1:0/3004280156", 2024-09-10T23:36:30.228 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.1:0/3004280156", 2024-09-10T23:36:30.228 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2024-09-10T23:36:30.228 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 43, 2024-09-10T23:36:30.228 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 69, 2024-09-10T23:36:30.228 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:36:30.228 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2024-09-10T23:36:30.228 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2024-09-10T23:36:30.228 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T23:36:30.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:30.242 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2024-09-10T23:36:30.242 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T23:36:30.251 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2024-09-10T23:36:30.251 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T23:36:30.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:30.251 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:36:30.313 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T23:36:30.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b7600640 1 -- 172.21.15.94:0/3754628931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0b015cbc0 msgr2=0x7fd0b00cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:30.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b7600640 1 --2- 172.21.15.94:0/3754628931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0b015cbc0 0x7fd0b00cde20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd0980097f0 tx=0x7fd09801c660 comp rx=0 tx=0).stop 2024-09-10T23:36:30.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b7600640 1 -- 172.21.15.94:0/3754628931 shutdown_connections 2024-09-10T23:36:30.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b7600640 1 --2- 172.21.15.94:0/3754628931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0b015cbc0 0x7fd0b00cde20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:30.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b7600640 1 -- 172.21.15.94:0/3754628931 >> 172.21.15.94:0/3754628931 conn(0x7fd0b006cdf0 msgr2=0x7fd0b006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:30.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b7600640 1 -- 172.21.15.94:0/3754628931 shutdown_connections 2024-09-10T23:36:30.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b7600640 1 -- 172.21.15.94:0/3754628931 wait complete. 2024-09-10T23:36:30.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b7600640 1 Processor -- start 2024-09-10T23:36:30.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b7600640 1 -- start start 2024-09-10T23:36:30.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b7600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0b00cfd50 0x7fd0b014bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:30.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b7600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b0152c10 0x7fd0b006fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:30.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b7600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0b015cbc0 0x7fd0b014e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:30.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b7600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b016a4d0 con 0x7fd0b0152c10 2024-09-10T23:36:30.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b7600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b016a350 con 0x7fd0b015cbc0 2024-09-10T23:36:30.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b7600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd0b016a650 con 0x7fd0b00cfd50 2024-09-10T23:36:30.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0a7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b0152c10 0x7fd0b006fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:30.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0a7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b0152c10 0x7fd0b006fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51154/0 (socket says 172.21.15.94:51154) 2024-09-10T23:36:30.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0a7e00640 1 -- 172.21.15.94:0/2332608774 learned_addr learned my addr 172.21.15.94:0/2332608774 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:30.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b4c00640 1 --2- 172.21.15.94:0/2332608774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0b00cfd50 0x7fd0b014bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:30.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b5600640 1 --2- 172.21.15.94:0/2332608774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0b015cbc0 0x7fd0b014e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:30.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0a7e00640 1 -- 172.21.15.94:0/2332608774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0b00cfd50 msgr2=0x7fd0b014bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:30.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0a7e00640 1 --2- 172.21.15.94:0/2332608774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0b00cfd50 0x7fd0b014bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:30.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0a7e00640 1 -- 172.21.15.94:0/2332608774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0b015cbc0 msgr2=0x7fd0b014e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:30.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0a7e00640 1 --2- 172.21.15.94:0/2332608774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0b015cbc0 0x7fd0b014e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:30.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0a7e00640 1 -- 172.21.15.94:0/2332608774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b0156220 con 0x7fd0b0152c10 2024-09-10T23:36:30.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.470+0000 7fd0b5600640 1 --2- 172.21.15.94:0/2332608774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0b015cbc0 0x7fd0b014e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:30.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.474+0000 7fd0a7e00640 1 --2- 172.21.15.94:0/2332608774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b0152c10 0x7fd0b006fe80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd09801cb40 tx=0x7fd0980a6540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:30.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.474+0000 7fd0ad600640 1 -- 172.21.15.94:0/2332608774 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098004240 con 0x7fd0b0152c10 2024-09-10T23:36:30.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.474+0000 7fd0ad600640 1 -- 172.21.15.94:0/2332608774 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0980bf070 con 0x7fd0b0152c10 2024-09-10T23:36:30.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.474+0000 7fd0ad600640 1 -- 172.21.15.94:0/2332608774 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0980044f0 con 0x7fd0b0152c10 2024-09-10T23:36:30.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.474+0000 7fd0b7600640 1 -- 172.21.15.94:0/2332608774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b0156500 con 0x7fd0b0152c10 2024-09-10T23:36:30.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.474+0000 7fd0b7600640 1 -- 172.21.15.94:0/2332608774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b0156ac0 con 0x7fd0b0152c10 2024-09-10T23:36:30.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.474+0000 7fd0b7600640 1 -- 172.21.15.94:0/2332608774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b0156d00 con 0x7fd0b0152c10 2024-09-10T23:36:30.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.474+0000 7fd0ad600640 1 -- 172.21.15.94:0/2332608774 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7fd098003860 con 0x7fd0b0152c10 2024-09-10T23:36:30.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.474+0000 7fd0ad600640 1 --2- 172.21.15.94:0/2332608774 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd08c072940 0x7fd08c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:30.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.474+0000 7fd0ad600640 1 -- 172.21.15.94:0/2332608774 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fd09812ea70 con 0x7fd0b0152c10 2024-09-10T23:36:30.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.474+0000 7fd0b4c00640 1 --2- 172.21.15.94:0/2332608774 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd08c072940 0x7fd08c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:30.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.478+0000 7fd0b4c00640 1 --2- 172.21.15.94:0/2332608774 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd08c072940 0x7fd08c074e00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0007fd0 tx=0x7fd0a0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:30.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.478+0000 7fd0ad600640 1 -- 172.21.15.94:0/2332608774 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd098016310 con 0x7fd0b0152c10 2024-09-10T23:36:30.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:30.678+0000 7fd0b7600640 1 -- 172.21.15.94:0/2332608774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fd0b00cde20 con 0x7fd0b0152c10 2024-09-10T23:36:31.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.050+0000 7fd0ad600640 1 -- 172.21.15.94:0/2332608774 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v44) ==== 136+0+0 (secure 0 0 0) 0x7fd0980f9bf0 con 0x7fd0b0152c10 2024-09-10T23:36:31.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.054+0000 7fd0b7600640 1 -- 172.21.15.94:0/2332608774 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd08c072940 msgr2=0x7fd08c074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:31.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.054+0000 7fd0b7600640 1 --2- 172.21.15.94:0/2332608774 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd08c072940 0x7fd08c074e00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0007fd0 tx=0x7fd0a0009040 comp rx=0 tx=0).stop 2024-09-10T23:36:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.054+0000 7fd0b7600640 1 -- 172.21.15.94:0/2332608774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b0152c10 msgr2=0x7fd0b006fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.054+0000 7fd0b7600640 1 --2- 172.21.15.94:0/2332608774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b0152c10 0x7fd0b006fe80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd09801cb40 tx=0x7fd0980a6540 comp rx=0 tx=0).stop 2024-09-10T23:36:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.054+0000 7fd0b7600640 1 -- 172.21.15.94:0/2332608774 shutdown_connections 2024-09-10T23:36:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.054+0000 7fd0b7600640 1 --2- 172.21.15.94:0/2332608774 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fd08c072940 0x7fd08c074e00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.054+0000 7fd0b7600640 1 --2- 172.21.15.94:0/2332608774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0b015cbc0 0x7fd0b014e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.054+0000 7fd0b7600640 1 --2- 172.21.15.94:0/2332608774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0b0152c10 0x7fd0b006fe80 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.054+0000 7fd0b7600640 1 --2- 172.21.15.94:0/2332608774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0b00cfd50 0x7fd0b014bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.054+0000 7fd0b7600640 1 -- 172.21.15.94:0/2332608774 >> 172.21.15.94:0/2332608774 conn(0x7fd0b006cdf0 msgr2=0x7fd0b006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.054+0000 7fd0b7600640 1 -- 172.21.15.94:0/2332608774 shutdown_connections 2024-09-10T23:36:31.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.054+0000 7fd0b7600640 1 -- 172.21.15.94:0/2332608774 wait complete. 2024-09-10T23:36:31.075 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T23:36:31.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff472c00640 1 -- 172.21.15.94:0/3976723836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff46c0cea40 msgr2=0x7ff46c0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:31.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff472c00640 1 --2- 172.21.15.94:0/3976723836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff46c0cea40 0x7ff46c0b31a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff464009c80 tx=0x7ff46401f560 comp rx=0 tx=0).stop 2024-09-10T23:36:31.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff472c00640 1 -- 172.21.15.94:0/3976723836 shutdown_connections 2024-09-10T23:36:31.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff472c00640 1 --2- 172.21.15.94:0/3976723836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff46c0cea40 0x7ff46c0b31a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff472c00640 1 --2- 172.21.15.94:0/3976723836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff46c15adc0 0x7ff46c0cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff472c00640 1 -- 172.21.15.94:0/3976723836 >> 172.21.15.94:0/3976723836 conn(0x7ff46c074240 msgr2=0x7ff46c074620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff472c00640 1 -- 172.21.15.94:0/3976723836 shutdown_connections 2024-09-10T23:36:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff472c00640 1 -- 172.21.15.94:0/3976723836 wait complete. 2024-09-10T23:36:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff472c00640 1 Processor -- start 2024-09-10T23:36:31.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff472c00640 1 -- start start 2024-09-10T23:36:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff472c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff46c0cab60 0x7ff46c14e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff472c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff46c0cea40 0x7ff46c15d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff472c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff46c15adc0 0x7ff46c15d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff472c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c0cd530 con 0x7ff46c0cab60 2024-09-10T23:36:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff472c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c0cd3b0 con 0x7ff46c15adc0 2024-09-10T23:36:31.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff472c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff46c0cd6b0 con 0x7ff46c0cea40 2024-09-10T23:36:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff46be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff46c0cab60 0x7ff46c14e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff46be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff46c0cab60 0x7ff46c14e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51156/0 (socket says 172.21.15.94:51156) 2024-09-10T23:36:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff46be00640 1 -- 172.21.15.94:0/4135571079 learned_addr learned my addr 172.21.15.94:0/4135571079 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff470c00640 1 --2- 172.21.15.94:0/4135571079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff46c15adc0 0x7ff46c15d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff463400640 1 --2- 172.21.15.94:0/4135571079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff46c0cea40 0x7ff46c15d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff46be00640 1 -- 172.21.15.94:0/4135571079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff46c0cea40 msgr2=0x7ff46c15d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff46be00640 1 --2- 172.21.15.94:0/4135571079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff46c0cea40 0x7ff46c15d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff46be00640 1 -- 172.21.15.94:0/4135571079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff46c15adc0 msgr2=0x7ff46c15d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff46be00640 1 --2- 172.21.15.94:0/4135571079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff46c15adc0 0x7ff46c15d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff46be00640 1 -- 172.21.15.94:0/4135571079 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46c1f4770 con 0x7ff46c0cab60 2024-09-10T23:36:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff463400640 1 --2- 172.21.15.94:0/4135571079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff46c0cea40 0x7ff46c15d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:31.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.218+0000 7ff46be00640 1 --2- 172.21.15.94:0/4135571079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff46c0cab60 0x7ff46c14e2d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00e7a0 tx=0x7ff45c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.222+0000 7ff468c00640 1 -- 172.21.15.94:0/4135571079 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c00ccc0 con 0x7ff46c0cab60 2024-09-10T23:36:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.222+0000 7ff468c00640 1 -- 172.21.15.94:0/4135571079 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff45c010430 con 0x7ff46c0cab60 2024-09-10T23:36:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.222+0000 7ff468c00640 1 -- 172.21.15.94:0/4135571079 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c010860 con 0x7ff46c0cab60 2024-09-10T23:36:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.222+0000 7ff472c00640 1 -- 172.21.15.94:0/4135571079 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46c1f4a50 con 0x7ff46c0cab60 2024-09-10T23:36:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.222+0000 7ff472c00640 1 -- 172.21.15.94:0/4135571079 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff46c1f5010 con 0x7ff46c0cab60 2024-09-10T23:36:31.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.222+0000 7ff472c00640 1 -- 172.21.15.94:0/4135571079 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff46c0b3980 con 0x7ff46c0cab60 2024-09-10T23:36:31.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.222+0000 7ff468c00640 1 -- 172.21.15.94:0/4135571079 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7ff45c010a00 con 0x7ff46c0cab60 2024-09-10T23:36:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.222+0000 7ff468c00640 1 --2- 172.21.15.94:0/4135571079 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff448072940 0x7ff448074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.222+0000 7ff468c00640 1 -- 172.21.15.94:0/4135571079 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7ff45c0942f0 con 0x7ff46c0cab60 2024-09-10T23:36:31.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.226+0000 7ff463400640 1 --2- 172.21.15.94:0/4135571079 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff448072940 0x7ff448074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:31.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.226+0000 7ff463400640 1 --2- 172.21.15.94:0/4135571079 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff448072940 0x7ff448074e00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff46c159c60 tx=0x7ff454009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:31.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.226+0000 7ff468c00640 1 -- 172.21.15.94:0/4135571079 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff45c05eda0 con 0x7ff46c0cab60 2024-09-10T23:36:31.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.414+0000 7ff472c00640 1 -- 172.21.15.94:0/4135571079 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7ff46c0ce4d0 con 0x7ff448072940 2024-09-10T23:36:31.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.414+0000 7ff468c00640 1 -- 172.21.15.94:0/4135571079 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7ff46c0ce4d0 con 0x7ff448072940 2024-09-10T23:36:31.416 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:36:31.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:31.416 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephfs" 2024-09-10T23:36:31.416 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:31.416 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:36:31.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.418+0000 7ff472c00640 1 -- 172.21.15.94:0/4135571079 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff448072940 msgr2=0x7ff448074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:31.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.418+0000 7ff472c00640 1 --2- 172.21.15.94:0/4135571079 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff448072940 0x7ff448074e00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff46c159c60 tx=0x7ff454009040 comp rx=0 tx=0).stop 2024-09-10T23:36:31.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.418+0000 7ff472c00640 1 -- 172.21.15.94:0/4135571079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff46c0cab60 msgr2=0x7ff46c14e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:31.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.418+0000 7ff472c00640 1 --2- 172.21.15.94:0/4135571079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff46c0cab60 0x7ff46c14e2d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00e7a0 tx=0x7ff45c00ec60 comp rx=0 tx=0).stop 2024-09-10T23:36:31.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.418+0000 7ff472c00640 1 -- 172.21.15.94:0/4135571079 shutdown_connections 2024-09-10T23:36:31.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.418+0000 7ff472c00640 1 --2- 172.21.15.94:0/4135571079 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7ff448072940 0x7ff448074e00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.418+0000 7ff472c00640 1 --2- 172.21.15.94:0/4135571079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff46c15adc0 0x7ff46c15d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.418+0000 7ff472c00640 1 --2- 172.21.15.94:0/4135571079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff46c0cea40 0x7ff46c15d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.418+0000 7ff472c00640 1 --2- 172.21.15.94:0/4135571079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff46c0cab60 0x7ff46c14e2d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.418+0000 7ff472c00640 1 -- 172.21.15.94:0/4135571079 >> 172.21.15.94:0/4135571079 conn(0x7ff46c074240 msgr2=0x7ff46c0d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:31.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.418+0000 7ff472c00640 1 -- 172.21.15.94:0/4135571079 shutdown_connections 2024-09-10T23:36:31.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.418+0000 7ff472c00640 1 -- 172.21.15.94:0/4135571079 wait complete. 2024-09-10T23:36:31.437 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:36:31.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.578+0000 7f5e41c00640 1 -- 172.21.15.94:0/1767580440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3c15bf60 msgr2=0x7f5e3c0d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:31.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.578+0000 7f5e41c00640 1 --2- 172.21.15.94:0/1767580440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3c15bf60 0x7f5e3c0d2780 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009900 tx=0x7f5e2c01c550 comp rx=0 tx=0).stop 2024-09-10T23:36:31.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.578+0000 7f5e41c00640 1 -- 172.21.15.94:0/1767580440 shutdown_connections 2024-09-10T23:36:31.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.578+0000 7f5e41c00640 1 --2- 172.21.15.94:0/1767580440 >> v2:172.21.15.181:3300/0 conn(0x7f5e3c0d0660 0x7f5e3c0ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.578+0000 7f5e41c00640 1 --2- 172.21.15.94:0/1767580440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e3c0cfce0 0x7f5e3c0d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.578+0000 7f5e41c00640 1 --2- 172.21.15.94:0/1767580440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3c15bf60 0x7f5e3c0d2780 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.578+0000 7f5e41c00640 1 -- 172.21.15.94:0/1767580440 >> 172.21.15.94:0/1767580440 conn(0x7f5e3c0d1c60 msgr2=0x7f5e3c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:31.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.578+0000 7f5e41c00640 1 -- 172.21.15.94:0/1767580440 shutdown_connections 2024-09-10T23:36:31.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.578+0000 7f5e41c00640 1 -- 172.21.15.94:0/1767580440 wait complete. 2024-09-10T23:36:31.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e41c00640 1 Processor -- start 2024-09-10T23:36:31.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e41c00640 1 -- start start 2024-09-10T23:36:31.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e41c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e3c0cfce0 0x7f5e3c0cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:31.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e41c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e3c0d0660 0x7f5e3c147ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:31.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e41c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3c15bf60 0x7f5e3c148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:31.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e41c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c0c6a70 con 0x7f5e3c15bf60 2024-09-10T23:36:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e41c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c0c68f0 con 0x7f5e3c0cfce0 2024-09-10T23:36:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e41c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5e3c0c6bf0 con 0x7f5e3c0d0660 2024-09-10T23:36:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e3b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3c15bf60 0x7f5e3c148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e3b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3c15bf60 0x7f5e3c148220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51188/0 (socket says 172.21.15.94:51188) 2024-09-10T23:36:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e3b400640 1 -- 172.21.15.94:0/678143936 learned_addr learned my addr 172.21.15.94:0/678143936 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e32000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e3c0d0660 0x7f5e3c147ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:31.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e3aa00640 1 --2- 172.21.15.94:0/678143936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e3c0cfce0 0x7f5e3c0cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:31.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e3b400640 1 -- 172.21.15.94:0/678143936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e3c0d0660 msgr2=0x7f5e3c147ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:31.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e3b400640 1 --2- 172.21.15.94:0/678143936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e3c0d0660 0x7f5e3c147ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e3b400640 1 -- 172.21.15.94:0/678143936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e3c0cfce0 msgr2=0x7f5e3c0cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:31.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e3b400640 1 --2- 172.21.15.94:0/678143936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e3c0cfce0 0x7f5e3c0cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e3b400640 1 -- 172.21.15.94:0/678143936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3c0ca0c0 con 0x7f5e3c15bf60 2024-09-10T23:36:31.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e3b400640 1 --2- 172.21.15.94:0/678143936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3c15bf60 0x7f5e3c148220 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400d750 tx=0x7f5e3400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:31.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e33400640 1 -- 172.21.15.94:0/678143936 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34017070 con 0x7f5e3c15bf60 2024-09-10T23:36:31.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e33400640 1 -- 172.21.15.94:0/678143936 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e34009d20 con 0x7f5e3c15bf60 2024-09-10T23:36:31.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e41c00640 1 -- 172.21.15.94:0/678143936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3c0ca3a0 con 0x7f5e3c15bf60 2024-09-10T23:36:31.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e33400640 1 -- 172.21.15.94:0/678143936 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400de60 con 0x7f5e3c15bf60 2024-09-10T23:36:31.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.582+0000 7f5e41c00640 1 -- 172.21.15.94:0/678143936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e3c0ca960 con 0x7f5e3c15bf60 2024-09-10T23:36:31.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.586+0000 7f5e41c00640 1 -- 172.21.15.94:0/678143936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e3c0ce3f0 con 0x7f5e3c15bf60 2024-09-10T23:36:31.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.586+0000 7f5e33400640 1 -- 172.21.15.94:0/678143936 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f5e34005280 con 0x7f5e3c15bf60 2024-09-10T23:36:31.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.586+0000 7f5e33400640 1 --2- 172.21.15.94:0/678143936 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5e18072940 0x7f5e18074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:31.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.586+0000 7f5e33400640 1 -- 172.21.15.94:0/678143936 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f5e34093a50 con 0x7f5e3c15bf60 2024-09-10T23:36:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.586+0000 7f5e3aa00640 1 --2- 172.21.15.94:0/678143936 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5e18072940 0x7f5e18074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.586+0000 7f5e3aa00640 1 --2- 172.21.15.94:0/678143936 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5e18072940 0x7f5e18074e00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0af000 tx=0x7f5e2c01ad90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.590+0000 7f5e33400640 1 -- 172.21.15.94:0/678143936 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5e3405e480 con 0x7f5e3c15bf60 2024-09-10T23:36:31.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.758+0000 7f5e41c00640 1 -- 172.21.15.94:0/678143936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5e3c0b3030 con 0x7f5e3c15bf60 2024-09-10T23:36:31.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.758+0000 7f5e33400640 1 -- 172.21.15.94:0/678143936 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f5e340671f0 con 0x7f5e3c15bf60 2024-09-10T23:36:31.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.758+0000 7f5e41c00640 1 -- 172.21.15.94:0/678143936 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5e18072940 msgr2=0x7f5e18074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:31.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.758+0000 7f5e41c00640 1 --2- 172.21.15.94:0/678143936 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5e18072940 0x7f5e18074e00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0af000 tx=0x7f5e2c01ad90 comp rx=0 tx=0).stop 2024-09-10T23:36:31.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.758+0000 7f5e41c00640 1 -- 172.21.15.94:0/678143936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3c15bf60 msgr2=0x7f5e3c148220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:31.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.758+0000 7f5e41c00640 1 --2- 172.21.15.94:0/678143936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3c15bf60 0x7f5e3c148220 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400d750 tx=0x7f5e3400dc10 comp rx=0 tx=0).stop 2024-09-10T23:36:31.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.758+0000 7f5e41c00640 1 -- 172.21.15.94:0/678143936 shutdown_connections 2024-09-10T23:36:31.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.758+0000 7f5e41c00640 1 --2- 172.21.15.94:0/678143936 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f5e18072940 0x7f5e18074e00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.758+0000 7f5e41c00640 1 --2- 172.21.15.94:0/678143936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3c15bf60 0x7f5e3c148220 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.758+0000 7f5e41c00640 1 --2- 172.21.15.94:0/678143936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e3c0d0660 0x7f5e3c147ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.758+0000 7f5e41c00640 1 --2- 172.21.15.94:0/678143936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e3c0cfce0 0x7f5e3c0cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.758+0000 7f5e41c00640 1 -- 172.21.15.94:0/678143936 >> 172.21.15.94:0/678143936 conn(0x7f5e3c0d1c60 msgr2=0x7f5e3c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:31.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.758+0000 7f5e41c00640 1 -- 172.21.15.94:0/678143936 shutdown_connections 2024-09-10T23:36:31.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.758+0000 7f5e41c00640 1 -- 172.21.15.94:0/678143936 wait complete. 2024-09-10T23:36:31.780 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:36:31.780 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:31.789 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:36:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c7c200640 1 -- 172.21.15.94:0/4102953264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7406e7c0 msgr2=0x7f2c7415af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c7c200640 1 --2- 172.21.15.94:0/4102953264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7406e7c0 0x7f2c7415af90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c00aa10 tx=0x7f2c6c01dc30 comp rx=0 tx=0).stop 2024-09-10T23:36:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c7c200640 1 -- 172.21.15.94:0/4102953264 shutdown_connections 2024-09-10T23:36:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c7c200640 1 --2- 172.21.15.94:0/4102953264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7406e7c0 0x7f2c7415af90 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c7c200640 1 --2- 172.21.15.94:0/4102953264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c740ce280 0x7f2c740cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c7c200640 1 -- 172.21.15.94:0/4102953264 >> 172.21.15.94:0/4102953264 conn(0x7f2c740cfaf0 msgr2=0x7f2c740cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c7c200640 1 -- 172.21.15.94:0/4102953264 shutdown_connections 2024-09-10T23:36:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c7c200640 1 -- 172.21.15.94:0/4102953264 wait complete. 2024-09-10T23:36:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c7c200640 1 Processor -- start 2024-09-10T23:36:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c7c200640 1 -- start start 2024-09-10T23:36:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c7c200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7406e7c0 0x7f2c7414c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c7c200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c740c64f0 0x7f2c74072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c7c200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c740ce280 0x7f2c740734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c7c200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c741676d0 con 0x7f2c7406e7c0 2024-09-10T23:36:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c7c200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2c74167550 con 0x7f2c740c64f0 2024-09-10T23:36:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c7c200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2c74167850 con 0x7f2c740ce280 2024-09-10T23:36:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.974+0000 7f2c79800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7406e7c0 0x7f2c7414c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c79800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7406e7c0 0x7f2c7414c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51198/0 (socket says 172.21.15.94:51198) 2024-09-10T23:36:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c79800640 1 -- 172.21.15.94:0/4093529197 learned_addr learned my addr 172.21.15.94:0/4093529197 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c7a200640 1 --2- 172.21.15.94:0/4093529197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c740ce280 0x7f2c740734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c78e00640 1 --2- 172.21.15.94:0/4093529197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c740c64f0 0x7f2c74072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c79800640 1 -- 172.21.15.94:0/4093529197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c740ce280 msgr2=0x7f2c740734b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c79800640 1 --2- 172.21.15.94:0/4093529197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c740ce280 0x7f2c740734b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c79800640 1 -- 172.21.15.94:0/4093529197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c740c64f0 msgr2=0x7f2c74072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c79800640 1 --2- 172.21.15.94:0/4093529197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c740c64f0 0x7f2c74072f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c79800640 1 -- 172.21.15.94:0/4093529197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c740b4040 con 0x7f2c7406e7c0 2024-09-10T23:36:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c79800640 1 --2- 172.21.15.94:0/4093529197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7406e7c0 0x7f2c7414c4a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800eaa0 tx=0x7f2c6800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:31.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c71600640 1 -- 172.21.15.94:0/4093529197 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6800ce10 con 0x7f2c7406e7c0 2024-09-10T23:36:31.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c71600640 1 -- 172.21.15.94:0/4093529197 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c68011470 con 0x7f2c7406e7c0 2024-09-10T23:36:31.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c71600640 1 -- 172.21.15.94:0/4093529197 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c680118a0 con 0x7f2c7406e7c0 2024-09-10T23:36:31.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c7c200640 1 -- 172.21.15.94:0/4093529197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c740b4320 con 0x7f2c7406e7c0 2024-09-10T23:36:31.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c7c200640 1 -- 172.21.15.94:0/4093529197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c740b48e0 con 0x7f2c7406e7c0 2024-09-10T23:36:31.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c7c200640 1 -- 172.21.15.94:0/4093529197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c7415af90 con 0x7f2c7406e7c0 2024-09-10T23:36:31.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.978+0000 7f2c71600640 1 -- 172.21.15.94:0/4093529197 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f2c68022030 con 0x7f2c7406e7c0 2024-09-10T23:36:31.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.982+0000 7f2c71600640 1 --2- 172.21.15.94:0/4093529197 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2c54072940 0x7f2c54074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:31.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.982+0000 7f2c71600640 1 -- 172.21.15.94:0/4093529197 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f2c68094300 con 0x7f2c7406e7c0 2024-09-10T23:36:31.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.982+0000 7f2c78e00640 1 --2- 172.21.15.94:0/4093529197 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2c54072940 0x7f2c54074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.982+0000 7f2c78e00640 1 --2- 172.21.15.94:0/4093529197 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2c54072940 0x7f2c54074e00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2c741652b0 tx=0x7f2c60009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:31.982+0000 7f2c71600640 1 -- 172.21.15.94:0/4093529197 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2c68016030 con 0x7f2c7406e7c0 2024-09-10T23:36:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.162+0000 7f2c7c200640 1 -- 172.21.15.94:0/4093529197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2c740d0850 con 0x7f2c7406e7c0 2024-09-10T23:36:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.162+0000 7f2c71600640 1 -- 172.21.15.94:0/4093529197 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) ==== 80+0+0 (secure 0 0 0) 0x7f2c6805f410 con 0x7f2c7406e7c0 2024-09-10T23:36:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:36:32.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.166+0000 7f2c7c200640 1 -- 172.21.15.94:0/4093529197 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2c54072940 msgr2=0x7f2c54074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:32.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.166+0000 7f2c7c200640 1 --2- 172.21.15.94:0/4093529197 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2c54072940 0x7f2c54074e00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2c741652b0 tx=0x7f2c60009040 comp rx=0 tx=0).stop 2024-09-10T23:36:32.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.166+0000 7f2c7c200640 1 -- 172.21.15.94:0/4093529197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7406e7c0 msgr2=0x7f2c7414c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:32.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.166+0000 7f2c7c200640 1 --2- 172.21.15.94:0/4093529197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7406e7c0 0x7f2c7414c4a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800eaa0 tx=0x7f2c6800ef60 comp rx=0 tx=0).stop 2024-09-10T23:36:32.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.166+0000 7f2c7c200640 1 -- 172.21.15.94:0/4093529197 shutdown_connections 2024-09-10T23:36:32.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.166+0000 7f2c7c200640 1 --2- 172.21.15.94:0/4093529197 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2c54072940 0x7f2c54074e00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:32.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.166+0000 7f2c7c200640 1 --2- 172.21.15.94:0/4093529197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c740c64f0 0x7f2c74072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:32.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.166+0000 7f2c7c200640 1 --2- 172.21.15.94:0/4093529197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c740ce280 0x7f2c740734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:32.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.166+0000 7f2c7c200640 1 --2- 172.21.15.94:0/4093529197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c7406e7c0 0x7f2c7414c4a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:32.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.166+0000 7f2c7c200640 1 -- 172.21.15.94:0/4093529197 >> 172.21.15.94:0/4093529197 conn(0x7f2c740cfaf0 msgr2=0x7f2c740c5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:32.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.166+0000 7f2c7c200640 1 -- 172.21.15.94:0/4093529197 shutdown_connections 2024-09-10T23:36:32.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.166+0000 7f2c7c200640 1 -- 172.21.15.94:0/4093529197 wait complete. 2024-09-10T23:36:32.185 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:36:32.185 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:32.194 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:36:32.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.378+0000 7f378d200640 1 -- 172.21.15.94:0/1515112524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37880b2570 msgr2=0x7f37880b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:32.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.378+0000 7f378d200640 1 --2- 172.21.15.94:0/1515112524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37880b2570 0x7f37880b29a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3788069920 tx=0x7f378001ccf0 comp rx=0 tx=0).stop 2024-09-10T23:36:32.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.378+0000 7f378d200640 1 -- 172.21.15.94:0/1515112524 shutdown_connections 2024-09-10T23:36:32.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.378+0000 7f378d200640 1 --2- 172.21.15.94:0/1515112524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37880b2570 0x7f37880b29a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:32.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.378+0000 7f378d200640 1 --2- 172.21.15.94:0/1515112524 >> v2:172.21.15.181:3300/0 conn(0x7f37880b3150 0x7f37880d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:32.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.378+0000 7f378d200640 1 -- 172.21.15.94:0/1515112524 >> 172.21.15.94:0/1515112524 conn(0x7f37880d1cb0 msgr2=0x7f37880d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:32.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.378+0000 7f378d200640 1 -- 172.21.15.94:0/1515112524 shutdown_connections 2024-09-10T23:36:32.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.378+0000 7f378d200640 1 -- 172.21.15.94:0/1515112524 wait complete. 2024-09-10T23:36:32.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f378d200640 1 Processor -- start 2024-09-10T23:36:32.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f378d200640 1 -- start start 2024-09-10T23:36:32.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f378d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37880b2570 0x7f37880b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:32.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f378d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37880b3150 0x7f37881557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:32.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f378d200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37881613c0 0x7f3788071be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:32.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f378d200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f378816b0d0 con 0x7f37880b2570 2024-09-10T23:36:32.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f378d200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f378816af50 con 0x7f37880b3150 2024-09-10T23:36:32.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f378d200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f378816b250 con 0x7f37881613c0 2024-09-10T23:36:32.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f3785e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37880b2570 0x7f37880b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:32.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f3785e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37880b2570 0x7f37880b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51224/0 (socket says 172.21.15.94:51224) 2024-09-10T23:36:32.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f3785e00640 1 -- 172.21.15.94:0/2749370713 learned_addr learned my addr 172.21.15.94:0/2749370713 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:32.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f377d400640 1 --2- 172.21.15.94:0/2749370713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37880b3150 0x7f37881557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:32.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f3786800640 1 --2- 172.21.15.94:0/2749370713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37881613c0 0x7f3788071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:32.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f3785e00640 1 -- 172.21.15.94:0/2749370713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37881613c0 msgr2=0x7f3788071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:32.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f3785e00640 1 --2- 172.21.15.94:0/2749370713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37881613c0 0x7f3788071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:32.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f3785e00640 1 -- 172.21.15.94:0/2749370713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37880b3150 msgr2=0x7f37881557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:32.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f3785e00640 1 --2- 172.21.15.94:0/2749370713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37880b3150 0x7f37881557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:32.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f3785e00640 1 -- 172.21.15.94:0/2749370713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37880b4e50 con 0x7f37880b2570 2024-09-10T23:36:32.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f377d400640 1 --2- 172.21.15.94:0/2749370713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37880b3150 0x7f37881557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:32.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f3785e00640 1 --2- 172.21.15.94:0/2749370713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37880b2570 0x7f37880b6d00 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f377800c3c0 tx=0x7f377800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:32.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f377ea00640 1 -- 172.21.15.94:0/2749370713 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778017070 con 0x7f37880b2570 2024-09-10T23:36:32.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f377ea00640 1 -- 172.21.15.94:0/2749370713 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37780053c0 con 0x7f37880b2570 2024-09-10T23:36:32.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f377ea00640 1 -- 172.21.15.94:0/2749370713 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778003860 con 0x7f37880b2570 2024-09-10T23:36:32.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f378d200640 1 -- 172.21.15.94:0/2749370713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37880b5130 con 0x7f37880b2570 2024-09-10T23:36:32.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f378d200640 1 -- 172.21.15.94:0/2749370713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37880b56f0 con 0x7f37880b2570 2024-09-10T23:36:32.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.382+0000 7f378d200640 1 -- 172.21.15.94:0/2749370713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37880b29a0 con 0x7f37880b2570 2024-09-10T23:36:32.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.386+0000 7f377ea00640 1 -- 172.21.15.94:0/2749370713 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f3778003a00 con 0x7f37880b2570 2024-09-10T23:36:32.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.386+0000 7f377ea00640 1 --2- 172.21.15.94:0/2749370713 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3764072940 0x7f3764074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:32.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.386+0000 7f377ea00640 1 -- 172.21.15.94:0/2749370713 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f3778092ec0 con 0x7f37880b2570 2024-09-10T23:36:32.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.386+0000 7f377d400640 1 --2- 172.21.15.94:0/2749370713 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3764072940 0x7f3764074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:32.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.386+0000 7f377d400640 1 --2- 172.21.15.94:0/2749370713 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3764072940 0x7f3764074e00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f37700058d0 tx=0x7f3770005860 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:32.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.390+0000 7f377ea00640 1 -- 172.21.15.94:0/2749370713 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f377805e080 con 0x7f37880b2570 2024-09-10T23:36:32.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.566+0000 7f378d200640 1 -- 172.21.15.94:0/2749370713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3788159e70 con 0x7f37880b2570 2024-09-10T23:36:32.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.566+0000 7f377ea00640 1 -- 172.21.15.94:0/2749370713 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) ==== 80+0+0 (secure 0 0 0) 0x7f377805d980 con 0x7f37880b2570 2024-09-10T23:36:32.569 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:36:32.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.570+0000 7f378d200640 1 -- 172.21.15.94:0/2749370713 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3764072940 msgr2=0x7f3764074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:32.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.570+0000 7f378d200640 1 --2- 172.21.15.94:0/2749370713 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3764072940 0x7f3764074e00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f37700058d0 tx=0x7f3770005860 comp rx=0 tx=0).stop 2024-09-10T23:36:32.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.570+0000 7f378d200640 1 -- 172.21.15.94:0/2749370713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37880b2570 msgr2=0x7f37880b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:32.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.570+0000 7f378d200640 1 --2- 172.21.15.94:0/2749370713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37880b2570 0x7f37880b6d00 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f377800c3c0 tx=0x7f377800c880 comp rx=0 tx=0).stop 2024-09-10T23:36:32.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.570+0000 7f378d200640 1 -- 172.21.15.94:0/2749370713 shutdown_connections 2024-09-10T23:36:32.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.570+0000 7f378d200640 1 --2- 172.21.15.94:0/2749370713 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f3764072940 0x7f3764074e00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:32.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.570+0000 7f378d200640 1 --2- 172.21.15.94:0/2749370713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37880b3150 0x7f37881557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:32.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.570+0000 7f378d200640 1 --2- 172.21.15.94:0/2749370713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37881613c0 0x7f3788071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:32.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.570+0000 7f378d200640 1 --2- 172.21.15.94:0/2749370713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37880b2570 0x7f37880b6d00 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:32.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.570+0000 7f378d200640 1 -- 172.21.15.94:0/2749370713 >> 172.21.15.94:0/2749370713 conn(0x7f37880d1cb0 msgr2=0x7f37880707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:32.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.570+0000 7f378d200640 1 -- 172.21.15.94:0/2749370713 shutdown_connections 2024-09-10T23:36:32.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.570+0000 7f378d200640 1 -- 172.21.15.94:0/2749370713 wait complete. 2024-09-10T23:36:32.590 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-09-10T23:36:32.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:32.590 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T23:36:32.607 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[41891]: fuse finished with error 0 and tester_r 0 2024-09-10T23:36:32.610 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:36:32.671 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:36:32.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:32.671 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T23:36:32.680 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:36:32.681 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:32.690 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:36:32.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.866+0000 7f8b93400640 1 -- 172.21.15.94:0/443508166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b8c1455d0 msgr2=0x7f8b8c0c6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:32.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.866+0000 7f8b93400640 1 --2- 172.21.15.94:0/443508166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b8c1455d0 0x7f8b8c0c6460 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c06e760 tx=0x7f8b7c01c820 comp rx=0 tx=0).stop 2024-09-10T23:36:32.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.866+0000 7f8b93400640 1 -- 172.21.15.94:0/443508166 shutdown_connections 2024-09-10T23:36:32.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.866+0000 7f8b93400640 1 --2- 172.21.15.94:0/443508166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b8c1455d0 0x7f8b8c0c6460 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:32.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.866+0000 7f8b93400640 1 -- 172.21.15.94:0/443508166 >> 172.21.15.94:0/443508166 conn(0x7f8b8c0cfaf0 msgr2=0x7f8b8c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:32.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.866+0000 7f8b93400640 1 -- 172.21.15.94:0/443508166 shutdown_connections 2024-09-10T23:36:32.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.866+0000 7f8b93400640 1 -- 172.21.15.94:0/443508166 wait complete. 2024-09-10T23:36:32.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.866+0000 7f8b93400640 1 Processor -- start 2024-09-10T23:36:32.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.866+0000 7f8b93400640 1 -- start start 2024-09-10T23:36:32.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b93400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b8c0cd9f0 0x7f8b8c0cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:32.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b93400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b8c0d0710 0x7f8b8c13ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:32.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b93400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b8c1455d0 0x7f8b8c157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b93400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c16a360 con 0x7f8b8c0cd9f0 2024-09-10T23:36:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b93400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c16a1e0 con 0x7f8b8c0d0710 2024-09-10T23:36:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b93400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8b8c16a4e0 con 0x7f8b8c1455d0 2024-09-10T23:36:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b90a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b8c0cd9f0 0x7f8b8c0cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b90a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b8c0cd9f0 0x7f8b8c0cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51256/0 (socket says 172.21.15.94:51256) 2024-09-10T23:36:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b90a00640 1 -- 172.21.15.94:0/2336642110 learned_addr learned my addr 172.21.15.94:0/2336642110 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b90a00640 1 -- 172.21.15.94:0/2336642110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b8c1455d0 msgr2=0x7f8b8c157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b90a00640 1 --2- 172.21.15.94:0/2336642110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b8c1455d0 0x7f8b8c157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b90a00640 1 -- 172.21.15.94:0/2336642110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b8c0d0710 msgr2=0x7f8b8c13ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b90a00640 1 --2- 172.21.15.94:0/2336642110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b8c0d0710 0x7f8b8c13ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:32.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b90a00640 1 -- 172.21.15.94:0/2336642110 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c0c7330 con 0x7f8b8c0cd9f0 2024-09-10T23:36:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b90a00640 1 --2- 172.21.15.94:0/2336642110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b8c0cd9f0 0x7f8b8c0cbd00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c000c10 tx=0x7f8b7c0a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b89600640 1 -- 172.21.15.94:0/2336642110 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c0bb070 con 0x7f8b8c0cd9f0 2024-09-10T23:36:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b89600640 1 -- 172.21.15.94:0/2336642110 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b7c004470 con 0x7f8b8c0cd9f0 2024-09-10T23:36:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b93400640 1 -- 172.21.15.94:0/2336642110 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8c0c7050 con 0x7f8b8c0cd9f0 2024-09-10T23:36:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b89600640 1 -- 172.21.15.94:0/2336642110 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c0b2040 con 0x7f8b8c0cd9f0 2024-09-10T23:36:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b93400640 1 -- 172.21.15.94:0/2336642110 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b8c0b52a0 con 0x7f8b8c0cd9f0 2024-09-10T23:36:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b93400640 1 -- 172.21.15.94:0/2336642110 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b8c0c6460 con 0x7f8b8c0cd9f0 2024-09-10T23:36:32.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.870+0000 7f8b89600640 1 -- 172.21.15.94:0/2336642110 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f8b7c0b5030 con 0x7f8b8c0cd9f0 2024-09-10T23:36:32.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.874+0000 7f8b89600640 1 --2- 172.21.15.94:0/2336642110 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b68072940 0x7f8b68074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:32.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.874+0000 7f8b89600640 1 -- 172.21.15.94:0/2336642110 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f8b7c12d220 con 0x7f8b8c0cd9f0 2024-09-10T23:36:32.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.874+0000 7f8b83e00640 1 --2- 172.21.15.94:0/2336642110 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b68072940 0x7f8b68074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:32.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.874+0000 7f8b83e00640 1 --2- 172.21.15.94:0/2336642110 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b68072940 0x7f8b68074e00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8b740046c0 tx=0x7f8b740012c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:32.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:32.878+0000 7f8b89600640 1 -- 172.21.15.94:0/2336642110 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8b7c016410 con 0x7f8b8c0cd9f0 2024-09-10T23:36:33.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.054+0000 7f8b93400640 1 -- 172.21.15.94:0/2336642110 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8b8c069e20 con 0x7f8b8c0cd9f0 2024-09-10T23:36:33.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.058+0000 7f8b89600640 1 -- 172.21.15.94:0/2336642110 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) ==== 80+0+0 (secure 0 0 0) 0x7f8b8c069e20 con 0x7f8b8c0cd9f0 2024-09-10T23:36:33.058 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:36:33.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.058+0000 7f8b93400640 1 -- 172.21.15.94:0/2336642110 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b68072940 msgr2=0x7f8b68074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:33.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.058+0000 7f8b93400640 1 --2- 172.21.15.94:0/2336642110 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b68072940 0x7f8b68074e00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8b740046c0 tx=0x7f8b740012c0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.058+0000 7f8b93400640 1 -- 172.21.15.94:0/2336642110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b8c0cd9f0 msgr2=0x7f8b8c0cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:33.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.058+0000 7f8b93400640 1 --2- 172.21.15.94:0/2336642110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b8c0cd9f0 0x7f8b8c0cbd00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c000c10 tx=0x7f8b7c0a5650 comp rx=0 tx=0).stop 2024-09-10T23:36:33.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.058+0000 7f8b93400640 1 -- 172.21.15.94:0/2336642110 shutdown_connections 2024-09-10T23:36:33.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.058+0000 7f8b93400640 1 --2- 172.21.15.94:0/2336642110 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f8b68072940 0x7f8b68074e00 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.058+0000 7f8b93400640 1 --2- 172.21.15.94:0/2336642110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b8c0d0710 0x7f8b8c13ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.058+0000 7f8b93400640 1 --2- 172.21.15.94:0/2336642110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b8c1455d0 0x7f8b8c157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.058+0000 7f8b93400640 1 --2- 172.21.15.94:0/2336642110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b8c0cd9f0 0x7f8b8c0cbd00 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.058+0000 7f8b93400640 1 -- 172.21.15.94:0/2336642110 >> 172.21.15.94:0/2336642110 conn(0x7f8b8c0cfaf0 msgr2=0x7f8b8c0afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:33.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.058+0000 7f8b93400640 1 -- 172.21.15.94:0/2336642110 shutdown_connections 2024-09-10T23:36:33.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.058+0000 7f8b93400640 1 -- 172.21.15.94:0/2336642110 wait complete. 2024-09-10T23:36:33.080 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:36:33.080 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:33.089 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:36:33.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.230+0000 7f297c000640 1 -- 172.21.15.94:0/277129179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2974158790 msgr2=0x7f297406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:33.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.230+0000 7f297c000640 1 --2- 172.21.15.94:0/277129179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2974158790 0x7f297406fc00 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2974069640 tx=0x7f296801cd80 comp rx=0 tx=0).stop 2024-09-10T23:36:33.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.230+0000 7f297c000640 1 -- 172.21.15.94:0/277129179 shutdown_connections 2024-09-10T23:36:33.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.230+0000 7f297c000640 1 --2- 172.21.15.94:0/277129179 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29740cb300 0x7f297415e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.230+0000 7f297c000640 1 --2- 172.21.15.94:0/277129179 >> v2:172.21.15.181:3300/0 conn(0x7f29740c49b0 0x7f29740cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.230+0000 7f297c000640 1 --2- 172.21.15.94:0/277129179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2974158790 0x7f297406fc00 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.230+0000 7f297c000640 1 -- 172.21.15.94:0/277129179 >> 172.21.15.94:0/277129179 conn(0x7f29740cfaf0 msgr2=0x7f29740cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:33.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.230+0000 7f297c000640 1 -- 172.21.15.94:0/277129179 shutdown_connections 2024-09-10T23:36:33.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.230+0000 7f297c000640 1 -- 172.21.15.94:0/277129179 wait complete. 2024-09-10T23:36:33.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f297c000640 1 Processor -- start 2024-09-10T23:36:33.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f297c000640 1 -- start start 2024-09-10T23:36:33.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f297c000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740c49b0 0x7f29741f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:33.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f297c000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29740cb300 0x7f29740b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:33.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f297c000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2974158790 0x7f2974157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:33.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f297c000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f297415b8d0 con 0x7f29740c49b0 2024-09-10T23:36:33.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f297c000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f297415b750 con 0x7f29740cb300 2024-09-10T23:36:33.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f297c000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f297415ba50 con 0x7f2974158790 2024-09-10T23:36:33.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f2979600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740c49b0 0x7f29741f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:33.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f2979600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740c49b0 0x7f29741f7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51264/0 (socket says 172.21.15.94:51264) 2024-09-10T23:36:33.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f2979600640 1 -- 172.21.15.94:0/20327070 learned_addr learned my addr 172.21.15.94:0/20327070 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:33.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f297a000640 1 --2- 172.21.15.94:0/20327070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2974158790 0x7f2974157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:33.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f2978c00640 1 --2- 172.21.15.94:0/20327070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29740cb300 0x7f29740b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:33.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f2979600640 1 -- 172.21.15.94:0/20327070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2974158790 msgr2=0x7f2974157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:33.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f2979600640 1 --2- 172.21.15.94:0/20327070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2974158790 0x7f2974157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f2979600640 1 -- 172.21.15.94:0/20327070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29740cb300 msgr2=0x7f29740b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:33.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f2979600640 1 --2- 172.21.15.94:0/20327070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29740cb300 0x7f29740b2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f2979600640 1 -- 172.21.15.94:0/20327070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2974145570 con 0x7f29740c49b0 2024-09-10T23:36:33.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f2978c00640 1 --2- 172.21.15.94:0/20327070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29740cb300 0x7f29740b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:33.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f2979600640 1 --2- 172.21.15.94:0/20327070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740c49b0 0x7f29741f7e40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2968000f90 tx=0x7f2968009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:33.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f2971600640 1 -- 172.21.15.94:0/20327070 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29680bb070 con 0x7f29740c49b0 2024-09-10T23:36:33.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f2971600640 1 -- 172.21.15.94:0/20327070 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2968004d40 con 0x7f29740c49b0 2024-09-10T23:36:33.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f297c000640 1 -- 172.21.15.94:0/20327070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29741fb930 con 0x7f29740c49b0 2024-09-10T23:36:33.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f2971600640 1 -- 172.21.15.94:0/20327070 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29680b2040 con 0x7f29740c49b0 2024-09-10T23:36:33.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.234+0000 7f297c000640 1 -- 172.21.15.94:0/20327070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29741fbef0 con 0x7f29740c49b0 2024-09-10T23:36:33.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.238+0000 7f297c000640 1 -- 172.21.15.94:0/20327070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f297415bf60 con 0x7f29740c49b0 2024-09-10T23:36:33.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.238+0000 7f2971600640 1 -- 172.21.15.94:0/20327070 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f29680b5030 con 0x7f29740c49b0 2024-09-10T23:36:33.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.238+0000 7f2971600640 1 --2- 172.21.15.94:0/20327070 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2954072940 0x7f2954074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:33.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.238+0000 7f2971600640 1 -- 172.21.15.94:0/20327070 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f296812d550 con 0x7f29740c49b0 2024-09-10T23:36:33.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.238+0000 7f2978c00640 1 --2- 172.21.15.94:0/20327070 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2954072940 0x7f2954074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:33.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.238+0000 7f2978c00640 1 --2- 172.21.15.94:0/20327070 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2954072940 0x7f2954074e00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2960005fd0 tx=0x7f2960005f20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:33.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.242+0000 7f2971600640 1 -- 172.21.15.94:0/20327070 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2968016970 con 0x7f29740c49b0 2024-09-10T23:36:33.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.422+0000 7f297c000640 1 -- 172.21.15.94:0/20327070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f29740cb070 con 0x7f29740c49b0 2024-09-10T23:36:33.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.422+0000 7f2971600640 1 -- 172.21.15.94:0/20327070 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) ==== 80+0+0 (secure 0 0 0) 0x7f29680f85e0 con 0x7f29740c49b0 2024-09-10T23:36:33.423 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:36:33.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.422+0000 7f297c000640 1 -- 172.21.15.94:0/20327070 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2954072940 msgr2=0x7f2954074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:33.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.422+0000 7f297c000640 1 --2- 172.21.15.94:0/20327070 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2954072940 0x7f2954074e00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2960005fd0 tx=0x7f2960005f20 comp rx=0 tx=0).stop 2024-09-10T23:36:33.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.422+0000 7f297c000640 1 -- 172.21.15.94:0/20327070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740c49b0 msgr2=0x7f29741f7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:33.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.422+0000 7f297c000640 1 --2- 172.21.15.94:0/20327070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740c49b0 0x7f29741f7e40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2968000f90 tx=0x7f2968009360 comp rx=0 tx=0).stop 2024-09-10T23:36:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.422+0000 7f297c000640 1 -- 172.21.15.94:0/20327070 shutdown_connections 2024-09-10T23:36:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.422+0000 7f297c000640 1 --2- 172.21.15.94:0/20327070 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f2954072940 0x7f2954074e00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.422+0000 7f297c000640 1 --2- 172.21.15.94:0/20327070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29740cb300 0x7f29740b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.422+0000 7f297c000640 1 --2- 172.21.15.94:0/20327070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2974158790 0x7f2974157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.422+0000 7f297c000640 1 --2- 172.21.15.94:0/20327070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740c49b0 0x7f29741f7e40 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.422+0000 7f297c000640 1 -- 172.21.15.94:0/20327070 >> 172.21.15.94:0/20327070 conn(0x7f29740cfaf0 msgr2=0x7f29740af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.422+0000 7f297c000640 1 -- 172.21.15.94:0/20327070 shutdown_connections 2024-09-10T23:36:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.422+0000 7f297c000640 1 -- 172.21.15.94:0/20327070 wait complete. 2024-09-10T23:36:33.445 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi181.front.sepia.ceph.com... 2024-09-10T23:36:33.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:33.445 DEBUG:teuthology.orchestra.run.smithi181:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T23:36:33.463 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[35085]: fuse finished with error 0 and tester_r 0 2024-09-10T23:36:33.466 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:36:33.499 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:36:33.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:33.499 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T23:36:33.550 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:33.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.690+0000 7fb186c00640 1 -- 172.21.15.94:0/894603905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb18015f5c0 msgr2=0x7fb1800cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:33.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.690+0000 7fb186c00640 1 --2- 172.21.15.94:0/894603905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb18015f5c0 0x7fb1800cfb60 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb18006d8b0 tx=0x7fb17001c7a0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.690+0000 7fb186c00640 1 -- 172.21.15.94:0/894603905 shutdown_connections 2024-09-10T23:36:33.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.690+0000 7fb186c00640 1 --2- 172.21.15.94:0/894603905 >> v2:172.21.15.181:3300/0 conn(0x7fb1800b2bb0 0x7fb1800d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.690+0000 7fb186c00640 1 --2- 172.21.15.94:0/894603905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb18015f5c0 0x7fb1800cfb60 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.690+0000 7fb186c00640 1 -- 172.21.15.94:0/894603905 >> 172.21.15.94:0/894603905 conn(0x7fb1800d1c60 msgr2=0x7fb1800d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:33.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.690+0000 7fb186c00640 1 -- 172.21.15.94:0/894603905 shutdown_connections 2024-09-10T23:36:33.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.690+0000 7fb186c00640 1 -- 172.21.15.94:0/894603905 wait complete. 2024-09-10T23:36:33.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb186c00640 1 Processor -- start 2024-09-10T23:36:33.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb186c00640 1 -- start start 2024-09-10T23:36:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb186c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1800b2bb0 0x7fb180146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb186c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1800d05e0 0x7fb18006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb186c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb18015f5c0 0x7fb18006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb186c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb180169170 con 0x7fb18015f5c0 2024-09-10T23:36:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb186c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb180168ff0 con 0x7fb1800b2bb0 2024-09-10T23:36:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb186c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb1801692f0 con 0x7fb1800d05e0 2024-09-10T23:36:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb184c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb18015f5c0 0x7fb18006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb184c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb18015f5c0 0x7fb18006f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51288/0 (socket says 172.21.15.94:51288) 2024-09-10T23:36:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb184c00640 1 -- 172.21.15.94:0/780071839 learned_addr learned my addr 172.21.15.94:0/780071839 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb177400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1800d05e0 0x7fb18006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb17fe00640 1 --2- 172.21.15.94:0/780071839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1800b2bb0 0x7fb180146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb184c00640 1 -- 172.21.15.94:0/780071839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1800d05e0 msgr2=0x7fb18006eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb184c00640 1 --2- 172.21.15.94:0/780071839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1800d05e0 0x7fb18006eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb184c00640 1 -- 172.21.15.94:0/780071839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1800b2bb0 msgr2=0x7fb180146bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb184c00640 1 --2- 172.21.15.94:0/780071839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1800b2bb0 0x7fb180146bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb184c00640 1 -- 172.21.15.94:0/780071839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1800c7ab0 con 0x7fb18015f5c0 2024-09-10T23:36:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb184c00640 1 --2- 172.21.15.94:0/780071839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb18015f5c0 0x7fb18006f3f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb17800c7d0 tx=0x7fb17800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb17cc00640 1 -- 172.21.15.94:0/780071839 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178003b80 con 0x7fb18015f5c0 2024-09-10T23:36:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb17cc00640 1 -- 172.21.15.94:0/780071839 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb178003d40 con 0x7fb18015f5c0 2024-09-10T23:36:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb186c00640 1 -- 172.21.15.94:0/780071839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1800c7d90 con 0x7fb18015f5c0 2024-09-10T23:36:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb17cc00640 1 -- 172.21.15.94:0/780071839 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17800aab0 con 0x7fb18015f5c0 2024-09-10T23:36:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb186c00640 1 -- 172.21.15.94:0/780071839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb180147130 con 0x7fb18015f5c0 2024-09-10T23:36:33.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.694+0000 7fb186c00640 1 -- 172.21.15.94:0/780071839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1800b15c0 con 0x7fb18015f5c0 2024-09-10T23:36:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.698+0000 7fb17cc00640 1 -- 172.21.15.94:0/780071839 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7fb17800ac50 con 0x7fb18015f5c0 2024-09-10T23:36:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.698+0000 7fb17cc00640 1 --2- 172.21.15.94:0/780071839 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb15c072940 0x7fb15c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.698+0000 7fb17cc00640 1 -- 172.21.15.94:0/780071839 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fb178093640 con 0x7fb18015f5c0 2024-09-10T23:36:33.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.698+0000 7fb17fe00640 1 --2- 172.21.15.94:0/780071839 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb15c072940 0x7fb15c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:33.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.698+0000 7fb17fe00640 1 --2- 172.21.15.94:0/780071839 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb15c072940 0x7fb15c074e00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb1700af000 tx=0x7fb1700b3040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:33.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.702+0000 7fb17cc00640 1 -- 172.21.15.94:0/780071839 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb17805e100 con 0x7fb18015f5c0 2024-09-10T23:36:33.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.902+0000 7fb186c00640 1 -- 172.21.15.94:0/780071839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1800b1a80 con 0x7fb18015f5c0 2024-09-10T23:36:33.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.902+0000 7fb17cc00640 1 -- 172.21.15.94:0/780071839 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5238 (secure 0 0 0) 0x7fb178065e20 con 0x7fb18015f5c0 2024-09-10T23:36:33.906 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:33.906 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":44,"btime":"2024-09-10T23:36:31:051584+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3320339747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3320339747},{"type":"v1","addr":"172.21.15.94:6834","nonce":3320339747}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3123300514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3123300514},{"type":"v1","addr":"172.21.15.94:6835","nonce":3123300514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/339523200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":339523200},{"type":"v1","addr":"172.21.15.181:6837","nonce":339523200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:36:19.404249+0000","modified":"2024-09-10T23:36:22.404999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195389":{"gid":195389,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/2999113739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2999113739},{"type":"v1","addr":"172.21.15.181:6836","nonce":2999113739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195389,"qdb_cluster":[195389]},"id":5}]} 2024-09-10T23:36:33.906 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 44 2024-09-10T23:36:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.906+0000 7fb186c00640 1 -- 172.21.15.94:0/780071839 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb15c072940 msgr2=0x7fb15c074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.906+0000 7fb186c00640 1 --2- 172.21.15.94:0/780071839 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb15c072940 0x7fb15c074e00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb1700af000 tx=0x7fb1700b3040 comp rx=0 tx=0).stop 2024-09-10T23:36:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.906+0000 7fb186c00640 1 -- 172.21.15.94:0/780071839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb18015f5c0 msgr2=0x7fb18006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.906+0000 7fb186c00640 1 --2- 172.21.15.94:0/780071839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb18015f5c0 0x7fb18006f3f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb17800c7d0 tx=0x7fb17800cc90 comp rx=0 tx=0).stop 2024-09-10T23:36:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.906+0000 7fb186c00640 1 -- 172.21.15.94:0/780071839 shutdown_connections 2024-09-10T23:36:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.906+0000 7fb186c00640 1 --2- 172.21.15.94:0/780071839 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fb15c072940 0x7fb15c074e00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.906+0000 7fb186c00640 1 --2- 172.21.15.94:0/780071839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb18015f5c0 0x7fb18006f3f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.906+0000 7fb186c00640 1 --2- 172.21.15.94:0/780071839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1800d05e0 0x7fb18006eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.906+0000 7fb186c00640 1 --2- 172.21.15.94:0/780071839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1800b2bb0 0x7fb180146bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.906+0000 7fb186c00640 1 -- 172.21.15.94:0/780071839 >> 172.21.15.94:0/780071839 conn(0x7fb1800d1c60 msgr2=0x7fb1800d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.906+0000 7fb186c00640 1 -- 172.21.15.94:0/780071839 shutdown_connections 2024-09-10T23:36:33.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:33.906+0000 7fb186c00640 1 -- 172.21.15.94:0/780071839 wait complete. 2024-09-10T23:36:33.928 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:36:34.092 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2024-09-10T23:36:34.092 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-10T23:36:34.092 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol71860 2024-09-10T23:36:34.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.234+0000 7fccfa800640 1 -- 172.21.15.94:0/1715997989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccf40cb560 msgr2=0x7fccf40700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:34.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.234+0000 7fccfa800640 1 --2- 172.21.15.94:0/1715997989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccf40cb560 0x7fccf40700c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009840 tx=0x7fcce001c820 comp rx=0 tx=0).stop 2024-09-10T23:36:34.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.234+0000 7fccfa800640 1 -- 172.21.15.94:0/1715997989 shutdown_connections 2024-09-10T23:36:34.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.234+0000 7fccfa800640 1 --2- 172.21.15.94:0/1715997989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccf40cb560 0x7fccf40700c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.234+0000 7fccfa800640 1 --2- 172.21.15.94:0/1715997989 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccf41517e0 0x7fccf40cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.234+0000 7fccfa800640 1 -- 172.21.15.94:0/1715997989 >> 172.21.15.94:0/1715997989 conn(0x7fccf40d0100 msgr2=0x7fccf40d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:34.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.234+0000 7fccfa800640 1 -- 172.21.15.94:0/1715997989 shutdown_connections 2024-09-10T23:36:34.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.234+0000 7fccfa800640 1 -- 172.21.15.94:0/1715997989 wait complete. 2024-09-10T23:36:34.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccfa800640 1 Processor -- start 2024-09-10T23:36:34.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccfa800640 1 -- start start 2024-09-10T23:36:34.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccfa800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccf4070600 0x7fccf41481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:34.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccfa800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccf40cb560 0x7fccf40c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:34.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccfa800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccf41517e0 0x7fccf40c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:34.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccfa800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fccf4167c20 con 0x7fccf4070600 2024-09-10T23:36:34.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccfa800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fccf4167aa0 con 0x7fccf41517e0 2024-09-10T23:36:34.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccfa800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fccf4167da0 con 0x7fccf40cb560 2024-09-10T23:36:34.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccf9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccf4070600 0x7fccf41481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:34.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccf9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccf4070600 0x7fccf41481c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48636/0 (socket says 172.21.15.94:48636) 2024-09-10T23:36:34.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccf9400640 1 -- 172.21.15.94:0/2308838764 learned_addr learned my addr 172.21.15.94:0/2308838764 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:34.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccf8a00640 1 --2- 172.21.15.94:0/2308838764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccf40cb560 0x7fccf40c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:34.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccf9e00640 1 --2- 172.21.15.94:0/2308838764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccf41517e0 0x7fccf40c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:34.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccf9400640 1 -- 172.21.15.94:0/2308838764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccf40cb560 msgr2=0x7fccf40c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:34.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccf9400640 1 --2- 172.21.15.94:0/2308838764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccf40cb560 0x7fccf40c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccf9400640 1 -- 172.21.15.94:0/2308838764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccf41517e0 msgr2=0x7fccf40c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:34.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccf9400640 1 --2- 172.21.15.94:0/2308838764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccf41517e0 0x7fccf40c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccf9400640 1 -- 172.21.15.94:0/2308838764 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf406a2a0 con 0x7fccf4070600 2024-09-10T23:36:34.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccf9400640 1 --2- 172.21.15.94:0/2308838764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccf4070600 0x7fccf41481c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fcce800d950 tx=0x7fcce800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:34.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccf1600640 1 -- 172.21.15.94:0/2308838764 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8017070 con 0x7fccf4070600 2024-09-10T23:36:34.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccf1600640 1 -- 172.21.15.94:0/2308838764 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcce8009d20 con 0x7fccf4070600 2024-09-10T23:36:34.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccf1600640 1 -- 172.21.15.94:0/2308838764 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce80106c0 con 0x7fccf4070600 2024-09-10T23:36:34.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccfa800640 1 -- 172.21.15.94:0/2308838764 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf41595e0 con 0x7fccf4070600 2024-09-10T23:36:34.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccfa800640 1 -- 172.21.15.94:0/2308838764 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf4159ba0 con 0x7fccf4070600 2024-09-10T23:36:34.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.238+0000 7fccfa800640 1 -- 172.21.15.94:0/2308838764 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf40700c0 con 0x7fccf4070600 2024-09-10T23:36:34.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.242+0000 7fccf1600640 1 -- 172.21.15.94:0/2308838764 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7fcce8010860 con 0x7fccf4070600 2024-09-10T23:36:34.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.242+0000 7fccf1600640 1 --2- 172.21.15.94:0/2308838764 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fccd4072940 0x7fccd4074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:34.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.242+0000 7fccf1600640 1 -- 172.21.15.94:0/2308838764 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fcce8093e60 con 0x7fccf4070600 2024-09-10T23:36:34.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.242+0000 7fccf8a00640 1 --2- 172.21.15.94:0/2308838764 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fccd4072940 0x7fccd4074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:34.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.242+0000 7fccf8a00640 1 --2- 172.21.15.94:0/2308838764 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fccd4072940 0x7fccd4074e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcce00af000 tx=0x7fcce001b060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:34.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.246+0000 7fccf1600640 1 -- 172.21.15.94:0/2308838764 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcce805eee0 con 0x7fccf4070600 2024-09-10T23:36:34.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.434+0000 7fccfa800640 1 -- 172.21.15.94:0/2308838764 --> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol71860", "target": ["mon-mgr", ""]}) -- 0x7fccf4148700 con 0x7fccd4072940 2024-09-10T23:36:34.444 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:36:34.442+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:34.444 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:36:34.442+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:34.444 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:36:34.442+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:34.444 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:36:34.442+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:34.444 INFO:tasks.ceph.mgr.x.smithi181.stderr:2024-09-10T23:36:34.442+0000 7fb908c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:34.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.470+0000 7fccf1600640 1 -- 172.21.15.94:0/2308838764 <== mgr.4123 v2:172.21.15.181:6800/3310396812 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcce00042a0 con 0x7fccd4072940 2024-09-10T23:36:34.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.474+0000 7fccfa800640 1 -- 172.21.15.94:0/2308838764 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fccd4072940 msgr2=0x7fccd4074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:34.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.474+0000 7fccfa800640 1 --2- 172.21.15.94:0/2308838764 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fccd4072940 0x7fccd4074e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcce00af000 tx=0x7fcce001b060 comp rx=0 tx=0).stop 2024-09-10T23:36:34.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.474+0000 7fccfa800640 1 -- 172.21.15.94:0/2308838764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccf4070600 msgr2=0x7fccf41481c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:34.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.474+0000 7fccfa800640 1 --2- 172.21.15.94:0/2308838764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccf4070600 0x7fccf41481c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fcce800d950 tx=0x7fcce800de10 comp rx=0 tx=0).stop 2024-09-10T23:36:34.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.474+0000 7fccfa800640 1 -- 172.21.15.94:0/2308838764 shutdown_connections 2024-09-10T23:36:34.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.474+0000 7fccfa800640 1 --2- 172.21.15.94:0/2308838764 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fccd4072940 0x7fccd4074e00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.474+0000 7fccfa800640 1 --2- 172.21.15.94:0/2308838764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccf41517e0 0x7fccf40c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.474+0000 7fccfa800640 1 --2- 172.21.15.94:0/2308838764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccf40cb560 0x7fccf40c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.474+0000 7fccfa800640 1 --2- 172.21.15.94:0/2308838764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fccf4070600 0x7fccf41481c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.474+0000 7fccfa800640 1 -- 172.21.15.94:0/2308838764 >> 172.21.15.94:0/2308838764 conn(0x7fccf40d0100 msgr2=0x7fccf415a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:34.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.474+0000 7fccfa800640 1 -- 172.21.15.94:0/2308838764 shutdown_connections 2024-09-10T23:36:34.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.474+0000 7fccfa800640 1 -- 172.21.15.94:0/2308838764 wait complete. 2024-09-10T23:36:34.495 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:34.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.634+0000 7f4f0ca00640 1 -- 172.21.15.94:0/3367758671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f08071a20 msgr2=0x7f4f0815f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:34.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.634+0000 7f4f0ca00640 1 --2- 172.21.15.94:0/3367758671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f08071a20 0x7f4f0815f610 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009840 tx=0x7f4ef001c790 comp rx=0 tx=0).stop 2024-09-10T23:36:34.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.634+0000 7f4f0ca00640 1 -- 172.21.15.94:0/3367758671 shutdown_connections 2024-09-10T23:36:34.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.634+0000 7f4f0ca00640 1 --2- 172.21.15.94:0/3367758671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f08071a20 0x7f4f0815f610 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.634+0000 7f4f0ca00640 1 --2- 172.21.15.94:0/3367758671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f0806cc20 0x7f4f0806d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.634+0000 7f4f0ca00640 1 -- 172.21.15.94:0/3367758671 >> 172.21.15.94:0/3367758671 conn(0x7f4f080d0b70 msgr2=0x7f4f080d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:34.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.634+0000 7f4f0ca00640 1 -- 172.21.15.94:0/3367758671 shutdown_connections 2024-09-10T23:36:34.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.634+0000 7f4f0ca00640 1 -- 172.21.15.94:0/3367758671 wait complete. 2024-09-10T23:36:34.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.634+0000 7f4f0ca00640 1 Processor -- start 2024-09-10T23:36:34.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.634+0000 7f4f0ca00640 1 -- start start 2024-09-10T23:36:34.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4f0ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f0806cc20 0x7f4f08144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:34.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4f0ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f08071a20 0x7f4f081fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:34.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4f0ca00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f080d1690 0x7f4f080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:34.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4f0ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08169130 con 0x7f4f08071a20 2024-09-10T23:36:34.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4f0ca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08168fb0 con 0x7f4f0806cc20 2024-09-10T23:36:34.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4f0ca00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f4f081692b0 con 0x7f4f080d1690 2024-09-10T23:36:34.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4efd400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f08071a20 0x7f4f081fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:34.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4efd400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f08071a20 0x7f4f081fbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48652/0 (socket says 172.21.15.94:48652) 2024-09-10T23:36:34.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4efd400640 1 -- 172.21.15.94:0/619505114 learned_addr learned my addr 172.21.15.94:0/619505114 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:34.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4f05e00640 1 --2- 172.21.15.94:0/619505114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f0806cc20 0x7f4f08144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:34.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4f06800640 1 --2- 172.21.15.94:0/619505114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f080d1690 0x7f4f080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:34.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4efd400640 1 -- 172.21.15.94:0/619505114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f080d1690 msgr2=0x7f4f080b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:34.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4efd400640 1 --2- 172.21.15.94:0/619505114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f080d1690 0x7f4f080b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4efd400640 1 -- 172.21.15.94:0/619505114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f0806cc20 msgr2=0x7f4f08144170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:34.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4efd400640 1 --2- 172.21.15.94:0/619505114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f0806cc20 0x7f4f08144170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4efd400640 1 -- 172.21.15.94:0/619505114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f081f7dd0 con 0x7f4f08071a20 2024-09-10T23:36:34.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4f06800640 1 --2- 172.21.15.94:0/619505114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f080d1690 0x7f4f080b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:34.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4efd400640 1 --2- 172.21.15.94:0/619505114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f08071a20 0x7f4f081fbb70 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4ef001d030 tx=0x7f4ef00a6f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:34.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4efea00640 1 -- 172.21.15.94:0/619505114 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef00ac070 con 0x7f4f08071a20 2024-09-10T23:36:34.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4efea00640 1 -- 172.21.15.94:0/619505114 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ef0005130 con 0x7f4f08071a20 2024-09-10T23:36:34.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4efea00640 1 -- 172.21.15.94:0/619505114 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0005560 con 0x7f4f08071a20 2024-09-10T23:36:34.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4f0ca00640 1 -- 172.21.15.94:0/619505114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f081f80b0 con 0x7f4f08071a20 2024-09-10T23:36:34.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4f0ca00640 1 -- 172.21.15.94:0/619505114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f081f8670 con 0x7f4f08071a20 2024-09-10T23:36:34.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4f0ca00640 1 -- 172.21.15.94:0/619505114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f0806d000 con 0x7f4f08071a20 2024-09-10T23:36:34.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.638+0000 7f4efea00640 1 -- 172.21.15.94:0/619505114 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f4ef00046d0 con 0x7f4f08071a20 2024-09-10T23:36:34.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.642+0000 7f4efea00640 1 --2- 172.21.15.94:0/619505114 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4ee4072940 0x7f4ee4074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:34.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.642+0000 7f4efea00640 1 -- 172.21.15.94:0/619505114 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f4ef00dae00 con 0x7f4f08071a20 2024-09-10T23:36:34.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.642+0000 7f4f05e00640 1 --2- 172.21.15.94:0/619505114 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4ee4072940 0x7f4ee4074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:34.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.642+0000 7f4f05e00640 1 --2- 172.21.15.94:0/619505114 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4ee4072940 0x7f4ee4074e00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009be0 tx=0x7f4ef80091e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:34.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.642+0000 7f4efea00640 1 -- 172.21.15.94:0/619505114 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4ef0016410 con 0x7f4f08071a20 2024-09-10T23:36:34.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.838+0000 7f4f0ca00640 1 -- 172.21.15.94:0/619505114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f08156410 con 0x7f4f08071a20 2024-09-10T23:36:34.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.838+0000 7f4efea00640 1 -- 172.21.15.94:0/619505114 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5238 (secure 0 0 0) 0x7f4ef00f9050 con 0x7f4f08071a20 2024-09-10T23:36:34.841 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:34.842 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":44,"btime":"2024-09-10T23:36:31:051584+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3320339747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3320339747},{"type":"v1","addr":"172.21.15.94:6834","nonce":3320339747}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3123300514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3123300514},{"type":"v1","addr":"172.21.15.94:6835","nonce":3123300514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/339523200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":339523200},{"type":"v1","addr":"172.21.15.181:6837","nonce":339523200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:36:19.404249+0000","modified":"2024-09-10T23:36:22.404999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195389":{"gid":195389,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/2999113739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2999113739},{"type":"v1","addr":"172.21.15.181:6836","nonce":2999113739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195389,"qdb_cluster":[195389]},"id":5}]} 2024-09-10T23:36:34.842 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 44 2024-09-10T23:36:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.842+0000 7f4f0ca00640 1 -- 172.21.15.94:0/619505114 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4ee4072940 msgr2=0x7f4ee4074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.842+0000 7f4f0ca00640 1 --2- 172.21.15.94:0/619505114 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4ee4072940 0x7f4ee4074e00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009be0 tx=0x7f4ef80091e0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.842+0000 7f4f0ca00640 1 -- 172.21.15.94:0/619505114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f08071a20 msgr2=0x7f4f081fbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.842+0000 7f4f0ca00640 1 --2- 172.21.15.94:0/619505114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f08071a20 0x7f4f081fbb70 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4ef001d030 tx=0x7f4ef00a6f90 comp rx=0 tx=0).stop 2024-09-10T23:36:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.842+0000 7f4f0ca00640 1 -- 172.21.15.94:0/619505114 shutdown_connections 2024-09-10T23:36:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.842+0000 7f4f0ca00640 1 --2- 172.21.15.94:0/619505114 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f4ee4072940 0x7f4ee4074e00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.842+0000 7f4f0ca00640 1 --2- 172.21.15.94:0/619505114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f080d1690 0x7f4f080b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.842+0000 7f4f0ca00640 1 --2- 172.21.15.94:0/619505114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f08071a20 0x7f4f081fbb70 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.842+0000 7f4f0ca00640 1 --2- 172.21.15.94:0/619505114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f0806cc20 0x7f4f08144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.842+0000 7f4f0ca00640 1 -- 172.21.15.94:0/619505114 >> 172.21.15.94:0/619505114 conn(0x7f4f080d0b70 msgr2=0x7f4f08165ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.842+0000 7f4f0ca00640 1 -- 172.21.15.94:0/619505114 shutdown_connections 2024-09-10T23:36:34.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:34.842+0000 7f4f0ca00640 1 -- 172.21.15.94:0/619505114 wait complete. 2024-09-10T23:36:34.862 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:36:34.893 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:36:34.897 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:36:35.025 INFO:teuthology.orchestra.run.smithi181.stdout:[{"id":6118,"entity":{"name":{"type":"client","num":6118},"addr":{"type":"any","addr":"172.21.15.181:0","nonce":1922952906}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.55999426299999999,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.972061108276474,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6118 172.21.15.181:0/1922952906","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8f219c476cb4f26fc5719c661c0323719a45cb30","ceph_version":"ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)","entity_id":"x","hostname":"smithi181","pid":"28823","root":"/"}}] 2024-09-10T23:36:35.026 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi181", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28823", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.181:0", "nonce": 1922952906, "type": "any" }, "name": { "num": 6118, "type": "client" } }, "id": 6118, "inst": "client.6118 172.21.15.181:0/1922952906", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.972061108276474 }, "state": "open", "uptime": 0.559994263 } ] 2024-09-10T23:36:35.026 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-10T23:36:35.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.166+0000 7f0b95c00640 1 -- 172.21.15.94:0/2978999026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b740039f0 msgr2=0x7f0b74003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:35.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.166+0000 7f0b95c00640 1 --2- 172.21.15.94:0/2978999026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b740039f0 0x7f0b74003dd0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000bb10 tx=0x7f0b8001cd70 comp rx=0 tx=0).stop 2024-09-10T23:36:35.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.166+0000 7f0b8f400640 1 -- 172.21.15.94:0/2978999026 reap_dead start 2024-09-10T23:36:35.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b95c00640 1 -- 172.21.15.94:0/2978999026 shutdown_connections 2024-09-10T23:36:35.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b95c00640 1 -- 172.21.15.94:0/2978999026 >> 172.21.15.94:0/2978999026 conn(0x7f0b9006cdf0 msgr2=0x7f0b9006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:35.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b95c00640 1 -- 172.21.15.94:0/2978999026 shutdown_connections 2024-09-10T23:36:35.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b95c00640 1 -- 172.21.15.94:0/2978999026 wait complete. 2024-09-10T23:36:35.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b95c00640 1 Processor -- start 2024-09-10T23:36:35.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b95c00640 1 -- start start 2024-09-10T23:36:35.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b95c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b740039f0 0x7f0b9013a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:35.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b95c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b900b0b80 0x7f0b900b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:35.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b95c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006f6f0 0x7f0b9006fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b95c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b90167be0 con 0x7f0b9006f6f0 2024-09-10T23:36:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b95c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0b90167a60 con 0x7f0b740039f0 2024-09-10T23:36:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b95c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0b90167d60 con 0x7f0b900b0b80 2024-09-10T23:36:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b8f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006f6f0 0x7f0b9006fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b87e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b900b0b80 0x7f0b900b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b8ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b740039f0 0x7f0b9013a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b8f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006f6f0 0x7f0b9006fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48666/0 (socket says 172.21.15.94:48666) 2024-09-10T23:36:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b8ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b740039f0 0x7f0b9013a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:35892/0 (socket says 172.21.15.94:35892) 2024-09-10T23:36:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b8f400640 1 -- 172.21.15.94:0/2750809858 learned_addr learned my addr 172.21.15.94:0/2750809858 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b87e00640 1 -- 172.21.15.94:0/2750809858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b740039f0 msgr2=0x7f0b9013a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b87e00640 1 --2- 172.21.15.94:0/2750809858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b740039f0 0x7f0b9013a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b87e00640 1 -- 172.21.15.94:0/2750809858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006f6f0 msgr2=0x7f0b9006fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b87e00640 1 --2- 172.21.15.94:0/2750809858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006f6f0 0x7f0b9006fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:35.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b87e00640 1 -- 172.21.15.94:0/2750809858 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b900b1640 con 0x7f0b900b0b80 2024-09-10T23:36:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b8ea00640 1 --2- 172.21.15.94:0/2750809858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b740039f0 0x7f0b9013a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:36:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.170+0000 7f0b8f400640 1 --2- 172.21.15.94:0/2750809858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006f6f0 0x7f0b9006fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:36:35.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.174+0000 7f0b87e00640 1 --2- 172.21.15.94:0/2750809858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b900b0b80 0x7f0b900b1000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0b780011b0 tx=0x7f0b78001720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.174+0000 7f0b86a00640 1 -- 172.21.15.94:0/2750809858 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78015070 con 0x7f0b900b0b80 2024-09-10T23:36:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.174+0000 7f0b86a00640 1 -- 172.21.15.94:0/2750809858 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b78011900 con 0x7f0b900b0b80 2024-09-10T23:36:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.174+0000 7f0b95c00640 1 -- 172.21.15.94:0/2750809858 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b900b1920 con 0x7f0b900b0b80 2024-09-10T23:36:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.174+0000 7f0b86a00640 1 -- 172.21.15.94:0/2750809858 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78011d50 con 0x7f0b900b0b80 2024-09-10T23:36:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.174+0000 7f0b95c00640 1 -- 172.21.15.94:0/2750809858 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b900b1ee0 con 0x7f0b900b0b80 2024-09-10T23:36:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.174+0000 7f0b95c00640 1 -- 172.21.15.94:0/2750809858 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b900cfcb0 con 0x7f0b900b0b80 2024-09-10T23:36:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.174+0000 7f0b86a00640 1 -- 172.21.15.94:0/2750809858 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 11) ==== 96700+0+0 (secure 0 0 0) 0x7f0b780103c0 con 0x7f0b900b0b80 2024-09-10T23:36:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.174+0000 7f0b86a00640 1 --2- 172.21.15.94:0/2750809858 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b6c072940 0x7f0b6c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.174+0000 7f0b86a00640 1 -- 172.21.15.94:0/2750809858 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f0b780948f0 con 0x7f0b900b0b80 2024-09-10T23:36:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.174+0000 7f0b8ea00640 1 --2- 172.21.15.94:0/2750809858 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b6c072940 0x7f0b6c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.178+0000 7f0b8ea00640 1 --2- 172.21.15.94:0/2750809858 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b6c072940 0x7f0b6c074e00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0b800b0000 tx=0x7f0b800a7040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:35.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.178+0000 7f0b86a00640 1 -- 172.21.15.94:0/2750809858 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0b7805f320 con 0x7f0b900b0b80 2024-09-10T23:36:35.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.386+0000 7f0b95c00640 1 -- 172.21.15.94:0/2750809858 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f0b9014d190 con 0x7f0b900b0b80 2024-09-10T23:36:35.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.394+0000 7f0b86a00640 1 -- 172.21.15.94:0/2750809858 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v11) ==== 81+0+503250 (secure 0 0 0) 0x7f0b78063d50 con 0x7f0b900b0b80 2024-09-10T23:36:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:35.400 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:36:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 11, 2024-09-10T23:36:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "active_gid": 4123, 2024-09-10T23:36:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "active_name": "x", 2024-09-10T23:36:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "active_addrs": { 2024-09-10T23:36:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-10T23:36:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-10T23:36:35.400 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.181:6800", 2024-09-10T23:36:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3310396812 2024-09-10T23:36:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:36:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.181:6801", 2024-09-10T23:36:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3310396812 2024-09-10T23:36:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:36:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "active_addr": "172.21.15.181:6801/3310396812", 2024-09-10T23:36:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "active_change": "2024-09-10T23:32:21.329421+0000", 2024-09-10T23:36:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "active_mgr_features": 4540719139924082687, 2024-09-10T23:36:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "available": true, 2024-09-10T23:36:35.401 INFO:teuthology.orchestra.run.smithi094.stdout: "standbys": [ 2024-09-10T23:36:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 4146, 2024-09-10T23:36:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "y", 2024-09-10T23:36:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_features": 4540719139924082687, 2024-09-10T23:36:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-09-10T23:36:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-09-10T23:36:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-10T23:36:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-10T23:36:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.402 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-10T23:36:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T23:36:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.403 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.404 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.405 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.406 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.407 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.408 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-09-10T23:36:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-09-10T23:36:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.409 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T23:36:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-09-10T23:36:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-09-10T23:36:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-09-10T23:36:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.410 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-09-10T23:36:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-09-10T23:36:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-09-10T23:36:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.411 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-09-10T23:36:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-09-10T23:36:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-09-10T23:36:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.412 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-09-10T23:36:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-09-10T23:36:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-09-10T23:36:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.413 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-09-10T23:36:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-09-10T23:36:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-09-10T23:36:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-09-10T23:36:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.414 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-09-10T23:36:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-09-10T23:36:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-09-10T23:36:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.415 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T23:36:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-09-10T23:36:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-09-10T23:36:35.416 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-09-10T23:36:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.417 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.418 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-09-10T23:36:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-09-10T23:36:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-09-10T23:36:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.419 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T23:36:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-09-10T23:36:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-09-10T23:36:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.420 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-09-10T23:36:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T23:36:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:36:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-09-10T23:36:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-09-10T23:36:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-10T23:36:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.421 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-09-10T23:36:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-09-10T23:36:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T23:36:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:36:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-09-10T23:36:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-09-10T23:36:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-10T23:36:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.422 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-09-10T23:36:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-10T23:36:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-09-10T23:36:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T23:36:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-09-10T23:36:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-09-10T23:36:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.423 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T23:36:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T23:36:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T23:36:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-09-10T23:36:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-09-10T23:36:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-10T23:36:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.424 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-09-10T23:36:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-09-10T23:36:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-09-10T23:36:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-09-10T23:36:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T23:36:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-09-10T23:36:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-09-10T23:36:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.425 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-09-10T23:36:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T23:36:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:36:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-09-10T23:36:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-09-10T23:36:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-10T23:36:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.426 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-09-10T23:36:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-09-10T23:36:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T23:36:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:36:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.427 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.428 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.429 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.431 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-09-10T23:36:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-09-10T23:36:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-10T23:36:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T23:36:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-09-10T23:36:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-09-10T23:36:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-09-10T23:36:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-09-10T23:36:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-09-10T23:36:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-09-10T23:36:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-09-10T23:36:35.434 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-09-10T23:36:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-09-10T23:36:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-09-10T23:36:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-09-10T23:36:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.435 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T23:36:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-10T23:36:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-10T23:36:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.436 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-10T23:36:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T23:36:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.437 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.438 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.439 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-09-10T23:36:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-09-10T23:36:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-10T23:36:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-10T23:36:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T23:36:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T23:36:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.440 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-09-10T23:36:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-09-10T23:36:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-10T23:36:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-10T23:36:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T23:36:35.441 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-09-10T23:36:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-09-10T23:36:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-09-10T23:36:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-10T23:36:35.442 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-09-10T23:36:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T23:36:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-09-10T23:36:35.443 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-09-10T23:36:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-09-10T23:36:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T23:36:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.444 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-09-10T23:36:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-09-10T23:36:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-09-10T23:36:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T23:36:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.445 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-09-10T23:36:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-09-10T23:36:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T23:36:35.446 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T23:36:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-09-10T23:36:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-09-10T23:36:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-10T23:36:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.447 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T23:36:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-09-10T23:36:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T23:36:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-10T23:36:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-09-10T23:36:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T23:36:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-09-10T23:36:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-09-10T23:36:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-09-10T23:36:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-09-10T23:36:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-09-10T23:36:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T23:36:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-09-10T23:36:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-09-10T23:36:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T23:36:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-09-10T23:36:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-09-10T23:36:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T23:36:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-09-10T23:36:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-09-10T23:36:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T23:36:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-09-10T23:36:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-09-10T23:36:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T23:36:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-10T23:36:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-09-10T23:36:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-09-10T23:36:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T23:36:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-09-10T23:36:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-09-10T23:36:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-09-10T23:36:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T23:36:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-09-10T23:36:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-09-10T23:36:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-09-10T23:36:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T23:36:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-10T23:36:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-09-10T23:36:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-09-10T23:36:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T23:36:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-09-10T23:36:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-09-10T23:36:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-09-10T23:36:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T23:36:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-09-10T23:36:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-09-10T23:36:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-09-10T23:36:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T23:36:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-09-10T23:36:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-09-10T23:36:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-09-10T23:36:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T23:36:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-09-10T23:36:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-09-10T23:36:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-09-10T23:36:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T23:36:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-09-10T23:36:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-09-10T23:36:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-09-10T23:36:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T23:36:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-09-10T23:36:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nginx": { 2024-09-10T23:36:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nginx", 2024-09-10T23:36:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T23:36:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nginx container image", 2024-09-10T23:36:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-09-10T23:36:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-09-10T23:36:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T23:36:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-10T23:36:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-09-10T23:36:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-09-10T23:36:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T23:36:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-09-10T23:36:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_oauth2_proxy": { 2024-09-10T23:36:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T23:36:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T23:36:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "oauth2-proxy container image", 2024-09-10T23:36:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-09-10T23:36:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-09-10T23:36:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T23:36:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-10T23:36:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-09-10T23:36:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-09-10T23:36:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T23:36:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-09-10T23:36:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_samba": { 2024-09-10T23:36:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_samba", 2024-09-10T23:36:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T23:36:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Samba/SMB container image", 2024-09-10T23:36:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_samba_metrics": { 2024-09-10T23:36:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_samba_metrics", 2024-09-10T23:36:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T23:36:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T23:36:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-09-10T23:36:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-09-10T23:36:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T23:36:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-09-10T23:36:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-09-10T23:36:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-09-10T23:36:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T23:36:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-09-10T23:36:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-09-10T23:36:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-10T23:36:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T23:36:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-09-10T23:36:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T23:36:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-09-10T23:36:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T23:36:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-09-10T23:36:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-09-10T23:36:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-09-10T23:36:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T23:36:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-09-10T23:36:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-09-10T23:36:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-09-10T23:36:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-09-10T23:36:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-09-10T23:36:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-09-10T23:36:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T23:36:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-09-10T23:36:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-09-10T23:36:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-10T23:36:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-09-10T23:36:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "grafana_dashboards_path": { 2024-09-10T23:36:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "grafana_dashboards_path", 2024-09-10T23:36:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T23:36:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T23:36:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-09-10T23:36:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-09-10T23:36:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-10T23:36:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-09-10T23:36:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-09-10T23:36:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-09-10T23:36:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T23:36:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-09-10T23:36:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-09-10T23:36:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T23:36:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-09-10T23:36:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-09-10T23:36:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T23:36:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T23:36:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.490 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.491 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T23:36:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T23:36:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.492 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T23:36:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T23:36:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T23:36:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.493 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-09-10T23:36:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T23:36:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-09-10T23:36:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-09-10T23:36:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.494 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-10T23:36:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-09-10T23:36:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-09-10T23:36:35.495 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-09-10T23:36:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-10T23:36:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T23:36:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.496 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-09-10T23:36:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-09-10T23:36:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-09-10T23:36:35.497 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-09-10T23:36:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-09-10T23:36:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-09-10T23:36:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.498 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-09-10T23:36:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-09-10T23:36:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T23:36:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-09-10T23:36:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-09-10T23:36:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.499 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-09-10T23:36:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T23:36:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-09-10T23:36:35.500 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-09-10T23:36:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T23:36:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T23:36:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-09-10T23:36:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-09-10T23:36:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T23:36:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.502 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-09-10T23:36:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-09-10T23:36:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T23:36:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.503 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-09-10T23:36:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-09-10T23:36:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T23:36:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-09-10T23:36:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-09-10T23:36:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T23:36:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-09-10T23:36:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-09-10T23:36:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T23:36:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-09-10T23:36:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-09-10T23:36:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-09-10T23:36:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-09-10T23:36:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.508 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-09-10T23:36:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-09-10T23:36:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T23:36:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.509 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-09-10T23:36:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-09-10T23:36:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T23:36:35.510 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-09-10T23:36:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-09-10T23:36:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T23:36:35.511 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-09-10T23:36:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-09-10T23:36:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T23:36:35.512 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-09-10T23:36:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-09-10T23:36:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.513 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T23:36:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-09-10T23:36:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-09-10T23:36:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.514 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T23:36:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-09-10T23:36:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.515 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.516 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.517 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.518 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.519 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-09-10T23:36:35.520 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-09-10T23:36:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-09-10T23:36:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T23:36:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.521 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-09-10T23:36:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-09-10T23:36:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-09-10T23:36:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T23:36:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.523 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-09-10T23:36:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T23:36:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T23:36:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.524 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-10T23:36:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T23:36:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T23:36:35.525 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T23:36:35.526 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T23:36:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-09-10T23:36:35.527 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T23:36:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T23:36:35.528 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T23:36:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.529 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T23:36:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T23:36:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T23:36:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T23:36:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T23:36:35.531 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T23:36:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.532 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T23:36:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T23:36:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.533 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T23:36:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T23:36:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.534 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T23:36:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T23:36:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.535 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T23:36:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T23:36:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.536 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T23:36:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T23:36:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.537 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T23:36:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T23:36:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.538 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T23:36:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T23:36:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-09-10T23:36:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.539 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T23:36:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T23:36:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.540 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-09-10T23:36:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-09-10T23:36:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.541 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T23:36:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T23:36:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-09-10T23:36:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.542 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T23:36:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T23:36:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.543 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T23:36:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T23:36:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.544 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T23:36:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T23:36:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.545 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T23:36:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T23:36:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.546 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T23:36:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T23:36:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.547 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "[]", 2024-09-10T23:36:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T23:36:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T23:36:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.548 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "{}", 2024-09-10T23:36:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T23:36:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T23:36:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.549 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T23:36:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T23:36:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.550 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T23:36:35.551 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T23:36:35.552 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.552 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.552 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.552 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.553 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T23:36:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T23:36:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T23:36:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T23:36:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.555 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T23:36:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T23:36:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.556 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T23:36:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T23:36:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.557 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T23:36:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T23:36:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.558 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T23:36:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T23:36:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.559 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T23:36:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T23:36:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.560 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T23:36:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T23:36:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.561 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T23:36:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T23:36:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T23:36:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.562 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-10T23:36:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T23:36:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T23:36:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.563 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-09-10T23:36:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T23:36:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T23:36:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.564 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-09-10T23:36:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T23:36:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T23:36:35.565 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T23:36:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T23:36:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-09-10T23:36:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T23:36:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T23:36:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T23:36:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T23:36:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T23:36:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T23:36:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T23:36:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T23:36:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T23:36:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T23:36:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-10T23:36:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T23:36:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T23:36:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-10T23:36:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-09-10T23:36:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-09-10T23:36:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-09-10T23:36:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-09-10T23:36:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-09-10T23:36:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-09-10T23:36:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-09-10T23:36:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-09-10T23:36:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-09-10T23:36:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-09-10T23:36:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-09-10T23:36:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-09-10T23:36:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-09-10T23:36:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-09-10T23:36:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-09-10T23:36:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-09-10T23:36:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T23:36:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-09-10T23:36:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-09-10T23:36:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-09-10T23:36:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-09-10T23:36:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-09-10T23:36:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-09-10T23:36:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-09-10T23:36:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-09-10T23:36:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-09-10T23:36:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-09-10T23:36:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-09-10T23:36:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-09-10T23:36:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-09-10T23:36:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-09-10T23:36:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-09-10T23:36:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-09-10T23:36:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-09-10T23:36:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-09-10T23:36:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-09-10T23:36:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-09-10T23:36:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-09-10T23:36:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-09-10T23:36:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-09-10T23:36:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-09-10T23:36:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-09-10T23:36:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-09-10T23:36:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-09-10T23:36:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-09-10T23:36:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-09-10T23:36:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T23:36:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-09-10T23:36:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-09-10T23:36:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-09-10T23:36:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T23:36:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-09-10T23:36:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-09-10T23:36:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-09-10T23:36:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-09-10T23:36:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-09-10T23:36:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-09-10T23:36:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-09-10T23:36:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T23:36:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-09-10T23:36:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-09-10T23:36:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T23:36:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-10T23:36:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-10T23:36:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-10T23:36:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T23:36:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-09-10T23:36:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-09-10T23:36:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-09-10T23:36:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T23:36:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-09-10T23:36:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-09-10T23:36:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-09-10T23:36:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-09-10T23:36:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-09-10T23:36:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-09-10T23:36:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-09-10T23:36:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-10T23:36:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-10T23:36:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-10T23:36:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-09-10T23:36:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-09-10T23:36:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-09-10T23:36:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-09-10T23:36:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-09-10T23:36:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-09-10T23:36:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T23:36:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-09-10T23:36:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-09-10T23:36:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-09-10T23:36:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T23:36:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-09-10T23:36:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-09-10T23:36:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-09-10T23:36:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-10T23:36:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-10T23:36:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-09-10T23:36:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-09-10T23:36:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T23:36:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-09-10T23:36:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-09-10T23:36:35.632 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-09-10T23:36:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-09-10T23:36:35.633 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-09-10T23:36:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-09-10T23:36:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-09-10T23:36:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.634 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.635 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-09-10T23:36:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-09-10T23:36:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-09-10T23:36:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:36:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-09-10T23:36:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-09-10T23:36:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-10T23:36:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-10T23:36:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-09-10T23:36:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T23:36:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.637 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-09-10T23:36:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-09-10T23:36:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-09-10T23:36:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.638 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-09-10T23:36:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-09-10T23:36:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-09-10T23:36:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:36:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.639 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-09-10T23:36:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.640 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.641 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.642 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.643 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.646 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-09-10T23:36:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.647 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.648 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.649 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.650 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.651 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.652 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-09-10T23:36:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-09-10T23:36:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-09-10T23:36:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.654 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-09-10T23:36:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-09-10T23:36:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T23:36:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.655 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.656 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.657 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.658 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.659 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-09-10T23:36:35.660 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-09-10T23:36:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-09-10T23:36:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T23:36:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-09-10T23:36:35.661 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-09-10T23:36:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-09-10T23:36:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-09-10T23:36:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-09-10T23:36:35.662 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-09-10T23:36:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-09-10T23:36:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T23:36:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.663 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-09-10T23:36:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-09-10T23:36:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-09-10T23:36:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.664 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-09-10T23:36:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-09-10T23:36:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-09-10T23:36:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T23:36:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T23:36:35.666 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-09-10T23:36:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.667 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.668 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.669 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.670 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.671 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.672 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.673 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-09-10T23:36:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.674 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.675 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.676 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.677 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.678 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.679 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.680 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-09-10T23:36:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-09-10T23:36:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-09-10T23:36:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.681 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T23:36:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.682 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.683 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.684 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.685 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.686 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-09-10T23:36:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-09-10T23:36:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.687 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-09-10T23:36:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-09-10T23:36:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-09-10T23:36:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-09-10T23:36:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.689 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-09-10T23:36:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.690 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.691 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.692 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.693 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.694 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.695 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-09-10T23:36:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.697 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.698 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.700 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.701 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.702 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-09-10T23:36:35.703 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.704 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.705 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.706 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.707 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.708 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-10T23:36:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-10T23:36:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-10T23:36:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.709 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.710 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-09-10T23:36:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-09-10T23:36:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-10T23:36:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.711 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-09-10T23:36:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-09-10T23:36:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-09-10T23:36:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T23:36:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-09-10T23:36:35.712 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-09-10T23:36:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-09-10T23:36:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T23:36:35.713 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-09-10T23:36:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-09-10T23:36:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.714 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.715 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.716 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.718 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.719 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-09-10T23:36:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-09-10T23:36:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.720 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-09-10T23:36:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-09-10T23:36:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-10T23:36:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-10T23:36:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.721 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-10T23:36:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-09-10T23:36:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.722 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-09-10T23:36:35.723 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-09-10T23:36:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-09-10T23:36:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.724 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-09-10T23:36:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-09-10T23:36:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T23:36:35.725 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T23:36:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.726 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.727 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.728 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.729 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.730 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-09-10T23:36:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-09-10T23:36:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.731 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T23:36:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T23:36:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-09-10T23:36:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.732 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-09-10T23:36:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-09-10T23:36:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-10T23:36:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-09-10T23:36:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.733 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-09-10T23:36:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-09-10T23:36:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-09-10T23:36:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.734 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T23:36:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-09-10T23:36:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-09-10T23:36:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-09-10T23:36:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.735 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T23:36:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.736 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-09-10T23:36:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-09-10T23:36:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-09-10T23:36:35.737 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-09-10T23:36:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-09-10T23:36:35.738 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-09-10T23:36:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-09-10T23:36:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-09-10T23:36:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-09-10T23:36:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-09-10T23:36:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-09-10T23:36:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-09-10T23:36:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-09-10T23:36:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.740 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-09-10T23:36:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.741 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.742 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.743 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.744 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.745 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-09-10T23:36:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-09-10T23:36:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.746 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-10T23:36:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-09-10T23:36:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T23:36:35.747 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.748 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-09-10T23:36:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-09-10T23:36:35.749 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-09-10T23:36:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-09-10T23:36:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-09-10T23:36:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.751 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-09-10T23:36:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-09-10T23:36:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.752 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.753 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.755 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.756 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "max_requests": { 2024-09-10T23:36:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_requests", 2024-09-10T23:36:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.758 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-09-10T23:36:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T23:36:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-09-10T23:36:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-09-10T23:36:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.759 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-09-10T23:36:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-09-10T23:36:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.760 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.761 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-09-10T23:36:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.763 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.764 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.765 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.766 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.767 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T23:36:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T23:36:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-10T23:36:35.768 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T23:36:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.769 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-09-10T23:36:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.770 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.771 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.772 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.773 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.774 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.775 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-09-10T23:36:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-09-10T23:36:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.776 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-09-10T23:36:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-09-10T23:36:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.777 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-09-10T23:36:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-09-10T23:36:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-09-10T23:36:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.778 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-09-10T23:36:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-09-10T23:36:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.779 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-09-10T23:36:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-09-10T23:36:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-10T23:36:35.780 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-09-10T23:36:35.781 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-09-10T23:36:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.782 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-09-10T23:36:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-09-10T23:36:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-09-10T23:36:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-09-10T23:36:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.784 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-09-10T23:36:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-09-10T23:36:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-10T23:36:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-09-10T23:36:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.785 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.786 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-09-10T23:36:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-09-10T23:36:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.787 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-09-10T23:36:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-09-10T23:36:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.788 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-09-10T23:36:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-09-10T23:36:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.789 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-09-10T23:36:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-09-10T23:36:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-09-10T23:36:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.790 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T23:36:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-09-10T23:36:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-09-10T23:36:35.791 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-09-10T23:36:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.792 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.793 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.794 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.795 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.796 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.798 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-09-10T23:36:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.799 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.800 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.801 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.802 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.803 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.804 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.805 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-09-10T23:36:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.806 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.807 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.809 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.810 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.811 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.812 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-09-10T23:36:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-09-10T23:36:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-09-10T23:36:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T23:36:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.813 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-10T23:36:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-10T23:36:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-09-10T23:36:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.814 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.815 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.816 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.818 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.819 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.820 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.821 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-09-10T23:36:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-09-10T23:36:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-09-10T23:36:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T23:36:35.822 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-09-10T23:36:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-09-10T23:36:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.823 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T23:36:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-09-10T23:36:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-09-10T23:36:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.824 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T23:36:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-09-10T23:36:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-09-10T23:36:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.825 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T23:36:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-09-10T23:36:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-09-10T23:36:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.826 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T23:36:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-09-10T23:36:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-09-10T23:36:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.827 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-09-10T23:36:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-09-10T23:36:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.828 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-09-10T23:36:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-09-10T23:36:35.829 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T23:36:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-09-10T23:36:35.830 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-09-10T23:36:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-10T23:36:35.831 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-10T23:36:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-09-10T23:36:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-09-10T23:36:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-09-10T23:36:35.832 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-09-10T23:36:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-09-10T23:36:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.833 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-09-10T23:36:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-09-10T23:36:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.834 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-09-10T23:36:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-09-10T23:36:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.836 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.837 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.838 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.839 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.840 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-09-10T23:36:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-09-10T23:36:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.841 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-09-10T23:36:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-09-10T23:36:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.842 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.843 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-09-10T23:36:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-09-10T23:36:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T23:36:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.844 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-09-10T23:36:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.845 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.846 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.847 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.848 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.849 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.852 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-09-10T23:36:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.853 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.855 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.856 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.857 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-09-10T23:36:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-09-10T23:36:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.858 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-09-10T23:36:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T23:36:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-09-10T23:36:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-09-10T23:36:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.859 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-09-10T23:36:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-09-10T23:36:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-09-10T23:36:35.860 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T23:36:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_no_wait": { 2024-09-10T23:36:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T23:36:35.861 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T23:36:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.862 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-09-10T23:36:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-09-10T23:36:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-09-10T23:36:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-10T23:36:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-09-10T23:36:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.864 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-09-10T23:36:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-09-10T23:36:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.865 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-10T23:36:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-10T23:36:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.866 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-10T23:36:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.867 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.868 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.869 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.870 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.871 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.872 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-09-10T23:36:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-09-10T23:36:35.873 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-09-10T23:36:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-09-10T23:36:35.874 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-09-10T23:36:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-09-10T23:36:35.875 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-09-10T23:36:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T23:36:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:36:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "modules": [ 2024-09-10T23:36:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "iostat", 2024-09-10T23:36:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "nfs", 2024-09-10T23:36:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "restful" 2024-09-10T23:36:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-09-10T23:36:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-09-10T23:36:35.877 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-10T23:36:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-10T23:36:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-10T23:36:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T23:36:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.878 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.879 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.880 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.881 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.883 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-09-10T23:36:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-09-10T23:36:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T23:36:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.884 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-09-10T23:36:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-09-10T23:36:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-09-10T23:36:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-09-10T23:36:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.885 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-09-10T23:36:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-09-10T23:36:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-09-10T23:36:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.886 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-09-10T23:36:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-09-10T23:36:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-09-10T23:36:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-09-10T23:36:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-09-10T23:36:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-09-10T23:36:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-09-10T23:36:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.888 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-09-10T23:36:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-09-10T23:36:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-09-10T23:36:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.889 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-09-10T23:36:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-09-10T23:36:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T23:36:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.890 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-09-10T23:36:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-09-10T23:36:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-09-10T23:36:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.891 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.892 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-09-10T23:36:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-09-10T23:36:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-09-10T23:36:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.893 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T23:36:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-09-10T23:36:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-09-10T23:36:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.894 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-09-10T23:36:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T23:36:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:36:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-09-10T23:36:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-09-10T23:36:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-10T23:36:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.895 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-09-10T23:36:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-09-10T23:36:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T23:36:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:36:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-09-10T23:36:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-09-10T23:36:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-10T23:36:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.896 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-09-10T23:36:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-10T23:36:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-09-10T23:36:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T23:36:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-09-10T23:36:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-09-10T23:36:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.897 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T23:36:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T23:36:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T23:36:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-09-10T23:36:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-09-10T23:36:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-10T23:36:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.898 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-09-10T23:36:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-09-10T23:36:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-09-10T23:36:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-09-10T23:36:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T23:36:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-09-10T23:36:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-09-10T23:36:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.899 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-09-10T23:36:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T23:36:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T23:36:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-09-10T23:36:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-09-10T23:36:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-10T23:36:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.900 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-09-10T23:36:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-09-10T23:36:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T23:36:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T23:36:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.901 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.902 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.903 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.904 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.905 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.906 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-09-10T23:36:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-09-10T23:36:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-10T23:36:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T23:36:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-09-10T23:36:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-09-10T23:36:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-09-10T23:36:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-09-10T23:36:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-09-10T23:36:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-09-10T23:36:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-09-10T23:36:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-09-10T23:36:35.908 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-09-10T23:36:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-09-10T23:36:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-09-10T23:36:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T23:36:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-10T23:36:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-10T23:36:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-10T23:36:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.910 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T23:36:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.911 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-09-10T23:36:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-09-10T23:36:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-10T23:36:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-10T23:36:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.912 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T23:36:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T23:36:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-09-10T23:36:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-09-10T23:36:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-10T23:36:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-10T23:36:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.913 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T23:36:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-09-10T23:36:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.914 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-09-10T23:36:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-09-10T23:36:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-10T23:36:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-09-10T23:36:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T23:36:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-09-10T23:36:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-09-10T23:36:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-09-10T23:36:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T23:36:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.916 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-09-10T23:36:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-09-10T23:36:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-09-10T23:36:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T23:36:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-09-10T23:36:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-09-10T23:36:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.918 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T23:36:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T23:36:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-09-10T23:36:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-09-10T23:36:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-10T23:36:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.919 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T23:36:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-09-10T23:36:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T23:36:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-10T23:36:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-09-10T23:36:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.920 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T23:36:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-09-10T23:36:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-09-10T23:36:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.921 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-09-10T23:36:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-09-10T23:36:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-09-10T23:36:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T23:36:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-09-10T23:36:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-09-10T23:36:35.922 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T23:36:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-09-10T23:36:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-09-10T23:36:35.923 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T23:36:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-09-10T23:36:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-09-10T23:36:35.924 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T23:36:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-09-10T23:36:35.925 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-09-10T23:36:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T23:36:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-10T23:36:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-09-10T23:36:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-09-10T23:36:35.926 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T23:36:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-09-10T23:36:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-09-10T23:36:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-09-10T23:36:35.927 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T23:36:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-09-10T23:36:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-09-10T23:36:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-09-10T23:36:35.928 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T23:36:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-10T23:36:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-09-10T23:36:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-09-10T23:36:35.929 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T23:36:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-09-10T23:36:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-09-10T23:36:35.930 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-09-10T23:36:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T23:36:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-09-10T23:36:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.931 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-09-10T23:36:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-09-10T23:36:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T23:36:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-09-10T23:36:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-09-10T23:36:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-09-10T23:36:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T23:36:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-09-10T23:36:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-09-10T23:36:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-09-10T23:36:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T23:36:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-09-10T23:36:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-09-10T23:36:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-09-10T23:36:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T23:36:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-09-10T23:36:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nginx": { 2024-09-10T23:36:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nginx", 2024-09-10T23:36:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T23:36:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nginx container image", 2024-09-10T23:36:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.936 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-09-10T23:36:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-09-10T23:36:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T23:36:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-10T23:36:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.937 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-09-10T23:36:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-09-10T23:36:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T23:36:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-09-10T23:36:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.938 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_oauth2_proxy": { 2024-09-10T23:36:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T23:36:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T23:36:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "oauth2-proxy container image", 2024-09-10T23:36:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.939 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-09-10T23:36:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-09-10T23:36:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T23:36:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-09-10T23:36:35.940 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-09-10T23:36:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-09-10T23:36:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T23:36:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.941 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-09-10T23:36:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_samba": { 2024-09-10T23:36:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_samba", 2024-09-10T23:36:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T23:36:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.942 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Samba/SMB container image", 2024-09-10T23:36:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_samba_metrics": { 2024-09-10T23:36:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_samba_metrics", 2024-09-10T23:36:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T23:36:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.943 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T23:36:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-09-10T23:36:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-09-10T23:36:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T23:36:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.944 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-09-10T23:36:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-09-10T23:36:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-09-10T23:36:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.945 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T23:36:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-09-10T23:36:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-09-10T23:36:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.946 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-10T23:36:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T23:36:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-09-10T23:36:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T23:36:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.947 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-09-10T23:36:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T23:36:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-09-10T23:36:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-09-10T23:36:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.948 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-09-10T23:36:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T23:36:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-09-10T23:36:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-09-10T23:36:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.949 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-09-10T23:36:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-09-10T23:36:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-09-10T23:36:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-09-10T23:36:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.950 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T23:36:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-09-10T23:36:35.951 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-09-10T23:36:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-10T23:36:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-09-10T23:36:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "grafana_dashboards_path": { 2024-09-10T23:36:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "grafana_dashboards_path", 2024-09-10T23:36:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T23:36:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T23:36:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.953 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-09-10T23:36:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-09-10T23:36:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-10T23:36:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-09-10T23:36:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.954 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-09-10T23:36:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-09-10T23:36:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T23:36:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.955 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-09-10T23:36:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-09-10T23:36:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T23:36:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.956 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.957 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-09-10T23:36:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-09-10T23:36:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.958 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T23:36:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.959 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T23:36:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.960 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.961 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.962 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T23:36:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T23:36:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.963 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T23:36:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T23:36:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T23:36:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.964 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-09-10T23:36:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T23:36:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-09-10T23:36:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-09-10T23:36:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.965 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-10T23:36:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-09-10T23:36:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-09-10T23:36:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-09-10T23:36:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.966 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-10T23:36:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T23:36:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-09-10T23:36:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-09-10T23:36:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-09-10T23:36:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-09-10T23:36:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-09-10T23:36:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-09-10T23:36:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-09-10T23:36:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-09-10T23:36:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T23:36:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-09-10T23:36:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-09-10T23:36:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-09-10T23:36:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T23:36:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-09-10T23:36:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-09-10T23:36:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T23:36:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T23:36:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-09-10T23:36:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-09-10T23:36:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T23:36:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-09-10T23:36:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-09-10T23:36:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T23:36:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-09-10T23:36:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-09-10T23:36:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T23:36:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-09-10T23:36:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-09-10T23:36:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T23:36:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-09-10T23:36:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-09-10T23:36:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T23:36:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-09-10T23:36:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-09-10T23:36:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-09-10T23:36:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-09-10T23:36:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-09-10T23:36:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-09-10T23:36:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T23:36:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-09-10T23:36:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-09-10T23:36:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T23:36:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-09-10T23:36:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-09-10T23:36:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T23:36:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-09-10T23:36:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-09-10T23:36:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T23:36:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-09-10T23:36:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-09-10T23:36:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T23:36:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-09-10T23:36:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-09-10T23:36:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T23:36:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-09-10T23:36:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-09-10T23:36:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-09-10T23:36:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-09-10T23:36:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T23:36:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-09-10T23:36:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-09-10T23:36:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-09-10T23:36:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T23:36:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-09-10T23:36:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T23:36:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T23:36:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-10T23:36:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T23:36:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T23:36:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T23:36:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T23:36:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-09-10T23:36:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T23:36:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T23:36:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T23:36:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T23:36:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T23:36:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T23:36:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T23:36:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T23:36:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T23:36:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T23:36:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T23:36:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T23:36:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T23:36:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T23:36:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T23:36:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T23:36:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T23:36:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T23:36:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T23:36:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T23:36:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T23:36:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T23:36:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T23:36:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-09-10T23:36:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T23:36:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T23:36:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-09-10T23:36:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-09-10T23:36:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.012 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T23:36:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T23:36:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-09-10T23:36:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T23:36:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T23:36:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T23:36:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T23:36:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T23:36:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T23:36:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T23:36:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T23:36:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T23:36:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T23:36:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "[]", 2024-09-10T23:36:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T23:36:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T23:36:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "{}", 2024-09-10T23:36:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T23:36:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T23:36:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T23:36:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T23:36:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T23:36:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T23:36:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T23:36:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T23:36:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T23:36:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T23:36:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T23:36:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T23:36:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T23:36:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T23:36:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T23:36:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T23:36:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T23:36:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T23:36:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T23:36:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T23:36:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T23:36:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T23:36:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T23:36:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T23:36:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T23:36:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-10T23:36:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T23:36:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T23:36:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-09-10T23:36:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T23:36:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T23:36:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-09-10T23:36:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T23:36:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T23:36:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T23:36:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T23:36:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-09-10T23:36:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T23:36:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T23:36:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T23:36:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T23:36:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T23:36:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T23:36:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T23:36:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T23:36:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T23:36:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T23:36:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-10T23:36:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T23:36:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T23:36:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-10T23:36:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-09-10T23:36:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-09-10T23:36:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-09-10T23:36:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-09-10T23:36:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-09-10T23:36:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-09-10T23:36:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-09-10T23:36:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-09-10T23:36:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-09-10T23:36:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-09-10T23:36:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-09-10T23:36:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-09-10T23:36:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-09-10T23:36:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-09-10T23:36:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-09-10T23:36:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-09-10T23:36:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T23:36:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-09-10T23:36:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-09-10T23:36:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-09-10T23:36:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-09-10T23:36:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-09-10T23:36:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-09-10T23:36:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-09-10T23:36:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-09-10T23:36:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-09-10T23:36:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-09-10T23:36:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-09-10T23:36:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-09-10T23:36:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-09-10T23:36:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-09-10T23:36:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-09-10T23:36:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-09-10T23:36:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-09-10T23:36:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-09-10T23:36:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-09-10T23:36:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-09-10T23:36:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-09-10T23:36:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-09-10T23:36:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-09-10T23:36:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-09-10T23:36:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-09-10T23:36:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-09-10T23:36:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-09-10T23:36:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-09-10T23:36:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-09-10T23:36:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T23:36:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-09-10T23:36:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-09-10T23:36:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-09-10T23:36:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T23:36:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-09-10T23:36:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-09-10T23:36:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-09-10T23:36:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-09-10T23:36:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-09-10T23:36:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-09-10T23:36:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-09-10T23:36:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T23:36:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-09-10T23:36:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-09-10T23:36:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T23:36:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-10T23:36:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-10T23:36:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-10T23:36:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T23:36:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-09-10T23:36:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-09-10T23:36:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-09-10T23:36:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T23:36:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-09-10T23:36:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-09-10T23:36:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-09-10T23:36:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-09-10T23:36:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-09-10T23:36:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-09-10T23:36:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-09-10T23:36:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-10T23:36:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-10T23:36:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-09-10T23:36:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-09-10T23:36:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-09-10T23:36:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-09-10T23:36:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-09-10T23:36:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-09-10T23:36:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-09-10T23:36:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T23:36:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-09-10T23:36:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-09-10T23:36:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-09-10T23:36:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T23:36:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-09-10T23:36:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-09-10T23:36:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-09-10T23:36:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-10T23:36:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-10T23:36:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-09-10T23:36:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-09-10T23:36:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T23:36:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-09-10T23:36:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-09-10T23:36:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-09-10T23:36:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-09-10T23:36:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-09-10T23:36:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-09-10T23:36:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-09-10T23:36:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-09-10T23:36:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-09-10T23:36:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-09-10T23:36:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:36:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-09-10T23:36:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-09-10T23:36:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-10T23:36:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-10T23:36:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-09-10T23:36:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T23:36:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-09-10T23:36:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-09-10T23:36:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-09-10T23:36:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-09-10T23:36:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-09-10T23:36:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-09-10T23:36:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T23:36:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-09-10T23:36:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-09-10T23:36:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-09-10T23:36:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-09-10T23:36:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-09-10T23:36:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-09-10T23:36:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-09-10T23:36:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T23:36:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-09-10T23:36:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-09-10T23:36:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-09-10T23:36:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T23:36:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-09-10T23:36:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-09-10T23:36:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-09-10T23:36:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-09-10T23:36:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-09-10T23:36:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-09-10T23:36:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-09-10T23:36:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T23:36:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-09-10T23:36:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-09-10T23:36:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-09-10T23:36:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.127 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-09-10T23:36:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-09-10T23:36:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-09-10T23:36:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T23:36:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T23:36:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-09-10T23:36:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.135 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-09-10T23:36:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.137 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.138 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.139 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.140 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.141 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.142 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-09-10T23:36:36.143 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-09-10T23:36:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-09-10T23:36:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T23:36:36.144 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.145 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.146 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.147 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.148 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.149 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-09-10T23:36:36.150 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-09-10T23:36:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-09-10T23:36:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-09-10T23:36:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-09-10T23:36:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-09-10T23:36:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.151 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.152 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-09-10T23:36:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.153 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.154 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.155 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.156 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.157 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.158 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-09-10T23:36:36.159 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.160 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.161 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.162 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.163 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.164 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.165 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-09-10T23:36:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.166 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.167 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.169 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.171 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-10T23:36:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-10T23:36:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-10T23:36:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-09-10T23:36:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-09-10T23:36:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-10T23:36:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-09-10T23:36:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-09-10T23:36:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-09-10T23:36:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T23:36:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.174 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-09-10T23:36:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-09-10T23:36:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-09-10T23:36:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.175 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T23:36:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-09-10T23:36:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-09-10T23:36:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.176 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.177 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.180 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.181 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-09-10T23:36:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-09-10T23:36:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.182 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-09-10T23:36:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-09-10T23:36:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-09-10T23:36:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-09-10T23:36:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.183 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-10T23:36:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-09-10T23:36:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.184 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.185 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-09-10T23:36:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-09-10T23:36:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-09-10T23:36:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.186 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-09-10T23:36:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-09-10T23:36:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.187 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T23:36:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T23:36:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.188 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.190 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.192 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-09-10T23:36:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-09-10T23:36:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.193 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T23:36:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T23:36:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-09-10T23:36:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.194 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-09-10T23:36:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-09-10T23:36:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-10T23:36:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-09-10T23:36:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.195 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-09-10T23:36:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-09-10T23:36:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-09-10T23:36:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.196 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T23:36:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-09-10T23:36:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-09-10T23:36:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-09-10T23:36:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.197 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T23:36:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.198 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-09-10T23:36:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-09-10T23:36:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-09-10T23:36:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.199 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-09-10T23:36:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-09-10T23:36:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-09-10T23:36:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.200 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-09-10T23:36:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-09-10T23:36:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-09-10T23:36:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-09-10T23:36:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.201 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-09-10T23:36:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-09-10T23:36:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-09-10T23:36:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-09-10T23:36:36.202 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.203 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.204 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.205 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.206 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.207 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-09-10T23:36:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-09-10T23:36:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-09-10T23:36:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.208 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-09-10T23:36:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T23:36:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.209 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.210 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-09-10T23:36:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-09-10T23:36:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.211 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-09-10T23:36:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.212 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-09-10T23:36:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-09-10T23:36:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-09-10T23:36:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-09-10T23:36:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.214 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.215 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.216 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.217 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.218 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "max_requests": { 2024-09-10T23:36:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_requests", 2024-09-10T23:36:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-09-10T23:36:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.220 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T23:36:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-09-10T23:36:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-09-10T23:36:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.221 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-09-10T23:36:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-09-10T23:36:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.222 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.223 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-09-10T23:36:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.224 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.225 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.226 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.227 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T23:36:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T23:36:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-09-10T23:36:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T23:36:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.231 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-09-10T23:36:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.233 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.234 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.235 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.236 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-09-10T23:36:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-09-10T23:36:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.237 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-09-10T23:36:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-09-10T23:36:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.238 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-09-10T23:36:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-09-10T23:36:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-09-10T23:36:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.239 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-09-10T23:36:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-09-10T23:36:36.240 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-09-10T23:36:36.241 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-09-10T23:36:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-09-10T23:36:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-09-10T23:36:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-09-10T23:36:36.242 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-09-10T23:36:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-09-10T23:36:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.243 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-09-10T23:36:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-09-10T23:36:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-09-10T23:36:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-09-10T23:36:36.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-09-10T23:36:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-09-10T23:36:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.246 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-09-10T23:36:36.247 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-09-10T23:36:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.248 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-09-10T23:36:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-09-10T23:36:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-09-10T23:36:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-09-10T23:36:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.250 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-09-10T23:36:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-09-10T23:36:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-09-10T23:36:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.251 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T23:36:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-09-10T23:36:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-09-10T23:36:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.252 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-09-10T23:36:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.253 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.254 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.255 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.256 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.257 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.258 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-09-10T23:36:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.260 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.261 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.262 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.263 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.264 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.265 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-09-10T23:36:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.266 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.267 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.268 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.270 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.271 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.272 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-09-10T23:36:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.273 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-09-10T23:36:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-09-10T23:36:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T23:36:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.274 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-10T23:36:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-10T23:36:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-09-10T23:36:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.276 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.277 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.278 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.279 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.280 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-09-10T23:36:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-09-10T23:36:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-09-10T23:36:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T23:36:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-09-10T23:36:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-09-10T23:36:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T23:36:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-09-10T23:36:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-09-10T23:36:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T23:36:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-09-10T23:36:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-09-10T23:36:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T23:36:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-09-10T23:36:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-09-10T23:36:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T23:36:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-09-10T23:36:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-09-10T23:36:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-09-10T23:36:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-09-10T23:36:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-09-10T23:36:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-09-10T23:36:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T23:36:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-09-10T23:36:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-09-10T23:36:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-10T23:36:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-10T23:36:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-09-10T23:36:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-09-10T23:36:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-09-10T23:36:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-09-10T23:36:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-09-10T23:36:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-09-10T23:36:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-09-10T23:36:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-09-10T23:36:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-09-10T23:36:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-09-10T23:36:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-09-10T23:36:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-09-10T23:36:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-09-10T23:36:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-09-10T23:36:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-09-10T23:36:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T23:36:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-09-10T23:36:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-09-10T23:36:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-09-10T23:36:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-09-10T23:36:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-09-10T23:36:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T23:36:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-09-10T23:36:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-09-10T23:36:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-09-10T23:36:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-09-10T23:36:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-09-10T23:36:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T23:36:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_no_wait": { 2024-09-10T23:36:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T23:36:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-09-10T23:36:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T23:36:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-09-10T23:36:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-09-10T23:36:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-09-10T23:36:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-09-10T23:36:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-09-10T23:36:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-09-10T23:36:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-09-10T23:36:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-09-10T23:36:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-09-10T23:36:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-09-10T23:36:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-09-10T23:36:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-09-10T23:36:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-09-10T23:36:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-09-10T23:36:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-09-10T23:36:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-09-10T23:36:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-09-10T23:36:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-09-10T23:36:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-09-10T23:36:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-09-10T23:36:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-09-10T23:36:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-09-10T23:36:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-09-10T23:36:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-09-10T23:36:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-09-10T23:36:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-09-10T23:36:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-09-10T23:36:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-09-10T23:36:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-09-10T23:36:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-09-10T23:36:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-09-10T23:36:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-09-10T23:36:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "sqlite3_killpoint": { 2024-09-10T23:36:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sqlite3_killpoint", 2024-09-10T23:36:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "dev", 2024-09-10T23:36:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-10T23:36:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-09-10T23:36:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-09-10T23:36:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-09-10T23:36:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-09-10T23:36:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-09-10T23:36:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-09-10T23:36:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-09-10T23:36:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-09-10T23:36:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-09-10T23:36:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-09-10T23:36:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-09-10T23:36:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-09-10T23:36:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-09-10T23:36:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T23:36:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-09-10T23:36:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-09-10T23:36:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-09-10T23:36:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-09-10T23:36:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-09-10T23:36:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-09-10T23:36:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-09-10T23:36:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "services": {}, 2024-09-10T23:36:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "always_on_modules": { 2024-09-10T23:36:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "octopus": [ 2024-09-10T23:36:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-09-10T23:36:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-09-10T23:36:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-09-10T23:36:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-09-10T23:36:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-09-10T23:36:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-09-10T23:36:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-09-10T23:36:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-09-10T23:36:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-09-10T23:36:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-09-10T23:36:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "pacific": [ 2024-09-10T23:36:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-09-10T23:36:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-09-10T23:36:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-09-10T23:36:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-09-10T23:36:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-09-10T23:36:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-09-10T23:36:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-09-10T23:36:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-09-10T23:36:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-09-10T23:36:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-09-10T23:36:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "quincy": [ 2024-09-10T23:36:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-09-10T23:36:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-09-10T23:36:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-09-10T23:36:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-09-10T23:36:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-09-10T23:36:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-09-10T23:36:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-09-10T23:36:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-09-10T23:36:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-09-10T23:36:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-09-10T23:36:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "reef": [ 2024-09-10T23:36:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-09-10T23:36:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-09-10T23:36:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-09-10T23:36:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-09-10T23:36:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-09-10T23:36:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-09-10T23:36:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-09-10T23:36:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-09-10T23:36:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-09-10T23:36:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-09-10T23:36:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:36:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "squid": [ 2024-09-10T23:36:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-09-10T23:36:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-09-10T23:36:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-09-10T23:36:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-09-10T23:36:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-09-10T23:36:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-09-10T23:36:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-09-10T23:36:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-09-10T23:36:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-09-10T23:36:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-09-10T23:36:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:36:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "force_disabled_modules": {}, 2024-09-10T23:36:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "last_failure_osd_epoch": 0, 2024-09-10T23:36:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "active_clients": [ 2024-09-10T23:36:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-09-10T23:36:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-10T23:36:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-10T23:36:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.181:0", 2024-09-10T23:36:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1584210907 2024-09-10T23:36:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:36:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "libcephsqlite", 2024-09-10T23:36:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-10T23:36:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-10T23:36:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.181:0", 2024-09-10T23:36:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1346366560 2024-09-10T23:36:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:36:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-09-10T23:36:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-10T23:36:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-10T23:36:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.181:0", 2024-09-10T23:36:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2231303511 2024-09-10T23:36:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:36:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-09-10T23:36:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-10T23:36:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-10T23:36:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.181:0", 2024-09-10T23:36:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2211863778 2024-09-10T23:36:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:36:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-09-10T23:36:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-10T23:36:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-10T23:36:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.181:0", 2024-09-10T23:36:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2614085240 2024-09-10T23:36:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:36:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-09-10T23:36:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-10T23:36:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-10T23:36:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.181:0", 2024-09-10T23:36:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 749967229 2024-09-10T23:36:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:36:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-09-10T23:36:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-10T23:36:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-10T23:36:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.181:0", 2024-09-10T23:36:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1339348859 2024-09-10T23:36:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:36:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-09-10T23:36:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-10T23:36:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-10T23:36:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.181:0", 2024-09-10T23:36:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2150218902 2024-09-10T23:36:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:36:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-09-10T23:36:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-10T23:36:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-10T23:36:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.181:0", 2024-09-10T23:36:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2484932838 2024-09-10T23:36:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:36:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-09-10T23:36:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-10T23:36:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-10T23:36:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.181:0", 2024-09-10T23:36:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3942792329 2024-09-10T23:36:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:36:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:36:36.347 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:36:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.398+0000 7f0b95c00640 1 -- 172.21.15.94:0/2750809858 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b6c072940 msgr2=0x7f0b6c074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.398+0000 7f0b95c00640 1 --2- 172.21.15.94:0/2750809858 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b6c072940 0x7f0b6c074e00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0b800b0000 tx=0x7f0b800a7040 comp rx=0 tx=0).stop 2024-09-10T23:36:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.398+0000 7f0b95c00640 1 -- 172.21.15.94:0/2750809858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b900b0b80 msgr2=0x7f0b900b1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.398+0000 7f0b95c00640 1 --2- 172.21.15.94:0/2750809858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b900b0b80 0x7f0b900b1000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0b780011b0 tx=0x7f0b78001720 comp rx=0 tx=0).stop 2024-09-10T23:36:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.398+0000 7f0b95c00640 1 -- 172.21.15.94:0/2750809858 shutdown_connections 2024-09-10T23:36:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.398+0000 7f0b95c00640 1 --2- 172.21.15.94:0/2750809858 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7f0b6c072940 0x7f0b6c074e00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.398+0000 7f0b95c00640 1 --2- 172.21.15.94:0/2750809858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b9006f6f0 0x7f0b9006fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.398+0000 7f0b95c00640 1 --2- 172.21.15.94:0/2750809858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b900b0b80 0x7f0b900b1000 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.398+0000 7f0b95c00640 1 --2- 172.21.15.94:0/2750809858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b740039f0 0x7f0b9013a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.398+0000 7f0b95c00640 1 -- 172.21.15.94:0/2750809858 >> 172.21.15.94:0/2750809858 conn(0x7f0b9006cdf0 msgr2=0x7f0b9006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.398+0000 7f0b95c00640 1 -- 172.21.15.94:0/2750809858 shutdown_connections 2024-09-10T23:36:36.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:35.398+0000 7f0b95c00640 1 -- 172.21.15.94:0/2750809858 wait complete. 2024-09-10T23:36:36.354 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail x 2024-09-10T23:36:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa193200640 1 -- 172.21.15.94:0/1832756387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1940b1640 msgr2=0x7fa1940b27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa193200640 1 --2- 172.21.15.94:0/1832756387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1940b1640 0x7fa1940b27f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa1940cf5f0 tx=0x7fa18401c830 comp rx=0 tx=0).stop 2024-09-10T23:36:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa193200640 1 -- 172.21.15.94:0/1832756387 shutdown_connections 2024-09-10T23:36:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa193200640 1 --2- 172.21.15.94:0/1832756387 >> v2:172.21.15.181:3300/0 conn(0x7fa1940d0520 0x7fa1940b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa193200640 1 --2- 172.21.15.94:0/1832756387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1940b1640 0x7fa1940b27f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa193200640 1 -- 172.21.15.94:0/1832756387 >> 172.21.15.94:0/1832756387 conn(0x7fa1940d1af0 msgr2=0x7fa1940d1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa193200640 1 -- 172.21.15.94:0/1832756387 shutdown_connections 2024-09-10T23:36:36.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa193200640 1 -- 172.21.15.94:0/1832756387 wait complete. 2024-09-10T23:36:36.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa193200640 1 Processor -- start 2024-09-10T23:36:36.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa193200640 1 -- start start 2024-09-10T23:36:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa193200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1940b1640 0x7fa1940ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa193200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1940d00d0 0x7fa19415ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa193200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1940d0520 0x7fa19415bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa193200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1941609d0 con 0x7fa1940b1640 2024-09-10T23:36:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa193200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa194160850 con 0x7fa1940d0520 2024-09-10T23:36:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa193200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa194160b50 con 0x7fa1940d00d0 2024-09-10T23:36:36.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.498+0000 7fa191e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1940b1640 0x7fa1940ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa191e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1940b1640 0x7fa1940ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48670/0 (socket says 172.21.15.94:48670) 2024-09-10T23:36:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa191e00640 1 -- 172.21.15.94:0/4198105194 learned_addr learned my addr 172.21.15.94:0/4198105194 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa191400640 1 --2- 172.21.15.94:0/4198105194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1940d00d0 0x7fa19415ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa192800640 1 --2- 172.21.15.94:0/4198105194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1940d0520 0x7fa19415bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa191e00640 1 -- 172.21.15.94:0/4198105194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1940d00d0 msgr2=0x7fa19415ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa191e00640 1 --2- 172.21.15.94:0/4198105194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1940d00d0 0x7fa19415ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa191e00640 1 -- 172.21.15.94:0/4198105194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1940d0520 msgr2=0x7fa19415bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa191e00640 1 --2- 172.21.15.94:0/4198105194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1940d0520 0x7fa19415bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa191e00640 1 -- 172.21.15.94:0/4198105194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1940ce7d0 con 0x7fa1940b1640 2024-09-10T23:36:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa191e00640 1 --2- 172.21.15.94:0/4198105194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1940b1640 0x7fa1940ce290 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa18401aee0 tx=0x7fa1840a6750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa18a000640 1 -- 172.21.15.94:0/4198105194 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1840bd070 con 0x7fa1940b1640 2024-09-10T23:36:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa18a000640 1 -- 172.21.15.94:0/4198105194 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa184004430 con 0x7fa1940b1640 2024-09-10T23:36:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa18a000640 1 -- 172.21.15.94:0/4198105194 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184005290 con 0x7fa1940b1640 2024-09-10T23:36:36.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa193200640 1 -- 172.21.15.94:0/4198105194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1941f5d10 con 0x7fa1940b1640 2024-09-10T23:36:36.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa193200640 1 -- 172.21.15.94:0/4198105194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1941f62d0 con 0x7fa1940b1640 2024-09-10T23:36:36.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa193200640 1 -- 172.21.15.94:0/4198105194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1940b27f0 con 0x7fa1940b1640 2024-09-10T23:36:36.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.502+0000 7fa18a000640 1 -- 172.21.15.94:0/4198105194 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) ==== 96751+0+0 (secure 0 0 0) 0x7fa184005430 con 0x7fa1940b1640 2024-09-10T23:36:36.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.506+0000 7fa18a000640 1 --2- 172.21.15.94:0/4198105194 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa174072a00 0x7fa174074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:36.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.506+0000 7fa18a000640 1 -- 172.21.15.94:0/4198105194 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7fa18412db00 con 0x7fa1940b1640 2024-09-10T23:36:36.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.506+0000 7fa191400640 1 --2- 172.21.15.94:0/4198105194 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa174072a00 0x7fa174074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:36.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.506+0000 7fa191400640 1 --2- 172.21.15.94:0/4198105194 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa174072a00 0x7fa174074ec0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa1940cbe80 tx=0x7fa180007e40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:36.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.510+0000 7fa18a000640 1 -- 172.21.15.94:0/4198105194 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa1840164b0 con 0x7fa1940b1640 2024-09-10T23:36:36.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.714+0000 7fa193200640 1 -- 172.21.15.94:0/4198105194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr fail", "who": "x"} v 0) -- 0x7fa19415d4e0 con 0x7fa1940b1640 2024-09-10T23:36:36.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.718+0000 7fa18a000640 1 -- 172.21.15.94:0/4198105194 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr fail", "who": "x"}]=0 v13) ==== 68+0+0 (secure 0 0 0) 0x7fa1840f8460 con 0x7fa1940b1640 2024-09-10T23:36:36.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.722+0000 7fa18a000640 1 -- 172.21.15.94:0/4198105194 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 13) ==== 48237+0+0 (secure 0 0 0) 0x7fa1840f46d0 con 0x7fa1940b1640 2024-09-10T23:36:36.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.722+0000 7fa18a000640 1 -- 172.21.15.94:0/4198105194 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa174072a00 msgr2=0x7fa174074ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:36.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.722+0000 7fa18a000640 1 --2- 172.21.15.94:0/4198105194 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa174072a00 0x7fa174074ec0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa1940cbe80 tx=0x7fa180007e40 comp rx=0 tx=0).stop 2024-09-10T23:36:36.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.722+0000 7fa193200640 1 -- 172.21.15.94:0/4198105194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1940b1640 msgr2=0x7fa1940ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:36.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.722+0000 7fa193200640 1 --2- 172.21.15.94:0/4198105194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1940b1640 0x7fa1940ce290 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa18401aee0 tx=0x7fa1840a6750 comp rx=0 tx=0).stop 2024-09-10T23:36:36.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.722+0000 7fa193200640 1 -- 172.21.15.94:0/4198105194 shutdown_connections 2024-09-10T23:36:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.722+0000 7fa193200640 1 --2- 172.21.15.94:0/4198105194 >> [v2:172.21.15.181:6800/3310396812,v1:172.21.15.181:6801/3310396812] conn(0x7fa174072a00 0x7fa174074ec0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.722+0000 7fa193200640 1 --2- 172.21.15.94:0/4198105194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1940d0520 0x7fa19415bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.722+0000 7fa193200640 1 --2- 172.21.15.94:0/4198105194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1940d00d0 0x7fa19415ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.722+0000 7fa193200640 1 --2- 172.21.15.94:0/4198105194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1940b1640 0x7fa1940ce290 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.722+0000 7fa193200640 1 -- 172.21.15.94:0/4198105194 >> 172.21.15.94:0/4198105194 conn(0x7fa1940d1af0 msgr2=0x7fa194070c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.722+0000 7fa193200640 1 -- 172.21.15.94:0/4198105194 shutdown_connections 2024-09-10T23:36:36.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.722+0000 7fa193200640 1 -- 172.21.15.94:0/4198105194 wait complete. 2024-09-10T23:36:36.756 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:36.844 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:36:36.842+0000 7f9ea3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:36.844 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:36:36.842+0000 7f9ea3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:36.844 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:36:36.842+0000 7f9ea3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:36.844 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:36:36.842+0000 7f9ea3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:36.844 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:36:36.842+0000 7f9ea3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:36.844 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:36:36.842+0000 7f9ea7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:36.844 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:36:36.842+0000 7f9ea7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:36.844 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:36:36.842+0000 7f9ea7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:36.844 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:36:36.842+0000 7f9ea7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:36.844 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:36:36.842+0000 7f9ea7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:36:36.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.930+0000 7f89df800640 1 -- 172.21.15.94:0/1555380868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d806f600 msgr2=0x7f89d80b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:36.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.930+0000 7f89df800640 1 --2- 172.21.15.94:0/1555380868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d806f600 0x7f89d80b35c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f89d80b2760 tx=0x7f89d001dbe0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.930+0000 7f89df800640 1 -- 172.21.15.94:0/1555380868 shutdown_connections 2024-09-10T23:36:36.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.930+0000 7f89df800640 1 --2- 172.21.15.94:0/1555380868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d806f600 0x7f89d80b35c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.930+0000 7f89df800640 1 --2- 172.21.15.94:0/1555380868 >> v2:172.21.15.181:3300/0 conn(0x7f89d8157c60 0x7f89d8072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.930+0000 7f89df800640 1 -- 172.21.15.94:0/1555380868 >> 172.21.15.94:0/1555380868 conn(0x7f89d806cdf0 msgr2=0x7f89d806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:36.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.930+0000 7f89df800640 1 -- 172.21.15.94:0/1555380868 shutdown_connections 2024-09-10T23:36:36.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.930+0000 7f89df800640 1 -- 172.21.15.94:0/1555380868 wait complete. 2024-09-10T23:36:36.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.930+0000 7f89df800640 1 Processor -- start 2024-09-10T23:36:36.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.930+0000 7f89df800640 1 -- start start 2024-09-10T23:36:36.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.930+0000 7f89df800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89d806f600 0x7f89d8150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:36.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89df800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d8070150 0x7f89d80c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:36.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89df800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89d8157c60 0x7f89d80c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:36.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89df800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f89d815b2c0 con 0x7f89d8070150 2024-09-10T23:36:36.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89df800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f89d815b140 con 0x7f89d806f600 2024-09-10T23:36:36.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89df800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f89d815b440 con 0x7f89d8157c60 2024-09-10T23:36:36.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89cfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d8070150 0x7f89d80c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:36.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89dce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89d806f600 0x7f89d8150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:36.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89cfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d8070150 0x7f89d80c5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48800/0 (socket says 172.21.15.94:48800) 2024-09-10T23:36:36.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89dce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89d806f600 0x7f89d8150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:36046/0 (socket says 172.21.15.94:36046) 2024-09-10T23:36:36.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89cfe00640 1 -- 172.21.15.94:0/837346409 learned_addr learned my addr 172.21.15.94:0/837346409 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:36.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89dd800640 1 --2- 172.21.15.94:0/837346409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89d8157c60 0x7f89d80c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:36.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89cfe00640 1 -- 172.21.15.94:0/837346409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89d8157c60 msgr2=0x7f89d80c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:36.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89cfe00640 1 --2- 172.21.15.94:0/837346409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89d8157c60 0x7f89d80c55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89cfe00640 1 -- 172.21.15.94:0/837346409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89d806f600 msgr2=0x7f89d8150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:36.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89cfe00640 1 --2- 172.21.15.94:0/837346409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89d806f600 0x7f89d8150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:36.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89cfe00640 1 -- 172.21.15.94:0/837346409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d8151260 con 0x7f89d8070150 2024-09-10T23:36:36.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89cfe00640 1 --2- 172.21.15.94:0/837346409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d8070150 0x7f89d80c5080 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f89c000c500 tx=0x7f89c000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:36.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89d5600640 1 -- 172.21.15.94:0/837346409 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0017070 con 0x7f89d8070150 2024-09-10T23:36:36.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89d5600640 1 -- 172.21.15.94:0/837346409 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89c00053c0 con 0x7f89d8070150 2024-09-10T23:36:36.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89d5600640 1 -- 172.21.15.94:0/837346409 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0003860 con 0x7f89d8070150 2024-09-10T23:36:36.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89df800640 1 -- 172.21.15.94:0/837346409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d8151500 con 0x7f89d8070150 2024-09-10T23:36:36.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89df800640 1 -- 172.21.15.94:0/837346409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d8151ac0 con 0x7f89d8070150 2024-09-10T23:36:36.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89df800640 1 -- 172.21.15.94:0/837346409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d80b35c0 con 0x7f89d8070150 2024-09-10T23:36:36.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89d5600640 1 -- 172.21.15.94:0/837346409 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 48237+0+0 (secure 0 0 0) 0x7f89c0003a00 con 0x7f89d8070150 2024-09-10T23:36:36.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.934+0000 7f89d5600640 1 -- 172.21.15.94:0/837346409 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6268+0+0 (secure 0 0 0) 0x7f89c004ef40 con 0x7f89d8070150 2024-09-10T23:36:36.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:36.938+0000 7f89d5600640 1 -- 172.21.15.94:0/837346409 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f89c0011d40 con 0x7f89d8070150 2024-09-10T23:36:37.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.142+0000 7f89df800640 1 -- 172.21.15.94:0/837346409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89d806e7e0 con 0x7f89d8070150 2024-09-10T23:36:37.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.142+0000 7f89d5600640 1 -- 172.21.15.94:0/837346409 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5238 (secure 0 0 0) 0x7f89c00192c0 con 0x7f89d8070150 2024-09-10T23:36:37.144 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:37.144 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":44,"btime":"2024-09-10T23:36:31:051584+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3320339747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3320339747},{"type":"v1","addr":"172.21.15.94:6834","nonce":3320339747}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3123300514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3123300514},{"type":"v1","addr":"172.21.15.94:6835","nonce":3123300514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/339523200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":339523200},{"type":"v1","addr":"172.21.15.181:6837","nonce":339523200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:36:19.404249+0000","modified":"2024-09-10T23:36:22.404999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195389":{"gid":195389,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/2999113739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2999113739},{"type":"v1","addr":"172.21.15.181:6836","nonce":2999113739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195389,"qdb_cluster":[195389]},"id":5}]} 2024-09-10T23:36:37.144 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 44 2024-09-10T23:36:37.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.146+0000 7f89df800640 1 -- 172.21.15.94:0/837346409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d8070150 msgr2=0x7f89d80c5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:37.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.146+0000 7f89df800640 1 --2- 172.21.15.94:0/837346409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d8070150 0x7f89d80c5080 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f89c000c500 tx=0x7f89c000c9c0 comp rx=0 tx=0).stop 2024-09-10T23:36:37.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.146+0000 7f89df800640 1 -- 172.21.15.94:0/837346409 shutdown_connections 2024-09-10T23:36:37.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.146+0000 7f89df800640 1 --2- 172.21.15.94:0/837346409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89d8157c60 0x7f89d80c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:37.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.146+0000 7f89df800640 1 --2- 172.21.15.94:0/837346409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d8070150 0x7f89d80c5080 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:37.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.146+0000 7f89df800640 1 --2- 172.21.15.94:0/837346409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89d806f600 0x7f89d8150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:37.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.146+0000 7f89df800640 1 -- 172.21.15.94:0/837346409 >> 172.21.15.94:0/837346409 conn(0x7f89d806cdf0 msgr2=0x7f89d8159e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:37.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.146+0000 7f89df800640 1 -- 172.21.15.94:0/837346409 shutdown_connections 2024-09-10T23:36:37.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.146+0000 7f89df800640 1 -- 172.21.15.94:0/837346409 wait complete. 2024-09-10T23:36:37.165 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-10T23:36:37.329 INFO:teuthology.orchestra.run.smithi181.stdout:[{"id":195593,"entity":{"name":{"type":"client","num":195593},"addr":{"type":"any","addr":"172.21.15.94:0","nonce":3371089655}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.46399525200000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9978754995770893,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.195593 172.21.15.94:0/3371089655","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8f219c476cb4f26fc5719c661c0323719a45cb30","ceph_version":"ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)","entity_id":"y","hostname":"smithi094","pid":"28898","root":"/"}},{"id":6507,"entity":{"name":{"type":"client","num":6507},"addr":{"type":"any","addr":"172.21.15.94:0","nonce":2420181232}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.46399525200000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9978754995770893,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6507 172.21.15.94:0/2420181232","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"8f219c476cb4f26fc5719c661c0323719a45cb30","ceph_version":"ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)","entity_id":"y","hostname":"smithi094","pid":"28898","root":"/"}}] 2024-09-10T23:36:37.330 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi094", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28898", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.94:0", "nonce": 3371089655, "type": "any" }, "name": { "num": 195593, "type": "client" } }, "id": 195593, "inst": "client.195593 172.21.15.94:0/3371089655", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9978754995770893 }, "state": "open", "uptime": 0.463995252 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi094", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28898", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.94:0", "nonce": 2420181232, "type": "any" }, "name": { "num": 6507, "type": "client" } }, "id": 6507, "inst": "client.6507 172.21.15.94:0/2420181232", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9978754995770893 }, "state": "open", "uptime": 0.463995252 } ] 2024-09-10T23:36:37.330 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T23:36:37.330 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:36:37.330 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:36:37.424 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:36:37.508 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:36:37.509 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:36:37.509 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:36:37.509 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:37.517 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:36:37.517 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:36:37.518 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:36:37.518 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:37.526 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:36:37.526 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:36:37.527 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:36:37.527 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:37.569 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:36:37.569 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:36:37.569 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:36:37.570 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:37.578 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:36:37.578 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:37.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad1a200640 1 -- 172.21.15.94:0/392190730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facfc0051d0 msgr2=0x7facfc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:37.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad1a200640 1 --2- 172.21.15.94:0/392190730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facfc0051d0 0x7facfc00b9d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fad0c001dc0 tx=0x7fad0c01cb30 comp rx=0 tx=0).stop 2024-09-10T23:36:37.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad1a200640 1 -- 172.21.15.94:0/392190730 shutdown_connections 2024-09-10T23:36:37.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad1a200640 1 --2- 172.21.15.94:0/392190730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facfc0051d0 0x7facfc00b9d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:37.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad1a200640 1 -- 172.21.15.94:0/392190730 >> 172.21.15.94:0/392190730 conn(0x7fad1409aa70 msgr2=0x7fad1409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:37.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad1a200640 1 -- 172.21.15.94:0/392190730 shutdown_connections 2024-09-10T23:36:37.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad1a200640 1 -- 172.21.15.94:0/392190730 wait complete. 2024-09-10T23:36:37.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad1a200640 1 Processor -- start 2024-09-10T23:36:37.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad1a200640 1 -- start start 2024-09-10T23:36:37.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad1a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facfc0051d0 0x7fad140c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:37.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad1a200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad140b53e0 0x7fad140b7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:37.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad1a200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad141f2f80 0x7fad14156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:37.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad1a200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad140b34d0 con 0x7facfc0051d0 2024-09-10T23:36:37.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad1a200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fad140b3350 con 0x7fad141f2f80 2024-09-10T23:36:37.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad1a200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fad140d01e0 con 0x7fad140b53e0 2024-09-10T23:36:37.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad18e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facfc0051d0 0x7fad140c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:37.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad18e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facfc0051d0 0x7fad140c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48816/0 (socket says 172.21.15.94:48816) 2024-09-10T23:36:37.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad18e00640 1 -- 172.21.15.94:0/2368905075 learned_addr learned my addr 172.21.15.94:0/2368905075 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:37.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad13e00640 1 --2- 172.21.15.94:0/2368905075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad140b53e0 0x7fad140b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:37.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad19800640 1 --2- 172.21.15.94:0/2368905075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad141f2f80 0x7fad14156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:37.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad18e00640 1 -- 172.21.15.94:0/2368905075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad140b53e0 msgr2=0x7fad140b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:37.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad18e00640 1 --2- 172.21.15.94:0/2368905075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad140b53e0 0x7fad140b7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:37.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad18e00640 1 -- 172.21.15.94:0/2368905075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad141f2f80 msgr2=0x7fad14156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:37.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad18e00640 1 --2- 172.21.15.94:0/2368905075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad141f2f80 0x7fad14156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:37.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.762+0000 7fad18e00640 1 -- 172.21.15.94:0/2368905075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad140b58e0 con 0x7facfc0051d0 2024-09-10T23:36:37.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.766+0000 7fad18e00640 1 --2- 172.21.15.94:0/2368905075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facfc0051d0 0x7fad140c8370 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fad0800e660 tx=0x7fad0800ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:37.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.766+0000 7fad10c00640 1 -- 172.21.15.94:0/2368905075 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad08015020 con 0x7facfc0051d0 2024-09-10T23:36:37.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.766+0000 7fad10c00640 1 -- 172.21.15.94:0/2368905075 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad0800ca00 con 0x7facfc0051d0 2024-09-10T23:36:37.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.766+0000 7fad10c00640 1 -- 172.21.15.94:0/2368905075 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0800ce50 con 0x7facfc0051d0 2024-09-10T23:36:37.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.766+0000 7fad1a200640 1 -- 172.21.15.94:0/2368905075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1415a990 con 0x7facfc0051d0 2024-09-10T23:36:37.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.766+0000 7fad1a200640 1 -- 172.21.15.94:0/2368905075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad1415af50 con 0x7facfc0051d0 2024-09-10T23:36:37.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.766+0000 7fad1a200640 1 -- 172.21.15.94:0/2368905075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad141556e0 con 0x7facfc0051d0 2024-09-10T23:36:37.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.766+0000 7fad10c00640 1 -- 172.21.15.94:0/2368905075 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 14) ==== 48364+0+0 (secure 0 0 0) 0x7fad0801a400 con 0x7facfc0051d0 2024-09-10T23:36:37.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.766+0000 7fad10c00640 1 --2- 172.21.15.94:0/2368905075 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7facf403b3a0 0x7facf403d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:37.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.766+0000 7fad10c00640 1 -- 172.21.15.94:0/2368905075 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6268+0+0 (secure 0 0 0) 0x7fad08050200 con 0x7facfc0051d0 2024-09-10T23:36:37.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.766+0000 7fad13e00640 1 --2- 172.21.15.94:0/2368905075 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7facf403b3a0 0x7facf403d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:37.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.766+0000 7fad13e00640 1 --2- 172.21.15.94:0/2368905075 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7facf403b3a0 0x7facf403d860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad140c8ac0 tx=0x7fad000091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:37.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.770+0000 7fad10c00640 1 -- 172.21.15.94:0/2368905075 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fad0801f020 con 0x7facfc0051d0 2024-09-10T23:36:37.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.974+0000 7fad1a200640 1 -- 172.21.15.94:0/2368905075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad140afe10 con 0x7facfc0051d0 2024-09-10T23:36:37.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.974+0000 7fad10c00640 1 -- 172.21.15.94:0/2368905075 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5238 (secure 0 0 0) 0x7fad08021d10 con 0x7facfc0051d0 2024-09-10T23:36:37.976 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:37.976 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":44,"btime":"2024-09-10T23:36:31:051584+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3320339747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3320339747},{"type":"v1","addr":"172.21.15.94:6834","nonce":3320339747}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3123300514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3123300514},{"type":"v1","addr":"172.21.15.94:6835","nonce":3123300514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/339523200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":339523200},{"type":"v1","addr":"172.21.15.181:6837","nonce":339523200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:36:19.404249+0000","modified":"2024-09-10T23:36:22.404999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195389":{"gid":195389,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/2999113739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2999113739},{"type":"v1","addr":"172.21.15.181:6836","nonce":2999113739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195389,"qdb_cluster":[195389]},"id":5}]} 2024-09-10T23:36:37.976 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 44 2024-09-10T23:36:37.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.978+0000 7fad1a200640 1 -- 172.21.15.94:0/2368905075 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7facf403b3a0 msgr2=0x7facf403d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:37.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.978+0000 7fad1a200640 1 --2- 172.21.15.94:0/2368905075 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7facf403b3a0 0x7facf403d860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad140c8ac0 tx=0x7fad000091e0 comp rx=0 tx=0).stop 2024-09-10T23:36:37.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.978+0000 7fad1a200640 1 -- 172.21.15.94:0/2368905075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facfc0051d0 msgr2=0x7fad140c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:37.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.978+0000 7fad1a200640 1 --2- 172.21.15.94:0/2368905075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facfc0051d0 0x7fad140c8370 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fad0800e660 tx=0x7fad0800ea60 comp rx=0 tx=0).stop 2024-09-10T23:36:37.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.978+0000 7fad1a200640 1 -- 172.21.15.94:0/2368905075 shutdown_connections 2024-09-10T23:36:37.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.978+0000 7fad1a200640 1 --2- 172.21.15.94:0/2368905075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad141f2f80 0x7fad14156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:37.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.978+0000 7fad1a200640 1 --2- 172.21.15.94:0/2368905075 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7facf403b3a0 0x7facf403d860 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:37.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.978+0000 7fad1a200640 1 --2- 172.21.15.94:0/2368905075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad140b53e0 0x7fad140b7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:37.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.978+0000 7fad1a200640 1 --2- 172.21.15.94:0/2368905075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facfc0051d0 0x7fad140c8370 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:37.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.978+0000 7fad1a200640 1 -- 172.21.15.94:0/2368905075 >> 172.21.15.94:0/2368905075 conn(0x7fad1409aa70 msgr2=0x7fad1409ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:37.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.978+0000 7fad1a200640 1 -- 172.21.15.94:0/2368905075 shutdown_connections 2024-09-10T23:36:37.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:37.978+0000 7fad1a200640 1 -- 172.21.15.94:0/2368905075 wait complete. 2024-09-10T23:36:37.998 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:38.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.138+0000 7f6b30000640 1 -- 172.21.15.94:0/2749230680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b280b0320 msgr2=0x7f6b28166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:38.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.138+0000 7f6b30000640 1 --2- 172.21.15.94:0/2749230680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b280b0320 0x7f6b28166aa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009bc0 tx=0x7f6b2001ca80 comp rx=0 tx=0).stop 2024-09-10T23:36:38.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.138+0000 7f6b30000640 1 -- 172.21.15.94:0/2749230680 shutdown_connections 2024-09-10T23:36:38.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.138+0000 7f6b30000640 1 --2- 172.21.15.94:0/2749230680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b280b0320 0x7f6b28166aa0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.138+0000 7f6b30000640 1 -- 172.21.15.94:0/2749230680 >> 172.21.15.94:0/2749230680 conn(0x7f6b280d1c60 msgr2=0x7f6b280d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:38.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.138+0000 7f6b30000640 1 -- 172.21.15.94:0/2749230680 shutdown_connections 2024-09-10T23:36:38.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.138+0000 7f6b30000640 1 -- 172.21.15.94:0/2749230680 wait complete. 2024-09-10T23:36:38.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b30000640 1 Processor -- start 2024-09-10T23:36:38.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b30000640 1 -- start start 2024-09-10T23:36:38.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b30000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b280b0320 0x7f6b28150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:38.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b30000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b280b2bb0 0x7f6b280cef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:38.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b30000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b2815f5c0 0x7f6b2815e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:38.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b30000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28169f00 con 0x7f6b280b2bb0 2024-09-10T23:36:38.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b30000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28169d80 con 0x7f6b2815f5c0 2024-09-10T23:36:38.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b30000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6b2816a080 con 0x7f6b280b0320 2024-09-10T23:36:38.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b2cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b280b2bb0 0x7f6b280cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:38.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b2d600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b280b0320 0x7f6b28150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:38.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b2d600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b280b0320 0x7f6b28150c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:46348/0 (socket says 172.21.15.94:46348) 2024-09-10T23:36:38.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b2e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b2815f5c0 0x7f6b2815e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:38.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b2d600640 1 -- 172.21.15.94:0/3668331447 learned_addr learned my addr 172.21.15.94:0/3668331447 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:38.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b2cc00640 1 -- 172.21.15.94:0/3668331447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b280b0320 msgr2=0x7f6b28150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:38.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b2cc00640 1 --2- 172.21.15.94:0/3668331447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b280b0320 0x7f6b28150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b2cc00640 1 -- 172.21.15.94:0/3668331447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b2815f5c0 msgr2=0x7f6b2815e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:38.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b2cc00640 1 --2- 172.21.15.94:0/3668331447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b2815f5c0 0x7f6b2815e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b2cc00640 1 -- 172.21.15.94:0/3668331447 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b28151640 con 0x7f6b280b2bb0 2024-09-10T23:36:38.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b2d600640 1 --2- 172.21.15.94:0/3668331447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b280b0320 0x7f6b28150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:36:38.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b2cc00640 1 --2- 172.21.15.94:0/3668331447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b280b2bb0 0x7f6b280cef80 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400c460 tx=0x7f6b1400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:38.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b25600640 1 -- 172.21.15.94:0/3668331447 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14011790 con 0x7f6b280b2bb0 2024-09-10T23:36:38.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b25600640 1 -- 172.21.15.94:0/3668331447 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b14017070 con 0x7f6b280b2bb0 2024-09-10T23:36:38.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b25600640 1 -- 172.21.15.94:0/3668331447 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14011ce0 con 0x7f6b280b2bb0 2024-09-10T23:36:38.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b30000640 1 -- 172.21.15.94:0/3668331447 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b280cc8f0 con 0x7f6b280b2bb0 2024-09-10T23:36:38.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.142+0000 7f6b30000640 1 -- 172.21.15.94:0/3668331447 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b280c59e0 con 0x7f6b280b2bb0 2024-09-10T23:36:38.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.146+0000 7f6b30000640 1 -- 172.21.15.94:0/3668331447 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b280cfb60 con 0x7f6b280b2bb0 2024-09-10T23:36:38.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.146+0000 7f6b25600640 1 -- 172.21.15.94:0/3668331447 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 14) ==== 48364+0+0 (secure 0 0 0) 0x7f6b1401c460 con 0x7f6b280b2bb0 2024-09-10T23:36:38.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.146+0000 7f6b25600640 1 --2- 172.21.15.94:0/3668331447 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6b0803b3a0 0x7f6b0803d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:38.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.146+0000 7f6b25600640 1 -- 172.21.15.94:0/3668331447 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6268+0+0 (secure 0 0 0) 0x7f6b1404f7d0 con 0x7f6b280b2bb0 2024-09-10T23:36:38.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.146+0000 7f6b2d600640 1 --2- 172.21.15.94:0/3668331447 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6b0803b3a0 0x7f6b0803d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:38.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.146+0000 7f6b2d600640 1 --2- 172.21.15.94:0/3668331447 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6b0803b3a0 0x7f6b0803d860 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c000c10 tx=0x7f6b1c009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:38.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.150+0000 7f6b25600640 1 -- 172.21.15.94:0/3668331447 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6b14010cd0 con 0x7f6b280b2bb0 2024-09-10T23:36:38.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.346+0000 7f6b30000640 1 -- 172.21.15.94:0/3668331447 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b280b4620 con 0x7f6b280b2bb0 2024-09-10T23:36:38.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.346+0000 7f6b25600640 1 -- 172.21.15.94:0/3668331447 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5238 (secure 0 0 0) 0x7f6b14012420 con 0x7f6b280b2bb0 2024-09-10T23:36:38.349 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:38.349 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":44,"btime":"2024-09-10T23:36:31:051584+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3320339747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3320339747},{"type":"v1","addr":"172.21.15.94:6834","nonce":3320339747}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3123300514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3123300514},{"type":"v1","addr":"172.21.15.94:6835","nonce":3123300514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/339523200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":339523200},{"type":"v1","addr":"172.21.15.181:6837","nonce":339523200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:36:19.404249+0000","modified":"2024-09-10T23:36:22.404999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195389":{"gid":195389,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/2999113739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2999113739},{"type":"v1","addr":"172.21.15.181:6836","nonce":2999113739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195389,"qdb_cluster":[195389]},"id":5}]} 2024-09-10T23:36:38.350 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 44 2024-09-10T23:36:38.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.350+0000 7f6b30000640 1 -- 172.21.15.94:0/3668331447 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6b0803b3a0 msgr2=0x7f6b0803d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:38.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.350+0000 7f6b30000640 1 --2- 172.21.15.94:0/3668331447 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6b0803b3a0 0x7f6b0803d860 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c000c10 tx=0x7f6b1c009040 comp rx=0 tx=0).stop 2024-09-10T23:36:38.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.350+0000 7f6b30000640 1 -- 172.21.15.94:0/3668331447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b280b2bb0 msgr2=0x7f6b280cef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:38.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.350+0000 7f6b30000640 1 --2- 172.21.15.94:0/3668331447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b280b2bb0 0x7f6b280cef80 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400c460 tx=0x7f6b1400c920 comp rx=0 tx=0).stop 2024-09-10T23:36:38.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.350+0000 7f6b30000640 1 -- 172.21.15.94:0/3668331447 shutdown_connections 2024-09-10T23:36:38.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.350+0000 7f6b30000640 1 --2- 172.21.15.94:0/3668331447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b2815f5c0 0x7f6b2815e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.350+0000 7f6b30000640 1 --2- 172.21.15.94:0/3668331447 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6b0803b3a0 0x7f6b0803d860 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.350+0000 7f6b30000640 1 --2- 172.21.15.94:0/3668331447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b280b2bb0 0x7f6b280cef80 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.350+0000 7f6b30000640 1 --2- 172.21.15.94:0/3668331447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b280b0320 0x7f6b28150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.350+0000 7f6b30000640 1 -- 172.21.15.94:0/3668331447 >> 172.21.15.94:0/3668331447 conn(0x7f6b280d1c60 msgr2=0x7f6b2806f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:38.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.350+0000 7f6b30000640 1 -- 172.21.15.94:0/3668331447 shutdown_connections 2024-09-10T23:36:38.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.350+0000 7f6b30000640 1 -- 172.21.15.94:0/3668331447 wait complete. 2024-09-10T23:36:38.370 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:36:38.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.510+0000 7f4059c00640 1 -- 172.21.15.94:0/3064387553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f405415ecb0 msgr2=0x7f405406f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:38.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.510+0000 7f4059c00640 1 --2- 172.21.15.94:0/3064387553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f405415ecb0 0x7f405406f4b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f404c009c80 tx=0x7f404c01cd70 comp rx=0 tx=0).stop 2024-09-10T23:36:38.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.510+0000 7f4059c00640 1 -- 172.21.15.94:0/3064387553 shutdown_connections 2024-09-10T23:36:38.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.510+0000 7f4059c00640 1 --2- 172.21.15.94:0/3064387553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f405415ecb0 0x7f405406f4b0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.510+0000 7f4059c00640 1 --2- 172.21.15.94:0/3064387553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40540d04b0 0x7f405415e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.510+0000 7f4059c00640 1 -- 172.21.15.94:0/3064387553 >> 172.21.15.94:0/3064387553 conn(0x7f405406cdf0 msgr2=0x7f405406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:38.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.510+0000 7f4059c00640 1 -- 172.21.15.94:0/3064387553 shutdown_connections 2024-09-10T23:36:38.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.510+0000 7f4059c00640 1 -- 172.21.15.94:0/3064387553 wait complete. 2024-09-10T23:36:38.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.510+0000 7f4059c00640 1 Processor -- start 2024-09-10T23:36:38.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.510+0000 7f4059c00640 1 -- start start 2024-09-10T23:36:38.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.510+0000 7f4059c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054073ff0 0x7f40541fb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:38.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.510+0000 7f4059c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40540d04b0 0x7f40541445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:38.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.510+0000 7f4059c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f405415ecb0 0x7f40540cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:38.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.510+0000 7f4059c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4054167930 con 0x7f40540d04b0 2024-09-10T23:36:38.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.510+0000 7f4059c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f40541677b0 con 0x7f4054073ff0 2024-09-10T23:36:38.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.510+0000 7f4059c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f4054167ab0 con 0x7f405415ecb0 2024-09-10T23:36:38.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f404a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40540d04b0 0x7f40541445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:38.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f404a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40540d04b0 0x7f40541445c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48836/0 (socket says 172.21.15.94:48836) 2024-09-10T23:36:38.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f404a000640 1 -- 172.21.15.94:0/532040357 learned_addr learned my addr 172.21.15.94:0/532040357 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:38.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f4052a00640 1 --2- 172.21.15.94:0/532040357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054073ff0 0x7f40541fb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f4053400640 1 --2- 172.21.15.94:0/532040357 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f405415ecb0 0x7f40540cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f404a000640 1 -- 172.21.15.94:0/532040357 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f405415ecb0 msgr2=0x7f40540cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f404a000640 1 --2- 172.21.15.94:0/532040357 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f405415ecb0 0x7f40540cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f404a000640 1 -- 172.21.15.94:0/532040357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054073ff0 msgr2=0x7f40541fb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f404a000640 1 --2- 172.21.15.94:0/532040357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054073ff0 0x7f40541fb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f404a000640 1 -- 172.21.15.94:0/532040357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4054150bf0 con 0x7f40540d04b0 2024-09-10T23:36:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f404a000640 1 --2- 172.21.15.94:0/532040357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40540d04b0 0x7f40541445c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f403c00d950 tx=0x7f403c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f404b400640 1 -- 172.21.15.94:0/532040357 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c017070 con 0x7f40540d04b0 2024-09-10T23:36:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f404b400640 1 -- 172.21.15.94:0/532040357 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f403c009d20 con 0x7f40540d04b0 2024-09-10T23:36:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f4059c00640 1 -- 172.21.15.94:0/532040357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4054163070 con 0x7f40540d04b0 2024-09-10T23:36:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f404b400640 1 -- 172.21.15.94:0/532040357 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c0106c0 con 0x7f40540d04b0 2024-09-10T23:36:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f4059c00640 1 -- 172.21.15.94:0/532040357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4054163630 con 0x7f40540d04b0 2024-09-10T23:36:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f404b400640 1 -- 172.21.15.94:0/532040357 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 14) ==== 48364+0+0 (secure 0 0 0) 0x7f403c010860 con 0x7f40540d04b0 2024-09-10T23:36:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f4059c00640 1 -- 172.21.15.94:0/532040357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f405415e770 con 0x7f40540d04b0 2024-09-10T23:36:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f404b400640 1 --2- 172.21.15.94:0/532040357 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f403003b3a0 0x7f403003d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f404b400640 1 -- 172.21.15.94:0/532040357 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6268+0+0 (secure 0 0 0) 0x7f403c04fde0 con 0x7f40540d04b0 2024-09-10T23:36:38.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f4052a00640 1 --2- 172.21.15.94:0/532040357 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f403003b3a0 0x7f403003d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:38.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.514+0000 7f4052a00640 1 --2- 172.21.15.94:0/532040357 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f403003b3a0 0x7f403003d860 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4044000f70 tx=0x7f4044009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:38.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.518+0000 7f404b400640 1 -- 172.21.15.94:0/532040357 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4054150bf0 con 0x7f40540d04b0 2024-09-10T23:36:38.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.686+0000 7f4059c00640 1 -- 172.21.15.94:0/532040357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40540b40a0 con 0x7f40540d04b0 2024-09-10T23:36:38.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.686+0000 7f404b400640 1 -- 172.21.15.94:0/532040357 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+17479 (secure 0 0 0) 0x7f403c00ee10 con 0x7f40540d04b0 2024-09-10T23:36:38.688 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:38.688 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":70,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:36:36.718602+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:36:17.484207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T23:36:18.356053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1922952906":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/2150218902":"2024-09-11T23:36:36.718561+0000","172.21.15.181:6801/3310396812":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/2484932838":"2024-09-11T23:36:36.718561+0000","172.21.15.181:6800/3310396812":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/3942792329":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/749967229":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/1339348859":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/2211863778":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/2614085240":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/1346366560":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/1584210907":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/2231303511":"2024-09-11T23:36:36.718561+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:36:38.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.690+0000 7f4059c00640 1 -- 172.21.15.94:0/532040357 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f403003b3a0 msgr2=0x7f403003d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:38.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.690+0000 7f4059c00640 1 --2- 172.21.15.94:0/532040357 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f403003b3a0 0x7f403003d860 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4044000f70 tx=0x7f4044009040 comp rx=0 tx=0).stop 2024-09-10T23:36:38.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.690+0000 7f4059c00640 1 -- 172.21.15.94:0/532040357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40540d04b0 msgr2=0x7f40541445c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:38.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.690+0000 7f4059c00640 1 --2- 172.21.15.94:0/532040357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40540d04b0 0x7f40541445c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f403c00d950 tx=0x7f403c00de10 comp rx=0 tx=0).stop 2024-09-10T23:36:38.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.690+0000 7f4059c00640 1 -- 172.21.15.94:0/532040357 shutdown_connections 2024-09-10T23:36:38.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.690+0000 7f4059c00640 1 --2- 172.21.15.94:0/532040357 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f403003b3a0 0x7f403003d860 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.690+0000 7f4059c00640 1 --2- 172.21.15.94:0/532040357 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f405415ecb0 0x7f40540cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.690+0000 7f4059c00640 1 --2- 172.21.15.94:0/532040357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40540d04b0 0x7f40541445c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.690+0000 7f4059c00640 1 --2- 172.21.15.94:0/532040357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054073ff0 0x7f40541fb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.690+0000 7f4059c00640 1 -- 172.21.15.94:0/532040357 >> 172.21.15.94:0/532040357 conn(0x7f405406cdf0 msgr2=0x7f4054150df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:38.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.690+0000 7f4059c00640 1 -- 172.21.15.94:0/532040357 shutdown_connections 2024-09-10T23:36:38.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.690+0000 7f4059c00640 1 -- 172.21.15.94:0/532040357 wait complete. 2024-09-10T23:36:38.713 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T23:36:38.713 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:38.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.858+0000 7ff44ce00640 1 -- 172.21.15.94:0/1605040341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff448073ff0 msgr2=0x7ff4480cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:38.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.858+0000 7ff44ce00640 1 --2- 172.21.15.94:0/1605040341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff448073ff0 0x7ff4480cfda0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff438009840 tx=0x7ff43801c820 comp rx=0 tx=0).stop 2024-09-10T23:36:38.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.858+0000 7ff44ce00640 1 -- 172.21.15.94:0/1605040341 shutdown_connections 2024-09-10T23:36:38.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.858+0000 7ff44ce00640 1 --2- 172.21.15.94:0/1605040341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff448073ff0 0x7ff4480cfda0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.858+0000 7ff44ce00640 1 -- 172.21.15.94:0/1605040341 >> 172.21.15.94:0/1605040341 conn(0x7ff44806cdf0 msgr2=0x7ff44806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:38.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.858+0000 7ff44ce00640 1 -- 172.21.15.94:0/1605040341 shutdown_connections 2024-09-10T23:36:38.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.858+0000 7ff44ce00640 1 -- 172.21.15.94:0/1605040341 wait complete. 2024-09-10T23:36:38.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.858+0000 7ff44ce00640 1 Processor -- start 2024-09-10T23:36:38.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.858+0000 7ff44ce00640 1 -- start start 2024-09-10T23:36:38.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.858+0000 7ff44ce00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff448073ff0 0x7ff44814f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:38.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff44ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4480d02e0 0x7ff448153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:38.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff44ce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff44815ee80 0x7ff448158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:38.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff44ce00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff448168df0 con 0x7ff44815ee80 2024-09-10T23:36:38.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff44ce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff448168c70 con 0x7ff4480d02e0 2024-09-10T23:36:38.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff44ce00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff448168f70 con 0x7ff448073ff0 2024-09-10T23:36:38.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff446800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff44815ee80 0x7ff448158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:38.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff446800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff44815ee80 0x7ff448158940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48852/0 (socket says 172.21.15.94:48852) 2024-09-10T23:36:38.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff43d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4480d02e0 0x7ff448153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:38.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff446800640 1 -- 172.21.15.94:0/3143269820 learned_addr learned my addr 172.21.15.94:0/3143269820 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:38.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff445e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff448073ff0 0x7ff44814f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:38.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff445e00640 1 -- 172.21.15.94:0/3143269820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4480d02e0 msgr2=0x7ff448153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:38.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff445e00640 1 --2- 172.21.15.94:0/3143269820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4480d02e0 0x7ff448153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff445e00640 1 -- 172.21.15.94:0/3143269820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff44815ee80 msgr2=0x7ff448158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:38.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff445e00640 1 --2- 172.21.15.94:0/3143269820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff44815ee80 0x7ff448158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:38.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff445e00640 1 -- 172.21.15.94:0/3143269820 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4481546b0 con 0x7ff448073ff0 2024-09-10T23:36:38.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff43d400640 1 --2- 172.21.15.94:0/3143269820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4480d02e0 0x7ff448153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:36:38.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff446800640 1 --2- 172.21.15.94:0/3143269820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff44815ee80 0x7ff448158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:36:38.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff445e00640 1 --2- 172.21.15.94:0/3143269820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff448073ff0 0x7ff44814f090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff4380ac000 tx=0x7ff4380a5650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:38.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff43ea00640 1 -- 172.21.15.94:0/3143269820 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4380b2040 con 0x7ff448073ff0 2024-09-10T23:36:38.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff43ea00640 1 -- 172.21.15.94:0/3143269820 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff438004470 con 0x7ff448073ff0 2024-09-10T23:36:38.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff43ea00640 1 -- 172.21.15.94:0/3143269820 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438005280 con 0x7ff448073ff0 2024-09-10T23:36:38.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff44ce00640 1 -- 172.21.15.94:0/3143269820 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff448154990 con 0x7ff448073ff0 2024-09-10T23:36:38.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff44ce00640 1 -- 172.21.15.94:0/3143269820 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff44814f920 con 0x7ff448073ff0 2024-09-10T23:36:38.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff44ce00640 1 -- 172.21.15.94:0/3143269820 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff448154bd0 con 0x7ff448073ff0 2024-09-10T23:36:38.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff43ea00640 1 -- 172.21.15.94:0/3143269820 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 14) ==== 48364+0+0 (secure 0 0 0) 0x7ff4380b5070 con 0x7ff448073ff0 2024-09-10T23:36:38.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff43ea00640 1 --2- 172.21.15.94:0/3143269820 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff42403b3d0 0x7ff42403d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:38.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff43ea00640 1 -- 172.21.15.94:0/3143269820 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(70..70 src has 1..70) ==== 6268+0+0 (secure 0 0 0) 0x7ff4380e91e0 con 0x7ff448073ff0 2024-09-10T23:36:38.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.862+0000 7ff43d400640 1 --2- 172.21.15.94:0/3143269820 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff42403b3d0 0x7ff42403d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:38.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.866+0000 7ff43d400640 1 --2- 172.21.15.94:0/3143269820 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff42403b3d0 0x7ff42403d890 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff448159610 tx=0x7ff430007cd0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:38.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:38.870+0000 7ff43ea00640 1 -- 172.21.15.94:0/3143269820 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff4380bc060 con 0x7ff448073ff0 2024-09-10T23:36:39.026 INFO:tasks.ceph.mgr.x.smithi181.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-10T23:36:39.040 INFO:tasks.ceph.mgr.x.smithi181.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-10T23:36:39.040 INFO:tasks.ceph.mgr.x.smithi181.stderr: from numpy import show_config as show_numpy_config 2024-09-10T23:36:39.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.066+0000 7ff44ce00640 1 -- 172.21.15.94:0/3143269820 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4480cfda0 con 0x7ff448073ff0 2024-09-10T23:36:39.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.066+0000 7ff43ea00640 1 -- 172.21.15.94:0/3143269820 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5238 (secure 0 0 0) 0x7ff438016410 con 0x7ff448073ff0 2024-09-10T23:36:39.068 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:39.068 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":44,"btime":"2024-09-10T23:36:31:051584+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3320339747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3320339747},{"type":"v1","addr":"172.21.15.94:6834","nonce":3320339747}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3123300514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3123300514},{"type":"v1","addr":"172.21.15.94:6835","nonce":3123300514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/339523200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":339523200},{"type":"v1","addr":"172.21.15.181:6837","nonce":339523200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:36:19.404249+0000","modified":"2024-09-10T23:36:22.404999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195389":{"gid":195389,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/2999113739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2999113739},{"type":"v1","addr":"172.21.15.181:6836","nonce":2999113739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195389,"qdb_cluster":[195389]},"id":5}]} 2024-09-10T23:36:39.068 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 44 2024-09-10T23:36:39.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.070+0000 7ff44ce00640 1 -- 172.21.15.94:0/3143269820 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff42403b3d0 msgr2=0x7ff42403d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:39.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.070+0000 7ff44ce00640 1 --2- 172.21.15.94:0/3143269820 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff42403b3d0 0x7ff42403d890 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff448159610 tx=0x7ff430007cd0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.070+0000 7ff44ce00640 1 -- 172.21.15.94:0/3143269820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff448073ff0 msgr2=0x7ff44814f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:39.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.070+0000 7ff44ce00640 1 --2- 172.21.15.94:0/3143269820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff448073ff0 0x7ff44814f090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff4380ac000 tx=0x7ff4380a5650 comp rx=0 tx=0).stop 2024-09-10T23:36:39.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.070+0000 7ff44ce00640 1 -- 172.21.15.94:0/3143269820 shutdown_connections 2024-09-10T23:36:39.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.070+0000 7ff44ce00640 1 --2- 172.21.15.94:0/3143269820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4480d02e0 0x7ff448153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.070+0000 7ff44ce00640 1 --2- 172.21.15.94:0/3143269820 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff42403b3d0 0x7ff42403d890 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.070+0000 7ff44ce00640 1 --2- 172.21.15.94:0/3143269820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff44815ee80 0x7ff448158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.070+0000 7ff44ce00640 1 --2- 172.21.15.94:0/3143269820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff448073ff0 0x7ff44814f090 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.070+0000 7ff44ce00640 1 -- 172.21.15.94:0/3143269820 >> 172.21.15.94:0/3143269820 conn(0x7ff44806cdf0 msgr2=0x7ff4480af640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.070+0000 7ff44ce00640 1 -- 172.21.15.94:0/3143269820 shutdown_connections 2024-09-10T23:36:39.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.070+0000 7ff44ce00640 1 -- 172.21.15.94:0/3143269820 wait complete. 2024-09-10T23:36:39.090 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:39.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.230+0000 7f993ca00640 1 -- 172.21.15.94:0/780804466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99380cdea0 msgr2=0x7f993815ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:39.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.230+0000 7f993ca00640 1 --2- 172.21.15.94:0/780804466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99380cdea0 0x7f993815ec00 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f993806dcc0 tx=0x7f992001c830 comp rx=0 tx=0).stop 2024-09-10T23:36:39.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.230+0000 7f993ca00640 1 -- 172.21.15.94:0/780804466 shutdown_connections 2024-09-10T23:36:39.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.230+0000 7f993ca00640 1 --2- 172.21.15.94:0/780804466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f993815f140 0x7f993815f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.230+0000 7f993ca00640 1 --2- 172.21.15.94:0/780804466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99380cdea0 0x7f993815ec00 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.230+0000 7f993ca00640 1 --2- 172.21.15.94:0/780804466 >> v2:172.21.15.181:3300/0 conn(0x7f9938073970 0x7f9938073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.230+0000 7f993ca00640 1 -- 172.21.15.94:0/780804466 >> 172.21.15.94:0/780804466 conn(0x7f99380cc900 msgr2=0x7f99380ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:39.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f993ca00640 1 -- 172.21.15.94:0/780804466 shutdown_connections 2024-09-10T23:36:39.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f993ca00640 1 -- 172.21.15.94:0/780804466 wait complete. 2024-09-10T23:36:39.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f993ca00640 1 Processor -- start 2024-09-10T23:36:39.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f993ca00640 1 -- start start 2024-09-10T23:36:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f993ca00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9938073970 0x7f993809d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f993ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99380cdea0 0x7f99381fa540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f993ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f993815f140 0x7f99381fae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f993ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f99380cb7a0 con 0x7f99380cdea0 2024-09-10T23:36:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f993ca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f99380cb620 con 0x7f993815f140 2024-09-10T23:36:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f993ca00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f99380cb920 con 0x7f9938073970 2024-09-10T23:36:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f992d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99380cdea0 0x7f99381fa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f992d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99380cdea0 0x7f99381fa540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48866/0 (socket says 172.21.15.94:48866) 2024-09-10T23:36:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f992d400640 1 -- 172.21.15.94:0/1271396702 learned_addr learned my addr 172.21.15.94:0/1271396702 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f9935e00640 1 --2- 172.21.15.94:0/1271396702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9938073970 0x7f993809d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:39.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f9936800640 1 --2- 172.21.15.94:0/1271396702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f993815f140 0x7f99381fae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f992d400640 1 -- 172.21.15.94:0/1271396702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9938073970 msgr2=0x7f993809d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f992d400640 1 --2- 172.21.15.94:0/1271396702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9938073970 0x7f993809d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f992d400640 1 -- 172.21.15.94:0/1271396702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f993815f140 msgr2=0x7f99381fae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f992d400640 1 --2- 172.21.15.94:0/1271396702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f993815f140 0x7f99381fae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f992d400640 1 -- 172.21.15.94:0/1271396702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993809b5b0 con 0x7f99380cdea0 2024-09-10T23:36:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f9936800640 1 --2- 172.21.15.94:0/1271396702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f993815f140 0x7f99381fae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.234+0000 7f992d400640 1 --2- 172.21.15.94:0/1271396702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99380cdea0 0x7f99381fa540 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f99200b0040 tx=0x7f99200a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.238+0000 7f992ea00640 1 -- 172.21.15.94:0/1271396702 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99200b2040 con 0x7f99380cdea0 2024-09-10T23:36:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.238+0000 7f992ea00640 1 -- 172.21.15.94:0/1271396702 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9920005070 con 0x7f99380cdea0 2024-09-10T23:36:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.238+0000 7f992ea00640 1 -- 172.21.15.94:0/1271396702 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99200044f0 con 0x7f99380cdea0 2024-09-10T23:36:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.238+0000 7f993ca00640 1 -- 172.21.15.94:0/1271396702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993809b890 con 0x7f99380cdea0 2024-09-10T23:36:39.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.238+0000 7f993ca00640 1 -- 172.21.15.94:0/1271396702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f993809be50 con 0x7f99380cdea0 2024-09-10T23:36:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.238+0000 7f993ca00640 1 -- 172.21.15.94:0/1271396702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f993815ec00 con 0x7f99380cdea0 2024-09-10T23:36:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.238+0000 7f992ea00640 1 -- 172.21.15.94:0/1271396702 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 15) ==== 48572+0+0 (secure 0 0 0) 0x7f99200054c0 con 0x7f99380cdea0 2024-09-10T23:36:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.238+0000 7f992ea00640 1 --2- 172.21.15.94:0/1271396702 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f991403b5e0 0x7f991403daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:39.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.238+0000 7f992ea00640 1 -- 172.21.15.94:0/1271396702 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6268+0+0 (secure 0 0 0) 0x7f99200e9520 con 0x7f99380cdea0 2024-09-10T23:36:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.238+0000 7f9935e00640 1 --2- 172.21.15.94:0/1271396702 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f991403b5e0 0x7f991403daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:39.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.238+0000 7f9935e00640 1 --2- 172.21.15.94:0/1271396702 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f991403b5e0 0x7f991403daa0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9928009e70 tx=0x7f99280095e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:39.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.242+0000 7f992ea00640 1 -- 172.21.15.94:0/1271396702 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f99200164b0 con 0x7f99380cdea0 2024-09-10T23:36:39.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.438+0000 7f993ca00640 1 -- 172.21.15.94:0/1271396702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f993806c7b0 con 0x7f99380cdea0 2024-09-10T23:36:39.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.442+0000 7f992ea00640 1 -- 172.21.15.94:0/1271396702 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5238 (secure 0 0 0) 0x7f99200b53c0 con 0x7f99380cdea0 2024-09-10T23:36:39.442 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:39.442 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":44,"btime":"2024-09-10T23:36:31:051584+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3320339747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3320339747},{"type":"v1","addr":"172.21.15.94:6834","nonce":3320339747}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3123300514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3123300514},{"type":"v1","addr":"172.21.15.94:6835","nonce":3123300514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/339523200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":339523200},{"type":"v1","addr":"172.21.15.181:6837","nonce":339523200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:36:19.404249+0000","modified":"2024-09-10T23:36:22.404999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195389":{"gid":195389,"name":"d","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/2999113739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2999113739},{"type":"v1","addr":"172.21.15.181:6836","nonce":2999113739}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195389,"qdb_cluster":[195389]},"id":5}]} 2024-09-10T23:36:39.442 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 44 2024-09-10T23:36:39.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.442+0000 7f993ca00640 1 -- 172.21.15.94:0/1271396702 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f991403b5e0 msgr2=0x7f991403daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:39.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.442+0000 7f993ca00640 1 --2- 172.21.15.94:0/1271396702 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f991403b5e0 0x7f991403daa0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9928009e70 tx=0x7f99280095e0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.442+0000 7f993ca00640 1 -- 172.21.15.94:0/1271396702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99380cdea0 msgr2=0x7f99381fa540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.442+0000 7f993ca00640 1 --2- 172.21.15.94:0/1271396702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99380cdea0 0x7f99381fa540 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f99200b0040 tx=0x7f99200a56b0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.442+0000 7f993ca00640 1 -- 172.21.15.94:0/1271396702 shutdown_connections 2024-09-10T23:36:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.442+0000 7f993ca00640 1 --2- 172.21.15.94:0/1271396702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f993815f140 0x7f99381fae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.442+0000 7f993ca00640 1 --2- 172.21.15.94:0/1271396702 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f991403b5e0 0x7f991403daa0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.442+0000 7f993ca00640 1 --2- 172.21.15.94:0/1271396702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99380cdea0 0x7f99381fa540 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.442+0000 7f993ca00640 1 --2- 172.21.15.94:0/1271396702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9938073970 0x7f993809d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.442+0000 7f993ca00640 1 -- 172.21.15.94:0/1271396702 >> 172.21.15.94:0/1271396702 conn(0x7f99380cc900 msgr2=0x7f993815da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.442+0000 7f993ca00640 1 -- 172.21.15.94:0/1271396702 shutdown_connections 2024-09-10T23:36:39.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.442+0000 7f993ca00640 1 -- 172.21.15.94:0/1271396702 wait complete. 2024-09-10T23:36:39.464 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:36:39.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53cd400640 1 -- 172.21.15.94:0/1154805748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53c8071a20 msgr2=0x7f53c80cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53cd400640 1 --2- 172.21.15.94:0/1154805748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53c8071a20 0x7f53c80cf390 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009900 tx=0x7f53b001c7c0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53cd400640 1 -- 172.21.15.94:0/1154805748 shutdown_connections 2024-09-10T23:36:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53cd400640 1 --2- 172.21.15.94:0/1154805748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53c80d1690 0x7f53c80cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53cd400640 1 --2- 172.21.15.94:0/1154805748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53c8071a20 0x7f53c80cf390 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53cd400640 1 -- 172.21.15.94:0/1154805748 >> 172.21.15.94:0/1154805748 conn(0x7f53c80d0b70 msgr2=0x7f53c80d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53cd400640 1 -- 172.21.15.94:0/1154805748 shutdown_connections 2024-09-10T23:36:39.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53cd400640 1 -- 172.21.15.94:0/1154805748 wait complete. 2024-09-10T23:36:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53cd400640 1 Processor -- start 2024-09-10T23:36:39.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53cd400640 1 -- start start 2024-09-10T23:36:39.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53cd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c806cc20 0x7f53c815a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:39.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53cd400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53c8071a20 0x7f53c8155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:39.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53cd400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53c80d1690 0x7f53c814f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:39.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53cd400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53c8168e60 con 0x7f53c8071a20 2024-09-10T23:36:39.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53cd400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f53c8168ce0 con 0x7f53c806cc20 2024-09-10T23:36:39.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53cd400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f53c8168fe0 con 0x7f53c80d1690 2024-09-10T23:36:39.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53bde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53c8071a20 0x7f53c8155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:39.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53bde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53c8071a20 0x7f53c8155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48892/0 (socket says 172.21.15.94:48892) 2024-09-10T23:36:39.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53bde00640 1 -- 172.21.15.94:0/3675937614 learned_addr learned my addr 172.21.15.94:0/3675937614 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53c7200640 1 --2- 172.21.15.94:0/3675937614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53c80d1690 0x7f53c814f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53c6800640 1 --2- 172.21.15.94:0/3675937614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c806cc20 0x7f53c815a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53bde00640 1 -- 172.21.15.94:0/3675937614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53c80d1690 msgr2=0x7f53c814f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53bde00640 1 --2- 172.21.15.94:0/3675937614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53c80d1690 0x7f53c814f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53bde00640 1 -- 172.21.15.94:0/3675937614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c806cc20 msgr2=0x7f53c815a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53bde00640 1 --2- 172.21.15.94:0/3675937614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c806cc20 0x7f53c815a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53bde00640 1 -- 172.21.15.94:0/3675937614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c81fb120 con 0x7f53c8071a20 2024-09-10T23:36:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.606+0000 7f53c6800640 1 --2- 172.21.15.94:0/3675937614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c806cc20 0x7f53c815a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.610+0000 7f53bde00640 1 --2- 172.21.15.94:0/3675937614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53c8071a20 0x7f53c8155bf0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f53b001a3f0 tx=0x7f53b0005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.610+0000 7f53bf400640 1 -- 172.21.15.94:0/3675937614 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b00ac070 con 0x7f53c8071a20 2024-09-10T23:36:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.610+0000 7f53bf400640 1 -- 172.21.15.94:0/3675937614 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53b0004320 con 0x7f53c8071a20 2024-09-10T23:36:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.610+0000 7f53bf400640 1 -- 172.21.15.94:0/3675937614 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b00052a0 con 0x7f53c8071a20 2024-09-10T23:36:39.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.610+0000 7f53cd400640 1 -- 172.21.15.94:0/3675937614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c81fb400 con 0x7f53c8071a20 2024-09-10T23:36:39.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.610+0000 7f53cd400640 1 -- 172.21.15.94:0/3675937614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c81fb9c0 con 0x7f53c8071a20 2024-09-10T23:36:39.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.610+0000 7f53cd400640 1 -- 172.21.15.94:0/3675937614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c806d000 con 0x7f53c8071a20 2024-09-10T23:36:39.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.610+0000 7f53bf400640 1 -- 172.21.15.94:0/3675937614 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 15) ==== 48572+0+0 (secure 0 0 0) 0x7f53b0005440 con 0x7f53c8071a20 2024-09-10T23:36:39.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.610+0000 7f53bf400640 1 --2- 172.21.15.94:0/3675937614 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f53a403b610 0x7f53a403dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:39.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.610+0000 7f53bf400640 1 -- 172.21.15.94:0/3675937614 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6268+0+0 (secure 0 0 0) 0x7f53b00ea350 con 0x7f53c8071a20 2024-09-10T23:36:39.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.610+0000 7f53c6800640 1 --2- 172.21.15.94:0/3675937614 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f53a403b610 0x7f53a403dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:39.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.610+0000 7f53c6800640 1 --2- 172.21.15.94:0/3675937614 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f53a403b610 0x7f53a403dad0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f53b80012f0 tx=0x7f53b8009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:39.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.614+0000 7f53bf400640 1 -- 172.21.15.94:0/3675937614 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f53b00164d0 con 0x7f53c8071a20 2024-09-10T23:36:39.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.782+0000 7f53cd400640 1 -- 172.21.15.94:0/3675937614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53c81fc010 con 0x7f53c8071a20 2024-09-10T23:36:39.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.782+0000 7f53bf400640 1 -- 172.21.15.94:0/3675937614 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+17479 (secure 0 0 0) 0x7f53b00a6690 con 0x7f53c8071a20 2024-09-10T23:36:39.784 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:39.784 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":70,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:36:36.718602+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:36:17.484207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T23:36:18.356053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1922952906":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/2150218902":"2024-09-11T23:36:36.718561+0000","172.21.15.181:6801/3310396812":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/2484932838":"2024-09-11T23:36:36.718561+0000","172.21.15.181:6800/3310396812":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/3942792329":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/749967229":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/1339348859":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/2211863778":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/2614085240":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/1346366560":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/1584210907":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/2231303511":"2024-09-11T23:36:36.718561+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:36:39.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.786+0000 7f53cd400640 1 -- 172.21.15.94:0/3675937614 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f53a403b610 msgr2=0x7f53a403dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:39.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.786+0000 7f53cd400640 1 --2- 172.21.15.94:0/3675937614 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f53a403b610 0x7f53a403dad0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f53b80012f0 tx=0x7f53b8009040 comp rx=0 tx=0).stop 2024-09-10T23:36:39.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.786+0000 7f53cd400640 1 -- 172.21.15.94:0/3675937614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53c8071a20 msgr2=0x7f53c8155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:39.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.786+0000 7f53cd400640 1 --2- 172.21.15.94:0/3675937614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53c8071a20 0x7f53c8155bf0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f53b001a3f0 tx=0x7f53b0005e90 comp rx=0 tx=0).stop 2024-09-10T23:36:39.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.786+0000 7f53cd400640 1 -- 172.21.15.94:0/3675937614 shutdown_connections 2024-09-10T23:36:39.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.786+0000 7f53cd400640 1 --2- 172.21.15.94:0/3675937614 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f53a403b610 0x7f53a403dad0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.786+0000 7f53cd400640 1 --2- 172.21.15.94:0/3675937614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53c80d1690 0x7f53c814f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.786+0000 7f53cd400640 1 --2- 172.21.15.94:0/3675937614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53c8071a20 0x7f53c8155bf0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.786+0000 7f53cd400640 1 --2- 172.21.15.94:0/3675937614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53c806cc20 0x7f53c815a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.786+0000 7f53cd400640 1 -- 172.21.15.94:0/3675937614 >> 172.21.15.94:0/3675937614 conn(0x7f53c80d0b70 msgr2=0x7f53c8165cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:39.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.786+0000 7f53cd400640 1 -- 172.21.15.94:0/3675937614 shutdown_connections 2024-09-10T23:36:39.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.786+0000 7f53cd400640 1 -- 172.21.15.94:0/3675937614 wait complete. 2024-09-10T23:36:39.806 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T23:36:39.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.946+0000 7fabdfe00640 1 -- 172.21.15.94:0/2892381605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe015f140 msgr2=0x7fabe00b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:39.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.946+0000 7fabdfe00640 1 --2- 172.21.15.94:0/2892381605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe015f140 0x7fabe00b1170 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009900 tx=0x7fabd001c850 comp rx=0 tx=0).stop 2024-09-10T23:36:39.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.946+0000 7fabdfe00640 1 -- 172.21.15.94:0/2892381605 shutdown_connections 2024-09-10T23:36:39.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.946+0000 7fabdfe00640 1 --2- 172.21.15.94:0/2892381605 >> v2:172.21.15.181:3300/0 conn(0x7fabe00b2ac0 0x7fabe00b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.946+0000 7fabdfe00640 1 --2- 172.21.15.94:0/2892381605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe015f140 0x7fabe00b1170 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.946+0000 7fabdfe00640 1 -- 172.21.15.94:0/2892381605 >> 172.21.15.94:0/2892381605 conn(0x7fabe00d1b00 msgr2=0x7fabe00d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:39.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.946+0000 7fabdfe00640 1 -- 172.21.15.94:0/2892381605 shutdown_connections 2024-09-10T23:36:39.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.946+0000 7fabdfe00640 1 -- 172.21.15.94:0/2892381605 wait complete. 2024-09-10T23:36:39.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdfe00640 1 Processor -- start 2024-09-10T23:36:39.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdfe00640 1 -- start start 2024-09-10T23:36:39.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdfe00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe00b1bf0 0x7fabe006f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:39.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe00b2ac0 0x7fabe0148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:39.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe015f140 0x7fabe0149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:39.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdfe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0168ad0 con 0x7fabe015f140 2024-09-10T23:36:39.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdfe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0168950 con 0x7fabe00b2ac0 2024-09-10T23:36:39.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdfe00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fabe0168c50 con 0x7fabe00b1bf0 2024-09-10T23:36:39.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe015f140 0x7fabe0149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:39.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdea00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe00b1bf0 0x7fabe006f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:39.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdea00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe00b1bf0 0x7fabe006f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:46426/0 (socket says 172.21.15.94:46426) 2024-09-10T23:36:39.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdea00640 1 -- 172.21.15.94:0/2770675604 learned_addr learned my addr 172.21.15.94:0/2770675604 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:39.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe015f140 0x7fabe0149530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48898/0 (socket says 172.21.15.94:48898) 2024-09-10T23:36:39.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabde000640 1 --2- 172.21.15.94:0/2770675604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe00b2ac0 0x7fabe0148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:39.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdea00640 1 -- 172.21.15.94:0/2770675604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe00b2ac0 msgr2=0x7fabe0148ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:39.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdea00640 1 --2- 172.21.15.94:0/2770675604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe00b2ac0 0x7fabe0148ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdea00640 1 -- 172.21.15.94:0/2770675604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe015f140 msgr2=0x7fabe0149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:39.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdea00640 1 --2- 172.21.15.94:0/2770675604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe015f140 0x7fabe0149530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:39.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdea00640 1 -- 172.21.15.94:0/2770675604 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe01fa900 con 0x7fabe00b1bf0 2024-09-10T23:36:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdea00640 1 --2- 172.21.15.94:0/2770675604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe00b1bf0 0x7fabe006f940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009850 tx=0x7fabd0005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabd6a00640 1 -- 172.21.15.94:0/2770675604 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd00b2040 con 0x7fabe00b1bf0 2024-09-10T23:36:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabd6a00640 1 -- 172.21.15.94:0/2770675604 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabd0004310 con 0x7fabe00b1bf0 2024-09-10T23:36:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdfe00640 1 -- 172.21.15.94:0/2770675604 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe01fabe0 con 0x7fabe00b1bf0 2024-09-10T23:36:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabd6a00640 1 -- 172.21.15.94:0/2770675604 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd0005290 con 0x7fabe00b1bf0 2024-09-10T23:36:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.950+0000 7fabdfe00640 1 -- 172.21.15.94:0/2770675604 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe01fb1a0 con 0x7fabe00b1bf0 2024-09-10T23:36:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.954+0000 7fabdfe00640 1 -- 172.21.15.94:0/2770675604 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe00cfa00 con 0x7fabe00b1bf0 2024-09-10T23:36:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.954+0000 7fabd6a00640 1 -- 172.21.15.94:0/2770675604 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 15) ==== 48572+0+0 (secure 0 0 0) 0x7fabd00bf060 con 0x7fabe00b1bf0 2024-09-10T23:36:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.954+0000 7fabd6a00640 1 --2- 172.21.15.94:0/2770675604 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fabc003b5e0 0x7fabc003daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.954+0000 7fabd6a00640 1 -- 172.21.15.94:0/2770675604 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(70..70 src has 1..70) ==== 6268+0+0 (secure 0 0 0) 0x7fabd00e8fe0 con 0x7fabe00b1bf0 2024-09-10T23:36:39.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.954+0000 7fabde000640 1 --2- 172.21.15.94:0/2770675604 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fabc003b5e0 0x7fabc003daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:39.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.954+0000 7fabde000640 1 --2- 172.21.15.94:0/2770675604 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fabc003b5e0 0x7fabc003daa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fabe0074810 tx=0x7fabcc0091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:39.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:39.958+0000 7fabd6a00640 1 -- 172.21.15.94:0/2770675604 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fabd00164d0 con 0x7fabe00b1bf0 2024-09-10T23:36:40.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.154+0000 7fabdfe00640 1 -- 172.21.15.94:0/2770675604 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fabe00d22f0 con 0x7fabe00b1bf0 2024-09-10T23:36:40.157 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-10T23:36:40.158+0000 7f4045a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:36:40.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.166+0000 7fabd6a00640 1 -- 172.21.15.94:0/2770675604 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) ==== 161+0+0 (secure 0 0 0) 0x7fabd00e7360 con 0x7fabe00b1bf0 2024-09-10T23:36:40.167 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T23:36:40.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.166+0000 7fabdfe00640 1 -- 172.21.15.94:0/2770675604 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fabc003b5e0 msgr2=0x7fabc003daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:40.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.166+0000 7fabdfe00640 1 --2- 172.21.15.94:0/2770675604 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fabc003b5e0 0x7fabc003daa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fabe0074810 tx=0x7fabcc0091e0 comp rx=0 tx=0).stop 2024-09-10T23:36:40.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.166+0000 7fabdfe00640 1 -- 172.21.15.94:0/2770675604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe00b1bf0 msgr2=0x7fabe006f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:40.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.166+0000 7fabdfe00640 1 --2- 172.21.15.94:0/2770675604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe00b1bf0 0x7fabe006f940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009850 tx=0x7fabd0005de0 comp rx=0 tx=0).stop 2024-09-10T23:36:40.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.166+0000 7fabdfe00640 1 -- 172.21.15.94:0/2770675604 shutdown_connections 2024-09-10T23:36:40.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.166+0000 7fabdfe00640 1 --2- 172.21.15.94:0/2770675604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabe00b2ac0 0x7fabe0148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:40.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.166+0000 7fabdfe00640 1 --2- 172.21.15.94:0/2770675604 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fabc003b5e0 0x7fabc003daa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:40.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.166+0000 7fabdfe00640 1 --2- 172.21.15.94:0/2770675604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabe015f140 0x7fabe0149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:40.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.166+0000 7fabdfe00640 1 --2- 172.21.15.94:0/2770675604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabe00b1bf0 0x7fabe006f940 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:40.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.166+0000 7fabdfe00640 1 -- 172.21.15.94:0/2770675604 >> 172.21.15.94:0/2770675604 conn(0x7fabe00d1b00 msgr2=0x7fabe015d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:40.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.166+0000 7fabdfe00640 1 -- 172.21.15.94:0/2770675604 shutdown_connections 2024-09-10T23:36:40.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.166+0000 7fabdfe00640 1 -- 172.21.15.94:0/2770675604 wait complete. 2024-09-10T23:36:40.179 INFO:tasks.ceph.mds.d.smithi181.stderr: -6988> 2024-09-10T23:36:21.986+0000 7fca07a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:36:40.189 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T23:36:40.229 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2024-09-10T23:36:40.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b2e600640 1 -- 172.21.15.94:0/2401269359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b28158790 msgr2=0x7f4b2806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:40.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b2e600640 1 --2- 172.21.15.94:0/2401269359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b28158790 0x7f4b2806fc00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009c80 tx=0x7f4b1801cda0 comp rx=0 tx=0).stop 2024-09-10T23:36:40.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b2e600640 1 -- 172.21.15.94:0/2401269359 shutdown_connections 2024-09-10T23:36:40.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b2e600640 1 --2- 172.21.15.94:0/2401269359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b280c49b0 0x7f4b280cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:40.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b2e600640 1 --2- 172.21.15.94:0/2401269359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b28158790 0x7f4b2806fc00 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:40.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b2e600640 1 -- 172.21.15.94:0/2401269359 >> 172.21.15.94:0/2401269359 conn(0x7f4b280cfaf0 msgr2=0x7f4b280cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:40.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b2e600640 1 -- 172.21.15.94:0/2401269359 shutdown_connections 2024-09-10T23:36:40.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b2e600640 1 -- 172.21.15.94:0/2401269359 wait complete. 2024-09-10T23:36:40.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b2e600640 1 Processor -- start 2024-09-10T23:36:40.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b2e600640 1 -- start start 2024-09-10T23:36:40.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b2e600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b280c49b0 0x7f4b280b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:40.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b2e600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b280cb300 0x7f4b28154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:40.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b2e600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b28158790 0x7f4b281575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:40.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b2e600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28168230 con 0x7f4b28158790 2024-09-10T23:36:40.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b2e600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4b281680b0 con 0x7f4b280cb300 2024-09-10T23:36:40.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b2e600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f4b281683b0 con 0x7f4b280c49b0 2024-09-10T23:36:40.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b27400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b280c49b0 0x7f4b280b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:40.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b1ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b280cb300 0x7f4b28154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:40.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b27e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b28158790 0x7f4b281575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:40.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b27400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b280c49b0 0x7f4b280b5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:46458/0 (socket says 172.21.15.94:46458) 2024-09-10T23:36:40.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.334+0000 7f4b27400640 1 -- 172.21.15.94:0/275313183 learned_addr learned my addr 172.21.15.94:0/275313183 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:40.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b27400640 1 -- 172.21.15.94:0/275313183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b280cb300 msgr2=0x7f4b28154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:40.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b27400640 1 --2- 172.21.15.94:0/275313183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b280cb300 0x7f4b28154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:40.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b27400640 1 -- 172.21.15.94:0/275313183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b28158790 msgr2=0x7f4b281575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:40.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b27400640 1 --2- 172.21.15.94:0/275313183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b28158790 0x7f4b281575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:40.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b27400640 1 -- 172.21.15.94:0/275313183 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b281fb730 con 0x7f4b280c49b0 2024-09-10T23:36:40.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b27e00640 1 --2- 172.21.15.94:0/275313183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b28158790 0x7f4b281575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:36:40.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b27400640 1 --2- 172.21.15.94:0/275313183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b280c49b0 0x7f4b280b5410 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4b18000f90 tx=0x7f4b180092c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:40.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b1fe00640 1 -- 172.21.15.94:0/275313183 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b180bb070 con 0x7f4b280c49b0 2024-09-10T23:36:40.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b1fe00640 1 -- 172.21.15.94:0/275313183 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b18004d40 con 0x7f4b280c49b0 2024-09-10T23:36:40.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b1fe00640 1 -- 172.21.15.94:0/275313183 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b180b2040 con 0x7f4b280c49b0 2024-09-10T23:36:40.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b2e600640 1 -- 172.21.15.94:0/275313183 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b28154960 con 0x7f4b280c49b0 2024-09-10T23:36:40.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b2e600640 1 -- 172.21.15.94:0/275313183 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b281fbc80 con 0x7f4b280c49b0 2024-09-10T23:36:40.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b1fe00640 1 -- 172.21.15.94:0/275313183 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 15) ==== 48572+0+0 (secure 0 0 0) 0x7f4b180b5070 con 0x7f4b280c49b0 2024-09-10T23:36:40.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b2e600640 1 -- 172.21.15.94:0/275313183 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b280cadc0 con 0x7f4b280c49b0 2024-09-10T23:36:40.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b1fe00640 1 --2- 172.21.15.94:0/275313183 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4b0403b5e0 0x7f4b0403daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:40.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b1fe00640 1 -- 172.21.15.94:0/275313183 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(71..71 src has 1..71) ==== 6354+0+0 (secure 0 0 0) 0x7f4b180e9c40 con 0x7f4b280c49b0 2024-09-10T23:36:40.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b1ea00640 1 --2- 172.21.15.94:0/275313183 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4b0403b5e0 0x7f4b0403daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:40.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.338+0000 7f4b1ea00640 1 --2- 172.21.15.94:0/275313183 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4b0403b5e0 0x7f4b0403daa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4b280b51d0 tx=0x7f4b100091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:40.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.342+0000 7f4b1fe00640 1 -- 172.21.15.94:0/275313183 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4b28154960 con 0x7f4b280c49b0 2024-09-10T23:36:40.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:40.542+0000 7f4b2e600640 1 -- 172.21.15.94:0/275313183 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4b280d0850 con 0x7f4b280c49b0 2024-09-10T23:36:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.170+0000 7f4b1fe00640 1 -- 172.21.15.94:0/275313183 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f4b180a5af0 con 0x7f4b280c49b0 2024-09-10T23:36:41.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.174+0000 7f4b1fe00640 1 -- 172.21.15.94:0/275313183 <== mon.2 v2:172.21.15.181:3301/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) ==== 104+0+0 (secure 0 0 0) 0x7f4b180a5e20 con 0x7f4b280c49b0 2024-09-10T23:36:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.174+0000 7f4b2e600640 1 -- 172.21.15.94:0/275313183 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4b0403b5e0 msgr2=0x7f4b0403daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.174+0000 7f4b2e600640 1 --2- 172.21.15.94:0/275313183 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4b0403b5e0 0x7f4b0403daa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4b280b51d0 tx=0x7f4b100091e0 comp rx=0 tx=0).stop 2024-09-10T23:36:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.174+0000 7f4b2e600640 1 -- 172.21.15.94:0/275313183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b280c49b0 msgr2=0x7f4b280b5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:41.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.174+0000 7f4b2e600640 1 --2- 172.21.15.94:0/275313183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b280c49b0 0x7f4b280b5410 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4b18000f90 tx=0x7f4b180092c0 comp rx=0 tx=0).stop 2024-09-10T23:36:41.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.174+0000 7f4b2e600640 1 -- 172.21.15.94:0/275313183 shutdown_connections 2024-09-10T23:36:41.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.174+0000 7f4b2e600640 1 --2- 172.21.15.94:0/275313183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b280cb300 0x7f4b28154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:41.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.174+0000 7f4b2e600640 1 --2- 172.21.15.94:0/275313183 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4b0403b5e0 0x7f4b0403daa0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:41.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.174+0000 7f4b2e600640 1 --2- 172.21.15.94:0/275313183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b28158790 0x7f4b281575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:41.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.174+0000 7f4b2e600640 1 --2- 172.21.15.94:0/275313183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b280c49b0 0x7f4b280b5410 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:41.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.174+0000 7f4b2e600640 1 -- 172.21.15.94:0/275313183 >> 172.21.15.94:0/275313183 conn(0x7f4b280cfaf0 msgr2=0x7f4b28156d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:41.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.174+0000 7f4b2e600640 1 -- 172.21.15.94:0/275313183 shutdown_connections 2024-09-10T23:36:41.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.174+0000 7f4b2e600640 1 -- 172.21.15.94:0/275313183 wait complete. 2024-09-10T23:36:41.197 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T23:36:41.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6acc00640 1 -- 172.21.15.94:0/4108181349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a815bd30 msgr2=0x7fe6a815c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:41.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6acc00640 1 --2- 172.21.15.94:0/4108181349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a815bd30 0x7fe6a815c110 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8073fc0 tx=0x7fe69801ccf0 comp rx=0 tx=0).stop 2024-09-10T23:36:41.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6acc00640 1 -- 172.21.15.94:0/4108181349 shutdown_connections 2024-09-10T23:36:41.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6acc00640 1 --2- 172.21.15.94:0/4108181349 >> v2:172.21.15.181:3300/0 conn(0x7fe6a80ae980 0x7fe6a80d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:41.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6acc00640 1 --2- 172.21.15.94:0/4108181349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a815bd30 0x7fe6a815c110 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:41.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6acc00640 1 -- 172.21.15.94:0/4108181349 >> 172.21.15.94:0/4108181349 conn(0x7fe6a809aa70 msgr2=0x7fe6a809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:41.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6acc00640 1 -- 172.21.15.94:0/4108181349 shutdown_connections 2024-09-10T23:36:41.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6acc00640 1 -- 172.21.15.94:0/4108181349 wait complete. 2024-09-10T23:36:41.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6acc00640 1 Processor -- start 2024-09-10T23:36:41.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6acc00640 1 -- start start 2024-09-10T23:36:41.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6acc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a80ae980 0x7fe6a8144da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:41.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6acc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a80b2ec0 0x7fe6a8143400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:41.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6acc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a815bd30 0x7fe6a80b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:41.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6acc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a8169320 con 0x7fe6a80ae980 2024-09-10T23:36:41.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6acc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a81691a0 con 0x7fe6a815bd30 2024-09-10T23:36:41.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6acc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a81694a0 con 0x7fe6a80b2ec0 2024-09-10T23:36:41.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.342+0000 7fe6a5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a80ae980 0x7fe6a8144da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:41.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe6a5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a80ae980 0x7fe6a8144da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48952/0 (socket says 172.21.15.94:48952) 2024-09-10T23:36:41.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe6a5e00640 1 -- 172.21.15.94:0/1929138672 learned_addr learned my addr 172.21.15.94:0/1929138672 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:41.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe6a6800640 1 --2- 172.21.15.94:0/1929138672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a815bd30 0x7fe6a80b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:41.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe6a5400640 1 --2- 172.21.15.94:0/1929138672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a80b2ec0 0x7fe6a8143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:41.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe6a5e00640 1 -- 172.21.15.94:0/1929138672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a80b2ec0 msgr2=0x7fe6a8143400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:41.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe6a5e00640 1 --2- 172.21.15.94:0/1929138672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a80b2ec0 0x7fe6a8143400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:41.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe6a5e00640 1 -- 172.21.15.94:0/1929138672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a815bd30 msgr2=0x7fe6a80b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe6a5e00640 1 --2- 172.21.15.94:0/1929138672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a815bd30 0x7fe6a80b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe6a5e00640 1 -- 172.21.15.94:0/1929138672 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a8143d00 con 0x7fe6a80ae980 2024-09-10T23:36:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe6a5e00640 1 --2- 172.21.15.94:0/1929138672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a80ae980 0x7fe6a8144da0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe698000f90 tx=0x7fe698009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe69e000640 1 -- 172.21.15.94:0/1929138672 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6980bb070 con 0x7fe6a80ae980 2024-09-10T23:36:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe69e000640 1 -- 172.21.15.94:0/1929138672 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe698004d40 con 0x7fe6a80ae980 2024-09-10T23:36:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe6acc00640 1 -- 172.21.15.94:0/1929138672 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a8143fe0 con 0x7fe6a80ae980 2024-09-10T23:36:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe69e000640 1 -- 172.21.15.94:0/1929138672 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6980b2040 con 0x7fe6a80ae980 2024-09-10T23:36:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe6acc00640 1 -- 172.21.15.94:0/1929138672 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a81445a0 con 0x7fe6a80ae980 2024-09-10T23:36:41.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe6acc00640 1 -- 172.21.15.94:0/1929138672 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a80d1b80 con 0x7fe6a80ae980 2024-09-10T23:36:41.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.346+0000 7fe69e000640 1 -- 172.21.15.94:0/1929138672 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fe6980b5030 con 0x7fe6a80ae980 2024-09-10T23:36:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.350+0000 7fe69e000640 1 --2- 172.21.15.94:0/1929138672 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe688072700 0x7fe688074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.350+0000 7fe69e000640 1 -- 172.21.15.94:0/1929138672 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6354+0+0 (secure 0 0 0) 0x7fe69812d460 con 0x7fe6a80ae980 2024-09-10T23:36:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.350+0000 7fe6a5400640 1 --2- 172.21.15.94:0/1929138672 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe688072700 0x7fe688074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.350+0000 7fe6a5400640 1 --2- 172.21.15.94:0/1929138672 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe688072700 0x7fe688074bc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8072420 tx=0x7fe69400b390 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:41.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.354+0000 7fe69e000640 1 -- 172.21.15.94:0/1929138672 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe698016970 con 0x7fe6a80ae980 2024-09-10T23:36:41.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:41.518+0000 7fe6acc00640 1 -- 172.21.15.94:0/1929138672 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe6a80d28a0 con 0x7fe6a80ae980 2024-09-10T23:36:42.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.166+0000 7fe69e000640 1 -- 172.21.15.94:0/1929138672 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) ==== 181+0+0 (secure 0 0 0) 0x7fe6980f7fb0 con 0x7fe6a80ae980 2024-09-10T23:36:42.170 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-09-10T23:36:42.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.170+0000 7fe6acc00640 1 -- 172.21.15.94:0/1929138672 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe688072700 msgr2=0x7fe688074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:42.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.170+0000 7fe6acc00640 1 --2- 172.21.15.94:0/1929138672 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe688072700 0x7fe688074bc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8072420 tx=0x7fe69400b390 comp rx=0 tx=0).stop 2024-09-10T23:36:42.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.170+0000 7fe6acc00640 1 -- 172.21.15.94:0/1929138672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a80ae980 msgr2=0x7fe6a8144da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:42.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.170+0000 7fe6acc00640 1 --2- 172.21.15.94:0/1929138672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a80ae980 0x7fe6a8144da0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe698000f90 tx=0x7fe698009360 comp rx=0 tx=0).stop 2024-09-10T23:36:42.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.170+0000 7fe6acc00640 1 -- 172.21.15.94:0/1929138672 shutdown_connections 2024-09-10T23:36:42.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.170+0000 7fe6acc00640 1 --2- 172.21.15.94:0/1929138672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a815bd30 0x7fe6a80b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:42.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.170+0000 7fe6acc00640 1 --2- 172.21.15.94:0/1929138672 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe688072700 0x7fe688074bc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:42.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.170+0000 7fe6acc00640 1 --2- 172.21.15.94:0/1929138672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a80b2ec0 0x7fe6a8143400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:42.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.170+0000 7fe6acc00640 1 --2- 172.21.15.94:0/1929138672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a80ae980 0x7fe6a8144da0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:42.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.170+0000 7fe6acc00640 1 -- 172.21.15.94:0/1929138672 >> 172.21.15.94:0/1929138672 conn(0x7fe6a809aa70 msgr2=0x7fe6a815e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:42.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.170+0000 7fe6acc00640 1 -- 172.21.15.94:0/1929138672 shutdown_connections 2024-09-10T23:36:42.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.170+0000 7fe6acc00640 1 -- 172.21.15.94:0/1929138672 wait complete. 2024-09-10T23:36:42.196 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T23:36:42.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ee00640 1 -- 172.21.15.94:0/1200295814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f613c003fe0 msgr2=0x7f613c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:42.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ee00640 1 --2- 172.21.15.94:0/1200295814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f613c003fe0 0x7f613c004810 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f61580ca880 tx=0x7f614001cf60 comp rx=0 tx=0).stop 2024-09-10T23:36:42.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ce00640 1 -- 172.21.15.94:0/1200295814 reap_dead start 2024-09-10T23:36:42.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ee00640 1 -- 172.21.15.94:0/1200295814 shutdown_connections 2024-09-10T23:36:42.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ee00640 1 -- 172.21.15.94:0/1200295814 >> 172.21.15.94:0/1200295814 conn(0x7f61580d02d0 msgr2=0x7f61580d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:42.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ee00640 1 -- 172.21.15.94:0/1200295814 shutdown_connections 2024-09-10T23:36:42.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ee00640 1 -- 172.21.15.94:0/1200295814 wait complete. 2024-09-10T23:36:42.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ee00640 1 Processor -- start 2024-09-10T23:36:42.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ee00640 1 -- start start 2024-09-10T23:36:42.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ee00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f613c003fe0 0x7f6158163f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:42.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ee00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61580c6ac0 0x7f61581f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:42.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ee00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61581fb110 0x7f6158070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:42.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ee00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f615806da30 con 0x7f61581fb110 2024-09-10T23:36:42.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ee00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f615806d8b0 con 0x7f61580c6ac0 2024-09-10T23:36:42.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ee00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6158165bb0 con 0x7f613c003fe0 2024-09-10T23:36:42.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61581fb110 0x7f6158070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:42.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61581fb110 0x7f6158070270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48968/0 (socket says 172.21.15.94:48968) 2024-09-10T23:36:42.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ce00640 1 -- 172.21.15.94:0/3245308111 learned_addr learned my addr 172.21.15.94:0/3245308111 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:42.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f6157e00640 1 --2- 172.21.15.94:0/3245308111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f613c003fe0 0x7f6158163f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:42.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f614f400640 1 --2- 172.21.15.94:0/3245308111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61580c6ac0 0x7f61581f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:42.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ce00640 1 -- 172.21.15.94:0/3245308111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f613c003fe0 msgr2=0x7f6158163f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:42.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ce00640 1 --2- 172.21.15.94:0/3245308111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f613c003fe0 0x7f6158163f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:42.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ce00640 1 -- 172.21.15.94:0/3245308111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61580c6ac0 msgr2=0x7f61581f8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:42.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ce00640 1 --2- 172.21.15.94:0/3245308111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61580c6ac0 0x7f61581f8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:42.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f615ce00640 1 -- 172.21.15.94:0/3245308111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61581f8a70 con 0x7f61581fb110 2024-09-10T23:36:42.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.350+0000 7f614f400640 1 --2- 172.21.15.94:0/3245308111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61580c6ac0 0x7f61581f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:42.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.354+0000 7f615ce00640 1 --2- 172.21.15.94:0/3245308111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61581fb110 0x7f6158070270 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6158072de0 tx=0x7f6150010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:42.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.354+0000 7f6154c00640 1 -- 172.21.15.94:0/3245308111 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615000edb0 con 0x7f61581fb110 2024-09-10T23:36:42.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.354+0000 7f6154c00640 1 -- 172.21.15.94:0/3245308111 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6150015070 con 0x7f61581fb110 2024-09-10T23:36:42.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.354+0000 7f6154c00640 1 -- 172.21.15.94:0/3245308111 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150003be0 con 0x7f61581fb110 2024-09-10T23:36:42.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.354+0000 7f615ee00640 1 -- 172.21.15.94:0/3245308111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61581fba80 con 0x7f61581fb110 2024-09-10T23:36:42.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.354+0000 7f615ee00640 1 -- 172.21.15.94:0/3245308111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61581f7d30 con 0x7f61581fb110 2024-09-10T23:36:42.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.354+0000 7f615ee00640 1 -- 172.21.15.94:0/3245308111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61581fbfa0 con 0x7f61581fb110 2024-09-10T23:36:42.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.354+0000 7f6154c00640 1 -- 172.21.15.94:0/3245308111 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f6150004030 con 0x7f61581fb110 2024-09-10T23:36:42.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.354+0000 7f6154c00640 1 --2- 172.21.15.94:0/3245308111 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6134072730 0x7f6134074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:42.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.354+0000 7f6154c00640 1 -- 172.21.15.94:0/3245308111 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5916+0+0 (secure 0 0 0) 0x7f6150093e90 con 0x7f61581fb110 2024-09-10T23:36:42.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.354+0000 7f6157e00640 1 --2- 172.21.15.94:0/3245308111 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6134072730 0x7f6134074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:42.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.358+0000 7f6157e00640 1 --2- 172.21.15.94:0/3245308111 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6134072730 0x7f6134074bf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6148009b80 tx=0x7f61480091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:42.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.358+0000 7f6154c00640 1 -- 172.21.15.94:0/3245308111 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f61581fbfa0 con 0x7f61581fb110 2024-09-10T23:36:42.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:42.522+0000 7f615ee00640 1 -- 172.21.15.94:0/3245308111 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f61580b6250 con 0x7f61581fb110 2024-09-10T23:36:43.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.174+0000 7f6154c00640 1 -- 172.21.15.94:0/3245308111 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) ==== 169+0+0 (secure 0 0 0) 0x7f61581f8a70 con 0x7f61581fb110 2024-09-10T23:36:43.176 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-09-10T23:36:43.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.178+0000 7f615ee00640 1 -- 172.21.15.94:0/3245308111 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6134072730 msgr2=0x7f6134074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:43.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.178+0000 7f615ee00640 1 --2- 172.21.15.94:0/3245308111 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6134072730 0x7f6134074bf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6148009b80 tx=0x7f61480091e0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.178+0000 7f615ee00640 1 -- 172.21.15.94:0/3245308111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61581fb110 msgr2=0x7f6158070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:43.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.178+0000 7f615ee00640 1 --2- 172.21.15.94:0/3245308111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61581fb110 0x7f6158070270 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6158072de0 tx=0x7f6150010e80 comp rx=0 tx=0).stop 2024-09-10T23:36:43.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.178+0000 7f615ee00640 1 -- 172.21.15.94:0/3245308111 shutdown_connections 2024-09-10T23:36:43.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.178+0000 7f615ee00640 1 --2- 172.21.15.94:0/3245308111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61580c6ac0 0x7f61581f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.178+0000 7f615ee00640 1 --2- 172.21.15.94:0/3245308111 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6134072730 0x7f6134074bf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.178+0000 7f615ee00640 1 --2- 172.21.15.94:0/3245308111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61581fb110 0x7f6158070270 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.178+0000 7f615ee00640 1 --2- 172.21.15.94:0/3245308111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f613c003fe0 0x7f6158163f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.178+0000 7f615ee00640 1 -- 172.21.15.94:0/3245308111 >> 172.21.15.94:0/3245308111 conn(0x7f61580d02d0 msgr2=0x7f61580b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:43.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.178+0000 7f615ee00640 1 -- 172.21.15.94:0/3245308111 shutdown_connections 2024-09-10T23:36:43.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.178+0000 7f615ee00640 1 -- 172.21.15.94:0/3245308111 wait complete. 2024-09-10T23:36:43.206 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T23:36:43.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ef600640 1 -- 172.21.15.94:0/4164689278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60e815f2d0 msgr2=0x7f60e80b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:43.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ef600640 1 --2- 172.21.15.94:0/4164689278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60e815f2d0 0x7f60e80b0300 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009900 tx=0x7f60dc01c850 comp rx=0 tx=0).stop 2024-09-10T23:36:43.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ef600640 1 -- 172.21.15.94:0/4164689278 shutdown_connections 2024-09-10T23:36:43.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ef600640 1 --2- 172.21.15.94:0/4164689278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60e80cbb30 0x7f60e80ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ef600640 1 --2- 172.21.15.94:0/4164689278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60e815f2d0 0x7f60e80b0300 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ef600640 1 -- 172.21.15.94:0/4164689278 >> 172.21.15.94:0/4164689278 conn(0x7f60e80b1de0 msgr2=0x7f60e80b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:43.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ef600640 1 -- 172.21.15.94:0/4164689278 shutdown_connections 2024-09-10T23:36:43.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ef600640 1 -- 172.21.15.94:0/4164689278 wait complete. 2024-09-10T23:36:43.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ef600640 1 Processor -- start 2024-09-10T23:36:43.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ef600640 1 -- start start 2024-09-10T23:36:43.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ef600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60e80cbb30 0x7f60e80c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:43.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ef600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60e80cc310 0x7f60e80ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:43.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ef600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60e815f2d0 0x7f60e80cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:43.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ef600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60e80cc0e0 con 0x7f60e815f2d0 2024-09-10T23:36:43.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ef600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f60e80cbf60 con 0x7f60e80cbb30 2024-09-10T23:36:43.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ef600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f60e81683f0 con 0x7f60e80cc310 2024-09-10T23:36:43.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ed600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60e815f2d0 0x7f60e80cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:43.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ed600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60e815f2d0 0x7f60e80cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48980/0 (socket says 172.21.15.94:48980) 2024-09-10T23:36:43.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ed600640 1 -- 172.21.15.94:0/1027730573 learned_addr learned my addr 172.21.15.94:0/1027730573 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:43.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ecc00640 1 --2- 172.21.15.94:0/1027730573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60e80cbb30 0x7f60e80c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:43.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60e7e00640 1 --2- 172.21.15.94:0/1027730573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60e80cc310 0x7f60e80ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:43.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ed600640 1 -- 172.21.15.94:0/1027730573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60e80cc310 msgr2=0x7f60e80ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:43.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ed600640 1 --2- 172.21.15.94:0/1027730573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60e80cc310 0x7f60e80ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ed600640 1 -- 172.21.15.94:0/1027730573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60e80cbb30 msgr2=0x7f60e80c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:43.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ed600640 1 --2- 172.21.15.94:0/1027730573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60e80cbb30 0x7f60e80c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.354+0000 7f60ed600640 1 -- 172.21.15.94:0/1027730573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e80c6b50 con 0x7f60e815f2d0 2024-09-10T23:36:43.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.358+0000 7f60ed600640 1 --2- 172.21.15.94:0/1027730573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60e815f2d0 0x7f60e80cd420 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f60e000b9a0 tx=0x7f60e000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:43.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.358+0000 7f60e4c00640 1 -- 172.21.15.94:0/1027730573 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e0019070 con 0x7f60e815f2d0 2024-09-10T23:36:43.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.358+0000 7f60e4c00640 1 -- 172.21.15.94:0/1027730573 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60e000ccb0 con 0x7f60e815f2d0 2024-09-10T23:36:43.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.358+0000 7f60ef600640 1 -- 172.21.15.94:0/1027730573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e815e550 con 0x7f60e815f2d0 2024-09-10T23:36:43.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.358+0000 7f60e4c00640 1 -- 172.21.15.94:0/1027730573 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e00146a0 con 0x7f60e815f2d0 2024-09-10T23:36:43.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.358+0000 7f60ef600640 1 -- 172.21.15.94:0/1027730573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e81fb5f0 con 0x7f60e815f2d0 2024-09-10T23:36:43.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.358+0000 7f60ef600640 1 -- 172.21.15.94:0/1027730573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e80b6270 con 0x7f60e815f2d0 2024-09-10T23:36:43.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.358+0000 7f60e4c00640 1 -- 172.21.15.94:0/1027730573 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f60e001d3b0 con 0x7f60e815f2d0 2024-09-10T23:36:43.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.358+0000 7f60e4c00640 1 --2- 172.21.15.94:0/1027730573 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f60c8072700 0x7f60c8074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:43.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.358+0000 7f60e4c00640 1 -- 172.21.15.94:0/1027730573 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7f60e0093380 con 0x7f60e815f2d0 2024-09-10T23:36:43.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.358+0000 7f60ecc00640 1 --2- 172.21.15.94:0/1027730573 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f60c8072700 0x7f60c8074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:43.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.362+0000 7f60ecc00640 1 --2- 172.21.15.94:0/1027730573 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f60c8072700 0x7f60c8074bc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0af000 tx=0x7f60dc01b090 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:43.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.362+0000 7f60e4c00640 1 -- 172.21.15.94:0/1027730573 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f60e005e2b0 con 0x7f60e815f2d0 2024-09-10T23:36:43.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.530+0000 7f60ef600640 1 -- 172.21.15.94:0/1027730573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f60e815b1b0 con 0x7f60e815f2d0 2024-09-10T23:36:43.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.534+0000 7f60e4c00640 1 -- 172.21.15.94:0/1027730573 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v35) ==== 71+0+0 (secure 0 0 0) 0x7f60e0062ce0 con 0x7f60e815f2d0 2024-09-10T23:36:43.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.538+0000 7f60ef600640 1 -- 172.21.15.94:0/1027730573 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f60c8072700 msgr2=0x7f60c8074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:43.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.538+0000 7f60ef600640 1 --2- 172.21.15.94:0/1027730573 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f60c8072700 0x7f60c8074bc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0af000 tx=0x7f60dc01b090 comp rx=0 tx=0).stop 2024-09-10T23:36:43.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.538+0000 7f60ef600640 1 -- 172.21.15.94:0/1027730573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60e815f2d0 msgr2=0x7f60e80cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:43.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.538+0000 7f60ef600640 1 --2- 172.21.15.94:0/1027730573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60e815f2d0 0x7f60e80cd420 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f60e000b9a0 tx=0x7f60e000be60 comp rx=0 tx=0).stop 2024-09-10T23:36:43.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.538+0000 7f60ef600640 1 -- 172.21.15.94:0/1027730573 shutdown_connections 2024-09-10T23:36:43.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.538+0000 7f60ef600640 1 --2- 172.21.15.94:0/1027730573 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f60c8072700 0x7f60c8074bc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.538+0000 7f60ef600640 1 --2- 172.21.15.94:0/1027730573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60e815f2d0 0x7f60e80cd420 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.538+0000 7f60ef600640 1 --2- 172.21.15.94:0/1027730573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60e80cc310 0x7f60e80ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.538+0000 7f60ef600640 1 --2- 172.21.15.94:0/1027730573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60e80cbb30 0x7f60e80c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.538+0000 7f60ef600640 1 -- 172.21.15.94:0/1027730573 >> 172.21.15.94:0/1027730573 conn(0x7f60e80b1de0 msgr2=0x7f60e80b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:43.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.538+0000 7f60ef600640 1 -- 172.21.15.94:0/1027730573 shutdown_connections 2024-09-10T23:36:43.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.538+0000 7f60ef600640 1 -- 172.21.15.94:0/1027730573 wait complete. 2024-09-10T23:36:43.556 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-10T23:36:43.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.694+0000 7f58b5e00640 1 -- 172.21.15.94:0/2100848290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58b0070150 msgr2=0x7f58b0072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:43.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.694+0000 7f58b5e00640 1 --2- 172.21.15.94:0/2100848290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58b0070150 0x7f58b0072610 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f58a0009bc0 tx=0x7f58a001cb10 comp rx=0 tx=0).stop 2024-09-10T23:36:43.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.694+0000 7f58b5e00640 1 -- 172.21.15.94:0/2100848290 shutdown_connections 2024-09-10T23:36:43.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.694+0000 7f58b5e00640 1 --2- 172.21.15.94:0/2100848290 >> v2:172.21.15.181:3300/0 conn(0x7f58b00b35c0 0x7f58b006eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.694+0000 7f58b5e00640 1 --2- 172.21.15.94:0/2100848290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58b0070150 0x7f58b0072610 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.694+0000 7f58b5e00640 1 -- 172.21.15.94:0/2100848290 >> 172.21.15.94:0/2100848290 conn(0x7f58b006cdf0 msgr2=0x7f58b006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:43.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.694+0000 7f58b5e00640 1 -- 172.21.15.94:0/2100848290 shutdown_connections 2024-09-10T23:36:43.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.694+0000 7f58b5e00640 1 -- 172.21.15.94:0/2100848290 wait complete. 2024-09-10T23:36:43.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58b5e00640 1 Processor -- start 2024-09-10T23:36:43.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58b5e00640 1 -- start start 2024-09-10T23:36:43.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58b5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58b0070150 0x7f58b01fba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:43.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58b5e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58b00b35c0 0x7f58b013c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:43.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58b5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0157c60 0x7f58b013ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:43.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58b5e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58b0167f90 con 0x7f58b0157c60 2024-09-10T23:36:43.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58b5e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f58b015b5b0 con 0x7f58b0070150 2024-09-10T23:36:43.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58b5e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f58b0168110 con 0x7f58b00b35c0 2024-09-10T23:36:43.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58aea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58b0070150 0x7f58b01fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:43.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58af400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0157c60 0x7f58b013ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:43.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58aea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58b0070150 0x7f58b01fba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:36272/0 (socket says 172.21.15.94:36272) 2024-09-10T23:36:43.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58aea00640 1 -- 172.21.15.94:0/3204649862 learned_addr learned my addr 172.21.15.94:0/3204649862 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:43.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58af400640 1 --2- 172.21.15.94:0/3204649862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0157c60 0x7f58b013ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48990/0 (socket says 172.21.15.94:48990) 2024-09-10T23:36:43.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58a6000640 1 --2- 172.21.15.94:0/3204649862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58b00b35c0 0x7f58b013c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:43.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58af400640 1 -- 172.21.15.94:0/3204649862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58b00b35c0 msgr2=0x7f58b013c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:43.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58af400640 1 --2- 172.21.15.94:0/3204649862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58b00b35c0 0x7f58b013c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58af400640 1 -- 172.21.15.94:0/3204649862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58b0070150 msgr2=0x7f58b01fba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:43.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58af400640 1 --2- 172.21.15.94:0/3204649862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58b0070150 0x7f58b01fba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:43.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58af400640 1 -- 172.21.15.94:0/3204649862 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b013f530 con 0x7f58b0157c60 2024-09-10T23:36:43.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58af400640 1 --2- 172.21.15.94:0/3204649862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0157c60 0x7f58b013ea90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f58a800e950 tx=0x7f58a800ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:43.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58a7e00640 1 -- 172.21.15.94:0/3204649862 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a800cd80 con 0x7f58b0157c60 2024-09-10T23:36:43.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58a7e00640 1 -- 172.21.15.94:0/3204649862 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58a8004030 con 0x7f58b0157c60 2024-09-10T23:36:43.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58a7e00640 1 -- 172.21.15.94:0/3204649862 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8004480 con 0x7f58b0157c60 2024-09-10T23:36:43.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.698+0000 7f58b5e00640 1 -- 172.21.15.94:0/3204649862 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b013f810 con 0x7f58b0157c60 2024-09-10T23:36:43.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.702+0000 7f58b5e00640 1 -- 172.21.15.94:0/3204649862 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b013c670 con 0x7f58b0157c60 2024-09-10T23:36:43.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.702+0000 7f58b5e00640 1 -- 172.21.15.94:0/3204649862 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b01fb050 con 0x7f58b0157c60 2024-09-10T23:36:43.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.702+0000 7f58a7e00640 1 -- 172.21.15.94:0/3204649862 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f58a8003600 con 0x7f58b0157c60 2024-09-10T23:36:43.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.702+0000 7f58a7e00640 1 --2- 172.21.15.94:0/3204649862 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f588c072730 0x7f588c074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:43.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.702+0000 7f58a7e00640 1 -- 172.21.15.94:0/3204649862 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7f58a8093d60 con 0x7f58b0157c60 2024-09-10T23:36:43.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.702+0000 7f58aea00640 1 --2- 172.21.15.94:0/3204649862 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f588c072730 0x7f588c074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:43.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.702+0000 7f58aea00640 1 --2- 172.21.15.94:0/3204649862 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f588c072730 0x7f588c074bf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f58a001a740 tx=0x7f58a001b350 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:43.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.706+0000 7f58a7e00640 1 -- 172.21.15.94:0/3204649862 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f58a805ec00 con 0x7f58b0157c60 2024-09-10T23:36:43.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:43.942+0000 7f58b5e00640 1 -- 172.21.15.94:0/3204649862 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7f58b013f9e0 con 0x7f58b0157c60 2024-09-10T23:36:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.174+0000 7f58a7e00640 1 -- 172.21.15.94:0/3204649862 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v227) ==== 131+0+0 (secure 0 0 0) 0x7f58b013f9e0 con 0x7f58b0157c60 2024-09-10T23:36:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.178+0000 7f58b5e00640 1 -- 172.21.15.94:0/3204649862 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f588c072730 msgr2=0x7f588c074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.178+0000 7f58b5e00640 1 --2- 172.21.15.94:0/3204649862 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f588c072730 0x7f588c074bf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f58a001a740 tx=0x7f58a001b350 comp rx=0 tx=0).stop 2024-09-10T23:36:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.178+0000 7f58b5e00640 1 -- 172.21.15.94:0/3204649862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0157c60 msgr2=0x7f58b013ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.178+0000 7f58b5e00640 1 --2- 172.21.15.94:0/3204649862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0157c60 0x7f58b013ea90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f58a800e950 tx=0x7f58a800ee10 comp rx=0 tx=0).stop 2024-09-10T23:36:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.178+0000 7f58b5e00640 1 -- 172.21.15.94:0/3204649862 shutdown_connections 2024-09-10T23:36:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.178+0000 7f58b5e00640 1 --2- 172.21.15.94:0/3204649862 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f588c072730 0x7f588c074bf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.178+0000 7f58b5e00640 1 --2- 172.21.15.94:0/3204649862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0157c60 0x7f58b013ea90 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.178+0000 7f58b5e00640 1 --2- 172.21.15.94:0/3204649862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58b00b35c0 0x7f58b013c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.178+0000 7f58b5e00640 1 --2- 172.21.15.94:0/3204649862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58b0070150 0x7f58b01fba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.178+0000 7f58b5e00640 1 -- 172.21.15.94:0/3204649862 >> 172.21.15.94:0/3204649862 conn(0x7f58b006cdf0 msgr2=0x7f58b0153720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:44.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.178+0000 7f58b5e00640 1 -- 172.21.15.94:0/3204649862 shutdown_connections 2024-09-10T23:36:44.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.178+0000 7f58b5e00640 1 -- 172.21.15.94:0/3204649862 wait complete. 2024-09-10T23:36:44.200 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-10T23:36:44.200 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2024-09-10T23:36:44.200 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T23:36:44.201 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:36:44.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.342+0000 7f68a2c00640 1 -- 172.21.15.94:0/2341949744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f689c0d04b0 msgr2=0x7f689c15e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:44.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.342+0000 7f68a2c00640 1 --2- 172.21.15.94:0/2341949744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f689c0d04b0 0x7f689c15e770 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6888009840 tx=0x7f688801c7f0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.342+0000 7f68a2c00640 1 -- 172.21.15.94:0/2341949744 shutdown_connections 2024-09-10T23:36:44.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.342+0000 7f68a2c00640 1 --2- 172.21.15.94:0/2341949744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f689c15ecb0 0x7f689c06f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.342+0000 7f68a2c00640 1 --2- 172.21.15.94:0/2341949744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f689c0d04b0 0x7f689c15e770 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.342+0000 7f68a2c00640 1 -- 172.21.15.94:0/2341949744 >> 172.21.15.94:0/2341949744 conn(0x7f689c06cdf0 msgr2=0x7f689c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:44.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.342+0000 7f68a2c00640 1 -- 172.21.15.94:0/2341949744 shutdown_connections 2024-09-10T23:36:44.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.342+0000 7f68a2c00640 1 -- 172.21.15.94:0/2341949744 wait complete. 2024-09-10T23:36:44.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f68a2c00640 1 Processor -- start 2024-09-10T23:36:44.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f68a2c00640 1 -- start start 2024-09-10T23:36:44.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f68a2c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f689c073ff0 0x7f689c149010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:44.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f68a2c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f689c0d04b0 0x7f689c1facc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:44.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f68a2c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f689c15ecb0 0x7f689c1fb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:44.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f68a2c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f689c167930 con 0x7f689c073ff0 2024-09-10T23:36:44.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f68a2c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f689c1677b0 con 0x7f689c15ecb0 2024-09-10T23:36:44.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f68a2c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f689c167ab0 con 0x7f689c0d04b0 2024-09-10T23:36:44.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f689be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f689c073ff0 0x7f689c149010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:44.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f689be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f689c073ff0 0x7f689c149010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60346/0 (socket says 172.21.15.94:60346) 2024-09-10T23:36:44.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f689be00640 1 -- 172.21.15.94:0/1316148960 learned_addr learned my addr 172.21.15.94:0/1316148960 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:44.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f689b400640 1 --2- 172.21.15.94:0/1316148960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f689c0d04b0 0x7f689c1facc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:44.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f68a0c00640 1 --2- 172.21.15.94:0/1316148960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f689c15ecb0 0x7f689c1fb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:44.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f689be00640 1 -- 172.21.15.94:0/1316148960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f689c0d04b0 msgr2=0x7f689c1facc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:44.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f689be00640 1 --2- 172.21.15.94:0/1316148960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f689c0d04b0 0x7f689c1facc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f689be00640 1 -- 172.21.15.94:0/1316148960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f689c15ecb0 msgr2=0x7f689c1fb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:44.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f689be00640 1 --2- 172.21.15.94:0/1316148960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f689c15ecb0 0x7f689c1fb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f689be00640 1 -- 172.21.15.94:0/1316148960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c14b9a0 con 0x7f689c073ff0 2024-09-10T23:36:44.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f68a0c00640 1 --2- 172.21.15.94:0/1316148960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f689c15ecb0 0x7f689c1fb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:44.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f689b400640 1 --2- 172.21.15.94:0/1316148960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f689c0d04b0 0x7f689c1facc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:44.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f689be00640 1 --2- 172.21.15.94:0/1316148960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f689c073ff0 0x7f689c149010 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f688c00c450 tx=0x7f688c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:44.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f6893e00640 1 -- 172.21.15.94:0/1316148960 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c016020 con 0x7f689c073ff0 2024-09-10T23:36:44.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f6893e00640 1 -- 172.21.15.94:0/1316148960 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f688c012920 con 0x7f689c073ff0 2024-09-10T23:36:44.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f68a2c00640 1 -- 172.21.15.94:0/1316148960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689c14bc80 con 0x7f689c073ff0 2024-09-10T23:36:44.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f6893e00640 1 -- 172.21.15.94:0/1316148960 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c012d70 con 0x7f689c073ff0 2024-09-10T23:36:44.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f68a2c00640 1 -- 172.21.15.94:0/1316148960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f689c14c240 con 0x7f689c073ff0 2024-09-10T23:36:44.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.346+0000 7f68a2c00640 1 -- 172.21.15.94:0/1316148960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f689c15e770 con 0x7f689c073ff0 2024-09-10T23:36:44.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.350+0000 7f6893e00640 1 -- 172.21.15.94:0/1316148960 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f688c01b400 con 0x7f689c073ff0 2024-09-10T23:36:44.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.350+0000 7f6893e00640 1 --2- 172.21.15.94:0/1316148960 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f687c072700 0x7f687c074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:44.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.350+0000 7f6893e00640 1 -- 172.21.15.94:0/1316148960 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7f688c093720 con 0x7f689c073ff0 2024-09-10T23:36:44.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.350+0000 7f689b400640 1 --2- 172.21.15.94:0/1316148960 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f687c072700 0x7f687c074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:44.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.350+0000 7f689b400640 1 --2- 172.21.15.94:0/1316148960 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f687c072700 0x7f687c074bc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f68880af000 tx=0x7f688801b030 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:44.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.354+0000 7f6893e00640 1 -- 172.21.15.94:0/1316148960 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f688c020020 con 0x7f689c073ff0 2024-09-10T23:36:44.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.522+0000 7f68a2c00640 1 -- 172.21.15.94:0/1316148960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f689c165580 con 0x7f689c073ff0 2024-09-10T23:36:44.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.522+0000 7f6893e00640 1 -- 172.21.15.94:0/1316148960 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+13847 (secure 0 0 0) 0x7f688c05ec30 con 0x7f689c073ff0 2024-09-10T23:36:44.525 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:44.525 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":73,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:36:43.171308+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6836/2999113739":"2024-09-11T23:36:40.159064+0000","172.21.15.181:6834/2999113739":"2024-09-11T23:36:40.159064+0000","172.21.15.181:0/2231303511":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/1584210907":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/2614085240":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/1346366560":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/2211863778":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/1339348859":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/3942792329":"2024-09-11T23:36:36.718561+0000","172.21.15.181:6800/3310396812":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/2484932838":"2024-09-11T23:36:36.718561+0000","172.21.15.181:6801/3310396812":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/749967229":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/2150218902":"2024-09-11T23:36:36.718561+0000","172.21.15.181:0/1922952906":"2024-09-11T23:36:36.718561+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:36:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.526+0000 7f68a2c00640 1 -- 172.21.15.94:0/1316148960 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f687c072700 msgr2=0x7f687c074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.526+0000 7f68a2c00640 1 --2- 172.21.15.94:0/1316148960 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f687c072700 0x7f687c074bc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f68880af000 tx=0x7f688801b030 comp rx=0 tx=0).stop 2024-09-10T23:36:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.526+0000 7f68a2c00640 1 -- 172.21.15.94:0/1316148960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f689c073ff0 msgr2=0x7f689c149010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.526+0000 7f68a2c00640 1 --2- 172.21.15.94:0/1316148960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f689c073ff0 0x7f689c149010 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f688c00c450 tx=0x7f688c00c910 comp rx=0 tx=0).stop 2024-09-10T23:36:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.526+0000 7f68a2c00640 1 -- 172.21.15.94:0/1316148960 shutdown_connections 2024-09-10T23:36:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.526+0000 7f68a2c00640 1 --2- 172.21.15.94:0/1316148960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f689c15ecb0 0x7f689c1fb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.526+0000 7f68a2c00640 1 --2- 172.21.15.94:0/1316148960 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f687c072700 0x7f687c074bc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.526+0000 7f68a2c00640 1 --2- 172.21.15.94:0/1316148960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f689c0d04b0 0x7f689c1facc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.526+0000 7f68a2c00640 1 --2- 172.21.15.94:0/1316148960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f689c073ff0 0x7f689c149010 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.526+0000 7f68a2c00640 1 -- 172.21.15.94:0/1316148960 >> 172.21.15.94:0/1316148960 conn(0x7f689c06cdf0 msgr2=0x7f689c0cd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.526+0000 7f68a2c00640 1 -- 172.21.15.94:0/1316148960 shutdown_connections 2024-09-10T23:36:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.526+0000 7f68a2c00640 1 -- 172.21.15.94:0/1316148960 wait complete. 2024-09-10T23:36:44.546 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T23:32:31.372521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T23:36:44.547 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T23:36:44.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 -- 172.21.15.94:0/3933545273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60d015bf60 msgr2=0x7f60d00d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:44.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 --2- 172.21.15.94:0/3933545273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60d015bf60 0x7f60d00d2780 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f60d00731c0 tx=0x7f60c001c570 comp rx=0 tx=0).stop 2024-09-10T23:36:44.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 -- 172.21.15.94:0/3933545273 shutdown_connections 2024-09-10T23:36:44.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 --2- 172.21.15.94:0/3933545273 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60d00d0660 0x7f60d00ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 --2- 172.21.15.94:0/3933545273 >> v2:172.21.15.181:3300/0 conn(0x7f60d00cfce0 0x7f60d00d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 --2- 172.21.15.94:0/3933545273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60d015bf60 0x7f60d00d2780 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 -- 172.21.15.94:0/3933545273 >> 172.21.15.94:0/3933545273 conn(0x7f60d00d1c60 msgr2=0x7f60d00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:44.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 -- 172.21.15.94:0/3933545273 shutdown_connections 2024-09-10T23:36:44.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 -- 172.21.15.94:0/3933545273 wait complete. 2024-09-10T23:36:44.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 Processor -- start 2024-09-10T23:36:44.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 -- start start 2024-09-10T23:36:44.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60d00cfce0 0x7f60d0152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:44.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d00d0660 0x7f60d00c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:44.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60d015bf60 0x7f60d01f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:44.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60d00c6a40 con 0x7f60d00cfce0 2024-09-10T23:36:44.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f60d00c68c0 con 0x7f60d00d0660 2024-09-10T23:36:44.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60d4e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f60d00c6bc0 con 0x7f60d015bf60 2024-09-10T23:36:44.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60cde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60d00cfce0 0x7f60d0152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:44.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60cde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60d00cfce0 0x7f60d0152cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60350/0 (socket says 172.21.15.94:60350) 2024-09-10T23:36:44.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60cde00640 1 -- 172.21.15.94:0/2233028891 learned_addr learned my addr 172.21.15.94:0/2233028891 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:44.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60cd400640 1 --2- 172.21.15.94:0/2233028891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d00d0660 0x7f60d00c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:44.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60ce800640 1 --2- 172.21.15.94:0/2233028891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60d015bf60 0x7f60d01f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:44.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60cde00640 1 -- 172.21.15.94:0/2233028891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60d015bf60 msgr2=0x7f60d01f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:44.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60cde00640 1 --2- 172.21.15.94:0/2233028891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60d015bf60 0x7f60d01f5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60cde00640 1 -- 172.21.15.94:0/2233028891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d00d0660 msgr2=0x7f60d00c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:44.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60cde00640 1 --2- 172.21.15.94:0/2233028891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d00d0660 0x7f60d00c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.690+0000 7f60cde00640 1 -- 172.21.15.94:0/2233028891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d00c9270 con 0x7f60d00cfce0 2024-09-10T23:36:44.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.694+0000 7f60cde00640 1 --2- 172.21.15.94:0/2233028891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60d00cfce0 0x7f60d0152cb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009850 tx=0x7f60c00a6420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:44.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.694+0000 7f60c6000640 1 -- 172.21.15.94:0/2233028891 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c00ac070 con 0x7f60d00cfce0 2024-09-10T23:36:44.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.694+0000 7f60c6000640 1 -- 172.21.15.94:0/2233028891 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60c00050c0 con 0x7f60d00cfce0 2024-09-10T23:36:44.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.694+0000 7f60c6000640 1 -- 172.21.15.94:0/2233028891 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c00054b0 con 0x7f60d00cfce0 2024-09-10T23:36:44.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.694+0000 7f60d4e00640 1 -- 172.21.15.94:0/2233028891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d01f6820 con 0x7f60d00cfce0 2024-09-10T23:36:44.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.694+0000 7f60d4e00640 1 -- 172.21.15.94:0/2233028891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d01f6de0 con 0x7f60d00cfce0 2024-09-10T23:36:44.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.694+0000 7f60d4e00640 1 -- 172.21.15.94:0/2233028891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d00ce3f0 con 0x7f60d00cfce0 2024-09-10T23:36:44.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.694+0000 7f60c6000640 1 -- 172.21.15.94:0/2233028891 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f60c00b9070 con 0x7f60d00cfce0 2024-09-10T23:36:44.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.694+0000 7f60c6000640 1 --2- 172.21.15.94:0/2233028891 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f60b0072700 0x7f60b0074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:44.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.694+0000 7f60c6000640 1 -- 172.21.15.94:0/2233028891 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7f60c012d360 con 0x7f60d00cfce0 2024-09-10T23:36:44.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.694+0000 7f60cd400640 1 --2- 172.21.15.94:0/2233028891 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f60b0072700 0x7f60b0074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:44.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.698+0000 7f60cd400640 1 --2- 172.21.15.94:0/2233028891 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f60b0072700 0x7f60b0074bc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f60d009f830 tx=0x7f60bc00b2e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:44.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.698+0000 7f60c6000640 1 -- 172.21.15.94:0/2233028891 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f60c00164b0 con 0x7f60d00cfce0 2024-09-10T23:36:44.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.862+0000 7f60d4e00640 1 -- 172.21.15.94:0/2233028891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f60d00c76b0 con 0x7f60d00cfce0 2024-09-10T23:36:44.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.862+0000 7f60c6000640 1 -- 172.21.15.94:0/2233028891 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f60c00f88e0 con 0x7f60d00cfce0 2024-09-10T23:36:44.866 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-09-10T23:36:44.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.866+0000 7f60d4e00640 1 -- 172.21.15.94:0/2233028891 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f60b0072700 msgr2=0x7f60b0074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:44.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.866+0000 7f60d4e00640 1 --2- 172.21.15.94:0/2233028891 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f60b0072700 0x7f60b0074bc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f60d009f830 tx=0x7f60bc00b2e0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.866+0000 7f60d4e00640 1 -- 172.21.15.94:0/2233028891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60d00cfce0 msgr2=0x7f60d0152cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:44.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.866+0000 7f60d4e00640 1 --2- 172.21.15.94:0/2233028891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60d00cfce0 0x7f60d0152cb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009850 tx=0x7f60c00a6420 comp rx=0 tx=0).stop 2024-09-10T23:36:44.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.866+0000 7f60d4e00640 1 -- 172.21.15.94:0/2233028891 shutdown_connections 2024-09-10T23:36:44.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.866+0000 7f60d4e00640 1 --2- 172.21.15.94:0/2233028891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60d00d0660 0x7f60d00c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.866+0000 7f60d4e00640 1 --2- 172.21.15.94:0/2233028891 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f60b0072700 0x7f60b0074bc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.866+0000 7f60d4e00640 1 --2- 172.21.15.94:0/2233028891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60d015bf60 0x7f60d01f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.866+0000 7f60d4e00640 1 --2- 172.21.15.94:0/2233028891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60d00cfce0 0x7f60d0152cb0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:44.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.866+0000 7f60d4e00640 1 -- 172.21.15.94:0/2233028891 >> 172.21.15.94:0/2233028891 conn(0x7f60d00d1c60 msgr2=0x7f60d00b4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:44.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.866+0000 7f60d4e00640 1 -- 172.21.15.94:0/2233028891 shutdown_connections 2024-09-10T23:36:44.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:44.866+0000 7f60d4e00640 1 -- 172.21.15.94:0/2233028891 wait complete. 2024-09-10T23:36:44.887 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-10T23:36:45.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.026+0000 7fe609800640 1 -- 172.21.15.94:0/2439426354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e8004810 msgr2=0x7fe5e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:45.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.026+0000 7fe609800640 1 --2- 172.21.15.94:0/2439426354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e8004810 0x7fe5e8004c90 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec006f60 tx=0x7fe5ec01cee0 comp rx=0 tx=0).stop 2024-09-10T23:36:45.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe609800640 1 -- 172.21.15.94:0/2439426354 shutdown_connections 2024-09-10T23:36:45.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe609800640 1 --2- 172.21.15.94:0/2439426354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e8004810 0x7fe5e8004c90 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:45.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe609800640 1 -- 172.21.15.94:0/2439426354 >> 172.21.15.94:0/2439426354 conn(0x7fe6040d1af0 msgr2=0x7fe6040d1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:45.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe609800640 1 -- 172.21.15.94:0/2439426354 shutdown_connections 2024-09-10T23:36:45.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe609800640 1 -- 172.21.15.94:0/2439426354 wait complete. 2024-09-10T23:36:45.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe609800640 1 Processor -- start 2024-09-10T23:36:45.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe609800640 1 -- start start 2024-09-10T23:36:45.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe609800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e8004810 0x7fe60415b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:45.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe609800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6040c8bc0 0x7fe6040b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:45.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe609800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6040c9040 0x7fe6040b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:45.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe609800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe6040b25c0 con 0x7fe6040c8bc0 2024-09-10T23:36:45.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe609800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe6040b2440 con 0x7fe6040c9040 2024-09-10T23:36:45.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe609800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe6040ae820 con 0x7fe5e8004810 2024-09-10T23:36:45.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe5fb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6040c8bc0 0x7fe6040b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:45.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe5fb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6040c8bc0 0x7fe6040b5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60374/0 (socket says 172.21.15.94:60374) 2024-09-10T23:36:45.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe5fb400640 1 -- 172.21.15.94:0/1072693415 learned_addr learned my addr 172.21.15.94:0/1072693415 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:45.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe603e00640 1 --2- 172.21.15.94:0/1072693415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e8004810 0x7fe60415b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:45.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe608e00640 1 --2- 172.21.15.94:0/1072693415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6040c9040 0x7fe6040b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:45.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe5fb400640 1 -- 172.21.15.94:0/1072693415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e8004810 msgr2=0x7fe60415b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:45.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe5fb400640 1 --2- 172.21.15.94:0/1072693415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e8004810 0x7fe60415b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:45.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe5fb400640 1 -- 172.21.15.94:0/1072693415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6040c9040 msgr2=0x7fe6040b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:45.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe5fb400640 1 --2- 172.21.15.94:0/1072693415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6040c9040 0x7fe6040b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:45.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe5fb400640 1 -- 172.21.15.94:0/1072693415 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6040c9600 con 0x7fe6040c8bc0 2024-09-10T23:36:45.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe5fb400640 1 --2- 172.21.15.94:0/1072693415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6040c8bc0 0x7fe6040b5c40 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec098920 tx=0x7fe5ec0a5c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:45.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe600c00640 1 -- 172.21.15.94:0/1072693415 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec001680 con 0x7fe6040c8bc0 2024-09-10T23:36:45.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.030+0000 7fe600c00640 1 -- 172.21.15.94:0/1072693415 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5ec0060d0 con 0x7fe6040c8bc0 2024-09-10T23:36:45.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.034+0000 7fe609800640 1 -- 172.21.15.94:0/1072693415 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6040c98e0 con 0x7fe6040c8bc0 2024-09-10T23:36:45.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.034+0000 7fe600c00640 1 -- 172.21.15.94:0/1072693415 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec006520 con 0x7fe6040c8bc0 2024-09-10T23:36:45.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.034+0000 7fe609800640 1 -- 172.21.15.94:0/1072693415 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6040b4920 con 0x7fe6040c8bc0 2024-09-10T23:36:45.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.034+0000 7fe609800640 1 -- 172.21.15.94:0/1072693415 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6040b32e0 con 0x7fe6040c8bc0 2024-09-10T23:36:45.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.034+0000 7fe600c00640 1 -- 172.21.15.94:0/1072693415 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fe5ec003600 con 0x7fe6040c8bc0 2024-09-10T23:36:45.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.034+0000 7fe600c00640 1 --2- 172.21.15.94:0/1072693415 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5e0072730 0x7fe5e0074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:45.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.034+0000 7fe600c00640 1 -- 172.21.15.94:0/1072693415 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7fe5ec0b5020 con 0x7fe6040c8bc0 2024-09-10T23:36:45.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.034+0000 7fe603e00640 1 --2- 172.21.15.94:0/1072693415 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5e0072730 0x7fe5e0074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:45.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.034+0000 7fe603e00640 1 --2- 172.21.15.94:0/1072693415 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5e0072730 0x7fe5e0074bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4006fd0 tx=0x7fe5f4006cf0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:45.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.038+0000 7fe600c00640 1 -- 172.21.15.94:0/1072693415 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe5ec016c90 con 0x7fe6040c8bc0 2024-09-10T23:36:45.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:45.262+0000 7fe609800640 1 -- 172.21.15.94:0/1072693415 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7fe6040cfeb0 con 0x7fe6040c8bc0 2024-09-10T23:36:46.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.182+0000 7fe600c00640 1 -- 172.21.15.94:0/1072693415 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v229) ==== 149+0+0 (secure 0 0 0) 0x7fe5ec0f7b90 con 0x7fe6040c8bc0 2024-09-10T23:36:46.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.186+0000 7fe609800640 1 -- 172.21.15.94:0/1072693415 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5e0072730 msgr2=0x7fe5e0074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:46.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.186+0000 7fe609800640 1 --2- 172.21.15.94:0/1072693415 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5e0072730 0x7fe5e0074bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4006fd0 tx=0x7fe5f4006cf0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.186+0000 7fe609800640 1 -- 172.21.15.94:0/1072693415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6040c8bc0 msgr2=0x7fe6040b5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:46.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.186+0000 7fe609800640 1 --2- 172.21.15.94:0/1072693415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6040c8bc0 0x7fe6040b5c40 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec098920 tx=0x7fe5ec0a5c00 comp rx=0 tx=0).stop 2024-09-10T23:36:46.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.186+0000 7fe609800640 1 -- 172.21.15.94:0/1072693415 shutdown_connections 2024-09-10T23:36:46.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.186+0000 7fe609800640 1 --2- 172.21.15.94:0/1072693415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6040c9040 0x7fe6040b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.186+0000 7fe609800640 1 --2- 172.21.15.94:0/1072693415 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5e0072730 0x7fe5e0074bf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.186+0000 7fe609800640 1 --2- 172.21.15.94:0/1072693415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6040c8bc0 0x7fe6040b5c40 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.186+0000 7fe609800640 1 --2- 172.21.15.94:0/1072693415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e8004810 0x7fe60415b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.186+0000 7fe609800640 1 -- 172.21.15.94:0/1072693415 >> 172.21.15.94:0/1072693415 conn(0x7fe6040d1af0 msgr2=0x7fe60415bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:46.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.186+0000 7fe609800640 1 -- 172.21.15.94:0/1072693415 shutdown_connections 2024-09-10T23:36:46.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.186+0000 7fe609800640 1 -- 172.21.15.94:0/1072693415 wait complete. 2024-09-10T23:36:46.207 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T23:36:46.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9381600640 1 -- 172.21.15.94:0/1264405850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93600052f0 msgr2=0x7f936000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:46.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9381600640 1 --2- 172.21.15.94:0/1264405850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93600052f0 0x7f936000b9d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f93740016c0 tx=0x7f937401c790 comp rx=0 tx=0).stop 2024-09-10T23:36:46.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9381600640 1 -- 172.21.15.94:0/1264405850 shutdown_connections 2024-09-10T23:36:46.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9381600640 1 --2- 172.21.15.94:0/1264405850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93600052f0 0x7f936000b9d0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9381600640 1 -- 172.21.15.94:0/1264405850 >> 172.21.15.94:0/1264405850 conn(0x7f937c0d0b70 msgr2=0x7f937c0d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:46.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9381600640 1 -- 172.21.15.94:0/1264405850 shutdown_connections 2024-09-10T23:36:46.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9381600640 1 -- 172.21.15.94:0/1264405850 wait complete. 2024-09-10T23:36:46.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9381600640 1 Processor -- start 2024-09-10T23:36:46.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9381600640 1 -- start start 2024-09-10T23:36:46.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9381600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93600052f0 0x7f937c144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:46.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9381600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c15eb80 0x7f937c13d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:46.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9381600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f937c151650 0x7f937c151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:46.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9381600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f937c152e10 con 0x7f937c15eb80 2024-09-10T23:36:46.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9381600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f937c15caa0 con 0x7f93600052f0 2024-09-10T23:36:46.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9381600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f937c0d1690 con 0x7f937c151650 2024-09-10T23:36:46.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9372000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c15eb80 0x7f937c13d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:46.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9372000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c15eb80 0x7f937c13d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60392/0 (socket says 172.21.15.94:60392) 2024-09-10T23:36:46.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9372000640 1 -- 172.21.15.94:0/827420292 learned_addr learned my addr 172.21.15.94:0/827420292 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:46.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f937b400640 1 --2- 172.21.15.94:0/827420292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f937c151650 0x7f937c151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:46.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f937aa00640 1 --2- 172.21.15.94:0/827420292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93600052f0 0x7f937c144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:46.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9372000640 1 -- 172.21.15.94:0/827420292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f937c151650 msgr2=0x7f937c151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:46.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9372000640 1 --2- 172.21.15.94:0/827420292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f937c151650 0x7f937c151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9372000640 1 -- 172.21.15.94:0/827420292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93600052f0 msgr2=0x7f937c144bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:46.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9372000640 1 --2- 172.21.15.94:0/827420292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93600052f0 0x7f937c144bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.350+0000 7f9372000640 1 -- 172.21.15.94:0/827420292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c153d10 con 0x7f937c15eb80 2024-09-10T23:36:46.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.354+0000 7f9372000640 1 --2- 172.21.15.94:0/827420292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c15eb80 0x7f937c13d4e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9364001870 tx=0x7f9364012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:46.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.354+0000 7f9373400640 1 -- 172.21.15.94:0/827420292 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364010ae0 con 0x7f937c15eb80 2024-09-10T23:36:46.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.354+0000 7f9373400640 1 -- 172.21.15.94:0/827420292 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9364010c80 con 0x7f937c15eb80 2024-09-10T23:36:46.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.354+0000 7f9373400640 1 -- 172.21.15.94:0/827420292 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93640136b0 con 0x7f937c15eb80 2024-09-10T23:36:46.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.354+0000 7f9381600640 1 -- 172.21.15.94:0/827420292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937c15f120 con 0x7f937c15eb80 2024-09-10T23:36:46.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.354+0000 7f9381600640 1 -- 172.21.15.94:0/827420292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f937c15f6e0 con 0x7f937c15eb80 2024-09-10T23:36:46.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.354+0000 7f9381600640 1 -- 172.21.15.94:0/827420292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f937c06d000 con 0x7f937c15eb80 2024-09-10T23:36:46.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.354+0000 7f9373400640 1 -- 172.21.15.94:0/827420292 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f9364013850 con 0x7f937c15eb80 2024-09-10T23:36:46.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.354+0000 7f9373400640 1 --2- 172.21.15.94:0/827420292 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9358072730 0x7f9358074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:46.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.354+0000 7f9373400640 1 -- 172.21.15.94:0/827420292 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7f9364096e90 con 0x7f937c15eb80 2024-09-10T23:36:46.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.354+0000 7f937aa00640 1 --2- 172.21.15.94:0/827420292 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9358072730 0x7f9358074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:46.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.358+0000 7f937aa00640 1 --2- 172.21.15.94:0/827420292 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9358072730 0x7f9358074bf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f936c009f10 tx=0x7f936c009500 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.358+0000 7f9373400640 1 -- 172.21.15.94:0/827420292 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9364012e60 con 0x7f937c15eb80 2024-09-10T23:36:46.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.538+0000 7f9381600640 1 -- 172.21.15.94:0/827420292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f937c153ee0 con 0x7f937c15eb80 2024-09-10T23:36:46.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.542+0000 7f9373400640 1 -- 172.21.15.94:0/827420292 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v35) ==== 59+0+1998 (secure 0 0 0) 0x7f9364098020 con 0x7f937c15eb80 2024-09-10T23:36:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T23:36:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T23:36:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T23:36:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T23:36:46.542 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3944.074506 2024-09-10T23:36:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3865.464527 2024-09-10T23:36:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3898.861024 2024-09-10T23:36:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4010.048529 2024-09-10T23:36:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4650.562436 2024-09-10T23:36:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4724.018776 2024-09-10T23:36:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5255.753663 2024-09-10T23:36:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4718.009443 2024-09-10T23:36:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds 20/20 2024-09-10T23:36:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T23:36:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_ms 2/2 2024-09-10T23:36:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_frag true 2024-09-10T23:36:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_scatterstat true 2024-09-10T23:36:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T23:36:46.543 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_verify_scatter true 2024-09-10T23:36:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T23:36:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T23:36:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T23:36:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced client_mount_timeout 600 2024-09-10T23:36:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_client 20/20 2024-09-10T23:36:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_ms 1/1 2024-09-10T23:36:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T23:36:46.544 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T23:36:46.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.542+0000 7f9381600640 1 -- 172.21.15.94:0/827420292 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9358072730 msgr2=0x7f9358074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:46.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.542+0000 7f9381600640 1 --2- 172.21.15.94:0/827420292 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9358072730 0x7f9358074bf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f936c009f10 tx=0x7f936c009500 comp rx=0 tx=0).stop 2024-09-10T23:36:46.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.542+0000 7f9381600640 1 -- 172.21.15.94:0/827420292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c15eb80 msgr2=0x7f937c13d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:46.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.542+0000 7f9381600640 1 --2- 172.21.15.94:0/827420292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c15eb80 0x7f937c13d4e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9364001870 tx=0x7f9364012b20 comp rx=0 tx=0).stop 2024-09-10T23:36:46.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.542+0000 7f9381600640 1 -- 172.21.15.94:0/827420292 shutdown_connections 2024-09-10T23:36:46.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.542+0000 7f9381600640 1 --2- 172.21.15.94:0/827420292 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9358072730 0x7f9358074bf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.542+0000 7f9381600640 1 --2- 172.21.15.94:0/827420292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f937c151650 0x7f937c151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.542+0000 7f9381600640 1 --2- 172.21.15.94:0/827420292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c15eb80 0x7f937c13d4e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.542+0000 7f9381600640 1 --2- 172.21.15.94:0/827420292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93600052f0 0x7f937c144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.542+0000 7f9381600640 1 -- 172.21.15.94:0/827420292 >> 172.21.15.94:0/827420292 conn(0x7f937c0d0b70 msgr2=0x7f937c06f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:46.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.542+0000 7f9381600640 1 -- 172.21.15.94:0/827420292 shutdown_connections 2024-09-10T23:36:46.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.542+0000 7f9381600640 1 -- 172.21.15.94:0/827420292 wait complete. 2024-09-10T23:36:46.563 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:36:46.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.706+0000 7f6518000640 1 -- 172.21.15.94:0/617470345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f651015f2d0 msgr2=0x7f65100b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:46.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.706+0000 7f6518000640 1 --2- 172.21.15.94:0/617470345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f651015f2d0 0x7f65100b15f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6504009c80 tx=0x7f650401f5a0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.706+0000 7f6518000640 1 -- 172.21.15.94:0/617470345 shutdown_connections 2024-09-10T23:36:46.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.706+0000 7f6518000640 1 --2- 172.21.15.94:0/617470345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65100cfc00 0x7f65100b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.706+0000 7f6518000640 1 --2- 172.21.15.94:0/617470345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f651015f2d0 0x7f65100b15f0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.706+0000 7f6518000640 1 -- 172.21.15.94:0/617470345 >> 172.21.15.94:0/617470345 conn(0x7f65100d1c90 msgr2=0x7f65100d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:46.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.706+0000 7f6518000640 1 -- 172.21.15.94:0/617470345 shutdown_connections 2024-09-10T23:36:46.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.706+0000 7f6518000640 1 -- 172.21.15.94:0/617470345 wait complete. 2024-09-10T23:36:46.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.706+0000 7f6518000640 1 Processor -- start 2024-09-10T23:36:46.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.706+0000 7f6518000640 1 -- start start 2024-09-10T23:36:46.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6518000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65100b2790 0x7f65101396c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:46.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6518000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65100cfc00 0x7f651013d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:46.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6518000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f651015f2d0 0x7f65100ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:46.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6518000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f651015f830 con 0x7f65100cfc00 2024-09-10T23:36:46.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6518000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f651015f6b0 con 0x7f651015f2d0 2024-09-10T23:36:46.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6518000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f65101661e0 con 0x7f65100b2790 2024-09-10T23:36:46.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6514c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65100cfc00 0x7f651013d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:46.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6514c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65100cfc00 0x7f651013d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60416/0 (socket says 172.21.15.94:60416) 2024-09-10T23:36:46.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6514c00640 1 -- 172.21.15.94:0/2580366353 learned_addr learned my addr 172.21.15.94:0/2580366353 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:46.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6515600640 1 --2- 172.21.15.94:0/2580366353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65100b2790 0x7f65101396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:46.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6516000640 1 --2- 172.21.15.94:0/2580366353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f651015f2d0 0x7f65100ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:46.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6514c00640 1 -- 172.21.15.94:0/2580366353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65100b2790 msgr2=0x7f65101396c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:46.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6514c00640 1 --2- 172.21.15.94:0/2580366353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65100b2790 0x7f65101396c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6514c00640 1 -- 172.21.15.94:0/2580366353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f651015f2d0 msgr2=0x7f65100ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:46.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6514c00640 1 --2- 172.21.15.94:0/2580366353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f651015f2d0 0x7f65100ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6514c00640 1 -- 172.21.15.94:0/2580366353 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651013da40 con 0x7f65100cfc00 2024-09-10T23:36:46.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6516000640 1 --2- 172.21.15.94:0/2580366353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f651015f2d0 0x7f65100ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:36:46.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6515600640 1 --2- 172.21.15.94:0/2580366353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65100b2790 0x7f65101396c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:46.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6514c00640 1 --2- 172.21.15.94:0/2580366353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65100cfc00 0x7f651013d500 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00c970 tx=0x7f64fc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:46.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f650d600640 1 -- 172.21.15.94:0/2580366353 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc007c10 con 0x7f65100cfc00 2024-09-10T23:36:46.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f650d600640 1 -- 172.21.15.94:0/2580366353 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64fc00f450 con 0x7f65100cfc00 2024-09-10T23:36:46.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6518000640 1 -- 172.21.15.94:0/2580366353 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65101f7bf0 con 0x7f65100cfc00 2024-09-10T23:36:46.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f650d600640 1 -- 172.21.15.94:0/2580366353 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc00f8a0 con 0x7f65100cfc00 2024-09-10T23:36:46.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6518000640 1 -- 172.21.15.94:0/2580366353 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65101f81b0 con 0x7f65100cfc00 2024-09-10T23:36:46.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.710+0000 7f6518000640 1 -- 172.21.15.94:0/2580366353 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6510150c30 con 0x7f65100cfc00 2024-09-10T23:36:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.714+0000 7f650d600640 1 -- 172.21.15.94:0/2580366353 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f64fc00fa40 con 0x7f65100cfc00 2024-09-10T23:36:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.714+0000 7f650d600640 1 --2- 172.21.15.94:0/2580366353 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64f0072700 0x7f64f0074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.714+0000 7f650d600640 1 -- 172.21.15.94:0/2580366353 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7f64fc093ba0 con 0x7f65100cfc00 2024-09-10T23:36:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.714+0000 7f6515600640 1 --2- 172.21.15.94:0/2580366353 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64f0072700 0x7f64f0074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.714+0000 7f6515600640 1 --2- 172.21.15.94:0/2580366353 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64f0072700 0x7f64f0074bc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f65040b3000 tx=0x7f650401dde0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:46.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.718+0000 7f650d600640 1 -- 172.21.15.94:0/2580366353 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f64fc014030 con 0x7f65100cfc00 2024-09-10T23:36:46.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.882+0000 7f6518000640 1 -- 172.21.15.94:0/2580366353 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6510070b20 con 0x7f65100cfc00 2024-09-10T23:36:46.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.886+0000 7f650d600640 1 -- 172.21.15.94:0/2580366353 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f64fc05f130 con 0x7f65100cfc00 2024-09-10T23:36:46.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.890+0000 7f6518000640 1 -- 172.21.15.94:0/2580366353 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64f0072700 msgr2=0x7f64f0074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:46.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.890+0000 7f6518000640 1 --2- 172.21.15.94:0/2580366353 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64f0072700 0x7f64f0074bc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f65040b3000 tx=0x7f650401dde0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.890+0000 7f6518000640 1 -- 172.21.15.94:0/2580366353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65100cfc00 msgr2=0x7f651013d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:46.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.890+0000 7f6518000640 1 --2- 172.21.15.94:0/2580366353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65100cfc00 0x7f651013d500 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00c970 tx=0x7f64fc00ce30 comp rx=0 tx=0).stop 2024-09-10T23:36:46.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.890+0000 7f6518000640 1 -- 172.21.15.94:0/2580366353 shutdown_connections 2024-09-10T23:36:46.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.890+0000 7f6518000640 1 --2- 172.21.15.94:0/2580366353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f651015f2d0 0x7f65100ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.890+0000 7f6518000640 1 --2- 172.21.15.94:0/2580366353 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64f0072700 0x7f64f0074bc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.890+0000 7f6518000640 1 --2- 172.21.15.94:0/2580366353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65100cfc00 0x7f651013d500 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.890+0000 7f6518000640 1 --2- 172.21.15.94:0/2580366353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65100b2790 0x7f65101396c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:46.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.890+0000 7f6518000640 1 -- 172.21.15.94:0/2580366353 >> 172.21.15.94:0/2580366353 conn(0x7f65100d1c90 msgr2=0x7f6510155fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:46.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.890+0000 7f6518000640 1 -- 172.21.15.94:0/2580366353 shutdown_connections 2024-09-10T23:36:46.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:46.890+0000 7f6518000640 1 -- 172.21.15.94:0/2580366353 wait complete. 2024-09-10T23:36:46.912 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:36:46.912 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:36:47.016 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:36:47.116 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:36:47.116 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:36:47.116 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:47.125 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:36:47.125 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:36:47.125 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:47.134 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:36:47.134 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:36:47.134 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:47.173 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:36:47.173 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:36:47.173 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:36:47.190 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:36:47.190 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T23:36:47.190 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbc2a00640 1 -- 172.21.15.94:0/3693407174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbbc15bd30 msgr2=0x7fbbbc15c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbc2a00640 1 --2- 172.21.15.94:0/3693407174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbbc15bd30 0x7fbbbc15c110 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc073fc0 tx=0x7fbbac01c830 comp rx=0 tx=0).stop 2024-09-10T23:36:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbc2a00640 1 -- 172.21.15.94:0/3693407174 shutdown_connections 2024-09-10T23:36:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbc2a00640 1 --2- 172.21.15.94:0/3693407174 >> v2:172.21.15.181:3300/0 conn(0x7fbbbc0ae980 0x7fbbbc0d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbc2a00640 1 --2- 172.21.15.94:0/3693407174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbbc15bd30 0x7fbbbc15c110 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbc2a00640 1 -- 172.21.15.94:0/3693407174 >> 172.21.15.94:0/3693407174 conn(0x7fbbbc09aa70 msgr2=0x7fbbbc09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbc2a00640 1 -- 172.21.15.94:0/3693407174 shutdown_connections 2024-09-10T23:36:47.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbc2a00640 1 -- 172.21.15.94:0/3693407174 wait complete. 2024-09-10T23:36:47.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbc2a00640 1 Processor -- start 2024-09-10T23:36:47.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbc2a00640 1 -- start start 2024-09-10T23:36:47.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbc2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbbc0ae980 0x7fbbbc157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:47.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbc2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbbc0b2ec0 0x7fbbbc0ced70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:47.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbc2a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbbc15bd30 0x7fbbbc0ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:47.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbc2a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc169320 con 0x7fbbbc0ae980 2024-09-10T23:36:47.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbc2a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc1691a0 con 0x7fbbbc0b2ec0 2024-09-10T23:36:47.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbc2a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fbbbc1694a0 con 0x7fbbbc15bd30 2024-09-10T23:36:47.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.358+0000 7fbbbbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbbc0ae980 0x7fbbbc157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:47.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbbbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbbc0ae980 0x7fbbbc157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60436/0 (socket says 172.21.15.94:60436) 2024-09-10T23:36:47.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbbbe00640 1 -- 172.21.15.94:0/4015423731 learned_addr learned my addr 172.21.15.94:0/4015423731 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:47.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbbb400640 1 --2- 172.21.15.94:0/4015423731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbbc0b2ec0 0x7fbbbc0ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:47.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbc0a00640 1 --2- 172.21.15.94:0/4015423731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbbc15bd30 0x7fbbbc0ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:47.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbbbe00640 1 -- 172.21.15.94:0/4015423731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbbc15bd30 msgr2=0x7fbbbc0ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:47.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbbbe00640 1 --2- 172.21.15.94:0/4015423731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbbc15bd30 0x7fbbbc0ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:47.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbbbe00640 1 -- 172.21.15.94:0/4015423731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbbc0b2ec0 msgr2=0x7fbbbc0ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:47.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbbbe00640 1 --2- 172.21.15.94:0/4015423731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbbc0b2ec0 0x7fbbbc0ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:47.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbbbe00640 1 -- 172.21.15.94:0/4015423731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbbc0cbbd0 con 0x7fbbbc0ae980 2024-09-10T23:36:47.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbbb400640 1 --2- 172.21.15.94:0/4015423731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbbc0b2ec0 0x7fbbbc0ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:47.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbbbe00640 1 --2- 172.21.15.94:0/4015423731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbbc0ae980 0x7fbbbc157780 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbbac019440 tx=0x7fbbac0a5600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:47.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbb3e00640 1 -- 172.21.15.94:0/4015423731 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0bb070 con 0x7fbbbc0ae980 2024-09-10T23:36:47.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbb3e00640 1 -- 172.21.15.94:0/4015423731 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbac004320 con 0x7fbbbc0ae980 2024-09-10T23:36:47.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbb3e00640 1 -- 172.21.15.94:0/4015423731 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0b2040 con 0x7fbbbc0ae980 2024-09-10T23:36:47.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbc2a00640 1 -- 172.21.15.94:0/4015423731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbbc0cc360 con 0x7fbbbc0ae980 2024-09-10T23:36:47.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbc2a00640 1 -- 172.21.15.94:0/4015423731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbbc0ca0c0 con 0x7fbbbc0ae980 2024-09-10T23:36:47.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbc2a00640 1 -- 172.21.15.94:0/4015423731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbbc0d28a0 con 0x7fbbbc0ae980 2024-09-10T23:36:47.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.362+0000 7fbbb3e00640 1 -- 172.21.15.94:0/4015423731 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fbbac0052d0 con 0x7fbbbc0ae980 2024-09-10T23:36:47.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.366+0000 7fbbb3e00640 1 --2- 172.21.15.94:0/4015423731 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbb9c072700 0x7fbb9c074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:47.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.366+0000 7fbbb3e00640 1 -- 172.21.15.94:0/4015423731 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7fbbac12cad0 con 0x7fbbbc0ae980 2024-09-10T23:36:47.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.366+0000 7fbbbb400640 1 --2- 172.21.15.94:0/4015423731 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbb9c072700 0x7fbb9c074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:47.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.366+0000 7fbbbb400640 1 --2- 172.21.15.94:0/4015423731 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbb9c072700 0x7fbb9c074bc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbba80058d0 tx=0x7fbba8005860 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:47.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.370+0000 7fbbb3e00640 1 -- 172.21.15.94:0/4015423731 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbbac0164b0 con 0x7fbbbc0ae980 2024-09-10T23:36:47.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.566+0000 7fbbc2a00640 1 -- 172.21.15.94:0/4015423731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbbc0d1b60 con 0x7fbbbc0ae980 2024-09-10T23:36:47.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.570+0000 7fbbb3e00640 1 -- 172.21.15.94:0/4015423731 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+3914 (secure 0 0 0) 0x7fbbac0f7fc0 con 0x7fbbbc0ae980 2024-09-10T23:36:47.570 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:47.570 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":46,"btime":"2024-09-10T23:36:41:161281+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/3320339747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3320339747},{"type":"v1","addr":"172.21.15.94:6834","nonce":3320339747}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3123300514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3123300514},{"type":"v1","addr":"172.21.15.94:6835","nonce":3123300514}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/2058500499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2058500499},{"type":"v1","addr":"172.21.15.181:6801","nonce":2058500499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/339523200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":339523200},{"type":"v1","addr":"172.21.15.181:6837","nonce":339523200}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[]} 2024-09-10T23:36:47.570 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 46 2024-09-10T23:36:47.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.570+0000 7fbbc2a00640 1 -- 172.21.15.94:0/4015423731 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbb9c072700 msgr2=0x7fbb9c074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:47.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.570+0000 7fbbc2a00640 1 --2- 172.21.15.94:0/4015423731 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbb9c072700 0x7fbb9c074bc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbba80058d0 tx=0x7fbba8005860 comp rx=0 tx=0).stop 2024-09-10T23:36:47.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.570+0000 7fbbc2a00640 1 -- 172.21.15.94:0/4015423731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbbc0ae980 msgr2=0x7fbbbc157780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:47.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.570+0000 7fbbc2a00640 1 --2- 172.21.15.94:0/4015423731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbbc0ae980 0x7fbbbc157780 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbbac019440 tx=0x7fbbac0a5600 comp rx=0 tx=0).stop 2024-09-10T23:36:47.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.570+0000 7fbbc2a00640 1 -- 172.21.15.94:0/4015423731 shutdown_connections 2024-09-10T23:36:47.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.570+0000 7fbbc2a00640 1 --2- 172.21.15.94:0/4015423731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbbc0b2ec0 0x7fbbbc0ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.570+0000 7fbbc2a00640 1 --2- 172.21.15.94:0/4015423731 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbb9c072700 0x7fbb9c074bc0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.570+0000 7fbbc2a00640 1 --2- 172.21.15.94:0/4015423731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbbc15bd30 0x7fbbbc0ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.570+0000 7fbbc2a00640 1 --2- 172.21.15.94:0/4015423731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbbc0ae980 0x7fbbbc157780 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.570+0000 7fbbc2a00640 1 -- 172.21.15.94:0/4015423731 >> 172.21.15.94:0/4015423731 conn(0x7fbbbc09aa70 msgr2=0x7fbbbc0b6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.570+0000 7fbbc2a00640 1 -- 172.21.15.94:0/4015423731 shutdown_connections 2024-09-10T23:36:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.570+0000 7fbbc2a00640 1 -- 172.21.15.94:0/4015423731 wait complete. 2024-09-10T23:36:47.591 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T23:36:47.591 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T23:36:47.591 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T23:36:47.592 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:36:47.592 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T23:36:47.592 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T23:36:47.592 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T23:36:47.592 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:36:47.592 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T23:36:47.593 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T23:36:47.593 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T23:36:47.593 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:36:47.593 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T23:36:47.593 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T23:36:47.593 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T23:36:47.594 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:36:47.605 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T23:36:47.605 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T23:36:47.606 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T23:36:47.607 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T23:36:47.609 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T23:36:47.609 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T23:36:47.610 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T23:36:47.610 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T23:36:47.651 INFO:tasks.ceph.mds.a:Started 2024-09-10T23:36:47.652 DEBUG:teuthology.parallel:result is None 2024-09-10T23:36:47.652 INFO:tasks.ceph.mds.b:Started 2024-09-10T23:36:47.652 DEBUG:teuthology.parallel:result is None 2024-09-10T23:36:47.653 INFO:tasks.ceph.mds.c:Started 2024-09-10T23:36:47.653 DEBUG:teuthology.parallel:result is None 2024-09-10T23:36:47.654 INFO:tasks.ceph.mds.d:Started 2024-09-10T23:36:47.654 DEBUG:teuthology.parallel:result is None 2024-09-10T23:36:47.654 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T23:36:47.737 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-09-10T23:36:47.737 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-09-10T23:36:47.743 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:36:47.743 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2024-09-10T23:36:47.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.802+0000 7f34aee00640 1 -- 172.21.15.94:0/2656418118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34a80ce890 msgr2=0x7f34a80cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:47.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.802+0000 7f34aee00640 1 --2- 172.21.15.94:0/2656418118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34a80ce890 0x7f34a80cd710 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f3498009840 tx=0x7f349801c790 comp rx=0 tx=0).stop 2024-09-10T23:36:47.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.802+0000 7f34aee00640 1 -- 172.21.15.94:0/2656418118 shutdown_connections 2024-09-10T23:36:47.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.802+0000 7f34aee00640 1 --2- 172.21.15.94:0/2656418118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34a80ce890 0x7f34a80cd710 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:47.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.802+0000 7f34aee00640 1 -- 172.21.15.94:0/2656418118 >> 172.21.15.94:0/2656418118 conn(0x7f34a80d1c60 msgr2=0x7f34a80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:47.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.802+0000 7f34aee00640 1 -- 172.21.15.94:0/2656418118 shutdown_connections 2024-09-10T23:36:47.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.802+0000 7f34aee00640 1 -- 172.21.15.94:0/2656418118 wait complete. 2024-09-10T23:36:47.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34aee00640 1 Processor -- start 2024-09-10T23:36:47.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34aee00640 1 -- start start 2024-09-10T23:36:47.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34aee00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34a80b1100 0x7f34a80b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:47.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34aee00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34a80ce1e0 0x7f34a815c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:47.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34aee00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34a80ce890 0x7f34a815bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:47.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34aee00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f34a816a350 con 0x7f34a80ce1e0 2024-09-10T23:36:47.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34aee00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f34a816a1d0 con 0x7f34a80b1100 2024-09-10T23:36:47.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34aee00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f34a816a4d0 con 0x7f34a80ce890 2024-09-10T23:36:47.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f349f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34a80ce1e0 0x7f34a815c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:47.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f349f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34a80ce1e0 0x7f34a815c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60472/0 (socket says 172.21.15.94:60472) 2024-09-10T23:36:47.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f349f400640 1 -- 172.21.15.94:0/816173866 learned_addr learned my addr 172.21.15.94:0/816173866 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:47.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34a7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34a80b1100 0x7f34a80b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:47.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34ace00640 1 --2- 172.21.15.94:0/816173866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34a80ce890 0x7f34a815bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:47.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34a7e00640 1 -- 172.21.15.94:0/816173866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34a80ce890 msgr2=0x7f34a815bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:47.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34a7e00640 1 --2- 172.21.15.94:0/816173866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34a80ce890 0x7f34a815bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:47.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34a7e00640 1 -- 172.21.15.94:0/816173866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34a80ce1e0 msgr2=0x7f34a815c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:47.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34a7e00640 1 --2- 172.21.15.94:0/816173866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34a80ce1e0 0x7f34a815c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:47.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34a7e00640 1 -- 172.21.15.94:0/816173866 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a81f7d70 con 0x7f34a80b1100 2024-09-10T23:36:47.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f349f400640 1 --2- 172.21.15.94:0/816173866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34a80ce1e0 0x7f34a815c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:36:47.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34a7e00640 1 --2- 172.21.15.94:0/816173866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34a80b1100 0x7f34a80b6ac0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f34980ac000 tx=0x7f34980a6740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:47.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34a4c00640 1 -- 172.21.15.94:0/816173866 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34980053d0 con 0x7f34a80b1100 2024-09-10T23:36:47.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34a4c00640 1 -- 172.21.15.94:0/816173866 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3498005570 con 0x7f34a80b1100 2024-09-10T23:36:47.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34aee00640 1 -- 172.21.15.94:0/816173866 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a80b70a0 con 0x7f34a80b1100 2024-09-10T23:36:47.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34a4c00640 1 -- 172.21.15.94:0/816173866 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498004290 con 0x7f34a80b1100 2024-09-10T23:36:47.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.806+0000 7f34aee00640 1 -- 172.21.15.94:0/816173866 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a815c050 con 0x7f34a80b1100 2024-09-10T23:36:47.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.810+0000 7f34aee00640 1 -- 172.21.15.94:0/816173866 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a80cd710 con 0x7f34a80b1100 2024-09-10T23:36:47.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.810+0000 7f34a4c00640 1 -- 172.21.15.94:0/816173866 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f34980046d0 con 0x7f34a80b1100 2024-09-10T23:36:47.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.810+0000 7f34a4c00640 1 --2- 172.21.15.94:0/816173866 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3484072700 0x7f3484074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:47.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.810+0000 7f34a4c00640 1 -- 172.21.15.94:0/816173866 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5502+0+0 (secure 0 0 0) 0x7f349812d540 con 0x7f34a80b1100 2024-09-10T23:36:47.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.810+0000 7f349f400640 1 --2- 172.21.15.94:0/816173866 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3484072700 0x7f3484074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:47.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.810+0000 7f349f400640 1 --2- 172.21.15.94:0/816173866 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3484072700 0x7f3484074bc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f34a80c9a50 tx=0x7f3490009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:47.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.814+0000 7f34a4c00640 1 -- 172.21.15.94:0/816173866 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f34980bf030 con 0x7f34a80b1100 2024-09-10T23:36:47.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:47.990+0000 7f34aee00640 1 -- 172.21.15.94:0/816173866 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f34a80b4640 con 0x7f34a80b1100 2024-09-10T23:36:48.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.206+0000 7f34a4c00640 1 -- 172.21.15.94:0/816173866 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v75) ==== 97+0+0 (secure 0 0 0) 0x7f34980ac030 con 0x7f34a80b1100 2024-09-10T23:36:48.207 INFO:teuthology.orchestra.run.smithi094.stderr: removed all blocklist entries 2024-09-10T23:36:48.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.206+0000 7f34aee00640 1 -- 172.21.15.94:0/816173866 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3484072700 msgr2=0x7f3484074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:48.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.206+0000 7f34aee00640 1 --2- 172.21.15.94:0/816173866 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3484072700 0x7f3484074bc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f34a80c9a50 tx=0x7f3490009040 comp rx=0 tx=0).stop 2024-09-10T23:36:48.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.206+0000 7f34aee00640 1 -- 172.21.15.94:0/816173866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34a80b1100 msgr2=0x7f34a80b6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:48.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.206+0000 7f34aee00640 1 --2- 172.21.15.94:0/816173866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34a80b1100 0x7f34a80b6ac0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f34980ac000 tx=0x7f34980a6740 comp rx=0 tx=0).stop 2024-09-10T23:36:48.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.206+0000 7f34aee00640 1 -- 172.21.15.94:0/816173866 shutdown_connections 2024-09-10T23:36:48.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.206+0000 7f34aee00640 1 --2- 172.21.15.94:0/816173866 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3484072700 0x7f3484074bc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.206+0000 7f34aee00640 1 --2- 172.21.15.94:0/816173866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34a80ce890 0x7f34a815bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.206+0000 7f34aee00640 1 --2- 172.21.15.94:0/816173866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34a80ce1e0 0x7f34a815c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.206+0000 7f34aee00640 1 --2- 172.21.15.94:0/816173866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34a80b1100 0x7f34a80b6ac0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.206+0000 7f34aee00640 1 -- 172.21.15.94:0/816173866 >> 172.21.15.94:0/816173866 conn(0x7f34a80d1c60 msgr2=0x7f34a8071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:48.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.206+0000 7f34aee00640 1 -- 172.21.15.94:0/816173866 shutdown_connections 2024-09-10T23:36:48.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.206+0000 7f34aee00640 1 -- 172.21.15.94:0/816173866 wait complete. 2024-09-10T23:36:48.231 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T23:36:48.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dfa00640 1 -- 172.21.15.94:0/3376484340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22d806f600 msgr2=0x7f22d8072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:48.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dfa00640 1 --2- 172.21.15.94:0/3376484340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22d806f600 0x7f22d8072610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009870 tx=0x7f22c001c740 comp rx=0 tx=0).stop 2024-09-10T23:36:48.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dfa00640 1 -- 172.21.15.94:0/3376484340 shutdown_connections 2024-09-10T23:36:48.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dfa00640 1 --2- 172.21.15.94:0/3376484340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22d806f600 0x7f22d8072610 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dfa00640 1 -- 172.21.15.94:0/3376484340 >> 172.21.15.94:0/3376484340 conn(0x7f22d806cdf0 msgr2=0x7f22d806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:48.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dfa00640 1 -- 172.21.15.94:0/3376484340 shutdown_connections 2024-09-10T23:36:48.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dfa00640 1 -- 172.21.15.94:0/3376484340 wait complete. 2024-09-10T23:36:48.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dfa00640 1 Processor -- start 2024-09-10T23:36:48.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dfa00640 1 -- start start 2024-09-10T23:36:48.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dfa00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22d806f600 0x7f22d8161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:48.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dfa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22d8070150 0x7f22d813e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:48.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dfa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d8157c60 0x7f22d81f9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:48.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dfa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8168380 con 0x7f22d8070150 2024-09-10T23:36:48.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dfa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8168200 con 0x7f22d8157c60 2024-09-10T23:36:48.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dfa00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f22d8168500 con 0x7f22d806f600 2024-09-10T23:36:48.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22cfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22d8070150 0x7f22d813e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:48.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22cfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22d8070150 0x7f22d813e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60482/0 (socket says 172.21.15.94:60482) 2024-09-10T23:36:48.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22cfe00640 1 -- 172.21.15.94:0/741672458 learned_addr learned my addr 172.21.15.94:0/741672458 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:48.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dd000640 1 --2- 172.21.15.94:0/741672458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22d806f600 0x7f22d8161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:48.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.374+0000 7f22dda00640 1 --2- 172.21.15.94:0/741672458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d8157c60 0x7f22d81f9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:48.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.378+0000 7f22cfe00640 1 -- 172.21.15.94:0/741672458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22d806f600 msgr2=0x7f22d8161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:48.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.378+0000 7f22cfe00640 1 --2- 172.21.15.94:0/741672458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22d806f600 0x7f22d8161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.378+0000 7f22cfe00640 1 -- 172.21.15.94:0/741672458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d8157c60 msgr2=0x7f22d81f9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:48.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.378+0000 7f22cfe00640 1 --2- 172.21.15.94:0/741672458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d8157c60 0x7f22d81f9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.378+0000 7f22cfe00640 1 -- 172.21.15.94:0/741672458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d813e570 con 0x7f22d8070150 2024-09-10T23:36:48.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.378+0000 7f22cfe00640 1 --2- 172.21.15.94:0/741672458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22d8070150 0x7f22d813e030 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f22c0000c10 tx=0x7f22c00a65c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:48.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.378+0000 7f22d5600640 1 -- 172.21.15.94:0/741672458 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c0005030 con 0x7f22d8070150 2024-09-10T23:36:48.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.378+0000 7f22d5600640 1 -- 172.21.15.94:0/741672458 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22c00051d0 con 0x7f22d8070150 2024-09-10T23:36:48.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.378+0000 7f22d5600640 1 -- 172.21.15.94:0/741672458 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c0005600 con 0x7f22d8070150 2024-09-10T23:36:48.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.378+0000 7f22dfa00640 1 -- 172.21.15.94:0/741672458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d81603e0 con 0x7f22d8070150 2024-09-10T23:36:48.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.378+0000 7f22dfa00640 1 -- 172.21.15.94:0/741672458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d81609a0 con 0x7f22d8070150 2024-09-10T23:36:48.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.378+0000 7f22dfa00640 1 -- 172.21.15.94:0/741672458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d8072610 con 0x7f22d8070150 2024-09-10T23:36:48.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.378+0000 7f22d5600640 1 -- 172.21.15.94:0/741672458 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f22c00046d0 con 0x7f22d8070150 2024-09-10T23:36:48.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.378+0000 7f22d5600640 1 --2- 172.21.15.94:0/741672458 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f22b4072700 0x7f22b4074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:48.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.382+0000 7f22d5600640 1 -- 172.21.15.94:0/741672458 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4857+0+0 (secure 0 0 0) 0x7f22c012d080 con 0x7f22d8070150 2024-09-10T23:36:48.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.382+0000 7f22dd000640 1 --2- 172.21.15.94:0/741672458 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f22b4072700 0x7f22b4074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:48.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.382+0000 7f22dd000640 1 --2- 172.21.15.94:0/741672458 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f22b4072700 0x7f22b4074bc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f22c8001870 tx=0x7f22c8001380 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:48.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.382+0000 7f22d5600640 1 -- 172.21.15.94:0/741672458 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f22c00bf060 con 0x7f22d8070150 2024-09-10T23:36:48.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.610+0000 7f22dfa00640 1 -- 172.21.15.94:0/741672458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f22d8155730 con 0x7f22d8070150 2024-09-10T23:36:48.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.610+0000 7f22d5600640 1 -- 172.21.15.94:0/741672458 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v33) ==== 80+0+5304 (secure 0 0 0) 0x7f22c00ac030 con 0x7f22d8070150 2024-09-10T23:36:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:48.611 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:36:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_dump": [ 2024-09-10T23:36:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.a", 2024-09-10T23:36:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/2m3LxAAr0ErpLJg+0+I2r74zQya4w==", 2024-09-10T23:36:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:36:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:36:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:36:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.b", 2024-09-10T23:36:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmc8pYABAAw90zu8D+9IwK15h4HIlGBA==", 2024-09-10T23:36:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:36:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:36:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:36:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.c", 2024-09-10T23:36:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmk98KNxAA44e9N4VHbw/94Lh1V9zugQ==", 2024-09-10T23:36:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:36:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:36:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:36:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.d", 2024-09-10T23:36:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmVAKzBxAAJehOw6aphTF/g3ucoQfw9A==", 2024-09-10T23:36:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:36:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:36:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:36:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.0", 2024-09-10T23:36:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDa1uBmSAD3KBAAJTF1OEC44Ml599aL7iEuDw==", 2024-09-10T23:36:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:36:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:36:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.1", 2024-09-10T23:36:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDe1uBma1L9ABAAfEjW1/Hb4fh4Kwv/ZUgFAQ==", 2024-09-10T23:36:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:36:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:36:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.2", 2024-09-10T23:36:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDh1uBmtLUQFhAAweeFHj6BRQVC9BX+VLiWEQ==", 2024-09-10T23:36:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:36:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:36:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.3", 2024-09-10T23:36:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDk1uBm1KFCKRAAev28iYcrEYOViX95uo6Kbw==", 2024-09-10T23:36:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:36:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:36:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.4", 2024-09-10T23:36:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDq1uBm/Hg9LhAAvB2D0rj4DeRPxTrGmXYIsA==", 2024-09-10T23:36:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:36:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:36:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.5", 2024-09-10T23:36:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDu1uBm1fAKCBAAFHMjnGFAItLdaACF/UjE/w==", 2024-09-10T23:36:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:36:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:36:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.6", 2024-09-10T23:36:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDx1uBmXgYfHhAA/6wwtM4mDRXsIpoTIHKJ0w==", 2024-09-10T23:36:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:36:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:36:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.7", 2024-09-10T23:36:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQD01uBmw6hQMxAAk7oGAg+iIplNK5KrokGwNA==", 2024-09-10T23:36:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:36:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:36:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.0", 2024-09-10T23:36:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g==", 2024-09-10T23:36:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:36:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow r", 2024-09-10T23:36:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T23:36:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.1", 2024-09-10T23:36:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmuSkhFRAA1R6WNXLImkGfhW9/XJzEAQ==", 2024-09-10T23:36:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:36:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:36:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:36:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:36:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.2", 2024-09-10T23:36:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmfvaqERAASpL0KtglFcAuRf4a/0z5JQ==", 2024-09-10T23:36:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:36:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:36:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:36:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:36:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.3", 2024-09-10T23:36:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBm8xUtGxAARc0ywfAhMxBNd57TzBoV3g==", 2024-09-10T23:36:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:36:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:36:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:36:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:36:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.admin", 2024-09-10T23:36:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmOpg1FBAA9NV6Jqt6doWMuAfcxztlOg==", 2024-09-10T23:36:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:36:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:36:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:36:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.x", 2024-09-10T23:36:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/EPVKxAA/Myv8Zjf7BejbzlkxyCC1w==", 2024-09-10T23:36:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:36:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:36:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.y", 2024-09-10T23:36:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmhHvtJxAACe+79nYhv2ODwtmNhA45FA==", 2024-09-10T23:36:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:36:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:36:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:36:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:36:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:36:48.624 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:36:48.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.610+0000 7f22dfa00640 1 -- 172.21.15.94:0/741672458 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f22b4072700 msgr2=0x7f22b4074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:48.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.610+0000 7f22dfa00640 1 --2- 172.21.15.94:0/741672458 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f22b4072700 0x7f22b4074bc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f22c8001870 tx=0x7f22c8001380 comp rx=0 tx=0).stop 2024-09-10T23:36:48.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.610+0000 7f22dfa00640 1 -- 172.21.15.94:0/741672458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22d8070150 msgr2=0x7f22d813e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:48.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.610+0000 7f22dfa00640 1 --2- 172.21.15.94:0/741672458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22d8070150 0x7f22d813e030 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f22c0000c10 tx=0x7f22c00a65c0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.610+0000 7f22dfa00640 1 -- 172.21.15.94:0/741672458 shutdown_connections 2024-09-10T23:36:48.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.610+0000 7f22dfa00640 1 --2- 172.21.15.94:0/741672458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d8157c60 0x7f22d81f9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.610+0000 7f22dfa00640 1 --2- 172.21.15.94:0/741672458 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f22b4072700 0x7f22b4074bc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.610+0000 7f22dfa00640 1 --2- 172.21.15.94:0/741672458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22d8070150 0x7f22d813e030 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.610+0000 7f22dfa00640 1 --2- 172.21.15.94:0/741672458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22d806f600 0x7f22d8161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.610+0000 7f22dfa00640 1 -- 172.21.15.94:0/741672458 >> 172.21.15.94:0/741672458 conn(0x7f22d806cdf0 msgr2=0x7f22d80ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:48.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.610+0000 7f22dfa00640 1 -- 172.21.15.94:0/741672458 shutdown_connections 2024-09-10T23:36:48.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.610+0000 7f22dfa00640 1 -- 172.21.15.94:0/741672458 wait complete. 2024-09-10T23:36:48.632 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T23:36:48.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa70200640 1 -- 172.21.15.94:0/17104809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa6815bd30 msgr2=0x7faa6815c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:48.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa70200640 1 --2- 172.21.15.94:0/17104809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa6815bd30 0x7faa6815c110 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009900 tx=0x7faa5c01c850 comp rx=0 tx=0).stop 2024-09-10T23:36:48.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa70200640 1 -- 172.21.15.94:0/17104809 shutdown_connections 2024-09-10T23:36:48.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa70200640 1 --2- 172.21.15.94:0/17104809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa680ae980 0x7faa680d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa70200640 1 --2- 172.21.15.94:0/17104809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa6815bd30 0x7faa6815c110 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa70200640 1 -- 172.21.15.94:0/17104809 >> 172.21.15.94:0/17104809 conn(0x7faa6809aa70 msgr2=0x7faa6809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:48.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa70200640 1 -- 172.21.15.94:0/17104809 shutdown_connections 2024-09-10T23:36:48.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa70200640 1 -- 172.21.15.94:0/17104809 wait complete. 2024-09-10T23:36:48.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa70200640 1 Processor -- start 2024-09-10T23:36:48.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa70200640 1 -- start start 2024-09-10T23:36:48.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa70200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa680ae980 0x7faa681fb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:48.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa70200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa680b2ec0 0x7faa68157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:48.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa70200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6815bd30 0x7faa680b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:48.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa70200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa681692f0 con 0x7faa6815bd30 2024-09-10T23:36:48.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa70200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faa68169170 con 0x7faa680ae980 2024-09-10T23:36:48.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa70200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7faa68169470 con 0x7faa680b2ec0 2024-09-10T23:36:48.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa6e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6815bd30 0x7faa680b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:48.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa6d800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa680ae980 0x7faa681fb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:48.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa6ce00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa680b2ec0 0x7faa68157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:48.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa6d800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa680ae980 0x7faa681fb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:50146/0 (socket says 172.21.15.94:50146) 2024-09-10T23:36:48.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa6d800640 1 -- 172.21.15.94:0/2801662095 learned_addr learned my addr 172.21.15.94:0/2801662095 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:48.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa6d800640 1 -- 172.21.15.94:0/2801662095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa680b2ec0 msgr2=0x7faa68157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:48.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa6d800640 1 --2- 172.21.15.94:0/2801662095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa680b2ec0 0x7faa68157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa6d800640 1 -- 172.21.15.94:0/2801662095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6815bd30 msgr2=0x7faa680b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:48.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa6d800640 1 --2- 172.21.15.94:0/2801662095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6815bd30 0x7faa680b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa6d800640 1 -- 172.21.15.94:0/2801662095 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa680b5860 con 0x7faa680ae980 2024-09-10T23:36:48.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.774+0000 7faa6d800640 1 --2- 172.21.15.94:0/2801662095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa680ae980 0x7faa681fb9a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0b0040 tx=0x7faa5c005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:48.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.778+0000 7faa65600640 1 -- 172.21.15.94:0/2801662095 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0b2040 con 0x7faa680ae980 2024-09-10T23:36:48.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.778+0000 7faa65600640 1 -- 172.21.15.94:0/2801662095 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa5c004230 con 0x7faa680ae980 2024-09-10T23:36:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.778+0000 7faa65600640 1 -- 172.21.15.94:0/2801662095 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c005030 con 0x7faa680ae980 2024-09-10T23:36:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.778+0000 7faa70200640 1 -- 172.21.15.94:0/2801662095 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa681fc110 con 0x7faa680ae980 2024-09-10T23:36:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.778+0000 7faa70200640 1 -- 172.21.15.94:0/2801662095 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa681fc6d0 con 0x7faa680ae980 2024-09-10T23:36:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.778+0000 7faa70200640 1 -- 172.21.15.94:0/2801662095 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6806fc70 con 0x7faa680ae980 2024-09-10T23:36:48.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.778+0000 7faa65600640 1 -- 172.21.15.94:0/2801662095 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7faa5c0051d0 con 0x7faa680ae980 2024-09-10T23:36:48.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.778+0000 7faa65600640 1 --2- 172.21.15.94:0/2801662095 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa48072700 0x7faa48074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:48.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.778+0000 7faa65600640 1 -- 172.21.15.94:0/2801662095 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4857+0+0 (secure 0 0 0) 0x7faa5c12cb00 con 0x7faa680ae980 2024-09-10T23:36:48.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.778+0000 7faa6ce00640 1 --2- 172.21.15.94:0/2801662095 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa48072700 0x7faa48074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:48.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.782+0000 7faa6ce00640 1 --2- 172.21.15.94:0/2801662095 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa48072700 0x7faa48074bc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faa680c55a0 tx=0x7faa540091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:48.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.782+0000 7faa65600640 1 -- 172.21.15.94:0/2801662095 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faa5c0164d0 con 0x7faa680ae980 2024-09-10T23:36:48.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.950+0000 7faa70200640 1 -- 172.21.15.94:0/2801662095 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7faa680d1b80 con 0x7faa680ae980 2024-09-10T23:36:48.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.950+0000 7faa65600640 1 -- 172.21.15.94:0/2801662095 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7faa5c0f7d30 con 0x7faa680ae980 2024-09-10T23:36:48.952 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:48.953 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:36:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:36:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-09-10T23:36:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-09-10T23:36:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:48.953 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:36:48.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.950+0000 7faa70200640 1 -- 172.21.15.94:0/2801662095 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa48072700 msgr2=0x7faa48074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:48.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.950+0000 7faa70200640 1 --2- 172.21.15.94:0/2801662095 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa48072700 0x7faa48074bc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faa680c55a0 tx=0x7faa540091e0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.950+0000 7faa70200640 1 -- 172.21.15.94:0/2801662095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa680ae980 msgr2=0x7faa681fb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:48.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.950+0000 7faa70200640 1 --2- 172.21.15.94:0/2801662095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa680ae980 0x7faa681fb9a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0b0040 tx=0x7faa5c005de0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.950+0000 7faa70200640 1 -- 172.21.15.94:0/2801662095 shutdown_connections 2024-09-10T23:36:48.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.950+0000 7faa70200640 1 --2- 172.21.15.94:0/2801662095 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa48072700 0x7faa48074bc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.950+0000 7faa70200640 1 --2- 172.21.15.94:0/2801662095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6815bd30 0x7faa680b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.950+0000 7faa70200640 1 --2- 172.21.15.94:0/2801662095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa680b2ec0 0x7faa68157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.950+0000 7faa70200640 1 --2- 172.21.15.94:0/2801662095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa680ae980 0x7faa681fb9a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:48.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.950+0000 7faa70200640 1 -- 172.21.15.94:0/2801662095 >> 172.21.15.94:0/2801662095 conn(0x7faa6809aa70 msgr2=0x7faa6815a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:48.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.950+0000 7faa70200640 1 -- 172.21.15.94:0/2801662095 shutdown_connections 2024-09-10T23:36:48.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:48.950+0000 7faa70200640 1 -- 172.21.15.94:0/2801662095 wait complete. 2024-09-10T23:36:48.972 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T23:36:48.972 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T23:36:49.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.114+0000 7f3210200640 1 -- 172.21.15.94:0/3631913776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32080cdea0 msgr2=0x7f32080ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:49.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.114+0000 7f3210200640 1 --2- 172.21.15.94:0/3631913776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32080cdea0 0x7f32080ca5d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f31f4009900 tx=0x7f31f401c850 comp rx=0 tx=0).stop 2024-09-10T23:36:49.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.114+0000 7f3210200640 1 -- 172.21.15.94:0/3631913776 shutdown_connections 2024-09-10T23:36:49.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.114+0000 7f3210200640 1 --2- 172.21.15.94:0/3631913776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32080cdea0 0x7f32080ca5d0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:49.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.114+0000 7f3210200640 1 -- 172.21.15.94:0/3631913776 >> 172.21.15.94:0/3631913776 conn(0x7f32080cc900 msgr2=0x7f32080ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:49.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.114+0000 7f3210200640 1 -- 172.21.15.94:0/3631913776 shutdown_connections 2024-09-10T23:36:49.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.114+0000 7f3210200640 1 -- 172.21.15.94:0/3631913776 wait complete. 2024-09-10T23:36:49.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f3210200640 1 Processor -- start 2024-09-10T23:36:49.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f3210200640 1 -- start start 2024-09-10T23:36:49.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f3210200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3208073970 0x7f320814e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:49.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f3210200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32080cab10 0x7f320809d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:49.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f3210200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32080cdea0 0x7f3208156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:49.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f3210200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3208168ef0 con 0x7f32080cdea0 2024-09-10T23:36:49.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f3210200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3208168d70 con 0x7f3208073970 2024-09-10T23:36:49.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f3210200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3208169070 con 0x7f32080cab10 2024-09-10T23:36:49.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f320e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32080cdea0 0x7f3208156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:49.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f320e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32080cdea0 0x7f3208156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60498/0 (socket says 172.21.15.94:60498) 2024-09-10T23:36:49.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f320e200640 1 -- 172.21.15.94:0/492523929 learned_addr learned my addr 172.21.15.94:0/492523929 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:49.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f320d800640 1 --2- 172.21.15.94:0/492523929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3208073970 0x7f320814e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:49.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f320ce00640 1 --2- 172.21.15.94:0/492523929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32080cab10 0x7f320809d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f320e200640 1 -- 172.21.15.94:0/492523929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32080cab10 msgr2=0x7f320809d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f320e200640 1 --2- 172.21.15.94:0/492523929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32080cab10 0x7f320809d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f320e200640 1 -- 172.21.15.94:0/492523929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3208073970 msgr2=0x7f320814e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f320e200640 1 --2- 172.21.15.94:0/492523929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3208073970 0x7f320814e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f320e200640 1 -- 172.21.15.94:0/492523929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320814eb50 con 0x7f32080cdea0 2024-09-10T23:36:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f320e200640 1 --2- 172.21.15.94:0/492523929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32080cdea0 0x7f3208156910 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f320000c770 tx=0x7f320000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f3205600640 1 -- 172.21.15.94:0/492523929 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200017070 con 0x7f32080cdea0 2024-09-10T23:36:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f3205600640 1 -- 172.21.15.94:0/492523929 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3200005070 con 0x7f32080cdea0 2024-09-10T23:36:49.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f3205600640 1 -- 172.21.15.94:0/492523929 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32000054c0 con 0x7f32080cdea0 2024-09-10T23:36:49.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f3210200640 1 -- 172.21.15.94:0/492523929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320809b840 con 0x7f32080cdea0 2024-09-10T23:36:49.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f3210200640 1 -- 172.21.15.94:0/492523929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32081563f0 con 0x7f32080cdea0 2024-09-10T23:36:49.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.118+0000 7f3210200640 1 -- 172.21.15.94:0/492523929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f320809bd40 con 0x7f32080cdea0 2024-09-10T23:36:49.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.122+0000 7f3205600640 1 -- 172.21.15.94:0/492523929 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f3200005660 con 0x7f32080cdea0 2024-09-10T23:36:49.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.122+0000 7f3205600640 1 --2- 172.21.15.94:0/492523929 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f31e8072730 0x7f31e8074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:49.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.122+0000 7f3205600640 1 -- 172.21.15.94:0/492523929 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4857+0+0 (secure 0 0 0) 0x7f32000929a0 con 0x7f32080cdea0 2024-09-10T23:36:49.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.122+0000 7f320d800640 1 --2- 172.21.15.94:0/492523929 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f31e8072730 0x7f31e8074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:49.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.122+0000 7f320d800640 1 --2- 172.21.15.94:0/492523929 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f31e8072730 0x7f31e8074bf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3208072f90 tx=0x7f31fc007e40 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:49.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.126+0000 7f3205600640 1 -- 172.21.15.94:0/492523929 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f320809bd40 con 0x7f32080cdea0 2024-09-10T23:36:49.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:49.294+0000 7f3210200640 1 -- 172.21.15.94:0/492523929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f32080ca5b0 con 0x7f32080cdea0 2024-09-10T23:36:50.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.214+0000 7f3205600640 1 -- 172.21.15.94:0/492523929 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7f320005db60 con 0x7f32080cdea0 2024-09-10T23:36:50.215 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-09-10T23:36:50.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.218+0000 7f3210200640 1 -- 172.21.15.94:0/492523929 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f31e8072730 msgr2=0x7f31e8074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:50.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.218+0000 7f3210200640 1 --2- 172.21.15.94:0/492523929 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f31e8072730 0x7f31e8074bf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3208072f90 tx=0x7f31fc007e40 comp rx=0 tx=0).stop 2024-09-10T23:36:50.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.218+0000 7f3210200640 1 -- 172.21.15.94:0/492523929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32080cdea0 msgr2=0x7f3208156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:50.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.218+0000 7f3210200640 1 --2- 172.21.15.94:0/492523929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32080cdea0 0x7f3208156910 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f320000c770 tx=0x7f320000cc30 comp rx=0 tx=0).stop 2024-09-10T23:36:50.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.218+0000 7f3210200640 1 -- 172.21.15.94:0/492523929 shutdown_connections 2024-09-10T23:36:50.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.218+0000 7f3210200640 1 --2- 172.21.15.94:0/492523929 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f31e8072730 0x7f31e8074bf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:50.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.218+0000 7f3210200640 1 --2- 172.21.15.94:0/492523929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32080cdea0 0x7f3208156910 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:50.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.218+0000 7f3210200640 1 --2- 172.21.15.94:0/492523929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32080cab10 0x7f320809d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:50.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.218+0000 7f3210200640 1 --2- 172.21.15.94:0/492523929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3208073970 0x7f320814e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:50.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.218+0000 7f3210200640 1 -- 172.21.15.94:0/492523929 >> 172.21.15.94:0/492523929 conn(0x7f32080cc900 msgr2=0x7f3208073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:50.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.218+0000 7f3210200640 1 -- 172.21.15.94:0/492523929 shutdown_connections 2024-09-10T23:36:50.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.218+0000 7f3210200640 1 -- 172.21.15.94:0/492523929 wait complete. 2024-09-10T23:36:50.247 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T23:36:50.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cf200640 1 -- 172.21.15.94:0/2976350088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c80c49b0 msgr2=0x7f95c80cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:50.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cf200640 1 --2- 172.21.15.94:0/2976350088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c80c49b0 0x7f95c80cadc0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009900 tx=0x7f95b001c8e0 comp rx=0 tx=0).stop 2024-09-10T23:36:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cf200640 1 -- 172.21.15.94:0/2976350088 shutdown_connections 2024-09-10T23:36:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cf200640 1 --2- 172.21.15.94:0/2976350088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c80c49b0 0x7f95c80cadc0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cf200640 1 --2- 172.21.15.94:0/2976350088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95c8158790 0x7f95c806fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cf200640 1 -- 172.21.15.94:0/2976350088 >> 172.21.15.94:0/2976350088 conn(0x7f95c80cfaf0 msgr2=0x7f95c80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cf200640 1 -- 172.21.15.94:0/2976350088 shutdown_connections 2024-09-10T23:36:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cf200640 1 -- 172.21.15.94:0/2976350088 wait complete. 2024-09-10T23:36:50.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cf200640 1 Processor -- start 2024-09-10T23:36:50.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cf200640 1 -- start start 2024-09-10T23:36:50.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cf200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95c80c49b0 0x7f95c8073f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:50.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cf200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c80cb300 0x7f95c80ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:50.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cf200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c8158790 0x7f95c80aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:50.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cf200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95c8168290 con 0x7f95c8158790 2024-09-10T23:36:50.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cf200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f95c8168110 con 0x7f95c80cb300 2024-09-10T23:36:50.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cf200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f95c8168410 con 0x7f95c80c49b0 2024-09-10T23:36:50.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cd200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c8158790 0x7f95c80aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:50.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cd200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c8158790 0x7f95c80aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60504/0 (socket says 172.21.15.94:60504) 2024-09-10T23:36:50.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95c7e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95c80c49b0 0x7f95c8073f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:50.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cd200640 1 -- 172.21.15.94:0/2546989085 learned_addr learned my addr 172.21.15.94:0/2546989085 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:50.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95bf400640 1 --2- 172.21.15.94:0/2546989085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c80cb300 0x7f95c80ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:50.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cd200640 1 -- 172.21.15.94:0/2546989085 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95c80c49b0 msgr2=0x7f95c8073f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:50.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cd200640 1 --2- 172.21.15.94:0/2546989085 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95c80c49b0 0x7f95c8073f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:50.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cd200640 1 -- 172.21.15.94:0/2546989085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c80cb300 msgr2=0x7f95c80ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:50.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cd200640 1 --2- 172.21.15.94:0/2546989085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c80cb300 0x7f95c80ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:50.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.398+0000 7f95cd200640 1 -- 172.21.15.94:0/2546989085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c81f8f40 con 0x7f95c8158790 2024-09-10T23:36:50.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.402+0000 7f95cd200640 1 --2- 172.21.15.94:0/2546989085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c8158790 0x7f95c80aee30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f95c000c780 tx=0x7f95c000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:50.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.402+0000 7f95c4c00640 1 -- 172.21.15.94:0/2546989085 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c0003ba0 con 0x7f95c8158790 2024-09-10T23:36:50.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.402+0000 7f95cf200640 1 -- 172.21.15.94:0/2546989085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c81f9220 con 0x7f95c8158790 2024-09-10T23:36:50.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.402+0000 7f95c4c00640 1 -- 172.21.15.94:0/2546989085 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95c0003d60 con 0x7f95c8158790 2024-09-10T23:36:50.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.402+0000 7f95c4c00640 1 -- 172.21.15.94:0/2546989085 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c000aab0 con 0x7f95c8158790 2024-09-10T23:36:50.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.402+0000 7f95cf200640 1 -- 172.21.15.94:0/2546989085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c8073720 con 0x7f95c8158790 2024-09-10T23:36:50.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.402+0000 7f95cf200640 1 -- 172.21.15.94:0/2546989085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c80cadc0 con 0x7f95c8158790 2024-09-10T23:36:50.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.402+0000 7f95c4c00640 1 -- 172.21.15.94:0/2546989085 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f95c000ac50 con 0x7f95c8158790 2024-09-10T23:36:50.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.402+0000 7f95c4c00640 1 --2- 172.21.15.94:0/2546989085 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95a4072700 0x7f95a4074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:50.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.402+0000 7f95c4c00640 1 -- 172.21.15.94:0/2546989085 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5227+0+0 (secure 0 0 0) 0x7f95c0093bb0 con 0x7f95c8158790 2024-09-10T23:36:50.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.402+0000 7f95c7e00640 1 --2- 172.21.15.94:0/2546989085 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95a4072700 0x7f95a4074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:50.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.406+0000 7f95c7e00640 1 --2- 172.21.15.94:0/2546989085 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95a4072700 0x7f95a4074bc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f95c80c9f10 tx=0x7f95b80091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:50.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.406+0000 7f95c4c00640 1 -- 172.21.15.94:0/2546989085 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f95c005eb60 con 0x7f95c8158790 2024-09-10T23:36:50.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:50.570+0000 7f95cf200640 1 -- 172.21.15.94:0/2546989085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f95c80d0850 con 0x7f95c8158790 2024-09-10T23:36:51.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.222+0000 7f95c4c00640 1 -- 172.21.15.94:0/2546989085 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) ==== 170+0+0 (secure 0 0 0) 0x7f95c0063590 con 0x7f95c8158790 2024-09-10T23:36:51.225 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-09-10T23:36:51.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.226+0000 7f95cf200640 1 -- 172.21.15.94:0/2546989085 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95a4072700 msgr2=0x7f95a4074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:51.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.226+0000 7f95cf200640 1 --2- 172.21.15.94:0/2546989085 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95a4072700 0x7f95a4074bc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f95c80c9f10 tx=0x7f95b80091e0 comp rx=0 tx=0).stop 2024-09-10T23:36:51.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.226+0000 7f95cf200640 1 -- 172.21.15.94:0/2546989085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c8158790 msgr2=0x7f95c80aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:51.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.226+0000 7f95cf200640 1 --2- 172.21.15.94:0/2546989085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c8158790 0x7f95c80aee30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f95c000c780 tx=0x7f95c000cc40 comp rx=0 tx=0).stop 2024-09-10T23:36:51.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.226+0000 7f95cf200640 1 -- 172.21.15.94:0/2546989085 shutdown_connections 2024-09-10T23:36:51.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.226+0000 7f95cf200640 1 --2- 172.21.15.94:0/2546989085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c80cb300 0x7f95c80ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:51.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.226+0000 7f95cf200640 1 --2- 172.21.15.94:0/2546989085 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95a4072700 0x7f95a4074bc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:51.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.226+0000 7f95cf200640 1 --2- 172.21.15.94:0/2546989085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c8158790 0x7f95c80aee30 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:51.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.226+0000 7f95cf200640 1 --2- 172.21.15.94:0/2546989085 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95c80c49b0 0x7f95c8073f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:51.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.226+0000 7f95cf200640 1 -- 172.21.15.94:0/2546989085 >> 172.21.15.94:0/2546989085 conn(0x7f95c80cfaf0 msgr2=0x7f95c80b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:51.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.226+0000 7f95cf200640 1 -- 172.21.15.94:0/2546989085 shutdown_connections 2024-09-10T23:36:51.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.226+0000 7f95cf200640 1 -- 172.21.15.94:0/2546989085 wait complete. 2024-09-10T23:36:51.267 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T23:36:51.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.410+0000 7f25eca00640 1 -- 172.21.15.94:0/2067596630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25e815f570 msgr2=0x7f25e80b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:51.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.410+0000 7f25eca00640 1 --2- 172.21.15.94:0/2067596630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25e815f570 0x7f25e80b30b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009bc0 tx=0x7f25e001cb10 comp rx=0 tx=0).stop 2024-09-10T23:36:51.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.410+0000 7f25eca00640 1 -- 172.21.15.94:0/2067596630 shutdown_connections 2024-09-10T23:36:51.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.410+0000 7f25eca00640 1 --2- 172.21.15.94:0/2067596630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25e815f570 0x7f25e80b30b0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:51.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.410+0000 7f25eca00640 1 -- 172.21.15.94:0/2067596630 >> 172.21.15.94:0/2067596630 conn(0x7f25e80d1b10 msgr2=0x7f25e80d1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:51.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.410+0000 7f25eca00640 1 -- 172.21.15.94:0/2067596630 shutdown_connections 2024-09-10T23:36:51.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.410+0000 7f25eca00640 1 -- 172.21.15.94:0/2067596630 wait complete. 2024-09-10T23:36:51.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25eca00640 1 Processor -- start 2024-09-10T23:36:51.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25eca00640 1 -- start start 2024-09-10T23:36:51.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25eca00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25e80cfc70 0x7f25e8151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:51.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25eca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25e815f570 0x7f25e80c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:51.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25eca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25e80c6ae0 0x7f25e815cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:51.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25eca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f25e8169d00 con 0x7f25e815f570 2024-09-10T23:36:51.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25eca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f25e8169b80 con 0x7f25e80c6ae0 2024-09-10T23:36:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25eca00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f25e8169e80 con 0x7f25e80cfc70 2024-09-10T23:36:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25e5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25e815f570 0x7f25e80c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25e5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25e815f570 0x7f25e80c65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60522/0 (socket says 172.21.15.94:60522) 2024-09-10T23:36:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25e5e00640 1 -- 172.21.15.94:0/4119819269 learned_addr learned my addr 172.21.15.94:0/4119819269 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25e6800640 1 --2- 172.21.15.94:0/4119819269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25e80cfc70 0x7f25e8151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25e7200640 1 --2- 172.21.15.94:0/4119819269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25e80c6ae0 0x7f25e815cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25e5e00640 1 -- 172.21.15.94:0/4119819269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25e80cfc70 msgr2=0x7f25e8151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25e5e00640 1 --2- 172.21.15.94:0/4119819269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25e80cfc70 0x7f25e8151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25e5e00640 1 -- 172.21.15.94:0/4119819269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25e80c6ae0 msgr2=0x7f25e815cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25e5e00640 1 --2- 172.21.15.94:0/4119819269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25e80c6ae0 0x7f25e815cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25e5e00640 1 -- 172.21.15.94:0/4119819269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e80c7040 con 0x7f25e815f570 2024-09-10T23:36:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25e5e00640 1 --2- 172.21.15.94:0/4119819269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25e815f570 0x7f25e80c65a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f25d400c420 tx=0x7f25d400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:51.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25dea00640 1 -- 172.21.15.94:0/4119819269 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4017070 con 0x7f25e815f570 2024-09-10T23:36:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25dea00640 1 -- 172.21.15.94:0/4119819269 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25d4003600 con 0x7f25e815f570 2024-09-10T23:36:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25dea00640 1 -- 172.21.15.94:0/4119819269 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4003a90 con 0x7f25e815f570 2024-09-10T23:36:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25eca00640 1 -- 172.21.15.94:0/4119819269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e80c7320 con 0x7f25e815f570 2024-09-10T23:36:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.414+0000 7f25eca00640 1 -- 172.21.15.94:0/4119819269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e8159670 con 0x7f25e815f570 2024-09-10T23:36:51.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.418+0000 7f25eca00640 1 -- 172.21.15.94:0/4119819269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e80b30b0 con 0x7f25e815f570 2024-09-10T23:36:51.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.418+0000 7f25dea00640 1 -- 172.21.15.94:0/4119819269 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f25d4003c30 con 0x7f25e815f570 2024-09-10T23:36:51.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.422+0000 7f25dea00640 1 --2- 172.21.15.94:0/4119819269 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f25c8072700 0x7f25c8074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:51.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.422+0000 7f25dea00640 1 -- 172.21.15.94:0/4119819269 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5609+0+0 (secure 0 0 0) 0x7f25d4092ce0 con 0x7f25e815f570 2024-09-10T23:36:51.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.422+0000 7f25e6800640 1 --2- 172.21.15.94:0/4119819269 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f25c8072700 0x7f25c8074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:51.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.422+0000 7f25e6800640 1 --2- 172.21.15.94:0/4119819269 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f25c8072700 0x7f25c8074bc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f25d8007fd0 tx=0x7f25d8009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:51.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.426+0000 7f25dea00640 1 -- 172.21.15.94:0/4119819269 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f25d4021020 con 0x7f25e815f570 2024-09-10T23:36:51.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:51.622+0000 7f25eca00640 1 -- 172.21.15.94:0/4119819269 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f25e80d2100 con 0x7f25e815f570 2024-09-10T23:36:52.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.226+0000 7f25dea00640 1 -- 172.21.15.94:0/4119819269 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v49) ==== 174+0+0 (secure 0 0 0) 0x7f25d405dc90 con 0x7f25e815f570 2024-09-10T23:36:52.228 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 12 and data pool 13 2024-09-10T23:36:52.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.226+0000 7f25eca00640 1 -- 172.21.15.94:0/4119819269 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f25c8072700 msgr2=0x7f25c8074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:52.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.226+0000 7f25eca00640 1 --2- 172.21.15.94:0/4119819269 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f25c8072700 0x7f25c8074bc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f25d8007fd0 tx=0x7f25d8009040 comp rx=0 tx=0).stop 2024-09-10T23:36:52.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.226+0000 7f25eca00640 1 -- 172.21.15.94:0/4119819269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25e815f570 msgr2=0x7f25e80c65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:52.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.226+0000 7f25eca00640 1 --2- 172.21.15.94:0/4119819269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25e815f570 0x7f25e80c65a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f25d400c420 tx=0x7f25d400c8e0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.226+0000 7f25eca00640 1 -- 172.21.15.94:0/4119819269 shutdown_connections 2024-09-10T23:36:52.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.226+0000 7f25eca00640 1 --2- 172.21.15.94:0/4119819269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25e80c6ae0 0x7f25e815cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.226+0000 7f25eca00640 1 --2- 172.21.15.94:0/4119819269 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f25c8072700 0x7f25c8074bc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.226+0000 7f25eca00640 1 --2- 172.21.15.94:0/4119819269 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25e815f570 0x7f25e80c65a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.226+0000 7f25eca00640 1 --2- 172.21.15.94:0/4119819269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25e80cfc70 0x7f25e8151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.230+0000 7f25eca00640 1 -- 172.21.15.94:0/4119819269 >> 172.21.15.94:0/4119819269 conn(0x7f25e80d1b10 msgr2=0x7f25e815b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:52.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.230+0000 7f25eca00640 1 -- 172.21.15.94:0/4119819269 shutdown_connections 2024-09-10T23:36:52.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.230+0000 7f25eca00640 1 -- 172.21.15.94:0/4119819269 wait complete. 2024-09-10T23:36:52.270 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:36:52.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.410+0000 7f9ae0800640 1 -- 172.21.15.94:0/3419607282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad80d02e0 msgr2=0x7f9ad815e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:52.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.410+0000 7f9ae0800640 1 --2- 172.21.15.94:0/3419607282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad80d02e0 0x7f9ad815e770 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80d0b80 tx=0x7f9ac801c890 comp rx=0 tx=0).stop 2024-09-10T23:36:52.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.410+0000 7f9ae0800640 1 -- 172.21.15.94:0/3419607282 shutdown_connections 2024-09-10T23:36:52.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.410+0000 7f9ae0800640 1 --2- 172.21.15.94:0/3419607282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad80d02e0 0x7f9ad815e770 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.410+0000 7f9ae0800640 1 --2- 172.21.15.94:0/3419607282 >> v2:172.21.15.181:3300/0 conn(0x7f9ad8073ff0 0x7f9ad80cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.410+0000 7f9ae0800640 1 -- 172.21.15.94:0/3419607282 >> 172.21.15.94:0/3419607282 conn(0x7f9ad806cdf0 msgr2=0x7f9ad806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:52.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.410+0000 7f9ae0800640 1 -- 172.21.15.94:0/3419607282 shutdown_connections 2024-09-10T23:36:52.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.410+0000 7f9ae0800640 1 -- 172.21.15.94:0/3419607282 wait complete. 2024-09-10T23:36:52.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9ae0800640 1 Processor -- start 2024-09-10T23:36:52.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9ae0800640 1 -- start start 2024-09-10T23:36:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9ae0800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad8073ff0 0x7f9ad815e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9ae0800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ad80d02e0 0x7f9ad814f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9ae0800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ad815ecb0 0x7f9ad80cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9ae0800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad81679c0 con 0x7f9ad8073ff0 2024-09-10T23:36:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9ae0800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad815f230 con 0x7f9ad815ecb0 2024-09-10T23:36:52.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9ae0800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad8167b40 con 0x7f9ad80d02e0 2024-09-10T23:36:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9adde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad8073ff0 0x7f9ad815e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9adde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad8073ff0 0x7f9ad815e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60536/0 (socket says 172.21.15.94:60536) 2024-09-10T23:36:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9adde00640 1 -- 172.21.15.94:0/197273869 learned_addr learned my addr 172.21.15.94:0/197273869 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9add400640 1 --2- 172.21.15.94:0/197273869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ad80d02e0 0x7f9ad814f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9ade800640 1 --2- 172.21.15.94:0/197273869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ad815ecb0 0x7f9ad80cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9adde00640 1 -- 172.21.15.94:0/197273869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ad80d02e0 msgr2=0x7f9ad814f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9adde00640 1 --2- 172.21.15.94:0/197273869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ad80d02e0 0x7f9ad814f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9adde00640 1 -- 172.21.15.94:0/197273869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ad815ecb0 msgr2=0x7f9ad80cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9adde00640 1 --2- 172.21.15.94:0/197273869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ad815ecb0 0x7f9ad80cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9adde00640 1 -- 172.21.15.94:0/197273869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad814f710 con 0x7f9ad8073ff0 2024-09-10T23:36:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9adde00640 1 --2- 172.21.15.94:0/197273869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad8073ff0 0x7f9ad815e710 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00e9f0 tx=0x7f9acc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9ad2000640 1 -- 172.21.15.94:0/197273869 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc00cd80 con 0x7f9ad8073ff0 2024-09-10T23:36:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9ad2000640 1 -- 172.21.15.94:0/197273869 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9acc010430 con 0x7f9ad8073ff0 2024-09-10T23:36:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9ad2000640 1 -- 172.21.15.94:0/197273869 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc010860 con 0x7f9ad8073ff0 2024-09-10T23:36:52.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9ae0800640 1 -- 172.21.15.94:0/197273869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad81f7ac0 con 0x7f9ad8073ff0 2024-09-10T23:36:52.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.414+0000 7f9ae0800640 1 -- 172.21.15.94:0/197273869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad81f8080 con 0x7f9ad8073ff0 2024-09-10T23:36:52.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.418+0000 7f9ae0800640 1 -- 172.21.15.94:0/197273869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad80abca0 con 0x7f9ad8073ff0 2024-09-10T23:36:52.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.418+0000 7f9ad2000640 1 -- 172.21.15.94:0/197273869 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f9acc010a00 con 0x7f9ad8073ff0 2024-09-10T23:36:52.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.418+0000 7f9ad2000640 1 --2- 172.21.15.94:0/197273869 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9abc072700 0x7f9abc074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:52.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.418+0000 7f9ad2000640 1 -- 172.21.15.94:0/197273869 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5709+0+0 (secure 0 0 0) 0x7f9acc093ff0 con 0x7f9ad8073ff0 2024-09-10T23:36:52.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.418+0000 7f9add400640 1 --2- 172.21.15.94:0/197273869 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9abc072700 0x7f9abc074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:52.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.418+0000 7f9add400640 1 --2- 172.21.15.94:0/197273869 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9abc072700 0x7f9abc074bc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80af000 tx=0x7f9ac801b0d0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:52.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.422+0000 7f9ad2000640 1 -- 172.21.15.94:0/197273869 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9acc05ef20 con 0x7f9ad8073ff0 2024-09-10T23:36:52.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.590+0000 7f9ae0800640 1 -- 172.21.15.94:0/197273869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ad80cfda0 con 0x7f9ad8073ff0 2024-09-10T23:36:52.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.590+0000 7f9ad2000640 1 -- 172.21.15.94:0/197273869 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16733 (secure 0 0 0) 0x7f9acc063950 con 0x7f9ad8073ff0 2024-09-10T23:36:52.592 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:52.592 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":80,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:36:51.627561+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:36:49.308477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T23:36:50.584653+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:36:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.594+0000 7f9ae0800640 1 -- 172.21.15.94:0/197273869 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9abc072700 msgr2=0x7f9abc074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.594+0000 7f9ae0800640 1 --2- 172.21.15.94:0/197273869 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9abc072700 0x7f9abc074bc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80af000 tx=0x7f9ac801b0d0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.594+0000 7f9ae0800640 1 -- 172.21.15.94:0/197273869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad8073ff0 msgr2=0x7f9ad815e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.594+0000 7f9ae0800640 1 --2- 172.21.15.94:0/197273869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad8073ff0 0x7f9ad815e710 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00e9f0 tx=0x7f9acc00eeb0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.594+0000 7f9ae0800640 1 -- 172.21.15.94:0/197273869 shutdown_connections 2024-09-10T23:36:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.594+0000 7f9ae0800640 1 --2- 172.21.15.94:0/197273869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ad815ecb0 0x7f9ad80cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.594+0000 7f9ae0800640 1 --2- 172.21.15.94:0/197273869 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9abc072700 0x7f9abc074bc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.594+0000 7f9ae0800640 1 --2- 172.21.15.94:0/197273869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ad80d02e0 0x7f9ad814f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.594+0000 7f9ae0800640 1 --2- 172.21.15.94:0/197273869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad8073ff0 0x7f9ad815e710 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.594+0000 7f9ae0800640 1 -- 172.21.15.94:0/197273869 >> 172.21.15.94:0/197273869 conn(0x7f9ad806cdf0 msgr2=0x7f9ad8164e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.594+0000 7f9ae0800640 1 -- 172.21.15.94:0/197273869 shutdown_connections 2024-09-10T23:36:52.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.594+0000 7f9ae0800640 1 -- 172.21.15.94:0/197273869 wait complete. 2024-09-10T23:36:52.614 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:36:52.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.754+0000 7f24c3e00640 1 -- 172.21.15.94:0/670810275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc0b35c0 msgr2=0x7f24bc06eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:52.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.754+0000 7f24c3e00640 1 --2- 172.21.15.94:0/670810275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc0b35c0 0x7f24bc06eef0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009bc0 tx=0x7f24b401cb10 comp rx=0 tx=0).stop 2024-09-10T23:36:52.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.754+0000 7f24c3e00640 1 -- 172.21.15.94:0/670810275 shutdown_connections 2024-09-10T23:36:52.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.754+0000 7f24c3e00640 1 --2- 172.21.15.94:0/670810275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc0b35c0 0x7f24bc06eef0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.754+0000 7f24c3e00640 1 --2- 172.21.15.94:0/670810275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24bc070150 0x7f24bc072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.754+0000 7f24c3e00640 1 -- 172.21.15.94:0/670810275 >> 172.21.15.94:0/670810275 conn(0x7f24bc06cdf0 msgr2=0x7f24bc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:52.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.754+0000 7f24c3e00640 1 -- 172.21.15.94:0/670810275 shutdown_connections 2024-09-10T23:36:52.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.754+0000 7f24c3e00640 1 -- 172.21.15.94:0/670810275 wait complete. 2024-09-10T23:36:52.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c3e00640 1 Processor -- start 2024-09-10T23:36:52.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c3e00640 1 -- start start 2024-09-10T23:36:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24bc070150 0x7f24bc14e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc0b35c0 0x7f24bc152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24bc157c60 0x7f24bc153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:52.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c3e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc167f90 con 0x7f24bc0b35c0 2024-09-10T23:36:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c3e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc15b630 con 0x7f24bc070150 2024-09-10T23:36:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c3e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f24bc168110 con 0x7f24bc157c60 2024-09-10T23:36:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc0b35c0 0x7f24bc152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc0b35c0 0x7f24bc152580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60554/0 (socket says 172.21.15.94:60554) 2024-09-10T23:36:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c0a00640 1 -- 172.21.15.94:0/895630873 learned_addr learned my addr 172.21.15.94:0/895630873 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c1e00640 1 --2- 172.21.15.94:0/895630873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24bc157c60 0x7f24bc153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c1400640 1 --2- 172.21.15.94:0/895630873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24bc070150 0x7f24bc14e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c0a00640 1 -- 172.21.15.94:0/895630873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24bc157c60 msgr2=0x7f24bc153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c0a00640 1 --2- 172.21.15.94:0/895630873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24bc157c60 0x7f24bc153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c0a00640 1 -- 172.21.15.94:0/895630873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24bc070150 msgr2=0x7f24bc14e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c0a00640 1 --2- 172.21.15.94:0/895630873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24bc070150 0x7f24bc14e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c0a00640 1 -- 172.21.15.94:0/895630873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24bc152f20 con 0x7f24bc0b35c0 2024-09-10T23:36:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c1400640 1 --2- 172.21.15.94:0/895630873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24bc070150 0x7f24bc14e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c0a00640 1 --2- 172.21.15.94:0/895630873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc0b35c0 0x7f24bc152580 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f24a800c3c0 tx=0x7f24a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24b9600640 1 -- 172.21.15.94:0/895630873 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8011790 con 0x7f24bc0b35c0 2024-09-10T23:36:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24b9600640 1 -- 172.21.15.94:0/895630873 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24a8017070 con 0x7f24bc0b35c0 2024-09-10T23:36:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24b9600640 1 -- 172.21.15.94:0/895630873 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8011ce0 con 0x7f24bc0b35c0 2024-09-10T23:36:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c3e00640 1 -- 172.21.15.94:0/895630873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24bc1584a0 con 0x7f24bc0b35c0 2024-09-10T23:36:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c3e00640 1 -- 172.21.15.94:0/895630873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24bc14e790 con 0x7f24bc0b35c0 2024-09-10T23:36:52.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.758+0000 7f24c3e00640 1 -- 172.21.15.94:0/895630873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24bc072610 con 0x7f24bc0b35c0 2024-09-10T23:36:52.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.762+0000 7f24b9600640 1 -- 172.21.15.94:0/895630873 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f24a801c460 con 0x7f24bc0b35c0 2024-09-10T23:36:52.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.762+0000 7f24b9600640 1 --2- 172.21.15.94:0/895630873 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f249c072700 0x7f249c074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:52.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.762+0000 7f24b9600640 1 -- 172.21.15.94:0/895630873 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5709+0+0 (secure 0 0 0) 0x7f24a8093150 con 0x7f24bc0b35c0 2024-09-10T23:36:52.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.762+0000 7f24c1400640 1 --2- 172.21.15.94:0/895630873 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f249c072700 0x7f249c074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:52.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.762+0000 7f24c1400640 1 --2- 172.21.15.94:0/895630873 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f249c072700 0x7f249c074bc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009c30 tx=0x7f24b00091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:52.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.766+0000 7f24b9600640 1 -- 172.21.15.94:0/895630873 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f24a805e600 con 0x7f24bc0b35c0 2024-09-10T23:36:52.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.934+0000 7f24c3e00640 1 -- 172.21.15.94:0/895630873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24bc06e7e0 con 0x7f24bc0b35c0 2024-09-10T23:36:52.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.934+0000 7f24b9600640 1 -- 172.21.15.94:0/895630873 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16733 (secure 0 0 0) 0x7f24a805df00 con 0x7f24bc0b35c0 2024-09-10T23:36:52.935 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:52.936 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":80,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:36:51.627561+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:36:49.308477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T23:36:50.584653+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:36:52.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.934+0000 7f24c3e00640 1 -- 172.21.15.94:0/895630873 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f249c072700 msgr2=0x7f249c074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:52.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.934+0000 7f24c3e00640 1 --2- 172.21.15.94:0/895630873 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f249c072700 0x7f249c074bc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009c30 tx=0x7f24b00091e0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.934+0000 7f24c3e00640 1 -- 172.21.15.94:0/895630873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc0b35c0 msgr2=0x7f24bc152580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:52.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.934+0000 7f24c3e00640 1 --2- 172.21.15.94:0/895630873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc0b35c0 0x7f24bc152580 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f24a800c3c0 tx=0x7f24a800c880 comp rx=0 tx=0).stop 2024-09-10T23:36:52.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.934+0000 7f24c3e00640 1 -- 172.21.15.94:0/895630873 shutdown_connections 2024-09-10T23:36:52.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.934+0000 7f24c3e00640 1 --2- 172.21.15.94:0/895630873 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f249c072700 0x7f249c074bc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.934+0000 7f24c3e00640 1 --2- 172.21.15.94:0/895630873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24bc157c60 0x7f24bc153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.934+0000 7f24c3e00640 1 --2- 172.21.15.94:0/895630873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24bc0b35c0 0x7f24bc152580 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.934+0000 7f24c3e00640 1 --2- 172.21.15.94:0/895630873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24bc070150 0x7f24bc14e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:52.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.934+0000 7f24c3e00640 1 -- 172.21.15.94:0/895630873 >> 172.21.15.94:0/895630873 conn(0x7f24bc06cdf0 msgr2=0x7f24bc0ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:52.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.934+0000 7f24c3e00640 1 -- 172.21.15.94:0/895630873 shutdown_connections 2024-09-10T23:36:52.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:52.938+0000 7f24c3e00640 1 -- 172.21.15.94:0/895630873 wait complete. 2024-09-10T23:36:52.957 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T23:36:53.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb077400640 1 -- 172.21.15.94:0/2443553263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0700cf3d0 msgr2=0x7fb0700cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:53.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb077400640 1 --2- 172.21.15.94:0/2443553263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0700cf3d0 0x7fb0700cf7e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb070072410 tx=0x7fb06801ccf0 comp rx=0 tx=0).stop 2024-09-10T23:36:53.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb077400640 1 -- 172.21.15.94:0/2443553263 shutdown_connections 2024-09-10T23:36:53.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb077400640 1 --2- 172.21.15.94:0/2443553263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0700cf3d0 0x7fb0700cf7e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:53.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb077400640 1 --2- 172.21.15.94:0/2443553263 >> v2:172.21.15.181:3300/0 conn(0x7fb07015def0 0x7fb070161be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:53.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb077400640 1 -- 172.21.15.94:0/2443553263 >> 172.21.15.94:0/2443553263 conn(0x7fb070074240 msgr2=0x7fb070074620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:53.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb077400640 1 -- 172.21.15.94:0/2443553263 shutdown_connections 2024-09-10T23:36:53.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb077400640 1 -- 172.21.15.94:0/2443553263 wait complete. 2024-09-10T23:36:53.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb077400640 1 Processor -- start 2024-09-10T23:36:53.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb077400640 1 -- start start 2024-09-10T23:36:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb077400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb070074d30 0x7fb070098aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb077400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0700cf3d0 0x7fb0700c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb077400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb07015def0 0x7fb0700992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb077400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb07016aef0 con 0x7fb070074d30 2024-09-10T23:36:53.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb077400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb07016ad70 con 0x7fb0700cf3d0 2024-09-10T23:36:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb077400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb07016b070 con 0x7fb07015def0 2024-09-10T23:36:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb074a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb070074d30 0x7fb070098aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb074a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb070074d30 0x7fb070098aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60574/0 (socket says 172.21.15.94:60574) 2024-09-10T23:36:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb074a00640 1 -- 172.21.15.94:0/432499677 learned_addr learned my addr 172.21.15.94:0/432499677 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb075400640 1 --2- 172.21.15.94:0/432499677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb07015def0 0x7fb0700992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb067e00640 1 --2- 172.21.15.94:0/432499677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0700cf3d0 0x7fb0700c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb074a00640 1 -- 172.21.15.94:0/432499677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb07015def0 msgr2=0x7fb0700992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb074a00640 1 --2- 172.21.15.94:0/432499677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb07015def0 0x7fb0700992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb074a00640 1 -- 172.21.15.94:0/432499677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0700cf3d0 msgr2=0x7fb0700c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb074a00640 1 --2- 172.21.15.94:0/432499677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0700cf3d0 0x7fb0700c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb074a00640 1 -- 172.21.15.94:0/432499677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0700b33e0 con 0x7fb070074d30 2024-09-10T23:36:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb075400640 1 --2- 172.21.15.94:0/432499677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb07015def0 0x7fb0700992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:36:53.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.102+0000 7fb074a00640 1 --2- 172.21.15.94:0/432499677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb070074d30 0x7fb070098aa0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb06000c9f0 tx=0x7fb06000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:53.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.106+0000 7fb06d600640 1 -- 172.21.15.94:0/432499677 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060007bb0 con 0x7fb070074d30 2024-09-10T23:36:53.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.106+0000 7fb06d600640 1 -- 172.21.15.94:0/432499677 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb060007e40 con 0x7fb070074d30 2024-09-10T23:36:53.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.106+0000 7fb06d600640 1 -- 172.21.15.94:0/432499677 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06000f690 con 0x7fb070074d30 2024-09-10T23:36:53.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.106+0000 7fb077400640 1 -- 172.21.15.94:0/432499677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0700c4840 con 0x7fb070074d30 2024-09-10T23:36:53.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.106+0000 7fb077400640 1 -- 172.21.15.94:0/432499677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07009a0f0 con 0x7fb070074d30 2024-09-10T23:36:53.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.106+0000 7fb077400640 1 -- 172.21.15.94:0/432499677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0700cf7e0 con 0x7fb070074d30 2024-09-10T23:36:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.106+0000 7fb06d600640 1 -- 172.21.15.94:0/432499677 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fb06000f830 con 0x7fb070074d30 2024-09-10T23:36:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.106+0000 7fb06d600640 1 --2- 172.21.15.94:0/432499677 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb04c072730 0x7fb04c074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.106+0000 7fb06d600640 1 -- 172.21.15.94:0/432499677 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5709+0+0 (secure 0 0 0) 0x7fb060093a90 con 0x7fb070074d30 2024-09-10T23:36:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.106+0000 7fb067e00640 1 --2- 172.21.15.94:0/432499677 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb04c072730 0x7fb04c074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:53.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.106+0000 7fb067e00640 1 --2- 172.21.15.94:0/432499677 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb04c072730 0x7fb04c074bf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb07006fb30 tx=0x7fb058007e40 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.110+0000 7fb06d600640 1 -- 172.21.15.94:0/432499677 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb0700b33e0 con 0x7fb070074d30 2024-09-10T23:36:53.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:53.314+0000 7fb077400640 1 -- 172.21.15.94:0/432499677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb070161730 con 0x7fb070074d30 2024-09-10T23:36:54.208 INFO:tasks.ceph.mds.b.smithi181.stderr:2024-09-10T23:36:54.206+0000 7f3e26c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:36:54.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.258+0000 7fb06d600640 1 -- 172.21.15.94:0/432499677 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) ==== 118+0+0 (secure 0 0 0) 0x7fb060014030 con 0x7fb070074d30 2024-09-10T23:36:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.258+0000 7fb077400640 1 -- 172.21.15.94:0/432499677 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb04c072730 msgr2=0x7fb04c074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.258+0000 7fb077400640 1 --2- 172.21.15.94:0/432499677 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb04c072730 0x7fb04c074bf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb07006fb30 tx=0x7fb058007e40 comp rx=0 tx=0).stop 2024-09-10T23:36:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.258+0000 7fb077400640 1 -- 172.21.15.94:0/432499677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb070074d30 msgr2=0x7fb070098aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.258+0000 7fb077400640 1 --2- 172.21.15.94:0/432499677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb070074d30 0x7fb070098aa0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb06000c9f0 tx=0x7fb06000ceb0 comp rx=0 tx=0).stop 2024-09-10T23:36:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.258+0000 7fb077400640 1 -- 172.21.15.94:0/432499677 shutdown_connections 2024-09-10T23:36:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.258+0000 7fb077400640 1 --2- 172.21.15.94:0/432499677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0700cf3d0 0x7fb0700c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.258+0000 7fb077400640 1 --2- 172.21.15.94:0/432499677 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb04c072730 0x7fb04c074bf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.258+0000 7fb077400640 1 --2- 172.21.15.94:0/432499677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb07015def0 0x7fb0700992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.258+0000 7fb077400640 1 --2- 172.21.15.94:0/432499677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb070074d30 0x7fb070098aa0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:54.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.258+0000 7fb077400640 1 -- 172.21.15.94:0/432499677 >> 172.21.15.94:0/432499677 conn(0x7fb070074240 msgr2=0x7fb07009dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:54.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.262+0000 7fb077400640 1 -- 172.21.15.94:0/432499677 shutdown_connections 2024-09-10T23:36:54.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.262+0000 7fb077400640 1 -- 172.21.15.94:0/432499677 wait complete. 2024-09-10T23:36:54.283 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T23:36:54.283 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T23:36:54.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.426+0000 7f9cf6e00640 1 -- 172.21.15.94:0/423346112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd4005220 msgr2=0x7f9cd400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:54.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.426+0000 7f9cf6e00640 1 --2- 172.21.15.94:0/423346112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd4005220 0x7f9cd400b9d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800c730 tx=0x7f9ce801f980 comp rx=0 tx=0).stop 2024-09-10T23:36:54.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.426+0000 7f9cf6e00640 1 -- 172.21.15.94:0/423346112 shutdown_connections 2024-09-10T23:36:54.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.426+0000 7f9cf6e00640 1 --2- 172.21.15.94:0/423346112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd4005220 0x7f9cd400b9d0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:54.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.426+0000 7f9cf6e00640 1 -- 172.21.15.94:0/423346112 >> 172.21.15.94:0/423346112 conn(0x7f9cf00d0d80 msgr2=0x7f9cf00d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:54.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.426+0000 7f9cf6e00640 1 -- 172.21.15.94:0/423346112 shutdown_connections 2024-09-10T23:36:54.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.426+0000 7f9cf6e00640 1 -- 172.21.15.94:0/423346112 wait complete. 2024-09-10T23:36:54.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cf6e00640 1 Processor -- start 2024-09-10T23:36:54.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cf6e00640 1 -- start start 2024-09-10T23:36:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cf6e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd4005220 0x7f9cf015b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cf6e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf0146c80 0x7f9cf0148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cf6e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf0148670 0x7f9cf00cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cf6e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf015f9a0 con 0x7f9cd4005220 2024-09-10T23:36:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cf6e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf015f820 con 0x7f9cf0146c80 2024-09-10T23:36:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cf6e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf015fb20 con 0x7f9cf0148670 2024-09-10T23:36:54.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cefe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd4005220 0x7f9cf015b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cefe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd4005220 0x7f9cf015b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47568/0 (socket says 172.21.15.94:47568) 2024-09-10T23:36:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cefe00640 1 -- 172.21.15.94:0/2294666532 learned_addr learned my addr 172.21.15.94:0/2294666532 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cf4e00640 1 --2- 172.21.15.94:0/2294666532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf0148670 0x7f9cf00cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9ce7400640 1 --2- 172.21.15.94:0/2294666532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf0146c80 0x7f9cf0148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cefe00640 1 -- 172.21.15.94:0/2294666532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf0148670 msgr2=0x7f9cf00cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cefe00640 1 --2- 172.21.15.94:0/2294666532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf0148670 0x7f9cf00cc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cefe00640 1 -- 172.21.15.94:0/2294666532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf0146c80 msgr2=0x7f9cf0148130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cefe00640 1 --2- 172.21.15.94:0/2294666532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf0146c80 0x7f9cf0148130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cefe00640 1 -- 172.21.15.94:0/2294666532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf0148b00 con 0x7f9cd4005220 2024-09-10T23:36:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cf4e00640 1 --2- 172.21.15.94:0/2294666532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf0148670 0x7f9cf00cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9ce7400640 1 --2- 172.21.15.94:0/2294666532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf0146c80 0x7f9cf0148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:54.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cefe00640 1 --2- 172.21.15.94:0/2294666532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd4005220 0x7f9cf015b070 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0004bf0 tx=0x7f9ce0001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:54.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cecc00640 1 -- 172.21.15.94:0/2294666532 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0015020 con 0x7f9cd4005220 2024-09-10T23:36:54.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cecc00640 1 -- 172.21.15.94:0/2294666532 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ce00119a0 con 0x7f9cd4005220 2024-09-10T23:36:54.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cecc00640 1 -- 172.21.15.94:0/2294666532 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0004e60 con 0x7f9cd4005220 2024-09-10T23:36:54.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cf6e00640 1 -- 172.21.15.94:0/2294666532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf01f4690 con 0x7f9cd4005220 2024-09-10T23:36:54.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.430+0000 7f9cf6e00640 1 -- 172.21.15.94:0/2294666532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf01f4f30 con 0x7f9cd4005220 2024-09-10T23:36:54.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.434+0000 7f9cf6e00640 1 -- 172.21.15.94:0/2294666532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf0073a90 con 0x7f9cd4005220 2024-09-10T23:36:54.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.434+0000 7f9cecc00640 1 -- 172.21.15.94:0/2294666532 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f9ce001a400 con 0x7f9cd4005220 2024-09-10T23:36:54.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.434+0000 7f9cecc00640 1 --2- 172.21.15.94:0/2294666532 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9ccc072700 0x7f9ccc074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:54.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.434+0000 7f9cecc00640 1 -- 172.21.15.94:0/2294666532 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f9ce0094240 con 0x7f9cd4005220 2024-09-10T23:36:54.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.434+0000 7f9ce7400640 1 --2- 172.21.15.94:0/2294666532 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9ccc072700 0x7f9ccc074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:54.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.434+0000 7f9ce7400640 1 --2- 172.21.15.94:0/2294666532 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9ccc072700 0x7f9ccc074bc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0146b50 tx=0x7f9cd80091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:54.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.438+0000 7f9cecc00640 1 -- 172.21.15.94:0/2294666532 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9ce001f020 con 0x7f9cd4005220 2024-09-10T23:36:54.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:54.638+0000 7f9cf6e00640 1 -- 172.21.15.94:0/2294666532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9cf0073700 con 0x7f9cd4005220 2024-09-10T23:36:55.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.262+0000 7f9cecc00640 1 -- 172.21.15.94:0/2294666532 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) ==== 122+0+0 (secure 0 0 0) 0x7f9ce005f660 con 0x7f9cd4005220 2024-09-10T23:36:55.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.266+0000 7f9cf6e00640 1 -- 172.21.15.94:0/2294666532 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9ccc072700 msgr2=0x7f9ccc074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:55.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.266+0000 7f9cf6e00640 1 --2- 172.21.15.94:0/2294666532 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9ccc072700 0x7f9ccc074bc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0146b50 tx=0x7f9cd80091e0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.266+0000 7f9cf6e00640 1 -- 172.21.15.94:0/2294666532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd4005220 msgr2=0x7f9cf015b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:55.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.266+0000 7f9cf6e00640 1 --2- 172.21.15.94:0/2294666532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd4005220 0x7f9cf015b070 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0004bf0 tx=0x7f9ce0001810 comp rx=0 tx=0).stop 2024-09-10T23:36:55.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.266+0000 7f9cf6e00640 1 -- 172.21.15.94:0/2294666532 shutdown_connections 2024-09-10T23:36:55.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.266+0000 7f9cf6e00640 1 --2- 172.21.15.94:0/2294666532 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf0146c80 0x7f9cf0148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.266+0000 7f9cf6e00640 1 --2- 172.21.15.94:0/2294666532 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9ccc072700 0x7f9ccc074bc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.266+0000 7f9cf6e00640 1 --2- 172.21.15.94:0/2294666532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf0148670 0x7f9cf00cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.266+0000 7f9cf6e00640 1 --2- 172.21.15.94:0/2294666532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd4005220 0x7f9cf015b070 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.266+0000 7f9cf6e00640 1 -- 172.21.15.94:0/2294666532 >> 172.21.15.94:0/2294666532 conn(0x7f9cf00d0d80 msgr2=0x7f9cf006e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:55.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.266+0000 7f9cf6e00640 1 -- 172.21.15.94:0/2294666532 shutdown_connections 2024-09-10T23:36:55.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.266+0000 7f9cf6e00640 1 -- 172.21.15.94:0/2294666532 wait complete. 2024-09-10T23:36:55.288 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T23:36:55.288 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:55.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.430+0000 7f2447800640 1 -- 172.21.15.94:0/653012020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f244006f600 msgr2=0x7f24400b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:55.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.430+0000 7f2447800640 1 --2- 172.21.15.94:0/653012020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f244006f600 0x7f24400b35c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f243800ad90 tx=0x7f243801dc20 comp rx=0 tx=0).stop 2024-09-10T23:36:55.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.430+0000 7f2447800640 1 -- 172.21.15.94:0/653012020 shutdown_connections 2024-09-10T23:36:55.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.430+0000 7f2447800640 1 --2- 172.21.15.94:0/653012020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f244006f600 0x7f24400b35c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.430+0000 7f2447800640 1 --2- 172.21.15.94:0/653012020 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2440157c60 0x7f2440072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.430+0000 7f2447800640 1 -- 172.21.15.94:0/653012020 >> 172.21.15.94:0/653012020 conn(0x7f244006cdf0 msgr2=0x7f244006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:55.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.430+0000 7f2447800640 1 -- 172.21.15.94:0/653012020 shutdown_connections 2024-09-10T23:36:55.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.430+0000 7f2447800640 1 -- 172.21.15.94:0/653012020 wait complete. 2024-09-10T23:36:55.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.430+0000 7f2447800640 1 Processor -- start 2024-09-10T23:36:55.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.430+0000 7f2447800640 1 -- start start 2024-09-10T23:36:55.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2447800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f244006f600 0x7f244015bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:55.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2447800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440070150 0x7f24401636a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:55.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2447800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2440157c60 0x7f24401f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:55.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2447800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f244015b3b0 con 0x7f2440070150 2024-09-10T23:36:55.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2447800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f244015b230 con 0x7f244006f600 2024-09-10T23:36:55.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2447800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f244015b530 con 0x7f2440157c60 2024-09-10T23:36:55.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2437e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440070150 0x7f24401636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:55.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2444e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f244006f600 0x7f244015bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:55.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2437e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440070150 0x7f24401636a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47578/0 (socket says 172.21.15.94:47578) 2024-09-10T23:36:55.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2437e00640 1 -- 172.21.15.94:0/3360098573 learned_addr learned my addr 172.21.15.94:0/3360098573 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:55.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2445800640 1 --2- 172.21.15.94:0/3360098573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2440157c60 0x7f24401f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2437e00640 1 -- 172.21.15.94:0/3360098573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2440157c60 msgr2=0x7f24401f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2437e00640 1 --2- 172.21.15.94:0/3360098573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2440157c60 0x7f24401f57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2437e00640 1 -- 172.21.15.94:0/3360098573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f244006f600 msgr2=0x7f244015bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2437e00640 1 --2- 172.21.15.94:0/3360098573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f244006f600 0x7f244015bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2437e00640 1 -- 172.21.15.94:0/3360098573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24401f90f0 con 0x7f2440070150 2024-09-10T23:36:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2444e00640 1 --2- 172.21.15.94:0/3360098573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f244006f600 0x7f244015bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:36:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2437e00640 1 --2- 172.21.15.94:0/3360098573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440070150 0x7f24401636a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f242800c480 tx=0x7f242800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f243d600640 1 -- 172.21.15.94:0/3360098573 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24280052d0 con 0x7f2440070150 2024-09-10T23:36:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f243d600640 1 -- 172.21.15.94:0/3360098573 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2428005470 con 0x7f2440070150 2024-09-10T23:36:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2447800640 1 -- 172.21.15.94:0/3360098573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2440163be0 con 0x7f2440070150 2024-09-10T23:36:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f243d600640 1 -- 172.21.15.94:0/3360098573 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428012870 con 0x7f2440070150 2024-09-10T23:36:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2447800640 1 -- 172.21.15.94:0/3360098573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24401f6700 con 0x7f2440070150 2024-09-10T23:36:55.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.434+0000 7f2447800640 1 -- 172.21.15.94:0/3360098573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24400b35c0 con 0x7f2440070150 2024-09-10T23:36:55.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.438+0000 7f243d600640 1 -- 172.21.15.94:0/3360098573 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f2428003600 con 0x7f2440070150 2024-09-10T23:36:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.438+0000 7f243d600640 1 --2- 172.21.15.94:0/3360098573 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f241c072700 0x7f241c074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.438+0000 7f243d600640 1 -- 172.21.15.94:0/3360098573 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f2428092ff0 con 0x7f2440070150 2024-09-10T23:36:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.438+0000 7f2444e00640 1 --2- 172.21.15.94:0/3360098573 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f241c072700 0x7f241c074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.438+0000 7f2444e00640 1 --2- 172.21.15.94:0/3360098573 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f241c072700 0x7f241c074bc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2430009600 tx=0x7f24300091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.442+0000 7f243d600640 1 -- 172.21.15.94:0/3360098573 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f24401f90f0 con 0x7f2440070150 2024-09-10T23:36:55.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.642+0000 7f2447800640 1 -- 172.21.15.94:0/3360098573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2440074070 con 0x7f2440070150 2024-09-10T23:36:55.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.646+0000 7f243d600640 1 -- 172.21.15.94:0/3360098573 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5240 (secure 0 0 0) 0x7f242805de10 con 0x7f2440070150 2024-09-10T23:36:55.646 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:55.646 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":53,"btime":"2024-09-10T23:36:55:260109+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1215179751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1215179751},{"type":"v1","addr":"172.21.15.94:6834","nonce":1215179751}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6672,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3193009978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":3193009978},{"type":"v1","addr":"172.21.15.181:6835","nonce":3193009978}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":195698,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/736952701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":736952701},{"type":"v1","addr":"172.21.15.94:6835","nonce":736952701}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:36:51.627529+0000","modified":"2024-09-10T23:36:54.641240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195701":{"gid":195701,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6834/1214870454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1214870454},{"type":"v1","addr":"172.21.15.181:6834","nonce":1214870454}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195701,"qdb_cluster":[195701]},"id":6}]} 2024-09-10T23:36:55.646 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 53 2024-09-10T23:36:55.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.646+0000 7f2447800640 1 -- 172.21.15.94:0/3360098573 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f241c072700 msgr2=0x7f241c074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:55.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.646+0000 7f2447800640 1 --2- 172.21.15.94:0/3360098573 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f241c072700 0x7f241c074bc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2430009600 tx=0x7f24300091e0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.646+0000 7f2447800640 1 -- 172.21.15.94:0/3360098573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440070150 msgr2=0x7f24401636a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:55.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.646+0000 7f2447800640 1 --2- 172.21.15.94:0/3360098573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440070150 0x7f24401636a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f242800c480 tx=0x7f242800c940 comp rx=0 tx=0).stop 2024-09-10T23:36:55.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.646+0000 7f2447800640 1 -- 172.21.15.94:0/3360098573 shutdown_connections 2024-09-10T23:36:55.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.646+0000 7f2447800640 1 --2- 172.21.15.94:0/3360098573 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f241c072700 0x7f241c074bc0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.646+0000 7f2447800640 1 --2- 172.21.15.94:0/3360098573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2440157c60 0x7f24401f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.646+0000 7f2447800640 1 --2- 172.21.15.94:0/3360098573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440070150 0x7f24401636a0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.646+0000 7f2447800640 1 --2- 172.21.15.94:0/3360098573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f244006f600 0x7f244015bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.646+0000 7f2447800640 1 -- 172.21.15.94:0/3360098573 >> 172.21.15.94:0/3360098573 conn(0x7f244006cdf0 msgr2=0x7f2440156dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:55.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.646+0000 7f2447800640 1 -- 172.21.15.94:0/3360098573 shutdown_connections 2024-09-10T23:36:55.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.646+0000 7f2447800640 1 -- 172.21.15.94:0/3360098573 wait complete. 2024-09-10T23:36:55.668 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:36:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.806+0000 7f5957a00640 1 -- 172.21.15.94:0/784103078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5950157c60 msgr2=0x7f5950072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.806+0000 7f5957a00640 1 --2- 172.21.15.94:0/784103078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5950157c60 0x7f5950072610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f594000a980 tx=0x7f594001da20 comp rx=0 tx=0).stop 2024-09-10T23:36:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.806+0000 7f5957a00640 1 -- 172.21.15.94:0/784103078 shutdown_connections 2024-09-10T23:36:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.806+0000 7f5957a00640 1 --2- 172.21.15.94:0/784103078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5950157c60 0x7f5950072610 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.806+0000 7f5957a00640 1 -- 172.21.15.94:0/784103078 >> 172.21.15.94:0/784103078 conn(0x7f595006cdf0 msgr2=0x7f595006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.806+0000 7f5957a00640 1 -- 172.21.15.94:0/784103078 shutdown_connections 2024-09-10T23:36:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5957a00640 1 -- 172.21.15.94:0/784103078 wait complete. 2024-09-10T23:36:55.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5957a00640 1 Processor -- start 2024-09-10T23:36:55.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5957a00640 1 -- start start 2024-09-10T23:36:55.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5957a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595006f600 0x7f59500cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:55.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5957a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5950070150 0x7f5950159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:55.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5957a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5950157c60 0x7f5950151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:55.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5957a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5950168270 con 0x7f595006f600 2024-09-10T23:36:55.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5957a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f59501680f0 con 0x7f5950070150 2024-09-10T23:36:55.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5957a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f59501683f0 con 0x7f5950157c60 2024-09-10T23:36:55.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5955000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595006f600 0x7f59500cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5955000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595006f600 0x7f59500cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47590/0 (socket says 172.21.15.94:47590) 2024-09-10T23:36:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5955000640 1 -- 172.21.15.94:0/757364528 learned_addr learned my addr 172.21.15.94:0/757364528 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5947e00640 1 --2- 172.21.15.94:0/757364528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5950070150 0x7f5950159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5955a00640 1 --2- 172.21.15.94:0/757364528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5950157c60 0x7f5950151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5955000640 1 -- 172.21.15.94:0/757364528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5950157c60 msgr2=0x7f5950151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5955000640 1 --2- 172.21.15.94:0/757364528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5950157c60 0x7f5950151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5955000640 1 -- 172.21.15.94:0/757364528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5950070150 msgr2=0x7f5950159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5955000640 1 --2- 172.21.15.94:0/757364528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5950070150 0x7f5950159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5955000640 1 -- 172.21.15.94:0/757364528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59500cb890 con 0x7f595006f600 2024-09-10T23:36:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.810+0000 7f5955000640 1 --2- 172.21.15.94:0/757364528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595006f600 0x7f59500cb350 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5940000c10 tx=0x7f5940004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:55.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.814+0000 7f594d600640 1 -- 172.21.15.94:0/757364528 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59400b3040 con 0x7f595006f600 2024-09-10T23:36:55.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.814+0000 7f594d600640 1 -- 172.21.15.94:0/757364528 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59400a7c00 con 0x7f595006f600 2024-09-10T23:36:55.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.814+0000 7f594d600640 1 -- 172.21.15.94:0/757364528 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5940003600 con 0x7f595006f600 2024-09-10T23:36:55.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.814+0000 7f5957a00640 1 -- 172.21.15.94:0/757364528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59501fba80 con 0x7f595006f600 2024-09-10T23:36:55.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.814+0000 7f5957a00640 1 -- 172.21.15.94:0/757364528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59501fbd20 con 0x7f595006f600 2024-09-10T23:36:55.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.814+0000 7f5957a00640 1 -- 172.21.15.94:0/757364528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5950072610 con 0x7f595006f600 2024-09-10T23:36:55.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.814+0000 7f594d600640 1 -- 172.21.15.94:0/757364528 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f594000b040 con 0x7f595006f600 2024-09-10T23:36:55.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.814+0000 7f594d600640 1 --2- 172.21.15.94:0/757364528 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f592c072730 0x7f592c074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:55.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.814+0000 7f594d600640 1 -- 172.21.15.94:0/757364528 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f594012d3a0 con 0x7f595006f600 2024-09-10T23:36:55.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.818+0000 7f5947e00640 1 --2- 172.21.15.94:0/757364528 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f592c072730 0x7f592c074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:55.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.818+0000 7f5947e00640 1 --2- 172.21.15.94:0/757364528 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f592c072730 0x7f592c074bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f59500ced50 tx=0x7f5938009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:55.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.818+0000 7f594d600640 1 -- 172.21.15.94:0/757364528 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f59400176a0 con 0x7f595006f600 2024-09-10T23:36:55.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.986+0000 7f5957a00640 1 -- 172.21.15.94:0/757364528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f595006e7e0 con 0x7f595006f600 2024-09-10T23:36:55.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.986+0000 7f594d600640 1 -- 172.21.15.94:0/757364528 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16720 (secure 0 0 0) 0x7f59400f82a0 con 0x7f595006f600 2024-09-10T23:36:55.987 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:55.987 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:36:53.228418+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:36:49.308477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T23:36:50.584653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:36:55.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.986+0000 7f5957a00640 1 -- 172.21.15.94:0/757364528 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f592c072730 msgr2=0x7f592c074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:55.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.986+0000 7f5957a00640 1 --2- 172.21.15.94:0/757364528 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f592c072730 0x7f592c074bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f59500ced50 tx=0x7f5938009040 comp rx=0 tx=0).stop 2024-09-10T23:36:55.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.986+0000 7f5957a00640 1 -- 172.21.15.94:0/757364528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595006f600 msgr2=0x7f59500cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:55.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.986+0000 7f5957a00640 1 --2- 172.21.15.94:0/757364528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595006f600 0x7f59500cb350 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5940000c10 tx=0x7f5940004620 comp rx=0 tx=0).stop 2024-09-10T23:36:55.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.986+0000 7f5957a00640 1 -- 172.21.15.94:0/757364528 shutdown_connections 2024-09-10T23:36:55.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.986+0000 7f5957a00640 1 --2- 172.21.15.94:0/757364528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5950070150 0x7f5950159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.986+0000 7f5957a00640 1 --2- 172.21.15.94:0/757364528 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f592c072730 0x7f592c074bf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.986+0000 7f5957a00640 1 --2- 172.21.15.94:0/757364528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5950157c60 0x7f5950151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.986+0000 7f5957a00640 1 --2- 172.21.15.94:0/757364528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f595006f600 0x7f59500cb350 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:55.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.986+0000 7f5957a00640 1 -- 172.21.15.94:0/757364528 >> 172.21.15.94:0/757364528 conn(0x7f595006cdf0 msgr2=0x7f59501585e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:55.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.986+0000 7f5957a00640 1 -- 172.21.15.94:0/757364528 shutdown_connections 2024-09-10T23:36:55.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:55.986+0000 7f5957a00640 1 -- 172.21.15.94:0/757364528 wait complete. 2024-09-10T23:36:56.010 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T23:36:56.010 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:36:56.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.150+0000 7fc24e600640 1 -- 172.21.15.94:0/2570105701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2480ae980 msgr2=0x7fc2480d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:56.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.150+0000 7fc24e600640 1 --2- 172.21.15.94:0/2570105701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2480ae980 0x7fc2480d1b80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc234009840 tx=0x7fc23401c820 comp rx=0 tx=0).stop 2024-09-10T23:36:56.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.150+0000 7fc24e600640 1 -- 172.21.15.94:0/2570105701 shutdown_connections 2024-09-10T23:36:56.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.150+0000 7fc24e600640 1 --2- 172.21.15.94:0/2570105701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2480ae980 0x7fc2480d1b80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.150+0000 7fc24e600640 1 --2- 172.21.15.94:0/2570105701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc24815bd30 0x7fc24815c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.150+0000 7fc24e600640 1 -- 172.21.15.94:0/2570105701 >> 172.21.15.94:0/2570105701 conn(0x7fc24809aa70 msgr2=0x7fc24809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:56.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.150+0000 7fc24e600640 1 -- 172.21.15.94:0/2570105701 shutdown_connections 2024-09-10T23:36:56.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.150+0000 7fc24e600640 1 -- 172.21.15.94:0/2570105701 wait complete. 2024-09-10T23:36:56.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.150+0000 7fc24e600640 1 Processor -- start 2024-09-10T23:36:56.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.150+0000 7fc24e600640 1 -- start start 2024-09-10T23:36:56.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc24e600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2480ae980 0x7fc248152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:56.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc24e600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2480b2ec0 0x7fc2480cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:56.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc24e600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc24815bd30 0x7fc248158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:56.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc24e600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc248169340 con 0x7fc2480ae980 2024-09-10T23:36:56.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc24e600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc2481691c0 con 0x7fc24815bd30 2024-09-10T23:36:56.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc24e600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc2481694c0 con 0x7fc2480b2ec0 2024-09-10T23:36:56.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc247400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2480ae980 0x7fc248152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:56.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc247400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2480ae980 0x7fc248152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47594/0 (socket says 172.21.15.94:47594) 2024-09-10T23:36:56.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc247400640 1 -- 172.21.15.94:0/3113133900 learned_addr learned my addr 172.21.15.94:0/3113133900 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:56.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc246a00640 1 --2- 172.21.15.94:0/3113133900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2480b2ec0 0x7fc2480cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:56.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc247400640 1 -- 172.21.15.94:0/3113133900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2480b2ec0 msgr2=0x7fc2480cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:56.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc247400640 1 --2- 172.21.15.94:0/3113133900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2480b2ec0 0x7fc2480cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc247400640 1 -- 172.21.15.94:0/3113133900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc24815bd30 msgr2=0x7fc248158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:56.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc247400640 1 --2- 172.21.15.94:0/3113133900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc24815bd30 0x7fc248158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc247400640 1 -- 172.21.15.94:0/3113133900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2480ca6b0 con 0x7fc2480ae980 2024-09-10T23:36:56.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc247400640 1 --2- 172.21.15.94:0/3113133900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2480ae980 0x7fc248152ed0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fc23800e7a0 tx=0x7fc23800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:56.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc23f400640 1 -- 172.21.15.94:0/3113133900 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23800ccc0 con 0x7fc2480ae980 2024-09-10T23:36:56.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc23f400640 1 -- 172.21.15.94:0/3113133900 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc238010430 con 0x7fc2480ae980 2024-09-10T23:36:56.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc23f400640 1 -- 172.21.15.94:0/3113133900 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238010880 con 0x7fc2480ae980 2024-09-10T23:36:56.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc24e600640 1 -- 172.21.15.94:0/3113133900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2480ca990 con 0x7fc2480ae980 2024-09-10T23:36:56.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc24e600640 1 -- 172.21.15.94:0/3113133900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2480caf50 con 0x7fc2480ae980 2024-09-10T23:36:56.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc24e600640 1 -- 172.21.15.94:0/3113133900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2480d1b80 con 0x7fc2480ae980 2024-09-10T23:36:56.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.154+0000 7fc23f400640 1 -- 172.21.15.94:0/3113133900 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fc238010a20 con 0x7fc2480ae980 2024-09-10T23:36:56.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.158+0000 7fc23f400640 1 --2- 172.21.15.94:0/3113133900 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc228072700 0x7fc228074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:56.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.158+0000 7fc23f400640 1 -- 172.21.15.94:0/3113133900 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fc238093ff0 con 0x7fc2480ae980 2024-09-10T23:36:56.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.158+0000 7fc246a00640 1 --2- 172.21.15.94:0/3113133900 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc228072700 0x7fc228074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:56.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.158+0000 7fc246a00640 1 --2- 172.21.15.94:0/3113133900 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc228072700 0x7fc228074bc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc2340af000 tx=0x7fc23401b060 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:56.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.162+0000 7fc23f400640 1 -- 172.21.15.94:0/3113133900 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc23805f500 con 0x7fc2480ae980 2024-09-10T23:36:56.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.326+0000 7fc24e600640 1 -- 172.21.15.94:0/3113133900 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc248153410 con 0x7fc228072700 2024-09-10T23:36:56.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.330+0000 7fc23f400640 1 -- 172.21.15.94:0/3113133900 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252059 (secure 0 0 0) 0x7fc248153410 con 0x7fc228072700 2024-09-10T23:36:56.334 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:56.336 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:36:56.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.334+0000 7fc24e600640 1 -- 172.21.15.94:0/3113133900 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc228072700 msgr2=0x7fc228074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:56.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.334+0000 7fc24e600640 1 --2- 172.21.15.94:0/3113133900 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc228072700 0x7fc228074bc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc2340af000 tx=0x7fc23401b060 comp rx=0 tx=0).stop 2024-09-10T23:36:56.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.334+0000 7fc24e600640 1 -- 172.21.15.94:0/3113133900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2480ae980 msgr2=0x7fc248152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:56.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.334+0000 7fc24e600640 1 --2- 172.21.15.94:0/3113133900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2480ae980 0x7fc248152ed0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fc23800e7a0 tx=0x7fc23800ec60 comp rx=0 tx=0).stop 2024-09-10T23:36:56.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.334+0000 7fc24e600640 1 -- 172.21.15.94:0/3113133900 shutdown_connections 2024-09-10T23:36:56.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.334+0000 7fc24e600640 1 --2- 172.21.15.94:0/3113133900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc24815bd30 0x7fc248158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.334+0000 7fc24e600640 1 --2- 172.21.15.94:0/3113133900 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc228072700 0x7fc228074bc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.334+0000 7fc24e600640 1 --2- 172.21.15.94:0/3113133900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2480b2ec0 0x7fc2480cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.334+0000 7fc24e600640 1 --2- 172.21.15.94:0/3113133900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2480ae980 0x7fc248152ed0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.334+0000 7fc24e600640 1 -- 172.21.15.94:0/3113133900 >> 172.21.15.94:0/3113133900 conn(0x7fc24809aa70 msgr2=0x7fc24815e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:56.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.334+0000 7fc24e600640 1 -- 172.21.15.94:0/3113133900 shutdown_connections 2024-09-10T23:36:56.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.334+0000 7fc24e600640 1 -- 172.21.15.94:0/3113133900 wait complete. 2024-09-10T23:36:56.355 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-10T23:36:54.738146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258256,"kb_used_data":8728,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716464,"statfs":{"total":773094113280,"available":772829659136,"internally_reserved":0,"allocated":8937472,"data_stored":4457639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.521037"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.248385+0000","last_change":"2024-09-10T23:36:52.243739+0000","last_active":"2024-09-10T23:36:52.248385+0000","last_peered":"2024-09-10T23:36:52.248385+0000","last_clean":"2024-09-10T23:36:52.248385+0000","last_became_active":"2024-09-10T23:36:52.243464+0000","last_became_peered":"2024-09-10T23:36:52.243464+0000","last_unstale":"2024-09-10T23:36:52.248385+0000","last_undegraded":"2024-09-10T23:36:52.248385+0000","last_fullsized":"2024-09-10T23:36:52.248385+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:34.455988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.250949+0000","last_change":"2024-09-10T23:36:52.244795+0000","last_active":"2024-09-10T23:36:52.250949+0000","last_peered":"2024-09-10T23:36:52.250949+0000","last_clean":"2024-09-10T23:36:52.250949+0000","last_became_active":"2024-09-10T23:36:52.244384+0000","last_became_peered":"2024-09-10T23:36:52.244384+0000","last_unstale":"2024-09-10T23:36:52.250949+0000","last_undegraded":"2024-09-10T23:36:52.250949+0000","last_fullsized":"2024-09-10T23:36:52.250949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:41.704048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.266773+0000","last_change":"2024-09-10T23:36:52.263412+0000","last_active":"2024-09-10T23:36:52.266773+0000","last_peered":"2024-09-10T23:36:52.266773+0000","last_clean":"2024-09-10T23:36:52.266773+0000","last_became_active":"2024-09-10T23:36:52.263117+0000","last_became_peered":"2024-09-10T23:36:52.263117+0000","last_unstale":"2024-09-10T23:36:52.266773+0000","last_undegraded":"2024-09-10T23:36:52.266773+0000","last_fullsized":"2024-09-10T23:36:52.266773+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:53:21.829334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.265821+0000","last_change":"2024-09-10T23:36:52.262496+0000","last_active":"2024-09-10T23:36:52.265821+0000","last_peered":"2024-09-10T23:36:52.265821+0000","last_clean":"2024-09-10T23:36:52.265821+0000","last_became_active":"2024-09-10T23:36:52.261927+0000","last_became_peered":"2024-09-10T23:36:52.261927+0000","last_unstale":"2024-09-10T23:36:52.265821+0000","last_undegraded":"2024-09-10T23:36:52.265821+0000","last_fullsized":"2024-09-10T23:36:52.265821+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:52.152410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.269437+0000","last_change":"2024-09-10T23:36:52.265738+0000","last_active":"2024-09-10T23:36:52.269437+0000","last_peered":"2024-09-10T23:36:52.269437+0000","last_clean":"2024-09-10T23:36:52.269437+0000","last_became_active":"2024-09-10T23:36:52.265489+0000","last_became_peered":"2024-09-10T23:36:52.265489+0000","last_unstale":"2024-09-10T23:36:52.269437+0000","last_undegraded":"2024-09-10T23:36:52.269437+0000","last_fullsized":"2024-09-10T23:36:52.269437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:49.404949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.262507+0000","last_change":"2024-09-10T23:36:52.251410+0000","last_active":"2024-09-10T23:36:52.262507+0000","last_peered":"2024-09-10T23:36:52.262507+0000","last_clean":"2024-09-10T23:36:52.262507+0000","last_became_active":"2024-09-10T23:36:52.251157+0000","last_became_peered":"2024-09-10T23:36:52.251157+0000","last_unstale":"2024-09-10T23:36:52.262507+0000","last_undegraded":"2024-09-10T23:36:52.262507+0000","last_fullsized":"2024-09-10T23:36:52.262507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:52.621073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.266487+0000","last_change":"2024-09-10T23:36:52.257597+0000","last_active":"2024-09-10T23:36:52.266487+0000","last_peered":"2024-09-10T23:36:52.266487+0000","last_clean":"2024-09-10T23:36:52.266487+0000","last_became_active":"2024-09-10T23:36:52.257330+0000","last_became_peered":"2024-09-10T23:36:52.257330+0000","last_unstale":"2024-09-10T23:36:52.266487+0000","last_undegraded":"2024-09-10T23:36:52.266487+0000","last_fullsized":"2024-09-10T23:36:52.266487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:13.263641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.250846+0000","last_change":"2024-09-10T23:36:52.248454+0000","last_active":"2024-09-10T23:36:52.250846+0000","last_peered":"2024-09-10T23:36:52.250846+0000","last_clean":"2024-09-10T23:36:52.250846+0000","last_became_active":"2024-09-10T23:36:52.248180+0000","last_became_peered":"2024-09-10T23:36:52.248180+0000","last_unstale":"2024-09-10T23:36:52.250846+0000","last_undegraded":"2024-09-10T23:36:52.250846+0000","last_fullsized":"2024-09-10T23:36:52.250846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:13.263641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.256970+0000","last_change":"2024-09-10T23:36:52.250730+0000","last_active":"2024-09-10T23:36:52.256970+0000","last_peered":"2024-09-10T23:36:52.256970+0000","last_clean":"2024-09-10T23:36:52.256970+0000","last_became_active":"2024-09-10T23:36:52.250159+0000","last_became_peered":"2024-09-10T23:36:52.250159+0000","last_unstale":"2024-09-10T23:36:52.256970+0000","last_undegraded":"2024-09-10T23:36:52.256970+0000","last_fullsized":"2024-09-10T23:36:52.256970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:54.263092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.255082+0000","last_change":"2024-09-10T23:36:52.246249+0000","last_active":"2024-09-10T23:36:52.255082+0000","last_peered":"2024-09-10T23:36:52.255082+0000","last_clean":"2024-09-10T23:36:52.255082+0000","last_became_active":"2024-09-10T23:36:52.245928+0000","last_became_peered":"2024-09-10T23:36:52.245928+0000","last_unstale":"2024-09-10T23:36:52.255082+0000","last_undegraded":"2024-09-10T23:36:52.255082+0000","last_fullsized":"2024-09-10T23:36:52.255082+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:52.799130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.251234+0000","last_change":"2024-09-10T23:36:52.247770+0000","last_active":"2024-09-10T23:36:52.251234+0000","last_peered":"2024-09-10T23:36:52.251234+0000","last_clean":"2024-09-10T23:36:52.251234+0000","last_became_active":"2024-09-10T23:36:52.247492+0000","last_became_peered":"2024-09-10T23:36:52.247492+0000","last_unstale":"2024-09-10T23:36:52.251234+0000","last_undegraded":"2024-09-10T23:36:52.251234+0000","last_fullsized":"2024-09-10T23:36:52.251234+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:55.905682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.268741+0000","last_change":"2024-09-10T23:36:52.266099+0000","last_active":"2024-09-10T23:36:52.268741+0000","last_peered":"2024-09-10T23:36:52.268741+0000","last_clean":"2024-09-10T23:36:52.268741+0000","last_became_active":"2024-09-10T23:36:52.264893+0000","last_became_peered":"2024-09-10T23:36:52.264893+0000","last_unstale":"2024-09-10T23:36:52.268741+0000","last_undegraded":"2024-09-10T23:36:52.268741+0000","last_fullsized":"2024-09-10T23:36:52.268741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:18.346281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.266870+0000","last_change":"2024-09-10T23:36:52.255909+0000","last_active":"2024-09-10T23:36:52.266870+0000","last_peered":"2024-09-10T23:36:52.266870+0000","last_clean":"2024-09-10T23:36:52.266870+0000","last_became_active":"2024-09-10T23:36:52.255633+0000","last_became_peered":"2024-09-10T23:36:52.255633+0000","last_unstale":"2024-09-10T23:36:52.266870+0000","last_undegraded":"2024-09-10T23:36:52.266870+0000","last_fullsized":"2024-09-10T23:36:52.266870+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:43.555918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.271127+0000","last_change":"2024-09-10T23:36:52.267389+0000","last_active":"2024-09-10T23:36:52.271127+0000","last_peered":"2024-09-10T23:36:52.271127+0000","last_clean":"2024-09-10T23:36:52.271127+0000","last_became_active":"2024-09-10T23:36:52.267069+0000","last_became_peered":"2024-09-10T23:36:52.267069+0000","last_unstale":"2024-09-10T23:36:52.271127+0000","last_undegraded":"2024-09-10T23:36:52.271127+0000","last_fullsized":"2024-09-10T23:36:52.271127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:24:45.585197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.254642+0000","last_change":"2024-09-10T23:36:52.250542+0000","last_active":"2024-09-10T23:36:52.254642+0000","last_peered":"2024-09-10T23:36:52.254642+0000","last_clean":"2024-09-10T23:36:52.254642+0000","last_became_active":"2024-09-10T23:36:52.250262+0000","last_became_peered":"2024-09-10T23:36:52.250262+0000","last_unstale":"2024-09-10T23:36:52.254642+0000","last_undegraded":"2024-09-10T23:36:52.254642+0000","last_fullsized":"2024-09-10T23:36:52.254642+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:17.852801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.259453+0000","last_change":"2024-09-10T23:36:52.254567+0000","last_active":"2024-09-10T23:36:52.259453+0000","last_peered":"2024-09-10T23:36:52.259453+0000","last_clean":"2024-09-10T23:36:52.259453+0000","last_became_active":"2024-09-10T23:36:52.253855+0000","last_became_peered":"2024-09-10T23:36:52.253855+0000","last_unstale":"2024-09-10T23:36:52.259453+0000","last_undegraded":"2024-09-10T23:36:52.259453+0000","last_fullsized":"2024-09-10T23:36:52.259453+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:05.948035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.263093+0000","last_change":"2024-09-10T23:36:52.261920+0000","last_active":"2024-09-10T23:36:52.263093+0000","last_peered":"2024-09-10T23:36:52.263093+0000","last_clean":"2024-09-10T23:36:52.263093+0000","last_became_active":"2024-09-10T23:36:52.259123+0000","last_became_peered":"2024-09-10T23:36:52.259123+0000","last_unstale":"2024-09-10T23:36:52.263093+0000","last_undegraded":"2024-09-10T23:36:52.263093+0000","last_fullsized":"2024-09-10T23:36:52.263093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:10.084354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.251668+0000","last_change":"2024-09-10T23:36:52.249342+0000","last_active":"2024-09-10T23:36:52.251668+0000","last_peered":"2024-09-10T23:36:52.251668+0000","last_clean":"2024-09-10T23:36:52.251668+0000","last_became_active":"2024-09-10T23:36:52.249092+0000","last_became_peered":"2024-09-10T23:36:52.249092+0000","last_unstale":"2024-09-10T23:36:52.251668+0000","last_undegraded":"2024-09-10T23:36:52.251668+0000","last_fullsized":"2024-09-10T23:36:52.251668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:10.576351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.271190+0000","last_change":"2024-09-10T23:36:52.267327+0000","last_active":"2024-09-10T23:36:52.271190+0000","last_peered":"2024-09-10T23:36:52.271190+0000","last_clean":"2024-09-10T23:36:52.271190+0000","last_became_active":"2024-09-10T23:36:52.267011+0000","last_became_peered":"2024-09-10T23:36:52.267011+0000","last_unstale":"2024-09-10T23:36:52.271190+0000","last_undegraded":"2024-09-10T23:36:52.271190+0000","last_fullsized":"2024-09-10T23:36:52.271190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:15.868540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.273207+0000","last_change":"2024-09-10T23:36:52.267758+0000","last_active":"2024-09-10T23:36:52.273207+0000","last_peered":"2024-09-10T23:36:52.273207+0000","last_clean":"2024-09-10T23:36:52.273207+0000","last_became_active":"2024-09-10T23:36:52.267322+0000","last_became_peered":"2024-09-10T23:36:52.267322+0000","last_unstale":"2024-09-10T23:36:52.273207+0000","last_undegraded":"2024-09-10T23:36:52.273207+0000","last_fullsized":"2024-09-10T23:36:52.273207+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:51.483384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.263795+0000","last_change":"2024-09-10T23:36:52.252266+0000","last_active":"2024-09-10T23:36:52.263795+0000","last_peered":"2024-09-10T23:36:52.263795+0000","last_clean":"2024-09-10T23:36:52.263795+0000","last_became_active":"2024-09-10T23:36:52.251683+0000","last_became_peered":"2024-09-10T23:36:52.251683+0000","last_unstale":"2024-09-10T23:36:52.263795+0000","last_undegraded":"2024-09-10T23:36:52.263795+0000","last_fullsized":"2024-09-10T23:36:52.263795+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:57.163842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.273346+0000","last_change":"2024-09-10T23:36:52.269242+0000","last_active":"2024-09-10T23:36:52.273346+0000","last_peered":"2024-09-10T23:36:52.273346+0000","last_clean":"2024-09-10T23:36:52.273346+0000","last_became_active":"2024-09-10T23:36:52.268157+0000","last_became_peered":"2024-09-10T23:36:52.268157+0000","last_unstale":"2024-09-10T23:36:52.273346+0000","last_undegraded":"2024-09-10T23:36:52.273346+0000","last_fullsized":"2024-09-10T23:36:52.273346+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:31.967692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.267444+0000","last_change":"2024-09-10T23:36:52.264159+0000","last_active":"2024-09-10T23:36:52.267444+0000","last_peered":"2024-09-10T23:36:52.267444+0000","last_clean":"2024-09-10T23:36:52.267444+0000","last_became_active":"2024-09-10T23:36:52.263842+0000","last_became_peered":"2024-09-10T23:36:52.263842+0000","last_unstale":"2024-09-10T23:36:52.267444+0000","last_undegraded":"2024-09-10T23:36:52.267444+0000","last_fullsized":"2024-09-10T23:36:52.267444+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:19.173697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.264554+0000","last_change":"2024-09-10T23:36:52.261763+0000","last_active":"2024-09-10T23:36:52.264554+0000","last_peered":"2024-09-10T23:36:52.264554+0000","last_clean":"2024-09-10T23:36:52.264554+0000","last_became_active":"2024-09-10T23:36:52.261500+0000","last_became_peered":"2024-09-10T23:36:52.261500+0000","last_unstale":"2024-09-10T23:36:52.264554+0000","last_undegraded":"2024-09-10T23:36:52.264554+0000","last_fullsized":"2024-09-10T23:36:52.264554+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:18.401283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.270833+0000","last_change":"2024-09-10T23:36:52.265930+0000","last_active":"2024-09-10T23:36:52.270833+0000","last_peered":"2024-09-10T23:36:52.270833+0000","last_clean":"2024-09-10T23:36:52.270833+0000","last_became_active":"2024-09-10T23:36:52.265541+0000","last_became_peered":"2024-09-10T23:36:52.265541+0000","last_unstale":"2024-09-10T23:36:52.270833+0000","last_undegraded":"2024-09-10T23:36:52.270833+0000","last_fullsized":"2024-09-10T23:36:52.270833+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:24.782214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.268573+0000","last_change":"2024-09-10T23:36:52.256559+0000","last_active":"2024-09-10T23:36:52.268573+0000","last_peered":"2024-09-10T23:36:52.268573+0000","last_clean":"2024-09-10T23:36:52.268573+0000","last_became_active":"2024-09-10T23:36:52.256321+0000","last_became_peered":"2024-09-10T23:36:52.256321+0000","last_unstale":"2024-09-10T23:36:52.268573+0000","last_undegraded":"2024-09-10T23:36:52.268573+0000","last_fullsized":"2024-09-10T23:36:52.268573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:33.966515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.269119+0000","last_change":"2024-09-10T23:36:52.261074+0000","last_active":"2024-09-10T23:36:52.269119+0000","last_peered":"2024-09-10T23:36:52.269119+0000","last_clean":"2024-09-10T23:36:52.269119+0000","last_became_active":"2024-09-10T23:36:52.254194+0000","last_became_peered":"2024-09-10T23:36:52.254194+0000","last_unstale":"2024-09-10T23:36:52.269119+0000","last_undegraded":"2024-09-10T23:36:52.269119+0000","last_fullsized":"2024-09-10T23:36:52.269119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:31.839877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.270581+0000","last_change":"2024-09-10T23:36:52.269141+0000","last_active":"2024-09-10T23:36:52.270581+0000","last_peered":"2024-09-10T23:36:52.270581+0000","last_clean":"2024-09-10T23:36:52.270581+0000","last_became_active":"2024-09-10T23:36:52.268857+0000","last_became_peered":"2024-09-10T23:36:52.268857+0000","last_unstale":"2024-09-10T23:36:52.270581+0000","last_undegraded":"2024-09-10T23:36:52.270581+0000","last_fullsized":"2024-09-10T23:36:52.270581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:56.174167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.263771+0000","last_change":"2024-09-10T23:36:52.251948+0000","last_active":"2024-09-10T23:36:52.263771+0000","last_peered":"2024-09-10T23:36:52.263771+0000","last_clean":"2024-09-10T23:36:52.263771+0000","last_became_active":"2024-09-10T23:36:52.251371+0000","last_became_peered":"2024-09-10T23:36:52.251371+0000","last_unstale":"2024-09-10T23:36:52.263771+0000","last_undegraded":"2024-09-10T23:36:52.263771+0000","last_fullsized":"2024-09-10T23:36:52.263771+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:57.457285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.273242+0000","last_change":"2024-09-10T23:36:52.268185+0000","last_active":"2024-09-10T23:36:52.273242+0000","last_peered":"2024-09-10T23:36:52.273242+0000","last_clean":"2024-09-10T23:36:52.273242+0000","last_became_active":"2024-09-10T23:36:52.267670+0000","last_became_peered":"2024-09-10T23:36:52.267670+0000","last_unstale":"2024-09-10T23:36:52.273242+0000","last_undegraded":"2024-09-10T23:36:52.273242+0000","last_fullsized":"2024-09-10T23:36:52.273242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:25:05.695698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.260746+0000","last_change":"2024-09-10T23:36:52.253057+0000","last_active":"2024-09-10T23:36:52.260746+0000","last_peered":"2024-09-10T23:36:52.260746+0000","last_clean":"2024-09-10T23:36:52.260746+0000","last_became_active":"2024-09-10T23:36:52.252820+0000","last_became_peered":"2024-09-10T23:36:52.252820+0000","last_unstale":"2024-09-10T23:36:52.260746+0000","last_undegraded":"2024-09-10T23:36:52.260746+0000","last_fullsized":"2024-09-10T23:36:52.260746+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:34.147787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.259426+0000","last_change":"2024-09-10T23:36:52.252449+0000","last_active":"2024-09-10T23:36:52.259426+0000","last_peered":"2024-09-10T23:36:52.259426+0000","last_clean":"2024-09-10T23:36:52.259426+0000","last_became_active":"2024-09-10T23:36:52.252229+0000","last_became_peered":"2024-09-10T23:36:52.252229+0000","last_unstale":"2024-09-10T23:36:52.259426+0000","last_undegraded":"2024-09-10T23:36:52.259426+0000","last_fullsized":"2024-09-10T23:36:52.259426+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:10.111868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.265618+0000","last_change":"2024-09-10T23:36:52.262019+0000","last_active":"2024-09-10T23:36:52.265618+0000","last_peered":"2024-09-10T23:36:52.265618+0000","last_clean":"2024-09-10T23:36:52.265618+0000","last_became_active":"2024-09-10T23:36:52.261761+0000","last_became_peered":"2024-09-10T23:36:52.261761+0000","last_unstale":"2024-09-10T23:36:52.265618+0000","last_undegraded":"2024-09-10T23:36:52.265618+0000","last_fullsized":"2024-09-10T23:36:52.265618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:37.753986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"80'4","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.271559+0000","last_change":"2024-09-10T23:36:51.250986+0000","last_active":"2024-09-10T23:36:52.271559+0000","last_peered":"2024-09-10T23:36:52.271559+0000","last_clean":"2024-09-10T23:36:52.271559+0000","last_became_active":"2024-09-10T23:36:51.249834+0000","last_became_peered":"2024-09-10T23:36:51.249834+0000","last_unstale":"2024-09-10T23:36:52.271559+0000","last_undegraded":"2024-09-10T23:36:52.271559+0000","last_fullsized":"2024-09-10T23:36:52.271559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:12.153127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.255473+0000","last_change":"2024-09-10T23:36:52.248789+0000","last_active":"2024-09-10T23:36:52.255473+0000","last_peered":"2024-09-10T23:36:52.255473+0000","last_clean":"2024-09-10T23:36:52.255473+0000","last_became_active":"2024-09-10T23:36:52.248202+0000","last_became_peered":"2024-09-10T23:36:52.248202+0000","last_unstale":"2024-09-10T23:36:52.255473+0000","last_undegraded":"2024-09-10T23:36:52.255473+0000","last_fullsized":"2024-09-10T23:36:52.255473+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:15.602323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.266192+0000","last_change":"2024-09-10T23:36:51.249624+0000","last_active":"2024-09-10T23:36:52.266192+0000","last_peered":"2024-09-10T23:36:52.266192+0000","last_clean":"2024-09-10T23:36:52.266192+0000","last_became_active":"2024-09-10T23:36:51.249303+0000","last_became_peered":"2024-09-10T23:36:51.249303+0000","last_unstale":"2024-09-10T23:36:52.266192+0000","last_undegraded":"2024-09-10T23:36:52.266192+0000","last_fullsized":"2024-09-10T23:36:52.266192+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:11.778368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.257114+0000","last_change":"2024-09-10T23:36:52.254194+0000","last_active":"2024-09-10T23:36:52.257114+0000","last_peered":"2024-09-10T23:36:52.257114+0000","last_clean":"2024-09-10T23:36:52.257114+0000","last_became_active":"2024-09-10T23:36:52.248701+0000","last_became_peered":"2024-09-10T23:36:52.248701+0000","last_unstale":"2024-09-10T23:36:52.257114+0000","last_undegraded":"2024-09-10T23:36:52.257114+0000","last_fullsized":"2024-09-10T23:36:52.257114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:15:13.644272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.271514+0000","last_change":"2024-09-10T23:36:51.249893+0000","last_active":"2024-09-10T23:36:52.271514+0000","last_peered":"2024-09-10T23:36:52.271514+0000","last_clean":"2024-09-10T23:36:52.271514+0000","last_became_active":"2024-09-10T23:36:51.249618+0000","last_became_peered":"2024-09-10T23:36:51.249618+0000","last_unstale":"2024-09-10T23:36:52.271514+0000","last_undegraded":"2024-09-10T23:36:52.271514+0000","last_fullsized":"2024-09-10T23:36:52.271514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:15.221785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.256894+0000","last_change":"2024-09-10T23:36:52.252748+0000","last_active":"2024-09-10T23:36:52.256894+0000","last_peered":"2024-09-10T23:36:52.256894+0000","last_clean":"2024-09-10T23:36:52.256894+0000","last_became_active":"2024-09-10T23:36:52.249665+0000","last_became_peered":"2024-09-10T23:36:52.249665+0000","last_unstale":"2024-09-10T23:36:52.256894+0000","last_undegraded":"2024-09-10T23:36:52.256894+0000","last_fullsized":"2024-09-10T23:36:52.256894+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:22.543318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.270778+0000","last_change":"2024-09-10T23:36:51.251958+0000","last_active":"2024-09-10T23:36:52.270778+0000","last_peered":"2024-09-10T23:36:52.270778+0000","last_clean":"2024-09-10T23:36:52.270778+0000","last_became_active":"2024-09-10T23:36:51.251386+0000","last_became_peered":"2024-09-10T23:36:51.251386+0000","last_unstale":"2024-09-10T23:36:52.270778+0000","last_undegraded":"2024-09-10T23:36:52.270778+0000","last_fullsized":"2024-09-10T23:36:52.270778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:53:20.826875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.265091+0000","last_change":"2024-09-10T23:36:52.250800+0000","last_active":"2024-09-10T23:36:52.265091+0000","last_peered":"2024-09-10T23:36:52.265091+0000","last_clean":"2024-09-10T23:36:52.265091+0000","last_became_active":"2024-09-10T23:36:52.250476+0000","last_became_peered":"2024-09-10T23:36:52.250476+0000","last_unstale":"2024-09-10T23:36:52.265091+0000","last_undegraded":"2024-09-10T23:36:52.265091+0000","last_fullsized":"2024-09-10T23:36:52.265091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:16:47.505665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.259994+0000","last_change":"2024-09-10T23:36:51.254300+0000","last_active":"2024-09-10T23:36:52.259994+0000","last_peered":"2024-09-10T23:36:52.259994+0000","last_clean":"2024-09-10T23:36:52.259994+0000","last_became_active":"2024-09-10T23:36:51.254006+0000","last_became_peered":"2024-09-10T23:36:51.254006+0000","last_unstale":"2024-09-10T23:36:52.259994+0000","last_undegraded":"2024-09-10T23:36:52.259994+0000","last_fullsized":"2024-09-10T23:36:52.259994+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:27.522259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.266133+0000","last_change":"2024-09-10T23:36:52.264649+0000","last_active":"2024-09-10T23:36:52.266133+0000","last_peered":"2024-09-10T23:36:52.266133+0000","last_clean":"2024-09-10T23:36:52.266133+0000","last_became_active":"2024-09-10T23:36:52.264217+0000","last_became_peered":"2024-09-10T23:36:52.264217+0000","last_unstale":"2024-09-10T23:36:52.266133+0000","last_undegraded":"2024-09-10T23:36:52.266133+0000","last_fullsized":"2024-09-10T23:36:52.266133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:57.457285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.270468+0000","last_change":"2024-09-10T23:36:51.246870+0000","last_active":"2024-09-10T23:36:52.270468+0000","last_peered":"2024-09-10T23:36:52.270468+0000","last_clean":"2024-09-10T23:36:52.270468+0000","last_became_active":"2024-09-10T23:36:51.246555+0000","last_became_peered":"2024-09-10T23:36:51.246555+0000","last_unstale":"2024-09-10T23:36:52.270468+0000","last_undegraded":"2024-09-10T23:36:52.270468+0000","last_fullsized":"2024-09-10T23:36:52.270468+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:49:08.273176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.256829+0000","last_change":"2024-09-10T23:36:52.252083+0000","last_active":"2024-09-10T23:36:52.256829+0000","last_peered":"2024-09-10T23:36:52.256829+0000","last_clean":"2024-09-10T23:36:52.256829+0000","last_became_active":"2024-09-10T23:36:52.249587+0000","last_became_peered":"2024-09-10T23:36:52.249587+0000","last_unstale":"2024-09-10T23:36:52.256829+0000","last_undegraded":"2024-09-10T23:36:52.256829+0000","last_fullsized":"2024-09-10T23:36:52.256829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:30.281670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.255794+0000","last_change":"2024-09-10T23:36:51.249469+0000","last_active":"2024-09-10T23:36:52.255794+0000","last_peered":"2024-09-10T23:36:52.255794+0000","last_clean":"2024-09-10T23:36:52.255794+0000","last_became_active":"2024-09-10T23:36:51.249165+0000","last_became_peered":"2024-09-10T23:36:51.249165+0000","last_unstale":"2024-09-10T23:36:52.255794+0000","last_undegraded":"2024-09-10T23:36:52.255794+0000","last_fullsized":"2024-09-10T23:36:52.255794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:49:02.808530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.273437+0000","last_change":"2024-09-10T23:36:52.269981+0000","last_active":"2024-09-10T23:36:52.273437+0000","last_peered":"2024-09-10T23:36:52.273437+0000","last_clean":"2024-09-10T23:36:52.273437+0000","last_became_active":"2024-09-10T23:36:52.269690+0000","last_became_peered":"2024-09-10T23:36:52.269690+0000","last_unstale":"2024-09-10T23:36:52.273437+0000","last_undegraded":"2024-09-10T23:36:52.273437+0000","last_fullsized":"2024-09-10T23:36:52.273437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:59.619749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.254433+0000","last_change":"2024-09-10T23:36:51.252955+0000","last_active":"2024-09-10T23:36:52.254433+0000","last_peered":"2024-09-10T23:36:52.254433+0000","last_clean":"2024-09-10T23:36:52.254433+0000","last_became_active":"2024-09-10T23:36:51.252649+0000","last_became_peered":"2024-09-10T23:36:51.252649+0000","last_unstale":"2024-09-10T23:36:52.254433+0000","last_undegraded":"2024-09-10T23:36:52.254433+0000","last_fullsized":"2024-09-10T23:36:52.254433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:55.232022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.265034+0000","last_change":"2024-09-10T23:36:52.260178+0000","last_active":"2024-09-10T23:36:52.265034+0000","last_peered":"2024-09-10T23:36:52.265034+0000","last_clean":"2024-09-10T23:36:52.265034+0000","last_became_active":"2024-09-10T23:36:52.259849+0000","last_became_peered":"2024-09-10T23:36:52.259849+0000","last_unstale":"2024-09-10T23:36:52.265034+0000","last_undegraded":"2024-09-10T23:36:52.265034+0000","last_fullsized":"2024-09-10T23:36:52.265034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:18.664368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.233179+0000","last_change":"2024-09-10T23:36:51.251994+0000","last_active":"2024-09-10T23:36:52.233179+0000","last_peered":"2024-09-10T23:36:52.233179+0000","last_clean":"2024-09-10T23:36:52.233179+0000","last_became_active":"2024-09-10T23:36:51.251676+0000","last_became_peered":"2024-09-10T23:36:51.251676+0000","last_unstale":"2024-09-10T23:36:52.233179+0000","last_undegraded":"2024-09-10T23:36:52.233179+0000","last_fullsized":"2024-09-10T23:36:52.233179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:13.300624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.256950+0000","last_change":"2024-09-10T23:36:52.250424+0000","last_active":"2024-09-10T23:36:52.256950+0000","last_peered":"2024-09-10T23:36:52.256950+0000","last_clean":"2024-09-10T23:36:52.256950+0000","last_became_active":"2024-09-10T23:36:52.249829+0000","last_became_peered":"2024-09-10T23:36:52.249829+0000","last_unstale":"2024-09-10T23:36:52.256950+0000","last_undegraded":"2024-09-10T23:36:52.256950+0000","last_fullsized":"2024-09-10T23:36:52.256950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:32.885162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.237993+0000","last_change":"2024-09-10T23:36:51.238852+0000","last_active":"2024-09-10T23:36:52.237993+0000","last_peered":"2024-09-10T23:36:52.237993+0000","last_clean":"2024-09-10T23:36:52.237993+0000","last_became_active":"2024-09-10T23:36:51.238599+0000","last_became_peered":"2024-09-10T23:36:51.238599+0000","last_unstale":"2024-09-10T23:36:52.237993+0000","last_undegraded":"2024-09-10T23:36:52.237993+0000","last_fullsized":"2024-09-10T23:36:52.237993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:08.162686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.263755+0000","last_change":"2024-09-10T23:36:52.261680+0000","last_active":"2024-09-10T23:36:52.263755+0000","last_peered":"2024-09-10T23:36:52.263755+0000","last_clean":"2024-09-10T23:36:52.263755+0000","last_became_active":"2024-09-10T23:36:52.261123+0000","last_became_peered":"2024-09-10T23:36:52.261123+0000","last_unstale":"2024-09-10T23:36:52.263755+0000","last_undegraded":"2024-09-10T23:36:52.263755+0000","last_fullsized":"2024-09-10T23:36:52.263755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:59.321006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"80'3","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.268423+0000","last_change":"2024-09-10T23:36:51.253740+0000","last_active":"2024-09-10T23:36:52.268423+0000","last_peered":"2024-09-10T23:36:52.268423+0000","last_clean":"2024-09-10T23:36:52.268423+0000","last_became_active":"2024-09-10T23:36:51.253049+0000","last_became_peered":"2024-09-10T23:36:51.253049+0000","last_unstale":"2024-09-10T23:36:52.268423+0000","last_undegraded":"2024-09-10T23:36:52.268423+0000","last_fullsized":"2024-09-10T23:36:52.268423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:29.077715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.269143+0000","last_change":"2024-09-10T23:36:52.261610+0000","last_active":"2024-09-10T23:36:52.269143+0000","last_peered":"2024-09-10T23:36:52.269143+0000","last_clean":"2024-09-10T23:36:52.269143+0000","last_became_active":"2024-09-10T23:36:52.254487+0000","last_became_peered":"2024-09-10T23:36:52.254487+0000","last_unstale":"2024-09-10T23:36:52.269143+0000","last_undegraded":"2024-09-10T23:36:52.269143+0000","last_fullsized":"2024-09-10T23:36:52.269143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:26.511936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"80'6","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.273844+0000","last_change":"2024-09-10T23:36:51.245606+0000","last_active":"2024-09-10T23:36:52.273844+0000","last_peered":"2024-09-10T23:36:52.273844+0000","last_clean":"2024-09-10T23:36:52.273844+0000","last_became_active":"2024-09-10T23:36:51.245347+0000","last_became_peered":"2024-09-10T23:36:51.245347+0000","last_unstale":"2024-09-10T23:36:52.273844+0000","last_undegraded":"2024-09-10T23:36:52.273844+0000","last_fullsized":"2024-09-10T23:36:52.273844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:57.414029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.258931+0000","last_change":"2024-09-10T23:36:52.253082+0000","last_active":"2024-09-10T23:36:52.258931+0000","last_peered":"2024-09-10T23:36:52.258931+0000","last_clean":"2024-09-10T23:36:52.258931+0000","last_became_active":"2024-09-10T23:36:52.252807+0000","last_became_peered":"2024-09-10T23:36:52.252807+0000","last_unstale":"2024-09-10T23:36:52.258931+0000","last_undegraded":"2024-09-10T23:36:52.258931+0000","last_fullsized":"2024-09-10T23:36:52.258931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:03.400629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.267308+0000","last_change":"2024-09-10T23:36:51.247636+0000","last_active":"2024-09-10T23:36:52.267308+0000","last_peered":"2024-09-10T23:36:52.267308+0000","last_clean":"2024-09-10T23:36:52.267308+0000","last_became_active":"2024-09-10T23:36:51.247240+0000","last_became_peered":"2024-09-10T23:36:51.247240+0000","last_unstale":"2024-09-10T23:36:52.267308+0000","last_undegraded":"2024-09-10T23:36:52.267308+0000","last_fullsized":"2024-09-10T23:36:52.267308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:51.618614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.256682+0000","last_change":"2024-09-10T23:36:52.250394+0000","last_active":"2024-09-10T23:36:52.256682+0000","last_peered":"2024-09-10T23:36:52.256682+0000","last_clean":"2024-09-10T23:36:52.256682+0000","last_became_active":"2024-09-10T23:36:52.250128+0000","last_became_peered":"2024-09-10T23:36:52.250128+0000","last_unstale":"2024-09-10T23:36:52.256682+0000","last_undegraded":"2024-09-10T23:36:52.256682+0000","last_fullsized":"2024-09-10T23:36:52.256682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:28:10.501354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.240255+0000","last_change":"2024-09-10T23:36:51.238946+0000","last_active":"2024-09-10T23:36:52.240255+0000","last_peered":"2024-09-10T23:36:52.240255+0000","last_clean":"2024-09-10T23:36:52.240255+0000","last_became_active":"2024-09-10T23:36:51.238554+0000","last_became_peered":"2024-09-10T23:36:51.238554+0000","last_unstale":"2024-09-10T23:36:52.240255+0000","last_undegraded":"2024-09-10T23:36:52.240255+0000","last_fullsized":"2024-09-10T23:36:52.240255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:27.522259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.273457+0000","last_change":"2024-09-10T23:36:52.269710+0000","last_active":"2024-09-10T23:36:52.273457+0000","last_peered":"2024-09-10T23:36:52.273457+0000","last_clean":"2024-09-10T23:36:52.273457+0000","last_became_active":"2024-09-10T23:36:52.269444+0000","last_became_peered":"2024-09-10T23:36:52.269444+0000","last_unstale":"2024-09-10T23:36:52.273457+0000","last_undegraded":"2024-09-10T23:36:52.273457+0000","last_fullsized":"2024-09-10T23:36:52.273457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:05.738262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.237695+0000","last_change":"2024-09-10T23:36:51.249715+0000","last_active":"2024-09-10T23:36:52.237695+0000","last_peered":"2024-09-10T23:36:52.237695+0000","last_clean":"2024-09-10T23:36:52.237695+0000","last_became_active":"2024-09-10T23:36:51.249457+0000","last_became_peered":"2024-09-10T23:36:51.249457+0000","last_unstale":"2024-09-10T23:36:52.237695+0000","last_undegraded":"2024-09-10T23:36:52.237695+0000","last_fullsized":"2024-09-10T23:36:52.237695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:23:25.253315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.257537+0000","last_change":"2024-09-10T23:36:52.252699+0000","last_active":"2024-09-10T23:36:52.257537+0000","last_peered":"2024-09-10T23:36:52.257537+0000","last_clean":"2024-09-10T23:36:52.257537+0000","last_became_active":"2024-09-10T23:36:52.252399+0000","last_became_peered":"2024-09-10T23:36:52.252399+0000","last_unstale":"2024-09-10T23:36:52.257537+0000","last_undegraded":"2024-09-10T23:36:52.257537+0000","last_fullsized":"2024-09-10T23:36:52.257537+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:03.704223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"80'8","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.271656+0000","last_change":"2024-09-10T23:36:51.250076+0000","last_active":"2024-09-10T23:36:52.271656+0000","last_peered":"2024-09-10T23:36:52.271656+0000","last_clean":"2024-09-10T23:36:52.271656+0000","last_became_active":"2024-09-10T23:36:51.249800+0000","last_became_peered":"2024-09-10T23:36:51.249800+0000","last_unstale":"2024-09-10T23:36:52.271656+0000","last_undegraded":"2024-09-10T23:36:52.271656+0000","last_fullsized":"2024-09-10T23:36:52.271656+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:31.580397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.248813+0000","last_change":"2024-09-10T23:36:52.243894+0000","last_active":"2024-09-10T23:36:52.248813+0000","last_peered":"2024-09-10T23:36:52.248813+0000","last_clean":"2024-09-10T23:36:52.248813+0000","last_became_active":"2024-09-10T23:36:52.243639+0000","last_became_peered":"2024-09-10T23:36:52.243639+0000","last_unstale":"2024-09-10T23:36:52.248813+0000","last_undegraded":"2024-09-10T23:36:52.248813+0000","last_fullsized":"2024-09-10T23:36:52.248813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:24.282604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.268290+0000","last_change":"2024-09-10T23:36:51.250367+0000","last_active":"2024-09-10T23:36:52.268290+0000","last_peered":"2024-09-10T23:36:52.268290+0000","last_clean":"2024-09-10T23:36:52.268290+0000","last_became_active":"2024-09-10T23:36:51.250069+0000","last_became_peered":"2024-09-10T23:36:51.250069+0000","last_unstale":"2024-09-10T23:36:52.268290+0000","last_undegraded":"2024-09-10T23:36:52.268290+0000","last_fullsized":"2024-09-10T23:36:52.268290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:08.014677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.266094+0000","last_change":"2024-09-10T23:36:52.264246+0000","last_active":"2024-09-10T23:36:52.266094+0000","last_peered":"2024-09-10T23:36:52.266094+0000","last_clean":"2024-09-10T23:36:52.266094+0000","last_became_active":"2024-09-10T23:36:52.263999+0000","last_became_peered":"2024-09-10T23:36:52.263999+0000","last_unstale":"2024-09-10T23:36:52.266094+0000","last_undegraded":"2024-09-10T23:36:52.266094+0000","last_fullsized":"2024-09-10T23:36:52.266094+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:16:47.505665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.269770+0000","last_change":"2024-09-10T23:36:51.252590+0000","last_active":"2024-09-10T23:36:52.269770+0000","last_peered":"2024-09-10T23:36:52.269770+0000","last_clean":"2024-09-10T23:36:52.269770+0000","last_became_active":"2024-09-10T23:36:51.252332+0000","last_became_peered":"2024-09-10T23:36:51.252332+0000","last_unstale":"2024-09-10T23:36:52.269770+0000","last_undegraded":"2024-09-10T23:36:52.269770+0000","last_fullsized":"2024-09-10T23:36:52.269770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:54:02.579561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.247405+0000","last_change":"2024-09-10T23:36:52.242754+0000","last_active":"2024-09-10T23:36:52.247405+0000","last_peered":"2024-09-10T23:36:52.247405+0000","last_clean":"2024-09-10T23:36:52.247405+0000","last_became_active":"2024-09-10T23:36:52.242484+0000","last_became_peered":"2024-09-10T23:36:52.242484+0000","last_unstale":"2024-09-10T23:36:52.247405+0000","last_undegraded":"2024-09-10T23:36:52.247405+0000","last_fullsized":"2024-09-10T23:36:52.247405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:17.852801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.253746+0000","last_change":"2024-09-10T23:36:51.251584+0000","last_active":"2024-09-10T23:36:52.253746+0000","last_peered":"2024-09-10T23:36:52.253746+0000","last_clean":"2024-09-10T23:36:52.253746+0000","last_became_active":"2024-09-10T23:36:51.251286+0000","last_became_peered":"2024-09-10T23:36:51.251286+0000","last_unstale":"2024-09-10T23:36:52.253746+0000","last_undegraded":"2024-09-10T23:36:52.253746+0000","last_fullsized":"2024-09-10T23:36:52.253746+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:43.539868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.254603+0000","last_change":"2024-09-10T23:36:52.250582+0000","last_active":"2024-09-10T23:36:52.254603+0000","last_peered":"2024-09-10T23:36:52.254603+0000","last_clean":"2024-09-10T23:36:52.254603+0000","last_became_active":"2024-09-10T23:36:52.250316+0000","last_became_peered":"2024-09-10T23:36:52.250316+0000","last_unstale":"2024-09-10T23:36:52.254603+0000","last_undegraded":"2024-09-10T23:36:52.254603+0000","last_fullsized":"2024-09-10T23:36:52.254603+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:24.282604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.268231+0000","last_change":"2024-09-10T23:36:51.254512+0000","last_active":"2024-09-10T23:36:52.268231+0000","last_peered":"2024-09-10T23:36:52.268231+0000","last_clean":"2024-09-10T23:36:52.268231+0000","last_became_active":"2024-09-10T23:36:51.254202+0000","last_became_peered":"2024-09-10T23:36:51.254202+0000","last_unstale":"2024-09-10T23:36:52.268231+0000","last_undegraded":"2024-09-10T23:36:52.268231+0000","last_fullsized":"2024-09-10T23:36:52.268231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:10.349358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.255814+0000","last_change":"2024-09-10T23:36:52.250037+0000","last_active":"2024-09-10T23:36:52.255814+0000","last_peered":"2024-09-10T23:36:52.255814+0000","last_clean":"2024-09-10T23:36:52.255814+0000","last_became_active":"2024-09-10T23:36:52.249754+0000","last_became_peered":"2024-09-10T23:36:52.249754+0000","last_unstale":"2024-09-10T23:36:52.255814+0000","last_undegraded":"2024-09-10T23:36:52.255814+0000","last_fullsized":"2024-09-10T23:36:52.255814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:20.586094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.239791+0000","last_change":"2024-09-10T23:36:51.243419+0000","last_active":"2024-09-10T23:36:52.239791+0000","last_peered":"2024-09-10T23:36:52.239791+0000","last_clean":"2024-09-10T23:36:52.239791+0000","last_became_active":"2024-09-10T23:36:51.242377+0000","last_became_peered":"2024-09-10T23:36:51.242377+0000","last_unstale":"2024-09-10T23:36:52.239791+0000","last_undegraded":"2024-09-10T23:36:52.239791+0000","last_fullsized":"2024-09-10T23:36:52.239791+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:57.414029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.265703+0000","last_change":"2024-09-10T23:36:52.258113+0000","last_active":"2024-09-10T23:36:52.265703+0000","last_peered":"2024-09-10T23:36:52.265703+0000","last_clean":"2024-09-10T23:36:52.265703+0000","last_became_active":"2024-09-10T23:36:52.257848+0000","last_became_peered":"2024-09-10T23:36:52.257848+0000","last_unstale":"2024-09-10T23:36:52.265703+0000","last_undegraded":"2024-09-10T23:36:52.265703+0000","last_fullsized":"2024-09-10T23:36:52.265703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:30.795006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.243829+0000","last_change":"2024-09-10T23:36:51.247599+0000","last_active":"2024-09-10T23:36:52.243829+0000","last_peered":"2024-09-10T23:36:52.243829+0000","last_clean":"2024-09-10T23:36:52.243829+0000","last_became_active":"2024-09-10T23:36:51.246845+0000","last_became_peered":"2024-09-10T23:36:51.246845+0000","last_unstale":"2024-09-10T23:36:52.243829+0000","last_undegraded":"2024-09-10T23:36:52.243829+0000","last_fullsized":"2024-09-10T23:36:52.243829+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:56.443755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.250235+0000","last_change":"2024-09-10T23:36:52.249537+0000","last_active":"2024-09-10T23:36:52.250235+0000","last_peered":"2024-09-10T23:36:52.250235+0000","last_clean":"2024-09-10T23:36:52.250235+0000","last_became_active":"2024-09-10T23:36:52.249293+0000","last_became_peered":"2024-09-10T23:36:52.249293+0000","last_unstale":"2024-09-10T23:36:52.250235+0000","last_undegraded":"2024-09-10T23:36:52.250235+0000","last_fullsized":"2024-09-10T23:36:52.250235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:27.984279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.254642+0000","last_change":"2024-09-10T23:36:51.251718+0000","last_active":"2024-09-10T23:36:52.254642+0000","last_peered":"2024-09-10T23:36:52.254642+0000","last_clean":"2024-09-10T23:36:52.254642+0000","last_became_active":"2024-09-10T23:36:51.251260+0000","last_became_peered":"2024-09-10T23:36:51.251260+0000","last_unstale":"2024-09-10T23:36:52.254642+0000","last_undegraded":"2024-09-10T23:36:52.254642+0000","last_fullsized":"2024-09-10T23:36:52.254642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:10.354321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.265982+0000","last_change":"2024-09-10T23:36:52.255169+0000","last_active":"2024-09-10T23:36:52.265982+0000","last_peered":"2024-09-10T23:36:52.265982+0000","last_clean":"2024-09-10T23:36:52.265982+0000","last_became_active":"2024-09-10T23:36:52.254812+0000","last_became_peered":"2024-09-10T23:36:52.254812+0000","last_unstale":"2024-09-10T23:36:52.265982+0000","last_undegraded":"2024-09-10T23:36:52.265982+0000","last_fullsized":"2024-09-10T23:36:52.265982+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:16:32.579522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.267131+0000","last_change":"2024-09-10T23:36:51.254207+0000","last_active":"2024-09-10T23:36:52.267131+0000","last_peered":"2024-09-10T23:36:52.267131+0000","last_clean":"2024-09-10T23:36:52.267131+0000","last_became_active":"2024-09-10T23:36:51.253877+0000","last_became_peered":"2024-09-10T23:36:51.253877+0000","last_unstale":"2024-09-10T23:36:52.267131+0000","last_undegraded":"2024-09-10T23:36:52.267131+0000","last_fullsized":"2024-09-10T23:36:52.267131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:51.964881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.261304+0000","last_change":"2024-09-10T23:36:52.254595+0000","last_active":"2024-09-10T23:36:52.261304+0000","last_peered":"2024-09-10T23:36:52.261304+0000","last_clean":"2024-09-10T23:36:52.261304+0000","last_became_active":"2024-09-10T23:36:52.254290+0000","last_became_peered":"2024-09-10T23:36:52.254290+0000","last_unstale":"2024-09-10T23:36:52.261304+0000","last_undegraded":"2024-09-10T23:36:52.261304+0000","last_fullsized":"2024-09-10T23:36:52.261304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:34:20.035632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.242692+0000","last_change":"2024-09-10T23:36:51.249202+0000","last_active":"2024-09-10T23:36:52.242692+0000","last_peered":"2024-09-10T23:36:52.242692+0000","last_clean":"2024-09-10T23:36:52.242692+0000","last_became_active":"2024-09-10T23:36:51.248905+0000","last_became_peered":"2024-09-10T23:36:51.248905+0000","last_unstale":"2024-09-10T23:36:52.242692+0000","last_undegraded":"2024-09-10T23:36:52.242692+0000","last_fullsized":"2024-09-10T23:36:52.242692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:30.899676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.267588+0000","last_change":"2024-09-10T23:36:52.257937+0000","last_active":"2024-09-10T23:36:52.267588+0000","last_peered":"2024-09-10T23:36:52.267588+0000","last_clean":"2024-09-10T23:36:52.267588+0000","last_became_active":"2024-09-10T23:36:52.257521+0000","last_became_peered":"2024-09-10T23:36:52.257521+0000","last_unstale":"2024-09-10T23:36:52.267588+0000","last_undegraded":"2024-09-10T23:36:52.267588+0000","last_fullsized":"2024-09-10T23:36:52.267588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:46.298248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.239799+0000","last_change":"2024-09-10T23:36:51.247320+0000","last_active":"2024-09-10T23:36:52.239799+0000","last_peered":"2024-09-10T23:36:52.239799+0000","last_clean":"2024-09-10T23:36:52.239799+0000","last_became_active":"2024-09-10T23:36:51.247016+0000","last_became_peered":"2024-09-10T23:36:51.247016+0000","last_unstale":"2024-09-10T23:36:52.239799+0000","last_undegraded":"2024-09-10T23:36:52.239799+0000","last_fullsized":"2024-09-10T23:36:52.239799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:40.701589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.267558+0000","last_change":"2024-09-10T23:36:52.263486+0000","last_active":"2024-09-10T23:36:52.267558+0000","last_peered":"2024-09-10T23:36:52.267558+0000","last_clean":"2024-09-10T23:36:52.267558+0000","last_became_active":"2024-09-10T23:36:52.263240+0000","last_became_peered":"2024-09-10T23:36:52.263240+0000","last_unstale":"2024-09-10T23:36:52.267558+0000","last_undegraded":"2024-09-10T23:36:52.267558+0000","last_fullsized":"2024-09-10T23:36:52.267558+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:57.163842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.236204+0000","last_change":"2024-09-10T23:36:51.248214+0000","last_active":"2024-09-10T23:36:52.236204+0000","last_peered":"2024-09-10T23:36:52.236204+0000","last_clean":"2024-09-10T23:36:52.236204+0000","last_became_active":"2024-09-10T23:36:51.247905+0000","last_became_peered":"2024-09-10T23:36:51.247905+0000","last_unstale":"2024-09-10T23:36:52.236204+0000","last_undegraded":"2024-09-10T23:36:52.236204+0000","last_fullsized":"2024-09-10T23:36:52.236204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:11.892931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.247251+0000","last_change":"2024-09-10T23:36:52.242929+0000","last_active":"2024-09-10T23:36:52.247251+0000","last_peered":"2024-09-10T23:36:52.247251+0000","last_clean":"2024-09-10T23:36:52.247251+0000","last_became_active":"2024-09-10T23:36:52.242511+0000","last_became_peered":"2024-09-10T23:36:52.242511+0000","last_unstale":"2024-09-10T23:36:52.247251+0000","last_undegraded":"2024-09-10T23:36:52.247251+0000","last_fullsized":"2024-09-10T23:36:52.247251+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:14.361040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.239699+0000","last_change":"2024-09-10T23:36:51.240180+0000","last_active":"2024-09-10T23:36:52.239699+0000","last_peered":"2024-09-10T23:36:52.239699+0000","last_clean":"2024-09-10T23:36:52.239699+0000","last_became_active":"2024-09-10T23:36:51.239910+0000","last_became_peered":"2024-09-10T23:36:51.239910+0000","last_unstale":"2024-09-10T23:36:52.239699+0000","last_undegraded":"2024-09-10T23:36:52.239699+0000","last_fullsized":"2024-09-10T23:36:52.239699+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:54.155581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.270722+0000","last_change":"2024-09-10T23:36:52.265382+0000","last_active":"2024-09-10T23:36:52.270722+0000","last_peered":"2024-09-10T23:36:52.270722+0000","last_clean":"2024-09-10T23:36:52.270722+0000","last_became_active":"2024-09-10T23:36:52.265086+0000","last_became_peered":"2024-09-10T23:36:52.265086+0000","last_unstale":"2024-09-10T23:36:52.270722+0000","last_undegraded":"2024-09-10T23:36:52.270722+0000","last_fullsized":"2024-09-10T23:36:52.270722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:17.214930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.237184+0000","last_change":"2024-09-10T23:36:51.242468+0000","last_active":"2024-09-10T23:36:52.237184+0000","last_peered":"2024-09-10T23:36:52.237184+0000","last_clean":"2024-09-10T23:36:52.237184+0000","last_became_active":"2024-09-10T23:36:51.242199+0000","last_became_peered":"2024-09-10T23:36:51.242199+0000","last_unstale":"2024-09-10T23:36:52.237184+0000","last_undegraded":"2024-09-10T23:36:52.237184+0000","last_fullsized":"2024-09-10T23:36:52.237184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:55.223076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.247331+0000","last_change":"2024-09-10T23:36:52.242664+0000","last_active":"2024-09-10T23:36:52.247331+0000","last_peered":"2024-09-10T23:36:52.247331+0000","last_clean":"2024-09-10T23:36:52.247331+0000","last_became_active":"2024-09-10T23:36:52.242038+0000","last_became_peered":"2024-09-10T23:36:52.242038+0000","last_unstale":"2024-09-10T23:36:52.247331+0000","last_undegraded":"2024-09-10T23:36:52.247331+0000","last_fullsized":"2024-09-10T23:36:52.247331+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:46.820926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.231673+0000","last_change":"2024-09-10T23:36:51.248559+0000","last_active":"2024-09-10T23:36:52.231673+0000","last_peered":"2024-09-10T23:36:52.231673+0000","last_clean":"2024-09-10T23:36:52.231673+0000","last_became_active":"2024-09-10T23:36:51.248327+0000","last_became_peered":"2024-09-10T23:36:51.248327+0000","last_unstale":"2024-09-10T23:36:52.231673+0000","last_undegraded":"2024-09-10T23:36:52.231673+0000","last_fullsized":"2024-09-10T23:36:52.231673+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:23.705570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.256912+0000","last_change":"2024-09-10T23:36:52.253914+0000","last_active":"2024-09-10T23:36:52.256912+0000","last_peered":"2024-09-10T23:36:52.256912+0000","last_clean":"2024-09-10T23:36:52.256912+0000","last_became_active":"2024-09-10T23:36:52.250835+0000","last_became_peered":"2024-09-10T23:36:52.250835+0000","last_unstale":"2024-09-10T23:36:52.256912+0000","last_undegraded":"2024-09-10T23:36:52.256912+0000","last_fullsized":"2024-09-10T23:36:52.256912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:37.496155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.237873+0000","last_change":"2024-09-10T23:36:51.249119+0000","last_active":"2024-09-10T23:36:52.237873+0000","last_peered":"2024-09-10T23:36:52.237873+0000","last_clean":"2024-09-10T23:36:52.237873+0000","last_became_active":"2024-09-10T23:36:51.248829+0000","last_became_peered":"2024-09-10T23:36:51.248829+0000","last_unstale":"2024-09-10T23:36:52.237873+0000","last_undegraded":"2024-09-10T23:36:52.237873+0000","last_fullsized":"2024-09-10T23:36:52.237873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:54.155581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.263727+0000","last_change":"2024-09-10T23:36:52.261387+0000","last_active":"2024-09-10T23:36:52.263727+0000","last_peered":"2024-09-10T23:36:52.263727+0000","last_clean":"2024-09-10T23:36:52.263727+0000","last_became_active":"2024-09-10T23:36:52.260720+0000","last_became_peered":"2024-09-10T23:36:52.260720+0000","last_unstale":"2024-09-10T23:36:52.263727+0000","last_undegraded":"2024-09-10T23:36:52.263727+0000","last_fullsized":"2024-09-10T23:36:52.263727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:33.594454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.269239+0000","last_change":"2024-09-10T23:36:51.249710+0000","last_active":"2024-09-10T23:36:52.269239+0000","last_peered":"2024-09-10T23:36:52.269239+0000","last_clean":"2024-09-10T23:36:52.269239+0000","last_became_active":"2024-09-10T23:36:51.249452+0000","last_became_peered":"2024-09-10T23:36:51.249452+0000","last_unstale":"2024-09-10T23:36:52.269239+0000","last_undegraded":"2024-09-10T23:36:52.269239+0000","last_fullsized":"2024-09-10T23:36:52.269239+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:42.368336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":244,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.231964+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:36:52.231964+0000","last_peered":"2024-09-10T23:36:52.231964+0000","last_clean":"2024-09-10T23:36:52.231964+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:36:52.231964+0000","last_undegraded":"2024-09-10T23:36:52.231964+0000","last_fullsized":"2024-09-10T23:36:52.231964+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509498,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":962560,"data_stored":406977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542201,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":983040,"data_stored":414424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509497,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1556480,"data_stored":997345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509495,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":962560,"data_stored":406977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509496,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":962560,"data_stored":406977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509497,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":974848,"data_stored":413752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509496,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1572864,"data_stored":1004210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509496,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":962560,"data_stored":406977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:36:56.360 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:36:56.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.538+0000 7f781e400640 1 -- 172.21.15.94:0/3023022385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78180aea80 msgr2=0x7f78180aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:56.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.538+0000 7f781e400640 1 --2- 172.21.15.94:0/3023022385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78180aea80 0x7f78180aee60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7808009870 tx=0x7f780801c7d0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.538+0000 7f781e400640 1 -- 172.21.15.94:0/3023022385 shutdown_connections 2024-09-10T23:36:56.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.538+0000 7f781e400640 1 --2- 172.21.15.94:0/3023022385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78180aea80 0x7f78180aee60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.538+0000 7f781e400640 1 -- 172.21.15.94:0/3023022385 >> 172.21.15.94:0/3023022385 conn(0x7f78180cc900 msgr2=0x7f78180ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:56.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.538+0000 7f781e400640 1 -- 172.21.15.94:0/3023022385 shutdown_connections 2024-09-10T23:36:56.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.538+0000 7f781e400640 1 -- 172.21.15.94:0/3023022385 wait complete. 2024-09-10T23:36:56.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.538+0000 7f781e400640 1 Processor -- start 2024-09-10T23:36:56.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.538+0000 7f781e400640 1 -- start start 2024-09-10T23:36:56.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f781e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78180aea80 0x7f7818155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:56.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f781e400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f781815ccc0 0x7f7818154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:56.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f781e400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f781815d300 0x7f7818072c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:56.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f781e400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7818169f40 con 0x7f78180aea80 2024-09-10T23:36:56.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f781e400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7818169dc0 con 0x7f781815ccc0 2024-09-10T23:36:56.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f781e400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f781816a0c0 con 0x7f781815d300 2024-09-10T23:36:56.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f7817400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78180aea80 0x7f7818155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:56.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f7817400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78180aea80 0x7f7818155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47604/0 (socket says 172.21.15.94:47604) 2024-09-10T23:36:56.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f7817400640 1 -- 172.21.15.94:0/4147833598 learned_addr learned my addr 172.21.15.94:0/4147833598 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:56.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f780fe00640 1 --2- 172.21.15.94:0/4147833598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f781815ccc0 0x7f7818154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:56.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f7817e00640 1 --2- 172.21.15.94:0/4147833598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f781815d300 0x7f7818072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:56.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f7817400640 1 -- 172.21.15.94:0/4147833598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f781815d300 msgr2=0x7f7818072c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:56.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f7817400640 1 --2- 172.21.15.94:0/4147833598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f781815d300 0x7f7818072c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f7817400640 1 -- 172.21.15.94:0/4147833598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f781815ccc0 msgr2=0x7f7818154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:56.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f7817400640 1 --2- 172.21.15.94:0/4147833598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f781815ccc0 0x7f7818154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f7817400640 1 -- 172.21.15.94:0/4147833598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78180c46a0 con 0x7f78180aea80 2024-09-10T23:36:56.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f780fe00640 1 --2- 172.21.15.94:0/4147833598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f781815ccc0 0x7f7818154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:56.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f7817400640 1 --2- 172.21.15.94:0/4147833598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78180aea80 0x7f7818155350 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7808000c10 tx=0x7f78080a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:56.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f780f400640 1 -- 172.21.15.94:0/4147833598 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78080bc070 con 0x7f78180aea80 2024-09-10T23:36:56.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f780f400640 1 -- 172.21.15.94:0/4147833598 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78080043b0 con 0x7f78180aea80 2024-09-10T23:36:56.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f780f400640 1 -- 172.21.15.94:0/4147833598 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78080b2040 con 0x7f78180aea80 2024-09-10T23:36:56.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f781e400640 1 -- 172.21.15.94:0/4147833598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7818155070 con 0x7f78180aea80 2024-09-10T23:36:56.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f781e400640 1 -- 172.21.15.94:0/4147833598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78180c3ea0 con 0x7f78180aea80 2024-09-10T23:36:56.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f781e400640 1 -- 172.21.15.94:0/4147833598 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78180acda0 con 0x7f78180aea80 2024-09-10T23:36:56.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.542+0000 7f780f400640 1 -- 172.21.15.94:0/4147833598 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f78080b5070 con 0x7f78180aea80 2024-09-10T23:36:56.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.546+0000 7f780f400640 1 --2- 172.21.15.94:0/4147833598 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f77f4072700 0x7f77f4074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:56.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.546+0000 7f780f400640 1 -- 172.21.15.94:0/4147833598 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f780812ca80 con 0x7f78180aea80 2024-09-10T23:36:56.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.546+0000 7f780fe00640 1 --2- 172.21.15.94:0/4147833598 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f77f4072700 0x7f77f4074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:56.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.546+0000 7f780fe00640 1 --2- 172.21.15.94:0/4147833598 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f77f4072700 0x7f77f4074bc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f78180b5f50 tx=0x7f7800009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:56.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.550+0000 7f780f400640 1 -- 172.21.15.94:0/4147833598 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7808016450 con 0x7f78180aea80 2024-09-10T23:36:56.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.710+0000 7f781e400640 1 -- 172.21.15.94:0/4147833598 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78180c3a10 con 0x7f77f4072700 2024-09-10T23:36:56.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.714+0000 7f780f400640 1 -- 172.21.15.94:0/4147833598 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252059 (secure 0 0 0) 0x7f78180c3a10 con 0x7f77f4072700 2024-09-10T23:36:56.719 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:56.720 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:36:56.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.718+0000 7f781e400640 1 -- 172.21.15.94:0/4147833598 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f77f4072700 msgr2=0x7f77f4074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:56.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.718+0000 7f781e400640 1 --2- 172.21.15.94:0/4147833598 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f77f4072700 0x7f77f4074bc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f78180b5f50 tx=0x7f7800009040 comp rx=0 tx=0).stop 2024-09-10T23:36:56.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.718+0000 7f781e400640 1 -- 172.21.15.94:0/4147833598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78180aea80 msgr2=0x7f7818155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:56.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.718+0000 7f781e400640 1 --2- 172.21.15.94:0/4147833598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78180aea80 0x7f7818155350 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7808000c10 tx=0x7f78080a55a0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.718+0000 7f781e400640 1 -- 172.21.15.94:0/4147833598 shutdown_connections 2024-09-10T23:36:56.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.718+0000 7f781e400640 1 --2- 172.21.15.94:0/4147833598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f781815ccc0 0x7f7818154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.718+0000 7f781e400640 1 --2- 172.21.15.94:0/4147833598 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f77f4072700 0x7f77f4074bc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.718+0000 7f781e400640 1 --2- 172.21.15.94:0/4147833598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f781815d300 0x7f7818072c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.718+0000 7f781e400640 1 --2- 172.21.15.94:0/4147833598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78180aea80 0x7f7818155350 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.718+0000 7f781e400640 1 -- 172.21.15.94:0/4147833598 >> 172.21.15.94:0/4147833598 conn(0x7f78180cc900 msgr2=0x7f7818159330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:56.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.718+0000 7f781e400640 1 -- 172.21.15.94:0/4147833598 shutdown_connections 2024-09-10T23:36:56.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.718+0000 7f781e400640 1 -- 172.21.15.94:0/4147833598 wait complete. 2024-09-10T23:36:56.739 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-10T23:36:54.738146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258256,"kb_used_data":8728,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716464,"statfs":{"total":773094113280,"available":772829659136,"internally_reserved":0,"allocated":8937472,"data_stored":4457639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.521037"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.248385+0000","last_change":"2024-09-10T23:36:52.243739+0000","last_active":"2024-09-10T23:36:52.248385+0000","last_peered":"2024-09-10T23:36:52.248385+0000","last_clean":"2024-09-10T23:36:52.248385+0000","last_became_active":"2024-09-10T23:36:52.243464+0000","last_became_peered":"2024-09-10T23:36:52.243464+0000","last_unstale":"2024-09-10T23:36:52.248385+0000","last_undegraded":"2024-09-10T23:36:52.248385+0000","last_fullsized":"2024-09-10T23:36:52.248385+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:34.455988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.250949+0000","last_change":"2024-09-10T23:36:52.244795+0000","last_active":"2024-09-10T23:36:52.250949+0000","last_peered":"2024-09-10T23:36:52.250949+0000","last_clean":"2024-09-10T23:36:52.250949+0000","last_became_active":"2024-09-10T23:36:52.244384+0000","last_became_peered":"2024-09-10T23:36:52.244384+0000","last_unstale":"2024-09-10T23:36:52.250949+0000","last_undegraded":"2024-09-10T23:36:52.250949+0000","last_fullsized":"2024-09-10T23:36:52.250949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:41.704048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.266773+0000","last_change":"2024-09-10T23:36:52.263412+0000","last_active":"2024-09-10T23:36:52.266773+0000","last_peered":"2024-09-10T23:36:52.266773+0000","last_clean":"2024-09-10T23:36:52.266773+0000","last_became_active":"2024-09-10T23:36:52.263117+0000","last_became_peered":"2024-09-10T23:36:52.263117+0000","last_unstale":"2024-09-10T23:36:52.266773+0000","last_undegraded":"2024-09-10T23:36:52.266773+0000","last_fullsized":"2024-09-10T23:36:52.266773+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:53:21.829334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.265821+0000","last_change":"2024-09-10T23:36:52.262496+0000","last_active":"2024-09-10T23:36:52.265821+0000","last_peered":"2024-09-10T23:36:52.265821+0000","last_clean":"2024-09-10T23:36:52.265821+0000","last_became_active":"2024-09-10T23:36:52.261927+0000","last_became_peered":"2024-09-10T23:36:52.261927+0000","last_unstale":"2024-09-10T23:36:52.265821+0000","last_undegraded":"2024-09-10T23:36:52.265821+0000","last_fullsized":"2024-09-10T23:36:52.265821+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:52.152410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.269437+0000","last_change":"2024-09-10T23:36:52.265738+0000","last_active":"2024-09-10T23:36:52.269437+0000","last_peered":"2024-09-10T23:36:52.269437+0000","last_clean":"2024-09-10T23:36:52.269437+0000","last_became_active":"2024-09-10T23:36:52.265489+0000","last_became_peered":"2024-09-10T23:36:52.265489+0000","last_unstale":"2024-09-10T23:36:52.269437+0000","last_undegraded":"2024-09-10T23:36:52.269437+0000","last_fullsized":"2024-09-10T23:36:52.269437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:49.404949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.262507+0000","last_change":"2024-09-10T23:36:52.251410+0000","last_active":"2024-09-10T23:36:52.262507+0000","last_peered":"2024-09-10T23:36:52.262507+0000","last_clean":"2024-09-10T23:36:52.262507+0000","last_became_active":"2024-09-10T23:36:52.251157+0000","last_became_peered":"2024-09-10T23:36:52.251157+0000","last_unstale":"2024-09-10T23:36:52.262507+0000","last_undegraded":"2024-09-10T23:36:52.262507+0000","last_fullsized":"2024-09-10T23:36:52.262507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:52.621073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.266487+0000","last_change":"2024-09-10T23:36:52.257597+0000","last_active":"2024-09-10T23:36:52.266487+0000","last_peered":"2024-09-10T23:36:52.266487+0000","last_clean":"2024-09-10T23:36:52.266487+0000","last_became_active":"2024-09-10T23:36:52.257330+0000","last_became_peered":"2024-09-10T23:36:52.257330+0000","last_unstale":"2024-09-10T23:36:52.266487+0000","last_undegraded":"2024-09-10T23:36:52.266487+0000","last_fullsized":"2024-09-10T23:36:52.266487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:13.263641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.250846+0000","last_change":"2024-09-10T23:36:52.248454+0000","last_active":"2024-09-10T23:36:52.250846+0000","last_peered":"2024-09-10T23:36:52.250846+0000","last_clean":"2024-09-10T23:36:52.250846+0000","last_became_active":"2024-09-10T23:36:52.248180+0000","last_became_peered":"2024-09-10T23:36:52.248180+0000","last_unstale":"2024-09-10T23:36:52.250846+0000","last_undegraded":"2024-09-10T23:36:52.250846+0000","last_fullsized":"2024-09-10T23:36:52.250846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:13.263641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.256970+0000","last_change":"2024-09-10T23:36:52.250730+0000","last_active":"2024-09-10T23:36:52.256970+0000","last_peered":"2024-09-10T23:36:52.256970+0000","last_clean":"2024-09-10T23:36:52.256970+0000","last_became_active":"2024-09-10T23:36:52.250159+0000","last_became_peered":"2024-09-10T23:36:52.250159+0000","last_unstale":"2024-09-10T23:36:52.256970+0000","last_undegraded":"2024-09-10T23:36:52.256970+0000","last_fullsized":"2024-09-10T23:36:52.256970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:54.263092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.255082+0000","last_change":"2024-09-10T23:36:52.246249+0000","last_active":"2024-09-10T23:36:52.255082+0000","last_peered":"2024-09-10T23:36:52.255082+0000","last_clean":"2024-09-10T23:36:52.255082+0000","last_became_active":"2024-09-10T23:36:52.245928+0000","last_became_peered":"2024-09-10T23:36:52.245928+0000","last_unstale":"2024-09-10T23:36:52.255082+0000","last_undegraded":"2024-09-10T23:36:52.255082+0000","last_fullsized":"2024-09-10T23:36:52.255082+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:52.799130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.251234+0000","last_change":"2024-09-10T23:36:52.247770+0000","last_active":"2024-09-10T23:36:52.251234+0000","last_peered":"2024-09-10T23:36:52.251234+0000","last_clean":"2024-09-10T23:36:52.251234+0000","last_became_active":"2024-09-10T23:36:52.247492+0000","last_became_peered":"2024-09-10T23:36:52.247492+0000","last_unstale":"2024-09-10T23:36:52.251234+0000","last_undegraded":"2024-09-10T23:36:52.251234+0000","last_fullsized":"2024-09-10T23:36:52.251234+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:55.905682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.268741+0000","last_change":"2024-09-10T23:36:52.266099+0000","last_active":"2024-09-10T23:36:52.268741+0000","last_peered":"2024-09-10T23:36:52.268741+0000","last_clean":"2024-09-10T23:36:52.268741+0000","last_became_active":"2024-09-10T23:36:52.264893+0000","last_became_peered":"2024-09-10T23:36:52.264893+0000","last_unstale":"2024-09-10T23:36:52.268741+0000","last_undegraded":"2024-09-10T23:36:52.268741+0000","last_fullsized":"2024-09-10T23:36:52.268741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:18.346281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.266870+0000","last_change":"2024-09-10T23:36:52.255909+0000","last_active":"2024-09-10T23:36:52.266870+0000","last_peered":"2024-09-10T23:36:52.266870+0000","last_clean":"2024-09-10T23:36:52.266870+0000","last_became_active":"2024-09-10T23:36:52.255633+0000","last_became_peered":"2024-09-10T23:36:52.255633+0000","last_unstale":"2024-09-10T23:36:52.266870+0000","last_undegraded":"2024-09-10T23:36:52.266870+0000","last_fullsized":"2024-09-10T23:36:52.266870+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:43.555918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.271127+0000","last_change":"2024-09-10T23:36:52.267389+0000","last_active":"2024-09-10T23:36:52.271127+0000","last_peered":"2024-09-10T23:36:52.271127+0000","last_clean":"2024-09-10T23:36:52.271127+0000","last_became_active":"2024-09-10T23:36:52.267069+0000","last_became_peered":"2024-09-10T23:36:52.267069+0000","last_unstale":"2024-09-10T23:36:52.271127+0000","last_undegraded":"2024-09-10T23:36:52.271127+0000","last_fullsized":"2024-09-10T23:36:52.271127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:24:45.585197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.254642+0000","last_change":"2024-09-10T23:36:52.250542+0000","last_active":"2024-09-10T23:36:52.254642+0000","last_peered":"2024-09-10T23:36:52.254642+0000","last_clean":"2024-09-10T23:36:52.254642+0000","last_became_active":"2024-09-10T23:36:52.250262+0000","last_became_peered":"2024-09-10T23:36:52.250262+0000","last_unstale":"2024-09-10T23:36:52.254642+0000","last_undegraded":"2024-09-10T23:36:52.254642+0000","last_fullsized":"2024-09-10T23:36:52.254642+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:17.852801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.259453+0000","last_change":"2024-09-10T23:36:52.254567+0000","last_active":"2024-09-10T23:36:52.259453+0000","last_peered":"2024-09-10T23:36:52.259453+0000","last_clean":"2024-09-10T23:36:52.259453+0000","last_became_active":"2024-09-10T23:36:52.253855+0000","last_became_peered":"2024-09-10T23:36:52.253855+0000","last_unstale":"2024-09-10T23:36:52.259453+0000","last_undegraded":"2024-09-10T23:36:52.259453+0000","last_fullsized":"2024-09-10T23:36:52.259453+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:05.948035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.263093+0000","last_change":"2024-09-10T23:36:52.261920+0000","last_active":"2024-09-10T23:36:52.263093+0000","last_peered":"2024-09-10T23:36:52.263093+0000","last_clean":"2024-09-10T23:36:52.263093+0000","last_became_active":"2024-09-10T23:36:52.259123+0000","last_became_peered":"2024-09-10T23:36:52.259123+0000","last_unstale":"2024-09-10T23:36:52.263093+0000","last_undegraded":"2024-09-10T23:36:52.263093+0000","last_fullsized":"2024-09-10T23:36:52.263093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:10.084354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.251668+0000","last_change":"2024-09-10T23:36:52.249342+0000","last_active":"2024-09-10T23:36:52.251668+0000","last_peered":"2024-09-10T23:36:52.251668+0000","last_clean":"2024-09-10T23:36:52.251668+0000","last_became_active":"2024-09-10T23:36:52.249092+0000","last_became_peered":"2024-09-10T23:36:52.249092+0000","last_unstale":"2024-09-10T23:36:52.251668+0000","last_undegraded":"2024-09-10T23:36:52.251668+0000","last_fullsized":"2024-09-10T23:36:52.251668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:10.576351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.271190+0000","last_change":"2024-09-10T23:36:52.267327+0000","last_active":"2024-09-10T23:36:52.271190+0000","last_peered":"2024-09-10T23:36:52.271190+0000","last_clean":"2024-09-10T23:36:52.271190+0000","last_became_active":"2024-09-10T23:36:52.267011+0000","last_became_peered":"2024-09-10T23:36:52.267011+0000","last_unstale":"2024-09-10T23:36:52.271190+0000","last_undegraded":"2024-09-10T23:36:52.271190+0000","last_fullsized":"2024-09-10T23:36:52.271190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:15.868540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.273207+0000","last_change":"2024-09-10T23:36:52.267758+0000","last_active":"2024-09-10T23:36:52.273207+0000","last_peered":"2024-09-10T23:36:52.273207+0000","last_clean":"2024-09-10T23:36:52.273207+0000","last_became_active":"2024-09-10T23:36:52.267322+0000","last_became_peered":"2024-09-10T23:36:52.267322+0000","last_unstale":"2024-09-10T23:36:52.273207+0000","last_undegraded":"2024-09-10T23:36:52.273207+0000","last_fullsized":"2024-09-10T23:36:52.273207+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:51.483384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.263795+0000","last_change":"2024-09-10T23:36:52.252266+0000","last_active":"2024-09-10T23:36:52.263795+0000","last_peered":"2024-09-10T23:36:52.263795+0000","last_clean":"2024-09-10T23:36:52.263795+0000","last_became_active":"2024-09-10T23:36:52.251683+0000","last_became_peered":"2024-09-10T23:36:52.251683+0000","last_unstale":"2024-09-10T23:36:52.263795+0000","last_undegraded":"2024-09-10T23:36:52.263795+0000","last_fullsized":"2024-09-10T23:36:52.263795+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:57.163842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.273346+0000","last_change":"2024-09-10T23:36:52.269242+0000","last_active":"2024-09-10T23:36:52.273346+0000","last_peered":"2024-09-10T23:36:52.273346+0000","last_clean":"2024-09-10T23:36:52.273346+0000","last_became_active":"2024-09-10T23:36:52.268157+0000","last_became_peered":"2024-09-10T23:36:52.268157+0000","last_unstale":"2024-09-10T23:36:52.273346+0000","last_undegraded":"2024-09-10T23:36:52.273346+0000","last_fullsized":"2024-09-10T23:36:52.273346+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:31.967692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.267444+0000","last_change":"2024-09-10T23:36:52.264159+0000","last_active":"2024-09-10T23:36:52.267444+0000","last_peered":"2024-09-10T23:36:52.267444+0000","last_clean":"2024-09-10T23:36:52.267444+0000","last_became_active":"2024-09-10T23:36:52.263842+0000","last_became_peered":"2024-09-10T23:36:52.263842+0000","last_unstale":"2024-09-10T23:36:52.267444+0000","last_undegraded":"2024-09-10T23:36:52.267444+0000","last_fullsized":"2024-09-10T23:36:52.267444+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:19.173697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.264554+0000","last_change":"2024-09-10T23:36:52.261763+0000","last_active":"2024-09-10T23:36:52.264554+0000","last_peered":"2024-09-10T23:36:52.264554+0000","last_clean":"2024-09-10T23:36:52.264554+0000","last_became_active":"2024-09-10T23:36:52.261500+0000","last_became_peered":"2024-09-10T23:36:52.261500+0000","last_unstale":"2024-09-10T23:36:52.264554+0000","last_undegraded":"2024-09-10T23:36:52.264554+0000","last_fullsized":"2024-09-10T23:36:52.264554+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:18.401283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.270833+0000","last_change":"2024-09-10T23:36:52.265930+0000","last_active":"2024-09-10T23:36:52.270833+0000","last_peered":"2024-09-10T23:36:52.270833+0000","last_clean":"2024-09-10T23:36:52.270833+0000","last_became_active":"2024-09-10T23:36:52.265541+0000","last_became_peered":"2024-09-10T23:36:52.265541+0000","last_unstale":"2024-09-10T23:36:52.270833+0000","last_undegraded":"2024-09-10T23:36:52.270833+0000","last_fullsized":"2024-09-10T23:36:52.270833+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:24.782214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.268573+0000","last_change":"2024-09-10T23:36:52.256559+0000","last_active":"2024-09-10T23:36:52.268573+0000","last_peered":"2024-09-10T23:36:52.268573+0000","last_clean":"2024-09-10T23:36:52.268573+0000","last_became_active":"2024-09-10T23:36:52.256321+0000","last_became_peered":"2024-09-10T23:36:52.256321+0000","last_unstale":"2024-09-10T23:36:52.268573+0000","last_undegraded":"2024-09-10T23:36:52.268573+0000","last_fullsized":"2024-09-10T23:36:52.268573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:33.966515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.269119+0000","last_change":"2024-09-10T23:36:52.261074+0000","last_active":"2024-09-10T23:36:52.269119+0000","last_peered":"2024-09-10T23:36:52.269119+0000","last_clean":"2024-09-10T23:36:52.269119+0000","last_became_active":"2024-09-10T23:36:52.254194+0000","last_became_peered":"2024-09-10T23:36:52.254194+0000","last_unstale":"2024-09-10T23:36:52.269119+0000","last_undegraded":"2024-09-10T23:36:52.269119+0000","last_fullsized":"2024-09-10T23:36:52.269119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:31.839877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.270581+0000","last_change":"2024-09-10T23:36:52.269141+0000","last_active":"2024-09-10T23:36:52.270581+0000","last_peered":"2024-09-10T23:36:52.270581+0000","last_clean":"2024-09-10T23:36:52.270581+0000","last_became_active":"2024-09-10T23:36:52.268857+0000","last_became_peered":"2024-09-10T23:36:52.268857+0000","last_unstale":"2024-09-10T23:36:52.270581+0000","last_undegraded":"2024-09-10T23:36:52.270581+0000","last_fullsized":"2024-09-10T23:36:52.270581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:56.174167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.263771+0000","last_change":"2024-09-10T23:36:52.251948+0000","last_active":"2024-09-10T23:36:52.263771+0000","last_peered":"2024-09-10T23:36:52.263771+0000","last_clean":"2024-09-10T23:36:52.263771+0000","last_became_active":"2024-09-10T23:36:52.251371+0000","last_became_peered":"2024-09-10T23:36:52.251371+0000","last_unstale":"2024-09-10T23:36:52.263771+0000","last_undegraded":"2024-09-10T23:36:52.263771+0000","last_fullsized":"2024-09-10T23:36:52.263771+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:57.457285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.273242+0000","last_change":"2024-09-10T23:36:52.268185+0000","last_active":"2024-09-10T23:36:52.273242+0000","last_peered":"2024-09-10T23:36:52.273242+0000","last_clean":"2024-09-10T23:36:52.273242+0000","last_became_active":"2024-09-10T23:36:52.267670+0000","last_became_peered":"2024-09-10T23:36:52.267670+0000","last_unstale":"2024-09-10T23:36:52.273242+0000","last_undegraded":"2024-09-10T23:36:52.273242+0000","last_fullsized":"2024-09-10T23:36:52.273242+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:25:05.695698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.260746+0000","last_change":"2024-09-10T23:36:52.253057+0000","last_active":"2024-09-10T23:36:52.260746+0000","last_peered":"2024-09-10T23:36:52.260746+0000","last_clean":"2024-09-10T23:36:52.260746+0000","last_became_active":"2024-09-10T23:36:52.252820+0000","last_became_peered":"2024-09-10T23:36:52.252820+0000","last_unstale":"2024-09-10T23:36:52.260746+0000","last_undegraded":"2024-09-10T23:36:52.260746+0000","last_fullsized":"2024-09-10T23:36:52.260746+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:34.147787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.259426+0000","last_change":"2024-09-10T23:36:52.252449+0000","last_active":"2024-09-10T23:36:52.259426+0000","last_peered":"2024-09-10T23:36:52.259426+0000","last_clean":"2024-09-10T23:36:52.259426+0000","last_became_active":"2024-09-10T23:36:52.252229+0000","last_became_peered":"2024-09-10T23:36:52.252229+0000","last_unstale":"2024-09-10T23:36:52.259426+0000","last_undegraded":"2024-09-10T23:36:52.259426+0000","last_fullsized":"2024-09-10T23:36:52.259426+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:10.111868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.265618+0000","last_change":"2024-09-10T23:36:52.262019+0000","last_active":"2024-09-10T23:36:52.265618+0000","last_peered":"2024-09-10T23:36:52.265618+0000","last_clean":"2024-09-10T23:36:52.265618+0000","last_became_active":"2024-09-10T23:36:52.261761+0000","last_became_peered":"2024-09-10T23:36:52.261761+0000","last_unstale":"2024-09-10T23:36:52.265618+0000","last_undegraded":"2024-09-10T23:36:52.265618+0000","last_fullsized":"2024-09-10T23:36:52.265618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:37.753986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"80'4","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.271559+0000","last_change":"2024-09-10T23:36:51.250986+0000","last_active":"2024-09-10T23:36:52.271559+0000","last_peered":"2024-09-10T23:36:52.271559+0000","last_clean":"2024-09-10T23:36:52.271559+0000","last_became_active":"2024-09-10T23:36:51.249834+0000","last_became_peered":"2024-09-10T23:36:51.249834+0000","last_unstale":"2024-09-10T23:36:52.271559+0000","last_undegraded":"2024-09-10T23:36:52.271559+0000","last_fullsized":"2024-09-10T23:36:52.271559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:12.153127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.255473+0000","last_change":"2024-09-10T23:36:52.248789+0000","last_active":"2024-09-10T23:36:52.255473+0000","last_peered":"2024-09-10T23:36:52.255473+0000","last_clean":"2024-09-10T23:36:52.255473+0000","last_became_active":"2024-09-10T23:36:52.248202+0000","last_became_peered":"2024-09-10T23:36:52.248202+0000","last_unstale":"2024-09-10T23:36:52.255473+0000","last_undegraded":"2024-09-10T23:36:52.255473+0000","last_fullsized":"2024-09-10T23:36:52.255473+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:15.602323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.266192+0000","last_change":"2024-09-10T23:36:51.249624+0000","last_active":"2024-09-10T23:36:52.266192+0000","last_peered":"2024-09-10T23:36:52.266192+0000","last_clean":"2024-09-10T23:36:52.266192+0000","last_became_active":"2024-09-10T23:36:51.249303+0000","last_became_peered":"2024-09-10T23:36:51.249303+0000","last_unstale":"2024-09-10T23:36:52.266192+0000","last_undegraded":"2024-09-10T23:36:52.266192+0000","last_fullsized":"2024-09-10T23:36:52.266192+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:11.778368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.257114+0000","last_change":"2024-09-10T23:36:52.254194+0000","last_active":"2024-09-10T23:36:52.257114+0000","last_peered":"2024-09-10T23:36:52.257114+0000","last_clean":"2024-09-10T23:36:52.257114+0000","last_became_active":"2024-09-10T23:36:52.248701+0000","last_became_peered":"2024-09-10T23:36:52.248701+0000","last_unstale":"2024-09-10T23:36:52.257114+0000","last_undegraded":"2024-09-10T23:36:52.257114+0000","last_fullsized":"2024-09-10T23:36:52.257114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:15:13.644272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.271514+0000","last_change":"2024-09-10T23:36:51.249893+0000","last_active":"2024-09-10T23:36:52.271514+0000","last_peered":"2024-09-10T23:36:52.271514+0000","last_clean":"2024-09-10T23:36:52.271514+0000","last_became_active":"2024-09-10T23:36:51.249618+0000","last_became_peered":"2024-09-10T23:36:51.249618+0000","last_unstale":"2024-09-10T23:36:52.271514+0000","last_undegraded":"2024-09-10T23:36:52.271514+0000","last_fullsized":"2024-09-10T23:36:52.271514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:15.221785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.256894+0000","last_change":"2024-09-10T23:36:52.252748+0000","last_active":"2024-09-10T23:36:52.256894+0000","last_peered":"2024-09-10T23:36:52.256894+0000","last_clean":"2024-09-10T23:36:52.256894+0000","last_became_active":"2024-09-10T23:36:52.249665+0000","last_became_peered":"2024-09-10T23:36:52.249665+0000","last_unstale":"2024-09-10T23:36:52.256894+0000","last_undegraded":"2024-09-10T23:36:52.256894+0000","last_fullsized":"2024-09-10T23:36:52.256894+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:22.543318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.270778+0000","last_change":"2024-09-10T23:36:51.251958+0000","last_active":"2024-09-10T23:36:52.270778+0000","last_peered":"2024-09-10T23:36:52.270778+0000","last_clean":"2024-09-10T23:36:52.270778+0000","last_became_active":"2024-09-10T23:36:51.251386+0000","last_became_peered":"2024-09-10T23:36:51.251386+0000","last_unstale":"2024-09-10T23:36:52.270778+0000","last_undegraded":"2024-09-10T23:36:52.270778+0000","last_fullsized":"2024-09-10T23:36:52.270778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:53:20.826875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.265091+0000","last_change":"2024-09-10T23:36:52.250800+0000","last_active":"2024-09-10T23:36:52.265091+0000","last_peered":"2024-09-10T23:36:52.265091+0000","last_clean":"2024-09-10T23:36:52.265091+0000","last_became_active":"2024-09-10T23:36:52.250476+0000","last_became_peered":"2024-09-10T23:36:52.250476+0000","last_unstale":"2024-09-10T23:36:52.265091+0000","last_undegraded":"2024-09-10T23:36:52.265091+0000","last_fullsized":"2024-09-10T23:36:52.265091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:16:47.505665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.259994+0000","last_change":"2024-09-10T23:36:51.254300+0000","last_active":"2024-09-10T23:36:52.259994+0000","last_peered":"2024-09-10T23:36:52.259994+0000","last_clean":"2024-09-10T23:36:52.259994+0000","last_became_active":"2024-09-10T23:36:51.254006+0000","last_became_peered":"2024-09-10T23:36:51.254006+0000","last_unstale":"2024-09-10T23:36:52.259994+0000","last_undegraded":"2024-09-10T23:36:52.259994+0000","last_fullsized":"2024-09-10T23:36:52.259994+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:27.522259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.266133+0000","last_change":"2024-09-10T23:36:52.264649+0000","last_active":"2024-09-10T23:36:52.266133+0000","last_peered":"2024-09-10T23:36:52.266133+0000","last_clean":"2024-09-10T23:36:52.266133+0000","last_became_active":"2024-09-10T23:36:52.264217+0000","last_became_peered":"2024-09-10T23:36:52.264217+0000","last_unstale":"2024-09-10T23:36:52.266133+0000","last_undegraded":"2024-09-10T23:36:52.266133+0000","last_fullsized":"2024-09-10T23:36:52.266133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:57.457285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.270468+0000","last_change":"2024-09-10T23:36:51.246870+0000","last_active":"2024-09-10T23:36:52.270468+0000","last_peered":"2024-09-10T23:36:52.270468+0000","last_clean":"2024-09-10T23:36:52.270468+0000","last_became_active":"2024-09-10T23:36:51.246555+0000","last_became_peered":"2024-09-10T23:36:51.246555+0000","last_unstale":"2024-09-10T23:36:52.270468+0000","last_undegraded":"2024-09-10T23:36:52.270468+0000","last_fullsized":"2024-09-10T23:36:52.270468+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:49:08.273176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.256829+0000","last_change":"2024-09-10T23:36:52.252083+0000","last_active":"2024-09-10T23:36:52.256829+0000","last_peered":"2024-09-10T23:36:52.256829+0000","last_clean":"2024-09-10T23:36:52.256829+0000","last_became_active":"2024-09-10T23:36:52.249587+0000","last_became_peered":"2024-09-10T23:36:52.249587+0000","last_unstale":"2024-09-10T23:36:52.256829+0000","last_undegraded":"2024-09-10T23:36:52.256829+0000","last_fullsized":"2024-09-10T23:36:52.256829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:30.281670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.255794+0000","last_change":"2024-09-10T23:36:51.249469+0000","last_active":"2024-09-10T23:36:52.255794+0000","last_peered":"2024-09-10T23:36:52.255794+0000","last_clean":"2024-09-10T23:36:52.255794+0000","last_became_active":"2024-09-10T23:36:51.249165+0000","last_became_peered":"2024-09-10T23:36:51.249165+0000","last_unstale":"2024-09-10T23:36:52.255794+0000","last_undegraded":"2024-09-10T23:36:52.255794+0000","last_fullsized":"2024-09-10T23:36:52.255794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:49:02.808530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.273437+0000","last_change":"2024-09-10T23:36:52.269981+0000","last_active":"2024-09-10T23:36:52.273437+0000","last_peered":"2024-09-10T23:36:52.273437+0000","last_clean":"2024-09-10T23:36:52.273437+0000","last_became_active":"2024-09-10T23:36:52.269690+0000","last_became_peered":"2024-09-10T23:36:52.269690+0000","last_unstale":"2024-09-10T23:36:52.273437+0000","last_undegraded":"2024-09-10T23:36:52.273437+0000","last_fullsized":"2024-09-10T23:36:52.273437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:59.619749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.254433+0000","last_change":"2024-09-10T23:36:51.252955+0000","last_active":"2024-09-10T23:36:52.254433+0000","last_peered":"2024-09-10T23:36:52.254433+0000","last_clean":"2024-09-10T23:36:52.254433+0000","last_became_active":"2024-09-10T23:36:51.252649+0000","last_became_peered":"2024-09-10T23:36:51.252649+0000","last_unstale":"2024-09-10T23:36:52.254433+0000","last_undegraded":"2024-09-10T23:36:52.254433+0000","last_fullsized":"2024-09-10T23:36:52.254433+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:55.232022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.265034+0000","last_change":"2024-09-10T23:36:52.260178+0000","last_active":"2024-09-10T23:36:52.265034+0000","last_peered":"2024-09-10T23:36:52.265034+0000","last_clean":"2024-09-10T23:36:52.265034+0000","last_became_active":"2024-09-10T23:36:52.259849+0000","last_became_peered":"2024-09-10T23:36:52.259849+0000","last_unstale":"2024-09-10T23:36:52.265034+0000","last_undegraded":"2024-09-10T23:36:52.265034+0000","last_fullsized":"2024-09-10T23:36:52.265034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:18.664368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.233179+0000","last_change":"2024-09-10T23:36:51.251994+0000","last_active":"2024-09-10T23:36:52.233179+0000","last_peered":"2024-09-10T23:36:52.233179+0000","last_clean":"2024-09-10T23:36:52.233179+0000","last_became_active":"2024-09-10T23:36:51.251676+0000","last_became_peered":"2024-09-10T23:36:51.251676+0000","last_unstale":"2024-09-10T23:36:52.233179+0000","last_undegraded":"2024-09-10T23:36:52.233179+0000","last_fullsized":"2024-09-10T23:36:52.233179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:13.300624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.256950+0000","last_change":"2024-09-10T23:36:52.250424+0000","last_active":"2024-09-10T23:36:52.256950+0000","last_peered":"2024-09-10T23:36:52.256950+0000","last_clean":"2024-09-10T23:36:52.256950+0000","last_became_active":"2024-09-10T23:36:52.249829+0000","last_became_peered":"2024-09-10T23:36:52.249829+0000","last_unstale":"2024-09-10T23:36:52.256950+0000","last_undegraded":"2024-09-10T23:36:52.256950+0000","last_fullsized":"2024-09-10T23:36:52.256950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:32.885162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.237993+0000","last_change":"2024-09-10T23:36:51.238852+0000","last_active":"2024-09-10T23:36:52.237993+0000","last_peered":"2024-09-10T23:36:52.237993+0000","last_clean":"2024-09-10T23:36:52.237993+0000","last_became_active":"2024-09-10T23:36:51.238599+0000","last_became_peered":"2024-09-10T23:36:51.238599+0000","last_unstale":"2024-09-10T23:36:52.237993+0000","last_undegraded":"2024-09-10T23:36:52.237993+0000","last_fullsized":"2024-09-10T23:36:52.237993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:08.162686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.263755+0000","last_change":"2024-09-10T23:36:52.261680+0000","last_active":"2024-09-10T23:36:52.263755+0000","last_peered":"2024-09-10T23:36:52.263755+0000","last_clean":"2024-09-10T23:36:52.263755+0000","last_became_active":"2024-09-10T23:36:52.261123+0000","last_became_peered":"2024-09-10T23:36:52.261123+0000","last_unstale":"2024-09-10T23:36:52.263755+0000","last_undegraded":"2024-09-10T23:36:52.263755+0000","last_fullsized":"2024-09-10T23:36:52.263755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:59.321006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"80'3","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.268423+0000","last_change":"2024-09-10T23:36:51.253740+0000","last_active":"2024-09-10T23:36:52.268423+0000","last_peered":"2024-09-10T23:36:52.268423+0000","last_clean":"2024-09-10T23:36:52.268423+0000","last_became_active":"2024-09-10T23:36:51.253049+0000","last_became_peered":"2024-09-10T23:36:51.253049+0000","last_unstale":"2024-09-10T23:36:52.268423+0000","last_undegraded":"2024-09-10T23:36:52.268423+0000","last_fullsized":"2024-09-10T23:36:52.268423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:29.077715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.269143+0000","last_change":"2024-09-10T23:36:52.261610+0000","last_active":"2024-09-10T23:36:52.269143+0000","last_peered":"2024-09-10T23:36:52.269143+0000","last_clean":"2024-09-10T23:36:52.269143+0000","last_became_active":"2024-09-10T23:36:52.254487+0000","last_became_peered":"2024-09-10T23:36:52.254487+0000","last_unstale":"2024-09-10T23:36:52.269143+0000","last_undegraded":"2024-09-10T23:36:52.269143+0000","last_fullsized":"2024-09-10T23:36:52.269143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:26.511936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"80'6","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.273844+0000","last_change":"2024-09-10T23:36:51.245606+0000","last_active":"2024-09-10T23:36:52.273844+0000","last_peered":"2024-09-10T23:36:52.273844+0000","last_clean":"2024-09-10T23:36:52.273844+0000","last_became_active":"2024-09-10T23:36:51.245347+0000","last_became_peered":"2024-09-10T23:36:51.245347+0000","last_unstale":"2024-09-10T23:36:52.273844+0000","last_undegraded":"2024-09-10T23:36:52.273844+0000","last_fullsized":"2024-09-10T23:36:52.273844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:57.414029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.258931+0000","last_change":"2024-09-10T23:36:52.253082+0000","last_active":"2024-09-10T23:36:52.258931+0000","last_peered":"2024-09-10T23:36:52.258931+0000","last_clean":"2024-09-10T23:36:52.258931+0000","last_became_active":"2024-09-10T23:36:52.252807+0000","last_became_peered":"2024-09-10T23:36:52.252807+0000","last_unstale":"2024-09-10T23:36:52.258931+0000","last_undegraded":"2024-09-10T23:36:52.258931+0000","last_fullsized":"2024-09-10T23:36:52.258931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:03.400629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.267308+0000","last_change":"2024-09-10T23:36:51.247636+0000","last_active":"2024-09-10T23:36:52.267308+0000","last_peered":"2024-09-10T23:36:52.267308+0000","last_clean":"2024-09-10T23:36:52.267308+0000","last_became_active":"2024-09-10T23:36:51.247240+0000","last_became_peered":"2024-09-10T23:36:51.247240+0000","last_unstale":"2024-09-10T23:36:52.267308+0000","last_undegraded":"2024-09-10T23:36:52.267308+0000","last_fullsized":"2024-09-10T23:36:52.267308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:51.618614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.256682+0000","last_change":"2024-09-10T23:36:52.250394+0000","last_active":"2024-09-10T23:36:52.256682+0000","last_peered":"2024-09-10T23:36:52.256682+0000","last_clean":"2024-09-10T23:36:52.256682+0000","last_became_active":"2024-09-10T23:36:52.250128+0000","last_became_peered":"2024-09-10T23:36:52.250128+0000","last_unstale":"2024-09-10T23:36:52.256682+0000","last_undegraded":"2024-09-10T23:36:52.256682+0000","last_fullsized":"2024-09-10T23:36:52.256682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:28:10.501354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.240255+0000","last_change":"2024-09-10T23:36:51.238946+0000","last_active":"2024-09-10T23:36:52.240255+0000","last_peered":"2024-09-10T23:36:52.240255+0000","last_clean":"2024-09-10T23:36:52.240255+0000","last_became_active":"2024-09-10T23:36:51.238554+0000","last_became_peered":"2024-09-10T23:36:51.238554+0000","last_unstale":"2024-09-10T23:36:52.240255+0000","last_undegraded":"2024-09-10T23:36:52.240255+0000","last_fullsized":"2024-09-10T23:36:52.240255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:27.522259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.273457+0000","last_change":"2024-09-10T23:36:52.269710+0000","last_active":"2024-09-10T23:36:52.273457+0000","last_peered":"2024-09-10T23:36:52.273457+0000","last_clean":"2024-09-10T23:36:52.273457+0000","last_became_active":"2024-09-10T23:36:52.269444+0000","last_became_peered":"2024-09-10T23:36:52.269444+0000","last_unstale":"2024-09-10T23:36:52.273457+0000","last_undegraded":"2024-09-10T23:36:52.273457+0000","last_fullsized":"2024-09-10T23:36:52.273457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:05.738262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.237695+0000","last_change":"2024-09-10T23:36:51.249715+0000","last_active":"2024-09-10T23:36:52.237695+0000","last_peered":"2024-09-10T23:36:52.237695+0000","last_clean":"2024-09-10T23:36:52.237695+0000","last_became_active":"2024-09-10T23:36:51.249457+0000","last_became_peered":"2024-09-10T23:36:51.249457+0000","last_unstale":"2024-09-10T23:36:52.237695+0000","last_undegraded":"2024-09-10T23:36:52.237695+0000","last_fullsized":"2024-09-10T23:36:52.237695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:23:25.253315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.257537+0000","last_change":"2024-09-10T23:36:52.252699+0000","last_active":"2024-09-10T23:36:52.257537+0000","last_peered":"2024-09-10T23:36:52.257537+0000","last_clean":"2024-09-10T23:36:52.257537+0000","last_became_active":"2024-09-10T23:36:52.252399+0000","last_became_peered":"2024-09-10T23:36:52.252399+0000","last_unstale":"2024-09-10T23:36:52.257537+0000","last_undegraded":"2024-09-10T23:36:52.257537+0000","last_fullsized":"2024-09-10T23:36:52.257537+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:03.704223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"80'8","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.271656+0000","last_change":"2024-09-10T23:36:51.250076+0000","last_active":"2024-09-10T23:36:52.271656+0000","last_peered":"2024-09-10T23:36:52.271656+0000","last_clean":"2024-09-10T23:36:52.271656+0000","last_became_active":"2024-09-10T23:36:51.249800+0000","last_became_peered":"2024-09-10T23:36:51.249800+0000","last_unstale":"2024-09-10T23:36:52.271656+0000","last_undegraded":"2024-09-10T23:36:52.271656+0000","last_fullsized":"2024-09-10T23:36:52.271656+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:31.580397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.248813+0000","last_change":"2024-09-10T23:36:52.243894+0000","last_active":"2024-09-10T23:36:52.248813+0000","last_peered":"2024-09-10T23:36:52.248813+0000","last_clean":"2024-09-10T23:36:52.248813+0000","last_became_active":"2024-09-10T23:36:52.243639+0000","last_became_peered":"2024-09-10T23:36:52.243639+0000","last_unstale":"2024-09-10T23:36:52.248813+0000","last_undegraded":"2024-09-10T23:36:52.248813+0000","last_fullsized":"2024-09-10T23:36:52.248813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:24.282604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.268290+0000","last_change":"2024-09-10T23:36:51.250367+0000","last_active":"2024-09-10T23:36:52.268290+0000","last_peered":"2024-09-10T23:36:52.268290+0000","last_clean":"2024-09-10T23:36:52.268290+0000","last_became_active":"2024-09-10T23:36:51.250069+0000","last_became_peered":"2024-09-10T23:36:51.250069+0000","last_unstale":"2024-09-10T23:36:52.268290+0000","last_undegraded":"2024-09-10T23:36:52.268290+0000","last_fullsized":"2024-09-10T23:36:52.268290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:08.014677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.266094+0000","last_change":"2024-09-10T23:36:52.264246+0000","last_active":"2024-09-10T23:36:52.266094+0000","last_peered":"2024-09-10T23:36:52.266094+0000","last_clean":"2024-09-10T23:36:52.266094+0000","last_became_active":"2024-09-10T23:36:52.263999+0000","last_became_peered":"2024-09-10T23:36:52.263999+0000","last_unstale":"2024-09-10T23:36:52.266094+0000","last_undegraded":"2024-09-10T23:36:52.266094+0000","last_fullsized":"2024-09-10T23:36:52.266094+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:16:47.505665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.269770+0000","last_change":"2024-09-10T23:36:51.252590+0000","last_active":"2024-09-10T23:36:52.269770+0000","last_peered":"2024-09-10T23:36:52.269770+0000","last_clean":"2024-09-10T23:36:52.269770+0000","last_became_active":"2024-09-10T23:36:51.252332+0000","last_became_peered":"2024-09-10T23:36:51.252332+0000","last_unstale":"2024-09-10T23:36:52.269770+0000","last_undegraded":"2024-09-10T23:36:52.269770+0000","last_fullsized":"2024-09-10T23:36:52.269770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:54:02.579561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.247405+0000","last_change":"2024-09-10T23:36:52.242754+0000","last_active":"2024-09-10T23:36:52.247405+0000","last_peered":"2024-09-10T23:36:52.247405+0000","last_clean":"2024-09-10T23:36:52.247405+0000","last_became_active":"2024-09-10T23:36:52.242484+0000","last_became_peered":"2024-09-10T23:36:52.242484+0000","last_unstale":"2024-09-10T23:36:52.247405+0000","last_undegraded":"2024-09-10T23:36:52.247405+0000","last_fullsized":"2024-09-10T23:36:52.247405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:17.852801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.253746+0000","last_change":"2024-09-10T23:36:51.251584+0000","last_active":"2024-09-10T23:36:52.253746+0000","last_peered":"2024-09-10T23:36:52.253746+0000","last_clean":"2024-09-10T23:36:52.253746+0000","last_became_active":"2024-09-10T23:36:51.251286+0000","last_became_peered":"2024-09-10T23:36:51.251286+0000","last_unstale":"2024-09-10T23:36:52.253746+0000","last_undegraded":"2024-09-10T23:36:52.253746+0000","last_fullsized":"2024-09-10T23:36:52.253746+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:43.539868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.254603+0000","last_change":"2024-09-10T23:36:52.250582+0000","last_active":"2024-09-10T23:36:52.254603+0000","last_peered":"2024-09-10T23:36:52.254603+0000","last_clean":"2024-09-10T23:36:52.254603+0000","last_became_active":"2024-09-10T23:36:52.250316+0000","last_became_peered":"2024-09-10T23:36:52.250316+0000","last_unstale":"2024-09-10T23:36:52.254603+0000","last_undegraded":"2024-09-10T23:36:52.254603+0000","last_fullsized":"2024-09-10T23:36:52.254603+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:24.282604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.268231+0000","last_change":"2024-09-10T23:36:51.254512+0000","last_active":"2024-09-10T23:36:52.268231+0000","last_peered":"2024-09-10T23:36:52.268231+0000","last_clean":"2024-09-10T23:36:52.268231+0000","last_became_active":"2024-09-10T23:36:51.254202+0000","last_became_peered":"2024-09-10T23:36:51.254202+0000","last_unstale":"2024-09-10T23:36:52.268231+0000","last_undegraded":"2024-09-10T23:36:52.268231+0000","last_fullsized":"2024-09-10T23:36:52.268231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:10.349358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.255814+0000","last_change":"2024-09-10T23:36:52.250037+0000","last_active":"2024-09-10T23:36:52.255814+0000","last_peered":"2024-09-10T23:36:52.255814+0000","last_clean":"2024-09-10T23:36:52.255814+0000","last_became_active":"2024-09-10T23:36:52.249754+0000","last_became_peered":"2024-09-10T23:36:52.249754+0000","last_unstale":"2024-09-10T23:36:52.255814+0000","last_undegraded":"2024-09-10T23:36:52.255814+0000","last_fullsized":"2024-09-10T23:36:52.255814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:20.586094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.239791+0000","last_change":"2024-09-10T23:36:51.243419+0000","last_active":"2024-09-10T23:36:52.239791+0000","last_peered":"2024-09-10T23:36:52.239791+0000","last_clean":"2024-09-10T23:36:52.239791+0000","last_became_active":"2024-09-10T23:36:51.242377+0000","last_became_peered":"2024-09-10T23:36:51.242377+0000","last_unstale":"2024-09-10T23:36:52.239791+0000","last_undegraded":"2024-09-10T23:36:52.239791+0000","last_fullsized":"2024-09-10T23:36:52.239791+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:57.414029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.265703+0000","last_change":"2024-09-10T23:36:52.258113+0000","last_active":"2024-09-10T23:36:52.265703+0000","last_peered":"2024-09-10T23:36:52.265703+0000","last_clean":"2024-09-10T23:36:52.265703+0000","last_became_active":"2024-09-10T23:36:52.257848+0000","last_became_peered":"2024-09-10T23:36:52.257848+0000","last_unstale":"2024-09-10T23:36:52.265703+0000","last_undegraded":"2024-09-10T23:36:52.265703+0000","last_fullsized":"2024-09-10T23:36:52.265703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:30.795006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.243829+0000","last_change":"2024-09-10T23:36:51.247599+0000","last_active":"2024-09-10T23:36:52.243829+0000","last_peered":"2024-09-10T23:36:52.243829+0000","last_clean":"2024-09-10T23:36:52.243829+0000","last_became_active":"2024-09-10T23:36:51.246845+0000","last_became_peered":"2024-09-10T23:36:51.246845+0000","last_unstale":"2024-09-10T23:36:52.243829+0000","last_undegraded":"2024-09-10T23:36:52.243829+0000","last_fullsized":"2024-09-10T23:36:52.243829+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:56.443755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.250235+0000","last_change":"2024-09-10T23:36:52.249537+0000","last_active":"2024-09-10T23:36:52.250235+0000","last_peered":"2024-09-10T23:36:52.250235+0000","last_clean":"2024-09-10T23:36:52.250235+0000","last_became_active":"2024-09-10T23:36:52.249293+0000","last_became_peered":"2024-09-10T23:36:52.249293+0000","last_unstale":"2024-09-10T23:36:52.250235+0000","last_undegraded":"2024-09-10T23:36:52.250235+0000","last_fullsized":"2024-09-10T23:36:52.250235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:27.984279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.254642+0000","last_change":"2024-09-10T23:36:51.251718+0000","last_active":"2024-09-10T23:36:52.254642+0000","last_peered":"2024-09-10T23:36:52.254642+0000","last_clean":"2024-09-10T23:36:52.254642+0000","last_became_active":"2024-09-10T23:36:51.251260+0000","last_became_peered":"2024-09-10T23:36:51.251260+0000","last_unstale":"2024-09-10T23:36:52.254642+0000","last_undegraded":"2024-09-10T23:36:52.254642+0000","last_fullsized":"2024-09-10T23:36:52.254642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:10.354321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.265982+0000","last_change":"2024-09-10T23:36:52.255169+0000","last_active":"2024-09-10T23:36:52.265982+0000","last_peered":"2024-09-10T23:36:52.265982+0000","last_clean":"2024-09-10T23:36:52.265982+0000","last_became_active":"2024-09-10T23:36:52.254812+0000","last_became_peered":"2024-09-10T23:36:52.254812+0000","last_unstale":"2024-09-10T23:36:52.265982+0000","last_undegraded":"2024-09-10T23:36:52.265982+0000","last_fullsized":"2024-09-10T23:36:52.265982+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:16:32.579522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"80'2","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.267131+0000","last_change":"2024-09-10T23:36:51.254207+0000","last_active":"2024-09-10T23:36:52.267131+0000","last_peered":"2024-09-10T23:36:52.267131+0000","last_clean":"2024-09-10T23:36:52.267131+0000","last_became_active":"2024-09-10T23:36:51.253877+0000","last_became_peered":"2024-09-10T23:36:51.253877+0000","last_unstale":"2024-09-10T23:36:52.267131+0000","last_undegraded":"2024-09-10T23:36:52.267131+0000","last_fullsized":"2024-09-10T23:36:52.267131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:51.964881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.261304+0000","last_change":"2024-09-10T23:36:52.254595+0000","last_active":"2024-09-10T23:36:52.261304+0000","last_peered":"2024-09-10T23:36:52.261304+0000","last_clean":"2024-09-10T23:36:52.261304+0000","last_became_active":"2024-09-10T23:36:52.254290+0000","last_became_peered":"2024-09-10T23:36:52.254290+0000","last_unstale":"2024-09-10T23:36:52.261304+0000","last_undegraded":"2024-09-10T23:36:52.261304+0000","last_fullsized":"2024-09-10T23:36:52.261304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:34:20.035632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.242692+0000","last_change":"2024-09-10T23:36:51.249202+0000","last_active":"2024-09-10T23:36:52.242692+0000","last_peered":"2024-09-10T23:36:52.242692+0000","last_clean":"2024-09-10T23:36:52.242692+0000","last_became_active":"2024-09-10T23:36:51.248905+0000","last_became_peered":"2024-09-10T23:36:51.248905+0000","last_unstale":"2024-09-10T23:36:52.242692+0000","last_undegraded":"2024-09-10T23:36:52.242692+0000","last_fullsized":"2024-09-10T23:36:52.242692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:30.899676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.267588+0000","last_change":"2024-09-10T23:36:52.257937+0000","last_active":"2024-09-10T23:36:52.267588+0000","last_peered":"2024-09-10T23:36:52.267588+0000","last_clean":"2024-09-10T23:36:52.267588+0000","last_became_active":"2024-09-10T23:36:52.257521+0000","last_became_peered":"2024-09-10T23:36:52.257521+0000","last_unstale":"2024-09-10T23:36:52.267588+0000","last_undegraded":"2024-09-10T23:36:52.267588+0000","last_fullsized":"2024-09-10T23:36:52.267588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:46.298248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.239799+0000","last_change":"2024-09-10T23:36:51.247320+0000","last_active":"2024-09-10T23:36:52.239799+0000","last_peered":"2024-09-10T23:36:52.239799+0000","last_clean":"2024-09-10T23:36:52.239799+0000","last_became_active":"2024-09-10T23:36:51.247016+0000","last_became_peered":"2024-09-10T23:36:51.247016+0000","last_unstale":"2024-09-10T23:36:52.239799+0000","last_undegraded":"2024-09-10T23:36:52.239799+0000","last_fullsized":"2024-09-10T23:36:52.239799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:40.701589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.267558+0000","last_change":"2024-09-10T23:36:52.263486+0000","last_active":"2024-09-10T23:36:52.267558+0000","last_peered":"2024-09-10T23:36:52.267558+0000","last_clean":"2024-09-10T23:36:52.267558+0000","last_became_active":"2024-09-10T23:36:52.263240+0000","last_became_peered":"2024-09-10T23:36:52.263240+0000","last_unstale":"2024-09-10T23:36:52.267558+0000","last_undegraded":"2024-09-10T23:36:52.267558+0000","last_fullsized":"2024-09-10T23:36:52.267558+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:57.163842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.236204+0000","last_change":"2024-09-10T23:36:51.248214+0000","last_active":"2024-09-10T23:36:52.236204+0000","last_peered":"2024-09-10T23:36:52.236204+0000","last_clean":"2024-09-10T23:36:52.236204+0000","last_became_active":"2024-09-10T23:36:51.247905+0000","last_became_peered":"2024-09-10T23:36:51.247905+0000","last_unstale":"2024-09-10T23:36:52.236204+0000","last_undegraded":"2024-09-10T23:36:52.236204+0000","last_fullsized":"2024-09-10T23:36:52.236204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:11.892931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.247251+0000","last_change":"2024-09-10T23:36:52.242929+0000","last_active":"2024-09-10T23:36:52.247251+0000","last_peered":"2024-09-10T23:36:52.247251+0000","last_clean":"2024-09-10T23:36:52.247251+0000","last_became_active":"2024-09-10T23:36:52.242511+0000","last_became_peered":"2024-09-10T23:36:52.242511+0000","last_unstale":"2024-09-10T23:36:52.247251+0000","last_undegraded":"2024-09-10T23:36:52.247251+0000","last_fullsized":"2024-09-10T23:36:52.247251+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:14.361040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.239699+0000","last_change":"2024-09-10T23:36:51.240180+0000","last_active":"2024-09-10T23:36:52.239699+0000","last_peered":"2024-09-10T23:36:52.239699+0000","last_clean":"2024-09-10T23:36:52.239699+0000","last_became_active":"2024-09-10T23:36:51.239910+0000","last_became_peered":"2024-09-10T23:36:51.239910+0000","last_unstale":"2024-09-10T23:36:52.239699+0000","last_undegraded":"2024-09-10T23:36:52.239699+0000","last_fullsized":"2024-09-10T23:36:52.239699+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:54.155581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.270722+0000","last_change":"2024-09-10T23:36:52.265382+0000","last_active":"2024-09-10T23:36:52.270722+0000","last_peered":"2024-09-10T23:36:52.270722+0000","last_clean":"2024-09-10T23:36:52.270722+0000","last_became_active":"2024-09-10T23:36:52.265086+0000","last_became_peered":"2024-09-10T23:36:52.265086+0000","last_unstale":"2024-09-10T23:36:52.270722+0000","last_undegraded":"2024-09-10T23:36:52.270722+0000","last_fullsized":"2024-09-10T23:36:52.270722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:17.214930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.237184+0000","last_change":"2024-09-10T23:36:51.242468+0000","last_active":"2024-09-10T23:36:52.237184+0000","last_peered":"2024-09-10T23:36:52.237184+0000","last_clean":"2024-09-10T23:36:52.237184+0000","last_became_active":"2024-09-10T23:36:51.242199+0000","last_became_peered":"2024-09-10T23:36:51.242199+0000","last_unstale":"2024-09-10T23:36:52.237184+0000","last_undegraded":"2024-09-10T23:36:52.237184+0000","last_fullsized":"2024-09-10T23:36:52.237184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:55.223076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.247331+0000","last_change":"2024-09-10T23:36:52.242664+0000","last_active":"2024-09-10T23:36:52.247331+0000","last_peered":"2024-09-10T23:36:52.247331+0000","last_clean":"2024-09-10T23:36:52.247331+0000","last_became_active":"2024-09-10T23:36:52.242038+0000","last_became_peered":"2024-09-10T23:36:52.242038+0000","last_unstale":"2024-09-10T23:36:52.247331+0000","last_undegraded":"2024-09-10T23:36:52.247331+0000","last_fullsized":"2024-09-10T23:36:52.247331+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:46.820926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.231673+0000","last_change":"2024-09-10T23:36:51.248559+0000","last_active":"2024-09-10T23:36:52.231673+0000","last_peered":"2024-09-10T23:36:52.231673+0000","last_clean":"2024-09-10T23:36:52.231673+0000","last_became_active":"2024-09-10T23:36:51.248327+0000","last_became_peered":"2024-09-10T23:36:51.248327+0000","last_unstale":"2024-09-10T23:36:52.231673+0000","last_undegraded":"2024-09-10T23:36:52.231673+0000","last_fullsized":"2024-09-10T23:36:52.231673+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:23.705570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.256912+0000","last_change":"2024-09-10T23:36:52.253914+0000","last_active":"2024-09-10T23:36:52.256912+0000","last_peered":"2024-09-10T23:36:52.256912+0000","last_clean":"2024-09-10T23:36:52.256912+0000","last_became_active":"2024-09-10T23:36:52.250835+0000","last_became_peered":"2024-09-10T23:36:52.250835+0000","last_unstale":"2024-09-10T23:36:52.256912+0000","last_undegraded":"2024-09-10T23:36:52.256912+0000","last_fullsized":"2024-09-10T23:36:52.256912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:37.496155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.237873+0000","last_change":"2024-09-10T23:36:51.249119+0000","last_active":"2024-09-10T23:36:52.237873+0000","last_peered":"2024-09-10T23:36:52.237873+0000","last_clean":"2024-09-10T23:36:52.237873+0000","last_became_active":"2024-09-10T23:36:51.248829+0000","last_became_peered":"2024-09-10T23:36:51.248829+0000","last_unstale":"2024-09-10T23:36:52.237873+0000","last_undegraded":"2024-09-10T23:36:52.237873+0000","last_fullsized":"2024-09-10T23:36:52.237873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:54.155581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.263727+0000","last_change":"2024-09-10T23:36:52.261387+0000","last_active":"2024-09-10T23:36:52.263727+0000","last_peered":"2024-09-10T23:36:52.263727+0000","last_clean":"2024-09-10T23:36:52.263727+0000","last_became_active":"2024-09-10T23:36:52.260720+0000","last_became_peered":"2024-09-10T23:36:52.260720+0000","last_unstale":"2024-09-10T23:36:52.263727+0000","last_undegraded":"2024-09-10T23:36:52.263727+0000","last_fullsized":"2024-09-10T23:36:52.263727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:51.216776+0000","last_clean_scrub_stamp":"2024-09-10T23:36:51.216776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:33.594454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.269239+0000","last_change":"2024-09-10T23:36:51.249710+0000","last_active":"2024-09-10T23:36:52.269239+0000","last_peered":"2024-09-10T23:36:52.269239+0000","last_clean":"2024-09-10T23:36:52.269239+0000","last_became_active":"2024-09-10T23:36:51.249452+0000","last_became_peered":"2024-09-10T23:36:51.249452+0000","last_unstale":"2024-09-10T23:36:52.269239+0000","last_undegraded":"2024-09-10T23:36:52.269239+0000","last_fullsized":"2024-09-10T23:36:52.269239+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:36:50.214317+0000","last_clean_scrub_stamp":"2024-09-10T23:36:50.214317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:42.368336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":244,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-10T23:36:52.231964+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:36:52.231964+0000","last_peered":"2024-09-10T23:36:52.231964+0000","last_clean":"2024-09-10T23:36:52.231964+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:36:52.231964+0000","last_undegraded":"2024-09-10T23:36:52.231964+0000","last_fullsized":"2024-09-10T23:36:52.231964+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509498,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":962560,"data_stored":406977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542201,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":983040,"data_stored":414424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509497,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1556480,"data_stored":997345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509495,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":962560,"data_stored":406977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509496,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":962560,"data_stored":406977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509497,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":974848,"data_stored":413752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509496,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1572864,"data_stored":1004210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509496,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":962560,"data_stored":406977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:36:56.743 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T23:36:56.743 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T23:36:56.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.930+0000 7f5b1b600640 1 -- 172.21.15.94:0/3521645281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b140c49b0 msgr2=0x7f5b140cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:56.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.930+0000 7f5b1b600640 1 --2- 172.21.15.94:0/3521645281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b140c49b0 0x7f5b140cadc0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009900 tx=0x7f5afc01c8e0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.930+0000 7f5b1b600640 1 -- 172.21.15.94:0/3521645281 shutdown_connections 2024-09-10T23:36:56.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.930+0000 7f5b1b600640 1 --2- 172.21.15.94:0/3521645281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b140c49b0 0x7f5b140cadc0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.930+0000 7f5b1b600640 1 --2- 172.21.15.94:0/3521645281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b14158790 0x7f5b1406fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.930+0000 7f5b1b600640 1 -- 172.21.15.94:0/3521645281 >> 172.21.15.94:0/3521645281 conn(0x7f5b140cfaf0 msgr2=0x7f5b140cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:56.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.930+0000 7f5b1b600640 1 -- 172.21.15.94:0/3521645281 shutdown_connections 2024-09-10T23:36:56.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.930+0000 7f5b1b600640 1 -- 172.21.15.94:0/3521645281 wait complete. 2024-09-10T23:36:56.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.930+0000 7f5b1b600640 1 Processor -- start 2024-09-10T23:36:56.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.930+0000 7f5b1b600640 1 -- start start 2024-09-10T23:36:56.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b1b600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b140c49b0 0x7f5b1414b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:56.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b1b600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b140cb300 0x7f5b141fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:56.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b1b600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b14158790 0x7f5b14155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:56.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b1b600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b14168290 con 0x7f5b140c49b0 2024-09-10T23:36:56.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b1b600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5b14168110 con 0x7f5b14158790 2024-09-10T23:36:56.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b1b600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5b14168410 con 0x7f5b140cb300 2024-09-10T23:36:56.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b18c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b140c49b0 0x7f5b1414b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:56.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b18c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b140c49b0 0x7f5b1414b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47628/0 (socket says 172.21.15.94:47628) 2024-09-10T23:36:56.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b18c00640 1 -- 172.21.15.94:0/1493387368 learned_addr learned my addr 172.21.15.94:0/1493387368 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:56.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b0be00640 1 --2- 172.21.15.94:0/1493387368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b140cb300 0x7f5b141fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:56.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b19600640 1 --2- 172.21.15.94:0/1493387368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b14158790 0x7f5b14155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:56.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b18c00640 1 -- 172.21.15.94:0/1493387368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b140cb300 msgr2=0x7f5b141fb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b18c00640 1 --2- 172.21.15.94:0/1493387368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b140cb300 0x7f5b141fb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b18c00640 1 -- 172.21.15.94:0/1493387368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b14158790 msgr2=0x7f5b14155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b18c00640 1 --2- 172.21.15.94:0/1493387368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b14158790 0x7f5b14155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b18c00640 1 -- 172.21.15.94:0/1493387368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b141fbe80 con 0x7f5b140c49b0 2024-09-10T23:36:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b18c00640 1 --2- 172.21.15.94:0/1493387368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b140c49b0 0x7f5b1414b750 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400d720 tx=0x7f5b0400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b11600640 1 -- 172.21.15.94:0/1493387368 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04017070 con 0x7f5b140c49b0 2024-09-10T23:36:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b11600640 1 -- 172.21.15.94:0/1493387368 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b04009d20 con 0x7f5b140c49b0 2024-09-10T23:36:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b11600640 1 -- 172.21.15.94:0/1493387368 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04005280 con 0x7f5b140c49b0 2024-09-10T23:36:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b1b600640 1 -- 172.21.15.94:0/1493387368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b14072d80 con 0x7f5b140c49b0 2024-09-10T23:36:56.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b1b600640 1 -- 172.21.15.94:0/1493387368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b14073300 con 0x7f5b140c49b0 2024-09-10T23:36:56.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b1b600640 1 -- 172.21.15.94:0/1493387368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b140cadc0 con 0x7f5b140c49b0 2024-09-10T23:36:56.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.934+0000 7f5b11600640 1 -- 172.21.15.94:0/1493387368 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f5b04005420 con 0x7f5b140c49b0 2024-09-10T23:36:56.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.938+0000 7f5b11600640 1 --2- 172.21.15.94:0/1493387368 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5af0072700 0x7f5af0074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:56.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.938+0000 7f5b11600640 1 -- 172.21.15.94:0/1493387368 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f5b04093580 con 0x7f5b140c49b0 2024-09-10T23:36:56.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.938+0000 7f5b0be00640 1 --2- 172.21.15.94:0/1493387368 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5af0072700 0x7f5af0074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:56.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.938+0000 7f5b0be00640 1 --2- 172.21.15.94:0/1493387368 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5af0072700 0x7f5af0074bc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0af000 tx=0x7f5afc01b120 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:56.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:56.942+0000 7f5b11600640 1 -- 172.21.15.94:0/1493387368 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5b141fbe80 con 0x7f5b140c49b0 2024-09-10T23:36:57.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.166+0000 7f5b1b600640 1 -- 172.21.15.94:0/1493387368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5b140d0850 con 0x7f5b140c49b0 2024-09-10T23:36:57.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.166+0000 7f5b11600640 1 -- 172.21.15.94:0/1493387368 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v34) ==== 190+0+147 (secure 0 0 0) 0x7f5b040670d0 con 0x7f5b140c49b0 2024-09-10T23:36:57.169 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-10T23:36:57.169 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g== 2024-09-10T23:36:57.169 INFO:teuthology.orchestra.run.smithi094.stdout: caps mds = "allow" 2024-09-10T23:36:57.169 INFO:teuthology.orchestra.run.smithi094.stdout: caps mon = "allow r" 2024-09-10T23:36:57.169 INFO:teuthology.orchestra.run.smithi094.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T23:36:57.169 INFO:teuthology.orchestra.run.smithi094.stderr:updated caps for client.0 2024-09-10T23:36:57.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.170+0000 7f5b1b600640 1 -- 172.21.15.94:0/1493387368 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5af0072700 msgr2=0x7f5af0074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:57.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.170+0000 7f5b1b600640 1 --2- 172.21.15.94:0/1493387368 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5af0072700 0x7f5af0074bc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0af000 tx=0x7f5afc01b120 comp rx=0 tx=0).stop 2024-09-10T23:36:57.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.170+0000 7f5b1b600640 1 -- 172.21.15.94:0/1493387368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b140c49b0 msgr2=0x7f5b1414b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:57.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.170+0000 7f5b1b600640 1 --2- 172.21.15.94:0/1493387368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b140c49b0 0x7f5b1414b750 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400d720 tx=0x7f5b0400dbe0 comp rx=0 tx=0).stop 2024-09-10T23:36:57.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.170+0000 7f5b1b600640 1 -- 172.21.15.94:0/1493387368 shutdown_connections 2024-09-10T23:36:57.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.170+0000 7f5b1b600640 1 --2- 172.21.15.94:0/1493387368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b14158790 0x7f5b14155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:57.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.170+0000 7f5b1b600640 1 --2- 172.21.15.94:0/1493387368 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5af0072700 0x7f5af0074bc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:57.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.170+0000 7f5b1b600640 1 --2- 172.21.15.94:0/1493387368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b140cb300 0x7f5b141fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:57.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.170+0000 7f5b1b600640 1 --2- 172.21.15.94:0/1493387368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b140c49b0 0x7f5b1414b750 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:57.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.170+0000 7f5b1b600640 1 -- 172.21.15.94:0/1493387368 >> 172.21.15.94:0/1493387368 conn(0x7f5b140cfaf0 msgr2=0x7f5b140ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:57.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.170+0000 7f5b1b600640 1 -- 172.21.15.94:0/1493387368 shutdown_connections 2024-09-10T23:36:57.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.170+0000 7f5b1b600640 1 -- 172.21.15.94:0/1493387368 wait complete. 2024-09-10T23:36:57.190 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:36:57.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.330+0000 7f638a400640 1 -- 172.21.15.94:0/1568738033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63840d1490 msgr2=0x7f63840cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:57.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.330+0000 7f638a400640 1 --2- 172.21.15.94:0/1568738033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63840d1490 0x7f63840cebf0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f636c009900 tx=0x7f636c01c850 comp rx=0 tx=0).stop 2024-09-10T23:36:57.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.330+0000 7f638a400640 1 -- 172.21.15.94:0/1568738033 shutdown_connections 2024-09-10T23:36:57.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.330+0000 7f638a400640 1 --2- 172.21.15.94:0/1568738033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63840d1490 0x7f63840cebf0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:57.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.330+0000 7f638a400640 1 -- 172.21.15.94:0/1568738033 >> 172.21.15.94:0/1568738033 conn(0x7f63840d1c60 msgr2=0x7f63840d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:57.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.330+0000 7f638a400640 1 -- 172.21.15.94:0/1568738033 shutdown_connections 2024-09-10T23:36:57.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.330+0000 7f638a400640 1 -- 172.21.15.94:0/1568738033 wait complete. 2024-09-10T23:36:57.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f638a400640 1 Processor -- start 2024-09-10T23:36:57.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f638a400640 1 -- start start 2024-09-10T23:36:57.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f638a400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f638406dd30 0x7f63840b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:57.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f638a400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f638406e470 0x7f63841fb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:57.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f638a400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63840d1490 0x7f63841fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:57.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f638a400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f638416c5a0 con 0x7f638406e470 2024-09-10T23:36:57.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f638a400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f638416c420 con 0x7f63840d1490 2024-09-10T23:36:57.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f638a400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f638416c720 con 0x7f638406dd30 2024-09-10T23:36:57.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f637aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f638406e470 0x7f63841fb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:57.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f637aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f638406e470 0x7f63841fb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47648/0 (socket says 172.21.15.94:47648) 2024-09-10T23:36:57.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f637aa00640 1 -- 172.21.15.94:0/1139213963 learned_addr learned my addr 172.21.15.94:0/1139213963 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:57.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f6383400640 1 --2- 172.21.15.94:0/1139213963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f638406dd30 0x7f63840b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:57.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f6383e00640 1 --2- 172.21.15.94:0/1139213963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63840d1490 0x7f63841fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:57.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f637aa00640 1 -- 172.21.15.94:0/1139213963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f638406dd30 msgr2=0x7f63840b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:57.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f637aa00640 1 --2- 172.21.15.94:0/1139213963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f638406dd30 0x7f63840b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:57.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f637aa00640 1 -- 172.21.15.94:0/1139213963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63840d1490 msgr2=0x7f63841fbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:57.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f637aa00640 1 --2- 172.21.15.94:0/1139213963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63840d1490 0x7f63841fbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:57.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f637aa00640 1 -- 172.21.15.94:0/1139213963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6384141590 con 0x7f638406e470 2024-09-10T23:36:57.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f6383400640 1 --2- 172.21.15.94:0/1139213963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f638406dd30 0x7f63840b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:57.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f6383e00640 1 --2- 172.21.15.94:0/1139213963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63840d1490 0x7f63841fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:57.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f637aa00640 1 --2- 172.21.15.94:0/1139213963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f638406e470 0x7f63841fb680 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f636c009850 tx=0x7f636c0a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:57.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f637be00640 1 -- 172.21.15.94:0/1139213963 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c0ba020 con 0x7f638406e470 2024-09-10T23:36:57.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f637be00640 1 -- 172.21.15.94:0/1139213963 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f636c004430 con 0x7f638406e470 2024-09-10T23:36:57.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f638a400640 1 -- 172.21.15.94:0/1139213963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63841fc1f0 con 0x7f638406e470 2024-09-10T23:36:57.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f637be00640 1 -- 172.21.15.94:0/1139213963 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c0b2040 con 0x7f638406e470 2024-09-10T23:36:57.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f638a400640 1 -- 172.21.15.94:0/1139213963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63840b3870 con 0x7f638406e470 2024-09-10T23:36:57.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.334+0000 7f638a400640 1 -- 172.21.15.94:0/1139213963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63840cca20 con 0x7f638406e470 2024-09-10T23:36:57.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.338+0000 7f637be00640 1 -- 172.21.15.94:0/1139213963 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f636c003860 con 0x7f638406e470 2024-09-10T23:36:57.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.338+0000 7f637be00640 1 --2- 172.21.15.94:0/1139213963 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6360072700 0x7f6360074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:57.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.338+0000 7f637be00640 1 -- 172.21.15.94:0/1139213963 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f636c12cfb0 con 0x7f638406e470 2024-09-10T23:36:57.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.338+0000 7f6383400640 1 --2- 172.21.15.94:0/1139213963 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6360072700 0x7f6360074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:57.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.338+0000 7f6383400640 1 --2- 172.21.15.94:0/1139213963 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6360072700 0x7f6360074bc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6374001750 tx=0x7f6374009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:57.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.342+0000 7f637be00640 1 -- 172.21.15.94:0/1139213963 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f636c0164d0 con 0x7f638406e470 2024-09-10T23:36:57.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.538+0000 7f638a400640 1 -- 172.21.15.94:0/1139213963 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63840749c0 con 0x7f638406e470 2024-09-10T23:36:57.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.538+0000 7f637be00640 1 -- 172.21.15.94:0/1139213963 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5240 (secure 0 0 0) 0x7f636c0f8520 con 0x7f638406e470 2024-09-10T23:36:57.541 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:36:57.542 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":53,"btime":"2024-09-10T23:36:55:260109+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1215179751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1215179751},{"type":"v1","addr":"172.21.15.94:6834","nonce":1215179751}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6672,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3193009978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":3193009978},{"type":"v1","addr":"172.21.15.181:6835","nonce":3193009978}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":195698,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/736952701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":736952701},{"type":"v1","addr":"172.21.15.94:6835","nonce":736952701}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:36:51.627529+0000","modified":"2024-09-10T23:36:54.641240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195701":{"gid":195701,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6834/1214870454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1214870454},{"type":"v1","addr":"172.21.15.181:6834","nonce":1214870454}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195701,"qdb_cluster":[195701]},"id":6}]} 2024-09-10T23:36:57.542 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 53 2024-09-10T23:36:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.542+0000 7f638a400640 1 -- 172.21.15.94:0/1139213963 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6360072700 msgr2=0x7f6360074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.542+0000 7f638a400640 1 --2- 172.21.15.94:0/1139213963 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6360072700 0x7f6360074bc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6374001750 tx=0x7f6374009040 comp rx=0 tx=0).stop 2024-09-10T23:36:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.542+0000 7f638a400640 1 -- 172.21.15.94:0/1139213963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f638406e470 msgr2=0x7f63841fb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.542+0000 7f638a400640 1 --2- 172.21.15.94:0/1139213963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f638406e470 0x7f63841fb680 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f636c009850 tx=0x7f636c0a5620 comp rx=0 tx=0).stop 2024-09-10T23:36:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.542+0000 7f638a400640 1 -- 172.21.15.94:0/1139213963 shutdown_connections 2024-09-10T23:36:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.542+0000 7f638a400640 1 --2- 172.21.15.94:0/1139213963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63840d1490 0x7f63841fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.542+0000 7f638a400640 1 --2- 172.21.15.94:0/1139213963 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6360072700 0x7f6360074bc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.542+0000 7f638a400640 1 --2- 172.21.15.94:0/1139213963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f638406e470 0x7f63841fb680 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.542+0000 7f638a400640 1 --2- 172.21.15.94:0/1139213963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f638406dd30 0x7f63840b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.542+0000 7f638a400640 1 -- 172.21.15.94:0/1139213963 >> 172.21.15.94:0/1139213963 conn(0x7f63840d1c60 msgr2=0x7f63840b4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:57.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.542+0000 7f638a400640 1 -- 172.21.15.94:0/1139213963 shutdown_connections 2024-09-10T23:36:57.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:57.542+0000 7f638a400640 1 -- 172.21.15.94:0/1139213963 wait complete. 2024-09-10T23:36:57.968 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T23:36:51.627529+0000', 'modified': '2024-09-10T23:36:54.641240+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 195701}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_195701': {'gid': 195701, 'name': 'b', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6834/1214870454', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6800', 'nonce': 1214870454}, {'type': 'v1', 'addr': '172.21.15.181:6834', 'nonce': 1214870454}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 195701, 'qdb_cluster': [195701]} 2024-09-10T23:36:57.969 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T23:36:57.969 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T23:36:58.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.110+0000 7f9430c00640 1 -- 172.21.15.94:0/3602298307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f942c0c64f0 msgr2=0x7f942c0ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:58.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.110+0000 7f9430c00640 1 --2- 172.21.15.94:0/3602298307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f942c0c64f0 0x7f942c0ce660 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f942c0ca500 tx=0x7f941401c8a0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.110+0000 7f9430c00640 1 -- 172.21.15.94:0/3602298307 shutdown_connections 2024-09-10T23:36:58.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.110+0000 7f9430c00640 1 --2- 172.21.15.94:0/3602298307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f942c0c64f0 0x7f942c0ce660 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.110+0000 7f9430c00640 1 --2- 172.21.15.94:0/3602298307 >> v2:172.21.15.181:3300/0 conn(0x7f942c0ce280 0x7f942c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.110+0000 7f9430c00640 1 -- 172.21.15.94:0/3602298307 >> 172.21.15.94:0/3602298307 conn(0x7f942c0cfaf0 msgr2=0x7f942c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:58.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.110+0000 7f9430c00640 1 -- 172.21.15.94:0/3602298307 shutdown_connections 2024-09-10T23:36:58.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.110+0000 7f9430c00640 1 -- 172.21.15.94:0/3602298307 wait complete. 2024-09-10T23:36:58.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.110+0000 7f9430c00640 1 Processor -- start 2024-09-10T23:36:58.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.110+0000 7f9430c00640 1 -- start start 2024-09-10T23:36:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9430c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f942c06e7c0 0x7f942c0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9430c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f942c0c64f0 0x7f942c13ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9430c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f942c0ce280 0x7f942c13e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9430c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f942c0cdd80 con 0x7f942c0c64f0 2024-09-10T23:36:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9430c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f942c0cdc00 con 0x7f942c06e7c0 2024-09-10T23:36:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9430c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f942c0cdf00 con 0x7f942c0ce280 2024-09-10T23:36:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9421400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f942c0c64f0 0x7f942c13ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9421400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f942c0c64f0 0x7f942c13ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47666/0 (socket says 172.21.15.94:47666) 2024-09-10T23:36:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9421400640 1 -- 172.21.15.94:0/3466297305 learned_addr learned my addr 172.21.15.94:0/3466297305 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f942a800640 1 --2- 172.21.15.94:0/3466297305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f942c0ce280 0x7f942c13e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9429e00640 1 --2- 172.21.15.94:0/3466297305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f942c06e7c0 0x7f942c0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9421400640 1 -- 172.21.15.94:0/3466297305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f942c0ce280 msgr2=0x7f942c13e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9421400640 1 --2- 172.21.15.94:0/3466297305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f942c0ce280 0x7f942c13e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9421400640 1 -- 172.21.15.94:0/3466297305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f942c06e7c0 msgr2=0x7f942c0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9421400640 1 --2- 172.21.15.94:0/3466297305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f942c06e7c0 0x7f942c0ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9421400640 1 -- 172.21.15.94:0/3466297305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942c1f9580 con 0x7f942c0c64f0 2024-09-10T23:36:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9429e00640 1 --2- 172.21.15.94:0/3466297305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f942c06e7c0 0x7f942c0ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f942a800640 1 --2- 172.21.15.94:0/3466297305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f942c0ce280 0x7f942c13e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9421400640 1 --2- 172.21.15.94:0/3466297305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f942c0c64f0 0x7f942c13ded0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f94140b0040 tx=0x7f94140041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9422a00640 1 -- 172.21.15.94:0/3466297305 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94140b2040 con 0x7f942c0c64f0 2024-09-10T23:36:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9422a00640 1 -- 172.21.15.94:0/3466297305 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9414003600 con 0x7f942c0c64f0 2024-09-10T23:36:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9422a00640 1 -- 172.21.15.94:0/3466297305 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414003a70 con 0x7f942c0c64f0 2024-09-10T23:36:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9430c00640 1 -- 172.21.15.94:0/3466297305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942c1f9860 con 0x7f942c0c64f0 2024-09-10T23:36:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.114+0000 7f9430c00640 1 -- 172.21.15.94:0/3466297305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942c1f9e20 con 0x7f942c0c64f0 2024-09-10T23:36:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f9422a00640 1 -- 172.21.15.94:0/3466297305 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f9414003c10 con 0x7f942c0c64f0 2024-09-10T23:36:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f9422a00640 1 --2- 172.21.15.94:0/3466297305 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9408072700 0x7f9408074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f943401c000 1 Processor -- start 2024-09-10T23:36:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f9422a00640 1 -- 172.21.15.94:0/3466297305 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f941412d300 con 0x7f942c0c64f0 2024-09-10T23:36:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f943401c000 1 -- start start 2024-09-10T23:36:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f9429e00640 1 --2- 172.21.15.94:0/3466297305 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9408072700 0x7f9408074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f9429e00640 1 --2- 172.21.15.94:0/3466297305 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9408072700 0x7f9408074bc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f941c005c50 tx=0x7f941c00a450 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f943401c000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29bd4f00 0x555f29bd52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f943401c000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29bd5820 0x555f29b8acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f943401c000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29b8b230 0x555f29b8b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f943401c000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555f29aa53f0 con 0x555f29bd4f00 2024-09-10T23:36:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f943401c000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555f299e2b20 con 0x555f29b8b230 2024-09-10T23:36:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f943401c000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x555f299d1f50 con 0x555f29bd5820 2024-09-10T23:36:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f9429e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29bd4f00 0x555f29bd52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f9429e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29bd4f00 0x555f29bd52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47680/0 (socket says 172.21.15.94:47680) 2024-09-10T23:36:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f9429e00640 1 -- 172.21.15.94:0/2863921691 learned_addr learned my addr 172.21.15.94:0/2863921691 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f9421400640 1 --2- 172.21.15.94:0/2863921691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29bd5820 0x555f29b8acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.118+0000 7f942a800640 1 --2- 172.21.15.94:0/2863921691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29b8b230 0x555f29b8b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f9429e00640 1 -- 172.21.15.94:0/2863921691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29bd5820 msgr2=0x555f29b8acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:36:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f9429e00640 1 --2- 172.21.15.94:0/2863921691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29bd5820 0x555f29b8acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f9429e00640 1 -- 172.21.15.94:0/2863921691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29b8b230 msgr2=0x555f29b8b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:36:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f9429e00640 1 --2- 172.21.15.94:0/2863921691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29b8b230 0x555f29b8b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f9429e00640 1 -- 172.21.15.94:0/2863921691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f29999590 con 0x555f29bd4f00 2024-09-10T23:36:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f9429e00640 1 --2- 172.21.15.94:0/2863921691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29bd4f00 0x555f29bd52e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f941c010560 tx=0x7f941c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d70005c729128bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f941aa00640 1 -- 172.21.15.94:0/2863921691 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c021070 con 0x555f29bd4f00 2024-09-10T23:36:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f941aa00640 1 -- 172.21.15.94:0/2863921691 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f941c00f980 con 0x555f29bd4f00 2024-09-10T23:36:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f941aa00640 1 -- 172.21.15.94:0/2863921691 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c00fd30 con 0x555f29bd4f00 2024-09-10T23:36:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 -- 172.21.15.94:0/2863921691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29bd4f00 msgr2=0x555f29bd52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:58.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 --2- 172.21.15.94:0/2863921691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29bd4f00 0x555f29bd52e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f941c010560 tx=0x7f941c011000 comp rx=0 tx=0).stop 2024-09-10T23:36:58.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 -- 172.21.15.94:0/2863921691 shutdown_connections 2024-09-10T23:36:58.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 --2- 172.21.15.94:0/2863921691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29b8b230 0x555f29b8b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 --2- 172.21.15.94:0/2863921691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29bd5820 0x555f29b8acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 --2- 172.21.15.94:0/2863921691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29bd4f00 0x555f29bd52e0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 -- 172.21.15.94:0/2863921691 >> 172.21.15.94:0/2863921691 conn(0x555f29a6e130 msgr2=0x555f29b8ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:58.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 -- 172.21.15.94:0/2863921691 shutdown_connections 2024-09-10T23:36:58.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 -- 172.21.15.94:0/2863921691 wait complete. 2024-09-10T23:36:58.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 -- start start 2024-09-10T23:36:58.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 Processor -- start 2024-09-10T23:36:58.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29b8b230 0x555f29bbf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29bd4f00 0x555f29bbfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd5820 0x555f29bc03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555f2996c5d0 con 0x555f29bd4f00 2024-09-10T23:36:58.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555f299d5d60 con 0x555f29bd5820 2024-09-10T23:36:58.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x555f29978500 con 0x555f29b8b230 2024-09-10T23:36:58.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f943401c000 10 client.0 _ll_register_callbacks cb 0x555f2997c810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:36:58.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f9421400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29bd4f00 0x555f29bbfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f9421400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29bd4f00 0x555f29bbfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47690/0 (socket says 172.21.15.94:47690) 2024-09-10T23:36:58.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f9421400640 1 -- 172.21.15.94:0/3726570382 learned_addr learned my addr 172.21.15.94:0/3726570382 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:58.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f9429e00640 1 --2- 172.21.15.94:0/3726570382 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29b8b230 0x555f29bbf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.122+0000 7f942a800640 1 --2- 172.21.15.94:0/3726570382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd5820 0x555f29bc03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f9421400640 1 -- 172.21.15.94:0/3726570382 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29b8b230 msgr2=0x555f29bbf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:36:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f9421400640 1 --2- 172.21.15.94:0/3726570382 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29b8b230 0x555f29bbf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f9421400640 1 -- 172.21.15.94:0/3726570382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd5820 msgr2=0x555f29bc03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:36:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f9421400640 1 --2- 172.21.15.94:0/3726570382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd5820 0x555f29bc03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f9421400640 1 -- 172.21.15.94:0/3726570382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f29970170 con 0x555f29bd4f00 2024-09-10T23:36:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f942a800640 1 --2- 172.21.15.94:0/3726570382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd5820 0x555f29bc03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f9421400640 1 --2- 172.21.15.94:0/3726570382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29bd4f00 0x555f29bbfe60 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9414000c10 tx=0x7f94140f7a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbf4f3e0ada5218 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f940f400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:36:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f940f400640 1 -- 172.21.15.94:0/3726570382 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941411d1c0 con 0x555f29bd4f00 2024-09-10T23:36:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f940f400640 1 -- 172.21.15.94:0/3726570382 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f9414101330 con 0x555f29bd4f00 2024-09-10T23:36:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 1 -- 172.21.15.94:0/3726570382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555f2996a0f0 con 0x555f29bd4f00 2024-09-10T23:36:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f940f400640 1 -- 172.21.15.94:0/3726570382 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f94140f7bc0 con 0x555f29bd4f00 2024-09-10T23:36:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 10 client.6786 fetch_fsmap learned FSMap version 53 2024-09-10T23:36:58.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 1 -- 172.21.15.94:0/3726570382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x555f29a56ef0 con 0x555f29bd4f00 2024-09-10T23:36:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f940f400640 1 -- 172.21.15.94:0/3726570382 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555f2996a0f0 con 0x555f29bd4f00 2024-09-10T23:36:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f940f400640 1 -- 172.21.15.94:0/3726570382 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7f941410dba0 con 0x555f29bd4f00 2024-09-10T23:36:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 10 client.6786 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T23:36:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 10 client.6786 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T23:36:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 20 client.6786 populate_metadata read hostname 'smithi094' 2024-09-10T23:36:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 1 --2- 172.21.15.94:0/3726570382 >> [v2:172.21.15.181:6800/1214870454,v1:172.21.15.181:6834/1214870454] conn(0x555f29bc0db0 0x555f29bc1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 4 client.6786 mds_command: new command op to 195701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:36:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 1 -- 172.21.15.94:0/3726570382 --> [v2:172.21.15.181:6800/1214870454,v1:172.21.15.181:6834/1214870454] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f299e4a70 con 0x555f29bc0db0 2024-09-10T23:36:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f9429e00640 1 --2- 172.21.15.94:0/3726570382 >> [v2:172.21.15.181:6800/1214870454,v1:172.21.15.181:6834/1214870454] conn(0x555f29bc0db0 0x555f29bc1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f940f400640 10 client.6786 ms_handle_connect on v2:172.21.15.181:6800/1214870454 2024-09-10T23:36:58.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f9429e00640 1 --2- 172.21.15.94:0/3726570382 >> [v2:172.21.15.181:6800/1214870454,v1:172.21.15.181:6834/1214870454] conn(0x555f29bc0db0 0x555f29bc1190 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36d7f4185c55d423 server_cookie=fd006d53da156bef in_seq=0 out_seq=0 2024-09-10T23:36:58.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f940f400640 1 -- 172.21.15.94:0/3726570382 <== mds.0 v2:172.21.15.181:6800/1214870454 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555f299e4a70 con 0x555f29bc0db0 2024-09-10T23:36:58.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f940f400640 10 client.6786 handle_command_reply: tid=0 2024-09-10T23:36:58.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 1 client.6786 shutdown 2024-09-10T23:36:58.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 1 -- 172.21.15.94:0/3726570382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29bd4f00 msgr2=0x555f29bbfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:58.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 1 --2- 172.21.15.94:0/3726570382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29bd4f00 0x555f29bbfe60 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9414000c10 tx=0x7f94140f7a00 comp rx=0 tx=0).stop 2024-09-10T23:36:58.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 1 -- 172.21.15.94:0/3726570382 shutdown_connections 2024-09-10T23:36:58.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 1 --2- 172.21.15.94:0/3726570382 >> [v2:172.21.15.181:6800/1214870454,v1:172.21.15.181:6834/1214870454] conn(0x555f29bc0db0 0x555f29bc1190 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 1 --2- 172.21.15.94:0/3726570382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd5820 0x555f29bc03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 1 --2- 172.21.15.94:0/3726570382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29bd4f00 0x555f29bbfe60 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 1 --2- 172.21.15.94:0/3726570382 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29b8b230 0x555f29bbf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 1 -- 172.21.15.94:0/3726570382 >> 172.21.15.94:0/3726570382 conn(0x555f29a6e130 msgr2=0x555f29b8b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:58.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f940f400640 0 client.6786 ms_handle_reset on v2:172.21.15.181:6800/1214870454 2024-09-10T23:36:58.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 1 -- 172.21.15.94:0/3726570382 shutdown_connections 2024-09-10T23:36:58.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 1 -- 172.21.15.94:0/3726570382 wait complete. 2024-09-10T23:36:58.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.126+0000 7f943401c000 20 client.6786 trim_cache size 0 max 16384 2024-09-10T23:36:58.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f943401c000 1 Processor -- start 2024-09-10T23:36:58.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f943401c000 1 -- start start 2024-09-10T23:36:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f943401c000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29b8b230 0x555f29bec630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f943401c000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd4f00 0x555f29becb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f943401c000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29bd5820 0x555f29bd3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f943401c000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f942c15a480 con 0x555f29b8b230 2024-09-10T23:36:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f943401c000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f942c0b3e50 con 0x555f29bd4f00 2024-09-10T23:36:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f943401c000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f942c0c7970 con 0x555f29bd5820 2024-09-10T23:36:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f9429e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29b8b230 0x555f29bec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f9429e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29b8b230 0x555f29bec630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47706/0 (socket says 172.21.15.94:47706) 2024-09-10T23:36:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f9429e00640 1 -- 172.21.15.94:0/4046591062 learned_addr learned my addr 172.21.15.94:0/4046591062 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f9421400640 1 --2- 172.21.15.94:0/4046591062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd4f00 0x555f29becb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f942a800640 1 --2- 172.21.15.94:0/4046591062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29bd5820 0x555f29bd3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f9429e00640 1 -- 172.21.15.94:0/4046591062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29bd5820 msgr2=0x555f29bd3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:36:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f9429e00640 1 --2- 172.21.15.94:0/4046591062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29bd5820 0x555f29bd3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f9429e00640 1 -- 172.21.15.94:0/4046591062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd4f00 msgr2=0x555f29becb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:36:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f9429e00640 1 --2- 172.21.15.94:0/4046591062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd4f00 0x555f29becb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.146+0000 7f9429e00640 1 -- 172.21.15.94:0/4046591062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f29bd5650 con 0x555f29b8b230 2024-09-10T23:36:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f9421400640 1 --2- 172.21.15.94:0/4046591062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd4f00 0x555f29becb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:36:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f9429e00640 1 --2- 172.21.15.94:0/4046591062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29b8b230 0x555f29bec630 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x555f299e1890 tx=0x7f941c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e3d16a8a077118 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f941a000640 1 -- 172.21.15.94:0/4046591062 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c014070 con 0x555f29b8b230 2024-09-10T23:36:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f941a000640 1 -- 172.21.15.94:0/4046591062 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f941c00fb90 con 0x555f29b8b230 2024-09-10T23:36:58.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f941a000640 1 -- 172.21.15.94:0/4046591062 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c021040 con 0x555f29b8b230 2024-09-10T23:36:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 -- 172.21.15.94:0/4046591062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29b8b230 msgr2=0x555f29bec630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 --2- 172.21.15.94:0/4046591062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29b8b230 0x555f29bec630 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x555f299e1890 tx=0x7f941c004ac0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 -- 172.21.15.94:0/4046591062 shutdown_connections 2024-09-10T23:36:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 --2- 172.21.15.94:0/4046591062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd4f00 0x555f29becb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 --2- 172.21.15.94:0/4046591062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29bd5820 0x555f29bd3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 --2- 172.21.15.94:0/4046591062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29b8b230 0x555f29bec630 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 -- 172.21.15.94:0/4046591062 >> 172.21.15.94:0/4046591062 conn(0x555f29a6e130 msgr2=0x555f29b89ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 -- 172.21.15.94:0/4046591062 shutdown_connections 2024-09-10T23:36:58.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 -- 172.21.15.94:0/4046591062 wait complete. 2024-09-10T23:36:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 -- start start 2024-09-10T23:36:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 Processor -- start 2024-09-10T23:36:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29b8b230 0x555f29c4b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29bd4f00 0x555f29c4bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd5820 0x555f29c4c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x555f2999dec0 con 0x555f29b8b230 2024-09-10T23:36:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f942c0b3800 con 0x555f29bd5820 2024-09-10T23:36:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x555f2996c5d0 con 0x555f29bd4f00 2024-09-10T23:36:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f943401c000 10 client.0 _ll_register_callbacks cb 0x7f94140f4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:36:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f9429e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29b8b230 0x555f29c4b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f9429e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29b8b230 0x555f29c4b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47710/0 (socket says 172.21.15.94:47710) 2024-09-10T23:36:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f9429e00640 1 -- 172.21.15.94:0/224125414 learned_addr learned my addr 172.21.15.94:0/224125414 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f9421400640 1 --2- 172.21.15.94:0/224125414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29bd4f00 0x555f29c4bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f942a800640 1 --2- 172.21.15.94:0/224125414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd5820 0x555f29c4c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f9429e00640 1 -- 172.21.15.94:0/224125414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29bd4f00 msgr2=0x555f29c4bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:36:58.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f9429e00640 1 --2- 172.21.15.94:0/224125414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29bd4f00 0x555f29c4bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f9429e00640 1 -- 172.21.15.94:0/224125414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd5820 msgr2=0x555f29c4c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:36:58.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f9429e00640 1 --2- 172.21.15.94:0/224125414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd5820 0x555f29c4c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.150+0000 7f9429e00640 1 -- 172.21.15.94:0/224125414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f29a2dd90 con 0x555f29b8b230 2024-09-10T23:36:58.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f9429e00640 1 --2- 172.21.15.94:0/224125414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29b8b230 0x555f29c4b5c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x555f299d35b0 tx=0x7f941c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1eb30d6c80fe0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:58.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f940f400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:36:58.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f940f400640 1 -- 172.21.15.94:0/224125414 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c014070 con 0x555f29b8b230 2024-09-10T23:36:58.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f940f400640 1 -- 172.21.15.94:0/224125414 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f941c004d30 con 0x555f29b8b230 2024-09-10T23:36:58.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 1 -- 172.21.15.94:0/224125414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f941c00fb90 con 0x555f29b8b230 2024-09-10T23:36:58.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f940f400640 1 -- 172.21.15.94:0/224125414 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f941c049050 con 0x555f29b8b230 2024-09-10T23:36:58.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 10 client.6792 fetch_fsmap learned FSMap version 53 2024-09-10T23:36:58.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 1 -- 172.21.15.94:0/224125414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x555f29998a90 con 0x555f29b8b230 2024-09-10T23:36:58.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f940f400640 1 -- 172.21.15.94:0/224125414 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f941c00fb90 con 0x555f29b8b230 2024-09-10T23:36:58.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f940f400640 1 -- 172.21.15.94:0/224125414 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7f941c04c020 con 0x555f29b8b230 2024-09-10T23:36:58.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 10 client.6792 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T23:36:58.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 10 client.6792 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T23:36:58.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 20 client.6792 populate_metadata read hostname 'smithi094' 2024-09-10T23:36:58.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 1 --2- 172.21.15.94:0/224125414 >> [v2:172.21.15.181:6800/1214870454,v1:172.21.15.181:6834/1214870454] conn(0x555f29be8fd0 0x555f29be93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 4 client.6792 mds_command: new command op to 195701 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T23:36:58.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 1 -- 172.21.15.94:0/224125414 --> [v2:172.21.15.181:6800/1214870454,v1:172.21.15.181:6834/1214870454] -- command(tid 0: {"prefix": "status"}) -- 0x555f299e0520 con 0x555f29be8fd0 2024-09-10T23:36:58.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f9421400640 1 --2- 172.21.15.94:0/224125414 >> [v2:172.21.15.181:6800/1214870454,v1:172.21.15.181:6834/1214870454] conn(0x555f29be8fd0 0x555f29be93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f9421400640 1 --2- 172.21.15.94:0/224125414 >> [v2:172.21.15.181:6800/1214870454,v1:172.21.15.181:6834/1214870454] conn(0x555f29be8fd0 0x555f29be93b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ca7a86bf3a74872 server_cookie=f8fab11c411c9944 in_seq=0 out_seq=0 2024-09-10T23:36:58.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f940f400640 10 client.6792 ms_handle_connect on v2:172.21.15.181:6800/1214870454 2024-09-10T23:36:58.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f940f400640 1 -- 172.21.15.94:0/224125414 <== mds.0 v2:172.21.15.181:6800/1214870454 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555f299e0520 con 0x555f29be8fd0 2024-09-10T23:36:58.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f940f400640 10 client.6792 handle_command_reply: tid=0 2024-09-10T23:36:58.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 1 client.6792 shutdown 2024-09-10T23:36:58.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 1 -- 172.21.15.94:0/224125414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29b8b230 msgr2=0x555f29c4b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:58.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 1 --2- 172.21.15.94:0/224125414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29b8b230 0x555f29c4b5c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x555f299d35b0 tx=0x7f941c00f690 comp rx=0 tx=0).stop 2024-09-10T23:36:58.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 1 -- 172.21.15.94:0/224125414 shutdown_connections 2024-09-10T23:36:58.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 1 --2- 172.21.15.94:0/224125414 >> [v2:172.21.15.181:6800/1214870454,v1:172.21.15.181:6834/1214870454] conn(0x555f29be8fd0 0x555f29be93b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 1 --2- 172.21.15.94:0/224125414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f29bd5820 0x555f29c4c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 1 --2- 172.21.15.94:0/224125414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x555f29bd4f00 0x555f29c4bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f940f400640 0 client.6792 ms_handle_reset on v2:172.21.15.181:6800/1214870454 2024-09-10T23:36:58.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 1 --2- 172.21.15.94:0/224125414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555f29b8b230 0x555f29c4b5c0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 1 -- 172.21.15.94:0/224125414 >> 172.21.15.94:0/224125414 conn(0x555f29a6e130 msgr2=0x555f29b8a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:58.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 1 -- 172.21.15.94:0/224125414 shutdown_connections 2024-09-10T23:36:58.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 1 -- 172.21.15.94:0/224125414 wait complete. 2024-09-10T23:36:58.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f943401c000 20 client.6792 trim_cache size 0 max 16384 2024-09-10T23:36:58.162 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:36:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "1798cdb9-9937-4918-a569-86b41a92817e", 2024-09-10T23:36:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-09-10T23:36:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 195701, 2024-09-10T23:36:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-10T23:36:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-10T23:36:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-10T23:36:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 5.9258390280000004, 2024-09-10T23:36:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 53, 2024-09-10T23:36:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 81, 2024-09-10T23:36:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 81, 2024-09-10T23:36:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 10.410503552 2024-09-10T23:36:58.162 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:36:58.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f9430c00640 1 -- 172.21.15.94:0/3466297305 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9408072700 msgr2=0x7f9408074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:58.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f9430c00640 1 --2- 172.21.15.94:0/3466297305 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9408072700 0x7f9408074bc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f941c005c50 tx=0x7f941c00a450 comp rx=0 tx=0).stop 2024-09-10T23:36:58.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f9430c00640 1 -- 172.21.15.94:0/3466297305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f942c0c64f0 msgr2=0x7f942c13ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:58.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f9430c00640 1 --2- 172.21.15.94:0/3466297305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f942c0c64f0 0x7f942c13ded0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f94140b0040 tx=0x7f94140041a0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f9430c00640 1 -- 172.21.15.94:0/3466297305 shutdown_connections 2024-09-10T23:36:58.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f9430c00640 1 --2- 172.21.15.94:0/3466297305 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9408072700 0x7f9408074bc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f9430c00640 1 --2- 172.21.15.94:0/3466297305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f942c0ce280 0x7f942c13e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f9430c00640 1 --2- 172.21.15.94:0/3466297305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f942c0c64f0 0x7f942c13ded0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f9430c00640 1 --2- 172.21.15.94:0/3466297305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f942c06e7c0 0x7f942c0ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f9430c00640 1 -- 172.21.15.94:0/3466297305 >> 172.21.15.94:0/3466297305 conn(0x7f942c0cfaf0 msgr2=0x7f942c073f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:58.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f9430c00640 1 -- 172.21.15.94:0/3466297305 shutdown_connections 2024-09-10T23:36:58.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.154+0000 7f9430c00640 1 -- 172.21.15.94:0/3466297305 wait complete. 2024-09-10T23:36:58.175 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:36:58.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.314+0000 7fac71000640 1 -- 172.21.15.94:0/2643112027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac6c15bdc0 msgr2=0x7fac6c0d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:58.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.314+0000 7fac71000640 1 --2- 172.21.15.94:0/2643112027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac6c15bdc0 0x7fac6c0d0240 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0b33a0 tx=0x7fac5c01c8a0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac71000640 1 -- 172.21.15.94:0/2643112027 shutdown_connections 2024-09-10T23:36:58.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac71000640 1 --2- 172.21.15.94:0/2643112027 >> v2:172.21.15.181:3300/0 conn(0x7fac6c0d2210 0x7fac6c0ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac71000640 1 --2- 172.21.15.94:0/2643112027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac6c15bdc0 0x7fac6c0d0240 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac71000640 1 -- 172.21.15.94:0/2643112027 >> 172.21.15.94:0/2643112027 conn(0x7fac6c09aa70 msgr2=0x7fac6c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:58.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac71000640 1 -- 172.21.15.94:0/2643112027 shutdown_connections 2024-09-10T23:36:58.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac71000640 1 -- 172.21.15.94:0/2643112027 wait complete. 2024-09-10T23:36:58.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac71000640 1 Processor -- start 2024-09-10T23:36:58.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac71000640 1 -- start start 2024-09-10T23:36:58.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac71000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac6c0d0780 0x7fac6c1f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac71000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac6c0d2210 0x7fac6c0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac71000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac6c15bdc0 0x7fac6c1571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac71000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c168ee0 con 0x7fac6c15bdc0 2024-09-10T23:36:58.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac71000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c168d60 con 0x7fac6c0d0780 2024-09-10T23:36:58.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac71000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fac6c169060 con 0x7fac6c0d2210 2024-09-10T23:36:58.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac6a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac6c15bdc0 0x7fac6c1571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac6a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac6c15bdc0 0x7fac6c1571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47732/0 (socket says 172.21.15.94:47732) 2024-09-10T23:36:58.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac6a800640 1 -- 172.21.15.94:0/3584616667 learned_addr learned my addr 172.21.15.94:0/3584616667 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:58.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac69e00640 1 --2- 172.21.15.94:0/3584616667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac6c0d0780 0x7fac6c1f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac61400640 1 --2- 172.21.15.94:0/3584616667 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac6c0d2210 0x7fac6c0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac6a800640 1 -- 172.21.15.94:0/3584616667 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac6c0d2210 msgr2=0x7fac6c0cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:58.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac6a800640 1 --2- 172.21.15.94:0/3584616667 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac6c0d2210 0x7fac6c0cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac6a800640 1 -- 172.21.15.94:0/3584616667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac6c0d0780 msgr2=0x7fac6c1f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:58.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac6a800640 1 --2- 172.21.15.94:0/3584616667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac6c0d0780 0x7fac6c1f5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac6a800640 1 -- 172.21.15.94:0/3584616667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c1507f0 con 0x7fac6c15bdc0 2024-09-10T23:36:58.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac6a800640 1 --2- 172.21.15.94:0/3584616667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac6c15bdc0 0x7fac6c1571e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fac6400cce0 tx=0x7fac6400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:58.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.318+0000 7fac62a00640 1 -- 172.21.15.94:0/3584616667 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6400f450 con 0x7fac6c15bdc0 2024-09-10T23:36:58.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.322+0000 7fac62a00640 1 -- 172.21.15.94:0/3584616667 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac6400f700 con 0x7fac6c15bdc0 2024-09-10T23:36:58.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.322+0000 7fac62a00640 1 -- 172.21.15.94:0/3584616667 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6400fb50 con 0x7fac6c15bdc0 2024-09-10T23:36:58.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.322+0000 7fac71000640 1 -- 172.21.15.94:0/3584616667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6c150ad0 con 0x7fac6c15bdc0 2024-09-10T23:36:58.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.322+0000 7fac71000640 1 -- 172.21.15.94:0/3584616667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac6c151090 con 0x7fac6c15bdc0 2024-09-10T23:36:58.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.322+0000 7fac71000640 1 -- 172.21.15.94:0/3584616667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac6c0ae870 con 0x7fac6c15bdc0 2024-09-10T23:36:58.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.322+0000 7fac62a00640 1 -- 172.21.15.94:0/3584616667 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fac6400e3b0 con 0x7fac6c15bdc0 2024-09-10T23:36:58.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.322+0000 7fac62a00640 1 --2- 172.21.15.94:0/3584616667 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fac48072730 0x7fac48074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:58.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.322+0000 7fac62a00640 1 -- 172.21.15.94:0/3584616667 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fac64093f30 con 0x7fac6c15bdc0 2024-09-10T23:36:58.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.322+0000 7fac69e00640 1 --2- 172.21.15.94:0/3584616667 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fac48072730 0x7fac48074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:58.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.322+0000 7fac69e00640 1 --2- 172.21.15.94:0/3584616667 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fac48072730 0x7fac48074bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0af000 tx=0x7fac5c01b0e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:58.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.326+0000 7fac62a00640 1 -- 172.21.15.94:0/3584616667 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fac64014030 con 0x7fac6c15bdc0 2024-09-10T23:36:58.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.526+0000 7fac71000640 1 -- 172.21.15.94:0/3584616667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fac6c0d2b10 con 0x7fac6c15bdc0 2024-09-10T23:36:58.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.526+0000 7fac62a00640 1 -- 172.21.15.94:0/3584616667 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7fac6405ed90 con 0x7fac6c15bdc0 2024-09-10T23:36:58.527 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:36:58.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.526+0000 7fac71000640 1 -- 172.21.15.94:0/3584616667 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fac48072730 msgr2=0x7fac48074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:58.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.526+0000 7fac71000640 1 --2- 172.21.15.94:0/3584616667 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fac48072730 0x7fac48074bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0af000 tx=0x7fac5c01b0e0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.526+0000 7fac71000640 1 -- 172.21.15.94:0/3584616667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac6c15bdc0 msgr2=0x7fac6c1571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:58.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.526+0000 7fac71000640 1 --2- 172.21.15.94:0/3584616667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac6c15bdc0 0x7fac6c1571e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fac6400cce0 tx=0x7fac6400a620 comp rx=0 tx=0).stop 2024-09-10T23:36:58.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.526+0000 7fac71000640 1 -- 172.21.15.94:0/3584616667 shutdown_connections 2024-09-10T23:36:58.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.526+0000 7fac71000640 1 --2- 172.21.15.94:0/3584616667 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fac48072730 0x7fac48074bf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.526+0000 7fac71000640 1 --2- 172.21.15.94:0/3584616667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac6c15bdc0 0x7fac6c1571e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.526+0000 7fac71000640 1 --2- 172.21.15.94:0/3584616667 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac6c0d2210 0x7fac6c0cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.526+0000 7fac71000640 1 --2- 172.21.15.94:0/3584616667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac6c0d0780 0x7fac6c1f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:58.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.526+0000 7fac71000640 1 -- 172.21.15.94:0/3584616667 >> 172.21.15.94:0/3584616667 conn(0x7fac6c09aa70 msgr2=0x7fac6c071ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:58.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.526+0000 7fac71000640 1 -- 172.21.15.94:0/3584616667 shutdown_connections 2024-09-10T23:36:58.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58.526+0000 7fac71000640 1 -- 172.21.15.94:0/3584616667 wait complete. 2024-09-10T23:36:58.547 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:36:58.547 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:36:58.547 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T23:36:58.547 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T23:36:58.547 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T23:36:58.548 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:36:58.548 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:36:58.548 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:36:58.548 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:36:58.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:58.548 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T23:36:58.557 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:36:58.557 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:36:58.558 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:36:58.558 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:58.558 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T23:36:58.558 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:58.558 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:36:58.558 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:58.558 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T23:36:58.558 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:36:58.558 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:58.558 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T23:36:58.558 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:36:58.558 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:58.558 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:36:58.558 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:58.559 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:36:58.559 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42193sec preferred_lft 42193sec 2024-09-10T23:36:58.559 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-09-10T23:36:58.559 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:58.559 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:36:58.559 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:ed:7d:46:db brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:58.559 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:36:58.559 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:58.559 INFO:teuthology.orchestra.run.smithi094.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:36:58.559 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:36:58.559 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:36:58.559 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:58.559 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:36:58.559 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:58.559 INFO:teuthology.orchestra.run.smithi094.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:36:58.560 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:36:58.560 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:36:58.560 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:58.560 INFO:teuthology.orchestra.run.smithi094.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:36:58.560 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:36:58.560 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:36:58.560 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:36:58.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:58.560 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:36:58.618 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T23:36:58.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:58.621 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T23:36:58.673 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T23:36:58.674 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:36:58.674 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:36:58.674 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:36:58.674 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:36:58.674 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:36:58.674 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:36:58.674 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:36:58.674 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:36:58.674 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:36:58.674 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:36:58.674 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:36:58.675 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:36:58.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:36:58.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:36:58.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:36:58.675 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:36:58.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:36:58.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:36:58.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:58.811 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T23:36:58.820 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T23:36:58.820 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T23:36:58.821 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:36:58.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:58.821 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-09-10T23:36:58.884 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T23:36:58.884 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:36:58.884 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T23:36:58.937 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:36:58.937 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:36:58.937 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:36:58.992 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:36:59.052 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:36:59.106 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:36:59.107 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:36:59.107 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:36:59.107 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:36:59.107 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:36:59.107 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:36:59.107 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:36:59.107 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:36:59.107 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:36:59.107 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:36:59.107 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:36:59.107 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:36:59.107 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:36:59.108 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:36:59.108 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:36:59.108 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:36:59.108 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:36:59.108 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:36:59.108 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:36:59.108 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:36:59.108 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:36:59.108 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:36:59.108 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:36:59.108 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:36:59.108 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:36:59.108 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:36:59.109 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:36:59.109 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:36:59.109 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:36:59.109 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:36:59.109 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:36:59.109 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:36:59.109 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:36:59.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:59.110 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:36:59.161 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:36:59.161 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T23:36:59.208 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:36:59.230 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:36:59.287 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:36:59.287 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:36:59.287 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:36:59.287 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:36:59.287 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:36:59.287 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:36:59.288 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:36:59.288 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:36:59.288 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:36:59.288 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:36:59.288 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:36:59.288 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:36:59.288 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:36:59.288 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:36:59.288 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:36:59.288 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:36:59.288 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:36:59.288 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:36:59.289 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:36:59.289 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:36:59.289 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:36:59.289 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:36:59.289 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:36:59.289 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:36:59.289 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:36:59.289 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:36:59.289 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:36:59.289 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:36:59.289 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:36:59.289 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:36:59.290 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:36:59.290 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:36:59.290 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:36:59.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:59.291 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:36:59.304 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:2024-09-10T23:36:59.302+0000 7f4cbe2f0580 -1 init, newargv = 0x556b6466fc60 newargc=15 2024-09-10T23:36:59.304 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[43489]: starting ceph client 2024-09-10T23:36:59.311 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[43489]: starting fuse 2024-09-10T23:36:59.313 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T23:36:59.313 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:36:59.313 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:36:59.313 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-09-10T23:36:59.313 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-09-10T23:36:59.313 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-09-10T23:36:59.313 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-09-10T23:36:59.313 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:36:59.313 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-09-10T23:36:59.314 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T23:36:59.314 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-09-10T23:36:59.314 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T23:36:59.314 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:36:59.314 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-09-10T23:36:59.314 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-09-10T23:36:59.314 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-09-10T23:36:59.314 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-09-10T23:36:59.314 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:36:59.314 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-09-10T23:36:59.314 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:36:59.314 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:36:59.314 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:36:59.314 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-09-10T23:36:59.314 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-09-10T23:36:59.314 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-09-10T23:36:59.315 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:36:59.315 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:36:59.315 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.0")) 2024-09-10T23:36:59.315 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-09-10T23:36:59.437 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.0.43489.asok 2024-09-10T23:36:59.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:36:59.448 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43489.asok 2024-09-10T23:36:59.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:59.448 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43489.asok status 2024-09-10T23:36:59.621 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:36:59.621 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-09-10T23:36:59.621 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:36:59.622 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:36:59.622 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "0", 2024-09-10T23:36:59.622 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-09-10T23:36:59.622 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T23:36:59.622 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "43489", 2024-09-10T23:36:59.622 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-09-10T23:36:59.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:59.622 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-09-10T23:36:59.622 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-09-10T23:36:59.622 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 6804, 2024-09-10T23:36:59.622 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-09-10T23:36:59.622 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-09-10T23:36:59.623 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-09-10T23:36:59.623 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 6804 2024-09-10T23:36:59.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:59.623 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:36:59.623 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:36:59.623 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:36:59.623 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 96764277 2024-09-10T23:36:59.623 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:36:59.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:59.623 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:36:59.623 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:36:59.623 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:36:59.623 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 96764277 2024-09-10T23:36:59.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:36:59.624 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.6804 192.168.0.1:0/96764277", 2024-09-10T23:36:59.624 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/96764277", 2024-09-10T23:36:59.624 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-09-10T23:36:59.624 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 53, 2024-09-10T23:36:59.624 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 81, 2024-09-10T23:36:59.624 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:36:59.624 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-09-10T23:36:59.624 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-09-10T23:36:59.624 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:36:59.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:59.636 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-09-10T23:36:59.636 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T23:36:59.645 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-09-10T23:36:59.645 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T23:36:59.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:36:59.646 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:36:59.709 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:36:59.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.894+0000 7feac8400640 1 -- 172.21.15.94:0/1392575818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa8004d50 msgr2=0x7feaa80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:59.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.894+0000 7feac8400640 1 --2- 172.21.15.94:0/1392575818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa8004d50 0x7feaa80051d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7feab8001e60 tx=0x7feab801cbe0 comp rx=0 tx=0).stop 2024-09-10T23:36:59.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.894+0000 7feac6400640 1 -- 172.21.15.94:0/1392575818 reap_dead start 2024-09-10T23:36:59.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.894+0000 7feac8400640 1 -- 172.21.15.94:0/1392575818 shutdown_connections 2024-09-10T23:36:59.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.894+0000 7feac8400640 1 -- 172.21.15.94:0/1392575818 >> 172.21.15.94:0/1392575818 conn(0x7feac006cdf0 msgr2=0x7feac006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:36:59.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.894+0000 7feac8400640 1 -- 172.21.15.94:0/1392575818 shutdown_connections 2024-09-10T23:36:59.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.894+0000 7feac8400640 1 -- 172.21.15.94:0/1392575818 wait complete. 2024-09-10T23:36:59.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.894+0000 7feac8400640 1 Processor -- start 2024-09-10T23:36:59.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.894+0000 7feac8400640 1 -- start start 2024-09-10T23:36:59.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac8400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa8004d50 0x7feac00cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:59.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac8400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feac01f7f90 0x7feac01f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:59.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac8400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac014e5c0 0x7feac014ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:59.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac8400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feac00cff20 con 0x7feac014e5c0 2024-09-10T23:36:59.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac8400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feac00cfda0 con 0x7feaa8004d50 2024-09-10T23:36:59.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac8400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7feac00d00a0 con 0x7feac01f7f90 2024-09-10T23:36:59.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac6400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac014e5c0 0x7feac014ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:59.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac6400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac014e5c0 0x7feac014ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47758/0 (socket says 172.21.15.94:47758) 2024-09-10T23:36:59.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac6400640 1 -- 172.21.15.94:0/803343546 learned_addr learned my addr 172.21.15.94:0/803343546 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:36:59.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac5000640 1 --2- 172.21.15.94:0/803343546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feac01f7f90 0x7feac01f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:59.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac5a00640 1 --2- 172.21.15.94:0/803343546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa8004d50 0x7feac00cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:59.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac6400640 1 -- 172.21.15.94:0/803343546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feac01f7f90 msgr2=0x7feac01f4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:59.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac6400640 1 --2- 172.21.15.94:0/803343546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feac01f7f90 0x7feac01f4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:59.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac6400640 1 -- 172.21.15.94:0/803343546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa8004d50 msgr2=0x7feac00cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:36:59.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac6400640 1 --2- 172.21.15.94:0/803343546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa8004d50 0x7feac00cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:36:59.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac6400640 1 -- 172.21.15.94:0/803343546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac00b1f10 con 0x7feac014e5c0 2024-09-10T23:36:59.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac6400640 1 --2- 172.21.15.94:0/803343546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac014e5c0 0x7feac014ea50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7feab801ca10 tx=0x7feab80a7f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:59.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feabd600640 1 -- 172.21.15.94:0/803343546 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8006c50 con 0x7feac014e5c0 2024-09-10T23:36:59.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feabd600640 1 -- 172.21.15.94:0/803343546 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feab80a5ae0 con 0x7feac014e5c0 2024-09-10T23:36:59.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac8400640 1 -- 172.21.15.94:0/803343546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac00b21f0 con 0x7feac014e5c0 2024-09-10T23:36:59.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feabd600640 1 -- 172.21.15.94:0/803343546 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8003980 con 0x7feac014e5c0 2024-09-10T23:36:59.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac8400640 1 -- 172.21.15.94:0/803343546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac01f4e70 con 0x7feac014e5c0 2024-09-10T23:36:59.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.898+0000 7feac8400640 1 -- 172.21.15.94:0/803343546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac015f400 con 0x7feac014e5c0 2024-09-10T23:36:59.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.902+0000 7feabd600640 1 -- 172.21.15.94:0/803343546 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7feab80b7070 con 0x7feac014e5c0 2024-09-10T23:36:59.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.902+0000 7feabd600640 1 --2- 172.21.15.94:0/803343546 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feaa0072730 0x7feaa0074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:36:59.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.902+0000 7feabd600640 1 -- 172.21.15.94:0/803343546 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7feab812c890 con 0x7feac014e5c0 2024-09-10T23:36:59.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.902+0000 7feac5a00640 1 --2- 172.21.15.94:0/803343546 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feaa0072730 0x7feaa0074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:36:59.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.902+0000 7feac5a00640 1 --2- 172.21.15.94:0/803343546 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feaa0072730 0x7feaa0074bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feab400baf0 tx=0x7feab400b260 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:36:59.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:36:59.906+0000 7feabd600640 1 -- 172.21.15.94:0/803343546 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feab80168f0 con 0x7feac014e5c0 2024-09-10T23:37:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:00.110+0000 7feac8400640 1 -- 172.21.15.94:0/803343546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7feac01f3f10 con 0x7feac014e5c0 2024-09-10T23:37:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:00.110+0000 7feabd600640 1 -- 172.21.15.94:0/803343546 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7feab80f76f0 con 0x7feac014e5c0 2024-09-10T23:37:00.113 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:37:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:00.114+0000 7feac8400640 1 -- 172.21.15.94:0/803343546 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feaa0072730 msgr2=0x7feaa0074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:00.114+0000 7feac8400640 1 --2- 172.21.15.94:0/803343546 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feaa0072730 0x7feaa0074bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feab400baf0 tx=0x7feab400b260 comp rx=0 tx=0).stop 2024-09-10T23:37:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:00.114+0000 7feac8400640 1 -- 172.21.15.94:0/803343546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac014e5c0 msgr2=0x7feac014ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:00.114+0000 7feac8400640 1 --2- 172.21.15.94:0/803343546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac014e5c0 0x7feac014ea50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7feab801ca10 tx=0x7feab80a7f80 comp rx=0 tx=0).stop 2024-09-10T23:37:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:00.114+0000 7feac8400640 1 -- 172.21.15.94:0/803343546 shutdown_connections 2024-09-10T23:37:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:00.114+0000 7feac8400640 1 --2- 172.21.15.94:0/803343546 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feaa0072730 0x7feaa0074bf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:00.114+0000 7feac8400640 1 --2- 172.21.15.94:0/803343546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feac014e5c0 0x7feac014ea50 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:00.114+0000 7feac8400640 1 --2- 172.21.15.94:0/803343546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feac01f7f90 0x7feac01f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:00.114+0000 7feac8400640 1 --2- 172.21.15.94:0/803343546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa8004d50 0x7feac00cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:00.114+0000 7feac8400640 1 -- 172.21.15.94:0/803343546 >> 172.21.15.94:0/803343546 conn(0x7feac006cdf0 msgr2=0x7feac00cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:00.114+0000 7feac8400640 1 -- 172.21.15.94:0/803343546 shutdown_connections 2024-09-10T23:37:00.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:00.114+0000 7feac8400640 1 -- 172.21.15.94:0/803343546 wait complete. 2024-09-10T23:37:00.135 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:37:00.135 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:37:00.135 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-09-10T23:37:00.135 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T23:37:00.135 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T23:37:00.135 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:37:00.135 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:37:00.135 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:37:00.135 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:37:00.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:00.136 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-09-10T23:37:00.145 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:37:00.145 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:37:00.145 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:37:00.145 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:00.145 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-09-10T23:37:00.145 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:00.145 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:37:00.145 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:00.145 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-09-10T23:37:00.145 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:37:00.145 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:00.145 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-09-10T23:37:00.146 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:37:00.146 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:00.146 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:37:00.146 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:00.146 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:37:00.146 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42195sec preferred_lft 42195sec 2024-09-10T23:37:00.146 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-09-10T23:37:00.146 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:00.146 INFO:teuthology.orchestra.run.smithi181.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:37:00.146 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 02:42:3b:2e:09:2d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:00.146 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:37:00.146 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:00.147 INFO:teuthology.orchestra.run.smithi181.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:37:00.147 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:00.147 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:37:00.147 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:00.147 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:37:00.147 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:00.147 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:37:00.147 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:37:00.147 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:37:00.147 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:00.147 INFO:teuthology.orchestra.run.smithi181.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:37:00.147 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:37:00.147 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:37:00.147 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:00.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:00.148 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:37:00.207 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-09-10T23:37:00.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:00.209 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-09-10T23:37:00.269 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-09-10T23:37:00.269 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:37:00.269 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:37:00.269 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:37:00.270 INFO:teuthology.orchestra.run.smithi181.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:37:00.270 INFO:teuthology.orchestra.run.smithi181.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:37:00.270 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:37:00.270 INFO:teuthology.orchestra.run.smithi181.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:37:00.270 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:37:00.270 INFO:teuthology.orchestra.run.smithi181.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:37:00.270 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:37:00.270 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:37:00.270 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:37:00.270 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:37:00.271 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:37:00.271 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:37:00.271 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:37:00.359 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:37:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:37:00.401 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:37:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:37:00.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:00.408 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-09-10T23:37:00.417 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T23:37:00.417 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T23:37:00.417 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:37:00.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:00.417 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-09-10T23:37:00.480 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T23:37:00.480 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:37:00.480 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T23:37:00.529 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:37:00.529 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:37:00.530 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:37:00.580 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:37:00.640 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:37:00.694 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:37:00.694 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:37:00.694 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:37:00.694 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:37:00.694 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:37:00.694 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:37:00.694 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:37:00.694 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:37:00.694 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:37:00.695 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:37:00.695 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:37:00.695 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:37:00.695 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:37:00.695 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:37:00.695 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:37:00.695 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:37:00.695 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:37:00.695 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:37:00.695 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:37:00.695 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:37:00.695 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:37:00.695 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:37:00.695 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:37:00.696 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:37:00.696 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:37:00.696 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:37:00.696 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:37:00.696 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:37:00.696 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:37:00.696 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:37:00.696 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:37:00.696 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:37:00.696 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:37:00.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:00.697 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:37:00.753 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:37:00.753 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T23:37:00.803 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:37:00.824 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:37:00.875 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:37:00.875 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:37:00.875 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:37:00.875 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:37:00.875 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:37:00.875 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:37:00.875 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:37:00.875 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:37:00.875 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:37:00.875 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:37:00.875 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:37:00.876 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:37:00.876 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:37:00.876 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:37:00.876 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:37:00.876 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:37:00.876 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:37:00.876 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:37:00.876 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:37:00.876 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:37:00.876 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:37:00.876 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:37:00.876 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:37:00.876 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:37:00.877 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:37:00.877 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:37:00.877 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:37:00.877 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:37:00.877 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:37:00.877 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:37:00.877 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:37:00.877 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:37:00.877 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:37:00.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:00.878 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:37:00.898 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:2024-09-10T23:37:00.898+0000 7f66fe0ee580 -1 init, newargv = 0x5597f1559aa0 newargc=15 2024-09-10T23:37:00.898 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[35320]: starting ceph client 2024-09-10T23:37:00.905 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[35320]: starting fuse 2024-09-10T23:37:00.906 INFO:teuthology.orchestra.run.smithi181.stdout:44 2024-09-10T23:37:00.906 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:37:00.906 DEBUG:teuthology.orchestra.run.smithi181:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:37:00.907 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2024-09-10T23:37:00.907 DEBUG:teuthology.orchestra.run.smithi181:> import re 2024-09-10T23:37:00.907 DEBUG:teuthology.orchestra.run.smithi181:> import os 2024-09-10T23:37:00.907 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2024-09-10T23:37:00.907 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:37:00.907 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2024-09-10T23:37:00.907 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T23:37:00.907 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2024-09-10T23:37:00.907 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T23:37:00.907 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:37:00.907 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2024-09-10T23:37:00.907 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2024-09-10T23:37:00.907 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2024-09-10T23:37:00.907 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2024-09-10T23:37:00.907 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:37:00.907 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2024-09-10T23:37:00.908 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:37:00.908 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:37:00.908 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:37:00.908 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2024-09-10T23:37:00.908 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2024-09-10T23:37:00.908 DEBUG:teuthology.orchestra.run.smithi181:> return f 2024-09-10T23:37:00.908 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:37:00.908 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:37:00.908 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.1")) 2024-09-10T23:37:00.908 DEBUG:teuthology.orchestra.run.smithi181:> ' 2024-09-10T23:37:01.028 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.1.35320.asok 2024-09-10T23:37:01.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:37:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:37:01.039 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35320.asok 2024-09-10T23:37:01.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:01.040 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35320.asok status 2024-09-10T23:37:01.208 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T23:37:01.208 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2024-09-10T23:37:01.208 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:37:01.208 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:37:01.209 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "1", 2024-09-10T23:37:01.209 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2024-09-10T23:37:01.209 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T23:37:01.209 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "35320", 2024-09-10T23:37:01.209 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2024-09-10T23:37:01.209 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:37:01.209 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2024-09-10T23:37:01.209 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2024-09-10T23:37:01.209 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 195806, 2024-09-10T23:37:01.209 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2024-09-10T23:37:01.209 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2024-09-10T23:37:01.209 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2024-09-10T23:37:01.209 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 195806 2024-09-10T23:37:01.209 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:37:01.209 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:37:01.210 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:37:01.210 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:37:01.210 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 842627111 2024-09-10T23:37:01.210 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T23:37:01.210 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:37:01.210 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:37:01.210 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:37:01.210 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:37:01.210 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 842627111 2024-09-10T23:37:01.210 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:37:01.210 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.195806 192.168.0.1:0/842627111", 2024-09-10T23:37:01.210 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.1:0/842627111", 2024-09-10T23:37:01.210 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2024-09-10T23:37:01.210 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 53, 2024-09-10T23:37:01.210 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 81, 2024-09-10T23:37:01.211 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:37:01.211 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2024-09-10T23:37:01.211 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2024-09-10T23:37:01.211 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T23:37:01.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:01.223 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2024-09-10T23:37:01.223 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T23:37:01.232 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2024-09-10T23:37:01.232 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T23:37:01.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:01.232 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:37:01.297 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T23:37:01.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.454+0000 7ff0e4800640 1 -- 172.21.15.94:0/497123036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0d05a0 msgr2=0x7ff0dc15db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.454+0000 7ff0e4800640 1 --2- 172.21.15.94:0/497123036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0d05a0 0x7ff0dc15db80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00aa10 tx=0x7ff0cc01dc00 comp rx=0 tx=0).stop 2024-09-10T23:37:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.454+0000 7ff0e4800640 1 -- 172.21.15.94:0/497123036 shutdown_connections 2024-09-10T23:37:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.454+0000 7ff0e4800640 1 --2- 172.21.15.94:0/497123036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0dc0cbfc0 0x7ff0dc15e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.454+0000 7ff0e4800640 1 --2- 172.21.15.94:0/497123036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0d05a0 0x7ff0dc15db80 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.454+0000 7ff0e4800640 1 -- 172.21.15.94:0/497123036 >> 172.21.15.94:0/497123036 conn(0x7ff0dc06cdf0 msgr2=0x7ff0dc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.454+0000 7ff0e4800640 1 -- 172.21.15.94:0/497123036 shutdown_connections 2024-09-10T23:37:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.454+0000 7ff0e4800640 1 -- 172.21.15.94:0/497123036 wait complete. 2024-09-10T23:37:01.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.454+0000 7ff0e4800640 1 Processor -- start 2024-09-10T23:37:01.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.454+0000 7ff0e4800640 1 -- start start 2024-09-10T23:37:01.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e4800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0dc0cbbe0 0x7ff0dc138280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:01.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e4800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0cbfc0 0x7ff0dc1f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:01.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e4800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0dc0d05a0 0x7ff0dc1f9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:01.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e4800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc13f710 con 0x7ff0dc0cbfc0 2024-09-10T23:37:01.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e4800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc13f590 con 0x7ff0dc0cbbe0 2024-09-10T23:37:01.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e4800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff0dc13f890 con 0x7ff0dc0d05a0 2024-09-10T23:37:01.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0cbfc0 0x7ff0dc1f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:01.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0cbfc0 0x7ff0dc1f8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47788/0 (socket says 172.21.15.94:47788) 2024-09-10T23:37:01.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e1400640 1 -- 172.21.15.94:0/544862067 learned_addr learned my addr 172.21.15.94:0/544862067 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:01.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e1e00640 1 --2- 172.21.15.94:0/544862067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0dc0cbbe0 0x7ff0dc138280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:01.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e2800640 1 --2- 172.21.15.94:0/544862067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0dc0d05a0 0x7ff0dc1f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:01.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e1400640 1 -- 172.21.15.94:0/544862067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0dc0d05a0 msgr2=0x7ff0dc1f9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:01.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e1400640 1 --2- 172.21.15.94:0/544862067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0dc0d05a0 0x7ff0dc1f9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:01.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e1400640 1 -- 172.21.15.94:0/544862067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0dc0cbbe0 msgr2=0x7ff0dc138280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:01.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e1400640 1 --2- 172.21.15.94:0/544862067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0dc0cbbe0 0x7ff0dc138280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:01.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e1400640 1 -- 172.21.15.94:0/544862067 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0dc1392b0 con 0x7ff0dc0cbfc0 2024-09-10T23:37:01.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e2800640 1 --2- 172.21.15.94:0/544862067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0dc0d05a0 0x7ff0dc1f9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:01.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e1400640 1 --2- 172.21.15.94:0/544862067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0cbfc0 0x7ff0dc1f8f00 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00a960 tx=0x7ff0cc0a6ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:01.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0d6000640 1 -- 172.21.15.94:0/544862067 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc0b3070 con 0x7ff0dc0cbfc0 2024-09-10T23:37:01.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0d6000640 1 -- 172.21.15.94:0/544862067 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0cc0055f0 con 0x7ff0dc0cbfc0 2024-09-10T23:37:01.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e4800640 1 -- 172.21.15.94:0/544862067 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0dc1541e0 con 0x7ff0dc0cbfc0 2024-09-10T23:37:01.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0d6000640 1 -- 172.21.15.94:0/544862067 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc004930 con 0x7ff0dc0cbfc0 2024-09-10T23:37:01.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e4800640 1 -- 172.21.15.94:0/544862067 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0dc154480 con 0x7ff0dc0cbfc0 2024-09-10T23:37:01.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0e4800640 1 -- 172.21.15.94:0/544862067 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0dc13fdf0 con 0x7ff0dc0cbfc0 2024-09-10T23:37:01.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.458+0000 7ff0d6000640 1 -- 172.21.15.94:0/544862067 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7ff0cc00b040 con 0x7ff0dc0cbfc0 2024-09-10T23:37:01.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.462+0000 7ff0d6000640 1 --2- 172.21.15.94:0/544862067 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff0c0072700 0x7ff0c0074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:01.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.462+0000 7ff0d6000640 1 -- 172.21.15.94:0/544862067 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7ff0cc12d700 con 0x7ff0dc0cbfc0 2024-09-10T23:37:01.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.462+0000 7ff0e1e00640 1 --2- 172.21.15.94:0/544862067 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff0c0072700 0x7ff0c0074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:01.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.462+0000 7ff0e1e00640 1 --2- 172.21.15.94:0/544862067 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff0c0072700 0x7ff0c0074bc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0007fd0 tx=0x7ff0d0009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:01.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.466+0000 7ff0d6000640 1 -- 172.21.15.94:0/544862067 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff0dc1392b0 con 0x7ff0dc0cbfc0 2024-09-10T23:37:01.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:01.662+0000 7ff0e4800640 1 -- 172.21.15.94:0/544862067 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7ff0dc15c590 con 0x7ff0dc0cbfc0 2024-09-10T23:37:02.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.294+0000 7ff0d6000640 1 -- 172.21.15.94:0/544862067 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v54) ==== 136+0+0 (secure 0 0 0) 0x7ff0cc017820 con 0x7ff0dc0cbfc0 2024-09-10T23:37:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.298+0000 7ff0e4800640 1 -- 172.21.15.94:0/544862067 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff0c0072700 msgr2=0x7ff0c0074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.298+0000 7ff0e4800640 1 --2- 172.21.15.94:0/544862067 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff0c0072700 0x7ff0c0074bc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0007fd0 tx=0x7ff0d0009040 comp rx=0 tx=0).stop 2024-09-10T23:37:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.298+0000 7ff0e4800640 1 -- 172.21.15.94:0/544862067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0cbfc0 msgr2=0x7ff0dc1f8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.298+0000 7ff0e4800640 1 --2- 172.21.15.94:0/544862067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0cbfc0 0x7ff0dc1f8f00 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00a960 tx=0x7ff0cc0a6ac0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.298+0000 7ff0e4800640 1 -- 172.21.15.94:0/544862067 shutdown_connections 2024-09-10T23:37:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.298+0000 7ff0e4800640 1 --2- 172.21.15.94:0/544862067 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff0c0072700 0x7ff0c0074bc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.298+0000 7ff0e4800640 1 --2- 172.21.15.94:0/544862067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0dc0d05a0 0x7ff0dc1f9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.298+0000 7ff0e4800640 1 --2- 172.21.15.94:0/544862067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0cbfc0 0x7ff0dc1f8f00 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.298+0000 7ff0e4800640 1 --2- 172.21.15.94:0/544862067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0dc0cbbe0 0x7ff0dc138280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.298+0000 7ff0e4800640 1 -- 172.21.15.94:0/544862067 >> 172.21.15.94:0/544862067 conn(0x7ff0dc06cdf0 msgr2=0x7ff0dc0cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.298+0000 7ff0e4800640 1 -- 172.21.15.94:0/544862067 shutdown_connections 2024-09-10T23:37:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.298+0000 7ff0e4800640 1 -- 172.21.15.94:0/544862067 wait complete. 2024-09-10T23:37:02.321 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T23:37:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.462+0000 7f97d5400640 1 -- 172.21.15.94:0/4150189445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97d006ce80 msgr2=0x7f97d015f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.462+0000 7f97d5400640 1 --2- 172.21.15.94:0/4150189445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97d006ce80 0x7f97d015f820 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009900 tx=0x7f97c001c6d0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.462+0000 7f97d5400640 1 -- 172.21.15.94:0/4150189445 shutdown_connections 2024-09-10T23:37:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.462+0000 7f97d5400640 1 --2- 172.21.15.94:0/4150189445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97d006d260 0x7f97d015ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.462+0000 7f97d5400640 1 --2- 172.21.15.94:0/4150189445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97d006ce80 0x7f97d015f820 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.462+0000 7f97d5400640 1 -- 172.21.15.94:0/4150189445 >> 172.21.15.94:0/4150189445 conn(0x7f97d00d0d80 msgr2=0x7f97d00d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97d5400640 1 -- 172.21.15.94:0/4150189445 shutdown_connections 2024-09-10T23:37:02.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97d5400640 1 -- 172.21.15.94:0/4150189445 wait complete. 2024-09-10T23:37:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97d5400640 1 Processor -- start 2024-09-10T23:37:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97d5400640 1 -- start start 2024-09-10T23:37:02.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97d5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97d006ce80 0x7f97d013cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:02.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97d5400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97d006d260 0x7f97d013d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:02.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97d5400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d013d790 0x7f97d013e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:02.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97d5400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f97d0141970 con 0x7f97d006ce80 2024-09-10T23:37:02.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97d5400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f97d01417f0 con 0x7f97d013d790 2024-09-10T23:37:02.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97d5400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f97d0141af0 con 0x7f97d006d260 2024-09-10T23:37:02.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97ce800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97d006ce80 0x7f97d013cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:02.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97ce800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97d006ce80 0x7f97d013cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47804/0 (socket says 172.21.15.94:47804) 2024-09-10T23:37:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97ce800640 1 -- 172.21.15.94:0/3205839115 learned_addr learned my addr 172.21.15.94:0/3205839115 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97cf200640 1 --2- 172.21.15.94:0/3205839115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d013d790 0x7f97d013e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97c5e00640 1 --2- 172.21.15.94:0/3205839115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97d006d260 0x7f97d013d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97ce800640 1 -- 172.21.15.94:0/3205839115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97d006d260 msgr2=0x7f97d013d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97ce800640 1 --2- 172.21.15.94:0/3205839115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97d006d260 0x7f97d013d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97ce800640 1 -- 172.21.15.94:0/3205839115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d013d790 msgr2=0x7f97d013e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97ce800640 1 --2- 172.21.15.94:0/3205839115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d013d790 0x7f97d013e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97ce800640 1 -- 172.21.15.94:0/3205839115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d0153e00 con 0x7f97d006ce80 2024-09-10T23:37:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97c5e00640 1 --2- 172.21.15.94:0/3205839115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97d006d260 0x7f97d013d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97ce800640 1 --2- 172.21.15.94:0/3205839115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97d006ce80 0x7f97d013cb60 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f97c001cf70 tx=0x7f97c0005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97c7400640 1 -- 172.21.15.94:0/3205839115 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c00ac070 con 0x7f97d006ce80 2024-09-10T23:37:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97c7400640 1 -- 172.21.15.94:0/3205839115 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97c0004320 con 0x7f97d006ce80 2024-09-10T23:37:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97c7400640 1 -- 172.21.15.94:0/3205839115 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c00052a0 con 0x7f97d006ce80 2024-09-10T23:37:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.466+0000 7f97d5400640 1 -- 172.21.15.94:0/3205839115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d01540e0 con 0x7f97d006ce80 2024-09-10T23:37:02.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.470+0000 7f97d5400640 1 -- 172.21.15.94:0/3205839115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d01546a0 con 0x7f97d006ce80 2024-09-10T23:37:02.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.470+0000 7f97d5400640 1 -- 172.21.15.94:0/3205839115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d0142070 con 0x7f97d006ce80 2024-09-10T23:37:02.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.470+0000 7f97c7400640 1 -- 172.21.15.94:0/3205839115 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f97c00b8020 con 0x7f97d006ce80 2024-09-10T23:37:02.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.470+0000 7f97c7400640 1 --2- 172.21.15.94:0/3205839115 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f97ac072700 0x7f97ac074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:02.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.470+0000 7f97c7400640 1 -- 172.21.15.94:0/3205839115 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f97c012d150 con 0x7f97d006ce80 2024-09-10T23:37:02.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.470+0000 7f97c5e00640 1 --2- 172.21.15.94:0/3205839115 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f97ac072700 0x7f97ac074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:02.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.470+0000 7f97c5e00640 1 --2- 172.21.15.94:0/3205839115 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f97ac072700 0x7f97ac074bc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f97b80058d0 tx=0x7f97b8005860 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.474+0000 7f97c7400640 1 -- 172.21.15.94:0/3205839115 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f97c00bf070 con 0x7f97d006ce80 2024-09-10T23:37:02.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.662+0000 7f97d5400640 1 -- 172.21.15.94:0/3205839115 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f97d0073700 con 0x7f97ac072700 2024-09-10T23:37:02.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.666+0000 7f97c7400640 1 -- 172.21.15.94:0/3205839115 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f97d0073700 con 0x7f97ac072700 2024-09-10T23:37:02.666 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:37:02.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:02.666 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephfs" 2024-09-10T23:37:02.667 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:02.667 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:37:02.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.666+0000 7f97d5400640 1 -- 172.21.15.94:0/3205839115 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f97ac072700 msgr2=0x7f97ac074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:02.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.666+0000 7f97d5400640 1 --2- 172.21.15.94:0/3205839115 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f97ac072700 0x7f97ac074bc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f97b80058d0 tx=0x7f97b8005860 comp rx=0 tx=0).stop 2024-09-10T23:37:02.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.666+0000 7f97d5400640 1 -- 172.21.15.94:0/3205839115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97d006ce80 msgr2=0x7f97d013cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:02.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.666+0000 7f97d5400640 1 --2- 172.21.15.94:0/3205839115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97d006ce80 0x7f97d013cb60 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f97c001cf70 tx=0x7f97c0005e90 comp rx=0 tx=0).stop 2024-09-10T23:37:02.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.666+0000 7f97d5400640 1 -- 172.21.15.94:0/3205839115 shutdown_connections 2024-09-10T23:37:02.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.666+0000 7f97d5400640 1 --2- 172.21.15.94:0/3205839115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97d013d790 0x7f97d013e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.666+0000 7f97d5400640 1 --2- 172.21.15.94:0/3205839115 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f97ac072700 0x7f97ac074bc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.666+0000 7f97d5400640 1 --2- 172.21.15.94:0/3205839115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97d006d260 0x7f97d013d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.666+0000 7f97d5400640 1 --2- 172.21.15.94:0/3205839115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97d006ce80 0x7f97d013cb60 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.666+0000 7f97d5400640 1 -- 172.21.15.94:0/3205839115 >> 172.21.15.94:0/3205839115 conn(0x7f97d00d0d80 msgr2=0x7f97d0073920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:02.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.666+0000 7f97d5400640 1 -- 172.21.15.94:0/3205839115 shutdown_connections 2024-09-10T23:37:02.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.666+0000 7f97d5400640 1 -- 172.21.15.94:0/3205839115 wait complete. 2024-09-10T23:37:02.687 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:37:02.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.830+0000 7f3373a00640 1 -- 172.21.15.94:0/3966849466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f336c0d0710 msgr2=0x7f336c0c5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:02.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.830+0000 7f3373a00640 1 --2- 172.21.15.94:0/3966849466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f336c0d0710 0x7f336c0c5530 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3354009900 tx=0x7f335401c870 comp rx=0 tx=0).stop 2024-09-10T23:37:02.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3373a00640 1 -- 172.21.15.94:0/3966849466 shutdown_connections 2024-09-10T23:37:02.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3373a00640 1 --2- 172.21.15.94:0/3966849466 >> v2:172.21.15.181:3300/0 conn(0x7f336c0c5a70 0x7f336c0cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3373a00640 1 --2- 172.21.15.94:0/3966849466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f336c0d0710 0x7f336c0c5530 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3373a00640 1 --2- 172.21.15.94:0/3966849466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f336c1455d0 0x7f336c0c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3373a00640 1 -- 172.21.15.94:0/3966849466 >> 172.21.15.94:0/3966849466 conn(0x7f336c0cfaf0 msgr2=0x7f336c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3373a00640 1 -- 172.21.15.94:0/3966849466 shutdown_connections 2024-09-10T23:37:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3373a00640 1 -- 172.21.15.94:0/3966849466 wait complete. 2024-09-10T23:37:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3373a00640 1 Processor -- start 2024-09-10T23:37:02.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3373a00640 1 -- start start 2024-09-10T23:37:02.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3373a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f336c0c5a70 0x7f336c13b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:02.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3373a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f336c0d0710 0x7f336c0ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:02.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3373a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f336c1455d0 0x7f336c158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:02.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3373a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f336c15ff00 con 0x7f336c0c5a70 2024-09-10T23:37:02.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3373a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f336c15fd80 con 0x7f336c0d0710 2024-09-10T23:37:02.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3373a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f336c160080 con 0x7f336c1455d0 2024-09-10T23:37:02.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3371000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f336c0c5a70 0x7f336c13b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:02.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3371000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f336c0c5a70 0x7f336c13b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47822/0 (socket says 172.21.15.94:47822) 2024-09-10T23:37:02.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3371000640 1 -- 172.21.15.94:0/740699038 learned_addr learned my addr 172.21.15.94:0/740699038 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:02.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3371a00640 1 --2- 172.21.15.94:0/740699038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f336c1455d0 0x7f336c158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:02.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3363e00640 1 --2- 172.21.15.94:0/740699038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f336c0d0710 0x7f336c0ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:02.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3371000640 1 -- 172.21.15.94:0/740699038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f336c1455d0 msgr2=0x7f336c158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:02.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3371000640 1 --2- 172.21.15.94:0/740699038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f336c1455d0 0x7f336c158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3371000640 1 -- 172.21.15.94:0/740699038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f336c0d0710 msgr2=0x7f336c0ae2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:02.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3371000640 1 --2- 172.21.15.94:0/740699038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f336c0d0710 0x7f336c0ae2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:02.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3371000640 1 -- 172.21.15.94:0/740699038 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c13bbe0 con 0x7f336c0c5a70 2024-09-10T23:37:02.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3371a00640 1 --2- 172.21.15.94:0/740699038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f336c1455d0 0x7f336c158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:37:02.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3371000640 1 --2- 172.21.15.94:0/740699038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f336c0c5a70 0x7f336c13b350 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f335c0058d0 tx=0x7f335c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:02.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3369600640 1 -- 172.21.15.94:0/740699038 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c017070 con 0x7f336c0c5a70 2024-09-10T23:37:02.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3369600640 1 -- 172.21.15.94:0/740699038 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f335c009d20 con 0x7f336c0c5a70 2024-09-10T23:37:02.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3373a00640 1 -- 172.21.15.94:0/740699038 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336c1401e0 con 0x7f336c0c5a70 2024-09-10T23:37:02.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.834+0000 7f3369600640 1 -- 172.21.15.94:0/740699038 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0106c0 con 0x7f336c0c5a70 2024-09-10T23:37:02.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.838+0000 7f3373a00640 1 -- 172.21.15.94:0/740699038 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336c0ca2e0 con 0x7f336c0c5a70 2024-09-10T23:37:02.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.838+0000 7f3373a00640 1 -- 172.21.15.94:0/740699038 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f336c160610 con 0x7f336c0c5a70 2024-09-10T23:37:02.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.838+0000 7f3369600640 1 -- 172.21.15.94:0/740699038 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f335c010860 con 0x7f336c0c5a70 2024-09-10T23:37:02.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.838+0000 7f3369600640 1 --2- 172.21.15.94:0/740699038 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3348072730 0x7f3348074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:02.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.838+0000 7f3369600640 1 -- 172.21.15.94:0/740699038 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f335c093990 con 0x7f336c0c5a70 2024-09-10T23:37:02.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.838+0000 7f3363e00640 1 --2- 172.21.15.94:0/740699038 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3348072730 0x7f3348074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:02.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.838+0000 7f3363e00640 1 --2- 172.21.15.94:0/740699038 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3348072730 0x7f3348074bf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f33540af000 tx=0x7f335401b0b0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:02.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:02.842+0000 7f3369600640 1 -- 172.21.15.94:0/740699038 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f335c05edc0 con 0x7f336c0c5a70 2024-09-10T23:37:03.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.010+0000 7f3373a00640 1 -- 172.21.15.94:0/740699038 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f336c0cf6e0 con 0x7f336c0c5a70 2024-09-10T23:37:03.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.010+0000 7f3369600640 1 -- 172.21.15.94:0/740699038 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f335c05e6c0 con 0x7f336c0c5a70 2024-09-10T23:37:03.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.014+0000 7f3373a00640 1 -- 172.21.15.94:0/740699038 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3348072730 msgr2=0x7f3348074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:03.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.014+0000 7f3373a00640 1 --2- 172.21.15.94:0/740699038 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3348072730 0x7f3348074bf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f33540af000 tx=0x7f335401b0b0 comp rx=0 tx=0).stop 2024-09-10T23:37:03.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.014+0000 7f3373a00640 1 -- 172.21.15.94:0/740699038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f336c0c5a70 msgr2=0x7f336c13b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:03.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.014+0000 7f3373a00640 1 --2- 172.21.15.94:0/740699038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f336c0c5a70 0x7f336c13b350 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f335c0058d0 tx=0x7f335c00dde0 comp rx=0 tx=0).stop 2024-09-10T23:37:03.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.014+0000 7f3373a00640 1 -- 172.21.15.94:0/740699038 shutdown_connections 2024-09-10T23:37:03.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.014+0000 7f3373a00640 1 --2- 172.21.15.94:0/740699038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f336c0d0710 0x7f336c0ae2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:03.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.014+0000 7f3373a00640 1 --2- 172.21.15.94:0/740699038 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3348072730 0x7f3348074bf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:03.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.014+0000 7f3373a00640 1 --2- 172.21.15.94:0/740699038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f336c1455d0 0x7f336c158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:03.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.014+0000 7f3373a00640 1 --2- 172.21.15.94:0/740699038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f336c0c5a70 0x7f336c13b350 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:03.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.014+0000 7f3373a00640 1 -- 172.21.15.94:0/740699038 >> 172.21.15.94:0/740699038 conn(0x7f336c0cfaf0 msgr2=0x7f336c0b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:03.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.014+0000 7f3373a00640 1 -- 172.21.15.94:0/740699038 shutdown_connections 2024-09-10T23:37:03.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.014+0000 7f3373a00640 1 -- 172.21.15.94:0/740699038 wait complete. 2024-09-10T23:37:03.034 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2024-09-10T23:37:03.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.174+0000 7fa37c000640 1 -- 172.21.15.94:0/3373616503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3740ce1e0 msgr2=0x7fa3740cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:03.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.174+0000 7fa37c000640 1 --2- 172.21.15.94:0/3373616503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3740ce1e0 0x7fa3740cec70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa360009900 tx=0x7fa36001c820 comp rx=0 tx=0).stop 2024-09-10T23:37:03.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.174+0000 7fa37c000640 1 -- 172.21.15.94:0/3373616503 shutdown_connections 2024-09-10T23:37:03.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.174+0000 7fa37c000640 1 --2- 172.21.15.94:0/3373616503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3740ce1e0 0x7fa3740cec70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:03.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.174+0000 7fa37c000640 1 --2- 172.21.15.94:0/3373616503 >> v2:172.21.15.181:3300/0 conn(0x7fa3740ce890 0x7fa3740cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:03.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.174+0000 7fa37c000640 1 -- 172.21.15.94:0/3373616503 >> 172.21.15.94:0/3373616503 conn(0x7fa3740d1c60 msgr2=0x7fa3740d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:03.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.174+0000 7fa37c000640 1 -- 172.21.15.94:0/3373616503 shutdown_connections 2024-09-10T23:37:03.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.174+0000 7fa37c000640 1 -- 172.21.15.94:0/3373616503 wait complete. 2024-09-10T23:37:03.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa37c000640 1 Processor -- start 2024-09-10T23:37:03.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa37c000640 1 -- start start 2024-09-10T23:37:03.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa37c000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3740b1100 0x7fa3740b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:03.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa37c000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3740ce1e0 0x7fa37415e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:03.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa37c000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3740ce890 0x7fa374158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:03.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa37c000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa3740d08e0 con 0x7fa3740b1100 2024-09-10T23:37:03.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa37c000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa3740d0760 con 0x7fa3740ce890 2024-09-10T23:37:03.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa37c000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa3740d0a60 con 0x7fa3740ce1e0 2024-09-10T23:37:03.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa379600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3740b1100 0x7fa3740b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:03.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa379600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3740b1100 0x7fa3740b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47828/0 (socket says 172.21.15.94:47828) 2024-09-10T23:37:03.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa379600640 1 -- 172.21.15.94:0/811139904 learned_addr learned my addr 172.21.15.94:0/811139904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:03.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa379600640 1 -- 172.21.15.94:0/811139904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3740ce1e0 msgr2=0x7fa37415e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:03.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa379600640 1 --2- 172.21.15.94:0/811139904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3740ce1e0 0x7fa37415e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:03.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa379600640 1 -- 172.21.15.94:0/811139904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3740ce890 msgr2=0x7fa374158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:03.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa379600640 1 --2- 172.21.15.94:0/811139904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3740ce890 0x7fa374158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:03.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa379600640 1 -- 172.21.15.94:0/811139904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3740b73d0 con 0x7fa3740b1100 2024-09-10T23:37:03.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa379600640 1 --2- 172.21.15.94:0/811139904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3740b1100 0x7fa3740b4cc0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa36800ed00 tx=0x7fa36800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:03.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa371600640 1 -- 172.21.15.94:0/811139904 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368003b80 con 0x7fa3740b1100 2024-09-10T23:37:03.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa371600640 1 -- 172.21.15.94:0/811139904 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa368003d20 con 0x7fa3740b1100 2024-09-10T23:37:03.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa371600640 1 -- 172.21.15.94:0/811139904 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368004280 con 0x7fa3740b1100 2024-09-10T23:37:03.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa37c000640 1 -- 172.21.15.94:0/811139904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3740b76b0 con 0x7fa3740b1100 2024-09-10T23:37:03.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.178+0000 7fa37c000640 1 -- 172.21.15.94:0/811139904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3740b58d0 con 0x7fa3740b1100 2024-09-10T23:37:03.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.182+0000 7fa37c000640 1 -- 172.21.15.94:0/811139904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3740cec70 con 0x7fa3740b1100 2024-09-10T23:37:03.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.182+0000 7fa371600640 1 -- 172.21.15.94:0/811139904 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fa368004420 con 0x7fa3740b1100 2024-09-10T23:37:03.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.182+0000 7fa371600640 1 --2- 172.21.15.94:0/811139904 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa354072700 0x7fa354074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:03.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.182+0000 7fa371600640 1 -- 172.21.15.94:0/811139904 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fa368094240 con 0x7fa3740b1100 2024-09-10T23:37:03.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.182+0000 7fa378c00640 1 --2- 172.21.15.94:0/811139904 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa354072700 0x7fa354074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:03.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.182+0000 7fa378c00640 1 --2- 172.21.15.94:0/811139904 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa354072700 0x7fa354074bc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa3600af000 tx=0x7fa3600195c0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:03.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.186+0000 7fa371600640 1 -- 172.21.15.94:0/811139904 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa36805f6f0 con 0x7fa3740b1100 2024-09-10T23:37:03.377 INFO:teuthology.orchestra.run.smithi094.stderr:Invalid command: invalid chars @# in abcd@# 2024-09-10T23:37:03.377 INFO:teuthology.orchestra.run.smithi094.stderr:fs volume create [] : Create a CephFS volume 2024-09-10T23:37:03.377 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-10T23:37:03.384 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T23:37:03.385 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2024-09-10T23:37:03.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.526+0000 7fa9d8e00640 1 -- 172.21.15.94:0/3254456010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d40cb4d0 msgr2=0x7fa9d415b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.526+0000 7fa9d8e00640 1 --2- 172.21.15.94:0/3254456010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d40cb4d0 0x7fa9d415b190 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40c6b50 tx=0x7fa9cc01c8c0 comp rx=0 tx=0).stop 2024-09-10T23:37:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.526+0000 7fa9d8e00640 1 -- 172.21.15.94:0/3254456010 shutdown_connections 2024-09-10T23:37:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.526+0000 7fa9d8e00640 1 --2- 172.21.15.94:0/3254456010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d40cb4d0 0x7fa9d415b190 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.526+0000 7fa9d8e00640 1 --2- 172.21.15.94:0/3254456010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9d4158790 0x7fa9d406fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.526+0000 7fa9d8e00640 1 -- 172.21.15.94:0/3254456010 >> 172.21.15.94:0/3254456010 conn(0x7fa9d40cfaf0 msgr2=0x7fa9d40cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.526+0000 7fa9d8e00640 1 -- 172.21.15.94:0/3254456010 shutdown_connections 2024-09-10T23:37:03.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.526+0000 7fa9d8e00640 1 -- 172.21.15.94:0/3254456010 wait complete. 2024-09-10T23:37:03.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d8e00640 1 Processor -- start 2024-09-10T23:37:03.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d8e00640 1 -- start start 2024-09-10T23:37:03.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d40c49b0 0x7fa9d4149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:03.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d8e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9d40cb4d0 0x7fa9d41fb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:03.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d8e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9d4158790 0x7fa9d40ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d8e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4167fc0 con 0x7fa9d40c49b0 2024-09-10T23:37:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d8e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4167e40 con 0x7fa9d40cb4d0 2024-09-10T23:37:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d8e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d4168140 con 0x7fa9d4158790 2024-09-10T23:37:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d40c49b0 0x7fa9d4149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d40c49b0 0x7fa9d4149150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47838/0 (socket says 172.21.15.94:47838) 2024-09-10T23:37:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d1e00640 1 -- 172.21.15.94:0/3734036647 learned_addr learned my addr 172.21.15.94:0/3734036647 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d1e00640 1 -- 172.21.15.94:0/3734036647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9d4158790 msgr2=0x7fa9d40ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d1e00640 1 --2- 172.21.15.94:0/3734036647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9d4158790 0x7fa9d40ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d1e00640 1 -- 172.21.15.94:0/3734036647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9d40cb4d0 msgr2=0x7fa9d41fb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d1e00640 1 --2- 172.21.15.94:0/3734036647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9d40cb4d0 0x7fa9d41fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d1e00640 1 -- 172.21.15.94:0/3734036647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d4074380 con 0x7fa9d40c49b0 2024-09-10T23:37:03.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d1e00640 1 --2- 172.21.15.94:0/3734036647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d40c49b0 0x7fa9d4149150 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400eaa0 tx=0x7fa9c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:03.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9ca000640 1 -- 172.21.15.94:0/3734036647 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400cdb0 con 0x7fa9d40c49b0 2024-09-10T23:37:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9ca000640 1 -- 172.21.15.94:0/3734036647 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9c4011470 con 0x7fa9d40c49b0 2024-09-10T23:37:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9ca000640 1 -- 172.21.15.94:0/3734036647 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c40118a0 con 0x7fa9d40c49b0 2024-09-10T23:37:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d8e00640 1 -- 172.21.15.94:0/3734036647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d4074660 con 0x7fa9d40c49b0 2024-09-10T23:37:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d8e00640 1 -- 172.21.15.94:0/3734036647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d41497b0 con 0x7fa9d40c49b0 2024-09-10T23:37:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.530+0000 7fa9d8e00640 1 -- 172.21.15.94:0/3734036647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d41fb2e0 con 0x7fa9d40c49b0 2024-09-10T23:37:03.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.534+0000 7fa9ca000640 1 -- 172.21.15.94:0/3734036647 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fa9c400f040 con 0x7fa9d40c49b0 2024-09-10T23:37:03.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.534+0000 7fa9ca000640 1 --2- 172.21.15.94:0/3734036647 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa9b4072730 0x7fa9b4074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:03.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.534+0000 7fa9ca000640 1 -- 172.21.15.94:0/3734036647 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fa9c4093e00 con 0x7fa9d40c49b0 2024-09-10T23:37:03.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.534+0000 7fa9d1400640 1 --2- 172.21.15.94:0/3734036647 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa9b4072730 0x7fa9b4074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.534+0000 7fa9d1400640 1 --2- 172.21.15.94:0/3734036647 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa9b4072730 0x7fa9b4074bf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa9d41518c0 tx=0x7fa9c00079c0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:03.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.538+0000 7fa9ca000640 1 -- 172.21.15.94:0/3734036647 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa9c4016030 con 0x7fa9d40c49b0 2024-09-10T23:37:03.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:03.726+0000 7fa9d8e00640 1 -- 172.21.15.94:0/3734036647 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) -- 0x7fa9d415b190 con 0x7fa9b4072730 2024-09-10T23:37:05.114 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:37:05.118 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:37:06.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.326+0000 7fa9ca000640 1 -- 172.21.15.94:0/3734036647 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) ==== 60+0+0 (secure 0 0 0) 0x7fa9d415b190 con 0x7fa9b4072730 2024-09-10T23:37:06.330 INFO:teuthology.orchestra.run.smithi094.stderr:Volume created successfully (no MDS daemons created) 2024-09-10T23:37:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.334+0000 7fa9d8e00640 1 -- 172.21.15.94:0/3734036647 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa9b4072730 msgr2=0x7fa9b4074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.334+0000 7fa9d8e00640 1 --2- 172.21.15.94:0/3734036647 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa9b4072730 0x7fa9b4074bf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa9d41518c0 tx=0x7fa9c00079c0 comp rx=0 tx=0).stop 2024-09-10T23:37:06.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.334+0000 7fa9d8e00640 1 -- 172.21.15.94:0/3734036647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d40c49b0 msgr2=0x7fa9d4149150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.334+0000 7fa9d8e00640 1 --2- 172.21.15.94:0/3734036647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d40c49b0 0x7fa9d4149150 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400eaa0 tx=0x7fa9c400ef60 comp rx=0 tx=0).stop 2024-09-10T23:37:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.334+0000 7fa9d8e00640 1 -- 172.21.15.94:0/3734036647 shutdown_connections 2024-09-10T23:37:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.334+0000 7fa9d8e00640 1 --2- 172.21.15.94:0/3734036647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9d40cb4d0 0x7fa9d41fb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.334+0000 7fa9d8e00640 1 --2- 172.21.15.94:0/3734036647 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa9b4072730 0x7fa9b4074bf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.334+0000 7fa9d8e00640 1 --2- 172.21.15.94:0/3734036647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9d4158790 0x7fa9d40ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.334+0000 7fa9d8e00640 1 --2- 172.21.15.94:0/3734036647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9d40c49b0 0x7fa9d4149150 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.334+0000 7fa9d8e00640 1 -- 172.21.15.94:0/3734036647 >> 172.21.15.94:0/3734036647 conn(0x7fa9d40cfaf0 msgr2=0x7fa9d40b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.334+0000 7fa9d8e00640 1 -- 172.21.15.94:0/3734036647 shutdown_connections 2024-09-10T23:37:06.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.334+0000 7fa9d8e00640 1 -- 172.21.15.94:0/3734036647 wait complete. 2024-09-10T23:37:06.365 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2024-09-10T23:37:06.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.510+0000 7f485aa00640 1 -- 172.21.15.94:0/4232175737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48540c6e60 msgr2=0x7f4854166970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:06.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.510+0000 7f485aa00640 1 --2- 172.21.15.94:0/4232175737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48540c6e60 0x7f4854166970 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f484c009cb0 tx=0x7f484c01cd10 comp rx=0 tx=0).stop 2024-09-10T23:37:06.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.510+0000 7f485aa00640 1 -- 172.21.15.94:0/4232175737 shutdown_connections 2024-09-10T23:37:06.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.510+0000 7f485aa00640 1 --2- 172.21.15.94:0/4232175737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48540c6e60 0x7f4854166970 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:06.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.510+0000 7f485aa00640 1 -- 172.21.15.94:0/4232175737 >> 172.21.15.94:0/4232175737 conn(0x7f48540d1c60 msgr2=0x7f48540d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:06.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.510+0000 7f485aa00640 1 -- 172.21.15.94:0/4232175737 shutdown_connections 2024-09-10T23:37:06.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.510+0000 7f485aa00640 1 -- 172.21.15.94:0/4232175737 wait complete. 2024-09-10T23:37:06.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.510+0000 7f485aa00640 1 Processor -- start 2024-09-10T23:37:06.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.510+0000 7f485aa00640 1 -- start start 2024-09-10T23:37:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f485aa00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48540c6e60 0x7f48540b7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f485aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48540d2780 0x7f485415e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f485aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f485415bf60 0x7f4854150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f485aa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4854169ce0 con 0x7f48540d2780 2024-09-10T23:37:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f485aa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4854169b60 con 0x7f485415bf60 2024-09-10T23:37:06.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f485aa00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f4854169e60 con 0x7f48540c6e60 2024-09-10T23:37:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f484be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48540d2780 0x7f485415e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f484be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48540d2780 0x7f485415e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54944/0 (socket says 172.21.15.94:54944) 2024-09-10T23:37:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f484be00640 1 -- 172.21.15.94:0/2386383709 learned_addr learned my addr 172.21.15.94:0/2386383709 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f4853e00640 1 --2- 172.21.15.94:0/2386383709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48540c6e60 0x7f48540b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f4858a00640 1 --2- 172.21.15.94:0/2386383709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f485415bf60 0x7f4854150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f484be00640 1 -- 172.21.15.94:0/2386383709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48540c6e60 msgr2=0x7f48540b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f484be00640 1 --2- 172.21.15.94:0/2386383709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48540c6e60 0x7f48540b7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:06.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f484be00640 1 -- 172.21.15.94:0/2386383709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f485415bf60 msgr2=0x7f4854150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f484be00640 1 --2- 172.21.15.94:0/2386383709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f485415bf60 0x7f4854150ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f484be00640 1 -- 172.21.15.94:0/2386383709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48540b6570 con 0x7f48540d2780 2024-09-10T23:37:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f484be00640 1 --2- 172.21.15.94:0/2386383709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48540d2780 0x7f485415e860 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f483c00c450 tx=0x7f483c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f4850c00640 1 -- 172.21.15.94:0/2386383709 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c012830 con 0x7f48540d2780 2024-09-10T23:37:06.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f4850c00640 1 -- 172.21.15.94:0/2386383709 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f483c017070 con 0x7f48540d2780 2024-09-10T23:37:06.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f485aa00640 1 -- 172.21.15.94:0/2386383709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48540b6850 con 0x7f48540d2780 2024-09-10T23:37:06.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f4850c00640 1 -- 172.21.15.94:0/2386383709 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c012d80 con 0x7f48540d2780 2024-09-10T23:37:06.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f485aa00640 1 -- 172.21.15.94:0/2386383709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48540b6e10 con 0x7f48540d2780 2024-09-10T23:37:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.514+0000 7f485aa00640 1 -- 172.21.15.94:0/2386383709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48540cfce0 con 0x7f48540d2780 2024-09-10T23:37:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.518+0000 7f4850c00640 1 -- 172.21.15.94:0/2386383709 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f483c01c460 con 0x7f48540d2780 2024-09-10T23:37:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.518+0000 7f4850c00640 1 --2- 172.21.15.94:0/2386383709 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4830072700 0x7f4830074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.518+0000 7f4850c00640 1 -- 172.21.15.94:0/2386383709 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6543+0+0 (secure 0 0 0) 0x7f483c093620 con 0x7f48540d2780 2024-09-10T23:37:06.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.518+0000 7f4853e00640 1 --2- 172.21.15.94:0/2386383709 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4830072700 0x7f4830074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:06.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.518+0000 7f4853e00640 1 --2- 172.21.15.94:0/2386383709 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4830072700 0x7f4830074bc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4844000f70 tx=0x7f4844009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:06.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.522+0000 7f4850c00640 1 -- 172.21.15.94:0/2386383709 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f483c05e790 con 0x7f48540d2780 2024-09-10T23:37:06.711 INFO:teuthology.orchestra.run.smithi094.stderr:Invalid command: invalid chars @# in abcd@# 2024-09-10T23:37:06.711 INFO:teuthology.orchestra.run.smithi094.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2024-09-10T23:37:06.711 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-10T23:37:06.718 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T23:37:06.719 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2024-09-10T23:37:06.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.862+0000 7fb9d8a00640 1 -- 172.21.15.94:0/2683732045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b8005220 msgr2=0x7fb9b800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:06.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.862+0000 7fb9d8a00640 1 --2- 172.21.15.94:0/2683732045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b8005220 0x7fb9b800b9d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc006fd0 tx=0x7fb9cc01cad0 comp rx=0 tx=0).stop 2024-09-10T23:37:06.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.862+0000 7fb9d8a00640 1 -- 172.21.15.94:0/2683732045 shutdown_connections 2024-09-10T23:37:06.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.862+0000 7fb9d8a00640 1 --2- 172.21.15.94:0/2683732045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9b8005220 0x7fb9b800b9d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:06.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.862+0000 7fb9d8a00640 1 -- 172.21.15.94:0/2683732045 >> 172.21.15.94:0/2683732045 conn(0x7fb9d406e610 msgr2=0x7fb9d40b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:06.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.862+0000 7fb9d8a00640 1 -- 172.21.15.94:0/2683732045 shutdown_connections 2024-09-10T23:37:06.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.862+0000 7fb9d8a00640 1 -- 172.21.15.94:0/2683732045 wait complete. 2024-09-10T23:37:06.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9d8a00640 1 Processor -- start 2024-09-10T23:37:06.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9d8a00640 1 -- start start 2024-09-10T23:37:06.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9d8a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b8005220 0x7fb9d40c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:06.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9d8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9d41f8c70 0x7fb9d40c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:06.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9d8a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9d414f620 0x7fb9d414faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:06.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9d8a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d40af540 con 0x7fb9d41f8c70 2024-09-10T23:37:06.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9d8a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d40cd3e0 con 0x7fb9b8005220 2024-09-10T23:37:06.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9d8a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb9d40af6c0 con 0x7fb9d414f620 2024-09-10T23:37:06.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9c9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9d41f8c70 0x7fb9d40c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:06.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9c9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9d41f8c70 0x7fb9d40c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54958/0 (socket says 172.21.15.94:54958) 2024-09-10T23:37:06.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9d2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b8005220 0x7fb9d40c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9c9e00640 1 -- 172.21.15.94:0/506165916 learned_addr learned my addr 172.21.15.94:0/506165916 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9d3200640 1 --2- 172.21.15.94:0/506165916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9d414f620 0x7fb9d414faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9c9e00640 1 -- 172.21.15.94:0/506165916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9d414f620 msgr2=0x7fb9d414faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9c9e00640 1 --2- 172.21.15.94:0/506165916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9d414f620 0x7fb9d414faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9c9e00640 1 -- 172.21.15.94:0/506165916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b8005220 msgr2=0x7fb9d40c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9c9e00640 1 --2- 172.21.15.94:0/506165916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b8005220 0x7fb9d40c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9c9e00640 1 -- 172.21.15.94:0/506165916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d40ca330 con 0x7fb9d41f8c70 2024-09-10T23:37:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9d3200640 1 --2- 172.21.15.94:0/506165916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9d414f620 0x7fb9d414faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:06.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9c9e00640 1 --2- 172.21.15.94:0/506165916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9d41f8c70 0x7fb9d40c6f00 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc011ae0 tx=0x7fb9bc009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9cb400640 1 -- 172.21.15.94:0/506165916 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc01e070 con 0x7fb9d41f8c70 2024-09-10T23:37:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9cb400640 1 -- 172.21.15.94:0/506165916 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9bc004460 con 0x7fb9d41f8c70 2024-09-10T23:37:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9cb400640 1 -- 172.21.15.94:0/506165916 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc00fbe0 con 0x7fb9d41f8c70 2024-09-10T23:37:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.866+0000 7fb9d8a00640 1 -- 172.21.15.94:0/506165916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d40c8fc0 con 0x7fb9d41f8c70 2024-09-10T23:37:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.870+0000 7fb9d8a00640 1 -- 172.21.15.94:0/506165916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d40c9570 con 0x7fb9d41f8c70 2024-09-10T23:37:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.870+0000 7fb9d8a00640 1 -- 172.21.15.94:0/506165916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9d4074180 con 0x7fb9d41f8c70 2024-09-10T23:37:06.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.870+0000 7fb9cb400640 1 -- 172.21.15.94:0/506165916 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fb9bc02b080 con 0x7fb9d41f8c70 2024-09-10T23:37:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.874+0000 7fb9cb400640 1 --2- 172.21.15.94:0/506165916 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb9b0072730 0x7fb9b0074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.874+0000 7fb9cb400640 1 -- 172.21.15.94:0/506165916 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6543+0+0 (secure 0 0 0) 0x7fb9bc094970 con 0x7fb9d41f8c70 2024-09-10T23:37:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.874+0000 7fb9d2800640 1 --2- 172.21.15.94:0/506165916 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb9b0072730 0x7fb9b0074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.874+0000 7fb9d2800640 1 --2- 172.21.15.94:0/506165916 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb9b0072730 0x7fb9b0074bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4006ae0 tx=0x7fb9c4006730 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:06.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:06.874+0000 7fb9cb400640 1 -- 172.21.15.94:0/506165916 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb9bc05f480 con 0x7fb9d41f8c70 2024-09-10T23:37:07.065 INFO:teuthology.orchestra.run.smithi094.stderr:Invalid command: invalid chars #@ in abcd@# 2024-09-10T23:37:07.065 INFO:teuthology.orchestra.run.smithi094.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2024-09-10T23:37:07.065 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-10T23:37:07.072 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-10T23:37:07.072 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2024-09-10T23:37:07.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.213+0000 7f91f8c00640 1 -- 172.21.15.94:0/3694773098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d8003a10 msgr2=0x7f91d8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:07.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.213+0000 7f91f8c00640 1 --2- 172.21.15.94:0/3694773098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d8003a10 0x7f91d8003e30 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009b90 tx=0x7f91e401d270 comp rx=0 tx=0).stop 2024-09-10T23:37:07.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.213+0000 7f91f2800640 1 -- 172.21.15.94:0/3694773098 reap_dead start 2024-09-10T23:37:07.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.213+0000 7f91f8c00640 1 -- 172.21.15.94:0/3694773098 shutdown_connections 2024-09-10T23:37:07.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.213+0000 7f91f8c00640 1 -- 172.21.15.94:0/3694773098 >> 172.21.15.94:0/3694773098 conn(0x7f91f40cfaf0 msgr2=0x7f91f40cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:07.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.213+0000 7f91f8c00640 1 -- 172.21.15.94:0/3694773098 shutdown_connections 2024-09-10T23:37:07.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.213+0000 7f91f8c00640 1 -- 172.21.15.94:0/3694773098 wait complete. 2024-09-10T23:37:07.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f8c00640 1 Processor -- start 2024-09-10T23:37:07.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f8c00640 1 -- start start 2024-09-10T23:37:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f8c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d8003a10 0x7f91f4166550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f8c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91f413b6e0 0x7f91f41fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f8c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91f4143c20 0x7f91f41440a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f8c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91f41670d0 con 0x7f91f413b6e0 2024-09-10T23:37:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f8c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f91f4166f50 con 0x7f91d8003a10 2024-09-10T23:37:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f8c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f91f4167250 con 0x7f91f4143c20 2024-09-10T23:37:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91f413b6e0 0x7f91f41fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91f413b6e0 0x7f91f41fbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54972/0 (socket says 172.21.15.94:54972) 2024-09-10T23:37:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f1400640 1 -- 172.21.15.94:0/133457352 learned_addr learned my addr 172.21.15.94:0/133457352 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f1e00640 1 --2- 172.21.15.94:0/133457352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d8003a10 0x7f91f4166550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f2800640 1 --2- 172.21.15.94:0/133457352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91f4143c20 0x7f91f41440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f1400640 1 -- 172.21.15.94:0/133457352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91f4143c20 msgr2=0x7f91f41440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f1400640 1 --2- 172.21.15.94:0/133457352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91f4143c20 0x7f91f41440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f1400640 1 -- 172.21.15.94:0/133457352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d8003a10 msgr2=0x7f91f4166550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f1400640 1 --2- 172.21.15.94:0/133457352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d8003a10 0x7f91f4166550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f1400640 1 -- 172.21.15.94:0/133457352 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f41f85a0 con 0x7f91f413b6e0 2024-09-10T23:37:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f1400640 1 --2- 172.21.15.94:0/133457352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91f413b6e0 0x7f91f41fbb70 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f91e0006660 tx=0x7f91e00039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91ea000640 1 -- 172.21.15.94:0/133457352 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0011700 con 0x7f91f413b6e0 2024-09-10T23:37:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91ea000640 1 -- 172.21.15.94:0/133457352 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91e0016020 con 0x7f91f413b6e0 2024-09-10T23:37:07.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91ea000640 1 -- 172.21.15.94:0/133457352 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0011b40 con 0x7f91f413b6e0 2024-09-10T23:37:07.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f8c00640 1 -- 172.21.15.94:0/133457352 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f41416f0 con 0x7f91f413b6e0 2024-09-10T23:37:07.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.217+0000 7f91f8c00640 1 -- 172.21.15.94:0/133457352 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f413f930 con 0x7f91f413b6e0 2024-09-10T23:37:07.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.221+0000 7f91f8c00640 1 -- 172.21.15.94:0/133457352 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91f40cdea0 con 0x7f91f413b6e0 2024-09-10T23:37:07.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.221+0000 7f91ea000640 1 -- 172.21.15.94:0/133457352 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f91e0011ce0 con 0x7f91f413b6e0 2024-09-10T23:37:07.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.221+0000 7f91ea000640 1 --2- 172.21.15.94:0/133457352 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f91d4072730 0x7f91d4074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:07.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.221+0000 7f91ea000640 1 -- 172.21.15.94:0/133457352 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6543+0+0 (secure 0 0 0) 0x7f91e0094220 con 0x7f91f413b6e0 2024-09-10T23:37:07.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.221+0000 7f91f1e00640 1 --2- 172.21.15.94:0/133457352 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f91d4072730 0x7f91d4074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:07.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.225+0000 7f91f1e00640 1 --2- 172.21.15.94:0/133457352 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f91d4072730 0x7f91d4074bf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f91e40ae000 tx=0x7f91e401a010 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:07.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.225+0000 7f91ea000640 1 -- 172.21.15.94:0/133457352 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f91e0020020 con 0x7f91f413b6e0 2024-09-10T23:37:07.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:07.437+0000 7f91f8c00640 1 -- 172.21.15.94:0/133457352 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7f91f40cf6e0 con 0x7f91d4072730 2024-09-10T23:37:07.443 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-10T23:37:07.441+0000 7f4045a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:37:07.490 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-09-10T23:37:10.464 INFO:teuthology.orchestra.run.smithi094.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2024-09-10T23:37:10.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.461+0000 7f91ea000640 1 -- 172.21.15.94:0/133457352 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+78 (secure 0 0 0) 0x7f91f40cf6e0 con 0x7f91d4072730 2024-09-10T23:37:10.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.465+0000 7f91f8c00640 1 -- 172.21.15.94:0/133457352 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f91d4072730 msgr2=0x7f91d4074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:10.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.465+0000 7f91f8c00640 1 --2- 172.21.15.94:0/133457352 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f91d4072730 0x7f91d4074bf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f91e40ae000 tx=0x7f91e401a010 comp rx=0 tx=0).stop 2024-09-10T23:37:10.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.465+0000 7f91f8c00640 1 -- 172.21.15.94:0/133457352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91f413b6e0 msgr2=0x7f91f41fbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:10.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.465+0000 7f91f8c00640 1 --2- 172.21.15.94:0/133457352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91f413b6e0 0x7f91f41fbb70 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f91e0006660 tx=0x7f91e00039d0 comp rx=0 tx=0).stop 2024-09-10T23:37:10.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.465+0000 7f91f8c00640 1 -- 172.21.15.94:0/133457352 shutdown_connections 2024-09-10T23:37:10.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.465+0000 7f91f8c00640 1 --2- 172.21.15.94:0/133457352 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f91d4072730 0x7f91d4074bf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:10.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.465+0000 7f91f8c00640 1 --2- 172.21.15.94:0/133457352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91f4143c20 0x7f91f41440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:10.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.465+0000 7f91f8c00640 1 --2- 172.21.15.94:0/133457352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91f413b6e0 0x7f91f41fbb70 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:10.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.465+0000 7f91f8c00640 1 --2- 172.21.15.94:0/133457352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d8003a10 0x7f91f4166550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:10.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.465+0000 7f91f8c00640 1 -- 172.21.15.94:0/133457352 >> 172.21.15.94:0/133457352 conn(0x7f91f40cfaf0 msgr2=0x7f91f415e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:10.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.465+0000 7f91f8c00640 1 -- 172.21.15.94:0/133457352 shutdown_connections 2024-09-10T23:37:10.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.465+0000 7f91f8c00640 1 -- 172.21.15.94:0/133457352 wait complete. 2024-09-10T23:37:10.490 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:37:10.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:37:10.580 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:37:10.676 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:37:10.676 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:37:10.676 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:37:10.676 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:10.685 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:37:10.685 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:10.737 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:37:10.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.929+0000 7fecf3c00640 1 -- 172.21.15.94:0/2118152428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec13cfe0 msgr2=0x7fecec0cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.929+0000 7fecf3c00640 1 --2- 172.21.15.94:0/2118152428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec13cfe0 0x7fecec0cec60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fece400ad90 tx=0x7fece401dbf0 comp rx=0 tx=0).stop 2024-09-10T23:37:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.929+0000 7fecf3c00640 1 -- 172.21.15.94:0/2118152428 shutdown_connections 2024-09-10T23:37:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.929+0000 7fecf3c00640 1 --2- 172.21.15.94:0/2118152428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec13cfe0 0x7fecec0cec60 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.929+0000 7fecf3c00640 1 --2- 172.21.15.94:0/2118152428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecec15e180 0x7fecec0ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.929+0000 7fecf3c00640 1 -- 172.21.15.94:0/2118152428 >> 172.21.15.94:0/2118152428 conn(0x7fecec0cfaf0 msgr2=0x7fecec0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.929+0000 7fecf3c00640 1 -- 172.21.15.94:0/2118152428 shutdown_connections 2024-09-10T23:37:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.929+0000 7fecf3c00640 1 -- 172.21.15.94:0/2118152428 wait complete. 2024-09-10T23:37:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.929+0000 7fecf3c00640 1 Processor -- start 2024-09-10T23:37:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.929+0000 7fecf3c00640 1 -- start start 2024-09-10T23:37:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecf3c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecec13cc00 0x7fecec13c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecf3c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec13cfe0 0x7fecec151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecf3c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecec15e180 0x7fecec06f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecf3c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fecec13f800 con 0x7fecec13cfe0 2024-09-10T23:37:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecf3c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fecec13f680 con 0x7fecec13cc00 2024-09-10T23:37:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecf3c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fecec13f980 con 0x7fecec15e180 2024-09-10T23:37:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecebe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec13cfe0 0x7fecec151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecebe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec13cfe0 0x7fecec151880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55008/0 (socket says 172.21.15.94:55008) 2024-09-10T23:37:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecebe00640 1 -- 172.21.15.94:0/560662240 learned_addr learned my addr 172.21.15.94:0/560662240 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecf1c00640 1 --2- 172.21.15.94:0/560662240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecec15e180 0x7fecec06f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecf1200640 1 --2- 172.21.15.94:0/560662240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecec13cc00 0x7fecec13c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecebe00640 1 -- 172.21.15.94:0/560662240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecec15e180 msgr2=0x7fecec06f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecebe00640 1 --2- 172.21.15.94:0/560662240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecec15e180 0x7fecec06f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecebe00640 1 -- 172.21.15.94:0/560662240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecec13cc00 msgr2=0x7fecec13c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecebe00640 1 --2- 172.21.15.94:0/560662240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecec13cc00 0x7fecec13c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecebe00640 1 -- 172.21.15.94:0/560662240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecec074530 con 0x7fecec13cfe0 2024-09-10T23:37:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecebe00640 1 --2- 172.21.15.94:0/560662240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec13cfe0 0x7fecec151880 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fecd800e7a0 tx=0x7fecd800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fece8c00640 1 -- 172.21.15.94:0/560662240 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd800ccc0 con 0x7fecec13cfe0 2024-09-10T23:37:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fece8c00640 1 -- 172.21.15.94:0/560662240 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecd8010430 con 0x7fecec13cfe0 2024-09-10T23:37:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fece8c00640 1 -- 172.21.15.94:0/560662240 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8010860 con 0x7fecec13cfe0 2024-09-10T23:37:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecf3c00640 1 -- 172.21.15.94:0/560662240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecec1ee160 con 0x7fecec13cfe0 2024-09-10T23:37:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecf3c00640 1 -- 172.21.15.94:0/560662240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecec072e90 con 0x7fecec13cfe0 2024-09-10T23:37:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.933+0000 7fecf3c00640 1 -- 172.21.15.94:0/560662240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecec13ff10 con 0x7fecec13cfe0 2024-09-10T23:37:10.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.937+0000 7fece8c00640 1 -- 172.21.15.94:0/560662240 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fecd8029080 con 0x7fecec13cfe0 2024-09-10T23:37:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.937+0000 7fece8c00640 1 --2- 172.21.15.94:0/560662240 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feccc072700 0x7feccc074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.937+0000 7fece8c00640 1 -- 172.21.15.94:0/560662240 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5795+0+0 (secure 0 0 0) 0x7fecd8094400 con 0x7fecec13cfe0 2024-09-10T23:37:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.937+0000 7fecf1200640 1 --2- 172.21.15.94:0/560662240 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feccc072700 0x7feccc074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.937+0000 7fecf1200640 1 --2- 172.21.15.94:0/560662240 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feccc072700 0x7feccc074bc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fece0007fd0 tx=0x7fece0009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:10.937+0000 7fece8c00640 1 -- 172.21.15.94:0/560662240 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fecec074530 con 0x7fecec13cfe0 2024-09-10T23:37:11.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.121+0000 7fecf3c00640 1 -- 172.21.15.94:0/560662240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fecec1ee750 con 0x7fecec13cfe0 2024-09-10T23:37:11.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.121+0000 7fece8c00640 1 -- 172.21.15.94:0/560662240 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v88) ==== 80+0+120 (secure 0 0 0) 0x7fecd805f1f0 con 0x7fecec13cfe0 2024-09-10T23:37:11.123 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6835/736952701 2024-09-11T23:37:07.444153+0000 2024-09-10T23:37:11.123 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6833/736952701 2024-09-11T23:37:07.444153+0000 2024-09-10T23:37:11.123 INFO:teuthology.orchestra.run.smithi094.stderr:listed 2 entries 2024-09-10T23:37:11.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.121+0000 7fecf3c00640 1 -- 172.21.15.94:0/560662240 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feccc072700 msgr2=0x7feccc074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:11.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.121+0000 7fecf3c00640 1 --2- 172.21.15.94:0/560662240 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feccc072700 0x7feccc074bc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fece0007fd0 tx=0x7fece0009040 comp rx=0 tx=0).stop 2024-09-10T23:37:11.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.121+0000 7fecf3c00640 1 -- 172.21.15.94:0/560662240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec13cfe0 msgr2=0x7fecec151880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:11.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.121+0000 7fecf3c00640 1 --2- 172.21.15.94:0/560662240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec13cfe0 0x7fecec151880 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fecd800e7a0 tx=0x7fecd800ec60 comp rx=0 tx=0).stop 2024-09-10T23:37:11.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.121+0000 7fecf3c00640 1 -- 172.21.15.94:0/560662240 shutdown_connections 2024-09-10T23:37:11.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.121+0000 7fecf3c00640 1 --2- 172.21.15.94:0/560662240 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feccc072700 0x7feccc074bc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:11.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.121+0000 7fecf3c00640 1 --2- 172.21.15.94:0/560662240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecec15e180 0x7fecec06f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:11.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.121+0000 7fecf3c00640 1 --2- 172.21.15.94:0/560662240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecec13cfe0 0x7fecec151880 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:11.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.121+0000 7fecf3c00640 1 --2- 172.21.15.94:0/560662240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecec13cc00 0x7fecec13c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:11.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.121+0000 7fecf3c00640 1 -- 172.21.15.94:0/560662240 >> 172.21.15.94:0/560662240 conn(0x7fecec0cfaf0 msgr2=0x7fecec0cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:11.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.121+0000 7fecf3c00640 1 -- 172.21.15.94:0/560662240 shutdown_connections 2024-09-10T23:37:11.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.121+0000 7fecf3c00640 1 -- 172.21.15.94:0/560662240 wait complete. 2024-09-10T23:37:11.144 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-09-10T23:37:11.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:11.144 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T23:37:11.161 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[43489]: fuse finished with error 0 and tester_r 0 2024-09-10T23:37:11.164 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:37:11.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:11.164 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T23:37:11.299 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:37:11.299 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:37:11.300 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:11.309 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:37:11.309 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:11.362 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:37:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fda200640 1 -- 172.21.15.94:0/3991737834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb8003ab0 msgr2=0x7f0fb8003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fda200640 1 --2- 172.21.15.94:0/3991737834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb8003ab0 0x7f0fb8003ef0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009b60 tx=0x7f0fc401d270 comp rx=0 tx=0).stop 2024-09-10T23:37:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fda200640 1 -- 172.21.15.94:0/3991737834 shutdown_connections 2024-09-10T23:37:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fda200640 1 --2- 172.21.15.94:0/3991737834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb8003ab0 0x7f0fb8003ef0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fda200640 1 -- 172.21.15.94:0/3991737834 >> 172.21.15.94:0/3991737834 conn(0x7f0fd40cf8d0 msgr2=0x7f0fd40cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fda200640 1 -- 172.21.15.94:0/3991737834 shutdown_connections 2024-09-10T23:37:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fda200640 1 -- 172.21.15.94:0/3991737834 wait complete. 2024-09-10T23:37:11.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fda200640 1 Processor -- start 2024-09-10T23:37:11.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fda200640 1 -- start start 2024-09-10T23:37:11.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fda200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb8003ab0 0x7f0fd41f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:11.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fda200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fd40c6140 0x7f0fd4154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:11.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fda200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd40c3400 0x7f0fd4157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:11.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fda200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd406d000 con 0x7f0fb8003ab0 2024-09-10T23:37:11.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fda200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd40cb2a0 con 0x7f0fd40c6140 2024-09-10T23:37:11.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fda200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd406d180 con 0x7f0fd40c3400 2024-09-10T23:37:11.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fd8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb8003ab0 0x7f0fd41f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:11.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fd8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb8003ab0 0x7f0fd41f6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55012/0 (socket says 172.21.15.94:55012) 2024-09-10T23:37:11.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fd8e00640 1 -- 172.21.15.94:0/1652575288 learned_addr learned my addr 172.21.15.94:0/1652575288 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:11.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.505+0000 7f0fd9800640 1 --2- 172.21.15.94:0/1652575288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd40c3400 0x7f0fd4157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.509+0000 7f0fd8e00640 1 -- 172.21.15.94:0/1652575288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd40c3400 msgr2=0x7f0fd4157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.509+0000 7f0fd8e00640 1 --2- 172.21.15.94:0/1652575288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd40c3400 0x7f0fd4157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.509+0000 7f0fd8e00640 1 -- 172.21.15.94:0/1652575288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fd40c6140 msgr2=0x7f0fd4154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.509+0000 7f0fcbe00640 1 --2- 172.21.15.94:0/1652575288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fd40c6140 0x7f0fd4154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.509+0000 7f0fd8e00640 1 --2- 172.21.15.94:0/1652575288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fd40c6140 0x7f0fd4154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.509+0000 7f0fd8e00640 1 -- 172.21.15.94:0/1652575288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd41551c0 con 0x7f0fb8003ab0 2024-09-10T23:37:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.509+0000 7f0fcbe00640 1 --2- 172.21.15.94:0/1652575288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fd40c6140 0x7f0fd4154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:37:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.509+0000 7f0fd8e00640 1 --2- 172.21.15.94:0/1652575288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb8003ab0 0x7f0fd41f6b80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40af040 tx=0x7f0fc40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.509+0000 7f0fd1600640 1 -- 172.21.15.94:0/1652575288 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40ba070 con 0x7f0fb8003ab0 2024-09-10T23:37:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.509+0000 7f0fd1600640 1 -- 172.21.15.94:0/1652575288 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fc40acaa0 con 0x7f0fb8003ab0 2024-09-10T23:37:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.509+0000 7f0fd1600640 1 -- 172.21.15.94:0/1652575288 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40b1040 con 0x7f0fb8003ab0 2024-09-10T23:37:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.509+0000 7f0fda200640 1 -- 172.21.15.94:0/1652575288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd415cc20 con 0x7f0fb8003ab0 2024-09-10T23:37:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.509+0000 7f0fda200640 1 -- 172.21.15.94:0/1652575288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd415c940 con 0x7f0fb8003ab0 2024-09-10T23:37:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.509+0000 7f0fda200640 1 -- 172.21.15.94:0/1652575288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd406ea20 con 0x7f0fb8003ab0 2024-09-10T23:37:11.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.509+0000 7f0fd1600640 1 -- 172.21.15.94:0/1652575288 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f0fc40c7070 con 0x7f0fb8003ab0 2024-09-10T23:37:11.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.513+0000 7f0fd1600640 1 --2- 172.21.15.94:0/1652575288 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0fb0072700 0x7f0fb0074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:11.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.513+0000 7f0fd1600640 1 -- 172.21.15.94:0/1652575288 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5795+0+0 (secure 0 0 0) 0x7f0fc412ec30 con 0x7f0fb8003ab0 2024-09-10T23:37:11.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.513+0000 7f0fcbe00640 1 --2- 172.21.15.94:0/1652575288 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0fb0072700 0x7f0fb0074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:11.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.513+0000 7f0fcbe00640 1 --2- 172.21.15.94:0/1652575288 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0fb0072700 0x7f0fb0074bc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0fd40c5430 tx=0x7f0fbc001e50 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:11.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.513+0000 7f0fd1600640 1 -- 172.21.15.94:0/1652575288 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0fc40c4020 con 0x7f0fb8003ab0 2024-09-10T23:37:11.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.689+0000 7f0fda200640 1 -- 172.21.15.94:0/1652575288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0fd406f980 con 0x7f0fb8003ab0 2024-09-10T23:37:11.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.689+0000 7f0fd1600640 1 -- 172.21.15.94:0/1652575288 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v88) ==== 80+0+120 (secure 0 0 0) 0x7f0fd41551c0 con 0x7f0fb8003ab0 2024-09-10T23:37:11.693 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6835/736952701 2024-09-11T23:37:07.444153+0000 2024-09-10T23:37:11.694 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6833/736952701 2024-09-11T23:37:07.444153+0000 2024-09-10T23:37:11.694 INFO:teuthology.orchestra.run.smithi094.stderr:listed 2 entries 2024-09-10T23:37:11.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.693+0000 7f0fda200640 1 -- 172.21.15.94:0/1652575288 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0fb0072700 msgr2=0x7f0fb0074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:11.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.693+0000 7f0fda200640 1 --2- 172.21.15.94:0/1652575288 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0fb0072700 0x7f0fb0074bc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0fd40c5430 tx=0x7f0fbc001e50 comp rx=0 tx=0).stop 2024-09-10T23:37:11.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.693+0000 7f0fda200640 1 -- 172.21.15.94:0/1652575288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb8003ab0 msgr2=0x7f0fd41f6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:11.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.693+0000 7f0fda200640 1 --2- 172.21.15.94:0/1652575288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb8003ab0 0x7f0fd41f6b80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40af040 tx=0x7f0fc40056a0 comp rx=0 tx=0).stop 2024-09-10T23:37:11.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.693+0000 7f0fda200640 1 -- 172.21.15.94:0/1652575288 shutdown_connections 2024-09-10T23:37:11.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.693+0000 7f0fda200640 1 --2- 172.21.15.94:0/1652575288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fd40c6140 0x7f0fd4154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:11.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.693+0000 7f0fda200640 1 --2- 172.21.15.94:0/1652575288 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0fb0072700 0x7f0fb0074bc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:11.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.693+0000 7f0fda200640 1 --2- 172.21.15.94:0/1652575288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fd40c3400 0x7f0fd4157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:11.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.693+0000 7f0fda200640 1 --2- 172.21.15.94:0/1652575288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb8003ab0 0x7f0fd41f6b80 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:11.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.693+0000 7f0fda200640 1 -- 172.21.15.94:0/1652575288 >> 172.21.15.94:0/1652575288 conn(0x7f0fd40cf8d0 msgr2=0x7f0fd40b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:11.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.693+0000 7f0fda200640 1 -- 172.21.15.94:0/1652575288 shutdown_connections 2024-09-10T23:37:11.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:11.693+0000 7f0fda200640 1 -- 172.21.15.94:0/1652575288 wait complete. 2024-09-10T23:37:11.714 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi181.front.sepia.ceph.com... 2024-09-10T23:37:11.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:11.715 DEBUG:teuthology.orchestra.run.smithi181:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T23:37:11.734 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[35320]: fuse finished with error 0 and tester_r 0 2024-09-10T23:37:11.737 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:37:11.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:11.737 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T23:37:11.898 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:37:11.898 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:37:11.899 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:11.907 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:37:11.907 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:37:11.908 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:37:11.908 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:11.916 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:37:11.916 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:37:12.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.089+0000 7f777fe00640 1 -- 172.21.15.94:0/3221774855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f777806ee20 msgr2=0x7f7778070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:12.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.089+0000 7f777fe00640 1 --2- 172.21.15.94:0/3221774855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f777806ee20 0x7f7778070880 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7764009840 tx=0x7f776401c820 comp rx=0 tx=0).stop 2024-09-10T23:37:12.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.089+0000 7f777fe00640 1 -- 172.21.15.94:0/3221774855 shutdown_connections 2024-09-10T23:37:12.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.089+0000 7f777fe00640 1 --2- 172.21.15.94:0/3221774855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f777806ee20 0x7f7778070880 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.089+0000 7f777fe00640 1 -- 172.21.15.94:0/3221774855 >> 172.21.15.94:0/3221774855 conn(0x7f77780b16d0 msgr2=0x7f77780b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:12.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.089+0000 7f777fe00640 1 -- 172.21.15.94:0/3221774855 shutdown_connections 2024-09-10T23:37:12.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.089+0000 7f777fe00640 1 -- 172.21.15.94:0/3221774855 wait complete. 2024-09-10T23:37:12.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777fe00640 1 Processor -- start 2024-09-10T23:37:12.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777fe00640 1 -- start start 2024-09-10T23:37:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777fe00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f777806ee20 0x7f77781571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77780b3fe0 0x7f77780af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7778164e60 0x7f77780afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7778178f60 con 0x7f7778164e60 2024-09-10T23:37:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777fe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7778178de0 con 0x7f77780b3fe0 2024-09-10T23:37:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777fe00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f77781790e0 con 0x7f777806ee20 2024-09-10T23:37:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7778164e60 0x7f77780afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7778164e60 0x7f77780afc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55034/0 (socket says 172.21.15.94:55034) 2024-09-10T23:37:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777de00640 1 -- 172.21.15.94:0/1661415644 learned_addr learned my addr 172.21.15.94:0/1661415644 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777ca00640 1 --2- 172.21.15.94:0/1661415644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77780b3fe0 0x7f77780af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777d400640 1 --2- 172.21.15.94:0/1661415644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f777806ee20 0x7f77781571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:12.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777de00640 1 -- 172.21.15.94:0/1661415644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f777806ee20 msgr2=0x7f77781571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777de00640 1 --2- 172.21.15.94:0/1661415644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f777806ee20 0x7f77781571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777de00640 1 -- 172.21.15.94:0/1661415644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77780b3fe0 msgr2=0x7f77780af6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777de00640 1 --2- 172.21.15.94:0/1661415644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77780b3fe0 0x7f77780af6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777de00640 1 -- 172.21.15.94:0/1661415644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777816ad70 con 0x7f7778164e60 2024-09-10T23:37:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777de00640 1 --2- 172.21.15.94:0/1661415644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7778164e60 0x7f77780afc10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f777000c390 tx=0x7f777000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f7775600640 1 -- 172.21.15.94:0/1661415644 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770011790 con 0x7f7778164e60 2024-09-10T23:37:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f7775600640 1 -- 172.21.15.94:0/1661415644 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7770017070 con 0x7f7778164e60 2024-09-10T23:37:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f7775600640 1 -- 172.21.15.94:0/1661415644 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770011cc0 con 0x7f7778164e60 2024-09-10T23:37:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777fe00640 1 -- 172.21.15.94:0/1661415644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777816a480 con 0x7f7778164e60 2024-09-10T23:37:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.093+0000 7f777fe00640 1 -- 172.21.15.94:0/1661415644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7778176290 con 0x7f7778164e60 2024-09-10T23:37:12.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.097+0000 7f777fe00640 1 -- 172.21.15.94:0/1661415644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77780b1c50 con 0x7f7778164e60 2024-09-10T23:37:12.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.097+0000 7f7775600640 1 -- 172.21.15.94:0/1661415644 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f777001c460 con 0x7f7778164e60 2024-09-10T23:37:12.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.097+0000 7f7775600640 1 --2- 172.21.15.94:0/1661415644 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7758072700 0x7f7758074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:12.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.097+0000 7f7775600640 1 -- 172.21.15.94:0/1661415644 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5795+0+0 (secure 0 0 0) 0x7f7770092fd0 con 0x7f7778164e60 2024-09-10T23:37:12.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.097+0000 7f777d400640 1 --2- 172.21.15.94:0/1661415644 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7758072700 0x7f7758074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:12.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.097+0000 7f777d400640 1 --2- 172.21.15.94:0/1661415644 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7758072700 0x7f7758074bc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f776c0012f0 tx=0x7f776c009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:12.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.101+0000 7f7775600640 1 -- 172.21.15.94:0/1661415644 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7770021020 con 0x7f7778164e60 2024-09-10T23:37:12.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.301+0000 7f777fe00640 1 -- 172.21.15.94:0/1661415644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7778070880 con 0x7f7778164e60 2024-09-10T23:37:12.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.301+0000 7f7775600640 1 -- 172.21.15.94:0/1661415644 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5241 (secure 0 0 0) 0x7f777005ddd0 con 0x7f7778164e60 2024-09-10T23:37:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:12.304 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":59,"btime":"2024-09-10T23:37:08:446193+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1215179751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1215179751},{"type":"v1","addr":"172.21.15.94:6834","nonce":1215179751}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6672,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3193009978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":3193009978},{"type":"v1","addr":"172.21.15.181:6835","nonce":3193009978}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1426560042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":1426560042},{"type":"v1","addr":"172.21.15.94:6835","nonce":1426560042}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:36:51.627529+0000","modified":"2024-09-10T23:36:54.641240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195701":{"gid":195701,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6834/1214870454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1214870454},{"type":"v1","addr":"172.21.15.181:6834","nonce":1214870454}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195701,"qdb_cluster":[195701]},"id":6}]} 2024-09-10T23:37:12.304 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 59 2024-09-10T23:37:12.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.305+0000 7f777fe00640 1 -- 172.21.15.94:0/1661415644 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7758072700 msgr2=0x7f7758074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:12.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.305+0000 7f777fe00640 1 --2- 172.21.15.94:0/1661415644 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7758072700 0x7f7758074bc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f776c0012f0 tx=0x7f776c009040 comp rx=0 tx=0).stop 2024-09-10T23:37:12.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.305+0000 7f777fe00640 1 -- 172.21.15.94:0/1661415644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7778164e60 msgr2=0x7f77780afc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:12.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.305+0000 7f777fe00640 1 --2- 172.21.15.94:0/1661415644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7778164e60 0x7f77780afc10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f777000c390 tx=0x7f777000c850 comp rx=0 tx=0).stop 2024-09-10T23:37:12.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.305+0000 7f777fe00640 1 -- 172.21.15.94:0/1661415644 shutdown_connections 2024-09-10T23:37:12.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.305+0000 7f777fe00640 1 --2- 172.21.15.94:0/1661415644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77780b3fe0 0x7f77780af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.305+0000 7f777fe00640 1 --2- 172.21.15.94:0/1661415644 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7758072700 0x7f7758074bc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.305+0000 7f777fe00640 1 --2- 172.21.15.94:0/1661415644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7778164e60 0x7f77780afc10 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.305+0000 7f777fe00640 1 --2- 172.21.15.94:0/1661415644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f777806ee20 0x7f77781571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.305+0000 7f777fe00640 1 -- 172.21.15.94:0/1661415644 >> 172.21.15.94:0/1661415644 conn(0x7f77780b16d0 msgr2=0x7f77781641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:12.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.305+0000 7f777fe00640 1 -- 172.21.15.94:0/1661415644 shutdown_connections 2024-09-10T23:37:12.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.305+0000 7f777fe00640 1 -- 172.21.15.94:0/1661415644 wait complete. 2024-09-10T23:37:12.325 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:37:12.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.465+0000 7f615c600640 1 -- 172.21.15.94:0/3627265842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f615415f3f0 msgr2=0x7f6154068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:12.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.465+0000 7f615c600640 1 --2- 172.21.15.94:0/3627265842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f615415f3f0 0x7f6154068990 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f6148009900 tx=0x7f614801c850 comp rx=0 tx=0).stop 2024-09-10T23:37:12.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.465+0000 7f615c600640 1 -- 172.21.15.94:0/3627265842 shutdown_connections 2024-09-10T23:37:12.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.465+0000 7f615c600640 1 --2- 172.21.15.94:0/3627265842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f615415f3f0 0x7f6154068990 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.465+0000 7f615c600640 1 -- 172.21.15.94:0/3627265842 >> 172.21.15.94:0/3627265842 conn(0x7f61540d02d0 msgr2=0x7f61540d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:12.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.465+0000 7f615c600640 1 -- 172.21.15.94:0/3627265842 shutdown_connections 2024-09-10T23:37:12.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.465+0000 7f615c600640 1 -- 172.21.15.94:0/3627265842 wait complete. 2024-09-10T23:37:12.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.465+0000 7f615c600640 1 Processor -- start 2024-09-10T23:37:12.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.465+0000 7f615c600640 1 -- start start 2024-09-10T23:37:12.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f615c600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61540cc2f0 0x7f61540b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:12.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f615c600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61540cdea0 0x7f61540b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:12.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f615c600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f615415f3f0 0x7f61540b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:12.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f615c600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f615406da30 con 0x7f61540cc2f0 2024-09-10T23:37:12.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f615c600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f615406d8b0 con 0x7f61540cdea0 2024-09-10T23:37:12.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f615c600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f61541697f0 con 0x7f615415f3f0 2024-09-10T23:37:12.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f6159c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61540cc2f0 0x7f61540b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:12.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f6159c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61540cc2f0 0x7f61540b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55048/0 (socket says 172.21.15.94:55048) 2024-09-10T23:37:12.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f6159c00640 1 -- 172.21.15.94:0/303710628 learned_addr learned my addr 172.21.15.94:0/303710628 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:12.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f6159200640 1 --2- 172.21.15.94:0/303710628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61540cdea0 0x7f61540b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:12.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f6159c00640 1 -- 172.21.15.94:0/303710628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f615415f3f0 msgr2=0x7f61540b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:12.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f6159c00640 1 --2- 172.21.15.94:0/303710628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f615415f3f0 0x7f61540b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f6159c00640 1 -- 172.21.15.94:0/303710628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61540cdea0 msgr2=0x7f61540b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f6159c00640 1 --2- 172.21.15.94:0/303710628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61540cdea0 0x7f61540b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f6159c00640 1 -- 172.21.15.94:0/303710628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61540d1190 con 0x7f61540cc2f0 2024-09-10T23:37:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f6159c00640 1 --2- 172.21.15.94:0/303710628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61540cc2f0 0x7f61540b47c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6148009850 tx=0x7f61480a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f6151600640 1 -- 172.21.15.94:0/303710628 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61480b2040 con 0x7f61540cc2f0 2024-09-10T23:37:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f6151600640 1 -- 172.21.15.94:0/303710628 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6148004310 con 0x7f61540cc2f0 2024-09-10T23:37:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f6151600640 1 -- 172.21.15.94:0/303710628 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148003860 con 0x7f61540cc2f0 2024-09-10T23:37:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f615c600640 1 -- 172.21.15.94:0/303710628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61541f6060 con 0x7f61540cc2f0 2024-09-10T23:37:12.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f615c600640 1 -- 172.21.15.94:0/303710628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61541f65e0 con 0x7f61540cc2f0 2024-09-10T23:37:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f615c600640 1 -- 172.21.15.94:0/303710628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6154151890 con 0x7f61540cc2f0 2024-09-10T23:37:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.469+0000 7f6151600640 1 -- 172.21.15.94:0/303710628 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f6148003a00 con 0x7f61540cc2f0 2024-09-10T23:37:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.473+0000 7f6151600640 1 --2- 172.21.15.94:0/303710628 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6134072730 0x7f6134074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.473+0000 7f6151600640 1 -- 172.21.15.94:0/303710628 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5795+0+0 (secure 0 0 0) 0x7f614812d970 con 0x7f61540cc2f0 2024-09-10T23:37:12.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.473+0000 7f6159200640 1 --2- 172.21.15.94:0/303710628 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6134072730 0x7f6134074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:12.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.473+0000 7f6159200640 1 --2- 172.21.15.94:0/303710628 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6134072730 0x7f6134074bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f61541fa4b0 tx=0x7f6140009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:12.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.477+0000 7f6151600640 1 -- 172.21.15.94:0/303710628 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f61480c6020 con 0x7f61540cc2f0 2024-09-10T23:37:12.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.673+0000 7f615c600640 1 -- 172.21.15.94:0/303710628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61540b6250 con 0x7f61540cc2f0 2024-09-10T23:37:12.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.673+0000 7f6151600640 1 -- 172.21.15.94:0/303710628 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5241 (secure 0 0 0) 0x7f61480164d0 con 0x7f61540cc2f0 2024-09-10T23:37:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:12.674 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":59,"btime":"2024-09-10T23:37:08:446193+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1215179751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1215179751},{"type":"v1","addr":"172.21.15.94:6834","nonce":1215179751}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6672,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3193009978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":3193009978},{"type":"v1","addr":"172.21.15.181:6835","nonce":3193009978}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1426560042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":1426560042},{"type":"v1","addr":"172.21.15.94:6835","nonce":1426560042}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:36:51.627529+0000","modified":"2024-09-10T23:36:54.641240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195701":{"gid":195701,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6834/1214870454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1214870454},{"type":"v1","addr":"172.21.15.181:6834","nonce":1214870454}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195701,"qdb_cluster":[195701]},"id":6}]} 2024-09-10T23:37:12.675 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 59 2024-09-10T23:37:12.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.673+0000 7f615c600640 1 -- 172.21.15.94:0/303710628 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6134072730 msgr2=0x7f6134074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:12.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.673+0000 7f615c600640 1 --2- 172.21.15.94:0/303710628 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6134072730 0x7f6134074bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f61541fa4b0 tx=0x7f6140009040 comp rx=0 tx=0).stop 2024-09-10T23:37:12.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.673+0000 7f615c600640 1 -- 172.21.15.94:0/303710628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61540cc2f0 msgr2=0x7f61540b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.673+0000 7f615c600640 1 --2- 172.21.15.94:0/303710628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61540cc2f0 0x7f61540b47c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6148009850 tx=0x7f61480a5730 comp rx=0 tx=0).stop 2024-09-10T23:37:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.673+0000 7f615c600640 1 -- 172.21.15.94:0/303710628 shutdown_connections 2024-09-10T23:37:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.673+0000 7f615c600640 1 --2- 172.21.15.94:0/303710628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61540cdea0 0x7f61540b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.673+0000 7f615c600640 1 --2- 172.21.15.94:0/303710628 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6134072730 0x7f6134074bf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.673+0000 7f615c600640 1 --2- 172.21.15.94:0/303710628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f615415f3f0 0x7f61540b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.673+0000 7f615c600640 1 --2- 172.21.15.94:0/303710628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61540cc2f0 0x7f61540b47c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.673+0000 7f615c600640 1 -- 172.21.15.94:0/303710628 >> 172.21.15.94:0/303710628 conn(0x7f61540d02d0 msgr2=0x7f61540ad770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.673+0000 7f615c600640 1 -- 172.21.15.94:0/303710628 shutdown_connections 2024-09-10T23:37:12.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.673+0000 7f615c600640 1 -- 172.21.15.94:0/303710628 wait complete. 2024-09-10T23:37:12.695 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:37:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a8e00640 1 -- 172.21.15.94:0/1456975263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a4071a20 msgr2=0x7fa3a415f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a8e00640 1 --2- 172.21.15.94:0/1456975263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a4071a20 0x7fa3a415f610 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009900 tx=0x7fa38c01c850 comp rx=0 tx=0).stop 2024-09-10T23:37:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a8e00640 1 -- 172.21.15.94:0/1456975263 shutdown_connections 2024-09-10T23:37:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a8e00640 1 --2- 172.21.15.94:0/1456975263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a4071a20 0x7fa3a415f610 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a8e00640 1 --2- 172.21.15.94:0/1456975263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3a406cc20 0x7fa3a406d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a8e00640 1 -- 172.21.15.94:0/1456975263 >> 172.21.15.94:0/1456975263 conn(0x7fa3a40d0b70 msgr2=0x7fa3a40d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a8e00640 1 -- 172.21.15.94:0/1456975263 shutdown_connections 2024-09-10T23:37:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a8e00640 1 -- 172.21.15.94:0/1456975263 wait complete. 2024-09-10T23:37:12.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a8e00640 1 Processor -- start 2024-09-10T23:37:12.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a8e00640 1 -- start start 2024-09-10T23:37:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a406cc20 0x7fa3a4155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a8e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3a4071a20 0x7fa3a4146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a8e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3a40d1690 0x7fa3a4150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a8e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41691c0 con 0x7fa3a406cc20 2024-09-10T23:37:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a8e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a4160420 con 0x7fa3a4071a20 2024-09-10T23:37:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a8e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a4169340 con 0x7fa3a40d1690 2024-09-10T23:37:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.837+0000 7fa3a1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a406cc20 0x7fa3a4155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa3a1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a406cc20 0x7fa3a4155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55074/0 (socket says 172.21.15.94:55074) 2024-09-10T23:37:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa3a1e00640 1 -- 172.21.15.94:0/1589290414 learned_addr learned my addr 172.21.15.94:0/1589290414 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa39be00640 1 --2- 172.21.15.94:0/1589290414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3a4071a20 0x7fa3a4146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa3a2800640 1 --2- 172.21.15.94:0/1589290414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3a40d1690 0x7fa3a4150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa3a1e00640 1 -- 172.21.15.94:0/1589290414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3a40d1690 msgr2=0x7fa3a4150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:12.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa3a1e00640 1 --2- 172.21.15.94:0/1589290414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3a40d1690 0x7fa3a4150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa3a1e00640 1 -- 172.21.15.94:0/1589290414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3a4071a20 msgr2=0x7fa3a4146300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:12.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa3a1e00640 1 --2- 172.21.15.94:0/1589290414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3a4071a20 0x7fa3a4146300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:12.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa3a1e00640 1 -- 172.21.15.94:0/1589290414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a4151760 con 0x7fa3a406cc20 2024-09-10T23:37:12.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa3a1e00640 1 --2- 172.21.15.94:0/1589290414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a406cc20 0x7fa3a4155490 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fa39400d950 tx=0x7fa39400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:12.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa39a000640 1 -- 172.21.15.94:0/1589290414 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa394017070 con 0x7fa3a406cc20 2024-09-10T23:37:12.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa39a000640 1 -- 172.21.15.94:0/1589290414 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa394009d20 con 0x7fa3a406cc20 2024-09-10T23:37:12.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa3a8e00640 1 -- 172.21.15.94:0/1589290414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a41549b0 con 0x7fa3a406cc20 2024-09-10T23:37:12.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa39a000640 1 -- 172.21.15.94:0/1589290414 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3940106c0 con 0x7fa3a406cc20 2024-09-10T23:37:12.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa3a8e00640 1 -- 172.21.15.94:0/1589290414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a4154f70 con 0x7fa3a406cc20 2024-09-10T23:37:12.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa3a8e00640 1 -- 172.21.15.94:0/1589290414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a406d000 con 0x7fa3a406cc20 2024-09-10T23:37:12.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.841+0000 7fa39a000640 1 -- 172.21.15.94:0/1589290414 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fa394010860 con 0x7fa3a406cc20 2024-09-10T23:37:12.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.845+0000 7fa39a000640 1 --2- 172.21.15.94:0/1589290414 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa380072700 0x7fa380074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:12.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.845+0000 7fa39a000640 1 -- 172.21.15.94:0/1589290414 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5795+0+0 (secure 0 0 0) 0x7fa394093da0 con 0x7fa3a406cc20 2024-09-10T23:37:12.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.845+0000 7fa39be00640 1 --2- 172.21.15.94:0/1589290414 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa380072700 0x7fa380074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:12.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.845+0000 7fa39be00640 1 --2- 172.21.15.94:0/1589290414 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa380072700 0x7fa380074bc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0af000 tx=0x7fa38c01b090 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:12.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:12.845+0000 7fa39a000640 1 -- 172.21.15.94:0/1589290414 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa39405eb90 con 0x7fa3a406cc20 2024-09-10T23:37:13.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.013+0000 7fa3a8e00640 1 -- 172.21.15.94:0/1589290414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3a415d450 con 0x7fa3a406cc20 2024-09-10T23:37:13.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.013+0000 7fa39a000640 1 -- 172.21.15.94:0/1589290414 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+16847 (secure 0 0 0) 0x7fa3940635c0 con 0x7fa3a406cc20 2024-09-10T23:37:13.016 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:13.016 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":88,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:37:10.454435+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:36:49.308477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T23:36:50.584653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6835/736952701":"2024-09-11T23:37:07.444153+0000","172.21.15.94:6833/736952701":"2024-09-11T23:37:07.444153+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:37:13.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.017+0000 7fa3a8e00640 1 -- 172.21.15.94:0/1589290414 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa380072700 msgr2=0x7fa380074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:13.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.017+0000 7fa3a8e00640 1 --2- 172.21.15.94:0/1589290414 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa380072700 0x7fa380074bc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0af000 tx=0x7fa38c01b090 comp rx=0 tx=0).stop 2024-09-10T23:37:13.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.017+0000 7fa3a8e00640 1 -- 172.21.15.94:0/1589290414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a406cc20 msgr2=0x7fa3a4155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:13.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.017+0000 7fa3a8e00640 1 --2- 172.21.15.94:0/1589290414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a406cc20 0x7fa3a4155490 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fa39400d950 tx=0x7fa39400de10 comp rx=0 tx=0).stop 2024-09-10T23:37:13.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.017+0000 7fa3a8e00640 1 -- 172.21.15.94:0/1589290414 shutdown_connections 2024-09-10T23:37:13.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.017+0000 7fa3a8e00640 1 --2- 172.21.15.94:0/1589290414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3a4071a20 0x7fa3a4146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.017+0000 7fa3a8e00640 1 --2- 172.21.15.94:0/1589290414 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa380072700 0x7fa380074bc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.017+0000 7fa3a8e00640 1 --2- 172.21.15.94:0/1589290414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3a40d1690 0x7fa3a4150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.017+0000 7fa3a8e00640 1 --2- 172.21.15.94:0/1589290414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3a406cc20 0x7fa3a4155490 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.017+0000 7fa3a8e00640 1 -- 172.21.15.94:0/1589290414 >> 172.21.15.94:0/1589290414 conn(0x7fa3a40d0b70 msgr2=0x7fa3a414c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:13.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.017+0000 7fa3a8e00640 1 -- 172.21.15.94:0/1589290414 shutdown_connections 2024-09-10T23:37:13.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.017+0000 7fa3a8e00640 1 -- 172.21.15.94:0/1589290414 wait complete. 2024-09-10T23:37:13.037 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T23:37:13.037 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:37:13.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.177+0000 7f8cf0a00640 1 -- 172.21.15.94:0/4179627712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec160120 msgr2=0x7f8cec160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:13.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.177+0000 7f8cf0a00640 1 --2- 172.21.15.94:0/4179627712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec160120 0x7f8cec160590 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009900 tx=0x7f8ce401f0d0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.177+0000 7f8cf0a00640 1 -- 172.21.15.94:0/4179627712 shutdown_connections 2024-09-10T23:37:13.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.177+0000 7f8cf0a00640 1 --2- 172.21.15.94:0/4179627712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec160120 0x7f8cec160590 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.177+0000 7f8cf0a00640 1 --2- 172.21.15.94:0/4179627712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cec0d0010 0x7f8cec15fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.177+0000 7f8cf0a00640 1 -- 172.21.15.94:0/4179627712 >> 172.21.15.94:0/4179627712 conn(0x7f8cec0d1bf0 msgr2=0x7f8cec0d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:13.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.177+0000 7f8cf0a00640 1 -- 172.21.15.94:0/4179627712 shutdown_connections 2024-09-10T23:37:13.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.177+0000 7f8cf0a00640 1 -- 172.21.15.94:0/4179627712 wait complete. 2024-09-10T23:37:13.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.177+0000 7f8cf0a00640 1 Processor -- start 2024-09-10T23:37:13.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.177+0000 7f8cf0a00640 1 -- start start 2024-09-10T23:37:13.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8cf0a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cec0cfc30 0x7f8cec150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:13.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8cf0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec0d0010 0x7f8cec1535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:13.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8cf0a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cec160120 0x7f8cec0b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:13.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8cf0a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec0b1c10 con 0x7f8cec0d0010 2024-09-10T23:37:13.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8cf0a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec0b1a90 con 0x7f8cec160120 2024-09-10T23:37:13.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8cf0a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8cec0b1d90 con 0x7f8cec0cfc30 2024-09-10T23:37:13.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8ce9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec0d0010 0x7f8cec1535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:13.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8ce9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec0d0010 0x7f8cec1535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55100/0 (socket says 172.21.15.94:55100) 2024-09-10T23:37:13.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8ce9400640 1 -- 172.21.15.94:0/4206122166 learned_addr learned my addr 172.21.15.94:0/4206122166 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:13.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8cea800640 1 --2- 172.21.15.94:0/4206122166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cec160120 0x7f8cec0b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:13.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8ce9e00640 1 --2- 172.21.15.94:0/4206122166 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cec0cfc30 0x7f8cec150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:13.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8ce9400640 1 -- 172.21.15.94:0/4206122166 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cec0cfc30 msgr2=0x7f8cec150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:13.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8ce9400640 1 --2- 172.21.15.94:0/4206122166 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cec0cfc30 0x7f8cec150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8ce9400640 1 -- 172.21.15.94:0/4206122166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cec160120 msgr2=0x7f8cec0b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:13.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8ce9400640 1 --2- 172.21.15.94:0/4206122166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cec160120 0x7f8cec0b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8ce9400640 1 -- 172.21.15.94:0/4206122166 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cec153ae0 con 0x7f8cec0d0010 2024-09-10T23:37:13.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8cea800640 1 --2- 172.21.15.94:0/4206122166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cec160120 0x7f8cec0b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:13.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8ce9e00640 1 --2- 172.21.15.94:0/4206122166 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cec0cfc30 0x7f8cec150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:13.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8ce9400640 1 --2- 172.21.15.94:0/4206122166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec0d0010 0x7f8cec1535a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800c970 tx=0x7f8cd800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:13.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8ce2000640 1 -- 172.21.15.94:0/4206122166 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8007c70 con 0x7f8cec0d0010 2024-09-10T23:37:13.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8ce2000640 1 -- 172.21.15.94:0/4206122166 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cd800f450 con 0x7f8cec0d0010 2024-09-10T23:37:13.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8cf0a00640 1 -- 172.21.15.94:0/4206122166 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cec151010 con 0x7f8cec0d0010 2024-09-10T23:37:13.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8ce2000640 1 -- 172.21.15.94:0/4206122166 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd800f8a0 con 0x7f8cec0d0010 2024-09-10T23:37:13.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8cf0a00640 1 -- 172.21.15.94:0/4206122166 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cec1515d0 con 0x7f8cec0d0010 2024-09-10T23:37:13.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8cf0a00640 1 -- 172.21.15.94:0/4206122166 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cec15fbe0 con 0x7f8cec0d0010 2024-09-10T23:37:13.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.181+0000 7f8ce2000640 1 -- 172.21.15.94:0/4206122166 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f8cd8020030 con 0x7f8cec0d0010 2024-09-10T23:37:13.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.185+0000 7f8ce2000640 1 --2- 172.21.15.94:0/4206122166 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ccc072730 0x7f8ccc074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:13.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.185+0000 7f8ce2000640 1 -- 172.21.15.94:0/4206122166 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5795+0+0 (secure 0 0 0) 0x7f8cd8093920 con 0x7f8cec0d0010 2024-09-10T23:37:13.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.185+0000 7f8ce9e00640 1 --2- 172.21.15.94:0/4206122166 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ccc072730 0x7f8ccc074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:13.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.185+0000 7f8ce9e00640 1 --2- 172.21.15.94:0/4206122166 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ccc072730 0x7f8ccc074bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0045c0 tx=0x7f8cdc009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:13.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.189+0000 7f8ce2000640 1 -- 172.21.15.94:0/4206122166 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8cd8014030 con 0x7f8cec0d0010 2024-09-10T23:37:13.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.389+0000 7f8cf0a00640 1 -- 172.21.15.94:0/4206122166 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cec050ef0 con 0x7f8cec0d0010 2024-09-10T23:37:13.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.389+0000 7f8ce2000640 1 -- 172.21.15.94:0/4206122166 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5241 (secure 0 0 0) 0x7f8cd805ee10 con 0x7f8cec0d0010 2024-09-10T23:37:13.390 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:13.390 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":59,"btime":"2024-09-10T23:37:08:446193+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1215179751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1215179751},{"type":"v1","addr":"172.21.15.94:6834","nonce":1215179751}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6672,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3193009978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":3193009978},{"type":"v1","addr":"172.21.15.181:6835","nonce":3193009978}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1426560042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":1426560042},{"type":"v1","addr":"172.21.15.94:6835","nonce":1426560042}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:36:51.627529+0000","modified":"2024-09-10T23:36:54.641240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195701":{"gid":195701,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6834/1214870454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1214870454},{"type":"v1","addr":"172.21.15.181:6834","nonce":1214870454}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195701,"qdb_cluster":[195701]},"id":6}]} 2024-09-10T23:37:13.391 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 59 2024-09-10T23:37:13.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.389+0000 7f8cf0a00640 1 -- 172.21.15.94:0/4206122166 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ccc072730 msgr2=0x7f8ccc074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:13.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.389+0000 7f8cf0a00640 1 --2- 172.21.15.94:0/4206122166 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ccc072730 0x7f8ccc074bf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0045c0 tx=0x7f8cdc009040 comp rx=0 tx=0).stop 2024-09-10T23:37:13.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.389+0000 7f8cf0a00640 1 -- 172.21.15.94:0/4206122166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec0d0010 msgr2=0x7f8cec1535a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:13.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.389+0000 7f8cf0a00640 1 --2- 172.21.15.94:0/4206122166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec0d0010 0x7f8cec1535a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800c970 tx=0x7f8cd800ce30 comp rx=0 tx=0).stop 2024-09-10T23:37:13.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.389+0000 7f8cf0a00640 1 -- 172.21.15.94:0/4206122166 shutdown_connections 2024-09-10T23:37:13.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.389+0000 7f8cf0a00640 1 --2- 172.21.15.94:0/4206122166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cec160120 0x7f8cec0b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.389+0000 7f8cf0a00640 1 --2- 172.21.15.94:0/4206122166 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ccc072730 0x7f8ccc074bf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.389+0000 7f8cf0a00640 1 --2- 172.21.15.94:0/4206122166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cec0d0010 0x7f8cec1535a0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.389+0000 7f8cf0a00640 1 --2- 172.21.15.94:0/4206122166 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cec0cfc30 0x7f8cec150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.389+0000 7f8cf0a00640 1 -- 172.21.15.94:0/4206122166 >> 172.21.15.94:0/4206122166 conn(0x7f8cec0d1bf0 msgr2=0x7f8cec0b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:13.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.389+0000 7f8cf0a00640 1 -- 172.21.15.94:0/4206122166 shutdown_connections 2024-09-10T23:37:13.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.389+0000 7f8cf0a00640 1 -- 172.21.15.94:0/4206122166 wait complete. 2024-09-10T23:37:13.411 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:37:13.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38f2400640 1 -- 172.21.15.94:0/2351228553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38ec157c60 msgr2=0x7f38ec072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:13.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38f2400640 1 --2- 172.21.15.94:0/2351228553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38ec157c60 0x7f38ec072610 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0cc090 tx=0x7f38dc01ccf0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38f2400640 1 -- 172.21.15.94:0/2351228553 shutdown_connections 2024-09-10T23:37:13.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38f2400640 1 --2- 172.21.15.94:0/2351228553 >> v2:172.21.15.181:3300/0 conn(0x7f38ec070150 0x7f38ec0b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38f2400640 1 --2- 172.21.15.94:0/2351228553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38ec157c60 0x7f38ec072610 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38f2400640 1 -- 172.21.15.94:0/2351228553 >> 172.21.15.94:0/2351228553 conn(0x7f38ec06cdf0 msgr2=0x7f38ec06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:13.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38f2400640 1 -- 172.21.15.94:0/2351228553 shutdown_connections 2024-09-10T23:37:13.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38f2400640 1 -- 172.21.15.94:0/2351228553 wait complete. 2024-09-10T23:37:13.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38f2400640 1 Processor -- start 2024-09-10T23:37:13.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38f2400640 1 -- start start 2024-09-10T23:37:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38f2400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38ec06eef0 0x7f38ec073550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38f2400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38ec070150 0x7f38ec0c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38f2400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38ec157c60 0x7f38ec0c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38f2400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec15b2c0 con 0x7f38ec06eef0 2024-09-10T23:37:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38f2400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec15b140 con 0x7f38ec070150 2024-09-10T23:37:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38f2400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f38ec15b440 con 0x7f38ec157c60 2024-09-10T23:37:13.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38eb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38ec06eef0 0x7f38ec073550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38eb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38ec06eef0 0x7f38ec073550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55122/0 (socket says 172.21.15.94:55122) 2024-09-10T23:37:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38eb400640 1 -- 172.21.15.94:0/232186772 learned_addr learned my addr 172.21.15.94:0/232186772 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38e2a00640 1 --2- 172.21.15.94:0/232186772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38ec070150 0x7f38ec0c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38ebe00640 1 --2- 172.21.15.94:0/232186772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38ec157c60 0x7f38ec0c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38eb400640 1 -- 172.21.15.94:0/232186772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38ec157c60 msgr2=0x7f38ec0c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38eb400640 1 --2- 172.21.15.94:0/232186772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38ec157c60 0x7f38ec0c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38eb400640 1 -- 172.21.15.94:0/232186772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38ec070150 msgr2=0x7f38ec0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38eb400640 1 --2- 172.21.15.94:0/232186772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38ec070150 0x7f38ec0c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.553+0000 7f38eb400640 1 -- 172.21.15.94:0/232186772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec0cc730 con 0x7f38ec06eef0 2024-09-10T23:37:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.557+0000 7f38eb400640 1 --2- 172.21.15.94:0/232186772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38ec06eef0 0x7f38ec073550 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f38dc000f90 tx=0x7f38dc009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.557+0000 7f38e3e00640 1 -- 172.21.15.94:0/232186772 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0bb070 con 0x7f38ec06eef0 2024-09-10T23:37:13.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.557+0000 7f38e3e00640 1 -- 172.21.15.94:0/232186772 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38dc004d40 con 0x7f38ec06eef0 2024-09-10T23:37:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.557+0000 7f38e3e00640 1 -- 172.21.15.94:0/232186772 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0b2040 con 0x7f38ec06eef0 2024-09-10T23:37:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.557+0000 7f38f2400640 1 -- 172.21.15.94:0/232186772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ec0cca10 con 0x7f38ec06eef0 2024-09-10T23:37:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.557+0000 7f38f2400640 1 -- 172.21.15.94:0/232186772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ec0ccfd0 con 0x7f38ec06eef0 2024-09-10T23:37:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.557+0000 7f38f2400640 1 -- 172.21.15.94:0/232186772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ec0b35c0 con 0x7f38ec06eef0 2024-09-10T23:37:13.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.557+0000 7f38e3e00640 1 -- 172.21.15.94:0/232186772 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f38dc0b5030 con 0x7f38ec06eef0 2024-09-10T23:37:13.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.557+0000 7f38e3e00640 1 --2- 172.21.15.94:0/232186772 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f38c8072700 0x7f38c8074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:13.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.557+0000 7f38e3e00640 1 -- 172.21.15.94:0/232186772 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5795+0+0 (secure 0 0 0) 0x7f38dc12cef0 con 0x7f38ec06eef0 2024-09-10T23:37:13.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.557+0000 7f38e2a00640 1 --2- 172.21.15.94:0/232186772 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f38c8072700 0x7f38c8074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:13.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.561+0000 7f38e2a00640 1 --2- 172.21.15.94:0/232186772 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f38c8072700 0x7f38c8074bc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f38ec164810 tx=0x7f38d400b490 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:13.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.561+0000 7f38e3e00640 1 -- 172.21.15.94:0/232186772 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f38dc016970 con 0x7f38ec06eef0 2024-09-10T23:37:13.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.761+0000 7f38f2400640 1 -- 172.21.15.94:0/232186772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38ec0ceb70 con 0x7f38ec06eef0 2024-09-10T23:37:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.761+0000 7f38e3e00640 1 -- 172.21.15.94:0/232186772 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5241 (secure 0 0 0) 0x7f38dc0f7d80 con 0x7f38ec06eef0 2024-09-10T23:37:13.764 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:13.764 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":59,"btime":"2024-09-10T23:37:08:446193+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1215179751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1215179751},{"type":"v1","addr":"172.21.15.94:6834","nonce":1215179751}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6672,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3193009978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":3193009978},{"type":"v1","addr":"172.21.15.181:6835","nonce":3193009978}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1426560042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":1426560042},{"type":"v1","addr":"172.21.15.94:6835","nonce":1426560042}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:36:51.627529+0000","modified":"2024-09-10T23:36:54.641240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195701":{"gid":195701,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6834/1214870454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1214870454},{"type":"v1","addr":"172.21.15.181:6834","nonce":1214870454}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195701,"qdb_cluster":[195701]},"id":6}]} 2024-09-10T23:37:13.764 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 59 2024-09-10T23:37:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.765+0000 7f38f2400640 1 -- 172.21.15.94:0/232186772 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f38c8072700 msgr2=0x7f38c8074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.765+0000 7f38f2400640 1 --2- 172.21.15.94:0/232186772 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f38c8072700 0x7f38c8074bc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f38ec164810 tx=0x7f38d400b490 comp rx=0 tx=0).stop 2024-09-10T23:37:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.765+0000 7f38f2400640 1 -- 172.21.15.94:0/232186772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38ec06eef0 msgr2=0x7f38ec073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.765+0000 7f38f2400640 1 --2- 172.21.15.94:0/232186772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38ec06eef0 0x7f38ec073550 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f38dc000f90 tx=0x7f38dc009360 comp rx=0 tx=0).stop 2024-09-10T23:37:13.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.765+0000 7f38f2400640 1 -- 172.21.15.94:0/232186772 shutdown_connections 2024-09-10T23:37:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.765+0000 7f38f2400640 1 --2- 172.21.15.94:0/232186772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38ec070150 0x7f38ec0c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.765+0000 7f38f2400640 1 --2- 172.21.15.94:0/232186772 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f38c8072700 0x7f38c8074bc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.765+0000 7f38f2400640 1 --2- 172.21.15.94:0/232186772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38ec157c60 0x7f38ec0c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.765+0000 7f38f2400640 1 --2- 172.21.15.94:0/232186772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38ec06eef0 0x7f38ec073550 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.765+0000 7f38f2400640 1 -- 172.21.15.94:0/232186772 >> 172.21.15.94:0/232186772 conn(0x7f38ec06cdf0 msgr2=0x7f38ec156b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.765+0000 7f38f2400640 1 -- 172.21.15.94:0/232186772 shutdown_connections 2024-09-10T23:37:13.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.765+0000 7f38f2400640 1 -- 172.21.15.94:0/232186772 wait complete. 2024-09-10T23:37:13.785 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:37:13.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.925+0000 7fd638600640 1 -- 172.21.15.94:0/607431832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd63015cbc0 msgr2=0x7fd6300cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:13.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.925+0000 7fd638600640 1 --2- 172.21.15.94:0/607431832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd63015cbc0 0x7fd6300cde20 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd6300cd3d0 tx=0x7fd61c01c800 comp rx=0 tx=0).stop 2024-09-10T23:37:13.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.925+0000 7fd638600640 1 -- 172.21.15.94:0/607431832 shutdown_connections 2024-09-10T23:37:13.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.925+0000 7fd638600640 1 --2- 172.21.15.94:0/607431832 >> v2:172.21.15.181:3300/0 conn(0x7fd6300cfd50 0x7fd6300d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.925+0000 7fd638600640 1 --2- 172.21.15.94:0/607431832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd63015cbc0 0x7fd6300cde20 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.925+0000 7fd638600640 1 -- 172.21.15.94:0/607431832 >> 172.21.15.94:0/607431832 conn(0x7fd63006cdf0 msgr2=0x7fd63006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:13.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.925+0000 7fd638600640 1 -- 172.21.15.94:0/607431832 shutdown_connections 2024-09-10T23:37:13.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.925+0000 7fd638600640 1 -- 172.21.15.94:0/607431832 wait complete. 2024-09-10T23:37:13.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.925+0000 7fd638600640 1 Processor -- start 2024-09-10T23:37:13.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.925+0000 7fd638600640 1 -- start start 2024-09-10T23:37:13.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd638600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6300cfd50 0x7fd63014a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:13.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd638600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd630152c10 0x7fd63006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:13.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd638600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd63015cbc0 0x7fd63006fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:13.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd638600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd630168c60 con 0x7fd630152c10 2024-09-10T23:37:13.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd638600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd630168ae0 con 0x7fd63015cbc0 2024-09-10T23:37:13.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd638600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd630168de0 con 0x7fd6300cfd50 2024-09-10T23:37:13.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd635200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd630152c10 0x7fd63006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:13.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd635200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd630152c10 0x7fd63006f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58920/0 (socket says 172.21.15.94:58920) 2024-09-10T23:37:13.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd635200640 1 -- 172.21.15.94:0/2115955188 learned_addr learned my addr 172.21.15.94:0/2115955188 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:13.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd635c00640 1 --2- 172.21.15.94:0/2115955188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6300cfd50 0x7fd63014a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:13.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd636600640 1 --2- 172.21.15.94:0/2115955188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd63015cbc0 0x7fd63006fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:13.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd635200640 1 -- 172.21.15.94:0/2115955188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6300cfd50 msgr2=0x7fd63014a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:13.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd635200640 1 --2- 172.21.15.94:0/2115955188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6300cfd50 0x7fd63014a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd635200640 1 -- 172.21.15.94:0/2115955188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd63015cbc0 msgr2=0x7fd63006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:13.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd635200640 1 --2- 172.21.15.94:0/2115955188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd63015cbc0 0x7fd63006fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:13.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd635200640 1 -- 172.21.15.94:0/2115955188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63014d380 con 0x7fd630152c10 2024-09-10T23:37:13.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd635c00640 1 --2- 172.21.15.94:0/2115955188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6300cfd50 0x7fd63014a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:13.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd635200640 1 --2- 172.21.15.94:0/2115955188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd630152c10 0x7fd63006f740 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd61c000c10 tx=0x7fd61c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:13.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd62d600640 1 -- 172.21.15.94:0/2115955188 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0bc070 con 0x7fd630152c10 2024-09-10T23:37:13.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd62d600640 1 -- 172.21.15.94:0/2115955188 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd61c003600 con 0x7fd630152c10 2024-09-10T23:37:13.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd638600640 1 -- 172.21.15.94:0/2115955188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63014d660 con 0x7fd630152c10 2024-09-10T23:37:13.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd62d600640 1 -- 172.21.15.94:0/2115955188 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0b2040 con 0x7fd630152c10 2024-09-10T23:37:13.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd638600640 1 -- 172.21.15.94:0/2115955188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63014dc20 con 0x7fd630152c10 2024-09-10T23:37:13.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd638600640 1 -- 172.21.15.94:0/2115955188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6300d0200 con 0x7fd630152c10 2024-09-10T23:37:13.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.929+0000 7fd62d600640 1 -- 172.21.15.94:0/2115955188 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fd61c0b5070 con 0x7fd630152c10 2024-09-10T23:37:13.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.933+0000 7fd62d600640 1 --2- 172.21.15.94:0/2115955188 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd610072700 0x7fd610074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:13.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.933+0000 7fd62d600640 1 -- 172.21.15.94:0/2115955188 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5795+0+0 (secure 0 0 0) 0x7fd61c12cdd0 con 0x7fd630152c10 2024-09-10T23:37:13.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.933+0000 7fd635c00640 1 --2- 172.21.15.94:0/2115955188 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd610072700 0x7fd610074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:13.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.933+0000 7fd635c00640 1 --2- 172.21.15.94:0/2115955188 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd610072700 0x7fd610074bc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd624007fd0 tx=0x7fd624009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:13.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:13.933+0000 7fd62d600640 1 -- 172.21.15.94:0/2115955188 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd61c0163f0 con 0x7fd630152c10 2024-09-10T23:37:14.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.101+0000 7fd638600640 1 -- 172.21.15.94:0/2115955188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd630069850 con 0x7fd630152c10 2024-09-10T23:37:14.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.101+0000 7fd62d600640 1 -- 172.21.15.94:0/2115955188 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+16847 (secure 0 0 0) 0x7fd61c0f83a0 con 0x7fd630152c10 2024-09-10T23:37:14.105 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:14.105 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":88,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:37:10.454435+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:36:49.308477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T23:36:50.584653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6835/736952701":"2024-09-11T23:37:07.444153+0000","172.21.15.94:6833/736952701":"2024-09-11T23:37:07.444153+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:37:14.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.105+0000 7fd638600640 1 -- 172.21.15.94:0/2115955188 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd610072700 msgr2=0x7fd610074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:14.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.105+0000 7fd638600640 1 --2- 172.21.15.94:0/2115955188 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd610072700 0x7fd610074bc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd624007fd0 tx=0x7fd624009040 comp rx=0 tx=0).stop 2024-09-10T23:37:14.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.105+0000 7fd638600640 1 -- 172.21.15.94:0/2115955188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd630152c10 msgr2=0x7fd63006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:14.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.105+0000 7fd638600640 1 --2- 172.21.15.94:0/2115955188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd630152c10 0x7fd63006f740 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd61c000c10 tx=0x7fd61c004040 comp rx=0 tx=0).stop 2024-09-10T23:37:14.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.105+0000 7fd638600640 1 -- 172.21.15.94:0/2115955188 shutdown_connections 2024-09-10T23:37:14.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.105+0000 7fd638600640 1 --2- 172.21.15.94:0/2115955188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd63015cbc0 0x7fd63006fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.105+0000 7fd638600640 1 --2- 172.21.15.94:0/2115955188 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd610072700 0x7fd610074bc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.105+0000 7fd638600640 1 --2- 172.21.15.94:0/2115955188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd630152c10 0x7fd63006f740 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.105+0000 7fd638600640 1 --2- 172.21.15.94:0/2115955188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6300cfd50 0x7fd63014a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.105+0000 7fd638600640 1 -- 172.21.15.94:0/2115955188 >> 172.21.15.94:0/2115955188 conn(0x7fd63006cdf0 msgr2=0x7fd630151bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:14.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.105+0000 7fd638600640 1 -- 172.21.15.94:0/2115955188 shutdown_connections 2024-09-10T23:37:14.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.105+0000 7fd638600640 1 -- 172.21.15.94:0/2115955188 wait complete. 2024-09-10T23:37:14.127 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T23:37:14.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f4400640 1 -- 172.21.15.94:0/2018493405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ec0cb4d0 msgr2=0x7f37ec15b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:14.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f4400640 1 --2- 172.21.15.94:0/2018493405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ec0cb4d0 0x7f37ec15b190 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f37ec06b000 tx=0x7f37e401cb80 comp rx=0 tx=0).stop 2024-09-10T23:37:14.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f4400640 1 -- 172.21.15.94:0/2018493405 shutdown_connections 2024-09-10T23:37:14.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f4400640 1 --2- 172.21.15.94:0/2018493405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ec0cb4d0 0x7f37ec15b190 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f4400640 1 --2- 172.21.15.94:0/2018493405 >> v2:172.21.15.181:3300/0 conn(0x7f37ec158790 0x7f37ec06fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f4400640 1 -- 172.21.15.94:0/2018493405 >> 172.21.15.94:0/2018493405 conn(0x7f37ec0cfaf0 msgr2=0x7f37ec0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:14.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f4400640 1 -- 172.21.15.94:0/2018493405 shutdown_connections 2024-09-10T23:37:14.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f4400640 1 -- 172.21.15.94:0/2018493405 wait complete. 2024-09-10T23:37:14.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f4400640 1 Processor -- start 2024-09-10T23:37:14.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f4400640 1 -- start start 2024-09-10T23:37:14.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f4400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ec0c49b0 0x7f37ec153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:14.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f4400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37ec0cb4d0 0x7f37ec1540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:14.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f4400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37ec158790 0x7f37ec0adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:14.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f4400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec167fc0 con 0x7f37ec0c49b0 2024-09-10T23:37:14.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f4400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec167e40 con 0x7f37ec0cb4d0 2024-09-10T23:37:14.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f4400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f37ec168140 con 0x7f37ec158790 2024-09-10T23:37:14.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f1a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ec0c49b0 0x7f37ec153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:14.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f1a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ec0c49b0 0x7f37ec153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58938/0 (socket says 172.21.15.94:58938) 2024-09-10T23:37:14.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f1a00640 1 -- 172.21.15.94:0/268487601 learned_addr learned my addr 172.21.15.94:0/268487601 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:14.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f1000640 1 --2- 172.21.15.94:0/268487601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37ec0cb4d0 0x7f37ec1540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:14.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.269+0000 7f37f2400640 1 --2- 172.21.15.94:0/268487601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37ec158790 0x7f37ec0adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:14.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.273+0000 7f37f1a00640 1 -- 172.21.15.94:0/268487601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37ec158790 msgr2=0x7f37ec0adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:14.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.273+0000 7f37f1a00640 1 --2- 172.21.15.94:0/268487601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37ec158790 0x7f37ec0adb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.273+0000 7f37f1a00640 1 -- 172.21.15.94:0/268487601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37ec0cb4d0 msgr2=0x7f37ec1540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:14.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.273+0000 7f37f1a00640 1 --2- 172.21.15.94:0/268487601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37ec0cb4d0 0x7f37ec1540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.273+0000 7f37f1a00640 1 -- 172.21.15.94:0/268487601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec154600 con 0x7f37ec0c49b0 2024-09-10T23:37:14.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.273+0000 7f37f1a00640 1 --2- 172.21.15.94:0/268487601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ec0c49b0 0x7f37ec153890 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f37e00058d0 tx=0x7f37e000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:14.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.273+0000 7f37e9600640 1 -- 172.21.15.94:0/268487601 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0017070 con 0x7f37ec0c49b0 2024-09-10T23:37:14.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.273+0000 7f37e9600640 1 -- 172.21.15.94:0/268487601 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37e0009cc0 con 0x7f37ec0c49b0 2024-09-10T23:37:14.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.273+0000 7f37f4400640 1 -- 172.21.15.94:0/268487601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ec1fb2c0 con 0x7f37ec0c49b0 2024-09-10T23:37:14.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.273+0000 7f37e9600640 1 -- 172.21.15.94:0/268487601 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e00106c0 con 0x7f37ec0c49b0 2024-09-10T23:37:14.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.273+0000 7f37f4400640 1 -- 172.21.15.94:0/268487601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ec1fb880 con 0x7f37ec0c49b0 2024-09-10T23:37:14.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.273+0000 7f37f4400640 1 -- 172.21.15.94:0/268487601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ec15b190 con 0x7f37ec0c49b0 2024-09-10T23:37:14.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.273+0000 7f37e9600640 1 -- 172.21.15.94:0/268487601 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f37e0010860 con 0x7f37ec0c49b0 2024-09-10T23:37:14.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.277+0000 7f37e9600640 1 --2- 172.21.15.94:0/268487601 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f37cc072700 0x7f37cc074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:14.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.277+0000 7f37e9600640 1 -- 172.21.15.94:0/268487601 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5795+0+0 (secure 0 0 0) 0x7f37e00938c0 con 0x7f37ec0c49b0 2024-09-10T23:37:14.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.277+0000 7f37f1000640 1 --2- 172.21.15.94:0/268487601 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f37cc072700 0x7f37cc074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:14.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.277+0000 7f37f1000640 1 --2- 172.21.15.94:0/268487601 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f37cc072700 0x7f37cc074bc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0aec70 tx=0x7f37d8009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:14.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.281+0000 7f37e9600640 1 -- 172.21.15.94:0/268487601 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f37e005e6c0 con 0x7f37ec0c49b0 2024-09-10T23:37:14.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.477+0000 7f37f4400640 1 -- 172.21.15.94:0/268487601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f37ec0d0850 con 0x7f37ec0c49b0 2024-09-10T23:37:14.481 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-10T23:37:14.477+0000 7f4045a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:37:14.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.481+0000 7f37e9600640 1 -- 172.21.15.94:0/268487601 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) ==== 161+0+0 (secure 0 0 0) 0x7f37e00630f0 con 0x7f37ec0c49b0 2024-09-10T23:37:14.483 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T23:37:14.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.485+0000 7f37f4400640 1 -- 172.21.15.94:0/268487601 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f37cc072700 msgr2=0x7f37cc074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:14.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.485+0000 7f37f4400640 1 --2- 172.21.15.94:0/268487601 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f37cc072700 0x7f37cc074bc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0aec70 tx=0x7f37d8009040 comp rx=0 tx=0).stop 2024-09-10T23:37:14.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.485+0000 7f37f4400640 1 -- 172.21.15.94:0/268487601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ec0c49b0 msgr2=0x7f37ec153890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:14.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.485+0000 7f37f4400640 1 --2- 172.21.15.94:0/268487601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ec0c49b0 0x7f37ec153890 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f37e00058d0 tx=0x7f37e000dde0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.485+0000 7f37f4400640 1 -- 172.21.15.94:0/268487601 shutdown_connections 2024-09-10T23:37:14.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.485+0000 7f37f4400640 1 --2- 172.21.15.94:0/268487601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37ec0cb4d0 0x7f37ec1540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.485+0000 7f37f4400640 1 --2- 172.21.15.94:0/268487601 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f37cc072700 0x7f37cc074bc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.485+0000 7f37f4400640 1 --2- 172.21.15.94:0/268487601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37ec158790 0x7f37ec0adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.485+0000 7f37f4400640 1 --2- 172.21.15.94:0/268487601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37ec0c49b0 0x7f37ec153890 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.485+0000 7f37f4400640 1 -- 172.21.15.94:0/268487601 >> 172.21.15.94:0/268487601 conn(0x7f37ec0cfaf0 msgr2=0x7f37ec06f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:14.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.485+0000 7f37f4400640 1 -- 172.21.15.94:0/268487601 shutdown_connections 2024-09-10T23:37:14.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.485+0000 7f37f4400640 1 -- 172.21.15.94:0/268487601 wait complete. 2024-09-10T23:37:14.492 INFO:tasks.ceph.mds.b.smithi181.stderr: -1487> 2024-09-10T23:36:54.206+0000 7f3e26c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:37:14.505 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T23:37:14.538 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:37:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.649+0000 7fc9e6200640 1 -- 172.21.15.94:0/606342583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e00b0ee0 msgr2=0x7fc9e006e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.649+0000 7fc9e6200640 1 --2- 172.21.15.94:0/606342583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e00b0ee0 0x7fc9e006e270 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0073830 tx=0x7fc9d801ccf0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.649+0000 7fc9e6200640 1 -- 172.21.15.94:0/606342583 shutdown_connections 2024-09-10T23:37:14.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.649+0000 7fc9e6200640 1 --2- 172.21.15.94:0/606342583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e00b0ee0 0x7fc9e006e270 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.649+0000 7fc9e6200640 1 --2- 172.21.15.94:0/606342583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9e00cee00 0x7fc9e006dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.649+0000 7fc9e6200640 1 --2- 172.21.15.94:0/606342583 >> v2:172.21.15.181:3300/0 conn(0x7fc9e00ce4e0 0x7fc9e00ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.649+0000 7fc9e6200640 1 -- 172.21.15.94:0/606342583 >> 172.21.15.94:0/606342583 conn(0x7fc9e00d1c60 msgr2=0x7fc9e00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.649+0000 7fc9e6200640 1 -- 172.21.15.94:0/606342583 shutdown_connections 2024-09-10T23:37:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.649+0000 7fc9e6200640 1 -- 172.21.15.94:0/606342583 wait complete. 2024-09-10T23:37:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.649+0000 7fc9e6200640 1 Processor -- start 2024-09-10T23:37:14.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9e6200640 1 -- start start 2024-09-10T23:37:14.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9e6200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9e00b0ee0 0x7fc9e00c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:14.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9e6200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e00ce4e0 0x7fc9e0156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:14.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9e6200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9e00cee00 0x7fc9e0157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:14.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9e6200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e00cd300 con 0x7fc9e00ce4e0 2024-09-10T23:37:14.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9e6200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e00cd180 con 0x7fc9e00cee00 2024-09-10T23:37:14.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9e6200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e00cd480 con 0x7fc9e00b0ee0 2024-09-10T23:37:14.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9d6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e00ce4e0 0x7fc9e0156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:14.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9d6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e00ce4e0 0x7fc9e0156610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58944/0 (socket says 172.21.15.94:58944) 2024-09-10T23:37:14.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9d6a00640 1 -- 172.21.15.94:0/443310930 learned_addr learned my addr 172.21.15.94:0/443310930 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:14.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9df400640 1 --2- 172.21.15.94:0/443310930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9e00b0ee0 0x7fc9e00c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:14.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9dfe00640 1 --2- 172.21.15.94:0/443310930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9e00cee00 0x7fc9e0157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:14.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9d6a00640 1 -- 172.21.15.94:0/443310930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9e00b0ee0 msgr2=0x7fc9e00c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:14.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9d6a00640 1 --2- 172.21.15.94:0/443310930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9e00b0ee0 0x7fc9e00c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9d6a00640 1 -- 172.21.15.94:0/443310930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9e00cee00 msgr2=0x7fc9e0157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:14.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9d6a00640 1 --2- 172.21.15.94:0/443310930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9e00cee00 0x7fc9e0157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:14.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9d6a00640 1 -- 172.21.15.94:0/443310930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e015aa50 con 0x7fc9e00ce4e0 2024-09-10T23:37:14.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9dfe00640 1 --2- 172.21.15.94:0/443310930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9e00cee00 0x7fc9e0157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:37:14.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9d6a00640 1 --2- 172.21.15.94:0/443310930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e00ce4e0 0x7fc9e0156610 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8005d10 tx=0x7fc9c800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:14.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9d7e00640 1 -- 172.21.15.94:0/443310930 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8017070 con 0x7fc9e00ce4e0 2024-09-10T23:37:14.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9d7e00640 1 -- 172.21.15.94:0/443310930 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9c8009b20 con 0x7fc9e00ce4e0 2024-09-10T23:37:14.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9d7e00640 1 -- 172.21.15.94:0/443310930 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8005020 con 0x7fc9e00ce4e0 2024-09-10T23:37:14.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9e6200640 1 -- 172.21.15.94:0/443310930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e00aeac0 con 0x7fc9e00ce4e0 2024-09-10T23:37:14.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9e6200640 1 -- 172.21.15.94:0/443310930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e015a640 con 0x7fc9e00ce4e0 2024-09-10T23:37:14.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.653+0000 7fc9e6200640 1 -- 172.21.15.94:0/443310930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e00ce8c0 con 0x7fc9e00ce4e0 2024-09-10T23:37:14.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.657+0000 7fc9d7e00640 1 -- 172.21.15.94:0/443310930 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fc9c80051c0 con 0x7fc9e00ce4e0 2024-09-10T23:37:14.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.657+0000 7fc9d7e00640 1 --2- 172.21.15.94:0/443310930 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc9bc072730 0x7fc9bc074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:14.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.657+0000 7fc9d7e00640 1 -- 172.21.15.94:0/443310930 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5881+0+0 (secure 0 0 0) 0x7fc9c8093630 con 0x7fc9e00ce4e0 2024-09-10T23:37:14.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.657+0000 7fc9df400640 1 --2- 172.21.15.94:0/443310930 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc9bc072730 0x7fc9bc074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:14.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.657+0000 7fc9df400640 1 --2- 172.21.15.94:0/443310930 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc9bc072730 0x7fc9bc074bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000bb70 tx=0x7fc9d000b2e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:14.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.661+0000 7fc9d7e00640 1 -- 172.21.15.94:0/443310930 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc9c805e3e0 con 0x7fc9e00ce4e0 2024-09-10T23:37:14.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:14.861+0000 7fc9e6200640 1 -- 172.21.15.94:0/443310930 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc9e00d15b0 con 0x7fc9e00ce4e0 2024-09-10T23:37:15.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.485+0000 7fc9d7e00640 1 -- 172.21.15.94:0/443310930 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) ==== 104+0+0 (secure 0 0 0) 0x7fc9c8062e10 con 0x7fc9e00ce4e0 2024-09-10T23:37:15.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.489+0000 7fc9e6200640 1 -- 172.21.15.94:0/443310930 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc9bc072730 msgr2=0x7fc9bc074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:15.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.489+0000 7fc9e6200640 1 --2- 172.21.15.94:0/443310930 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc9bc072730 0x7fc9bc074bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000bb70 tx=0x7fc9d000b2e0 comp rx=0 tx=0).stop 2024-09-10T23:37:15.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.489+0000 7fc9e6200640 1 -- 172.21.15.94:0/443310930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e00ce4e0 msgr2=0x7fc9e0156610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:15.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.489+0000 7fc9e6200640 1 --2- 172.21.15.94:0/443310930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e00ce4e0 0x7fc9e0156610 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8005d10 tx=0x7fc9c800dbb0 comp rx=0 tx=0).stop 2024-09-10T23:37:15.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.489+0000 7fc9e6200640 1 -- 172.21.15.94:0/443310930 shutdown_connections 2024-09-10T23:37:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.489+0000 7fc9e6200640 1 --2- 172.21.15.94:0/443310930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9e00cee00 0x7fc9e0157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.489+0000 7fc9e6200640 1 --2- 172.21.15.94:0/443310930 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc9bc072730 0x7fc9bc074bf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.489+0000 7fc9e6200640 1 --2- 172.21.15.94:0/443310930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9e00ce4e0 0x7fc9e0156610 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.489+0000 7fc9e6200640 1 --2- 172.21.15.94:0/443310930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9e00b0ee0 0x7fc9e00c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.489+0000 7fc9e6200640 1 -- 172.21.15.94:0/443310930 >> 172.21.15.94:0/443310930 conn(0x7fc9e00d1c60 msgr2=0x7fc9e00b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.489+0000 7fc9e6200640 1 -- 172.21.15.94:0/443310930 shutdown_connections 2024-09-10T23:37:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.489+0000 7fc9e6200640 1 -- 172.21.15.94:0/443310930 wait complete. 2024-09-10T23:37:15.511 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T23:37:15.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.653+0000 7fc36de00640 1 -- 172.21.15.94:0/3887444567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc368070150 msgr2=0x7fc3680b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:15.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.653+0000 7fc36de00640 1 --2- 172.21.15.94:0/3887444567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc368070150 0x7fc3680b35c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc360009c80 tx=0x7fc36001cc80 comp rx=0 tx=0).stop 2024-09-10T23:37:15.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.653+0000 7fc36de00640 1 -- 172.21.15.94:0/3887444567 shutdown_connections 2024-09-10T23:37:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.653+0000 7fc36de00640 1 --2- 172.21.15.94:0/3887444567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc368070150 0x7fc3680b35c0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.653+0000 7fc36de00640 1 --2- 172.21.15.94:0/3887444567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc36806f600 0x7fc368072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.653+0000 7fc36de00640 1 -- 172.21.15.94:0/3887444567 >> 172.21.15.94:0/3887444567 conn(0x7fc36806cdf0 msgr2=0x7fc36806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.653+0000 7fc36de00640 1 -- 172.21.15.94:0/3887444567 shutdown_connections 2024-09-10T23:37:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.653+0000 7fc36de00640 1 -- 172.21.15.94:0/3887444567 wait complete. 2024-09-10T23:37:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc36de00640 1 Processor -- start 2024-09-10T23:37:15.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc36de00640 1 -- start start 2024-09-10T23:37:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc36de00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc36806f600 0x7fc3681488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc36de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc368070150 0x7fc368155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc36de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc368157c60 0x7fc368152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc36de00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc36815b3b0 con 0x7fc368157c60 2024-09-10T23:37:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc36de00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc36815b230 con 0x7fc368070150 2024-09-10T23:37:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc36de00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc36815b530 con 0x7fc36806f600 2024-09-10T23:37:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc367400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc368157c60 0x7fc368152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc367400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc368157c60 0x7fc368152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58966/0 (socket says 172.21.15.94:58966) 2024-09-10T23:37:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc367400640 1 -- 172.21.15.94:0/3212855800 learned_addr learned my addr 172.21.15.94:0/3212855800 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc366a00640 1 --2- 172.21.15.94:0/3212855800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc36806f600 0x7fc3681488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc366000640 1 --2- 172.21.15.94:0/3212855800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc368070150 0x7fc368155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc367400640 1 -- 172.21.15.94:0/3212855800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc36806f600 msgr2=0x7fc3681488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc367400640 1 --2- 172.21.15.94:0/3212855800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc36806f600 0x7fc3681488d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc367400640 1 -- 172.21.15.94:0/3212855800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc368070150 msgr2=0x7fc368155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:15.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc367400640 1 --2- 172.21.15.94:0/3212855800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc368070150 0x7fc368155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:15.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc367400640 1 -- 172.21.15.94:0/3212855800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3681564e0 con 0x7fc368157c60 2024-09-10T23:37:15.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc367400640 1 --2- 172.21.15.94:0/3212855800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc368157c60 0x7fc368152350 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc360009bd0 tx=0x7fc360009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:15.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc35f400640 1 -- 172.21.15.94:0/3212855800 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3600bb070 con 0x7fc368157c60 2024-09-10T23:37:15.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc35f400640 1 -- 172.21.15.94:0/3212855800 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc360004e00 con 0x7fc368157c60 2024-09-10T23:37:15.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc35f400640 1 -- 172.21.15.94:0/3212855800 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3600b2040 con 0x7fc368157c60 2024-09-10T23:37:15.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc36de00640 1 -- 172.21.15.94:0/3212855800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36814f8c0 con 0x7fc368157c60 2024-09-10T23:37:15.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.657+0000 7fc36de00640 1 -- 172.21.15.94:0/3212855800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3680ce3f0 con 0x7fc368157c60 2024-09-10T23:37:15.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.661+0000 7fc36de00640 1 -- 172.21.15.94:0/3212855800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc368072610 con 0x7fc368157c60 2024-09-10T23:37:15.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.661+0000 7fc35f400640 1 -- 172.21.15.94:0/3212855800 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fc3600b5030 con 0x7fc368157c60 2024-09-10T23:37:15.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.661+0000 7fc35f400640 1 --2- 172.21.15.94:0/3212855800 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc348072700 0x7fc348074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:15.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.661+0000 7fc35f400640 1 -- 172.21.15.94:0/3212855800 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5881+0+0 (secure 0 0 0) 0x7fc36012cff0 con 0x7fc368157c60 2024-09-10T23:37:15.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.661+0000 7fc366a00640 1 --2- 172.21.15.94:0/3212855800 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc348072700 0x7fc348074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:15.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.661+0000 7fc366a00640 1 --2- 172.21.15.94:0/3212855800 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc348072700 0x7fc348074bc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc358007fd0 tx=0x7fc358009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:15.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.665+0000 7fc35f400640 1 -- 172.21.15.94:0/3212855800 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc360016990 con 0x7fc368157c60 2024-09-10T23:37:15.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:15.833+0000 7fc36de00640 1 -- 172.21.15.94:0/3212855800 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc36806e7e0 con 0x7fc368157c60 2024-09-10T23:37:16.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.489+0000 7fc35f400640 1 -- 172.21.15.94:0/3212855800 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) ==== 181+0+0 (secure 0 0 0) 0x7fc3600f7d30 con 0x7fc368157c60 2024-09-10T23:37:16.490 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-09-10T23:37:16.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.489+0000 7fc36de00640 1 -- 172.21.15.94:0/3212855800 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc348072700 msgr2=0x7fc348074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:16.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.489+0000 7fc36de00640 1 --2- 172.21.15.94:0/3212855800 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc348072700 0x7fc348074bc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc358007fd0 tx=0x7fc358009040 comp rx=0 tx=0).stop 2024-09-10T23:37:16.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.489+0000 7fc36de00640 1 -- 172.21.15.94:0/3212855800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc368157c60 msgr2=0x7fc368152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:16.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.489+0000 7fc36de00640 1 --2- 172.21.15.94:0/3212855800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc368157c60 0x7fc368152350 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc360009bd0 tx=0x7fc360009390 comp rx=0 tx=0).stop 2024-09-10T23:37:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.489+0000 7fc36de00640 1 -- 172.21.15.94:0/3212855800 shutdown_connections 2024-09-10T23:37:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.489+0000 7fc36de00640 1 --2- 172.21.15.94:0/3212855800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc368070150 0x7fc368155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.489+0000 7fc36de00640 1 --2- 172.21.15.94:0/3212855800 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc348072700 0x7fc348074bc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.489+0000 7fc36de00640 1 --2- 172.21.15.94:0/3212855800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc368157c60 0x7fc368152350 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.489+0000 7fc36de00640 1 --2- 172.21.15.94:0/3212855800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc36806f600 0x7fc3681488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.489+0000 7fc36de00640 1 -- 172.21.15.94:0/3212855800 >> 172.21.15.94:0/3212855800 conn(0x7fc36806cdf0 msgr2=0x7fc36806eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.493+0000 7fc36de00640 1 -- 172.21.15.94:0/3212855800 shutdown_connections 2024-09-10T23:37:16.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.493+0000 7fc36de00640 1 -- 172.21.15.94:0/3212855800 wait complete. 2024-09-10T23:37:16.516 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T23:37:16.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795c600640 1 -- 172.21.15.94:0/3825608720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7954073970 msgr2=0x7f7954073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:16.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795c600640 1 --2- 172.21.15.94:0/3825608720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7954073970 0x7f7954073d50 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f795406a320 tx=0x7f794801ccf0 comp rx=0 tx=0).stop 2024-09-10T23:37:16.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795c600640 1 -- 172.21.15.94:0/3825608720 shutdown_connections 2024-09-10T23:37:16.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795c600640 1 --2- 172.21.15.94:0/3825608720 >> v2:172.21.15.181:3300/0 conn(0x7f79540cdea0 0x7f795415ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:16.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795c600640 1 --2- 172.21.15.94:0/3825608720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7954073970 0x7f7954073d50 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:16.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795c600640 1 -- 172.21.15.94:0/3825608720 >> 172.21.15.94:0/3825608720 conn(0x7f79540cc900 msgr2=0x7f79540ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:16.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795c600640 1 -- 172.21.15.94:0/3825608720 shutdown_connections 2024-09-10T23:37:16.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795c600640 1 -- 172.21.15.94:0/3825608720 wait complete. 2024-09-10T23:37:16.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795c600640 1 Processor -- start 2024-09-10T23:37:16.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795c600640 1 -- start start 2024-09-10T23:37:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795c600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7954073970 0x7f795414c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795c600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79540cdea0 0x7f795414b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795c600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f795415f140 0x7f79540c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:16.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795c600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f79541683c0 con 0x7f795415f140 2024-09-10T23:37:16.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795c600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7954168240 con 0x7f79540cdea0 2024-09-10T23:37:16.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795c600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f7954168540 con 0x7f7954073970 2024-09-10T23:37:16.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.661+0000 7f795a600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f795415f140 0x7f79540c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:16.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f795a600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f795415f140 0x7f79540c7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58984/0 (socket says 172.21.15.94:58984) 2024-09-10T23:37:16.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f795a600640 1 -- 172.21.15.94:0/859991587 learned_addr learned my addr 172.21.15.94:0/859991587 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:16.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f7959c00640 1 --2- 172.21.15.94:0/859991587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7954073970 0x7f795414c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:16.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f7959200640 1 --2- 172.21.15.94:0/859991587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79540cdea0 0x7f795414b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:16.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f795a600640 1 -- 172.21.15.94:0/859991587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7954073970 msgr2=0x7f795414c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:16.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f795a600640 1 --2- 172.21.15.94:0/859991587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7954073970 0x7f795414c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:16.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f795a600640 1 -- 172.21.15.94:0/859991587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79540cdea0 msgr2=0x7f795414b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:16.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f795a600640 1 --2- 172.21.15.94:0/859991587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79540cdea0 0x7f795414b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:16.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f795a600640 1 -- 172.21.15.94:0/859991587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795414f0f0 con 0x7f795415f140 2024-09-10T23:37:16.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f795a600640 1 --2- 172.21.15.94:0/859991587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f795415f140 0x7f79540c7630 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c390 tx=0x7f794c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:16.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f7951600640 1 -- 172.21.15.94:0/859991587 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c017070 con 0x7f795415f140 2024-09-10T23:37:16.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f7951600640 1 -- 172.21.15.94:0/859991587 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f794c0053c0 con 0x7f795415f140 2024-09-10T23:37:16.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f7951600640 1 -- 172.21.15.94:0/859991587 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c003860 con 0x7f795415f140 2024-09-10T23:37:16.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f795c600640 1 -- 172.21.15.94:0/859991587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795414bb20 con 0x7f795415f140 2024-09-10T23:37:16.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f795c600640 1 -- 172.21.15.94:0/859991587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795414e7c0 con 0x7f795415f140 2024-09-10T23:37:16.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f795c600640 1 -- 172.21.15.94:0/859991587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79540cd650 con 0x7f795415f140 2024-09-10T23:37:16.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.665+0000 7f7951600640 1 -- 172.21.15.94:0/859991587 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f794c003a00 con 0x7f795415f140 2024-09-10T23:37:16.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.669+0000 7f7951600640 1 --2- 172.21.15.94:0/859991587 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7934072730 0x7f7934074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:16.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.669+0000 7f7951600640 1 -- 172.21.15.94:0/859991587 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5443+0+0 (secure 0 0 0) 0x7f794c0928f0 con 0x7f795415f140 2024-09-10T23:37:16.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.669+0000 7f7959c00640 1 --2- 172.21.15.94:0/859991587 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7934072730 0x7f7934074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:16.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.669+0000 7f7959c00640 1 --2- 172.21.15.94:0/859991587 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7934072730 0x7f7934074bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f79480af000 tx=0x7f794801b530 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:16.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.673+0000 7f7951600640 1 -- 172.21.15.94:0/859991587 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f794c05d8f0 con 0x7f795415f140 2024-09-10T23:37:16.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:16.845+0000 7f795c600640 1 -- 172.21.15.94:0/859991587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f79541fb8d0 con 0x7f795415f140 2024-09-10T23:37:17.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.493+0000 7f7951600640 1 -- 172.21.15.94:0/859991587 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) ==== 169+0+0 (secure 0 0 0) 0x7f794c062320 con 0x7f795415f140 2024-09-10T23:37:17.496 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-09-10T23:37:17.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.497+0000 7f795c600640 1 -- 172.21.15.94:0/859991587 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7934072730 msgr2=0x7f7934074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:17.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.497+0000 7f795c600640 1 --2- 172.21.15.94:0/859991587 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7934072730 0x7f7934074bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f79480af000 tx=0x7f794801b530 comp rx=0 tx=0).stop 2024-09-10T23:37:17.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.497+0000 7f795c600640 1 -- 172.21.15.94:0/859991587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f795415f140 msgr2=0x7f79540c7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:17.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.497+0000 7f795c600640 1 --2- 172.21.15.94:0/859991587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f795415f140 0x7f79540c7630 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c390 tx=0x7f794c00c850 comp rx=0 tx=0).stop 2024-09-10T23:37:17.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.497+0000 7f795c600640 1 -- 172.21.15.94:0/859991587 shutdown_connections 2024-09-10T23:37:17.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.497+0000 7f795c600640 1 --2- 172.21.15.94:0/859991587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79540cdea0 0x7f795414b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:17.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.497+0000 7f795c600640 1 --2- 172.21.15.94:0/859991587 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7934072730 0x7f7934074bf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:17.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.497+0000 7f795c600640 1 --2- 172.21.15.94:0/859991587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f795415f140 0x7f79540c7630 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:17.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.497+0000 7f795c600640 1 --2- 172.21.15.94:0/859991587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7954073970 0x7f795414c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:17.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.497+0000 7f795c600640 1 -- 172.21.15.94:0/859991587 >> 172.21.15.94:0/859991587 conn(0x7f79540cc900 msgr2=0x7f7954165440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:17.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.497+0000 7f795c600640 1 -- 172.21.15.94:0/859991587 shutdown_connections 2024-09-10T23:37:17.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.497+0000 7f795c600640 1 -- 172.21.15.94:0/859991587 wait complete. 2024-09-10T23:37:17.527 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T23:37:17.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.669+0000 7ff25f200640 1 -- 172.21.15.94:0/1583714274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff25815b880 msgr2=0x7ff25806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:17.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.669+0000 7ff25f200640 1 --2- 172.21.15.94:0/1583714274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff25815b880 0x7ff25806f5c0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff258072370 tx=0x7ff25401ccc0 comp rx=0 tx=0).stop 2024-09-10T23:37:17.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.669+0000 7ff25f200640 1 -- 172.21.15.94:0/1583714274 shutdown_connections 2024-09-10T23:37:17.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.669+0000 7ff25f200640 1 --2- 172.21.15.94:0/1583714274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff25815b880 0x7ff25806f5c0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:17.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.669+0000 7ff25f200640 1 --2- 172.21.15.94:0/1583714274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2580cbee0 0x7ff258073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:17.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.669+0000 7ff25f200640 1 --2- 172.21.15.94:0/1583714274 >> v2:172.21.15.181:3300/0 conn(0x7ff2580738b0 0x7ff25815f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:17.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.669+0000 7ff25f200640 1 -- 172.21.15.94:0/1583714274 >> 172.21.15.94:0/1583714274 conn(0x7ff2580cfca0 msgr2=0x7ff2580d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:17.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.669+0000 7ff25f200640 1 -- 172.21.15.94:0/1583714274 shutdown_connections 2024-09-10T23:37:17.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.669+0000 7ff25f200640 1 -- 172.21.15.94:0/1583714274 wait complete. 2024-09-10T23:37:17.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25f200640 1 Processor -- start 2024-09-10T23:37:17.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25f200640 1 -- start start 2024-09-10T23:37:17.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2580738b0 0x7ff258155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:17.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2580cbee0 0x7ff2580c2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:17.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25f200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25815b880 0x7ff2581597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:17.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff258073190 con 0x7ff2580738b0 2024-09-10T23:37:17.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25f200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff258073010 con 0x7ff2580cbee0 2024-09-10T23:37:17.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25f200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff258073310 con 0x7ff25815b880 2024-09-10T23:37:17.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2580738b0 0x7ff258155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:17.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2580738b0 0x7ff258155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59008/0 (socket says 172.21.15.94:59008) 2024-09-10T23:37:17.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25de00640 1 -- 172.21.15.94:0/2011254376 learned_addr learned my addr 172.21.15.94:0/2011254376 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:17.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25e800640 1 --2- 172.21.15.94:0/2011254376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25815b880 0x7ff2581597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:17.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25d400640 1 --2- 172.21.15.94:0/2011254376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2580cbee0 0x7ff2580c2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:17.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25de00640 1 -- 172.21.15.94:0/2011254376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25815b880 msgr2=0x7ff2581597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:17.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25de00640 1 --2- 172.21.15.94:0/2011254376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25815b880 0x7ff2581597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:17.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25de00640 1 -- 172.21.15.94:0/2011254376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2580cbee0 msgr2=0x7ff2580c2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:17.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25de00640 1 --2- 172.21.15.94:0/2011254376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2580cbee0 0x7ff2580c2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:17.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25de00640 1 -- 172.21.15.94:0/2011254376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff258152e20 con 0x7ff2580738b0 2024-09-10T23:37:17.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25de00640 1 --2- 172.21.15.94:0/2011254376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2580738b0 0x7ff258155ee0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00d720 tx=0x7ff24c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:17.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff252000640 1 -- 172.21.15.94:0/2011254376 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c017070 con 0x7ff2580738b0 2024-09-10T23:37:17.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff252000640 1 -- 172.21.15.94:0/2011254376 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff24c009d20 con 0x7ff2580738b0 2024-09-10T23:37:17.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff252000640 1 -- 172.21.15.94:0/2011254376 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c004280 con 0x7ff2580738b0 2024-09-10T23:37:17.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25f200640 1 -- 172.21.15.94:0/2011254376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff258153100 con 0x7ff2580738b0 2024-09-10T23:37:17.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.673+0000 7ff25f200640 1 -- 172.21.15.94:0/2011254376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2580c3250 con 0x7ff2580738b0 2024-09-10T23:37:17.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.677+0000 7ff25f200640 1 -- 172.21.15.94:0/2011254376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff258073c90 con 0x7ff2580738b0 2024-09-10T23:37:17.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.677+0000 7ff252000640 1 -- 172.21.15.94:0/2011254376 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7ff24c005020 con 0x7ff2580738b0 2024-09-10T23:37:17.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.677+0000 7ff252000640 1 --2- 172.21.15.94:0/2011254376 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff23c072730 0x7ff23c074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:17.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.677+0000 7ff252000640 1 -- 172.21.15.94:0/2011254376 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5029+0+0 (secure 0 0 0) 0x7ff24c093710 con 0x7ff2580738b0 2024-09-10T23:37:17.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.677+0000 7ff25d400640 1 --2- 172.21.15.94:0/2011254376 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff23c072730 0x7ff23c074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:17.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.677+0000 7ff25d400640 1 --2- 172.21.15.94:0/2011254376 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff23c072730 0x7ff23c074bf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff2480058d0 tx=0x7ff248005860 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:17.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.681+0000 7ff252000640 1 -- 172.21.15.94:0/2011254376 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff24c020020 con 0x7ff2580738b0 2024-09-10T23:37:17.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.857+0000 7ff25f200640 1 -- 172.21.15.94:0/2011254376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7ff2580b5c20 con 0x7ff2580738b0 2024-09-10T23:37:17.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.861+0000 7ff252000640 1 -- 172.21.15.94:0/2011254376 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v37) ==== 71+0+0 (secure 0 0 0) 0x7ff24c05edd0 con 0x7ff2580738b0 2024-09-10T23:37:17.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.865+0000 7ff25f200640 1 -- 172.21.15.94:0/2011254376 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff23c072730 msgr2=0x7ff23c074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.865+0000 7ff25f200640 1 --2- 172.21.15.94:0/2011254376 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff23c072730 0x7ff23c074bf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff2480058d0 tx=0x7ff248005860 comp rx=0 tx=0).stop 2024-09-10T23:37:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.865+0000 7ff25f200640 1 -- 172.21.15.94:0/2011254376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2580738b0 msgr2=0x7ff258155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.865+0000 7ff25f200640 1 --2- 172.21.15.94:0/2011254376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2580738b0 0x7ff258155ee0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00d720 tx=0x7ff24c00dbe0 comp rx=0 tx=0).stop 2024-09-10T23:37:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.865+0000 7ff25f200640 1 -- 172.21.15.94:0/2011254376 shutdown_connections 2024-09-10T23:37:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.865+0000 7ff25f200640 1 --2- 172.21.15.94:0/2011254376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2580cbee0 0x7ff2580c2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.865+0000 7ff25f200640 1 --2- 172.21.15.94:0/2011254376 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff23c072730 0x7ff23c074bf0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:17.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.865+0000 7ff25f200640 1 --2- 172.21.15.94:0/2011254376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25815b880 0x7ff2581597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:17.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.865+0000 7ff25f200640 1 --2- 172.21.15.94:0/2011254376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2580738b0 0x7ff258155ee0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:17.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.865+0000 7ff25f200640 1 -- 172.21.15.94:0/2011254376 >> 172.21.15.94:0/2011254376 conn(0x7ff2580cfca0 msgr2=0x7ff25815c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:17.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.865+0000 7ff25f200640 1 -- 172.21.15.94:0/2011254376 shutdown_connections 2024-09-10T23:37:17.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:17.865+0000 7ff25f200640 1 -- 172.21.15.94:0/2011254376 wait complete. 2024-09-10T23:37:17.886 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-10T23:37:18.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.025+0000 7f81c4600640 1 -- 172.21.15.94:0/3806322385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81bc073ff0 msgr2=0x7f81bc0cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:18.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.025+0000 7f81c4600640 1 --2- 172.21.15.94:0/3806322385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81bc073ff0 0x7f81bc0cfda0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f81bc06f240 tx=0x7f81b001c890 comp rx=0 tx=0).stop 2024-09-10T23:37:18.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.025+0000 7f81c4600640 1 -- 172.21.15.94:0/3806322385 shutdown_connections 2024-09-10T23:37:18.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.025+0000 7f81c4600640 1 --2- 172.21.15.94:0/3806322385 >> v2:172.21.15.181:3300/0 conn(0x7f81bc0d02e0 0x7f81bc15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.025+0000 7f81c4600640 1 --2- 172.21.15.94:0/3806322385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81bc073ff0 0x7f81bc0cfda0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.025+0000 7f81c4600640 1 -- 172.21.15.94:0/3806322385 >> 172.21.15.94:0/3806322385 conn(0x7f81bc06cdf0 msgr2=0x7f81bc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:18.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.025+0000 7f81c4600640 1 -- 172.21.15.94:0/3806322385 shutdown_connections 2024-09-10T23:37:18.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.025+0000 7f81c4600640 1 -- 172.21.15.94:0/3806322385 wait complete. 2024-09-10T23:37:18.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.025+0000 7f81c4600640 1 Processor -- start 2024-09-10T23:37:18.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.025+0000 7f81c4600640 1 -- start start 2024-09-10T23:37:18.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c4600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81bc073ff0 0x7f81bc13b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:18.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c4600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81bc0d02e0 0x7f81bc13b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:18.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c4600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81bc15ecb0 0x7f81bc06f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:18.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c4600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc167930 con 0x7f81bc073ff0 2024-09-10T23:37:18.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c4600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc1677b0 con 0x7f81bc0d02e0 2024-09-10T23:37:18.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c4600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f81bc167ab0 con 0x7f81bc15ecb0 2024-09-10T23:37:18.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c1c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81bc073ff0 0x7f81bc13b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:18.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c1c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81bc073ff0 0x7f81bc13b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59028/0 (socket says 172.21.15.94:59028) 2024-09-10T23:37:18.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c1c00640 1 -- 172.21.15.94:0/2161242919 learned_addr learned my addr 172.21.15.94:0/2161242919 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:18.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c1200640 1 --2- 172.21.15.94:0/2161242919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81bc0d02e0 0x7f81bc13b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:18.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c2600640 1 --2- 172.21.15.94:0/2161242919 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81bc15ecb0 0x7f81bc06f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:18.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c1c00640 1 -- 172.21.15.94:0/2161242919 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81bc15ecb0 msgr2=0x7f81bc06f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:18.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c1c00640 1 --2- 172.21.15.94:0/2161242919 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81bc15ecb0 0x7f81bc06f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c1c00640 1 -- 172.21.15.94:0/2161242919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81bc0d02e0 msgr2=0x7f81bc13b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:18.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c1c00640 1 --2- 172.21.15.94:0/2161242919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81bc0d02e0 0x7f81bc13b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c1c00640 1 -- 172.21.15.94:0/2161242919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81bc13b540 con 0x7f81bc073ff0 2024-09-10T23:37:18.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c2600640 1 --2- 172.21.15.94:0/2161242919 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81bc15ecb0 0x7f81bc06f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:37:18.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c1c00640 1 --2- 172.21.15.94:0/2161242919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81bc073ff0 0x7f81bc13b000 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f81b00b0040 tx=0x7f81b0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:18.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81b9600640 1 -- 172.21.15.94:0/2161242919 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b00b2040 con 0x7f81bc073ff0 2024-09-10T23:37:18.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81b9600640 1 -- 172.21.15.94:0/2161242919 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81b0005030 con 0x7f81bc073ff0 2024-09-10T23:37:18.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81b9600640 1 -- 172.21.15.94:0/2161242919 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0005480 con 0x7f81bc073ff0 2024-09-10T23:37:18.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c4600640 1 -- 172.21.15.94:0/2161242919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81bc1f8ea0 con 0x7f81bc073ff0 2024-09-10T23:37:18.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c4600640 1 -- 172.21.15.94:0/2161242919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81bc1f9460 con 0x7f81bc073ff0 2024-09-10T23:37:18.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.029+0000 7f81c4600640 1 -- 172.21.15.94:0/2161242919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81bc0cfda0 con 0x7f81bc073ff0 2024-09-10T23:37:18.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.033+0000 7f81b9600640 1 -- 172.21.15.94:0/2161242919 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f81b00046d0 con 0x7f81bc073ff0 2024-09-10T23:37:18.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.033+0000 7f81b9600640 1 --2- 172.21.15.94:0/2161242919 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f819c072700 0x7f819c074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:18.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.033+0000 7f81b9600640 1 -- 172.21.15.94:0/2161242919 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5029+0+0 (secure 0 0 0) 0x7f81b012cde0 con 0x7f81bc073ff0 2024-09-10T23:37:18.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.033+0000 7f81c1200640 1 --2- 172.21.15.94:0/2161242919 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f819c072700 0x7f819c074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:18.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.033+0000 7f81c1200640 1 --2- 172.21.15.94:0/2161242919 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f819c072700 0x7f819c074bc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f81a80058d0 tx=0x7f81a8005860 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:18.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.037+0000 7f81b9600640 1 -- 172.21.15.94:0/2161242919 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f81b00bf070 con 0x7f81bc073ff0 2024-09-10T23:37:18.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.261+0000 7f81c4600640 1 -- 172.21.15.94:0/2161242919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7f81bc13bd70 con 0x7f81bc073ff0 2024-09-10T23:37:18.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.497+0000 7f81b9600640 1 -- 172.21.15.94:0/2161242919 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v261) ==== 146+0+0 (secure 0 0 0) 0x7f81b00164b0 con 0x7f81bc073ff0 2024-09-10T23:37:18.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.497+0000 7f81c4600640 1 -- 172.21.15.94:0/2161242919 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f819c072700 msgr2=0x7f819c074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:18.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.497+0000 7f81c4600640 1 --2- 172.21.15.94:0/2161242919 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f819c072700 0x7f819c074bc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f81a80058d0 tx=0x7f81a8005860 comp rx=0 tx=0).stop 2024-09-10T23:37:18.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.497+0000 7f81c4600640 1 -- 172.21.15.94:0/2161242919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81bc073ff0 msgr2=0x7f81bc13b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:18.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.497+0000 7f81c4600640 1 --2- 172.21.15.94:0/2161242919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81bc073ff0 0x7f81bc13b000 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f81b00b0040 tx=0x7f81b0004040 comp rx=0 tx=0).stop 2024-09-10T23:37:18.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.497+0000 7f81c4600640 1 -- 172.21.15.94:0/2161242919 shutdown_connections 2024-09-10T23:37:18.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.497+0000 7f81c4600640 1 --2- 172.21.15.94:0/2161242919 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81bc0d02e0 0x7f81bc13b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.497+0000 7f81c4600640 1 --2- 172.21.15.94:0/2161242919 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f819c072700 0x7f819c074bc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.497+0000 7f81c4600640 1 --2- 172.21.15.94:0/2161242919 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81bc15ecb0 0x7f81bc06f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.497+0000 7f81c4600640 1 --2- 172.21.15.94:0/2161242919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81bc073ff0 0x7f81bc13b000 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.497+0000 7f81c4600640 1 -- 172.21.15.94:0/2161242919 >> 172.21.15.94:0/2161242919 conn(0x7f81bc06cdf0 msgr2=0x7f81bc150d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:18.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.497+0000 7f81c4600640 1 -- 172.21.15.94:0/2161242919 shutdown_connections 2024-09-10T23:37:18.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.497+0000 7f81c4600640 1 -- 172.21.15.94:0/2161242919 wait complete. 2024-09-10T23:37:18.520 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2024-09-10T23:37:18.520 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-10T23:37:18.521 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T23:37:18.521 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:37:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b19600640 1 -- 172.21.15.94:0/1004839504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b140d02e0 msgr2=0x7f8b1415e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b19600640 1 --2- 172.21.15.94:0/1004839504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b140d02e0 0x7f8b1415e770 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8b140d0b80 tx=0x7f8afc01c890 comp rx=0 tx=0).stop 2024-09-10T23:37:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b19600640 1 -- 172.21.15.94:0/1004839504 shutdown_connections 2024-09-10T23:37:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b19600640 1 --2- 172.21.15.94:0/1004839504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b140d02e0 0x7f8b1415e770 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b19600640 1 --2- 172.21.15.94:0/1004839504 >> v2:172.21.15.181:3300/0 conn(0x7f8b14073ff0 0x7f8b140cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b19600640 1 -- 172.21.15.94:0/1004839504 >> 172.21.15.94:0/1004839504 conn(0x7f8b1406cdf0 msgr2=0x7f8b1406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:18.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b19600640 1 -- 172.21.15.94:0/1004839504 shutdown_connections 2024-09-10T23:37:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b19600640 1 -- 172.21.15.94:0/1004839504 wait complete. 2024-09-10T23:37:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b19600640 1 Processor -- start 2024-09-10T23:37:18.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b19600640 1 -- start start 2024-09-10T23:37:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b19600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b14073ff0 0x7f8b141362d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b19600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b140d02e0 0x7f8b14163070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b19600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b1415ecb0 0x7f8b141635b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b19600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8b141679c0 con 0x7f8b1415ecb0 2024-09-10T23:37:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b19600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8b14167b40 con 0x7f8b14073ff0 2024-09-10T23:37:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b19600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8b14167cc0 con 0x7f8b140d02e0 2024-09-10T23:37:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b13400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b1415ecb0 0x7f8b141635b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b13400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b1415ecb0 0x7f8b141635b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59040/0 (socket says 172.21.15.94:59040) 2024-09-10T23:37:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b13400640 1 -- 172.21.15.94:0/1282446759 learned_addr learned my addr 172.21.15.94:0/1282446759 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b12a00640 1 --2- 172.21.15.94:0/1282446759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b14073ff0 0x7f8b141362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b0a000640 1 --2- 172.21.15.94:0/1282446759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b140d02e0 0x7f8b14163070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b13400640 1 -- 172.21.15.94:0/1282446759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b140d02e0 msgr2=0x7f8b14163070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b13400640 1 --2- 172.21.15.94:0/1282446759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b140d02e0 0x7f8b14163070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b13400640 1 -- 172.21.15.94:0/1282446759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b14073ff0 msgr2=0x7f8b141362d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:18.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b13400640 1 --2- 172.21.15.94:0/1282446759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b14073ff0 0x7f8b141362d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b13400640 1 -- 172.21.15.94:0/1282446759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b14138b80 con 0x7f8b1415ecb0 2024-09-10T23:37:18.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.661+0000 7f8b13400640 1 --2- 172.21.15.94:0/1282446759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b1415ecb0 0x7f8b141635b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00b9a0 tx=0x7f8b0c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:18.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.665+0000 7f8b0b400640 1 -- 172.21.15.94:0/1282446759 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c019070 con 0x7f8b1415ecb0 2024-09-10T23:37:18.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.665+0000 7f8b0b400640 1 -- 172.21.15.94:0/1282446759 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b0c00cc30 con 0x7f8b1415ecb0 2024-09-10T23:37:18.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.665+0000 7f8b19600640 1 -- 172.21.15.94:0/1282446759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b14138e60 con 0x7f8b1415ecb0 2024-09-10T23:37:18.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.665+0000 7f8b0b400640 1 -- 172.21.15.94:0/1282446759 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0146a0 con 0x7f8b1415ecb0 2024-09-10T23:37:18.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.665+0000 7f8b19600640 1 -- 172.21.15.94:0/1282446759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b14139420 con 0x7f8b1415ecb0 2024-09-10T23:37:18.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.665+0000 7f8b19600640 1 -- 172.21.15.94:0/1282446759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b140cfda0 con 0x7f8b1415ecb0 2024-09-10T23:37:18.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.665+0000 7f8b0b400640 1 -- 172.21.15.94:0/1282446759 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f8b0c01e440 con 0x7f8b1415ecb0 2024-09-10T23:37:18.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.665+0000 7f8b0b400640 1 --2- 172.21.15.94:0/1282446759 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8af0072700 0x7f8af0074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:18.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.665+0000 7f8b0b400640 1 -- 172.21.15.94:0/1282446759 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5029+0+0 (secure 0 0 0) 0x7f8b0c093280 con 0x7f8b1415ecb0 2024-09-10T23:37:18.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.665+0000 7f8b12a00640 1 --2- 172.21.15.94:0/1282446759 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8af0072700 0x7f8af0074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:18.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.665+0000 7f8b12a00640 1 --2- 172.21.15.94:0/1282446759 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8af0072700 0x7f8af0074bc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8b04005fd0 tx=0x7f8b04005db0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:18.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.669+0000 7f8b0b400640 1 -- 172.21.15.94:0/1282446759 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8b0c00ce20 con 0x7f8b1415ecb0 2024-09-10T23:37:18.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.837+0000 7f8b19600640 1 -- 172.21.15.94:0/1282446759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b140cf3a0 con 0x7f8b1415ecb0 2024-09-10T23:37:18.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.837+0000 7f8b0b400640 1 -- 172.21.15.94:0/1282446759 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+13151 (secure 0 0 0) 0x7f8b0c05e8b0 con 0x7f8b1415ecb0 2024-09-10T23:37:18.841 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:18.841 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":91,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:37:17.491604+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6834/1214870454":"2024-09-11T23:37:14.482841+0000","172.21.15.181:6800/1214870454":"2024-09-11T23:37:14.482841+0000","172.21.15.94:6835/736952701":"2024-09-11T23:37:07.444153+0000","172.21.15.94:6833/736952701":"2024-09-11T23:37:07.444153+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:37:18.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.841+0000 7f8b19600640 1 -- 172.21.15.94:0/1282446759 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8af0072700 msgr2=0x7f8af0074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:18.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.841+0000 7f8b19600640 1 --2- 172.21.15.94:0/1282446759 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8af0072700 0x7f8af0074bc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8b04005fd0 tx=0x7f8b04005db0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.841+0000 7f8b19600640 1 -- 172.21.15.94:0/1282446759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b1415ecb0 msgr2=0x7f8b141635b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:18.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.841+0000 7f8b19600640 1 --2- 172.21.15.94:0/1282446759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b1415ecb0 0x7f8b141635b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00b9a0 tx=0x7f8b0c00be60 comp rx=0 tx=0).stop 2024-09-10T23:37:18.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.841+0000 7f8b19600640 1 -- 172.21.15.94:0/1282446759 shutdown_connections 2024-09-10T23:37:18.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.841+0000 7f8b19600640 1 --2- 172.21.15.94:0/1282446759 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8af0072700 0x7f8af0074bc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.841+0000 7f8b19600640 1 --2- 172.21.15.94:0/1282446759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b1415ecb0 0x7f8b141635b0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.841+0000 7f8b19600640 1 --2- 172.21.15.94:0/1282446759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b140d02e0 0x7f8b14163070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.841+0000 7f8b19600640 1 --2- 172.21.15.94:0/1282446759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b14073ff0 0x7f8b141362d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:18.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.841+0000 7f8b19600640 1 -- 172.21.15.94:0/1282446759 >> 172.21.15.94:0/1282446759 conn(0x7f8b1406cdf0 msgr2=0x7f8b14072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:18.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.841+0000 7f8b19600640 1 -- 172.21.15.94:0/1282446759 shutdown_connections 2024-09-10T23:37:18.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:18.841+0000 7f8b19600640 1 -- 172.21.15.94:0/1282446759 wait complete. 2024-09-10T23:37:18.863 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T23:32:31.372521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T23:37:18.863 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T23:37:19.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.005+0000 7fa7a2600640 1 -- 172.21.15.94:0/1427703047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa784004010 msgr2=0x7fa784004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.005+0000 7fa7a2600640 1 --2- 172.21.15.94:0/1427703047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa784004010 0x7fa784004810 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa788001690 tx=0x7fa78801c770 comp rx=0 tx=0).stop 2024-09-10T23:37:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.005+0000 7fa7a1c00640 1 -- 172.21.15.94:0/1427703047 reap_dead start 2024-09-10T23:37:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.005+0000 7fa7a2600640 1 -- 172.21.15.94:0/1427703047 shutdown_connections 2024-09-10T23:37:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.005+0000 7fa7a2600640 1 -- 172.21.15.94:0/1427703047 >> 172.21.15.94:0/1427703047 conn(0x7fa79c06e610 msgr2=0x7fa79c0b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.005+0000 7fa7a2600640 1 -- 172.21.15.94:0/1427703047 shutdown_connections 2024-09-10T23:37:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.005+0000 7fa7a2600640 1 -- 172.21.15.94:0/1427703047 wait complete. 2024-09-10T23:37:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.005+0000 7fa7a2600640 1 Processor -- start 2024-09-10T23:37:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.005+0000 7fa7a2600640 1 -- start start 2024-09-10T23:37:19.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa7a2600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa784004010 0x7fa79c0c5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:19.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa7a2600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa79c1f4f60 0x7fa79c1fb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:19.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa7a2600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa79c1f53e0 0x7fa79c1fb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:19.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa7a2600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c0cf7b0 con 0x7fa79c1f4f60 2024-09-10T23:37:19.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa7a2600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c0cf630 con 0x7fa79c1f53e0 2024-09-10T23:37:19.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa7a2600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa79c0cf930 con 0x7fa784004010 2024-09-10T23:37:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa79be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa79c1f4f60 0x7fa79c1fb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa79be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa79c1f4f60 0x7fa79c1fb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59056/0 (socket says 172.21.15.94:59056) 2024-09-10T23:37:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa79be00640 1 -- 172.21.15.94:0/2156925019 learned_addr learned my addr 172.21.15.94:0/2156925019 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa7a1200640 1 --2- 172.21.15.94:0/2156925019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa784004010 0x7fa79c0c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa7a1c00640 1 --2- 172.21.15.94:0/2156925019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa79c1f53e0 0x7fa79c1fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa79be00640 1 -- 172.21.15.94:0/2156925019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa784004010 msgr2=0x7fa79c0c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa79be00640 1 --2- 172.21.15.94:0/2156925019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa784004010 0x7fa79c0c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa79be00640 1 -- 172.21.15.94:0/2156925019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa79c1f53e0 msgr2=0x7fa79c1fb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa79be00640 1 --2- 172.21.15.94:0/2156925019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa79c1f53e0 0x7fa79c1fb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa79be00640 1 -- 172.21.15.94:0/2156925019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79c155ad0 con 0x7fa79c1f4f60 2024-09-10T23:37:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa79be00640 1 --2- 172.21.15.94:0/2156925019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa79c1f4f60 0x7fa79c1fb1c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fa78801c5a0 tx=0x7fa788001330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa798c00640 1 -- 172.21.15.94:0/2156925019 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7880036c0 con 0x7fa79c1f4f60 2024-09-10T23:37:19.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa798c00640 1 -- 172.21.15.94:0/2156925019 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa788003970 con 0x7fa79c1f4f60 2024-09-10T23:37:19.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa798c00640 1 -- 172.21.15.94:0/2156925019 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788003dc0 con 0x7fa79c1f4f60 2024-09-10T23:37:19.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa7a2600640 1 -- 172.21.15.94:0/2156925019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79c0c5ea0 con 0x7fa79c1f4f60 2024-09-10T23:37:19.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa7a2600640 1 -- 172.21.15.94:0/2156925019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa79c0c6420 con 0x7fa79c1f4f60 2024-09-10T23:37:19.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.009+0000 7fa7a2600640 1 -- 172.21.15.94:0/2156925019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa79c0cd2a0 con 0x7fa79c1f4f60 2024-09-10T23:37:19.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.013+0000 7fa798c00640 1 -- 172.21.15.94:0/2156925019 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fa7880b6020 con 0x7fa79c1f4f60 2024-09-10T23:37:19.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.013+0000 7fa798c00640 1 --2- 172.21.15.94:0/2156925019 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa77c072730 0x7fa77c074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:19.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.013+0000 7fa798c00640 1 -- 172.21.15.94:0/2156925019 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5029+0+0 (secure 0 0 0) 0x7fa78812c5f0 con 0x7fa79c1f4f60 2024-09-10T23:37:19.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.013+0000 7fa7a1200640 1 --2- 172.21.15.94:0/2156925019 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa77c072730 0x7fa77c074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:19.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.013+0000 7fa7a1200640 1 --2- 172.21.15.94:0/2156925019 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa77c072730 0x7fa77c074bf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa790001250 tx=0x7fa790006040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:19.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.017+0000 7fa798c00640 1 -- 172.21.15.94:0/2156925019 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa7880bf030 con 0x7fa79c1f4f60 2024-09-10T23:37:19.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.181+0000 7fa7a2600640 1 -- 172.21.15.94:0/2156925019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa79c0b68e0 con 0x7fa79c1f4f60 2024-09-10T23:37:19.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.181+0000 7fa798c00640 1 -- 172.21.15.94:0/2156925019 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fa79c155ad0 con 0x7fa79c1f4f60 2024-09-10T23:37:19.183 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-09-10T23:37:19.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.181+0000 7fa7a2600640 1 -- 172.21.15.94:0/2156925019 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa77c072730 msgr2=0x7fa77c074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:19.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.181+0000 7fa7a2600640 1 --2- 172.21.15.94:0/2156925019 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa77c072730 0x7fa77c074bf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa790001250 tx=0x7fa790006040 comp rx=0 tx=0).stop 2024-09-10T23:37:19.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.181+0000 7fa7a2600640 1 -- 172.21.15.94:0/2156925019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa79c1f4f60 msgr2=0x7fa79c1fb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:19.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.181+0000 7fa7a2600640 1 --2- 172.21.15.94:0/2156925019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa79c1f4f60 0x7fa79c1fb1c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fa78801c5a0 tx=0x7fa788001330 comp rx=0 tx=0).stop 2024-09-10T23:37:19.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.181+0000 7fa7a2600640 1 -- 172.21.15.94:0/2156925019 shutdown_connections 2024-09-10T23:37:19.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.181+0000 7fa7a2600640 1 --2- 172.21.15.94:0/2156925019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa79c1f53e0 0x7fa79c1fb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:19.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.181+0000 7fa7a2600640 1 --2- 172.21.15.94:0/2156925019 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa77c072730 0x7fa77c074bf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:19.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.181+0000 7fa7a2600640 1 --2- 172.21.15.94:0/2156925019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa79c1f4f60 0x7fa79c1fb1c0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:19.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.181+0000 7fa7a2600640 1 --2- 172.21.15.94:0/2156925019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa784004010 0x7fa79c0c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:19.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.181+0000 7fa7a2600640 1 -- 172.21.15.94:0/2156925019 >> 172.21.15.94:0/2156925019 conn(0x7fa79c06e610 msgr2=0x7fa79c06d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:19.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.181+0000 7fa7a2600640 1 -- 172.21.15.94:0/2156925019 shutdown_connections 2024-09-10T23:37:19.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.181+0000 7fa7a2600640 1 -- 172.21.15.94:0/2156925019 wait complete. 2024-09-10T23:37:19.205 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-10T23:37:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.341+0000 7fd6f6c00640 1 -- 172.21.15.94:0/3386299875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f00b1a20 msgr2=0x7fd6f00b1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.341+0000 7fd6f6c00640 1 --2- 172.21.15.94:0/3386299875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f00b1a20 0x7fd6f00b1e90 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00b7b70 tx=0x7fd6e801c800 comp rx=0 tx=0).stop 2024-09-10T23:37:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.341+0000 7fd6f6c00640 1 -- 172.21.15.94:0/3386299875 shutdown_connections 2024-09-10T23:37:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.341+0000 7fd6f6c00640 1 --2- 172.21.15.94:0/3386299875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f00b1a20 0x7fd6f00b1e90 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.341+0000 7fd6f6c00640 1 --2- 172.21.15.94:0/3386299875 >> v2:172.21.15.181:3300/0 conn(0x7fd6f00d00d0 0x7fd6f00d0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.341+0000 7fd6f6c00640 1 -- 172.21.15.94:0/3386299875 >> 172.21.15.94:0/3386299875 conn(0x7fd6f00d1af0 msgr2=0x7fd6f00d1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.341+0000 7fd6f6c00640 1 -- 172.21.15.94:0/3386299875 shutdown_connections 2024-09-10T23:37:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.341+0000 7fd6f6c00640 1 -- 172.21.15.94:0/3386299875 wait complete. 2024-09-10T23:37:19.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f6c00640 1 Processor -- start 2024-09-10T23:37:19.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f6c00640 1 -- start start 2024-09-10T23:37:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f6c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f00b1640 0x7fd6f00cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f6c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f00b1a20 0x7fd6f0151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f6c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f00d00d0 0x7fd6f01523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f6c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01607c0 con 0x7fd6f00b1640 2024-09-10T23:37:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f6c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f0160640 con 0x7fd6f00b1a20 2024-09-10T23:37:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f6c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f0160940 con 0x7fd6f00d00d0 2024-09-10T23:37:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f5800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f00b1640 0x7fd6f00cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f5800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f00b1640 0x7fd6f00cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59080/0 (socket says 172.21.15.94:59080) 2024-09-10T23:37:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f5800640 1 -- 172.21.15.94:0/691178366 learned_addr learned my addr 172.21.15.94:0/691178366 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f6200640 1 --2- 172.21.15.94:0/691178366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f00d00d0 0x7fd6f01523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f4e00640 1 --2- 172.21.15.94:0/691178366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f00b1a20 0x7fd6f0151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f5800640 1 -- 172.21.15.94:0/691178366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f00d00d0 msgr2=0x7fd6f01523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:19.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f5800640 1 --2- 172.21.15.94:0/691178366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f00d00d0 0x7fd6f01523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:19.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f5800640 1 -- 172.21.15.94:0/691178366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f00b1a20 msgr2=0x7fd6f0151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:19.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f5800640 1 --2- 172.21.15.94:0/691178366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f00b1a20 0x7fd6f0151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:19.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f5800640 1 -- 172.21.15.94:0/691178366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f00ce1b0 con 0x7fd6f00b1640 2024-09-10T23:37:19.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.345+0000 7fd6f5800640 1 --2- 172.21.15.94:0/691178366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f00b1640 0x7fd6f00cd6b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400c7c0 tx=0x7fd6e400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:19.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.349+0000 7fd6ed600640 1 -- 172.21.15.94:0/691178366 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4016020 con 0x7fd6f00b1640 2024-09-10T23:37:19.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.349+0000 7fd6ed600640 1 -- 172.21.15.94:0/691178366 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6e400ab40 con 0x7fd6f00b1640 2024-09-10T23:37:19.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.349+0000 7fd6ed600640 1 -- 172.21.15.94:0/691178366 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4012420 con 0x7fd6f00b1640 2024-09-10T23:37:19.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.349+0000 7fd6f6c00640 1 -- 172.21.15.94:0/691178366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f00ce470 con 0x7fd6f00b1640 2024-09-10T23:37:19.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.349+0000 7fd6f6c00640 1 -- 172.21.15.94:0/691178366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f006ea00 con 0x7fd6f00b1640 2024-09-10T23:37:19.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.349+0000 7fd6f6c00640 1 -- 172.21.15.94:0/691178366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f00b1e90 con 0x7fd6f00b1640 2024-09-10T23:37:19.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.349+0000 7fd6ed600640 1 -- 172.21.15.94:0/691178366 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fd6e4006b60 con 0x7fd6f00b1640 2024-09-10T23:37:19.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.349+0000 7fd6ed600640 1 --2- 172.21.15.94:0/691178366 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd6d0072700 0x7fd6d0074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:19.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.349+0000 7fd6ed600640 1 -- 172.21.15.94:0/691178366 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5029+0+0 (secure 0 0 0) 0x7fd6e4093a60 con 0x7fd6f00b1640 2024-09-10T23:37:19.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.349+0000 7fd6f4e00640 1 --2- 172.21.15.94:0/691178366 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd6d0072700 0x7fd6d0074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:19.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.353+0000 7fd6f4e00640 1 --2- 172.21.15.94:0/691178366 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd6d0072700 0x7fd6d0074bc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd6f01fa4e0 tx=0x7fd6dc00a250 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:19.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.353+0000 7fd6ed600640 1 -- 172.21.15.94:0/691178366 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd6e4020020 con 0x7fd6f00b1640 2024-09-10T23:37:19.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:19.577+0000 7fd6f6c00640 1 -- 172.21.15.94:0/691178366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7fd6f00cd2d0 con 0x7fd6f00b1640 2024-09-10T23:37:20.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.505+0000 7fd6ed600640 1 -- 172.21.15.94:0/691178366 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v263) ==== 152+0+0 (secure 0 0 0) 0x7fd6e405ece0 con 0x7fd6f00b1640 2024-09-10T23:37:20.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.509+0000 7fd6f6c00640 1 -- 172.21.15.94:0/691178366 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd6d0072700 msgr2=0x7fd6d0074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:20.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.509+0000 7fd6f6c00640 1 --2- 172.21.15.94:0/691178366 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd6d0072700 0x7fd6d0074bc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd6f01fa4e0 tx=0x7fd6dc00a250 comp rx=0 tx=0).stop 2024-09-10T23:37:20.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.509+0000 7fd6f6c00640 1 -- 172.21.15.94:0/691178366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f00b1640 msgr2=0x7fd6f00cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:20.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.509+0000 7fd6f6c00640 1 --2- 172.21.15.94:0/691178366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f00b1640 0x7fd6f00cd6b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400c7c0 tx=0x7fd6e400cc80 comp rx=0 tx=0).stop 2024-09-10T23:37:20.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.509+0000 7fd6f6c00640 1 -- 172.21.15.94:0/691178366 shutdown_connections 2024-09-10T23:37:20.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.509+0000 7fd6f6c00640 1 --2- 172.21.15.94:0/691178366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6f00b1a20 0x7fd6f0151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:20.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.509+0000 7fd6f6c00640 1 --2- 172.21.15.94:0/691178366 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd6d0072700 0x7fd6d0074bc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:20.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.509+0000 7fd6f6c00640 1 --2- 172.21.15.94:0/691178366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6f00d00d0 0x7fd6f01523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:20.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.509+0000 7fd6f6c00640 1 --2- 172.21.15.94:0/691178366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f00b1640 0x7fd6f00cd6b0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:20.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.509+0000 7fd6f6c00640 1 -- 172.21.15.94:0/691178366 >> 172.21.15.94:0/691178366 conn(0x7fd6f00d1af0 msgr2=0x7fd6f015d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:20.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.509+0000 7fd6f6c00640 1 -- 172.21.15.94:0/691178366 shutdown_connections 2024-09-10T23:37:20.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.509+0000 7fd6f6c00640 1 -- 172.21.15.94:0/691178366 wait complete. 2024-09-10T23:37:20.533 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T23:37:20.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.677+0000 7fe598800640 1 -- 172.21.15.94:0/1076287047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe590152c10 msgr2=0x7fe5900b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:20.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.677+0000 7fe598800640 1 --2- 172.21.15.94:0/1076287047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe590152c10 0x7fe5900b29d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe584009900 tx=0x7fe58401c8e0 comp rx=0 tx=0).stop 2024-09-10T23:37:20.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.677+0000 7fe598800640 1 -- 172.21.15.94:0/1076287047 shutdown_connections 2024-09-10T23:37:20.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.677+0000 7fe598800640 1 --2- 172.21.15.94:0/1076287047 >> v2:172.21.15.181:3300/0 conn(0x7fe59015cbc0 0x7fe5900cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:20.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.677+0000 7fe598800640 1 --2- 172.21.15.94:0/1076287047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe590152c10 0x7fe5900b29d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:20.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.677+0000 7fe598800640 1 -- 172.21.15.94:0/1076287047 >> 172.21.15.94:0/1076287047 conn(0x7fe59006cdf0 msgr2=0x7fe59006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:20.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.677+0000 7fe598800640 1 -- 172.21.15.94:0/1076287047 shutdown_connections 2024-09-10T23:37:20.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.677+0000 7fe598800640 1 -- 172.21.15.94:0/1076287047 wait complete. 2024-09-10T23:37:20.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.677+0000 7fe598800640 1 Processor -- start 2024-09-10T23:37:20.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.677+0000 7fe598800640 1 -- start start 2024-09-10T23:37:20.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.677+0000 7fe598800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5900cfd50 0x7fe590140e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:20.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.677+0000 7fe598800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe590152c10 0x7fe5901f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:20.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.677+0000 7fe598800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe59015cbc0 0x7fe59014e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:20.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.677+0000 7fe598800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901693b0 con 0x7fe590152c10 2024-09-10T23:37:20.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.677+0000 7fe598800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe590169230 con 0x7fe5900cfd50 2024-09-10T23:37:20.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.677+0000 7fe598800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe590169530 con 0x7fe59015cbc0 2024-09-10T23:37:20.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe595400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe590152c10 0x7fe5901f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:20.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe595400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe590152c10 0x7fe5901f7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59086/0 (socket says 172.21.15.94:59086) 2024-09-10T23:37:20.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe595400640 1 -- 172.21.15.94:0/1676046345 learned_addr learned my addr 172.21.15.94:0/1676046345 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:20.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe595e00640 1 --2- 172.21.15.94:0/1676046345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5900cfd50 0x7fe590140e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:20.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe596800640 1 --2- 172.21.15.94:0/1676046345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe59015cbc0 0x7fe59014e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:20.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe595400640 1 -- 172.21.15.94:0/1676046345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe59015cbc0 msgr2=0x7fe59014e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:20.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe595400640 1 --2- 172.21.15.94:0/1676046345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe59015cbc0 0x7fe59014e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:20.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe595400640 1 -- 172.21.15.94:0/1676046345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5900cfd50 msgr2=0x7fe590140e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:20.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe595400640 1 --2- 172.21.15.94:0/1676046345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5900cfd50 0x7fe590140e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:20.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe595400640 1 -- 172.21.15.94:0/1676046345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5901f8df0 con 0x7fe590152c10 2024-09-10T23:37:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe595400640 1 --2- 172.21.15.94:0/1676046345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe590152c10 0x7fe5901f7eb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fe58000e7a0 tx=0x7fe58000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe58a000640 1 -- 172.21.15.94:0/1676046345 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58000ccc0 con 0x7fe590152c10 2024-09-10T23:37:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe58a000640 1 -- 172.21.15.94:0/1676046345 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe580010430 con 0x7fe590152c10 2024-09-10T23:37:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe58a000640 1 -- 172.21.15.94:0/1676046345 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580010860 con 0x7fe590152c10 2024-09-10T23:37:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe598800640 1 -- 172.21.15.94:0/1676046345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5901f8830 con 0x7fe590152c10 2024-09-10T23:37:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe598800640 1 -- 172.21.15.94:0/1676046345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5901541c0 con 0x7fe590152c10 2024-09-10T23:37:20.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe598800640 1 -- 172.21.15.94:0/1676046345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5900b29d0 con 0x7fe590152c10 2024-09-10T23:37:20.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.681+0000 7fe58a000640 1 -- 172.21.15.94:0/1676046345 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fe580010a00 con 0x7fe590152c10 2024-09-10T23:37:20.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.685+0000 7fe58a000640 1 --2- 172.21.15.94:0/1676046345 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe574072700 0x7fe574074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:20.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.685+0000 7fe58a000640 1 -- 172.21.15.94:0/1676046345 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5029+0+0 (secure 0 0 0) 0x7fe580093ba0 con 0x7fe590152c10 2024-09-10T23:37:20.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.685+0000 7fe595e00640 1 --2- 172.21.15.94:0/1676046345 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe574072700 0x7fe574074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:20.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.685+0000 7fe595e00640 1 --2- 172.21.15.94:0/1676046345 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe574072700 0x7fe574074bc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe5840af000 tx=0x7fe58401b120 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:20.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.685+0000 7fe58a000640 1 -- 172.21.15.94:0/1676046345 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe58005eca0 con 0x7fe590152c10 2024-09-10T23:37:20.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.853+0000 7fe598800640 1 -- 172.21.15.94:0/1676046345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fe5901664c0 con 0x7fe590152c10 2024-09-10T23:37:20.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.857+0000 7fe58a000640 1 -- 172.21.15.94:0/1676046345 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+1998 (secure 0 0 0) 0x7fe5800636d0 con 0x7fe590152c10 2024-09-10T23:37:20.858 INFO:teuthology.orchestra.run.smithi094.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T23:37:20.858 INFO:teuthology.orchestra.run.smithi094.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T23:37:20.858 INFO:teuthology.orchestra.run.smithi094.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T23:37:20.859 INFO:teuthology.orchestra.run.smithi094.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T23:37:20.859 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3944.074506 2024-09-10T23:37:20.859 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3865.464527 2024-09-10T23:37:20.859 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3898.861024 2024-09-10T23:37:20.859 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4010.048529 2024-09-10T23:37:20.859 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4650.562436 2024-09-10T23:37:20.859 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4724.018776 2024-09-10T23:37:20.859 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5255.753663 2024-09-10T23:37:20.859 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4718.009443 2024-09-10T23:37:20.859 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds 20/20 2024-09-10T23:37:20.859 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T23:37:20.859 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_ms 2/2 2024-09-10T23:37:20.859 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_frag true 2024-09-10T23:37:20.859 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_scatterstat true 2024-09-10T23:37:20.860 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T23:37:20.860 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_verify_scatter true 2024-09-10T23:37:20.860 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T23:37:20.860 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T23:37:20.860 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T23:37:20.860 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced client_mount_timeout 600 2024-09-10T23:37:20.860 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_client 20/20 2024-09-10T23:37:20.860 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_ms 1/1 2024-09-10T23:37:20.860 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T23:37:20.860 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T23:37:20.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.857+0000 7fe598800640 1 -- 172.21.15.94:0/1676046345 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe574072700 msgr2=0x7fe574074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:20.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.857+0000 7fe598800640 1 --2- 172.21.15.94:0/1676046345 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe574072700 0x7fe574074bc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe5840af000 tx=0x7fe58401b120 comp rx=0 tx=0).stop 2024-09-10T23:37:20.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.857+0000 7fe598800640 1 -- 172.21.15.94:0/1676046345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe590152c10 msgr2=0x7fe5901f7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:20.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.857+0000 7fe598800640 1 --2- 172.21.15.94:0/1676046345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe590152c10 0x7fe5901f7eb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fe58000e7a0 tx=0x7fe58000ec60 comp rx=0 tx=0).stop 2024-09-10T23:37:20.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.857+0000 7fe598800640 1 -- 172.21.15.94:0/1676046345 shutdown_connections 2024-09-10T23:37:20.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.857+0000 7fe598800640 1 --2- 172.21.15.94:0/1676046345 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe574072700 0x7fe574074bc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:20.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.857+0000 7fe598800640 1 --2- 172.21.15.94:0/1676046345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe59015cbc0 0x7fe59014e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:20.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.857+0000 7fe598800640 1 --2- 172.21.15.94:0/1676046345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe590152c10 0x7fe5901f7eb0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:20.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.857+0000 7fe598800640 1 --2- 172.21.15.94:0/1676046345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5900cfd50 0x7fe590140e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:20.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.857+0000 7fe598800640 1 -- 172.21.15.94:0/1676046345 >> 172.21.15.94:0/1676046345 conn(0x7fe59006cdf0 msgr2=0x7fe59015d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:20.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.857+0000 7fe598800640 1 -- 172.21.15.94:0/1676046345 shutdown_connections 2024-09-10T23:37:20.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:20.857+0000 7fe598800640 1 -- 172.21.15.94:0/1676046345 wait complete. 2024-09-10T23:37:20.880 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:37:21.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092b200640 1 -- 172.21.15.94:0/1654463020 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09241615d0 msgr2=0x7f09240d2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:21.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092b200640 1 --2- 172.21.15.94:0/1654463020 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09241615d0 0x7f09240d2490 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0914009900 tx=0x7f091401f0d0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092b200640 1 -- 172.21.15.94:0/1654463020 shutdown_connections 2024-09-10T23:37:21.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092b200640 1 --2- 172.21.15.94:0/1654463020 >> v2:172.21.15.181:3300/0 conn(0x7f09240d4820 0x7f09240b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092b200640 1 --2- 172.21.15.94:0/1654463020 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09241615d0 0x7f09240d2490 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092b200640 1 -- 172.21.15.94:0/1654463020 >> 172.21.15.94:0/1654463020 conn(0x7f09240d3d00 msgr2=0x7f09240d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:21.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092b200640 1 -- 172.21.15.94:0/1654463020 shutdown_connections 2024-09-10T23:37:21.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092b200640 1 -- 172.21.15.94:0/1654463020 wait complete. 2024-09-10T23:37:21.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092b200640 1 Processor -- start 2024-09-10T23:37:21.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092b200640 1 -- start start 2024-09-10T23:37:21.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092b200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09240d4820 0x7f09240cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:21.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09241611f0 0x7f0924152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:21.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09241615d0 0x7f09241f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:21.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f09240b7970 con 0x7f09241611f0 2024-09-10T23:37:21.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092b200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f09240b77f0 con 0x7f09241615d0 2024-09-10T23:37:21.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092b200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f09240b7af0 con 0x7f09240d4820 2024-09-10T23:37:21.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f0929400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09241611f0 0x7f0924152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:21.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f0929400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09241611f0 0x7f0924152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59090/0 (socket says 172.21.15.94:59090) 2024-09-10T23:37:21.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f0929400640 1 -- 172.21.15.94:0/1140877274 learned_addr learned my addr 172.21.15.94:0/1140877274 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:21.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f0929e00640 1 --2- 172.21.15.94:0/1140877274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09240d4820 0x7f09240cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:21.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f092a800640 1 --2- 172.21.15.94:0/1140877274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09241615d0 0x7f09241f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:21.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f0929400640 1 -- 172.21.15.94:0/1140877274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09240d4820 msgr2=0x7f09240cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:21.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f0929400640 1 --2- 172.21.15.94:0/1140877274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09240d4820 0x7f09240cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f0929400640 1 -- 172.21.15.94:0/1140877274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09241615d0 msgr2=0x7f09241f4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:21.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f0929400640 1 --2- 172.21.15.94:0/1140877274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09241615d0 0x7f09241f4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.025+0000 7f0929400640 1 -- 172.21.15.94:0/1140877274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09241f5390 con 0x7f09241611f0 2024-09-10T23:37:21.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.029+0000 7f0929400640 1 --2- 172.21.15.94:0/1140877274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09241611f0 0x7f0924152270 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0914009850 tx=0x7f0914005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:21.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.029+0000 7f091e000640 1 -- 172.21.15.94:0/1140877274 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09140be070 con 0x7f09241611f0 2024-09-10T23:37:21.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.029+0000 7f091e000640 1 -- 172.21.15.94:0/1140877274 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09140038f0 con 0x7f09241611f0 2024-09-10T23:37:21.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.029+0000 7f091e000640 1 -- 172.21.15.94:0/1140877274 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09140b5040 con 0x7f09241611f0 2024-09-10T23:37:21.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.029+0000 7f092b200640 1 -- 172.21.15.94:0/1140877274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09240b4480 con 0x7f09241611f0 2024-09-10T23:37:21.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.029+0000 7f092b200640 1 -- 172.21.15.94:0/1140877274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09240cd7c0 con 0x7f09241611f0 2024-09-10T23:37:21.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.029+0000 7f092b200640 1 -- 172.21.15.94:0/1140877274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09240b70a0 con 0x7f09241611f0 2024-09-10T23:37:21.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.029+0000 7f091e000640 1 -- 172.21.15.94:0/1140877274 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f09140b8070 con 0x7f09241611f0 2024-09-10T23:37:21.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.029+0000 7f091e000640 1 --2- 172.21.15.94:0/1140877274 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0908072700 0x7f0908074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:21.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.029+0000 7f091e000640 1 -- 172.21.15.94:0/1140877274 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5029+0+0 (secure 0 0 0) 0x7f091412fd40 con 0x7f09241611f0 2024-09-10T23:37:21.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.033+0000 7f0929e00640 1 --2- 172.21.15.94:0/1140877274 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0908072700 0x7f0908074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:21.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.033+0000 7f0929e00640 1 --2- 172.21.15.94:0/1140877274 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0908072700 0x7f0908074bc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f09240cd020 tx=0x7f0918009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:21.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.037+0000 7f091e000640 1 -- 172.21.15.94:0/1140877274 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f09241f5390 con 0x7f09241611f0 2024-09-10T23:37:21.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.201+0000 7f092b200640 1 -- 172.21.15.94:0/1140877274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0924069780 con 0x7f09241611f0 2024-09-10T23:37:21.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.205+0000 7f091e000640 1 -- 172.21.15.94:0/1140877274 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f0914018c30 con 0x7f09241611f0 2024-09-10T23:37:21.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.205+0000 7f092b200640 1 -- 172.21.15.94:0/1140877274 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0908072700 msgr2=0x7f0908074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:21.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.205+0000 7f092b200640 1 --2- 172.21.15.94:0/1140877274 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0908072700 0x7f0908074bc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f09240cd020 tx=0x7f0918009040 comp rx=0 tx=0).stop 2024-09-10T23:37:21.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.205+0000 7f092b200640 1 -- 172.21.15.94:0/1140877274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09241611f0 msgr2=0x7f0924152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:21.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.205+0000 7f092b200640 1 --2- 172.21.15.94:0/1140877274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09241611f0 0x7f0924152270 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0914009850 tx=0x7f0914005790 comp rx=0 tx=0).stop 2024-09-10T23:37:21.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.205+0000 7f092b200640 1 -- 172.21.15.94:0/1140877274 shutdown_connections 2024-09-10T23:37:21.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.205+0000 7f092b200640 1 --2- 172.21.15.94:0/1140877274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09241615d0 0x7f09241f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.205+0000 7f092b200640 1 --2- 172.21.15.94:0/1140877274 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0908072700 0x7f0908074bc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.205+0000 7f092b200640 1 --2- 172.21.15.94:0/1140877274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09241611f0 0x7f0924152270 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.205+0000 7f092b200640 1 --2- 172.21.15.94:0/1140877274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09240d4820 0x7f09240cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.205+0000 7f092b200640 1 -- 172.21.15.94:0/1140877274 >> 172.21.15.94:0/1140877274 conn(0x7f09240d3d00 msgr2=0x7f09240c9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:21.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.205+0000 7f092b200640 1 -- 172.21.15.94:0/1140877274 shutdown_connections 2024-09-10T23:37:21.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.205+0000 7f092b200640 1 -- 172.21.15.94:0/1140877274 wait complete. 2024-09-10T23:37:21.230 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:37:21.230 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:37:21.328 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:37:21.416 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:37:21.416 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:37:21.416 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:21.424 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:37:21.425 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:37:21.425 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:21.434 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:37:21.434 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:37:21.434 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:21.474 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:37:21.474 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:37:21.475 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:21.485 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:37:21.486 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T23:37:21.486 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:37:21.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.657+0000 7fd6c1e00640 1 -- 172.21.15.94:0/1968724682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6bc0d04b0 msgr2=0x7fd6bc15e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.657+0000 7fd6c1e00640 1 --2- 172.21.15.94:0/1968724682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6bc0d04b0 0x7fd6bc15e770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009900 tx=0x7fd6a401c8b0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.657+0000 7fd6c1e00640 1 -- 172.21.15.94:0/1968724682 shutdown_connections 2024-09-10T23:37:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.657+0000 7fd6c1e00640 1 --2- 172.21.15.94:0/1968724682 >> v2:172.21.15.181:3300/0 conn(0x7fd6bc15ecb0 0x7fd6bc06f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.657+0000 7fd6c1e00640 1 --2- 172.21.15.94:0/1968724682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6bc0d04b0 0x7fd6bc15e770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.657+0000 7fd6c1e00640 1 -- 172.21.15.94:0/1968724682 >> 172.21.15.94:0/1968724682 conn(0x7fd6bc06cdf0 msgr2=0x7fd6bc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6c1e00640 1 -- 172.21.15.94:0/1968724682 shutdown_connections 2024-09-10T23:37:21.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6c1e00640 1 -- 172.21.15.94:0/1968724682 wait complete. 2024-09-10T23:37:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6c1e00640 1 Processor -- start 2024-09-10T23:37:21.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6c1e00640 1 -- start start 2024-09-10T23:37:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6c1e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6bc073ff0 0x7fd6bc1f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6c1e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6bc0d04b0 0x7fd6bc145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:21.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6c1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6bc15ecb0 0x7fd6bc14c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6c1e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc167930 con 0x7fd6bc15ecb0 2024-09-10T23:37:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6c1e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc1677b0 con 0x7fd6bc073ff0 2024-09-10T23:37:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6c1e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd6bc167ab0 con 0x7fd6bc0d04b0 2024-09-10T23:37:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6bb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6bc15ecb0 0x7fd6bc14c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6bb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6bc15ecb0 0x7fd6bc14c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59100/0 (socket says 172.21.15.94:59100) 2024-09-10T23:37:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6bb400640 1 -- 172.21.15.94:0/739305163 learned_addr learned my addr 172.21.15.94:0/739305163 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6baa00640 1 --2- 172.21.15.94:0/739305163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6bc073ff0 0x7fd6bc1f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:21.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6b2000640 1 --2- 172.21.15.94:0/739305163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6bc0d04b0 0x7fd6bc145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:21.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6bb400640 1 -- 172.21.15.94:0/739305163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6bc0d04b0 msgr2=0x7fd6bc145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:21.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6bb400640 1 --2- 172.21.15.94:0/739305163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6bc0d04b0 0x7fd6bc145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6bb400640 1 -- 172.21.15.94:0/739305163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6bc073ff0 msgr2=0x7fd6bc1f7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6bb400640 1 --2- 172.21.15.94:0/739305163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6bc073ff0 0x7fd6bc1f7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6bb400640 1 -- 172.21.15.94:0/739305163 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc1f82a0 con 0x7fd6bc15ecb0 2024-09-10T23:37:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6bb400640 1 --2- 172.21.15.94:0/739305163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6bc15ecb0 0x7fd6bc14c950 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400ea80 tx=0x7fd6b400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.661+0000 7fd6b3e00640 1 -- 172.21.15.94:0/739305163 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4004b30 con 0x7fd6bc15ecb0 2024-09-10T23:37:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.665+0000 7fd6b3e00640 1 -- 172.21.15.94:0/739305163 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6b4004cf0 con 0x7fd6bc15ecb0 2024-09-10T23:37:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.665+0000 7fd6c1e00640 1 -- 172.21.15.94:0/739305163 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6bc146330 con 0x7fd6bc15ecb0 2024-09-10T23:37:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.665+0000 7fd6b3e00640 1 -- 172.21.15.94:0/739305163 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4005280 con 0x7fd6bc15ecb0 2024-09-10T23:37:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.665+0000 7fd6c1e00640 1 -- 172.21.15.94:0/739305163 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6bc14b090 con 0x7fd6bc15ecb0 2024-09-10T23:37:21.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.665+0000 7fd6c1e00640 1 -- 172.21.15.94:0/739305163 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6bc15e770 con 0x7fd6bc15ecb0 2024-09-10T23:37:21.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.665+0000 7fd6b3e00640 1 -- 172.21.15.94:0/739305163 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fd6b4005420 con 0x7fd6bc15ecb0 2024-09-10T23:37:21.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.665+0000 7fd6b3e00640 1 --2- 172.21.15.94:0/739305163 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd698072730 0x7fd698074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:21.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.665+0000 7fd6b3e00640 1 -- 172.21.15.94:0/739305163 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5029+0+0 (secure 0 0 0) 0x7fd6b4093ef0 con 0x7fd6bc15ecb0 2024-09-10T23:37:21.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.669+0000 7fd6baa00640 1 --2- 172.21.15.94:0/739305163 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd698072730 0x7fd698074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:21.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.669+0000 7fd6baa00640 1 --2- 172.21.15.94:0/739305163 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd698072730 0x7fd698074bf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac007fd0 tx=0x7fd6ac009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:21.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.669+0000 7fd6b3e00640 1 -- 172.21.15.94:0/739305163 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd6bc1f82a0 con 0x7fd6bc15ecb0 2024-09-10T23:37:21.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.869+0000 7fd6c1e00640 1 -- 172.21.15.94:0/739305163 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6bc06d270 con 0x7fd6bc15ecb0 2024-09-10T23:37:21.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.869+0000 7fd6b3e00640 1 -- 172.21.15.94:0/739305163 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+3913 (secure 0 0 0) 0x7fd6b405eee0 con 0x7fd6bc15ecb0 2024-09-10T23:37:21.870 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:21.871 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":61,"btime":"2024-09-10T23:37:15:484998+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1215179751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1215179751},{"type":"v1","addr":"172.21.15.94:6834","nonce":1215179751}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6672,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3193009978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":3193009978},{"type":"v1","addr":"172.21.15.181:6835","nonce":3193009978}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1426560042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":1426560042},{"type":"v1","addr":"172.21.15.94:6835","nonce":1426560042}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":195878,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6834/64212827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":64212827},{"type":"v1","addr":"172.21.15.181:6834","nonce":64212827}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[]} 2024-09-10T23:37:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 61 2024-09-10T23:37:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.869+0000 7fd6c1e00640 1 -- 172.21.15.94:0/739305163 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd698072730 msgr2=0x7fd698074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.869+0000 7fd6c1e00640 1 --2- 172.21.15.94:0/739305163 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd698072730 0x7fd698074bf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac007fd0 tx=0x7fd6ac009040 comp rx=0 tx=0).stop 2024-09-10T23:37:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.869+0000 7fd6c1e00640 1 -- 172.21.15.94:0/739305163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6bc15ecb0 msgr2=0x7fd6bc14c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.869+0000 7fd6c1e00640 1 --2- 172.21.15.94:0/739305163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6bc15ecb0 0x7fd6bc14c950 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400ea80 tx=0x7fd6b400ef40 comp rx=0 tx=0).stop 2024-09-10T23:37:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.869+0000 7fd6c1e00640 1 -- 172.21.15.94:0/739305163 shutdown_connections 2024-09-10T23:37:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.869+0000 7fd6c1e00640 1 --2- 172.21.15.94:0/739305163 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd698072730 0x7fd698074bf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.869+0000 7fd6c1e00640 1 --2- 172.21.15.94:0/739305163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6bc15ecb0 0x7fd6bc14c950 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.869+0000 7fd6c1e00640 1 --2- 172.21.15.94:0/739305163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6bc0d04b0 0x7fd6bc145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.869+0000 7fd6c1e00640 1 --2- 172.21.15.94:0/739305163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6bc073ff0 0x7fd6bc1f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.869+0000 7fd6c1e00640 1 -- 172.21.15.94:0/739305163 >> 172.21.15.94:0/739305163 conn(0x7fd6bc06cdf0 msgr2=0x7fd6bc0b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.869+0000 7fd6c1e00640 1 -- 172.21.15.94:0/739305163 shutdown_connections 2024-09-10T23:37:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:21.869+0000 7fd6c1e00640 1 -- 172.21.15.94:0/739305163 wait complete. 2024-09-10T23:37:21.892 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T23:37:21.892 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T23:37:21.892 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T23:37:21.892 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:37:21.893 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T23:37:21.893 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T23:37:21.893 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T23:37:21.893 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:37:21.893 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T23:37:21.893 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T23:37:21.893 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T23:37:21.894 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:37:21.894 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T23:37:21.894 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T23:37:21.894 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T23:37:21.894 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:37:21.904 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T23:37:21.904 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T23:37:21.905 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T23:37:21.906 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T23:37:21.909 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T23:37:21.909 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T23:37:21.910 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T23:37:21.910 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T23:37:21.951 INFO:tasks.ceph.mds.c:Started 2024-09-10T23:37:21.951 DEBUG:teuthology.parallel:result is None 2024-09-10T23:37:21.953 INFO:tasks.ceph.mds.a:Started 2024-09-10T23:37:21.953 DEBUG:teuthology.parallel:result is None 2024-09-10T23:37:21.960 INFO:tasks.ceph.mds.d:Started 2024-09-10T23:37:21.960 DEBUG:teuthology.parallel:result is None 2024-09-10T23:37:21.962 INFO:tasks.ceph.mds.b:Started 2024-09-10T23:37:21.962 DEBUG:teuthology.parallel:result is None 2024-09-10T23:37:21.962 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T23:37:22.037 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-09-10T23:37:22.037 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-09-10T23:37:22.055 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:37:22.056 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2024-09-10T23:37:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.109+0000 7f8b62400640 1 -- 172.21.15.94:0/3508670826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b5c13cfe0 msgr2=0x7f8b5c0cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.109+0000 7f8b62400640 1 --2- 172.21.15.94:0/3508670826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b5c13cfe0 0x7f8b5c0cec60 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0c6b20 tx=0x7f8b5401dbb0 comp rx=0 tx=0).stop 2024-09-10T23:37:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.109+0000 7f8b62400640 1 -- 172.21.15.94:0/3508670826 shutdown_connections 2024-09-10T23:37:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.109+0000 7f8b62400640 1 --2- 172.21.15.94:0/3508670826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b5c13cfe0 0x7f8b5c0cec60 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.109+0000 7f8b62400640 1 --2- 172.21.15.94:0/3508670826 >> v2:172.21.15.181:3300/0 conn(0x7f8b5c15e180 0x7f8b5c0ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:22.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.109+0000 7f8b62400640 1 -- 172.21.15.94:0/3508670826 >> 172.21.15.94:0/3508670826 conn(0x7f8b5c0cfaf0 msgr2=0x7f8b5c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:22.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.109+0000 7f8b62400640 1 -- 172.21.15.94:0/3508670826 shutdown_connections 2024-09-10T23:37:22.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.109+0000 7f8b62400640 1 -- 172.21.15.94:0/3508670826 wait complete. 2024-09-10T23:37:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.109+0000 7f8b62400640 1 Processor -- start 2024-09-10T23:37:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.109+0000 7f8b62400640 1 -- start start 2024-09-10T23:37:22.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b62400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5c13cc00 0x7f8b5c0b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:22.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b62400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b5c13cfe0 0x7f8b5c142f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:22.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b62400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b5c15e180 0x7f8b5c072540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:22.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b62400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c13f7e0 con 0x7f8b5c15e180 2024-09-10T23:37:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b62400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c13f660 con 0x7f8b5c13cc00 2024-09-10T23:37:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b62400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8b5c13f960 con 0x7f8b5c13cfe0 2024-09-10T23:37:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b5b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5c13cc00 0x7f8b5c0b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b5be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b5c15e180 0x7f8b5c072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b5b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5c13cc00 0x7f8b5c0b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:56638/0 (socket says 172.21.15.94:56638) 2024-09-10T23:37:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b5b400640 1 -- 172.21.15.94:0/2917124972 learned_addr learned my addr 172.21.15.94:0/2917124972 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b52a00640 1 --2- 172.21.15.94:0/2917124972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b5c13cfe0 0x7f8b5c142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b5be00640 1 -- 172.21.15.94:0/2917124972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b5c13cfe0 msgr2=0x7f8b5c142f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b5be00640 1 --2- 172.21.15.94:0/2917124972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b5c13cfe0 0x7f8b5c142f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:22.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b5be00640 1 -- 172.21.15.94:0/2917124972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5c13cc00 msgr2=0x7f8b5c0b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:22.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b5be00640 1 --2- 172.21.15.94:0/2917124972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5c13cc00 0x7f8b5c0b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:22.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b5be00640 1 -- 172.21.15.94:0/2917124972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5c15d620 con 0x7f8b5c15e180 2024-09-10T23:37:22.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b5b400640 1 --2- 172.21.15.94:0/2917124972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5c13cc00 0x7f8b5c0b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:37:22.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b5be00640 1 --2- 172.21.15.94:0/2917124972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b5c15e180 0x7f8b5c072540 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8b540ad000 tx=0x7f8b5400a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:22.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b53e00640 1 -- 172.21.15.94:0/2917124972 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b540b3040 con 0x7f8b5c15e180 2024-09-10T23:37:22.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b53e00640 1 -- 172.21.15.94:0/2917124972 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b54004c50 con 0x7f8b5c15e180 2024-09-10T23:37:22.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b62400640 1 -- 172.21.15.94:0/2917124972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5c15b730 con 0x7f8b5c15e180 2024-09-10T23:37:22.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b53e00640 1 -- 172.21.15.94:0/2917124972 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54004220 con 0x7f8b5c15e180 2024-09-10T23:37:22.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b62400640 1 -- 172.21.15.94:0/2917124972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5c15bcf0 con 0x7f8b5c15e180 2024-09-10T23:37:22.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.113+0000 7f8b62400640 1 -- 172.21.15.94:0/2917124972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b5c13fe80 con 0x7f8b5c15e180 2024-09-10T23:37:22.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.117+0000 7f8b53e00640 1 -- 172.21.15.94:0/2917124972 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f8b540043c0 con 0x7f8b5c15e180 2024-09-10T23:37:22.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.117+0000 7f8b53e00640 1 --2- 172.21.15.94:0/2917124972 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8b38072700 0x7f8b38074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:22.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.117+0000 7f8b53e00640 1 -- 172.21.15.94:0/2917124972 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5029+0+0 (secure 0 0 0) 0x7f8b5412d760 con 0x7f8b5c15e180 2024-09-10T23:37:22.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.117+0000 7f8b5b400640 1 --2- 172.21.15.94:0/2917124972 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8b38072700 0x7f8b38074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:22.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.117+0000 7f8b5b400640 1 --2- 172.21.15.94:0/2917124972 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8b38072700 0x7f8b38074bc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c001d00 tx=0x7f8b4c009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:22.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.121+0000 7f8b53e00640 1 -- 172.21.15.94:0/2917124972 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8b540c5060 con 0x7f8b5c15e180 2024-09-10T23:37:22.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.297+0000 7f8b62400640 1 -- 172.21.15.94:0/2917124972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f8b5c0cf770 con 0x7f8b5c15e180 2024-09-10T23:37:22.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.521+0000 7f8b53e00640 1 -- 172.21.15.94:0/2917124972 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v93) ==== 97+0+0 (secure 0 0 0) 0x7f8b54017860 con 0x7f8b5c15e180 2024-09-10T23:37:22.524 INFO:teuthology.orchestra.run.smithi094.stderr: removed all blocklist entries 2024-09-10T23:37:22.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.525+0000 7f8b62400640 1 -- 172.21.15.94:0/2917124972 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8b38072700 msgr2=0x7f8b38074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:22.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.525+0000 7f8b62400640 1 --2- 172.21.15.94:0/2917124972 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8b38072700 0x7f8b38074bc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c001d00 tx=0x7f8b4c009040 comp rx=0 tx=0).stop 2024-09-10T23:37:22.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.525+0000 7f8b62400640 1 -- 172.21.15.94:0/2917124972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b5c15e180 msgr2=0x7f8b5c072540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:22.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.525+0000 7f8b62400640 1 --2- 172.21.15.94:0/2917124972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b5c15e180 0x7f8b5c072540 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8b540ad000 tx=0x7f8b5400a440 comp rx=0 tx=0).stop 2024-09-10T23:37:22.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.525+0000 7f8b62400640 1 -- 172.21.15.94:0/2917124972 shutdown_connections 2024-09-10T23:37:22.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.525+0000 7f8b62400640 1 --2- 172.21.15.94:0/2917124972 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8b38072700 0x7f8b38074bc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:22.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.525+0000 7f8b62400640 1 --2- 172.21.15.94:0/2917124972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b5c15e180 0x7f8b5c072540 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:22.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.525+0000 7f8b62400640 1 --2- 172.21.15.94:0/2917124972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b5c13cfe0 0x7f8b5c142f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:22.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.525+0000 7f8b62400640 1 --2- 172.21.15.94:0/2917124972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5c13cc00 0x7f8b5c0b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:22.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.525+0000 7f8b62400640 1 -- 172.21.15.94:0/2917124972 >> 172.21.15.94:0/2917124972 conn(0x7f8b5c0cfaf0 msgr2=0x7f8b5c0af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:22.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.525+0000 7f8b62400640 1 -- 172.21.15.94:0/2917124972 shutdown_connections 2024-09-10T23:37:22.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.525+0000 7f8b62400640 1 -- 172.21.15.94:0/2917124972 wait complete. 2024-09-10T23:37:22.551 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T23:37:22.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.689+0000 7f1b23e00640 1 -- 172.21.15.94:0/1974119499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0ae910 msgr2=0x7f1b1c0cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:22.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.689+0000 7f1b23e00640 1 --2- 172.21.15.94:0/1974119499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0ae910 0x7f1b1c0cace0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009840 tx=0x7f1b1401c790 comp rx=0 tx=0).stop 2024-09-10T23:37:22.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.689+0000 7f1b23e00640 1 -- 172.21.15.94:0/1974119499 shutdown_connections 2024-09-10T23:37:22.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.689+0000 7f1b23e00640 1 --2- 172.21.15.94:0/1974119499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0ae910 0x7f1b1c0cace0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:22.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.689+0000 7f1b23e00640 1 -- 172.21.15.94:0/1974119499 >> 172.21.15.94:0/1974119499 conn(0x7f1b1c0cc900 msgr2=0x7f1b1c0ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:22.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.689+0000 7f1b23e00640 1 -- 172.21.15.94:0/1974119499 shutdown_connections 2024-09-10T23:37:22.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.689+0000 7f1b23e00640 1 -- 172.21.15.94:0/1974119499 wait complete. 2024-09-10T23:37:22.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b23e00640 1 Processor -- start 2024-09-10T23:37:22.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b23e00640 1 -- start start 2024-09-10T23:37:22.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b23e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b1c073970 0x7f1b1c14fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:22.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b23e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0ae910 0x7f1b1c1574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:22.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b23e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b1c0cdea0 0x7f1b1c1579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:22.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b23e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c168df0 con 0x7f1b1c0ae910 2024-09-10T23:37:22.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b23e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c168c70 con 0x7f1b1c073970 2024-09-10T23:37:22.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b23e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1b1c168f70 con 0x7f1b1c0cdea0 2024-09-10T23:37:22.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b20a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0ae910 0x7f1b1c1574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b20a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0ae910 0x7f1b1c1574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59162/0 (socket says 172.21.15.94:59162) 2024-09-10T23:37:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b20a00640 1 -- 172.21.15.94:0/1105778135 learned_addr learned my addr 172.21.15.94:0/1105778135 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b21400640 1 --2- 172.21.15.94:0/1105778135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b1c073970 0x7f1b1c14fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b21e00640 1 --2- 172.21.15.94:0/1105778135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b1c0cdea0 0x7f1b1c1579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b20a00640 1 -- 172.21.15.94:0/1105778135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b1c0cdea0 msgr2=0x7f1b1c1579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b20a00640 1 --2- 172.21.15.94:0/1105778135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b1c0cdea0 0x7f1b1c1579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b20a00640 1 -- 172.21.15.94:0/1105778135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b1c073970 msgr2=0x7f1b1c14fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b20a00640 1 --2- 172.21.15.94:0/1105778135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b1c073970 0x7f1b1c14fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b20a00640 1 -- 172.21.15.94:0/1105778135 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c074290 con 0x7f1b1c0ae910 2024-09-10T23:37:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b21400640 1 --2- 172.21.15.94:0/1105778135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b1c073970 0x7f1b1c14fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b20a00640 1 --2- 172.21.15.94:0/1105778135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0ae910 0x7f1b1c1574b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800c3a0 tx=0x7f1b0800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b19600640 1 -- 172.21.15.94:0/1105778135 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b08017070 con 0x7f1b1c0ae910 2024-09-10T23:37:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b19600640 1 -- 172.21.15.94:0/1105778135 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b08003600 con 0x7f1b1c0ae910 2024-09-10T23:37:22.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b19600640 1 -- 172.21.15.94:0/1105778135 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b08003a50 con 0x7f1b1c0ae910 2024-09-10T23:37:22.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b23e00640 1 -- 172.21.15.94:0/1105778135 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1c1f5d80 con 0x7f1b1c0ae910 2024-09-10T23:37:22.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.693+0000 7f1b23e00640 1 -- 172.21.15.94:0/1105778135 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1c1f6340 con 0x7f1b1c0ae910 2024-09-10T23:37:22.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.697+0000 7f1b23e00640 1 -- 172.21.15.94:0/1105778135 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b1c073d50 con 0x7f1b1c0ae910 2024-09-10T23:37:22.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.697+0000 7f1b19600640 1 -- 172.21.15.94:0/1105778135 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f1b08003bf0 con 0x7f1b1c0ae910 2024-09-10T23:37:22.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.697+0000 7f1b19600640 1 --2- 172.21.15.94:0/1105778135 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1afc072730 0x7f1afc074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:22.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.697+0000 7f1b19600640 1 -- 172.21.15.94:0/1105778135 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4857+0+0 (secure 0 0 0) 0x7f1b08092770 con 0x7f1b1c0ae910 2024-09-10T23:37:22.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.697+0000 7f1b21400640 1 --2- 172.21.15.94:0/1105778135 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1afc072730 0x7f1afc074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:22.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.701+0000 7f1b21400640 1 --2- 172.21.15.94:0/1105778135 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1afc072730 0x7f1afc074bf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0c6830 tx=0x7f1b10009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:22.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.701+0000 7f1b19600640 1 -- 172.21.15.94:0/1105778135 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1b0805dee0 con 0x7f1b1c0ae910 2024-09-10T23:37:22.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.925+0000 7f1b23e00640 1 -- 172.21.15.94:0/1105778135 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1b1c074050 con 0x7f1b1c0ae910 2024-09-10T23:37:22.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.925+0000 7f1b19600640 1 -- 172.21.15.94:0/1105778135 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v34) ==== 80+0+5304 (secure 0 0 0) 0x7f1b0805d7e0 con 0x7f1b1c0ae910 2024-09-10T23:37:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:22.928 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:37:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_dump": [ 2024-09-10T23:37:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.a", 2024-09-10T23:37:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/2m3LxAAr0ErpLJg+0+I2r74zQya4w==", 2024-09-10T23:37:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:37:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:37:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:37:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:37:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.b", 2024-09-10T23:37:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmc8pYABAAw90zu8D+9IwK15h4HIlGBA==", 2024-09-10T23:37:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:37:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:37:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:37:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:37:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.c", 2024-09-10T23:37:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmk98KNxAA44e9N4VHbw/94Lh1V9zugQ==", 2024-09-10T23:37:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:37:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:37:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:37:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:37:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.d", 2024-09-10T23:37:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmVAKzBxAAJehOw6aphTF/g3ucoQfw9A==", 2024-09-10T23:37:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:37:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:37:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:37:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:37:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.0", 2024-09-10T23:37:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDa1uBmSAD3KBAAJTF1OEC44Ml599aL7iEuDw==", 2024-09-10T23:37:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:37:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:37:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:37:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.1", 2024-09-10T23:37:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDe1uBma1L9ABAAfEjW1/Hb4fh4Kwv/ZUgFAQ==", 2024-09-10T23:37:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:37:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:37:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:37:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.2", 2024-09-10T23:37:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDh1uBmtLUQFhAAweeFHj6BRQVC9BX+VLiWEQ==", 2024-09-10T23:37:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:37:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:37:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:37:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.3", 2024-09-10T23:37:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDk1uBm1KFCKRAAev28iYcrEYOViX95uo6Kbw==", 2024-09-10T23:37:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:37:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:37:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:37:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.4", 2024-09-10T23:37:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDq1uBm/Hg9LhAAvB2D0rj4DeRPxTrGmXYIsA==", 2024-09-10T23:37:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:37:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:37:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:37:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.5", 2024-09-10T23:37:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDu1uBm1fAKCBAAFHMjnGFAItLdaACF/UjE/w==", 2024-09-10T23:37:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:37:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:37:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:37:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.6", 2024-09-10T23:37:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDx1uBmXgYfHhAA/6wwtM4mDRXsIpoTIHKJ0w==", 2024-09-10T23:37:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:37:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:37:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:37:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.7", 2024-09-10T23:37:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQD01uBmw6hQMxAAk7oGAg+iIplNK5KrokGwNA==", 2024-09-10T23:37:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:37:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:37:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:37:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.0", 2024-09-10T23:37:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g==", 2024-09-10T23:37:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:37:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow r", 2024-09-10T23:37:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T23:37:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.1", 2024-09-10T23:37:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmuSkhFRAA1R6WNXLImkGfhW9/XJzEAQ==", 2024-09-10T23:37:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:37:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:37:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:37:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:37:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.2", 2024-09-10T23:37:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmfvaqERAASpL0KtglFcAuRf4a/0z5JQ==", 2024-09-10T23:37:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:37:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:37:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:37:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:37:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.3", 2024-09-10T23:37:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBm8xUtGxAARc0ywfAhMxBNd57TzBoV3g==", 2024-09-10T23:37:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:37:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:37:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:37:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:37:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.admin", 2024-09-10T23:37:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmOpg1FBAA9NV6Jqt6doWMuAfcxztlOg==", 2024-09-10T23:37:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:37:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:37:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:37:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:37:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.x", 2024-09-10T23:37:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/EPVKxAA/Myv8Zjf7BejbzlkxyCC1w==", 2024-09-10T23:37:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:37:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:37:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:37:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.y", 2024-09-10T23:37:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmhHvtJxAACe+79nYhv2ODwtmNhA45FA==", 2024-09-10T23:37:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:37:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:37:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:37:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:37:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:37:22.943 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:37:22.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.925+0000 7f1b23e00640 1 -- 172.21.15.94:0/1105778135 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1afc072730 msgr2=0x7f1afc074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:22.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.925+0000 7f1b23e00640 1 --2- 172.21.15.94:0/1105778135 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1afc072730 0x7f1afc074bf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0c6830 tx=0x7f1b10009040 comp rx=0 tx=0).stop 2024-09-10T23:37:22.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.925+0000 7f1b23e00640 1 -- 172.21.15.94:0/1105778135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0ae910 msgr2=0x7f1b1c1574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:22.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.925+0000 7f1b23e00640 1 --2- 172.21.15.94:0/1105778135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0ae910 0x7f1b1c1574b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800c3a0 tx=0x7f1b0800c860 comp rx=0 tx=0).stop 2024-09-10T23:37:22.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.925+0000 7f1b23e00640 1 -- 172.21.15.94:0/1105778135 shutdown_connections 2024-09-10T23:37:22.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.925+0000 7f1b23e00640 1 --2- 172.21.15.94:0/1105778135 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1afc072730 0x7f1afc074bf0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:22.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.925+0000 7f1b23e00640 1 --2- 172.21.15.94:0/1105778135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b1c0cdea0 0x7f1b1c1579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:22.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.925+0000 7f1b23e00640 1 --2- 172.21.15.94:0/1105778135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c0ae910 0x7f1b1c1574b0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:22.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.925+0000 7f1b23e00640 1 --2- 172.21.15.94:0/1105778135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b1c073970 0x7f1b1c14fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:22.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.929+0000 7f1b23e00640 1 -- 172.21.15.94:0/1105778135 >> 172.21.15.94:0/1105778135 conn(0x7f1b1c0cc900 msgr2=0x7f1b1c0b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:22.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.929+0000 7f1b23e00640 1 -- 172.21.15.94:0/1105778135 shutdown_connections 2024-09-10T23:37:22.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:22.929+0000 7f1b23e00640 1 -- 172.21.15.94:0/1105778135 wait complete. 2024-09-10T23:37:22.949 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T23:37:23.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdae3000640 1 -- 172.21.15.94:0/4283924492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdadc0d03c0 msgr2=0x7fdadc15e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:23.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdae3000640 1 --2- 172.21.15.94:0/4283924492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdadc0d03c0 0x7fdadc15e590 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009900 tx=0x7fdac401c790 comp rx=0 tx=0).stop 2024-09-10T23:37:23.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdae3000640 1 -- 172.21.15.94:0/4283924492 shutdown_connections 2024-09-10T23:37:23.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdae3000640 1 --2- 172.21.15.94:0/4283924492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdadc0d03c0 0x7fdadc15e590 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:23.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdae3000640 1 -- 172.21.15.94:0/4283924492 >> 172.21.15.94:0/4283924492 conn(0x7fdadc06cdf0 msgr2=0x7fdadc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:23.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdae3000640 1 -- 172.21.15.94:0/4283924492 shutdown_connections 2024-09-10T23:37:23.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdae3000640 1 -- 172.21.15.94:0/4283924492 wait complete. 2024-09-10T23:37:23.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdae3000640 1 Processor -- start 2024-09-10T23:37:23.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdae3000640 1 -- start start 2024-09-10T23:37:23.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdae3000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdadc0cf3c0 0x7fdadc1fb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:23.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdae3000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdadc0d03c0 0x7fdadc14d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:23.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdae3000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdadc15ead0 0x7fdadc158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:23.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdae3000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc168ac0 con 0x7fdadc0cf3c0 2024-09-10T23:37:23.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdae3000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc168940 con 0x7fdadc15ead0 2024-09-10T23:37:23.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdae3000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fdadc168c40 con 0x7fdadc0d03c0 2024-09-10T23:37:23.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdadbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdadc0cf3c0 0x7fdadc1fb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:23.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdadbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdadc0cf3c0 0x7fdadc1fb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59180/0 (socket says 172.21.15.94:59180) 2024-09-10T23:37:23.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdadbe00640 1 -- 172.21.15.94:0/3656679780 learned_addr learned my addr 172.21.15.94:0/3656679780 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:23.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdae1000640 1 --2- 172.21.15.94:0/3656679780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdadc15ead0 0x7fdadc158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:23.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.089+0000 7fdad3400640 1 --2- 172.21.15.94:0/3656679780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdadc0d03c0 0x7fdadc14d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:23.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.093+0000 7fdadbe00640 1 -- 172.21.15.94:0/3656679780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdadc0d03c0 msgr2=0x7fdadc14d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:23.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.093+0000 7fdadbe00640 1 --2- 172.21.15.94:0/3656679780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdadc0d03c0 0x7fdadc14d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:23.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.093+0000 7fdadbe00640 1 -- 172.21.15.94:0/3656679780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdadc15ead0 msgr2=0x7fdadc158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:23.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.093+0000 7fdadbe00640 1 --2- 172.21.15.94:0/3656679780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdadc15ead0 0x7fdadc158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:23.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.093+0000 7fdadbe00640 1 -- 172.21.15.94:0/3656679780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdadc1fbb80 con 0x7fdadc0cf3c0 2024-09-10T23:37:23.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.093+0000 7fdadbe00640 1 --2- 172.21.15.94:0/3656679780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdadc0cf3c0 0x7fdadc1fb640 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00c7c0 tx=0x7fdacc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:23.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.093+0000 7fdad8c00640 1 -- 172.21.15.94:0/3656679780 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc016020 con 0x7fdadc0cf3c0 2024-09-10T23:37:23.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.093+0000 7fdad8c00640 1 -- 172.21.15.94:0/3656679780 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdacc00ab40 con 0x7fdadc0cf3c0 2024-09-10T23:37:23.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.093+0000 7fdae3000640 1 -- 172.21.15.94:0/3656679780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdadc1555d0 con 0x7fdadc0cf3c0 2024-09-10T23:37:23.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.093+0000 7fdad8c00640 1 -- 172.21.15.94:0/3656679780 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc012420 con 0x7fdadc0cf3c0 2024-09-10T23:37:23.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.093+0000 7fdae3000640 1 -- 172.21.15.94:0/3656679780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdadc163a40 con 0x7fdadc0cf3c0 2024-09-10T23:37:23.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.093+0000 7fdae3000640 1 -- 172.21.15.94:0/3656679780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdadc06f930 con 0x7fdadc0cf3c0 2024-09-10T23:37:23.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.093+0000 7fdad8c00640 1 -- 172.21.15.94:0/3656679780 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fdacc01b400 con 0x7fdadc0cf3c0 2024-09-10T23:37:23.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.097+0000 7fdad8c00640 1 --2- 172.21.15.94:0/3656679780 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdab8072730 0x7fdab8074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:23.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.097+0000 7fdad8c00640 1 -- 172.21.15.94:0/3656679780 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4857+0+0 (secure 0 0 0) 0x7fdacc0935d0 con 0x7fdadc0cf3c0 2024-09-10T23:37:23.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.097+0000 7fdad3400640 1 --2- 172.21.15.94:0/3656679780 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdab8072730 0x7fdab8074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:23.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.097+0000 7fdad3400640 1 --2- 172.21.15.94:0/3656679780 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdab8072730 0x7fdab8074bf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdac40af000 tx=0x7fdac401afd0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:23.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.101+0000 7fdad8c00640 1 -- 172.21.15.94:0/3656679780 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdacc020030 con 0x7fdadc0cf3c0 2024-09-10T23:37:23.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.269+0000 7fdae3000640 1 -- 172.21.15.94:0/3656679780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdadc0008e0 con 0x7fdadc0cf3c0 2024-09-10T23:37:23.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.269+0000 7fdad8c00640 1 -- 172.21.15.94:0/3656679780 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v95) ==== 84+0+65 (secure 0 0 0) 0x7fdadc1fbb80 con 0x7fdadc0cf3c0 2024-09-10T23:37:23.274 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:23.274 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:37:23.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:23.274 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-09-10T23:37:23.274 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-09-10T23:37:23.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:23.275 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:37:23.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.273+0000 7fdae3000640 1 -- 172.21.15.94:0/3656679780 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdab8072730 msgr2=0x7fdab8074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:23.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.273+0000 7fdae3000640 1 --2- 172.21.15.94:0/3656679780 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdab8072730 0x7fdab8074bf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdac40af000 tx=0x7fdac401afd0 comp rx=0 tx=0).stop 2024-09-10T23:37:23.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.273+0000 7fdae3000640 1 -- 172.21.15.94:0/3656679780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdadc0cf3c0 msgr2=0x7fdadc1fb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:23.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.273+0000 7fdae3000640 1 --2- 172.21.15.94:0/3656679780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdadc0cf3c0 0x7fdadc1fb640 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00c7c0 tx=0x7fdacc00cc80 comp rx=0 tx=0).stop 2024-09-10T23:37:23.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.273+0000 7fdae3000640 1 -- 172.21.15.94:0/3656679780 shutdown_connections 2024-09-10T23:37:23.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.273+0000 7fdae3000640 1 --2- 172.21.15.94:0/3656679780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdadc15ead0 0x7fdadc158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:23.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.273+0000 7fdae3000640 1 --2- 172.21.15.94:0/3656679780 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdab8072730 0x7fdab8074bf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:23.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.273+0000 7fdae3000640 1 --2- 172.21.15.94:0/3656679780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdadc0d03c0 0x7fdadc14d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:23.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.273+0000 7fdae3000640 1 --2- 172.21.15.94:0/3656679780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdadc0cf3c0 0x7fdadc1fb640 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:23.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.273+0000 7fdae3000640 1 -- 172.21.15.94:0/3656679780 >> 172.21.15.94:0/3656679780 conn(0x7fdadc06cdf0 msgr2=0x7fdadc15cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:23.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.273+0000 7fdae3000640 1 -- 172.21.15.94:0/3656679780 shutdown_connections 2024-09-10T23:37:23.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.273+0000 7fdae3000640 1 -- 172.21.15.94:0/3656679780 wait complete. 2024-09-10T23:37:23.294 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T23:37:23.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T23:37:23.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.437+0000 7f41a1400640 1 -- 172.21.15.94:0/3821575281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f419c15eca0 msgr2=0x7f419c165710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:23.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.437+0000 7f41a1400640 1 --2- 172.21.15.94:0/3821575281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f419c15eca0 0x7f419c165710 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4194009bc0 tx=0x7f419401ca80 comp rx=0 tx=0).stop 2024-09-10T23:37:23.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.437+0000 7f41a1400640 1 -- 172.21.15.94:0/3821575281 shutdown_connections 2024-09-10T23:37:23.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.437+0000 7f41a1400640 1 --2- 172.21.15.94:0/3821575281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f419c15eca0 0x7f419c165710 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:23.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.437+0000 7f41a1400640 1 -- 172.21.15.94:0/3821575281 >> 172.21.15.94:0/3821575281 conn(0x7f419c06cdf0 msgr2=0x7f419c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:23.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.437+0000 7f41a1400640 1 -- 172.21.15.94:0/3821575281 shutdown_connections 2024-09-10T23:37:23.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.437+0000 7f41a1400640 1 -- 172.21.15.94:0/3821575281 wait complete. 2024-09-10T23:37:23.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f41a1400640 1 Processor -- start 2024-09-10T23:37:23.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f41a1400640 1 -- start start 2024-09-10T23:37:23.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f41a1400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f419c0cf3c0 0x7f419c154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:23.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f41a1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f419c0d03c0 0x7f419c1529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:23.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f41a1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f419c15eca0 0x7f419c1433c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:23.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f41a1400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f419c1689c0 con 0x7f419c0d03c0 2024-09-10T23:37:23.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f41a1400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f419c168840 con 0x7f419c15eca0 2024-09-10T23:37:23.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f41a1400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f419c168b40 con 0x7f419c0cf3c0 2024-09-10T23:37:23.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f4191e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f419c0d03c0 0x7f419c1529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:23.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f4191e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f419c0d03c0 0x7f419c1529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59192/0 (socket says 172.21.15.94:59192) 2024-09-10T23:37:23.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f4191e00640 1 -- 172.21.15.94:0/3108991673 learned_addr learned my addr 172.21.15.94:0/3108991673 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:23.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f4191e00640 1 -- 172.21.15.94:0/3108991673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f419c0cf3c0 msgr2=0x7f419c154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:23.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f419a800640 1 --2- 172.21.15.94:0/3108991673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f419c0cf3c0 0x7f419c154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:23.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f419b200640 1 --2- 172.21.15.94:0/3108991673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f419c15eca0 0x7f419c1433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:23.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f4191e00640 1 --2- 172.21.15.94:0/3108991673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f419c0cf3c0 0x7f419c154c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:23.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f4191e00640 1 -- 172.21.15.94:0/3108991673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f419c15eca0 msgr2=0x7f419c1433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:23.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f4191e00640 1 --2- 172.21.15.94:0/3108991673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f419c15eca0 0x7f419c1433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:23.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f4191e00640 1 -- 172.21.15.94:0/3108991673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419c152f30 con 0x7f419c0d03c0 2024-09-10T23:37:23.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f419b200640 1 --2- 172.21.15.94:0/3108991673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f419c15eca0 0x7f419c1433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:37:23.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f419a800640 1 --2- 172.21.15.94:0/3108991673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f419c0cf3c0 0x7f419c154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:37:23.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f4191e00640 1 --2- 172.21.15.94:0/3108991673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f419c0d03c0 0x7f419c1529f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f418400c4d0 tx=0x7f418400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:23.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f4193400640 1 -- 172.21.15.94:0/3108991673 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184016020 con 0x7f419c0d03c0 2024-09-10T23:37:23.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f4193400640 1 -- 172.21.15.94:0/3108991673 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4184012920 con 0x7f419c0d03c0 2024-09-10T23:37:23.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f4193400640 1 -- 172.21.15.94:0/3108991673 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184012d50 con 0x7f419c0d03c0 2024-09-10T23:37:23.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f41a1400640 1 -- 172.21.15.94:0/3108991673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419c1531b0 con 0x7f419c0d03c0 2024-09-10T23:37:23.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.441+0000 7f41a1400640 1 -- 172.21.15.94:0/3108991673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f419c144d40 con 0x7f419c0d03c0 2024-09-10T23:37:23.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.445+0000 7f41a1400640 1 -- 172.21.15.94:0/3108991673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f419c0cfcb0 con 0x7f419c0d03c0 2024-09-10T23:37:23.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.445+0000 7f4193400640 1 -- 172.21.15.94:0/3108991673 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f418401a3b0 con 0x7f419c0d03c0 2024-09-10T23:37:23.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.445+0000 7f4193400640 1 --2- 172.21.15.94:0/3108991673 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4178072730 0x7f4178074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:23.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.445+0000 7f4193400640 1 -- 172.21.15.94:0/3108991673 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4857+0+0 (secure 0 0 0) 0x7f41840930d0 con 0x7f419c0d03c0 2024-09-10T23:37:23.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.445+0000 7f419a800640 1 --2- 172.21.15.94:0/3108991673 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4178072730 0x7f4178074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:23.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.445+0000 7f419a800640 1 --2- 172.21.15.94:0/3108991673 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4178072730 0x7f4178074bf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f418c000f70 tx=0x7f418c009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:23.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.449+0000 7f4193400640 1 -- 172.21.15.94:0/3108991673 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4184020030 con 0x7f419c0d03c0 2024-09-10T23:37:23.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:23.617+0000 7f41a1400640 1 -- 172.21.15.94:0/3108991673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f419c06d4b0 con 0x7f419c0d03c0 2024-09-10T23:37:24.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.537+0000 7f4193400640 1 -- 172.21.15.94:0/3108991673 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v96) ==== 138+0+0 (secure 0 0 0) 0x7f419c152f30 con 0x7f419c0d03c0 2024-09-10T23:37:24.539 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-09-10T23:37:24.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.541+0000 7f41a1400640 1 -- 172.21.15.94:0/3108991673 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4178072730 msgr2=0x7f4178074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:24.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.541+0000 7f41a1400640 1 --2- 172.21.15.94:0/3108991673 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4178072730 0x7f4178074bf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f418c000f70 tx=0x7f418c009040 comp rx=0 tx=0).stop 2024-09-10T23:37:24.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.541+0000 7f41a1400640 1 -- 172.21.15.94:0/3108991673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f419c0d03c0 msgr2=0x7f419c1529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:24.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.541+0000 7f41a1400640 1 --2- 172.21.15.94:0/3108991673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f419c0d03c0 0x7f419c1529f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f418400c4d0 tx=0x7f418400c990 comp rx=0 tx=0).stop 2024-09-10T23:37:24.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.541+0000 7f41a1400640 1 -- 172.21.15.94:0/3108991673 shutdown_connections 2024-09-10T23:37:24.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.541+0000 7f41a1400640 1 --2- 172.21.15.94:0/3108991673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f419c15eca0 0x7f419c1433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:24.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.541+0000 7f41a1400640 1 --2- 172.21.15.94:0/3108991673 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4178072730 0x7f4178074bf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:24.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.541+0000 7f41a1400640 1 --2- 172.21.15.94:0/3108991673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f419c0d03c0 0x7f419c1529f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:24.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.541+0000 7f41a1400640 1 --2- 172.21.15.94:0/3108991673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f419c0cf3c0 0x7f419c154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:24.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.541+0000 7f41a1400640 1 -- 172.21.15.94:0/3108991673 >> 172.21.15.94:0/3108991673 conn(0x7f419c06cdf0 msgr2=0x7f419c14f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:24.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.541+0000 7f41a1400640 1 -- 172.21.15.94:0/3108991673 shutdown_connections 2024-09-10T23:37:24.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.541+0000 7f41a1400640 1 -- 172.21.15.94:0/3108991673 wait complete. 2024-09-10T23:37:24.568 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T23:37:24.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.721+0000 7f2dcfc00640 1 -- 172.21.15.94:0/3023371285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc80c5a70 msgr2=0x7f2dc80cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:24.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.721+0000 7f2dcfc00640 1 --2- 172.21.15.94:0/3023371285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc80c5a70 0x7f2dc80cd9f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009cb0 tx=0x7f2dc001cb90 comp rx=0 tx=0).stop 2024-09-10T23:37:24.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.721+0000 7f2dcfc00640 1 -- 172.21.15.94:0/3023371285 shutdown_connections 2024-09-10T23:37:24.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.721+0000 7f2dcfc00640 1 --2- 172.21.15.94:0/3023371285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc80c5a70 0x7f2dc80cd9f0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:24.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.721+0000 7f2dcfc00640 1 --2- 172.21.15.94:0/3023371285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc80d0710 0x7f2dc80c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:24.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.721+0000 7f2dcfc00640 1 -- 172.21.15.94:0/3023371285 >> 172.21.15.94:0/3023371285 conn(0x7f2dc80cfaf0 msgr2=0x7f2dc80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:24.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.721+0000 7f2dcfc00640 1 -- 172.21.15.94:0/3023371285 shutdown_connections 2024-09-10T23:37:24.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.721+0000 7f2dcfc00640 1 -- 172.21.15.94:0/3023371285 wait complete. 2024-09-10T23:37:24.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.721+0000 7f2dcfc00640 1 Processor -- start 2024-09-10T23:37:24.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.721+0000 7f2dcfc00640 1 -- start start 2024-09-10T23:37:24.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dcfc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc80c5a70 0x7f2dc8072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:24.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dcfc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc80d0710 0x7f2dc8136020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:24.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dcfc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc81455d0 0x7f2dc80afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:24.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dcfc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc81694a0 con 0x7f2dc80d0710 2024-09-10T23:37:24.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dcfc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc8169320 con 0x7f2dc80c5a70 2024-09-10T23:37:24.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dcfc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc8169620 con 0x7f2dc81455d0 2024-09-10T23:37:24.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dbfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc80d0710 0x7f2dc8136020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:24.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dbfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc80d0710 0x7f2dc8136020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60526/0 (socket says 172.21.15.94:60526) 2024-09-10T23:37:24.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dbfe00640 1 -- 172.21.15.94:0/2553833880 learned_addr learned my addr 172.21.15.94:0/2553833880 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:24.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dcdc00640 1 --2- 172.21.15.94:0/2553833880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc81455d0 0x7f2dc80afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:24.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dcd200640 1 --2- 172.21.15.94:0/2553833880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc80c5a70 0x7f2dc8072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:24.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dbfe00640 1 -- 172.21.15.94:0/2553833880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc81455d0 msgr2=0x7f2dc80afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:24.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dbfe00640 1 --2- 172.21.15.94:0/2553833880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc81455d0 0x7f2dc80afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:24.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dbfe00640 1 -- 172.21.15.94:0/2553833880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc80c5a70 msgr2=0x7f2dc8072560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:24.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dbfe00640 1 --2- 172.21.15.94:0/2553833880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc80c5a70 0x7f2dc8072560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:24.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dbfe00640 1 -- 172.21.15.94:0/2553833880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc8143c50 con 0x7f2dc80d0710 2024-09-10T23:37:24.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dbfe00640 1 --2- 172.21.15.94:0/2553833880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc80d0710 0x7f2dc8136020 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2db000c970 tx=0x7f2db000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:24.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dc5600640 1 -- 172.21.15.94:0/2553833880 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0007c10 con 0x7f2dc80d0710 2024-09-10T23:37:24.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dc5600640 1 -- 172.21.15.94:0/2553833880 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2db000f450 con 0x7f2dc80d0710 2024-09-10T23:37:24.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dcfc00640 1 -- 172.21.15.94:0/2553833880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc81433b0 con 0x7f2dc80d0710 2024-09-10T23:37:24.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dc5600640 1 -- 172.21.15.94:0/2553833880 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db000f8a0 con 0x7f2dc80d0710 2024-09-10T23:37:24.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dcfc00640 1 -- 172.21.15.94:0/2553833880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc8143970 con 0x7f2dc80d0710 2024-09-10T23:37:24.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.725+0000 7f2dcfc00640 1 -- 172.21.15.94:0/2553833880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dc80cd9f0 con 0x7f2dc80d0710 2024-09-10T23:37:24.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.729+0000 7f2dc5600640 1 -- 172.21.15.94:0/2553833880 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f2db000fa40 con 0x7f2dc80d0710 2024-09-10T23:37:24.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.729+0000 7f2dc5600640 1 --2- 172.21.15.94:0/2553833880 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2da4072700 0x7f2da4074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:24.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.729+0000 7f2dc5600640 1 -- 172.21.15.94:0/2553833880 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5227+0+0 (secure 0 0 0) 0x7f2db0093b00 con 0x7f2dc80d0710 2024-09-10T23:37:24.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.729+0000 7f2dcd200640 1 --2- 172.21.15.94:0/2553833880 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2da4072700 0x7f2da4074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:24.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.729+0000 7f2dcd200640 1 --2- 172.21.15.94:0/2553833880 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2da4072700 0x7f2da4074bc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2dc815ff90 tx=0x7f2db8009060 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:24.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.733+0000 7f2dc5600640 1 -- 172.21.15.94:0/2553833880 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2db0014030 con 0x7f2dc80d0710 2024-09-10T23:37:24.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:24.901+0000 7f2dcfc00640 1 -- 172.21.15.94:0/2553833880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2dc80cf6e0 con 0x7f2dc80d0710 2024-09-10T23:37:25.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.545+0000 7f2dc5600640 1 -- 172.21.15.94:0/2553833880 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v97) ==== 170+0+0 (secure 0 0 0) 0x7f2db005ebc0 con 0x7f2dc80d0710 2024-09-10T23:37:25.546 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-09-10T23:37:25.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.545+0000 7f2dcfc00640 1 -- 172.21.15.94:0/2553833880 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2da4072700 msgr2=0x7f2da4074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:25.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.545+0000 7f2dcfc00640 1 --2- 172.21.15.94:0/2553833880 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2da4072700 0x7f2da4074bc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2dc815ff90 tx=0x7f2db8009060 comp rx=0 tx=0).stop 2024-09-10T23:37:25.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.545+0000 7f2dcfc00640 1 -- 172.21.15.94:0/2553833880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc80d0710 msgr2=0x7f2dc8136020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.545+0000 7f2dcfc00640 1 --2- 172.21.15.94:0/2553833880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc80d0710 0x7f2dc8136020 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2db000c970 tx=0x7f2db000ce30 comp rx=0 tx=0).stop 2024-09-10T23:37:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.545+0000 7f2dcfc00640 1 -- 172.21.15.94:0/2553833880 shutdown_connections 2024-09-10T23:37:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.545+0000 7f2dcfc00640 1 --2- 172.21.15.94:0/2553833880 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2da4072700 0x7f2da4074bc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.545+0000 7f2dcfc00640 1 --2- 172.21.15.94:0/2553833880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc81455d0 0x7f2dc80afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.545+0000 7f2dcfc00640 1 --2- 172.21.15.94:0/2553833880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dc80d0710 0x7f2dc8136020 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.545+0000 7f2dcfc00640 1 --2- 172.21.15.94:0/2553833880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc80c5a70 0x7f2dc8072560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.545+0000 7f2dcfc00640 1 -- 172.21.15.94:0/2553833880 >> 172.21.15.94:0/2553833880 conn(0x7f2dc80cfaf0 msgr2=0x7f2dc8159b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.545+0000 7f2dcfc00640 1 -- 172.21.15.94:0/2553833880 shutdown_connections 2024-09-10T23:37:25.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.545+0000 7f2dcfc00640 1 -- 172.21.15.94:0/2553833880 wait complete. 2024-09-10T23:37:25.586 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T23:37:25.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.729+0000 7f853ce00640 1 -- 172.21.15.94:0/2469393178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85380b3200 msgr2=0x7f85380d49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:25.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.729+0000 7f853ce00640 1 --2- 172.21.15.94:0/2469393178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85380b3200 0x7f85380d49c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8520009870 tx=0x7f852001c750 comp rx=0 tx=0).stop 2024-09-10T23:37:25.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.729+0000 7f853ce00640 1 -- 172.21.15.94:0/2469393178 shutdown_connections 2024-09-10T23:37:25.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.729+0000 7f853ce00640 1 --2- 172.21.15.94:0/2469393178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85380b3200 0x7f85380d49c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:25.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.729+0000 7f853ce00640 1 -- 172.21.15.94:0/2469393178 >> 172.21.15.94:0/2469393178 conn(0x7f85380d3ea0 msgr2=0x7f85380d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:25.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.729+0000 7f853ce00640 1 -- 172.21.15.94:0/2469393178 shutdown_connections 2024-09-10T23:37:25.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.729+0000 7f853ce00640 1 -- 172.21.15.94:0/2469393178 wait complete. 2024-09-10T23:37:25.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.729+0000 7f853ce00640 1 Processor -- start 2024-09-10T23:37:25.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.729+0000 7f853ce00640 1 -- start start 2024-09-10T23:37:25.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f853ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85380b3200 0x7f853815a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:25.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f853ce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85380d0ea0 0x7f8538157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:25.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f853ce00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f853815df80 0x7f853815d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:25.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f853ce00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f853816c000 con 0x7f85380d0ea0 2024-09-10T23:37:25.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f853ce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f853816be80 con 0x7f85380b3200 2024-09-10T23:37:25.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f853ce00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f853816c180 con 0x7f853815df80 2024-09-10T23:37:25.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f852d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85380d0ea0 0x7f8538157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:25.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f852d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85380d0ea0 0x7f8538157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60534/0 (socket says 172.21.15.94:60534) 2024-09-10T23:37:25.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f852d400640 1 -- 172.21.15.94:0/3435264700 learned_addr learned my addr 172.21.15.94:0/3435264700 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:25.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f8535e00640 1 --2- 172.21.15.94:0/3435264700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85380b3200 0x7f853815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:25.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f8536800640 1 --2- 172.21.15.94:0/3435264700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f853815df80 0x7f853815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:25.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f852d400640 1 -- 172.21.15.94:0/3435264700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f853815df80 msgr2=0x7f853815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:25.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f852d400640 1 --2- 172.21.15.94:0/3435264700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f853815df80 0x7f853815d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:25.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f852d400640 1 -- 172.21.15.94:0/3435264700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85380b3200 msgr2=0x7f853815a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:25.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f852d400640 1 --2- 172.21.15.94:0/3435264700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85380b3200 0x7f853815a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:25.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f852d400640 1 -- 172.21.15.94:0/3435264700 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853815c590 con 0x7f85380d0ea0 2024-09-10T23:37:25.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f852d400640 1 --2- 172.21.15.94:0/3435264700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85380d0ea0 0x7f8538157580 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f852001a380 tx=0x7f8520005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:25.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f852ea00640 1 -- 172.21.15.94:0/3435264700 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85200be020 con 0x7f85380d0ea0 2024-09-10T23:37:25.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f852ea00640 1 -- 172.21.15.94:0/3435264700 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85200a7be0 con 0x7f85380d0ea0 2024-09-10T23:37:25.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f853ce00640 1 -- 172.21.15.94:0/3435264700 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853815c870 con 0x7f85380d0ea0 2024-09-10T23:37:25.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f852ea00640 1 -- 172.21.15.94:0/3435264700 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520003600 con 0x7f85380d0ea0 2024-09-10T23:37:25.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f853ce00640 1 -- 172.21.15.94:0/3435264700 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85380cba20 con 0x7f85380d0ea0 2024-09-10T23:37:25.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.733+0000 7f853ce00640 1 -- 172.21.15.94:0/3435264700 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85380d49c0 con 0x7f85380d0ea0 2024-09-10T23:37:25.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.737+0000 7f852ea00640 1 -- 172.21.15.94:0/3435264700 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f85200037a0 con 0x7f85380d0ea0 2024-09-10T23:37:25.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.737+0000 7f852ea00640 1 --2- 172.21.15.94:0/3435264700 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8514072700 0x7f8514074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:25.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.737+0000 7f852ea00640 1 -- 172.21.15.94:0/3435264700 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5609+0+0 (secure 0 0 0) 0x7f852012ef20 con 0x7f85380d0ea0 2024-09-10T23:37:25.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.737+0000 7f8535e00640 1 --2- 172.21.15.94:0/3435264700 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8514072700 0x7f8514074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:25.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.737+0000 7f8535e00640 1 --2- 172.21.15.94:0/3435264700 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8514072700 0x7f8514074bc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8528007fd0 tx=0x7f8528009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:25.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.741+0000 7f852ea00640 1 -- 172.21.15.94:0/3435264700 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f85200c8030 con 0x7f85380d0ea0 2024-09-10T23:37:25.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:25.937+0000 7f853ce00640 1 -- 172.21.15.94:0/3435264700 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8538069dc0 con 0x7f85380d0ea0 2024-09-10T23:37:26.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.545+0000 7f852ea00640 1 -- 172.21.15.94:0/3435264700 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v64) ==== 174+0+0 (secure 0 0 0) 0x7f8520016370 con 0x7f85380d0ea0 2024-09-10T23:37:26.549 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 16 and data pool 17 2024-09-10T23:37:26.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.549+0000 7f853ce00640 1 -- 172.21.15.94:0/3435264700 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8514072700 msgr2=0x7f8514074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:26.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.549+0000 7f853ce00640 1 --2- 172.21.15.94:0/3435264700 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8514072700 0x7f8514074bc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8528007fd0 tx=0x7f8528009040 comp rx=0 tx=0).stop 2024-09-10T23:37:26.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.549+0000 7f853ce00640 1 -- 172.21.15.94:0/3435264700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85380d0ea0 msgr2=0x7f8538157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:26.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.549+0000 7f853ce00640 1 --2- 172.21.15.94:0/3435264700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85380d0ea0 0x7f8538157580 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f852001a380 tx=0x7f8520005ca0 comp rx=0 tx=0).stop 2024-09-10T23:37:26.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.549+0000 7f853ce00640 1 -- 172.21.15.94:0/3435264700 shutdown_connections 2024-09-10T23:37:26.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.549+0000 7f853ce00640 1 --2- 172.21.15.94:0/3435264700 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8514072700 0x7f8514074bc0 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8528007fd0 tx=0x7f8528009040 comp rx=0 tx=0).stop 2024-09-10T23:37:26.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.549+0000 7f853ce00640 1 --2- 172.21.15.94:0/3435264700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f853815df80 0x7f853815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:26.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.549+0000 7f853ce00640 1 --2- 172.21.15.94:0/3435264700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85380d0ea0 0x7f8538157580 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:26.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.549+0000 7f853ce00640 1 --2- 172.21.15.94:0/3435264700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85380b3200 0x7f853815a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:26.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.549+0000 7f853ce00640 1 -- 172.21.15.94:0/3435264700 >> 172.21.15.94:0/3435264700 conn(0x7f85380d3ea0 msgr2=0x7f85380b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:26.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.549+0000 7f853ce00640 1 -- 172.21.15.94:0/3435264700 shutdown_connections 2024-09-10T23:37:26.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.549+0000 7f853ce00640 1 -- 172.21.15.94:0/3435264700 wait complete. 2024-09-10T23:37:26.594 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:37:26.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.733+0000 7f192aa00640 1 -- 172.21.15.94:0/2036241117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f192406e7d0 msgr2=0x7f19240ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:26.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.733+0000 7f192aa00640 1 --2- 172.21.15.94:0/2036241117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f192406e7d0 0x7f19240ce680 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f19240b47d0 tx=0x7f191c01ca90 comp rx=0 tx=0).stop 2024-09-10T23:37:26.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.733+0000 7f192aa00640 1 -- 172.21.15.94:0/2036241117 shutdown_connections 2024-09-10T23:37:26.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.733+0000 7f192aa00640 1 --2- 172.21.15.94:0/2036241117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f192406e7d0 0x7f19240ce680 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:26.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.733+0000 7f192aa00640 1 --2- 172.21.15.94:0/2036241117 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1924162560 0x7f192406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:26.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.733+0000 7f192aa00640 1 --2- 172.21.15.94:0/2036241117 >> v2:172.21.15.181:3300/0 conn(0x7f192415dd50 0x7f19240b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:26.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.733+0000 7f192aa00640 1 -- 172.21.15.94:0/2036241117 >> 172.21.15.94:0/2036241117 conn(0x7f192409aa70 msgr2=0x7f192409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:26.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.733+0000 7f192aa00640 1 -- 172.21.15.94:0/2036241117 shutdown_connections 2024-09-10T23:37:26.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.733+0000 7f192aa00640 1 -- 172.21.15.94:0/2036241117 wait complete. 2024-09-10T23:37:26.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.733+0000 7f192aa00640 1 Processor -- start 2024-09-10T23:37:26.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f192aa00640 1 -- start start 2024-09-10T23:37:26.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f192aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f192406e7d0 0x7f192414e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:26.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f192aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f192415dd50 0x7f192414a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:26.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f192aa00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1924162560 0x7f19240cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:26.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f192aa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f19240cc6a0 con 0x7f192415dd50 2024-09-10T23:37:26.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f192aa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f19240cc520 con 0x7f192406e7d0 2024-09-10T23:37:26.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f192aa00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f19240cc820 con 0x7f1924162560 2024-09-10T23:37:26.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f1923400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f192415dd50 0x7f192414a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:26.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f1923400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f192415dd50 0x7f192414a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60552/0 (socket says 172.21.15.94:60552) 2024-09-10T23:37:26.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f1923400640 1 -- 172.21.15.94:0/1167426165 learned_addr learned my addr 172.21.15.94:0/1167426165 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:26.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f1923e00640 1 --2- 172.21.15.94:0/1167426165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f192406e7d0 0x7f192414e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:26.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f1928a00640 1 --2- 172.21.15.94:0/1167426165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1924162560 0x7f19240cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:26.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f1923400640 1 -- 172.21.15.94:0/1167426165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1924162560 msgr2=0x7f19240cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:26.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f1923400640 1 --2- 172.21.15.94:0/1167426165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1924162560 0x7f19240cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:26.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f1923400640 1 -- 172.21.15.94:0/1167426165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f192406e7d0 msgr2=0x7f192414e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:26.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f1923400640 1 --2- 172.21.15.94:0/1167426165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f192406e7d0 0x7f192414e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:26.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f1923400640 1 -- 172.21.15.94:0/1167426165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192414ab60 con 0x7f192415dd50 2024-09-10T23:37:26.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f1923400640 1 --2- 172.21.15.94:0/1167426165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f192415dd50 0x7f192414a620 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f191000e7a0 tx=0x7f191000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:26.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f191be00640 1 -- 172.21.15.94:0/1167426165 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191000ccc0 con 0x7f192415dd50 2024-09-10T23:37:26.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f191be00640 1 -- 172.21.15.94:0/1167426165 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1910010430 con 0x7f192415dd50 2024-09-10T23:37:26.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f191be00640 1 -- 172.21.15.94:0/1167426165 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910010860 con 0x7f192415dd50 2024-09-10T23:37:26.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f192aa00640 1 -- 172.21.15.94:0/1167426165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19240b6e10 con 0x7f192415dd50 2024-09-10T23:37:26.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f192aa00640 1 -- 172.21.15.94:0/1167426165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19240c9f80 con 0x7f192415dd50 2024-09-10T23:37:26.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.737+0000 7f192aa00640 1 -- 172.21.15.94:0/1167426165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19240cae30 con 0x7f192415dd50 2024-09-10T23:37:26.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.741+0000 7f191be00640 1 -- 172.21.15.94:0/1167426165 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f1910010a00 con 0x7f192415dd50 2024-09-10T23:37:26.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.741+0000 7f191be00640 1 --2- 172.21.15.94:0/1167426165 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1904072700 0x7f1904074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:26.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.741+0000 7f191be00640 1 -- 172.21.15.94:0/1167426165 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7f1910093ed0 con 0x7f192415dd50 2024-09-10T23:37:26.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.741+0000 7f1923e00640 1 --2- 172.21.15.94:0/1167426165 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1904072700 0x7f1904074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:26.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.741+0000 7f1923e00640 1 --2- 172.21.15.94:0/1167426165 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1904072700 0x7f1904074bc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1914009b60 tx=0x7f19140091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:26.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.745+0000 7f191be00640 1 -- 172.21.15.94:0/1167426165 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f191005eca0 con 0x7f192415dd50 2024-09-10T23:37:26.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.913+0000 7f192aa00640 1 -- 172.21.15.94:0/1167426165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19240b1b80 con 0x7f192415dd50 2024-09-10T23:37:26.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.913+0000 7f191be00640 1 -- 172.21.15.94:0/1167426165 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+16733 (secure 0 0 0) 0x7f19100636d0 con 0x7f192415dd50 2024-09-10T23:37:26.917 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:26.917 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":98,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:37:25.941810+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:37:23.632549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T23:37:24.916517+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:37:26.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.917+0000 7f192aa00640 1 -- 172.21.15.94:0/1167426165 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1904072700 msgr2=0x7f1904074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:26.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.917+0000 7f192aa00640 1 --2- 172.21.15.94:0/1167426165 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1904072700 0x7f1904074bc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1914009b60 tx=0x7f19140091e0 comp rx=0 tx=0).stop 2024-09-10T23:37:26.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.917+0000 7f192aa00640 1 -- 172.21.15.94:0/1167426165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f192415dd50 msgr2=0x7f192414a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:26.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.917+0000 7f192aa00640 1 --2- 172.21.15.94:0/1167426165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f192415dd50 0x7f192414a620 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f191000e7a0 tx=0x7f191000ec60 comp rx=0 tx=0).stop 2024-09-10T23:37:26.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.917+0000 7f192aa00640 1 -- 172.21.15.94:0/1167426165 shutdown_connections 2024-09-10T23:37:26.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.917+0000 7f192aa00640 1 --2- 172.21.15.94:0/1167426165 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1904072700 0x7f1904074bc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:26.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.917+0000 7f192aa00640 1 --2- 172.21.15.94:0/1167426165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1924162560 0x7f19240cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:26.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.917+0000 7f192aa00640 1 --2- 172.21.15.94:0/1167426165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f192415dd50 0x7f192414a620 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:26.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.917+0000 7f192aa00640 1 --2- 172.21.15.94:0/1167426165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f192406e7d0 0x7f192414e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:26.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.917+0000 7f192aa00640 1 -- 172.21.15.94:0/1167426165 >> 172.21.15.94:0/1167426165 conn(0x7f192409aa70 msgr2=0x7f1924071200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:26.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.917+0000 7f192aa00640 1 -- 172.21.15.94:0/1167426165 shutdown_connections 2024-09-10T23:37:26.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:26.917+0000 7f192aa00640 1 -- 172.21.15.94:0/1167426165 wait complete. 2024-09-10T23:37:26.939 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:37:27.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.077+0000 7fc3c5e00640 1 -- 172.21.15.94:0/2036649340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00b2bb0 msgr2=0x7fc3c00d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:27.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.077+0000 7fc3c5e00640 1 --2- 172.21.15.94:0/2036649340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00b2bb0 0x7fc3c00d00a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009840 tx=0x7fc3a801c700 comp rx=0 tx=0).stop 2024-09-10T23:37:27.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.077+0000 7fc3c5e00640 1 -- 172.21.15.94:0/2036649340 shutdown_connections 2024-09-10T23:37:27.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.077+0000 7fc3c5e00640 1 --2- 172.21.15.94:0/2036649340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00b2bb0 0x7fc3c00d00a0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:27.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.077+0000 7fc3c5e00640 1 --2- 172.21.15.94:0/2036649340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3c015f5c0 0x7fc3c00cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:27.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.077+0000 7fc3c5e00640 1 -- 172.21.15.94:0/2036649340 >> 172.21.15.94:0/2036649340 conn(0x7fc3c00d1c60 msgr2=0x7fc3c00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:27.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.077+0000 7fc3c5e00640 1 -- 172.21.15.94:0/2036649340 shutdown_connections 2024-09-10T23:37:27.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.077+0000 7fc3c5e00640 1 -- 172.21.15.94:0/2036649340 wait complete. 2024-09-10T23:37:27.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3c5e00640 1 Processor -- start 2024-09-10T23:37:27.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3c5e00640 1 -- start start 2024-09-10T23:37:27.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3c5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00b2bb0 0x7fc3c014ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:27.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3c5e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3c00d05e0 0x7fc3c00b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:27.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3c5e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3c015f5c0 0x7fc3c015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:27.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3c5e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c0169250 con 0x7fc3c00b2bb0 2024-09-10T23:37:27.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3c5e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c01690d0 con 0x7fc3c00d05e0 2024-09-10T23:37:27.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3c5e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c01693d0 con 0x7fc3c015f5c0 2024-09-10T23:37:27.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3bea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00b2bb0 0x7fc3c014ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:27.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3bea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00b2bb0 0x7fc3c014ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60576/0 (socket says 172.21.15.94:60576) 2024-09-10T23:37:27.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3bea00640 1 -- 172.21.15.94:0/4274300840 learned_addr learned my addr 172.21.15.94:0/4274300840 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:27.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3b6000640 1 --2- 172.21.15.94:0/4274300840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3c00d05e0 0x7fc3c00b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:27.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3bf400640 1 --2- 172.21.15.94:0/4274300840 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3c015f5c0 0x7fc3c015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:27.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3bea00640 1 -- 172.21.15.94:0/4274300840 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3c015f5c0 msgr2=0x7fc3c015c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:27.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3bea00640 1 --2- 172.21.15.94:0/4274300840 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3c015f5c0 0x7fc3c015c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:27.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3bea00640 1 -- 172.21.15.94:0/4274300840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3c00d05e0 msgr2=0x7fc3c00b71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:27.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3bea00640 1 --2- 172.21.15.94:0/4274300840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3c00d05e0 0x7fc3c00b71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:27.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3bea00640 1 -- 172.21.15.94:0/4274300840 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c00b52a0 con 0x7fc3c00b2bb0 2024-09-10T23:37:27.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3bea00640 1 --2- 172.21.15.94:0/4274300840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00b2bb0 0x7fc3c014ca20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00058d0 tx=0x7fc3b000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:27.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3b7e00640 1 -- 172.21.15.94:0/4274300840 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0017070 con 0x7fc3c00b2bb0 2024-09-10T23:37:27.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3b7e00640 1 -- 172.21.15.94:0/4274300840 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3b0009d20 con 0x7fc3c00b2bb0 2024-09-10T23:37:27.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3c5e00640 1 -- 172.21.15.94:0/4274300840 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c00b4ce0 con 0x7fc3c00b2bb0 2024-09-10T23:37:27.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3b7e00640 1 -- 172.21.15.94:0/4274300840 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b00106c0 con 0x7fc3c00b2bb0 2024-09-10T23:37:27.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.081+0000 7fc3c5e00640 1 -- 172.21.15.94:0/4274300840 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c00b69a0 con 0x7fc3c00b2bb0 2024-09-10T23:37:27.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.085+0000 7fc3c5e00640 1 -- 172.21.15.94:0/4274300840 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c00b15c0 con 0x7fc3c00b2bb0 2024-09-10T23:37:27.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.085+0000 7fc3b7e00640 1 -- 172.21.15.94:0/4274300840 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fc3b0010860 con 0x7fc3c00b2bb0 2024-09-10T23:37:27.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.085+0000 7fc3b7e00640 1 --2- 172.21.15.94:0/4274300840 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc39c072700 0x7fc39c074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:27.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.085+0000 7fc3b7e00640 1 -- 172.21.15.94:0/4274300840 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fc3b00938f0 con 0x7fc3c00b2bb0 2024-09-10T23:37:27.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.085+0000 7fc3b6000640 1 --2- 172.21.15.94:0/4274300840 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc39c072700 0x7fc39c074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:27.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.085+0000 7fc3b6000640 1 --2- 172.21.15.94:0/4274300840 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc39c072700 0x7fc39c074bc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80af000 tx=0x7fc3a801af40 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:27.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.089+0000 7fc3b7e00640 1 -- 172.21.15.94:0/4274300840 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc3c00b52a0 con 0x7fc3c00b2bb0 2024-09-10T23:37:27.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.253+0000 7fc3c5e00640 1 -- 172.21.15.94:0/4274300840 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3c00749c0 con 0x7fc3c00b2bb0 2024-09-10T23:37:27.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.257+0000 7fc3b7e00640 1 -- 172.21.15.94:0/4274300840 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+16733 (secure 0 0 0) 0x7fc3b005e6c0 con 0x7fc3c00b2bb0 2024-09-10T23:37:27.258 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:27.258 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":98,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:37:25.941810+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:37:23.632549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T23:37:24.916517+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:37:27.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.257+0000 7fc3c5e00640 1 -- 172.21.15.94:0/4274300840 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc39c072700 msgr2=0x7fc39c074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:27.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.257+0000 7fc3c5e00640 1 --2- 172.21.15.94:0/4274300840 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc39c072700 0x7fc39c074bc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80af000 tx=0x7fc3a801af40 comp rx=0 tx=0).stop 2024-09-10T23:37:27.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.257+0000 7fc3c5e00640 1 -- 172.21.15.94:0/4274300840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00b2bb0 msgr2=0x7fc3c014ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:27.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.257+0000 7fc3c5e00640 1 --2- 172.21.15.94:0/4274300840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00b2bb0 0x7fc3c014ca20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00058d0 tx=0x7fc3b000dde0 comp rx=0 tx=0).stop 2024-09-10T23:37:27.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.257+0000 7fc3c5e00640 1 -- 172.21.15.94:0/4274300840 shutdown_connections 2024-09-10T23:37:27.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.257+0000 7fc3c5e00640 1 --2- 172.21.15.94:0/4274300840 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3c00d05e0 0x7fc3c00b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:27.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.257+0000 7fc3c5e00640 1 --2- 172.21.15.94:0/4274300840 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc39c072700 0x7fc39c074bc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:27.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.257+0000 7fc3c5e00640 1 --2- 172.21.15.94:0/4274300840 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3c015f5c0 0x7fc3c015c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:27.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.257+0000 7fc3c5e00640 1 --2- 172.21.15.94:0/4274300840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c00b2bb0 0x7fc3c014ca20 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:27.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.257+0000 7fc3c5e00640 1 -- 172.21.15.94:0/4274300840 >> 172.21.15.94:0/4274300840 conn(0x7fc3c00d1c60 msgr2=0x7fc3c006e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:27.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.257+0000 7fc3c5e00640 1 -- 172.21.15.94:0/4274300840 shutdown_connections 2024-09-10T23:37:27.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.257+0000 7fc3c5e00640 1 -- 172.21.15.94:0/4274300840 wait complete. 2024-09-10T23:37:27.280 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T23:37:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.421+0000 7fe60d000640 1 -- 172.21.15.94:0/278464933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6080ce890 msgr2=0x7fe6080cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.421+0000 7fe60d000640 1 --2- 172.21.15.94:0/278464933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6080ce890 0x7fe6080cd710 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009900 tx=0x7fe5f801c820 comp rx=0 tx=0).stop 2024-09-10T23:37:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.421+0000 7fe60d000640 1 -- 172.21.15.94:0/278464933 shutdown_connections 2024-09-10T23:37:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.421+0000 7fe60d000640 1 --2- 172.21.15.94:0/278464933 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6080ce1e0 0x7fe6080cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.421+0000 7fe60d000640 1 --2- 172.21.15.94:0/278464933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6080ce890 0x7fe6080cd710 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.421+0000 7fe60d000640 1 -- 172.21.15.94:0/278464933 >> 172.21.15.94:0/278464933 conn(0x7fe6080d1c60 msgr2=0x7fe6080d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.421+0000 7fe60d000640 1 -- 172.21.15.94:0/278464933 shutdown_connections 2024-09-10T23:37:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.421+0000 7fe60d000640 1 -- 172.21.15.94:0/278464933 wait complete. 2024-09-10T23:37:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe60d000640 1 Processor -- start 2024-09-10T23:37:27.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe60d000640 1 -- start start 2024-09-10T23:37:27.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe60d000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6080b1100 0x7fe60814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:27.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe60d000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6080ce1e0 0x7fe608071600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe60d000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6080ce890 0x7fe608071b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe60d000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe6080d08e0 con 0x7fe6080ce890 2024-09-10T23:37:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe60d000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe6080d0760 con 0x7fe6080ce1e0 2024-09-10T23:37:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe60d000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe6080d0a60 con 0x7fe6080b1100 2024-09-10T23:37:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe606800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6080ce890 0x7fe608071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe605e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6080b1100 0x7fe60814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe605e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6080b1100 0x7fe60814ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:57958/0 (socket says 172.21.15.94:57958) 2024-09-10T23:37:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe605e00640 1 -- 172.21.15.94:0/1854202070 learned_addr learned my addr 172.21.15.94:0/1854202070 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe5fd400640 1 --2- 172.21.15.94:0/1854202070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6080ce1e0 0x7fe608071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe605e00640 1 -- 172.21.15.94:0/1854202070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6080ce1e0 msgr2=0x7fe608071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe605e00640 1 --2- 172.21.15.94:0/1854202070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6080ce1e0 0x7fe608071600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:27.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe605e00640 1 -- 172.21.15.94:0/1854202070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6080ce890 msgr2=0x7fe608071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe605e00640 1 --2- 172.21.15.94:0/1854202070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6080ce890 0x7fe608071b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe605e00640 1 -- 172.21.15.94:0/1854202070 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe608159300 con 0x7fe6080b1100 2024-09-10T23:37:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe606800640 1 --2- 172.21.15.94:0/1854202070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6080ce890 0x7fe608071b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:37:27.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe605e00640 1 --2- 172.21.15.94:0/1854202070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6080b1100 0x7fe60814ca20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009850 tx=0x7fe5f80a5860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe5fea00640 1 -- 172.21.15.94:0/1854202070 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80b2040 con 0x7fe6080b1100 2024-09-10T23:37:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe5fea00640 1 -- 172.21.15.94:0/1854202070 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5f80043c0 con 0x7fe6080b1100 2024-09-10T23:37:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe60d000640 1 -- 172.21.15.94:0/1854202070 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60815de10 con 0x7fe6080b1100 2024-09-10T23:37:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe5fea00640 1 -- 172.21.15.94:0/1854202070 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8005290 con 0x7fe6080b1100 2024-09-10T23:37:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.425+0000 7fe60d000640 1 -- 172.21.15.94:0/1854202070 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60815e3d0 con 0x7fe6080b1100 2024-09-10T23:37:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.429+0000 7fe60d000640 1 -- 172.21.15.94:0/1854202070 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6080cec70 con 0x7fe6080b1100 2024-09-10T23:37:27.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.429+0000 7fe5fea00640 1 -- 172.21.15.94:0/1854202070 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fe5f8005430 con 0x7fe6080b1100 2024-09-10T23:37:27.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.429+0000 7fe5fea00640 1 --2- 172.21.15.94:0/1854202070 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5e4072700 0x7fe5e4074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.429+0000 7fe5fea00640 1 -- 172.21.15.94:0/1854202070 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(98..98 src has 1..98) ==== 5709+0+0 (secure 0 0 0) 0x7fe5f812cff0 con 0x7fe6080b1100 2024-09-10T23:37:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.429+0000 7fe5fd400640 1 --2- 172.21.15.94:0/1854202070 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5e4072700 0x7fe5e4074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:27.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.429+0000 7fe5fd400640 1 --2- 172.21.15.94:0/1854202070 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5e4072700 0x7fe5e4074bc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe6080c82f0 tx=0x7fe5f00091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:27.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.433+0000 7fe5fea00640 1 -- 172.21.15.94:0/1854202070 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe5f80164d0 con 0x7fe6080b1100 2024-09-10T23:37:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:27.633+0000 7fe60d000640 1 -- 172.21.15.94:0/1854202070 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe608070800 con 0x7fe6080b1100 2024-09-10T23:37:28.538 INFO:tasks.ceph.mds.d.smithi181.stderr:2024-09-10T23:37:28.538+0000 7f6fd0000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:37:28.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.585+0000 7fe5fea00640 1 -- 172.21.15.94:0/1854202070 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v67) ==== 118+0+0 (secure 0 0 0) 0x7fe5f80f7e70 con 0x7fe6080b1100 2024-09-10T23:37:28.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.589+0000 7fe60d000640 1 -- 172.21.15.94:0/1854202070 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5e4072700 msgr2=0x7fe5e4074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:28.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.589+0000 7fe60d000640 1 --2- 172.21.15.94:0/1854202070 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5e4072700 0x7fe5e4074bc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe6080c82f0 tx=0x7fe5f00091e0 comp rx=0 tx=0).stop 2024-09-10T23:37:28.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.589+0000 7fe60d000640 1 -- 172.21.15.94:0/1854202070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6080b1100 msgr2=0x7fe60814ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:28.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.589+0000 7fe60d000640 1 --2- 172.21.15.94:0/1854202070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6080b1100 0x7fe60814ca20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009850 tx=0x7fe5f80a5860 comp rx=0 tx=0).stop 2024-09-10T23:37:28.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.589+0000 7fe60d000640 1 -- 172.21.15.94:0/1854202070 shutdown_connections 2024-09-10T23:37:28.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.589+0000 7fe60d000640 1 --2- 172.21.15.94:0/1854202070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6080ce1e0 0x7fe608071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:28.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.589+0000 7fe60d000640 1 --2- 172.21.15.94:0/1854202070 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5e4072700 0x7fe5e4074bc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:28.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.589+0000 7fe60d000640 1 --2- 172.21.15.94:0/1854202070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6080ce890 0x7fe608071b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:28.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.589+0000 7fe60d000640 1 --2- 172.21.15.94:0/1854202070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6080b1100 0x7fe60814ca20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:28.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.589+0000 7fe60d000640 1 -- 172.21.15.94:0/1854202070 >> 172.21.15.94:0/1854202070 conn(0x7fe6080d1c60 msgr2=0x7fe6080c8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:28.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.589+0000 7fe60d000640 1 -- 172.21.15.94:0/1854202070 shutdown_connections 2024-09-10T23:37:28.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.589+0000 7fe60d000640 1 -- 172.21.15.94:0/1854202070 wait complete. 2024-09-10T23:37:28.614 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T23:37:28.614 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T23:37:28.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.761+0000 7f2326c00640 1 -- 172.21.15.94:0/1413321054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23200af150 msgr2=0x7f23200af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:28.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.761+0000 7f2326c00640 1 --2- 172.21.15.94:0/1413321054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23200af150 0x7f23200af5e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2320072eb0 tx=0x7f231801c890 comp rx=0 tx=0).stop 2024-09-10T23:37:28.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.761+0000 7f2326c00640 1 -- 172.21.15.94:0/1413321054 shutdown_connections 2024-09-10T23:37:28.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.761+0000 7f2326c00640 1 --2- 172.21.15.94:0/1413321054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23200af150 0x7f23200af5e0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:28.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.761+0000 7f2326c00640 1 --2- 172.21.15.94:0/1413321054 >> v2:172.21.15.181:3300/0 conn(0x7f23200b2ac0 0x7f23200b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:28.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.761+0000 7f2326c00640 1 -- 172.21.15.94:0/1413321054 >> 172.21.15.94:0/1413321054 conn(0x7f23200d1b00 msgr2=0x7f23200d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:28.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.761+0000 7f2326c00640 1 -- 172.21.15.94:0/1413321054 shutdown_connections 2024-09-10T23:37:28.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.761+0000 7f2326c00640 1 -- 172.21.15.94:0/1413321054 wait complete. 2024-09-10T23:37:28.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.761+0000 7f2326c00640 1 Processor -- start 2024-09-10T23:37:28.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.761+0000 7f2326c00640 1 -- start start 2024-09-10T23:37:28.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.761+0000 7f2326c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23200af150 0x7f232014d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:28.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.761+0000 7f2326c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23200b2ac0 0x7f232006eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:28.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.761+0000 7f2326c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f232015f140 0x7f23201581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:28.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.761+0000 7f2326c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2320168ad0 con 0x7f232015f140 2024-09-10T23:37:28.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.761+0000 7f2326c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2320168950 con 0x7f23200af150 2024-09-10T23:37:28.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.761+0000 7f2326c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2320168c50 con 0x7f23200b2ac0 2024-09-10T23:37:28.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f2326200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f232015f140 0x7f23201581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:28.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f2326200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f232015f140 0x7f23201581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60602/0 (socket says 172.21.15.94:60602) 2024-09-10T23:37:28.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f2326200640 1 -- 172.21.15.94:0/1228889054 learned_addr learned my addr 172.21.15.94:0/1228889054 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:28.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f2325800640 1 --2- 172.21.15.94:0/1228889054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23200af150 0x7f232014d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:28.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f2324e00640 1 --2- 172.21.15.94:0/1228889054 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23200b2ac0 0x7f232006eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:28.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f2326200640 1 -- 172.21.15.94:0/1228889054 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23200b2ac0 msgr2=0x7f232006eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:28.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f2326200640 1 --2- 172.21.15.94:0/1228889054 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23200b2ac0 0x7f232006eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:28.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f2326200640 1 -- 172.21.15.94:0/1228889054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23200af150 msgr2=0x7f232014d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:28.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f2326200640 1 --2- 172.21.15.94:0/1228889054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23200af150 0x7f232014d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:28.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f2326200640 1 -- 172.21.15.94:0/1228889054 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232014e1f0 con 0x7f232015f140 2024-09-10T23:37:28.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f2324e00640 1 --2- 172.21.15.94:0/1228889054 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23200b2ac0 0x7f232006eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:28.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f2326200640 1 --2- 172.21.15.94:0/1228889054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f232015f140 0x7f23201581b0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2318009850 tx=0x7f2318004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:28.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f231d600640 1 -- 172.21.15.94:0/1228889054 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23180b2040 con 0x7f232015f140 2024-09-10T23:37:28.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f231d600640 1 -- 172.21.15.94:0/1228889054 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2318003600 con 0x7f232015f140 2024-09-10T23:37:28.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f2326c00640 1 -- 172.21.15.94:0/1228889054 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2320156c50 con 0x7f232015f140 2024-09-10T23:37:28.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f231d600640 1 -- 172.21.15.94:0/1228889054 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318003a50 con 0x7f232015f140 2024-09-10T23:37:28.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f2326c00640 1 -- 172.21.15.94:0/1228889054 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2320157210 con 0x7f232015f140 2024-09-10T23:37:28.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f2326c00640 1 -- 172.21.15.94:0/1228889054 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23200af5e0 con 0x7f232015f140 2024-09-10T23:37:28.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.765+0000 7f231d600640 1 -- 172.21.15.94:0/1228889054 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f2318003bf0 con 0x7f232015f140 2024-09-10T23:37:28.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.769+0000 7f231d600640 1 --2- 172.21.15.94:0/1228889054 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2300072700 0x7f2300074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:28.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.769+0000 7f231d600640 1 -- 172.21.15.94:0/1228889054 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f231812d6d0 con 0x7f232015f140 2024-09-10T23:37:28.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.769+0000 7f2325800640 1 --2- 172.21.15.94:0/1228889054 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2300072700 0x7f2300074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:28.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.769+0000 7f2325800640 1 --2- 172.21.15.94:0/1228889054 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2300072700 0x7f2300074bc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f231400b450 tx=0x7f231400d040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:28.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.773+0000 7f231d600640 1 -- 172.21.15.94:0/1228889054 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f232014e1f0 con 0x7f232015f140 2024-09-10T23:37:28.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:28.973+0000 7f2326c00640 1 -- 172.21.15.94:0/1228889054 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f232015e8b0 con 0x7f232015f140 2024-09-10T23:37:29.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.589+0000 7f231d600640 1 -- 172.21.15.94:0/1228889054 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v68) ==== 122+0+0 (secure 0 0 0) 0x7f23180164b0 con 0x7f232015f140 2024-09-10T23:37:29.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.593+0000 7f2326c00640 1 -- 172.21.15.94:0/1228889054 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2300072700 msgr2=0x7f2300074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:29.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.593+0000 7f2326c00640 1 --2- 172.21.15.94:0/1228889054 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2300072700 0x7f2300074bc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f231400b450 tx=0x7f231400d040 comp rx=0 tx=0).stop 2024-09-10T23:37:29.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.593+0000 7f2326c00640 1 -- 172.21.15.94:0/1228889054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f232015f140 msgr2=0x7f23201581b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:29.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.593+0000 7f2326c00640 1 --2- 172.21.15.94:0/1228889054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f232015f140 0x7f23201581b0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2318009850 tx=0x7f2318004040 comp rx=0 tx=0).stop 2024-09-10T23:37:29.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.593+0000 7f2326c00640 1 -- 172.21.15.94:0/1228889054 shutdown_connections 2024-09-10T23:37:29.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.593+0000 7f2326c00640 1 --2- 172.21.15.94:0/1228889054 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2300072700 0x7f2300074bc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:29.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.593+0000 7f2326c00640 1 --2- 172.21.15.94:0/1228889054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f232015f140 0x7f23201581b0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:29.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.593+0000 7f2326c00640 1 --2- 172.21.15.94:0/1228889054 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23200b2ac0 0x7f232006eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:29.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.593+0000 7f2326c00640 1 --2- 172.21.15.94:0/1228889054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23200af150 0x7f232014d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:29.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.593+0000 7f2326c00640 1 -- 172.21.15.94:0/1228889054 >> 172.21.15.94:0/1228889054 conn(0x7f23200d1b00 msgr2=0x7f23200cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:29.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.593+0000 7f2326c00640 1 -- 172.21.15.94:0/1228889054 shutdown_connections 2024-09-10T23:37:29.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.593+0000 7f2326c00640 1 -- 172.21.15.94:0/1228889054 wait complete. 2024-09-10T23:37:29.615 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T23:37:29.616 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:37:29.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba36000640 1 -- 172.21.15.94:0/361755416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba30070150 msgr2=0x7fba30072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:29.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba36000640 1 --2- 172.21.15.94:0/361755416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba30070150 0x7fba30072610 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fba2800a950 tx=0x7fba2801dae0 comp rx=0 tx=0).stop 2024-09-10T23:37:29.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba36000640 1 -- 172.21.15.94:0/361755416 shutdown_connections 2024-09-10T23:37:29.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba36000640 1 --2- 172.21.15.94:0/361755416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba30070150 0x7fba30072610 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:29.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba36000640 1 -- 172.21.15.94:0/361755416 >> 172.21.15.94:0/361755416 conn(0x7fba3006cdf0 msgr2=0x7fba3006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:29.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba36000640 1 -- 172.21.15.94:0/361755416 shutdown_connections 2024-09-10T23:37:29.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba36000640 1 -- 172.21.15.94:0/361755416 wait complete. 2024-09-10T23:37:29.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba36000640 1 Processor -- start 2024-09-10T23:37:29.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba36000640 1 -- start start 2024-09-10T23:37:29.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba36000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba3006f600 0x7fba300cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:29.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba36000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba30070150 0x7fba3015bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:29.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba36000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba30157c60 0x7fba3015c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:29.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba36000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fba30168380 con 0x7fba30070150 2024-09-10T23:37:29.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba36000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fba30168200 con 0x7fba3006f600 2024-09-10T23:37:29.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba36000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fba30168500 con 0x7fba30157c60 2024-09-10T23:37:29.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba2ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba30070150 0x7fba3015bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:29.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba2ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba30070150 0x7fba3015bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60630/0 (socket says 172.21.15.94:60630) 2024-09-10T23:37:29.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba2ea00640 1 -- 172.21.15.94:0/2327880367 learned_addr learned my addr 172.21.15.94:0/2327880367 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:29.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba2f400640 1 --2- 172.21.15.94:0/2327880367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba3006f600 0x7fba300cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.757+0000 7fba2fe00640 1 --2- 172.21.15.94:0/2327880367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba30157c60 0x7fba3015c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba2ea00640 1 -- 172.21.15.94:0/2327880367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba30157c60 msgr2=0x7fba3015c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba2ea00640 1 --2- 172.21.15.94:0/2327880367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba30157c60 0x7fba3015c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba2ea00640 1 -- 172.21.15.94:0/2327880367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba3006f600 msgr2=0x7fba300cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba2ea00640 1 --2- 172.21.15.94:0/2327880367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba3006f600 0x7fba300cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba2ea00640 1 -- 172.21.15.94:0/2327880367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3014fae0 con 0x7fba30070150 2024-09-10T23:37:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba2fe00640 1 --2- 172.21.15.94:0/2327880367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba30157c60 0x7fba3015c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba2f400640 1 --2- 172.21.15.94:0/2327880367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba3006f600 0x7fba300cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba2ea00640 1 --2- 172.21.15.94:0/2327880367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba30070150 0x7fba3015bc10 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00c480 tx=0x7fba1c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba27400640 1 -- 172.21.15.94:0/2327880367 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0052d0 con 0x7fba30070150 2024-09-10T23:37:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba27400640 1 -- 172.21.15.94:0/2327880367 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba1c005470 con 0x7fba30070150 2024-09-10T23:37:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba36000640 1 -- 172.21.15.94:0/2327880367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3014f800 con 0x7fba30070150 2024-09-10T23:37:29.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba27400640 1 -- 172.21.15.94:0/2327880367 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c012870 con 0x7fba30070150 2024-09-10T23:37:29.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba36000640 1 -- 172.21.15.94:0/2327880367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3014dc40 con 0x7fba30070150 2024-09-10T23:37:29.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba36000640 1 -- 172.21.15.94:0/2327880367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba30072610 con 0x7fba30070150 2024-09-10T23:37:29.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba27400640 1 -- 172.21.15.94:0/2327880367 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fba1c003600 con 0x7fba30070150 2024-09-10T23:37:29.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba27400640 1 --2- 172.21.15.94:0/2327880367 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fba10072700 0x7fba10074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:29.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.761+0000 7fba27400640 1 -- 172.21.15.94:0/2327880367 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7fba1c093010 con 0x7fba30070150 2024-09-10T23:37:29.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.765+0000 7fba2f400640 1 --2- 172.21.15.94:0/2327880367 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fba10072700 0x7fba10074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:29.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.765+0000 7fba2f400640 1 --2- 172.21.15.94:0/2327880367 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fba10072700 0x7fba10074bc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fba300ce8d0 tx=0x7fba20009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:29.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.765+0000 7fba27400640 1 -- 172.21.15.94:0/2327880367 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fba1c05e560 con 0x7fba30070150 2024-09-10T23:37:29.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.969+0000 7fba36000640 1 -- 172.21.15.94:0/2327880367 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba3006e7e0 con 0x7fba30070150 2024-09-10T23:37:29.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.969+0000 7fba27400640 1 -- 172.21.15.94:0/2327880367 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5243 (secure 0 0 0) 0x7fba1c05de60 con 0x7fba30070150 2024-09-10T23:37:29.971 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:29.971 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":68,"btime":"2024-09-10T23:37:29:587275+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7011,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1105858824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1105858824},{"type":"v1","addr":"172.21.15.94:6834","nonce":1105858824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3277955689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3277955689},{"type":"v1","addr":"172.21.15.94:6835","nonce":3277955689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":195947,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/2607191387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2607191387},{"type":"v1","addr":"172.21.15.181:6801","nonce":2607191387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:37:25.941783+0000","modified":"2024-09-10T23:37:28.978060+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195950},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195950":{"gid":195950,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/2513770511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2513770511},{"type":"v1","addr":"172.21.15.181:6835","nonce":2513770511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195950,"qdb_cluster":[195950]},"id":8}]} 2024-09-10T23:37:29.971 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 68 2024-09-10T23:37:29.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.969+0000 7fba36000640 1 -- 172.21.15.94:0/2327880367 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fba10072700 msgr2=0x7fba10074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:29.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.969+0000 7fba36000640 1 --2- 172.21.15.94:0/2327880367 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fba10072700 0x7fba10074bc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fba300ce8d0 tx=0x7fba20009040 comp rx=0 tx=0).stop 2024-09-10T23:37:29.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.969+0000 7fba36000640 1 -- 172.21.15.94:0/2327880367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba30070150 msgr2=0x7fba3015bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:29.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.969+0000 7fba36000640 1 --2- 172.21.15.94:0/2327880367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba30070150 0x7fba3015bc10 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00c480 tx=0x7fba1c00c940 comp rx=0 tx=0).stop 2024-09-10T23:37:29.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.969+0000 7fba36000640 1 -- 172.21.15.94:0/2327880367 shutdown_connections 2024-09-10T23:37:29.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.969+0000 7fba36000640 1 --2- 172.21.15.94:0/2327880367 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fba10072700 0x7fba10074bc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:29.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.969+0000 7fba36000640 1 --2- 172.21.15.94:0/2327880367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba30157c60 0x7fba3015c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:29.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.969+0000 7fba36000640 1 --2- 172.21.15.94:0/2327880367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba30070150 0x7fba3015bc10 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:29.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.969+0000 7fba36000640 1 --2- 172.21.15.94:0/2327880367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba3006f600 0x7fba300cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:29.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.969+0000 7fba36000640 1 -- 172.21.15.94:0/2327880367 >> 172.21.15.94:0/2327880367 conn(0x7fba3006cdf0 msgr2=0x7fba3006eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:29.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.969+0000 7fba36000640 1 -- 172.21.15.94:0/2327880367 shutdown_connections 2024-09-10T23:37:29.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:29.969+0000 7fba36000640 1 -- 172.21.15.94:0/2327880367 wait complete. 2024-09-10T23:37:29.993 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:37:30.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.133+0000 7f24eb000640 1 -- 172.21.15.94:0/3649822737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e40af310 msgr2=0x7f24e40af7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:30.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.133+0000 7f24eb000640 1 --2- 172.21.15.94:0/3649822737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e40af310 0x7f24e40af7c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009900 tx=0x7f24dc01c7c0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.133+0000 7f24eb000640 1 -- 172.21.15.94:0/3649822737 shutdown_connections 2024-09-10T23:37:30.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.133+0000 7f24eb000640 1 --2- 172.21.15.94:0/3649822737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e40af310 0x7f24e40af7c0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.133+0000 7f24eb000640 1 --2- 172.21.15.94:0/3649822737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24e40b2bb0 0x7f24e40d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.133+0000 7f24eb000640 1 -- 172.21.15.94:0/3649822737 >> 172.21.15.94:0/3649822737 conn(0x7f24e40d1c60 msgr2=0x7f24e40d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:30.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.133+0000 7f24eb000640 1 -- 172.21.15.94:0/3649822737 shutdown_connections 2024-09-10T23:37:30.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.133+0000 7f24eb000640 1 -- 172.21.15.94:0/3649822737 wait complete. 2024-09-10T23:37:30.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24eb000640 1 Processor -- start 2024-09-10T23:37:30.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24eb000640 1 -- start start 2024-09-10T23:37:30.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24eb000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e40af310 0x7f24e415cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:30.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24eb000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24e40b2bb0 0x7f24e40c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24eb000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24e415f5c0 0x7f24e4151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24eb000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41690c0 con 0x7f24e40af310 2024-09-10T23:37:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24eb000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f24e4168f40 con 0x7f24e415f5c0 2024-09-10T23:37:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24eb000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f24e4169240 con 0x7f24e40b2bb0 2024-09-10T23:37:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24e3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e40af310 0x7f24e415cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24e3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e40af310 0x7f24e415cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60652/0 (socket says 172.21.15.94:60652) 2024-09-10T23:37:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24e3e00640 1 -- 172.21.15.94:0/168317960 learned_addr learned my addr 172.21.15.94:0/168317960 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:30.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24e9000640 1 --2- 172.21.15.94:0/168317960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24e415f5c0 0x7f24e4151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:30.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24db400640 1 --2- 172.21.15.94:0/168317960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24e40b2bb0 0x7f24e40c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:30.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24e3e00640 1 -- 172.21.15.94:0/168317960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24e40b2bb0 msgr2=0x7f24e40c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:30.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24e3e00640 1 --2- 172.21.15.94:0/168317960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24e40b2bb0 0x7f24e40c4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24e3e00640 1 -- 172.21.15.94:0/168317960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24e415f5c0 msgr2=0x7f24e4151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:30.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24e3e00640 1 --2- 172.21.15.94:0/168317960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24e415f5c0 0x7f24e4151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24e3e00640 1 -- 172.21.15.94:0/168317960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e414ef60 con 0x7f24e40af310 2024-09-10T23:37:30.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24e3e00640 1 --2- 172.21.15.94:0/168317960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e40af310 0x7f24e415cba0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f24d400ba50 tx=0x7f24d400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:30.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24e0c00640 1 -- 172.21.15.94:0/168317960 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4019070 con 0x7f24e40af310 2024-09-10T23:37:30.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24e0c00640 1 -- 172.21.15.94:0/168317960 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24d400cd10 con 0x7f24e40af310 2024-09-10T23:37:30.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24eb000640 1 -- 172.21.15.94:0/168317960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e414f240 con 0x7f24e40af310 2024-09-10T23:37:30.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24e0c00640 1 -- 172.21.15.94:0/168317960 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d40146a0 con 0x7f24e40af310 2024-09-10T23:37:30.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.137+0000 7f24eb000640 1 -- 172.21.15.94:0/168317960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e41529a0 con 0x7f24e40af310 2024-09-10T23:37:30.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.141+0000 7f24eb000640 1 -- 172.21.15.94:0/168317960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e40af7c0 con 0x7f24e40af310 2024-09-10T23:37:30.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.141+0000 7f24e0c00640 1 -- 172.21.15.94:0/168317960 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f24d4014840 con 0x7f24e40af310 2024-09-10T23:37:30.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.141+0000 7f24e0c00640 1 --2- 172.21.15.94:0/168317960 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f24c0072700 0x7f24c0074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:30.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.141+0000 7f24e0c00640 1 -- 172.21.15.94:0/168317960 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f24d4093530 con 0x7f24e40af310 2024-09-10T23:37:30.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.141+0000 7f24db400640 1 --2- 172.21.15.94:0/168317960 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f24c0072700 0x7f24c0074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:30.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.141+0000 7f24db400640 1 --2- 172.21.15.94:0/168317960 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f24c0072700 0x7f24c0074bc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f24e415a3a0 tx=0x7f24cc00b390 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:30.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.145+0000 7f24e0c00640 1 -- 172.21.15.94:0/168317960 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f24d405e300 con 0x7f24e40af310 2024-09-10T23:37:30.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.313+0000 7f24eb000640 1 -- 172.21.15.94:0/168317960 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24e414d5d0 con 0x7f24e40af310 2024-09-10T23:37:30.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.313+0000 7f24e0c00640 1 -- 172.21.15.94:0/168317960 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16720 (secure 0 0 0) 0x7f24d4062d30 con 0x7f24e40af310 2024-09-10T23:37:30.317 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:30.317 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":99,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:37:27.568506+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:37:23.632549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T23:37:24.916517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:37:30.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.317+0000 7f24eb000640 1 -- 172.21.15.94:0/168317960 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f24c0072700 msgr2=0x7f24c0074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:30.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.317+0000 7f24eb000640 1 --2- 172.21.15.94:0/168317960 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f24c0072700 0x7f24c0074bc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f24e415a3a0 tx=0x7f24cc00b390 comp rx=0 tx=0).stop 2024-09-10T23:37:30.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.317+0000 7f24eb000640 1 -- 172.21.15.94:0/168317960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e40af310 msgr2=0x7f24e415cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:30.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.317+0000 7f24eb000640 1 --2- 172.21.15.94:0/168317960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e40af310 0x7f24e415cba0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f24d400ba50 tx=0x7f24d400bf10 comp rx=0 tx=0).stop 2024-09-10T23:37:30.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.317+0000 7f24eb000640 1 -- 172.21.15.94:0/168317960 shutdown_connections 2024-09-10T23:37:30.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.317+0000 7f24eb000640 1 --2- 172.21.15.94:0/168317960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24e415f5c0 0x7f24e4151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.317+0000 7f24eb000640 1 --2- 172.21.15.94:0/168317960 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f24c0072700 0x7f24c0074bc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.317+0000 7f24eb000640 1 --2- 172.21.15.94:0/168317960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24e40b2bb0 0x7f24e40c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.317+0000 7f24eb000640 1 --2- 172.21.15.94:0/168317960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e40af310 0x7f24e415cba0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.317+0000 7f24eb000640 1 -- 172.21.15.94:0/168317960 >> 172.21.15.94:0/168317960 conn(0x7f24e40d1c60 msgr2=0x7f24e415e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:30.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.317+0000 7f24eb000640 1 -- 172.21.15.94:0/168317960 shutdown_connections 2024-09-10T23:37:30.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.317+0000 7f24eb000640 1 -- 172.21.15.94:0/168317960 wait complete. 2024-09-10T23:37:30.339 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T23:37:30.339 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:37:30.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.477+0000 7f4463400640 1 -- 172.21.15.94:0/1289316451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f445c15f5c0 msgr2=0x7f445c0cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:30.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.477+0000 7f4463400640 1 --2- 172.21.15.94:0/1289316451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f445c15f5c0 0x7f445c0cfb60 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f445c15f9a0 tx=0x7f445001c7a0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.477+0000 7f4463400640 1 -- 172.21.15.94:0/1289316451 shutdown_connections 2024-09-10T23:37:30.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.477+0000 7f4463400640 1 --2- 172.21.15.94:0/1289316451 >> v2:172.21.15.181:3300/0 conn(0x7f445c0af310 0x7f445c0af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.477+0000 7f4463400640 1 --2- 172.21.15.94:0/1289316451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f445c15f5c0 0x7f445c0cfb60 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.477+0000 7f4463400640 1 -- 172.21.15.94:0/1289316451 >> 172.21.15.94:0/1289316451 conn(0x7f445c0d1c60 msgr2=0x7f445c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:30.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.477+0000 7f4463400640 1 -- 172.21.15.94:0/1289316451 shutdown_connections 2024-09-10T23:37:30.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.477+0000 7f4463400640 1 -- 172.21.15.94:0/1289316451 wait complete. 2024-09-10T23:37:30.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.477+0000 7f4463400640 1 Processor -- start 2024-09-10T23:37:30.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.477+0000 7f4463400640 1 -- start start 2024-09-10T23:37:30.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f4463400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f445c0af310 0x7f445c0b6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:30.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f4463400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f445c0b2bb0 0x7f445c1f8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:30.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f4463400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f445c15f5c0 0x7f445c0b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:30.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f4463400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f445c168fa0 con 0x7f445c0b2bb0 2024-09-10T23:37:30.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f4463400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f445c168e20 con 0x7f445c0af310 2024-09-10T23:37:30.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f4463400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f445c169120 con 0x7f445c15f5c0 2024-09-10T23:37:30.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f445be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f445c0b2bb0 0x7f445c1f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:30.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f445be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f445c0b2bb0 0x7f445c1f8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60666/0 (socket says 172.21.15.94:60666) 2024-09-10T23:37:30.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f445be00640 1 -- 172.21.15.94:0/3893728347 learned_addr learned my addr 172.21.15.94:0/3893728347 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:30.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f4460a00640 1 --2- 172.21.15.94:0/3893728347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f445c0af310 0x7f445c0b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:30.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f4461400640 1 --2- 172.21.15.94:0/3893728347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f445c15f5c0 0x7f445c0b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:30.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f445be00640 1 -- 172.21.15.94:0/3893728347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f445c15f5c0 msgr2=0x7f445c0b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:30.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f445be00640 1 --2- 172.21.15.94:0/3893728347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f445c15f5c0 0x7f445c0b7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f445be00640 1 -- 172.21.15.94:0/3893728347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f445c0af310 msgr2=0x7f445c0b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:30.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f445be00640 1 --2- 172.21.15.94:0/3893728347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f445c0af310 0x7f445c0b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f445be00640 1 -- 172.21.15.94:0/3893728347 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445c0b77b0 con 0x7f445c0b2bb0 2024-09-10T23:37:30.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f445be00640 1 --2- 172.21.15.94:0/3893728347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f445c0b2bb0 0x7f445c1f8960 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f444800c450 tx=0x7f444800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:30.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f4458c00640 1 -- 172.21.15.94:0/3893728347 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448005480 con 0x7f445c0b2bb0 2024-09-10T23:37:30.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f4458c00640 1 -- 172.21.15.94:0/3893728347 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4448003600 con 0x7f445c0b2bb0 2024-09-10T23:37:30.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f4463400640 1 -- 172.21.15.94:0/3893728347 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445c0b5860 con 0x7f445c0b2bb0 2024-09-10T23:37:30.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f4458c00640 1 -- 172.21.15.94:0/3893728347 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448003a30 con 0x7f445c0b2bb0 2024-09-10T23:37:30.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f4463400640 1 -- 172.21.15.94:0/3893728347 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f445c0b5e20 con 0x7f445c0b2bb0 2024-09-10T23:37:30.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.481+0000 7f4463400640 1 -- 172.21.15.94:0/3893728347 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f445c0af7a0 con 0x7f445c0b2bb0 2024-09-10T23:37:30.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.485+0000 7f4458c00640 1 -- 172.21.15.94:0/3893728347 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f4448003bd0 con 0x7f445c0b2bb0 2024-09-10T23:37:30.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.485+0000 7f4458c00640 1 --2- 172.21.15.94:0/3893728347 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f443c072700 0x7f443c074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:30.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.485+0000 7f4458c00640 1 -- 172.21.15.94:0/3893728347 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f44480930d0 con 0x7f445c0b2bb0 2024-09-10T23:37:30.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.485+0000 7f4460a00640 1 --2- 172.21.15.94:0/3893728347 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f443c072700 0x7f443c074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:30.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.485+0000 7f4460a00640 1 --2- 172.21.15.94:0/3893728347 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f443c072700 0x7f443c074bc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f44500af000 tx=0x7f445001afe0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:30.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.489+0000 7f4458c00640 1 -- 172.21.15.94:0/3893728347 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f444805e4f0 con 0x7f445c0b2bb0 2024-09-10T23:37:30.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.653+0000 7f4463400640 1 -- 172.21.15.94:0/3893728347 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f445c09fc70 con 0x7f443c072700 2024-09-10T23:37:30.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.657+0000 7f4458c00640 1 -- 172.21.15.94:0/3893728347 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252065 (secure 0 0 0) 0x7f445c09fc70 con 0x7f443c072700 2024-09-10T23:37:30.662 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:37:30.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.661+0000 7f4463400640 1 -- 172.21.15.94:0/3893728347 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f443c072700 msgr2=0x7f443c074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:30.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.661+0000 7f4463400640 1 --2- 172.21.15.94:0/3893728347 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f443c072700 0x7f443c074bc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f44500af000 tx=0x7f445001afe0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.661+0000 7f4463400640 1 -- 172.21.15.94:0/3893728347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f445c0b2bb0 msgr2=0x7f445c1f8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:30.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.661+0000 7f4463400640 1 --2- 172.21.15.94:0/3893728347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f445c0b2bb0 0x7f445c1f8960 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f444800c450 tx=0x7f444800c910 comp rx=0 tx=0).stop 2024-09-10T23:37:30.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.661+0000 7f4463400640 1 -- 172.21.15.94:0/3893728347 shutdown_connections 2024-09-10T23:37:30.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.661+0000 7f4463400640 1 --2- 172.21.15.94:0/3893728347 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f443c072700 0x7f443c074bc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.661+0000 7f4463400640 1 --2- 172.21.15.94:0/3893728347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f445c15f5c0 0x7f445c0b7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.661+0000 7f4463400640 1 --2- 172.21.15.94:0/3893728347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f445c0b2bb0 0x7f445c1f8960 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.661+0000 7f4463400640 1 --2- 172.21.15.94:0/3893728347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f445c0af310 0x7f445c0b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.661+0000 7f4463400640 1 -- 172.21.15.94:0/3893728347 >> 172.21.15.94:0/3893728347 conn(0x7f445c0d1c60 msgr2=0x7f445c0b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:30.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.661+0000 7f4463400640 1 -- 172.21.15.94:0/3893728347 shutdown_connections 2024-09-10T23:37:30.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.661+0000 7f4463400640 1 -- 172.21.15.94:0/3893728347 wait complete. 2024-09-10T23:37:30.683 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-09-10T23:37:28.745340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260232,"kb_used_data":10696,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714488,"statfs":{"total":773094113280,"available":772827635712,"internally_reserved":0,"allocated":10952704,"data_stored":5396100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.204428"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583842+0000","last_change":"2024-09-10T23:37:26.572290+0000","last_active":"2024-09-10T23:37:27.583842+0000","last_peered":"2024-09-10T23:37:27.583842+0000","last_clean":"2024-09-10T23:37:27.583842+0000","last_became_active":"2024-09-10T23:37:26.572038+0000","last_became_peered":"2024-09-10T23:37:26.572038+0000","last_unstale":"2024-09-10T23:37:27.583842+0000","last_undegraded":"2024-09-10T23:37:27.583842+0000","last_fullsized":"2024-09-10T23:37:27.583842+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:01.439200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.574323+0000","last_change":"2024-09-10T23:37:26.568617+0000","last_active":"2024-09-10T23:37:26.574323+0000","last_peered":"2024-09-10T23:37:26.574323+0000","last_clean":"2024-09-10T23:37:26.574323+0000","last_became_active":"2024-09-10T23:37:26.568204+0000","last_became_peered":"2024-09-10T23:37:26.568204+0000","last_unstale":"2024-09-10T23:37:26.574323+0000","last_undegraded":"2024-09-10T23:37:26.574323+0000","last_fullsized":"2024-09-10T23:37:26.574323+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:59:34.242358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.573067+0000","last_change":"2024-09-10T23:37:26.569482+0000","last_active":"2024-09-10T23:37:26.573067+0000","last_peered":"2024-09-10T23:37:26.573067+0000","last_clean":"2024-09-10T23:37:26.573067+0000","last_became_active":"2024-09-10T23:37:26.569003+0000","last_became_peered":"2024-09-10T23:37:26.569003+0000","last_unstale":"2024-09-10T23:37:26.573067+0000","last_undegraded":"2024-09-10T23:37:26.573067+0000","last_fullsized":"2024-09-10T23:37:26.573067+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:34:54.359719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583807+0000","last_change":"2024-09-10T23:37:26.571109+0000","last_active":"2024-09-10T23:37:27.583807+0000","last_peered":"2024-09-10T23:37:27.583807+0000","last_clean":"2024-09-10T23:37:27.583807+0000","last_became_active":"2024-09-10T23:37:26.570655+0000","last_became_peered":"2024-09-10T23:37:26.570655+0000","last_unstale":"2024-09-10T23:37:27.583807+0000","last_undegraded":"2024-09-10T23:37:27.583807+0000","last_fullsized":"2024-09-10T23:37:27.583807+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:04.968567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.597178+0000","last_change":"2024-09-10T23:37:26.592064+0000","last_active":"2024-09-10T23:37:26.597178+0000","last_peered":"2024-09-10T23:37:26.597178+0000","last_clean":"2024-09-10T23:37:26.597178+0000","last_became_active":"2024-09-10T23:37:26.591737+0000","last_became_peered":"2024-09-10T23:37:26.591737+0000","last_unstale":"2024-09-10T23:37:26.597178+0000","last_undegraded":"2024-09-10T23:37:26.597178+0000","last_fullsized":"2024-09-10T23:37:26.597178+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:09.768476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.575505+0000","last_change":"2024-09-10T23:37:26.570231+0000","last_active":"2024-09-10T23:37:26.575505+0000","last_peered":"2024-09-10T23:37:26.575505+0000","last_clean":"2024-09-10T23:37:26.575505+0000","last_became_active":"2024-09-10T23:37:26.569922+0000","last_became_peered":"2024-09-10T23:37:26.569922+0000","last_unstale":"2024-09-10T23:37:26.575505+0000","last_undegraded":"2024-09-10T23:37:26.575505+0000","last_fullsized":"2024-09-10T23:37:26.575505+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:07.013347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578269+0000","last_change":"2024-09-10T23:37:26.570469+0000","last_active":"2024-09-10T23:37:27.578269+0000","last_peered":"2024-09-10T23:37:27.578269+0000","last_clean":"2024-09-10T23:37:27.578269+0000","last_became_active":"2024-09-10T23:37:26.570161+0000","last_became_peered":"2024-09-10T23:37:26.570161+0000","last_unstale":"2024-09-10T23:37:27.578269+0000","last_undegraded":"2024-09-10T23:37:27.578269+0000","last_fullsized":"2024-09-10T23:37:27.578269+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:31:31.064875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578272+0000","last_change":"2024-09-10T23:37:26.572452+0000","last_active":"2024-09-10T23:37:27.578272+0000","last_peered":"2024-09-10T23:37:27.578272+0000","last_clean":"2024-09-10T23:37:27.578272+0000","last_became_active":"2024-09-10T23:37:26.572163+0000","last_became_peered":"2024-09-10T23:37:26.572163+0000","last_unstale":"2024-09-10T23:37:27.578272+0000","last_undegraded":"2024-09-10T23:37:27.578272+0000","last_fullsized":"2024-09-10T23:37:27.578272+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:45.392321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584096+0000","last_change":"2024-09-10T23:37:26.583435+0000","last_active":"2024-09-10T23:37:27.584096+0000","last_peered":"2024-09-10T23:37:27.584096+0000","last_clean":"2024-09-10T23:37:27.584096+0000","last_became_active":"2024-09-10T23:37:26.582937+0000","last_became_peered":"2024-09-10T23:37:26.582937+0000","last_unstale":"2024-09-10T23:37:27.584096+0000","last_undegraded":"2024-09-10T23:37:27.584096+0000","last_fullsized":"2024-09-10T23:37:27.584096+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:53.799014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.597294+0000","last_change":"2024-09-10T23:37:26.593305+0000","last_active":"2024-09-10T23:37:26.597294+0000","last_peered":"2024-09-10T23:37:26.597294+0000","last_clean":"2024-09-10T23:37:26.597294+0000","last_became_active":"2024-09-10T23:37:26.592924+0000","last_became_peered":"2024-09-10T23:37:26.592924+0000","last_unstale":"2024-09-10T23:37:26.597294+0000","last_undegraded":"2024-09-10T23:37:26.597294+0000","last_fullsized":"2024-09-10T23:37:26.597294+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:47.479672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578820+0000","last_change":"2024-09-10T23:37:26.572828+0000","last_active":"2024-09-10T23:37:27.578820+0000","last_peered":"2024-09-10T23:37:27.578820+0000","last_clean":"2024-09-10T23:37:27.578820+0000","last_became_active":"2024-09-10T23:37:26.572496+0000","last_became_peered":"2024-09-10T23:37:26.572496+0000","last_unstale":"2024-09-10T23:37:27.578820+0000","last_undegraded":"2024-09-10T23:37:27.578820+0000","last_fullsized":"2024-09-10T23:37:27.578820+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:28:23.356135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.599082+0000","last_change":"2024-09-10T23:37:26.597675+0000","last_active":"2024-09-10T23:37:26.599082+0000","last_peered":"2024-09-10T23:37:26.599082+0000","last_clean":"2024-09-10T23:37:26.599082+0000","last_became_active":"2024-09-10T23:37:26.597286+0000","last_became_peered":"2024-09-10T23:37:26.597286+0000","last_unstale":"2024-09-10T23:37:26.599082+0000","last_undegraded":"2024-09-10T23:37:26.599082+0000","last_fullsized":"2024-09-10T23:37:26.599082+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:54.910181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.594984+0000","last_change":"2024-09-10T23:37:26.593290+0000","last_active":"2024-09-10T23:37:26.594984+0000","last_peered":"2024-09-10T23:37:26.594984+0000","last_clean":"2024-09-10T23:37:26.594984+0000","last_became_active":"2024-09-10T23:37:26.592808+0000","last_became_peered":"2024-09-10T23:37:26.592808+0000","last_unstale":"2024-09-10T23:37:26.594984+0000","last_undegraded":"2024-09-10T23:37:26.594984+0000","last_fullsized":"2024-09-10T23:37:26.594984+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:08.150387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.573943+0000","last_change":"2024-09-10T23:37:26.571006+0000","last_active":"2024-09-10T23:37:26.573943+0000","last_peered":"2024-09-10T23:37:26.573943+0000","last_clean":"2024-09-10T23:37:26.573943+0000","last_became_active":"2024-09-10T23:37:26.570597+0000","last_became_peered":"2024-09-10T23:37:26.570597+0000","last_unstale":"2024-09-10T23:37:26.573943+0000","last_undegraded":"2024-09-10T23:37:26.573943+0000","last_fullsized":"2024-09-10T23:37:26.573943+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:49.340330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.582932+0000","last_change":"2024-09-10T23:37:26.590703+0000","last_active":"2024-09-10T23:37:27.582932+0000","last_peered":"2024-09-10T23:37:27.582932+0000","last_clean":"2024-09-10T23:37:27.582932+0000","last_became_active":"2024-09-10T23:37:26.590424+0000","last_became_peered":"2024-09-10T23:37:26.590424+0000","last_unstale":"2024-09-10T23:37:27.582932+0000","last_undegraded":"2024-09-10T23:37:27.582932+0000","last_fullsized":"2024-09-10T23:37:27.582932+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:06.059219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.591998+0000","last_change":"2024-09-10T23:37:26.589407+0000","last_active":"2024-09-10T23:37:26.591998+0000","last_peered":"2024-09-10T23:37:26.591998+0000","last_clean":"2024-09-10T23:37:26.591998+0000","last_became_active":"2024-09-10T23:37:26.587702+0000","last_became_peered":"2024-09-10T23:37:26.587702+0000","last_unstale":"2024-09-10T23:37:26.591998+0000","last_undegraded":"2024-09-10T23:37:26.591998+0000","last_fullsized":"2024-09-10T23:37:26.591998+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:19.135723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.569844+0000","last_change":"2024-09-10T23:37:26.565785+0000","last_active":"2024-09-10T23:37:26.569844+0000","last_peered":"2024-09-10T23:37:26.569844+0000","last_clean":"2024-09-10T23:37:26.569844+0000","last_became_active":"2024-09-10T23:37:26.564340+0000","last_became_peered":"2024-09-10T23:37:26.564340+0000","last_unstale":"2024-09-10T23:37:26.569844+0000","last_undegraded":"2024-09-10T23:37:26.569844+0000","last_fullsized":"2024-09-10T23:37:26.569844+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:43.341222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578850+0000","last_change":"2024-09-10T23:37:26.574009+0000","last_active":"2024-09-10T23:37:27.578850+0000","last_peered":"2024-09-10T23:37:27.578850+0000","last_clean":"2024-09-10T23:37:27.578850+0000","last_became_active":"2024-09-10T23:37:26.573740+0000","last_became_peered":"2024-09-10T23:37:26.573740+0000","last_unstale":"2024-09-10T23:37:27.578850+0000","last_undegraded":"2024-09-10T23:37:27.578850+0000","last_fullsized":"2024-09-10T23:37:27.578850+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:19.321987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.592286+0000","last_change":"2024-09-10T23:37:26.591258+0000","last_active":"2024-09-10T23:37:26.592286+0000","last_peered":"2024-09-10T23:37:26.592286+0000","last_clean":"2024-09-10T23:37:26.592286+0000","last_became_active":"2024-09-10T23:37:26.590931+0000","last_became_peered":"2024-09-10T23:37:26.590931+0000","last_unstale":"2024-09-10T23:37:26.592286+0000","last_undegraded":"2024-09-10T23:37:26.592286+0000","last_fullsized":"2024-09-10T23:37:26.592286+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:08.290602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.571217+0000","last_change":"2024-09-10T23:37:26.566788+0000","last_active":"2024-09-10T23:37:26.571217+0000","last_peered":"2024-09-10T23:37:26.571217+0000","last_clean":"2024-09-10T23:37:26.571217+0000","last_became_active":"2024-09-10T23:37:26.566537+0000","last_became_peered":"2024-09-10T23:37:26.566537+0000","last_unstale":"2024-09-10T23:37:26.571217+0000","last_undegraded":"2024-09-10T23:37:26.571217+0000","last_fullsized":"2024-09-10T23:37:26.571217+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:54.910181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578559+0000","last_change":"2024-09-10T23:37:26.572733+0000","last_active":"2024-09-10T23:37:27.578559+0000","last_peered":"2024-09-10T23:37:27.578559+0000","last_clean":"2024-09-10T23:37:27.578559+0000","last_became_active":"2024-09-10T23:37:26.569327+0000","last_became_peered":"2024-09-10T23:37:26.569327+0000","last_unstale":"2024-09-10T23:37:27.578559+0000","last_undegraded":"2024-09-10T23:37:27.578559+0000","last_fullsized":"2024-09-10T23:37:27.578559+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:52.847810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.575481+0000","last_change":"2024-09-10T23:37:26.569472+0000","last_active":"2024-09-10T23:37:26.575481+0000","last_peered":"2024-09-10T23:37:26.575481+0000","last_clean":"2024-09-10T23:37:26.575481+0000","last_became_active":"2024-09-10T23:37:26.569145+0000","last_became_peered":"2024-09-10T23:37:26.569145+0000","last_unstale":"2024-09-10T23:37:26.575481+0000","last_undegraded":"2024-09-10T23:37:26.575481+0000","last_fullsized":"2024-09-10T23:37:26.575481+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:23.851300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.574044+0000","last_change":"2024-09-10T23:37:26.571874+0000","last_active":"2024-09-10T23:37:26.574044+0000","last_peered":"2024-09-10T23:37:26.574044+0000","last_clean":"2024-09-10T23:37:26.574044+0000","last_became_active":"2024-09-10T23:37:26.571523+0000","last_became_peered":"2024-09-10T23:37:26.571523+0000","last_unstale":"2024-09-10T23:37:26.574044+0000","last_undegraded":"2024-09-10T23:37:26.574044+0000","last_fullsized":"2024-09-10T23:37:26.574044+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:53.386615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.582146+0000","last_change":"2024-09-10T23:37:26.569387+0000","last_active":"2024-09-10T23:37:27.582146+0000","last_peered":"2024-09-10T23:37:27.582146+0000","last_clean":"2024-09-10T23:37:27.582146+0000","last_became_active":"2024-09-10T23:37:26.568874+0000","last_became_peered":"2024-09-10T23:37:26.568874+0000","last_unstale":"2024-09-10T23:37:27.582146+0000","last_undegraded":"2024-09-10T23:37:27.582146+0000","last_fullsized":"2024-09-10T23:37:27.582146+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:15:06.855063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578875+0000","last_change":"2024-09-10T23:37:26.572047+0000","last_active":"2024-09-10T23:37:27.578875+0000","last_peered":"2024-09-10T23:37:27.578875+0000","last_clean":"2024-09-10T23:37:27.578875+0000","last_became_active":"2024-09-10T23:37:26.571800+0000","last_became_peered":"2024-09-10T23:37:26.571800+0000","last_unstale":"2024-09-10T23:37:27.578875+0000","last_undegraded":"2024-09-10T23:37:27.578875+0000","last_fullsized":"2024-09-10T23:37:27.578875+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:52.936434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.585742+0000","last_change":"2024-09-10T23:37:26.574974+0000","last_active":"2024-09-10T23:37:26.585742+0000","last_peered":"2024-09-10T23:37:26.585742+0000","last_clean":"2024-09-10T23:37:26.585742+0000","last_became_active":"2024-09-10T23:37:26.574632+0000","last_became_peered":"2024-09-10T23:37:26.574632+0000","last_unstale":"2024-09-10T23:37:26.585742+0000","last_undegraded":"2024-09-10T23:37:26.585742+0000","last_fullsized":"2024-09-10T23:37:26.585742+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:15:06.855063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584327+0000","last_change":"2024-09-10T23:37:26.570460+0000","last_active":"2024-09-10T23:37:27.584327+0000","last_peered":"2024-09-10T23:37:27.584327+0000","last_clean":"2024-09-10T23:37:27.584327+0000","last_became_active":"2024-09-10T23:37:26.570179+0000","last_became_peered":"2024-09-10T23:37:26.570179+0000","last_unstale":"2024-09-10T23:37:27.584327+0000","last_undegraded":"2024-09-10T23:37:27.584327+0000","last_fullsized":"2024-09-10T23:37:27.584327+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:35.652590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.586488+0000","last_change":"2024-09-10T23:37:26.576961+0000","last_active":"2024-09-10T23:37:26.586488+0000","last_peered":"2024-09-10T23:37:26.586488+0000","last_clean":"2024-09-10T23:37:26.586488+0000","last_became_active":"2024-09-10T23:37:26.576664+0000","last_became_peered":"2024-09-10T23:37:26.576664+0000","last_unstale":"2024-09-10T23:37:26.586488+0000","last_undegraded":"2024-09-10T23:37:26.586488+0000","last_fullsized":"2024-09-10T23:37:26.586488+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:33.373338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.573459+0000","last_change":"2024-09-10T23:37:26.570558+0000","last_active":"2024-09-10T23:37:26.573459+0000","last_peered":"2024-09-10T23:37:26.573459+0000","last_clean":"2024-09-10T23:37:26.573459+0000","last_became_active":"2024-09-10T23:37:26.570255+0000","last_became_peered":"2024-09-10T23:37:26.570255+0000","last_unstale":"2024-09-10T23:37:26.573459+0000","last_undegraded":"2024-09-10T23:37:26.573459+0000","last_fullsized":"2024-09-10T23:37:26.573459+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:13:59.361625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578443+0000","last_change":"2024-09-10T23:37:26.571146+0000","last_active":"2024-09-10T23:37:27.578443+0000","last_peered":"2024-09-10T23:37:27.578443+0000","last_clean":"2024-09-10T23:37:27.578443+0000","last_became_active":"2024-09-10T23:37:26.570830+0000","last_became_peered":"2024-09-10T23:37:26.570830+0000","last_unstale":"2024-09-10T23:37:27.578443+0000","last_undegraded":"2024-09-10T23:37:27.578443+0000","last_fullsized":"2024-09-10T23:37:27.578443+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:06.546726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583478+0000","last_change":"2024-09-10T23:37:26.572980+0000","last_active":"2024-09-10T23:37:27.583478+0000","last_peered":"2024-09-10T23:37:27.583478+0000","last_clean":"2024-09-10T23:37:27.583478+0000","last_became_active":"2024-09-10T23:37:26.572698+0000","last_became_peered":"2024-09-10T23:37:26.572698+0000","last_unstale":"2024-09-10T23:37:27.583478+0000","last_undegraded":"2024-09-10T23:37:27.583478+0000","last_fullsized":"2024-09-10T23:37:27.583478+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:04.358361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.592037+0000","last_change":"2024-09-10T23:37:26.589579+0000","last_active":"2024-09-10T23:37:26.592037+0000","last_peered":"2024-09-10T23:37:26.592037+0000","last_clean":"2024-09-10T23:37:26.592037+0000","last_became_active":"2024-09-10T23:37:26.589260+0000","last_became_peered":"2024-09-10T23:37:26.589260+0000","last_unstale":"2024-09-10T23:37:26.592037+0000","last_undegraded":"2024-09-10T23:37:26.592037+0000","last_fullsized":"2024-09-10T23:37:26.592037+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:30.790821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583682+0000","last_change":"2024-09-10T23:37:26.580274+0000","last_active":"2024-09-10T23:37:27.583682+0000","last_peered":"2024-09-10T23:37:27.583682+0000","last_clean":"2024-09-10T23:37:27.583682+0000","last_became_active":"2024-09-10T23:37:26.579920+0000","last_became_peered":"2024-09-10T23:37:26.579920+0000","last_unstale":"2024-09-10T23:37:27.583682+0000","last_undegraded":"2024-09-10T23:37:27.583682+0000","last_fullsized":"2024-09-10T23:37:27.583682+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:18.088525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"98'4","reported_seq":17,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.582617+0000","last_change":"2024-09-10T23:37:25.566145+0000","last_active":"2024-09-10T23:37:26.582617+0000","last_peered":"2024-09-10T23:37:26.582617+0000","last_clean":"2024-09-10T23:37:26.582617+0000","last_became_active":"2024-09-10T23:37:25.565826+0000","last_became_peered":"2024-09-10T23:37:25.565826+0000","last_unstale":"2024-09-10T23:37:26.582617+0000","last_undegraded":"2024-09-10T23:37:26.582617+0000","last_fullsized":"2024-09-10T23:37:26.582617+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:12.094932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.578010+0000","last_change":"2024-09-10T23:37:26.574542+0000","last_active":"2024-09-10T23:37:26.578010+0000","last_peered":"2024-09-10T23:37:26.578010+0000","last_clean":"2024-09-10T23:37:26.578010+0000","last_became_active":"2024-09-10T23:37:26.573984+0000","last_became_peered":"2024-09-10T23:37:26.573984+0000","last_unstale":"2024-09-10T23:37:26.578010+0000","last_undegraded":"2024-09-10T23:37:26.578010+0000","last_fullsized":"2024-09-10T23:37:26.578010+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:00.463133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"98'1","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.580854+0000","last_change":"2024-09-10T23:37:25.563742+0000","last_active":"2024-09-10T23:37:26.580854+0000","last_peered":"2024-09-10T23:37:26.580854+0000","last_clean":"2024-09-10T23:37:26.580854+0000","last_became_active":"2024-09-10T23:37:25.563167+0000","last_became_peered":"2024-09-10T23:37:25.563167+0000","last_unstale":"2024-09-10T23:37:26.580854+0000","last_undegraded":"2024-09-10T23:37:26.580854+0000","last_fullsized":"2024-09-10T23:37:26.580854+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:48:59.532474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.574942+0000","last_change":"2024-09-10T23:37:26.572840+0000","last_active":"2024-09-10T23:37:26.574942+0000","last_peered":"2024-09-10T23:37:26.574942+0000","last_clean":"2024-09-10T23:37:26.574942+0000","last_became_active":"2024-09-10T23:37:26.572565+0000","last_became_peered":"2024-09-10T23:37:26.572565+0000","last_unstale":"2024-09-10T23:37:26.574942+0000","last_undegraded":"2024-09-10T23:37:26.574942+0000","last_fullsized":"2024-09-10T23:37:26.574942+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:08.191727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"98'2","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583376+0000","last_change":"2024-09-10T23:37:25.573677+0000","last_active":"2024-09-10T23:37:27.583376+0000","last_peered":"2024-09-10T23:37:27.583376+0000","last_clean":"2024-09-10T23:37:27.583376+0000","last_became_active":"2024-09-10T23:37:25.573325+0000","last_became_peered":"2024-09-10T23:37:25.573325+0000","last_unstale":"2024-09-10T23:37:27.583376+0000","last_undegraded":"2024-09-10T23:37:27.583376+0000","last_fullsized":"2024-09-10T23:37:27.583376+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:21.697861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.589544+0000","last_change":"2024-09-10T23:37:26.589006+0000","last_active":"2024-09-10T23:37:26.589544+0000","last_peered":"2024-09-10T23:37:26.589544+0000","last_clean":"2024-09-10T23:37:26.589544+0000","last_became_active":"2024-09-10T23:37:26.588387+0000","last_became_peered":"2024-09-10T23:37:26.588387+0000","last_unstale":"2024-09-10T23:37:26.589544+0000","last_undegraded":"2024-09-10T23:37:26.589544+0000","last_fullsized":"2024-09-10T23:37:26.589544+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:50:17.880005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"98'2","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583976+0000","last_change":"2024-09-10T23:37:25.576105+0000","last_active":"2024-09-10T23:37:27.583976+0000","last_peered":"2024-09-10T23:37:27.583976+0000","last_clean":"2024-09-10T23:37:27.583976+0000","last_became_active":"2024-09-10T23:37:25.575750+0000","last_became_peered":"2024-09-10T23:37:25.575750+0000","last_unstale":"2024-09-10T23:37:27.583976+0000","last_undegraded":"2024-09-10T23:37:27.583976+0000","last_fullsized":"2024-09-10T23:37:27.583976+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:59:33.239484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.582842+0000","last_change":"2024-09-10T23:37:26.574146+0000","last_active":"2024-09-10T23:37:27.582842+0000","last_peered":"2024-09-10T23:37:27.582842+0000","last_clean":"2024-09-10T23:37:27.582842+0000","last_became_active":"2024-09-10T23:37:26.571996+0000","last_became_peered":"2024-09-10T23:37:26.571996+0000","last_unstale":"2024-09-10T23:37:27.582842+0000","last_undegraded":"2024-09-10T23:37:27.582842+0000","last_fullsized":"2024-09-10T23:37:27.582842+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:56.212613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"98'1","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.587513+0000","last_change":"2024-09-10T23:37:25.566909+0000","last_active":"2024-09-10T23:37:26.587513+0000","last_peered":"2024-09-10T23:37:26.587513+0000","last_clean":"2024-09-10T23:37:26.587513+0000","last_became_active":"2024-09-10T23:37:25.566558+0000","last_became_peered":"2024-09-10T23:37:25.566558+0000","last_unstale":"2024-09-10T23:37:26.587513+0000","last_undegraded":"2024-09-10T23:37:26.587513+0000","last_fullsized":"2024-09-10T23:37:26.587513+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:16.692007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578376+0000","last_change":"2024-09-10T23:37:26.592072+0000","last_active":"2024-09-10T23:37:27.578376+0000","last_peered":"2024-09-10T23:37:27.578376+0000","last_clean":"2024-09-10T23:37:27.578376+0000","last_became_active":"2024-09-10T23:37:26.591739+0000","last_became_peered":"2024-09-10T23:37:26.591739+0000","last_unstale":"2024-09-10T23:37:27.578376+0000","last_undegraded":"2024-09-10T23:37:27.578376+0000","last_fullsized":"2024-09-10T23:37:27.578376+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:02.820297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"98'1","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583164+0000","last_change":"2024-09-10T23:37:25.568577+0000","last_active":"2024-09-10T23:37:27.583164+0000","last_peered":"2024-09-10T23:37:27.583164+0000","last_clean":"2024-09-10T23:37:27.583164+0000","last_became_active":"2024-09-10T23:37:25.568032+0000","last_became_peered":"2024-09-10T23:37:25.568032+0000","last_unstale":"2024-09-10T23:37:27.583164+0000","last_undegraded":"2024-09-10T23:37:27.583164+0000","last_fullsized":"2024-09-10T23:37:27.583164+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:52.305088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.597271+0000","last_change":"2024-09-10T23:37:26.593029+0000","last_active":"2024-09-10T23:37:26.597271+0000","last_peered":"2024-09-10T23:37:26.597271+0000","last_clean":"2024-09-10T23:37:26.597271+0000","last_became_active":"2024-09-10T23:37:26.592707+0000","last_became_peered":"2024-09-10T23:37:26.592707+0000","last_unstale":"2024-09-10T23:37:26.597271+0000","last_undegraded":"2024-09-10T23:37:26.597271+0000","last_fullsized":"2024-09-10T23:37:26.597271+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:17.694882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578367+0000","last_change":"2024-09-10T23:37:25.574164+0000","last_active":"2024-09-10T23:37:27.578367+0000","last_peered":"2024-09-10T23:37:27.578367+0000","last_clean":"2024-09-10T23:37:27.578367+0000","last_became_active":"2024-09-10T23:37:25.573880+0000","last_became_peered":"2024-09-10T23:37:25.573880+0000","last_unstale":"2024-09-10T23:37:27.578367+0000","last_undegraded":"2024-09-10T23:37:27.578367+0000","last_fullsized":"2024-09-10T23:37:27.578367+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:46.195152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578454+0000","last_change":"2024-09-10T23:37:26.575378+0000","last_active":"2024-09-10T23:37:27.578454+0000","last_peered":"2024-09-10T23:37:27.578454+0000","last_clean":"2024-09-10T23:37:27.578454+0000","last_became_active":"2024-09-10T23:37:26.575087+0000","last_became_peered":"2024-09-10T23:37:26.575087+0000","last_unstale":"2024-09-10T23:37:27.578454+0000","last_undegraded":"2024-09-10T23:37:27.578454+0000","last_fullsized":"2024-09-10T23:37:27.578454+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:28.905331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.574867+0000","last_change":"2024-09-10T23:37:25.577564+0000","last_active":"2024-09-10T23:37:26.574867+0000","last_peered":"2024-09-10T23:37:26.574867+0000","last_clean":"2024-09-10T23:37:26.574867+0000","last_became_active":"2024-09-10T23:37:25.577232+0000","last_became_peered":"2024-09-10T23:37:25.577232+0000","last_unstale":"2024-09-10T23:37:26.574867+0000","last_undegraded":"2024-09-10T23:37:26.574867+0000","last_fullsized":"2024-09-10T23:37:26.574867+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:52.529265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.574299+0000","last_change":"2024-09-10T23:37:26.568480+0000","last_active":"2024-09-10T23:37:26.574299+0000","last_peered":"2024-09-10T23:37:26.574299+0000","last_clean":"2024-09-10T23:37:26.574299+0000","last_became_active":"2024-09-10T23:37:26.567873+0000","last_became_peered":"2024-09-10T23:37:26.567873+0000","last_unstale":"2024-09-10T23:37:26.574299+0000","last_undegraded":"2024-09-10T23:37:26.574299+0000","last_fullsized":"2024-09-10T23:37:26.574299+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:37.843703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584564+0000","last_change":"2024-09-10T23:37:25.576592+0000","last_active":"2024-09-10T23:37:27.584564+0000","last_peered":"2024-09-10T23:37:27.584564+0000","last_clean":"2024-09-10T23:37:27.584564+0000","last_became_active":"2024-09-10T23:37:25.575766+0000","last_became_peered":"2024-09-10T23:37:25.575766+0000","last_unstale":"2024-09-10T23:37:27.584564+0000","last_undegraded":"2024-09-10T23:37:27.584564+0000","last_fullsized":"2024-09-10T23:37:27.584564+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:28:17.590140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.594972+0000","last_change":"2024-09-10T23:37:26.593226+0000","last_active":"2024-09-10T23:37:26.594972+0000","last_peered":"2024-09-10T23:37:26.594972+0000","last_clean":"2024-09-10T23:37:26.594972+0000","last_became_active":"2024-09-10T23:37:26.592775+0000","last_became_peered":"2024-09-10T23:37:26.592775+0000","last_unstale":"2024-09-10T23:37:26.594972+0000","last_undegraded":"2024-09-10T23:37:26.594972+0000","last_fullsized":"2024-09-10T23:37:26.594972+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:59.908076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.580106+0000","last_change":"2024-09-10T23:37:25.577535+0000","last_active":"2024-09-10T23:37:26.580106+0000","last_peered":"2024-09-10T23:37:26.580106+0000","last_clean":"2024-09-10T23:37:26.580106+0000","last_became_active":"2024-09-10T23:37:25.577252+0000","last_became_peered":"2024-09-10T23:37:25.577252+0000","last_unstale":"2024-09-10T23:37:26.580106+0000","last_undegraded":"2024-09-10T23:37:26.580106+0000","last_fullsized":"2024-09-10T23:37:26.580106+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:39.232432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.574917+0000","last_change":"2024-09-10T23:37:26.572237+0000","last_active":"2024-09-10T23:37:26.574917+0000","last_peered":"2024-09-10T23:37:26.574917+0000","last_clean":"2024-09-10T23:37:26.574917+0000","last_became_active":"2024-09-10T23:37:26.571958+0000","last_became_peered":"2024-09-10T23:37:26.571958+0000","last_unstale":"2024-09-10T23:37:26.574917+0000","last_undegraded":"2024-09-10T23:37:26.574917+0000","last_fullsized":"2024-09-10T23:37:26.574917+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:26.017918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"98'3","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.598333+0000","last_change":"2024-09-10T23:37:25.580213+0000","last_active":"2024-09-10T23:37:26.598333+0000","last_peered":"2024-09-10T23:37:26.598333+0000","last_clean":"2024-09-10T23:37:26.598333+0000","last_became_active":"2024-09-10T23:37:25.579912+0000","last_became_peered":"2024-09-10T23:37:25.579912+0000","last_unstale":"2024-09-10T23:37:26.598333+0000","last_undegraded":"2024-09-10T23:37:26.598333+0000","last_fullsized":"2024-09-10T23:37:26.598333+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:30.606174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.579017+0000","last_change":"2024-09-10T23:37:26.577142+0000","last_active":"2024-09-10T23:37:27.579017+0000","last_peered":"2024-09-10T23:37:27.579017+0000","last_clean":"2024-09-10T23:37:27.579017+0000","last_became_active":"2024-09-10T23:37:26.574767+0000","last_became_peered":"2024-09-10T23:37:26.574767+0000","last_unstale":"2024-09-10T23:37:27.579017+0000","last_undegraded":"2024-09-10T23:37:27.579017+0000","last_fullsized":"2024-09-10T23:37:27.579017+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:29.277753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"98'6","reported_seq":21,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584083+0000","last_change":"2024-09-10T23:37:25.576828+0000","last_active":"2024-09-10T23:37:27.584083+0000","last_peered":"2024-09-10T23:37:27.584083+0000","last_clean":"2024-09-10T23:37:27.584083+0000","last_became_active":"2024-09-10T23:37:25.576361+0000","last_became_peered":"2024-09-10T23:37:25.576361+0000","last_unstale":"2024-09-10T23:37:27.584083+0000","last_undegraded":"2024-09-10T23:37:27.584083+0000","last_fullsized":"2024-09-10T23:37:27.584083+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:58.905202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.592069+0000","last_change":"2024-09-10T23:37:26.589331+0000","last_active":"2024-09-10T23:37:26.592069+0000","last_peered":"2024-09-10T23:37:26.592069+0000","last_clean":"2024-09-10T23:37:26.592069+0000","last_became_active":"2024-09-10T23:37:26.588041+0000","last_became_peered":"2024-09-10T23:37:26.588041+0000","last_unstale":"2024-09-10T23:37:26.592069+0000","last_undegraded":"2024-09-10T23:37:26.592069+0000","last_fullsized":"2024-09-10T23:37:26.592069+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:40.408564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"98'2","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.595820+0000","last_change":"2024-09-10T23:37:25.579349+0000","last_active":"2024-09-10T23:37:26.595820+0000","last_peered":"2024-09-10T23:37:26.595820+0000","last_clean":"2024-09-10T23:37:26.595820+0000","last_became_active":"2024-09-10T23:37:25.578471+0000","last_became_peered":"2024-09-10T23:37:25.578471+0000","last_unstale":"2024-09-10T23:37:26.595820+0000","last_undegraded":"2024-09-10T23:37:26.595820+0000","last_fullsized":"2024-09-10T23:37:26.595820+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:18.725719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583149+0000","last_change":"2024-09-10T23:37:26.560176+0000","last_active":"2024-09-10T23:37:27.583149+0000","last_peered":"2024-09-10T23:37:27.583149+0000","last_clean":"2024-09-10T23:37:27.583149+0000","last_became_active":"2024-09-10T23:37:26.559916+0000","last_became_peered":"2024-09-10T23:37:26.559916+0000","last_unstale":"2024-09-10T23:37:27.583149+0000","last_undegraded":"2024-09-10T23:37:27.583149+0000","last_fullsized":"2024-09-10T23:37:27.583149+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:33.887526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.558061+0000","last_change":"2024-09-10T23:37:25.564537+0000","last_active":"2024-09-10T23:37:26.558061+0000","last_peered":"2024-09-10T23:37:26.558061+0000","last_clean":"2024-09-10T23:37:26.558061+0000","last_became_active":"2024-09-10T23:37:25.564273+0000","last_became_peered":"2024-09-10T23:37:25.564273+0000","last_unstale":"2024-09-10T23:37:26.558061+0000","last_undegraded":"2024-09-10T23:37:26.558061+0000","last_fullsized":"2024-09-10T23:37:26.558061+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:08.765602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.582335+0000","last_change":"2024-09-10T23:37:26.569821+0000","last_active":"2024-09-10T23:37:27.582335+0000","last_peered":"2024-09-10T23:37:27.582335+0000","last_clean":"2024-09-10T23:37:27.582335+0000","last_became_active":"2024-09-10T23:37:26.569396+0000","last_became_peered":"2024-09-10T23:37:26.569396+0000","last_unstale":"2024-09-10T23:37:27.582335+0000","last_undegraded":"2024-09-10T23:37:27.582335+0000","last_fullsized":"2024-09-10T23:37:27.582335+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:15:20.676151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"98'8","reported_seq":23,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.582824+0000","last_change":"2024-09-10T23:37:25.567845+0000","last_active":"2024-09-10T23:37:27.582824+0000","last_peered":"2024-09-10T23:37:27.582824+0000","last_clean":"2024-09-10T23:37:27.582824+0000","last_became_active":"2024-09-10T23:37:25.567562+0000","last_became_peered":"2024-09-10T23:37:25.567562+0000","last_unstale":"2024-09-10T23:37:27.582824+0000","last_undegraded":"2024-09-10T23:37:27.582824+0000","last_fullsized":"2024-09-10T23:37:27.582824+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:56.739863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583724+0000","last_change":"2024-09-10T23:37:26.575323+0000","last_active":"2024-09-10T23:37:27.583724+0000","last_peered":"2024-09-10T23:37:27.583724+0000","last_clean":"2024-09-10T23:37:27.583724+0000","last_became_active":"2024-09-10T23:37:26.575012+0000","last_became_peered":"2024-09-10T23:37:26.575012+0000","last_unstale":"2024-09-10T23:37:27.583724+0000","last_undegraded":"2024-09-10T23:37:27.583724+0000","last_fullsized":"2024-09-10T23:37:27.583724+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:43.438318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"98'1","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.586816+0000","last_change":"2024-09-10T23:37:25.570549+0000","last_active":"2024-09-10T23:37:26.586816+0000","last_peered":"2024-09-10T23:37:26.586816+0000","last_clean":"2024-09-10T23:37:26.586816+0000","last_became_active":"2024-09-10T23:37:25.570220+0000","last_became_peered":"2024-09-10T23:37:25.570220+0000","last_unstale":"2024-09-10T23:37:26.586816+0000","last_undegraded":"2024-09-10T23:37:26.586816+0000","last_fullsized":"2024-09-10T23:37:26.586816+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:57.188321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584305+0000","last_change":"2024-09-10T23:37:26.583855+0000","last_active":"2024-09-10T23:37:27.584305+0000","last_peered":"2024-09-10T23:37:27.584305+0000","last_clean":"2024-09-10T23:37:27.584305+0000","last_became_active":"2024-09-10T23:37:26.583558+0000","last_became_peered":"2024-09-10T23:37:26.583558+0000","last_unstale":"2024-09-10T23:37:27.584305+0000","last_undegraded":"2024-09-10T23:37:27.584305+0000","last_fullsized":"2024-09-10T23:37:27.584305+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:35.668760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"98'2","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.595791+0000","last_change":"2024-09-10T23:37:25.578477+0000","last_active":"2024-09-10T23:37:26.595791+0000","last_peered":"2024-09-10T23:37:26.595791+0000","last_clean":"2024-09-10T23:37:26.595791+0000","last_became_active":"2024-09-10T23:37:25.578094+0000","last_became_peered":"2024-09-10T23:37:25.578094+0000","last_unstale":"2024-09-10T23:37:26.595791+0000","last_undegraded":"2024-09-10T23:37:26.595791+0000","last_fullsized":"2024-09-10T23:37:26.595791+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:29.345192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578778+0000","last_change":"2024-09-10T23:37:26.571689+0000","last_active":"2024-09-10T23:37:27.578778+0000","last_peered":"2024-09-10T23:37:27.578778+0000","last_clean":"2024-09-10T23:37:27.578778+0000","last_became_active":"2024-09-10T23:37:26.571292+0000","last_became_peered":"2024-09-10T23:37:26.571292+0000","last_unstale":"2024-09-10T23:37:27.578778+0000","last_undegraded":"2024-09-10T23:37:27.578778+0000","last_fullsized":"2024-09-10T23:37:27.578778+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:39.091655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"98'1","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.588364+0000","last_change":"2024-09-10T23:37:25.567905+0000","last_active":"2024-09-10T23:37:26.588364+0000","last_peered":"2024-09-10T23:37:26.588364+0000","last_clean":"2024-09-10T23:37:26.588364+0000","last_became_active":"2024-09-10T23:37:25.567572+0000","last_became_peered":"2024-09-10T23:37:25.567572+0000","last_unstale":"2024-09-10T23:37:26.588364+0000","last_undegraded":"2024-09-10T23:37:26.588364+0000","last_fullsized":"2024-09-10T23:37:26.588364+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:13.807191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.571733+0000","last_change":"2024-09-10T23:37:26.567098+0000","last_active":"2024-09-10T23:37:26.571733+0000","last_peered":"2024-09-10T23:37:26.571733+0000","last_clean":"2024-09-10T23:37:26.571733+0000","last_became_active":"2024-09-10T23:37:26.566842+0000","last_became_peered":"2024-09-10T23:37:26.566842+0000","last_unstale":"2024-09-10T23:37:26.571733+0000","last_undegraded":"2024-09-10T23:37:26.571733+0000","last_fullsized":"2024-09-10T23:37:26.571733+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:17.323896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.553187+0000","last_change":"2024-09-10T23:37:25.572473+0000","last_active":"2024-09-10T23:37:26.553187+0000","last_peered":"2024-09-10T23:37:26.553187+0000","last_clean":"2024-09-10T23:37:26.553187+0000","last_became_active":"2024-09-10T23:37:25.572166+0000","last_became_peered":"2024-09-10T23:37:25.572166+0000","last_unstale":"2024-09-10T23:37:26.553187+0000","last_undegraded":"2024-09-10T23:37:26.553187+0000","last_fullsized":"2024-09-10T23:37:26.553187+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:11.968920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.567463+0000","last_change":"2024-09-10T23:37:26.565334+0000","last_active":"2024-09-10T23:37:26.567463+0000","last_peered":"2024-09-10T23:37:26.567463+0000","last_clean":"2024-09-10T23:37:26.567463+0000","last_became_active":"2024-09-10T23:37:26.564958+0000","last_became_peered":"2024-09-10T23:37:26.564958+0000","last_unstale":"2024-09-10T23:37:26.567463+0000","last_undegraded":"2024-09-10T23:37:26.567463+0000","last_fullsized":"2024-09-10T23:37:26.567463+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:47.479672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"98'2","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.592932+0000","last_change":"2024-09-10T23:37:25.579209+0000","last_active":"2024-09-10T23:37:26.592932+0000","last_peered":"2024-09-10T23:37:26.592932+0000","last_clean":"2024-09-10T23:37:26.592932+0000","last_became_active":"2024-09-10T23:37:25.578867+0000","last_became_peered":"2024-09-10T23:37:25.578867+0000","last_unstale":"2024-09-10T23:37:26.592932+0000","last_undegraded":"2024-09-10T23:37:26.592932+0000","last_fullsized":"2024-09-10T23:37:26.592932+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:26.084444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584230+0000","last_change":"2024-09-10T23:37:26.586650+0000","last_active":"2024-09-10T23:37:27.584230+0000","last_peered":"2024-09-10T23:37:27.584230+0000","last_clean":"2024-09-10T23:37:27.584230+0000","last_became_active":"2024-09-10T23:37:26.585199+0000","last_became_peered":"2024-09-10T23:37:26.585199+0000","last_unstale":"2024-09-10T23:37:27.584230+0000","last_undegraded":"2024-09-10T23:37:27.584230+0000","last_fullsized":"2024-09-10T23:37:27.584230+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:22.127741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583266+0000","last_change":"2024-09-10T23:37:25.572436+0000","last_active":"2024-09-10T23:37:27.583266+0000","last_peered":"2024-09-10T23:37:27.583266+0000","last_clean":"2024-09-10T23:37:27.583266+0000","last_became_active":"2024-09-10T23:37:25.572114+0000","last_became_peered":"2024-09-10T23:37:25.572114+0000","last_unstale":"2024-09-10T23:37:27.583266+0000","last_undegraded":"2024-09-10T23:37:27.583266+0000","last_fullsized":"2024-09-10T23:37:27.583266+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:33.537215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.571272+0000","last_change":"2024-09-10T23:37:26.567713+0000","last_active":"2024-09-10T23:37:26.571272+0000","last_peered":"2024-09-10T23:37:26.571272+0000","last_clean":"2024-09-10T23:37:26.571272+0000","last_became_active":"2024-09-10T23:37:26.567468+0000","last_became_peered":"2024-09-10T23:37:26.567468+0000","last_unstale":"2024-09-10T23:37:26.571272+0000","last_undegraded":"2024-09-10T23:37:26.571272+0000","last_fullsized":"2024-09-10T23:37:26.571272+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:15.911072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.582474+0000","last_change":"2024-09-10T23:37:25.571999+0000","last_active":"2024-09-10T23:37:27.582474+0000","last_peered":"2024-09-10T23:37:27.582474+0000","last_clean":"2024-09-10T23:37:27.582474+0000","last_became_active":"2024-09-10T23:37:25.571738+0000","last_became_peered":"2024-09-10T23:37:25.571738+0000","last_unstale":"2024-09-10T23:37:27.582474+0000","last_undegraded":"2024-09-10T23:37:27.582474+0000","last_fullsized":"2024-09-10T23:37:27.582474+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:57.188321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.597946+0000","last_change":"2024-09-10T23:37:26.596316+0000","last_active":"2024-09-10T23:37:26.597946+0000","last_peered":"2024-09-10T23:37:26.597946+0000","last_clean":"2024-09-10T23:37:26.597946+0000","last_became_active":"2024-09-10T23:37:26.596058+0000","last_became_peered":"2024-09-10T23:37:26.596058+0000","last_unstale":"2024-09-10T23:37:26.597946+0000","last_undegraded":"2024-09-10T23:37:26.597946+0000","last_fullsized":"2024-09-10T23:37:26.597946+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:43.341222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.582278+0000","last_change":"2024-09-10T23:37:25.568933+0000","last_active":"2024-09-10T23:37:27.582278+0000","last_peered":"2024-09-10T23:37:27.582278+0000","last_clean":"2024-09-10T23:37:27.582278+0000","last_became_active":"2024-09-10T23:37:25.565677+0000","last_became_peered":"2024-09-10T23:37:25.565677+0000","last_unstale":"2024-09-10T23:37:27.582278+0000","last_undegraded":"2024-09-10T23:37:27.582278+0000","last_fullsized":"2024-09-10T23:37:27.582278+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:36.840828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.570403+0000","last_change":"2024-09-10T23:37:26.566541+0000","last_active":"2024-09-10T23:37:26.570403+0000","last_peered":"2024-09-10T23:37:26.570403+0000","last_clean":"2024-09-10T23:37:26.570403+0000","last_became_active":"2024-09-10T23:37:26.566241+0000","last_became_peered":"2024-09-10T23:37:26.566241+0000","last_unstale":"2024-09-10T23:37:26.570403+0000","last_undegraded":"2024-09-10T23:37:26.570403+0000","last_fullsized":"2024-09-10T23:37:26.570403+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:15.703723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"98'2","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584551+0000","last_change":"2024-09-10T23:37:25.576595+0000","last_active":"2024-09-10T23:37:27.584551+0000","last_peered":"2024-09-10T23:37:27.584551+0000","last_clean":"2024-09-10T23:37:27.584551+0000","last_became_active":"2024-09-10T23:37:25.575857+0000","last_became_peered":"2024-09-10T23:37:25.575857+0000","last_unstale":"2024-09-10T23:37:27.584551+0000","last_undegraded":"2024-09-10T23:37:27.584551+0000","last_fullsized":"2024-09-10T23:37:27.584551+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:47:32.605533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.575236+0000","last_change":"2024-09-10T23:37:26.571603+0000","last_active":"2024-09-10T23:37:26.575236+0000","last_peered":"2024-09-10T23:37:26.575236+0000","last_clean":"2024-09-10T23:37:26.575236+0000","last_became_active":"2024-09-10T23:37:26.571247+0000","last_became_peered":"2024-09-10T23:37:26.571247+0000","last_unstale":"2024-09-10T23:37:26.575236+0000","last_undegraded":"2024-09-10T23:37:26.575236+0000","last_fullsized":"2024-09-10T23:37:26.575236+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:04.526930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578754+0000","last_change":"2024-09-10T23:37:25.568397+0000","last_active":"2024-09-10T23:37:27.578754+0000","last_peered":"2024-09-10T23:37:27.578754+0000","last_clean":"2024-09-10T23:37:27.578754+0000","last_became_active":"2024-09-10T23:37:25.568069+0000","last_became_peered":"2024-09-10T23:37:25.568069+0000","last_unstale":"2024-09-10T23:37:27.578754+0000","last_undegraded":"2024-09-10T23:37:27.578754+0000","last_fullsized":"2024-09-10T23:37:27.578754+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:48.534455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584123+0000","last_change":"2024-09-10T23:37:26.590474+0000","last_active":"2024-09-10T23:37:27.584123+0000","last_peered":"2024-09-10T23:37:27.584123+0000","last_clean":"2024-09-10T23:37:27.584123+0000","last_became_active":"2024-09-10T23:37:26.590189+0000","last_became_peered":"2024-09-10T23:37:26.590189+0000","last_unstale":"2024-09-10T23:37:27.584123+0000","last_undegraded":"2024-09-10T23:37:27.584123+0000","last_fullsized":"2024-09-10T23:37:27.584123+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:31.790594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583463+0000","last_change":"2024-09-10T23:37:25.565291+0000","last_active":"2024-09-10T23:37:27.583463+0000","last_peered":"2024-09-10T23:37:27.583463+0000","last_clean":"2024-09-10T23:37:27.583463+0000","last_became_active":"2024-09-10T23:37:25.565000+0000","last_became_peered":"2024-09-10T23:37:25.565000+0000","last_unstale":"2024-09-10T23:37:27.583463+0000","last_undegraded":"2024-09-10T23:37:27.583463+0000","last_fullsized":"2024-09-10T23:37:27.583463+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:07.561326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.571770+0000","last_change":"2024-09-10T23:37:26.569739+0000","last_active":"2024-09-10T23:37:26.571770+0000","last_peered":"2024-09-10T23:37:26.571770+0000","last_clean":"2024-09-10T23:37:26.571770+0000","last_became_active":"2024-09-10T23:37:26.569482+0000","last_became_peered":"2024-09-10T23:37:26.569482+0000","last_unstale":"2024-09-10T23:37:26.571770+0000","last_undegraded":"2024-09-10T23:37:26.571770+0000","last_fullsized":"2024-09-10T23:37:26.571770+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:45.411596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584183+0000","last_change":"2024-09-10T23:37:25.573201+0000","last_active":"2024-09-10T23:37:27.584183+0000","last_peered":"2024-09-10T23:37:27.584183+0000","last_clean":"2024-09-10T23:37:27.584183+0000","last_became_active":"2024-09-10T23:37:25.572274+0000","last_became_peered":"2024-09-10T23:37:25.572274+0000","last_unstale":"2024-09-10T23:37:27.584183+0000","last_undegraded":"2024-09-10T23:37:27.584183+0000","last_fullsized":"2024-09-10T23:37:27.584183+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:07.147512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.573028+0000","last_change":"2024-09-10T23:37:26.569607+0000","last_active":"2024-09-10T23:37:26.573028+0000","last_peered":"2024-09-10T23:37:26.573028+0000","last_clean":"2024-09-10T23:37:26.573028+0000","last_became_active":"2024-09-10T23:37:26.569378+0000","last_became_peered":"2024-09-10T23:37:26.569378+0000","last_unstale":"2024-09-10T23:37:26.573028+0000","last_undegraded":"2024-09-10T23:37:26.573028+0000","last_fullsized":"2024-09-10T23:37:26.573028+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:17:06.903609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578595+0000","last_change":"2024-09-10T23:37:25.566534+0000","last_active":"2024-09-10T23:37:27.578595+0000","last_peered":"2024-09-10T23:37:27.578595+0000","last_clean":"2024-09-10T23:37:27.578595+0000","last_became_active":"2024-09-10T23:37:25.565243+0000","last_became_peered":"2024-09-10T23:37:25.565243+0000","last_unstale":"2024-09-10T23:37:27.578595+0000","last_undegraded":"2024-09-10T23:37:27.578595+0000","last_fullsized":"2024-09-10T23:37:27.578595+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:16:27.374539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.595061+0000","last_change":"2024-09-10T23:37:26.594268+0000","last_active":"2024-09-10T23:37:26.595061+0000","last_peered":"2024-09-10T23:37:26.595061+0000","last_clean":"2024-09-10T23:37:26.595061+0000","last_became_active":"2024-09-10T23:37:26.593921+0000","last_became_peered":"2024-09-10T23:37:26.593921+0000","last_unstale":"2024-09-10T23:37:26.595061+0000","last_undegraded":"2024-09-10T23:37:26.595061+0000","last_fullsized":"2024-09-10T23:37:26.595061+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:41.033610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.553574+0000","last_change":"2024-09-10T23:37:25.569342+0000","last_active":"2024-09-10T23:37:26.553574+0000","last_peered":"2024-09-10T23:37:26.553574+0000","last_clean":"2024-09-10T23:37:26.553574+0000","last_became_active":"2024-09-10T23:37:25.569030+0000","last_became_peered":"2024-09-10T23:37:25.569030+0000","last_unstale":"2024-09-10T23:37:26.553574+0000","last_undegraded":"2024-09-10T23:37:26.553574+0000","last_fullsized":"2024-09-10T23:37:26.553574+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:31.571574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.567937+0000","last_change":"2024-09-10T23:37:26.564083+0000","last_active":"2024-09-10T23:37:26.567937+0000","last_peered":"2024-09-10T23:37:26.567937+0000","last_clean":"2024-09-10T23:37:26.567937+0000","last_became_active":"2024-09-10T23:37:26.563810+0000","last_became_peered":"2024-09-10T23:37:26.563810+0000","last_unstale":"2024-09-10T23:37:26.567937+0000","last_undegraded":"2024-09-10T23:37:26.567937+0000","last_fullsized":"2024-09-10T23:37:26.567937+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:07.710400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583597+0000","last_change":"2024-09-10T23:37:25.568287+0000","last_active":"2024-09-10T23:37:27.583597+0000","last_peered":"2024-09-10T23:37:27.583597+0000","last_clean":"2024-09-10T23:37:27.583597+0000","last_became_active":"2024-09-10T23:37:25.567908+0000","last_became_peered":"2024-09-10T23:37:25.567908+0000","last_unstale":"2024-09-10T23:37:27.583597+0000","last_undegraded":"2024-09-10T23:37:27.583597+0000","last_fullsized":"2024-09-10T23:37:27.583597+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:42.198784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.571054+0000","last_change":"2024-09-10T23:37:26.568353+0000","last_active":"2024-09-10T23:37:26.571054+0000","last_peered":"2024-09-10T23:37:26.571054+0000","last_clean":"2024-09-10T23:37:26.571054+0000","last_became_active":"2024-09-10T23:37:26.567857+0000","last_became_peered":"2024-09-10T23:37:26.567857+0000","last_unstale":"2024-09-10T23:37:26.571054+0000","last_undegraded":"2024-09-10T23:37:26.571054+0000","last_fullsized":"2024-09-10T23:37:26.571054+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:24.051687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.564851+0000","last_change":"2024-09-10T23:37:25.570114+0000","last_active":"2024-09-10T23:37:26.564851+0000","last_peered":"2024-09-10T23:37:26.564851+0000","last_clean":"2024-09-10T23:37:26.564851+0000","last_became_active":"2024-09-10T23:37:25.569763+0000","last_became_peered":"2024-09-10T23:37:25.569763+0000","last_unstale":"2024-09-10T23:37:26.564851+0000","last_undegraded":"2024-09-10T23:37:26.564851+0000","last_fullsized":"2024-09-10T23:37:26.564851+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:18:28.119346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.582243+0000","last_change":"2024-09-10T23:37:26.569887+0000","last_active":"2024-09-10T23:37:27.582243+0000","last_peered":"2024-09-10T23:37:27.582243+0000","last_clean":"2024-09-10T23:37:27.582243+0000","last_became_active":"2024-09-10T23:37:26.569594+0000","last_became_peered":"2024-09-10T23:37:26.569594+0000","last_unstale":"2024-09-10T23:37:27.582243+0000","last_undegraded":"2024-09-10T23:37:27.582243+0000","last_fullsized":"2024-09-10T23:37:27.582243+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:33.373338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.562574+0000","last_change":"2024-09-10T23:37:25.573222+0000","last_active":"2024-09-10T23:37:26.562574+0000","last_peered":"2024-09-10T23:37:26.562574+0000","last_clean":"2024-09-10T23:37:26.562574+0000","last_became_active":"2024-09-10T23:37:25.572806+0000","last_became_peered":"2024-09-10T23:37:25.572806+0000","last_unstale":"2024-09-10T23:37:26.562574+0000","last_undegraded":"2024-09-10T23:37:26.562574+0000","last_fullsized":"2024-09-10T23:37:26.562574+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:04.445805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":278,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.557967+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:37:26.557967+0000","last_peered":"2024-09-10T23:37:26.557967+0000","last_clean":"2024-09-10T23:37:26.557967+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:37:26.557967+0000","last_undegraded":"2024-09-10T23:37:26.557967+0000","last_fullsized":"2024-09-10T23:37:26.557967+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509505,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1232896,"data_stored":528054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542208,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":526318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509504,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32960,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338880,"statfs":{"total":96636764160,"available":96603013120,"internally_reserved":0,"allocated":1814528,"data_stored":1116778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509502,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1228800,"data_stored":528020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509503,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1212416,"data_stored":526318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509504,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1224704,"data_stored":526956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509503,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32952,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338888,"statfs":{"total":96636764160,"available":96603021312,"internally_reserved":0,"allocated":1806336,"data_stored":1116686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509503,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1220608,"data_stored":526970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:37:30.688 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:37:30.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.865+0000 7f5d2ec00640 1 -- 172.21.15.94:0/3073212084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d0c004810 msgr2=0x7f5d0c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:30.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.865+0000 7f5d2ec00640 1 --2- 172.21.15.94:0/3073212084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d0c004810 0x7f5d0c004c90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000c260 tx=0x7f5d1001cf60 comp rx=0 tx=0).stop 2024-09-10T23:37:30.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.865+0000 7f5d2cc00640 1 -- 172.21.15.94:0/3073212084 reap_dead start 2024-09-10T23:37:30.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.865+0000 7f5d2ec00640 1 -- 172.21.15.94:0/3073212084 shutdown_connections 2024-09-10T23:37:30.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.865+0000 7f5d2ec00640 1 -- 172.21.15.94:0/3073212084 >> 172.21.15.94:0/3073212084 conn(0x7f5d2806cdf0 msgr2=0x7f5d2806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:30.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.865+0000 7f5d2ec00640 1 -- 172.21.15.94:0/3073212084 shutdown_connections 2024-09-10T23:37:30.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.865+0000 7f5d2ec00640 1 -- 172.21.15.94:0/3073212084 wait complete. 2024-09-10T23:37:30.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.865+0000 7f5d2ec00640 1 Processor -- start 2024-09-10T23:37:30.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.865+0000 7f5d2ec00640 1 -- start start 2024-09-10T23:37:30.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2ec00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0c004810 0x7f5d28150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2ec00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d280ccca0 0x7f5d280c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2ec00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d280cd120 0x7f5d280c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2ec00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2815b5f0 con 0x7f5d280cd120 2024-09-10T23:37:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2ec00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2815b470 con 0x7f5d0c004810 2024-09-10T23:37:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2ec00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5d28165f80 con 0x7f5d280ccca0 2024-09-10T23:37:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d280cd120 0x7f5d280c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d280cd120 0x7f5d280c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60682/0 (socket says 172.21.15.94:60682) 2024-09-10T23:37:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2cc00640 1 -- 172.21.15.94:0/1224945486 learned_addr learned my addr 172.21.15.94:0/1224945486 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d27e00640 1 --2- 172.21.15.94:0/1224945486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0c004810 0x7f5d28150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d1f400640 1 --2- 172.21.15.94:0/1224945486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d280ccca0 0x7f5d280c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2cc00640 1 -- 172.21.15.94:0/1224945486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d280ccca0 msgr2=0x7f5d280c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2cc00640 1 --2- 172.21.15.94:0/1224945486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d280ccca0 0x7f5d280c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2cc00640 1 -- 172.21.15.94:0/1224945486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0c004810 msgr2=0x7f5d28150930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:30.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2cc00640 1 --2- 172.21.15.94:0/1224945486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0c004810 0x7f5d28150930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:30.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2cc00640 1 -- 172.21.15.94:0/1224945486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d280c91a0 con 0x7f5d280cd120 2024-09-10T23:37:30.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d27e00640 1 --2- 172.21.15.94:0/1224945486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0c004810 0x7f5d28150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:30.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d1f400640 1 --2- 172.21.15.94:0/1224945486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d280ccca0 0x7f5d280c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:30.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2cc00640 1 --2- 172.21.15.94:0/1224945486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d280cd120 0x7f5d280c9ca0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5d280ced80 tx=0x7f5d2000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:30.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d24c00640 1 -- 172.21.15.94:0/1224945486 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20004b30 con 0x7f5d280cd120 2024-09-10T23:37:30.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d24c00640 1 -- 172.21.15.94:0/1224945486 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d20004cd0 con 0x7f5d280cd120 2024-09-10T23:37:30.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d24c00640 1 -- 172.21.15.94:0/1224945486 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20003be0 con 0x7f5d280cd120 2024-09-10T23:37:30.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2ec00640 1 -- 172.21.15.94:0/1224945486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d280ce0e0 con 0x7f5d280cd120 2024-09-10T23:37:30.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2ec00640 1 -- 172.21.15.94:0/1224945486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d280ce6a0 con 0x7f5d280cd120 2024-09-10T23:37:30.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.869+0000 7f5d2ec00640 1 -- 172.21.15.94:0/1224945486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d280ce870 con 0x7f5d280cd120 2024-09-10T23:37:30.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.873+0000 7f5d24c00640 1 -- 172.21.15.94:0/1224945486 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f5d20018070 con 0x7f5d280cd120 2024-09-10T23:37:30.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.873+0000 7f5d24c00640 1 --2- 172.21.15.94:0/1224945486 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5d04072700 0x7f5d04074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:30.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.873+0000 7f5d24c00640 1 -- 172.21.15.94:0/1224945486 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f5d20093b00 con 0x7f5d280cd120 2024-09-10T23:37:30.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.873+0000 7f5d27e00640 1 --2- 172.21.15.94:0/1224945486 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5d04072700 0x7f5d04074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:30.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.873+0000 7f5d27e00640 1 --2- 172.21.15.94:0/1224945486 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5d04072700 0x7f5d04074bc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5d2815dbd0 tx=0x7f5d180091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:30.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:30.877+0000 7f5d24c00640 1 -- 172.21.15.94:0/1224945486 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5d2005ef20 con 0x7f5d280cd120 2024-09-10T23:37:31.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.037+0000 7f5d2ec00640 1 -- 172.21.15.94:0/1224945486 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5d280cebc0 con 0x7f5d04072700 2024-09-10T23:37:31.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.041+0000 7f5d24c00640 1 -- 172.21.15.94:0/1224945486 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252065 (secure 0 0 0) 0x7f5d18012bd0 con 0x7f5d04072700 2024-09-10T23:37:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:31.049 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:37:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.045+0000 7f5d2ec00640 1 -- 172.21.15.94:0/1224945486 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5d04072700 msgr2=0x7f5d04074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.045+0000 7f5d2ec00640 1 --2- 172.21.15.94:0/1224945486 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5d04072700 0x7f5d04074bc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5d2815dbd0 tx=0x7f5d180091e0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.045+0000 7f5d2ec00640 1 -- 172.21.15.94:0/1224945486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d280cd120 msgr2=0x7f5d280c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.045+0000 7f5d2ec00640 1 --2- 172.21.15.94:0/1224945486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d280cd120 0x7f5d280c9ca0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5d280ced80 tx=0x7f5d2000f3c0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.045+0000 7f5d2ec00640 1 -- 172.21.15.94:0/1224945486 shutdown_connections 2024-09-10T23:37:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.045+0000 7f5d2ec00640 1 --2- 172.21.15.94:0/1224945486 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5d04072700 0x7f5d04074bc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.045+0000 7f5d2ec00640 1 --2- 172.21.15.94:0/1224945486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d280cd120 0x7f5d280c9ca0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.045+0000 7f5d2ec00640 1 --2- 172.21.15.94:0/1224945486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d280ccca0 0x7f5d280c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.045+0000 7f5d2ec00640 1 --2- 172.21.15.94:0/1224945486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0c004810 0x7f5d28150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.045+0000 7f5d2ec00640 1 -- 172.21.15.94:0/1224945486 >> 172.21.15.94:0/1224945486 conn(0x7f5d2806cdf0 msgr2=0x7f5d280cfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:31.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.045+0000 7f5d2ec00640 1 -- 172.21.15.94:0/1224945486 shutdown_connections 2024-09-10T23:37:31.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.045+0000 7f5d2ec00640 1 -- 172.21.15.94:0/1224945486 wait complete. 2024-09-10T23:37:31.068 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-09-10T23:37:30.745787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260232,"kb_used_data":10696,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714488,"statfs":{"total":773094113280,"available":772827635712,"internally_reserved":0,"allocated":10952704,"data_stored":5396100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001095"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583842+0000","last_change":"2024-09-10T23:37:26.572290+0000","last_active":"2024-09-10T23:37:27.583842+0000","last_peered":"2024-09-10T23:37:27.583842+0000","last_clean":"2024-09-10T23:37:27.583842+0000","last_became_active":"2024-09-10T23:37:26.572038+0000","last_became_peered":"2024-09-10T23:37:26.572038+0000","last_unstale":"2024-09-10T23:37:27.583842+0000","last_undegraded":"2024-09-10T23:37:27.583842+0000","last_fullsized":"2024-09-10T23:37:27.583842+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:01.439200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.574323+0000","last_change":"2024-09-10T23:37:26.568617+0000","last_active":"2024-09-10T23:37:26.574323+0000","last_peered":"2024-09-10T23:37:26.574323+0000","last_clean":"2024-09-10T23:37:26.574323+0000","last_became_active":"2024-09-10T23:37:26.568204+0000","last_became_peered":"2024-09-10T23:37:26.568204+0000","last_unstale":"2024-09-10T23:37:26.574323+0000","last_undegraded":"2024-09-10T23:37:26.574323+0000","last_fullsized":"2024-09-10T23:37:26.574323+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:59:34.242358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.573067+0000","last_change":"2024-09-10T23:37:26.569482+0000","last_active":"2024-09-10T23:37:26.573067+0000","last_peered":"2024-09-10T23:37:26.573067+0000","last_clean":"2024-09-10T23:37:26.573067+0000","last_became_active":"2024-09-10T23:37:26.569003+0000","last_became_peered":"2024-09-10T23:37:26.569003+0000","last_unstale":"2024-09-10T23:37:26.573067+0000","last_undegraded":"2024-09-10T23:37:26.573067+0000","last_fullsized":"2024-09-10T23:37:26.573067+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:34:54.359719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583807+0000","last_change":"2024-09-10T23:37:26.571109+0000","last_active":"2024-09-10T23:37:27.583807+0000","last_peered":"2024-09-10T23:37:27.583807+0000","last_clean":"2024-09-10T23:37:27.583807+0000","last_became_active":"2024-09-10T23:37:26.570655+0000","last_became_peered":"2024-09-10T23:37:26.570655+0000","last_unstale":"2024-09-10T23:37:27.583807+0000","last_undegraded":"2024-09-10T23:37:27.583807+0000","last_fullsized":"2024-09-10T23:37:27.583807+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:04.968567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.597178+0000","last_change":"2024-09-10T23:37:26.592064+0000","last_active":"2024-09-10T23:37:26.597178+0000","last_peered":"2024-09-10T23:37:26.597178+0000","last_clean":"2024-09-10T23:37:26.597178+0000","last_became_active":"2024-09-10T23:37:26.591737+0000","last_became_peered":"2024-09-10T23:37:26.591737+0000","last_unstale":"2024-09-10T23:37:26.597178+0000","last_undegraded":"2024-09-10T23:37:26.597178+0000","last_fullsized":"2024-09-10T23:37:26.597178+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:09.768476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.575505+0000","last_change":"2024-09-10T23:37:26.570231+0000","last_active":"2024-09-10T23:37:26.575505+0000","last_peered":"2024-09-10T23:37:26.575505+0000","last_clean":"2024-09-10T23:37:26.575505+0000","last_became_active":"2024-09-10T23:37:26.569922+0000","last_became_peered":"2024-09-10T23:37:26.569922+0000","last_unstale":"2024-09-10T23:37:26.575505+0000","last_undegraded":"2024-09-10T23:37:26.575505+0000","last_fullsized":"2024-09-10T23:37:26.575505+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:07.013347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578269+0000","last_change":"2024-09-10T23:37:26.570469+0000","last_active":"2024-09-10T23:37:27.578269+0000","last_peered":"2024-09-10T23:37:27.578269+0000","last_clean":"2024-09-10T23:37:27.578269+0000","last_became_active":"2024-09-10T23:37:26.570161+0000","last_became_peered":"2024-09-10T23:37:26.570161+0000","last_unstale":"2024-09-10T23:37:27.578269+0000","last_undegraded":"2024-09-10T23:37:27.578269+0000","last_fullsized":"2024-09-10T23:37:27.578269+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:31:31.064875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578272+0000","last_change":"2024-09-10T23:37:26.572452+0000","last_active":"2024-09-10T23:37:27.578272+0000","last_peered":"2024-09-10T23:37:27.578272+0000","last_clean":"2024-09-10T23:37:27.578272+0000","last_became_active":"2024-09-10T23:37:26.572163+0000","last_became_peered":"2024-09-10T23:37:26.572163+0000","last_unstale":"2024-09-10T23:37:27.578272+0000","last_undegraded":"2024-09-10T23:37:27.578272+0000","last_fullsized":"2024-09-10T23:37:27.578272+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:45.392321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584096+0000","last_change":"2024-09-10T23:37:26.583435+0000","last_active":"2024-09-10T23:37:27.584096+0000","last_peered":"2024-09-10T23:37:27.584096+0000","last_clean":"2024-09-10T23:37:27.584096+0000","last_became_active":"2024-09-10T23:37:26.582937+0000","last_became_peered":"2024-09-10T23:37:26.582937+0000","last_unstale":"2024-09-10T23:37:27.584096+0000","last_undegraded":"2024-09-10T23:37:27.584096+0000","last_fullsized":"2024-09-10T23:37:27.584096+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:53.799014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.597294+0000","last_change":"2024-09-10T23:37:26.593305+0000","last_active":"2024-09-10T23:37:26.597294+0000","last_peered":"2024-09-10T23:37:26.597294+0000","last_clean":"2024-09-10T23:37:26.597294+0000","last_became_active":"2024-09-10T23:37:26.592924+0000","last_became_peered":"2024-09-10T23:37:26.592924+0000","last_unstale":"2024-09-10T23:37:26.597294+0000","last_undegraded":"2024-09-10T23:37:26.597294+0000","last_fullsized":"2024-09-10T23:37:26.597294+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:47.479672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578820+0000","last_change":"2024-09-10T23:37:26.572828+0000","last_active":"2024-09-10T23:37:27.578820+0000","last_peered":"2024-09-10T23:37:27.578820+0000","last_clean":"2024-09-10T23:37:27.578820+0000","last_became_active":"2024-09-10T23:37:26.572496+0000","last_became_peered":"2024-09-10T23:37:26.572496+0000","last_unstale":"2024-09-10T23:37:27.578820+0000","last_undegraded":"2024-09-10T23:37:27.578820+0000","last_fullsized":"2024-09-10T23:37:27.578820+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:28:23.356135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.599082+0000","last_change":"2024-09-10T23:37:26.597675+0000","last_active":"2024-09-10T23:37:26.599082+0000","last_peered":"2024-09-10T23:37:26.599082+0000","last_clean":"2024-09-10T23:37:26.599082+0000","last_became_active":"2024-09-10T23:37:26.597286+0000","last_became_peered":"2024-09-10T23:37:26.597286+0000","last_unstale":"2024-09-10T23:37:26.599082+0000","last_undegraded":"2024-09-10T23:37:26.599082+0000","last_fullsized":"2024-09-10T23:37:26.599082+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:54.910181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.594984+0000","last_change":"2024-09-10T23:37:26.593290+0000","last_active":"2024-09-10T23:37:26.594984+0000","last_peered":"2024-09-10T23:37:26.594984+0000","last_clean":"2024-09-10T23:37:26.594984+0000","last_became_active":"2024-09-10T23:37:26.592808+0000","last_became_peered":"2024-09-10T23:37:26.592808+0000","last_unstale":"2024-09-10T23:37:26.594984+0000","last_undegraded":"2024-09-10T23:37:26.594984+0000","last_fullsized":"2024-09-10T23:37:26.594984+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:08.150387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.573943+0000","last_change":"2024-09-10T23:37:26.571006+0000","last_active":"2024-09-10T23:37:26.573943+0000","last_peered":"2024-09-10T23:37:26.573943+0000","last_clean":"2024-09-10T23:37:26.573943+0000","last_became_active":"2024-09-10T23:37:26.570597+0000","last_became_peered":"2024-09-10T23:37:26.570597+0000","last_unstale":"2024-09-10T23:37:26.573943+0000","last_undegraded":"2024-09-10T23:37:26.573943+0000","last_fullsized":"2024-09-10T23:37:26.573943+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:49.340330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.582932+0000","last_change":"2024-09-10T23:37:26.590703+0000","last_active":"2024-09-10T23:37:27.582932+0000","last_peered":"2024-09-10T23:37:27.582932+0000","last_clean":"2024-09-10T23:37:27.582932+0000","last_became_active":"2024-09-10T23:37:26.590424+0000","last_became_peered":"2024-09-10T23:37:26.590424+0000","last_unstale":"2024-09-10T23:37:27.582932+0000","last_undegraded":"2024-09-10T23:37:27.582932+0000","last_fullsized":"2024-09-10T23:37:27.582932+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:06.059219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.591998+0000","last_change":"2024-09-10T23:37:26.589407+0000","last_active":"2024-09-10T23:37:26.591998+0000","last_peered":"2024-09-10T23:37:26.591998+0000","last_clean":"2024-09-10T23:37:26.591998+0000","last_became_active":"2024-09-10T23:37:26.587702+0000","last_became_peered":"2024-09-10T23:37:26.587702+0000","last_unstale":"2024-09-10T23:37:26.591998+0000","last_undegraded":"2024-09-10T23:37:26.591998+0000","last_fullsized":"2024-09-10T23:37:26.591998+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:19.135723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.569844+0000","last_change":"2024-09-10T23:37:26.565785+0000","last_active":"2024-09-10T23:37:26.569844+0000","last_peered":"2024-09-10T23:37:26.569844+0000","last_clean":"2024-09-10T23:37:26.569844+0000","last_became_active":"2024-09-10T23:37:26.564340+0000","last_became_peered":"2024-09-10T23:37:26.564340+0000","last_unstale":"2024-09-10T23:37:26.569844+0000","last_undegraded":"2024-09-10T23:37:26.569844+0000","last_fullsized":"2024-09-10T23:37:26.569844+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:43.341222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578850+0000","last_change":"2024-09-10T23:37:26.574009+0000","last_active":"2024-09-10T23:37:27.578850+0000","last_peered":"2024-09-10T23:37:27.578850+0000","last_clean":"2024-09-10T23:37:27.578850+0000","last_became_active":"2024-09-10T23:37:26.573740+0000","last_became_peered":"2024-09-10T23:37:26.573740+0000","last_unstale":"2024-09-10T23:37:27.578850+0000","last_undegraded":"2024-09-10T23:37:27.578850+0000","last_fullsized":"2024-09-10T23:37:27.578850+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:19.321987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.592286+0000","last_change":"2024-09-10T23:37:26.591258+0000","last_active":"2024-09-10T23:37:26.592286+0000","last_peered":"2024-09-10T23:37:26.592286+0000","last_clean":"2024-09-10T23:37:26.592286+0000","last_became_active":"2024-09-10T23:37:26.590931+0000","last_became_peered":"2024-09-10T23:37:26.590931+0000","last_unstale":"2024-09-10T23:37:26.592286+0000","last_undegraded":"2024-09-10T23:37:26.592286+0000","last_fullsized":"2024-09-10T23:37:26.592286+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:08.290602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.571217+0000","last_change":"2024-09-10T23:37:26.566788+0000","last_active":"2024-09-10T23:37:26.571217+0000","last_peered":"2024-09-10T23:37:26.571217+0000","last_clean":"2024-09-10T23:37:26.571217+0000","last_became_active":"2024-09-10T23:37:26.566537+0000","last_became_peered":"2024-09-10T23:37:26.566537+0000","last_unstale":"2024-09-10T23:37:26.571217+0000","last_undegraded":"2024-09-10T23:37:26.571217+0000","last_fullsized":"2024-09-10T23:37:26.571217+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:54.910181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578559+0000","last_change":"2024-09-10T23:37:26.572733+0000","last_active":"2024-09-10T23:37:27.578559+0000","last_peered":"2024-09-10T23:37:27.578559+0000","last_clean":"2024-09-10T23:37:27.578559+0000","last_became_active":"2024-09-10T23:37:26.569327+0000","last_became_peered":"2024-09-10T23:37:26.569327+0000","last_unstale":"2024-09-10T23:37:27.578559+0000","last_undegraded":"2024-09-10T23:37:27.578559+0000","last_fullsized":"2024-09-10T23:37:27.578559+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:52.847810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.575481+0000","last_change":"2024-09-10T23:37:26.569472+0000","last_active":"2024-09-10T23:37:26.575481+0000","last_peered":"2024-09-10T23:37:26.575481+0000","last_clean":"2024-09-10T23:37:26.575481+0000","last_became_active":"2024-09-10T23:37:26.569145+0000","last_became_peered":"2024-09-10T23:37:26.569145+0000","last_unstale":"2024-09-10T23:37:26.575481+0000","last_undegraded":"2024-09-10T23:37:26.575481+0000","last_fullsized":"2024-09-10T23:37:26.575481+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:23.851300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.574044+0000","last_change":"2024-09-10T23:37:26.571874+0000","last_active":"2024-09-10T23:37:26.574044+0000","last_peered":"2024-09-10T23:37:26.574044+0000","last_clean":"2024-09-10T23:37:26.574044+0000","last_became_active":"2024-09-10T23:37:26.571523+0000","last_became_peered":"2024-09-10T23:37:26.571523+0000","last_unstale":"2024-09-10T23:37:26.574044+0000","last_undegraded":"2024-09-10T23:37:26.574044+0000","last_fullsized":"2024-09-10T23:37:26.574044+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:53.386615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.582146+0000","last_change":"2024-09-10T23:37:26.569387+0000","last_active":"2024-09-10T23:37:27.582146+0000","last_peered":"2024-09-10T23:37:27.582146+0000","last_clean":"2024-09-10T23:37:27.582146+0000","last_became_active":"2024-09-10T23:37:26.568874+0000","last_became_peered":"2024-09-10T23:37:26.568874+0000","last_unstale":"2024-09-10T23:37:27.582146+0000","last_undegraded":"2024-09-10T23:37:27.582146+0000","last_fullsized":"2024-09-10T23:37:27.582146+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:15:06.855063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578875+0000","last_change":"2024-09-10T23:37:26.572047+0000","last_active":"2024-09-10T23:37:27.578875+0000","last_peered":"2024-09-10T23:37:27.578875+0000","last_clean":"2024-09-10T23:37:27.578875+0000","last_became_active":"2024-09-10T23:37:26.571800+0000","last_became_peered":"2024-09-10T23:37:26.571800+0000","last_unstale":"2024-09-10T23:37:27.578875+0000","last_undegraded":"2024-09-10T23:37:27.578875+0000","last_fullsized":"2024-09-10T23:37:27.578875+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:52.936434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.585742+0000","last_change":"2024-09-10T23:37:26.574974+0000","last_active":"2024-09-10T23:37:26.585742+0000","last_peered":"2024-09-10T23:37:26.585742+0000","last_clean":"2024-09-10T23:37:26.585742+0000","last_became_active":"2024-09-10T23:37:26.574632+0000","last_became_peered":"2024-09-10T23:37:26.574632+0000","last_unstale":"2024-09-10T23:37:26.585742+0000","last_undegraded":"2024-09-10T23:37:26.585742+0000","last_fullsized":"2024-09-10T23:37:26.585742+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:15:06.855063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584327+0000","last_change":"2024-09-10T23:37:26.570460+0000","last_active":"2024-09-10T23:37:27.584327+0000","last_peered":"2024-09-10T23:37:27.584327+0000","last_clean":"2024-09-10T23:37:27.584327+0000","last_became_active":"2024-09-10T23:37:26.570179+0000","last_became_peered":"2024-09-10T23:37:26.570179+0000","last_unstale":"2024-09-10T23:37:27.584327+0000","last_undegraded":"2024-09-10T23:37:27.584327+0000","last_fullsized":"2024-09-10T23:37:27.584327+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:35.652590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.586488+0000","last_change":"2024-09-10T23:37:26.576961+0000","last_active":"2024-09-10T23:37:26.586488+0000","last_peered":"2024-09-10T23:37:26.586488+0000","last_clean":"2024-09-10T23:37:26.586488+0000","last_became_active":"2024-09-10T23:37:26.576664+0000","last_became_peered":"2024-09-10T23:37:26.576664+0000","last_unstale":"2024-09-10T23:37:26.586488+0000","last_undegraded":"2024-09-10T23:37:26.586488+0000","last_fullsized":"2024-09-10T23:37:26.586488+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:33.373338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.573459+0000","last_change":"2024-09-10T23:37:26.570558+0000","last_active":"2024-09-10T23:37:26.573459+0000","last_peered":"2024-09-10T23:37:26.573459+0000","last_clean":"2024-09-10T23:37:26.573459+0000","last_became_active":"2024-09-10T23:37:26.570255+0000","last_became_peered":"2024-09-10T23:37:26.570255+0000","last_unstale":"2024-09-10T23:37:26.573459+0000","last_undegraded":"2024-09-10T23:37:26.573459+0000","last_fullsized":"2024-09-10T23:37:26.573459+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:13:59.361625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578443+0000","last_change":"2024-09-10T23:37:26.571146+0000","last_active":"2024-09-10T23:37:27.578443+0000","last_peered":"2024-09-10T23:37:27.578443+0000","last_clean":"2024-09-10T23:37:27.578443+0000","last_became_active":"2024-09-10T23:37:26.570830+0000","last_became_peered":"2024-09-10T23:37:26.570830+0000","last_unstale":"2024-09-10T23:37:27.578443+0000","last_undegraded":"2024-09-10T23:37:27.578443+0000","last_fullsized":"2024-09-10T23:37:27.578443+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:06.546726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583478+0000","last_change":"2024-09-10T23:37:26.572980+0000","last_active":"2024-09-10T23:37:27.583478+0000","last_peered":"2024-09-10T23:37:27.583478+0000","last_clean":"2024-09-10T23:37:27.583478+0000","last_became_active":"2024-09-10T23:37:26.572698+0000","last_became_peered":"2024-09-10T23:37:26.572698+0000","last_unstale":"2024-09-10T23:37:27.583478+0000","last_undegraded":"2024-09-10T23:37:27.583478+0000","last_fullsized":"2024-09-10T23:37:27.583478+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:04.358361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.592037+0000","last_change":"2024-09-10T23:37:26.589579+0000","last_active":"2024-09-10T23:37:26.592037+0000","last_peered":"2024-09-10T23:37:26.592037+0000","last_clean":"2024-09-10T23:37:26.592037+0000","last_became_active":"2024-09-10T23:37:26.589260+0000","last_became_peered":"2024-09-10T23:37:26.589260+0000","last_unstale":"2024-09-10T23:37:26.592037+0000","last_undegraded":"2024-09-10T23:37:26.592037+0000","last_fullsized":"2024-09-10T23:37:26.592037+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:30.790821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583682+0000","last_change":"2024-09-10T23:37:26.580274+0000","last_active":"2024-09-10T23:37:27.583682+0000","last_peered":"2024-09-10T23:37:27.583682+0000","last_clean":"2024-09-10T23:37:27.583682+0000","last_became_active":"2024-09-10T23:37:26.579920+0000","last_became_peered":"2024-09-10T23:37:26.579920+0000","last_unstale":"2024-09-10T23:37:27.583682+0000","last_undegraded":"2024-09-10T23:37:27.583682+0000","last_fullsized":"2024-09-10T23:37:27.583682+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:18.088525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"98'4","reported_seq":17,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.582617+0000","last_change":"2024-09-10T23:37:25.566145+0000","last_active":"2024-09-10T23:37:26.582617+0000","last_peered":"2024-09-10T23:37:26.582617+0000","last_clean":"2024-09-10T23:37:26.582617+0000","last_became_active":"2024-09-10T23:37:25.565826+0000","last_became_peered":"2024-09-10T23:37:25.565826+0000","last_unstale":"2024-09-10T23:37:26.582617+0000","last_undegraded":"2024-09-10T23:37:26.582617+0000","last_fullsized":"2024-09-10T23:37:26.582617+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:12.094932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.578010+0000","last_change":"2024-09-10T23:37:26.574542+0000","last_active":"2024-09-10T23:37:26.578010+0000","last_peered":"2024-09-10T23:37:26.578010+0000","last_clean":"2024-09-10T23:37:26.578010+0000","last_became_active":"2024-09-10T23:37:26.573984+0000","last_became_peered":"2024-09-10T23:37:26.573984+0000","last_unstale":"2024-09-10T23:37:26.578010+0000","last_undegraded":"2024-09-10T23:37:26.578010+0000","last_fullsized":"2024-09-10T23:37:26.578010+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:00.463133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"98'1","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.580854+0000","last_change":"2024-09-10T23:37:25.563742+0000","last_active":"2024-09-10T23:37:26.580854+0000","last_peered":"2024-09-10T23:37:26.580854+0000","last_clean":"2024-09-10T23:37:26.580854+0000","last_became_active":"2024-09-10T23:37:25.563167+0000","last_became_peered":"2024-09-10T23:37:25.563167+0000","last_unstale":"2024-09-10T23:37:26.580854+0000","last_undegraded":"2024-09-10T23:37:26.580854+0000","last_fullsized":"2024-09-10T23:37:26.580854+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:48:59.532474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.574942+0000","last_change":"2024-09-10T23:37:26.572840+0000","last_active":"2024-09-10T23:37:26.574942+0000","last_peered":"2024-09-10T23:37:26.574942+0000","last_clean":"2024-09-10T23:37:26.574942+0000","last_became_active":"2024-09-10T23:37:26.572565+0000","last_became_peered":"2024-09-10T23:37:26.572565+0000","last_unstale":"2024-09-10T23:37:26.574942+0000","last_undegraded":"2024-09-10T23:37:26.574942+0000","last_fullsized":"2024-09-10T23:37:26.574942+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:08.191727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"98'2","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583376+0000","last_change":"2024-09-10T23:37:25.573677+0000","last_active":"2024-09-10T23:37:27.583376+0000","last_peered":"2024-09-10T23:37:27.583376+0000","last_clean":"2024-09-10T23:37:27.583376+0000","last_became_active":"2024-09-10T23:37:25.573325+0000","last_became_peered":"2024-09-10T23:37:25.573325+0000","last_unstale":"2024-09-10T23:37:27.583376+0000","last_undegraded":"2024-09-10T23:37:27.583376+0000","last_fullsized":"2024-09-10T23:37:27.583376+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:21.697861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.589544+0000","last_change":"2024-09-10T23:37:26.589006+0000","last_active":"2024-09-10T23:37:26.589544+0000","last_peered":"2024-09-10T23:37:26.589544+0000","last_clean":"2024-09-10T23:37:26.589544+0000","last_became_active":"2024-09-10T23:37:26.588387+0000","last_became_peered":"2024-09-10T23:37:26.588387+0000","last_unstale":"2024-09-10T23:37:26.589544+0000","last_undegraded":"2024-09-10T23:37:26.589544+0000","last_fullsized":"2024-09-10T23:37:26.589544+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:50:17.880005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"98'2","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583976+0000","last_change":"2024-09-10T23:37:25.576105+0000","last_active":"2024-09-10T23:37:27.583976+0000","last_peered":"2024-09-10T23:37:27.583976+0000","last_clean":"2024-09-10T23:37:27.583976+0000","last_became_active":"2024-09-10T23:37:25.575750+0000","last_became_peered":"2024-09-10T23:37:25.575750+0000","last_unstale":"2024-09-10T23:37:27.583976+0000","last_undegraded":"2024-09-10T23:37:27.583976+0000","last_fullsized":"2024-09-10T23:37:27.583976+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:59:33.239484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.582842+0000","last_change":"2024-09-10T23:37:26.574146+0000","last_active":"2024-09-10T23:37:27.582842+0000","last_peered":"2024-09-10T23:37:27.582842+0000","last_clean":"2024-09-10T23:37:27.582842+0000","last_became_active":"2024-09-10T23:37:26.571996+0000","last_became_peered":"2024-09-10T23:37:26.571996+0000","last_unstale":"2024-09-10T23:37:27.582842+0000","last_undegraded":"2024-09-10T23:37:27.582842+0000","last_fullsized":"2024-09-10T23:37:27.582842+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:56.212613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"98'1","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.587513+0000","last_change":"2024-09-10T23:37:25.566909+0000","last_active":"2024-09-10T23:37:26.587513+0000","last_peered":"2024-09-10T23:37:26.587513+0000","last_clean":"2024-09-10T23:37:26.587513+0000","last_became_active":"2024-09-10T23:37:25.566558+0000","last_became_peered":"2024-09-10T23:37:25.566558+0000","last_unstale":"2024-09-10T23:37:26.587513+0000","last_undegraded":"2024-09-10T23:37:26.587513+0000","last_fullsized":"2024-09-10T23:37:26.587513+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:16.692007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578376+0000","last_change":"2024-09-10T23:37:26.592072+0000","last_active":"2024-09-10T23:37:27.578376+0000","last_peered":"2024-09-10T23:37:27.578376+0000","last_clean":"2024-09-10T23:37:27.578376+0000","last_became_active":"2024-09-10T23:37:26.591739+0000","last_became_peered":"2024-09-10T23:37:26.591739+0000","last_unstale":"2024-09-10T23:37:27.578376+0000","last_undegraded":"2024-09-10T23:37:27.578376+0000","last_fullsized":"2024-09-10T23:37:27.578376+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:02.820297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"98'1","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583164+0000","last_change":"2024-09-10T23:37:25.568577+0000","last_active":"2024-09-10T23:37:27.583164+0000","last_peered":"2024-09-10T23:37:27.583164+0000","last_clean":"2024-09-10T23:37:27.583164+0000","last_became_active":"2024-09-10T23:37:25.568032+0000","last_became_peered":"2024-09-10T23:37:25.568032+0000","last_unstale":"2024-09-10T23:37:27.583164+0000","last_undegraded":"2024-09-10T23:37:27.583164+0000","last_fullsized":"2024-09-10T23:37:27.583164+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:52.305088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.597271+0000","last_change":"2024-09-10T23:37:26.593029+0000","last_active":"2024-09-10T23:37:26.597271+0000","last_peered":"2024-09-10T23:37:26.597271+0000","last_clean":"2024-09-10T23:37:26.597271+0000","last_became_active":"2024-09-10T23:37:26.592707+0000","last_became_peered":"2024-09-10T23:37:26.592707+0000","last_unstale":"2024-09-10T23:37:26.597271+0000","last_undegraded":"2024-09-10T23:37:26.597271+0000","last_fullsized":"2024-09-10T23:37:26.597271+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:17.694882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578367+0000","last_change":"2024-09-10T23:37:25.574164+0000","last_active":"2024-09-10T23:37:27.578367+0000","last_peered":"2024-09-10T23:37:27.578367+0000","last_clean":"2024-09-10T23:37:27.578367+0000","last_became_active":"2024-09-10T23:37:25.573880+0000","last_became_peered":"2024-09-10T23:37:25.573880+0000","last_unstale":"2024-09-10T23:37:27.578367+0000","last_undegraded":"2024-09-10T23:37:27.578367+0000","last_fullsized":"2024-09-10T23:37:27.578367+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:46.195152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578454+0000","last_change":"2024-09-10T23:37:26.575378+0000","last_active":"2024-09-10T23:37:27.578454+0000","last_peered":"2024-09-10T23:37:27.578454+0000","last_clean":"2024-09-10T23:37:27.578454+0000","last_became_active":"2024-09-10T23:37:26.575087+0000","last_became_peered":"2024-09-10T23:37:26.575087+0000","last_unstale":"2024-09-10T23:37:27.578454+0000","last_undegraded":"2024-09-10T23:37:27.578454+0000","last_fullsized":"2024-09-10T23:37:27.578454+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:28.905331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.574867+0000","last_change":"2024-09-10T23:37:25.577564+0000","last_active":"2024-09-10T23:37:26.574867+0000","last_peered":"2024-09-10T23:37:26.574867+0000","last_clean":"2024-09-10T23:37:26.574867+0000","last_became_active":"2024-09-10T23:37:25.577232+0000","last_became_peered":"2024-09-10T23:37:25.577232+0000","last_unstale":"2024-09-10T23:37:26.574867+0000","last_undegraded":"2024-09-10T23:37:26.574867+0000","last_fullsized":"2024-09-10T23:37:26.574867+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:52.529265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.574299+0000","last_change":"2024-09-10T23:37:26.568480+0000","last_active":"2024-09-10T23:37:26.574299+0000","last_peered":"2024-09-10T23:37:26.574299+0000","last_clean":"2024-09-10T23:37:26.574299+0000","last_became_active":"2024-09-10T23:37:26.567873+0000","last_became_peered":"2024-09-10T23:37:26.567873+0000","last_unstale":"2024-09-10T23:37:26.574299+0000","last_undegraded":"2024-09-10T23:37:26.574299+0000","last_fullsized":"2024-09-10T23:37:26.574299+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:37.843703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584564+0000","last_change":"2024-09-10T23:37:25.576592+0000","last_active":"2024-09-10T23:37:27.584564+0000","last_peered":"2024-09-10T23:37:27.584564+0000","last_clean":"2024-09-10T23:37:27.584564+0000","last_became_active":"2024-09-10T23:37:25.575766+0000","last_became_peered":"2024-09-10T23:37:25.575766+0000","last_unstale":"2024-09-10T23:37:27.584564+0000","last_undegraded":"2024-09-10T23:37:27.584564+0000","last_fullsized":"2024-09-10T23:37:27.584564+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:28:17.590140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.594972+0000","last_change":"2024-09-10T23:37:26.593226+0000","last_active":"2024-09-10T23:37:26.594972+0000","last_peered":"2024-09-10T23:37:26.594972+0000","last_clean":"2024-09-10T23:37:26.594972+0000","last_became_active":"2024-09-10T23:37:26.592775+0000","last_became_peered":"2024-09-10T23:37:26.592775+0000","last_unstale":"2024-09-10T23:37:26.594972+0000","last_undegraded":"2024-09-10T23:37:26.594972+0000","last_fullsized":"2024-09-10T23:37:26.594972+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:59.908076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.580106+0000","last_change":"2024-09-10T23:37:25.577535+0000","last_active":"2024-09-10T23:37:26.580106+0000","last_peered":"2024-09-10T23:37:26.580106+0000","last_clean":"2024-09-10T23:37:26.580106+0000","last_became_active":"2024-09-10T23:37:25.577252+0000","last_became_peered":"2024-09-10T23:37:25.577252+0000","last_unstale":"2024-09-10T23:37:26.580106+0000","last_undegraded":"2024-09-10T23:37:26.580106+0000","last_fullsized":"2024-09-10T23:37:26.580106+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:39.232432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.574917+0000","last_change":"2024-09-10T23:37:26.572237+0000","last_active":"2024-09-10T23:37:26.574917+0000","last_peered":"2024-09-10T23:37:26.574917+0000","last_clean":"2024-09-10T23:37:26.574917+0000","last_became_active":"2024-09-10T23:37:26.571958+0000","last_became_peered":"2024-09-10T23:37:26.571958+0000","last_unstale":"2024-09-10T23:37:26.574917+0000","last_undegraded":"2024-09-10T23:37:26.574917+0000","last_fullsized":"2024-09-10T23:37:26.574917+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:26.017918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"98'3","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.598333+0000","last_change":"2024-09-10T23:37:25.580213+0000","last_active":"2024-09-10T23:37:26.598333+0000","last_peered":"2024-09-10T23:37:26.598333+0000","last_clean":"2024-09-10T23:37:26.598333+0000","last_became_active":"2024-09-10T23:37:25.579912+0000","last_became_peered":"2024-09-10T23:37:25.579912+0000","last_unstale":"2024-09-10T23:37:26.598333+0000","last_undegraded":"2024-09-10T23:37:26.598333+0000","last_fullsized":"2024-09-10T23:37:26.598333+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:30.606174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.579017+0000","last_change":"2024-09-10T23:37:26.577142+0000","last_active":"2024-09-10T23:37:27.579017+0000","last_peered":"2024-09-10T23:37:27.579017+0000","last_clean":"2024-09-10T23:37:27.579017+0000","last_became_active":"2024-09-10T23:37:26.574767+0000","last_became_peered":"2024-09-10T23:37:26.574767+0000","last_unstale":"2024-09-10T23:37:27.579017+0000","last_undegraded":"2024-09-10T23:37:27.579017+0000","last_fullsized":"2024-09-10T23:37:27.579017+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:29.277753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"98'6","reported_seq":21,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584083+0000","last_change":"2024-09-10T23:37:25.576828+0000","last_active":"2024-09-10T23:37:27.584083+0000","last_peered":"2024-09-10T23:37:27.584083+0000","last_clean":"2024-09-10T23:37:27.584083+0000","last_became_active":"2024-09-10T23:37:25.576361+0000","last_became_peered":"2024-09-10T23:37:25.576361+0000","last_unstale":"2024-09-10T23:37:27.584083+0000","last_undegraded":"2024-09-10T23:37:27.584083+0000","last_fullsized":"2024-09-10T23:37:27.584083+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:58.905202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.592069+0000","last_change":"2024-09-10T23:37:26.589331+0000","last_active":"2024-09-10T23:37:26.592069+0000","last_peered":"2024-09-10T23:37:26.592069+0000","last_clean":"2024-09-10T23:37:26.592069+0000","last_became_active":"2024-09-10T23:37:26.588041+0000","last_became_peered":"2024-09-10T23:37:26.588041+0000","last_unstale":"2024-09-10T23:37:26.592069+0000","last_undegraded":"2024-09-10T23:37:26.592069+0000","last_fullsized":"2024-09-10T23:37:26.592069+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:40.408564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"98'2","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.595820+0000","last_change":"2024-09-10T23:37:25.579349+0000","last_active":"2024-09-10T23:37:26.595820+0000","last_peered":"2024-09-10T23:37:26.595820+0000","last_clean":"2024-09-10T23:37:26.595820+0000","last_became_active":"2024-09-10T23:37:25.578471+0000","last_became_peered":"2024-09-10T23:37:25.578471+0000","last_unstale":"2024-09-10T23:37:26.595820+0000","last_undegraded":"2024-09-10T23:37:26.595820+0000","last_fullsized":"2024-09-10T23:37:26.595820+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:18.725719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583149+0000","last_change":"2024-09-10T23:37:26.560176+0000","last_active":"2024-09-10T23:37:27.583149+0000","last_peered":"2024-09-10T23:37:27.583149+0000","last_clean":"2024-09-10T23:37:27.583149+0000","last_became_active":"2024-09-10T23:37:26.559916+0000","last_became_peered":"2024-09-10T23:37:26.559916+0000","last_unstale":"2024-09-10T23:37:27.583149+0000","last_undegraded":"2024-09-10T23:37:27.583149+0000","last_fullsized":"2024-09-10T23:37:27.583149+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:33.887526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.558061+0000","last_change":"2024-09-10T23:37:25.564537+0000","last_active":"2024-09-10T23:37:26.558061+0000","last_peered":"2024-09-10T23:37:26.558061+0000","last_clean":"2024-09-10T23:37:26.558061+0000","last_became_active":"2024-09-10T23:37:25.564273+0000","last_became_peered":"2024-09-10T23:37:25.564273+0000","last_unstale":"2024-09-10T23:37:26.558061+0000","last_undegraded":"2024-09-10T23:37:26.558061+0000","last_fullsized":"2024-09-10T23:37:26.558061+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:08.765602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.582335+0000","last_change":"2024-09-10T23:37:26.569821+0000","last_active":"2024-09-10T23:37:27.582335+0000","last_peered":"2024-09-10T23:37:27.582335+0000","last_clean":"2024-09-10T23:37:27.582335+0000","last_became_active":"2024-09-10T23:37:26.569396+0000","last_became_peered":"2024-09-10T23:37:26.569396+0000","last_unstale":"2024-09-10T23:37:27.582335+0000","last_undegraded":"2024-09-10T23:37:27.582335+0000","last_fullsized":"2024-09-10T23:37:27.582335+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:15:20.676151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"98'8","reported_seq":23,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.582824+0000","last_change":"2024-09-10T23:37:25.567845+0000","last_active":"2024-09-10T23:37:27.582824+0000","last_peered":"2024-09-10T23:37:27.582824+0000","last_clean":"2024-09-10T23:37:27.582824+0000","last_became_active":"2024-09-10T23:37:25.567562+0000","last_became_peered":"2024-09-10T23:37:25.567562+0000","last_unstale":"2024-09-10T23:37:27.582824+0000","last_undegraded":"2024-09-10T23:37:27.582824+0000","last_fullsized":"2024-09-10T23:37:27.582824+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:56.739863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583724+0000","last_change":"2024-09-10T23:37:26.575323+0000","last_active":"2024-09-10T23:37:27.583724+0000","last_peered":"2024-09-10T23:37:27.583724+0000","last_clean":"2024-09-10T23:37:27.583724+0000","last_became_active":"2024-09-10T23:37:26.575012+0000","last_became_peered":"2024-09-10T23:37:26.575012+0000","last_unstale":"2024-09-10T23:37:27.583724+0000","last_undegraded":"2024-09-10T23:37:27.583724+0000","last_fullsized":"2024-09-10T23:37:27.583724+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:43.438318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"98'1","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.586816+0000","last_change":"2024-09-10T23:37:25.570549+0000","last_active":"2024-09-10T23:37:26.586816+0000","last_peered":"2024-09-10T23:37:26.586816+0000","last_clean":"2024-09-10T23:37:26.586816+0000","last_became_active":"2024-09-10T23:37:25.570220+0000","last_became_peered":"2024-09-10T23:37:25.570220+0000","last_unstale":"2024-09-10T23:37:26.586816+0000","last_undegraded":"2024-09-10T23:37:26.586816+0000","last_fullsized":"2024-09-10T23:37:26.586816+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:57.188321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584305+0000","last_change":"2024-09-10T23:37:26.583855+0000","last_active":"2024-09-10T23:37:27.584305+0000","last_peered":"2024-09-10T23:37:27.584305+0000","last_clean":"2024-09-10T23:37:27.584305+0000","last_became_active":"2024-09-10T23:37:26.583558+0000","last_became_peered":"2024-09-10T23:37:26.583558+0000","last_unstale":"2024-09-10T23:37:27.584305+0000","last_undegraded":"2024-09-10T23:37:27.584305+0000","last_fullsized":"2024-09-10T23:37:27.584305+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:35.668760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"98'2","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.595791+0000","last_change":"2024-09-10T23:37:25.578477+0000","last_active":"2024-09-10T23:37:26.595791+0000","last_peered":"2024-09-10T23:37:26.595791+0000","last_clean":"2024-09-10T23:37:26.595791+0000","last_became_active":"2024-09-10T23:37:25.578094+0000","last_became_peered":"2024-09-10T23:37:25.578094+0000","last_unstale":"2024-09-10T23:37:26.595791+0000","last_undegraded":"2024-09-10T23:37:26.595791+0000","last_fullsized":"2024-09-10T23:37:26.595791+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:29.345192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578778+0000","last_change":"2024-09-10T23:37:26.571689+0000","last_active":"2024-09-10T23:37:27.578778+0000","last_peered":"2024-09-10T23:37:27.578778+0000","last_clean":"2024-09-10T23:37:27.578778+0000","last_became_active":"2024-09-10T23:37:26.571292+0000","last_became_peered":"2024-09-10T23:37:26.571292+0000","last_unstale":"2024-09-10T23:37:27.578778+0000","last_undegraded":"2024-09-10T23:37:27.578778+0000","last_fullsized":"2024-09-10T23:37:27.578778+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:39.091655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"98'1","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.588364+0000","last_change":"2024-09-10T23:37:25.567905+0000","last_active":"2024-09-10T23:37:26.588364+0000","last_peered":"2024-09-10T23:37:26.588364+0000","last_clean":"2024-09-10T23:37:26.588364+0000","last_became_active":"2024-09-10T23:37:25.567572+0000","last_became_peered":"2024-09-10T23:37:25.567572+0000","last_unstale":"2024-09-10T23:37:26.588364+0000","last_undegraded":"2024-09-10T23:37:26.588364+0000","last_fullsized":"2024-09-10T23:37:26.588364+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:13.807191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.571733+0000","last_change":"2024-09-10T23:37:26.567098+0000","last_active":"2024-09-10T23:37:26.571733+0000","last_peered":"2024-09-10T23:37:26.571733+0000","last_clean":"2024-09-10T23:37:26.571733+0000","last_became_active":"2024-09-10T23:37:26.566842+0000","last_became_peered":"2024-09-10T23:37:26.566842+0000","last_unstale":"2024-09-10T23:37:26.571733+0000","last_undegraded":"2024-09-10T23:37:26.571733+0000","last_fullsized":"2024-09-10T23:37:26.571733+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:17.323896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.553187+0000","last_change":"2024-09-10T23:37:25.572473+0000","last_active":"2024-09-10T23:37:26.553187+0000","last_peered":"2024-09-10T23:37:26.553187+0000","last_clean":"2024-09-10T23:37:26.553187+0000","last_became_active":"2024-09-10T23:37:25.572166+0000","last_became_peered":"2024-09-10T23:37:25.572166+0000","last_unstale":"2024-09-10T23:37:26.553187+0000","last_undegraded":"2024-09-10T23:37:26.553187+0000","last_fullsized":"2024-09-10T23:37:26.553187+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:11.968920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.567463+0000","last_change":"2024-09-10T23:37:26.565334+0000","last_active":"2024-09-10T23:37:26.567463+0000","last_peered":"2024-09-10T23:37:26.567463+0000","last_clean":"2024-09-10T23:37:26.567463+0000","last_became_active":"2024-09-10T23:37:26.564958+0000","last_became_peered":"2024-09-10T23:37:26.564958+0000","last_unstale":"2024-09-10T23:37:26.567463+0000","last_undegraded":"2024-09-10T23:37:26.567463+0000","last_fullsized":"2024-09-10T23:37:26.567463+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:47.479672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"98'2","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.592932+0000","last_change":"2024-09-10T23:37:25.579209+0000","last_active":"2024-09-10T23:37:26.592932+0000","last_peered":"2024-09-10T23:37:26.592932+0000","last_clean":"2024-09-10T23:37:26.592932+0000","last_became_active":"2024-09-10T23:37:25.578867+0000","last_became_peered":"2024-09-10T23:37:25.578867+0000","last_unstale":"2024-09-10T23:37:26.592932+0000","last_undegraded":"2024-09-10T23:37:26.592932+0000","last_fullsized":"2024-09-10T23:37:26.592932+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:26.084444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584230+0000","last_change":"2024-09-10T23:37:26.586650+0000","last_active":"2024-09-10T23:37:27.584230+0000","last_peered":"2024-09-10T23:37:27.584230+0000","last_clean":"2024-09-10T23:37:27.584230+0000","last_became_active":"2024-09-10T23:37:26.585199+0000","last_became_peered":"2024-09-10T23:37:26.585199+0000","last_unstale":"2024-09-10T23:37:27.584230+0000","last_undegraded":"2024-09-10T23:37:27.584230+0000","last_fullsized":"2024-09-10T23:37:27.584230+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:22.127741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583266+0000","last_change":"2024-09-10T23:37:25.572436+0000","last_active":"2024-09-10T23:37:27.583266+0000","last_peered":"2024-09-10T23:37:27.583266+0000","last_clean":"2024-09-10T23:37:27.583266+0000","last_became_active":"2024-09-10T23:37:25.572114+0000","last_became_peered":"2024-09-10T23:37:25.572114+0000","last_unstale":"2024-09-10T23:37:27.583266+0000","last_undegraded":"2024-09-10T23:37:27.583266+0000","last_fullsized":"2024-09-10T23:37:27.583266+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:33.537215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.571272+0000","last_change":"2024-09-10T23:37:26.567713+0000","last_active":"2024-09-10T23:37:26.571272+0000","last_peered":"2024-09-10T23:37:26.571272+0000","last_clean":"2024-09-10T23:37:26.571272+0000","last_became_active":"2024-09-10T23:37:26.567468+0000","last_became_peered":"2024-09-10T23:37:26.567468+0000","last_unstale":"2024-09-10T23:37:26.571272+0000","last_undegraded":"2024-09-10T23:37:26.571272+0000","last_fullsized":"2024-09-10T23:37:26.571272+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:15.911072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.582474+0000","last_change":"2024-09-10T23:37:25.571999+0000","last_active":"2024-09-10T23:37:27.582474+0000","last_peered":"2024-09-10T23:37:27.582474+0000","last_clean":"2024-09-10T23:37:27.582474+0000","last_became_active":"2024-09-10T23:37:25.571738+0000","last_became_peered":"2024-09-10T23:37:25.571738+0000","last_unstale":"2024-09-10T23:37:27.582474+0000","last_undegraded":"2024-09-10T23:37:27.582474+0000","last_fullsized":"2024-09-10T23:37:27.582474+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:57.188321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.597946+0000","last_change":"2024-09-10T23:37:26.596316+0000","last_active":"2024-09-10T23:37:26.597946+0000","last_peered":"2024-09-10T23:37:26.597946+0000","last_clean":"2024-09-10T23:37:26.597946+0000","last_became_active":"2024-09-10T23:37:26.596058+0000","last_became_peered":"2024-09-10T23:37:26.596058+0000","last_unstale":"2024-09-10T23:37:26.597946+0000","last_undegraded":"2024-09-10T23:37:26.597946+0000","last_fullsized":"2024-09-10T23:37:26.597946+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:43.341222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.582278+0000","last_change":"2024-09-10T23:37:25.568933+0000","last_active":"2024-09-10T23:37:27.582278+0000","last_peered":"2024-09-10T23:37:27.582278+0000","last_clean":"2024-09-10T23:37:27.582278+0000","last_became_active":"2024-09-10T23:37:25.565677+0000","last_became_peered":"2024-09-10T23:37:25.565677+0000","last_unstale":"2024-09-10T23:37:27.582278+0000","last_undegraded":"2024-09-10T23:37:27.582278+0000","last_fullsized":"2024-09-10T23:37:27.582278+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:36.840828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.570403+0000","last_change":"2024-09-10T23:37:26.566541+0000","last_active":"2024-09-10T23:37:26.570403+0000","last_peered":"2024-09-10T23:37:26.570403+0000","last_clean":"2024-09-10T23:37:26.570403+0000","last_became_active":"2024-09-10T23:37:26.566241+0000","last_became_peered":"2024-09-10T23:37:26.566241+0000","last_unstale":"2024-09-10T23:37:26.570403+0000","last_undegraded":"2024-09-10T23:37:26.570403+0000","last_fullsized":"2024-09-10T23:37:26.570403+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:15.703723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"98'2","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584551+0000","last_change":"2024-09-10T23:37:25.576595+0000","last_active":"2024-09-10T23:37:27.584551+0000","last_peered":"2024-09-10T23:37:27.584551+0000","last_clean":"2024-09-10T23:37:27.584551+0000","last_became_active":"2024-09-10T23:37:25.575857+0000","last_became_peered":"2024-09-10T23:37:25.575857+0000","last_unstale":"2024-09-10T23:37:27.584551+0000","last_undegraded":"2024-09-10T23:37:27.584551+0000","last_fullsized":"2024-09-10T23:37:27.584551+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:47:32.605533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.575236+0000","last_change":"2024-09-10T23:37:26.571603+0000","last_active":"2024-09-10T23:37:26.575236+0000","last_peered":"2024-09-10T23:37:26.575236+0000","last_clean":"2024-09-10T23:37:26.575236+0000","last_became_active":"2024-09-10T23:37:26.571247+0000","last_became_peered":"2024-09-10T23:37:26.571247+0000","last_unstale":"2024-09-10T23:37:26.575236+0000","last_undegraded":"2024-09-10T23:37:26.575236+0000","last_fullsized":"2024-09-10T23:37:26.575236+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:04.526930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578754+0000","last_change":"2024-09-10T23:37:25.568397+0000","last_active":"2024-09-10T23:37:27.578754+0000","last_peered":"2024-09-10T23:37:27.578754+0000","last_clean":"2024-09-10T23:37:27.578754+0000","last_became_active":"2024-09-10T23:37:25.568069+0000","last_became_peered":"2024-09-10T23:37:25.568069+0000","last_unstale":"2024-09-10T23:37:27.578754+0000","last_undegraded":"2024-09-10T23:37:27.578754+0000","last_fullsized":"2024-09-10T23:37:27.578754+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:48.534455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584123+0000","last_change":"2024-09-10T23:37:26.590474+0000","last_active":"2024-09-10T23:37:27.584123+0000","last_peered":"2024-09-10T23:37:27.584123+0000","last_clean":"2024-09-10T23:37:27.584123+0000","last_became_active":"2024-09-10T23:37:26.590189+0000","last_became_peered":"2024-09-10T23:37:26.590189+0000","last_unstale":"2024-09-10T23:37:27.584123+0000","last_undegraded":"2024-09-10T23:37:27.584123+0000","last_fullsized":"2024-09-10T23:37:27.584123+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:31.790594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583463+0000","last_change":"2024-09-10T23:37:25.565291+0000","last_active":"2024-09-10T23:37:27.583463+0000","last_peered":"2024-09-10T23:37:27.583463+0000","last_clean":"2024-09-10T23:37:27.583463+0000","last_became_active":"2024-09-10T23:37:25.565000+0000","last_became_peered":"2024-09-10T23:37:25.565000+0000","last_unstale":"2024-09-10T23:37:27.583463+0000","last_undegraded":"2024-09-10T23:37:27.583463+0000","last_fullsized":"2024-09-10T23:37:27.583463+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:07.561326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.571770+0000","last_change":"2024-09-10T23:37:26.569739+0000","last_active":"2024-09-10T23:37:26.571770+0000","last_peered":"2024-09-10T23:37:26.571770+0000","last_clean":"2024-09-10T23:37:26.571770+0000","last_became_active":"2024-09-10T23:37:26.569482+0000","last_became_peered":"2024-09-10T23:37:26.569482+0000","last_unstale":"2024-09-10T23:37:26.571770+0000","last_undegraded":"2024-09-10T23:37:26.571770+0000","last_fullsized":"2024-09-10T23:37:26.571770+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:45.411596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.584183+0000","last_change":"2024-09-10T23:37:25.573201+0000","last_active":"2024-09-10T23:37:27.584183+0000","last_peered":"2024-09-10T23:37:27.584183+0000","last_clean":"2024-09-10T23:37:27.584183+0000","last_became_active":"2024-09-10T23:37:25.572274+0000","last_became_peered":"2024-09-10T23:37:25.572274+0000","last_unstale":"2024-09-10T23:37:27.584183+0000","last_undegraded":"2024-09-10T23:37:27.584183+0000","last_fullsized":"2024-09-10T23:37:27.584183+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:07.147512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.573028+0000","last_change":"2024-09-10T23:37:26.569607+0000","last_active":"2024-09-10T23:37:26.573028+0000","last_peered":"2024-09-10T23:37:26.573028+0000","last_clean":"2024-09-10T23:37:26.573028+0000","last_became_active":"2024-09-10T23:37:26.569378+0000","last_became_peered":"2024-09-10T23:37:26.569378+0000","last_unstale":"2024-09-10T23:37:26.573028+0000","last_undegraded":"2024-09-10T23:37:26.573028+0000","last_fullsized":"2024-09-10T23:37:26.573028+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:17:06.903609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.578595+0000","last_change":"2024-09-10T23:37:25.566534+0000","last_active":"2024-09-10T23:37:27.578595+0000","last_peered":"2024-09-10T23:37:27.578595+0000","last_clean":"2024-09-10T23:37:27.578595+0000","last_became_active":"2024-09-10T23:37:25.565243+0000","last_became_peered":"2024-09-10T23:37:25.565243+0000","last_unstale":"2024-09-10T23:37:27.578595+0000","last_undegraded":"2024-09-10T23:37:27.578595+0000","last_fullsized":"2024-09-10T23:37:27.578595+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:16:27.374539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.595061+0000","last_change":"2024-09-10T23:37:26.594268+0000","last_active":"2024-09-10T23:37:26.595061+0000","last_peered":"2024-09-10T23:37:26.595061+0000","last_clean":"2024-09-10T23:37:26.595061+0000","last_became_active":"2024-09-10T23:37:26.593921+0000","last_became_peered":"2024-09-10T23:37:26.593921+0000","last_unstale":"2024-09-10T23:37:26.595061+0000","last_undegraded":"2024-09-10T23:37:26.595061+0000","last_fullsized":"2024-09-10T23:37:26.595061+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:41.033610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.553574+0000","last_change":"2024-09-10T23:37:25.569342+0000","last_active":"2024-09-10T23:37:26.553574+0000","last_peered":"2024-09-10T23:37:26.553574+0000","last_clean":"2024-09-10T23:37:26.553574+0000","last_became_active":"2024-09-10T23:37:25.569030+0000","last_became_peered":"2024-09-10T23:37:25.569030+0000","last_unstale":"2024-09-10T23:37:26.553574+0000","last_undegraded":"2024-09-10T23:37:26.553574+0000","last_fullsized":"2024-09-10T23:37:26.553574+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:31.571574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.567937+0000","last_change":"2024-09-10T23:37:26.564083+0000","last_active":"2024-09-10T23:37:26.567937+0000","last_peered":"2024-09-10T23:37:26.567937+0000","last_clean":"2024-09-10T23:37:26.567937+0000","last_became_active":"2024-09-10T23:37:26.563810+0000","last_became_peered":"2024-09-10T23:37:26.563810+0000","last_unstale":"2024-09-10T23:37:26.567937+0000","last_undegraded":"2024-09-10T23:37:26.567937+0000","last_fullsized":"2024-09-10T23:37:26.567937+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:07.710400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.583597+0000","last_change":"2024-09-10T23:37:25.568287+0000","last_active":"2024-09-10T23:37:27.583597+0000","last_peered":"2024-09-10T23:37:27.583597+0000","last_clean":"2024-09-10T23:37:27.583597+0000","last_became_active":"2024-09-10T23:37:25.567908+0000","last_became_peered":"2024-09-10T23:37:25.567908+0000","last_unstale":"2024-09-10T23:37:27.583597+0000","last_undegraded":"2024-09-10T23:37:27.583597+0000","last_fullsized":"2024-09-10T23:37:27.583597+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:42.198784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.571054+0000","last_change":"2024-09-10T23:37:26.568353+0000","last_active":"2024-09-10T23:37:26.571054+0000","last_peered":"2024-09-10T23:37:26.571054+0000","last_clean":"2024-09-10T23:37:26.571054+0000","last_became_active":"2024-09-10T23:37:26.567857+0000","last_became_peered":"2024-09-10T23:37:26.567857+0000","last_unstale":"2024-09-10T23:37:26.571054+0000","last_undegraded":"2024-09-10T23:37:26.571054+0000","last_fullsized":"2024-09-10T23:37:26.571054+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:24.051687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.564851+0000","last_change":"2024-09-10T23:37:25.570114+0000","last_active":"2024-09-10T23:37:26.564851+0000","last_peered":"2024-09-10T23:37:26.564851+0000","last_clean":"2024-09-10T23:37:26.564851+0000","last_became_active":"2024-09-10T23:37:25.569763+0000","last_became_peered":"2024-09-10T23:37:25.569763+0000","last_unstale":"2024-09-10T23:37:26.564851+0000","last_undegraded":"2024-09-10T23:37:26.564851+0000","last_fullsized":"2024-09-10T23:37:26.564851+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:18:28.119346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T23:37:27.582243+0000","last_change":"2024-09-10T23:37:26.569887+0000","last_active":"2024-09-10T23:37:27.582243+0000","last_peered":"2024-09-10T23:37:27.582243+0000","last_clean":"2024-09-10T23:37:27.582243+0000","last_became_active":"2024-09-10T23:37:26.569594+0000","last_became_peered":"2024-09-10T23:37:26.569594+0000","last_unstale":"2024-09-10T23:37:27.582243+0000","last_undegraded":"2024-09-10T23:37:27.582243+0000","last_fullsized":"2024-09-10T23:37:27.582243+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:25.540863+0000","last_clean_scrub_stamp":"2024-09-10T23:37:25.540863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:33.373338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.562574+0000","last_change":"2024-09-10T23:37:25.573222+0000","last_active":"2024-09-10T23:37:26.562574+0000","last_peered":"2024-09-10T23:37:26.562574+0000","last_clean":"2024-09-10T23:37:26.562574+0000","last_became_active":"2024-09-10T23:37:25.572806+0000","last_became_peered":"2024-09-10T23:37:25.572806+0000","last_unstale":"2024-09-10T23:37:26.562574+0000","last_undegraded":"2024-09-10T23:37:26.562574+0000","last_fullsized":"2024-09-10T23:37:26.562574+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:37:24.537988+0000","last_clean_scrub_stamp":"2024-09-10T23:37:24.537988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:04.445805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":278,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T23:37:26.557967+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:37:26.557967+0000","last_peered":"2024-09-10T23:37:26.557967+0000","last_clean":"2024-09-10T23:37:26.557967+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:37:26.557967+0000","last_undegraded":"2024-09-10T23:37:26.557967+0000","last_fullsized":"2024-09-10T23:37:26.557967+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509505,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1232896,"data_stored":528054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542208,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":526318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509504,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32960,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338880,"statfs":{"total":96636764160,"available":96603013120,"internally_reserved":0,"allocated":1814528,"data_stored":1116778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509502,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1228800,"data_stored":528020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509503,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1212416,"data_stored":526318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509504,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1224704,"data_stored":526956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509503,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32952,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338888,"statfs":{"total":96636764160,"available":96603021312,"internally_reserved":0,"allocated":1806336,"data_stored":1116686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509503,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1220608,"data_stored":526970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:37:31.072 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T23:37:31.072 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T23:37:31.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f844e000640 1 -- 172.21.15.94:0/4108266610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f842c004810 msgr2=0x7f842c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:31.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f844e000640 1 --2- 172.21.15.94:0/4108266610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f842c004810 0x7f842c004c90 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f8430006fd0 tx=0x7f843001cfa0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f8447e00640 1 -- 172.21.15.94:0/4108266610 reap_dead start 2024-09-10T23:37:31.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f844e000640 1 -- 172.21.15.94:0/4108266610 shutdown_connections 2024-09-10T23:37:31.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f844e000640 1 -- 172.21.15.94:0/4108266610 >> 172.21.15.94:0/4108266610 conn(0x7f84480d1c60 msgr2=0x7f84480d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:31.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f844e000640 1 -- 172.21.15.94:0/4108266610 shutdown_connections 2024-09-10T23:37:31.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f844e000640 1 -- 172.21.15.94:0/4108266610 wait complete. 2024-09-10T23:37:31.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f844e000640 1 Processor -- start 2024-09-10T23:37:31.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f844e000640 1 -- start start 2024-09-10T23:37:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f844e000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f842c004810 0x7f844814af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f844e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f844806ef80 0x7f844806f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f844e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84481fa540 0x7f84481fa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f844e000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f84480c6a80 con 0x7f84481fa540 2024-09-10T23:37:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f844e000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f84480c6900 con 0x7f844806ef80 2024-09-10T23:37:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f844e000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f84480c6c00 con 0x7f842c004810 2024-09-10T23:37:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f8447e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84481fa540 0x7f84481fa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f8447e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84481fa540 0x7f84481fa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60698/0 (socket says 172.21.15.94:60698) 2024-09-10T23:37:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f8447e00640 1 -- 172.21.15.94:0/796392295 learned_addr learned my addr 172.21.15.94:0/796392295 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f8447400640 1 --2- 172.21.15.94:0/796392295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f842c004810 0x7f844814af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f843ea00640 1 --2- 172.21.15.94:0/796392295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f844806ef80 0x7f844806f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f8447e00640 1 -- 172.21.15.94:0/796392295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f842c004810 msgr2=0x7f844814af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f8447e00640 1 --2- 172.21.15.94:0/796392295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f842c004810 0x7f844814af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f8447e00640 1 -- 172.21.15.94:0/796392295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f844806ef80 msgr2=0x7f844806f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f8447e00640 1 --2- 172.21.15.94:0/796392295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f844806ef80 0x7f844806f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f8447e00640 1 -- 172.21.15.94:0/796392295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84481fb2a0 con 0x7f84481fa540 2024-09-10T23:37:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.257+0000 7f8447e00640 1 --2- 172.21.15.94:0/796392295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84481fa540 0x7f84481fa9c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f84480d1c30 tx=0x7f844000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.261+0000 7f843fe00640 1 -- 172.21.15.94:0/796392295 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440003980 con 0x7f84481fa540 2024-09-10T23:37:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.261+0000 7f843fe00640 1 -- 172.21.15.94:0/796392295 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8440003b20 con 0x7f84481fa540 2024-09-10T23:37:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.261+0000 7f843fe00640 1 -- 172.21.15.94:0/796392295 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440004030 con 0x7f84481fa540 2024-09-10T23:37:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.261+0000 7f844e000640 1 -- 172.21.15.94:0/796392295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84480ae980 con 0x7f84481fa540 2024-09-10T23:37:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.261+0000 7f844e000640 1 -- 172.21.15.94:0/796392295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84480aef40 con 0x7f84481fa540 2024-09-10T23:37:31.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.261+0000 7f844e000640 1 -- 172.21.15.94:0/796392295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f844806f940 con 0x7f84481fa540 2024-09-10T23:37:31.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.261+0000 7f843fe00640 1 -- 172.21.15.94:0/796392295 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f8440018070 con 0x7f84481fa540 2024-09-10T23:37:31.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.261+0000 7f843fe00640 1 --2- 172.21.15.94:0/796392295 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8424072730 0x7f8424074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:31.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.261+0000 7f843fe00640 1 -- 172.21.15.94:0/796392295 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f8440093e20 con 0x7f84481fa540 2024-09-10T23:37:31.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.261+0000 7f8447400640 1 --2- 172.21.15.94:0/796392295 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8424072730 0x7f8424074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:31.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.265+0000 7f8447400640 1 --2- 172.21.15.94:0/796392295 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8424072730 0x7f8424074bf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8438009ba0 tx=0x7f84380091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:31.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.265+0000 7f843fe00640 1 -- 172.21.15.94:0/796392295 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f844005f2f0 con 0x7f84481fa540 2024-09-10T23:37:31.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.485+0000 7f844e000640 1 -- 172.21.15.94:0/796392295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f84480d2780 con 0x7f84481fa540 2024-09-10T23:37:31.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.489+0000 7f843fe00640 1 -- 172.21.15.94:0/796392295 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v35) ==== 190+0+147 (secure 0 0 0) 0x7f844005ebf0 con 0x7f84481fa540 2024-09-10T23:37:31.492 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-10T23:37:31.492 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g== 2024-09-10T23:37:31.492 INFO:teuthology.orchestra.run.smithi094.stdout: caps mds = "allow" 2024-09-10T23:37:31.492 INFO:teuthology.orchestra.run.smithi094.stdout: caps mon = "allow r" 2024-09-10T23:37:31.492 INFO:teuthology.orchestra.run.smithi094.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T23:37:31.493 INFO:teuthology.orchestra.run.smithi094.stderr:updated caps for client.0 2024-09-10T23:37:31.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.493+0000 7f844e000640 1 -- 172.21.15.94:0/796392295 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8424072730 msgr2=0x7f8424074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:31.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.493+0000 7f844e000640 1 --2- 172.21.15.94:0/796392295 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8424072730 0x7f8424074bf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8438009ba0 tx=0x7f84380091e0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.493+0000 7f844e000640 1 -- 172.21.15.94:0/796392295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84481fa540 msgr2=0x7f84481fa9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:31.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.493+0000 7f844e000640 1 --2- 172.21.15.94:0/796392295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84481fa540 0x7f84481fa9c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f84480d1c30 tx=0x7f844000f3c0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.493+0000 7f844e000640 1 -- 172.21.15.94:0/796392295 shutdown_connections 2024-09-10T23:37:31.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.493+0000 7f844e000640 1 --2- 172.21.15.94:0/796392295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f844806ef80 0x7f844806f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.493+0000 7f844e000640 1 --2- 172.21.15.94:0/796392295 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8424072730 0x7f8424074bf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.493+0000 7f844e000640 1 --2- 172.21.15.94:0/796392295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84481fa540 0x7f84481fa9c0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.493+0000 7f844e000640 1 --2- 172.21.15.94:0/796392295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f842c004810 0x7f844814af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.493+0000 7f844e000640 1 -- 172.21.15.94:0/796392295 >> 172.21.15.94:0/796392295 conn(0x7f84480d1c60 msgr2=0x7f844815a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:31.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.493+0000 7f844e000640 1 -- 172.21.15.94:0/796392295 shutdown_connections 2024-09-10T23:37:31.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.493+0000 7f844e000640 1 -- 172.21.15.94:0/796392295 wait complete. 2024-09-10T23:37:31.513 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:37:31.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.653+0000 7f2cdbe00640 1 -- 172.21.15.94:0/3012236221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd4071a20 msgr2=0x7f2cd415f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:31.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.653+0000 7f2cdbe00640 1 --2- 172.21.15.94:0/3012236221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd4071a20 0x7f2cd415f610 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40b6ca0 tx=0x7f2cc001c810 comp rx=0 tx=0).stop 2024-09-10T23:37:31.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.653+0000 7f2cdbe00640 1 -- 172.21.15.94:0/3012236221 shutdown_connections 2024-09-10T23:37:31.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.653+0000 7f2cdbe00640 1 --2- 172.21.15.94:0/3012236221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cd40d1690 0x7f2cd415fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.653+0000 7f2cdbe00640 1 --2- 172.21.15.94:0/3012236221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd4071a20 0x7f2cd415f610 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.653+0000 7f2cdbe00640 1 --2- 172.21.15.94:0/3012236221 >> v2:172.21.15.181:3300/0 conn(0x7f2cd406cc20 0x7f2cd406d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.653+0000 7f2cdbe00640 1 -- 172.21.15.94:0/3012236221 >> 172.21.15.94:0/3012236221 conn(0x7f2cd40d0b70 msgr2=0x7f2cd40d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:31.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.653+0000 7f2cdbe00640 1 -- 172.21.15.94:0/3012236221 shutdown_connections 2024-09-10T23:37:31.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.653+0000 7f2cdbe00640 1 -- 172.21.15.94:0/3012236221 wait complete. 2024-09-10T23:37:31.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cdbe00640 1 Processor -- start 2024-09-10T23:37:31.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cdbe00640 1 -- start start 2024-09-10T23:37:31.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cdbe00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cd406cc20 0x7f2cd4155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:31.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cdbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd4071a20 0x7f2cd4158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:31.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cdbe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd40d1690 0x7f2cd40705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:31.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cdbe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd40cff30 con 0x7f2cd4071a20 2024-09-10T23:37:31.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cdbe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd40cfdb0 con 0x7f2cd40d1690 2024-09-10T23:37:31.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cdbe00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd40d00b0 con 0x7f2cd406cc20 2024-09-10T23:37:31.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cd8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd4071a20 0x7f2cd4158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:31.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cd8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd4071a20 0x7f2cd4158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60726/0 (socket says 172.21.15.94:60726) 2024-09-10T23:37:31.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cd8a00640 1 -- 172.21.15.94:0/1218417925 learned_addr learned my addr 172.21.15.94:0/1218417925 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:31.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cd9e00640 1 --2- 172.21.15.94:0/1218417925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd40d1690 0x7f2cd40705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:31.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cd9400640 1 --2- 172.21.15.94:0/1218417925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cd406cc20 0x7f2cd4155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:31.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cd8a00640 1 -- 172.21.15.94:0/1218417925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cd406cc20 msgr2=0x7f2cd4155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:31.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cd8a00640 1 --2- 172.21.15.94:0/1218417925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cd406cc20 0x7f2cd4155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cd8a00640 1 -- 172.21.15.94:0/1218417925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd40d1690 msgr2=0x7f2cd40705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:31.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cd8a00640 1 --2- 172.21.15.94:0/1218417925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd40d1690 0x7f2cd40705e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cd8a00640 1 -- 172.21.15.94:0/1218417925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd406f540 con 0x7f2cd4071a20 2024-09-10T23:37:31.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cd9400640 1 --2- 172.21.15.94:0/1218417925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cd406cc20 0x7f2cd4155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:31.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cd8a00640 1 --2- 172.21.15.94:0/1218417925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd4071a20 0x7f2cd4158dc0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2cc001ccf0 tx=0x7f2cc0005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:31.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cd1600640 1 -- 172.21.15.94:0/1218417925 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00ac070 con 0x7f2cd4071a20 2024-09-10T23:37:31.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cd1600640 1 -- 172.21.15.94:0/1218417925 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cc0004310 con 0x7f2cd4071a20 2024-09-10T23:37:31.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cdbe00640 1 -- 172.21.15.94:0/1218417925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd406f820 con 0x7f2cd4071a20 2024-09-10T23:37:31.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cd1600640 1 -- 172.21.15.94:0/1218417925 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0005290 con 0x7f2cd4071a20 2024-09-10T23:37:31.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.657+0000 7f2cdbe00640 1 -- 172.21.15.94:0/1218417925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd406fde0 con 0x7f2cd4071a20 2024-09-10T23:37:31.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.661+0000 7f2cdbe00640 1 -- 172.21.15.94:0/1218417925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd40d05f0 con 0x7f2cd4071a20 2024-09-10T23:37:31.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.661+0000 7f2cd1600640 1 -- 172.21.15.94:0/1218417925 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f2cc00b8020 con 0x7f2cd4071a20 2024-09-10T23:37:31.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.661+0000 7f2cd1600640 1 --2- 172.21.15.94:0/1218417925 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2cb4072700 0x7f2cb4074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:31.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.661+0000 7f2cd1600640 1 -- 172.21.15.94:0/1218417925 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f2cc012d6a0 con 0x7f2cd4071a20 2024-09-10T23:37:31.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.661+0000 7f2cd9400640 1 --2- 172.21.15.94:0/1218417925 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2cb4072700 0x7f2cb4074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:31.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.661+0000 7f2cd9400640 1 --2- 172.21.15.94:0/1218417925 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2cb4072700 0x7f2cb4074bc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8005e10 tx=0x7f2cc8005d60 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:31.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.665+0000 7f2cd1600640 1 -- 172.21.15.94:0/1218417925 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2cc00bf030 con 0x7f2cd4071a20 2024-09-10T23:37:31.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.865+0000 7f2cdbe00640 1 -- 172.21.15.94:0/1218417925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cd4150950 con 0x7f2cd4071a20 2024-09-10T23:37:31.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.869+0000 7f2cd1600640 1 -- 172.21.15.94:0/1218417925 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5243 (secure 0 0 0) 0x7f2cc0016490 con 0x7f2cd4071a20 2024-09-10T23:37:31.870 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:31.870 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":68,"btime":"2024-09-10T23:37:29:587275+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7011,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1105858824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1105858824},{"type":"v1","addr":"172.21.15.94:6834","nonce":1105858824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3277955689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3277955689},{"type":"v1","addr":"172.21.15.94:6835","nonce":3277955689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":195947,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/2607191387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2607191387},{"type":"v1","addr":"172.21.15.181:6801","nonce":2607191387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:37:25.941783+0000","modified":"2024-09-10T23:37:28.978060+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195950},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195950":{"gid":195950,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/2513770511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2513770511},{"type":"v1","addr":"172.21.15.181:6835","nonce":2513770511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195950,"qdb_cluster":[195950]},"id":8}]} 2024-09-10T23:37:31.870 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 68 2024-09-10T23:37:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.869+0000 7f2cdbe00640 1 -- 172.21.15.94:0/1218417925 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2cb4072700 msgr2=0x7f2cb4074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.869+0000 7f2cdbe00640 1 --2- 172.21.15.94:0/1218417925 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2cb4072700 0x7f2cb4074bc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8005e10 tx=0x7f2cc8005d60 comp rx=0 tx=0).stop 2024-09-10T23:37:31.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.869+0000 7f2cdbe00640 1 -- 172.21.15.94:0/1218417925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd4071a20 msgr2=0x7f2cd4158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.869+0000 7f2cdbe00640 1 --2- 172.21.15.94:0/1218417925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd4071a20 0x7f2cd4158dc0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2cc001ccf0 tx=0x7f2cc0005e90 comp rx=0 tx=0).stop 2024-09-10T23:37:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.869+0000 7f2cdbe00640 1 -- 172.21.15.94:0/1218417925 shutdown_connections 2024-09-10T23:37:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.869+0000 7f2cdbe00640 1 --2- 172.21.15.94:0/1218417925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd40d1690 0x7f2cd40705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.869+0000 7f2cdbe00640 1 --2- 172.21.15.94:0/1218417925 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2cb4072700 0x7f2cb4074bc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.869+0000 7f2cdbe00640 1 --2- 172.21.15.94:0/1218417925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cd4071a20 0x7f2cd4158dc0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.869+0000 7f2cdbe00640 1 --2- 172.21.15.94:0/1218417925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cd406cc20 0x7f2cd4155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.869+0000 7f2cdbe00640 1 -- 172.21.15.94:0/1218417925 >> 172.21.15.94:0/1218417925 conn(0x7f2cd40d0b70 msgr2=0x7f2cd414f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.869+0000 7f2cdbe00640 1 -- 172.21.15.94:0/1218417925 shutdown_connections 2024-09-10T23:37:31.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:31.869+0000 7f2cdbe00640 1 -- 172.21.15.94:0/1218417925 wait complete. 2024-09-10T23:37:32.294 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T23:37:25.941783+0000', 'modified': '2024-09-10T23:37:28.978060+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 195950}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_195950': {'gid': 195950, 'name': 'd', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6835/2513770511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 2513770511}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 2513770511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 195950, 'qdb_cluster': [195950]} 2024-09-10T23:37:32.294 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T23:37:32.294 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T23:37:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.433+0000 7f13f6a00640 1 -- 172.21.15.94:0/1463255513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13f00c5a70 msgr2=0x7f13f00cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.433+0000 7f13f6a00640 1 --2- 172.21.15.94:0/1463255513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13f00c5a70 0x7f13f00cd9f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f13f006f970 tx=0x7f13e801cb70 comp rx=0 tx=0).stop 2024-09-10T23:37:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.433+0000 7f13f6a00640 1 -- 172.21.15.94:0/1463255513 shutdown_connections 2024-09-10T23:37:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.433+0000 7f13f6a00640 1 --2- 172.21.15.94:0/1463255513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13f00c5a70 0x7f13f00cd9f0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.433+0000 7f13f6a00640 1 --2- 172.21.15.94:0/1463255513 >> v2:172.21.15.181:3300/0 conn(0x7f13f00d0710 0x7f13f00c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.433+0000 7f13f6a00640 1 -- 172.21.15.94:0/1463255513 >> 172.21.15.94:0/1463255513 conn(0x7f13f00cfaf0 msgr2=0x7f13f00cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.433+0000 7f13f6a00640 1 -- 172.21.15.94:0/1463255513 shutdown_connections 2024-09-10T23:37:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.433+0000 7f13f6a00640 1 -- 172.21.15.94:0/1463255513 wait complete. 2024-09-10T23:37:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13f6a00640 1 Processor -- start 2024-09-10T23:37:32.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13f6a00640 1 -- start start 2024-09-10T23:37:32.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13f6a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13f00c5a70 0x7f13f00afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13f6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13f00d0710 0x7f13f00cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13f6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13f01455d0 0x7f13f0073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13f6a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f13f0169400 con 0x7f13f00d0710 2024-09-10T23:37:32.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13f6a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f13f0169280 con 0x7f13f01455d0 2024-09-10T23:37:32.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13f6a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f13f0169580 con 0x7f13f00c5a70 2024-09-10T23:37:32.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13e7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13f00d0710 0x7f13f00cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13e7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13f00d0710 0x7f13f00cbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60746/0 (socket says 172.21.15.94:60746) 2024-09-10T23:37:32.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13e7400640 1 -- 172.21.15.94:0/139127684 learned_addr learned my addr 172.21.15.94:0/139127684 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:32.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13f4a00640 1 --2- 172.21.15.94:0/139127684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13f01455d0 0x7f13f0073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13efe00640 1 --2- 172.21.15.94:0/139127684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13f00c5a70 0x7f13f00afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13e7400640 1 -- 172.21.15.94:0/139127684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13f00c5a70 msgr2=0x7f13f00afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:32.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13e7400640 1 --2- 172.21.15.94:0/139127684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13f00c5a70 0x7f13f00afe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13e7400640 1 -- 172.21.15.94:0/139127684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13f01455d0 msgr2=0x7f13f0073cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:32.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13e7400640 1 --2- 172.21.15.94:0/139127684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13f01455d0 0x7f13f0073cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13e7400640 1 -- 172.21.15.94:0/139127684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f00af7d0 con 0x7f13f00d0710 2024-09-10T23:37:32.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13f4a00640 1 --2- 172.21.15.94:0/139127684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13f01455d0 0x7f13f0073cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:32.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13e7400640 1 --2- 172.21.15.94:0/139127684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13f00d0710 0x7f13f00cbd60 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f13d800d980 tx=0x7f13d800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:32.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13ecc00640 1 -- 172.21.15.94:0/139127684 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8017070 con 0x7f13f00d0710 2024-09-10T23:37:32.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13ecc00640 1 -- 172.21.15.94:0/139127684 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13d8009d60 con 0x7f13f00d0710 2024-09-10T23:37:32.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13ecc00640 1 -- 172.21.15.94:0/139127684 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8005280 con 0x7f13f00d0710 2024-09-10T23:37:32.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13f6a00640 1 -- 172.21.15.94:0/139127684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f00b04d0 con 0x7f13f00d0710 2024-09-10T23:37:32.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.437+0000 7f13f6a00640 1 -- 172.21.15.94:0/139127684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f0140cf0 con 0x7f13f00d0710 2024-09-10T23:37:32.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.441+0000 7f13ecc00640 1 -- 172.21.15.94:0/139127684 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f13d8005420 con 0x7f13f00d0710 2024-09-10T23:37:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.441+0000 7f13ecc00640 1 --2- 172.21.15.94:0/139127684 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f13cc072730 0x7f13cc074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.441+0000 7f13ecc00640 1 -- 172.21.15.94:0/139127684 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f13d8093730 con 0x7f13f00d0710 2024-09-10T23:37:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.441+0000 7f13f9dab000 1 Processor -- start 2024-09-10T23:37:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.441+0000 7f13efe00640 1 --2- 172.21.15.94:0/139127684 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f13cc072730 0x7f13cc074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.441+0000 7f13f9dab000 1 -- start start 2024-09-10T23:37:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.441+0000 7f13efe00640 1 --2- 172.21.15.94:0/139127684 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f13cc072730 0x7f13cc074bf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f13e000b810 tx=0x7f13e0008040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.441+0000 7f13f9dab000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073379d0 0x55b507337db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.441+0000 7f13f9dab000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073382f0 0x55b5072eb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.441+0000 7f13f9dab000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5072ec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.441+0000 7f13f9dab000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b50720c3f0 con 0x55b5072ebe10 2024-09-10T23:37:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.441+0000 7f13f9dab000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b507149b20 con 0x55b5073379d0 2024-09-10T23:37:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.441+0000 7f13f9dab000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x55b507138f50 con 0x55b5073382f0 2024-09-10T23:37:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5072ec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5072ec1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60760/0 (socket says 172.21.15.94:60760) 2024-09-10T23:37:32.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f4a00640 1 -- 172.21.15.94:0/521519426 learned_addr learned my addr 172.21.15.94:0/521519426 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:32.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13efe00640 1 --2- 172.21.15.94:0/521519426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073379d0 0x55b507337db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13e7400640 1 --2- 172.21.15.94:0/521519426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073382f0 0x55b5072eb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f4a00640 1 -- 172.21.15.94:0/521519426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073382f0 msgr2=0x55b5072eb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:37:32.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f4a00640 1 --2- 172.21.15.94:0/521519426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073382f0 0x55b5072eb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f4a00640 1 -- 172.21.15.94:0/521519426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073379d0 msgr2=0x55b507337db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:37:32.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f4a00640 1 --2- 172.21.15.94:0/521519426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073379d0 0x55b507337db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f4a00640 1 -- 172.21.15.94:0/521519426 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b507100590 con 0x55b5072ebe10 2024-09-10T23:37:32.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f4a00640 1 --2- 172.21.15.94:0/521519426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5072ec1f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f13e80b0040 tx=0x7f13e8019910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d445d8031a1288 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:32.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13e4c00640 1 -- 172.21.15.94:0/521519426 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e80b2040 con 0x55b5072ebe10 2024-09-10T23:37:32.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13e4c00640 1 -- 172.21.15.94:0/521519426 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13e80a5b20 con 0x55b5072ebe10 2024-09-10T23:37:32.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13e4c00640 1 -- 172.21.15.94:0/521519426 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8004cb0 con 0x55b5072ebe10 2024-09-10T23:37:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 -- 172.21.15.94:0/521519426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 msgr2=0x55b5072ec1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 --2- 172.21.15.94:0/521519426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5072ec1f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f13e80b0040 tx=0x7f13e8019910 comp rx=0 tx=0).stop 2024-09-10T23:37:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 -- 172.21.15.94:0/521519426 shutdown_connections 2024-09-10T23:37:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 --2- 172.21.15.94:0/521519426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5072ec1f0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 --2- 172.21.15.94:0/521519426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073382f0 0x55b5072eb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 --2- 172.21.15.94:0/521519426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073379d0 0x55b507337db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 -- 172.21.15.94:0/521519426 >> 172.21.15.94:0/521519426 conn(0x55b5071d5130 msgr2=0x55b50731ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 -- 172.21.15.94:0/521519426 shutdown_connections 2024-09-10T23:37:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 -- 172.21.15.94:0/521519426 wait complete. 2024-09-10T23:37:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 -- start start 2024-09-10T23:37:32.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 Processor -- start 2024-09-10T23:37:32.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5073186a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073379d0 0x55b507318e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073382f0 0x55b5073193b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b5070d35d0 con 0x55b5072ebe10 2024-09-10T23:37:32.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b50713cd60 con 0x55b5073382f0 2024-09-10T23:37:32.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x55b5070df500 con 0x55b5073379d0 2024-09-10T23:37:32.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.445+0000 7f13f9dab000 10 client.0 _ll_register_callbacks cb 0x55b5070e3810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:37:32.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5073186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5073186a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60768/0 (socket says 172.21.15.94:60768) 2024-09-10T23:37:32.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13f4a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073382f0 0x55b5073193b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13e7400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073379d0 0x55b507318e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13efe00640 1 -- 172.21.15.94:0/719888661 learned_addr learned my addr 172.21.15.94:0/719888661 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:32.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13efe00640 1 -- 172.21.15.94:0/719888661 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073379d0 msgr2=0x55b507318e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:37:32.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13efe00640 1 --2- 172.21.15.94:0/719888661 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073379d0 0x55b507318e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13efe00640 1 -- 172.21.15.94:0/719888661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073382f0 msgr2=0x55b5073193b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:37:32.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13efe00640 1 --2- 172.21.15.94:0/719888661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073382f0 0x55b5073193b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13efe00640 1 -- 172.21.15.94:0/719888661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5070d7170 con 0x55b5072ebe10 2024-09-10T23:37:32.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13efe00640 1 --2- 172.21.15.94:0/719888661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5073186a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f13e000df10 tx=0x7f13e0004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755ee5bc7cff9ea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:32.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13dd600640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:37:32.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13dd600640 1 -- 172.21.15.94:0/719888661 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0024020 con 0x55b5072ebe10 2024-09-10T23:37:32.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13f9dab000 1 -- 172.21.15.94:0/719888661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b5070d10f0 con 0x55b5072ebe10 2024-09-10T23:37:32.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13dd600640 1 -- 172.21.15.94:0/719888661 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f13e000f040 con 0x55b5072ebe10 2024-09-10T23:37:32.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13dd600640 1 -- 172.21.15.94:0/719888661 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f13e002e050 con 0x55b5072ebe10 2024-09-10T23:37:32.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13f9dab000 10 client.7143 fetch_fsmap learned FSMap version 68 2024-09-10T23:37:32.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13f9dab000 1 -- 172.21.15.94:0/719888661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55b5071bdef0 con 0x55b5072ebe10 2024-09-10T23:37:32.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13dd600640 1 -- 172.21.15.94:0/719888661 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b5070d10f0 con 0x55b5072ebe10 2024-09-10T23:37:32.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13dd600640 1 -- 172.21.15.94:0/719888661 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 68) ==== 3502+0+0 (secure 0 0 0) 0x7f13e0031020 con 0x55b5072ebe10 2024-09-10T23:37:32.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13f9dab000 10 client.7143 fetch_fsmap finished waiting for FSMap version 68 2024-09-10T23:37:32.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13f9dab000 10 client.7143 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T23:37:32.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13f9dab000 20 client.7143 populate_metadata read hostname 'smithi094' 2024-09-10T23:37:32.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13f9dab000 1 --2- 172.21.15.94:0/719888661 >> [v2:172.21.15.181:6834/2513770511,v1:172.21.15.181:6835/2513770511] conn(0x55b50731a0f0 0x55b507349c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13f9dab000 4 client.7143 mds_command: new command op to 195950 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:37:32.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13f9dab000 1 -- 172.21.15.94:0/719888661 --> [v2:172.21.15.181:6834/2513770511,v1:172.21.15.181:6835/2513770511] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b50714ba70 con 0x55b50731a0f0 2024-09-10T23:37:32.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13e7400640 1 --2- 172.21.15.94:0/719888661 >> [v2:172.21.15.181:6834/2513770511,v1:172.21.15.181:6835/2513770511] conn(0x55b50731a0f0 0x55b507349c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13e7400640 1 --2- 172.21.15.94:0/719888661 >> [v2:172.21.15.181:6834/2513770511,v1:172.21.15.181:6835/2513770511] conn(0x55b50731a0f0 0x55b507349c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=488fbd758781357e server_cookie=742a850f56632462 in_seq=0 out_seq=0 2024-09-10T23:37:32.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13dd600640 10 client.7143 ms_handle_connect on v2:172.21.15.181:6834/2513770511 2024-09-10T23:37:32.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13dd600640 1 -- 172.21.15.94:0/719888661 <== mds.0 v2:172.21.15.181:6834/2513770511 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b50714ba70 con 0x55b50731a0f0 2024-09-10T23:37:32.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13dd600640 10 client.7143 handle_command_reply: tid=0 2024-09-10T23:37:32.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13f9dab000 1 client.7143 shutdown 2024-09-10T23:37:32.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13f9dab000 1 -- 172.21.15.94:0/719888661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 msgr2=0x55b5073186a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:32.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.449+0000 7f13f9dab000 1 --2- 172.21.15.94:0/719888661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5073186a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f13e000df10 tx=0x7f13e0004700 comp rx=0 tx=0).stop 2024-09-10T23:37:32.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.453+0000 7f13f9dab000 1 -- 172.21.15.94:0/719888661 shutdown_connections 2024-09-10T23:37:32.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.453+0000 7f13f9dab000 1 --2- 172.21.15.94:0/719888661 >> [v2:172.21.15.181:6834/2513770511,v1:172.21.15.181:6835/2513770511] conn(0x55b50731a0f0 0x55b507349c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.453+0000 7f13f9dab000 1 --2- 172.21.15.94:0/719888661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073382f0 0x55b5073193b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.453+0000 7f13f9dab000 1 --2- 172.21.15.94:0/719888661 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073379d0 0x55b507318e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.453+0000 7f13dd600640 0 client.7143 ms_handle_reset on v2:172.21.15.181:6834/2513770511 2024-09-10T23:37:32.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.453+0000 7f13f9dab000 1 --2- 172.21.15.94:0/719888661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5073186a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.453+0000 7f13f9dab000 1 -- 172.21.15.94:0/719888661 >> 172.21.15.94:0/719888661 conn(0x55b5071d5130 msgr2=0x55b5072eafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:32.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.453+0000 7f13f9dab000 1 -- 172.21.15.94:0/719888661 shutdown_connections 2024-09-10T23:37:32.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.453+0000 7f13f9dab000 1 -- 172.21.15.94:0/719888661 wait complete. 2024-09-10T23:37:32.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.453+0000 7f13f9dab000 20 client.7143 trim_cache size 0 max 16384 2024-09-10T23:37:32.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.469+0000 7f13f9dab000 1 Processor -- start 2024-09-10T23:37:32.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.469+0000 7f13f9dab000 1 -- start start 2024-09-10T23:37:32.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.469+0000 7f13f9dab000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5073a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.469+0000 7f13f9dab000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073379d0 0x55b5073a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.469+0000 7f13f9dab000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073382f0 0x55b5073aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.469+0000 7f13f9dab000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f13f0204590 con 0x55b5072ebe10 2024-09-10T23:37:32.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.469+0000 7f13f9dab000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f13f0203700 con 0x55b5073382f0 2024-09-10T23:37:32.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.469+0000 7f13f9dab000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f13f01fc0b0 con 0x55b5073379d0 2024-09-10T23:37:32.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.469+0000 7f13efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5073a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.469+0000 7f13efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5073a99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60778/0 (socket says 172.21.15.94:60778) 2024-09-10T23:37:32.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.469+0000 7f13efe00640 1 -- 172.21.15.94:0/227255568 learned_addr learned my addr 172.21.15.94:0/227255568 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:32.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.469+0000 7f13e7400640 1 --2- 172.21.15.94:0/227255568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073379d0 0x55b5073a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.469+0000 7f13f4a00640 1 --2- 172.21.15.94:0/227255568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073382f0 0x55b5073aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13efe00640 1 -- 172.21.15.94:0/227255568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073379d0 msgr2=0x55b5073a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:37:32.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13efe00640 1 --2- 172.21.15.94:0/227255568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073379d0 0x55b5073a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13efe00640 1 -- 172.21.15.94:0/227255568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073382f0 msgr2=0x55b5073aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:37:32.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13efe00640 1 --2- 172.21.15.94:0/227255568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073382f0 0x55b5073aa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13efe00640 1 -- 172.21.15.94:0/227255568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b507194d90 con 0x55b5072ebe10 2024-09-10T23:37:32.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f4a00640 1 --2- 172.21.15.94:0/227255568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073382f0 0x55b5073aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:32.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13efe00640 1 --2- 172.21.15.94:0/227255568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5073a99b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f13e00046d0 tx=0x7f13e000d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f52e84c7aff31e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:32.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13dfe00640 1 -- 172.21.15.94:0/227255568 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0037070 con 0x55b5072ebe10 2024-09-10T23:37:32.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13dfe00640 1 -- 172.21.15.94:0/227255568 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13e0020600 con 0x55b5072ebe10 2024-09-10T23:37:32.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13dfe00640 1 -- 172.21.15.94:0/227255568 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0024030 con 0x55b5072ebe10 2024-09-10T23:37:32.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 -- 172.21.15.94:0/227255568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 msgr2=0x55b5073a99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:32.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 --2- 172.21.15.94:0/227255568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5073a99b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f13e00046d0 tx=0x7f13e000d830 comp rx=0 tx=0).stop 2024-09-10T23:37:32.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 -- 172.21.15.94:0/227255568 shutdown_connections 2024-09-10T23:37:32.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 --2- 172.21.15.94:0/227255568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073382f0 0x55b5073aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 --2- 172.21.15.94:0/227255568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073379d0 0x55b5073a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 --2- 172.21.15.94:0/227255568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5072ebe10 0x55b5073a99b0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 -- 172.21.15.94:0/227255568 >> 172.21.15.94:0/227255568 conn(0x55b5071d5130 msgr2=0x55b507344fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:32.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 -- 172.21.15.94:0/227255568 shutdown_connections 2024-09-10T23:37:32.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 -- 172.21.15.94:0/227255568 wait complete. 2024-09-10T23:37:32.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 -- start start 2024-09-10T23:37:32.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 Processor -- start 2024-09-10T23:37:32.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5073379d0 0x55b5073adc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073382f0 0x55b5073ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073ae680 0x55b50734b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b50734ece0 con 0x55b5073379d0 2024-09-10T23:37:32.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b5072e5d80 con 0x55b5073382f0 2024-09-10T23:37:32.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x55b50713c070 con 0x55b5073ae680 2024-09-10T23:37:32.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f9dab000 10 client.0 _ll_register_callbacks cb 0x7f13e000b4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:37:32.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5073379d0 0x55b5073adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5073379d0 0x55b5073adc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60794/0 (socket says 172.21.15.94:60794) 2024-09-10T23:37:32.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13efe00640 1 -- 172.21.15.94:0/2460207411 learned_addr learned my addr 172.21.15.94:0/2460207411 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:32.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13e7400640 1 --2- 172.21.15.94:0/2460207411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073382f0 0x55b5073ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13f4a00640 1 --2- 172.21.15.94:0/2460207411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073ae680 0x55b50734b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13efe00640 1 -- 172.21.15.94:0/2460207411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073ae680 msgr2=0x55b50734b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:37:32.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13efe00640 1 --2- 172.21.15.94:0/2460207411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073ae680 0x55b50734b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13efe00640 1 -- 172.21.15.94:0/2460207411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073382f0 msgr2=0x55b5073ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:37:32.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13efe00640 1 --2- 172.21.15.94:0/2460207411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073382f0 0x55b5073ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.473+0000 7f13efe00640 1 -- 172.21.15.94:0/2460207411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5070ffa90 con 0x55b5073379d0 2024-09-10T23:37:32.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13efe00640 1 --2- 172.21.15.94:0/2460207411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5073379d0 0x55b5073adc00 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f13e0035040 tx=0x7f13e0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d5e6212c9868cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:32.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13dd600640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:37:32.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13dd600640 1 -- 172.21.15.94:0/2460207411 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0037070 con 0x55b5073379d0 2024-09-10T23:37:32.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13dd600640 1 -- 172.21.15.94:0/2460207411 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f13e003cab0 con 0x55b5073379d0 2024-09-10T23:37:32.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 1 -- 172.21.15.94:0/2460207411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e0020600 con 0x55b5073379d0 2024-09-10T23:37:32.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13dd600640 1 -- 172.21.15.94:0/2460207411 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f13e0024040 con 0x55b5073379d0 2024-09-10T23:37:32.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 10 client.7149 fetch_fsmap learned FSMap version 68 2024-09-10T23:37:32.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 1 -- 172.21.15.94:0/2460207411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55b5070de050 con 0x55b5073379d0 2024-09-10T23:37:32.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13dd600640 1 -- 172.21.15.94:0/2460207411 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13e0020600 con 0x55b5073379d0 2024-09-10T23:37:32.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13dd600640 1 -- 172.21.15.94:0/2460207411 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 68) ==== 3502+0+0 (secure 0 0 0) 0x7f13e0031050 con 0x55b5073379d0 2024-09-10T23:37:32.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 10 client.7149 fetch_fsmap finished waiting for FSMap version 68 2024-09-10T23:37:32.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 10 client.7149 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T23:37:32.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 20 client.7149 populate_metadata read hostname 'smithi094' 2024-09-10T23:37:32.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 1 --2- 172.21.15.94:0/2460207411 >> [v2:172.21.15.181:6834/2513770511,v1:172.21.15.181:6835/2513770511] conn(0x55b50734c150 0x55b50734c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 4 client.7149 mds_command: new command op to 195950 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T23:37:32.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 1 -- 172.21.15.94:0/2460207411 --> [v2:172.21.15.181:6834/2513770511,v1:172.21.15.181:6835/2513770511] -- command(tid 0: {"prefix": "status"}) -- 0x55b507147520 con 0x55b50734c150 2024-09-10T23:37:32.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13e7400640 1 --2- 172.21.15.94:0/2460207411 >> [v2:172.21.15.181:6834/2513770511,v1:172.21.15.181:6835/2513770511] conn(0x55b50734c150 0x55b50734c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13dd600640 10 client.7149 ms_handle_connect on v2:172.21.15.181:6834/2513770511 2024-09-10T23:37:32.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13e7400640 1 --2- 172.21.15.94:0/2460207411 >> [v2:172.21.15.181:6834/2513770511,v1:172.21.15.181:6835/2513770511] conn(0x55b50734c150 0x55b50734c530 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=902075bac5a31870 server_cookie=c11779b254129a01 in_seq=0 out_seq=0 2024-09-10T23:37:32.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13dd600640 1 -- 172.21.15.94:0/2460207411 <== mds.0 v2:172.21.15.181:6834/2513770511 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55b507147520 con 0x55b50734c150 2024-09-10T23:37:32.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13dd600640 10 client.7149 handle_command_reply: tid=0 2024-09-10T23:37:32.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 1 client.7149 shutdown 2024-09-10T23:37:32.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 1 -- 172.21.15.94:0/2460207411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5073379d0 msgr2=0x55b5073adc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:32.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 1 --2- 172.21.15.94:0/2460207411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5073379d0 0x55b5073adc00 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f13e0035040 tx=0x7f13e0010000 comp rx=0 tx=0).stop 2024-09-10T23:37:32.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 1 -- 172.21.15.94:0/2460207411 shutdown_connections 2024-09-10T23:37:32.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 1 --2- 172.21.15.94:0/2460207411 >> [v2:172.21.15.181:6834/2513770511,v1:172.21.15.181:6835/2513770511] conn(0x55b50734c150 0x55b50734c530 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 1 --2- 172.21.15.94:0/2460207411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b5073382f0 0x55b5073ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 1 --2- 172.21.15.94:0/2460207411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b5073ae680 0x55b50734b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 1 --2- 172.21.15.94:0/2460207411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5073379d0 0x55b5073adc00 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13dd600640 0 client.7149 ms_handle_reset on v2:172.21.15.181:6834/2513770511 2024-09-10T23:37:32.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 1 -- 172.21.15.94:0/2460207411 >> 172.21.15.94:0/2460207411 conn(0x55b5071d5130 msgr2=0x55b5072f3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:32.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 1 -- 172.21.15.94:0/2460207411 shutdown_connections 2024-09-10T23:37:32.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 1 -- 172.21.15.94:0/2460207411 wait complete. 2024-09-10T23:37:32.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f9dab000 20 client.7149 trim_cache size 0 max 16384 2024-09-10T23:37:32.485 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:37:32.486 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "1798cdb9-9937-4918-a569-86b41a92817e", 2024-09-10T23:37:32.486 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-09-10T23:37:32.486 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 195950, 2024-09-10T23:37:32.486 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-10T23:37:32.486 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-10T23:37:32.486 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-10T23:37:32.486 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 5.9282682089999996, 2024-09-10T23:37:32.486 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 68, 2024-09-10T23:37:32.486 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 99, 2024-09-10T23:37:32.486 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 99, 2024-09-10T23:37:32.486 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 10.420197461000001 2024-09-10T23:37:32.486 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:37:32.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f6a00640 1 -- 172.21.15.94:0/139127684 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f13cc072730 msgr2=0x7f13cc074bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:32.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f6a00640 1 --2- 172.21.15.94:0/139127684 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f13cc072730 0x7f13cc074bf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f13e000b810 tx=0x7f13e0008040 comp rx=0 tx=0).stop 2024-09-10T23:37:32.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f6a00640 1 -- 172.21.15.94:0/139127684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13f00d0710 msgr2=0x7f13f00cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:32.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f6a00640 1 --2- 172.21.15.94:0/139127684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13f00d0710 0x7f13f00cbd60 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f13d800d980 tx=0x7f13d800de40 comp rx=0 tx=0).stop 2024-09-10T23:37:32.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f6a00640 1 -- 172.21.15.94:0/139127684 shutdown_connections 2024-09-10T23:37:32.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f6a00640 1 --2- 172.21.15.94:0/139127684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13f01455d0 0x7f13f0073cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f6a00640 1 --2- 172.21.15.94:0/139127684 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f13cc072730 0x7f13cc074bf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f6a00640 1 --2- 172.21.15.94:0/139127684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13f00d0710 0x7f13f00cbd60 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f6a00640 1 --2- 172.21.15.94:0/139127684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13f00c5a70 0x7f13f00afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f6a00640 1 -- 172.21.15.94:0/139127684 >> 172.21.15.94:0/139127684 conn(0x7f13f00cfaf0 msgr2=0x7f13f00c4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:32.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f6a00640 1 -- 172.21.15.94:0/139127684 shutdown_connections 2024-09-10T23:37:32.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.477+0000 7f13f6a00640 1 -- 172.21.15.94:0/139127684 wait complete. 2024-09-10T23:37:32.500 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:37:32.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.641+0000 7f4b41c00640 1 -- 172.21.15.94:0/3499094605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3c073ff0 msgr2=0x7f4b3c0cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:32.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.641+0000 7f4b41c00640 1 --2- 172.21.15.94:0/3499094605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3c073ff0 0x7f4b3c0cfda0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c06eb40 tx=0x7f4b2c01c890 comp rx=0 tx=0).stop 2024-09-10T23:37:32.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.641+0000 7f4b41c00640 1 -- 172.21.15.94:0/3499094605 shutdown_connections 2024-09-10T23:37:32.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.641+0000 7f4b41c00640 1 --2- 172.21.15.94:0/3499094605 >> v2:172.21.15.181:3300/0 conn(0x7f4b3c15ecb0 0x7f4b3c0b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.641+0000 7f4b41c00640 1 --2- 172.21.15.94:0/3499094605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b3c0d02e0 0x7f4b3c15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.641+0000 7f4b41c00640 1 --2- 172.21.15.94:0/3499094605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3c073ff0 0x7f4b3c0cfda0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.641+0000 7f4b41c00640 1 -- 172.21.15.94:0/3499094605 >> 172.21.15.94:0/3499094605 conn(0x7f4b3c06cdf0 msgr2=0x7f4b3c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:32.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.641+0000 7f4b41c00640 1 -- 172.21.15.94:0/3499094605 shutdown_connections 2024-09-10T23:37:32.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.641+0000 7f4b41c00640 1 -- 172.21.15.94:0/3499094605 wait complete. 2024-09-10T23:37:32.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b41c00640 1 Processor -- start 2024-09-10T23:37:32.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b41c00640 1 -- start start 2024-09-10T23:37:32.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b41c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3c073ff0 0x7f4b3c156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b41c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b3c0d02e0 0x7f4b3c157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b41c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b3c15ecb0 0x7f4b3c1556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b41c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c06f840 con 0x7f4b3c073ff0 2024-09-10T23:37:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b41c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c06f6c0 con 0x7f4b3c15ecb0 2024-09-10T23:37:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b41c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f4b3c06f9c0 con 0x7f4b3c0d02e0 2024-09-10T23:37:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b3aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3c073ff0 0x7f4b3c156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b3aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3c073ff0 0x7f4b3c156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60818/0 (socket says 172.21.15.94:60818) 2024-09-10T23:37:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b3aa00640 1 -- 172.21.15.94:0/145701806 learned_addr learned my addr 172.21.15.94:0/145701806 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b32000640 1 --2- 172.21.15.94:0/145701806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b3c0d02e0 0x7f4b3c157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b3b400640 1 --2- 172.21.15.94:0/145701806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b3c15ecb0 0x7f4b3c1556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b3aa00640 1 -- 172.21.15.94:0/145701806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b3c0d02e0 msgr2=0x7f4b3c157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b3aa00640 1 --2- 172.21.15.94:0/145701806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b3c0d02e0 0x7f4b3c157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b3aa00640 1 -- 172.21.15.94:0/145701806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b3c15ecb0 msgr2=0x7f4b3c1556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b3aa00640 1 --2- 172.21.15.94:0/145701806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b3c15ecb0 0x7f4b3c1556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b3aa00640 1 -- 172.21.15.94:0/145701806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c1596d0 con 0x7f4b3c073ff0 2024-09-10T23:37:32.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b3aa00640 1 --2- 172.21.15.94:0/145701806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3c073ff0 0x7f4b3c156fd0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c01af40 tx=0x7f4b2c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b33400640 1 -- 172.21.15.94:0/145701806 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c0b2040 con 0x7f4b3c073ff0 2024-09-10T23:37:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b33400640 1 -- 172.21.15.94:0/145701806 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b2c005030 con 0x7f4b3c073ff0 2024-09-10T23:37:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b33400640 1 -- 172.21.15.94:0/145701806 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c005480 con 0x7f4b3c073ff0 2024-09-10T23:37:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b41c00640 1 -- 172.21.15.94:0/145701806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3c1599b0 con 0x7f4b3c073ff0 2024-09-10T23:37:32.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.645+0000 7f4b41c00640 1 -- 172.21.15.94:0/145701806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3c159f70 con 0x7f4b3c073ff0 2024-09-10T23:37:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.649+0000 7f4b41c00640 1 -- 172.21.15.94:0/145701806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b3c15e770 con 0x7f4b3c073ff0 2024-09-10T23:37:32.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.649+0000 7f4b33400640 1 -- 172.21.15.94:0/145701806 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f4b2c0046d0 con 0x7f4b3c073ff0 2024-09-10T23:37:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.649+0000 7f4b33400640 1 --2- 172.21.15.94:0/145701806 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4b18072700 0x7f4b18074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.649+0000 7f4b33400640 1 -- 172.21.15.94:0/145701806 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f4b2c12ce50 con 0x7f4b3c073ff0 2024-09-10T23:37:32.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.649+0000 7f4b32000640 1 --2- 172.21.15.94:0/145701806 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4b18072700 0x7f4b18074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:32.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.649+0000 7f4b32000640 1 --2- 172.21.15.94:0/145701806 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4b18072700 0x7f4b18074bc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4b24005fd0 tx=0x7f4b24005e60 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:32.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.653+0000 7f4b33400640 1 -- 172.21.15.94:0/145701806 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4b2c0164b0 con 0x7f4b3c073ff0 2024-09-10T23:37:32.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.853+0000 7f4b41c00640 1 -- 172.21.15.94:0/145701806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4b3c1511d0 con 0x7f4b3c073ff0 2024-09-10T23:37:32.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.853+0000 7f4b33400640 1 -- 172.21.15.94:0/145701806 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) ==== 53+0+73 (secure 0 0 0) 0x7f4b2c0f8270 con 0x7f4b3c073ff0 2024-09-10T23:37:32.856 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:37:32.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.857+0000 7f4b41c00640 1 -- 172.21.15.94:0/145701806 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4b18072700 msgr2=0x7f4b18074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.857+0000 7f4b41c00640 1 --2- 172.21.15.94:0/145701806 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4b18072700 0x7f4b18074bc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4b24005fd0 tx=0x7f4b24005e60 comp rx=0 tx=0).stop 2024-09-10T23:37:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.857+0000 7f4b41c00640 1 -- 172.21.15.94:0/145701806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3c073ff0 msgr2=0x7f4b3c156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.857+0000 7f4b41c00640 1 --2- 172.21.15.94:0/145701806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3c073ff0 0x7f4b3c156fd0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c01af40 tx=0x7f4b2c004040 comp rx=0 tx=0).stop 2024-09-10T23:37:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.857+0000 7f4b41c00640 1 -- 172.21.15.94:0/145701806 shutdown_connections 2024-09-10T23:37:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.857+0000 7f4b41c00640 1 --2- 172.21.15.94:0/145701806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b3c15ecb0 0x7f4b3c1556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.857+0000 7f4b41c00640 1 --2- 172.21.15.94:0/145701806 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4b18072700 0x7f4b18074bc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.857+0000 7f4b41c00640 1 --2- 172.21.15.94:0/145701806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b3c0d02e0 0x7f4b3c157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.857+0000 7f4b41c00640 1 --2- 172.21.15.94:0/145701806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b3c073ff0 0x7f4b3c156fd0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.857+0000 7f4b41c00640 1 -- 172.21.15.94:0/145701806 >> 172.21.15.94:0/145701806 conn(0x7f4b3c06cdf0 msgr2=0x7f4b3c0b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.857+0000 7f4b41c00640 1 -- 172.21.15.94:0/145701806 shutdown_connections 2024-09-10T23:37:32.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:32.857+0000 7f4b41c00640 1 -- 172.21.15.94:0/145701806 wait complete. 2024-09-10T23:37:32.877 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:37:32.877 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:37:32.877 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T23:37:32.877 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T23:37:32.877 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T23:37:32.877 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:37:32.877 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:37:32.878 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:37:32.878 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:37:32.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:32.878 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T23:37:32.886 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:37:32.887 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:37:32.887 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:37:32.887 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:32.887 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T23:37:32.887 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:32.887 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:37:32.887 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:32.887 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T23:37:32.887 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:37:32.887 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:32.888 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T23:37:32.888 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:37:32.888 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:32.888 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:37:32.888 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:32.888 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:37:32.888 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42158sec preferred_lft 42158sec 2024-09-10T23:37:32.888 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-09-10T23:37:32.888 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:32.888 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:37:32.888 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:ed:7d:46:db brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:32.888 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:37:32.888 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:32.889 INFO:teuthology.orchestra.run.smithi094.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:37:32.889 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:32.889 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:37:32.889 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:32.889 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:37:32.889 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:32.889 INFO:teuthology.orchestra.run.smithi094.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:37:32.889 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:37:32.889 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:37:32.889 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:32.889 INFO:teuthology.orchestra.run.smithi094.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:37:32.889 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:37:32.889 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:37:32.890 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:32.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:32.891 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:37:32.954 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T23:37:32.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:32.957 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T23:37:33.008 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T23:37:33.009 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:37:33.009 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:37:33.009 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:37:33.009 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:37:33.009 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:37:33.009 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:37:33.009 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:37:33.009 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:37:33.009 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:37:33.009 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:37:33.010 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:37:33.010 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:37:33.010 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:37:33.010 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:37:33.010 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:37:33.010 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:37:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:37:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:37:33.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:33.145 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T23:37:33.153 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T23:37:33.154 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T23:37:33.154 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:37:33.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:33.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-09-10T23:37:33.220 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T23:37:33.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:37:33.220 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T23:37:33.269 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:37:33.269 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:37:33.269 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:37:33.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:37:33.381 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:37:33.434 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:37:33.434 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:37:33.435 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:37:33.435 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:37:33.435 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:37:33.435 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:37:33.435 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:37:33.435 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:37:33.435 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:37:33.435 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:37:33.435 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:37:33.435 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:37:33.435 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:37:33.435 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:37:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:37:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:37:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:37:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:37:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:37:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:37:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:37:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:37:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:37:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:37:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:37:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:37:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:37:33.436 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:37:33.437 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:37:33.437 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:37:33.437 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:37:33.437 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:37:33.437 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:37:33.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:33.438 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:37:33.489 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:37:33.489 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T23:37:33.531 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:37:33.551 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:37:33.607 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:37:33.607 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:37:33.607 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:37:33.607 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:37:33.607 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:37:33.607 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:37:33.607 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:37:33.607 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:37:33.608 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:37:33.608 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:37:33.608 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:37:33.608 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:37:33.608 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:37:33.608 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:37:33.608 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:37:33.608 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:37:33.608 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:37:33.608 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:37:33.608 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:37:33.608 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:37:33.608 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:37:33.608 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:37:33.608 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:37:33.609 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:37:33.609 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:37:33.609 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:37:33.609 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:37:33.609 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:37:33.609 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:37:33.609 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:37:33.609 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:37:33.609 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:37:33.609 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:37:33.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:33.610 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:37:33.629 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:2024-09-10T23:37:33.625+0000 7f2cf2d85580 -1 init, newargv = 0x56416e8e5c20 newargc=15 2024-09-10T23:37:33.629 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[44916]: starting ceph client 2024-09-10T23:37:33.636 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[44916]: starting fuse 2024-09-10T23:37:33.637 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T23:37:33.637 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:37:33.638 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:37:33.638 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-09-10T23:37:33.638 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-09-10T23:37:33.638 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-09-10T23:37:33.638 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-09-10T23:37:33.638 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:37:33.638 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-09-10T23:37:33.638 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T23:37:33.638 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-09-10T23:37:33.638 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T23:37:33.638 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:37:33.638 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-09-10T23:37:33.638 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-09-10T23:37:33.638 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-09-10T23:37:33.639 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-09-10T23:37:33.639 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:37:33.639 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-09-10T23:37:33.639 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:37:33.639 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:37:33.639 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:37:33.639 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-09-10T23:37:33.639 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-09-10T23:37:33.639 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-09-10T23:37:33.639 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:37:33.639 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:37:33.639 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.0")) 2024-09-10T23:37:33.639 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-09-10T23:37:33.764 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.0.44916.asok 2024-09-10T23:37:33.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:37:33.776 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44916.asok 2024-09-10T23:37:33.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:33.776 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44916.asok status 2024-09-10T23:37:33.949 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:37:33.949 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-09-10T23:37:33.949 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:37:33.950 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:37:33.950 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "0", 2024-09-10T23:37:33.950 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-09-10T23:37:33.950 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T23:37:33.950 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "44916", 2024-09-10T23:37:33.950 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-09-10T23:37:33.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:33.950 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-09-10T23:37:33.950 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-09-10T23:37:33.950 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 7164, 2024-09-10T23:37:33.950 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-09-10T23:37:33.950 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-09-10T23:37:33.950 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-09-10T23:37:33.950 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 7164 2024-09-10T23:37:33.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:33.950 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:37:33.951 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:37:33.951 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:37:33.951 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1507996979 2024-09-10T23:37:33.951 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:33.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:33.951 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:37:33.951 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:37:33.951 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:37:33.951 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1507996979 2024-09-10T23:37:33.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:33.951 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.7164 192.168.0.1:0/1507996979", 2024-09-10T23:37:33.951 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/1507996979", 2024-09-10T23:37:33.951 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-09-10T23:37:33.951 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 68, 2024-09-10T23:37:33.951 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 99, 2024-09-10T23:37:33.952 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:37:33.952 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-09-10T23:37:33.952 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-09-10T23:37:33.952 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:37:33.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:33.964 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-09-10T23:37:33.964 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T23:37:33.973 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-09-10T23:37:33.973 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T23:37:33.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:33.973 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:37:34.037 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:37:34.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.225+0000 7f1e8d200640 1 -- 172.21.15.94:0/3557252686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e88070150 msgr2=0x7f1e880b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:34.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.225+0000 7f1e8d200640 1 --2- 172.21.15.94:0/3557252686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e88070150 0x7f1e880b35c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000aa10 tx=0x7f1e7001dba0 comp rx=0 tx=0).stop 2024-09-10T23:37:34.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.225+0000 7f1e8d200640 1 -- 172.21.15.94:0/3557252686 shutdown_connections 2024-09-10T23:37:34.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.225+0000 7f1e8d200640 1 --2- 172.21.15.94:0/3557252686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e88070150 0x7f1e880b35c0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:34.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.225+0000 7f1e8d200640 1 --2- 172.21.15.94:0/3557252686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e88157c60 0x7f1e88072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:34.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.225+0000 7f1e8d200640 1 -- 172.21.15.94:0/3557252686 >> 172.21.15.94:0/3557252686 conn(0x7f1e8806cdf0 msgr2=0x7f1e8806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:34.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.225+0000 7f1e8d200640 1 -- 172.21.15.94:0/3557252686 shutdown_connections 2024-09-10T23:37:34.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.225+0000 7f1e8d200640 1 -- 172.21.15.94:0/3557252686 wait complete. 2024-09-10T23:37:34.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.225+0000 7f1e8d200640 1 Processor -- start 2024-09-10T23:37:34.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.225+0000 7f1e8d200640 1 -- start start 2024-09-10T23:37:34.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e8d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e8806eef0 0x7f1e881fbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:34.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e8d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e88070150 0x7f1e88142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:34.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e8d200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e88157c60 0x7f1e881f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:34.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e8d200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e88167210 con 0x7f1e88070150 2024-09-10T23:37:34.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e8d200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1e88167090 con 0x7f1e8806eef0 2024-09-10T23:37:34.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e8d200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1e88167390 con 0x7f1e88157c60 2024-09-10T23:37:34.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e7d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e88070150 0x7f1e88142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:34.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e7d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e88070150 0x7f1e88142450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54864/0 (socket says 172.21.15.94:54864) 2024-09-10T23:37:34.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e7d400640 1 -- 172.21.15.94:0/4088877162 learned_addr learned my addr 172.21.15.94:0/4088877162 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:34.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e86800640 1 --2- 172.21.15.94:0/4088877162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e88157c60 0x7f1e881f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:34.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e85e00640 1 --2- 172.21.15.94:0/4088877162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e8806eef0 0x7f1e881fbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:34.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e7d400640 1 -- 172.21.15.94:0/4088877162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e88157c60 msgr2=0x7f1e881f7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:34.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e7d400640 1 --2- 172.21.15.94:0/4088877162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e88157c60 0x7f1e881f7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:34.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e7d400640 1 -- 172.21.15.94:0/4088877162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e8806eef0 msgr2=0x7f1e881fbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:34.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e7d400640 1 --2- 172.21.15.94:0/4088877162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e8806eef0 0x7f1e881fbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:34.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e7d400640 1 -- 172.21.15.94:0/4088877162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e881f8a10 con 0x7f1e88070150 2024-09-10T23:37:34.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e86800640 1 --2- 172.21.15.94:0/4088877162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e88157c60 0x7f1e881f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:34.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e7d400640 1 --2- 172.21.15.94:0/4088877162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e88070150 0x7f1e88142450 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000a960 tx=0x7f1e700a6970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e7ea00640 1 -- 172.21.15.94:0/4088877162 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e700b3070 con 0x7f1e88070150 2024-09-10T23:37:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e7ea00640 1 -- 172.21.15.94:0/4088877162 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e700055f0 con 0x7f1e88070150 2024-09-10T23:37:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e7ea00640 1 -- 172.21.15.94:0/4088877162 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70004930 con 0x7f1e88070150 2024-09-10T23:37:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e8d200640 1 -- 172.21.15.94:0/4088877162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e881f8cf0 con 0x7f1e88070150 2024-09-10T23:37:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e8d200640 1 -- 172.21.15.94:0/4088877162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e881fb9c0 con 0x7f1e88070150 2024-09-10T23:37:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.229+0000 7f1e8d200640 1 -- 172.21.15.94:0/4088877162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e880b35c0 con 0x7f1e88070150 2024-09-10T23:37:34.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.233+0000 7f1e7ea00640 1 -- 172.21.15.94:0/4088877162 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f1e7000b040 con 0x7f1e88070150 2024-09-10T23:37:34.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.233+0000 7f1e7ea00640 1 --2- 172.21.15.94:0/4088877162 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1e64072700 0x7f1e64074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:34.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.233+0000 7f1e7ea00640 1 -- 172.21.15.94:0/4088877162 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f1e7012dd00 con 0x7f1e88070150 2024-09-10T23:37:34.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.233+0000 7f1e85e00640 1 --2- 172.21.15.94:0/4088877162 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1e64072700 0x7f1e64074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:34.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.233+0000 7f1e85e00640 1 --2- 172.21.15.94:0/4088877162 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1e64072700 0x7f1e64074bc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009e60 tx=0x7f1e780095b0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:34.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.237+0000 7f1e7ea00640 1 -- 172.21.15.94:0/4088877162 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1e70017820 con 0x7f1e88070150 2024-09-10T23:37:34.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.433+0000 7f1e8d200640 1 -- 172.21.15.94:0/4088877162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1e8806e7e0 con 0x7f1e88070150 2024-09-10T23:37:34.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.437+0000 7f1e7ea00640 1 -- 172.21.15.94:0/4088877162 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) ==== 53+0+73 (secure 0 0 0) 0x7f1e700f9330 con 0x7f1e88070150 2024-09-10T23:37:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:37:34.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.437+0000 7f1e8d200640 1 -- 172.21.15.94:0/4088877162 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1e64072700 msgr2=0x7f1e64074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:34.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.437+0000 7f1e8d200640 1 --2- 172.21.15.94:0/4088877162 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1e64072700 0x7f1e64074bc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009e60 tx=0x7f1e780095b0 comp rx=0 tx=0).stop 2024-09-10T23:37:34.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.437+0000 7f1e8d200640 1 -- 172.21.15.94:0/4088877162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e88070150 msgr2=0x7f1e88142450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:34.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.437+0000 7f1e8d200640 1 --2- 172.21.15.94:0/4088877162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e88070150 0x7f1e88142450 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000a960 tx=0x7f1e700a6970 comp rx=0 tx=0).stop 2024-09-10T23:37:34.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.437+0000 7f1e8d200640 1 -- 172.21.15.94:0/4088877162 shutdown_connections 2024-09-10T23:37:34.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.437+0000 7f1e8d200640 1 --2- 172.21.15.94:0/4088877162 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1e64072700 0x7f1e64074bc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.437+0000 7f1e8d200640 1 --2- 172.21.15.94:0/4088877162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e88157c60 0x7f1e881f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.437+0000 7f1e8d200640 1 --2- 172.21.15.94:0/4088877162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e88070150 0x7f1e88142450 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.437+0000 7f1e8d200640 1 --2- 172.21.15.94:0/4088877162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e8806eef0 0x7f1e881fbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.437+0000 7f1e8d200640 1 -- 172.21.15.94:0/4088877162 >> 172.21.15.94:0/4088877162 conn(0x7f1e8806cdf0 msgr2=0x7f1e880ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.437+0000 7f1e8d200640 1 -- 172.21.15.94:0/4088877162 shutdown_connections 2024-09-10T23:37:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:34.437+0000 7f1e8d200640 1 -- 172.21.15.94:0/4088877162 wait complete. 2024-09-10T23:37:34.459 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:37:34.459 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:37:34.459 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-09-10T23:37:34.459 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T23:37:34.459 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T23:37:34.459 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:37:34.459 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:37:34.459 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:37:34.459 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:37:34.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:34.459 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-09-10T23:37:34.470 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:37:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:37:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:37:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-09-10T23:37:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:34.470 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:37:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-09-10T23:37:34.471 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:37:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-09-10T23:37:34.471 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:37:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:34.471 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:37:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:37:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42161sec preferred_lft 42161sec 2024-09-10T23:37:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-09-10T23:37:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:34.471 INFO:teuthology.orchestra.run.smithi181.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:37:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 02:42:3b:2e:09:2d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:37:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:34.472 INFO:teuthology.orchestra.run.smithi181.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:37:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:37:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:37:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:37:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:34.472 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:37:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:37:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:37:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:34.472 INFO:teuthology.orchestra.run.smithi181.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:37:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:37:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:37:34.473 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:37:34.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:34.473 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:37:34.536 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-09-10T23:37:34.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:34.539 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-09-10T23:37:34.598 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-09-10T23:37:34.598 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:37:34.598 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:37:34.598 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:37:34.598 INFO:teuthology.orchestra.run.smithi181.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:37:34.598 INFO:teuthology.orchestra.run.smithi181.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:37:34.598 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:37:34.598 INFO:teuthology.orchestra.run.smithi181.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:37:34.598 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:37:34.598 INFO:teuthology.orchestra.run.smithi181.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:37:34.599 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:37:34.599 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:37:34.599 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:37:34.599 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:37:34.599 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:37:34.599 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:37:34.599 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:37:34.694 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:37:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:37:34.737 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:37:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:37:34.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:34.744 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-09-10T23:37:34.753 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T23:37:34.753 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T23:37:34.754 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:37:34.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:34.754 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-09-10T23:37:34.816 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T23:37:34.816 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:37:34.816 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T23:37:34.864 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:37:34.865 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:37:34.865 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:37:34.916 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:37:34.981 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:37:35.038 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:37:35.038 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:37:35.038 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:37:35.038 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:37:35.038 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:37:35.038 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:37:35.038 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:37:35.038 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:37:35.038 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:37:35.038 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:37:35.038 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:37:35.038 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:37:35.039 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:37:35.039 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:37:35.039 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:37:35.039 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:37:35.039 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:37:35.039 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:37:35.039 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:37:35.039 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:37:35.039 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:37:35.039 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:37:35.039 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:37:35.039 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:37:35.039 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:37:35.039 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:37:35.039 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:37:35.040 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:37:35.040 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:37:35.040 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:37:35.040 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:37:35.040 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:37:35.040 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:37:35.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:35.041 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:37:35.096 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:37:35.097 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T23:37:35.139 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:37:35.159 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:37:35.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:37:35.185 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:37:35.210 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:37:35.210 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:37:35.210 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:37:35.211 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:37:35.211 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:37:35.211 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:37:35.211 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:37:35.211 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:37:35.211 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:37:35.211 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:37:35.211 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:37:35.211 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:37:35.211 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:37:35.211 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:37:35.211 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:37:35.211 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:37:35.211 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:37:35.212 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:37:35.212 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:37:35.212 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:37:35.212 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:37:35.212 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:37:35.212 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:37:35.212 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:37:35.212 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:37:35.212 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:37:35.212 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:37:35.212 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:37:35.212 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:37:35.212 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:37:35.212 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:37:35.213 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:37:35.213 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:37:35.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:35.214 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:37:35.233 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:2024-09-10T23:37:35.230+0000 7fc4f622a580 -1 init, newargv = 0x5599e8eefc60 newargc=15 2024-09-10T23:37:35.233 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[35501]: starting ceph client 2024-09-10T23:37:35.240 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[35501]: starting fuse 2024-09-10T23:37:35.242 INFO:teuthology.orchestra.run.smithi181.stdout:44 2024-09-10T23:37:35.242 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:37:35.243 DEBUG:teuthology.orchestra.run.smithi181:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:37:35.243 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2024-09-10T23:37:35.243 DEBUG:teuthology.orchestra.run.smithi181:> import re 2024-09-10T23:37:35.243 DEBUG:teuthology.orchestra.run.smithi181:> import os 2024-09-10T23:37:35.243 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2024-09-10T23:37:35.243 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:37:35.243 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2024-09-10T23:37:35.243 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T23:37:35.243 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2024-09-10T23:37:35.243 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T23:37:35.243 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:37:35.243 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2024-09-10T23:37:35.243 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2024-09-10T23:37:35.243 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2024-09-10T23:37:35.244 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2024-09-10T23:37:35.244 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:37:35.244 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2024-09-10T23:37:35.244 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:37:35.244 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:37:35.244 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:37:35.244 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2024-09-10T23:37:35.244 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2024-09-10T23:37:35.244 DEBUG:teuthology.orchestra.run.smithi181:> return f 2024-09-10T23:37:35.244 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:37:35.244 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:37:35.244 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.1")) 2024-09-10T23:37:35.244 DEBUG:teuthology.orchestra.run.smithi181:> ' 2024-09-10T23:37:35.367 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.1.35501.asok 2024-09-10T23:37:35.370 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:37:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:37:35.379 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35501.asok 2024-09-10T23:37:35.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:35.379 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35501.asok status 2024-09-10T23:37:35.549 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T23:37:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2024-09-10T23:37:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:37:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:37:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "1", 2024-09-10T23:37:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2024-09-10T23:37:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T23:37:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "35501", 2024-09-10T23:37:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2024-09-10T23:37:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:37:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2024-09-10T23:37:35.549 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2024-09-10T23:37:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 6781, 2024-09-10T23:37:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2024-09-10T23:37:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2024-09-10T23:37:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2024-09-10T23:37:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 6781 2024-09-10T23:37:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:37:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:37:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:37:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:37:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 2375658643 2024-09-10T23:37:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T23:37:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:37:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:37:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:37:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:37:35.550 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 2375658643 2024-09-10T23:37:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:37:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.6781 192.168.0.1:0/2375658643", 2024-09-10T23:37:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.1:0/2375658643", 2024-09-10T23:37:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2024-09-10T23:37:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 68, 2024-09-10T23:37:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 99, 2024-09-10T23:37:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:37:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2024-09-10T23:37:35.551 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2024-09-10T23:37:35.551 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T23:37:35.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:35.563 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2024-09-10T23:37:35.563 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T23:37:35.572 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2024-09-10T23:37:35.572 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T23:37:35.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:35.572 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:37:35.636 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T23:37:35.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.785+0000 7f3d3e400640 1 -- 172.21.15.94:0/3077459872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d38152c10 msgr2=0x7f3d380b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:35.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.785+0000 7f3d3e400640 1 --2- 172.21.15.94:0/3077459872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d38152c10 0x7f3d380b29d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009840 tx=0x7f3d2001c820 comp rx=0 tx=0).stop 2024-09-10T23:37:35.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.785+0000 7f3d3e400640 1 -- 172.21.15.94:0/3077459872 shutdown_connections 2024-09-10T23:37:35.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.785+0000 7f3d3e400640 1 --2- 172.21.15.94:0/3077459872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d380cfd50 0x7f3d380d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:35.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.785+0000 7f3d3e400640 1 --2- 172.21.15.94:0/3077459872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d38152c10 0x7f3d380b29d0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:35.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.785+0000 7f3d3e400640 1 -- 172.21.15.94:0/3077459872 >> 172.21.15.94:0/3077459872 conn(0x7f3d3806cdf0 msgr2=0x7f3d3806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:35.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.785+0000 7f3d3e400640 1 -- 172.21.15.94:0/3077459872 shutdown_connections 2024-09-10T23:37:35.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.785+0000 7f3d3e400640 1 -- 172.21.15.94:0/3077459872 wait complete. 2024-09-10T23:37:35.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.785+0000 7f3d3e400640 1 Processor -- start 2024-09-10T23:37:35.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.785+0000 7f3d3e400640 1 -- start start 2024-09-10T23:37:35.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d3e400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d380cfd50 0x7f3d380701e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:35.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d3e400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d38152c10 0x7f3d3815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:35.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d3e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d3815cbc0 0x7f3d3815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:35.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d3e400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d38168dc0 con 0x7f3d3815cbc0 2024-09-10T23:37:35.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d3e400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3d38168c40 con 0x7f3d380cfd50 2024-09-10T23:37:35.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d3e400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3d38168f40 con 0x7f3d38152c10 2024-09-10T23:37:35.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d37e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d3815cbc0 0x7f3d3815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:35.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d37e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d3815cbc0 0x7f3d3815afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54874/0 (socket says 172.21.15.94:54874) 2024-09-10T23:37:35.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d37e00640 1 -- 172.21.15.94:0/1698156809 learned_addr learned my addr 172.21.15.94:0/1698156809 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:35.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d36a00640 1 --2- 172.21.15.94:0/1698156809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d38152c10 0x7f3d3815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:35.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d37400640 1 --2- 172.21.15.94:0/1698156809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d380cfd50 0x7f3d380701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:35.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d37e00640 1 -- 172.21.15.94:0/1698156809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d38152c10 msgr2=0x7f3d3815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:35.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d37e00640 1 --2- 172.21.15.94:0/1698156809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d38152c10 0x7f3d3815aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:35.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d37e00640 1 -- 172.21.15.94:0/1698156809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d380cfd50 msgr2=0x7f3d380701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:35.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d37e00640 1 --2- 172.21.15.94:0/1698156809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d380cfd50 0x7f3d380701e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:35.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d37e00640 1 -- 172.21.15.94:0/1698156809 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d381f84d0 con 0x7f3d3815cbc0 2024-09-10T23:37:35.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d37e00640 1 --2- 172.21.15.94:0/1698156809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d3815cbc0 0x7f3d3815afc0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000ea20 tx=0x7f3d3000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:35.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d2f400640 1 -- 172.21.15.94:0/1698156809 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3000cde0 con 0x7f3d3815cbc0 2024-09-10T23:37:35.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d2f400640 1 -- 172.21.15.94:0/1698156809 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d30004c40 con 0x7f3d3815cbc0 2024-09-10T23:37:35.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d2f400640 1 -- 172.21.15.94:0/1698156809 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30005280 con 0x7f3d3815cbc0 2024-09-10T23:37:35.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d3e400640 1 -- 172.21.15.94:0/1698156809 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d38153250 con 0x7f3d3815cbc0 2024-09-10T23:37:35.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d3e400640 1 -- 172.21.15.94:0/1698156809 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d38153810 con 0x7f3d3815cbc0 2024-09-10T23:37:35.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.789+0000 7f3d3e400640 1 -- 172.21.15.94:0/1698156809 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d381f8260 con 0x7f3d3815cbc0 2024-09-10T23:37:35.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.793+0000 7f3d2f400640 1 -- 172.21.15.94:0/1698156809 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f3d3000f040 con 0x7f3d3815cbc0 2024-09-10T23:37:35.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.793+0000 7f3d2f400640 1 --2- 172.21.15.94:0/1698156809 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3d14072700 0x7f3d14074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:35.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.793+0000 7f3d2f400640 1 -- 172.21.15.94:0/1698156809 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f3d30093ab0 con 0x7f3d3815cbc0 2024-09-10T23:37:35.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.793+0000 7f3d37400640 1 --2- 172.21.15.94:0/1698156809 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3d14072700 0x7f3d14074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:35.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.793+0000 7f3d37400640 1 --2- 172.21.15.94:0/1698156809 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3d14072700 0x7f3d14074bc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3d200af000 tx=0x7f3d2001b060 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:35.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.797+0000 7f3d2f400640 1 -- 172.21.15.94:0/1698156809 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3d30016030 con 0x7f3d3815cbc0 2024-09-10T23:37:35.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:35.997+0000 7f3d3e400640 1 -- 172.21.15.94:0/1698156809 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f3d380d0200 con 0x7f3d3815cbc0 2024-09-10T23:37:36.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.621+0000 7f3d2f400640 1 -- 172.21.15.94:0/1698156809 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v69) ==== 136+0+0 (secure 0 0 0) 0x7f3d3005e920 con 0x7f3d3815cbc0 2024-09-10T23:37:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.625+0000 7f3d3e400640 1 -- 172.21.15.94:0/1698156809 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3d14072700 msgr2=0x7f3d14074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.625+0000 7f3d3e400640 1 --2- 172.21.15.94:0/1698156809 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3d14072700 0x7f3d14074bc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3d200af000 tx=0x7f3d2001b060 comp rx=0 tx=0).stop 2024-09-10T23:37:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.625+0000 7f3d3e400640 1 -- 172.21.15.94:0/1698156809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d3815cbc0 msgr2=0x7f3d3815afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.625+0000 7f3d3e400640 1 --2- 172.21.15.94:0/1698156809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d3815cbc0 0x7f3d3815afc0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000ea20 tx=0x7f3d3000eee0 comp rx=0 tx=0).stop 2024-09-10T23:37:36.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.625+0000 7f3d3e400640 1 -- 172.21.15.94:0/1698156809 shutdown_connections 2024-09-10T23:37:36.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.625+0000 7f3d3e400640 1 --2- 172.21.15.94:0/1698156809 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3d14072700 0x7f3d14074bc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:36.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.625+0000 7f3d3e400640 1 --2- 172.21.15.94:0/1698156809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d3815cbc0 0x7f3d3815afc0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:36.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.625+0000 7f3d3e400640 1 --2- 172.21.15.94:0/1698156809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d38152c10 0x7f3d3815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:36.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.625+0000 7f3d3e400640 1 --2- 172.21.15.94:0/1698156809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d380cfd50 0x7f3d380701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:36.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.625+0000 7f3d3e400640 1 -- 172.21.15.94:0/1698156809 >> 172.21.15.94:0/1698156809 conn(0x7f3d3806cdf0 msgr2=0x7f3d380b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:36.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.625+0000 7f3d3e400640 1 -- 172.21.15.94:0/1698156809 shutdown_connections 2024-09-10T23:37:36.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.625+0000 7f3d3e400640 1 -- 172.21.15.94:0/1698156809 wait complete. 2024-09-10T23:37:36.647 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T23:37:36.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.789+0000 7f9fd6a00640 1 -- 172.21.15.94:0/476356263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd00c6e60 msgr2=0x7f9fd0166970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:36.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.789+0000 7f9fd6a00640 1 --2- 172.21.15.94:0/476356263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd00c6e60 0x7f9fd0166970 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009870 tx=0x7f9fc801c7d0 comp rx=0 tx=0).stop 2024-09-10T23:37:36.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.789+0000 7f9fd6a00640 1 -- 172.21.15.94:0/476356263 shutdown_connections 2024-09-10T23:37:36.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.789+0000 7f9fd6a00640 1 --2- 172.21.15.94:0/476356263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd00c6e60 0x7f9fd0166970 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:36.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.789+0000 7f9fd6a00640 1 -- 172.21.15.94:0/476356263 >> 172.21.15.94:0/476356263 conn(0x7f9fd00d1c60 msgr2=0x7f9fd00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:36.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.789+0000 7f9fd6a00640 1 -- 172.21.15.94:0/476356263 shutdown_connections 2024-09-10T23:37:36.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.789+0000 7f9fd6a00640 1 -- 172.21.15.94:0/476356263 wait complete. 2024-09-10T23:37:36.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.789+0000 7f9fd6a00640 1 Processor -- start 2024-09-10T23:37:36.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.789+0000 7f9fd6a00640 1 -- start start 2024-09-10T23:37:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd6a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd00c6e60 0x7f9fd0152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fd00d2780 0x7f9fd015c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd015bf60 0x7f9fd015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd6a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd0169d90 con 0x7f9fd015bf60 2024-09-10T23:37:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd6a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd0169c10 con 0x7f9fd00d2780 2024-09-10T23:37:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd6a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd0169f10 con 0x7f9fd00c6e60 2024-09-10T23:37:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd015bf60 0x7f9fd015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd015bf60 0x7f9fd015ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54878/0 (socket says 172.21.15.94:54878) 2024-09-10T23:37:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd4a00640 1 -- 172.21.15.94:0/584284013 learned_addr learned my addr 172.21.15.94:0/584284013 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fcfe00640 1 --2- 172.21.15.94:0/584284013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd00c6e60 0x7f9fd0152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fcf400640 1 --2- 172.21.15.94:0/584284013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fd00d2780 0x7f9fd015c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd4a00640 1 -- 172.21.15.94:0/584284013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd00c6e60 msgr2=0x7f9fd0152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd4a00640 1 --2- 172.21.15.94:0/584284013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd00c6e60 0x7f9fd0152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd4a00640 1 -- 172.21.15.94:0/584284013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fd00d2780 msgr2=0x7f9fd015c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd4a00640 1 --2- 172.21.15.94:0/584284013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fd00d2780 0x7f9fd015c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd4a00640 1 -- 172.21.15.94:0/584284013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd0152da0 con 0x7f9fd015bf60 2024-09-10T23:37:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd4a00640 1 --2- 172.21.15.94:0/584284013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd015bf60 0x7f9fd015ce30 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f9fc801ccb0 tx=0x7f9fc80a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fc7e00640 1 -- 172.21.15.94:0/584284013 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc80bb070 con 0x7f9fd015bf60 2024-09-10T23:37:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fc7e00640 1 -- 172.21.15.94:0/584284013 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fc8004470 con 0x7f9fd015bf60 2024-09-10T23:37:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fc7e00640 1 -- 172.21.15.94:0/584284013 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc80b2040 con 0x7f9fd015bf60 2024-09-10T23:37:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd6a00640 1 -- 172.21.15.94:0/584284013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd00cb450 con 0x7f9fd015bf60 2024-09-10T23:37:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd6a00640 1 -- 172.21.15.94:0/584284013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd006dd30 con 0x7f9fd015bf60 2024-09-10T23:37:36.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fd6a00640 1 -- 172.21.15.94:0/584284013 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd00cfce0 con 0x7f9fd015bf60 2024-09-10T23:37:36.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.793+0000 7f9fc7e00640 1 -- 172.21.15.94:0/584284013 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f9fc80b5070 con 0x7f9fd015bf60 2024-09-10T23:37:36.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.797+0000 7f9fc7e00640 1 --2- 172.21.15.94:0/584284013 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9fb0072700 0x7f9fb0074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:36.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.797+0000 7f9fc7e00640 1 -- 172.21.15.94:0/584284013 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f9fc812ca00 con 0x7f9fd015bf60 2024-09-10T23:37:36.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.797+0000 7f9fcfe00640 1 --2- 172.21.15.94:0/584284013 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9fb0072700 0x7f9fb0074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:36.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.797+0000 7f9fcfe00640 1 --2- 172.21.15.94:0/584284013 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9fb0072700 0x7f9fb0074bc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0002100 tx=0x7f9fc0009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:36.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.801+0000 7f9fc7e00640 1 -- 172.21.15.94:0/584284013 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9fc80f1de0 con 0x7f9fd015bf60 2024-09-10T23:37:36.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.989+0000 7f9fd6a00640 1 -- 172.21.15.94:0/584284013 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f9fd00b7850 con 0x7f9fb0072700 2024-09-10T23:37:36.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.993+0000 7f9fc7e00640 1 -- 172.21.15.94:0/584284013 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f9fd00b7850 con 0x7f9fb0072700 2024-09-10T23:37:36.995 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:37:36.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:36.995 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephfs" 2024-09-10T23:37:36.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:36.995 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:37:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.993+0000 7f9fd6a00640 1 -- 172.21.15.94:0/584284013 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9fb0072700 msgr2=0x7f9fb0074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.993+0000 7f9fd6a00640 1 --2- 172.21.15.94:0/584284013 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9fb0072700 0x7f9fb0074bc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0002100 tx=0x7f9fc0009040 comp rx=0 tx=0).stop 2024-09-10T23:37:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.993+0000 7f9fd6a00640 1 -- 172.21.15.94:0/584284013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd015bf60 msgr2=0x7f9fd015ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.993+0000 7f9fd6a00640 1 --2- 172.21.15.94:0/584284013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd015bf60 0x7f9fd015ce30 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f9fc801ccb0 tx=0x7f9fc80a5690 comp rx=0 tx=0).stop 2024-09-10T23:37:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.993+0000 7f9fd6a00640 1 -- 172.21.15.94:0/584284013 shutdown_connections 2024-09-10T23:37:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.993+0000 7f9fd6a00640 1 --2- 172.21.15.94:0/584284013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fd00d2780 0x7f9fd015c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.993+0000 7f9fd6a00640 1 --2- 172.21.15.94:0/584284013 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9fb0072700 0x7f9fb0074bc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:36.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.993+0000 7f9fd6a00640 1 --2- 172.21.15.94:0/584284013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd015bf60 0x7f9fd015ce30 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:36.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.993+0000 7f9fd6a00640 1 --2- 172.21.15.94:0/584284013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd00c6e60 0x7f9fd0152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:36.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.993+0000 7f9fd6a00640 1 -- 172.21.15.94:0/584284013 >> 172.21.15.94:0/584284013 conn(0x7f9fd00d1c60 msgr2=0x7f9fd00c7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:36.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.993+0000 7f9fd6a00640 1 -- 172.21.15.94:0/584284013 shutdown_connections 2024-09-10T23:37:36.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:36.993+0000 7f9fd6a00640 1 -- 172.21.15.94:0/584284013 wait complete. 2024-09-10T23:37:37.015 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:37:37.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.157+0000 7f9989a00640 1 -- 172.21.15.94:0/2568473083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99840b3170 msgr2=0x7f99840b1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:37.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.157+0000 7f9989a00640 1 --2- 172.21.15.94:0/2568473083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99840b3170 0x7f99840b1110 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f996c009900 tx=0x7f996c01c8b0 comp rx=0 tx=0).stop 2024-09-10T23:37:37.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.157+0000 7f9989a00640 1 -- 172.21.15.94:0/2568473083 shutdown_connections 2024-09-10T23:37:37.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.157+0000 7f9989a00640 1 --2- 172.21.15.94:0/2568473083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99840d02f0 0x7f9984165790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:37.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.157+0000 7f9989a00640 1 --2- 172.21.15.94:0/2568473083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99840b3170 0x7f99840b1110 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:37.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.157+0000 7f9989a00640 1 -- 172.21.15.94:0/2568473083 >> 172.21.15.94:0/2568473083 conn(0x7f998409aa70 msgr2=0x7f998409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:37.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.157+0000 7f9989a00640 1 -- 172.21.15.94:0/2568473083 shutdown_connections 2024-09-10T23:37:37.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.157+0000 7f9989a00640 1 -- 172.21.15.94:0/2568473083 wait complete. 2024-09-10T23:37:37.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.157+0000 7f9989a00640 1 Processor -- start 2024-09-10T23:37:37.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f9989a00640 1 -- start start 2024-09-10T23:37:37.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f9989a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99840b3170 0x7f998414ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:37.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f9989a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99840d02f0 0x7f99841f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:37.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f9989a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99840d1bd0 0x7f99840c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:37.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f9989a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9984168b50 con 0x7f99840d02f0 2024-09-10T23:37:37.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f9989a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f99841689d0 con 0x7f99840d1bd0 2024-09-10T23:37:37.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f9989a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9984168cd0 con 0x7f99840b3170 2024-09-10T23:37:37.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f997be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99840d02f0 0x7f99841f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:37.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f997be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99840d02f0 0x7f99841f8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54896/0 (socket says 172.21.15.94:54896) 2024-09-10T23:37:37.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f997be00640 1 -- 172.21.15.94:0/1923830514 learned_addr learned my addr 172.21.15.94:0/1923830514 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:37.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f9982a00640 1 --2- 172.21.15.94:0/1923830514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99840b3170 0x7f998414ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:37.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f9983400640 1 --2- 172.21.15.94:0/1923830514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99840d1bd0 0x7f99840c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:37.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f997be00640 1 -- 172.21.15.94:0/1923830514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99840b3170 msgr2=0x7f998414ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:37.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f997be00640 1 --2- 172.21.15.94:0/1923830514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99840b3170 0x7f998414ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:37.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f997be00640 1 -- 172.21.15.94:0/1923830514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99840d1bd0 msgr2=0x7f99840c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:37.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f997be00640 1 --2- 172.21.15.94:0/1923830514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99840d1bd0 0x7f99840c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:37.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f997be00640 1 -- 172.21.15.94:0/1923830514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99840c5970 con 0x7f99840d02f0 2024-09-10T23:37:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f997be00640 1 --2- 172.21.15.94:0/1923830514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99840d02f0 0x7f99841f8230 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f996c009850 tx=0x7f996c0a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f997aa00640 1 -- 172.21.15.94:0/1923830514 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c0ba020 con 0x7f99840d02f0 2024-09-10T23:37:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f997aa00640 1 -- 172.21.15.94:0/1923830514 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f996c0053e0 con 0x7f99840d02f0 2024-09-10T23:37:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f997aa00640 1 -- 172.21.15.94:0/1923830514 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c0b2040 con 0x7f99840d02f0 2024-09-10T23:37:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f9989a00640 1 -- 172.21.15.94:0/1923830514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99840c7560 con 0x7f99840d02f0 2024-09-10T23:37:37.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f9989a00640 1 -- 172.21.15.94:0/1923830514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99840c5c10 con 0x7f99840d02f0 2024-09-10T23:37:37.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.161+0000 7f9989a00640 1 -- 172.21.15.94:0/1923830514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f998406d6d0 con 0x7f99840d02f0 2024-09-10T23:37:37.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.165+0000 7f997aa00640 1 -- 172.21.15.94:0/1923830514 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7f996c0046d0 con 0x7f99840d02f0 2024-09-10T23:37:37.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.165+0000 7f997aa00640 1 --2- 172.21.15.94:0/1923830514 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f99600726f0 0x7f9960074bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:37.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.165+0000 7f997aa00640 1 -- 172.21.15.94:0/1923830514 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f996c12db40 con 0x7f99840d02f0 2024-09-10T23:37:37.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.165+0000 7f9982a00640 1 --2- 172.21.15.94:0/1923830514 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f99600726f0 0x7f9960074bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:37.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.165+0000 7f9982a00640 1 --2- 172.21.15.94:0/1923830514 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f99600726f0 0x7f9960074bb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9974007fd0 tx=0x7f9974009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:37.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.169+0000 7f997aa00640 1 -- 172.21.15.94:0/1923830514 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f996c0c4020 con 0x7f99840d02f0 2024-09-10T23:37:37.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.333+0000 7f9989a00640 1 -- 172.21.15.94:0/1923830514 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f99840b1110 con 0x7f99840d02f0 2024-09-10T23:37:37.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.333+0000 7f997aa00640 1 -- 172.21.15.94:0/1923830514 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f996c0164d0 con 0x7f99840d02f0 2024-09-10T23:37:37.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.337+0000 7f9989a00640 1 -- 172.21.15.94:0/1923830514 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f99600726f0 msgr2=0x7f9960074bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:37.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.337+0000 7f9989a00640 1 --2- 172.21.15.94:0/1923830514 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f99600726f0 0x7f9960074bb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9974007fd0 tx=0x7f9974009040 comp rx=0 tx=0).stop 2024-09-10T23:37:37.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.337+0000 7f9989a00640 1 -- 172.21.15.94:0/1923830514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99840d02f0 msgr2=0x7f99841f8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:37.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.337+0000 7f9989a00640 1 --2- 172.21.15.94:0/1923830514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99840d02f0 0x7f99841f8230 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f996c009850 tx=0x7f996c0a5770 comp rx=0 tx=0).stop 2024-09-10T23:37:37.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.337+0000 7f9989a00640 1 -- 172.21.15.94:0/1923830514 shutdown_connections 2024-09-10T23:37:37.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.337+0000 7f9989a00640 1 --2- 172.21.15.94:0/1923830514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99840d1bd0 0x7f99840c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:37.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.337+0000 7f9989a00640 1 --2- 172.21.15.94:0/1923830514 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f99600726f0 0x7f9960074bb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:37.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.337+0000 7f9989a00640 1 --2- 172.21.15.94:0/1923830514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99840d02f0 0x7f99841f8230 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:37.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.337+0000 7f9989a00640 1 --2- 172.21.15.94:0/1923830514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99840b3170 0x7f998414ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:37.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.337+0000 7f9989a00640 1 -- 172.21.15.94:0/1923830514 >> 172.21.15.94:0/1923830514 conn(0x7f998409aa70 msgr2=0x7f99840d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:37.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.337+0000 7f9989a00640 1 -- 172.21.15.94:0/1923830514 shutdown_connections 2024-09-10T23:37:37.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.337+0000 7f9989a00640 1 -- 172.21.15.94:0/1923830514 wait complete. 2024-09-10T23:37:37.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:37.358 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad) 2024-09-10T23:37:37.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:37.394 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/fake) 2024-09-10T23:37:37.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:37.471 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T23:37:37.525 INFO:teuthology.orchestra.run.smithi094.stdout:cephfs_data 2024-09-10T23:37:37.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:37.525 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad) 2024-09-10T23:37:37.595 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:37:37.596 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol55860/.meta 2024-09-10T23:37:37.662 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol55860 2024-09-10T23:37:37.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.849+0000 7fb6a5a00640 1 -- 172.21.15.94:0/4022129995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a00cfc00 msgr2=0x7fb6a00b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:37.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.849+0000 7fb6a5a00640 1 --2- 172.21.15.94:0/4022129995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a00cfc00 0x7fb6a00b1b30 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb688009900 tx=0x7fb68801f0d0 comp rx=0 tx=0).stop 2024-09-10T23:37:37.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.849+0000 7fb6a5a00640 1 -- 172.21.15.94:0/4022129995 shutdown_connections 2024-09-10T23:37:37.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.849+0000 7fb6a5a00640 1 --2- 172.21.15.94:0/4022129995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a00cfc00 0x7fb6a00b1b30 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:37.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.849+0000 7fb6a5a00640 1 --2- 172.21.15.94:0/4022129995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a015f2d0 0x7fb6a00b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:37.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.849+0000 7fb6a5a00640 1 -- 172.21.15.94:0/4022129995 >> 172.21.15.94:0/4022129995 conn(0x7fb6a00d1c90 msgr2=0x7fb6a00d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:37.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.849+0000 7fb6a5a00640 1 -- 172.21.15.94:0/4022129995 shutdown_connections 2024-09-10T23:37:37.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.849+0000 7fb6a5a00640 1 -- 172.21.15.94:0/4022129995 wait complete. 2024-09-10T23:37:37.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb6a5a00640 1 Processor -- start 2024-09-10T23:37:37.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb6a5a00640 1 -- start start 2024-09-10T23:37:37.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb6a5a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a00b2790 0x7fb6a00c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:37.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb6a5a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a00cfc00 0x7fb6a00caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:37.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb6a5a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6a015f2d0 0x7fb6a01f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:37.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb6a5a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a015f830 con 0x7fb6a00cfc00 2024-09-10T23:37:37.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb6a5a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a015f6b0 con 0x7fb6a015f2d0 2024-09-10T23:37:37.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb6a5a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a01661e0 con 0x7fb6a00b2790 2024-09-10T23:37:37.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb69ea00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a00b2790 0x7fb6a00c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:37.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb69f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6a015f2d0 0x7fb6a01f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:37.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb69ea00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a00b2790 0x7fb6a00c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:48740/0 (socket says 172.21.15.94:48740) 2024-09-10T23:37:37.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb69ea00640 1 -- 172.21.15.94:0/3634824880 learned_addr learned my addr 172.21.15.94:0/3634824880 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:37.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb696000640 1 --2- 172.21.15.94:0/3634824880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a00cfc00 0x7fb6a00caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:37.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb696000640 1 -- 172.21.15.94:0/3634824880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a00b2790 msgr2=0x7fb6a00c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:37.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb696000640 1 --2- 172.21.15.94:0/3634824880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a00b2790 0x7fb6a00c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:37.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb696000640 1 -- 172.21.15.94:0/3634824880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6a015f2d0 msgr2=0x7fb6a01f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:37.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb696000640 1 --2- 172.21.15.94:0/3634824880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6a015f2d0 0x7fb6a01f8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:37.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb696000640 1 -- 172.21.15.94:0/3634824880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a00cb390 con 0x7fb6a00cfc00 2024-09-10T23:37:37.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb69ea00640 1 --2- 172.21.15.94:0/3634824880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a00b2790 0x7fb6a00c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:37:37.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb696000640 1 --2- 172.21.15.94:0/3634824880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a00cfc00 0x7fb6a00caad0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fb6880af000 tx=0x7fb6880a8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:37.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb697400640 1 -- 172.21.15.94:0/3634824880 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6880b5040 con 0x7fb6a00cfc00 2024-09-10T23:37:37.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb697400640 1 -- 172.21.15.94:0/3634824880 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb688005340 con 0x7fb6a00cfc00 2024-09-10T23:37:37.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb6a5a00640 1 -- 172.21.15.94:0/3634824880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a00cb670 con 0x7fb6a00cfc00 2024-09-10T23:37:37.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb697400640 1 -- 172.21.15.94:0/3634824880 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6880046d0 con 0x7fb6a00cfc00 2024-09-10T23:37:37.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.853+0000 7fb6a5a00640 1 -- 172.21.15.94:0/3634824880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a006deb0 con 0x7fb6a00cfc00 2024-09-10T23:37:37.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.857+0000 7fb6a5a00640 1 -- 172.21.15.94:0/3634824880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a00cc130 con 0x7fb6a00cfc00 2024-09-10T23:37:37.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.857+0000 7fb697400640 1 -- 172.21.15.94:0/3634824880 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fb688004890 con 0x7fb6a00cfc00 2024-09-10T23:37:37.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.857+0000 7fb697400640 1 --2- 172.21.15.94:0/3634824880 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb67c072700 0x7fb67c074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:37.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.857+0000 7fb697400640 1 -- 172.21.15.94:0/3634824880 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5796+0+0 (secure 0 0 0) 0x7fb688130640 con 0x7fb6a00cfc00 2024-09-10T23:37:37.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.857+0000 7fb69ea00640 1 --2- 172.21.15.94:0/3634824880 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb67c072700 0x7fb67c074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:37.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.857+0000 7fb69ea00640 1 --2- 172.21.15.94:0/3634824880 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb67c072700 0x7fb67c074bc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb690005fd0 tx=0x7fb69000b390 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:37.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:37.861+0000 7fb697400640 1 -- 172.21.15.94:0/3634824880 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb6880bf060 con 0x7fb6a00cfc00 2024-09-10T23:37:38.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.057+0000 7fb6a5a00640 1 -- 172.21.15.94:0/3634824880 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol55860", "target": ["mon-mgr", ""]}) -- 0x7fb6a00c8550 con 0x7fb67c072700 2024-09-10T23:37:38.064 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:38.061+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:38.064 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:38.061+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:38.064 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:38.061+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:38.064 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:38.061+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:38.064 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:38.061+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:38.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.201+0000 7fb697400640 1 -- 172.21.15.94:0/3634824880 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7fb69000bc80 con 0x7fb67c072700 2024-09-10T23:37:38.205 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad 2024-09-10T23:37:38.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.205+0000 7fb6a5a00640 1 -- 172.21.15.94:0/3634824880 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb67c072700 msgr2=0x7fb67c074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:38.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.205+0000 7fb6a5a00640 1 --2- 172.21.15.94:0/3634824880 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb67c072700 0x7fb67c074bc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb690005fd0 tx=0x7fb69000b390 comp rx=0 tx=0).stop 2024-09-10T23:37:38.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.205+0000 7fb6a5a00640 1 -- 172.21.15.94:0/3634824880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a00cfc00 msgr2=0x7fb6a00caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:38.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.205+0000 7fb6a5a00640 1 --2- 172.21.15.94:0/3634824880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a00cfc00 0x7fb6a00caad0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fb6880af000 tx=0x7fb6880a8c60 comp rx=0 tx=0).stop 2024-09-10T23:37:38.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.205+0000 7fb6a5a00640 1 -- 172.21.15.94:0/3634824880 shutdown_connections 2024-09-10T23:37:38.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.205+0000 7fb6a5a00640 1 --2- 172.21.15.94:0/3634824880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6a015f2d0 0x7fb6a01f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.205+0000 7fb6a5a00640 1 --2- 172.21.15.94:0/3634824880 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb67c072700 0x7fb67c074bc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.205+0000 7fb6a5a00640 1 --2- 172.21.15.94:0/3634824880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a00cfc00 0x7fb6a00caad0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.205+0000 7fb6a5a00640 1 --2- 172.21.15.94:0/3634824880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a00b2790 0x7fb6a00c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.205+0000 7fb6a5a00640 1 -- 172.21.15.94:0/3634824880 >> 172.21.15.94:0/3634824880 conn(0x7fb6a00d1c90 msgr2=0x7fb6a00afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:38.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.205+0000 7fb6a5a00640 1 -- 172.21.15.94:0/3634824880 shutdown_connections 2024-09-10T23:37:38.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.205+0000 7fb6a5a00640 1 -- 172.21.15.94:0/3634824880 wait complete. 2024-09-10T23:37:38.226 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-10T23:37:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.369+0000 7fca94000640 1 -- 172.21.15.94:0/3690071980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca740052f0 msgr2=0x7fca7400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.369+0000 7fca94000640 1 --2- 172.21.15.94:0/3690071980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca740052f0 0x7fca7400b9d0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0c72f0 tx=0x7fca8401d5d0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.369+0000 7fca94000640 1 -- 172.21.15.94:0/3690071980 shutdown_connections 2024-09-10T23:37:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.369+0000 7fca94000640 1 --2- 172.21.15.94:0/3690071980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca740052f0 0x7fca7400b9d0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.369+0000 7fca94000640 1 -- 172.21.15.94:0/3690071980 >> 172.21.15.94:0/3690071980 conn(0x7fca8c0d1c60 msgr2=0x7fca8c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:38.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.369+0000 7fca94000640 1 -- 172.21.15.94:0/3690071980 shutdown_connections 2024-09-10T23:37:38.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.369+0000 7fca94000640 1 -- 172.21.15.94:0/3690071980 wait complete. 2024-09-10T23:37:38.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.369+0000 7fca94000640 1 Processor -- start 2024-09-10T23:37:38.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.369+0000 7fca94000640 1 -- start start 2024-09-10T23:37:38.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca94000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca740052f0 0x7fca8c15de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:38.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca94000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca8c1e9170 0x7fca8c06ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:38.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca94000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca8c1e95d0 0x7fca8c06f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:38.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca94000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c0cfce0 con 0x7fca8c1e9170 2024-09-10T23:37:38.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca94000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c0cd660 con 0x7fca8c1e95d0 2024-09-10T23:37:38.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca94000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fca8c0cfe60 con 0x7fca740052f0 2024-09-10T23:37:38.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca90c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca8c1e9170 0x7fca8c06ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:38.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca90c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca8c1e9170 0x7fca8c06ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54954/0 (socket says 172.21.15.94:54954) 2024-09-10T23:37:38.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca90c00640 1 -- 172.21.15.94:0/3814389964 learned_addr learned my addr 172.21.15.94:0/3814389964 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:38.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca92000640 1 --2- 172.21.15.94:0/3814389964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca8c1e95d0 0x7fca8c06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:38.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca91600640 1 --2- 172.21.15.94:0/3814389964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca740052f0 0x7fca8c15de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:38.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca90c00640 1 -- 172.21.15.94:0/3814389964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca740052f0 msgr2=0x7fca8c15de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:38.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca90c00640 1 --2- 172.21.15.94:0/3814389964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca740052f0 0x7fca8c15de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca90c00640 1 -- 172.21.15.94:0/3814389964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca8c1e95d0 msgr2=0x7fca8c06f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:38.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca90c00640 1 --2- 172.21.15.94:0/3814389964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca8c1e95d0 0x7fca8c06f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca90c00640 1 -- 172.21.15.94:0/3814389964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8c06c690 con 0x7fca8c1e9170 2024-09-10T23:37:38.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca91600640 1 --2- 172.21.15.94:0/3814389964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca740052f0 0x7fca8c15de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:38.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca90c00640 1 --2- 172.21.15.94:0/3814389964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca8c1e9170 0x7fca8c06ebf0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fca78004e00 tx=0x7fca78001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:38.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca89600640 1 -- 172.21.15.94:0/3814389964 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78015020 con 0x7fca8c1e9170 2024-09-10T23:37:38.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca89600640 1 -- 172.21.15.94:0/3814389964 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca78011960 con 0x7fca8c1e9170 2024-09-10T23:37:38.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca94000640 1 -- 172.21.15.94:0/3814389964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca8c0b6b60 con 0x7fca8c1e9170 2024-09-10T23:37:38.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca89600640 1 -- 172.21.15.94:0/3814389964 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78011d90 con 0x7fca8c1e9170 2024-09-10T23:37:38.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca94000640 1 -- 172.21.15.94:0/3814389964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca8c0b7120 con 0x7fca8c1e9170 2024-09-10T23:37:38.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca94000640 1 -- 172.21.15.94:0/3814389964 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca8c0d02f0 con 0x7fca8c1e9170 2024-09-10T23:37:38.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.373+0000 7fca89600640 1 -- 172.21.15.94:0/3814389964 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 96496+0+0 (secure 0 0 0) 0x7fca7801a400 con 0x7fca8c1e9170 2024-09-10T23:37:38.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.377+0000 7fca89600640 1 --2- 172.21.15.94:0/3814389964 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fca6c072700 0x7fca6c074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:38.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.377+0000 7fca89600640 1 -- 172.21.15.94:0/3814389964 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5796+0+0 (secure 0 0 0) 0x7fca78094300 con 0x7fca8c1e9170 2024-09-10T23:37:38.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.377+0000 7fca91600640 1 --2- 172.21.15.94:0/3814389964 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fca6c072700 0x7fca6c074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:38.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.377+0000 7fca91600640 1 --2- 172.21.15.94:0/3814389964 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fca6c072700 0x7fca6c074bc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fca80000fc0 tx=0x7fca800067b0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:38.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.381+0000 7fca89600640 1 -- 172.21.15.94:0/3814389964 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fca7801f020 con 0x7fca8c1e9170 2024-09-10T23:37:38.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.569+0000 7fca94000640 1 -- 172.21.15.94:0/3814389964 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7fca8c0b7920 con 0x7fca6c072700 2024-09-10T23:37:38.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.573+0000 7fca89600640 1 -- 172.21.15.94:0/3814389964 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fca800044a0 con 0x7fca6c072700 2024-09-10T23:37:38.574 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:37:38.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:38.575 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subvol55860" 2024-09-10T23:37:38.575 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:38.575 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:37:38.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.573+0000 7fca94000640 1 -- 172.21.15.94:0/3814389964 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fca6c072700 msgr2=0x7fca6c074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:38.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.573+0000 7fca94000640 1 --2- 172.21.15.94:0/3814389964 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fca6c072700 0x7fca6c074bc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fca80000fc0 tx=0x7fca800067b0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.573+0000 7fca94000640 1 -- 172.21.15.94:0/3814389964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca8c1e9170 msgr2=0x7fca8c06ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:38.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.573+0000 7fca94000640 1 --2- 172.21.15.94:0/3814389964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca8c1e9170 0x7fca8c06ebf0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fca78004e00 tx=0x7fca78001640 comp rx=0 tx=0).stop 2024-09-10T23:37:38.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.573+0000 7fca94000640 1 -- 172.21.15.94:0/3814389964 shutdown_connections 2024-09-10T23:37:38.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.573+0000 7fca94000640 1 --2- 172.21.15.94:0/3814389964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca8c1e95d0 0x7fca8c06f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.573+0000 7fca94000640 1 --2- 172.21.15.94:0/3814389964 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fca6c072700 0x7fca6c074bc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.573+0000 7fca94000640 1 --2- 172.21.15.94:0/3814389964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fca8c1e9170 0x7fca8c06ebf0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.573+0000 7fca94000640 1 --2- 172.21.15.94:0/3814389964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca740052f0 0x7fca8c15de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.573+0000 7fca94000640 1 -- 172.21.15.94:0/3814389964 >> 172.21.15.94:0/3814389964 conn(0x7fca8c0d1c60 msgr2=0x7fca8c0707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:38.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.573+0000 7fca94000640 1 -- 172.21.15.94:0/3814389964 shutdown_connections 2024-09-10T23:37:38.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.573+0000 7fca94000640 1 -- 172.21.15.94:0/3814389964 wait complete. 2024-09-10T23:37:38.596 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol55860 2024-09-10T23:37:38.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.745+0000 7f8770e00640 1 -- 172.21.15.94:0/4090479314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f876c0d0840 msgr2=0x7f876c0cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:38.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.745+0000 7f8770e00640 1 --2- 172.21.15.94:0/4090479314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f876c0d0840 0x7f876c0cd710 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8764009840 tx=0x7f876401c790 comp rx=0 tx=0).stop 2024-09-10T23:37:38.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.745+0000 7f8770e00640 1 -- 172.21.15.94:0/4090479314 shutdown_connections 2024-09-10T23:37:38.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.745+0000 7f8770e00640 1 --2- 172.21.15.94:0/4090479314 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f876c0d0840 0x7f876c0cd710 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.745+0000 7f8770e00640 1 -- 172.21.15.94:0/4090479314 >> 172.21.15.94:0/4090479314 conn(0x7f876c0d1c60 msgr2=0x7f876c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:38.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.745+0000 7f8770e00640 1 -- 172.21.15.94:0/4090479314 shutdown_connections 2024-09-10T23:37:38.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.745+0000 7f8770e00640 1 -- 172.21.15.94:0/4090479314 wait complete. 2024-09-10T23:37:38.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8770e00640 1 Processor -- start 2024-09-10T23:37:38.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8770e00640 1 -- start start 2024-09-10T23:37:38.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8770e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c0ce1e0 0x7f876c1652b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:38.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8770e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f876c0ce890 0x7f876c151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:38.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8770e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f876c0d0840 0x7f876c1568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:38.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8770e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f876c16a3d0 con 0x7f876c0ce1e0 2024-09-10T23:37:38.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8770e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f876c16a250 con 0x7f876c0ce890 2024-09-10T23:37:38.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8770e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f876c16a550 con 0x7f876c0d0840 2024-09-10T23:37:38.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8769e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c0ce1e0 0x7f876c1652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:38.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8769e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c0ce1e0 0x7f876c1652b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54960/0 (socket says 172.21.15.94:54960) 2024-09-10T23:37:38.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8769e00640 1 -- 172.21.15.94:0/2429044903 learned_addr learned my addr 172.21.15.94:0/2429044903 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:38.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8769e00640 1 -- 172.21.15.94:0/2429044903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f876c0d0840 msgr2=0x7f876c1568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:38.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8769e00640 1 --2- 172.21.15.94:0/2429044903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f876c0d0840 0x7f876c1568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8769e00640 1 -- 172.21.15.94:0/2429044903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f876c0ce890 msgr2=0x7f876c151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:38.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8769e00640 1 --2- 172.21.15.94:0/2429044903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f876c0ce890 0x7f876c151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8769e00640 1 -- 172.21.15.94:0/2429044903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876c06f0a0 con 0x7f876c0ce1e0 2024-09-10T23:37:38.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8769e00640 1 --2- 172.21.15.94:0/2429044903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c0ce1e0 0x7f876c1652b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f875c00c870 tx=0x7f875c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:38.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8762a00640 1 -- 172.21.15.94:0/2429044903 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c003b80 con 0x7f876c0ce1e0 2024-09-10T23:37:38.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8762a00640 1 -- 172.21.15.94:0/2429044903 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f875c003d40 con 0x7f876c0ce1e0 2024-09-10T23:37:38.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8762a00640 1 -- 172.21.15.94:0/2429044903 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c00aab0 con 0x7f876c0ce1e0 2024-09-10T23:37:38.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8770e00640 1 -- 172.21.15.94:0/2429044903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876c06eae0 con 0x7f876c0ce1e0 2024-09-10T23:37:38.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.749+0000 7f8770e00640 1 -- 172.21.15.94:0/2429044903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f876c1560b0 con 0x7f876c0ce1e0 2024-09-10T23:37:38.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.753+0000 7f8770e00640 1 -- 172.21.15.94:0/2429044903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f876c0cd710 con 0x7f876c0ce1e0 2024-09-10T23:37:38.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.753+0000 7f8762a00640 1 -- 172.21.15.94:0/2429044903 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 96547+0+0 (secure 0 0 0) 0x7f875c00ac50 con 0x7f876c0ce1e0 2024-09-10T23:37:38.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.753+0000 7f8762a00640 1 --2- 172.21.15.94:0/2429044903 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87480727c0 0x7f8748074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:38.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.753+0000 7f8762a00640 1 -- 172.21.15.94:0/2429044903 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7f875c093650 con 0x7f876c0ce1e0 2024-09-10T23:37:38.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.753+0000 7f8761400640 1 --2- 172.21.15.94:0/2429044903 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87480727c0 0x7f8748074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:38.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.753+0000 7f8761400640 1 --2- 172.21.15.94:0/2429044903 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87480727c0 0x7f8748074c80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f876c0b0b30 tx=0x7f87540078b0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.757+0000 7f8762a00640 1 -- 172.21.15.94:0/2429044903 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f875c05e9d0 con 0x7f876c0ce1e0 2024-09-10T23:37:38.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.949+0000 7f8770e00640 1 -- 172.21.15.94:0/2429044903 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol55860", "target": ["mon-mgr", ""]}) -- 0x7f876c06d810 con 0x7f87480727c0 2024-09-10T23:37:38.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.965+0000 7f8762a00640 1 -- 172.21.15.94:0/2429044903 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+686 (secure 0 0 0) 0x7f876c06d810 con 0x7f87480727c0 2024-09-10T23:37:38.968 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:37:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: "atime": "2024-09-10 23:37:37", 2024-09-10T23:37:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_pcent": "undefined", 2024-09-10T23:37:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_quota": "infinite", 2024-09-10T23:37:38.968 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_used": 0, 2024-09-10T23:37:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "created_at": "2024-09-10 23:37:37", 2024-09-10T23:37:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "ctime": "2024-09-10 23:37:38", 2024-09-10T23:37:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "data_pool": "cephfs_data", 2024-09-10T23:37:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "features": [ 2024-09-10T23:37:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot-clone", 2024-09-10T23:37:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot-autoprotect" 2024-09-10T23:37:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:37:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "flavor": 1, 2024-09-10T23:37:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 0, 2024-09-10T23:37:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": 16877, 2024-09-10T23:37:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_addrs": [ 2024-09-10T23:37:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "172.21.15.94:6789", 2024-09-10T23:37:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: "172.21.15.181:6789", 2024-09-10T23:37:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "172.21.15.181:6790" 2024-09-10T23:37:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:37:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "mtime": "2024-09-10 23:37:37", 2024-09-10T23:37:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad", 2024-09-10T23:37:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_namespace": "", 2024-09-10T23:37:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "complete", 2024-09-10T23:37:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "subvolume", 2024-09-10T23:37:38.970 INFO:teuthology.orchestra.run.smithi094.stdout: "uid": 0 2024-09-10T23:37:38.970 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:37:38.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.969+0000 7f8770e00640 1 -- 172.21.15.94:0/2429044903 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87480727c0 msgr2=0x7f8748074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:38.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.969+0000 7f8770e00640 1 --2- 172.21.15.94:0/2429044903 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87480727c0 0x7f8748074c80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f876c0b0b30 tx=0x7f87540078b0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.969+0000 7f8770e00640 1 -- 172.21.15.94:0/2429044903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c0ce1e0 msgr2=0x7f876c1652b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:38.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.969+0000 7f8770e00640 1 --2- 172.21.15.94:0/2429044903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c0ce1e0 0x7f876c1652b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f875c00c870 tx=0x7f875c00cd30 comp rx=0 tx=0).stop 2024-09-10T23:37:38.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.969+0000 7f8770e00640 1 -- 172.21.15.94:0/2429044903 shutdown_connections 2024-09-10T23:37:38.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.969+0000 7f8770e00640 1 --2- 172.21.15.94:0/2429044903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f876c0ce890 0x7f876c151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.969+0000 7f8770e00640 1 --2- 172.21.15.94:0/2429044903 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87480727c0 0x7f8748074c80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.969+0000 7f8770e00640 1 --2- 172.21.15.94:0/2429044903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f876c0d0840 0x7f876c1568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.969+0000 7f8770e00640 1 --2- 172.21.15.94:0/2429044903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f876c0ce1e0 0x7f876c1652b0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:38.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.969+0000 7f8770e00640 1 -- 172.21.15.94:0/2429044903 >> 172.21.15.94:0/2429044903 conn(0x7f876c0d1c60 msgr2=0x7f876c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:38.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.969+0000 7f8770e00640 1 -- 172.21.15.94:0/2429044903 shutdown_connections 2024-09-10T23:37:38.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:38.969+0000 7f8770e00640 1 -- 172.21.15.94:0/2429044903 wait complete. 2024-09-10T23:37:38.990 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvol55860 10485760 2024-09-10T23:37:39.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.133+0000 7f48af200640 1 -- 172.21.15.94:0/3802990876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48a80b2890 msgr2=0x7f48a81602f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:39.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.133+0000 7f48af200640 1 --2- 172.21.15.94:0/3802990876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48a80b2890 0x7f48a81602f0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009900 tx=0x7f48a001f0d0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.133+0000 7f48af200640 1 -- 172.21.15.94:0/3802990876 shutdown_connections 2024-09-10T23:37:39.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.133+0000 7f48af200640 1 --2- 172.21.15.94:0/3802990876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48a80b2890 0x7f48a81602f0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.133+0000 7f48af200640 1 -- 172.21.15.94:0/3802990876 >> 172.21.15.94:0/3802990876 conn(0x7f48a80d1bf0 msgr2=0x7f48a80d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:39.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.133+0000 7f48af200640 1 -- 172.21.15.94:0/3802990876 shutdown_connections 2024-09-10T23:37:39.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.133+0000 7f48af200640 1 -- 172.21.15.94:0/3802990876 wait complete. 2024-09-10T23:37:39.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.133+0000 7f48af200640 1 Processor -- start 2024-09-10T23:37:39.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.133+0000 7f48af200640 1 -- start start 2024-09-10T23:37:39.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48af200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48a80b2890 0x7f48a8158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:39.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48af200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48a80cfc30 0x7f48a81599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:39.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48af200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48a80d0010 0x7f48a815de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:39.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48af200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8167020 con 0x7f48a80d0010 2024-09-10T23:37:39.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48af200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8166ea0 con 0x7f48a80cfc30 2024-09-10T23:37:39.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48af200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f48a81671a0 con 0x7f48a80b2890 2024-09-10T23:37:39.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48ad200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48a80d0010 0x7f48a815de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:39.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48ad200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48a80d0010 0x7f48a815de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54978/0 (socket says 172.21.15.94:54978) 2024-09-10T23:37:39.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48ad200640 1 -- 172.21.15.94:0/1880152862 learned_addr learned my addr 172.21.15.94:0/1880152862 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:39.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48a7400640 1 --2- 172.21.15.94:0/1880152862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48a80cfc30 0x7f48a81599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:39.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48a7e00640 1 --2- 172.21.15.94:0/1880152862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48a80b2890 0x7f48a8158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:39.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48ad200640 1 -- 172.21.15.94:0/1880152862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48a80b2890 msgr2=0x7f48a8158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:39.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48ad200640 1 --2- 172.21.15.94:0/1880152862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48a80b2890 0x7f48a8158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48ad200640 1 -- 172.21.15.94:0/1880152862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48a80cfc30 msgr2=0x7f48a81599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:39.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48ad200640 1 --2- 172.21.15.94:0/1880152862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48a80cfc30 0x7f48a81599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48ad200640 1 -- 172.21.15.94:0/1880152862 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a80c8df0 con 0x7f48a80d0010 2024-09-10T23:37:39.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48a7400640 1 --2- 172.21.15.94:0/1880152862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48a80cfc30 0x7f48a81599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:37:39.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48a7e00640 1 --2- 172.21.15.94:0/1880152862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48a80b2890 0x7f48a8158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:37:39.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48ad200640 1 --2- 172.21.15.94:0/1880152862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48a80d0010 0x7f48a815de90 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f48a00b3040 tx=0x7f48a00a8d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:39.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f489fe00640 1 -- 172.21.15.94:0/1880152862 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a00b5040 con 0x7f48a80d0010 2024-09-10T23:37:39.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f489fe00640 1 -- 172.21.15.94:0/1880152862 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48a0005310 con 0x7f48a80d0010 2024-09-10T23:37:39.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f489fe00640 1 -- 172.21.15.94:0/1880152862 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a00046d0 con 0x7f48a80d0010 2024-09-10T23:37:39.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48af200640 1 -- 172.21.15.94:0/1880152862 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a80c90d0 con 0x7f48a80d0010 2024-09-10T23:37:39.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48af200640 1 -- 172.21.15.94:0/1880152862 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a80c9690 con 0x7f48a80d0010 2024-09-10T23:37:39.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f48af200640 1 -- 172.21.15.94:0/1880152862 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a80ccfa0 con 0x7f48a80d0010 2024-09-10T23:37:39.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.137+0000 7f489fe00640 1 -- 172.21.15.94:0/1880152862 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 96547+0+0 (secure 0 0 0) 0x7f48a00b8070 con 0x7f48a80d0010 2024-09-10T23:37:39.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.141+0000 7f489fe00640 1 --2- 172.21.15.94:0/1880152862 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f48880727c0 0x7f4888074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:39.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.141+0000 7f489fe00640 1 -- 172.21.15.94:0/1880152862 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7f48a012fe30 con 0x7f48a80d0010 2024-09-10T23:37:39.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.141+0000 7f48a7e00640 1 --2- 172.21.15.94:0/1880152862 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f48880727c0 0x7f4888074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:39.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.141+0000 7f48a7e00640 1 --2- 172.21.15.94:0/1880152862 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f48880727c0 0x7f4888074c80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4898007fd0 tx=0x7f4898009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:39.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.145+0000 7f489fe00640 1 -- 172.21.15.94:0/1880152862 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f48a0018c30 con 0x7f48a80d0010 2024-09-10T23:37:39.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.337+0000 7f48af200640 1 -- 172.21.15.94:0/1880152862 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvol55860", "new_size": "10485760", "target": ["mon-mgr", ""]}) -- 0x7f48a80cea40 con 0x7f48880727c0 2024-09-10T23:37:39.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.353+0000 7f489fe00640 1 -- 172.21.15.94:0/1880152862 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+128 (secure 0 0 0) 0x7f48a80cea40 con 0x7f48880727c0 2024-09-10T23:37:39.357 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:37:39.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:39.358 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_used": 0 2024-09-10T23:37:39.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:39.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:39.358 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_quota": 10485760 2024-09-10T23:37:39.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:39.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:39.358 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_pcent": "0.00" 2024-09-10T23:37:39.358 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:39.358 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:37:39.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.357+0000 7f48af200640 1 -- 172.21.15.94:0/1880152862 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f48880727c0 msgr2=0x7f4888074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:39.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.357+0000 7f48af200640 1 --2- 172.21.15.94:0/1880152862 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f48880727c0 0x7f4888074c80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4898007fd0 tx=0x7f4898009040 comp rx=0 tx=0).stop 2024-09-10T23:37:39.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.357+0000 7f48af200640 1 -- 172.21.15.94:0/1880152862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48a80d0010 msgr2=0x7f48a815de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:39.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.357+0000 7f48af200640 1 --2- 172.21.15.94:0/1880152862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48a80d0010 0x7f48a815de90 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f48a00b3040 tx=0x7f48a00a8d00 comp rx=0 tx=0).stop 2024-09-10T23:37:39.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.357+0000 7f48af200640 1 -- 172.21.15.94:0/1880152862 shutdown_connections 2024-09-10T23:37:39.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.357+0000 7f48af200640 1 --2- 172.21.15.94:0/1880152862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48a80cfc30 0x7f48a81599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.357+0000 7f48af200640 1 --2- 172.21.15.94:0/1880152862 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f48880727c0 0x7f4888074c80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.357+0000 7f48af200640 1 --2- 172.21.15.94:0/1880152862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48a80d0010 0x7f48a815de90 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.357+0000 7f48af200640 1 --2- 172.21.15.94:0/1880152862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48a80b2890 0x7f48a8158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.357+0000 7f48af200640 1 -- 172.21.15.94:0/1880152862 >> 172.21.15.94:0/1880152862 conn(0x7f48a80d1bf0 msgr2=0x7f48a80b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:39.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.357+0000 7f48af200640 1 -- 172.21.15.94:0/1880152862 shutdown_connections 2024-09-10T23:37:39.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.357+0000 7f48af200640 1 -- 172.21.15.94:0/1880152862 wait complete. 2024-09-10T23:37:39.379 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol55860 2024-09-10T23:37:39.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8778a00640 1 -- 172.21.15.94:0/4245164257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87740af150 msgr2=0x7f87740af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:39.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8778a00640 1 --2- 172.21.15.94:0/4245164257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87740af150 0x7f87740af5e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f876c009c80 tx=0x7f876c01cd70 comp rx=0 tx=0).stop 2024-09-10T23:37:39.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8778a00640 1 -- 172.21.15.94:0/4245164257 shutdown_connections 2024-09-10T23:37:39.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8778a00640 1 --2- 172.21.15.94:0/4245164257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87740af150 0x7f87740af5e0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8778a00640 1 --2- 172.21.15.94:0/4245164257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f877415f140 0x7f87740b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8778a00640 1 -- 172.21.15.94:0/4245164257 >> 172.21.15.94:0/4245164257 conn(0x7f87740d1b00 msgr2=0x7f87740d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:39.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8778a00640 1 -- 172.21.15.94:0/4245164257 shutdown_connections 2024-09-10T23:37:39.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8778a00640 1 -- 172.21.15.94:0/4245164257 wait complete. 2024-09-10T23:37:39.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8778a00640 1 Processor -- start 2024-09-10T23:37:39.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8778a00640 1 -- start start 2024-09-10T23:37:39.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8778a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87740af150 0x7f8774148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:39.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8778a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87740b2ac0 0x7f877415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:39.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8778a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f877415f140 0x7f8774151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:39.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8778a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8774168ad0 con 0x7f87740af150 2024-09-10T23:37:39.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8778a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8774168950 con 0x7f87740b2ac0 2024-09-10T23:37:39.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8778a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8774168c50 con 0x7f877415f140 2024-09-10T23:37:39.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.521+0000 7f8772800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87740af150 0x7f8774148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:39.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f8772800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87740af150 0x7f8774148ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55004/0 (socket says 172.21.15.94:55004) 2024-09-10T23:37:39.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f8772800640 1 -- 172.21.15.94:0/293091279 learned_addr learned my addr 172.21.15.94:0/293091279 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:39.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f8773200640 1 --2- 172.21.15.94:0/293091279 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f877415f140 0x7f8774151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:39.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f8769e00640 1 --2- 172.21.15.94:0/293091279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87740b2ac0 0x7f877415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:39.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f8772800640 1 -- 172.21.15.94:0/293091279 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f877415f140 msgr2=0x7f8774151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:39.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f8772800640 1 --2- 172.21.15.94:0/293091279 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f877415f140 0x7f8774151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f8772800640 1 -- 172.21.15.94:0/293091279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87740b2ac0 msgr2=0x7f877415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:39.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f8772800640 1 --2- 172.21.15.94:0/293091279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87740b2ac0 0x7f877415ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f8772800640 1 -- 172.21.15.94:0/293091279 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87741492f0 con 0x7f87740af150 2024-09-10T23:37:39.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f8769e00640 1 --2- 172.21.15.94:0/293091279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87740b2ac0 0x7f877415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:39.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f8772800640 1 --2- 172.21.15.94:0/293091279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87740af150 0x7f8774148ab0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f876400d750 tx=0x7f876400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:39.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f876b400640 1 -- 172.21.15.94:0/293091279 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764017070 con 0x7f87740af150 2024-09-10T23:37:39.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f876b400640 1 -- 172.21.15.94:0/293091279 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8764009cc0 con 0x7f87740af150 2024-09-10T23:37:39.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f876b400640 1 -- 172.21.15.94:0/293091279 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764005280 con 0x7f87740af150 2024-09-10T23:37:39.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f8778a00640 1 -- 172.21.15.94:0/293091279 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87741522d0 con 0x7f87740af150 2024-09-10T23:37:39.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f8778a00640 1 -- 172.21.15.94:0/293091279 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f877415a630 con 0x7f87740af150 2024-09-10T23:37:39.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f8778a00640 1 -- 172.21.15.94:0/293091279 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87740af5e0 con 0x7f87740af150 2024-09-10T23:37:39.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.525+0000 7f876b400640 1 -- 172.21.15.94:0/293091279 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 96547+0+0 (secure 0 0 0) 0x7f8764005420 con 0x7f87740af150 2024-09-10T23:37:39.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.529+0000 7f876b400640 1 --2- 172.21.15.94:0/293091279 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8750072790 0x7f8750074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:39.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.529+0000 7f876b400640 1 -- 172.21.15.94:0/293091279 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7f87640936d0 con 0x7f87740af150 2024-09-10T23:37:39.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.529+0000 7f8769e00640 1 --2- 172.21.15.94:0/293091279 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8750072790 0x7f8750074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:39.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.529+0000 7f8769e00640 1 --2- 172.21.15.94:0/293091279 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8750072790 0x7f8750074c50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f877415c580 tx=0x7f875c009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:39.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.533+0000 7f876b400640 1 -- 172.21.15.94:0/293091279 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f876400de60 con 0x7f87740af150 2024-09-10T23:37:39.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.725+0000 7f8778a00640 1 -- 172.21.15.94:0/293091279 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol55860", "target": ["mon-mgr", ""]}) -- 0x7f877415e760 con 0x7f8750072790 2024-09-10T23:37:39.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.737+0000 7f876b400640 1 -- 172.21.15.94:0/293091279 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+679 (secure 0 0 0) 0x7f877415e760 con 0x7f8750072790 2024-09-10T23:37:39.741 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:37:39.741 INFO:teuthology.orchestra.run.smithi094.stdout: "atime": "2024-09-10 23:37:37", 2024-09-10T23:37:39.741 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_pcent": "0.00", 2024-09-10T23:37:39.741 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_quota": 10485760, 2024-09-10T23:37:39.741 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_used": 0, 2024-09-10T23:37:39.741 INFO:teuthology.orchestra.run.smithi094.stdout: "created_at": "2024-09-10 23:37:37", 2024-09-10T23:37:39.741 INFO:teuthology.orchestra.run.smithi094.stdout: "ctime": "2024-09-10 23:37:39", 2024-09-10T23:37:39.742 INFO:teuthology.orchestra.run.smithi094.stdout: "data_pool": "cephfs_data", 2024-09-10T23:37:39.742 INFO:teuthology.orchestra.run.smithi094.stdout: "features": [ 2024-09-10T23:37:39.742 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot-clone", 2024-09-10T23:37:39.742 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot-autoprotect" 2024-09-10T23:37:39.742 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:37:39.742 INFO:teuthology.orchestra.run.smithi094.stdout: "flavor": 1, 2024-09-10T23:37:39.742 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 0, 2024-09-10T23:37:39.742 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": 16877, 2024-09-10T23:37:39.742 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_addrs": [ 2024-09-10T23:37:39.742 INFO:teuthology.orchestra.run.smithi094.stdout: "172.21.15.94:6789", 2024-09-10T23:37:39.742 INFO:teuthology.orchestra.run.smithi094.stdout: "172.21.15.181:6789", 2024-09-10T23:37:39.742 INFO:teuthology.orchestra.run.smithi094.stdout: "172.21.15.181:6790" 2024-09-10T23:37:39.742 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:37:39.743 INFO:teuthology.orchestra.run.smithi094.stdout: "mtime": "2024-09-10 23:37:37", 2024-09-10T23:37:39.743 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad", 2024-09-10T23:37:39.743 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_namespace": "", 2024-09-10T23:37:39.743 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "complete", 2024-09-10T23:37:39.743 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "subvolume", 2024-09-10T23:37:39.743 INFO:teuthology.orchestra.run.smithi094.stdout: "uid": 0 2024-09-10T23:37:39.743 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:37:39.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.741+0000 7f8778a00640 1 -- 172.21.15.94:0/293091279 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8750072790 msgr2=0x7f8750074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:39.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.741+0000 7f8778a00640 1 --2- 172.21.15.94:0/293091279 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8750072790 0x7f8750074c50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f877415c580 tx=0x7f875c009040 comp rx=0 tx=0).stop 2024-09-10T23:37:39.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.741+0000 7f8778a00640 1 -- 172.21.15.94:0/293091279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87740af150 msgr2=0x7f8774148ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:39.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.741+0000 7f8778a00640 1 --2- 172.21.15.94:0/293091279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87740af150 0x7f8774148ab0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f876400d750 tx=0x7f876400dc10 comp rx=0 tx=0).stop 2024-09-10T23:37:39.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.741+0000 7f8778a00640 1 -- 172.21.15.94:0/293091279 shutdown_connections 2024-09-10T23:37:39.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.741+0000 7f8778a00640 1 --2- 172.21.15.94:0/293091279 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87740b2ac0 0x7f877415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.741+0000 7f8778a00640 1 --2- 172.21.15.94:0/293091279 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8750072790 0x7f8750074c50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.741+0000 7f8778a00640 1 --2- 172.21.15.94:0/293091279 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f877415f140 0x7f8774151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.741+0000 7f8778a00640 1 --2- 172.21.15.94:0/293091279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87740af150 0x7f8774148ab0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.741+0000 7f8778a00640 1 -- 172.21.15.94:0/293091279 >> 172.21.15.94:0/293091279 conn(0x7f87740d1b00 msgr2=0x7f87740cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:39.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.741+0000 7f8778a00640 1 -- 172.21.15.94:0/293091279 shutdown_connections 2024-09-10T23:37:39.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.741+0000 7f8778a00640 1 -- 172.21.15.94:0/293091279 wait complete. 2024-09-10T23:37:39.763 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol55860 --mode 777 --uid 1000 --gid 1000 2024-09-10T23:37:39.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd90c400640 1 -- 172.21.15.94:0/1921577799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd90415a5a0 msgr2=0x7fd904074080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:39.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd90c400640 1 --2- 172.21.15.94:0/1921577799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd90415a5a0 0x7fd904074080 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd9040749b0 tx=0x7fd8f801c6b0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd90c400640 1 -- 172.21.15.94:0/1921577799 shutdown_connections 2024-09-10T23:37:39.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd90c400640 1 --2- 172.21.15.94:0/1921577799 >> v2:172.21.15.181:3300/0 conn(0x7fd9040ca310 0x7fd9040ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd90c400640 1 --2- 172.21.15.94:0/1921577799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd90415a5a0 0x7fd904074080 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd90c400640 1 -- 172.21.15.94:0/1921577799 >> 172.21.15.94:0/1921577799 conn(0x7fd9040cfa20 msgr2=0x7fd9040cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:39.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd90c400640 1 -- 172.21.15.94:0/1921577799 shutdown_connections 2024-09-10T23:37:39.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd90c400640 1 -- 172.21.15.94:0/1921577799 wait complete. 2024-09-10T23:37:39.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd90c400640 1 Processor -- start 2024-09-10T23:37:39.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd90c400640 1 -- start start 2024-09-10T23:37:39.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd90c400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9040ca310 0x7fd9041fb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:39.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd90c400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd90415a5a0 0x7fd904158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:39.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd90c400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd90415a980 0x7fd9041541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:39.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd90c400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd9041675a0 con 0x7fd9040ca310 2024-09-10T23:37:39.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd90c400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd9040ce1b0 con 0x7fd90415a980 2024-09-10T23:37:39.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd90c400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd904167720 con 0x7fd90415a5a0 2024-09-10T23:37:39.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd909a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9040ca310 0x7fd9041fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:39.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd909a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9040ca310 0x7fd9041fb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55026/0 (socket says 172.21.15.94:55026) 2024-09-10T23:37:39.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd909a00640 1 -- 172.21.15.94:0/253648345 learned_addr learned my addr 172.21.15.94:0/253648345 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:39.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.913+0000 7fd909000640 1 --2- 172.21.15.94:0/253648345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd90415a5a0 0x7fd904158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:39.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd90a400640 1 --2- 172.21.15.94:0/253648345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd90415a980 0x7fd9041541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:39.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd909a00640 1 -- 172.21.15.94:0/253648345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd90415a5a0 msgr2=0x7fd904158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:39.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd909a00640 1 --2- 172.21.15.94:0/253648345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd90415a5a0 0x7fd904158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd909a00640 1 -- 172.21.15.94:0/253648345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd90415a980 msgr2=0x7fd9041541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:39.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd909a00640 1 --2- 172.21.15.94:0/253648345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd90415a980 0x7fd9041541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:39.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd909a00640 1 -- 172.21.15.94:0/253648345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9041f78c0 con 0x7fd9040ca310 2024-09-10T23:37:39.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd909000640 1 --2- 172.21.15.94:0/253648345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd90415a5a0 0x7fd904158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:39.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd90a400640 1 --2- 172.21.15.94:0/253648345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd90415a980 0x7fd9041541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:39.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd909a00640 1 --2- 172.21.15.94:0/253648345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9040ca310 0x7fd9041fb980 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80b0040 tx=0x7fd8f80a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:39.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd901600640 1 -- 172.21.15.94:0/253648345 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f80b2040 con 0x7fd9040ca310 2024-09-10T23:37:39.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd901600640 1 -- 172.21.15.94:0/253648345 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8f80043e0 con 0x7fd9040ca310 2024-09-10T23:37:39.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd90c400640 1 -- 172.21.15.94:0/253648345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9041f7b30 con 0x7fd9040ca310 2024-09-10T23:37:39.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd901600640 1 -- 172.21.15.94:0/253648345 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8005280 con 0x7fd9040ca310 2024-09-10T23:37:39.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd90c400640 1 -- 172.21.15.94:0/253648345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9041f80f0 con 0x7fd9040ca310 2024-09-10T23:37:39.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd90c400640 1 -- 172.21.15.94:0/253648345 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9040ca720 con 0x7fd9040ca310 2024-09-10T23:37:39.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd901600640 1 -- 172.21.15.94:0/253648345 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 96547+0+0 (secure 0 0 0) 0x7fd8f8005420 con 0x7fd9040ca310 2024-09-10T23:37:39.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.917+0000 7fd901600640 1 --2- 172.21.15.94:0/253648345 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd8e4072790 0x7fd8e4074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:39.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.921+0000 7fd901600640 1 -- 172.21.15.94:0/253648345 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7fd8f812d410 con 0x7fd9040ca310 2024-09-10T23:37:39.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.921+0000 7fd909000640 1 --2- 172.21.15.94:0/253648345 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd8e4072790 0x7fd8e4074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:39.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.921+0000 7fd909000640 1 --2- 172.21.15.94:0/253648345 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd8e4072790 0x7fd8e4074c50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd904159c50 tx=0x7fd8f0008040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:39.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:39.921+0000 7fd901600640 1 -- 172.21.15.94:0/253648345 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd9040cac60 con 0x7fd9040ca310 2024-09-10T23:37:40.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.109+0000 7fd90c400640 1 -- 172.21.15.94:0/253648345 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol55860", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) -- 0x7fd9040d0720 con 0x7fd8e4072790 2024-09-10T23:37:40.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.125+0000 7fd901600640 1 -- 172.21.15.94:0/253648345 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd9040d0720 con 0x7fd8e4072790 2024-09-10T23:37:40.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.129+0000 7fd90c400640 1 -- 172.21.15.94:0/253648345 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd8e4072790 msgr2=0x7fd8e4074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:40.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.129+0000 7fd90c400640 1 --2- 172.21.15.94:0/253648345 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd8e4072790 0x7fd8e4074c50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd904159c50 tx=0x7fd8f0008040 comp rx=0 tx=0).stop 2024-09-10T23:37:40.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.129+0000 7fd90c400640 1 -- 172.21.15.94:0/253648345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9040ca310 msgr2=0x7fd9041fb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:40.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.129+0000 7fd90c400640 1 --2- 172.21.15.94:0/253648345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9040ca310 0x7fd9041fb980 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80b0040 tx=0x7fd8f80a5690 comp rx=0 tx=0).stop 2024-09-10T23:37:40.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.129+0000 7fd90c400640 1 -- 172.21.15.94:0/253648345 shutdown_connections 2024-09-10T23:37:40.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.129+0000 7fd90c400640 1 --2- 172.21.15.94:0/253648345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd90415a980 0x7fd9041541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:40.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.129+0000 7fd90c400640 1 --2- 172.21.15.94:0/253648345 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd8e4072790 0x7fd8e4074c50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:40.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.129+0000 7fd90c400640 1 --2- 172.21.15.94:0/253648345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd90415a5a0 0x7fd904158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:40.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.129+0000 7fd90c400640 1 --2- 172.21.15.94:0/253648345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9040ca310 0x7fd9041fb980 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:40.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.129+0000 7fd90c400640 1 -- 172.21.15.94:0/253648345 >> 172.21.15.94:0/253648345 conn(0x7fd9040cfa20 msgr2=0x7fd9041642e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:40.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.129+0000 7fd90c400640 1 -- 172.21.15.94:0/253648345 shutdown_connections 2024-09-10T23:37:40.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.129+0000 7fd90c400640 1 -- 172.21.15.94:0/253648345 wait complete. 2024-09-10T23:37:40.150 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol55860 2024-09-10T23:37:40.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.293+0000 7f72d4a00640 1 -- 172.21.15.94:0/4117956140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d00affd0 msgr2=0x7f72d00b0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:40.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.293+0000 7f72d4a00640 1 --2- 172.21.15.94:0/4117956140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d00affd0 0x7f72d00b0500 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f72d0068b10 tx=0x7f72b801c7c0 comp rx=0 tx=0).stop 2024-09-10T23:37:40.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.293+0000 7f72d4a00640 1 -- 172.21.15.94:0/4117956140 shutdown_connections 2024-09-10T23:37:40.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.293+0000 7f72d4a00640 1 --2- 172.21.15.94:0/4117956140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72d00b1180 0x7f72d00ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:40.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.293+0000 7f72d4a00640 1 --2- 172.21.15.94:0/4117956140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d00affd0 0x7f72d00b0500 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:40.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.293+0000 7f72d4a00640 1 -- 172.21.15.94:0/4117956140 >> 172.21.15.94:0/4117956140 conn(0x7f72d0074240 msgr2=0x7f72d0074620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:40.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.293+0000 7f72d4a00640 1 -- 172.21.15.94:0/4117956140 shutdown_connections 2024-09-10T23:37:40.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.293+0000 7f72d4a00640 1 -- 172.21.15.94:0/4117956140 wait complete. 2024-09-10T23:37:40.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.293+0000 7f72d4a00640 1 Processor -- start 2024-09-10T23:37:40.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.293+0000 7f72d4a00640 1 -- start start 2024-09-10T23:37:40.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72d4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d00b1180 0x7f72d00b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:40.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72d4a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d015bf80 0x7f72d0156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:40.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72d4a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72d0156b60 0x7f72d00ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:40.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72d4a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0168e40 con 0x7f72d00b1180 2024-09-10T23:37:40.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72d4a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0168cc0 con 0x7f72d015bf80 2024-09-10T23:37:40.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72d4a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f72d0168fc0 con 0x7f72d0156b60 2024-09-10T23:37:40.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72cde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d00b1180 0x7f72d00b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:40.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72cde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d00b1180 0x7f72d00b5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55028/0 (socket says 172.21.15.94:55028) 2024-09-10T23:37:40.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72cde00640 1 -- 172.21.15.94:0/1466515284 learned_addr learned my addr 172.21.15.94:0/1466515284 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:40.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72c5400640 1 --2- 172.21.15.94:0/1466515284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d015bf80 0x7f72d0156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:40.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72ce800640 1 --2- 172.21.15.94:0/1466515284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72d0156b60 0x7f72d00ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:40.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72cde00640 1 -- 172.21.15.94:0/1466515284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72d0156b60 msgr2=0x7f72d00ae2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:40.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72cde00640 1 --2- 172.21.15.94:0/1466515284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72d0156b60 0x7f72d00ae2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:40.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72cde00640 1 -- 172.21.15.94:0/1466515284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d015bf80 msgr2=0x7f72d0156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:40.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72cde00640 1 --2- 172.21.15.94:0/1466515284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d015bf80 0x7f72d0156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:40.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72cde00640 1 -- 172.21.15.94:0/1466515284 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d01fb2f0 con 0x7f72d00b1180 2024-09-10T23:37:40.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72cde00640 1 --2- 172.21.15.94:0/1466515284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d00b1180 0x7f72d00b5880 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f72c000cbc0 tx=0x7f72c000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:40.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72c6a00640 1 -- 172.21.15.94:0/1466515284 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0016020 con 0x7f72d00b1180 2024-09-10T23:37:40.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72c6a00640 1 -- 172.21.15.94:0/1466515284 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72c000ab40 con 0x7f72d00b1180 2024-09-10T23:37:40.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72d4a00640 1 -- 172.21.15.94:0/1466515284 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d01fb590 con 0x7f72d00b1180 2024-09-10T23:37:40.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72c6a00640 1 -- 172.21.15.94:0/1466515284 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c000ce40 con 0x7f72d00b1180 2024-09-10T23:37:40.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72d4a00640 1 -- 172.21.15.94:0/1466515284 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d01fbb50 con 0x7f72d00b1180 2024-09-10T23:37:40.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72d4a00640 1 -- 172.21.15.94:0/1466515284 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d00ad060 con 0x7f72d00b1180 2024-09-10T23:37:40.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.297+0000 7f72c6a00640 1 -- 172.21.15.94:0/1466515284 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 96547+0+0 (secure 0 0 0) 0x7f72c001a3b0 con 0x7f72d00b1180 2024-09-10T23:37:40.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.301+0000 7f72c6a00640 1 --2- 172.21.15.94:0/1466515284 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f72ac072790 0x7f72ac074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:40.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.301+0000 7f72c6a00640 1 -- 172.21.15.94:0/1466515284 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7f72c0093830 con 0x7f72d00b1180 2024-09-10T23:37:40.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.301+0000 7f72c5400640 1 --2- 172.21.15.94:0/1466515284 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f72ac072790 0x7f72ac074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:40.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.301+0000 7f72c5400640 1 --2- 172.21.15.94:0/1466515284 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f72ac072790 0x7f72ac074c50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f72b80af000 tx=0x7f72b801b060 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:40.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.305+0000 7f72c6a00640 1 -- 172.21.15.94:0/1466515284 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f72c0020030 con 0x7f72d00b1180 2024-09-10T23:37:40.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.497+0000 7f72d4a00640 1 -- 172.21.15.94:0/1466515284 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol55860", "target": ["mon-mgr", ""]}) -- 0x7f72d015abe0 con 0x7f72ac072790 2024-09-10T23:37:40.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.521+0000 7f72c6a00640 1 -- 172.21.15.94:0/1466515284 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f72d015abe0 con 0x7f72ac072790 2024-09-10T23:37:40.522 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad 2024-09-10T23:37:40.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.521+0000 7f72d4a00640 1 -- 172.21.15.94:0/1466515284 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f72ac072790 msgr2=0x7f72ac074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.521+0000 7f72d4a00640 1 --2- 172.21.15.94:0/1466515284 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f72ac072790 0x7f72ac074c50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f72b80af000 tx=0x7f72b801b060 comp rx=0 tx=0).stop 2024-09-10T23:37:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.521+0000 7f72d4a00640 1 -- 172.21.15.94:0/1466515284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d00b1180 msgr2=0x7f72d00b5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.521+0000 7f72d4a00640 1 --2- 172.21.15.94:0/1466515284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d00b1180 0x7f72d00b5880 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f72c000cbc0 tx=0x7f72c000cbf0 comp rx=0 tx=0).stop 2024-09-10T23:37:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.521+0000 7f72d4a00640 1 -- 172.21.15.94:0/1466515284 shutdown_connections 2024-09-10T23:37:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.521+0000 7f72d4a00640 1 --2- 172.21.15.94:0/1466515284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d015bf80 0x7f72d0156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.521+0000 7f72d4a00640 1 --2- 172.21.15.94:0/1466515284 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f72ac072790 0x7f72ac074c50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.521+0000 7f72d4a00640 1 --2- 172.21.15.94:0/1466515284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72d0156b60 0x7f72d00ae2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.521+0000 7f72d4a00640 1 --2- 172.21.15.94:0/1466515284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d00b1180 0x7f72d00b5880 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.521+0000 7f72d4a00640 1 -- 172.21.15.94:0/1466515284 >> 172.21.15.94:0/1466515284 conn(0x7f72d0074240 msgr2=0x7f72d006fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.521+0000 7f72d4a00640 1 -- 172.21.15.94:0/1466515284 shutdown_connections 2024-09-10T23:37:40.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:40.521+0000 7f72d4a00640 1 -- 172.21.15.94:0/1466515284 wait complete. 2024-09-10T23:37:40.543 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol55860 with 8 files each 1MB size under directory volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad 2024-09-10T23:37:40.544 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:37:40.544 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:40.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:40.553 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T23:37:40.628 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T23:37:40.628 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T23:37:40.628 INFO:teuthology.orchestra.run.smithi094.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191265 s, 54.8 MB/s 2024-09-10T23:37:40.629 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:37:40.629 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:40.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:40.681 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T23:37:40.761 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T23:37:40.761 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T23:37:40.761 INFO:teuthology.orchestra.run.smithi094.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206484 s, 50.8 MB/s 2024-09-10T23:37:40.762 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:37:40.762 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:40.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:40.814 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T23:37:40.890 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T23:37:40.890 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T23:37:40.890 INFO:teuthology.orchestra.run.smithi094.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173193 s, 60.5 MB/s 2024-09-10T23:37:40.891 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:37:40.891 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:40.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:40.941 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T23:37:41.017 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T23:37:41.017 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T23:37:41.018 INFO:teuthology.orchestra.run.smithi094.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016689 s, 62.8 MB/s 2024-09-10T23:37:41.019 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:37:41.019 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:41.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:41.070 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T23:37:41.145 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T23:37:41.145 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T23:37:41.145 INFO:teuthology.orchestra.run.smithi094.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164841 s, 63.6 MB/s 2024-09-10T23:37:41.146 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:37:41.147 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:41.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:41.202 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T23:37:41.275 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T23:37:41.276 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T23:37:41.276 INFO:teuthology.orchestra.run.smithi094.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146473 s, 71.6 MB/s 2024-09-10T23:37:41.277 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:37:41.277 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:41.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:41.330 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T23:37:41.407 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T23:37:41.407 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T23:37:41.407 INFO:teuthology.orchestra.run.smithi094.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187118 s, 56.0 MB/s 2024-09-10T23:37:41.408 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:37:41.408 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:37:41.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:41.457 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T23:37:41.531 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T23:37:41.531 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T23:37:41.531 INFO:teuthology.orchestra.run.smithi094.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148673 s, 70.5 MB/s 2024-09-10T23:37:41.532 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol55860 subvol_snap51870 2024-09-10T23:37:41.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.721+0000 7faf5f200640 1 -- 172.21.15.94:0/278424487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf580b4c40 msgr2=0x7faf580cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:41.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.721+0000 7faf5f200640 1 --2- 172.21.15.94:0/278424487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf580b4c40 0x7faf580cfcb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faf44009840 tx=0x7faf4401c790 comp rx=0 tx=0).stop 2024-09-10T23:37:41.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.721+0000 7faf5f200640 1 -- 172.21.15.94:0/278424487 shutdown_connections 2024-09-10T23:37:41.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.721+0000 7faf5f200640 1 --2- 172.21.15.94:0/278424487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf580b4c40 0x7faf580cfcb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:41.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.721+0000 7faf5f200640 1 --2- 172.21.15.94:0/278424487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf5815ba50 0x7faf580b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:41.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.721+0000 7faf5f200640 1 -- 172.21.15.94:0/278424487 >> 172.21.15.94:0/278424487 conn(0x7faf5806cdf0 msgr2=0x7faf5806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:41.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.721+0000 7faf5f200640 1 -- 172.21.15.94:0/278424487 shutdown_connections 2024-09-10T23:37:41.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.721+0000 7faf5f200640 1 -- 172.21.15.94:0/278424487 wait complete. 2024-09-10T23:37:41.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf5f200640 1 Processor -- start 2024-09-10T23:37:41.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf5f200640 1 -- start start 2024-09-10T23:37:41.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf5f200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf580b4c40 0x7faf5813e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:41.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf5f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf580d01f0 0x7faf5813f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:41.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf5f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf5815ba50 0x7faf5815d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:41.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf5f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf58168be0 con 0x7faf580d01f0 2024-09-10T23:37:41.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf5f200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faf58168a60 con 0x7faf5815ba50 2024-09-10T23:37:41.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf5f200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7faf58168d60 con 0x7faf580b4c40 2024-09-10T23:37:41.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf57400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf580d01f0 0x7faf5813f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:41.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf57400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf580d01f0 0x7faf5813f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55042/0 (socket says 172.21.15.94:55042) 2024-09-10T23:37:41.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf57400640 1 -- 172.21.15.94:0/1493384177 learned_addr learned my addr 172.21.15.94:0/1493384177 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:41.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf5d200640 1 --2- 172.21.15.94:0/1493384177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf5815ba50 0x7faf5815d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:41.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf57e00640 1 --2- 172.21.15.94:0/1493384177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf580b4c40 0x7faf5813e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:41.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf57400640 1 -- 172.21.15.94:0/1493384177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf580b4c40 msgr2=0x7faf5813e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:41.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf57400640 1 --2- 172.21.15.94:0/1493384177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf580b4c40 0x7faf5813e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:41.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf57400640 1 -- 172.21.15.94:0/1493384177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf5815ba50 msgr2=0x7faf5815d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:41.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf57400640 1 --2- 172.21.15.94:0/1493384177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf5815ba50 0x7faf5815d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:41.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf57400640 1 -- 172.21.15.94:0/1493384177 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5815c700 con 0x7faf580d01f0 2024-09-10T23:37:41.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf57400640 1 --2- 172.21.15.94:0/1493384177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf580d01f0 0x7faf5813f350 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7faf44000c10 tx=0x7faf440a64f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:41.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf4fe00640 1 -- 172.21.15.94:0/1493384177 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf440ac070 con 0x7faf580d01f0 2024-09-10T23:37:41.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf4fe00640 1 -- 172.21.15.94:0/1493384177 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf44004310 con 0x7faf580d01f0 2024-09-10T23:37:41.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf4fe00640 1 -- 172.21.15.94:0/1493384177 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf440052e0 con 0x7faf580d01f0 2024-09-10T23:37:41.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf5f200640 1 -- 172.21.15.94:0/1493384177 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5815c9e0 con 0x7faf580d01f0 2024-09-10T23:37:41.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.725+0000 7faf5f200640 1 -- 172.21.15.94:0/1493384177 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf5815cfa0 con 0x7faf580d01f0 2024-09-10T23:37:41.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.729+0000 7faf5f200640 1 -- 172.21.15.94:0/1493384177 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf580cfcb0 con 0x7faf580d01f0 2024-09-10T23:37:41.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.729+0000 7faf4fe00640 1 -- 172.21.15.94:0/1493384177 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 96547+0+0 (secure 0 0 0) 0x7faf44005480 con 0x7faf580d01f0 2024-09-10T23:37:41.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.729+0000 7faf4fe00640 1 --2- 172.21.15.94:0/1493384177 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faf380727c0 0x7faf38074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:41.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.729+0000 7faf4fe00640 1 -- 172.21.15.94:0/1493384177 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7faf4412d9e0 con 0x7faf580d01f0 2024-09-10T23:37:41.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.729+0000 7faf57e00640 1 --2- 172.21.15.94:0/1493384177 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faf380727c0 0x7faf38074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:41.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.729+0000 7faf57e00640 1 --2- 172.21.15.94:0/1493384177 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faf380727c0 0x7faf38074c80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faf48009600 tx=0x7faf480091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:41.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.733+0000 7faf4fe00640 1 -- 172.21.15.94:0/1493384177 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faf440b6070 con 0x7faf580d01f0 2024-09-10T23:37:41.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.929+0000 7faf5f200640 1 -- 172.21.15.94:0/1493384177 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol55860", "snap_name": "subvol_snap51870", "target": ["mon-mgr", ""]}) -- 0x7faf58156d90 con 0x7faf380727c0 2024-09-10T23:37:41.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.957+0000 7faf4fe00640 1 -- 172.21.15.94:0/1493384177 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faf58156d90 con 0x7faf380727c0 2024-09-10T23:37:41.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.957+0000 7faf5f200640 1 -- 172.21.15.94:0/1493384177 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faf380727c0 msgr2=0x7faf38074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:41.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.957+0000 7faf5f200640 1 --2- 172.21.15.94:0/1493384177 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faf380727c0 0x7faf38074c80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faf48009600 tx=0x7faf480091e0 comp rx=0 tx=0).stop 2024-09-10T23:37:41.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.957+0000 7faf5f200640 1 -- 172.21.15.94:0/1493384177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf580d01f0 msgr2=0x7faf5813f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:41.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.957+0000 7faf5f200640 1 --2- 172.21.15.94:0/1493384177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf580d01f0 0x7faf5813f350 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7faf44000c10 tx=0x7faf440a64f0 comp rx=0 tx=0).stop 2024-09-10T23:37:41.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.957+0000 7faf5f200640 1 -- 172.21.15.94:0/1493384177 shutdown_connections 2024-09-10T23:37:41.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.957+0000 7faf5f200640 1 --2- 172.21.15.94:0/1493384177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf5815ba50 0x7faf5815d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:41.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.957+0000 7faf5f200640 1 --2- 172.21.15.94:0/1493384177 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faf380727c0 0x7faf38074c80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:41.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.957+0000 7faf5f200640 1 --2- 172.21.15.94:0/1493384177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf580d01f0 0x7faf5813f350 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:41.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.957+0000 7faf5f200640 1 --2- 172.21.15.94:0/1493384177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf580b4c40 0x7faf5813e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:41.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.957+0000 7faf5f200640 1 -- 172.21.15.94:0/1493384177 >> 172.21.15.94:0/1493384177 conn(0x7faf5806cdf0 msgr2=0x7faf5815f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:41.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.957+0000 7faf5f200640 1 -- 172.21.15.94:0/1493384177 shutdown_connections 2024-09-10T23:37:41.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:41.957+0000 7faf5f200640 1 -- 172.21.15.94:0/1493384177 wait complete. 2024-09-10T23:37:41.981 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol55860 subvol_snap51870 subvol_clone39050 2024-09-10T23:37:42.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.125+0000 7f5f73e00640 1 -- 172.21.15.94:0/487050225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f7415bcc0 msgr2=0x7f5f740cff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:42.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.125+0000 7f5f73e00640 1 --2- 172.21.15.94:0/487050225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f7415bcc0 0x7f5f740cff00 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009900 tx=0x7f5f6401c8e0 comp rx=0 tx=0).stop 2024-09-10T23:37:42.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.125+0000 7f5f73e00640 1 -- 172.21.15.94:0/487050225 shutdown_connections 2024-09-10T23:37:42.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.125+0000 7f5f73e00640 1 --2- 172.21.15.94:0/487050225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f7415bcc0 0x7f5f740cff00 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:42.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.125+0000 7f5f73e00640 1 -- 172.21.15.94:0/487050225 >> 172.21.15.94:0/487050225 conn(0x7f5f740d1b50 msgr2=0x7f5f740d1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:42.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73e00640 1 -- 172.21.15.94:0/487050225 shutdown_connections 2024-09-10T23:37:42.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73e00640 1 -- 172.21.15.94:0/487050225 wait complete. 2024-09-10T23:37:42.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73e00640 1 Processor -- start 2024-09-10T23:37:42.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73e00640 1 -- start start 2024-09-10T23:37:42.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f74071610 0x7f5f740b6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:42.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f740d2670 0x7f5f7414f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:42.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f7415bcc0 0x7f5f740b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:42.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7416a5b0 con 0x7f5f7415bcc0 2024-09-10T23:37:42.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7416a430 con 0x7f5f74071610 2024-09-10T23:37:42.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5f7416a730 con 0x7f5f740d2670 2024-09-10T23:37:42.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f7415bcc0 0x7f5f740b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:42.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f7415bcc0 0x7f5f740b59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55056/0 (socket says 172.21.15.94:55056) 2024-09-10T23:37:42.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73400640 1 -- 172.21.15.94:0/1449718501 learned_addr learned my addr 172.21.15.94:0/1449718501 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:42.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f72a00640 1 --2- 172.21.15.94:0/1449718501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f74071610 0x7f5f740b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:42.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f6be00640 1 --2- 172.21.15.94:0/1449718501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f740d2670 0x7f5f7414f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:42.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73400640 1 -- 172.21.15.94:0/1449718501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f740d2670 msgr2=0x7f5f7414f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:42.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73400640 1 --2- 172.21.15.94:0/1449718501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f740d2670 0x7f5f7414f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:42.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73400640 1 -- 172.21.15.94:0/1449718501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f74071610 msgr2=0x7f5f740b6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:42.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73400640 1 --2- 172.21.15.94:0/1449718501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f74071610 0x7f5f740b6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:42.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73400640 1 -- 172.21.15.94:0/1449718501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f740cad30 con 0x7f5f7415bcc0 2024-09-10T23:37:42.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73400640 1 --2- 172.21.15.94:0/1449718501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f7415bcc0 0x7f5f740b59f0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00c7c0 tx=0x7f5f6c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f6aa00640 1 -- 172.21.15.94:0/1449718501 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c003b80 con 0x7f5f7415bcc0 2024-09-10T23:37:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f6aa00640 1 -- 172.21.15.94:0/1449718501 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f6c003d20 con 0x7f5f7415bcc0 2024-09-10T23:37:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f73e00640 1 -- 172.21.15.94:0/1449718501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f740cafd0 con 0x7f5f7415bcc0 2024-09-10T23:37:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.129+0000 7f5f6aa00640 1 -- 172.21.15.94:0/1449718501 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c00aab0 con 0x7f5f7415bcc0 2024-09-10T23:37:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.133+0000 7f5f73e00640 1 -- 172.21.15.94:0/1449718501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f740cb590 con 0x7f5f7415bcc0 2024-09-10T23:37:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.133+0000 7f5f73e00640 1 -- 172.21.15.94:0/1449718501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f740cff00 con 0x7f5f7415bcc0 2024-09-10T23:37:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.133+0000 7f5f6aa00640 1 -- 172.21.15.94:0/1449718501 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 96547+0+0 (secure 0 0 0) 0x7f5f6c00ac50 con 0x7f5f7415bcc0 2024-09-10T23:37:42.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.133+0000 7f5f6aa00640 1 --2- 172.21.15.94:0/1449718501 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f50072790 0x7f5f50074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:42.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.133+0000 7f5f6aa00640 1 -- 172.21.15.94:0/1449718501 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7f5f6c093470 con 0x7f5f7415bcc0 2024-09-10T23:37:42.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.133+0000 7f5f72a00640 1 --2- 172.21.15.94:0/1449718501 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f50072790 0x7f5f50074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:42.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.133+0000 7f5f72a00640 1 --2- 172.21.15.94:0/1449718501 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f50072790 0x7f5f50074c50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5f640af000 tx=0x7f5f6401b120 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:42.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.137+0000 7f5f6aa00640 1 -- 172.21.15.94:0/1449718501 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5f6c05e100 con 0x7f5f7415bcc0 2024-09-10T23:37:42.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.337+0000 7f5f73e00640 1 -- 172.21.15.94:0/1449718501 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol55860", "snap_name": "subvol_snap51870", "target_sub_name": "subvol_clone39050", "target": ["mon-mgr", ""]}) -- 0x7f5f7406f9c0 con 0x7f5f50072790 2024-09-10T23:37:42.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.429+0000 7f5f6aa00640 1 -- 172.21.15.94:0/1449718501 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5f7406f9c0 con 0x7f5f50072790 2024-09-10T23:37:42.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.429+0000 7f5f73e00640 1 -- 172.21.15.94:0/1449718501 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f50072790 msgr2=0x7f5f50074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:42.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.429+0000 7f5f73e00640 1 --2- 172.21.15.94:0/1449718501 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f50072790 0x7f5f50074c50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5f640af000 tx=0x7f5f6401b120 comp rx=0 tx=0).stop 2024-09-10T23:37:42.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.429+0000 7f5f73e00640 1 -- 172.21.15.94:0/1449718501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f7415bcc0 msgr2=0x7f5f740b59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:42.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.429+0000 7f5f73e00640 1 --2- 172.21.15.94:0/1449718501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f7415bcc0 0x7f5f740b59f0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00c7c0 tx=0x7f5f6c00cc80 comp rx=0 tx=0).stop 2024-09-10T23:37:42.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.433+0000 7f5f73e00640 1 -- 172.21.15.94:0/1449718501 shutdown_connections 2024-09-10T23:37:42.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.433+0000 7f5f73e00640 1 --2- 172.21.15.94:0/1449718501 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f50072790 0x7f5f50074c50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:42.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.433+0000 7f5f73e00640 1 --2- 172.21.15.94:0/1449718501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f7415bcc0 0x7f5f740b59f0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:42.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.433+0000 7f5f73e00640 1 --2- 172.21.15.94:0/1449718501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f740d2670 0x7f5f7414f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:42.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.433+0000 7f5f73e00640 1 --2- 172.21.15.94:0/1449718501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f74071610 0x7f5f740b6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:42.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.433+0000 7f5f73e00640 1 -- 172.21.15.94:0/1449718501 >> 172.21.15.94:0/1449718501 conn(0x7f5f740d1b50 msgr2=0x7f5f740ce560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:42.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.433+0000 7f5f73e00640 1 -- 172.21.15.94:0/1449718501 shutdown_connections 2024-09-10T23:37:42.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.433+0000 7f5f73e00640 1 -- 172.21.15.94:0/1449718501 wait complete. 2024-09-10T23:37:42.436 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:42.433+0000 7f9ea7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:42.436 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:42.433+0000 7f9ea7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:42.436 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:42.433+0000 7f9ea7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:42.436 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:42.433+0000 7f9ea7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:42.436 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:42.433+0000 7f9ea7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:42.455 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:42.453+0000 7f9ea6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:42.455 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:42.453+0000 7f9ea6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:42.455 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:42.453+0000 7f9ea6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:42.455 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:42.453+0000 7f9ea6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:42.455 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:42.453+0000 7f9ea6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:42.457 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone39050 2024-09-10T23:37:42.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.621+0000 7f5f34400640 1 -- 172.21.15.94:0/3477906347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c1455d0 msgr2=0x7f5f2c0c6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:42.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.621+0000 7f5f34400640 1 --2- 172.21.15.94:0/3477906347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c1455d0 0x7f5f2c0c6460 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c06e760 tx=0x7f5f2001c820 comp rx=0 tx=0).stop 2024-09-10T23:37:42.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.621+0000 7f5f34400640 1 -- 172.21.15.94:0/3477906347 shutdown_connections 2024-09-10T23:37:42.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.621+0000 7f5f34400640 1 --2- 172.21.15.94:0/3477906347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c1455d0 0x7f5f2c0c6460 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:42.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.621+0000 7f5f34400640 1 -- 172.21.15.94:0/3477906347 >> 172.21.15.94:0/3477906347 conn(0x7f5f2c0cfaf0 msgr2=0x7f5f2c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:42.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.621+0000 7f5f34400640 1 -- 172.21.15.94:0/3477906347 shutdown_connections 2024-09-10T23:37:42.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.621+0000 7f5f34400640 1 -- 172.21.15.94:0/3477906347 wait complete. 2024-09-10T23:37:42.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f34400640 1 Processor -- start 2024-09-10T23:37:42.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f34400640 1 -- start start 2024-09-10T23:37:42.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f34400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0cd9f0 0x7f5f2c160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:42.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f34400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c0d0710 0x7f5f2c1fb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:42.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f34400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f2c1455d0 0x7f5f2c073070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:42.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f34400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c16a310 con 0x7f5f2c0cd9f0 2024-09-10T23:37:42.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f34400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c16a190 con 0x7f5f2c0d0710 2024-09-10T23:37:42.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f34400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5f2c16a490 con 0x7f5f2c1455d0 2024-09-10T23:37:42.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f31a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0cd9f0 0x7f5f2c160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:42.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f31000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c0d0710 0x7f5f2c1fb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:42.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f31a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0cd9f0 0x7f5f2c160060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55112/0 (socket says 172.21.15.94:55112) 2024-09-10T23:37:42.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f31a00640 1 -- 172.21.15.94:0/1716700319 learned_addr learned my addr 172.21.15.94:0/1716700319 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:42.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f32400640 1 --2- 172.21.15.94:0/1716700319 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f2c1455d0 0x7f5f2c073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:42.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f31000640 1 -- 172.21.15.94:0/1716700319 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f2c1455d0 msgr2=0x7f5f2c073070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:42.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f31000640 1 --2- 172.21.15.94:0/1716700319 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f2c1455d0 0x7f5f2c073070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:42.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f31000640 1 -- 172.21.15.94:0/1716700319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0cd9f0 msgr2=0x7f5f2c160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:42.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f31000640 1 --2- 172.21.15.94:0/1716700319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0cd9f0 0x7f5f2c160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:42.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f31000640 1 -- 172.21.15.94:0/1716700319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2c072600 con 0x7f5f2c0d0710 2024-09-10T23:37:42.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f31a00640 1 --2- 172.21.15.94:0/1716700319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0cd9f0 0x7f5f2c160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:37:42.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f31000640 1 --2- 172.21.15.94:0/1716700319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c0d0710 0x7f5f2c1fb9e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800c400 tx=0x7f5f1800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:42.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f29600640 1 -- 172.21.15.94:0/1716700319 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18017070 con 0x7f5f2c0d0710 2024-09-10T23:37:42.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f29600640 1 -- 172.21.15.94:0/1716700319 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f180053c0 con 0x7f5f2c0d0710 2024-09-10T23:37:42.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f29600640 1 -- 172.21.15.94:0/1716700319 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18003860 con 0x7f5f2c0d0710 2024-09-10T23:37:42.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f34400640 1 -- 172.21.15.94:0/1716700319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2c074400 con 0x7f5f2c0d0710 2024-09-10T23:37:42.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.625+0000 7f5f34400640 1 -- 172.21.15.94:0/1716700319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2c072b50 con 0x7f5f2c0d0710 2024-09-10T23:37:42.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.629+0000 7f5f34400640 1 -- 172.21.15.94:0/1716700319 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f2c0c6460 con 0x7f5f2c0d0710 2024-09-10T23:37:42.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.629+0000 7f5f29600640 1 -- 172.21.15.94:0/1716700319 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 96547+0+0 (secure 0 0 0) 0x7f5f18003a00 con 0x7f5f2c0d0710 2024-09-10T23:37:42.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.629+0000 7f5f29600640 1 --2- 172.21.15.94:0/1716700319 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f0c0727e0 0x7f5f0c074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:42.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.629+0000 7f5f29600640 1 -- 172.21.15.94:0/1716700319 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7f5f18092c10 con 0x7f5f2c0d0710 2024-09-10T23:37:42.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.629+0000 7f5f31a00640 1 --2- 172.21.15.94:0/1716700319 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f0c0727e0 0x7f5f0c074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:42.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.629+0000 7f5f31a00640 1 --2- 172.21.15.94:0/1716700319 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f0c0727e0 0x7f5f0c074ca0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5f2001a450 tx=0x7f5f2001b060 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:42.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.637+0000 7f5f29600640 1 -- 172.21.15.94:0/1716700319 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5f1805da90 con 0x7f5f2c0d0710 2024-09-10T23:37:42.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:42.845+0000 7f5f34400640 1 -- 172.21.15.94:0/1716700319 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone39050", "target": ["mon-mgr", ""]}) -- 0x7f5f2c069e20 con 0x7f5f0c0727e0 2024-09-10T23:37:43.435 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:43.433+0000 7f9e88200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:43.435 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:43.433+0000 7f9e88200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:43.435 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:43.433+0000 7f9e88200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:43.435 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:43.433+0000 7f9e88200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:43.435 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:37:43.433+0000 7f9e88200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:37:43.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.445+0000 7f5f29600640 1 -- 172.21.15.94:0/1716700319 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f5f2c069e20 con 0x7f5f0c0727e0 2024-09-10T23:37:43.446 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:37:43.446 INFO:teuthology.orchestra.run.smithi094.stdout: "status": { 2024-09-10T23:37:43.446 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "complete" 2024-09-10T23:37:43.446 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:43.446 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:37:43.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.445+0000 7f5f34400640 1 -- 172.21.15.94:0/1716700319 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f0c0727e0 msgr2=0x7f5f0c074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:43.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.445+0000 7f5f34400640 1 --2- 172.21.15.94:0/1716700319 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f0c0727e0 0x7f5f0c074ca0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5f2001a450 tx=0x7f5f2001b060 comp rx=0 tx=0).stop 2024-09-10T23:37:43.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.445+0000 7f5f34400640 1 -- 172.21.15.94:0/1716700319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c0d0710 msgr2=0x7f5f2c1fb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:43.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.445+0000 7f5f34400640 1 --2- 172.21.15.94:0/1716700319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c0d0710 0x7f5f2c1fb9e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800c400 tx=0x7f5f1800c8c0 comp rx=0 tx=0).stop 2024-09-10T23:37:43.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.445+0000 7f5f34400640 1 -- 172.21.15.94:0/1716700319 shutdown_connections 2024-09-10T23:37:43.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.445+0000 7f5f34400640 1 --2- 172.21.15.94:0/1716700319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f2c0d0710 0x7f5f2c1fb9e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:43.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.445+0000 7f5f34400640 1 --2- 172.21.15.94:0/1716700319 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f0c0727e0 0x7f5f0c074ca0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:43.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.445+0000 7f5f34400640 1 --2- 172.21.15.94:0/1716700319 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f2c1455d0 0x7f5f2c073070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:43.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.445+0000 7f5f34400640 1 --2- 172.21.15.94:0/1716700319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0cd9f0 0x7f5f2c160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:43.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.445+0000 7f5f34400640 1 -- 172.21.15.94:0/1716700319 >> 172.21.15.94:0/1716700319 conn(0x7f5f2c0cfaf0 msgr2=0x7f5f2c0b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:43.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.445+0000 7f5f34400640 1 -- 172.21.15.94:0/1716700319 shutdown_connections 2024-09-10T23:37:43.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.445+0000 7f5f34400640 1 -- 172.21.15.94:0/1716700319 wait complete. 2024-09-10T23:37:43.468 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-10T23:37:43.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:43.469 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone39050/.meta) 2024-09-10T23:37:43.493 INFO:teuthology.orchestra.run.smithi094.stdout:[GLOBAL] 2024-09-10T23:37:43.493 INFO:teuthology.orchestra.run.smithi094.stdout:version = 2 2024-09-10T23:37:43.494 INFO:teuthology.orchestra.run.smithi094.stdout:type = clone 2024-09-10T23:37:43.494 INFO:teuthology.orchestra.run.smithi094.stdout:path = /volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6 2024-09-10T23:37:43.494 INFO:teuthology.orchestra.run.smithi094.stdout:state = complete 2024-09-10T23:37:43.494 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:43.496 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol55860 2024-09-10T23:37:43.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.681+0000 7f173b800640 1 -- 172.21.15.94:0/3043652596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17340cb4d0 msgr2=0x7f173415b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:43.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.681+0000 7f173b800640 1 --2- 172.21.15.94:0/3043652596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17340cb4d0 0x7f173415b190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f172c009900 tx=0x7f172c01c8e0 comp rx=0 tx=0).stop 2024-09-10T23:37:43.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.681+0000 7f173b800640 1 -- 172.21.15.94:0/3043652596 shutdown_connections 2024-09-10T23:37:43.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.681+0000 7f173b800640 1 --2- 172.21.15.94:0/3043652596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17340cb4d0 0x7f173415b190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:43.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.681+0000 7f173b800640 1 --2- 172.21.15.94:0/3043652596 >> v2:172.21.15.181:3300/0 conn(0x7f173406fc00 0x7f17340c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:43.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.681+0000 7f173b800640 1 -- 172.21.15.94:0/3043652596 >> 172.21.15.94:0/3043652596 conn(0x7f17340cfaf0 msgr2=0x7f17340cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:43.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.681+0000 7f173b800640 1 -- 172.21.15.94:0/3043652596 shutdown_connections 2024-09-10T23:37:43.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.681+0000 7f173b800640 1 -- 172.21.15.94:0/3043652596 wait complete. 2024-09-10T23:37:43.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.681+0000 7f173b800640 1 Processor -- start 2024-09-10T23:37:43.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.681+0000 7f173b800640 1 -- start start 2024-09-10T23:37:43.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.681+0000 7f173b800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f173406fc00 0x7f17341fb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.681+0000 7f173b800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17340cb4d0 0x7f173414c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.681+0000 7f173b800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1734158790 0x7f173415a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.681+0000 7f173b800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1734168070 con 0x7f17340cb4d0 2024-09-10T23:37:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.681+0000 7f173b800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1734167ef0 con 0x7f1734158790 2024-09-10T23:37:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.681+0000 7f173b800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f17341681f0 con 0x7f173406fc00 2024-09-10T23:37:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f1733e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17340cb4d0 0x7f173414c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f1733e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17340cb4d0 0x7f173414c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55140/0 (socket says 172.21.15.94:55140) 2024-09-10T23:37:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f1733e00640 1 -- 172.21.15.94:0/3186707916 learned_addr learned my addr 172.21.15.94:0/3186707916 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f1738e00640 1 --2- 172.21.15.94:0/3186707916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f173406fc00 0x7f17341fb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f1739800640 1 --2- 172.21.15.94:0/3186707916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1734158790 0x7f173415a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f1733e00640 1 -- 172.21.15.94:0/3186707916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f173406fc00 msgr2=0x7f17341fb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f1733e00640 1 --2- 172.21.15.94:0/3186707916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f173406fc00 0x7f17341fb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f1733e00640 1 -- 172.21.15.94:0/3186707916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1734158790 msgr2=0x7f173415a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f1733e00640 1 --2- 172.21.15.94:0/3186707916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1734158790 0x7f173415a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f1733e00640 1 -- 172.21.15.94:0/3186707916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17341fb980 con 0x7f17340cb4d0 2024-09-10T23:37:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f1738e00640 1 --2- 172.21.15.94:0/3186707916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f173406fc00 0x7f17341fb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f1733e00640 1 --2- 172.21.15.94:0/3186707916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17340cb4d0 0x7f173414c180 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f172000e9f0 tx=0x7f172000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f1730c00640 1 -- 172.21.15.94:0/3186707916 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172000cd80 con 0x7f17340cb4d0 2024-09-10T23:37:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f1730c00640 1 -- 172.21.15.94:0/3186707916 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1720010430 con 0x7f17340cb4d0 2024-09-10T23:37:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f173b800640 1 -- 172.21.15.94:0/3186707916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17340ae510 con 0x7f17340cb4d0 2024-09-10T23:37:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f1730c00640 1 -- 172.21.15.94:0/3186707916 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720010860 con 0x7f17340cb4d0 2024-09-10T23:37:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f173b800640 1 -- 172.21.15.94:0/3186707916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17340aead0 con 0x7f17340cb4d0 2024-09-10T23:37:43.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f173b800640 1 -- 172.21.15.94:0/3186707916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17340c49b0 con 0x7f17340cb4d0 2024-09-10T23:37:43.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.685+0000 7f1730c00640 1 -- 172.21.15.94:0/3186707916 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 96547+0+0 (secure 0 0 0) 0x7f1720010a00 con 0x7f17340cb4d0 2024-09-10T23:37:43.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.689+0000 7f1730c00640 1 --2- 172.21.15.94:0/3186707916 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1714072790 0x7f1714074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:43.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.689+0000 7f1730c00640 1 -- 172.21.15.94:0/3186707916 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7f1720094340 con 0x7f17340cb4d0 2024-09-10T23:37:43.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.689+0000 7f1738e00640 1 --2- 172.21.15.94:0/3186707916 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1714072790 0x7f1714074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:43.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.689+0000 7f1738e00640 1 --2- 172.21.15.94:0/3186707916 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1714072790 0x7f1714074c50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1734070a90 tx=0x7f1724009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:43.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.693+0000 7f1730c00640 1 -- 172.21.15.94:0/3186707916 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f172005f670 con 0x7f17340cb4d0 2024-09-10T23:37:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.881+0000 7f173b800640 1 -- 172.21.15.94:0/3186707916 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol55860", "target": ["mon-mgr", ""]}) -- 0x7f17340d0850 con 0x7f1714072790 2024-09-10T23:37:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.901+0000 7f1730c00640 1 -- 172.21.15.94:0/3186707916 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f17340d0850 con 0x7f1714072790 2024-09-10T23:37:43.902 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad 2024-09-10T23:37:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.901+0000 7f173b800640 1 -- 172.21.15.94:0/3186707916 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1714072790 msgr2=0x7f1714074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.901+0000 7f173b800640 1 --2- 172.21.15.94:0/3186707916 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1714072790 0x7f1714074c50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1734070a90 tx=0x7f1724009040 comp rx=0 tx=0).stop 2024-09-10T23:37:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.901+0000 7f173b800640 1 -- 172.21.15.94:0/3186707916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17340cb4d0 msgr2=0x7f173414c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.901+0000 7f173b800640 1 --2- 172.21.15.94:0/3186707916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17340cb4d0 0x7f173414c180 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f172000e9f0 tx=0x7f172000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:37:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.901+0000 7f173b800640 1 -- 172.21.15.94:0/3186707916 shutdown_connections 2024-09-10T23:37:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.901+0000 7f173b800640 1 --2- 172.21.15.94:0/3186707916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1734158790 0x7f173415a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.901+0000 7f173b800640 1 --2- 172.21.15.94:0/3186707916 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1714072790 0x7f1714074c50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.901+0000 7f173b800640 1 --2- 172.21.15.94:0/3186707916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17340cb4d0 0x7f173414c180 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.901+0000 7f173b800640 1 --2- 172.21.15.94:0/3186707916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f173406fc00 0x7f17341fb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.901+0000 7f173b800640 1 -- 172.21.15.94:0/3186707916 >> 172.21.15.94:0/3186707916 conn(0x7f17340cfaf0 msgr2=0x7f17340b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.901+0000 7f173b800640 1 -- 172.21.15.94:0/3186707916 shutdown_connections 2024-09-10T23:37:43.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:43.901+0000 7f173b800640 1 -- 172.21.15.94:0/3186707916 wait complete. 2024-09-10T23:37:43.924 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_clone39050 2024-09-10T23:37:44.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d18e00640 1 -- 172.21.15.94:0/1300530535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d140ce890 msgr2=0x7f6d140cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:44.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d18e00640 1 --2- 172.21.15.94:0/1300530535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d140ce890 0x7f6d140cd710 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f6d140d18b0 tx=0x7f6d0401caf0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d18e00640 1 -- 172.21.15.94:0/1300530535 shutdown_connections 2024-09-10T23:37:44.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d18e00640 1 --2- 172.21.15.94:0/1300530535 >> v2:172.21.15.181:3300/0 conn(0x7f6d140b1100 0x7f6d140b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d18e00640 1 --2- 172.21.15.94:0/1300530535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d140ce890 0x7f6d140cd710 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d18e00640 1 -- 172.21.15.94:0/1300530535 >> 172.21.15.94:0/1300530535 conn(0x7f6d140d1c60 msgr2=0x7f6d140d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:44.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d18e00640 1 -- 172.21.15.94:0/1300530535 shutdown_connections 2024-09-10T23:37:44.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d18e00640 1 -- 172.21.15.94:0/1300530535 wait complete. 2024-09-10T23:37:44.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d18e00640 1 Processor -- start 2024-09-10T23:37:44.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d18e00640 1 -- start start 2024-09-10T23:37:44.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d18e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d140b1100 0x7f6d140b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:44.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d18e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d140ce1e0 0x7f6d140718d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:44.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d18e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d140ce890 0x7f6d14071e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:44.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d18e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6d140d0a90 con 0x7f6d140ce1e0 2024-09-10T23:37:44.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d18e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d140d0910 con 0x7f6d140ce890 2024-09-10T23:37:44.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d18e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6d14169110 con 0x7f6d140b1100 2024-09-10T23:37:44.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d09400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d140ce1e0 0x7f6d140718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:44.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d11e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d140b1100 0x7f6d140b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:44.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d09400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d140ce1e0 0x7f6d140718d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55586/0 (socket says 172.21.15.94:55586) 2024-09-10T23:37:44.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d09400640 1 -- 172.21.15.94:0/3032125136 learned_addr learned my addr 172.21.15.94:0/3032125136 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d12800640 1 --2- 172.21.15.94:0/3032125136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d140ce890 0x7f6d14071e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d09400640 1 -- 172.21.15.94:0/3032125136 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d140b1100 msgr2=0x7f6d140b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d09400640 1 --2- 172.21.15.94:0/3032125136 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d140b1100 0x7f6d140b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d09400640 1 -- 172.21.15.94:0/3032125136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d140ce890 msgr2=0x7f6d14071e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d09400640 1 --2- 172.21.15.94:0/3032125136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d140ce890 0x7f6d14071e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.069+0000 7f6d09400640 1 -- 172.21.15.94:0/3032125136 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d140b7270 con 0x7f6d140ce1e0 2024-09-10T23:37:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.073+0000 7f6d09400640 1 --2- 172.21.15.94:0/3032125136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d140ce1e0 0x7f6d140718d0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00c420 tx=0x7f6cfc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.073+0000 7f6d0aa00640 1 -- 172.21.15.94:0/3032125136 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc017070 con 0x7f6d140ce1e0 2024-09-10T23:37:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.073+0000 7f6d0aa00640 1 -- 172.21.15.94:0/3032125136 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cfc003600 con 0x7f6d140ce1e0 2024-09-10T23:37:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.073+0000 7f6d0aa00640 1 -- 172.21.15.94:0/3032125136 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc003a90 con 0x7f6d140ce1e0 2024-09-10T23:37:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.073+0000 7f6d18e00640 1 -- 172.21.15.94:0/3032125136 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d140b7550 con 0x7f6d140ce1e0 2024-09-10T23:37:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.073+0000 7f6d18e00640 1 -- 172.21.15.94:0/3032125136 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d140b5760 con 0x7f6d140ce1e0 2024-09-10T23:37:44.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.073+0000 7f6d18e00640 1 -- 172.21.15.94:0/3032125136 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d140b7a00 con 0x7f6d140ce1e0 2024-09-10T23:37:44.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.073+0000 7f6d0aa00640 1 -- 172.21.15.94:0/3032125136 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 96547+0+0 (secure 0 0 0) 0x7f6cfc003c30 con 0x7f6d140ce1e0 2024-09-10T23:37:44.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.073+0000 7f6d0aa00640 1 --2- 172.21.15.94:0/3032125136 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6cf00727c0 0x7f6cf0074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:44.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.073+0000 7f6d0aa00640 1 -- 172.21.15.94:0/3032125136 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7f6cfc092bd0 con 0x7f6d140ce1e0 2024-09-10T23:37:44.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.073+0000 7f6d11e00640 1 --2- 172.21.15.94:0/3032125136 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6cf00727c0 0x7f6cf0074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:44.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.077+0000 7f6d11e00640 1 --2- 172.21.15.94:0/3032125136 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6cf00727c0 0x7f6cf0074c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6d040af000 tx=0x7f6d0401b330 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:44.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.077+0000 7f6d0aa00640 1 -- 172.21.15.94:0/3032125136 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6cfc05d880 con 0x7f6d140ce1e0 2024-09-10T23:37:44.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.269+0000 7f6d18e00640 1 -- 172.21.15.94:0/3032125136 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_clone39050", "target": ["mon-mgr", ""]}) -- 0x7f6d14158ef0 con 0x7f6cf00727c0 2024-09-10T23:37:44.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.277+0000 7f6d0aa00640 1 -- 172.21.15.94:0/3032125136 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+73 (secure 0 0 0) 0x7f6d14158ef0 con 0x7f6cf00727c0 2024-09-10T23:37:44.279 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6 2024-09-10T23:37:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.277+0000 7f6d18e00640 1 -- 172.21.15.94:0/3032125136 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6cf00727c0 msgr2=0x7f6cf0074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.277+0000 7f6d18e00640 1 --2- 172.21.15.94:0/3032125136 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6cf00727c0 0x7f6cf0074c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6d040af000 tx=0x7f6d0401b330 comp rx=0 tx=0).stop 2024-09-10T23:37:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.277+0000 7f6d18e00640 1 -- 172.21.15.94:0/3032125136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d140ce1e0 msgr2=0x7f6d140718d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.277+0000 7f6d18e00640 1 --2- 172.21.15.94:0/3032125136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d140ce1e0 0x7f6d140718d0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00c420 tx=0x7f6cfc00c8e0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.281+0000 7f6d18e00640 1 -- 172.21.15.94:0/3032125136 shutdown_connections 2024-09-10T23:37:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.281+0000 7f6d18e00640 1 --2- 172.21.15.94:0/3032125136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d140ce890 0x7f6d14071e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.281+0000 7f6d18e00640 1 --2- 172.21.15.94:0/3032125136 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6cf00727c0 0x7f6cf0074c80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.281+0000 7f6d18e00640 1 --2- 172.21.15.94:0/3032125136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d140ce1e0 0x7f6d140718d0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.281+0000 7f6d18e00640 1 --2- 172.21.15.94:0/3032125136 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d140b1100 0x7f6d140b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.281+0000 7f6d18e00640 1 -- 172.21.15.94:0/3032125136 >> 172.21.15.94:0/3032125136 conn(0x7f6d140d1c60 msgr2=0x7f6d140ae8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.281+0000 7f6d18e00640 1 -- 172.21.15.94:0/3032125136 shutdown_connections 2024-09-10T23:37:44.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.281+0000 7f6d18e00640 1 -- 172.21.15.94:0/3032125136 wait complete. 2024-09-10T23:37:44.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:44.301 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870) 2024-09-10T23:37:44.322 INFO:teuthology.orchestra.run.smithi094.stdout:9 2024-09-10T23:37:44.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:44.322 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6) 2024-09-10T23:37:44.383 INFO:teuthology.orchestra.run.smithi094.stdout:9 2024-09-10T23:37:44.384 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol_clone39050 2024-09-10T23:37:44.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.565+0000 7ff93c200640 1 -- 172.21.15.94:0/1748537714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340ce890 msgr2=0x7ff9340cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:44.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.565+0000 7ff93c200640 1 --2- 172.21.15.94:0/1748537714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340ce890 0x7ff9340cd710 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ff928009840 tx=0x7ff92801c760 comp rx=0 tx=0).stop 2024-09-10T23:37:44.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.565+0000 7ff93c200640 1 -- 172.21.15.94:0/1748537714 shutdown_connections 2024-09-10T23:37:44.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.565+0000 7ff93c200640 1 --2- 172.21.15.94:0/1748537714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9340ce1e0 0x7ff9340cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.565+0000 7ff93c200640 1 --2- 172.21.15.94:0/1748537714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340ce890 0x7ff9340cd710 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.565+0000 7ff93c200640 1 -- 172.21.15.94:0/1748537714 >> 172.21.15.94:0/1748537714 conn(0x7ff9340d1c60 msgr2=0x7ff9340d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:44.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.565+0000 7ff93c200640 1 -- 172.21.15.94:0/1748537714 shutdown_connections 2024-09-10T23:37:44.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.565+0000 7ff93c200640 1 -- 172.21.15.94:0/1748537714 wait complete. 2024-09-10T23:37:44.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93c200640 1 Processor -- start 2024-09-10T23:37:44.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93c200640 1 -- start start 2024-09-10T23:37:44.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93c200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9340b1100 0x7ff9340b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:44.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93c200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9340ce1e0 0x7ff9341fb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:44.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93c200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340ce890 0x7ff934152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:44.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93c200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff9340d0960 con 0x7ff9340ce890 2024-09-10T23:37:44.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93c200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff9340d07e0 con 0x7ff9340ce1e0 2024-09-10T23:37:44.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93c200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff9340d0ae0 con 0x7ff9340b1100 2024-09-10T23:37:44.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340ce890 0x7ff934152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:44.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340ce890 0x7ff934152820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55602/0 (socket says 172.21.15.94:55602) 2024-09-10T23:37:44.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93a200640 1 -- 172.21.15.94:0/2531158129 learned_addr learned my addr 172.21.15.94:0/2531158129 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:44.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff939800640 1 --2- 172.21.15.94:0/2531158129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9340b1100 0x7ff9340b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:44.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff938e00640 1 --2- 172.21.15.94:0/2531158129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9340ce1e0 0x7ff9341fb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:44.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93a200640 1 -- 172.21.15.94:0/2531158129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9340b1100 msgr2=0x7ff9340b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:44.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93a200640 1 --2- 172.21.15.94:0/2531158129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9340b1100 0x7ff9340b75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93a200640 1 -- 172.21.15.94:0/2531158129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9340ce1e0 msgr2=0x7ff9341fb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:44.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93a200640 1 --2- 172.21.15.94:0/2531158129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9340ce1e0 0x7ff9341fb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93a200640 1 -- 172.21.15.94:0/2531158129 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9340b5d60 con 0x7ff9340ce890 2024-09-10T23:37:44.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff939800640 1 --2- 172.21.15.94:0/2531158129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9340b1100 0x7ff9340b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:44.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93a200640 1 --2- 172.21.15.94:0/2531158129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340ce890 0x7ff934152820 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00c770 tx=0x7ff92c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:44.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff931600640 1 -- 172.21.15.94:0/2531158129 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c017070 con 0x7ff9340ce890 2024-09-10T23:37:44.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff931600640 1 -- 172.21.15.94:0/2531158129 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff92c005070 con 0x7ff9340ce890 2024-09-10T23:37:44.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93c200640 1 -- 172.21.15.94:0/2531158129 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9341f7ba0 con 0x7ff9340ce890 2024-09-10T23:37:44.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff931600640 1 -- 172.21.15.94:0/2531158129 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0054c0 con 0x7ff9340ce890 2024-09-10T23:37:44.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.569+0000 7ff93c200640 1 -- 172.21.15.94:0/2531158129 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9341f8160 con 0x7ff9340ce890 2024-09-10T23:37:44.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.573+0000 7ff93c200640 1 -- 172.21.15.94:0/2531158129 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9340cd710 con 0x7ff9340ce890 2024-09-10T23:37:44.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.573+0000 7ff931600640 1 -- 172.21.15.94:0/2531158129 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 96547+0+0 (secure 0 0 0) 0x7ff92c005660 con 0x7ff9340ce890 2024-09-10T23:37:44.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.573+0000 7ff931600640 1 --2- 172.21.15.94:0/2531158129 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff914072790 0x7ff914074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:44.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.573+0000 7ff931600640 1 -- 172.21.15.94:0/2531158129 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7ff92c092ff0 con 0x7ff9340ce890 2024-09-10T23:37:44.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.573+0000 7ff939800640 1 --2- 172.21.15.94:0/2531158129 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff914072790 0x7ff914074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:44.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.573+0000 7ff939800640 1 --2- 172.21.15.94:0/2531158129 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff914072790 0x7ff914074c50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff9280af000 tx=0x7ff92801afa0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:44.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.577+0000 7ff931600640 1 -- 172.21.15.94:0/2531158129 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff92c05dc50 con 0x7ff9340ce890 2024-09-10T23:37:44.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.669+0000 7ff931600640 1 -- 172.21.15.94:0/2531158129 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 18) ==== 96700+0+0 (secure 0 0 0) 0x7ff92c059ec0 con 0x7ff9340ce890 2024-09-10T23:37:44.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.777+0000 7ff93c200640 1 -- 172.21.15.94:0/2531158129 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol_clone39050", "target": ["mon-mgr", ""]}) -- 0x7ff9340c8af0 con 0x7ff914072790 2024-09-10T23:37:44.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.785+0000 7ff931600640 1 -- 172.21.15.94:0/2531158129 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+724 (secure 0 0 0) 0x7ff9340c8af0 con 0x7ff914072790 2024-09-10T23:37:44.786 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:37:44.786 INFO:teuthology.orchestra.run.smithi094.stdout: "atime": "2024-09-10 23:37:37", 2024-09-10T23:37:44.786 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_pcent": "80.00", 2024-09-10T23:37:44.786 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_quota": 10485760, 2024-09-10T23:37:44.786 INFO:teuthology.orchestra.run.smithi094.stdout: "bytes_used": 8388608, 2024-09-10T23:37:44.786 INFO:teuthology.orchestra.run.smithi094.stdout: "created_at": "2024-09-10 23:37:42", 2024-09-10T23:37:44.786 INFO:teuthology.orchestra.run.smithi094.stdout: "ctime": "2024-09-10 23:37:42", 2024-09-10T23:37:44.786 INFO:teuthology.orchestra.run.smithi094.stdout: "data_pool": "cephfs_data", 2024-09-10T23:37:44.786 INFO:teuthology.orchestra.run.smithi094.stdout: "features": [ 2024-09-10T23:37:44.787 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot-clone", 2024-09-10T23:37:44.787 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot-autoprotect", 2024-09-10T23:37:44.787 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot-retention" 2024-09-10T23:37:44.787 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:37:44.787 INFO:teuthology.orchestra.run.smithi094.stdout: "flavor": 2, 2024-09-10T23:37:44.787 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 1000, 2024-09-10T23:37:44.787 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": 16895, 2024-09-10T23:37:44.787 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_addrs": [ 2024-09-10T23:37:44.787 INFO:teuthology.orchestra.run.smithi094.stdout: "172.21.15.94:6789", 2024-09-10T23:37:44.787 INFO:teuthology.orchestra.run.smithi094.stdout: "172.21.15.181:6789", 2024-09-10T23:37:44.787 INFO:teuthology.orchestra.run.smithi094.stdout: "172.21.15.181:6790" 2024-09-10T23:37:44.787 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-10T23:37:44.788 INFO:teuthology.orchestra.run.smithi094.stdout: "mtime": "2024-09-10 23:37:41", 2024-09-10T23:37:44.788 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6", 2024-09-10T23:37:44.788 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_namespace": "", 2024-09-10T23:37:44.788 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "complete", 2024-09-10T23:37:44.788 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "clone", 2024-09-10T23:37:44.788 INFO:teuthology.orchestra.run.smithi094.stdout: "uid": 1000 2024-09-10T23:37:44.788 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:37:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.785+0000 7ff93c200640 1 -- 172.21.15.94:0/2531158129 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff914072790 msgr2=0x7ff914074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.785+0000 7ff93c200640 1 --2- 172.21.15.94:0/2531158129 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff914072790 0x7ff914074c50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff9280af000 tx=0x7ff92801afa0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.785+0000 7ff93c200640 1 -- 172.21.15.94:0/2531158129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340ce890 msgr2=0x7ff934152820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.785+0000 7ff93c200640 1 --2- 172.21.15.94:0/2531158129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340ce890 0x7ff934152820 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00c770 tx=0x7ff92c00cc30 comp rx=0 tx=0).stop 2024-09-10T23:37:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.785+0000 7ff93c200640 1 -- 172.21.15.94:0/2531158129 shutdown_connections 2024-09-10T23:37:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.785+0000 7ff93c200640 1 --2- 172.21.15.94:0/2531158129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9340ce1e0 0x7ff9341fb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.785+0000 7ff93c200640 1 --2- 172.21.15.94:0/2531158129 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff914072790 0x7ff914074c50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.785+0000 7ff93c200640 1 --2- 172.21.15.94:0/2531158129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9340ce890 0x7ff934152820 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.785+0000 7ff93c200640 1 --2- 172.21.15.94:0/2531158129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9340b1100 0x7ff9340b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.785+0000 7ff93c200640 1 -- 172.21.15.94:0/2531158129 >> 172.21.15.94:0/2531158129 conn(0x7ff9340d1c60 msgr2=0x7ff9340d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.785+0000 7ff93c200640 1 -- 172.21.15.94:0/2531158129 shutdown_connections 2024-09-10T23:37:44.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:44.785+0000 7ff93c200640 1 -- 172.21.15.94:0/2531158129 wait complete. 2024-09-10T23:37:44.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:44.807 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870) 2024-09-10T23:37:44.825 INFO:teuthology.orchestra.run.smithi094.stdout:10485760 2024-09-10T23:37:44.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:44.825 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870) 2024-09-10T23:37:44.886 INFO:teuthology.orchestra.run.smithi094.stdout:cephfs_data 2024-09-10T23:37:44.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:44.886 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870) 2024-09-10T23:37:44.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:44.947 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870) 2024-09-10T23:37:45.010 INFO:teuthology.orchestra.run.smithi094.stdout:volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870 2024-09-10T23:37:45.010 INFO:teuthology.orchestra.run.smithi094.stdout:volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.2 2024-09-10T23:37:45.010 INFO:teuthology.orchestra.run.smithi094.stdout:volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.5 2024-09-10T23:37:45.011 INFO:teuthology.orchestra.run.smithi094.stdout:volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.4 2024-09-10T23:37:45.011 INFO:teuthology.orchestra.run.smithi094.stdout:volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.7 2024-09-10T23:37:45.011 INFO:teuthology.orchestra.run.smithi094.stdout:volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.6 2024-09-10T23:37:45.011 INFO:teuthology.orchestra.run.smithi094.stdout:volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.0 2024-09-10T23:37:45.011 INFO:teuthology.orchestra.run.smithi094.stdout:volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.3 2024-09-10T23:37:45.011 INFO:teuthology.orchestra.run.smithi094.stdout:volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.1 2024-09-10T23:37:45.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.012 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870) 2024-09-10T23:37:45.064 INFO:teuthology.orchestra.run.smithi094.stdout:41ff 2024-09-10T23:37:45.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.065 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/) 2024-09-10T23:37:45.114 INFO:teuthology.orchestra.run.smithi094.stdout:41ff 2024-09-10T23:37:45.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.115 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870) 2024-09-10T23:37:45.168 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:45.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.169 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/) 2024-09-10T23:37:45.218 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:45.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.219 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870) 2024-09-10T23:37:45.272 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:45.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.273 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/) 2024-09-10T23:37:45.322 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:45.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.323 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870) 2024-09-10T23:37:45.380 INFO:teuthology.orchestra.run.smithi094.stdout:1726011461 2024-09-10T23:37:45.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.382 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/) 2024-09-10T23:37:45.434 INFO:teuthology.orchestra.run.smithi094.stdout:1726011461 2024-09-10T23:37:45.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.435 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.2) 2024-09-10T23:37:45.492 INFO:teuthology.orchestra.run.smithi094.stdout:81b4 2024-09-10T23:37:45.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.493 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.2) 2024-09-10T23:37:45.555 INFO:teuthology.orchestra.run.smithi094.stdout:81b4 2024-09-10T23:37:45.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.556 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.2) 2024-09-10T23:37:45.609 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:45.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.610 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.2) 2024-09-10T23:37:45.662 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:45.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.663 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.2) 2024-09-10T23:37:45.717 INFO:teuthology.orchestra.run.smithi094.stdout:1271 2024-09-10T23:37:45.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.718 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.2) 2024-09-10T23:37:45.770 INFO:teuthology.orchestra.run.smithi094.stdout:1271 2024-09-10T23:37:45.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.771 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.2) 2024-09-10T23:37:45.828 INFO:teuthology.orchestra.run.smithi094.stdout:1726011460 2024-09-10T23:37:45.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.829 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.2) 2024-09-10T23:37:45.882 INFO:teuthology.orchestra.run.smithi094.stdout:1726011460 2024-09-10T23:37:45.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.883 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.5) 2024-09-10T23:37:45.936 INFO:teuthology.orchestra.run.smithi094.stdout:81b4 2024-09-10T23:37:45.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.937 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.5) 2024-09-10T23:37:45.998 INFO:teuthology.orchestra.run.smithi094.stdout:81b4 2024-09-10T23:37:45.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:45.999 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.5) 2024-09-10T23:37:46.052 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:46.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.053 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.5) 2024-09-10T23:37:46.106 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:46.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.107 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.5) 2024-09-10T23:37:46.160 INFO:teuthology.orchestra.run.smithi094.stdout:1271 2024-09-10T23:37:46.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.161 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.5) 2024-09-10T23:37:46.214 INFO:teuthology.orchestra.run.smithi094.stdout:1271 2024-09-10T23:37:46.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.215 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.5) 2024-09-10T23:37:46.269 INFO:teuthology.orchestra.run.smithi094.stdout:1726011461 2024-09-10T23:37:46.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.270 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.5) 2024-09-10T23:37:46.322 INFO:teuthology.orchestra.run.smithi094.stdout:1726011461 2024-09-10T23:37:46.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.323 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.4) 2024-09-10T23:37:46.376 INFO:teuthology.orchestra.run.smithi094.stdout:81b4 2024-09-10T23:37:46.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.377 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.4) 2024-09-10T23:37:46.442 INFO:teuthology.orchestra.run.smithi094.stdout:81b4 2024-09-10T23:37:46.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.444 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.4) 2024-09-10T23:37:46.496 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:46.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.497 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.4) 2024-09-10T23:37:46.550 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:46.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.552 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.4) 2024-09-10T23:37:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:1271 2024-09-10T23:37:46.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.610 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.4) 2024-09-10T23:37:46.662 INFO:teuthology.orchestra.run.smithi094.stdout:1271 2024-09-10T23:37:46.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.664 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.4) 2024-09-10T23:37:46.720 INFO:teuthology.orchestra.run.smithi094.stdout:1726011461 2024-09-10T23:37:46.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.721 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.4) 2024-09-10T23:37:46.774 INFO:teuthology.orchestra.run.smithi094.stdout:1726011461 2024-09-10T23:37:46.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.775 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.7) 2024-09-10T23:37:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:81b4 2024-09-10T23:37:46.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.830 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.7) 2024-09-10T23:37:46.893 INFO:teuthology.orchestra.run.smithi094.stdout:81b4 2024-09-10T23:37:46.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.894 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.7) 2024-09-10T23:37:46.948 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:46.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:46.950 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.7) 2024-09-10T23:37:47.002 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:47.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.004 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.7) 2024-09-10T23:37:47.056 INFO:teuthology.orchestra.run.smithi094.stdout:1271 2024-09-10T23:37:47.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.057 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.7) 2024-09-10T23:37:47.110 INFO:teuthology.orchestra.run.smithi094.stdout:1271 2024-09-10T23:37:47.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.111 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.7) 2024-09-10T23:37:47.163 INFO:teuthology.orchestra.run.smithi094.stdout:1726011461 2024-09-10T23:37:47.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.165 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.7) 2024-09-10T23:37:47.218 INFO:teuthology.orchestra.run.smithi094.stdout:1726011461 2024-09-10T23:37:47.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.219 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.6) 2024-09-10T23:37:47.276 INFO:teuthology.orchestra.run.smithi094.stdout:81b4 2024-09-10T23:37:47.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.277 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.6) 2024-09-10T23:37:47.341 INFO:teuthology.orchestra.run.smithi094.stdout:81b4 2024-09-10T23:37:47.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.342 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.6) 2024-09-10T23:37:47.400 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:47.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.401 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.6) 2024-09-10T23:37:47.458 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:47.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.459 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.6) 2024-09-10T23:37:47.512 INFO:teuthology.orchestra.run.smithi094.stdout:1271 2024-09-10T23:37:47.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.514 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.6) 2024-09-10T23:37:47.567 INFO:teuthology.orchestra.run.smithi094.stdout:1271 2024-09-10T23:37:47.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.568 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.6) 2024-09-10T23:37:47.624 INFO:teuthology.orchestra.run.smithi094.stdout:1726011461 2024-09-10T23:37:47.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.626 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.6) 2024-09-10T23:37:47.678 INFO:teuthology.orchestra.run.smithi094.stdout:1726011461 2024-09-10T23:37:47.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.679 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.0) 2024-09-10T23:37:47.731 INFO:teuthology.orchestra.run.smithi094.stdout:81b4 2024-09-10T23:37:47.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.732 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.0) 2024-09-10T23:37:47.790 INFO:teuthology.orchestra.run.smithi094.stdout:81b4 2024-09-10T23:37:47.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.791 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.0) 2024-09-10T23:37:47.845 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:47.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.846 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.0) 2024-09-10T23:37:47.898 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:47.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.899 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.0) 2024-09-10T23:37:47.952 INFO:teuthology.orchestra.run.smithi094.stdout:1271 2024-09-10T23:37:47.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:47.953 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.0) 2024-09-10T23:37:48.010 INFO:teuthology.orchestra.run.smithi094.stdout:1271 2024-09-10T23:37:48.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.011 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.0) 2024-09-10T23:37:48.064 INFO:teuthology.orchestra.run.smithi094.stdout:1726011460 2024-09-10T23:37:48.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.065 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.0) 2024-09-10T23:37:48.118 INFO:teuthology.orchestra.run.smithi094.stdout:1726011460 2024-09-10T23:37:48.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.119 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.3) 2024-09-10T23:37:48.172 INFO:teuthology.orchestra.run.smithi094.stdout:81b4 2024-09-10T23:37:48.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.173 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.3) 2024-09-10T23:37:48.235 INFO:teuthology.orchestra.run.smithi094.stdout:81b4 2024-09-10T23:37:48.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.236 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.3) 2024-09-10T23:37:48.288 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:48.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.289 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.3) 2024-09-10T23:37:48.346 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:48.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.347 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.3) 2024-09-10T23:37:48.400 INFO:teuthology.orchestra.run.smithi094.stdout:1271 2024-09-10T23:37:48.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.401 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.3) 2024-09-10T23:37:48.450 INFO:teuthology.orchestra.run.smithi094.stdout:1271 2024-09-10T23:37:48.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.451 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.3) 2024-09-10T23:37:48.504 INFO:teuthology.orchestra.run.smithi094.stdout:1726011461 2024-09-10T23:37:48.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.505 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.3) 2024-09-10T23:37:48.563 INFO:teuthology.orchestra.run.smithi094.stdout:1726011461 2024-09-10T23:37:48.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.564 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.1) 2024-09-10T23:37:48.620 INFO:teuthology.orchestra.run.smithi094.stdout:81b4 2024-09-10T23:37:48.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.621 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.1) 2024-09-10T23:37:48.682 INFO:teuthology.orchestra.run.smithi094.stdout:81b4 2024-09-10T23:37:48.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.683 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.1) 2024-09-10T23:37:48.736 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:48.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.738 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.1) 2024-09-10T23:37:48.790 INFO:teuthology.orchestra.run.smithi094.stdout:1000 2024-09-10T23:37:48.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.791 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.1) 2024-09-10T23:37:48.844 INFO:teuthology.orchestra.run.smithi094.stdout:1271 2024-09-10T23:37:48.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.845 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.1) 2024-09-10T23:37:48.898 INFO:teuthology.orchestra.run.smithi094.stdout:1271 2024-09-10T23:37:48.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.899 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad/.snap/subvol_snap51870/subvolume_file.1) 2024-09-10T23:37:48.955 INFO:teuthology.orchestra.run.smithi094.stdout:1726011460 2024-09-10T23:37:48.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:48.957 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone39050/8eaadb60-383f-4342-9528-f28377af92c6/subvolume_file.1) 2024-09-10T23:37:49.006 INFO:teuthology.orchestra.run.smithi094.stdout:1726011460 2024-09-10T23:37:49.007 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol55860 fake subvol_clone47961 2024-09-10T23:37:49.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.197+0000 7ffb2b800640 1 -- 172.21.15.94:0/2409339496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb24161580 msgr2=0x7ffb240d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:49.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.197+0000 7ffb2b800640 1 --2- 172.21.15.94:0/2409339496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb24161580 0x7ffb240d1e20 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ffb240b4f40 tx=0x7ffb1401c780 comp rx=0 tx=0).stop 2024-09-10T23:37:49.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.197+0000 7ffb2b800640 1 -- 172.21.15.94:0/2409339496 shutdown_connections 2024-09-10T23:37:49.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.197+0000 7ffb2b800640 1 --2- 172.21.15.94:0/2409339496 >> v2:172.21.15.181:3300/0 conn(0x7ffb240ce970 0x7ffb240ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:49.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.197+0000 7ffb2b800640 1 --2- 172.21.15.94:0/2409339496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb24161580 0x7ffb240d1e20 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:49.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.197+0000 7ffb2b800640 1 -- 172.21.15.94:0/2409339496 >> 172.21.15.94:0/2409339496 conn(0x7ffb240d3eb0 msgr2=0x7ffb240d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:49.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.197+0000 7ffb2b800640 1 -- 172.21.15.94:0/2409339496 shutdown_connections 2024-09-10T23:37:49.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.197+0000 7ffb2b800640 1 -- 172.21.15.94:0/2409339496 wait complete. 2024-09-10T23:37:49.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb2b800640 1 Processor -- start 2024-09-10T23:37:49.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb2b800640 1 -- start start 2024-09-10T23:37:49.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb2b800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb240b4f90 0x7ffb2415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:49.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb2b800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb240ce970 0x7ffb2414f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:49.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb2b800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb24161580 0x7ffb24147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:49.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb2b800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2416aef0 con 0x7ffb240ce970 2024-09-10T23:37:49.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb2b800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2416ad70 con 0x7ffb240b4f90 2024-09-10T23:37:49.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb2b800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ffb2416b070 con 0x7ffb24161580 2024-09-10T23:37:49.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb1be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb240ce970 0x7ffb2414f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:49.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb1be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb240ce970 0x7ffb2414f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55620/0 (socket says 172.21.15.94:55620) 2024-09-10T23:37:49.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb1be00640 1 -- 172.21.15.94:0/3893831990 learned_addr learned my addr 172.21.15.94:0/3893831990 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:49.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb28e00640 1 --2- 172.21.15.94:0/3893831990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb240b4f90 0x7ffb2415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:49.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb29800640 1 --2- 172.21.15.94:0/3893831990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb24161580 0x7ffb24147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:49.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb1be00640 1 -- 172.21.15.94:0/3893831990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb24161580 msgr2=0x7ffb24147e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:49.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb1be00640 1 --2- 172.21.15.94:0/3893831990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb24161580 0x7ffb24147e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:49.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb1be00640 1 -- 172.21.15.94:0/3893831990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb240b4f90 msgr2=0x7ffb2415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:49.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb1be00640 1 --2- 172.21.15.94:0/3893831990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb240b4f90 0x7ffb2415e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:49.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb1be00640 1 -- 172.21.15.94:0/3893831990 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb24151a20 con 0x7ffb240ce970 2024-09-10T23:37:49.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb28e00640 1 --2- 172.21.15.94:0/3893831990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb240b4f90 0x7ffb2415e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:37:49.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb1be00640 1 --2- 172.21.15.94:0/3893831990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb240ce970 0x7ffb2414f700 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c490 tx=0x7ffb0c001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:49.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb21600640 1 -- 172.21.15.94:0/3893831990 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c017070 con 0x7ffb240ce970 2024-09-10T23:37:49.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb21600640 1 -- 172.21.15.94:0/3893831990 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb0c003600 con 0x7ffb240ce970 2024-09-10T23:37:49.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb2b800640 1 -- 172.21.15.94:0/3893831990 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb240b1fa0 con 0x7ffb240ce970 2024-09-10T23:37:49.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb21600640 1 -- 172.21.15.94:0/3893831990 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c003a90 con 0x7ffb240ce970 2024-09-10T23:37:49.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb2b800640 1 -- 172.21.15.94:0/3893831990 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb24151d00 con 0x7ffb240ce970 2024-09-10T23:37:49.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.201+0000 7ffb2b800640 1 -- 172.21.15.94:0/3893831990 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb240ced80 con 0x7ffb240ce970 2024-09-10T23:37:49.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.205+0000 7ffb21600640 1 -- 172.21.15.94:0/3893831990 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 96700+0+0 (secure 0 0 0) 0x7ffb0c003c30 con 0x7ffb240ce970 2024-09-10T23:37:49.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.205+0000 7ffb21600640 1 --2- 172.21.15.94:0/3893831990 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ffb00072970 0x7ffb00074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:49.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.205+0000 7ffb21600640 1 -- 172.21.15.94:0/3893831990 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7ffb0c092e80 con 0x7ffb240ce970 2024-09-10T23:37:49.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.205+0000 7ffb28e00640 1 --2- 172.21.15.94:0/3893831990 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ffb00072970 0x7ffb00074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:49.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.205+0000 7ffb28e00640 1 --2- 172.21.15.94:0/3893831990 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ffb00072970 0x7ffb00074e30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffb140af000 tx=0x7ffb1401afc0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:49.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.209+0000 7ffb21600640 1 -- 172.21.15.94:0/3893831990 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ffb0c05e030 con 0x7ffb240ce970 2024-09-10T23:37:49.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.405+0000 7ffb2b800640 1 -- 172.21.15.94:0/3893831990 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol55860", "snap_name": "fake", "target_sub_name": "subvol_clone47961", "target": ["mon-mgr", ""]}) -- 0x7ffb24152fe0 con 0x7ffb00072970 2024-09-10T23:37:49.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.477+0000 7ffb21600640 1 -- 172.21.15.94:0/3893831990 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffb24152fe0 con 0x7ffb00072970 2024-09-10T23:37:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.481+0000 7ffb2b800640 1 -- 172.21.15.94:0/3893831990 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ffb00072970 msgr2=0x7ffb00074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.481+0000 7ffb2b800640 1 --2- 172.21.15.94:0/3893831990 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ffb00072970 0x7ffb00074e30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffb140af000 tx=0x7ffb1401afc0 comp rx=0 tx=0).stop 2024-09-10T23:37:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.481+0000 7ffb2b800640 1 -- 172.21.15.94:0/3893831990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb240ce970 msgr2=0x7ffb2414f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.481+0000 7ffb2b800640 1 --2- 172.21.15.94:0/3893831990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb240ce970 0x7ffb2414f700 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c490 tx=0x7ffb0c001e80 comp rx=0 tx=0).stop 2024-09-10T23:37:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.481+0000 7ffb2b800640 1 -- 172.21.15.94:0/3893831990 shutdown_connections 2024-09-10T23:37:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.481+0000 7ffb2b800640 1 --2- 172.21.15.94:0/3893831990 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ffb00072970 0x7ffb00074e30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.481+0000 7ffb2b800640 1 --2- 172.21.15.94:0/3893831990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb24161580 0x7ffb24147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.481+0000 7ffb2b800640 1 --2- 172.21.15.94:0/3893831990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb240ce970 0x7ffb2414f700 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.481+0000 7ffb2b800640 1 --2- 172.21.15.94:0/3893831990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb240b4f90 0x7ffb2415e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.481+0000 7ffb2b800640 1 -- 172.21.15.94:0/3893831990 >> 172.21.15.94:0/3893831990 conn(0x7ffb240d3eb0 msgr2=0x7ffb240d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.481+0000 7ffb2b800640 1 -- 172.21.15.94:0/3893831990 shutdown_connections 2024-09-10T23:37:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.481+0000 7ffb2b800640 1 -- 172.21.15.94:0/3893831990 wait complete. 2024-09-10T23:37:49.502 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone47961 2024-09-10T23:37:49.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.645+0000 7f3e29400640 1 -- 172.21.15.94:0/3797794436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e240b2ac0 msgr2=0x7f3e240b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:49.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.645+0000 7f3e29400640 1 --2- 172.21.15.94:0/3797794436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e240b2ac0 0x7f3e240b16b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009bc0 tx=0x7f3e0c01cb10 comp rx=0 tx=0).stop 2024-09-10T23:37:49.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.645+0000 7f3e29400640 1 -- 172.21.15.94:0/3797794436 shutdown_connections 2024-09-10T23:37:49.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.645+0000 7f3e29400640 1 --2- 172.21.15.94:0/3797794436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e240b2ac0 0x7f3e240b16b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:49.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.645+0000 7f3e29400640 1 --2- 172.21.15.94:0/3797794436 >> v2:172.21.15.181:3300/0 conn(0x7f3e2415f140 0x7f3e240b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:49.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.645+0000 7f3e29400640 1 -- 172.21.15.94:0/3797794436 >> 172.21.15.94:0/3797794436 conn(0x7f3e240d1b00 msgr2=0x7f3e240d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:49.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.645+0000 7f3e29400640 1 -- 172.21.15.94:0/3797794436 shutdown_connections 2024-09-10T23:37:49.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.645+0000 7f3e29400640 1 -- 172.21.15.94:0/3797794436 wait complete. 2024-09-10T23:37:49.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e29400640 1 Processor -- start 2024-09-10T23:37:49.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e29400640 1 -- start start 2024-09-10T23:37:49.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e29400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e240b1bf0 0x7f3e2415b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e29400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e240b2ac0 0x7f3e2415b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e29400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e2415f140 0x7f3e240c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e29400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3e24168ad0 con 0x7f3e2415f140 2024-09-10T23:37:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e29400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3e24168950 con 0x7f3e240b2ac0 2024-09-10T23:37:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e29400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3e24168c50 con 0x7f3e240b1bf0 2024-09-10T23:37:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e28a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e2415f140 0x7f3e240c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e28a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e2415f140 0x7f3e240c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55624/0 (socket says 172.21.15.94:55624) 2024-09-10T23:37:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e28a00640 1 -- 172.21.15.94:0/284812079 learned_addr learned my addr 172.21.15.94:0/284812079 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e23e00640 1 --2- 172.21.15.94:0/284812079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e240b1bf0 0x7f3e2415b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e1b400640 1 --2- 172.21.15.94:0/284812079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e240b2ac0 0x7f3e2415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e28a00640 1 -- 172.21.15.94:0/284812079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e240b1bf0 msgr2=0x7f3e2415b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e28a00640 1 --2- 172.21.15.94:0/284812079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e240b1bf0 0x7f3e2415b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e28a00640 1 -- 172.21.15.94:0/284812079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e240b2ac0 msgr2=0x7f3e2415b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e28a00640 1 --2- 172.21.15.94:0/284812079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e240b2ac0 0x7f3e2415b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e28a00640 1 -- 172.21.15.94:0/284812079 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e240cea40 con 0x7f3e2415f140 2024-09-10T23:37:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e28a00640 1 --2- 172.21.15.94:0/284812079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e2415f140 0x7f3e240c7810 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00c3f0 tx=0x7f3e1c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e20c00640 1 -- 172.21.15.94:0/284812079 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c017070 con 0x7f3e2415f140 2024-09-10T23:37:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e20c00640 1 -- 172.21.15.94:0/284812079 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e1c003600 con 0x7f3e2415f140 2024-09-10T23:37:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e20c00640 1 -- 172.21.15.94:0/284812079 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c003a50 con 0x7f3e2415f140 2024-09-10T23:37:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e29400640 1 -- 172.21.15.94:0/284812079 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e240ced20 con 0x7f3e2415f140 2024-09-10T23:37:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.649+0000 7f3e29400640 1 -- 172.21.15.94:0/284812079 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e240cf2e0 con 0x7f3e2415f140 2024-09-10T23:37:49.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.653+0000 7f3e29400640 1 -- 172.21.15.94:0/284812079 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e240cfa00 con 0x7f3e2415f140 2024-09-10T23:37:49.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.653+0000 7f3e20c00640 1 -- 172.21.15.94:0/284812079 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 96700+0+0 (secure 0 0 0) 0x7f3e1c003bf0 con 0x7f3e2415f140 2024-09-10T23:37:49.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.653+0000 7f3e20c00640 1 --2- 172.21.15.94:0/284812079 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3e00072970 0x7f3e00074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:49.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.653+0000 7f3e20c00640 1 -- 172.21.15.94:0/284812079 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7f3e1c092ed0 con 0x7f3e2415f140 2024-09-10T23:37:49.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.653+0000 7f3e23e00640 1 --2- 172.21.15.94:0/284812079 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3e00072970 0x7f3e00074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:49.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.653+0000 7f3e23e00640 1 --2- 172.21.15.94:0/284812079 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3e00072970 0x7f3e00074e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009ad0 tx=0x7f3e140091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:49.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.657+0000 7f3e20c00640 1 -- 172.21.15.94:0/284812079 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3e1c05dff0 con 0x7f3e2415f140 2024-09-10T23:37:49.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:49.857+0000 7f3e29400640 1 -- 172.21.15.94:0/284812079 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone47961", "target": ["mon-mgr", ""]}) -- 0x7f3e24159a60 con 0x7f3e00072970 2024-09-10T23:37:52.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:52.089+0000 7f3e20c00640 1 -- 172.21.15.94:0/284812079 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+153 (secure 0 0 0) 0x7f3e24159a60 con 0x7f3e00072970 2024-09-10T23:37:52.091 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:37:52.091 INFO:teuthology.orchestra.run.smithi094.stdout: "status": { 2024-09-10T23:37:52.091 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "pending", 2024-09-10T23:37:52.091 INFO:teuthology.orchestra.run.smithi094.stdout: "source": { 2024-09-10T23:37:52.091 INFO:teuthology.orchestra.run.smithi094.stdout: "volume": "cephfs", 2024-09-10T23:37:52.091 INFO:teuthology.orchestra.run.smithi094.stdout: "subvolume": "subvol55860", 2024-09-10T23:37:52.091 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot": "fake" 2024-09-10T23:37:52.091 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:52.091 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:52.092 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:37:52.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:52.089+0000 7f3e29400640 1 -- 172.21.15.94:0/284812079 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3e00072970 msgr2=0x7f3e00074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:52.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:52.089+0000 7f3e29400640 1 --2- 172.21.15.94:0/284812079 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3e00072970 0x7f3e00074e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009ad0 tx=0x7f3e140091e0 comp rx=0 tx=0).stop 2024-09-10T23:37:52.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:52.089+0000 7f3e29400640 1 -- 172.21.15.94:0/284812079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e2415f140 msgr2=0x7f3e240c7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:52.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:52.089+0000 7f3e29400640 1 --2- 172.21.15.94:0/284812079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e2415f140 0x7f3e240c7810 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00c3f0 tx=0x7f3e1c00c8b0 comp rx=0 tx=0).stop 2024-09-10T23:37:52.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:52.089+0000 7f3e29400640 1 -- 172.21.15.94:0/284812079 shutdown_connections 2024-09-10T23:37:52.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:52.089+0000 7f3e29400640 1 --2- 172.21.15.94:0/284812079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e240b2ac0 0x7f3e2415b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:52.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:52.089+0000 7f3e29400640 1 --2- 172.21.15.94:0/284812079 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3e00072970 0x7f3e00074e30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:52.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:52.089+0000 7f3e29400640 1 --2- 172.21.15.94:0/284812079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e2415f140 0x7f3e240c7810 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:52.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:52.089+0000 7f3e29400640 1 --2- 172.21.15.94:0/284812079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e240b1bf0 0x7f3e2415b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:52.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:52.089+0000 7f3e29400640 1 -- 172.21.15.94:0/284812079 >> 172.21.15.94:0/284812079 conn(0x7f3e240d1b00 msgr2=0x7f3e240af590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:52.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:52.093+0000 7f3e29400640 1 -- 172.21.15.94:0/284812079 shutdown_connections 2024-09-10T23:37:52.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:52.093+0000 7f3e29400640 1 -- 172.21.15.94:0/284812079 wait complete. 2024-09-10T23:37:52.119 DEBUG:tasks.cephfs.test_volumes:current clone state = pending 2024-09-10T23:37:53.119 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone47961 2024-09-10T23:37:53.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.273+0000 7fe7a0a00640 1 -- 172.21.15.94:0/1966167982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe79c06fc00 msgr2=0x7fe79c0c49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:53.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.273+0000 7fe7a0a00640 1 --2- 172.21.15.94:0/1966167982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe79c06fc00 0x7fe79c0c49b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe784009900 tx=0x7fe78401c8e0 comp rx=0 tx=0).stop 2024-09-10T23:37:53.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.273+0000 7fe7a0a00640 1 -- 172.21.15.94:0/1966167982 shutdown_connections 2024-09-10T23:37:53.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.273+0000 7fe7a0a00640 1 --2- 172.21.15.94:0/1966167982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe79c0cb4d0 0x7fe79c15b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:53.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.273+0000 7fe7a0a00640 1 --2- 172.21.15.94:0/1966167982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe79c06fc00 0x7fe79c0c49b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.273+0000 7fe7a0a00640 1 -- 172.21.15.94:0/1966167982 >> 172.21.15.94:0/1966167982 conn(0x7fe79c0cfaf0 msgr2=0x7fe79c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.273+0000 7fe7a0a00640 1 -- 172.21.15.94:0/1966167982 shutdown_connections 2024-09-10T23:37:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.273+0000 7fe7a0a00640 1 -- 172.21.15.94:0/1966167982 wait complete. 2024-09-10T23:37:53.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe7a0a00640 1 Processor -- start 2024-09-10T23:37:53.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe7a0a00640 1 -- start start 2024-09-10T23:37:53.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe7a0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe79c06fc00 0x7fe79c151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:53.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe7a0a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe79c0cb4d0 0x7fe79c15ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:53.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe7a0a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe79c158790 0x7fe79c0ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:53.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe7a0a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1680c0 con 0x7fe79c06fc00 2024-09-10T23:37:53.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe7a0a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c167f40 con 0x7fe79c0cb4d0 2024-09-10T23:37:53.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe7a0a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe79c168240 con 0x7fe79c158790 2024-09-10T23:37:53.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe799e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe79c06fc00 0x7fe79c151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:53.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe799e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe79c06fc00 0x7fe79c151460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55630/0 (socket says 172.21.15.94:55630) 2024-09-10T23:37:53.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe799e00640 1 -- 172.21.15.94:0/2661657883 learned_addr learned my addr 172.21.15.94:0/2661657883 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:53.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe791400640 1 --2- 172.21.15.94:0/2661657883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe79c0cb4d0 0x7fe79c15ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:53.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe79a800640 1 --2- 172.21.15.94:0/2661657883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe79c158790 0x7fe79c0ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:53.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe799e00640 1 -- 172.21.15.94:0/2661657883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe79c158790 msgr2=0x7fe79c0ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:53.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe799e00640 1 --2- 172.21.15.94:0/2661657883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe79c158790 0x7fe79c0ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:53.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe799e00640 1 -- 172.21.15.94:0/2661657883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe79c0cb4d0 msgr2=0x7fe79c15ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:53.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe799e00640 1 --2- 172.21.15.94:0/2661657883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe79c0cb4d0 0x7fe79c15ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:53.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe799e00640 1 -- 172.21.15.94:0/2661657883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c0ae800 con 0x7fe79c06fc00 2024-09-10T23:37:53.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe799e00640 1 --2- 172.21.15.94:0/2661657883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe79c06fc00 0x7fe79c151460 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00c3c0 tx=0x7fe78c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:53.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe792a00640 1 -- 172.21.15.94:0/2661657883 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0052d0 con 0x7fe79c06fc00 2024-09-10T23:37:53.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe792a00640 1 -- 172.21.15.94:0/2661657883 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe78c007020 con 0x7fe79c06fc00 2024-09-10T23:37:53.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe792a00640 1 -- 172.21.15.94:0/2661657883 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c007470 con 0x7fe79c06fc00 2024-09-10T23:37:53.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe7a0a00640 1 -- 172.21.15.94:0/2661657883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79c15f0b0 con 0x7fe79c06fc00 2024-09-10T23:37:53.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe7a0a00640 1 -- 172.21.15.94:0/2661657883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79c0aef80 con 0x7fe79c06fc00 2024-09-10T23:37:53.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.277+0000 7fe7a0a00640 1 -- 172.21.15.94:0/2661657883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79c0c49b0 con 0x7fe79c06fc00 2024-09-10T23:37:53.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.281+0000 7fe792a00640 1 -- 172.21.15.94:0/2661657883 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 96700+0+0 (secure 0 0 0) 0x7fe78c0067e0 con 0x7fe79c06fc00 2024-09-10T23:37:53.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.281+0000 7fe792a00640 1 --2- 172.21.15.94:0/2661657883 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe778072940 0x7fe778074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:53.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.281+0000 7fe792a00640 1 -- 172.21.15.94:0/2661657883 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7fe78c092cd0 con 0x7fe79c06fc00 2024-09-10T23:37:53.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.281+0000 7fe791400640 1 --2- 172.21.15.94:0/2661657883 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe778072940 0x7fe778074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:53.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.281+0000 7fe791400640 1 --2- 172.21.15.94:0/2661657883 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe778072940 0x7fe778074e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe7840af000 tx=0x7fe78401b120 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:53.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.285+0000 7fe792a00640 1 -- 172.21.15.94:0/2661657883 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe78c001090 con 0x7fe79c06fc00 2024-09-10T23:37:53.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:53.485+0000 7fe7a0a00640 1 -- 172.21.15.94:0/2661657883 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone47961", "target": ["mon-mgr", ""]}) -- 0x7fe79c0d0850 con 0x7fe778072940 2024-09-10T23:37:57.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.073+0000 7fe792a00640 1 -- 172.21.15.94:0/2661657883 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fe79c0d0850 con 0x7fe778072940 2024-09-10T23:37:57.074 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:37:57.074 INFO:teuthology.orchestra.run.smithi094.stdout: "status": { 2024-09-10T23:37:57.074 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "complete" 2024-09-10T23:37:57.074 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:57.074 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:37:57.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.073+0000 7fe7a0a00640 1 -- 172.21.15.94:0/2661657883 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe778072940 msgr2=0x7fe778074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:57.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.073+0000 7fe7a0a00640 1 --2- 172.21.15.94:0/2661657883 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe778072940 0x7fe778074e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe7840af000 tx=0x7fe78401b120 comp rx=0 tx=0).stop 2024-09-10T23:37:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.073+0000 7fe7a0a00640 1 -- 172.21.15.94:0/2661657883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe79c06fc00 msgr2=0x7fe79c151460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.073+0000 7fe7a0a00640 1 --2- 172.21.15.94:0/2661657883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe79c06fc00 0x7fe79c151460 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00c3c0 tx=0x7fe78c00c880 comp rx=0 tx=0).stop 2024-09-10T23:37:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.073+0000 7fe7a0a00640 1 -- 172.21.15.94:0/2661657883 shutdown_connections 2024-09-10T23:37:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.073+0000 7fe7a0a00640 1 --2- 172.21.15.94:0/2661657883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe79c0cb4d0 0x7fe79c15ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.073+0000 7fe7a0a00640 1 --2- 172.21.15.94:0/2661657883 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe778072940 0x7fe778074e00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.073+0000 7fe7a0a00640 1 --2- 172.21.15.94:0/2661657883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe79c158790 0x7fe79c0ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.073+0000 7fe7a0a00640 1 --2- 172.21.15.94:0/2661657883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe79c06fc00 0x7fe79c151460 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.073+0000 7fe7a0a00640 1 -- 172.21.15.94:0/2661657883 >> 172.21.15.94:0/2661657883 conn(0x7fe79c0cfaf0 msgr2=0x7fe79c1571f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.073+0000 7fe7a0a00640 1 -- 172.21.15.94:0/2661657883 shutdown_connections 2024-09-10T23:37:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.073+0000 7fe7a0a00640 1 -- 172.21.15.94:0/2661657883 wait complete. 2024-09-10T23:37:57.099 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-10T23:37:57.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:57.100 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone47961/.meta) 2024-09-10T23:37:57.125 INFO:teuthology.orchestra.run.smithi094.stdout:[GLOBAL] 2024-09-10T23:37:57.125 INFO:teuthology.orchestra.run.smithi094.stdout:version = 2 2024-09-10T23:37:57.125 INFO:teuthology.orchestra.run.smithi094.stdout:type = clone 2024-09-10T23:37:57.125 INFO:teuthology.orchestra.run.smithi094.stdout:path = /volumes/_nogroup/subvol_clone47961/e042b62e-3b2e-4dee-8dd3-6a74631841cd 2024-09-10T23:37:57.125 INFO:teuthology.orchestra.run.smithi094.stdout:state = complete 2024-09-10T23:37:57.125 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:37:57.128 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol55860 subvol_snap51870 2024-09-10T23:37:57.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 -- 172.21.15.94:0/896088600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa415ba30 msgr2=0x7f3fa40b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:57.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 --2- 172.21.15.94:0/896088600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa415ba30 0x7f3fa40b1bd0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40c5320 tx=0x7f3f9401c5d0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 -- 172.21.15.94:0/896088600 shutdown_connections 2024-09-10T23:37:57.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 --2- 172.21.15.94:0/896088600 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fa40d02e0 0x7f3fa415f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 --2- 172.21.15.94:0/896088600 >> v2:172.21.15.181:3300/0 conn(0x7f3fa40b2110 0x7f3fa40cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 --2- 172.21.15.94:0/896088600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa415ba30 0x7f3fa40b1bd0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 -- 172.21.15.94:0/896088600 >> 172.21.15.94:0/896088600 conn(0x7f3fa409aa70 msgr2=0x7f3fa409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 -- 172.21.15.94:0/896088600 shutdown_connections 2024-09-10T23:37:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 -- 172.21.15.94:0/896088600 wait complete. 2024-09-10T23:37:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 Processor -- start 2024-09-10T23:37:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 -- start start 2024-09-10T23:37:57.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fa40b2110 0x7f3fa40b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:57.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fa40d02e0 0x7f3fa4153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:57.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa415ba30 0x7f3fa415cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa40af2d0 con 0x7f3fa415ba30 2024-09-10T23:37:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa40af150 con 0x7f3fa40b2110 2024-09-10T23:37:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa40af450 con 0x7f3fa40d02e0 2024-09-10T23:37:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa415ba30 0x7f3fa415cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa415ba30 0x7f3fa415cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38458/0 (socket says 172.21.15.94:38458) 2024-09-10T23:37:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fa40b2110 0x7f3fa40b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3fa3400640 1 -- 172.21.15.94:0/4261072327 learned_addr learned my addr 172.21.15.94:0/4261072327 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.321+0000 7f3f9a000640 1 --2- 172.21.15.94:0/4261072327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fa40d02e0 0x7f3fa4153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.325+0000 7f3fa3400640 1 -- 172.21.15.94:0/4261072327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fa40d02e0 msgr2=0x7f3fa4153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.325+0000 7f3fa3400640 1 --2- 172.21.15.94:0/4261072327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fa40d02e0 0x7f3fa4153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.325+0000 7f3fa3400640 1 -- 172.21.15.94:0/4261072327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fa40b2110 msgr2=0x7f3fa40b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.325+0000 7f3fa3400640 1 --2- 172.21.15.94:0/4261072327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fa40b2110 0x7f3fa40b4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.325+0000 7f3fa3400640 1 -- 172.21.15.94:0/4261072327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa40b5030 con 0x7f3fa415ba30 2024-09-10T23:37:57.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.325+0000 7f3fa3400640 1 --2- 172.21.15.94:0/4261072327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa415ba30 0x7f3fa415cf60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00ea20 tx=0x7f3f9c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:57.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.325+0000 7f3f9b400640 1 -- 172.21.15.94:0/4261072327 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c00cde0 con 0x7f3fa415ba30 2024-09-10T23:37:57.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.325+0000 7f3f9b400640 1 -- 172.21.15.94:0/4261072327 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f9c004c40 con 0x7f3fa415ba30 2024-09-10T23:37:57.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.325+0000 7f3fa3e00640 1 -- 172.21.15.94:0/4261072327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa415c1e0 con 0x7f3fa415ba30 2024-09-10T23:37:57.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.325+0000 7f3f9b400640 1 -- 172.21.15.94:0/4261072327 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c005280 con 0x7f3fa415ba30 2024-09-10T23:37:57.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.325+0000 7f3fa3e00640 1 -- 172.21.15.94:0/4261072327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa415c760 con 0x7f3fa415ba30 2024-09-10T23:37:57.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.325+0000 7f3fa3e00640 1 -- 172.21.15.94:0/4261072327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa415f540 con 0x7f3fa415ba30 2024-09-10T23:37:57.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.325+0000 7f3f9b400640 1 -- 172.21.15.94:0/4261072327 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 96700+0+0 (secure 0 0 0) 0x7f3f9c00f040 con 0x7f3fa415ba30 2024-09-10T23:37:57.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.325+0000 7f3f9b400640 1 --2- 172.21.15.94:0/4261072327 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3f80072940 0x7f3f80074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:57.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.325+0000 7f3f9b400640 1 -- 172.21.15.94:0/4261072327 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7f3f9c094080 con 0x7f3fa415ba30 2024-09-10T23:37:57.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.325+0000 7f3fa2a00640 1 --2- 172.21.15.94:0/4261072327 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3f80072940 0x7f3f80074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:57.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.329+0000 7f3fa2a00640 1 --2- 172.21.15.94:0/4261072327 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3f80072940 0x7f3f80074e00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3f940ae000 tx=0x7f3f9401ae10 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:57.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.329+0000 7f3f9b400640 1 -- 172.21.15.94:0/4261072327 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3f9c016030 con 0x7f3fa415ba30 2024-09-10T23:37:57.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.529+0000 7f3fa3e00640 1 -- 172.21.15.94:0/4261072327 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol55860", "snap_name": "subvol_snap51870", "target": ["mon-mgr", ""]}) -- 0x7f3fa4157b80 con 0x7f3f80072940 2024-09-10T23:37:57.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.545+0000 7f3f9b400640 1 -- 172.21.15.94:0/4261072327 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+115 (secure 0 0 0) 0x7f3fa4157b80 con 0x7f3f80072940 2024-09-10T23:37:57.547 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:37:57.548 INFO:teuthology.orchestra.run.smithi094.stdout: "created_at": "2024-09-10 23:37:41.952176", 2024-09-10T23:37:57.548 INFO:teuthology.orchestra.run.smithi094.stdout: "data_pool": "cephfs_data", 2024-09-10T23:37:57.548 INFO:teuthology.orchestra.run.smithi094.stdout: "has_pending_clones": "no" 2024-09-10T23:37:57.548 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:37:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.549+0000 7f3fa3e00640 1 -- 172.21.15.94:0/4261072327 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3f80072940 msgr2=0x7f3f80074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.549+0000 7f3fa3e00640 1 --2- 172.21.15.94:0/4261072327 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3f80072940 0x7f3f80074e00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3f940ae000 tx=0x7f3f9401ae10 comp rx=0 tx=0).stop 2024-09-10T23:37:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.549+0000 7f3fa3e00640 1 -- 172.21.15.94:0/4261072327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa415ba30 msgr2=0x7f3fa415cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.549+0000 7f3fa3e00640 1 --2- 172.21.15.94:0/4261072327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa415ba30 0x7f3fa415cf60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00ea20 tx=0x7f3f9c00eee0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.549+0000 7f3fa3e00640 1 -- 172.21.15.94:0/4261072327 shutdown_connections 2024-09-10T23:37:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.549+0000 7f3fa3e00640 1 --2- 172.21.15.94:0/4261072327 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3f80072940 0x7f3f80074e00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.549+0000 7f3fa3e00640 1 --2- 172.21.15.94:0/4261072327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fa415ba30 0x7f3fa415cf60 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.549+0000 7f3fa3e00640 1 --2- 172.21.15.94:0/4261072327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fa40d02e0 0x7f3fa4153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.549+0000 7f3fa3e00640 1 --2- 172.21.15.94:0/4261072327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fa40b2110 0x7f3fa40b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.549+0000 7f3fa3e00640 1 -- 172.21.15.94:0/4261072327 >> 172.21.15.94:0/4261072327 conn(0x7f3fa409aa70 msgr2=0x7f3fa40cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:57.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.549+0000 7f3fa3e00640 1 -- 172.21.15.94:0/4261072327 shutdown_connections 2024-09-10T23:37:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.549+0000 7f3fa3e00640 1 -- 172.21.15.94:0/4261072327 wait complete. 2024-09-10T23:37:57.569 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs subvol55860 2024-09-10T23:37:57.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.713+0000 7f2433200640 1 -- 172.21.15.94:0/4101729908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f242c15ba30 msgr2=0x7f242c0b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:57.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.713+0000 7f2433200640 1 --2- 172.21.15.94:0/4101729908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f242c15ba30 0x7f242c0b1bd0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f242c0c5320 tx=0x7f242001c890 comp rx=0 tx=0).stop 2024-09-10T23:37:57.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.713+0000 7f2433200640 1 -- 172.21.15.94:0/4101729908 shutdown_connections 2024-09-10T23:37:57.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.713+0000 7f2433200640 1 --2- 172.21.15.94:0/4101729908 >> v2:172.21.15.181:3300/0 conn(0x7f242c0b2110 0x7f242c0cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.713+0000 7f2433200640 1 --2- 172.21.15.94:0/4101729908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f242c15ba30 0x7f242c0b1bd0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.713+0000 7f2433200640 1 -- 172.21.15.94:0/4101729908 >> 172.21.15.94:0/4101729908 conn(0x7f242c09aa70 msgr2=0x7f242c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:57.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.713+0000 7f2433200640 1 -- 172.21.15.94:0/4101729908 shutdown_connections 2024-09-10T23:37:57.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.713+0000 7f2433200640 1 -- 172.21.15.94:0/4101729908 wait complete. 2024-09-10T23:37:57.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.713+0000 7f2433200640 1 Processor -- start 2024-09-10T23:37:57.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.713+0000 7f2433200640 1 -- start start 2024-09-10T23:37:57.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.713+0000 7f2433200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f242c0b2110 0x7f242c0c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:57.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.713+0000 7f2433200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f242c0d02e0 0x7f242c14ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:57.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.713+0000 7f2433200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f242c15ba30 0x7f242c0b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:57.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.713+0000 7f2433200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f242c168b50 con 0x7f242c0d02e0 2024-09-10T23:37:57.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.713+0000 7f2433200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f242c1689d0 con 0x7f242c15ba30 2024-09-10T23:37:57.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.713+0000 7f2433200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f242c168cd0 con 0x7f242c0b2110 2024-09-10T23:37:57.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2431400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f242c0d02e0 0x7f242c14ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:57.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2431400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f242c0d02e0 0x7f242c14ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38474/0 (socket says 172.21.15.94:38474) 2024-09-10T23:37:57.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2431400640 1 -- 172.21.15.94:0/3535834884 learned_addr learned my addr 172.21.15.94:0/3535834884 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:57.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2431e00640 1 --2- 172.21.15.94:0/3535834884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f242c0b2110 0x7f242c0c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2432800640 1 --2- 172.21.15.94:0/3535834884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f242c15ba30 0x7f242c0b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2431400640 1 -- 172.21.15.94:0/3535834884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f242c0b2110 msgr2=0x7f242c0c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2431400640 1 --2- 172.21.15.94:0/3535834884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f242c0b2110 0x7f242c0c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2431400640 1 -- 172.21.15.94:0/3535834884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f242c15ba30 msgr2=0x7f242c0b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2431400640 1 --2- 172.21.15.94:0/3535834884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f242c15ba30 0x7f242c0b4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2431400640 1 -- 172.21.15.94:0/3535834884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c0b5030 con 0x7f242c0d02e0 2024-09-10T23:37:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2431400640 1 --2- 172.21.15.94:0/3535834884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f242c0d02e0 0x7f242c14ecb0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f241c00c9a0 tx=0x7f241c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:57.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2426000640 1 -- 172.21.15.94:0/3535834884 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c007c70 con 0x7f242c0d02e0 2024-09-10T23:37:57.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2426000640 1 -- 172.21.15.94:0/3535834884 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f241c00f450 con 0x7f242c0d02e0 2024-09-10T23:37:57.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2433200640 1 -- 172.21.15.94:0/3535834884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242c0c7ef0 con 0x7f242c0d02e0 2024-09-10T23:37:57.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2426000640 1 -- 172.21.15.94:0/3535834884 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c00f8a0 con 0x7f242c0d02e0 2024-09-10T23:37:57.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2433200640 1 -- 172.21.15.94:0/3535834884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242c0cbc20 con 0x7f242c0d02e0 2024-09-10T23:37:57.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2433200640 1 -- 172.21.15.94:0/3535834884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f242c0cfda0 con 0x7f242c0d02e0 2024-09-10T23:37:57.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.717+0000 7f2426000640 1 -- 172.21.15.94:0/3535834884 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 96700+0+0 (secure 0 0 0) 0x7f241c020030 con 0x7f242c0d02e0 2024-09-10T23:37:57.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.721+0000 7f2426000640 1 --2- 172.21.15.94:0/3535834884 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2410072940 0x7f2410074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:57.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.721+0000 7f2426000640 1 -- 172.21.15.94:0/3535834884 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7f241c093f70 con 0x7f242c0d02e0 2024-09-10T23:37:57.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.721+0000 7f2431e00640 1 --2- 172.21.15.94:0/3535834884 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2410072940 0x7f2410074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:57.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.721+0000 7f2431e00640 1 --2- 172.21.15.94:0/3535834884 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2410072940 0x7f2410074e00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f24200af000 tx=0x7f242001b0d0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:57.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.725+0000 7f2426000640 1 -- 172.21.15.94:0/3535834884 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f241c014030 con 0x7f242c0d02e0 2024-09-10T23:37:57.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.917+0000 7f2433200640 1 -- 172.21.15.94:0/3535834884 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "subvol55860", "target": ["mon-mgr", ""]}) -- 0x7f242c076110 con 0x7f2410072940 2024-09-10T23:37:57.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.937+0000 7f2426000640 1 -- 172.21.15.94:0/3535834884 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+87 (secure 0 0 0) 0x7f242c076110 con 0x7f2410072940 2024-09-10T23:37:57.940 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:37:57.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:57.940 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fake" 2024-09-10T23:37:57.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:37:57.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:37:57.941 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subvol_snap51870" 2024-09-10T23:37:57.941 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:37:57.941 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:37:57.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.941+0000 7f2433200640 1 -- 172.21.15.94:0/3535834884 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2410072940 msgr2=0x7f2410074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:57.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.941+0000 7f2433200640 1 --2- 172.21.15.94:0/3535834884 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2410072940 0x7f2410074e00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f24200af000 tx=0x7f242001b0d0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.941+0000 7f2433200640 1 -- 172.21.15.94:0/3535834884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f242c0d02e0 msgr2=0x7f242c14ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:57.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.941+0000 7f2433200640 1 --2- 172.21.15.94:0/3535834884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f242c0d02e0 0x7f242c14ecb0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f241c00c9a0 tx=0x7f241c00ce60 comp rx=0 tx=0).stop 2024-09-10T23:37:57.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.941+0000 7f2433200640 1 -- 172.21.15.94:0/3535834884 shutdown_connections 2024-09-10T23:37:57.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.941+0000 7f2433200640 1 --2- 172.21.15.94:0/3535834884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f242c15ba30 0x7f242c0b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.941+0000 7f2433200640 1 --2- 172.21.15.94:0/3535834884 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2410072940 0x7f2410074e00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.941+0000 7f2433200640 1 --2- 172.21.15.94:0/3535834884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f242c0d02e0 0x7f242c14ecb0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.941+0000 7f2433200640 1 --2- 172.21.15.94:0/3535834884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f242c0b2110 0x7f242c0c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:57.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.941+0000 7f2433200640 1 -- 172.21.15.94:0/3535834884 >> 172.21.15.94:0/3535834884 conn(0x7f242c09aa70 msgr2=0x7f242c0aeda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:57.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.941+0000 7f2433200640 1 -- 172.21.15.94:0/3535834884 shutdown_connections 2024-09-10T23:37:57.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:57.941+0000 7f2433200640 1 -- 172.21.15.94:0/3535834884 wait complete. 2024-09-10T23:37:57.961 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol55860 subvol_snap51870 2024-09-10T23:37:58.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.105+0000 7f2485200640 1 -- 172.21.15.94:0/1932266413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f248015de30 msgr2=0x7f24800b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:58.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.105+0000 7f2485200640 1 --2- 172.21.15.94:0/1932266413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f248015de30 0x7f24800b2fc0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2470009870 tx=0x7f247001c7f0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.105+0000 7f2485200640 1 -- 172.21.15.94:0/1932266413 shutdown_connections 2024-09-10T23:37:58.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.105+0000 7f2485200640 1 --2- 172.21.15.94:0/1932266413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24800d2020 0x7f24800cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.105+0000 7f2485200640 1 --2- 172.21.15.94:0/1932266413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f248015de30 0x7f24800b2fc0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.105+0000 7f2485200640 1 -- 172.21.15.94:0/1932266413 >> 172.21.15.94:0/1932266413 conn(0x7f248009aa70 msgr2=0x7f248009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:58.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.105+0000 7f2485200640 1 -- 172.21.15.94:0/1932266413 shutdown_connections 2024-09-10T23:37:58.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.105+0000 7f2485200640 1 -- 172.21.15.94:0/1932266413 wait complete. 2024-09-10T23:37:58.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f2485200640 1 Processor -- start 2024-09-10T23:37:58.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f2485200640 1 -- start start 2024-09-10T23:37:58.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f2485200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24800b25c0 0x7f24800b7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:58.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f2485200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24800d2020 0x7f2480152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:58.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f2485200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f248015de30 0x7f24800b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:58.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f2485200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f248016b140 con 0x7f248015de30 2024-09-10T23:37:58.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f2485200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f248016afc0 con 0x7f24800d2020 2024-09-10T23:37:58.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f2485200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f248016b2c0 con 0x7f24800b25c0 2024-09-10T23:37:58.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f247fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f248015de30 0x7f24800b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:58.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f247fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f248015de30 0x7f24800b4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38492/0 (socket says 172.21.15.94:38492) 2024-09-10T23:37:58.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f247fe00640 1 -- 172.21.15.94:0/4161591411 learned_addr learned my addr 172.21.15.94:0/4161591411 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:58.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f247f400640 1 --2- 172.21.15.94:0/4161591411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24800b25c0 0x7f24800b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:58.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f247fe00640 1 -- 172.21.15.94:0/4161591411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24800b25c0 msgr2=0x7f24800b7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:58.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f2477e00640 1 --2- 172.21.15.94:0/4161591411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24800d2020 0x7f2480152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:58.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f247fe00640 1 --2- 172.21.15.94:0/4161591411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24800b25c0 0x7f24800b7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f247fe00640 1 -- 172.21.15.94:0/4161591411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24800d2020 msgr2=0x7f2480152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f247fe00640 1 --2- 172.21.15.94:0/4161591411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24800d2020 0x7f2480152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f247fe00640 1 -- 172.21.15.94:0/4161591411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24801541d0 con 0x7f248015de30 2024-09-10T23:37:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f2477e00640 1 --2- 172.21.15.94:0/4161591411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24800d2020 0x7f2480152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:37:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f247fe00640 1 --2- 172.21.15.94:0/4161591411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f248015de30 0x7f24800b4d90 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f247800cce0 tx=0x7f247800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f2477400640 1 -- 172.21.15.94:0/4161591411 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247800f450 con 0x7f248015de30 2024-09-10T23:37:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f2477400640 1 -- 172.21.15.94:0/4161591411 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f247800f6e0 con 0x7f248015de30 2024-09-10T23:37:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f2485200640 1 -- 172.21.15.94:0/4161591411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24801544b0 con 0x7f248015de30 2024-09-10T23:37:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f2477400640 1 -- 172.21.15.94:0/4161591411 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247800fb30 con 0x7f248015de30 2024-09-10T23:37:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f2485200640 1 -- 172.21.15.94:0/4161591411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2480154a70 con 0x7f248015de30 2024-09-10T23:37:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.109+0000 7f2485200640 1 -- 172.21.15.94:0/4161591411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24800cfc70 con 0x7f248015de30 2024-09-10T23:37:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.113+0000 7f2477400640 1 -- 172.21.15.94:0/4161591411 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 96700+0+0 (secure 0 0 0) 0x7f247800fcd0 con 0x7f248015de30 2024-09-10T23:37:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.113+0000 7f2477400640 1 --2- 172.21.15.94:0/4161591411 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f245c072940 0x7f245c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.113+0000 7f2477400640 1 -- 172.21.15.94:0/4161591411 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7f2478094590 con 0x7f248015de30 2024-09-10T23:37:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.113+0000 7f247f400640 1 --2- 172.21.15.94:0/4161591411 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f245c072940 0x7f245c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.113+0000 7f247f400640 1 --2- 172.21.15.94:0/4161591411 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f245c072940 0x7f245c074e00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f24700af000 tx=0x7f247001b030 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.117+0000 7f2477400640 1 -- 172.21.15.94:0/4161591411 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2478014030 con 0x7f248015de30 2024-09-10T23:37:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.313+0000 7f2485200640 1 -- 172.21.15.94:0/4161591411 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol55860", "snap_name": "subvol_snap51870", "target": ["mon-mgr", ""]}) -- 0x7f248009b130 con 0x7f245c072940 2024-09-10T23:37:58.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.341+0000 7f2477400640 1 -- 172.21.15.94:0/4161591411 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f248009b130 con 0x7f245c072940 2024-09-10T23:37:58.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.345+0000 7f2485200640 1 -- 172.21.15.94:0/4161591411 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f245c072940 msgr2=0x7f245c074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:58.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.345+0000 7f2485200640 1 --2- 172.21.15.94:0/4161591411 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f245c072940 0x7f245c074e00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f24700af000 tx=0x7f247001b030 comp rx=0 tx=0).stop 2024-09-10T23:37:58.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.345+0000 7f2485200640 1 -- 172.21.15.94:0/4161591411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f248015de30 msgr2=0x7f24800b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:58.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.345+0000 7f2485200640 1 --2- 172.21.15.94:0/4161591411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f248015de30 0x7f24800b4d90 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f247800cce0 tx=0x7f247800a620 comp rx=0 tx=0).stop 2024-09-10T23:37:58.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.345+0000 7f2485200640 1 -- 172.21.15.94:0/4161591411 shutdown_connections 2024-09-10T23:37:58.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.345+0000 7f2485200640 1 --2- 172.21.15.94:0/4161591411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24800d2020 0x7f2480152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.345+0000 7f2485200640 1 --2- 172.21.15.94:0/4161591411 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f245c072940 0x7f245c074e00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.345+0000 7f2485200640 1 --2- 172.21.15.94:0/4161591411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f248015de30 0x7f24800b4d90 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.345+0000 7f2485200640 1 --2- 172.21.15.94:0/4161591411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24800b25c0 0x7f24800b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.345+0000 7f2485200640 1 -- 172.21.15.94:0/4161591411 >> 172.21.15.94:0/4161591411 conn(0x7f248009aa70 msgr2=0x7f24800707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:58.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.345+0000 7f2485200640 1 -- 172.21.15.94:0/4161591411 shutdown_connections 2024-09-10T23:37:58.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.345+0000 7f2485200640 1 -- 172.21.15.94:0/4161591411 wait complete. 2024-09-10T23:37:58.366 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol55860 fake 2024-09-10T23:37:58.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.509+0000 7fa717e00640 1 -- 172.21.15.94:0/313303600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7100d19b0 msgr2=0x7fa7100d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:58.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.509+0000 7fa717e00640 1 --2- 172.21.15.94:0/313303600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7100d19b0 0x7fa7100d0590 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc009840 tx=0x7fa6fc01f010 comp rx=0 tx=0).stop 2024-09-10T23:37:58.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.509+0000 7fa717e00640 1 -- 172.21.15.94:0/313303600 shutdown_connections 2024-09-10T23:37:58.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.509+0000 7fa717e00640 1 --2- 172.21.15.94:0/313303600 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7100d19b0 0x7fa7100d0590 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.509+0000 7fa717e00640 1 --2- 172.21.15.94:0/313303600 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa71015dec0 0x7fa7100cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.509+0000 7fa717e00640 1 -- 172.21.15.94:0/313303600 >> 172.21.15.94:0/313303600 conn(0x7fa7100d3de0 msgr2=0x7fa7100d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:58.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.509+0000 7fa717e00640 1 -- 172.21.15.94:0/313303600 shutdown_connections 2024-09-10T23:37:58.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.509+0000 7fa717e00640 1 -- 172.21.15.94:0/313303600 wait complete. 2024-09-10T23:37:58.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa717e00640 1 Processor -- start 2024-09-10T23:37:58.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa717e00640 1 -- start start 2024-09-10T23:37:58.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa717e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7100d19b0 0x7fa71015d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:58.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa717e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7100d4900 0x7fa7100c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:58.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa717e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa71015dec0 0x7fa7100ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:58.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa717e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa7100b74c0 con 0x7fa7100d19b0 2024-09-10T23:37:58.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa717e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa7100b7340 con 0x7fa71015dec0 2024-09-10T23:37:58.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa717e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa7100b7640 con 0x7fa7100d4900 2024-09-10T23:37:58.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa715400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7100d19b0 0x7fa71015d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:58.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa715400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7100d19b0 0x7fa71015d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38506/0 (socket says 172.21.15.94:38506) 2024-09-10T23:37:58.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa715400640 1 -- 172.21.15.94:0/95325301 learned_addr learned my addr 172.21.15.94:0/95325301 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:37:58.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa714a00640 1 --2- 172.21.15.94:0/95325301 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7100d4900 0x7fa7100c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:58.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa715e00640 1 --2- 172.21.15.94:0/95325301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa71015dec0 0x7fa7100ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:58.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa715400640 1 -- 172.21.15.94:0/95325301 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7100d4900 msgr2=0x7fa7100c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:58.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa715400640 1 --2- 172.21.15.94:0/95325301 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7100d4900 0x7fa7100c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa715400640 1 -- 172.21.15.94:0/95325301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa71015dec0 msgr2=0x7fa7100ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:58.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa715400640 1 --2- 172.21.15.94:0/95325301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa71015dec0 0x7fa7100ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa715400640 1 -- 172.21.15.94:0/95325301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71009e790 con 0x7fa7100d19b0 2024-09-10T23:37:58.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa715400640 1 --2- 172.21.15.94:0/95325301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7100d19b0 0x7fa71015d3e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fa70400e7a0 tx=0x7fa70400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:58.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa70d600640 1 -- 172.21.15.94:0/95325301 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70400ccc0 con 0x7fa7100d19b0 2024-09-10T23:37:58.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa70d600640 1 -- 172.21.15.94:0/95325301 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa704010430 con 0x7fa7100d19b0 2024-09-10T23:37:58.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa70d600640 1 -- 172.21.15.94:0/95325301 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704010880 con 0x7fa7100d19b0 2024-09-10T23:37:58.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa717e00640 1 -- 172.21.15.94:0/95325301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7100c79b0 con 0x7fa7100d19b0 2024-09-10T23:37:58.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.513+0000 7fa717e00640 1 -- 172.21.15.94:0/95325301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7100c7f70 con 0x7fa7100d19b0 2024-09-10T23:37:58.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.517+0000 7fa717e00640 1 -- 172.21.15.94:0/95325301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7100d0c90 con 0x7fa7100d19b0 2024-09-10T23:37:58.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.517+0000 7fa70d600640 1 -- 172.21.15.94:0/95325301 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 96700+0+0 (secure 0 0 0) 0x7fa704010a20 con 0x7fa7100d19b0 2024-09-10T23:37:58.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.517+0000 7fa70d600640 1 --2- 172.21.15.94:0/95325301 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa6f0072940 0x7fa6f0074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:37:58.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.517+0000 7fa70d600640 1 -- 172.21.15.94:0/95325301 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7fa704094400 con 0x7fa7100d19b0 2024-09-10T23:37:58.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.517+0000 7fa714a00640 1 --2- 172.21.15.94:0/95325301 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa6f0072940 0x7fa6f0074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:37:58.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.517+0000 7fa714a00640 1 --2- 172.21.15.94:0/95325301 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa6f0072940 0x7fa6f0074e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0b2000 tx=0x7fa6fc01d850 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:37:58.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.521+0000 7fa70d600640 1 -- 172.21.15.94:0/95325301 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa70405eed0 con 0x7fa7100d19b0 2024-09-10T23:37:58.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.721+0000 7fa717e00640 1 -- 172.21.15.94:0/95325301 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol55860", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7fa7100d1390 con 0x7fa6f0072940 2024-09-10T23:37:58.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.753+0000 7fa70d600640 1 -- 172.21.15.94:0/95325301 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa7100d1390 con 0x7fa6f0072940 2024-09-10T23:37:58.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.753+0000 7fa717e00640 1 -- 172.21.15.94:0/95325301 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa6f0072940 msgr2=0x7fa6f0074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:58.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.753+0000 7fa717e00640 1 --2- 172.21.15.94:0/95325301 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa6f0072940 0x7fa6f0074e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0b2000 tx=0x7fa6fc01d850 comp rx=0 tx=0).stop 2024-09-10T23:37:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.753+0000 7fa717e00640 1 -- 172.21.15.94:0/95325301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7100d19b0 msgr2=0x7fa71015d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:37:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.753+0000 7fa717e00640 1 --2- 172.21.15.94:0/95325301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7100d19b0 0x7fa71015d3e0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fa70400e7a0 tx=0x7fa70400ec60 comp rx=0 tx=0).stop 2024-09-10T23:37:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.753+0000 7fa717e00640 1 -- 172.21.15.94:0/95325301 shutdown_connections 2024-09-10T23:37:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.753+0000 7fa717e00640 1 --2- 172.21.15.94:0/95325301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa71015dec0 0x7fa7100ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.753+0000 7fa717e00640 1 --2- 172.21.15.94:0/95325301 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa6f0072940 0x7fa6f0074e00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.753+0000 7fa717e00640 1 --2- 172.21.15.94:0/95325301 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7100d4900 0x7fa7100c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.753+0000 7fa717e00640 1 --2- 172.21.15.94:0/95325301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7100d19b0 0x7fa71015d3e0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:37:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.753+0000 7fa717e00640 1 -- 172.21.15.94:0/95325301 >> 172.21.15.94:0/95325301 conn(0x7fa7100d3de0 msgr2=0x7fa7100d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:37:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.753+0000 7fa717e00640 1 -- 172.21.15.94:0/95325301 shutdown_connections 2024-09-10T23:37:58.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:37:58.753+0000 7fa717e00640 1 -- 172.21.15.94:0/95325301 wait complete. 2024-09-10T23:37:58.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:37:58.778 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol55860/.meta) 2024-09-10T23:38:02.069 INFO:teuthology.orchestra.run.smithi094.stdout:[GLOBAL] 2024-09-10T23:38:02.069 INFO:teuthology.orchestra.run.smithi094.stdout:version = 1 2024-09-10T23:38:02.069 INFO:teuthology.orchestra.run.smithi094.stdout:type = subvolume 2024-09-10T23:38:02.070 INFO:teuthology.orchestra.run.smithi094.stdout:path = /volumes/_nogroup/subvol55860/c3ce770d-1b60-4dd9-b33d-2a3112de0fad 2024-09-10T23:38:02.070 INFO:teuthology.orchestra.run.smithi094.stdout:state = complete 2024-09-10T23:38:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:02.073 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol55860 2024-09-10T23:38:02.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.225+0000 7f2e9b800640 1 -- 172.21.15.94:0/1240256685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e940c5a70 msgr2=0x7f2e940cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:02.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.225+0000 7f2e9b800640 1 --2- 172.21.15.94:0/1240256685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e940c5a70 0x7f2e940cd9f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009cb0 tx=0x7f2e8c01cb90 comp rx=0 tx=0).stop 2024-09-10T23:38:02.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.225+0000 7f2e9b800640 1 -- 172.21.15.94:0/1240256685 shutdown_connections 2024-09-10T23:38:02.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.225+0000 7f2e9b800640 1 --2- 172.21.15.94:0/1240256685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e940c5a70 0x7f2e940cd9f0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.225+0000 7f2e9b800640 1 --2- 172.21.15.94:0/1240256685 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e940d0710 0x7f2e940c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.225+0000 7f2e9b800640 1 -- 172.21.15.94:0/1240256685 >> 172.21.15.94:0/1240256685 conn(0x7f2e940cfaf0 msgr2=0x7f2e940cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:02.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.225+0000 7f2e9b800640 1 -- 172.21.15.94:0/1240256685 shutdown_connections 2024-09-10T23:38:02.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.225+0000 7f2e9b800640 1 -- 172.21.15.94:0/1240256685 wait complete. 2024-09-10T23:38:02.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.225+0000 7f2e9b800640 1 Processor -- start 2024-09-10T23:38:02.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.225+0000 7f2e9b800640 1 -- start start 2024-09-10T23:38:02.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e9b800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e940c5a70 0x7f2e94137130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:02.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e9b800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e940d0710 0x7f2e94158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:02.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e9b800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e941455d0 0x7f2e94153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:02.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e9b800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e941694f0 con 0x7f2e940c5a70 2024-09-10T23:38:02.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e9b800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2e94169370 con 0x7f2e940d0710 2024-09-10T23:38:02.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e9b800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2e94169670 con 0x7f2e941455d0 2024-09-10T23:38:02.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e98e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e940c5a70 0x7f2e94137130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:02.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e98e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e940c5a70 0x7f2e94137130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38532/0 (socket says 172.21.15.94:38532) 2024-09-10T23:38:02.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e98e00640 1 -- 172.21.15.94:0/3880831241 learned_addr learned my addr 172.21.15.94:0/3880831241 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:02.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e93e00640 1 --2- 172.21.15.94:0/3880831241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e940d0710 0x7f2e94158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:02.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e99800640 1 --2- 172.21.15.94:0/3880831241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e941455d0 0x7f2e94153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:02.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e98e00640 1 -- 172.21.15.94:0/3880831241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e941455d0 msgr2=0x7f2e94153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:02.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e98e00640 1 --2- 172.21.15.94:0/3880831241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e941455d0 0x7f2e94153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e98e00640 1 -- 172.21.15.94:0/3880831241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e940d0710 msgr2=0x7f2e94158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:02.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e98e00640 1 --2- 172.21.15.94:0/3880831241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e940d0710 0x7f2e94158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e98e00640 1 -- 172.21.15.94:0/3880831241 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e940b0520 con 0x7f2e940c5a70 2024-09-10T23:38:02.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e98e00640 1 --2- 172.21.15.94:0/3880831241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e940c5a70 0x7f2e94137130 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800b3e0 tx=0x7f2e8800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:02.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e90c00640 1 -- 172.21.15.94:0/3880831241 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88015020 con 0x7f2e940c5a70 2024-09-10T23:38:02.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e90c00640 1 -- 172.21.15.94:0/3880831241 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e88011940 con 0x7f2e940c5a70 2024-09-10T23:38:02.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e9b800640 1 -- 172.21.15.94:0/3880831241 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e940b0800 con 0x7f2e940c5a70 2024-09-10T23:38:02.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e90c00640 1 -- 172.21.15.94:0/3880831241 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88011d90 con 0x7f2e940c5a70 2024-09-10T23:38:02.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e9b800640 1 -- 172.21.15.94:0/3880831241 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e940b0dc0 con 0x7f2e940c5a70 2024-09-10T23:38:02.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e9b800640 1 -- 172.21.15.94:0/3880831241 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e940cd9f0 con 0x7f2e940c5a70 2024-09-10T23:38:02.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.229+0000 7f2e90c00640 1 -- 172.21.15.94:0/3880831241 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 96700+0+0 (secure 0 0 0) 0x7f2e8801a400 con 0x7f2e940c5a70 2024-09-10T23:38:02.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.233+0000 7f2e90c00640 1 --2- 172.21.15.94:0/3880831241 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2e74072940 0x7f2e74074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:02.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.233+0000 7f2e90c00640 1 -- 172.21.15.94:0/3880831241 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7f2e88093b50 con 0x7f2e940c5a70 2024-09-10T23:38:02.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.233+0000 7f2e93e00640 1 --2- 172.21.15.94:0/3880831241 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2e74072940 0x7f2e74074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:02.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.233+0000 7f2e93e00640 1 --2- 172.21.15.94:0/3880831241 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2e74072940 0x7f2e74074e00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2e80006fd0 tx=0x7f2e80008040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:02.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.237+0000 7f2e90c00640 1 -- 172.21.15.94:0/3880831241 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2e8801f020 con 0x7f2e940c5a70 2024-09-10T23:38:02.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.425+0000 7f2e9b800640 1 -- 172.21.15.94:0/3880831241 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol55860", "target": ["mon-mgr", ""]}) -- 0x7f2e940cf6e0 con 0x7f2e74072940 2024-09-10T23:38:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.465+0000 7f2e90c00640 1 -- 172.21.15.94:0/3880831241 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2e940cf6e0 con 0x7f2e74072940 2024-09-10T23:38:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f2e9b800640 1 -- 172.21.15.94:0/3880831241 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2e74072940 msgr2=0x7f2e74074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f2e9b800640 1 --2- 172.21.15.94:0/3880831241 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2e74072940 0x7f2e74074e00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2e80006fd0 tx=0x7f2e80008040 comp rx=0 tx=0).stop 2024-09-10T23:38:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f2e9b800640 1 -- 172.21.15.94:0/3880831241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e940c5a70 msgr2=0x7f2e94137130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f2e9b800640 1 --2- 172.21.15.94:0/3880831241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e940c5a70 0x7f2e94137130 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800b3e0 tx=0x7f2e8800b8a0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f2e9b800640 1 -- 172.21.15.94:0/3880831241 shutdown_connections 2024-09-10T23:38:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f2e9b800640 1 --2- 172.21.15.94:0/3880831241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e940d0710 0x7f2e94158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f2e9b800640 1 --2- 172.21.15.94:0/3880831241 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2e74072940 0x7f2e74074e00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f2e9b800640 1 --2- 172.21.15.94:0/3880831241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e941455d0 0x7f2e94153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f2e9b800640 1 --2- 172.21.15.94:0/3880831241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e940c5a70 0x7f2e94137130 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f2e9b800640 1 -- 172.21.15.94:0/3880831241 >> 172.21.15.94:0/3880831241 conn(0x7f2e940cfaf0 msgr2=0x7f2e94159b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:02.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f2e9b800640 1 -- 172.21.15.94:0/3880831241 shutdown_connections 2024-09-10T23:38:02.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f2e9b800640 1 -- 172.21.15.94:0/3880831241 wait complete. 2024-09-10T23:38:02.473 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:02.473 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:02.473 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:02.473 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:02.473 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:02.469+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:02.490 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:02.489+0000 7f9ea3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:02.490 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:02.489+0000 7f9ea3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:02.490 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:02.489+0000 7f9ea3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:02.490 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:02.489+0000 7f9ea3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:02.490 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:02.489+0000 7f9ea3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:02.494 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone39050 2024-09-10T23:38:02.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.641+0000 7f2d51800640 1 -- 172.21.15.94:0/160388489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d4c0b2960 msgr2=0x7f2d4c06e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:02.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.641+0000 7f2d51800640 1 --2- 172.21.15.94:0/160388489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d4c0b2960 0x7f2d4c06e270 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009900 tx=0x7f2d3401c8e0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.641+0000 7f2d51800640 1 -- 172.21.15.94:0/160388489 shutdown_connections 2024-09-10T23:38:02.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.641+0000 7f2d51800640 1 --2- 172.21.15.94:0/160388489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d4c0b2960 0x7f2d4c06e270 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.641+0000 7f2d51800640 1 --2- 172.21.15.94:0/160388489 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d4c161600 0x7f2d4c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.641+0000 7f2d51800640 1 -- 172.21.15.94:0/160388489 >> 172.21.15.94:0/160388489 conn(0x7f2d4c0d1c60 msgr2=0x7f2d4c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:02.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.641+0000 7f2d51800640 1 -- 172.21.15.94:0/160388489 shutdown_connections 2024-09-10T23:38:02.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.641+0000 7f2d51800640 1 -- 172.21.15.94:0/160388489 wait complete. 2024-09-10T23:38:02.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d51800640 1 Processor -- start 2024-09-10T23:38:02.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d51800640 1 -- start start 2024-09-10T23:38:02.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d51800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4c0b2960 0x7f2d4c1504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:02.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d51800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d4c0b5cf0 0x7f2d4c157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:02.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d51800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d4c161600 0x7f2d4c158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:02.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d51800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c16b360 con 0x7f2d4c0b5cf0 2024-09-10T23:38:02.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d51800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c16b1e0 con 0x7f2d4c0b2960 2024-09-10T23:38:02.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d51800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2d4c16b4e0 con 0x7f2d4c161600 2024-09-10T23:38:02.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d42000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d4c0b5cf0 0x7f2d4c157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:02.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d42000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d4c0b5cf0 0x7f2d4c157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38580/0 (socket says 172.21.15.94:38580) 2024-09-10T23:38:02.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d42000640 1 -- 172.21.15.94:0/1278957511 learned_addr learned my addr 172.21.15.94:0/1278957511 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:02.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d4b400640 1 --2- 172.21.15.94:0/1278957511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d4c161600 0x7f2d4c158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:02.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d4aa00640 1 --2- 172.21.15.94:0/1278957511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4c0b2960 0x7f2d4c1504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:02.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d42000640 1 -- 172.21.15.94:0/1278957511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d4c161600 msgr2=0x7f2d4c158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:02.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d42000640 1 --2- 172.21.15.94:0/1278957511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d4c161600 0x7f2d4c158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d42000640 1 -- 172.21.15.94:0/1278957511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4c0b2960 msgr2=0x7f2d4c1504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:02.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d42000640 1 --2- 172.21.15.94:0/1278957511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4c0b2960 0x7f2d4c1504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d42000640 1 -- 172.21.15.94:0/1278957511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4c0c6d70 con 0x7f2d4c0b5cf0 2024-09-10T23:38:02.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d42000640 1 --2- 172.21.15.94:0/1278957511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d4c0b5cf0 0x7f2d4c157730 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009850 tx=0x7f2d340a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:02.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d43400640 1 -- 172.21.15.94:0/1278957511 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d340b2040 con 0x7f2d4c0b5cf0 2024-09-10T23:38:02.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d43400640 1 -- 172.21.15.94:0/1278957511 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d34005070 con 0x7f2d4c0b5cf0 2024-09-10T23:38:02.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d43400640 1 -- 172.21.15.94:0/1278957511 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d340044f0 con 0x7f2d4c0b5cf0 2024-09-10T23:38:02.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d51800640 1 -- 172.21.15.94:0/1278957511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4c157c70 con 0x7f2d4c0b5cf0 2024-09-10T23:38:02.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.645+0000 7f2d51800640 1 -- 172.21.15.94:0/1278957511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d4c0c72d0 con 0x7f2d4c0b5cf0 2024-09-10T23:38:02.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.649+0000 7f2d51800640 1 -- 172.21.15.94:0/1278957511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d4c0b6160 con 0x7f2d4c0b5cf0 2024-09-10T23:38:02.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.649+0000 7f2d43400640 1 -- 172.21.15.94:0/1278957511 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 96700+0+0 (secure 0 0 0) 0x7f2d340054a0 con 0x7f2d4c0b5cf0 2024-09-10T23:38:02.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.649+0000 7f2d43400640 1 --2- 172.21.15.94:0/1278957511 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2d28072940 0x7f2d28074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:02.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.649+0000 7f2d43400640 1 -- 172.21.15.94:0/1278957511 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5796+0+0 (secure 0 0 0) 0x7f2d3412cf40 con 0x7f2d4c0b5cf0 2024-09-10T23:38:02.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.649+0000 7f2d4aa00640 1 --2- 172.21.15.94:0/1278957511 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2d28072940 0x7f2d28074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:02.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.653+0000 7f2d4aa00640 1 --2- 172.21.15.94:0/1278957511 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2d28072940 0x7f2d28074e00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009e60 tx=0x7f2d3c0095b0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.653+0000 7f2d43400640 1 -- 172.21.15.94:0/1278957511 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2d4c0c6d70 con 0x7f2d4c0b5cf0 2024-09-10T23:38:02.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.849+0000 7f2d51800640 1 -- 172.21.15.94:0/1278957511 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone39050", "target": ["mon-mgr", ""]}) -- 0x7f2d4c15baa0 con 0x7f2d28072940 2024-09-10T23:38:02.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.865+0000 7f2d43400640 1 -- 172.21.15.94:0/1278957511 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2d4c15baa0 con 0x7f2d28072940 2024-09-10T23:38:02.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.869+0000 7f2d51800640 1 -- 172.21.15.94:0/1278957511 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2d28072940 msgr2=0x7f2d28074e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:02.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.869+0000 7f2d51800640 1 --2- 172.21.15.94:0/1278957511 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2d28072940 0x7f2d28074e00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009e60 tx=0x7f2d3c0095b0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.869+0000 7f2d51800640 1 -- 172.21.15.94:0/1278957511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d4c0b5cf0 msgr2=0x7f2d4c157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:02.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.869+0000 7f2d51800640 1 --2- 172.21.15.94:0/1278957511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d4c0b5cf0 0x7f2d4c157730 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009850 tx=0x7f2d340a56b0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.869+0000 7f2d51800640 1 -- 172.21.15.94:0/1278957511 shutdown_connections 2024-09-10T23:38:02.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.869+0000 7f2d51800640 1 --2- 172.21.15.94:0/1278957511 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2d28072940 0x7f2d28074e00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.869+0000 7f2d51800640 1 --2- 172.21.15.94:0/1278957511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d4c161600 0x7f2d4c158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.869+0000 7f2d51800640 1 --2- 172.21.15.94:0/1278957511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d4c0b5cf0 0x7f2d4c157730 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.869+0000 7f2d51800640 1 --2- 172.21.15.94:0/1278957511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d4c0b2960 0x7f2d4c1504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:02.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.869+0000 7f2d51800640 1 -- 172.21.15.94:0/1278957511 >> 172.21.15.94:0/1278957511 conn(0x7f2d4c0d1c60 msgr2=0x7f2d4c15bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:02.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.869+0000 7f2d51800640 1 -- 172.21.15.94:0/1278957511 shutdown_connections 2024-09-10T23:38:02.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:02.869+0000 7f2d51800640 1 -- 172.21.15.94:0/1278957511 wait complete. 2024-09-10T23:38:02.893 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone47961 2024-09-10T23:38:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.041+0000 7f323f000640 1 -- 172.21.15.94:0/680242231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3220003a60 msgr2=0x7f3220003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.041+0000 7f323f000640 1 --2- 172.21.15.94:0/680242231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3220003a60 0x7f3220003e80 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f322c0098a0 tx=0x7f322c01f5b0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.041+0000 7f323f000640 1 -- 172.21.15.94:0/680242231 shutdown_connections 2024-09-10T23:38:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.041+0000 7f323f000640 1 --2- 172.21.15.94:0/680242231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3220003a60 0x7f3220003e80 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.041+0000 7f323f000640 1 -- 172.21.15.94:0/680242231 >> 172.21.15.94:0/680242231 conn(0x7f323806e610 msgr2=0x7f32380b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.041+0000 7f323f000640 1 -- 172.21.15.94:0/680242231 shutdown_connections 2024-09-10T23:38:03.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.041+0000 7f323f000640 1 -- 172.21.15.94:0/680242231 wait complete. 2024-09-10T23:38:03.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.041+0000 7f323f000640 1 Processor -- start 2024-09-10T23:38:03.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323f000640 1 -- start start 2024-09-10T23:38:03.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323f000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3220003a60 0x7f32381f8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:03.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323f000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32380ca140 0x7f32380c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:03.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323f000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32380ca5c0 0x7f32380c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:03.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323f000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f32380cef70 con 0x7f3220003a60 2024-09-10T23:38:03.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323f000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f32380cedf0 con 0x7f32380ca5c0 2024-09-10T23:38:03.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323f000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f32380cf0f0 con 0x7f32380ca140 2024-09-10T23:38:03.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323dc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3220003a60 0x7f32381f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:03.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323dc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3220003a60 0x7f32381f8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38592/0 (socket says 172.21.15.94:38592) 2024-09-10T23:38:03.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323dc00640 1 -- 172.21.15.94:0/2845362842 learned_addr learned my addr 172.21.15.94:0/2845362842 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323d200640 1 --2- 172.21.15.94:0/2845362842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32380ca140 0x7f32380c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323e600640 1 --2- 172.21.15.94:0/2845362842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32380ca5c0 0x7f32380c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323dc00640 1 -- 172.21.15.94:0/2845362842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32380ca140 msgr2=0x7f32380c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323dc00640 1 --2- 172.21.15.94:0/2845362842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32380ca140 0x7f32380c9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323dc00640 1 -- 172.21.15.94:0/2845362842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32380ca5c0 msgr2=0x7f32380c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323dc00640 1 --2- 172.21.15.94:0/2845362842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32380ca5c0 0x7f32380c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323dc00640 1 -- 172.21.15.94:0/2845362842 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32380c4ad0 con 0x7f3220003a60 2024-09-10T23:38:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323dc00640 1 --2- 172.21.15.94:0/2845362842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3220003a60 0x7f32381f8760 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f32380ce850 tx=0x7f322c011460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f3235600640 1 -- 172.21.15.94:0/2845362842 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c004230 con 0x7f3220003a60 2024-09-10T23:38:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f3235600640 1 -- 172.21.15.94:0/2845362842 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f322c0044c0 con 0x7f3220003a60 2024-09-10T23:38:03.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f3235600640 1 -- 172.21.15.94:0/2845362842 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c0abad0 con 0x7f3220003a60 2024-09-10T23:38:03.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323f000640 1 -- 172.21.15.94:0/2845362842 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32380c4db0 con 0x7f3220003a60 2024-09-10T23:38:03.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323f000640 1 -- 172.21.15.94:0/2845362842 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32380c5370 con 0x7f3220003a60 2024-09-10T23:38:03.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.045+0000 7f323f000640 1 -- 172.21.15.94:0/2845362842 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32380cf650 con 0x7f3220003a60 2024-09-10T23:38:03.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.049+0000 7f3235600640 1 -- 172.21.15.94:0/2845362842 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 96700+0+0 (secure 0 0 0) 0x7f322c0ba070 con 0x7f3220003a60 2024-09-10T23:38:03.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.049+0000 7f3235600640 1 --2- 172.21.15.94:0/2845362842 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f32180728e0 0x7f3218074da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:03.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.049+0000 7f3235600640 1 -- 172.21.15.94:0/2845362842 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5852+0+0 (secure 0 0 0) 0x7f322c12e770 con 0x7f3220003a60 2024-09-10T23:38:03.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.049+0000 7f323d200640 1 --2- 172.21.15.94:0/2845362842 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f32180728e0 0x7f3218074da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:03.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.049+0000 7f323d200640 1 --2- 172.21.15.94:0/2845362842 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f32180728e0 0x7f3218074da0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f323815d210 tx=0x7f32240067b0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:03.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.053+0000 7f3235600640 1 -- 172.21.15.94:0/2845362842 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f322c0af030 con 0x7f3220003a60 2024-09-10T23:38:03.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.245+0000 7f323f000640 1 -- 172.21.15.94:0/2845362842 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone47961", "target": ["mon-mgr", ""]}) -- 0x7f32380b68e0 con 0x7f32180728e0 2024-09-10T23:38:03.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.261+0000 7f3235600640 1 -- 172.21.15.94:0/2845362842 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f32380b68e0 con 0x7f32180728e0 2024-09-10T23:38:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.265+0000 7f323f000640 1 -- 172.21.15.94:0/2845362842 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f32180728e0 msgr2=0x7f3218074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.265+0000 7f323f000640 1 --2- 172.21.15.94:0/2845362842 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f32180728e0 0x7f3218074da0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f323815d210 tx=0x7f32240067b0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.265+0000 7f323f000640 1 -- 172.21.15.94:0/2845362842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3220003a60 msgr2=0x7f32381f8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:03.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.265+0000 7f323f000640 1 --2- 172.21.15.94:0/2845362842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3220003a60 0x7f32381f8760 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f32380ce850 tx=0x7f322c011460 comp rx=0 tx=0).stop 2024-09-10T23:38:03.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.265+0000 7f323f000640 1 -- 172.21.15.94:0/2845362842 shutdown_connections 2024-09-10T23:38:03.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.265+0000 7f323f000640 1 --2- 172.21.15.94:0/2845362842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32380ca5c0 0x7f32380c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.265+0000 7f323f000640 1 --2- 172.21.15.94:0/2845362842 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f32180728e0 0x7f3218074da0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.265+0000 7f323f000640 1 --2- 172.21.15.94:0/2845362842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32380ca140 0x7f32380c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.265+0000 7f323f000640 1 --2- 172.21.15.94:0/2845362842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3220003a60 0x7f32381f8760 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.265+0000 7f323f000640 1 -- 172.21.15.94:0/2845362842 >> 172.21.15.94:0/2845362842 conn(0x7f323806e610 msgr2=0x7f32380d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:03.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.265+0000 7f323f000640 1 -- 172.21.15.94:0/2845362842 shutdown_connections 2024-09-10T23:38:03.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.265+0000 7f323f000640 1 -- 172.21.15.94:0/2845362842 wait complete. 2024-09-10T23:38:03.290 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T23:38:03.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T23:38:03.346 INFO:teuthology.orchestra.run.smithi094.stdout:2 2024-09-10T23:38:03.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:38:03.352 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:38:03.352 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:38:03.431 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:38:03.519 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:38:03.520 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:38:03.520 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:38:03.520 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:03.529 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:38:03.529 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:03.581 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:38:03.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.765+0000 7f7d2b200640 1 -- 172.21.15.94:0/384789079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d2406fc00 msgr2=0x7f7d240c49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:03.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.765+0000 7f7d2b200640 1 --2- 172.21.15.94:0/384789079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d2406fc00 0x7f7d240c49b0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7d2415f840 tx=0x7f7d0c01c800 comp rx=0 tx=0).stop 2024-09-10T23:38:03.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.765+0000 7f7d2b200640 1 -- 172.21.15.94:0/384789079 shutdown_connections 2024-09-10T23:38:03.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.765+0000 7f7d2b200640 1 --2- 172.21.15.94:0/384789079 >> v2:172.21.15.181:3300/0 conn(0x7f7d240cb4d0 0x7f7d2415b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.765+0000 7f7d2b200640 1 --2- 172.21.15.94:0/384789079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d2406fc00 0x7f7d240c49b0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.765+0000 7f7d2b200640 1 -- 172.21.15.94:0/384789079 >> 172.21.15.94:0/384789079 conn(0x7f7d240cfaf0 msgr2=0x7f7d240cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:03.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.765+0000 7f7d2b200640 1 -- 172.21.15.94:0/384789079 shutdown_connections 2024-09-10T23:38:03.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.765+0000 7f7d2b200640 1 -- 172.21.15.94:0/384789079 wait complete. 2024-09-10T23:38:03.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d2b200640 1 Processor -- start 2024-09-10T23:38:03.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d2b200640 1 -- start start 2024-09-10T23:38:03.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d2b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d2406fc00 0x7f7d241f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:03.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d2b200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d240cb4d0 0x7f7d2414b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:03.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d2b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d24158790 0x7f7d240736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:03.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d2b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7d24168070 con 0x7f7d2406fc00 2024-09-10T23:38:03.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d2b200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7d24167ef0 con 0x7f7d24158790 2024-09-10T23:38:03.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d2b200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f7d241681f0 con 0x7f7d240cb4d0 2024-09-10T23:38:03.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d23e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d2406fc00 0x7f7d241f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:03.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d23e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d2406fc00 0x7f7d241f7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47502/0 (socket says 172.21.15.94:47502) 2024-09-10T23:38:03.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d23e00640 1 -- 172.21.15.94:0/3084754143 learned_addr learned my addr 172.21.15.94:0/3084754143 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:03.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d1b400640 1 --2- 172.21.15.94:0/3084754143 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d240cb4d0 0x7f7d2414b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:03.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d29200640 1 --2- 172.21.15.94:0/3084754143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d24158790 0x7f7d240736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:03.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d23e00640 1 -- 172.21.15.94:0/3084754143 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d240cb4d0 msgr2=0x7f7d2414b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:03.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d23e00640 1 --2- 172.21.15.94:0/3084754143 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d240cb4d0 0x7f7d2414b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d23e00640 1 -- 172.21.15.94:0/3084754143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d24158790 msgr2=0x7f7d240736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:03.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d23e00640 1 --2- 172.21.15.94:0/3084754143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d24158790 0x7f7d240736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d23e00640 1 -- 172.21.15.94:0/3084754143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d241f8340 con 0x7f7d2406fc00 2024-09-10T23:38:03.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d23e00640 1 --2- 172.21.15.94:0/3084754143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d2406fc00 0x7f7d241f7ba0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400c3c0 tx=0x7f7d1400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:03.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d20c00640 1 -- 172.21.15.94:0/3084754143 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14011790 con 0x7f7d2406fc00 2024-09-10T23:38:03.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d20c00640 1 -- 172.21.15.94:0/3084754143 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d14017070 con 0x7f7d2406fc00 2024-09-10T23:38:03.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d20c00640 1 -- 172.21.15.94:0/3084754143 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14011d40 con 0x7f7d2406fc00 2024-09-10T23:38:03.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d2b200640 1 -- 172.21.15.94:0/3084754143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d241f8620 con 0x7f7d2406fc00 2024-09-10T23:38:03.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d2b200640 1 -- 172.21.15.94:0/3084754143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2414b6b0 con 0x7f7d2406fc00 2024-09-10T23:38:03.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.769+0000 7f7d2b200640 1 -- 172.21.15.94:0/3084754143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d240c49b0 con 0x7f7d2406fc00 2024-09-10T23:38:03.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.773+0000 7f7d20c00640 1 -- 172.21.15.94:0/3084754143 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 96700+0+0 (secure 0 0 0) 0x7f7d1401c460 con 0x7f7d2406fc00 2024-09-10T23:38:03.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.773+0000 7f7d20c00640 1 --2- 172.21.15.94:0/3084754143 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7d00072970 0x7f7d00074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:03.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.773+0000 7f7d20c00640 1 -- 172.21.15.94:0/3084754143 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5852+0+0 (secure 0 0 0) 0x7f7d14093570 con 0x7f7d2406fc00 2024-09-10T23:38:03.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.773+0000 7f7d1b400640 1 --2- 172.21.15.94:0/3084754143 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7d00072970 0x7f7d00074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:03.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.773+0000 7f7d1b400640 1 --2- 172.21.15.94:0/3084754143 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7d00072970 0x7f7d00074e30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0af000 tx=0x7f7d0c01b040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:03.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.777+0000 7f7d20c00640 1 -- 172.21.15.94:0/3084754143 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7d1405e0a0 con 0x7f7d2406fc00 2024-09-10T23:38:03.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.957+0000 7f7d2b200640 1 -- 172.21.15.94:0/3084754143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7d240d0850 con 0x7f7d2406fc00 2024-09-10T23:38:03.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.957+0000 7f7d20c00640 1 -- 172.21.15.94:0/3084754143 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v103) ==== 80+0+0 (secure 0 0 0) 0x7f7d14062ad0 con 0x7f7d2406fc00 2024-09-10T23:38:03.961 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:38:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.961+0000 7f7d2b200640 1 -- 172.21.15.94:0/3084754143 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7d00072970 msgr2=0x7f7d00074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.961+0000 7f7d2b200640 1 --2- 172.21.15.94:0/3084754143 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7d00072970 0x7f7d00074e30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0af000 tx=0x7f7d0c01b040 comp rx=0 tx=0).stop 2024-09-10T23:38:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.961+0000 7f7d2b200640 1 -- 172.21.15.94:0/3084754143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d2406fc00 msgr2=0x7f7d241f7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.961+0000 7f7d2b200640 1 --2- 172.21.15.94:0/3084754143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d2406fc00 0x7f7d241f7ba0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400c3c0 tx=0x7f7d1400c880 comp rx=0 tx=0).stop 2024-09-10T23:38:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.961+0000 7f7d2b200640 1 -- 172.21.15.94:0/3084754143 shutdown_connections 2024-09-10T23:38:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.961+0000 7f7d2b200640 1 --2- 172.21.15.94:0/3084754143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d24158790 0x7f7d240736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.961+0000 7f7d2b200640 1 --2- 172.21.15.94:0/3084754143 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7d00072970 0x7f7d00074e30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.961+0000 7f7d2b200640 1 --2- 172.21.15.94:0/3084754143 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d240cb4d0 0x7f7d2414b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.961+0000 7f7d2b200640 1 --2- 172.21.15.94:0/3084754143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d2406fc00 0x7f7d241f7ba0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.961+0000 7f7d2b200640 1 -- 172.21.15.94:0/3084754143 >> 172.21.15.94:0/3084754143 conn(0x7f7d240cfaf0 msgr2=0x7f7d24151c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.961+0000 7f7d2b200640 1 -- 172.21.15.94:0/3084754143 shutdown_connections 2024-09-10T23:38:03.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:03.961+0000 7f7d2b200640 1 -- 172.21.15.94:0/3084754143 wait complete. 2024-09-10T23:38:03.982 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-09-10T23:38:03.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:03.982 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T23:38:04.000 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[44916]: fuse finished with error 0 and tester_r 0 2024-09-10T23:38:04.002 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:38:04.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:04.002 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T23:38:04.176 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:38:04.176 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:38:04.176 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:04.185 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:38:04.185 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:04.236 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:38:04.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.377+0000 7f010d600640 1 -- 172.21.15.94:0/1901047507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01080cc760 msgr2=0x7f00ec004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:04.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.377+0000 7f010d600640 1 --2- 172.21.15.94:0/1901047507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01080cc760 0x7f00ec004810 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009b60 tx=0x7f00f001d2b0 comp rx=0 tx=0).stop 2024-09-10T23:38:04.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.377+0000 7f0107400640 1 -- 172.21.15.94:0/1901047507 reap_dead start 2024-09-10T23:38:04.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.377+0000 7f010d600640 1 -- 172.21.15.94:0/1901047507 shutdown_connections 2024-09-10T23:38:04.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.377+0000 7f010d600640 1 -- 172.21.15.94:0/1901047507 >> 172.21.15.94:0/1901047507 conn(0x7f01080d1c90 msgr2=0x7f01080d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:04.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.377+0000 7f010d600640 1 -- 172.21.15.94:0/1901047507 shutdown_connections 2024-09-10T23:38:04.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.377+0000 7f010d600640 1 -- 172.21.15.94:0/1901047507 wait complete. 2024-09-10T23:38:04.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f010d600640 1 Processor -- start 2024-09-10T23:38:04.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f010d600640 1 -- start start 2024-09-10T23:38:04.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f010d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01080cc760 0x7f010815b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:04.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f010d600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01080b75d0 0x7f01081e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:04.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f010d600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01080ca6f0 0x7f01081e7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:04.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f010d600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f010806eae0 con 0x7f01080cc760 2024-09-10T23:38:04.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f010d600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f01080d0ba0 con 0x7f01080b75d0 2024-09-10T23:38:04.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f010d600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f010806ec60 con 0x7f01080ca6f0 2024-09-10T23:38:04.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f0106a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01080cc760 0x7f010815b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:04.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f0106a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01080cc760 0x7f010815b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47514/0 (socket says 172.21.15.94:47514) 2024-09-10T23:38:04.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f0106a00640 1 -- 172.21.15.94:0/1649232477 learned_addr learned my addr 172.21.15.94:0/1649232477 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:04.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f00fe000640 1 --2- 172.21.15.94:0/1649232477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01080b75d0 0x7f01081e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:04.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f0107400640 1 --2- 172.21.15.94:0/1649232477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01080ca6f0 0x7f01081e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:04.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f0106a00640 1 -- 172.21.15.94:0/1649232477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01080ca6f0 msgr2=0x7f01081e7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:04.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f0106a00640 1 --2- 172.21.15.94:0/1649232477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01080ca6f0 0x7f01081e7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:04.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f0106a00640 1 -- 172.21.15.94:0/1649232477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01080b75d0 msgr2=0x7f01081e6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:04.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f0106a00640 1 --2- 172.21.15.94:0/1649232477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01080b75d0 0x7f01081e6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:04.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f0106a00640 1 -- 172.21.15.94:0/1649232477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01081fc570 con 0x7f01080cc760 2024-09-10T23:38:04.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f0107400640 1 --2- 172.21.15.94:0/1649232477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01080ca6f0 0x7f01081e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:04.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f0106a00640 1 --2- 172.21.15.94:0/1649232477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01080cc760 0x7f010815b120 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f00f8011ea0 tx=0x7f00f8011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:04.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f00ff400640 1 -- 172.21.15.94:0/1649232477 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f801e070 con 0x7f01080cc760 2024-09-10T23:38:04.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f00ff400640 1 -- 172.21.15.94:0/1649232477 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00f800e5c0 con 0x7f01080cc760 2024-09-10T23:38:04.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f010d600640 1 -- 172.21.15.94:0/1649232477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01081fc850 con 0x7f01080cc760 2024-09-10T23:38:04.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.381+0000 7f00ff400640 1 -- 172.21.15.94:0/1649232477 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f800e9f0 con 0x7f01080cc760 2024-09-10T23:38:04.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.385+0000 7f010d600640 1 -- 172.21.15.94:0/1649232477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01081fce10 con 0x7f01080cc760 2024-09-10T23:38:04.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.385+0000 7f010d600640 1 -- 172.21.15.94:0/1649232477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01080cec10 con 0x7f01080cc760 2024-09-10T23:38:04.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.385+0000 7f00ff400640 1 -- 172.21.15.94:0/1649232477 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 96700+0+0 (secure 0 0 0) 0x7f00f800eb90 con 0x7f01080cc760 2024-09-10T23:38:04.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.385+0000 7f00ff400640 1 --2- 172.21.15.94:0/1649232477 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f00e4072970 0x7f00e4074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:04.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.385+0000 7f00ff400640 1 -- 172.21.15.94:0/1649232477 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5868+0+0 (secure 0 0 0) 0x7f00f80949f0 con 0x7f01080cc760 2024-09-10T23:38:04.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.385+0000 7f00fe000640 1 --2- 172.21.15.94:0/1649232477 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f00e4072970 0x7f00e4074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:04.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.389+0000 7f00fe000640 1 --2- 172.21.15.94:0/1649232477 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f00e4072970 0x7f00e4074e30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f00f00af000 tx=0x7f00f001a050 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:04.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.389+0000 7f00ff400640 1 -- 172.21.15.94:0/1649232477 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f00f805f450 con 0x7f01080cc760 2024-09-10T23:38:04.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.565+0000 7f010d600640 1 -- 172.21.15.94:0/1649232477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f01080cc0c0 con 0x7f01080cc760 2024-09-10T23:38:04.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.569+0000 7f00ff400640 1 -- 172.21.15.94:0/1649232477 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v103) ==== 80+0+0 (secure 0 0 0) 0x7f00f8063e80 con 0x7f01080cc760 2024-09-10T23:38:04.570 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:38:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.569+0000 7f010d600640 1 -- 172.21.15.94:0/1649232477 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f00e4072970 msgr2=0x7f00e4074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.569+0000 7f010d600640 1 --2- 172.21.15.94:0/1649232477 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f00e4072970 0x7f00e4074e30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f00f00af000 tx=0x7f00f001a050 comp rx=0 tx=0).stop 2024-09-10T23:38:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.569+0000 7f010d600640 1 -- 172.21.15.94:0/1649232477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01080cc760 msgr2=0x7f010815b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.569+0000 7f010d600640 1 --2- 172.21.15.94:0/1649232477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01080cc760 0x7f010815b120 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f00f8011ea0 tx=0x7f00f8011f90 comp rx=0 tx=0).stop 2024-09-10T23:38:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.569+0000 7f010d600640 1 -- 172.21.15.94:0/1649232477 shutdown_connections 2024-09-10T23:38:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.569+0000 7f010d600640 1 --2- 172.21.15.94:0/1649232477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01080b75d0 0x7f01081e6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.569+0000 7f010d600640 1 --2- 172.21.15.94:0/1649232477 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f00e4072970 0x7f00e4074e30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.569+0000 7f010d600640 1 --2- 172.21.15.94:0/1649232477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01080ca6f0 0x7f01081e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.569+0000 7f010d600640 1 --2- 172.21.15.94:0/1649232477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01080cc760 0x7f010815b120 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:04.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.569+0000 7f010d600640 1 -- 172.21.15.94:0/1649232477 >> 172.21.15.94:0/1649232477 conn(0x7f01080d1c90 msgr2=0x7f01080b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:04.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.569+0000 7f010d600640 1 -- 172.21.15.94:0/1649232477 shutdown_connections 2024-09-10T23:38:04.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.569+0000 7f010d600640 1 -- 172.21.15.94:0/1649232477 wait complete. 2024-09-10T23:38:04.591 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi181.front.sepia.ceph.com... 2024-09-10T23:38:04.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:04.592 DEBUG:teuthology.orchestra.run.smithi181:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T23:38:04.610 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[35501]: fuse finished with error 0 and tester_r 0 2024-09-10T23:38:04.614 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:38:04.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:04.614 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T23:38:04.708 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:38:04.708 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:38:04.708 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:04.717 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:38:04.717 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:38:04.717 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:38:04.717 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:04.756 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:38:04.757 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:38:04.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.901+0000 7f2a81400640 1 -- 172.21.15.94:0/3291283533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a7c0cc2f0 msgr2=0x7f2a7c06f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:04.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.901+0000 7f2a81400640 1 --2- 172.21.15.94:0/3291283533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a7c0cc2f0 0x7f2a7c06f900 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0c7330 tx=0x7f2a6401ccf0 comp rx=0 tx=0).stop 2024-09-10T23:38:04.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.901+0000 7f2a81400640 1 -- 172.21.15.94:0/3291283533 shutdown_connections 2024-09-10T23:38:04.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.901+0000 7f2a81400640 1 --2- 172.21.15.94:0/3291283533 >> v2:172.21.15.181:3300/0 conn(0x7f2a7c06fe40 0x7f2a7c0cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:04.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.901+0000 7f2a81400640 1 --2- 172.21.15.94:0/3291283533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a7c0cc2f0 0x7f2a7c06f900 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:04.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.901+0000 7f2a81400640 1 --2- 172.21.15.94:0/3291283533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a7c15f3f0 0x7f2a7c068990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:04.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.901+0000 7f2a81400640 1 -- 172.21.15.94:0/3291283533 >> 172.21.15.94:0/3291283533 conn(0x7f2a7c0d02d0 msgr2=0x7f2a7c0d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:04.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.901+0000 7f2a81400640 1 -- 172.21.15.94:0/3291283533 shutdown_connections 2024-09-10T23:38:04.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.901+0000 7f2a81400640 1 -- 172.21.15.94:0/3291283533 wait complete. 2024-09-10T23:38:04.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a81400640 1 Processor -- start 2024-09-10T23:38:04.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a81400640 1 -- start start 2024-09-10T23:38:04.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a81400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a7c06fe40 0x7f2a7c098dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:04.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a81400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a7c0cc2f0 0x7f2a7c155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:04.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a81400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a7c15f3f0 0x7f2a7c15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:04.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a81400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c0cb490 con 0x7f2a7c15f3f0 2024-09-10T23:38:04.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a81400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c0cb310 con 0x7f2a7c0cc2f0 2024-09-10T23:38:04.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a81400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7c0cb610 con 0x7f2a7c06fe40 2024-09-10T23:38:04.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a7b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a7c15f3f0 0x7f2a7c15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:04.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a7b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a7c15f3f0 0x7f2a7c15d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47534/0 (socket says 172.21.15.94:47534) 2024-09-10T23:38:04.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a7b200640 1 -- 172.21.15.94:0/2262572555 learned_addr learned my addr 172.21.15.94:0/2262572555 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:04.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a7a800640 1 --2- 172.21.15.94:0/2262572555 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a7c06fe40 0x7f2a7c098dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:04.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a71e00640 1 --2- 172.21.15.94:0/2262572555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a7c0cc2f0 0x7f2a7c155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:04.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a7b200640 1 -- 172.21.15.94:0/2262572555 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a7c06fe40 msgr2=0x7f2a7c098dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a7b200640 1 --2- 172.21.15.94:0/2262572555 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a7c06fe40 0x7f2a7c098dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a7b200640 1 -- 172.21.15.94:0/2262572555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a7c0cc2f0 msgr2=0x7f2a7c155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a7b200640 1 --2- 172.21.15.94:0/2262572555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a7c0cc2f0 0x7f2a7c155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a7b200640 1 -- 172.21.15.94:0/2262572555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c099300 con 0x7f2a7c15f3f0 2024-09-10T23:38:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a7b200640 1 --2- 172.21.15.94:0/2262572555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a7c15f3f0 0x7f2a7c15d1e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400e940 tx=0x7f2a7400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a73400640 1 -- 172.21.15.94:0/2262572555 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74004440 con 0x7f2a7c15f3f0 2024-09-10T23:38:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a73400640 1 -- 172.21.15.94:0/2262572555 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a7400c9c0 con 0x7f2a7c15f3f0 2024-09-10T23:38:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a73400640 1 -- 172.21.15.94:0/2262572555 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7400ce10 con 0x7f2a7c15f3f0 2024-09-10T23:38:04.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a81400640 1 -- 172.21.15.94:0/2262572555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7c15e5f0 con 0x7f2a7c15f3f0 2024-09-10T23:38:04.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.905+0000 7f2a81400640 1 -- 172.21.15.94:0/2262572555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7c0b5830 con 0x7f2a7c15f3f0 2024-09-10T23:38:04.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.909+0000 7f2a81400640 1 -- 172.21.15.94:0/2262572555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a7c06f900 con 0x7f2a7c15f3f0 2024-09-10T23:38:04.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.909+0000 7f2a73400640 1 -- 172.21.15.94:0/2262572555 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f2a74020020 con 0x7f2a7c15f3f0 2024-09-10T23:38:04.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.909+0000 7f2a73400640 1 --2- 172.21.15.94:0/2262572555 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2a58072a60 0x7f2a58074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.909+0000 7f2a73400640 1 -- 172.21.15.94:0/2262572555 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5868+0+0 (secure 0 0 0) 0x7f2a74093c00 con 0x7f2a7c15f3f0 2024-09-10T23:38:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.909+0000 7f2a7a800640 1 --2- 172.21.15.94:0/2262572555 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2a58072a60 0x7f2a58074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:04.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.913+0000 7f2a7a800640 1 --2- 172.21.15.94:0/2262572555 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2a58072a60 0x7f2a58074f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009b40 tx=0x7f2a6c0091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:04.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:04.913+0000 7f2a73400640 1 -- 172.21.15.94:0/2262572555 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2a7405e470 con 0x7f2a7c15f3f0 2024-09-10T23:38:05.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.113+0000 7f2a81400640 1 -- 172.21.15.94:0/2262572555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a7c0b6250 con 0x7f2a7c15f3f0 2024-09-10T23:38:05.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.113+0000 7f2a73400640 1 -- 172.21.15.94:0/2262572555 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5243 (secure 0 0 0) 0x7f2a74062ea0 con 0x7f2a7c15f3f0 2024-09-10T23:38:05.116 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:05.116 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":69,"btime":"2024-09-10T23:37:36:617338+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7011,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1105858824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1105858824},{"type":"v1","addr":"172.21.15.94:6834","nonce":1105858824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3277955689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3277955689},{"type":"v1","addr":"172.21.15.94:6835","nonce":3277955689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":195947,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/2607191387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2607191387},{"type":"v1","addr":"172.21.15.181:6801","nonce":2607191387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:37:25.941783+0000","modified":"2024-09-10T23:37:28.978060+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195950},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195950":{"gid":195950,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/2513770511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2513770511},{"type":"v1","addr":"172.21.15.181:6835","nonce":2513770511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195950,"qdb_cluster":[195950]},"id":8}]} 2024-09-10T23:38:05.116 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 69 2024-09-10T23:38:05.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.117+0000 7f2a81400640 1 -- 172.21.15.94:0/2262572555 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2a58072a60 msgr2=0x7f2a58074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:05.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.117+0000 7f2a81400640 1 --2- 172.21.15.94:0/2262572555 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2a58072a60 0x7f2a58074f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009b40 tx=0x7f2a6c0091e0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.117+0000 7f2a81400640 1 -- 172.21.15.94:0/2262572555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a7c15f3f0 msgr2=0x7f2a7c15d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:05.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.117+0000 7f2a81400640 1 --2- 172.21.15.94:0/2262572555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a7c15f3f0 0x7f2a7c15d1e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400e940 tx=0x7f2a7400ee00 comp rx=0 tx=0).stop 2024-09-10T23:38:05.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.117+0000 7f2a81400640 1 -- 172.21.15.94:0/2262572555 shutdown_connections 2024-09-10T23:38:05.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.117+0000 7f2a81400640 1 --2- 172.21.15.94:0/2262572555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a7c0cc2f0 0x7f2a7c155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.117+0000 7f2a81400640 1 --2- 172.21.15.94:0/2262572555 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2a58072a60 0x7f2a58074f20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.117+0000 7f2a81400640 1 --2- 172.21.15.94:0/2262572555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a7c15f3f0 0x7f2a7c15d1e0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.117+0000 7f2a81400640 1 --2- 172.21.15.94:0/2262572555 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a7c06fe40 0x7f2a7c098dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.117+0000 7f2a81400640 1 -- 172.21.15.94:0/2262572555 >> 172.21.15.94:0/2262572555 conn(0x7f2a7c0d02d0 msgr2=0x7f2a7c06eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:05.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.117+0000 7f2a81400640 1 -- 172.21.15.94:0/2262572555 shutdown_connections 2024-09-10T23:38:05.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.117+0000 7f2a81400640 1 -- 172.21.15.94:0/2262572555 wait complete. 2024-09-10T23:38:05.137 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:38:05.246 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:38:05.249 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:38:05.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.277+0000 7feeee600640 1 -- 172.21.15.94:0/3878889707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee815c210 msgr2=0x7feee8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:05.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.277+0000 7feeee600640 1 --2- 172.21.15.94:0/3878889707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee815c210 0x7feee8068990 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7feee80cc2d0 tx=0x7feedc01c830 comp rx=0 tx=0).stop 2024-09-10T23:38:05.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.277+0000 7feeee600640 1 -- 172.21.15.94:0/3878889707 shutdown_connections 2024-09-10T23:38:05.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.277+0000 7feeee600640 1 --2- 172.21.15.94:0/3878889707 >> v2:172.21.15.181:3300/0 conn(0x7feee815f530 0x7feee815f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.277+0000 7feeee600640 1 --2- 172.21.15.94:0/3878889707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee815c210 0x7feee8068990 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.277+0000 7feeee600640 1 -- 172.21.15.94:0/3878889707 >> 172.21.15.94:0/3878889707 conn(0x7feee80d0120 msgr2=0x7feee80d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:05.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.277+0000 7feeee600640 1 -- 172.21.15.94:0/3878889707 shutdown_connections 2024-09-10T23:38:05.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.277+0000 7feeee600640 1 -- 172.21.15.94:0/3878889707 wait complete. 2024-09-10T23:38:05.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.277+0000 7feeee600640 1 Processor -- start 2024-09-10T23:38:05.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.277+0000 7feeee600640 1 -- start start 2024-09-10T23:38:05.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feeee600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feee80ccaa0 0x7feee80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:05.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feeee600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee815c210 0x7feee815b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:05.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feeee600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feee815f530 0x7feee81f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:05.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feeee600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feee8160690 con 0x7feee815c210 2024-09-10T23:38:05.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feeee600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feee815cde0 con 0x7feee815f530 2024-09-10T23:38:05.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feeee600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7feee8160810 con 0x7feee80ccaa0 2024-09-10T23:38:05.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feee7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee815c210 0x7feee815b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:05.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feee7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee815c210 0x7feee815b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47556/0 (socket says 172.21.15.94:47556) 2024-09-10T23:38:05.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feee7e00640 1 -- 172.21.15.94:0/3853255128 learned_addr learned my addr 172.21.15.94:0/3853255128 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:05.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feeed200640 1 --2- 172.21.15.94:0/3853255128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feee80ccaa0 0x7feee80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:05.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feeedc00640 1 --2- 172.21.15.94:0/3853255128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feee815f530 0x7feee81f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:05.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feee7e00640 1 -- 172.21.15.94:0/3853255128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feee80ccaa0 msgr2=0x7feee80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:05.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feee7e00640 1 --2- 172.21.15.94:0/3853255128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feee80ccaa0 0x7feee80b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feee7e00640 1 -- 172.21.15.94:0/3853255128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feee815f530 msgr2=0x7feee81f77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:05.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feee7e00640 1 --2- 172.21.15.94:0/3853255128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feee815f530 0x7feee81f77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feee7e00640 1 -- 172.21.15.94:0/3853255128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee80738b0 con 0x7feee815c210 2024-09-10T23:38:05.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feee7e00640 1 --2- 172.21.15.94:0/3853255128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee815c210 0x7feee815b130 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7feed400c500 tx=0x7feed400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:05.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feee4c00640 1 -- 172.21.15.94:0/3853255128 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4017070 con 0x7feee815c210 2024-09-10T23:38:05.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feee4c00640 1 -- 172.21.15.94:0/3853255128 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feed4003600 con 0x7feee815c210 2024-09-10T23:38:05.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feee4c00640 1 -- 172.21.15.94:0/3853255128 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4003a50 con 0x7feee815c210 2024-09-10T23:38:05.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feeee600640 1 -- 172.21.15.94:0/3853255128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee8073330 con 0x7feee815c210 2024-09-10T23:38:05.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feeee600640 1 -- 172.21.15.94:0/3853255128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee80735d0 con 0x7feee815c210 2024-09-10T23:38:05.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.281+0000 7feeee600640 1 -- 172.21.15.94:0/3853255128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee8068990 con 0x7feee815c210 2024-09-10T23:38:05.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.285+0000 7feee4c00640 1 -- 172.21.15.94:0/3853255128 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7feed4003bf0 con 0x7feee815c210 2024-09-10T23:38:05.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.285+0000 7feee4c00640 1 --2- 172.21.15.94:0/3853255128 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feec8072a90 0x7feec8074f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:05.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.285+0000 7feee4c00640 1 -- 172.21.15.94:0/3853255128 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5868+0+0 (secure 0 0 0) 0x7feed4093380 con 0x7feee815c210 2024-09-10T23:38:05.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.285+0000 7feeed200640 1 --2- 172.21.15.94:0/3853255128 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feec8072a90 0x7feec8074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:05.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.285+0000 7feeed200640 1 --2- 172.21.15.94:0/3853255128 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feec8072a90 0x7feec8074f50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feedc0af000 tx=0x7feedc01b070 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:05.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.289+0000 7feee4c00640 1 -- 172.21.15.94:0/3853255128 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feed405e300 con 0x7feee815c210 2024-09-10T23:38:05.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.489+0000 7feeee600640 1 -- 172.21.15.94:0/3853255128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feee815f940 con 0x7feee815c210 2024-09-10T23:38:05.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.489+0000 7feee4c00640 1 -- 172.21.15.94:0/3853255128 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5243 (secure 0 0 0) 0x7feed405dc00 con 0x7feee815c210 2024-09-10T23:38:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:05.492 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":69,"btime":"2024-09-10T23:37:36:617338+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7011,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1105858824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1105858824},{"type":"v1","addr":"172.21.15.94:6834","nonce":1105858824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3277955689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3277955689},{"type":"v1","addr":"172.21.15.94:6835","nonce":3277955689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":195947,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/2607191387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2607191387},{"type":"v1","addr":"172.21.15.181:6801","nonce":2607191387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:37:25.941783+0000","modified":"2024-09-10T23:37:28.978060+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195950},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195950":{"gid":195950,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/2513770511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2513770511},{"type":"v1","addr":"172.21.15.181:6835","nonce":2513770511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195950,"qdb_cluster":[195950]},"id":8}]} 2024-09-10T23:38:05.492 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 69 2024-09-10T23:38:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.493+0000 7feeee600640 1 -- 172.21.15.94:0/3853255128 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feec8072a90 msgr2=0x7feec8074f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.493+0000 7feeee600640 1 --2- 172.21.15.94:0/3853255128 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feec8072a90 0x7feec8074f50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feedc0af000 tx=0x7feedc01b070 comp rx=0 tx=0).stop 2024-09-10T23:38:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.493+0000 7feeee600640 1 -- 172.21.15.94:0/3853255128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee815c210 msgr2=0x7feee815b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.493+0000 7feeee600640 1 --2- 172.21.15.94:0/3853255128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee815c210 0x7feee815b130 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7feed400c500 tx=0x7feed400c9c0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.493+0000 7feeee600640 1 -- 172.21.15.94:0/3853255128 shutdown_connections 2024-09-10T23:38:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.493+0000 7feeee600640 1 --2- 172.21.15.94:0/3853255128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feee815f530 0x7feee81f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.493+0000 7feeee600640 1 --2- 172.21.15.94:0/3853255128 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7feec8072a90 0x7feec8074f50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.493+0000 7feeee600640 1 --2- 172.21.15.94:0/3853255128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feee815c210 0x7feee815b130 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.493+0000 7feeee600640 1 --2- 172.21.15.94:0/3853255128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feee80ccaa0 0x7feee80b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.493+0000 7feeee600640 1 -- 172.21.15.94:0/3853255128 >> 172.21.15.94:0/3853255128 conn(0x7feee80d0120 msgr2=0x7feee8073b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:05.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.493+0000 7feeee600640 1 -- 172.21.15.94:0/3853255128 shutdown_connections 2024-09-10T23:38:05.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.493+0000 7feeee600640 1 -- 172.21.15.94:0/3853255128 wait complete. 2024-09-10T23:38:05.512 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:38:05.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.653+0000 7feab2800640 1 -- 172.21.15.94:0/1166126477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaac06cc20 msgr2=0x7feaac06d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:05.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.653+0000 7feab2800640 1 --2- 172.21.15.94:0/1166126477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaac06cc20 0x7feaac06d000 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7feaac073620 tx=0x7fea9c01c810 comp rx=0 tx=0).stop 2024-09-10T23:38:05.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.653+0000 7feab2800640 1 -- 172.21.15.94:0/1166126477 shutdown_connections 2024-09-10T23:38:05.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.653+0000 7feab2800640 1 --2- 172.21.15.94:0/1166126477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaac0d1690 0x7feaac15fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.653+0000 7feab2800640 1 --2- 172.21.15.94:0/1166126477 >> v2:172.21.15.181:3300/0 conn(0x7feaac071a20 0x7feaac15f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.653+0000 7feab2800640 1 --2- 172.21.15.94:0/1166126477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaac06cc20 0x7feaac06d000 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.653+0000 7feab2800640 1 -- 172.21.15.94:0/1166126477 >> 172.21.15.94:0/1166126477 conn(0x7feaac0d0b70 msgr2=0x7feaac0d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:05.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.653+0000 7feab2800640 1 -- 172.21.15.94:0/1166126477 shutdown_connections 2024-09-10T23:38:05.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.653+0000 7feab2800640 1 -- 172.21.15.94:0/1166126477 wait complete. 2024-09-10T23:38:05.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feab2800640 1 Processor -- start 2024-09-10T23:38:05.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feab2800640 1 -- start start 2024-09-10T23:38:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feab2800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaac06cc20 0x7feaac14b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feab2800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaac071a20 0x7feaac142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feab2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaac0d1690 0x7feaac15dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:05.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feab2800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feaac0cff30 con 0x7feaac0d1690 2024-09-10T23:38:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feab2800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feaac0cfdb0 con 0x7feaac071a20 2024-09-10T23:38:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feab2800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7feaac0d00b0 con 0x7feaac06cc20 2024-09-10T23:38:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feaabe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaac0d1690 0x7feaac15dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feaabe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaac0d1690 0x7feaac15dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47580/0 (socket says 172.21.15.94:47580) 2024-09-10T23:38:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feaabe00640 1 -- 172.21.15.94:0/2377109869 learned_addr learned my addr 172.21.15.94:0/2377109869 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feaaaa00640 1 --2- 172.21.15.94:0/2377109869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaac071a20 0x7feaac142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feaab400640 1 --2- 172.21.15.94:0/2377109869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaac06cc20 0x7feaac14b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feaabe00640 1 -- 172.21.15.94:0/2377109869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaac06cc20 msgr2=0x7feaac14b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feaabe00640 1 --2- 172.21.15.94:0/2377109869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaac06cc20 0x7feaac14b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feaabe00640 1 -- 172.21.15.94:0/2377109869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaac071a20 msgr2=0x7feaac142150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feaabe00640 1 --2- 172.21.15.94:0/2377109869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaac071a20 0x7feaac142150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feaabe00640 1 -- 172.21.15.94:0/2377109869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaac15fec0 con 0x7feaac0d1690 2024-09-10T23:38:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feaab400640 1 --2- 172.21.15.94:0/2377109869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaac06cc20 0x7feaac14b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:05.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feaaaa00640 1 --2- 172.21.15.94:0/2377109869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaac071a20 0x7feaac142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feaabe00640 1 --2- 172.21.15.94:0/2377109869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaac0d1690 0x7feaac15dd00 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7feaa400b9a0 tx=0x7feaa400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feaa3400640 1 -- 172.21.15.94:0/2377109869 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa4019070 con 0x7feaac0d1690 2024-09-10T23:38:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feaa3400640 1 -- 172.21.15.94:0/2377109869 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feaa400ccb0 con 0x7feaac0d1690 2024-09-10T23:38:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feab2800640 1 -- 172.21.15.94:0/2377109869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaac1fbfe0 con 0x7feaac0d1690 2024-09-10T23:38:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feaa3400640 1 -- 172.21.15.94:0/2377109869 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa40146a0 con 0x7feaac0d1690 2024-09-10T23:38:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.657+0000 7feab2800640 1 -- 172.21.15.94:0/2377109869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaac14b120 con 0x7feaac0d1690 2024-09-10T23:38:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.661+0000 7feab2800640 1 -- 172.21.15.94:0/2377109869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaac06d000 con 0x7feaac0d1690 2024-09-10T23:38:05.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.661+0000 7feaa3400640 1 -- 172.21.15.94:0/2377109869 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7feaa401e440 con 0x7feaac0d1690 2024-09-10T23:38:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.661+0000 7feaa3400640 1 --2- 172.21.15.94:0/2377109869 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea8c072a60 0x7fea8c074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:05.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.661+0000 7feaa3400640 1 -- 172.21.15.94:0/2377109869 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5868+0+0 (secure 0 0 0) 0x7feaa4093cb0 con 0x7feaac0d1690 2024-09-10T23:38:05.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.661+0000 7feaab400640 1 --2- 172.21.15.94:0/2377109869 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea8c072a60 0x7fea8c074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:05.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.661+0000 7feaab400640 1 --2- 172.21.15.94:0/2377109869 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea8c072a60 0x7fea8c074f20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0ae000 tx=0x7fea9c01b050 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:05.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.665+0000 7feaa3400640 1 -- 172.21.15.94:0/2377109869 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feaa405eb70 con 0x7feaac0d1690 2024-09-10T23:38:05.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.829+0000 7feab2800640 1 -- 172.21.15.94:0/2377109869 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feaac14dfb0 con 0x7feaac0d1690 2024-09-10T23:38:05.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.833+0000 7feaa3400640 1 -- 172.21.15.94:0/2377109869 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16992 (secure 0 0 0) 0x7feaa405e470 con 0x7feaac0d1690 2024-09-10T23:38:05.834 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:05.834 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":103,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:38:03.839881+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:37:23.632549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T23:37:24.916517+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.5","mappings":[{"from":4,"to":3}]},{"pgid":"17.25","mappings":[{"from":6,"to":5}]},{"pgid":"17.31","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:38:05.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.833+0000 7feab2800640 1 -- 172.21.15.94:0/2377109869 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea8c072a60 msgr2=0x7fea8c074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:05.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.833+0000 7feab2800640 1 --2- 172.21.15.94:0/2377109869 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea8c072a60 0x7fea8c074f20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0ae000 tx=0x7fea9c01b050 comp rx=0 tx=0).stop 2024-09-10T23:38:05.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.833+0000 7feab2800640 1 -- 172.21.15.94:0/2377109869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaac0d1690 msgr2=0x7feaac15dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:05.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.833+0000 7feab2800640 1 --2- 172.21.15.94:0/2377109869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaac0d1690 0x7feaac15dd00 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7feaa400b9a0 tx=0x7feaa400be60 comp rx=0 tx=0).stop 2024-09-10T23:38:05.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.833+0000 7feab2800640 1 -- 172.21.15.94:0/2377109869 shutdown_connections 2024-09-10T23:38:05.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.833+0000 7feab2800640 1 --2- 172.21.15.94:0/2377109869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaac071a20 0x7feaac142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.833+0000 7feab2800640 1 --2- 172.21.15.94:0/2377109869 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea8c072a60 0x7fea8c074f20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.833+0000 7feab2800640 1 --2- 172.21.15.94:0/2377109869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaac0d1690 0x7feaac15dd00 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.833+0000 7feab2800640 1 --2- 172.21.15.94:0/2377109869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaac06cc20 0x7feaac14b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.833+0000 7feab2800640 1 -- 172.21.15.94:0/2377109869 >> 172.21.15.94:0/2377109869 conn(0x7feaac0d0b70 msgr2=0x7feaac0b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:05.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.833+0000 7feab2800640 1 -- 172.21.15.94:0/2377109869 shutdown_connections 2024-09-10T23:38:05.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.833+0000 7feab2800640 1 -- 172.21.15.94:0/2377109869 wait complete. 2024-09-10T23:38:05.856 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T23:38:05.856 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:38:05.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.997+0000 7fdcd2e00640 1 -- 172.21.15.94:0/2374679171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdccc068c30 msgr2=0x7fdccc0ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:05.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.997+0000 7fdcd2e00640 1 --2- 172.21.15.94:0/2374679171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdccc068c30 0x7fdccc0ad5a0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0691f0 tx=0x7fdcb401c830 comp rx=0 tx=0).stop 2024-09-10T23:38:05.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.997+0000 7fdcd2e00640 1 -- 172.21.15.94:0/2374679171 shutdown_connections 2024-09-10T23:38:05.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.997+0000 7fdcd2e00640 1 --2- 172.21.15.94:0/2374679171 >> v2:172.21.15.181:3300/0 conn(0x7fdccc0cf690 0x7fdccc0cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.997+0000 7fdcd2e00640 1 --2- 172.21.15.94:0/2374679171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdccc068c30 0x7fdccc0ad5a0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:05.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.997+0000 7fdcd2e00640 1 --2- 172.21.15.94:0/2374679171 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdccc15bcc0 0x7fdccc0ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.997+0000 7fdcd2e00640 1 -- 172.21.15.94:0/2374679171 >> 172.21.15.94:0/2374679171 conn(0x7fdccc0d02d0 msgr2=0x7fdccc0d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:06.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.997+0000 7fdcd2e00640 1 -- 172.21.15.94:0/2374679171 shutdown_connections 2024-09-10T23:38:06.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.997+0000 7fdcd2e00640 1 -- 172.21.15.94:0/2374679171 wait complete. 2024-09-10T23:38:06.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.997+0000 7fdcd2e00640 1 Processor -- start 2024-09-10T23:38:06.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:05.997+0000 7fdcd2e00640 1 -- start start 2024-09-10T23:38:06.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdcd2e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdccc068c30 0x7fdccc0704f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:06.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdcd2e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdccc0cf690 0x7fdccc070a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdcd2e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdccc15bcc0 0x7fdccc1fb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdcd2e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc0cbc90 con 0x7fdccc068c30 2024-09-10T23:38:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdcd2e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc0cbb10 con 0x7fdccc15bcc0 2024-09-10T23:38:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdcd2e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fdccc0cbe10 con 0x7fdccc0cf690 2024-09-10T23:38:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdccbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdccc068c30 0x7fdccc0704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdccbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdccc068c30 0x7fdccc0704f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47602/0 (socket says 172.21.15.94:47602) 2024-09-10T23:38:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdccbe00640 1 -- 172.21.15.94:0/1196655599 learned_addr learned my addr 172.21.15.94:0/1196655599 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdcc3400640 1 --2- 172.21.15.94:0/1196655599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdccc0cf690 0x7fdccc070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdcd0e00640 1 --2- 172.21.15.94:0/1196655599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdccc15bcc0 0x7fdccc1fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdccbe00640 1 -- 172.21.15.94:0/1196655599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdccc0cf690 msgr2=0x7fdccc070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdccbe00640 1 --2- 172.21.15.94:0/1196655599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdccc0cf690 0x7fdccc070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdccbe00640 1 -- 172.21.15.94:0/1196655599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdccc15bcc0 msgr2=0x7fdccc1fb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:06.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdccbe00640 1 --2- 172.21.15.94:0/1196655599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdccc15bcc0 0x7fdccc1fb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdccbe00640 1 -- 172.21.15.94:0/1196655599 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdccc0cd420 con 0x7fdccc068c30 2024-09-10T23:38:06.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdccbe00640 1 --2- 172.21.15.94:0/1196655599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdccc068c30 0x7fdccc0704f0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00d720 tx=0x7fdcbc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:06.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdcc8c00640 1 -- 172.21.15.94:0/1196655599 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc017070 con 0x7fdccc068c30 2024-09-10T23:38:06.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdcc8c00640 1 -- 172.21.15.94:0/1196655599 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcbc009d20 con 0x7fdccc068c30 2024-09-10T23:38:06.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdcc8c00640 1 -- 172.21.15.94:0/1196655599 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc005280 con 0x7fdccc068c30 2024-09-10T23:38:06.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdcd2e00640 1 -- 172.21.15.94:0/1196655599 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdccc099b20 con 0x7fdccc068c30 2024-09-10T23:38:06.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdcd2e00640 1 -- 172.21.15.94:0/1196655599 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdccc0cd140 con 0x7fdccc068c30 2024-09-10T23:38:06.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdcd2e00640 1 -- 172.21.15.94:0/1196655599 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdccc0cfb40 con 0x7fdccc068c30 2024-09-10T23:38:06.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.001+0000 7fdcc8c00640 1 -- 172.21.15.94:0/1196655599 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7fdcbc005420 con 0x7fdccc068c30 2024-09-10T23:38:06.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.005+0000 7fdcc8c00640 1 --2- 172.21.15.94:0/1196655599 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdca8072a60 0x7fdca8074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:06.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.005+0000 7fdcc8c00640 1 -- 172.21.15.94:0/1196655599 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5868+0+0 (secure 0 0 0) 0x7fdcbc093d60 con 0x7fdccc068c30 2024-09-10T23:38:06.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.005+0000 7fdcc3400640 1 --2- 172.21.15.94:0/1196655599 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdca8072a60 0x7fdca8074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:06.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.005+0000 7fdcc3400640 1 --2- 172.21.15.94:0/1196655599 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdca8072a60 0x7fdca8074f20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40af000 tx=0x7fdcb401b070 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:06.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.009+0000 7fdcc8c00640 1 -- 172.21.15.94:0/1196655599 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdcbc05e540 con 0x7fdccc068c30 2024-09-10T23:38:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.205+0000 7fdcd2e00640 1 -- 172.21.15.94:0/1196655599 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdccc0ae0d0 con 0x7fdccc068c30 2024-09-10T23:38:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.205+0000 7fdcc8c00640 1 -- 172.21.15.94:0/1196655599 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5243 (secure 0 0 0) 0x7fdcbc0672b0 con 0x7fdccc068c30 2024-09-10T23:38:06.209 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:06.210 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":69,"btime":"2024-09-10T23:37:36:617338+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7011,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1105858824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1105858824},{"type":"v1","addr":"172.21.15.94:6834","nonce":1105858824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3277955689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3277955689},{"type":"v1","addr":"172.21.15.94:6835","nonce":3277955689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":195947,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/2607191387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2607191387},{"type":"v1","addr":"172.21.15.181:6801","nonce":2607191387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:37:25.941783+0000","modified":"2024-09-10T23:37:28.978060+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195950},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195950":{"gid":195950,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/2513770511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2513770511},{"type":"v1","addr":"172.21.15.181:6835","nonce":2513770511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195950,"qdb_cluster":[195950]},"id":8}]} 2024-09-10T23:38:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 69 2024-09-10T23:38:06.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.209+0000 7fdcd2e00640 1 -- 172.21.15.94:0/1196655599 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdca8072a60 msgr2=0x7fdca8074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.209+0000 7fdcd2e00640 1 --2- 172.21.15.94:0/1196655599 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdca8072a60 0x7fdca8074f20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40af000 tx=0x7fdcb401b070 comp rx=0 tx=0).stop 2024-09-10T23:38:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.209+0000 7fdcd2e00640 1 -- 172.21.15.94:0/1196655599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdccc068c30 msgr2=0x7fdccc0704f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.209+0000 7fdcd2e00640 1 --2- 172.21.15.94:0/1196655599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdccc068c30 0x7fdccc0704f0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00d720 tx=0x7fdcbc00dbe0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.209+0000 7fdcd2e00640 1 -- 172.21.15.94:0/1196655599 shutdown_connections 2024-09-10T23:38:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.209+0000 7fdcd2e00640 1 --2- 172.21.15.94:0/1196655599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdccc15bcc0 0x7fdccc1fb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.209+0000 7fdcd2e00640 1 --2- 172.21.15.94:0/1196655599 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdca8072a60 0x7fdca8074f20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.209+0000 7fdcd2e00640 1 --2- 172.21.15.94:0/1196655599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdccc0cf690 0x7fdccc070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.209+0000 7fdcd2e00640 1 --2- 172.21.15.94:0/1196655599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdccc068c30 0x7fdccc0704f0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.209+0000 7fdcd2e00640 1 -- 172.21.15.94:0/1196655599 >> 172.21.15.94:0/1196655599 conn(0x7fdccc0d02d0 msgr2=0x7fdccc156600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.209+0000 7fdcd2e00640 1 -- 172.21.15.94:0/1196655599 shutdown_connections 2024-09-10T23:38:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.209+0000 7fdcd2e00640 1 -- 172.21.15.94:0/1196655599 wait complete. 2024-09-10T23:38:06.230 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:38:06.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.369+0000 7fd298200640 1 -- 172.21.15.94:0/3371011925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2900af310 msgr2=0x7fd2900af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:06.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.369+0000 7fd298200640 1 --2- 172.21.15.94:0/3371011925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2900af310 0x7fd2900af7a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd290073820 tx=0x7fd28801cc60 comp rx=0 tx=0).stop 2024-09-10T23:38:06.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.369+0000 7fd298200640 1 -- 172.21.15.94:0/3371011925 shutdown_connections 2024-09-10T23:38:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.369+0000 7fd298200640 1 --2- 172.21.15.94:0/3371011925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2900af310 0x7fd2900af7a0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.369+0000 7fd298200640 1 --2- 172.21.15.94:0/3371011925 >> v2:172.21.15.181:3300/0 conn(0x7fd29015f5c0 0x7fd2900cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.369+0000 7fd298200640 1 -- 172.21.15.94:0/3371011925 >> 172.21.15.94:0/3371011925 conn(0x7fd2900d1c60 msgr2=0x7fd2900d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.369+0000 7fd298200640 1 -- 172.21.15.94:0/3371011925 shutdown_connections 2024-09-10T23:38:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.369+0000 7fd298200640 1 -- 172.21.15.94:0/3371011925 wait complete. 2024-09-10T23:38:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd298200640 1 Processor -- start 2024-09-10T23:38:06.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd298200640 1 -- start start 2024-09-10T23:38:06.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd298200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2900af310 0x7fd29006f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd298200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2900b2bb0 0x7fd2900c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd298200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd29015f5c0 0x7fd2901fb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd298200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd2901690b0 con 0x7fd2900af310 2024-09-10T23:38:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd298200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd290168f30 con 0x7fd29015f5c0 2024-09-10T23:38:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd298200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd290169230 con 0x7fd2900b2bb0 2024-09-10T23:38:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd295800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2900af310 0x7fd29006f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd295800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2900af310 0x7fd29006f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47622/0 (socket says 172.21.15.94:47622) 2024-09-10T23:38:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd295800640 1 -- 172.21.15.94:0/909860755 learned_addr learned my addr 172.21.15.94:0/909860755 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd296200640 1 --2- 172.21.15.94:0/909860755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd29015f5c0 0x7fd2901fb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd294e00640 1 --2- 172.21.15.94:0/909860755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2900b2bb0 0x7fd2900c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd295800640 1 -- 172.21.15.94:0/909860755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2900b2bb0 msgr2=0x7fd2900c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:06.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd295800640 1 --2- 172.21.15.94:0/909860755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2900b2bb0 0x7fd2900c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd295800640 1 -- 172.21.15.94:0/909860755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd29015f5c0 msgr2=0x7fd2901fb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd295800640 1 --2- 172.21.15.94:0/909860755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd29015f5c0 0x7fd2901fb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd295800640 1 -- 172.21.15.94:0/909860755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2901fb290 con 0x7fd2900af310 2024-09-10T23:38:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd294e00640 1 --2- 172.21.15.94:0/909860755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2900b2bb0 0x7fd2900c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd295800640 1 --2- 172.21.15.94:0/909860755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2900af310 0x7fd29006f900 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fd28400d720 tx=0x7fd28400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd28d600640 1 -- 172.21.15.94:0/909860755 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284017070 con 0x7fd2900af310 2024-09-10T23:38:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd28d600640 1 -- 172.21.15.94:0/909860755 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd284009d20 con 0x7fd2900af310 2024-09-10T23:38:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd28d600640 1 -- 172.21.15.94:0/909860755 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284005280 con 0x7fd2900af310 2024-09-10T23:38:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd298200640 1 -- 172.21.15.94:0/909860755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2901f8a60 con 0x7fd2900af310 2024-09-10T23:38:06.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd298200640 1 -- 172.21.15.94:0/909860755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd290166640 con 0x7fd2900af310 2024-09-10T23:38:06.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.373+0000 7fd298200640 1 -- 172.21.15.94:0/909860755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2900af7a0 con 0x7fd2900af310 2024-09-10T23:38:06.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.377+0000 7fd28d600640 1 -- 172.21.15.94:0/909860755 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7fd284005420 con 0x7fd2900af310 2024-09-10T23:38:06.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.377+0000 7fd28d600640 1 --2- 172.21.15.94:0/909860755 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd270072a60 0x7fd270074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:06.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.377+0000 7fd28d600640 1 -- 172.21.15.94:0/909860755 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5868+0+0 (secure 0 0 0) 0x7fd284093d10 con 0x7fd2900af310 2024-09-10T23:38:06.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.377+0000 7fd294e00640 1 --2- 172.21.15.94:0/909860755 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd270072a60 0x7fd270074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:06.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.377+0000 7fd294e00640 1 --2- 172.21.15.94:0/909860755 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd270072a60 0x7fd270074f20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd2900c7b80 tx=0x7fd27c009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:06.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.381+0000 7fd28d600640 1 -- 172.21.15.94:0/909860755 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd28405e4f0 con 0x7fd2900af310 2024-09-10T23:38:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.581+0000 7fd298200640 1 -- 172.21.15.94:0/909860755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2901fb460 con 0x7fd2900af310 2024-09-10T23:38:06.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.581+0000 7fd28d600640 1 -- 172.21.15.94:0/909860755 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5243 (secure 0 0 0) 0x7fd284067260 con 0x7fd2900af310 2024-09-10T23:38:06.583 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:06.583 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":69,"btime":"2024-09-10T23:37:36:617338+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7011,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1105858824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1105858824},{"type":"v1","addr":"172.21.15.94:6834","nonce":1105858824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3277955689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3277955689},{"type":"v1","addr":"172.21.15.94:6835","nonce":3277955689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":195947,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/2607191387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2607191387},{"type":"v1","addr":"172.21.15.181:6801","nonce":2607191387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:37:25.941783+0000","modified":"2024-09-10T23:37:28.978060+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":195950},"failed":[],"damaged":[],"stopped":[],"info":{"gid_195950":{"gid":195950,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/2513770511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2513770511},{"type":"v1","addr":"172.21.15.181:6835","nonce":2513770511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":195950,"qdb_cluster":[195950]},"id":8}]} 2024-09-10T23:38:06.584 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 69 2024-09-10T23:38:06.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.585+0000 7fd298200640 1 -- 172.21.15.94:0/909860755 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd270072a60 msgr2=0x7fd270074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.585+0000 7fd298200640 1 --2- 172.21.15.94:0/909860755 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd270072a60 0x7fd270074f20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd2900c7b80 tx=0x7fd27c009040 comp rx=0 tx=0).stop 2024-09-10T23:38:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.585+0000 7fd298200640 1 -- 172.21.15.94:0/909860755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2900af310 msgr2=0x7fd29006f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.585+0000 7fd298200640 1 --2- 172.21.15.94:0/909860755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2900af310 0x7fd29006f900 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fd28400d720 tx=0x7fd28400dbe0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.585+0000 7fd298200640 1 -- 172.21.15.94:0/909860755 shutdown_connections 2024-09-10T23:38:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.585+0000 7fd298200640 1 --2- 172.21.15.94:0/909860755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd29015f5c0 0x7fd2901fb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.585+0000 7fd298200640 1 --2- 172.21.15.94:0/909860755 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd270072a60 0x7fd270074f20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.585+0000 7fd298200640 1 --2- 172.21.15.94:0/909860755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2900b2bb0 0x7fd2900c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.585+0000 7fd298200640 1 --2- 172.21.15.94:0/909860755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2900af310 0x7fd29006f900 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.585+0000 7fd298200640 1 -- 172.21.15.94:0/909860755 >> 172.21.15.94:0/909860755 conn(0x7fd2900d1c60 msgr2=0x7fd2900b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.585+0000 7fd298200640 1 -- 172.21.15.94:0/909860755 shutdown_connections 2024-09-10T23:38:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.585+0000 7fd298200640 1 -- 172.21.15.94:0/909860755 wait complete. 2024-09-10T23:38:06.605 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:38:06.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.745+0000 7f9633c00640 1 -- 172.21.15.94:0/1254602839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f962c15f5c0 msgr2=0x7f962c0cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:06.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.745+0000 7f9633c00640 1 --2- 172.21.15.94:0/1254602839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f962c15f5c0 0x7f962c0cfb60 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f962c074670 tx=0x7f962001c500 comp rx=0 tx=0).stop 2024-09-10T23:38:06.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.745+0000 7f9633c00640 1 -- 172.21.15.94:0/1254602839 shutdown_connections 2024-09-10T23:38:06.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.745+0000 7f9633c00640 1 --2- 172.21.15.94:0/1254602839 >> v2:172.21.15.181:3300/0 conn(0x7f962c0d05e0 0x7f962c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.745+0000 7f9633c00640 1 --2- 172.21.15.94:0/1254602839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f962c0b2bb0 0x7f962c0d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.745+0000 7f9633c00640 1 --2- 172.21.15.94:0/1254602839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f962c15f5c0 0x7f962c0cfb60 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.745+0000 7f9633c00640 1 -- 172.21.15.94:0/1254602839 >> 172.21.15.94:0/1254602839 conn(0x7f962c0d1c60 msgr2=0x7f962c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:06.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.745+0000 7f9633c00640 1 -- 172.21.15.94:0/1254602839 shutdown_connections 2024-09-10T23:38:06.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.745+0000 7f9633c00640 1 -- 172.21.15.94:0/1254602839 wait complete. 2024-09-10T23:38:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.745+0000 7f9633c00640 1 Processor -- start 2024-09-10T23:38:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.745+0000 7f9633c00640 1 -- start start 2024-09-10T23:38:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9633c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f962c0b2bb0 0x7f962c070f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9633c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f962c0d05e0 0x7f962c15b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9633c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f962c15f5c0 0x7f962c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9633c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f962c0afb80 con 0x7f962c0b2bb0 2024-09-10T23:38:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9633c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f962c0afa00 con 0x7f962c0d05e0 2024-09-10T23:38:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9633c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f962c0afd00 con 0x7f962c15f5c0 2024-09-10T23:38:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9631200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f962c0b2bb0 0x7f962c070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:06.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9631200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f962c0b2bb0 0x7f962c070f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47648/0 (socket says 172.21.15.94:47648) 2024-09-10T23:38:06.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9631200640 1 -- 172.21.15.94:0/1885893855 learned_addr learned my addr 172.21.15.94:0/1885893855 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:06.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9631200640 1 -- 172.21.15.94:0/1885893855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f962c15f5c0 msgr2=0x7f962c06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:06.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f962be00640 1 --2- 172.21.15.94:0/1885893855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f962c0d05e0 0x7f962c15b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:06.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9631c00640 1 --2- 172.21.15.94:0/1885893855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f962c15f5c0 0x7f962c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9631200640 1 --2- 172.21.15.94:0/1885893855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f962c15f5c0 0x7f962c06dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9631200640 1 -- 172.21.15.94:0/1885893855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f962c0d05e0 msgr2=0x7f962c15b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9631200640 1 --2- 172.21.15.94:0/1885893855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f962c0d05e0 0x7f962c15b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9631200640 1 -- 172.21.15.94:0/1885893855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962c150a50 con 0x7f962c0b2bb0 2024-09-10T23:38:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f962be00640 1 --2- 172.21.15.94:0/1885893855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f962c0d05e0 0x7f962c15b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:38:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9631c00640 1 --2- 172.21.15.94:0/1885893855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f962c15f5c0 0x7f962c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:38:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9631200640 1 --2- 172.21.15.94:0/1885893855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f962c0b2bb0 0x7f962c070f00 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f96200ac040 tx=0x7f9620005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9628c00640 1 -- 172.21.15.94:0/1885893855 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620005430 con 0x7f962c0b2bb0 2024-09-10T23:38:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9628c00640 1 -- 172.21.15.94:0/1885893855 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96200aa730 con 0x7f962c0b2bb0 2024-09-10T23:38:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9633c00640 1 -- 172.21.15.94:0/1885893855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962c0714e0 con 0x7f962c0b2bb0 2024-09-10T23:38:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9628c00640 1 -- 172.21.15.94:0/1885893855 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96200aab00 con 0x7f962c0b2bb0 2024-09-10T23:38:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9633c00640 1 -- 172.21.15.94:0/1885893855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962c06ec50 con 0x7f962c0b2bb0 2024-09-10T23:38:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.749+0000 7f9633c00640 1 -- 172.21.15.94:0/1885893855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f962c0b02d0 con 0x7f962c0b2bb0 2024-09-10T23:38:06.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.753+0000 7f9628c00640 1 -- 172.21.15.94:0/1885893855 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f96200c6070 con 0x7f962c0b2bb0 2024-09-10T23:38:06.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.753+0000 7f9628c00640 1 --2- 172.21.15.94:0/1885893855 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f960c072a60 0x7f960c074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:06.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.753+0000 7f9628c00640 1 -- 172.21.15.94:0/1885893855 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5868+0+0 (secure 0 0 0) 0x7f962012ef40 con 0x7f962c0b2bb0 2024-09-10T23:38:06.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.753+0000 7f962be00640 1 --2- 172.21.15.94:0/1885893855 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f960c072a60 0x7f960c074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:06.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.753+0000 7f962be00640 1 --2- 172.21.15.94:0/1885893855 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f960c072a60 0x7f960c074f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f962c0cc140 tx=0x7f9618008040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.753+0000 7f9628c00640 1 -- 172.21.15.94:0/1885893855 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f96200c3060 con 0x7f962c0b2bb0 2024-09-10T23:38:06.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.925+0000 7f9633c00640 1 -- 172.21.15.94:0/1885893855 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f962c0749c0 con 0x7f962c0b2bb0 2024-09-10T23:38:06.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.925+0000 7f9628c00640 1 -- 172.21.15.94:0/1885893855 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16992 (secure 0 0 0) 0x7f96200164d0 con 0x7f962c0b2bb0 2024-09-10T23:38:06.927 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:06.927 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":103,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:38:03.839881+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:37:23.632549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T23:37:24.916517+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.5","mappings":[{"from":4,"to":3}]},{"pgid":"17.25","mappings":[{"from":6,"to":5}]},{"pgid":"17.31","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:38:06.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.925+0000 7f9633c00640 1 -- 172.21.15.94:0/1885893855 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f960c072a60 msgr2=0x7f960c074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:06.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.925+0000 7f9633c00640 1 --2- 172.21.15.94:0/1885893855 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f960c072a60 0x7f960c074f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f962c0cc140 tx=0x7f9618008040 comp rx=0 tx=0).stop 2024-09-10T23:38:06.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.925+0000 7f9633c00640 1 -- 172.21.15.94:0/1885893855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f962c0b2bb0 msgr2=0x7f962c070f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:06.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.925+0000 7f9633c00640 1 --2- 172.21.15.94:0/1885893855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f962c0b2bb0 0x7f962c070f00 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f96200ac040 tx=0x7f9620005040 comp rx=0 tx=0).stop 2024-09-10T23:38:06.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.925+0000 7f9633c00640 1 -- 172.21.15.94:0/1885893855 shutdown_connections 2024-09-10T23:38:06.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.925+0000 7f9633c00640 1 --2- 172.21.15.94:0/1885893855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f962c0d05e0 0x7f962c15b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.925+0000 7f9633c00640 1 --2- 172.21.15.94:0/1885893855 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f960c072a60 0x7f960c074f20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.925+0000 7f9633c00640 1 --2- 172.21.15.94:0/1885893855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f962c15f5c0 0x7f962c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.925+0000 7f9633c00640 1 --2- 172.21.15.94:0/1885893855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f962c0b2bb0 0x7f962c070f00 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:06.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.925+0000 7f9633c00640 1 -- 172.21.15.94:0/1885893855 >> 172.21.15.94:0/1885893855 conn(0x7f962c0d1c60 msgr2=0x7f962c151b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:06.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.925+0000 7f9633c00640 1 -- 172.21.15.94:0/1885893855 shutdown_connections 2024-09-10T23:38:06.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:06.925+0000 7f9633c00640 1 -- 172.21.15.94:0/1885893855 wait complete. 2024-09-10T23:38:06.948 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T23:38:07.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a7a00640 1 -- 172.21.15.94:0/1808164769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a013cc00 msgr2=0x7f42a015da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:07.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a7a00640 1 --2- 172.21.15.94:0/1808164769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a013cc00 0x7f42a015da70 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f429000b330 tx=0x7f429001d000 comp rx=0 tx=0).stop 2024-09-10T23:38:07.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a7a00640 1 -- 172.21.15.94:0/1808164769 shutdown_connections 2024-09-10T23:38:07.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a7a00640 1 --2- 172.21.15.94:0/1808164769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a013cc00 0x7f42a015da70 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:07.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a7a00640 1 -- 172.21.15.94:0/1808164769 >> 172.21.15.94:0/1808164769 conn(0x7f42a00cfaf0 msgr2=0x7f42a00cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:07.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a7a00640 1 -- 172.21.15.94:0/1808164769 shutdown_connections 2024-09-10T23:38:07.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a7a00640 1 -- 172.21.15.94:0/1808164769 wait complete. 2024-09-10T23:38:07.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a7a00640 1 Processor -- start 2024-09-10T23:38:07.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a7a00640 1 -- start start 2024-09-10T23:38:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a7a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42a013cc00 0x7f42a013b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a7a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42a013cfe0 0x7f42a0151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a7a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a013d530 0x7f42a01f9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a7a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f42a01685e0 con 0x7f42a013d530 2024-09-10T23:38:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a7a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f42a0168460 con 0x7f42a013cc00 2024-09-10T23:38:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a7a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f42a0168760 con 0x7f42a013cfe0 2024-09-10T23:38:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a5a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a013d530 0x7f42a01f9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a5a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a013d530 0x7f42a01f9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47674/0 (socket says 172.21.15.94:47674) 2024-09-10T23:38:07.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.089+0000 7f42a5a00640 1 -- 172.21.15.94:0/1108253451 learned_addr learned my addr 172.21.15.94:0/1108253451 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:07.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f42a5000640 1 --2- 172.21.15.94:0/1108253451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42a013cc00 0x7f42a013b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:07.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f42a5a00640 1 -- 172.21.15.94:0/1108253451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42a013cfe0 msgr2=0x7f42a0151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:07.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f4297e00640 1 --2- 172.21.15.94:0/1108253451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42a013cfe0 0x7f42a0151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:07.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f42a5a00640 1 --2- 172.21.15.94:0/1108253451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42a013cfe0 0x7f42a0151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:07.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f42a5a00640 1 -- 172.21.15.94:0/1108253451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42a013cc00 msgr2=0x7f42a013b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:07.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f42a5a00640 1 --2- 172.21.15.94:0/1108253451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42a013cc00 0x7f42a013b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:07.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f42a5a00640 1 -- 172.21.15.94:0/1108253451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a0151eb0 con 0x7f42a013d530 2024-09-10T23:38:07.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f42a5000640 1 --2- 172.21.15.94:0/1108253451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42a013cc00 0x7f42a013b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:38:07.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f4297e00640 1 --2- 172.21.15.94:0/1108253451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42a013cfe0 0x7f42a0151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:38:07.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f42a5a00640 1 --2- 172.21.15.94:0/1108253451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a013d530 0x7f42a01f9400 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4298002240 tx=0x7f429800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:07.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f429d600640 1 -- 172.21.15.94:0/1108253451 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298017070 con 0x7f42a013d530 2024-09-10T23:38:07.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f429d600640 1 -- 172.21.15.94:0/1108253451 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4298003c70 con 0x7f42a013d530 2024-09-10T23:38:07.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f429d600640 1 -- 172.21.15.94:0/1108253451 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42980052d0 con 0x7f42a013d530 2024-09-10T23:38:07.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f42a7a00640 1 -- 172.21.15.94:0/1108253451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a0152190 con 0x7f42a013d530 2024-09-10T23:38:07.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f42a7a00640 1 -- 172.21.15.94:0/1108253451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a01f8380 con 0x7f42a013d530 2024-09-10T23:38:07.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f42a7a00640 1 -- 172.21.15.94:0/1108253451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a01f8580 con 0x7f42a013d530 2024-09-10T23:38:07.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.093+0000 7f429d600640 1 -- 172.21.15.94:0/1108253451 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f4298005680 con 0x7f42a013d530 2024-09-10T23:38:07.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.097+0000 7f429d600640 1 --2- 172.21.15.94:0/1108253451 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f427c072a60 0x7f427c074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:07.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.097+0000 7f429d600640 1 -- 172.21.15.94:0/1108253451 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5868+0+0 (secure 0 0 0) 0x7f42980936d0 con 0x7f42a013d530 2024-09-10T23:38:07.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.097+0000 7f42a5000640 1 --2- 172.21.15.94:0/1108253451 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f427c072a60 0x7f427c074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:07.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.097+0000 7f42a5000640 1 --2- 172.21.15.94:0/1108253451 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f427c072a60 0x7f427c074f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f42900ae000 tx=0x7f429001b840 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:07.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.101+0000 7f429d600640 1 -- 172.21.15.94:0/1108253451 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f42a01f8580 con 0x7f42a013d530 2024-09-10T23:38:07.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.297+0000 7f42a7a00640 1 -- 172.21.15.94:0/1108253451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f42a015da70 con 0x7f42a013d530 2024-09-10T23:38:07.299 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-10T23:38:07.297+0000 7f4045a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:38:07.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.297+0000 7f429d600640 1 -- 172.21.15.94:0/1108253451 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) ==== 161+0+0 (secure 0 0 0) 0x7f429805e5d0 con 0x7f42a013d530 2024-09-10T23:38:07.301 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T23:38:07.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.301+0000 7f42a7a00640 1 -- 172.21.15.94:0/1108253451 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f427c072a60 msgr2=0x7f427c074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:07.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.301+0000 7f42a7a00640 1 --2- 172.21.15.94:0/1108253451 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f427c072a60 0x7f427c074f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f42900ae000 tx=0x7f429001b840 comp rx=0 tx=0).stop 2024-09-10T23:38:07.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.301+0000 7f42a7a00640 1 -- 172.21.15.94:0/1108253451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a013d530 msgr2=0x7f42a01f9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:07.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.301+0000 7f42a7a00640 1 --2- 172.21.15.94:0/1108253451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a013d530 0x7f42a01f9400 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4298002240 tx=0x7f429800cb90 comp rx=0 tx=0).stop 2024-09-10T23:38:07.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.301+0000 7f42a7a00640 1 -- 172.21.15.94:0/1108253451 shutdown_connections 2024-09-10T23:38:07.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.301+0000 7f42a7a00640 1 --2- 172.21.15.94:0/1108253451 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f427c072a60 0x7f427c074f20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:07.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.301+0000 7f42a7a00640 1 --2- 172.21.15.94:0/1108253451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a013d530 0x7f42a01f9400 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:07.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.301+0000 7f42a7a00640 1 --2- 172.21.15.94:0/1108253451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42a013cfe0 0x7f42a0151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:07.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.301+0000 7f42a7a00640 1 --2- 172.21.15.94:0/1108253451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42a013cc00 0x7f42a013b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:07.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.301+0000 7f42a7a00640 1 -- 172.21.15.94:0/1108253451 >> 172.21.15.94:0/1108253451 conn(0x7f42a00cfaf0 msgr2=0x7f42a013c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:07.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.301+0000 7f42a7a00640 1 -- 172.21.15.94:0/1108253451 shutdown_connections 2024-09-10T23:38:07.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.301+0000 7f42a7a00640 1 -- 172.21.15.94:0/1108253451 wait complete. 2024-09-10T23:38:07.323 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T23:38:07.372 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2024-09-10T23:38:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.465+0000 7fd7f3e00640 1 -- 172.21.15.94:0/387734497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7f415f140 msgr2=0x7fd7f40b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.465+0000 7fd7f3e00640 1 --2- 172.21.15.94:0/387734497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7f415f140 0x7fd7f40b1170 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4072ee0 tx=0x7fd7e401c890 comp rx=0 tx=0).stop 2024-09-10T23:38:07.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.465+0000 7fd7f3e00640 1 -- 172.21.15.94:0/387734497 shutdown_connections 2024-09-10T23:38:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.465+0000 7fd7f3e00640 1 --2- 172.21.15.94:0/387734497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7f415f140 0x7fd7f40b1170 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.465+0000 7fd7f3e00640 1 -- 172.21.15.94:0/387734497 >> 172.21.15.94:0/387734497 conn(0x7fd7f40d1b00 msgr2=0x7fd7f40d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.465+0000 7fd7f3e00640 1 -- 172.21.15.94:0/387734497 shutdown_connections 2024-09-10T23:38:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.465+0000 7fd7f3e00640 1 -- 172.21.15.94:0/387734497 wait complete. 2024-09-10T23:38:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f3e00640 1 Processor -- start 2024-09-10T23:38:07.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f3e00640 1 -- start start 2024-09-10T23:38:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7f40af150 0x7fd7f414b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7f40b2ac0 0x7fd7f40cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7f415f140 0x7fd7f414aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f3e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4169ae0 con 0x7fd7f415f140 2024-09-10T23:38:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f3e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4169960 con 0x7fd7f40b2ac0 2024-09-10T23:38:07.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f3e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f4169c60 con 0x7fd7f40af150 2024-09-10T23:38:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7f415f140 0x7fd7f414aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f2a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7f40af150 0x7fd7f414b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7f415f140 0x7fd7f414aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47696/0 (socket says 172.21.15.94:47696) 2024-09-10T23:38:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f3400640 1 -- 172.21.15.94:0/44245680 learned_addr learned my addr 172.21.15.94:0/44245680 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7ea000640 1 --2- 172.21.15.94:0/44245680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7f40b2ac0 0x7fd7f40cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f2a00640 1 -- 172.21.15.94:0/44245680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7f40b2ac0 msgr2=0x7fd7f40cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f2a00640 1 --2- 172.21.15.94:0/44245680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7f40b2ac0 0x7fd7f40cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f2a00640 1 -- 172.21.15.94:0/44245680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7f415f140 msgr2=0x7fd7f414aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f2a00640 1 --2- 172.21.15.94:0/44245680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7f415f140 0x7fd7f414aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:07.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f2a00640 1 -- 172.21.15.94:0/44245680 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f41fbab0 con 0x7fd7f40af150 2024-09-10T23:38:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f2a00640 1 --2- 172.21.15.94:0/44245680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7f40af150 0x7fd7f414b1e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40ac000 tx=0x7fd7e40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7eb400640 1 -- 172.21.15.94:0/44245680 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e40b2040 con 0x7fd7f40af150 2024-09-10T23:38:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7eb400640 1 -- 172.21.15.94:0/44245680 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7e4003600 con 0x7fd7f40af150 2024-09-10T23:38:07.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7eb400640 1 -- 172.21.15.94:0/44245680 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4003a50 con 0x7fd7f40af150 2024-09-10T23:38:07.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f3e00640 1 -- 172.21.15.94:0/44245680 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f40c6580 con 0x7fd7f40af150 2024-09-10T23:38:07.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.469+0000 7fd7f3e00640 1 -- 172.21.15.94:0/44245680 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f40cd6f0 con 0x7fd7f40af150 2024-09-10T23:38:07.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.473+0000 7fd7f3e00640 1 -- 172.21.15.94:0/44245680 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f40af5c0 con 0x7fd7f40af150 2024-09-10T23:38:07.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.473+0000 7fd7eb400640 1 -- 172.21.15.94:0/44245680 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7fd7e4003bf0 con 0x7fd7f40af150 2024-09-10T23:38:07.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.473+0000 7fd7eb400640 1 --2- 172.21.15.94:0/44245680 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd7d0072a90 0x7fd7d0074f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:07.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.473+0000 7fd7eb400640 1 -- 172.21.15.94:0/44245680 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(104..104 src has 1..104) ==== 5910+0+0 (secure 0 0 0) 0x7fd7e412db20 con 0x7fd7f40af150 2024-09-10T23:38:07.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.473+0000 7fd7ea000640 1 --2- 172.21.15.94:0/44245680 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd7d0072a90 0x7fd7d0074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:07.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.473+0000 7fd7ea000640 1 --2- 172.21.15.94:0/44245680 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd7d0072a90 0x7fd7d0074f50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40cc040 tx=0x7fd7dc007e40 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:07.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.477+0000 7fd7eb400640 1 -- 172.21.15.94:0/44245680 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd7e40164b0 con 0x7fd7f40af150 2024-09-10T23:38:07.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:07.677+0000 7fd7f3e00640 1 -- 172.21.15.94:0/44245680 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd7f406f2f0 con 0x7fd7f40af150 2024-09-10T23:38:08.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.305+0000 7fd7eb400640 1 -- 172.21.15.94:0/44245680 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v71) ==== 104+0+0 (secure 0 0 0) 0x7fd7e40f83e0 con 0x7fd7f40af150 2024-09-10T23:38:08.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.305+0000 7fd7f3e00640 1 -- 172.21.15.94:0/44245680 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd7d0072a90 msgr2=0x7fd7d0074f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:08.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.305+0000 7fd7f3e00640 1 --2- 172.21.15.94:0/44245680 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd7d0072a90 0x7fd7d0074f50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40cc040 tx=0x7fd7dc007e40 comp rx=0 tx=0).stop 2024-09-10T23:38:08.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.305+0000 7fd7f3e00640 1 -- 172.21.15.94:0/44245680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7f40af150 msgr2=0x7fd7f414b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:08.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.305+0000 7fd7f3e00640 1 --2- 172.21.15.94:0/44245680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7f40af150 0x7fd7f414b1e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40ac000 tx=0x7fd7e40040c0 comp rx=0 tx=0).stop 2024-09-10T23:38:08.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.309+0000 7fd7f3e00640 1 -- 172.21.15.94:0/44245680 shutdown_connections 2024-09-10T23:38:08.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.309+0000 7fd7f3e00640 1 --2- 172.21.15.94:0/44245680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7f40b2ac0 0x7fd7f40cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:08.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.309+0000 7fd7f3e00640 1 --2- 172.21.15.94:0/44245680 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd7d0072a90 0x7fd7d0074f50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:08.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.309+0000 7fd7f3e00640 1 --2- 172.21.15.94:0/44245680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7f415f140 0x7fd7f414aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:08.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.309+0000 7fd7f3e00640 1 --2- 172.21.15.94:0/44245680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7f40af150 0x7fd7f414b1e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:08.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.309+0000 7fd7f3e00640 1 -- 172.21.15.94:0/44245680 >> 172.21.15.94:0/44245680 conn(0x7fd7f40d1b00 msgr2=0x7fd7f406f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:08.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.309+0000 7fd7f3e00640 1 -- 172.21.15.94:0/44245680 shutdown_connections 2024-09-10T23:38:08.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.309+0000 7fd7f3e00640 1 -- 172.21.15.94:0/44245680 wait complete. 2024-09-10T23:38:08.331 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T23:38:08.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.477+0000 7f44e8e00640 1 -- 172.21.15.94:0/3417772968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c8004d50 msgr2=0x7f44c80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:08.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.477+0000 7f44e8e00640 1 --2- 172.21.15.94:0/3417772968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c8004d50 0x7f44c80051d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009b60 tx=0x7f44dc01d270 comp rx=0 tx=0).stop 2024-09-10T23:38:08.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.477+0000 7f44e3e00640 1 -- 172.21.15.94:0/3417772968 reap_dead start 2024-09-10T23:38:08.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.477+0000 7f44e8e00640 1 -- 172.21.15.94:0/3417772968 shutdown_connections 2024-09-10T23:38:08.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.477+0000 7f44e8e00640 1 -- 172.21.15.94:0/3417772968 >> 172.21.15.94:0/3417772968 conn(0x7f44e409aa70 msgr2=0x7f44e409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:08.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.477+0000 7f44e8e00640 1 -- 172.21.15.94:0/3417772968 shutdown_connections 2024-09-10T23:38:08.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.477+0000 7f44e8e00640 1 -- 172.21.15.94:0/3417772968 wait complete. 2024-09-10T23:38:08.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.477+0000 7f44e8e00640 1 Processor -- start 2024-09-10T23:38:08.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.477+0000 7f44e8e00640 1 -- start start 2024-09-10T23:38:08.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.477+0000 7f44e8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c8004d50 0x7f44e414bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:08.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.477+0000 7f44e8e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44e40b76c0 0x7f44e41f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:08.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.477+0000 7f44e8e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44e40caa10 0x7f44e40cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:08.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.477+0000 7f44e8e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f44e40b34d0 con 0x7f44c8004d50 2024-09-10T23:38:08.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.477+0000 7f44e8e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f44e40b3350 con 0x7f44e40b76c0 2024-09-10T23:38:08.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.477+0000 7f44e8e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f44e40af2c0 con 0x7f44e40caa10 2024-09-10T23:38:08.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44e3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c8004d50 0x7f44e414bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:08.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44e3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c8004d50 0x7f44e414bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47720/0 (socket says 172.21.15.94:47720) 2024-09-10T23:38:08.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44e3400640 1 -- 172.21.15.94:0/3575664480 learned_addr learned my addr 172.21.15.94:0/3575664480 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:08.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44e2a00640 1 --2- 172.21.15.94:0/3575664480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44e40b76c0 0x7f44e41f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:08.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44e3e00640 1 --2- 172.21.15.94:0/3575664480 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44e40caa10 0x7f44e40cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:08.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44e3400640 1 -- 172.21.15.94:0/3575664480 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44e40caa10 msgr2=0x7f44e40cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:08.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44e3400640 1 --2- 172.21.15.94:0/3575664480 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44e40caa10 0x7f44e40cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:08.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44e3400640 1 -- 172.21.15.94:0/3575664480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44e40b76c0 msgr2=0x7f44e41f1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:08.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44e3400640 1 --2- 172.21.15.94:0/3575664480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44e40b76c0 0x7f44e41f1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:08.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44e3400640 1 -- 172.21.15.94:0/3575664480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e40d2280 con 0x7f44c8004d50 2024-09-10T23:38:08.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44e3400640 1 --2- 172.21.15.94:0/3575664480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c8004d50 0x7f44e414bee0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f44d4006290 tx=0x7f44d4006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44db400640 1 -- 172.21.15.94:0/3575664480 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4015070 con 0x7f44c8004d50 2024-09-10T23:38:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44db400640 1 -- 172.21.15.94:0/3575664480 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44d400fd60 con 0x7f44c8004d50 2024-09-10T23:38:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44db400640 1 -- 172.21.15.94:0/3575664480 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4011690 con 0x7f44c8004d50 2024-09-10T23:38:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44e8e00640 1 -- 172.21.15.94:0/3575664480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e415c170 con 0x7f44c8004d50 2024-09-10T23:38:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44e8e00640 1 -- 172.21.15.94:0/3575664480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e415ee70 con 0x7f44c8004d50 2024-09-10T23:38:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44e8e00640 1 -- 172.21.15.94:0/3575664480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e40d01e0 con 0x7f44c8004d50 2024-09-10T23:38:08.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.481+0000 7f44db400640 1 -- 172.21.15.94:0/3575664480 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f44d4011830 con 0x7f44c8004d50 2024-09-10T23:38:08.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.485+0000 7f44db400640 1 --2- 172.21.15.94:0/3575664480 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44c4072a60 0x7f44c4074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:08.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.485+0000 7f44db400640 1 -- 172.21.15.94:0/3575664480 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5910+0+0 (secure 0 0 0) 0x7f44d4094c20 con 0x7f44c8004d50 2024-09-10T23:38:08.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.485+0000 7f44e2a00640 1 --2- 172.21.15.94:0/3575664480 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44c4072a60 0x7f44c4074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.485+0000 7f44e2a00640 1 --2- 172.21.15.94:0/3575664480 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44c4072a60 0x7f44c4074f20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f44e414e4f0 tx=0x7f44d00091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:08.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.489+0000 7f44db400640 1 -- 172.21.15.94:0/3575664480 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f44d405f460 con 0x7f44c8004d50 2024-09-10T23:38:08.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:08.653+0000 7f44e8e00640 1 -- 172.21.15.94:0/3575664480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f44e40b7aa0 con 0x7f44c8004d50 2024-09-10T23:38:09.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.305+0000 7f44db400640 1 -- 172.21.15.94:0/3575664480 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) ==== 181+0+0 (secure 0 0 0) 0x7f44d4063e90 con 0x7f44c8004d50 2024-09-10T23:38:09.309 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-09-10T23:38:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.309+0000 7f44e8e00640 1 -- 172.21.15.94:0/3575664480 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44c4072a60 msgr2=0x7f44c4074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.309+0000 7f44e8e00640 1 --2- 172.21.15.94:0/3575664480 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44c4072a60 0x7f44c4074f20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f44e414e4f0 tx=0x7f44d00091e0 comp rx=0 tx=0).stop 2024-09-10T23:38:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.309+0000 7f44e8e00640 1 -- 172.21.15.94:0/3575664480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c8004d50 msgr2=0x7f44e414bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.309+0000 7f44e8e00640 1 --2- 172.21.15.94:0/3575664480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c8004d50 0x7f44e414bee0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f44d4006290 tx=0x7f44d4006750 comp rx=0 tx=0).stop 2024-09-10T23:38:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.309+0000 7f44e8e00640 1 -- 172.21.15.94:0/3575664480 shutdown_connections 2024-09-10T23:38:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.309+0000 7f44e8e00640 1 --2- 172.21.15.94:0/3575664480 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44e40b76c0 0x7f44e41f1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.309+0000 7f44e8e00640 1 --2- 172.21.15.94:0/3575664480 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44c4072a60 0x7f44c4074f20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.309+0000 7f44e8e00640 1 --2- 172.21.15.94:0/3575664480 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44e40caa10 0x7f44e40cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.309+0000 7f44e8e00640 1 --2- 172.21.15.94:0/3575664480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c8004d50 0x7f44e414bee0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.309+0000 7f44e8e00640 1 -- 172.21.15.94:0/3575664480 >> 172.21.15.94:0/3575664480 conn(0x7f44e409aa70 msgr2=0x7f44e40b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.309+0000 7f44e8e00640 1 -- 172.21.15.94:0/3575664480 shutdown_connections 2024-09-10T23:38:09.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.309+0000 7f44e8e00640 1 -- 172.21.15.94:0/3575664480 wait complete. 2024-09-10T23:38:09.333 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T23:38:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.477+0000 7ff235400640 1 -- 172.21.15.94:0/2786301707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff214004810 msgr2=0x7ff214004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.477+0000 7ff235400640 1 --2- 172.21.15.94:0/2786301707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff214004810 0x7ff214004c90 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff218009b30 tx=0x7ff21801d220 comp rx=0 tx=0).stop 2024-09-10T23:38:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.477+0000 7ff22f200640 1 -- 172.21.15.94:0/2786301707 reap_dead start 2024-09-10T23:38:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.477+0000 7ff235400640 1 -- 172.21.15.94:0/2786301707 shutdown_connections 2024-09-10T23:38:09.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.477+0000 7ff235400640 1 -- 172.21.15.94:0/2786301707 >> 172.21.15.94:0/2786301707 conn(0x7ff23009aa70 msgr2=0x7ff23009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:09.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.477+0000 7ff235400640 1 -- 172.21.15.94:0/2786301707 shutdown_connections 2024-09-10T23:38:09.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.477+0000 7ff235400640 1 -- 172.21.15.94:0/2786301707 wait complete. 2024-09-10T23:38:09.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.477+0000 7ff235400640 1 Processor -- start 2024-09-10T23:38:09.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff235400640 1 -- start start 2024-09-10T23:38:09.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff235400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff214004810 0x7ff2300b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:09.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff235400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2300cbab0 0x7ff2300b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:09.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff235400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2300af630 0x7ff2301f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:09.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff235400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff230166cf0 con 0x7ff2300cbab0 2024-09-10T23:38:09.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff235400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff230166b70 con 0x7ff214004810 2024-09-10T23:38:09.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff235400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff230166e70 con 0x7ff2300af630 2024-09-10T23:38:09.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff225e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2300cbab0 0x7ff2300b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:09.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff225e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2300cbab0 0x7ff2300b7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47742/0 (socket says 172.21.15.94:47742) 2024-09-10T23:38:09.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff225e00640 1 -- 172.21.15.94:0/2159216406 learned_addr learned my addr 172.21.15.94:0/2159216406 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:09.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff22e800640 1 --2- 172.21.15.94:0/2159216406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff214004810 0x7ff2300b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:09.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff22f200640 1 --2- 172.21.15.94:0/2159216406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2300af630 0x7ff2301f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:09.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff225e00640 1 -- 172.21.15.94:0/2159216406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2300af630 msgr2=0x7ff2301f7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:09.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff225e00640 1 --2- 172.21.15.94:0/2159216406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2300af630 0x7ff2301f7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:09.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff225e00640 1 -- 172.21.15.94:0/2159216406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff214004810 msgr2=0x7ff2300b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:09.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff225e00640 1 --2- 172.21.15.94:0/2159216406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff214004810 0x7ff2300b53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:09.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff225e00640 1 -- 172.21.15.94:0/2159216406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2300cc900 con 0x7ff2300cbab0 2024-09-10T23:38:09.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff22f200640 1 --2- 172.21.15.94:0/2159216406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2300af630 0x7ff2301f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:09.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff22e800640 1 --2- 172.21.15.94:0/2159216406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff214004810 0x7ff2300b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:09.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff225e00640 1 --2- 172.21.15.94:0/2159216406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2300cbab0 0x7ff2300b7280 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff218009a80 tx=0x7ff2180acbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:09.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff227400640 1 -- 172.21.15.94:0/2159216406 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2180b1040 con 0x7ff2300cbab0 2024-09-10T23:38:09.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff227400640 1 -- 172.21.15.94:0/2159216406 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2180bc070 con 0x7ff2300cbab0 2024-09-10T23:38:09.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff235400640 1 -- 172.21.15.94:0/2159216406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2300ccbe0 con 0x7ff2300cbab0 2024-09-10T23:38:09.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff227400640 1 -- 172.21.15.94:0/2159216406 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff218005290 con 0x7ff2300cbab0 2024-09-10T23:38:09.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff235400640 1 -- 172.21.15.94:0/2159216406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2300afab0 con 0x7ff2300cbab0 2024-09-10T23:38:09.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.481+0000 7ff235400640 1 -- 172.21.15.94:0/2159216406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2300d1b80 con 0x7ff2300cbab0 2024-09-10T23:38:09.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.485+0000 7ff227400640 1 -- 172.21.15.94:0/2159216406 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7ff218005430 con 0x7ff2300cbab0 2024-09-10T23:38:09.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.485+0000 7ff227400640 1 --2- 172.21.15.94:0/2159216406 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff20c072a60 0x7ff20c074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:09.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.485+0000 7ff227400640 1 -- 172.21.15.94:0/2159216406 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5443+0+0 (secure 0 0 0) 0x7ff21812eb30 con 0x7ff2300cbab0 2024-09-10T23:38:09.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.485+0000 7ff22e800640 1 --2- 172.21.15.94:0/2159216406 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff20c072a60 0x7ff20c074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:09.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.485+0000 7ff22e800640 1 --2- 172.21.15.94:0/2159216406 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff20c072a60 0x7ff20c074f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff220002240 tx=0x7ff2200019d0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:09.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.489+0000 7ff227400640 1 -- 172.21.15.94:0/2159216406 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff218016f60 con 0x7ff2300cbab0 2024-09-10T23:38:09.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:09.661+0000 7ff235400640 1 -- 172.21.15.94:0/2159216406 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff2300afce0 con 0x7ff2300cbab0 2024-09-10T23:38:10.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.309+0000 7ff227400640 1 -- 172.21.15.94:0/2159216406 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) ==== 169+0+0 (secure 0 0 0) 0x7ff2180f9c30 con 0x7ff2300cbab0 2024-09-10T23:38:10.311 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-09-10T23:38:10.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.313+0000 7ff235400640 1 -- 172.21.15.94:0/2159216406 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff20c072a60 msgr2=0x7ff20c074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.313+0000 7ff235400640 1 --2- 172.21.15.94:0/2159216406 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff20c072a60 0x7ff20c074f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff220002240 tx=0x7ff2200019d0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.313+0000 7ff235400640 1 -- 172.21.15.94:0/2159216406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2300cbab0 msgr2=0x7ff2300b7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.313+0000 7ff235400640 1 --2- 172.21.15.94:0/2159216406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2300cbab0 0x7ff2300b7280 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff218009a80 tx=0x7ff2180acbb0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.313+0000 7ff235400640 1 -- 172.21.15.94:0/2159216406 shutdown_connections 2024-09-10T23:38:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.313+0000 7ff235400640 1 --2- 172.21.15.94:0/2159216406 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff20c072a60 0x7ff20c074f20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.313+0000 7ff235400640 1 --2- 172.21.15.94:0/2159216406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2300af630 0x7ff2301f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.313+0000 7ff235400640 1 --2- 172.21.15.94:0/2159216406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2300cbab0 0x7ff2300b7280 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.313+0000 7ff235400640 1 --2- 172.21.15.94:0/2159216406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff214004810 0x7ff2300b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.313+0000 7ff235400640 1 -- 172.21.15.94:0/2159216406 >> 172.21.15.94:0/2159216406 conn(0x7ff23009aa70 msgr2=0x7ff23015b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.313+0000 7ff235400640 1 -- 172.21.15.94:0/2159216406 shutdown_connections 2024-09-10T23:38:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.313+0000 7ff235400640 1 -- 172.21.15.94:0/2159216406 wait complete. 2024-09-10T23:38:10.340 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T23:38:10.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.481+0000 7fbd1f200640 1 -- 172.21.15.94:0/2123846961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd200b80b0 msgr2=0x7fbd20168ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.481+0000 7fbd1f200640 1 --2- 172.21.15.94:0/2123846961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd200b80b0 0x7fbd20168ab0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009c80 tx=0x7fbd1801f5a0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.481+0000 7fbd1f200640 1 -- 172.21.15.94:0/2123846961 shutdown_connections 2024-09-10T23:38:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.481+0000 7fbd1f200640 1 --2- 172.21.15.94:0/2123846961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd200b80b0 0x7fbd20168ab0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.481+0000 7fbd1f200640 1 -- 172.21.15.94:0/2123846961 >> 172.21.15.94:0/2123846961 conn(0x7fbd200d3d00 msgr2=0x7fbd200d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.481+0000 7fbd1f200640 1 -- 172.21.15.94:0/2123846961 shutdown_connections 2024-09-10T23:38:10.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.481+0000 7fbd1f200640 1 -- 172.21.15.94:0/2123846961 wait complete. 2024-09-10T23:38:10.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd1f200640 1 Processor -- start 2024-09-10T23:38:10.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd1f200640 1 -- start start 2024-09-10T23:38:10.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd1f200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd200b80b0 0x7fbd2015de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:10.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd1f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd201611f0 0x7fbd200ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:10.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd1f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd201615d0 0x7fbd20158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:10.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd1f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbd20168ff0 con 0x7fbd201611f0 2024-09-10T23:38:10.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd1f200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbd200d4a30 con 0x7fbd201615d0 2024-09-10T23:38:10.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd1f200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fbd20169170 con 0x7fbd200b80b0 2024-09-10T23:38:10.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd15400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd201611f0 0x7fbd200ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:10.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd15400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd201611f0 0x7fbd200ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47766/0 (socket says 172.21.15.94:47766) 2024-09-10T23:38:10.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd15400640 1 -- 172.21.15.94:0/3520713071 learned_addr learned my addr 172.21.15.94:0/3520713071 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:10.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd1de00640 1 --2- 172.21.15.94:0/3520713071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd200b80b0 0x7fbd2015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:10.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd1e800640 1 --2- 172.21.15.94:0/3520713071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd201615d0 0x7fbd20158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:10.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd15400640 1 -- 172.21.15.94:0/3520713071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd200b80b0 msgr2=0x7fbd2015de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:10.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd15400640 1 --2- 172.21.15.94:0/3520713071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd200b80b0 0x7fbd2015de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd15400640 1 -- 172.21.15.94:0/3520713071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd201615d0 msgr2=0x7fbd20158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:10.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd15400640 1 --2- 172.21.15.94:0/3520713071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd201615d0 0x7fbd20158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd15400640 1 -- 172.21.15.94:0/3520713071 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd201f87a0 con 0x7fbd201611f0 2024-09-10T23:38:10.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd15400640 1 --2- 172.21.15.94:0/3520713071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd201611f0 0x7fbd200ccc20 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800c3c0 tx=0x7fbd0800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:10.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd16a00640 1 -- 172.21.15.94:0/3520713071 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08017070 con 0x7fbd201611f0 2024-09-10T23:38:10.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd16a00640 1 -- 172.21.15.94:0/3520713071 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd080053c0 con 0x7fbd201611f0 2024-09-10T23:38:10.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd16a00640 1 -- 172.21.15.94:0/3520713071 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08003860 con 0x7fbd201611f0 2024-09-10T23:38:10.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd1f200640 1 -- 172.21.15.94:0/3520713071 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd201f8a80 con 0x7fbd201611f0 2024-09-10T23:38:10.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd1f200640 1 -- 172.21.15.94:0/3520713071 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd201f4930 con 0x7fbd201611f0 2024-09-10T23:38:10.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.485+0000 7fbd1f200640 1 -- 172.21.15.94:0/3520713071 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd201f85d0 con 0x7fbd201611f0 2024-09-10T23:38:10.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.489+0000 7fbd16a00640 1 -- 172.21.15.94:0/3520713071 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7fbd08003a00 con 0x7fbd201611f0 2024-09-10T23:38:10.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.489+0000 7fbd16a00640 1 --2- 172.21.15.94:0/3520713071 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbcfc072a60 0x7fbcfc074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:10.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.489+0000 7fbd16a00640 1 -- 172.21.15.94:0/3520713071 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7fbd08092e80 con 0x7fbd201611f0 2024-09-10T23:38:10.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.489+0000 7fbd1de00640 1 --2- 172.21.15.94:0/3520713071 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbcfc072a60 0x7fbcfc074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:10.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.489+0000 7fbd1de00640 1 --2- 172.21.15.94:0/3520713071 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbcfc072a60 0x7fbcfc074f20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbd10004e30 tx=0x7fbd10009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:10.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.493+0000 7fbd16a00640 1 -- 172.21.15.94:0/3520713071 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbd0805e0c0 con 0x7fbd201611f0 2024-09-10T23:38:10.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.661+0000 7fbd1f200640 1 -- 172.21.15.94:0/3520713071 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fbd200d1d80 con 0x7fbd201611f0 2024-09-10T23:38:10.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.665+0000 7fbd16a00640 1 -- 172.21.15.94:0/3520713071 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v39) ==== 71+0+0 (secure 0 0 0) 0x7fbd0805d9c0 con 0x7fbd201611f0 2024-09-10T23:38:10.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.665+0000 7fbd1f200640 1 -- 172.21.15.94:0/3520713071 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbcfc072a60 msgr2=0x7fbcfc074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:10.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.665+0000 7fbd1f200640 1 --2- 172.21.15.94:0/3520713071 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbcfc072a60 0x7fbcfc074f20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbd10004e30 tx=0x7fbd10009040 comp rx=0 tx=0).stop 2024-09-10T23:38:10.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.665+0000 7fbd1f200640 1 -- 172.21.15.94:0/3520713071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd201611f0 msgr2=0x7fbd200ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:10.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.665+0000 7fbd1f200640 1 --2- 172.21.15.94:0/3520713071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd201611f0 0x7fbd200ccc20 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800c3c0 tx=0x7fbd0800c880 comp rx=0 tx=0).stop 2024-09-10T23:38:10.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.665+0000 7fbd1f200640 1 -- 172.21.15.94:0/3520713071 shutdown_connections 2024-09-10T23:38:10.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.665+0000 7fbd1f200640 1 --2- 172.21.15.94:0/3520713071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd201615d0 0x7fbd20158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.665+0000 7fbd1f200640 1 --2- 172.21.15.94:0/3520713071 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbcfc072a60 0x7fbcfc074f20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.665+0000 7fbd1f200640 1 --2- 172.21.15.94:0/3520713071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd201611f0 0x7fbd200ccc20 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.665+0000 7fbd1f200640 1 --2- 172.21.15.94:0/3520713071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd200b80b0 0x7fbd2015de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.665+0000 7fbd1f200640 1 -- 172.21.15.94:0/3520713071 >> 172.21.15.94:0/3520713071 conn(0x7fbd200d3d00 msgr2=0x7fbd2006d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:10.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.665+0000 7fbd1f200640 1 -- 172.21.15.94:0/3520713071 shutdown_connections 2024-09-10T23:38:10.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.665+0000 7fbd1f200640 1 -- 172.21.15.94:0/3520713071 wait complete. 2024-09-10T23:38:10.688 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-10T23:38:10.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.829+0000 7fa80b600640 1 -- 172.21.15.94:0/2027201899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7e8004810 msgr2=0x7fa7e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:10.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.829+0000 7fa80b600640 1 --2- 172.21.15.94:0/2027201899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7e8004810 0x7fa7e8004c90 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa8040ce2b0 tx=0x7fa7ec01d1f0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.829+0000 7fa80b600640 1 -- 172.21.15.94:0/2027201899 shutdown_connections 2024-09-10T23:38:10.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.829+0000 7fa80b600640 1 --2- 172.21.15.94:0/2027201899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7e8004810 0x7fa7e8004c90 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.829+0000 7fa80b600640 1 -- 172.21.15.94:0/2027201899 >> 172.21.15.94:0/2027201899 conn(0x7fa8040cc900 msgr2=0x7fa8040ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:10.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.829+0000 7fa80b600640 1 -- 172.21.15.94:0/2027201899 shutdown_connections 2024-09-10T23:38:10.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.829+0000 7fa80b600640 1 -- 172.21.15.94:0/2027201899 wait complete. 2024-09-10T23:38:10.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa80b600640 1 Processor -- start 2024-09-10T23:38:10.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa80b600640 1 -- start start 2024-09-10T23:38:10.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa80b600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7e8004810 0x7fa804156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:10.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa80b600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8040c9330 0x7fa8040c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:10.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa80b600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8041fb620 0x7fa8041fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:10.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa80b600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa80415f410 con 0x7fa8041fb620 2024-09-10T23:38:10.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa80b600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa8040cb460 con 0x7fa8040c9330 2024-09-10T23:38:10.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa80b600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa80415f590 con 0x7fa7e8004810 2024-09-10T23:38:10.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa809600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8041fb620 0x7fa8041fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:10.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa809600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8041fb620 0x7fa8041fbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47770/0 (socket says 172.21.15.94:47770) 2024-09-10T23:38:10.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa809600640 1 -- 172.21.15.94:0/1678177391 learned_addr learned my addr 172.21.15.94:0/1678177391 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:10.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa808c00640 1 --2- 172.21.15.94:0/1678177391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7e8004810 0x7fa804156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:10.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa7fbe00640 1 --2- 172.21.15.94:0/1678177391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8040c9330 0x7fa8040c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:10.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa809600640 1 -- 172.21.15.94:0/1678177391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7e8004810 msgr2=0x7fa804156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:10.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa809600640 1 --2- 172.21.15.94:0/1678177391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7e8004810 0x7fa804156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa809600640 1 -- 172.21.15.94:0/1678177391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8040c9330 msgr2=0x7fa8040c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:10.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa809600640 1 --2- 172.21.15.94:0/1678177391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8040c9330 0x7fa8040c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:10.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa809600640 1 -- 172.21.15.94:0/1678177391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80406fb20 con 0x7fa8041fb620 2024-09-10T23:38:10.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa809600640 1 --2- 172.21.15.94:0/1678177391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8041fb620 0x7fa8041fbaa0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa804072990 tx=0x7fa7fc00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:10.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa801600640 1 -- 172.21.15.94:0/1678177391 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc00c940 con 0x7fa8041fb620 2024-09-10T23:38:10.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa801600640 1 -- 172.21.15.94:0/1678177391 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7fc017070 con 0x7fa8041fb620 2024-09-10T23:38:10.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa801600640 1 -- 172.21.15.94:0/1678177391 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc00ce70 con 0x7fa8041fb620 2024-09-10T23:38:10.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa80b600640 1 -- 172.21.15.94:0/1678177391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80406fe00 con 0x7fa8041fb620 2024-09-10T23:38:10.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa80b600640 1 -- 172.21.15.94:0/1678177391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8041fc270 con 0x7fa8041fb620 2024-09-10T23:38:10.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.833+0000 7fa80b600640 1 -- 172.21.15.94:0/1678177391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8040ad8c0 con 0x7fa8041fb620 2024-09-10T23:38:10.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.837+0000 7fa801600640 1 -- 172.21.15.94:0/1678177391 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7fa7fc01c460 con 0x7fa8041fb620 2024-09-10T23:38:10.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.837+0000 7fa801600640 1 --2- 172.21.15.94:0/1678177391 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa7e0072a60 0x7fa7e0074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:10.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.837+0000 7fa801600640 1 -- 172.21.15.94:0/1678177391 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7fa7fc093a30 con 0x7fa8041fb620 2024-09-10T23:38:10.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.837+0000 7fa808c00640 1 --2- 172.21.15.94:0/1678177391 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa7e0072a60 0x7fa7e0074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:10.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.837+0000 7fa808c00640 1 --2- 172.21.15.94:0/1678177391 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa7e0072a60 0x7fa7e0074f20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4006fd0 tx=0x7fa7f4006df0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:10.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:10.841+0000 7fa801600640 1 -- 172.21.15.94:0/1678177391 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa7fc05e6c0 con 0x7fa8041fb620 2024-09-10T23:38:11.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.065+0000 7fa80b600640 1 -- 172.21.15.94:0/1678177391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7fa8040c6930 con 0x7fa8041fb620 2024-09-10T23:38:11.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.309+0000 7fa801600640 1 -- 172.21.15.94:0/1678177391 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v307) ==== 149+0+0 (secure 0 0 0) 0x7fa7fc067430 con 0x7fa8041fb620 2024-09-10T23:38:11.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.313+0000 7fa80b600640 1 -- 172.21.15.94:0/1678177391 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa7e0072a60 msgr2=0x7fa7e0074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:11.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.313+0000 7fa80b600640 1 --2- 172.21.15.94:0/1678177391 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa7e0072a60 0x7fa7e0074f20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4006fd0 tx=0x7fa7f4006df0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.313+0000 7fa80b600640 1 -- 172.21.15.94:0/1678177391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8041fb620 msgr2=0x7fa8041fbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:11.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.313+0000 7fa80b600640 1 --2- 172.21.15.94:0/1678177391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8041fb620 0x7fa8041fbaa0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa804072990 tx=0x7fa7fc00ed10 comp rx=0 tx=0).stop 2024-09-10T23:38:11.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.313+0000 7fa80b600640 1 -- 172.21.15.94:0/1678177391 shutdown_connections 2024-09-10T23:38:11.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.313+0000 7fa80b600640 1 --2- 172.21.15.94:0/1678177391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8040c9330 0x7fa8040c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.313+0000 7fa80b600640 1 --2- 172.21.15.94:0/1678177391 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa7e0072a60 0x7fa7e0074f20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.313+0000 7fa80b600640 1 --2- 172.21.15.94:0/1678177391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8041fb620 0x7fa8041fbaa0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.313+0000 7fa80b600640 1 --2- 172.21.15.94:0/1678177391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7e8004810 0x7fa804156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.313+0000 7fa80b600640 1 -- 172.21.15.94:0/1678177391 >> 172.21.15.94:0/1678177391 conn(0x7fa8040cc900 msgr2=0x7fa8040ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:11.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.313+0000 7fa80b600640 1 -- 172.21.15.94:0/1678177391 shutdown_connections 2024-09-10T23:38:11.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.313+0000 7fa80b600640 1 -- 172.21.15.94:0/1678177391 wait complete. 2024-09-10T23:38:11.335 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-10T23:38:11.336 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-10T23:38:11.336 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T23:38:11.336 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:38:11.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ee00640 1 -- 172.21.15.94:0/3073570189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f778815cbc0 msgr2=0x7f77880cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:11.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ee00640 1 --2- 172.21.15.94:0/3073570189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f778815cbc0 0x7f77880cde20 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f77880d08b0 tx=0x7f777001c890 comp rx=0 tx=0).stop 2024-09-10T23:38:11.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ee00640 1 -- 172.21.15.94:0/3073570189 shutdown_connections 2024-09-10T23:38:11.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ee00640 1 --2- 172.21.15.94:0/3073570189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f778815cbc0 0x7f77880cde20 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ee00640 1 --2- 172.21.15.94:0/3073570189 >> v2:172.21.15.181:3300/0 conn(0x7f7788152c10 0x7f77880b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ee00640 1 -- 172.21.15.94:0/3073570189 >> 172.21.15.94:0/3073570189 conn(0x7f778806cdf0 msgr2=0x7f778806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:11.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ee00640 1 -- 172.21.15.94:0/3073570189 shutdown_connections 2024-09-10T23:38:11.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ee00640 1 -- 172.21.15.94:0/3073570189 wait complete. 2024-09-10T23:38:11.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ee00640 1 Processor -- start 2024-09-10T23:38:11.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ee00640 1 -- start start 2024-09-10T23:38:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ee00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77880cfd50 0x7f7788147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ee00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7788152c10 0x7f7788147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ee00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f778815cbc0 0x7f7788149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ee00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7788169360 con 0x7f7788152c10 2024-09-10T23:38:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ee00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f77881691e0 con 0x7f778815cbc0 2024-09-10T23:38:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ee00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f77881694e0 con 0x7f77880cfd50 2024-09-10T23:38:11.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f777f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7788152c10 0x7f7788147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f777f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7788152c10 0x7f7788147d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47790/0 (socket says 172.21.15.94:47790) 2024-09-10T23:38:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f777f400640 1 -- 172.21.15.94:0/1844063576 learned_addr learned my addr 172.21.15.94:0/1844063576 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f778ce00640 1 --2- 172.21.15.94:0/1844063576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f778815cbc0 0x7f7788149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.477+0000 7f7787e00640 1 --2- 172.21.15.94:0/1844063576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77880cfd50 0x7f7788147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f777f400640 1 -- 172.21.15.94:0/1844063576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77880cfd50 msgr2=0x7f7788147830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f777f400640 1 --2- 172.21.15.94:0/1844063576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77880cfd50 0x7f7788147830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f777f400640 1 -- 172.21.15.94:0/1844063576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f778815cbc0 msgr2=0x7f7788149d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f777f400640 1 --2- 172.21.15.94:0/1844063576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f778815cbc0 0x7f7788149d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f777f400640 1 -- 172.21.15.94:0/1844063576 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7788164af0 con 0x7f7788152c10 2024-09-10T23:38:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f778ce00640 1 --2- 172.21.15.94:0/1844063576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f778815cbc0 0x7f7788149d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f777f400640 1 --2- 172.21.15.94:0/1844063576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7788152c10 0x7f7788147d70 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f77700b0040 tx=0x7f7770005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:11.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f7784c00640 1 -- 172.21.15.94:0/1844063576 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77700b2040 con 0x7f7788152c10 2024-09-10T23:38:11.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f7784c00640 1 -- 172.21.15.94:0/1844063576 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7770005310 con 0x7f7788152c10 2024-09-10T23:38:11.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f7784c00640 1 -- 172.21.15.94:0/1844063576 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77700046d0 con 0x7f7788152c10 2024-09-10T23:38:11.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f778ee00640 1 -- 172.21.15.94:0/1844063576 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7788164dd0 con 0x7f7788152c10 2024-09-10T23:38:11.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f778ee00640 1 -- 172.21.15.94:0/1844063576 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7788165390 con 0x7f7788152c10 2024-09-10T23:38:11.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f778ee00640 1 -- 172.21.15.94:0/1844063576 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77880b29d0 con 0x7f7788152c10 2024-09-10T23:38:11.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f7784c00640 1 -- 172.21.15.94:0/1844063576 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f7770004890 con 0x7f7788152c10 2024-09-10T23:38:11.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f7784c00640 1 --2- 172.21.15.94:0/1844063576 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7764072a60 0x7f7764074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:11.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f7784c00640 1 -- 172.21.15.94:0/1844063576 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7f777012df40 con 0x7f7788152c10 2024-09-10T23:38:11.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.481+0000 7f7787e00640 1 --2- 172.21.15.94:0/1844063576 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7764072a60 0x7f7764074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:11.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.485+0000 7f7787e00640 1 --2- 172.21.15.94:0/1844063576 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7764072a60 0x7f7764074f20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7778005c50 tx=0x7f7778005be0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:11.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.485+0000 7f7784c00640 1 -- 172.21.15.94:0/1844063576 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f77700164b0 con 0x7f7788152c10 2024-09-10T23:38:11.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.653+0000 7f778ee00640 1 -- 172.21.15.94:0/1844063576 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77880b2560 con 0x7f7788152c10 2024-09-10T23:38:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.653+0000 7f7784c00640 1 -- 172.21.15.94:0/1844063576 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+13071 (secure 0 0 0) 0x7f77700f9100 con 0x7f7788152c10 2024-09-10T23:38:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:11.655 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":106,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:38:10.310953+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6835/2513770511":"2024-09-11T23:38:07.300686+0000","172.21.15.181:6834/2513770511":"2024-09-11T23:38:07.300686+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:38:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.653+0000 7f778ee00640 1 -- 172.21.15.94:0/1844063576 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7764072a60 msgr2=0x7f7764074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.653+0000 7f778ee00640 1 --2- 172.21.15.94:0/1844063576 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7764072a60 0x7f7764074f20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7778005c50 tx=0x7f7778005be0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.653+0000 7f778ee00640 1 -- 172.21.15.94:0/1844063576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7788152c10 msgr2=0x7f7788147d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.653+0000 7f778ee00640 1 --2- 172.21.15.94:0/1844063576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7788152c10 0x7f7788147d70 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f77700b0040 tx=0x7f7770005e00 comp rx=0 tx=0).stop 2024-09-10T23:38:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.653+0000 7f778ee00640 1 -- 172.21.15.94:0/1844063576 shutdown_connections 2024-09-10T23:38:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.653+0000 7f778ee00640 1 --2- 172.21.15.94:0/1844063576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f778815cbc0 0x7f7788149d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.653+0000 7f778ee00640 1 --2- 172.21.15.94:0/1844063576 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7764072a60 0x7f7764074f20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.653+0000 7f778ee00640 1 --2- 172.21.15.94:0/1844063576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7788152c10 0x7f7788147d70 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.653+0000 7f778ee00640 1 --2- 172.21.15.94:0/1844063576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77880cfd50 0x7f7788147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.653+0000 7f778ee00640 1 -- 172.21.15.94:0/1844063576 >> 172.21.15.94:0/1844063576 conn(0x7f778806cdf0 msgr2=0x7f7788150690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.653+0000 7f778ee00640 1 -- 172.21.15.94:0/1844063576 shutdown_connections 2024-09-10T23:38:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.653+0000 7f778ee00640 1 -- 172.21.15.94:0/1844063576 wait complete. 2024-09-10T23:38:11.678 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T23:32:31.372521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T23:38:11.679 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T23:38:11.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.821+0000 7f647a200640 1 -- 172.21.15.94:0/3288660151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64740c49b0 msgr2=0x7f64740cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:11.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.821+0000 7f647a200640 1 --2- 172.21.15.94:0/3288660151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64740c49b0 0x7f64740cadc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6460009840 tx=0x7f646001c820 comp rx=0 tx=0).stop 2024-09-10T23:38:11.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.821+0000 7f647a200640 1 -- 172.21.15.94:0/3288660151 shutdown_connections 2024-09-10T23:38:11.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.821+0000 7f647a200640 1 --2- 172.21.15.94:0/3288660151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64740c49b0 0x7f64740cadc0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.821+0000 7f647a200640 1 --2- 172.21.15.94:0/3288660151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6474158790 0x7f647406fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.821+0000 7f647a200640 1 -- 172.21.15.94:0/3288660151 >> 172.21.15.94:0/3288660151 conn(0x7f64740cfaf0 msgr2=0x7f64740cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:11.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.821+0000 7f647a200640 1 -- 172.21.15.94:0/3288660151 shutdown_connections 2024-09-10T23:38:11.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.821+0000 7f647a200640 1 -- 172.21.15.94:0/3288660151 wait complete. 2024-09-10T23:38:11.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.821+0000 7f647a200640 1 Processor -- start 2024-09-10T23:38:11.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f647a200640 1 -- start start 2024-09-10T23:38:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f647a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64740c49b0 0x7f64740c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f647a200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64740cb300 0x7f647414e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f647a200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6474158790 0x7f6474154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f647a200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6474168230 con 0x7f64740c49b0 2024-09-10T23:38:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f647a200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f64741680b0 con 0x7f6474158790 2024-09-10T23:38:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f647a200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f64741683b0 con 0x7f64740cb300 2024-09-10T23:38:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f6473400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64740c49b0 0x7f64740c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f6473400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64740c49b0 0x7f64740c8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47800/0 (socket says 172.21.15.94:47800) 2024-09-10T23:38:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f6473400640 1 -- 172.21.15.94:0/2069960219 learned_addr learned my addr 172.21.15.94:0/2069960219 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f6473e00640 1 --2- 172.21.15.94:0/2069960219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6474158790 0x7f6474154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f6472a00640 1 --2- 172.21.15.94:0/2069960219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64740cb300 0x7f647414e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f6473400640 1 -- 172.21.15.94:0/2069960219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64740cb300 msgr2=0x7f647414e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f6473400640 1 --2- 172.21.15.94:0/2069960219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64740cb300 0x7f647414e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f6473400640 1 -- 172.21.15.94:0/2069960219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6474158790 msgr2=0x7f6474154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f6473400640 1 --2- 172.21.15.94:0/2069960219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6474158790 0x7f6474154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f6473400640 1 -- 172.21.15.94:0/2069960219 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64740cf000 con 0x7f64740c49b0 2024-09-10T23:38:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f6473e00640 1 --2- 172.21.15.94:0/2069960219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6474158790 0x7f6474154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:11.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f6473400640 1 --2- 172.21.15.94:0/2069960219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64740c49b0 0x7f64740c8e70 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f646400e970 tx=0x7f646400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f646b400640 1 -- 172.21.15.94:0/2069960219 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646400cd80 con 0x7f64740c49b0 2024-09-10T23:38:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f646b400640 1 -- 172.21.15.94:0/2069960219 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6464004020 con 0x7f64740c49b0 2024-09-10T23:38:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f646b400640 1 -- 172.21.15.94:0/2069960219 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6464004470 con 0x7f64740c49b0 2024-09-10T23:38:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f647a200640 1 -- 172.21.15.94:0/2069960219 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64740c80f0 con 0x7f64740c49b0 2024-09-10T23:38:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f647a200640 1 -- 172.21.15.94:0/2069960219 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64740c8390 con 0x7f64740c49b0 2024-09-10T23:38:11.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.825+0000 7f647a200640 1 -- 172.21.15.94:0/2069960219 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64740cadc0 con 0x7f64740c49b0 2024-09-10T23:38:11.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.829+0000 7f646b400640 1 -- 172.21.15.94:0/2069960219 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f6464003600 con 0x7f64740c49b0 2024-09-10T23:38:11.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.829+0000 7f646b400640 1 --2- 172.21.15.94:0/2069960219 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6454072a90 0x7f6454074f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:11.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.829+0000 7f646b400640 1 -- 172.21.15.94:0/2069960219 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7f6464094230 con 0x7f64740c49b0 2024-09-10T23:38:11.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.829+0000 7f6472a00640 1 --2- 172.21.15.94:0/2069960219 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6454072a90 0x7f6454074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:11.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.829+0000 7f6472a00640 1 --2- 172.21.15.94:0/2069960219 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6454072a90 0x7f6454074f50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f64600af000 tx=0x7f646001b060 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:11.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.833+0000 7f646b400640 1 -- 172.21.15.94:0/2069960219 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f646405f470 con 0x7f64740c49b0 2024-09-10T23:38:12.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:11.997+0000 7f647a200640 1 -- 172.21.15.94:0/2069960219 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f64740d0850 con 0x7f64740c49b0 2024-09-10T23:38:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.001+0000 7f646b400640 1 -- 172.21.15.94:0/2069960219 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7f6464092e20 con 0x7f64740c49b0 2024-09-10T23:38:12.002 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-09-10T23:38:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.001+0000 7f647a200640 1 -- 172.21.15.94:0/2069960219 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6454072a90 msgr2=0x7f6454074f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.001+0000 7f647a200640 1 --2- 172.21.15.94:0/2069960219 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6454072a90 0x7f6454074f50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f64600af000 tx=0x7f646001b060 comp rx=0 tx=0).stop 2024-09-10T23:38:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.001+0000 7f647a200640 1 -- 172.21.15.94:0/2069960219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64740c49b0 msgr2=0x7f64740c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.001+0000 7f647a200640 1 --2- 172.21.15.94:0/2069960219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64740c49b0 0x7f64740c8e70 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f646400e970 tx=0x7f646400ee30 comp rx=0 tx=0).stop 2024-09-10T23:38:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.001+0000 7f647a200640 1 -- 172.21.15.94:0/2069960219 shutdown_connections 2024-09-10T23:38:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.001+0000 7f647a200640 1 --2- 172.21.15.94:0/2069960219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6474158790 0x7f6474154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.001+0000 7f647a200640 1 --2- 172.21.15.94:0/2069960219 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6454072a90 0x7f6454074f50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.001+0000 7f647a200640 1 --2- 172.21.15.94:0/2069960219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64740cb300 0x7f647414e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.001+0000 7f647a200640 1 --2- 172.21.15.94:0/2069960219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64740c49b0 0x7f64740c8e70 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.001+0000 7f647a200640 1 -- 172.21.15.94:0/2069960219 >> 172.21.15.94:0/2069960219 conn(0x7f64740cfaf0 msgr2=0x7f64740ccd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.001+0000 7f647a200640 1 -- 172.21.15.94:0/2069960219 shutdown_connections 2024-09-10T23:38:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.001+0000 7f647a200640 1 -- 172.21.15.94:0/2069960219 wait complete. 2024-09-10T23:38:12.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-10T23:38:12.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.161+0000 7f82e3800640 1 -- 172.21.15.94:0/2139744388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82dc15e290 msgr2=0x7f82dc0d03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:12.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.161+0000 7f82e3800640 1 --2- 172.21.15.94:0/2139744388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82dc15e290 0x7f82dc0d03d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f82dc06f830 tx=0x7f82d401cd50 comp rx=0 tx=0).stop 2024-09-10T23:38:12.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.161+0000 7f82e3800640 1 -- 172.21.15.94:0/2139744388 shutdown_connections 2024-09-10T23:38:12.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.161+0000 7f82e3800640 1 --2- 172.21.15.94:0/2139744388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82dc15e290 0x7f82dc0d03d0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:12.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.161+0000 7f82e3800640 1 --2- 172.21.15.94:0/2139744388 >> v2:172.21.15.181:3300/0 conn(0x7f82dc0cbbe0 0x7f82dc0cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:12.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.161+0000 7f82e3800640 1 -- 172.21.15.94:0/2139744388 >> 172.21.15.94:0/2139744388 conn(0x7f82dc06cdf0 msgr2=0x7f82dc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:12.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.161+0000 7f82e3800640 1 -- 172.21.15.94:0/2139744388 shutdown_connections 2024-09-10T23:38:12.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.161+0000 7f82e3800640 1 -- 172.21.15.94:0/2139744388 wait complete. 2024-09-10T23:38:12.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e3800640 1 Processor -- start 2024-09-10T23:38:12.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e3800640 1 -- start start 2024-09-10T23:38:12.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e3800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82dc0cbbe0 0x7f82dc1500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:12.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e3800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82dc0cbfc0 0x7f82dc138090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:12.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e3800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82dc15e290 0x7f82dc13ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:12.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e3800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc13f710 con 0x7f82dc0cbbe0 2024-09-10T23:38:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e3800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc13f590 con 0x7f82dc15e290 2024-09-10T23:38:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e3800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f82dc13f890 con 0x7f82dc0cbfc0 2024-09-10T23:38:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e0e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82dc0cbbe0 0x7f82dc1500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e0e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82dc0cbbe0 0x7f82dc1500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47812/0 (socket says 172.21.15.94:47812) 2024-09-10T23:38:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e0e00640 1 -- 172.21.15.94:0/281920969 learned_addr learned my addr 172.21.15.94:0/281920969 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e1800640 1 --2- 172.21.15.94:0/281920969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82dc15e290 0x7f82dc13ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82d3e00640 1 --2- 172.21.15.94:0/281920969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82dc0cbfc0 0x7f82dc138090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e0e00640 1 -- 172.21.15.94:0/281920969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82dc0cbfc0 msgr2=0x7f82dc138090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e0e00640 1 --2- 172.21.15.94:0/281920969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82dc0cbfc0 0x7f82dc138090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e0e00640 1 -- 172.21.15.94:0/281920969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82dc15e290 msgr2=0x7f82dc13ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e0e00640 1 --2- 172.21.15.94:0/281920969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82dc15e290 0x7f82dc13ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e0e00640 1 -- 172.21.15.94:0/281920969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82dc155580 con 0x7f82dc0cbbe0 2024-09-10T23:38:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e1800640 1 --2- 172.21.15.94:0/281920969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82dc15e290 0x7f82dc13ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:12.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e0e00640 1 --2- 172.21.15.94:0/281920969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82dc0cbbe0 0x7f82dc1500f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0095d0 tx=0x7f82cc009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82d9600640 1 -- 172.21.15.94:0/281920969 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc018020 con 0x7f82dc0cbbe0 2024-09-10T23:38:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82d9600640 1 -- 172.21.15.94:0/281920969 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82cc014910 con 0x7f82dc0cbbe0 2024-09-10T23:38:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e3800640 1 -- 172.21.15.94:0/281920969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82dc155860 con 0x7f82dc0cbbe0 2024-09-10T23:38:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82d9600640 1 -- 172.21.15.94:0/281920969 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc014d40 con 0x7f82dc0cbbe0 2024-09-10T23:38:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e3800640 1 -- 172.21.15.94:0/281920969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82dc155e20 con 0x7f82dc0cbbe0 2024-09-10T23:38:12.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.165+0000 7f82e3800640 1 -- 172.21.15.94:0/281920969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82dc13fde0 con 0x7f82dc0cbbe0 2024-09-10T23:38:12.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.169+0000 7f82d9600640 1 -- 172.21.15.94:0/281920969 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f82cc026430 con 0x7f82dc0cbbe0 2024-09-10T23:38:12.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.169+0000 7f82d9600640 1 --2- 172.21.15.94:0/281920969 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f82b8072a60 0x7f82b8074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:12.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.169+0000 7f82d9600640 1 -- 172.21.15.94:0/281920969 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7f82cc094180 con 0x7f82dc0cbbe0 2024-09-10T23:38:12.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.169+0000 7f82d3e00640 1 --2- 172.21.15.94:0/281920969 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f82b8072a60 0x7f82b8074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:12.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.169+0000 7f82d3e00640 1 --2- 172.21.15.94:0/281920969 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f82b8072a60 0x7f82b8074f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0cf940 tx=0x7f82c4009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:12.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.173+0000 7f82d9600640 1 -- 172.21.15.94:0/281920969 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f82cc022030 con 0x7f82dc0cbbe0 2024-09-10T23:38:12.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:12.401+0000 7f82e3800640 1 -- 172.21.15.94:0/281920969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7f82dc0b2c80 con 0x7f82dc0cbbe0 2024-09-10T23:38:13.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.321+0000 7f82d9600640 1 -- 172.21.15.94:0/281920969 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v309) ==== 151+0+0 (secure 0 0 0) 0x7f82cc011070 con 0x7f82dc0cbbe0 2024-09-10T23:38:13.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.325+0000 7f82e3800640 1 -- 172.21.15.94:0/281920969 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f82b8072a60 msgr2=0x7f82b8074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:13.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.325+0000 7f82e3800640 1 --2- 172.21.15.94:0/281920969 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f82b8072a60 0x7f82b8074f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0cf940 tx=0x7f82c4009040 comp rx=0 tx=0).stop 2024-09-10T23:38:13.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.325+0000 7f82e3800640 1 -- 172.21.15.94:0/281920969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82dc0cbbe0 msgr2=0x7f82dc1500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:13.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.325+0000 7f82e3800640 1 --2- 172.21.15.94:0/281920969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82dc0cbbe0 0x7f82dc1500f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0095d0 tx=0x7f82cc009a90 comp rx=0 tx=0).stop 2024-09-10T23:38:13.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.325+0000 7f82e3800640 1 -- 172.21.15.94:0/281920969 shutdown_connections 2024-09-10T23:38:13.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.325+0000 7f82e3800640 1 --2- 172.21.15.94:0/281920969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82dc15e290 0x7f82dc13ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.325+0000 7f82e3800640 1 --2- 172.21.15.94:0/281920969 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f82b8072a60 0x7f82b8074f20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.325+0000 7f82e3800640 1 --2- 172.21.15.94:0/281920969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82dc0cbfc0 0x7f82dc138090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.325+0000 7f82e3800640 1 --2- 172.21.15.94:0/281920969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f82dc0cbbe0 0x7f82dc1500f0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.325+0000 7f82e3800640 1 -- 172.21.15.94:0/281920969 >> 172.21.15.94:0/281920969 conn(0x7f82dc06cdf0 msgr2=0x7f82dc06e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:13.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.325+0000 7f82e3800640 1 -- 172.21.15.94:0/281920969 shutdown_connections 2024-09-10T23:38:13.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.325+0000 7f82e3800640 1 -- 172.21.15.94:0/281920969 wait complete. 2024-09-10T23:38:13.349 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T23:38:13.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.493+0000 7f117ea00640 1 -- 172.21.15.94:0/2133382710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f117815f5c0 msgr2=0x7f11780cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:13.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.493+0000 7f117ea00640 1 --2- 172.21.15.94:0/2133382710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f117815f5c0 0x7f11780cfb60 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f117806d8b0 tx=0x7f116801c7a0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.493+0000 7f117ea00640 1 -- 172.21.15.94:0/2133382710 shutdown_connections 2024-09-10T23:38:13.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.493+0000 7f117ea00640 1 --2- 172.21.15.94:0/2133382710 >> v2:172.21.15.181:3300/0 conn(0x7f11780b2bb0 0x7f11780d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.493+0000 7f117ea00640 1 --2- 172.21.15.94:0/2133382710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f117815f5c0 0x7f11780cfb60 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.493+0000 7f117ea00640 1 -- 172.21.15.94:0/2133382710 >> 172.21.15.94:0/2133382710 conn(0x7f11780d1c60 msgr2=0x7f11780d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:13.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.493+0000 7f117ea00640 1 -- 172.21.15.94:0/2133382710 shutdown_connections 2024-09-10T23:38:13.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.493+0000 7f117ea00640 1 -- 172.21.15.94:0/2133382710 wait complete. 2024-09-10T23:38:13.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f117ea00640 1 Processor -- start 2024-09-10T23:38:13.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f117ea00640 1 -- start start 2024-09-10T23:38:13.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f117ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780b2bb0 0x7f11780c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:13.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f117ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11780d05e0 0x7f1178155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:13.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f117ea00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f117815f5c0 0x7f1178151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:13.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f117ea00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11781692b0 con 0x7f11780b2bb0 2024-09-10T23:38:13.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f117ea00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1178169130 con 0x7f11780d05e0 2024-09-10T23:38:13.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f117ea00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1178169430 con 0x7f117815f5c0 2024-09-10T23:38:13.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f1177e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780b2bb0 0x7f11780c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:13.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f1177e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780b2bb0 0x7f11780c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47826/0 (socket says 172.21.15.94:47826) 2024-09-10T23:38:13.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f1177e00640 1 -- 172.21.15.94:0/3775444616 learned_addr learned my addr 172.21.15.94:0/3775444616 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:13.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f1177e00640 1 -- 172.21.15.94:0/3775444616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f117815f5c0 msgr2=0x7f1178151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:13.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f1177e00640 1 --2- 172.21.15.94:0/3775444616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f117815f5c0 0x7f1178151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f1177e00640 1 -- 172.21.15.94:0/3775444616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11780d05e0 msgr2=0x7f1178155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:13.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f1177e00640 1 --2- 172.21.15.94:0/3775444616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11780d05e0 0x7f1178155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f1177e00640 1 -- 172.21.15.94:0/3775444616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1178155fa0 con 0x7f11780b2bb0 2024-09-10T23:38:13.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f1177e00640 1 --2- 172.21.15.94:0/3775444616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780b2bb0 0x7f11780c70d0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f11680b0040 tx=0x7f1168005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:13.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f116fe00640 1 -- 172.21.15.94:0/3775444616 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11680b2040 con 0x7f11780b2bb0 2024-09-10T23:38:13.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f116fe00640 1 -- 172.21.15.94:0/3775444616 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11680a5ae0 con 0x7f11780b2bb0 2024-09-10T23:38:13.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f116fe00640 1 -- 172.21.15.94:0/3775444616 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168003600 con 0x7f11780b2bb0 2024-09-10T23:38:13.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f117ea00640 1 -- 172.21.15.94:0/3775444616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11780c76c0 con 0x7f11780b2bb0 2024-09-10T23:38:13.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f117ea00640 1 -- 172.21.15.94:0/3775444616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11780c8e60 con 0x7f11780b2bb0 2024-09-10T23:38:13.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.497+0000 7f117ea00640 1 -- 172.21.15.94:0/3775444616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11780b15c0 con 0x7f11780b2bb0 2024-09-10T23:38:13.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.501+0000 7f116fe00640 1 -- 172.21.15.94:0/3775444616 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f11680037a0 con 0x7f11780b2bb0 2024-09-10T23:38:13.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.501+0000 7f116fe00640 1 --2- 172.21.15.94:0/3775444616 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1158072a60 0x7f1158074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:13.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.501+0000 7f116fe00640 1 -- 172.21.15.94:0/3775444616 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7f116812d470 con 0x7f11780b2bb0 2024-09-10T23:38:13.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.501+0000 7f1177400640 1 --2- 172.21.15.94:0/3775444616 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1158072a60 0x7f1158074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:13.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.501+0000 7f1177400640 1 --2- 172.21.15.94:0/3775444616 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1158072a60 0x7f1158074f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1178152bb0 tx=0x7f1164005c80 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:13.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.505+0000 7f116fe00640 1 -- 172.21.15.94:0/3775444616 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f11680164b0 con 0x7f11780b2bb0 2024-09-10T23:38:13.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.669+0000 7f117ea00640 1 -- 172.21.15.94:0/3775444616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f11780749c0 con 0x7f11780b2bb0 2024-09-10T23:38:13.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.673+0000 7f116fe00640 1 -- 172.21.15.94:0/3775444616 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v39) ==== 59+0+1998 (secure 0 0 0) 0x7f11680f8710 con 0x7f11780b2bb0 2024-09-10T23:38:13.674 INFO:teuthology.orchestra.run.smithi094.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T23:38:13.674 INFO:teuthology.orchestra.run.smithi094.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T23:38:13.674 INFO:teuthology.orchestra.run.smithi094.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T23:38:13.674 INFO:teuthology.orchestra.run.smithi094.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T23:38:13.674 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3944.074506 2024-09-10T23:38:13.674 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3865.464527 2024-09-10T23:38:13.674 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3898.861024 2024-09-10T23:38:13.674 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4010.048529 2024-09-10T23:38:13.674 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4650.562436 2024-09-10T23:38:13.674 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4724.018776 2024-09-10T23:38:13.674 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5255.753663 2024-09-10T23:38:13.675 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4718.009443 2024-09-10T23:38:13.675 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds 20/20 2024-09-10T23:38:13.675 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T23:38:13.675 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_ms 2/2 2024-09-10T23:38:13.675 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_frag true 2024-09-10T23:38:13.675 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_scatterstat true 2024-09-10T23:38:13.675 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T23:38:13.675 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_verify_scatter true 2024-09-10T23:38:13.675 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T23:38:13.675 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T23:38:13.675 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T23:38:13.675 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced client_mount_timeout 600 2024-09-10T23:38:13.675 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_client 20/20 2024-09-10T23:38:13.676 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_ms 1/1 2024-09-10T23:38:13.676 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T23:38:13.676 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T23:38:13.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.673+0000 7f117ea00640 1 -- 172.21.15.94:0/3775444616 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1158072a60 msgr2=0x7f1158074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:13.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.673+0000 7f117ea00640 1 --2- 172.21.15.94:0/3775444616 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1158072a60 0x7f1158074f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1178152bb0 tx=0x7f1164005c80 comp rx=0 tx=0).stop 2024-09-10T23:38:13.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.673+0000 7f117ea00640 1 -- 172.21.15.94:0/3775444616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780b2bb0 msgr2=0x7f11780c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:13.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.673+0000 7f117ea00640 1 --2- 172.21.15.94:0/3775444616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780b2bb0 0x7f11780c70d0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f11680b0040 tx=0x7f1168005790 comp rx=0 tx=0).stop 2024-09-10T23:38:13.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.673+0000 7f117ea00640 1 -- 172.21.15.94:0/3775444616 shutdown_connections 2024-09-10T23:38:13.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.673+0000 7f117ea00640 1 --2- 172.21.15.94:0/3775444616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11780d05e0 0x7f1178155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.673+0000 7f117ea00640 1 --2- 172.21.15.94:0/3775444616 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1158072a60 0x7f1158074f20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.673+0000 7f117ea00640 1 --2- 172.21.15.94:0/3775444616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f117815f5c0 0x7f1178151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.673+0000 7f117ea00640 1 --2- 172.21.15.94:0/3775444616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11780b2bb0 0x7f11780c70d0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.673+0000 7f117ea00640 1 -- 172.21.15.94:0/3775444616 >> 172.21.15.94:0/3775444616 conn(0x7f11780d1c60 msgr2=0x7f11780d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:13.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.673+0000 7f117ea00640 1 -- 172.21.15.94:0/3775444616 shutdown_connections 2024-09-10T23:38:13.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.673+0000 7f117ea00640 1 -- 172.21.15.94:0/3775444616 wait complete. 2024-09-10T23:38:13.695 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:38:13.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.837+0000 7fe45e600640 1 -- 172.21.15.94:0/2846289785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe43c004810 msgr2=0x7fe43c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:13.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.837+0000 7fe45e600640 1 --2- 172.21.15.94:0/2846289785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe43c004810 0x7fe43c004c90 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe44000c470 tx=0x7fe44001f6a0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.837+0000 7fe45e600640 1 -- 172.21.15.94:0/2846289785 shutdown_connections 2024-09-10T23:38:13.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.837+0000 7fe45e600640 1 --2- 172.21.15.94:0/2846289785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe43c004810 0x7fe43c004c90 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.837+0000 7fe45e600640 1 -- 172.21.15.94:0/2846289785 >> 172.21.15.94:0/2846289785 conn(0x7fe4580cfaf0 msgr2=0x7fe4580cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:13.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.837+0000 7fe45e600640 1 -- 172.21.15.94:0/2846289785 shutdown_connections 2024-09-10T23:38:13.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.837+0000 7fe45e600640 1 -- 172.21.15.94:0/2846289785 wait complete. 2024-09-10T23:38:13.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe45e600640 1 Processor -- start 2024-09-10T23:38:13.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe45e600640 1 -- start start 2024-09-10T23:38:13.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe45e600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe43c004810 0x7fe4580aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:13.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe45e600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4581f8560 0x7fe4581f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:13.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe45e600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4580c6bc0 0x7fe4580c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:13.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe45e600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4580cde30 con 0x7fe4581f8560 2024-09-10T23:38:13.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe45e600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe4580c66c0 con 0x7fe4580c6bc0 2024-09-10T23:38:13.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe45e600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe4580cdfb0 con 0x7fe43c004810 2024-09-10T23:38:13.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe44ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4581f8560 0x7fe4581f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:13.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe44ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4581f8560 0x7fe4581f89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33198/0 (socket says 172.21.15.94:33198) 2024-09-10T23:38:13.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe44ea00640 1 -- 172.21.15.94:0/1597079962 learned_addr learned my addr 172.21.15.94:0/1597079962 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:13.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe457e00640 1 --2- 172.21.15.94:0/1597079962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4580c6bc0 0x7fe4580c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:13.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe457400640 1 --2- 172.21.15.94:0/1597079962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe43c004810 0x7fe4580aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:13.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe44ea00640 1 -- 172.21.15.94:0/1597079962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe43c004810 msgr2=0x7fe4580aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:13.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe44ea00640 1 --2- 172.21.15.94:0/1597079962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe43c004810 0x7fe4580aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe44ea00640 1 -- 172.21.15.94:0/1597079962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4580c6bc0 msgr2=0x7fe4580c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe44ea00640 1 --2- 172.21.15.94:0/1597079962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4580c6bc0 0x7fe4580c7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe44ea00640 1 -- 172.21.15.94:0/1597079962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4580ad8e0 con 0x7fe4581f8560 2024-09-10T23:38:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe457400640 1 --2- 172.21.15.94:0/1597079962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe43c004810 0x7fe4580aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe44ea00640 1 --2- 172.21.15.94:0/1597079962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4581f8560 0x7fe4581f89a0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe44001dda0 tx=0x7fe4400011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe44fe00640 1 -- 172.21.15.94:0/1597079962 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4400bc020 con 0x7fe4581f8560 2024-09-10T23:38:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe44fe00640 1 -- 172.21.15.94:0/1597079962 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe440006510 con 0x7fe4581f8560 2024-09-10T23:38:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe44fe00640 1 -- 172.21.15.94:0/1597079962 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4400b3040 con 0x7fe4581f8560 2024-09-10T23:38:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe45e600640 1 -- 172.21.15.94:0/1597079962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4580adba0 con 0x7fe4581f8560 2024-09-10T23:38:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.841+0000 7fe45e600640 1 -- 172.21.15.94:0/1597079962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4580ae160 con 0x7fe4581f8560 2024-09-10T23:38:13.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.845+0000 7fe45e600640 1 -- 172.21.15.94:0/1597079962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4580c5530 con 0x7fe4581f8560 2024-09-10T23:38:13.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.845+0000 7fe44fe00640 1 -- 172.21.15.94:0/1597079962 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7fe440003860 con 0x7fe4581f8560 2024-09-10T23:38:13.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.845+0000 7fe44fe00640 1 --2- 172.21.15.94:0/1597079962 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe434072a90 0x7fe434074f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:13.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.845+0000 7fe44fe00640 1 -- 172.21.15.94:0/1597079962 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7fe4401312f0 con 0x7fe4581f8560 2024-09-10T23:38:13.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.845+0000 7fe457400640 1 --2- 172.21.15.94:0/1597079962 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe434072a90 0x7fe434074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:13.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.845+0000 7fe457400640 1 --2- 172.21.15.94:0/1597079962 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe434072a90 0x7fe434074f50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe458052890 tx=0x7fe4480091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:13.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:13.849+0000 7fe44fe00640 1 -- 172.21.15.94:0/1597079962 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe4400c6030 con 0x7fe4581f8560 2024-09-10T23:38:14.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.017+0000 7fe45e600640 1 -- 172.21.15.94:0/1597079962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe4580cf6e0 con 0x7fe4581f8560 2024-09-10T23:38:14.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.017+0000 7fe44fe00640 1 -- 172.21.15.94:0/1597079962 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7fe4580701c0 con 0x7fe4581f8560 2024-09-10T23:38:14.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.021+0000 7fe45e600640 1 -- 172.21.15.94:0/1597079962 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe434072a90 msgr2=0x7fe434074f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:14.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.021+0000 7fe45e600640 1 --2- 172.21.15.94:0/1597079962 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe434072a90 0x7fe434074f50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe458052890 tx=0x7fe4480091e0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.021+0000 7fe45e600640 1 -- 172.21.15.94:0/1597079962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4581f8560 msgr2=0x7fe4581f89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:14.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.021+0000 7fe45e600640 1 --2- 172.21.15.94:0/1597079962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4581f8560 0x7fe4581f89a0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe44001dda0 tx=0x7fe4400011a0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.021+0000 7fe45e600640 1 -- 172.21.15.94:0/1597079962 shutdown_connections 2024-09-10T23:38:14.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.021+0000 7fe45e600640 1 --2- 172.21.15.94:0/1597079962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4580c6bc0 0x7fe4580c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.021+0000 7fe45e600640 1 --2- 172.21.15.94:0/1597079962 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe434072a90 0x7fe434074f50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.021+0000 7fe45e600640 1 --2- 172.21.15.94:0/1597079962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4581f8560 0x7fe4581f89a0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.021+0000 7fe45e600640 1 --2- 172.21.15.94:0/1597079962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe43c004810 0x7fe4580aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.021+0000 7fe45e600640 1 -- 172.21.15.94:0/1597079962 >> 172.21.15.94:0/1597079962 conn(0x7fe4580cfaf0 msgr2=0x7fe4580c5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:14.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.021+0000 7fe45e600640 1 -- 172.21.15.94:0/1597079962 shutdown_connections 2024-09-10T23:38:14.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.021+0000 7fe45e600640 1 -- 172.21.15.94:0/1597079962 wait complete. 2024-09-10T23:38:14.043 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:38:14.043 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:38:14.131 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:38:14.224 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:38:14.224 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:38:14.224 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:14.232 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:38:14.233 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:38:14.233 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:14.242 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:38:14.242 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:38:14.243 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:14.285 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:38:14.285 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:38:14.285 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:14.297 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:38:14.297 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T23:38:14.297 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:38:14.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe75aa00640 1 -- 172.21.15.94:0/1221880452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe75415ee80 msgr2=0x7fe7541657a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:14.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe75aa00640 1 --2- 172.21.15.94:0/1221880452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe75415ee80 0x7fe7541657a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009900 tx=0x7fe74c01c8e0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe75aa00640 1 -- 172.21.15.94:0/1221880452 shutdown_connections 2024-09-10T23:38:14.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe75aa00640 1 --2- 172.21.15.94:0/1221880452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe75415ee80 0x7fe7541657a0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe75aa00640 1 -- 172.21.15.94:0/1221880452 >> 172.21.15.94:0/1221880452 conn(0x7fe75406cdf0 msgr2=0x7fe75406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:14.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe75aa00640 1 -- 172.21.15.94:0/1221880452 shutdown_connections 2024-09-10T23:38:14.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe75aa00640 1 -- 172.21.15.94:0/1221880452 wait complete. 2024-09-10T23:38:14.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe75aa00640 1 Processor -- start 2024-09-10T23:38:14.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe75aa00640 1 -- start start 2024-09-10T23:38:14.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe754073ff0 0x7fe754152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:14.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7540d04b0 0x7fe75414df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:14.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe75415ee80 0x7fe754150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:14.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe75aa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe754168b60 con 0x7fe754073ff0 2024-09-10T23:38:14.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe75aa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe7541689e0 con 0x7fe75415ee80 2024-09-10T23:38:14.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe75aa00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe754168ce0 con 0x7fe7540d04b0 2024-09-10T23:38:14.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe753e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe754073ff0 0x7fe754152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:14.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe753e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe754073ff0 0x7fe754152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33210/0 (socket says 172.21.15.94:33210) 2024-09-10T23:38:14.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe753e00640 1 -- 172.21.15.94:0/1705968843 learned_addr learned my addr 172.21.15.94:0/1705968843 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:14.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe758a00640 1 --2- 172.21.15.94:0/1705968843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe75415ee80 0x7fe754150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:14.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe753e00640 1 -- 172.21.15.94:0/1705968843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7540d04b0 msgr2=0x7fe75414df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:14.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.473+0000 7fe753400640 1 --2- 172.21.15.94:0/1705968843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7540d04b0 0x7fe75414df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:14.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe753e00640 1 --2- 172.21.15.94:0/1705968843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7540d04b0 0x7fe75414df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe753e00640 1 -- 172.21.15.94:0/1705968843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe75415ee80 msgr2=0x7fe754150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:14.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe753e00640 1 --2- 172.21.15.94:0/1705968843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe75415ee80 0x7fe754150f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe753e00640 1 -- 172.21.15.94:0/1705968843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7541fb820 con 0x7fe754073ff0 2024-09-10T23:38:14.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe753400640 1 --2- 172.21.15.94:0/1705968843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7540d04b0 0x7fe75414df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:38:14.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe758a00640 1 --2- 172.21.15.94:0/1705968843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe75415ee80 0x7fe754150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:38:14.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe753e00640 1 --2- 172.21.15.94:0/1705968843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe754073ff0 0x7fe754152d40 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe74400c8a0 tx=0x7fe74400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:14.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe74be00640 1 -- 172.21.15.94:0/1705968843 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe744003b80 con 0x7fe754073ff0 2024-09-10T23:38:14.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe74be00640 1 -- 172.21.15.94:0/1705968843 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe744003d40 con 0x7fe754073ff0 2024-09-10T23:38:14.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe74be00640 1 -- 172.21.15.94:0/1705968843 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74400aab0 con 0x7fe754073ff0 2024-09-10T23:38:14.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe75aa00640 1 -- 172.21.15.94:0/1705968843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7541fb540 con 0x7fe754073ff0 2024-09-10T23:38:14.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe75aa00640 1 -- 172.21.15.94:0/1705968843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe754152540 con 0x7fe754073ff0 2024-09-10T23:38:14.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe75aa00640 1 -- 172.21.15.94:0/1705968843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7540cfda0 con 0x7fe754073ff0 2024-09-10T23:38:14.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe74be00640 1 -- 172.21.15.94:0/1705968843 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7fe74400ac50 con 0x7fe754073ff0 2024-09-10T23:38:14.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe74be00640 1 --2- 172.21.15.94:0/1705968843 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe734072a60 0x7fe734074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:14.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe74be00640 1 -- 172.21.15.94:0/1705968843 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7fe7440936d0 con 0x7fe754073ff0 2024-09-10T23:38:14.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.477+0000 7fe753400640 1 --2- 172.21.15.94:0/1705968843 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe734072a60 0x7fe734074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:14.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.481+0000 7fe753400640 1 --2- 172.21.15.94:0/1705968843 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe734072a60 0x7fe734074f20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe7540b2190 tx=0x7fe740009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:14.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.481+0000 7fe74be00640 1 -- 172.21.15.94:0/1705968843 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe74405e9b0 con 0x7fe754073ff0 2024-09-10T23:38:14.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.681+0000 7fe75aa00640 1 -- 172.21.15.94:0/1705968843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7540cf320 con 0x7fe754073ff0 2024-09-10T23:38:14.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.681+0000 7fe74be00640 1 -- 172.21.15.94:0/1705968843 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+3919 (secure 0 0 0) 0x7fe7440670d0 con 0x7fe754073ff0 2024-09-10T23:38:14.684 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:14.684 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":71,"btime":"2024-09-10T23:38:08:302621+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7009,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2420699740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2420699740},{"type":"v1","addr":"172.21.15.181:6835","nonce":2420699740}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7011,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1105858824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1105858824},{"type":"v1","addr":"172.21.15.94:6834","nonce":1105858824}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7014,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3277955689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3277955689},{"type":"v1","addr":"172.21.15.94:6835","nonce":3277955689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":195947,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/2607191387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2607191387},{"type":"v1","addr":"172.21.15.181:6801","nonce":2607191387}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[]} 2024-09-10T23:38:14.684 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 71 2024-09-10T23:38:14.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.685+0000 7fe75aa00640 1 -- 172.21.15.94:0/1705968843 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe734072a60 msgr2=0x7fe734074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:14.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.685+0000 7fe75aa00640 1 --2- 172.21.15.94:0/1705968843 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe734072a60 0x7fe734074f20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe7540b2190 tx=0x7fe740009040 comp rx=0 tx=0).stop 2024-09-10T23:38:14.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.685+0000 7fe75aa00640 1 -- 172.21.15.94:0/1705968843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe754073ff0 msgr2=0x7fe754152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:14.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.685+0000 7fe75aa00640 1 --2- 172.21.15.94:0/1705968843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe754073ff0 0x7fe754152d40 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe74400c8a0 tx=0x7fe74400cd60 comp rx=0 tx=0).stop 2024-09-10T23:38:14.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.685+0000 7fe75aa00640 1 -- 172.21.15.94:0/1705968843 shutdown_connections 2024-09-10T23:38:14.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.685+0000 7fe75aa00640 1 --2- 172.21.15.94:0/1705968843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe75415ee80 0x7fe754150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.685+0000 7fe75aa00640 1 --2- 172.21.15.94:0/1705968843 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe734072a60 0x7fe734074f20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.685+0000 7fe75aa00640 1 --2- 172.21.15.94:0/1705968843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7540d04b0 0x7fe75414df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.685+0000 7fe75aa00640 1 --2- 172.21.15.94:0/1705968843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe754073ff0 0x7fe754152d40 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.685+0000 7fe75aa00640 1 -- 172.21.15.94:0/1705968843 >> 172.21.15.94:0/1705968843 conn(0x7fe75406cdf0 msgr2=0x7fe754159b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:14.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.685+0000 7fe75aa00640 1 -- 172.21.15.94:0/1705968843 shutdown_connections 2024-09-10T23:38:14.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.685+0000 7fe75aa00640 1 -- 172.21.15.94:0/1705968843 wait complete. 2024-09-10T23:38:14.705 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T23:38:14.706 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T23:38:14.706 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T23:38:14.706 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:38:14.706 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T23:38:14.706 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T23:38:14.706 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T23:38:14.706 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:38:14.707 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T23:38:14.707 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T23:38:14.707 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T23:38:14.707 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:38:14.707 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T23:38:14.707 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T23:38:14.707 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T23:38:14.707 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:38:14.720 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T23:38:14.720 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T23:38:14.721 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T23:38:14.721 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T23:38:14.723 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T23:38:14.723 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T23:38:14.724 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T23:38:14.724 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T23:38:14.769 INFO:tasks.ceph.mds.a:Started 2024-09-10T23:38:14.769 INFO:tasks.ceph.mds.d:Started 2024-09-10T23:38:14.769 DEBUG:teuthology.parallel:result is None 2024-09-10T23:38:14.770 DEBUG:teuthology.parallel:result is None 2024-09-10T23:38:14.770 INFO:tasks.ceph.mds.b:Started 2024-09-10T23:38:14.770 DEBUG:teuthology.parallel:result is None 2024-09-10T23:38:14.771 INFO:tasks.ceph.mds.c:Started 2024-09-10T23:38:14.771 DEBUG:teuthology.parallel:result is None 2024-09-10T23:38:14.771 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T23:38:14.854 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-09-10T23:38:14.859 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-09-10T23:38:14.861 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2024-09-10T23:38:14.861 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:38:14.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2974c00640 1 -- 172.21.15.94:0/2661980699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700ce4e0 msgr2=0x7f29700ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:14.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2974c00640 1 --2- 172.21.15.94:0/2661980699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700ce4e0 0x7f29700ce8c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f2960009900 tx=0x7f296001c850 comp rx=0 tx=0).stop 2024-09-10T23:38:14.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2974c00640 1 -- 172.21.15.94:0/2661980699 shutdown_connections 2024-09-10T23:38:14.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2974c00640 1 --2- 172.21.15.94:0/2661980699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29700cd470 0x7f297006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2974c00640 1 --2- 172.21.15.94:0/2661980699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700ce4e0 0x7f29700ce8c0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2974c00640 1 -- 172.21.15.94:0/2661980699 >> 172.21.15.94:0/2661980699 conn(0x7f29700d1c60 msgr2=0x7f29700d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2974c00640 1 -- 172.21.15.94:0/2661980699 shutdown_connections 2024-09-10T23:38:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2974c00640 1 -- 172.21.15.94:0/2661980699 wait complete. 2024-09-10T23:38:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2974c00640 1 Processor -- start 2024-09-10T23:38:14.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2974c00640 1 -- start start 2024-09-10T23:38:14.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2974c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29700cd470 0x7f2970071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:14.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2974c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700ce4e0 0x7f297014ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:14.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2974c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29700cee00 0x7f297014f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:14.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2974c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f297016b5e0 con 0x7f29700ce4e0 2024-09-10T23:38:14.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2974c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f297016b460 con 0x7f29700cd470 2024-09-10T23:38:14.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2974c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f297016b760 con 0x7f29700cee00 2024-09-10T23:38:14.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2965400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700ce4e0 0x7f297014ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:14.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2965400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700ce4e0 0x7f297014ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33246/0 (socket says 172.21.15.94:33246) 2024-09-10T23:38:14.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f2965400640 1 -- 172.21.15.94:0/978431152 learned_addr learned my addr 172.21.15.94:0/978431152 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:14.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f296de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29700cd470 0x7f2970071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:14.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.921+0000 7f296e800640 1 --2- 172.21.15.94:0/978431152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29700cee00 0x7f297014f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:14.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.925+0000 7f2965400640 1 -- 172.21.15.94:0/978431152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29700cee00 msgr2=0x7f297014f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:14.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.925+0000 7f2965400640 1 --2- 172.21.15.94:0/978431152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29700cee00 0x7f297014f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.925+0000 7f2965400640 1 -- 172.21.15.94:0/978431152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29700cd470 msgr2=0x7f2970071bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:14.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.925+0000 7f2965400640 1 --2- 172.21.15.94:0/978431152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29700cd470 0x7f2970071bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:14.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.925+0000 7f2965400640 1 -- 172.21.15.94:0/978431152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2970153d80 con 0x7f29700ce4e0 2024-09-10T23:38:14.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.925+0000 7f296de00640 1 --2- 172.21.15.94:0/978431152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29700cd470 0x7f2970071bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:38:14.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.925+0000 7f2965400640 1 --2- 172.21.15.94:0/978431152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700ce4e0 0x7f297014ef60 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f295800c430 tx=0x7f295800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:14.927 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-10T23:38:14.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.925+0000 7f2966a00640 1 -- 172.21.15.94:0/978431152 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958017070 con 0x7f29700ce4e0 2024-09-10T23:38:14.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.925+0000 7f2966a00640 1 -- 172.21.15.94:0/978431152 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29580053c0 con 0x7f29700ce4e0 2024-09-10T23:38:14.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.925+0000 7f2966a00640 1 -- 172.21.15.94:0/978431152 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958003860 con 0x7f29700ce4e0 2024-09-10T23:38:14.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.925+0000 7f2974c00640 1 -- 172.21.15.94:0/978431152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2970154060 con 0x7f29700ce4e0 2024-09-10T23:38:14.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.925+0000 7f2974c00640 1 -- 172.21.15.94:0/978431152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2970154620 con 0x7f29700ce4e0 2024-09-10T23:38:14.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.925+0000 7f2974c00640 1 -- 172.21.15.94:0/978431152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29700ce8c0 con 0x7f29700ce4e0 2024-09-10T23:38:14.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.925+0000 7f2966a00640 1 -- 172.21.15.94:0/978431152 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f2958003a00 con 0x7f29700ce4e0 2024-09-10T23:38:14.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.925+0000 7f2966a00640 1 --2- 172.21.15.94:0/978431152 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f294c072a90 0x7f294c074f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:14.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.929+0000 7f2966a00640 1 -- 172.21.15.94:0/978431152 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4971+0+0 (secure 0 0 0) 0x7f2958092dd0 con 0x7f29700ce4e0 2024-09-10T23:38:14.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.929+0000 7f296de00640 1 --2- 172.21.15.94:0/978431152 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f294c072a90 0x7f294c074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:14.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.929+0000 7f296de00640 1 --2- 172.21.15.94:0/978431152 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f294c072a90 0x7f294c074f50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f29600af000 tx=0x7f296001b090 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:14.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:14.929+0000 7f2966a00640 1 -- 172.21.15.94:0/978431152 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f29580623f0 con 0x7f29700ce4e0 2024-09-10T23:38:15.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.105+0000 7f2974c00640 1 -- 172.21.15.94:0/978431152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f297015b6a0 con 0x7f29700ce4e0 2024-09-10T23:38:15.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.337+0000 7f2966a00640 1 -- 172.21.15.94:0/978431152 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v108) ==== 97+0+0 (secure 0 0 0) 0x7f2958066e10 con 0x7f29700ce4e0 2024-09-10T23:38:15.338 INFO:teuthology.orchestra.run.smithi094.stderr: removed all blocklist entries 2024-09-10T23:38:15.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.337+0000 7f2974c00640 1 -- 172.21.15.94:0/978431152 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f294c072a90 msgr2=0x7f294c074f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:15.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.337+0000 7f2974c00640 1 --2- 172.21.15.94:0/978431152 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f294c072a90 0x7f294c074f50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f29600af000 tx=0x7f296001b090 comp rx=0 tx=0).stop 2024-09-10T23:38:15.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.337+0000 7f2974c00640 1 -- 172.21.15.94:0/978431152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700ce4e0 msgr2=0x7f297014ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:15.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.337+0000 7f2974c00640 1 --2- 172.21.15.94:0/978431152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700ce4e0 0x7f297014ef60 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f295800c430 tx=0x7f295800c8f0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.337+0000 7f2974c00640 1 -- 172.21.15.94:0/978431152 shutdown_connections 2024-09-10T23:38:15.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.337+0000 7f2974c00640 1 --2- 172.21.15.94:0/978431152 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f294c072a90 0x7f294c074f50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.337+0000 7f2974c00640 1 --2- 172.21.15.94:0/978431152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29700cee00 0x7f297014f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.337+0000 7f2974c00640 1 --2- 172.21.15.94:0/978431152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29700ce4e0 0x7f297014ef60 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.337+0000 7f2974c00640 1 --2- 172.21.15.94:0/978431152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29700cd470 0x7f2970071bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.337+0000 7f2974c00640 1 -- 172.21.15.94:0/978431152 >> 172.21.15.94:0/978431152 conn(0x7f29700d1c60 msgr2=0x7f29700b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:15.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.337+0000 7f2974c00640 1 -- 172.21.15.94:0/978431152 shutdown_connections 2024-09-10T23:38:15.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.337+0000 7f2974c00640 1 -- 172.21.15.94:0/978431152 wait complete. 2024-09-10T23:38:15.362 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T23:38:15.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.505+0000 7f87fda00640 1 -- 172.21.15.94:0/1048950292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87dc0052f0 msgr2=0x7f87dc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:15.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.505+0000 7f87fda00640 1 --2- 172.21.15.94:0/1048950292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87dc0052f0 0x7f87dc00b9d0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f87f000c610 tx=0x7f87f001fa40 comp rx=0 tx=0).stop 2024-09-10T23:38:15.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.505+0000 7f87fda00640 1 -- 172.21.15.94:0/1048950292 shutdown_connections 2024-09-10T23:38:15.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.505+0000 7f87fda00640 1 --2- 172.21.15.94:0/1048950292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87dc0052f0 0x7f87dc00b9d0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.505+0000 7f87fda00640 1 -- 172.21.15.94:0/1048950292 >> 172.21.15.94:0/1048950292 conn(0x7f87f80d0b70 msgr2=0x7f87f80d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:15.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.505+0000 7f87fda00640 1 -- 172.21.15.94:0/1048950292 shutdown_connections 2024-09-10T23:38:15.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.505+0000 7f87fda00640 1 -- 172.21.15.94:0/1048950292 wait complete. 2024-09-10T23:38:15.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.505+0000 7f87fda00640 1 Processor -- start 2024-09-10T23:38:15.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.505+0000 7f87fda00640 1 -- start start 2024-09-10T23:38:15.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87fda00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87dc0052f0 0x7f87f8142a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:15.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87fda00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87f81e8760 0x7f87f80cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:15.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87fda00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f81f7d10 0x7f87f81f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:15.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87fda00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87f8070cb0 con 0x7f87f81f7d10 2024-09-10T23:38:15.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87fda00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f87f815b1a0 con 0x7f87dc0052f0 2024-09-10T23:38:15.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87fda00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f87f8070e30 con 0x7f87f81e8760 2024-09-10T23:38:15.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87f7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f81f7d10 0x7f87f81f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:15.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87f7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f81f7d10 0x7f87f81f81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33258/0 (socket says 172.21.15.94:33258) 2024-09-10T23:38:15.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87f7400640 1 -- 172.21.15.94:0/3859416740 learned_addr learned my addr 172.21.15.94:0/3859416740 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:15.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87f6a00640 1 --2- 172.21.15.94:0/3859416740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87dc0052f0 0x7f87f8142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:15.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87ee000640 1 --2- 172.21.15.94:0/3859416740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87f81e8760 0x7f87f80cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:15.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87f7400640 1 -- 172.21.15.94:0/3859416740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87f81e8760 msgr2=0x7f87f80cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:15.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87f7400640 1 --2- 172.21.15.94:0/3859416740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87f81e8760 0x7f87f80cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87f7400640 1 -- 172.21.15.94:0/3859416740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87dc0052f0 msgr2=0x7f87f8142a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87f7400640 1 --2- 172.21.15.94:0/3859416740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87dc0052f0 0x7f87f8142a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87f7400640 1 -- 172.21.15.94:0/3859416740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f81e8c20 con 0x7f87f81f7d10 2024-09-10T23:38:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87f6a00640 1 --2- 172.21.15.94:0/3859416740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87dc0052f0 0x7f87f8142a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:38:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87f7400640 1 --2- 172.21.15.94:0/3859416740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f81f7d10 0x7f87f81f81c0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f87f00202e0 tx=0x7f87f000c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87ef400640 1 -- 172.21.15.94:0/3859416740 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0004cd0 con 0x7f87f81f7d10 2024-09-10T23:38:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87ef400640 1 -- 172.21.15.94:0/3859416740 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87f00b7030 con 0x7f87f81f7d10 2024-09-10T23:38:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87ef400640 1 -- 172.21.15.94:0/3859416740 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00a9870 con 0x7f87f81f7d10 2024-09-10T23:38:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87fda00640 1 -- 172.21.15.94:0/3859416740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f81e8f00 con 0x7f87f81f7d10 2024-09-10T23:38:15.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87fda00640 1 -- 172.21.15.94:0/3859416740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f81e94c0 con 0x7f87f81f7d10 2024-09-10T23:38:15.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87fda00640 1 -- 172.21.15.94:0/3859416740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f815b320 con 0x7f87f81f7d10 2024-09-10T23:38:15.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.509+0000 7f87ef400640 1 -- 172.21.15.94:0/3859416740 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f87f0006070 con 0x7f87f81f7d10 2024-09-10T23:38:15.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.513+0000 7f87ef400640 1 --2- 172.21.15.94:0/3859416740 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87d4072a60 0x7f87d4074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:15.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.513+0000 7f87ef400640 1 -- 172.21.15.94:0/3859416740 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4885+0+0 (secure 0 0 0) 0x7f87f01307c0 con 0x7f87f81f7d10 2024-09-10T23:38:15.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.513+0000 7f87f6a00640 1 --2- 172.21.15.94:0/3859416740 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87d4072a60 0x7f87d4074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:15.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.513+0000 7f87f6a00640 1 --2- 172.21.15.94:0/3859416740 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87d4072a60 0x7f87d4074f20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f87e8000c10 tx=0x7f87e8006b80 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:15.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.517+0000 7f87ef400640 1 -- 172.21.15.94:0/3859416740 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f87f00c2020 con 0x7f87f81f7d10 2024-09-10T23:38:15.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.737+0000 7f87fda00640 1 -- 172.21.15.94:0/3859416740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f87f80b4c30 con 0x7f87f81f7d10 2024-09-10T23:38:15.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.737+0000 7f87ef400640 1 -- 172.21.15.94:0/3859416740 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v35) ==== 80+0+5304 (secure 0 0 0) 0x7f87f00197f0 con 0x7f87f81f7d10 2024-09-10T23:38:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:15.740 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:38:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_dump": [ 2024-09-10T23:38:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.a", 2024-09-10T23:38:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/2m3LxAAr0ErpLJg+0+I2r74zQya4w==", 2024-09-10T23:38:15.740 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:38:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:38:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:38:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.b", 2024-09-10T23:38:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmc8pYABAAw90zu8D+9IwK15h4HIlGBA==", 2024-09-10T23:38:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:38:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:38:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:38:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.c", 2024-09-10T23:38:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmk98KNxAA44e9N4VHbw/94Lh1V9zugQ==", 2024-09-10T23:38:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:38:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:38:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:38:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.d", 2024-09-10T23:38:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmVAKzBxAAJehOw6aphTF/g3ucoQfw9A==", 2024-09-10T23:38:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:38:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:38:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:38:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.0", 2024-09-10T23:38:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDa1uBmSAD3KBAAJTF1OEC44Ml599aL7iEuDw==", 2024-09-10T23:38:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.743 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:38:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:38:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.1", 2024-09-10T23:38:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDe1uBma1L9ABAAfEjW1/Hb4fh4Kwv/ZUgFAQ==", 2024-09-10T23:38:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:38:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:38:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.2", 2024-09-10T23:38:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDh1uBmtLUQFhAAweeFHj6BRQVC9BX+VLiWEQ==", 2024-09-10T23:38:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:38:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:38:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.3", 2024-09-10T23:38:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDk1uBm1KFCKRAAev28iYcrEYOViX95uo6Kbw==", 2024-09-10T23:38:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.745 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:38:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:38:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.4", 2024-09-10T23:38:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDq1uBm/Hg9LhAAvB2D0rj4DeRPxTrGmXYIsA==", 2024-09-10T23:38:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:38:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:38:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.5", 2024-09-10T23:38:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDu1uBm1fAKCBAAFHMjnGFAItLdaACF/UjE/w==", 2024-09-10T23:38:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:38:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:38:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.6", 2024-09-10T23:38:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDx1uBmXgYfHhAA/6wwtM4mDRXsIpoTIHKJ0w==", 2024-09-10T23:38:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:38:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:38:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:15.747 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.7", 2024-09-10T23:38:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQD01uBmw6hQMxAAk7oGAg+iIplNK5KrokGwNA==", 2024-09-10T23:38:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:38:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:38:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.0", 2024-09-10T23:38:15.748 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g==", 2024-09-10T23:38:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:38:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow r", 2024-09-10T23:38:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T23:38:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.1", 2024-09-10T23:38:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmuSkhFRAA1R6WNXLImkGfhW9/XJzEAQ==", 2024-09-10T23:38:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:38:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:38:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:38:15.749 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:38:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.2", 2024-09-10T23:38:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmfvaqERAASpL0KtglFcAuRf4a/0z5JQ==", 2024-09-10T23:38:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:38:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:38:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:38:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:38:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.3", 2024-09-10T23:38:15.750 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBm8xUtGxAARc0ywfAhMxBNd57TzBoV3g==", 2024-09-10T23:38:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:38:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:38:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:38:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:38:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.admin", 2024-09-10T23:38:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmOpg1FBAA9NV6Jqt6doWMuAfcxztlOg==", 2024-09-10T23:38:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.751 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:38:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:38:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:38:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.x", 2024-09-10T23:38:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/EPVKxAA/Myv8Zjf7BejbzlkxyCC1w==", 2024-09-10T23:38:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:38:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:38:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:15.752 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.y", 2024-09-10T23:38:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmhHvtJxAACe+79nYhv2ODwtmNhA45FA==", 2024-09-10T23:38:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:38:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:38:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:15.753 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:38:15.753 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:38:15.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.741+0000 7f87fda00640 1 -- 172.21.15.94:0/3859416740 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87d4072a60 msgr2=0x7f87d4074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:15.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.741+0000 7f87fda00640 1 --2- 172.21.15.94:0/3859416740 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87d4072a60 0x7f87d4074f20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f87e8000c10 tx=0x7f87e8006b80 comp rx=0 tx=0).stop 2024-09-10T23:38:15.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.741+0000 7f87fda00640 1 -- 172.21.15.94:0/3859416740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f81f7d10 msgr2=0x7f87f81f81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:15.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.741+0000 7f87fda00640 1 --2- 172.21.15.94:0/3859416740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f81f7d10 0x7f87f81f81c0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f87f00202e0 tx=0x7f87f000c260 comp rx=0 tx=0).stop 2024-09-10T23:38:15.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.741+0000 7f87fda00640 1 -- 172.21.15.94:0/3859416740 shutdown_connections 2024-09-10T23:38:15.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.741+0000 7f87fda00640 1 --2- 172.21.15.94:0/3859416740 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87d4072a60 0x7f87d4074f20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.741+0000 7f87fda00640 1 --2- 172.21.15.94:0/3859416740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87f81f7d10 0x7f87f81f81c0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.741+0000 7f87fda00640 1 --2- 172.21.15.94:0/3859416740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87f81e8760 0x7f87f80cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.741+0000 7f87fda00640 1 --2- 172.21.15.94:0/3859416740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87dc0052f0 0x7f87f8142a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.741+0000 7f87fda00640 1 -- 172.21.15.94:0/3859416740 >> 172.21.15.94:0/3859416740 conn(0x7f87f80d0b70 msgr2=0x7f87f80b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:15.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.741+0000 7f87fda00640 1 -- 172.21.15.94:0/3859416740 shutdown_connections 2024-09-10T23:38:15.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.741+0000 7f87fda00640 1 -- 172.21.15.94:0/3859416740 wait complete. 2024-09-10T23:38:15.762 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T23:38:15.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.901+0000 7fb55f400640 1 -- 172.21.15.94:0/2554076662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb558073ff0 msgr2=0x7fb5580cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:15.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.901+0000 7fb55f400640 1 --2- 172.21.15.94:0/2554076662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb558073ff0 0x7fb5580cfda0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb548009900 tx=0x7fb54801c8e0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.901+0000 7fb55f400640 1 -- 172.21.15.94:0/2554076662 shutdown_connections 2024-09-10T23:38:15.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.901+0000 7fb55f400640 1 --2- 172.21.15.94:0/2554076662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb55815ee80 0x7fb55806f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.901+0000 7fb55f400640 1 --2- 172.21.15.94:0/2554076662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb558073ff0 0x7fb5580cfda0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.901+0000 7fb55f400640 1 -- 172.21.15.94:0/2554076662 >> 172.21.15.94:0/2554076662 conn(0x7fb55806cdf0 msgr2=0x7fb55806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:15.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.901+0000 7fb55f400640 1 -- 172.21.15.94:0/2554076662 shutdown_connections 2024-09-10T23:38:15.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.901+0000 7fb55f400640 1 -- 172.21.15.94:0/2554076662 wait complete. 2024-09-10T23:38:15.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb55f400640 1 Processor -- start 2024-09-10T23:38:15.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb55f400640 1 -- start start 2024-09-10T23:38:15.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb55f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb558073ff0 0x7fb558153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:15.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb55f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5580d02e0 0x7fb55815f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:15.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb55f400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb55815ee80 0x7fb55814bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:15.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb55f400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb558167930 con 0x7fb5580d02e0 2024-09-10T23:38:15.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb55f400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb5581677b0 con 0x7fb558073ff0 2024-09-10T23:38:15.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb55f400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb558167ab0 con 0x7fb55815ee80 2024-09-10T23:38:15.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb54fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5580d02e0 0x7fb55815f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:15.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb54fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5580d02e0 0x7fb55815f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33272/0 (socket says 172.21.15.94:33272) 2024-09-10T23:38:15.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb54fe00640 1 -- 172.21.15.94:0/3891428754 learned_addr learned my addr 172.21.15.94:0/3891428754 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:15.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb55ca00640 1 --2- 172.21.15.94:0/3891428754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb558073ff0 0x7fb558153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:15.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb55d400640 1 --2- 172.21.15.94:0/3891428754 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb55815ee80 0x7fb55814bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:15.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb54fe00640 1 -- 172.21.15.94:0/3891428754 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb55815ee80 msgr2=0x7fb55814bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:15.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb54fe00640 1 --2- 172.21.15.94:0/3891428754 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb55815ee80 0x7fb55814bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb54fe00640 1 -- 172.21.15.94:0/3891428754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb558073ff0 msgr2=0x7fb558153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:15.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb54fe00640 1 --2- 172.21.15.94:0/3891428754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb558073ff0 0x7fb558153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:15.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb54fe00640 1 -- 172.21.15.94:0/3891428754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5581550b0 con 0x7fb5580d02e0 2024-09-10T23:38:15.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb55ca00640 1 --2- 172.21.15.94:0/3891428754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb558073ff0 0x7fb558153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:15.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb54fe00640 1 --2- 172.21.15.94:0/3891428754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5580d02e0 0x7fb55815f750 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fb54000c4b0 tx=0x7fb54000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:15.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb555600640 1 -- 172.21.15.94:0/3891428754 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5400052d0 con 0x7fb5580d02e0 2024-09-10T23:38:15.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb555600640 1 -- 172.21.15.94:0/3891428754 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb540005470 con 0x7fb5580d02e0 2024-09-10T23:38:15.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb555600640 1 -- 172.21.15.94:0/3891428754 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540012870 con 0x7fb5580d02e0 2024-09-10T23:38:15.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb55f400640 1 -- 172.21.15.94:0/3891428754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb558155390 con 0x7fb5580d02e0 2024-09-10T23:38:15.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.905+0000 7fb55f400640 1 -- 172.21.15.94:0/3891428754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb558151880 con 0x7fb5580d02e0 2024-09-10T23:38:15.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.909+0000 7fb55f400640 1 -- 172.21.15.94:0/3891428754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5580cfda0 con 0x7fb5580d02e0 2024-09-10T23:38:15.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.909+0000 7fb555600640 1 -- 172.21.15.94:0/3891428754 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7fb540003600 con 0x7fb5580d02e0 2024-09-10T23:38:15.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.909+0000 7fb555600640 1 --2- 172.21.15.94:0/3891428754 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb534072a90 0x7fb534074f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:15.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.909+0000 7fb555600640 1 -- 172.21.15.94:0/3891428754 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4885+0+0 (secure 0 0 0) 0x7fb540093270 con 0x7fb5580d02e0 2024-09-10T23:38:15.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.909+0000 7fb55ca00640 1 --2- 172.21.15.94:0/3891428754 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb534072a90 0x7fb534074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:15.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.909+0000 7fb55ca00640 1 --2- 172.21.15.94:0/3891428754 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb534072a90 0x7fb534074f50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb5480af000 tx=0x7fb54801b120 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:15.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:15.913+0000 7fb555600640 1 -- 172.21.15.94:0/3891428754 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb54005e630 con 0x7fb5580d02e0 2024-09-10T23:38:16.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.077+0000 7fb55f400640 1 -- 172.21.15.94:0/3891428754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb55806d300 con 0x7fb5580d02e0 2024-09-10T23:38:16.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.077+0000 7fb555600640 1 -- 172.21.15.94:0/3891428754 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v110) ==== 84+0+65 (secure 0 0 0) 0x7fb54005df30 con 0x7fb5580d02e0 2024-09-10T23:38:16.081 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:16.081 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:38:16.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:16.081 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-09-10T23:38:16.081 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-09-10T23:38:16.081 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:16.081 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:38:16.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.081+0000 7fb55f400640 1 -- 172.21.15.94:0/3891428754 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb534072a90 msgr2=0x7fb534074f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:16.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.081+0000 7fb55f400640 1 --2- 172.21.15.94:0/3891428754 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb534072a90 0x7fb534074f50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb5480af000 tx=0x7fb54801b120 comp rx=0 tx=0).stop 2024-09-10T23:38:16.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.081+0000 7fb55f400640 1 -- 172.21.15.94:0/3891428754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5580d02e0 msgr2=0x7fb55815f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:16.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.081+0000 7fb55f400640 1 --2- 172.21.15.94:0/3891428754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5580d02e0 0x7fb55815f750 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fb54000c4b0 tx=0x7fb54000c970 comp rx=0 tx=0).stop 2024-09-10T23:38:16.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.081+0000 7fb55f400640 1 -- 172.21.15.94:0/3891428754 shutdown_connections 2024-09-10T23:38:16.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.081+0000 7fb55f400640 1 --2- 172.21.15.94:0/3891428754 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb534072a90 0x7fb534074f50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:16.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.081+0000 7fb55f400640 1 --2- 172.21.15.94:0/3891428754 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb55815ee80 0x7fb55814bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:16.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.081+0000 7fb55f400640 1 --2- 172.21.15.94:0/3891428754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5580d02e0 0x7fb55815f750 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:16.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.081+0000 7fb55f400640 1 --2- 172.21.15.94:0/3891428754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb558073ff0 0x7fb558153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:16.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.081+0000 7fb55f400640 1 -- 172.21.15.94:0/3891428754 >> 172.21.15.94:0/3891428754 conn(0x7fb55806cdf0 msgr2=0x7fb558150530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:16.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.081+0000 7fb55f400640 1 -- 172.21.15.94:0/3891428754 shutdown_connections 2024-09-10T23:38:16.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.081+0000 7fb55f400640 1 -- 172.21.15.94:0/3891428754 wait complete. 2024-09-10T23:38:16.101 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T23:38:16.102 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T23:38:16.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.245+0000 7f19bfa00640 1 -- 172.21.15.94:0/1606192065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f199c0052f0 msgr2=0x7f199c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:16.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.245+0000 7f19bfa00640 1 --2- 172.21.15.94:0/1606192065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f199c0052f0 0x7f199c00b9d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f19b000c630 tx=0x7f19b001d2c0 comp rx=0 tx=0).stop 2024-09-10T23:38:16.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.245+0000 7f19bfa00640 1 -- 172.21.15.94:0/1606192065 shutdown_connections 2024-09-10T23:38:16.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.245+0000 7f19bfa00640 1 --2- 172.21.15.94:0/1606192065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f199c0052f0 0x7f199c00b9d0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:16.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.245+0000 7f19bfa00640 1 -- 172.21.15.94:0/1606192065 >> 172.21.15.94:0/1606192065 conn(0x7f19b806cdf0 msgr2=0x7f19b806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:16.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.245+0000 7f19bfa00640 1 -- 172.21.15.94:0/1606192065 shutdown_connections 2024-09-10T23:38:16.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.245+0000 7f19bfa00640 1 -- 172.21.15.94:0/1606192065 wait complete. 2024-09-10T23:38:16.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19bfa00640 1 Processor -- start 2024-09-10T23:38:16.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19bfa00640 1 -- start start 2024-09-10T23:38:16.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19bfa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f199c0052f0 0x7f19b81f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:16.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19bfa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19b8150f10 0x7f19b81498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:16.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19bfa00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19b81f4b60 0x7f19b814c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:16.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19bfa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f19b80d0250 con 0x7f19b8150f10 2024-09-10T23:38:16.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19bfa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f19b80d00d0 con 0x7f199c0052f0 2024-09-10T23:38:16.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19bfa00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f19b80d03d0 con 0x7f19b81f4b60 2024-09-10T23:38:16.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19afe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19b8150f10 0x7f19b81498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:16.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19afe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19b8150f10 0x7f19b81498c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33286/0 (socket says 172.21.15.94:33286) 2024-09-10T23:38:16.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19afe00640 1 -- 172.21.15.94:0/1342382645 learned_addr learned my addr 172.21.15.94:0/1342382645 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:16.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19bda00640 1 --2- 172.21.15.94:0/1342382645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19b81f4b60 0x7f19b814c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:16.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19bd000640 1 --2- 172.21.15.94:0/1342382645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f199c0052f0 0x7f19b81f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:16.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19afe00640 1 -- 172.21.15.94:0/1342382645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19b81f4b60 msgr2=0x7f19b814c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:16.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19afe00640 1 --2- 172.21.15.94:0/1342382645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19b81f4b60 0x7f19b814c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:16.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19afe00640 1 -- 172.21.15.94:0/1342382645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f199c0052f0 msgr2=0x7f19b81f7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:16.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19afe00640 1 --2- 172.21.15.94:0/1342382645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f199c0052f0 0x7f19b81f7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:16.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19afe00640 1 -- 172.21.15.94:0/1342382645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b80732c0 con 0x7f19b8150f10 2024-09-10T23:38:16.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19afe00640 1 --2- 172.21.15.94:0/1342382645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19b8150f10 0x7f19b81498c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f19a000f9d0 tx=0x7f19a000fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:16.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19b5600640 1 -- 172.21.15.94:0/1342382645 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a000bcb0 con 0x7f19b8150f10 2024-09-10T23:38:16.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19b5600640 1 -- 172.21.15.94:0/1342382645 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19a0011440 con 0x7f19b8150f10 2024-09-10T23:38:16.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19bfa00640 1 -- 172.21.15.94:0/1342382645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b8151390 con 0x7f19b8150f10 2024-09-10T23:38:16.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19b5600640 1 -- 172.21.15.94:0/1342382645 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0011870 con 0x7f19b8150f10 2024-09-10T23:38:16.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19bfa00640 1 -- 172.21.15.94:0/1342382645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b81f8400 con 0x7f19b8150f10 2024-09-10T23:38:16.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.249+0000 7f19bfa00640 1 -- 172.21.15.94:0/1342382645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19b81f4450 con 0x7f19b8150f10 2024-09-10T23:38:16.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.253+0000 7f19b5600640 1 -- 172.21.15.94:0/1342382645 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f19a0011a10 con 0x7f19b8150f10 2024-09-10T23:38:16.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.253+0000 7f19b5600640 1 --2- 172.21.15.94:0/1342382645 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1994072a60 0x7f1994074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:16.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.253+0000 7f19b5600640 1 -- 172.21.15.94:0/1342382645 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4885+0+0 (secure 0 0 0) 0x7f19a00945a0 con 0x7f19b8150f10 2024-09-10T23:38:16.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.253+0000 7f19bd000640 1 --2- 172.21.15.94:0/1342382645 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1994072a60 0x7f1994074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:16.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.253+0000 7f19bd000640 1 --2- 172.21.15.94:0/1342382645 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1994072a60 0x7f1994074f20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f19a8006fd0 tx=0x7f19a8006b80 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:16.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.257+0000 7f19b5600640 1 -- 172.21.15.94:0/1342382645 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f19a005f720 con 0x7f19b8150f10 2024-09-10T23:38:16.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:16.429+0000 7f19bfa00640 1 -- 172.21.15.94:0/1342382645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f19b81576f0 con 0x7f19b8150f10 2024-09-10T23:38:17.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.345+0000 7f19b5600640 1 -- 172.21.15.94:0/1342382645 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v111) ==== 138+0+0 (secure 0 0 0) 0x7f19a005f020 con 0x7f19b8150f10 2024-09-10T23:38:17.346 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-09-10T23:38:17.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.345+0000 7f19bfa00640 1 -- 172.21.15.94:0/1342382645 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1994072a60 msgr2=0x7f1994074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:17.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.345+0000 7f19bfa00640 1 --2- 172.21.15.94:0/1342382645 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1994072a60 0x7f1994074f20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f19a8006fd0 tx=0x7f19a8006b80 comp rx=0 tx=0).stop 2024-09-10T23:38:17.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.345+0000 7f19bfa00640 1 -- 172.21.15.94:0/1342382645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19b8150f10 msgr2=0x7f19b81498c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:17.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.345+0000 7f19bfa00640 1 --2- 172.21.15.94:0/1342382645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19b8150f10 0x7f19b81498c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f19a000f9d0 tx=0x7f19a000fe90 comp rx=0 tx=0).stop 2024-09-10T23:38:17.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.349+0000 7f19bfa00640 1 -- 172.21.15.94:0/1342382645 shutdown_connections 2024-09-10T23:38:17.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.349+0000 7f19bfa00640 1 --2- 172.21.15.94:0/1342382645 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1994072a60 0x7f1994074f20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:17.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.349+0000 7f19bfa00640 1 --2- 172.21.15.94:0/1342382645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19b81f4b60 0x7f19b814c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:17.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.349+0000 7f19bfa00640 1 --2- 172.21.15.94:0/1342382645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19b8150f10 0x7f19b81498c0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:17.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.349+0000 7f19bfa00640 1 --2- 172.21.15.94:0/1342382645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f199c0052f0 0x7f19b81f7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:17.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.349+0000 7f19bfa00640 1 -- 172.21.15.94:0/1342382645 >> 172.21.15.94:0/1342382645 conn(0x7f19b806cdf0 msgr2=0x7f19b815e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:17.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.349+0000 7f19bfa00640 1 -- 172.21.15.94:0/1342382645 shutdown_connections 2024-09-10T23:38:17.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.349+0000 7f19bfa00640 1 -- 172.21.15.94:0/1342382645 wait complete. 2024-09-10T23:38:17.375 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T23:38:17.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.525+0000 7fc0de600640 1 -- 172.21.15.94:0/3914127690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0bc003c90 msgr2=0x7fc0bc004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:17.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.525+0000 7fc0de600640 1 --2- 172.21.15.94:0/3914127690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0bc003c90 0x7fc0bc004090 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8001f50 tx=0x7fc0c801cbc0 comp rx=0 tx=0).stop 2024-09-10T23:38:17.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.525+0000 7fc0de600640 1 -- 172.21.15.94:0/3914127690 shutdown_connections 2024-09-10T23:38:17.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.525+0000 7fc0de600640 1 --2- 172.21.15.94:0/3914127690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0bc003c90 0x7fc0bc004090 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:17.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.525+0000 7fc0de600640 1 -- 172.21.15.94:0/3914127690 >> 172.21.15.94:0/3914127690 conn(0x7fc0d80cfaf0 msgr2=0x7fc0d80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:17.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.525+0000 7fc0de600640 1 -- 172.21.15.94:0/3914127690 shutdown_connections 2024-09-10T23:38:17.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.525+0000 7fc0de600640 1 -- 172.21.15.94:0/3914127690 wait complete. 2024-09-10T23:38:17.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.525+0000 7fc0de600640 1 Processor -- start 2024-09-10T23:38:17.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.525+0000 7fc0de600640 1 -- start start 2024-09-10T23:38:17.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.525+0000 7fc0de600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0bc003c90 0x7fc0d81f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:17.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.525+0000 7fc0de600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0d80b52b0 0x7fc0d80b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.525+0000 7fc0de600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0d8158db0 0x7fc0d81591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.525+0000 7fc0de600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d815e780 con 0x7fc0bc003c90 2024-09-10T23:38:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.525+0000 7fc0de600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d80cb950 con 0x7fc0d80b52b0 2024-09-10T23:38:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.525+0000 7fc0de600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d815e900 con 0x7fc0d8158db0 2024-09-10T23:38:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0d7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0bc003c90 0x7fc0d81f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0d7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0bc003c90 0x7fc0d81f6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33300/0 (socket says 172.21.15.94:33300) 2024-09-10T23:38:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0d7400640 1 -- 172.21.15.94:0/103369501 learned_addr learned my addr 172.21.15.94:0/103369501 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0d7e00640 1 --2- 172.21.15.94:0/103369501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0d8158db0 0x7fc0d81591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0d6a00640 1 --2- 172.21.15.94:0/103369501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0d80b52b0 0x7fc0d80b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0d7400640 1 -- 172.21.15.94:0/103369501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0d8158db0 msgr2=0x7fc0d81591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0d7400640 1 --2- 172.21.15.94:0/103369501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0d8158db0 0x7fc0d81591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:17.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0d7400640 1 -- 172.21.15.94:0/103369501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0d80b52b0 msgr2=0x7fc0d80b5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:17.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0d7400640 1 --2- 172.21.15.94:0/103369501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0d80b52b0 0x7fc0d80b5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:17.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0d7400640 1 -- 172.21.15.94:0/103369501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d80b2be0 con 0x7fc0bc003c90 2024-09-10T23:38:17.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0d7400640 1 --2- 172.21.15.94:0/103369501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0bc003c90 0x7fc0d81f6f70 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fc0c801b2c0 tx=0x7fc0c80a5e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:17.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0cf400640 1 -- 172.21.15.94:0/103369501 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8006c50 con 0x7fc0bc003c90 2024-09-10T23:38:17.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0cf400640 1 -- 172.21.15.94:0/103369501 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0c80b4030 con 0x7fc0bc003c90 2024-09-10T23:38:17.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0de600640 1 -- 172.21.15.94:0/103369501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d80b2ec0 con 0x7fc0bc003c90 2024-09-10T23:38:17.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0cf400640 1 -- 172.21.15.94:0/103369501 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8006370 con 0x7fc0bc003c90 2024-09-10T23:38:17.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0de600640 1 -- 172.21.15.94:0/103369501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d80b3480 con 0x7fc0bc003c90 2024-09-10T23:38:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0de600640 1 -- 172.21.15.94:0/103369501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d815ee60 con 0x7fc0bc003c90 2024-09-10T23:38:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.529+0000 7fc0cf400640 1 -- 172.21.15.94:0/103369501 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7fc0c8006510 con 0x7fc0bc003c90 2024-09-10T23:38:17.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.533+0000 7fc0cf400640 1 --2- 172.21.15.94:0/103369501 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc0b8072a60 0x7fc0b8074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:17.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.533+0000 7fc0cf400640 1 -- 172.21.15.94:0/103369501 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5255+0+0 (secure 0 0 0) 0x7fc0c812e1f0 con 0x7fc0bc003c90 2024-09-10T23:38:17.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.533+0000 7fc0d6a00640 1 --2- 172.21.15.94:0/103369501 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc0b8072a60 0x7fc0b8074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:17.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.533+0000 7fc0d6a00640 1 --2- 172.21.15.94:0/103369501 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc0b8072a60 0x7fc0b8074f20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4001000 tx=0x7fc0c4000f50 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:17.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.537+0000 7fc0cf400640 1 -- 172.21.15.94:0/103369501 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc0c8016970 con 0x7fc0bc003c90 2024-09-10T23:38:17.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:17.701+0000 7fc0de600640 1 -- 172.21.15.94:0/103369501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc0d806f3d0 con 0x7fc0bc003c90 2024-09-10T23:38:18.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.353+0000 7fc0cf400640 1 -- 172.21.15.94:0/103369501 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v112) ==== 170+0+0 (secure 0 0 0) 0x7fc0c8006830 con 0x7fc0bc003c90 2024-09-10T23:38:18.355 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-09-10T23:38:18.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.353+0000 7fc0de600640 1 -- 172.21.15.94:0/103369501 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc0b8072a60 msgr2=0x7fc0b8074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:18.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.353+0000 7fc0de600640 1 --2- 172.21.15.94:0/103369501 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc0b8072a60 0x7fc0b8074f20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4001000 tx=0x7fc0c4000f50 comp rx=0 tx=0).stop 2024-09-10T23:38:18.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.353+0000 7fc0de600640 1 -- 172.21.15.94:0/103369501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0bc003c90 msgr2=0x7fc0d81f6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:18.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.353+0000 7fc0de600640 1 --2- 172.21.15.94:0/103369501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0bc003c90 0x7fc0d81f6f70 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fc0c801b2c0 tx=0x7fc0c80a5e30 comp rx=0 tx=0).stop 2024-09-10T23:38:18.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.357+0000 7fc0de600640 1 -- 172.21.15.94:0/103369501 shutdown_connections 2024-09-10T23:38:18.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.357+0000 7fc0de600640 1 --2- 172.21.15.94:0/103369501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0d80b52b0 0x7fc0d80b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:18.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.357+0000 7fc0de600640 1 --2- 172.21.15.94:0/103369501 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc0b8072a60 0x7fc0b8074f20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:18.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.357+0000 7fc0de600640 1 --2- 172.21.15.94:0/103369501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0d8158db0 0x7fc0d81591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:18.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.357+0000 7fc0de600640 1 --2- 172.21.15.94:0/103369501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0bc003c90 0x7fc0d81f6f70 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:18.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.357+0000 7fc0de600640 1 -- 172.21.15.94:0/103369501 >> 172.21.15.94:0/103369501 conn(0x7fc0d80cfaf0 msgr2=0x7fc0d8156a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:18.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.357+0000 7fc0de600640 1 -- 172.21.15.94:0/103369501 shutdown_connections 2024-09-10T23:38:18.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.357+0000 7fc0de600640 1 -- 172.21.15.94:0/103369501 wait complete. 2024-09-10T23:38:18.395 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T23:38:18.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.537+0000 7f24e4000640 1 -- 172.21.15.94:0/3221792499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24dc06ce80 msgr2=0x7f24dc15f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:18.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.537+0000 7f24e4000640 1 --2- 172.21.15.94:0/3221792499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24dc06ce80 0x7f24dc15f820 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009840 tx=0x7f24d001c7f0 comp rx=0 tx=0).stop 2024-09-10T23:38:18.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.537+0000 7f24e4000640 1 -- 172.21.15.94:0/3221792499 shutdown_connections 2024-09-10T23:38:18.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.537+0000 7f24e4000640 1 --2- 172.21.15.94:0/3221792499 >> v2:172.21.15.181:3300/0 conn(0x7f24dc073920 0x7f24dc073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:18.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.537+0000 7f24e4000640 1 --2- 172.21.15.94:0/3221792499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24dc06ce80 0x7f24dc15f820 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:18.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.537+0000 7f24e4000640 1 -- 172.21.15.94:0/3221792499 >> 172.21.15.94:0/3221792499 conn(0x7f24dc0d0d80 msgr2=0x7f24dc0d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:18.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.537+0000 7f24e4000640 1 -- 172.21.15.94:0/3221792499 shutdown_connections 2024-09-10T23:38:18.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.537+0000 7f24e4000640 1 -- 172.21.15.94:0/3221792499 wait complete. 2024-09-10T23:38:18.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.537+0000 7f24e4000640 1 Processor -- start 2024-09-10T23:38:18.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.537+0000 7f24e4000640 1 -- start start 2024-09-10T23:38:18.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e4000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24dc06ce80 0x7f24dc153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:18.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e4000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24dc06d260 0x7f24dc150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:18.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e4000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24dc073920 0x7f24dc1512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:18.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e4000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc141540 con 0x7f24dc06d260 2024-09-10T23:38:18.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e4000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc1413c0 con 0x7f24dc06ce80 2024-09-10T23:38:18.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e4000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f24dc1416c0 con 0x7f24dc073920 2024-09-10T23:38:18.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24dc06d260 0x7f24dc150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:18.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24dc06d260 0x7f24dc150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33310/0 (socket says 172.21.15.94:33310) 2024-09-10T23:38:18.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e0c00640 1 -- 172.21.15.94:0/397654706 learned_addr learned my addr 172.21.15.94:0/397654706 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:18.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e1600640 1 --2- 172.21.15.94:0/397654706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24dc06ce80 0x7f24dc153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:18.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e2000640 1 --2- 172.21.15.94:0/397654706 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24dc073920 0x7f24dc1512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:18.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e0c00640 1 -- 172.21.15.94:0/397654706 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24dc073920 msgr2=0x7f24dc1512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:18.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e0c00640 1 --2- 172.21.15.94:0/397654706 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24dc073920 0x7f24dc1512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:18.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e0c00640 1 -- 172.21.15.94:0/397654706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24dc06ce80 msgr2=0x7f24dc153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:18.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e0c00640 1 --2- 172.21.15.94:0/397654706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24dc06ce80 0x7f24dc153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:18.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e0c00640 1 -- 172.21.15.94:0/397654706 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24dc152ed0 con 0x7f24dc06d260 2024-09-10T23:38:18.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e1600640 1 --2- 172.21.15.94:0/397654706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24dc06ce80 0x7f24dc153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:18.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e0c00640 1 --2- 172.21.15.94:0/397654706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24dc06d260 0x7f24dc150d90 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f24c800c730 tx=0x7f24c800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:18.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24d9600640 1 -- 172.21.15.94:0/397654706 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8006aa0 con 0x7f24dc06d260 2024-09-10T23:38:18.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24d9600640 1 -- 172.21.15.94:0/397654706 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24c8017070 con 0x7f24dc06d260 2024-09-10T23:38:18.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24d9600640 1 -- 172.21.15.94:0/397654706 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8007050 con 0x7f24dc06d260 2024-09-10T23:38:18.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e4000640 1 -- 172.21.15.94:0/397654706 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24dc13e1b0 con 0x7f24dc06d260 2024-09-10T23:38:18.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e4000640 1 -- 172.21.15.94:0/397654706 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24dc139700 con 0x7f24dc06d260 2024-09-10T23:38:18.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.541+0000 7f24e4000640 1 -- 172.21.15.94:0/397654706 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24dc15f820 con 0x7f24dc06d260 2024-09-10T23:38:18.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.545+0000 7f24d9600640 1 -- 172.21.15.94:0/397654706 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f24c80071f0 con 0x7f24dc06d260 2024-09-10T23:38:18.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.545+0000 7f24d9600640 1 --2- 172.21.15.94:0/397654706 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f24bc072a60 0x7f24bc074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:18.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.545+0000 7f24d9600640 1 -- 172.21.15.94:0/397654706 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5637+0+0 (secure 0 0 0) 0x7f24c80942a0 con 0x7f24dc06d260 2024-09-10T23:38:18.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.545+0000 7f24e1600640 1 --2- 172.21.15.94:0/397654706 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f24bc072a60 0x7f24bc074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:18.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.545+0000 7f24e1600640 1 --2- 172.21.15.94:0/397654706 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f24bc072a60 0x7f24bc074f20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f24d00af000 tx=0x7f24d001b030 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:18.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.549+0000 7f24d9600640 1 -- 172.21.15.94:0/397654706 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f24dc15c270 con 0x7f24dc06d260 2024-09-10T23:38:18.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:18.753+0000 7f24e4000640 1 -- 172.21.15.94:0/397654706 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f24dc1411a0 con 0x7f24dc06d260 2024-09-10T23:38:19.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.365+0000 7f24d9600640 1 -- 172.21.15.94:0/397654706 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v74) ==== 174+0+0 (secure 0 0 0) 0x7f24c805f3f0 con 0x7f24dc06d260 2024-09-10T23:38:19.372 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 18 and data pool 19 2024-09-10T23:38:19.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.377+0000 7f24e4000640 1 -- 172.21.15.94:0/397654706 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f24bc072a60 msgr2=0x7f24bc074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:19.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.377+0000 7f24e4000640 1 --2- 172.21.15.94:0/397654706 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f24bc072a60 0x7f24bc074f20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f24d00af000 tx=0x7f24d001b030 comp rx=0 tx=0).stop 2024-09-10T23:38:19.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.377+0000 7f24e4000640 1 -- 172.21.15.94:0/397654706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24dc06d260 msgr2=0x7f24dc150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:19.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.377+0000 7f24e4000640 1 --2- 172.21.15.94:0/397654706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24dc06d260 0x7f24dc150d90 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f24c800c730 tx=0x7f24c800cbf0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.381+0000 7f24e4000640 1 -- 172.21.15.94:0/397654706 shutdown_connections 2024-09-10T23:38:19.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.381+0000 7f24e4000640 1 --2- 172.21.15.94:0/397654706 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f24bc072a60 0x7f24bc074f20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.381+0000 7f24e4000640 1 --2- 172.21.15.94:0/397654706 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24dc073920 0x7f24dc1512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.381+0000 7f24e4000640 1 --2- 172.21.15.94:0/397654706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24dc06d260 0x7f24dc150d90 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.381+0000 7f24e4000640 1 --2- 172.21.15.94:0/397654706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24dc06ce80 0x7f24dc153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.381+0000 7f24e4000640 1 -- 172.21.15.94:0/397654706 >> 172.21.15.94:0/397654706 conn(0x7f24dc0d0d80 msgr2=0x7f24dc137e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:19.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.381+0000 7f24e4000640 1 -- 172.21.15.94:0/397654706 shutdown_connections 2024-09-10T23:38:19.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.381+0000 7f24e4000640 1 -- 172.21.15.94:0/397654706 wait complete. 2024-09-10T23:38:19.413 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:38:19.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 -- 172.21.15.94:0/455173502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec06eef0 msgr2=0x7f73ec06f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:19.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 --2- 172.21.15.94:0/455173502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec06eef0 0x7f73ec06f330 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0b2760 tx=0x7f73e401ccf0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 -- 172.21.15.94:0/455173502 shutdown_connections 2024-09-10T23:38:19.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 --2- 172.21.15.94:0/455173502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec06eef0 0x7f73ec06f330 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 --2- 172.21.15.94:0/455173502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec070150 0x7f73ec0b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 --2- 172.21.15.94:0/455173502 >> v2:172.21.15.181:3300/0 conn(0x7f73ec157c60 0x7f73ec072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 -- 172.21.15.94:0/455173502 >> 172.21.15.94:0/455173502 conn(0x7f73ec06cdf0 msgr2=0x7f73ec06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:19.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 -- 172.21.15.94:0/455173502 shutdown_connections 2024-09-10T23:38:19.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 -- 172.21.15.94:0/455173502 wait complete. 2024-09-10T23:38:19.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 Processor -- start 2024-09-10T23:38:19.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 -- start start 2024-09-10T23:38:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec06eef0 0x7f73ec154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73ec070150 0x7f73ec14f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec157c60 0x7f73ec1f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec15b140 con 0x7f73ec157c60 2024-09-10T23:38:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec15afc0 con 0x7f73ec070150 2024-09-10T23:38:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73f1e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f73ec15b2c0 con 0x7f73ec06eef0 2024-09-10T23:38:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73eb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec157c60 0x7f73ec1f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73eb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec157c60 0x7f73ec1f7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33338/0 (socket says 172.21.15.94:33338) 2024-09-10T23:38:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73eb400640 1 -- 172.21.15.94:0/738979940 learned_addr learned my addr 172.21.15.94:0/738979940 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73eaa00640 1 --2- 172.21.15.94:0/738979940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec06eef0 0x7f73ec154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:19.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.553+0000 7f73e2000640 1 --2- 172.21.15.94:0/738979940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73ec070150 0x7f73ec14f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:19.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.557+0000 7f73eb400640 1 -- 172.21.15.94:0/738979940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec06eef0 msgr2=0x7f73ec154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:19.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.557+0000 7f73eb400640 1 --2- 172.21.15.94:0/738979940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec06eef0 0x7f73ec154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.557+0000 7f73eb400640 1 -- 172.21.15.94:0/738979940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73ec070150 msgr2=0x7f73ec14f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:19.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.557+0000 7f73eb400640 1 --2- 172.21.15.94:0/738979940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73ec070150 0x7f73ec14f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.557+0000 7f73eb400640 1 -- 172.21.15.94:0/738979940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec14deb0 con 0x7f73ec157c60 2024-09-10T23:38:19.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.557+0000 7f73eb400640 1 --2- 172.21.15.94:0/738979940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec157c60 0x7f73ec1f7c30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009bd0 tx=0x7f73e4003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:19.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.557+0000 7f73e3e00640 1 -- 172.21.15.94:0/738979940 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e40bb070 con 0x7f73ec157c60 2024-09-10T23:38:19.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.557+0000 7f73e3e00640 1 -- 172.21.15.94:0/738979940 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73e4004e00 con 0x7f73ec157c60 2024-09-10T23:38:19.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.557+0000 7f73f1e00640 1 -- 172.21.15.94:0/738979940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec1fb560 con 0x7f73ec157c60 2024-09-10T23:38:19.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.557+0000 7f73e3e00640 1 -- 172.21.15.94:0/738979940 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e40b2040 con 0x7f73ec157c60 2024-09-10T23:38:19.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.557+0000 7f73f1e00640 1 -- 172.21.15.94:0/738979940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec14e960 con 0x7f73ec157c60 2024-09-10T23:38:19.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.557+0000 7f73f1e00640 1 -- 172.21.15.94:0/738979940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ec0b35c0 con 0x7f73ec157c60 2024-09-10T23:38:19.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.557+0000 7f73e3e00640 1 -- 172.21.15.94:0/738979940 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f73e40b5030 con 0x7f73ec157c60 2024-09-10T23:38:19.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.557+0000 7f73e3e00640 1 --2- 172.21.15.94:0/738979940 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73c8072a60 0x7f73c8074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:19.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.557+0000 7f73e3e00640 1 -- 172.21.15.94:0/738979940 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5737+0+0 (secure 0 0 0) 0x7f73e412d800 con 0x7f73ec157c60 2024-09-10T23:38:19.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.561+0000 7f73eaa00640 1 --2- 172.21.15.94:0/738979940 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73c8072a60 0x7f73c8074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:19.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.561+0000 7f73eaa00640 1 --2- 172.21.15.94:0/738979940 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73c8072a60 0x7f73c8074f20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00bad0 tx=0x7f73dc00b2e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:19.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.561+0000 7f73e3e00640 1 -- 172.21.15.94:0/738979940 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f73e4016970 con 0x7f73ec157c60 2024-09-10T23:38:19.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.729+0000 7f73f1e00640 1 -- 172.21.15.94:0/738979940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73ec06f650 con 0x7f73ec157c60 2024-09-10T23:38:19.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.729+0000 7f73e3e00640 1 -- 172.21.15.94:0/738979940 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16732 (secure 0 0 0) 0x7f73e41010d0 con 0x7f73ec157c60 2024-09-10T23:38:19.733 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:19.733 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":113,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:38:18.757635+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:38:16.443089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T23:38:17.716329+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:38:19.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.733+0000 7f73f1e00640 1 -- 172.21.15.94:0/738979940 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73c8072a60 msgr2=0x7f73c8074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:19.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.733+0000 7f73f1e00640 1 --2- 172.21.15.94:0/738979940 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73c8072a60 0x7f73c8074f20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00bad0 tx=0x7f73dc00b2e0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.733+0000 7f73f1e00640 1 -- 172.21.15.94:0/738979940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec157c60 msgr2=0x7f73ec1f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:19.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.733+0000 7f73f1e00640 1 --2- 172.21.15.94:0/738979940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec157c60 0x7f73ec1f7c30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009bd0 tx=0x7f73e4003f60 comp rx=0 tx=0).stop 2024-09-10T23:38:19.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.733+0000 7f73f1e00640 1 -- 172.21.15.94:0/738979940 shutdown_connections 2024-09-10T23:38:19.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.733+0000 7f73f1e00640 1 --2- 172.21.15.94:0/738979940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73ec070150 0x7f73ec14f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.733+0000 7f73f1e00640 1 --2- 172.21.15.94:0/738979940 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73c8072a60 0x7f73c8074f20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.733+0000 7f73f1e00640 1 --2- 172.21.15.94:0/738979940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73ec157c60 0x7f73ec1f7c30 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.733+0000 7f73f1e00640 1 --2- 172.21.15.94:0/738979940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73ec06eef0 0x7f73ec154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.733+0000 7f73f1e00640 1 -- 172.21.15.94:0/738979940 >> 172.21.15.94:0/738979940 conn(0x7f73ec06cdf0 msgr2=0x7f73ec072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:19.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.733+0000 7f73f1e00640 1 -- 172.21.15.94:0/738979940 shutdown_connections 2024-09-10T23:38:19.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.733+0000 7f73f1e00640 1 -- 172.21.15.94:0/738979940 wait complete. 2024-09-10T23:38:19.755 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:38:19.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.893+0000 7efe91600640 1 -- 172.21.15.94:0/3926991943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe8c15ead0 msgr2=0x7efe8c06f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:19.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.893+0000 7efe91600640 1 --2- 172.21.15.94:0/3926991943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe8c15ead0 0x7efe8c06f790 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0cc870 tx=0x7efe8401cc30 comp rx=0 tx=0).stop 2024-09-10T23:38:19.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.893+0000 7efe91600640 1 -- 172.21.15.94:0/3926991943 shutdown_connections 2024-09-10T23:38:19.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.893+0000 7efe91600640 1 --2- 172.21.15.94:0/3926991943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe8c15ead0 0x7efe8c06f790 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.893+0000 7efe91600640 1 --2- 172.21.15.94:0/3926991943 >> v2:172.21.15.181:3300/0 conn(0x7efe8c0d03c0 0x7efe8c15e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.893+0000 7efe91600640 1 -- 172.21.15.94:0/3926991943 >> 172.21.15.94:0/3926991943 conn(0x7efe8c06cdf0 msgr2=0x7efe8c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:19.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.893+0000 7efe91600640 1 -- 172.21.15.94:0/3926991943 shutdown_connections 2024-09-10T23:38:19.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.893+0000 7efe91600640 1 -- 172.21.15.94:0/3926991943 wait complete. 2024-09-10T23:38:19.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe91600640 1 Processor -- start 2024-09-10T23:38:19.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe91600640 1 -- start start 2024-09-10T23:38:19.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe91600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe8c0cf3c0 0x7efe8c159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:19.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe91600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe8c0d03c0 0x7efe8c149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:19.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe91600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe8c15ead0 0x7efe8c072610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:19.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe91600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c165710 con 0x7efe8c0cf3c0 2024-09-10T23:38:19.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe91600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c070770 con 0x7efe8c15ead0 2024-09-10T23:38:19.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe91600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7efe8c165890 con 0x7efe8c0d03c0 2024-09-10T23:38:19.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe8aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe8c0cf3c0 0x7efe8c159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:19.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe8aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe8c0cf3c0 0x7efe8c159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33352/0 (socket says 172.21.15.94:33352) 2024-09-10T23:38:19.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe8aa00640 1 -- 172.21.15.94:0/1039992671 learned_addr learned my addr 172.21.15.94:0/1039992671 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:19.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe82000640 1 --2- 172.21.15.94:0/1039992671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe8c0d03c0 0x7efe8c149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:19.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe8b400640 1 --2- 172.21.15.94:0/1039992671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe8c15ead0 0x7efe8c072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:19.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe8aa00640 1 -- 172.21.15.94:0/1039992671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe8c0d03c0 msgr2=0x7efe8c149300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:19.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe8aa00640 1 --2- 172.21.15.94:0/1039992671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe8c0d03c0 0x7efe8c149300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe8aa00640 1 -- 172.21.15.94:0/1039992671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe8c15ead0 msgr2=0x7efe8c072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:19.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe8aa00640 1 --2- 172.21.15.94:0/1039992671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe8c15ead0 0x7efe8c072610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:19.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe8aa00640 1 -- 172.21.15.94:0/1039992671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8c156cd0 con 0x7efe8c0cf3c0 2024-09-10T23:38:19.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe8aa00640 1 --2- 172.21.15.94:0/1039992671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe8c0cf3c0 0x7efe8c159de0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00c500 tx=0x7efe7c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:19.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe83400640 1 -- 172.21.15.94:0/1039992671 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c017070 con 0x7efe8c0cf3c0 2024-09-10T23:38:19.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe83400640 1 -- 172.21.15.94:0/1039992671 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe7c003600 con 0x7efe8c0cf3c0 2024-09-10T23:38:19.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe91600640 1 -- 172.21.15.94:0/1039992671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8c14f390 con 0x7efe8c0cf3c0 2024-09-10T23:38:19.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe83400640 1 -- 172.21.15.94:0/1039992671 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c003a50 con 0x7efe8c0cf3c0 2024-09-10T23:38:19.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.897+0000 7efe91600640 1 -- 172.21.15.94:0/1039992671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8c14f950 con 0x7efe8c0cf3c0 2024-09-10T23:38:19.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.901+0000 7efe91600640 1 -- 172.21.15.94:0/1039992671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8c15e590 con 0x7efe8c0cf3c0 2024-09-10T23:38:19.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.901+0000 7efe83400640 1 -- 172.21.15.94:0/1039992671 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7efe7c003bf0 con 0x7efe8c0cf3c0 2024-09-10T23:38:19.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.901+0000 7efe83400640 1 --2- 172.21.15.94:0/1039992671 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efe68072a60 0x7efe68074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:19.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.901+0000 7efe83400640 1 -- 172.21.15.94:0/1039992671 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5737+0+0 (secure 0 0 0) 0x7efe7c093250 con 0x7efe8c0cf3c0 2024-09-10T23:38:19.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.901+0000 7efe82000640 1 --2- 172.21.15.94:0/1039992671 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efe68072a60 0x7efe68074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:19.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.901+0000 7efe82000640 1 --2- 172.21.15.94:0/1039992671 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efe68072a60 0x7efe68074f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efe740058d0 tx=0x7efe74005860 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:19.905+0000 7efe83400640 1 -- 172.21.15.94:0/1039992671 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efe7c05dc40 con 0x7efe8c0cf3c0 2024-09-10T23:38:20.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.077+0000 7efe91600640 1 -- 172.21.15.94:0/1039992671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe8c0ce230 con 0x7efe8c0cf3c0 2024-09-10T23:38:20.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.077+0000 7efe83400640 1 -- 172.21.15.94:0/1039992671 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16732 (secure 0 0 0) 0x7efe7c062670 con 0x7efe8c0cf3c0 2024-09-10T23:38:20.079 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:20.079 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":113,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:38:18.757635+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:38:16.443089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T23:38:17.716329+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:38:20.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.077+0000 7efe91600640 1 -- 172.21.15.94:0/1039992671 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efe68072a60 msgr2=0x7efe68074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:20.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.077+0000 7efe91600640 1 --2- 172.21.15.94:0/1039992671 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efe68072a60 0x7efe68074f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efe740058d0 tx=0x7efe74005860 comp rx=0 tx=0).stop 2024-09-10T23:38:20.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.077+0000 7efe91600640 1 -- 172.21.15.94:0/1039992671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe8c0cf3c0 msgr2=0x7efe8c159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:20.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.077+0000 7efe91600640 1 --2- 172.21.15.94:0/1039992671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe8c0cf3c0 0x7efe8c159de0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00c500 tx=0x7efe7c00c9c0 comp rx=0 tx=0).stop 2024-09-10T23:38:20.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.077+0000 7efe91600640 1 -- 172.21.15.94:0/1039992671 shutdown_connections 2024-09-10T23:38:20.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.077+0000 7efe91600640 1 --2- 172.21.15.94:0/1039992671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe8c15ead0 0x7efe8c072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:20.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.077+0000 7efe91600640 1 --2- 172.21.15.94:0/1039992671 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efe68072a60 0x7efe68074f20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:20.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.077+0000 7efe91600640 1 --2- 172.21.15.94:0/1039992671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe8c0d03c0 0x7efe8c149300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:20.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.077+0000 7efe91600640 1 --2- 172.21.15.94:0/1039992671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe8c0cf3c0 0x7efe8c159de0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:20.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.081+0000 7efe91600640 1 -- 172.21.15.94:0/1039992671 >> 172.21.15.94:0/1039992671 conn(0x7efe8c06cdf0 msgr2=0x7efe8c073f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:20.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.081+0000 7efe91600640 1 -- 172.21.15.94:0/1039992671 shutdown_connections 2024-09-10T23:38:20.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.081+0000 7efe91600640 1 -- 172.21.15.94:0/1039992671 wait complete. 2024-09-10T23:38:20.102 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T23:38:20.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.245+0000 7fe8eb800640 1 -- 172.21.15.94:0/1592414197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e415bf60 msgr2=0x7fe8e40d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:20.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.245+0000 7fe8eb800640 1 --2- 172.21.15.94:0/1592414197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e415bf60 0x7fe8e40d2780 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009930 tx=0x7fe8d801c850 comp rx=0 tx=0).stop 2024-09-10T23:38:20.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.245+0000 7fe8eb800640 1 -- 172.21.15.94:0/1592414197 shutdown_connections 2024-09-10T23:38:20.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.245+0000 7fe8eb800640 1 --2- 172.21.15.94:0/1592414197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e40cfce0 0x7fe8e40d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:20.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.245+0000 7fe8eb800640 1 --2- 172.21.15.94:0/1592414197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e415bf60 0x7fe8e40d2780 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:20.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.245+0000 7fe8eb800640 1 -- 172.21.15.94:0/1592414197 >> 172.21.15.94:0/1592414197 conn(0x7fe8e40d1c60 msgr2=0x7fe8e40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:20.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.245+0000 7fe8eb800640 1 -- 172.21.15.94:0/1592414197 shutdown_connections 2024-09-10T23:38:20.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.245+0000 7fe8eb800640 1 -- 172.21.15.94:0/1592414197 wait complete. 2024-09-10T23:38:20.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.245+0000 7fe8eb800640 1 Processor -- start 2024-09-10T23:38:20.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.245+0000 7fe8eb800640 1 -- start start 2024-09-10T23:38:20.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8eb800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e40cfce0 0x7fe8e415eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:20.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8eb800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e40d0660 0x7fe8e40b7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:20.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8eb800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e415bf60 0x7fe8e40b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:20.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8eb800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e4168dc0 con 0x7fe8e40d0660 2024-09-10T23:38:20.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8eb800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e4168c40 con 0x7fe8e40cfce0 2024-09-10T23:38:20.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8eb800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e4168f40 con 0x7fe8e415bf60 2024-09-10T23:38:20.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8e3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e40d0660 0x7fe8e40b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:20.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8e3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e40d0660 0x7fe8e40b7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33368/0 (socket says 172.21.15.94:33368) 2024-09-10T23:38:20.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8e3e00640 1 -- 172.21.15.94:0/4039243817 learned_addr learned my addr 172.21.15.94:0/4039243817 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:20.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8e9800640 1 --2- 172.21.15.94:0/4039243817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e415bf60 0x7fe8e40b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:20.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8e8e00640 1 --2- 172.21.15.94:0/4039243817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e40cfce0 0x7fe8e415eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:20.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8e3e00640 1 -- 172.21.15.94:0/4039243817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e415bf60 msgr2=0x7fe8e40b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:20.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8e3e00640 1 --2- 172.21.15.94:0/4039243817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e415bf60 0x7fe8e40b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:20.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8e3e00640 1 -- 172.21.15.94:0/4039243817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e40cfce0 msgr2=0x7fe8e415eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:20.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8e3e00640 1 --2- 172.21.15.94:0/4039243817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e40cfce0 0x7fe8e415eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:20.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8e3e00640 1 -- 172.21.15.94:0/4039243817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e41fc0c0 con 0x7fe8e40d0660 2024-09-10T23:38:20.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8e9800640 1 --2- 172.21.15.94:0/4039243817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e415bf60 0x7fe8e40b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:20.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8e3e00640 1 --2- 172.21.15.94:0/4039243817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e40d0660 0x7fe8e40b7060 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000e9c0 tx=0x7fe8d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:20.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8e0c00640 1 -- 172.21.15.94:0/4039243817 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d000cd80 con 0x7fe8e40d0660 2024-09-10T23:38:20.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8e0c00640 1 -- 172.21.15.94:0/4039243817 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8d0004020 con 0x7fe8e40d0660 2024-09-10T23:38:20.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8eb800640 1 -- 172.21.15.94:0/4039243817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e41f7fd0 con 0x7fe8e40d0660 2024-09-10T23:38:20.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8e0c00640 1 -- 172.21.15.94:0/4039243817 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0004450 con 0x7fe8e40d0660 2024-09-10T23:38:20.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8eb800640 1 -- 172.21.15.94:0/4039243817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e41f8590 con 0x7fe8e40d0660 2024-09-10T23:38:20.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8eb800640 1 -- 172.21.15.94:0/4039243817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e40d0120 con 0x7fe8e40d0660 2024-09-10T23:38:20.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.249+0000 7fe8e0c00640 1 -- 172.21.15.94:0/4039243817 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7fe8d0003600 con 0x7fe8e40d0660 2024-09-10T23:38:20.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.253+0000 7fe8e0c00640 1 --2- 172.21.15.94:0/4039243817 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe8c4072a60 0x7fe8c4074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:20.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.253+0000 7fe8e0c00640 1 -- 172.21.15.94:0/4039243817 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5737+0+0 (secure 0 0 0) 0x7fe8d00945a0 con 0x7fe8e40d0660 2024-09-10T23:38:20.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.253+0000 7fe8e8e00640 1 --2- 172.21.15.94:0/4039243817 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe8c4072a60 0x7fe8c4074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:20.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.253+0000 7fe8e8e00640 1 --2- 172.21.15.94:0/4039243817 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe8c4072a60 0x7fe8c4074f20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80af000 tx=0x7fe8d801b090 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:20.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.257+0000 7fe8e0c00640 1 -- 172.21.15.94:0/4039243817 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe8d005f520 con 0x7fe8e40d0660 2024-09-10T23:38:20.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:20.461+0000 7fe8eb800640 1 -- 172.21.15.94:0/4039243817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe8e4156fa0 con 0x7fe8e40d0660 2024-09-10T23:38:21.346 INFO:tasks.ceph.mds.b.smithi181.stderr:2024-09-10T23:38:21.345+0000 7f49ac400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:38:21.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.377+0000 7fe8e0c00640 1 -- 172.21.15.94:0/4039243817 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v77) ==== 118+0+0 (secure 0 0 0) 0x7fe8d005ee20 con 0x7fe8e40d0660 2024-09-10T23:38:21.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.381+0000 7fe8eb800640 1 -- 172.21.15.94:0/4039243817 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe8c4072a60 msgr2=0x7fe8c4074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:21.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.381+0000 7fe8eb800640 1 --2- 172.21.15.94:0/4039243817 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe8c4072a60 0x7fe8c4074f20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80af000 tx=0x7fe8d801b090 comp rx=0 tx=0).stop 2024-09-10T23:38:21.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.381+0000 7fe8eb800640 1 -- 172.21.15.94:0/4039243817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e40d0660 msgr2=0x7fe8e40b7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:21.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.381+0000 7fe8eb800640 1 --2- 172.21.15.94:0/4039243817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e40d0660 0x7fe8e40b7060 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000e9c0 tx=0x7fe8d000ee80 comp rx=0 tx=0).stop 2024-09-10T23:38:21.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.381+0000 7fe8eb800640 1 -- 172.21.15.94:0/4039243817 shutdown_connections 2024-09-10T23:38:21.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.381+0000 7fe8eb800640 1 --2- 172.21.15.94:0/4039243817 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe8c4072a60 0x7fe8c4074f20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:21.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.381+0000 7fe8eb800640 1 --2- 172.21.15.94:0/4039243817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e415bf60 0x7fe8e40b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:21.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.381+0000 7fe8eb800640 1 --2- 172.21.15.94:0/4039243817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8e40d0660 0x7fe8e40b7060 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:21.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.381+0000 7fe8eb800640 1 --2- 172.21.15.94:0/4039243817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e40cfce0 0x7fe8e415eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:21.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.381+0000 7fe8eb800640 1 -- 172.21.15.94:0/4039243817 >> 172.21.15.94:0/4039243817 conn(0x7fe8e40d1c60 msgr2=0x7fe8e4071570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:21.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.381+0000 7fe8eb800640 1 -- 172.21.15.94:0/4039243817 shutdown_connections 2024-09-10T23:38:21.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.381+0000 7fe8eb800640 1 -- 172.21.15.94:0/4039243817 wait complete. 2024-09-10T23:38:21.404 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T23:38:21.404 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T23:38:21.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.549+0000 7f2cd3400640 1 -- 172.21.15.94:0/2433563724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cb0004810 msgr2=0x7f2cb0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:21.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.549+0000 7f2cd3400640 1 --2- 172.21.15.94:0/2433563724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2cb0004810 0x7f2cb0004c90 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009b60 tx=0x7f2cb401d210 comp rx=0 tx=0).stop 2024-09-10T23:38:21.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.549+0000 7f2cd1400640 1 -- 172.21.15.94:0/2433563724 reap_dead start 2024-09-10T23:38:21.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.549+0000 7f2cd3400640 1 -- 172.21.15.94:0/2433563724 shutdown_connections 2024-09-10T23:38:21.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.549+0000 7f2cd3400640 1 -- 172.21.15.94:0/2433563724 >> 172.21.15.94:0/2433563724 conn(0x7f2ccc0cfaf0 msgr2=0x7f2ccc0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:21.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.549+0000 7f2cd3400640 1 -- 172.21.15.94:0/2433563724 shutdown_connections 2024-09-10T23:38:21.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.549+0000 7f2cd3400640 1 -- 172.21.15.94:0/2433563724 wait complete. 2024-09-10T23:38:21.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cd3400640 1 Processor -- start 2024-09-10T23:38:21.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cd3400640 1 -- start start 2024-09-10T23:38:21.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cd3400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cb0004810 0x7f2ccc159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:21.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cd3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ccc072980 0x7f2ccc072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:21.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cd3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ccc1f8250 0x7f2ccc1f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:21.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cd3400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc0cf400 con 0x7f2ccc072980 2024-09-10T23:38:21.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cd3400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc0cf280 con 0x7f2ccc1f8250 2024-09-10T23:38:21.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cd3400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2ccc0cf580 con 0x7f2cb0004810 2024-09-10T23:38:21.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cc3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ccc072980 0x7f2ccc072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:21.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cd0a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cb0004810 0x7f2ccc159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:21.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cc3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ccc072980 0x7f2ccc072e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33394/0 (socket says 172.21.15.94:33394) 2024-09-10T23:38:21.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cc3e00640 1 -- 172.21.15.94:0/1429106908 learned_addr learned my addr 172.21.15.94:0/1429106908 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:21.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cd1400640 1 --2- 172.21.15.94:0/1429106908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ccc1f8250 0x7f2ccc1f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:21.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cc3e00640 1 -- 172.21.15.94:0/1429106908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cb0004810 msgr2=0x7f2ccc159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:21.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cc3e00640 1 --2- 172.21.15.94:0/1429106908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cb0004810 0x7f2ccc159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:21.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cc3e00640 1 -- 172.21.15.94:0/1429106908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ccc1f8250 msgr2=0x7f2ccc1f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:21.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cc3e00640 1 --2- 172.21.15.94:0/1429106908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ccc1f8250 0x7f2ccc1f86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:21.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cc3e00640 1 -- 172.21.15.94:0/1429106908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc1f4c60 con 0x7f2ccc072980 2024-09-10T23:38:21.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cc3e00640 1 --2- 172.21.15.94:0/1429106908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ccc072980 0x7f2ccc072e00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f2cb401d040 tx=0x7f2cb4005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:21.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cc9600640 1 -- 172.21.15.94:0/1429106908 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40ba070 con 0x7f2ccc072980 2024-09-10T23:38:21.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cc9600640 1 -- 172.21.15.94:0/1429106908 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cb40ac8a0 con 0x7f2ccc072980 2024-09-10T23:38:21.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cc9600640 1 -- 172.21.15.94:0/1429106908 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40b1040 con 0x7f2ccc072980 2024-09-10T23:38:21.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cd3400640 1 -- 172.21.15.94:0/1429106908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ccc1f8f60 con 0x7f2ccc072980 2024-09-10T23:38:21.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.553+0000 7f2cd3400640 1 -- 172.21.15.94:0/1429106908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ccc1f51c0 con 0x7f2ccc072980 2024-09-10T23:38:21.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.557+0000 7f2cd3400640 1 -- 172.21.15.94:0/1429106908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ccc06ec50 con 0x7f2ccc072980 2024-09-10T23:38:21.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.557+0000 7f2cc9600640 1 -- 172.21.15.94:0/1429106908 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f2cb40046d0 con 0x7f2ccc072980 2024-09-10T23:38:21.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.557+0000 7f2cc9600640 1 --2- 172.21.15.94:0/1429106908 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2ca8072a60 0x7f2ca8074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:21.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.557+0000 7f2cc9600640 1 -- 172.21.15.94:0/1429106908 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f2cb412e870 con 0x7f2ccc072980 2024-09-10T23:38:21.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.557+0000 7f2cd0a00640 1 --2- 172.21.15.94:0/1429106908 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2ca8072a60 0x7f2ca8074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:21.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.557+0000 7f2cd0a00640 1 --2- 172.21.15.94:0/1429106908 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2ca8072a60 0x7f2ca8074f20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc003fb0 tx=0x7f2cbc001500 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:21.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.561+0000 7f2cc9600640 1 -- 172.21.15.94:0/1429106908 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2cb40c4020 con 0x7f2ccc072980 2024-09-10T23:38:21.639 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-10T23:38:21.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:21.761+0000 7f2cd3400640 1 -- 172.21.15.94:0/1429106908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2ccc0d0850 con 0x7f2ccc072980 2024-09-10T23:38:22.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.385+0000 7f2cc9600640 1 -- 172.21.15.94:0/1429106908 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v78) ==== 122+0+0 (secure 0 0 0) 0x7f2ccc1f4c60 con 0x7f2ccc072980 2024-09-10T23:38:22.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.389+0000 7f2cd3400640 1 -- 172.21.15.94:0/1429106908 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2ca8072a60 msgr2=0x7f2ca8074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:22.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.389+0000 7f2cd3400640 1 --2- 172.21.15.94:0/1429106908 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2ca8072a60 0x7f2ca8074f20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc003fb0 tx=0x7f2cbc001500 comp rx=0 tx=0).stop 2024-09-10T23:38:22.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.389+0000 7f2cd3400640 1 -- 172.21.15.94:0/1429106908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ccc072980 msgr2=0x7f2ccc072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:22.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.389+0000 7f2cd3400640 1 --2- 172.21.15.94:0/1429106908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ccc072980 0x7f2ccc072e00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f2cb401d040 tx=0x7f2cb4005620 comp rx=0 tx=0).stop 2024-09-10T23:38:22.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.389+0000 7f2cd3400640 1 -- 172.21.15.94:0/1429106908 shutdown_connections 2024-09-10T23:38:22.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.389+0000 7f2cd3400640 1 --2- 172.21.15.94:0/1429106908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ccc1f8250 0x7f2ccc1f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.389+0000 7f2cd3400640 1 --2- 172.21.15.94:0/1429106908 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2ca8072a60 0x7f2ca8074f20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.389+0000 7f2cd3400640 1 --2- 172.21.15.94:0/1429106908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ccc072980 0x7f2ccc072e00 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.389+0000 7f2cd3400640 1 --2- 172.21.15.94:0/1429106908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cb0004810 0x7f2ccc159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.389+0000 7f2cd3400640 1 -- 172.21.15.94:0/1429106908 >> 172.21.15.94:0/1429106908 conn(0x7f2ccc0cfaf0 msgr2=0x7f2ccc0c49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:22.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.389+0000 7f2cd3400640 1 -- 172.21.15.94:0/1429106908 shutdown_connections 2024-09-10T23:38:22.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.389+0000 7f2cd3400640 1 -- 172.21.15.94:0/1429106908 wait complete. 2024-09-10T23:38:22.414 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T23:38:22.414 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:38:22.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.557+0000 7ff38cc00640 1 -- 172.21.15.94:0/1062166876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff36c003a10 msgr2=0x7ff36c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:22.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.557+0000 7ff38cc00640 1 --2- 172.21.15.94:0/1062166876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff36c003a10 0x7ff36c003e30 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7ff37800c310 tx=0x7ff37801cfc0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.557+0000 7ff386800640 1 -- 172.21.15.94:0/1062166876 reap_dead start 2024-09-10T23:38:22.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.557+0000 7ff38cc00640 1 -- 172.21.15.94:0/1062166876 shutdown_connections 2024-09-10T23:38:22.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.557+0000 7ff38cc00640 1 -- 172.21.15.94:0/1062166876 >> 172.21.15.94:0/1062166876 conn(0x7ff3880d1c60 msgr2=0x7ff3880d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:22.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.557+0000 7ff38cc00640 1 -- 172.21.15.94:0/1062166876 shutdown_connections 2024-09-10T23:38:22.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.557+0000 7ff38cc00640 1 -- 172.21.15.94:0/1062166876 wait complete. 2024-09-10T23:38:22.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff38cc00640 1 Processor -- start 2024-09-10T23:38:22.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff38cc00640 1 -- start start 2024-09-10T23:38:22.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff38cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff36c003a10 0x7ff388150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:22.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff38cc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff38813f5b0 0x7ff3880c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:22.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff38cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3880c4d90 0x7ff388161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:22.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff38cc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3880b05c0 con 0x7ff36c003a10 2024-09-10T23:38:22.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff38cc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff3880b0440 con 0x7ff3880c4d90 2024-09-10T23:38:22.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff38cc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff388166aa0 con 0x7ff38813f5b0 2024-09-10T23:38:22.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff385e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff36c003a10 0x7ff388150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:22.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff385e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff36c003a10 0x7ff388150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33412/0 (socket says 172.21.15.94:33412) 2024-09-10T23:38:22.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff385e00640 1 -- 172.21.15.94:0/2118272278 learned_addr learned my addr 172.21.15.94:0/2118272278 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:22.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff37d400640 1 --2- 172.21.15.94:0/2118272278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff38813f5b0 0x7ff3880c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:22.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff386800640 1 --2- 172.21.15.94:0/2118272278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3880c4d90 0x7ff388161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:22.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff385e00640 1 -- 172.21.15.94:0/2118272278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff38813f5b0 msgr2=0x7ff3880c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:22.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff385e00640 1 --2- 172.21.15.94:0/2118272278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff38813f5b0 0x7ff3880c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff385e00640 1 -- 172.21.15.94:0/2118272278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3880c4d90 msgr2=0x7ff388161ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:22.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff385e00640 1 --2- 172.21.15.94:0/2118272278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3880c4d90 0x7ff388161ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff385e00640 1 -- 172.21.15.94:0/2118272278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38815fb40 con 0x7ff36c003a10 2024-09-10T23:38:22.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff386800640 1 --2- 172.21.15.94:0/2118272278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3880c4d90 0x7ff388161ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:22.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff37d400640 1 --2- 172.21.15.94:0/2118272278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff38813f5b0 0x7ff3880c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:22.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff385e00640 1 --2- 172.21.15.94:0/2118272278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff36c003a10 0x7ff388150c50 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff3780989a0 tx=0x7ff378001030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:22.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff37ea00640 1 -- 172.21.15.94:0/2118272278 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378006090 con 0x7ff36c003a10 2024-09-10T23:38:22.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff37ea00640 1 -- 172.21.15.94:0/2118272278 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff378006250 con 0x7ff36c003a10 2024-09-10T23:38:22.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff38cc00640 1 -- 172.21.15.94:0/2118272278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38815fde0 con 0x7ff36c003a10 2024-09-10T23:38:22.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff37ea00640 1 -- 172.21.15.94:0/2118272278 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378003600 con 0x7ff36c003a10 2024-09-10T23:38:22.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.561+0000 7ff38cc00640 1 -- 172.21.15.94:0/2118272278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff388151190 con 0x7ff36c003a10 2024-09-10T23:38:22.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.565+0000 7ff38cc00640 1 -- 172.21.15.94:0/2118272278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3880af7c0 con 0x7ff36c003a10 2024-09-10T23:38:22.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.565+0000 7ff37ea00640 1 -- 172.21.15.94:0/2118272278 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7ff3780037a0 con 0x7ff36c003a10 2024-09-10T23:38:22.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.565+0000 7ff37ea00640 1 --2- 172.21.15.94:0/2118272278 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff364072a90 0x7ff364074f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:22.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.565+0000 7ff37ea00640 1 -- 172.21.15.94:0/2118272278 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7ff37812d5e0 con 0x7ff36c003a10 2024-09-10T23:38:22.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.565+0000 7ff37d400640 1 --2- 172.21.15.94:0/2118272278 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff364072a90 0x7ff364074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:22.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.565+0000 7ff37d400640 1 --2- 172.21.15.94:0/2118272278 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff364072a90 0x7ff364074f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff38814ebd0 tx=0x7ff3700091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:22.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.569+0000 7ff37ea00640 1 -- 172.21.15.94:0/2118272278 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff378016d70 con 0x7ff36c003a10 2024-09-10T23:38:22.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.773+0000 7ff38cc00640 1 -- 172.21.15.94:0/2118272278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3880749c0 con 0x7ff36c003a10 2024-09-10T23:38:22.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.773+0000 7ff37ea00640 1 -- 172.21.15.94:0/2118272278 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5241 (secure 0 0 0) 0x7ff3780f85d0 con 0x7ff36c003a10 2024-09-10T23:38:22.774 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:22.774 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":78,"btime":"2024-09-10T23:38:22:381526+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7090,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3097381199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3097381199},{"type":"v1","addr":"172.21.15.181:6835","nonce":3097381199}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2654663117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2654663117},{"type":"v1","addr":"172.21.15.94:6833","nonce":2654663117}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7530,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1334394570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1334394570},{"type":"v1","addr":"172.21.15.94:6835","nonce":1334394570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:38:18.757602+0000","modified":"2024-09-10T23:38:21.765704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196283":{"gid":196283,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6834/1748173202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":1748173202},{"type":"v1","addr":"172.21.15.181:6834","nonce":1748173202}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196283,"qdb_cluster":[196283]},"id":9}]} 2024-09-10T23:38:22.775 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 78 2024-09-10T23:38:22.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.773+0000 7ff38cc00640 1 -- 172.21.15.94:0/2118272278 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff364072a90 msgr2=0x7ff364074f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:22.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.773+0000 7ff38cc00640 1 --2- 172.21.15.94:0/2118272278 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff364072a90 0x7ff364074f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff38814ebd0 tx=0x7ff3700091e0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.773+0000 7ff38cc00640 1 -- 172.21.15.94:0/2118272278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff36c003a10 msgr2=0x7ff388150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:22.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.773+0000 7ff38cc00640 1 --2- 172.21.15.94:0/2118272278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff36c003a10 0x7ff388150c50 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff3780989a0 tx=0x7ff378001030 comp rx=0 tx=0).stop 2024-09-10T23:38:22.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.773+0000 7ff38cc00640 1 -- 172.21.15.94:0/2118272278 shutdown_connections 2024-09-10T23:38:22.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.773+0000 7ff38cc00640 1 --2- 172.21.15.94:0/2118272278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3880c4d90 0x7ff388161ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.773+0000 7ff38cc00640 1 --2- 172.21.15.94:0/2118272278 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff364072a90 0x7ff364074f50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.773+0000 7ff38cc00640 1 --2- 172.21.15.94:0/2118272278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff38813f5b0 0x7ff3880c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.773+0000 7ff38cc00640 1 --2- 172.21.15.94:0/2118272278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff36c003a10 0x7ff388150c50 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.773+0000 7ff38cc00640 1 -- 172.21.15.94:0/2118272278 >> 172.21.15.94:0/2118272278 conn(0x7ff3880d1c60 msgr2=0x7ff388165a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:22.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.773+0000 7ff38cc00640 1 -- 172.21.15.94:0/2118272278 shutdown_connections 2024-09-10T23:38:22.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.773+0000 7ff38cc00640 1 -- 172.21.15.94:0/2118272278 wait complete. 2024-09-10T23:38:22.795 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:38:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.937+0000 7fc711c00640 1 -- 172.21.15.94:0/886662924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc70c15ba50 msgr2=0x7fc70c0b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.937+0000 7fc711c00640 1 --2- 172.21.15.94:0/886662924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc70c15ba50 0x7fc70c0b4700 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0d0890 tx=0x7fc6fc01c7c0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.937+0000 7fc711c00640 1 -- 172.21.15.94:0/886662924 shutdown_connections 2024-09-10T23:38:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.937+0000 7fc711c00640 1 --2- 172.21.15.94:0/886662924 >> v2:172.21.15.181:3300/0 conn(0x7fc70c0d03c0 0x7fc70c0cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.937+0000 7fc711c00640 1 --2- 172.21.15.94:0/886662924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc70c15ba50 0x7fc70c0b4700 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.937+0000 7fc711c00640 1 -- 172.21.15.94:0/886662924 >> 172.21.15.94:0/886662924 conn(0x7fc70c06cdf0 msgr2=0x7fc70c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:22.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.937+0000 7fc711c00640 1 -- 172.21.15.94:0/886662924 shutdown_connections 2024-09-10T23:38:22.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.937+0000 7fc711c00640 1 -- 172.21.15.94:0/886662924 wait complete. 2024-09-10T23:38:22.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.937+0000 7fc711c00640 1 Processor -- start 2024-09-10T23:38:22.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.937+0000 7fc711c00640 1 -- start start 2024-09-10T23:38:22.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc711c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc70c0b4c40 0x7fc70c1425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:22.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc711c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc70c0d03c0 0x7fc70c14b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:22.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc711c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc70c15ba50 0x7fc70c15e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:22.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc711c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1689d0 con 0x7fc70c0b4c40 2024-09-10T23:38:22.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc711c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c168850 con 0x7fc70c0d03c0 2024-09-10T23:38:22.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc711c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc70c168b50 con 0x7fc70c15ba50 2024-09-10T23:38:22.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc70aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc70c0b4c40 0x7fc70c1425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:22.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc70aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc70c0b4c40 0x7fc70c1425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33422/0 (socket says 172.21.15.94:33422) 2024-09-10T23:38:22.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc70aa00640 1 -- 172.21.15.94:0/1838674682 learned_addr learned my addr 172.21.15.94:0/1838674682 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:22.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc70b400640 1 --2- 172.21.15.94:0/1838674682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc70c15ba50 0x7fc70c15e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:22.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc70a000640 1 --2- 172.21.15.94:0/1838674682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc70c0d03c0 0x7fc70c14b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:22.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc70aa00640 1 -- 172.21.15.94:0/1838674682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc70c15ba50 msgr2=0x7fc70c15e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:22.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc70aa00640 1 --2- 172.21.15.94:0/1838674682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc70c15ba50 0x7fc70c15e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc70aa00640 1 -- 172.21.15.94:0/1838674682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc70c0d03c0 msgr2=0x7fc70c14b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:22.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc70aa00640 1 --2- 172.21.15.94:0/1838674682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc70c0d03c0 0x7fc70c14b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:22.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc70aa00640 1 -- 172.21.15.94:0/1838674682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c149e00 con 0x7fc70c0b4c40 2024-09-10T23:38:22.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc70aa00640 1 --2- 172.21.15.94:0/1838674682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc70c0b4c40 0x7fc70c1425a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0b0040 tx=0x7fc6fc0a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:22.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc702a00640 1 -- 172.21.15.94:0/1838674682 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc0b2040 con 0x7fc70c0b4c40 2024-09-10T23:38:22.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc702a00640 1 -- 172.21.15.94:0/1838674682 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6fc004340 con 0x7fc70c0b4c40 2024-09-10T23:38:22.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc711c00640 1 -- 172.21.15.94:0/1838674682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c14a0e0 con 0x7fc70c0b4c40 2024-09-10T23:38:22.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc702a00640 1 -- 172.21.15.94:0/1838674682 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc005290 con 0x7fc70c0b4c40 2024-09-10T23:38:22.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc711c00640 1 -- 172.21.15.94:0/1838674682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c145900 con 0x7fc70c0b4c40 2024-09-10T23:38:22.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc711c00640 1 -- 172.21.15.94:0/1838674682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70c0cddb0 con 0x7fc70c0b4c40 2024-09-10T23:38:22.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.941+0000 7fc702a00640 1 -- 172.21.15.94:0/1838674682 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7fc6fc005430 con 0x7fc70c0b4c40 2024-09-10T23:38:22.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.945+0000 7fc702a00640 1 --2- 172.21.15.94:0/1838674682 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc6ec072a90 0x7fc6ec074f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:22.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.945+0000 7fc702a00640 1 -- 172.21.15.94:0/1838674682 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fc6fc12d640 con 0x7fc70c0b4c40 2024-09-10T23:38:22.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.945+0000 7fc70a000640 1 --2- 172.21.15.94:0/1838674682 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc6ec072a90 0x7fc6ec074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:22.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.945+0000 7fc70a000640 1 --2- 172.21.15.94:0/1838674682 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc6ec072a90 0x7fc6ec074f50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc70c143140 tx=0x7fc6f8009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:22.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:22.949+0000 7fc702a00640 1 -- 172.21.15.94:0/1838674682 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc6fc016470 con 0x7fc70c0b4c40 2024-09-10T23:38:23.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.113+0000 7fc711c00640 1 -- 172.21.15.94:0/1838674682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc70c14cd60 con 0x7fc70c0b4c40 2024-09-10T23:38:23.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.117+0000 7fc702a00640 1 -- 172.21.15.94:0/1838674682 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+16719 (secure 0 0 0) 0x7fc6fc0f85c0 con 0x7fc70c0b4c40 2024-09-10T23:38:23.118 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:23.119 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":114,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:38:20.369832+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:38:16.443089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T23:38:17.716329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:38:23.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.117+0000 7fc711c00640 1 -- 172.21.15.94:0/1838674682 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc6ec072a90 msgr2=0x7fc6ec074f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:23.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.117+0000 7fc711c00640 1 --2- 172.21.15.94:0/1838674682 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc6ec072a90 0x7fc6ec074f50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc70c143140 tx=0x7fc6f8009040 comp rx=0 tx=0).stop 2024-09-10T23:38:23.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.117+0000 7fc711c00640 1 -- 172.21.15.94:0/1838674682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc70c0b4c40 msgr2=0x7fc70c1425a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:23.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.117+0000 7fc711c00640 1 --2- 172.21.15.94:0/1838674682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc70c0b4c40 0x7fc70c1425a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0b0040 tx=0x7fc6fc0a5750 comp rx=0 tx=0).stop 2024-09-10T23:38:23.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.117+0000 7fc711c00640 1 -- 172.21.15.94:0/1838674682 shutdown_connections 2024-09-10T23:38:23.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.117+0000 7fc711c00640 1 --2- 172.21.15.94:0/1838674682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc70c0d03c0 0x7fc70c14b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.117+0000 7fc711c00640 1 --2- 172.21.15.94:0/1838674682 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc6ec072a90 0x7fc6ec074f50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.117+0000 7fc711c00640 1 --2- 172.21.15.94:0/1838674682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc70c15ba50 0x7fc70c15e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.117+0000 7fc711c00640 1 --2- 172.21.15.94:0/1838674682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc70c0b4c40 0x7fc70c1425a0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.117+0000 7fc711c00640 1 -- 172.21.15.94:0/1838674682 >> 172.21.15.94:0/1838674682 conn(0x7fc70c06cdf0 msgr2=0x7fc70c0cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:23.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.117+0000 7fc711c00640 1 -- 172.21.15.94:0/1838674682 shutdown_connections 2024-09-10T23:38:23.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.117+0000 7fc711c00640 1 -- 172.21.15.94:0/1838674682 wait complete. 2024-09-10T23:38:23.141 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T23:38:23.141 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:38:23.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.281+0000 7f46fa200640 1 -- 172.21.15.94:0/2724316545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f4070860 msgr2=0x7f46f4070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:23.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.281+0000 7f46fa200640 1 --2- 172.21.15.94:0/2724316545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f4070860 0x7f46f4070cb0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f46f40b6ca0 tx=0x7f46dc01c830 comp rx=0 tx=0).stop 2024-09-10T23:38:23.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.281+0000 7f46fa200640 1 -- 172.21.15.94:0/2724316545 shutdown_connections 2024-09-10T23:38:23.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.281+0000 7f46fa200640 1 --2- 172.21.15.94:0/2724316545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f4070860 0x7f46f4070cb0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.281+0000 7f46fa200640 1 --2- 172.21.15.94:0/2724316545 >> v2:172.21.15.181:3300/0 conn(0x7f46f40d0770 0x7f46f415ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.281+0000 7f46fa200640 1 -- 172.21.15.94:0/2724316545 >> 172.21.15.94:0/2724316545 conn(0x7f46f40d0b70 msgr2=0x7f46f40d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:23.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.281+0000 7f46fa200640 1 -- 172.21.15.94:0/2724316545 shutdown_connections 2024-09-10T23:38:23.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.281+0000 7f46fa200640 1 -- 172.21.15.94:0/2724316545 wait complete. 2024-09-10T23:38:23.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.281+0000 7f46fa200640 1 Processor -- start 2024-09-10T23:38:23.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.281+0000 7f46fa200640 1 -- start start 2024-09-10T23:38:23.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46fa200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f406f410 0x7f46f414be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:23.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46fa200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46f4070860 0x7f46f415d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:23.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46fa200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46f40d0770 0x7f46f4154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:23.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46fa200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41682b0 con 0x7f46f406f410 2024-09-10T23:38:23.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46fa200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f46f4168130 con 0x7f46f4070860 2024-09-10T23:38:23.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46fa200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f46f4168430 con 0x7f46f40d0770 2024-09-10T23:38:23.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46f3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f406f410 0x7f46f414be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:23.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46f3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f406f410 0x7f46f414be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33432/0 (socket says 172.21.15.94:33432) 2024-09-10T23:38:23.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46f3400640 1 -- 172.21.15.94:0/3176490762 learned_addr learned my addr 172.21.15.94:0/3176490762 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:23.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46eaa00640 1 --2- 172.21.15.94:0/3176490762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46f4070860 0x7f46f415d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:23.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46f3e00640 1 --2- 172.21.15.94:0/3176490762 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46f40d0770 0x7f46f4154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:23.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46f3400640 1 -- 172.21.15.94:0/3176490762 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46f40d0770 msgr2=0x7f46f4154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:23.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46f3400640 1 --2- 172.21.15.94:0/3176490762 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46f40d0770 0x7f46f4154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46f3400640 1 -- 172.21.15.94:0/3176490762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46f4070860 msgr2=0x7f46f415d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:23.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46f3400640 1 --2- 172.21.15.94:0/3176490762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46f4070860 0x7f46f415d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46f3400640 1 -- 172.21.15.94:0/3176490762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f414fa70 con 0x7f46f406f410 2024-09-10T23:38:23.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46f3400640 1 --2- 172.21.15.94:0/3176490762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f406f410 0x7f46f414be40 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f46e400d720 tx=0x7f46e400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:23.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46ebe00640 1 -- 172.21.15.94:0/3176490762 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4017070 con 0x7f46f406f410 2024-09-10T23:38:23.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46ebe00640 1 -- 172.21.15.94:0/3176490762 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46e4009d20 con 0x7f46f406f410 2024-09-10T23:38:23.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46ebe00640 1 -- 172.21.15.94:0/3176490762 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4005280 con 0x7f46f406f410 2024-09-10T23:38:23.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46fa200640 1 -- 172.21.15.94:0/3176490762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f414f1d0 con 0x7f46f406f410 2024-09-10T23:38:23.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46fa200640 1 -- 172.21.15.94:0/3176490762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f414f790 con 0x7f46f406f410 2024-09-10T23:38:23.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46fa200640 1 -- 172.21.15.94:0/3176490762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f4070cb0 con 0x7f46f406f410 2024-09-10T23:38:23.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.285+0000 7f46ebe00640 1 -- 172.21.15.94:0/3176490762 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f46e4005420 con 0x7f46f406f410 2024-09-10T23:38:23.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.289+0000 7f46ebe00640 1 --2- 172.21.15.94:0/3176490762 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f46d0072a60 0x7f46d0074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:23.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.289+0000 7f46ebe00640 1 -- 172.21.15.94:0/3176490762 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f46e4093c70 con 0x7f46f406f410 2024-09-10T23:38:23.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.289+0000 7f46eaa00640 1 --2- 172.21.15.94:0/3176490762 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f46d0072a60 0x7f46d0074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:23.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.289+0000 7f46eaa00640 1 --2- 172.21.15.94:0/3176490762 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f46d0072a60 0x7f46d0074f20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0af000 tx=0x7f46dc01b070 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:23.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.293+0000 7f46ebe00640 1 -- 172.21.15.94:0/3176490762 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f46e405ebf0 con 0x7f46f406f410 2024-09-10T23:38:23.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.457+0000 7f46fa200640 1 -- 172.21.15.94:0/3176490762 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46f40cff10 con 0x7f46d0072a60 2024-09-10T23:38:23.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.461+0000 7f46ebe00640 1 -- 172.21.15.94:0/3176490762 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252478 (secure 0 0 0) 0x7f46f40cff10 con 0x7f46d0072a60 2024-09-10T23:38:23.465 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:23.466 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:38:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.465+0000 7f46fa200640 1 -- 172.21.15.94:0/3176490762 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f46d0072a60 msgr2=0x7f46d0074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.465+0000 7f46fa200640 1 --2- 172.21.15.94:0/3176490762 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f46d0072a60 0x7f46d0074f20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0af000 tx=0x7f46dc01b070 comp rx=0 tx=0).stop 2024-09-10T23:38:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.465+0000 7f46fa200640 1 -- 172.21.15.94:0/3176490762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f406f410 msgr2=0x7f46f414be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.465+0000 7f46fa200640 1 --2- 172.21.15.94:0/3176490762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f406f410 0x7f46f414be40 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f46e400d720 tx=0x7f46e400dbe0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.465+0000 7f46fa200640 1 -- 172.21.15.94:0/3176490762 shutdown_connections 2024-09-10T23:38:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.465+0000 7f46fa200640 1 --2- 172.21.15.94:0/3176490762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46f4070860 0x7f46f415d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.465+0000 7f46fa200640 1 --2- 172.21.15.94:0/3176490762 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f46d0072a60 0x7f46d0074f20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.465+0000 7f46fa200640 1 --2- 172.21.15.94:0/3176490762 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46f40d0770 0x7f46f4154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.465+0000 7f46fa200640 1 --2- 172.21.15.94:0/3176490762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f406f410 0x7f46f414be40 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.465+0000 7f46fa200640 1 -- 172.21.15.94:0/3176490762 >> 172.21.15.94:0/3176490762 conn(0x7f46f40d0b70 msgr2=0x7f46f40b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:23.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.465+0000 7f46fa200640 1 -- 172.21.15.94:0/3176490762 shutdown_connections 2024-09-10T23:38:23.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.465+0000 7f46fa200640 1 -- 172.21.15.94:0/3176490762 wait complete. 2024-09-10T23:38:23.486 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-10T23:38:22.756898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261824,"kb_used_data":12296,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712896,"statfs":{"total":773094113280,"available":772826005504,"internally_reserved":0,"allocated":12591104,"data_stored":6156748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.409013"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379877+0000","last_change":"2024-09-10T23:38:19.391041+0000","last_active":"2024-09-10T23:38:20.379877+0000","last_peered":"2024-09-10T23:38:20.379877+0000","last_clean":"2024-09-10T23:38:20.379877+0000","last_became_active":"2024-09-10T23:38:19.390779+0000","last_became_peered":"2024-09-10T23:38:19.390779+0000","last_unstale":"2024-09-10T23:38:20.379877+0000","last_undegraded":"2024-09-10T23:38:20.379877+0000","last_fullsized":"2024-09-10T23:38:20.379877+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:35.192273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380328+0000","last_change":"2024-09-10T23:38:19.388129+0000","last_active":"2024-09-10T23:38:20.380328+0000","last_peered":"2024-09-10T23:38:20.380328+0000","last_clean":"2024-09-10T23:38:20.380328+0000","last_became_active":"2024-09-10T23:38:19.387780+0000","last_became_peered":"2024-09-10T23:38:19.387780+0000","last_unstale":"2024-09-10T23:38:20.380328+0000","last_undegraded":"2024-09-10T23:38:20.380328+0000","last_fullsized":"2024-09-10T23:38:20.380328+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:24.573075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256275+0000","last_change":"2024-09-10T23:38:19.380225+0000","last_active":"2024-09-10T23:38:21.256275+0000","last_peered":"2024-09-10T23:38:21.256275+0000","last_clean":"2024-09-10T23:38:21.256275+0000","last_became_active":"2024-09-10T23:38:19.379727+0000","last_became_peered":"2024-09-10T23:38:19.379727+0000","last_unstale":"2024-09-10T23:38:21.256275+0000","last_undegraded":"2024-09-10T23:38:21.256275+0000","last_fullsized":"2024-09-10T23:38:21.256275+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:11:57.180791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256491+0000","last_change":"2024-09-10T23:38:19.396970+0000","last_active":"2024-09-10T23:38:21.256491+0000","last_peered":"2024-09-10T23:38:21.256491+0000","last_clean":"2024-09-10T23:38:21.256491+0000","last_became_active":"2024-09-10T23:38:19.394882+0000","last_became_peered":"2024-09-10T23:38:19.394882+0000","last_unstale":"2024-09-10T23:38:21.256491+0000","last_undegraded":"2024-09-10T23:38:21.256491+0000","last_fullsized":"2024-09-10T23:38:21.256491+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:47.082303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.377999+0000","last_change":"2024-09-10T23:38:19.377197+0000","last_active":"2024-09-10T23:38:20.377999+0000","last_peered":"2024-09-10T23:38:20.377999+0000","last_clean":"2024-09-10T23:38:20.377999+0000","last_became_active":"2024-09-10T23:38:19.376899+0000","last_became_peered":"2024-09-10T23:38:19.376899+0000","last_unstale":"2024-09-10T23:38:20.377999+0000","last_undegraded":"2024-09-10T23:38:20.377999+0000","last_fullsized":"2024-09-10T23:38:20.377999+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:30.531015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384056+0000","last_change":"2024-09-10T23:38:19.381650+0000","last_active":"2024-09-10T23:38:20.384056+0000","last_peered":"2024-09-10T23:38:20.384056+0000","last_clean":"2024-09-10T23:38:20.384056+0000","last_became_active":"2024-09-10T23:38:19.381348+0000","last_became_peered":"2024-09-10T23:38:19.381348+0000","last_unstale":"2024-09-10T23:38:20.384056+0000","last_undegraded":"2024-09-10T23:38:20.384056+0000","last_fullsized":"2024-09-10T23:38:20.384056+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:38.302286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.377778+0000","last_change":"2024-09-10T23:38:19.369944+0000","last_active":"2024-09-10T23:38:20.377778+0000","last_peered":"2024-09-10T23:38:20.377778+0000","last_clean":"2024-09-10T23:38:20.377778+0000","last_became_active":"2024-09-10T23:38:19.369004+0000","last_became_peered":"2024-09-10T23:38:19.369004+0000","last_unstale":"2024-09-10T23:38:20.377778+0000","last_undegraded":"2024-09-10T23:38:20.377778+0000","last_fullsized":"2024-09-10T23:38:20.377778+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:59.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.377772+0000","last_change":"2024-09-10T23:38:19.380217+0000","last_active":"2024-09-10T23:38:20.377772+0000","last_peered":"2024-09-10T23:38:20.377772+0000","last_clean":"2024-09-10T23:38:20.377772+0000","last_became_active":"2024-09-10T23:38:19.379970+0000","last_became_peered":"2024-09-10T23:38:19.379970+0000","last_unstale":"2024-09-10T23:38:20.377772+0000","last_undegraded":"2024-09-10T23:38:20.377772+0000","last_fullsized":"2024-09-10T23:38:20.377772+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:49.402756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256093+0000","last_change":"2024-09-10T23:38:19.381019+0000","last_active":"2024-09-10T23:38:21.256093+0000","last_peered":"2024-09-10T23:38:21.256093+0000","last_clean":"2024-09-10T23:38:21.256093+0000","last_became_active":"2024-09-10T23:38:19.380762+0000","last_became_peered":"2024-09-10T23:38:19.380762+0000","last_unstale":"2024-09-10T23:38:21.256093+0000","last_undegraded":"2024-09-10T23:38:21.256093+0000","last_fullsized":"2024-09-10T23:38:21.256093+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:52.195279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380381+0000","last_change":"2024-09-10T23:38:19.387927+0000","last_active":"2024-09-10T23:38:20.380381+0000","last_peered":"2024-09-10T23:38:20.380381+0000","last_clean":"2024-09-10T23:38:20.380381+0000","last_became_active":"2024-09-10T23:38:19.387498+0000","last_became_peered":"2024-09-10T23:38:19.387498+0000","last_unstale":"2024-09-10T23:38:20.380381+0000","last_undegraded":"2024-09-10T23:38:20.380381+0000","last_fullsized":"2024-09-10T23:38:20.380381+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:27.140310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256543+0000","last_change":"2024-09-10T23:38:19.397733+0000","last_active":"2024-09-10T23:38:21.256543+0000","last_peered":"2024-09-10T23:38:21.256543+0000","last_clean":"2024-09-10T23:38:21.256543+0000","last_became_active":"2024-09-10T23:38:19.397347+0000","last_became_peered":"2024-09-10T23:38:19.397347+0000","last_unstale":"2024-09-10T23:38:21.256543+0000","last_undegraded":"2024-09-10T23:38:21.256543+0000","last_fullsized":"2024-09-10T23:38:21.256543+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:42.847739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384735+0000","last_change":"2024-09-10T23:38:19.382379+0000","last_active":"2024-09-10T23:38:20.384735+0000","last_peered":"2024-09-10T23:38:20.384735+0000","last_clean":"2024-09-10T23:38:20.384735+0000","last_became_active":"2024-09-10T23:38:19.382088+0000","last_became_peered":"2024-09-10T23:38:19.382088+0000","last_unstale":"2024-09-10T23:38:20.384735+0000","last_undegraded":"2024-09-10T23:38:20.384735+0000","last_fullsized":"2024-09-10T23:38:20.384735+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:47.082303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256373+0000","last_change":"2024-09-10T23:38:19.397890+0000","last_active":"2024-09-10T23:38:21.256373+0000","last_peered":"2024-09-10T23:38:21.256373+0000","last_clean":"2024-09-10T23:38:21.256373+0000","last_became_active":"2024-09-10T23:38:19.396940+0000","last_became_peered":"2024-09-10T23:38:19.396940+0000","last_unstale":"2024-09-10T23:38:21.256373+0000","last_undegraded":"2024-09-10T23:38:21.256373+0000","last_fullsized":"2024-09-10T23:38:21.256373+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:56.434591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380478+0000","last_change":"2024-09-10T23:38:19.378937+0000","last_active":"2024-09-10T23:38:20.380478+0000","last_peered":"2024-09-10T23:38:20.380478+0000","last_clean":"2024-09-10T23:38:20.380478+0000","last_became_active":"2024-09-10T23:38:19.376939+0000","last_became_peered":"2024-09-10T23:38:19.376939+0000","last_unstale":"2024-09-10T23:38:20.380478+0000","last_undegraded":"2024-09-10T23:38:20.380478+0000","last_fullsized":"2024-09-10T23:38:20.380478+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:00.549139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.751783+0000","last_change":"2024-09-10T23:38:19.383837+0000","last_active":"2024-09-10T23:38:20.751783+0000","last_peered":"2024-09-10T23:38:20.751783+0000","last_clean":"2024-09-10T23:38:20.751783+0000","last_became_active":"2024-09-10T23:38:19.383519+0000","last_became_peered":"2024-09-10T23:38:19.383519+0000","last_unstale":"2024-09-10T23:38:20.751783+0000","last_undegraded":"2024-09-10T23:38:20.751783+0000","last_fullsized":"2024-09-10T23:38:20.751783+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:58.025635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379484+0000","last_change":"2024-09-10T23:38:19.371075+0000","last_active":"2024-09-10T23:38:20.379484+0000","last_peered":"2024-09-10T23:38:20.379484+0000","last_clean":"2024-09-10T23:38:20.379484+0000","last_became_active":"2024-09-10T23:38:19.370761+0000","last_became_peered":"2024-09-10T23:38:19.370761+0000","last_unstale":"2024-09-10T23:38:20.379484+0000","last_undegraded":"2024-09-10T23:38:20.379484+0000","last_fullsized":"2024-09-10T23:38:20.379484+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:54.021452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256381+0000","last_change":"2024-09-10T23:38:19.399486+0000","last_active":"2024-09-10T23:38:21.256381+0000","last_peered":"2024-09-10T23:38:21.256381+0000","last_clean":"2024-09-10T23:38:21.256381+0000","last_became_active":"2024-09-10T23:38:19.399162+0000","last_became_peered":"2024-09-10T23:38:19.399162+0000","last_unstale":"2024-09-10T23:38:21.256381+0000","last_undegraded":"2024-09-10T23:38:21.256381+0000","last_fullsized":"2024-09-10T23:38:21.256381+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:23.857889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.752063+0000","last_change":"2024-09-10T23:38:19.383285+0000","last_active":"2024-09-10T23:38:20.752063+0000","last_peered":"2024-09-10T23:38:20.752063+0000","last_clean":"2024-09-10T23:38:20.752063+0000","last_became_active":"2024-09-10T23:38:19.382999+0000","last_became_peered":"2024-09-10T23:38:19.382999+0000","last_unstale":"2024-09-10T23:38:20.752063+0000","last_undegraded":"2024-09-10T23:38:20.752063+0000","last_fullsized":"2024-09-10T23:38:20.752063+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:56.528988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380642+0000","last_change":"2024-09-10T23:38:19.388697+0000","last_active":"2024-09-10T23:38:20.380642+0000","last_peered":"2024-09-10T23:38:20.380642+0000","last_clean":"2024-09-10T23:38:20.380642+0000","last_became_active":"2024-09-10T23:38:19.388436+0000","last_became_peered":"2024-09-10T23:38:19.388436+0000","last_unstale":"2024-09-10T23:38:20.380642+0000","last_undegraded":"2024-09-10T23:38:20.380642+0000","last_fullsized":"2024-09-10T23:38:20.380642+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:08.931506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.752119+0000","last_change":"2024-09-10T23:38:19.396084+0000","last_active":"2024-09-10T23:38:20.752119+0000","last_peered":"2024-09-10T23:38:20.752119+0000","last_clean":"2024-09-10T23:38:20.752119+0000","last_became_active":"2024-09-10T23:38:19.395760+0000","last_became_peered":"2024-09-10T23:38:19.395760+0000","last_unstale":"2024-09-10T23:38:20.752119+0000","last_undegraded":"2024-09-10T23:38:20.752119+0000","last_fullsized":"2024-09-10T23:38:20.752119+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:55.207747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384251+0000","last_change":"2024-09-10T23:38:19.382191+0000","last_active":"2024-09-10T23:38:20.384251+0000","last_peered":"2024-09-10T23:38:20.384251+0000","last_clean":"2024-09-10T23:38:20.384251+0000","last_became_active":"2024-09-10T23:38:19.381832+0000","last_became_peered":"2024-09-10T23:38:19.381832+0000","last_unstale":"2024-09-10T23:38:20.384251+0000","last_undegraded":"2024-09-10T23:38:20.384251+0000","last_fullsized":"2024-09-10T23:38:20.384251+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:12.194094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379864+0000","last_change":"2024-09-10T23:38:19.400711+0000","last_active":"2024-09-10T23:38:20.379864+0000","last_peered":"2024-09-10T23:38:20.379864+0000","last_clean":"2024-09-10T23:38:20.379864+0000","last_became_active":"2024-09-10T23:38:19.400430+0000","last_became_peered":"2024-09-10T23:38:19.400430+0000","last_unstale":"2024-09-10T23:38:20.379864+0000","last_undegraded":"2024-09-10T23:38:20.379864+0000","last_fullsized":"2024-09-10T23:38:20.379864+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:10.243290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.255684+0000","last_change":"2024-09-10T23:38:19.376402+0000","last_active":"2024-09-10T23:38:21.255684+0000","last_peered":"2024-09-10T23:38:21.255684+0000","last_clean":"2024-09-10T23:38:21.255684+0000","last_became_active":"2024-09-10T23:38:19.374000+0000","last_became_peered":"2024-09-10T23:38:19.374000+0000","last_unstale":"2024-09-10T23:38:21.255684+0000","last_undegraded":"2024-09-10T23:38:21.255684+0000","last_fullsized":"2024-09-10T23:38:21.255684+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:19.392566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.751584+0000","last_change":"2024-09-10T23:38:19.381604+0000","last_active":"2024-09-10T23:38:20.751584+0000","last_peered":"2024-09-10T23:38:20.751584+0000","last_clean":"2024-09-10T23:38:20.751584+0000","last_became_active":"2024-09-10T23:38:19.377833+0000","last_became_peered":"2024-09-10T23:38:19.377833+0000","last_unstale":"2024-09-10T23:38:20.751584+0000","last_undegraded":"2024-09-10T23:38:20.751584+0000","last_fullsized":"2024-09-10T23:38:20.751584+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:12.536624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256789+0000","last_change":"2024-09-10T23:38:19.396434+0000","last_active":"2024-09-10T23:38:21.256789+0000","last_peered":"2024-09-10T23:38:21.256789+0000","last_clean":"2024-09-10T23:38:21.256789+0000","last_became_active":"2024-09-10T23:38:19.396157+0000","last_became_peered":"2024-09-10T23:38:19.396157+0000","last_unstale":"2024-09-10T23:38:21.256789+0000","last_undegraded":"2024-09-10T23:38:21.256789+0000","last_fullsized":"2024-09-10T23:38:21.256789+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:31.455382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380597+0000","last_change":"2024-09-10T23:38:19.389694+0000","last_active":"2024-09-10T23:38:20.380597+0000","last_peered":"2024-09-10T23:38:20.380597+0000","last_clean":"2024-09-10T23:38:20.380597+0000","last_became_active":"2024-09-10T23:38:19.388300+0000","last_became_peered":"2024-09-10T23:38:19.388300+0000","last_unstale":"2024-09-10T23:38:20.380597+0000","last_undegraded":"2024-09-10T23:38:20.380597+0000","last_fullsized":"2024-09-10T23:38:20.380597+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:38.434030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.381456+0000","last_change":"2024-09-10T23:38:19.377376+0000","last_active":"2024-09-10T23:38:20.381456+0000","last_peered":"2024-09-10T23:38:20.381456+0000","last_clean":"2024-09-10T23:38:20.381456+0000","last_became_active":"2024-09-10T23:38:19.376063+0000","last_became_peered":"2024-09-10T23:38:19.376063+0000","last_unstale":"2024-09-10T23:38:20.381456+0000","last_undegraded":"2024-09-10T23:38:20.381456+0000","last_fullsized":"2024-09-10T23:38:20.381456+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:11.943937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256111+0000","last_change":"2024-09-10T23:38:19.379464+0000","last_active":"2024-09-10T23:38:21.256111+0000","last_peered":"2024-09-10T23:38:21.256111+0000","last_clean":"2024-09-10T23:38:21.256111+0000","last_became_active":"2024-09-10T23:38:19.379132+0000","last_became_peered":"2024-09-10T23:38:19.379132+0000","last_unstale":"2024-09-10T23:38:21.256111+0000","last_undegraded":"2024-09-10T23:38:21.256111+0000","last_fullsized":"2024-09-10T23:38:21.256111+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:07.671893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380181+0000","last_change":"2024-09-10T23:38:19.378864+0000","last_active":"2024-09-10T23:38:20.380181+0000","last_peered":"2024-09-10T23:38:20.380181+0000","last_clean":"2024-09-10T23:38:20.380181+0000","last_became_active":"2024-09-10T23:38:19.378570+0000","last_became_peered":"2024-09-10T23:38:19.378570+0000","last_unstale":"2024-09-10T23:38:20.380181+0000","last_undegraded":"2024-09-10T23:38:20.380181+0000","last_fullsized":"2024-09-10T23:38:20.380181+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:27.379766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379959+0000","last_change":"2024-09-10T23:38:19.385373+0000","last_active":"2024-09-10T23:38:20.379959+0000","last_peered":"2024-09-10T23:38:20.379959+0000","last_clean":"2024-09-10T23:38:20.379959+0000","last_became_active":"2024-09-10T23:38:19.382817+0000","last_became_peered":"2024-09-10T23:38:19.382817+0000","last_unstale":"2024-09-10T23:38:20.379959+0000","last_undegraded":"2024-09-10T23:38:20.379959+0000","last_fullsized":"2024-09-10T23:38:20.379959+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:18.863734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.255870+0000","last_change":"2024-09-10T23:38:19.377065+0000","last_active":"2024-09-10T23:38:21.255870+0000","last_peered":"2024-09-10T23:38:21.255870+0000","last_clean":"2024-09-10T23:38:21.255870+0000","last_became_active":"2024-09-10T23:38:19.376797+0000","last_became_peered":"2024-09-10T23:38:19.376797+0000","last_unstale":"2024-09-10T23:38:21.255870+0000","last_undegraded":"2024-09-10T23:38:21.255870+0000","last_fullsized":"2024-09-10T23:38:21.255870+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:42.344220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.255848+0000","last_change":"2024-09-10T23:38:19.378344+0000","last_active":"2024-09-10T23:38:21.255848+0000","last_peered":"2024-09-10T23:38:21.255848+0000","last_clean":"2024-09-10T23:38:21.255848+0000","last_became_active":"2024-09-10T23:38:19.378069+0000","last_became_peered":"2024-09-10T23:38:19.378069+0000","last_unstale":"2024-09-10T23:38:21.255848+0000","last_undegraded":"2024-09-10T23:38:21.255848+0000","last_fullsized":"2024-09-10T23:38:21.255848+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:03.898897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.381524+0000","last_change":"2024-09-10T23:38:19.370192+0000","last_active":"2024-09-10T23:38:20.381524+0000","last_peered":"2024-09-10T23:38:20.381524+0000","last_clean":"2024-09-10T23:38:20.381524+0000","last_became_active":"2024-09-10T23:38:19.369891+0000","last_became_peered":"2024-09-10T23:38:19.369891+0000","last_unstale":"2024-09-10T23:38:20.381524+0000","last_undegraded":"2024-09-10T23:38:20.381524+0000","last_fullsized":"2024-09-10T23:38:20.381524+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:57.277190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"113'4","reported_seq":19,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.378307+0000","last_change":"2024-09-10T23:38:18.375480+0000","last_active":"2024-09-10T23:38:20.378307+0000","last_peered":"2024-09-10T23:38:20.378307+0000","last_clean":"2024-09-10T23:38:20.378307+0000","last_became_active":"2024-09-10T23:38:18.374761+0000","last_became_peered":"2024-09-10T23:38:18.374761+0000","last_unstale":"2024-09-10T23:38:20.378307+0000","last_undegraded":"2024-09-10T23:38:20.378307+0000","last_fullsized":"2024-09-10T23:38:20.378307+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:41.337331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256528+0000","last_change":"2024-09-10T23:38:19.399680+0000","last_active":"2024-09-10T23:38:21.256528+0000","last_peered":"2024-09-10T23:38:21.256528+0000","last_clean":"2024-09-10T23:38:21.256528+0000","last_became_active":"2024-09-10T23:38:19.399424+0000","last_became_peered":"2024-09-10T23:38:19.399424+0000","last_unstale":"2024-09-10T23:38:21.256528+0000","last_undegraded":"2024-09-10T23:38:21.256528+0000","last_fullsized":"2024-09-10T23:38:21.256528+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:11.160958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256347+0000","last_change":"2024-09-10T23:38:18.373280+0000","last_active":"2024-09-10T23:38:21.256347+0000","last_peered":"2024-09-10T23:38:21.256347+0000","last_clean":"2024-09-10T23:38:21.256347+0000","last_became_active":"2024-09-10T23:38:18.372998+0000","last_became_peered":"2024-09-10T23:38:18.372998+0000","last_unstale":"2024-09-10T23:38:21.256347+0000","last_undegraded":"2024-09-10T23:38:21.256347+0000","last_fullsized":"2024-09-10T23:38:21.256347+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:30.949271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256479+0000","last_change":"2024-09-10T23:38:19.373062+0000","last_active":"2024-09-10T23:38:21.256479+0000","last_peered":"2024-09-10T23:38:21.256479+0000","last_clean":"2024-09-10T23:38:21.256479+0000","last_became_active":"2024-09-10T23:38:19.372815+0000","last_became_peered":"2024-09-10T23:38:19.372815+0000","last_unstale":"2024-09-10T23:38:21.256479+0000","last_undegraded":"2024-09-10T23:38:21.256479+0000","last_fullsized":"2024-09-10T23:38:21.256479+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:10.566874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256633+0000","last_change":"2024-09-10T23:38:18.373866+0000","last_active":"2024-09-10T23:38:21.256633+0000","last_peered":"2024-09-10T23:38:21.256633+0000","last_clean":"2024-09-10T23:38:21.256633+0000","last_became_active":"2024-09-10T23:38:18.373476+0000","last_became_peered":"2024-09-10T23:38:18.373476+0000","last_unstale":"2024-09-10T23:38:21.256633+0000","last_undegraded":"2024-09-10T23:38:21.256633+0000","last_fullsized":"2024-09-10T23:38:21.256633+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:51:35.159965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.385004+0000","last_change":"2024-09-10T23:38:19.379954+0000","last_active":"2024-09-10T23:38:20.385004+0000","last_peered":"2024-09-10T23:38:20.385004+0000","last_clean":"2024-09-10T23:38:20.385004+0000","last_became_active":"2024-09-10T23:38:19.379611+0000","last_became_peered":"2024-09-10T23:38:19.379611+0000","last_unstale":"2024-09-10T23:38:20.385004+0000","last_undegraded":"2024-09-10T23:38:20.385004+0000","last_fullsized":"2024-09-10T23:38:20.385004+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:31.455382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379832+0000","last_change":"2024-09-10T23:38:18.383056+0000","last_active":"2024-09-10T23:38:20.379832+0000","last_peered":"2024-09-10T23:38:20.379832+0000","last_clean":"2024-09-10T23:38:20.379832+0000","last_became_active":"2024-09-10T23:38:18.382477+0000","last_became_peered":"2024-09-10T23:38:18.382477+0000","last_unstale":"2024-09-10T23:38:20.379832+0000","last_undegraded":"2024-09-10T23:38:20.379832+0000","last_fullsized":"2024-09-10T23:38:20.379832+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:07.751630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.751800+0000","last_change":"2024-09-10T23:38:19.385468+0000","last_active":"2024-09-10T23:38:20.751800+0000","last_peered":"2024-09-10T23:38:20.751800+0000","last_clean":"2024-09-10T23:38:20.751800+0000","last_became_active":"2024-09-10T23:38:19.384288+0000","last_became_peered":"2024-09-10T23:38:19.384288+0000","last_unstale":"2024-09-10T23:38:20.751800+0000","last_undegraded":"2024-09-10T23:38:20.751800+0000","last_fullsized":"2024-09-10T23:38:20.751800+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:30.446896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380214+0000","last_change":"2024-09-10T23:38:18.385147+0000","last_active":"2024-09-10T23:38:20.380214+0000","last_peered":"2024-09-10T23:38:20.380214+0000","last_clean":"2024-09-10T23:38:20.380214+0000","last_became_active":"2024-09-10T23:38:18.384626+0000","last_became_peered":"2024-09-10T23:38:18.384626+0000","last_unstale":"2024-09-10T23:38:20.380214+0000","last_undegraded":"2024-09-10T23:38:20.380214+0000","last_fullsized":"2024-09-10T23:38:20.380214+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:22.829046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384281+0000","last_change":"2024-09-10T23:38:19.379900+0000","last_active":"2024-09-10T23:38:20.384281+0000","last_peered":"2024-09-10T23:38:20.384281+0000","last_clean":"2024-09-10T23:38:20.384281+0000","last_became_active":"2024-09-10T23:38:19.379405+0000","last_became_peered":"2024-09-10T23:38:19.379405+0000","last_unstale":"2024-09-10T23:38:20.384281+0000","last_undegraded":"2024-09-10T23:38:20.384281+0000","last_fullsized":"2024-09-10T23:38:20.384281+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:14.287193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.751807+0000","last_change":"2024-09-10T23:38:18.385106+0000","last_active":"2024-09-10T23:38:20.751807+0000","last_peered":"2024-09-10T23:38:20.751807+0000","last_clean":"2024-09-10T23:38:20.751807+0000","last_became_active":"2024-09-10T23:38:18.384811+0000","last_became_peered":"2024-09-10T23:38:18.384811+0000","last_unstale":"2024-09-10T23:38:20.751807+0000","last_undegraded":"2024-09-10T23:38:20.751807+0000","last_fullsized":"2024-09-10T23:38:20.751807+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:33.755052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.383971+0000","last_change":"2024-09-10T23:38:19.382660+0000","last_active":"2024-09-10T23:38:20.383971+0000","last_peered":"2024-09-10T23:38:20.383971+0000","last_clean":"2024-09-10T23:38:20.383971+0000","last_became_active":"2024-09-10T23:38:19.382216+0000","last_became_peered":"2024-09-10T23:38:19.382216+0000","last_unstale":"2024-09-10T23:38:20.383971+0000","last_undegraded":"2024-09-10T23:38:20.383971+0000","last_fullsized":"2024-09-10T23:38:20.383971+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:39.902596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.255572+0000","last_change":"2024-09-10T23:38:18.371252+0000","last_active":"2024-09-10T23:38:21.255572+0000","last_peered":"2024-09-10T23:38:21.255572+0000","last_clean":"2024-09-10T23:38:21.255572+0000","last_became_active":"2024-09-10T23:38:18.370978+0000","last_became_peered":"2024-09-10T23:38:18.370978+0000","last_unstale":"2024-09-10T23:38:21.255572+0000","last_undegraded":"2024-09-10T23:38:21.255572+0000","last_fullsized":"2024-09-10T23:38:21.255572+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:32.543277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379639+0000","last_change":"2024-09-10T23:38:19.398008+0000","last_active":"2024-09-10T23:38:20.379639+0000","last_peered":"2024-09-10T23:38:20.379639+0000","last_clean":"2024-09-10T23:38:20.379639+0000","last_became_active":"2024-09-10T23:38:19.396607+0000","last_became_peered":"2024-09-10T23:38:19.396607+0000","last_unstale":"2024-09-10T23:38:20.379639+0000","last_undegraded":"2024-09-10T23:38:20.379639+0000","last_fullsized":"2024-09-10T23:38:20.379639+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:40.863447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379493+0000","last_change":"2024-09-10T23:38:18.383288+0000","last_active":"2024-09-10T23:38:20.379493+0000","last_peered":"2024-09-10T23:38:20.379493+0000","last_clean":"2024-09-10T23:38:20.379493+0000","last_became_active":"2024-09-10T23:38:18.382239+0000","last_became_peered":"2024-09-10T23:38:18.382239+0000","last_unstale":"2024-09-10T23:38:20.379493+0000","last_undegraded":"2024-09-10T23:38:20.379493+0000","last_fullsized":"2024-09-10T23:38:20.379493+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:58.216892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380982+0000","last_change":"2024-09-10T23:38:19.397564+0000","last_active":"2024-09-10T23:38:20.380982+0000","last_peered":"2024-09-10T23:38:20.380982+0000","last_clean":"2024-09-10T23:38:20.380982+0000","last_became_active":"2024-09-10T23:38:19.397177+0000","last_became_peered":"2024-09-10T23:38:19.397177+0000","last_unstale":"2024-09-10T23:38:20.380982+0000","last_undegraded":"2024-09-10T23:38:20.380982+0000","last_fullsized":"2024-09-10T23:38:20.380982+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:56.284299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384966+0000","last_change":"2024-09-10T23:38:18.379265+0000","last_active":"2024-09-10T23:38:20.384966+0000","last_peered":"2024-09-10T23:38:20.384966+0000","last_clean":"2024-09-10T23:38:20.384966+0000","last_became_active":"2024-09-10T23:38:18.378985+0000","last_became_peered":"2024-09-10T23:38:18.378985+0000","last_unstale":"2024-09-10T23:38:20.384966+0000","last_undegraded":"2024-09-10T23:38:20.384966+0000","last_fullsized":"2024-09-10T23:38:20.384966+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:14.170106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.378275+0000","last_change":"2024-09-10T23:38:19.377453+0000","last_active":"2024-09-10T23:38:20.378275+0000","last_peered":"2024-09-10T23:38:20.378275+0000","last_clean":"2024-09-10T23:38:20.378275+0000","last_became_active":"2024-09-10T23:38:19.377208+0000","last_became_peered":"2024-09-10T23:38:19.377208+0000","last_unstale":"2024-09-10T23:38:20.378275+0000","last_undegraded":"2024-09-10T23:38:20.378275+0000","last_fullsized":"2024-09-10T23:38:20.378275+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:15.176995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.751806+0000","last_change":"2024-09-10T23:38:18.379055+0000","last_active":"2024-09-10T23:38:20.751806+0000","last_peered":"2024-09-10T23:38:20.751806+0000","last_clean":"2024-09-10T23:38:20.751806+0000","last_became_active":"2024-09-10T23:38:18.378628+0000","last_became_peered":"2024-09-10T23:38:18.378628+0000","last_unstale":"2024-09-10T23:38:20.751806+0000","last_undegraded":"2024-09-10T23:38:20.751806+0000","last_fullsized":"2024-09-10T23:38:20.751806+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:00.102925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384796+0000","last_change":"2024-09-10T23:38:19.376562+0000","last_active":"2024-09-10T23:38:20.384796+0000","last_peered":"2024-09-10T23:38:20.384796+0000","last_clean":"2024-09-10T23:38:20.384796+0000","last_became_active":"2024-09-10T23:38:19.376164+0000","last_became_peered":"2024-09-10T23:38:19.376164+0000","last_unstale":"2024-09-10T23:38:20.384796+0000","last_undegraded":"2024-09-10T23:38:20.384796+0000","last_fullsized":"2024-09-10T23:38:20.384796+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:42.847739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"113'3","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.377996+0000","last_change":"2024-09-10T23:38:18.372216+0000","last_active":"2024-09-10T23:38:20.377996+0000","last_peered":"2024-09-10T23:38:20.377996+0000","last_clean":"2024-09-10T23:38:20.377996+0000","last_became_active":"2024-09-10T23:38:18.371982+0000","last_became_peered":"2024-09-10T23:38:18.371982+0000","last_unstale":"2024-09-10T23:38:20.377996+0000","last_undegraded":"2024-09-10T23:38:20.377996+0000","last_fullsized":"2024-09-10T23:38:20.377996+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:46.042598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.255952+0000","last_change":"2024-09-10T23:38:19.381538+0000","last_active":"2024-09-10T23:38:21.255952+0000","last_peered":"2024-09-10T23:38:21.255952+0000","last_clean":"2024-09-10T23:38:21.255952+0000","last_became_active":"2024-09-10T23:38:19.381294+0000","last_became_peered":"2024-09-10T23:38:19.381294+0000","last_unstale":"2024-09-10T23:38:21.255952+0000","last_undegraded":"2024-09-10T23:38:21.255952+0000","last_fullsized":"2024-09-10T23:38:21.255952+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:40.004917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"113'6","reported_seq":21,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256012+0000","last_change":"2024-09-10T23:38:18.375147+0000","last_active":"2024-09-10T23:38:21.256012+0000","last_peered":"2024-09-10T23:38:21.256012+0000","last_clean":"2024-09-10T23:38:21.256012+0000","last_became_active":"2024-09-10T23:38:18.374845+0000","last_became_peered":"2024-09-10T23:38:18.374845+0000","last_unstale":"2024-09-10T23:38:21.256012+0000","last_undegraded":"2024-09-10T23:38:21.256012+0000","last_fullsized":"2024-09-10T23:38:21.256012+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:27.452591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.381152+0000","last_change":"2024-09-10T23:38:19.399437+0000","last_active":"2024-09-10T23:38:20.381152+0000","last_peered":"2024-09-10T23:38:20.381152+0000","last_clean":"2024-09-10T23:38:20.381152+0000","last_became_active":"2024-09-10T23:38:19.398838+0000","last_became_peered":"2024-09-10T23:38:19.398838+0000","last_unstale":"2024-09-10T23:38:20.381152+0000","last_undegraded":"2024-09-10T23:38:20.381152+0000","last_fullsized":"2024-09-10T23:38:20.381152+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:49.239269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380770+0000","last_change":"2024-09-10T23:38:18.383148+0000","last_active":"2024-09-10T23:38:20.380770+0000","last_peered":"2024-09-10T23:38:20.380770+0000","last_clean":"2024-09-10T23:38:20.380770+0000","last_became_active":"2024-09-10T23:38:18.382525+0000","last_became_peered":"2024-09-10T23:38:18.382525+0000","last_unstale":"2024-09-10T23:38:20.380770+0000","last_undegraded":"2024-09-10T23:38:20.380770+0000","last_fullsized":"2024-09-10T23:38:20.380770+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:12.476365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380337+0000","last_change":"2024-09-10T23:38:19.370305+0000","last_active":"2024-09-10T23:38:20.380337+0000","last_peered":"2024-09-10T23:38:20.380337+0000","last_clean":"2024-09-10T23:38:20.380337+0000","last_became_active":"2024-09-10T23:38:19.369980+0000","last_became_peered":"2024-09-10T23:38:19.369980+0000","last_unstale":"2024-09-10T23:38:20.380337+0000","last_undegraded":"2024-09-10T23:38:20.380337+0000","last_fullsized":"2024-09-10T23:38:20.380337+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:53.229557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256038+0000","last_change":"2024-09-10T23:38:18.380342+0000","last_active":"2024-09-10T23:38:21.256038+0000","last_peered":"2024-09-10T23:38:21.256038+0000","last_clean":"2024-09-10T23:38:21.256038+0000","last_became_active":"2024-09-10T23:38:18.380064+0000","last_became_peered":"2024-09-10T23:38:18.380064+0000","last_unstale":"2024-09-10T23:38:21.256038+0000","last_undegraded":"2024-09-10T23:38:21.256038+0000","last_fullsized":"2024-09-10T23:38:21.256038+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:38.895707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384069+0000","last_change":"2024-09-10T23:38:19.375606+0000","last_active":"2024-09-10T23:38:20.384069+0000","last_peered":"2024-09-10T23:38:20.384069+0000","last_clean":"2024-09-10T23:38:20.384069+0000","last_became_active":"2024-09-10T23:38:19.375346+0000","last_became_peered":"2024-09-10T23:38:19.375346+0000","last_unstale":"2024-09-10T23:38:20.384069+0000","last_undegraded":"2024-09-10T23:38:20.384069+0000","last_fullsized":"2024-09-10T23:38:20.384069+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:20.050463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"113'8","reported_seq":23,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.383874+0000","last_change":"2024-09-10T23:38:18.380372+0000","last_active":"2024-09-10T23:38:20.383874+0000","last_peered":"2024-09-10T23:38:20.383874+0000","last_clean":"2024-09-10T23:38:20.383874+0000","last_became_active":"2024-09-10T23:38:18.380070+0000","last_became_peered":"2024-09-10T23:38:18.380070+0000","last_unstale":"2024-09-10T23:38:20.383874+0000","last_undegraded":"2024-09-10T23:38:20.383874+0000","last_fullsized":"2024-09-10T23:38:20.383874+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:37.192323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.378114+0000","last_change":"2024-09-10T23:38:19.378545+0000","last_active":"2024-09-10T23:38:20.378114+0000","last_peered":"2024-09-10T23:38:20.378114+0000","last_clean":"2024-09-10T23:38:20.378114+0000","last_became_active":"2024-09-10T23:38:19.378297+0000","last_became_peered":"2024-09-10T23:38:19.378297+0000","last_unstale":"2024-09-10T23:38:20.378114+0000","last_undegraded":"2024-09-10T23:38:20.378114+0000","last_fullsized":"2024-09-10T23:38:20.378114+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:22.259216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.385011+0000","last_change":"2024-09-10T23:38:18.377800+0000","last_active":"2024-09-10T23:38:20.385011+0000","last_peered":"2024-09-10T23:38:20.385011+0000","last_clean":"2024-09-10T23:38:20.385011+0000","last_became_active":"2024-09-10T23:38:18.377551+0000","last_became_peered":"2024-09-10T23:38:18.377551+0000","last_unstale":"2024-09-10T23:38:20.385011+0000","last_undegraded":"2024-09-10T23:38:20.385011+0000","last_fullsized":"2024-09-10T23:38:20.385011+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:47.941526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.378294+0000","last_change":"2024-09-10T23:38:19.397958+0000","last_active":"2024-09-10T23:38:20.378294+0000","last_peered":"2024-09-10T23:38:20.378294+0000","last_clean":"2024-09-10T23:38:20.378294+0000","last_became_active":"2024-09-10T23:38:19.397490+0000","last_became_peered":"2024-09-10T23:38:19.397490+0000","last_unstale":"2024-09-10T23:38:20.378294+0000","last_undegraded":"2024-09-10T23:38:20.378294+0000","last_fullsized":"2024-09-10T23:38:20.378294+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:48.948415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256376+0000","last_change":"2024-09-10T23:38:18.377068+0000","last_active":"2024-09-10T23:38:21.256376+0000","last_peered":"2024-09-10T23:38:21.256376+0000","last_clean":"2024-09-10T23:38:21.256376+0000","last_became_active":"2024-09-10T23:38:18.376777+0000","last_became_peered":"2024-09-10T23:38:18.376777+0000","last_unstale":"2024-09-10T23:38:21.256376+0000","last_undegraded":"2024-09-10T23:38:21.256376+0000","last_fullsized":"2024-09-10T23:38:21.256376+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:37.295397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380191+0000","last_change":"2024-09-10T23:38:19.386807+0000","last_active":"2024-09-10T23:38:20.380191+0000","last_peered":"2024-09-10T23:38:20.380191+0000","last_clean":"2024-09-10T23:38:20.380191+0000","last_became_active":"2024-09-10T23:38:19.386534+0000","last_became_peered":"2024-09-10T23:38:19.386534+0000","last_unstale":"2024-09-10T23:38:20.380191+0000","last_undegraded":"2024-09-10T23:38:20.380191+0000","last_fullsized":"2024-09-10T23:38:20.380191+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:57.494220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256347+0000","last_change":"2024-09-10T23:38:18.377255+0000","last_active":"2024-09-10T23:38:21.256347+0000","last_peered":"2024-09-10T23:38:21.256347+0000","last_clean":"2024-09-10T23:38:21.256347+0000","last_became_active":"2024-09-10T23:38:18.376970+0000","last_became_peered":"2024-09-10T23:38:18.376970+0000","last_unstale":"2024-09-10T23:38:21.256347+0000","last_undegraded":"2024-09-10T23:38:21.256347+0000","last_fullsized":"2024-09-10T23:38:21.256347+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:43.974712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380682+0000","last_change":"2024-09-10T23:38:19.399728+0000","last_active":"2024-09-10T23:38:20.380682+0000","last_peered":"2024-09-10T23:38:20.380682+0000","last_clean":"2024-09-10T23:38:20.380682+0000","last_became_active":"2024-09-10T23:38:19.399447+0000","last_became_peered":"2024-09-10T23:38:19.399447+0000","last_unstale":"2024-09-10T23:38:20.380682+0000","last_undegraded":"2024-09-10T23:38:20.380682+0000","last_fullsized":"2024-09-10T23:38:20.380682+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:58.972436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.377958+0000","last_change":"2024-09-10T23:38:18.383424+0000","last_active":"2024-09-10T23:38:20.377958+0000","last_peered":"2024-09-10T23:38:20.377958+0000","last_clean":"2024-09-10T23:38:20.377958+0000","last_became_active":"2024-09-10T23:38:18.383034+0000","last_became_peered":"2024-09-10T23:38:18.383034+0000","last_unstale":"2024-09-10T23:38:20.377958+0000","last_undegraded":"2024-09-10T23:38:20.377958+0000","last_fullsized":"2024-09-10T23:38:20.377958+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:17:20.177415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380104+0000","last_change":"2024-09-10T23:38:19.385679+0000","last_active":"2024-09-10T23:38:20.380104+0000","last_peered":"2024-09-10T23:38:20.380104+0000","last_clean":"2024-09-10T23:38:20.380104+0000","last_became_active":"2024-09-10T23:38:19.384976+0000","last_became_peered":"2024-09-10T23:38:19.384976+0000","last_unstale":"2024-09-10T23:38:20.380104+0000","last_undegraded":"2024-09-10T23:38:20.380104+0000","last_fullsized":"2024-09-10T23:38:20.380104+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:24.440181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256203+0000","last_change":"2024-09-10T23:38:18.376656+0000","last_active":"2024-09-10T23:38:21.256203+0000","last_peered":"2024-09-10T23:38:21.256203+0000","last_clean":"2024-09-10T23:38:21.256203+0000","last_became_active":"2024-09-10T23:38:18.376404+0000","last_became_peered":"2024-09-10T23:38:18.376404+0000","last_unstale":"2024-09-10T23:38:21.256203+0000","last_undegraded":"2024-09-10T23:38:21.256203+0000","last_fullsized":"2024-09-10T23:38:21.256203+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:44.189752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.255744+0000","last_change":"2024-09-10T23:38:19.371004+0000","last_active":"2024-09-10T23:38:21.255744+0000","last_peered":"2024-09-10T23:38:21.255744+0000","last_clean":"2024-09-10T23:38:21.255744+0000","last_became_active":"2024-09-10T23:38:19.370691+0000","last_became_peered":"2024-09-10T23:38:19.370691+0000","last_unstale":"2024-09-10T23:38:21.255744+0000","last_undegraded":"2024-09-10T23:38:21.255744+0000","last_fullsized":"2024-09-10T23:38:21.255744+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:17:21.184304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.751904+0000","last_change":"2024-09-10T23:38:18.382296+0000","last_active":"2024-09-10T23:38:20.751904+0000","last_peered":"2024-09-10T23:38:20.751904+0000","last_clean":"2024-09-10T23:38:20.751904+0000","last_became_active":"2024-09-10T23:38:18.380974+0000","last_became_peered":"2024-09-10T23:38:18.380974+0000","last_unstale":"2024-09-10T23:38:20.751904+0000","last_undegraded":"2024-09-10T23:38:20.751904+0000","last_fullsized":"2024-09-10T23:38:20.751904+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:39.791907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379779+0000","last_change":"2024-09-10T23:38:19.374817+0000","last_active":"2024-09-10T23:38:20.379779+0000","last_peered":"2024-09-10T23:38:20.379779+0000","last_clean":"2024-09-10T23:38:20.379779+0000","last_became_active":"2024-09-10T23:38:19.374445+0000","last_became_peered":"2024-09-10T23:38:19.374445+0000","last_unstale":"2024-09-10T23:38:20.379779+0000","last_undegraded":"2024-09-10T23:38:20.379779+0000","last_fullsized":"2024-09-10T23:38:20.379779+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:23.336535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380151+0000","last_change":"2024-09-10T23:38:18.378009+0000","last_active":"2024-09-10T23:38:20.380151+0000","last_peered":"2024-09-10T23:38:20.380151+0000","last_clean":"2024-09-10T23:38:20.380151+0000","last_became_active":"2024-09-10T23:38:18.377249+0000","last_became_peered":"2024-09-10T23:38:18.377249+0000","last_unstale":"2024-09-10T23:38:20.380151+0000","last_undegraded":"2024-09-10T23:38:20.380151+0000","last_fullsized":"2024-09-10T23:38:20.380151+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:07.004388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384097+0000","last_change":"2024-09-10T23:38:19.377322+0000","last_active":"2024-09-10T23:38:20.384097+0000","last_peered":"2024-09-10T23:38:20.384097+0000","last_clean":"2024-09-10T23:38:20.384097+0000","last_became_active":"2024-09-10T23:38:19.377088+0000","last_became_peered":"2024-09-10T23:38:19.377088+0000","last_unstale":"2024-09-10T23:38:20.384097+0000","last_undegraded":"2024-09-10T23:38:20.384097+0000","last_fullsized":"2024-09-10T23:38:20.384097+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:04.775309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256026+0000","last_change":"2024-09-10T23:38:18.378023+0000","last_active":"2024-09-10T23:38:21.256026+0000","last_peered":"2024-09-10T23:38:21.256026+0000","last_clean":"2024-09-10T23:38:21.256026+0000","last_became_active":"2024-09-10T23:38:18.377774+0000","last_became_peered":"2024-09-10T23:38:18.377774+0000","last_unstale":"2024-09-10T23:38:21.256026+0000","last_undegraded":"2024-09-10T23:38:21.256026+0000","last_fullsized":"2024-09-10T23:38:21.256026+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:23.451248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256266+0000","last_change":"2024-09-10T23:38:19.377645+0000","last_active":"2024-09-10T23:38:21.256266+0000","last_peered":"2024-09-10T23:38:21.256266+0000","last_clean":"2024-09-10T23:38:21.256266+0000","last_became_active":"2024-09-10T23:38:19.377334+0000","last_became_peered":"2024-09-10T23:38:19.377334+0000","last_unstale":"2024-09-10T23:38:21.256266+0000","last_undegraded":"2024-09-10T23:38:21.256266+0000","last_fullsized":"2024-09-10T23:38:21.256266+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:28.697385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.751992+0000","last_change":"2024-09-10T23:38:18.381848+0000","last_active":"2024-09-10T23:38:20.751992+0000","last_peered":"2024-09-10T23:38:20.751992+0000","last_clean":"2024-09-10T23:38:20.751992+0000","last_became_active":"2024-09-10T23:38:18.380901+0000","last_became_peered":"2024-09-10T23:38:18.380901+0000","last_unstale":"2024-09-10T23:38:20.751992+0000","last_undegraded":"2024-09-10T23:38:20.751992+0000","last_fullsized":"2024-09-10T23:38:20.751992+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:17:21.047034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380393+0000","last_change":"2024-09-10T23:38:19.385080+0000","last_active":"2024-09-10T23:38:20.380393+0000","last_peered":"2024-09-10T23:38:20.380393+0000","last_clean":"2024-09-10T23:38:20.380393+0000","last_became_active":"2024-09-10T23:38:19.384810+0000","last_became_peered":"2024-09-10T23:38:19.384810+0000","last_unstale":"2024-09-10T23:38:20.380393+0000","last_undegraded":"2024-09-10T23:38:20.380393+0000","last_fullsized":"2024-09-10T23:38:20.380393+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:38.036081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"114'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.378069+0000","last_change":"2024-09-10T23:38:18.372313+0000","last_active":"2024-09-10T23:38:20.378069+0000","last_peered":"2024-09-10T23:38:20.378069+0000","last_clean":"2024-09-10T23:38:20.378069+0000","last_became_active":"2024-09-10T23:38:18.371745+0000","last_became_peered":"2024-09-10T23:38:18.371745+0000","last_unstale":"2024-09-10T23:38:20.378069+0000","last_undegraded":"2024-09-10T23:38:20.378069+0000","last_fullsized":"2024-09-10T23:38:20.378069+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:18.385677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.381200+0000","last_change":"2024-09-10T23:38:19.375455+0000","last_active":"2024-09-10T23:38:20.381200+0000","last_peered":"2024-09-10T23:38:20.381200+0000","last_clean":"2024-09-10T23:38:20.381200+0000","last_became_active":"2024-09-10T23:38:19.374472+0000","last_became_peered":"2024-09-10T23:38:19.374472+0000","last_unstale":"2024-09-10T23:38:20.381200+0000","last_undegraded":"2024-09-10T23:38:20.381200+0000","last_fullsized":"2024-09-10T23:38:20.381200+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:38.307389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.752052+0000","last_change":"2024-09-10T23:38:18.375834+0000","last_active":"2024-09-10T23:38:20.752052+0000","last_peered":"2024-09-10T23:38:20.752052+0000","last_clean":"2024-09-10T23:38:20.752052+0000","last_became_active":"2024-09-10T23:38:18.375538+0000","last_became_peered":"2024-09-10T23:38:18.375538+0000","last_unstale":"2024-09-10T23:38:20.752052+0000","last_undegraded":"2024-09-10T23:38:20.752052+0000","last_fullsized":"2024-09-10T23:38:20.752052+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:45.172467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256454+0000","last_change":"2024-09-10T23:38:19.379972+0000","last_active":"2024-09-10T23:38:21.256454+0000","last_peered":"2024-09-10T23:38:21.256454+0000","last_clean":"2024-09-10T23:38:21.256454+0000","last_became_active":"2024-09-10T23:38:19.379418+0000","last_became_peered":"2024-09-10T23:38:19.379418+0000","last_unstale":"2024-09-10T23:38:21.256454+0000","last_undegraded":"2024-09-10T23:38:21.256454+0000","last_fullsized":"2024-09-10T23:38:21.256454+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:31.392621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.381095+0000","last_change":"2024-09-10T23:38:18.381436+0000","last_active":"2024-09-10T23:38:20.381095+0000","last_peered":"2024-09-10T23:38:20.381095+0000","last_clean":"2024-09-10T23:38:20.381095+0000","last_became_active":"2024-09-10T23:38:18.381134+0000","last_became_peered":"2024-09-10T23:38:18.381134+0000","last_unstale":"2024-09-10T23:38:20.381095+0000","last_undegraded":"2024-09-10T23:38:20.381095+0000","last_fullsized":"2024-09-10T23:38:20.381095+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:54.849657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384572+0000","last_change":"2024-09-10T23:38:19.376422+0000","last_active":"2024-09-10T23:38:20.384572+0000","last_peered":"2024-09-10T23:38:20.384572+0000","last_clean":"2024-09-10T23:38:20.384572+0000","last_became_active":"2024-09-10T23:38:19.376002+0000","last_became_peered":"2024-09-10T23:38:19.376002+0000","last_unstale":"2024-09-10T23:38:20.384572+0000","last_undegraded":"2024-09-10T23:38:20.384572+0000","last_fullsized":"2024-09-10T23:38:20.384572+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:56.434591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380260+0000","last_change":"2024-09-10T23:38:18.383845+0000","last_active":"2024-09-10T23:38:20.380260+0000","last_peered":"2024-09-10T23:38:20.380260+0000","last_clean":"2024-09-10T23:38:20.380260+0000","last_became_active":"2024-09-10T23:38:18.383430+0000","last_became_peered":"2024-09-10T23:38:18.383430+0000","last_unstale":"2024-09-10T23:38:20.380260+0000","last_undegraded":"2024-09-10T23:38:20.380260+0000","last_fullsized":"2024-09-10T23:38:20.380260+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:54.546403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384451+0000","last_change":"2024-09-10T23:38:19.399489+0000","last_active":"2024-09-10T23:38:20.384451+0000","last_peered":"2024-09-10T23:38:20.384451+0000","last_clean":"2024-09-10T23:38:20.384451+0000","last_became_active":"2024-09-10T23:38:19.399219+0000","last_became_peered":"2024-09-10T23:38:19.399219+0000","last_unstale":"2024-09-10T23:38:20.384451+0000","last_undegraded":"2024-09-10T23:38:20.384451+0000","last_fullsized":"2024-09-10T23:38:20.384451+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:38.886159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384116+0000","last_change":"2024-09-10T23:38:18.371469+0000","last_active":"2024-09-10T23:38:20.384116+0000","last_peered":"2024-09-10T23:38:20.384116+0000","last_clean":"2024-09-10T23:38:20.384116+0000","last_became_active":"2024-09-10T23:38:18.371213+0000","last_became_peered":"2024-09-10T23:38:18.371213+0000","last_unstale":"2024-09-10T23:38:20.384116+0000","last_undegraded":"2024-09-10T23:38:20.384116+0000","last_fullsized":"2024-09-10T23:38:20.384116+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:22.864876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.751921+0000","last_change":"2024-09-10T23:38:19.390957+0000","last_active":"2024-09-10T23:38:20.751921+0000","last_peered":"2024-09-10T23:38:20.751921+0000","last_clean":"2024-09-10T23:38:20.751921+0000","last_became_active":"2024-09-10T23:38:19.390694+0000","last_became_peered":"2024-09-10T23:38:19.390694+0000","last_unstale":"2024-09-10T23:38:20.751921+0000","last_undegraded":"2024-09-10T23:38:20.751921+0000","last_fullsized":"2024-09-10T23:38:20.751921+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:55.859293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256147+0000","last_change":"2024-09-10T23:38:18.376639+0000","last_active":"2024-09-10T23:38:21.256147+0000","last_peered":"2024-09-10T23:38:21.256147+0000","last_clean":"2024-09-10T23:38:21.256147+0000","last_became_active":"2024-09-10T23:38:18.376337+0000","last_became_peered":"2024-09-10T23:38:18.376337+0000","last_unstale":"2024-09-10T23:38:21.256147+0000","last_undegraded":"2024-09-10T23:38:21.256147+0000","last_fullsized":"2024-09-10T23:38:21.256147+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:19.043574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380665+0000","last_change":"2024-09-10T23:38:19.373475+0000","last_active":"2024-09-10T23:38:20.380665+0000","last_peered":"2024-09-10T23:38:20.380665+0000","last_clean":"2024-09-10T23:38:20.380665+0000","last_became_active":"2024-09-10T23:38:19.373205+0000","last_became_peered":"2024-09-10T23:38:19.373205+0000","last_unstale":"2024-09-10T23:38:20.380665+0000","last_undegraded":"2024-09-10T23:38:20.380665+0000","last_fullsized":"2024-09-10T23:38:20.380665+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:49.934280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379847+0000","last_change":"2024-09-10T23:38:18.382530+0000","last_active":"2024-09-10T23:38:20.379847+0000","last_peered":"2024-09-10T23:38:20.379847+0000","last_clean":"2024-09-10T23:38:20.379847+0000","last_became_active":"2024-09-10T23:38:18.381654+0000","last_became_peered":"2024-09-10T23:38:18.381654+0000","last_unstale":"2024-09-10T23:38:20.379847+0000","last_undegraded":"2024-09-10T23:38:20.379847+0000","last_fullsized":"2024-09-10T23:38:20.379847+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:59.553134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256016+0000","last_change":"2024-09-10T23:38:19.396149+0000","last_active":"2024-09-10T23:38:21.256016+0000","last_peered":"2024-09-10T23:38:21.256016+0000","last_clean":"2024-09-10T23:38:21.256016+0000","last_became_active":"2024-09-10T23:38:19.395879+0000","last_became_peered":"2024-09-10T23:38:19.395879+0000","last_unstale":"2024-09-10T23:38:21.256016+0000","last_undegraded":"2024-09-10T23:38:21.256016+0000","last_fullsized":"2024-09-10T23:38:21.256016+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:48.172091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384404+0000","last_change":"2024-09-10T23:38:18.372768+0000","last_active":"2024-09-10T23:38:20.384404+0000","last_peered":"2024-09-10T23:38:20.384404+0000","last_clean":"2024-09-10T23:38:20.384404+0000","last_became_active":"2024-09-10T23:38:18.372508+0000","last_became_peered":"2024-09-10T23:38:18.372508+0000","last_unstale":"2024-09-10T23:38:20.384404+0000","last_undegraded":"2024-09-10T23:38:20.384404+0000","last_fullsized":"2024-09-10T23:38:20.384404+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:54.620298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":307,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.255383+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:38:21.255383+0000","last_peered":"2024-09-10T23:38:21.255383+0000","last_clean":"2024-09-10T23:38:21.255383+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:38:21.255383+0000","last_undegraded":"2024-09-10T23:38:21.255383+0000","last_fullsized":"2024-09-10T23:38:21.255383+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509516,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1421312,"data_stored":621433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542219,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1417216,"data_stored":621399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509515,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2023424,"data_stored":1212509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509513,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1417216,"data_stored":621399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509514,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1421312,"data_stored":621469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509515,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1425408,"data_stored":622071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509514,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2027520,"data_stored":1213401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509514,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1437696,"data_stored":623067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:38:23.490 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:38:23.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.665+0000 7f1b62400640 1 -- 172.21.15.94:0/106540036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b5c15cbc0 msgr2=0x7f1b5c0cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:23.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.665+0000 7f1b62400640 1 --2- 172.21.15.94:0/106540036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b5c15cbc0 0x7f1b5c0cde20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009840 tx=0x7f1b4401c820 comp rx=0 tx=0).stop 2024-09-10T23:38:23.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b62400640 1 -- 172.21.15.94:0/106540036 shutdown_connections 2024-09-10T23:38:23.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b62400640 1 --2- 172.21.15.94:0/106540036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b5c0cfd50 0x7f1b5c0d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b62400640 1 --2- 172.21.15.94:0/106540036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b5c15cbc0 0x7f1b5c0cde20 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b62400640 1 -- 172.21.15.94:0/106540036 >> 172.21.15.94:0/106540036 conn(0x7f1b5c06cdf0 msgr2=0x7f1b5c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:23.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b62400640 1 -- 172.21.15.94:0/106540036 shutdown_connections 2024-09-10T23:38:23.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b62400640 1 -- 172.21.15.94:0/106540036 wait complete. 2024-09-10T23:38:23.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b62400640 1 Processor -- start 2024-09-10T23:38:23.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b62400640 1 -- start start 2024-09-10T23:38:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b62400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b5c0cfd50 0x7f1b5c163550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b62400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b5c152c10 0x7f1b5c146c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b62400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b5c15cbc0 0x7f1b5c163db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b62400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c168c60 con 0x7f1b5c15cbc0 2024-09-10T23:38:23.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b62400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c168ae0 con 0x7f1b5c0cfd50 2024-09-10T23:38:23.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b62400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5c168de0 con 0x7f1b5c152c10 2024-09-10T23:38:23.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b5b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b5c0cfd50 0x7f1b5c163550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:23.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b5be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b5c15cbc0 0x7f1b5c163db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:23.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b5b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b5c0cfd50 0x7f1b5c163550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:40480/0 (socket says 172.21.15.94:40480) 2024-09-10T23:38:23.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b5b400640 1 -- 172.21.15.94:0/870979109 learned_addr learned my addr 172.21.15.94:0/870979109 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:23.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b52a00640 1 --2- 172.21.15.94:0/870979109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b5c152c10 0x7f1b5c146c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:23.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b5b400640 1 -- 172.21.15.94:0/870979109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b5c152c10 msgr2=0x7f1b5c146c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:23.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b5b400640 1 --2- 172.21.15.94:0/870979109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b5c152c10 0x7f1b5c146c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b5b400640 1 -- 172.21.15.94:0/870979109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b5c15cbc0 msgr2=0x7f1b5c163db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:23.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b5b400640 1 --2- 172.21.15.94:0/870979109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b5c15cbc0 0x7f1b5c163db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b5b400640 1 -- 172.21.15.94:0/870979109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5c06e660 con 0x7f1b5c0cfd50 2024-09-10T23:38:23.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.669+0000 7f1b5b400640 1 --2- 172.21.15.94:0/870979109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b5c0cfd50 0x7f1b5c163550 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00c8a0 tx=0x7f1b4c00cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.673+0000 7f1b53e00640 1 -- 172.21.15.94:0/870979109 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c003b80 con 0x7f1b5c0cfd50 2024-09-10T23:38:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.673+0000 7f1b53e00640 1 -- 172.21.15.94:0/870979109 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b4c003d40 con 0x7f1b5c0cfd50 2024-09-10T23:38:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.673+0000 7f1b53e00640 1 -- 172.21.15.94:0/870979109 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c00aa50 con 0x7f1b5c0cfd50 2024-09-10T23:38:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.673+0000 7f1b62400640 1 -- 172.21.15.94:0/870979109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5c149f30 con 0x7f1b5c0cfd50 2024-09-10T23:38:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.673+0000 7f1b62400640 1 -- 172.21.15.94:0/870979109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5c1fb8a0 con 0x7f1b5c0cfd50 2024-09-10T23:38:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.673+0000 7f1b62400640 1 -- 172.21.15.94:0/870979109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b5c0d0200 con 0x7f1b5c0cfd50 2024-09-10T23:38:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.673+0000 7f1b53e00640 1 -- 172.21.15.94:0/870979109 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f1b4c00abf0 con 0x7f1b5c0cfd50 2024-09-10T23:38:23.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.673+0000 7f1b53e00640 1 --2- 172.21.15.94:0/870979109 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1b38072a90 0x7f1b38074f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:23.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.673+0000 7f1b53e00640 1 -- 172.21.15.94:0/870979109 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f1b4c093cb0 con 0x7f1b5c0cfd50 2024-09-10T23:38:23.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.673+0000 7f1b52a00640 1 --2- 172.21.15.94:0/870979109 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1b38072a90 0x7f1b38074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:23.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.677+0000 7f1b52a00640 1 --2- 172.21.15.94:0/870979109 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1b38072a90 0x7f1b38074f50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1b440af000 tx=0x7f1b4401b060 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:23.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.677+0000 7f1b53e00640 1 -- 172.21.15.94:0/870979109 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1b4c05e6c0 con 0x7f1b5c0cfd50 2024-09-10T23:38:23.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.845+0000 7f1b62400640 1 -- 172.21.15.94:0/870979109 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1b5c0b46c0 con 0x7f1b38072a90 2024-09-10T23:38:23.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.849+0000 7f1b53e00640 1 -- 172.21.15.94:0/870979109 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252478 (secure 0 0 0) 0x7f1b5c0b46c0 con 0x7f1b38072a90 2024-09-10T23:38:23.854 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:23.855 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:38:23.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.853+0000 7f1b62400640 1 -- 172.21.15.94:0/870979109 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1b38072a90 msgr2=0x7f1b38074f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:23.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.853+0000 7f1b62400640 1 --2- 172.21.15.94:0/870979109 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1b38072a90 0x7f1b38074f50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1b440af000 tx=0x7f1b4401b060 comp rx=0 tx=0).stop 2024-09-10T23:38:23.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.853+0000 7f1b62400640 1 -- 172.21.15.94:0/870979109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b5c0cfd50 msgr2=0x7f1b5c163550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:23.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.853+0000 7f1b62400640 1 --2- 172.21.15.94:0/870979109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b5c0cfd50 0x7f1b5c163550 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00c8a0 tx=0x7f1b4c00cd60 comp rx=0 tx=0).stop 2024-09-10T23:38:23.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.853+0000 7f1b62400640 1 -- 172.21.15.94:0/870979109 shutdown_connections 2024-09-10T23:38:23.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.853+0000 7f1b62400640 1 --2- 172.21.15.94:0/870979109 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1b38072a90 0x7f1b38074f50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.853+0000 7f1b62400640 1 --2- 172.21.15.94:0/870979109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b5c15cbc0 0x7f1b5c163db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.853+0000 7f1b62400640 1 --2- 172.21.15.94:0/870979109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b5c152c10 0x7f1b5c146c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.853+0000 7f1b62400640 1 --2- 172.21.15.94:0/870979109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b5c0cfd50 0x7f1b5c163550 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:23.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.853+0000 7f1b62400640 1 -- 172.21.15.94:0/870979109 >> 172.21.15.94:0/870979109 conn(0x7f1b5c06cdf0 msgr2=0x7f1b5c06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:23.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.853+0000 7f1b62400640 1 -- 172.21.15.94:0/870979109 shutdown_connections 2024-09-10T23:38:23.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:23.853+0000 7f1b62400640 1 -- 172.21.15.94:0/870979109 wait complete. 2024-09-10T23:38:23.872 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-10T23:38:22.756898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261824,"kb_used_data":12296,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712896,"statfs":{"total":773094113280,"available":772826005504,"internally_reserved":0,"allocated":12591104,"data_stored":6156748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.409013"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379877+0000","last_change":"2024-09-10T23:38:19.391041+0000","last_active":"2024-09-10T23:38:20.379877+0000","last_peered":"2024-09-10T23:38:20.379877+0000","last_clean":"2024-09-10T23:38:20.379877+0000","last_became_active":"2024-09-10T23:38:19.390779+0000","last_became_peered":"2024-09-10T23:38:19.390779+0000","last_unstale":"2024-09-10T23:38:20.379877+0000","last_undegraded":"2024-09-10T23:38:20.379877+0000","last_fullsized":"2024-09-10T23:38:20.379877+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:35.192273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380328+0000","last_change":"2024-09-10T23:38:19.388129+0000","last_active":"2024-09-10T23:38:20.380328+0000","last_peered":"2024-09-10T23:38:20.380328+0000","last_clean":"2024-09-10T23:38:20.380328+0000","last_became_active":"2024-09-10T23:38:19.387780+0000","last_became_peered":"2024-09-10T23:38:19.387780+0000","last_unstale":"2024-09-10T23:38:20.380328+0000","last_undegraded":"2024-09-10T23:38:20.380328+0000","last_fullsized":"2024-09-10T23:38:20.380328+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:24.573075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256275+0000","last_change":"2024-09-10T23:38:19.380225+0000","last_active":"2024-09-10T23:38:21.256275+0000","last_peered":"2024-09-10T23:38:21.256275+0000","last_clean":"2024-09-10T23:38:21.256275+0000","last_became_active":"2024-09-10T23:38:19.379727+0000","last_became_peered":"2024-09-10T23:38:19.379727+0000","last_unstale":"2024-09-10T23:38:21.256275+0000","last_undegraded":"2024-09-10T23:38:21.256275+0000","last_fullsized":"2024-09-10T23:38:21.256275+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:11:57.180791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256491+0000","last_change":"2024-09-10T23:38:19.396970+0000","last_active":"2024-09-10T23:38:21.256491+0000","last_peered":"2024-09-10T23:38:21.256491+0000","last_clean":"2024-09-10T23:38:21.256491+0000","last_became_active":"2024-09-10T23:38:19.394882+0000","last_became_peered":"2024-09-10T23:38:19.394882+0000","last_unstale":"2024-09-10T23:38:21.256491+0000","last_undegraded":"2024-09-10T23:38:21.256491+0000","last_fullsized":"2024-09-10T23:38:21.256491+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:47.082303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.377999+0000","last_change":"2024-09-10T23:38:19.377197+0000","last_active":"2024-09-10T23:38:20.377999+0000","last_peered":"2024-09-10T23:38:20.377999+0000","last_clean":"2024-09-10T23:38:20.377999+0000","last_became_active":"2024-09-10T23:38:19.376899+0000","last_became_peered":"2024-09-10T23:38:19.376899+0000","last_unstale":"2024-09-10T23:38:20.377999+0000","last_undegraded":"2024-09-10T23:38:20.377999+0000","last_fullsized":"2024-09-10T23:38:20.377999+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:30.531015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384056+0000","last_change":"2024-09-10T23:38:19.381650+0000","last_active":"2024-09-10T23:38:20.384056+0000","last_peered":"2024-09-10T23:38:20.384056+0000","last_clean":"2024-09-10T23:38:20.384056+0000","last_became_active":"2024-09-10T23:38:19.381348+0000","last_became_peered":"2024-09-10T23:38:19.381348+0000","last_unstale":"2024-09-10T23:38:20.384056+0000","last_undegraded":"2024-09-10T23:38:20.384056+0000","last_fullsized":"2024-09-10T23:38:20.384056+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:38.302286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.377778+0000","last_change":"2024-09-10T23:38:19.369944+0000","last_active":"2024-09-10T23:38:20.377778+0000","last_peered":"2024-09-10T23:38:20.377778+0000","last_clean":"2024-09-10T23:38:20.377778+0000","last_became_active":"2024-09-10T23:38:19.369004+0000","last_became_peered":"2024-09-10T23:38:19.369004+0000","last_unstale":"2024-09-10T23:38:20.377778+0000","last_undegraded":"2024-09-10T23:38:20.377778+0000","last_fullsized":"2024-09-10T23:38:20.377778+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:59.038709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.377772+0000","last_change":"2024-09-10T23:38:19.380217+0000","last_active":"2024-09-10T23:38:20.377772+0000","last_peered":"2024-09-10T23:38:20.377772+0000","last_clean":"2024-09-10T23:38:20.377772+0000","last_became_active":"2024-09-10T23:38:19.379970+0000","last_became_peered":"2024-09-10T23:38:19.379970+0000","last_unstale":"2024-09-10T23:38:20.377772+0000","last_undegraded":"2024-09-10T23:38:20.377772+0000","last_fullsized":"2024-09-10T23:38:20.377772+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:49.402756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256093+0000","last_change":"2024-09-10T23:38:19.381019+0000","last_active":"2024-09-10T23:38:21.256093+0000","last_peered":"2024-09-10T23:38:21.256093+0000","last_clean":"2024-09-10T23:38:21.256093+0000","last_became_active":"2024-09-10T23:38:19.380762+0000","last_became_peered":"2024-09-10T23:38:19.380762+0000","last_unstale":"2024-09-10T23:38:21.256093+0000","last_undegraded":"2024-09-10T23:38:21.256093+0000","last_fullsized":"2024-09-10T23:38:21.256093+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:52.195279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380381+0000","last_change":"2024-09-10T23:38:19.387927+0000","last_active":"2024-09-10T23:38:20.380381+0000","last_peered":"2024-09-10T23:38:20.380381+0000","last_clean":"2024-09-10T23:38:20.380381+0000","last_became_active":"2024-09-10T23:38:19.387498+0000","last_became_peered":"2024-09-10T23:38:19.387498+0000","last_unstale":"2024-09-10T23:38:20.380381+0000","last_undegraded":"2024-09-10T23:38:20.380381+0000","last_fullsized":"2024-09-10T23:38:20.380381+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:27.140310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256543+0000","last_change":"2024-09-10T23:38:19.397733+0000","last_active":"2024-09-10T23:38:21.256543+0000","last_peered":"2024-09-10T23:38:21.256543+0000","last_clean":"2024-09-10T23:38:21.256543+0000","last_became_active":"2024-09-10T23:38:19.397347+0000","last_became_peered":"2024-09-10T23:38:19.397347+0000","last_unstale":"2024-09-10T23:38:21.256543+0000","last_undegraded":"2024-09-10T23:38:21.256543+0000","last_fullsized":"2024-09-10T23:38:21.256543+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:42.847739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384735+0000","last_change":"2024-09-10T23:38:19.382379+0000","last_active":"2024-09-10T23:38:20.384735+0000","last_peered":"2024-09-10T23:38:20.384735+0000","last_clean":"2024-09-10T23:38:20.384735+0000","last_became_active":"2024-09-10T23:38:19.382088+0000","last_became_peered":"2024-09-10T23:38:19.382088+0000","last_unstale":"2024-09-10T23:38:20.384735+0000","last_undegraded":"2024-09-10T23:38:20.384735+0000","last_fullsized":"2024-09-10T23:38:20.384735+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:47.082303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256373+0000","last_change":"2024-09-10T23:38:19.397890+0000","last_active":"2024-09-10T23:38:21.256373+0000","last_peered":"2024-09-10T23:38:21.256373+0000","last_clean":"2024-09-10T23:38:21.256373+0000","last_became_active":"2024-09-10T23:38:19.396940+0000","last_became_peered":"2024-09-10T23:38:19.396940+0000","last_unstale":"2024-09-10T23:38:21.256373+0000","last_undegraded":"2024-09-10T23:38:21.256373+0000","last_fullsized":"2024-09-10T23:38:21.256373+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:56.434591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380478+0000","last_change":"2024-09-10T23:38:19.378937+0000","last_active":"2024-09-10T23:38:20.380478+0000","last_peered":"2024-09-10T23:38:20.380478+0000","last_clean":"2024-09-10T23:38:20.380478+0000","last_became_active":"2024-09-10T23:38:19.376939+0000","last_became_peered":"2024-09-10T23:38:19.376939+0000","last_unstale":"2024-09-10T23:38:20.380478+0000","last_undegraded":"2024-09-10T23:38:20.380478+0000","last_fullsized":"2024-09-10T23:38:20.380478+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:00.549139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.751783+0000","last_change":"2024-09-10T23:38:19.383837+0000","last_active":"2024-09-10T23:38:20.751783+0000","last_peered":"2024-09-10T23:38:20.751783+0000","last_clean":"2024-09-10T23:38:20.751783+0000","last_became_active":"2024-09-10T23:38:19.383519+0000","last_became_peered":"2024-09-10T23:38:19.383519+0000","last_unstale":"2024-09-10T23:38:20.751783+0000","last_undegraded":"2024-09-10T23:38:20.751783+0000","last_fullsized":"2024-09-10T23:38:20.751783+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:58.025635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379484+0000","last_change":"2024-09-10T23:38:19.371075+0000","last_active":"2024-09-10T23:38:20.379484+0000","last_peered":"2024-09-10T23:38:20.379484+0000","last_clean":"2024-09-10T23:38:20.379484+0000","last_became_active":"2024-09-10T23:38:19.370761+0000","last_became_peered":"2024-09-10T23:38:19.370761+0000","last_unstale":"2024-09-10T23:38:20.379484+0000","last_undegraded":"2024-09-10T23:38:20.379484+0000","last_fullsized":"2024-09-10T23:38:20.379484+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:54.021452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256381+0000","last_change":"2024-09-10T23:38:19.399486+0000","last_active":"2024-09-10T23:38:21.256381+0000","last_peered":"2024-09-10T23:38:21.256381+0000","last_clean":"2024-09-10T23:38:21.256381+0000","last_became_active":"2024-09-10T23:38:19.399162+0000","last_became_peered":"2024-09-10T23:38:19.399162+0000","last_unstale":"2024-09-10T23:38:21.256381+0000","last_undegraded":"2024-09-10T23:38:21.256381+0000","last_fullsized":"2024-09-10T23:38:21.256381+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:23.857889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.752063+0000","last_change":"2024-09-10T23:38:19.383285+0000","last_active":"2024-09-10T23:38:20.752063+0000","last_peered":"2024-09-10T23:38:20.752063+0000","last_clean":"2024-09-10T23:38:20.752063+0000","last_became_active":"2024-09-10T23:38:19.382999+0000","last_became_peered":"2024-09-10T23:38:19.382999+0000","last_unstale":"2024-09-10T23:38:20.752063+0000","last_undegraded":"2024-09-10T23:38:20.752063+0000","last_fullsized":"2024-09-10T23:38:20.752063+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:56.528988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380642+0000","last_change":"2024-09-10T23:38:19.388697+0000","last_active":"2024-09-10T23:38:20.380642+0000","last_peered":"2024-09-10T23:38:20.380642+0000","last_clean":"2024-09-10T23:38:20.380642+0000","last_became_active":"2024-09-10T23:38:19.388436+0000","last_became_peered":"2024-09-10T23:38:19.388436+0000","last_unstale":"2024-09-10T23:38:20.380642+0000","last_undegraded":"2024-09-10T23:38:20.380642+0000","last_fullsized":"2024-09-10T23:38:20.380642+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:08.931506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.752119+0000","last_change":"2024-09-10T23:38:19.396084+0000","last_active":"2024-09-10T23:38:20.752119+0000","last_peered":"2024-09-10T23:38:20.752119+0000","last_clean":"2024-09-10T23:38:20.752119+0000","last_became_active":"2024-09-10T23:38:19.395760+0000","last_became_peered":"2024-09-10T23:38:19.395760+0000","last_unstale":"2024-09-10T23:38:20.752119+0000","last_undegraded":"2024-09-10T23:38:20.752119+0000","last_fullsized":"2024-09-10T23:38:20.752119+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:55.207747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384251+0000","last_change":"2024-09-10T23:38:19.382191+0000","last_active":"2024-09-10T23:38:20.384251+0000","last_peered":"2024-09-10T23:38:20.384251+0000","last_clean":"2024-09-10T23:38:20.384251+0000","last_became_active":"2024-09-10T23:38:19.381832+0000","last_became_peered":"2024-09-10T23:38:19.381832+0000","last_unstale":"2024-09-10T23:38:20.384251+0000","last_undegraded":"2024-09-10T23:38:20.384251+0000","last_fullsized":"2024-09-10T23:38:20.384251+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:12.194094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379864+0000","last_change":"2024-09-10T23:38:19.400711+0000","last_active":"2024-09-10T23:38:20.379864+0000","last_peered":"2024-09-10T23:38:20.379864+0000","last_clean":"2024-09-10T23:38:20.379864+0000","last_became_active":"2024-09-10T23:38:19.400430+0000","last_became_peered":"2024-09-10T23:38:19.400430+0000","last_unstale":"2024-09-10T23:38:20.379864+0000","last_undegraded":"2024-09-10T23:38:20.379864+0000","last_fullsized":"2024-09-10T23:38:20.379864+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:10.243290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.255684+0000","last_change":"2024-09-10T23:38:19.376402+0000","last_active":"2024-09-10T23:38:21.255684+0000","last_peered":"2024-09-10T23:38:21.255684+0000","last_clean":"2024-09-10T23:38:21.255684+0000","last_became_active":"2024-09-10T23:38:19.374000+0000","last_became_peered":"2024-09-10T23:38:19.374000+0000","last_unstale":"2024-09-10T23:38:21.255684+0000","last_undegraded":"2024-09-10T23:38:21.255684+0000","last_fullsized":"2024-09-10T23:38:21.255684+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:19.392566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.751584+0000","last_change":"2024-09-10T23:38:19.381604+0000","last_active":"2024-09-10T23:38:20.751584+0000","last_peered":"2024-09-10T23:38:20.751584+0000","last_clean":"2024-09-10T23:38:20.751584+0000","last_became_active":"2024-09-10T23:38:19.377833+0000","last_became_peered":"2024-09-10T23:38:19.377833+0000","last_unstale":"2024-09-10T23:38:20.751584+0000","last_undegraded":"2024-09-10T23:38:20.751584+0000","last_fullsized":"2024-09-10T23:38:20.751584+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:12.536624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256789+0000","last_change":"2024-09-10T23:38:19.396434+0000","last_active":"2024-09-10T23:38:21.256789+0000","last_peered":"2024-09-10T23:38:21.256789+0000","last_clean":"2024-09-10T23:38:21.256789+0000","last_became_active":"2024-09-10T23:38:19.396157+0000","last_became_peered":"2024-09-10T23:38:19.396157+0000","last_unstale":"2024-09-10T23:38:21.256789+0000","last_undegraded":"2024-09-10T23:38:21.256789+0000","last_fullsized":"2024-09-10T23:38:21.256789+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:31.455382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380597+0000","last_change":"2024-09-10T23:38:19.389694+0000","last_active":"2024-09-10T23:38:20.380597+0000","last_peered":"2024-09-10T23:38:20.380597+0000","last_clean":"2024-09-10T23:38:20.380597+0000","last_became_active":"2024-09-10T23:38:19.388300+0000","last_became_peered":"2024-09-10T23:38:19.388300+0000","last_unstale":"2024-09-10T23:38:20.380597+0000","last_undegraded":"2024-09-10T23:38:20.380597+0000","last_fullsized":"2024-09-10T23:38:20.380597+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:38.434030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.381456+0000","last_change":"2024-09-10T23:38:19.377376+0000","last_active":"2024-09-10T23:38:20.381456+0000","last_peered":"2024-09-10T23:38:20.381456+0000","last_clean":"2024-09-10T23:38:20.381456+0000","last_became_active":"2024-09-10T23:38:19.376063+0000","last_became_peered":"2024-09-10T23:38:19.376063+0000","last_unstale":"2024-09-10T23:38:20.381456+0000","last_undegraded":"2024-09-10T23:38:20.381456+0000","last_fullsized":"2024-09-10T23:38:20.381456+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:11.943937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256111+0000","last_change":"2024-09-10T23:38:19.379464+0000","last_active":"2024-09-10T23:38:21.256111+0000","last_peered":"2024-09-10T23:38:21.256111+0000","last_clean":"2024-09-10T23:38:21.256111+0000","last_became_active":"2024-09-10T23:38:19.379132+0000","last_became_peered":"2024-09-10T23:38:19.379132+0000","last_unstale":"2024-09-10T23:38:21.256111+0000","last_undegraded":"2024-09-10T23:38:21.256111+0000","last_fullsized":"2024-09-10T23:38:21.256111+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:07.671893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380181+0000","last_change":"2024-09-10T23:38:19.378864+0000","last_active":"2024-09-10T23:38:20.380181+0000","last_peered":"2024-09-10T23:38:20.380181+0000","last_clean":"2024-09-10T23:38:20.380181+0000","last_became_active":"2024-09-10T23:38:19.378570+0000","last_became_peered":"2024-09-10T23:38:19.378570+0000","last_unstale":"2024-09-10T23:38:20.380181+0000","last_undegraded":"2024-09-10T23:38:20.380181+0000","last_fullsized":"2024-09-10T23:38:20.380181+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:27.379766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379959+0000","last_change":"2024-09-10T23:38:19.385373+0000","last_active":"2024-09-10T23:38:20.379959+0000","last_peered":"2024-09-10T23:38:20.379959+0000","last_clean":"2024-09-10T23:38:20.379959+0000","last_became_active":"2024-09-10T23:38:19.382817+0000","last_became_peered":"2024-09-10T23:38:19.382817+0000","last_unstale":"2024-09-10T23:38:20.379959+0000","last_undegraded":"2024-09-10T23:38:20.379959+0000","last_fullsized":"2024-09-10T23:38:20.379959+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:18.863734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.255870+0000","last_change":"2024-09-10T23:38:19.377065+0000","last_active":"2024-09-10T23:38:21.255870+0000","last_peered":"2024-09-10T23:38:21.255870+0000","last_clean":"2024-09-10T23:38:21.255870+0000","last_became_active":"2024-09-10T23:38:19.376797+0000","last_became_peered":"2024-09-10T23:38:19.376797+0000","last_unstale":"2024-09-10T23:38:21.255870+0000","last_undegraded":"2024-09-10T23:38:21.255870+0000","last_fullsized":"2024-09-10T23:38:21.255870+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:42.344220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.255848+0000","last_change":"2024-09-10T23:38:19.378344+0000","last_active":"2024-09-10T23:38:21.255848+0000","last_peered":"2024-09-10T23:38:21.255848+0000","last_clean":"2024-09-10T23:38:21.255848+0000","last_became_active":"2024-09-10T23:38:19.378069+0000","last_became_peered":"2024-09-10T23:38:19.378069+0000","last_unstale":"2024-09-10T23:38:21.255848+0000","last_undegraded":"2024-09-10T23:38:21.255848+0000","last_fullsized":"2024-09-10T23:38:21.255848+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:03.898897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.381524+0000","last_change":"2024-09-10T23:38:19.370192+0000","last_active":"2024-09-10T23:38:20.381524+0000","last_peered":"2024-09-10T23:38:20.381524+0000","last_clean":"2024-09-10T23:38:20.381524+0000","last_became_active":"2024-09-10T23:38:19.369891+0000","last_became_peered":"2024-09-10T23:38:19.369891+0000","last_unstale":"2024-09-10T23:38:20.381524+0000","last_undegraded":"2024-09-10T23:38:20.381524+0000","last_fullsized":"2024-09-10T23:38:20.381524+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:57.277190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"113'4","reported_seq":19,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.378307+0000","last_change":"2024-09-10T23:38:18.375480+0000","last_active":"2024-09-10T23:38:20.378307+0000","last_peered":"2024-09-10T23:38:20.378307+0000","last_clean":"2024-09-10T23:38:20.378307+0000","last_became_active":"2024-09-10T23:38:18.374761+0000","last_became_peered":"2024-09-10T23:38:18.374761+0000","last_unstale":"2024-09-10T23:38:20.378307+0000","last_undegraded":"2024-09-10T23:38:20.378307+0000","last_fullsized":"2024-09-10T23:38:20.378307+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:41.337331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256528+0000","last_change":"2024-09-10T23:38:19.399680+0000","last_active":"2024-09-10T23:38:21.256528+0000","last_peered":"2024-09-10T23:38:21.256528+0000","last_clean":"2024-09-10T23:38:21.256528+0000","last_became_active":"2024-09-10T23:38:19.399424+0000","last_became_peered":"2024-09-10T23:38:19.399424+0000","last_unstale":"2024-09-10T23:38:21.256528+0000","last_undegraded":"2024-09-10T23:38:21.256528+0000","last_fullsized":"2024-09-10T23:38:21.256528+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:11.160958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256347+0000","last_change":"2024-09-10T23:38:18.373280+0000","last_active":"2024-09-10T23:38:21.256347+0000","last_peered":"2024-09-10T23:38:21.256347+0000","last_clean":"2024-09-10T23:38:21.256347+0000","last_became_active":"2024-09-10T23:38:18.372998+0000","last_became_peered":"2024-09-10T23:38:18.372998+0000","last_unstale":"2024-09-10T23:38:21.256347+0000","last_undegraded":"2024-09-10T23:38:21.256347+0000","last_fullsized":"2024-09-10T23:38:21.256347+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:30.949271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256479+0000","last_change":"2024-09-10T23:38:19.373062+0000","last_active":"2024-09-10T23:38:21.256479+0000","last_peered":"2024-09-10T23:38:21.256479+0000","last_clean":"2024-09-10T23:38:21.256479+0000","last_became_active":"2024-09-10T23:38:19.372815+0000","last_became_peered":"2024-09-10T23:38:19.372815+0000","last_unstale":"2024-09-10T23:38:21.256479+0000","last_undegraded":"2024-09-10T23:38:21.256479+0000","last_fullsized":"2024-09-10T23:38:21.256479+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:10.566874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256633+0000","last_change":"2024-09-10T23:38:18.373866+0000","last_active":"2024-09-10T23:38:21.256633+0000","last_peered":"2024-09-10T23:38:21.256633+0000","last_clean":"2024-09-10T23:38:21.256633+0000","last_became_active":"2024-09-10T23:38:18.373476+0000","last_became_peered":"2024-09-10T23:38:18.373476+0000","last_unstale":"2024-09-10T23:38:21.256633+0000","last_undegraded":"2024-09-10T23:38:21.256633+0000","last_fullsized":"2024-09-10T23:38:21.256633+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:51:35.159965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.385004+0000","last_change":"2024-09-10T23:38:19.379954+0000","last_active":"2024-09-10T23:38:20.385004+0000","last_peered":"2024-09-10T23:38:20.385004+0000","last_clean":"2024-09-10T23:38:20.385004+0000","last_became_active":"2024-09-10T23:38:19.379611+0000","last_became_peered":"2024-09-10T23:38:19.379611+0000","last_unstale":"2024-09-10T23:38:20.385004+0000","last_undegraded":"2024-09-10T23:38:20.385004+0000","last_fullsized":"2024-09-10T23:38:20.385004+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:31.455382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379832+0000","last_change":"2024-09-10T23:38:18.383056+0000","last_active":"2024-09-10T23:38:20.379832+0000","last_peered":"2024-09-10T23:38:20.379832+0000","last_clean":"2024-09-10T23:38:20.379832+0000","last_became_active":"2024-09-10T23:38:18.382477+0000","last_became_peered":"2024-09-10T23:38:18.382477+0000","last_unstale":"2024-09-10T23:38:20.379832+0000","last_undegraded":"2024-09-10T23:38:20.379832+0000","last_fullsized":"2024-09-10T23:38:20.379832+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:07.751630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.751800+0000","last_change":"2024-09-10T23:38:19.385468+0000","last_active":"2024-09-10T23:38:20.751800+0000","last_peered":"2024-09-10T23:38:20.751800+0000","last_clean":"2024-09-10T23:38:20.751800+0000","last_became_active":"2024-09-10T23:38:19.384288+0000","last_became_peered":"2024-09-10T23:38:19.384288+0000","last_unstale":"2024-09-10T23:38:20.751800+0000","last_undegraded":"2024-09-10T23:38:20.751800+0000","last_fullsized":"2024-09-10T23:38:20.751800+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:30.446896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380214+0000","last_change":"2024-09-10T23:38:18.385147+0000","last_active":"2024-09-10T23:38:20.380214+0000","last_peered":"2024-09-10T23:38:20.380214+0000","last_clean":"2024-09-10T23:38:20.380214+0000","last_became_active":"2024-09-10T23:38:18.384626+0000","last_became_peered":"2024-09-10T23:38:18.384626+0000","last_unstale":"2024-09-10T23:38:20.380214+0000","last_undegraded":"2024-09-10T23:38:20.380214+0000","last_fullsized":"2024-09-10T23:38:20.380214+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:22.829046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384281+0000","last_change":"2024-09-10T23:38:19.379900+0000","last_active":"2024-09-10T23:38:20.384281+0000","last_peered":"2024-09-10T23:38:20.384281+0000","last_clean":"2024-09-10T23:38:20.384281+0000","last_became_active":"2024-09-10T23:38:19.379405+0000","last_became_peered":"2024-09-10T23:38:19.379405+0000","last_unstale":"2024-09-10T23:38:20.384281+0000","last_undegraded":"2024-09-10T23:38:20.384281+0000","last_fullsized":"2024-09-10T23:38:20.384281+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:14.287193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.751807+0000","last_change":"2024-09-10T23:38:18.385106+0000","last_active":"2024-09-10T23:38:20.751807+0000","last_peered":"2024-09-10T23:38:20.751807+0000","last_clean":"2024-09-10T23:38:20.751807+0000","last_became_active":"2024-09-10T23:38:18.384811+0000","last_became_peered":"2024-09-10T23:38:18.384811+0000","last_unstale":"2024-09-10T23:38:20.751807+0000","last_undegraded":"2024-09-10T23:38:20.751807+0000","last_fullsized":"2024-09-10T23:38:20.751807+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:33.755052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.383971+0000","last_change":"2024-09-10T23:38:19.382660+0000","last_active":"2024-09-10T23:38:20.383971+0000","last_peered":"2024-09-10T23:38:20.383971+0000","last_clean":"2024-09-10T23:38:20.383971+0000","last_became_active":"2024-09-10T23:38:19.382216+0000","last_became_peered":"2024-09-10T23:38:19.382216+0000","last_unstale":"2024-09-10T23:38:20.383971+0000","last_undegraded":"2024-09-10T23:38:20.383971+0000","last_fullsized":"2024-09-10T23:38:20.383971+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:39.902596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.255572+0000","last_change":"2024-09-10T23:38:18.371252+0000","last_active":"2024-09-10T23:38:21.255572+0000","last_peered":"2024-09-10T23:38:21.255572+0000","last_clean":"2024-09-10T23:38:21.255572+0000","last_became_active":"2024-09-10T23:38:18.370978+0000","last_became_peered":"2024-09-10T23:38:18.370978+0000","last_unstale":"2024-09-10T23:38:21.255572+0000","last_undegraded":"2024-09-10T23:38:21.255572+0000","last_fullsized":"2024-09-10T23:38:21.255572+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:32.543277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379639+0000","last_change":"2024-09-10T23:38:19.398008+0000","last_active":"2024-09-10T23:38:20.379639+0000","last_peered":"2024-09-10T23:38:20.379639+0000","last_clean":"2024-09-10T23:38:20.379639+0000","last_became_active":"2024-09-10T23:38:19.396607+0000","last_became_peered":"2024-09-10T23:38:19.396607+0000","last_unstale":"2024-09-10T23:38:20.379639+0000","last_undegraded":"2024-09-10T23:38:20.379639+0000","last_fullsized":"2024-09-10T23:38:20.379639+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:40.863447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379493+0000","last_change":"2024-09-10T23:38:18.383288+0000","last_active":"2024-09-10T23:38:20.379493+0000","last_peered":"2024-09-10T23:38:20.379493+0000","last_clean":"2024-09-10T23:38:20.379493+0000","last_became_active":"2024-09-10T23:38:18.382239+0000","last_became_peered":"2024-09-10T23:38:18.382239+0000","last_unstale":"2024-09-10T23:38:20.379493+0000","last_undegraded":"2024-09-10T23:38:20.379493+0000","last_fullsized":"2024-09-10T23:38:20.379493+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:58.216892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380982+0000","last_change":"2024-09-10T23:38:19.397564+0000","last_active":"2024-09-10T23:38:20.380982+0000","last_peered":"2024-09-10T23:38:20.380982+0000","last_clean":"2024-09-10T23:38:20.380982+0000","last_became_active":"2024-09-10T23:38:19.397177+0000","last_became_peered":"2024-09-10T23:38:19.397177+0000","last_unstale":"2024-09-10T23:38:20.380982+0000","last_undegraded":"2024-09-10T23:38:20.380982+0000","last_fullsized":"2024-09-10T23:38:20.380982+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:56.284299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384966+0000","last_change":"2024-09-10T23:38:18.379265+0000","last_active":"2024-09-10T23:38:20.384966+0000","last_peered":"2024-09-10T23:38:20.384966+0000","last_clean":"2024-09-10T23:38:20.384966+0000","last_became_active":"2024-09-10T23:38:18.378985+0000","last_became_peered":"2024-09-10T23:38:18.378985+0000","last_unstale":"2024-09-10T23:38:20.384966+0000","last_undegraded":"2024-09-10T23:38:20.384966+0000","last_fullsized":"2024-09-10T23:38:20.384966+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:14.170106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.378275+0000","last_change":"2024-09-10T23:38:19.377453+0000","last_active":"2024-09-10T23:38:20.378275+0000","last_peered":"2024-09-10T23:38:20.378275+0000","last_clean":"2024-09-10T23:38:20.378275+0000","last_became_active":"2024-09-10T23:38:19.377208+0000","last_became_peered":"2024-09-10T23:38:19.377208+0000","last_unstale":"2024-09-10T23:38:20.378275+0000","last_undegraded":"2024-09-10T23:38:20.378275+0000","last_fullsized":"2024-09-10T23:38:20.378275+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:15.176995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.751806+0000","last_change":"2024-09-10T23:38:18.379055+0000","last_active":"2024-09-10T23:38:20.751806+0000","last_peered":"2024-09-10T23:38:20.751806+0000","last_clean":"2024-09-10T23:38:20.751806+0000","last_became_active":"2024-09-10T23:38:18.378628+0000","last_became_peered":"2024-09-10T23:38:18.378628+0000","last_unstale":"2024-09-10T23:38:20.751806+0000","last_undegraded":"2024-09-10T23:38:20.751806+0000","last_fullsized":"2024-09-10T23:38:20.751806+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:00.102925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384796+0000","last_change":"2024-09-10T23:38:19.376562+0000","last_active":"2024-09-10T23:38:20.384796+0000","last_peered":"2024-09-10T23:38:20.384796+0000","last_clean":"2024-09-10T23:38:20.384796+0000","last_became_active":"2024-09-10T23:38:19.376164+0000","last_became_peered":"2024-09-10T23:38:19.376164+0000","last_unstale":"2024-09-10T23:38:20.384796+0000","last_undegraded":"2024-09-10T23:38:20.384796+0000","last_fullsized":"2024-09-10T23:38:20.384796+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:42.847739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"113'3","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.377996+0000","last_change":"2024-09-10T23:38:18.372216+0000","last_active":"2024-09-10T23:38:20.377996+0000","last_peered":"2024-09-10T23:38:20.377996+0000","last_clean":"2024-09-10T23:38:20.377996+0000","last_became_active":"2024-09-10T23:38:18.371982+0000","last_became_peered":"2024-09-10T23:38:18.371982+0000","last_unstale":"2024-09-10T23:38:20.377996+0000","last_undegraded":"2024-09-10T23:38:20.377996+0000","last_fullsized":"2024-09-10T23:38:20.377996+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:46.042598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.255952+0000","last_change":"2024-09-10T23:38:19.381538+0000","last_active":"2024-09-10T23:38:21.255952+0000","last_peered":"2024-09-10T23:38:21.255952+0000","last_clean":"2024-09-10T23:38:21.255952+0000","last_became_active":"2024-09-10T23:38:19.381294+0000","last_became_peered":"2024-09-10T23:38:19.381294+0000","last_unstale":"2024-09-10T23:38:21.255952+0000","last_undegraded":"2024-09-10T23:38:21.255952+0000","last_fullsized":"2024-09-10T23:38:21.255952+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:40.004917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"113'6","reported_seq":21,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256012+0000","last_change":"2024-09-10T23:38:18.375147+0000","last_active":"2024-09-10T23:38:21.256012+0000","last_peered":"2024-09-10T23:38:21.256012+0000","last_clean":"2024-09-10T23:38:21.256012+0000","last_became_active":"2024-09-10T23:38:18.374845+0000","last_became_peered":"2024-09-10T23:38:18.374845+0000","last_unstale":"2024-09-10T23:38:21.256012+0000","last_undegraded":"2024-09-10T23:38:21.256012+0000","last_fullsized":"2024-09-10T23:38:21.256012+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:27.452591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.381152+0000","last_change":"2024-09-10T23:38:19.399437+0000","last_active":"2024-09-10T23:38:20.381152+0000","last_peered":"2024-09-10T23:38:20.381152+0000","last_clean":"2024-09-10T23:38:20.381152+0000","last_became_active":"2024-09-10T23:38:19.398838+0000","last_became_peered":"2024-09-10T23:38:19.398838+0000","last_unstale":"2024-09-10T23:38:20.381152+0000","last_undegraded":"2024-09-10T23:38:20.381152+0000","last_fullsized":"2024-09-10T23:38:20.381152+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:49.239269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380770+0000","last_change":"2024-09-10T23:38:18.383148+0000","last_active":"2024-09-10T23:38:20.380770+0000","last_peered":"2024-09-10T23:38:20.380770+0000","last_clean":"2024-09-10T23:38:20.380770+0000","last_became_active":"2024-09-10T23:38:18.382525+0000","last_became_peered":"2024-09-10T23:38:18.382525+0000","last_unstale":"2024-09-10T23:38:20.380770+0000","last_undegraded":"2024-09-10T23:38:20.380770+0000","last_fullsized":"2024-09-10T23:38:20.380770+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:12.476365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380337+0000","last_change":"2024-09-10T23:38:19.370305+0000","last_active":"2024-09-10T23:38:20.380337+0000","last_peered":"2024-09-10T23:38:20.380337+0000","last_clean":"2024-09-10T23:38:20.380337+0000","last_became_active":"2024-09-10T23:38:19.369980+0000","last_became_peered":"2024-09-10T23:38:19.369980+0000","last_unstale":"2024-09-10T23:38:20.380337+0000","last_undegraded":"2024-09-10T23:38:20.380337+0000","last_fullsized":"2024-09-10T23:38:20.380337+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:53.229557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256038+0000","last_change":"2024-09-10T23:38:18.380342+0000","last_active":"2024-09-10T23:38:21.256038+0000","last_peered":"2024-09-10T23:38:21.256038+0000","last_clean":"2024-09-10T23:38:21.256038+0000","last_became_active":"2024-09-10T23:38:18.380064+0000","last_became_peered":"2024-09-10T23:38:18.380064+0000","last_unstale":"2024-09-10T23:38:21.256038+0000","last_undegraded":"2024-09-10T23:38:21.256038+0000","last_fullsized":"2024-09-10T23:38:21.256038+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:38.895707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384069+0000","last_change":"2024-09-10T23:38:19.375606+0000","last_active":"2024-09-10T23:38:20.384069+0000","last_peered":"2024-09-10T23:38:20.384069+0000","last_clean":"2024-09-10T23:38:20.384069+0000","last_became_active":"2024-09-10T23:38:19.375346+0000","last_became_peered":"2024-09-10T23:38:19.375346+0000","last_unstale":"2024-09-10T23:38:20.384069+0000","last_undegraded":"2024-09-10T23:38:20.384069+0000","last_fullsized":"2024-09-10T23:38:20.384069+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:20.050463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"113'8","reported_seq":23,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.383874+0000","last_change":"2024-09-10T23:38:18.380372+0000","last_active":"2024-09-10T23:38:20.383874+0000","last_peered":"2024-09-10T23:38:20.383874+0000","last_clean":"2024-09-10T23:38:20.383874+0000","last_became_active":"2024-09-10T23:38:18.380070+0000","last_became_peered":"2024-09-10T23:38:18.380070+0000","last_unstale":"2024-09-10T23:38:20.383874+0000","last_undegraded":"2024-09-10T23:38:20.383874+0000","last_fullsized":"2024-09-10T23:38:20.383874+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:37.192323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.378114+0000","last_change":"2024-09-10T23:38:19.378545+0000","last_active":"2024-09-10T23:38:20.378114+0000","last_peered":"2024-09-10T23:38:20.378114+0000","last_clean":"2024-09-10T23:38:20.378114+0000","last_became_active":"2024-09-10T23:38:19.378297+0000","last_became_peered":"2024-09-10T23:38:19.378297+0000","last_unstale":"2024-09-10T23:38:20.378114+0000","last_undegraded":"2024-09-10T23:38:20.378114+0000","last_fullsized":"2024-09-10T23:38:20.378114+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:22.259216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.385011+0000","last_change":"2024-09-10T23:38:18.377800+0000","last_active":"2024-09-10T23:38:20.385011+0000","last_peered":"2024-09-10T23:38:20.385011+0000","last_clean":"2024-09-10T23:38:20.385011+0000","last_became_active":"2024-09-10T23:38:18.377551+0000","last_became_peered":"2024-09-10T23:38:18.377551+0000","last_unstale":"2024-09-10T23:38:20.385011+0000","last_undegraded":"2024-09-10T23:38:20.385011+0000","last_fullsized":"2024-09-10T23:38:20.385011+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:47.941526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.378294+0000","last_change":"2024-09-10T23:38:19.397958+0000","last_active":"2024-09-10T23:38:20.378294+0000","last_peered":"2024-09-10T23:38:20.378294+0000","last_clean":"2024-09-10T23:38:20.378294+0000","last_became_active":"2024-09-10T23:38:19.397490+0000","last_became_peered":"2024-09-10T23:38:19.397490+0000","last_unstale":"2024-09-10T23:38:20.378294+0000","last_undegraded":"2024-09-10T23:38:20.378294+0000","last_fullsized":"2024-09-10T23:38:20.378294+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:48.948415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256376+0000","last_change":"2024-09-10T23:38:18.377068+0000","last_active":"2024-09-10T23:38:21.256376+0000","last_peered":"2024-09-10T23:38:21.256376+0000","last_clean":"2024-09-10T23:38:21.256376+0000","last_became_active":"2024-09-10T23:38:18.376777+0000","last_became_peered":"2024-09-10T23:38:18.376777+0000","last_unstale":"2024-09-10T23:38:21.256376+0000","last_undegraded":"2024-09-10T23:38:21.256376+0000","last_fullsized":"2024-09-10T23:38:21.256376+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:37.295397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380191+0000","last_change":"2024-09-10T23:38:19.386807+0000","last_active":"2024-09-10T23:38:20.380191+0000","last_peered":"2024-09-10T23:38:20.380191+0000","last_clean":"2024-09-10T23:38:20.380191+0000","last_became_active":"2024-09-10T23:38:19.386534+0000","last_became_peered":"2024-09-10T23:38:19.386534+0000","last_unstale":"2024-09-10T23:38:20.380191+0000","last_undegraded":"2024-09-10T23:38:20.380191+0000","last_fullsized":"2024-09-10T23:38:20.380191+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:57.494220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256347+0000","last_change":"2024-09-10T23:38:18.377255+0000","last_active":"2024-09-10T23:38:21.256347+0000","last_peered":"2024-09-10T23:38:21.256347+0000","last_clean":"2024-09-10T23:38:21.256347+0000","last_became_active":"2024-09-10T23:38:18.376970+0000","last_became_peered":"2024-09-10T23:38:18.376970+0000","last_unstale":"2024-09-10T23:38:21.256347+0000","last_undegraded":"2024-09-10T23:38:21.256347+0000","last_fullsized":"2024-09-10T23:38:21.256347+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:43.974712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380682+0000","last_change":"2024-09-10T23:38:19.399728+0000","last_active":"2024-09-10T23:38:20.380682+0000","last_peered":"2024-09-10T23:38:20.380682+0000","last_clean":"2024-09-10T23:38:20.380682+0000","last_became_active":"2024-09-10T23:38:19.399447+0000","last_became_peered":"2024-09-10T23:38:19.399447+0000","last_unstale":"2024-09-10T23:38:20.380682+0000","last_undegraded":"2024-09-10T23:38:20.380682+0000","last_fullsized":"2024-09-10T23:38:20.380682+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:58.972436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.377958+0000","last_change":"2024-09-10T23:38:18.383424+0000","last_active":"2024-09-10T23:38:20.377958+0000","last_peered":"2024-09-10T23:38:20.377958+0000","last_clean":"2024-09-10T23:38:20.377958+0000","last_became_active":"2024-09-10T23:38:18.383034+0000","last_became_peered":"2024-09-10T23:38:18.383034+0000","last_unstale":"2024-09-10T23:38:20.377958+0000","last_undegraded":"2024-09-10T23:38:20.377958+0000","last_fullsized":"2024-09-10T23:38:20.377958+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:17:20.177415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380104+0000","last_change":"2024-09-10T23:38:19.385679+0000","last_active":"2024-09-10T23:38:20.380104+0000","last_peered":"2024-09-10T23:38:20.380104+0000","last_clean":"2024-09-10T23:38:20.380104+0000","last_became_active":"2024-09-10T23:38:19.384976+0000","last_became_peered":"2024-09-10T23:38:19.384976+0000","last_unstale":"2024-09-10T23:38:20.380104+0000","last_undegraded":"2024-09-10T23:38:20.380104+0000","last_fullsized":"2024-09-10T23:38:20.380104+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:24.440181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256203+0000","last_change":"2024-09-10T23:38:18.376656+0000","last_active":"2024-09-10T23:38:21.256203+0000","last_peered":"2024-09-10T23:38:21.256203+0000","last_clean":"2024-09-10T23:38:21.256203+0000","last_became_active":"2024-09-10T23:38:18.376404+0000","last_became_peered":"2024-09-10T23:38:18.376404+0000","last_unstale":"2024-09-10T23:38:21.256203+0000","last_undegraded":"2024-09-10T23:38:21.256203+0000","last_fullsized":"2024-09-10T23:38:21.256203+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:44.189752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.255744+0000","last_change":"2024-09-10T23:38:19.371004+0000","last_active":"2024-09-10T23:38:21.255744+0000","last_peered":"2024-09-10T23:38:21.255744+0000","last_clean":"2024-09-10T23:38:21.255744+0000","last_became_active":"2024-09-10T23:38:19.370691+0000","last_became_peered":"2024-09-10T23:38:19.370691+0000","last_unstale":"2024-09-10T23:38:21.255744+0000","last_undegraded":"2024-09-10T23:38:21.255744+0000","last_fullsized":"2024-09-10T23:38:21.255744+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:17:21.184304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.751904+0000","last_change":"2024-09-10T23:38:18.382296+0000","last_active":"2024-09-10T23:38:20.751904+0000","last_peered":"2024-09-10T23:38:20.751904+0000","last_clean":"2024-09-10T23:38:20.751904+0000","last_became_active":"2024-09-10T23:38:18.380974+0000","last_became_peered":"2024-09-10T23:38:18.380974+0000","last_unstale":"2024-09-10T23:38:20.751904+0000","last_undegraded":"2024-09-10T23:38:20.751904+0000","last_fullsized":"2024-09-10T23:38:20.751904+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:39.791907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379779+0000","last_change":"2024-09-10T23:38:19.374817+0000","last_active":"2024-09-10T23:38:20.379779+0000","last_peered":"2024-09-10T23:38:20.379779+0000","last_clean":"2024-09-10T23:38:20.379779+0000","last_became_active":"2024-09-10T23:38:19.374445+0000","last_became_peered":"2024-09-10T23:38:19.374445+0000","last_unstale":"2024-09-10T23:38:20.379779+0000","last_undegraded":"2024-09-10T23:38:20.379779+0000","last_fullsized":"2024-09-10T23:38:20.379779+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:23.336535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380151+0000","last_change":"2024-09-10T23:38:18.378009+0000","last_active":"2024-09-10T23:38:20.380151+0000","last_peered":"2024-09-10T23:38:20.380151+0000","last_clean":"2024-09-10T23:38:20.380151+0000","last_became_active":"2024-09-10T23:38:18.377249+0000","last_became_peered":"2024-09-10T23:38:18.377249+0000","last_unstale":"2024-09-10T23:38:20.380151+0000","last_undegraded":"2024-09-10T23:38:20.380151+0000","last_fullsized":"2024-09-10T23:38:20.380151+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:07.004388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384097+0000","last_change":"2024-09-10T23:38:19.377322+0000","last_active":"2024-09-10T23:38:20.384097+0000","last_peered":"2024-09-10T23:38:20.384097+0000","last_clean":"2024-09-10T23:38:20.384097+0000","last_became_active":"2024-09-10T23:38:19.377088+0000","last_became_peered":"2024-09-10T23:38:19.377088+0000","last_unstale":"2024-09-10T23:38:20.384097+0000","last_undegraded":"2024-09-10T23:38:20.384097+0000","last_fullsized":"2024-09-10T23:38:20.384097+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:04.775309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256026+0000","last_change":"2024-09-10T23:38:18.378023+0000","last_active":"2024-09-10T23:38:21.256026+0000","last_peered":"2024-09-10T23:38:21.256026+0000","last_clean":"2024-09-10T23:38:21.256026+0000","last_became_active":"2024-09-10T23:38:18.377774+0000","last_became_peered":"2024-09-10T23:38:18.377774+0000","last_unstale":"2024-09-10T23:38:21.256026+0000","last_undegraded":"2024-09-10T23:38:21.256026+0000","last_fullsized":"2024-09-10T23:38:21.256026+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:23.451248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256266+0000","last_change":"2024-09-10T23:38:19.377645+0000","last_active":"2024-09-10T23:38:21.256266+0000","last_peered":"2024-09-10T23:38:21.256266+0000","last_clean":"2024-09-10T23:38:21.256266+0000","last_became_active":"2024-09-10T23:38:19.377334+0000","last_became_peered":"2024-09-10T23:38:19.377334+0000","last_unstale":"2024-09-10T23:38:21.256266+0000","last_undegraded":"2024-09-10T23:38:21.256266+0000","last_fullsized":"2024-09-10T23:38:21.256266+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:28.697385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.751992+0000","last_change":"2024-09-10T23:38:18.381848+0000","last_active":"2024-09-10T23:38:20.751992+0000","last_peered":"2024-09-10T23:38:20.751992+0000","last_clean":"2024-09-10T23:38:20.751992+0000","last_became_active":"2024-09-10T23:38:18.380901+0000","last_became_peered":"2024-09-10T23:38:18.380901+0000","last_unstale":"2024-09-10T23:38:20.751992+0000","last_undegraded":"2024-09-10T23:38:20.751992+0000","last_fullsized":"2024-09-10T23:38:20.751992+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:17:21.047034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380393+0000","last_change":"2024-09-10T23:38:19.385080+0000","last_active":"2024-09-10T23:38:20.380393+0000","last_peered":"2024-09-10T23:38:20.380393+0000","last_clean":"2024-09-10T23:38:20.380393+0000","last_became_active":"2024-09-10T23:38:19.384810+0000","last_became_peered":"2024-09-10T23:38:19.384810+0000","last_unstale":"2024-09-10T23:38:20.380393+0000","last_undegraded":"2024-09-10T23:38:20.380393+0000","last_fullsized":"2024-09-10T23:38:20.380393+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:38.036081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"114'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.378069+0000","last_change":"2024-09-10T23:38:18.372313+0000","last_active":"2024-09-10T23:38:20.378069+0000","last_peered":"2024-09-10T23:38:20.378069+0000","last_clean":"2024-09-10T23:38:20.378069+0000","last_became_active":"2024-09-10T23:38:18.371745+0000","last_became_peered":"2024-09-10T23:38:18.371745+0000","last_unstale":"2024-09-10T23:38:20.378069+0000","last_undegraded":"2024-09-10T23:38:20.378069+0000","last_fullsized":"2024-09-10T23:38:20.378069+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:18.385677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.381200+0000","last_change":"2024-09-10T23:38:19.375455+0000","last_active":"2024-09-10T23:38:20.381200+0000","last_peered":"2024-09-10T23:38:20.381200+0000","last_clean":"2024-09-10T23:38:20.381200+0000","last_became_active":"2024-09-10T23:38:19.374472+0000","last_became_peered":"2024-09-10T23:38:19.374472+0000","last_unstale":"2024-09-10T23:38:20.381200+0000","last_undegraded":"2024-09-10T23:38:20.381200+0000","last_fullsized":"2024-09-10T23:38:20.381200+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:38.307389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.752052+0000","last_change":"2024-09-10T23:38:18.375834+0000","last_active":"2024-09-10T23:38:20.752052+0000","last_peered":"2024-09-10T23:38:20.752052+0000","last_clean":"2024-09-10T23:38:20.752052+0000","last_became_active":"2024-09-10T23:38:18.375538+0000","last_became_peered":"2024-09-10T23:38:18.375538+0000","last_unstale":"2024-09-10T23:38:20.752052+0000","last_undegraded":"2024-09-10T23:38:20.752052+0000","last_fullsized":"2024-09-10T23:38:20.752052+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:45.172467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256454+0000","last_change":"2024-09-10T23:38:19.379972+0000","last_active":"2024-09-10T23:38:21.256454+0000","last_peered":"2024-09-10T23:38:21.256454+0000","last_clean":"2024-09-10T23:38:21.256454+0000","last_became_active":"2024-09-10T23:38:19.379418+0000","last_became_peered":"2024-09-10T23:38:19.379418+0000","last_unstale":"2024-09-10T23:38:21.256454+0000","last_undegraded":"2024-09-10T23:38:21.256454+0000","last_fullsized":"2024-09-10T23:38:21.256454+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:31.392621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.381095+0000","last_change":"2024-09-10T23:38:18.381436+0000","last_active":"2024-09-10T23:38:20.381095+0000","last_peered":"2024-09-10T23:38:20.381095+0000","last_clean":"2024-09-10T23:38:20.381095+0000","last_became_active":"2024-09-10T23:38:18.381134+0000","last_became_peered":"2024-09-10T23:38:18.381134+0000","last_unstale":"2024-09-10T23:38:20.381095+0000","last_undegraded":"2024-09-10T23:38:20.381095+0000","last_fullsized":"2024-09-10T23:38:20.381095+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:54.849657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384572+0000","last_change":"2024-09-10T23:38:19.376422+0000","last_active":"2024-09-10T23:38:20.384572+0000","last_peered":"2024-09-10T23:38:20.384572+0000","last_clean":"2024-09-10T23:38:20.384572+0000","last_became_active":"2024-09-10T23:38:19.376002+0000","last_became_peered":"2024-09-10T23:38:19.376002+0000","last_unstale":"2024-09-10T23:38:20.384572+0000","last_undegraded":"2024-09-10T23:38:20.384572+0000","last_fullsized":"2024-09-10T23:38:20.384572+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:56.434591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380260+0000","last_change":"2024-09-10T23:38:18.383845+0000","last_active":"2024-09-10T23:38:20.380260+0000","last_peered":"2024-09-10T23:38:20.380260+0000","last_clean":"2024-09-10T23:38:20.380260+0000","last_became_active":"2024-09-10T23:38:18.383430+0000","last_became_peered":"2024-09-10T23:38:18.383430+0000","last_unstale":"2024-09-10T23:38:20.380260+0000","last_undegraded":"2024-09-10T23:38:20.380260+0000","last_fullsized":"2024-09-10T23:38:20.380260+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:54.546403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384451+0000","last_change":"2024-09-10T23:38:19.399489+0000","last_active":"2024-09-10T23:38:20.384451+0000","last_peered":"2024-09-10T23:38:20.384451+0000","last_clean":"2024-09-10T23:38:20.384451+0000","last_became_active":"2024-09-10T23:38:19.399219+0000","last_became_peered":"2024-09-10T23:38:19.399219+0000","last_unstale":"2024-09-10T23:38:20.384451+0000","last_undegraded":"2024-09-10T23:38:20.384451+0000","last_fullsized":"2024-09-10T23:38:20.384451+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:38.886159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384116+0000","last_change":"2024-09-10T23:38:18.371469+0000","last_active":"2024-09-10T23:38:20.384116+0000","last_peered":"2024-09-10T23:38:20.384116+0000","last_clean":"2024-09-10T23:38:20.384116+0000","last_became_active":"2024-09-10T23:38:18.371213+0000","last_became_peered":"2024-09-10T23:38:18.371213+0000","last_unstale":"2024-09-10T23:38:20.384116+0000","last_undegraded":"2024-09-10T23:38:20.384116+0000","last_fullsized":"2024-09-10T23:38:20.384116+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:22.864876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.751921+0000","last_change":"2024-09-10T23:38:19.390957+0000","last_active":"2024-09-10T23:38:20.751921+0000","last_peered":"2024-09-10T23:38:20.751921+0000","last_clean":"2024-09-10T23:38:20.751921+0000","last_became_active":"2024-09-10T23:38:19.390694+0000","last_became_peered":"2024-09-10T23:38:19.390694+0000","last_unstale":"2024-09-10T23:38:20.751921+0000","last_undegraded":"2024-09-10T23:38:20.751921+0000","last_fullsized":"2024-09-10T23:38:20.751921+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:55.859293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256147+0000","last_change":"2024-09-10T23:38:18.376639+0000","last_active":"2024-09-10T23:38:21.256147+0000","last_peered":"2024-09-10T23:38:21.256147+0000","last_clean":"2024-09-10T23:38:21.256147+0000","last_became_active":"2024-09-10T23:38:18.376337+0000","last_became_peered":"2024-09-10T23:38:18.376337+0000","last_unstale":"2024-09-10T23:38:21.256147+0000","last_undegraded":"2024-09-10T23:38:21.256147+0000","last_fullsized":"2024-09-10T23:38:21.256147+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:19.043574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.380665+0000","last_change":"2024-09-10T23:38:19.373475+0000","last_active":"2024-09-10T23:38:20.380665+0000","last_peered":"2024-09-10T23:38:20.380665+0000","last_clean":"2024-09-10T23:38:20.380665+0000","last_became_active":"2024-09-10T23:38:19.373205+0000","last_became_peered":"2024-09-10T23:38:19.373205+0000","last_unstale":"2024-09-10T23:38:20.380665+0000","last_undegraded":"2024-09-10T23:38:20.380665+0000","last_fullsized":"2024-09-10T23:38:20.380665+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:49.934280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.379847+0000","last_change":"2024-09-10T23:38:18.382530+0000","last_active":"2024-09-10T23:38:20.379847+0000","last_peered":"2024-09-10T23:38:20.379847+0000","last_clean":"2024-09-10T23:38:20.379847+0000","last_became_active":"2024-09-10T23:38:18.381654+0000","last_became_peered":"2024-09-10T23:38:18.381654+0000","last_unstale":"2024-09-10T23:38:20.379847+0000","last_undegraded":"2024-09-10T23:38:20.379847+0000","last_fullsized":"2024-09-10T23:38:20.379847+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:59.553134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.256016+0000","last_change":"2024-09-10T23:38:19.396149+0000","last_active":"2024-09-10T23:38:21.256016+0000","last_peered":"2024-09-10T23:38:21.256016+0000","last_clean":"2024-09-10T23:38:21.256016+0000","last_became_active":"2024-09-10T23:38:19.395879+0000","last_became_peered":"2024-09-10T23:38:19.395879+0000","last_unstale":"2024-09-10T23:38:21.256016+0000","last_undegraded":"2024-09-10T23:38:21.256016+0000","last_fullsized":"2024-09-10T23:38:21.256016+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:18.347753+0000","last_clean_scrub_stamp":"2024-09-10T23:38:18.347753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:48.172091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:20.384404+0000","last_change":"2024-09-10T23:38:18.372768+0000","last_active":"2024-09-10T23:38:20.384404+0000","last_peered":"2024-09-10T23:38:20.384404+0000","last_clean":"2024-09-10T23:38:20.384404+0000","last_became_active":"2024-09-10T23:38:18.372508+0000","last_became_peered":"2024-09-10T23:38:18.372508+0000","last_unstale":"2024-09-10T23:38:20.384404+0000","last_undegraded":"2024-09-10T23:38:20.384404+0000","last_fullsized":"2024-09-10T23:38:20.384404+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:17.340864+0000","last_clean_scrub_stamp":"2024-09-10T23:38:17.340864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:54.620298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":307,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T23:38:21.255383+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:38:21.255383+0000","last_peered":"2024-09-10T23:38:21.255383+0000","last_clean":"2024-09-10T23:38:21.255383+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:38:21.255383+0000","last_undegraded":"2024-09-10T23:38:21.255383+0000","last_fullsized":"2024-09-10T23:38:21.255383+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509516,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1421312,"data_stored":621433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542219,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1417216,"data_stored":621399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509515,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2023424,"data_stored":1212509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509513,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1417216,"data_stored":621399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509514,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1421312,"data_stored":621469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509515,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1425408,"data_stored":622071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509514,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2027520,"data_stored":1213401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509514,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1437696,"data_stored":623067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:38:23.876 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T23:38:23.876 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T23:38:24.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.057+0000 7f1f2e800640 1 -- 172.21.15.94:0/618265341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f280c49b0 msgr2=0x7f1f28165b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:24.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.057+0000 7f1f2e800640 1 --2- 172.21.15.94:0/618265341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f280c49b0 0x7f1f28165b10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009bc0 tx=0x7f1f2001cba0 comp rx=0 tx=0).stop 2024-09-10T23:38:24.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.057+0000 7f1f2e800640 1 -- 172.21.15.94:0/618265341 shutdown_connections 2024-09-10T23:38:24.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.057+0000 7f1f2e800640 1 --2- 172.21.15.94:0/618265341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f280c49b0 0x7f1f28165b10 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:24.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.057+0000 7f1f2e800640 1 -- 172.21.15.94:0/618265341 >> 172.21.15.94:0/618265341 conn(0x7f1f280cfaf0 msgr2=0x7f1f280cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:24.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.057+0000 7f1f2e800640 1 -- 172.21.15.94:0/618265341 shutdown_connections 2024-09-10T23:38:24.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.057+0000 7f1f2e800640 1 -- 172.21.15.94:0/618265341 wait complete. 2024-09-10T23:38:24.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f2e800640 1 Processor -- start 2024-09-10T23:38:24.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f2e800640 1 -- start start 2024-09-10T23:38:24.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f2e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f2806fc00 0x7f1f28153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:24.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f2e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f280c49b0 0x7f1f281507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:24.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f2e800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f28158790 0x7f1f281fb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:24.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f2e800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f28168f80 con 0x7f1f2806fc00 2024-09-10T23:38:24.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f2e800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1f28168e00 con 0x7f1f280c49b0 2024-09-10T23:38:24.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f2e800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1f28169100 con 0x7f1f28158790 2024-09-10T23:38:24.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f27400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f2806fc00 0x7f1f28153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:24.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f27400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f2806fc00 0x7f1f28153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48694/0 (socket says 172.21.15.94:48694) 2024-09-10T23:38:24.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f27400640 1 -- 172.21.15.94:0/3254853655 learned_addr learned my addr 172.21.15.94:0/3254853655 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:24.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f1ea00640 1 --2- 172.21.15.94:0/3254853655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f280c49b0 0x7f1f281507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:24.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f27400640 1 -- 172.21.15.94:0/3254853655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f28158790 msgr2=0x7f1f281fb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:24.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f27400640 1 --2- 172.21.15.94:0/3254853655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f28158790 0x7f1f281fb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:24.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f27400640 1 -- 172.21.15.94:0/3254853655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f280c49b0 msgr2=0x7f1f281507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:24.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f27400640 1 --2- 172.21.15.94:0/3254853655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f280c49b0 0x7f1f281507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:24.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f27400640 1 -- 172.21.15.94:0/3254853655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f281512b0 con 0x7f1f2806fc00 2024-09-10T23:38:24.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f1ea00640 1 --2- 172.21.15.94:0/3254853655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f280c49b0 0x7f1f281507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f27400640 1 --2- 172.21.15.94:0/3254853655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f2806fc00 0x7f1f28153180 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800c400 tx=0x7f1f1800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f1fe00640 1 -- 172.21.15.94:0/3254853655 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18011790 con 0x7f1f2806fc00 2024-09-10T23:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f1fe00640 1 -- 172.21.15.94:0/3254853655 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f18017070 con 0x7f1f2806fc00 2024-09-10T23:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f1fe00640 1 -- 172.21.15.94:0/3254853655 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18011ce0 con 0x7f1f2806fc00 2024-09-10T23:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f2e800640 1 -- 172.21.15.94:0/3254853655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f28158f50 con 0x7f1f2806fc00 2024-09-10T23:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.061+0000 7f1f2e800640 1 -- 172.21.15.94:0/3254853655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f28151f30 con 0x7f1f2806fc00 2024-09-10T23:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.065+0000 7f1f2e800640 1 -- 172.21.15.94:0/3254853655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f280cadc0 con 0x7f1f2806fc00 2024-09-10T23:38:24.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.065+0000 7f1f1fe00640 1 -- 172.21.15.94:0/3254853655 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f1f1801c460 con 0x7f1f2806fc00 2024-09-10T23:38:24.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.065+0000 7f1f1fe00640 1 --2- 172.21.15.94:0/3254853655 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1f04072a60 0x7f1f04074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:24.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.065+0000 7f1f1fe00640 1 -- 172.21.15.94:0/3254853655 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f1f180938e0 con 0x7f1f2806fc00 2024-09-10T23:38:24.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.065+0000 7f1f1ea00640 1 --2- 172.21.15.94:0/3254853655 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1f04072a60 0x7f1f04074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:24.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.065+0000 7f1f1ea00640 1 --2- 172.21.15.94:0/3254853655 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1f04072a60 0x7f1f04074f20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1f281519e0 tx=0x7f1f10009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.069+0000 7f1f1fe00640 1 -- 172.21.15.94:0/3254853655 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1f1805e910 con 0x7f1f2806fc00 2024-09-10T23:38:24.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.293+0000 7f1f2e800640 1 -- 172.21.15.94:0/3254853655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f1f280ae7c0 con 0x7f1f2806fc00 2024-09-10T23:38:24.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.297+0000 7f1f1fe00640 1 -- 172.21.15.94:0/3254853655 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v36) ==== 190+0+147 (secure 0 0 0) 0x7f1f1805e210 con 0x7f1f2806fc00 2024-09-10T23:38:24.299 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-10T23:38:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g== 2024-09-10T23:38:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: caps mds = "allow" 2024-09-10T23:38:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: caps mon = "allow r" 2024-09-10T23:38:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T23:38:24.299 INFO:teuthology.orchestra.run.smithi094.stderr:updated caps for client.0 2024-09-10T23:38:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.297+0000 7f1f2e800640 1 -- 172.21.15.94:0/3254853655 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1f04072a60 msgr2=0x7f1f04074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.297+0000 7f1f2e800640 1 --2- 172.21.15.94:0/3254853655 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1f04072a60 0x7f1f04074f20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1f281519e0 tx=0x7f1f10009040 comp rx=0 tx=0).stop 2024-09-10T23:38:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.297+0000 7f1f2e800640 1 -- 172.21.15.94:0/3254853655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f2806fc00 msgr2=0x7f1f28153180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.297+0000 7f1f2e800640 1 --2- 172.21.15.94:0/3254853655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f2806fc00 0x7f1f28153180 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800c400 tx=0x7f1f1800c8c0 comp rx=0 tx=0).stop 2024-09-10T23:38:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.297+0000 7f1f2e800640 1 -- 172.21.15.94:0/3254853655 shutdown_connections 2024-09-10T23:38:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.297+0000 7f1f2e800640 1 --2- 172.21.15.94:0/3254853655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f280c49b0 0x7f1f281507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:24.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.297+0000 7f1f2e800640 1 --2- 172.21.15.94:0/3254853655 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1f04072a60 0x7f1f04074f20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:24.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.297+0000 7f1f2e800640 1 --2- 172.21.15.94:0/3254853655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f28158790 0x7f1f281fb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:24.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.297+0000 7f1f2e800640 1 --2- 172.21.15.94:0/3254853655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f2806fc00 0x7f1f28153180 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:24.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.297+0000 7f1f2e800640 1 -- 172.21.15.94:0/3254853655 >> 172.21.15.94:0/3254853655 conn(0x7f1f280cfaf0 msgr2=0x7f1f280cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:24.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.297+0000 7f1f2e800640 1 -- 172.21.15.94:0/3254853655 shutdown_connections 2024-09-10T23:38:24.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.297+0000 7f1f2e800640 1 -- 172.21.15.94:0/3254853655 wait complete. 2024-09-10T23:38:24.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:38:24.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.461+0000 7f64a2600640 1 -- 172.21.15.94:0/4019814870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6480003a40 msgr2=0x7f6480003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:24.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.461+0000 7f64a2600640 1 --2- 172.21.15.94:0/4019814870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6480003a40 0x7f6480003e60 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f648c0016f0 tx=0x7f648c01c7b0 comp rx=0 tx=0).stop 2024-09-10T23:38:24.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.461+0000 7f649be00640 1 -- 172.21.15.94:0/4019814870 reap_dead start 2024-09-10T23:38:24.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.461+0000 7f64a2600640 1 -- 172.21.15.94:0/4019814870 shutdown_connections 2024-09-10T23:38:24.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.461+0000 7f64a2600640 1 -- 172.21.15.94:0/4019814870 >> 172.21.15.94:0/4019814870 conn(0x7f649c06cdf0 msgr2=0x7f649c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:24.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.461+0000 7f64a2600640 1 -- 172.21.15.94:0/4019814870 shutdown_connections 2024-09-10T23:38:24.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.461+0000 7f64a2600640 1 -- 172.21.15.94:0/4019814870 wait complete. 2024-09-10T23:38:24.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f64a2600640 1 Processor -- start 2024-09-10T23:38:24.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f64a2600640 1 -- start start 2024-09-10T23:38:24.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f64a2600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6480003a40 0x7f649c146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:24.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f64a2600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f649c14f9b0 0x7f649c14fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:24.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f64a2600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649c152460 0x7f649c152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:24.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f64a2600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f649c166550 con 0x7f649c14f9b0 2024-09-10T23:38:24.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f64a2600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f649c1663d0 con 0x7f649c152460 2024-09-10T23:38:24.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f64a2600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f649c1666d0 con 0x7f6480003a40 2024-09-10T23:38:24.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f649aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f649c14f9b0 0x7f649c14fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:24.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f649aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f649c14f9b0 0x7f649c14fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48722/0 (socket says 172.21.15.94:48722) 2024-09-10T23:38:24.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f649aa00640 1 -- 172.21.15.94:0/1453107219 learned_addr learned my addr 172.21.15.94:0/1453107219 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:24.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f649b400640 1 --2- 172.21.15.94:0/1453107219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6480003a40 0x7f649c146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:24.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f649be00640 1 --2- 172.21.15.94:0/1453107219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649c152460 0x7f649c152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:24.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f649aa00640 1 -- 172.21.15.94:0/1453107219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6480003a40 msgr2=0x7f649c146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:24.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f649aa00640 1 --2- 172.21.15.94:0/1453107219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6480003a40 0x7f649c146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:24.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f649aa00640 1 -- 172.21.15.94:0/1453107219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649c152460 msgr2=0x7f649c152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:24.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f649aa00640 1 --2- 172.21.15.94:0/1453107219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649c152460 0x7f649c152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:24.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f649aa00640 1 -- 172.21.15.94:0/1453107219 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c1f89d0 con 0x7f649c14f9b0 2024-09-10T23:38:24.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f649aa00640 1 --2- 172.21.15.94:0/1453107219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f649c14f9b0 0x7f649c14fdf0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f648800e600 tx=0x7f648800ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:24.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f6493400640 1 -- 172.21.15.94:0/1453107219 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488015020 con 0x7f649c14f9b0 2024-09-10T23:38:24.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f6493400640 1 -- 172.21.15.94:0/1453107219 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f648800c9e0 con 0x7f649c14f9b0 2024-09-10T23:38:24.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f64a2600640 1 -- 172.21.15.94:0/1453107219 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649c1f8cb0 con 0x7f649c14f9b0 2024-09-10T23:38:24.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f6493400640 1 -- 172.21.15.94:0/1453107219 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648800ce30 con 0x7f649c14f9b0 2024-09-10T23:38:24.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f64a2600640 1 -- 172.21.15.94:0/1453107219 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f649c1f9270 con 0x7f649c14f9b0 2024-09-10T23:38:24.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.465+0000 7f64a2600640 1 -- 172.21.15.94:0/1453107219 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f649c1f9470 con 0x7f649c14f9b0 2024-09-10T23:38:24.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.469+0000 7f6493400640 1 -- 172.21.15.94:0/1453107219 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f6488011460 con 0x7f649c14f9b0 2024-09-10T23:38:24.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.469+0000 7f6493400640 1 --2- 172.21.15.94:0/1453107219 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f647c072a90 0x7f647c074f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:24.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.469+0000 7f6493400640 1 -- 172.21.15.94:0/1453107219 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f64880943a0 con 0x7f649c14f9b0 2024-09-10T23:38:24.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.469+0000 7f649b400640 1 --2- 172.21.15.94:0/1453107219 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f647c072a90 0x7f647c074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:24.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.469+0000 7f649b400640 1 --2- 172.21.15.94:0/1453107219 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f647c072a90 0x7f647c074f50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f648c0016c0 tx=0x7f648c0b1040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:24.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.473+0000 7f6493400640 1 -- 172.21.15.94:0/1453107219 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f649c1f9470 con 0x7f649c14f9b0 2024-09-10T23:38:24.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.669+0000 7f64a2600640 1 -- 172.21.15.94:0/1453107219 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f649c0cfda0 con 0x7f649c14f9b0 2024-09-10T23:38:24.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.673+0000 7f6493400640 1 -- 172.21.15.94:0/1453107219 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5241 (secure 0 0 0) 0x7f648801f020 con 0x7f649c14f9b0 2024-09-10T23:38:24.674 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:24.674 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":78,"btime":"2024-09-10T23:38:22:381526+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7090,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3097381199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3097381199},{"type":"v1","addr":"172.21.15.181:6835","nonce":3097381199}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2654663117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2654663117},{"type":"v1","addr":"172.21.15.94:6833","nonce":2654663117}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7530,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1334394570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1334394570},{"type":"v1","addr":"172.21.15.94:6835","nonce":1334394570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:38:18.757602+0000","modified":"2024-09-10T23:38:21.765704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196283":{"gid":196283,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6834/1748173202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":1748173202},{"type":"v1","addr":"172.21.15.181:6834","nonce":1748173202}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196283,"qdb_cluster":[196283]},"id":9}]} 2024-09-10T23:38:24.674 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 78 2024-09-10T23:38:24.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.673+0000 7f64a2600640 1 -- 172.21.15.94:0/1453107219 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f647c072a90 msgr2=0x7f647c074f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.673+0000 7f64a2600640 1 --2- 172.21.15.94:0/1453107219 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f647c072a90 0x7f647c074f50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f648c0016c0 tx=0x7f648c0b1040 comp rx=0 tx=0).stop 2024-09-10T23:38:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.673+0000 7f64a2600640 1 -- 172.21.15.94:0/1453107219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f649c14f9b0 msgr2=0x7f649c14fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.673+0000 7f64a2600640 1 --2- 172.21.15.94:0/1453107219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f649c14f9b0 0x7f649c14fdf0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f648800e600 tx=0x7f648800ea00 comp rx=0 tx=0).stop 2024-09-10T23:38:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.673+0000 7f64a2600640 1 -- 172.21.15.94:0/1453107219 shutdown_connections 2024-09-10T23:38:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.673+0000 7f64a2600640 1 --2- 172.21.15.94:0/1453107219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649c152460 0x7f649c152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.673+0000 7f64a2600640 1 --2- 172.21.15.94:0/1453107219 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f647c072a90 0x7f647c074f50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.673+0000 7f64a2600640 1 --2- 172.21.15.94:0/1453107219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f649c14f9b0 0x7f649c14fdf0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.673+0000 7f64a2600640 1 --2- 172.21.15.94:0/1453107219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6480003a40 0x7f649c146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.673+0000 7f64a2600640 1 -- 172.21.15.94:0/1453107219 >> 172.21.15.94:0/1453107219 conn(0x7f649c06cdf0 msgr2=0x7f649c0b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.673+0000 7f64a2600640 1 -- 172.21.15.94:0/1453107219 shutdown_connections 2024-09-10T23:38:24.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:24.673+0000 7f64a2600640 1 -- 172.21.15.94:0/1453107219 wait complete. 2024-09-10T23:38:25.097 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T23:38:18.757602+0000', 'modified': '2024-09-10T23:38:21.765704+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 196283}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_196283': {'gid': 196283, 'name': 'b', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6834/1748173202', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6801', 'nonce': 1748173202}, {'type': 'v1', 'addr': '172.21.15.181:6834', 'nonce': 1748173202}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 196283, 'qdb_cluster': [196283]} 2024-09-10T23:38:25.097 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T23:38:25.097 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T23:38:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce8400640 1 -- 172.21.15.94:0/4017844335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce006e7c0 msgr2=0x7f2ce006ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce8400640 1 --2- 172.21.15.94:0/4017844335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce006e7c0 0x7f2ce006ec50 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800acd0 tx=0x7f2cd801def0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce8400640 1 -- 172.21.15.94:0/4017844335 shutdown_connections 2024-09-10T23:38:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce8400640 1 --2- 172.21.15.94:0/4017844335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce006e7c0 0x7f2ce006ec50 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce8400640 1 --2- 172.21.15.94:0/4017844335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ce00c64f0 0x7f2ce00cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce8400640 1 -- 172.21.15.94:0/4017844335 >> 172.21.15.94:0/4017844335 conn(0x7f2ce00cfaf0 msgr2=0x7f2ce00cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce8400640 1 -- 172.21.15.94:0/4017844335 shutdown_connections 2024-09-10T23:38:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce8400640 1 -- 172.21.15.94:0/4017844335 wait complete. 2024-09-10T23:38:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce8400640 1 Processor -- start 2024-09-10T23:38:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce8400640 1 -- start start 2024-09-10T23:38:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce8400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ce006e7c0 0x7f2ce0153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce8400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce00c64f0 0x7f2ce014af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce8400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ce00ce280 0x7f2ce00b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce8400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce015b530 con 0x7f2ce00c64f0 2024-09-10T23:38:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce8400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce015b3b0 con 0x7f2ce006e7c0 2024-09-10T23:38:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce8400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce015b6b0 con 0x7f2ce00ce280 2024-09-10T23:38:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce5000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce00c64f0 0x7f2ce014af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce5000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce00c64f0 0x7f2ce014af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48750/0 (socket says 172.21.15.94:48750) 2024-09-10T23:38:25.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce5000640 1 -- 172.21.15.94:0/22222271 learned_addr learned my addr 172.21.15.94:0/22222271 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:25.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce6400640 1 --2- 172.21.15.94:0/22222271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ce00ce280 0x7f2ce00b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.241+0000 7f2ce5a00640 1 --2- 172.21.15.94:0/22222271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ce006e7c0 0x7f2ce0153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.245+0000 7f2ce5000640 1 -- 172.21.15.94:0/22222271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ce00ce280 msgr2=0x7f2ce00b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:25.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.245+0000 7f2ce5000640 1 --2- 172.21.15.94:0/22222271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ce00ce280 0x7f2ce00b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.245+0000 7f2ce5000640 1 -- 172.21.15.94:0/22222271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ce006e7c0 msgr2=0x7f2ce0153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:25.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.245+0000 7f2ce5000640 1 --2- 172.21.15.94:0/22222271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ce006e7c0 0x7f2ce0153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.245+0000 7f2ce5000640 1 -- 172.21.15.94:0/22222271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce015a530 con 0x7f2ce00c64f0 2024-09-10T23:38:25.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.245+0000 7f2ce6400640 1 --2- 172.21.15.94:0/22222271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ce00ce280 0x7f2ce00b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:25.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.245+0000 7f2ce5a00640 1 --2- 172.21.15.94:0/22222271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ce006e7c0 0x7f2ce0153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:25.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.245+0000 7f2ce5000640 1 --2- 172.21.15.94:0/22222271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce00c64f0 0x7f2ce014af40 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00d950 tx=0x7f2ccc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:25.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.245+0000 7f2cdd600640 1 -- 172.21.15.94:0/22222271 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc017070 con 0x7f2ce00c64f0 2024-09-10T23:38:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.245+0000 7f2cdd600640 1 -- 172.21.15.94:0/22222271 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ccc009d20 con 0x7f2ce00c64f0 2024-09-10T23:38:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.245+0000 7f2ce8400640 1 -- 172.21.15.94:0/22222271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce0153a70 con 0x7f2ce00c64f0 2024-09-10T23:38:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.245+0000 7f2cdd600640 1 -- 172.21.15.94:0/22222271 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0106c0 con 0x7f2ce00c64f0 2024-09-10T23:38:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.245+0000 7f2ce8400640 1 -- 172.21.15.94:0/22222271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce0158ac0 con 0x7f2ce00c64f0 2024-09-10T23:38:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.245+0000 7f2cdd600640 1 -- 172.21.15.94:0/22222271 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f2ccc010860 con 0x7f2ce00c64f0 2024-09-10T23:38:25.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.245+0000 7f2cdd600640 1 --2- 172.21.15.94:0/22222271 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2cc0072a60 0x7f2cc0074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.245+0000 7f2cdd600640 1 -- 172.21.15.94:0/22222271 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f2ccc093f30 con 0x7f2ce00c64f0 2024-09-10T23:38:25.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ce5a00640 1 --2- 172.21.15.94:0/22222271 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2cc0072a60 0x7f2cc0074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 Processor -- start 2024-09-10T23:38:25.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 -- start start 2024-09-10T23:38:25.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ce5a00640 1 --2- 172.21.15.94:0/22222271 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2cc0072a60 0x7f2cc0074f20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4000f70 tx=0x7f2cd4009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:25.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f619d0 0x558b42f61db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f622f0 0x558b42f158d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f15e10 0x558b42f161f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558b42e363f0 con 0x558b42f622f0 2024-09-10T23:38:25.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558b42d73b20 con 0x558b42f15e10 2024-09-10T23:38:25.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x558b42d62f50 con 0x558b42f619d0 2024-09-10T23:38:25.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ce5000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f622f0 0x558b42f158d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ce5000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f622f0 0x558b42f158d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48762/0 (socket says 172.21.15.94:48762) 2024-09-10T23:38:25.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ce5000640 1 -- 172.21.15.94:0/1097372917 learned_addr learned my addr 172.21.15.94:0/1097372917 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:25.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ce5a00640 1 --2- 172.21.15.94:0/1097372917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f619d0 0x558b42f61db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ce6400640 1 --2- 172.21.15.94:0/1097372917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f15e10 0x558b42f161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ce5000640 1 -- 172.21.15.94:0/1097372917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f619d0 msgr2=0x558b42f61db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:38:25.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ce5000640 1 --2- 172.21.15.94:0/1097372917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f619d0 0x558b42f61db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ce5000640 1 -- 172.21.15.94:0/1097372917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f15e10 msgr2=0x558b42f161f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:38:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ce5000640 1 --2- 172.21.15.94:0/1097372917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f15e10 0x558b42f161f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ce5000640 1 -- 172.21.15.94:0/1097372917 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b42d2a590 con 0x558b42f622f0 2024-09-10T23:38:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ce6400640 1 --2- 172.21.15.94:0/1097372917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f15e10 0x558b42f161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:38:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ce5000640 1 --2- 172.21.15.94:0/1097372917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f622f0 0x558b42f158d0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc05e740 tx=0x7f2ccc05e2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befc0b722eba7cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2cd2000640 1 -- 172.21.15.94:0/1097372917 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc063030 con 0x558b42f622f0 2024-09-10T23:38:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2cd2000640 1 -- 172.21.15.94:0/1097372917 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ccc05e7b0 con 0x558b42f622f0 2024-09-10T23:38:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2cd2000640 1 -- 172.21.15.94:0/1097372917 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc020040 con 0x558b42f622f0 2024-09-10T23:38:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1097372917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f622f0 msgr2=0x558b42f158d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/1097372917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f622f0 0x558b42f158d0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc05e740 tx=0x7f2ccc05e2c0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1097372917 shutdown_connections 2024-09-10T23:38:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/1097372917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f15e10 0x558b42f161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/1097372917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f622f0 0x558b42f158d0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/1097372917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f619d0 0x558b42f61db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1097372917 >> 172.21.15.94:0/1097372917 conn(0x558b42dff130 msgr2=0x558b42f44e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1097372917 shutdown_connections 2024-09-10T23:38:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1097372917 wait complete. 2024-09-10T23:38:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.249+0000 7f2ceb7d0000 1 -- start start 2024-09-10T23:38:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 1 Processor -- start 2024-09-10T23:38:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f15e10 0x558b42f426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f619d0 0x558b42f42e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f622f0 0x558b42f433b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558b42cfd5d0 con 0x558b42f15e10 2024-09-10T23:38:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558b42d66d60 con 0x558b42f619d0 2024-09-10T23:38:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x558b42d09500 con 0x558b42f622f0 2024-09-10T23:38:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 10 client.0 _ll_register_callbacks cb 0x558b42d0d810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:38:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ce5a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f15e10 0x558b42f426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ce5a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f15e10 0x558b42f426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48776/0 (socket says 172.21.15.94:48776) 2024-09-10T23:38:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ce5a00640 1 -- 172.21.15.94:0/491024723 learned_addr learned my addr 172.21.15.94:0/491024723 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ce5000640 1 --2- 172.21.15.94:0/491024723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f619d0 0x558b42f42e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ce6400640 1 --2- 172.21.15.94:0/491024723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f622f0 0x558b42f433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ce5a00640 1 -- 172.21.15.94:0/491024723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f622f0 msgr2=0x558b42f433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:38:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ce5a00640 1 --2- 172.21.15.94:0/491024723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f622f0 0x558b42f433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ce5a00640 1 -- 172.21.15.94:0/491024723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f619d0 msgr2=0x558b42f42e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:38:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ce5a00640 1 --2- 172.21.15.94:0/491024723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f619d0 0x558b42f42e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ce5a00640 1 -- 172.21.15.94:0/491024723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b42d01170 con 0x558b42f15e10 2024-09-10T23:38:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ce5a00640 1 --2- 172.21.15.94:0/491024723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f15e10 0x558b42f426a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400ea30 tx=0x7f2cd4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7446ee9fe8d9d69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2cc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:38:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2cc6a00640 1 -- 172.21.15.94:0/491024723 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4027060 con 0x558b42f15e10 2024-09-10T23:38:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2cc6a00640 1 -- 172.21.15.94:0/491024723 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f2cd4010040 con 0x558b42f15e10 2024-09-10T23:38:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/491024723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558b42cfb0f0 con 0x558b42f15e10 2024-09-10T23:38:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2cc6a00640 1 -- 172.21.15.94:0/491024723 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f2cd40054a0 con 0x558b42f15e10 2024-09-10T23:38:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 10 client.7647 fetch_fsmap learned FSMap version 78 2024-09-10T23:38:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/491024723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x558b42de7ef0 con 0x558b42f15e10 2024-09-10T23:38:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2cc6a00640 1 -- 172.21.15.94:0/491024723 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x558b42cfb0f0 con 0x558b42f15e10 2024-09-10T23:38:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2cc6a00640 1 -- 172.21.15.94:0/491024723 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 78) ==== 3502+0+0 (secure 0 0 0) 0x7f2cd4013420 con 0x558b42f15e10 2024-09-10T23:38:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 10 client.7647 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T23:38:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 10 client.7647 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T23:38:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 20 client.7647 populate_metadata read hostname 'smithi094' 2024-09-10T23:38:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/491024723 >> [v2:172.21.15.181:6801/1748173202,v1:172.21.15.181:6834/1748173202] conn(0x558b42f440f0 0x558b42f73c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 4 client.7647 mds_command: new command op to 196283 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:38:25.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/491024723 --> [v2:172.21.15.181:6801/1748173202,v1:172.21.15.181:6834/1748173202] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b42d75a70 con 0x558b42f440f0 2024-09-10T23:38:25.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ce5000640 1 --2- 172.21.15.94:0/491024723 >> [v2:172.21.15.181:6801/1748173202,v1:172.21.15.181:6834/1748173202] conn(0x558b42f440f0 0x558b42f73c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2ce5000640 1 --2- 172.21.15.94:0/491024723 >> [v2:172.21.15.181:6801/1748173202,v1:172.21.15.181:6834/1748173202] conn(0x558b42f440f0 0x558b42f73c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9654e8236757aa82 server_cookie=3324e7f22b5c99b6 in_seq=0 out_seq=0 2024-09-10T23:38:25.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.253+0000 7f2cc6a00640 10 client.7647 ms_handle_connect on v2:172.21.15.181:6801/1748173202 2024-09-10T23:38:25.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.257+0000 7f2cc6a00640 1 -- 172.21.15.94:0/491024723 <== mds.0 v2:172.21.15.181:6801/1748173202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558b42d75a70 con 0x558b42f440f0 2024-09-10T23:38:25.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.257+0000 7f2cc6a00640 10 client.7647 handle_command_reply: tid=0 2024-09-10T23:38:25.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.257+0000 7f2ceb7d0000 1 client.7647 shutdown 2024-09-10T23:38:25.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.257+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/491024723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f15e10 msgr2=0x558b42f426a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:25.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.257+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/491024723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f15e10 0x558b42f426a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400ea30 tx=0x7f2cd4000ee0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.257+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/491024723 shutdown_connections 2024-09-10T23:38:25.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.257+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/491024723 >> [v2:172.21.15.181:6801/1748173202,v1:172.21.15.181:6834/1748173202] conn(0x558b42f440f0 0x558b42f73c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.257+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/491024723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f619d0 0x558b42f42e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.257+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/491024723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f622f0 0x558b42f433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.257+0000 7f2cc6a00640 0 client.7647 ms_handle_reset on v2:172.21.15.181:6801/1748173202 2024-09-10T23:38:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.257+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/491024723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f15e10 0x558b42f426a0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.257+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/491024723 >> 172.21.15.94:0/491024723 conn(0x558b42dff130 msgr2=0x558b42f14fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.257+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/491024723 shutdown_connections 2024-09-10T23:38:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.257+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/491024723 wait complete. 2024-09-10T23:38:25.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.257+0000 7f2ceb7d0000 20 client.7647 trim_cache size 0 max 16384 2024-09-10T23:38:25.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.273+0000 7f2ceb7d0000 1 Processor -- start 2024-09-10T23:38:25.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 -- start start 2024-09-10T23:38:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f15e10 0x558b42fdc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f619d0 0x558b42fdcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f622f0 0x558b42fdd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0203ca0 con 0x558b42f619d0 2024-09-10T23:38:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0202e60 con 0x558b42f15e10 2024-09-10T23:38:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce00af810 con 0x558b42f622f0 2024-09-10T23:38:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ce5000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f619d0 0x558b42fdcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ce5000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f619d0 0x558b42fdcd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48792/0 (socket says 172.21.15.94:48792) 2024-09-10T23:38:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ce5000640 1 -- 172.21.15.94:0/1013802817 learned_addr learned my addr 172.21.15.94:0/1013802817 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ce5000640 1 -- 172.21.15.94:0/1013802817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f622f0 msgr2=0x558b42fdd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:38:25.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ce5000640 1 --2- 172.21.15.94:0/1013802817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f622f0 0x558b42fdd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ce5000640 1 -- 172.21.15.94:0/1013802817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f15e10 msgr2=0x558b42fdc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:38:25.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ce5000640 1 --2- 172.21.15.94:0/1013802817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f15e10 0x558b42fdc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ce5000640 1 -- 172.21.15.94:0/1013802817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b42dbed90 con 0x558b42f619d0 2024-09-10T23:38:25.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ce5000640 1 --2- 172.21.15.94:0/1013802817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f619d0 0x558b42fdcd30 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc05e4b0 tx=0x7f2ccc01d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95db342a7d11390b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:25.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2cd1600640 1 -- 172.21.15.94:0/1013802817 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc08f070 con 0x558b42f619d0 2024-09-10T23:38:25.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2cd1600640 1 -- 172.21.15.94:0/1013802817 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ccc067cb0 con 0x558b42f619d0 2024-09-10T23:38:25.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2cd1600640 1 -- 172.21.15.94:0/1013802817 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc020040 con 0x558b42f619d0 2024-09-10T23:38:25.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1013802817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f619d0 msgr2=0x558b42fdcd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:25.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/1013802817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f619d0 0x558b42fdcd30 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc05e4b0 tx=0x7f2ccc01d070 comp rx=0 tx=0).stop 2024-09-10T23:38:25.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1013802817 shutdown_connections 2024-09-10T23:38:25.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/1013802817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f622f0 0x558b42fdd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/1013802817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f619d0 0x558b42fdcd30 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/1013802817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f15e10 0x558b42fdc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1013802817 >> 172.21.15.94:0/1013802817 conn(0x558b42dff130 msgr2=0x558b42f1d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:25.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1013802817 shutdown_connections 2024-09-10T23:38:25.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1013802817 wait complete. 2024-09-10T23:38:25.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 -- start start 2024-09-10T23:38:25.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.277+0000 7f2ceb7d0000 1 Processor -- start 2024-09-10T23:38:25.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f15e10 0x558b42fe0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f619d0 0x558b42fe0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f622f0 0x558b42fe12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce00afe60 con 0x558b42f15e10 2024-09-10T23:38:25.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558b42cfd5d0 con 0x558b42f619d0 2024-09-10T23:38:25.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x558b42f78ce0 con 0x558b42f622f0 2024-09-10T23:38:25.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 10 client.0 _ll_register_callbacks cb 0x7f2cd40013b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:38:25.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ce5a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f15e10 0x558b42fe0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ce5a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f15e10 0x558b42fe0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48796/0 (socket says 172.21.15.94:48796) 2024-09-10T23:38:25.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ce5a00640 1 -- 172.21.15.94:0/1645171280 learned_addr learned my addr 172.21.15.94:0/1645171280 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:25.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ce5000640 1 --2- 172.21.15.94:0/1645171280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f619d0 0x558b42fe0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ce6400640 1 --2- 172.21.15.94:0/1645171280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f622f0 0x558b42fe12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ce5a00640 1 -- 172.21.15.94:0/1645171280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f622f0 msgr2=0x558b42fe12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:38:25.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ce5a00640 1 --2- 172.21.15.94:0/1645171280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f622f0 0x558b42fe12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ce5a00640 1 -- 172.21.15.94:0/1645171280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f619d0 msgr2=0x558b42fe0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:38:25.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ce5a00640 1 --2- 172.21.15.94:0/1645171280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f619d0 0x558b42fe0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ce5a00640 1 -- 172.21.15.94:0/1645171280 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b42d29a90 con 0x558b42f15e10 2024-09-10T23:38:25.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ce5000640 1 --2- 172.21.15.94:0/1645171280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f619d0 0x558b42fe0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:38:25.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ce6400640 1 --2- 172.21.15.94:0/1645171280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f622f0 0x558b42fe12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:38:25.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ce5a00640 1 --2- 172.21.15.94:0/1645171280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f15e10 0x558b42fe0830 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009010 tx=0x7f2cd4005430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5afd4337e6c78f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:25.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2cc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:38:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2cc6a00640 1 -- 172.21.15.94:0/1645171280 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4013970 con 0x558b42f15e10 2024-09-10T23:38:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2cc6a00640 1 -- 172.21.15.94:0/1645171280 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f2cd4027840 con 0x558b42f15e10 2024-09-10T23:38:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1645171280 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ccc067cb0 con 0x558b42f15e10 2024-09-10T23:38:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2cc6a00640 1 -- 172.21.15.94:0/1645171280 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f2cd4013b10 con 0x558b42f15e10 2024-09-10T23:38:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 10 client.7653 fetch_fsmap learned FSMap version 78 2024-09-10T23:38:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1645171280 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x558b42d08050 con 0x558b42f15e10 2024-09-10T23:38:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2cc6a00640 1 -- 172.21.15.94:0/1645171280 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ccc067cb0 con 0x558b42f15e10 2024-09-10T23:38:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2cc6a00640 1 -- 172.21.15.94:0/1645171280 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 78) ==== 3502+0+0 (secure 0 0 0) 0x7f2cd4027a60 con 0x558b42f15e10 2024-09-10T23:38:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 10 client.7653 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T23:38:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 10 client.7653 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T23:38:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 20 client.7653 populate_metadata read hostname 'smithi094' 2024-09-10T23:38:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/1645171280 >> [v2:172.21.15.181:6801/1748173202,v1:172.21.15.181:6834/1748173202] conn(0x558b42f76b60 0x558b42f75ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 4 client.7653 mds_command: new command op to 196283 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T23:38:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1645171280 --> [v2:172.21.15.181:6801/1748173202,v1:172.21.15.181:6834/1748173202] -- command(tid 0: {"prefix": "status"}) -- 0x558b42d71520 con 0x558b42f76b60 2024-09-10T23:38:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ce5000640 1 --2- 172.21.15.94:0/1645171280 >> [v2:172.21.15.181:6801/1748173202,v1:172.21.15.181:6834/1748173202] conn(0x558b42f76b60 0x558b42f75ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2cc6a00640 10 client.7653 ms_handle_connect on v2:172.21.15.181:6801/1748173202 2024-09-10T23:38:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ce5000640 1 --2- 172.21.15.94:0/1645171280 >> [v2:172.21.15.181:6801/1748173202,v1:172.21.15.181:6834/1748173202] conn(0x558b42f76b60 0x558b42f75ca0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bfb93e03365b349 server_cookie=bb79f2bf987b38ff in_seq=0 out_seq=0 2024-09-10T23:38:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2cc6a00640 1 -- 172.21.15.94:0/1645171280 <== mds.0 v2:172.21.15.181:6801/1748173202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x558b42d71520 con 0x558b42f76b60 2024-09-10T23:38:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2cc6a00640 10 client.7653 handle_command_reply: tid=0 2024-09-10T23:38:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 client.7653 shutdown 2024-09-10T23:38:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1645171280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f15e10 msgr2=0x558b42fe0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/1645171280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f15e10 0x558b42fe0830 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009010 tx=0x7f2cd4005430 comp rx=0 tx=0).stop 2024-09-10T23:38:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1645171280 shutdown_connections 2024-09-10T23:38:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/1645171280 >> [v2:172.21.15.181:6801/1748173202,v1:172.21.15.181:6834/1748173202] conn(0x558b42f76b60 0x558b42f75ca0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/1645171280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558b42f619d0 0x558b42fe0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/1645171280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558b42f622f0 0x558b42fe12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 --2- 172.21.15.94:0/1645171280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b42f15e10 0x558b42fe0830 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2cc6a00640 0 client.7653 ms_handle_reset on v2:172.21.15.181:6801/1748173202 2024-09-10T23:38:25.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1645171280 >> 172.21.15.94:0/1645171280 conn(0x558b42dff130 msgr2=0x558b42f1db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:25.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1645171280 shutdown_connections 2024-09-10T23:38:25.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 1 -- 172.21.15.94:0/1645171280 wait complete. 2024-09-10T23:38:25.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.281+0000 7f2ceb7d0000 20 client.7653 trim_cache size 0 max 16384 2024-09-10T23:38:25.290 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:38:25.290 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "1798cdb9-9937-4918-a569-86b41a92817e", 2024-09-10T23:38:25.290 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-09-10T23:38:25.290 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 196283, 2024-09-10T23:38:25.290 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-10T23:38:25.290 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-10T23:38:25.291 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-10T23:38:25.291 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 5.9120620410000004, 2024-09-10T23:38:25.291 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 78, 2024-09-10T23:38:25.291 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 114, 2024-09-10T23:38:25.291 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 114, 2024-09-10T23:38:25.291 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 10.421179292 2024-09-10T23:38:25.291 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:38:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.285+0000 7f2ce8400640 1 -- 172.21.15.94:0/22222271 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2cc0072a60 msgr2=0x7f2cc0074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.285+0000 7f2ce8400640 1 --2- 172.21.15.94:0/22222271 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2cc0072a60 0x7f2cc0074f20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4000f70 tx=0x7f2cd4009040 comp rx=0 tx=0).stop 2024-09-10T23:38:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.285+0000 7f2ce8400640 1 -- 172.21.15.94:0/22222271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce00c64f0 msgr2=0x7f2ce014af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.285+0000 7f2ce8400640 1 --2- 172.21.15.94:0/22222271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce00c64f0 0x7f2ce014af40 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00d950 tx=0x7f2ccc00de10 comp rx=0 tx=0).stop 2024-09-10T23:38:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.285+0000 7f2ce8400640 1 -- 172.21.15.94:0/22222271 shutdown_connections 2024-09-10T23:38:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.285+0000 7f2ce8400640 1 --2- 172.21.15.94:0/22222271 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2cc0072a60 0x7f2cc0074f20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.285+0000 7f2ce8400640 1 --2- 172.21.15.94:0/22222271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ce00ce280 0x7f2ce00b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.285+0000 7f2ce8400640 1 --2- 172.21.15.94:0/22222271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ce00c64f0 0x7f2ce014af40 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.285+0000 7f2ce8400640 1 --2- 172.21.15.94:0/22222271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ce006e7c0 0x7f2ce0153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.285+0000 7f2ce8400640 1 -- 172.21.15.94:0/22222271 >> 172.21.15.94:0/22222271 conn(0x7f2ce00cfaf0 msgr2=0x7f2ce0072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:25.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.285+0000 7f2ce8400640 1 -- 172.21.15.94:0/22222271 shutdown_connections 2024-09-10T23:38:25.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.285+0000 7f2ce8400640 1 -- 172.21.15.94:0/22222271 wait complete. 2024-09-10T23:38:25.305 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:38:25.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 -- 172.21.15.94:0/2553339723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8c15ecb0 msgr2=0x7faa8c0b35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:25.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 --2- 172.21.15.94:0/2553339723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8c15ecb0 0x7faa8c0b35f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0d0b80 tx=0x7faa8401cd50 comp rx=0 tx=0).stop 2024-09-10T23:38:25.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 -- 172.21.15.94:0/2553339723 shutdown_connections 2024-09-10T23:38:25.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 --2- 172.21.15.94:0/2553339723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8c15ecb0 0x7faa8c0b35f0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 --2- 172.21.15.94:0/2553339723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa8c0d02e0 0x7faa8c15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 --2- 172.21.15.94:0/2553339723 >> v2:172.21.15.181:3300/0 conn(0x7faa8c073ff0 0x7faa8c0cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 -- 172.21.15.94:0/2553339723 >> 172.21.15.94:0/2553339723 conn(0x7faa8c06cdf0 msgr2=0x7faa8c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:25.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 -- 172.21.15.94:0/2553339723 shutdown_connections 2024-09-10T23:38:25.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 -- 172.21.15.94:0/2553339723 wait complete. 2024-09-10T23:38:25.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 Processor -- start 2024-09-10T23:38:25.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 -- start start 2024-09-10T23:38:25.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8c073ff0 0x7faa8c0b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa8c0d02e0 0x7faa8c1f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa8c15ecb0 0x7faa8c159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c06f840 con 0x7faa8c073ff0 2024-09-10T23:38:25.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c06f6c0 con 0x7faa8c0d02e0 2024-09-10T23:38:25.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa94000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7faa8c06f9c0 con 0x7faa8c15ecb0 2024-09-10T23:38:25.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa91600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8c073ff0 0x7faa8c0b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa91600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8c073ff0 0x7faa8c0b1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48814/0 (socket says 172.21.15.94:48814) 2024-09-10T23:38:25.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa91600640 1 -- 172.21.15.94:0/1726525160 learned_addr learned my addr 172.21.15.94:0/1726525160 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:25.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa90c00640 1 --2- 172.21.15.94:0/1726525160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa8c0d02e0 0x7faa8c1f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa92000640 1 --2- 172.21.15.94:0/1726525160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa8c15ecb0 0x7faa8c159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa91600640 1 -- 172.21.15.94:0/1726525160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa8c15ecb0 msgr2=0x7faa8c159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:25.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa91600640 1 --2- 172.21.15.94:0/1726525160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa8c15ecb0 0x7faa8c159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa91600640 1 -- 172.21.15.94:0/1726525160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa8c0d02e0 msgr2=0x7faa8c1f69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:25.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa91600640 1 --2- 172.21.15.94:0/1726525160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa8c0d02e0 0x7faa8c1f69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa91600640 1 -- 172.21.15.94:0/1726525160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8c1643c0 con 0x7faa8c073ff0 2024-09-10T23:38:25.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.449+0000 7faa90c00640 1 --2- 172.21.15.94:0/1726525160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa8c0d02e0 0x7faa8c1f69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:25.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.453+0000 7faa91600640 1 --2- 172.21.15.94:0/1726525160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8c073ff0 0x7faa8c0b1270 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7faa8000c940 tx=0x7faa8000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:25.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.453+0000 7faa89600640 1 -- 172.21.15.94:0/1726525160 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80007bb0 con 0x7faa8c073ff0 2024-09-10T23:38:25.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.453+0000 7faa89600640 1 -- 172.21.15.94:0/1726525160 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa8000f450 con 0x7faa8c073ff0 2024-09-10T23:38:25.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.453+0000 7faa94000640 1 -- 172.21.15.94:0/1726525160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8c1646a0 con 0x7faa8c073ff0 2024-09-10T23:38:25.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.453+0000 7faa89600640 1 -- 172.21.15.94:0/1726525160 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80007e60 con 0x7faa8c073ff0 2024-09-10T23:38:25.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.453+0000 7faa94000640 1 -- 172.21.15.94:0/1726525160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa8c154fe0 con 0x7faa8c073ff0 2024-09-10T23:38:25.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.453+0000 7faa94000640 1 -- 172.21.15.94:0/1726525160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa8c151950 con 0x7faa8c073ff0 2024-09-10T23:38:25.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.453+0000 7faa89600640 1 -- 172.21.15.94:0/1726525160 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7faa80020030 con 0x7faa8c073ff0 2024-09-10T23:38:25.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.453+0000 7faa89600640 1 --2- 172.21.15.94:0/1726525160 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa6c072a60 0x7faa6c074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:25.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.453+0000 7faa89600640 1 -- 172.21.15.94:0/1726525160 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7faa80094140 con 0x7faa8c073ff0 2024-09-10T23:38:25.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.453+0000 7faa90c00640 1 --2- 172.21.15.94:0/1726525160 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa6c072a60 0x7faa6c074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:25.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.457+0000 7faa90c00640 1 --2- 172.21.15.94:0/1726525160 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa6c072a60 0x7faa6c074f20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0b2370 tx=0x7faa78005ed0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:25.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.457+0000 7faa89600640 1 -- 172.21.15.94:0/1726525160 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faa80014030 con 0x7faa8c073ff0 2024-09-10T23:38:25.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.657+0000 7faa94000640 1 -- 172.21.15.94:0/1726525160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faa8c069850 con 0x7faa8c073ff0 2024-09-10T23:38:25.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.657+0000 7faa89600640 1 -- 172.21.15.94:0/1726525160 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) ==== 53+0+73 (secure 0 0 0) 0x7faa8005f0c0 con 0x7faa8c073ff0 2024-09-10T23:38:25.661 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:38:25.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.661+0000 7faa94000640 1 -- 172.21.15.94:0/1726525160 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa6c072a60 msgr2=0x7faa6c074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:25.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.661+0000 7faa94000640 1 --2- 172.21.15.94:0/1726525160 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa6c072a60 0x7faa6c074f20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0b2370 tx=0x7faa78005ed0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.661+0000 7faa94000640 1 -- 172.21.15.94:0/1726525160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8c073ff0 msgr2=0x7faa8c0b1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:25.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.661+0000 7faa94000640 1 --2- 172.21.15.94:0/1726525160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8c073ff0 0x7faa8c0b1270 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7faa8000c940 tx=0x7faa8000ce00 comp rx=0 tx=0).stop 2024-09-10T23:38:25.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.661+0000 7faa94000640 1 -- 172.21.15.94:0/1726525160 shutdown_connections 2024-09-10T23:38:25.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.661+0000 7faa94000640 1 --2- 172.21.15.94:0/1726525160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa8c0d02e0 0x7faa8c1f69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.661+0000 7faa94000640 1 --2- 172.21.15.94:0/1726525160 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa6c072a60 0x7faa6c074f20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.661+0000 7faa94000640 1 --2- 172.21.15.94:0/1726525160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa8c15ecb0 0x7faa8c159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.661+0000 7faa94000640 1 --2- 172.21.15.94:0/1726525160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa8c073ff0 0x7faa8c0b1270 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:25.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.661+0000 7faa94000640 1 -- 172.21.15.94:0/1726525160 >> 172.21.15.94:0/1726525160 conn(0x7faa8c06cdf0 msgr2=0x7faa8c14da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:25.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.661+0000 7faa94000640 1 -- 172.21.15.94:0/1726525160 shutdown_connections 2024-09-10T23:38:25.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25.661+0000 7faa94000640 1 -- 172.21.15.94:0/1726525160 wait complete. 2024-09-10T23:38:25.681 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:38:25.682 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:38:25.682 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T23:38:25.682 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T23:38:25.682 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T23:38:25.682 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:38:25.682 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:38:25.682 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:38:25.682 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:38:25.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:25.682 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T23:38:25.691 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:38:25.692 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:38:25.692 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:38:25.692 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:25.692 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T23:38:25.692 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:25.692 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:38:25.692 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:25.692 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T23:38:25.692 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:38:25.692 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:25.692 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T23:38:25.692 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:38:25.693 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:25.693 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:38:25.693 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:25.693 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:38:25.693 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42105sec preferred_lft 42105sec 2024-09-10T23:38:25.693 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-09-10T23:38:25.693 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:25.693 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:38:25.693 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:ed:7d:46:db brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:25.693 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:38:25.693 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:25.693 INFO:teuthology.orchestra.run.smithi094.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:38:25.693 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:25.694 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:38:25.694 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:25.694 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:38:25.694 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:25.694 INFO:teuthology.orchestra.run.smithi094.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:38:25.694 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:38:25.694 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:38:25.694 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:25.694 INFO:teuthology.orchestra.run.smithi094.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:38:25.694 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:38:25.694 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:38:25.694 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:25.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:25.695 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:38:25.750 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T23:38:25.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:25.752 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T23:38:25.804 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T23:38:25.804 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:38:25.804 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:38:25.805 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:38:25.805 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:38:25.805 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:38:25.805 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:38:25.805 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:38:25.805 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:38:25.805 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:38:25.805 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:38:25.805 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:38:25.805 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:38:25.806 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:38:25.806 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:38:25.806 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:38:25.806 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:38:25.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:38:25.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:38:25.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:25.942 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T23:38:25.951 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T23:38:25.951 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T23:38:25.952 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:38:25.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:25.952 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-09-10T23:38:26.016 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T23:38:26.017 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:38:26.017 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T23:38:26.065 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:38:26.065 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:38:26.065 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:38:26.116 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:38:26.180 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:38:26.230 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:38:26.230 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:38:26.230 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:38:26.230 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:38:26.230 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:38:26.231 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:38:26.231 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:38:26.231 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:38:26.231 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:38:26.231 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:38:26.231 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:38:26.231 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:38:26.231 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:38:26.231 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:38:26.231 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:38:26.231 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:38:26.231 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:38:26.231 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:38:26.231 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:38:26.232 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:38:26.232 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:38:26.232 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:38:26.232 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:38:26.232 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:38:26.232 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:38:26.232 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:38:26.232 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:38:26.232 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:38:26.232 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:38:26.232 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:38:26.232 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:38:26.232 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:38:26.233 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:38:26.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:26.234 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:38:26.284 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:38:26.285 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T23:38:26.331 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:38:26.352 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:38:26.406 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:38:26.407 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:38:26.407 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:38:26.407 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:38:26.407 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:38:26.407 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:38:26.407 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:38:26.407 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:38:26.407 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:38:26.407 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:38:26.407 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:38:26.407 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:38:26.407 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:38:26.407 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:38:26.408 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:38:26.408 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:38:26.408 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:38:26.408 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:38:26.408 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:38:26.408 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:38:26.408 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:38:26.408 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:38:26.408 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:38:26.408 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:38:26.408 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:38:26.408 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:38:26.408 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:38:26.409 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:38:26.409 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:38:26.409 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:38:26.409 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:38:26.409 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:38:26.409 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:38:26.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:26.410 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:38:26.427 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:2024-09-10T23:38:26.425+0000 7f9cd8803580 -1 init, newargv = 0x558995e2dbc0 newargc=15 2024-09-10T23:38:26.428 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[47067]: starting ceph client 2024-09-10T23:38:26.434 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[47067]: starting fuse 2024-09-10T23:38:26.436 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T23:38:26.436 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:38:26.436 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:38:26.436 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-09-10T23:38:26.437 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-09-10T23:38:26.437 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-09-10T23:38:26.437 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-09-10T23:38:26.437 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:38:26.437 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-09-10T23:38:26.437 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T23:38:26.437 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-09-10T23:38:26.437 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T23:38:26.437 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:38:26.437 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-09-10T23:38:26.437 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-09-10T23:38:26.437 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-09-10T23:38:26.437 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-09-10T23:38:26.437 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:38:26.437 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-09-10T23:38:26.438 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:38:26.438 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:38:26.438 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:38:26.438 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-09-10T23:38:26.438 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-09-10T23:38:26.438 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-09-10T23:38:26.438 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:38:26.438 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:38:26.439 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.0")) 2024-09-10T23:38:26.439 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-09-10T23:38:26.560 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.0.47067.asok 2024-09-10T23:38:26.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:38:26.572 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47067.asok 2024-09-10T23:38:26.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:26.572 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47067.asok status 2024-09-10T23:38:26.739 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:38:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-09-10T23:38:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:38:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:38:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "0", 2024-09-10T23:38:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-09-10T23:38:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T23:38:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "47067", 2024-09-10T23:38:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-09-10T23:38:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-09-10T23:38:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-09-10T23:38:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 7668, 2024-09-10T23:38:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-09-10T23:38:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-09-10T23:38:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-09-10T23:38:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 7668 2024-09-10T23:38:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:38:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:38:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:38:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3940112033 2024-09-10T23:38:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:38:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:38:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:38:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3940112033 2024-09-10T23:38:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.7668 192.168.0.1:0/3940112033", 2024-09-10T23:38:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/3940112033", 2024-09-10T23:38:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-09-10T23:38:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 78, 2024-09-10T23:38:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 114, 2024-09-10T23:38:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:38:26.744 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-09-10T23:38:26.744 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-09-10T23:38:26.744 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:38:26.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:26.754 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-09-10T23:38:26.754 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T23:38:26.763 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-09-10T23:38:26.763 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T23:38:26.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:26.764 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:38:26.830 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:38:27.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.013+0000 7efc73c00640 1 -- 172.21.15.94:0/1241821684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc6c157c60 msgr2=0x7efc6c072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:27.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.013+0000 7efc73c00640 1 --2- 172.21.15.94:0/1241821684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc6c157c60 0x7efc6c072610 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7efc6c0cc090 tx=0x7efc6001c830 comp rx=0 tx=0).stop 2024-09-10T23:38:27.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc73c00640 1 -- 172.21.15.94:0/1241821684 shutdown_connections 2024-09-10T23:38:27.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc73c00640 1 --2- 172.21.15.94:0/1241821684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc6c06eef0 0x7efc6c06f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:27.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc73c00640 1 --2- 172.21.15.94:0/1241821684 >> v2:172.21.15.181:3300/0 conn(0x7efc6c070150 0x7efc6c0b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:27.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc73c00640 1 --2- 172.21.15.94:0/1241821684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc6c157c60 0x7efc6c072610 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:27.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc73c00640 1 -- 172.21.15.94:0/1241821684 >> 172.21.15.94:0/1241821684 conn(0x7efc6c06cdf0 msgr2=0x7efc6c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:27.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc73c00640 1 -- 172.21.15.94:0/1241821684 shutdown_connections 2024-09-10T23:38:27.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc73c00640 1 -- 172.21.15.94:0/1241821684 wait complete. 2024-09-10T23:38:27.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc73c00640 1 Processor -- start 2024-09-10T23:38:27.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc73c00640 1 -- start start 2024-09-10T23:38:27.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc73c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc6c06eef0 0x7efc6c14b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:27.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc73c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc6c070150 0x7efc6c165530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:27.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc73c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc6c157c60 0x7efc6c0cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:27.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc73c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c15b050 con 0x7efc6c06eef0 2024-09-10T23:38:27.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc73c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c15aed0 con 0x7efc6c070150 2024-09-10T23:38:27.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc73c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7efc6c15b1d0 con 0x7efc6c157c60 2024-09-10T23:38:27.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc71200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc6c06eef0 0x7efc6c14b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:27.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc71200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc6c06eef0 0x7efc6c14b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48844/0 (socket says 172.21.15.94:48844) 2024-09-10T23:38:27.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc71200640 1 -- 172.21.15.94:0/2917059075 learned_addr learned my addr 172.21.15.94:0/2917059075 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:27.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc6be00640 1 --2- 172.21.15.94:0/2917059075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc6c070150 0x7efc6c165530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:27.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc71c00640 1 --2- 172.21.15.94:0/2917059075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc6c157c60 0x7efc6c0cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:27.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc71200640 1 -- 172.21.15.94:0/2917059075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc6c157c60 msgr2=0x7efc6c0cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:27.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc71200640 1 --2- 172.21.15.94:0/2917059075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc6c157c60 0x7efc6c0cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:27.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc71200640 1 -- 172.21.15.94:0/2917059075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc6c070150 msgr2=0x7efc6c165530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:27.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc71200640 1 --2- 172.21.15.94:0/2917059075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc6c070150 0x7efc6c165530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:27.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc71200640 1 -- 172.21.15.94:0/2917059075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6c15bda0 con 0x7efc6c06eef0 2024-09-10T23:38:27.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc6be00640 1 --2- 172.21.15.94:0/2917059075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc6c070150 0x7efc6c165530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:27.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc71200640 1 --2- 172.21.15.94:0/2917059075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc6c06eef0 0x7efc6c14b170 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7efc600b0040 tx=0x7efc600a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:27.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc68c00640 1 -- 172.21.15.94:0/2917059075 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc600b2040 con 0x7efc6c06eef0 2024-09-10T23:38:27.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc68c00640 1 -- 172.21.15.94:0/2917059075 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc600043b0 con 0x7efc6c06eef0 2024-09-10T23:38:27.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc68c00640 1 -- 172.21.15.94:0/2917059075 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60003860 con 0x7efc6c06eef0 2024-09-10T23:38:27.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.017+0000 7efc73c00640 1 -- 172.21.15.94:0/2917059075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6c14bcd0 con 0x7efc6c06eef0 2024-09-10T23:38:27.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.021+0000 7efc73c00640 1 -- 172.21.15.94:0/2917059075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6c1f8fa0 con 0x7efc6c06eef0 2024-09-10T23:38:27.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.021+0000 7efc73c00640 1 -- 172.21.15.94:0/2917059075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc6c0b35c0 con 0x7efc6c06eef0 2024-09-10T23:38:27.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.021+0000 7efc68c00640 1 -- 172.21.15.94:0/2917059075 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7efc600c0060 con 0x7efc6c06eef0 2024-09-10T23:38:27.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.021+0000 7efc68c00640 1 --2- 172.21.15.94:0/2917059075 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efc4c072a60 0x7efc4c074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:27.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.021+0000 7efc68c00640 1 -- 172.21.15.94:0/2917059075 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7efc6012d2f0 con 0x7efc6c06eef0 2024-09-10T23:38:27.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.021+0000 7efc6be00640 1 --2- 172.21.15.94:0/2917059075 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efc4c072a60 0x7efc4c074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:27.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.025+0000 7efc6be00640 1 --2- 172.21.15.94:0/2917059075 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efc4c072a60 0x7efc4c074f20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efc58005fd0 tx=0x7efc5800a5e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:27.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.025+0000 7efc68c00640 1 -- 172.21.15.94:0/2917059075 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efc6c14bcd0 con 0x7efc6c06eef0 2024-09-10T23:38:27.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.225+0000 7efc73c00640 1 -- 172.21.15.94:0/2917059075 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efc6c06e7e0 con 0x7efc6c06eef0 2024-09-10T23:38:27.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.225+0000 7efc68c00640 1 -- 172.21.15.94:0/2917059075 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) ==== 53+0+73 (secure 0 0 0) 0x7efc600164b0 con 0x7efc6c06eef0 2024-09-10T23:38:27.228 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:38:27.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.229+0000 7efc73c00640 1 -- 172.21.15.94:0/2917059075 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efc4c072a60 msgr2=0x7efc4c074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:27.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.229+0000 7efc73c00640 1 --2- 172.21.15.94:0/2917059075 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efc4c072a60 0x7efc4c074f20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efc58005fd0 tx=0x7efc5800a5e0 comp rx=0 tx=0).stop 2024-09-10T23:38:27.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.229+0000 7efc73c00640 1 -- 172.21.15.94:0/2917059075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc6c06eef0 msgr2=0x7efc6c14b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:27.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.229+0000 7efc73c00640 1 --2- 172.21.15.94:0/2917059075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc6c06eef0 0x7efc6c14b170 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7efc600b0040 tx=0x7efc600a55a0 comp rx=0 tx=0).stop 2024-09-10T23:38:27.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.229+0000 7efc73c00640 1 -- 172.21.15.94:0/2917059075 shutdown_connections 2024-09-10T23:38:27.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.229+0000 7efc73c00640 1 --2- 172.21.15.94:0/2917059075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc6c070150 0x7efc6c165530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:27.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.229+0000 7efc73c00640 1 --2- 172.21.15.94:0/2917059075 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efc4c072a60 0x7efc4c074f20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:27.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.229+0000 7efc73c00640 1 --2- 172.21.15.94:0/2917059075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc6c157c60 0x7efc6c0cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:27.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.229+0000 7efc73c00640 1 --2- 172.21.15.94:0/2917059075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc6c06eef0 0x7efc6c14b170 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:27.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.229+0000 7efc73c00640 1 -- 172.21.15.94:0/2917059075 >> 172.21.15.94:0/2917059075 conn(0x7efc6c06cdf0 msgr2=0x7efc6c15b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:27.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.229+0000 7efc73c00640 1 -- 172.21.15.94:0/2917059075 shutdown_connections 2024-09-10T23:38:27.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:27.229+0000 7efc73c00640 1 -- 172.21.15.94:0/2917059075 wait complete. 2024-09-10T23:38:27.249 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:38:27.254 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:38:27.254 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-09-10T23:38:27.254 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T23:38:27.254 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T23:38:27.254 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:38:27.254 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:38:27.254 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:38:27.254 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:38:27.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:27.255 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-09-10T23:38:27.265 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:38:27.265 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:38:27.265 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:38:27.265 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:27.265 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-09-10T23:38:27.265 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:27.265 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:38:27.265 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:27.266 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-09-10T23:38:27.266 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:38:27.266 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:27.266 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-09-10T23:38:27.266 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:38:27.266 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:27.266 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:38:27.266 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:27.266 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:38:27.266 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42108sec preferred_lft 42108sec 2024-09-10T23:38:27.266 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-09-10T23:38:27.266 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:27.266 INFO:teuthology.orchestra.run.smithi181.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:38:27.266 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 02:42:3b:2e:09:2d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:27.267 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:38:27.267 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:27.267 INFO:teuthology.orchestra.run.smithi181.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:38:27.267 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:27.267 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:38:27.267 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:27.267 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:38:27.267 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:27.267 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:38:27.267 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:38:27.267 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:38:27.267 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:27.267 INFO:teuthology.orchestra.run.smithi181.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:38:27.267 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:38:27.268 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:38:27.268 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:27.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:27.269 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:38:27.332 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-09-10T23:38:27.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:27.334 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-09-10T23:38:27.393 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-09-10T23:38:27.393 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:38:27.393 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:38:27.393 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:38:27.393 INFO:teuthology.orchestra.run.smithi181.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:38:27.394 INFO:teuthology.orchestra.run.smithi181.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:38:27.394 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:38:27.394 INFO:teuthology.orchestra.run.smithi181.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:38:27.394 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:38:27.394 INFO:teuthology.orchestra.run.smithi181.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:38:27.394 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:38:27.394 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:38:27.394 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:38:27.394 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:38:27.394 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:38:27.394 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:38:27.395 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:38:27.489 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:38:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:38:27.530 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:38:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:38:27.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:27.537 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-09-10T23:38:27.546 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T23:38:27.546 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T23:38:27.547 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:38:27.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:27.547 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-09-10T23:38:27.608 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T23:38:27.608 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:38:27.608 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T23:38:27.661 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:38:27.662 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:38:27.662 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:38:27.716 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:38:27.776 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:38:27.830 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:38:27.830 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:38:27.830 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:38:27.830 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:38:27.830 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:38:27.830 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:38:27.830 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:38:27.830 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:38:27.830 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:38:27.830 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:38:27.830 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:38:27.830 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:38:27.830 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:38:27.831 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:38:27.831 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:38:27.831 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:38:27.831 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:38:27.831 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:38:27.831 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:38:27.831 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:38:27.831 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:38:27.831 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:38:27.831 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:38:27.831 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:38:27.831 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:38:27.831 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:38:27.831 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:38:27.832 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:38:27.832 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:38:27.832 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:38:27.832 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:38:27.832 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:38:27.832 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:38:27.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:27.833 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:38:27.888 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:38:27.888 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T23:38:27.939 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:38:27.959 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:38:28.011 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:38:28.011 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:38:28.011 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:38:28.011 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:38:28.011 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:38:28.011 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:38:28.011 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:38:28.012 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:38:28.012 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:38:28.012 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:38:28.012 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:38:28.012 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:38:28.012 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:38:28.012 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:38:28.012 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:38:28.012 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:38:28.012 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:38:28.012 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:38:28.013 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:38:28.013 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:38:28.013 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:38:28.013 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:38:28.013 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:38:28.013 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:38:28.013 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:38:28.013 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:38:28.013 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:38:28.013 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:38:28.013 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:38:28.013 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:38:28.013 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:38:28.014 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:38:28.014 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:38:28.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:28.015 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:38:28.033 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:2024-09-10T23:38:28.033+0000 7fa06c474580 -1 init, newargv = 0x562cccee0bc0 newargc=15 2024-09-10T23:38:28.037 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[35684]: starting ceph client 2024-09-10T23:38:28.040 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[35684]: starting fuse 2024-09-10T23:38:28.042 INFO:teuthology.orchestra.run.smithi181.stdout:44 2024-09-10T23:38:28.043 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:38:28.043 DEBUG:teuthology.orchestra.run.smithi181:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:38:28.043 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2024-09-10T23:38:28.044 DEBUG:teuthology.orchestra.run.smithi181:> import re 2024-09-10T23:38:28.044 DEBUG:teuthology.orchestra.run.smithi181:> import os 2024-09-10T23:38:28.044 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2024-09-10T23:38:28.044 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:38:28.044 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2024-09-10T23:38:28.044 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T23:38:28.044 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2024-09-10T23:38:28.045 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T23:38:28.045 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:38:28.045 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2024-09-10T23:38:28.045 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2024-09-10T23:38:28.045 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2024-09-10T23:38:28.045 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2024-09-10T23:38:28.046 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:38:28.046 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2024-09-10T23:38:28.046 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:38:28.046 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:38:28.046 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:38:28.046 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2024-09-10T23:38:28.046 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2024-09-10T23:38:28.046 DEBUG:teuthology.orchestra.run.smithi181:> return f 2024-09-10T23:38:28.046 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:38:28.047 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:38:28.047 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.1")) 2024-09-10T23:38:28.047 DEBUG:teuthology.orchestra.run.smithi181:> ' 2024-09-10T23:38:28.164 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.1.35684.asok 2024-09-10T23:38:28.168 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:38:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:38:28.175 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35684.asok 2024-09-10T23:38:28.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:28.176 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35684.asok status 2024-09-10T23:38:28.344 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T23:38:28.344 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2024-09-10T23:38:28.344 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:38:28.344 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:38:28.344 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "1", 2024-09-10T23:38:28.344 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2024-09-10T23:38:28.344 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T23:38:28.344 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "35684", 2024-09-10T23:38:28.344 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2024-09-10T23:38:28.344 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:38:28.345 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2024-09-10T23:38:28.345 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2024-09-10T23:38:28.345 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 7225, 2024-09-10T23:38:28.345 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2024-09-10T23:38:28.345 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2024-09-10T23:38:28.345 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2024-09-10T23:38:28.345 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 7225 2024-09-10T23:38:28.345 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:38:28.345 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:38:28.345 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:38:28.345 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:38:28.345 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 2578414749 2024-09-10T23:38:28.345 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T23:38:28.345 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:38:28.346 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:38:28.346 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:38:28.346 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:38:28.346 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 2578414749 2024-09-10T23:38:28.346 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:38:28.346 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.7225 192.168.0.1:0/2578414749", 2024-09-10T23:38:28.346 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.1:0/2578414749", 2024-09-10T23:38:28.346 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2024-09-10T23:38:28.346 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 78, 2024-09-10T23:38:28.346 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 114, 2024-09-10T23:38:28.346 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:38:28.346 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2024-09-10T23:38:28.346 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2024-09-10T23:38:28.346 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T23:38:28.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:28.359 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2024-09-10T23:38:28.359 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T23:38:28.368 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2024-09-10T23:38:28.368 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T23:38:28.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:28.368 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:38:28.427 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T23:38:28.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.577+0000 7f28ec400640 1 -- 172.21.15.94:0/946830997 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28e40cc2f0 msgr2=0x7f28cc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:28.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.577+0000 7f28ec400640 1 --2- 172.21.15.94:0/946830997 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28e40cc2f0 0x7f28cc004810 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f28e4071140 tx=0x7f28d001d250 comp rx=0 tx=0).stop 2024-09-10T23:38:28.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.577+0000 7f28ea400640 1 -- 172.21.15.94:0/946830997 reap_dead start 2024-09-10T23:38:28.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.577+0000 7f28ec400640 1 -- 172.21.15.94:0/946830997 shutdown_connections 2024-09-10T23:38:28.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.577+0000 7f28ec400640 1 -- 172.21.15.94:0/946830997 >> 172.21.15.94:0/946830997 conn(0x7f28e40d02d0 msgr2=0x7f28e40d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:28.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.577+0000 7f28ec400640 1 -- 172.21.15.94:0/946830997 shutdown_connections 2024-09-10T23:38:28.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.577+0000 7f28ec400640 1 -- 172.21.15.94:0/946830997 wait complete. 2024-09-10T23:38:28.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ec400640 1 Processor -- start 2024-09-10T23:38:28.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ec400640 1 -- start start 2024-09-10T23:38:28.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ec400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28e40cc2f0 0x7f28e40b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:28.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ec400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e4160f10 0x7f28e4161390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:28.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ec400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e4152070 0x7f28e41524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:28.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ec400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f28e40cd6e0 con 0x7f28e4152070 2024-09-10T23:38:28.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ec400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f28e40cd560 con 0x7f28e4160f10 2024-09-10T23:38:28.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ec400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f28e40cd860 con 0x7f28e40cc2f0 2024-09-10T23:38:28.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ea400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e4152070 0x7f28e41524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:28.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ea400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e4152070 0x7f28e41524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48872/0 (socket says 172.21.15.94:48872) 2024-09-10T23:38:28.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ea400640 1 -- 172.21.15.94:0/234416235 learned_addr learned my addr 172.21.15.94:0/234416235 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:28.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28e9a00640 1 --2- 172.21.15.94:0/234416235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28e40cc2f0 0x7f28e40b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:28.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28e9000640 1 --2- 172.21.15.94:0/234416235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e4160f10 0x7f28e4161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:28.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ea400640 1 -- 172.21.15.94:0/234416235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28e40cc2f0 msgr2=0x7f28e40b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:28.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ea400640 1 --2- 172.21.15.94:0/234416235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28e40cc2f0 0x7f28e40b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:28.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ea400640 1 -- 172.21.15.94:0/234416235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e4160f10 msgr2=0x7f28e4161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:28.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ea400640 1 --2- 172.21.15.94:0/234416235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e4160f10 0x7f28e4161390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:28.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ea400640 1 -- 172.21.15.94:0/234416235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e41f8300 con 0x7f28e4152070 2024-09-10T23:38:28.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28e9000640 1 --2- 172.21.15.94:0/234416235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e4160f10 0x7f28e4161390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:28.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ea400640 1 --2- 172.21.15.94:0/234416235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e4152070 0x7f28e41524f0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f28e40d0fa0 tx=0x7f28dc006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:28.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28e1600640 1 -- 172.21.15.94:0/234416235 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc011430 con 0x7f28e4152070 2024-09-10T23:38:28.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28e1600640 1 -- 172.21.15.94:0/234416235 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28dc016020 con 0x7f28e4152070 2024-09-10T23:38:28.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28e1600640 1 -- 172.21.15.94:0/234416235 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc011870 con 0x7f28e4152070 2024-09-10T23:38:28.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ec400640 1 -- 172.21.15.94:0/234416235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e41f85c0 con 0x7f28e4152070 2024-09-10T23:38:28.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ec400640 1 -- 172.21.15.94:0/234416235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e41f8b80 con 0x7f28e4152070 2024-09-10T23:38:28.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.581+0000 7f28ec400640 1 -- 172.21.15.94:0/234416235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e40cdea0 con 0x7f28e4152070 2024-09-10T23:38:28.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.585+0000 7f28e1600640 1 -- 172.21.15.94:0/234416235 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f28dc011a10 con 0x7f28e4152070 2024-09-10T23:38:28.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.585+0000 7f28e1600640 1 --2- 172.21.15.94:0/234416235 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f28c4072a90 0x7f28c4074f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:28.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.585+0000 7f28e1600640 1 -- 172.21.15.94:0/234416235 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f28dc0948a0 con 0x7f28e4152070 2024-09-10T23:38:28.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.585+0000 7f28e9a00640 1 --2- 172.21.15.94:0/234416235 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f28c4072a90 0x7f28c4074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:28.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.585+0000 7f28e9a00640 1 --2- 172.21.15.94:0/234416235 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f28c4072a90 0x7f28c4074f50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f28d8000c10 tx=0x7f28d8006110 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:28.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.589+0000 7f28e1600640 1 -- 172.21.15.94:0/234416235 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f28dc020020 con 0x7f28e4152070 2024-09-10T23:38:28.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.789+0000 7f28ec400640 1 -- 172.21.15.94:0/234416235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f28e406f6e0 con 0x7f28e4152070 2024-09-10T23:38:28.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.841+0000 7f28e1600640 1 -- 172.21.15.94:0/234416235 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v79) ==== 136+0+0 (secure 0 0 0) 0x7f28dc05f750 con 0x7f28e4152070 2024-09-10T23:38:28.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.841+0000 7f28ec400640 1 -- 172.21.15.94:0/234416235 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f28c4072a90 msgr2=0x7f28c4074f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:28.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.841+0000 7f28ec400640 1 --2- 172.21.15.94:0/234416235 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f28c4072a90 0x7f28c4074f50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f28d8000c10 tx=0x7f28d8006110 comp rx=0 tx=0).stop 2024-09-10T23:38:28.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.841+0000 7f28ec400640 1 -- 172.21.15.94:0/234416235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e4152070 msgr2=0x7f28e41524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:28.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.841+0000 7f28ec400640 1 --2- 172.21.15.94:0/234416235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e4152070 0x7f28e41524f0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f28e40d0fa0 tx=0x7f28dc006430 comp rx=0 tx=0).stop 2024-09-10T23:38:28.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.841+0000 7f28ec400640 1 -- 172.21.15.94:0/234416235 shutdown_connections 2024-09-10T23:38:28.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.841+0000 7f28ec400640 1 --2- 172.21.15.94:0/234416235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28e4160f10 0x7f28e4161390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:28.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.841+0000 7f28ec400640 1 --2- 172.21.15.94:0/234416235 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f28c4072a90 0x7f28c4074f50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:28.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.841+0000 7f28ec400640 1 --2- 172.21.15.94:0/234416235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e4152070 0x7f28e41524f0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:28.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.841+0000 7f28ec400640 1 --2- 172.21.15.94:0/234416235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28e40cc2f0 0x7f28e40b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:28.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.841+0000 7f28ec400640 1 -- 172.21.15.94:0/234416235 >> 172.21.15.94:0/234416235 conn(0x7f28e40d02d0 msgr2=0x7f28e415f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:28.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.841+0000 7f28ec400640 1 -- 172.21.15.94:0/234416235 shutdown_connections 2024-09-10T23:38:28.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:28.841+0000 7f28ec400640 1 -- 172.21.15.94:0/234416235 wait complete. 2024-09-10T23:38:28.865 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T23:38:29.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.005+0000 7f5c38800640 1 -- 172.21.15.94:0/2443596293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c300ce890 msgr2=0x7f5c300cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:29.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.005+0000 7f5c38800640 1 --2- 172.21.15.94:0/2443596293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c300ce890 0x7f5c300cd710 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009840 tx=0x7f5c2401c760 comp rx=0 tx=0).stop 2024-09-10T23:38:29.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.005+0000 7f5c38800640 1 -- 172.21.15.94:0/2443596293 shutdown_connections 2024-09-10T23:38:29.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.005+0000 7f5c38800640 1 --2- 172.21.15.94:0/2443596293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c300ce1e0 0x7f5c300cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.005+0000 7f5c38800640 1 --2- 172.21.15.94:0/2443596293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c300ce890 0x7f5c300cd710 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.005+0000 7f5c38800640 1 -- 172.21.15.94:0/2443596293 >> 172.21.15.94:0/2443596293 conn(0x7f5c300d1c60 msgr2=0x7f5c300d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:29.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.005+0000 7f5c38800640 1 -- 172.21.15.94:0/2443596293 shutdown_connections 2024-09-10T23:38:29.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.005+0000 7f5c38800640 1 -- 172.21.15.94:0/2443596293 wait complete. 2024-09-10T23:38:29.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c38800640 1 Processor -- start 2024-09-10T23:38:29.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c38800640 1 -- start start 2024-09-10T23:38:29.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c38800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c300b1100 0x7f5c300b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:29.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c38800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c300ce1e0 0x7f5c3014db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:29.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c38800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c300ce890 0x7f5c3014e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:29.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c38800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5c300d08e0 con 0x7f5c300ce890 2024-09-10T23:38:29.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c38800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5c300d0760 con 0x7f5c300b1100 2024-09-10T23:38:29.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c38800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5c300d0a60 con 0x7f5c300ce1e0 2024-09-10T23:38:29.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c36800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c300ce890 0x7f5c3014e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c36800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c300ce890 0x7f5c3014e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48888/0 (socket says 172.21.15.94:48888) 2024-09-10T23:38:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c36800640 1 -- 172.21.15.94:0/2094215903 learned_addr learned my addr 172.21.15.94:0/2094215903 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c35e00640 1 --2- 172.21.15.94:0/2094215903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c300b1100 0x7f5c300b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:29.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c35400640 1 --2- 172.21.15.94:0/2094215903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c300ce1e0 0x7f5c3014db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:29.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c35e00640 1 -- 172.21.15.94:0/2094215903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c300ce1e0 msgr2=0x7f5c3014db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:29.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c35e00640 1 --2- 172.21.15.94:0/2094215903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c300ce1e0 0x7f5c3014db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c35e00640 1 -- 172.21.15.94:0/2094215903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c300ce890 msgr2=0x7f5c3014e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:29.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c35e00640 1 --2- 172.21.15.94:0/2094215903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c300ce890 0x7f5c3014e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c35e00640 1 -- 172.21.15.94:0/2094215903 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3015e170 con 0x7f5c300b1100 2024-09-10T23:38:29.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c36800640 1 --2- 172.21.15.94:0/2094215903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c300ce890 0x7f5c3014e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:38:29.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c35e00640 1 --2- 172.21.15.94:0/2094215903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c300b1100 0x7f5c300b6ae0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5c240ac000 tx=0x7f5c24005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:29.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c2a000640 1 -- 172.21.15.94:0/2094215903 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c240b2040 con 0x7f5c300b1100 2024-09-10T23:38:29.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c2a000640 1 -- 172.21.15.94:0/2094215903 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c240042f0 con 0x7f5c300b1100 2024-09-10T23:38:29.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c2a000640 1 -- 172.21.15.94:0/2094215903 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c240052d0 con 0x7f5c300b1100 2024-09-10T23:38:29.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c38800640 1 -- 172.21.15.94:0/2094215903 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c300b5ac0 con 0x7f5c300b1100 2024-09-10T23:38:29.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.009+0000 7f5c38800640 1 -- 172.21.15.94:0/2094215903 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c300b6080 con 0x7f5c300b1100 2024-09-10T23:38:29.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.013+0000 7f5c38800640 1 -- 172.21.15.94:0/2094215903 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c300cd710 con 0x7f5c300b1100 2024-09-10T23:38:29.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.013+0000 7f5c2a000640 1 -- 172.21.15.94:0/2094215903 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f5c240b5070 con 0x7f5c300b1100 2024-09-10T23:38:29.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.013+0000 7f5c2a000640 1 --2- 172.21.15.94:0/2094215903 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5c14072a60 0x7f5c14074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:29.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.013+0000 7f5c2a000640 1 -- 172.21.15.94:0/2094215903 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f5c2412d500 con 0x7f5c300b1100 2024-09-10T23:38:29.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.013+0000 7f5c35400640 1 --2- 172.21.15.94:0/2094215903 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5c14072a60 0x7f5c14074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:29.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.013+0000 7f5c35400640 1 --2- 172.21.15.94:0/2094215903 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5c14072a60 0x7f5c14074f20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5c300c9840 tx=0x7f5c200091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:29.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.017+0000 7f5c2a000640 1 -- 172.21.15.94:0/2094215903 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5c24016410 con 0x7f5c300b1100 2024-09-10T23:38:29.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.205+0000 7f5c38800640 1 -- 172.21.15.94:0/2094215903 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f5c300ceec0 con 0x7f5c14072a60 2024-09-10T23:38:29.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.205+0000 7f5c2a000640 1 -- 172.21.15.94:0/2094215903 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f5c300ceec0 con 0x7f5c14072a60 2024-09-10T23:38:29.208 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:38:29.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:29.208 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephfs" 2024-09-10T23:38:29.208 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:29.208 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:38:29.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.209+0000 7f5c38800640 1 -- 172.21.15.94:0/2094215903 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5c14072a60 msgr2=0x7f5c14074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:29.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.209+0000 7f5c38800640 1 --2- 172.21.15.94:0/2094215903 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5c14072a60 0x7f5c14074f20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5c300c9840 tx=0x7f5c200091e0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.209+0000 7f5c38800640 1 -- 172.21.15.94:0/2094215903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c300b1100 msgr2=0x7f5c300b6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:29.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.209+0000 7f5c38800640 1 --2- 172.21.15.94:0/2094215903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c300b1100 0x7f5c300b6ae0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5c240ac000 tx=0x7f5c24005dd0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.209+0000 7f5c38800640 1 -- 172.21.15.94:0/2094215903 shutdown_connections 2024-09-10T23:38:29.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.209+0000 7f5c38800640 1 --2- 172.21.15.94:0/2094215903 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5c14072a60 0x7f5c14074f20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.209+0000 7f5c38800640 1 --2- 172.21.15.94:0/2094215903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c300ce890 0x7f5c3014e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.209+0000 7f5c38800640 1 --2- 172.21.15.94:0/2094215903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c300ce1e0 0x7f5c3014db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.209+0000 7f5c38800640 1 --2- 172.21.15.94:0/2094215903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c300b1100 0x7f5c300b6ae0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.209+0000 7f5c38800640 1 -- 172.21.15.94:0/2094215903 >> 172.21.15.94:0/2094215903 conn(0x7f5c300d1c60 msgr2=0x7f5c300b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:29.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.209+0000 7f5c38800640 1 -- 172.21.15.94:0/2094215903 shutdown_connections 2024-09-10T23:38:29.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.209+0000 7f5c38800640 1 -- 172.21.15.94:0/2094215903 wait complete. 2024-09-10T23:38:29.230 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:38:29.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f0442400640 1 -- 172.21.15.94:0/3020652722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c0cb4d0 msgr2=0x7f043c15b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:29.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f0442400640 1 --2- 172.21.15.94:0/3020652722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c0cb4d0 0x7f043c15b190 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f0434009bc0 tx=0x7f043401cba0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f0442400640 1 -- 172.21.15.94:0/3020652722 shutdown_connections 2024-09-10T23:38:29.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f0442400640 1 --2- 172.21.15.94:0/3020652722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c0cb4d0 0x7f043c15b190 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f0442400640 1 --2- 172.21.15.94:0/3020652722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f043c06fc00 0x7f043c0c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f0442400640 1 -- 172.21.15.94:0/3020652722 >> 172.21.15.94:0/3020652722 conn(0x7f043c0cfaf0 msgr2=0x7f043c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:29.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f0442400640 1 -- 172.21.15.94:0/3020652722 shutdown_connections 2024-09-10T23:38:29.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f0442400640 1 -- 172.21.15.94:0/3020652722 wait complete. 2024-09-10T23:38:29.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f0442400640 1 Processor -- start 2024-09-10T23:38:29.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f0442400640 1 -- start start 2024-09-10T23:38:29.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f0442400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f043c06fc00 0x7f043c150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:29.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f0442400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c0cb4d0 0x7f043c0b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:29.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f0442400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f043c158790 0x7f043c15df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:29.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f0442400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f043c1680c0 con 0x7f043c0cb4d0 2024-09-10T23:38:29.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f0442400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f043c167f40 con 0x7f043c06fc00 2024-09-10T23:38:29.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f0442400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f043c168240 con 0x7f043c158790 2024-09-10T23:38:29.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f043aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c0cb4d0 0x7f043c0b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:29.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f043aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c0cb4d0 0x7f043c0b2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48908/0 (socket says 172.21.15.94:48908) 2024-09-10T23:38:29.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f043aa00640 1 -- 172.21.15.94:0/335928681 learned_addr learned my addr 172.21.15.94:0/335928681 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:29.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f043b400640 1 --2- 172.21.15.94:0/335928681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f043c06fc00 0x7f043c150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f043be00640 1 --2- 172.21.15.94:0/335928681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f043c158790 0x7f043c15df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f043aa00640 1 -- 172.21.15.94:0/335928681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f043c158790 msgr2=0x7f043c15df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f043aa00640 1 --2- 172.21.15.94:0/335928681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f043c158790 0x7f043c15df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f043aa00640 1 -- 172.21.15.94:0/335928681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f043c06fc00 msgr2=0x7f043c150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f043aa00640 1 --2- 172.21.15.94:0/335928681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f043c06fc00 0x7f043c150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.373+0000 7f043aa00640 1 -- 172.21.15.94:0/335928681 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043c0b31b0 con 0x7f043c0cb4d0 2024-09-10T23:38:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.377+0000 7f043aa00640 1 --2- 172.21.15.94:0/335928681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c0cb4d0 0x7f043c0b2a40 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f042800d980 tx=0x7f042800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.377+0000 7f0433400640 1 -- 172.21.15.94:0/335928681 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428017070 con 0x7f043c0cb4d0 2024-09-10T23:38:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.377+0000 7f0433400640 1 -- 172.21.15.94:0/335928681 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0428009d20 con 0x7f043c0cb4d0 2024-09-10T23:38:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.377+0000 7f0442400640 1 -- 172.21.15.94:0/335928681 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043c0b3490 con 0x7f043c0cb4d0 2024-09-10T23:38:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.377+0000 7f0433400640 1 -- 172.21.15.94:0/335928681 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428004280 con 0x7f043c0cb4d0 2024-09-10T23:38:29.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.377+0000 7f0442400640 1 -- 172.21.15.94:0/335928681 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f043c0b3a50 con 0x7f043c0cb4d0 2024-09-10T23:38:29.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.377+0000 7f0442400640 1 -- 172.21.15.94:0/335928681 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f043c0c49b0 con 0x7f043c0cb4d0 2024-09-10T23:38:29.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.377+0000 7f0433400640 1 -- 172.21.15.94:0/335928681 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f0428003600 con 0x7f043c0cb4d0 2024-09-10T23:38:29.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.377+0000 7f0433400640 1 --2- 172.21.15.94:0/335928681 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f041c072a60 0x7f041c074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:29.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.377+0000 7f0433400640 1 -- 172.21.15.94:0/335928681 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f04280940d0 con 0x7f043c0cb4d0 2024-09-10T23:38:29.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.377+0000 7f043b400640 1 --2- 172.21.15.94:0/335928681 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f041c072a60 0x7f041c074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:29.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.381+0000 7f043b400640 1 --2- 172.21.15.94:0/335928681 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f041c072a60 0x7f041c074f20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f042c0046c0 tx=0x7f042c009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:29.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.381+0000 7f0433400640 1 -- 172.21.15.94:0/335928681 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f042805f010 con 0x7f043c0cb4d0 2024-09-10T23:38:29.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.549+0000 7f0442400640 1 -- 172.21.15.94:0/335928681 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f043c0d0850 con 0x7f043c0cb4d0 2024-09-10T23:38:29.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.549+0000 7f0433400640 1 -- 172.21.15.94:0/335928681 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7f042805e910 con 0x7f043c0cb4d0 2024-09-10T23:38:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.553+0000 7f0442400640 1 -- 172.21.15.94:0/335928681 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f041c072a60 msgr2=0x7f041c074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.553+0000 7f0442400640 1 --2- 172.21.15.94:0/335928681 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f041c072a60 0x7f041c074f20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f042c0046c0 tx=0x7f042c009040 comp rx=0 tx=0).stop 2024-09-10T23:38:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.553+0000 7f0442400640 1 -- 172.21.15.94:0/335928681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c0cb4d0 msgr2=0x7f043c0b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.553+0000 7f0442400640 1 --2- 172.21.15.94:0/335928681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c0cb4d0 0x7f043c0b2a40 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f042800d980 tx=0x7f042800de40 comp rx=0 tx=0).stop 2024-09-10T23:38:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.553+0000 7f0442400640 1 -- 172.21.15.94:0/335928681 shutdown_connections 2024-09-10T23:38:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.553+0000 7f0442400640 1 --2- 172.21.15.94:0/335928681 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f041c072a60 0x7f041c074f20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.553+0000 7f0442400640 1 --2- 172.21.15.94:0/335928681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f043c158790 0x7f043c15df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.553+0000 7f0442400640 1 --2- 172.21.15.94:0/335928681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c0cb4d0 0x7f043c0b2a40 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.553+0000 7f0442400640 1 --2- 172.21.15.94:0/335928681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f043c06fc00 0x7f043c150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.553+0000 7f0442400640 1 -- 172.21.15.94:0/335928681 >> 172.21.15.94:0/335928681 conn(0x7f043c0cfaf0 msgr2=0x7f043c0cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:29.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.553+0000 7f0442400640 1 -- 172.21.15.94:0/335928681 shutdown_connections 2024-09-10T23:38:29.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:29.553+0000 7f0442400640 1 -- 172.21.15.94:0/335928681 wait complete. 2024-09-10T23:38:29.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:29.574 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol10810/aeccf0fb-972d-488d-a02e-2e6cfb5661c7) 2024-09-10T23:38:29.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:29.610 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol10810/aeccf0fb-972d-488d-a02e-2e6cfb5661c7/.snap/fake) 2024-09-10T23:38:29.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:29.689 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T23:38:29.744 INFO:teuthology.orchestra.run.smithi094.stdout:cephfs_data 2024-09-10T23:38:29.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:29.745 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol10810/aeccf0fb-972d-488d-a02e-2e6cfb5661c7) 2024-09-10T23:38:29.810 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:38:29.810 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol10810/.meta 2024-09-10T23:38:29.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:29.878 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp51760/subvol38681/f648b0ef-e7c2-471f-ba1e-197b79fd4d86) 2024-09-10T23:38:29.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:29.951 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp51760/subvol38681/f648b0ef-e7c2-471f-ba1e-197b79fd4d86/.snap/fake) 2024-09-10T23:38:30.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:30.025 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T23:38:30.084 INFO:teuthology.orchestra.run.smithi094.stdout:cephfs_data 2024-09-10T23:38:30.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:30.085 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp51760/subvol38681/f648b0ef-e7c2-471f-ba1e-197b79fd4d86) 2024-09-10T23:38:30.151 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:38:30.151 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp51760/subvol38681/.meta 2024-09-10T23:38:30.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:30.216 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol34412/95f4d40c-75cd-46d6-8e20-3be0ed14e557) 2024-09-10T23:38:30.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:30.283 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T23:38:30.340 INFO:teuthology.orchestra.run.smithi094.stdout:cephfs_data 2024-09-10T23:38:30.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:30.340 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol34412/95f4d40c-75cd-46d6-8e20-3be0ed14e557) 2024-09-10T23:38:30.407 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:38:30.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol34412/.meta 2024-09-10T23:38:30.474 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol10810 2024-09-10T23:38:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.661+0000 7efe79a00640 1 -- 172.21.15.94:0/848574135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7415f560 msgr2=0x7efe740cfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:30.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.661+0000 7efe79a00640 1 --2- 172.21.15.94:0/848574135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7415f560 0x7efe740cfa90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7efe74073710 tx=0x7efe5c01c8c0 comp rx=0 tx=0).stop 2024-09-10T23:38:30.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.661+0000 7efe79a00640 1 -- 172.21.15.94:0/848574135 shutdown_connections 2024-09-10T23:38:30.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.661+0000 7efe79a00640 1 --2- 172.21.15.94:0/848574135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7415f560 0x7efe740cfa90 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:30.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.661+0000 7efe79a00640 1 --2- 172.21.15.94:0/848574135 >> v2:172.21.15.181:3300/0 conn(0x7efe7415f180 0x7efe740cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:30.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.661+0000 7efe79a00640 1 -- 172.21.15.94:0/848574135 >> 172.21.15.94:0/848574135 conn(0x7efe740d1cb0 msgr2=0x7efe740d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:30.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.661+0000 7efe79a00640 1 -- 172.21.15.94:0/848574135 shutdown_connections 2024-09-10T23:38:30.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.661+0000 7efe79a00640 1 -- 172.21.15.94:0/848574135 wait complete. 2024-09-10T23:38:30.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe79a00640 1 Processor -- start 2024-09-10T23:38:30.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe79a00640 1 -- start start 2024-09-10T23:38:30.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe79a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe740cffd0 0x7efe74158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:30.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe79a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7415f180 0x7efe740b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:30.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe79a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe7415f560 0x7efe741f8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:30.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe79a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efe741692a0 con 0x7efe7415f180 2024-09-10T23:38:30.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe79a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efe74169120 con 0x7efe740cffd0 2024-09-10T23:38:30.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe79a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7efe74169420 con 0x7efe7415f560 2024-09-10T23:38:30.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe6a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7415f180 0x7efe740b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:30.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe6a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7415f180 0x7efe740b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48918/0 (socket says 172.21.15.94:48918) 2024-09-10T23:38:30.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe6a000640 1 -- 172.21.15.94:0/3106910007 learned_addr learned my addr 172.21.15.94:0/3106910007 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:30.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe72a00640 1 --2- 172.21.15.94:0/3106910007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe740cffd0 0x7efe74158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:30.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe73400640 1 --2- 172.21.15.94:0/3106910007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe7415f560 0x7efe741f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:30.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe6a000640 1 -- 172.21.15.94:0/3106910007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe7415f560 msgr2=0x7efe741f8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:30.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe6a000640 1 --2- 172.21.15.94:0/3106910007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe7415f560 0x7efe741f8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:30.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe6a000640 1 -- 172.21.15.94:0/3106910007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe740cffd0 msgr2=0x7efe74158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:30.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe6a000640 1 --2- 172.21.15.94:0/3106910007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe740cffd0 0x7efe74158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:30.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe6a000640 1 -- 172.21.15.94:0/3106910007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe740b5230 con 0x7efe7415f180 2024-09-10T23:38:30.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe6a000640 1 --2- 172.21.15.94:0/3106910007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7415f180 0x7efe740b4ae0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0b0040 tx=0x7efe5c0a57c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:30.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe6b400640 1 -- 172.21.15.94:0/3106910007 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c0b2040 con 0x7efe7415f180 2024-09-10T23:38:30.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe6b400640 1 -- 172.21.15.94:0/3106910007 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe5c0044b0 con 0x7efe7415f180 2024-09-10T23:38:30.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe6b400640 1 -- 172.21.15.94:0/3106910007 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c003860 con 0x7efe7415f180 2024-09-10T23:38:30.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe79a00640 1 -- 172.21.15.94:0/3106910007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe740b5510 con 0x7efe7415f180 2024-09-10T23:38:30.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe79a00640 1 -- 172.21.15.94:0/3106910007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe741f8040 con 0x7efe7415f180 2024-09-10T23:38:30.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.665+0000 7efe79a00640 1 -- 172.21.15.94:0/3106910007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe740cb790 con 0x7efe7415f180 2024-09-10T23:38:30.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.669+0000 7efe6b400640 1 -- 172.21.15.94:0/3106910007 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7efe5c0c0060 con 0x7efe7415f180 2024-09-10T23:38:30.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.669+0000 7efe6b400640 1 --2- 172.21.15.94:0/3106910007 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efe50072a60 0x7efe50074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:30.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.669+0000 7efe6b400640 1 -- 172.21.15.94:0/3106910007 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7efe5c12d3b0 con 0x7efe7415f180 2024-09-10T23:38:30.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.669+0000 7efe72a00640 1 --2- 172.21.15.94:0/3106910007 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efe50072a60 0x7efe50074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:30.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.669+0000 7efe72a00640 1 --2- 172.21.15.94:0/3106910007 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efe50072a60 0x7efe50074f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efe64006fd0 tx=0x7efe64008040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:30.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.673+0000 7efe6b400640 1 -- 172.21.15.94:0/3106910007 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efe5c0164b0 con 0x7efe7415f180 2024-09-10T23:38:30.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.865+0000 7efe79a00640 1 -- 172.21.15.94:0/3106910007 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol10810", "target": ["mon-mgr", ""]}) -- 0x7efe740cd0a0 con 0x7efe50072a60 2024-09-10T23:38:30.877 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:30.873+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:30.877 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:30.873+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:30.877 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:30.873+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:30.877 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:30.873+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:30.877 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:30.873+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:30.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.909+0000 7efe6b400640 1 -- 172.21.15.94:0/3106910007 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7efe740cd0a0 con 0x7efe50072a60 2024-09-10T23:38:30.910 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/_nogroup/subvol10810/aeccf0fb-972d-488d-a02e-2e6cfb5661c7 2024-09-10T23:38:30.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.909+0000 7efe79a00640 1 -- 172.21.15.94:0/3106910007 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efe50072a60 msgr2=0x7efe50074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:30.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.909+0000 7efe79a00640 1 --2- 172.21.15.94:0/3106910007 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efe50072a60 0x7efe50074f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efe64006fd0 tx=0x7efe64008040 comp rx=0 tx=0).stop 2024-09-10T23:38:30.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.909+0000 7efe79a00640 1 -- 172.21.15.94:0/3106910007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7415f180 msgr2=0x7efe740b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:30.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.909+0000 7efe79a00640 1 --2- 172.21.15.94:0/3106910007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7415f180 0x7efe740b4ae0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0b0040 tx=0x7efe5c0a57c0 comp rx=0 tx=0).stop 2024-09-10T23:38:30.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.909+0000 7efe79a00640 1 -- 172.21.15.94:0/3106910007 shutdown_connections 2024-09-10T23:38:30.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.909+0000 7efe79a00640 1 --2- 172.21.15.94:0/3106910007 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efe50072a60 0x7efe50074f20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:30.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.909+0000 7efe79a00640 1 --2- 172.21.15.94:0/3106910007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe7415f560 0x7efe741f8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:30.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.909+0000 7efe79a00640 1 --2- 172.21.15.94:0/3106910007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7415f180 0x7efe740b4ae0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:30.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.909+0000 7efe79a00640 1 --2- 172.21.15.94:0/3106910007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe740cffd0 0x7efe74158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:30.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.909+0000 7efe79a00640 1 -- 172.21.15.94:0/3106910007 >> 172.21.15.94:0/3106910007 conn(0x7efe740d1cb0 msgr2=0x7efe7415b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:30.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.909+0000 7efe79a00640 1 -- 172.21.15.94:0/3106910007 shutdown_connections 2024-09-10T23:38:30.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:30.909+0000 7efe79a00640 1 -- 172.21.15.94:0/3106910007 wait complete. 2024-09-10T23:38:30.930 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol38681 subvol_grp51760 2024-09-10T23:38:31.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.073+0000 7ff57e000640 1 -- 172.21.15.94:0/1423497465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff57815af30 msgr2=0x7ff57806f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:31.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.073+0000 7ff57e000640 1 --2- 172.21.15.94:0/1423497465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff57815af30 0x7ff57806f3b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff564009900 tx=0x7ff56401c8b0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.073+0000 7ff57e000640 1 -- 172.21.15.94:0/1423497465 shutdown_connections 2024-09-10T23:38:31.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.073+0000 7ff57e000640 1 --2- 172.21.15.94:0/1423497465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff57815af30 0x7ff57806f3b0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.073+0000 7ff57e000640 1 --2- 172.21.15.94:0/1423497465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff578073630 0x7ff578073a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.073+0000 7ff57e000640 1 -- 172.21.15.94:0/1423497465 >> 172.21.15.94:0/1423497465 conn(0x7ff5780cd6b0 msgr2=0x7ff5780cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:31.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.073+0000 7ff57e000640 1 -- 172.21.15.94:0/1423497465 shutdown_connections 2024-09-10T23:38:31.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.073+0000 7ff57e000640 1 -- 172.21.15.94:0/1423497465 wait complete. 2024-09-10T23:38:31.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.073+0000 7ff57e000640 1 Processor -- start 2024-09-10T23:38:31.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57e000640 1 -- start start 2024-09-10T23:38:31.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57e000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff578073630 0x7ff5780c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:31.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff57815af30 0x7ff5780c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:31.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff57815e580 0x7ff5781fb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:31.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57e000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5781684c0 con 0x7ff57815e580 2024-09-10T23:38:31.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57e000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff578168340 con 0x7ff57815af30 2024-09-10T23:38:31.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57e000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff578168640 con 0x7ff578073630 2024-09-10T23:38:31.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff57815e580 0x7ff5781fb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:31.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff57815e580 0x7ff5781fb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48940/0 (socket says 172.21.15.94:48940) 2024-09-10T23:38:31.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57d600640 1 -- 172.21.15.94:0/277207537 learned_addr learned my addr 172.21.15.94:0/277207537 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:31.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57cc00640 1 --2- 172.21.15.94:0/277207537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff578073630 0x7ff5780c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:31.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff577e00640 1 --2- 172.21.15.94:0/277207537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff57815af30 0x7ff5780c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:31.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57d600640 1 -- 172.21.15.94:0/277207537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff578073630 msgr2=0x7ff5780c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:31.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57d600640 1 --2- 172.21.15.94:0/277207537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff578073630 0x7ff5780c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57d600640 1 -- 172.21.15.94:0/277207537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff57815af30 msgr2=0x7ff5780c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:31.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57d600640 1 --2- 172.21.15.94:0/277207537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff57815af30 0x7ff5780c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57d600640 1 -- 172.21.15.94:0/277207537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5780c7480 con 0x7ff57815e580 2024-09-10T23:38:31.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff577e00640 1 --2- 172.21.15.94:0/277207537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff57815af30 0x7ff5780c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:38:31.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57d600640 1 --2- 172.21.15.94:0/277207537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff57815e580 0x7ff5781fb660 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c710 tx=0x7ff57000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:31.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff574c00640 1 -- 172.21.15.94:0/277207537 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570003b80 con 0x7ff57815e580 2024-09-10T23:38:31.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff574c00640 1 -- 172.21.15.94:0/277207537 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff570003d20 con 0x7ff57815e580 2024-09-10T23:38:31.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57e000640 1 -- 172.21.15.94:0/277207537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5780ca840 con 0x7ff57815e580 2024-09-10T23:38:31.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff574c00640 1 -- 172.21.15.94:0/277207537 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57000aab0 con 0x7ff57815e580 2024-09-10T23:38:31.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57e000640 1 -- 172.21.15.94:0/277207537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff578154760 con 0x7ff57815e580 2024-09-10T23:38:31.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.077+0000 7ff57e000640 1 -- 172.21.15.94:0/277207537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff578073a10 con 0x7ff57815e580 2024-09-10T23:38:31.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.081+0000 7ff574c00640 1 -- 172.21.15.94:0/277207537 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7ff57000ac50 con 0x7ff57815e580 2024-09-10T23:38:31.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.081+0000 7ff574c00640 1 --2- 172.21.15.94:0/277207537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff558072a90 0x7ff558074f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:31.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.081+0000 7ff574c00640 1 -- 172.21.15.94:0/277207537 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7ff5700937a0 con 0x7ff57815e580 2024-09-10T23:38:31.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.081+0000 7ff57cc00640 1 --2- 172.21.15.94:0/277207537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff558072a90 0x7ff558074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:31.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.081+0000 7ff57cc00640 1 --2- 172.21.15.94:0/277207537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff558072a90 0x7ff558074f50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0058d0 tx=0x7ff56c005860 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:31.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.085+0000 7ff574c00640 1 -- 172.21.15.94:0/277207537 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff57000ce20 con 0x7ff57815e580 2024-09-10T23:38:31.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.277+0000 7ff57e000640 1 -- 172.21.15.94:0/277207537 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol38681", "group_name": "subvol_grp51760", "target": ["mon-mgr", ""]}) -- 0x7ff5780cabe0 con 0x7ff558072a90 2024-09-10T23:38:31.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.305+0000 7ff574c00640 1 -- 172.21.15.94:0/277207537 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7ff56c00b320 con 0x7ff558072a90 2024-09-10T23:38:31.308 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/subvol_grp51760/subvol38681/f648b0ef-e7c2-471f-ba1e-197b79fd4d86 2024-09-10T23:38:31.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.309+0000 7ff57e000640 1 -- 172.21.15.94:0/277207537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff558072a90 msgr2=0x7ff558074f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:31.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.309+0000 7ff57e000640 1 --2- 172.21.15.94:0/277207537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff558072a90 0x7ff558074f50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0058d0 tx=0x7ff56c005860 comp rx=0 tx=0).stop 2024-09-10T23:38:31.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.309+0000 7ff57e000640 1 -- 172.21.15.94:0/277207537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff57815e580 msgr2=0x7ff5781fb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:31.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.309+0000 7ff57e000640 1 --2- 172.21.15.94:0/277207537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff57815e580 0x7ff5781fb660 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c710 tx=0x7ff57000cbd0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.309+0000 7ff57e000640 1 -- 172.21.15.94:0/277207537 shutdown_connections 2024-09-10T23:38:31.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.309+0000 7ff57e000640 1 --2- 172.21.15.94:0/277207537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff57815af30 0x7ff5780c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.309+0000 7ff57e000640 1 --2- 172.21.15.94:0/277207537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff558072a90 0x7ff558074f50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.309+0000 7ff57e000640 1 --2- 172.21.15.94:0/277207537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff57815e580 0x7ff5781fb660 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.309+0000 7ff57e000640 1 --2- 172.21.15.94:0/277207537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff578073630 0x7ff5780c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.309+0000 7ff57e000640 1 -- 172.21.15.94:0/277207537 >> 172.21.15.94:0/277207537 conn(0x7ff5780cd6b0 msgr2=0x7ff5780d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:31.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.309+0000 7ff57e000640 1 -- 172.21.15.94:0/277207537 shutdown_connections 2024-09-10T23:38:31.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.309+0000 7ff57e000640 1 -- 172.21.15.94:0/277207537 wait complete. 2024-09-10T23:38:31.329 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol34412 2024-09-10T23:38:31.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.473+0000 7f8322a00640 1 -- 172.21.15.94:0/23558196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f831c0cf280 msgr2=0x7f831c15af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:31.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.473+0000 7f8322a00640 1 --2- 172.21.15.94:0/23558196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f831c0cf280 0x7f831c15af90 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8314009840 tx=0x7f831401c820 comp rx=0 tx=0).stop 2024-09-10T23:38:31.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.473+0000 7f8322a00640 1 -- 172.21.15.94:0/23558196 shutdown_connections 2024-09-10T23:38:31.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.473+0000 7f8322a00640 1 --2- 172.21.15.94:0/23558196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f831c0cf280 0x7f831c15af90 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.473+0000 7f8322a00640 1 -- 172.21.15.94:0/23558196 >> 172.21.15.94:0/23558196 conn(0x7f831c0cfaf0 msgr2=0x7f831c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:31.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.473+0000 7f8322a00640 1 -- 172.21.15.94:0/23558196 shutdown_connections 2024-09-10T23:38:31.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.473+0000 7f8322a00640 1 -- 172.21.15.94:0/23558196 wait complete. 2024-09-10T23:38:31.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.473+0000 7f8322a00640 1 Processor -- start 2024-09-10T23:38:31.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.473+0000 7f8322a00640 1 -- start start 2024-09-10T23:38:31.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f8322a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f831c0c64f0 0x7f831c1fba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:31.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f8322a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f831c0ce280 0x7f831c151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:31.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f8322a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f831c0cf280 0x7f831c0ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:31.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f8322a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f831c168c90 con 0x7f831c0c64f0 2024-09-10T23:38:31.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f8322a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f831c168b10 con 0x7f831c0ce280 2024-09-10T23:38:31.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f8322a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f831c168e10 con 0x7f831c0cf280 2024-09-10T23:38:31.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f831be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f831c0c64f0 0x7f831c1fba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:31.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f831be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f831c0c64f0 0x7f831c1fba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48960/0 (socket says 172.21.15.94:48960) 2024-09-10T23:38:31.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f831be00640 1 -- 172.21.15.94:0/3271279935 learned_addr learned my addr 172.21.15.94:0/3271279935 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:31.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f831be00640 1 -- 172.21.15.94:0/3271279935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f831c0cf280 msgr2=0x7f831c0ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:31.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f831be00640 1 --2- 172.21.15.94:0/3271279935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f831c0cf280 0x7f831c0ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f831be00640 1 -- 172.21.15.94:0/3271279935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f831c0ce280 msgr2=0x7f831c151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:31.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f831be00640 1 --2- 172.21.15.94:0/3271279935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f831c0ce280 0x7f831c151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f831be00640 1 -- 172.21.15.94:0/3271279935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c1fbf60 con 0x7f831c0c64f0 2024-09-10T23:38:31.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f831be00640 1 --2- 172.21.15.94:0/3271279935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f831c0c64f0 0x7f831c1fba20 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f830c00c8a0 tx=0x7f830c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:31.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f8313e00640 1 -- 172.21.15.94:0/3271279935 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c003b80 con 0x7f831c0c64f0 2024-09-10T23:38:31.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f8313e00640 1 -- 172.21.15.94:0/3271279935 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f830c003d40 con 0x7f831c0c64f0 2024-09-10T23:38:31.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f8322a00640 1 -- 172.21.15.94:0/3271279935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c0b0fa0 con 0x7f831c0c64f0 2024-09-10T23:38:31.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f8313e00640 1 -- 172.21.15.94:0/3271279935 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c00aab0 con 0x7f831c0c64f0 2024-09-10T23:38:31.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f8322a00640 1 -- 172.21.15.94:0/3271279935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c0c8030 con 0x7f831c0c64f0 2024-09-10T23:38:31.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f8322a00640 1 -- 172.21.15.94:0/3271279935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831c0b14c0 con 0x7f831c0c64f0 2024-09-10T23:38:31.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.477+0000 7f8313e00640 1 -- 172.21.15.94:0/3271279935 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f830c00ac50 con 0x7f831c0c64f0 2024-09-10T23:38:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.481+0000 7f8313e00640 1 --2- 172.21.15.94:0/3271279935 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f82fc072a60 0x7f82fc074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.481+0000 7f8313e00640 1 -- 172.21.15.94:0/3271279935 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f830c093920 con 0x7f831c0c64f0 2024-09-10T23:38:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.481+0000 7f831b400640 1 --2- 172.21.15.94:0/3271279935 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f82fc072a60 0x7f82fc074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.481+0000 7f831b400640 1 --2- 172.21.15.94:0/3271279935 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f82fc072a60 0x7f82fc074f20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8308001630 tx=0x7f8308007a00 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:31.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.485+0000 7f8313e00640 1 -- 172.21.15.94:0/3271279935 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f831c0b14c0 con 0x7f831c0c64f0 2024-09-10T23:38:31.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.681+0000 7f8322a00640 1 -- 172.21.15.94:0/3271279935 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol34412", "target": ["mon-mgr", ""]}) -- 0x7f831c0af330 con 0x7f82fc072a60 2024-09-10T23:38:31.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.705+0000 7f8313e00640 1 -- 172.21.15.94:0/3271279935 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+144 (secure 0 0 0) 0x7f831c0af330 con 0x7f82fc072a60 2024-09-10T23:38:31.707 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:38:31.707 INFO:teuthology.orchestra.run.smithi094.stdout: "status": { 2024-09-10T23:38:31.707 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "pending", 2024-09-10T23:38:31.707 INFO:teuthology.orchestra.run.smithi094.stdout: "source": { 2024-09-10T23:38:31.707 INFO:teuthology.orchestra.run.smithi094.stdout: "volume": "fake", 2024-09-10T23:38:31.707 INFO:teuthology.orchestra.run.smithi094.stdout: "subvolume": "fake", 2024-09-10T23:38:31.708 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot": "fake" 2024-09-10T23:38:31.708 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:31.708 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:31.708 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:38:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.705+0000 7f8322a00640 1 -- 172.21.15.94:0/3271279935 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f82fc072a60 msgr2=0x7f82fc074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.705+0000 7f8322a00640 1 --2- 172.21.15.94:0/3271279935 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f82fc072a60 0x7f82fc074f20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8308001630 tx=0x7f8308007a00 comp rx=0 tx=0).stop 2024-09-10T23:38:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.705+0000 7f8322a00640 1 -- 172.21.15.94:0/3271279935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f831c0c64f0 msgr2=0x7f831c1fba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.705+0000 7f8322a00640 1 --2- 172.21.15.94:0/3271279935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f831c0c64f0 0x7f831c1fba20 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f830c00c8a0 tx=0x7f830c00cd60 comp rx=0 tx=0).stop 2024-09-10T23:38:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.705+0000 7f8322a00640 1 -- 172.21.15.94:0/3271279935 shutdown_connections 2024-09-10T23:38:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.705+0000 7f8322a00640 1 --2- 172.21.15.94:0/3271279935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f831c0ce280 0x7f831c151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.705+0000 7f8322a00640 1 --2- 172.21.15.94:0/3271279935 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f82fc072a60 0x7f82fc074f20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.705+0000 7f8322a00640 1 --2- 172.21.15.94:0/3271279935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f831c0cf280 0x7f831c0ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.705+0000 7f8322a00640 1 --2- 172.21.15.94:0/3271279935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f831c0c64f0 0x7f831c1fba20 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.705+0000 7f8322a00640 1 -- 172.21.15.94:0/3271279935 >> 172.21.15.94:0/3271279935 conn(0x7f831c0cfaf0 msgr2=0x7f831c0af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.709+0000 7f8322a00640 1 -- 172.21.15.94:0/3271279935 shutdown_connections 2024-09-10T23:38:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.709+0000 7f8322a00640 1 -- 172.21.15.94:0/3271279935 wait complete. 2024-09-10T23:38:31.728 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol10810 fake 2024-09-10T23:38:31.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.873+0000 7f695ca00640 1 -- 172.21.15.94:0/2251605807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c004030 msgr2=0x7f693c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:31.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.873+0000 7f695ca00640 1 --2- 172.21.15.94:0/2251605807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c004030 0x7f693c004810 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f6944009b60 tx=0x7f694401d280 comp rx=0 tx=0).stop 2024-09-10T23:38:31.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.873+0000 7f6956800640 1 -- 172.21.15.94:0/2251605807 reap_dead start 2024-09-10T23:38:31.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.873+0000 7f695ca00640 1 -- 172.21.15.94:0/2251605807 shutdown_connections 2024-09-10T23:38:31.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.873+0000 7f695ca00640 1 -- 172.21.15.94:0/2251605807 >> 172.21.15.94:0/2251605807 conn(0x7f695806cdf0 msgr2=0x7f695806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:31.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.873+0000 7f695ca00640 1 -- 172.21.15.94:0/2251605807 shutdown_connections 2024-09-10T23:38:31.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.873+0000 7f695ca00640 1 -- 172.21.15.94:0/2251605807 wait complete. 2024-09-10T23:38:31.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f695ca00640 1 Processor -- start 2024-09-10T23:38:31.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f695ca00640 1 -- start start 2024-09-10T23:38:31.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f695ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c004030 0x7f69581473e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:31.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f695ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f695814be60 0x7f695814dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:31.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f695ca00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f695814e900 0x7f695814edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:31.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f695ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69580731b0 con 0x7f693c004030 2024-09-10T23:38:31.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f695ca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6958073030 con 0x7f695814be60 2024-09-10T23:38:31.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f695ca00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6958073330 con 0x7f695814e900 2024-09-10T23:38:31.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f6955e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c004030 0x7f69581473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:31.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f6955e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c004030 0x7f69581473e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48968/0 (socket says 172.21.15.94:48968) 2024-09-10T23:38:31.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f6955e00640 1 -- 172.21.15.94:0/2120912483 learned_addr learned my addr 172.21.15.94:0/2120912483 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:31.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f6956800640 1 --2- 172.21.15.94:0/2120912483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f695814e900 0x7f695814edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:31.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f6955400640 1 --2- 172.21.15.94:0/2120912483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f695814be60 0x7f695814dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:31.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f6955e00640 1 -- 172.21.15.94:0/2120912483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f695814e900 msgr2=0x7f695814edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:31.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f6955e00640 1 --2- 172.21.15.94:0/2120912483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f695814e900 0x7f695814edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f6955e00640 1 -- 172.21.15.94:0/2120912483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f695814be60 msgr2=0x7f695814dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:31.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f6955e00640 1 --2- 172.21.15.94:0/2120912483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f695814be60 0x7f695814dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:31.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f6955e00640 1 -- 172.21.15.94:0/2120912483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695814c2f0 con 0x7f693c004030 2024-09-10T23:38:31.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f6955e00640 1 --2- 172.21.15.94:0/2120912483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c004030 0x7f69581473e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f6948004fd0 tx=0x7f694800ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:31.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f694e000640 1 -- 172.21.15.94:0/2120912483 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948015070 con 0x7f693c004030 2024-09-10T23:38:31.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f694e000640 1 -- 172.21.15.94:0/2120912483 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f694800ca60 con 0x7f693c004030 2024-09-10T23:38:31.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f695ca00640 1 -- 172.21.15.94:0/2120912483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695814c5d0 con 0x7f693c004030 2024-09-10T23:38:31.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f694e000640 1 -- 172.21.15.94:0/2120912483 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948011460 con 0x7f693c004030 2024-09-10T23:38:31.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.877+0000 7f695ca00640 1 -- 172.21.15.94:0/2120912483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f695814cb90 con 0x7f693c004030 2024-09-10T23:38:31.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.881+0000 7f695ca00640 1 -- 172.21.15.94:0/2120912483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f695815e590 con 0x7f693c004030 2024-09-10T23:38:31.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.881+0000 7f694e000640 1 -- 172.21.15.94:0/2120912483 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7f6948011600 con 0x7f693c004030 2024-09-10T23:38:31.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.881+0000 7f694e000640 1 --2- 172.21.15.94:0/2120912483 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6938072a60 0x7f6938074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:31.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.881+0000 7f694e000640 1 -- 172.21.15.94:0/2120912483 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f69480942d0 con 0x7f693c004030 2024-09-10T23:38:31.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.881+0000 7f6955400640 1 --2- 172.21.15.94:0/2120912483 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6938072a60 0x7f6938074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:31.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.881+0000 7f6955400640 1 --2- 172.21.15.94:0/2120912483 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6938072a60 0x7f6938074f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f69440ae000 tx=0x7f694401a020 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:31.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:31.885+0000 7f694e000640 1 -- 172.21.15.94:0/2120912483 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f694805ebc0 con 0x7f693c004030 2024-09-10T23:38:32.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.081+0000 7f695ca00640 1 -- 172.21.15.94:0/2120912483 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol10810", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7f695814cdd0 con 0x7f6938072a60 2024-09-10T23:38:32.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.109+0000 7f694e000640 1 -- 172.21.15.94:0/2120912483 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f69440051b0 con 0x7f6938072a60 2024-09-10T23:38:32.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.113+0000 7f695ca00640 1 -- 172.21.15.94:0/2120912483 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6938072a60 msgr2=0x7f6938074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:32.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.113+0000 7f695ca00640 1 --2- 172.21.15.94:0/2120912483 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6938072a60 0x7f6938074f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f69440ae000 tx=0x7f694401a020 comp rx=0 tx=0).stop 2024-09-10T23:38:32.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.113+0000 7f695ca00640 1 -- 172.21.15.94:0/2120912483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c004030 msgr2=0x7f69581473e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:32.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.113+0000 7f695ca00640 1 --2- 172.21.15.94:0/2120912483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c004030 0x7f69581473e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f6948004fd0 tx=0x7f694800ea90 comp rx=0 tx=0).stop 2024-09-10T23:38:32.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.113+0000 7f695ca00640 1 -- 172.21.15.94:0/2120912483 shutdown_connections 2024-09-10T23:38:32.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.113+0000 7f695ca00640 1 --2- 172.21.15.94:0/2120912483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f695814be60 0x7f695814dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:32.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.113+0000 7f695ca00640 1 --2- 172.21.15.94:0/2120912483 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6938072a60 0x7f6938074f20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:32.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.113+0000 7f695ca00640 1 --2- 172.21.15.94:0/2120912483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f695814e900 0x7f695814edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:32.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.113+0000 7f695ca00640 1 --2- 172.21.15.94:0/2120912483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f693c004030 0x7f69581473e0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:32.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.113+0000 7f695ca00640 1 -- 172.21.15.94:0/2120912483 >> 172.21.15.94:0/2120912483 conn(0x7f695806cdf0 msgr2=0x7f6958158b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:32.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.113+0000 7f695ca00640 1 -- 172.21.15.94:0/2120912483 shutdown_connections 2024-09-10T23:38:32.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.113+0000 7f695ca00640 1 -- 172.21.15.94:0/2120912483 wait complete. 2024-09-10T23:38:32.135 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol38681 fake subvol_grp51760 2024-09-10T23:38:32.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.281+0000 7fdd97000640 1 -- 172.21.15.94:0/3487256811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd78003a10 msgr2=0x7fdd78003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:32.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.281+0000 7fdd97000640 1 --2- 172.21.15.94:0/3487256811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd78003a10 0x7fdd78003e30 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009b60 tx=0x7fdd8401d250 comp rx=0 tx=0).stop 2024-09-10T23:38:32.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.281+0000 7fdd96600640 1 -- 172.21.15.94:0/3487256811 reap_dead start 2024-09-10T23:38:32.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.281+0000 7fdd97000640 1 -- 172.21.15.94:0/3487256811 shutdown_connections 2024-09-10T23:38:32.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.281+0000 7fdd97000640 1 -- 172.21.15.94:0/3487256811 >> 172.21.15.94:0/3487256811 conn(0x7fdd900d0a40 msgr2=0x7fdd900d0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:32.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.281+0000 7fdd97000640 1 -- 172.21.15.94:0/3487256811 shutdown_connections 2024-09-10T23:38:32.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.281+0000 7fdd97000640 1 -- 172.21.15.94:0/3487256811 wait complete. 2024-09-10T23:38:32.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd97000640 1 Processor -- start 2024-09-10T23:38:32.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd97000640 1 -- start start 2024-09-10T23:38:32.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd97000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd78003a10 0x7fdd90156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:32.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd97000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd90154ed0 0x7fdd901fba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:32.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd97000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd90154860 0x7fdd900cf790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:32.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd97000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9015cdc0 con 0x7fdd90154ed0 2024-09-10T23:38:32.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd97000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9015cc40 con 0x7fdd90154860 2024-09-10T23:38:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd97000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fdd90166e80 con 0x7fdd78003a10 2024-09-10T23:38:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd95200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd90154ed0 0x7fdd901fba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd95200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd90154ed0 0x7fdd901fba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48992/0 (socket says 172.21.15.94:48992) 2024-09-10T23:38:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd95200640 1 -- 172.21.15.94:0/2020396750 learned_addr learned my addr 172.21.15.94:0/2020396750 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd96600640 1 --2- 172.21.15.94:0/2020396750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd90154860 0x7fdd900cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd95c00640 1 --2- 172.21.15.94:0/2020396750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd78003a10 0x7fdd90156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd95200640 1 -- 172.21.15.94:0/2020396750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd78003a10 msgr2=0x7fdd90156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd95200640 1 --2- 172.21.15.94:0/2020396750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd78003a10 0x7fdd90156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd95200640 1 -- 172.21.15.94:0/2020396750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd90154860 msgr2=0x7fdd900cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd95200640 1 --2- 172.21.15.94:0/2020396750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd90154860 0x7fdd900cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd95200640 1 -- 172.21.15.94:0/2020396750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd90154ce0 con 0x7fdd90154ed0 2024-09-10T23:38:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd95200640 1 --2- 172.21.15.94:0/2020396750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd90154ed0 0x7fdd901fba90 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c011ab0 tx=0x7fdd7c011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd8d600640 1 -- 172.21.15.94:0/2020396750 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c00fcf0 con 0x7fdd90154ed0 2024-09-10T23:38:32.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd8d600640 1 -- 172.21.15.94:0/2020396750 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd7c016070 con 0x7fdd90154ed0 2024-09-10T23:38:32.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd97000640 1 -- 172.21.15.94:0/2020396750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd90157090 con 0x7fdd90154ed0 2024-09-10T23:38:32.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd8d600640 1 -- 172.21.15.94:0/2020396750 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c012690 con 0x7fdd90154ed0 2024-09-10T23:38:32.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd97000640 1 -- 172.21.15.94:0/2020396750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd90157650 con 0x7fdd90154ed0 2024-09-10T23:38:32.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.285+0000 7fdd97000640 1 -- 172.21.15.94:0/2020396750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd901651e0 con 0x7fdd90154ed0 2024-09-10T23:38:32.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.289+0000 7fdd8d600640 1 -- 172.21.15.94:0/2020396750 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7fdd7c012830 con 0x7fdd90154ed0 2024-09-10T23:38:32.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.289+0000 7fdd8d600640 1 --2- 172.21.15.94:0/2020396750 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdd70072a90 0x7fdd70074f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:32.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.289+0000 7fdd8d600640 1 -- 172.21.15.94:0/2020396750 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fdd7c094550 con 0x7fdd90154ed0 2024-09-10T23:38:32.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.289+0000 7fdd95c00640 1 --2- 172.21.15.94:0/2020396750 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdd70072a90 0x7fdd70074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:32.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.289+0000 7fdd95c00640 1 --2- 172.21.15.94:0/2020396750 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdd70072a90 0x7fdd70074f50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdd840ae000 tx=0x7fdd8401a050 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:32.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.293+0000 7fdd8d600640 1 -- 172.21.15.94:0/2020396750 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdd7c05ee40 con 0x7fdd90154ed0 2024-09-10T23:38:32.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.489+0000 7fdd97000640 1 -- 172.21.15.94:0/2020396750 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol38681", "snap_name": "fake", "group_name": "subvol_grp51760", "target": ["mon-mgr", ""]}) -- 0x7fdd90157b70 con 0x7fdd70072a90 2024-09-10T23:38:32.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.525+0000 7fdd8d600640 1 -- 172.21.15.94:0/2020396750 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdd90157b70 con 0x7fdd70072a90 2024-09-10T23:38:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.525+0000 7fdd97000640 1 -- 172.21.15.94:0/2020396750 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdd70072a90 msgr2=0x7fdd70074f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.525+0000 7fdd97000640 1 --2- 172.21.15.94:0/2020396750 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdd70072a90 0x7fdd70074f50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdd840ae000 tx=0x7fdd8401a050 comp rx=0 tx=0).stop 2024-09-10T23:38:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.525+0000 7fdd97000640 1 -- 172.21.15.94:0/2020396750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd90154ed0 msgr2=0x7fdd901fba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.525+0000 7fdd97000640 1 --2- 172.21.15.94:0/2020396750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd90154ed0 0x7fdd901fba90 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c011ab0 tx=0x7fdd7c011f70 comp rx=0 tx=0).stop 2024-09-10T23:38:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.525+0000 7fdd97000640 1 -- 172.21.15.94:0/2020396750 shutdown_connections 2024-09-10T23:38:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.525+0000 7fdd97000640 1 --2- 172.21.15.94:0/2020396750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd90154860 0x7fdd900cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.525+0000 7fdd97000640 1 --2- 172.21.15.94:0/2020396750 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdd70072a90 0x7fdd70074f50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.525+0000 7fdd97000640 1 --2- 172.21.15.94:0/2020396750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd90154ed0 0x7fdd901fba90 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.525+0000 7fdd97000640 1 --2- 172.21.15.94:0/2020396750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd78003a10 0x7fdd90156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:32.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.525+0000 7fdd97000640 1 -- 172.21.15.94:0/2020396750 >> 172.21.15.94:0/2020396750 conn(0x7fdd900d0a40 msgr2=0x7fdd90158390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:32.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.525+0000 7fdd97000640 1 -- 172.21.15.94:0/2020396750 shutdown_connections 2024-09-10T23:38:32.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.525+0000 7fdd97000640 1 -- 172.21.15.94:0/2020396750 wait complete. 2024-09-10T23:38:32.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:32.547 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol10810/.meta) 2024-09-10T23:38:32.571 INFO:teuthology.orchestra.run.smithi094.stdout:[GLOBAL] 2024-09-10T23:38:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:version = 1 2024-09-10T23:38:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:type = subvolume 2024-09-10T23:38:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:path = /volumes/_nogroup/subvol10810/aeccf0fb-972d-488d-a02e-2e6cfb5661c7 2024-09-10T23:38:32.572 INFO:teuthology.orchestra.run.smithi094.stdout:state = complete 2024-09-10T23:38:32.572 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:32.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:32.574 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp51760/subvol38681/.meta) 2024-09-10T23:38:32.646 INFO:teuthology.orchestra.run.smithi094.stdout:[GLOBAL] 2024-09-10T23:38:32.646 INFO:teuthology.orchestra.run.smithi094.stdout:version = 1 2024-09-10T23:38:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:type = subvolume 2024-09-10T23:38:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:path = /volumes/subvol_grp51760/subvol38681/f648b0ef-e7c2-471f-ba1e-197b79fd4d86 2024-09-10T23:38:32.647 INFO:teuthology.orchestra.run.smithi094.stdout:state = complete 2024-09-10T23:38:32.647 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:32.649 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol10810 2024-09-10T23:38:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.837+0000 7faaaac00640 1 -- 172.21.15.94:0/319762827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa40c64f0 msgr2=0x7faaa40ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.837+0000 7faaaac00640 1 --2- 172.21.15.94:0/319762827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa40c64f0 0x7faaa40ce660 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009900 tx=0x7faa8c01c8e0 comp rx=0 tx=0).stop 2024-09-10T23:38:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.837+0000 7faaaac00640 1 -- 172.21.15.94:0/319762827 shutdown_connections 2024-09-10T23:38:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.837+0000 7faaaac00640 1 --2- 172.21.15.94:0/319762827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa40c64f0 0x7faaa40ce660 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.837+0000 7faaaac00640 1 --2- 172.21.15.94:0/319762827 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa40ce280 0x7faaa40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.837+0000 7faaaac00640 1 -- 172.21.15.94:0/319762827 >> 172.21.15.94:0/319762827 conn(0x7faaa40cfaf0 msgr2=0x7faaa40cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.837+0000 7faaaac00640 1 -- 172.21.15.94:0/319762827 shutdown_connections 2024-09-10T23:38:32.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.837+0000 7faaaac00640 1 -- 172.21.15.94:0/319762827 wait complete. 2024-09-10T23:38:32.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.837+0000 7faaaac00640 1 Processor -- start 2024-09-10T23:38:32.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.837+0000 7faaaac00640 1 -- start start 2024-09-10T23:38:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.837+0000 7faaaac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa406e7c0 0x7faaa414fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.837+0000 7faaaac00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa40c64f0 0x7faaa4151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.837+0000 7faaaac00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa40ce280 0x7faaa4151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:32.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.837+0000 7faaaac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faaa4167e90 con 0x7faaa406e7c0 2024-09-10T23:38:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.837+0000 7faaaac00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faaa4167d10 con 0x7faaa40c64f0 2024-09-10T23:38:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.837+0000 7faaaac00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7faaa4168010 con 0x7faaa40ce280 2024-09-10T23:38:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaa3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa406e7c0 0x7faaa414fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaa3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa406e7c0 0x7faaa414fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48998/0 (socket says 172.21.15.94:48998) 2024-09-10T23:38:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaa3e00640 1 -- 172.21.15.94:0/2407075274 learned_addr learned my addr 172.21.15.94:0/2407075274 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faa9b400640 1 --2- 172.21.15.94:0/2407075274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa40c64f0 0x7faaa4151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaa8c00640 1 --2- 172.21.15.94:0/2407075274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa40ce280 0x7faaa4151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaa3e00640 1 -- 172.21.15.94:0/2407075274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa40ce280 msgr2=0x7faaa4151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaa3e00640 1 --2- 172.21.15.94:0/2407075274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa40ce280 0x7faaa4151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaa3e00640 1 -- 172.21.15.94:0/2407075274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa40c64f0 msgr2=0x7faaa4151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaa3e00640 1 --2- 172.21.15.94:0/2407075274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa40c64f0 0x7faaa4151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaa3e00640 1 -- 172.21.15.94:0/2407075274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa40ae7f0 con 0x7faaa406e7c0 2024-09-10T23:38:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaa3e00640 1 --2- 172.21.15.94:0/2407075274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa406e7c0 0x7faaa414fbc0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7faa9400ba50 tx=0x7faa9400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:32.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaa0c00640 1 -- 172.21.15.94:0/2407075274 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94019070 con 0x7faaa406e7c0 2024-09-10T23:38:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaa0c00640 1 -- 172.21.15.94:0/2407075274 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa9400ccf0 con 0x7faaa406e7c0 2024-09-10T23:38:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaa0c00640 1 -- 172.21.15.94:0/2407075274 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa940146a0 con 0x7faaa406e7c0 2024-09-10T23:38:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaaac00640 1 -- 172.21.15.94:0/2407075274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa40aead0 con 0x7faaa406e7c0 2024-09-10T23:38:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaaac00640 1 -- 172.21.15.94:0/2407075274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa40af090 con 0x7faaa406e7c0 2024-09-10T23:38:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaaac00640 1 -- 172.21.15.94:0/2407075274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa40ce660 con 0x7faaa406e7c0 2024-09-10T23:38:32.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.841+0000 7faaa0c00640 1 -- 172.21.15.94:0/2407075274 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) ==== 96802+0+0 (secure 0 0 0) 0x7faa9401d3b0 con 0x7faaa406e7c0 2024-09-10T23:38:32.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.845+0000 7faaa0c00640 1 --2- 172.21.15.94:0/2407075274 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa80072a60 0x7faa80074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:32.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.845+0000 7faaa0c00640 1 -- 172.21.15.94:0/2407075274 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7faa94093b70 con 0x7faaa406e7c0 2024-09-10T23:38:32.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.845+0000 7faa9b400640 1 --2- 172.21.15.94:0/2407075274 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa80072a60 0x7faa80074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:32.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.845+0000 7faa9b400640 1 --2- 172.21.15.94:0/2407075274 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa80072a60 0x7faa80074f20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0af000 tx=0x7faa8c01b120 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:32.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.849+0000 7faaa0c00640 1 -- 172.21.15.94:0/2407075274 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faa9405e4a0 con 0x7faaa406e7c0 2024-09-10T23:38:32.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:32.869+0000 7faaa0c00640 1 -- 172.21.15.94:0/2407075274 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 20) ==== 96853+0+0 (secure 0 0 0) 0x7faa9405a710 con 0x7faaa406e7c0 2024-09-10T23:38:33.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.048+0000 7faaaac00640 1 -- 172.21.15.94:0/2407075274 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol10810", "target": ["mon-mgr", ""]}) -- 0x7faaa40d0830 con 0x7faa80072a60 2024-09-10T23:38:33.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.092+0000 7faaa0c00640 1 -- 172.21.15.94:0/2407075274 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faaa40d0830 con 0x7faa80072a60 2024-09-10T23:38:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.096+0000 7faaaac00640 1 -- 172.21.15.94:0/2407075274 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa80072a60 msgr2=0x7faa80074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.096+0000 7faaaac00640 1 --2- 172.21.15.94:0/2407075274 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa80072a60 0x7faa80074f20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0af000 tx=0x7faa8c01b120 comp rx=0 tx=0).stop 2024-09-10T23:38:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.096+0000 7faaaac00640 1 -- 172.21.15.94:0/2407075274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa406e7c0 msgr2=0x7faaa414fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.096+0000 7faaaac00640 1 --2- 172.21.15.94:0/2407075274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa406e7c0 0x7faaa414fbc0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7faa9400ba50 tx=0x7faa9400bf10 comp rx=0 tx=0).stop 2024-09-10T23:38:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.096+0000 7faaaac00640 1 -- 172.21.15.94:0/2407075274 shutdown_connections 2024-09-10T23:38:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.096+0000 7faaaac00640 1 --2- 172.21.15.94:0/2407075274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa40c64f0 0x7faaa4151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.096+0000 7faaaac00640 1 --2- 172.21.15.94:0/2407075274 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa80072a60 0x7faa80074f20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.096+0000 7faaaac00640 1 --2- 172.21.15.94:0/2407075274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa40ce280 0x7faaa4151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.096+0000 7faaaac00640 1 --2- 172.21.15.94:0/2407075274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa406e7c0 0x7faaa414fbc0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.096+0000 7faaaac00640 1 -- 172.21.15.94:0/2407075274 >> 172.21.15.94:0/2407075274 conn(0x7faaa40cfaf0 msgr2=0x7faaa40ca520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:33.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.096+0000 7faaaac00640 1 -- 172.21.15.94:0/2407075274 shutdown_connections 2024-09-10T23:38:33.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.096+0000 7faaaac00640 1 -- 172.21.15.94:0/2407075274 wait complete. 2024-09-10T23:38:33.102 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:33.100+0000 7f9ea2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:33.102 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:33.100+0000 7f9ea2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:33.102 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:33.100+0000 7f9ea2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:33.102 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:33.100+0000 7f9ea2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:33.102 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:33.100+0000 7f9ea2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:33.118 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:33.116+0000 7f9ea3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:33.118 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:33.116+0000 7f9ea3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:33.118 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:33.116+0000 7f9ea3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:33.118 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:33.116+0000 7f9ea3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:33.118 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:33.116+0000 7f9ea3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:38:33.122 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol38681 subvol_grp51760 2024-09-10T23:38:33.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 -- 172.21.15.94:0/2841810550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d00b2110 msgr2=0x7f36d00cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:33.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 --2- 172.21.15.94:0/2841810550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d00b2110 0x7f36d00cfda0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f36d0070530 tx=0x7f36b801c5d0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 -- 172.21.15.94:0/2841810550 shutdown_connections 2024-09-10T23:38:33.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 --2- 172.21.15.94:0/2841810550 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36d00d02e0 0x7f36d015f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 --2- 172.21.15.94:0/2841810550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d00b2110 0x7f36d00cfda0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 --2- 172.21.15.94:0/2841810550 >> v2:172.21.15.181:3300/0 conn(0x7f36d015ba30 0x7f36d00b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 -- 172.21.15.94:0/2841810550 >> 172.21.15.94:0/2841810550 conn(0x7f36d009aa70 msgr2=0x7f36d009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:33.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 -- 172.21.15.94:0/2841810550 shutdown_connections 2024-09-10T23:38:33.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 -- 172.21.15.94:0/2841810550 wait complete. 2024-09-10T23:38:33.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 Processor -- start 2024-09-10T23:38:33.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 -- start start 2024-09-10T23:38:33.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36d00b2110 0x7f36d0156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:33.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d00d02e0 0x7f36d00b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:33.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d015ba30 0x7f36d01f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:33.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f36d00af2d0 con 0x7f36d00d02e0 2024-09-10T23:38:33.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f36d00af150 con 0x7f36d015ba30 2024-09-10T23:38:33.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cf200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f36d00af450 con 0x7f36d00b2110 2024-09-10T23:38:33.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36c7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d00d02e0 0x7f36d00b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:33.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36c7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d00d02e0 0x7f36d00b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49028/0 (socket says 172.21.15.94:49028) 2024-09-10T23:38:33.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36c7e00640 1 -- 172.21.15.94:0/3589562028 learned_addr learned my addr 172.21.15.94:0/3589562028 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:33.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36ce800640 1 --2- 172.21.15.94:0/3589562028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d015ba30 0x7f36d01f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:33.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36cde00640 1 --2- 172.21.15.94:0/3589562028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36d00b2110 0x7f36d0156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:33.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36c7e00640 1 -- 172.21.15.94:0/3589562028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36d00b2110 msgr2=0x7f36d0156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:33.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36c7e00640 1 --2- 172.21.15.94:0/3589562028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36d00b2110 0x7f36d0156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36c7e00640 1 -- 172.21.15.94:0/3589562028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d015ba30 msgr2=0x7f36d01f6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:33.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36c7e00640 1 --2- 172.21.15.94:0/3589562028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d015ba30 0x7f36d01f6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36c7e00640 1 -- 172.21.15.94:0/3589562028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d01f78c0 con 0x7f36d00d02e0 2024-09-10T23:38:33.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.268+0000 7f36c7e00640 1 --2- 172.21.15.94:0/3589562028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d00d02e0 0x7f36d00b5420 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009850 tx=0x7f36b80a64e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:33.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.272+0000 7f36c6000640 1 -- 172.21.15.94:0/3589562028 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b80ac070 con 0x7f36d00d02e0 2024-09-10T23:38:33.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.272+0000 7f36c6000640 1 -- 172.21.15.94:0/3589562028 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36b8004320 con 0x7f36d00d02e0 2024-09-10T23:38:33.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.272+0000 7f36c6000640 1 -- 172.21.15.94:0/3589562028 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8005320 con 0x7f36d00d02e0 2024-09-10T23:38:33.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.272+0000 7f36cf200640 1 -- 172.21.15.94:0/3589562028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d01f7ba0 con 0x7f36d00d02e0 2024-09-10T23:38:33.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.272+0000 7f36cf200640 1 -- 172.21.15.94:0/3589562028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d00b4f00 con 0x7f36d00d02e0 2024-09-10T23:38:33.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.272+0000 7f36cf200640 1 -- 172.21.15.94:0/3589562028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d015f540 con 0x7f36d00d02e0 2024-09-10T23:38:33.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.272+0000 7f36c6000640 1 -- 172.21.15.94:0/3589562028 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 96853+0+0 (secure 0 0 0) 0x7f36b80054c0 con 0x7f36d00d02e0 2024-09-10T23:38:33.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.272+0000 7f36c6000640 1 --2- 172.21.15.94:0/3589562028 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f36ac072af0 0x7f36ac074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:33.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.272+0000 7f36c6000640 1 -- 172.21.15.94:0/3589562028 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f36b812e260 con 0x7f36d00d02e0 2024-09-10T23:38:33.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.272+0000 7f36cde00640 1 --2- 172.21.15.94:0/3589562028 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f36ac072af0 0x7f36ac074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:33.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.276+0000 7f36cde00640 1 --2- 172.21.15.94:0/3589562028 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f36ac072af0 0x7f36ac074fb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f36c000bad0 tx=0x7f36c000b2e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.276+0000 7f36c6000640 1 -- 172.21.15.94:0/3589562028 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f36b80164b0 con 0x7f36d00d02e0 2024-09-10T23:38:33.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.468+0000 7f36cf200640 1 -- 172.21.15.94:0/3589562028 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol38681", "group_name": "subvol_grp51760", "target": ["mon-mgr", ""]}) -- 0x7f36d01f7d70 con 0x7f36ac072af0 2024-09-10T23:38:33.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.508+0000 7f36c6000640 1 -- 172.21.15.94:0/3589562028 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f36d01f7d70 con 0x7f36ac072af0 2024-09-10T23:38:33.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.512+0000 7f36cf200640 1 -- 172.21.15.94:0/3589562028 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f36ac072af0 msgr2=0x7f36ac074fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:33.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.512+0000 7f36cf200640 1 --2- 172.21.15.94:0/3589562028 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f36ac072af0 0x7f36ac074fb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f36c000bad0 tx=0x7f36c000b2e0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.512+0000 7f36cf200640 1 -- 172.21.15.94:0/3589562028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d00d02e0 msgr2=0x7f36d00b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:33.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.512+0000 7f36cf200640 1 --2- 172.21.15.94:0/3589562028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d00d02e0 0x7f36d00b5420 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009850 tx=0x7f36b80a64e0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.512+0000 7f36cf200640 1 -- 172.21.15.94:0/3589562028 shutdown_connections 2024-09-10T23:38:33.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.512+0000 7f36cf200640 1 --2- 172.21.15.94:0/3589562028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d015ba30 0x7f36d01f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.512+0000 7f36cf200640 1 --2- 172.21.15.94:0/3589562028 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f36ac072af0 0x7f36ac074fb0 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f36c000bad0 tx=0x7f36c000b2e0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.512+0000 7f36cf200640 1 --2- 172.21.15.94:0/3589562028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d00d02e0 0x7f36d00b5420 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.512+0000 7f36cf200640 1 --2- 172.21.15.94:0/3589562028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36d00b2110 0x7f36d0156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.512+0000 7f36cf200640 1 -- 172.21.15.94:0/3589562028 >> 172.21.15.94:0/3589562028 conn(0x7f36d009aa70 msgr2=0x7f36d009ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:33.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.512+0000 7f36cf200640 1 -- 172.21.15.94:0/3589562028 shutdown_connections 2024-09-10T23:38:33.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.512+0000 7f36cf200640 1 -- 172.21.15.94:0/3589562028 wait complete. 2024-09-10T23:38:33.535 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol34412 2024-09-10T23:38:33.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.676+0000 7fb470400640 1 -- 172.21.15.94:0/2732560262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4680d0710 msgr2=0x7fb4680c5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:33.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.676+0000 7fb470400640 1 --2- 172.21.15.94:0/2732560262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4680d0710 0x7fb4680c5530 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fb454009930 tx=0x7fb45401c6d0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.676+0000 7fb470400640 1 -- 172.21.15.94:0/2732560262 shutdown_connections 2024-09-10T23:38:33.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.676+0000 7fb470400640 1 --2- 172.21.15.94:0/2732560262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4680d0710 0x7fb4680c5530 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.676+0000 7fb470400640 1 --2- 172.21.15.94:0/2732560262 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4681455d0 0x7fb4680c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.676+0000 7fb470400640 1 -- 172.21.15.94:0/2732560262 >> 172.21.15.94:0/2732560262 conn(0x7fb4680cfaf0 msgr2=0x7fb4680cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:33.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.676+0000 7fb470400640 1 -- 172.21.15.94:0/2732560262 shutdown_connections 2024-09-10T23:38:33.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.676+0000 7fb470400640 1 -- 172.21.15.94:0/2732560262 wait complete. 2024-09-10T23:38:33.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb470400640 1 Processor -- start 2024-09-10T23:38:33.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb470400640 1 -- start start 2024-09-10T23:38:33.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb470400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4680c5a70 0x7fb468139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:33.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb470400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4680d0710 0x7fb468139f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:33.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb470400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4681455d0 0x7fb4680ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:33.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb470400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb468169590 con 0x7fb4681455d0 2024-09-10T23:38:33.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb470400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb468169410 con 0x7fb4680c5a70 2024-09-10T23:38:33.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb470400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb468169710 con 0x7fb4680d0710 2024-09-10T23:38:33.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb46e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4681455d0 0x7fb4680ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:33.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb46da00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4680c5a70 0x7fb468139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:33.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb46e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4681455d0 0x7fb4680ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49050/0 (socket says 172.21.15.94:49050) 2024-09-10T23:38:33.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb46e400640 1 -- 172.21.15.94:0/4072954328 learned_addr learned my addr 172.21.15.94:0/4072954328 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:33.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb46d000640 1 --2- 172.21.15.94:0/4072954328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4680d0710 0x7fb468139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:33.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb46e400640 1 -- 172.21.15.94:0/4072954328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4680d0710 msgr2=0x7fb468139f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:33.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb46e400640 1 --2- 172.21.15.94:0/4072954328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4680d0710 0x7fb468139f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb46e400640 1 -- 172.21.15.94:0/4072954328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4680c5a70 msgr2=0x7fb468139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:33.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb46e400640 1 --2- 172.21.15.94:0/4072954328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4680c5a70 0x7fb468139940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:33.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb46e400640 1 -- 172.21.15.94:0/4072954328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46813a970 con 0x7fb4681455d0 2024-09-10T23:38:33.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb46e400640 1 --2- 172.21.15.94:0/4072954328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4681455d0 0x7fb4680ce550 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c7b0 tx=0x7fb46000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:33.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb465600640 1 -- 172.21.15.94:0/4072954328 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460016020 con 0x7fb4681455d0 2024-09-10T23:38:33.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb465600640 1 -- 172.21.15.94:0/4072954328 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb46000ab80 con 0x7fb4681455d0 2024-09-10T23:38:33.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb465600640 1 -- 172.21.15.94:0/4072954328 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46000ce60 con 0x7fb4681455d0 2024-09-10T23:38:33.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb470400640 1 -- 172.21.15.94:0/4072954328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46813ac50 con 0x7fb4681455d0 2024-09-10T23:38:33.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.680+0000 7fb470400640 1 -- 172.21.15.94:0/4072954328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46813b210 con 0x7fb4681455d0 2024-09-10T23:38:33.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.684+0000 7fb470400640 1 -- 172.21.15.94:0/4072954328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4680c5530 con 0x7fb4681455d0 2024-09-10T23:38:33.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.684+0000 7fb465600640 1 -- 172.21.15.94:0/4072954328 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 96853+0+0 (secure 0 0 0) 0x7fb46001a3b0 con 0x7fb4681455d0 2024-09-10T23:38:33.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.684+0000 7fb465600640 1 --2- 172.21.15.94:0/4072954328 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb448072b20 0x7fb448074fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:33.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.684+0000 7fb465600640 1 -- 172.21.15.94:0/4072954328 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fb460093e90 con 0x7fb4681455d0 2024-09-10T23:38:33.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.684+0000 7fb46da00640 1 --2- 172.21.15.94:0/4072954328 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb448072b20 0x7fb448074fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:33.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.684+0000 7fb46da00640 1 --2- 172.21.15.94:0/4072954328 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb448072b20 0x7fb448074fe0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00ab40 tx=0x7fb45c00a250 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:33.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.688+0000 7fb465600640 1 -- 172.21.15.94:0/4072954328 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb460020030 con 0x7fb4681455d0 2024-09-10T23:38:33.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.880+0000 7fb470400640 1 -- 172.21.15.94:0/4072954328 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol34412", "target": ["mon-mgr", ""]}) -- 0x7fb46813ba40 con 0x7fb448072b20 2024-09-10T23:38:33.891 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:38:33.888+0000 7f9e9fe00640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvolume 'subvol34412' is not ready for operation rm (use --force to override) 2024-09-10T23:38:33.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:33.888+0000 7fb465600640 1 -- 172.21.15.94:0/4072954328 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -11 subvolume 'subvol34412' is not ready for operation rm (use --force to override)) ==== 87+0+0 (secure 0 0 0) 0x7fb46813ba40 con 0x7fb448072b20 2024-09-10T23:38:33.891 INFO:teuthology.orchestra.run.smithi094.stderr:Error EAGAIN: subvolume 'subvol34412' is not ready for operation rm (use --force to override) 2024-09-10T23:38:33.901 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-09-10T23:38:33.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:33.902 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol34412/.meta) 2024-09-10T23:38:33.926 INFO:teuthology.orchestra.run.smithi094.stdout:[GLOBAL] 2024-09-10T23:38:33.926 INFO:teuthology.orchestra.run.smithi094.stdout:version = 1 2024-09-10T23:38:33.926 INFO:teuthology.orchestra.run.smithi094.stdout:type = clone 2024-09-10T23:38:33.926 INFO:teuthology.orchestra.run.smithi094.stdout:path = /volumes/_nogroup/subvol34412/95f4d40c-75cd-46d6-8e20-3be0ed14e557 2024-09-10T23:38:33.926 INFO:teuthology.orchestra.run.smithi094.stdout:state = pending 2024-09-10T23:38:33.926 INFO:teuthology.orchestra.run.smithi094.stdout:[source] 2024-09-10T23:38:33.926 INFO:teuthology.orchestra.run.smithi094.stdout:volume = fake 2024-09-10T23:38:33.927 INFO:teuthology.orchestra.run.smithi094.stdout:subvolume = fake 2024-09-10T23:38:33.927 INFO:teuthology.orchestra.run.smithi094.stdout:snapshot = fake 2024-09-10T23:38:33.928 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol34412 --force 2024-09-10T23:38:34.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf8c00640 1 -- 172.21.15.94:0/1644475638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd8004810 msgr2=0x7f5bd8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:34.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf8c00640 1 --2- 172.21.15.94:0/1644475638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd8004810 0x7f5bd8004c90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f5be00017f0 tx=0x7f5be001c8c0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf8c00640 1 -- 172.21.15.94:0/1644475638 shutdown_connections 2024-09-10T23:38:34.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf8c00640 1 --2- 172.21.15.94:0/1644475638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd8004810 0x7f5bd8004c90 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf8c00640 1 -- 172.21.15.94:0/1644475638 >> 172.21.15.94:0/1644475638 conn(0x7f5bf406cdf0 msgr2=0x7f5bf406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:34.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf8c00640 1 -- 172.21.15.94:0/1644475638 shutdown_connections 2024-09-10T23:38:34.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf8c00640 1 -- 172.21.15.94:0/1644475638 wait complete. 2024-09-10T23:38:34.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf8c00640 1 Processor -- start 2024-09-10T23:38:34.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf8c00640 1 -- start start 2024-09-10T23:38:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf8c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd8004810 0x7f5bf41585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf8c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bf41f6830 0x7f5bf41495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf8c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf40a9d30 0x7f5bf40b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf8c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf40b4190 con 0x7f5bd8004810 2024-09-10T23:38:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf8c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf40b4010 con 0x7f5bf40a9d30 2024-09-10T23:38:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf8c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf40b4310 con 0x7f5bf41f6830 2024-09-10T23:38:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd8004810 0x7f5bf41585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd8004810 0x7f5bf41585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45322/0 (socket says 172.21.15.94:45322) 2024-09-10T23:38:34.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf1e00640 1 -- 172.21.15.94:0/3777941066 learned_addr learned my addr 172.21.15.94:0/3777941066 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf1400640 1 --2- 172.21.15.94:0/3777941066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bf41f6830 0x7f5bf41495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.116+0000 7f5bf2800640 1 --2- 172.21.15.94:0/3777941066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf40a9d30 0x7f5bf40b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.120+0000 7f5bf1e00640 1 -- 172.21.15.94:0/3777941066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bf41f6830 msgr2=0x7f5bf41495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.120+0000 7f5bf1e00640 1 --2- 172.21.15.94:0/3777941066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bf41f6830 0x7f5bf41495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.120+0000 7f5bf1e00640 1 -- 172.21.15.94:0/3777941066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf40a9d30 msgr2=0x7f5bf40b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.120+0000 7f5bf1e00640 1 --2- 172.21.15.94:0/3777941066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf40a9d30 0x7f5bf40b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.120+0000 7f5bf1e00640 1 -- 172.21.15.94:0/3777941066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf40aa5a0 con 0x7f5bd8004810 2024-09-10T23:38:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.120+0000 7f5bf1e00640 1 --2- 172.21.15.94:0/3777941066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd8004810 0x7f5bf41585b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f5be4011dc0 tx=0x7f5be400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.120+0000 7f5bea000640 1 -- 172.21.15.94:0/3777941066 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4003980 con 0x7f5bd8004810 2024-09-10T23:38:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.120+0000 7f5bea000640 1 -- 172.21.15.94:0/3777941066 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5be4003b20 con 0x7f5bd8004810 2024-09-10T23:38:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.120+0000 7f5bea000640 1 -- 172.21.15.94:0/3777941066 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4004030 con 0x7f5bd8004810 2024-09-10T23:38:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.120+0000 7f5bf8c00640 1 -- 172.21.15.94:0/3777941066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf4157ad0 con 0x7f5bd8004810 2024-09-10T23:38:34.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.120+0000 7f5bf8c00640 1 -- 172.21.15.94:0/3777941066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf40aa190 con 0x7f5bd8004810 2024-09-10T23:38:34.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.120+0000 7f5bf8c00640 1 -- 172.21.15.94:0/3777941066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf41578c0 con 0x7f5bd8004810 2024-09-10T23:38:34.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.120+0000 7f5bea000640 1 -- 172.21.15.94:0/3777941066 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 96853+0+0 (secure 0 0 0) 0x7f5be40043e0 con 0x7f5bd8004810 2024-09-10T23:38:34.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.124+0000 7f5bea000640 1 --2- 172.21.15.94:0/3777941066 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bd4072af0 0x7f5bd4074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:34.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.124+0000 7f5bea000640 1 -- 172.21.15.94:0/3777941066 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7f5be4094ce0 con 0x7f5bd8004810 2024-09-10T23:38:34.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.124+0000 7f5bf1400640 1 --2- 172.21.15.94:0/3777941066 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bd4072af0 0x7f5bd4074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:34.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.124+0000 7f5bf1400640 1 --2- 172.21.15.94:0/3777941066 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bd4072af0 0x7f5bd4074fb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5be00017c0 tx=0x7f5be0019660 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:34.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.128+0000 7f5bea000640 1 -- 172.21.15.94:0/3777941066 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5be405f470 con 0x7f5bd8004810 2024-09-10T23:38:34.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.316+0000 7f5bf8c00640 1 -- 172.21.15.94:0/3777941066 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol34412", "force": true, "target": ["mon-mgr", ""]}) -- 0x7f5bf4158af0 con 0x7f5bd4072af0 2024-09-10T23:38:34.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.336+0000 7f5bea000640 1 -- 172.21.15.94:0/3777941066 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5be00ac040 con 0x7f5bd4072af0 2024-09-10T23:38:34.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.336+0000 7f5bf8c00640 1 -- 172.21.15.94:0/3777941066 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bd4072af0 msgr2=0x7f5bd4074fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:34.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.336+0000 7f5bf8c00640 1 --2- 172.21.15.94:0/3777941066 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bd4072af0 0x7f5bd4074fb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5be00017c0 tx=0x7f5be0019660 comp rx=0 tx=0).stop 2024-09-10T23:38:34.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.336+0000 7f5bf8c00640 1 -- 172.21.15.94:0/3777941066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd8004810 msgr2=0x7f5bf41585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:34.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.336+0000 7f5bf8c00640 1 --2- 172.21.15.94:0/3777941066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd8004810 0x7f5bf41585b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f5be4011dc0 tx=0x7f5be400f790 comp rx=0 tx=0).stop 2024-09-10T23:38:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.336+0000 7f5bf8c00640 1 -- 172.21.15.94:0/3777941066 shutdown_connections 2024-09-10T23:38:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.336+0000 7f5bf8c00640 1 --2- 172.21.15.94:0/3777941066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bf40a9d30 0x7f5bf40b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.336+0000 7f5bf8c00640 1 --2- 172.21.15.94:0/3777941066 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bd4072af0 0x7f5bd4074fb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.336+0000 7f5bf8c00640 1 --2- 172.21.15.94:0/3777941066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bf41f6830 0x7f5bf41495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.336+0000 7f5bf8c00640 1 --2- 172.21.15.94:0/3777941066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd8004810 0x7f5bf41585b0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.336+0000 7f5bf8c00640 1 -- 172.21.15.94:0/3777941066 >> 172.21.15.94:0/3777941066 conn(0x7f5bf406cdf0 msgr2=0x7f5bf40cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.336+0000 7f5bf8c00640 1 -- 172.21.15.94:0/3777941066 shutdown_connections 2024-09-10T23:38:34.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.336+0000 7f5bf8c00640 1 -- 172.21.15.94:0/3777941066 wait complete. 2024-09-10T23:38:34.362 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-10T23:38:34.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.508+0000 7fae3b200640 1 -- 172.21.15.94:0/654903645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae3c06fb10 msgr2=0x7fae3c06ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:34.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.508+0000 7fae3b200640 1 --2- 172.21.15.94:0/654903645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae3c06fb10 0x7fae3c06ff20 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0b0390 tx=0x7fae3401c890 comp rx=0 tx=0).stop 2024-09-10T23:38:34.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.508+0000 7fae3b200640 1 -- 172.21.15.94:0/654903645 shutdown_connections 2024-09-10T23:38:34.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.508+0000 7fae3b200640 1 --2- 172.21.15.94:0/654903645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae3c06fb10 0x7fae3c06ff20 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.508+0000 7fae3b200640 1 --2- 172.21.15.94:0/654903645 >> v2:172.21.15.181:3300/0 conn(0x7fae3c06d000 0x7fae3c073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.508+0000 7fae3b200640 1 -- 172.21.15.94:0/654903645 >> 172.21.15.94:0/654903645 conn(0x7fae3c06e610 msgr2=0x7fae3c0b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:34.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae3b200640 1 -- 172.21.15.94:0/654903645 shutdown_connections 2024-09-10T23:38:34.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae3b200640 1 -- 172.21.15.94:0/654903645 wait complete. 2024-09-10T23:38:34.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae3b200640 1 Processor -- start 2024-09-10T23:38:34.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae3b200640 1 -- start start 2024-09-10T23:38:34.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae3b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae3c06cc20 0x7fae3c0c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:34.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae3b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae3c06d000 0x7fae3c0ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:34.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae3b200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae3c06fb10 0x7fae3c1fb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:34.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae3b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1690b0 con 0x7fae3c06cc20 2024-09-10T23:38:34.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae3b200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c0cb180 con 0x7fae3c06d000 2024-09-10T23:38:34.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae3b200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fae3c169230 con 0x7fae3c06fb10 2024-09-10T23:38:34.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae39e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae3c06cc20 0x7fae3c0c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:34.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae39e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae3c06cc20 0x7fae3c0c6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45340/0 (socket says 172.21.15.94:45340) 2024-09-10T23:38:34.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae39e00640 1 -- 172.21.15.94:0/377347286 learned_addr learned my addr 172.21.15.94:0/377347286 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae31400640 1 --2- 172.21.15.94:0/377347286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae3c06d000 0x7fae3c0ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae3a800640 1 --2- 172.21.15.94:0/377347286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae3c06fb10 0x7fae3c1fb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae39e00640 1 -- 172.21.15.94:0/377347286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae3c06fb10 msgr2=0x7fae3c1fb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae39e00640 1 --2- 172.21.15.94:0/377347286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae3c06fb10 0x7fae3c1fb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae39e00640 1 -- 172.21.15.94:0/377347286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae3c06d000 msgr2=0x7fae3c0ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae39e00640 1 --2- 172.21.15.94:0/377347286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae3c06d000 0x7fae3c0ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae39e00640 1 -- 172.21.15.94:0/377347286 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3c0c7bd0 con 0x7fae3c06cc20 2024-09-10T23:38:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae39e00640 1 --2- 172.21.15.94:0/377347286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae3c06cc20 0x7fae3c0c6ad0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00c8a0 tx=0x7fae2c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae32a00640 1 -- 172.21.15.94:0/377347286 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c003b80 con 0x7fae3c06cc20 2024-09-10T23:38:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae32a00640 1 -- 172.21.15.94:0/377347286 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae2c003d40 con 0x7fae3c06cc20 2024-09-10T23:38:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae3b200640 1 -- 172.21.15.94:0/377347286 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3c0c7e70 con 0x7fae3c06cc20 2024-09-10T23:38:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae32a00640 1 -- 172.21.15.94:0/377347286 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00aa50 con 0x7fae3c06cc20 2024-09-10T23:38:34.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.512+0000 7fae3b200640 1 -- 172.21.15.94:0/377347286 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae3c0b0310 con 0x7fae3c06cc20 2024-09-10T23:38:34.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.516+0000 7fae3b200640 1 -- 172.21.15.94:0/377347286 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae3c073fd0 con 0x7fae3c06cc20 2024-09-10T23:38:34.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.516+0000 7fae32a00640 1 -- 172.21.15.94:0/377347286 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 96853+0+0 (secure 0 0 0) 0x7fae2c00abf0 con 0x7fae3c06cc20 2024-09-10T23:38:34.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.516+0000 7fae32a00640 1 --2- 172.21.15.94:0/377347286 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fae18072af0 0x7fae18074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:34.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.516+0000 7fae32a00640 1 -- 172.21.15.94:0/377347286 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5737+0+0 (secure 0 0 0) 0x7fae2c093b00 con 0x7fae3c06cc20 2024-09-10T23:38:34.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.516+0000 7fae31400640 1 --2- 172.21.15.94:0/377347286 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fae18072af0 0x7fae18074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:34.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.516+0000 7fae31400640 1 --2- 172.21.15.94:0/377347286 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fae18072af0 0x7fae18074fb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0701d0 tx=0x7fae240091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:34.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.520+0000 7fae32a00640 1 -- 172.21.15.94:0/377347286 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fae2c095e20 con 0x7fae3c06cc20 2024-09-10T23:38:34.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.708+0000 7fae3b200640 1 -- 172.21.15.94:0/377347286 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7fae3c15dba0 con 0x7fae18072af0 2024-09-10T23:38:34.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.712+0000 7fae32a00640 1 -- 172.21.15.94:0/377347286 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fae3c15dba0 con 0x7fae18072af0 2024-09-10T23:38:34.717 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-09-10T23:38:34.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.716+0000 7fae3b200640 1 -- 172.21.15.94:0/377347286 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fae18072af0 msgr2=0x7fae18074fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:34.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.716+0000 7fae3b200640 1 --2- 172.21.15.94:0/377347286 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fae18072af0 0x7fae18074fb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0701d0 tx=0x7fae240091e0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.716+0000 7fae3b200640 1 -- 172.21.15.94:0/377347286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae3c06cc20 msgr2=0x7fae3c0c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:34.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.716+0000 7fae3b200640 1 --2- 172.21.15.94:0/377347286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae3c06cc20 0x7fae3c0c6ad0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00c8a0 tx=0x7fae2c00cd60 comp rx=0 tx=0).stop 2024-09-10T23:38:34.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.716+0000 7fae3b200640 1 -- 172.21.15.94:0/377347286 shutdown_connections 2024-09-10T23:38:34.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.716+0000 7fae3b200640 1 --2- 172.21.15.94:0/377347286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae3c06d000 0x7fae3c0ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.716+0000 7fae3b200640 1 --2- 172.21.15.94:0/377347286 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fae18072af0 0x7fae18074fb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.716+0000 7fae3b200640 1 --2- 172.21.15.94:0/377347286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae3c06fb10 0x7fae3c1fb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.716+0000 7fae3b200640 1 --2- 172.21.15.94:0/377347286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae3c06cc20 0x7fae3c0c6ad0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:34.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.716+0000 7fae3b200640 1 -- 172.21.15.94:0/377347286 >> 172.21.15.94:0/377347286 conn(0x7fae3c06e610 msgr2=0x7fae3c0ce880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:34.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.716+0000 7fae3b200640 1 -- 172.21.15.94:0/377347286 shutdown_connections 2024-09-10T23:38:34.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34.716+0000 7fae3b200640 1 -- 172.21.15.94:0/377347286 wait complete. 2024-09-10T23:38:34.738 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T23:38:34.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T23:38:34.791 INFO:teuthology.orchestra.run.smithi094.stdout:2 2024-09-10T23:38:34.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:38:34.798 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:38:34.798 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:38:34.920 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:38:35.007 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:38:35.007 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:38:35.008 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:38:35.008 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:35.017 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:38:35.017 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:35.069 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:38:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.256+0000 7fb6d9800640 1 -- 172.21.15.94:0/134893517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6b8004810 msgr2=0x7fb6b8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.256+0000 7fb6d9800640 1 --2- 172.21.15.94:0/134893517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6b8004810 0x7fb6b8004c90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00ac10 tx=0x7fb6bc01cdd0 comp rx=0 tx=0).stop 2024-09-10T23:38:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.256+0000 7fb6d3400640 1 -- 172.21.15.94:0/134893517 reap_dead start 2024-09-10T23:38:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.256+0000 7fb6d9800640 1 -- 172.21.15.94:0/134893517 shutdown_connections 2024-09-10T23:38:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.256+0000 7fb6d9800640 1 -- 172.21.15.94:0/134893517 >> 172.21.15.94:0/134893517 conn(0x7fb6d40cfaf0 msgr2=0x7fb6d40cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.256+0000 7fb6d9800640 1 -- 172.21.15.94:0/134893517 shutdown_connections 2024-09-10T23:38:35.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.256+0000 7fb6d9800640 1 -- 172.21.15.94:0/134893517 wait complete. 2024-09-10T23:38:35.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d9800640 1 Processor -- start 2024-09-10T23:38:35.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d9800640 1 -- start start 2024-09-10T23:38:35.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d9800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6b8004810 0x7fb6d414d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:35.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d9800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6d41e28d0 0x7fb6d40c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:35.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d9800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d4156820 0x7fb6d40c5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d9800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41583e0 con 0x7fb6d4156820 2024-09-10T23:38:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d9800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d415b8b0 con 0x7fb6d41e28d0 2024-09-10T23:38:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d9800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d406ec50 con 0x7fb6b8004810 2024-09-10T23:38:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d4156820 0x7fb6d40c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d2a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6b8004810 0x7fb6d414d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d4156820 0x7fb6d40c5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45358/0 (socket says 172.21.15.94:45358) 2024-09-10T23:38:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d3400640 1 -- 172.21.15.94:0/3332759346 learned_addr learned my addr 172.21.15.94:0/3332759346 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6ca000640 1 --2- 172.21.15.94:0/3332759346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6d41e28d0 0x7fb6d40c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d3400640 1 -- 172.21.15.94:0/3332759346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6b8004810 msgr2=0x7fb6d414d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d3400640 1 --2- 172.21.15.94:0/3332759346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6b8004810 0x7fb6d414d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d3400640 1 -- 172.21.15.94:0/3332759346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6d41e28d0 msgr2=0x7fb6d40c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d3400640 1 --2- 172.21.15.94:0/3332759346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6d41e28d0 0x7fb6d40c4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:35.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d3400640 1 -- 172.21.15.94:0/3332759346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d41e5380 con 0x7fb6d4156820 2024-09-10T23:38:35.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d3400640 1 --2- 172.21.15.94:0/3332759346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d4156820 0x7fb6d40c5500 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40763b0 tx=0x7fb6cc006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:35.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6cb400640 1 -- 172.21.15.94:0/3332759346 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc00fa80 con 0x7fb6d4156820 2024-09-10T23:38:35.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6cb400640 1 -- 172.21.15.94:0/3332759346 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6cc017070 con 0x7fb6d4156820 2024-09-10T23:38:35.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6cb400640 1 -- 172.21.15.94:0/3332759346 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc011430 con 0x7fb6d4156820 2024-09-10T23:38:35.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d9800640 1 -- 172.21.15.94:0/3332759346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d4072c80 con 0x7fb6d4156820 2024-09-10T23:38:35.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.260+0000 7fb6d9800640 1 -- 172.21.15.94:0/3332759346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d4072f40 con 0x7fb6d4156820 2024-09-10T23:38:35.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.264+0000 7fb6d9800640 1 -- 172.21.15.94:0/3332759346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d40d0850 con 0x7fb6d4156820 2024-09-10T23:38:35.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.264+0000 7fb6cb400640 1 -- 172.21.15.94:0/3332759346 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7fb6cc0115d0 con 0x7fb6d4156820 2024-09-10T23:38:35.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.264+0000 7fb6cb400640 1 --2- 172.21.15.94:0/3332759346 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb6b0072c10 0x7fb6b00750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:35.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.264+0000 7fb6cb400640 1 -- 172.21.15.94:0/3332759346 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5793+0+0 (secure 0 0 0) 0x7fb6cc094670 con 0x7fb6d4156820 2024-09-10T23:38:35.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.264+0000 7fb6d2a00640 1 --2- 172.21.15.94:0/3332759346 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb6b0072c10 0x7fb6b00750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:35.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.264+0000 7fb6d2a00640 1 --2- 172.21.15.94:0/3332759346 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb6b0072c10 0x7fb6b00750d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4006fd0 tx=0x7fb6c4006b20 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.268+0000 7fb6cb400640 1 -- 172.21.15.94:0/3332759346 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb6d41e5380 con 0x7fb6d4156820 2024-09-10T23:38:35.301 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:38:35.304 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:38:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.456+0000 7fb6d9800640 1 -- 172.21.15.94:0/3332759346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb6d40d0380 con 0x7fb6d4156820 2024-09-10T23:38:35.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.456+0000 7fb6cb400640 1 -- 172.21.15.94:0/3332759346 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v115) ==== 80+0+0 (secure 0 0 0) 0x7fb6cc05f260 con 0x7fb6d4156820 2024-09-10T23:38:35.458 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:38:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.456+0000 7fb6d9800640 1 -- 172.21.15.94:0/3332759346 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb6b0072c10 msgr2=0x7fb6b00750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:35.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.456+0000 7fb6d9800640 1 --2- 172.21.15.94:0/3332759346 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb6b0072c10 0x7fb6b00750d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4006fd0 tx=0x7fb6c4006b20 comp rx=0 tx=0).stop 2024-09-10T23:38:35.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.456+0000 7fb6d9800640 1 -- 172.21.15.94:0/3332759346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d4156820 msgr2=0x7fb6d40c5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:35.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.456+0000 7fb6d9800640 1 --2- 172.21.15.94:0/3332759346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d4156820 0x7fb6d40c5500 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40763b0 tx=0x7fb6cc006660 comp rx=0 tx=0).stop 2024-09-10T23:38:35.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.456+0000 7fb6d9800640 1 -- 172.21.15.94:0/3332759346 shutdown_connections 2024-09-10T23:38:35.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.456+0000 7fb6d9800640 1 --2- 172.21.15.94:0/3332759346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6d41e28d0 0x7fb6d40c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:35.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.456+0000 7fb6d9800640 1 --2- 172.21.15.94:0/3332759346 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb6b0072c10 0x7fb6b00750d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:35.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.456+0000 7fb6d9800640 1 --2- 172.21.15.94:0/3332759346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d4156820 0x7fb6d40c5500 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:35.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.456+0000 7fb6d9800640 1 --2- 172.21.15.94:0/3332759346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6b8004810 0x7fb6d414d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:35.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.456+0000 7fb6d9800640 1 -- 172.21.15.94:0/3332759346 >> 172.21.15.94:0/3332759346 conn(0x7fb6d40cfaf0 msgr2=0x7fb6d40ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:35.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.456+0000 7fb6d9800640 1 -- 172.21.15.94:0/3332759346 shutdown_connections 2024-09-10T23:38:35.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.456+0000 7fb6d9800640 1 -- 172.21.15.94:0/3332759346 wait complete. 2024-09-10T23:38:35.480 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-09-10T23:38:35.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:35.480 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T23:38:35.497 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[47067]: fuse finished with error 0 and tester_r 0 2024-09-10T23:38:35.500 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:38:35.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:35.500 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T23:38:35.619 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:38:35.619 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:38:35.619 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:35.628 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:38:35.628 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:35.681 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:38:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.820+0000 7f3519800640 1 -- 172.21.15.94:0/3198726027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35140c5a70 msgr2=0x7f35140cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.820+0000 7f3519800640 1 --2- 172.21.15.94:0/3198726027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35140c5a70 0x7f35140cd9f0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f350c009cb0 tx=0x7f350c01cd70 comp rx=0 tx=0).stop 2024-09-10T23:38:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.820+0000 7f3519800640 1 -- 172.21.15.94:0/3198726027 shutdown_connections 2024-09-10T23:38:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.820+0000 7f3519800640 1 --2- 172.21.15.94:0/3198726027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35140c5a70 0x7f35140cd9f0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.820+0000 7f3519800640 1 --2- 172.21.15.94:0/3198726027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35140d0710 0x7f35140c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.820+0000 7f3519800640 1 -- 172.21.15.94:0/3198726027 >> 172.21.15.94:0/3198726027 conn(0x7f35140cfaf0 msgr2=0x7f35140cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.820+0000 7f3519800640 1 -- 172.21.15.94:0/3198726027 shutdown_connections 2024-09-10T23:38:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.820+0000 7f3519800640 1 -- 172.21.15.94:0/3198726027 wait complete. 2024-09-10T23:38:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.820+0000 7f3519800640 1 Processor -- start 2024-09-10T23:38:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.820+0000 7f3519800640 1 -- start start 2024-09-10T23:38:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3519800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35140c5a70 0x7f3514074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3519800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35140d0710 0x7f3514158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3519800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35141455d0 0x7f3514134ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3519800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f35141694a0 con 0x7f35140c5a70 2024-09-10T23:38:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3519800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3514169320 con 0x7f35141455d0 2024-09-10T23:38:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3519800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3514169620 con 0x7f35140d0710 2024-09-10T23:38:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3512a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35140c5a70 0x7f3514074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3512a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35140c5a70 0x7f3514074020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45378/0 (socket says 172.21.15.94:45378) 2024-09-10T23:38:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3512a00640 1 -- 172.21.15.94:0/3723227512 learned_addr learned my addr 172.21.15.94:0/3723227512 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f350a000640 1 --2- 172.21.15.94:0/3723227512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35140d0710 0x7f3514158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3513400640 1 --2- 172.21.15.94:0/3723227512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35141455d0 0x7f3514134ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3512a00640 1 -- 172.21.15.94:0/3723227512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35140d0710 msgr2=0x7f3514158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3512a00640 1 --2- 172.21.15.94:0/3723227512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35140d0710 0x7f3514158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3512a00640 1 -- 172.21.15.94:0/3723227512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35141455d0 msgr2=0x7f3514134ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3512a00640 1 --2- 172.21.15.94:0/3723227512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35141455d0 0x7f3514134ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3512a00640 1 -- 172.21.15.94:0/3723227512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351413a150 con 0x7f35140c5a70 2024-09-10T23:38:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3512a00640 1 --2- 172.21.15.94:0/3723227512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35140c5a70 0x7f3514074020 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f350400c3c0 tx=0x7f350400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f350b400640 1 -- 172.21.15.94:0/3723227512 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504017070 con 0x7f35140c5a70 2024-09-10T23:38:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f350b400640 1 -- 172.21.15.94:0/3723227512 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35040053c0 con 0x7f35140c5a70 2024-09-10T23:38:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f350b400640 1 -- 172.21.15.94:0/3723227512 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504003860 con 0x7f35140c5a70 2024-09-10T23:38:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3519800640 1 -- 172.21.15.94:0/3723227512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351413a430 con 0x7f35140c5a70 2024-09-10T23:38:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3519800640 1 -- 172.21.15.94:0/3723227512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3514158330 con 0x7f35140c5a70 2024-09-10T23:38:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f3519800640 1 -- 172.21.15.94:0/3723227512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35140cd9f0 con 0x7f35140c5a70 2024-09-10T23:38:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.824+0000 7f350b400640 1 -- 172.21.15.94:0/3723227512 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f3504003a00 con 0x7f35140c5a70 2024-09-10T23:38:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.828+0000 7f350b400640 1 --2- 172.21.15.94:0/3723227512 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f34f0072c10 0x7f34f00750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.828+0000 7f350b400640 1 -- 172.21.15.94:0/3723227512 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5793+0+0 (secure 0 0 0) 0x7f3504093450 con 0x7f35140c5a70 2024-09-10T23:38:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.828+0000 7f350a000640 1 --2- 172.21.15.94:0/3723227512 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f34f0072c10 0x7f34f00750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:35.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.828+0000 7f350a000640 1 --2- 172.21.15.94:0/3723227512 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f34f0072c10 0x7f34f00750d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f35140cdcb0 tx=0x7f34fc0091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:35.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:35.832+0000 7f350b400640 1 -- 172.21.15.94:0/3723227512 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f35140701c0 con 0x7f35140c5a70 2024-09-10T23:38:36.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.012+0000 7f3519800640 1 -- 172.21.15.94:0/3723227512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3514074840 con 0x7f35140c5a70 2024-09-10T23:38:36.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.012+0000 7f350b400640 1 -- 172.21.15.94:0/3723227512 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v115) ==== 80+0+0 (secure 0 0 0) 0x7f350405da80 con 0x7f35140c5a70 2024-09-10T23:38:36.016 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:38:36.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.016+0000 7f3519800640 1 -- 172.21.15.94:0/3723227512 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f34f0072c10 msgr2=0x7f34f00750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:36.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.016+0000 7f3519800640 1 --2- 172.21.15.94:0/3723227512 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f34f0072c10 0x7f34f00750d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f35140cdcb0 tx=0x7f34fc0091e0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.016+0000 7f3519800640 1 -- 172.21.15.94:0/3723227512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35140c5a70 msgr2=0x7f3514074020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:36.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.016+0000 7f3519800640 1 --2- 172.21.15.94:0/3723227512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35140c5a70 0x7f3514074020 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f350400c3c0 tx=0x7f350400c880 comp rx=0 tx=0).stop 2024-09-10T23:38:36.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.016+0000 7f3519800640 1 -- 172.21.15.94:0/3723227512 shutdown_connections 2024-09-10T23:38:36.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.016+0000 7f3519800640 1 --2- 172.21.15.94:0/3723227512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35141455d0 0x7f3514134ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.016+0000 7f3519800640 1 --2- 172.21.15.94:0/3723227512 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f34f0072c10 0x7f34f00750d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.016+0000 7f3519800640 1 --2- 172.21.15.94:0/3723227512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35140d0710 0x7f3514158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.016+0000 7f3519800640 1 --2- 172.21.15.94:0/3723227512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35140c5a70 0x7f3514074020 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.016+0000 7f3519800640 1 -- 172.21.15.94:0/3723227512 >> 172.21.15.94:0/3723227512 conn(0x7f35140cfaf0 msgr2=0x7f351413b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:36.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.016+0000 7f3519800640 1 -- 172.21.15.94:0/3723227512 shutdown_connections 2024-09-10T23:38:36.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.016+0000 7f3519800640 1 -- 172.21.15.94:0/3723227512 wait complete. 2024-09-10T23:38:36.038 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi181.front.sepia.ceph.com... 2024-09-10T23:38:36.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:36.038 DEBUG:teuthology.orchestra.run.smithi181:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T23:38:36.056 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[35684]: fuse finished with error 0 and tester_r 0 2024-09-10T23:38:36.058 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:38:36.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:36.059 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T23:38:36.225 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:38:36.225 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:38:36.226 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:36.234 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:38:36.234 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:38:36.234 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:38:36.234 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:36.276 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:38:36.276 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:38:36.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.416+0000 7f73ee400640 1 -- 172.21.15.94:0/3300700786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73cc003a10 msgr2=0x7f73cc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:36.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.416+0000 7f73ee400640 1 --2- 172.21.15.94:0/3300700786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73cc003a10 0x7f73cc003e30 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009b90 tx=0x7f73d801d270 comp rx=0 tx=0).stop 2024-09-10T23:38:36.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.416+0000 7f73e7e00640 1 -- 172.21.15.94:0/3300700786 reap_dead start 2024-09-10T23:38:36.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.416+0000 7f73ee400640 1 -- 172.21.15.94:0/3300700786 shutdown_connections 2024-09-10T23:38:36.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.416+0000 7f73ee400640 1 -- 172.21.15.94:0/3300700786 >> 172.21.15.94:0/3300700786 conn(0x7f73e809aa70 msgr2=0x7f73e809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:36.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.416+0000 7f73ee400640 1 -- 172.21.15.94:0/3300700786 shutdown_connections 2024-09-10T23:38:36.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.416+0000 7f73ee400640 1 -- 172.21.15.94:0/3300700786 wait complete. 2024-09-10T23:38:36.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73ee400640 1 Processor -- start 2024-09-10T23:38:36.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73ee400640 1 -- start start 2024-09-10T23:38:36.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73ee400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73cc003a10 0x7f73e80b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:36.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73ee400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e80b5e90 0x7f73e80c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:36.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73ee400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73e815a7b0 0x7f73e80aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:36.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73ee400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f73e8166b70 con 0x7f73e80b5e90 2024-09-10T23:38:36.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73ee400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f73e80b2bb0 con 0x7f73e815a7b0 2024-09-10T23:38:36.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73ee400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f73e8166cf0 con 0x7f73cc003a10 2024-09-10T23:38:36.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73e6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e80b5e90 0x7f73e80c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:36.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73e6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e80b5e90 0x7f73e80c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45382/0 (socket says 172.21.15.94:45382) 2024-09-10T23:38:36.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73e6a00640 1 -- 172.21.15.94:0/1209614993 learned_addr learned my addr 172.21.15.94:0/1209614993 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:36.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73e7e00640 1 --2- 172.21.15.94:0/1209614993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73e815a7b0 0x7f73e80aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:36.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73e7400640 1 --2- 172.21.15.94:0/1209614993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73cc003a10 0x7f73e80b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:36.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73e6a00640 1 -- 172.21.15.94:0/1209614993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73cc003a10 msgr2=0x7f73e80b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:36.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73e6a00640 1 --2- 172.21.15.94:0/1209614993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73cc003a10 0x7f73e80b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73e6a00640 1 -- 172.21.15.94:0/1209614993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73e815a7b0 msgr2=0x7f73e80aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:36.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73e6a00640 1 --2- 172.21.15.94:0/1209614993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73e815a7b0 0x7f73e80aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73e6a00640 1 -- 172.21.15.94:0/1209614993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e815af80 con 0x7f73e80b5e90 2024-09-10T23:38:36.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73e6a00640 1 --2- 172.21.15.94:0/1209614993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e80b5e90 0x7f73e80c8090 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f73d40066e0 tx=0x7f73d40039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:36.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73df400640 1 -- 172.21.15.94:0/1209614993 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4015070 con 0x7f73e80b5e90 2024-09-10T23:38:36.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73df400640 1 -- 172.21.15.94:0/1209614993 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73d4011900 con 0x7f73e80b5e90 2024-09-10T23:38:36.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73df400640 1 -- 172.21.15.94:0/1209614993 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4011d50 con 0x7f73e80b5e90 2024-09-10T23:38:36.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73ee400640 1 -- 172.21.15.94:0/1209614993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e80b50d0 con 0x7f73e80b5e90 2024-09-10T23:38:36.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.420+0000 7f73ee400640 1 -- 172.21.15.94:0/1209614993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e80b5690 con 0x7f73e80b5e90 2024-09-10T23:38:36.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.424+0000 7f73ee400640 1 -- 172.21.15.94:0/1209614993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73e80b1e70 con 0x7f73e80b5e90 2024-09-10T23:38:36.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.424+0000 7f73df400640 1 -- 172.21.15.94:0/1209614993 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f73d40103c0 con 0x7f73e80b5e90 2024-09-10T23:38:36.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.424+0000 7f73df400640 1 --2- 172.21.15.94:0/1209614993 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73c8072c40 0x7f73c8075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:36.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.424+0000 7f73df400640 1 -- 172.21.15.94:0/1209614993 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5793+0+0 (secure 0 0 0) 0x7f73d4094e70 con 0x7f73e80b5e90 2024-09-10T23:38:36.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.424+0000 7f73e7400640 1 --2- 172.21.15.94:0/1209614993 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73c8072c40 0x7f73c8075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:36.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.424+0000 7f73e7400640 1 --2- 172.21.15.94:0/1209614993 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73c8072c40 0x7f73c8075100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f73d80ae000 tx=0x7f73d801a010 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:36.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.428+0000 7f73df400640 1 -- 172.21.15.94:0/1209614993 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f73d405faa0 con 0x7f73e80b5e90 2024-09-10T23:38:36.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.624+0000 7f73ee400640 1 -- 172.21.15.94:0/1209614993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73e80d0710 con 0x7f73e80b5e90 2024-09-10T23:38:36.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.628+0000 7f73df400640 1 -- 172.21.15.94:0/1209614993 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5241 (secure 0 0 0) 0x7f73d405f3a0 con 0x7f73e80b5e90 2024-09-10T23:38:36.629 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:36.630 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":79,"btime":"2024-09-10T23:38:28:841812+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7090,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3097381199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3097381199},{"type":"v1","addr":"172.21.15.181:6835","nonce":3097381199}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2654663117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2654663117},{"type":"v1","addr":"172.21.15.94:6833","nonce":2654663117}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7530,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1334394570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1334394570},{"type":"v1","addr":"172.21.15.94:6835","nonce":1334394570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:38:18.757602+0000","modified":"2024-09-10T23:38:21.765704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196283":{"gid":196283,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6834/1748173202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":1748173202},{"type":"v1","addr":"172.21.15.181:6834","nonce":1748173202}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196283,"qdb_cluster":[196283]},"id":9}]} 2024-09-10T23:38:36.630 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 79 2024-09-10T23:38:36.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.628+0000 7f73ee400640 1 -- 172.21.15.94:0/1209614993 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73c8072c40 msgr2=0x7f73c8075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:36.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.628+0000 7f73ee400640 1 --2- 172.21.15.94:0/1209614993 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73c8072c40 0x7f73c8075100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f73d80ae000 tx=0x7f73d801a010 comp rx=0 tx=0).stop 2024-09-10T23:38:36.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.628+0000 7f73ee400640 1 -- 172.21.15.94:0/1209614993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e80b5e90 msgr2=0x7f73e80c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:36.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.628+0000 7f73ee400640 1 --2- 172.21.15.94:0/1209614993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e80b5e90 0x7f73e80c8090 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f73d40066e0 tx=0x7f73d40039d0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.628+0000 7f73ee400640 1 -- 172.21.15.94:0/1209614993 shutdown_connections 2024-09-10T23:38:36.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.628+0000 7f73ee400640 1 --2- 172.21.15.94:0/1209614993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73e815a7b0 0x7f73e80aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.628+0000 7f73ee400640 1 --2- 172.21.15.94:0/1209614993 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73c8072c40 0x7f73c8075100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.628+0000 7f73ee400640 1 --2- 172.21.15.94:0/1209614993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e80b5e90 0x7f73e80c8090 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.628+0000 7f73ee400640 1 --2- 172.21.15.94:0/1209614993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73cc003a10 0x7f73e80b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.628+0000 7f73ee400640 1 -- 172.21.15.94:0/1209614993 >> 172.21.15.94:0/1209614993 conn(0x7f73e809aa70 msgr2=0x7f73e815b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:36.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.628+0000 7f73ee400640 1 -- 172.21.15.94:0/1209614993 shutdown_connections 2024-09-10T23:38:36.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.628+0000 7f73ee400640 1 -- 172.21.15.94:0/1209614993 wait complete. 2024-09-10T23:38:36.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:38:36.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.788+0000 7f8a77200640 1 -- 172.21.15.94:0/4132050139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a78073630 msgr2=0x7f8a78073a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:36.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.788+0000 7f8a77200640 1 --2- 172.21.15.94:0/4132050139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a78073630 0x7f8a78073a10 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009900 tx=0x7f8a6801c8b0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.788+0000 7f8a77200640 1 -- 172.21.15.94:0/4132050139 shutdown_connections 2024-09-10T23:38:36.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.788+0000 7f8a77200640 1 --2- 172.21.15.94:0/4132050139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a7815af30 0x7f8a7806f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.788+0000 7f8a77200640 1 --2- 172.21.15.94:0/4132050139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a78073630 0x7f8a78073a10 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.788+0000 7f8a77200640 1 -- 172.21.15.94:0/4132050139 >> 172.21.15.94:0/4132050139 conn(0x7f8a780cd6b0 msgr2=0x7f8a780cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:36.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.788+0000 7f8a77200640 1 -- 172.21.15.94:0/4132050139 shutdown_connections 2024-09-10T23:38:36.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.788+0000 7f8a77200640 1 -- 172.21.15.94:0/4132050139 wait complete. 2024-09-10T23:38:36.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.788+0000 7f8a77200640 1 Processor -- start 2024-09-10T23:38:36.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.788+0000 7f8a77200640 1 -- start start 2024-09-10T23:38:36.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a77200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a78073630 0x7f8a780c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:36.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a77200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a7815af30 0x7f8a7814db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:36.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a77200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a7815e580 0x7f8a78151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:36.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a77200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8a781684c0 con 0x7f8a7815af30 2024-09-10T23:38:36.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a77200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8a78168340 con 0x7f8a7815e580 2024-09-10T23:38:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a77200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8a78168640 con 0x7f8a78073630 2024-09-10T23:38:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a75400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a7815af30 0x7f8a7814db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a75400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a7815af30 0x7f8a7814db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45408/0 (socket says 172.21.15.94:45408) 2024-09-10T23:38:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a75400640 1 -- 172.21.15.94:0/303787189 learned_addr learned my addr 172.21.15.94:0/303787189 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a75e00640 1 --2- 172.21.15.94:0/303787189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a78073630 0x7f8a780c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a76800640 1 --2- 172.21.15.94:0/303787189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a7815e580 0x7f8a78151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a75400640 1 -- 172.21.15.94:0/303787189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a78073630 msgr2=0x7f8a780c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a75400640 1 --2- 172.21.15.94:0/303787189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a78073630 0x7f8a780c39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a75400640 1 -- 172.21.15.94:0/303787189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a7815e580 msgr2=0x7f8a78151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a75400640 1 --2- 172.21.15.94:0/303787189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a7815e580 0x7f8a78151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a75400640 1 -- 172.21.15.94:0/303787189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7814ee80 con 0x7f8a7815af30 2024-09-10T23:38:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a75400640 1 --2- 172.21.15.94:0/303787189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a7815af30 0x7f8a7814db40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400d980 tx=0x7f8a6400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:36.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a6e000640 1 -- 172.21.15.94:0/303787189 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64017070 con 0x7f8a7815af30 2024-09-10T23:38:36.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a6e000640 1 -- 172.21.15.94:0/303787189 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a64009cc0 con 0x7f8a7815af30 2024-09-10T23:38:36.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a6e000640 1 -- 172.21.15.94:0/303787189 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a640106c0 con 0x7f8a7815af30 2024-09-10T23:38:36.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a77200640 1 -- 172.21.15.94:0/303787189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7814d060 con 0x7f8a7815af30 2024-09-10T23:38:36.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a77200640 1 -- 172.21.15.94:0/303787189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a7814d620 con 0x7f8a7815af30 2024-09-10T23:38:36.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.792+0000 7f8a77200640 1 -- 172.21.15.94:0/303787189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a78073a10 con 0x7f8a7815af30 2024-09-10T23:38:36.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.796+0000 7f8a6e000640 1 -- 172.21.15.94:0/303787189 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f8a64010860 con 0x7f8a7815af30 2024-09-10T23:38:36.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.796+0000 7f8a6e000640 1 --2- 172.21.15.94:0/303787189 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8a58072c10 0x7f8a580750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:36.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.796+0000 7f8a6e000640 1 -- 172.21.15.94:0/303787189 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5793+0+0 (secure 0 0 0) 0x7f8a64094240 con 0x7f8a7815af30 2024-09-10T23:38:36.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.796+0000 7f8a75e00640 1 --2- 172.21.15.94:0/303787189 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8a58072c10 0x7f8a580750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:36.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.796+0000 7f8a75e00640 1 --2- 172.21.15.94:0/303787189 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8a58072c10 0x7f8a580750d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8a680af000 tx=0x7f8a6801b0f0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:36.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.800+0000 7f8a6e000640 1 -- 172.21.15.94:0/303787189 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8a6405e7e0 con 0x7f8a7815af30 2024-09-10T23:38:37.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:36.996+0000 7f8a77200640 1 -- 172.21.15.94:0/303787189 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a7815c990 con 0x7f8a7815af30 2024-09-10T23:38:37.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.000+0000 7f8a6e000640 1 -- 172.21.15.94:0/303787189 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5241 (secure 0 0 0) 0x7f8a64063210 con 0x7f8a7815af30 2024-09-10T23:38:37.002 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:37.002 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":79,"btime":"2024-09-10T23:38:28:841812+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7090,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3097381199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3097381199},{"type":"v1","addr":"172.21.15.181:6835","nonce":3097381199}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2654663117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2654663117},{"type":"v1","addr":"172.21.15.94:6833","nonce":2654663117}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7530,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1334394570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1334394570},{"type":"v1","addr":"172.21.15.94:6835","nonce":1334394570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:38:18.757602+0000","modified":"2024-09-10T23:38:21.765704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196283":{"gid":196283,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6834/1748173202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":1748173202},{"type":"v1","addr":"172.21.15.181:6834","nonce":1748173202}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196283,"qdb_cluster":[196283]},"id":9}]} 2024-09-10T23:38:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 79 2024-09-10T23:38:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.000+0000 7f8a77200640 1 -- 172.21.15.94:0/303787189 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8a58072c10 msgr2=0x7f8a580750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.000+0000 7f8a77200640 1 --2- 172.21.15.94:0/303787189 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8a58072c10 0x7f8a580750d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8a680af000 tx=0x7f8a6801b0f0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.000+0000 7f8a77200640 1 -- 172.21.15.94:0/303787189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a7815af30 msgr2=0x7f8a7814db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.000+0000 7f8a77200640 1 --2- 172.21.15.94:0/303787189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a7815af30 0x7f8a7814db40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400d980 tx=0x7f8a6400de40 comp rx=0 tx=0).stop 2024-09-10T23:38:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.000+0000 7f8a77200640 1 -- 172.21.15.94:0/303787189 shutdown_connections 2024-09-10T23:38:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.000+0000 7f8a77200640 1 --2- 172.21.15.94:0/303787189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a7815e580 0x7f8a78151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.000+0000 7f8a77200640 1 --2- 172.21.15.94:0/303787189 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8a58072c10 0x7f8a580750d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.000+0000 7f8a77200640 1 --2- 172.21.15.94:0/303787189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a7815af30 0x7f8a7814db40 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.000+0000 7f8a77200640 1 --2- 172.21.15.94:0/303787189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a78073630 0x7f8a780c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.000+0000 7f8a77200640 1 -- 172.21.15.94:0/303787189 >> 172.21.15.94:0/303787189 conn(0x7f8a780cd6b0 msgr2=0x7f8a7814bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.000+0000 7f8a77200640 1 -- 172.21.15.94:0/303787189 shutdown_connections 2024-09-10T23:38:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.000+0000 7f8a77200640 1 -- 172.21.15.94:0/303787189 wait complete. 2024-09-10T23:38:37.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:38:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.160+0000 7fb667400640 1 -- 172.21.15.94:0/595008334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb66015f5c0 msgr2=0x7fb6600cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.160+0000 7fb667400640 1 --2- 172.21.15.94:0/595008334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb66015f5c0 0x7fb6600cfb60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb650009840 tx=0x7fb65001c700 comp rx=0 tx=0).stop 2024-09-10T23:38:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.160+0000 7fb667400640 1 -- 172.21.15.94:0/595008334 shutdown_connections 2024-09-10T23:38:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.160+0000 7fb667400640 1 --2- 172.21.15.94:0/595008334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb66015f5c0 0x7fb6600cfb60 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.160+0000 7fb667400640 1 -- 172.21.15.94:0/595008334 >> 172.21.15.94:0/595008334 conn(0x7fb6600d1c60 msgr2=0x7fb6600d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.160+0000 7fb667400640 1 -- 172.21.15.94:0/595008334 shutdown_connections 2024-09-10T23:38:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.160+0000 7fb667400640 1 -- 172.21.15.94:0/595008334 wait complete. 2024-09-10T23:38:37.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb667400640 1 Processor -- start 2024-09-10T23:38:37.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb667400640 1 -- start start 2024-09-10T23:38:37.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb667400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6600af310 0x7fb6601f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:37.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb667400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6600b2bb0 0x7fb6600c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:37.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb667400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb66015f5c0 0x7fb6601f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:37.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb667400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb66016a0c0 con 0x7fb66015f5c0 2024-09-10T23:38:37.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb667400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb660169f40 con 0x7fb6600af310 2024-09-10T23:38:37.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb667400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb66016a240 con 0x7fb6600b2bb0 2024-09-10T23:38:37.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb665400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb66015f5c0 0x7fb6601f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:37.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb664a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6600af310 0x7fb6601f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:37.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb665400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb66015f5c0 0x7fb6601f6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45422/0 (socket says 172.21.15.94:45422) 2024-09-10T23:38:37.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb665400640 1 -- 172.21.15.94:0/210304709 learned_addr learned my addr 172.21.15.94:0/210304709 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:37.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb657e00640 1 --2- 172.21.15.94:0/210304709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6600b2bb0 0x7fb6600c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:37.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb664a00640 1 -- 172.21.15.94:0/210304709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6600b2bb0 msgr2=0x7fb6600c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:37.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb664a00640 1 --2- 172.21.15.94:0/210304709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6600b2bb0 0x7fb6600c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb664a00640 1 -- 172.21.15.94:0/210304709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb66015f5c0 msgr2=0x7fb6601f6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:37.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb664a00640 1 --2- 172.21.15.94:0/210304709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb66015f5c0 0x7fb6601f6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb664a00640 1 -- 172.21.15.94:0/210304709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6601f9500 con 0x7fb6600af310 2024-09-10T23:38:37.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb664a00640 1 --2- 172.21.15.94:0/210304709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6600af310 0x7fb6601f8c00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb6500ac000 tx=0x7fb6500a6560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:37.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb65d600640 1 -- 172.21.15.94:0/210304709 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6500bd020 con 0x7fb6600af310 2024-09-10T23:38:37.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb65d600640 1 -- 172.21.15.94:0/210304709 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb650004130 con 0x7fb6600af310 2024-09-10T23:38:37.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb65d600640 1 -- 172.21.15.94:0/210304709 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650003600 con 0x7fb6600af310 2024-09-10T23:38:37.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb667400640 1 -- 172.21.15.94:0/210304709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6601f97e0 con 0x7fb6600af310 2024-09-10T23:38:37.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.164+0000 7fb667400640 1 -- 172.21.15.94:0/210304709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6600cc960 con 0x7fb6600af310 2024-09-10T23:38:37.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.168+0000 7fb667400640 1 -- 172.21.15.94:0/210304709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6600af7a0 con 0x7fb6600af310 2024-09-10T23:38:37.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.168+0000 7fb65d600640 1 -- 172.21.15.94:0/210304709 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7fb6500ca060 con 0x7fb6600af310 2024-09-10T23:38:37.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.168+0000 7fb65d600640 1 --2- 172.21.15.94:0/210304709 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb63c072c10 0x7fb63c0750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:37.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.168+0000 7fb65d600640 1 -- 172.21.15.94:0/210304709 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5793+0+0 (secure 0 0 0) 0x7fb65012eb20 con 0x7fb6600af310 2024-09-10T23:38:37.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.168+0000 7fb657e00640 1 --2- 172.21.15.94:0/210304709 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb63c072c10 0x7fb63c0750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:37.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.168+0000 7fb657e00640 1 --2- 172.21.15.94:0/210304709 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb63c072c10 0x7fb63c0750d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb6601f8b60 tx=0x7fb648007cd0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:37.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.172+0000 7fb65d600640 1 -- 172.21.15.94:0/210304709 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb650016410 con 0x7fb6600af310 2024-09-10T23:38:37.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.340+0000 7fb667400640 1 -- 172.21.15.94:0/210304709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb660156de0 con 0x7fb6600af310 2024-09-10T23:38:37.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.340+0000 7fb65d600640 1 -- 172.21.15.94:0/210304709 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16831 (secure 0 0 0) 0x7fb6500f9270 con 0x7fb6600af310 2024-09-10T23:38:37.343 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:37.343 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":115,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:38:34.921450+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:38:16.443089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T23:38:17.716329+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:38:37.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.340+0000 7fb667400640 1 -- 172.21.15.94:0/210304709 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb63c072c10 msgr2=0x7fb63c0750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:37.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.340+0000 7fb667400640 1 --2- 172.21.15.94:0/210304709 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb63c072c10 0x7fb63c0750d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb6601f8b60 tx=0x7fb648007cd0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.340+0000 7fb667400640 1 -- 172.21.15.94:0/210304709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6600af310 msgr2=0x7fb6601f8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:37.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.340+0000 7fb667400640 1 --2- 172.21.15.94:0/210304709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6600af310 0x7fb6601f8c00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb6500ac000 tx=0x7fb6500a6560 comp rx=0 tx=0).stop 2024-09-10T23:38:37.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.340+0000 7fb667400640 1 -- 172.21.15.94:0/210304709 shutdown_connections 2024-09-10T23:38:37.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.340+0000 7fb667400640 1 --2- 172.21.15.94:0/210304709 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb63c072c10 0x7fb63c0750d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.340+0000 7fb667400640 1 --2- 172.21.15.94:0/210304709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb66015f5c0 0x7fb6601f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.340+0000 7fb667400640 1 --2- 172.21.15.94:0/210304709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6600b2bb0 0x7fb6600c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.340+0000 7fb667400640 1 --2- 172.21.15.94:0/210304709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6600af310 0x7fb6601f8c00 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.340+0000 7fb667400640 1 -- 172.21.15.94:0/210304709 >> 172.21.15.94:0/210304709 conn(0x7fb6600d1c60 msgr2=0x7fb66015d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:37.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.340+0000 7fb667400640 1 -- 172.21.15.94:0/210304709 shutdown_connections 2024-09-10T23:38:37.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.340+0000 7fb667400640 1 -- 172.21.15.94:0/210304709 wait complete. 2024-09-10T23:38:37.365 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T23:38:37.365 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:38:37.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.504+0000 7faa72e00640 1 -- 172.21.15.94:0/3626906176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa6c15e1e0 msgr2=0x7faa6c0740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.504+0000 7faa72e00640 1 --2- 172.21.15.94:0/3626906176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa6c15e1e0 0x7faa6c0740e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00aa10 tx=0x7faa5c01dc00 comp rx=0 tx=0).stop 2024-09-10T23:38:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.504+0000 7faa72e00640 1 -- 172.21.15.94:0/3626906176 shutdown_connections 2024-09-10T23:38:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.504+0000 7faa72e00640 1 --2- 172.21.15.94:0/3626906176 >> v2:172.21.15.181:3300/0 conn(0x7faa6c074620 0x7faa6c0b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.504+0000 7faa72e00640 1 --2- 172.21.15.94:0/3626906176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa6c15e1e0 0x7faa6c0740e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.504+0000 7faa72e00640 1 -- 172.21.15.94:0/3626906176 >> 172.21.15.94:0/3626906176 conn(0x7faa6c0b1840 msgr2=0x7faa6c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.504+0000 7faa72e00640 1 -- 172.21.15.94:0/3626906176 shutdown_connections 2024-09-10T23:38:37.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.504+0000 7faa72e00640 1 -- 172.21.15.94:0/3626906176 wait complete. 2024-09-10T23:38:37.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.504+0000 7faa72e00640 1 Processor -- start 2024-09-10T23:38:37.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.504+0000 7faa72e00640 1 -- start start 2024-09-10T23:38:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa72e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa6c073d00 0x7faa6c13d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa72e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c074620 0x7faa6c1f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa72e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa6c15e1e0 0x7faa6c14f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa72e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c0b40a0 con 0x7faa6c074620 2024-09-10T23:38:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa72e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c0b3f20 con 0x7faa6c15e1e0 2024-09-10T23:38:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa72e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7faa6c0b4220 con 0x7faa6c073d00 2024-09-10T23:38:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa71000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c074620 0x7faa6c1f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa71000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c074620 0x7faa6c1f74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45424/0 (socket says 172.21.15.94:45424) 2024-09-10T23:38:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa72400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa6c15e1e0 0x7faa6c14f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa71000640 1 -- 172.21.15.94:0/1284089423 learned_addr learned my addr 172.21.15.94:0/1284089423 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa71a00640 1 --2- 172.21.15.94:0/1284089423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa6c073d00 0x7faa6c13d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa71000640 1 -- 172.21.15.94:0/1284089423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa6c073d00 msgr2=0x7faa6c13d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa71000640 1 --2- 172.21.15.94:0/1284089423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa6c073d00 0x7faa6c13d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa71000640 1 -- 172.21.15.94:0/1284089423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa6c15e1e0 msgr2=0x7faa6c14f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa71000640 1 --2- 172.21.15.94:0/1284089423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa6c15e1e0 0x7faa6c14f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa71000640 1 -- 172.21.15.94:0/1284089423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c074420 con 0x7faa6c074620 2024-09-10T23:38:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa71000640 1 --2- 172.21.15.94:0/1284089423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c074620 0x7faa6c1f74a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00a960 tx=0x7faa5c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa69600640 1 -- 172.21.15.94:0/1284089423 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0b3070 con 0x7faa6c074620 2024-09-10T23:38:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa69600640 1 -- 172.21.15.94:0/1284089423 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa5c0055f0 con 0x7faa6c074620 2024-09-10T23:38:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa69600640 1 -- 172.21.15.94:0/1284089423 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c004930 con 0x7faa6c074620 2024-09-10T23:38:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa72e00640 1 -- 172.21.15.94:0/1284089423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c1f8420 con 0x7faa6c074620 2024-09-10T23:38:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa72e00640 1 -- 172.21.15.94:0/1284089423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c1f8700 con 0x7faa6c074620 2024-09-10T23:38:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.508+0000 7faa72e00640 1 -- 172.21.15.94:0/1284089423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6c0b34d0 con 0x7faa6c074620 2024-09-10T23:38:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.512+0000 7faa69600640 1 -- 172.21.15.94:0/1284089423 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7faa5c00b040 con 0x7faa6c074620 2024-09-10T23:38:37.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.512+0000 7faa69600640 1 --2- 172.21.15.94:0/1284089423 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa50072c10 0x7faa500750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:37.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.512+0000 7faa69600640 1 -- 172.21.15.94:0/1284089423 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5793+0+0 (secure 0 0 0) 0x7faa5c12de20 con 0x7faa6c074620 2024-09-10T23:38:37.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.512+0000 7faa71a00640 1 --2- 172.21.15.94:0/1284089423 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa50072c10 0x7faa500750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:37.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.512+0000 7faa71a00640 1 --2- 172.21.15.94:0/1284089423 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa50072c10 0x7faa500750d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faa60007fd0 tx=0x7faa60009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:37.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.516+0000 7faa69600640 1 -- 172.21.15.94:0/1284089423 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faa5c017820 con 0x7faa6c074620 2024-09-10T23:38:37.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.716+0000 7faa72e00640 1 -- 172.21.15.94:0/1284089423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa6c0b4790 con 0x7faa6c074620 2024-09-10T23:38:37.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.716+0000 7faa69600640 1 -- 172.21.15.94:0/1284089423 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5241 (secure 0 0 0) 0x7faa5c0f8450 con 0x7faa6c074620 2024-09-10T23:38:37.718 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:37.718 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":79,"btime":"2024-09-10T23:38:28:841812+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7090,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3097381199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3097381199},{"type":"v1","addr":"172.21.15.181:6835","nonce":3097381199}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2654663117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2654663117},{"type":"v1","addr":"172.21.15.94:6833","nonce":2654663117}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7530,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1334394570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1334394570},{"type":"v1","addr":"172.21.15.94:6835","nonce":1334394570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:38:18.757602+0000","modified":"2024-09-10T23:38:21.765704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196283":{"gid":196283,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6834/1748173202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":1748173202},{"type":"v1","addr":"172.21.15.181:6834","nonce":1748173202}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196283,"qdb_cluster":[196283]},"id":9}]} 2024-09-10T23:38:37.718 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 79 2024-09-10T23:38:37.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.716+0000 7faa72e00640 1 -- 172.21.15.94:0/1284089423 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa50072c10 msgr2=0x7faa500750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:37.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.716+0000 7faa72e00640 1 --2- 172.21.15.94:0/1284089423 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa50072c10 0x7faa500750d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faa60007fd0 tx=0x7faa60009040 comp rx=0 tx=0).stop 2024-09-10T23:38:37.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.716+0000 7faa72e00640 1 -- 172.21.15.94:0/1284089423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c074620 msgr2=0x7faa6c1f74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:37.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.716+0000 7faa72e00640 1 --2- 172.21.15.94:0/1284089423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c074620 0x7faa6c1f74a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00a960 tx=0x7faa5c005e70 comp rx=0 tx=0).stop 2024-09-10T23:38:37.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.716+0000 7faa72e00640 1 -- 172.21.15.94:0/1284089423 shutdown_connections 2024-09-10T23:38:37.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.716+0000 7faa72e00640 1 --2- 172.21.15.94:0/1284089423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa6c15e1e0 0x7faa6c14f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.716+0000 7faa72e00640 1 --2- 172.21.15.94:0/1284089423 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faa50072c10 0x7faa500750d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.716+0000 7faa72e00640 1 --2- 172.21.15.94:0/1284089423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c074620 0x7faa6c1f74a0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.716+0000 7faa72e00640 1 --2- 172.21.15.94:0/1284089423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa6c073d00 0x7faa6c13d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.716+0000 7faa72e00640 1 -- 172.21.15.94:0/1284089423 >> 172.21.15.94:0/1284089423 conn(0x7faa6c0b1840 msgr2=0x7faa6c0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:37.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.716+0000 7faa72e00640 1 -- 172.21.15.94:0/1284089423 shutdown_connections 2024-09-10T23:38:37.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.716+0000 7faa72e00640 1 -- 172.21.15.94:0/1284089423 wait complete. 2024-09-10T23:38:37.739 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:38:37.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.880+0000 7f5fcc800640 1 -- 172.21.15.94:0/4176211237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fac003a60 msgr2=0x7f5fac003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:37.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.880+0000 7f5fcc800640 1 --2- 172.21.15.94:0/4176211237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fac003a60 0x7f5fac003e80 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009b30 tx=0x7f5fb801d210 comp rx=0 tx=0).stop 2024-09-10T23:38:37.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.880+0000 7f5fcc800640 1 -- 172.21.15.94:0/4176211237 shutdown_connections 2024-09-10T23:38:37.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.880+0000 7f5fcc800640 1 --2- 172.21.15.94:0/4176211237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fac003a60 0x7f5fac003e80 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.880+0000 7f5fcc800640 1 -- 172.21.15.94:0/4176211237 >> 172.21.15.94:0/4176211237 conn(0x7f5fc406cdf0 msgr2=0x7f5fc406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:37.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.880+0000 7f5fcc800640 1 -- 172.21.15.94:0/4176211237 shutdown_connections 2024-09-10T23:38:37.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.880+0000 7f5fcc800640 1 -- 172.21.15.94:0/4176211237 wait complete. 2024-09-10T23:38:37.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.880+0000 7f5fcc800640 1 Processor -- start 2024-09-10T23:38:37.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.880+0000 7f5fcc800640 1 -- start start 2024-09-10T23:38:37.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.880+0000 7f5fcc800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fac003a60 0x7f5fc41e6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:37.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.880+0000 7f5fcc800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fc4146620 0x7f5fc41e75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:37.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.880+0000 7f5fc9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fc4146620 0x7f5fc41e75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:37.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.880+0000 7f5fc9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fc4146620 0x7f5fc41e75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45432/0 (socket says 172.21.15.94:45432) 2024-09-10T23:38:37.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.880+0000 7f5fc9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fac003a60 0x7f5fc41e6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:37.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.880+0000 7f5fc9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fac003a60 0x7f5fc41e6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:52882/0 (socket says 172.21.15.94:52882) 2024-09-10T23:38:37.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fcc800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5fc40cb5d0 0x7f5fc415c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:37.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fcc800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc406f880 con 0x7f5fc4146620 2024-09-10T23:38:37.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fcc800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc406f700 con 0x7f5fac003a60 2024-09-10T23:38:37.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fcc800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5fc415aa50 con 0x7f5fc40cb5d0 2024-09-10T23:38:37.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fc9400640 1 -- 172.21.15.94:0/2096469588 learned_addr learned my addr 172.21.15.94:0/2096469588 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:37.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fca800640 1 --2- 172.21.15.94:0/2096469588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5fc40cb5d0 0x7f5fc415c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fc9400640 1 -- 172.21.15.94:0/2096469588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5fc40cb5d0 msgr2=0x7f5fc415c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fc9400640 1 --2- 172.21.15.94:0/2096469588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5fc40cb5d0 0x7f5fc415c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fc9400640 1 -- 172.21.15.94:0/2096469588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fac003a60 msgr2=0x7f5fc41e6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fc9400640 1 --2- 172.21.15.94:0/2096469588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fac003a60 0x7f5fc41e6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fc9400640 1 -- 172.21.15.94:0/2096469588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc4146aa0 con 0x7f5fc4146620 2024-09-10T23:38:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fca800640 1 --2- 172.21.15.94:0/2096469588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5fc40cb5d0 0x7f5fc415c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:38:37.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fc9400640 1 --2- 172.21.15.94:0/2096469588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fc4146620 0x7f5fc41e75d0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4004fd0 tx=0x7f5fb400e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:37.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fbe000640 1 -- 172.21.15.94:0/2096469588 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4015070 con 0x7f5fc4146620 2024-09-10T23:38:37.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fcc800640 1 -- 172.21.15.94:0/2096469588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc40ca810 con 0x7f5fc4146620 2024-09-10T23:38:37.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fbe000640 1 -- 172.21.15.94:0/2096469588 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fb4010980 con 0x7f5fc4146620 2024-09-10T23:38:37.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fbe000640 1 -- 172.21.15.94:0/2096469588 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4010dd0 con 0x7f5fc4146620 2024-09-10T23:38:37.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fcc800640 1 -- 172.21.15.94:0/2096469588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc40cadd0 con 0x7f5fc4146620 2024-09-10T23:38:37.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.884+0000 7f5fcc800640 1 -- 172.21.15.94:0/2096469588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc415afb0 con 0x7f5fc4146620 2024-09-10T23:38:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.888+0000 7f5fbe000640 1 -- 172.21.15.94:0/2096469588 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f5fb4011460 con 0x7f5fc4146620 2024-09-10T23:38:37.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.888+0000 7f5fbe000640 1 --2- 172.21.15.94:0/2096469588 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5fa8072c40 0x7f5fa8075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:37.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.888+0000 7f5fbe000640 1 -- 172.21.15.94:0/2096469588 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5793+0+0 (secure 0 0 0) 0x7f5fb40943f0 con 0x7f5fc4146620 2024-09-10T23:38:37.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.888+0000 7f5fc9e00640 1 --2- 172.21.15.94:0/2096469588 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5fa8072c40 0x7f5fa8075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:37.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.888+0000 7f5fc9e00640 1 --2- 172.21.15.94:0/2096469588 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5fa8072c40 0x7f5fa8075100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80ae000 tx=0x7f5fb8019fb0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:37.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:37.892+0000 7f5fbe000640 1 -- 172.21.15.94:0/2096469588 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5fb405e990 con 0x7f5fc4146620 2024-09-10T23:38:38.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.092+0000 7f5fcc800640 1 -- 172.21.15.94:0/2096469588 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fc406e7e0 con 0x7f5fc4146620 2024-09-10T23:38:38.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.096+0000 7f5fbe000640 1 -- 172.21.15.94:0/2096469588 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5241 (secure 0 0 0) 0x7f5fb40633c0 con 0x7f5fc4146620 2024-09-10T23:38:38.099 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:38.099 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":79,"btime":"2024-09-10T23:38:28:841812+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7090,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3097381199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3097381199},{"type":"v1","addr":"172.21.15.181:6835","nonce":3097381199}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2654663117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2654663117},{"type":"v1","addr":"172.21.15.94:6833","nonce":2654663117}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7530,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1334394570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1334394570},{"type":"v1","addr":"172.21.15.94:6835","nonce":1334394570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:38:18.757602+0000","modified":"2024-09-10T23:38:21.765704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196283":{"gid":196283,"name":"b","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6834/1748173202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":1748173202},{"type":"v1","addr":"172.21.15.181:6834","nonce":1748173202}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196283,"qdb_cluster":[196283]},"id":9}]} 2024-09-10T23:38:38.099 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 79 2024-09-10T23:38:38.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.096+0000 7f5fcc800640 1 -- 172.21.15.94:0/2096469588 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5fa8072c40 msgr2=0x7f5fa8075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:38.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.096+0000 7f5fcc800640 1 --2- 172.21.15.94:0/2096469588 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5fa8072c40 0x7f5fa8075100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80ae000 tx=0x7f5fb8019fb0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.096+0000 7f5fcc800640 1 -- 172.21.15.94:0/2096469588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fc4146620 msgr2=0x7f5fc41e75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:38.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.096+0000 7f5fcc800640 1 --2- 172.21.15.94:0/2096469588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fc4146620 0x7f5fc41e75d0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4004fd0 tx=0x7f5fb400e960 comp rx=0 tx=0).stop 2024-09-10T23:38:38.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.096+0000 7f5fcc800640 1 -- 172.21.15.94:0/2096469588 shutdown_connections 2024-09-10T23:38:38.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.096+0000 7f5fcc800640 1 --2- 172.21.15.94:0/2096469588 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5fa8072c40 0x7f5fa8075100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.096+0000 7f5fcc800640 1 --2- 172.21.15.94:0/2096469588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5fc40cb5d0 0x7f5fc415c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.096+0000 7f5fcc800640 1 --2- 172.21.15.94:0/2096469588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fc4146620 0x7f5fc41e75d0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.096+0000 7f5fcc800640 1 --2- 172.21.15.94:0/2096469588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fac003a60 0x7f5fc41e6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.096+0000 7f5fcc800640 1 -- 172.21.15.94:0/2096469588 >> 172.21.15.94:0/2096469588 conn(0x7f5fc406cdf0 msgr2=0x7f5fc41587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:38.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.096+0000 7f5fcc800640 1 -- 172.21.15.94:0/2096469588 shutdown_connections 2024-09-10T23:38:38.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.096+0000 7f5fcc800640 1 -- 172.21.15.94:0/2096469588 wait complete. 2024-09-10T23:38:38.119 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:38:38.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f6442400640 1 -- 172.21.15.94:0/3614051058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6420003a10 msgr2=0x7f6420003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:38.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f6442400640 1 --2- 172.21.15.94:0/3614051058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6420003a10 0x7f6420003e30 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f642c009eb0 tx=0x7f642c01d5d0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f643be00640 1 -- 172.21.15.94:0/3614051058 reap_dead start 2024-09-10T23:38:38.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f6442400640 1 -- 172.21.15.94:0/3614051058 shutdown_connections 2024-09-10T23:38:38.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f6442400640 1 -- 172.21.15.94:0/3614051058 >> 172.21.15.94:0/3614051058 conn(0x7f643c0d1c60 msgr2=0x7f643c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:38.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f6442400640 1 -- 172.21.15.94:0/3614051058 shutdown_connections 2024-09-10T23:38:38.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f6442400640 1 -- 172.21.15.94:0/3614051058 wait complete. 2024-09-10T23:38:38.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f6442400640 1 Processor -- start 2024-09-10T23:38:38.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f6442400640 1 -- start start 2024-09-10T23:38:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f6442400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6420003a10 0x7f643c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f6442400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f643c0cee60 0x7f643c1f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f6442400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f643c0cf2e0 0x7f643c1f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f6442400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f643c0b0480 con 0x7f643c0cee60 2024-09-10T23:38:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f6442400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f643c0b0300 con 0x7f6420003a10 2024-09-10T23:38:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f6442400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f643c166aa0 con 0x7f643c0cf2e0 2024-09-10T23:38:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f643aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f643c0cee60 0x7f643c1f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f643aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f643c0cee60 0x7f643c1f7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45446/0 (socket says 172.21.15.94:45446) 2024-09-10T23:38:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f643aa00640 1 -- 172.21.15.94:0/1236432579 learned_addr learned my addr 172.21.15.94:0/1236432579 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f643b400640 1 --2- 172.21.15.94:0/1236432579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6420003a10 0x7f643c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f643be00640 1 --2- 172.21.15.94:0/1236432579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f643c0cf2e0 0x7f643c1f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f643aa00640 1 -- 172.21.15.94:0/1236432579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f643c0cf2e0 msgr2=0x7f643c1f83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f643aa00640 1 --2- 172.21.15.94:0/1236432579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f643c0cf2e0 0x7f643c1f83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f643aa00640 1 -- 172.21.15.94:0/1236432579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6420003a10 msgr2=0x7f643c06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f643aa00640 1 --2- 172.21.15.94:0/1236432579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6420003a10 0x7f643c06dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f643aa00640 1 -- 172.21.15.94:0/1236432579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c0b3d30 con 0x7f643c0cee60 2024-09-10T23:38:38.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.260+0000 7f643aa00640 1 --2- 172.21.15.94:0/1236432579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f643c0cee60 0x7f643c1f7e70 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6428000fc0 tx=0x7f64280039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:38.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.264+0000 7f6433400640 1 -- 172.21.15.94:0/1236432579 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428015070 con 0x7f643c0cee60 2024-09-10T23:38:38.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.264+0000 7f6433400640 1 -- 172.21.15.94:0/1236432579 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6428011880 con 0x7f643c0cee60 2024-09-10T23:38:38.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.264+0000 7f6442400640 1 -- 172.21.15.94:0/1236432579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c0cd690 con 0x7f643c0cee60 2024-09-10T23:38:38.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.264+0000 7f6433400640 1 -- 172.21.15.94:0/1236432579 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428011cd0 con 0x7f643c0cee60 2024-09-10T23:38:38.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.264+0000 7f6442400640 1 -- 172.21.15.94:0/1236432579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c0cd950 con 0x7f643c0cee60 2024-09-10T23:38:38.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.264+0000 7f6442400640 1 -- 172.21.15.94:0/1236432579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643c0cfb60 con 0x7f643c0cee60 2024-09-10T23:38:38.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.264+0000 7f6433400640 1 -- 172.21.15.94:0/1236432579 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f64280103c0 con 0x7f643c0cee60 2024-09-10T23:38:38.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.264+0000 7f6433400640 1 --2- 172.21.15.94:0/1236432579 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f641c072c40 0x7f641c075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:38.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.264+0000 7f6433400640 1 -- 172.21.15.94:0/1236432579 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5793+0+0 (secure 0 0 0) 0x7f6428095000 con 0x7f643c0cee60 2024-09-10T23:38:38.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.264+0000 7f643b400640 1 --2- 172.21.15.94:0/1236432579 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f641c072c40 0x7f641c075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:38.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.264+0000 7f643b400640 1 --2- 172.21.15.94:0/1236432579 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f641c072c40 0x7f641c075100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f642c0af000 tx=0x7f642c01a370 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:38.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.268+0000 7f6433400640 1 -- 172.21.15.94:0/1236432579 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f642805fbf0 con 0x7f643c0cee60 2024-09-10T23:38:38.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.436+0000 7f6442400640 1 -- 172.21.15.94:0/1236432579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f643c0749c0 con 0x7f643c0cee60 2024-09-10T23:38:38.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.436+0000 7f6433400640 1 -- 172.21.15.94:0/1236432579 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16831 (secure 0 0 0) 0x7f642805f4f0 con 0x7f643c0cee60 2024-09-10T23:38:38.439 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:38.439 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":115,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:38:34.921450+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:38:16.443089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T23:38:17.716329+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]},{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:38:38.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.440+0000 7f6442400640 1 -- 172.21.15.94:0/1236432579 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f641c072c40 msgr2=0x7f641c075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:38.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.440+0000 7f6442400640 1 --2- 172.21.15.94:0/1236432579 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f641c072c40 0x7f641c075100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f642c0af000 tx=0x7f642c01a370 comp rx=0 tx=0).stop 2024-09-10T23:38:38.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.440+0000 7f6442400640 1 -- 172.21.15.94:0/1236432579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f643c0cee60 msgr2=0x7f643c1f7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:38.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.440+0000 7f6442400640 1 --2- 172.21.15.94:0/1236432579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f643c0cee60 0x7f643c1f7e70 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6428000fc0 tx=0x7f64280039d0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.440+0000 7f6442400640 1 -- 172.21.15.94:0/1236432579 shutdown_connections 2024-09-10T23:38:38.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.440+0000 7f6442400640 1 --2- 172.21.15.94:0/1236432579 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f641c072c40 0x7f641c075100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.440+0000 7f6442400640 1 --2- 172.21.15.94:0/1236432579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f643c0cf2e0 0x7f643c1f83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.440+0000 7f6442400640 1 --2- 172.21.15.94:0/1236432579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f643c0cee60 0x7f643c1f7e70 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.440+0000 7f6442400640 1 --2- 172.21.15.94:0/1236432579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6420003a10 0x7f643c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.440+0000 7f6442400640 1 -- 172.21.15.94:0/1236432579 >> 172.21.15.94:0/1236432579 conn(0x7f643c0d1c60 msgr2=0x7f643c150410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:38.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.440+0000 7f6442400640 1 -- 172.21.15.94:0/1236432579 shutdown_connections 2024-09-10T23:38:38.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.440+0000 7f6442400640 1 -- 172.21.15.94:0/1236432579 wait complete. 2024-09-10T23:38:38.461 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T23:38:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.600+0000 7f8c21600640 1 -- 172.21.15.94:0/3600003815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c1c0c49b0 msgr2=0x7f8c1c0cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.600+0000 7f8c21600640 1 --2- 172.21.15.94:0/3600003815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c1c0c49b0 0x7f8c1c0cadc0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0c6b50 tx=0x7f8c0401c8c0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.600+0000 7f8c21600640 1 -- 172.21.15.94:0/3600003815 shutdown_connections 2024-09-10T23:38:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.600+0000 7f8c21600640 1 --2- 172.21.15.94:0/3600003815 >> v2:172.21.15.181:3300/0 conn(0x7f8c1c0cb300 0x7f8c1c15e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.600+0000 7f8c21600640 1 --2- 172.21.15.94:0/3600003815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c1c0c49b0 0x7f8c1c0cadc0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.600+0000 7f8c21600640 1 --2- 172.21.15.94:0/3600003815 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c158790 0x7f8c1c06fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.600+0000 7f8c21600640 1 -- 172.21.15.94:0/3600003815 >> 172.21.15.94:0/3600003815 conn(0x7f8c1c0cfaf0 msgr2=0x7f8c1c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.600+0000 7f8c21600640 1 -- 172.21.15.94:0/3600003815 shutdown_connections 2024-09-10T23:38:38.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.600+0000 7f8c21600640 1 -- 172.21.15.94:0/3600003815 wait complete. 2024-09-10T23:38:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.600+0000 7f8c21600640 1 Processor -- start 2024-09-10T23:38:38.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.600+0000 7f8c21600640 1 -- start start 2024-09-10T23:38:38.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.600+0000 7f8c21600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c1c0c49b0 0x7f8c1c1fbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:38.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.600+0000 7f8c21600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c1c0cb300 0x7f8c1c14bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:38.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c21600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c158790 0x7f8c1c14c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:38.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c21600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c15b8d0 con 0x7f8c1c0cb300 2024-09-10T23:38:38.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c21600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c15b750 con 0x7f8c1c0c49b0 2024-09-10T23:38:38.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c21600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8c1c15ba50 con 0x7f8c1c158790 2024-09-10T23:38:38.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c13e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c1c0cb300 0x7f8c1c14bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:38.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c1aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c1c0c49b0 0x7f8c1c1fbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:38.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c13e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c1c0cb300 0x7f8c1c14bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45470/0 (socket says 172.21.15.94:45470) 2024-09-10T23:38:38.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c13e00640 1 -- 172.21.15.94:0/304280542 learned_addr learned my addr 172.21.15.94:0/304280542 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:38.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c1b400640 1 --2- 172.21.15.94:0/304280542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c158790 0x7f8c1c14c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:38.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c13e00640 1 -- 172.21.15.94:0/304280542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c158790 msgr2=0x7f8c1c14c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:38.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c13e00640 1 --2- 172.21.15.94:0/304280542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c158790 0x7f8c1c14c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c13e00640 1 -- 172.21.15.94:0/304280542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c1c0c49b0 msgr2=0x7f8c1c1fbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:38.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c13e00640 1 --2- 172.21.15.94:0/304280542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c1c0c49b0 0x7f8c1c1fbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c13e00640 1 -- 172.21.15.94:0/304280542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c159320 con 0x7f8c1c0cb300 2024-09-10T23:38:38.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c13e00640 1 --2- 172.21.15.94:0/304280542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c1c0cb300 0x7f8c1c14bcf0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8c040b0040 tx=0x7f8c040a55c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:38.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c12a00640 1 -- 172.21.15.94:0/304280542 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c040b2040 con 0x7f8c1c0cb300 2024-09-10T23:38:38.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c12a00640 1 -- 172.21.15.94:0/304280542 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c04004380 con 0x7f8c1c0cb300 2024-09-10T23:38:38.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c12a00640 1 -- 172.21.15.94:0/304280542 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c040052d0 con 0x7f8c1c0cb300 2024-09-10T23:38:38.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c21600640 1 -- 172.21.15.94:0/304280542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1c159600 con 0x7f8c1c0cb300 2024-09-10T23:38:38.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c21600640 1 -- 172.21.15.94:0/304280542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1c159bc0 con 0x7f8c1c0cb300 2024-09-10T23:38:38.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c21600640 1 -- 172.21.15.94:0/304280542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c1c15bfb0 con 0x7f8c1c0cb300 2024-09-10T23:38:38.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.604+0000 7f8c12a00640 1 -- 172.21.15.94:0/304280542 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f8c04005470 con 0x7f8c1c0cb300 2024-09-10T23:38:38.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.608+0000 7f8c12a00640 1 --2- 172.21.15.94:0/304280542 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8bf8072c10 0x7f8bf80750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:38.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.608+0000 7f8c12a00640 1 -- 172.21.15.94:0/304280542 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5793+0+0 (secure 0 0 0) 0x7f8c0412d8d0 con 0x7f8c1c0cb300 2024-09-10T23:38:38.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.608+0000 7f8c1aa00640 1 --2- 172.21.15.94:0/304280542 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8bf8072c10 0x7f8bf80750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:38.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.608+0000 7f8c1aa00640 1 --2- 172.21.15.94:0/304280542 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8bf8072c10 0x7f8bf80750d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00bb50 tx=0x7f8c0c00b2e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:38.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.612+0000 7f8c12a00640 1 -- 172.21.15.94:0/304280542 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8c040164b0 con 0x7f8c1c0cb300 2024-09-10T23:38:38.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.808+0000 7f8c21600640 1 -- 172.21.15.94:0/304280542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8c1c0d0850 con 0x7f8c1c0cb300 2024-09-10T23:38:38.811 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-10T23:38:38.808+0000 7f4045a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:38:38.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.808+0000 7f8c12a00640 1 -- 172.21.15.94:0/304280542 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v80) ==== 161+0+0 (secure 0 0 0) 0x7f8c040f7e80 con 0x7f8c1c0cb300 2024-09-10T23:38:38.813 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T23:38:38.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.812+0000 7f8c21600640 1 -- 172.21.15.94:0/304280542 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8bf8072c10 msgr2=0x7f8bf80750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:38.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.812+0000 7f8c21600640 1 --2- 172.21.15.94:0/304280542 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8bf8072c10 0x7f8bf80750d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00bb50 tx=0x7f8c0c00b2e0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.812+0000 7f8c21600640 1 -- 172.21.15.94:0/304280542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c1c0cb300 msgr2=0x7f8c1c14bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:38.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.812+0000 7f8c21600640 1 --2- 172.21.15.94:0/304280542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c1c0cb300 0x7f8c1c14bcf0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8c040b0040 tx=0x7f8c040a55c0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.812+0000 7f8c21600640 1 -- 172.21.15.94:0/304280542 shutdown_connections 2024-09-10T23:38:38.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.812+0000 7f8c21600640 1 --2- 172.21.15.94:0/304280542 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8bf8072c10 0x7f8bf80750d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.812+0000 7f8c21600640 1 --2- 172.21.15.94:0/304280542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c158790 0x7f8c1c14c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.812+0000 7f8c21600640 1 --2- 172.21.15.94:0/304280542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c1c0cb300 0x7f8c1c14bcf0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.812+0000 7f8c21600640 1 --2- 172.21.15.94:0/304280542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c1c0c49b0 0x7f8c1c1fbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.812+0000 7f8c21600640 1 -- 172.21.15.94:0/304280542 >> 172.21.15.94:0/304280542 conn(0x7f8c1c0cfaf0 msgr2=0x7f8c1c0ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:38.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.812+0000 7f8c21600640 1 -- 172.21.15.94:0/304280542 shutdown_connections 2024-09-10T23:38:38.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.812+0000 7f8c21600640 1 -- 172.21.15.94:0/304280542 wait complete. 2024-09-10T23:38:38.836 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T23:38:38.884 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:38:38.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.980+0000 7f6a00e00640 1 -- 172.21.15.94:0/2648970363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69fc06e7c0 msgr2=0x7f69fc06ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:38.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.980+0000 7f6a00e00640 1 --2- 172.21.15.94:0/2648970363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69fc06e7c0 0x7f69fc06ebf0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0cd580 tx=0x7f69f401cd80 comp rx=0 tx=0).stop 2024-09-10T23:38:38.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.980+0000 7f6a00e00640 1 -- 172.21.15.94:0/2648970363 shutdown_connections 2024-09-10T23:38:38.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.980+0000 7f6a00e00640 1 --2- 172.21.15.94:0/2648970363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69fc06e7c0 0x7f69fc06ebf0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.980+0000 7f6a00e00640 1 --2- 172.21.15.94:0/2648970363 >> v2:172.21.15.181:3300/0 conn(0x7f69fc0c64f0 0x7f69fc0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.980+0000 7f6a00e00640 1 -- 172.21.15.94:0/2648970363 >> 172.21.15.94:0/2648970363 conn(0x7f69fc0cfaf0 msgr2=0x7f69fc0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:38.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.980+0000 7f6a00e00640 1 -- 172.21.15.94:0/2648970363 shutdown_connections 2024-09-10T23:38:38.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.980+0000 7f6a00e00640 1 -- 172.21.15.94:0/2648970363 wait complete. 2024-09-10T23:38:38.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.980+0000 7f6a00e00640 1 Processor -- start 2024-09-10T23:38:38.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.980+0000 7f6a00e00640 1 -- start start 2024-09-10T23:38:38.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f6a00e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69fc06e7c0 0x7f69fc154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:38.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f6a00e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc0c64f0 0x7f69fc152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:38.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f6a00e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69fc0ce280 0x7f69fc1fb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:38.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f6a00e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc1678f0 con 0x7f69fc06e7c0 2024-09-10T23:38:38.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f6a00e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc167770 con 0x7f69fc0c64f0 2024-09-10T23:38:38.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f6a00e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f69fc167a70 con 0x7f69fc0ce280 2024-09-10T23:38:38.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f69f9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69fc06e7c0 0x7f69fc154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:38.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f69f9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69fc06e7c0 0x7f69fc154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45478/0 (socket says 172.21.15.94:45478) 2024-09-10T23:38:38.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f69f9e00640 1 -- 172.21.15.94:0/905768818 learned_addr learned my addr 172.21.15.94:0/905768818 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:38.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f69fa800640 1 --2- 172.21.15.94:0/905768818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69fc0ce280 0x7f69fc1fb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:38.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f69f9400640 1 --2- 172.21.15.94:0/905768818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc0c64f0 0x7f69fc152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:38.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f69f9e00640 1 -- 172.21.15.94:0/905768818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69fc0ce280 msgr2=0x7f69fc1fb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:38.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f69f9e00640 1 --2- 172.21.15.94:0/905768818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69fc0ce280 0x7f69fc1fb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f69f9e00640 1 -- 172.21.15.94:0/905768818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc0c64f0 msgr2=0x7f69fc152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:38.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f69f9e00640 1 --2- 172.21.15.94:0/905768818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc0c64f0 0x7f69fc152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:38.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f69f9e00640 1 -- 172.21.15.94:0/905768818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc0c5960 con 0x7f69fc06e7c0 2024-09-10T23:38:38.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f69f9400640 1 --2- 172.21.15.94:0/905768818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc0c64f0 0x7f69fc152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:38.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f69f9e00640 1 --2- 172.21.15.94:0/905768818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69fc06e7c0 0x7f69fc154c30 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00c3f0 tx=0x7f69ec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:38.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f69f2000640 1 -- 172.21.15.94:0/905768818 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec017070 con 0x7f69fc06e7c0 2024-09-10T23:38:38.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f69f2000640 1 -- 172.21.15.94:0/905768818 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69ec0053c0 con 0x7f69fc06e7c0 2024-09-10T23:38:38.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f6a00e00640 1 -- 172.21.15.94:0/905768818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69fc1fbd30 con 0x7f69fc06e7c0 2024-09-10T23:38:38.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f69f2000640 1 -- 172.21.15.94:0/905768818 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec003860 con 0x7f69fc06e7c0 2024-09-10T23:38:38.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f6a00e00640 1 -- 172.21.15.94:0/905768818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69fc154170 con 0x7f69fc06e7c0 2024-09-10T23:38:38.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.984+0000 7f6a00e00640 1 -- 172.21.15.94:0/905768818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69fc06ebf0 con 0x7f69fc06e7c0 2024-09-10T23:38:38.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.988+0000 7f69f2000640 1 -- 172.21.15.94:0/905768818 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f69ec003a00 con 0x7f69fc06e7c0 2024-09-10T23:38:38.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.988+0000 7f69f2000640 1 --2- 172.21.15.94:0/905768818 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f69dc072c40 0x7f69dc075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:38.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.988+0000 7f69f2000640 1 -- 172.21.15.94:0/905768818 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5851+0+0 (secure 0 0 0) 0x7f69ec093510 con 0x7f69fc06e7c0 2024-09-10T23:38:38.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.988+0000 7f69f9400640 1 --2- 172.21.15.94:0/905768818 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f69dc072c40 0x7f69dc075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:38.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.988+0000 7f69f9400640 1 --2- 172.21.15.94:0/905768818 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f69dc072c40 0x7f69dc075100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f69e8006fd0 tx=0x7f69e8008040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:38.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:38.992+0000 7f69f2000640 1 -- 172.21.15.94:0/905768818 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f69ec05da80 con 0x7f69fc06e7c0 2024-09-10T23:38:39.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:39.188+0000 7f6a00e00640 1 -- 172.21.15.94:0/905768818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f69fc06f090 con 0x7f69fc06e7c0 2024-09-10T23:38:39.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:39.892+0000 7f69f2000640 1 -- 172.21.15.94:0/905768818 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v81) ==== 104+0+0 (secure 0 0 0) 0x7f69ec0624b0 con 0x7f69fc06e7c0 2024-09-10T23:38:39.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:39.892+0000 7f6a00e00640 1 -- 172.21.15.94:0/905768818 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f69dc072c40 msgr2=0x7f69dc075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:39.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:39.892+0000 7f6a00e00640 1 --2- 172.21.15.94:0/905768818 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f69dc072c40 0x7f69dc075100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f69e8006fd0 tx=0x7f69e8008040 comp rx=0 tx=0).stop 2024-09-10T23:38:39.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:39.892+0000 7f6a00e00640 1 -- 172.21.15.94:0/905768818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69fc06e7c0 msgr2=0x7f69fc154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:39.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:39.892+0000 7f6a00e00640 1 --2- 172.21.15.94:0/905768818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69fc06e7c0 0x7f69fc154c30 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00c3f0 tx=0x7f69ec00c8b0 comp rx=0 tx=0).stop 2024-09-10T23:38:39.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:39.892+0000 7f6a00e00640 1 -- 172.21.15.94:0/905768818 shutdown_connections 2024-09-10T23:38:39.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:39.892+0000 7f6a00e00640 1 --2- 172.21.15.94:0/905768818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69fc0c64f0 0x7f69fc152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:39.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:39.892+0000 7f6a00e00640 1 --2- 172.21.15.94:0/905768818 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f69dc072c40 0x7f69dc075100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:39.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:39.892+0000 7f6a00e00640 1 --2- 172.21.15.94:0/905768818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69fc0ce280 0x7f69fc1fb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:39.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:39.892+0000 7f6a00e00640 1 --2- 172.21.15.94:0/905768818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69fc06e7c0 0x7f69fc154c30 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:39.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:39.892+0000 7f6a00e00640 1 -- 172.21.15.94:0/905768818 >> 172.21.15.94:0/905768818 conn(0x7f69fc0cfaf0 msgr2=0x7f69fc0ca530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:39.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:39.892+0000 7f6a00e00640 1 -- 172.21.15.94:0/905768818 shutdown_connections 2024-09-10T23:38:39.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:39.892+0000 7f6a00e00640 1 -- 172.21.15.94:0/905768818 wait complete. 2024-09-10T23:38:39.922 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T23:38:40.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.068+0000 7f87d7000640 1 -- 172.21.15.94:0/2015353310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d015ba30 msgr2=0x7f87d00b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:40.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.068+0000 7f87d7000640 1 --2- 172.21.15.94:0/2015353310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d015ba30 0x7f87d00b1bd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009840 tx=0x7f87c401c820 comp rx=0 tx=0).stop 2024-09-10T23:38:40.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.068+0000 7f87d7000640 1 -- 172.21.15.94:0/2015353310 shutdown_connections 2024-09-10T23:38:40.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.068+0000 7f87d7000640 1 --2- 172.21.15.94:0/2015353310 >> v2:172.21.15.181:3300/0 conn(0x7f87d00b2110 0x7f87d00cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:40.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.068+0000 7f87d7000640 1 --2- 172.21.15.94:0/2015353310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d015ba30 0x7f87d00b1bd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:40.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.068+0000 7f87d7000640 1 -- 172.21.15.94:0/2015353310 >> 172.21.15.94:0/2015353310 conn(0x7f87d009aa70 msgr2=0x7f87d009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:40.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.068+0000 7f87d7000640 1 -- 172.21.15.94:0/2015353310 shutdown_connections 2024-09-10T23:38:40.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.068+0000 7f87d7000640 1 -- 172.21.15.94:0/2015353310 wait complete. 2024-09-10T23:38:40.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.068+0000 7f87d7000640 1 Processor -- start 2024-09-10T23:38:40.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d7000640 1 -- start start 2024-09-10T23:38:40.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d7000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d00b2110 0x7f87d0071c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:40.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d7000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d00d02e0 0x7f87d01fac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:40.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d7000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87d015ba30 0x7f87d01fb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:40.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d7000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87d0168c60 con 0x7f87d015ba30 2024-09-10T23:38:40.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d7000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f87d0168ae0 con 0x7f87d00b2110 2024-09-10T23:38:40.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d7000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f87d0168de0 con 0x7f87d00d02e0 2024-09-10T23:38:40.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d6600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87d015ba30 0x7f87d01fb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:40.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d6600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87d015ba30 0x7f87d01fb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45488/0 (socket says 172.21.15.94:45488) 2024-09-10T23:38:40.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d6600640 1 -- 172.21.15.94:0/3924994816 learned_addr learned my addr 172.21.15.94:0/3924994816 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:40.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d5c00640 1 --2- 172.21.15.94:0/3924994816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d00b2110 0x7f87d0071c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:40.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d5200640 1 --2- 172.21.15.94:0/3924994816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d00d02e0 0x7f87d01fac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:40.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d6600640 1 -- 172.21.15.94:0/3924994816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d00d02e0 msgr2=0x7f87d01fac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:40.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d6600640 1 --2- 172.21.15.94:0/3924994816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d00d02e0 0x7f87d01fac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:40.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d6600640 1 -- 172.21.15.94:0/3924994816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d00b2110 msgr2=0x7f87d0071c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:40.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d6600640 1 --2- 172.21.15.94:0/3924994816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d00b2110 0x7f87d0071c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:40.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d6600640 1 -- 172.21.15.94:0/3924994816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d01652f0 con 0x7f87d015ba30 2024-09-10T23:38:40.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d6600640 1 --2- 172.21.15.94:0/3924994816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87d015ba30 0x7f87d01fb1d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f87c800cbc0 tx=0x7f87c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:40.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87cd600640 1 -- 172.21.15.94:0/3924994816 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8016020 con 0x7f87d015ba30 2024-09-10T23:38:40.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87cd600640 1 -- 172.21.15.94:0/3924994816 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87c800ab00 con 0x7f87d015ba30 2024-09-10T23:38:40.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87cd600640 1 -- 172.21.15.94:0/3924994816 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8012420 con 0x7f87d015ba30 2024-09-10T23:38:40.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d7000640 1 -- 172.21.15.94:0/3924994816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d006ef70 con 0x7f87d015ba30 2024-09-10T23:38:40.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d7000640 1 -- 172.21.15.94:0/3924994816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d006e1b0 con 0x7f87d015ba30 2024-09-10T23:38:40.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.072+0000 7f87d7000640 1 -- 172.21.15.94:0/3924994816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d00cfda0 con 0x7f87d015ba30 2024-09-10T23:38:40.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.076+0000 7f87cd600640 1 -- 172.21.15.94:0/3924994816 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f87c80125c0 con 0x7f87d015ba30 2024-09-10T23:38:40.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.076+0000 7f87cd600640 1 --2- 172.21.15.94:0/3924994816 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87b0072c10 0x7f87b00750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:40.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.076+0000 7f87cd600640 1 -- 172.21.15.94:0/3924994816 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5851+0+0 (secure 0 0 0) 0x7f87c80945c0 con 0x7f87d015ba30 2024-09-10T23:38:40.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.076+0000 7f87d5c00640 1 --2- 172.21.15.94:0/3924994816 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87b0072c10 0x7f87b00750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:40.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.076+0000 7f87d5c00640 1 --2- 172.21.15.94:0/3924994816 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87b0072c10 0x7f87b00750d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f87c40af000 tx=0x7f87c401b060 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:40.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.080+0000 7f87cd600640 1 -- 172.21.15.94:0/3924994816 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f87c8020020 con 0x7f87d015ba30 2024-09-10T23:38:40.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.248+0000 7f87d7000640 1 -- 172.21.15.94:0/3924994816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f87d009aff0 con 0x7f87d015ba30 2024-09-10T23:38:40.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.900+0000 7f87cd600640 1 -- 172.21.15.94:0/3924994816 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v117) ==== 181+0+0 (secure 0 0 0) 0x7f87c805f180 con 0x7f87d015ba30 2024-09-10T23:38:40.905 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-09-10T23:38:40.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.904+0000 7f87d7000640 1 -- 172.21.15.94:0/3924994816 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87b0072c10 msgr2=0x7f87b00750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:40.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.904+0000 7f87d7000640 1 --2- 172.21.15.94:0/3924994816 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87b0072c10 0x7f87b00750d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f87c40af000 tx=0x7f87c401b060 comp rx=0 tx=0).stop 2024-09-10T23:38:40.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.904+0000 7f87d7000640 1 -- 172.21.15.94:0/3924994816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87d015ba30 msgr2=0x7f87d01fb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:40.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.904+0000 7f87d7000640 1 --2- 172.21.15.94:0/3924994816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87d015ba30 0x7f87d01fb1d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f87c800cbc0 tx=0x7f87c800ccb0 comp rx=0 tx=0).stop 2024-09-10T23:38:40.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.904+0000 7f87d7000640 1 -- 172.21.15.94:0/3924994816 shutdown_connections 2024-09-10T23:38:40.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.904+0000 7f87d7000640 1 --2- 172.21.15.94:0/3924994816 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f87b0072c10 0x7f87b00750d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:40.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.904+0000 7f87d7000640 1 --2- 172.21.15.94:0/3924994816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87d015ba30 0x7f87d01fb1d0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:40.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.904+0000 7f87d7000640 1 --2- 172.21.15.94:0/3924994816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d00d02e0 0x7f87d01fac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:40.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.904+0000 7f87d7000640 1 --2- 172.21.15.94:0/3924994816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d00b2110 0x7f87d0071c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:40.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.904+0000 7f87d7000640 1 -- 172.21.15.94:0/3924994816 >> 172.21.15.94:0/3924994816 conn(0x7f87d009aa70 msgr2=0x7f87d00ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:40.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.904+0000 7f87d7000640 1 -- 172.21.15.94:0/3924994816 shutdown_connections 2024-09-10T23:38:40.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:40.904+0000 7f87d7000640 1 -- 172.21.15.94:0/3924994816 wait complete. 2024-09-10T23:38:40.933 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T23:38:41.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.080+0000 7f0450a00640 1 -- 172.21.15.94:0/2713423691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0430003a10 msgr2=0x7f0430003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:41.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.080+0000 7f0450a00640 1 --2- 172.21.15.94:0/2713423691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0430003a10 0x7f0430003e30 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f043c00c310 tx=0x7f043c01cf90 comp rx=0 tx=0).stop 2024-09-10T23:38:41.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.080+0000 7f044a800640 1 -- 172.21.15.94:0/2713423691 reap_dead start 2024-09-10T23:38:41.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.080+0000 7f0450a00640 1 -- 172.21.15.94:0/2713423691 shutdown_connections 2024-09-10T23:38:41.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.080+0000 7f0450a00640 1 -- 172.21.15.94:0/2713423691 >> 172.21.15.94:0/2713423691 conn(0x7f044c06cdf0 msgr2=0x7f044c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:41.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.080+0000 7f0450a00640 1 -- 172.21.15.94:0/2713423691 shutdown_connections 2024-09-10T23:38:41.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.080+0000 7f0450a00640 1 -- 172.21.15.94:0/2713423691 wait complete. 2024-09-10T23:38:41.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0450a00640 1 Processor -- start 2024-09-10T23:38:41.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0450a00640 1 -- start start 2024-09-10T23:38:41.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0450a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0430003a10 0x7f044c141520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:41.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0450a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f044c15ce00 0x7f044c15d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:41.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0450a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f044c0caa30 0x7f044c0caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:41.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0450a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f044c072fb0 con 0x7f0430003a10 2024-09-10T23:38:41.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0450a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f044c072e30 con 0x7f044c15ce00 2024-09-10T23:38:41.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0450a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f044c073130 con 0x7f044c0caa30 2024-09-10T23:38:41.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0449e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0430003a10 0x7f044c141520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:41.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0449e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0430003a10 0x7f044c141520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45512/0 (socket says 172.21.15.94:45512) 2024-09-10T23:38:41.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0449e00640 1 -- 172.21.15.94:0/1527326387 learned_addr learned my addr 172.21.15.94:0/1527326387 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:41.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f044a800640 1 --2- 172.21.15.94:0/1527326387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f044c0caa30 0x7f044c0caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:41.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0449400640 1 --2- 172.21.15.94:0/1527326387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f044c15ce00 0x7f044c15d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:41.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0449e00640 1 -- 172.21.15.94:0/1527326387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f044c0caa30 msgr2=0x7f044c0caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:41.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0449e00640 1 --2- 172.21.15.94:0/1527326387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f044c0caa30 0x7f044c0caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:41.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0449e00640 1 -- 172.21.15.94:0/1527326387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f044c15ce00 msgr2=0x7f044c15d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:41.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0449e00640 1 --2- 172.21.15.94:0/1527326387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f044c15ce00 0x7f044c15d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:41.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0449e00640 1 -- 172.21.15.94:0/1527326387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044c1fa940 con 0x7f0430003a10 2024-09-10T23:38:41.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0449e00640 1 --2- 172.21.15.94:0/1527326387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0430003a10 0x7f044c141520 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f043c01b690 tx=0x7f043c006cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:41.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0442000640 1 -- 172.21.15.94:0/1527326387 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c0060c0 con 0x7f0430003a10 2024-09-10T23:38:41.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0442000640 1 -- 172.21.15.94:0/1527326387 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f043c003600 con 0x7f0430003a10 2024-09-10T23:38:41.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0450a00640 1 -- 172.21.15.94:0/1527326387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044c1fac20 con 0x7f0430003a10 2024-09-10T23:38:41.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0442000640 1 -- 172.21.15.94:0/1527326387 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c003a10 con 0x7f0430003a10 2024-09-10T23:38:41.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0450a00640 1 -- 172.21.15.94:0/1527326387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f044c1fb1e0 con 0x7f0430003a10 2024-09-10T23:38:41.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.084+0000 7f0450a00640 1 -- 172.21.15.94:0/1527326387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f044c0736c0 con 0x7f0430003a10 2024-09-10T23:38:41.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.088+0000 7f0442000640 1 -- 172.21.15.94:0/1527326387 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f043c003bb0 con 0x7f0430003a10 2024-09-10T23:38:41.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.088+0000 7f0442000640 1 --2- 172.21.15.94:0/1527326387 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f042c072c40 0x7f042c075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:41.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.088+0000 7f0442000640 1 -- 172.21.15.94:0/1527326387 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5413+0+0 (secure 0 0 0) 0x7f043c12e060 con 0x7f0430003a10 2024-09-10T23:38:41.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.088+0000 7f0449400640 1 --2- 172.21.15.94:0/1527326387 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f042c072c40 0x7f042c075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:41.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.088+0000 7f0449400640 1 --2- 172.21.15.94:0/1527326387 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f042c072c40 0x7f042c075100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f044c14a5c0 tx=0x7f043800a2f0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:41.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.092+0000 7f0442000640 1 -- 172.21.15.94:0/1527326387 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f043c016d70 con 0x7f0430003a10 2024-09-10T23:38:41.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.260+0000 7f0450a00640 1 -- 172.21.15.94:0/1527326387 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f044c06f040 con 0x7f0430003a10 2024-09-10T23:38:41.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.936+0000 7f0442000640 1 -- 172.21.15.94:0/1527326387 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v119) ==== 169+0+0 (secure 0 0 0) 0x7f043c0f8ec0 con 0x7f0430003a10 2024-09-10T23:38:41.938 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-09-10T23:38:41.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.940+0000 7f0450a00640 1 -- 172.21.15.94:0/1527326387 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f042c072c40 msgr2=0x7f042c075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:41.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.940+0000 7f0450a00640 1 --2- 172.21.15.94:0/1527326387 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f042c072c40 0x7f042c075100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f044c14a5c0 tx=0x7f043800a2f0 comp rx=0 tx=0).stop 2024-09-10T23:38:41.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.940+0000 7f0450a00640 1 -- 172.21.15.94:0/1527326387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0430003a10 msgr2=0x7f044c141520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:41.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.940+0000 7f0450a00640 1 --2- 172.21.15.94:0/1527326387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0430003a10 0x7f044c141520 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f043c01b690 tx=0x7f043c006cf0 comp rx=0 tx=0).stop 2024-09-10T23:38:41.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.940+0000 7f0450a00640 1 -- 172.21.15.94:0/1527326387 shutdown_connections 2024-09-10T23:38:41.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.940+0000 7f0450a00640 1 --2- 172.21.15.94:0/1527326387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f044c15ce00 0x7f044c15d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:41.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.940+0000 7f0450a00640 1 --2- 172.21.15.94:0/1527326387 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f042c072c40 0x7f042c075100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:41.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.940+0000 7f0450a00640 1 --2- 172.21.15.94:0/1527326387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f044c0caa30 0x7f044c0caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:41.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.940+0000 7f0450a00640 1 --2- 172.21.15.94:0/1527326387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0430003a10 0x7f044c141520 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:41.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.940+0000 7f0450a00640 1 -- 172.21.15.94:0/1527326387 >> 172.21.15.94:0/1527326387 conn(0x7f044c06cdf0 msgr2=0x7f044c15b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:41.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.940+0000 7f0450a00640 1 -- 172.21.15.94:0/1527326387 shutdown_connections 2024-09-10T23:38:41.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:41.940+0000 7f0450a00640 1 -- 172.21.15.94:0/1527326387 wait complete. 2024-09-10T23:38:41.964 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T23:38:42.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.104+0000 7f065d800640 1 -- 172.21.15.94:0/1752021569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06580c49b0 msgr2=0x7f0658165b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:42.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.104+0000 7f065d800640 1 --2- 172.21.15.94:0/1752021569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06580c49b0 0x7f0658165b10 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0650009c80 tx=0x7f065001cda0 comp rx=0 tx=0).stop 2024-09-10T23:38:42.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.104+0000 7f065d800640 1 -- 172.21.15.94:0/1752021569 shutdown_connections 2024-09-10T23:38:42.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.104+0000 7f065d800640 1 --2- 172.21.15.94:0/1752021569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06580c49b0 0x7f0658165b10 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:42.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.104+0000 7f065d800640 1 -- 172.21.15.94:0/1752021569 >> 172.21.15.94:0/1752021569 conn(0x7f06580cfaf0 msgr2=0x7f06580cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:42.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.104+0000 7f065d800640 1 -- 172.21.15.94:0/1752021569 shutdown_connections 2024-09-10T23:38:42.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.104+0000 7f065d800640 1 -- 172.21.15.94:0/1752021569 wait complete. 2024-09-10T23:38:42.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.104+0000 7f065d800640 1 Processor -- start 2024-09-10T23:38:42.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.104+0000 7f065d800640 1 -- start start 2024-09-10T23:38:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f065d800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065806fc00 0x7f0658150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f065d800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06580c49b0 0x7f06580ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f065d800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0658158790 0x7f06580ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f065d800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0658168fe0 con 0x7f0658158790 2024-09-10T23:38:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f065d800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0658168e60 con 0x7f065806fc00 2024-09-10T23:38:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f065d800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0658169160 con 0x7f06580c49b0 2024-09-10T23:38:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f0657400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0658158790 0x7f06580ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f0657400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0658158790 0x7f06580ae3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45528/0 (socket says 172.21.15.94:45528) 2024-09-10T23:38:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f0657400640 1 -- 172.21.15.94:0/1868475431 learned_addr learned my addr 172.21.15.94:0/1868475431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f0656a00640 1 --2- 172.21.15.94:0/1868475431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065806fc00 0x7f0658150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f0656000640 1 --2- 172.21.15.94:0/1868475431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06580c49b0 0x7f06580ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f0657400640 1 -- 172.21.15.94:0/1868475431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06580c49b0 msgr2=0x7f06580ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f0657400640 1 --2- 172.21.15.94:0/1868475431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06580c49b0 0x7f06580ade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f0657400640 1 -- 172.21.15.94:0/1868475431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065806fc00 msgr2=0x7f0658150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f0657400640 1 --2- 172.21.15.94:0/1868475431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065806fc00 0x7f0658150fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f0657400640 1 -- 172.21.15.94:0/1868475431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06580c5d00 con 0x7f0658158790 2024-09-10T23:38:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f0656a00640 1 --2- 172.21.15.94:0/1868475431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065806fc00 0x7f0658150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f0657400640 1 --2- 172.21.15.94:0/1868475431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0658158790 0x7f06580ae3a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f0650009bd0 tx=0x7f06500a5a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f064ea00640 1 -- 172.21.15.94:0/1868475431 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06500bb070 con 0x7f0658158790 2024-09-10T23:38:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f064ea00640 1 -- 172.21.15.94:0/1868475431 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0650004c50 con 0x7f0658158790 2024-09-10T23:38:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f065d800640 1 -- 172.21.15.94:0/1868475431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06580c5fe0 con 0x7f0658158790 2024-09-10T23:38:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f064ea00640 1 -- 172.21.15.94:0/1868475431 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06500b2040 con 0x7f0658158790 2024-09-10T23:38:42.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f065d800640 1 -- 172.21.15.94:0/1868475431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06581595b0 con 0x7f0658158790 2024-09-10T23:38:42.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f065d800640 1 -- 172.21.15.94:0/1868475431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06580c6220 con 0x7f0658158790 2024-09-10T23:38:42.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.108+0000 7f064ea00640 1 -- 172.21.15.94:0/1868475431 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f06500b5030 con 0x7f0658158790 2024-09-10T23:38:42.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.112+0000 7f064ea00640 1 --2- 172.21.15.94:0/1868475431 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0638072c10 0x7f06380750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:42.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.112+0000 7f064ea00640 1 -- 172.21.15.94:0/1868475431 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4971+0+0 (secure 0 0 0) 0x7f065012d770 con 0x7f0658158790 2024-09-10T23:38:42.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.112+0000 7f0656a00640 1 --2- 172.21.15.94:0/1868475431 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0638072c10 0x7f06380750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:42.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.112+0000 7f0656a00640 1 --2- 172.21.15.94:0/1868475431 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0638072c10 0x7f06380750d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f06580cf6b0 tx=0x7f0648009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:42.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.116+0000 7f064ea00640 1 -- 172.21.15.94:0/1868475431 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0650016990 con 0x7f0658158790 2024-09-10T23:38:42.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.284+0000 7f065d800640 1 -- 172.21.15.94:0/1868475431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f06580cadc0 con 0x7f0658158790 2024-09-10T23:38:42.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.288+0000 7f064ea00640 1 -- 172.21.15.94:0/1868475431 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v41) ==== 71+0+0 (secure 0 0 0) 0x7f06500f86e0 con 0x7f0658158790 2024-09-10T23:38:42.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.288+0000 7f065d800640 1 -- 172.21.15.94:0/1868475431 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0638072c10 msgr2=0x7f06380750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:42.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.288+0000 7f065d800640 1 --2- 172.21.15.94:0/1868475431 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0638072c10 0x7f06380750d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f06580cf6b0 tx=0x7f0648009040 comp rx=0 tx=0).stop 2024-09-10T23:38:42.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.288+0000 7f065d800640 1 -- 172.21.15.94:0/1868475431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0658158790 msgr2=0x7f06580ae3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:42.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.288+0000 7f065d800640 1 --2- 172.21.15.94:0/1868475431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0658158790 0x7f06580ae3a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f0650009bd0 tx=0x7f06500a5a90 comp rx=0 tx=0).stop 2024-09-10T23:38:42.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.288+0000 7f065d800640 1 -- 172.21.15.94:0/1868475431 shutdown_connections 2024-09-10T23:38:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.288+0000 7f065d800640 1 --2- 172.21.15.94:0/1868475431 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0638072c10 0x7f06380750d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.288+0000 7f065d800640 1 --2- 172.21.15.94:0/1868475431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0658158790 0x7f06580ae3a0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.288+0000 7f065d800640 1 --2- 172.21.15.94:0/1868475431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06580c49b0 0x7f06580ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.288+0000 7f065d800640 1 --2- 172.21.15.94:0/1868475431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f065806fc00 0x7f0658150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.288+0000 7f065d800640 1 -- 172.21.15.94:0/1868475431 >> 172.21.15.94:0/1868475431 conn(0x7f06580cfaf0 msgr2=0x7f06581579f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.288+0000 7f065d800640 1 -- 172.21.15.94:0/1868475431 shutdown_connections 2024-09-10T23:38:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.288+0000 7f065d800640 1 -- 172.21.15.94:0/1868475431 wait complete. 2024-09-10T23:38:42.311 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-10T23:38:42.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.448+0000 7f1933200640 1 -- 172.21.15.94:0/3481584746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19340d01e0 msgr2=0x7f19340af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.448+0000 7f1933200640 1 --2- 172.21.15.94:0/3481584746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19340d01e0 0x7f19340af540 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f192c009c80 tx=0x7f192c01cd70 comp rx=0 tx=0).stop 2024-09-10T23:38:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.448+0000 7f1933200640 1 -- 172.21.15.94:0/3481584746 shutdown_connections 2024-09-10T23:38:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.448+0000 7f1933200640 1 --2- 172.21.15.94:0/3481584746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19340d01e0 0x7f19340af540 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.448+0000 7f1933200640 1 -- 172.21.15.94:0/3481584746 >> 172.21.15.94:0/3481584746 conn(0x7f193409aa70 msgr2=0x7f193409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1933200640 1 -- 172.21.15.94:0/3481584746 shutdown_connections 2024-09-10T23:38:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1933200640 1 -- 172.21.15.94:0/3481584746 wait complete. 2024-09-10T23:38:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1933200640 1 Processor -- start 2024-09-10T23:38:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1933200640 1 -- start start 2024-09-10T23:38:42.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1933200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19340cfda0 0x7f19341531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:42.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1933200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19340d01e0 0x7f19341649e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:42.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1933200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f193415ba30 0x7f1934164f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:42.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1933200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1934169a50 con 0x7f19340cfda0 2024-09-10T23:38:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1933200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f19341698d0 con 0x7f19340d01e0 2024-09-10T23:38:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1933200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1934169bd0 con 0x7f193415ba30 2024-09-10T23:38:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1931e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19340cfda0 0x7f19341531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1931e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19340cfda0 0x7f19341531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45542/0 (socket says 172.21.15.94:45542) 2024-09-10T23:38:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1931e00640 1 -- 172.21.15.94:0/1856428423 learned_addr learned my addr 172.21.15.94:0/1856428423 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1932800640 1 --2- 172.21.15.94:0/1856428423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f193415ba30 0x7f1934164f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f192be00640 1 --2- 172.21.15.94:0/1856428423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19340d01e0 0x7f19341649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1931e00640 1 -- 172.21.15.94:0/1856428423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f193415ba30 msgr2=0x7f1934164f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1931e00640 1 --2- 172.21.15.94:0/1856428423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f193415ba30 0x7f1934164f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1931e00640 1 -- 172.21.15.94:0/1856428423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19340d01e0 msgr2=0x7f19341649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1931e00640 1 --2- 172.21.15.94:0/1856428423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19340d01e0 0x7f19341649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1931e00640 1 -- 172.21.15.94:0/1856428423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19341536f0 con 0x7f19340cfda0 2024-09-10T23:38:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1932800640 1 --2- 172.21.15.94:0/1856428423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f193415ba30 0x7f1934164f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f192be00640 1 --2- 172.21.15.94:0/1856428423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19340d01e0 0x7f19341649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1931e00640 1 --2- 172.21.15.94:0/1856428423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19340cfda0 0x7f19341531b0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f192400c3c0 tx=0x7f192400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f192a000640 1 -- 172.21.15.94:0/1856428423 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924017070 con 0x7f19340cfda0 2024-09-10T23:38:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f192a000640 1 -- 172.21.15.94:0/1856428423 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19240053c0 con 0x7f19340cfda0 2024-09-10T23:38:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f1933200640 1 -- 172.21.15.94:0/1856428423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193415b080 con 0x7f19340cfda0 2024-09-10T23:38:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.452+0000 7f192a000640 1 -- 172.21.15.94:0/1856428423 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924003860 con 0x7f19340cfda0 2024-09-10T23:38:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.456+0000 7f1933200640 1 -- 172.21.15.94:0/1856428423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193415c8c0 con 0x7f19340cfda0 2024-09-10T23:38:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.456+0000 7f1933200640 1 -- 172.21.15.94:0/1856428423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1934153cb0 con 0x7f19340cfda0 2024-09-10T23:38:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.456+0000 7f192a000640 1 -- 172.21.15.94:0/1856428423 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f1924003a00 con 0x7f19340cfda0 2024-09-10T23:38:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.456+0000 7f192a000640 1 --2- 172.21.15.94:0/1856428423 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1910072c10 0x7f19100750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.456+0000 7f192a000640 1 -- 172.21.15.94:0/1856428423 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4971+0+0 (secure 0 0 0) 0x7f19240932f0 con 0x7f19340cfda0 2024-09-10T23:38:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.456+0000 7f192be00640 1 --2- 172.21.15.94:0/1856428423 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1910072c10 0x7f19100750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.456+0000 7f192be00640 1 --2- 172.21.15.94:0/1856428423 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1910072c10 0x7f19100750d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19341faa30 tx=0x7f191c009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.460+0000 7f192a000640 1 -- 172.21.15.94:0/1856428423 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f192405e230 con 0x7f19340cfda0 2024-09-10T23:38:42.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.688+0000 7f1933200640 1 -- 172.21.15.94:0/1856428423 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7f1934070900 con 0x7f19340cfda0 2024-09-10T23:38:42.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.936+0000 7f192a000640 1 -- 172.21.15.94:0/1856428423 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v338) ==== 148+0+0 (secure 0 0 0) 0x7f192405db30 con 0x7f19340cfda0 2024-09-10T23:38:42.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.940+0000 7f1933200640 1 -- 172.21.15.94:0/1856428423 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1910072c10 msgr2=0x7f19100750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:42.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.940+0000 7f1933200640 1 --2- 172.21.15.94:0/1856428423 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1910072c10 0x7f19100750d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19341faa30 tx=0x7f191c009040 comp rx=0 tx=0).stop 2024-09-10T23:38:42.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.940+0000 7f1933200640 1 -- 172.21.15.94:0/1856428423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19340cfda0 msgr2=0x7f19341531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.940+0000 7f1933200640 1 --2- 172.21.15.94:0/1856428423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19340cfda0 0x7f19341531b0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f192400c3c0 tx=0x7f192400c880 comp rx=0 tx=0).stop 2024-09-10T23:38:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.940+0000 7f1933200640 1 -- 172.21.15.94:0/1856428423 shutdown_connections 2024-09-10T23:38:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.940+0000 7f1933200640 1 --2- 172.21.15.94:0/1856428423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19340d01e0 0x7f19341649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.940+0000 7f1933200640 1 --2- 172.21.15.94:0/1856428423 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1910072c10 0x7f19100750d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.940+0000 7f1933200640 1 --2- 172.21.15.94:0/1856428423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f193415ba30 0x7f1934164f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.940+0000 7f1933200640 1 --2- 172.21.15.94:0/1856428423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19340cfda0 0x7f19341531b0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.940+0000 7f1933200640 1 -- 172.21.15.94:0/1856428423 >> 172.21.15.94:0/1856428423 conn(0x7f193409aa70 msgr2=0x7f19341541d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.940+0000 7f1933200640 1 -- 172.21.15.94:0/1856428423 shutdown_connections 2024-09-10T23:38:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:42.940+0000 7f1933200640 1 -- 172.21.15.94:0/1856428423 wait complete. 2024-09-10T23:38:42.962 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-10T23:38:42.962 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T23:38:42.962 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:38:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.100+0000 7fbec5200640 1 -- 172.21.15.94:0/936386226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbec00b2bb0 msgr2=0x7fbec00d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.100+0000 7fbec5200640 1 --2- 172.21.15.94:0/936386226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbec00b2bb0 0x7fbec00d00a0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fbec0074670 tx=0x7fbea801c4e0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.100+0000 7fbec5200640 1 -- 172.21.15.94:0/936386226 shutdown_connections 2024-09-10T23:38:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.100+0000 7fbec5200640 1 --2- 172.21.15.94:0/936386226 >> v2:172.21.15.181:3300/0 conn(0x7fbec00d05e0 0x7fbec00b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.100+0000 7fbec5200640 1 --2- 172.21.15.94:0/936386226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbec00b2bb0 0x7fbec00d00a0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.100+0000 7fbec5200640 1 --2- 172.21.15.94:0/936386226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbec015f5c0 0x7fbec00cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.100+0000 7fbec5200640 1 -- 172.21.15.94:0/936386226 >> 172.21.15.94:0/936386226 conn(0x7fbec00d1c60 msgr2=0x7fbec00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.100+0000 7fbec5200640 1 -- 172.21.15.94:0/936386226 shutdown_connections 2024-09-10T23:38:43.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.100+0000 7fbec5200640 1 -- 172.21.15.94:0/936386226 wait complete. 2024-09-10T23:38:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.100+0000 7fbec5200640 1 Processor -- start 2024-09-10T23:38:43.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbec5200640 1 -- start start 2024-09-10T23:38:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbec5200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbec00b2bb0 0x7fbec014a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbec5200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbec00d05e0 0x7fbec006f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbec5200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbec015f5c0 0x7fbec006f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbec5200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbec00afb30 con 0x7fbec00b2bb0 2024-09-10T23:38:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbec5200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbec00af9b0 con 0x7fbec00d05e0 2024-09-10T23:38:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbec5200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fbec00afcb0 con 0x7fbec015f5c0 2024-09-10T23:38:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbebde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbec00b2bb0 0x7fbec014a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbebde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbec00b2bb0 0x7fbec014a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45564/0 (socket says 172.21.15.94:45564) 2024-09-10T23:38:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbebde00640 1 -- 172.21.15.94:0/1152365152 learned_addr learned my addr 172.21.15.94:0/1152365152 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbeb5400640 1 --2- 172.21.15.94:0/1152365152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbec00d05e0 0x7fbec006f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbebe800640 1 --2- 172.21.15.94:0/1152365152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbec015f5c0 0x7fbec006f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbebde00640 1 -- 172.21.15.94:0/1152365152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbec015f5c0 msgr2=0x7fbec006f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:43.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbebde00640 1 --2- 172.21.15.94:0/1152365152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbec015f5c0 0x7fbec006f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbebde00640 1 -- 172.21.15.94:0/1152365152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbec00d05e0 msgr2=0x7fbec006f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbebde00640 1 --2- 172.21.15.94:0/1152365152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbec00d05e0 0x7fbec006f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbebde00640 1 -- 172.21.15.94:0/1152365152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec014ae20 con 0x7fbec00b2bb0 2024-09-10T23:38:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbebde00640 1 --2- 172.21.15.94:0/1152365152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbec00b2bb0 0x7fbec014a8e0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000c9f0 tx=0x7fbeb000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:43.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbeb6a00640 1 -- 172.21.15.94:0/1152365152 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0007bb0 con 0x7fbec00b2bb0 2024-09-10T23:38:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbeb6a00640 1 -- 172.21.15.94:0/1152365152 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbeb0007e40 con 0x7fbec00b2bb0 2024-09-10T23:38:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbec5200640 1 -- 172.21.15.94:0/1152365152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec014f3c0 con 0x7fbec00b2bb0 2024-09-10T23:38:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbeb6a00640 1 -- 172.21.15.94:0/1152365152 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb000f690 con 0x7fbec00b2bb0 2024-09-10T23:38:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbec5200640 1 -- 172.21.15.94:0/1152365152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec014f980 con 0x7fbec00b2bb0 2024-09-10T23:38:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.104+0000 7fbec5200640 1 -- 172.21.15.94:0/1152365152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec00b15c0 con 0x7fbec00b2bb0 2024-09-10T23:38:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.108+0000 7fbeb6a00640 1 -- 172.21.15.94:0/1152365152 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7fbeb0020030 con 0x7fbec00b2bb0 2024-09-10T23:38:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.108+0000 7fbeb6a00640 1 --2- 172.21.15.94:0/1152365152 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbe9c072c10 0x7fbe9c0750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.108+0000 7fbeb6a00640 1 -- 172.21.15.94:0/1152365152 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4971+0+0 (secure 0 0 0) 0x7fbeb0094120 con 0x7fbec00b2bb0 2024-09-10T23:38:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.108+0000 7fbeb5400640 1 --2- 172.21.15.94:0/1152365152 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbe9c072c10 0x7fbe9c0750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:43.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.108+0000 7fbeb5400640 1 --2- 172.21.15.94:0/1152365152 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbe9c072c10 0x7fbe9c0750d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbea80ae000 tx=0x7fbea801ad20 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:43.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.112+0000 7fbeb6a00640 1 -- 172.21.15.94:0/1152365152 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbeb0014030 con 0x7fbec00b2bb0 2024-09-10T23:38:43.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.292+0000 7fbec5200640 1 -- 172.21.15.94:0/1152365152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbec0153c20 con 0x7fbec00b2bb0 2024-09-10T23:38:43.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.296+0000 7fbeb6a00640 1 -- 172.21.15.94:0/1152365152 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+13076 (secure 0 0 0) 0x7fbeb005ea00 con 0x7fbec00b2bb0 2024-09-10T23:38:43.298 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:43.298 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":119,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:38:41.937778+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6834/1748173202":"2024-09-11T23:38:38.812829+0000","172.21.15.181:6801/1748173202":"2024-09-11T23:38:38.812829+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:38:43.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.296+0000 7fbec5200640 1 -- 172.21.15.94:0/1152365152 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbe9c072c10 msgr2=0x7fbe9c0750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:43.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.296+0000 7fbec5200640 1 --2- 172.21.15.94:0/1152365152 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbe9c072c10 0x7fbe9c0750d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbea80ae000 tx=0x7fbea801ad20 comp rx=0 tx=0).stop 2024-09-10T23:38:43.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.296+0000 7fbec5200640 1 -- 172.21.15.94:0/1152365152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbec00b2bb0 msgr2=0x7fbec014a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:43.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.296+0000 7fbec5200640 1 --2- 172.21.15.94:0/1152365152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbec00b2bb0 0x7fbec014a8e0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000c9f0 tx=0x7fbeb000ceb0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.296+0000 7fbec5200640 1 -- 172.21.15.94:0/1152365152 shutdown_connections 2024-09-10T23:38:43.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.296+0000 7fbec5200640 1 --2- 172.21.15.94:0/1152365152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbec00d05e0 0x7fbec006f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.296+0000 7fbec5200640 1 --2- 172.21.15.94:0/1152365152 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbe9c072c10 0x7fbe9c0750d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.296+0000 7fbec5200640 1 --2- 172.21.15.94:0/1152365152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbec015f5c0 0x7fbec006f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.296+0000 7fbec5200640 1 --2- 172.21.15.94:0/1152365152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbec00b2bb0 0x7fbec014a8e0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.296+0000 7fbec5200640 1 -- 172.21.15.94:0/1152365152 >> 172.21.15.94:0/1152365152 conn(0x7fbec00d1c60 msgr2=0x7fbec015caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:43.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.296+0000 7fbec5200640 1 -- 172.21.15.94:0/1152365152 shutdown_connections 2024-09-10T23:38:43.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.296+0000 7fbec5200640 1 -- 172.21.15.94:0/1152365152 wait complete. 2024-09-10T23:38:43.319 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T23:32:31.372521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T23:38:43.319 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T23:38:43.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 -- 172.21.15.94:0/3127203381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13480b1bf0 msgr2=0x7f13480cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:43.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 --2- 172.21.15.94:0/3127203381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13480b1bf0 0x7f13480cfa00 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f1348072ee0 tx=0x7f134001cd50 comp rx=0 tx=0).stop 2024-09-10T23:38:43.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 -- 172.21.15.94:0/3127203381 shutdown_connections 2024-09-10T23:38:43.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 --2- 172.21.15.94:0/3127203381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13480b1bf0 0x7f13480cfa00 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 --2- 172.21.15.94:0/3127203381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13480b2ac0 0x7f13480b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 --2- 172.21.15.94:0/3127203381 >> v2:172.21.15.181:3300/0 conn(0x7f134815f140 0x7f13480b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 -- 172.21.15.94:0/3127203381 >> 172.21.15.94:0/3127203381 conn(0x7f13480d1b00 msgr2=0x7f13480d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:43.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 -- 172.21.15.94:0/3127203381 shutdown_connections 2024-09-10T23:38:43.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 -- 172.21.15.94:0/3127203381 wait complete. 2024-09-10T23:38:43.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 Processor -- start 2024-09-10T23:38:43.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 -- start start 2024-09-10T23:38:43.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13480b1bf0 0x7f1348152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:43.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13480b2ac0 0x7f13480cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:43.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f134815f140 0x7f134815cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:43.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f13480af4c0 con 0x7f13480b1bf0 2024-09-10T23:38:43.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f13480af340 con 0x7f134815f140 2024-09-10T23:38:43.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1347200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f13480af640 con 0x7f13480b2ac0 2024-09-10T23:38:43.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.464+0000 7f1345e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13480b1bf0 0x7f1348152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:43.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f1345e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13480b1bf0 0x7f1348152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45580/0 (socket says 172.21.15.94:45580) 2024-09-10T23:38:43.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f1345e00640 1 -- 172.21.15.94:0/2582646271 learned_addr learned my addr 172.21.15.94:0/2582646271 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:43.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f133d400640 1 --2- 172.21.15.94:0/2582646271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13480b2ac0 0x7f13480cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:43.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f1346800640 1 --2- 172.21.15.94:0/2582646271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f134815f140 0x7f134815cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:43.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f1345e00640 1 -- 172.21.15.94:0/2582646271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13480b2ac0 msgr2=0x7f13480cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:43.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f1345e00640 1 --2- 172.21.15.94:0/2582646271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13480b2ac0 0x7f13480cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f1345e00640 1 -- 172.21.15.94:0/2582646271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f134815f140 msgr2=0x7f134815cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:43.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f1345e00640 1 --2- 172.21.15.94:0/2582646271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f134815f140 0x7f134815cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f1345e00640 1 -- 172.21.15.94:0/2582646271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1348152ff0 con 0x7f13480b1bf0 2024-09-10T23:38:43.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f1346800640 1 --2- 172.21.15.94:0/2582646271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f134815f140 0x7f134815cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:43.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f1345e00640 1 --2- 172.21.15.94:0/2582646271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13480b1bf0 0x7f1348152150 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f13380058d0 tx=0x7f133800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:43.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f133ea00640 1 -- 172.21.15.94:0/2582646271 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338017070 con 0x7f13480b1bf0 2024-09-10T23:38:43.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f133ea00640 1 -- 172.21.15.94:0/2582646271 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1338009d20 con 0x7f13480b1bf0 2024-09-10T23:38:43.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f133ea00640 1 -- 172.21.15.94:0/2582646271 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13380106c0 con 0x7f13480b1bf0 2024-09-10T23:38:43.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f1347200640 1 -- 172.21.15.94:0/2582646271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13480c56d0 con 0x7f13480b1bf0 2024-09-10T23:38:43.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f1347200640 1 -- 172.21.15.94:0/2582646271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13480c7fa0 con 0x7f13480b1bf0 2024-09-10T23:38:43.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f1347200640 1 -- 172.21.15.94:0/2582646271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13480cfa00 con 0x7f13480b1bf0 2024-09-10T23:38:43.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f133ea00640 1 -- 172.21.15.94:0/2582646271 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f1338010860 con 0x7f13480b1bf0 2024-09-10T23:38:43.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.468+0000 7f133ea00640 1 --2- 172.21.15.94:0/2582646271 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1324072c40 0x7f1324075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:43.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.472+0000 7f133ea00640 1 -- 172.21.15.94:0/2582646271 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4971+0+0 (secure 0 0 0) 0x7f1338093f50 con 0x7f13480b1bf0 2024-09-10T23:38:43.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.472+0000 7f133d400640 1 --2- 172.21.15.94:0/2582646271 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1324072c40 0x7f1324075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:43.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.472+0000 7f133d400640 1 --2- 172.21.15.94:0/2582646271 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1324072c40 0x7f1324075100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1348153710 tx=0x7f133000b310 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:43.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.472+0000 7f133ea00640 1 -- 172.21.15.94:0/2582646271 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f133805ef10 con 0x7f13480b1bf0 2024-09-10T23:38:43.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.636+0000 7f1347200640 1 -- 172.21.15.94:0/2582646271 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1348070f50 con 0x7f13480b1bf0 2024-09-10T23:38:43.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.640+0000 7f133ea00640 1 -- 172.21.15.94:0/2582646271 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7f133805e810 con 0x7f13480b1bf0 2024-09-10T23:38:43.642 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-09-10T23:38:43.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.640+0000 7f1347200640 1 -- 172.21.15.94:0/2582646271 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1324072c40 msgr2=0x7f1324075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:43.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.640+0000 7f1347200640 1 --2- 172.21.15.94:0/2582646271 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1324072c40 0x7f1324075100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1348153710 tx=0x7f133000b310 comp rx=0 tx=0).stop 2024-09-10T23:38:43.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.640+0000 7f1347200640 1 -- 172.21.15.94:0/2582646271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13480b1bf0 msgr2=0x7f1348152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:43.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.640+0000 7f1347200640 1 --2- 172.21.15.94:0/2582646271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13480b1bf0 0x7f1348152150 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f13380058d0 tx=0x7f133800dd20 comp rx=0 tx=0).stop 2024-09-10T23:38:43.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.640+0000 7f1347200640 1 -- 172.21.15.94:0/2582646271 shutdown_connections 2024-09-10T23:38:43.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.640+0000 7f1347200640 1 --2- 172.21.15.94:0/2582646271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f134815f140 0x7f134815cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.640+0000 7f1347200640 1 --2- 172.21.15.94:0/2582646271 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1324072c40 0x7f1324075100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.640+0000 7f1347200640 1 --2- 172.21.15.94:0/2582646271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13480b2ac0 0x7f13480cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.640+0000 7f1347200640 1 --2- 172.21.15.94:0/2582646271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13480b1bf0 0x7f1348152150 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.640+0000 7f1347200640 1 -- 172.21.15.94:0/2582646271 >> 172.21.15.94:0/2582646271 conn(0x7f13480d1b00 msgr2=0x7f1348153970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:43.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.640+0000 7f1347200640 1 -- 172.21.15.94:0/2582646271 shutdown_connections 2024-09-10T23:38:43.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.640+0000 7f1347200640 1 -- 172.21.15.94:0/2582646271 wait complete. 2024-09-10T23:38:43.663 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-10T23:38:43.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.800+0000 7f22e2400640 1 -- 172.21.15.94:0/2054107640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc0af310 msgr2=0x7f22dc0af7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:43.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.800+0000 7f22e2400640 1 --2- 172.21.15.94:0/2054107640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc0af310 0x7f22dc0af7c0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009900 tx=0x7f22d401c7e0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.800+0000 7f22e2400640 1 -- 172.21.15.94:0/2054107640 shutdown_connections 2024-09-10T23:38:43.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.800+0000 7f22e2400640 1 --2- 172.21.15.94:0/2054107640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc0af310 0x7f22dc0af7c0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.800+0000 7f22e2400640 1 --2- 172.21.15.94:0/2054107640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22dc0b2bb0 0x7f22dc0d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.800+0000 7f22e2400640 1 -- 172.21.15.94:0/2054107640 >> 172.21.15.94:0/2054107640 conn(0x7f22dc0d1c60 msgr2=0x7f22dc0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:43.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.800+0000 7f22e2400640 1 -- 172.21.15.94:0/2054107640 shutdown_connections 2024-09-10T23:38:43.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.800+0000 7f22e2400640 1 -- 172.21.15.94:0/2054107640 wait complete. 2024-09-10T23:38:43.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22e2400640 1 Processor -- start 2024-09-10T23:38:43.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22e2400640 1 -- start start 2024-09-10T23:38:43.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22e2400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22dc0af310 0x7f22dc166540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:43.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22e2400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22dc0b2bb0 0x7f22dc1fc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:43.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22e2400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc15f5c0 0x7f22dc1501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:43.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22e2400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc1690c0 con 0x7f22dc15f5c0 2024-09-10T23:38:43.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22e2400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc168f40 con 0x7f22dc0b2bb0 2024-09-10T23:38:43.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22e2400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc169240 con 0x7f22dc0af310 2024-09-10T23:38:43.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22dbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc15f5c0 0x7f22dc1501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22dbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc15f5c0 0x7f22dc1501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53168/0 (socket says 172.21.15.94:53168) 2024-09-10T23:38:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22dbe00640 1 -- 172.21.15.94:0/3413266569 learned_addr learned my addr 172.21.15.94:0/3413266569 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22db400640 1 --2- 172.21.15.94:0/3413266569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22dc0af310 0x7f22dc166540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22d2a00640 1 --2- 172.21.15.94:0/3413266569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22dc0b2bb0 0x7f22dc1fc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22dbe00640 1 -- 172.21.15.94:0/3413266569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22dc0af310 msgr2=0x7f22dc166540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22dbe00640 1 --2- 172.21.15.94:0/3413266569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22dc0af310 0x7f22dc166540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22dbe00640 1 -- 172.21.15.94:0/3413266569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22dc0b2bb0 msgr2=0x7f22dc1fc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22dbe00640 1 --2- 172.21.15.94:0/3413266569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22dc0b2bb0 0x7f22dc1fc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22dbe00640 1 -- 172.21.15.94:0/3413266569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc14fc00 con 0x7f22dc15f5c0 2024-09-10T23:38:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22d2a00640 1 --2- 172.21.15.94:0/3413266569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22dc0b2bb0 0x7f22dc1fc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22dbe00640 1 --2- 172.21.15.94:0/3413266569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc15f5c0 0x7f22dc1501c0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f22d40b0040 tx=0x7f22d40a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22d3e00640 1 -- 172.21.15.94:0/3413266569 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d40b2040 con 0x7f22dc15f5c0 2024-09-10T23:38:43.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22d3e00640 1 -- 172.21.15.94:0/3413266569 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22d40043c0 con 0x7f22dc15f5c0 2024-09-10T23:38:43.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22e2400640 1 -- 172.21.15.94:0/3413266569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc14fee0 con 0x7f22dc15f5c0 2024-09-10T23:38:43.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22d3e00640 1 -- 172.21.15.94:0/3413266569 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d40052d0 con 0x7f22dc15f5c0 2024-09-10T23:38:43.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22e2400640 1 -- 172.21.15.94:0/3413266569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc1fb880 con 0x7f22dc15f5c0 2024-09-10T23:38:43.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.804+0000 7f22e2400640 1 -- 172.21.15.94:0/3413266569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc0af7c0 con 0x7f22dc15f5c0 2024-09-10T23:38:43.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.808+0000 7f22d3e00640 1 -- 172.21.15.94:0/3413266569 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f22d40046d0 con 0x7f22dc15f5c0 2024-09-10T23:38:43.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.808+0000 7f22d3e00640 1 --2- 172.21.15.94:0/3413266569 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f22b8072c40 0x7f22b8075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:43.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.808+0000 7f22d3e00640 1 -- 172.21.15.94:0/3413266569 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4971+0+0 (secure 0 0 0) 0x7f22d412d820 con 0x7f22dc15f5c0 2024-09-10T23:38:43.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.808+0000 7f22db400640 1 --2- 172.21.15.94:0/3413266569 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f22b8072c40 0x7f22b8075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:43.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.808+0000 7f22db400640 1 --2- 172.21.15.94:0/3413266569 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f22b8072c40 0x7f22b8075100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f22cc001cf0 tx=0x7f22cc009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:43.812+0000 7f22d3e00640 1 -- 172.21.15.94:0/3413266569 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f22d40164f0 con 0x7f22dc15f5c0 2024-09-10T23:38:44.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:44.040+0000 7f22e2400640 1 -- 172.21.15.94:0/3413266569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7f22dc14d620 con 0x7f22dc15f5c0 2024-09-10T23:38:44.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:44.944+0000 7f22d3e00640 1 -- 172.21.15.94:0/3413266569 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v340) ==== 154+0+0 (secure 0 0 0) 0x7f22d40f87e0 con 0x7f22dc15f5c0 2024-09-10T23:38:44.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:44.948+0000 7f22e2400640 1 -- 172.21.15.94:0/3413266569 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f22b8072c40 msgr2=0x7f22b8075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:44.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:44.948+0000 7f22e2400640 1 --2- 172.21.15.94:0/3413266569 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f22b8072c40 0x7f22b8075100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f22cc001cf0 tx=0x7f22cc009040 comp rx=0 tx=0).stop 2024-09-10T23:38:44.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:44.948+0000 7f22e2400640 1 -- 172.21.15.94:0/3413266569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc15f5c0 msgr2=0x7f22dc1501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:44.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:44.948+0000 7f22e2400640 1 --2- 172.21.15.94:0/3413266569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc15f5c0 0x7f22dc1501c0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f22d40b0040 tx=0x7f22d40a5710 comp rx=0 tx=0).stop 2024-09-10T23:38:44.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:44.948+0000 7f22e2400640 1 -- 172.21.15.94:0/3413266569 shutdown_connections 2024-09-10T23:38:44.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:44.948+0000 7f22e2400640 1 --2- 172.21.15.94:0/3413266569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22dc0b2bb0 0x7f22dc1fc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:44.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:44.948+0000 7f22e2400640 1 --2- 172.21.15.94:0/3413266569 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f22b8072c40 0x7f22b8075100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:44.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:44.948+0000 7f22e2400640 1 --2- 172.21.15.94:0/3413266569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22dc15f5c0 0x7f22dc1501c0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:44.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:44.948+0000 7f22e2400640 1 --2- 172.21.15.94:0/3413266569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22dc0af310 0x7f22dc166540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:44.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:44.948+0000 7f22e2400640 1 -- 172.21.15.94:0/3413266569 >> 172.21.15.94:0/3413266569 conn(0x7f22dc0d1c60 msgr2=0x7f22dc14d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:44.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:44.948+0000 7f22e2400640 1 -- 172.21.15.94:0/3413266569 shutdown_connections 2024-09-10T23:38:44.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:44.948+0000 7f22e2400640 1 -- 172.21.15.94:0/3413266569 wait complete. 2024-09-10T23:38:44.973 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T23:38:45.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.112+0000 7f44cf000640 1 -- 172.21.15.94:0/3234542344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c8073ff0 msgr2=0x7f44c80cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:45.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.112+0000 7f44cf000640 1 --2- 172.21.15.94:0/3234542344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c8073ff0 0x7f44c80cfda0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009900 tx=0x7f44b801c8b0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.112+0000 7f44cf000640 1 -- 172.21.15.94:0/3234542344 shutdown_connections 2024-09-10T23:38:45.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.112+0000 7f44cf000640 1 --2- 172.21.15.94:0/3234542344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c80d02e0 0x7f44c815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.112+0000 7f44cf000640 1 --2- 172.21.15.94:0/3234542344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c8073ff0 0x7f44c80cfda0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.112+0000 7f44cf000640 1 -- 172.21.15.94:0/3234542344 >> 172.21.15.94:0/3234542344 conn(0x7f44c806cdf0 msgr2=0x7f44c806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:45.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44cf000640 1 -- 172.21.15.94:0/3234542344 shutdown_connections 2024-09-10T23:38:45.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44cf000640 1 -- 172.21.15.94:0/3234542344 wait complete. 2024-09-10T23:38:45.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44cf000640 1 Processor -- start 2024-09-10T23:38:45.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44cf000640 1 -- start start 2024-09-10T23:38:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44cf000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c8073ff0 0x7f44c80b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44cf000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c80d02e0 0x7f44c80b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44cf000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c815ecb0 0x7f44c8140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44cf000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f44c8167930 con 0x7f44c80d02e0 2024-09-10T23:38:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44cf000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f44c81677b0 con 0x7f44c8073ff0 2024-09-10T23:38:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44cf000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f44c8167ab0 con 0x7f44c815ecb0 2024-09-10T23:38:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44bf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c80d02e0 0x7f44c80b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44bf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c80d02e0 0x7f44c80b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53184/0 (socket says 172.21.15.94:53184) 2024-09-10T23:38:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44bf400640 1 -- 172.21.15.94:0/3082808950 learned_addr learned my addr 172.21.15.94:0/3082808950 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44bf400640 1 -- 172.21.15.94:0/3082808950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c815ecb0 msgr2=0x7f44c8140040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:45.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44bf400640 1 --2- 172.21.15.94:0/3082808950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c815ecb0 0x7f44c8140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44bf400640 1 -- 172.21.15.94:0/3082808950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c8073ff0 msgr2=0x7f44c80b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44bf400640 1 --2- 172.21.15.94:0/3082808950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c8073ff0 0x7f44c80b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44bf400640 1 -- 172.21.15.94:0/3082808950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c81443c0 con 0x7f44c80d02e0 2024-09-10T23:38:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44bf400640 1 --2- 172.21.15.94:0/3082808950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c80d02e0 0x7f44c80b1d50 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f44b000d950 tx=0x7f44b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44c4c00640 1 -- 172.21.15.94:0/3082808950 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0017070 con 0x7f44c80d02e0 2024-09-10T23:38:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44c4c00640 1 -- 172.21.15.94:0/3082808950 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44b0009d20 con 0x7f44c80d02e0 2024-09-10T23:38:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44c4c00640 1 -- 172.21.15.94:0/3082808950 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b00106c0 con 0x7f44c80d02e0 2024-09-10T23:38:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.116+0000 7f44cf000640 1 -- 172.21.15.94:0/3082808950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c8140c60 con 0x7f44c80d02e0 2024-09-10T23:38:45.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.120+0000 7f44cf000640 1 -- 172.21.15.94:0/3082808950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c813e090 con 0x7f44c80d02e0 2024-09-10T23:38:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.120+0000 7f44cf000640 1 -- 172.21.15.94:0/3082808950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c80cfda0 con 0x7f44c80d02e0 2024-09-10T23:38:45.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.120+0000 7f44c4c00640 1 -- 172.21.15.94:0/3082808950 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f44b0010860 con 0x7f44c80d02e0 2024-09-10T23:38:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.120+0000 7f44c4c00640 1 --2- 172.21.15.94:0/3082808950 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44a4072c40 0x7f44a4075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.120+0000 7f44c4c00640 1 -- 172.21.15.94:0/3082808950 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4971+0+0 (secure 0 0 0) 0x7f44b0093ff0 con 0x7f44c80d02e0 2024-09-10T23:38:45.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.120+0000 7f44c7e00640 1 --2- 172.21.15.94:0/3082808950 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44a4072c40 0x7f44a4075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:45.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.120+0000 7f44c7e00640 1 --2- 172.21.15.94:0/3082808950 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44a4072c40 0x7f44a4075100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f44b80af000 tx=0x7f44b8019650 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:45.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.124+0000 7f44c4c00640 1 -- 172.21.15.94:0/3082808950 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f44c81443c0 con 0x7f44c80d02e0 2024-09-10T23:38:45.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.288+0000 7f44cf000640 1 -- 172.21.15.94:0/3082808950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f44c80cd650 con 0x7f44c80d02e0 2024-09-10T23:38:45.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.292+0000 7f44c4c00640 1 -- 172.21.15.94:0/3082808950 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v41) ==== 59+0+1998 (secure 0 0 0) 0x7f44b005ef20 con 0x7f44c80d02e0 2024-09-10T23:38:45.294 INFO:teuthology.orchestra.run.smithi094.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T23:38:45.294 INFO:teuthology.orchestra.run.smithi094.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T23:38:45.294 INFO:teuthology.orchestra.run.smithi094.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T23:38:45.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T23:38:45.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3944.074506 2024-09-10T23:38:45.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3865.464527 2024-09-10T23:38:45.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3898.861024 2024-09-10T23:38:45.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4010.048529 2024-09-10T23:38:45.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4650.562436 2024-09-10T23:38:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4724.018776 2024-09-10T23:38:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5255.753663 2024-09-10T23:38:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4718.009443 2024-09-10T23:38:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds 20/20 2024-09-10T23:38:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T23:38:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_ms 2/2 2024-09-10T23:38:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_frag true 2024-09-10T23:38:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_scatterstat true 2024-09-10T23:38:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T23:38:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_verify_scatter true 2024-09-10T23:38:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T23:38:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T23:38:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T23:38:45.295 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced client_mount_timeout 600 2024-09-10T23:38:45.296 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_client 20/20 2024-09-10T23:38:45.296 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_ms 1/1 2024-09-10T23:38:45.296 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T23:38:45.296 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T23:38:45.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.292+0000 7f44cf000640 1 -- 172.21.15.94:0/3082808950 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44a4072c40 msgr2=0x7f44a4075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:45.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.292+0000 7f44cf000640 1 --2- 172.21.15.94:0/3082808950 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44a4072c40 0x7f44a4075100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f44b80af000 tx=0x7f44b8019650 comp rx=0 tx=0).stop 2024-09-10T23:38:45.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.292+0000 7f44cf000640 1 -- 172.21.15.94:0/3082808950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c80d02e0 msgr2=0x7f44c80b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:45.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.292+0000 7f44cf000640 1 --2- 172.21.15.94:0/3082808950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c80d02e0 0x7f44c80b1d50 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f44b000d950 tx=0x7f44b000de10 comp rx=0 tx=0).stop 2024-09-10T23:38:45.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.292+0000 7f44cf000640 1 -- 172.21.15.94:0/3082808950 shutdown_connections 2024-09-10T23:38:45.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.292+0000 7f44cf000640 1 --2- 172.21.15.94:0/3082808950 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44a4072c40 0x7f44a4075100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.292+0000 7f44cf000640 1 --2- 172.21.15.94:0/3082808950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c815ecb0 0x7f44c8140040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.292+0000 7f44cf000640 1 --2- 172.21.15.94:0/3082808950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44c80d02e0 0x7f44c80b1d50 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.292+0000 7f44cf000640 1 --2- 172.21.15.94:0/3082808950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c8073ff0 0x7f44c80b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.292+0000 7f44cf000640 1 -- 172.21.15.94:0/3082808950 >> 172.21.15.94:0/3082808950 conn(0x7f44c806cdf0 msgr2=0x7f44c806f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:45.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.292+0000 7f44cf000640 1 -- 172.21.15.94:0/3082808950 shutdown_connections 2024-09-10T23:38:45.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.292+0000 7f44cf000640 1 -- 172.21.15.94:0/3082808950 wait complete. 2024-09-10T23:38:45.315 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:38:45.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.456+0000 7f838e400640 1 -- 172.21.15.94:0/1425877956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83880cb4d0 msgr2=0x7f838815b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:45.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.456+0000 7f838e400640 1 --2- 172.21.15.94:0/1425877956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83880cb4d0 0x7f838815b190 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f838806b000 tx=0x7f838001cd80 comp rx=0 tx=0).stop 2024-09-10T23:38:45.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.456+0000 7f838e400640 1 -- 172.21.15.94:0/1425877956 shutdown_connections 2024-09-10T23:38:45.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.456+0000 7f838e400640 1 --2- 172.21.15.94:0/1425877956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83880cb4d0 0x7f838815b190 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.456+0000 7f838e400640 1 --2- 172.21.15.94:0/1425877956 >> v2:172.21.15.181:3300/0 conn(0x7f8388158790 0x7f838806fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.456+0000 7f838e400640 1 -- 172.21.15.94:0/1425877956 >> 172.21.15.94:0/1425877956 conn(0x7f83880cfaf0 msgr2=0x7f83880cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:45.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.456+0000 7f838e400640 1 -- 172.21.15.94:0/1425877956 shutdown_connections 2024-09-10T23:38:45.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.456+0000 7f838e400640 1 -- 172.21.15.94:0/1425877956 wait complete. 2024-09-10T23:38:45.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.456+0000 7f838e400640 1 Processor -- start 2024-09-10T23:38:45.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.456+0000 7f838e400640 1 -- start start 2024-09-10T23:38:45.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f838e400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83880c49b0 0x7f83881fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:45.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f838e400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83880cb4d0 0x7f83880c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:45.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f838e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8388158790 0x7f83880b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:45.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f838e400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8388167f60 con 0x7f8388158790 2024-09-10T23:38:45.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f838e400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8388167de0 con 0x7f83880cb4d0 2024-09-10T23:38:45.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f838e400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f83881680e0 con 0x7f83880c49b0 2024-09-10T23:38:45.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f8387e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8388158790 0x7f83880b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:45.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f8387e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8388158790 0x7f83880b4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53202/0 (socket says 172.21.15.94:53202) 2024-09-10T23:38:45.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f8387e00640 1 -- 172.21.15.94:0/1762103707 learned_addr learned my addr 172.21.15.94:0/1762103707 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:45.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f8387400640 1 --2- 172.21.15.94:0/1762103707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83880c49b0 0x7f83881fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:45.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f8386a00640 1 --2- 172.21.15.94:0/1762103707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83880cb4d0 0x7f83880c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:45.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f8387e00640 1 -- 172.21.15.94:0/1762103707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83880c49b0 msgr2=0x7f83881fbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:45.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f8387e00640 1 --2- 172.21.15.94:0/1762103707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83880c49b0 0x7f83881fbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f8387e00640 1 -- 172.21.15.94:0/1762103707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83880cb4d0 msgr2=0x7f83880c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:45.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f8387e00640 1 --2- 172.21.15.94:0/1762103707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83880cb4d0 0x7f83880c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f8387e00640 1 -- 172.21.15.94:0/1762103707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83880b5b00 con 0x7f8388158790 2024-09-10T23:38:45.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f8387e00640 1 --2- 172.21.15.94:0/1762103707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8388158790 0x7f83880b4fb0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8380019990 tx=0x7f8380003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:45.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f837f400640 1 -- 172.21.15.94:0/1762103707 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83800bb070 con 0x7f8388158790 2024-09-10T23:38:45.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f837f400640 1 -- 172.21.15.94:0/1762103707 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8380004e00 con 0x7f8388158790 2024-09-10T23:38:45.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f837f400640 1 -- 172.21.15.94:0/1762103707 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83800b2040 con 0x7f8388158790 2024-09-10T23:38:45.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f838e400640 1 -- 172.21.15.94:0/1762103707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83881fb300 con 0x7f8388158790 2024-09-10T23:38:45.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f838e400640 1 -- 172.21.15.94:0/1762103707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83881fb5a0 con 0x7f8388158790 2024-09-10T23:38:45.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.460+0000 7f838e400640 1 -- 172.21.15.94:0/1762103707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f838815b190 con 0x7f8388158790 2024-09-10T23:38:45.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.464+0000 7f837f400640 1 -- 172.21.15.94:0/1762103707 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f83800b5030 con 0x7f8388158790 2024-09-10T23:38:45.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.464+0000 7f837f400640 1 --2- 172.21.15.94:0/1762103707 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8368072c40 0x7f8368075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:45.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.464+0000 7f837f400640 1 -- 172.21.15.94:0/1762103707 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4971+0+0 (secure 0 0 0) 0x7f838012d9e0 con 0x7f8388158790 2024-09-10T23:38:45.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.464+0000 7f8387400640 1 --2- 172.21.15.94:0/1762103707 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8368072c40 0x7f8368075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:45.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.464+0000 7f8387400640 1 --2- 172.21.15.94:0/1762103707 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8368072c40 0x7f8368075100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f837800bb50 tx=0x7f837800b2e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:45.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.468+0000 7f837f400640 1 -- 172.21.15.94:0/1762103707 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8380016970 con 0x7f8388158790 2024-09-10T23:38:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.632+0000 7f838e400640 1 -- 172.21.15.94:0/1762103707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8388069e20 con 0x7f8388158790 2024-09-10T23:38:45.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.632+0000 7f837f400640 1 -- 172.21.15.94:0/1762103707 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7f8388069e20 con 0x7f8388158790 2024-09-10T23:38:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.636+0000 7f838e400640 1 -- 172.21.15.94:0/1762103707 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8368072c40 msgr2=0x7f8368075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.636+0000 7f838e400640 1 --2- 172.21.15.94:0/1762103707 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8368072c40 0x7f8368075100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f837800bb50 tx=0x7f837800b2e0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.636+0000 7f838e400640 1 -- 172.21.15.94:0/1762103707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8388158790 msgr2=0x7f83880b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.636+0000 7f838e400640 1 --2- 172.21.15.94:0/1762103707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8388158790 0x7f83880b4fb0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8380019990 tx=0x7f8380003f60 comp rx=0 tx=0).stop 2024-09-10T23:38:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.636+0000 7f838e400640 1 -- 172.21.15.94:0/1762103707 shutdown_connections 2024-09-10T23:38:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.636+0000 7f838e400640 1 --2- 172.21.15.94:0/1762103707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83880cb4d0 0x7f83880c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.636+0000 7f838e400640 1 --2- 172.21.15.94:0/1762103707 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8368072c40 0x7f8368075100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.636+0000 7f838e400640 1 --2- 172.21.15.94:0/1762103707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8388158790 0x7f83880b4fb0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.636+0000 7f838e400640 1 --2- 172.21.15.94:0/1762103707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83880c49b0 0x7f83881fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.636+0000 7f838e400640 1 -- 172.21.15.94:0/1762103707 >> 172.21.15.94:0/1762103707 conn(0x7f83880cfaf0 msgr2=0x7f838806f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.636+0000 7f838e400640 1 -- 172.21.15.94:0/1762103707 shutdown_connections 2024-09-10T23:38:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:45.636+0000 7f838e400640 1 -- 172.21.15.94:0/1762103707 wait complete. 2024-09-10T23:38:45.659 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:38:45.659 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:38:45.735 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:38:45.839 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:38:45.839 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:38:45.840 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:45.848 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:38:45.848 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:38:45.849 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:45.858 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:38:45.858 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:38:45.858 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:45.897 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:38:45.897 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:38:45.898 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:38:45.912 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:38:45.913 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T23:38:45.913 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:38:46.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.080+0000 7fc665000640 1 -- 172.21.15.94:0/1036202500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6600ae980 msgr2=0x7fc6600d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:46.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.080+0000 7fc665000640 1 --2- 172.21.15.94:0/1036202500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6600ae980 0x7fc6600d1b80 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fc6600cf6b0 tx=0x7fc64801c8c0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.080+0000 7fc665000640 1 -- 172.21.15.94:0/1036202500 shutdown_connections 2024-09-10T23:38:46.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.080+0000 7fc665000640 1 --2- 172.21.15.94:0/1036202500 >> v2:172.21.15.181:3300/0 conn(0x7fc6600b2ec0 0x7fc6600d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.080+0000 7fc665000640 1 --2- 172.21.15.94:0/1036202500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6600ae980 0x7fc6600d1b80 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.080+0000 7fc665000640 1 --2- 172.21.15.94:0/1036202500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc66015bd30 0x7fc66015c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.080+0000 7fc665000640 1 -- 172.21.15.94:0/1036202500 >> 172.21.15.94:0/1036202500 conn(0x7fc66009aa70 msgr2=0x7fc66009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:46.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.080+0000 7fc665000640 1 -- 172.21.15.94:0/1036202500 shutdown_connections 2024-09-10T23:38:46.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.080+0000 7fc665000640 1 -- 172.21.15.94:0/1036202500 wait complete. 2024-09-10T23:38:46.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.080+0000 7fc665000640 1 Processor -- start 2024-09-10T23:38:46.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.080+0000 7fc665000640 1 -- start start 2024-09-10T23:38:46.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc665000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6600ae980 0x7fc6600b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:46.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc665000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6600b2ec0 0x7fc660154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:46.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc665000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc66015bd30 0x7fc660155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:46.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc665000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc6600b1ff0 con 0x7fc6600ae980 2024-09-10T23:38:46.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc665000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc6600b1e70 con 0x7fc66015bd30 2024-09-10T23:38:46.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc665000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc6600b2170 con 0x7fc6600b2ec0 2024-09-10T23:38:46.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc65de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6600ae980 0x7fc6600b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:46.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc65de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6600ae980 0x7fc6600b6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53218/0 (socket says 172.21.15.94:53218) 2024-09-10T23:38:46.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc65de00640 1 -- 172.21.15.94:0/678058721 learned_addr learned my addr 172.21.15.94:0/678058721 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:46.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc655400640 1 --2- 172.21.15.94:0/678058721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6600b2ec0 0x7fc660154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:46.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc65e800640 1 --2- 172.21.15.94:0/678058721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc66015bd30 0x7fc660155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:46.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc65de00640 1 -- 172.21.15.94:0/678058721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6600b2ec0 msgr2=0x7fc660154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:46.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc65de00640 1 --2- 172.21.15.94:0/678058721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6600b2ec0 0x7fc660154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc65de00640 1 -- 172.21.15.94:0/678058721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc66015bd30 msgr2=0x7fc660155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:46.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc65de00640 1 --2- 172.21.15.94:0/678058721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc66015bd30 0x7fc660155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc65de00640 1 -- 172.21.15.94:0/678058721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6600b56d0 con 0x7fc6600ae980 2024-09-10T23:38:46.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc65e800640 1 --2- 172.21.15.94:0/678058721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc66015bd30 0x7fc660155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:46.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc65de00640 1 --2- 172.21.15.94:0/678058721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6600ae980 0x7fc6600b6b40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fc65000d720 tx=0x7fc65000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:46.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc656a00640 1 -- 172.21.15.94:0/678058721 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650017070 con 0x7fc6600ae980 2024-09-10T23:38:46.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc656a00640 1 -- 172.21.15.94:0/678058721 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc650009cc0 con 0x7fc6600ae980 2024-09-10T23:38:46.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc656a00640 1 -- 172.21.15.94:0/678058721 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650005280 con 0x7fc6600ae980 2024-09-10T23:38:46.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc665000640 1 -- 172.21.15.94:0/678058721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6600b7170 con 0x7fc6600ae980 2024-09-10T23:38:46.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc665000640 1 -- 172.21.15.94:0/678058721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6600b7730 con 0x7fc6600ae980 2024-09-10T23:38:46.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc665000640 1 -- 172.21.15.94:0/678058721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc66015c110 con 0x7fc6600ae980 2024-09-10T23:38:46.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.084+0000 7fc656a00640 1 -- 172.21.15.94:0/678058721 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7fc650005420 con 0x7fc6600ae980 2024-09-10T23:38:46.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.088+0000 7fc656a00640 1 --2- 172.21.15.94:0/678058721 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc63c072c40 0x7fc63c075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:46.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.088+0000 7fc656a00640 1 -- 172.21.15.94:0/678058721 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4971+0+0 (secure 0 0 0) 0x7fc650093b70 con 0x7fc6600ae980 2024-09-10T23:38:46.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.088+0000 7fc655400640 1 --2- 172.21.15.94:0/678058721 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc63c072c40 0x7fc63c075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:46.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.088+0000 7fc655400640 1 --2- 172.21.15.94:0/678058721 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc63c072c40 0x7fc63c075100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc6480af000 tx=0x7fc64801b100 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:46.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.092+0000 7fc656a00640 1 -- 172.21.15.94:0/678058721 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc65005e4e0 con 0x7fc6600ae980 2024-09-10T23:38:46.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.288+0000 7fc665000640 1 -- 172.21.15.94:0/678058721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6600d28a0 con 0x7fc6600ae980 2024-09-10T23:38:46.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.292+0000 7fc656a00640 1 -- 172.21.15.94:0/678058721 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+3916 (secure 0 0 0) 0x7fc650067250 con 0x7fc6600ae980 2024-09-10T23:38:46.294 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:46.294 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"btime":"2024-09-10T23:38:39:893112+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7090,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3097381199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3097381199},{"type":"v1","addr":"172.21.15.181:6835","nonce":3097381199}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/2654663117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2654663117},{"type":"v1","addr":"172.21.15.94:6833","nonce":2654663117}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":72},{"gid":7530,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1334394570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1334394570},{"type":"v1","addr":"172.21.15.94:6835","nonce":1334394570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":196502,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6834/798604600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":798604600},{"type":"v1","addr":"172.21.15.181:6834","nonce":798604600}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[]} 2024-09-10T23:38:46.294 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 81 2024-09-10T23:38:46.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.292+0000 7fc665000640 1 -- 172.21.15.94:0/678058721 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc63c072c40 msgr2=0x7fc63c075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:46.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.292+0000 7fc665000640 1 --2- 172.21.15.94:0/678058721 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc63c072c40 0x7fc63c075100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc6480af000 tx=0x7fc64801b100 comp rx=0 tx=0).stop 2024-09-10T23:38:46.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.292+0000 7fc665000640 1 -- 172.21.15.94:0/678058721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6600ae980 msgr2=0x7fc6600b6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:46.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.292+0000 7fc665000640 1 --2- 172.21.15.94:0/678058721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6600ae980 0x7fc6600b6b40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fc65000d720 tx=0x7fc65000dbe0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.292+0000 7fc665000640 1 -- 172.21.15.94:0/678058721 shutdown_connections 2024-09-10T23:38:46.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.292+0000 7fc665000640 1 --2- 172.21.15.94:0/678058721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc66015bd30 0x7fc660155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.292+0000 7fc665000640 1 --2- 172.21.15.94:0/678058721 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc63c072c40 0x7fc63c075100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.292+0000 7fc665000640 1 --2- 172.21.15.94:0/678058721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6600b2ec0 0x7fc660154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.292+0000 7fc665000640 1 --2- 172.21.15.94:0/678058721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6600ae980 0x7fc6600b6b40 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.292+0000 7fc665000640 1 -- 172.21.15.94:0/678058721 >> 172.21.15.94:0/678058721 conn(0x7fc66009aa70 msgr2=0x7fc66015ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:46.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.292+0000 7fc665000640 1 -- 172.21.15.94:0/678058721 shutdown_connections 2024-09-10T23:38:46.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.292+0000 7fc665000640 1 -- 172.21.15.94:0/678058721 wait complete. 2024-09-10T23:38:46.315 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T23:38:46.316 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T23:38:46.316 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T23:38:46.316 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:38:46.316 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T23:38:46.316 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T23:38:46.316 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T23:38:46.317 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:38:46.317 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T23:38:46.317 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T23:38:46.317 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T23:38:46.317 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:38:46.317 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T23:38:46.317 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T23:38:46.318 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T23:38:46.318 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:38:46.329 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T23:38:46.329 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T23:38:46.330 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T23:38:46.330 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T23:38:46.332 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T23:38:46.332 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T23:38:46.333 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T23:38:46.333 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T23:38:46.375 INFO:tasks.ceph.mds.b:Started 2024-09-10T23:38:46.376 DEBUG:teuthology.parallel:result is None 2024-09-10T23:38:46.377 INFO:tasks.ceph.mds.c:Started 2024-09-10T23:38:46.377 DEBUG:teuthology.parallel:result is None 2024-09-10T23:38:46.377 INFO:tasks.ceph.mds.d:Started 2024-09-10T23:38:46.378 INFO:tasks.ceph.mds.a:Started 2024-09-10T23:38:46.378 DEBUG:teuthology.parallel:result is None 2024-09-10T23:38:46.378 DEBUG:teuthology.parallel:result is None 2024-09-10T23:38:46.378 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T23:38:46.461 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-09-10T23:38:46.462 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-09-10T23:38:46.466 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2024-09-10T23:38:46.467 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:38:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.524+0000 7fc69aa00640 1 -- 172.21.15.94:0/77566372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6940ce1e0 msgr2=0x7fc6940cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.524+0000 7fc69aa00640 1 --2- 172.21.15.94:0/77566372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6940ce1e0 0x7fc6940cd710 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc680009840 tx=0x7fc68001c790 comp rx=0 tx=0).stop 2024-09-10T23:38:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.524+0000 7fc69aa00640 1 -- 172.21.15.94:0/77566372 shutdown_connections 2024-09-10T23:38:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.524+0000 7fc69aa00640 1 --2- 172.21.15.94:0/77566372 >> v2:172.21.15.181:3300/0 conn(0x7fc6940b1100 0x7fc6940b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.524+0000 7fc69aa00640 1 --2- 172.21.15.94:0/77566372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6940ce1e0 0x7fc6940cd710 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.524+0000 7fc69aa00640 1 -- 172.21.15.94:0/77566372 >> 172.21.15.94:0/77566372 conn(0x7fc6940d1c60 msgr2=0x7fc6940d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.524+0000 7fc69aa00640 1 -- 172.21.15.94:0/77566372 shutdown_connections 2024-09-10T23:38:46.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.524+0000 7fc69aa00640 1 -- 172.21.15.94:0/77566372 wait complete. 2024-09-10T23:38:46.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc69aa00640 1 Processor -- start 2024-09-10T23:38:46.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc69aa00640 1 -- start start 2024-09-10T23:38:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc69aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6940b1100 0x7fc6940ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc69aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6940ce1e0 0x7fc694151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc69aa00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6940ce890 0x7fc694158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc69aa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc694169110 con 0x7fc6940b1100 2024-09-10T23:38:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc69aa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc6940d09c0 con 0x7fc6940ce1e0 2024-09-10T23:38:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc69aa00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc694169290 con 0x7fc6940ce890 2024-09-10T23:38:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc693e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6940b1100 0x7fc6940ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc693e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6940b1100 0x7fc6940ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53246/0 (socket says 172.21.15.94:53246) 2024-09-10T23:38:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc693e00640 1 -- 172.21.15.94:0/1516363781 learned_addr learned my addr 172.21.15.94:0/1516363781 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc698a00640 1 --2- 172.21.15.94:0/1516363781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6940ce890 0x7fc694158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc693400640 1 --2- 172.21.15.94:0/1516363781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6940ce1e0 0x7fc694151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc693e00640 1 -- 172.21.15.94:0/1516363781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6940ce890 msgr2=0x7fc694158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc693e00640 1 --2- 172.21.15.94:0/1516363781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6940ce890 0x7fc694158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc693e00640 1 -- 172.21.15.94:0/1516363781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6940ce1e0 msgr2=0x7fc694151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc693e00640 1 --2- 172.21.15.94:0/1516363781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6940ce1e0 0x7fc694151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc693e00640 1 -- 172.21.15.94:0/1516363781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6940b4be0 con 0x7fc6940b1100 2024-09-10T23:38:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc693e00640 1 --2- 172.21.15.94:0/1516363781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6940b1100 0x7fc6940ca280 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fc68400c460 tx=0x7fc68400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:46.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc68be00640 1 -- 172.21.15.94:0/1516363781 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684016020 con 0x7fc6940b1100 2024-09-10T23:38:46.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc68be00640 1 -- 172.21.15.94:0/1516363781 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc684012940 con 0x7fc6940b1100 2024-09-10T23:38:46.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc68be00640 1 -- 172.21.15.94:0/1516363781 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684012d90 con 0x7fc6940b1100 2024-09-10T23:38:46.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc69aa00640 1 -- 172.21.15.94:0/1516363781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6940b4ec0 con 0x7fc6940b1100 2024-09-10T23:38:46.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.528+0000 7fc69aa00640 1 -- 172.21.15.94:0/1516363781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6940b5480 con 0x7fc6940b1100 2024-09-10T23:38:46.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.532+0000 7fc69aa00640 1 -- 172.21.15.94:0/1516363781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6940b15b0 con 0x7fc6940b1100 2024-09-10T23:38:46.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.532+0000 7fc68be00640 1 -- 172.21.15.94:0/1516363781 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7fc68401b400 con 0x7fc6940b1100 2024-09-10T23:38:46.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.532+0000 7fc68be00640 1 --2- 172.21.15.94:0/1516363781 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc674072c10 0x7fc6740750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:46.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.532+0000 7fc68be00640 1 -- 172.21.15.94:0/1516363781 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4971+0+0 (secure 0 0 0) 0x7fc6840939c0 con 0x7fc6940b1100 2024-09-10T23:38:46.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.532+0000 7fc693400640 1 --2- 172.21.15.94:0/1516363781 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc674072c10 0x7fc6740750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:46.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.536+0000 7fc693400640 1 --2- 172.21.15.94:0/1516363781 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc674072c10 0x7fc6740750d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc6800af000 tx=0x7fc68001afd0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:46.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.536+0000 7fc68be00640 1 -- 172.21.15.94:0/1516363781 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc684020030 con 0x7fc6940b1100 2024-09-10T23:38:46.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.712+0000 7fc69aa00640 1 -- 172.21.15.94:0/1516363781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc69415d890 con 0x7fc6940b1100 2024-09-10T23:38:46.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.960+0000 7fc68be00640 1 -- 172.21.15.94:0/1516363781 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v121) ==== 97+0+0 (secure 0 0 0) 0x7fc68405e3f0 con 0x7fc6940b1100 2024-09-10T23:38:46.963 INFO:teuthology.orchestra.run.smithi094.stderr: removed all blocklist entries 2024-09-10T23:38:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.964+0000 7fc69aa00640 1 -- 172.21.15.94:0/1516363781 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc674072c10 msgr2=0x7fc6740750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.964+0000 7fc69aa00640 1 --2- 172.21.15.94:0/1516363781 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc674072c10 0x7fc6740750d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc6800af000 tx=0x7fc68001afd0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.964+0000 7fc69aa00640 1 -- 172.21.15.94:0/1516363781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6940b1100 msgr2=0x7fc6940ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.964+0000 7fc69aa00640 1 --2- 172.21.15.94:0/1516363781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6940b1100 0x7fc6940ca280 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fc68400c460 tx=0x7fc68400c920 comp rx=0 tx=0).stop 2024-09-10T23:38:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.964+0000 7fc69aa00640 1 -- 172.21.15.94:0/1516363781 shutdown_connections 2024-09-10T23:38:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.964+0000 7fc69aa00640 1 --2- 172.21.15.94:0/1516363781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6940ce1e0 0x7fc694151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.964+0000 7fc69aa00640 1 --2- 172.21.15.94:0/1516363781 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc674072c10 0x7fc6740750d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.964+0000 7fc69aa00640 1 --2- 172.21.15.94:0/1516363781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6940ce890 0x7fc694158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.964+0000 7fc69aa00640 1 --2- 172.21.15.94:0/1516363781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6940b1100 0x7fc6940ca280 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.964+0000 7fc69aa00640 1 -- 172.21.15.94:0/1516363781 >> 172.21.15.94:0/1516363781 conn(0x7fc6940d1c60 msgr2=0x7fc69415fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.964+0000 7fc69aa00640 1 -- 172.21.15.94:0/1516363781 shutdown_connections 2024-09-10T23:38:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:46.964+0000 7fc69aa00640 1 -- 172.21.15.94:0/1516363781 wait complete. 2024-09-10T23:38:46.989 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T23:38:47.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.128+0000 7ff9bdc00640 1 -- 172.21.15.94:0/2536877201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9b815bd30 msgr2=0x7ff9b815c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:47.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.128+0000 7ff9bdc00640 1 --2- 172.21.15.94:0/2536877201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9b815bd30 0x7ff9b815c110 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009900 tx=0x7ff9a801c8e0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.128+0000 7ff9bdc00640 1 -- 172.21.15.94:0/2536877201 shutdown_connections 2024-09-10T23:38:47.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.128+0000 7ff9bdc00640 1 --2- 172.21.15.94:0/2536877201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b80d2290 0x7ff9b80b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.128+0000 7ff9bdc00640 1 --2- 172.21.15.94:0/2536877201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9b815bd30 0x7ff9b815c110 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.128+0000 7ff9bdc00640 1 -- 172.21.15.94:0/2536877201 >> 172.21.15.94:0/2536877201 conn(0x7ff9b809aa70 msgr2=0x7ff9b809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:47.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.128+0000 7ff9bdc00640 1 -- 172.21.15.94:0/2536877201 shutdown_connections 2024-09-10T23:38:47.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.128+0000 7ff9bdc00640 1 -- 172.21.15.94:0/2536877201 wait complete. 2024-09-10T23:38:47.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9bdc00640 1 Processor -- start 2024-09-10T23:38:47.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9bdc00640 1 -- start start 2024-09-10T23:38:47.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9bdc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9b80ae980 0x7ff9b8159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:47.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9bdc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b80d2290 0x7ff9b815a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:47.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9bdc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b815bd30 0x7ff9b814e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:47.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9bdc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b81695a0 con 0x7ff9b80ae980 2024-09-10T23:38:47.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9bdc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b8169420 con 0x7ff9b815bd30 2024-09-10T23:38:47.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9bdc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b8169720 con 0x7ff9b80d2290 2024-09-10T23:38:47.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9b6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9b80ae980 0x7ff9b8159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:47.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9b6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9b80ae980 0x7ff9b8159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53264/0 (socket says 172.21.15.94:53264) 2024-09-10T23:38:47.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9b6a00640 1 -- 172.21.15.94:0/686396818 learned_addr learned my addr 172.21.15.94:0/686396818 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:47.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9b6000640 1 --2- 172.21.15.94:0/686396818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b80d2290 0x7ff9b815a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:47.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9b7400640 1 --2- 172.21.15.94:0/686396818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b815bd30 0x7ff9b814e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:47.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9b6a00640 1 -- 172.21.15.94:0/686396818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b80d2290 msgr2=0x7ff9b815a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:47.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9b6a00640 1 --2- 172.21.15.94:0/686396818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b80d2290 0x7ff9b815a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9b6a00640 1 -- 172.21.15.94:0/686396818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b815bd30 msgr2=0x7ff9b814e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:47.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9b6a00640 1 --2- 172.21.15.94:0/686396818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b815bd30 0x7ff9b814e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9b6a00640 1 -- 172.21.15.94:0/686396818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b80cd750 con 0x7ff9b80ae980 2024-09-10T23:38:47.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9b6a00640 1 --2- 172.21.15.94:0/686396818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9b80ae980 0x7ff9b8159e50 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009850 tx=0x7ff9a8005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:47.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9aea00640 1 -- 172.21.15.94:0/686396818 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a80b2040 con 0x7ff9b80ae980 2024-09-10T23:38:47.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9aea00640 1 -- 172.21.15.94:0/686396818 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9a8005320 con 0x7ff9b80ae980 2024-09-10T23:38:47.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9aea00640 1 -- 172.21.15.94:0/686396818 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a80046d0 con 0x7ff9b80ae980 2024-09-10T23:38:47.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9bdc00640 1 -- 172.21.15.94:0/686396818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b80cda30 con 0x7ff9b80ae980 2024-09-10T23:38:47.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.132+0000 7ff9bdc00640 1 -- 172.21.15.94:0/686396818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b814f2e0 con 0x7ff9b80ae980 2024-09-10T23:38:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.136+0000 7ff9bdc00640 1 -- 172.21.15.94:0/686396818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b80b1e70 con 0x7ff9b80ae980 2024-09-10T23:38:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.136+0000 7ff9aea00640 1 -- 172.21.15.94:0/686396818 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7ff9a8004870 con 0x7ff9b80ae980 2024-09-10T23:38:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.136+0000 7ff9aea00640 1 --2- 172.21.15.94:0/686396818 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff998072c10 0x7ff9980750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.136+0000 7ff9aea00640 1 -- 172.21.15.94:0/686396818 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4885+0+0 (secure 0 0 0) 0x7ff9a812da10 con 0x7ff9b80ae980 2024-09-10T23:38:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.136+0000 7ff9b6000640 1 --2- 172.21.15.94:0/686396818 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff998072c10 0x7ff9980750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:47.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.136+0000 7ff9b6000640 1 --2- 172.21.15.94:0/686396818 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff998072c10 0x7ff9980750d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80c88c0 tx=0x7ff9a4007e40 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:47.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.140+0000 7ff9aea00640 1 -- 172.21.15.94:0/686396818 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff9a80164d0 con 0x7ff9b80ae980 2024-09-10T23:38:47.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.364+0000 7ff9bdc00640 1 -- 172.21.15.94:0/686396818 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ff9b80cdc00 con 0x7ff9b80ae980 2024-09-10T23:38:47.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.364+0000 7ff9aea00640 1 -- 172.21.15.94:0/686396818 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v36) ==== 80+0+5304 (secure 0 0 0) 0x7ff9a80f8430 con 0x7ff9b80ae980 2024-09-10T23:38:47.368 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:47.368 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:38:47.369 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_dump": [ 2024-09-10T23:38:47.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.369 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.a", 2024-09-10T23:38:47.369 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/2m3LxAAr0ErpLJg+0+I2r74zQya4w==", 2024-09-10T23:38:47.369 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.369 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:38:47.369 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:38:47.369 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:38:47.369 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:47.369 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.369 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.b", 2024-09-10T23:38:47.369 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmc8pYABAAw90zu8D+9IwK15h4HIlGBA==", 2024-09-10T23:38:47.370 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.370 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:38:47.370 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:38:47.370 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:38:47.370 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:47.370 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.370 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.c", 2024-09-10T23:38:47.370 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmk98KNxAA44e9N4VHbw/94Lh1V9zugQ==", 2024-09-10T23:38:47.370 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.370 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:38:47.370 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:38:47.370 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:38:47.371 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:47.371 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.371 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.d", 2024-09-10T23:38:47.371 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmVAKzBxAAJehOw6aphTF/g3ucoQfw9A==", 2024-09-10T23:38:47.371 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.371 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:38:47.371 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:38:47.371 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:38:47.371 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:47.371 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.372 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.0", 2024-09-10T23:38:47.372 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDa1uBmSAD3KBAAJTF1OEC44Ml599aL7iEuDw==", 2024-09-10T23:38:47.372 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.372 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:38:47.372 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:38:47.372 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:47.372 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.372 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.1", 2024-09-10T23:38:47.372 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDe1uBma1L9ABAAfEjW1/Hb4fh4Kwv/ZUgFAQ==", 2024-09-10T23:38:47.372 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.372 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:38:47.372 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:38:47.373 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:47.373 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.373 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.2", 2024-09-10T23:38:47.373 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDh1uBmtLUQFhAAweeFHj6BRQVC9BX+VLiWEQ==", 2024-09-10T23:38:47.373 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.373 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:38:47.373 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:38:47.373 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:47.373 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.373 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.3", 2024-09-10T23:38:47.374 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDk1uBm1KFCKRAAev28iYcrEYOViX95uo6Kbw==", 2024-09-10T23:38:47.374 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.374 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:38:47.374 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:38:47.374 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:47.374 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.374 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.4", 2024-09-10T23:38:47.374 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDq1uBm/Hg9LhAAvB2D0rj4DeRPxTrGmXYIsA==", 2024-09-10T23:38:47.374 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.374 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:38:47.374 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:38:47.374 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:47.375 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.375 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.5", 2024-09-10T23:38:47.375 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDu1uBm1fAKCBAAFHMjnGFAItLdaACF/UjE/w==", 2024-09-10T23:38:47.375 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.375 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:38:47.375 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:38:47.375 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:47.375 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.375 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.6", 2024-09-10T23:38:47.375 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDx1uBmXgYfHhAA/6wwtM4mDRXsIpoTIHKJ0w==", 2024-09-10T23:38:47.376 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.376 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:38:47.376 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:38:47.376 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:47.376 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.376 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.7", 2024-09-10T23:38:47.376 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQD01uBmw6hQMxAAk7oGAg+iIplNK5KrokGwNA==", 2024-09-10T23:38:47.376 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.376 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:38:47.376 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:38:47.376 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:47.377 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.377 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.0", 2024-09-10T23:38:47.377 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g==", 2024-09-10T23:38:47.377 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.377 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:38:47.377 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow r", 2024-09-10T23:38:47.377 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T23:38:47.377 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.377 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.1", 2024-09-10T23:38:47.377 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmuSkhFRAA1R6WNXLImkGfhW9/XJzEAQ==", 2024-09-10T23:38:47.378 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.378 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:38:47.378 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:38:47.378 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:38:47.378 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:38:47.378 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.378 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.2", 2024-09-10T23:38:47.378 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmfvaqERAASpL0KtglFcAuRf4a/0z5JQ==", 2024-09-10T23:38:47.378 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.378 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:38:47.378 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:38:47.378 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:38:47.379 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:38:47.379 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.379 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.3", 2024-09-10T23:38:47.379 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBm8xUtGxAARc0ywfAhMxBNd57TzBoV3g==", 2024-09-10T23:38:47.379 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.379 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:38:47.379 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:38:47.379 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:38:47.379 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:38:47.379 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.380 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.admin", 2024-09-10T23:38:47.380 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmOpg1FBAA9NV6Jqt6doWMuAfcxztlOg==", 2024-09-10T23:38:47.380 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.380 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:38:47.380 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:38:47.380 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:38:47.380 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:47.380 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.380 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.x", 2024-09-10T23:38:47.380 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/EPVKxAA/Myv8Zjf7BejbzlkxyCC1w==", 2024-09-10T23:38:47.380 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.381 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:38:47.381 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:38:47.381 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:47.381 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:47.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.381 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.y", 2024-09-10T23:38:47.381 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmhHvtJxAACe+79nYhv2ODwtmNhA45FA==", 2024-09-10T23:38:47.381 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:38:47.381 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:38:47.381 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:38:47.381 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:38:47.381 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.381 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.382 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:38:47.382 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:38:47.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.368+0000 7ff9bdc00640 1 -- 172.21.15.94:0/686396818 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff998072c10 msgr2=0x7ff9980750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:47.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.368+0000 7ff9bdc00640 1 --2- 172.21.15.94:0/686396818 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff998072c10 0x7ff9980750d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80c88c0 tx=0x7ff9a4007e40 comp rx=0 tx=0).stop 2024-09-10T23:38:47.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.368+0000 7ff9bdc00640 1 -- 172.21.15.94:0/686396818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9b80ae980 msgr2=0x7ff9b8159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:47.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.368+0000 7ff9bdc00640 1 --2- 172.21.15.94:0/686396818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9b80ae980 0x7ff9b8159e50 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009850 tx=0x7ff9a8005760 comp rx=0 tx=0).stop 2024-09-10T23:38:47.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.368+0000 7ff9bdc00640 1 -- 172.21.15.94:0/686396818 shutdown_connections 2024-09-10T23:38:47.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.368+0000 7ff9bdc00640 1 --2- 172.21.15.94:0/686396818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b815bd30 0x7ff9b814e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.368+0000 7ff9bdc00640 1 --2- 172.21.15.94:0/686396818 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff998072c10 0x7ff9980750d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.368+0000 7ff9bdc00640 1 --2- 172.21.15.94:0/686396818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b80d2290 0x7ff9b815a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.368+0000 7ff9bdc00640 1 --2- 172.21.15.94:0/686396818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9b80ae980 0x7ff9b8159e50 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.368+0000 7ff9bdc00640 1 -- 172.21.15.94:0/686396818 >> 172.21.15.94:0/686396818 conn(0x7ff9b809aa70 msgr2=0x7ff9b8159400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:47.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.368+0000 7ff9bdc00640 1 -- 172.21.15.94:0/686396818 shutdown_connections 2024-09-10T23:38:47.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.368+0000 7ff9bdc00640 1 -- 172.21.15.94:0/686396818 wait complete. 2024-09-10T23:38:47.390 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T23:38:47.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.528+0000 7fd233c00640 1 -- 172.21.15.94:0/3119057589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd22c0d0010 msgr2=0x7fd22c15fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:47.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.528+0000 7fd233c00640 1 --2- 172.21.15.94:0/3119057589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd22c0d0010 0x7fd22c15fbe0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fd22c06d820 tx=0x7fd21801f0b0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.528+0000 7fd233c00640 1 -- 172.21.15.94:0/3119057589 shutdown_connections 2024-09-10T23:38:47.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.528+0000 7fd233c00640 1 --2- 172.21.15.94:0/3119057589 >> v2:172.21.15.181:3300/0 conn(0x7fd22c160120 0x7fd22c160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.528+0000 7fd233c00640 1 --2- 172.21.15.94:0/3119057589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd22c0d0010 0x7fd22c15fbe0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.528+0000 7fd233c00640 1 -- 172.21.15.94:0/3119057589 >> 172.21.15.94:0/3119057589 conn(0x7fd22c0d1bf0 msgr2=0x7fd22c0d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:47.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.528+0000 7fd233c00640 1 -- 172.21.15.94:0/3119057589 shutdown_connections 2024-09-10T23:38:47.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.528+0000 7fd233c00640 1 -- 172.21.15.94:0/3119057589 wait complete. 2024-09-10T23:38:47.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.528+0000 7fd233c00640 1 Processor -- start 2024-09-10T23:38:47.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.528+0000 7fd233c00640 1 -- start start 2024-09-10T23:38:47.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd233c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd22c0cfc30 0x7fd22c09ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:47.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd233c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd22c0d0010 0x7fd22c0ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:47.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd233c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd22c160120 0x7fd22c0ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:47.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd233c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c0b1c10 con 0x7fd22c0cfc30 2024-09-10T23:38:47.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd233c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c0b1a90 con 0x7fd22c160120 2024-09-10T23:38:47.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd233c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd22c0b1d90 con 0x7fd22c0d0010 2024-09-10T23:38:47.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd231200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd22c0cfc30 0x7fd22c09ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:47.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd231200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd22c0cfc30 0x7fd22c09ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53270/0 (socket says 172.21.15.94:53270) 2024-09-10T23:38:47.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd231200640 1 -- 172.21.15.94:0/3636374349 learned_addr learned my addr 172.21.15.94:0/3636374349 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:47.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd22be00640 1 --2- 172.21.15.94:0/3636374349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd22c0d0010 0x7fd22c0ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:47.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd231c00640 1 --2- 172.21.15.94:0/3636374349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd22c160120 0x7fd22c0ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:47.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd231200640 1 -- 172.21.15.94:0/3636374349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd22c0d0010 msgr2=0x7fd22c0ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:47.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd231200640 1 --2- 172.21.15.94:0/3636374349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd22c0d0010 0x7fd22c0ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd231200640 1 -- 172.21.15.94:0/3636374349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd22c160120 msgr2=0x7fd22c0ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:47.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd231200640 1 --2- 172.21.15.94:0/3636374349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd22c160120 0x7fd22c0ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd231200640 1 -- 172.21.15.94:0/3636374349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22c160600 con 0x7fd22c0cfc30 2024-09-10T23:38:47.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd231200640 1 --2- 172.21.15.94:0/3636374349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd22c0cfc30 0x7fd22c09ba80 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fd22000c500 tx=0x7fd22000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:47.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd228c00640 1 -- 172.21.15.94:0/3636374349 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220017070 con 0x7fd22c0cfc30 2024-09-10T23:38:47.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd228c00640 1 -- 172.21.15.94:0/3636374349 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2200053c0 con 0x7fd22c0cfc30 2024-09-10T23:38:47.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd228c00640 1 -- 172.21.15.94:0/3636374349 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220012890 con 0x7fd22c0cfc30 2024-09-10T23:38:47.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd233c00640 1 -- 172.21.15.94:0/3636374349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22c0cbec0 con 0x7fd22c0cfc30 2024-09-10T23:38:47.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd233c00640 1 -- 172.21.15.94:0/3636374349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd22c0c9120 con 0x7fd22c0cfc30 2024-09-10T23:38:47.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.532+0000 7fd233c00640 1 -- 172.21.15.94:0/3636374349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd22c15fbe0 con 0x7fd22c0cfc30 2024-09-10T23:38:47.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.536+0000 7fd228c00640 1 -- 172.21.15.94:0/3636374349 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7fd220003600 con 0x7fd22c0cfc30 2024-09-10T23:38:47.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.536+0000 7fd228c00640 1 --2- 172.21.15.94:0/3636374349 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd20c072c40 0x7fd20c075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:47.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.536+0000 7fd228c00640 1 -- 172.21.15.94:0/3636374349 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4885+0+0 (secure 0 0 0) 0x7fd2200930d0 con 0x7fd22c0cfc30 2024-09-10T23:38:47.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.536+0000 7fd22be00640 1 --2- 172.21.15.94:0/3636374349 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd20c072c40 0x7fd20c075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:47.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.536+0000 7fd22be00640 1 --2- 172.21.15.94:0/3636374349 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd20c072c40 0x7fd20c075100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd2180b2000 tx=0x7fd21801d8f0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:47.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.540+0000 7fd228c00640 1 -- 172.21.15.94:0/3636374349 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd22005da50 con 0x7fd22c0cfc30 2024-09-10T23:38:47.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.704+0000 7fd233c00640 1 -- 172.21.15.94:0/3636374349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd22c0704f0 con 0x7fd22c0cfc30 2024-09-10T23:38:47.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.704+0000 7fd228c00640 1 -- 172.21.15.94:0/3636374349 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v123) ==== 84+0+65 (secure 0 0 0) 0x7fd220062480 con 0x7fd22c0cfc30 2024-09-10T23:38:47.709 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:47.709 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:38:47.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:38:47.709 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-09-10T23:38:47.709 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-09-10T23:38:47.709 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:47.709 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:38:47.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.708+0000 7fd233c00640 1 -- 172.21.15.94:0/3636374349 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd20c072c40 msgr2=0x7fd20c075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:47.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.708+0000 7fd233c00640 1 --2- 172.21.15.94:0/3636374349 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd20c072c40 0x7fd20c075100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd2180b2000 tx=0x7fd21801d8f0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.708+0000 7fd233c00640 1 -- 172.21.15.94:0/3636374349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd22c0cfc30 msgr2=0x7fd22c09ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:47.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.708+0000 7fd233c00640 1 --2- 172.21.15.94:0/3636374349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd22c0cfc30 0x7fd22c09ba80 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fd22000c500 tx=0x7fd22000c9c0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.708+0000 7fd233c00640 1 -- 172.21.15.94:0/3636374349 shutdown_connections 2024-09-10T23:38:47.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.708+0000 7fd233c00640 1 --2- 172.21.15.94:0/3636374349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd22c160120 0x7fd22c0ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.708+0000 7fd233c00640 1 --2- 172.21.15.94:0/3636374349 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd20c072c40 0x7fd20c075100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.708+0000 7fd233c00640 1 --2- 172.21.15.94:0/3636374349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd22c0d0010 0x7fd22c0ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.708+0000 7fd233c00640 1 --2- 172.21.15.94:0/3636374349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd22c0cfc30 0x7fd22c09ba80 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.708+0000 7fd233c00640 1 -- 172.21.15.94:0/3636374349 >> 172.21.15.94:0/3636374349 conn(0x7fd22c0d1bf0 msgr2=0x7fd22c071f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:47.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.708+0000 7fd233c00640 1 -- 172.21.15.94:0/3636374349 shutdown_connections 2024-09-10T23:38:47.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.708+0000 7fd233c00640 1 -- 172.21.15.94:0/3636374349 wait complete. 2024-09-10T23:38:47.730 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T23:38:47.730 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T23:38:47.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.872+0000 7feaa9800640 1 -- 172.21.15.94:0/1045932711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa40cfda0 msgr2=0x7feaa40d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:47.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.872+0000 7feaa9800640 1 --2- 172.21.15.94:0/1045932711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa40cfda0 0x7feaa40d01e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7feaa40b16b0 tx=0x7fea9001c890 comp rx=0 tx=0).stop 2024-09-10T23:38:47.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.872+0000 7feaa9800640 1 -- 172.21.15.94:0/1045932711 shutdown_connections 2024-09-10T23:38:47.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.872+0000 7feaa9800640 1 --2- 172.21.15.94:0/1045932711 >> v2:172.21.15.181:3300/0 conn(0x7feaa40aee30 0x7feaa40af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.872+0000 7feaa9800640 1 --2- 172.21.15.94:0/1045932711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa40cfda0 0x7feaa40d01e0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.872+0000 7feaa9800640 1 -- 172.21.15.94:0/1045932711 >> 172.21.15.94:0/1045932711 conn(0x7feaa409aa70 msgr2=0x7feaa409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:47.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.872+0000 7feaa9800640 1 -- 172.21.15.94:0/1045932711 shutdown_connections 2024-09-10T23:38:47.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.872+0000 7feaa9800640 1 -- 172.21.15.94:0/1045932711 wait complete. 2024-09-10T23:38:47.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa9800640 1 Processor -- start 2024-09-10T23:38:47.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa9800640 1 -- start start 2024-09-10T23:38:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa9800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa40aee30 0x7feaa4153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa9800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa40cfda0 0x7feaa414f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa9800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa415ba30 0x7feaa414fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa9800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feaa4168a40 con 0x7feaa415ba30 2024-09-10T23:38:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa9800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7feaa41688c0 con 0x7feaa40aee30 2024-09-10T23:38:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa9800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7feaa4168bc0 con 0x7feaa40cfda0 2024-09-10T23:38:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa415ba30 0x7feaa414fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa415ba30 0x7feaa414fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53294/0 (socket says 172.21.15.94:53294) 2024-09-10T23:38:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa8e00640 1 -- 172.21.15.94:0/1589028272 learned_addr learned my addr 172.21.15.94:0/1589028272 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa8e00640 1 -- 172.21.15.94:0/1589028272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa40cfda0 msgr2=0x7feaa414f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa8e00640 1 --2- 172.21.15.94:0/1589028272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa40cfda0 0x7feaa414f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa8e00640 1 -- 172.21.15.94:0/1589028272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa40aee30 msgr2=0x7feaa4153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:47.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa3e00640 1 --2- 172.21.15.94:0/1589028272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa40aee30 0x7feaa4153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:47.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa8e00640 1 --2- 172.21.15.94:0/1589028272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa40aee30 0x7feaa4153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:47.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa8e00640 1 -- 172.21.15.94:0/1589028272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa40d1f20 con 0x7feaa415ba30 2024-09-10T23:38:47.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa3e00640 1 --2- 172.21.15.94:0/1589028272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa40aee30 0x7feaa4153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:38:47.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa8e00640 1 --2- 172.21.15.94:0/1589028272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa415ba30 0x7feaa414fb20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00d750 tx=0x7fea9c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:47.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7fea9be00640 1 -- 172.21.15.94:0/1589028272 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c017070 con 0x7feaa415ba30 2024-09-10T23:38:47.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7fea9be00640 1 -- 172.21.15.94:0/1589028272 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea9c009d20 con 0x7feaa415ba30 2024-09-10T23:38:47.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa9800640 1 -- 172.21.15.94:0/1589028272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa40c93c0 con 0x7feaa415ba30 2024-09-10T23:38:47.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7fea9be00640 1 -- 172.21.15.94:0/1589028272 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c00de60 con 0x7feaa415ba30 2024-09-10T23:38:47.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.876+0000 7feaa9800640 1 -- 172.21.15.94:0/1589028272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa4150e80 con 0x7feaa415ba30 2024-09-10T23:38:47.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.880+0000 7feaa9800640 1 -- 172.21.15.94:0/1589028272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa40af280 con 0x7feaa415ba30 2024-09-10T23:38:47.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.880+0000 7fea9be00640 1 -- 172.21.15.94:0/1589028272 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7fea9c005280 con 0x7feaa415ba30 2024-09-10T23:38:47.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.880+0000 7fea9be00640 1 --2- 172.21.15.94:0/1589028272 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea84072c10 0x7fea840750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:47.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.880+0000 7fea9be00640 1 -- 172.21.15.94:0/1589028272 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4885+0+0 (secure 0 0 0) 0x7fea9c093d00 con 0x7feaa415ba30 2024-09-10T23:38:47.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.880+0000 7feaa3e00640 1 --2- 172.21.15.94:0/1589028272 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea84072c10 0x7fea840750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:47.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.880+0000 7feaa3e00640 1 --2- 172.21.15.94:0/1589028272 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea84072c10 0x7fea840750d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea94007fd0 tx=0x7fea94009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:47.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:47.884+0000 7fea9be00640 1 -- 172.21.15.94:0/1589028272 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fea9c05ed10 con 0x7feaa415ba30 2024-09-10T23:38:48.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:48.056+0000 7feaa9800640 1 -- 172.21.15.94:0/1589028272 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7feaa409b130 con 0x7feaa415ba30 2024-09-10T23:38:48.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:48.968+0000 7fea9be00640 1 -- 172.21.15.94:0/1589028272 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v124) ==== 138+0+0 (secure 0 0 0) 0x7fea9c067430 con 0x7feaa415ba30 2024-09-10T23:38:48.970 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-09-10T23:38:48.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:48.968+0000 7feaa9800640 1 -- 172.21.15.94:0/1589028272 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea84072c10 msgr2=0x7fea840750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:48.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:48.968+0000 7feaa9800640 1 --2- 172.21.15.94:0/1589028272 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea84072c10 0x7fea840750d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea94007fd0 tx=0x7fea94009040 comp rx=0 tx=0).stop 2024-09-10T23:38:48.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:48.968+0000 7feaa9800640 1 -- 172.21.15.94:0/1589028272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa415ba30 msgr2=0x7feaa414fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:48.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:48.968+0000 7feaa9800640 1 --2- 172.21.15.94:0/1589028272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa415ba30 0x7feaa414fb20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00d750 tx=0x7fea9c00dc10 comp rx=0 tx=0).stop 2024-09-10T23:38:48.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:48.968+0000 7feaa9800640 1 -- 172.21.15.94:0/1589028272 shutdown_connections 2024-09-10T23:38:48.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:48.968+0000 7feaa9800640 1 --2- 172.21.15.94:0/1589028272 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea84072c10 0x7fea840750d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:48.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:48.968+0000 7feaa9800640 1 --2- 172.21.15.94:0/1589028272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaa415ba30 0x7feaa414fb20 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:48.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:48.968+0000 7feaa9800640 1 --2- 172.21.15.94:0/1589028272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa40cfda0 0x7feaa414f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:48.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:48.968+0000 7feaa9800640 1 --2- 172.21.15.94:0/1589028272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa40aee30 0x7feaa4153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:48.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:48.968+0000 7feaa9800640 1 -- 172.21.15.94:0/1589028272 >> 172.21.15.94:0/1589028272 conn(0x7feaa409aa70 msgr2=0x7feaa4159090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:48.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:48.972+0000 7feaa9800640 1 -- 172.21.15.94:0/1589028272 shutdown_connections 2024-09-10T23:38:48.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:48.972+0000 7feaa9800640 1 -- 172.21.15.94:0/1589028272 wait complete. 2024-09-10T23:38:49.003 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T23:38:49.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 -- 172.21.15.94:0/118680191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a9c0c5a70 msgr2=0x7f1a9c0cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:49.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 --2- 172.21.15.94:0/118680191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a9c0c5a70 0x7f1a9c0cd9f0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f1a94009c80 tx=0x7f1a9401cd30 comp rx=0 tx=0).stop 2024-09-10T23:38:49.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 -- 172.21.15.94:0/118680191 shutdown_connections 2024-09-10T23:38:49.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 --2- 172.21.15.94:0/118680191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a9c0c5a70 0x7f1a9c0cd9f0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:49.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 --2- 172.21.15.94:0/118680191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a9c0d0710 0x7f1a9c0c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:49.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 --2- 172.21.15.94:0/118680191 >> v2:172.21.15.181:3300/0 conn(0x7f1a9c1455d0 0x7f1a9c0c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:49.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 -- 172.21.15.94:0/118680191 >> 172.21.15.94:0/118680191 conn(0x7f1a9c0cfaf0 msgr2=0x7f1a9c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:49.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 -- 172.21.15.94:0/118680191 shutdown_connections 2024-09-10T23:38:49.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 -- 172.21.15.94:0/118680191 wait complete. 2024-09-10T23:38:49.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 Processor -- start 2024-09-10T23:38:49.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 -- start start 2024-09-10T23:38:49.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a9c0c5a70 0x7f1a9c158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:49.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a9c0d0710 0x7f1a9c0c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:49.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a9c1455d0 0x7f1a9c0c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:49.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c15fe80 con 0x7f1a9c0c5a70 2024-09-10T23:38:49.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c15fd00 con 0x7f1a9c1455d0 2024-09-10T23:38:49.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1aa2200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1a9c160000 con 0x7f1a9c0d0710 2024-09-10T23:38:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1a9b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a9c0c5a70 0x7f1a9c158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.152+0000 7f1a9b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a9c0c5a70 0x7f1a9c158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53314/0 (socket says 172.21.15.94:53314) 2024-09-10T23:38:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1a9b400640 1 -- 172.21.15.94:0/864726951 learned_addr learned my addr 172.21.15.94:0/864726951 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1a92a00640 1 --2- 172.21.15.94:0/864726951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a9c0d0710 0x7f1a9c0c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1a9be00640 1 --2- 172.21.15.94:0/864726951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a9c1455d0 0x7f1a9c0c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1a9b400640 1 -- 172.21.15.94:0/864726951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a9c0d0710 msgr2=0x7f1a9c0c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1a9b400640 1 --2- 172.21.15.94:0/864726951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a9c0d0710 0x7f1a9c0c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1a9b400640 1 -- 172.21.15.94:0/864726951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a9c1455d0 msgr2=0x7f1a9c0c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1a9b400640 1 --2- 172.21.15.94:0/864726951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a9c1455d0 0x7f1a9c0c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1a9b400640 1 -- 172.21.15.94:0/864726951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9c0b2a40 con 0x7f1a9c0c5a70 2024-09-10T23:38:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1a9be00640 1 --2- 172.21.15.94:0/864726951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a9c1455d0 0x7f1a9c0c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:38:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1a9b400640 1 --2- 172.21.15.94:0/864726951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a9c0c5a70 0x7f1a9c158e70 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00e7a0 tx=0x7f1a8c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1a93e00640 1 -- 172.21.15.94:0/864726951 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c00ccc0 con 0x7f1a9c0c5a70 2024-09-10T23:38:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1a93e00640 1 -- 172.21.15.94:0/864726951 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a8c010430 con 0x7f1a9c0c5a70 2024-09-10T23:38:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1aa2200640 1 -- 172.21.15.94:0/864726951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9c142c70 con 0x7f1a9c0c5a70 2024-09-10T23:38:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1a93e00640 1 -- 172.21.15.94:0/864726951 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c010860 con 0x7f1a9c0c5a70 2024-09-10T23:38:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1aa2200640 1 -- 172.21.15.94:0/864726951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a9c143230 con 0x7f1a9c0c5a70 2024-09-10T23:38:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1aa2200640 1 -- 172.21.15.94:0/864726951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a9c1605d0 con 0x7f1a9c0c5a70 2024-09-10T23:38:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.156+0000 7f1a93e00640 1 -- 172.21.15.94:0/864726951 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f1a8c029080 con 0x7f1a9c0c5a70 2024-09-10T23:38:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.160+0000 7f1a93e00640 1 --2- 172.21.15.94:0/864726951 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1a78072c40 0x7f1a78075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.160+0000 7f1a93e00640 1 -- 172.21.15.94:0/864726951 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5255+0+0 (secure 0 0 0) 0x7f1a8c094bf0 con 0x7f1a9c0c5a70 2024-09-10T23:38:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.160+0000 7f1a92a00640 1 --2- 172.21.15.94:0/864726951 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1a78072c40 0x7f1a78075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:49.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.160+0000 7f1a92a00640 1 --2- 172.21.15.94:0/864726951 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1a78072c40 0x7f1a78075100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0cf670 tx=0x7f1a840091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:49.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.160+0000 7f1a93e00640 1 -- 172.21.15.94:0/864726951 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1a8c05fb10 con 0x7f1a9c0c5a70 2024-09-10T23:38:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.328+0000 7f1aa2200640 1 -- 172.21.15.94:0/864726951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1a9c0cf6e0 con 0x7f1a9c0c5a70 2024-09-10T23:38:49.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.972+0000 7f1a93e00640 1 -- 172.21.15.94:0/864726951 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v125) ==== 170+0+0 (secure 0 0 0) 0x7f1a8c05f410 con 0x7f1a9c0c5a70 2024-09-10T23:38:49.974 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-09-10T23:38:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.972+0000 7f1aa2200640 1 -- 172.21.15.94:0/864726951 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1a78072c40 msgr2=0x7f1a78075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.972+0000 7f1aa2200640 1 --2- 172.21.15.94:0/864726951 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1a78072c40 0x7f1a78075100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0cf670 tx=0x7f1a840091e0 comp rx=0 tx=0).stop 2024-09-10T23:38:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.972+0000 7f1aa2200640 1 -- 172.21.15.94:0/864726951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a9c0c5a70 msgr2=0x7f1a9c158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.972+0000 7f1aa2200640 1 --2- 172.21.15.94:0/864726951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a9c0c5a70 0x7f1a9c158e70 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00e7a0 tx=0x7f1a8c00ec60 comp rx=0 tx=0).stop 2024-09-10T23:38:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.972+0000 7f1aa2200640 1 -- 172.21.15.94:0/864726951 shutdown_connections 2024-09-10T23:38:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.972+0000 7f1aa2200640 1 --2- 172.21.15.94:0/864726951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a9c1455d0 0x7f1a9c0c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:49.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.972+0000 7f1aa2200640 1 --2- 172.21.15.94:0/864726951 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1a78072c40 0x7f1a78075100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.972+0000 7f1aa2200640 1 --2- 172.21.15.94:0/864726951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a9c0d0710 0x7f1a9c0c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.972+0000 7f1aa2200640 1 --2- 172.21.15.94:0/864726951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a9c0c5a70 0x7f1a9c158e70 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.972+0000 7f1aa2200640 1 -- 172.21.15.94:0/864726951 >> 172.21.15.94:0/864726951 conn(0x7f1a9c0cfaf0 msgr2=0x7f1a9c15b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.976+0000 7f1aa2200640 1 -- 172.21.15.94:0/864726951 shutdown_connections 2024-09-10T23:38:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:49.976+0000 7f1aa2200640 1 -- 172.21.15.94:0/864726951 wait complete. 2024-09-10T23:38:50.016 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T23:38:50.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.160+0000 7f5bdb400640 1 -- 172.21.15.94:0/251930898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd40c64f0 msgr2=0x7f5bd40cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:50.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.160+0000 7f5bdb400640 1 --2- 172.21.15.94:0/251930898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd40c64f0 0x7f5bd40cf280 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4072510 tx=0x7f5bbc01dbf0 comp rx=0 tx=0).stop 2024-09-10T23:38:50.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.160+0000 7f5bdb400640 1 -- 172.21.15.94:0/251930898 shutdown_connections 2024-09-10T23:38:50.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.160+0000 7f5bdb400640 1 --2- 172.21.15.94:0/251930898 >> v2:172.21.15.181:3300/0 conn(0x7f5bd406e7c0 0x7f5bd406ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:50.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.160+0000 7f5bdb400640 1 --2- 172.21.15.94:0/251930898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd40c64f0 0x7f5bd40cf280 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:50.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.160+0000 7f5bdb400640 1 -- 172.21.15.94:0/251930898 >> 172.21.15.94:0/251930898 conn(0x7f5bd40cfaf0 msgr2=0x7f5bd40cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:50.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.160+0000 7f5bdb400640 1 -- 172.21.15.94:0/251930898 shutdown_connections 2024-09-10T23:38:50.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.160+0000 7f5bdb400640 1 -- 172.21.15.94:0/251930898 wait complete. 2024-09-10T23:38:50.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.160+0000 7f5bdb400640 1 Processor -- start 2024-09-10T23:38:50.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.160+0000 7f5bdb400640 1 -- start start 2024-09-10T23:38:50.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bdb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd406e7c0 0x7f5bd40c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:50.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bdb400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bd40c64f0 0x7f5bd40c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:50.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bdb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bd40ce280 0x7f5bd4073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bdb400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd415b5c0 con 0x7f5bd406e7c0 2024-09-10T23:38:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bdb400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd415b440 con 0x7f5bd40ce280 2024-09-10T23:38:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bdb400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd415b740 con 0x7f5bd40c64f0 2024-09-10T23:38:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bd8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd406e7c0 0x7f5bd40c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bd8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd406e7c0 0x7f5bd40c9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53336/0 (socket says 172.21.15.94:53336) 2024-09-10T23:38:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bd8a00640 1 -- 172.21.15.94:0/3909584437 learned_addr learned my addr 172.21.15.94:0/3909584437 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bd9400640 1 --2- 172.21.15.94:0/3909584437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bd40ce280 0x7f5bd4073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bcbe00640 1 --2- 172.21.15.94:0/3909584437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bd40c64f0 0x7f5bd40c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bd8a00640 1 -- 172.21.15.94:0/3909584437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bd40c64f0 msgr2=0x7f5bd40c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bd8a00640 1 --2- 172.21.15.94:0/3909584437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bd40c64f0 0x7f5bd40c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bd8a00640 1 -- 172.21.15.94:0/3909584437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bd40ce280 msgr2=0x7f5bd4073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:50.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bd8a00640 1 --2- 172.21.15.94:0/3909584437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bd40ce280 0x7f5bd4073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bd8a00640 1 -- 172.21.15.94:0/3909584437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd40c7050 con 0x7f5bd406e7c0 2024-09-10T23:38:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bcbe00640 1 --2- 172.21.15.94:0/3909584437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bd40c64f0 0x7f5bd40c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bd8a00640 1 --2- 172.21.15.94:0/3909584437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd406e7c0 0x7f5bd40c9010 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400c3f0 tx=0x7f5bc400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:50.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bd1600640 1 -- 172.21.15.94:0/3909584437 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4017070 con 0x7f5bd406e7c0 2024-09-10T23:38:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bd1600640 1 -- 172.21.15.94:0/3909584437 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bc40053c0 con 0x7f5bd406e7c0 2024-09-10T23:38:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bdb400640 1 -- 172.21.15.94:0/3909584437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd40c7330 con 0x7f5bd406e7c0 2024-09-10T23:38:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bd1600640 1 -- 172.21.15.94:0/3909584437 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4003860 con 0x7f5bd406e7c0 2024-09-10T23:38:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bdb400640 1 -- 172.21.15.94:0/3909584437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd40c78f0 con 0x7f5bd406e7c0 2024-09-10T23:38:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.164+0000 7f5bdb400640 1 -- 172.21.15.94:0/3909584437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd406ebf0 con 0x7f5bd406e7c0 2024-09-10T23:38:50.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.168+0000 7f5bd1600640 1 -- 172.21.15.94:0/3909584437 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f5bc4003a00 con 0x7f5bd406e7c0 2024-09-10T23:38:50.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.168+0000 7f5bd1600640 1 --2- 172.21.15.94:0/3909584437 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bb0072c40 0x7f5bb0075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:50.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.168+0000 7f5bd1600640 1 -- 172.21.15.94:0/3909584437 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5637+0+0 (secure 0 0 0) 0x7f5bc40937b0 con 0x7f5bd406e7c0 2024-09-10T23:38:50.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.168+0000 7f5bcbe00640 1 --2- 172.21.15.94:0/3909584437 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bb0072c40 0x7f5bb0075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:50.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.168+0000 7f5bcbe00640 1 --2- 172.21.15.94:0/3909584437 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bb0072c40 0x7f5bb0075100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0b0000 tx=0x7f5bbc01c430 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:50.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.172+0000 7f5bd1600640 1 -- 172.21.15.94:0/3909584437 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5bc4021020 con 0x7f5bd406e7c0 2024-09-10T23:38:50.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.372+0000 7f5bdb400640 1 -- 172.21.15.94:0/3909584437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5bd40d0850 con 0x7f5bd406e7c0 2024-09-10T23:38:51.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:50.996+0000 7f5bd1600640 1 -- 172.21.15.94:0/3909584437 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v84) ==== 174+0+0 (secure 0 0 0) 0x7f5bc405e550 con 0x7f5bd406e7c0 2024-09-10T23:38:51.002 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 20 and data pool 21 2024-09-10T23:38:51.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.004+0000 7f5bdb400640 1 -- 172.21.15.94:0/3909584437 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bb0072c40 msgr2=0x7f5bb0075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:51.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.004+0000 7f5bdb400640 1 --2- 172.21.15.94:0/3909584437 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bb0072c40 0x7f5bb0075100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0b0000 tx=0x7f5bbc01c430 comp rx=0 tx=0).stop 2024-09-10T23:38:51.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.004+0000 7f5bdb400640 1 -- 172.21.15.94:0/3909584437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd406e7c0 msgr2=0x7f5bd40c9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:51.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.004+0000 7f5bdb400640 1 --2- 172.21.15.94:0/3909584437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd406e7c0 0x7f5bd40c9010 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400c3f0 tx=0x7f5bc400c8b0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.008+0000 7f5bdb400640 1 -- 172.21.15.94:0/3909584437 shutdown_connections 2024-09-10T23:38:51.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.008+0000 7f5bdb400640 1 --2- 172.21.15.94:0/3909584437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bd40ce280 0x7f5bd4073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.008+0000 7f5bdb400640 1 --2- 172.21.15.94:0/3909584437 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bb0072c40 0x7f5bb0075100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.008+0000 7f5bdb400640 1 --2- 172.21.15.94:0/3909584437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bd40c64f0 0x7f5bd40c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.008+0000 7f5bdb400640 1 --2- 172.21.15.94:0/3909584437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bd406e7c0 0x7f5bd40c9010 secure :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400c3f0 tx=0x7f5bc400c8b0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.008+0000 7f5bdb400640 1 -- 172.21.15.94:0/3909584437 >> 172.21.15.94:0/3909584437 conn(0x7f5bd40cfaf0 msgr2=0x7f5bd40afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:51.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.012+0000 7f5bdb400640 1 -- 172.21.15.94:0/3909584437 shutdown_connections 2024-09-10T23:38:51.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.012+0000 7f5bdb400640 1 -- 172.21.15.94:0/3909584437 wait complete. 2024-09-10T23:38:51.045 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:38:51.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd8e00640 1 -- 172.21.15.94:0/2050202936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fd4162560 msgr2=0x7f6fd406e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:51.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd8e00640 1 --2- 172.21.15.94:0/2050202936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fd4162560 0x7f6fd406e290 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009900 tx=0x7f6fc001c8e0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd8e00640 1 -- 172.21.15.94:0/2050202936 shutdown_connections 2024-09-10T23:38:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd8e00640 1 --2- 172.21.15.94:0/2050202936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fd406e7d0 0x7f6fd40cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd8e00640 1 --2- 172.21.15.94:0/2050202936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fd4162560 0x7f6fd406e290 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd8e00640 1 -- 172.21.15.94:0/2050202936 >> 172.21.15.94:0/2050202936 conn(0x7f6fd409aa70 msgr2=0x7f6fd409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd8e00640 1 -- 172.21.15.94:0/2050202936 shutdown_connections 2024-09-10T23:38:51.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd8e00640 1 -- 172.21.15.94:0/2050202936 wait complete. 2024-09-10T23:38:51.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd8e00640 1 Processor -- start 2024-09-10T23:38:51.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd8e00640 1 -- start start 2024-09-10T23:38:51.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fd406e7d0 0x7f6fd414c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:51.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd8e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fd415dd50 0x7f6fd40b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:51.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd8e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fd4162560 0x7f6fd40b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:51.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd8e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd416ae70 con 0x7f6fd406e7d0 2024-09-10T23:38:51.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd8e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd416acf0 con 0x7f6fd415dd50 2024-09-10T23:38:51.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd8e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd416aff0 con 0x7f6fd4162560 2024-09-10T23:38:51.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fd406e7d0 0x7f6fd414c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:51.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fd406e7d0 0x7f6fd414c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53356/0 (socket says 172.21.15.94:53356) 2024-09-10T23:38:51.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.184+0000 7f6fd1e00640 1 -- 172.21.15.94:0/3243285383 learned_addr learned my addr 172.21.15.94:0/3243285383 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:51.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.188+0000 7f6fd1400640 1 --2- 172.21.15.94:0/3243285383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fd415dd50 0x7f6fd40b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:51.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.188+0000 7f6fd2800640 1 --2- 172.21.15.94:0/3243285383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fd4162560 0x7f6fd40b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:51.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.188+0000 7f6fd1e00640 1 -- 172.21.15.94:0/3243285383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fd4162560 msgr2=0x7f6fd40b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:51.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.188+0000 7f6fd1e00640 1 --2- 172.21.15.94:0/3243285383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fd4162560 0x7f6fd40b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.188+0000 7f6fd1e00640 1 -- 172.21.15.94:0/3243285383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fd415dd50 msgr2=0x7f6fd40b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:51.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.188+0000 7f6fd1e00640 1 --2- 172.21.15.94:0/3243285383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fd415dd50 0x7f6fd40b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.188+0000 7f6fd1e00640 1 -- 172.21.15.94:0/3243285383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd4152360 con 0x7f6fd406e7d0 2024-09-10T23:38:51.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.188+0000 7f6fd2800640 1 --2- 172.21.15.94:0/3243285383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fd4162560 0x7f6fd40b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:51.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.188+0000 7f6fd1e00640 1 --2- 172.21.15.94:0/3243285383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fd406e7d0 0x7f6fd414c560 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400c500 tx=0x7f6fc400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:51.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.188+0000 7f6fca000640 1 -- 172.21.15.94:0/3243285383 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4017070 con 0x7f6fd406e7d0 2024-09-10T23:38:51.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.188+0000 7f6fca000640 1 -- 172.21.15.94:0/3243285383 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fc40053c0 con 0x7f6fd406e7d0 2024-09-10T23:38:51.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.188+0000 7f6fd8e00640 1 -- 172.21.15.94:0/3243285383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd414caa0 con 0x7f6fd406e7d0 2024-09-10T23:38:51.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.188+0000 7f6fca000640 1 -- 172.21.15.94:0/3243285383 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4012890 con 0x7f6fd406e7d0 2024-09-10T23:38:51.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.188+0000 7f6fd8e00640 1 -- 172.21.15.94:0/3243285383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd40afb50 con 0x7f6fd406e7d0 2024-09-10T23:38:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.188+0000 7f6fd8e00640 1 -- 172.21.15.94:0/3243285383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd40afe90 con 0x7f6fd406e7d0 2024-09-10T23:38:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.188+0000 7f6fca000640 1 -- 172.21.15.94:0/3243285383 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f6fc4003600 con 0x7f6fd406e7d0 2024-09-10T23:38:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.192+0000 7f6fca000640 1 --2- 172.21.15.94:0/3243285383 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6fb4072c10 0x7f6fb40750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:51.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.192+0000 7f6fca000640 1 -- 172.21.15.94:0/3243285383 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5737+0+0 (secure 0 0 0) 0x7f6fc40934d0 con 0x7f6fd406e7d0 2024-09-10T23:38:51.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.192+0000 7f6fd1400640 1 --2- 172.21.15.94:0/3243285383 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6fb4072c10 0x7f6fb40750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:51.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.192+0000 7f6fd1400640 1 --2- 172.21.15.94:0/3243285383 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6fb4072c10 0x7f6fb40750d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00af000 tx=0x7f6fc001b120 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:51.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.196+0000 7f6fca000640 1 -- 172.21.15.94:0/3243285383 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6fd40afe90 con 0x7f6fd406e7d0 2024-09-10T23:38:51.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.360+0000 7f6fd8e00640 1 -- 172.21.15.94:0/3243285383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fd40d29c0 con 0x7f6fd406e7d0 2024-09-10T23:38:51.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.364+0000 7f6fca000640 1 -- 172.21.15.94:0/3243285383 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16784 (secure 0 0 0) 0x7f6fd4152360 con 0x7f6fd406e7d0 2024-09-10T23:38:51.366 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:51.366 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":126,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:38:50.375721+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:38:48.069795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T23:38:49.345087+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:38:51.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.364+0000 7f6fd8e00640 1 -- 172.21.15.94:0/3243285383 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6fb4072c10 msgr2=0x7f6fb40750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:51.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.364+0000 7f6fd8e00640 1 --2- 172.21.15.94:0/3243285383 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6fb4072c10 0x7f6fb40750d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00af000 tx=0x7f6fc001b120 comp rx=0 tx=0).stop 2024-09-10T23:38:51.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.364+0000 7f6fd8e00640 1 -- 172.21.15.94:0/3243285383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fd406e7d0 msgr2=0x7f6fd414c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:51.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.364+0000 7f6fd8e00640 1 --2- 172.21.15.94:0/3243285383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fd406e7d0 0x7f6fd414c560 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400c500 tx=0x7f6fc400c9c0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.364+0000 7f6fd8e00640 1 -- 172.21.15.94:0/3243285383 shutdown_connections 2024-09-10T23:38:51.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.364+0000 7f6fd8e00640 1 --2- 172.21.15.94:0/3243285383 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fd415dd50 0x7f6fd40b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.364+0000 7f6fd8e00640 1 --2- 172.21.15.94:0/3243285383 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6fb4072c10 0x7f6fb40750d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.364+0000 7f6fd8e00640 1 --2- 172.21.15.94:0/3243285383 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fd4162560 0x7f6fd40b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.364+0000 7f6fd8e00640 1 --2- 172.21.15.94:0/3243285383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6fd406e7d0 0x7f6fd414c560 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.364+0000 7f6fd8e00640 1 -- 172.21.15.94:0/3243285383 >> 172.21.15.94:0/3243285383 conn(0x7f6fd409aa70 msgr2=0x7f6fd40cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:51.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.364+0000 7f6fd8e00640 1 -- 172.21.15.94:0/3243285383 shutdown_connections 2024-09-10T23:38:51.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.364+0000 7f6fd8e00640 1 -- 172.21.15.94:0/3243285383 wait complete. 2024-09-10T23:38:51.387 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:38:51.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.524+0000 7f00b2000640 1 -- 172.21.15.94:0/3036177233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac15ba50 msgr2=0x7f00ac0b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:51.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.524+0000 7f00b2000640 1 --2- 172.21.15.94:0/3036177233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac15ba50 0x7f00ac0b4700 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0cd8f0 tx=0x7f009c01c800 comp rx=0 tx=0).stop 2024-09-10T23:38:51.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.524+0000 7f00b2000640 1 -- 172.21.15.94:0/3036177233 shutdown_connections 2024-09-10T23:38:51.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.524+0000 7f00b2000640 1 --2- 172.21.15.94:0/3036177233 >> v2:172.21.15.181:3300/0 conn(0x7f00ac0d03c0 0x7f00ac0cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.524+0000 7f00b2000640 1 --2- 172.21.15.94:0/3036177233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac15ba50 0x7f00ac0b4700 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.524+0000 7f00b2000640 1 -- 172.21.15.94:0/3036177233 >> 172.21.15.94:0/3036177233 conn(0x7f00ac06cdf0 msgr2=0x7f00ac06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:51.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.524+0000 7f00b2000640 1 -- 172.21.15.94:0/3036177233 shutdown_connections 2024-09-10T23:38:51.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.524+0000 7f00b2000640 1 -- 172.21.15.94:0/3036177233 wait complete. 2024-09-10T23:38:51.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00b2000640 1 Processor -- start 2024-09-10T23:38:51.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00b2000640 1 -- start start 2024-09-10T23:38:51.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00b2000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00ac0b4c40 0x7f00ac158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:51.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00b2000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac0d03c0 0x7f00ac150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:51.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00b2000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac15ba50 0x7f00ac150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:51.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00b2000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac1689d0 con 0x7f00ac0d03c0 2024-09-10T23:38:51.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00b2000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f00ac168850 con 0x7f00ac0b4c40 2024-09-10T23:38:51.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00b2000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f00ac168b50 con 0x7f00ac15ba50 2024-09-10T23:38:51.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00a2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac0d03c0 0x7f00ac150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:51.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00a2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac0d03c0 0x7f00ac150750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53374/0 (socket says 172.21.15.94:53374) 2024-09-10T23:38:51.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00a2a00640 1 -- 172.21.15.94:0/199613366 learned_addr learned my addr 172.21.15.94:0/199613366 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:51.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00ab400640 1 --2- 172.21.15.94:0/199613366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00ac0b4c40 0x7f00ac158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:51.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00abe00640 1 --2- 172.21.15.94:0/199613366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac15ba50 0x7f00ac150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:51.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00a2a00640 1 -- 172.21.15.94:0/199613366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac15ba50 msgr2=0x7f00ac150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:51.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00a2a00640 1 --2- 172.21.15.94:0/199613366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac15ba50 0x7f00ac150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00a2a00640 1 -- 172.21.15.94:0/199613366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00ac0b4c40 msgr2=0x7f00ac158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:51.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00a2a00640 1 --2- 172.21.15.94:0/199613366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00ac0b4c40 0x7f00ac158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00a2a00640 1 -- 172.21.15.94:0/199613366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ac1f7be0 con 0x7f00ac0d03c0 2024-09-10T23:38:51.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00a2a00640 1 --2- 172.21.15.94:0/199613366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac0d03c0 0x7f00ac150750 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f009400c4d0 tx=0x7f009400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:51.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00a3e00640 1 -- 172.21.15.94:0/199613366 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00940052d0 con 0x7f00ac0d03c0 2024-09-10T23:38:51.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00a3e00640 1 -- 172.21.15.94:0/199613366 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0094005470 con 0x7f00ac0d03c0 2024-09-10T23:38:51.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00a3e00640 1 -- 172.21.15.94:0/199613366 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094012890 con 0x7f00ac0d03c0 2024-09-10T23:38:51.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00b2000640 1 -- 172.21.15.94:0/199613366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ac1f7ec0 con 0x7f00ac0d03c0 2024-09-10T23:38:51.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.528+0000 7f00b2000640 1 -- 172.21.15.94:0/199613366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ac1f8480 con 0x7f00ac0d03c0 2024-09-10T23:38:51.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.532+0000 7f00b2000640 1 -- 172.21.15.94:0/199613366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ac0cddb0 con 0x7f00ac0d03c0 2024-09-10T23:38:51.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.532+0000 7f00a3e00640 1 -- 172.21.15.94:0/199613366 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f0094003600 con 0x7f00ac0d03c0 2024-09-10T23:38:51.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.532+0000 7f00a3e00640 1 --2- 172.21.15.94:0/199613366 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0088072c10 0x7f00880750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:51.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.532+0000 7f00a3e00640 1 -- 172.21.15.94:0/199613366 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5737+0+0 (secure 0 0 0) 0x7f00940939e0 con 0x7f00ac0d03c0 2024-09-10T23:38:51.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.532+0000 7f00ab400640 1 --2- 172.21.15.94:0/199613366 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0088072c10 0x7f00880750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:51.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.532+0000 7f00ab400640 1 --2- 172.21.15.94:0/199613366 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0088072c10 0x7f00880750d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f009c0af000 tx=0x7f009c01b040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:51.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.536+0000 7f00a3e00640 1 -- 172.21.15.94:0/199613366 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f009405e6a0 con 0x7f00ac0d03c0 2024-09-10T23:38:51.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.700+0000 7f00b2000640 1 -- 172.21.15.94:0/199613366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00ac156960 con 0x7f00ac0d03c0 2024-09-10T23:38:51.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.704+0000 7f00a3e00640 1 -- 172.21.15.94:0/199613366 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16784 (secure 0 0 0) 0x7f009405dfa0 con 0x7f00ac0d03c0 2024-09-10T23:38:51.706 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:51.706 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":126,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:38:50.375721+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:38:48.069795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T23:38:49.345087+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:38:51.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.704+0000 7f00b2000640 1 -- 172.21.15.94:0/199613366 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0088072c10 msgr2=0x7f00880750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:51.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.704+0000 7f00b2000640 1 --2- 172.21.15.94:0/199613366 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0088072c10 0x7f00880750d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f009c0af000 tx=0x7f009c01b040 comp rx=0 tx=0).stop 2024-09-10T23:38:51.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.704+0000 7f00b2000640 1 -- 172.21.15.94:0/199613366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac0d03c0 msgr2=0x7f00ac150750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:51.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.704+0000 7f00b2000640 1 --2- 172.21.15.94:0/199613366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac0d03c0 0x7f00ac150750 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f009400c4d0 tx=0x7f009400c990 comp rx=0 tx=0).stop 2024-09-10T23:38:51.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.704+0000 7f00b2000640 1 -- 172.21.15.94:0/199613366 shutdown_connections 2024-09-10T23:38:51.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.704+0000 7f00b2000640 1 --2- 172.21.15.94:0/199613366 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0088072c10 0x7f00880750d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.704+0000 7f00b2000640 1 --2- 172.21.15.94:0/199613366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00ac15ba50 0x7f00ac150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.704+0000 7f00b2000640 1 --2- 172.21.15.94:0/199613366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00ac0d03c0 0x7f00ac150750 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.704+0000 7f00b2000640 1 --2- 172.21.15.94:0/199613366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00ac0b4c40 0x7f00ac158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.704+0000 7f00b2000640 1 -- 172.21.15.94:0/199613366 >> 172.21.15.94:0/199613366 conn(0x7f00ac06cdf0 msgr2=0x7f00ac157160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:51.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.704+0000 7f00b2000640 1 -- 172.21.15.94:0/199613366 shutdown_connections 2024-09-10T23:38:51.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.704+0000 7f00b2000640 1 -- 172.21.15.94:0/199613366 wait complete. 2024-09-10T23:38:51.728 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T23:38:51.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.868+0000 7f26a9600640 1 -- 172.21.15.94:0/3370899696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a40ce1e0 msgr2=0x7f26a40cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:51.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.868+0000 7f26a9600640 1 --2- 172.21.15.94:0/3370899696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a40ce1e0 0x7f26a40cec70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2690009900 tx=0x7f269001c820 comp rx=0 tx=0).stop 2024-09-10T23:38:51.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.868+0000 7f26a9600640 1 -- 172.21.15.94:0/3370899696 shutdown_connections 2024-09-10T23:38:51.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.868+0000 7f26a9600640 1 --2- 172.21.15.94:0/3370899696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a40ce1e0 0x7f26a40cec70 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.868+0000 7f26a9600640 1 --2- 172.21.15.94:0/3370899696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26a40ce890 0x7f26a40cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.868+0000 7f26a9600640 1 -- 172.21.15.94:0/3370899696 >> 172.21.15.94:0/3370899696 conn(0x7f26a40d1c60 msgr2=0x7f26a40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:51.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.868+0000 7f26a9600640 1 -- 172.21.15.94:0/3370899696 shutdown_connections 2024-09-10T23:38:51.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.868+0000 7f26a9600640 1 -- 172.21.15.94:0/3370899696 wait complete. 2024-09-10T23:38:51.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a9600640 1 Processor -- start 2024-09-10T23:38:51.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a9600640 1 -- start start 2024-09-10T23:38:51.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a9600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a40b1100 0x7f26a414f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:51.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a9600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a40ce1e0 0x7f26a40b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:51.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a9600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26a40ce890 0x7f26a414ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:51.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a9600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f26a40d0960 con 0x7f26a40b1100 2024-09-10T23:38:51.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a9600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f26a40d07e0 con 0x7f26a40ce1e0 2024-09-10T23:38:51.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a9600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f26a40d0ae0 con 0x7f26a40ce890 2024-09-10T23:38:51.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a40b1100 0x7f26a414f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:51.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a40b1100 0x7f26a414f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53398/0 (socket says 172.21.15.94:53398) 2024-09-10T23:38:51.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a2a00640 1 -- 172.21.15.94:0/1323716644 learned_addr learned my addr 172.21.15.94:0/1323716644 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:51.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a3400640 1 --2- 172.21.15.94:0/1323716644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26a40ce890 0x7f26a414ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:51.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a2000640 1 --2- 172.21.15.94:0/1323716644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a40ce1e0 0x7f26a40b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:51.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a2a00640 1 -- 172.21.15.94:0/1323716644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26a40ce890 msgr2=0x7f26a414ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:51.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a2a00640 1 --2- 172.21.15.94:0/1323716644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26a40ce890 0x7f26a414ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a2a00640 1 -- 172.21.15.94:0/1323716644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a40ce1e0 msgr2=0x7f26a40b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:51.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a2a00640 1 --2- 172.21.15.94:0/1323716644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a40ce1e0 0x7f26a40b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:51.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a2a00640 1 -- 172.21.15.94:0/1323716644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a415ed00 con 0x7f26a40b1100 2024-09-10T23:38:51.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a2a00640 1 --2- 172.21.15.94:0/1323716644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a40b1100 0x7f26a414f210 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f269400ccb0 tx=0x7f269400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:51.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f269aa00640 1 -- 172.21.15.94:0/1323716644 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269400f450 con 0x7f26a40b1100 2024-09-10T23:38:51.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f269aa00640 1 -- 172.21.15.94:0/1323716644 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f269400f6e0 con 0x7f26a40b1100 2024-09-10T23:38:51.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f269aa00640 1 -- 172.21.15.94:0/1323716644 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269400fb30 con 0x7f26a40b1100 2024-09-10T23:38:51.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a9600640 1 -- 172.21.15.94:0/1323716644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a415efe0 con 0x7f26a40b1100 2024-09-10T23:38:51.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a9600640 1 -- 172.21.15.94:0/1323716644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a415c1c0 con 0x7f26a40b1100 2024-09-10T23:38:51.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.872+0000 7f26a9600640 1 -- 172.21.15.94:0/1323716644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a40cec70 con 0x7f26a40b1100 2024-09-10T23:38:51.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.876+0000 7f269aa00640 1 -- 172.21.15.94:0/1323716644 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f269400fcd0 con 0x7f26a40b1100 2024-09-10T23:38:51.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.876+0000 7f269aa00640 1 --2- 172.21.15.94:0/1323716644 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2684072c10 0x7f26840750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:51.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.876+0000 7f269aa00640 1 -- 172.21.15.94:0/1323716644 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5737+0+0 (secure 0 0 0) 0x7f2694094950 con 0x7f26a40b1100 2024-09-10T23:38:51.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.876+0000 7f26a2000640 1 --2- 172.21.15.94:0/1323716644 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2684072c10 0x7f26840750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:51.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.876+0000 7f26a2000640 1 --2- 172.21.15.94:0/1323716644 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2684072c10 0x7f26840750d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f26900af000 tx=0x7f269001b060 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:51.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:51.880+0000 7f269aa00640 1 -- 172.21.15.94:0/1323716644 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2694014030 con 0x7f26a40b1100 2024-09-10T23:38:52.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:52.084+0000 7f26a9600640 1 -- 172.21.15.94:0/1323716644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f26a40b1830 con 0x7f26a40b1100 2024-09-10T23:38:52.968 INFO:tasks.ceph.mds.d.smithi181.stderr:2024-09-10T23:38:52.968+0000 7f494f600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:38:53.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.008+0000 7f269aa00640 1 -- 172.21.15.94:0/1323716644 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v87) ==== 118+0+0 (secure 0 0 0) 0x7f269405ef30 con 0x7f26a40b1100 2024-09-10T23:38:53.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.012+0000 7f26a9600640 1 -- 172.21.15.94:0/1323716644 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2684072c10 msgr2=0x7f26840750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:53.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.012+0000 7f26a9600640 1 --2- 172.21.15.94:0/1323716644 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2684072c10 0x7f26840750d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f26900af000 tx=0x7f269001b060 comp rx=0 tx=0).stop 2024-09-10T23:38:53.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.012+0000 7f26a9600640 1 -- 172.21.15.94:0/1323716644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a40b1100 msgr2=0x7f26a414f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:53.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.012+0000 7f26a9600640 1 --2- 172.21.15.94:0/1323716644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a40b1100 0x7f26a414f210 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f269400ccb0 tx=0x7f269400a620 comp rx=0 tx=0).stop 2024-09-10T23:38:53.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.012+0000 7f26a9600640 1 -- 172.21.15.94:0/1323716644 shutdown_connections 2024-09-10T23:38:53.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.012+0000 7f26a9600640 1 --2- 172.21.15.94:0/1323716644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26a40ce1e0 0x7f26a40b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:53.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.012+0000 7f26a9600640 1 --2- 172.21.15.94:0/1323716644 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2684072c10 0x7f26840750d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:53.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.012+0000 7f26a9600640 1 --2- 172.21.15.94:0/1323716644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26a40ce890 0x7f26a414ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:53.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.012+0000 7f26a9600640 1 --2- 172.21.15.94:0/1323716644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26a40b1100 0x7f26a414f210 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:53.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.012+0000 7f26a9600640 1 -- 172.21.15.94:0/1323716644 >> 172.21.15.94:0/1323716644 conn(0x7f26a40d1c60 msgr2=0x7f26a414fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:53.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.012+0000 7f26a9600640 1 -- 172.21.15.94:0/1323716644 shutdown_connections 2024-09-10T23:38:53.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.012+0000 7f26a9600640 1 -- 172.21.15.94:0/1323716644 wait complete. 2024-09-10T23:38:53.038 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T23:38:53.038 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T23:38:53.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.184+0000 7fb29f600640 1 -- 172.21.15.94:0/924740863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb27c003a90 msgr2=0x7fb27c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:53.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.184+0000 7fb29f600640 1 --2- 172.21.15.94:0/924740863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb27c003a90 0x7fb27c003eb0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fb29815f870 tx=0x7fb28801cec0 comp rx=0 tx=0).stop 2024-09-10T23:38:53.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.184+0000 7fb29d600640 1 -- 172.21.15.94:0/924740863 reap_dead start 2024-09-10T23:38:53.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.184+0000 7fb29f600640 1 -- 172.21.15.94:0/924740863 shutdown_connections 2024-09-10T23:38:53.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.184+0000 7fb29f600640 1 -- 172.21.15.94:0/924740863 >> 172.21.15.94:0/924740863 conn(0x7fb2980cfaf0 msgr2=0x7fb2980cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:53.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.184+0000 7fb29f600640 1 -- 172.21.15.94:0/924740863 shutdown_connections 2024-09-10T23:38:53.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.184+0000 7fb29f600640 1 -- 172.21.15.94:0/924740863 wait complete. 2024-09-10T23:38:53.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.184+0000 7fb29f600640 1 Processor -- start 2024-09-10T23:38:53.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.184+0000 7fb29f600640 1 -- start start 2024-09-10T23:38:53.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29f600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb27c003a90 0x7fb2981fb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:53.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29f600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb29815a8a0 0x7fb29814a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29f600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb298152ab0 0x7fb298152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29f600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb298166a30 con 0x7fb27c003a90 2024-09-10T23:38:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29f600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb2981668b0 con 0x7fb298152ab0 2024-09-10T23:38:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29f600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb298166bb0 con 0x7fb29815a8a0 2024-09-10T23:38:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb27c003a90 0x7fb2981fb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb27c003a90 0x7fb2981fb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53414/0 (socket says 172.21.15.94:53414) 2024-09-10T23:38:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29cc00640 1 -- 172.21.15.94:0/382198106 learned_addr learned my addr 172.21.15.94:0/382198106 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29d600640 1 --2- 172.21.15.94:0/382198106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb298152ab0 0x7fb298152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb28fe00640 1 --2- 172.21.15.94:0/382198106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb29815a8a0 0x7fb29814a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29cc00640 1 -- 172.21.15.94:0/382198106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb29815a8a0 msgr2=0x7fb29814a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29cc00640 1 --2- 172.21.15.94:0/382198106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb29815a8a0 0x7fb29814a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29cc00640 1 -- 172.21.15.94:0/382198106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb298152ab0 msgr2=0x7fb298152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:53.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29cc00640 1 --2- 172.21.15.94:0/382198106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb298152ab0 0x7fb298152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:53.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29cc00640 1 -- 172.21.15.94:0/382198106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb298072640 con 0x7fb27c003a90 2024-09-10T23:38:53.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29cc00640 1 --2- 172.21.15.94:0/382198106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb27c003a90 0x7fb2981fb9b0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fb28801d3a0 tx=0x7fb288006be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:53.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb295600640 1 -- 172.21.15.94:0/382198106 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288001460 con 0x7fb27c003a90 2024-09-10T23:38:53.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb295600640 1 -- 172.21.15.94:0/382198106 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb288001600 con 0x7fb27c003a90 2024-09-10T23:38:53.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29f600640 1 -- 172.21.15.94:0/382198106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb298072920 con 0x7fb27c003a90 2024-09-10T23:38:53.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb295600640 1 -- 172.21.15.94:0/382198106 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288006330 con 0x7fb27c003a90 2024-09-10T23:38:53.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29f600640 1 -- 172.21.15.94:0/382198106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb298072ee0 con 0x7fb27c003a90 2024-09-10T23:38:53.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.188+0000 7fb29f600640 1 -- 172.21.15.94:0/382198106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2980cadc0 con 0x7fb27c003a90 2024-09-10T23:38:53.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.192+0000 7fb295600640 1 -- 172.21.15.94:0/382198106 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7fb288003620 con 0x7fb27c003a90 2024-09-10T23:38:53.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.192+0000 7fb295600640 1 --2- 172.21.15.94:0/382198106 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb274072c40 0x7fb274075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:53.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.192+0000 7fb295600640 1 -- 172.21.15.94:0/382198106 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7fb28812d6c0 con 0x7fb27c003a90 2024-09-10T23:38:53.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.192+0000 7fb28fe00640 1 --2- 172.21.15.94:0/382198106 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb274072c40 0x7fb274075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:53.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.192+0000 7fb28fe00640 1 --2- 172.21.15.94:0/382198106 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb274072c40 0x7fb274075100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb298153760 tx=0x7fb280006b80 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:53.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.196+0000 7fb295600640 1 -- 172.21.15.94:0/382198106 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb2880bf060 con 0x7fb27c003a90 2024-09-10T23:38:53.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:53.396+0000 7fb29f600640 1 -- 172.21.15.94:0/382198106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb2980d0850 con 0x7fb27c003a90 2024-09-10T23:38:54.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.012+0000 7fb295600640 1 -- 172.21.15.94:0/382198106 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v88) ==== 122+0+0 (secure 0 0 0) 0x7fb288016c70 con 0x7fb27c003a90 2024-09-10T23:38:54.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.012+0000 7fb29f600640 1 -- 172.21.15.94:0/382198106 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb274072c40 msgr2=0x7fb274075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:54.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.012+0000 7fb29f600640 1 --2- 172.21.15.94:0/382198106 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb274072c40 0x7fb274075100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb298153760 tx=0x7fb280006b80 comp rx=0 tx=0).stop 2024-09-10T23:38:54.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.012+0000 7fb29f600640 1 -- 172.21.15.94:0/382198106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb27c003a90 msgr2=0x7fb2981fb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:54.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.012+0000 7fb29f600640 1 --2- 172.21.15.94:0/382198106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb27c003a90 0x7fb2981fb9b0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fb28801d3a0 tx=0x7fb288006be0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.012+0000 7fb29f600640 1 -- 172.21.15.94:0/382198106 shutdown_connections 2024-09-10T23:38:54.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.012+0000 7fb29f600640 1 --2- 172.21.15.94:0/382198106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb298152ab0 0x7fb298152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.012+0000 7fb29f600640 1 --2- 172.21.15.94:0/382198106 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb274072c40 0x7fb274075100 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.012+0000 7fb29f600640 1 --2- 172.21.15.94:0/382198106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb29815a8a0 0x7fb29814a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.012+0000 7fb29f600640 1 --2- 172.21.15.94:0/382198106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb27c003a90 0x7fb2981fb9b0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.012+0000 7fb29f600640 1 -- 172.21.15.94:0/382198106 >> 172.21.15.94:0/382198106 conn(0x7fb2980cfaf0 msgr2=0x7fb298157820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:54.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.012+0000 7fb29f600640 1 -- 172.21.15.94:0/382198106 shutdown_connections 2024-09-10T23:38:54.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.012+0000 7fb29f600640 1 -- 172.21.15.94:0/382198106 wait complete. 2024-09-10T23:38:54.038 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T23:38:54.038 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:38:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.176+0000 7f8989600640 1 -- 172.21.15.94:0/4213475501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f898415cbc0 msgr2=0x7f89840cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.176+0000 7f8989600640 1 --2- 172.21.15.94:0/4213475501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f898415cbc0 0x7f89840cde20 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f89840d08b0 tx=0x7f896c01c890 comp rx=0 tx=0).stop 2024-09-10T23:38:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.176+0000 7f8989600640 1 -- 172.21.15.94:0/4213475501 shutdown_connections 2024-09-10T23:38:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.176+0000 7f8989600640 1 --2- 172.21.15.94:0/4213475501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f898415cbc0 0x7f89840cde20 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.176+0000 7f8989600640 1 --2- 172.21.15.94:0/4213475501 >> v2:172.21.15.181:3300/0 conn(0x7f8984152c10 0x7f89840b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.176+0000 7f8989600640 1 -- 172.21.15.94:0/4213475501 >> 172.21.15.94:0/4213475501 conn(0x7f898406cdf0 msgr2=0x7f898406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.176+0000 7f8989600640 1 -- 172.21.15.94:0/4213475501 shutdown_connections 2024-09-10T23:38:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.176+0000 7f8989600640 1 -- 172.21.15.94:0/4213475501 wait complete. 2024-09-10T23:38:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f8989600640 1 Processor -- start 2024-09-10T23:38:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f8989600640 1 -- start start 2024-09-10T23:38:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f8989600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89840cfd50 0x7f898414b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f8989600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8984152c10 0x7f89841518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f8989600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f898415cbc0 0x7f8984162fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f8989600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f89841693b0 con 0x7f8984152c10 2024-09-10T23:38:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f8989600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8984169230 con 0x7f89840cfd50 2024-09-10T23:38:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f8989600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8984169530 con 0x7f898415cbc0 2024-09-10T23:38:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f897a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8984152c10 0x7f89841518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f897a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8984152c10 0x7f89841518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55650/0 (socket says 172.21.15.94:55650) 2024-09-10T23:38:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f897a000640 1 -- 172.21.15.94:0/2490875333 learned_addr learned my addr 172.21.15.94:0/2490875333 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f8982a00640 1 --2- 172.21.15.94:0/2490875333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89840cfd50 0x7f898414b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f8983400640 1 --2- 172.21.15.94:0/2490875333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f898415cbc0 0x7f8984162fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f897a000640 1 -- 172.21.15.94:0/2490875333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f898415cbc0 msgr2=0x7f8984162fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f897a000640 1 --2- 172.21.15.94:0/2490875333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f898415cbc0 0x7f8984162fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f897a000640 1 -- 172.21.15.94:0/2490875333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89840cfd50 msgr2=0x7f898414b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f897a000640 1 --2- 172.21.15.94:0/2490875333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89840cfd50 0x7f898414b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f897a000640 1 -- 172.21.15.94:0/2490875333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89840cc1c0 con 0x7f8984152c10 2024-09-10T23:38:54.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f897a000640 1 --2- 172.21.15.94:0/2490875333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8984152c10 0x7f89841518f0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f896c0b0040 tx=0x7f896c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:54.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f897b400640 1 -- 172.21.15.94:0/2490875333 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c0b2040 con 0x7f8984152c10 2024-09-10T23:38:54.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f897b400640 1 -- 172.21.15.94:0/2490875333 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f896c005030 con 0x7f8984152c10 2024-09-10T23:38:54.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f897b400640 1 -- 172.21.15.94:0/2490875333 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c005460 con 0x7f8984152c10 2024-09-10T23:38:54.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f8989600640 1 -- 172.21.15.94:0/2490875333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89840cc4a0 con 0x7f8984152c10 2024-09-10T23:38:54.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.180+0000 7f8989600640 1 -- 172.21.15.94:0/2490875333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f898415da70 con 0x7f8984152c10 2024-09-10T23:38:54.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.184+0000 7f8989600640 1 -- 172.21.15.94:0/2490875333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f898415d860 con 0x7f8984152c10 2024-09-10T23:38:54.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.184+0000 7f897b400640 1 -- 172.21.15.94:0/2490875333 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f896c0bf060 con 0x7f8984152c10 2024-09-10T23:38:54.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.184+0000 7f897b400640 1 --2- 172.21.15.94:0/2490875333 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8960072c10 0x7f89600750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:54.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.184+0000 7f897b400640 1 -- 172.21.15.94:0/2490875333 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f896c12d060 con 0x7f8984152c10 2024-09-10T23:38:54.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.184+0000 7f8982a00640 1 --2- 172.21.15.94:0/2490875333 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8960072c10 0x7f89600750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.184+0000 7f8982a00640 1 --2- 172.21.15.94:0/2490875333 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8960072c10 0x7f89600750d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8974004630 tx=0x7f897400b390 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:54.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.188+0000 7f897b400640 1 -- 172.21.15.94:0/2490875333 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f896c0164b0 con 0x7f8984152c10 2024-09-10T23:38:54.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.384+0000 7f8989600640 1 -- 172.21.15.94:0/2490875333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89840b46c0 con 0x7f8984152c10 2024-09-10T23:38:54.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.384+0000 7f897b400640 1 -- 172.21.15.94:0/2490875333 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5234 (secure 0 0 0) 0x7f896c0f75c0 con 0x7f8984152c10 2024-09-10T23:38:54.389 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:54.389 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":88,"btime":"2024-09-10T23:38:54:013296+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3851553379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3851553379},{"type":"v1","addr":"172.21.15.181:6835","nonce":3851553379}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/90080224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":90080224},{"type":"v1","addr":"172.21.15.94:6833","nonce":90080224}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":7929,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/334167733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":334167733},{"type":"v1","addr":"172.21.15.94:6835","nonce":334167733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:38:50.375690+0000","modified":"2024-09-10T23:38:53.400898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196562":{"gid":196562,"name":"d","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6801/2109170417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2109170417},{"type":"v1","addr":"172.21.15.181:6801","nonce":2109170417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196562,"qdb_cluster":[196562]},"id":10}]} 2024-09-10T23:38:54.389 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 88 2024-09-10T23:38:54.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.388+0000 7f8989600640 1 -- 172.21.15.94:0/2490875333 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8960072c10 msgr2=0x7f89600750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:54.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.388+0000 7f8989600640 1 --2- 172.21.15.94:0/2490875333 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8960072c10 0x7f89600750d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8974004630 tx=0x7f897400b390 comp rx=0 tx=0).stop 2024-09-10T23:38:54.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.388+0000 7f8989600640 1 -- 172.21.15.94:0/2490875333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8984152c10 msgr2=0x7f89841518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:54.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.388+0000 7f8989600640 1 --2- 172.21.15.94:0/2490875333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8984152c10 0x7f89841518f0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f896c0b0040 tx=0x7f896c004040 comp rx=0 tx=0).stop 2024-09-10T23:38:54.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.388+0000 7f8989600640 1 -- 172.21.15.94:0/2490875333 shutdown_connections 2024-09-10T23:38:54.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.388+0000 7f8989600640 1 --2- 172.21.15.94:0/2490875333 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8960072c10 0x7f89600750d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.388+0000 7f8989600640 1 --2- 172.21.15.94:0/2490875333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f898415cbc0 0x7f8984162fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.388+0000 7f8989600640 1 --2- 172.21.15.94:0/2490875333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8984152c10 0x7f89841518f0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.388+0000 7f8989600640 1 --2- 172.21.15.94:0/2490875333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89840cfd50 0x7f898414b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.388+0000 7f8989600640 1 -- 172.21.15.94:0/2490875333 >> 172.21.15.94:0/2490875333 conn(0x7f898406cdf0 msgr2=0x7f898415d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:54.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.388+0000 7f8989600640 1 -- 172.21.15.94:0/2490875333 shutdown_connections 2024-09-10T23:38:54.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.388+0000 7f8989600640 1 -- 172.21.15.94:0/2490875333 wait complete. 2024-09-10T23:38:54.410 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:38:54.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 -- 172.21.15.94:0/2960871484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39800af310 msgr2=0x7f39800af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 --2- 172.21.15.94:0/2960871484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39800af310 0x7f39800af7a0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f3980073820 tx=0x7f397801c7a0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 -- 172.21.15.94:0/2960871484 shutdown_connections 2024-09-10T23:38:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 --2- 172.21.15.94:0/2960871484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39800af310 0x7f39800af7a0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 --2- 172.21.15.94:0/2960871484 >> v2:172.21.15.181:3300/0 conn(0x7f398015f5c0 0x7f39800cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 -- 172.21.15.94:0/2960871484 >> 172.21.15.94:0/2960871484 conn(0x7f39800d1c60 msgr2=0x7f39800d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 -- 172.21.15.94:0/2960871484 shutdown_connections 2024-09-10T23:38:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 -- 172.21.15.94:0/2960871484 wait complete. 2024-09-10T23:38:54.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 Processor -- start 2024-09-10T23:38:54.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 -- start start 2024-09-10T23:38:54.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39800af310 0x7f3980148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:54.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39800b2bb0 0x7f398014dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f398015f5c0 0x7f398014e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f39801690b0 con 0x7f39800af310 2024-09-10T23:38:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3980168f30 con 0x7f39800b2bb0 2024-09-10T23:38:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3980169230 con 0x7f398015f5c0 2024-09-10T23:38:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3985600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39800af310 0x7f3980148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3985600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39800af310 0x7f3980148c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55668/0 (socket says 172.21.15.94:55668) 2024-09-10T23:38:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3985600640 1 -- 172.21.15.94:0/3390108537 learned_addr learned my addr 172.21.15.94:0/3390108537 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3986000640 1 --2- 172.21.15.94:0/3390108537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f398015f5c0 0x7f398014e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3984c00640 1 --2- 172.21.15.94:0/3390108537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39800b2bb0 0x7f398014dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3985600640 1 -- 172.21.15.94:0/3390108537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f398015f5c0 msgr2=0x7f398014e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3985600640 1 --2- 172.21.15.94:0/3390108537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f398015f5c0 0x7f398014e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3985600640 1 -- 172.21.15.94:0/3390108537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39800b2bb0 msgr2=0x7f398014dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3985600640 1 --2- 172.21.15.94:0/3390108537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39800b2bb0 0x7f398014dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3985600640 1 -- 172.21.15.94:0/3390108537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39800c7d70 con 0x7f39800af310 2024-09-10T23:38:54.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3985600640 1 --2- 172.21.15.94:0/3390108537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39800af310 0x7f3980148c10 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f397400ebf0 tx=0x7f397400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:54.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f397d600640 1 -- 172.21.15.94:0/3390108537 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974010630 con 0x7f39800af310 2024-09-10T23:38:54.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f397d600640 1 -- 172.21.15.94:0/3390108537 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39740108c0 con 0x7f39800af310 2024-09-10T23:38:54.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f3988000640 1 -- 172.21.15.94:0/3390108537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398014ef80 con 0x7f39800af310 2024-09-10T23:38:54.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.552+0000 7f397d600640 1 -- 172.21.15.94:0/3390108537 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974010cf0 con 0x7f39800af310 2024-09-10T23:38:54.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.556+0000 7f3988000640 1 -- 172.21.15.94:0/3390108537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f398014d3b0 con 0x7f39800af310 2024-09-10T23:38:54.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.556+0000 7f3988000640 1 -- 172.21.15.94:0/3390108537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39800af7a0 con 0x7f39800af310 2024-09-10T23:38:54.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.556+0000 7f397d600640 1 -- 172.21.15.94:0/3390108537 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f3974018420 con 0x7f39800af310 2024-09-10T23:38:54.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.556+0000 7f397d600640 1 --2- 172.21.15.94:0/3390108537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3960072c10 0x7f39600750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:54.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.556+0000 7f397d600640 1 -- 172.21.15.94:0/3390108537 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f3974094c90 con 0x7f39800af310 2024-09-10T23:38:54.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.556+0000 7f3984c00640 1 --2- 172.21.15.94:0/3390108537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3960072c10 0x7f39600750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:54.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.556+0000 7f3984c00640 1 --2- 172.21.15.94:0/3390108537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3960072c10 0x7f39600750d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f39800c87b0 tx=0x7f396c009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:54.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.560+0000 7f397d600640 1 -- 172.21.15.94:0/3390108537 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f39800c7d70 con 0x7f39800af310 2024-09-10T23:38:54.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.724+0000 7f3988000640 1 -- 172.21.15.94:0/3390108537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39800749c0 con 0x7f39800af310 2024-09-10T23:38:54.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.728+0000 7f397d600640 1 -- 172.21.15.94:0/3390108537 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+16771 (secure 0 0 0) 0x7f397405f8c0 con 0x7f39800af310 2024-09-10T23:38:54.730 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:54.730 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":127,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:38:51.988498+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:38:48.069795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T23:38:49.345087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:38:54.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.728+0000 7f3988000640 1 -- 172.21.15.94:0/3390108537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3960072c10 msgr2=0x7f39600750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:54.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.728+0000 7f3988000640 1 --2- 172.21.15.94:0/3390108537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3960072c10 0x7f39600750d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f39800c87b0 tx=0x7f396c009040 comp rx=0 tx=0).stop 2024-09-10T23:38:54.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.728+0000 7f3988000640 1 -- 172.21.15.94:0/3390108537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39800af310 msgr2=0x7f3980148c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:54.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.728+0000 7f3988000640 1 --2- 172.21.15.94:0/3390108537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39800af310 0x7f3980148c10 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f397400ebf0 tx=0x7f397400c6a0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.728+0000 7f3988000640 1 -- 172.21.15.94:0/3390108537 shutdown_connections 2024-09-10T23:38:54.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.728+0000 7f3988000640 1 --2- 172.21.15.94:0/3390108537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39800b2bb0 0x7f398014dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.728+0000 7f3988000640 1 --2- 172.21.15.94:0/3390108537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3960072c10 0x7f39600750d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.728+0000 7f3988000640 1 --2- 172.21.15.94:0/3390108537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f398015f5c0 0x7f398014e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.728+0000 7f3988000640 1 --2- 172.21.15.94:0/3390108537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39800af310 0x7f3980148c10 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.728+0000 7f3988000640 1 -- 172.21.15.94:0/3390108537 >> 172.21.15.94:0/3390108537 conn(0x7f39800d1c60 msgr2=0x7f398015dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:54.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.728+0000 7f3988000640 1 -- 172.21.15.94:0/3390108537 shutdown_connections 2024-09-10T23:38:54.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.728+0000 7f3988000640 1 -- 172.21.15.94:0/3390108537 wait complete. 2024-09-10T23:38:54.752 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T23:38:54.752 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:38:54.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.892+0000 7fc6da000640 1 -- 172.21.15.94:0/2552057384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6b80053d0 msgr2=0x7fc6b800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:54.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.892+0000 7fc6da000640 1 --2- 172.21.15.94:0/2552057384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6b80053d0 0x7fc6b800b9d0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009eb0 tx=0x7fc6cc01d5f0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.892+0000 7fc6da000640 1 -- 172.21.15.94:0/2552057384 shutdown_connections 2024-09-10T23:38:54.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.892+0000 7fc6da000640 1 --2- 172.21.15.94:0/2552057384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6b80053d0 0x7fc6b800b9d0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.892+0000 7fc6da000640 1 -- 172.21.15.94:0/2552057384 >> 172.21.15.94:0/2552057384 conn(0x7fc6d40d1c60 msgr2=0x7fc6d40d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:54.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.892+0000 7fc6da000640 1 -- 172.21.15.94:0/2552057384 shutdown_connections 2024-09-10T23:38:54.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.892+0000 7fc6da000640 1 -- 172.21.15.94:0/2552057384 wait complete. 2024-09-10T23:38:54.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6da000640 1 Processor -- start 2024-09-10T23:38:54.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6da000640 1 -- start start 2024-09-10T23:38:54.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6da000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6b80053d0 0x7fc6d4148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:54.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6da000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6d40c5790 0x7fc6d414b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:54.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6da000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6d4070780 0x7fc6d4070c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:54.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6da000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d40cffb0 con 0x7fc6b80053d0 2024-09-10T23:38:54.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6da000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d40cfe30 con 0x7fc6d4070780 2024-09-10T23:38:54.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6da000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc6d40d0130 con 0x7fc6d40c5790 2024-09-10T23:38:54.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6d3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6b80053d0 0x7fc6d4148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:54.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6d3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6b80053d0 0x7fc6d4148510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55692/0 (socket says 172.21.15.94:55692) 2024-09-10T23:38:54.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6d3400640 1 -- 172.21.15.94:0/1354455886 learned_addr learned my addr 172.21.15.94:0/1354455886 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:54.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6d3400640 1 -- 172.21.15.94:0/1354455886 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6d40c5790 msgr2=0x7fc6d414b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:54.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6d3400640 1 --2- 172.21.15.94:0/1354455886 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6d40c5790 0x7fc6d414b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6d3400640 1 -- 172.21.15.94:0/1354455886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6d4070780 msgr2=0x7fc6d4070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:54.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6d3e00640 1 --2- 172.21.15.94:0/1354455886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6d4070780 0x7fc6d4070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:54.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6d3400640 1 --2- 172.21.15.94:0/1354455886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6d4070780 0x7fc6d4070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:54.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6d3400640 1 -- 172.21.15.94:0/1354455886 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d4148310 con 0x7fc6b80053d0 2024-09-10T23:38:54.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6d3e00640 1 --2- 172.21.15.94:0/1354455886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6d4070780 0x7fc6d4070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:38:54.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6d3400640 1 --2- 172.21.15.94:0/1354455886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6b80053d0 0x7fc6d4148510 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400e5c0 tx=0x7fc6c400ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:54.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6cb400640 1 -- 172.21.15.94:0/1354455886 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4015020 con 0x7fc6b80053d0 2024-09-10T23:38:54.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6cb400640 1 -- 172.21.15.94:0/1354455886 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6c40109a0 con 0x7fc6b80053d0 2024-09-10T23:38:54.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6da000640 1 -- 172.21.15.94:0/1354455886 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d40c85e0 con 0x7fc6b80053d0 2024-09-10T23:38:54.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6cb400640 1 -- 172.21.15.94:0/1354455886 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4010df0 con 0x7fc6b80053d0 2024-09-10T23:38:54.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6da000640 1 -- 172.21.15.94:0/1354455886 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d4151020 con 0x7fc6b80053d0 2024-09-10T23:38:54.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.896+0000 7fc6da000640 1 -- 172.21.15.94:0/1354455886 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d40d0690 con 0x7fc6b80053d0 2024-09-10T23:38:54.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.900+0000 7fc6cb400640 1 -- 172.21.15.94:0/1354455886 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7fc6c4011460 con 0x7fc6b80053d0 2024-09-10T23:38:54.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.900+0000 7fc6cb400640 1 --2- 172.21.15.94:0/1354455886 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc6b4072c10 0x7fc6b40750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:54.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.900+0000 7fc6cb400640 1 -- 172.21.15.94:0/1354455886 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7fc6c40943b0 con 0x7fc6b80053d0 2024-09-10T23:38:54.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.900+0000 7fc6d2a00640 1 --2- 172.21.15.94:0/1354455886 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc6b4072c10 0x7fc6b40750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:54.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.900+0000 7fc6d2a00640 1 --2- 172.21.15.94:0/1354455886 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc6b4072c10 0x7fc6b40750d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0006c90 tx=0x7fc6c00060b0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:54.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:54.904+0000 7fc6cb400640 1 -- 172.21.15.94:0/1354455886 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc6c401f030 con 0x7fc6b80053d0 2024-09-10T23:38:55.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.068+0000 7fc6da000640 1 -- 172.21.15.94:0/1354455886 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc6d409fc70 con 0x7fc6b4072c10 2024-09-10T23:38:55.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.072+0000 7fc6cb400640 1 -- 172.21.15.94:0/1354455886 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252474 (secure 0 0 0) 0x7fc6d409fc70 con 0x7fc6b4072c10 2024-09-10T23:38:55.079 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:55.081 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:38:55.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.076+0000 7fc6da000640 1 -- 172.21.15.94:0/1354455886 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc6b4072c10 msgr2=0x7fc6b40750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:55.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.076+0000 7fc6da000640 1 --2- 172.21.15.94:0/1354455886 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc6b4072c10 0x7fc6b40750d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0006c90 tx=0x7fc6c00060b0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.076+0000 7fc6da000640 1 -- 172.21.15.94:0/1354455886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6b80053d0 msgr2=0x7fc6d4148510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:55.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.076+0000 7fc6da000640 1 --2- 172.21.15.94:0/1354455886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6b80053d0 0x7fc6d4148510 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400e5c0 tx=0x7fc6c400ea80 comp rx=0 tx=0).stop 2024-09-10T23:38:55.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.076+0000 7fc6da000640 1 -- 172.21.15.94:0/1354455886 shutdown_connections 2024-09-10T23:38:55.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.076+0000 7fc6da000640 1 --2- 172.21.15.94:0/1354455886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6d4070780 0x7fc6d4070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.076+0000 7fc6da000640 1 --2- 172.21.15.94:0/1354455886 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc6b4072c10 0x7fc6b40750d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.076+0000 7fc6da000640 1 --2- 172.21.15.94:0/1354455886 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6d40c5790 0x7fc6d414b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.076+0000 7fc6da000640 1 --2- 172.21.15.94:0/1354455886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6b80053d0 0x7fc6d4148510 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.076+0000 7fc6da000640 1 -- 172.21.15.94:0/1354455886 >> 172.21.15.94:0/1354455886 conn(0x7fc6d40d1c60 msgr2=0x7fc6d415e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:55.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.076+0000 7fc6da000640 1 -- 172.21.15.94:0/1354455886 shutdown_connections 2024-09-10T23:38:55.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.076+0000 7fc6da000640 1 -- 172.21.15.94:0/1354455886 wait complete. 2024-09-10T23:38:55.099 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-10T23:38:54.763665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263100,"kb_used_data":13572,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711620,"statfs":{"total":773094113280,"available":772824698880,"internally_reserved":0,"allocated":13897728,"data_stored":6742700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.791320"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.996139+0000","last_change":"2024-09-10T23:38:51.018222+0000","last_active":"2024-09-10T23:38:51.996139+0000","last_peered":"2024-09-10T23:38:51.996139+0000","last_clean":"2024-09-10T23:38:51.996139+0000","last_became_active":"2024-09-10T23:38:51.017759+0000","last_became_peered":"2024-09-10T23:38:51.017759+0000","last_unstale":"2024-09-10T23:38:51.996139+0000","last_undegraded":"2024-09-10T23:38:51.996139+0000","last_fullsized":"2024-09-10T23:38:51.996139+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:45:47.988677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021773+0000","last_change":"2024-09-10T23:38:51.020141+0000","last_active":"2024-09-10T23:38:52.021773+0000","last_peered":"2024-09-10T23:38:52.021773+0000","last_clean":"2024-09-10T23:38:52.021773+0000","last_became_active":"2024-09-10T23:38:51.019862+0000","last_became_peered":"2024-09-10T23:38:51.019862+0000","last_unstale":"2024-09-10T23:38:52.021773+0000","last_undegraded":"2024-09-10T23:38:52.021773+0000","last_fullsized":"2024-09-10T23:38:52.021773+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:56.196878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156942+0000","last_change":"2024-09-10T23:38:51.010558+0000","last_active":"2024-09-10T23:38:52.156942+0000","last_peered":"2024-09-10T23:38:52.156942+0000","last_clean":"2024-09-10T23:38:52.156942+0000","last_became_active":"2024-09-10T23:38:51.010245+0000","last_became_peered":"2024-09-10T23:38:51.010245+0000","last_unstale":"2024-09-10T23:38:52.156942+0000","last_undegraded":"2024-09-10T23:38:52.156942+0000","last_fullsized":"2024-09-10T23:38:52.156942+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:38.373318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993275+0000","last_change":"2024-09-10T23:38:51.009845+0000","last_active":"2024-09-10T23:38:51.993275+0000","last_peered":"2024-09-10T23:38:51.993275+0000","last_clean":"2024-09-10T23:38:51.993275+0000","last_became_active":"2024-09-10T23:38:51.009573+0000","last_became_peered":"2024-09-10T23:38:51.009573+0000","last_unstale":"2024-09-10T23:38:51.993275+0000","last_undegraded":"2024-09-10T23:38:51.993275+0000","last_fullsized":"2024-09-10T23:38:51.993275+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:07.447754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157626+0000","last_change":"2024-09-10T23:38:51.013311+0000","last_active":"2024-09-10T23:38:52.157626+0000","last_peered":"2024-09-10T23:38:52.157626+0000","last_clean":"2024-09-10T23:38:52.157626+0000","last_became_active":"2024-09-10T23:38:51.012608+0000","last_became_peered":"2024-09-10T23:38:51.012608+0000","last_unstale":"2024-09-10T23:38:52.157626+0000","last_undegraded":"2024-09-10T23:38:52.157626+0000","last_fullsized":"2024-09-10T23:38:52.157626+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:09.926090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994984+0000","last_change":"2024-09-10T23:38:51.003749+0000","last_active":"2024-09-10T23:38:51.994984+0000","last_peered":"2024-09-10T23:38:51.994984+0000","last_clean":"2024-09-10T23:38:51.994984+0000","last_became_active":"2024-09-10T23:38:51.003413+0000","last_became_peered":"2024-09-10T23:38:51.003413+0000","last_unstale":"2024-09-10T23:38:51.994984+0000","last_undegraded":"2024-09-10T23:38:51.994984+0000","last_fullsized":"2024-09-10T23:38:51.994984+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:06.111238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156576+0000","last_change":"2024-09-10T23:38:51.017303+0000","last_active":"2024-09-10T23:38:52.156576+0000","last_peered":"2024-09-10T23:38:52.156576+0000","last_clean":"2024-09-10T23:38:52.156576+0000","last_became_active":"2024-09-10T23:38:51.016992+0000","last_became_peered":"2024-09-10T23:38:51.016992+0000","last_unstale":"2024-09-10T23:38:52.156576+0000","last_undegraded":"2024-09-10T23:38:52.156576+0000","last_fullsized":"2024-09-10T23:38:52.156576+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:13.966506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994448+0000","last_change":"2024-09-10T23:38:50.997375+0000","last_active":"2024-09-10T23:38:51.994448+0000","last_peered":"2024-09-10T23:38:51.994448+0000","last_clean":"2024-09-10T23:38:51.994448+0000","last_became_active":"2024-09-10T23:38:50.996786+0000","last_became_peered":"2024-09-10T23:38:50.996786+0000","last_unstale":"2024-09-10T23:38:51.994448+0000","last_undegraded":"2024-09-10T23:38:51.994448+0000","last_fullsized":"2024-09-10T23:38:51.994448+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:09.062977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995563+0000","last_change":"2024-09-10T23:38:51.005819+0000","last_active":"2024-09-10T23:38:51.995563+0000","last_peered":"2024-09-10T23:38:51.995563+0000","last_clean":"2024-09-10T23:38:51.995563+0000","last_became_active":"2024-09-10T23:38:51.005475+0000","last_became_peered":"2024-09-10T23:38:51.005475+0000","last_unstale":"2024-09-10T23:38:51.995563+0000","last_undegraded":"2024-09-10T23:38:51.995563+0000","last_fullsized":"2024-09-10T23:38:51.995563+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:34.802364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157147+0000","last_change":"2024-09-10T23:38:51.022181+0000","last_active":"2024-09-10T23:38:52.157147+0000","last_peered":"2024-09-10T23:38:52.157147+0000","last_clean":"2024-09-10T23:38:52.157147+0000","last_became_active":"2024-09-10T23:38:51.021871+0000","last_became_peered":"2024-09-10T23:38:51.021871+0000","last_unstale":"2024-09-10T23:38:52.157147+0000","last_undegraded":"2024-09-10T23:38:52.157147+0000","last_fullsized":"2024-09-10T23:38:52.157147+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.322453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995295+0000","last_change":"2024-09-10T23:38:51.008680+0000","last_active":"2024-09-10T23:38:51.995295+0000","last_peered":"2024-09-10T23:38:51.995295+0000","last_clean":"2024-09-10T23:38:51.995295+0000","last_became_active":"2024-09-10T23:38:51.008356+0000","last_became_peered":"2024-09-10T23:38:51.008356+0000","last_unstale":"2024-09-10T23:38:51.995295+0000","last_undegraded":"2024-09-10T23:38:51.995295+0000","last_fullsized":"2024-09-10T23:38:51.995295+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:37.993279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157244+0000","last_change":"2024-09-10T23:38:51.011435+0000","last_active":"2024-09-10T23:38:52.157244+0000","last_peered":"2024-09-10T23:38:52.157244+0000","last_clean":"2024-09-10T23:38:52.157244+0000","last_became_active":"2024-09-10T23:38:51.011077+0000","last_became_peered":"2024-09-10T23:38:51.011077+0000","last_unstale":"2024-09-10T23:38:52.157244+0000","last_undegraded":"2024-09-10T23:38:52.157244+0000","last_fullsized":"2024-09-10T23:38:52.157244+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:00.419380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157836+0000","last_change":"2024-09-10T23:38:51.013585+0000","last_active":"2024-09-10T23:38:52.157836+0000","last_peered":"2024-09-10T23:38:52.157836+0000","last_clean":"2024-09-10T23:38:52.157836+0000","last_became_active":"2024-09-10T23:38:51.013002+0000","last_became_peered":"2024-09-10T23:38:51.013002+0000","last_unstale":"2024-09-10T23:38:52.157836+0000","last_undegraded":"2024-09-10T23:38:52.157836+0000","last_fullsized":"2024-09-10T23:38:52.157836+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:51.674266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157365+0000","last_change":"2024-09-10T23:38:51.003902+0000","last_active":"2024-09-10T23:38:52.157365+0000","last_peered":"2024-09-10T23:38:52.157365+0000","last_clean":"2024-09-10T23:38:52.157365+0000","last_became_active":"2024-09-10T23:38:51.003591+0000","last_became_peered":"2024-09-10T23:38:51.003591+0000","last_unstale":"2024-09-10T23:38:52.157365+0000","last_undegraded":"2024-09-10T23:38:52.157365+0000","last_fullsized":"2024-09-10T23:38:52.157365+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:19.795894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156610+0000","last_change":"2024-09-10T23:38:50.999932+0000","last_active":"2024-09-10T23:38:52.156610+0000","last_peered":"2024-09-10T23:38:52.156610+0000","last_clean":"2024-09-10T23:38:52.156610+0000","last_became_active":"2024-09-10T23:38:50.999458+0000","last_became_peered":"2024-09-10T23:38:50.999458+0000","last_unstale":"2024-09-10T23:38:52.156610+0000","last_undegraded":"2024-09-10T23:38:52.156610+0000","last_fullsized":"2024-09-10T23:38:52.156610+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:11.526399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156620+0000","last_change":"2024-09-10T23:38:51.010919+0000","last_active":"2024-09-10T23:38:52.156620+0000","last_peered":"2024-09-10T23:38:52.156620+0000","last_clean":"2024-09-10T23:38:52.156620+0000","last_became_active":"2024-09-10T23:38:51.010626+0000","last_became_peered":"2024-09-10T23:38:51.010626+0000","last_unstale":"2024-09-10T23:38:52.156620+0000","last_undegraded":"2024-09-10T23:38:52.156620+0000","last_fullsized":"2024-09-10T23:38:52.156620+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:36.508752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157809+0000","last_change":"2024-09-10T23:38:51.017242+0000","last_active":"2024-09-10T23:38:52.157809+0000","last_peered":"2024-09-10T23:38:52.157809+0000","last_clean":"2024-09-10T23:38:52.157809+0000","last_became_active":"2024-09-10T23:38:51.016989+0000","last_became_peered":"2024-09-10T23:38:51.016989+0000","last_unstale":"2024-09-10T23:38:52.157809+0000","last_undegraded":"2024-09-10T23:38:52.157809+0000","last_fullsized":"2024-09-10T23:38:52.157809+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:10:16.219440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.158133+0000","last_change":"2024-09-10T23:38:51.008074+0000","last_active":"2024-09-10T23:38:52.158133+0000","last_peered":"2024-09-10T23:38:52.158133+0000","last_clean":"2024-09-10T23:38:52.158133+0000","last_became_active":"2024-09-10T23:38:51.006439+0000","last_became_peered":"2024-09-10T23:38:51.006439+0000","last_unstale":"2024-09-10T23:38:52.158133+0000","last_undegraded":"2024-09-10T23:38:52.158133+0000","last_fullsized":"2024-09-10T23:38:52.158133+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:43.817898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995782+0000","last_change":"2024-09-10T23:38:50.998685+0000","last_active":"2024-09-10T23:38:51.995782+0000","last_peered":"2024-09-10T23:38:51.995782+0000","last_clean":"2024-09-10T23:38:51.995782+0000","last_became_active":"2024-09-10T23:38:50.998378+0000","last_became_peered":"2024-09-10T23:38:50.998378+0000","last_unstale":"2024-09-10T23:38:51.995782+0000","last_undegraded":"2024-09-10T23:38:51.995782+0000","last_fullsized":"2024-09-10T23:38:51.995782+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:45.667770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995757+0000","last_change":"2024-09-10T23:38:51.007136+0000","last_active":"2024-09-10T23:38:51.995757+0000","last_peered":"2024-09-10T23:38:51.995757+0000","last_clean":"2024-09-10T23:38:51.995757+0000","last_became_active":"2024-09-10T23:38:51.006882+0000","last_became_peered":"2024-09-10T23:38:51.006882+0000","last_unstale":"2024-09-10T23:38:51.995757+0000","last_undegraded":"2024-09-10T23:38:51.995757+0000","last_fullsized":"2024-09-10T23:38:51.995757+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:33.127810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157618+0000","last_change":"2024-09-10T23:38:51.016399+0000","last_active":"2024-09-10T23:38:52.157618+0000","last_peered":"2024-09-10T23:38:52.157618+0000","last_clean":"2024-09-10T23:38:52.157618+0000","last_became_active":"2024-09-10T23:38:51.016092+0000","last_became_peered":"2024-09-10T23:38:51.016092+0000","last_unstale":"2024-09-10T23:38:52.157618+0000","last_undegraded":"2024-09-10T23:38:52.157618+0000","last_fullsized":"2024-09-10T23:38:52.157618+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:10.998657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156905+0000","last_change":"2024-09-10T23:38:51.013935+0000","last_active":"2024-09-10T23:38:52.156905+0000","last_peered":"2024-09-10T23:38:52.156905+0000","last_clean":"2024-09-10T23:38:52.156905+0000","last_became_active":"2024-09-10T23:38:51.012090+0000","last_became_peered":"2024-09-10T23:38:51.012090+0000","last_unstale":"2024-09-10T23:38:52.156905+0000","last_undegraded":"2024-09-10T23:38:52.156905+0000","last_fullsized":"2024-09-10T23:38:52.156905+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:35.176499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992714+0000","last_change":"2024-09-10T23:38:51.014060+0000","last_active":"2024-09-10T23:38:51.992714+0000","last_peered":"2024-09-10T23:38:51.992714+0000","last_clean":"2024-09-10T23:38:51.992714+0000","last_became_active":"2024-09-10T23:38:51.013517+0000","last_became_peered":"2024-09-10T23:38:51.013517+0000","last_unstale":"2024-09-10T23:38:51.992714+0000","last_undegraded":"2024-09-10T23:38:51.992714+0000","last_fullsized":"2024-09-10T23:38:51.992714+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:10.057833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156429+0000","last_change":"2024-09-10T23:38:51.001717+0000","last_active":"2024-09-10T23:38:52.156429+0000","last_peered":"2024-09-10T23:38:52.156429+0000","last_clean":"2024-09-10T23:38:52.156429+0000","last_became_active":"2024-09-10T23:38:51.000793+0000","last_became_peered":"2024-09-10T23:38:51.000793+0000","last_unstale":"2024-09-10T23:38:52.156429+0000","last_undegraded":"2024-09-10T23:38:52.156429+0000","last_fullsized":"2024-09-10T23:38:52.156429+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:07.790657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021721+0000","last_change":"2024-09-10T23:38:51.013055+0000","last_active":"2024-09-10T23:38:52.021721+0000","last_peered":"2024-09-10T23:38:52.021721+0000","last_clean":"2024-09-10T23:38:52.021721+0000","last_became_active":"2024-09-10T23:38:51.012769+0000","last_became_peered":"2024-09-10T23:38:51.012769+0000","last_unstale":"2024-09-10T23:38:52.021721+0000","last_undegraded":"2024-09-10T23:38:52.021721+0000","last_fullsized":"2024-09-10T23:38:52.021721+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:29.118023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.996104+0000","last_change":"2024-09-10T23:38:51.007965+0000","last_active":"2024-09-10T23:38:51.996104+0000","last_peered":"2024-09-10T23:38:51.996104+0000","last_clean":"2024-09-10T23:38:51.996104+0000","last_became_active":"2024-09-10T23:38:51.006185+0000","last_became_peered":"2024-09-10T23:38:51.006185+0000","last_unstale":"2024-09-10T23:38:51.996104+0000","last_undegraded":"2024-09-10T23:38:51.996104+0000","last_fullsized":"2024-09-10T23:38:51.996104+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:47.762556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021374+0000","last_change":"2024-09-10T23:38:51.011470+0000","last_active":"2024-09-10T23:38:52.021374+0000","last_peered":"2024-09-10T23:38:52.021374+0000","last_clean":"2024-09-10T23:38:52.021374+0000","last_became_active":"2024-09-10T23:38:51.011224+0000","last_became_peered":"2024-09-10T23:38:51.011224+0000","last_unstale":"2024-09-10T23:38:52.021374+0000","last_undegraded":"2024-09-10T23:38:52.021374+0000","last_fullsized":"2024-09-10T23:38:52.021374+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:06.816076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021675+0000","last_change":"2024-09-10T23:38:51.019262+0000","last_active":"2024-09-10T23:38:52.021675+0000","last_peered":"2024-09-10T23:38:52.021675+0000","last_clean":"2024-09-10T23:38:52.021675+0000","last_became_active":"2024-09-10T23:38:51.018945+0000","last_became_peered":"2024-09-10T23:38:51.018945+0000","last_unstale":"2024-09-10T23:38:52.021675+0000","last_undegraded":"2024-09-10T23:38:52.021675+0000","last_fullsized":"2024-09-10T23:38:52.021675+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:40.555309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994782+0000","last_change":"2024-09-10T23:38:51.012443+0000","last_active":"2024-09-10T23:38:51.994782+0000","last_peered":"2024-09-10T23:38:51.994782+0000","last_clean":"2024-09-10T23:38:51.994782+0000","last_became_active":"2024-09-10T23:38:51.011778+0000","last_became_peered":"2024-09-10T23:38:51.011778+0000","last_unstale":"2024-09-10T23:38:51.994782+0000","last_undegraded":"2024-09-10T23:38:51.994782+0000","last_fullsized":"2024-09-10T23:38:51.994782+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:03.303492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157109+0000","last_change":"2024-09-10T23:38:51.011728+0000","last_active":"2024-09-10T23:38:52.157109+0000","last_peered":"2024-09-10T23:38:52.157109+0000","last_clean":"2024-09-10T23:38:52.157109+0000","last_became_active":"2024-09-10T23:38:51.011458+0000","last_became_peered":"2024-09-10T23:38:51.011458+0000","last_unstale":"2024-09-10T23:38:52.157109+0000","last_undegraded":"2024-09-10T23:38:52.157109+0000","last_fullsized":"2024-09-10T23:38:52.157109+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:26.219894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021225+0000","last_change":"2024-09-10T23:38:51.015961+0000","last_active":"2024-09-10T23:38:52.021225+0000","last_peered":"2024-09-10T23:38:52.021225+0000","last_clean":"2024-09-10T23:38:52.021225+0000","last_became_active":"2024-09-10T23:38:51.015411+0000","last_became_peered":"2024-09-10T23:38:51.015411+0000","last_unstale":"2024-09-10T23:38:52.021225+0000","last_undegraded":"2024-09-10T23:38:52.021225+0000","last_fullsized":"2024-09-10T23:38:52.021225+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:12.487250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021548+0000","last_change":"2024-09-10T23:38:51.013266+0000","last_active":"2024-09-10T23:38:52.021548+0000","last_peered":"2024-09-10T23:38:52.021548+0000","last_clean":"2024-09-10T23:38:52.021548+0000","last_became_active":"2024-09-10T23:38:51.012935+0000","last_became_peered":"2024-09-10T23:38:51.012935+0000","last_unstale":"2024-09-10T23:38:52.021548+0000","last_undegraded":"2024-09-10T23:38:52.021548+0000","last_fullsized":"2024-09-10T23:38:52.021548+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:50.487537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.158176+0000","last_change":"2024-09-10T23:38:51.015320+0000","last_active":"2024-09-10T23:38:52.158176+0000","last_peered":"2024-09-10T23:38:52.158176+0000","last_clean":"2024-09-10T23:38:52.158176+0000","last_became_active":"2024-09-10T23:38:51.014187+0000","last_became_peered":"2024-09-10T23:38:51.014187+0000","last_unstale":"2024-09-10T23:38:52.158176+0000","last_undegraded":"2024-09-10T23:38:52.158176+0000","last_fullsized":"2024-09-10T23:38:52.158176+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:45.910996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"126'4","reported_seq":19,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157753+0000","last_change":"2024-09-10T23:38:50.001083+0000","last_active":"2024-09-10T23:38:52.157753+0000","last_peered":"2024-09-10T23:38:52.157753+0000","last_clean":"2024-09-10T23:38:52.157753+0000","last_became_active":"2024-09-10T23:38:50.000792+0000","last_became_peered":"2024-09-10T23:38:50.000792+0000","last_unstale":"2024-09-10T23:38:52.157753+0000","last_undegraded":"2024-09-10T23:38:52.157753+0000","last_fullsized":"2024-09-10T23:38:52.157753+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:09.107698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157163+0000","last_change":"2024-09-10T23:38:51.011028+0000","last_active":"2024-09-10T23:38:52.157163+0000","last_peered":"2024-09-10T23:38:52.157163+0000","last_clean":"2024-09-10T23:38:52.157163+0000","last_became_active":"2024-09-10T23:38:51.010772+0000","last_became_peered":"2024-09-10T23:38:51.010772+0000","last_unstale":"2024-09-10T23:38:52.157163+0000","last_undegraded":"2024-09-10T23:38:52.157163+0000","last_fullsized":"2024-09-10T23:38:52.157163+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:31.010655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995480+0000","last_change":"2024-09-10T23:38:49.996561+0000","last_active":"2024-09-10T23:38:51.995480+0000","last_peered":"2024-09-10T23:38:51.995480+0000","last_clean":"2024-09-10T23:38:51.995480+0000","last_became_active":"2024-09-10T23:38:49.996215+0000","last_became_peered":"2024-09-10T23:38:49.996215+0000","last_unstale":"2024-09-10T23:38:51.995480+0000","last_undegraded":"2024-09-10T23:38:51.995480+0000","last_fullsized":"2024-09-10T23:38:51.995480+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:19.142799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994781+0000","last_change":"2024-09-10T23:38:51.003624+0000","last_active":"2024-09-10T23:38:51.994781+0000","last_peered":"2024-09-10T23:38:51.994781+0000","last_clean":"2024-09-10T23:38:51.994781+0000","last_became_active":"2024-09-10T23:38:51.003342+0000","last_became_peered":"2024-09-10T23:38:51.003342+0000","last_unstale":"2024-09-10T23:38:51.994781+0000","last_undegraded":"2024-09-10T23:38:51.994781+0000","last_fullsized":"2024-09-10T23:38:51.994781+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:35.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157004+0000","last_change":"2024-09-10T23:38:49.998832+0000","last_active":"2024-09-10T23:38:52.157004+0000","last_peered":"2024-09-10T23:38:52.157004+0000","last_clean":"2024-09-10T23:38:52.157004+0000","last_became_active":"2024-09-10T23:38:49.998579+0000","last_became_peered":"2024-09-10T23:38:49.998579+0000","last_unstale":"2024-09-10T23:38:52.157004+0000","last_undegraded":"2024-09-10T23:38:52.157004+0000","last_fullsized":"2024-09-10T23:38:52.157004+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:56.126627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993427+0000","last_change":"2024-09-10T23:38:51.019178+0000","last_active":"2024-09-10T23:38:51.993427+0000","last_peered":"2024-09-10T23:38:51.993427+0000","last_clean":"2024-09-10T23:38:51.993427+0000","last_became_active":"2024-09-10T23:38:51.018907+0000","last_became_peered":"2024-09-10T23:38:51.018907+0000","last_unstale":"2024-09-10T23:38:51.993427+0000","last_undegraded":"2024-09-10T23:38:51.993427+0000","last_fullsized":"2024-09-10T23:38:51.993427+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:07.963755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157502+0000","last_change":"2024-09-10T23:38:50.000573+0000","last_active":"2024-09-10T23:38:52.157502+0000","last_peered":"2024-09-10T23:38:52.157502+0000","last_clean":"2024-09-10T23:38:52.157502+0000","last_became_active":"2024-09-10T23:38:50.000255+0000","last_became_peered":"2024-09-10T23:38:50.000255+0000","last_unstale":"2024-09-10T23:38:52.157502+0000","last_undegraded":"2024-09-10T23:38:52.157502+0000","last_fullsized":"2024-09-10T23:38:52.157502+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:12.020186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021760+0000","last_change":"2024-09-10T23:38:51.015563+0000","last_active":"2024-09-10T23:38:52.021760+0000","last_peered":"2024-09-10T23:38:52.021760+0000","last_clean":"2024-09-10T23:38:52.021760+0000","last_became_active":"2024-09-10T23:38:51.015282+0000","last_became_peered":"2024-09-10T23:38:51.015282+0000","last_unstale":"2024-09-10T23:38:52.021760+0000","last_undegraded":"2024-09-10T23:38:52.021760+0000","last_fullsized":"2024-09-10T23:38:52.021760+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:58.764114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994925+0000","last_change":"2024-09-10T23:38:49.984678+0000","last_active":"2024-09-10T23:38:51.994925+0000","last_peered":"2024-09-10T23:38:51.994925+0000","last_clean":"2024-09-10T23:38:51.994925+0000","last_became_active":"2024-09-10T23:38:49.984330+0000","last_became_peered":"2024-09-10T23:38:49.984330+0000","last_unstale":"2024-09-10T23:38:51.994925+0000","last_undegraded":"2024-09-10T23:38:51.994925+0000","last_fullsized":"2024-09-10T23:38:51.994925+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:08:33.519585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993109+0000","last_change":"2024-09-10T23:38:51.012311+0000","last_active":"2024-09-10T23:38:51.993109+0000","last_peered":"2024-09-10T23:38:51.993109+0000","last_clean":"2024-09-10T23:38:51.993109+0000","last_became_active":"2024-09-10T23:38:51.011809+0000","last_became_peered":"2024-09-10T23:38:51.011809+0000","last_unstale":"2024-09-10T23:38:51.993109+0000","last_undegraded":"2024-09-10T23:38:51.993109+0000","last_fullsized":"2024-09-10T23:38:51.993109+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:20.095934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156832+0000","last_change":"2024-09-10T23:38:49.994554+0000","last_active":"2024-09-10T23:38:52.156832+0000","last_peered":"2024-09-10T23:38:52.156832+0000","last_clean":"2024-09-10T23:38:52.156832+0000","last_became_active":"2024-09-10T23:38:49.994268+0000","last_became_peered":"2024-09-10T23:38:49.994268+0000","last_unstale":"2024-09-10T23:38:52.156832+0000","last_undegraded":"2024-09-10T23:38:52.156832+0000","last_fullsized":"2024-09-10T23:38:52.156832+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:03.005349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021568+0000","last_change":"2024-09-10T23:38:51.019154+0000","last_active":"2024-09-10T23:38:52.021568+0000","last_peered":"2024-09-10T23:38:52.021568+0000","last_clean":"2024-09-10T23:38:52.021568+0000","last_became_active":"2024-09-10T23:38:51.018782+0000","last_became_peered":"2024-09-10T23:38:51.018782+0000","last_unstale":"2024-09-10T23:38:52.021568+0000","last_undegraded":"2024-09-10T23:38:52.021568+0000","last_fullsized":"2024-09-10T23:38:52.021568+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:22.736229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994445+0000","last_change":"2024-09-10T23:38:49.997027+0000","last_active":"2024-09-10T23:38:51.994445+0000","last_peered":"2024-09-10T23:38:51.994445+0000","last_clean":"2024-09-10T23:38:51.994445+0000","last_became_active":"2024-09-10T23:38:49.996716+0000","last_became_peered":"2024-09-10T23:38:49.996716+0000","last_unstale":"2024-09-10T23:38:51.994445+0000","last_undegraded":"2024-09-10T23:38:51.994445+0000","last_fullsized":"2024-09-10T23:38:51.994445+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:21.154567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992856+0000","last_change":"2024-09-10T23:38:51.017366+0000","last_active":"2024-09-10T23:38:51.992856+0000","last_peered":"2024-09-10T23:38:51.992856+0000","last_clean":"2024-09-10T23:38:51.992856+0000","last_became_active":"2024-09-10T23:38:51.014336+0000","last_became_peered":"2024-09-10T23:38:51.014336+0000","last_unstale":"2024-09-10T23:38:51.992856+0000","last_undegraded":"2024-09-10T23:38:51.992856+0000","last_fullsized":"2024-09-10T23:38:51.992856+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:49.821245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156212+0000","last_change":"2024-09-10T23:38:50.007729+0000","last_active":"2024-09-10T23:38:52.156212+0000","last_peered":"2024-09-10T23:38:52.156212+0000","last_clean":"2024-09-10T23:38:52.156212+0000","last_became_active":"2024-09-10T23:38:50.007415+0000","last_became_peered":"2024-09-10T23:38:50.007415+0000","last_unstale":"2024-09-10T23:38:52.156212+0000","last_undegraded":"2024-09-10T23:38:52.156212+0000","last_fullsized":"2024-09-10T23:38:52.156212+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:17.415987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993585+0000","last_change":"2024-09-10T23:38:51.009272+0000","last_active":"2024-09-10T23:38:51.993585+0000","last_peered":"2024-09-10T23:38:51.993585+0000","last_clean":"2024-09-10T23:38:51.993585+0000","last_became_active":"2024-09-10T23:38:51.009006+0000","last_became_peered":"2024-09-10T23:38:51.009006+0000","last_unstale":"2024-09-10T23:38:51.993585+0000","last_undegraded":"2024-09-10T23:38:51.993585+0000","last_fullsized":"2024-09-10T23:38:51.993585+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:27.155391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995673+0000","last_change":"2024-09-10T23:38:49.990274+0000","last_active":"2024-09-10T23:38:51.995673+0000","last_peered":"2024-09-10T23:38:51.995673+0000","last_clean":"2024-09-10T23:38:51.995673+0000","last_became_active":"2024-09-10T23:38:49.989988+0000","last_became_peered":"2024-09-10T23:38:49.989988+0000","last_unstale":"2024-09-10T23:38:51.995673+0000","last_undegraded":"2024-09-10T23:38:51.995673+0000","last_fullsized":"2024-09-10T23:38:51.995673+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:19:55.106626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156858+0000","last_change":"2024-09-10T23:38:51.005665+0000","last_active":"2024-09-10T23:38:52.156858+0000","last_peered":"2024-09-10T23:38:52.156858+0000","last_clean":"2024-09-10T23:38:52.156858+0000","last_became_active":"2024-09-10T23:38:51.005389+0000","last_became_peered":"2024-09-10T23:38:51.005389+0000","last_unstale":"2024-09-10T23:38:52.156858+0000","last_undegraded":"2024-09-10T23:38:52.156858+0000","last_fullsized":"2024-09-10T23:38:52.156858+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:58.218590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.158268+0000","last_change":"2024-09-10T23:38:49.991719+0000","last_active":"2024-09-10T23:38:52.158268+0000","last_peered":"2024-09-10T23:38:52.158268+0000","last_clean":"2024-09-10T23:38:52.158268+0000","last_became_active":"2024-09-10T23:38:49.991454+0000","last_became_peered":"2024-09-10T23:38:49.991454+0000","last_unstale":"2024-09-10T23:38:52.158268+0000","last_undegraded":"2024-09-10T23:38:52.158268+0000","last_fullsized":"2024-09-10T23:38:52.158268+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:05.922315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157174+0000","last_change":"2024-09-10T23:38:51.008649+0000","last_active":"2024-09-10T23:38:52.157174+0000","last_peered":"2024-09-10T23:38:52.157174+0000","last_clean":"2024-09-10T23:38:52.157174+0000","last_became_active":"2024-09-10T23:38:51.008339+0000","last_became_peered":"2024-09-10T23:38:51.008339+0000","last_unstale":"2024-09-10T23:38:52.157174+0000","last_undegraded":"2024-09-10T23:38:52.157174+0000","last_fullsized":"2024-09-10T23:38:52.157174+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:19.004763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"126'6","reported_seq":21,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156876+0000","last_change":"2024-09-10T23:38:49.994548+0000","last_active":"2024-09-10T23:38:52.156876+0000","last_peered":"2024-09-10T23:38:52.156876+0000","last_clean":"2024-09-10T23:38:52.156876+0000","last_became_active":"2024-09-10T23:38:49.994202+0000","last_became_peered":"2024-09-10T23:38:49.994202+0000","last_unstale":"2024-09-10T23:38:52.156876+0000","last_undegraded":"2024-09-10T23:38:52.156876+0000","last_fullsized":"2024-09-10T23:38:52.156876+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:15.911264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992743+0000","last_change":"2024-09-10T23:38:51.014250+0000","last_active":"2024-09-10T23:38:51.992743+0000","last_peered":"2024-09-10T23:38:51.992743+0000","last_clean":"2024-09-10T23:38:51.992743+0000","last_became_active":"2024-09-10T23:38:51.013954+0000","last_became_peered":"2024-09-10T23:38:51.013954+0000","last_unstale":"2024-09-10T23:38:51.992743+0000","last_undegraded":"2024-09-10T23:38:51.992743+0000","last_fullsized":"2024-09-10T23:38:51.992743+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:30.596239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157561+0000","last_change":"2024-09-10T23:38:50.001391+0000","last_active":"2024-09-10T23:38:52.157561+0000","last_peered":"2024-09-10T23:38:52.157561+0000","last_clean":"2024-09-10T23:38:52.157561+0000","last_became_active":"2024-09-10T23:38:50.000839+0000","last_became_peered":"2024-09-10T23:38:50.000839+0000","last_unstale":"2024-09-10T23:38:52.157561+0000","last_undegraded":"2024-09-10T23:38:52.157561+0000","last_fullsized":"2024-09-10T23:38:52.157561+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:23.703292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994784+0000","last_change":"2024-09-10T23:38:51.012723+0000","last_active":"2024-09-10T23:38:51.994784+0000","last_peered":"2024-09-10T23:38:51.994784+0000","last_clean":"2024-09-10T23:38:51.994784+0000","last_became_active":"2024-09-10T23:38:51.012159+0000","last_became_peered":"2024-09-10T23:38:51.012159+0000","last_unstale":"2024-09-10T23:38:51.994784+0000","last_undegraded":"2024-09-10T23:38:51.994784+0000","last_fullsized":"2024-09-10T23:38:51.994784+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:24.188587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157400+0000","last_change":"2024-09-10T23:38:49.999593+0000","last_active":"2024-09-10T23:38:52.157400+0000","last_peered":"2024-09-10T23:38:52.157400+0000","last_clean":"2024-09-10T23:38:52.157400+0000","last_became_active":"2024-09-10T23:38:49.999258+0000","last_became_peered":"2024-09-10T23:38:49.999258+0000","last_unstale":"2024-09-10T23:38:52.157400+0000","last_undegraded":"2024-09-10T23:38:52.157400+0000","last_fullsized":"2024-09-10T23:38:52.157400+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:03.695248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992600+0000","last_change":"2024-09-10T23:38:51.009127+0000","last_active":"2024-09-10T23:38:51.992600+0000","last_peered":"2024-09-10T23:38:51.992600+0000","last_clean":"2024-09-10T23:38:51.992600+0000","last_became_active":"2024-09-10T23:38:51.008867+0000","last_became_peered":"2024-09-10T23:38:51.008867+0000","last_unstale":"2024-09-10T23:38:51.992600+0000","last_undegraded":"2024-09-10T23:38:51.992600+0000","last_fullsized":"2024-09-10T23:38:51.992600+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:09.659885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021334+0000","last_change":"2024-09-10T23:38:50.000383+0000","last_active":"2024-09-10T23:38:52.021334+0000","last_peered":"2024-09-10T23:38:52.021334+0000","last_clean":"2024-09-10T23:38:52.021334+0000","last_became_active":"2024-09-10T23:38:49.999943+0000","last_became_peered":"2024-09-10T23:38:49.999943+0000","last_unstale":"2024-09-10T23:38:52.021334+0000","last_undegraded":"2024-09-10T23:38:52.021334+0000","last_fullsized":"2024-09-10T23:38:52.021334+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:26.170538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992917+0000","last_change":"2024-09-10T23:38:51.013767+0000","last_active":"2024-09-10T23:38:51.992917+0000","last_peered":"2024-09-10T23:38:51.992917+0000","last_clean":"2024-09-10T23:38:51.992917+0000","last_became_active":"2024-09-10T23:38:51.010462+0000","last_became_peered":"2024-09-10T23:38:51.010462+0000","last_unstale":"2024-09-10T23:38:51.992917+0000","last_undegraded":"2024-09-10T23:38:51.992917+0000","last_fullsized":"2024-09-10T23:38:51.992917+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:03.810193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"126'8","reported_seq":23,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994578+0000","last_change":"2024-09-10T23:38:50.001418+0000","last_active":"2024-09-10T23:38:51.994578+0000","last_peered":"2024-09-10T23:38:51.994578+0000","last_clean":"2024-09-10T23:38:51.994578+0000","last_became_active":"2024-09-10T23:38:50.001055+0000","last_became_peered":"2024-09-10T23:38:50.001055+0000","last_unstale":"2024-09-10T23:38:51.994578+0000","last_undegraded":"2024-09-10T23:38:51.994578+0000","last_fullsized":"2024-09-10T23:38:51.994578+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:41.655333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.158268+0000","last_change":"2024-09-10T23:38:51.006705+0000","last_active":"2024-09-10T23:38:52.158268+0000","last_peered":"2024-09-10T23:38:52.158268+0000","last_clean":"2024-09-10T23:38:52.158268+0000","last_became_active":"2024-09-10T23:38:51.006091+0000","last_became_peered":"2024-09-10T23:38:51.006091+0000","last_unstale":"2024-09-10T23:38:52.158268+0000","last_undegraded":"2024-09-10T23:38:52.158268+0000","last_fullsized":"2024-09-10T23:38:52.158268+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:10.509963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993621+0000","last_change":"2024-09-10T23:38:50.005199+0000","last_active":"2024-09-10T23:38:51.993621+0000","last_peered":"2024-09-10T23:38:51.993621+0000","last_clean":"2024-09-10T23:38:51.993621+0000","last_became_active":"2024-09-10T23:38:50.004764+0000","last_became_peered":"2024-09-10T23:38:50.004764+0000","last_unstale":"2024-09-10T23:38:51.993621+0000","last_undegraded":"2024-09-10T23:38:51.993621+0000","last_fullsized":"2024-09-10T23:38:51.993621+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:35.392555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021749+0000","last_change":"2024-09-10T23:38:51.020237+0000","last_active":"2024-09-10T23:38:52.021749+0000","last_peered":"2024-09-10T23:38:52.021749+0000","last_clean":"2024-09-10T23:38:52.021749+0000","last_became_active":"2024-09-10T23:38:51.019623+0000","last_became_peered":"2024-09-10T23:38:51.019623+0000","last_unstale":"2024-09-10T23:38:52.021749+0000","last_undegraded":"2024-09-10T23:38:52.021749+0000","last_fullsized":"2024-09-10T23:38:52.021749+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:18.958122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995945+0000","last_change":"2024-09-10T23:38:49.995217+0000","last_active":"2024-09-10T23:38:51.995945+0000","last_peered":"2024-09-10T23:38:51.995945+0000","last_clean":"2024-09-10T23:38:51.995945+0000","last_became_active":"2024-09-10T23:38:49.994905+0000","last_became_peered":"2024-09-10T23:38:49.994905+0000","last_unstale":"2024-09-10T23:38:51.995945+0000","last_undegraded":"2024-09-10T23:38:51.995945+0000","last_fullsized":"2024-09-10T23:38:51.995945+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:38.014917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157869+0000","last_change":"2024-09-10T23:38:51.001392+0000","last_active":"2024-09-10T23:38:52.157869+0000","last_peered":"2024-09-10T23:38:52.157869+0000","last_clean":"2024-09-10T23:38:52.157869+0000","last_became_active":"2024-09-10T23:38:51.001116+0000","last_became_peered":"2024-09-10T23:38:51.001116+0000","last_unstale":"2024-09-10T23:38:52.157869+0000","last_undegraded":"2024-09-10T23:38:52.157869+0000","last_fullsized":"2024-09-10T23:38:52.157869+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:36.399112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157042+0000","last_change":"2024-09-10T23:38:49.994999+0000","last_active":"2024-09-10T23:38:52.157042+0000","last_peered":"2024-09-10T23:38:52.157042+0000","last_clean":"2024-09-10T23:38:52.157042+0000","last_became_active":"2024-09-10T23:38:49.994677+0000","last_became_peered":"2024-09-10T23:38:49.994677+0000","last_unstale":"2024-09-10T23:38:52.157042+0000","last_undegraded":"2024-09-10T23:38:52.157042+0000","last_fullsized":"2024-09-10T23:38:52.157042+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:56.573683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995344+0000","last_change":"2024-09-10T23:38:51.004574+0000","last_active":"2024-09-10T23:38:51.995344+0000","last_peered":"2024-09-10T23:38:51.995344+0000","last_clean":"2024-09-10T23:38:51.995344+0000","last_became_active":"2024-09-10T23:38:51.004244+0000","last_became_peered":"2024-09-10T23:38:51.004244+0000","last_unstale":"2024-09-10T23:38:51.995344+0000","last_undegraded":"2024-09-10T23:38:51.995344+0000","last_fullsized":"2024-09-10T23:38:51.995344+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:46.365535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021583+0000","last_change":"2024-09-10T23:38:49.995698+0000","last_active":"2024-09-10T23:38:52.021583+0000","last_peered":"2024-09-10T23:38:52.021583+0000","last_clean":"2024-09-10T23:38:52.021583+0000","last_became_active":"2024-09-10T23:38:49.995420+0000","last_became_peered":"2024-09-10T23:38:49.995420+0000","last_unstale":"2024-09-10T23:38:52.021583+0000","last_undegraded":"2024-09-10T23:38:52.021583+0000","last_fullsized":"2024-09-10T23:38:52.021583+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:13.801146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157304+0000","last_change":"2024-09-10T23:38:51.002583+0000","last_active":"2024-09-10T23:38:52.157304+0000","last_peered":"2024-09-10T23:38:52.157304+0000","last_clean":"2024-09-10T23:38:52.157304+0000","last_became_active":"2024-09-10T23:38:51.001781+0000","last_became_peered":"2024-09-10T23:38:51.001781+0000","last_unstale":"2024-09-10T23:38:52.157304+0000","last_undegraded":"2024-09-10T23:38:52.157304+0000","last_fullsized":"2024-09-10T23:38:52.157304+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:35.522701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994408+0000","last_change":"2024-09-10T23:38:49.995991+0000","last_active":"2024-09-10T23:38:51.994408+0000","last_peered":"2024-09-10T23:38:51.994408+0000","last_clean":"2024-09-10T23:38:51.994408+0000","last_became_active":"2024-09-10T23:38:49.995646+0000","last_became_peered":"2024-09-10T23:38:49.995646+0000","last_unstale":"2024-09-10T23:38:51.994408+0000","last_undegraded":"2024-09-10T23:38:51.994408+0000","last_fullsized":"2024-09-10T23:38:51.994408+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:05.537992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021333+0000","last_change":"2024-09-10T23:38:51.016657+0000","last_active":"2024-09-10T23:38:52.021333+0000","last_peered":"2024-09-10T23:38:52.021333+0000","last_clean":"2024-09-10T23:38:52.021333+0000","last_became_active":"2024-09-10T23:38:51.016362+0000","last_became_peered":"2024-09-10T23:38:51.016362+0000","last_unstale":"2024-09-10T23:38:52.021333+0000","last_undegraded":"2024-09-10T23:38:52.021333+0000","last_fullsized":"2024-09-10T23:38:52.021333+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:41.867094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992698+0000","last_change":"2024-09-10T23:38:50.008339+0000","last_active":"2024-09-10T23:38:51.992698+0000","last_peered":"2024-09-10T23:38:51.992698+0000","last_clean":"2024-09-10T23:38:51.992698+0000","last_became_active":"2024-09-10T23:38:50.008050+0000","last_became_peered":"2024-09-10T23:38:50.008050+0000","last_unstale":"2024-09-10T23:38:51.992698+0000","last_undegraded":"2024-09-10T23:38:51.992698+0000","last_fullsized":"2024-09-10T23:38:51.992698+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:44.904439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156730+0000","last_change":"2024-09-10T23:38:51.002280+0000","last_active":"2024-09-10T23:38:52.156730+0000","last_peered":"2024-09-10T23:38:52.156730+0000","last_clean":"2024-09-10T23:38:52.156730+0000","last_became_active":"2024-09-10T23:38:51.001949+0000","last_became_peered":"2024-09-10T23:38:51.001949+0000","last_unstale":"2024-09-10T23:38:52.156730+0000","last_undegraded":"2024-09-10T23:38:52.156730+0000","last_fullsized":"2024-09-10T23:38:52.156730+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:37:38.619339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992949+0000","last_change":"2024-09-10T23:38:50.004257+0000","last_active":"2024-09-10T23:38:51.992949+0000","last_peered":"2024-09-10T23:38:51.992949+0000","last_clean":"2024-09-10T23:38:51.992949+0000","last_became_active":"2024-09-10T23:38:50.003746+0000","last_became_peered":"2024-09-10T23:38:50.003746+0000","last_unstale":"2024-09-10T23:38:51.992949+0000","last_undegraded":"2024-09-10T23:38:51.992949+0000","last_fullsized":"2024-09-10T23:38:51.992949+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:50.667709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021208+0000","last_change":"2024-09-10T23:38:51.015678+0000","last_active":"2024-09-10T23:38:52.021208+0000","last_peered":"2024-09-10T23:38:52.021208+0000","last_clean":"2024-09-10T23:38:52.021208+0000","last_became_active":"2024-09-10T23:38:51.015026+0000","last_became_peered":"2024-09-10T23:38:51.015026+0000","last_unstale":"2024-09-10T23:38:52.021208+0000","last_undegraded":"2024-09-10T23:38:52.021208+0000","last_fullsized":"2024-09-10T23:38:52.021208+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:25.873667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992710+0000","last_change":"2024-09-10T23:38:50.008309+0000","last_active":"2024-09-10T23:38:51.992710+0000","last_peered":"2024-09-10T23:38:51.992710+0000","last_clean":"2024-09-10T23:38:51.992710+0000","last_became_active":"2024-09-10T23:38:50.008007+0000","last_became_peered":"2024-09-10T23:38:50.008007+0000","last_unstale":"2024-09-10T23:38:51.992710+0000","last_undegraded":"2024-09-10T23:38:51.992710+0000","last_fullsized":"2024-09-10T23:38:51.992710+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:08.919532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993645+0000","last_change":"2024-09-10T23:38:51.007392+0000","last_active":"2024-09-10T23:38:51.993645+0000","last_peered":"2024-09-10T23:38:51.993645+0000","last_clean":"2024-09-10T23:38:51.993645+0000","last_became_active":"2024-09-10T23:38:51.006642+0000","last_became_peered":"2024-09-10T23:38:51.006642+0000","last_unstale":"2024-09-10T23:38:51.993645+0000","last_undegraded":"2024-09-10T23:38:51.993645+0000","last_fullsized":"2024-09-10T23:38:51.993645+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:57.110746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993744+0000","last_change":"2024-09-10T23:38:50.001623+0000","last_active":"2024-09-10T23:38:51.993744+0000","last_peered":"2024-09-10T23:38:51.993744+0000","last_clean":"2024-09-10T23:38:51.993744+0000","last_became_active":"2024-09-10T23:38:50.001315+0000","last_became_peered":"2024-09-10T23:38:50.001315+0000","last_unstale":"2024-09-10T23:38:51.993744+0000","last_undegraded":"2024-09-10T23:38:51.993744+0000","last_fullsized":"2024-09-10T23:38:51.993744+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:42.811340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993661+0000","last_change":"2024-09-10T23:38:51.009829+0000","last_active":"2024-09-10T23:38:51.993661+0000","last_peered":"2024-09-10T23:38:51.993661+0000","last_clean":"2024-09-10T23:38:51.993661+0000","last_became_active":"2024-09-10T23:38:51.009248+0000","last_became_peered":"2024-09-10T23:38:51.009248+0000","last_unstale":"2024-09-10T23:38:51.993661+0000","last_undegraded":"2024-09-10T23:38:51.993661+0000","last_fullsized":"2024-09-10T23:38:51.993661+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:04.701805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"127'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.023515+0000","last_change":"2024-09-10T23:38:50.005377+0000","last_active":"2024-09-10T23:38:52.023515+0000","last_peered":"2024-09-10T23:38:52.023515+0000","last_clean":"2024-09-10T23:38:52.023515+0000","last_became_active":"2024-09-10T23:38:50.005061+0000","last_became_peered":"2024-09-10T23:38:50.005061+0000","last_unstale":"2024-09-10T23:38:52.023515+0000","last_undegraded":"2024-09-10T23:38:52.023515+0000","last_fullsized":"2024-09-10T23:38:52.023515+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:39.375765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995992+0000","last_change":"2024-09-10T23:38:51.006670+0000","last_active":"2024-09-10T23:38:51.995992+0000","last_peered":"2024-09-10T23:38:51.995992+0000","last_clean":"2024-09-10T23:38:51.995992+0000","last_became_active":"2024-09-10T23:38:51.006405+0000","last_became_peered":"2024-09-10T23:38:51.006405+0000","last_unstale":"2024-09-10T23:38:51.995992+0000","last_undegraded":"2024-09-10T23:38:51.995992+0000","last_fullsized":"2024-09-10T23:38:51.995992+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:50.087651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157124+0000","last_change":"2024-09-10T23:38:49.991369+0000","last_active":"2024-09-10T23:38:52.157124+0000","last_peered":"2024-09-10T23:38:52.157124+0000","last_clean":"2024-09-10T23:38:52.157124+0000","last_became_active":"2024-09-10T23:38:49.990807+0000","last_became_peered":"2024-09-10T23:38:49.990807+0000","last_unstale":"2024-09-10T23:38:52.157124+0000","last_undegraded":"2024-09-10T23:38:52.157124+0000","last_fullsized":"2024-09-10T23:38:52.157124+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:34:28.178846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995998+0000","last_change":"2024-09-10T23:38:51.014424+0000","last_active":"2024-09-10T23:38:51.995998+0000","last_peered":"2024-09-10T23:38:51.995998+0000","last_clean":"2024-09-10T23:38:51.995998+0000","last_became_active":"2024-09-10T23:38:51.013953+0000","last_became_peered":"2024-09-10T23:38:51.013953+0000","last_unstale":"2024-09-10T23:38:51.995998+0000","last_undegraded":"2024-09-10T23:38:51.995998+0000","last_fullsized":"2024-09-10T23:38:51.995998+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:34.717688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157041+0000","last_change":"2024-09-10T23:38:49.992314+0000","last_active":"2024-09-10T23:38:52.157041+0000","last_peered":"2024-09-10T23:38:52.157041+0000","last_clean":"2024-09-10T23:38:52.157041+0000","last_became_active":"2024-09-10T23:38:49.992007+0000","last_became_peered":"2024-09-10T23:38:49.992007+0000","last_unstale":"2024-09-10T23:38:52.157041+0000","last_undegraded":"2024-09-10T23:38:52.157041+0000","last_fullsized":"2024-09-10T23:38:52.157041+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:06.441197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156909+0000","last_change":"2024-09-10T23:38:51.022021+0000","last_active":"2024-09-10T23:38:52.156909+0000","last_peered":"2024-09-10T23:38:52.156909+0000","last_clean":"2024-09-10T23:38:52.156909+0000","last_became_active":"2024-09-10T23:38:51.021689+0000","last_became_peered":"2024-09-10T23:38:51.021689+0000","last_unstale":"2024-09-10T23:38:52.156909+0000","last_undegraded":"2024-09-10T23:38:52.156909+0000","last_fullsized":"2024-09-10T23:38:52.156909+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:23.666220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021176+0000","last_change":"2024-09-10T23:38:49.998035+0000","last_active":"2024-09-10T23:38:52.021176+0000","last_peered":"2024-09-10T23:38:52.021176+0000","last_clean":"2024-09-10T23:38:52.021176+0000","last_became_active":"2024-09-10T23:38:49.997429+0000","last_became_peered":"2024-09-10T23:38:49.997429+0000","last_unstale":"2024-09-10T23:38:52.021176+0000","last_undegraded":"2024-09-10T23:38:52.021176+0000","last_fullsized":"2024-09-10T23:38:52.021176+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:18:57.051526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993237+0000","last_change":"2024-09-10T23:38:51.014876+0000","last_active":"2024-09-10T23:38:51.993237+0000","last_peered":"2024-09-10T23:38:51.993237+0000","last_clean":"2024-09-10T23:38:51.993237+0000","last_became_active":"2024-09-10T23:38:51.014594+0000","last_became_peered":"2024-09-10T23:38:51.014594+0000","last_unstale":"2024-09-10T23:38:51.993237+0000","last_undegraded":"2024-09-10T23:38:51.993237+0000","last_fullsized":"2024-09-10T23:38:51.993237+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:21.080920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157241+0000","last_change":"2024-09-10T23:38:49.996415+0000","last_active":"2024-09-10T23:38:52.157241+0000","last_peered":"2024-09-10T23:38:52.157241+0000","last_clean":"2024-09-10T23:38:52.157241+0000","last_became_active":"2024-09-10T23:38:49.996072+0000","last_became_peered":"2024-09-10T23:38:49.996072+0000","last_unstale":"2024-09-10T23:38:52.157241+0000","last_undegraded":"2024-09-10T23:38:52.157241+0000","last_fullsized":"2024-09-10T23:38:52.157241+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:25.597983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992862+0000","last_change":"2024-09-10T23:38:51.009829+0000","last_active":"2024-09-10T23:38:51.992862+0000","last_peered":"2024-09-10T23:38:51.992862+0000","last_clean":"2024-09-10T23:38:51.992862+0000","last_became_active":"2024-09-10T23:38:51.009481+0000","last_became_peered":"2024-09-10T23:38:51.009481+0000","last_unstale":"2024-09-10T23:38:51.992862+0000","last_undegraded":"2024-09-10T23:38:51.992862+0000","last_fullsized":"2024-09-10T23:38:51.992862+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:12:04.685418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156527+0000","last_change":"2024-09-10T23:38:49.996805+0000","last_active":"2024-09-10T23:38:52.156527+0000","last_peered":"2024-09-10T23:38:52.156527+0000","last_clean":"2024-09-10T23:38:52.156527+0000","last_became_active":"2024-09-10T23:38:49.996458+0000","last_became_peered":"2024-09-10T23:38:49.996458+0000","last_unstale":"2024-09-10T23:38:52.156527+0000","last_undegraded":"2024-09-10T23:38:52.156527+0000","last_fullsized":"2024-09-10T23:38:52.156527+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:26.148834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156430+0000","last_change":"2024-09-10T23:38:51.018198+0000","last_active":"2024-09-10T23:38:52.156430+0000","last_peered":"2024-09-10T23:38:52.156430+0000","last_clean":"2024-09-10T23:38:52.156430+0000","last_became_active":"2024-09-10T23:38:51.017893+0000","last_became_peered":"2024-09-10T23:38:51.017893+0000","last_unstale":"2024-09-10T23:38:52.156430+0000","last_undegraded":"2024-09-10T23:38:52.156430+0000","last_fullsized":"2024-09-10T23:38:52.156430+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:33.560706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994035+0000","last_change":"2024-09-10T23:38:49.993427+0000","last_active":"2024-09-10T23:38:51.994035+0000","last_peered":"2024-09-10T23:38:51.994035+0000","last_clean":"2024-09-10T23:38:51.994035+0000","last_became_active":"2024-09-10T23:38:49.992941+0000","last_became_peered":"2024-09-10T23:38:49.992941+0000","last_unstale":"2024-09-10T23:38:51.994035+0000","last_undegraded":"2024-09-10T23:38:51.994035+0000","last_fullsized":"2024-09-10T23:38:51.994035+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:55.592712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157013+0000","last_change":"2024-09-10T23:38:51.011553+0000","last_active":"2024-09-10T23:38:52.157013+0000","last_peered":"2024-09-10T23:38:52.157013+0000","last_clean":"2024-09-10T23:38:52.157013+0000","last_became_active":"2024-09-10T23:38:51.011292+0000","last_became_peered":"2024-09-10T23:38:51.011292+0000","last_unstale":"2024-09-10T23:38:52.157013+0000","last_undegraded":"2024-09-10T23:38:52.157013+0000","last_fullsized":"2024-09-10T23:38:52.157013+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:16.455092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"126'3","reported_seq":18,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156892+0000","last_change":"2024-09-10T23:38:49.997220+0000","last_active":"2024-09-10T23:38:52.156892+0000","last_peered":"2024-09-10T23:38:52.156892+0000","last_clean":"2024-09-10T23:38:52.156892+0000","last_became_active":"2024-09-10T23:38:49.996952+0000","last_became_peered":"2024-09-10T23:38:49.996952+0000","last_unstale":"2024-09-10T23:38:52.156892+0000","last_undegraded":"2024-09-10T23:38:52.156892+0000","last_fullsized":"2024-09-10T23:38:52.156892+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:56.104189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":331,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156490+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:38:52.156490+0000","last_peered":"2024-09-10T23:38:52.156490+0000","last_clean":"2024-09-10T23:38:52.156490+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:38:52.156490+0000","last_undegraded":"2024-09-10T23:38:52.156490+0000","last_fullsized":"2024-09-10T23:38:52.156490+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509522,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1585152,"data_stored":697016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542225,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1585152,"data_stored":697016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509521,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33328,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338512,"statfs":{"total":96636764160,"available":96602636288,"internally_reserved":0,"allocated":2191360,"data_stored":1282862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509519,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1593344,"data_stored":692904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509520,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1593344,"data_stored":697632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509521,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1589248,"data_stored":697038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509520,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33324,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338516,"statfs":{"total":96636764160,"available":96602640384,"internally_reserved":0,"allocated":2183168,"data_stored":1287454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509520,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1576960,"data_stored":690778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:38:55.103 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:38:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.284+0000 7f4703200640 1 -- 172.21.15.94:0/626224639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46fc0cfba0 msgr2=0x7f46fc0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.284+0000 7f4703200640 1 --2- 172.21.15.94:0/626224639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46fc0cfba0 0x7f46fc0b3f20 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009c80 tx=0x7f46f401cce0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.284+0000 7f4703200640 1 -- 172.21.15.94:0/626224639 shutdown_connections 2024-09-10T23:38:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.284+0000 7f4703200640 1 --2- 172.21.15.94:0/626224639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46fc0cfba0 0x7f46fc0b3f20 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.284+0000 7f4703200640 1 --2- 172.21.15.94:0/626224639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46fc073630 0x7f46fc073a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.284+0000 7f4703200640 1 -- 172.21.15.94:0/626224639 >> 172.21.15.94:0/626224639 conn(0x7f46fc06dd30 msgr2=0x7f46fc06e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.284+0000 7f4703200640 1 -- 172.21.15.94:0/626224639 shutdown_connections 2024-09-10T23:38:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.284+0000 7f4703200640 1 -- 172.21.15.94:0/626224639 wait complete. 2024-09-10T23:38:55.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f4703200640 1 Processor -- start 2024-09-10T23:38:55.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f4703200640 1 -- start start 2024-09-10T23:38:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f4703200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46fc073630 0x7f46fc146240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f4703200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46fc0b3ac0 0x7f46fc159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f4703200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46fc0cfba0 0x7f46fc15a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f4703200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc167d50 con 0x7f46fc073630 2024-09-10T23:38:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f4703200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc167bd0 con 0x7f46fc0cfba0 2024-09-10T23:38:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f4703200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f46fc167ed0 con 0x7f46fc0b3ac0 2024-09-10T23:38:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46fc073630 0x7f46fc146240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46fc073630 0x7f46fc146240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55716/0 (socket says 172.21.15.94:55716) 2024-09-10T23:38:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f46fbe00640 1 -- 172.21.15.94:0/1629436523 learned_addr learned my addr 172.21.15.94:0/1629436523 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f46fb400640 1 --2- 172.21.15.94:0/1629436523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46fc0b3ac0 0x7f46fc159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f4701200640 1 --2- 172.21.15.94:0/1629436523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46fc0cfba0 0x7f46fc15a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f46fbe00640 1 -- 172.21.15.94:0/1629436523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46fc0b3ac0 msgr2=0x7f46fc159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f46fbe00640 1 --2- 172.21.15.94:0/1629436523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46fc0b3ac0 0x7f46fc159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f46fbe00640 1 -- 172.21.15.94:0/1629436523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46fc0cfba0 msgr2=0x7f46fc15a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f46fbe00640 1 --2- 172.21.15.94:0/1629436523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46fc0cfba0 0x7f46fc15a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f46fbe00640 1 -- 172.21.15.94:0/1629436523 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc1f8bd0 con 0x7f46fc073630 2024-09-10T23:38:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f46fbe00640 1 --2- 172.21.15.94:0/1629436523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46fc073630 0x7f46fc146240 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00c9f0 tx=0x7f46ec00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f46f3e00640 1 -- 172.21.15.94:0/1629436523 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec007bb0 con 0x7f46fc073630 2024-09-10T23:38:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f46f3e00640 1 -- 172.21.15.94:0/1629436523 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46ec007e40 con 0x7f46fc073630 2024-09-10T23:38:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f46f3e00640 1 -- 172.21.15.94:0/1629436523 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec00f690 con 0x7f46fc073630 2024-09-10T23:38:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f4703200640 1 -- 172.21.15.94:0/1629436523 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46fc146780 con 0x7f46fc073630 2024-09-10T23:38:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.288+0000 7f4703200640 1 -- 172.21.15.94:0/1629436523 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46fc1f80f0 con 0x7f46fc073630 2024-09-10T23:38:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.292+0000 7f4703200640 1 -- 172.21.15.94:0/1629436523 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46fc073a10 con 0x7f46fc073630 2024-09-10T23:38:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.292+0000 7f46f3e00640 1 -- 172.21.15.94:0/1629436523 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f46ec00f830 con 0x7f46fc073630 2024-09-10T23:38:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.292+0000 7f46f3e00640 1 --2- 172.21.15.94:0/1629436523 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f46dc072c10 0x7f46dc0750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.292+0000 7f46f3e00640 1 -- 172.21.15.94:0/1629436523 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f46ec0946a0 con 0x7f46fc073630 2024-09-10T23:38:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.292+0000 7f46fb400640 1 --2- 172.21.15.94:0/1629436523 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f46dc072c10 0x7f46dc0750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.292+0000 7f46fb400640 1 --2- 172.21.15.94:0/1629436523 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f46dc072c10 0x7f46dc0750d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f46fc14fcb0 tx=0x7f46e8009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.296+0000 7f46f3e00640 1 -- 172.21.15.94:0/1629436523 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f46ec014030 con 0x7f46fc073630 2024-09-10T23:38:55.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.456+0000 7f4703200640 1 -- 172.21.15.94:0/1629436523 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46fc0cce60 con 0x7f46dc072c10 2024-09-10T23:38:55.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.460+0000 7f46f3e00640 1 -- 172.21.15.94:0/1629436523 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252474 (secure 0 0 0) 0x7f46fc0cce60 con 0x7f46dc072c10 2024-09-10T23:38:55.467 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:38:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.464+0000 7f4703200640 1 -- 172.21.15.94:0/1629436523 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f46dc072c10 msgr2=0x7f46dc0750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.464+0000 7f4703200640 1 --2- 172.21.15.94:0/1629436523 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f46dc072c10 0x7f46dc0750d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f46fc14fcb0 tx=0x7f46e8009040 comp rx=0 tx=0).stop 2024-09-10T23:38:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.464+0000 7f4703200640 1 -- 172.21.15.94:0/1629436523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46fc073630 msgr2=0x7f46fc146240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.464+0000 7f4703200640 1 --2- 172.21.15.94:0/1629436523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46fc073630 0x7f46fc146240 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00c9f0 tx=0x7f46ec00ceb0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.464+0000 7f4703200640 1 -- 172.21.15.94:0/1629436523 shutdown_connections 2024-09-10T23:38:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.464+0000 7f4703200640 1 --2- 172.21.15.94:0/1629436523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46fc0cfba0 0x7f46fc15a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.464+0000 7f4703200640 1 --2- 172.21.15.94:0/1629436523 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f46dc072c10 0x7f46dc0750d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.464+0000 7f4703200640 1 --2- 172.21.15.94:0/1629436523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46fc0b3ac0 0x7f46fc159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.464+0000 7f4703200640 1 --2- 172.21.15.94:0/1629436523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46fc073630 0x7f46fc146240 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.464+0000 7f4703200640 1 -- 172.21.15.94:0/1629436523 >> 172.21.15.94:0/1629436523 conn(0x7f46fc06dd30 msgr2=0x7f46fc151170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.464+0000 7f4703200640 1 -- 172.21.15.94:0/1629436523 shutdown_connections 2024-09-10T23:38:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.464+0000 7f4703200640 1 -- 172.21.15.94:0/1629436523 wait complete. 2024-09-10T23:38:55.487 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-10T23:38:54.763665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263100,"kb_used_data":13572,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711620,"statfs":{"total":773094113280,"available":772824698880,"internally_reserved":0,"allocated":13897728,"data_stored":6742700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.791320"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.996139+0000","last_change":"2024-09-10T23:38:51.018222+0000","last_active":"2024-09-10T23:38:51.996139+0000","last_peered":"2024-09-10T23:38:51.996139+0000","last_clean":"2024-09-10T23:38:51.996139+0000","last_became_active":"2024-09-10T23:38:51.017759+0000","last_became_peered":"2024-09-10T23:38:51.017759+0000","last_unstale":"2024-09-10T23:38:51.996139+0000","last_undegraded":"2024-09-10T23:38:51.996139+0000","last_fullsized":"2024-09-10T23:38:51.996139+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:45:47.988677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021773+0000","last_change":"2024-09-10T23:38:51.020141+0000","last_active":"2024-09-10T23:38:52.021773+0000","last_peered":"2024-09-10T23:38:52.021773+0000","last_clean":"2024-09-10T23:38:52.021773+0000","last_became_active":"2024-09-10T23:38:51.019862+0000","last_became_peered":"2024-09-10T23:38:51.019862+0000","last_unstale":"2024-09-10T23:38:52.021773+0000","last_undegraded":"2024-09-10T23:38:52.021773+0000","last_fullsized":"2024-09-10T23:38:52.021773+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:56.196878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156942+0000","last_change":"2024-09-10T23:38:51.010558+0000","last_active":"2024-09-10T23:38:52.156942+0000","last_peered":"2024-09-10T23:38:52.156942+0000","last_clean":"2024-09-10T23:38:52.156942+0000","last_became_active":"2024-09-10T23:38:51.010245+0000","last_became_peered":"2024-09-10T23:38:51.010245+0000","last_unstale":"2024-09-10T23:38:52.156942+0000","last_undegraded":"2024-09-10T23:38:52.156942+0000","last_fullsized":"2024-09-10T23:38:52.156942+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:38.373318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993275+0000","last_change":"2024-09-10T23:38:51.009845+0000","last_active":"2024-09-10T23:38:51.993275+0000","last_peered":"2024-09-10T23:38:51.993275+0000","last_clean":"2024-09-10T23:38:51.993275+0000","last_became_active":"2024-09-10T23:38:51.009573+0000","last_became_peered":"2024-09-10T23:38:51.009573+0000","last_unstale":"2024-09-10T23:38:51.993275+0000","last_undegraded":"2024-09-10T23:38:51.993275+0000","last_fullsized":"2024-09-10T23:38:51.993275+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:07.447754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157626+0000","last_change":"2024-09-10T23:38:51.013311+0000","last_active":"2024-09-10T23:38:52.157626+0000","last_peered":"2024-09-10T23:38:52.157626+0000","last_clean":"2024-09-10T23:38:52.157626+0000","last_became_active":"2024-09-10T23:38:51.012608+0000","last_became_peered":"2024-09-10T23:38:51.012608+0000","last_unstale":"2024-09-10T23:38:52.157626+0000","last_undegraded":"2024-09-10T23:38:52.157626+0000","last_fullsized":"2024-09-10T23:38:52.157626+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:09.926090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994984+0000","last_change":"2024-09-10T23:38:51.003749+0000","last_active":"2024-09-10T23:38:51.994984+0000","last_peered":"2024-09-10T23:38:51.994984+0000","last_clean":"2024-09-10T23:38:51.994984+0000","last_became_active":"2024-09-10T23:38:51.003413+0000","last_became_peered":"2024-09-10T23:38:51.003413+0000","last_unstale":"2024-09-10T23:38:51.994984+0000","last_undegraded":"2024-09-10T23:38:51.994984+0000","last_fullsized":"2024-09-10T23:38:51.994984+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:06.111238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156576+0000","last_change":"2024-09-10T23:38:51.017303+0000","last_active":"2024-09-10T23:38:52.156576+0000","last_peered":"2024-09-10T23:38:52.156576+0000","last_clean":"2024-09-10T23:38:52.156576+0000","last_became_active":"2024-09-10T23:38:51.016992+0000","last_became_peered":"2024-09-10T23:38:51.016992+0000","last_unstale":"2024-09-10T23:38:52.156576+0000","last_undegraded":"2024-09-10T23:38:52.156576+0000","last_fullsized":"2024-09-10T23:38:52.156576+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:13.966506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994448+0000","last_change":"2024-09-10T23:38:50.997375+0000","last_active":"2024-09-10T23:38:51.994448+0000","last_peered":"2024-09-10T23:38:51.994448+0000","last_clean":"2024-09-10T23:38:51.994448+0000","last_became_active":"2024-09-10T23:38:50.996786+0000","last_became_peered":"2024-09-10T23:38:50.996786+0000","last_unstale":"2024-09-10T23:38:51.994448+0000","last_undegraded":"2024-09-10T23:38:51.994448+0000","last_fullsized":"2024-09-10T23:38:51.994448+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:09.062977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995563+0000","last_change":"2024-09-10T23:38:51.005819+0000","last_active":"2024-09-10T23:38:51.995563+0000","last_peered":"2024-09-10T23:38:51.995563+0000","last_clean":"2024-09-10T23:38:51.995563+0000","last_became_active":"2024-09-10T23:38:51.005475+0000","last_became_peered":"2024-09-10T23:38:51.005475+0000","last_unstale":"2024-09-10T23:38:51.995563+0000","last_undegraded":"2024-09-10T23:38:51.995563+0000","last_fullsized":"2024-09-10T23:38:51.995563+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:34.802364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157147+0000","last_change":"2024-09-10T23:38:51.022181+0000","last_active":"2024-09-10T23:38:52.157147+0000","last_peered":"2024-09-10T23:38:52.157147+0000","last_clean":"2024-09-10T23:38:52.157147+0000","last_became_active":"2024-09-10T23:38:51.021871+0000","last_became_peered":"2024-09-10T23:38:51.021871+0000","last_unstale":"2024-09-10T23:38:52.157147+0000","last_undegraded":"2024-09-10T23:38:52.157147+0000","last_fullsized":"2024-09-10T23:38:52.157147+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.322453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995295+0000","last_change":"2024-09-10T23:38:51.008680+0000","last_active":"2024-09-10T23:38:51.995295+0000","last_peered":"2024-09-10T23:38:51.995295+0000","last_clean":"2024-09-10T23:38:51.995295+0000","last_became_active":"2024-09-10T23:38:51.008356+0000","last_became_peered":"2024-09-10T23:38:51.008356+0000","last_unstale":"2024-09-10T23:38:51.995295+0000","last_undegraded":"2024-09-10T23:38:51.995295+0000","last_fullsized":"2024-09-10T23:38:51.995295+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:37.993279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157244+0000","last_change":"2024-09-10T23:38:51.011435+0000","last_active":"2024-09-10T23:38:52.157244+0000","last_peered":"2024-09-10T23:38:52.157244+0000","last_clean":"2024-09-10T23:38:52.157244+0000","last_became_active":"2024-09-10T23:38:51.011077+0000","last_became_peered":"2024-09-10T23:38:51.011077+0000","last_unstale":"2024-09-10T23:38:52.157244+0000","last_undegraded":"2024-09-10T23:38:52.157244+0000","last_fullsized":"2024-09-10T23:38:52.157244+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:00.419380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157836+0000","last_change":"2024-09-10T23:38:51.013585+0000","last_active":"2024-09-10T23:38:52.157836+0000","last_peered":"2024-09-10T23:38:52.157836+0000","last_clean":"2024-09-10T23:38:52.157836+0000","last_became_active":"2024-09-10T23:38:51.013002+0000","last_became_peered":"2024-09-10T23:38:51.013002+0000","last_unstale":"2024-09-10T23:38:52.157836+0000","last_undegraded":"2024-09-10T23:38:52.157836+0000","last_fullsized":"2024-09-10T23:38:52.157836+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:51.674266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157365+0000","last_change":"2024-09-10T23:38:51.003902+0000","last_active":"2024-09-10T23:38:52.157365+0000","last_peered":"2024-09-10T23:38:52.157365+0000","last_clean":"2024-09-10T23:38:52.157365+0000","last_became_active":"2024-09-10T23:38:51.003591+0000","last_became_peered":"2024-09-10T23:38:51.003591+0000","last_unstale":"2024-09-10T23:38:52.157365+0000","last_undegraded":"2024-09-10T23:38:52.157365+0000","last_fullsized":"2024-09-10T23:38:52.157365+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:19.795894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156610+0000","last_change":"2024-09-10T23:38:50.999932+0000","last_active":"2024-09-10T23:38:52.156610+0000","last_peered":"2024-09-10T23:38:52.156610+0000","last_clean":"2024-09-10T23:38:52.156610+0000","last_became_active":"2024-09-10T23:38:50.999458+0000","last_became_peered":"2024-09-10T23:38:50.999458+0000","last_unstale":"2024-09-10T23:38:52.156610+0000","last_undegraded":"2024-09-10T23:38:52.156610+0000","last_fullsized":"2024-09-10T23:38:52.156610+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:11.526399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156620+0000","last_change":"2024-09-10T23:38:51.010919+0000","last_active":"2024-09-10T23:38:52.156620+0000","last_peered":"2024-09-10T23:38:52.156620+0000","last_clean":"2024-09-10T23:38:52.156620+0000","last_became_active":"2024-09-10T23:38:51.010626+0000","last_became_peered":"2024-09-10T23:38:51.010626+0000","last_unstale":"2024-09-10T23:38:52.156620+0000","last_undegraded":"2024-09-10T23:38:52.156620+0000","last_fullsized":"2024-09-10T23:38:52.156620+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:36.508752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157809+0000","last_change":"2024-09-10T23:38:51.017242+0000","last_active":"2024-09-10T23:38:52.157809+0000","last_peered":"2024-09-10T23:38:52.157809+0000","last_clean":"2024-09-10T23:38:52.157809+0000","last_became_active":"2024-09-10T23:38:51.016989+0000","last_became_peered":"2024-09-10T23:38:51.016989+0000","last_unstale":"2024-09-10T23:38:52.157809+0000","last_undegraded":"2024-09-10T23:38:52.157809+0000","last_fullsized":"2024-09-10T23:38:52.157809+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:10:16.219440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.158133+0000","last_change":"2024-09-10T23:38:51.008074+0000","last_active":"2024-09-10T23:38:52.158133+0000","last_peered":"2024-09-10T23:38:52.158133+0000","last_clean":"2024-09-10T23:38:52.158133+0000","last_became_active":"2024-09-10T23:38:51.006439+0000","last_became_peered":"2024-09-10T23:38:51.006439+0000","last_unstale":"2024-09-10T23:38:52.158133+0000","last_undegraded":"2024-09-10T23:38:52.158133+0000","last_fullsized":"2024-09-10T23:38:52.158133+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:43.817898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995782+0000","last_change":"2024-09-10T23:38:50.998685+0000","last_active":"2024-09-10T23:38:51.995782+0000","last_peered":"2024-09-10T23:38:51.995782+0000","last_clean":"2024-09-10T23:38:51.995782+0000","last_became_active":"2024-09-10T23:38:50.998378+0000","last_became_peered":"2024-09-10T23:38:50.998378+0000","last_unstale":"2024-09-10T23:38:51.995782+0000","last_undegraded":"2024-09-10T23:38:51.995782+0000","last_fullsized":"2024-09-10T23:38:51.995782+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:45.667770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995757+0000","last_change":"2024-09-10T23:38:51.007136+0000","last_active":"2024-09-10T23:38:51.995757+0000","last_peered":"2024-09-10T23:38:51.995757+0000","last_clean":"2024-09-10T23:38:51.995757+0000","last_became_active":"2024-09-10T23:38:51.006882+0000","last_became_peered":"2024-09-10T23:38:51.006882+0000","last_unstale":"2024-09-10T23:38:51.995757+0000","last_undegraded":"2024-09-10T23:38:51.995757+0000","last_fullsized":"2024-09-10T23:38:51.995757+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:33.127810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157618+0000","last_change":"2024-09-10T23:38:51.016399+0000","last_active":"2024-09-10T23:38:52.157618+0000","last_peered":"2024-09-10T23:38:52.157618+0000","last_clean":"2024-09-10T23:38:52.157618+0000","last_became_active":"2024-09-10T23:38:51.016092+0000","last_became_peered":"2024-09-10T23:38:51.016092+0000","last_unstale":"2024-09-10T23:38:52.157618+0000","last_undegraded":"2024-09-10T23:38:52.157618+0000","last_fullsized":"2024-09-10T23:38:52.157618+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:10.998657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156905+0000","last_change":"2024-09-10T23:38:51.013935+0000","last_active":"2024-09-10T23:38:52.156905+0000","last_peered":"2024-09-10T23:38:52.156905+0000","last_clean":"2024-09-10T23:38:52.156905+0000","last_became_active":"2024-09-10T23:38:51.012090+0000","last_became_peered":"2024-09-10T23:38:51.012090+0000","last_unstale":"2024-09-10T23:38:52.156905+0000","last_undegraded":"2024-09-10T23:38:52.156905+0000","last_fullsized":"2024-09-10T23:38:52.156905+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:35.176499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992714+0000","last_change":"2024-09-10T23:38:51.014060+0000","last_active":"2024-09-10T23:38:51.992714+0000","last_peered":"2024-09-10T23:38:51.992714+0000","last_clean":"2024-09-10T23:38:51.992714+0000","last_became_active":"2024-09-10T23:38:51.013517+0000","last_became_peered":"2024-09-10T23:38:51.013517+0000","last_unstale":"2024-09-10T23:38:51.992714+0000","last_undegraded":"2024-09-10T23:38:51.992714+0000","last_fullsized":"2024-09-10T23:38:51.992714+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:10.057833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156429+0000","last_change":"2024-09-10T23:38:51.001717+0000","last_active":"2024-09-10T23:38:52.156429+0000","last_peered":"2024-09-10T23:38:52.156429+0000","last_clean":"2024-09-10T23:38:52.156429+0000","last_became_active":"2024-09-10T23:38:51.000793+0000","last_became_peered":"2024-09-10T23:38:51.000793+0000","last_unstale":"2024-09-10T23:38:52.156429+0000","last_undegraded":"2024-09-10T23:38:52.156429+0000","last_fullsized":"2024-09-10T23:38:52.156429+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:07.790657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021721+0000","last_change":"2024-09-10T23:38:51.013055+0000","last_active":"2024-09-10T23:38:52.021721+0000","last_peered":"2024-09-10T23:38:52.021721+0000","last_clean":"2024-09-10T23:38:52.021721+0000","last_became_active":"2024-09-10T23:38:51.012769+0000","last_became_peered":"2024-09-10T23:38:51.012769+0000","last_unstale":"2024-09-10T23:38:52.021721+0000","last_undegraded":"2024-09-10T23:38:52.021721+0000","last_fullsized":"2024-09-10T23:38:52.021721+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:29.118023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.996104+0000","last_change":"2024-09-10T23:38:51.007965+0000","last_active":"2024-09-10T23:38:51.996104+0000","last_peered":"2024-09-10T23:38:51.996104+0000","last_clean":"2024-09-10T23:38:51.996104+0000","last_became_active":"2024-09-10T23:38:51.006185+0000","last_became_peered":"2024-09-10T23:38:51.006185+0000","last_unstale":"2024-09-10T23:38:51.996104+0000","last_undegraded":"2024-09-10T23:38:51.996104+0000","last_fullsized":"2024-09-10T23:38:51.996104+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:47.762556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021374+0000","last_change":"2024-09-10T23:38:51.011470+0000","last_active":"2024-09-10T23:38:52.021374+0000","last_peered":"2024-09-10T23:38:52.021374+0000","last_clean":"2024-09-10T23:38:52.021374+0000","last_became_active":"2024-09-10T23:38:51.011224+0000","last_became_peered":"2024-09-10T23:38:51.011224+0000","last_unstale":"2024-09-10T23:38:52.021374+0000","last_undegraded":"2024-09-10T23:38:52.021374+0000","last_fullsized":"2024-09-10T23:38:52.021374+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:06.816076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021675+0000","last_change":"2024-09-10T23:38:51.019262+0000","last_active":"2024-09-10T23:38:52.021675+0000","last_peered":"2024-09-10T23:38:52.021675+0000","last_clean":"2024-09-10T23:38:52.021675+0000","last_became_active":"2024-09-10T23:38:51.018945+0000","last_became_peered":"2024-09-10T23:38:51.018945+0000","last_unstale":"2024-09-10T23:38:52.021675+0000","last_undegraded":"2024-09-10T23:38:52.021675+0000","last_fullsized":"2024-09-10T23:38:52.021675+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:40.555309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994782+0000","last_change":"2024-09-10T23:38:51.012443+0000","last_active":"2024-09-10T23:38:51.994782+0000","last_peered":"2024-09-10T23:38:51.994782+0000","last_clean":"2024-09-10T23:38:51.994782+0000","last_became_active":"2024-09-10T23:38:51.011778+0000","last_became_peered":"2024-09-10T23:38:51.011778+0000","last_unstale":"2024-09-10T23:38:51.994782+0000","last_undegraded":"2024-09-10T23:38:51.994782+0000","last_fullsized":"2024-09-10T23:38:51.994782+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:03.303492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157109+0000","last_change":"2024-09-10T23:38:51.011728+0000","last_active":"2024-09-10T23:38:52.157109+0000","last_peered":"2024-09-10T23:38:52.157109+0000","last_clean":"2024-09-10T23:38:52.157109+0000","last_became_active":"2024-09-10T23:38:51.011458+0000","last_became_peered":"2024-09-10T23:38:51.011458+0000","last_unstale":"2024-09-10T23:38:52.157109+0000","last_undegraded":"2024-09-10T23:38:52.157109+0000","last_fullsized":"2024-09-10T23:38:52.157109+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:26.219894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021225+0000","last_change":"2024-09-10T23:38:51.015961+0000","last_active":"2024-09-10T23:38:52.021225+0000","last_peered":"2024-09-10T23:38:52.021225+0000","last_clean":"2024-09-10T23:38:52.021225+0000","last_became_active":"2024-09-10T23:38:51.015411+0000","last_became_peered":"2024-09-10T23:38:51.015411+0000","last_unstale":"2024-09-10T23:38:52.021225+0000","last_undegraded":"2024-09-10T23:38:52.021225+0000","last_fullsized":"2024-09-10T23:38:52.021225+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:12.487250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021548+0000","last_change":"2024-09-10T23:38:51.013266+0000","last_active":"2024-09-10T23:38:52.021548+0000","last_peered":"2024-09-10T23:38:52.021548+0000","last_clean":"2024-09-10T23:38:52.021548+0000","last_became_active":"2024-09-10T23:38:51.012935+0000","last_became_peered":"2024-09-10T23:38:51.012935+0000","last_unstale":"2024-09-10T23:38:52.021548+0000","last_undegraded":"2024-09-10T23:38:52.021548+0000","last_fullsized":"2024-09-10T23:38:52.021548+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:50.487537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.158176+0000","last_change":"2024-09-10T23:38:51.015320+0000","last_active":"2024-09-10T23:38:52.158176+0000","last_peered":"2024-09-10T23:38:52.158176+0000","last_clean":"2024-09-10T23:38:52.158176+0000","last_became_active":"2024-09-10T23:38:51.014187+0000","last_became_peered":"2024-09-10T23:38:51.014187+0000","last_unstale":"2024-09-10T23:38:52.158176+0000","last_undegraded":"2024-09-10T23:38:52.158176+0000","last_fullsized":"2024-09-10T23:38:52.158176+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:45.910996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"126'4","reported_seq":19,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157753+0000","last_change":"2024-09-10T23:38:50.001083+0000","last_active":"2024-09-10T23:38:52.157753+0000","last_peered":"2024-09-10T23:38:52.157753+0000","last_clean":"2024-09-10T23:38:52.157753+0000","last_became_active":"2024-09-10T23:38:50.000792+0000","last_became_peered":"2024-09-10T23:38:50.000792+0000","last_unstale":"2024-09-10T23:38:52.157753+0000","last_undegraded":"2024-09-10T23:38:52.157753+0000","last_fullsized":"2024-09-10T23:38:52.157753+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:09.107698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157163+0000","last_change":"2024-09-10T23:38:51.011028+0000","last_active":"2024-09-10T23:38:52.157163+0000","last_peered":"2024-09-10T23:38:52.157163+0000","last_clean":"2024-09-10T23:38:52.157163+0000","last_became_active":"2024-09-10T23:38:51.010772+0000","last_became_peered":"2024-09-10T23:38:51.010772+0000","last_unstale":"2024-09-10T23:38:52.157163+0000","last_undegraded":"2024-09-10T23:38:52.157163+0000","last_fullsized":"2024-09-10T23:38:52.157163+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:31.010655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995480+0000","last_change":"2024-09-10T23:38:49.996561+0000","last_active":"2024-09-10T23:38:51.995480+0000","last_peered":"2024-09-10T23:38:51.995480+0000","last_clean":"2024-09-10T23:38:51.995480+0000","last_became_active":"2024-09-10T23:38:49.996215+0000","last_became_peered":"2024-09-10T23:38:49.996215+0000","last_unstale":"2024-09-10T23:38:51.995480+0000","last_undegraded":"2024-09-10T23:38:51.995480+0000","last_fullsized":"2024-09-10T23:38:51.995480+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:19.142799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994781+0000","last_change":"2024-09-10T23:38:51.003624+0000","last_active":"2024-09-10T23:38:51.994781+0000","last_peered":"2024-09-10T23:38:51.994781+0000","last_clean":"2024-09-10T23:38:51.994781+0000","last_became_active":"2024-09-10T23:38:51.003342+0000","last_became_peered":"2024-09-10T23:38:51.003342+0000","last_unstale":"2024-09-10T23:38:51.994781+0000","last_undegraded":"2024-09-10T23:38:51.994781+0000","last_fullsized":"2024-09-10T23:38:51.994781+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:35.345357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157004+0000","last_change":"2024-09-10T23:38:49.998832+0000","last_active":"2024-09-10T23:38:52.157004+0000","last_peered":"2024-09-10T23:38:52.157004+0000","last_clean":"2024-09-10T23:38:52.157004+0000","last_became_active":"2024-09-10T23:38:49.998579+0000","last_became_peered":"2024-09-10T23:38:49.998579+0000","last_unstale":"2024-09-10T23:38:52.157004+0000","last_undegraded":"2024-09-10T23:38:52.157004+0000","last_fullsized":"2024-09-10T23:38:52.157004+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:56.126627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993427+0000","last_change":"2024-09-10T23:38:51.019178+0000","last_active":"2024-09-10T23:38:51.993427+0000","last_peered":"2024-09-10T23:38:51.993427+0000","last_clean":"2024-09-10T23:38:51.993427+0000","last_became_active":"2024-09-10T23:38:51.018907+0000","last_became_peered":"2024-09-10T23:38:51.018907+0000","last_unstale":"2024-09-10T23:38:51.993427+0000","last_undegraded":"2024-09-10T23:38:51.993427+0000","last_fullsized":"2024-09-10T23:38:51.993427+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:07.963755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157502+0000","last_change":"2024-09-10T23:38:50.000573+0000","last_active":"2024-09-10T23:38:52.157502+0000","last_peered":"2024-09-10T23:38:52.157502+0000","last_clean":"2024-09-10T23:38:52.157502+0000","last_became_active":"2024-09-10T23:38:50.000255+0000","last_became_peered":"2024-09-10T23:38:50.000255+0000","last_unstale":"2024-09-10T23:38:52.157502+0000","last_undegraded":"2024-09-10T23:38:52.157502+0000","last_fullsized":"2024-09-10T23:38:52.157502+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:12.020186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021760+0000","last_change":"2024-09-10T23:38:51.015563+0000","last_active":"2024-09-10T23:38:52.021760+0000","last_peered":"2024-09-10T23:38:52.021760+0000","last_clean":"2024-09-10T23:38:52.021760+0000","last_became_active":"2024-09-10T23:38:51.015282+0000","last_became_peered":"2024-09-10T23:38:51.015282+0000","last_unstale":"2024-09-10T23:38:52.021760+0000","last_undegraded":"2024-09-10T23:38:52.021760+0000","last_fullsized":"2024-09-10T23:38:52.021760+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:58.764114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994925+0000","last_change":"2024-09-10T23:38:49.984678+0000","last_active":"2024-09-10T23:38:51.994925+0000","last_peered":"2024-09-10T23:38:51.994925+0000","last_clean":"2024-09-10T23:38:51.994925+0000","last_became_active":"2024-09-10T23:38:49.984330+0000","last_became_peered":"2024-09-10T23:38:49.984330+0000","last_unstale":"2024-09-10T23:38:51.994925+0000","last_undegraded":"2024-09-10T23:38:51.994925+0000","last_fullsized":"2024-09-10T23:38:51.994925+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:08:33.519585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993109+0000","last_change":"2024-09-10T23:38:51.012311+0000","last_active":"2024-09-10T23:38:51.993109+0000","last_peered":"2024-09-10T23:38:51.993109+0000","last_clean":"2024-09-10T23:38:51.993109+0000","last_became_active":"2024-09-10T23:38:51.011809+0000","last_became_peered":"2024-09-10T23:38:51.011809+0000","last_unstale":"2024-09-10T23:38:51.993109+0000","last_undegraded":"2024-09-10T23:38:51.993109+0000","last_fullsized":"2024-09-10T23:38:51.993109+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:20.095934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156832+0000","last_change":"2024-09-10T23:38:49.994554+0000","last_active":"2024-09-10T23:38:52.156832+0000","last_peered":"2024-09-10T23:38:52.156832+0000","last_clean":"2024-09-10T23:38:52.156832+0000","last_became_active":"2024-09-10T23:38:49.994268+0000","last_became_peered":"2024-09-10T23:38:49.994268+0000","last_unstale":"2024-09-10T23:38:52.156832+0000","last_undegraded":"2024-09-10T23:38:52.156832+0000","last_fullsized":"2024-09-10T23:38:52.156832+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:03.005349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021568+0000","last_change":"2024-09-10T23:38:51.019154+0000","last_active":"2024-09-10T23:38:52.021568+0000","last_peered":"2024-09-10T23:38:52.021568+0000","last_clean":"2024-09-10T23:38:52.021568+0000","last_became_active":"2024-09-10T23:38:51.018782+0000","last_became_peered":"2024-09-10T23:38:51.018782+0000","last_unstale":"2024-09-10T23:38:52.021568+0000","last_undegraded":"2024-09-10T23:38:52.021568+0000","last_fullsized":"2024-09-10T23:38:52.021568+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:22.736229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994445+0000","last_change":"2024-09-10T23:38:49.997027+0000","last_active":"2024-09-10T23:38:51.994445+0000","last_peered":"2024-09-10T23:38:51.994445+0000","last_clean":"2024-09-10T23:38:51.994445+0000","last_became_active":"2024-09-10T23:38:49.996716+0000","last_became_peered":"2024-09-10T23:38:49.996716+0000","last_unstale":"2024-09-10T23:38:51.994445+0000","last_undegraded":"2024-09-10T23:38:51.994445+0000","last_fullsized":"2024-09-10T23:38:51.994445+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:21.154567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992856+0000","last_change":"2024-09-10T23:38:51.017366+0000","last_active":"2024-09-10T23:38:51.992856+0000","last_peered":"2024-09-10T23:38:51.992856+0000","last_clean":"2024-09-10T23:38:51.992856+0000","last_became_active":"2024-09-10T23:38:51.014336+0000","last_became_peered":"2024-09-10T23:38:51.014336+0000","last_unstale":"2024-09-10T23:38:51.992856+0000","last_undegraded":"2024-09-10T23:38:51.992856+0000","last_fullsized":"2024-09-10T23:38:51.992856+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:49.821245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156212+0000","last_change":"2024-09-10T23:38:50.007729+0000","last_active":"2024-09-10T23:38:52.156212+0000","last_peered":"2024-09-10T23:38:52.156212+0000","last_clean":"2024-09-10T23:38:52.156212+0000","last_became_active":"2024-09-10T23:38:50.007415+0000","last_became_peered":"2024-09-10T23:38:50.007415+0000","last_unstale":"2024-09-10T23:38:52.156212+0000","last_undegraded":"2024-09-10T23:38:52.156212+0000","last_fullsized":"2024-09-10T23:38:52.156212+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:17.415987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993585+0000","last_change":"2024-09-10T23:38:51.009272+0000","last_active":"2024-09-10T23:38:51.993585+0000","last_peered":"2024-09-10T23:38:51.993585+0000","last_clean":"2024-09-10T23:38:51.993585+0000","last_became_active":"2024-09-10T23:38:51.009006+0000","last_became_peered":"2024-09-10T23:38:51.009006+0000","last_unstale":"2024-09-10T23:38:51.993585+0000","last_undegraded":"2024-09-10T23:38:51.993585+0000","last_fullsized":"2024-09-10T23:38:51.993585+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:27.155391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995673+0000","last_change":"2024-09-10T23:38:49.990274+0000","last_active":"2024-09-10T23:38:51.995673+0000","last_peered":"2024-09-10T23:38:51.995673+0000","last_clean":"2024-09-10T23:38:51.995673+0000","last_became_active":"2024-09-10T23:38:49.989988+0000","last_became_peered":"2024-09-10T23:38:49.989988+0000","last_unstale":"2024-09-10T23:38:51.995673+0000","last_undegraded":"2024-09-10T23:38:51.995673+0000","last_fullsized":"2024-09-10T23:38:51.995673+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:19:55.106626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156858+0000","last_change":"2024-09-10T23:38:51.005665+0000","last_active":"2024-09-10T23:38:52.156858+0000","last_peered":"2024-09-10T23:38:52.156858+0000","last_clean":"2024-09-10T23:38:52.156858+0000","last_became_active":"2024-09-10T23:38:51.005389+0000","last_became_peered":"2024-09-10T23:38:51.005389+0000","last_unstale":"2024-09-10T23:38:52.156858+0000","last_undegraded":"2024-09-10T23:38:52.156858+0000","last_fullsized":"2024-09-10T23:38:52.156858+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:58.218590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.158268+0000","last_change":"2024-09-10T23:38:49.991719+0000","last_active":"2024-09-10T23:38:52.158268+0000","last_peered":"2024-09-10T23:38:52.158268+0000","last_clean":"2024-09-10T23:38:52.158268+0000","last_became_active":"2024-09-10T23:38:49.991454+0000","last_became_peered":"2024-09-10T23:38:49.991454+0000","last_unstale":"2024-09-10T23:38:52.158268+0000","last_undegraded":"2024-09-10T23:38:52.158268+0000","last_fullsized":"2024-09-10T23:38:52.158268+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:05.922315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157174+0000","last_change":"2024-09-10T23:38:51.008649+0000","last_active":"2024-09-10T23:38:52.157174+0000","last_peered":"2024-09-10T23:38:52.157174+0000","last_clean":"2024-09-10T23:38:52.157174+0000","last_became_active":"2024-09-10T23:38:51.008339+0000","last_became_peered":"2024-09-10T23:38:51.008339+0000","last_unstale":"2024-09-10T23:38:52.157174+0000","last_undegraded":"2024-09-10T23:38:52.157174+0000","last_fullsized":"2024-09-10T23:38:52.157174+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:19.004763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"126'6","reported_seq":21,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156876+0000","last_change":"2024-09-10T23:38:49.994548+0000","last_active":"2024-09-10T23:38:52.156876+0000","last_peered":"2024-09-10T23:38:52.156876+0000","last_clean":"2024-09-10T23:38:52.156876+0000","last_became_active":"2024-09-10T23:38:49.994202+0000","last_became_peered":"2024-09-10T23:38:49.994202+0000","last_unstale":"2024-09-10T23:38:52.156876+0000","last_undegraded":"2024-09-10T23:38:52.156876+0000","last_fullsized":"2024-09-10T23:38:52.156876+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:15.911264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992743+0000","last_change":"2024-09-10T23:38:51.014250+0000","last_active":"2024-09-10T23:38:51.992743+0000","last_peered":"2024-09-10T23:38:51.992743+0000","last_clean":"2024-09-10T23:38:51.992743+0000","last_became_active":"2024-09-10T23:38:51.013954+0000","last_became_peered":"2024-09-10T23:38:51.013954+0000","last_unstale":"2024-09-10T23:38:51.992743+0000","last_undegraded":"2024-09-10T23:38:51.992743+0000","last_fullsized":"2024-09-10T23:38:51.992743+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:30.596239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157561+0000","last_change":"2024-09-10T23:38:50.001391+0000","last_active":"2024-09-10T23:38:52.157561+0000","last_peered":"2024-09-10T23:38:52.157561+0000","last_clean":"2024-09-10T23:38:52.157561+0000","last_became_active":"2024-09-10T23:38:50.000839+0000","last_became_peered":"2024-09-10T23:38:50.000839+0000","last_unstale":"2024-09-10T23:38:52.157561+0000","last_undegraded":"2024-09-10T23:38:52.157561+0000","last_fullsized":"2024-09-10T23:38:52.157561+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:23.703292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994784+0000","last_change":"2024-09-10T23:38:51.012723+0000","last_active":"2024-09-10T23:38:51.994784+0000","last_peered":"2024-09-10T23:38:51.994784+0000","last_clean":"2024-09-10T23:38:51.994784+0000","last_became_active":"2024-09-10T23:38:51.012159+0000","last_became_peered":"2024-09-10T23:38:51.012159+0000","last_unstale":"2024-09-10T23:38:51.994784+0000","last_undegraded":"2024-09-10T23:38:51.994784+0000","last_fullsized":"2024-09-10T23:38:51.994784+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:24.188587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157400+0000","last_change":"2024-09-10T23:38:49.999593+0000","last_active":"2024-09-10T23:38:52.157400+0000","last_peered":"2024-09-10T23:38:52.157400+0000","last_clean":"2024-09-10T23:38:52.157400+0000","last_became_active":"2024-09-10T23:38:49.999258+0000","last_became_peered":"2024-09-10T23:38:49.999258+0000","last_unstale":"2024-09-10T23:38:52.157400+0000","last_undegraded":"2024-09-10T23:38:52.157400+0000","last_fullsized":"2024-09-10T23:38:52.157400+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:03.695248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992600+0000","last_change":"2024-09-10T23:38:51.009127+0000","last_active":"2024-09-10T23:38:51.992600+0000","last_peered":"2024-09-10T23:38:51.992600+0000","last_clean":"2024-09-10T23:38:51.992600+0000","last_became_active":"2024-09-10T23:38:51.008867+0000","last_became_peered":"2024-09-10T23:38:51.008867+0000","last_unstale":"2024-09-10T23:38:51.992600+0000","last_undegraded":"2024-09-10T23:38:51.992600+0000","last_fullsized":"2024-09-10T23:38:51.992600+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:09.659885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021334+0000","last_change":"2024-09-10T23:38:50.000383+0000","last_active":"2024-09-10T23:38:52.021334+0000","last_peered":"2024-09-10T23:38:52.021334+0000","last_clean":"2024-09-10T23:38:52.021334+0000","last_became_active":"2024-09-10T23:38:49.999943+0000","last_became_peered":"2024-09-10T23:38:49.999943+0000","last_unstale":"2024-09-10T23:38:52.021334+0000","last_undegraded":"2024-09-10T23:38:52.021334+0000","last_fullsized":"2024-09-10T23:38:52.021334+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:26.170538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992917+0000","last_change":"2024-09-10T23:38:51.013767+0000","last_active":"2024-09-10T23:38:51.992917+0000","last_peered":"2024-09-10T23:38:51.992917+0000","last_clean":"2024-09-10T23:38:51.992917+0000","last_became_active":"2024-09-10T23:38:51.010462+0000","last_became_peered":"2024-09-10T23:38:51.010462+0000","last_unstale":"2024-09-10T23:38:51.992917+0000","last_undegraded":"2024-09-10T23:38:51.992917+0000","last_fullsized":"2024-09-10T23:38:51.992917+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:48:03.810193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"126'8","reported_seq":23,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994578+0000","last_change":"2024-09-10T23:38:50.001418+0000","last_active":"2024-09-10T23:38:51.994578+0000","last_peered":"2024-09-10T23:38:51.994578+0000","last_clean":"2024-09-10T23:38:51.994578+0000","last_became_active":"2024-09-10T23:38:50.001055+0000","last_became_peered":"2024-09-10T23:38:50.001055+0000","last_unstale":"2024-09-10T23:38:51.994578+0000","last_undegraded":"2024-09-10T23:38:51.994578+0000","last_fullsized":"2024-09-10T23:38:51.994578+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:41.655333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.158268+0000","last_change":"2024-09-10T23:38:51.006705+0000","last_active":"2024-09-10T23:38:52.158268+0000","last_peered":"2024-09-10T23:38:52.158268+0000","last_clean":"2024-09-10T23:38:52.158268+0000","last_became_active":"2024-09-10T23:38:51.006091+0000","last_became_peered":"2024-09-10T23:38:51.006091+0000","last_unstale":"2024-09-10T23:38:52.158268+0000","last_undegraded":"2024-09-10T23:38:52.158268+0000","last_fullsized":"2024-09-10T23:38:52.158268+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:10.509963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993621+0000","last_change":"2024-09-10T23:38:50.005199+0000","last_active":"2024-09-10T23:38:51.993621+0000","last_peered":"2024-09-10T23:38:51.993621+0000","last_clean":"2024-09-10T23:38:51.993621+0000","last_became_active":"2024-09-10T23:38:50.004764+0000","last_became_peered":"2024-09-10T23:38:50.004764+0000","last_unstale":"2024-09-10T23:38:51.993621+0000","last_undegraded":"2024-09-10T23:38:51.993621+0000","last_fullsized":"2024-09-10T23:38:51.993621+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:35.392555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021749+0000","last_change":"2024-09-10T23:38:51.020237+0000","last_active":"2024-09-10T23:38:52.021749+0000","last_peered":"2024-09-10T23:38:52.021749+0000","last_clean":"2024-09-10T23:38:52.021749+0000","last_became_active":"2024-09-10T23:38:51.019623+0000","last_became_peered":"2024-09-10T23:38:51.019623+0000","last_unstale":"2024-09-10T23:38:52.021749+0000","last_undegraded":"2024-09-10T23:38:52.021749+0000","last_fullsized":"2024-09-10T23:38:52.021749+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:18.958122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995945+0000","last_change":"2024-09-10T23:38:49.995217+0000","last_active":"2024-09-10T23:38:51.995945+0000","last_peered":"2024-09-10T23:38:51.995945+0000","last_clean":"2024-09-10T23:38:51.995945+0000","last_became_active":"2024-09-10T23:38:49.994905+0000","last_became_peered":"2024-09-10T23:38:49.994905+0000","last_unstale":"2024-09-10T23:38:51.995945+0000","last_undegraded":"2024-09-10T23:38:51.995945+0000","last_fullsized":"2024-09-10T23:38:51.995945+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:38.014917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157869+0000","last_change":"2024-09-10T23:38:51.001392+0000","last_active":"2024-09-10T23:38:52.157869+0000","last_peered":"2024-09-10T23:38:52.157869+0000","last_clean":"2024-09-10T23:38:52.157869+0000","last_became_active":"2024-09-10T23:38:51.001116+0000","last_became_peered":"2024-09-10T23:38:51.001116+0000","last_unstale":"2024-09-10T23:38:52.157869+0000","last_undegraded":"2024-09-10T23:38:52.157869+0000","last_fullsized":"2024-09-10T23:38:52.157869+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:36.399112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157042+0000","last_change":"2024-09-10T23:38:49.994999+0000","last_active":"2024-09-10T23:38:52.157042+0000","last_peered":"2024-09-10T23:38:52.157042+0000","last_clean":"2024-09-10T23:38:52.157042+0000","last_became_active":"2024-09-10T23:38:49.994677+0000","last_became_peered":"2024-09-10T23:38:49.994677+0000","last_unstale":"2024-09-10T23:38:52.157042+0000","last_undegraded":"2024-09-10T23:38:52.157042+0000","last_fullsized":"2024-09-10T23:38:52.157042+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:56.573683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995344+0000","last_change":"2024-09-10T23:38:51.004574+0000","last_active":"2024-09-10T23:38:51.995344+0000","last_peered":"2024-09-10T23:38:51.995344+0000","last_clean":"2024-09-10T23:38:51.995344+0000","last_became_active":"2024-09-10T23:38:51.004244+0000","last_became_peered":"2024-09-10T23:38:51.004244+0000","last_unstale":"2024-09-10T23:38:51.995344+0000","last_undegraded":"2024-09-10T23:38:51.995344+0000","last_fullsized":"2024-09-10T23:38:51.995344+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:46.365535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021583+0000","last_change":"2024-09-10T23:38:49.995698+0000","last_active":"2024-09-10T23:38:52.021583+0000","last_peered":"2024-09-10T23:38:52.021583+0000","last_clean":"2024-09-10T23:38:52.021583+0000","last_became_active":"2024-09-10T23:38:49.995420+0000","last_became_peered":"2024-09-10T23:38:49.995420+0000","last_unstale":"2024-09-10T23:38:52.021583+0000","last_undegraded":"2024-09-10T23:38:52.021583+0000","last_fullsized":"2024-09-10T23:38:52.021583+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:13.801146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157304+0000","last_change":"2024-09-10T23:38:51.002583+0000","last_active":"2024-09-10T23:38:52.157304+0000","last_peered":"2024-09-10T23:38:52.157304+0000","last_clean":"2024-09-10T23:38:52.157304+0000","last_became_active":"2024-09-10T23:38:51.001781+0000","last_became_peered":"2024-09-10T23:38:51.001781+0000","last_unstale":"2024-09-10T23:38:52.157304+0000","last_undegraded":"2024-09-10T23:38:52.157304+0000","last_fullsized":"2024-09-10T23:38:52.157304+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:35.522701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994408+0000","last_change":"2024-09-10T23:38:49.995991+0000","last_active":"2024-09-10T23:38:51.994408+0000","last_peered":"2024-09-10T23:38:51.994408+0000","last_clean":"2024-09-10T23:38:51.994408+0000","last_became_active":"2024-09-10T23:38:49.995646+0000","last_became_peered":"2024-09-10T23:38:49.995646+0000","last_unstale":"2024-09-10T23:38:51.994408+0000","last_undegraded":"2024-09-10T23:38:51.994408+0000","last_fullsized":"2024-09-10T23:38:51.994408+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:05.537992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021333+0000","last_change":"2024-09-10T23:38:51.016657+0000","last_active":"2024-09-10T23:38:52.021333+0000","last_peered":"2024-09-10T23:38:52.021333+0000","last_clean":"2024-09-10T23:38:52.021333+0000","last_became_active":"2024-09-10T23:38:51.016362+0000","last_became_peered":"2024-09-10T23:38:51.016362+0000","last_unstale":"2024-09-10T23:38:52.021333+0000","last_undegraded":"2024-09-10T23:38:52.021333+0000","last_fullsized":"2024-09-10T23:38:52.021333+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:41.867094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992698+0000","last_change":"2024-09-10T23:38:50.008339+0000","last_active":"2024-09-10T23:38:51.992698+0000","last_peered":"2024-09-10T23:38:51.992698+0000","last_clean":"2024-09-10T23:38:51.992698+0000","last_became_active":"2024-09-10T23:38:50.008050+0000","last_became_peered":"2024-09-10T23:38:50.008050+0000","last_unstale":"2024-09-10T23:38:51.992698+0000","last_undegraded":"2024-09-10T23:38:51.992698+0000","last_fullsized":"2024-09-10T23:38:51.992698+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:44.904439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156730+0000","last_change":"2024-09-10T23:38:51.002280+0000","last_active":"2024-09-10T23:38:52.156730+0000","last_peered":"2024-09-10T23:38:52.156730+0000","last_clean":"2024-09-10T23:38:52.156730+0000","last_became_active":"2024-09-10T23:38:51.001949+0000","last_became_peered":"2024-09-10T23:38:51.001949+0000","last_unstale":"2024-09-10T23:38:52.156730+0000","last_undegraded":"2024-09-10T23:38:52.156730+0000","last_fullsized":"2024-09-10T23:38:52.156730+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:37:38.619339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992949+0000","last_change":"2024-09-10T23:38:50.004257+0000","last_active":"2024-09-10T23:38:51.992949+0000","last_peered":"2024-09-10T23:38:51.992949+0000","last_clean":"2024-09-10T23:38:51.992949+0000","last_became_active":"2024-09-10T23:38:50.003746+0000","last_became_peered":"2024-09-10T23:38:50.003746+0000","last_unstale":"2024-09-10T23:38:51.992949+0000","last_undegraded":"2024-09-10T23:38:51.992949+0000","last_fullsized":"2024-09-10T23:38:51.992949+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:50.667709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021208+0000","last_change":"2024-09-10T23:38:51.015678+0000","last_active":"2024-09-10T23:38:52.021208+0000","last_peered":"2024-09-10T23:38:52.021208+0000","last_clean":"2024-09-10T23:38:52.021208+0000","last_became_active":"2024-09-10T23:38:51.015026+0000","last_became_peered":"2024-09-10T23:38:51.015026+0000","last_unstale":"2024-09-10T23:38:52.021208+0000","last_undegraded":"2024-09-10T23:38:52.021208+0000","last_fullsized":"2024-09-10T23:38:52.021208+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:25.873667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992710+0000","last_change":"2024-09-10T23:38:50.008309+0000","last_active":"2024-09-10T23:38:51.992710+0000","last_peered":"2024-09-10T23:38:51.992710+0000","last_clean":"2024-09-10T23:38:51.992710+0000","last_became_active":"2024-09-10T23:38:50.008007+0000","last_became_peered":"2024-09-10T23:38:50.008007+0000","last_unstale":"2024-09-10T23:38:51.992710+0000","last_undegraded":"2024-09-10T23:38:51.992710+0000","last_fullsized":"2024-09-10T23:38:51.992710+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:08.919532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993645+0000","last_change":"2024-09-10T23:38:51.007392+0000","last_active":"2024-09-10T23:38:51.993645+0000","last_peered":"2024-09-10T23:38:51.993645+0000","last_clean":"2024-09-10T23:38:51.993645+0000","last_became_active":"2024-09-10T23:38:51.006642+0000","last_became_peered":"2024-09-10T23:38:51.006642+0000","last_unstale":"2024-09-10T23:38:51.993645+0000","last_undegraded":"2024-09-10T23:38:51.993645+0000","last_fullsized":"2024-09-10T23:38:51.993645+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:57.110746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993744+0000","last_change":"2024-09-10T23:38:50.001623+0000","last_active":"2024-09-10T23:38:51.993744+0000","last_peered":"2024-09-10T23:38:51.993744+0000","last_clean":"2024-09-10T23:38:51.993744+0000","last_became_active":"2024-09-10T23:38:50.001315+0000","last_became_peered":"2024-09-10T23:38:50.001315+0000","last_unstale":"2024-09-10T23:38:51.993744+0000","last_undegraded":"2024-09-10T23:38:51.993744+0000","last_fullsized":"2024-09-10T23:38:51.993744+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:42.811340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993661+0000","last_change":"2024-09-10T23:38:51.009829+0000","last_active":"2024-09-10T23:38:51.993661+0000","last_peered":"2024-09-10T23:38:51.993661+0000","last_clean":"2024-09-10T23:38:51.993661+0000","last_became_active":"2024-09-10T23:38:51.009248+0000","last_became_peered":"2024-09-10T23:38:51.009248+0000","last_unstale":"2024-09-10T23:38:51.993661+0000","last_undegraded":"2024-09-10T23:38:51.993661+0000","last_fullsized":"2024-09-10T23:38:51.993661+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:04.701805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"127'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.023515+0000","last_change":"2024-09-10T23:38:50.005377+0000","last_active":"2024-09-10T23:38:52.023515+0000","last_peered":"2024-09-10T23:38:52.023515+0000","last_clean":"2024-09-10T23:38:52.023515+0000","last_became_active":"2024-09-10T23:38:50.005061+0000","last_became_peered":"2024-09-10T23:38:50.005061+0000","last_unstale":"2024-09-10T23:38:52.023515+0000","last_undegraded":"2024-09-10T23:38:52.023515+0000","last_fullsized":"2024-09-10T23:38:52.023515+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:39.375765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995992+0000","last_change":"2024-09-10T23:38:51.006670+0000","last_active":"2024-09-10T23:38:51.995992+0000","last_peered":"2024-09-10T23:38:51.995992+0000","last_clean":"2024-09-10T23:38:51.995992+0000","last_became_active":"2024-09-10T23:38:51.006405+0000","last_became_peered":"2024-09-10T23:38:51.006405+0000","last_unstale":"2024-09-10T23:38:51.995992+0000","last_undegraded":"2024-09-10T23:38:51.995992+0000","last_fullsized":"2024-09-10T23:38:51.995992+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:50.087651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157124+0000","last_change":"2024-09-10T23:38:49.991369+0000","last_active":"2024-09-10T23:38:52.157124+0000","last_peered":"2024-09-10T23:38:52.157124+0000","last_clean":"2024-09-10T23:38:52.157124+0000","last_became_active":"2024-09-10T23:38:49.990807+0000","last_became_peered":"2024-09-10T23:38:49.990807+0000","last_unstale":"2024-09-10T23:38:52.157124+0000","last_undegraded":"2024-09-10T23:38:52.157124+0000","last_fullsized":"2024-09-10T23:38:52.157124+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:34:28.178846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.995998+0000","last_change":"2024-09-10T23:38:51.014424+0000","last_active":"2024-09-10T23:38:51.995998+0000","last_peered":"2024-09-10T23:38:51.995998+0000","last_clean":"2024-09-10T23:38:51.995998+0000","last_became_active":"2024-09-10T23:38:51.013953+0000","last_became_peered":"2024-09-10T23:38:51.013953+0000","last_unstale":"2024-09-10T23:38:51.995998+0000","last_undegraded":"2024-09-10T23:38:51.995998+0000","last_fullsized":"2024-09-10T23:38:51.995998+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:34.717688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157041+0000","last_change":"2024-09-10T23:38:49.992314+0000","last_active":"2024-09-10T23:38:52.157041+0000","last_peered":"2024-09-10T23:38:52.157041+0000","last_clean":"2024-09-10T23:38:52.157041+0000","last_became_active":"2024-09-10T23:38:49.992007+0000","last_became_peered":"2024-09-10T23:38:49.992007+0000","last_unstale":"2024-09-10T23:38:52.157041+0000","last_undegraded":"2024-09-10T23:38:52.157041+0000","last_fullsized":"2024-09-10T23:38:52.157041+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:06.441197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156909+0000","last_change":"2024-09-10T23:38:51.022021+0000","last_active":"2024-09-10T23:38:52.156909+0000","last_peered":"2024-09-10T23:38:52.156909+0000","last_clean":"2024-09-10T23:38:52.156909+0000","last_became_active":"2024-09-10T23:38:51.021689+0000","last_became_peered":"2024-09-10T23:38:51.021689+0000","last_unstale":"2024-09-10T23:38:52.156909+0000","last_undegraded":"2024-09-10T23:38:52.156909+0000","last_fullsized":"2024-09-10T23:38:52.156909+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:23.666220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.021176+0000","last_change":"2024-09-10T23:38:49.998035+0000","last_active":"2024-09-10T23:38:52.021176+0000","last_peered":"2024-09-10T23:38:52.021176+0000","last_clean":"2024-09-10T23:38:52.021176+0000","last_became_active":"2024-09-10T23:38:49.997429+0000","last_became_peered":"2024-09-10T23:38:49.997429+0000","last_unstale":"2024-09-10T23:38:52.021176+0000","last_undegraded":"2024-09-10T23:38:52.021176+0000","last_fullsized":"2024-09-10T23:38:52.021176+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:18:57.051526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.993237+0000","last_change":"2024-09-10T23:38:51.014876+0000","last_active":"2024-09-10T23:38:51.993237+0000","last_peered":"2024-09-10T23:38:51.993237+0000","last_clean":"2024-09-10T23:38:51.993237+0000","last_became_active":"2024-09-10T23:38:51.014594+0000","last_became_peered":"2024-09-10T23:38:51.014594+0000","last_unstale":"2024-09-10T23:38:51.993237+0000","last_undegraded":"2024-09-10T23:38:51.993237+0000","last_fullsized":"2024-09-10T23:38:51.993237+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:21.080920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157241+0000","last_change":"2024-09-10T23:38:49.996415+0000","last_active":"2024-09-10T23:38:52.157241+0000","last_peered":"2024-09-10T23:38:52.157241+0000","last_clean":"2024-09-10T23:38:52.157241+0000","last_became_active":"2024-09-10T23:38:49.996072+0000","last_became_peered":"2024-09-10T23:38:49.996072+0000","last_unstale":"2024-09-10T23:38:52.157241+0000","last_undegraded":"2024-09-10T23:38:52.157241+0000","last_fullsized":"2024-09-10T23:38:52.157241+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:25.597983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.992862+0000","last_change":"2024-09-10T23:38:51.009829+0000","last_active":"2024-09-10T23:38:51.992862+0000","last_peered":"2024-09-10T23:38:51.992862+0000","last_clean":"2024-09-10T23:38:51.992862+0000","last_became_active":"2024-09-10T23:38:51.009481+0000","last_became_peered":"2024-09-10T23:38:51.009481+0000","last_unstale":"2024-09-10T23:38:51.992862+0000","last_undegraded":"2024-09-10T23:38:51.992862+0000","last_fullsized":"2024-09-10T23:38:51.992862+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:12:04.685418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156527+0000","last_change":"2024-09-10T23:38:49.996805+0000","last_active":"2024-09-10T23:38:52.156527+0000","last_peered":"2024-09-10T23:38:52.156527+0000","last_clean":"2024-09-10T23:38:52.156527+0000","last_became_active":"2024-09-10T23:38:49.996458+0000","last_became_peered":"2024-09-10T23:38:49.996458+0000","last_unstale":"2024-09-10T23:38:52.156527+0000","last_undegraded":"2024-09-10T23:38:52.156527+0000","last_fullsized":"2024-09-10T23:38:52.156527+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:26.148834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156430+0000","last_change":"2024-09-10T23:38:51.018198+0000","last_active":"2024-09-10T23:38:52.156430+0000","last_peered":"2024-09-10T23:38:52.156430+0000","last_clean":"2024-09-10T23:38:52.156430+0000","last_became_active":"2024-09-10T23:38:51.017893+0000","last_became_peered":"2024-09-10T23:38:51.017893+0000","last_unstale":"2024-09-10T23:38:52.156430+0000","last_undegraded":"2024-09-10T23:38:52.156430+0000","last_fullsized":"2024-09-10T23:38:52.156430+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:33.560706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:51.994035+0000","last_change":"2024-09-10T23:38:49.993427+0000","last_active":"2024-09-10T23:38:51.994035+0000","last_peered":"2024-09-10T23:38:51.994035+0000","last_clean":"2024-09-10T23:38:51.994035+0000","last_became_active":"2024-09-10T23:38:49.992941+0000","last_became_peered":"2024-09-10T23:38:49.992941+0000","last_unstale":"2024-09-10T23:38:51.994035+0000","last_undegraded":"2024-09-10T23:38:51.994035+0000","last_fullsized":"2024-09-10T23:38:51.994035+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:55.592712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.157013+0000","last_change":"2024-09-10T23:38:51.011553+0000","last_active":"2024-09-10T23:38:52.157013+0000","last_peered":"2024-09-10T23:38:52.157013+0000","last_clean":"2024-09-10T23:38:52.157013+0000","last_became_active":"2024-09-10T23:38:51.011292+0000","last_became_peered":"2024-09-10T23:38:51.011292+0000","last_unstale":"2024-09-10T23:38:52.157013+0000","last_undegraded":"2024-09-10T23:38:52.157013+0000","last_fullsized":"2024-09-10T23:38:52.157013+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:49.971557+0000","last_clean_scrub_stamp":"2024-09-10T23:38:49.971557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:16.455092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"126'3","reported_seq":18,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156892+0000","last_change":"2024-09-10T23:38:49.997220+0000","last_active":"2024-09-10T23:38:52.156892+0000","last_peered":"2024-09-10T23:38:52.156892+0000","last_clean":"2024-09-10T23:38:52.156892+0000","last_became_active":"2024-09-10T23:38:49.996952+0000","last_became_peered":"2024-09-10T23:38:49.996952+0000","last_unstale":"2024-09-10T23:38:52.156892+0000","last_undegraded":"2024-09-10T23:38:52.156892+0000","last_fullsized":"2024-09-10T23:38:52.156892+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:38:48.964999+0000","last_clean_scrub_stamp":"2024-09-10T23:38:48.964999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:09:56.104189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":331,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-10T23:38:52.156490+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:38:52.156490+0000","last_peered":"2024-09-10T23:38:52.156490+0000","last_clean":"2024-09-10T23:38:52.156490+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:38:52.156490+0000","last_undegraded":"2024-09-10T23:38:52.156490+0000","last_fullsized":"2024-09-10T23:38:52.156490+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509522,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1585152,"data_stored":697016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542225,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1585152,"data_stored":697016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509521,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33328,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338512,"statfs":{"total":96636764160,"available":96602636288,"internally_reserved":0,"allocated":2191360,"data_stored":1282862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509519,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1593344,"data_stored":692904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509520,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1593344,"data_stored":697632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509521,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1589248,"data_stored":697038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509520,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33324,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338516,"statfs":{"total":96636764160,"available":96602640384,"internally_reserved":0,"allocated":2183168,"data_stored":1287454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509520,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1576960,"data_stored":690778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:38:55.491 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T23:38:55.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T23:38:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.672+0000 7f16ba400640 1 -- 172.21.15.94:0/443736522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16b40d0660 msgr2=0x7f16b40ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.672+0000 7f16ba400640 1 --2- 172.21.15.94:0/443736522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16b40d0660 0x7f16b40ce3f0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009c80 tx=0x7f16ac01ca10 comp rx=0 tx=0).stop 2024-09-10T23:38:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.672+0000 7f16ba400640 1 -- 172.21.15.94:0/443736522 shutdown_connections 2024-09-10T23:38:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.672+0000 7f16ba400640 1 --2- 172.21.15.94:0/443736522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16b40d0660 0x7f16b40ce3f0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.672+0000 7f16ba400640 1 --2- 172.21.15.94:0/443736522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16b40cfce0 0x7f16b40d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.672+0000 7f16ba400640 1 --2- 172.21.15.94:0/443736522 >> v2:172.21.15.181:3300/0 conn(0x7f16b415bf60 0x7f16b40d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.672+0000 7f16ba400640 1 -- 172.21.15.94:0/443736522 >> 172.21.15.94:0/443736522 conn(0x7f16b40d1c60 msgr2=0x7f16b40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.672+0000 7f16ba400640 1 -- 172.21.15.94:0/443736522 shutdown_connections 2024-09-10T23:38:55.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.672+0000 7f16ba400640 1 -- 172.21.15.94:0/443736522 wait complete. 2024-09-10T23:38:55.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16ba400640 1 Processor -- start 2024-09-10T23:38:55.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16ba400640 1 -- start start 2024-09-10T23:38:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16ba400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16b40cfce0 0x7f16b4157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16ba400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16b40d0660 0x7f16b415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16ba400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16b415bf60 0x7f16b414c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16ba400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16b40c6b40 con 0x7f16b40cfce0 2024-09-10T23:38:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16ba400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f16b40c69c0 con 0x7f16b40d0660 2024-09-10T23:38:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16ba400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f16b40c6cc0 con 0x7f16b415bf60 2024-09-10T23:38:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16b3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16b40cfce0 0x7f16b4157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16b3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16b40cfce0 0x7f16b4157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55734/0 (socket says 172.21.15.94:55734) 2024-09-10T23:38:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16b3400640 1 -- 172.21.15.94:0/3797225323 learned_addr learned my addr 172.21.15.94:0/3797225323 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:55.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16aaa00640 1 --2- 172.21.15.94:0/3797225323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16b40d0660 0x7f16b415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16b3e00640 1 --2- 172.21.15.94:0/3797225323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16b415bf60 0x7f16b414c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16b3400640 1 -- 172.21.15.94:0/3797225323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16b415bf60 msgr2=0x7f16b414c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16b3400640 1 --2- 172.21.15.94:0/3797225323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16b415bf60 0x7f16b414c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16b3400640 1 -- 172.21.15.94:0/3797225323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16b40d0660 msgr2=0x7f16b415c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16b3400640 1 --2- 172.21.15.94:0/3797225323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16b40d0660 0x7f16b415c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16b3400640 1 -- 172.21.15.94:0/3797225323 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b415cad0 con 0x7f16b40cfce0 2024-09-10T23:38:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16b3400640 1 --2- 172.21.15.94:0/3797225323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16b40cfce0 0x7f16b4157ce0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f16a40058d0 tx=0x7f16a400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16abe00640 1 -- 172.21.15.94:0/3797225323 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4017070 con 0x7f16b40cfce0 2024-09-10T23:38:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16abe00640 1 -- 172.21.15.94:0/3797225323 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16a4009d20 con 0x7f16b40cfce0 2024-09-10T23:38:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16abe00640 1 -- 172.21.15.94:0/3797225323 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a40106c0 con 0x7f16b40cfce0 2024-09-10T23:38:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16ba400640 1 -- 172.21.15.94:0/3797225323 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b415cdb0 con 0x7f16b40cfce0 2024-09-10T23:38:55.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16ba400640 1 -- 172.21.15.94:0/3797225323 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b415d370 con 0x7f16b40cfce0 2024-09-10T23:38:55.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.676+0000 7f16ba400640 1 -- 172.21.15.94:0/3797225323 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b40ce3f0 con 0x7f16b40cfce0 2024-09-10T23:38:55.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.680+0000 7f16abe00640 1 -- 172.21.15.94:0/3797225323 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f16a4010860 con 0x7f16b40cfce0 2024-09-10T23:38:55.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.680+0000 7f16abe00640 1 --2- 172.21.15.94:0/3797225323 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1690072c40 0x7f1690075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:55.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.680+0000 7f16abe00640 1 -- 172.21.15.94:0/3797225323 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f16a4094320 con 0x7f16b40cfce0 2024-09-10T23:38:55.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.680+0000 7f16aaa00640 1 --2- 172.21.15.94:0/3797225323 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1690072c40 0x7f1690075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:55.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.680+0000 7f16aaa00640 1 --2- 172.21.15.94:0/3797225323 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1690072c40 0x7f1690075100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f16b40c79e0 tx=0x7f169c00b2e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:55.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.684+0000 7f16abe00640 1 -- 172.21.15.94:0/3797225323 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f16a405e900 con 0x7f16b40cfce0 2024-09-10T23:38:55.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.908+0000 7f16ba400640 1 -- 172.21.15.94:0/3797225323 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f16b40c7450 con 0x7f16b40cfce0 2024-09-10T23:38:55.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.912+0000 7f16abe00640 1 -- 172.21.15.94:0/3797225323 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v37) ==== 190+0+147 (secure 0 0 0) 0x7f16a4063330 con 0x7f16b40cfce0 2024-09-10T23:38:55.914 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-10T23:38:55.914 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g== 2024-09-10T23:38:55.914 INFO:teuthology.orchestra.run.smithi094.stdout: caps mds = "allow" 2024-09-10T23:38:55.914 INFO:teuthology.orchestra.run.smithi094.stdout: caps mon = "allow r" 2024-09-10T23:38:55.914 INFO:teuthology.orchestra.run.smithi094.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T23:38:55.914 INFO:teuthology.orchestra.run.smithi094.stderr:updated caps for client.0 2024-09-10T23:38:55.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.912+0000 7f16ba400640 1 -- 172.21.15.94:0/3797225323 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1690072c40 msgr2=0x7f1690075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:55.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.912+0000 7f16ba400640 1 --2- 172.21.15.94:0/3797225323 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1690072c40 0x7f1690075100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f16b40c79e0 tx=0x7f169c00b2e0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.912+0000 7f16ba400640 1 -- 172.21.15.94:0/3797225323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16b40cfce0 msgr2=0x7f16b4157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:55.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.912+0000 7f16ba400640 1 --2- 172.21.15.94:0/3797225323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16b40cfce0 0x7f16b4157ce0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f16a40058d0 tx=0x7f16a400dde0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.912+0000 7f16ba400640 1 -- 172.21.15.94:0/3797225323 shutdown_connections 2024-09-10T23:38:55.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.912+0000 7f16ba400640 1 --2- 172.21.15.94:0/3797225323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16b40d0660 0x7f16b415c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.912+0000 7f16ba400640 1 --2- 172.21.15.94:0/3797225323 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1690072c40 0x7f1690075100 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.912+0000 7f16ba400640 1 --2- 172.21.15.94:0/3797225323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16b415bf60 0x7f16b414c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.912+0000 7f16ba400640 1 --2- 172.21.15.94:0/3797225323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16b40cfce0 0x7f16b4157ce0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:55.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.912+0000 7f16ba400640 1 -- 172.21.15.94:0/3797225323 >> 172.21.15.94:0/3797225323 conn(0x7f16b40d1c60 msgr2=0x7f16b40cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:55.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.912+0000 7f16ba400640 1 -- 172.21.15.94:0/3797225323 shutdown_connections 2024-09-10T23:38:55.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:55.912+0000 7f16ba400640 1 -- 172.21.15.94:0/3797225323 wait complete. 2024-09-10T23:38:55.935 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:38:56.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.072+0000 7fdddac00640 1 -- 172.21.15.94:0/1440901882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddd40b2bb0 msgr2=0x7fddd40d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:56.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.072+0000 7fdddac00640 1 --2- 172.21.15.94:0/1440901882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddd40b2bb0 0x7fddd40d0270 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009840 tx=0x7fddbc01c700 comp rx=0 tx=0).stop 2024-09-10T23:38:56.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.072+0000 7fdddac00640 1 -- 172.21.15.94:0/1440901882 shutdown_connections 2024-09-10T23:38:56.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.072+0000 7fdddac00640 1 --2- 172.21.15.94:0/1440901882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddd40b2bb0 0x7fddd40d0270 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.072+0000 7fdddac00640 1 -- 172.21.15.94:0/1440901882 >> 172.21.15.94:0/1440901882 conn(0x7fddd40d1c60 msgr2=0x7fddd40d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:56.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.072+0000 7fdddac00640 1 -- 172.21.15.94:0/1440901882 shutdown_connections 2024-09-10T23:38:56.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.072+0000 7fdddac00640 1 -- 172.21.15.94:0/1440901882 wait complete. 2024-09-10T23:38:56.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fdddac00640 1 Processor -- start 2024-09-10T23:38:56.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fdddac00640 1 -- start start 2024-09-10T23:38:56.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fdddac00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddd40af310 0x7fddd41512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fdddac00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddd40b2bb0 0x7fddd40c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fdddac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd415f5c0 0x7fddd40c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fdddac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fddd416a0c0 con 0x7fddd415f5c0 2024-09-10T23:38:56.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fdddac00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fddd4169f40 con 0x7fddd40b2bb0 2024-09-10T23:38:56.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fdddac00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fddd416a240 con 0x7fddd40af310 2024-09-10T23:38:56.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fddd8c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd415f5c0 0x7fddd40c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fddd3e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddd40af310 0x7fddd41512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fddd8c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd415f5c0 0x7fddd40c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55744/0 (socket says 172.21.15.94:55744) 2024-09-10T23:38:56.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fddd8c00640 1 -- 172.21.15.94:0/3292573001 learned_addr learned my addr 172.21.15.94:0/3292573001 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:56.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fddcb400640 1 --2- 172.21.15.94:0/3292573001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddd40b2bb0 0x7fddd40c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fddd8c00640 1 -- 172.21.15.94:0/3292573001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddd40af310 msgr2=0x7fddd41512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:56.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fddd8c00640 1 --2- 172.21.15.94:0/3292573001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddd40af310 0x7fddd41512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fddd8c00640 1 -- 172.21.15.94:0/3292573001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddd40b2bb0 msgr2=0x7fddd40c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:56.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fddd8c00640 1 --2- 172.21.15.94:0/3292573001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddd40b2bb0 0x7fddd40c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fddd8c00640 1 -- 172.21.15.94:0/3292573001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd414dfc0 con 0x7fddd415f5c0 2024-09-10T23:38:56.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fddd8c00640 1 --2- 172.21.15.94:0/3292573001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd415f5c0 0x7fddd40c4f10 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00cbc0 tx=0x7fddcc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:56.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fddd0c00640 1 -- 172.21.15.94:0/3292573001 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc016020 con 0x7fddd415f5c0 2024-09-10T23:38:56.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fddd0c00640 1 -- 172.21.15.94:0/3292573001 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddcc00ab00 con 0x7fddd415f5c0 2024-09-10T23:38:56.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fddd0c00640 1 -- 172.21.15.94:0/3292573001 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc012420 con 0x7fddd415f5c0 2024-09-10T23:38:56.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fdddac00640 1 -- 172.21.15.94:0/3292573001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd414e2a0 con 0x7fddd415f5c0 2024-09-10T23:38:56.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.076+0000 7fdddac00640 1 -- 172.21.15.94:0/3292573001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd41536e0 con 0x7fddd415f5c0 2024-09-10T23:38:56.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.080+0000 7fdddac00640 1 -- 172.21.15.94:0/3292573001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd414dd30 con 0x7fddd415f5c0 2024-09-10T23:38:56.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.080+0000 7fddd0c00640 1 -- 172.21.15.94:0/3292573001 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7fddcc0125c0 con 0x7fddd415f5c0 2024-09-10T23:38:56.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.080+0000 7fddd0c00640 1 --2- 172.21.15.94:0/3292573001 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fddb0072c10 0x7fddb00750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.080+0000 7fddd0c00640 1 -- 172.21.15.94:0/3292573001 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7fddcc0943d0 con 0x7fddd415f5c0 2024-09-10T23:38:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.080+0000 7fddd3e00640 1 --2- 172.21.15.94:0/3292573001 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fddb0072c10 0x7fddb00750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.084+0000 7fddd3e00640 1 --2- 172.21.15.94:0/3292573001 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fddb0072c10 0x7fddb00750d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fddc4000c10 tx=0x7fddc4009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:56.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.084+0000 7fddd0c00640 1 -- 172.21.15.94:0/3292573001 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fddcc020020 con 0x7fddd415f5c0 2024-09-10T23:38:56.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.284+0000 7fdddac00640 1 -- 172.21.15.94:0/3292573001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddd41508e0 con 0x7fddd415f5c0 2024-09-10T23:38:56.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.284+0000 7fddd0c00640 1 -- 172.21.15.94:0/3292573001 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5234 (secure 0 0 0) 0x7fddcc05ea40 con 0x7fddd415f5c0 2024-09-10T23:38:56.289 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:38:56.289 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":88,"btime":"2024-09-10T23:38:54:013296+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3851553379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3851553379},{"type":"v1","addr":"172.21.15.181:6835","nonce":3851553379}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/90080224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":90080224},{"type":"v1","addr":"172.21.15.94:6833","nonce":90080224}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":7929,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/334167733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":334167733},{"type":"v1","addr":"172.21.15.94:6835","nonce":334167733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:38:50.375690+0000","modified":"2024-09-10T23:38:53.400898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196562":{"gid":196562,"name":"d","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6801/2109170417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2109170417},{"type":"v1","addr":"172.21.15.181:6801","nonce":2109170417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196562,"qdb_cluster":[196562]},"id":10}]} 2024-09-10T23:38:56.289 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 88 2024-09-10T23:38:56.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.288+0000 7fdddac00640 1 -- 172.21.15.94:0/3292573001 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fddb0072c10 msgr2=0x7fddb00750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:56.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.288+0000 7fdddac00640 1 --2- 172.21.15.94:0/3292573001 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fddb0072c10 0x7fddb00750d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fddc4000c10 tx=0x7fddc4009040 comp rx=0 tx=0).stop 2024-09-10T23:38:56.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.288+0000 7fdddac00640 1 -- 172.21.15.94:0/3292573001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd415f5c0 msgr2=0x7fddd40c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:56.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.288+0000 7fdddac00640 1 --2- 172.21.15.94:0/3292573001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd415f5c0 0x7fddd40c4f10 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00cbc0 tx=0x7fddcc00ccb0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.288+0000 7fdddac00640 1 -- 172.21.15.94:0/3292573001 shutdown_connections 2024-09-10T23:38:56.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.288+0000 7fdddac00640 1 --2- 172.21.15.94:0/3292573001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddd40b2bb0 0x7fddd40c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.288+0000 7fdddac00640 1 --2- 172.21.15.94:0/3292573001 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fddb0072c10 0x7fddb00750d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.288+0000 7fdddac00640 1 --2- 172.21.15.94:0/3292573001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddd415f5c0 0x7fddd40c4f10 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.288+0000 7fdddac00640 1 --2- 172.21.15.94:0/3292573001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddd40af310 0x7fddd41512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.288+0000 7fdddac00640 1 -- 172.21.15.94:0/3292573001 >> 172.21.15.94:0/3292573001 conn(0x7fddd40d1c60 msgr2=0x7fddd40b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:56.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.288+0000 7fdddac00640 1 -- 172.21.15.94:0/3292573001 shutdown_connections 2024-09-10T23:38:56.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.288+0000 7fdddac00640 1 -- 172.21.15.94:0/3292573001 wait complete. 2024-09-10T23:38:56.712 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T23:38:50.375690+0000', 'modified': '2024-09-10T23:38:53.400898+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 196562}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_196562': {'gid': 196562, 'name': 'd', 'rank': 0, 'incarnation': 84, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6801/2109170417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6800', 'nonce': 2109170417}, {'type': 'v1', 'addr': '172.21.15.181:6801', 'nonce': 2109170417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 196562, 'qdb_cluster': [196562]} 2024-09-10T23:38:56.713 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T23:38:56.713 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T23:38:56.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe499200640 1 -- 172.21.15.94:0/2770758664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe49413d530 msgr2=0x7fe49415dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:56.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe499200640 1 --2- 172.21.15.94:0/2770758664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe49413d530 0x7fe49415dfb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009900 tx=0x7fe48c01c790 comp rx=0 tx=0).stop 2024-09-10T23:38:56.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe499200640 1 -- 172.21.15.94:0/2770758664 shutdown_connections 2024-09-10T23:38:56.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe499200640 1 --2- 172.21.15.94:0/2770758664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe49413d530 0x7fe49415dfb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe499200640 1 --2- 172.21.15.94:0/2770758664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49413cc00 0x7fe49415da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe499200640 1 -- 172.21.15.94:0/2770758664 >> 172.21.15.94:0/2770758664 conn(0x7fe4940cfaf0 msgr2=0x7fe4940cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:56.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe499200640 1 -- 172.21.15.94:0/2770758664 shutdown_connections 2024-09-10T23:38:56.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe499200640 1 -- 172.21.15.94:0/2770758664 wait complete. 2024-09-10T23:38:56.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe499200640 1 Processor -- start 2024-09-10T23:38:56.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe499200640 1 -- start start 2024-09-10T23:38:56.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe499200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe49413cc00 0x7fe49413a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe499200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe49413cfe0 0x7fe4940b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe499200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49413d530 0x7fe49415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe499200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe49413f420 con 0x7fe49413cc00 2024-09-10T23:38:56.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe499200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe49413f2a0 con 0x7fe49413cfe0 2024-09-10T23:38:56.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe499200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe49413f5a0 con 0x7fe49413d530 2024-09-10T23:38:56.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe491e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe49413cc00 0x7fe49413a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe491e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe49413cc00 0x7fe49413a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55758/0 (socket says 172.21.15.94:55758) 2024-09-10T23:38:56.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.856+0000 7fe491e00640 1 -- 172.21.15.94:0/1556370031 learned_addr learned my addr 172.21.15.94:0/1556370031 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:56.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.860+0000 7fe489400640 1 --2- 172.21.15.94:0/1556370031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe49413cfe0 0x7fe4940b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.860+0000 7fe492800640 1 --2- 172.21.15.94:0/1556370031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49413d530 0x7fe49415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.860+0000 7fe491e00640 1 -- 172.21.15.94:0/1556370031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49413d530 msgr2=0x7fe49415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:56.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.860+0000 7fe491e00640 1 --2- 172.21.15.94:0/1556370031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49413d530 0x7fe49415a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.860+0000 7fe491e00640 1 -- 172.21.15.94:0/1556370031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe49413cfe0 msgr2=0x7fe4940b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:56.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.860+0000 7fe491e00640 1 --2- 172.21.15.94:0/1556370031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe49413cfe0 0x7fe4940b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.860+0000 7fe491e00640 1 -- 172.21.15.94:0/1556370031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49413aa60 con 0x7fe49413cc00 2024-09-10T23:38:56.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.860+0000 7fe489400640 1 --2- 172.21.15.94:0/1556370031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe49413cfe0 0x7fe4940b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:56.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.860+0000 7fe491e00640 1 --2- 172.21.15.94:0/1556370031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe49413cc00 0x7fe49413a520 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fe48400a690 tx=0x7fe48400ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:56.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.860+0000 7fe48aa00640 1 -- 172.21.15.94:0/1556370031 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48400c9b0 con 0x7fe49413cc00 2024-09-10T23:38:56.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.860+0000 7fe48aa00640 1 -- 172.21.15.94:0/1556370031 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe48400cb50 con 0x7fe49413cc00 2024-09-10T23:38:56.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.860+0000 7fe48aa00640 1 -- 172.21.15.94:0/1556370031 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484007430 con 0x7fe49413cc00 2024-09-10T23:38:56.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.860+0000 7fe499200640 1 -- 172.21.15.94:0/1556370031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4941580b0 con 0x7fe49413cc00 2024-09-10T23:38:56.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.860+0000 7fe499200640 1 -- 172.21.15.94:0/1556370031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe494158350 con 0x7fe49413cc00 2024-09-10T23:38:56.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.860+0000 7fe48aa00640 1 -- 172.21.15.94:0/1556370031 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7fe484013070 con 0x7fe49413cc00 2024-09-10T23:38:56.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe48aa00640 1 --2- 172.21.15.94:0/1556370031 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe470072c10 0x7fe4700750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe48aa00640 1 -- 172.21.15.94:0/1556370031 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7fe484093d00 con 0x7fe49413cc00 2024-09-10T23:38:56.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 Processor -- start 2024-09-10T23:38:56.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe489400640 1 --2- 172.21.15.94:0/1556370031 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe470072c10 0x7fe4700750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 -- start start 2024-09-10T23:38:56.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe489400640 1 --2- 172.21.15.94:0/1556370031 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe470072c10 0x7fe4700750d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe4941f6f90 tx=0x7fe47c009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:56.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf719d0 0x55fcdbf71db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 0x55fcdbf258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbf261f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fcdbe463f0 con 0x55fcdbf25e10 2024-09-10T23:38:56.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fcdbd83b20 con 0x55fcdbf722f0 2024-09-10T23:38:56.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x55fcdbd72f50 con 0x55fcdbf719d0 2024-09-10T23:38:56.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe492800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbf261f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe492800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbf261f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55772/0 (socket says 172.21.15.94:55772) 2024-09-10T23:38:56.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe492800640 1 -- 172.21.15.94:0/1704861360 learned_addr learned my addr 172.21.15.94:0/1704861360 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:56.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe489400640 1 --2- 172.21.15.94:0/1704861360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 0x55fcdbf258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe491e00640 1 --2- 172.21.15.94:0/1704861360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf719d0 0x55fcdbf71db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe492800640 1 -- 172.21.15.94:0/1704861360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf719d0 msgr2=0x55fcdbf71db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:38:56.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe492800640 1 --2- 172.21.15.94:0/1704861360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf719d0 0x55fcdbf71db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe492800640 1 -- 172.21.15.94:0/1704861360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 msgr2=0x55fcdbf258d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:38:56.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe492800640 1 --2- 172.21.15.94:0/1704861360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 0x55fcdbf258d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe492800640 1 -- 172.21.15.94:0/1704861360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcdbd3a590 con 0x55fcdbf25e10 2024-09-10T23:38:56.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe492800640 1 --2- 172.21.15.94:0/1704861360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbf261f0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009850 tx=0x7fe48c01afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb35aeef165d7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:56.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe482a00640 1 -- 172.21.15.94:0/1704861360 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c0b2050 con 0x55fcdbf25e10 2024-09-10T23:38:56.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe482a00640 1 -- 172.21.15.94:0/1704861360 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe48c0b3040 con 0x55fcdbf25e10 2024-09-10T23:38:56.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe482a00640 1 -- 172.21.15.94:0/1704861360 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c005d60 con 0x55fcdbf25e10 2024-09-10T23:38:56.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 -- 172.21.15.94:0/1704861360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 msgr2=0x55fcdbf261f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:56.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/1704861360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbf261f0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009850 tx=0x7fe48c01afd0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 -- 172.21.15.94:0/1704861360 shutdown_connections 2024-09-10T23:38:56.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/1704861360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 0x55fcdbf258d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/1704861360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbf261f0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/1704861360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf719d0 0x55fcdbf71db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 -- 172.21.15.94:0/1704861360 >> 172.21.15.94:0/1704861360 conn(0x55fcdbe0f130 msgr2=0x55fcdbf54e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:56.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 -- 172.21.15.94:0/1704861360 shutdown_connections 2024-09-10T23:38:56.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.864+0000 7fe49c6b7000 1 -- 172.21.15.94:0/1704861360 wait complete. 2024-09-10T23:38:56.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 1 -- start start 2024-09-10T23:38:56.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 1 Processor -- start 2024-09-10T23:38:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf25e10 0x55fcdbf526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf719d0 0x55fcdbf52e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 0x55fcdbf533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fcdbd0d5d0 con 0x55fcdbf719d0 2024-09-10T23:38:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fcdbd76d60 con 0x55fcdbf722f0 2024-09-10T23:38:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x55fcdbd19500 con 0x55fcdbf25e10 2024-09-10T23:38:56.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 10 client.0 _ll_register_callbacks cb 0x55fcdbd1d810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:38:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe489400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf719d0 0x55fcdbf52e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe491e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf25e10 0x55fcdbf526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe489400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf719d0 0x55fcdbf52e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55776/0 (socket says 172.21.15.94:55776) 2024-09-10T23:38:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe489400640 1 -- 172.21.15.94:0/4147704210 learned_addr learned my addr 172.21.15.94:0/4147704210 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe492800640 1 --2- 172.21.15.94:0/4147704210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 0x55fcdbf533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe489400640 1 -- 172.21.15.94:0/4147704210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf25e10 msgr2=0x55fcdbf526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:38:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe489400640 1 --2- 172.21.15.94:0/4147704210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf25e10 0x55fcdbf526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe489400640 1 -- 172.21.15.94:0/4147704210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 msgr2=0x55fcdbf533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:38:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe489400640 1 --2- 172.21.15.94:0/4147704210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 0x55fcdbf533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe489400640 1 -- 172.21.15.94:0/4147704210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcdbd11170 con 0x55fcdbf719d0 2024-09-10T23:38:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe489400640 1 --2- 172.21.15.94:0/4147704210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf719d0 0x55fcdbf52e70 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00e9e0 tx=0x7fe47c000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308b4e4cd156be5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe477400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:38:56.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe477400640 1 -- 172.21.15.94:0/4147704210 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c028060 con 0x55fcdbf719d0 2024-09-10T23:38:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe477400640 1 -- 172.21.15.94:0/4147704210 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7fe47c010040 con 0x55fcdbf719d0 2024-09-10T23:38:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 1 -- 172.21.15.94:0/4147704210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fcdbd0b0f0 con 0x55fcdbf719d0 2024-09-10T23:38:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe477400640 1 -- 172.21.15.94:0/4147704210 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fe47c00eea0 con 0x55fcdbf719d0 2024-09-10T23:38:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 10 client.8049 fetch_fsmap learned FSMap version 88 2024-09-10T23:38:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 1 -- 172.21.15.94:0/4147704210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55fcdbdf7ef0 con 0x55fcdbf719d0 2024-09-10T23:38:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe477400640 1 -- 172.21.15.94:0/4147704210 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55fcdbd0b0f0 con 0x55fcdbf719d0 2024-09-10T23:38:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe477400640 1 -- 172.21.15.94:0/4147704210 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 88) ==== 3502+0+0 (secure 0 0 0) 0x7fe47c013420 con 0x55fcdbf719d0 2024-09-10T23:38:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 10 client.8049 fetch_fsmap finished waiting for FSMap version 88 2024-09-10T23:38:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 10 client.8049 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T23:38:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 20 client.8049 populate_metadata read hostname 'smithi094' 2024-09-10T23:38:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/4147704210 >> [v2:172.21.15.181:6800/2109170417,v1:172.21.15.181:6801/2109170417] conn(0x55fcdbf540f0 0x55fcdbf83c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 4 client.8049 mds_command: new command op to 196562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:38:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe49c6b7000 1 -- 172.21.15.94:0/4147704210 --> [v2:172.21.15.181:6800/2109170417,v1:172.21.15.181:6801/2109170417] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcdbd85a70 con 0x55fcdbf540f0 2024-09-10T23:38:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.868+0000 7fe491e00640 1 --2- 172.21.15.94:0/4147704210 >> [v2:172.21.15.181:6800/2109170417,v1:172.21.15.181:6801/2109170417] conn(0x55fcdbf540f0 0x55fcdbf83c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe491e00640 1 --2- 172.21.15.94:0/4147704210 >> [v2:172.21.15.181:6800/2109170417,v1:172.21.15.181:6801/2109170417] conn(0x55fcdbf540f0 0x55fcdbf83c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d0d92f0581dea79 server_cookie=47bb5661995e7bc in_seq=0 out_seq=0 2024-09-10T23:38:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe477400640 10 client.8049 ms_handle_connect on v2:172.21.15.181:6800/2109170417 2024-09-10T23:38:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe477400640 1 -- 172.21.15.94:0/4147704210 <== mds.0 v2:172.21.15.181:6800/2109170417 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fcdbd85a70 con 0x55fcdbf540f0 2024-09-10T23:38:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe477400640 10 client.8049 handle_command_reply: tid=0 2024-09-10T23:38:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe49c6b7000 1 client.8049 shutdown 2024-09-10T23:38:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe49c6b7000 1 -- 172.21.15.94:0/4147704210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf719d0 msgr2=0x55fcdbf52e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/4147704210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf719d0 0x55fcdbf52e70 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00e9e0 tx=0x7fe47c000ee0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe49c6b7000 1 -- 172.21.15.94:0/4147704210 shutdown_connections 2024-09-10T23:38:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/4147704210 >> [v2:172.21.15.181:6800/2109170417,v1:172.21.15.181:6801/2109170417] conn(0x55fcdbf540f0 0x55fcdbf83c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/4147704210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 0x55fcdbf533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/4147704210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf719d0 0x55fcdbf52e70 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe477400640 0 client.8049 ms_handle_reset on v2:172.21.15.181:6800/2109170417 2024-09-10T23:38:56.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/4147704210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf25e10 0x55fcdbf526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe49c6b7000 1 -- 172.21.15.94:0/4147704210 >> 172.21.15.94:0/4147704210 conn(0x55fcdbe0f130 msgr2=0x55fcdbf24fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:56.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe49c6b7000 1 -- 172.21.15.94:0/4147704210 shutdown_connections 2024-09-10T23:38:56.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe49c6b7000 1 -- 172.21.15.94:0/4147704210 wait complete. 2024-09-10T23:38:56.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.872+0000 7fe49c6b7000 20 client.8049 trim_cache size 0 max 16384 2024-09-10T23:38:56.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 Processor -- start 2024-09-10T23:38:56.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 -- start start 2024-09-10T23:38:56.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbfe3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf719d0 0x55fcdbfe4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 0x55fcdbfe4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941ffe70 con 0x55fcdbf25e10 2024-09-10T23:38:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe49415d340 con 0x55fcdbf722f0 2024-09-10T23:38:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe49415c6d0 con 0x55fcdbf719d0 2024-09-10T23:38:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe491e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbfe3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe491e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbfe3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55778/0 (socket says 172.21.15.94:55778) 2024-09-10T23:38:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe491e00640 1 -- 172.21.15.94:0/3554272741 learned_addr learned my addr 172.21.15.94:0/3554272741 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe491e00640 1 -- 172.21.15.94:0/3554272741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf719d0 msgr2=0x55fcdbfe4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:38:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe491e00640 1 --2- 172.21.15.94:0/3554272741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf719d0 0x55fcdbfe4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe491e00640 1 -- 172.21.15.94:0/3554272741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 msgr2=0x55fcdbfe4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:38:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe491e00640 1 --2- 172.21.15.94:0/3554272741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 0x55fcdbfe4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe491e00640 1 -- 172.21.15.94:0/3554272741 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcdbdced90 con 0x55fcdbf25e10 2024-09-10T23:38:56.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe491e00640 1 --2- 172.21.15.94:0/3554272741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbfe3dd0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fe48405e3c0 tx=0x7fe484062040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5be8cb14b16eb8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe482000640 1 -- 172.21.15.94:0/3554272741 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48405eb30 con 0x55fcdbf25e10 2024-09-10T23:38:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe482000640 1 -- 172.21.15.94:0/3554272741 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe48405e430 con 0x55fcdbf25e10 2024-09-10T23:38:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe482000640 1 -- 172.21.15.94:0/3554272741 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48405d8d0 con 0x55fcdbf25e10 2024-09-10T23:38:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 -- 172.21.15.94:0/3554272741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 msgr2=0x55fcdbfe3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/3554272741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbfe3dd0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fe48405e3c0 tx=0x7fe484062040 comp rx=0 tx=0).stop 2024-09-10T23:38:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 -- 172.21.15.94:0/3554272741 shutdown_connections 2024-09-10T23:38:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/3554272741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 0x55fcdbfe4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/3554272741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf719d0 0x55fcdbfe4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/3554272741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbfe3dd0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 -- 172.21.15.94:0/3554272741 >> 172.21.15.94:0/3554272741 conn(0x55fcdbe0f130 msgr2=0x55fcdbf2da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:56.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 -- 172.21.15.94:0/3554272741 shutdown_connections 2024-09-10T23:38:56.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 -- 172.21.15.94:0/3554272741 wait complete. 2024-09-10T23:38:56.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 -- start start 2024-09-10T23:38:56.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.892+0000 7fe49c6b7000 1 Processor -- start 2024-09-10T23:38:56.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbfe7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf719d0 0x55fcdbfe8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 0x55fcdbfe8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fcdbe32730 con 0x55fcdbf25e10 2024-09-10T23:38:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55fcdbd7b390 con 0x55fcdbf722f0 2024-09-10T23:38:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x55fcdbd828f0 con 0x55fcdbf719d0 2024-09-10T23:38:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 10 client.0 _ll_register_callbacks cb 0x7fe47c001490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:38:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe491e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbfe7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe491e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbfe7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55782/0 (socket says 172.21.15.94:55782) 2024-09-10T23:38:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe491e00640 1 -- 172.21.15.94:0/1165318539 learned_addr learned my addr 172.21.15.94:0/1165318539 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe492800640 1 --2- 172.21.15.94:0/1165318539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 0x55fcdbfe8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe489400640 1 --2- 172.21.15.94:0/1165318539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf719d0 0x55fcdbfe8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe491e00640 1 -- 172.21.15.94:0/1165318539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf719d0 msgr2=0x55fcdbfe8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:38:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe491e00640 1 --2- 172.21.15.94:0/1165318539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf719d0 0x55fcdbfe8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe491e00640 1 -- 172.21.15.94:0/1165318539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 msgr2=0x55fcdbfe8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:38:56.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe491e00640 1 --2- 172.21.15.94:0/1165318539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 0x55fcdbfe8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe491e00640 1 -- 172.21.15.94:0/1165318539 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcdbd39a90 con 0x55fcdbf25e10 2024-09-10T23:38:56.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe492800640 1 --2- 172.21.15.94:0/1165318539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 0x55fcdbfe8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:56.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe491e00640 1 --2- 172.21.15.94:0/1165318539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbfe7c00 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fe48405e130 tx=0x7fe48405a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c5789befcf74a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:56.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe477400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:38:56.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe477400640 1 -- 172.21.15.94:0/1165318539 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484083050 con 0x55fcdbf25e10 2024-09-10T23:38:56.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe477400640 1 -- 172.21.15.94:0/1165318539 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7fe48408ab20 con 0x55fcdbf25e10 2024-09-10T23:38:56.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 -- 172.21.15.94:0/1165318539 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe48405e430 con 0x55fcdbf25e10 2024-09-10T23:38:56.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe477400640 1 -- 172.21.15.94:0/1165318539 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fe48405eb30 con 0x55fcdbf25e10 2024-09-10T23:38:56.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 10 client.8055 fetch_fsmap learned FSMap version 88 2024-09-10T23:38:56.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 -- 172.21.15.94:0/1165318539 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55fcdbd18050 con 0x55fcdbf25e10 2024-09-10T23:38:56.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe477400640 1 -- 172.21.15.94:0/1165318539 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe48405e430 con 0x55fcdbf25e10 2024-09-10T23:38:56.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe477400640 1 -- 172.21.15.94:0/1165318539 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 88) ==== 3502+0+0 (secure 0 0 0) 0x7fe4840831f0 con 0x55fcdbf25e10 2024-09-10T23:38:56.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 10 client.8055 fetch_fsmap finished waiting for FSMap version 88 2024-09-10T23:38:56.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 10 client.8055 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T23:38:56.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 20 client.8055 populate_metadata read hostname 'smithi094' 2024-09-10T23:38:56.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/1165318539 >> [v2:172.21.15.181:6800/2109170417,v1:172.21.15.181:6801/2109170417] conn(0x55fcdbf86330 0x55fcdbf86710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:56.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 4 client.8055 mds_command: new command op to 196562 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T23:38:56.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 -- 172.21.15.94:0/1165318539 --> [v2:172.21.15.181:6800/2109170417,v1:172.21.15.181:6801/2109170417] -- command(tid 0: {"prefix": "status"}) -- 0x55fcdbd81520 con 0x55fcdbf86330 2024-09-10T23:38:56.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe489400640 1 --2- 172.21.15.94:0/1165318539 >> [v2:172.21.15.181:6800/2109170417,v1:172.21.15.181:6801/2109170417] conn(0x55fcdbf86330 0x55fcdbf86710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:56.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe489400640 1 --2- 172.21.15.94:0/1165318539 >> [v2:172.21.15.181:6800/2109170417,v1:172.21.15.181:6801/2109170417] conn(0x55fcdbf86330 0x55fcdbf86710 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d21eb8b4758e58c server_cookie=a6658d9e10d8956 in_seq=0 out_seq=0 2024-09-10T23:38:56.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe477400640 10 client.8055 ms_handle_connect on v2:172.21.15.181:6800/2109170417 2024-09-10T23:38:56.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe477400640 1 -- 172.21.15.94:0/1165318539 <== mds.0 v2:172.21.15.181:6800/2109170417 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55fcdbd81520 con 0x55fcdbf86330 2024-09-10T23:38:56.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe477400640 10 client.8055 handle_command_reply: tid=0 2024-09-10T23:38:56.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 client.8055 shutdown 2024-09-10T23:38:56.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 -- 172.21.15.94:0/1165318539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 msgr2=0x55fcdbfe7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:56.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/1165318539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbfe7c00 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fe48405e130 tx=0x7fe48405a8b0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 -- 172.21.15.94:0/1165318539 shutdown_connections 2024-09-10T23:38:56.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/1165318539 >> [v2:172.21.15.181:6800/2109170417,v1:172.21.15.181:6801/2109170417] conn(0x55fcdbf86330 0x55fcdbf86710 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/1165318539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fcdbf722f0 0x55fcdbfe8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/1165318539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fcdbf719d0 0x55fcdbfe8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe477400640 0 client.8055 ms_handle_reset on v2:172.21.15.181:6800/2109170417 2024-09-10T23:38:56.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 --2- 172.21.15.94:0/1165318539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fcdbf25e10 0x55fcdbfe7c00 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 -- 172.21.15.94:0/1165318539 >> 172.21.15.94:0/1165318539 conn(0x55fcdbe0f130 msgr2=0x55fcdbf705b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:56.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 -- 172.21.15.94:0/1165318539 shutdown_connections 2024-09-10T23:38:56.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 1 -- 172.21.15.94:0/1165318539 wait complete. 2024-09-10T23:38:56.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.896+0000 7fe49c6b7000 20 client.8055 trim_cache size 0 max 16384 2024-09-10T23:38:56.906 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:38:56.906 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "1798cdb9-9937-4918-a569-86b41a92817e", 2024-09-10T23:38:56.906 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-09-10T23:38:56.906 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 196562, 2024-09-10T23:38:56.906 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-10T23:38:56.906 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-10T23:38:56.906 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-10T23:38:56.906 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 5.8941381369999997, 2024-09-10T23:38:56.906 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 88, 2024-09-10T23:38:56.906 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 127, 2024-09-10T23:38:56.906 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 127, 2024-09-10T23:38:56.906 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 10.433667610000001 2024-09-10T23:38:56.907 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:38:56.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.900+0000 7fe499200640 1 -- 172.21.15.94:0/1556370031 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe470072c10 msgr2=0x7fe4700750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:56.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.900+0000 7fe499200640 1 --2- 172.21.15.94:0/1556370031 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe470072c10 0x7fe4700750d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe4941f6f90 tx=0x7fe47c009040 comp rx=0 tx=0).stop 2024-09-10T23:38:56.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.900+0000 7fe499200640 1 -- 172.21.15.94:0/1556370031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe49413cc00 msgr2=0x7fe49413a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:56.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.900+0000 7fe499200640 1 --2- 172.21.15.94:0/1556370031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe49413cc00 0x7fe49413a520 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fe48400a690 tx=0x7fe48400ab50 comp rx=0 tx=0).stop 2024-09-10T23:38:56.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.900+0000 7fe499200640 1 -- 172.21.15.94:0/1556370031 shutdown_connections 2024-09-10T23:38:56.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.900+0000 7fe499200640 1 --2- 172.21.15.94:0/1556370031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe49413cfe0 0x7fe4940b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.900+0000 7fe499200640 1 --2- 172.21.15.94:0/1556370031 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe470072c10 0x7fe4700750d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.900+0000 7fe499200640 1 --2- 172.21.15.94:0/1556370031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49413d530 0x7fe49415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.900+0000 7fe499200640 1 --2- 172.21.15.94:0/1556370031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe49413cc00 0x7fe49413a520 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:56.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.900+0000 7fe499200640 1 -- 172.21.15.94:0/1556370031 >> 172.21.15.94:0/1556370031 conn(0x7fe4940cfaf0 msgr2=0x7fe4940cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:56.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.900+0000 7fe499200640 1 -- 172.21.15.94:0/1556370031 shutdown_connections 2024-09-10T23:38:56.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:56.900+0000 7fe499200640 1 -- 172.21.15.94:0/1556370031 wait complete. 2024-09-10T23:38:56.921 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:38:57.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.060+0000 7ff330400640 1 -- 172.21.15.94:0/1262771521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff32815f5c0 msgr2=0x7ff3280cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:57.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.060+0000 7ff330400640 1 --2- 172.21.15.94:0/1262771521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff32815f5c0 0x7ff3280cfb60 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009900 tx=0x7ff31c01c7c0 comp rx=0 tx=0).stop 2024-09-10T23:38:57.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.060+0000 7ff330400640 1 -- 172.21.15.94:0/1262771521 shutdown_connections 2024-09-10T23:38:57.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.060+0000 7ff330400640 1 --2- 172.21.15.94:0/1262771521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff32815f5c0 0x7ff3280cfb60 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:57.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.060+0000 7ff330400640 1 -- 172.21.15.94:0/1262771521 >> 172.21.15.94:0/1262771521 conn(0x7ff3280d1c60 msgr2=0x7ff3280d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:57.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.060+0000 7ff330400640 1 -- 172.21.15.94:0/1262771521 shutdown_connections 2024-09-10T23:38:57.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.060+0000 7ff330400640 1 -- 172.21.15.94:0/1262771521 wait complete. 2024-09-10T23:38:57.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff330400640 1 Processor -- start 2024-09-10T23:38:57.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff330400640 1 -- start start 2024-09-10T23:38:57.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff330400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3280af310 0x7ff3280c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:57.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff330400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3280b2bb0 0x7ff32814f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:57.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff330400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff32815f5c0 0x7ff32815b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:57.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff330400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff32816a120 con 0x7ff32815f5c0 2024-09-10T23:38:57.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff330400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff328169fa0 con 0x7ff3280af310 2024-09-10T23:38:57.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff330400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff32816a2a0 con 0x7ff3280b2bb0 2024-09-10T23:38:57.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff32e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff32815f5c0 0x7ff32815b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:57.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff32e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff32815f5c0 0x7ff32815b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55798/0 (socket says 172.21.15.94:55798) 2024-09-10T23:38:57.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff32e400640 1 -- 172.21.15.94:0/1443664409 learned_addr learned my addr 172.21.15.94:0/1443664409 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:57.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff32da00640 1 --2- 172.21.15.94:0/1443664409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3280af310 0x7ff3280c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:57.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff32d000640 1 --2- 172.21.15.94:0/1443664409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3280b2bb0 0x7ff32814f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:57.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff32e400640 1 -- 172.21.15.94:0/1443664409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3280b2bb0 msgr2=0x7ff32814f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:57.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff32e400640 1 --2- 172.21.15.94:0/1443664409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3280b2bb0 0x7ff32814f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:57.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff32e400640 1 -- 172.21.15.94:0/1443664409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3280af310 msgr2=0x7ff3280c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:57.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff32e400640 1 --2- 172.21.15.94:0/1443664409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3280af310 0x7ff3280c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:57.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff32e400640 1 -- 172.21.15.94:0/1443664409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3281553d0 con 0x7ff32815f5c0 2024-09-10T23:38:57.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff32da00640 1 --2- 172.21.15.94:0/1443664409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3280af310 0x7ff3280c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:38:57.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff32e400640 1 --2- 172.21.15.94:0/1443664409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff32815f5c0 0x7ff32815b4e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7ff32000c7f0 tx=0x7ff32000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:57.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff325600640 1 -- 172.21.15.94:0/1443664409 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff320003b80 con 0x7ff32815f5c0 2024-09-10T23:38:57.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff325600640 1 -- 172.21.15.94:0/1443664409 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff320003d20 con 0x7ff32815f5c0 2024-09-10T23:38:57.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff325600640 1 -- 172.21.15.94:0/1443664409 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32000aab0 con 0x7ff32815f5c0 2024-09-10T23:38:57.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff330400640 1 -- 172.21.15.94:0/1443664409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3281547c0 con 0x7ff32815f5c0 2024-09-10T23:38:57.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff330400640 1 -- 172.21.15.94:0/1443664409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff328154d80 con 0x7ff32815f5c0 2024-09-10T23:38:57.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.064+0000 7ff330400640 1 -- 172.21.15.94:0/1443664409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3280af7a0 con 0x7ff32815f5c0 2024-09-10T23:38:57.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.068+0000 7ff325600640 1 -- 172.21.15.94:0/1443664409 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7ff32000ac50 con 0x7ff32815f5c0 2024-09-10T23:38:57.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.068+0000 7ff325600640 1 --2- 172.21.15.94:0/1443664409 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff308072c10 0x7ff3080750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:57.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.068+0000 7ff325600640 1 -- 172.21.15.94:0/1443664409 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7ff320093db0 con 0x7ff32815f5c0 2024-09-10T23:38:57.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.068+0000 7ff32da00640 1 --2- 172.21.15.94:0/1443664409 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff308072c10 0x7ff3080750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:57.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.068+0000 7ff32da00640 1 --2- 172.21.15.94:0/1443664409 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff308072c10 0x7ff3080750d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff31c0af000 tx=0x7ff31c01b000 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.072+0000 7ff325600640 1 -- 172.21.15.94:0/1443664409 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff32005e9e0 con 0x7ff32815f5c0 2024-09-10T23:38:57.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.272+0000 7ff330400640 1 -- 172.21.15.94:0/1443664409 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff3280749c0 con 0x7ff32815f5c0 2024-09-10T23:38:57.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.272+0000 7ff325600640 1 -- 172.21.15.94:0/1443664409 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v88) ==== 53+0+73 (secure 0 0 0) 0x7ff320067100 con 0x7ff32815f5c0 2024-09-10T23:38:57.276 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:38:57.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.276+0000 7ff330400640 1 -- 172.21.15.94:0/1443664409 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff308072c10 msgr2=0x7ff3080750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:57.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.276+0000 7ff330400640 1 --2- 172.21.15.94:0/1443664409 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff308072c10 0x7ff3080750d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff31c0af000 tx=0x7ff31c01b000 comp rx=0 tx=0).stop 2024-09-10T23:38:57.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.276+0000 7ff330400640 1 -- 172.21.15.94:0/1443664409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff32815f5c0 msgr2=0x7ff32815b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:57.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.276+0000 7ff330400640 1 --2- 172.21.15.94:0/1443664409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff32815f5c0 0x7ff32815b4e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7ff32000c7f0 tx=0x7ff32000ccb0 comp rx=0 tx=0).stop 2024-09-10T23:38:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.276+0000 7ff330400640 1 -- 172.21.15.94:0/1443664409 shutdown_connections 2024-09-10T23:38:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.276+0000 7ff330400640 1 --2- 172.21.15.94:0/1443664409 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff308072c10 0x7ff3080750d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.276+0000 7ff330400640 1 --2- 172.21.15.94:0/1443664409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff32815f5c0 0x7ff32815b4e0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.276+0000 7ff330400640 1 --2- 172.21.15.94:0/1443664409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3280b2bb0 0x7ff32814f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.276+0000 7ff330400640 1 --2- 172.21.15.94:0/1443664409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3280af310 0x7ff3280c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.276+0000 7ff330400640 1 -- 172.21.15.94:0/1443664409 >> 172.21.15.94:0/1443664409 conn(0x7ff3280d1c60 msgr2=0x7ff3281567e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.276+0000 7ff330400640 1 -- 172.21.15.94:0/1443664409 shutdown_connections 2024-09-10T23:38:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57.276+0000 7ff330400640 1 -- 172.21.15.94:0/1443664409 wait complete. 2024-09-10T23:38:57.297 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:38:57.297 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:38:57.297 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T23:38:57.297 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T23:38:57.298 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T23:38:57.298 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:38:57.298 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:38:57.298 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:38:57.298 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:38:57.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:57.298 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T23:38:57.308 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:38:57.308 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:38:57.308 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:38:57.308 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:57.308 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T23:38:57.308 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:57.308 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:38:57.308 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:57.308 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T23:38:57.308 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:38:57.309 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:57.309 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T23:38:57.309 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:38:57.309 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:57.309 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:38:57.309 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:57.309 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:38:57.309 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42074sec preferred_lft 42074sec 2024-09-10T23:38:57.309 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-09-10T23:38:57.309 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:57.309 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:38:57.309 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:ed:7d:46:db brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:57.309 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:38:57.309 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:57.309 INFO:teuthology.orchestra.run.smithi094.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:38:57.310 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:57.310 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:38:57.310 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:57.310 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:38:57.310 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:57.310 INFO:teuthology.orchestra.run.smithi094.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:38:57.310 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:38:57.310 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:38:57.310 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:57.310 INFO:teuthology.orchestra.run.smithi094.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:38:57.310 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:38:57.310 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:38:57.310 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:57.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:57.311 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:38:57.366 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T23:38:57.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:57.368 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T23:38:57.421 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T23:38:57.421 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:38:57.421 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:38:57.421 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:38:57.422 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:38:57.422 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:38:57.422 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:38:57.422 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:38:57.422 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:38:57.422 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:38:57.422 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:38:57.422 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:38:57.422 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:38:57.422 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:38:57.422 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:38:57.422 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:38:57.423 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:38:57.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:38:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:38:57.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:57.558 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T23:38:57.566 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T23:38:57.566 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T23:38:57.567 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:38:57.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:57.567 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-09-10T23:38:57.628 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T23:38:57.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:38:57.628 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T23:38:57.677 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:38:57.677 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:38:57.677 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:38:57.728 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:38:57.789 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:38:57.846 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:38:57.846 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:38:57.846 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:38:57.846 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:38:57.846 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:38:57.846 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:38:57.847 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:38:57.847 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:38:57.847 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:38:57.847 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:38:57.847 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:38:57.847 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:38:57.847 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:38:57.847 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:38:57.847 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:38:57.847 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:38:57.847 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:38:57.847 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:38:57.847 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:38:57.848 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:38:57.848 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:38:57.848 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:38:57.848 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:38:57.848 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:38:57.848 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:38:57.848 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:38:57.848 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:38:57.848 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:38:57.848 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:38:57.848 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:38:57.848 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:38:57.848 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:38:57.849 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:38:57.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:57.849 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:38:57.901 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:38:57.901 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T23:38:57.947 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:38:57.969 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:38:58.026 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:38:58.026 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:38:58.026 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:38:58.026 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:38:58.026 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:38:58.026 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:38:58.027 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:38:58.027 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:38:58.027 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:38:58.027 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:38:58.027 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:38:58.027 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:38:58.027 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:38:58.027 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:38:58.027 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:38:58.027 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:38:58.027 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:38:58.027 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:38:58.027 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:38:58.028 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:38:58.028 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:38:58.028 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:38:58.028 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:38:58.028 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:38:58.028 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:38:58.028 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:38:58.028 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:38:58.028 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:38:58.028 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:38:58.028 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:38:58.028 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:38:58.028 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:38:58.028 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:38:58.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:58.029 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:38:58.043 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:2024-09-10T23:38:58.040+0000 7f02be61d580 -1 init, newargv = 0x55cc2b475ed0 newargc=15 2024-09-10T23:38:58.043 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[48680]: starting ceph client 2024-09-10T23:38:58.050 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[48680]: starting fuse 2024-09-10T23:38:58.052 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T23:38:58.053 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:38:58.053 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:38:58.053 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-09-10T23:38:58.053 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-09-10T23:38:58.053 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-09-10T23:38:58.053 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-09-10T23:38:58.053 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:38:58.053 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-09-10T23:38:58.053 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T23:38:58.053 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-09-10T23:38:58.053 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T23:38:58.054 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:38:58.054 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-09-10T23:38:58.054 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-09-10T23:38:58.054 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-09-10T23:38:58.054 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-09-10T23:38:58.054 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:38:58.054 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-09-10T23:38:58.054 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:38:58.054 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:38:58.054 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:38:58.054 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-09-10T23:38:58.054 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-09-10T23:38:58.055 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-09-10T23:38:58.055 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:38:58.055 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:38:58.055 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.0")) 2024-09-10T23:38:58.055 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-09-10T23:38:58.177 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.0.48680.asok 2024-09-10T23:38:58.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:38:58.188 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48680.asok 2024-09-10T23:38:58.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:58.189 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48680.asok status 2024-09-10T23:38:58.360 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:38:58.360 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-09-10T23:38:58.361 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:38:58.361 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:38:58.361 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "0", 2024-09-10T23:38:58.361 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-09-10T23:38:58.361 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T23:38:58.361 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "48680", 2024-09-10T23:38:58.361 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-09-10T23:38:58.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:58.361 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-09-10T23:38:58.361 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-09-10T23:38:58.361 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 8067, 2024-09-10T23:38:58.361 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-09-10T23:38:58.361 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-09-10T23:38:58.361 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-09-10T23:38:58.362 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 8067 2024-09-10T23:38:58.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:58.362 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:38:58.362 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:38:58.362 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:38:58.362 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3621953280 2024-09-10T23:38:58.362 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:38:58.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:58.362 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:38:58.362 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:38:58.362 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:38:58.362 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3621953280 2024-09-10T23:38:58.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:38:58.363 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.8067 192.168.0.1:0/3621953280", 2024-09-10T23:38:58.363 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/3621953280", 2024-09-10T23:38:58.363 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-09-10T23:38:58.363 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 88, 2024-09-10T23:38:58.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 127, 2024-09-10T23:38:58.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:38:58.363 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-09-10T23:38:58.363 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-09-10T23:38:58.363 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:38:58.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:58.375 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-09-10T23:38:58.375 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T23:38:58.384 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-09-10T23:38:58.384 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T23:38:58.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:58.385 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:38:58.444 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:38:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd7400640 1 -- 172.21.15.94:0/3335209466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd015ead0 msgr2=0x7f2dd006f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd7400640 1 --2- 172.21.15.94:0/3335209466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd015ead0 0x7f2dd006f790 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009bc0 tx=0x7f2dc801cae0 comp rx=0 tx=0).stop 2024-09-10T23:38:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd7400640 1 -- 172.21.15.94:0/3335209466 shutdown_connections 2024-09-10T23:38:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd7400640 1 --2- 172.21.15.94:0/3335209466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd015ead0 0x7f2dd006f790 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd7400640 1 --2- 172.21.15.94:0/3335209466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dd00d03c0 0x7f2dd015e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd7400640 1 -- 172.21.15.94:0/3335209466 >> 172.21.15.94:0/3335209466 conn(0x7f2dd006cdf0 msgr2=0x7f2dd006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd7400640 1 -- 172.21.15.94:0/3335209466 shutdown_connections 2024-09-10T23:38:58.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd7400640 1 -- 172.21.15.94:0/3335209466 wait complete. 2024-09-10T23:38:58.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd7400640 1 Processor -- start 2024-09-10T23:38:58.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd7400640 1 -- start start 2024-09-10T23:38:58.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd00cf3c0 0x7f2dd0142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:58.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dd00d03c0 0x7f2dd00cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:58.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd7400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dd015ead0 0x7f2dd00cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:58.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd7400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0072610 con 0x7f2dd00cf3c0 2024-09-10T23:38:58.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd7400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0070770 con 0x7f2dd00d03c0 2024-09-10T23:38:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd7400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2dd0072790 con 0x7f2dd015ead0 2024-09-10T23:38:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd00cf3c0 0x7f2dd0142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd00cf3c0 0x7f2dd0142e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55818/0 (socket says 172.21.15.94:55818) 2024-09-10T23:38:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd4a00640 1 -- 172.21.15.94:0/1885573874 learned_addr learned my addr 172.21.15.94:0/1885573874 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:38:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dc7e00640 1 --2- 172.21.15.94:0/1885573874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dd00d03c0 0x7f2dd00cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd5400640 1 --2- 172.21.15.94:0/1885573874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dd015ead0 0x7f2dd00cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd4a00640 1 -- 172.21.15.94:0/1885573874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dd015ead0 msgr2=0x7f2dd00cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd4a00640 1 --2- 172.21.15.94:0/1885573874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dd015ead0 0x7f2dd00cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd4a00640 1 -- 172.21.15.94:0/1885573874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dd00d03c0 msgr2=0x7f2dd00cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd4a00640 1 --2- 172.21.15.94:0/1885573874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dd00d03c0 0x7f2dd00cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.624+0000 7f2dd4a00640 1 -- 172.21.15.94:0/1885573874 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd014a470 con 0x7f2dd00cf3c0 2024-09-10T23:38:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.628+0000 7f2dd4a00640 1 --2- 172.21.15.94:0/1885573874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd00cf3c0 0x7f2dd0142e50 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000c500 tx=0x7f2dc000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.628+0000 7f2dcd600640 1 -- 172.21.15.94:0/1885573874 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0017070 con 0x7f2dd00cf3c0 2024-09-10T23:38:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.628+0000 7f2dcd600640 1 -- 172.21.15.94:0/1885573874 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dc0003600 con 0x7f2dd00cf3c0 2024-09-10T23:38:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.628+0000 7f2dcd600640 1 -- 172.21.15.94:0/1885573874 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0003a90 con 0x7f2dd00cf3c0 2024-09-10T23:38:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.628+0000 7f2dd7400640 1 -- 172.21.15.94:0/1885573874 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd01493d0 con 0x7f2dd00cf3c0 2024-09-10T23:38:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.628+0000 7f2dd7400640 1 -- 172.21.15.94:0/1885573874 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd0149990 con 0x7f2dd00cf3c0 2024-09-10T23:38:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.628+0000 7f2dd7400640 1 -- 172.21.15.94:0/1885573874 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd015e590 con 0x7f2dd00cf3c0 2024-09-10T23:38:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.628+0000 7f2dcd600640 1 -- 172.21.15.94:0/1885573874 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f2dc0003c30 con 0x7f2dd00cf3c0 2024-09-10T23:38:58.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.628+0000 7f2dcd600640 1 --2- 172.21.15.94:0/1885573874 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2dac072c40 0x7f2dac075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:38:58.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.628+0000 7f2dcd600640 1 -- 172.21.15.94:0/1885573874 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f2dc0093610 con 0x7f2dd00cf3c0 2024-09-10T23:38:58.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.628+0000 7f2dc7e00640 1 --2- 172.21.15.94:0/1885573874 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2dac072c40 0x7f2dac075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:38:58.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.632+0000 7f2dc7e00640 1 --2- 172.21.15.94:0/1885573874 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2dac072c40 0x7f2dac075100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2db80058d0 tx=0x7f2db8005860 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:38:58.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.632+0000 7f2dcd600640 1 -- 172.21.15.94:0/1885573874 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2dc005e390 con 0x7f2dd00cf3c0 2024-09-10T23:38:58.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.836+0000 7f2dd7400640 1 -- 172.21.15.94:0/1885573874 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2dd01588c0 con 0x7f2dd00cf3c0 2024-09-10T23:38:58.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.836+0000 7f2dcd600640 1 -- 172.21.15.94:0/1885573874 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v88) ==== 53+0+73 (secure 0 0 0) 0x7f2dc005dc90 con 0x7f2dd00cf3c0 2024-09-10T23:38:58.839 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:38:58.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.840+0000 7f2dd7400640 1 -- 172.21.15.94:0/1885573874 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2dac072c40 msgr2=0x7f2dac075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:58.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.840+0000 7f2dd7400640 1 --2- 172.21.15.94:0/1885573874 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2dac072c40 0x7f2dac075100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2db80058d0 tx=0x7f2db8005860 comp rx=0 tx=0).stop 2024-09-10T23:38:58.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.840+0000 7f2dd7400640 1 -- 172.21.15.94:0/1885573874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd00cf3c0 msgr2=0x7f2dd0142e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:38:58.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.840+0000 7f2dd7400640 1 --2- 172.21.15.94:0/1885573874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd00cf3c0 0x7f2dd0142e50 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000c500 tx=0x7f2dc000c9c0 comp rx=0 tx=0).stop 2024-09-10T23:38:58.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.840+0000 7f2dd7400640 1 -- 172.21.15.94:0/1885573874 shutdown_connections 2024-09-10T23:38:58.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.840+0000 7f2dd7400640 1 --2- 172.21.15.94:0/1885573874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dd00d03c0 0x7f2dd00cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:58.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.840+0000 7f2dd7400640 1 --2- 172.21.15.94:0/1885573874 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2dac072c40 0x7f2dac075100 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:58.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.840+0000 7f2dd7400640 1 --2- 172.21.15.94:0/1885573874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dd015ead0 0x7f2dd00cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:58.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.840+0000 7f2dd7400640 1 --2- 172.21.15.94:0/1885573874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2dd00cf3c0 0x7f2dd0142e50 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:38:58.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.840+0000 7f2dd7400640 1 -- 172.21.15.94:0/1885573874 >> 172.21.15.94:0/1885573874 conn(0x7f2dd006cdf0 msgr2=0x7f2dd00b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:38:58.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.840+0000 7f2dd7400640 1 -- 172.21.15.94:0/1885573874 shutdown_connections 2024-09-10T23:38:58.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:38:58.840+0000 7f2dd7400640 1 -- 172.21.15.94:0/1885573874 wait complete. 2024-09-10T23:38:58.860 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:38:58.861 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:38:58.861 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-09-10T23:38:58.861 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T23:38:58.861 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T23:38:58.861 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:38:58.861 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:38:58.861 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:38:58.861 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:38:58.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:58.861 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-09-10T23:38:58.871 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:38:58.871 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:38:58.871 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:38:58.871 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:58.871 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-09-10T23:38:58.872 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:58.872 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:38:58.872 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:58.872 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-09-10T23:38:58.872 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:38:58.872 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:58.872 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-09-10T23:38:58.872 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:38:58.872 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:58.872 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:38:58.872 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:58.872 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:38:58.873 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42076sec preferred_lft 42076sec 2024-09-10T23:38:58.873 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-09-10T23:38:58.873 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:58.873 INFO:teuthology.orchestra.run.smithi181.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:38:58.873 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 02:42:3b:2e:09:2d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:58.873 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:38:58.873 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:58.873 INFO:teuthology.orchestra.run.smithi181.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:38:58.873 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:38:58.873 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:38:58.873 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:58.873 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:38:58.873 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:58.874 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:38:58.874 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:38:58.874 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:38:58.874 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:58.874 INFO:teuthology.orchestra.run.smithi181.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:38:58.874 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:38:58.874 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:38:58.874 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:38:58.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:58.875 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:38:58.936 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-09-10T23:38:58.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:58.939 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-09-10T23:38:58.993 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-09-10T23:38:58.993 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:38:58.993 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:38:58.993 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:38:58.993 INFO:teuthology.orchestra.run.smithi181.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:38:58.993 INFO:teuthology.orchestra.run.smithi181.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:38:58.993 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:38:58.993 INFO:teuthology.orchestra.run.smithi181.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:38:58.993 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:38:58.994 INFO:teuthology.orchestra.run.smithi181.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:38:58.994 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:38:58.994 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:38:58.994 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:38:58.994 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:38:58.994 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:38:58.994 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:38:58.994 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:38:59.090 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:38:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:38:59.132 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:38:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:38:59.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:59.139 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-09-10T23:38:59.147 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T23:38:59.147 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T23:38:59.148 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:38:59.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:59.148 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-09-10T23:38:59.216 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T23:38:59.216 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:38:59.216 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T23:38:59.268 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:38:59.268 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:38:59.269 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:38:59.323 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:38:59.384 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:38:59.441 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:38:59.442 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:38:59.442 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:38:59.442 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:38:59.442 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:38:59.442 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:38:59.442 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:38:59.442 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:38:59.442 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:38:59.442 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:38:59.442 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:38:59.442 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:38:59.442 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:38:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:38:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:38:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:38:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:38:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:38:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:38:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:38:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:38:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:38:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:38:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:38:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:38:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:38:59.443 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:38:59.444 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:38:59.444 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:38:59.444 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:38:59.444 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:38:59.444 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:38:59.444 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:38:59.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:59.445 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:38:59.500 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:38:59.500 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T23:38:59.543 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:38:59.564 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:38:59.621 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:38:59.622 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:38:59.622 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:38:59.622 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:38:59.622 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:38:59.622 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:38:59.622 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:38:59.622 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:38:59.622 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:38:59.622 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:38:59.623 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:38:59.623 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:38:59.623 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:38:59.623 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:38:59.623 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:38:59.623 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:38:59.623 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:38:59.623 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:38:59.623 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:38:59.623 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:38:59.623 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:38:59.623 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:38:59.624 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:38:59.624 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:38:59.624 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:38:59.624 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:38:59.624 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:38:59.624 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:38:59.624 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:38:59.624 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:38:59.624 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:38:59.624 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:38:59.624 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:38:59.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:59.625 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:38:59.637 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:2024-09-10T23:38:59.636+0000 7f8df3113580 -1 init, newargv = 0x561525fe5d50 newargc=15 2024-09-10T23:38:59.637 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[35866]: starting ceph client 2024-09-10T23:38:59.643 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[35866]: starting fuse 2024-09-10T23:38:59.645 INFO:teuthology.orchestra.run.smithi181.stdout:44 2024-09-10T23:38:59.645 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:38:59.646 DEBUG:teuthology.orchestra.run.smithi181:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:38:59.646 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2024-09-10T23:38:59.646 DEBUG:teuthology.orchestra.run.smithi181:> import re 2024-09-10T23:38:59.646 DEBUG:teuthology.orchestra.run.smithi181:> import os 2024-09-10T23:38:59.646 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2024-09-10T23:38:59.646 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:38:59.646 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2024-09-10T23:38:59.646 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T23:38:59.646 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2024-09-10T23:38:59.646 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T23:38:59.646 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:38:59.646 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2024-09-10T23:38:59.646 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2024-09-10T23:38:59.647 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2024-09-10T23:38:59.647 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2024-09-10T23:38:59.647 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:38:59.647 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2024-09-10T23:38:59.647 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:38:59.647 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:38:59.647 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:38:59.647 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2024-09-10T23:38:59.647 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2024-09-10T23:38:59.647 DEBUG:teuthology.orchestra.run.smithi181:> return f 2024-09-10T23:38:59.647 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:38:59.647 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:38:59.647 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.1")) 2024-09-10T23:38:59.647 DEBUG:teuthology.orchestra.run.smithi181:> ' 2024-09-10T23:38:59.771 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.1.35866.asok 2024-09-10T23:38:59.775 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:38:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:38:59.783 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35866.asok 2024-09-10T23:38:59.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:59.783 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35866.asok status 2024-09-10T23:38:59.952 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T23:38:59.952 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2024-09-10T23:38:59.952 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:38:59.952 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:38:59.952 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "1", 2024-09-10T23:38:59.952 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2024-09-10T23:38:59.952 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T23:38:59.952 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "35866", 2024-09-10T23:38:59.952 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2024-09-10T23:38:59.952 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:38:59.952 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2024-09-10T23:38:59.953 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2024-09-10T23:38:59.953 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 7564, 2024-09-10T23:38:59.953 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2024-09-10T23:38:59.953 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2024-09-10T23:38:59.953 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2024-09-10T23:38:59.953 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 7564 2024-09-10T23:38:59.953 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:38:59.953 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:38:59.953 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:38:59.953 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:38:59.953 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 1680117821 2024-09-10T23:38:59.953 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T23:38:59.953 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:38:59.953 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:38:59.954 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:38:59.954 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:38:59.954 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 1680117821 2024-09-10T23:38:59.954 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:38:59.954 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.7564 192.168.0.1:0/1680117821", 2024-09-10T23:38:59.954 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.1:0/1680117821", 2024-09-10T23:38:59.954 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2024-09-10T23:38:59.954 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 88, 2024-09-10T23:38:59.954 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 127, 2024-09-10T23:38:59.954 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:38:59.954 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2024-09-10T23:38:59.954 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2024-09-10T23:38:59.954 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T23:38:59.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:59.966 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2024-09-10T23:38:59.967 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T23:38:59.975 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2024-09-10T23:38:59.976 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T23:38:59.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:38:59.976 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:39:00.039 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T23:39:00.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.188+0000 7fac33000640 1 -- 172.21.15.94:0/3799301345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac2c0d04b0 msgr2=0x7fac2c15e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:00.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.188+0000 7fac33000640 1 --2- 172.21.15.94:0/3799301345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac2c0d04b0 0x7fac2c15e770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fac18009840 tx=0x7fac1801c710 comp rx=0 tx=0).stop 2024-09-10T23:39:00.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.188+0000 7fac33000640 1 -- 172.21.15.94:0/3799301345 shutdown_connections 2024-09-10T23:39:00.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.188+0000 7fac33000640 1 --2- 172.21.15.94:0/3799301345 >> v2:172.21.15.181:3300/0 conn(0x7fac2c15ecb0 0x7fac2c06f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.188+0000 7fac33000640 1 --2- 172.21.15.94:0/3799301345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac2c0d04b0 0x7fac2c15e770 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.188+0000 7fac33000640 1 -- 172.21.15.94:0/3799301345 >> 172.21.15.94:0/3799301345 conn(0x7fac2c06cdf0 msgr2=0x7fac2c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:00.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.188+0000 7fac33000640 1 -- 172.21.15.94:0/3799301345 shutdown_connections 2024-09-10T23:39:00.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.188+0000 7fac33000640 1 -- 172.21.15.94:0/3799301345 wait complete. 2024-09-10T23:39:00.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac33000640 1 Processor -- start 2024-09-10T23:39:00.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac33000640 1 -- start start 2024-09-10T23:39:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac33000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac2c073ff0 0x7fac2c149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac33000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac2c0d04b0 0x7fac2c14e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac33000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac2c15ecb0 0x7fac2c0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac33000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c167930 con 0x7fac2c15ecb0 2024-09-10T23:39:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac33000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c1677b0 con 0x7fac2c073ff0 2024-09-10T23:39:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac33000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fac2c167ab0 con 0x7fac2c0d04b0 2024-09-10T23:39:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac31000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac2c15ecb0 0x7fac2c0b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac31000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac2c15ecb0 0x7fac2c0b0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55834/0 (socket says 172.21.15.94:55834) 2024-09-10T23:39:00.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac31000640 1 -- 172.21.15.94:0/1161616631 learned_addr learned my addr 172.21.15.94:0/1161616631 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:00.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac2b400640 1 --2- 172.21.15.94:0/1161616631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac2c0d04b0 0x7fac2c14e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac2be00640 1 --2- 172.21.15.94:0/1161616631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac2c073ff0 0x7fac2c149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac31000640 1 -- 172.21.15.94:0/1161616631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac2c0d04b0 msgr2=0x7fac2c14e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac31000640 1 --2- 172.21.15.94:0/1161616631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac2c0d04b0 0x7fac2c14e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac31000640 1 -- 172.21.15.94:0/1161616631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac2c073ff0 msgr2=0x7fac2c149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac31000640 1 --2- 172.21.15.94:0/1161616631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac2c073ff0 0x7fac2c149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac31000640 1 -- 172.21.15.94:0/1161616631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2c149570 con 0x7fac2c15ecb0 2024-09-10T23:39:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac2b400640 1 --2- 172.21.15.94:0/1161616631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac2c0d04b0 0x7fac2c14e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac2be00640 1 --2- 172.21.15.94:0/1161616631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac2c073ff0 0x7fac2c149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac31000640 1 --2- 172.21.15.94:0/1161616631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac2c15ecb0 0x7fac2c0b0580 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fac2400ea80 tx=0x7fac2400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac23e00640 1 -- 172.21.15.94:0/1161616631 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2400ce40 con 0x7fac2c15ecb0 2024-09-10T23:39:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac23e00640 1 -- 172.21.15.94:0/1161616631 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac24004b30 con 0x7fac2c15ecb0 2024-09-10T23:39:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac23e00640 1 -- 172.21.15.94:0/1161616631 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24005020 con 0x7fac2c15ecb0 2024-09-10T23:39:00.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac33000640 1 -- 172.21.15.94:0/1161616631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2c14d7a0 con 0x7fac2c15ecb0 2024-09-10T23:39:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.192+0000 7fac33000640 1 -- 172.21.15.94:0/1161616631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2c14dd60 con 0x7fac2c15ecb0 2024-09-10T23:39:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.196+0000 7fac33000640 1 -- 172.21.15.94:0/1161616631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac2c15e770 con 0x7fac2c15ecb0 2024-09-10T23:39:00.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.196+0000 7fac23e00640 1 -- 172.21.15.94:0/1161616631 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7fac240051c0 con 0x7fac2c15ecb0 2024-09-10T23:39:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.196+0000 7fac23e00640 1 --2- 172.21.15.94:0/1161616631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fac0c072c40 0x7fac0c075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.196+0000 7fac23e00640 1 -- 172.21.15.94:0/1161616631 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7fac24094a70 con 0x7fac2c15ecb0 2024-09-10T23:39:00.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.196+0000 7fac2be00640 1 --2- 172.21.15.94:0/1161616631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fac0c072c40 0x7fac0c075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:00.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.196+0000 7fac2be00640 1 --2- 172.21.15.94:0/1161616631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fac0c072c40 0x7fac0c075100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fac1c001230 tx=0x7fac1c009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:00.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.200+0000 7fac23e00640 1 -- 172.21.15.94:0/1161616631 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fac2405f730 con 0x7fac2c15ecb0 2024-09-10T23:39:00.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.400+0000 7fac33000640 1 -- 172.21.15.94:0/1161616631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fac2c14e3b0 con 0x7fac2c15ecb0 2024-09-10T23:39:00.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.452+0000 7fac23e00640 1 -- 172.21.15.94:0/1161616631 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v89) ==== 136+0+0 (secure 0 0 0) 0x7fac2405f030 con 0x7fac2c15ecb0 2024-09-10T23:39:00.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.456+0000 7fac33000640 1 -- 172.21.15.94:0/1161616631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fac0c072c40 msgr2=0x7fac0c075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:00.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.456+0000 7fac33000640 1 --2- 172.21.15.94:0/1161616631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fac0c072c40 0x7fac0c075100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fac1c001230 tx=0x7fac1c009040 comp rx=0 tx=0).stop 2024-09-10T23:39:00.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.456+0000 7fac33000640 1 -- 172.21.15.94:0/1161616631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac2c15ecb0 msgr2=0x7fac2c0b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:00.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.456+0000 7fac33000640 1 --2- 172.21.15.94:0/1161616631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac2c15ecb0 0x7fac2c0b0580 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fac2400ea80 tx=0x7fac2400ef40 comp rx=0 tx=0).stop 2024-09-10T23:39:00.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.456+0000 7fac33000640 1 -- 172.21.15.94:0/1161616631 shutdown_connections 2024-09-10T23:39:00.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.456+0000 7fac33000640 1 --2- 172.21.15.94:0/1161616631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fac0c072c40 0x7fac0c075100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.456+0000 7fac33000640 1 --2- 172.21.15.94:0/1161616631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac2c15ecb0 0x7fac2c0b0580 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.456+0000 7fac33000640 1 --2- 172.21.15.94:0/1161616631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac2c0d04b0 0x7fac2c14e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.456+0000 7fac33000640 1 --2- 172.21.15.94:0/1161616631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac2c073ff0 0x7fac2c149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.456+0000 7fac33000640 1 -- 172.21.15.94:0/1161616631 >> 172.21.15.94:0/1161616631 conn(0x7fac2c06cdf0 msgr2=0x7fac2c072e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:00.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.456+0000 7fac33000640 1 -- 172.21.15.94:0/1161616631 shutdown_connections 2024-09-10T23:39:00.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.456+0000 7fac33000640 1 -- 172.21.15.94:0/1161616631 wait complete. 2024-09-10T23:39:00.478 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T23:39:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.616+0000 7f5134e00640 1 -- 172.21.15.94:0/3353025173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51300ce890 msgr2=0x7f51300cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.616+0000 7f5134e00640 1 --2- 172.21.15.94:0/3353025173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51300ce890 0x7f51300cd710 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f5120009c80 tx=0x7f512001cce0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.616+0000 7f5134e00640 1 -- 172.21.15.94:0/3353025173 shutdown_connections 2024-09-10T23:39:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.616+0000 7f5134e00640 1 --2- 172.21.15.94:0/3353025173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51300ce1e0 0x7f51300cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.616+0000 7f5134e00640 1 --2- 172.21.15.94:0/3353025173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51300ce890 0x7f51300cd710 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.616+0000 7f5134e00640 1 -- 172.21.15.94:0/3353025173 >> 172.21.15.94:0/3353025173 conn(0x7f51300d1c60 msgr2=0x7f51300d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.616+0000 7f5134e00640 1 -- 172.21.15.94:0/3353025173 shutdown_connections 2024-09-10T23:39:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.616+0000 7f5134e00640 1 -- 172.21.15.94:0/3353025173 wait complete. 2024-09-10T23:39:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f5134e00640 1 Processor -- start 2024-09-10T23:39:00.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f5134e00640 1 -- start start 2024-09-10T23:39:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f5134e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51300b1100 0x7f513015f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f5134e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51300ce1e0 0x7f513014f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f5134e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51300ce890 0x7f51301582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f5134e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51300d0960 con 0x7f51300ce1e0 2024-09-10T23:39:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f5134e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f51300d07e0 con 0x7f51300b1100 2024-09-10T23:39:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f5134e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f51300d0ae0 con 0x7f51300ce890 2024-09-10T23:39:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f512d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51300ce1e0 0x7f513014f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f512d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51300ce1e0 0x7f513014f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55842/0 (socket says 172.21.15.94:55842) 2024-09-10T23:39:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f512d400640 1 -- 172.21.15.94:0/4044752747 learned_addr learned my addr 172.21.15.94:0/4044752747 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f512e800640 1 --2- 172.21.15.94:0/4044752747 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51300ce890 0x7f51301582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f512de00640 1 --2- 172.21.15.94:0/4044752747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51300b1100 0x7f513015f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f512d400640 1 -- 172.21.15.94:0/4044752747 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51300ce890 msgr2=0x7f51301582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f512d400640 1 --2- 172.21.15.94:0/4044752747 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51300ce890 0x7f51301582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f512d400640 1 -- 172.21.15.94:0/4044752747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51300b1100 msgr2=0x7f513015f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f512d400640 1 --2- 172.21.15.94:0/4044752747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51300b1100 0x7f513015f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f512d400640 1 -- 172.21.15.94:0/4044752747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51301fb0b0 con 0x7f51300ce1e0 2024-09-10T23:39:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f512e800640 1 --2- 172.21.15.94:0/4044752747 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51300ce890 0x7f51301582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f512d400640 1 --2- 172.21.15.94:0/4044752747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51300ce1e0 0x7f513014f1a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f511c00c970 tx=0x7f511c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f5126000640 1 -- 172.21.15.94:0/4044752747 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c007c10 con 0x7f51300ce1e0 2024-09-10T23:39:00.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f5126000640 1 -- 172.21.15.94:0/4044752747 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f511c00f450 con 0x7f51300ce1e0 2024-09-10T23:39:00.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f5134e00640 1 -- 172.21.15.94:0/4044752747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51301fb390 con 0x7f51300ce1e0 2024-09-10T23:39:00.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f5126000640 1 -- 172.21.15.94:0/4044752747 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c00f8a0 con 0x7f51300ce1e0 2024-09-10T23:39:00.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f5134e00640 1 -- 172.21.15.94:0/4044752747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51301fb950 con 0x7f51300ce1e0 2024-09-10T23:39:00.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.620+0000 7f5134e00640 1 -- 172.21.15.94:0/4044752747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51300cec70 con 0x7f51300ce1e0 2024-09-10T23:39:00.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.624+0000 7f5126000640 1 -- 172.21.15.94:0/4044752747 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f511c00fa40 con 0x7f51300ce1e0 2024-09-10T23:39:00.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.624+0000 7f5126000640 1 --2- 172.21.15.94:0/4044752747 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5110072c10 0x7f51100750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:00.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.624+0000 7f5126000640 1 -- 172.21.15.94:0/4044752747 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f511c094660 con 0x7f51300ce1e0 2024-09-10T23:39:00.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.624+0000 7f512de00640 1 --2- 172.21.15.94:0/4044752747 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5110072c10 0x7f51100750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:00.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.624+0000 7f512de00640 1 --2- 172.21.15.94:0/4044752747 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5110072c10 0x7f51100750d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f51200af000 tx=0x7f512001b520 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:00.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.628+0000 7f5126000640 1 -- 172.21.15.94:0/4044752747 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f511c014030 con 0x7f51300ce1e0 2024-09-10T23:39:00.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.812+0000 7f5134e00640 1 -- 172.21.15.94:0/4044752747 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f51301fbd50 con 0x7f5110072c10 2024-09-10T23:39:00.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.812+0000 7f5126000640 1 -- 172.21.15.94:0/4044752747 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f51301fbd50 con 0x7f5110072c10 2024-09-10T23:39:00.817 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:39:00.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:00.817 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephfs" 2024-09-10T23:39:00.817 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:00.817 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:39:00.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.816+0000 7f5134e00640 1 -- 172.21.15.94:0/4044752747 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5110072c10 msgr2=0x7f51100750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:00.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.816+0000 7f5134e00640 1 --2- 172.21.15.94:0/4044752747 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5110072c10 0x7f51100750d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f51200af000 tx=0x7f512001b520 comp rx=0 tx=0).stop 2024-09-10T23:39:00.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.816+0000 7f5134e00640 1 -- 172.21.15.94:0/4044752747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51300ce1e0 msgr2=0x7f513014f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:00.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.816+0000 7f5134e00640 1 --2- 172.21.15.94:0/4044752747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51300ce1e0 0x7f513014f1a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f511c00c970 tx=0x7f511c00ce30 comp rx=0 tx=0).stop 2024-09-10T23:39:00.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.816+0000 7f5134e00640 1 -- 172.21.15.94:0/4044752747 shutdown_connections 2024-09-10T23:39:00.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.816+0000 7f5134e00640 1 --2- 172.21.15.94:0/4044752747 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5110072c10 0x7f51100750d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.816+0000 7f5134e00640 1 --2- 172.21.15.94:0/4044752747 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51300ce890 0x7f51301582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.816+0000 7f5134e00640 1 --2- 172.21.15.94:0/4044752747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51300ce1e0 0x7f513014f1a0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.816+0000 7f5134e00640 1 --2- 172.21.15.94:0/4044752747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51300b1100 0x7f513015f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.816+0000 7f5134e00640 1 -- 172.21.15.94:0/4044752747 >> 172.21.15.94:0/4044752747 conn(0x7f51300d1c60 msgr2=0x7f513014fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:00.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.816+0000 7f5134e00640 1 -- 172.21.15.94:0/4044752747 shutdown_connections 2024-09-10T23:39:00.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.816+0000 7f5134e00640 1 -- 172.21.15.94:0/4044752747 wait complete. 2024-09-10T23:39:00.838 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:39:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.980+0000 7f8eeda00640 1 -- 172.21.15.94:0/1375823759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee80cf500 msgr2=0x7f8ee80cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.980+0000 7f8eeda00640 1 --2- 172.21.15.94:0/1375823759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee80cf500 0x7f8ee80cf9b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009840 tx=0x7f8ee001c820 comp rx=0 tx=0).stop 2024-09-10T23:39:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.980+0000 7f8eeda00640 1 -- 172.21.15.94:0/1375823759 shutdown_connections 2024-09-10T23:39:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.980+0000 7f8eeda00640 1 --2- 172.21.15.94:0/1375823759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee80cf500 0x7f8ee80cf9b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.980+0000 7f8eeda00640 1 --2- 172.21.15.94:0/1375823759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ee80ccd60 0x7f8ee80cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.980+0000 7f8eeda00640 1 -- 172.21.15.94:0/1375823759 >> 172.21.15.94:0/1375823759 conn(0x7f8ee8074240 msgr2=0x7f8ee8074620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.980+0000 7f8eeda00640 1 -- 172.21.15.94:0/1375823759 shutdown_connections 2024-09-10T23:39:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.980+0000 7f8eeda00640 1 -- 172.21.15.94:0/1375823759 wait complete. 2024-09-10T23:39:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.980+0000 7f8eeda00640 1 Processor -- start 2024-09-10T23:39:00.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8eeda00640 1 -- start start 2024-09-10T23:39:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8eeda00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee80ccd60 0x7f8ee80b5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8eeda00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ee80cf500 0x7f8ee81579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8eeda00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee81595b0 0x7f8ee80ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8eeda00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8168ed0 con 0x7f8ee80ccd60 2024-09-10T23:39:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8eeda00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8168d50 con 0x7f8ee81595b0 2024-09-10T23:39:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8eeda00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8ee8169050 con 0x7f8ee80cf500 2024-09-10T23:39:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8ee6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee80ccd60 0x7f8ee80b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8ee6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee80ccd60 0x7f8ee80b5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55850/0 (socket says 172.21.15.94:55850) 2024-09-10T23:39:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8ee6a00640 1 -- 172.21.15.94:0/936218650 learned_addr learned my addr 172.21.15.94:0/936218650 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8ee6000640 1 --2- 172.21.15.94:0/936218650 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ee80cf500 0x7f8ee81579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8ee7400640 1 --2- 172.21.15.94:0/936218650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee81595b0 0x7f8ee80ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8ee6a00640 1 -- 172.21.15.94:0/936218650 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ee80cf500 msgr2=0x7f8ee81579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8ee6a00640 1 --2- 172.21.15.94:0/936218650 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ee80cf500 0x7f8ee81579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8ee6a00640 1 -- 172.21.15.94:0/936218650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee81595b0 msgr2=0x7f8ee80ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8ee6a00640 1 --2- 172.21.15.94:0/936218650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee81595b0 0x7f8ee80ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8ee6a00640 1 -- 172.21.15.94:0/936218650 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee8158c20 con 0x7f8ee80ccd60 2024-09-10T23:39:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8ee6a00640 1 --2- 172.21.15.94:0/936218650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee80ccd60 0x7f8ee80b5800 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800c770 tx=0x7f8ed800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8edea00640 1 -- 172.21.15.94:0/936218650 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8003b80 con 0x7f8ee80ccd60 2024-09-10T23:39:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8edea00640 1 -- 172.21.15.94:0/936218650 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ed8003d40 con 0x7f8ee80ccd60 2024-09-10T23:39:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8eeda00640 1 -- 172.21.15.94:0/936218650 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee8158f00 con 0x7f8ee80ccd60 2024-09-10T23:39:00.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8edea00640 1 -- 172.21.15.94:0/936218650 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed800aab0 con 0x7f8ee80ccd60 2024-09-10T23:39:00.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8eeda00640 1 -- 172.21.15.94:0/936218650 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee80c7670 con 0x7f8ee80ccd60 2024-09-10T23:39:00.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.984+0000 7f8eeda00640 1 -- 172.21.15.94:0/936218650 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee80c7870 con 0x7f8ee80ccd60 2024-09-10T23:39:00.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.988+0000 7f8edea00640 1 -- 172.21.15.94:0/936218650 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f8ed800ac50 con 0x7f8ee80ccd60 2024-09-10T23:39:00.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.988+0000 7f8edea00640 1 --2- 172.21.15.94:0/936218650 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ec8072c40 0x7f8ec8075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:00.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.988+0000 7f8edea00640 1 -- 172.21.15.94:0/936218650 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f8ed8093be0 con 0x7f8ee80ccd60 2024-09-10T23:39:00.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.988+0000 7f8ee6000640 1 --2- 172.21.15.94:0/936218650 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ec8072c40 0x7f8ec8075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:00.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.988+0000 7f8ee6000640 1 --2- 172.21.15.94:0/936218650 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ec8072c40 0x7f8ec8075100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80c6920 tx=0x7f8ed4009260 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:00.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:00.992+0000 7f8edea00640 1 -- 172.21.15.94:0/936218650 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8ee80c7870 con 0x7f8ee80ccd60 2024-09-10T23:39:01.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.156+0000 7f8eeda00640 1 -- 172.21.15.94:0/936218650 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8ee80cd1a0 con 0x7f8ee80ccd60 2024-09-10T23:39:01.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.156+0000 7f8edea00640 1 -- 172.21.15.94:0/936218650 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7f8ed805e8e0 con 0x7f8ee80ccd60 2024-09-10T23:39:01.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.160+0000 7f8eeda00640 1 -- 172.21.15.94:0/936218650 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ec8072c40 msgr2=0x7f8ec8075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:01.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.160+0000 7f8eeda00640 1 --2- 172.21.15.94:0/936218650 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ec8072c40 0x7f8ec8075100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80c6920 tx=0x7f8ed4009260 comp rx=0 tx=0).stop 2024-09-10T23:39:01.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.160+0000 7f8eeda00640 1 -- 172.21.15.94:0/936218650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee80ccd60 msgr2=0x7f8ee80b5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:01.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.160+0000 7f8eeda00640 1 --2- 172.21.15.94:0/936218650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee80ccd60 0x7f8ee80b5800 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800c770 tx=0x7f8ed800cc30 comp rx=0 tx=0).stop 2024-09-10T23:39:01.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.160+0000 7f8eeda00640 1 -- 172.21.15.94:0/936218650 shutdown_connections 2024-09-10T23:39:01.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.160+0000 7f8eeda00640 1 --2- 172.21.15.94:0/936218650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ee81595b0 0x7f8ee80ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:01.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.160+0000 7f8eeda00640 1 --2- 172.21.15.94:0/936218650 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ec8072c40 0x7f8ec8075100 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:01.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.160+0000 7f8eeda00640 1 --2- 172.21.15.94:0/936218650 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ee80cf500 0x7f8ee81579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:01.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.160+0000 7f8eeda00640 1 --2- 172.21.15.94:0/936218650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ee80ccd60 0x7f8ee80b5800 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:01.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.160+0000 7f8eeda00640 1 -- 172.21.15.94:0/936218650 >> 172.21.15.94:0/936218650 conn(0x7f8ee8074240 msgr2=0x7f8ee80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:01.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.160+0000 7f8eeda00640 1 -- 172.21.15.94:0/936218650 shutdown_connections 2024-09-10T23:39:01.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.160+0000 7f8eeda00640 1 -- 172.21.15.94:0/936218650 wait complete. 2024-09-10T23:39:01.183 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create non_existent_subvolume subvolume_1 2024-09-10T23:39:01.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.324+0000 7fc143c00640 1 -- 172.21.15.94:0/55804622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc120004b20 msgr2=0x7fc120004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:01.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.324+0000 7fc143c00640 1 --2- 172.21.15.94:0/55804622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc120004b20 0x7fc120004f00 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0040c0 tx=0x7fc12c01d8d0 comp rx=0 tx=0).stop 2024-09-10T23:39:01.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.324+0000 7fc143c00640 1 -- 172.21.15.94:0/55804622 shutdown_connections 2024-09-10T23:39:01.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.324+0000 7fc143c00640 1 --2- 172.21.15.94:0/55804622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc120004b20 0x7fc120004f00 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:01.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.324+0000 7fc143c00640 1 -- 172.21.15.94:0/55804622 >> 172.21.15.94:0/55804622 conn(0x7fc13c0cfaf0 msgr2=0x7fc13c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:01.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.324+0000 7fc143c00640 1 -- 172.21.15.94:0/55804622 shutdown_connections 2024-09-10T23:39:01.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.324+0000 7fc143c00640 1 -- 172.21.15.94:0/55804622 wait complete. 2024-09-10T23:39:01.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc143c00640 1 Processor -- start 2024-09-10T23:39:01.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc143c00640 1 -- start start 2024-09-10T23:39:01.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc143c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc120004b20 0x7fc13c159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:01.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc143c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc13c159c40 0x7fc13c0b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:01.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc143c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc13c0b2f80 0x7fc13c0b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:01.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc143c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c06e940 con 0x7fc120004b20 2024-09-10T23:39:01.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc143c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c06e7c0 con 0x7fc13c159c40 2024-09-10T23:39:01.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc143c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc13c06eac0 con 0x7fc13c0b2f80 2024-09-10T23:39:01.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc141200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc120004b20 0x7fc13c159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:01.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc141200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc120004b20 0x7fc13c159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55868/0 (socket says 172.21.15.94:55868) 2024-09-10T23:39:01.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc141200640 1 -- 172.21.15.94:0/498722775 learned_addr learned my addr 172.21.15.94:0/498722775 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:01.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc141c00640 1 --2- 172.21.15.94:0/498722775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc13c0b2f80 0x7fc13c0b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:01.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc133e00640 1 --2- 172.21.15.94:0/498722775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc13c159c40 0x7fc13c0b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:01.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc141200640 1 -- 172.21.15.94:0/498722775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc13c0b2f80 msgr2=0x7fc13c0b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:01.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc141200640 1 --2- 172.21.15.94:0/498722775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc13c0b2f80 0x7fc13c0b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:01.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc141200640 1 -- 172.21.15.94:0/498722775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc13c159c40 msgr2=0x7fc13c0b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:01.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc141200640 1 --2- 172.21.15.94:0/498722775 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc13c159c40 0x7fc13c0b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:01.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc141200640 1 -- 172.21.15.94:0/498722775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13c0b55a0 con 0x7fc120004b20 2024-09-10T23:39:01.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc141c00640 1 --2- 172.21.15.94:0/498722775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc13c0b2f80 0x7fc13c0b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:01.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc141200640 1 --2- 172.21.15.94:0/498722775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc120004b20 0x7fc13c159700 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc12c000c10 tx=0x7fc12c0078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:01.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc139600640 1 -- 172.21.15.94:0/498722775 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c007520 con 0x7fc120004b20 2024-09-10T23:39:01.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc139600640 1 -- 172.21.15.94:0/498722775 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc12c0b5020 con 0x7fc120004b20 2024-09-10T23:39:01.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc143c00640 1 -- 172.21.15.94:0/498722775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc13c0b5880 con 0x7fc120004b20 2024-09-10T23:39:01.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc139600640 1 -- 172.21.15.94:0/498722775 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c0018e0 con 0x7fc120004b20 2024-09-10T23:39:01.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc143c00640 1 -- 172.21.15.94:0/498722775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc13c0b4d60 con 0x7fc120004b20 2024-09-10T23:39:01.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.328+0000 7fc143c00640 1 -- 172.21.15.94:0/498722775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc13c06eee0 con 0x7fc120004b20 2024-09-10T23:39:01.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.332+0000 7fc139600640 1 -- 172.21.15.94:0/498722775 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7fc12c001500 con 0x7fc120004b20 2024-09-10T23:39:01.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.332+0000 7fc139600640 1 --2- 172.21.15.94:0/498722775 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc118072c00 0x7fc1180750c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:01.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.332+0000 7fc139600640 1 -- 172.21.15.94:0/498722775 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7fc12c12de20 con 0x7fc120004b20 2024-09-10T23:39:01.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.332+0000 7fc133e00640 1 --2- 172.21.15.94:0/498722775 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc118072c00 0x7fc1180750c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:01.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.332+0000 7fc133e00640 1 --2- 172.21.15.94:0/498722775 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc118072c00 0x7fc1180750c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc13c157260 tx=0x7fc124006e80 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:01.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.336+0000 7fc139600640 1 -- 172.21.15.94:0/498722775 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc12c0bf030 con 0x7fc120004b20 2024-09-10T23:39:01.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.528+0000 7fc143c00640 1 -- 172.21.15.94:0/498722775 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7fc13c0d0850 con 0x7fc118072c00 2024-09-10T23:39:01.530 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:01.528+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.528+0000 7fc139600640 1 -- 172.21.15.94:0/498722775 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fc13c0d0850 con 0x7fc118072c00 2024-09-10T23:39:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:01.540 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:01.541 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm non_existent_subvolume subvolume_1 2024-09-10T23:39:01.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9537800640 1 -- 172.21.15.94:0/1213327035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95300cb4d0 msgr2=0x7f953015b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:01.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9537800640 1 --2- 172.21.15.94:0/1213327035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95300cb4d0 0x7f953015b190 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9528009bc0 tx=0x7f952801cba0 comp rx=0 tx=0).stop 2024-09-10T23:39:01.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9537800640 1 -- 172.21.15.94:0/1213327035 shutdown_connections 2024-09-10T23:39:01.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9537800640 1 --2- 172.21.15.94:0/1213327035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95300cb4d0 0x7f953015b190 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:01.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9537800640 1 --2- 172.21.15.94:0/1213327035 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9530158790 0x7f953006fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:01.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9537800640 1 -- 172.21.15.94:0/1213327035 >> 172.21.15.94:0/1213327035 conn(0x7f95300cfaf0 msgr2=0x7f95300cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:01.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9537800640 1 -- 172.21.15.94:0/1213327035 shutdown_connections 2024-09-10T23:39:01.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9537800640 1 -- 172.21.15.94:0/1213327035 wait complete. 2024-09-10T23:39:01.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9537800640 1 Processor -- start 2024-09-10T23:39:01.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9537800640 1 -- start start 2024-09-10T23:39:01.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9537800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95300c49b0 0x7f95301f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:01.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9537800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95300cb4d0 0x7f95300b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:01.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9537800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9530158790 0x7f9530156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:01.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9537800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9530167f60 con 0x7f9530158790 2024-09-10T23:39:01.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9537800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9530167de0 con 0x7f95300cb4d0 2024-09-10T23:39:01.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9537800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f95301680e0 con 0x7f95300c49b0 2024-09-10T23:39:01.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9535800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9530158790 0x7f9530156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:01.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9535800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9530158790 0x7f9530156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55876/0 (socket says 172.21.15.94:55876) 2024-09-10T23:39:01.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9535800640 1 -- 172.21.15.94:0/2023581124 learned_addr learned my addr 172.21.15.94:0/2023581124 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:01.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9534e00640 1 --2- 172.21.15.94:0/2023581124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95300c49b0 0x7f95301f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:01.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.684+0000 7f9527e00640 1 --2- 172.21.15.94:0/2023581124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95300cb4d0 0x7f95300b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:01.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.688+0000 7f9535800640 1 -- 172.21.15.94:0/2023581124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95300c49b0 msgr2=0x7f95301f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:01.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.688+0000 7f9535800640 1 --2- 172.21.15.94:0/2023581124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95300c49b0 0x7f95301f8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:01.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.688+0000 7f9535800640 1 -- 172.21.15.94:0/2023581124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95300cb4d0 msgr2=0x7f95300b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:01.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.688+0000 7f9535800640 1 --2- 172.21.15.94:0/2023581124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95300cb4d0 0x7f95300b5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:01.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.688+0000 7f9535800640 1 -- 172.21.15.94:0/2023581124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95301fb4c0 con 0x7f9530158790 2024-09-10T23:39:01.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.688+0000 7f9527e00640 1 --2- 172.21.15.94:0/2023581124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95300cb4d0 0x7f95300b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:01.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.688+0000 7f9535800640 1 --2- 172.21.15.94:0/2023581124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9530158790 0x7f9530156560 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f9528009b10 tx=0x7f9528009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:01.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.688+0000 7f952d600640 1 -- 172.21.15.94:0/2023581124 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95280b2040 con 0x7f9530158790 2024-09-10T23:39:01.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.688+0000 7f952d600640 1 -- 172.21.15.94:0/2023581124 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9528004b40 con 0x7f9530158790 2024-09-10T23:39:01.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.688+0000 7f9537800640 1 -- 172.21.15.94:0/2023581124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95301fb7a0 con 0x7f9530158790 2024-09-10T23:39:01.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.688+0000 7f952d600640 1 -- 172.21.15.94:0/2023581124 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528004000 con 0x7f9530158790 2024-09-10T23:39:01.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.688+0000 7f9537800640 1 -- 172.21.15.94:0/2023581124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95301fbd60 con 0x7f9530158790 2024-09-10T23:39:01.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.688+0000 7f9537800640 1 -- 172.21.15.94:0/2023581124 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95301fc140 con 0x7f9530158790 2024-09-10T23:39:01.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.688+0000 7f952d600640 1 -- 172.21.15.94:0/2023581124 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f95280b5070 con 0x7f9530158790 2024-09-10T23:39:01.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.688+0000 7f952d600640 1 --2- 172.21.15.94:0/2023581124 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f950c072c10 0x7f950c0750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:01.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.688+0000 7f952d600640 1 -- 172.21.15.94:0/2023581124 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f952812d920 con 0x7f9530158790 2024-09-10T23:39:01.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.692+0000 7f9534e00640 1 --2- 172.21.15.94:0/2023581124 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f950c072c10 0x7f950c0750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:01.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.692+0000 7f9534e00640 1 --2- 172.21.15.94:0/2023581124 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f950c072c10 0x7f950c0750d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9520009b40 tx=0x7f95200091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:01.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.692+0000 7f952d600640 1 -- 172.21.15.94:0/2023581124 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f95301fc140 con 0x7f9530158790 2024-09-10T23:39:01.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.884+0000 7f9537800640 1 -- 172.21.15.94:0/2023581124 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f95301fb270 con 0x7f950c072c10 2024-09-10T23:39:01.888 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:01.884+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:01.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:01.884+0000 7f952d600640 1 -- 172.21.15.94:0/2023581124 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f95301fb270 con 0x7f950c072c10 2024-09-10T23:39:01.889 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:01.898 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:01.899 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath non_existent_subvolume subvolume_1 2024-09-10T23:39:02.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.040+0000 7f7353a00640 1 -- 172.21.15.94:0/492647354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f734c161600 msgr2=0x7f734c06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:02.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.040+0000 7f7353a00640 1 --2- 172.21.15.94:0/492647354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f734c161600 0x7f734c06dd30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f733c009840 tx=0x7f733c01c820 comp rx=0 tx=0).stop 2024-09-10T23:39:02.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.040+0000 7f7353a00640 1 -- 172.21.15.94:0/492647354 shutdown_connections 2024-09-10T23:39:02.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.040+0000 7f7353a00640 1 --2- 172.21.15.94:0/492647354 >> v2:172.21.15.181:3300/0 conn(0x7f734c0d0de0 0x7f734c06e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:02.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.040+0000 7f7353a00640 1 --2- 172.21.15.94:0/492647354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f734c161600 0x7f734c06dd30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:02.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.040+0000 7f7353a00640 1 -- 172.21.15.94:0/492647354 >> 172.21.15.94:0/492647354 conn(0x7f734c0d1c60 msgr2=0x7f734c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:02.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.040+0000 7f7353a00640 1 -- 172.21.15.94:0/492647354 shutdown_connections 2024-09-10T23:39:02.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.040+0000 7f7353a00640 1 -- 172.21.15.94:0/492647354 wait complete. 2024-09-10T23:39:02.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7353a00640 1 Processor -- start 2024-09-10T23:39:02.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7353a00640 1 -- start start 2024-09-10T23:39:02.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7353a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f734c0b2960 0x7f734c155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:02.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7353a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f734c0d0de0 0x7f734c15b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:02.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7353a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f734c161600 0x7f734c15b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:02.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7353a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f734c16aed0 con 0x7f734c0b2960 2024-09-10T23:39:02.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7353a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f734c16ad50 con 0x7f734c161600 2024-09-10T23:39:02.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7353a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f734c16b050 con 0x7f734c0d0de0 2024-09-10T23:39:02.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7351000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f734c0b2960 0x7f734c155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:02.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7351000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f734c0b2960 0x7f734c155ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55888/0 (socket says 172.21.15.94:55888) 2024-09-10T23:39:02.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7351000640 1 -- 172.21.15.94:0/4063243664 learned_addr learned my addr 172.21.15.94:0/4063243664 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:02.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7351a00640 1 --2- 172.21.15.94:0/4063243664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f734c161600 0x7f734c15b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:02.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f734be00640 1 --2- 172.21.15.94:0/4063243664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f734c0d0de0 0x7f734c15b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:02.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7351000640 1 -- 172.21.15.94:0/4063243664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f734c0d0de0 msgr2=0x7f734c15b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:02.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7351000640 1 --2- 172.21.15.94:0/4063243664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f734c0d0de0 0x7f734c15b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:02.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7351000640 1 -- 172.21.15.94:0/4063243664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f734c161600 msgr2=0x7f734c15b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:02.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7351000640 1 --2- 172.21.15.94:0/4063243664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f734c161600 0x7f734c15b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:02.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7351000640 1 -- 172.21.15.94:0/4063243664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c159d50 con 0x7f734c0b2960 2024-09-10T23:39:02.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7351a00640 1 --2- 172.21.15.94:0/4063243664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f734c161600 0x7f734c15b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:02.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f734be00640 1 --2- 172.21.15.94:0/4063243664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f734c0d0de0 0x7f734c15b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:02.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7351000640 1 --2- 172.21.15.94:0/4063243664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f734c0b2960 0x7f734c155ab0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f733c098200 tx=0x7f733c0a6570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:02.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7348c00640 1 -- 172.21.15.94:0/4063243664 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c004040 con 0x7f734c0b2960 2024-09-10T23:39:02.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7348c00640 1 -- 172.21.15.94:0/4063243664 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f733c0042d0 con 0x7f734c0b2960 2024-09-10T23:39:02.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7348c00640 1 -- 172.21.15.94:0/4063243664 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c003600 con 0x7f734c0b2960 2024-09-10T23:39:02.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7353a00640 1 -- 172.21.15.94:0/4063243664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734c15a030 con 0x7f734c0b2960 2024-09-10T23:39:02.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.044+0000 7f7353a00640 1 -- 172.21.15.94:0/4063243664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c15a5f0 con 0x7f734c0b2960 2024-09-10T23:39:02.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.048+0000 7f7353a00640 1 -- 172.21.15.94:0/4063243664 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f734c0b5cf0 con 0x7f734c0b2960 2024-09-10T23:39:02.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.048+0000 7f7348c00640 1 -- 172.21.15.94:0/4063243664 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f733c0037a0 con 0x7f734c0b2960 2024-09-10T23:39:02.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.048+0000 7f7348c00640 1 --2- 172.21.15.94:0/4063243664 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f732c072c10 0x7f732c0750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:02.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.048+0000 7f7348c00640 1 -- 172.21.15.94:0/4063243664 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f733c12efa0 con 0x7f734c0b2960 2024-09-10T23:39:02.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.048+0000 7f734be00640 1 --2- 172.21.15.94:0/4063243664 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f732c072c10 0x7f732c0750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:02.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.048+0000 7f734be00640 1 --2- 172.21.15.94:0/4063243664 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f732c072c10 0x7f732c0750d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7338007fd0 tx=0x7f7338009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:02.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.052+0000 7f7348c00640 1 -- 172.21.15.94:0/4063243664 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f733c016410 con 0x7f734c0b2960 2024-09-10T23:39:02.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.244+0000 7f7353a00640 1 -- 172.21.15.94:0/4063243664 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f734c1fae80 con 0x7f732c072c10 2024-09-10T23:39:02.248 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:02.244+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:02.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.244+0000 7f7348c00640 1 -- 172.21.15.94:0/4063243664 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f7338001420 con 0x7f732c072c10 2024-09-10T23:39:02.248 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:02.258 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:02.258 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info non_existent_subvolume subvolume_1 2024-09-10T23:39:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.400+0000 7faf1ce00640 1 -- 172.21.15.94:0/2225372266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf1806ce80 msgr2=0x7faf1815f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.400+0000 7faf1ce00640 1 --2- 172.21.15.94:0/2225372266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf1806ce80 0x7faf1815f820 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7faf18153320 tx=0x7faf0801cd50 comp rx=0 tx=0).stop 2024-09-10T23:39:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.400+0000 7faf1ce00640 1 -- 172.21.15.94:0/2225372266 shutdown_connections 2024-09-10T23:39:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.400+0000 7faf1ce00640 1 --2- 172.21.15.94:0/2225372266 >> v2:172.21.15.181:3300/0 conn(0x7faf1806d260 0x7faf1815ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.400+0000 7faf1ce00640 1 --2- 172.21.15.94:0/2225372266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf1806ce80 0x7faf1815f820 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.400+0000 7faf1ce00640 1 -- 172.21.15.94:0/2225372266 >> 172.21.15.94:0/2225372266 conn(0x7faf180d0d80 msgr2=0x7faf180d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:02.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.400+0000 7faf1ce00640 1 -- 172.21.15.94:0/2225372266 shutdown_connections 2024-09-10T23:39:02.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.400+0000 7faf1ce00640 1 -- 172.21.15.94:0/2225372266 wait complete. 2024-09-10T23:39:02.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.400+0000 7faf1ce00640 1 Processor -- start 2024-09-10T23:39:02.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.400+0000 7faf1ce00640 1 -- start start 2024-09-10T23:39:02.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf1ce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf1806ce80 0x7faf18148a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:02.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf1ce00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf1806d260 0x7faf181f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:02.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf1ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf18155be0 0x7faf1813a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:02.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf1ce00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf18141860 con 0x7faf1806ce80 2024-09-10T23:39:02.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf1ce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faf181416e0 con 0x7faf18155be0 2024-09-10T23:39:02.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf1ce00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7faf181419e0 con 0x7faf1806d260 2024-09-10T23:39:02.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf15e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf1806ce80 0x7faf18148a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:02.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf15e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf1806ce80 0x7faf18148a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55914/0 (socket says 172.21.15.94:55914) 2024-09-10T23:39:02.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf15e00640 1 -- 172.21.15.94:0/202383381 learned_addr learned my addr 172.21.15.94:0/202383381 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:02.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf16800640 1 --2- 172.21.15.94:0/202383381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf18155be0 0x7faf1813a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:02.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf0d400640 1 --2- 172.21.15.94:0/202383381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf1806d260 0x7faf181f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:02.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf15e00640 1 -- 172.21.15.94:0/202383381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf1806d260 msgr2=0x7faf181f7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:02.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf15e00640 1 --2- 172.21.15.94:0/202383381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf1806d260 0x7faf181f7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:02.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf15e00640 1 -- 172.21.15.94:0/202383381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf18155be0 msgr2=0x7faf1813a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:02.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf15e00640 1 --2- 172.21.15.94:0/202383381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf18155be0 0x7faf1813a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:02.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf15e00640 1 -- 172.21.15.94:0/202383381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1813bcd0 con 0x7faf1806ce80 2024-09-10T23:39:02.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf16800640 1 --2- 172.21.15.94:0/202383381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf18155be0 0x7faf1813a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:02.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf0d400640 1 --2- 172.21.15.94:0/202383381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf1806d260 0x7faf181f7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:02.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf15e00640 1 --2- 172.21.15.94:0/202383381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf1806ce80 0x7faf18148a40 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7faf08000f90 tx=0x7faf080a5ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:02.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf0ea00640 1 -- 172.21.15.94:0/202383381 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf080bb070 con 0x7faf1806ce80 2024-09-10T23:39:02.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf0ea00640 1 -- 172.21.15.94:0/202383381 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf08004c50 con 0x7faf1806ce80 2024-09-10T23:39:02.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf1ce00640 1 -- 172.21.15.94:0/202383381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1815dc20 con 0x7faf1806ce80 2024-09-10T23:39:02.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf0ea00640 1 -- 172.21.15.94:0/202383381 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf080b2040 con 0x7faf1806ce80 2024-09-10T23:39:02.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf1ce00640 1 -- 172.21.15.94:0/202383381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1815e1e0 con 0x7faf1806ce80 2024-09-10T23:39:02.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf1ce00640 1 -- 172.21.15.94:0/202383381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf18141f80 con 0x7faf1806ce80 2024-09-10T23:39:02.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.404+0000 7faf0ea00640 1 -- 172.21.15.94:0/202383381 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7faf080b5030 con 0x7faf1806ce80 2024-09-10T23:39:02.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.408+0000 7faf0ea00640 1 --2- 172.21.15.94:0/202383381 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faef4072c10 0x7faef40750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:02.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.408+0000 7faf0ea00640 1 -- 172.21.15.94:0/202383381 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7faf0812e090 con 0x7faf1806ce80 2024-09-10T23:39:02.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.408+0000 7faf0d400640 1 --2- 172.21.15.94:0/202383381 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faef4072c10 0x7faef40750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:02.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.408+0000 7faf0d400640 1 --2- 172.21.15.94:0/202383381 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7faef4072c10 0x7faef40750d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faf1815d670 tx=0x7faf00008040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:02.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.408+0000 7faf0ea00640 1 -- 172.21.15.94:0/202383381 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faf08016970 con 0x7faf1806ce80 2024-09-10T23:39:02.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.600+0000 7faf1ce00640 1 -- 172.21.15.94:0/202383381 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7faf18148f80 con 0x7faef4072c10 2024-09-10T23:39:02.605 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:02.604+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:02.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.604+0000 7faf0ea00640 1 -- 172.21.15.94:0/202383381 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7faf0000ae00 con 0x7faef4072c10 2024-09-10T23:39:02.606 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:02.614 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:02.615 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize non_existent_subvolume subvolname_1 inf 2024-09-10T23:39:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2fe00640 1 -- 172.21.15.94:0/3009764208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e300d1f00 msgr2=0x7f6e300ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2fe00640 1 --2- 172.21.15.94:0/3009764208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e300d1f00 0x7f6e300ce6f0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009c80 tx=0x7f6e2001cda0 comp rx=0 tx=0).stop 2024-09-10T23:39:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2fe00640 1 -- 172.21.15.94:0/3009764208 shutdown_connections 2024-09-10T23:39:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2fe00640 1 --2- 172.21.15.94:0/3009764208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e300d1f00 0x7f6e300ce6f0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2fe00640 1 -- 172.21.15.94:0/3009764208 >> 172.21.15.94:0/3009764208 conn(0x7f6e3009d1c0 msgr2=0x7f6e300d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2fe00640 1 -- 172.21.15.94:0/3009764208 shutdown_connections 2024-09-10T23:39:02.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2fe00640 1 -- 172.21.15.94:0/3009764208 wait complete. 2024-09-10T23:39:02.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2fe00640 1 Processor -- start 2024-09-10T23:39:02.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2fe00640 1 -- start start 2024-09-10T23:39:02.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2fe00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e300b3f30 0x7f6e3015fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:02.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e300cec30 0x7f6e3014d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:02.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e300d1f00 0x7f6e301fab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:02.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3016bec0 con 0x7f6e300d1f00 2024-09-10T23:39:02.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2fe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3016bd40 con 0x7f6e300cec30 2024-09-10T23:39:02.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2fe00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6e3016c040 con 0x7f6e300b3f30 2024-09-10T23:39:02.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e300d1f00 0x7f6e301fab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:02.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e300d1f00 0x7f6e301fab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55926/0 (socket says 172.21.15.94:55926) 2024-09-10T23:39:02.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2f400640 1 -- 172.21.15.94:0/1729283668 learned_addr learned my addr 172.21.15.94:0/1729283668 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:02.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e26000640 1 --2- 172.21.15.94:0/1729283668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e300cec30 0x7f6e3014d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:02.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.760+0000 7f6e2ea00640 1 --2- 172.21.15.94:0/1729283668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e300b3f30 0x7f6e3015fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:02.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.764+0000 7f6e2f400640 1 -- 172.21.15.94:0/1729283668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e300b3f30 msgr2=0x7f6e3015fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:02.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.764+0000 7f6e2f400640 1 --2- 172.21.15.94:0/1729283668 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e300b3f30 0x7f6e3015fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:02.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.764+0000 7f6e2f400640 1 -- 172.21.15.94:0/1729283668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e300cec30 msgr2=0x7f6e3014d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:02.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.764+0000 7f6e2f400640 1 --2- 172.21.15.94:0/1729283668 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e300cec30 0x7f6e3014d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:02.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.764+0000 7f6e2f400640 1 -- 172.21.15.94:0/1729283668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e30160090 con 0x7f6e300d1f00 2024-09-10T23:39:02.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.764+0000 7f6e2f400640 1 --2- 172.21.15.94:0/1729283668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e300d1f00 0x7f6e301fab50 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800c3c0 tx=0x7f6e2800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:02.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.764+0000 7f6e27400640 1 -- 172.21.15.94:0/1729283668 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28011790 con 0x7f6e300d1f00 2024-09-10T23:39:02.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.764+0000 7f6e27400640 1 -- 172.21.15.94:0/1729283668 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e28017070 con 0x7f6e300d1f00 2024-09-10T23:39:02.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.764+0000 7f6e2fe00640 1 -- 172.21.15.94:0/1729283668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3014d920 con 0x7f6e300d1f00 2024-09-10T23:39:02.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.764+0000 7f6e27400640 1 -- 172.21.15.94:0/1729283668 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28011cc0 con 0x7f6e300d1f00 2024-09-10T23:39:02.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.764+0000 7f6e2fe00640 1 -- 172.21.15.94:0/1729283668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e30161190 con 0x7f6e300d1f00 2024-09-10T23:39:02.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.764+0000 7f6e2fe00640 1 -- 172.21.15.94:0/1729283668 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e300ce6f0 con 0x7f6e300d1f00 2024-09-10T23:39:02.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.764+0000 7f6e27400640 1 -- 172.21.15.94:0/1729283668 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f6e2801c460 con 0x7f6e300d1f00 2024-09-10T23:39:02.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.768+0000 7f6e27400640 1 --2- 172.21.15.94:0/1729283668 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6e0c072c10 0x7f6e0c0750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:02.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.768+0000 7f6e27400640 1 -- 172.21.15.94:0/1729283668 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f6e28093bb0 con 0x7f6e300d1f00 2024-09-10T23:39:02.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.768+0000 7f6e2ea00640 1 --2- 172.21.15.94:0/1729283668 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6e0c072c10 0x7f6e0c0750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:02.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.768+0000 7f6e2ea00640 1 --2- 172.21.15.94:0/1729283668 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6e0c072c10 0x7f6e0c0750d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6e200af000 tx=0x7f6e2001b5e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:02.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.772+0000 7f6e27400640 1 -- 172.21.15.94:0/1729283668 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6e2805e220 con 0x7f6e300d1f00 2024-09-10T23:39:02.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.964+0000 7f6e2fe00640 1 -- 172.21.15.94:0/1729283668 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "new_size": "inf", "target": ["mon-mgr", ""]}) -- 0x7f6e3015a380 con 0x7f6e0c072c10 2024-09-10T23:39:02.970 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:02.968+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:02.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:02.968+0000 7f6e27400640 1 -- 172.21.15.94:0/1729283668 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f6e3015a380 con 0x7f6e0c072c10 2024-09-10T23:39:02.970 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:02.979 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:02.980 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume pin non_existent_subvolume subvolname_1 export 1 2024-09-10T23:39:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.124+0000 7f60c0000640 1 -- 172.21.15.94:0/3888621423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a0003a60 msgr2=0x7f60a0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.124+0000 7f60c0000640 1 --2- 172.21.15.94:0/3888621423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a0003a60 0x7f60a0003e40 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f60ac006f80 tx=0x7f60ac01cec0 comp rx=0 tx=0).stop 2024-09-10T23:39:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.124+0000 7f60c0000640 1 -- 172.21.15.94:0/3888621423 shutdown_connections 2024-09-10T23:39:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.124+0000 7f60c0000640 1 --2- 172.21.15.94:0/3888621423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60a0003a60 0x7f60a0003e40 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.124+0000 7f60c0000640 1 -- 172.21.15.94:0/3888621423 >> 172.21.15.94:0/3888621423 conn(0x7f60b806cdf0 msgr2=0x7f60b806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.124+0000 7f60c0000640 1 -- 172.21.15.94:0/3888621423 shutdown_connections 2024-09-10T23:39:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.124+0000 7f60c0000640 1 -- 172.21.15.94:0/3888621423 wait complete. 2024-09-10T23:39:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60c0000640 1 Processor -- start 2024-09-10T23:39:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60c0000640 1 -- start start 2024-09-10T23:39:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60c0000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60a0003a60 0x7f60b80b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60c0000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b80c71e0 0x7f60b80c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60c0000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b81455b0 0x7f60b8145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60c0000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60b806fe40 con 0x7f60b80c71e0 2024-09-10T23:39:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60c0000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f60b806fcc0 con 0x7f60b81455b0 2024-09-10T23:39:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60c0000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f60b806ffc0 con 0x7f60a0003a60 2024-09-10T23:39:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60bcc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b80c71e0 0x7f60b80c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60bcc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b80c71e0 0x7f60b80c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55946/0 (socket says 172.21.15.94:55946) 2024-09-10T23:39:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60bcc00640 1 -- 172.21.15.94:0/309490307 learned_addr learned my addr 172.21.15.94:0/309490307 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60bd600640 1 --2- 172.21.15.94:0/309490307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60a0003a60 0x7f60b80b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60be000640 1 --2- 172.21.15.94:0/309490307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b81455b0 0x7f60b8145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60bcc00640 1 -- 172.21.15.94:0/309490307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60a0003a60 msgr2=0x7f60b80b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60bcc00640 1 --2- 172.21.15.94:0/309490307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60a0003a60 0x7f60b80b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60bcc00640 1 -- 172.21.15.94:0/309490307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b81455b0 msgr2=0x7f60b8145a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60bcc00640 1 --2- 172.21.15.94:0/309490307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b81455b0 0x7f60b8145a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60bcc00640 1 -- 172.21.15.94:0/309490307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b806d1d0 con 0x7f60b80c71e0 2024-09-10T23:39:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60bd600640 1 --2- 172.21.15.94:0/309490307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60a0003a60 0x7f60b80b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60bcc00640 1 --2- 172.21.15.94:0/309490307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b80c71e0 0x7f60b80c7660 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f60a4003ee0 tx=0x7f60a4001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60b5600640 1 -- 172.21.15.94:0/309490307 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4011700 con 0x7f60b80c71e0 2024-09-10T23:39:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60b5600640 1 -- 172.21.15.94:0/309490307 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60a4016020 con 0x7f60b80c71e0 2024-09-10T23:39:03.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60b5600640 1 -- 172.21.15.94:0/309490307 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4011b60 con 0x7f60b80c71e0 2024-09-10T23:39:03.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60c0000640 1 -- 172.21.15.94:0/309490307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b80c7ba0 con 0x7f60b80c71e0 2024-09-10T23:39:03.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60c0000640 1 -- 172.21.15.94:0/309490307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b80b0100 con 0x7f60b80c71e0 2024-09-10T23:39:03.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.128+0000 7f60c0000640 1 -- 172.21.15.94:0/309490307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b80b35f0 con 0x7f60b80c71e0 2024-09-10T23:39:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.132+0000 7f60b5600640 1 -- 172.21.15.94:0/309490307 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f60a401b410 con 0x7f60b80c71e0 2024-09-10T23:39:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.132+0000 7f60b5600640 1 --2- 172.21.15.94:0/309490307 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6098072c40 0x7f6098075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.132+0000 7f60b5600640 1 -- 172.21.15.94:0/309490307 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f60a4094ae0 con 0x7f60b80c71e0 2024-09-10T23:39:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.132+0000 7f60bd600640 1 --2- 172.21.15.94:0/309490307 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6098072c40 0x7f6098075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.132+0000 7f60bd600640 1 --2- 172.21.15.94:0/309490307 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6098072c40 0x7f6098075100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0aa000 tx=0x7f60ac019d00 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:03.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.136+0000 7f60b5600640 1 -- 172.21.15.94:0/309490307 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f60a4020020 con 0x7f60b80c71e0 2024-09-10T23:39:03.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.332+0000 7f60c0000640 1 -- 172.21.15.94:0/309490307 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume pin", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "pin_type": "export", "pin_setting": "1", "target": ["mon-mgr", ""]}) -- 0x7f60b8073270 con 0x7f6098072c40 2024-09-10T23:39:03.337 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:03.336+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:03.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.336+0000 7f60b5600640 1 -- 172.21.15.94:0/309490307 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f60b8073270 con 0x7f6098072c40 2024-09-10T23:39:03.338 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:03.348 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:03.348 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls non_existent_subvolume 2024-09-10T23:39:03.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.492+0000 7f825cc00640 1 -- 172.21.15.94:0/1105028586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8258158790 msgr2=0x7f825806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:03.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.492+0000 7f825cc00640 1 --2- 172.21.15.94:0/1105028586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8258158790 0x7f825806fc00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8248009840 tx=0x7f824801c820 comp rx=0 tx=0).stop 2024-09-10T23:39:03.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.492+0000 7f825cc00640 1 -- 172.21.15.94:0/1105028586 shutdown_connections 2024-09-10T23:39:03.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.492+0000 7f825cc00640 1 --2- 172.21.15.94:0/1105028586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82580c49b0 0x7f82580cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:03.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.492+0000 7f825cc00640 1 --2- 172.21.15.94:0/1105028586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8258158790 0x7f825806fc00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:03.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.492+0000 7f825cc00640 1 -- 172.21.15.94:0/1105028586 >> 172.21.15.94:0/1105028586 conn(0x7f82580cfaf0 msgr2=0x7f82580cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:03.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.492+0000 7f825cc00640 1 -- 172.21.15.94:0/1105028586 shutdown_connections 2024-09-10T23:39:03.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.492+0000 7f825cc00640 1 -- 172.21.15.94:0/1105028586 wait complete. 2024-09-10T23:39:03.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.492+0000 7f825cc00640 1 Processor -- start 2024-09-10T23:39:03.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.492+0000 7f825cc00640 1 -- start start 2024-09-10T23:39:03.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f825cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82580c49b0 0x7f825815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f825cc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82580cb300 0x7f82581fbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f825cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258158790 0x7f825814bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f825cc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f82581681e0 con 0x7f8258158790 2024-09-10T23:39:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f825cc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8258168060 con 0x7f82580c49b0 2024-09-10T23:39:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f825cc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8258168360 con 0x7f82580cb300 2024-09-10T23:39:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f8256800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258158790 0x7f825814bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f8256800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258158790 0x7f825814bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55950/0 (socket says 172.21.15.94:55950) 2024-09-10T23:39:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f8256800640 1 -- 172.21.15.94:0/388216675 learned_addr learned my addr 172.21.15.94:0/388216675 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f8255e00640 1 --2- 172.21.15.94:0/388216675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82580c49b0 0x7f825815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f824d400640 1 --2- 172.21.15.94:0/388216675 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82580cb300 0x7f82581fbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f8256800640 1 -- 172.21.15.94:0/388216675 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82580cb300 msgr2=0x7f82581fbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f8256800640 1 --2- 172.21.15.94:0/388216675 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82580cb300 0x7f82581fbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f8256800640 1 -- 172.21.15.94:0/388216675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82580c49b0 msgr2=0x7f825815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f8256800640 1 --2- 172.21.15.94:0/388216675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82580c49b0 0x7f825815c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f8256800640 1 -- 172.21.15.94:0/388216675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825815ecb0 con 0x7f8258158790 2024-09-10T23:39:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f824d400640 1 --2- 172.21.15.94:0/388216675 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82580cb300 0x7f82581fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f8256800640 1 --2- 172.21.15.94:0/388216675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8258158790 0x7f825814bcf0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f825000cbb0 tx=0x7f825000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f824ea00640 1 -- 172.21.15.94:0/388216675 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250016020 con 0x7f8258158790 2024-09-10T23:39:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f824ea00640 1 -- 172.21.15.94:0/388216675 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f825000aae0 con 0x7f8258158790 2024-09-10T23:39:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f825cc00640 1 -- 172.21.15.94:0/388216675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8258073c80 con 0x7f8258158790 2024-09-10T23:39:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f824ea00640 1 -- 172.21.15.94:0/388216675 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250012420 con 0x7f8258158790 2024-09-10T23:39:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f825cc00640 1 -- 172.21.15.94:0/388216675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8258074240 con 0x7f8258158790 2024-09-10T23:39:03.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f825cc00640 1 -- 172.21.15.94:0/388216675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82580d01b0 con 0x7f8258158790 2024-09-10T23:39:03.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.496+0000 7f824ea00640 1 -- 172.21.15.94:0/388216675 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f82500125c0 con 0x7f8258158790 2024-09-10T23:39:03.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.500+0000 7f824ea00640 1 --2- 172.21.15.94:0/388216675 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8234072c40 0x7f8234075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:03.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.500+0000 7f824ea00640 1 -- 172.21.15.94:0/388216675 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f8250094350 con 0x7f8258158790 2024-09-10T23:39:03.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.500+0000 7f8255e00640 1 --2- 172.21.15.94:0/388216675 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8234072c40 0x7f8234075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:03.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.500+0000 7f8255e00640 1 --2- 172.21.15.94:0/388216675 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8234072c40 0x7f8234075100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f82480af000 tx=0x7f824801b060 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:03.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.504+0000 7f824ea00640 1 -- 172.21.15.94:0/388216675 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8250020020 con 0x7f8258158790 2024-09-10T23:39:03.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.692+0000 7f825cc00640 1 -- 172.21.15.94:0/388216675 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7f82580d0850 con 0x7f8234072c40 2024-09-10T23:39:03.697 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:03.696+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:03.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.696+0000 7f824ea00640 1 -- 172.21.15.94:0/388216675 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f82580d0850 con 0x7f8234072c40 2024-09-10T23:39:03.698 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:03.706 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:03.706 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create non_existent_subvolume subvolume_1 snapshot_1 2024-09-10T23:39:03.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.852+0000 7f4557200640 1 -- 172.21.15.94:0/2457893919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f453c004810 msgr2=0x7f453c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:03.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.852+0000 7f4557200640 1 --2- 172.21.15.94:0/2457893919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f453c004810 0x7f453c004c90 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f4540009b60 tx=0x7f454001d270 comp rx=0 tx=0).stop 2024-09-10T23:39:03.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.852+0000 7f4556800640 1 -- 172.21.15.94:0/2457893919 reap_dead start 2024-09-10T23:39:03.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.852+0000 7f4557200640 1 -- 172.21.15.94:0/2457893919 shutdown_connections 2024-09-10T23:39:03.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.852+0000 7f4557200640 1 -- 172.21.15.94:0/2457893919 >> 172.21.15.94:0/2457893919 conn(0x7f455809aa70 msgr2=0x7f455809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:03.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.852+0000 7f4557200640 1 -- 172.21.15.94:0/2457893919 shutdown_connections 2024-09-10T23:39:03.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.852+0000 7f4557200640 1 -- 172.21.15.94:0/2457893919 wait complete. 2024-09-10T23:39:03.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.852+0000 7f4557200640 1 Processor -- start 2024-09-10T23:39:03.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.852+0000 7f4557200640 1 -- start start 2024-09-10T23:39:03.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4557200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f453c004810 0x7f45580cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:03.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4557200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45580b6b70 0x7f45580b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:03.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4557200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45580b4ad0 0x7f45580b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:03.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4557200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f45580d0360 con 0x7f453c004810 2024-09-10T23:39:03.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4557200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f45580d01e0 con 0x7f45580b4ad0 2024-09-10T23:39:03.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4557200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f45580d04e0 con 0x7f45580b6b70 2024-09-10T23:39:03.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4555e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f453c004810 0x7f45580cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:03.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4555e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f453c004810 0x7f45580cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60930/0 (socket says 172.21.15.94:60930) 2024-09-10T23:39:03.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4555e00640 1 -- 172.21.15.94:0/688036454 learned_addr learned my addr 172.21.15.94:0/688036454 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:03.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f454fe00640 1 --2- 172.21.15.94:0/688036454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45580b6b70 0x7f45580b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:03.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4556800640 1 --2- 172.21.15.94:0/688036454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45580b4ad0 0x7f45580b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:03.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4555e00640 1 -- 172.21.15.94:0/688036454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45580b6b70 msgr2=0x7f45580b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:03.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4555e00640 1 --2- 172.21.15.94:0/688036454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45580b6b70 0x7f45580b6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:03.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4555e00640 1 -- 172.21.15.94:0/688036454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45580b4ad0 msgr2=0x7f45580b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:03.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4555e00640 1 --2- 172.21.15.94:0/688036454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45580b4ad0 0x7f45580b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:03.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4555e00640 1 -- 172.21.15.94:0/688036454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45580b76a0 con 0x7f453c004810 2024-09-10T23:39:03.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4555e00640 1 --2- 172.21.15.94:0/688036454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f453c004810 0x7f45580cb670 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f45480058d0 tx=0x7f4548005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:03.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f454e000640 1 -- 172.21.15.94:0/688036454 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548005320 con 0x7f453c004810 2024-09-10T23:39:03.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f454e000640 1 -- 172.21.15.94:0/688036454 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4548018020 con 0x7f453c004810 2024-09-10T23:39:03.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4557200640 1 -- 172.21.15.94:0/688036454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45580b7980 con 0x7f453c004810 2024-09-10T23:39:03.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f454e000640 1 -- 172.21.15.94:0/688036454 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548011b50 con 0x7f453c004810 2024-09-10T23:39:03.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4557200640 1 -- 172.21.15.94:0/688036454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455815cea0 con 0x7f453c004810 2024-09-10T23:39:03.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f4557200640 1 -- 172.21.15.94:0/688036454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45580af2c0 con 0x7f453c004810 2024-09-10T23:39:03.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.856+0000 7f454e000640 1 -- 172.21.15.94:0/688036454 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f4548010040 con 0x7f453c004810 2024-09-10T23:39:03.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.860+0000 7f454e000640 1 --2- 172.21.15.94:0/688036454 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4534072c10 0x7f45340750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:03.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.860+0000 7f454e000640 1 -- 172.21.15.94:0/688036454 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f4548097cd0 con 0x7f453c004810 2024-09-10T23:39:03.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.860+0000 7f454fe00640 1 --2- 172.21.15.94:0/688036454 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4534072c10 0x7f45340750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:03.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.860+0000 7f454fe00640 1 --2- 172.21.15.94:0/688036454 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4534072c10 0x7f45340750d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f45400ae000 tx=0x7f454001a010 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:03.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:03.864+0000 7f454e000640 1 -- 172.21.15.94:0/688036454 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4548022020 con 0x7f453c004810 2024-09-10T23:39:04.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.056+0000 7f4557200640 1 -- 172.21.15.94:0/688036454 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f455815ca00 con 0x7f4534072c10 2024-09-10T23:39:04.061 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:04.060+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:04.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.060+0000 7f454e000640 1 -- 172.21.15.94:0/688036454 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f4540004040 con 0x7f4534072c10 2024-09-10T23:39:04.062 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:04.070 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:04.070 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm non_existent_subvolume subvolume_1 snapshot_1 2024-09-10T23:39:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.212+0000 7f47ef600640 1 -- 172.21.15.94:0/358953920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47e80af310 msgr2=0x7f47e80af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.212+0000 7f47ef600640 1 --2- 172.21.15.94:0/358953920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47e80af310 0x7f47e80af7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009840 tx=0x7f47e001c700 comp rx=0 tx=0).stop 2024-09-10T23:39:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ef600640 1 -- 172.21.15.94:0/358953920 shutdown_connections 2024-09-10T23:39:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ef600640 1 --2- 172.21.15.94:0/358953920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47e80af310 0x7f47e80af7a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:04.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ef600640 1 --2- 172.21.15.94:0/358953920 >> v2:172.21.15.181:3300/0 conn(0x7f47e815f5c0 0x7f47e80cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:04.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ef600640 1 -- 172.21.15.94:0/358953920 >> 172.21.15.94:0/358953920 conn(0x7f47e80d1c60 msgr2=0x7f47e80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:04.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ef600640 1 -- 172.21.15.94:0/358953920 shutdown_connections 2024-09-10T23:39:04.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ef600640 1 -- 172.21.15.94:0/358953920 wait complete. 2024-09-10T23:39:04.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ef600640 1 Processor -- start 2024-09-10T23:39:04.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ef600640 1 -- start start 2024-09-10T23:39:04.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ef600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47e80af310 0x7f47e8146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:04.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ef600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47e80b2bb0 0x7f47e8152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:04.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ef600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e815f5c0 0x7f47e8153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ef600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8169000 con 0x7f47e815f5c0 2024-09-10T23:39:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ef600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8168e80 con 0x7f47e80b2bb0 2024-09-10T23:39:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ef600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f47e8169180 con 0x7f47e80af310 2024-09-10T23:39:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ed600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e815f5c0 0x7f47e8153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ed600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e815f5c0 0x7f47e8153400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60940/0 (socket says 172.21.15.94:60940) 2024-09-10T23:39:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ed600640 1 -- 172.21.15.94:0/1710031945 learned_addr learned my addr 172.21.15.94:0/1710031945 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ecc00640 1 --2- 172.21.15.94:0/1710031945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47e80af310 0x7f47e8146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47e7e00640 1 --2- 172.21.15.94:0/1710031945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47e80b2bb0 0x7f47e8152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ed600640 1 -- 172.21.15.94:0/1710031945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47e80af310 msgr2=0x7f47e8146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ed600640 1 --2- 172.21.15.94:0/1710031945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47e80af310 0x7f47e8146c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ed600640 1 -- 172.21.15.94:0/1710031945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47e80b2bb0 msgr2=0x7f47e8152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ed600640 1 --2- 172.21.15.94:0/1710031945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47e80b2bb0 0x7f47e8152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ed600640 1 -- 172.21.15.94:0/1710031945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e815c440 con 0x7f47e815f5c0 2024-09-10T23:39:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47e7e00640 1 --2- 172.21.15.94:0/1710031945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47e80b2bb0 0x7f47e8152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.216+0000 7f47ed600640 1 --2- 172.21.15.94:0/1710031945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47e815f5c0 0x7f47e8153400 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f47e001a330 tx=0x7f47e00a6640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.220+0000 7f47e4c00640 1 -- 172.21.15.94:0/1710031945 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e00bc020 con 0x7f47e815f5c0 2024-09-10T23:39:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.220+0000 7f47e4c00640 1 -- 172.21.15.94:0/1710031945 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47e0004330 con 0x7f47e815f5c0 2024-09-10T23:39:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.220+0000 7f47ef600640 1 -- 172.21.15.94:0/1710031945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e8147360 con 0x7f47e815f5c0 2024-09-10T23:39:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.220+0000 7f47e4c00640 1 -- 172.21.15.94:0/1710031945 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0005290 con 0x7f47e815f5c0 2024-09-10T23:39:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.220+0000 7f47ef600640 1 -- 172.21.15.94:0/1710031945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e806e220 con 0x7f47e815f5c0 2024-09-10T23:39:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.220+0000 7f47ef600640 1 -- 172.21.15.94:0/1710031945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e806dd30 con 0x7f47e815f5c0 2024-09-10T23:39:04.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.220+0000 7f47e4c00640 1 -- 172.21.15.94:0/1710031945 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f47e0005430 con 0x7f47e815f5c0 2024-09-10T23:39:04.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.220+0000 7f47e4c00640 1 --2- 172.21.15.94:0/1710031945 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f47c8072c40 0x7f47c8075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:04.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.220+0000 7f47e4c00640 1 -- 172.21.15.94:0/1710031945 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f47e012ef30 con 0x7f47e815f5c0 2024-09-10T23:39:04.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.220+0000 7f47ecc00640 1 --2- 172.21.15.94:0/1710031945 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f47c8072c40 0x7f47c8075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:04.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.224+0000 7f47ecc00640 1 --2- 172.21.15.94:0/1710031945 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f47c8072c40 0x7f47c8075100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009ec0 tx=0x7f47d8009500 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:04.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.224+0000 7f47e4c00640 1 -- 172.21.15.94:0/1710031945 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f47e0132030 con 0x7f47e815f5c0 2024-09-10T23:39:04.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.420+0000 7f47ef600640 1 -- 172.21.15.94:0/1710031945 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f47e80af7a0 con 0x7f47c8072c40 2024-09-10T23:39:04.423 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:04.420+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:04.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.420+0000 7f47e4c00640 1 -- 172.21.15.94:0/1710031945 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f47e80af7a0 con 0x7f47c8072c40 2024-09-10T23:39:04.424 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:04.431 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:04.432 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info non_existent_subvolume subvolume_1 snapshot_1 2024-09-10T23:39:04.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.576+0000 7fa695600640 1 -- 172.21.15.94:0/3567752205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6900ae980 msgr2=0x7fa6900d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:04.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.576+0000 7fa695600640 1 --2- 172.21.15.94:0/3567752205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6900ae980 0x7fa6900d1b80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa69015c9a0 tx=0x7fa67801c760 comp rx=0 tx=0).stop 2024-09-10T23:39:04.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.576+0000 7fa695600640 1 -- 172.21.15.94:0/3567752205 shutdown_connections 2024-09-10T23:39:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.576+0000 7fa695600640 1 --2- 172.21.15.94:0/3567752205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6900ae980 0x7fa6900d1b80 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.576+0000 7fa695600640 1 -- 172.21.15.94:0/3567752205 >> 172.21.15.94:0/3567752205 conn(0x7fa69009aa70 msgr2=0x7fa69009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.576+0000 7fa695600640 1 -- 172.21.15.94:0/3567752205 shutdown_connections 2024-09-10T23:39:04.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.576+0000 7fa695600640 1 -- 172.21.15.94:0/3567752205 wait complete. 2024-09-10T23:39:04.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.576+0000 7fa695600640 1 Processor -- start 2024-09-10T23:39:04.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa695600640 1 -- start start 2024-09-10T23:39:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa695600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6900ae980 0x7fa69009f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa695600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6900b2ec0 0x7fa69009ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa695600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa69015bd30 0x7fa6900c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa695600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa69016a3f0 con 0x7fa6900ae980 2024-09-10T23:39:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa695600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa69016a270 con 0x7fa6900b2ec0 2024-09-10T23:39:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa695600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa69016a570 con 0x7fa69015bd30 2024-09-10T23:39:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa68ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6900ae980 0x7fa69009f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa68ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6900ae980 0x7fa69009f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60946/0 (socket says 172.21.15.94:60946) 2024-09-10T23:39:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa68ea00640 1 -- 172.21.15.94:0/1750224976 learned_addr learned my addr 172.21.15.94:0/1750224976 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa686000640 1 --2- 172.21.15.94:0/1750224976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6900b2ec0 0x7fa69009ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa68f400640 1 --2- 172.21.15.94:0/1750224976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa69015bd30 0x7fa6900c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa68ea00640 1 -- 172.21.15.94:0/1750224976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa69015bd30 msgr2=0x7fa6900c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:04.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa68ea00640 1 --2- 172.21.15.94:0/1750224976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa69015bd30 0x7fa6900c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:04.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa68ea00640 1 -- 172.21.15.94:0/1750224976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6900b2ec0 msgr2=0x7fa69009ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:04.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa68ea00640 1 --2- 172.21.15.94:0/1750224976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6900b2ec0 0x7fa69009ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:04.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa68ea00640 1 -- 172.21.15.94:0/1750224976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6900b57d0 con 0x7fa6900ae980 2024-09-10T23:39:04.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa68f400640 1 --2- 172.21.15.94:0/1750224976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa69015bd30 0x7fa6900c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:04.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa686000640 1 --2- 172.21.15.94:0/1750224976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6900b2ec0 0x7fa69009ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:04.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa68ea00640 1 --2- 172.21.15.94:0/1750224976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6900ae980 0x7fa69009f500 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fa68000c3c0 tx=0x7fa68000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:04.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa687400640 1 -- 172.21.15.94:0/1750224976 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680011790 con 0x7fa6900ae980 2024-09-10T23:39:04.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa695600640 1 -- 172.21.15.94:0/1750224976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6900b5a90 con 0x7fa6900ae980 2024-09-10T23:39:04.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa687400640 1 -- 172.21.15.94:0/1750224976 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa680017070 con 0x7fa6900ae980 2024-09-10T23:39:04.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa687400640 1 -- 172.21.15.94:0/1750224976 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680011d20 con 0x7fa6900ae980 2024-09-10T23:39:04.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa695600640 1 -- 172.21.15.94:0/1750224976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6900b6050 con 0x7fa6900ae980 2024-09-10T23:39:04.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.580+0000 7fa695600640 1 -- 172.21.15.94:0/1750224976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6900d1b80 con 0x7fa6900ae980 2024-09-10T23:39:04.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.584+0000 7fa687400640 1 -- 172.21.15.94:0/1750224976 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7fa68001c460 con 0x7fa6900ae980 2024-09-10T23:39:04.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.584+0000 7fa687400640 1 --2- 172.21.15.94:0/1750224976 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa66c072c40 0x7fa66c075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:04.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.584+0000 7fa687400640 1 -- 172.21.15.94:0/1750224976 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7fa680093d00 con 0x7fa6900ae980 2024-09-10T23:39:04.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.584+0000 7fa686000640 1 --2- 172.21.15.94:0/1750224976 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa66c072c40 0x7fa66c075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:04.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.584+0000 7fa686000640 1 --2- 172.21.15.94:0/1750224976 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa66c072c40 0x7fa66c075100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa6780af000 tx=0x7fa67801afa0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:04.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.588+0000 7fa687400640 1 -- 172.21.15.94:0/1750224976 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa68005e940 con 0x7fa6900ae980 2024-09-10T23:39:04.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.780+0000 7fa695600640 1 -- 172.21.15.94:0/1750224976 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fa6900d28a0 con 0x7fa66c072c40 2024-09-10T23:39:04.784 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:04.780+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:04.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.780+0000 7fa687400640 1 -- 172.21.15.94:0/1750224976 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fa6900d28a0 con 0x7fa66c072c40 2024-09-10T23:39:04.784 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:04.792 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:04.793 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot protect non_existent_subvolume subvolume_1 snapshot_1 2024-09-10T23:39:04.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.936+0000 7f0e6ae00640 1 -- 172.21.15.94:0/926472464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e640b4c40 msgr2=0x7f0e640cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:04.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.936+0000 7f0e6ae00640 1 --2- 172.21.15.94:0/926472464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e640b4c40 0x7f0e640cfcb0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009900 tx=0x7f0e5001c850 comp rx=0 tx=0).stop 2024-09-10T23:39:04.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.936+0000 7f0e6ae00640 1 -- 172.21.15.94:0/926472464 shutdown_connections 2024-09-10T23:39:04.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.936+0000 7f0e6ae00640 1 --2- 172.21.15.94:0/926472464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e640b4c40 0x7f0e640cfcb0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:04.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.936+0000 7f0e6ae00640 1 --2- 172.21.15.94:0/926472464 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e6415ba50 0x7f0e640b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:04.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.936+0000 7f0e6ae00640 1 -- 172.21.15.94:0/926472464 >> 172.21.15.94:0/926472464 conn(0x7f0e6406cdf0 msgr2=0x7f0e6406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:04.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.936+0000 7f0e6ae00640 1 -- 172.21.15.94:0/926472464 shutdown_connections 2024-09-10T23:39:04.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.936+0000 7f0e6ae00640 1 -- 172.21.15.94:0/926472464 wait complete. 2024-09-10T23:39:04.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.936+0000 7f0e6ae00640 1 Processor -- start 2024-09-10T23:39:04.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e6ae00640 1 -- start start 2024-09-10T23:39:04.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e6ae00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e640b4c40 0x7f0e6415cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:04.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e6ae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e640d01f0 0x7f0e6413f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:04.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e6ae00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e6415ba50 0x7f0e64157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:04.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e6ae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0e64168be0 con 0x7f0e640d01f0 2024-09-10T23:39:04.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e6ae00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0e64168a60 con 0x7f0e640b4c40 2024-09-10T23:39:04.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e6ae00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0e64168d60 con 0x7f0e6415ba50 2024-09-10T23:39:04.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e63400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e640d01f0 0x7f0e6413f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:04.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e63400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e640d01f0 0x7f0e6413f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60954/0 (socket says 172.21.15.94:60954) 2024-09-10T23:39:04.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e63400640 1 -- 172.21.15.94:0/4005268370 learned_addr learned my addr 172.21.15.94:0/4005268370 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:04.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e63e00640 1 --2- 172.21.15.94:0/4005268370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e640b4c40 0x7f0e6415cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:04.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e68e00640 1 --2- 172.21.15.94:0/4005268370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e6415ba50 0x7f0e64157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:04.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e63400640 1 -- 172.21.15.94:0/4005268370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e6415ba50 msgr2=0x7f0e64157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:04.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e63400640 1 --2- 172.21.15.94:0/4005268370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e6415ba50 0x7f0e64157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:04.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e63400640 1 -- 172.21.15.94:0/4005268370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e640b4c40 msgr2=0x7f0e6415cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:04.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e63400640 1 --2- 172.21.15.94:0/4005268370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e640b4c40 0x7f0e6415cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:04.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e63400640 1 -- 172.21.15.94:0/4005268370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6414b570 con 0x7f0e640d01f0 2024-09-10T23:39:04.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e68e00640 1 --2- 172.21.15.94:0/4005268370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e6415ba50 0x7f0e64157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:04.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e63400640 1 --2- 172.21.15.94:0/4005268370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e640d01f0 0x7f0e6413f240 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009850 tx=0x7f0e50005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:04.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e5be00640 1 -- 172.21.15.94:0/4005268370 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e500b2040 con 0x7f0e640d01f0 2024-09-10T23:39:04.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e6ae00640 1 -- 172.21.15.94:0/4005268370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6413f780 con 0x7f0e640d01f0 2024-09-10T23:39:04.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e5be00640 1 -- 172.21.15.94:0/4005268370 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e50004300 con 0x7f0e640d01f0 2024-09-10T23:39:04.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e5be00640 1 -- 172.21.15.94:0/4005268370 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e500052d0 con 0x7f0e640d01f0 2024-09-10T23:39:04.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e6ae00640 1 -- 172.21.15.94:0/4005268370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e6415d530 con 0x7f0e640d01f0 2024-09-10T23:39:04.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.940+0000 7f0e6ae00640 1 -- 172.21.15.94:0/4005268370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e640cfcb0 con 0x7f0e640d01f0 2024-09-10T23:39:04.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.944+0000 7f0e5be00640 1 -- 172.21.15.94:0/4005268370 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f0e50005470 con 0x7f0e640d01f0 2024-09-10T23:39:04.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.944+0000 7f0e5be00640 1 --2- 172.21.15.94:0/4005268370 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0e44072c10 0x7f0e440750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:04.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.944+0000 7f0e5be00640 1 -- 172.21.15.94:0/4005268370 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f0e5012d830 con 0x7f0e640d01f0 2024-09-10T23:39:04.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.944+0000 7f0e63e00640 1 --2- 172.21.15.94:0/4005268370 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0e44072c10 0x7f0e440750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:04.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.944+0000 7f0e63e00640 1 --2- 172.21.15.94:0/4005268370 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0e44072c10 0x7f0e440750d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0e6414ef80 tx=0x7f0e540091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:04.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:04.948+0000 7f0e5be00640 1 -- 172.21.15.94:0/4005268370 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0e500164d0 con 0x7f0e640d01f0 2024-09-10T23:39:05.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.140+0000 7f0e6ae00640 1 -- 172.21.15.94:0/4005268370 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot protect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f0e64156460 con 0x7f0e44072c10 2024-09-10T23:39:05.146 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:05.144+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:05.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.144+0000 7f0e5be00640 1 -- 172.21.15.94:0/4005268370 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f0e64156460 con 0x7f0e44072c10 2024-09-10T23:39:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:05.155 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:05.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot unprotect non_existent_subvolume subvolume_1 snapshot_1 2024-09-10T23:39:05.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.300+0000 7f4ad1400640 1 -- 172.21.15.94:0/591951745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc070150 msgr2=0x7f4acc0b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.300+0000 7f4ad1400640 1 --2- 172.21.15.94:0/591951745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc070150 0x7f4acc0b35c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009bc0 tx=0x7f4ac401ca80 comp rx=0 tx=0).stop 2024-09-10T23:39:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.300+0000 7f4ad1400640 1 -- 172.21.15.94:0/591951745 shutdown_connections 2024-09-10T23:39:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.300+0000 7f4ad1400640 1 --2- 172.21.15.94:0/591951745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc070150 0x7f4acc0b35c0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.300+0000 7f4ad1400640 1 --2- 172.21.15.94:0/591951745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4acc06f600 0x7f4acc072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.300+0000 7f4ad1400640 1 -- 172.21.15.94:0/591951745 >> 172.21.15.94:0/591951745 conn(0x7f4acc06cdf0 msgr2=0x7f4acc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.300+0000 7f4ad1400640 1 -- 172.21.15.94:0/591951745 shutdown_connections 2024-09-10T23:39:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.300+0000 7f4ad1400640 1 -- 172.21.15.94:0/591951745 wait complete. 2024-09-10T23:39:05.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.300+0000 7f4ad1400640 1 Processor -- start 2024-09-10T23:39:05.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.300+0000 7f4ad1400640 1 -- start start 2024-09-10T23:39:05.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4ad1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc06f600 0x7f4acc0ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:05.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4ad1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4acc070150 0x7f4acc15d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:05.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4ad1400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4acc157c60 0x7f4acc0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4ad1400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc0d00c0 con 0x7f4acc06f600 2024-09-10T23:39:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4ad1400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc0cff40 con 0x7f4acc070150 2024-09-10T23:39:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4ad1400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f4acc0d0240 con 0x7f4acc157c60 2024-09-10T23:39:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4aca800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc06f600 0x7f4acc0ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4aca800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc06f600 0x7f4acc0ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60970/0 (socket says 172.21.15.94:60970) 2024-09-10T23:39:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4aca800640 1 -- 172.21.15.94:0/1716759386 learned_addr learned my addr 172.21.15.94:0/1716759386 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4ac9e00640 1 --2- 172.21.15.94:0/1716759386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4acc070150 0x7f4acc15d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4acb200640 1 --2- 172.21.15.94:0/1716759386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4acc157c60 0x7f4acc0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4aca800640 1 -- 172.21.15.94:0/1716759386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4acc157c60 msgr2=0x7f4acc0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4aca800640 1 --2- 172.21.15.94:0/1716759386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4acc157c60 0x7f4acc0ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4aca800640 1 -- 172.21.15.94:0/1716759386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4acc070150 msgr2=0x7f4acc15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4aca800640 1 --2- 172.21.15.94:0/1716759386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4acc070150 0x7f4acc15d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4aca800640 1 -- 172.21.15.94:0/1716759386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4acc159c10 con 0x7f4acc06f600 2024-09-10T23:39:05.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4acb200640 1 --2- 172.21.15.94:0/1716759386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4acc157c60 0x7f4acc0ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:05.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4aca800640 1 --2- 172.21.15.94:0/1716759386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc06f600 0x7f4acc0ce070 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00c400 tx=0x7f4abc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:05.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4ac2a00640 1 -- 172.21.15.94:0/1716759386 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc017070 con 0x7f4acc06f600 2024-09-10T23:39:05.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4ac2a00640 1 -- 172.21.15.94:0/1716759386 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4abc003600 con 0x7f4acc06f600 2024-09-10T23:39:05.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4ad1400640 1 -- 172.21.15.94:0/1716759386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4acc159ed0 con 0x7f4acc06f600 2024-09-10T23:39:05.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4ac2a00640 1 -- 172.21.15.94:0/1716759386 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc003a50 con 0x7f4acc06f600 2024-09-10T23:39:05.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4ad1400640 1 -- 172.21.15.94:0/1716759386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4acc0ce5b0 con 0x7f4acc06f600 2024-09-10T23:39:05.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4ad1400640 1 -- 172.21.15.94:0/1716759386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4acc072610 con 0x7f4acc06f600 2024-09-10T23:39:05.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.304+0000 7f4ac2a00640 1 -- 172.21.15.94:0/1716759386 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f4abc003bf0 con 0x7f4acc06f600 2024-09-10T23:39:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.308+0000 7f4ac2a00640 1 --2- 172.21.15.94:0/1716759386 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4aac072c10 0x7f4aac0750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.308+0000 7f4ac2a00640 1 -- 172.21.15.94:0/1716759386 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f4abc093470 con 0x7f4acc06f600 2024-09-10T23:39:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.308+0000 7f4ac9e00640 1 --2- 172.21.15.94:0/1716759386 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4aac072c10 0x7f4aac0750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.308+0000 7f4ac9e00640 1 --2- 172.21.15.94:0/1716759386 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4aac072c10 0x7f4aac0750d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4acc1481d0 tx=0x7f4ab800a2d0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:05.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.312+0000 7f4ac2a00640 1 -- 172.21.15.94:0/1716759386 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4abc05da60 con 0x7f4acc06f600 2024-09-10T23:39:05.358 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:39:05.361 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:39:05.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.516+0000 7f4ad1400640 1 -- 172.21.15.94:0/1716759386 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot unprotect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f4acc151d90 con 0x7f4aac072c10 2024-09-10T23:39:05.519 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:05.516+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:05.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.516+0000 7f4ac2a00640 1 -- 172.21.15.94:0/1716759386 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f4acc151d90 con 0x7f4aac072c10 2024-09-10T23:39:05.520 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:05.529 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:05.530 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls non_existent_subvolume subvolume_1 2024-09-10T23:39:05.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.672+0000 7fd50c400640 1 -- 172.21.15.94:0/1203915533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5040d04b0 msgr2=0x7fd50415e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:05.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.672+0000 7fd50c400640 1 --2- 172.21.15.94:0/1203915533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5040d04b0 0x7fd50415e770 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fd5040b2df0 tx=0x7fd4f001c890 comp rx=0 tx=0).stop 2024-09-10T23:39:05.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.672+0000 7fd50c400640 1 -- 172.21.15.94:0/1203915533 shutdown_connections 2024-09-10T23:39:05.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.672+0000 7fd50c400640 1 --2- 172.21.15.94:0/1203915533 >> v2:172.21.15.181:3300/0 conn(0x7fd50415ecb0 0x7fd50406f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:05.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.672+0000 7fd50c400640 1 --2- 172.21.15.94:0/1203915533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5040d04b0 0x7fd50415e770 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:05.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.672+0000 7fd50c400640 1 -- 172.21.15.94:0/1203915533 >> 172.21.15.94:0/1203915533 conn(0x7fd50406cdf0 msgr2=0x7fd50406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:05.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.672+0000 7fd50c400640 1 -- 172.21.15.94:0/1203915533 shutdown_connections 2024-09-10T23:39:05.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.672+0000 7fd50c400640 1 -- 172.21.15.94:0/1203915533 wait complete. 2024-09-10T23:39:05.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50c400640 1 Processor -- start 2024-09-10T23:39:05.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50c400640 1 -- start start 2024-09-10T23:39:05.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50c400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd504073ff0 0x7fd50414b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50c400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5040d04b0 0x7fd504148280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50c400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd50415ecb0 0x7fd5040b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50c400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd504167930 con 0x7fd50415ecb0 2024-09-10T23:39:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50c400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd5041677b0 con 0x7fd5040d04b0 2024-09-10T23:39:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50c400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd504167ab0 con 0x7fd504073ff0 2024-09-10T23:39:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50a400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd50415ecb0 0x7fd5040b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50a400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd50415ecb0 0x7fd5040b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60986/0 (socket says 172.21.15.94:60986) 2024-09-10T23:39:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50a400640 1 -- 172.21.15.94:0/3119144497 learned_addr learned my addr 172.21.15.94:0/3119144497 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd509000640 1 --2- 172.21.15.94:0/3119144497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5040d04b0 0x7fd504148280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd509a00640 1 --2- 172.21.15.94:0/3119144497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd504073ff0 0x7fd50414b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50a400640 1 -- 172.21.15.94:0/3119144497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd504073ff0 msgr2=0x7fd50414b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50a400640 1 --2- 172.21.15.94:0/3119144497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd504073ff0 0x7fd50414b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:05.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50a400640 1 -- 172.21.15.94:0/3119144497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5040d04b0 msgr2=0x7fd504148280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50a400640 1 --2- 172.21.15.94:0/3119144497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5040d04b0 0x7fd504148280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50a400640 1 -- 172.21.15.94:0/3119144497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd504148cc0 con 0x7fd50415ecb0 2024-09-10T23:39:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50a400640 1 --2- 172.21.15.94:0/3119144497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd50415ecb0 0x7fd5040b1950 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00b970 tx=0x7fd4fc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd501600640 1 -- 172.21.15.94:0/3119144497 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc019070 con 0x7fd50415ecb0 2024-09-10T23:39:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd501600640 1 -- 172.21.15.94:0/3119144497 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4fc00cc50 con 0x7fd50415ecb0 2024-09-10T23:39:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd501600640 1 -- 172.21.15.94:0/3119144497 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc0146a0 con 0x7fd50415ecb0 2024-09-10T23:39:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50c400640 1 -- 172.21.15.94:0/3119144497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5040b1490 con 0x7fd50415ecb0 2024-09-10T23:39:05.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50c400640 1 -- 172.21.15.94:0/3119144497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd50414b080 con 0x7fd50415ecb0 2024-09-10T23:39:05.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.676+0000 7fd50c400640 1 -- 172.21.15.94:0/3119144497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd50415e770 con 0x7fd50415ecb0 2024-09-10T23:39:05.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.680+0000 7fd501600640 1 -- 172.21.15.94:0/3119144497 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7fd4fc01d3b0 con 0x7fd50415ecb0 2024-09-10T23:39:05.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.680+0000 7fd501600640 1 --2- 172.21.15.94:0/3119144497 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd4e4072c40 0x7fd4e4075100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:05.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.680+0000 7fd501600640 1 -- 172.21.15.94:0/3119144497 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7fd4fc093fe0 con 0x7fd50415ecb0 2024-09-10T23:39:05.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.680+0000 7fd509a00640 1 --2- 172.21.15.94:0/3119144497 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd4e4072c40 0x7fd4e4075100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:05.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.680+0000 7fd509a00640 1 --2- 172.21.15.94:0/3119144497 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd4e4072c40 0x7fd4e4075100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8000f70 tx=0x7fd4f8009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:05.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.684+0000 7fd501600640 1 -- 172.21.15.94:0/3119144497 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd4fc05ec10 con 0x7fd50415ecb0 2024-09-10T23:39:05.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.880+0000 7fd50c400640 1 -- 172.21.15.94:0/3119144497 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7fd5040b1770 con 0x7fd4e4072c40 2024-09-10T23:39:05.882 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:05.880+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:05.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:05.880+0000 7fd501600640 1 -- 172.21.15.94:0/3119144497 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fd4f8001cd0 con 0x7fd4e4072c40 2024-09-10T23:39:05.883 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:05.891 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:05.891 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone non_existent_subvolume subvolume_1 snapshot_1 clone_1 2024-09-10T23:39:06.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62a00640 1 -- 172.21.15.94:0/1218143088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5c0d0610 msgr2=0x7f8f5c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:06.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62a00640 1 --2- 172.21.15.94:0/1218143088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5c0d0610 0x7f8f5c070600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009900 tx=0x7f8f4801c8e0 comp rx=0 tx=0).stop 2024-09-10T23:39:06.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62a00640 1 -- 172.21.15.94:0/1218143088 shutdown_connections 2024-09-10T23:39:06.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62a00640 1 --2- 172.21.15.94:0/1218143088 >> v2:172.21.15.181:3300/0 conn(0x7f8f5c0d2670 0x7f8f5c070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:06.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62a00640 1 --2- 172.21.15.94:0/1218143088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5c0d0610 0x7f8f5c070600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:06.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62a00640 1 -- 172.21.15.94:0/1218143088 >> 172.21.15.94:0/1218143088 conn(0x7f8f5c0d1b50 msgr2=0x7f8f5c0d1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:06.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62a00640 1 -- 172.21.15.94:0/1218143088 shutdown_connections 2024-09-10T23:39:06.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62a00640 1 -- 172.21.15.94:0/1218143088 wait complete. 2024-09-10T23:39:06.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62a00640 1 Processor -- start 2024-09-10T23:39:06.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62a00640 1 -- start start 2024-09-10T23:39:06.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f5c0d0610 0x7f8f5c0b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:06.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f5c0d2670 0x7f8f5c0c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:06.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5c15bcc0 0x7f8f5c1fb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:06.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c168ca0 con 0x7f8f5c0d2670 2024-09-10T23:39:06.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c168b20 con 0x7f8f5c0d0610 2024-09-10T23:39:06.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8f5c168e20 con 0x7f8f5c15bcc0 2024-09-10T23:39:06.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f60c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f5c0d2670 0x7f8f5c0c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:06.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f60c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f5c0d2670 0x7f8f5c0c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32770/0 (socket says 172.21.15.94:32770) 2024-09-10T23:39:06.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f60c00640 1 -- 172.21.15.94:0/3720058856 learned_addr learned my addr 172.21.15.94:0/3720058856 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:06.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f61600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f5c0d0610 0x7f8f5c0b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:06.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.040+0000 7f8f62000640 1 --2- 172.21.15.94:0/3720058856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5c15bcc0 0x7f8f5c1fb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:06.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.044+0000 7f8f60c00640 1 -- 172.21.15.94:0/3720058856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5c15bcc0 msgr2=0x7f8f5c1fb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:06.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.044+0000 7f8f60c00640 1 --2- 172.21.15.94:0/3720058856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5c15bcc0 0x7f8f5c1fb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:06.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.044+0000 7f8f60c00640 1 -- 172.21.15.94:0/3720058856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f5c0d0610 msgr2=0x7f8f5c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:06.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.044+0000 7f8f60c00640 1 --2- 172.21.15.94:0/3720058856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f5c0d0610 0x7f8f5c0b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:06.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.044+0000 7f8f60c00640 1 -- 172.21.15.94:0/3720058856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5c0cbd20 con 0x7f8f5c0d2670 2024-09-10T23:39:06.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.044+0000 7f8f60c00640 1 --2- 172.21.15.94:0/3720058856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f5c0d2670 0x7f8f5c0c8c20 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009850 tx=0x7f8f480a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:06.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.044+0000 7f8f59600640 1 -- 172.21.15.94:0/3720058856 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f480b2040 con 0x7f8f5c0d2670 2024-09-10T23:39:06.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.044+0000 7f8f59600640 1 -- 172.21.15.94:0/3720058856 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f480043b0 con 0x7f8f5c0d2670 2024-09-10T23:39:06.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.044+0000 7f8f62a00640 1 -- 172.21.15.94:0/3720058856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5c0cb760 con 0x7f8f5c0d2670 2024-09-10T23:39:06.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.044+0000 7f8f59600640 1 -- 172.21.15.94:0/3720058856 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48005290 con 0x7f8f5c0d2670 2024-09-10T23:39:06.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.044+0000 7f8f62a00640 1 -- 172.21.15.94:0/3720058856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f5c06e120 con 0x7f8f5c0d2670 2024-09-10T23:39:06.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.044+0000 7f8f62a00640 1 -- 172.21.15.94:0/3720058856 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f5c0cff00 con 0x7f8f5c0d2670 2024-09-10T23:39:06.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.044+0000 7f8f59600640 1 -- 172.21.15.94:0/3720058856 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f8f48005430 con 0x7f8f5c0d2670 2024-09-10T23:39:06.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.044+0000 7f8f59600640 1 --2- 172.21.15.94:0/3720058856 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8f3c072c10 0x7f8f3c0750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:06.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.044+0000 7f8f59600640 1 -- 172.21.15.94:0/3720058856 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f8f4812d410 con 0x7f8f5c0d2670 2024-09-10T23:39:06.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.048+0000 7f8f61600640 1 --2- 172.21.15.94:0/3720058856 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8f3c072c10 0x7f8f3c0750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:06.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.048+0000 7f8f61600640 1 --2- 172.21.15.94:0/3720058856 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8f3c072c10 0x7f8f3c0750d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8f50007fd0 tx=0x7f8f50009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:06.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.052+0000 7f8f59600640 1 -- 172.21.15.94:0/3720058856 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8f480164d0 con 0x7f8f5c0d2670 2024-09-10T23:39:06.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.244+0000 7f8f62a00640 1 -- 172.21.15.94:0/3720058856 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target_sub_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7f8f5c0b6e30 con 0x7f8f3c072c10 2024-09-10T23:39:06.255 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:06.252+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:06.255 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:06.252+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:06.255 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:06.252+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:06.255 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:06.252+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:06.255 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:06.252+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:06.261 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:06.260+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:06.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.260+0000 7f8f59600640 1 -- 172.21.15.94:0/3720058856 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f8f5c0b6e30 con 0x7f8f3c072c10 2024-09-10T23:39:06.262 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:06.271 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:06.272 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status non_existent_subvolume clone_1 2024-09-10T23:39:06.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.412+0000 7f04acc00640 1 -- 172.21.15.94:0/2211397331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f048c004810 msgr2=0x7f048c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:06.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.412+0000 7f04acc00640 1 --2- 172.21.15.94:0/2211397331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f048c004810 0x7f048c004c90 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f04900016c0 tx=0x7f049001c730 comp rx=0 tx=0).stop 2024-09-10T23:39:06.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.412+0000 7f04acc00640 1 -- 172.21.15.94:0/2211397331 shutdown_connections 2024-09-10T23:39:06.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.412+0000 7f04acc00640 1 --2- 172.21.15.94:0/2211397331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f048c004810 0x7f048c004c90 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:06.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.412+0000 7f04acc00640 1 -- 172.21.15.94:0/2211397331 >> 172.21.15.94:0/2211397331 conn(0x7f04a80d1c60 msgr2=0x7f04a80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:06.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.412+0000 7f04acc00640 1 -- 172.21.15.94:0/2211397331 shutdown_connections 2024-09-10T23:39:06.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.412+0000 7f04acc00640 1 -- 172.21.15.94:0/2211397331 wait complete. 2024-09-10T23:39:06.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04acc00640 1 Processor -- start 2024-09-10T23:39:06.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04acc00640 1 -- start start 2024-09-10T23:39:06.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04acc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f048c004810 0x7f04a815ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04acc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a81f8180 0x7f04a80ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04acc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a81f4a90 0x7f04a81facf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04acc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04a80ce3f0 con 0x7f048c004810 2024-09-10T23:39:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04acc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04a80ce570 con 0x7f04a81f4a90 2024-09-10T23:39:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04acc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f04a80ce6f0 con 0x7f04a81f8180 2024-09-10T23:39:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04a5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f048c004810 0x7f04a815ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04a5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f048c004810 0x7f04a815ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32812/0 (socket says 172.21.15.94:32812) 2024-09-10T23:39:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04a5e00640 1 -- 172.21.15.94:0/3566779083 learned_addr learned my addr 172.21.15.94:0/3566779083 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04a6800640 1 --2- 172.21.15.94:0/3566779083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a81f4a90 0x7f04a81facf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04a5e00640 1 -- 172.21.15.94:0/3566779083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a81f8180 msgr2=0x7f04a80ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f049d400640 1 --2- 172.21.15.94:0/3566779083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a81f8180 0x7f04a80ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04a5e00640 1 --2- 172.21.15.94:0/3566779083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a81f8180 0x7f04a80ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04a5e00640 1 -- 172.21.15.94:0/3566779083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a81f4a90 msgr2=0x7f04a81facf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04a5e00640 1 --2- 172.21.15.94:0/3566779083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a81f4a90 0x7f04a81facf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04a5e00640 1 -- 172.21.15.94:0/3566779083 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a80ca8c0 con 0x7f048c004810 2024-09-10T23:39:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04a6800640 1 --2- 172.21.15.94:0/3566779083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a81f4a90 0x7f04a81facf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f049d400640 1 --2- 172.21.15.94:0/3566779083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04a81f8180 0x7f04a80ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04a5e00640 1 --2- 172.21.15.94:0/3566779083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f048c004810 0x7f04a815ee20 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f0498000fc0 tx=0x7f04980066c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f049ea00640 1 -- 172.21.15.94:0/3566779083 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498015070 con 0x7f048c004810 2024-09-10T23:39:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f049ea00640 1 -- 172.21.15.94:0/3566779083 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0498011920 con 0x7f048c004810 2024-09-10T23:39:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f049ea00640 1 -- 172.21.15.94:0/3566779083 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498011d70 con 0x7f048c004810 2024-09-10T23:39:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04acc00640 1 -- 172.21.15.94:0/3566779083 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a815f410 con 0x7f048c004810 2024-09-10T23:39:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04acc00640 1 -- 172.21.15.94:0/3566779083 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a80b5f80 con 0x7f048c004810 2024-09-10T23:39:06.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.416+0000 7f04acc00640 1 -- 172.21.15.94:0/3566779083 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a80cec50 con 0x7f048c004810 2024-09-10T23:39:06.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.420+0000 7f049ea00640 1 -- 172.21.15.94:0/3566779083 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f04980103c0 con 0x7f048c004810 2024-09-10T23:39:06.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.420+0000 7f049ea00640 1 --2- 172.21.15.94:0/3566779083 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0484072c10 0x7f04840750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:06.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.420+0000 7f049ea00640 1 -- 172.21.15.94:0/3566779083 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f0498094e20 con 0x7f048c004810 2024-09-10T23:39:06.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.420+0000 7f049d400640 1 --2- 172.21.15.94:0/3566779083 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0484072c10 0x7f04840750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:06.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.420+0000 7f049d400640 1 --2- 172.21.15.94:0/3566779083 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0484072c10 0x7f04840750d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f048c0057f0 tx=0x7f0490019520 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:06.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.424+0000 7f049ea00640 1 -- 172.21.15.94:0/3566779083 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f049805fb20 con 0x7f048c004810 2024-09-10T23:39:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.620+0000 7f04acc00640 1 -- 172.21.15.94:0/3566779083 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "non_existent_subvolume", "clone_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7f04a80b4510 con 0x7f0484072c10 2024-09-10T23:39:06.626 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:06.624+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:06.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.624+0000 7f049ea00640 1 -- 172.21.15.94:0/3566779083 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f04a80b4510 con 0x7f0484072c10 2024-09-10T23:39:06.626 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:06.635 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:06.635 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create non_existent_subvolume group_1 2024-09-10T23:39:06.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.776+0000 7f0887000640 1 -- 172.21.15.94:0/1998584647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08800ce4e0 msgr2=0x7f08800ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:06.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.776+0000 7f0887000640 1 --2- 172.21.15.94:0/1998584647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08800ce4e0 0x7f08800ce8c0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f0870009bc0 tx=0x7f087001cb10 comp rx=0 tx=0).stop 2024-09-10T23:39:06.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.776+0000 7f0887000640 1 -- 172.21.15.94:0/1998584647 shutdown_connections 2024-09-10T23:39:06.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.776+0000 7f0887000640 1 --2- 172.21.15.94:0/1998584647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08800cee00 0x7f088006dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:06.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.776+0000 7f0887000640 1 --2- 172.21.15.94:0/1998584647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08800ce4e0 0x7f08800ce8c0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:06.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.776+0000 7f0887000640 1 -- 172.21.15.94:0/1998584647 >> 172.21.15.94:0/1998584647 conn(0x7f08800d1c60 msgr2=0x7f08800d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:06.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.776+0000 7f0887000640 1 -- 172.21.15.94:0/1998584647 shutdown_connections 2024-09-10T23:39:06.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.776+0000 7f0887000640 1 -- 172.21.15.94:0/1998584647 wait complete. 2024-09-10T23:39:06.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0887000640 1 Processor -- start 2024-09-10T23:39:06.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0887000640 1 -- start start 2024-09-10T23:39:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0887000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08800b0ee0 0x7f0880150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0887000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08800ce4e0 0x7f0880158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0887000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08800cee00 0x7f08800b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0887000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f088016b5e0 con 0x7f08800cee00 2024-09-10T23:39:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0887000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f088016b460 con 0x7f08800b0ee0 2024-09-10T23:39:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0887000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f088016b760 con 0x7f08800ce4e0 2024-09-10T23:39:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0885000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08800cee00 0x7f08800b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0885000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08800cee00 0x7f08800b63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32826/0 (socket says 172.21.15.94:32826) 2024-09-10T23:39:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0885000640 1 -- 172.21.15.94:0/1329874568 learned_addr learned my addr 172.21.15.94:0/1329874568 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f087fe00640 1 --2- 172.21.15.94:0/1329874568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08800b0ee0 0x7f0880150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f087f400640 1 --2- 172.21.15.94:0/1329874568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08800ce4e0 0x7f0880158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0885000640 1 -- 172.21.15.94:0/1329874568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08800ce4e0 msgr2=0x7f0880158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0885000640 1 --2- 172.21.15.94:0/1329874568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08800ce4e0 0x7f0880158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0885000640 1 -- 172.21.15.94:0/1329874568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08800b0ee0 msgr2=0x7f0880150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0885000640 1 --2- 172.21.15.94:0/1329874568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08800b0ee0 0x7f0880150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0885000640 1 -- 172.21.15.94:0/1329874568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08800b7660 con 0x7f08800cee00 2024-09-10T23:39:06.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0885000640 1 --2- 172.21.15.94:0/1329874568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08800cee00 0x7f08800b63f0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f087800c430 tx=0x7f087800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:06.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0877e00640 1 -- 172.21.15.94:0/1329874568 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878017070 con 0x7f08800cee00 2024-09-10T23:39:06.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0877e00640 1 -- 172.21.15.94:0/1329874568 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0878003600 con 0x7f08800cee00 2024-09-10T23:39:06.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0887000640 1 -- 172.21.15.94:0/1329874568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08800b7940 con 0x7f08800cee00 2024-09-10T23:39:06.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0877e00640 1 -- 172.21.15.94:0/1329874568 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878003a50 con 0x7f08800cee00 2024-09-10T23:39:06.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0887000640 1 -- 172.21.15.94:0/1329874568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08801508a0 con 0x7f08800cee00 2024-09-10T23:39:06.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.780+0000 7f0887000640 1 -- 172.21.15.94:0/1329874568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08800ce8c0 con 0x7f08800cee00 2024-09-10T23:39:06.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.784+0000 7f0877e00640 1 -- 172.21.15.94:0/1329874568 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f0878003bf0 con 0x7f08800cee00 2024-09-10T23:39:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.784+0000 7f0877e00640 1 --2- 172.21.15.94:0/1329874568 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0860072c10 0x7f08600750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.784+0000 7f0877e00640 1 -- 172.21.15.94:0/1329874568 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f08780934a0 con 0x7f08800cee00 2024-09-10T23:39:06.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.784+0000 7f087fe00640 1 --2- 172.21.15.94:0/1329874568 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0860072c10 0x7f08600750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:06.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.784+0000 7f087fe00640 1 --2- 172.21.15.94:0/1329874568 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0860072c10 0x7f08600750d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f08700af000 tx=0x7f087001b350 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.788+0000 7f0877e00640 1 -- 172.21.15.94:0/1329874568 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f087805da80 con 0x7f08800cee00 2024-09-10T23:39:06.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.976+0000 7f0887000640 1 -- 172.21.15.94:0/1329874568 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f088015d9f0 con 0x7f0860072c10 2024-09-10T23:39:06.982 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:06.980+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:06.980+0000 7f0877e00640 1 -- 172.21.15.94:0/1329874568 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f088015d9f0 con 0x7f0860072c10 2024-09-10T23:39:06.982 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:06.991 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:06.992 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm non_existent_subvolume group_1 2024-09-10T23:39:07.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.136+0000 7ff73ae00640 1 -- 172.21.15.94:0/420299695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7340b2570 msgr2=0x7ff7340b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:07.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.136+0000 7ff73ae00640 1 --2- 172.21.15.94:0/420299695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7340b2570 0x7ff7340b29a0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ff734069920 tx=0x7ff72c01ccf0 comp rx=0 tx=0).stop 2024-09-10T23:39:07.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.136+0000 7ff73ae00640 1 -- 172.21.15.94:0/420299695 shutdown_connections 2024-09-10T23:39:07.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.136+0000 7ff73ae00640 1 --2- 172.21.15.94:0/420299695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7340b2570 0x7ff7340b29a0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:07.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.136+0000 7ff73ae00640 1 --2- 172.21.15.94:0/420299695 >> v2:172.21.15.181:3300/0 conn(0x7ff7340b3150 0x7ff7340d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:07.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.136+0000 7ff73ae00640 1 -- 172.21.15.94:0/420299695 >> 172.21.15.94:0/420299695 conn(0x7ff7340d1cb0 msgr2=0x7ff7340d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:07.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.136+0000 7ff73ae00640 1 -- 172.21.15.94:0/420299695 shutdown_connections 2024-09-10T23:39:07.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.136+0000 7ff73ae00640 1 -- 172.21.15.94:0/420299695 wait complete. 2024-09-10T23:39:07.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.136+0000 7ff73ae00640 1 Processor -- start 2024-09-10T23:39:07.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.136+0000 7ff73ae00640 1 -- start start 2024-09-10T23:39:07.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff73ae00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7340b2570 0x7ff7340c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:07.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff73ae00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7340b3150 0x7ff73415a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:07.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff73ae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7341613c0 0x7ff7340c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:07.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff73ae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff73416b0d0 con 0x7ff7341613c0 2024-09-10T23:39:07.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff73ae00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff73416af50 con 0x7ff7340b2570 2024-09-10T23:39:07.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff73ae00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff73416b250 con 0x7ff7340b3150 2024-09-10T23:39:07.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff738e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7341613c0 0x7ff7340c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:07.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff738e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7341613c0 0x7ff7340c9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32840/0 (socket says 172.21.15.94:32840) 2024-09-10T23:39:07.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff738e00640 1 -- 172.21.15.94:0/2035253055 learned_addr learned my addr 172.21.15.94:0/2035253055 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:07.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff733e00640 1 --2- 172.21.15.94:0/2035253055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7340b2570 0x7ff7340c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:07.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff733400640 1 --2- 172.21.15.94:0/2035253055 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7340b3150 0x7ff73415a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:07.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff738e00640 1 -- 172.21.15.94:0/2035253055 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7340b3150 msgr2=0x7ff73415a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:07.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff738e00640 1 --2- 172.21.15.94:0/2035253055 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7340b3150 0x7ff73415a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:07.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff738e00640 1 -- 172.21.15.94:0/2035253055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7340b2570 msgr2=0x7ff7340c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:07.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff738e00640 1 --2- 172.21.15.94:0/2035253055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7340b2570 0x7ff7340c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:07.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff738e00640 1 -- 172.21.15.94:0/2035253055 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7341fb990 con 0x7ff7341613c0 2024-09-10T23:39:07.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff733e00640 1 --2- 172.21.15.94:0/2035253055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7340b2570 0x7ff7340c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:07.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff738e00640 1 --2- 172.21.15.94:0/2035253055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7341613c0 0x7ff7340c9d50 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7ff72c019900 tx=0x7ff72c003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:07.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff72be00640 1 -- 172.21.15.94:0/2035253055 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c0bb070 con 0x7ff7341613c0 2024-09-10T23:39:07.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff72be00640 1 -- 172.21.15.94:0/2035253055 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff72c004d40 con 0x7ff7341613c0 2024-09-10T23:39:07.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff72be00640 1 -- 172.21.15.94:0/2035253055 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c0b2040 con 0x7ff7341613c0 2024-09-10T23:39:07.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff73ae00640 1 -- 172.21.15.94:0/2035253055 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7341fbc70 con 0x7ff7341613c0 2024-09-10T23:39:07.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff73ae00640 1 -- 172.21.15.94:0/2035253055 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7341fc230 con 0x7ff7341613c0 2024-09-10T23:39:07.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.140+0000 7ff73ae00640 1 -- 172.21.15.94:0/2035253055 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7340cc1d0 con 0x7ff7341613c0 2024-09-10T23:39:07.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.144+0000 7ff72be00640 1 -- 172.21.15.94:0/2035253055 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7ff72c0b5030 con 0x7ff7341613c0 2024-09-10T23:39:07.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.144+0000 7ff72be00640 1 --2- 172.21.15.94:0/2035253055 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff714072c10 0x7ff7140750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:07.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.144+0000 7ff72be00640 1 -- 172.21.15.94:0/2035253055 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7ff72c12db40 con 0x7ff7341613c0 2024-09-10T23:39:07.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.144+0000 7ff733e00640 1 --2- 172.21.15.94:0/2035253055 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff714072c10 0x7ff7140750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:07.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.144+0000 7ff733e00640 1 --2- 172.21.15.94:0/2035253055 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff714072c10 0x7ff7140750d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff7240016b0 tx=0x7ff724009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:07.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.148+0000 7ff72be00640 1 -- 172.21.15.94:0/2035253055 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff72c016970 con 0x7ff7341613c0 2024-09-10T23:39:07.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.336+0000 7ff73ae00640 1 -- 172.21.15.94:0/2035253055 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7ff7340b29a0 con 0x7ff714072c10 2024-09-10T23:39:07.339 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:07.336+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:07.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.336+0000 7ff72be00640 1 -- 172.21.15.94:0/2035253055 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7ff7340b29a0 con 0x7ff714072c10 2024-09-10T23:39:07.339 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:07.348 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:07.349 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup getpath non_existent_subvolume group_1 2024-09-10T23:39:07.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd2c00640 1 -- 172.21.15.94:0/4040988894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bcc13cfe0 msgr2=0x7f3bcc0cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:07.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd2c00640 1 --2- 172.21.15.94:0/4040988894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bcc13cfe0 0x7f3bcc0cec60 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400aa10 tx=0x7f3bc401db70 comp rx=0 tx=0).stop 2024-09-10T23:39:07.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd2c00640 1 -- 172.21.15.94:0/4040988894 shutdown_connections 2024-09-10T23:39:07.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd2c00640 1 --2- 172.21.15.94:0/4040988894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bcc13cfe0 0x7f3bcc0cec60 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:07.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd2c00640 1 --2- 172.21.15.94:0/4040988894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bcc15e180 0x7f3bcc0ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:07.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd2c00640 1 -- 172.21.15.94:0/4040988894 >> 172.21.15.94:0/4040988894 conn(0x7f3bcc0cfaf0 msgr2=0x7f3bcc0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:07.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd2c00640 1 -- 172.21.15.94:0/4040988894 shutdown_connections 2024-09-10T23:39:07.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd2c00640 1 -- 172.21.15.94:0/4040988894 wait complete. 2024-09-10T23:39:07.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd2c00640 1 Processor -- start 2024-09-10T23:39:07.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd2c00640 1 -- start start 2024-09-10T23:39:07.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd2c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bcc13cc00 0x7f3bcc0cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:07.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd2c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bcc13cfe0 0x7f3bcc13b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:07.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd2c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bcc15e180 0x7f3bcc0c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:07.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd2c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc13f800 con 0x7f3bcc13cc00 2024-09-10T23:39:07.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd2c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc13f680 con 0x7f3bcc13cfe0 2024-09-10T23:39:07.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd2c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3bcc13f980 con 0x7f3bcc15e180 2024-09-10T23:39:07.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bcbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bcc13cc00 0x7f3bcc0cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:07.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bcbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bcc13cc00 0x7f3bcc0cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32866/0 (socket says 172.21.15.94:32866) 2024-09-10T23:39:07.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bcbe00640 1 -- 172.21.15.94:0/3707008456 learned_addr learned my addr 172.21.15.94:0/3707008456 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:07.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bcb400640 1 --2- 172.21.15.94:0/3707008456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bcc13cfe0 0x7f3bcc13b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:07.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.492+0000 7f3bd0c00640 1 --2- 172.21.15.94:0/3707008456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bcc15e180 0x7f3bcc0c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:07.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.496+0000 7f3bcbe00640 1 -- 172.21.15.94:0/3707008456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bcc15e180 msgr2=0x7f3bcc0c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:07.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.496+0000 7f3bcbe00640 1 --2- 172.21.15.94:0/3707008456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bcc15e180 0x7f3bcc0c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.496+0000 7f3bcbe00640 1 -- 172.21.15.94:0/3707008456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bcc13cfe0 msgr2=0x7f3bcc13b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.496+0000 7f3bcbe00640 1 --2- 172.21.15.94:0/3707008456 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bcc13cfe0 0x7f3bcc13b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.496+0000 7f3bcbe00640 1 -- 172.21.15.94:0/3707008456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc0cb6d0 con 0x7f3bcc13cc00 2024-09-10T23:39:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.496+0000 7f3bcbe00640 1 --2- 172.21.15.94:0/3707008456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bcc13cc00 0x7f3bcc0cb0f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00c500 tx=0x7f3bbc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.496+0000 7f3bc3e00640 1 -- 172.21.15.94:0/3707008456 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc017070 con 0x7f3bcc13cc00 2024-09-10T23:39:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.496+0000 7f3bc3e00640 1 -- 172.21.15.94:0/3707008456 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bbc0053c0 con 0x7f3bcc13cc00 2024-09-10T23:39:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.496+0000 7f3bc3e00640 1 -- 172.21.15.94:0/3707008456 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc012890 con 0x7f3bcc13cc00 2024-09-10T23:39:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.496+0000 7f3bd2c00640 1 -- 172.21.15.94:0/3707008456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bcc15c090 con 0x7f3bcc13cc00 2024-09-10T23:39:07.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.496+0000 7f3bd2c00640 1 -- 172.21.15.94:0/3707008456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bcc0cc1c0 con 0x7f3bcc13cc00 2024-09-10T23:39:07.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.496+0000 7f3bd2c00640 1 -- 172.21.15.94:0/3707008456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bcc13ff10 con 0x7f3bcc13cc00 2024-09-10T23:39:07.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.496+0000 7f3bc3e00640 1 -- 172.21.15.94:0/3707008456 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f3bbc003600 con 0x7f3bcc13cc00 2024-09-10T23:39:07.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.496+0000 7f3bc3e00640 1 --2- 172.21.15.94:0/3707008456 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3bac072c10 0x7f3bac0750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:07.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.496+0000 7f3bc3e00640 1 -- 172.21.15.94:0/3707008456 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f3bbc093460 con 0x7f3bcc13cc00 2024-09-10T23:39:07.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.496+0000 7f3bcb400640 1 --2- 172.21.15.94:0/3707008456 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3bac072c10 0x7f3bac0750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:07.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.500+0000 7f3bcb400640 1 --2- 172.21.15.94:0/3707008456 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3bac072c10 0x7f3bac0750d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0cc4e0 tx=0x7f3bb80095e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:07.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.500+0000 7f3bc3e00640 1 -- 172.21.15.94:0/3707008456 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3bbc05da50 con 0x7f3bcc13cc00 2024-09-10T23:39:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.692+0000 7f3bd2c00640 1 -- 172.21.15.94:0/3707008456 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup getpath", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f3bcc0cf770 con 0x7f3bac072c10 2024-09-10T23:39:07.695 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:07.692+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.692+0000 7f3bc3e00640 1 -- 172.21.15.94:0/3707008456 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f3bcc0cf770 con 0x7f3bac072c10 2024-09-10T23:39:07.696 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:07.704 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:07.705 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup pin non_existent_subvolume group_1 export 0 2024-09-10T23:39:07.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.848+0000 7f7ad6000640 1 -- 172.21.15.94:0/3128714623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad0071a20 msgr2=0x7f7ad015f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:07.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.848+0000 7f7ad6000640 1 --2- 172.21.15.94:0/3128714623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad0071a20 0x7f7ad015f610 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00b6ca0 tx=0x7f7ab801c830 comp rx=0 tx=0).stop 2024-09-10T23:39:07.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.848+0000 7f7ad6000640 1 -- 172.21.15.94:0/3128714623 shutdown_connections 2024-09-10T23:39:07.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.848+0000 7f7ad6000640 1 --2- 172.21.15.94:0/3128714623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad0071a20 0x7f7ad015f610 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:07.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.848+0000 7f7ad6000640 1 --2- 172.21.15.94:0/3128714623 >> v2:172.21.15.181:3300/0 conn(0x7f7ad006cc20 0x7f7ad006d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:07.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.848+0000 7f7ad6000640 1 -- 172.21.15.94:0/3128714623 >> 172.21.15.94:0/3128714623 conn(0x7f7ad00d0b70 msgr2=0x7f7ad00d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:07.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.848+0000 7f7ad6000640 1 -- 172.21.15.94:0/3128714623 shutdown_connections 2024-09-10T23:39:07.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.848+0000 7f7ad6000640 1 -- 172.21.15.94:0/3128714623 wait complete. 2024-09-10T23:39:07.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ad6000640 1 Processor -- start 2024-09-10T23:39:07.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ad6000640 1 -- start start 2024-09-10T23:39:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ad6000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ad006cc20 0x7f7ad0165d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ad6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad0071a20 0x7f7ad0143ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ad6000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad00d1690 0x7f7ad0147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ad6000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad0169130 con 0x7f7ad0071a20 2024-09-10T23:39:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ad6000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad0168fb0 con 0x7f7ad00d1690 2024-09-10T23:39:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ad6000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad01692b0 con 0x7f7ad006cc20 2024-09-10T23:39:07.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ac6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad0071a20 0x7f7ad0143ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ac6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad0071a20 0x7f7ad0143ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32888/0 (socket says 172.21.15.94:32888) 2024-09-10T23:39:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ac6a00640 1 -- 172.21.15.94:0/762910667 learned_addr learned my addr 172.21.15.94:0/762910667 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7acfe00640 1 --2- 172.21.15.94:0/762910667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad00d1690 0x7f7ad0147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7acf400640 1 --2- 172.21.15.94:0/762910667 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ad006cc20 0x7f7ad0165d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ac6a00640 1 -- 172.21.15.94:0/762910667 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ad006cc20 msgr2=0x7f7ad0165d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ac6a00640 1 --2- 172.21.15.94:0/762910667 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ad006cc20 0x7f7ad0165d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ac6a00640 1 -- 172.21.15.94:0/762910667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad00d1690 msgr2=0x7f7ad0147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ac6a00640 1 --2- 172.21.15.94:0/762910667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad00d1690 0x7f7ad0147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ac6a00640 1 -- 172.21.15.94:0/762910667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad014d040 con 0x7f7ad0071a20 2024-09-10T23:39:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7acfe00640 1 --2- 172.21.15.94:0/762910667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ad00d1690 0x7f7ad0147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:07.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ac6a00640 1 --2- 172.21.15.94:0/762910667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ad0071a20 0x7f7ad0143ce0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80b0040 tx=0x7f7ab8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:07.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ac7e00640 1 -- 172.21.15.94:0/762910667 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80bd030 con 0x7f7ad0071a20 2024-09-10T23:39:07.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ac7e00640 1 -- 172.21.15.94:0/762910667 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ab8004340 con 0x7f7ad0071a20 2024-09-10T23:39:07.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ac7e00640 1 -- 172.21.15.94:0/762910667 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8003860 con 0x7f7ad0071a20 2024-09-10T23:39:07.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ad6000640 1 -- 172.21.15.94:0/762910667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad0166290 con 0x7f7ad0071a20 2024-09-10T23:39:07.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ad6000640 1 -- 172.21.15.94:0/762910667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad01fb4d0 con 0x7f7ad0071a20 2024-09-10T23:39:07.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.852+0000 7f7ad6000640 1 -- 172.21.15.94:0/762910667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad006d000 con 0x7f7ad0071a20 2024-09-10T23:39:07.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.856+0000 7f7ac7e00640 1 -- 172.21.15.94:0/762910667 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f7ab8003a00 con 0x7f7ad0071a20 2024-09-10T23:39:07.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.856+0000 7f7ac7e00640 1 --2- 172.21.15.94:0/762910667 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7aac072c10 0x7f7aac0750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:07.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.856+0000 7f7ac7e00640 1 -- 172.21.15.94:0/762910667 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f7ab812ed60 con 0x7f7ad0071a20 2024-09-10T23:39:07.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.856+0000 7f7acf400640 1 --2- 172.21.15.94:0/762910667 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7aac072c10 0x7f7aac0750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:07.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.856+0000 7f7acf400640 1 --2- 172.21.15.94:0/762910667 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7aac072c10 0x7f7aac0750d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0005fd0 tx=0x7f7ac0007600 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:07.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:07.860+0000 7f7ac7e00640 1 -- 172.21.15.94:0/762910667 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7ab80c8020 con 0x7f7ad0071a20 2024-09-10T23:39:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.048+0000 7f7ad6000640 1 -- 172.21.15.94:0/762910667 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "non_existent_subvolume", "group_name": "group_1", "pin_type": "export", "pin_setting": "0", "target": ["mon-mgr", ""]}) -- 0x7f7ad0156410 con 0x7f7aac072c10 2024-09-10T23:39:08.054 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:08.052+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.052+0000 7f7ac7e00640 1 -- 172.21.15.94:0/762910667 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f7ad0156410 con 0x7f7aac072c10 2024-09-10T23:39:08.054 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:08.063 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:08.064 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup ls non_existent_subvolume 2024-09-10T23:39:08.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.204+0000 7f0d21400640 1 -- 172.21.15.94:0/2439275406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d1c0ce1e0 msgr2=0x7f0d1c0cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:08.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.204+0000 7f0d21400640 1 --2- 172.21.15.94:0/2439275406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d1c0ce1e0 0x7f0d1c0cd710 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0d18b0 tx=0x7f0d0801c830 comp rx=0 tx=0).stop 2024-09-10T23:39:08.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.204+0000 7f0d21400640 1 -- 172.21.15.94:0/2439275406 shutdown_connections 2024-09-10T23:39:08.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.204+0000 7f0d21400640 1 --2- 172.21.15.94:0/2439275406 >> v2:172.21.15.181:3300/0 conn(0x7f0d1c0b1100 0x7f0d1c0b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:08.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.204+0000 7f0d21400640 1 --2- 172.21.15.94:0/2439275406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d1c0ce1e0 0x7f0d1c0cd710 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:08.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.204+0000 7f0d21400640 1 -- 172.21.15.94:0/2439275406 >> 172.21.15.94:0/2439275406 conn(0x7f0d1c0d1c60 msgr2=0x7f0d1c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:08.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.204+0000 7f0d21400640 1 -- 172.21.15.94:0/2439275406 shutdown_connections 2024-09-10T23:39:08.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.204+0000 7f0d21400640 1 -- 172.21.15.94:0/2439275406 wait complete. 2024-09-10T23:39:08.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d21400640 1 Processor -- start 2024-09-10T23:39:08.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d21400640 1 -- start start 2024-09-10T23:39:08.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d21400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d1c0b1100 0x7f0d1c0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d21400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d1c0ce1e0 0x7f0d1c1f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d21400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d1c0ce890 0x7f0d1c150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d21400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c0d0b30 con 0x7f0d1c0ce890 2024-09-10T23:39:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d21400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c0d09b0 con 0x7f0d1c0ce1e0 2024-09-10T23:39:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d21400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1c169110 con 0x7f0d1c0b1100 2024-09-10T23:39:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d1b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d1c0ce890 0x7f0d1c150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d1b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d1c0ce890 0x7f0d1c150800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32902/0 (socket says 172.21.15.94:32902) 2024-09-10T23:39:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d1b200640 1 -- 172.21.15.94:0/1062093826 learned_addr learned my addr 172.21.15.94:0/1062093826 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d19e00640 1 --2- 172.21.15.94:0/1062093826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d1c0ce1e0 0x7f0d1c1f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d1a800640 1 --2- 172.21.15.94:0/1062093826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d1c0b1100 0x7f0d1c0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d1b200640 1 -- 172.21.15.94:0/1062093826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d1c0b1100 msgr2=0x7f0d1c0b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d1b200640 1 --2- 172.21.15.94:0/1062093826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d1c0b1100 0x7f0d1c0b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:08.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d1b200640 1 -- 172.21.15.94:0/1062093826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d1c0ce1e0 msgr2=0x7f0d1c1f82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d1b200640 1 --2- 172.21.15.94:0/1062093826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d1c0ce1e0 0x7f0d1c1f82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d1b200640 1 -- 172.21.15.94:0/1062093826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1c1f8810 con 0x7f0d1c0ce890 2024-09-10T23:39:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d1b200640 1 --2- 172.21.15.94:0/1062093826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d1c0ce890 0x7f0d1c150800 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400daa0 tx=0x7f0d1400df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d12a00640 1 -- 172.21.15.94:0/1062093826 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d14017070 con 0x7f0d1c0ce890 2024-09-10T23:39:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d12a00640 1 -- 172.21.15.94:0/1062093826 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d14004b30 con 0x7f0d1c0ce890 2024-09-10T23:39:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d12a00640 1 -- 172.21.15.94:0/1062093826 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d14005020 con 0x7f0d1c0ce890 2024-09-10T23:39:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d21400640 1 -- 172.21.15.94:0/1062093826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1c1514e0 con 0x7f0d1c0ce890 2024-09-10T23:39:08.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d21400640 1 -- 172.21.15.94:0/1062093826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1c1653e0 con 0x7f0d1c0ce890 2024-09-10T23:39:08.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.208+0000 7f0d21400640 1 -- 172.21.15.94:0/1062093826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1c0b1590 con 0x7f0d1c0ce890 2024-09-10T23:39:08.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.212+0000 7f0d12a00640 1 -- 172.21.15.94:0/1062093826 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 96955+0+0 (secure 0 0 0) 0x7f0d140051c0 con 0x7f0d1c0ce890 2024-09-10T23:39:08.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.212+0000 7f0d12a00640 1 --2- 172.21.15.94:0/1062093826 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0cfc072c10 0x7f0cfc0750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.212+0000 7f0d12a00640 1 -- 172.21.15.94:0/1062093826 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f0d140942e0 con 0x7f0d1c0ce890 2024-09-10T23:39:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.212+0000 7f0d1a800640 1 --2- 172.21.15.94:0/1062093826 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0cfc072c10 0x7f0cfc0750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.212+0000 7f0d1a800640 1 --2- 172.21.15.94:0/1062093826 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0cfc072c10 0x7f0cfc0750d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c007fd0 tx=0x7f0d0c009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.216+0000 7f0d12a00640 1 -- 172.21.15.94:0/1062093826 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0d14020070 con 0x7f0d1c0ce890 2024-09-10T23:39:08.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.404+0000 7f0d21400640 1 -- 172.21.15.94:0/1062093826 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7f0d1c06f0e0 con 0x7f0cfc072c10 2024-09-10T23:39:08.409 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:08.408+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:08.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.408+0000 7f0d12a00640 1 -- 172.21.15.94:0/1062093826 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f0d0c001d20 con 0x7f0cfc072c10 2024-09-10T23:39:08.410 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:08.418 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:08.419 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create non_existent_subvolume group_1 snapshot_1 2024-09-10T23:39:08.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.564+0000 7f34ece00640 1 -- 172.21.15.94:0/3948958443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34e80d0010 msgr2=0x7f34e815fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:08.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.564+0000 7f34ece00640 1 --2- 172.21.15.94:0/3948958443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34e80d0010 0x7f34e815fbe0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009900 tx=0x7f34d001f0d0 comp rx=0 tx=0).stop 2024-09-10T23:39:08.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.564+0000 7f34ece00640 1 -- 172.21.15.94:0/3948958443 shutdown_connections 2024-09-10T23:39:08.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.564+0000 7f34ece00640 1 --2- 172.21.15.94:0/3948958443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34e80d0010 0x7f34e815fbe0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:08.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.564+0000 7f34ece00640 1 -- 172.21.15.94:0/3948958443 >> 172.21.15.94:0/3948958443 conn(0x7f34e80d1bf0 msgr2=0x7f34e80d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:08.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.564+0000 7f34ece00640 1 -- 172.21.15.94:0/3948958443 shutdown_connections 2024-09-10T23:39:08.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.564+0000 7f34ece00640 1 -- 172.21.15.94:0/3948958443 wait complete. 2024-09-10T23:39:08.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34ece00640 1 Processor -- start 2024-09-10T23:39:08.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34ece00640 1 -- start start 2024-09-10T23:39:08.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34ece00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e80cfc30 0x7f34e806f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:08.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34ece00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34e80d0010 0x7f34e80c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:08.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34ece00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34e8160120 0x7f34e81f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:08.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34ece00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f34e8160a30 con 0x7f34e80d0010 2024-09-10T23:39:08.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34ece00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81608b0 con 0x7f34e8160120 2024-09-10T23:39:08.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34ece00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f34e80b1f90 con 0x7f34e80cfc30 2024-09-10T23:39:08.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34dd400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34e80d0010 0x7f34e80c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:08.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34dd400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34e80d0010 0x7f34e80c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32924/0 (socket says 172.21.15.94:32924) 2024-09-10T23:39:08.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34dd400640 1 -- 172.21.15.94:0/1316341286 learned_addr learned my addr 172.21.15.94:0/1316341286 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:08.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34e5e00640 1 --2- 172.21.15.94:0/1316341286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e80cfc30 0x7f34e806f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:08.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34e6800640 1 --2- 172.21.15.94:0/1316341286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34e8160120 0x7f34e81f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34dd400640 1 -- 172.21.15.94:0/1316341286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e80cfc30 msgr2=0x7f34e806f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34dd400640 1 --2- 172.21.15.94:0/1316341286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e80cfc30 0x7f34e806f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34dd400640 1 -- 172.21.15.94:0/1316341286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34e8160120 msgr2=0x7f34e81f7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34dd400640 1 --2- 172.21.15.94:0/1316341286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34e8160120 0x7f34e81f7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34dd400640 1 -- 172.21.15.94:0/1316341286 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e806fd20 con 0x7f34e80d0010 2024-09-10T23:39:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34e6800640 1 --2- 172.21.15.94:0/1316341286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34e8160120 0x7f34e81f7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34dd400640 1 --2- 172.21.15.94:0/1316341286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34e80d0010 0x7f34e80c7bc0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009850 tx=0x7f34d00a8ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34dea00640 1 -- 172.21.15.94:0/1316341286 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d00b5040 con 0x7f34e80d0010 2024-09-10T23:39:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34dea00640 1 -- 172.21.15.94:0/1316341286 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34d0003770 con 0x7f34e80d0010 2024-09-10T23:39:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34ece00640 1 -- 172.21.15.94:0/1316341286 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e806fa40 con 0x7f34e80d0010 2024-09-10T23:39:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34dea00640 1 -- 172.21.15.94:0/1316341286 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0003bc0 con 0x7f34e80d0010 2024-09-10T23:39:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.568+0000 7f34ece00640 1 -- 172.21.15.94:0/1316341286 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e815ace0 con 0x7f34e80d0010 2024-09-10T23:39:08.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.572+0000 7f34ece00640 1 -- 172.21.15.94:0/1316341286 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e815fbe0 con 0x7f34e80d0010 2024-09-10T23:39:08.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.572+0000 7f34dea00640 1 -- 172.21.15.94:0/1316341286 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f34d00b8050 con 0x7f34e80d0010 2024-09-10T23:39:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.572+0000 7f34dea00640 1 --2- 172.21.15.94:0/1316341286 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f34c4072ca0 0x7f34c4075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.572+0000 7f34dea00640 1 -- 172.21.15.94:0/1316341286 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f34d0130ee0 con 0x7f34e80d0010 2024-09-10T23:39:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.572+0000 7f34e5e00640 1 --2- 172.21.15.94:0/1316341286 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f34c4072ca0 0x7f34c4075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.572+0000 7f34e5e00640 1 --2- 172.21.15.94:0/1316341286 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f34c4072ca0 0x7f34c4075160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f34d80012f0 tx=0x7f34d8009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:08.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.576+0000 7f34dea00640 1 -- 172.21.15.94:0/1316341286 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f34d0018c30 con 0x7f34e80d0010 2024-09-10T23:39:08.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.768+0000 7f34ece00640 1 -- 172.21.15.94:0/1316341286 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f34e80c8100 con 0x7f34c4072ca0 2024-09-10T23:39:08.771 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:08.768+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:08.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.768+0000 7f34dea00640 1 -- 172.21.15.94:0/1316341286 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f34e80c8100 con 0x7f34c4072ca0 2024-09-10T23:39:08.772 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:08.780 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:08.780 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot rm non_existent_subvolume group_1 snapshot_1 2024-09-10T23:39:08.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.924+0000 7f21e3c00640 1 -- 172.21.15.94:0/2373836685 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21dc15f5c0 msgr2=0x7f21dc0cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.924+0000 7f21e3c00640 1 --2- 172.21.15.94:0/2373836685 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21dc15f5c0 0x7f21dc0cfb60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009870 tx=0x7f21d001c740 comp rx=0 tx=0).stop 2024-09-10T23:39:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.924+0000 7f21e3c00640 1 -- 172.21.15.94:0/2373836685 shutdown_connections 2024-09-10T23:39:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.924+0000 7f21e3c00640 1 --2- 172.21.15.94:0/2373836685 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21dc15f5c0 0x7f21dc0cfb60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.924+0000 7f21e3c00640 1 -- 172.21.15.94:0/2373836685 >> 172.21.15.94:0/2373836685 conn(0x7f21dc0d1c60 msgr2=0x7f21dc0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.924+0000 7f21e3c00640 1 -- 172.21.15.94:0/2373836685 shutdown_connections 2024-09-10T23:39:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.924+0000 7f21e3c00640 1 -- 172.21.15.94:0/2373836685 wait complete. 2024-09-10T23:39:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e3c00640 1 Processor -- start 2024-09-10T23:39:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e3c00640 1 -- start start 2024-09-10T23:39:08.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e3c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21dc0af310 0x7f21dc1461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:08.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e3c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21dc0b2bb0 0x7f21dc156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e3c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21dc15f5c0 0x7f21dc14d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e3c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc16a0c0 con 0x7f21dc15f5c0 2024-09-10T23:39:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e3c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc169f40 con 0x7f21dc0b2bb0 2024-09-10T23:39:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e3c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f21dc16a240 con 0x7f21dc0af310 2024-09-10T23:39:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e1c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21dc15f5c0 0x7f21dc14d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e1c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21dc15f5c0 0x7f21dc14d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32934/0 (socket says 172.21.15.94:32934) 2024-09-10T23:39:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e1c00640 1 -- 172.21.15.94:0/4168262830 learned_addr learned my addr 172.21.15.94:0/4168262830 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21dbe00640 1 --2- 172.21.15.94:0/4168262830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21dc0b2bb0 0x7f21dc156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e1200640 1 --2- 172.21.15.94:0/4168262830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21dc0af310 0x7f21dc1461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e1c00640 1 -- 172.21.15.94:0/4168262830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21dc0af310 msgr2=0x7f21dc1461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e1c00640 1 --2- 172.21.15.94:0/4168262830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21dc0af310 0x7f21dc1461c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e1c00640 1 -- 172.21.15.94:0/4168262830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21dc0b2bb0 msgr2=0x7f21dc156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:08.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e1c00640 1 --2- 172.21.15.94:0/4168262830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21dc0b2bb0 0x7f21dc156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:08.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e1c00640 1 -- 172.21.15.94:0/4168262830 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21dc0b2100 con 0x7f21dc15f5c0 2024-09-10T23:39:08.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e1c00640 1 --2- 172.21.15.94:0/4168262830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21dc15f5c0 0x7f21dc14d480 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f21d400c7c0 tx=0x7f21d400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:08.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21d8c00640 1 -- 172.21.15.94:0/4168262830 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4003b80 con 0x7f21dc15f5c0 2024-09-10T23:39:08.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21d8c00640 1 -- 172.21.15.94:0/4168262830 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21d4003d20 con 0x7f21dc15f5c0 2024-09-10T23:39:08.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21d8c00640 1 -- 172.21.15.94:0/4168262830 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d400aa30 con 0x7f21dc15f5c0 2024-09-10T23:39:08.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e3c00640 1 -- 172.21.15.94:0/4168262830 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21dc1fb510 con 0x7f21dc15f5c0 2024-09-10T23:39:08.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e3c00640 1 -- 172.21.15.94:0/4168262830 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21dc1fbad0 con 0x7f21dc15f5c0 2024-09-10T23:39:08.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.928+0000 7f21e3c00640 1 -- 172.21.15.94:0/4168262830 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21dc0af7c0 con 0x7f21dc15f5c0 2024-09-10T23:39:08.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.932+0000 7f21d8c00640 1 -- 172.21.15.94:0/4168262830 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f21d400abd0 con 0x7f21dc15f5c0 2024-09-10T23:39:08.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.932+0000 7f21d8c00640 1 --2- 172.21.15.94:0/4168262830 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f21bc072ca0 0x7f21bc075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:08.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.932+0000 7f21d8c00640 1 -- 172.21.15.94:0/4168262830 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f21d4093b10 con 0x7f21dc15f5c0 2024-09-10T23:39:08.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.932+0000 7f21e1200640 1 --2- 172.21.15.94:0/4168262830 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f21bc072ca0 0x7f21bc075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:08.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.932+0000 7f21e1200640 1 --2- 172.21.15.94:0/4168262830 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f21bc072ca0 0x7f21bc075160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f21d00af000 tx=0x7f21d00b2040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:08.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:08.936+0000 7f21d8c00640 1 -- 172.21.15.94:0/4168262830 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f21d405e230 con 0x7f21dc15f5c0 2024-09-10T23:39:09.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.128+0000 7f21e3c00640 1 -- 172.21.15.94:0/4168262830 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f21dc0749c0 con 0x7f21bc072ca0 2024-09-10T23:39:09.130 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:09.128+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:09.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.128+0000 7f21d8c00640 1 -- 172.21.15.94:0/4168262830 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f21dc0749c0 con 0x7f21bc072ca0 2024-09-10T23:39:09.131 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:09.140 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:09.140 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot ls non_existent_subvolume group_1 2024-09-10T23:39:09.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.284+0000 7fc2b0400640 1 -- 172.21.15.94:0/223650213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc290003f80 msgr2=0x7fc290004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:09.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.284+0000 7fc2b0400640 1 --2- 172.21.15.94:0/223650213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc290003f80 0x7fc290004810 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc294006f50 tx=0x7fc29401ce80 comp rx=0 tx=0).stop 2024-09-10T23:39:09.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.284+0000 7fc2ae400640 1 -- 172.21.15.94:0/223650213 reap_dead start 2024-09-10T23:39:09.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.284+0000 7fc2b0400640 1 -- 172.21.15.94:0/223650213 shutdown_connections 2024-09-10T23:39:09.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.284+0000 7fc2b0400640 1 -- 172.21.15.94:0/223650213 >> 172.21.15.94:0/223650213 conn(0x7fc2a806cdf0 msgr2=0x7fc2a806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:09.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.284+0000 7fc2b0400640 1 -- 172.21.15.94:0/223650213 shutdown_connections 2024-09-10T23:39:09.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.284+0000 7fc2b0400640 1 -- 172.21.15.94:0/223650213 wait complete. 2024-09-10T23:39:09.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.284+0000 7fc2b0400640 1 Processor -- start 2024-09-10T23:39:09.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2b0400640 1 -- start start 2024-09-10T23:39:09.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2b0400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290003f80 0x7fc2a8147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:09.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2b0400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a81e5830 0x7fc2a80ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:09.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2b0400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2a81569c0 0x7fc2a8156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:09.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2b0400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a80b35c0 con 0x7fc2a81569c0 2024-09-10T23:39:09.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2b0400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a80729e0 con 0x7fc2a81e5830 2024-09-10T23:39:09.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2b0400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a80b3740 con 0x7fc290003f80 2024-09-10T23:39:09.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2ae400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2a81569c0 0x7fc2a8156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:09.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2ae400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2a81569c0 0x7fc2a8156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32958/0 (socket says 172.21.15.94:32958) 2024-09-10T23:39:09.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2ae400640 1 -- 172.21.15.94:0/3215314508 learned_addr learned my addr 172.21.15.94:0/3215314508 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:09.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2ad000640 1 --2- 172.21.15.94:0/3215314508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a81e5830 0x7fc2a80ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:09.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2ae400640 1 -- 172.21.15.94:0/3215314508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290003f80 msgr2=0x7fc2a8147ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:09.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2ada00640 1 --2- 172.21.15.94:0/3215314508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290003f80 0x7fc2a8147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:09.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2ae400640 1 --2- 172.21.15.94:0/3215314508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290003f80 0x7fc2a8147ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:09.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2ae400640 1 -- 172.21.15.94:0/3215314508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a81e5830 msgr2=0x7fc2a80ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:09.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2ae400640 1 --2- 172.21.15.94:0/3215314508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a81e5830 0x7fc2a80ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:09.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2ae400640 1 -- 172.21.15.94:0/3215314508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a81483e0 con 0x7fc2a81569c0 2024-09-10T23:39:09.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2ad000640 1 --2- 172.21.15.94:0/3215314508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a81e5830 0x7fc2a80ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:09.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2ada00640 1 --2- 172.21.15.94:0/3215314508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc290003f80 0x7fc2a8147ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:09.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2ae400640 1 --2- 172.21.15.94:0/3215314508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2a81569c0 0x7fc2a8156e50 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80b3470 tx=0x7fc2a0012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:09.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2a5600640 1 -- 172.21.15.94:0/3215314508 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0010dc0 con 0x7fc2a81569c0 2024-09-10T23:39:09.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2a5600640 1 -- 172.21.15.94:0/3215314508 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2a0017030 con 0x7fc2a81569c0 2024-09-10T23:39:09.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2b0400640 1 -- 172.21.15.94:0/3215314508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a81e5cc0 con 0x7fc2a81569c0 2024-09-10T23:39:09.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2a5600640 1 -- 172.21.15.94:0/3215314508 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a00136b0 con 0x7fc2a81569c0 2024-09-10T23:39:09.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2b0400640 1 -- 172.21.15.94:0/3215314508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a80c8fa0 con 0x7fc2a81569c0 2024-09-10T23:39:09.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.288+0000 7fc2b0400640 1 -- 172.21.15.94:0/3215314508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a806e7e0 con 0x7fc2a81569c0 2024-09-10T23:39:09.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.292+0000 7fc2a5600640 1 -- 172.21.15.94:0/3215314508 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7fc2a0013850 con 0x7fc2a81569c0 2024-09-10T23:39:09.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.292+0000 7fc2a5600640 1 --2- 172.21.15.94:0/3215314508 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc288072cd0 0x7fc288075190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:09.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.292+0000 7fc2a5600640 1 -- 172.21.15.94:0/3215314508 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7fc2a0098000 con 0x7fc2a81569c0 2024-09-10T23:39:09.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.292+0000 7fc2ada00640 1 --2- 172.21.15.94:0/3215314508 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc288072cd0 0x7fc288075190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:09.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.292+0000 7fc2ada00640 1 --2- 172.21.15.94:0/3215314508 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc288072cd0 0x7fc288075190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00abd0 tx=0x7fc29c011040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:09.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.296+0000 7fc2a5600640 1 -- 172.21.15.94:0/3215314508 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc2a81483e0 con 0x7fc2a81569c0 2024-09-10T23:39:09.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.484+0000 7fc2b0400640 1 -- 172.21.15.94:0/3215314508 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot ls", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7fc2a8157390 con 0x7fc288072cd0 2024-09-10T23:39:09.489 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:09.488+0000 7f9e9fe00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-10T23:39:09.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.488+0000 7fc2a5600640 1 -- 172.21.15.94:0/3215314508 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fc29c004020 con 0x7fc288072cd0 2024-09-10T23:39:09.490 INFO:teuthology.orchestra.run.smithi094.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-10T23:39:09.498 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T23:39:09.499 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:39:09.499 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:39:09.587 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:39:09.655 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:39:09.656 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:39:09.656 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:39:09.656 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:09.668 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:39:09.668 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:09.721 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:39:09.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.904+0000 7f8013000640 1 -- 172.21.15.94:0/2749552083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c071a90 msgr2=0x7f800c0d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:09.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.904+0000 7f8013000640 1 --2- 172.21.15.94:0/2749552083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c071a90 0x7f800c0d02a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009840 tx=0x7f7ffc01c7f0 comp rx=0 tx=0).stop 2024-09-10T23:39:09.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.904+0000 7f8013000640 1 -- 172.21.15.94:0/2749552083 shutdown_connections 2024-09-10T23:39:09.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.904+0000 7f8013000640 1 --2- 172.21.15.94:0/2749552083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c071a90 0x7f800c0d02a0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:09.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.904+0000 7f8013000640 1 -- 172.21.15.94:0/2749552083 >> 172.21.15.94:0/2749552083 conn(0x7f800c0d0c00 msgr2=0x7f800c0d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:09.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.904+0000 7f8013000640 1 -- 172.21.15.94:0/2749552083 shutdown_connections 2024-09-10T23:39:09.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.904+0000 7f8013000640 1 -- 172.21.15.94:0/2749552083 wait complete. 2024-09-10T23:39:09.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.904+0000 7f8013000640 1 Processor -- start 2024-09-10T23:39:09.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8013000640 1 -- start start 2024-09-10T23:39:09.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8013000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f800c071a90 0x7f800c071240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:09.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8013000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f800c0ccfb0 0x7f800c155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:09.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8013000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c162460 0x7f800c159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:09.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8013000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f800c16c270 con 0x7f800c162460 2024-09-10T23:39:09.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8013000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f800c16c0f0 con 0x7f800c0ccfb0 2024-09-10T23:39:09.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8013000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f800c16c3f0 con 0x7f800c071a90 2024-09-10T23:39:09.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8011000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c162460 0x7f800c159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:09.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8011000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c162460 0x7f800c159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32964/0 (socket says 172.21.15.94:32964) 2024-09-10T23:39:09.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8011000640 1 -- 172.21.15.94:0/1121839375 learned_addr learned my addr 172.21.15.94:0/1121839375 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:09.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8011000640 1 -- 172.21.15.94:0/1121839375 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f800c071a90 msgr2=0x7f800c071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:09.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f800be00640 1 --2- 172.21.15.94:0/1121839375 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f800c071a90 0x7f800c071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:09.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8003400640 1 --2- 172.21.15.94:0/1121839375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f800c0ccfb0 0x7f800c155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:09.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8011000640 1 --2- 172.21.15.94:0/1121839375 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f800c071a90 0x7f800c071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:09.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8011000640 1 -- 172.21.15.94:0/1121839375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f800c0ccfb0 msgr2=0x7f800c155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:09.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8011000640 1 --2- 172.21.15.94:0/1121839375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f800c0ccfb0 0x7f800c155780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:09.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8011000640 1 -- 172.21.15.94:0/1121839375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c1546e0 con 0x7f800c162460 2024-09-10T23:39:09.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8011000640 1 --2- 172.21.15.94:0/1121839375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c162460 0x7f800c159d90 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f800400c780 tx=0x7f800400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:09.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8008c00640 1 -- 172.21.15.94:0/1121839375 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004004b70 con 0x7f800c162460 2024-09-10T23:39:09.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8008c00640 1 -- 172.21.15.94:0/1121839375 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8004004e00 con 0x7f800c162460 2024-09-10T23:39:09.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8013000640 1 -- 172.21.15.94:0/1121839375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800c1549c0 con 0x7f800c162460 2024-09-10T23:39:09.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8008c00640 1 -- 172.21.15.94:0/1121839375 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800400aab0 con 0x7f800c162460 2024-09-10T23:39:09.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8013000640 1 -- 172.21.15.94:0/1121839375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800c154f80 con 0x7f800c162460 2024-09-10T23:39:09.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.908+0000 7f8013000640 1 -- 172.21.15.94:0/1121839375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f800c0d02a0 con 0x7f800c162460 2024-09-10T23:39:09.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.912+0000 7f8008c00640 1 -- 172.21.15.94:0/1121839375 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f8004023060 con 0x7f800c162460 2024-09-10T23:39:09.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.912+0000 7f8008c00640 1 --2- 172.21.15.94:0/1121839375 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7fe8072cd0 0x7f7fe8075190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:09.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.912+0000 7f8008c00640 1 -- 172.21.15.94:0/1121839375 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f80040940e0 con 0x7f800c162460 2024-09-10T23:39:09.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.912+0000 7f800be00640 1 --2- 172.21.15.94:0/1121839375 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7fe8072cd0 0x7f7fe8075190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:09.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.912+0000 7f800be00640 1 --2- 172.21.15.94:0/1121839375 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7fe8072cd0 0x7f7fe8075190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0af000 tx=0x7f7ffc01b030 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:09.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:09.916+0000 7f8008c00640 1 -- 172.21.15.94:0/1121839375 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f800405ee50 con 0x7f800c162460 2024-09-10T23:39:10.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.096+0000 7f8013000640 1 -- 172.21.15.94:0/1121839375 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f800c0b6b80 con 0x7f800c162460 2024-09-10T23:39:10.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.096+0000 7f8008c00640 1 -- 172.21.15.94:0/1121839375 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) ==== 80+0+0 (secure 0 0 0) 0x7f800405e750 con 0x7f800c162460 2024-09-10T23:39:10.098 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:39:10.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.096+0000 7f8013000640 1 -- 172.21.15.94:0/1121839375 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7fe8072cd0 msgr2=0x7f7fe8075190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:10.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.096+0000 7f8013000640 1 --2- 172.21.15.94:0/1121839375 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7fe8072cd0 0x7f7fe8075190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0af000 tx=0x7f7ffc01b030 comp rx=0 tx=0).stop 2024-09-10T23:39:10.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.096+0000 7f8013000640 1 -- 172.21.15.94:0/1121839375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c162460 msgr2=0x7f800c159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:10.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.096+0000 7f8013000640 1 --2- 172.21.15.94:0/1121839375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c162460 0x7f800c159d90 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f800400c780 tx=0x7f800400cc40 comp rx=0 tx=0).stop 2024-09-10T23:39:10.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.096+0000 7f8013000640 1 -- 172.21.15.94:0/1121839375 shutdown_connections 2024-09-10T23:39:10.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.096+0000 7f8013000640 1 --2- 172.21.15.94:0/1121839375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f800c0ccfb0 0x7f800c155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:10.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.096+0000 7f8013000640 1 --2- 172.21.15.94:0/1121839375 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7fe8072cd0 0x7f7fe8075190 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:10.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.096+0000 7f8013000640 1 --2- 172.21.15.94:0/1121839375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c162460 0x7f800c159d90 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:10.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.096+0000 7f8013000640 1 --2- 172.21.15.94:0/1121839375 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f800c071a90 0x7f800c071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:10.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.096+0000 7f8013000640 1 -- 172.21.15.94:0/1121839375 >> 172.21.15.94:0/1121839375 conn(0x7f800c0d0c00 msgr2=0x7f800c0b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:10.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.096+0000 7f8013000640 1 -- 172.21.15.94:0/1121839375 shutdown_connections 2024-09-10T23:39:10.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.096+0000 7f8013000640 1 -- 172.21.15.94:0/1121839375 wait complete. 2024-09-10T23:39:10.119 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-09-10T23:39:10.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:10.119 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T23:39:10.137 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[48680]: fuse finished with error 0 and tester_r 0 2024-09-10T23:39:10.139 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:39:10.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:10.139 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T23:39:10.240 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:39:10.240 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:39:10.240 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:10.250 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:39:10.250 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:10.301 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:39:10.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.440+0000 7f9594800640 1 -- 172.21.15.94:0/3800563184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c158790 msgr2=0x7f958c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:10.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.440+0000 7f9594800640 1 --2- 172.21.15.94:0/3800563184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c158790 0x7f958c06fc00 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f958c069640 tx=0x7f958001c8c0 comp rx=0 tx=0).stop 2024-09-10T23:39:10.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.440+0000 7f9594800640 1 -- 172.21.15.94:0/3800563184 shutdown_connections 2024-09-10T23:39:10.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.440+0000 7f9594800640 1 --2- 172.21.15.94:0/3800563184 >> v2:172.21.15.181:3300/0 conn(0x7f958c0c49b0 0x7f958c0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:10.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.440+0000 7f9594800640 1 --2- 172.21.15.94:0/3800563184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c158790 0x7f958c06fc00 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:10.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.440+0000 7f9594800640 1 -- 172.21.15.94:0/3800563184 >> 172.21.15.94:0/3800563184 conn(0x7f958c0cfaf0 msgr2=0x7f958c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:10.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.440+0000 7f9594800640 1 -- 172.21.15.94:0/3800563184 shutdown_connections 2024-09-10T23:39:10.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.440+0000 7f9594800640 1 -- 172.21.15.94:0/3800563184 wait complete. 2024-09-10T23:39:10.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.440+0000 7f9594800640 1 Processor -- start 2024-09-10T23:39:10.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.440+0000 7f9594800640 1 -- start start 2024-09-10T23:39:10.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9594800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f958c0c49b0 0x7f958c15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:10.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9594800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f958c0cb300 0x7f958c148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:10.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9594800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c158790 0x7f958c1f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:10.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9594800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f958c168210 con 0x7f958c158790 2024-09-10T23:39:10.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9594800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f958c168090 con 0x7f958c0cb300 2024-09-10T23:39:10.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9594800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f958c168390 con 0x7f958c0c49b0 2024-09-10T23:39:10.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9592800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c158790 0x7f958c1f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:10.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9592800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c158790 0x7f958c1f6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32976/0 (socket says 172.21.15.94:32976) 2024-09-10T23:39:10.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9592800640 1 -- 172.21.15.94:0/3988398560 learned_addr learned my addr 172.21.15.94:0/3988398560 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:10.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9591400640 1 --2- 172.21.15.94:0/3988398560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f958c0cb300 0x7f958c148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:10.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9591e00640 1 --2- 172.21.15.94:0/3988398560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f958c0c49b0 0x7f958c15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:10.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9592800640 1 -- 172.21.15.94:0/3988398560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f958c0c49b0 msgr2=0x7f958c15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:10.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9592800640 1 --2- 172.21.15.94:0/3988398560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f958c0c49b0 0x7f958c15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:10.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9592800640 1 -- 172.21.15.94:0/3988398560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f958c0cb300 msgr2=0x7f958c148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:10.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9592800640 1 --2- 172.21.15.94:0/3988398560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f958c0cb300 0x7f958c148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:10.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9592800640 1 -- 172.21.15.94:0/3988398560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c149460 con 0x7f958c158790 2024-09-10T23:39:10.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9592800640 1 --2- 172.21.15.94:0/3988398560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c158790 0x7f958c1f6710 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f958800c780 tx=0x7f958800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:10.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9586000640 1 -- 172.21.15.94:0/3988398560 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588004b70 con 0x7f958c158790 2024-09-10T23:39:10.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9586000640 1 -- 172.21.15.94:0/3988398560 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9588004e00 con 0x7f958c158790 2024-09-10T23:39:10.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9586000640 1 -- 172.21.15.94:0/3988398560 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958800aab0 con 0x7f958c158790 2024-09-10T23:39:10.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9594800640 1 -- 172.21.15.94:0/3988398560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958c1f9da0 con 0x7f958c158790 2024-09-10T23:39:10.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9594800640 1 -- 172.21.15.94:0/3988398560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f958c1f5f10 con 0x7f958c158790 2024-09-10T23:39:10.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.444+0000 7f9594800640 1 -- 172.21.15.94:0/3988398560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f958c0d0850 con 0x7f958c158790 2024-09-10T23:39:10.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.448+0000 7f9586000640 1 -- 172.21.15.94:0/3988398560 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f958800ac50 con 0x7f958c158790 2024-09-10T23:39:10.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.448+0000 7f9586000640 1 --2- 172.21.15.94:0/3988398560 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9570072ca0 0x7f9570075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:10.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.448+0000 7f9586000640 1 -- 172.21.15.94:0/3988398560 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f9588093f80 con 0x7f958c158790 2024-09-10T23:39:10.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.448+0000 7f9591e00640 1 --2- 172.21.15.94:0/3988398560 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9570072ca0 0x7f9570075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:10.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.448+0000 7f9591e00640 1 --2- 172.21.15.94:0/3988398560 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9570072ca0 0x7f9570075160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f95800af000 tx=0x7f95800a7040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:10.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.452+0000 7f9586000640 1 -- 172.21.15.94:0/3988398560 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9588020070 con 0x7f958c158790 2024-09-10T23:39:10.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.628+0000 7f9594800640 1 -- 172.21.15.94:0/3988398560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f958c0cada0 con 0x7f958c158790 2024-09-10T23:39:10.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.632+0000 7f9586000640 1 -- 172.21.15.94:0/3988398560 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) ==== 80+0+0 (secure 0 0 0) 0x7f958c149460 con 0x7f958c158790 2024-09-10T23:39:10.633 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:39:10.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.632+0000 7f9594800640 1 -- 172.21.15.94:0/3988398560 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9570072ca0 msgr2=0x7f9570075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:10.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.632+0000 7f9594800640 1 --2- 172.21.15.94:0/3988398560 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9570072ca0 0x7f9570075160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f95800af000 tx=0x7f95800a7040 comp rx=0 tx=0).stop 2024-09-10T23:39:10.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.632+0000 7f9594800640 1 -- 172.21.15.94:0/3988398560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c158790 msgr2=0x7f958c1f6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:10.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.632+0000 7f9594800640 1 --2- 172.21.15.94:0/3988398560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c158790 0x7f958c1f6710 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f958800c780 tx=0x7f958800cc40 comp rx=0 tx=0).stop 2024-09-10T23:39:10.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.632+0000 7f9594800640 1 -- 172.21.15.94:0/3988398560 shutdown_connections 2024-09-10T23:39:10.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.632+0000 7f9594800640 1 --2- 172.21.15.94:0/3988398560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f958c0cb300 0x7f958c148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:10.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.632+0000 7f9594800640 1 --2- 172.21.15.94:0/3988398560 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9570072ca0 0x7f9570075160 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:10.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.632+0000 7f9594800640 1 --2- 172.21.15.94:0/3988398560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f958c158790 0x7f958c1f6710 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:10.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.632+0000 7f9594800640 1 --2- 172.21.15.94:0/3988398560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f958c0c49b0 0x7f958c15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:10.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.632+0000 7f9594800640 1 -- 172.21.15.94:0/3988398560 >> 172.21.15.94:0/3988398560 conn(0x7f958c0cfaf0 msgr2=0x7f958c0b0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:10.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.632+0000 7f9594800640 1 -- 172.21.15.94:0/3988398560 shutdown_connections 2024-09-10T23:39:10.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:10.632+0000 7f9594800640 1 -- 172.21.15.94:0/3988398560 wait complete. 2024-09-10T23:39:10.654 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi181.front.sepia.ceph.com... 2024-09-10T23:39:10.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:10.655 DEBUG:teuthology.orchestra.run.smithi181:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T23:39:10.675 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[35866]: fuse finished with error 0 and tester_r 0 2024-09-10T23:39:10.677 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:39:10.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:10.678 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T23:39:10.836 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:39:10.836 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:39:10.836 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:10.844 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:39:10.845 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:39:10.845 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:39:10.845 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:10.885 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:39:10.885 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:39:11.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.028+0000 7fcfa6400640 1 -- 172.21.15.94:0/2967650692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfa00d05e0 msgr2=0x7fcfa00b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:11.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.028+0000 7fcfa6400640 1 --2- 172.21.15.94:0/2967650692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfa00d05e0 0x7fcfa00b15c0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0073b30 tx=0x7fcf9801c9a0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.028+0000 7fcfa6400640 1 -- 172.21.15.94:0/2967650692 shutdown_connections 2024-09-10T23:39:11.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.028+0000 7fcfa6400640 1 --2- 172.21.15.94:0/2967650692 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfa00d05e0 0x7fcfa00b15c0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.028+0000 7fcfa6400640 1 --2- 172.21.15.94:0/2967650692 >> v2:172.21.15.181:3300/0 conn(0x7fcfa00b2bb0 0x7fcfa00d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.028+0000 7fcfa6400640 1 --2- 172.21.15.94:0/2967650692 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfa015f5c0 0x7fcfa00cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.028+0000 7fcfa6400640 1 -- 172.21.15.94:0/2967650692 >> 172.21.15.94:0/2967650692 conn(0x7fcfa00d1c60 msgr2=0x7fcfa00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:11.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.028+0000 7fcfa6400640 1 -- 172.21.15.94:0/2967650692 shutdown_connections 2024-09-10T23:39:11.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.028+0000 7fcfa6400640 1 -- 172.21.15.94:0/2967650692 wait complete. 2024-09-10T23:39:11.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.028+0000 7fcfa6400640 1 Processor -- start 2024-09-10T23:39:11.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcfa6400640 1 -- start start 2024-09-10T23:39:11.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcfa6400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfa00b2bb0 0x7fcfa0156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:11.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcfa6400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa00d05e0 0x7fcfa0150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:11.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcfa6400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfa015f5c0 0x7fcfa0150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:11.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcfa6400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa00afc20 con 0x7fcfa015f5c0 2024-09-10T23:39:11.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcfa6400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa00afaa0 con 0x7fcfa00d05e0 2024-09-10T23:39:11.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcfa6400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa00afda0 con 0x7fcfa00b2bb0 2024-09-10T23:39:11.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcf9fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfa015f5c0 0x7fcfa0150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:11.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcf9fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfa015f5c0 0x7fcfa0150b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32980/0 (socket says 172.21.15.94:32980) 2024-09-10T23:39:11.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcf9fe00640 1 -- 172.21.15.94:0/2540152246 learned_addr learned my addr 172.21.15.94:0/2540152246 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:11.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcf9f400640 1 --2- 172.21.15.94:0/2540152246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfa00b2bb0 0x7fcfa0156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:11.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcf9ea00640 1 --2- 172.21.15.94:0/2540152246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa00d05e0 0x7fcfa0150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcf9fe00640 1 -- 172.21.15.94:0/2540152246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfa00b2bb0 msgr2=0x7fcfa0156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcf9fe00640 1 --2- 172.21.15.94:0/2540152246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfa00b2bb0 0x7fcfa0156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcf9fe00640 1 -- 172.21.15.94:0/2540152246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa00d05e0 msgr2=0x7fcfa0150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcf9fe00640 1 --2- 172.21.15.94:0/2540152246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa00d05e0 0x7fcfa0150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcf9fe00640 1 -- 172.21.15.94:0/2540152246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa01f5a50 con 0x7fcfa015f5c0 2024-09-10T23:39:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcf9ea00640 1 --2- 172.21.15.94:0/2540152246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa00d05e0 0x7fcfa0150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcf9fe00640 1 --2- 172.21.15.94:0/2540152246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfa015f5c0 0x7fcfa0150b60 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fcf9801b050 tx=0x7fcf98003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcf97400640 1 -- 172.21.15.94:0/2540152246 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf980bb070 con 0x7fcfa015f5c0 2024-09-10T23:39:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcf97400640 1 -- 172.21.15.94:0/2540152246 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf98004e00 con 0x7fcfa015f5c0 2024-09-10T23:39:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcf97400640 1 -- 172.21.15.94:0/2540152246 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf980b2040 con 0x7fcfa015f5c0 2024-09-10T23:39:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcfa6400640 1 -- 172.21.15.94:0/2540152246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa01f5cd0 con 0x7fcfa015f5c0 2024-09-10T23:39:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcfa6400640 1 -- 172.21.15.94:0/2540152246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa01f6290 con 0x7fcfa015f5c0 2024-09-10T23:39:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.032+0000 7fcfa6400640 1 -- 172.21.15.94:0/2540152246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa00b0330 con 0x7fcfa015f5c0 2024-09-10T23:39:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.036+0000 7fcf97400640 1 -- 172.21.15.94:0/2540152246 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7fcf980b5030 con 0x7fcfa015f5c0 2024-09-10T23:39:11.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.036+0000 7fcf97400640 1 --2- 172.21.15.94:0/2540152246 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcf80072ca0 0x7fcf80075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:11.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.036+0000 7fcf97400640 1 -- 172.21.15.94:0/2540152246 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7fcf9812d620 con 0x7fcfa015f5c0 2024-09-10T23:39:11.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.036+0000 7fcf9f400640 1 --2- 172.21.15.94:0/2540152246 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcf80072ca0 0x7fcf80075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:11.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.036+0000 7fcf9f400640 1 --2- 172.21.15.94:0/2540152246 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcf80072ca0 0x7fcf80075160 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcf900058d0 tx=0x7fcf90005860 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:11.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.040+0000 7fcf97400640 1 -- 172.21.15.94:0/2540152246 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcfa01f5a50 con 0x7fcfa015f5c0 2024-09-10T23:39:11.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.240+0000 7fcfa6400640 1 -- 172.21.15.94:0/2540152246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfa015a7f0 con 0x7fcfa015f5c0 2024-09-10T23:39:11.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.240+0000 7fcf97400640 1 -- 172.21.15.94:0/2540152246 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5234 (secure 0 0 0) 0x7fcf98016970 con 0x7fcfa015f5c0 2024-09-10T23:39:11.242 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:11.242 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":89,"btime":"2024-09-10T23:39:00:456652+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3851553379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3851553379},{"type":"v1","addr":"172.21.15.181:6835","nonce":3851553379}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/90080224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":90080224},{"type":"v1","addr":"172.21.15.94:6833","nonce":90080224}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":7929,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/334167733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":334167733},{"type":"v1","addr":"172.21.15.94:6835","nonce":334167733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:38:50.375690+0000","modified":"2024-09-10T23:38:53.400898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196562":{"gid":196562,"name":"d","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6801/2109170417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2109170417},{"type":"v1","addr":"172.21.15.181:6801","nonce":2109170417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196562,"qdb_cluster":[196562]},"id":10}]} 2024-09-10T23:39:11.242 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 89 2024-09-10T23:39:11.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.240+0000 7fcfa6400640 1 -- 172.21.15.94:0/2540152246 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcf80072ca0 msgr2=0x7fcf80075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:11.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.240+0000 7fcfa6400640 1 --2- 172.21.15.94:0/2540152246 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcf80072ca0 0x7fcf80075160 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcf900058d0 tx=0x7fcf90005860 comp rx=0 tx=0).stop 2024-09-10T23:39:11.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.240+0000 7fcfa6400640 1 -- 172.21.15.94:0/2540152246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfa015f5c0 msgr2=0x7fcfa0150b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:11.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.240+0000 7fcfa6400640 1 --2- 172.21.15.94:0/2540152246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfa015f5c0 0x7fcfa0150b60 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fcf9801b050 tx=0x7fcf98003980 comp rx=0 tx=0).stop 2024-09-10T23:39:11.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.240+0000 7fcfa6400640 1 -- 172.21.15.94:0/2540152246 shutdown_connections 2024-09-10T23:39:11.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.240+0000 7fcfa6400640 1 --2- 172.21.15.94:0/2540152246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfa00d05e0 0x7fcfa0150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.240+0000 7fcfa6400640 1 --2- 172.21.15.94:0/2540152246 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcf80072ca0 0x7fcf80075160 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.240+0000 7fcfa6400640 1 --2- 172.21.15.94:0/2540152246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfa015f5c0 0x7fcfa0150b60 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.240+0000 7fcfa6400640 1 --2- 172.21.15.94:0/2540152246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcfa00b2bb0 0x7fcfa0156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.240+0000 7fcfa6400640 1 -- 172.21.15.94:0/2540152246 >> 172.21.15.94:0/2540152246 conn(0x7fcfa00d1c60 msgr2=0x7fcfa015daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:11.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.240+0000 7fcfa6400640 1 -- 172.21.15.94:0/2540152246 shutdown_connections 2024-09-10T23:39:11.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.240+0000 7fcfa6400640 1 -- 172.21.15.94:0/2540152246 wait complete. 2024-09-10T23:39:11.263 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:39:11.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.400+0000 7fdaca800640 1 -- 172.21.15.94:0/2009883541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdac415f530 msgr2=0x7fdac415f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:11.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.400+0000 7fdaca800640 1 --2- 172.21.15.94:0/2009883541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdac415f530 0x7fdac415f9c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009bc0 tx=0x7fdabc01cb10 comp rx=0 tx=0).stop 2024-09-10T23:39:11.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.400+0000 7fdaca800640 1 -- 172.21.15.94:0/2009883541 shutdown_connections 2024-09-10T23:39:11.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.400+0000 7fdaca800640 1 --2- 172.21.15.94:0/2009883541 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdac415f530 0x7fdac415f9c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.400+0000 7fdaca800640 1 --2- 172.21.15.94:0/2009883541 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdac4068990 0x7fdac40ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.400+0000 7fdaca800640 1 -- 172.21.15.94:0/2009883541 >> 172.21.15.94:0/2009883541 conn(0x7fdac40d0120 msgr2=0x7fdac40d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:11.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.400+0000 7fdaca800640 1 -- 172.21.15.94:0/2009883541 shutdown_connections 2024-09-10T23:39:11.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.400+0000 7fdaca800640 1 -- 172.21.15.94:0/2009883541 wait complete. 2024-09-10T23:39:11.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdaca800640 1 Processor -- start 2024-09-10T23:39:11.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdaca800640 1 -- start start 2024-09-10T23:39:11.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdaca800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdac4068990 0x7fdac4156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:11.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdaca800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdac415c210 0x7fdac414ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:11.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdaca800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdac415f530 0x7fdac414d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:11.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdaca800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdac4160890 con 0x7fdac415c210 2024-09-10T23:39:11.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdaca800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdac4160710 con 0x7fdac4068990 2024-09-10T23:39:11.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdaca800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fdac4160a10 con 0x7fdac415f530 2024-09-10T23:39:11.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdac8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdac415c210 0x7fdac414ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:11.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdac8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdac415c210 0x7fdac414ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32994/0 (socket says 172.21.15.94:32994) 2024-09-10T23:39:11.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdac8a00640 1 -- 172.21.15.94:0/2133148702 learned_addr learned my addr 172.21.15.94:0/2133148702 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:11.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdac9e00640 1 --2- 172.21.15.94:0/2133148702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdac415f530 0x7fdac414d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:11.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdac9400640 1 --2- 172.21.15.94:0/2133148702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdac4068990 0x7fdac4156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:11.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdac8a00640 1 -- 172.21.15.94:0/2133148702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdac415f530 msgr2=0x7fdac414d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:11.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdac8a00640 1 --2- 172.21.15.94:0/2133148702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdac415f530 0x7fdac414d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdac8a00640 1 -- 172.21.15.94:0/2133148702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdac4068990 msgr2=0x7fdac4156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:11.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdac8a00640 1 --2- 172.21.15.94:0/2133148702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdac4068990 0x7fdac4156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdac8a00640 1 -- 172.21.15.94:0/2133148702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac41f7e00 con 0x7fdac415c210 2024-09-10T23:39:11.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdac8a00640 1 --2- 172.21.15.94:0/2133148702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdac415c210 0x7fdac414ccc0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fdab000ea20 tx=0x7fdab000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:11.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdac1600640 1 -- 172.21.15.94:0/2133148702 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab000cd80 con 0x7fdac415c210 2024-09-10T23:39:11.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdac1600640 1 -- 172.21.15.94:0/2133148702 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdab0010430 con 0x7fdac415c210 2024-09-10T23:39:11.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdaca800640 1 -- 172.21.15.94:0/2133148702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac41f80e0 con 0x7fdac415c210 2024-09-10T23:39:11.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdac1600640 1 -- 172.21.15.94:0/2133148702 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0010860 con 0x7fdac415c210 2024-09-10T23:39:11.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdaca800640 1 -- 172.21.15.94:0/2133148702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac41584a0 con 0x7fdac415c210 2024-09-10T23:39:11.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.404+0000 7fdaca800640 1 -- 172.21.15.94:0/2133148702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac40ccaa0 con 0x7fdac415c210 2024-09-10T23:39:11.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.408+0000 7fdac1600640 1 -- 172.21.15.94:0/2133148702 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7fdab0010a00 con 0x7fdac415c210 2024-09-10T23:39:11.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.408+0000 7fdac1600640 1 --2- 172.21.15.94:0/2133148702 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdaa4072ca0 0x7fdaa4075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:11.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.408+0000 7fdac1600640 1 -- 172.21.15.94:0/2133148702 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7fdab0094bd0 con 0x7fdac415c210 2024-09-10T23:39:11.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.408+0000 7fdac9400640 1 --2- 172.21.15.94:0/2133148702 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdaa4072ca0 0x7fdaa4075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:11.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.408+0000 7fdac9400640 1 --2- 172.21.15.94:0/2133148702 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdaa4072ca0 0x7fdaa4075160 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdac415f460 tx=0x7fdab8009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:11.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.412+0000 7fdac1600640 1 -- 172.21.15.94:0/2133148702 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdab005f720 con 0x7fdac415c210 2024-09-10T23:39:11.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.612+0000 7fdaca800640 1 -- 172.21.15.94:0/2133148702 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdac40ce820 con 0x7fdac415c210 2024-09-10T23:39:11.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.612+0000 7fdac1600640 1 -- 172.21.15.94:0/2133148702 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5234 (secure 0 0 0) 0x7fdab005f020 con 0x7fdac415c210 2024-09-10T23:39:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:11.616 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":89,"btime":"2024-09-10T23:39:00:456652+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3851553379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3851553379},{"type":"v1","addr":"172.21.15.181:6835","nonce":3851553379}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/90080224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":90080224},{"type":"v1","addr":"172.21.15.94:6833","nonce":90080224}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":7929,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/334167733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":334167733},{"type":"v1","addr":"172.21.15.94:6835","nonce":334167733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:38:50.375690+0000","modified":"2024-09-10T23:38:53.400898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196562":{"gid":196562,"name":"d","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6801/2109170417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2109170417},{"type":"v1","addr":"172.21.15.181:6801","nonce":2109170417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196562,"qdb_cluster":[196562]},"id":10}]} 2024-09-10T23:39:11.617 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 89 2024-09-10T23:39:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.616+0000 7fdaca800640 1 -- 172.21.15.94:0/2133148702 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdaa4072ca0 msgr2=0x7fdaa4075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:11.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.616+0000 7fdaca800640 1 --2- 172.21.15.94:0/2133148702 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdaa4072ca0 0x7fdaa4075160 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdac415f460 tx=0x7fdab8009040 comp rx=0 tx=0).stop 2024-09-10T23:39:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.616+0000 7fdaca800640 1 -- 172.21.15.94:0/2133148702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdac415c210 msgr2=0x7fdac414ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.616+0000 7fdaca800640 1 --2- 172.21.15.94:0/2133148702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdac415c210 0x7fdac414ccc0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fdab000ea20 tx=0x7fdab000eee0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.616+0000 7fdaca800640 1 -- 172.21.15.94:0/2133148702 shutdown_connections 2024-09-10T23:39:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.616+0000 7fdaca800640 1 --2- 172.21.15.94:0/2133148702 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdaa4072ca0 0x7fdaa4075160 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.616+0000 7fdaca800640 1 --2- 172.21.15.94:0/2133148702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdac415f530 0x7fdac414d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.616+0000 7fdaca800640 1 --2- 172.21.15.94:0/2133148702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdac415c210 0x7fdac414ccc0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.616+0000 7fdaca800640 1 --2- 172.21.15.94:0/2133148702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdac4068990 0x7fdac4156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.616+0000 7fdaca800640 1 -- 172.21.15.94:0/2133148702 >> 172.21.15.94:0/2133148702 conn(0x7fdac40d0120 msgr2=0x7fdac415c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.616+0000 7fdaca800640 1 -- 172.21.15.94:0/2133148702 shutdown_connections 2024-09-10T23:39:11.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.616+0000 7fdaca800640 1 -- 172.21.15.94:0/2133148702 wait complete. 2024-09-10T23:39:11.638 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:39:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.776+0000 7fdb2c200640 1 -- 172.21.15.94:0/504720169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb240b0320 msgr2=0x7fdb24166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.776+0000 7fdb2c200640 1 --2- 172.21.15.94:0/504720169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb240b0320 0x7fdb24166aa0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c009840 tx=0x7fdb1c01c700 comp rx=0 tx=0).stop 2024-09-10T23:39:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.776+0000 7fdb2c200640 1 -- 172.21.15.94:0/504720169 shutdown_connections 2024-09-10T23:39:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.776+0000 7fdb2c200640 1 --2- 172.21.15.94:0/504720169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb240b0320 0x7fdb24166aa0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.776+0000 7fdb2c200640 1 -- 172.21.15.94:0/504720169 >> 172.21.15.94:0/504720169 conn(0x7fdb240d1c60 msgr2=0x7fdb240d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.776+0000 7fdb2c200640 1 -- 172.21.15.94:0/504720169 shutdown_connections 2024-09-10T23:39:11.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.776+0000 7fdb2c200640 1 -- 172.21.15.94:0/504720169 wait complete. 2024-09-10T23:39:11.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.776+0000 7fdb2c200640 1 Processor -- start 2024-09-10T23:39:11.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb2c200640 1 -- start start 2024-09-10T23:39:11.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb2c200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb240b0320 0x7fdb24146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:11.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb2c200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb240b2bb0 0x7fdb24152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:11.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb2c200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb2415f5c0 0x7fdb2414e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:11.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb2c200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24169eb0 con 0x7fdb240b0320 2024-09-10T23:39:11.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb2c200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24169d30 con 0x7fdb240b2bb0 2024-09-10T23:39:11.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb2c200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fdb2416a030 con 0x7fdb2415f5c0 2024-09-10T23:39:11.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb29800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb240b0320 0x7fdb24146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:11.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb29800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb240b0320 0x7fdb24146450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33024/0 (socket says 172.21.15.94:33024) 2024-09-10T23:39:11.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb29800640 1 -- 172.21.15.94:0/2919554276 learned_addr learned my addr 172.21.15.94:0/2919554276 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:11.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb2a200640 1 --2- 172.21.15.94:0/2919554276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb2415f5c0 0x7fdb2414e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:11.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb28e00640 1 --2- 172.21.15.94:0/2919554276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb240b2bb0 0x7fdb24152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:11.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb29800640 1 -- 172.21.15.94:0/2919554276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb2415f5c0 msgr2=0x7fdb2414e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:11.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb29800640 1 --2- 172.21.15.94:0/2919554276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb2415f5c0 0x7fdb2414e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb29800640 1 -- 172.21.15.94:0/2919554276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb240b2bb0 msgr2=0x7fdb24152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb29800640 1 --2- 172.21.15.94:0/2919554276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb240b2bb0 0x7fdb24152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb29800640 1 -- 172.21.15.94:0/2919554276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb24151180 con 0x7fdb240b0320 2024-09-10T23:39:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb29800640 1 --2- 172.21.15.94:0/2919554276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb240b0320 0x7fdb24146450 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800cbc0 tx=0x7fdb1800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb21600640 1 -- 172.21.15.94:0/2919554276 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb18016020 con 0x7fdb240b0320 2024-09-10T23:39:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb21600640 1 -- 172.21.15.94:0/2919554276 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb1800ab40 con 0x7fdb240b0320 2024-09-10T23:39:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb21600640 1 -- 172.21.15.94:0/2919554276 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1800ce40 con 0x7fdb240b0320 2024-09-10T23:39:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb2c200640 1 -- 172.21.15.94:0/2919554276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb24151460 con 0x7fdb240b0320 2024-09-10T23:39:11.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb2c200640 1 -- 172.21.15.94:0/2919554276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb24151a20 con 0x7fdb240b0320 2024-09-10T23:39:11.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.780+0000 7fdb2c200640 1 -- 172.21.15.94:0/2919554276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb240cfb60 con 0x7fdb240b0320 2024-09-10T23:39:11.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.784+0000 7fdb21600640 1 -- 172.21.15.94:0/2919554276 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7fdb1801a3b0 con 0x7fdb240b0320 2024-09-10T23:39:11.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.784+0000 7fdb21600640 1 --2- 172.21.15.94:0/2919554276 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdb04072ca0 0x7fdb04075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:11.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.784+0000 7fdb21600640 1 -- 172.21.15.94:0/2919554276 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7fdb18094330 con 0x7fdb240b0320 2024-09-10T23:39:11.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.784+0000 7fdb28e00640 1 --2- 172.21.15.94:0/2919554276 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdb04072ca0 0x7fdb04075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:11.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.784+0000 7fdb28e00640 1 --2- 172.21.15.94:0/2919554276 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdb04072ca0 0x7fdb04075160 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdb2415a3e0 tx=0x7fdb10009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:11.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.788+0000 7fdb21600640 1 -- 172.21.15.94:0/2919554276 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdb18020030 con 0x7fdb240b0320 2024-09-10T23:39:11.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.952+0000 7fdb2c200640 1 -- 172.21.15.94:0/2919554276 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb240749c0 con 0x7fdb240b0320 2024-09-10T23:39:11.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.956+0000 7fdb21600640 1 -- 172.21.15.94:0/2919554276 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+16771 (secure 0 0 0) 0x7fdb1805ee80 con 0x7fdb240b0320 2024-09-10T23:39:11.957 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:11.958 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":127,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:38:51.988498+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:38:48.069795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T23:38:49.345087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:39:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.956+0000 7fdb2c200640 1 -- 172.21.15.94:0/2919554276 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdb04072ca0 msgr2=0x7fdb04075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.956+0000 7fdb2c200640 1 --2- 172.21.15.94:0/2919554276 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdb04072ca0 0x7fdb04075160 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdb2415a3e0 tx=0x7fdb10009040 comp rx=0 tx=0).stop 2024-09-10T23:39:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.956+0000 7fdb2c200640 1 -- 172.21.15.94:0/2919554276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb240b0320 msgr2=0x7fdb24146450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.956+0000 7fdb2c200640 1 --2- 172.21.15.94:0/2919554276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb240b0320 0x7fdb24146450 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800cbc0 tx=0x7fdb1800cbf0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.956+0000 7fdb2c200640 1 -- 172.21.15.94:0/2919554276 shutdown_connections 2024-09-10T23:39:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.956+0000 7fdb2c200640 1 --2- 172.21.15.94:0/2919554276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb240b2bb0 0x7fdb24152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.956+0000 7fdb2c200640 1 --2- 172.21.15.94:0/2919554276 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdb04072ca0 0x7fdb04075160 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.956+0000 7fdb2c200640 1 --2- 172.21.15.94:0/2919554276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb2415f5c0 0x7fdb2414e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.956+0000 7fdb2c200640 1 --2- 172.21.15.94:0/2919554276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb240b0320 0x7fdb24146450 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.956+0000 7fdb2c200640 1 -- 172.21.15.94:0/2919554276 >> 172.21.15.94:0/2919554276 conn(0x7fdb240d1c60 msgr2=0x7fdb2415ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.956+0000 7fdb2c200640 1 -- 172.21.15.94:0/2919554276 shutdown_connections 2024-09-10T23:39:11.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:11.956+0000 7fdb2c200640 1 -- 172.21.15.94:0/2919554276 wait complete. 2024-09-10T23:39:11.981 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T23:39:11.981 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:39:12.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.120+0000 7f6f6fa00640 1 -- 172.21.15.94:0/2231582384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6815a060 msgr2=0x7f6f6806e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:12.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.120+0000 7f6f6fa00640 1 --2- 172.21.15.94:0/2231582384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6815a060 0x7f6f6806e5c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009900 tx=0x7f6f5801c820 comp rx=0 tx=0).stop 2024-09-10T23:39:12.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.120+0000 7f6f6fa00640 1 -- 172.21.15.94:0/2231582384 shutdown_connections 2024-09-10T23:39:12.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.120+0000 7f6f6fa00640 1 --2- 172.21.15.94:0/2231582384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6815a060 0x7f6f6806e5c0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.120+0000 7f6f6fa00640 1 -- 172.21.15.94:0/2231582384 >> 172.21.15.94:0/2231582384 conn(0x7f6f6806cdf0 msgr2=0x7f6f6806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:12.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.120+0000 7f6f6fa00640 1 -- 172.21.15.94:0/2231582384 shutdown_connections 2024-09-10T23:39:12.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.120+0000 7f6f6fa00640 1 -- 172.21.15.94:0/2231582384 wait complete. 2024-09-10T23:39:12.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.120+0000 7f6f6fa00640 1 Processor -- start 2024-09-10T23:39:12.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.120+0000 7f6f6fa00640 1 -- start start 2024-09-10T23:39:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f6fa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f6815a060 0x7f6f681507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f6fa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6815e3a0 0x7f6f6813c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f6fa00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f6815e8a0 0x7f6f68163ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:12.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f6fa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6f680d00e0 con 0x7f6f6815e3a0 2024-09-10T23:39:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f6fa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6f680cff60 con 0x7f6f6815a060 2024-09-10T23:39:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f6fa00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6f680d0260 con 0x7f6f6815e8a0 2024-09-10T23:39:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f5fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6815e3a0 0x7f6f6813c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f5fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6815e3a0 0x7f6f6813c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33034/0 (socket says 172.21.15.94:33034) 2024-09-10T23:39:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f5fe00640 1 -- 172.21.15.94:0/2051752745 learned_addr learned my addr 172.21.15.94:0/2051752745 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f6d000640 1 --2- 172.21.15.94:0/2051752745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f6815a060 0x7f6f681507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f6da00640 1 --2- 172.21.15.94:0/2051752745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f6815e8a0 0x7f6f68163ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f5fe00640 1 -- 172.21.15.94:0/2051752745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f6815e8a0 msgr2=0x7f6f68163ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f5fe00640 1 --2- 172.21.15.94:0/2051752745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f6815e8a0 0x7f6f68163ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f5fe00640 1 -- 172.21.15.94:0/2051752745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f6815a060 msgr2=0x7f6f681507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f5fe00640 1 --2- 172.21.15.94:0/2051752745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f6815a060 0x7f6f681507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f5fe00640 1 -- 172.21.15.94:0/2051752745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f68150540 con 0x7f6f6815e3a0 2024-09-10T23:39:12.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f5fe00640 1 --2- 172.21.15.94:0/2051752745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6815e3a0 0x7f6f6813c330 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f6f50001ec0 tx=0x7f6f5000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f65600640 1 -- 172.21.15.94:0/2051752745 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f50017070 con 0x7f6f6815e3a0 2024-09-10T23:39:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f65600640 1 -- 172.21.15.94:0/2051752745 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f500053c0 con 0x7f6f6815e3a0 2024-09-10T23:39:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f6fa00640 1 -- 172.21.15.94:0/2051752745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f68163300 con 0x7f6f6815e3a0 2024-09-10T23:39:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f65600640 1 -- 172.21.15.94:0/2051752745 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f50003860 con 0x7f6f6815e3a0 2024-09-10T23:39:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f6fa00640 1 -- 172.21.15.94:0/2051752745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f68150260 con 0x7f6f6815e3a0 2024-09-10T23:39:12.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f6fa00640 1 -- 172.21.15.94:0/2051752745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f6806e5c0 con 0x7f6f6815e3a0 2024-09-10T23:39:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.124+0000 7f6f65600640 1 -- 172.21.15.94:0/2051752745 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f6f50003a00 con 0x7f6f6815e3a0 2024-09-10T23:39:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.128+0000 7f6f65600640 1 --2- 172.21.15.94:0/2051752745 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6f44072ca0 0x7f6f44075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.128+0000 7f6f65600640 1 -- 172.21.15.94:0/2051752745 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f6f500936c0 con 0x7f6f6815e3a0 2024-09-10T23:39:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.128+0000 7f6f6d000640 1 --2- 172.21.15.94:0/2051752745 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6f44072ca0 0x7f6f44075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:12.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.128+0000 7f6f6d000640 1 --2- 172.21.15.94:0/2051752745 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6f44072ca0 0x7f6f44075160 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6f580af000 tx=0x7f6f5801b060 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:12.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.132+0000 7f6f65600640 1 -- 172.21.15.94:0/2051752745 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6f5005e2a0 con 0x7f6f6815e3a0 2024-09-10T23:39:12.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.328+0000 7f6f6fa00640 1 -- 172.21.15.94:0/2051752745 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f68157140 con 0x7f6f6815e3a0 2024-09-10T23:39:12.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.332+0000 7f6f65600640 1 -- 172.21.15.94:0/2051752745 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5234 (secure 0 0 0) 0x7f6f5005dba0 con 0x7f6f6815e3a0 2024-09-10T23:39:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:12.334 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":89,"btime":"2024-09-10T23:39:00:456652+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3851553379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3851553379},{"type":"v1","addr":"172.21.15.181:6835","nonce":3851553379}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/90080224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":90080224},{"type":"v1","addr":"172.21.15.94:6833","nonce":90080224}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":7929,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/334167733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":334167733},{"type":"v1","addr":"172.21.15.94:6835","nonce":334167733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:38:50.375690+0000","modified":"2024-09-10T23:38:53.400898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196562":{"gid":196562,"name":"d","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6801/2109170417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2109170417},{"type":"v1","addr":"172.21.15.181:6801","nonce":2109170417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196562,"qdb_cluster":[196562]},"id":10}]} 2024-09-10T23:39:12.334 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 89 2024-09-10T23:39:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.332+0000 7f6f6fa00640 1 -- 172.21.15.94:0/2051752745 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6f44072ca0 msgr2=0x7f6f44075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.332+0000 7f6f6fa00640 1 --2- 172.21.15.94:0/2051752745 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6f44072ca0 0x7f6f44075160 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6f580af000 tx=0x7f6f5801b060 comp rx=0 tx=0).stop 2024-09-10T23:39:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.332+0000 7f6f6fa00640 1 -- 172.21.15.94:0/2051752745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6815e3a0 msgr2=0x7f6f6813c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.332+0000 7f6f6fa00640 1 --2- 172.21.15.94:0/2051752745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6815e3a0 0x7f6f6813c330 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f6f50001ec0 tx=0x7f6f5000c890 comp rx=0 tx=0).stop 2024-09-10T23:39:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.332+0000 7f6f6fa00640 1 -- 172.21.15.94:0/2051752745 shutdown_connections 2024-09-10T23:39:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.332+0000 7f6f6fa00640 1 --2- 172.21.15.94:0/2051752745 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6f44072ca0 0x7f6f44075160 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.332+0000 7f6f6fa00640 1 --2- 172.21.15.94:0/2051752745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f6815e8a0 0x7f6f68163ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.332+0000 7f6f6fa00640 1 --2- 172.21.15.94:0/2051752745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6815e3a0 0x7f6f6813c330 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.332+0000 7f6f6fa00640 1 --2- 172.21.15.94:0/2051752745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f6815a060 0x7f6f681507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.332+0000 7f6f6fa00640 1 -- 172.21.15.94:0/2051752745 >> 172.21.15.94:0/2051752745 conn(0x7f6f6806cdf0 msgr2=0x7f6f6806eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.332+0000 7f6f6fa00640 1 -- 172.21.15.94:0/2051752745 shutdown_connections 2024-09-10T23:39:12.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.332+0000 7f6f6fa00640 1 -- 172.21.15.94:0/2051752745 wait complete. 2024-09-10T23:39:12.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:39:12.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 -- 172.21.15.94:0/4152634908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c6415bd30 msgr2=0x7f5c6415c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:12.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 --2- 172.21.15.94:0/4152634908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c6415bd30 0x7f5c6415c110 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f5c6409fc80 tx=0x7f5c5401c830 comp rx=0 tx=0).stop 2024-09-10T23:39:12.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 -- 172.21.15.94:0/4152634908 shutdown_connections 2024-09-10T23:39:12.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 --2- 172.21.15.94:0/4152634908 >> v2:172.21.15.181:3300/0 conn(0x7f5c640b2ec0 0x7f5c640d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 --2- 172.21.15.94:0/4152634908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c640ae980 0x7f5c640d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 --2- 172.21.15.94:0/4152634908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c6415bd30 0x7f5c6415c110 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 -- 172.21.15.94:0/4152634908 >> 172.21.15.94:0/4152634908 conn(0x7f5c6409aa70 msgr2=0x7f5c6409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:12.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 -- 172.21.15.94:0/4152634908 shutdown_connections 2024-09-10T23:39:12.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 -- 172.21.15.94:0/4152634908 wait complete. 2024-09-10T23:39:12.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 Processor -- start 2024-09-10T23:39:12.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 -- start start 2024-09-10T23:39:12.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c640ae980 0x7f5c64151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:12.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c640b2ec0 0x7f5c641514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:12.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c6415bd30 0x7f5c640aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:12.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5c640b1ff0 con 0x7f5c640b2ec0 2024-09-10T23:39:12.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5c640b1e70 con 0x7f5c6415bd30 2024-09-10T23:39:12.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c6ba00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5c640b2170 con 0x7f5c640ae980 2024-09-10T23:39:12.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c63e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c640b2ec0 0x7f5c641514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:12.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c63e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c640b2ec0 0x7f5c641514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33058/0 (socket says 172.21.15.94:33058) 2024-09-10T23:39:12.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c63e00640 1 -- 172.21.15.94:0/609031380 learned_addr learned my addr 172.21.15.94:0/609031380 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:12.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.496+0000 7f5c69000640 1 --2- 172.21.15.94:0/609031380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c640ae980 0x7f5c64151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:12.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.500+0000 7f5c69a00640 1 --2- 172.21.15.94:0/609031380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c6415bd30 0x7f5c640aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:12.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.500+0000 7f5c63e00640 1 -- 172.21.15.94:0/609031380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c640ae980 msgr2=0x7f5c64151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:12.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.500+0000 7f5c63e00640 1 --2- 172.21.15.94:0/609031380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c640ae980 0x7f5c64151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.500+0000 7f5c63e00640 1 -- 172.21.15.94:0/609031380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c6415bd30 msgr2=0x7f5c640aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:12.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.500+0000 7f5c63e00640 1 --2- 172.21.15.94:0/609031380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c6415bd30 0x7f5c640aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.500+0000 7f5c63e00640 1 -- 172.21.15.94:0/609031380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c64151a30 con 0x7f5c640b2ec0 2024-09-10T23:39:12.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.500+0000 7f5c63e00640 1 --2- 172.21.15.94:0/609031380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c640b2ec0 0x7f5c641514f0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000d770 tx=0x7f5c5000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:12.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.500+0000 7f5c60c00640 1 -- 172.21.15.94:0/609031380 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50017070 con 0x7f5c640b2ec0 2024-09-10T23:39:12.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.500+0000 7f5c60c00640 1 -- 172.21.15.94:0/609031380 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c50009d20 con 0x7f5c640b2ec0 2024-09-10T23:39:12.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.500+0000 7f5c6ba00640 1 -- 172.21.15.94:0/609031380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c6406de60 con 0x7f5c640b2ec0 2024-09-10T23:39:12.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.500+0000 7f5c60c00640 1 -- 172.21.15.94:0/609031380 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c500106c0 con 0x7f5c640b2ec0 2024-09-10T23:39:12.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.500+0000 7f5c6ba00640 1 -- 172.21.15.94:0/609031380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c64157290 con 0x7f5c640b2ec0 2024-09-10T23:39:12.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.500+0000 7f5c6ba00640 1 -- 172.21.15.94:0/609031380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c6415c110 con 0x7f5c640b2ec0 2024-09-10T23:39:12.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.500+0000 7f5c60c00640 1 -- 172.21.15.94:0/609031380 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f5c50010860 con 0x7f5c640b2ec0 2024-09-10T23:39:12.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.500+0000 7f5c60c00640 1 --2- 172.21.15.94:0/609031380 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5c44072ca0 0x7f5c44075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:12.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.504+0000 7f5c60c00640 1 -- 172.21.15.94:0/609031380 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7f5c50094410 con 0x7f5c640b2ec0 2024-09-10T23:39:12.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.504+0000 7f5c69000640 1 --2- 172.21.15.94:0/609031380 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5c44072ca0 0x7f5c44075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:12.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.504+0000 7f5c69000640 1 --2- 172.21.15.94:0/609031380 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5c44072ca0 0x7f5c44075160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5c540af000 tx=0x7f5c540a7040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:12.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.504+0000 7f5c60c00640 1 -- 172.21.15.94:0/609031380 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5c5005e9a0 con 0x7f5c640b2ec0 2024-09-10T23:39:12.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.708+0000 7f5c6ba00640 1 -- 172.21.15.94:0/609031380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c640d0710 con 0x7f5c640b2ec0 2024-09-10T23:39:12.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.708+0000 7f5c60c00640 1 -- 172.21.15.94:0/609031380 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5234 (secure 0 0 0) 0x7f5c500633d0 con 0x7f5c640b2ec0 2024-09-10T23:39:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:12.711 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":89,"btime":"2024-09-10T23:39:00:456652+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3851553379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3851553379},{"type":"v1","addr":"172.21.15.181:6835","nonce":3851553379}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/90080224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":90080224},{"type":"v1","addr":"172.21.15.94:6833","nonce":90080224}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":7929,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/334167733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":334167733},{"type":"v1","addr":"172.21.15.94:6835","nonce":334167733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:38:50.375690+0000","modified":"2024-09-10T23:38:53.400898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196562":{"gid":196562,"name":"d","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6801/2109170417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2109170417},{"type":"v1","addr":"172.21.15.181:6801","nonce":2109170417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196562,"qdb_cluster":[196562]},"id":10}]} 2024-09-10T23:39:12.711 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 89 2024-09-10T23:39:12.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.708+0000 7f5c6ba00640 1 -- 172.21.15.94:0/609031380 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5c44072ca0 msgr2=0x7f5c44075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:12.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.708+0000 7f5c6ba00640 1 --2- 172.21.15.94:0/609031380 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5c44072ca0 0x7f5c44075160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5c540af000 tx=0x7f5c540a7040 comp rx=0 tx=0).stop 2024-09-10T23:39:12.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.708+0000 7f5c6ba00640 1 -- 172.21.15.94:0/609031380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c640b2ec0 msgr2=0x7f5c641514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:12.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.708+0000 7f5c6ba00640 1 --2- 172.21.15.94:0/609031380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c640b2ec0 0x7f5c641514f0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000d770 tx=0x7f5c5000dc30 comp rx=0 tx=0).stop 2024-09-10T23:39:12.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.708+0000 7f5c6ba00640 1 -- 172.21.15.94:0/609031380 shutdown_connections 2024-09-10T23:39:12.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.708+0000 7f5c6ba00640 1 --2- 172.21.15.94:0/609031380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c6415bd30 0x7f5c640aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.708+0000 7f5c6ba00640 1 --2- 172.21.15.94:0/609031380 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5c44072ca0 0x7f5c44075160 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.708+0000 7f5c6ba00640 1 --2- 172.21.15.94:0/609031380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c640b2ec0 0x7f5c641514f0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.708+0000 7f5c6ba00640 1 --2- 172.21.15.94:0/609031380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c640ae980 0x7f5c64151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.708+0000 7f5c6ba00640 1 -- 172.21.15.94:0/609031380 >> 172.21.15.94:0/609031380 conn(0x7f5c6409aa70 msgr2=0x7f5c64070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:12.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.708+0000 7f5c6ba00640 1 -- 172.21.15.94:0/609031380 shutdown_connections 2024-09-10T23:39:12.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.708+0000 7f5c6ba00640 1 -- 172.21.15.94:0/609031380 wait complete. 2024-09-10T23:39:12.733 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:39:12.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.872+0000 7fcb5a400640 1 -- 172.21.15.94:0/1944149037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb540d2290 msgr2=0x7fcb540b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:12.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.872+0000 7fcb5a400640 1 --2- 172.21.15.94:0/1944149037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb540d2290 0x7fcb540b1e70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009c80 tx=0x7fcb4c01cda0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.872+0000 7fcb5a400640 1 -- 172.21.15.94:0/1944149037 shutdown_connections 2024-09-10T23:39:12.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.872+0000 7fcb5a400640 1 --2- 172.21.15.94:0/1944149037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb540d2290 0x7fcb540b1e70 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.872+0000 7fcb5a400640 1 --2- 172.21.15.94:0/1944149037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb5415bd30 0x7fcb5415c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.872+0000 7fcb5a400640 1 -- 172.21.15.94:0/1944149037 >> 172.21.15.94:0/1944149037 conn(0x7fcb5409aa70 msgr2=0x7fcb5409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:12.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.872+0000 7fcb5a400640 1 -- 172.21.15.94:0/1944149037 shutdown_connections 2024-09-10T23:39:12.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.872+0000 7fcb5a400640 1 -- 172.21.15.94:0/1944149037 wait complete. 2024-09-10T23:39:12.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb5a400640 1 Processor -- start 2024-09-10T23:39:12.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb5a400640 1 -- start start 2024-09-10T23:39:12.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb5a400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb540ae980 0x7fcb54159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:12.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb5a400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb540d2290 0x7fcb54164470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:12.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb5a400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5415bd30 0x7fcb540caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:12.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb5a400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcb541694d0 con 0x7fcb5415bd30 2024-09-10T23:39:12.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb5a400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcb54169350 con 0x7fcb540d2290 2024-09-10T23:39:12.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb5a400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fcb54169650 con 0x7fcb540ae980 2024-09-10T23:39:12.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb53e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5415bd30 0x7fcb540caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:12.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb53e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5415bd30 0x7fcb540caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33062/0 (socket says 172.21.15.94:33062) 2024-09-10T23:39:12.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb53e00640 1 -- 172.21.15.94:0/3644495957 learned_addr learned my addr 172.21.15.94:0/3644495957 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:12.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb53400640 1 --2- 172.21.15.94:0/3644495957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb540ae980 0x7fcb54159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:12.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb4aa00640 1 --2- 172.21.15.94:0/3644495957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb540d2290 0x7fcb54164470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:12.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb53e00640 1 -- 172.21.15.94:0/3644495957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb540ae980 msgr2=0x7fcb54159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:12.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb53e00640 1 --2- 172.21.15.94:0/3644495957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb540ae980 0x7fcb54159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb53e00640 1 -- 172.21.15.94:0/3644495957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb540d2290 msgr2=0x7fcb54164470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:12.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb53e00640 1 --2- 172.21.15.94:0/3644495957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb540d2290 0x7fcb54164470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:12.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb53e00640 1 -- 172.21.15.94:0/3644495957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb540b71e0 con 0x7fcb5415bd30 2024-09-10T23:39:12.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb53e00640 1 --2- 172.21.15.94:0/3644495957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5415bd30 0x7fcb540caf10 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009bd0 tx=0x7fcb4c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:12.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb4be00640 1 -- 172.21.15.94:0/3644495957 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c0bb070 con 0x7fcb5415bd30 2024-09-10T23:39:12.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb4be00640 1 -- 172.21.15.94:0/3644495957 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb4c004b40 con 0x7fcb5415bd30 2024-09-10T23:39:12.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb4be00640 1 -- 172.21.15.94:0/3644495957 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c0b2040 con 0x7fcb5415bd30 2024-09-10T23:39:12.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb5a400640 1 -- 172.21.15.94:0/3644495957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb540b74c0 con 0x7fcb5415bd30 2024-09-10T23:39:12.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.876+0000 7fcb5a400640 1 -- 172.21.15.94:0/3644495957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb54163f50 con 0x7fcb5415bd30 2024-09-10T23:39:12.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.880+0000 7fcb5a400640 1 -- 172.21.15.94:0/3644495957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb540b1e70 con 0x7fcb5415bd30 2024-09-10T23:39:12.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.880+0000 7fcb4be00640 1 -- 172.21.15.94:0/3644495957 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7fcb4c0b5030 con 0x7fcb5415bd30 2024-09-10T23:39:12.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.880+0000 7fcb4be00640 1 --2- 172.21.15.94:0/3644495957 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb30072ca0 0x7fcb30075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:12.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.880+0000 7fcb4be00640 1 -- 172.21.15.94:0/3644495957 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7fcb4c12d980 con 0x7fcb5415bd30 2024-09-10T23:39:12.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.880+0000 7fcb53400640 1 --2- 172.21.15.94:0/3644495957 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb30072ca0 0x7fcb30075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:12.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.880+0000 7fcb53400640 1 --2- 172.21.15.94:0/3644495957 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb30072ca0 0x7fcb30075160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009b40 tx=0x7fcb440091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:12.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:12.884+0000 7fcb4be00640 1 -- 172.21.15.94:0/3644495957 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcb4c016990 con 0x7fcb5415bd30 2024-09-10T23:39:13.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.052+0000 7fcb5a400640 1 -- 172.21.15.94:0/3644495957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb540d28a0 con 0x7fcb5415bd30 2024-09-10T23:39:13.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.052+0000 7fcb4be00640 1 -- 172.21.15.94:0/3644495957 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+16771 (secure 0 0 0) 0x7fcb4c0f8010 con 0x7fcb5415bd30 2024-09-10T23:39:13.055 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:13.056 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":127,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:38:51.988498+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:38:48.069795+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T23:38:49.345087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:39:13.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.056+0000 7fcb5a400640 1 -- 172.21.15.94:0/3644495957 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb30072ca0 msgr2=0x7fcb30075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:13.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.056+0000 7fcb5a400640 1 --2- 172.21.15.94:0/3644495957 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb30072ca0 0x7fcb30075160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009b40 tx=0x7fcb440091e0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.056+0000 7fcb5a400640 1 -- 172.21.15.94:0/3644495957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5415bd30 msgr2=0x7fcb540caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:13.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.056+0000 7fcb5a400640 1 --2- 172.21.15.94:0/3644495957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5415bd30 0x7fcb540caf10 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009bd0 tx=0x7fcb4c005ae0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.056+0000 7fcb5a400640 1 -- 172.21.15.94:0/3644495957 shutdown_connections 2024-09-10T23:39:13.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.056+0000 7fcb5a400640 1 --2- 172.21.15.94:0/3644495957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb540d2290 0x7fcb54164470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.056+0000 7fcb5a400640 1 --2- 172.21.15.94:0/3644495957 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb30072ca0 0x7fcb30075160 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.056+0000 7fcb5a400640 1 --2- 172.21.15.94:0/3644495957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5415bd30 0x7fcb540caf10 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.056+0000 7fcb5a400640 1 --2- 172.21.15.94:0/3644495957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb540ae980 0x7fcb54159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.056+0000 7fcb5a400640 1 -- 172.21.15.94:0/3644495957 >> 172.21.15.94:0/3644495957 conn(0x7fcb5409aa70 msgr2=0x7fcb5415b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:13.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.056+0000 7fcb5a400640 1 -- 172.21.15.94:0/3644495957 shutdown_connections 2024-09-10T23:39:13.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.056+0000 7fcb5a400640 1 -- 172.21.15.94:0/3644495957 wait complete. 2024-09-10T23:39:13.077 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T23:39:13.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.216+0000 7fb1bb200640 1 -- 172.21.15.94:0/502451250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1b40ce1e0 msgr2=0x7fb1b40cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:13.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.216+0000 7fb1bb200640 1 --2- 172.21.15.94:0/502451250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1b40ce1e0 0x7fb1b40cec70 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009900 tx=0x7fb1a001c820 comp rx=0 tx=0).stop 2024-09-10T23:39:13.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.216+0000 7fb1bb200640 1 -- 172.21.15.94:0/502451250 shutdown_connections 2024-09-10T23:39:13.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.216+0000 7fb1bb200640 1 --2- 172.21.15.94:0/502451250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1b40ce1e0 0x7fb1b40cec70 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.216+0000 7fb1bb200640 1 --2- 172.21.15.94:0/502451250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1b40ce890 0x7fb1b40cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.216+0000 7fb1bb200640 1 -- 172.21.15.94:0/502451250 >> 172.21.15.94:0/502451250 conn(0x7fb1b40d1c60 msgr2=0x7fb1b40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:13.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.216+0000 7fb1bb200640 1 -- 172.21.15.94:0/502451250 shutdown_connections 2024-09-10T23:39:13.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.216+0000 7fb1bb200640 1 -- 172.21.15.94:0/502451250 wait complete. 2024-09-10T23:39:13.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.216+0000 7fb1bb200640 1 Processor -- start 2024-09-10T23:39:13.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.216+0000 7fb1bb200640 1 -- start start 2024-09-10T23:39:13.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1bb200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1b40b1100 0x7fb1b4158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:13.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1bb200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1b40ce1e0 0x7fb1b40b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:13.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1bb200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1b40ce890 0x7fb1b40cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:13.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1bb200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b40d08e0 con 0x7fb1b40ce890 2024-09-10T23:39:13.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1bb200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b40d0760 con 0x7fb1b40ce1e0 2024-09-10T23:39:13.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1bb200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b40d0a60 con 0x7fb1b40b1100 2024-09-10T23:39:13.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1b9200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1b40ce890 0x7fb1b40cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:13.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1b9200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1b40ce890 0x7fb1b40cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33080/0 (socket says 172.21.15.94:33080) 2024-09-10T23:39:13.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1b9200640 1 -- 172.21.15.94:0/1810541942 learned_addr learned my addr 172.21.15.94:0/1810541942 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:13.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1b3400640 1 --2- 172.21.15.94:0/1810541942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1b40ce1e0 0x7fb1b40b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:13.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1b3e00640 1 --2- 172.21.15.94:0/1810541942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1b40b1100 0x7fb1b4158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:13.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1b9200640 1 -- 172.21.15.94:0/1810541942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1b40b1100 msgr2=0x7fb1b4158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:13.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1b9200640 1 --2- 172.21.15.94:0/1810541942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1b40b1100 0x7fb1b4158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1b9200640 1 -- 172.21.15.94:0/1810541942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1b40ce1e0 msgr2=0x7fb1b40b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:13.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1b9200640 1 --2- 172.21.15.94:0/1810541942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1b40ce1e0 0x7fb1b40b6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1b9200640 1 -- 172.21.15.94:0/1810541942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b40b5b00 con 0x7fb1b40ce890 2024-09-10T23:39:13.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1b3e00640 1 --2- 172.21.15.94:0/1810541942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1b40b1100 0x7fb1b4158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:13.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1b9200640 1 --2- 172.21.15.94:0/1810541942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1b40ce890 0x7fb1b40cb970 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00c7a0 tx=0x7fb1ac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:13.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1abe00640 1 -- 172.21.15.94:0/1810541942 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac003b80 con 0x7fb1b40ce890 2024-09-10T23:39:13.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1abe00640 1 -- 172.21.15.94:0/1810541942 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1ac003d40 con 0x7fb1b40ce890 2024-09-10T23:39:13.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1abe00640 1 -- 172.21.15.94:0/1810541942 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac00aa50 con 0x7fb1b40ce890 2024-09-10T23:39:13.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1bb200640 1 -- 172.21.15.94:0/1810541942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b40b5de0 con 0x7fb1b40ce890 2024-09-10T23:39:13.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1bb200640 1 -- 172.21.15.94:0/1810541942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b40b63a0 con 0x7fb1b40ce890 2024-09-10T23:39:13.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.220+0000 7fb1bb200640 1 -- 172.21.15.94:0/1810541942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b40cec70 con 0x7fb1b40ce890 2024-09-10T23:39:13.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.224+0000 7fb1abe00640 1 -- 172.21.15.94:0/1810541942 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7fb1ac00abf0 con 0x7fb1b40ce890 2024-09-10T23:39:13.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.224+0000 7fb1abe00640 1 --2- 172.21.15.94:0/1810541942 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb194072cd0 0x7fb194075190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:13.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.224+0000 7fb1abe00640 1 -- 172.21.15.94:0/1810541942 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5737+0+0 (secure 0 0 0) 0x7fb1ac093ac0 con 0x7fb1b40ce890 2024-09-10T23:39:13.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.224+0000 7fb1b3e00640 1 --2- 172.21.15.94:0/1810541942 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb194072cd0 0x7fb194075190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:13.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.224+0000 7fb1b3e00640 1 --2- 172.21.15.94:0/1810541942 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb194072cd0 0x7fb194075190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400bbd0 tx=0x7fb1a400b5b0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:13.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.228+0000 7fb1abe00640 1 -- 172.21.15.94:0/1810541942 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb1ac05e870 con 0x7fb1b40ce890 2024-09-10T23:39:13.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.428+0000 7fb1bb200640 1 -- 172.21.15.94:0/1810541942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb1b40c8af0 con 0x7fb1b40ce890 2024-09-10T23:39:13.431 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-10T23:39:13.428+0000 7f4045a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:39:13.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.432+0000 7fb1abe00640 1 -- 172.21.15.94:0/1810541942 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) ==== 161+0+0 (secure 0 0 0) 0x7fb1ac05e170 con 0x7fb1b40ce890 2024-09-10T23:39:13.433 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T23:39:13.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.432+0000 7fb1bb200640 1 -- 172.21.15.94:0/1810541942 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb194072cd0 msgr2=0x7fb194075190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.432+0000 7fb1bb200640 1 --2- 172.21.15.94:0/1810541942 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb194072cd0 0x7fb194075190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400bbd0 tx=0x7fb1a400b5b0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.432+0000 7fb1bb200640 1 -- 172.21.15.94:0/1810541942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1b40ce890 msgr2=0x7fb1b40cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.432+0000 7fb1bb200640 1 --2- 172.21.15.94:0/1810541942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1b40ce890 0x7fb1b40cb970 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00c7a0 tx=0x7fb1ac00cc60 comp rx=0 tx=0).stop 2024-09-10T23:39:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.432+0000 7fb1bb200640 1 -- 172.21.15.94:0/1810541942 shutdown_connections 2024-09-10T23:39:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.432+0000 7fb1bb200640 1 --2- 172.21.15.94:0/1810541942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1b40ce1e0 0x7fb1b40b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.432+0000 7fb1bb200640 1 --2- 172.21.15.94:0/1810541942 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb194072cd0 0x7fb194075190 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.432+0000 7fb1bb200640 1 --2- 172.21.15.94:0/1810541942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1b40ce890 0x7fb1b40cb970 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.432+0000 7fb1bb200640 1 --2- 172.21.15.94:0/1810541942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1b40b1100 0x7fb1b4158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.432+0000 7fb1bb200640 1 -- 172.21.15.94:0/1810541942 >> 172.21.15.94:0/1810541942 conn(0x7fb1b40d1c60 msgr2=0x7fb1b40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.432+0000 7fb1bb200640 1 -- 172.21.15.94:0/1810541942 shutdown_connections 2024-09-10T23:39:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.432+0000 7fb1bb200640 1 -- 172.21.15.94:0/1810541942 wait complete. 2024-09-10T23:39:13.443 INFO:tasks.ceph.mds.d.smithi181.stderr: -1665> 2024-09-10T23:38:52.968+0000 7f494f600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:39:13.457 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T23:39:13.488 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2024-09-10T23:39:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb318000640 1 -- 172.21.15.94:0/4039497540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb31015f560 msgr2=0x7fb3100cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb318000640 1 --2- 172.21.15.94:0/4039497540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb31015f560 0x7fb3100cd0a0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009840 tx=0x7fb2fc01c820 comp rx=0 tx=0).stop 2024-09-10T23:39:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb318000640 1 -- 172.21.15.94:0/4039497540 shutdown_connections 2024-09-10T23:39:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb318000640 1 --2- 172.21.15.94:0/4039497540 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3100d01a0 0x7fb3100b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb318000640 1 --2- 172.21.15.94:0/4039497540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb31015f560 0x7fb3100cd0a0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb318000640 1 -- 172.21.15.94:0/4039497540 >> 172.21.15.94:0/4039497540 conn(0x7fb3100d1cb0 msgr2=0x7fb3100d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb318000640 1 -- 172.21.15.94:0/4039497540 shutdown_connections 2024-09-10T23:39:13.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb318000640 1 -- 172.21.15.94:0/4039497540 wait complete. 2024-09-10T23:39:13.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb318000640 1 Processor -- start 2024-09-10T23:39:13.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb318000640 1 -- start start 2024-09-10T23:39:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb318000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3100d01a0 0x7fb310157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb318000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb31015f180 0x7fb3101fac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb318000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb31015f560 0x7fb3100b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb318000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb310168d10 con 0x7fb31015f560 2024-09-10T23:39:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb318000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb3100b29f0 con 0x7fb3100d01a0 2024-09-10T23:39:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb318000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb310168e90 con 0x7fb31015f180 2024-09-10T23:39:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb316000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb31015f560 0x7fb3100b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb316000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb31015f560 0x7fb3100b5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33102/0 (socket says 172.21.15.94:33102) 2024-09-10T23:39:13.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb316000640 1 -- 172.21.15.94:0/4189586320 learned_addr learned my addr 172.21.15.94:0/4189586320 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb314c00640 1 --2- 172.21.15.94:0/4189586320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb31015f180 0x7fb3101fac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb315600640 1 --2- 172.21.15.94:0/4189586320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3100d01a0 0x7fb310157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb316000640 1 -- 172.21.15.94:0/4189586320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb31015f180 msgr2=0x7fb3101fac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.600+0000 7fb316000640 1 --2- 172.21.15.94:0/4189586320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb31015f180 0x7fb3101fac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.604+0000 7fb316000640 1 -- 172.21.15.94:0/4189586320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3100d01a0 msgr2=0x7fb310157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.604+0000 7fb316000640 1 --2- 172.21.15.94:0/4189586320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3100d01a0 0x7fb310157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.604+0000 7fb316000640 1 -- 172.21.15.94:0/4189586320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3101fb6b0 con 0x7fb31015f560 2024-09-10T23:39:13.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.604+0000 7fb314c00640 1 --2- 172.21.15.94:0/4189586320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb31015f180 0x7fb3101fac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:39:13.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.604+0000 7fb316000640 1 --2- 172.21.15.94:0/4189586320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb31015f560 0x7fb3100b5cd0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb30800eb20 tx=0x7fb30800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:13.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.604+0000 7fb30d600640 1 -- 172.21.15.94:0/4189586320 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308004b30 con 0x7fb31015f560 2024-09-10T23:39:13.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.604+0000 7fb30d600640 1 -- 172.21.15.94:0/4189586320 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb308004cf0 con 0x7fb31015f560 2024-09-10T23:39:13.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.604+0000 7fb30d600640 1 -- 172.21.15.94:0/4189586320 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308005280 con 0x7fb31015f560 2024-09-10T23:39:13.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.604+0000 7fb318000640 1 -- 172.21.15.94:0/4189586320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3101fb990 con 0x7fb31015f560 2024-09-10T23:39:13.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.604+0000 7fb318000640 1 -- 172.21.15.94:0/4189586320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb310158f90 con 0x7fb31015f560 2024-09-10T23:39:13.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.604+0000 7fb318000640 1 -- 172.21.15.94:0/4189586320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb310158ad0 con 0x7fb31015f560 2024-09-10T23:39:13.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.604+0000 7fb30d600640 1 -- 172.21.15.94:0/4189586320 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7fb308005420 con 0x7fb31015f560 2024-09-10T23:39:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.604+0000 7fb30d600640 1 --2- 172.21.15.94:0/4189586320 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb2f0072ca0 0x7fb2f0075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.604+0000 7fb30d600640 1 -- 172.21.15.94:0/4189586320 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5823+0+0 (secure 0 0 0) 0x7fb308094a70 con 0x7fb31015f560 2024-09-10T23:39:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.604+0000 7fb315600640 1 --2- 172.21.15.94:0/4189586320 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb2f0072ca0 0x7fb2f0075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:13.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.608+0000 7fb315600640 1 --2- 172.21.15.94:0/4189586320 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb2f0072ca0 0x7fb2f0075160 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb304007fd0 tx=0x7fb304007ea0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:13.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.608+0000 7fb30d600640 1 -- 172.21.15.94:0/4189586320 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb310158ad0 con 0x7fb31015f560 2024-09-10T23:39:13.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:13.808+0000 7fb318000640 1 -- 172.21.15.94:0/4189586320 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb3100b16f0 con 0x7fb31015f560 2024-09-10T23:39:14.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.432+0000 7fb30d600640 1 -- 172.21.15.94:0/4189586320 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v91) ==== 104+0+0 (secure 0 0 0) 0x7fb30805f180 con 0x7fb31015f560 2024-09-10T23:39:14.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.436+0000 7fb318000640 1 -- 172.21.15.94:0/4189586320 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb2f0072ca0 msgr2=0x7fb2f0075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:14.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.436+0000 7fb318000640 1 --2- 172.21.15.94:0/4189586320 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb2f0072ca0 0x7fb2f0075160 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb304007fd0 tx=0x7fb304007ea0 comp rx=0 tx=0).stop 2024-09-10T23:39:14.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.436+0000 7fb318000640 1 -- 172.21.15.94:0/4189586320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb31015f560 msgr2=0x7fb3100b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:14.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.436+0000 7fb318000640 1 --2- 172.21.15.94:0/4189586320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb31015f560 0x7fb3100b5cd0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb30800eb20 tx=0x7fb30800c2d0 comp rx=0 tx=0).stop 2024-09-10T23:39:14.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.436+0000 7fb318000640 1 -- 172.21.15.94:0/4189586320 shutdown_connections 2024-09-10T23:39:14.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.436+0000 7fb318000640 1 --2- 172.21.15.94:0/4189586320 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb2f0072ca0 0x7fb2f0075160 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:14.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.436+0000 7fb318000640 1 --2- 172.21.15.94:0/4189586320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb31015f560 0x7fb3100b5cd0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:14.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.436+0000 7fb318000640 1 --2- 172.21.15.94:0/4189586320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb31015f180 0x7fb3101fac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:14.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.436+0000 7fb318000640 1 --2- 172.21.15.94:0/4189586320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3100d01a0 0x7fb310157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:14.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.436+0000 7fb318000640 1 -- 172.21.15.94:0/4189586320 >> 172.21.15.94:0/4189586320 conn(0x7fb3100d1cb0 msgr2=0x7fb31015d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:14.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.436+0000 7fb318000640 1 -- 172.21.15.94:0/4189586320 shutdown_connections 2024-09-10T23:39:14.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.436+0000 7fb318000640 1 -- 172.21.15.94:0/4189586320 wait complete. 2024-09-10T23:39:14.459 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T23:39:14.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.604+0000 7f5f56600640 1 -- 172.21.15.94:0/1079543010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f5006cc20 msgr2=0x7f5f5006d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:14.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.604+0000 7f5f56600640 1 --2- 172.21.15.94:0/1079543010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f5006cc20 0x7f5f5006d000 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f5f50073620 tx=0x7f5f4001c810 comp rx=0 tx=0).stop 2024-09-10T23:39:14.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.604+0000 7f5f56600640 1 -- 172.21.15.94:0/1079543010 shutdown_connections 2024-09-10T23:39:14.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.604+0000 7f5f56600640 1 --2- 172.21.15.94:0/1079543010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f500d1690 0x7f5f5015fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:14.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.604+0000 7f5f56600640 1 --2- 172.21.15.94:0/1079543010 >> v2:172.21.15.181:3300/0 conn(0x7f5f50071a20 0x7f5f5015f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:14.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.604+0000 7f5f56600640 1 --2- 172.21.15.94:0/1079543010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f5006cc20 0x7f5f5006d000 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:14.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.604+0000 7f5f56600640 1 -- 172.21.15.94:0/1079543010 >> 172.21.15.94:0/1079543010 conn(0x7f5f500d0b70 msgr2=0x7f5f500d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:14.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.604+0000 7f5f56600640 1 -- 172.21.15.94:0/1079543010 shutdown_connections 2024-09-10T23:39:14.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.604+0000 7f5f56600640 1 -- 172.21.15.94:0/1079543010 wait complete. 2024-09-10T23:39:14.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.604+0000 7f5f56600640 1 Processor -- start 2024-09-10T23:39:14.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.604+0000 7f5f56600640 1 -- start start 2024-09-10T23:39:14.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f56600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f5006cc20 0x7f5f501511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:14.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f56600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f50071a20 0x7f5f5014fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:14.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f56600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f500d1690 0x7f5f501501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:14.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f56600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f500cff30 con 0x7f5f5006cc20 2024-09-10T23:39:14.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f56600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5f500cfdb0 con 0x7f5f50071a20 2024-09-10T23:39:14.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f56600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5f500d00b0 con 0x7f5f500d1690 2024-09-10T23:39:14.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f4f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f5006cc20 0x7f5f501511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:14.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f4f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f5006cc20 0x7f5f501511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41804/0 (socket says 172.21.15.94:41804) 2024-09-10T23:39:14.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f4f400640 1 -- 172.21.15.94:0/2445641697 learned_addr learned my addr 172.21.15.94:0/2445641697 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:14.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f46a00640 1 --2- 172.21.15.94:0/2445641697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f50071a20 0x7f5f5014fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:14.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f4fe00640 1 --2- 172.21.15.94:0/2445641697 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f500d1690 0x7f5f501501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:14.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f4f400640 1 -- 172.21.15.94:0/2445641697 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f500d1690 msgr2=0x7f5f501501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:14.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f4f400640 1 --2- 172.21.15.94:0/2445641697 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f500d1690 0x7f5f501501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:14.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f4f400640 1 -- 172.21.15.94:0/2445641697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f50071a20 msgr2=0x7f5f5014fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:14.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f4f400640 1 --2- 172.21.15.94:0/2445641697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f50071a20 0x7f5f5014fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:14.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f4f400640 1 -- 172.21.15.94:0/2445641697 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f50151710 con 0x7f5f5006cc20 2024-09-10T23:39:14.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f4f400640 1 --2- 172.21.15.94:0/2445641697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f5006cc20 0x7f5f501511d0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f5f40000c10 tx=0x7f5f400a6590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:14.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f47e00640 1 -- 172.21.15.94:0/2445641697 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f400ac070 con 0x7f5f5006cc20 2024-09-10T23:39:14.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f47e00640 1 -- 172.21.15.94:0/2445641697 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f40004310 con 0x7f5f5006cc20 2024-09-10T23:39:14.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f47e00640 1 -- 172.21.15.94:0/2445641697 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f400053a0 con 0x7f5f5006cc20 2024-09-10T23:39:14.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f56600640 1 -- 172.21.15.94:0/2445641697 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f50160ce0 con 0x7f5f5006cc20 2024-09-10T23:39:14.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f56600640 1 -- 172.21.15.94:0/2445641697 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f50134570 con 0x7f5f5006cc20 2024-09-10T23:39:14.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f56600640 1 -- 172.21.15.94:0/2445641697 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f500d0610 con 0x7f5f5006cc20 2024-09-10T23:39:14.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.608+0000 7f5f47e00640 1 -- 172.21.15.94:0/2445641697 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f5f400046d0 con 0x7f5f5006cc20 2024-09-10T23:39:14.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.612+0000 7f5f47e00640 1 --2- 172.21.15.94:0/2445641697 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f2c072d00 0x7f5f2c0751c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:14.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.612+0000 7f5f47e00640 1 -- 172.21.15.94:0/2445641697 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5823+0+0 (secure 0 0 0) 0x7f5f4012e640 con 0x7f5f5006cc20 2024-09-10T23:39:14.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.612+0000 7f5f46a00640 1 --2- 172.21.15.94:0/2445641697 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f2c072d00 0x7f5f2c0751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:14.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.612+0000 7f5f46a00640 1 --2- 172.21.15.94:0/2445641697 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f2c072d00 0x7f5f2c0751c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f38005fd0 tx=0x7f5f38005db0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:14.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.616+0000 7f5f47e00640 1 -- 172.21.15.94:0/2445641697 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5f400b6070 con 0x7f5f5006cc20 2024-09-10T23:39:14.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:14.780+0000 7f5f56600640 1 -- 172.21.15.94:0/2445641697 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5f50159940 con 0x7f5f5006cc20 2024-09-10T23:39:15.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.524+0000 7f5f47e00640 1 -- 172.21.15.94:0/2445641697 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v129) ==== 181+0+0 (secure 0 0 0) 0x7f5f40016490 con 0x7f5f5006cc20 2024-09-10T23:39:15.529 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-09-10T23:39:15.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.528+0000 7f5f56600640 1 -- 172.21.15.94:0/2445641697 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f2c072d00 msgr2=0x7f5f2c0751c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:15.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.528+0000 7f5f56600640 1 --2- 172.21.15.94:0/2445641697 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f2c072d00 0x7f5f2c0751c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f38005fd0 tx=0x7f5f38005db0 comp rx=0 tx=0).stop 2024-09-10T23:39:15.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.528+0000 7f5f56600640 1 -- 172.21.15.94:0/2445641697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f5006cc20 msgr2=0x7f5f501511d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:15.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.528+0000 7f5f56600640 1 --2- 172.21.15.94:0/2445641697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f5006cc20 0x7f5f501511d0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f5f40000c10 tx=0x7f5f400a6590 comp rx=0 tx=0).stop 2024-09-10T23:39:15.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.528+0000 7f5f56600640 1 -- 172.21.15.94:0/2445641697 shutdown_connections 2024-09-10T23:39:15.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.528+0000 7f5f56600640 1 --2- 172.21.15.94:0/2445641697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f50071a20 0x7f5f5014fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:15.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.528+0000 7f5f56600640 1 --2- 172.21.15.94:0/2445641697 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f2c072d00 0x7f5f2c0751c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:15.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.528+0000 7f5f56600640 1 --2- 172.21.15.94:0/2445641697 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f500d1690 0x7f5f501501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:15.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.528+0000 7f5f56600640 1 --2- 172.21.15.94:0/2445641697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f5006cc20 0x7f5f501511d0 secure :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f5f40000c10 tx=0x7f5f400a6590 comp rx=0 tx=0).stop 2024-09-10T23:39:15.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.528+0000 7f5f56600640 1 -- 172.21.15.94:0/2445641697 >> 172.21.15.94:0/2445641697 conn(0x7f5f500d0b70 msgr2=0x7f5f50070c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:15.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.528+0000 7f5f56600640 1 -- 172.21.15.94:0/2445641697 shutdown_connections 2024-09-10T23:39:15.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.528+0000 7f5f56600640 1 -- 172.21.15.94:0/2445641697 wait complete. 2024-09-10T23:39:15.552 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T23:39:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.696+0000 7ff845400640 1 -- 172.21.15.94:0/1669600020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8240051d0 msgr2=0x7ff82400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.696+0000 7ff845400640 1 --2- 172.21.15.94:0/1669600020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8240051d0 0x7ff82400b9d0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7ff838009d10 tx=0x7ff83801d470 comp rx=0 tx=0).stop 2024-09-10T23:39:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.696+0000 7ff845400640 1 -- 172.21.15.94:0/1669600020 shutdown_connections 2024-09-10T23:39:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.696+0000 7ff845400640 1 --2- 172.21.15.94:0/1669600020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8240051d0 0x7ff82400b9d0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.696+0000 7ff845400640 1 -- 172.21.15.94:0/1669600020 >> 172.21.15.94:0/1669600020 conn(0x7ff8400d1a40 msgr2=0x7ff8400d1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.696+0000 7ff845400640 1 -- 172.21.15.94:0/1669600020 shutdown_connections 2024-09-10T23:39:15.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.696+0000 7ff845400640 1 -- 172.21.15.94:0/1669600020 wait complete. 2024-09-10T23:39:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.696+0000 7ff845400640 1 Processor -- start 2024-09-10T23:39:15.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.696+0000 7ff845400640 1 -- start start 2024-09-10T23:39:15.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff845400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8240051d0 0x7ff8401f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:15.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff845400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8400ca690 0x7ff8400caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:15.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff845400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff84014ba50 0x7ff84014bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff845400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff8400d28f0 con 0x7ff84014ba50 2024-09-10T23:39:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff845400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff8400d2770 con 0x7ff8400ca690 2024-09-10T23:39:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff845400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff8400cff80 con 0x7ff8240051d0 2024-09-10T23:39:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff844a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff84014ba50 0x7ff84014bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff844a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff84014ba50 0x7ff84014bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41826/0 (socket says 172.21.15.94:41826) 2024-09-10T23:39:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff844a00640 1 -- 172.21.15.94:0/1631436170 learned_addr learned my addr 172.21.15.94:0/1631436170 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff83fe00640 1 --2- 172.21.15.94:0/1631436170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8240051d0 0x7ff8401f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff837400640 1 --2- 172.21.15.94:0/1631436170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8400ca690 0x7ff8400caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff844a00640 1 -- 172.21.15.94:0/1631436170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8240051d0 msgr2=0x7ff8401f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff844a00640 1 --2- 172.21.15.94:0/1631436170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8240051d0 0x7ff8401f7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff844a00640 1 -- 172.21.15.94:0/1631436170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8400ca690 msgr2=0x7ff8400caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff844a00640 1 --2- 172.21.15.94:0/1631436170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8400ca690 0x7ff8400caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:15.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff844a00640 1 -- 172.21.15.94:0/1631436170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8400cb1c0 con 0x7ff84014ba50 2024-09-10T23:39:15.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff844a00640 1 --2- 172.21.15.94:0/1631436170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff84014ba50 0x7ff84014bf00 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7ff83801d2a0 tx=0x7ff838005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:15.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff83cc00640 1 -- 172.21.15.94:0/1631436170 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8380098e0 con 0x7ff84014ba50 2024-09-10T23:39:15.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff83cc00640 1 -- 172.21.15.94:0/1631436170 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8380bb070 con 0x7ff84014ba50 2024-09-10T23:39:15.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff83cc00640 1 -- 172.21.15.94:0/1631436170 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838005ca0 con 0x7ff84014ba50 2024-09-10T23:39:15.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff845400640 1 -- 172.21.15.94:0/1631436170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8400cb4a0 con 0x7ff84014ba50 2024-09-10T23:39:15.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff845400640 1 -- 172.21.15.94:0/1631436170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8400cba60 con 0x7ff84014ba50 2024-09-10T23:39:15.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff845400640 1 -- 172.21.15.94:0/1631436170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8400ce160 con 0x7ff84014ba50 2024-09-10T23:39:15.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.700+0000 7ff83cc00640 1 -- 172.21.15.94:0/1631436170 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7ff8380aa070 con 0x7ff84014ba50 2024-09-10T23:39:15.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.704+0000 7ff83cc00640 1 --2- 172.21.15.94:0/1631436170 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff81c072cd0 0x7ff81c075190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:15.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.704+0000 7ff83cc00640 1 -- 172.21.15.94:0/1631436170 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5385+0+0 (secure 0 0 0) 0x7ff83812ebf0 con 0x7ff84014ba50 2024-09-10T23:39:15.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.704+0000 7ff83fe00640 1 --2- 172.21.15.94:0/1631436170 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff81c072cd0 0x7ff81c075190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:15.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.704+0000 7ff83fe00640 1 --2- 172.21.15.94:0/1631436170 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff81c072cd0 0x7ff81c075190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff830001bf0 tx=0x7ff830006040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:15.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.708+0000 7ff83cc00640 1 -- 172.21.15.94:0/1631436170 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff838017180 con 0x7ff84014ba50 2024-09-10T23:39:15.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:15.872+0000 7ff845400640 1 -- 172.21.15.94:0/1631436170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff8400c4a60 con 0x7ff84014ba50 2024-09-10T23:39:16.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.528+0000 7ff83cc00640 1 -- 172.21.15.94:0/1631436170 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v130) ==== 169+0+0 (secure 0 0 0) 0x7ff8380f99a0 con 0x7ff84014ba50 2024-09-10T23:39:16.531 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-09-10T23:39:16.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.532+0000 7ff845400640 1 -- 172.21.15.94:0/1631436170 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff81c072cd0 msgr2=0x7ff81c075190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:16.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.532+0000 7ff845400640 1 --2- 172.21.15.94:0/1631436170 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff81c072cd0 0x7ff81c075190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff830001bf0 tx=0x7ff830006040 comp rx=0 tx=0).stop 2024-09-10T23:39:16.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.532+0000 7ff845400640 1 -- 172.21.15.94:0/1631436170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff84014ba50 msgr2=0x7ff84014bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:16.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.532+0000 7ff845400640 1 --2- 172.21.15.94:0/1631436170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff84014ba50 0x7ff84014bf00 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7ff83801d2a0 tx=0x7ff838005640 comp rx=0 tx=0).stop 2024-09-10T23:39:16.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.532+0000 7ff845400640 1 -- 172.21.15.94:0/1631436170 shutdown_connections 2024-09-10T23:39:16.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.532+0000 7ff845400640 1 --2- 172.21.15.94:0/1631436170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8400ca690 0x7ff8400caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:16.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.532+0000 7ff845400640 1 --2- 172.21.15.94:0/1631436170 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff81c072cd0 0x7ff81c075190 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:16.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.532+0000 7ff845400640 1 --2- 172.21.15.94:0/1631436170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff84014ba50 0x7ff84014bf00 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:16.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.532+0000 7ff845400640 1 --2- 172.21.15.94:0/1631436170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8240051d0 0x7ff8401f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:16.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.532+0000 7ff845400640 1 -- 172.21.15.94:0/1631436170 >> 172.21.15.94:0/1631436170 conn(0x7ff8400d1a40 msgr2=0x7ff8400b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:16.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.532+0000 7ff845400640 1 -- 172.21.15.94:0/1631436170 shutdown_connections 2024-09-10T23:39:16.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.532+0000 7ff845400640 1 -- 172.21.15.94:0/1631436170 wait complete. 2024-09-10T23:39:16.556 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T23:39:16.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.696+0000 7fecbee00640 1 -- 172.21.15.94:0/314195697 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecb80ce280 msgr2=0x7fecb80cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:16.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.696+0000 7fecbee00640 1 --2- 172.21.15.94:0/314195697 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecb80ce280 0x7fecb80cf280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feca800a950 tx=0x7feca801db70 comp rx=0 tx=0).stop 2024-09-10T23:39:16.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.696+0000 7fecbee00640 1 -- 172.21.15.94:0/314195697 shutdown_connections 2024-09-10T23:39:16.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.696+0000 7fecbee00640 1 --2- 172.21.15.94:0/314195697 >> v2:172.21.15.181:3300/0 conn(0x7fecb806e7c0 0x7fecb815af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:16.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.696+0000 7fecbee00640 1 --2- 172.21.15.94:0/314195697 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecb80ce280 0x7fecb80cf280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:16.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.696+0000 7fecbee00640 1 -- 172.21.15.94:0/314195697 >> 172.21.15.94:0/314195697 conn(0x7fecb80cfaf0 msgr2=0x7fecb80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:16.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.696+0000 7fecbee00640 1 -- 172.21.15.94:0/314195697 shutdown_connections 2024-09-10T23:39:16.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.696+0000 7fecbee00640 1 -- 172.21.15.94:0/314195697 wait complete. 2024-09-10T23:39:16.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecbee00640 1 Processor -- start 2024-09-10T23:39:16.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecbee00640 1 -- start start 2024-09-10T23:39:16.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecbee00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecb806e7c0 0x7fecb8072bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:16.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecbee00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb80c64f0 0x7fecb80c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:16.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecbee00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb80ce280 0x7fecb80c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:16.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecbee00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fecb8167680 con 0x7fecb80c64f0 2024-09-10T23:39:16.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecbee00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fecb8167500 con 0x7fecb80ce280 2024-09-10T23:39:16.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecbee00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fecb8167800 con 0x7fecb806e7c0 2024-09-10T23:39:16.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecaf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb80c64f0 0x7fecb80c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:16.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecaf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb80c64f0 0x7fecb80c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41838/0 (socket says 172.21.15.94:41838) 2024-09-10T23:39:16.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecaf400640 1 -- 172.21.15.94:0/2235764232 learned_addr learned my addr 172.21.15.94:0/2235764232 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:16.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecb7e00640 1 --2- 172.21.15.94:0/2235764232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecb806e7c0 0x7fecb8072bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:16.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecbce00640 1 --2- 172.21.15.94:0/2235764232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb80ce280 0x7fecb80c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:16.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecaf400640 1 -- 172.21.15.94:0/2235764232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecb806e7c0 msgr2=0x7fecb8072bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:16.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecaf400640 1 --2- 172.21.15.94:0/2235764232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecb806e7c0 0x7fecb8072bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:16.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecaf400640 1 -- 172.21.15.94:0/2235764232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb80ce280 msgr2=0x7fecb80c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:16.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecaf400640 1 --2- 172.21.15.94:0/2235764232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb80ce280 0x7fecb80c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:16.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecaf400640 1 -- 172.21.15.94:0/2235764232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb814ad40 con 0x7fecb80c64f0 2024-09-10T23:39:16.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecaf400640 1 --2- 172.21.15.94:0/2235764232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb80c64f0 0x7fecb80c6e20 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7feca000c460 tx=0x7feca000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:16.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecb4c00640 1 -- 172.21.15.94:0/2235764232 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0011790 con 0x7fecb80c64f0 2024-09-10T23:39:16.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecb4c00640 1 -- 172.21.15.94:0/2235764232 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feca0017070 con 0x7fecb80c64f0 2024-09-10T23:39:16.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecbee00640 1 -- 172.21.15.94:0/2235764232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb80c7a00 con 0x7fecb80c64f0 2024-09-10T23:39:16.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecb4c00640 1 -- 172.21.15.94:0/2235764232 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0011ca0 con 0x7fecb80c64f0 2024-09-10T23:39:16.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecbee00640 1 -- 172.21.15.94:0/2235764232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb8151b50 con 0x7fecb80c64f0 2024-09-10T23:39:16.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.700+0000 7fecbee00640 1 -- 172.21.15.94:0/2235764232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb815af90 con 0x7fecb80c64f0 2024-09-10T23:39:16.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.704+0000 7fecb4c00640 1 -- 172.21.15.94:0/2235764232 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7feca001c460 con 0x7fecb80c64f0 2024-09-10T23:39:16.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.704+0000 7fecb4c00640 1 --2- 172.21.15.94:0/2235764232 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fec94072cd0 0x7fec94075190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:16.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.704+0000 7fecb4c00640 1 -- 172.21.15.94:0/2235764232 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4971+0+0 (secure 0 0 0) 0x7feca00938e0 con 0x7fecb80c64f0 2024-09-10T23:39:16.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.704+0000 7fecb7e00640 1 --2- 172.21.15.94:0/2235764232 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fec94072cd0 0x7fec94075190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:16.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.704+0000 7fecb7e00640 1 --2- 172.21.15.94:0/2235764232 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fec94072cd0 0x7fec94075190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feca80b0000 tx=0x7feca801c3b0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:16.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.708+0000 7fecb4c00640 1 -- 172.21.15.94:0/2235764232 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fecb814ad40 con 0x7fecb80c64f0 2024-09-10T23:39:16.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.884+0000 7fecbee00640 1 -- 172.21.15.94:0/2235764232 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fecb80d0850 con 0x7fecb80c64f0 2024-09-10T23:39:16.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.888+0000 7fecb4c00640 1 -- 172.21.15.94:0/2235764232 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v43) ==== 71+0+0 (secure 0 0 0) 0x7feca005e8d0 con 0x7fecb80c64f0 2024-09-10T23:39:16.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.892+0000 7fecbee00640 1 -- 172.21.15.94:0/2235764232 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fec94072cd0 msgr2=0x7fec94075190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:16.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.892+0000 7fecbee00640 1 --2- 172.21.15.94:0/2235764232 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fec94072cd0 0x7fec94075190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feca80b0000 tx=0x7feca801c3b0 comp rx=0 tx=0).stop 2024-09-10T23:39:16.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.892+0000 7fecbee00640 1 -- 172.21.15.94:0/2235764232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb80c64f0 msgr2=0x7fecb80c6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:16.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.892+0000 7fecbee00640 1 --2- 172.21.15.94:0/2235764232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb80c64f0 0x7fecb80c6e20 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7feca000c460 tx=0x7feca000c860 comp rx=0 tx=0).stop 2024-09-10T23:39:16.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.892+0000 7fecbee00640 1 -- 172.21.15.94:0/2235764232 shutdown_connections 2024-09-10T23:39:16.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.892+0000 7fecbee00640 1 --2- 172.21.15.94:0/2235764232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecb80ce280 0x7fecb80c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:16.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.892+0000 7fecbee00640 1 --2- 172.21.15.94:0/2235764232 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fec94072cd0 0x7fec94075190 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:16.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.892+0000 7fecbee00640 1 --2- 172.21.15.94:0/2235764232 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb80c64f0 0x7fecb80c6e20 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:16.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.892+0000 7fecbee00640 1 --2- 172.21.15.94:0/2235764232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecb806e7c0 0x7fecb8072bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:16.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.892+0000 7fecbee00640 1 -- 172.21.15.94:0/2235764232 >> 172.21.15.94:0/2235764232 conn(0x7fecb80cfaf0 msgr2=0x7fecb814d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:16.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.892+0000 7fecbee00640 1 -- 172.21.15.94:0/2235764232 shutdown_connections 2024-09-10T23:39:16.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:16.892+0000 7fecbee00640 1 -- 172.21.15.94:0/2235764232 wait complete. 2024-09-10T23:39:16.913 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-10T23:39:17.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f0336800640 1 -- 172.21.15.94:0/3025376051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033015f3f0 msgr2=0x7f0330068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:17.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f0336800640 1 --2- 172.21.15.94:0/3025376051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033015f3f0 0x7f0330068990 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f0320009c80 tx=0x7f032001cd10 comp rx=0 tx=0).stop 2024-09-10T23:39:17.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f0336800640 1 -- 172.21.15.94:0/3025376051 shutdown_connections 2024-09-10T23:39:17.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f0336800640 1 --2- 172.21.15.94:0/3025376051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033015f3f0 0x7f0330068990 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:17.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f0336800640 1 -- 172.21.15.94:0/3025376051 >> 172.21.15.94:0/3025376051 conn(0x7f03300d02d0 msgr2=0x7f03300d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:17.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f0336800640 1 -- 172.21.15.94:0/3025376051 shutdown_connections 2024-09-10T23:39:17.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f0336800640 1 -- 172.21.15.94:0/3025376051 wait complete. 2024-09-10T23:39:17.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f0336800640 1 Processor -- start 2024-09-10T23:39:17.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f0336800640 1 -- start start 2024-09-10T23:39:17.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f0336800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03300cc2f0 0x7f03301f9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:17.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f0336800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03300cdea0 0x7f03301f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:17.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f0336800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033015f3f0 0x7f03300983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:17.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f0336800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f033006da30 con 0x7f033015f3f0 2024-09-10T23:39:17.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f0336800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f033006d8b0 con 0x7f03300cdea0 2024-09-10T23:39:17.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f0336800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f03301697f0 con 0x7f03300cc2f0 2024-09-10T23:39:17.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f032fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033015f3f0 0x7f03300983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:17.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f032fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033015f3f0 0x7f03300983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41850/0 (socket says 172.21.15.94:41850) 2024-09-10T23:39:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f032fe00640 1 -- 172.21.15.94:0/3495254169 learned_addr learned my addr 172.21.15.94:0/3495254169 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f032f400640 1 --2- 172.21.15.94:0/3495254169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03300cc2f0 0x7f03301f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.052+0000 7f032ea00640 1 --2- 172.21.15.94:0/3495254169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03300cdea0 0x7f03301f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.056+0000 7f032fe00640 1 -- 172.21.15.94:0/3495254169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03300cc2f0 msgr2=0x7f03301f9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.056+0000 7f032fe00640 1 --2- 172.21.15.94:0/3495254169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03300cc2f0 0x7f03301f9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.056+0000 7f032fe00640 1 -- 172.21.15.94:0/3495254169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03300cdea0 msgr2=0x7f03301f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.056+0000 7f032fe00640 1 --2- 172.21.15.94:0/3495254169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03300cdea0 0x7f03301f7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.056+0000 7f032fe00640 1 -- 172.21.15.94:0/3495254169 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03300c73f0 con 0x7f033015f3f0 2024-09-10T23:39:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.056+0000 7f032fe00640 1 --2- 172.21.15.94:0/3495254169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033015f3f0 0x7f03300983f0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f032800c3c0 tx=0x7f032800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.056+0000 7f0327400640 1 -- 172.21.15.94:0/3495254169 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328011790 con 0x7f033015f3f0 2024-09-10T23:39:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.056+0000 7f0327400640 1 -- 172.21.15.94:0/3495254169 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0328017070 con 0x7f033015f3f0 2024-09-10T23:39:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.056+0000 7f0327400640 1 -- 172.21.15.94:0/3495254169 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328011cc0 con 0x7f033015f3f0 2024-09-10T23:39:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.056+0000 7f0336800640 1 -- 172.21.15.94:0/3495254169 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03300b4710 con 0x7f033015f3f0 2024-09-10T23:39:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.056+0000 7f0336800640 1 -- 172.21.15.94:0/3495254169 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03300b29b0 con 0x7f033015f3f0 2024-09-10T23:39:17.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.056+0000 7f0336800640 1 -- 172.21.15.94:0/3495254169 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03300b6250 con 0x7f033015f3f0 2024-09-10T23:39:17.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.056+0000 7f0327400640 1 -- 172.21.15.94:0/3495254169 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f032801c460 con 0x7f033015f3f0 2024-09-10T23:39:17.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.056+0000 7f0327400640 1 --2- 172.21.15.94:0/3495254169 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0310072ca0 0x7f0310075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:17.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.060+0000 7f0327400640 1 -- 172.21.15.94:0/3495254169 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4971+0+0 (secure 0 0 0) 0x7f03280937e0 con 0x7f033015f3f0 2024-09-10T23:39:17.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.060+0000 7f032f400640 1 --2- 172.21.15.94:0/3495254169 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0310072ca0 0x7f0310075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:17.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.060+0000 7f032f400640 1 --2- 172.21.15.94:0/3495254169 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0310072ca0 0x7f0310075160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f03200af000 tx=0x7f03200a5f90 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:17.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.060+0000 7f0327400640 1 -- 172.21.15.94:0/3495254169 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f03300c73f0 con 0x7f033015f3f0 2024-09-10T23:39:17.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.288+0000 7f0336800640 1 -- 172.21.15.94:0/3495254169 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7f0330164250 con 0x7f033015f3f0 2024-09-10T23:39:17.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.544+0000 7f0327400640 1 -- 172.21.15.94:0/3495254169 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v372) ==== 151+0+0 (secure 0 0 0) 0x7f032805e200 con 0x7f033015f3f0 2024-09-10T23:39:17.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.548+0000 7f0336800640 1 -- 172.21.15.94:0/3495254169 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0310072ca0 msgr2=0x7f0310075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:17.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.548+0000 7f0336800640 1 --2- 172.21.15.94:0/3495254169 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0310072ca0 0x7f0310075160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f03200af000 tx=0x7f03200a5f90 comp rx=0 tx=0).stop 2024-09-10T23:39:17.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.548+0000 7f0336800640 1 -- 172.21.15.94:0/3495254169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033015f3f0 msgr2=0x7f03300983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:17.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.548+0000 7f0336800640 1 --2- 172.21.15.94:0/3495254169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033015f3f0 0x7f03300983f0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f032800c3c0 tx=0x7f032800c880 comp rx=0 tx=0).stop 2024-09-10T23:39:17.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.548+0000 7f0336800640 1 -- 172.21.15.94:0/3495254169 shutdown_connections 2024-09-10T23:39:17.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.548+0000 7f0336800640 1 --2- 172.21.15.94:0/3495254169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03300cdea0 0x7f03301f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:17.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.548+0000 7f0336800640 1 --2- 172.21.15.94:0/3495254169 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0310072ca0 0x7f0310075160 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:17.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.548+0000 7f0336800640 1 --2- 172.21.15.94:0/3495254169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033015f3f0 0x7f03300983f0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:17.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.548+0000 7f0336800640 1 --2- 172.21.15.94:0/3495254169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03300cc2f0 0x7f03301f9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:17.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.548+0000 7f0336800640 1 -- 172.21.15.94:0/3495254169 >> 172.21.15.94:0/3495254169 conn(0x7f03300d02d0 msgr2=0x7f0330157aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:17.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.548+0000 7f0336800640 1 -- 172.21.15.94:0/3495254169 shutdown_connections 2024-09-10T23:39:17.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.548+0000 7f0336800640 1 -- 172.21.15.94:0/3495254169 wait complete. 2024-09-10T23:39:17.571 INFO:tasks.cephfs_test_runner:test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-10T23:39:17.571 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- rather than going through a full upgrade cycle, 2024-09-10T23:39:17.571 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T23:39:17.572 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:39:17.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.708+0000 7fc575000640 1 -- 172.21.15.94:0/2763792480 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5700b2bb0 msgr2=0x7fc5700d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:17.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.708+0000 7fc575000640 1 --2- 172.21.15.94:0/2763792480 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5700b2bb0 0x7fc5700d00a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc558009900 tx=0x7fc55801c7c0 comp rx=0 tx=0).stop 2024-09-10T23:39:17.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.708+0000 7fc575000640 1 -- 172.21.15.94:0/2763792480 shutdown_connections 2024-09-10T23:39:17.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.708+0000 7fc575000640 1 --2- 172.21.15.94:0/2763792480 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5700b2bb0 0x7fc5700d00a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:17.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.708+0000 7fc575000640 1 --2- 172.21.15.94:0/2763792480 >> v2:172.21.15.181:3300/0 conn(0x7fc57015f5c0 0x7fc5700cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:17.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.708+0000 7fc575000640 1 -- 172.21.15.94:0/2763792480 >> 172.21.15.94:0/2763792480 conn(0x7fc5700d1c60 msgr2=0x7fc5700d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:17.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.708+0000 7fc575000640 1 -- 172.21.15.94:0/2763792480 shutdown_connections 2024-09-10T23:39:17.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.708+0000 7fc575000640 1 -- 172.21.15.94:0/2763792480 wait complete. 2024-09-10T23:39:17.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc575000640 1 Processor -- start 2024-09-10T23:39:17.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc575000640 1 -- start start 2024-09-10T23:39:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc575000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5700b2bb0 0x7fc5700c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc575000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5700d05e0 0x7fc5701f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc575000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc57015f5c0 0x7fc5701fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc575000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc570169200 con 0x7fc5700b2bb0 2024-09-10T23:39:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc575000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc570169080 con 0x7fc57015f5c0 2024-09-10T23:39:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc575000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc570169380 con 0x7fc5700d05e0 2024-09-10T23:39:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc56de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5700b2bb0 0x7fc5700c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc56de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5700b2bb0 0x7fc5700c81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41852/0 (socket says 172.21.15.94:41852) 2024-09-10T23:39:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc56de00640 1 -- 172.21.15.94:0/136996488 learned_addr learned my addr 172.21.15.94:0/136996488 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc565400640 1 --2- 172.21.15.94:0/136996488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5700d05e0 0x7fc5701f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:17.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc56e800640 1 --2- 172.21.15.94:0/136996488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc57015f5c0 0x7fc5701fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc56de00640 1 -- 172.21.15.94:0/136996488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5700d05e0 msgr2=0x7fc5701f58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc56de00640 1 --2- 172.21.15.94:0/136996488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5700d05e0 0x7fc5701f58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc56de00640 1 -- 172.21.15.94:0/136996488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc57015f5c0 msgr2=0x7fc5701fbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc56de00640 1 --2- 172.21.15.94:0/136996488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc57015f5c0 0x7fc5701fbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc56de00640 1 -- 172.21.15.94:0/136996488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc570165740 con 0x7fc5700b2bb0 2024-09-10T23:39:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc56de00640 1 --2- 172.21.15.94:0/136996488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5700b2bb0 0x7fc5700c81b0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fc56000e7d0 tx=0x7fc56000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc566a00640 1 -- 172.21.15.94:0/136996488 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56000ccc0 con 0x7fc5700b2bb0 2024-09-10T23:39:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc566a00640 1 -- 172.21.15.94:0/136996488 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc560010430 con 0x7fc5700b2bb0 2024-09-10T23:39:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc575000640 1 -- 172.21.15.94:0/136996488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5701530a0 con 0x7fc5700b2bb0 2024-09-10T23:39:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc566a00640 1 -- 172.21.15.94:0/136996488 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560010860 con 0x7fc5700b2bb0 2024-09-10T23:39:17.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.712+0000 7fc575000640 1 -- 172.21.15.94:0/136996488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc570153660 con 0x7fc5700b2bb0 2024-09-10T23:39:17.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.716+0000 7fc575000640 1 -- 172.21.15.94:0/136996488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5700b15c0 con 0x7fc5700b2bb0 2024-09-10T23:39:17.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.716+0000 7fc566a00640 1 -- 172.21.15.94:0/136996488 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7fc560010a00 con 0x7fc5700b2bb0 2024-09-10T23:39:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.716+0000 7fc566a00640 1 --2- 172.21.15.94:0/136996488 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc54c072cd0 0x7fc54c075190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.716+0000 7fc566a00640 1 -- 172.21.15.94:0/136996488 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4971+0+0 (secure 0 0 0) 0x7fc560094670 con 0x7fc5700b2bb0 2024-09-10T23:39:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.716+0000 7fc565400640 1 --2- 172.21.15.94:0/136996488 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc54c072cd0 0x7fc54c075190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:17.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.716+0000 7fc565400640 1 --2- 172.21.15.94:0/136996488 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc54c072cd0 0x7fc54c075190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc5580ae000 tx=0x7fc55801b000 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:17.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.720+0000 7fc566a00640 1 -- 172.21.15.94:0/136996488 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc56005ee70 con 0x7fc5700b2bb0 2024-09-10T23:39:17.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.888+0000 7fc575000640 1 -- 172.21.15.94:0/136996488 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc57015a410 con 0x7fc5700b2bb0 2024-09-10T23:39:17.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.888+0000 7fc566a00640 1 -- 172.21.15.94:0/136996488 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+13076 (secure 0 0 0) 0x7fc5600638a0 con 0x7fc5700b2bb0 2024-09-10T23:39:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:17.891 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":130,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:39:16.530656+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6801/2109170417":"2024-09-11T23:39:13.433144+0000","172.21.15.181:6800/2109170417":"2024-09-11T23:39:13.433144+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:39:17.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.892+0000 7fc575000640 1 -- 172.21.15.94:0/136996488 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc54c072cd0 msgr2=0x7fc54c075190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:17.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.892+0000 7fc575000640 1 --2- 172.21.15.94:0/136996488 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc54c072cd0 0x7fc54c075190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc5580ae000 tx=0x7fc55801b000 comp rx=0 tx=0).stop 2024-09-10T23:39:17.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.892+0000 7fc575000640 1 -- 172.21.15.94:0/136996488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5700b2bb0 msgr2=0x7fc5700c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:17.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.892+0000 7fc575000640 1 --2- 172.21.15.94:0/136996488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5700b2bb0 0x7fc5700c81b0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fc56000e7d0 tx=0x7fc56000ec90 comp rx=0 tx=0).stop 2024-09-10T23:39:17.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.892+0000 7fc575000640 1 -- 172.21.15.94:0/136996488 shutdown_connections 2024-09-10T23:39:17.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.892+0000 7fc575000640 1 --2- 172.21.15.94:0/136996488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc57015f5c0 0x7fc5701fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:17.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.892+0000 7fc575000640 1 --2- 172.21.15.94:0/136996488 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc54c072cd0 0x7fc54c075190 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:17.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.892+0000 7fc575000640 1 --2- 172.21.15.94:0/136996488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5700d05e0 0x7fc5701f58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:17.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.892+0000 7fc575000640 1 --2- 172.21.15.94:0/136996488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5700b2bb0 0x7fc5700c81b0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:17.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.892+0000 7fc575000640 1 -- 172.21.15.94:0/136996488 >> 172.21.15.94:0/136996488 conn(0x7fc5700d1c60 msgr2=0x7fc5700d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:17.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.892+0000 7fc575000640 1 -- 172.21.15.94:0/136996488 shutdown_connections 2024-09-10T23:39:17.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:17.892+0000 7fc575000640 1 -- 172.21.15.94:0/136996488 wait complete. 2024-09-10T23:39:17.914 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T23:32:31.372521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T23:39:17.914 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T23:39:18.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.056+0000 7f5bf3e00640 1 -- 172.21.15.94:0/2765016148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec0c49b0 msgr2=0x7f5bec0cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:18.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.056+0000 7f5bf3e00640 1 --2- 172.21.15.94:0/2765016148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec0c49b0 0x7f5bec0cadc0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009900 tx=0x7f5bd801c8e0 comp rx=0 tx=0).stop 2024-09-10T23:39:18.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.056+0000 7f5bf3e00640 1 -- 172.21.15.94:0/2765016148 shutdown_connections 2024-09-10T23:39:18.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.056+0000 7f5bf3e00640 1 --2- 172.21.15.94:0/2765016148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec0c49b0 0x7f5bec0cadc0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:18.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.056+0000 7f5bf3e00640 1 --2- 172.21.15.94:0/2765016148 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bec158790 0x7f5bec06fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:18.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.056+0000 7f5bf3e00640 1 -- 172.21.15.94:0/2765016148 >> 172.21.15.94:0/2765016148 conn(0x7f5bec0cfaf0 msgr2=0x7f5bec0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:18.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.056+0000 7f5bf3e00640 1 -- 172.21.15.94:0/2765016148 shutdown_connections 2024-09-10T23:39:18.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.056+0000 7f5bf3e00640 1 -- 172.21.15.94:0/2765016148 wait complete. 2024-09-10T23:39:18.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf3e00640 1 Processor -- start 2024-09-10T23:39:18.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf3e00640 1 -- start start 2024-09-10T23:39:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf3e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bec0c49b0 0x7f5bec14e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bec0cb300 0x7f5bec155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec158790 0x7f5bec0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf3e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec168290 con 0x7f5bec158790 2024-09-10T23:39:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf3e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec168110 con 0x7f5bec0cb300 2024-09-10T23:39:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf3e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5bec168410 con 0x7f5bec0c49b0 2024-09-10T23:39:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec158790 0x7f5bec0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec158790 0x7f5bec0b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41862/0 (socket says 172.21.15.94:41862) 2024-09-10T23:39:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf1e00640 1 -- 172.21.15.94:0/4021805315 learned_addr learned my addr 172.21.15.94:0/4021805315 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf1400640 1 --2- 172.21.15.94:0/4021805315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bec0c49b0 0x7f5bec14e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf0a00640 1 --2- 172.21.15.94:0/4021805315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bec0cb300 0x7f5bec155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf1e00640 1 -- 172.21.15.94:0/4021805315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bec0c49b0 msgr2=0x7f5bec14e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf1e00640 1 --2- 172.21.15.94:0/4021805315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bec0c49b0 0x7f5bec14e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf1e00640 1 -- 172.21.15.94:0/4021805315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bec0cb300 msgr2=0x7f5bec155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf1e00640 1 --2- 172.21.15.94:0/4021805315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bec0cb300 0x7f5bec155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf1e00640 1 -- 172.21.15.94:0/4021805315 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bec157bf0 con 0x7f5bec158790 2024-09-10T23:39:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf1e00640 1 --2- 172.21.15.94:0/4021805315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec158790 0x7f5bec0b0700 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f5be400c3a0 tx=0x7f5be400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5be9600640 1 -- 172.21.15.94:0/4021805315 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be40052d0 con 0x7f5bec158790 2024-09-10T23:39:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5be9600640 1 -- 172.21.15.94:0/4021805315 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5be4007020 con 0x7f5bec158790 2024-09-10T23:39:18.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5be9600640 1 -- 172.21.15.94:0/4021805315 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4007470 con 0x7f5bec158790 2024-09-10T23:39:18.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf3e00640 1 -- 172.21.15.94:0/4021805315 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bec157ed0 con 0x7f5bec158790 2024-09-10T23:39:18.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.060+0000 7f5bf3e00640 1 -- 172.21.15.94:0/4021805315 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bec15a6e0 con 0x7f5bec158790 2024-09-10T23:39:18.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.064+0000 7f5bf3e00640 1 -- 172.21.15.94:0/4021805315 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bec0cadc0 con 0x7f5bec158790 2024-09-10T23:39:18.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.064+0000 7f5be9600640 1 -- 172.21.15.94:0/4021805315 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f5be40067e0 con 0x7f5bec158790 2024-09-10T23:39:18.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.064+0000 7f5be9600640 1 --2- 172.21.15.94:0/4021805315 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bcc072ca0 0x7f5bcc075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:18.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.064+0000 7f5be9600640 1 -- 172.21.15.94:0/4021805315 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4971+0+0 (secure 0 0 0) 0x7f5be40930f0 con 0x7f5bec158790 2024-09-10T23:39:18.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.064+0000 7f5bf1400640 1 --2- 172.21.15.94:0/4021805315 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bcc072ca0 0x7f5bcc075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:18.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.064+0000 7f5bf1400640 1 --2- 172.21.15.94:0/4021805315 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bcc072ca0 0x7f5bcc075160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5bec15e680 tx=0x7f5be00095b0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:18.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.068+0000 7f5be9600640 1 -- 172.21.15.94:0/4021805315 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5be405d980 con 0x7f5bec158790 2024-09-10T23:39:18.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.236+0000 7f5bf3e00640 1 -- 172.21.15.94:0/4021805315 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5bec074840 con 0x7f5bec158790 2024-09-10T23:39:18.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.236+0000 7f5be9600640 1 -- 172.21.15.94:0/4021805315 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) ==== 93+0+10 (secure 0 0 0) 0x7f5be40623b0 con 0x7f5bec158790 2024-09-10T23:39:18.239 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-09-10T23:39:18.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.236+0000 7f5bf3e00640 1 -- 172.21.15.94:0/4021805315 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bcc072ca0 msgr2=0x7f5bcc075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:18.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.236+0000 7f5bf3e00640 1 --2- 172.21.15.94:0/4021805315 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bcc072ca0 0x7f5bcc075160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5bec15e680 tx=0x7f5be00095b0 comp rx=0 tx=0).stop 2024-09-10T23:39:18.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.236+0000 7f5bf3e00640 1 -- 172.21.15.94:0/4021805315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec158790 msgr2=0x7f5bec0b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:18.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.236+0000 7f5bf3e00640 1 --2- 172.21.15.94:0/4021805315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec158790 0x7f5bec0b0700 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f5be400c3a0 tx=0x7f5be400c860 comp rx=0 tx=0).stop 2024-09-10T23:39:18.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.240+0000 7f5bf3e00640 1 -- 172.21.15.94:0/4021805315 shutdown_connections 2024-09-10T23:39:18.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.240+0000 7f5bf3e00640 1 --2- 172.21.15.94:0/4021805315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bec0cb300 0x7f5bec155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:18.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.240+0000 7f5bf3e00640 1 --2- 172.21.15.94:0/4021805315 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5bcc072ca0 0x7f5bcc075160 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:18.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.240+0000 7f5bf3e00640 1 --2- 172.21.15.94:0/4021805315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec158790 0x7f5bec0b0700 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:18.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.240+0000 7f5bf3e00640 1 --2- 172.21.15.94:0/4021805315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bec0c49b0 0x7f5bec14e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:18.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.240+0000 7f5bf3e00640 1 -- 172.21.15.94:0/4021805315 >> 172.21.15.94:0/4021805315 conn(0x7f5bec0cfaf0 msgr2=0x7f5bec0ae500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:18.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.240+0000 7f5bf3e00640 1 -- 172.21.15.94:0/4021805315 shutdown_connections 2024-09-10T23:39:18.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.240+0000 7f5bf3e00640 1 -- 172.21.15.94:0/4021805315 wait complete. 2024-09-10T23:39:18.260 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-10T23:39:18.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.400+0000 7f9c34000640 1 -- 172.21.15.94:0/3508339504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c14003ab0 msgr2=0x7f9c14003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.400+0000 7f9c34000640 1 --2- 172.21.15.94:0/3508339504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c14003ab0 0x7f9c14003ed0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f9c20006fa0 tx=0x7f9c2001cf00 comp rx=0 tx=0).stop 2024-09-10T23:39:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.400+0000 7f9c34000640 1 -- 172.21.15.94:0/3508339504 shutdown_connections 2024-09-10T23:39:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.400+0000 7f9c34000640 1 --2- 172.21.15.94:0/3508339504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c14003ab0 0x7f9c14003ed0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.400+0000 7f9c34000640 1 -- 172.21.15.94:0/3508339504 >> 172.21.15.94:0/3508339504 conn(0x7f9c2c06cdf0 msgr2=0x7f9c2c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.400+0000 7f9c34000640 1 -- 172.21.15.94:0/3508339504 shutdown_connections 2024-09-10T23:39:18.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.400+0000 7f9c34000640 1 -- 172.21.15.94:0/3508339504 wait complete. 2024-09-10T23:39:18.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.400+0000 7f9c34000640 1 Processor -- start 2024-09-10T23:39:18.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.400+0000 7f9c34000640 1 -- start start 2024-09-10T23:39:18.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.400+0000 7f9c34000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c14003ab0 0x7f9c2c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:18.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.400+0000 7f9c34000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c2c148060 0x7f9c2c147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:18.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.400+0000 7f9c34000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c2c0ca650 0x7f9c2c0caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:18.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.400+0000 7f9c34000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c06f360 con 0x7f9c2c148060 2024-09-10T23:39:18.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.400+0000 7f9c34000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c06f1e0 con 0x7f9c2c0ca650 2024-09-10T23:39:18.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.400+0000 7f9c34000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9c2c15aa50 con 0x7f9c14003ab0 2024-09-10T23:39:18.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.400+0000 7f9c30c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c2c148060 0x7f9c2c147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:18.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c30c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c2c148060 0x7f9c2c147830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41874/0 (socket says 172.21.15.94:41874) 2024-09-10T23:39:18.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c31600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c14003ab0 0x7f9c2c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:18.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c30c00640 1 -- 172.21.15.94:0/2196160878 learned_addr learned my addr 172.21.15.94:0/2196160878 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:18.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c32000640 1 --2- 172.21.15.94:0/2196160878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c2c0ca650 0x7f9c2c0caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:18.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c30c00640 1 -- 172.21.15.94:0/2196160878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c14003ab0 msgr2=0x7f9c2c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:18.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c30c00640 1 --2- 172.21.15.94:0/2196160878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c14003ab0 0x7f9c2c0ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:18.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c30c00640 1 -- 172.21.15.94:0/2196160878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c2c0ca650 msgr2=0x7f9c2c0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:18.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c30c00640 1 --2- 172.21.15.94:0/2196160878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c2c0ca650 0x7f9c2c0caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:18.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c30c00640 1 -- 172.21.15.94:0/2196160878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2c1fbf40 con 0x7f9c2c148060 2024-09-10T23:39:18.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c30c00640 1 --2- 172.21.15.94:0/2196160878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c2c148060 0x7f9c2c147830 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800fbf0 tx=0x7f9c1800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:18.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c29600640 1 -- 172.21.15.94:0/2196160878 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18004460 con 0x7f9c2c148060 2024-09-10T23:39:18.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c29600640 1 -- 172.21.15.94:0/2196160878 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c180036f0 con 0x7f9c2c148060 2024-09-10T23:39:18.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c34000640 1 -- 172.21.15.94:0/2196160878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c2c1fc220 con 0x7f9c2c148060 2024-09-10T23:39:18.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c29600640 1 -- 172.21.15.94:0/2196160878 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18003b20 con 0x7f9c2c148060 2024-09-10T23:39:18.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c34000640 1 -- 172.21.15.94:0/2196160878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c2c151130 con 0x7f9c2c148060 2024-09-10T23:39:18.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c34000640 1 -- 172.21.15.94:0/2196160878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c2c15c130 con 0x7f9c2c148060 2024-09-10T23:39:18.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.404+0000 7f9c29600640 1 -- 172.21.15.94:0/2196160878 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f9c18020030 con 0x7f9c2c148060 2024-09-10T23:39:18.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.408+0000 7f9c29600640 1 --2- 172.21.15.94:0/2196160878 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9c0c072ca0 0x7f9c0c075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:18.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.408+0000 7f9c29600640 1 -- 172.21.15.94:0/2196160878 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4971+0+0 (secure 0 0 0) 0x7f9c18094bb0 con 0x7f9c2c148060 2024-09-10T23:39:18.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.408+0000 7f9c31600640 1 --2- 172.21.15.94:0/2196160878 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9c0c072ca0 0x7f9c0c075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:18.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.408+0000 7f9c31600640 1 --2- 172.21.15.94:0/2196160878 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9c0c072ca0 0x7f9c0c075160 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9c200ab000 tx=0x7f9c20019d40 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:18.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.412+0000 7f9c29600640 1 -- 172.21.15.94:0/2196160878 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9c1805fa00 con 0x7f9c2c148060 2024-09-10T23:39:18.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:18.636+0000 7f9c34000640 1 -- 172.21.15.94:0/2196160878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7f9c2c06e7e0 con 0x7f9c2c148060 2024-09-10T23:39:19.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.552+0000 7f9c29600640 1 -- 172.21.15.94:0/2196160878 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v374) ==== 152+0+0 (secure 0 0 0) 0x7f9c1805f300 con 0x7f9c2c148060 2024-09-10T23:39:19.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.552+0000 7f9c34000640 1 -- 172.21.15.94:0/2196160878 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9c0c072ca0 msgr2=0x7f9c0c075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.552+0000 7f9c34000640 1 --2- 172.21.15.94:0/2196160878 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9c0c072ca0 0x7f9c0c075160 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9c200ab000 tx=0x7f9c20019d40 comp rx=0 tx=0).stop 2024-09-10T23:39:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.552+0000 7f9c34000640 1 -- 172.21.15.94:0/2196160878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c2c148060 msgr2=0x7f9c2c147830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.552+0000 7f9c34000640 1 --2- 172.21.15.94:0/2196160878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c2c148060 0x7f9c2c147830 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800fbf0 tx=0x7f9c1800c7a0 comp rx=0 tx=0).stop 2024-09-10T23:39:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.556+0000 7f9c34000640 1 -- 172.21.15.94:0/2196160878 shutdown_connections 2024-09-10T23:39:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.556+0000 7f9c34000640 1 --2- 172.21.15.94:0/2196160878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c2c0ca650 0x7f9c2c0caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.556+0000 7f9c34000640 1 --2- 172.21.15.94:0/2196160878 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9c0c072ca0 0x7f9c0c075160 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.556+0000 7f9c34000640 1 --2- 172.21.15.94:0/2196160878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c2c148060 0x7f9c2c147830 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.556+0000 7f9c34000640 1 --2- 172.21.15.94:0/2196160878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c14003ab0 0x7f9c2c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.556+0000 7f9c34000640 1 -- 172.21.15.94:0/2196160878 >> 172.21.15.94:0/2196160878 conn(0x7f9c2c06cdf0 msgr2=0x7f9c2c155590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.556+0000 7f9c34000640 1 -- 172.21.15.94:0/2196160878 shutdown_connections 2024-09-10T23:39:19.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.556+0000 7f9c34000640 1 -- 172.21.15.94:0/2196160878 wait complete. 2024-09-10T23:39:19.578 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T23:39:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7af800640 1 -- 172.21.15.94:0/2937325281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7a815bd30 msgr2=0x7ff7a815c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7af800640 1 --2- 172.21.15.94:0/2937325281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7a815bd30 0x7ff7a815c110 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff798009870 tx=0x7ff79801c860 comp rx=0 tx=0).stop 2024-09-10T23:39:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7af800640 1 -- 172.21.15.94:0/2937325281 shutdown_connections 2024-09-10T23:39:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7af800640 1 --2- 172.21.15.94:0/2937325281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7a815bd30 0x7ff7a815c110 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:19.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7af800640 1 -- 172.21.15.94:0/2937325281 >> 172.21.15.94:0/2937325281 conn(0x7ff7a809aa70 msgr2=0x7ff7a809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7af800640 1 -- 172.21.15.94:0/2937325281 shutdown_connections 2024-09-10T23:39:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7af800640 1 -- 172.21.15.94:0/2937325281 wait complete. 2024-09-10T23:39:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7af800640 1 Processor -- start 2024-09-10T23:39:19.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7af800640 1 -- start start 2024-09-10T23:39:19.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7af800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7a80ae980 0x7ff7a80cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7af800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7a80d2290 0x7ff7a8144370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7af800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7a815bd30 0x7ff7a815e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7af800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a816a5c0 con 0x7ff7a815bd30 2024-09-10T23:39:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7af800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a816a440 con 0x7ff7a80ae980 2024-09-10T23:39:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7af800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff7a816a740 con 0x7ff7a80d2290 2024-09-10T23:39:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7ace00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7a80ae980 0x7ff7a80cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7ad800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7a815bd30 0x7ff7a815e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7ace00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7a80ae980 0x7ff7a80cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:39016/0 (socket says 172.21.15.94:39016) 2024-09-10T23:39:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7ace00640 1 -- 172.21.15.94:0/1783293766 learned_addr learned my addr 172.21.15.94:0/1783293766 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff79fe00640 1 --2- 172.21.15.94:0/1783293766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7a80d2290 0x7ff7a8144370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7ace00640 1 -- 172.21.15.94:0/1783293766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7a80d2290 msgr2=0x7ff7a8144370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7ace00640 1 --2- 172.21.15.94:0/1783293766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7a80d2290 0x7ff7a8144370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:19.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7ace00640 1 -- 172.21.15.94:0/1783293766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7a815bd30 msgr2=0x7ff7a815e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7ace00640 1 --2- 172.21.15.94:0/1783293766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7a815bd30 0x7ff7a815e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff7ace00640 1 -- 172.21.15.94:0/1783293766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a80c88a0 con 0x7ff7a80ae980 2024-09-10T23:39:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.720+0000 7ff79fe00640 1 --2- 172.21.15.94:0/1783293766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7a80d2290 0x7ff7a8144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.724+0000 7ff7ace00640 1 --2- 172.21.15.94:0/1783293766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7a80ae980 0x7ff7a80cc430 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff798019470 tx=0x7ff798005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.724+0000 7ff7a5600640 1 -- 172.21.15.94:0/1783293766 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980b2040 con 0x7ff7a80ae980 2024-09-10T23:39:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.724+0000 7ff7a5600640 1 -- 172.21.15.94:0/1783293766 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff798004040 con 0x7ff7a80ae980 2024-09-10T23:39:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.724+0000 7ff7a5600640 1 -- 172.21.15.94:0/1783293766 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798004430 con 0x7ff7a80ae980 2024-09-10T23:39:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.724+0000 7ff7af800640 1 -- 172.21.15.94:0/1783293766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a80b1080 con 0x7ff7a80ae980 2024-09-10T23:39:19.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.724+0000 7ff7af800640 1 -- 172.21.15.94:0/1783293766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a80cef40 con 0x7ff7a80ae980 2024-09-10T23:39:19.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.724+0000 7ff7af800640 1 -- 172.21.15.94:0/1783293766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a815c110 con 0x7ff7a80ae980 2024-09-10T23:39:19.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.724+0000 7ff7a5600640 1 -- 172.21.15.94:0/1783293766 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7ff7980b5070 con 0x7ff7a80ae980 2024-09-10T23:39:19.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.724+0000 7ff7a5600640 1 --2- 172.21.15.94:0/1783293766 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff784072ca0 0x7ff784075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:19.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.724+0000 7ff7a5600640 1 -- 172.21.15.94:0/1783293766 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4971+0+0 (secure 0 0 0) 0x7ff79812da60 con 0x7ff7a80ae980 2024-09-10T23:39:19.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.724+0000 7ff79fe00640 1 --2- 172.21.15.94:0/1783293766 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff784072ca0 0x7ff784075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:19.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.728+0000 7ff79fe00640 1 --2- 172.21.15.94:0/1783293766 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff784072ca0 0x7ff784075160 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8147980 tx=0x7ff790009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:19.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.728+0000 7ff7a5600640 1 -- 172.21.15.94:0/1783293766 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff7980c4020 con 0x7ff7a80ae980 2024-09-10T23:39:19.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.900+0000 7ff7af800640 1 -- 172.21.15.94:0/1783293766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7ff7a80d28a0 con 0x7ff7a80ae980 2024-09-10T23:39:19.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.904+0000 7ff7a5600640 1 -- 172.21.15.94:0/1783293766 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v43) ==== 59+0+1998 (secure 0 0 0) 0x7ff798016450 con 0x7ff7a80ae980 2024-09-10T23:39:19.905 INFO:teuthology.orchestra.run.smithi094.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T23:39:19.905 INFO:teuthology.orchestra.run.smithi094.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T23:39:19.905 INFO:teuthology.orchestra.run.smithi094.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T23:39:19.905 INFO:teuthology.orchestra.run.smithi094.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T23:39:19.905 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3944.074506 2024-09-10T23:39:19.906 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3865.464527 2024-09-10T23:39:19.906 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3898.861024 2024-09-10T23:39:19.906 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4010.048529 2024-09-10T23:39:19.906 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4650.562436 2024-09-10T23:39:19.906 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4724.018776 2024-09-10T23:39:19.906 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5255.753663 2024-09-10T23:39:19.906 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4718.009443 2024-09-10T23:39:19.906 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds 20/20 2024-09-10T23:39:19.906 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T23:39:19.906 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_ms 2/2 2024-09-10T23:39:19.906 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_frag true 2024-09-10T23:39:19.906 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_scatterstat true 2024-09-10T23:39:19.906 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T23:39:19.906 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_verify_scatter true 2024-09-10T23:39:19.907 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T23:39:19.907 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T23:39:19.907 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T23:39:19.907 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced client_mount_timeout 600 2024-09-10T23:39:19.907 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_client 20/20 2024-09-10T23:39:19.907 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_ms 1/1 2024-09-10T23:39:19.907 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T23:39:19.907 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T23:39:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.904+0000 7ff7af800640 1 -- 172.21.15.94:0/1783293766 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff784072ca0 msgr2=0x7ff784075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.904+0000 7ff7af800640 1 --2- 172.21.15.94:0/1783293766 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff784072ca0 0x7ff784075160 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8147980 tx=0x7ff790009040 comp rx=0 tx=0).stop 2024-09-10T23:39:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.904+0000 7ff7af800640 1 -- 172.21.15.94:0/1783293766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7a80ae980 msgr2=0x7ff7a80cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.904+0000 7ff7af800640 1 --2- 172.21.15.94:0/1783293766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7a80ae980 0x7ff7a80cc430 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff798019470 tx=0x7ff798005780 comp rx=0 tx=0).stop 2024-09-10T23:39:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.904+0000 7ff7af800640 1 -- 172.21.15.94:0/1783293766 shutdown_connections 2024-09-10T23:39:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.904+0000 7ff7af800640 1 --2- 172.21.15.94:0/1783293766 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff784072ca0 0x7ff784075160 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.904+0000 7ff7af800640 1 --2- 172.21.15.94:0/1783293766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7a815bd30 0x7ff7a815e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.904+0000 7ff7af800640 1 --2- 172.21.15.94:0/1783293766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7a80d2290 0x7ff7a8144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.904+0000 7ff7af800640 1 --2- 172.21.15.94:0/1783293766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7a80ae980 0x7ff7a80cc430 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.904+0000 7ff7af800640 1 -- 172.21.15.94:0/1783293766 >> 172.21.15.94:0/1783293766 conn(0x7ff7a809aa70 msgr2=0x7ff7a8070c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.904+0000 7ff7af800640 1 -- 172.21.15.94:0/1783293766 shutdown_connections 2024-09-10T23:39:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:19.904+0000 7ff7af800640 1 -- 172.21.15.94:0/1783293766 wait complete. 2024-09-10T23:39:19.928 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:39:20.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.068+0000 7f4f47400640 1 -- 172.21.15.94:0/1564694015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f400d2780 msgr2=0x7f4f400d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:20.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.068+0000 7f4f47400640 1 --2- 172.21.15.94:0/1564694015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f400d2780 0x7f4f400d03f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c009930 tx=0x7f4f2c01c850 comp rx=0 tx=0).stop 2024-09-10T23:39:20.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.068+0000 7f4f47400640 1 -- 172.21.15.94:0/1564694015 shutdown_connections 2024-09-10T23:39:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.068+0000 7f4f47400640 1 --2- 172.21.15.94:0/1564694015 >> v2:172.21.15.181:3300/0 conn(0x7f4f400c5e50 0x7f4f400c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.068+0000 7f4f47400640 1 --2- 172.21.15.94:0/1564694015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f400d2780 0x7f4f400d03f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.068+0000 7f4f47400640 1 -- 172.21.15.94:0/1564694015 >> 172.21.15.94:0/1564694015 conn(0x7f4f400d1c60 msgr2=0x7f4f400d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.068+0000 7f4f47400640 1 -- 172.21.15.94:0/1564694015 shutdown_connections 2024-09-10T23:39:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.068+0000 7f4f47400640 1 -- 172.21.15.94:0/1564694015 wait complete. 2024-09-10T23:39:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f47400640 1 Processor -- start 2024-09-10T23:39:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f47400640 1 -- start start 2024-09-10T23:39:20.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f47400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f400c5e50 0x7f4f401f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:20.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f47400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f400d2780 0x7f4f401f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:20.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f47400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f4015bf60 0x7f4f401f8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:20.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f47400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40168f20 con 0x7f4f400c5e50 2024-09-10T23:39:20.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f47400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40168da0 con 0x7f4f4015bf60 2024-09-10T23:39:20.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f47400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f4f401690a0 con 0x7f4f400d2780 2024-09-10T23:39:20.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f44a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f400c5e50 0x7f4f401f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:20.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f44a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f400c5e50 0x7f4f401f7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41892/0 (socket says 172.21.15.94:41892) 2024-09-10T23:39:20.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f44a00640 1 -- 172.21.15.94:0/1733198019 learned_addr learned my addr 172.21.15.94:0/1733198019 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:20.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f3fe00640 1 --2- 172.21.15.94:0/1733198019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f400d2780 0x7f4f401f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:20.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f45400640 1 --2- 172.21.15.94:0/1733198019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f4015bf60 0x7f4f401f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:20.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f44a00640 1 -- 172.21.15.94:0/1733198019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f400d2780 msgr2=0x7f4f401f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:20.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f44a00640 1 --2- 172.21.15.94:0/1733198019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f400d2780 0x7f4f401f81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:20.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f44a00640 1 -- 172.21.15.94:0/1733198019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f4015bf60 msgr2=0x7f4f401f8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:20.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f44a00640 1 --2- 172.21.15.94:0/1733198019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f4015bf60 0x7f4f401f8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:20.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f44a00640 1 -- 172.21.15.94:0/1733198019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f400b4ab0 con 0x7f4f400c5e50 2024-09-10T23:39:20.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f44a00640 1 --2- 172.21.15.94:0/1733198019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f400c5e50 0x7f4f401f7cb0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000c770 tx=0x7f4f3000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:20.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f3cc00640 1 -- 172.21.15.94:0/1733198019 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30003b80 con 0x7f4f400c5e50 2024-09-10T23:39:20.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f3cc00640 1 -- 172.21.15.94:0/1733198019 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f30003d40 con 0x7f4f400c5e50 2024-09-10T23:39:20.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f3cc00640 1 -- 172.21.15.94:0/1733198019 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30005330 con 0x7f4f400c5e50 2024-09-10T23:39:20.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f47400640 1 -- 172.21.15.94:0/1733198019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f400b4d90 con 0x7f4f400c5e50 2024-09-10T23:39:20.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.072+0000 7f4f47400640 1 -- 172.21.15.94:0/1733198019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4015ab60 con 0x7f4f400c5e50 2024-09-10T23:39:20.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.076+0000 7f4f47400640 1 -- 172.21.15.94:0/1733198019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f400c6300 con 0x7f4f400c5e50 2024-09-10T23:39:20.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.076+0000 7f4f3cc00640 1 -- 172.21.15.94:0/1733198019 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f4f300054d0 con 0x7f4f400c5e50 2024-09-10T23:39:20.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.076+0000 7f4f3cc00640 1 --2- 172.21.15.94:0/1733198019 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4f20072ca0 0x7f4f20075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:20.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.076+0000 7f4f3cc00640 1 -- 172.21.15.94:0/1733198019 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4971+0+0 (secure 0 0 0) 0x7f4f30093790 con 0x7f4f400c5e50 2024-09-10T23:39:20.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.076+0000 7f4f3fe00640 1 --2- 172.21.15.94:0/1733198019 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4f20072ca0 0x7f4f20075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:20.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.076+0000 7f4f3fe00640 1 --2- 172.21.15.94:0/1733198019 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4f20072ca0 0x7f4f20075160 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0af000 tx=0x7f4f2c0a7040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:20.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.080+0000 7f4f3cc00640 1 -- 172.21.15.94:0/1733198019 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4f3005e800 con 0x7f4f400c5e50 2024-09-10T23:39:20.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.244+0000 7f4f47400640 1 -- 172.21.15.94:0/1733198019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4f400d2320 con 0x7f4f400c5e50 2024-09-10T23:39:20.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.248+0000 7f4f3cc00640 1 -- 172.21.15.94:0/1733198019 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7f4f3005e100 con 0x7f4f400c5e50 2024-09-10T23:39:20.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.252+0000 7f4f47400640 1 -- 172.21.15.94:0/1733198019 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4f20072ca0 msgr2=0x7f4f20075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:20.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.252+0000 7f4f47400640 1 --2- 172.21.15.94:0/1733198019 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4f20072ca0 0x7f4f20075160 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0af000 tx=0x7f4f2c0a7040 comp rx=0 tx=0).stop 2024-09-10T23:39:20.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.252+0000 7f4f47400640 1 -- 172.21.15.94:0/1733198019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f400c5e50 msgr2=0x7f4f401f7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:20.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.252+0000 7f4f47400640 1 --2- 172.21.15.94:0/1733198019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f400c5e50 0x7f4f401f7cb0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000c770 tx=0x7f4f3000cb70 comp rx=0 tx=0).stop 2024-09-10T23:39:20.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.252+0000 7f4f47400640 1 -- 172.21.15.94:0/1733198019 shutdown_connections 2024-09-10T23:39:20.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.252+0000 7f4f47400640 1 --2- 172.21.15.94:0/1733198019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f4015bf60 0x7f4f401f8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:20.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.252+0000 7f4f47400640 1 --2- 172.21.15.94:0/1733198019 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4f20072ca0 0x7f4f20075160 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:20.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.252+0000 7f4f47400640 1 --2- 172.21.15.94:0/1733198019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f400d2780 0x7f4f401f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:20.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.252+0000 7f4f47400640 1 --2- 172.21.15.94:0/1733198019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f400c5e50 0x7f4f401f7cb0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:20.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.252+0000 7f4f47400640 1 -- 172.21.15.94:0/1733198019 >> 172.21.15.94:0/1733198019 conn(0x7f4f400d1c60 msgr2=0x7f4f400717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:20.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.252+0000 7f4f47400640 1 -- 172.21.15.94:0/1733198019 shutdown_connections 2024-09-10T23:39:20.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.252+0000 7f4f47400640 1 -- 172.21.15.94:0/1733198019 wait complete. 2024-09-10T23:39:20.275 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:39:20.275 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:39:20.383 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:39:20.471 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:39:20.471 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:39:20.471 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:20.480 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:39:20.480 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:39:20.480 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:20.489 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:39:20.490 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:39:20.490 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:20.529 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:39:20.529 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:39:20.529 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:20.540 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:39:20.540 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T23:39:20.541 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:39:20.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.712+0000 7fe0baa00640 1 -- 172.21.15.94:0/1532053799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe098004810 msgr2=0x7fe098004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:20.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.712+0000 7fe0baa00640 1 --2- 172.21.15.94:0/1532053799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe098004810 0x7fe098004c90 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00c310 tx=0x7fe09c01f6c0 comp rx=0 tx=0).stop 2024-09-10T23:39:20.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.712+0000 7fe0baa00640 1 -- 172.21.15.94:0/1532053799 shutdown_connections 2024-09-10T23:39:20.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.712+0000 7fe0baa00640 1 --2- 172.21.15.94:0/1532053799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe098004810 0x7fe098004c90 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:20.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.712+0000 7fe0baa00640 1 -- 172.21.15.94:0/1532053799 >> 172.21.15.94:0/1532053799 conn(0x7fe0b406cdf0 msgr2=0x7fe0b406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:20.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.712+0000 7fe0baa00640 1 -- 172.21.15.94:0/1532053799 shutdown_connections 2024-09-10T23:39:20.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.712+0000 7fe0baa00640 1 -- 172.21.15.94:0/1532053799 wait complete. 2024-09-10T23:39:20.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.712+0000 7fe0baa00640 1 Processor -- start 2024-09-10T23:39:20.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.712+0000 7fe0baa00640 1 -- start start 2024-09-10T23:39:20.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0baa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe098004810 0x7fe0b41f8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:20.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0baa00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0b41f19c0 0x7fe0b40ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:20.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0baa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0b415c180 0x7fe0b415c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:20.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0baa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b40cff20 con 0x7fe0b415c180 2024-09-10T23:39:20.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0baa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b40cfda0 con 0x7fe098004810 2024-09-10T23:39:20.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0baa00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe0b40d00a0 con 0x7fe0b41f19c0 2024-09-10T23:39:20.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0b8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0b415c180 0x7fe0b415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:20.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0b8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0b415c180 0x7fe0b415c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41916/0 (socket says 172.21.15.94:41916) 2024-09-10T23:39:20.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0b8a00640 1 -- 172.21.15.94:0/2462527464 learned_addr learned my addr 172.21.15.94:0/2462527464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:20.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0b3e00640 1 --2- 172.21.15.94:0/2462527464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe098004810 0x7fe0b41f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:20.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0ab400640 1 --2- 172.21.15.94:0/2462527464 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0b41f19c0 0x7fe0b40ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:20.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0b8a00640 1 -- 172.21.15.94:0/2462527464 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0b41f19c0 msgr2=0x7fe0b40ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:20.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0b8a00640 1 --2- 172.21.15.94:0/2462527464 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0b41f19c0 0x7fe0b40ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:20.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0b8a00640 1 -- 172.21.15.94:0/2462527464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe098004810 msgr2=0x7fe0b41f8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:20.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0b8a00640 1 --2- 172.21.15.94:0/2462527464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe098004810 0x7fe0b41f8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:20.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0b8a00640 1 -- 172.21.15.94:0/2462527464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b41f8ab0 con 0x7fe0b415c180 2024-09-10T23:39:20.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0b8a00640 1 --2- 172.21.15.94:0/2462527464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0b415c180 0x7fe0b415c630 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac006fd0 tx=0x7fe0ac001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:20.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0b0c00640 1 -- 172.21.15.94:0/2462527464 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac011430 con 0x7fe0b415c180 2024-09-10T23:39:20.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0b0c00640 1 -- 172.21.15.94:0/2462527464 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0ac0115d0 con 0x7fe0b415c180 2024-09-10T23:39:20.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0b0c00640 1 -- 172.21.15.94:0/2462527464 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac011a00 con 0x7fe0b415c180 2024-09-10T23:39:20.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0baa00640 1 -- 172.21.15.94:0/2462527464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b4146530 con 0x7fe0b415c180 2024-09-10T23:39:20.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0baa00640 1 -- 172.21.15.94:0/2462527464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b4146af0 con 0x7fe0b415c180 2024-09-10T23:39:20.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0baa00640 1 -- 172.21.15.94:0/2462527464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0b41f4630 con 0x7fe0b415c180 2024-09-10T23:39:20.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.716+0000 7fe0b0c00640 1 -- 172.21.15.94:0/2462527464 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7fe0ac011ba0 con 0x7fe0b415c180 2024-09-10T23:39:20.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.720+0000 7fe0b0c00640 1 --2- 172.21.15.94:0/2462527464 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe090072ca0 0x7fe090075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:20.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.720+0000 7fe0b0c00640 1 -- 172.21.15.94:0/2462527464 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4971+0+0 (secure 0 0 0) 0x7fe0ac0942b0 con 0x7fe0b415c180 2024-09-10T23:39:20.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.720+0000 7fe0b3e00640 1 --2- 172.21.15.94:0/2462527464 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe090072ca0 0x7fe090075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:20.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.720+0000 7fe0b3e00640 1 --2- 172.21.15.94:0/2462527464 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe090072ca0 0x7fe090075160 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe0b406b7e0 tx=0x7fe0a4009500 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:20.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.724+0000 7fe0b0c00640 1 -- 172.21.15.94:0/2462527464 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe0b41f4630 con 0x7fe0b415c180 2024-09-10T23:39:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.920+0000 7fe0baa00640 1 -- 172.21.15.94:0/2462527464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0b415eaf0 con 0x7fe0b415c180 2024-09-10T23:39:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.924+0000 7fe0b0c00640 1 -- 172.21.15.94:0/2462527464 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+3908 (secure 0 0 0) 0x7fe0b41f8ab0 con 0x7fe0b415c180 2024-09-10T23:39:20.926 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:20.926 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":91,"btime":"2024-09-10T23:39:14:435443+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3851553379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3851553379},{"type":"v1","addr":"172.21.15.181:6835","nonce":3851553379}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7768,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/1681326159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1681326159},{"type":"v1","addr":"172.21.15.181:6801","nonce":1681326159}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":7923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/90080224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":90080224},{"type":"v1","addr":"172.21.15.94:6833","nonce":90080224}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82},{"gid":7929,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/334167733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":334167733},{"type":"v1","addr":"172.21.15.94:6835","nonce":334167733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[]} 2024-09-10T23:39:20.926 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 91 2024-09-10T23:39:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.924+0000 7fe0baa00640 1 -- 172.21.15.94:0/2462527464 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe090072ca0 msgr2=0x7fe090075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.924+0000 7fe0baa00640 1 --2- 172.21.15.94:0/2462527464 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe090072ca0 0x7fe090075160 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe0b406b7e0 tx=0x7fe0a4009500 comp rx=0 tx=0).stop 2024-09-10T23:39:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.924+0000 7fe0baa00640 1 -- 172.21.15.94:0/2462527464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0b415c180 msgr2=0x7fe0b415c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.924+0000 7fe0baa00640 1 --2- 172.21.15.94:0/2462527464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0b415c180 0x7fe0b415c630 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac006fd0 tx=0x7fe0ac001640 comp rx=0 tx=0).stop 2024-09-10T23:39:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.924+0000 7fe0baa00640 1 -- 172.21.15.94:0/2462527464 shutdown_connections 2024-09-10T23:39:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.924+0000 7fe0baa00640 1 --2- 172.21.15.94:0/2462527464 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe090072ca0 0x7fe090075160 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.924+0000 7fe0baa00640 1 --2- 172.21.15.94:0/2462527464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0b415c180 0x7fe0b415c630 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.924+0000 7fe0baa00640 1 --2- 172.21.15.94:0/2462527464 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0b41f19c0 0x7fe0b40ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.924+0000 7fe0baa00640 1 --2- 172.21.15.94:0/2462527464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe098004810 0x7fe0b41f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:20.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.924+0000 7fe0baa00640 1 -- 172.21.15.94:0/2462527464 >> 172.21.15.94:0/2462527464 conn(0x7fe0b406cdf0 msgr2=0x7fe0b4072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:20.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.924+0000 7fe0baa00640 1 -- 172.21.15.94:0/2462527464 shutdown_connections 2024-09-10T23:39:20.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:20.924+0000 7fe0baa00640 1 -- 172.21.15.94:0/2462527464 wait complete. 2024-09-10T23:39:20.947 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T23:39:20.947 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T23:39:20.947 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T23:39:20.947 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:39:20.948 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T23:39:20.948 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T23:39:20.948 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T23:39:20.948 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:39:20.948 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T23:39:20.948 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T23:39:20.949 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T23:39:20.949 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:39:20.949 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T23:39:20.949 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T23:39:20.949 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T23:39:20.949 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:39:20.960 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T23:39:20.961 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T23:39:20.962 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T23:39:20.962 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T23:39:20.969 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T23:39:20.969 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T23:39:20.970 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T23:39:20.970 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T23:39:21.007 INFO:tasks.ceph.mds.a:Started 2024-09-10T23:39:21.008 DEBUG:teuthology.parallel:result is None 2024-09-10T23:39:21.009 INFO:tasks.ceph.mds.c:Started 2024-09-10T23:39:21.009 DEBUG:teuthology.parallel:result is None 2024-09-10T23:39:21.019 INFO:tasks.ceph.mds.d:Started 2024-09-10T23:39:21.020 DEBUG:teuthology.parallel:result is None 2024-09-10T23:39:21.021 INFO:tasks.ceph.mds.b:Started 2024-09-10T23:39:21.022 DEBUG:teuthology.parallel:result is None 2024-09-10T23:39:21.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T23:39:21.094 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-09-10T23:39:21.094 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-09-10T23:39:21.112 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:39:21.113 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2024-09-10T23:39:21.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.168+0000 7f0127e00640 1 -- 172.21.15.94:0/1643188774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01200ce1e0 msgr2=0x7f01200cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:21.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.168+0000 7f0127e00640 1 --2- 172.21.15.94:0/1643188774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01200ce1e0 0x7f01200cd710 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f01200d18b0 tx=0x7f010c01c830 comp rx=0 tx=0).stop 2024-09-10T23:39:21.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.168+0000 7f0127e00640 1 -- 172.21.15.94:0/1643188774 shutdown_connections 2024-09-10T23:39:21.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.168+0000 7f0127e00640 1 --2- 172.21.15.94:0/1643188774 >> v2:172.21.15.181:3300/0 conn(0x7f01200b1100 0x7f01200b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.168+0000 7f0127e00640 1 --2- 172.21.15.94:0/1643188774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01200ce1e0 0x7f01200cd710 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.168+0000 7f0127e00640 1 -- 172.21.15.94:0/1643188774 >> 172.21.15.94:0/1643188774 conn(0x7f01200d1c60 msgr2=0x7f01200d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:21.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.168+0000 7f0127e00640 1 -- 172.21.15.94:0/1643188774 shutdown_connections 2024-09-10T23:39:21.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.168+0000 7f0127e00640 1 -- 172.21.15.94:0/1643188774 wait complete. 2024-09-10T23:39:21.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0127e00640 1 Processor -- start 2024-09-10T23:39:21.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0127e00640 1 -- start start 2024-09-10T23:39:21.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0127e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01200b1100 0x7f0120153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:21.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0127e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01200ce1e0 0x7f01201f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:21.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0127e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01200ce890 0x7f012015f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:21.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0127e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f01200d0b30 con 0x7f01200b1100 2024-09-10T23:39:21.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0127e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f01200d09b0 con 0x7f01200ce1e0 2024-09-10T23:39:21.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0127e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0120169110 con 0x7f01200ce890 2024-09-10T23:39:21.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0125400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01200b1100 0x7f0120153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:21.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0125400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01200b1100 0x7f0120153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41966/0 (socket says 172.21.15.94:41966) 2024-09-10T23:39:21.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0125400640 1 -- 172.21.15.94:0/155408158 learned_addr learned my addr 172.21.15.94:0/155408158 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:21.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0124a00640 1 --2- 172.21.15.94:0/155408158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01200ce1e0 0x7f01201f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:21.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0125e00640 1 --2- 172.21.15.94:0/155408158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01200ce890 0x7f012015f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:21.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0125400640 1 -- 172.21.15.94:0/155408158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01200ce890 msgr2=0x7f012015f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:21.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0125400640 1 --2- 172.21.15.94:0/155408158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01200ce890 0x7f012015f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0125400640 1 -- 172.21.15.94:0/155408158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01200ce1e0 msgr2=0x7f01201f6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:21.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0125400640 1 --2- 172.21.15.94:0/155408158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01200ce1e0 0x7f01201f6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0125400640 1 -- 172.21.15.94:0/155408158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012015f810 con 0x7f01200b1100 2024-09-10T23:39:21.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0125400640 1 --2- 172.21.15.94:0/155408158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01200b1100 0x7f0120153550 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f011400b3e0 tx=0x7f011400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:21.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f011d600640 1 -- 172.21.15.94:0/155408158 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114015020 con 0x7f01200b1100 2024-09-10T23:39:21.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f011d600640 1 -- 172.21.15.94:0/155408158 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0114011940 con 0x7f01200b1100 2024-09-10T23:39:21.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0127e00640 1 -- 172.21.15.94:0/155408158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01201f8c20 con 0x7f01200b1100 2024-09-10T23:39:21.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f011d600640 1 -- 172.21.15.94:0/155408158 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114011d90 con 0x7f01200b1100 2024-09-10T23:39:21.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0127e00640 1 -- 172.21.15.94:0/155408158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01201f91e0 con 0x7f01200b1100 2024-09-10T23:39:21.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.172+0000 7f0127e00640 1 -- 172.21.15.94:0/155408158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01200b1590 con 0x7f01200b1100 2024-09-10T23:39:21.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.176+0000 7f011d600640 1 -- 172.21.15.94:0/155408158 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f011401a400 con 0x7f01200b1100 2024-09-10T23:39:21.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.176+0000 7f011d600640 1 --2- 172.21.15.94:0/155408158 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0100072ca0 0x7f0100075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:21.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.176+0000 7f011d600640 1 -- 172.21.15.94:0/155408158 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4971+0+0 (secure 0 0 0) 0x7f0114093e00 con 0x7f01200b1100 2024-09-10T23:39:21.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.176+0000 7f0124a00640 1 --2- 172.21.15.94:0/155408158 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0100072ca0 0x7f0100075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:21.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.176+0000 7f0124a00640 1 --2- 172.21.15.94:0/155408158 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0100072ca0 0x7f0100075160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f010c0af000 tx=0x7f010c01b070 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:21.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.180+0000 7f011d600640 1 -- 172.21.15.94:0/155408158 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f011401f020 con 0x7f01200b1100 2024-09-10T23:39:21.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.356+0000 7f0127e00640 1 -- 172.21.15.94:0/155408158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f01200b0270 con 0x7f01200b1100 2024-09-10T23:39:21.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.564+0000 7f011d600640 1 -- 172.21.15.94:0/155408158 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v132) ==== 97+0+0 (secure 0 0 0) 0x7f011405e690 con 0x7f01200b1100 2024-09-10T23:39:21.568 INFO:teuthology.orchestra.run.smithi094.stderr: removed all blocklist entries 2024-09-10T23:39:21.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.568+0000 7f0127e00640 1 -- 172.21.15.94:0/155408158 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0100072ca0 msgr2=0x7f0100075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:21.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.568+0000 7f0127e00640 1 --2- 172.21.15.94:0/155408158 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0100072ca0 0x7f0100075160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f010c0af000 tx=0x7f010c01b070 comp rx=0 tx=0).stop 2024-09-10T23:39:21.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.568+0000 7f0127e00640 1 -- 172.21.15.94:0/155408158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01200b1100 msgr2=0x7f0120153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:21.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.568+0000 7f0127e00640 1 --2- 172.21.15.94:0/155408158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01200b1100 0x7f0120153550 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f011400b3e0 tx=0x7f011400b8a0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.568+0000 7f0127e00640 1 -- 172.21.15.94:0/155408158 shutdown_connections 2024-09-10T23:39:21.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.568+0000 7f0127e00640 1 --2- 172.21.15.94:0/155408158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01200ce1e0 0x7f01201f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.568+0000 7f0127e00640 1 --2- 172.21.15.94:0/155408158 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0100072ca0 0x7f0100075160 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.568+0000 7f0127e00640 1 --2- 172.21.15.94:0/155408158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01200ce890 0x7f012015f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.568+0000 7f0127e00640 1 --2- 172.21.15.94:0/155408158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01200b1100 0x7f0120153550 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.568+0000 7f0127e00640 1 -- 172.21.15.94:0/155408158 >> 172.21.15.94:0/155408158 conn(0x7f01200d1c60 msgr2=0x7f01200ae870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:21.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.568+0000 7f0127e00640 1 -- 172.21.15.94:0/155408158 shutdown_connections 2024-09-10T23:39:21.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.568+0000 7f0127e00640 1 -- 172.21.15.94:0/155408158 wait complete. 2024-09-10T23:39:21.593 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T23:39:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.732+0000 7f1a60c00640 1 -- 172.21.15.94:0/2764147577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a5c0b2bb0 msgr2=0x7f1a5c0d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.732+0000 7f1a60c00640 1 --2- 172.21.15.94:0/2764147577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a5c0b2bb0 0x7f1a5c0d0270 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009900 tx=0x7f1a4401c7c0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.732+0000 7f1a60c00640 1 -- 172.21.15.94:0/2764147577 shutdown_connections 2024-09-10T23:39:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.732+0000 7f1a60c00640 1 --2- 172.21.15.94:0/2764147577 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a5c0af310 0x7f1a5c0af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.732+0000 7f1a60c00640 1 --2- 172.21.15.94:0/2764147577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a5c0b2bb0 0x7f1a5c0d0270 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.732+0000 7f1a60c00640 1 -- 172.21.15.94:0/2764147577 >> 172.21.15.94:0/2764147577 conn(0x7f1a5c0d1c60 msgr2=0x7f1a5c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.732+0000 7f1a60c00640 1 -- 172.21.15.94:0/2764147577 shutdown_connections 2024-09-10T23:39:21.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.732+0000 7f1a60c00640 1 -- 172.21.15.94:0/2764147577 wait complete. 2024-09-10T23:39:21.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a60c00640 1 Processor -- start 2024-09-10T23:39:21.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a60c00640 1 -- start start 2024-09-10T23:39:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a60c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a5c0af310 0x7f1a5c1644b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a60c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a5c0b2bb0 0x7f1a5c15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a60c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a5c15f5c0 0x7f1a5c15d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a60c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c169000 con 0x7f1a5c15f5c0 2024-09-10T23:39:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a60c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c168e80 con 0x7f1a5c0b2bb0 2024-09-10T23:39:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a60c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1a5c169180 con 0x7f1a5c0af310 2024-09-10T23:39:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a5a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a5c15f5c0 0x7f1a5c15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a5a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a5c15f5c0 0x7f1a5c15d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41974/0 (socket says 172.21.15.94:41974) 2024-09-10T23:39:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a5a800640 1 -- 172.21.15.94:0/4155876738 learned_addr learned my addr 172.21.15.94:0/4155876738 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a59e00640 1 --2- 172.21.15.94:0/4155876738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a5c0af310 0x7f1a5c1644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a53e00640 1 --2- 172.21.15.94:0/4155876738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a5c0b2bb0 0x7f1a5c15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a5a800640 1 -- 172.21.15.94:0/4155876738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a5c0af310 msgr2=0x7f1a5c1644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:21.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a5a800640 1 --2- 172.21.15.94:0/4155876738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a5c0af310 0x7f1a5c1644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a5a800640 1 -- 172.21.15.94:0/4155876738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a5c0b2bb0 msgr2=0x7f1a5c15cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:21.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a5a800640 1 --2- 172.21.15.94:0/4155876738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a5c0b2bb0 0x7f1a5c15cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a5a800640 1 -- 172.21.15.94:0/4155876738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5c153e70 con 0x7f1a5c15f5c0 2024-09-10T23:39:21.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a5a800640 1 --2- 172.21.15.94:0/4155876738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a5c15f5c0 0x7f1a5c15d280 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400ea20 tx=0x7f1a5400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:21.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a52000640 1 -- 172.21.15.94:0/4155876738 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5400cde0 con 0x7f1a5c15f5c0 2024-09-10T23:39:21.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a52000640 1 -- 172.21.15.94:0/4155876738 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a54004c40 con 0x7f1a5c15f5c0 2024-09-10T23:39:21.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a60c00640 1 -- 172.21.15.94:0/4155876738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5c152dd0 con 0x7f1a5c15f5c0 2024-09-10T23:39:21.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a52000640 1 -- 172.21.15.94:0/4155876738 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54005280 con 0x7f1a5c15f5c0 2024-09-10T23:39:21.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a60c00640 1 -- 172.21.15.94:0/4155876738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a5c153390 con 0x7f1a5c15f5c0 2024-09-10T23:39:21.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.736+0000 7f1a60c00640 1 -- 172.21.15.94:0/4155876738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a5c0af7a0 con 0x7f1a5c15f5c0 2024-09-10T23:39:21.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.740+0000 7f1a52000640 1 -- 172.21.15.94:0/4155876738 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f1a5400f040 con 0x7f1a5c15f5c0 2024-09-10T23:39:21.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.740+0000 7f1a52000640 1 --2- 172.21.15.94:0/4155876738 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1a38072cd0 0x7f1a38075190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:21.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.740+0000 7f1a52000640 1 -- 172.21.15.94:0/4155876738 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4885+0+0 (secure 0 0 0) 0x7f1a54094290 con 0x7f1a5c15f5c0 2024-09-10T23:39:21.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.740+0000 7f1a59e00640 1 --2- 172.21.15.94:0/4155876738 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1a38072cd0 0x7f1a38075190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:21.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.740+0000 7f1a59e00640 1 --2- 172.21.15.94:0/4155876738 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1a38072cd0 0x7f1a38075190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c007fd0 tx=0x7f1a4c009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:21.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.744+0000 7f1a52000640 1 -- 172.21.15.94:0/4155876738 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1a54016030 con 0x7f1a5c15f5c0 2024-09-10T23:39:21.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.968+0000 7f1a60c00640 1 -- 172.21.15.94:0/4155876738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1a5c0749c0 con 0x7f1a5c15f5c0 2024-09-10T23:39:21.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.968+0000 7f1a52000640 1 -- 172.21.15.94:0/4155876738 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v37) ==== 80+0+5304 (secure 0 0 0) 0x7f1a5405eb70 con 0x7f1a5c15f5c0 2024-09-10T23:39:21.971 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:21.971 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:39:21.971 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_dump": [ 2024-09-10T23:39:21.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.971 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.a", 2024-09-10T23:39:21.971 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/2m3LxAAr0ErpLJg+0+I2r74zQya4w==", 2024-09-10T23:39:21.971 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.971 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:39:21.972 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:39:21.972 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:39:21.972 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:21.972 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.972 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.b", 2024-09-10T23:39:21.972 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmc8pYABAAw90zu8D+9IwK15h4HIlGBA==", 2024-09-10T23:39:21.972 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.972 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:39:21.972 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:39:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:39:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.c", 2024-09-10T23:39:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmk98KNxAA44e9N4VHbw/94Lh1V9zugQ==", 2024-09-10T23:39:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:39:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:39:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:39:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.d", 2024-09-10T23:39:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmVAKzBxAAJehOw6aphTF/g3ucoQfw9A==", 2024-09-10T23:39:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:39:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:39:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:39:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.0", 2024-09-10T23:39:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDa1uBmSAD3KBAAJTF1OEC44Ml599aL7iEuDw==", 2024-09-10T23:39:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:39:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:39:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.1", 2024-09-10T23:39:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDe1uBma1L9ABAAfEjW1/Hb4fh4Kwv/ZUgFAQ==", 2024-09-10T23:39:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:39:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:39:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.2", 2024-09-10T23:39:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDh1uBmtLUQFhAAweeFHj6BRQVC9BX+VLiWEQ==", 2024-09-10T23:39:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:39:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:39:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.3", 2024-09-10T23:39:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDk1uBm1KFCKRAAev28iYcrEYOViX95uo6Kbw==", 2024-09-10T23:39:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:39:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:39:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.4", 2024-09-10T23:39:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDq1uBm/Hg9LhAAvB2D0rj4DeRPxTrGmXYIsA==", 2024-09-10T23:39:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:39:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:39:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.5", 2024-09-10T23:39:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDu1uBm1fAKCBAAFHMjnGFAItLdaACF/UjE/w==", 2024-09-10T23:39:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:39:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:39:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.6", 2024-09-10T23:39:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDx1uBmXgYfHhAA/6wwtM4mDRXsIpoTIHKJ0w==", 2024-09-10T23:39:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:39:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:39:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.7", 2024-09-10T23:39:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQD01uBmw6hQMxAAk7oGAg+iIplNK5KrokGwNA==", 2024-09-10T23:39:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:39:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:39:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.0", 2024-09-10T23:39:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g==", 2024-09-10T23:39:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:39:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow r", 2024-09-10T23:39:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T23:39:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.1", 2024-09-10T23:39:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmuSkhFRAA1R6WNXLImkGfhW9/XJzEAQ==", 2024-09-10T23:39:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:39:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:39:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:39:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:39:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.2", 2024-09-10T23:39:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmfvaqERAASpL0KtglFcAuRf4a/0z5JQ==", 2024-09-10T23:39:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:39:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:39:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:39:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:39:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.3", 2024-09-10T23:39:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBm8xUtGxAARc0ywfAhMxBNd57TzBoV3g==", 2024-09-10T23:39:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:39:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:39:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:39:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:39:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.admin", 2024-09-10T23:39:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmOpg1FBAA9NV6Jqt6doWMuAfcxztlOg==", 2024-09-10T23:39:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:39:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:39:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:39:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.x", 2024-09-10T23:39:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/EPVKxAA/Myv8Zjf7BejbzlkxyCC1w==", 2024-09-10T23:39:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:39:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:39:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.y", 2024-09-10T23:39:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmhHvtJxAACe+79nYhv2ODwtmNhA45FA==", 2024-09-10T23:39:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:39:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:39:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:39:21.985 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:39:21.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.968+0000 7f1a60c00640 1 -- 172.21.15.94:0/4155876738 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1a38072cd0 msgr2=0x7f1a38075190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:21.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.968+0000 7f1a60c00640 1 --2- 172.21.15.94:0/4155876738 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1a38072cd0 0x7f1a38075190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c007fd0 tx=0x7f1a4c009040 comp rx=0 tx=0).stop 2024-09-10T23:39:21.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.968+0000 7f1a60c00640 1 -- 172.21.15.94:0/4155876738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a5c15f5c0 msgr2=0x7f1a5c15d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:21.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.968+0000 7f1a60c00640 1 --2- 172.21.15.94:0/4155876738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a5c15f5c0 0x7f1a5c15d280 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400ea20 tx=0x7f1a5400eee0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.968+0000 7f1a60c00640 1 -- 172.21.15.94:0/4155876738 shutdown_connections 2024-09-10T23:39:21.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.968+0000 7f1a60c00640 1 --2- 172.21.15.94:0/4155876738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a5c0b2bb0 0x7f1a5c15cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.968+0000 7f1a60c00640 1 --2- 172.21.15.94:0/4155876738 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1a38072cd0 0x7f1a38075190 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.968+0000 7f1a60c00640 1 --2- 172.21.15.94:0/4155876738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a5c15f5c0 0x7f1a5c15d280 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.968+0000 7f1a60c00640 1 --2- 172.21.15.94:0/4155876738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a5c0af310 0x7f1a5c1644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:21.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.968+0000 7f1a60c00640 1 -- 172.21.15.94:0/4155876738 >> 172.21.15.94:0/4155876738 conn(0x7f1a5c0d1c60 msgr2=0x7f1a5c070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:21.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.968+0000 7f1a60c00640 1 -- 172.21.15.94:0/4155876738 shutdown_connections 2024-09-10T23:39:21.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:21.968+0000 7f1a60c00640 1 -- 172.21.15.94:0/4155876738 wait complete. 2024-09-10T23:39:21.993 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T23:39:22.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.132+0000 7f526e000640 1 -- 172.21.15.94:0/2987056979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52500051d0 msgr2=0x7f525000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:22.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.132+0000 7f526e000640 1 --2- 172.21.15.94:0/2987056979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52500051d0 0x7f525000b9d0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f5260001ef0 tx=0x7f526001cc30 comp rx=0 tx=0).stop 2024-09-10T23:39:22.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.132+0000 7f526d600640 1 -- 172.21.15.94:0/2987056979 reap_dead start 2024-09-10T23:39:22.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.132+0000 7f526e000640 1 -- 172.21.15.94:0/2987056979 shutdown_connections 2024-09-10T23:39:22.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.132+0000 7f526e000640 1 -- 172.21.15.94:0/2987056979 >> 172.21.15.94:0/2987056979 conn(0x7f526809aa70 msgr2=0x7f526809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.132+0000 7f526e000640 1 -- 172.21.15.94:0/2987056979 shutdown_connections 2024-09-10T23:39:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.132+0000 7f526e000640 1 -- 172.21.15.94:0/2987056979 wait complete. 2024-09-10T23:39:22.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f526e000640 1 Processor -- start 2024-09-10T23:39:22.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f526e000640 1 -- start start 2024-09-10T23:39:22.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f526e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52500051d0 0x7f52680cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:22.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f526e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52680ce480 0x7f52680c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:22.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f526e000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52680b67a0 0x7f52680b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:22.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f526e000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f52680d0360 con 0x7f52680ce480 2024-09-10T23:39:22.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f526e000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f52680d01e0 con 0x7f52500051d0 2024-09-10T23:39:22.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f526e000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f52680d04e0 con 0x7f52680b67a0 2024-09-10T23:39:22.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f5267e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52680ce480 0x7f52680c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:22.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f5267e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52680ce480 0x7f52680c7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41994/0 (socket says 172.21.15.94:41994) 2024-09-10T23:39:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f5267e00640 1 -- 172.21.15.94:0/1595538148 learned_addr learned my addr 172.21.15.94:0/1595538148 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f526d600640 1 --2- 172.21.15.94:0/1595538148 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52680b67a0 0x7f52680b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f526cc00640 1 --2- 172.21.15.94:0/1595538148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52500051d0 0x7f52680cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f5267e00640 1 -- 172.21.15.94:0/1595538148 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52680b67a0 msgr2=0x7f52680b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f5267e00640 1 --2- 172.21.15.94:0/1595538148 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52680b67a0 0x7f52680b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f5267e00640 1 -- 172.21.15.94:0/1595538148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52500051d0 msgr2=0x7f52680cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f5267e00640 1 --2- 172.21.15.94:0/1595538148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52500051d0 0x7f52680cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f5267e00640 1 -- 172.21.15.94:0/1595538148 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52680af800 con 0x7f52680ce480 2024-09-10T23:39:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f526d600640 1 --2- 172.21.15.94:0/1595538148 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52680b67a0 0x7f52680b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f5267e00640 1 --2- 172.21.15.94:0/1595538148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52680ce480 0x7f52680c7040 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f52540119b0 tx=0x7f5254011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f5264c00640 1 -- 172.21.15.94:0/1595538148 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525400fcf0 con 0x7f52680ce480 2024-09-10T23:39:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f5264c00640 1 -- 172.21.15.94:0/1595538148 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5254004030 con 0x7f52680ce480 2024-09-10T23:39:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f5264c00640 1 -- 172.21.15.94:0/1595538148 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254004480 con 0x7f52680ce480 2024-09-10T23:39:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f526e000640 1 -- 172.21.15.94:0/1595538148 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52681f5870 con 0x7f52680ce480 2024-09-10T23:39:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f526e000640 1 -- 172.21.15.94:0/1595538148 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52680cc920 con 0x7f52680ce480 2024-09-10T23:39:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.136+0000 7f526e000640 1 -- 172.21.15.94:0/1595538148 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52680af2c0 con 0x7f52680ce480 2024-09-10T23:39:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.140+0000 7f5264c00640 1 -- 172.21.15.94:0/1595538148 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f5254003600 con 0x7f52680ce480 2024-09-10T23:39:22.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.140+0000 7f5264c00640 1 --2- 172.21.15.94:0/1595538148 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5248072ca0 0x7f5248075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:22.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.140+0000 7f5264c00640 1 -- 172.21.15.94:0/1595538148 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4885+0+0 (secure 0 0 0) 0x7f5254094810 con 0x7f52680ce480 2024-09-10T23:39:22.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.140+0000 7f526cc00640 1 --2- 172.21.15.94:0/1595538148 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5248072ca0 0x7f5248075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:22.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.140+0000 7f526cc00640 1 --2- 172.21.15.94:0/1595538148 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5248072ca0 0x7f5248075160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f525c001060 tx=0x7f525c0016b0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.144+0000 7f5264c00640 1 -- 172.21.15.94:0/1595538148 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f525405f8d0 con 0x7f52680ce480 2024-09-10T23:39:22.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.312+0000 7f526e000640 1 -- 172.21.15.94:0/1595538148 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5268158c80 con 0x7f52680ce480 2024-09-10T23:39:22.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.312+0000 7f5264c00640 1 -- 172.21.15.94:0/1595538148 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v134) ==== 84+0+65 (secure 0 0 0) 0x7f525405f1d0 con 0x7f52680ce480 2024-09-10T23:39:22.315 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:22.316 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:39:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-09-10T23:39:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-09-10T23:39:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:22.316 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:39:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.316+0000 7f526e000640 1 -- 172.21.15.94:0/1595538148 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5248072ca0 msgr2=0x7f5248075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.316+0000 7f526e000640 1 --2- 172.21.15.94:0/1595538148 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5248072ca0 0x7f5248075160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f525c001060 tx=0x7f525c0016b0 comp rx=0 tx=0).stop 2024-09-10T23:39:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.316+0000 7f526e000640 1 -- 172.21.15.94:0/1595538148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52680ce480 msgr2=0x7f52680c7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.316+0000 7f526e000640 1 --2- 172.21.15.94:0/1595538148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52680ce480 0x7f52680c7040 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f52540119b0 tx=0x7f5254011e70 comp rx=0 tx=0).stop 2024-09-10T23:39:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.316+0000 7f526e000640 1 -- 172.21.15.94:0/1595538148 shutdown_connections 2024-09-10T23:39:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.316+0000 7f526e000640 1 --2- 172.21.15.94:0/1595538148 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5248072ca0 0x7f5248075160 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.316+0000 7f526e000640 1 --2- 172.21.15.94:0/1595538148 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52680b67a0 0x7f52680b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.316+0000 7f526e000640 1 --2- 172.21.15.94:0/1595538148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52680ce480 0x7f52680c7040 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.316+0000 7f526e000640 1 --2- 172.21.15.94:0/1595538148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52500051d0 0x7f52680cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.316+0000 7f526e000640 1 -- 172.21.15.94:0/1595538148 >> 172.21.15.94:0/1595538148 conn(0x7f526809aa70 msgr2=0x7f5268157f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:22.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.316+0000 7f526e000640 1 -- 172.21.15.94:0/1595538148 shutdown_connections 2024-09-10T23:39:22.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.316+0000 7f526e000640 1 -- 172.21.15.94:0/1595538148 wait complete. 2024-09-10T23:39:22.336 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T23:39:22.337 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T23:39:22.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 -- 172.21.15.94:0/73262266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5815ecb0 msgr2=0x7f7e580b35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:22.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 --2- 172.21.15.94:0/73262266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5815ecb0 0x7f7e580b35f0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f7e580d0b80 tx=0x7f7e5001cd50 comp rx=0 tx=0).stop 2024-09-10T23:39:22.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 -- 172.21.15.94:0/73262266 shutdown_connections 2024-09-10T23:39:22.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 --2- 172.21.15.94:0/73262266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5815ecb0 0x7f7e580b35f0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:22.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 --2- 172.21.15.94:0/73262266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e580d02e0 0x7f7e5815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:22.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 --2- 172.21.15.94:0/73262266 >> v2:172.21.15.181:3300/0 conn(0x7f7e58073ff0 0x7f7e580cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:22.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 -- 172.21.15.94:0/73262266 >> 172.21.15.94:0/73262266 conn(0x7f7e5806cdf0 msgr2=0x7f7e5806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:22.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 -- 172.21.15.94:0/73262266 shutdown_connections 2024-09-10T23:39:22.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 -- 172.21.15.94:0/73262266 wait complete. 2024-09-10T23:39:22.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 Processor -- start 2024-09-10T23:39:22.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 -- start start 2024-09-10T23:39:22.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e58073ff0 0x7f7e581f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:22.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e580d02e0 0x7f7e58138c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:22.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5815ecb0 0x7f7e581f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:22.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5806f840 con 0x7f7e5815ecb0 2024-09-10T23:39:22.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5806f6c0 con 0x7f7e58073ff0 2024-09-10T23:39:22.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e5e600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5806f9c0 con 0x7f7e580d02e0 2024-09-10T23:39:22.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e57e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5815ecb0 0x7f7e581f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:22.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e57e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5815ecb0 0x7f7e581f49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42012/0 (socket says 172.21.15.94:42012) 2024-09-10T23:39:22.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e57e00640 1 -- 172.21.15.94:0/2447978844 learned_addr learned my addr 172.21.15.94:0/2447978844 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:22.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.480+0000 7f7e57400640 1 --2- 172.21.15.94:0/2447978844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e58073ff0 0x7f7e581f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:22.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e4ea00640 1 --2- 172.21.15.94:0/2447978844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e580d02e0 0x7f7e58138c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:22.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e57e00640 1 -- 172.21.15.94:0/2447978844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e580d02e0 msgr2=0x7f7e58138c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:22.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e57e00640 1 --2- 172.21.15.94:0/2447978844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e580d02e0 0x7f7e58138c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:22.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e57e00640 1 -- 172.21.15.94:0/2447978844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e58073ff0 msgr2=0x7f7e581f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:22.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e57e00640 1 --2- 172.21.15.94:0/2447978844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e58073ff0 0x7f7e581f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:22.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e57e00640 1 -- 172.21.15.94:0/2447978844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e581559b0 con 0x7f7e5815ecb0 2024-09-10T23:39:22.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e57400640 1 --2- 172.21.15.94:0/2447978844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e58073ff0 0x7f7e581f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:22.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e57e00640 1 --2- 172.21.15.94:0/2447978844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5815ecb0 0x7f7e581f49d0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f7e5001b400 tx=0x7f7e50005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:22.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e4fe00640 1 -- 172.21.15.94:0/2447978844 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e500bb070 con 0x7f7e5815ecb0 2024-09-10T23:39:22.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e4fe00640 1 -- 172.21.15.94:0/2447978844 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e50004d40 con 0x7f7e5815ecb0 2024-09-10T23:39:22.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e4fe00640 1 -- 172.21.15.94:0/2447978844 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e500b2040 con 0x7f7e5815ecb0 2024-09-10T23:39:22.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e5e600640 1 -- 172.21.15.94:0/2447978844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e58139190 con 0x7f7e5815ecb0 2024-09-10T23:39:22.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e5e600640 1 -- 172.21.15.94:0/2447978844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e5813b450 con 0x7f7e5815ecb0 2024-09-10T23:39:22.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e5e600640 1 -- 172.21.15.94:0/2447978844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e5815e770 con 0x7f7e5815ecb0 2024-09-10T23:39:22.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e4fe00640 1 -- 172.21.15.94:0/2447978844 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f7e500b5030 con 0x7f7e5815ecb0 2024-09-10T23:39:22.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e4fe00640 1 --2- 172.21.15.94:0/2447978844 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7e34072ca0 0x7f7e34075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:22.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.484+0000 7f7e4fe00640 1 -- 172.21.15.94:0/2447978844 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4885+0+0 (secure 0 0 0) 0x7f7e5012d7e0 con 0x7f7e5815ecb0 2024-09-10T23:39:22.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.488+0000 7f7e57400640 1 --2- 172.21.15.94:0/2447978844 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7e34072ca0 0x7f7e34075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:22.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.488+0000 7f7e57400640 1 --2- 172.21.15.94:0/2447978844 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7e34072ca0 0x7f7e34075160 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7e48005fd0 tx=0x7f7e48005f20 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:22.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.488+0000 7f7e4fe00640 1 -- 172.21.15.94:0/2447978844 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7e50016970 con 0x7f7e5815ecb0 2024-09-10T23:39:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:22.660+0000 7f7e5e600640 1 -- 172.21.15.94:0/2447978844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7e581f8870 con 0x7f7e5815ecb0 2024-09-10T23:39:23.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.576+0000 7f7e4fe00640 1 -- 172.21.15.94:0/2447978844 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v135) ==== 138+0+0 (secure 0 0 0) 0x7f7e500f8810 con 0x7f7e5815ecb0 2024-09-10T23:39:23.578 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-09-10T23:39:23.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.580+0000 7f7e5e600640 1 -- 172.21.15.94:0/2447978844 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7e34072ca0 msgr2=0x7f7e34075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:23.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.580+0000 7f7e5e600640 1 --2- 172.21.15.94:0/2447978844 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7e34072ca0 0x7f7e34075160 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7e48005fd0 tx=0x7f7e48005f20 comp rx=0 tx=0).stop 2024-09-10T23:39:23.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.580+0000 7f7e5e600640 1 -- 172.21.15.94:0/2447978844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5815ecb0 msgr2=0x7f7e581f49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:23.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.580+0000 7f7e5e600640 1 --2- 172.21.15.94:0/2447978844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5815ecb0 0x7f7e581f49d0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f7e5001b400 tx=0x7f7e50005b00 comp rx=0 tx=0).stop 2024-09-10T23:39:23.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.580+0000 7f7e5e600640 1 -- 172.21.15.94:0/2447978844 shutdown_connections 2024-09-10T23:39:23.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.580+0000 7f7e5e600640 1 --2- 172.21.15.94:0/2447978844 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7e34072ca0 0x7f7e34075160 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:23.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.580+0000 7f7e5e600640 1 --2- 172.21.15.94:0/2447978844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e5815ecb0 0x7f7e581f49d0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:23.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.580+0000 7f7e5e600640 1 --2- 172.21.15.94:0/2447978844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e580d02e0 0x7f7e58138c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:23.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.580+0000 7f7e5e600640 1 --2- 172.21.15.94:0/2447978844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e58073ff0 0x7f7e581f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:23.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.580+0000 7f7e5e600640 1 -- 172.21.15.94:0/2447978844 >> 172.21.15.94:0/2447978844 conn(0x7f7e5806cdf0 msgr2=0x7f7e580af640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:23.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.580+0000 7f7e5e600640 1 -- 172.21.15.94:0/2447978844 shutdown_connections 2024-09-10T23:39:23.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.580+0000 7f7e5e600640 1 -- 172.21.15.94:0/2447978844 wait complete. 2024-09-10T23:39:23.606 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T23:39:23.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.752+0000 7f09c4a00640 1 -- 172.21.15.94:0/4149854044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c006e7c0 msgr2=0x7f09c006ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:23.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.752+0000 7f09c4a00640 1 --2- 172.21.15.94:0/4149854044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c006e7c0 0x7f09c006ebf0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f09c00cd580 tx=0x7f09b801cd80 comp rx=0 tx=0).stop 2024-09-10T23:39:23.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.752+0000 7f09c4a00640 1 -- 172.21.15.94:0/4149854044 shutdown_connections 2024-09-10T23:39:23.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.752+0000 7f09c4a00640 1 --2- 172.21.15.94:0/4149854044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c006e7c0 0x7f09c006ebf0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:23.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.752+0000 7f09c4a00640 1 --2- 172.21.15.94:0/4149854044 >> v2:172.21.15.181:3300/0 conn(0x7f09c00c64f0 0x7f09c00cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:23.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.752+0000 7f09c4a00640 1 -- 172.21.15.94:0/4149854044 >> 172.21.15.94:0/4149854044 conn(0x7f09c00cfaf0 msgr2=0x7f09c00cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:23.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.752+0000 7f09c4a00640 1 -- 172.21.15.94:0/4149854044 shutdown_connections 2024-09-10T23:39:23.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.752+0000 7f09c4a00640 1 -- 172.21.15.94:0/4149854044 wait complete. 2024-09-10T23:39:23.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09c4a00640 1 Processor -- start 2024-09-10T23:39:23.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09c4a00640 1 -- start start 2024-09-10T23:39:23.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09c4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c006e7c0 0x7f09c014d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:23.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09c4a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09c00c64f0 0x7f09c00aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:23.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09c4a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09c00ce280 0x7f09c00c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:23.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09c4a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f09c015b4b0 con 0x7f09c006e7c0 2024-09-10T23:39:23.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09c4a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f09c015b330 con 0x7f09c00ce280 2024-09-10T23:39:23.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09c4a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f09c015b630 con 0x7f09c00c64f0 2024-09-10T23:39:23.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09bde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c006e7c0 0x7f09c014d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:23.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09bde00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c006e7c0 0x7f09c014d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42028/0 (socket says 172.21.15.94:42028) 2024-09-10T23:39:23.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09bde00640 1 -- 172.21.15.94:0/2054197617 learned_addr learned my addr 172.21.15.94:0/2054197617 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:23.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09be800640 1 --2- 172.21.15.94:0/2054197617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09c00ce280 0x7f09c00c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:23.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09bd400640 1 --2- 172.21.15.94:0/2054197617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09c00c64f0 0x7f09c00aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:23.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09bde00640 1 -- 172.21.15.94:0/2054197617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09c00c64f0 msgr2=0x7f09c00aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:23.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09bde00640 1 --2- 172.21.15.94:0/2054197617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09c00c64f0 0x7f09c00aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:23.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09bde00640 1 -- 172.21.15.94:0/2054197617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09c00ce280 msgr2=0x7f09c00c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:23.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09bde00640 1 --2- 172.21.15.94:0/2054197617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09c00ce280 0x7f09c00c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:23.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09bde00640 1 -- 172.21.15.94:0/2054197617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c00adec0 con 0x7f09c006e7c0 2024-09-10T23:39:23.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09bde00640 1 --2- 172.21.15.94:0/2054197617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c006e7c0 0x7f09c014d190 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f09b000c3c0 tx=0x7f09b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:23.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09b6000640 1 -- 172.21.15.94:0/2054197617 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0017070 con 0x7f09c006e7c0 2024-09-10T23:39:23.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09b6000640 1 -- 172.21.15.94:0/2054197617 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09b00053c0 con 0x7f09c006e7c0 2024-09-10T23:39:23.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09b6000640 1 -- 172.21.15.94:0/2054197617 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0003860 con 0x7f09c006e7c0 2024-09-10T23:39:23.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09c4a00640 1 -- 172.21.15.94:0/2054197617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c00ae1a0 con 0x7f09c006e7c0 2024-09-10T23:39:23.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.756+0000 7f09c4a00640 1 -- 172.21.15.94:0/2054197617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c00ae760 con 0x7f09c006e7c0 2024-09-10T23:39:23.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.760+0000 7f09c4a00640 1 -- 172.21.15.94:0/2054197617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c006ebf0 con 0x7f09c006e7c0 2024-09-10T23:39:23.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.760+0000 7f09b6000640 1 -- 172.21.15.94:0/2054197617 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f09b0003a00 con 0x7f09c006e7c0 2024-09-10T23:39:23.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.760+0000 7f09b6000640 1 --2- 172.21.15.94:0/2054197617 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f09a0072ca0 0x7f09a0075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:23.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.760+0000 7f09b6000640 1 -- 172.21.15.94:0/2054197617 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5255+0+0 (secure 0 0 0) 0x7f09b00935f0 con 0x7f09c006e7c0 2024-09-10T23:39:23.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.760+0000 7f09bd400640 1 --2- 172.21.15.94:0/2054197617 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f09a0072ca0 0x7f09a0075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:23.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.764+0000 7f09bd400640 1 --2- 172.21.15.94:0/2054197617 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f09a0072ca0 0x7f09a0075160 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f09c00c9460 tx=0x7f09ac00b490 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:23.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.764+0000 7f09b6000640 1 -- 172.21.15.94:0/2054197617 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f09b0021020 con 0x7f09c006e7c0 2024-09-10T23:39:23.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:23.932+0000 7f09c4a00640 1 -- 172.21.15.94:0/2054197617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f09c00c6a70 con 0x7f09c006e7c0 2024-09-10T23:39:24.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.584+0000 7f09b6000640 1 -- 172.21.15.94:0/2054197617 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v136) ==== 170+0+0 (secure 0 0 0) 0x7f09b005dea0 con 0x7f09c006e7c0 2024-09-10T23:39:24.587 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-09-10T23:39:24.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.588+0000 7f09c4a00640 1 -- 172.21.15.94:0/2054197617 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f09a0072ca0 msgr2=0x7f09a0075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:24.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.588+0000 7f09c4a00640 1 --2- 172.21.15.94:0/2054197617 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f09a0072ca0 0x7f09a0075160 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f09c00c9460 tx=0x7f09ac00b490 comp rx=0 tx=0).stop 2024-09-10T23:39:24.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.588+0000 7f09c4a00640 1 -- 172.21.15.94:0/2054197617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c006e7c0 msgr2=0x7f09c014d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:24.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.588+0000 7f09c4a00640 1 --2- 172.21.15.94:0/2054197617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c006e7c0 0x7f09c014d190 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f09b000c3c0 tx=0x7f09b000c880 comp rx=0 tx=0).stop 2024-09-10T23:39:24.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.588+0000 7f09c4a00640 1 -- 172.21.15.94:0/2054197617 shutdown_connections 2024-09-10T23:39:24.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.588+0000 7f09c4a00640 1 --2- 172.21.15.94:0/2054197617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09c00ce280 0x7f09c00c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:24.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.588+0000 7f09c4a00640 1 --2- 172.21.15.94:0/2054197617 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f09a0072ca0 0x7f09a0075160 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:24.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.588+0000 7f09c4a00640 1 --2- 172.21.15.94:0/2054197617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09c00c64f0 0x7f09c00aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:24.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.588+0000 7f09c4a00640 1 --2- 172.21.15.94:0/2054197617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c006e7c0 0x7f09c014d190 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:24.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.588+0000 7f09c4a00640 1 -- 172.21.15.94:0/2054197617 >> 172.21.15.94:0/2054197617 conn(0x7f09c00cfaf0 msgr2=0x7f09c014d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:24.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.588+0000 7f09c4a00640 1 -- 172.21.15.94:0/2054197617 shutdown_connections 2024-09-10T23:39:24.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.588+0000 7f09c4a00640 1 -- 172.21.15.94:0/2054197617 wait complete. 2024-09-10T23:39:24.628 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T23:39:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f3260800640 1 -- 172.21.15.94:0/3373991446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f325806cc20 msgr2=0x7f325806d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f3260800640 1 --2- 172.21.15.94:0/3373991446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f325806cc20 0x7f325806d000 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f325806f150 tx=0x7f325001cc40 comp rx=0 tx=0).stop 2024-09-10T23:39:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f3260800640 1 -- 172.21.15.94:0/3373991446 shutdown_connections 2024-09-10T23:39:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f3260800640 1 --2- 172.21.15.94:0/3373991446 >> v2:172.21.15.181:3300/0 conn(0x7f32580cc8f0 0x7f32580ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f3260800640 1 --2- 172.21.15.94:0/3373991446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f325806cc20 0x7f325806d000 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f3260800640 1 -- 172.21.15.94:0/3373991446 >> 172.21.15.94:0/3373991446 conn(0x7f32580d0c30 msgr2=0x7f32580d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f3260800640 1 -- 172.21.15.94:0/3373991446 shutdown_connections 2024-09-10T23:39:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f3260800640 1 -- 172.21.15.94:0/3373991446 wait complete. 2024-09-10T23:39:24.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f3260800640 1 Processor -- start 2024-09-10T23:39:24.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f3260800640 1 -- start start 2024-09-10T23:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f3260800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f325806cc20 0x7f32580c4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f3260800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32580cc8f0 0x7f325815dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f3260800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f325815f6d0 0x7f325815e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:24.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f3260800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f32580cba90 con 0x7f325815f6d0 2024-09-10T23:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f3260800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3258074660 con 0x7f32580cc8f0 2024-09-10T23:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f3260800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f32580cbc10 con 0x7f325806cc20 2024-09-10T23:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f325e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f325815f6d0 0x7f325815e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f325e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f325815f6d0 0x7f325815e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50174/0 (socket says 172.21.15.94:50174) 2024-09-10T23:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f325e800640 1 -- 172.21.15.94:0/701062170 learned_addr learned my addr 172.21.15.94:0/701062170 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f325d400640 1 --2- 172.21.15.94:0/701062170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32580cc8f0 0x7f325815dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f325de00640 1 --2- 172.21.15.94:0/701062170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f325806cc20 0x7f32580c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f325e800640 1 -- 172.21.15.94:0/701062170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f325806cc20 msgr2=0x7f32580c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f325e800640 1 --2- 172.21.15.94:0/701062170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f325806cc20 0x7f32580c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f325e800640 1 -- 172.21.15.94:0/701062170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32580cc8f0 msgr2=0x7f325815dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f325e800640 1 --2- 172.21.15.94:0/701062170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32580cc8f0 0x7f325815dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f325e800640 1 -- 172.21.15.94:0/701062170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32581566d0 con 0x7f325815f6d0 2024-09-10T23:39:24.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f325e800640 1 --2- 172.21.15.94:0/701062170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f325815f6d0 0x7f325815e320 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f325400b410 tx=0x7f325400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:24.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.772+0000 7f324e000640 1 -- 172.21.15.94:0/701062170 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254015020 con 0x7f325815f6d0 2024-09-10T23:39:24.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.776+0000 7f324e000640 1 -- 172.21.15.94:0/701062170 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3254011960 con 0x7f325815f6d0 2024-09-10T23:39:24.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.776+0000 7f3260800640 1 -- 172.21.15.94:0/701062170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3258156970 con 0x7f325815f6d0 2024-09-10T23:39:24.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.776+0000 7f324e000640 1 -- 172.21.15.94:0/701062170 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254011db0 con 0x7f325815f6d0 2024-09-10T23:39:24.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.776+0000 7f3260800640 1 -- 172.21.15.94:0/701062170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3258158630 con 0x7f325815f6d0 2024-09-10T23:39:24.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.776+0000 7f3260800640 1 -- 172.21.15.94:0/701062170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f325806d000 con 0x7f325815f6d0 2024-09-10T23:39:24.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.776+0000 7f324e000640 1 -- 172.21.15.94:0/701062170 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f3254024480 con 0x7f325815f6d0 2024-09-10T23:39:24.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.776+0000 7f324e000640 1 --2- 172.21.15.94:0/701062170 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f323c072cd0 0x7f323c075190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:24.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.776+0000 7f324e000640 1 -- 172.21.15.94:0/701062170 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5637+0+0 (secure 0 0 0) 0x7f32540944c0 con 0x7f325815f6d0 2024-09-10T23:39:24.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.776+0000 7f325de00640 1 --2- 172.21.15.94:0/701062170 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f323c072cd0 0x7f323c075190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:24.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.780+0000 7f325de00640 1 --2- 172.21.15.94:0/701062170 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f323c072cd0 0x7f323c075190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f32500af000 tx=0x7f325001b480 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:24.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.780+0000 7f324e000640 1 -- 172.21.15.94:0/701062170 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f325401f020 con 0x7f325815f6d0 2024-09-10T23:39:24.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:24.984+0000 7f3260800640 1 -- 172.21.15.94:0/701062170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f32580b6bb0 con 0x7f325815f6d0 2024-09-10T23:39:25.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.600+0000 7f324e000640 1 -- 172.21.15.94:0/701062170 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v94) ==== 174+0+0 (secure 0 0 0) 0x7f325405eaa0 con 0x7f325815f6d0 2024-09-10T23:39:25.604 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 22 and data pool 23 2024-09-10T23:39:25.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.608+0000 7f3260800640 1 -- 172.21.15.94:0/701062170 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f323c072cd0 msgr2=0x7f323c075190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.608+0000 7f3260800640 1 --2- 172.21.15.94:0/701062170 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f323c072cd0 0x7f323c075190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f32500af000 tx=0x7f325001b480 comp rx=0 tx=0).stop 2024-09-10T23:39:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.608+0000 7f3260800640 1 -- 172.21.15.94:0/701062170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f325815f6d0 msgr2=0x7f325815e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.608+0000 7f3260800640 1 --2- 172.21.15.94:0/701062170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f325815f6d0 0x7f325815e320 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f325400b410 tx=0x7f325400b8d0 comp rx=0 tx=0).stop 2024-09-10T23:39:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.608+0000 7f3260800640 1 -- 172.21.15.94:0/701062170 shutdown_connections 2024-09-10T23:39:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.608+0000 7f3260800640 1 --2- 172.21.15.94:0/701062170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32580cc8f0 0x7f325815dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.608+0000 7f3260800640 1 --2- 172.21.15.94:0/701062170 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f323c072cd0 0x7f323c075190 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.608+0000 7f3260800640 1 --2- 172.21.15.94:0/701062170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f325815f6d0 0x7f325815e320 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.608+0000 7f3260800640 1 --2- 172.21.15.94:0/701062170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f325806cc20 0x7f32580c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.608+0000 7f3260800640 1 -- 172.21.15.94:0/701062170 >> 172.21.15.94:0/701062170 conn(0x7f32580d0c30 msgr2=0x7f32580b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.612+0000 7f3260800640 1 -- 172.21.15.94:0/701062170 shutdown_connections 2024-09-10T23:39:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.612+0000 7f3260800640 1 -- 172.21.15.94:0/701062170 wait complete. 2024-09-10T23:39:25.643 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:39:25.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.780+0000 7f9619600640 1 -- 172.21.15.94:0/185462372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96140b0320 msgr2=0x7f9614166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:25.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.780+0000 7f9619600640 1 --2- 172.21.15.94:0/185462372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96140b0320 0x7f9614166aa0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f960c009900 tx=0x7f960c01c7c0 comp rx=0 tx=0).stop 2024-09-10T23:39:25.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.780+0000 7f9619600640 1 -- 172.21.15.94:0/185462372 shutdown_connections 2024-09-10T23:39:25.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.780+0000 7f9619600640 1 --2- 172.21.15.94:0/185462372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96140b0320 0x7f9614166aa0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:25.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.780+0000 7f9619600640 1 -- 172.21.15.94:0/185462372 >> 172.21.15.94:0/185462372 conn(0x7f96140d1c60 msgr2=0x7f96140d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:25.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.780+0000 7f9619600640 1 -- 172.21.15.94:0/185462372 shutdown_connections 2024-09-10T23:39:25.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.780+0000 7f9619600640 1 -- 172.21.15.94:0/185462372 wait complete. 2024-09-10T23:39:25.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9619600640 1 Processor -- start 2024-09-10T23:39:25.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9619600640 1 -- start start 2024-09-10T23:39:25.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9619600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96140b0320 0x7f961414fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:25.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9619600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96140b2bb0 0x7f96141faca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:25.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9619600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961415f5c0 0x7f96141fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:25.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9619600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9614169f00 con 0x7f961415f5c0 2024-09-10T23:39:25.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9619600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9614169d80 con 0x7f96140b0320 2024-09-10T23:39:25.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9619600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f961416a080 con 0x7f96140b2bb0 2024-09-10T23:39:25.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9613400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961415f5c0 0x7f96141fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9613400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961415f5c0 0x7f96141fb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50198/0 (socket says 172.21.15.94:50198) 2024-09-10T23:39:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9613400640 1 -- 172.21.15.94:0/3949017278 learned_addr learned my addr 172.21.15.94:0/3949017278 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f960a000640 1 --2- 172.21.15.94:0/3949017278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96140b2bb0 0x7f96141faca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9612a00640 1 --2- 172.21.15.94:0/3949017278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96140b0320 0x7f961414fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9613400640 1 -- 172.21.15.94:0/3949017278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96140b2bb0 msgr2=0x7f96141faca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9613400640 1 --2- 172.21.15.94:0/3949017278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96140b2bb0 0x7f96141faca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9613400640 1 -- 172.21.15.94:0/3949017278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96140b0320 msgr2=0x7f961414fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9613400640 1 --2- 172.21.15.94:0/3949017278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96140b0320 0x7f961414fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9613400640 1 -- 172.21.15.94:0/3949017278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96141fb970 con 0x7f961415f5c0 2024-09-10T23:39:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9613400640 1 --2- 172.21.15.94:0/3949017278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961415f5c0 0x7f96141fb1e0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f960c0b0040 tx=0x7f960c0a5660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f960b400640 1 -- 172.21.15.94:0/3949017278 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c0b2040 con 0x7f961415f5c0 2024-09-10T23:39:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f960b400640 1 -- 172.21.15.94:0/3949017278 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f960c004340 con 0x7f961415f5c0 2024-09-10T23:39:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f960b400640 1 -- 172.21.15.94:0/3949017278 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c005290 con 0x7f961415f5c0 2024-09-10T23:39:25.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9619600640 1 -- 172.21.15.94:0/3949017278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96140c80d0 con 0x7f961415f5c0 2024-09-10T23:39:25.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9619600640 1 -- 172.21.15.94:0/3949017278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96140b4cc0 con 0x7f961415f5c0 2024-09-10T23:39:25.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.784+0000 7f9619600640 1 -- 172.21.15.94:0/3949017278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96140cfb60 con 0x7f961415f5c0 2024-09-10T23:39:25.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.788+0000 7f960b400640 1 -- 172.21.15.94:0/3949017278 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f960c005430 con 0x7f961415f5c0 2024-09-10T23:39:25.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.788+0000 7f960b400640 1 --2- 172.21.15.94:0/3949017278 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95f0072ca0 0x7f95f0075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:25.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.788+0000 7f960b400640 1 -- 172.21.15.94:0/3949017278 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f960c12dd70 con 0x7f961415f5c0 2024-09-10T23:39:25.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.788+0000 7f9612a00640 1 --2- 172.21.15.94:0/3949017278 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95f0072ca0 0x7f95f0075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:25.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.788+0000 7f9612a00640 1 --2- 172.21.15.94:0/3949017278 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95f0072ca0 0x7f95f0075160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f96040012f0 tx=0x7f9604009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:25.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.792+0000 7f960b400640 1 -- 172.21.15.94:0/3949017278 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f960c0c4060 con 0x7f961415f5c0 2024-09-10T23:39:25.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.960+0000 7f9619600640 1 -- 172.21.15.94:0/3949017278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96140749c0 con 0x7f961415f5c0 2024-09-10T23:39:25.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.960+0000 7f960b400640 1 -- 172.21.15.94:0/3949017278 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+16784 (secure 0 0 0) 0x7f960c0164d0 con 0x7f961415f5c0 2024-09-10T23:39:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:25.964 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":137,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:39:24.990839+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:39:22.673102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T23:39:23.946666+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:39:25.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.964+0000 7f9619600640 1 -- 172.21.15.94:0/3949017278 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95f0072ca0 msgr2=0x7f95f0075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:25.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.964+0000 7f9619600640 1 --2- 172.21.15.94:0/3949017278 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95f0072ca0 0x7f95f0075160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f96040012f0 tx=0x7f9604009040 comp rx=0 tx=0).stop 2024-09-10T23:39:25.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.964+0000 7f9619600640 1 -- 172.21.15.94:0/3949017278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961415f5c0 msgr2=0x7f96141fb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:25.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.964+0000 7f9619600640 1 --2- 172.21.15.94:0/3949017278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961415f5c0 0x7f96141fb1e0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f960c0b0040 tx=0x7f960c0a5660 comp rx=0 tx=0).stop 2024-09-10T23:39:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.964+0000 7f9619600640 1 -- 172.21.15.94:0/3949017278 shutdown_connections 2024-09-10T23:39:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.964+0000 7f9619600640 1 --2- 172.21.15.94:0/3949017278 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95f0072ca0 0x7f95f0075160 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.964+0000 7f9619600640 1 --2- 172.21.15.94:0/3949017278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f961415f5c0 0x7f96141fb1e0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.964+0000 7f9619600640 1 --2- 172.21.15.94:0/3949017278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96140b2bb0 0x7f96141faca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.964+0000 7f9619600640 1 --2- 172.21.15.94:0/3949017278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96140b0320 0x7f961414fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.964+0000 7f9619600640 1 -- 172.21.15.94:0/3949017278 >> 172.21.15.94:0/3949017278 conn(0x7f96140d1c60 msgr2=0x7f9614152220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.964+0000 7f9619600640 1 -- 172.21.15.94:0/3949017278 shutdown_connections 2024-09-10T23:39:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:25.964+0000 7f9619600640 1 -- 172.21.15.94:0/3949017278 wait complete. 2024-09-10T23:39:25.987 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:39:26.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.124+0000 7f255d600640 1 -- 172.21.15.94:0/84135083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f255815f3f0 msgr2=0x7f2558068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:26.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.124+0000 7f255d600640 1 --2- 172.21.15.94:0/84135083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f255815f3f0 0x7f2558068990 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f2548009840 tx=0x7f254801c790 comp rx=0 tx=0).stop 2024-09-10T23:39:26.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.124+0000 7f255d600640 1 -- 172.21.15.94:0/84135083 shutdown_connections 2024-09-10T23:39:26.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.124+0000 7f255d600640 1 --2- 172.21.15.94:0/84135083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f255815f3f0 0x7f2558068990 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:26.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.124+0000 7f255d600640 1 -- 172.21.15.94:0/84135083 >> 172.21.15.94:0/84135083 conn(0x7f25580d02d0 msgr2=0x7f25580d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:26.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.124+0000 7f255d600640 1 -- 172.21.15.94:0/84135083 shutdown_connections 2024-09-10T23:39:26.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.124+0000 7f255d600640 1 -- 172.21.15.94:0/84135083 wait complete. 2024-09-10T23:39:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f255d600640 1 Processor -- start 2024-09-10T23:39:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f255d600640 1 -- start start 2024-09-10T23:39:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f255d600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25580cc2f0 0x7f25580aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f255d600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25580cdea0 0x7f2558159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f255d600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f255815f3f0 0x7f25580b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f255d600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f255806da30 con 0x7f25580cdea0 2024-09-10T23:39:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f255d600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f255806d8b0 con 0x7f25580cc2f0 2024-09-10T23:39:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f255d600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f25581697f0 con 0x7f255815f3f0 2024-09-10T23:39:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f2556000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25580cdea0 0x7f2558159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f2556000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25580cdea0 0x7f2558159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50214/0 (socket says 172.21.15.94:50214) 2024-09-10T23:39:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f2556000640 1 -- 172.21.15.94:0/4122409161 learned_addr learned my addr 172.21.15.94:0/4122409161 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f2556a00640 1 --2- 172.21.15.94:0/4122409161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25580cc2f0 0x7f25580aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f2557400640 1 --2- 172.21.15.94:0/4122409161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f255815f3f0 0x7f25580b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f2556000640 1 -- 172.21.15.94:0/4122409161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f255815f3f0 msgr2=0x7f25580b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f2556000640 1 --2- 172.21.15.94:0/4122409161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f255815f3f0 0x7f25580b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f2556000640 1 -- 172.21.15.94:0/4122409161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25580cc2f0 msgr2=0x7f25580aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f2556000640 1 --2- 172.21.15.94:0/4122409161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25580cc2f0 0x7f25580aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f2556000640 1 -- 172.21.15.94:0/4122409161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2558166600 con 0x7f25580cdea0 2024-09-10T23:39:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f2556000640 1 --2- 172.21.15.94:0/4122409161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25580cdea0 0x7f2558159c90 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f254400c450 tx=0x7f254400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f254ea00640 1 -- 172.21.15.94:0/4122409161 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544017070 con 0x7f25580cdea0 2024-09-10T23:39:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f254ea00640 1 -- 172.21.15.94:0/4122409161 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2544003600 con 0x7f25580cdea0 2024-09-10T23:39:26.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.128+0000 7f254ea00640 1 -- 172.21.15.94:0/4122409161 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544003a50 con 0x7f25580cdea0 2024-09-10T23:39:26.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.132+0000 7f255d600640 1 -- 172.21.15.94:0/4122409161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25581668a0 con 0x7f25580cdea0 2024-09-10T23:39:26.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.132+0000 7f255d600640 1 -- 172.21.15.94:0/4122409161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25581fbf10 con 0x7f25580cdea0 2024-09-10T23:39:26.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.132+0000 7f255d600640 1 -- 172.21.15.94:0/4122409161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2558068990 con 0x7f25580cdea0 2024-09-10T23:39:26.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.132+0000 7f254ea00640 1 -- 172.21.15.94:0/4122409161 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f2544003bf0 con 0x7f25580cdea0 2024-09-10T23:39:26.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.132+0000 7f254ea00640 1 --2- 172.21.15.94:0/4122409161 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2538072ca0 0x7f2538075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:26.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.132+0000 7f254ea00640 1 -- 172.21.15.94:0/4122409161 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f2544093620 con 0x7f25580cdea0 2024-09-10T23:39:26.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.132+0000 7f2556a00640 1 --2- 172.21.15.94:0/4122409161 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2538072ca0 0x7f2538075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:26.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.132+0000 7f2556a00640 1 --2- 172.21.15.94:0/4122409161 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2538072ca0 0x7f2538075160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f25480af000 tx=0x7f254801afd0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:26.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.136+0000 7f254ea00640 1 -- 172.21.15.94:0/4122409161 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f254405dbe0 con 0x7f25580cdea0 2024-09-10T23:39:26.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.304+0000 7f255d600640 1 -- 172.21.15.94:0/4122409161 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25580b6250 con 0x7f25580cdea0 2024-09-10T23:39:26.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.304+0000 7f254ea00640 1 -- 172.21.15.94:0/4122409161 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+16784 (secure 0 0 0) 0x7f2544062610 con 0x7f25580cdea0 2024-09-10T23:39:26.308 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:26.308 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":137,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:39:24.990839+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:39:22.673102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T23:39:23.946666+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:39:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.308+0000 7f255d600640 1 -- 172.21.15.94:0/4122409161 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2538072ca0 msgr2=0x7f2538075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.308+0000 7f255d600640 1 --2- 172.21.15.94:0/4122409161 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2538072ca0 0x7f2538075160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f25480af000 tx=0x7f254801afd0 comp rx=0 tx=0).stop 2024-09-10T23:39:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.308+0000 7f255d600640 1 -- 172.21.15.94:0/4122409161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25580cdea0 msgr2=0x7f2558159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.308+0000 7f255d600640 1 --2- 172.21.15.94:0/4122409161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25580cdea0 0x7f2558159c90 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f254400c450 tx=0x7f254400c910 comp rx=0 tx=0).stop 2024-09-10T23:39:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.308+0000 7f255d600640 1 -- 172.21.15.94:0/4122409161 shutdown_connections 2024-09-10T23:39:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.308+0000 7f255d600640 1 --2- 172.21.15.94:0/4122409161 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2538072ca0 0x7f2538075160 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.308+0000 7f255d600640 1 --2- 172.21.15.94:0/4122409161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f255815f3f0 0x7f25580b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.308+0000 7f255d600640 1 --2- 172.21.15.94:0/4122409161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25580cdea0 0x7f2558159c90 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.308+0000 7f255d600640 1 --2- 172.21.15.94:0/4122409161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25580cc2f0 0x7f25580aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.308+0000 7f255d600640 1 -- 172.21.15.94:0/4122409161 >> 172.21.15.94:0/4122409161 conn(0x7f25580d02d0 msgr2=0x7f255806eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.308+0000 7f255d600640 1 -- 172.21.15.94:0/4122409161 shutdown_connections 2024-09-10T23:39:26.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.308+0000 7f255d600640 1 -- 172.21.15.94:0/4122409161 wait complete. 2024-09-10T23:39:26.330 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T23:39:26.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9269000640 1 -- 172.21.15.94:0/3113609991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92640b0160 msgr2=0x7f9264166940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:26.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9269000640 1 --2- 172.21.15.94:0/3113609991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92640b0160 0x7f9264166940 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f92640d1390 tx=0x7f925c01c7f0 comp rx=0 tx=0).stop 2024-09-10T23:39:26.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9269000640 1 -- 172.21.15.94:0/3113609991 shutdown_connections 2024-09-10T23:39:26.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9269000640 1 --2- 172.21.15.94:0/3113609991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92640b0160 0x7f9264166940 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:26.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9269000640 1 -- 172.21.15.94:0/3113609991 >> 172.21.15.94:0/3113609991 conn(0x7f92640d1b00 msgr2=0x7f92640d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:26.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9269000640 1 -- 172.21.15.94:0/3113609991 shutdown_connections 2024-09-10T23:39:26.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9269000640 1 -- 172.21.15.94:0/3113609991 wait complete. 2024-09-10T23:39:26.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9269000640 1 Processor -- start 2024-09-10T23:39:26.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9269000640 1 -- start start 2024-09-10T23:39:26.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9269000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92640b0160 0x7f9264163ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:26.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9269000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92640b2ac0 0x7f926406e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:26.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9269000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f926415f140 0x7f926415b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:26.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9269000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9264169a30 con 0x7f92640b2ac0 2024-09-10T23:39:26.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9269000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f92641698b0 con 0x7f92640b0160 2024-09-10T23:39:26.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9269000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9264169bb0 con 0x7f926415f140 2024-09-10T23:39:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f925aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92640b2ac0 0x7f926406e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f925aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92640b2ac0 0x7f926406e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50234/0 (socket says 172.21.15.94:50234) 2024-09-10T23:39:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f925aa00640 1 -- 172.21.15.94:0/2994423134 learned_addr learned my addr 172.21.15.94:0/2994423134 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9263400640 1 --2- 172.21.15.94:0/2994423134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92640b0160 0x7f9264163ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.472+0000 7f9263e00640 1 --2- 172.21.15.94:0/2994423134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f926415f140 0x7f926415b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.476+0000 7f925aa00640 1 -- 172.21.15.94:0/2994423134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f926415f140 msgr2=0x7f926415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.476+0000 7f925aa00640 1 --2- 172.21.15.94:0/2994423134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f926415f140 0x7f926415b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.476+0000 7f925aa00640 1 -- 172.21.15.94:0/2994423134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92640b0160 msgr2=0x7f9264163ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.476+0000 7f925aa00640 1 --2- 172.21.15.94:0/2994423134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92640b0160 0x7f9264163ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.476+0000 7f925aa00640 1 -- 172.21.15.94:0/2994423134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9264154be0 con 0x7f92640b2ac0 2024-09-10T23:39:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.476+0000 7f9263e00640 1 --2- 172.21.15.94:0/2994423134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f926415f140 0x7f926415b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.476+0000 7f925aa00640 1 --2- 172.21.15.94:0/2994423134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92640b2ac0 0x7f926406e740 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f924c00c480 tx=0x7f924c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:26.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.476+0000 7f925be00640 1 -- 172.21.15.94:0/2994423134 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c016020 con 0x7f92640b2ac0 2024-09-10T23:39:26.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.476+0000 7f925be00640 1 -- 172.21.15.94:0/2994423134 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f924c012920 con 0x7f92640b2ac0 2024-09-10T23:39:26.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.476+0000 7f925be00640 1 -- 172.21.15.94:0/2994423134 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c012d70 con 0x7f92640b2ac0 2024-09-10T23:39:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.476+0000 7f9269000640 1 -- 172.21.15.94:0/2994423134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9264154e80 con 0x7f92640b2ac0 2024-09-10T23:39:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.476+0000 7f9269000640 1 -- 172.21.15.94:0/2994423134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9264155440 con 0x7f92640b2ac0 2024-09-10T23:39:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.476+0000 7f9269000640 1 -- 172.21.15.94:0/2994423134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92640b1930 con 0x7f92640b2ac0 2024-09-10T23:39:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.476+0000 7f925be00640 1 -- 172.21.15.94:0/2994423134 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f924c025480 con 0x7f92640b2ac0 2024-09-10T23:39:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.476+0000 7f925be00640 1 --2- 172.21.15.94:0/2994423134 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9240072ca0 0x7f9240075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:26.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.476+0000 7f925be00640 1 -- 172.21.15.94:0/2994423134 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f924c094790 con 0x7f92640b2ac0 2024-09-10T23:39:26.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.480+0000 7f9263400640 1 --2- 172.21.15.94:0/2994423134 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9240072ca0 0x7f9240075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:26.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.480+0000 7f9263400640 1 --2- 172.21.15.94:0/2994423134 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9240072ca0 0x7f9240075160 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f925400b450 tx=0x7f925400d040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:26.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.480+0000 7f925be00640 1 -- 172.21.15.94:0/2994423134 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f924c020020 con 0x7f92640b2ac0 2024-09-10T23:39:26.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:26.684+0000 7f9269000640 1 -- 172.21.15.94:0/2994423134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f926414fd00 con 0x7f92640b2ac0 2024-09-10T23:39:27.581 INFO:tasks.ceph.mds.b.smithi181.stderr:2024-09-10T23:39:27.580+0000 7f983a000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:39:27.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.608+0000 7f925be00640 1 -- 172.21.15.94:0/2994423134 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v96) ==== 118+0+0 (secure 0 0 0) 0x7f924c05f3f0 con 0x7f92640b2ac0 2024-09-10T23:39:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.612+0000 7f9269000640 1 -- 172.21.15.94:0/2994423134 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9240072ca0 msgr2=0x7f9240075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.612+0000 7f9269000640 1 --2- 172.21.15.94:0/2994423134 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9240072ca0 0x7f9240075160 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f925400b450 tx=0x7f925400d040 comp rx=0 tx=0).stop 2024-09-10T23:39:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.612+0000 7f9269000640 1 -- 172.21.15.94:0/2994423134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92640b2ac0 msgr2=0x7f926406e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.612+0000 7f9269000640 1 --2- 172.21.15.94:0/2994423134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92640b2ac0 0x7f926406e740 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f924c00c480 tx=0x7f924c00c940 comp rx=0 tx=0).stop 2024-09-10T23:39:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.612+0000 7f9269000640 1 -- 172.21.15.94:0/2994423134 shutdown_connections 2024-09-10T23:39:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.612+0000 7f9269000640 1 --2- 172.21.15.94:0/2994423134 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9240072ca0 0x7f9240075160 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.612+0000 7f9269000640 1 --2- 172.21.15.94:0/2994423134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f926415f140 0x7f926415b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:27.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.612+0000 7f9269000640 1 --2- 172.21.15.94:0/2994423134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92640b2ac0 0x7f926406e740 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:27.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.612+0000 7f9269000640 1 --2- 172.21.15.94:0/2994423134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92640b0160 0x7f9264163ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:27.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.612+0000 7f9269000640 1 -- 172.21.15.94:0/2994423134 >> 172.21.15.94:0/2994423134 conn(0x7f92640d1b00 msgr2=0x7f92640b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:27.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.612+0000 7f9269000640 1 -- 172.21.15.94:0/2994423134 shutdown_connections 2024-09-10T23:39:27.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.612+0000 7f9269000640 1 -- 172.21.15.94:0/2994423134 wait complete. 2024-09-10T23:39:27.636 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T23:39:27.636 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T23:39:27.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff79be00640 1 -- 172.21.15.94:0/1353210929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7940c64f0 msgr2=0x7ff7940cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:27.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff79be00640 1 --2- 172.21.15.94:0/1353210929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7940c64f0 0x7ff7940cf280 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff794072510 tx=0x7ff78001c8c0 comp rx=0 tx=0).stop 2024-09-10T23:39:27.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff79be00640 1 -- 172.21.15.94:0/1353210929 shutdown_connections 2024-09-10T23:39:27.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff79be00640 1 --2- 172.21.15.94:0/1353210929 >> v2:172.21.15.181:3300/0 conn(0x7ff79406e7c0 0x7ff79406ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:27.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff79be00640 1 --2- 172.21.15.94:0/1353210929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7940c64f0 0x7ff7940cf280 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:27.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff79be00640 1 -- 172.21.15.94:0/1353210929 >> 172.21.15.94:0/1353210929 conn(0x7ff7940cfaf0 msgr2=0x7ff7940cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:27.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff79be00640 1 -- 172.21.15.94:0/1353210929 shutdown_connections 2024-09-10T23:39:27.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff79be00640 1 -- 172.21.15.94:0/1353210929 wait complete. 2024-09-10T23:39:27.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff79be00640 1 Processor -- start 2024-09-10T23:39:27.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff79be00640 1 -- start start 2024-09-10T23:39:27.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff79be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff79406e7c0 0x7ff7941f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:27.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff79be00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7940c64f0 0x7ff7940c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:27.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff79be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7940ce280 0x7ff7941f7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:27.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff79be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7941678f0 con 0x7ff79406e7c0 2024-09-10T23:39:27.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff79be00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff794167770 con 0x7ff7940ce280 2024-09-10T23:39:27.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff79be00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff794167a70 con 0x7ff7940c64f0 2024-09-10T23:39:27.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff799400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff79406e7c0 0x7ff7941f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:27.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff799400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff79406e7c0 0x7ff7941f85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50250/0 (socket says 172.21.15.94:50250) 2024-09-10T23:39:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff799400640 1 -- 172.21.15.94:0/3159492408 learned_addr learned my addr 172.21.15.94:0/3159492408 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff799e00640 1 --2- 172.21.15.94:0/3159492408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7940ce280 0x7ff7941f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff798a00640 1 --2- 172.21.15.94:0/3159492408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7940c64f0 0x7ff7940c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff799400640 1 -- 172.21.15.94:0/3159492408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7940c64f0 msgr2=0x7ff7940c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff799400640 1 --2- 172.21.15.94:0/3159492408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7940c64f0 0x7ff7940c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff799400640 1 -- 172.21.15.94:0/3159492408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7940ce280 msgr2=0x7ff7941f7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff799400640 1 --2- 172.21.15.94:0/3159492408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7940ce280 0x7ff7941f7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff799400640 1 -- 172.21.15.94:0/3159492408 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79415a730 con 0x7ff79406e7c0 2024-09-10T23:39:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.780+0000 7ff798a00640 1 --2- 172.21.15.94:0/3159492408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7940c64f0 0x7ff7940c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.784+0000 7ff799400640 1 --2- 172.21.15.94:0/3159492408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff79406e7c0 0x7ff7941f85e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff78800b410 tx=0x7ff78800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.784+0000 7ff791600640 1 -- 172.21.15.94:0/3159492408 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788015030 con 0x7ff79406e7c0 2024-09-10T23:39:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.784+0000 7ff791600640 1 -- 172.21.15.94:0/3159492408 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff788004020 con 0x7ff79406e7c0 2024-09-10T23:39:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.784+0000 7ff791600640 1 -- 172.21.15.94:0/3159492408 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7880044b0 con 0x7ff79406e7c0 2024-09-10T23:39:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.784+0000 7ff79be00640 1 -- 172.21.15.94:0/3159492408 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7940c9560 con 0x7ff79406e7c0 2024-09-10T23:39:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.784+0000 7ff79be00640 1 -- 172.21.15.94:0/3159492408 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7941f7de0 con 0x7ff79406e7c0 2024-09-10T23:39:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.784+0000 7ff79be00640 1 -- 172.21.15.94:0/3159492408 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff79406ebf0 con 0x7ff79406e7c0 2024-09-10T23:39:27.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.784+0000 7ff791600640 1 -- 172.21.15.94:0/3159492408 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7ff788022060 con 0x7ff79406e7c0 2024-09-10T23:39:27.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.784+0000 7ff791600640 1 --2- 172.21.15.94:0/3159492408 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff774072ca0 0x7ff774075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:27.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.784+0000 7ff791600640 1 -- 172.21.15.94:0/3159492408 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7ff788093ae0 con 0x7ff79406e7c0 2024-09-10T23:39:27.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.784+0000 7ff798a00640 1 --2- 172.21.15.94:0/3159492408 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff774072ca0 0x7ff774075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:27.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.788+0000 7ff798a00640 1 --2- 172.21.15.94:0/3159492408 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff774072ca0 0x7ff774075160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff7800af000 tx=0x7ff78001b100 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:27.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.788+0000 7ff791600640 1 -- 172.21.15.94:0/3159492408 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff78805e630 con 0x7ff79406e7c0 2024-09-10T23:39:27.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:27.992+0000 7ff79be00640 1 -- 172.21.15.94:0/3159492408 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff7940d0850 con 0x7ff79406e7c0 2024-09-10T23:39:28.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.620+0000 7ff791600640 1 -- 172.21.15.94:0/3159492408 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v97) ==== 122+0+0 (secure 0 0 0) 0x7ff78805df30 con 0x7ff79406e7c0 2024-09-10T23:39:28.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.620+0000 7ff79be00640 1 -- 172.21.15.94:0/3159492408 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff774072ca0 msgr2=0x7ff774075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:28.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.620+0000 7ff79be00640 1 --2- 172.21.15.94:0/3159492408 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff774072ca0 0x7ff774075160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff7800af000 tx=0x7ff78001b100 comp rx=0 tx=0).stop 2024-09-10T23:39:28.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.620+0000 7ff79be00640 1 -- 172.21.15.94:0/3159492408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff79406e7c0 msgr2=0x7ff7941f85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:28.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.620+0000 7ff79be00640 1 --2- 172.21.15.94:0/3159492408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff79406e7c0 0x7ff7941f85e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff78800b410 tx=0x7ff78800b8d0 comp rx=0 tx=0).stop 2024-09-10T23:39:28.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.620+0000 7ff79be00640 1 -- 172.21.15.94:0/3159492408 shutdown_connections 2024-09-10T23:39:28.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.620+0000 7ff79be00640 1 --2- 172.21.15.94:0/3159492408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7940ce280 0x7ff7941f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:28.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.620+0000 7ff79be00640 1 --2- 172.21.15.94:0/3159492408 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff774072ca0 0x7ff774075160 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:28.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.620+0000 7ff79be00640 1 --2- 172.21.15.94:0/3159492408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7940c64f0 0x7ff7940c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:28.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.620+0000 7ff79be00640 1 --2- 172.21.15.94:0/3159492408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff79406e7c0 0x7ff7941f85e0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:28.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.620+0000 7ff79be00640 1 -- 172.21.15.94:0/3159492408 >> 172.21.15.94:0/3159492408 conn(0x7ff7940cfaf0 msgr2=0x7ff7940c49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:28.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.620+0000 7ff79be00640 1 -- 172.21.15.94:0/3159492408 shutdown_connections 2024-09-10T23:39:28.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.620+0000 7ff79be00640 1 -- 172.21.15.94:0/3159492408 wait complete. 2024-09-10T23:39:28.646 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T23:39:28.646 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:39:28.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.788+0000 7f1efe600640 1 -- 172.21.15.94:0/1264620791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ef8073ff0 msgr2=0x7f1ef80cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.788+0000 7f1efe600640 1 --2- 172.21.15.94:0/1264620791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ef8073ff0 0x7f1ef80cfda0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009900 tx=0x7f1ee801c8e0 comp rx=0 tx=0).stop 2024-09-10T23:39:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.788+0000 7f1efe600640 1 -- 172.21.15.94:0/1264620791 shutdown_connections 2024-09-10T23:39:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.788+0000 7f1efe600640 1 --2- 172.21.15.94:0/1264620791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ef8073ff0 0x7f1ef80cfda0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.788+0000 7f1efe600640 1 -- 172.21.15.94:0/1264620791 >> 172.21.15.94:0/1264620791 conn(0x7f1ef806cdf0 msgr2=0x7f1ef806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.788+0000 7f1efe600640 1 -- 172.21.15.94:0/1264620791 shutdown_connections 2024-09-10T23:39:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.788+0000 7f1efe600640 1 -- 172.21.15.94:0/1264620791 wait complete. 2024-09-10T23:39:28.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.788+0000 7f1efe600640 1 Processor -- start 2024-09-10T23:39:28.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.788+0000 7f1efe600640 1 -- start start 2024-09-10T23:39:28.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.788+0000 7f1efe600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef8073ff0 0x7f1ef81f8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:28.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.788+0000 7f1efe600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef80d02e0 0x7f1ef813fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:28.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.788+0000 7f1efe600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ef815ee80 0x7f1ef8140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:28.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.788+0000 7f1efe600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef8168df0 con 0x7f1ef815ee80 2024-09-10T23:39:28.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.788+0000 7f1efe600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef8168c70 con 0x7f1ef80d02e0 2024-09-10T23:39:28.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.788+0000 7f1efe600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef8168f70 con 0x7f1ef8073ff0 2024-09-10T23:39:28.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1ef7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ef815ee80 0x7f1ef8140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1ef7400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef8073ff0 0x7f1ef81f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1ef7400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef8073ff0 0x7f1ef81f8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.94:48632/0 (socket says 172.21.15.94:48632) 2024-09-10T23:39:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1ef7400640 1 -- 172.21.15.94:0/3612152452 learned_addr learned my addr 172.21.15.94:0/3612152452 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1ef7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ef815ee80 0x7f1ef8140100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50268/0 (socket says 172.21.15.94:50268) 2024-09-10T23:39:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1eeea00640 1 --2- 172.21.15.94:0/3612152452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef80d02e0 0x7f1ef813fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1ef7e00640 1 -- 172.21.15.94:0/3612152452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef8073ff0 msgr2=0x7f1ef81f8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1ef7e00640 1 --2- 172.21.15.94:0/3612152452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef8073ff0 0x7f1ef81f8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1ef7e00640 1 -- 172.21.15.94:0/3612152452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef80d02e0 msgr2=0x7f1ef813fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1ef7e00640 1 --2- 172.21.15.94:0/3612152452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef80d02e0 0x7f1ef813fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1ef7e00640 1 -- 172.21.15.94:0/3612152452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef8150b10 con 0x7f1ef815ee80 2024-09-10T23:39:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1ef7400640 1 --2- 172.21.15.94:0/3612152452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef8073ff0 0x7f1ef81f8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:39:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1ef7e00640 1 --2- 172.21.15.94:0/3612152452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ef815ee80 0x7f1ef8140100 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0001e20 tx=0x7f1ef0010420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1eefe00640 1 -- 172.21.15.94:0/3612152452 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef000e0d0 con 0x7f1ef815ee80 2024-09-10T23:39:28.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1eefe00640 1 -- 172.21.15.94:0/3612152452 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ef000e270 con 0x7f1ef815ee80 2024-09-10T23:39:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1efe600640 1 -- 172.21.15.94:0/3612152452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef8150df0 con 0x7f1ef815ee80 2024-09-10T23:39:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1eefe00640 1 -- 172.21.15.94:0/3612152452 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0010d90 con 0x7f1ef815ee80 2024-09-10T23:39:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1efe600640 1 -- 172.21.15.94:0/3612152452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef81513b0 con 0x7f1ef815ee80 2024-09-10T23:39:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1efe600640 1 -- 172.21.15.94:0/3612152452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ef80cfda0 con 0x7f1ef815ee80 2024-09-10T23:39:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.792+0000 7f1eefe00640 1 -- 172.21.15.94:0/3612152452 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f1ef0016070 con 0x7f1ef815ee80 2024-09-10T23:39:28.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.796+0000 7f1eefe00640 1 --2- 172.21.15.94:0/3612152452 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1ed4072ca0 0x7f1ed4075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:28.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.796+0000 7f1eefe00640 1 -- 172.21.15.94:0/3612152452 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7f1ef00978d0 con 0x7f1ef815ee80 2024-09-10T23:39:28.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.796+0000 7f1ef7400640 1 --2- 172.21.15.94:0/3612152452 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1ed4072ca0 0x7f1ed4075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:28.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.796+0000 7f1ef7400640 1 --2- 172.21.15.94:0/3612152452 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1ed4072ca0 0x7f1ed4075160 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1ee801a510 tx=0x7f1ee801b120 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:28.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.800+0000 7f1eefe00640 1 -- 172.21.15.94:0/3612152452 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1ef0061f60 con 0x7f1ef815ee80 2024-09-10T23:39:28.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.996+0000 7f1efe600640 1 -- 172.21.15.94:0/3612152452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ef8050fb0 con 0x7f1ef815ee80 2024-09-10T23:39:29.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:28.996+0000 7f1eefe00640 1 -- 172.21.15.94:0/3612152452 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5231 (secure 0 0 0) 0x7f1ef0066990 con 0x7f1ef815ee80 2024-09-10T23:39:29.000 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:29.000 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":97,"btime":"2024-09-10T23:39:28:614931+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7864,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/188024700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":188024700},{"type":"v1","addr":"172.21.15.181:6835","nonce":188024700}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8376,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/35853213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":35853213},{"type":"v1","addr":"172.21.15.94:6834","nonce":35853213}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/800902310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":800902310},{"type":"v1","addr":"172.21.15.94:6835","nonce":800902310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:39:24.990806+0000","modified":"2024-09-10T23:39:27.995870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196874},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196874":{"gid":196874,"name":"b","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6801/2882789540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2882789540},{"type":"v1","addr":"172.21.15.181:6801","nonce":2882789540}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196874,"qdb_cluster":[196874]},"id":11}]} 2024-09-10T23:39:29.001 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 97 2024-09-10T23:39:29.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.000+0000 7f1efe600640 1 -- 172.21.15.94:0/3612152452 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1ed4072ca0 msgr2=0x7f1ed4075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:29.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.000+0000 7f1efe600640 1 --2- 172.21.15.94:0/3612152452 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1ed4072ca0 0x7f1ed4075160 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1ee801a510 tx=0x7f1ee801b120 comp rx=0 tx=0).stop 2024-09-10T23:39:29.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.000+0000 7f1efe600640 1 -- 172.21.15.94:0/3612152452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ef815ee80 msgr2=0x7f1ef8140100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:29.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.000+0000 7f1efe600640 1 --2- 172.21.15.94:0/3612152452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ef815ee80 0x7f1ef8140100 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0001e20 tx=0x7f1ef0010420 comp rx=0 tx=0).stop 2024-09-10T23:39:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.000+0000 7f1efe600640 1 -- 172.21.15.94:0/3612152452 shutdown_connections 2024-09-10T23:39:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.000+0000 7f1efe600640 1 --2- 172.21.15.94:0/3612152452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ef80d02e0 0x7f1ef813fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.000+0000 7f1efe600640 1 --2- 172.21.15.94:0/3612152452 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1ed4072ca0 0x7f1ed4075160 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.000+0000 7f1efe600640 1 --2- 172.21.15.94:0/3612152452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ef815ee80 0x7f1ef8140100 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.000+0000 7f1efe600640 1 --2- 172.21.15.94:0/3612152452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ef8073ff0 0x7f1ef81f8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.000+0000 7f1efe600640 1 -- 172.21.15.94:0/3612152452 >> 172.21.15.94:0/3612152452 conn(0x7f1ef806cdf0 msgr2=0x7f1ef80b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.000+0000 7f1efe600640 1 -- 172.21.15.94:0/3612152452 shutdown_connections 2024-09-10T23:39:29.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.000+0000 7f1efe600640 1 -- 172.21.15.94:0/3612152452 wait complete. 2024-09-10T23:39:29.021 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:39:29.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.160+0000 7fdd0d200640 1 -- 172.21.15.94:0/115423468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd08157c60 msgr2=0x7fdd08072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:29.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.160+0000 7fdd0d200640 1 --2- 172.21.15.94:0/115423468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd08157c60 0x7fdd08072610 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800aa10 tx=0x7fdcf801dba0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.160+0000 7fdd0d200640 1 -- 172.21.15.94:0/115423468 shutdown_connections 2024-09-10T23:39:29.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.160+0000 7fdd0d200640 1 --2- 172.21.15.94:0/115423468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd08157c60 0x7fdd08072610 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.160+0000 7fdd0d200640 1 -- 172.21.15.94:0/115423468 >> 172.21.15.94:0/115423468 conn(0x7fdd0806cdf0 msgr2=0x7fdd0806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:29.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.160+0000 7fdd0d200640 1 -- 172.21.15.94:0/115423468 shutdown_connections 2024-09-10T23:39:29.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.160+0000 7fdd0d200640 1 -- 172.21.15.94:0/115423468 wait complete. 2024-09-10T23:39:29.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.160+0000 7fdd0d200640 1 Processor -- start 2024-09-10T23:39:29.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.160+0000 7fdd0d200640 1 -- start start 2024-09-10T23:39:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd0d200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd0806f600 0x7fdd0814f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd0d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08070150 0x7fdd0815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd0d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd08157c60 0x7fdd0815d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd0d200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd081682d0 con 0x7fdd08157c60 2024-09-10T23:39:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd0d200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdd08168150 con 0x7fdd08070150 2024-09-10T23:39:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd0d200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fdd08168450 con 0x7fdd0806f600 2024-09-10T23:39:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd06800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd08157c60 0x7fdd0815d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd06800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd08157c60 0x7fdd0815d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50298/0 (socket says 172.21.15.94:50298) 2024-09-10T23:39:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd06800640 1 -- 172.21.15.94:0/2514606483 learned_addr learned my addr 172.21.15.94:0/2514606483 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd05400640 1 --2- 172.21.15.94:0/2514606483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08070150 0x7fdd0815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd05e00640 1 --2- 172.21.15.94:0/2514606483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd0806f600 0x7fdd0814f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd06800640 1 -- 172.21.15.94:0/2514606483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd0806f600 msgr2=0x7fdd0814f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd06800640 1 --2- 172.21.15.94:0/2514606483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd0806f600 0x7fdd0814f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd06800640 1 -- 172.21.15.94:0/2514606483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08070150 msgr2=0x7fdd0815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:29.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd06800640 1 --2- 172.21.15.94:0/2514606483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08070150 0x7fdd0815ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd06800640 1 -- 172.21.15.94:0/2514606483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd081f5f60 con 0x7fdd08157c60 2024-09-10T23:39:29.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd05400640 1 --2- 172.21.15.94:0/2514606483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08070150 0x7fdd0815ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:29.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd06800640 1 --2- 172.21.15.94:0/2514606483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd08157c60 0x7fdd0815d3f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000c780 tx=0x7fdd0000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:29.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdcfe000640 1 -- 172.21.15.94:0/2514606483 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00004b70 con 0x7fdd08157c60 2024-09-10T23:39:29.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdcfe000640 1 -- 172.21.15.94:0/2514606483 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd00004e00 con 0x7fdd08157c60 2024-09-10T23:39:29.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdcfe000640 1 -- 172.21.15.94:0/2514606483 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0000aab0 con 0x7fdd08157c60 2024-09-10T23:39:29.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd0d200640 1 -- 172.21.15.94:0/2514606483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0814f930 con 0x7fdd08157c60 2024-09-10T23:39:29.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd0d200640 1 -- 172.21.15.94:0/2514606483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd081f6500 con 0x7fdd08157c60 2024-09-10T23:39:29.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdd0d200640 1 -- 172.21.15.94:0/2514606483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd08072610 con 0x7fdd08157c60 2024-09-10T23:39:29.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.164+0000 7fdcfe000640 1 -- 172.21.15.94:0/2514606483 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7fdd0000ac50 con 0x7fdd08157c60 2024-09-10T23:39:29.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.168+0000 7fdcfe000640 1 --2- 172.21.15.94:0/2514606483 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdce8072ca0 0x7fdce8075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:29.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.168+0000 7fdcfe000640 1 -- 172.21.15.94:0/2514606483 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7fdd00094110 con 0x7fdd08157c60 2024-09-10T23:39:29.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.168+0000 7fdd05e00640 1 --2- 172.21.15.94:0/2514606483 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdce8072ca0 0x7fdce8075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:29.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.168+0000 7fdd05e00640 1 --2- 172.21.15.94:0/2514606483 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdce8072ca0 0x7fdce8075160 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80b0000 tx=0x7fdcf801c3e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:29.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.172+0000 7fdcfe000640 1 -- 172.21.15.94:0/2514606483 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdd00020070 con 0x7fdd08157c60 2024-09-10T23:39:29.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.336+0000 7fdd0d200640 1 -- 172.21.15.94:0/2514606483 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd080cbe70 con 0x7fdd08157c60 2024-09-10T23:39:29.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.336+0000 7fdcfe000640 1 -- 172.21.15.94:0/2514606483 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+16771 (secure 0 0 0) 0x7fdd0005ee80 con 0x7fdd08157c60 2024-09-10T23:39:29.341 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:29.341 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":138,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:39:26.602256+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:39:22.673102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T23:39:23.946666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:39:29.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.340+0000 7fdd0d200640 1 -- 172.21.15.94:0/2514606483 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdce8072ca0 msgr2=0x7fdce8075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:29.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.340+0000 7fdd0d200640 1 --2- 172.21.15.94:0/2514606483 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdce8072ca0 0x7fdce8075160 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80b0000 tx=0x7fdcf801c3e0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.340+0000 7fdd0d200640 1 -- 172.21.15.94:0/2514606483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd08157c60 msgr2=0x7fdd0815d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:29.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.340+0000 7fdd0d200640 1 --2- 172.21.15.94:0/2514606483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd08157c60 0x7fdd0815d3f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000c780 tx=0x7fdd0000cc40 comp rx=0 tx=0).stop 2024-09-10T23:39:29.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.340+0000 7fdd0d200640 1 -- 172.21.15.94:0/2514606483 shutdown_connections 2024-09-10T23:39:29.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.340+0000 7fdd0d200640 1 --2- 172.21.15.94:0/2514606483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd08070150 0x7fdd0815ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.340+0000 7fdd0d200640 1 --2- 172.21.15.94:0/2514606483 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdce8072ca0 0x7fdce8075160 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.340+0000 7fdd0d200640 1 --2- 172.21.15.94:0/2514606483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd08157c60 0x7fdd0815d3f0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.340+0000 7fdd0d200640 1 --2- 172.21.15.94:0/2514606483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd0806f600 0x7fdd0814f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.340+0000 7fdd0d200640 1 -- 172.21.15.94:0/2514606483 >> 172.21.15.94:0/2514606483 conn(0x7fdd0806cdf0 msgr2=0x7fdd0806eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:29.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.340+0000 7fdd0d200640 1 -- 172.21.15.94:0/2514606483 shutdown_connections 2024-09-10T23:39:29.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.340+0000 7fdd0d200640 1 -- 172.21.15.94:0/2514606483 wait complete. 2024-09-10T23:39:29.362 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T23:39:29.362 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:39:29.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.500+0000 7f252e000640 1 -- 172.21.15.94:0/2819929733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f250c0051d0 msgr2=0x7f250c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:29.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.500+0000 7f252e000640 1 --2- 172.21.15.94:0/2819929733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f250c0051d0 0x7f250c00b9d0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f2520009ee0 tx=0x7f252001d630 comp rx=0 tx=0).stop 2024-09-10T23:39:29.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.500+0000 7f2527e00640 1 -- 172.21.15.94:0/2819929733 reap_dead start 2024-09-10T23:39:29.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.500+0000 7f252e000640 1 -- 172.21.15.94:0/2819929733 shutdown_connections 2024-09-10T23:39:29.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.500+0000 7f252e000640 1 -- 172.21.15.94:0/2819929733 >> 172.21.15.94:0/2819929733 conn(0x7f25280d1c60 msgr2=0x7f25280d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:29.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.500+0000 7f252e000640 1 -- 172.21.15.94:0/2819929733 shutdown_connections 2024-09-10T23:39:29.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f252e000640 1 -- 172.21.15.94:0/2819929733 wait complete. 2024-09-10T23:39:29.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f252e000640 1 Processor -- start 2024-09-10T23:39:29.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f252e000640 1 -- start start 2024-09-10T23:39:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f252e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f250c0051d0 0x7f25281470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f252e000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2528151470 0x7f25280b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f252e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25280b4a90 0x7f252815caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f252e000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2528166e30 con 0x7f25280b4a90 2024-09-10T23:39:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f252e000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2528166cb0 con 0x7f250c0051d0 2024-09-10T23:39:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f252e000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2528166fb0 con 0x7f2528151470 2024-09-10T23:39:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f2527e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25280b4a90 0x7f252815caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f2527e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25280b4a90 0x7f252815caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50310/0 (socket says 172.21.15.94:50310) 2024-09-10T23:39:29.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f2527e00640 1 -- 172.21.15.94:0/2060743310 learned_addr learned my addr 172.21.15.94:0/2060743310 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f251fe00640 1 --2- 172.21.15.94:0/2060743310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2528151470 0x7f25280b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f2527400640 1 --2- 172.21.15.94:0/2060743310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f250c0051d0 0x7f25281470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f2527e00640 1 -- 172.21.15.94:0/2060743310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2528151470 msgr2=0x7f25280b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f2527e00640 1 --2- 172.21.15.94:0/2060743310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2528151470 0x7f25280b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f2527e00640 1 -- 172.21.15.94:0/2060743310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f250c0051d0 msgr2=0x7f25281470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f2527e00640 1 --2- 172.21.15.94:0/2060743310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f250c0051d0 0x7f25281470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f2527e00640 1 -- 172.21.15.94:0/2060743310 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2528151220 con 0x7f25280b4a90 2024-09-10T23:39:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f2527e00640 1 --2- 172.21.15.94:0/2060743310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25280b4a90 0x7f252815caa0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f2520009ee0 tx=0x7f25200ac9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f251f400640 1 -- 172.21.15.94:0/2060743310 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2520009ab0 con 0x7f25280b4a90 2024-09-10T23:39:29.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f251f400640 1 -- 172.21.15.94:0/2060743310 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25200ba070 con 0x7f25280b4a90 2024-09-10T23:39:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f251f400640 1 -- 172.21.15.94:0/2060743310 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25200acd50 con 0x7f25280b4a90 2024-09-10T23:39:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.504+0000 7f252e000640 1 -- 172.21.15.94:0/2060743310 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2528150f40 con 0x7f25280b4a90 2024-09-10T23:39:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.508+0000 7f252e000640 1 -- 172.21.15.94:0/2060743310 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25280cdf50 con 0x7f25280b4a90 2024-09-10T23:39:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.508+0000 7f252e000640 1 -- 172.21.15.94:0/2060743310 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25280af7c0 con 0x7f25280b4a90 2024-09-10T23:39:29.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.508+0000 7f251f400640 1 -- 172.21.15.94:0/2060743310 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f25200c7060 con 0x7f25280b4a90 2024-09-10T23:39:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.508+0000 7f251f400640 1 --2- 172.21.15.94:0/2060743310 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2504072cd0 0x7f2504075190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.508+0000 7f251f400640 1 -- 172.21.15.94:0/2060743310 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7f252012ef20 con 0x7f25280b4a90 2024-09-10T23:39:29.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.508+0000 7f2527400640 1 --2- 172.21.15.94:0/2060743310 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2504072cd0 0x7f2504075190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:29.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.508+0000 7f2527400640 1 --2- 172.21.15.94:0/2060743310 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2504072cd0 0x7f2504075190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2528166450 tx=0x7f25180091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:29.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.512+0000 7f251f400640 1 -- 172.21.15.94:0/2060743310 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2520017340 con 0x7f25280b4a90 2024-09-10T23:39:29.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.680+0000 7f252e000640 1 -- 172.21.15.94:0/2060743310 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25280d0510 con 0x7f2504072cd0 2024-09-10T23:39:29.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.684+0000 7f251f400640 1 -- 172.21.15.94:0/2060743310 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252479 (secure 0 0 0) 0x7f25280d0510 con 0x7f2504072cd0 2024-09-10T23:39:29.691 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:29.692 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:39:29.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.688+0000 7f252e000640 1 -- 172.21.15.94:0/2060743310 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2504072cd0 msgr2=0x7f2504075190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:29.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.688+0000 7f252e000640 1 --2- 172.21.15.94:0/2060743310 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2504072cd0 0x7f2504075190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2528166450 tx=0x7f25180091e0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.688+0000 7f252e000640 1 -- 172.21.15.94:0/2060743310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25280b4a90 msgr2=0x7f252815caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.688+0000 7f252e000640 1 --2- 172.21.15.94:0/2060743310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25280b4a90 0x7f252815caa0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f2520009ee0 tx=0x7f25200ac9e0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.688+0000 7f252e000640 1 -- 172.21.15.94:0/2060743310 shutdown_connections 2024-09-10T23:39:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.688+0000 7f252e000640 1 --2- 172.21.15.94:0/2060743310 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2504072cd0 0x7f2504075190 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.688+0000 7f252e000640 1 --2- 172.21.15.94:0/2060743310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25280b4a90 0x7f252815caa0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.688+0000 7f252e000640 1 --2- 172.21.15.94:0/2060743310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2528151470 0x7f25280b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.688+0000 7f252e000640 1 --2- 172.21.15.94:0/2060743310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f250c0051d0 0x7f25281470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.688+0000 7f252e000640 1 -- 172.21.15.94:0/2060743310 >> 172.21.15.94:0/2060743310 conn(0x7f25280d1c60 msgr2=0x7f25280708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.688+0000 7f252e000640 1 -- 172.21.15.94:0/2060743310 shutdown_connections 2024-09-10T23:39:29.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.688+0000 7f252e000640 1 -- 172.21.15.94:0/2060743310 wait complete. 2024-09-10T23:39:29.710 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-10T23:39:28.770028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268432,"kb_used_data":14728,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754706288,"statfs":{"total":773094113280,"available":772819238912,"internally_reserved":0,"allocated":15081472,"data_stored":7269748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.189738"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.624443+0000","last_change":"2024-09-10T23:39:25.610553+0000","last_active":"2024-09-10T23:39:26.624443+0000","last_peered":"2024-09-10T23:39:26.624443+0000","last_clean":"2024-09-10T23:39:26.624443+0000","last_became_active":"2024-09-10T23:39:25.610283+0000","last_became_peered":"2024-09-10T23:39:25.610283+0000","last_unstale":"2024-09-10T23:39:26.624443+0000","last_undegraded":"2024-09-10T23:39:26.624443+0000","last_fullsized":"2024-09-10T23:39:26.624443+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:16.475241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.624398+0000","last_change":"2024-09-10T23:39:25.613866+0000","last_active":"2024-09-10T23:39:26.624398+0000","last_peered":"2024-09-10T23:39:26.624398+0000","last_clean":"2024-09-10T23:39:26.624398+0000","last_became_active":"2024-09-10T23:39:25.613573+0000","last_became_peered":"2024-09-10T23:39:25.613573+0000","last_unstale":"2024-09-10T23:39:26.624398+0000","last_undegraded":"2024-09-10T23:39:26.624398+0000","last_fullsized":"2024-09-10T23:39:26.624398+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:18.794229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.624106+0000","last_change":"2024-09-10T23:39:25.612589+0000","last_active":"2024-09-10T23:39:26.624106+0000","last_peered":"2024-09-10T23:39:26.624106+0000","last_clean":"2024-09-10T23:39:26.624106+0000","last_became_active":"2024-09-10T23:39:25.611118+0000","last_became_peered":"2024-09-10T23:39:25.611118+0000","last_unstale":"2024-09-10T23:39:26.624106+0000","last_undegraded":"2024-09-10T23:39:26.624106+0000","last_fullsized":"2024-09-10T23:39:26.624106+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:16.981176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771709+0000","last_change":"2024-09-10T23:39:25.619277+0000","last_active":"2024-09-10T23:39:26.771709+0000","last_peered":"2024-09-10T23:39:26.771709+0000","last_clean":"2024-09-10T23:39:26.771709+0000","last_became_active":"2024-09-10T23:39:25.618895+0000","last_became_peered":"2024-09-10T23:39:25.618895+0000","last_unstale":"2024-09-10T23:39:26.771709+0000","last_undegraded":"2024-09-10T23:39:26.771709+0000","last_fullsized":"2024-09-10T23:39:26.771709+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:59.862302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771069+0000","last_change":"2024-09-10T23:39:25.609793+0000","last_active":"2024-09-10T23:39:26.771069+0000","last_peered":"2024-09-10T23:39:26.771069+0000","last_clean":"2024-09-10T23:39:26.771069+0000","last_became_active":"2024-09-10T23:39:25.609547+0000","last_became_peered":"2024-09-10T23:39:25.609547+0000","last_unstale":"2024-09-10T23:39:26.771069+0000","last_undegraded":"2024-09-10T23:39:26.771069+0000","last_fullsized":"2024-09-10T23:39:26.771069+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:45.535562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771185+0000","last_change":"2024-09-10T23:39:25.629619+0000","last_active":"2024-09-10T23:39:26.771185+0000","last_peered":"2024-09-10T23:39:26.771185+0000","last_clean":"2024-09-10T23:39:26.771185+0000","last_became_active":"2024-09-10T23:39:25.628881+0000","last_became_peered":"2024-09-10T23:39:25.628881+0000","last_unstale":"2024-09-10T23:39:26.771185+0000","last_undegraded":"2024-09-10T23:39:26.771185+0000","last_fullsized":"2024-09-10T23:39:26.771185+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:37.624879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.612659+0000","last_change":"2024-09-10T23:39:25.606249+0000","last_active":"2024-09-10T23:39:26.612659+0000","last_peered":"2024-09-10T23:39:26.612659+0000","last_clean":"2024-09-10T23:39:26.612659+0000","last_became_active":"2024-09-10T23:39:25.605661+0000","last_became_peered":"2024-09-10T23:39:25.605661+0000","last_unstale":"2024-09-10T23:39:26.612659+0000","last_undegraded":"2024-09-10T23:39:26.612659+0000","last_fullsized":"2024-09-10T23:39:26.612659+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:58.656048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770480+0000","last_change":"2024-09-10T23:39:25.611265+0000","last_active":"2024-09-10T23:39:26.770480+0000","last_peered":"2024-09-10T23:39:26.770480+0000","last_clean":"2024-09-10T23:39:26.770480+0000","last_became_active":"2024-09-10T23:39:25.610987+0000","last_became_peered":"2024-09-10T23:39:25.610987+0000","last_unstale":"2024-09-10T23:39:26.770480+0000","last_undegraded":"2024-09-10T23:39:26.770480+0000","last_fullsized":"2024-09-10T23:39:26.770480+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:09.732712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770429+0000","last_change":"2024-09-10T23:39:25.615276+0000","last_active":"2024-09-10T23:39:26.770429+0000","last_peered":"2024-09-10T23:39:26.770429+0000","last_clean":"2024-09-10T23:39:26.770429+0000","last_became_active":"2024-09-10T23:39:25.615026+0000","last_became_peered":"2024-09-10T23:39:25.615026+0000","last_unstale":"2024-09-10T23:39:26.770429+0000","last_undegraded":"2024-09-10T23:39:26.770429+0000","last_fullsized":"2024-09-10T23:39:26.770429+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:16.919508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770407+0000","last_change":"2024-09-10T23:39:25.619582+0000","last_active":"2024-09-10T23:39:26.770407+0000","last_peered":"2024-09-10T23:39:26.770407+0000","last_clean":"2024-09-10T23:39:26.770407+0000","last_became_active":"2024-09-10T23:39:25.619281+0000","last_became_peered":"2024-09-10T23:39:25.619281+0000","last_unstale":"2024-09-10T23:39:26.770407+0000","last_undegraded":"2024-09-10T23:39:26.770407+0000","last_fullsized":"2024-09-10T23:39:26.770407+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:19:30.378031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771271+0000","last_change":"2024-09-10T23:39:25.616973+0000","last_active":"2024-09-10T23:39:26.771271+0000","last_peered":"2024-09-10T23:39:26.771271+0000","last_clean":"2024-09-10T23:39:26.771271+0000","last_became_active":"2024-09-10T23:39:25.616717+0000","last_became_peered":"2024-09-10T23:39:25.616717+0000","last_unstale":"2024-09-10T23:39:26.771271+0000","last_undegraded":"2024-09-10T23:39:26.771271+0000","last_fullsized":"2024-09-10T23:39:26.771271+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:11.117207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.611160+0000","last_change":"2024-09-10T23:39:25.606766+0000","last_active":"2024-09-10T23:39:26.611160+0000","last_peered":"2024-09-10T23:39:26.611160+0000","last_clean":"2024-09-10T23:39:26.611160+0000","last_became_active":"2024-09-10T23:39:25.606515+0000","last_became_peered":"2024-09-10T23:39:25.606515+0000","last_unstale":"2024-09-10T23:39:26.611160+0000","last_undegraded":"2024-09-10T23:39:26.611160+0000","last_fullsized":"2024-09-10T23:39:26.611160+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:51.941533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.611101+0000","last_change":"2024-09-10T23:39:25.615546+0000","last_active":"2024-09-10T23:39:26.611101+0000","last_peered":"2024-09-10T23:39:26.611101+0000","last_clean":"2024-09-10T23:39:26.611101+0000","last_became_active":"2024-09-10T23:39:25.615088+0000","last_became_peered":"2024-09-10T23:39:25.615088+0000","last_unstale":"2024-09-10T23:39:26.611101+0000","last_undegraded":"2024-09-10T23:39:26.611101+0000","last_fullsized":"2024-09-10T23:39:26.611101+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:41.962177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770706+0000","last_change":"2024-09-10T23:39:25.612373+0000","last_active":"2024-09-10T23:39:26.770706+0000","last_peered":"2024-09-10T23:39:26.770706+0000","last_clean":"2024-09-10T23:39:26.770706+0000","last_became_active":"2024-09-10T23:39:25.612093+0000","last_became_peered":"2024-09-10T23:39:25.612093+0000","last_unstale":"2024-09-10T23:39:26.770706+0000","last_undegraded":"2024-09-10T23:39:26.770706+0000","last_fullsized":"2024-09-10T23:39:26.770706+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:16.475241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770475+0000","last_change":"2024-09-10T23:39:25.616597+0000","last_active":"2024-09-10T23:39:26.770475+0000","last_peered":"2024-09-10T23:39:26.770475+0000","last_clean":"2024-09-10T23:39:26.770475+0000","last_became_active":"2024-09-10T23:39:25.616307+0000","last_became_peered":"2024-09-10T23:39:25.616307+0000","last_unstale":"2024-09-10T23:39:26.770475+0000","last_undegraded":"2024-09-10T23:39:26.770475+0000","last_fullsized":"2024-09-10T23:39:26.770475+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:02.049706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.612200+0000","last_change":"2024-09-10T23:39:25.609362+0000","last_active":"2024-09-10T23:39:26.612200+0000","last_peered":"2024-09-10T23:39:26.612200+0000","last_clean":"2024-09-10T23:39:26.612200+0000","last_became_active":"2024-09-10T23:39:25.608449+0000","last_became_peered":"2024-09-10T23:39:25.608449+0000","last_unstale":"2024-09-10T23:39:26.612200+0000","last_undegraded":"2024-09-10T23:39:26.612200+0000","last_fullsized":"2024-09-10T23:39:26.612200+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:37.706104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771137+0000","last_change":"2024-09-10T23:39:25.621241+0000","last_active":"2024-09-10T23:39:26.771137+0000","last_peered":"2024-09-10T23:39:26.771137+0000","last_clean":"2024-09-10T23:39:26.771137+0000","last_became_active":"2024-09-10T23:39:25.620810+0000","last_became_peered":"2024-09-10T23:39:25.620810+0000","last_unstale":"2024-09-10T23:39:26.771137+0000","last_undegraded":"2024-09-10T23:39:26.771137+0000","last_fullsized":"2024-09-10T23:39:26.771137+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:30.090147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771656+0000","last_change":"2024-09-10T23:39:25.602781+0000","last_active":"2024-09-10T23:39:26.771656+0000","last_peered":"2024-09-10T23:39:26.771656+0000","last_clean":"2024-09-10T23:39:26.771656+0000","last_became_active":"2024-09-10T23:39:25.602117+0000","last_became_peered":"2024-09-10T23:39:25.602117+0000","last_unstale":"2024-09-10T23:39:26.771656+0000","last_undegraded":"2024-09-10T23:39:26.771656+0000","last_fullsized":"2024-09-10T23:39:26.771656+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:14.404875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613051+0000","last_change":"2024-09-10T23:39:25.618087+0000","last_active":"2024-09-10T23:39:26.613051+0000","last_peered":"2024-09-10T23:39:26.613051+0000","last_clean":"2024-09-10T23:39:26.613051+0000","last_became_active":"2024-09-10T23:39:25.617217+0000","last_became_peered":"2024-09-10T23:39:25.617217+0000","last_unstale":"2024-09-10T23:39:26.613051+0000","last_undegraded":"2024-09-10T23:39:26.613051+0000","last_fullsized":"2024-09-10T23:39:26.613051+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:18.469217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.611255+0000","last_change":"2024-09-10T23:39:25.605569+0000","last_active":"2024-09-10T23:39:26.611255+0000","last_peered":"2024-09-10T23:39:26.611255+0000","last_clean":"2024-09-10T23:39:26.611255+0000","last_became_active":"2024-09-10T23:39:25.605322+0000","last_became_peered":"2024-09-10T23:39:25.605322+0000","last_unstale":"2024-09-10T23:39:26.611255+0000","last_undegraded":"2024-09-10T23:39:26.611255+0000","last_fullsized":"2024-09-10T23:39:26.611255+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:19.954076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770551+0000","last_change":"2024-09-10T23:39:25.620120+0000","last_active":"2024-09-10T23:39:26.770551+0000","last_peered":"2024-09-10T23:39:26.770551+0000","last_clean":"2024-09-10T23:39:26.770551+0000","last_became_active":"2024-09-10T23:39:25.619814+0000","last_became_peered":"2024-09-10T23:39:25.619814+0000","last_unstale":"2024-09-10T23:39:26.770551+0000","last_undegraded":"2024-09-10T23:39:26.770551+0000","last_fullsized":"2024-09-10T23:39:26.770551+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:35.027835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771062+0000","last_change":"2024-09-10T23:39:25.617617+0000","last_active":"2024-09-10T23:39:26.771062+0000","last_peered":"2024-09-10T23:39:26.771062+0000","last_clean":"2024-09-10T23:39:26.771062+0000","last_became_active":"2024-09-10T23:39:25.617366+0000","last_became_peered":"2024-09-10T23:39:25.617366+0000","last_unstale":"2024-09-10T23:39:26.771062+0000","last_undegraded":"2024-09-10T23:39:26.771062+0000","last_fullsized":"2024-09-10T23:39:26.771062+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:10.722696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613658+0000","last_change":"2024-09-10T23:39:25.614289+0000","last_active":"2024-09-10T23:39:26.613658+0000","last_peered":"2024-09-10T23:39:26.613658+0000","last_clean":"2024-09-10T23:39:26.613658+0000","last_became_active":"2024-09-10T23:39:25.613951+0000","last_became_peered":"2024-09-10T23:39:25.613951+0000","last_unstale":"2024-09-10T23:39:26.613658+0000","last_undegraded":"2024-09-10T23:39:26.613658+0000","last_fullsized":"2024-09-10T23:39:26.613658+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:57.800986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770797+0000","last_change":"2024-09-10T23:39:25.606092+0000","last_active":"2024-09-10T23:39:26.770797+0000","last_peered":"2024-09-10T23:39:26.770797+0000","last_clean":"2024-09-10T23:39:26.770797+0000","last_became_active":"2024-09-10T23:39:25.605840+0000","last_became_peered":"2024-09-10T23:39:25.605840+0000","last_unstale":"2024-09-10T23:39:26.770797+0000","last_undegraded":"2024-09-10T23:39:26.770797+0000","last_fullsized":"2024-09-10T23:39:26.770797+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:10.795879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771241+0000","last_change":"2024-09-10T23:39:25.604741+0000","last_active":"2024-09-10T23:39:26.771241+0000","last_peered":"2024-09-10T23:39:26.771241+0000","last_clean":"2024-09-10T23:39:26.771241+0000","last_became_active":"2024-09-10T23:39:25.604500+0000","last_became_peered":"2024-09-10T23:39:25.604500+0000","last_unstale":"2024-09-10T23:39:26.771241+0000","last_undegraded":"2024-09-10T23:39:26.771241+0000","last_fullsized":"2024-09-10T23:39:26.771241+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:48.574961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.614155+0000","last_change":"2024-09-10T23:39:25.628400+0000","last_active":"2024-09-10T23:39:26.614155+0000","last_peered":"2024-09-10T23:39:26.614155+0000","last_clean":"2024-09-10T23:39:26.614155+0000","last_became_active":"2024-09-10T23:39:25.628145+0000","last_became_peered":"2024-09-10T23:39:25.628145+0000","last_unstale":"2024-09-10T23:39:26.614155+0000","last_undegraded":"2024-09-10T23:39:26.614155+0000","last_fullsized":"2024-09-10T23:39:26.614155+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:16.865263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771391+0000","last_change":"2024-09-10T23:39:25.601870+0000","last_active":"2024-09-10T23:39:26.771391+0000","last_peered":"2024-09-10T23:39:26.771391+0000","last_clean":"2024-09-10T23:39:26.771391+0000","last_became_active":"2024-09-10T23:39:25.601626+0000","last_became_peered":"2024-09-10T23:39:25.601626+0000","last_unstale":"2024-09-10T23:39:26.771391+0000","last_undegraded":"2024-09-10T23:39:26.771391+0000","last_fullsized":"2024-09-10T23:39:26.771391+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:58.274675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613082+0000","last_change":"2024-09-10T23:39:25.610039+0000","last_active":"2024-09-10T23:39:26.613082+0000","last_peered":"2024-09-10T23:39:26.613082+0000","last_clean":"2024-09-10T23:39:26.613082+0000","last_became_active":"2024-09-10T23:39:25.609750+0000","last_became_peered":"2024-09-10T23:39:25.609750+0000","last_unstale":"2024-09-10T23:39:26.613082+0000","last_undegraded":"2024-09-10T23:39:26.613082+0000","last_fullsized":"2024-09-10T23:39:26.613082+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:04.284460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623971+0000","last_change":"2024-09-10T23:39:25.604568+0000","last_active":"2024-09-10T23:39:26.623971+0000","last_peered":"2024-09-10T23:39:26.623971+0000","last_clean":"2024-09-10T23:39:26.623971+0000","last_became_active":"2024-09-10T23:39:25.602741+0000","last_became_peered":"2024-09-10T23:39:25.602741+0000","last_unstale":"2024-09-10T23:39:26.623971+0000","last_undegraded":"2024-09-10T23:39:26.623971+0000","last_fullsized":"2024-09-10T23:39:26.623971+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:02.049706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770660+0000","last_change":"2024-09-10T23:39:25.612221+0000","last_active":"2024-09-10T23:39:26.770660+0000","last_peered":"2024-09-10T23:39:26.770660+0000","last_clean":"2024-09-10T23:39:26.770660+0000","last_became_active":"2024-09-10T23:39:25.611960+0000","last_became_peered":"2024-09-10T23:39:25.611960+0000","last_unstale":"2024-09-10T23:39:26.770660+0000","last_undegraded":"2024-09-10T23:39:26.770660+0000","last_fullsized":"2024-09-10T23:39:26.770660+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:16.865263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623641+0000","last_change":"2024-09-10T23:39:25.605125+0000","last_active":"2024-09-10T23:39:26.623641+0000","last_peered":"2024-09-10T23:39:26.623641+0000","last_clean":"2024-09-10T23:39:26.623641+0000","last_became_active":"2024-09-10T23:39:25.604846+0000","last_became_peered":"2024-09-10T23:39:25.604846+0000","last_unstale":"2024-09-10T23:39:26.623641+0000","last_undegraded":"2024-09-10T23:39:26.623641+0000","last_fullsized":"2024-09-10T23:39:26.623641+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:04:09.326143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770570+0000","last_change":"2024-09-10T23:39:25.620896+0000","last_active":"2024-09-10T23:39:26.770570+0000","last_peered":"2024-09-10T23:39:26.770570+0000","last_clean":"2024-09-10T23:39:26.770570+0000","last_became_active":"2024-09-10T23:39:25.620458+0000","last_became_peered":"2024-09-10T23:39:25.620458+0000","last_unstale":"2024-09-10T23:39:26.770570+0000","last_undegraded":"2024-09-10T23:39:26.770570+0000","last_fullsized":"2024-09-10T23:39:26.770570+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:10.131156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771826+0000","last_change":"2024-09-10T23:39:25.622751+0000","last_active":"2024-09-10T23:39:26.771826+0000","last_peered":"2024-09-10T23:39:26.771826+0000","last_clean":"2024-09-10T23:39:26.771826+0000","last_became_active":"2024-09-10T23:39:25.622298+0000","last_became_peered":"2024-09-10T23:39:25.622298+0000","last_unstale":"2024-09-10T23:39:26.771826+0000","last_undegraded":"2024-09-10T23:39:26.771826+0000","last_fullsized":"2024-09-10T23:39:26.771826+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:15.498532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"137'4","reported_seq":19,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.624511+0000","last_change":"2024-09-10T23:39:24.613693+0000","last_active":"2024-09-10T23:39:26.624511+0000","last_peered":"2024-09-10T23:39:26.624511+0000","last_clean":"2024-09-10T23:39:26.624511+0000","last_became_active":"2024-09-10T23:39:24.613434+0000","last_became_peered":"2024-09-10T23:39:24.613434+0000","last_unstale":"2024-09-10T23:39:26.624511+0000","last_undegraded":"2024-09-10T23:39:26.624511+0000","last_fullsized":"2024-09-10T23:39:26.624511+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:12.622316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.624451+0000","last_change":"2024-09-10T23:39:25.611809+0000","last_active":"2024-09-10T23:39:26.624451+0000","last_peered":"2024-09-10T23:39:26.624451+0000","last_clean":"2024-09-10T23:39:26.624451+0000","last_became_active":"2024-09-10T23:39:25.611469+0000","last_became_peered":"2024-09-10T23:39:25.611469+0000","last_unstale":"2024-09-10T23:39:26.624451+0000","last_undegraded":"2024-09-10T23:39:26.624451+0000","last_fullsized":"2024-09-10T23:39:26.624451+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:37.443789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770571+0000","last_change":"2024-09-10T23:39:24.607480+0000","last_active":"2024-09-10T23:39:26.770571+0000","last_peered":"2024-09-10T23:39:26.770571+0000","last_clean":"2024-09-10T23:39:26.770571+0000","last_became_active":"2024-09-10T23:39:24.607219+0000","last_became_peered":"2024-09-10T23:39:24.607219+0000","last_unstale":"2024-09-10T23:39:26.770571+0000","last_undegraded":"2024-09-10T23:39:26.770571+0000","last_fullsized":"2024-09-10T23:39:26.770571+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:58.995560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771031+0000","last_change":"2024-09-10T23:39:25.622142+0000","last_active":"2024-09-10T23:39:26.771031+0000","last_peered":"2024-09-10T23:39:26.771031+0000","last_clean":"2024-09-10T23:39:26.771031+0000","last_became_active":"2024-09-10T23:39:25.621847+0000","last_became_peered":"2024-09-10T23:39:25.621847+0000","last_unstale":"2024-09-10T23:39:26.771031+0000","last_undegraded":"2024-09-10T23:39:26.771031+0000","last_fullsized":"2024-09-10T23:39:26.771031+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:13:03.413049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770835+0000","last_change":"2024-09-10T23:39:24.614764+0000","last_active":"2024-09-10T23:39:26.770835+0000","last_peered":"2024-09-10T23:39:26.770835+0000","last_clean":"2024-09-10T23:39:26.770835+0000","last_became_active":"2024-09-10T23:39:24.614433+0000","last_became_peered":"2024-09-10T23:39:24.614433+0000","last_unstale":"2024-09-10T23:39:26.770835+0000","last_undegraded":"2024-09-10T23:39:26.770835+0000","last_fullsized":"2024-09-10T23:39:26.770835+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:30.711587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770360+0000","last_change":"2024-09-10T23:39:25.620932+0000","last_active":"2024-09-10T23:39:26.770360+0000","last_peered":"2024-09-10T23:39:26.770360+0000","last_clean":"2024-09-10T23:39:26.770360+0000","last_became_active":"2024-09-10T23:39:25.620674+0000","last_became_peered":"2024-09-10T23:39:25.620674+0000","last_unstale":"2024-09-10T23:39:26.770360+0000","last_undegraded":"2024-09-10T23:39:26.770360+0000","last_fullsized":"2024-09-10T23:39:26.770360+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:04:09.326143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613713+0000","last_change":"2024-09-10T23:39:24.613135+0000","last_active":"2024-09-10T23:39:26.613713+0000","last_peered":"2024-09-10T23:39:26.613713+0000","last_clean":"2024-09-10T23:39:26.613713+0000","last_became_active":"2024-09-10T23:39:24.612820+0000","last_became_peered":"2024-09-10T23:39:24.612820+0000","last_unstale":"2024-09-10T23:39:26.613713+0000","last_undegraded":"2024-09-10T23:39:26.613713+0000","last_fullsized":"2024-09-10T23:39:26.613713+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:52.165161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771368+0000","last_change":"2024-09-10T23:39:25.615602+0000","last_active":"2024-09-10T23:39:26.771368+0000","last_peered":"2024-09-10T23:39:26.771368+0000","last_clean":"2024-09-10T23:39:26.771368+0000","last_became_active":"2024-09-10T23:39:25.615326+0000","last_became_peered":"2024-09-10T23:39:25.615326+0000","last_unstale":"2024-09-10T23:39:26.771368+0000","last_undegraded":"2024-09-10T23:39:26.771368+0000","last_fullsized":"2024-09-10T23:39:26.771368+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:50.984162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.624061+0000","last_change":"2024-09-10T23:39:24.611866+0000","last_active":"2024-09-10T23:39:26.624061+0000","last_peered":"2024-09-10T23:39:26.624061+0000","last_clean":"2024-09-10T23:39:26.624061+0000","last_became_active":"2024-09-10T23:39:24.609939+0000","last_became_peered":"2024-09-10T23:39:24.609939+0000","last_unstale":"2024-09-10T23:39:26.624061+0000","last_undegraded":"2024-09-10T23:39:26.624061+0000","last_fullsized":"2024-09-10T23:39:26.624061+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:09:08.126983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770754+0000","last_change":"2024-09-10T23:39:25.628310+0000","last_active":"2024-09-10T23:39:26.770754+0000","last_peered":"2024-09-10T23:39:26.770754+0000","last_clean":"2024-09-10T23:39:26.770754+0000","last_became_active":"2024-09-10T23:39:25.625779+0000","last_became_peered":"2024-09-10T23:39:25.625779+0000","last_unstale":"2024-09-10T23:39:26.770754+0000","last_undegraded":"2024-09-10T23:39:26.770754+0000","last_fullsized":"2024-09-10T23:39:26.770754+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:54.404349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771361+0000","last_change":"2024-09-10T23:39:24.611667+0000","last_active":"2024-09-10T23:39:26.771361+0000","last_peered":"2024-09-10T23:39:26.771361+0000","last_clean":"2024-09-10T23:39:26.771361+0000","last_became_active":"2024-09-10T23:39:24.611382+0000","last_became_peered":"2024-09-10T23:39:24.611382+0000","last_unstale":"2024-09-10T23:39:26.771361+0000","last_undegraded":"2024-09-10T23:39:26.771361+0000","last_fullsized":"2024-09-10T23:39:26.771361+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:51:30.734026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.612258+0000","last_change":"2024-09-10T23:39:25.614231+0000","last_active":"2024-09-10T23:39:26.612258+0000","last_peered":"2024-09-10T23:39:26.612258+0000","last_clean":"2024-09-10T23:39:26.612258+0000","last_became_active":"2024-09-10T23:39:25.613918+0000","last_became_peered":"2024-09-10T23:39:25.613918+0000","last_unstale":"2024-09-10T23:39:26.612258+0000","last_undegraded":"2024-09-10T23:39:26.612258+0000","last_fullsized":"2024-09-10T23:39:26.612258+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:24.696106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623699+0000","last_change":"2024-09-10T23:39:24.613602+0000","last_active":"2024-09-10T23:39:26.623699+0000","last_peered":"2024-09-10T23:39:26.623699+0000","last_clean":"2024-09-10T23:39:26.623699+0000","last_became_active":"2024-09-10T23:39:24.613218+0000","last_became_peered":"2024-09-10T23:39:24.613218+0000","last_unstale":"2024-09-10T23:39:26.623699+0000","last_undegraded":"2024-09-10T23:39:26.623699+0000","last_fullsized":"2024-09-10T23:39:26.623699+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:11.204076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770475+0000","last_change":"2024-09-10T23:39:25.608322+0000","last_active":"2024-09-10T23:39:26.770475+0000","last_peered":"2024-09-10T23:39:26.770475+0000","last_clean":"2024-09-10T23:39:26.770475+0000","last_became_active":"2024-09-10T23:39:25.608045+0000","last_became_peered":"2024-09-10T23:39:25.608045+0000","last_unstale":"2024-09-10T23:39:26.770475+0000","last_undegraded":"2024-09-10T23:39:26.770475+0000","last_fullsized":"2024-09-10T23:39:26.770475+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:21:26.921897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613054+0000","last_change":"2024-09-10T23:39:24.613507+0000","last_active":"2024-09-10T23:39:26.613054+0000","last_peered":"2024-09-10T23:39:26.613054+0000","last_clean":"2024-09-10T23:39:26.613054+0000","last_became_active":"2024-09-10T23:39:24.613241+0000","last_became_peered":"2024-09-10T23:39:24.613241+0000","last_unstale":"2024-09-10T23:39:26.613054+0000","last_undegraded":"2024-09-10T23:39:26.613054+0000","last_fullsized":"2024-09-10T23:39:26.613054+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:03.251113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613215+0000","last_change":"2024-09-10T23:39:25.620619+0000","last_active":"2024-09-10T23:39:26.613215+0000","last_peered":"2024-09-10T23:39:26.613215+0000","last_clean":"2024-09-10T23:39:26.613215+0000","last_became_active":"2024-09-10T23:39:25.620354+0000","last_became_peered":"2024-09-10T23:39:25.620354+0000","last_unstale":"2024-09-10T23:39:26.613215+0000","last_undegraded":"2024-09-10T23:39:26.613215+0000","last_fullsized":"2024-09-10T23:39:26.613215+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:11.219480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770943+0000","last_change":"2024-09-10T23:39:24.617299+0000","last_active":"2024-09-10T23:39:26.770943+0000","last_peered":"2024-09-10T23:39:26.770943+0000","last_clean":"2024-09-10T23:39:26.770943+0000","last_became_active":"2024-09-10T23:39:24.616499+0000","last_became_peered":"2024-09-10T23:39:24.616499+0000","last_unstale":"2024-09-10T23:39:26.770943+0000","last_undegraded":"2024-09-10T23:39:26.770943+0000","last_fullsized":"2024-09-10T23:39:26.770943+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:38.774810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613213+0000","last_change":"2024-09-10T23:39:25.626046+0000","last_active":"2024-09-10T23:39:26.613213+0000","last_peered":"2024-09-10T23:39:26.613213+0000","last_clean":"2024-09-10T23:39:26.613213+0000","last_became_active":"2024-09-10T23:39:25.625710+0000","last_became_peered":"2024-09-10T23:39:25.625710+0000","last_unstale":"2024-09-10T23:39:26.613213+0000","last_undegraded":"2024-09-10T23:39:26.613213+0000","last_fullsized":"2024-09-10T23:39:26.613213+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:11.300959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"137'3","reported_seq":18,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613953+0000","last_change":"2024-09-10T23:39:24.615133+0000","last_active":"2024-09-10T23:39:26.613953+0000","last_peered":"2024-09-10T23:39:26.613953+0000","last_clean":"2024-09-10T23:39:26.613953+0000","last_became_active":"2024-09-10T23:39:24.614835+0000","last_became_peered":"2024-09-10T23:39:24.614835+0000","last_unstale":"2024-09-10T23:39:26.613953+0000","last_undegraded":"2024-09-10T23:39:26.613953+0000","last_fullsized":"2024-09-10T23:39:26.613953+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:30.704948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.624143+0000","last_change":"2024-09-10T23:39:25.604223+0000","last_active":"2024-09-10T23:39:26.624143+0000","last_peered":"2024-09-10T23:39:26.624143+0000","last_clean":"2024-09-10T23:39:26.624143+0000","last_became_active":"2024-09-10T23:39:25.603911+0000","last_became_peered":"2024-09-10T23:39:25.603911+0000","last_unstale":"2024-09-10T23:39:26.624143+0000","last_undegraded":"2024-09-10T23:39:26.624143+0000","last_fullsized":"2024-09-10T23:39:26.624143+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:16.865263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"137'6","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771013+0000","last_change":"2024-09-10T23:39:24.605325+0000","last_active":"2024-09-10T23:39:26.771013+0000","last_peered":"2024-09-10T23:39:26.771013+0000","last_clean":"2024-09-10T23:39:26.771013+0000","last_became_active":"2024-09-10T23:39:24.605043+0000","last_became_peered":"2024-09-10T23:39:24.605043+0000","last_unstale":"2024-09-10T23:39:26.771013+0000","last_undegraded":"2024-09-10T23:39:26.771013+0000","last_fullsized":"2024-09-10T23:39:26.771013+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:39.200145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.612671+0000","last_change":"2024-09-10T23:39:25.609074+0000","last_active":"2024-09-10T23:39:26.612671+0000","last_peered":"2024-09-10T23:39:26.612671+0000","last_clean":"2024-09-10T23:39:26.612671+0000","last_became_active":"2024-09-10T23:39:25.608797+0000","last_became_peered":"2024-09-10T23:39:25.608797+0000","last_unstale":"2024-09-10T23:39:26.612671+0000","last_undegraded":"2024-09-10T23:39:26.612671+0000","last_fullsized":"2024-09-10T23:39:26.612671+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:43.837353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771433+0000","last_change":"2024-09-10T23:39:24.608738+0000","last_active":"2024-09-10T23:39:26.771433+0000","last_peered":"2024-09-10T23:39:26.771433+0000","last_clean":"2024-09-10T23:39:26.771433+0000","last_became_active":"2024-09-10T23:39:24.608463+0000","last_became_peered":"2024-09-10T23:39:24.608463+0000","last_unstale":"2024-09-10T23:39:26.771433+0000","last_undegraded":"2024-09-10T23:39:26.771433+0000","last_fullsized":"2024-09-10T23:39:26.771433+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:43.715096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613705+0000","last_change":"2024-09-10T23:39:25.613818+0000","last_active":"2024-09-10T23:39:26.613705+0000","last_peered":"2024-09-10T23:39:26.613705+0000","last_clean":"2024-09-10T23:39:26.613705+0000","last_became_active":"2024-09-10T23:39:25.613345+0000","last_became_peered":"2024-09-10T23:39:25.613345+0000","last_unstale":"2024-09-10T23:39:26.613705+0000","last_undegraded":"2024-09-10T23:39:26.613705+0000","last_fullsized":"2024-09-10T23:39:26.613705+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:22.371011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771025+0000","last_change":"2024-09-10T23:39:24.614850+0000","last_active":"2024-09-10T23:39:26.771025+0000","last_peered":"2024-09-10T23:39:26.771025+0000","last_clean":"2024-09-10T23:39:26.771025+0000","last_became_active":"2024-09-10T23:39:24.612966+0000","last_became_peered":"2024-09-10T23:39:24.612966+0000","last_unstale":"2024-09-10T23:39:26.771025+0000","last_undegraded":"2024-09-10T23:39:26.771025+0000","last_fullsized":"2024-09-10T23:39:26.771025+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:38.302646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770478+0000","last_change":"2024-09-10T23:39:25.620075+0000","last_active":"2024-09-10T23:39:26.770478+0000","last_peered":"2024-09-10T23:39:26.770478+0000","last_clean":"2024-09-10T23:39:26.770478+0000","last_became_active":"2024-09-10T23:39:25.619740+0000","last_became_peered":"2024-09-10T23:39:25.619740+0000","last_unstale":"2024-09-10T23:39:26.770478+0000","last_undegraded":"2024-09-10T23:39:26.770478+0000","last_fullsized":"2024-09-10T23:39:26.770478+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:00.828349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.612773+0000","last_change":"2024-09-10T23:39:24.609752+0000","last_active":"2024-09-10T23:39:26.612773+0000","last_peered":"2024-09-10T23:39:26.612773+0000","last_clean":"2024-09-10T23:39:26.612773+0000","last_became_active":"2024-09-10T23:39:24.609476+0000","last_became_peered":"2024-09-10T23:39:24.609476+0000","last_unstale":"2024-09-10T23:39:26.612773+0000","last_undegraded":"2024-09-10T23:39:26.612773+0000","last_fullsized":"2024-09-10T23:39:26.612773+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:21.564871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770685+0000","last_change":"2024-09-10T23:39:25.607888+0000","last_active":"2024-09-10T23:39:26.770685+0000","last_peered":"2024-09-10T23:39:26.770685+0000","last_clean":"2024-09-10T23:39:26.770685+0000","last_became_active":"2024-09-10T23:39:25.607637+0000","last_became_peered":"2024-09-10T23:39:25.607637+0000","last_unstale":"2024-09-10T23:39:26.770685+0000","last_undegraded":"2024-09-10T23:39:26.770685+0000","last_fullsized":"2024-09-10T23:39:26.770685+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:18.794229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"137'8","reported_seq":23,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770807+0000","last_change":"2024-09-10T23:39:24.604950+0000","last_active":"2024-09-10T23:39:26.770807+0000","last_peered":"2024-09-10T23:39:26.770807+0000","last_clean":"2024-09-10T23:39:26.770807+0000","last_became_active":"2024-09-10T23:39:24.604022+0000","last_became_peered":"2024-09-10T23:39:24.604022+0000","last_unstale":"2024-09-10T23:39:26.770807+0000","last_undegraded":"2024-09-10T23:39:26.770807+0000","last_fullsized":"2024-09-10T23:39:26.770807+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:16.225255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613286+0000","last_change":"2024-09-10T23:39:25.622750+0000","last_active":"2024-09-10T23:39:26.613286+0000","last_peered":"2024-09-10T23:39:26.613286+0000","last_clean":"2024-09-10T23:39:26.613286+0000","last_became_active":"2024-09-10T23:39:25.622483+0000","last_became_peered":"2024-09-10T23:39:25.622483+0000","last_unstale":"2024-09-10T23:39:26.613286+0000","last_undegraded":"2024-09-10T23:39:26.613286+0000","last_fullsized":"2024-09-10T23:39:26.613286+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:14.711437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771233+0000","last_change":"2024-09-10T23:39:24.605628+0000","last_active":"2024-09-10T23:39:26.771233+0000","last_peered":"2024-09-10T23:39:26.771233+0000","last_clean":"2024-09-10T23:39:26.771233+0000","last_became_active":"2024-09-10T23:39:24.605367+0000","last_became_peered":"2024-09-10T23:39:24.605367+0000","last_unstale":"2024-09-10T23:39:26.771233+0000","last_undegraded":"2024-09-10T23:39:26.771233+0000","last_fullsized":"2024-09-10T23:39:26.771233+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:01.508944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771775+0000","last_change":"2024-09-10T23:39:25.615157+0000","last_active":"2024-09-10T23:39:26.771775+0000","last_peered":"2024-09-10T23:39:26.771775+0000","last_clean":"2024-09-10T23:39:26.771775+0000","last_became_active":"2024-09-10T23:39:25.614824+0000","last_became_peered":"2024-09-10T23:39:25.614824+0000","last_unstale":"2024-09-10T23:39:26.771775+0000","last_undegraded":"2024-09-10T23:39:26.771775+0000","last_fullsized":"2024-09-10T23:39:26.771775+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:16.697918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770517+0000","last_change":"2024-09-10T23:39:24.611479+0000","last_active":"2024-09-10T23:39:26.770517+0000","last_peered":"2024-09-10T23:39:26.770517+0000","last_clean":"2024-09-10T23:39:26.770517+0000","last_became_active":"2024-09-10T23:39:24.609193+0000","last_became_peered":"2024-09-10T23:39:24.609193+0000","last_unstale":"2024-09-10T23:39:26.770517+0000","last_undegraded":"2024-09-10T23:39:26.770517+0000","last_fullsized":"2024-09-10T23:39:26.770517+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:28:59.738155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771891+0000","last_change":"2024-09-10T23:39:25.618336+0000","last_active":"2024-09-10T23:39:26.771891+0000","last_peered":"2024-09-10T23:39:26.771891+0000","last_clean":"2024-09-10T23:39:26.771891+0000","last_became_active":"2024-09-10T23:39:25.617985+0000","last_became_peered":"2024-09-10T23:39:25.617985+0000","last_unstale":"2024-09-10T23:39:26.771891+0000","last_undegraded":"2024-09-10T23:39:26.771891+0000","last_fullsized":"2024-09-10T23:39:26.771891+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:20.015474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771334+0000","last_change":"2024-09-10T23:39:24.602179+0000","last_active":"2024-09-10T23:39:26.771334+0000","last_peered":"2024-09-10T23:39:26.771334+0000","last_clean":"2024-09-10T23:39:26.771334+0000","last_became_active":"2024-09-10T23:39:24.601773+0000","last_became_peered":"2024-09-10T23:39:24.601773+0000","last_unstale":"2024-09-10T23:39:26.771334+0000","last_undegraded":"2024-09-10T23:39:26.771334+0000","last_fullsized":"2024-09-10T23:39:26.771334+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:31.181082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.614056+0000","last_change":"2024-09-10T23:39:25.616269+0000","last_active":"2024-09-10T23:39:26.614056+0000","last_peered":"2024-09-10T23:39:26.614056+0000","last_clean":"2024-09-10T23:39:26.614056+0000","last_became_active":"2024-09-10T23:39:25.616003+0000","last_became_peered":"2024-09-10T23:39:25.616003+0000","last_unstale":"2024-09-10T23:39:26.614056+0000","last_undegraded":"2024-09-10T23:39:26.614056+0000","last_fullsized":"2024-09-10T23:39:26.614056+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:09.732712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.611271+0000","last_change":"2024-09-10T23:39:24.615545+0000","last_active":"2024-09-10T23:39:26.611271+0000","last_peered":"2024-09-10T23:39:26.611271+0000","last_clean":"2024-09-10T23:39:26.611271+0000","last_became_active":"2024-09-10T23:39:24.615120+0000","last_became_peered":"2024-09-10T23:39:24.615120+0000","last_unstale":"2024-09-10T23:39:26.611271+0000","last_undegraded":"2024-09-10T23:39:26.611271+0000","last_fullsized":"2024-09-10T23:39:26.611271+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:25.247342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623731+0000","last_change":"2024-09-10T23:39:25.605665+0000","last_active":"2024-09-10T23:39:26.623731+0000","last_peered":"2024-09-10T23:39:26.623731+0000","last_clean":"2024-09-10T23:39:26.623731+0000","last_became_active":"2024-09-10T23:39:25.605411+0000","last_became_peered":"2024-09-10T23:39:25.605411+0000","last_unstale":"2024-09-10T23:39:26.623731+0000","last_undegraded":"2024-09-10T23:39:26.623731+0000","last_fullsized":"2024-09-10T23:39:26.623731+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:19:30.378031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613837+0000","last_change":"2024-09-10T23:39:24.611435+0000","last_active":"2024-09-10T23:39:26.613837+0000","last_peered":"2024-09-10T23:39:26.613837+0000","last_clean":"2024-09-10T23:39:26.613837+0000","last_became_active":"2024-09-10T23:39:24.611148+0000","last_became_peered":"2024-09-10T23:39:24.611148+0000","last_unstale":"2024-09-10T23:39:26.613837+0000","last_undegraded":"2024-09-10T23:39:26.613837+0000","last_fullsized":"2024-09-10T23:39:26.613837+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:17.956462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613770+0000","last_change":"2024-09-10T23:39:25.616933+0000","last_active":"2024-09-10T23:39:26.613770+0000","last_peered":"2024-09-10T23:39:26.613770+0000","last_clean":"2024-09-10T23:39:26.613770+0000","last_became_active":"2024-09-10T23:39:25.616682+0000","last_became_peered":"2024-09-10T23:39:25.616682+0000","last_unstale":"2024-09-10T23:39:26.613770+0000","last_undegraded":"2024-09-10T23:39:26.613770+0000","last_fullsized":"2024-09-10T23:39:26.613770+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:22.597132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.610967+0000","last_change":"2024-09-10T23:39:24.611840+0000","last_active":"2024-09-10T23:39:26.610967+0000","last_peered":"2024-09-10T23:39:26.610967+0000","last_clean":"2024-09-10T23:39:26.610967+0000","last_became_active":"2024-09-10T23:39:24.611521+0000","last_became_peered":"2024-09-10T23:39:24.611521+0000","last_unstale":"2024-09-10T23:39:26.610967+0000","last_undegraded":"2024-09-10T23:39:26.610967+0000","last_fullsized":"2024-09-10T23:39:26.610967+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:55.241122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623567+0000","last_change":"2024-09-10T23:39:25.606700+0000","last_active":"2024-09-10T23:39:26.623567+0000","last_peered":"2024-09-10T23:39:26.623567+0000","last_clean":"2024-09-10T23:39:26.623567+0000","last_became_active":"2024-09-10T23:39:25.606449+0000","last_became_peered":"2024-09-10T23:39:25.606449+0000","last_unstale":"2024-09-10T23:39:26.623567+0000","last_undegraded":"2024-09-10T23:39:26.623567+0000","last_fullsized":"2024-09-10T23:39:26.623567+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:45.863089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770993+0000","last_change":"2024-09-10T23:39:24.616440+0000","last_active":"2024-09-10T23:39:26.770993+0000","last_peered":"2024-09-10T23:39:26.770993+0000","last_clean":"2024-09-10T23:39:26.770993+0000","last_became_active":"2024-09-10T23:39:24.616027+0000","last_became_peered":"2024-09-10T23:39:24.616027+0000","last_unstale":"2024-09-10T23:39:26.770993+0000","last_undegraded":"2024-09-10T23:39:26.770993+0000","last_fullsized":"2024-09-10T23:39:26.770993+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:35:56.271120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.612544+0000","last_change":"2024-09-10T23:39:25.605960+0000","last_active":"2024-09-10T23:39:26.612544+0000","last_peered":"2024-09-10T23:39:26.612544+0000","last_clean":"2024-09-10T23:39:26.612544+0000","last_became_active":"2024-09-10T23:39:25.605289+0000","last_became_peered":"2024-09-10T23:39:25.605289+0000","last_unstale":"2024-09-10T23:39:26.612544+0000","last_undegraded":"2024-09-10T23:39:26.612544+0000","last_fullsized":"2024-09-10T23:39:26.612544+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:00.623663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.610996+0000","last_change":"2024-09-10T23:39:24.610231+0000","last_active":"2024-09-10T23:39:26.610996+0000","last_peered":"2024-09-10T23:39:26.610996+0000","last_clean":"2024-09-10T23:39:26.610996+0000","last_became_active":"2024-09-10T23:39:24.609932+0000","last_became_peered":"2024-09-10T23:39:24.609932+0000","last_unstale":"2024-09-10T23:39:26.610996+0000","last_undegraded":"2024-09-10T23:39:26.610996+0000","last_fullsized":"2024-09-10T23:39:26.610996+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:08.557847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.614153+0000","last_change":"2024-09-10T23:39:25.630602+0000","last_active":"2024-09-10T23:39:26.614153+0000","last_peered":"2024-09-10T23:39:26.614153+0000","last_clean":"2024-09-10T23:39:26.614153+0000","last_became_active":"2024-09-10T23:39:25.628124+0000","last_became_peered":"2024-09-10T23:39:25.628124+0000","last_unstale":"2024-09-10T23:39:26.614153+0000","last_undegraded":"2024-09-10T23:39:26.614153+0000","last_fullsized":"2024-09-10T23:39:26.614153+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:08.211937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770826+0000","last_change":"2024-09-10T23:39:24.614991+0000","last_active":"2024-09-10T23:39:26.770826+0000","last_peered":"2024-09-10T23:39:26.770826+0000","last_clean":"2024-09-10T23:39:26.770826+0000","last_became_active":"2024-09-10T23:39:24.614716+0000","last_became_peered":"2024-09-10T23:39:24.614716+0000","last_unstale":"2024-09-10T23:39:26.770826+0000","last_undegraded":"2024-09-10T23:39:26.770826+0000","last_fullsized":"2024-09-10T23:39:26.770826+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:50.421286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771873+0000","last_change":"2024-09-10T23:39:25.610111+0000","last_active":"2024-09-10T23:39:26.771873+0000","last_peered":"2024-09-10T23:39:26.771873+0000","last_clean":"2024-09-10T23:39:26.771873+0000","last_became_active":"2024-09-10T23:39:25.609828+0000","last_became_peered":"2024-09-10T23:39:25.609828+0000","last_unstale":"2024-09-10T23:39:26.771873+0000","last_undegraded":"2024-09-10T23:39:26.771873+0000","last_fullsized":"2024-09-10T23:39:26.771873+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:28.639580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"138'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.622884+0000","last_change":"2024-09-10T23:39:24.614541+0000","last_active":"2024-09-10T23:39:26.622884+0000","last_peered":"2024-09-10T23:39:26.622884+0000","last_clean":"2024-09-10T23:39:26.622884+0000","last_became_active":"2024-09-10T23:39:24.614160+0000","last_became_peered":"2024-09-10T23:39:24.614160+0000","last_unstale":"2024-09-10T23:39:26.622884+0000","last_undegraded":"2024-09-10T23:39:26.622884+0000","last_fullsized":"2024-09-10T23:39:26.622884+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:26.965773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771903+0000","last_change":"2024-09-10T23:39:25.603640+0000","last_active":"2024-09-10T23:39:26.771903+0000","last_peered":"2024-09-10T23:39:26.771903+0000","last_clean":"2024-09-10T23:39:26.771903+0000","last_became_active":"2024-09-10T23:39:25.603263+0000","last_became_peered":"2024-09-10T23:39:25.603263+0000","last_unstale":"2024-09-10T23:39:26.771903+0000","last_undegraded":"2024-09-10T23:39:26.771903+0000","last_fullsized":"2024-09-10T23:39:26.771903+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:39.571626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.624248+0000","last_change":"2024-09-10T23:39:24.616973+0000","last_active":"2024-09-10T23:39:26.624248+0000","last_peered":"2024-09-10T23:39:26.624248+0000","last_clean":"2024-09-10T23:39:26.624248+0000","last_became_active":"2024-09-10T23:39:24.616645+0000","last_became_peered":"2024-09-10T23:39:24.616645+0000","last_unstale":"2024-09-10T23:39:26.624248+0000","last_undegraded":"2024-09-10T23:39:26.624248+0000","last_fullsized":"2024-09-10T23:39:26.624248+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:29.254303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771574+0000","last_change":"2024-09-10T23:39:25.608971+0000","last_active":"2024-09-10T23:39:26.771574+0000","last_peered":"2024-09-10T23:39:26.771574+0000","last_clean":"2024-09-10T23:39:26.771574+0000","last_became_active":"2024-09-10T23:39:25.607331+0000","last_became_peered":"2024-09-10T23:39:25.607331+0000","last_unstale":"2024-09-10T23:39:26.771574+0000","last_undegraded":"2024-09-10T23:39:26.771574+0000","last_fullsized":"2024-09-10T23:39:26.771574+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:35:19.218449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613759+0000","last_change":"2024-09-10T23:39:24.617160+0000","last_active":"2024-09-10T23:39:26.613759+0000","last_peered":"2024-09-10T23:39:26.613759+0000","last_clean":"2024-09-10T23:39:26.613759+0000","last_became_active":"2024-09-10T23:39:24.616866+0000","last_became_peered":"2024-09-10T23:39:24.616866+0000","last_unstale":"2024-09-10T23:39:26.613759+0000","last_undegraded":"2024-09-10T23:39:26.613759+0000","last_fullsized":"2024-09-10T23:39:26.613759+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:34.154690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770908+0000","last_change":"2024-09-10T23:39:25.625272+0000","last_active":"2024-09-10T23:39:26.770908+0000","last_peered":"2024-09-10T23:39:26.770908+0000","last_clean":"2024-09-10T23:39:26.770908+0000","last_became_active":"2024-09-10T23:39:25.622945+0000","last_became_peered":"2024-09-10T23:39:25.622945+0000","last_unstale":"2024-09-10T23:39:26.770908+0000","last_undegraded":"2024-09-10T23:39:26.770908+0000","last_fullsized":"2024-09-10T23:39:26.770908+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:58.427537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771169+0000","last_change":"2024-09-10T23:39:24.608424+0000","last_active":"2024-09-10T23:39:26.771169+0000","last_peered":"2024-09-10T23:39:26.771169+0000","last_clean":"2024-09-10T23:39:26.771169+0000","last_became_active":"2024-09-10T23:39:24.607786+0000","last_became_peered":"2024-09-10T23:39:24.607786+0000","last_unstale":"2024-09-10T23:39:26.771169+0000","last_undegraded":"2024-09-10T23:39:26.771169+0000","last_fullsized":"2024-09-10T23:39:26.771169+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:26.211296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623952+0000","last_change":"2024-09-10T23:39:25.605405+0000","last_active":"2024-09-10T23:39:26.623952+0000","last_peered":"2024-09-10T23:39:26.623952+0000","last_clean":"2024-09-10T23:39:26.623952+0000","last_became_active":"2024-09-10T23:39:25.605050+0000","last_became_peered":"2024-09-10T23:39:25.605050+0000","last_unstale":"2024-09-10T23:39:26.623952+0000","last_undegraded":"2024-09-10T23:39:26.623952+0000","last_fullsized":"2024-09-10T23:39:26.623952+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:16.919508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.612749+0000","last_change":"2024-09-10T23:39:24.608720+0000","last_active":"2024-09-10T23:39:26.612749+0000","last_peered":"2024-09-10T23:39:26.612749+0000","last_clean":"2024-09-10T23:39:26.612749+0000","last_became_active":"2024-09-10T23:39:24.608415+0000","last_became_peered":"2024-09-10T23:39:24.608415+0000","last_unstale":"2024-09-10T23:39:26.612749+0000","last_undegraded":"2024-09-10T23:39:26.612749+0000","last_fullsized":"2024-09-10T23:39:26.612749+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:58.259236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.612214+0000","last_change":"2024-09-10T23:39:25.608083+0000","last_active":"2024-09-10T23:39:26.612214+0000","last_peered":"2024-09-10T23:39:26.612214+0000","last_clean":"2024-09-10T23:39:26.612214+0000","last_became_active":"2024-09-10T23:39:25.607187+0000","last_became_peered":"2024-09-10T23:39:25.607187+0000","last_unstale":"2024-09-10T23:39:26.612214+0000","last_undegraded":"2024-09-10T23:39:26.612214+0000","last_fullsized":"2024-09-10T23:39:26.612214+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:15.169982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623751+0000","last_change":"2024-09-10T23:39:24.618247+0000","last_active":"2024-09-10T23:39:26.623751+0000","last_peered":"2024-09-10T23:39:26.623751+0000","last_clean":"2024-09-10T23:39:26.623751+0000","last_became_active":"2024-09-10T23:39:24.617042+0000","last_became_peered":"2024-09-10T23:39:24.617042+0000","last_unstale":"2024-09-10T23:39:26.623751+0000","last_undegraded":"2024-09-10T23:39:26.623751+0000","last_fullsized":"2024-09-10T23:39:26.623751+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:16.262731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623643+0000","last_change":"2024-09-10T23:39:25.624133+0000","last_active":"2024-09-10T23:39:26.623643+0000","last_peered":"2024-09-10T23:39:26.623643+0000","last_clean":"2024-09-10T23:39:26.623643+0000","last_became_active":"2024-09-10T23:39:25.623865+0000","last_became_peered":"2024-09-10T23:39:25.623865+0000","last_unstale":"2024-09-10T23:39:26.623643+0000","last_undegraded":"2024-09-10T23:39:26.623643+0000","last_fullsized":"2024-09-10T23:39:26.623643+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:04:52.290850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623485+0000","last_change":"2024-09-10T23:39:24.610914+0000","last_active":"2024-09-10T23:39:26.623485+0000","last_peered":"2024-09-10T23:39:26.623485+0000","last_clean":"2024-09-10T23:39:26.623485+0000","last_became_active":"2024-09-10T23:39:24.608246+0000","last_became_peered":"2024-09-10T23:39:24.608246+0000","last_unstale":"2024-09-10T23:39:26.623485+0000","last_undegraded":"2024-09-10T23:39:26.623485+0000","last_fullsized":"2024-09-10T23:39:26.623485+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:51:17.812933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613339+0000","last_change":"2024-09-10T23:39:25.605894+0000","last_active":"2024-09-10T23:39:26.613339+0000","last_peered":"2024-09-10T23:39:26.613339+0000","last_clean":"2024-09-10T23:39:26.613339+0000","last_became_active":"2024-09-10T23:39:25.605517+0000","last_became_peered":"2024-09-10T23:39:25.605517+0000","last_unstale":"2024-09-10T23:39:26.613339+0000","last_undegraded":"2024-09-10T23:39:26.613339+0000","last_fullsized":"2024-09-10T23:39:26.613339+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:52.087639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771101+0000","last_change":"2024-09-10T23:39:24.605009+0000","last_active":"2024-09-10T23:39:26.771101+0000","last_peered":"2024-09-10T23:39:26.771101+0000","last_clean":"2024-09-10T23:39:26.771101+0000","last_became_active":"2024-09-10T23:39:24.604483+0000","last_became_peered":"2024-09-10T23:39:24.604483+0000","last_unstale":"2024-09-10T23:39:26.771101+0000","last_undegraded":"2024-09-10T23:39:26.771101+0000","last_fullsized":"2024-09-10T23:39:26.771101+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:04.197080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":350,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770893+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:39:26.770893+0000","last_peered":"2024-09-10T23:39:26.770893+0000","last_clean":"2024-09-10T23:39:26.770893+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:39:26.770893+0000","last_undegraded":"2024-09-10T23:39:26.770893+0000","last_fullsized":"2024-09-10T23:39:26.770893+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509529,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1728512,"data_stored":760524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1732608,"data_stored":760558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509528,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33472,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338368,"statfs":{"total":96636764160,"available":96602488832,"internally_reserved":0,"allocated":2330624,"data_stored":1351564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509526,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1732608,"data_stored":760558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509527,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1744896,"data_stored":762158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509528,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1732608,"data_stored":761106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509527,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37640,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334200,"statfs":{"total":96636764160,"available":96598220800,"internally_reserved":0,"allocated":2347008,"data_stored":1352686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509527,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1732608,"data_stored":760594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:39:29.717 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:39:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.892+0000 7f8387200640 1 -- 172.21.15.94:0/385933352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f838815ba30 msgr2=0x7f83880b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.892+0000 7f8387200640 1 --2- 172.21.15.94:0/385933352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f838815ba30 0x7f83880b1bd0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f8378009840 tx=0x7f837801c7f0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.892+0000 7f8387200640 1 -- 172.21.15.94:0/385933352 shutdown_connections 2024-09-10T23:39:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.892+0000 7f8387200640 1 --2- 172.21.15.94:0/385933352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83880d04b0 0x7f83880aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.892+0000 7f8387200640 1 --2- 172.21.15.94:0/385933352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f838815ba30 0x7f83880b1bd0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.892+0000 7f8387200640 1 -- 172.21.15.94:0/385933352 >> 172.21.15.94:0/385933352 conn(0x7f838809aa70 msgr2=0x7f838809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.892+0000 7f8387200640 1 -- 172.21.15.94:0/385933352 shutdown_connections 2024-09-10T23:39:29.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.892+0000 7f8387200640 1 -- 172.21.15.94:0/385933352 wait complete. 2024-09-10T23:39:29.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.892+0000 7f8387200640 1 Processor -- start 2024-09-10T23:39:29.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.892+0000 7f8387200640 1 -- start start 2024-09-10T23:39:29.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.892+0000 7f8387200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83880b2110 0x7f83880cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8387200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83880d04b0 0x7f8388150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8387200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f838815ba30 0x7f83880cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8387200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83881686b0 con 0x7f83880b2110 2024-09-10T23:39:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8387200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f83880b00f0 con 0x7f83880d04b0 2024-09-10T23:39:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8387200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8388168830 con 0x7f838815ba30 2024-09-10T23:39:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8385e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83880b2110 0x7f83880cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8385e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83880b2110 0x7f83880cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50340/0 (socket says 172.21.15.94:50340) 2024-09-10T23:39:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8385e00640 1 -- 172.21.15.94:0/2636499926 learned_addr learned my addr 172.21.15.94:0/2636499926 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8385e00640 1 -- 172.21.15.94:0/2636499926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f838815ba30 msgr2=0x7f83880cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8385e00640 1 --2- 172.21.15.94:0/2636499926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f838815ba30 0x7f83880cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8385e00640 1 -- 172.21.15.94:0/2636499926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83880d04b0 msgr2=0x7f8388150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8385e00640 1 --2- 172.21.15.94:0/2636499926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83880d04b0 0x7f8388150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:29.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8385e00640 1 -- 172.21.15.94:0/2636499926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8388165c20 con 0x7f83880b2110 2024-09-10T23:39:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8385e00640 1 --2- 172.21.15.94:0/2636499926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83880b2110 0x7f83880cd250 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f8378000c10 tx=0x7f83780a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f837ea00640 1 -- 172.21.15.94:0/2636499926 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83780b2040 con 0x7f83880b2110 2024-09-10T23:39:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f837ea00640 1 -- 172.21.15.94:0/2636499926 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83780043b0 con 0x7f83880b2110 2024-09-10T23:39:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8387200640 1 -- 172.21.15.94:0/2636499926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8388165940 con 0x7f83880b2110 2024-09-10T23:39:29.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f837ea00640 1 -- 172.21.15.94:0/2636499926 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378003800 con 0x7f83880b2110 2024-09-10T23:39:29.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8387200640 1 -- 172.21.15.94:0/2636499926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8388070b50 con 0x7f83880b2110 2024-09-10T23:39:29.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f8387200640 1 -- 172.21.15.94:0/2636499926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83880aee30 con 0x7f83880b2110 2024-09-10T23:39:29.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.896+0000 7f837ea00640 1 -- 172.21.15.94:0/2636499926 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f83780b5070 con 0x7f83880b2110 2024-09-10T23:39:29.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.900+0000 7f837ea00640 1 --2- 172.21.15.94:0/2636499926 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8364072cd0 0x7f8364075190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:29.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.900+0000 7f837ea00640 1 -- 172.21.15.94:0/2636499926 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7f837812d660 con 0x7f83880b2110 2024-09-10T23:39:29.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.900+0000 7f837d400640 1 --2- 172.21.15.94:0/2636499926 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8364072cd0 0x7f8364075190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:29.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.900+0000 7f837d400640 1 --2- 172.21.15.94:0/2636499926 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8364072cd0 0x7f8364075190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f838815ea50 tx=0x7f83700078d0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:29.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:29.904+0000 7f837ea00640 1 -- 172.21.15.94:0/2636499926 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8378016410 con 0x7f83880b2110 2024-09-10T23:39:30.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.064+0000 7f8387200640 1 -- 172.21.15.94:0/2636499926 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f838809b0a0 con 0x7f8364072cd0 2024-09-10T23:39:30.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.068+0000 7f837ea00640 1 -- 172.21.15.94:0/2636499926 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252479 (secure 0 0 0) 0x7f838809b0a0 con 0x7f8364072cd0 2024-09-10T23:39:30.074 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:30.075 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:39:30.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.072+0000 7f8387200640 1 -- 172.21.15.94:0/2636499926 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8364072cd0 msgr2=0x7f8364075190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:30.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.072+0000 7f8387200640 1 --2- 172.21.15.94:0/2636499926 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8364072cd0 0x7f8364075190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f838815ea50 tx=0x7f83700078d0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.072+0000 7f8387200640 1 -- 172.21.15.94:0/2636499926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83880b2110 msgr2=0x7f83880cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:30.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.072+0000 7f8387200640 1 --2- 172.21.15.94:0/2636499926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83880b2110 0x7f83880cd250 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f8378000c10 tx=0x7f83780a56b0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.072+0000 7f8387200640 1 -- 172.21.15.94:0/2636499926 shutdown_connections 2024-09-10T23:39:30.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.072+0000 7f8387200640 1 --2- 172.21.15.94:0/2636499926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f83880d04b0 0x7f8388150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.072+0000 7f8387200640 1 --2- 172.21.15.94:0/2636499926 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8364072cd0 0x7f8364075190 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.072+0000 7f8387200640 1 --2- 172.21.15.94:0/2636499926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f838815ba30 0x7f83880cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.072+0000 7f8387200640 1 --2- 172.21.15.94:0/2636499926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83880b2110 0x7f83880cd250 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.072+0000 7f8387200640 1 -- 172.21.15.94:0/2636499926 >> 172.21.15.94:0/2636499926 conn(0x7f838809aa70 msgr2=0x7f83881530d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:30.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.072+0000 7f8387200640 1 -- 172.21.15.94:0/2636499926 shutdown_connections 2024-09-10T23:39:30.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.072+0000 7f8387200640 1 -- 172.21.15.94:0/2636499926 wait complete. 2024-09-10T23:39:30.094 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-10T23:39:28.770028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268432,"kb_used_data":14728,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754706288,"statfs":{"total":773094113280,"available":772819238912,"internally_reserved":0,"allocated":15081472,"data_stored":7269748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.189738"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.624443+0000","last_change":"2024-09-10T23:39:25.610553+0000","last_active":"2024-09-10T23:39:26.624443+0000","last_peered":"2024-09-10T23:39:26.624443+0000","last_clean":"2024-09-10T23:39:26.624443+0000","last_became_active":"2024-09-10T23:39:25.610283+0000","last_became_peered":"2024-09-10T23:39:25.610283+0000","last_unstale":"2024-09-10T23:39:26.624443+0000","last_undegraded":"2024-09-10T23:39:26.624443+0000","last_fullsized":"2024-09-10T23:39:26.624443+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:16.475241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.624398+0000","last_change":"2024-09-10T23:39:25.613866+0000","last_active":"2024-09-10T23:39:26.624398+0000","last_peered":"2024-09-10T23:39:26.624398+0000","last_clean":"2024-09-10T23:39:26.624398+0000","last_became_active":"2024-09-10T23:39:25.613573+0000","last_became_peered":"2024-09-10T23:39:25.613573+0000","last_unstale":"2024-09-10T23:39:26.624398+0000","last_undegraded":"2024-09-10T23:39:26.624398+0000","last_fullsized":"2024-09-10T23:39:26.624398+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:18.794229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.624106+0000","last_change":"2024-09-10T23:39:25.612589+0000","last_active":"2024-09-10T23:39:26.624106+0000","last_peered":"2024-09-10T23:39:26.624106+0000","last_clean":"2024-09-10T23:39:26.624106+0000","last_became_active":"2024-09-10T23:39:25.611118+0000","last_became_peered":"2024-09-10T23:39:25.611118+0000","last_unstale":"2024-09-10T23:39:26.624106+0000","last_undegraded":"2024-09-10T23:39:26.624106+0000","last_fullsized":"2024-09-10T23:39:26.624106+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:16.981176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771709+0000","last_change":"2024-09-10T23:39:25.619277+0000","last_active":"2024-09-10T23:39:26.771709+0000","last_peered":"2024-09-10T23:39:26.771709+0000","last_clean":"2024-09-10T23:39:26.771709+0000","last_became_active":"2024-09-10T23:39:25.618895+0000","last_became_peered":"2024-09-10T23:39:25.618895+0000","last_unstale":"2024-09-10T23:39:26.771709+0000","last_undegraded":"2024-09-10T23:39:26.771709+0000","last_fullsized":"2024-09-10T23:39:26.771709+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:59.862302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771069+0000","last_change":"2024-09-10T23:39:25.609793+0000","last_active":"2024-09-10T23:39:26.771069+0000","last_peered":"2024-09-10T23:39:26.771069+0000","last_clean":"2024-09-10T23:39:26.771069+0000","last_became_active":"2024-09-10T23:39:25.609547+0000","last_became_peered":"2024-09-10T23:39:25.609547+0000","last_unstale":"2024-09-10T23:39:26.771069+0000","last_undegraded":"2024-09-10T23:39:26.771069+0000","last_fullsized":"2024-09-10T23:39:26.771069+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:45.535562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771185+0000","last_change":"2024-09-10T23:39:25.629619+0000","last_active":"2024-09-10T23:39:26.771185+0000","last_peered":"2024-09-10T23:39:26.771185+0000","last_clean":"2024-09-10T23:39:26.771185+0000","last_became_active":"2024-09-10T23:39:25.628881+0000","last_became_peered":"2024-09-10T23:39:25.628881+0000","last_unstale":"2024-09-10T23:39:26.771185+0000","last_undegraded":"2024-09-10T23:39:26.771185+0000","last_fullsized":"2024-09-10T23:39:26.771185+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:37.624879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.612659+0000","last_change":"2024-09-10T23:39:25.606249+0000","last_active":"2024-09-10T23:39:26.612659+0000","last_peered":"2024-09-10T23:39:26.612659+0000","last_clean":"2024-09-10T23:39:26.612659+0000","last_became_active":"2024-09-10T23:39:25.605661+0000","last_became_peered":"2024-09-10T23:39:25.605661+0000","last_unstale":"2024-09-10T23:39:26.612659+0000","last_undegraded":"2024-09-10T23:39:26.612659+0000","last_fullsized":"2024-09-10T23:39:26.612659+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:58.656048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770480+0000","last_change":"2024-09-10T23:39:25.611265+0000","last_active":"2024-09-10T23:39:26.770480+0000","last_peered":"2024-09-10T23:39:26.770480+0000","last_clean":"2024-09-10T23:39:26.770480+0000","last_became_active":"2024-09-10T23:39:25.610987+0000","last_became_peered":"2024-09-10T23:39:25.610987+0000","last_unstale":"2024-09-10T23:39:26.770480+0000","last_undegraded":"2024-09-10T23:39:26.770480+0000","last_fullsized":"2024-09-10T23:39:26.770480+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:09.732712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770429+0000","last_change":"2024-09-10T23:39:25.615276+0000","last_active":"2024-09-10T23:39:26.770429+0000","last_peered":"2024-09-10T23:39:26.770429+0000","last_clean":"2024-09-10T23:39:26.770429+0000","last_became_active":"2024-09-10T23:39:25.615026+0000","last_became_peered":"2024-09-10T23:39:25.615026+0000","last_unstale":"2024-09-10T23:39:26.770429+0000","last_undegraded":"2024-09-10T23:39:26.770429+0000","last_fullsized":"2024-09-10T23:39:26.770429+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:16.919508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770407+0000","last_change":"2024-09-10T23:39:25.619582+0000","last_active":"2024-09-10T23:39:26.770407+0000","last_peered":"2024-09-10T23:39:26.770407+0000","last_clean":"2024-09-10T23:39:26.770407+0000","last_became_active":"2024-09-10T23:39:25.619281+0000","last_became_peered":"2024-09-10T23:39:25.619281+0000","last_unstale":"2024-09-10T23:39:26.770407+0000","last_undegraded":"2024-09-10T23:39:26.770407+0000","last_fullsized":"2024-09-10T23:39:26.770407+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:19:30.378031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771271+0000","last_change":"2024-09-10T23:39:25.616973+0000","last_active":"2024-09-10T23:39:26.771271+0000","last_peered":"2024-09-10T23:39:26.771271+0000","last_clean":"2024-09-10T23:39:26.771271+0000","last_became_active":"2024-09-10T23:39:25.616717+0000","last_became_peered":"2024-09-10T23:39:25.616717+0000","last_unstale":"2024-09-10T23:39:26.771271+0000","last_undegraded":"2024-09-10T23:39:26.771271+0000","last_fullsized":"2024-09-10T23:39:26.771271+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:11.117207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.611160+0000","last_change":"2024-09-10T23:39:25.606766+0000","last_active":"2024-09-10T23:39:26.611160+0000","last_peered":"2024-09-10T23:39:26.611160+0000","last_clean":"2024-09-10T23:39:26.611160+0000","last_became_active":"2024-09-10T23:39:25.606515+0000","last_became_peered":"2024-09-10T23:39:25.606515+0000","last_unstale":"2024-09-10T23:39:26.611160+0000","last_undegraded":"2024-09-10T23:39:26.611160+0000","last_fullsized":"2024-09-10T23:39:26.611160+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:51.941533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.611101+0000","last_change":"2024-09-10T23:39:25.615546+0000","last_active":"2024-09-10T23:39:26.611101+0000","last_peered":"2024-09-10T23:39:26.611101+0000","last_clean":"2024-09-10T23:39:26.611101+0000","last_became_active":"2024-09-10T23:39:25.615088+0000","last_became_peered":"2024-09-10T23:39:25.615088+0000","last_unstale":"2024-09-10T23:39:26.611101+0000","last_undegraded":"2024-09-10T23:39:26.611101+0000","last_fullsized":"2024-09-10T23:39:26.611101+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:41.962177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770706+0000","last_change":"2024-09-10T23:39:25.612373+0000","last_active":"2024-09-10T23:39:26.770706+0000","last_peered":"2024-09-10T23:39:26.770706+0000","last_clean":"2024-09-10T23:39:26.770706+0000","last_became_active":"2024-09-10T23:39:25.612093+0000","last_became_peered":"2024-09-10T23:39:25.612093+0000","last_unstale":"2024-09-10T23:39:26.770706+0000","last_undegraded":"2024-09-10T23:39:26.770706+0000","last_fullsized":"2024-09-10T23:39:26.770706+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:16.475241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770475+0000","last_change":"2024-09-10T23:39:25.616597+0000","last_active":"2024-09-10T23:39:26.770475+0000","last_peered":"2024-09-10T23:39:26.770475+0000","last_clean":"2024-09-10T23:39:26.770475+0000","last_became_active":"2024-09-10T23:39:25.616307+0000","last_became_peered":"2024-09-10T23:39:25.616307+0000","last_unstale":"2024-09-10T23:39:26.770475+0000","last_undegraded":"2024-09-10T23:39:26.770475+0000","last_fullsized":"2024-09-10T23:39:26.770475+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:02.049706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.612200+0000","last_change":"2024-09-10T23:39:25.609362+0000","last_active":"2024-09-10T23:39:26.612200+0000","last_peered":"2024-09-10T23:39:26.612200+0000","last_clean":"2024-09-10T23:39:26.612200+0000","last_became_active":"2024-09-10T23:39:25.608449+0000","last_became_peered":"2024-09-10T23:39:25.608449+0000","last_unstale":"2024-09-10T23:39:26.612200+0000","last_undegraded":"2024-09-10T23:39:26.612200+0000","last_fullsized":"2024-09-10T23:39:26.612200+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:37.706104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771137+0000","last_change":"2024-09-10T23:39:25.621241+0000","last_active":"2024-09-10T23:39:26.771137+0000","last_peered":"2024-09-10T23:39:26.771137+0000","last_clean":"2024-09-10T23:39:26.771137+0000","last_became_active":"2024-09-10T23:39:25.620810+0000","last_became_peered":"2024-09-10T23:39:25.620810+0000","last_unstale":"2024-09-10T23:39:26.771137+0000","last_undegraded":"2024-09-10T23:39:26.771137+0000","last_fullsized":"2024-09-10T23:39:26.771137+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:30.090147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771656+0000","last_change":"2024-09-10T23:39:25.602781+0000","last_active":"2024-09-10T23:39:26.771656+0000","last_peered":"2024-09-10T23:39:26.771656+0000","last_clean":"2024-09-10T23:39:26.771656+0000","last_became_active":"2024-09-10T23:39:25.602117+0000","last_became_peered":"2024-09-10T23:39:25.602117+0000","last_unstale":"2024-09-10T23:39:26.771656+0000","last_undegraded":"2024-09-10T23:39:26.771656+0000","last_fullsized":"2024-09-10T23:39:26.771656+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:14.404875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613051+0000","last_change":"2024-09-10T23:39:25.618087+0000","last_active":"2024-09-10T23:39:26.613051+0000","last_peered":"2024-09-10T23:39:26.613051+0000","last_clean":"2024-09-10T23:39:26.613051+0000","last_became_active":"2024-09-10T23:39:25.617217+0000","last_became_peered":"2024-09-10T23:39:25.617217+0000","last_unstale":"2024-09-10T23:39:26.613051+0000","last_undegraded":"2024-09-10T23:39:26.613051+0000","last_fullsized":"2024-09-10T23:39:26.613051+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:18.469217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.611255+0000","last_change":"2024-09-10T23:39:25.605569+0000","last_active":"2024-09-10T23:39:26.611255+0000","last_peered":"2024-09-10T23:39:26.611255+0000","last_clean":"2024-09-10T23:39:26.611255+0000","last_became_active":"2024-09-10T23:39:25.605322+0000","last_became_peered":"2024-09-10T23:39:25.605322+0000","last_unstale":"2024-09-10T23:39:26.611255+0000","last_undegraded":"2024-09-10T23:39:26.611255+0000","last_fullsized":"2024-09-10T23:39:26.611255+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:19.954076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770551+0000","last_change":"2024-09-10T23:39:25.620120+0000","last_active":"2024-09-10T23:39:26.770551+0000","last_peered":"2024-09-10T23:39:26.770551+0000","last_clean":"2024-09-10T23:39:26.770551+0000","last_became_active":"2024-09-10T23:39:25.619814+0000","last_became_peered":"2024-09-10T23:39:25.619814+0000","last_unstale":"2024-09-10T23:39:26.770551+0000","last_undegraded":"2024-09-10T23:39:26.770551+0000","last_fullsized":"2024-09-10T23:39:26.770551+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:35.027835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771062+0000","last_change":"2024-09-10T23:39:25.617617+0000","last_active":"2024-09-10T23:39:26.771062+0000","last_peered":"2024-09-10T23:39:26.771062+0000","last_clean":"2024-09-10T23:39:26.771062+0000","last_became_active":"2024-09-10T23:39:25.617366+0000","last_became_peered":"2024-09-10T23:39:25.617366+0000","last_unstale":"2024-09-10T23:39:26.771062+0000","last_undegraded":"2024-09-10T23:39:26.771062+0000","last_fullsized":"2024-09-10T23:39:26.771062+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:10.722696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613658+0000","last_change":"2024-09-10T23:39:25.614289+0000","last_active":"2024-09-10T23:39:26.613658+0000","last_peered":"2024-09-10T23:39:26.613658+0000","last_clean":"2024-09-10T23:39:26.613658+0000","last_became_active":"2024-09-10T23:39:25.613951+0000","last_became_peered":"2024-09-10T23:39:25.613951+0000","last_unstale":"2024-09-10T23:39:26.613658+0000","last_undegraded":"2024-09-10T23:39:26.613658+0000","last_fullsized":"2024-09-10T23:39:26.613658+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:57.800986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770797+0000","last_change":"2024-09-10T23:39:25.606092+0000","last_active":"2024-09-10T23:39:26.770797+0000","last_peered":"2024-09-10T23:39:26.770797+0000","last_clean":"2024-09-10T23:39:26.770797+0000","last_became_active":"2024-09-10T23:39:25.605840+0000","last_became_peered":"2024-09-10T23:39:25.605840+0000","last_unstale":"2024-09-10T23:39:26.770797+0000","last_undegraded":"2024-09-10T23:39:26.770797+0000","last_fullsized":"2024-09-10T23:39:26.770797+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:10.795879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771241+0000","last_change":"2024-09-10T23:39:25.604741+0000","last_active":"2024-09-10T23:39:26.771241+0000","last_peered":"2024-09-10T23:39:26.771241+0000","last_clean":"2024-09-10T23:39:26.771241+0000","last_became_active":"2024-09-10T23:39:25.604500+0000","last_became_peered":"2024-09-10T23:39:25.604500+0000","last_unstale":"2024-09-10T23:39:26.771241+0000","last_undegraded":"2024-09-10T23:39:26.771241+0000","last_fullsized":"2024-09-10T23:39:26.771241+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:48.574961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.614155+0000","last_change":"2024-09-10T23:39:25.628400+0000","last_active":"2024-09-10T23:39:26.614155+0000","last_peered":"2024-09-10T23:39:26.614155+0000","last_clean":"2024-09-10T23:39:26.614155+0000","last_became_active":"2024-09-10T23:39:25.628145+0000","last_became_peered":"2024-09-10T23:39:25.628145+0000","last_unstale":"2024-09-10T23:39:26.614155+0000","last_undegraded":"2024-09-10T23:39:26.614155+0000","last_fullsized":"2024-09-10T23:39:26.614155+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:16.865263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771391+0000","last_change":"2024-09-10T23:39:25.601870+0000","last_active":"2024-09-10T23:39:26.771391+0000","last_peered":"2024-09-10T23:39:26.771391+0000","last_clean":"2024-09-10T23:39:26.771391+0000","last_became_active":"2024-09-10T23:39:25.601626+0000","last_became_peered":"2024-09-10T23:39:25.601626+0000","last_unstale":"2024-09-10T23:39:26.771391+0000","last_undegraded":"2024-09-10T23:39:26.771391+0000","last_fullsized":"2024-09-10T23:39:26.771391+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:58.274675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613082+0000","last_change":"2024-09-10T23:39:25.610039+0000","last_active":"2024-09-10T23:39:26.613082+0000","last_peered":"2024-09-10T23:39:26.613082+0000","last_clean":"2024-09-10T23:39:26.613082+0000","last_became_active":"2024-09-10T23:39:25.609750+0000","last_became_peered":"2024-09-10T23:39:25.609750+0000","last_unstale":"2024-09-10T23:39:26.613082+0000","last_undegraded":"2024-09-10T23:39:26.613082+0000","last_fullsized":"2024-09-10T23:39:26.613082+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:04.284460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623971+0000","last_change":"2024-09-10T23:39:25.604568+0000","last_active":"2024-09-10T23:39:26.623971+0000","last_peered":"2024-09-10T23:39:26.623971+0000","last_clean":"2024-09-10T23:39:26.623971+0000","last_became_active":"2024-09-10T23:39:25.602741+0000","last_became_peered":"2024-09-10T23:39:25.602741+0000","last_unstale":"2024-09-10T23:39:26.623971+0000","last_undegraded":"2024-09-10T23:39:26.623971+0000","last_fullsized":"2024-09-10T23:39:26.623971+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:02.049706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770660+0000","last_change":"2024-09-10T23:39:25.612221+0000","last_active":"2024-09-10T23:39:26.770660+0000","last_peered":"2024-09-10T23:39:26.770660+0000","last_clean":"2024-09-10T23:39:26.770660+0000","last_became_active":"2024-09-10T23:39:25.611960+0000","last_became_peered":"2024-09-10T23:39:25.611960+0000","last_unstale":"2024-09-10T23:39:26.770660+0000","last_undegraded":"2024-09-10T23:39:26.770660+0000","last_fullsized":"2024-09-10T23:39:26.770660+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:16.865263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623641+0000","last_change":"2024-09-10T23:39:25.605125+0000","last_active":"2024-09-10T23:39:26.623641+0000","last_peered":"2024-09-10T23:39:26.623641+0000","last_clean":"2024-09-10T23:39:26.623641+0000","last_became_active":"2024-09-10T23:39:25.604846+0000","last_became_peered":"2024-09-10T23:39:25.604846+0000","last_unstale":"2024-09-10T23:39:26.623641+0000","last_undegraded":"2024-09-10T23:39:26.623641+0000","last_fullsized":"2024-09-10T23:39:26.623641+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:04:09.326143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770570+0000","last_change":"2024-09-10T23:39:25.620896+0000","last_active":"2024-09-10T23:39:26.770570+0000","last_peered":"2024-09-10T23:39:26.770570+0000","last_clean":"2024-09-10T23:39:26.770570+0000","last_became_active":"2024-09-10T23:39:25.620458+0000","last_became_peered":"2024-09-10T23:39:25.620458+0000","last_unstale":"2024-09-10T23:39:26.770570+0000","last_undegraded":"2024-09-10T23:39:26.770570+0000","last_fullsized":"2024-09-10T23:39:26.770570+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:10.131156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771826+0000","last_change":"2024-09-10T23:39:25.622751+0000","last_active":"2024-09-10T23:39:26.771826+0000","last_peered":"2024-09-10T23:39:26.771826+0000","last_clean":"2024-09-10T23:39:26.771826+0000","last_became_active":"2024-09-10T23:39:25.622298+0000","last_became_peered":"2024-09-10T23:39:25.622298+0000","last_unstale":"2024-09-10T23:39:26.771826+0000","last_undegraded":"2024-09-10T23:39:26.771826+0000","last_fullsized":"2024-09-10T23:39:26.771826+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:15.498532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"137'4","reported_seq":19,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.624511+0000","last_change":"2024-09-10T23:39:24.613693+0000","last_active":"2024-09-10T23:39:26.624511+0000","last_peered":"2024-09-10T23:39:26.624511+0000","last_clean":"2024-09-10T23:39:26.624511+0000","last_became_active":"2024-09-10T23:39:24.613434+0000","last_became_peered":"2024-09-10T23:39:24.613434+0000","last_unstale":"2024-09-10T23:39:26.624511+0000","last_undegraded":"2024-09-10T23:39:26.624511+0000","last_fullsized":"2024-09-10T23:39:26.624511+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:12.622316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.624451+0000","last_change":"2024-09-10T23:39:25.611809+0000","last_active":"2024-09-10T23:39:26.624451+0000","last_peered":"2024-09-10T23:39:26.624451+0000","last_clean":"2024-09-10T23:39:26.624451+0000","last_became_active":"2024-09-10T23:39:25.611469+0000","last_became_peered":"2024-09-10T23:39:25.611469+0000","last_unstale":"2024-09-10T23:39:26.624451+0000","last_undegraded":"2024-09-10T23:39:26.624451+0000","last_fullsized":"2024-09-10T23:39:26.624451+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:37.443789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770571+0000","last_change":"2024-09-10T23:39:24.607480+0000","last_active":"2024-09-10T23:39:26.770571+0000","last_peered":"2024-09-10T23:39:26.770571+0000","last_clean":"2024-09-10T23:39:26.770571+0000","last_became_active":"2024-09-10T23:39:24.607219+0000","last_became_peered":"2024-09-10T23:39:24.607219+0000","last_unstale":"2024-09-10T23:39:26.770571+0000","last_undegraded":"2024-09-10T23:39:26.770571+0000","last_fullsized":"2024-09-10T23:39:26.770571+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:58.995560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771031+0000","last_change":"2024-09-10T23:39:25.622142+0000","last_active":"2024-09-10T23:39:26.771031+0000","last_peered":"2024-09-10T23:39:26.771031+0000","last_clean":"2024-09-10T23:39:26.771031+0000","last_became_active":"2024-09-10T23:39:25.621847+0000","last_became_peered":"2024-09-10T23:39:25.621847+0000","last_unstale":"2024-09-10T23:39:26.771031+0000","last_undegraded":"2024-09-10T23:39:26.771031+0000","last_fullsized":"2024-09-10T23:39:26.771031+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:13:03.413049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770835+0000","last_change":"2024-09-10T23:39:24.614764+0000","last_active":"2024-09-10T23:39:26.770835+0000","last_peered":"2024-09-10T23:39:26.770835+0000","last_clean":"2024-09-10T23:39:26.770835+0000","last_became_active":"2024-09-10T23:39:24.614433+0000","last_became_peered":"2024-09-10T23:39:24.614433+0000","last_unstale":"2024-09-10T23:39:26.770835+0000","last_undegraded":"2024-09-10T23:39:26.770835+0000","last_fullsized":"2024-09-10T23:39:26.770835+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:30.711587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770360+0000","last_change":"2024-09-10T23:39:25.620932+0000","last_active":"2024-09-10T23:39:26.770360+0000","last_peered":"2024-09-10T23:39:26.770360+0000","last_clean":"2024-09-10T23:39:26.770360+0000","last_became_active":"2024-09-10T23:39:25.620674+0000","last_became_peered":"2024-09-10T23:39:25.620674+0000","last_unstale":"2024-09-10T23:39:26.770360+0000","last_undegraded":"2024-09-10T23:39:26.770360+0000","last_fullsized":"2024-09-10T23:39:26.770360+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:04:09.326143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613713+0000","last_change":"2024-09-10T23:39:24.613135+0000","last_active":"2024-09-10T23:39:26.613713+0000","last_peered":"2024-09-10T23:39:26.613713+0000","last_clean":"2024-09-10T23:39:26.613713+0000","last_became_active":"2024-09-10T23:39:24.612820+0000","last_became_peered":"2024-09-10T23:39:24.612820+0000","last_unstale":"2024-09-10T23:39:26.613713+0000","last_undegraded":"2024-09-10T23:39:26.613713+0000","last_fullsized":"2024-09-10T23:39:26.613713+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:52.165161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771368+0000","last_change":"2024-09-10T23:39:25.615602+0000","last_active":"2024-09-10T23:39:26.771368+0000","last_peered":"2024-09-10T23:39:26.771368+0000","last_clean":"2024-09-10T23:39:26.771368+0000","last_became_active":"2024-09-10T23:39:25.615326+0000","last_became_peered":"2024-09-10T23:39:25.615326+0000","last_unstale":"2024-09-10T23:39:26.771368+0000","last_undegraded":"2024-09-10T23:39:26.771368+0000","last_fullsized":"2024-09-10T23:39:26.771368+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:50.984162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.624061+0000","last_change":"2024-09-10T23:39:24.611866+0000","last_active":"2024-09-10T23:39:26.624061+0000","last_peered":"2024-09-10T23:39:26.624061+0000","last_clean":"2024-09-10T23:39:26.624061+0000","last_became_active":"2024-09-10T23:39:24.609939+0000","last_became_peered":"2024-09-10T23:39:24.609939+0000","last_unstale":"2024-09-10T23:39:26.624061+0000","last_undegraded":"2024-09-10T23:39:26.624061+0000","last_fullsized":"2024-09-10T23:39:26.624061+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:09:08.126983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770754+0000","last_change":"2024-09-10T23:39:25.628310+0000","last_active":"2024-09-10T23:39:26.770754+0000","last_peered":"2024-09-10T23:39:26.770754+0000","last_clean":"2024-09-10T23:39:26.770754+0000","last_became_active":"2024-09-10T23:39:25.625779+0000","last_became_peered":"2024-09-10T23:39:25.625779+0000","last_unstale":"2024-09-10T23:39:26.770754+0000","last_undegraded":"2024-09-10T23:39:26.770754+0000","last_fullsized":"2024-09-10T23:39:26.770754+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:54.404349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771361+0000","last_change":"2024-09-10T23:39:24.611667+0000","last_active":"2024-09-10T23:39:26.771361+0000","last_peered":"2024-09-10T23:39:26.771361+0000","last_clean":"2024-09-10T23:39:26.771361+0000","last_became_active":"2024-09-10T23:39:24.611382+0000","last_became_peered":"2024-09-10T23:39:24.611382+0000","last_unstale":"2024-09-10T23:39:26.771361+0000","last_undegraded":"2024-09-10T23:39:26.771361+0000","last_fullsized":"2024-09-10T23:39:26.771361+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:51:30.734026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.612258+0000","last_change":"2024-09-10T23:39:25.614231+0000","last_active":"2024-09-10T23:39:26.612258+0000","last_peered":"2024-09-10T23:39:26.612258+0000","last_clean":"2024-09-10T23:39:26.612258+0000","last_became_active":"2024-09-10T23:39:25.613918+0000","last_became_peered":"2024-09-10T23:39:25.613918+0000","last_unstale":"2024-09-10T23:39:26.612258+0000","last_undegraded":"2024-09-10T23:39:26.612258+0000","last_fullsized":"2024-09-10T23:39:26.612258+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:24.696106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623699+0000","last_change":"2024-09-10T23:39:24.613602+0000","last_active":"2024-09-10T23:39:26.623699+0000","last_peered":"2024-09-10T23:39:26.623699+0000","last_clean":"2024-09-10T23:39:26.623699+0000","last_became_active":"2024-09-10T23:39:24.613218+0000","last_became_peered":"2024-09-10T23:39:24.613218+0000","last_unstale":"2024-09-10T23:39:26.623699+0000","last_undegraded":"2024-09-10T23:39:26.623699+0000","last_fullsized":"2024-09-10T23:39:26.623699+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:11.204076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770475+0000","last_change":"2024-09-10T23:39:25.608322+0000","last_active":"2024-09-10T23:39:26.770475+0000","last_peered":"2024-09-10T23:39:26.770475+0000","last_clean":"2024-09-10T23:39:26.770475+0000","last_became_active":"2024-09-10T23:39:25.608045+0000","last_became_peered":"2024-09-10T23:39:25.608045+0000","last_unstale":"2024-09-10T23:39:26.770475+0000","last_undegraded":"2024-09-10T23:39:26.770475+0000","last_fullsized":"2024-09-10T23:39:26.770475+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:21:26.921897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613054+0000","last_change":"2024-09-10T23:39:24.613507+0000","last_active":"2024-09-10T23:39:26.613054+0000","last_peered":"2024-09-10T23:39:26.613054+0000","last_clean":"2024-09-10T23:39:26.613054+0000","last_became_active":"2024-09-10T23:39:24.613241+0000","last_became_peered":"2024-09-10T23:39:24.613241+0000","last_unstale":"2024-09-10T23:39:26.613054+0000","last_undegraded":"2024-09-10T23:39:26.613054+0000","last_fullsized":"2024-09-10T23:39:26.613054+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:26:03.251113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613215+0000","last_change":"2024-09-10T23:39:25.620619+0000","last_active":"2024-09-10T23:39:26.613215+0000","last_peered":"2024-09-10T23:39:26.613215+0000","last_clean":"2024-09-10T23:39:26.613215+0000","last_became_active":"2024-09-10T23:39:25.620354+0000","last_became_peered":"2024-09-10T23:39:25.620354+0000","last_unstale":"2024-09-10T23:39:26.613215+0000","last_undegraded":"2024-09-10T23:39:26.613215+0000","last_fullsized":"2024-09-10T23:39:26.613215+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:11.219480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770943+0000","last_change":"2024-09-10T23:39:24.617299+0000","last_active":"2024-09-10T23:39:26.770943+0000","last_peered":"2024-09-10T23:39:26.770943+0000","last_clean":"2024-09-10T23:39:26.770943+0000","last_became_active":"2024-09-10T23:39:24.616499+0000","last_became_peered":"2024-09-10T23:39:24.616499+0000","last_unstale":"2024-09-10T23:39:26.770943+0000","last_undegraded":"2024-09-10T23:39:26.770943+0000","last_fullsized":"2024-09-10T23:39:26.770943+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:38.774810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613213+0000","last_change":"2024-09-10T23:39:25.626046+0000","last_active":"2024-09-10T23:39:26.613213+0000","last_peered":"2024-09-10T23:39:26.613213+0000","last_clean":"2024-09-10T23:39:26.613213+0000","last_became_active":"2024-09-10T23:39:25.625710+0000","last_became_peered":"2024-09-10T23:39:25.625710+0000","last_unstale":"2024-09-10T23:39:26.613213+0000","last_undegraded":"2024-09-10T23:39:26.613213+0000","last_fullsized":"2024-09-10T23:39:26.613213+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:11.300959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"137'3","reported_seq":18,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613953+0000","last_change":"2024-09-10T23:39:24.615133+0000","last_active":"2024-09-10T23:39:26.613953+0000","last_peered":"2024-09-10T23:39:26.613953+0000","last_clean":"2024-09-10T23:39:26.613953+0000","last_became_active":"2024-09-10T23:39:24.614835+0000","last_became_peered":"2024-09-10T23:39:24.614835+0000","last_unstale":"2024-09-10T23:39:26.613953+0000","last_undegraded":"2024-09-10T23:39:26.613953+0000","last_fullsized":"2024-09-10T23:39:26.613953+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:30.704948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.624143+0000","last_change":"2024-09-10T23:39:25.604223+0000","last_active":"2024-09-10T23:39:26.624143+0000","last_peered":"2024-09-10T23:39:26.624143+0000","last_clean":"2024-09-10T23:39:26.624143+0000","last_became_active":"2024-09-10T23:39:25.603911+0000","last_became_peered":"2024-09-10T23:39:25.603911+0000","last_unstale":"2024-09-10T23:39:26.624143+0000","last_undegraded":"2024-09-10T23:39:26.624143+0000","last_fullsized":"2024-09-10T23:39:26.624143+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:16.865263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"137'6","reported_seq":21,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771013+0000","last_change":"2024-09-10T23:39:24.605325+0000","last_active":"2024-09-10T23:39:26.771013+0000","last_peered":"2024-09-10T23:39:26.771013+0000","last_clean":"2024-09-10T23:39:26.771013+0000","last_became_active":"2024-09-10T23:39:24.605043+0000","last_became_peered":"2024-09-10T23:39:24.605043+0000","last_unstale":"2024-09-10T23:39:26.771013+0000","last_undegraded":"2024-09-10T23:39:26.771013+0000","last_fullsized":"2024-09-10T23:39:26.771013+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:39.200145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.612671+0000","last_change":"2024-09-10T23:39:25.609074+0000","last_active":"2024-09-10T23:39:26.612671+0000","last_peered":"2024-09-10T23:39:26.612671+0000","last_clean":"2024-09-10T23:39:26.612671+0000","last_became_active":"2024-09-10T23:39:25.608797+0000","last_became_peered":"2024-09-10T23:39:25.608797+0000","last_unstale":"2024-09-10T23:39:26.612671+0000","last_undegraded":"2024-09-10T23:39:26.612671+0000","last_fullsized":"2024-09-10T23:39:26.612671+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:43.837353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771433+0000","last_change":"2024-09-10T23:39:24.608738+0000","last_active":"2024-09-10T23:39:26.771433+0000","last_peered":"2024-09-10T23:39:26.771433+0000","last_clean":"2024-09-10T23:39:26.771433+0000","last_became_active":"2024-09-10T23:39:24.608463+0000","last_became_peered":"2024-09-10T23:39:24.608463+0000","last_unstale":"2024-09-10T23:39:26.771433+0000","last_undegraded":"2024-09-10T23:39:26.771433+0000","last_fullsized":"2024-09-10T23:39:26.771433+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:43.715096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613705+0000","last_change":"2024-09-10T23:39:25.613818+0000","last_active":"2024-09-10T23:39:26.613705+0000","last_peered":"2024-09-10T23:39:26.613705+0000","last_clean":"2024-09-10T23:39:26.613705+0000","last_became_active":"2024-09-10T23:39:25.613345+0000","last_became_peered":"2024-09-10T23:39:25.613345+0000","last_unstale":"2024-09-10T23:39:26.613705+0000","last_undegraded":"2024-09-10T23:39:26.613705+0000","last_fullsized":"2024-09-10T23:39:26.613705+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:22.371011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771025+0000","last_change":"2024-09-10T23:39:24.614850+0000","last_active":"2024-09-10T23:39:26.771025+0000","last_peered":"2024-09-10T23:39:26.771025+0000","last_clean":"2024-09-10T23:39:26.771025+0000","last_became_active":"2024-09-10T23:39:24.612966+0000","last_became_peered":"2024-09-10T23:39:24.612966+0000","last_unstale":"2024-09-10T23:39:26.771025+0000","last_undegraded":"2024-09-10T23:39:26.771025+0000","last_fullsized":"2024-09-10T23:39:26.771025+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:38.302646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770478+0000","last_change":"2024-09-10T23:39:25.620075+0000","last_active":"2024-09-10T23:39:26.770478+0000","last_peered":"2024-09-10T23:39:26.770478+0000","last_clean":"2024-09-10T23:39:26.770478+0000","last_became_active":"2024-09-10T23:39:25.619740+0000","last_became_peered":"2024-09-10T23:39:25.619740+0000","last_unstale":"2024-09-10T23:39:26.770478+0000","last_undegraded":"2024-09-10T23:39:26.770478+0000","last_fullsized":"2024-09-10T23:39:26.770478+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:00.828349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.612773+0000","last_change":"2024-09-10T23:39:24.609752+0000","last_active":"2024-09-10T23:39:26.612773+0000","last_peered":"2024-09-10T23:39:26.612773+0000","last_clean":"2024-09-10T23:39:26.612773+0000","last_became_active":"2024-09-10T23:39:24.609476+0000","last_became_peered":"2024-09-10T23:39:24.609476+0000","last_unstale":"2024-09-10T23:39:26.612773+0000","last_undegraded":"2024-09-10T23:39:26.612773+0000","last_fullsized":"2024-09-10T23:39:26.612773+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:21.564871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770685+0000","last_change":"2024-09-10T23:39:25.607888+0000","last_active":"2024-09-10T23:39:26.770685+0000","last_peered":"2024-09-10T23:39:26.770685+0000","last_clean":"2024-09-10T23:39:26.770685+0000","last_became_active":"2024-09-10T23:39:25.607637+0000","last_became_peered":"2024-09-10T23:39:25.607637+0000","last_unstale":"2024-09-10T23:39:26.770685+0000","last_undegraded":"2024-09-10T23:39:26.770685+0000","last_fullsized":"2024-09-10T23:39:26.770685+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:18.794229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"137'8","reported_seq":23,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770807+0000","last_change":"2024-09-10T23:39:24.604950+0000","last_active":"2024-09-10T23:39:26.770807+0000","last_peered":"2024-09-10T23:39:26.770807+0000","last_clean":"2024-09-10T23:39:26.770807+0000","last_became_active":"2024-09-10T23:39:24.604022+0000","last_became_peered":"2024-09-10T23:39:24.604022+0000","last_unstale":"2024-09-10T23:39:26.770807+0000","last_undegraded":"2024-09-10T23:39:26.770807+0000","last_fullsized":"2024-09-10T23:39:26.770807+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:16.225255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613286+0000","last_change":"2024-09-10T23:39:25.622750+0000","last_active":"2024-09-10T23:39:26.613286+0000","last_peered":"2024-09-10T23:39:26.613286+0000","last_clean":"2024-09-10T23:39:26.613286+0000","last_became_active":"2024-09-10T23:39:25.622483+0000","last_became_peered":"2024-09-10T23:39:25.622483+0000","last_unstale":"2024-09-10T23:39:26.613286+0000","last_undegraded":"2024-09-10T23:39:26.613286+0000","last_fullsized":"2024-09-10T23:39:26.613286+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:14.711437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771233+0000","last_change":"2024-09-10T23:39:24.605628+0000","last_active":"2024-09-10T23:39:26.771233+0000","last_peered":"2024-09-10T23:39:26.771233+0000","last_clean":"2024-09-10T23:39:26.771233+0000","last_became_active":"2024-09-10T23:39:24.605367+0000","last_became_peered":"2024-09-10T23:39:24.605367+0000","last_unstale":"2024-09-10T23:39:26.771233+0000","last_undegraded":"2024-09-10T23:39:26.771233+0000","last_fullsized":"2024-09-10T23:39:26.771233+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:01.508944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771775+0000","last_change":"2024-09-10T23:39:25.615157+0000","last_active":"2024-09-10T23:39:26.771775+0000","last_peered":"2024-09-10T23:39:26.771775+0000","last_clean":"2024-09-10T23:39:26.771775+0000","last_became_active":"2024-09-10T23:39:25.614824+0000","last_became_peered":"2024-09-10T23:39:25.614824+0000","last_unstale":"2024-09-10T23:39:26.771775+0000","last_undegraded":"2024-09-10T23:39:26.771775+0000","last_fullsized":"2024-09-10T23:39:26.771775+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:16.697918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770517+0000","last_change":"2024-09-10T23:39:24.611479+0000","last_active":"2024-09-10T23:39:26.770517+0000","last_peered":"2024-09-10T23:39:26.770517+0000","last_clean":"2024-09-10T23:39:26.770517+0000","last_became_active":"2024-09-10T23:39:24.609193+0000","last_became_peered":"2024-09-10T23:39:24.609193+0000","last_unstale":"2024-09-10T23:39:26.770517+0000","last_undegraded":"2024-09-10T23:39:26.770517+0000","last_fullsized":"2024-09-10T23:39:26.770517+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:28:59.738155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771891+0000","last_change":"2024-09-10T23:39:25.618336+0000","last_active":"2024-09-10T23:39:26.771891+0000","last_peered":"2024-09-10T23:39:26.771891+0000","last_clean":"2024-09-10T23:39:26.771891+0000","last_became_active":"2024-09-10T23:39:25.617985+0000","last_became_peered":"2024-09-10T23:39:25.617985+0000","last_unstale":"2024-09-10T23:39:26.771891+0000","last_undegraded":"2024-09-10T23:39:26.771891+0000","last_fullsized":"2024-09-10T23:39:26.771891+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:20.015474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"137'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771334+0000","last_change":"2024-09-10T23:39:24.602179+0000","last_active":"2024-09-10T23:39:26.771334+0000","last_peered":"2024-09-10T23:39:26.771334+0000","last_clean":"2024-09-10T23:39:26.771334+0000","last_became_active":"2024-09-10T23:39:24.601773+0000","last_became_peered":"2024-09-10T23:39:24.601773+0000","last_unstale":"2024-09-10T23:39:26.771334+0000","last_undegraded":"2024-09-10T23:39:26.771334+0000","last_fullsized":"2024-09-10T23:39:26.771334+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:31.181082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.614056+0000","last_change":"2024-09-10T23:39:25.616269+0000","last_active":"2024-09-10T23:39:26.614056+0000","last_peered":"2024-09-10T23:39:26.614056+0000","last_clean":"2024-09-10T23:39:26.614056+0000","last_became_active":"2024-09-10T23:39:25.616003+0000","last_became_peered":"2024-09-10T23:39:25.616003+0000","last_unstale":"2024-09-10T23:39:26.614056+0000","last_undegraded":"2024-09-10T23:39:26.614056+0000","last_fullsized":"2024-09-10T23:39:26.614056+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:09.732712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.611271+0000","last_change":"2024-09-10T23:39:24.615545+0000","last_active":"2024-09-10T23:39:26.611271+0000","last_peered":"2024-09-10T23:39:26.611271+0000","last_clean":"2024-09-10T23:39:26.611271+0000","last_became_active":"2024-09-10T23:39:24.615120+0000","last_became_peered":"2024-09-10T23:39:24.615120+0000","last_unstale":"2024-09-10T23:39:26.611271+0000","last_undegraded":"2024-09-10T23:39:26.611271+0000","last_fullsized":"2024-09-10T23:39:26.611271+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:25.247342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623731+0000","last_change":"2024-09-10T23:39:25.605665+0000","last_active":"2024-09-10T23:39:26.623731+0000","last_peered":"2024-09-10T23:39:26.623731+0000","last_clean":"2024-09-10T23:39:26.623731+0000","last_became_active":"2024-09-10T23:39:25.605411+0000","last_became_peered":"2024-09-10T23:39:25.605411+0000","last_unstale":"2024-09-10T23:39:26.623731+0000","last_undegraded":"2024-09-10T23:39:26.623731+0000","last_fullsized":"2024-09-10T23:39:26.623731+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:19:30.378031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613837+0000","last_change":"2024-09-10T23:39:24.611435+0000","last_active":"2024-09-10T23:39:26.613837+0000","last_peered":"2024-09-10T23:39:26.613837+0000","last_clean":"2024-09-10T23:39:26.613837+0000","last_became_active":"2024-09-10T23:39:24.611148+0000","last_became_peered":"2024-09-10T23:39:24.611148+0000","last_unstale":"2024-09-10T23:39:26.613837+0000","last_undegraded":"2024-09-10T23:39:26.613837+0000","last_fullsized":"2024-09-10T23:39:26.613837+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:17.956462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613770+0000","last_change":"2024-09-10T23:39:25.616933+0000","last_active":"2024-09-10T23:39:26.613770+0000","last_peered":"2024-09-10T23:39:26.613770+0000","last_clean":"2024-09-10T23:39:26.613770+0000","last_became_active":"2024-09-10T23:39:25.616682+0000","last_became_peered":"2024-09-10T23:39:25.616682+0000","last_unstale":"2024-09-10T23:39:26.613770+0000","last_undegraded":"2024-09-10T23:39:26.613770+0000","last_fullsized":"2024-09-10T23:39:26.613770+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:22.597132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.610967+0000","last_change":"2024-09-10T23:39:24.611840+0000","last_active":"2024-09-10T23:39:26.610967+0000","last_peered":"2024-09-10T23:39:26.610967+0000","last_clean":"2024-09-10T23:39:26.610967+0000","last_became_active":"2024-09-10T23:39:24.611521+0000","last_became_peered":"2024-09-10T23:39:24.611521+0000","last_unstale":"2024-09-10T23:39:26.610967+0000","last_undegraded":"2024-09-10T23:39:26.610967+0000","last_fullsized":"2024-09-10T23:39:26.610967+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:55.241122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623567+0000","last_change":"2024-09-10T23:39:25.606700+0000","last_active":"2024-09-10T23:39:26.623567+0000","last_peered":"2024-09-10T23:39:26.623567+0000","last_clean":"2024-09-10T23:39:26.623567+0000","last_became_active":"2024-09-10T23:39:25.606449+0000","last_became_peered":"2024-09-10T23:39:25.606449+0000","last_unstale":"2024-09-10T23:39:26.623567+0000","last_undegraded":"2024-09-10T23:39:26.623567+0000","last_fullsized":"2024-09-10T23:39:26.623567+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:45.863089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770993+0000","last_change":"2024-09-10T23:39:24.616440+0000","last_active":"2024-09-10T23:39:26.770993+0000","last_peered":"2024-09-10T23:39:26.770993+0000","last_clean":"2024-09-10T23:39:26.770993+0000","last_became_active":"2024-09-10T23:39:24.616027+0000","last_became_peered":"2024-09-10T23:39:24.616027+0000","last_unstale":"2024-09-10T23:39:26.770993+0000","last_undegraded":"2024-09-10T23:39:26.770993+0000","last_fullsized":"2024-09-10T23:39:26.770993+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:35:56.271120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.612544+0000","last_change":"2024-09-10T23:39:25.605960+0000","last_active":"2024-09-10T23:39:26.612544+0000","last_peered":"2024-09-10T23:39:26.612544+0000","last_clean":"2024-09-10T23:39:26.612544+0000","last_became_active":"2024-09-10T23:39:25.605289+0000","last_became_peered":"2024-09-10T23:39:25.605289+0000","last_unstale":"2024-09-10T23:39:26.612544+0000","last_undegraded":"2024-09-10T23:39:26.612544+0000","last_fullsized":"2024-09-10T23:39:26.612544+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:00.623663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.610996+0000","last_change":"2024-09-10T23:39:24.610231+0000","last_active":"2024-09-10T23:39:26.610996+0000","last_peered":"2024-09-10T23:39:26.610996+0000","last_clean":"2024-09-10T23:39:26.610996+0000","last_became_active":"2024-09-10T23:39:24.609932+0000","last_became_peered":"2024-09-10T23:39:24.609932+0000","last_unstale":"2024-09-10T23:39:26.610996+0000","last_undegraded":"2024-09-10T23:39:26.610996+0000","last_fullsized":"2024-09-10T23:39:26.610996+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:08.557847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.614153+0000","last_change":"2024-09-10T23:39:25.630602+0000","last_active":"2024-09-10T23:39:26.614153+0000","last_peered":"2024-09-10T23:39:26.614153+0000","last_clean":"2024-09-10T23:39:26.614153+0000","last_became_active":"2024-09-10T23:39:25.628124+0000","last_became_peered":"2024-09-10T23:39:25.628124+0000","last_unstale":"2024-09-10T23:39:26.614153+0000","last_undegraded":"2024-09-10T23:39:26.614153+0000","last_fullsized":"2024-09-10T23:39:26.614153+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:08.211937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"137'2","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770826+0000","last_change":"2024-09-10T23:39:24.614991+0000","last_active":"2024-09-10T23:39:26.770826+0000","last_peered":"2024-09-10T23:39:26.770826+0000","last_clean":"2024-09-10T23:39:26.770826+0000","last_became_active":"2024-09-10T23:39:24.614716+0000","last_became_peered":"2024-09-10T23:39:24.614716+0000","last_unstale":"2024-09-10T23:39:26.770826+0000","last_undegraded":"2024-09-10T23:39:26.770826+0000","last_fullsized":"2024-09-10T23:39:26.770826+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:18:50.421286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771873+0000","last_change":"2024-09-10T23:39:25.610111+0000","last_active":"2024-09-10T23:39:26.771873+0000","last_peered":"2024-09-10T23:39:26.771873+0000","last_clean":"2024-09-10T23:39:26.771873+0000","last_became_active":"2024-09-10T23:39:25.609828+0000","last_became_peered":"2024-09-10T23:39:25.609828+0000","last_unstale":"2024-09-10T23:39:26.771873+0000","last_undegraded":"2024-09-10T23:39:26.771873+0000","last_fullsized":"2024-09-10T23:39:26.771873+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:28.639580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"138'1","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.622884+0000","last_change":"2024-09-10T23:39:24.614541+0000","last_active":"2024-09-10T23:39:26.622884+0000","last_peered":"2024-09-10T23:39:26.622884+0000","last_clean":"2024-09-10T23:39:26.622884+0000","last_became_active":"2024-09-10T23:39:24.614160+0000","last_became_peered":"2024-09-10T23:39:24.614160+0000","last_unstale":"2024-09-10T23:39:26.622884+0000","last_undegraded":"2024-09-10T23:39:26.622884+0000","last_fullsized":"2024-09-10T23:39:26.622884+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:26.965773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771903+0000","last_change":"2024-09-10T23:39:25.603640+0000","last_active":"2024-09-10T23:39:26.771903+0000","last_peered":"2024-09-10T23:39:26.771903+0000","last_clean":"2024-09-10T23:39:26.771903+0000","last_became_active":"2024-09-10T23:39:25.603263+0000","last_became_peered":"2024-09-10T23:39:25.603263+0000","last_unstale":"2024-09-10T23:39:26.771903+0000","last_undegraded":"2024-09-10T23:39:26.771903+0000","last_fullsized":"2024-09-10T23:39:26.771903+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:39.571626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.624248+0000","last_change":"2024-09-10T23:39:24.616973+0000","last_active":"2024-09-10T23:39:26.624248+0000","last_peered":"2024-09-10T23:39:26.624248+0000","last_clean":"2024-09-10T23:39:26.624248+0000","last_became_active":"2024-09-10T23:39:24.616645+0000","last_became_peered":"2024-09-10T23:39:24.616645+0000","last_unstale":"2024-09-10T23:39:26.624248+0000","last_undegraded":"2024-09-10T23:39:26.624248+0000","last_fullsized":"2024-09-10T23:39:26.624248+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:29.254303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771574+0000","last_change":"2024-09-10T23:39:25.608971+0000","last_active":"2024-09-10T23:39:26.771574+0000","last_peered":"2024-09-10T23:39:26.771574+0000","last_clean":"2024-09-10T23:39:26.771574+0000","last_became_active":"2024-09-10T23:39:25.607331+0000","last_became_peered":"2024-09-10T23:39:25.607331+0000","last_unstale":"2024-09-10T23:39:26.771574+0000","last_undegraded":"2024-09-10T23:39:26.771574+0000","last_fullsized":"2024-09-10T23:39:26.771574+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:35:19.218449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613759+0000","last_change":"2024-09-10T23:39:24.617160+0000","last_active":"2024-09-10T23:39:26.613759+0000","last_peered":"2024-09-10T23:39:26.613759+0000","last_clean":"2024-09-10T23:39:26.613759+0000","last_became_active":"2024-09-10T23:39:24.616866+0000","last_became_peered":"2024-09-10T23:39:24.616866+0000","last_unstale":"2024-09-10T23:39:26.613759+0000","last_undegraded":"2024-09-10T23:39:26.613759+0000","last_fullsized":"2024-09-10T23:39:26.613759+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:34.154690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770908+0000","last_change":"2024-09-10T23:39:25.625272+0000","last_active":"2024-09-10T23:39:26.770908+0000","last_peered":"2024-09-10T23:39:26.770908+0000","last_clean":"2024-09-10T23:39:26.770908+0000","last_became_active":"2024-09-10T23:39:25.622945+0000","last_became_peered":"2024-09-10T23:39:25.622945+0000","last_unstale":"2024-09-10T23:39:26.770908+0000","last_undegraded":"2024-09-10T23:39:26.770908+0000","last_fullsized":"2024-09-10T23:39:26.770908+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:58.427537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771169+0000","last_change":"2024-09-10T23:39:24.608424+0000","last_active":"2024-09-10T23:39:26.771169+0000","last_peered":"2024-09-10T23:39:26.771169+0000","last_clean":"2024-09-10T23:39:26.771169+0000","last_became_active":"2024-09-10T23:39:24.607786+0000","last_became_peered":"2024-09-10T23:39:24.607786+0000","last_unstale":"2024-09-10T23:39:26.771169+0000","last_undegraded":"2024-09-10T23:39:26.771169+0000","last_fullsized":"2024-09-10T23:39:26.771169+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:26.211296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623952+0000","last_change":"2024-09-10T23:39:25.605405+0000","last_active":"2024-09-10T23:39:26.623952+0000","last_peered":"2024-09-10T23:39:26.623952+0000","last_clean":"2024-09-10T23:39:26.623952+0000","last_became_active":"2024-09-10T23:39:25.605050+0000","last_became_peered":"2024-09-10T23:39:25.605050+0000","last_unstale":"2024-09-10T23:39:26.623952+0000","last_undegraded":"2024-09-10T23:39:26.623952+0000","last_fullsized":"2024-09-10T23:39:26.623952+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:16.919508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.612749+0000","last_change":"2024-09-10T23:39:24.608720+0000","last_active":"2024-09-10T23:39:26.612749+0000","last_peered":"2024-09-10T23:39:26.612749+0000","last_clean":"2024-09-10T23:39:26.612749+0000","last_became_active":"2024-09-10T23:39:24.608415+0000","last_became_peered":"2024-09-10T23:39:24.608415+0000","last_unstale":"2024-09-10T23:39:26.612749+0000","last_undegraded":"2024-09-10T23:39:26.612749+0000","last_fullsized":"2024-09-10T23:39:26.612749+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:58.259236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.612214+0000","last_change":"2024-09-10T23:39:25.608083+0000","last_active":"2024-09-10T23:39:26.612214+0000","last_peered":"2024-09-10T23:39:26.612214+0000","last_clean":"2024-09-10T23:39:26.612214+0000","last_became_active":"2024-09-10T23:39:25.607187+0000","last_became_peered":"2024-09-10T23:39:25.607187+0000","last_unstale":"2024-09-10T23:39:26.612214+0000","last_undegraded":"2024-09-10T23:39:26.612214+0000","last_fullsized":"2024-09-10T23:39:26.612214+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:15.169982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623751+0000","last_change":"2024-09-10T23:39:24.618247+0000","last_active":"2024-09-10T23:39:26.623751+0000","last_peered":"2024-09-10T23:39:26.623751+0000","last_clean":"2024-09-10T23:39:26.623751+0000","last_became_active":"2024-09-10T23:39:24.617042+0000","last_became_peered":"2024-09-10T23:39:24.617042+0000","last_unstale":"2024-09-10T23:39:26.623751+0000","last_undegraded":"2024-09-10T23:39:26.623751+0000","last_fullsized":"2024-09-10T23:39:26.623751+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:16.262731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623643+0000","last_change":"2024-09-10T23:39:25.624133+0000","last_active":"2024-09-10T23:39:26.623643+0000","last_peered":"2024-09-10T23:39:26.623643+0000","last_clean":"2024-09-10T23:39:26.623643+0000","last_became_active":"2024-09-10T23:39:25.623865+0000","last_became_peered":"2024-09-10T23:39:25.623865+0000","last_unstale":"2024-09-10T23:39:26.623643+0000","last_undegraded":"2024-09-10T23:39:26.623643+0000","last_fullsized":"2024-09-10T23:39:26.623643+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:04:52.290850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.623485+0000","last_change":"2024-09-10T23:39:24.610914+0000","last_active":"2024-09-10T23:39:26.623485+0000","last_peered":"2024-09-10T23:39:26.623485+0000","last_clean":"2024-09-10T23:39:26.623485+0000","last_became_active":"2024-09-10T23:39:24.608246+0000","last_became_peered":"2024-09-10T23:39:24.608246+0000","last_unstale":"2024-09-10T23:39:26.623485+0000","last_undegraded":"2024-09-10T23:39:26.623485+0000","last_fullsized":"2024-09-10T23:39:26.623485+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:51:17.812933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.613339+0000","last_change":"2024-09-10T23:39:25.605894+0000","last_active":"2024-09-10T23:39:26.613339+0000","last_peered":"2024-09-10T23:39:26.613339+0000","last_clean":"2024-09-10T23:39:26.613339+0000","last_became_active":"2024-09-10T23:39:25.605517+0000","last_became_peered":"2024-09-10T23:39:25.605517+0000","last_unstale":"2024-09-10T23:39:26.613339+0000","last_undegraded":"2024-09-10T23:39:26.613339+0000","last_fullsized":"2024-09-10T23:39:26.613339+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:24.580012+0000","last_clean_scrub_stamp":"2024-09-10T23:39:24.580012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:52.087639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.771101+0000","last_change":"2024-09-10T23:39:24.605009+0000","last_active":"2024-09-10T23:39:26.771101+0000","last_peered":"2024-09-10T23:39:26.771101+0000","last_clean":"2024-09-10T23:39:26.771101+0000","last_became_active":"2024-09-10T23:39:24.604483+0000","last_became_peered":"2024-09-10T23:39:24.604483+0000","last_unstale":"2024-09-10T23:39:26.771101+0000","last_undegraded":"2024-09-10T23:39:26.771101+0000","last_fullsized":"2024-09-10T23:39:26.771101+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:39:23.572398+0000","last_clean_scrub_stamp":"2024-09-10T23:39:23.572398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:48:04.197080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":350,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T23:39:26.770893+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:39:26.770893+0000","last_peered":"2024-09-10T23:39:26.770893+0000","last_clean":"2024-09-10T23:39:26.770893+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:39:26.770893+0000","last_undegraded":"2024-09-10T23:39:26.770893+0000","last_fullsized":"2024-09-10T23:39:26.770893+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509529,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1728512,"data_stored":760524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1732608,"data_stored":760558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509528,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33472,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338368,"statfs":{"total":96636764160,"available":96602488832,"internally_reserved":0,"allocated":2330624,"data_stored":1351564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509526,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1732608,"data_stored":760558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509527,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":1744896,"data_stored":762158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509528,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1732608,"data_stored":761106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509527,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37640,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334200,"statfs":{"total":96636764160,"available":96598220800,"internally_reserved":0,"allocated":2347008,"data_stored":1352686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509527,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1732608,"data_stored":760594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:39:30.098 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T23:39:30.099 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T23:39:30.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.280+0000 7f8e88400640 1 -- 172.21.15.94:0/2510456854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8015f2d0 msgr2=0x7f8e800b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:30.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.280+0000 7f8e88400640 1 --2- 172.21.15.94:0/2510456854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8015f2d0 0x7f8e800b15f0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009c80 tx=0x7f8e7401f5a0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e88400640 1 -- 172.21.15.94:0/2510456854 shutdown_connections 2024-09-10T23:39:30.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e88400640 1 --2- 172.21.15.94:0/2510456854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e800cfc00 0x7f8e800b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e88400640 1 --2- 172.21.15.94:0/2510456854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8015f2d0 0x7f8e800b15f0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e88400640 1 -- 172.21.15.94:0/2510456854 >> 172.21.15.94:0/2510456854 conn(0x7f8e800d1c90 msgr2=0x7f8e800d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:30.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e88400640 1 -- 172.21.15.94:0/2510456854 shutdown_connections 2024-09-10T23:39:30.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e88400640 1 -- 172.21.15.94:0/2510456854 wait complete. 2024-09-10T23:39:30.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e88400640 1 Processor -- start 2024-09-10T23:39:30.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e88400640 1 -- start start 2024-09-10T23:39:30.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e88400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e800b2790 0x7f8e80143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:30.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e88400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e800cfc00 0x7f8e800c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:30.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e88400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8015f2d0 0x7f8e8006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:30.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e88400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8015f830 con 0x7f8e8015f2d0 2024-09-10T23:39:30.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e88400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8015f6b0 con 0x7f8e800b2790 2024-09-10T23:39:30.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e88400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8e801661e0 con 0x7f8e800cfc00 2024-09-10T23:39:30.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e86400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8015f2d0 0x7f8e8006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:30.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e86400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8015f2d0 0x7f8e8006ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50356/0 (socket says 172.21.15.94:50356) 2024-09-10T23:39:30.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e86400640 1 -- 172.21.15.94:0/1576601396 learned_addr learned my addr 172.21.15.94:0/1576601396 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:30.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e85a00640 1 --2- 172.21.15.94:0/1576601396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e800b2790 0x7f8e80143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:30.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e85000640 1 --2- 172.21.15.94:0/1576601396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e800cfc00 0x7f8e800c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:30.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e86400640 1 -- 172.21.15.94:0/1576601396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e800cfc00 msgr2=0x7f8e800c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:30.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e86400640 1 --2- 172.21.15.94:0/1576601396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e800cfc00 0x7f8e800c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e86400640 1 -- 172.21.15.94:0/1576601396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e800b2790 msgr2=0x7f8e80143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:30.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e86400640 1 --2- 172.21.15.94:0/1576601396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e800b2790 0x7f8e80143240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e86400640 1 -- 172.21.15.94:0/1576601396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e800c6280 con 0x7f8e8015f2d0 2024-09-10T23:39:30.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e85000640 1 --2- 172.21.15.94:0/1576601396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e800cfc00 0x7f8e800c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:30.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.284+0000 7f8e86400640 1 --2- 172.21.15.94:0/1576601396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8015f2d0 0x7f8e8006ef60 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800c3c0 tx=0x7f8e7800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:30.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.288+0000 7f8e7d600640 1 -- 172.21.15.94:0/1576601396 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78011790 con 0x7f8e8015f2d0 2024-09-10T23:39:30.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.288+0000 7f8e7d600640 1 -- 172.21.15.94:0/1576601396 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e78017070 con 0x7f8e8015f2d0 2024-09-10T23:39:30.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.288+0000 7f8e7d600640 1 -- 172.21.15.94:0/1576601396 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78011cc0 con 0x7f8e8015f2d0 2024-09-10T23:39:30.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.288+0000 7f8e88400640 1 -- 172.21.15.94:0/1576601396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e80143b20 con 0x7f8e8015f2d0 2024-09-10T23:39:30.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.288+0000 7f8e88400640 1 -- 172.21.15.94:0/1576601396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e80154af0 con 0x7f8e8015f2d0 2024-09-10T23:39:30.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.288+0000 7f8e88400640 1 -- 172.21.15.94:0/1576601396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e80071f00 con 0x7f8e8015f2d0 2024-09-10T23:39:30.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.288+0000 7f8e7d600640 1 -- 172.21.15.94:0/1576601396 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f8e7801c460 con 0x7f8e8015f2d0 2024-09-10T23:39:30.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.288+0000 7f8e7d600640 1 --2- 172.21.15.94:0/1576601396 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8e60072ca0 0x7f8e60075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:30.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.288+0000 7f8e7d600640 1 -- 172.21.15.94:0/1576601396 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7f8e78093b60 con 0x7f8e8015f2d0 2024-09-10T23:39:30.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.288+0000 7f8e85a00640 1 --2- 172.21.15.94:0/1576601396 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8e60072ca0 0x7f8e60075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:30.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.292+0000 7f8e85a00640 1 --2- 172.21.15.94:0/1576601396 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8e60072ca0 0x7f8e60075160 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8e740b3000 tx=0x7f8e7401dde0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:30.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.292+0000 7f8e7d600640 1 -- 172.21.15.94:0/1576601396 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8e7805e910 con 0x7f8e8015f2d0 2024-09-10T23:39:30.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.516+0000 7f8e88400640 1 -- 172.21.15.94:0/1576601396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f8e80071c20 con 0x7f8e8015f2d0 2024-09-10T23:39:30.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.520+0000 7f8e7d600640 1 -- 172.21.15.94:0/1576601396 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v38) ==== 190+0+147 (secure 0 0 0) 0x7f8e7805e210 con 0x7f8e8015f2d0 2024-09-10T23:39:30.522 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-10T23:39:30.522 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g== 2024-09-10T23:39:30.522 INFO:teuthology.orchestra.run.smithi094.stdout: caps mds = "allow" 2024-09-10T23:39:30.522 INFO:teuthology.orchestra.run.smithi094.stdout: caps mon = "allow r" 2024-09-10T23:39:30.522 INFO:teuthology.orchestra.run.smithi094.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T23:39:30.522 INFO:teuthology.orchestra.run.smithi094.stderr:updated caps for client.0 2024-09-10T23:39:30.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.520+0000 7f8e88400640 1 -- 172.21.15.94:0/1576601396 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8e60072ca0 msgr2=0x7f8e60075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:30.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.520+0000 7f8e88400640 1 --2- 172.21.15.94:0/1576601396 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8e60072ca0 0x7f8e60075160 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8e740b3000 tx=0x7f8e7401dde0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.520+0000 7f8e88400640 1 -- 172.21.15.94:0/1576601396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8015f2d0 msgr2=0x7f8e8006ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:30.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.520+0000 7f8e88400640 1 --2- 172.21.15.94:0/1576601396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8015f2d0 0x7f8e8006ef60 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f8e7800c3c0 tx=0x7f8e7800c880 comp rx=0 tx=0).stop 2024-09-10T23:39:30.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.520+0000 7f8e88400640 1 -- 172.21.15.94:0/1576601396 shutdown_connections 2024-09-10T23:39:30.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.520+0000 7f8e88400640 1 --2- 172.21.15.94:0/1576601396 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8e60072ca0 0x7f8e60075160 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.520+0000 7f8e88400640 1 --2- 172.21.15.94:0/1576601396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e8015f2d0 0x7f8e8006ef60 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.520+0000 7f8e88400640 1 --2- 172.21.15.94:0/1576601396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e800cfc00 0x7f8e800c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.520+0000 7f8e88400640 1 --2- 172.21.15.94:0/1576601396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e800b2790 0x7f8e80143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.520+0000 7f8e88400640 1 -- 172.21.15.94:0/1576601396 >> 172.21.15.94:0/1576601396 conn(0x7f8e800d1c90 msgr2=0x7f8e800d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:30.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.520+0000 7f8e88400640 1 -- 172.21.15.94:0/1576601396 shutdown_connections 2024-09-10T23:39:30.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.520+0000 7f8e88400640 1 -- 172.21.15.94:0/1576601396 wait complete. 2024-09-10T23:39:30.544 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:39:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.684+0000 7f0c66000640 1 -- 172.21.15.94:0/3441072071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c60073ff0 msgr2=0x7f0c600cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.684+0000 7f0c66000640 1 --2- 172.21.15.94:0/3441072071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c60073ff0 0x7f0c600cfda0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009840 tx=0x7f0c5001c820 comp rx=0 tx=0).stop 2024-09-10T23:39:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.684+0000 7f0c66000640 1 -- 172.21.15.94:0/3441072071 shutdown_connections 2024-09-10T23:39:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.684+0000 7f0c66000640 1 --2- 172.21.15.94:0/3441072071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c6015ee80 0x7f0c6006f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.684+0000 7f0c66000640 1 --2- 172.21.15.94:0/3441072071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c60073ff0 0x7f0c600cfda0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.684+0000 7f0c66000640 1 -- 172.21.15.94:0/3441072071 >> 172.21.15.94:0/3441072071 conn(0x7f0c6006cdf0 msgr2=0x7f0c6006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.684+0000 7f0c66000640 1 -- 172.21.15.94:0/3441072071 shutdown_connections 2024-09-10T23:39:30.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.684+0000 7f0c66000640 1 -- 172.21.15.94:0/3441072071 wait complete. 2024-09-10T23:39:30.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.684+0000 7f0c66000640 1 Processor -- start 2024-09-10T23:39:30.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.684+0000 7f0c66000640 1 -- start start 2024-09-10T23:39:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.684+0000 7f0c66000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c60073ff0 0x7f0c6013bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.684+0000 7f0c66000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c600d02e0 0x7f0c60160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c66000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6015ee80 0x7f0c601510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c66000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c601679c0 con 0x7f0c6015ee80 2024-09-10T23:39:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c66000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0c60070560 con 0x7f0c600d02e0 2024-09-10T23:39:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c66000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0c60167b40 con 0x7f0c60073ff0 2024-09-10T23:39:30.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c5fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6015ee80 0x7f0c601510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c5fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6015ee80 0x7f0c601510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50364/0 (socket says 172.21.15.94:50364) 2024-09-10T23:39:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c5fe00640 1 -- 172.21.15.94:0/3516438519 learned_addr learned my addr 172.21.15.94:0/3516438519 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c5fe00640 1 -- 172.21.15.94:0/3516438519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c60073ff0 msgr2=0x7f0c6013bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c5fe00640 1 --2- 172.21.15.94:0/3516438519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c60073ff0 0x7f0c6013bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c5fe00640 1 -- 172.21.15.94:0/3516438519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c600d02e0 msgr2=0x7f0c60160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c5fe00640 1 --2- 172.21.15.94:0/3516438519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c600d02e0 0x7f0c60160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c5fe00640 1 -- 172.21.15.94:0/3516438519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6013c0c0 con 0x7f0c6015ee80 2024-09-10T23:39:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c5fe00640 1 --2- 172.21.15.94:0/3516438519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6015ee80 0x7f0c601510b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800e9a0 tx=0x7f0c5800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c57e00640 1 -- 172.21.15.94:0/3516438519 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58004d30 con 0x7f0c6015ee80 2024-09-10T23:39:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c57e00640 1 -- 172.21.15.94:0/3516438519 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c5800cb20 con 0x7f0c6015ee80 2024-09-10T23:39:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c57e00640 1 -- 172.21.15.94:0/3516438519 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58005020 con 0x7f0c6015ee80 2024-09-10T23:39:30.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c66000640 1 -- 172.21.15.94:0/3516438519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c6013f360 con 0x7f0c6015ee80 2024-09-10T23:39:30.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c66000640 1 -- 172.21.15.94:0/3516438519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c6013ea30 con 0x7f0c6015ee80 2024-09-10T23:39:30.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c66000640 1 -- 172.21.15.94:0/3516438519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c600cfda0 con 0x7f0c6015ee80 2024-09-10T23:39:30.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.688+0000 7f0c57e00640 1 -- 172.21.15.94:0/3516438519 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f0c5800f040 con 0x7f0c6015ee80 2024-09-10T23:39:30.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.692+0000 7f0c57e00640 1 --2- 172.21.15.94:0/3516438519 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0c3c072ca0 0x7f0c3c075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.692+0000 7f0c57e00640 1 -- 172.21.15.94:0/3516438519 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7f0c58094630 con 0x7f0c6015ee80 2024-09-10T23:39:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.692+0000 7f0c5f400640 1 --2- 172.21.15.94:0/3516438519 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0c3c072ca0 0x7f0c3c075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:30.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.692+0000 7f0c5f400640 1 --2- 172.21.15.94:0/3516438519 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0c3c072ca0 0x7f0c3c075160 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0c500af000 tx=0x7f0c500195c0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:30.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.696+0000 7f0c57e00640 1 -- 172.21.15.94:0/3516438519 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0c58016030 con 0x7f0c6015ee80 2024-09-10T23:39:30.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.892+0000 7f0c66000640 1 -- 172.21.15.94:0/3516438519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c6015da30 con 0x7f0c6015ee80 2024-09-10T23:39:30.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.892+0000 7f0c57e00640 1 -- 172.21.15.94:0/3516438519 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5231 (secure 0 0 0) 0x7f0c5805f210 con 0x7f0c6015ee80 2024-09-10T23:39:30.897 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:30.897 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":97,"btime":"2024-09-10T23:39:28:614931+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7864,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/188024700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":188024700},{"type":"v1","addr":"172.21.15.181:6835","nonce":188024700}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8376,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/35853213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":35853213},{"type":"v1","addr":"172.21.15.94:6834","nonce":35853213}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/800902310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":800902310},{"type":"v1","addr":"172.21.15.94:6835","nonce":800902310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:39:24.990806+0000","modified":"2024-09-10T23:39:27.995870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196874},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196874":{"gid":196874,"name":"b","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6801/2882789540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2882789540},{"type":"v1","addr":"172.21.15.181:6801","nonce":2882789540}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196874,"qdb_cluster":[196874]},"id":11}]} 2024-09-10T23:39:30.897 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 97 2024-09-10T23:39:30.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.896+0000 7f0c66000640 1 -- 172.21.15.94:0/3516438519 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0c3c072ca0 msgr2=0x7f0c3c075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:30.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.896+0000 7f0c66000640 1 --2- 172.21.15.94:0/3516438519 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0c3c072ca0 0x7f0c3c075160 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0c500af000 tx=0x7f0c500195c0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.896+0000 7f0c66000640 1 -- 172.21.15.94:0/3516438519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6015ee80 msgr2=0x7f0c601510b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:30.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.896+0000 7f0c66000640 1 --2- 172.21.15.94:0/3516438519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6015ee80 0x7f0c601510b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800e9a0 tx=0x7f0c5800ee60 comp rx=0 tx=0).stop 2024-09-10T23:39:30.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.896+0000 7f0c66000640 1 -- 172.21.15.94:0/3516438519 shutdown_connections 2024-09-10T23:39:30.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.896+0000 7f0c66000640 1 --2- 172.21.15.94:0/3516438519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c600d02e0 0x7f0c60160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.896+0000 7f0c66000640 1 --2- 172.21.15.94:0/3516438519 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0c3c072ca0 0x7f0c3c075160 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.896+0000 7f0c66000640 1 --2- 172.21.15.94:0/3516438519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c6015ee80 0x7f0c601510b0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.896+0000 7f0c66000640 1 --2- 172.21.15.94:0/3516438519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c60073ff0 0x7f0c6013bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:30.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.896+0000 7f0c66000640 1 -- 172.21.15.94:0/3516438519 >> 172.21.15.94:0/3516438519 conn(0x7f0c6006cdf0 msgr2=0x7f0c600b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:30.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.896+0000 7f0c66000640 1 -- 172.21.15.94:0/3516438519 shutdown_connections 2024-09-10T23:39:30.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:30.896+0000 7f0c66000640 1 -- 172.21.15.94:0/3516438519 wait complete. 2024-09-10T23:39:31.321 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 97, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T23:39:24.990806+0000', 'modified': '2024-09-10T23:39:27.995870+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 196874}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_196874': {'gid': 196874, 'name': 'b', 'rank': 0, 'incarnation': 94, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6801/2882789540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6800', 'nonce': 2882789540}, {'type': 'v1', 'addr': '172.21.15.181:6801', 'nonce': 2882789540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 196874, 'qdb_cluster': [196874]} 2024-09-10T23:39:31.322 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T23:39:31.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T23:39:31.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f9800640 1 -- 172.21.15.94:0/1400706141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d80051d0 msgr2=0x7fb0d800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:31.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f9800640 1 --2- 172.21.15.94:0/1400706141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d80051d0 0x7fb0d800b9d0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec001dc0 tx=0x7fb0ec01cb30 comp rx=0 tx=0).stop 2024-09-10T23:39:31.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f9800640 1 -- 172.21.15.94:0/1400706141 shutdown_connections 2024-09-10T23:39:31.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f9800640 1 --2- 172.21.15.94:0/1400706141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d80051d0 0x7fb0d800b9d0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f9800640 1 -- 172.21.15.94:0/1400706141 >> 172.21.15.94:0/1400706141 conn(0x7fb0f40cfaf0 msgr2=0x7fb0f40cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:31.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f9800640 1 -- 172.21.15.94:0/1400706141 shutdown_connections 2024-09-10T23:39:31.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f9800640 1 -- 172.21.15.94:0/1400706141 wait complete. 2024-09-10T23:39:31.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f9800640 1 Processor -- start 2024-09-10T23:39:31.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f9800640 1 -- start start 2024-09-10T23:39:31.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f9800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d80051d0 0x7fb0f413b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f9800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0f40b5780 0x7fb0f40b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f9800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0f40b4d60 0x7fb0f40c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f9800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f40c6460 con 0x7fb0d80051d0 2024-09-10T23:39:31.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f9800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f40c5850 con 0x7fb0f40b4d60 2024-09-10T23:39:31.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f9800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f40c65e0 con 0x7fb0f40b5780 2024-09-10T23:39:31.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d80051d0 0x7fb0f413b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d80051d0 0x7fb0f413b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50376/0 (socket says 172.21.15.94:50376) 2024-09-10T23:39:31.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f2a00640 1 -- 172.21.15.94:0/938954391 learned_addr learned my addr 172.21.15.94:0/938954391 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:31.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0ea000640 1 --2- 172.21.15.94:0/938954391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0f40b5780 0x7fb0f40b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f3400640 1 --2- 172.21.15.94:0/938954391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0f40b4d60 0x7fb0f40c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f2a00640 1 -- 172.21.15.94:0/938954391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0f40b5780 msgr2=0x7fb0f40b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:31.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f2a00640 1 --2- 172.21.15.94:0/938954391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0f40b5780 0x7fb0f40b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f2a00640 1 -- 172.21.15.94:0/938954391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0f40b4d60 msgr2=0x7fb0f40c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:31.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f2a00640 1 --2- 172.21.15.94:0/938954391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0f40b4d60 0x7fb0f40c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f2a00640 1 -- 172.21.15.94:0/938954391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f4073dc0 con 0x7fb0d80051d0 2024-09-10T23:39:31.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0ea000640 1 --2- 172.21.15.94:0/938954391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0f40b5780 0x7fb0f40b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:31.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f3400640 1 --2- 172.21.15.94:0/938954391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0f40b4d60 0x7fb0f40c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:31.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0f2a00640 1 --2- 172.21.15.94:0/938954391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d80051d0 0x7fb0f413b220 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4001a80 tx=0x7fb0e40039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:31.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.464+0000 7fb0eb400640 1 -- 172.21.15.94:0/938954391 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4015070 con 0x7fb0d80051d0 2024-09-10T23:39:31.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.468+0000 7fb0eb400640 1 -- 172.21.15.94:0/938954391 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0e4011940 con 0x7fb0d80051d0 2024-09-10T23:39:31.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.468+0000 7fb0eb400640 1 -- 172.21.15.94:0/938954391 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4011d90 con 0x7fb0d80051d0 2024-09-10T23:39:31.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.468+0000 7fb0f9800640 1 -- 172.21.15.94:0/938954391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f40c7020 con 0x7fb0d80051d0 2024-09-10T23:39:31.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.468+0000 7fb0f9800640 1 -- 172.21.15.94:0/938954391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f40c75e0 con 0x7fb0d80051d0 2024-09-10T23:39:31.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.468+0000 7fb0eb400640 1 -- 172.21.15.94:0/938954391 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7fb0e40103c0 con 0x7fb0d80051d0 2024-09-10T23:39:31.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0eb400640 1 --2- 172.21.15.94:0/938954391 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb0d0072cd0 0x7fb0d0075190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0eb400640 1 -- 172.21.15.94:0/938954391 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7fb0e4094f70 con 0x7fb0d80051d0 2024-09-10T23:39:31.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 Processor -- start 2024-09-10T23:39:31.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 -- start start 2024-09-10T23:39:31.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0ea000640 1 --2- 172.21.15.94:0/938954391 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb0d0072cd0 0x7fb0d0075190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0ea000640 1 --2- 172.21.15.94:0/938954391 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb0d0072cd0 0x7fb0d0075190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40c9bc0 tx=0x7fb0dc0091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:31.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa429e9d0 0x556aa429edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429f2f0 0x556aa42528d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa4252e10 0x556aa42531f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556aa41733f0 con 0x556aa429f2f0 2024-09-10T23:39:31.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556aa40b0b20 con 0x556aa429e9d0 2024-09-10T23:39:31.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x556aa409ff50 con 0x556aa4252e10 2024-09-10T23:39:31.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0ea000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429f2f0 0x556aa42528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0ea000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429f2f0 0x556aa42528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50384/0 (socket says 172.21.15.94:50384) 2024-09-10T23:39:31.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0ea000640 1 -- 172.21.15.94:0/2783255461 learned_addr learned my addr 172.21.15.94:0/2783255461 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0f3400640 1 --2- 172.21.15.94:0/2783255461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa4252e10 0x556aa42531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0f2a00640 1 --2- 172.21.15.94:0/2783255461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa429e9d0 0x556aa429edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0ea000640 1 -- 172.21.15.94:0/2783255461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa4252e10 msgr2=0x556aa42531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:39:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0ea000640 1 --2- 172.21.15.94:0/2783255461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa4252e10 0x556aa42531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0ea000640 1 -- 172.21.15.94:0/2783255461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa429e9d0 msgr2=0x556aa429edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:39:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0ea000640 1 --2- 172.21.15.94:0/2783255461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa429e9d0 0x556aa429edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0ea000640 1 -- 172.21.15.94:0/2783255461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aa4067590 con 0x556aa429f2f0 2024-09-10T23:39:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0ea000640 1 --2- 172.21.15.94:0/2783255461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429f2f0 0x556aa42528d0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc013010 tx=0x7fb0dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6777aedc29e5ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0e3400640 1 -- 172.21.15.94:0/2783255461 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc003b80 con 0x556aa429f2f0 2024-09-10T23:39:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0e3400640 1 -- 172.21.15.94:0/2783255461 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0dc003d20 con 0x556aa429f2f0 2024-09-10T23:39:31.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0e3400640 1 -- 172.21.15.94:0/2783255461 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc01e690 con 0x556aa429f2f0 2024-09-10T23:39:31.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2783255461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429f2f0 msgr2=0x556aa42528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:31.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/2783255461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429f2f0 0x556aa42528d0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc013010 tx=0x7fb0dc017000 comp rx=0 tx=0).stop 2024-09-10T23:39:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2783255461 shutdown_connections 2024-09-10T23:39:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/2783255461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa4252e10 0x556aa42531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/2783255461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429f2f0 0x556aa42528d0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/2783255461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa429e9d0 0x556aa429edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2783255461 >> 172.21.15.94:0/2783255461 conn(0x556aa413c130 msgr2=0x556aa4281e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2783255461 shutdown_connections 2024-09-10T23:39:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.472+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2783255461 wait complete. 2024-09-10T23:39:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 1 -- start start 2024-09-10T23:39:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 1 Processor -- start 2024-09-10T23:39:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa4252e10 0x556aa42a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa429e9d0 0x556aa42a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa429f2f0 0x556aa42a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556aa403a5d0 con 0x556aa4252e10 2024-09-10T23:39:31.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556aa40a3d60 con 0x556aa429f2f0 2024-09-10T23:39:31.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x556aa4046500 con 0x556aa429e9d0 2024-09-10T23:39:31.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 10 client.0 _ll_register_callbacks cb 0x556aa404a810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:39:31.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0f2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa4252e10 0x556aa42a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0f2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa4252e10 0x556aa42a5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50390/0 (socket says 172.21.15.94:50390) 2024-09-10T23:39:31.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0f2a00640 1 -- 172.21.15.94:0/2347574946 learned_addr learned my addr 172.21.15.94:0/2347574946 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:31.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0f2a00640 1 -- 172.21.15.94:0/2347574946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa429e9d0 msgr2=0x556aa42a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:39:31.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0ea000640 1 --2- 172.21.15.94:0/2347574946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa429e9d0 0x556aa42a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0f3400640 1 --2- 172.21.15.94:0/2347574946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa429f2f0 0x556aa42a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0f2a00640 1 --2- 172.21.15.94:0/2347574946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa429e9d0 0x556aa42a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0f2a00640 1 -- 172.21.15.94:0/2347574946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa429f2f0 msgr2=0x556aa42a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:39:31.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0f2a00640 1 --2- 172.21.15.94:0/2347574946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa429f2f0 0x556aa42a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0f2a00640 1 -- 172.21.15.94:0/2347574946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aa403e170 con 0x556aa4252e10 2024-09-10T23:39:31.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0ea000640 1 --2- 172.21.15.94:0/2347574946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa429e9d0 0x556aa42a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:31.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0f3400640 1 --2- 172.21.15.94:0/2347574946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa429f2f0 0x556aa42a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:31.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0f2a00640 1 --2- 172.21.15.94:0/2347574946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa4252e10 0x556aa42a5d60 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40068f0 tx=0x7fb0e405f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b0c65dcea16f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:31.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:39:31.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0d7e00640 1 -- 172.21.15.94:0/2347574946 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e401f040 con 0x556aa4252e10 2024-09-10T23:39:31.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0d7e00640 1 -- 172.21.15.94:0/2347574946 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7fb0e4066180 con 0x556aa4252e10 2024-09-10T23:39:31.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2347574946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556aa40380f0 con 0x556aa4252e10 2024-09-10T23:39:31.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0d7e00640 1 -- 172.21.15.94:0/2347574946 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7fb0e405fb90 con 0x556aa4252e10 2024-09-10T23:39:31.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 10 client.8526 fetch_fsmap learned FSMap version 97 2024-09-10T23:39:31.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2347574946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) -- 0x556aa4124ef0 con 0x556aa4252e10 2024-09-10T23:39:31.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0d7e00640 1 -- 172.21.15.94:0/2347574946 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x556aa40380f0 con 0x556aa4252e10 2024-09-10T23:39:31.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0d7e00640 1 -- 172.21.15.94:0/2347574946 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 97) ==== 3502+0+0 (secure 0 0 0) 0x7fb0e408bb80 con 0x556aa4252e10 2024-09-10T23:39:31.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 10 client.8526 fetch_fsmap finished waiting for FSMap version 97 2024-09-10T23:39:31.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 10 client.8526 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T23:39:31.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 20 client.8526 populate_metadata read hostname 'smithi094' 2024-09-10T23:39:31.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/2347574946 >> [v2:172.21.15.181:6800/2882789540,v1:172.21.15.181:6801/2882789540] conn(0x556aa42ae130 0x556aa42b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 4 client.8526 mds_command: new command op to 196874 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:39:31.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2347574946 --> [v2:172.21.15.181:6800/2882789540,v1:172.21.15.181:6801/2882789540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aa40b2a70 con 0x556aa42ae130 2024-09-10T23:39:31.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.476+0000 7fb0ea000640 1 --2- 172.21.15.94:0/2347574946 >> [v2:172.21.15.181:6800/2882789540,v1:172.21.15.181:6801/2882789540] conn(0x556aa42ae130 0x556aa42b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0ea000640 1 --2- 172.21.15.94:0/2347574946 >> [v2:172.21.15.181:6800/2882789540,v1:172.21.15.181:6801/2882789540] conn(0x556aa42ae130 0x556aa42b0520 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c65b54b412e764b server_cookie=7ee8438d89b6b9e in_seq=0 out_seq=0 2024-09-10T23:39:31.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0d7e00640 10 client.8526 ms_handle_connect on v2:172.21.15.181:6800/2882789540 2024-09-10T23:39:31.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0d7e00640 1 -- 172.21.15.94:0/2347574946 <== mds.0 v2:172.21.15.181:6800/2882789540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556aa40b2a70 con 0x556aa42ae130 2024-09-10T23:39:31.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0d7e00640 10 client.8526 handle_command_reply: tid=0 2024-09-10T23:39:31.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0fcbbe000 1 client.8526 shutdown 2024-09-10T23:39:31.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2347574946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa4252e10 msgr2=0x556aa42a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:31.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/2347574946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa4252e10 0x556aa42a5d60 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40068f0 tx=0x7fb0e405f9e0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2347574946 shutdown_connections 2024-09-10T23:39:31.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/2347574946 >> [v2:172.21.15.181:6800/2882789540,v1:172.21.15.181:6801/2882789540] conn(0x556aa42ae130 0x556aa42b0520 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/2347574946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa429f2f0 0x556aa42a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/2347574946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa429e9d0 0x556aa42a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/2347574946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa4252e10 0x556aa42a5d60 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0d7e00640 0 client.8526 ms_handle_reset on v2:172.21.15.181:6800/2882789540 2024-09-10T23:39:31.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2347574946 >> 172.21.15.94:0/2347574946 conn(0x556aa413c130 msgr2=0x556aa4251fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:31.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2347574946 shutdown_connections 2024-09-10T23:39:31.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2347574946 wait complete. 2024-09-10T23:39:31.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.480+0000 7fb0fcbbe000 20 client.8526 trim_cache size 0 max 16384 2024-09-10T23:39:31.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 Processor -- start 2024-09-10T23:39:31.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 -- start start 2024-09-10T23:39:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa413c130 0x556aa4310e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa4252e10 0x556aa4311370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429e9d0 0x556aa43118b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f42016f0 con 0x556aa429e9d0 2024-09-10T23:39:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f4200790 con 0x556aa4252e10 2024-09-10T23:39:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f40731e0 con 0x556aa413c130 2024-09-10T23:39:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0f3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429e9d0 0x556aa43118b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0f3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429e9d0 0x556aa43118b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50392/0 (socket says 172.21.15.94:50392) 2024-09-10T23:39:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0f3400640 1 -- 172.21.15.94:0/3064236776 learned_addr learned my addr 172.21.15.94:0/3064236776 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0ea000640 1 --2- 172.21.15.94:0/3064236776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa4252e10 0x556aa4311370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0f2a00640 1 --2- 172.21.15.94:0/3064236776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa413c130 0x556aa4310e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0f3400640 1 -- 172.21.15.94:0/3064236776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa413c130 msgr2=0x556aa4310e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:39:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0f3400640 1 --2- 172.21.15.94:0/3064236776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa413c130 0x556aa4310e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0f3400640 1 -- 172.21.15.94:0/3064236776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa4252e10 msgr2=0x556aa4311370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:39:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0f3400640 1 --2- 172.21.15.94:0/3064236776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa4252e10 0x556aa4311370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0f3400640 1 -- 172.21.15.94:0/3064236776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aa40fbd90 con 0x556aa429e9d0 2024-09-10T23:39:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0f3400640 1 --2- 172.21.15.94:0/3064236776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429e9d0 0x556aa43118b0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec01b230 tx=0x7fb0d8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1f4ca051440d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0e2a00640 1 -- 172.21.15.94:0/3064236776 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec0a59a0 con 0x556aa429e9d0 2024-09-10T23:39:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0e2a00640 1 -- 172.21.15.94:0/3064236776 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0ec0a5b40 con 0x556aa429e9d0 2024-09-10T23:39:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0e2a00640 1 -- 172.21.15.94:0/3064236776 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec003980 con 0x556aa429e9d0 2024-09-10T23:39:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/3064236776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429e9d0 msgr2=0x556aa43118b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/3064236776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429e9d0 0x556aa43118b0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec01b230 tx=0x7fb0d8005840 comp rx=0 tx=0).stop 2024-09-10T23:39:31.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/3064236776 shutdown_connections 2024-09-10T23:39:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/3064236776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa4252e10 0x556aa4311370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/3064236776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429e9d0 0x556aa43118b0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/3064236776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa413c130 0x556aa4310e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/3064236776 >> 172.21.15.94:0/3064236776 conn(0x556aa429e2f0 msgr2=0x556aa42510a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/3064236776 shutdown_connections 2024-09-10T23:39:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/3064236776 wait complete. 2024-09-10T23:39:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 -- start start 2024-09-10T23:39:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.500+0000 7fb0fcbbe000 1 Processor -- start 2024-09-10T23:39:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa413c130 0x556aa42b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa4252e10 0x556aa42b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429e9d0 0x556aa42b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556aa406bec0 con 0x556aa429e9d0 2024-09-10T23:39:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f4073830 con 0x556aa4252e10 2024-09-10T23:39:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x556aa40a3070 con 0x556aa413c130 2024-09-10T23:39:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 10 client.0 _ll_register_callbacks cb 0x7fb0e405ede0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:39:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0f3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429e9d0 0x556aa42b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0f3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429e9d0 0x556aa42b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50398/0 (socket says 172.21.15.94:50398) 2024-09-10T23:39:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0f3400640 1 -- 172.21.15.94:0/2457867833 learned_addr learned my addr 172.21.15.94:0/2457867833 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0ea000640 1 --2- 172.21.15.94:0/2457867833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa4252e10 0x556aa42b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0f2a00640 1 --2- 172.21.15.94:0/2457867833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa413c130 0x556aa42b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0f3400640 1 -- 172.21.15.94:0/2457867833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa413c130 msgr2=0x556aa42b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:39:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0f3400640 1 --2- 172.21.15.94:0/2457867833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa413c130 0x556aa42b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0f3400640 1 -- 172.21.15.94:0/2457867833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa4252e10 msgr2=0x556aa42b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:39:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0f3400640 1 --2- 172.21.15.94:0/2457867833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa4252e10 0x556aa42b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0f3400640 1 -- 172.21.15.94:0/2457867833 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aa4066a90 con 0x556aa429e9d0 2024-09-10T23:39:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0f3400640 1 --2- 172.21.15.94:0/2457867833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429e9d0 0x556aa42b4c60 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec01b230 tx=0x7fb0ec0a5a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a46104b5299700e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:39:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0d7e00640 1 -- 172.21.15.94:0/2457867833 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec0c4060 con 0x556aa429e9d0 2024-09-10T23:39:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0d7e00640 1 -- 172.21.15.94:0/2457867833 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7fb0ec003a80 con 0x556aa429e9d0 2024-09-10T23:39:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2457867833 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0ec0a5b40 con 0x556aa429e9d0 2024-09-10T23:39:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0d7e00640 1 -- 172.21.15.94:0/2457867833 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7fb0ec0ca050 con 0x556aa429e9d0 2024-09-10T23:39:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 10 client.8532 fetch_fsmap learned FSMap version 97 2024-09-10T23:39:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2457867833 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) -- 0x556aa4045050 con 0x556aa429e9d0 2024-09-10T23:39:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0d7e00640 1 -- 172.21.15.94:0/2457867833 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0ec0a5b40 con 0x556aa429e9d0 2024-09-10T23:39:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0d7e00640 1 -- 172.21.15.94:0/2457867833 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 97) ==== 3502+0+0 (secure 0 0 0) 0x7fb0ec0cd020 con 0x556aa429e9d0 2024-09-10T23:39:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 10 client.8532 fetch_fsmap finished waiting for FSMap version 97 2024-09-10T23:39:31.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 10 client.8532 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T23:39:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 20 client.8532 populate_metadata read hostname 'smithi094' 2024-09-10T23:39:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/2457867833 >> [v2:172.21.15.181:6800/2882789540,v1:172.21.15.181:6801/2882789540] conn(0x556aa429f2f0 0x556aa42b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 4 client.8532 mds_command: new command op to 196874 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T23:39:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2457867833 --> [v2:172.21.15.181:6800/2882789540,v1:172.21.15.181:6801/2882789540] -- command(tid 0: {"prefix": "status"}) -- 0x556aa40ae520 con 0x556aa429f2f0 2024-09-10T23:39:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0f2a00640 1 --2- 172.21.15.94:0/2457867833 >> [v2:172.21.15.181:6800/2882789540,v1:172.21.15.181:6801/2882789540] conn(0x556aa429f2f0 0x556aa42b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0f2a00640 1 --2- 172.21.15.94:0/2457867833 >> [v2:172.21.15.181:6800/2882789540,v1:172.21.15.181:6801/2882789540] conn(0x556aa429f2f0 0x556aa42b51a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d40b9c92bc7de57d server_cookie=a4c1a52db782bdd7 in_seq=0 out_seq=0 2024-09-10T23:39:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0d7e00640 10 client.8532 ms_handle_connect on v2:172.21.15.181:6800/2882789540 2024-09-10T23:39:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0d7e00640 1 -- 172.21.15.94:0/2457867833 <== mds.0 v2:172.21.15.181:6800/2882789540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x556aa40ae520 con 0x556aa429f2f0 2024-09-10T23:39:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0d7e00640 10 client.8532 handle_command_reply: tid=0 2024-09-10T23:39:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 1 client.8532 shutdown 2024-09-10T23:39:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2457867833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429e9d0 msgr2=0x556aa42b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.504+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/2457867833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429e9d0 0x556aa42b4c60 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec01b230 tx=0x7fb0ec0a5a20 comp rx=0 tx=0).stop 2024-09-10T23:39:31.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2457867833 shutdown_connections 2024-09-10T23:39:31.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/2457867833 >> [v2:172.21.15.181:6800/2882789540,v1:172.21.15.181:6801/2882789540] conn(0x556aa429f2f0 0x556aa42b51a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/2457867833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556aa4252e10 0x556aa42b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/2457867833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556aa429e9d0 0x556aa42b4c60 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0d7e00640 0 client.8532 ms_handle_reset on v2:172.21.15.181:6800/2882789540 2024-09-10T23:39:31.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0fcbbe000 1 --2- 172.21.15.94:0/2457867833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556aa413c130 0x556aa42b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2457867833 >> 172.21.15.94:0/2457867833 conn(0x556aa429e2f0 msgr2=0x556aa42518c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:31.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2457867833 shutdown_connections 2024-09-10T23:39:31.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0fcbbe000 1 -- 172.21.15.94:0/2457867833 wait complete. 2024-09-10T23:39:31.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0fcbbe000 20 client.8532 trim_cache size 0 max 16384 2024-09-10T23:39:31.514 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:39:31.514 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "1798cdb9-9937-4918-a569-86b41a92817e", 2024-09-10T23:39:31.514 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-09-10T23:39:31.514 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 196874, 2024-09-10T23:39:31.514 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-10T23:39:31.514 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-10T23:39:31.514 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-10T23:39:31.514 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 5.9007842149999998, 2024-09-10T23:39:31.514 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 97, 2024-09-10T23:39:31.514 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 138, 2024-09-10T23:39:31.514 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 138, 2024-09-10T23:39:31.514 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 10.395708833 2024-09-10T23:39:31.514 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:39:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0f9800640 1 -- 172.21.15.94:0/938954391 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb0d0072cd0 msgr2=0x7fb0d0075190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0f9800640 1 --2- 172.21.15.94:0/938954391 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb0d0072cd0 0x7fb0d0075190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40c9bc0 tx=0x7fb0dc0091e0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0f9800640 1 -- 172.21.15.94:0/938954391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d80051d0 msgr2=0x7fb0f413b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0f9800640 1 --2- 172.21.15.94:0/938954391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d80051d0 0x7fb0f413b220 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4001a80 tx=0x7fb0e40039d0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0f9800640 1 -- 172.21.15.94:0/938954391 shutdown_connections 2024-09-10T23:39:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0f9800640 1 --2- 172.21.15.94:0/938954391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0f40b4d60 0x7fb0f40c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0f9800640 1 --2- 172.21.15.94:0/938954391 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb0d0072cd0 0x7fb0d0075190 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0f9800640 1 --2- 172.21.15.94:0/938954391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0f40b5780 0x7fb0f40b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0f9800640 1 --2- 172.21.15.94:0/938954391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d80051d0 0x7fb0f413b220 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0f9800640 1 -- 172.21.15.94:0/938954391 >> 172.21.15.94:0/938954391 conn(0x7fb0f40cfaf0 msgr2=0x7fb0f41404e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0f9800640 1 -- 172.21.15.94:0/938954391 shutdown_connections 2024-09-10T23:39:31.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.508+0000 7fb0f9800640 1 -- 172.21.15.94:0/938954391 wait complete. 2024-09-10T23:39:31.529 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:39:31.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.672+0000 7fc339000640 1 -- 172.21.15.94:0/2013543304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3340d05e0 msgr2=0x7fc3340b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:31.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.672+0000 7fc339000640 1 --2- 172.21.15.94:0/2013543304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3340d05e0 0x7fc3340b15c0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fc334073820 tx=0x7fc32c01c4e0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.672+0000 7fc339000640 1 -- 172.21.15.94:0/2013543304 shutdown_connections 2024-09-10T23:39:31.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.672+0000 7fc339000640 1 --2- 172.21.15.94:0/2013543304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3340d05e0 0x7fc3340b15c0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.672+0000 7fc339000640 1 --2- 172.21.15.94:0/2013543304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3340b2bb0 0x7fc3340d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.672+0000 7fc339000640 1 --2- 172.21.15.94:0/2013543304 >> v2:172.21.15.181:3300/0 conn(0x7fc33415f5c0 0x7fc3340cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.672+0000 7fc339000640 1 -- 172.21.15.94:0/2013543304 >> 172.21.15.94:0/2013543304 conn(0x7fc3340d1c60 msgr2=0x7fc3340d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:31.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.672+0000 7fc339000640 1 -- 172.21.15.94:0/2013543304 shutdown_connections 2024-09-10T23:39:31.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.672+0000 7fc339000640 1 -- 172.21.15.94:0/2013543304 wait complete. 2024-09-10T23:39:31.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.672+0000 7fc339000640 1 Processor -- start 2024-09-10T23:39:31.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.672+0000 7fc339000640 1 -- start start 2024-09-10T23:39:31.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc339000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3340b2bb0 0x7fc3340c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc339000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3340d05e0 0x7fc3340c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc339000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc33415f5c0 0x7fc3341fb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc339000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc3340afbd0 con 0x7fc3340d05e0 2024-09-10T23:39:31.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc339000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc3340afa50 con 0x7fc3340b2bb0 2024-09-10T23:39:31.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc339000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc3340afd50 con 0x7fc33415f5c0 2024-09-10T23:39:31.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc329400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3340d05e0 0x7fc3340c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc329400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3340d05e0 0x7fc3340c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50410/0 (socket says 172.21.15.94:50410) 2024-09-10T23:39:31.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc329400640 1 -- 172.21.15.94:0/942616060 learned_addr learned my addr 172.21.15.94:0/942616060 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc331e00640 1 --2- 172.21.15.94:0/942616060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3340b2bb0 0x7fc3340c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc332800640 1 --2- 172.21.15.94:0/942616060 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc33415f5c0 0x7fc3341fb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc329400640 1 -- 172.21.15.94:0/942616060 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc33415f5c0 msgr2=0x7fc3341fb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc329400640 1 --2- 172.21.15.94:0/942616060 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc33415f5c0 0x7fc3341fb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc329400640 1 -- 172.21.15.94:0/942616060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3340b2bb0 msgr2=0x7fc3340c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc329400640 1 --2- 172.21.15.94:0/942616060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3340b2bb0 0x7fc3340c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc329400640 1 -- 172.21.15.94:0/942616060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33406e240 con 0x7fc3340d05e0 2024-09-10T23:39:31.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc329400640 1 --2- 172.21.15.94:0/942616060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3340d05e0 0x7fc3340c7ce0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00cce0 tx=0x7fc31c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:31.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc32aa00640 1 -- 172.21.15.94:0/942616060 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c003980 con 0x7fc3340d05e0 2024-09-10T23:39:31.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc32aa00640 1 -- 172.21.15.94:0/942616060 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc31c003c10 con 0x7fc3340d05e0 2024-09-10T23:39:31.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc339000640 1 -- 172.21.15.94:0/942616060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33406e4e0 con 0x7fc3340d05e0 2024-09-10T23:39:31.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc32aa00640 1 -- 172.21.15.94:0/942616060 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c004030 con 0x7fc3340d05e0 2024-09-10T23:39:31.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc339000640 1 -- 172.21.15.94:0/942616060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3341f8060 con 0x7fc3340d05e0 2024-09-10T23:39:31.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc339000640 1 -- 172.21.15.94:0/942616060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3340b02e0 con 0x7fc3340d05e0 2024-09-10T23:39:31.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.676+0000 7fc32aa00640 1 -- 172.21.15.94:0/942616060 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7fc31c0041d0 con 0x7fc3340d05e0 2024-09-10T23:39:31.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.680+0000 7fc32aa00640 1 --2- 172.21.15.94:0/942616060 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc310072ca0 0x7fc310075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:31.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.680+0000 7fc32aa00640 1 -- 172.21.15.94:0/942616060 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7fc31c0948b0 con 0x7fc3340d05e0 2024-09-10T23:39:31.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.680+0000 7fc331e00640 1 --2- 172.21.15.94:0/942616060 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc310072ca0 0x7fc310075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:31.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.680+0000 7fc331e00640 1 --2- 172.21.15.94:0/942616060 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc310072ca0 0x7fc310075160 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc32400ab40 tx=0x7fc32400a250 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:31.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.684+0000 7fc32aa00640 1 -- 172.21.15.94:0/942616060 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc31c014070 con 0x7fc3340d05e0 2024-09-10T23:39:31.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.884+0000 7fc339000640 1 -- 172.21.15.94:0/942616060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc3340749c0 con 0x7fc3340d05e0 2024-09-10T23:39:31.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.884+0000 7fc32aa00640 1 -- 172.21.15.94:0/942616060 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v97) ==== 53+0+73 (secure 0 0 0) 0x7fc31c05ed20 con 0x7fc3340d05e0 2024-09-10T23:39:31.886 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:39:31.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.884+0000 7fc339000640 1 -- 172.21.15.94:0/942616060 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc310072ca0 msgr2=0x7fc310075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:31.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.884+0000 7fc339000640 1 --2- 172.21.15.94:0/942616060 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc310072ca0 0x7fc310075160 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc32400ab40 tx=0x7fc32400a250 comp rx=0 tx=0).stop 2024-09-10T23:39:31.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.884+0000 7fc339000640 1 -- 172.21.15.94:0/942616060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3340d05e0 msgr2=0x7fc3340c7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:31.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.884+0000 7fc339000640 1 --2- 172.21.15.94:0/942616060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3340d05e0 0x7fc3340c7ce0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00cce0 tx=0x7fc31c00a620 comp rx=0 tx=0).stop 2024-09-10T23:39:31.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.884+0000 7fc339000640 1 -- 172.21.15.94:0/942616060 shutdown_connections 2024-09-10T23:39:31.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.884+0000 7fc339000640 1 --2- 172.21.15.94:0/942616060 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc310072ca0 0x7fc310075160 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.884+0000 7fc339000640 1 --2- 172.21.15.94:0/942616060 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc33415f5c0 0x7fc3341fb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.884+0000 7fc339000640 1 --2- 172.21.15.94:0/942616060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3340d05e0 0x7fc3340c7ce0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.884+0000 7fc339000640 1 --2- 172.21.15.94:0/942616060 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3340b2bb0 0x7fc3340c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:31.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.884+0000 7fc339000640 1 -- 172.21.15.94:0/942616060 >> 172.21.15.94:0/942616060 conn(0x7fc3340d1c60 msgr2=0x7fc334155410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:31.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.884+0000 7fc339000640 1 -- 172.21.15.94:0/942616060 shutdown_connections 2024-09-10T23:39:31.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:31.884+0000 7fc339000640 1 -- 172.21.15.94:0/942616060 wait complete. 2024-09-10T23:39:31.908 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:39:31.908 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:39:31.908 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T23:39:31.909 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T23:39:31.909 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T23:39:31.909 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:39:31.909 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:39:31.909 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:39:31.909 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:39:31.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:31.909 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T23:39:31.918 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:39:31.919 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:39:31.919 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:39:31.919 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:39:31.919 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T23:39:31.919 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:39:31.919 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:39:31.919 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:39:31.919 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T23:39:31.919 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:39:31.919 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:39:31.919 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T23:39:31.919 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:39:31.919 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:39:31.920 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:39:31.920 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:39:31.920 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:39:31.920 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42039sec preferred_lft 42039sec 2024-09-10T23:39:31.920 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-09-10T23:39:31.920 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:39:31.920 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:39:31.920 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:ed:7d:46:db brd ff:ff:ff:ff:ff:ff 2024-09-10T23:39:31.920 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:39:31.920 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:39:31.920 INFO:teuthology.orchestra.run.smithi094.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:39:31.920 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:39:31.920 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:39:31.921 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:39:31.921 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:39:31.921 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:39:31.921 INFO:teuthology.orchestra.run.smithi094.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:39:31.921 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:39:31.921 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:39:31.921 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:39:31.921 INFO:teuthology.orchestra.run.smithi094.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:39:31.921 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:39:31.921 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:39:31.921 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:39:31.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:31.922 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:39:31.978 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T23:39:31.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:31.980 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T23:39:32.032 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T23:39:32.032 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:39:32.032 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:39:32.032 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:39:32.032 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:39:32.033 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:39:32.033 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:39:32.033 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:39:32.033 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:39:32.033 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:39:32.033 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:39:32.033 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:39:32.033 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:39:32.033 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:39:32.033 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:39:32.034 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:39:32.034 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:39:32.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:39:32.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:39:32.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:32.170 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T23:39:32.220 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T23:39:32.221 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T23:39:32.222 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:39:32.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:32.222 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-09-10T23:39:32.284 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T23:39:32.284 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:39:32.284 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T23:39:32.333 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:39:32.333 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:39:32.333 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:39:32.392 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:39:32.452 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:39:32.502 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:39:32.502 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:39:32.502 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:39:32.502 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:39:32.502 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:39:32.502 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:39:32.502 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:39:32.502 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:39:32.502 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:39:32.502 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:39:32.502 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:39:32.503 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:39:32.503 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:39:32.503 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:39:32.503 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:39:32.503 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:39:32.503 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:39:32.503 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:39:32.503 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:39:32.503 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:39:32.503 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:39:32.503 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:39:32.503 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:39:32.503 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:39:32.503 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:39:32.504 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:39:32.504 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:39:32.504 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:39:32.504 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:39:32.504 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:39:32.504 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:39:32.504 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:39:32.504 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:39:32.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:32.505 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:39:32.556 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:39:32.556 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T23:39:32.599 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:39:32.621 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:39:32.678 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:39:32.678 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:39:32.678 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:39:32.678 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:39:32.678 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:39:32.678 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:39:32.679 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:39:32.679 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:39:32.679 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:39:32.679 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:39:32.679 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:39:32.679 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:39:32.679 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:39:32.679 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:39:32.679 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:39:32.679 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:39:32.679 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:39:32.679 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:39:32.679 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:39:32.680 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:39:32.680 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:39:32.680 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:39:32.680 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:39:32.680 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:39:32.680 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:39:32.680 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:39:32.680 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:39:32.680 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:39:32.680 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:39:32.680 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:39:32.680 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:39:32.680 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:39:32.681 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:39:32.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:32.682 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:39:32.695 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:2024-09-10T23:39:32.692+0000 7f79e73d2580 -1 init, newargv = 0x5566e5075f00 newargc=15 2024-09-10T23:39:32.696 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[50535]: starting ceph client 2024-09-10T23:39:32.703 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[50535]: starting fuse 2024-09-10T23:39:32.704 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T23:39:32.704 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:39:32.705 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:39:32.705 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-09-10T23:39:32.705 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-09-10T23:39:32.705 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-09-10T23:39:32.705 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-09-10T23:39:32.705 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:39:32.705 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-09-10T23:39:32.705 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T23:39:32.705 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-09-10T23:39:32.705 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T23:39:32.705 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:39:32.705 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-09-10T23:39:32.705 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-09-10T23:39:32.705 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-09-10T23:39:32.705 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-09-10T23:39:32.706 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:39:32.706 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-09-10T23:39:32.706 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:39:32.706 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:39:32.706 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:39:32.706 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-09-10T23:39:32.706 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-09-10T23:39:32.706 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-09-10T23:39:32.706 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:39:32.706 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:39:32.706 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.0")) 2024-09-10T23:39:32.706 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-09-10T23:39:32.825 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.0.50535.asok 2024-09-10T23:39:32.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:39:32.837 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50535.asok 2024-09-10T23:39:32.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:32.837 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50535.asok status 2024-09-10T23:39:33.007 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:39:33.008 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-09-10T23:39:33.008 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:39:33.008 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:39:33.008 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "0", 2024-09-10T23:39:33.008 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-09-10T23:39:33.008 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T23:39:33.008 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "50535", 2024-09-10T23:39:33.008 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-09-10T23:39:33.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:33.008 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-09-10T23:39:33.008 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-09-10T23:39:33.009 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 8544, 2024-09-10T23:39:33.009 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-09-10T23:39:33.009 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-09-10T23:39:33.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-09-10T23:39:33.009 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 8544 2024-09-10T23:39:33.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:33.009 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:39:33.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:39:33.009 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:39:33.009 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2314287887 2024-09-10T23:39:33.009 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:33.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:33.009 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:39:33.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:39:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:39:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2314287887 2024-09-10T23:39:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.8544 192.168.0.1:0/2314287887", 2024-09-10T23:39:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/2314287887", 2024-09-10T23:39:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-09-10T23:39:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 97, 2024-09-10T23:39:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 138, 2024-09-10T23:39:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:39:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-09-10T23:39:33.010 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-09-10T23:39:33.010 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:39:33.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:33.023 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-09-10T23:39:33.023 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T23:39:33.032 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-09-10T23:39:33.032 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T23:39:33.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:33.033 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:39:33.092 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:39:33.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.272+0000 7f2a2b800640 1 -- 172.21.15.94:0/1742293473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a08004d50 msgr2=0x7f2a080051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:33.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.272+0000 7f2a2b800640 1 --2- 172.21.15.94:0/1742293473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a08004d50 0x7f2a080051d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009ee0 tx=0x7f2a1c01d5f0 comp rx=0 tx=0).stop 2024-09-10T23:39:33.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.272+0000 7f2a29800640 1 -- 172.21.15.94:0/1742293473 reap_dead start 2024-09-10T23:39:33.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.272+0000 7f2a2b800640 1 -- 172.21.15.94:0/1742293473 shutdown_connections 2024-09-10T23:39:33.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.272+0000 7f2a2b800640 1 -- 172.21.15.94:0/1742293473 >> 172.21.15.94:0/1742293473 conn(0x7f2a2406cdf0 msgr2=0x7f2a2406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:33.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.272+0000 7f2a2b800640 1 -- 172.21.15.94:0/1742293473 shutdown_connections 2024-09-10T23:39:33.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.272+0000 7f2a2b800640 1 -- 172.21.15.94:0/1742293473 wait complete. 2024-09-10T23:39:33.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.272+0000 7f2a2b800640 1 Processor -- start 2024-09-10T23:39:33.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.272+0000 7f2a2b800640 1 -- start start 2024-09-10T23:39:33.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a2b800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a08004d50 0x7f2a24146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:33.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a2b800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a24147710 0x7f2a240ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:33.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a2b800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a240c5250 0x7f2a240c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:33.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a2b800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2406f630 con 0x7f2a08004d50 2024-09-10T23:39:33.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a2b800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2406f4b0 con 0x7f2a240c5250 2024-09-10T23:39:33.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a2b800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2a2406f7b0 con 0x7f2a24147710 2024-09-10T23:39:33.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a28e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a08004d50 0x7f2a24146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a28e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a08004d50 0x7f2a24146e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50434/0 (socket says 172.21.15.94:50434) 2024-09-10T23:39:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a28e00640 1 -- 172.21.15.94:0/2791391560 learned_addr learned my addr 172.21.15.94:0/2791391560 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a1be00640 1 --2- 172.21.15.94:0/2791391560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a24147710 0x7f2a240ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a29800640 1 --2- 172.21.15.94:0/2791391560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a240c5250 0x7f2a240c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a28e00640 1 -- 172.21.15.94:0/2791391560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a24147710 msgr2=0x7f2a240ac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a28e00640 1 --2- 172.21.15.94:0/2791391560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a24147710 0x7f2a240ac0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a28e00640 1 -- 172.21.15.94:0/2791391560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a240c5250 msgr2=0x7f2a240c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a28e00640 1 --2- 172.21.15.94:0/2791391560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a240c5250 0x7f2a240c56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a28e00640 1 -- 172.21.15.94:0/2791391560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a240aca20 con 0x7f2a08004d50 2024-09-10T23:39:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a29800640 1 --2- 172.21.15.94:0/2791391560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a240c5250 0x7f2a240c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a28e00640 1 --2- 172.21.15.94:0/2791391560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a08004d50 0x7f2a24146e90 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400d730 tx=0x7f2a1400db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:33.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a21600640 1 -- 172.21.15.94:0/2791391560 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14017070 con 0x7f2a08004d50 2024-09-10T23:39:33.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a21600640 1 -- 172.21.15.94:0/2791391560 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a14004310 con 0x7f2a08004d50 2024-09-10T23:39:33.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a21600640 1 -- 172.21.15.94:0/2791391560 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14009a20 con 0x7f2a08004d50 2024-09-10T23:39:33.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a2b800640 1 -- 172.21.15.94:0/2791391560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a240acd00 con 0x7f2a08004d50 2024-09-10T23:39:33.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a2b800640 1 -- 172.21.15.94:0/2791391560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a240ad790 con 0x7f2a08004d50 2024-09-10T23:39:33.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.276+0000 7f2a2b800640 1 -- 172.21.15.94:0/2791391560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a240cfda0 con 0x7f2a08004d50 2024-09-10T23:39:33.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.280+0000 7f2a21600640 1 -- 172.21.15.94:0/2791391560 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f2a14009bc0 con 0x7f2a08004d50 2024-09-10T23:39:33.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.280+0000 7f2a21600640 1 --2- 172.21.15.94:0/2791391560 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2a00072ca0 0x7f2a00075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:33.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.280+0000 7f2a21600640 1 -- 172.21.15.94:0/2791391560 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7f2a14093fc0 con 0x7f2a08004d50 2024-09-10T23:39:33.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.280+0000 7f2a1be00640 1 --2- 172.21.15.94:0/2791391560 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2a00072ca0 0x7f2a00075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:33.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.280+0000 7f2a1be00640 1 --2- 172.21.15.94:0/2791391560 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2a00072ca0 0x7f2a00075160 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c001870 tx=0x7f2a0c006040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:33.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.284+0000 7f2a21600640 1 -- 172.21.15.94:0/2791391560 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2a1405eba0 con 0x7f2a08004d50 2024-09-10T23:39:33.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.484+0000 7f2a2b800640 1 -- 172.21.15.94:0/2791391560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2a241473d0 con 0x7f2a08004d50 2024-09-10T23:39:33.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.484+0000 7f2a21600640 1 -- 172.21.15.94:0/2791391560 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v97) ==== 53+0+73 (secure 0 0 0) 0x7f2a1405e4a0 con 0x7f2a08004d50 2024-09-10T23:39:33.487 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:39:33.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.484+0000 7f2a2b800640 1 -- 172.21.15.94:0/2791391560 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2a00072ca0 msgr2=0x7f2a00075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:33.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.484+0000 7f2a2b800640 1 --2- 172.21.15.94:0/2791391560 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2a00072ca0 0x7f2a00075160 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c001870 tx=0x7f2a0c006040 comp rx=0 tx=0).stop 2024-09-10T23:39:33.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.484+0000 7f2a2b800640 1 -- 172.21.15.94:0/2791391560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a08004d50 msgr2=0x7f2a24146e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:33.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.484+0000 7f2a2b800640 1 --2- 172.21.15.94:0/2791391560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a08004d50 0x7f2a24146e90 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400d730 tx=0x7f2a1400db30 comp rx=0 tx=0).stop 2024-09-10T23:39:33.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.484+0000 7f2a2b800640 1 -- 172.21.15.94:0/2791391560 shutdown_connections 2024-09-10T23:39:33.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.484+0000 7f2a2b800640 1 --2- 172.21.15.94:0/2791391560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a240c5250 0x7f2a240c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:33.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.484+0000 7f2a2b800640 1 --2- 172.21.15.94:0/2791391560 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2a00072ca0 0x7f2a00075160 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:33.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.484+0000 7f2a2b800640 1 --2- 172.21.15.94:0/2791391560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a24147710 0x7f2a240ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:33.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.484+0000 7f2a2b800640 1 --2- 172.21.15.94:0/2791391560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a08004d50 0x7f2a24146e90 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:33.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.484+0000 7f2a2b800640 1 -- 172.21.15.94:0/2791391560 >> 172.21.15.94:0/2791391560 conn(0x7f2a2406cdf0 msgr2=0x7f2a240b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:33.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.484+0000 7f2a2b800640 1 -- 172.21.15.94:0/2791391560 shutdown_connections 2024-09-10T23:39:33.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:33.484+0000 7f2a2b800640 1 -- 172.21.15.94:0/2791391560 wait complete. 2024-09-10T23:39:33.507 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:39:33.507 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:39:33.507 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-09-10T23:39:33.507 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T23:39:33.507 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T23:39:33.508 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:39:33.508 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:39:33.508 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:39:33.508 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:39:33.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:33.508 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-09-10T23:39:33.518 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:39:33.518 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:39:33.518 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:39:33.518 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:39:33.518 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-09-10T23:39:33.518 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:39:33.519 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:39:33.519 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:39:33.519 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-09-10T23:39:33.519 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:39:33.519 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:39:33.519 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-09-10T23:39:33.519 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:39:33.519 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:39:33.519 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:39:33.519 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:39:33.519 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:39:33.519 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42042sec preferred_lft 42042sec 2024-09-10T23:39:33.519 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-09-10T23:39:33.520 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:39:33.520 INFO:teuthology.orchestra.run.smithi181.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:39:33.520 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 02:42:3b:2e:09:2d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:39:33.520 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:39:33.520 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:39:33.520 INFO:teuthology.orchestra.run.smithi181.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:39:33.520 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:39:33.520 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:39:33.520 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:39:33.520 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:39:33.520 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:39:33.520 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:39:33.520 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:39:33.520 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:39:33.521 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:39:33.521 INFO:teuthology.orchestra.run.smithi181.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:39:33.521 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:39:33.521 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:39:33.521 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:39:33.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:33.522 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:39:33.584 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-09-10T23:39:33.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:33.587 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-09-10T23:39:33.640 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-09-10T23:39:33.640 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:39:33.640 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:39:33.640 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:39:33.640 INFO:teuthology.orchestra.run.smithi181.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:39:33.640 INFO:teuthology.orchestra.run.smithi181.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:39:33.640 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:39:33.640 INFO:teuthology.orchestra.run.smithi181.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:39:33.640 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:39:33.641 INFO:teuthology.orchestra.run.smithi181.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:39:33.641 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:39:33.642 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:39:33.642 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:39:33.642 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:39:33.642 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:39:33.642 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:39:33.642 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:39:33.735 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:39:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:39:33.776 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:39:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:39:33.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:33.784 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-09-10T23:39:33.792 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T23:39:33.792 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T23:39:33.793 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:39:33.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:33.794 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-09-10T23:39:33.860 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T23:39:33.860 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:39:33.860 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T23:39:33.908 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:39:33.909 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:39:33.909 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:39:33.959 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:39:34.024 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:39:34.077 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:39:34.077 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:39:34.077 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:39:34.077 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:39:34.078 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:39:34.078 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:39:34.078 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:39:34.078 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:39:34.078 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:39:34.078 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:39:34.078 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:39:34.078 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:39:34.078 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:39:34.078 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:39:34.078 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:39:34.078 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:39:34.078 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:39:34.079 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:39:34.079 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:39:34.079 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:39:34.079 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:39:34.079 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:39:34.079 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:39:34.079 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:39:34.079 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:39:34.079 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:39:34.079 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:39:34.079 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:39:34.079 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:39:34.079 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:39:34.079 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:39:34.080 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:39:34.080 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:39:34.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:34.081 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:39:34.132 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:39:34.132 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T23:39:34.175 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:39:34.195 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:39:34.249 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:39:34.249 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:39:34.249 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:39:34.249 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:39:34.250 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:39:34.250 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:39:34.250 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:39:34.250 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:39:34.250 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:39:34.250 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:39:34.250 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:39:34.250 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:39:34.250 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:39:34.250 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:39:34.250 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:39:34.250 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:39:34.250 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:39:34.250 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:39:34.251 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:39:34.251 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:39:34.251 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:39:34.251 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:39:34.251 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:39:34.251 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:39:34.251 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:39:34.251 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:39:34.251 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:39:34.251 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:39:34.251 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:39:34.251 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:39:34.251 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:39:34.251 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:39:34.251 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:39:34.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:34.252 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:39:34.269 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:2024-09-10T23:39:34.268+0000 7fa59cf0f580 -1 init, newargv = 0x55f694c80ed0 newargc=15 2024-09-10T23:39:34.269 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[36046]: starting ceph client 2024-09-10T23:39:34.276 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[36046]: starting fuse 2024-09-10T23:39:34.277 INFO:teuthology.orchestra.run.smithi181.stdout:44 2024-09-10T23:39:34.278 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:39:34.278 DEBUG:teuthology.orchestra.run.smithi181:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:39:34.278 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2024-09-10T23:39:34.278 DEBUG:teuthology.orchestra.run.smithi181:> import re 2024-09-10T23:39:34.278 DEBUG:teuthology.orchestra.run.smithi181:> import os 2024-09-10T23:39:34.278 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2024-09-10T23:39:34.278 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:39:34.278 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2024-09-10T23:39:34.278 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T23:39:34.278 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2024-09-10T23:39:34.278 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T23:39:34.278 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:39:34.279 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2024-09-10T23:39:34.279 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2024-09-10T23:39:34.279 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2024-09-10T23:39:34.279 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2024-09-10T23:39:34.279 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:39:34.279 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2024-09-10T23:39:34.279 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:39:34.279 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:39:34.279 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:39:34.279 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2024-09-10T23:39:34.279 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2024-09-10T23:39:34.279 DEBUG:teuthology.orchestra.run.smithi181:> return f 2024-09-10T23:39:34.279 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:39:34.279 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:39:34.279 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.1")) 2024-09-10T23:39:34.279 DEBUG:teuthology.orchestra.run.smithi181:> ' 2024-09-10T23:39:34.403 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.1.36046.asok 2024-09-10T23:39:34.406 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:39:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:39:34.414 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36046.asok 2024-09-10T23:39:34.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:34.415 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36046.asok status 2024-09-10T23:39:34.587 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T23:39:34.588 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2024-09-10T23:39:34.588 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:39:34.588 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:39:34.588 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "1", 2024-09-10T23:39:34.588 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2024-09-10T23:39:34.588 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T23:39:34.588 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "36046", 2024-09-10T23:39:34.588 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2024-09-10T23:39:34.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:39:34.588 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2024-09-10T23:39:34.588 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2024-09-10T23:39:34.589 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 7963, 2024-09-10T23:39:34.589 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2024-09-10T23:39:34.589 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2024-09-10T23:39:34.589 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2024-09-10T23:39:34.589 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 7963 2024-09-10T23:39:34.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:39:34.589 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:39:34.589 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:39:34.589 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:39:34.589 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 4070871116 2024-09-10T23:39:34.589 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T23:39:34.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:39:34.590 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:39:34.590 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:39:34.590 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:39:34.590 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 4070871116 2024-09-10T23:39:34.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:39:34.590 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.7963 192.168.0.1:0/4070871116", 2024-09-10T23:39:34.590 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.1:0/4070871116", 2024-09-10T23:39:34.590 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2024-09-10T23:39:34.590 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 97, 2024-09-10T23:39:34.590 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 138, 2024-09-10T23:39:34.590 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:39:34.590 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2024-09-10T23:39:34.591 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2024-09-10T23:39:34.591 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T23:39:34.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:34.602 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2024-09-10T23:39:34.603 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T23:39:34.612 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2024-09-10T23:39:34.612 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T23:39:34.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:34.612 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:39:34.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T23:39:34.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.824+0000 7fd885800640 1 -- 172.21.15.94:0/3549669472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880162560 msgr2=0x7fd88006e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:34.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.824+0000 7fd885800640 1 --2- 172.21.15.94:0/3549669472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880162560 0x7fd88006e290 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009900 tx=0x7fd86c01c8e0 comp rx=0 tx=0).stop 2024-09-10T23:39:34.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.824+0000 7fd885800640 1 -- 172.21.15.94:0/3549669472 shutdown_connections 2024-09-10T23:39:34.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.824+0000 7fd885800640 1 --2- 172.21.15.94:0/3549669472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880162560 0x7fd88006e290 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:34.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.824+0000 7fd885800640 1 -- 172.21.15.94:0/3549669472 >> 172.21.15.94:0/3549669472 conn(0x7fd88009aa70 msgr2=0x7fd88009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:34.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.824+0000 7fd885800640 1 -- 172.21.15.94:0/3549669472 shutdown_connections 2024-09-10T23:39:34.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.824+0000 7fd885800640 1 -- 172.21.15.94:0/3549669472 wait complete. 2024-09-10T23:39:34.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.824+0000 7fd885800640 1 Processor -- start 2024-09-10T23:39:34.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.824+0000 7fd885800640 1 -- start start 2024-09-10T23:39:34.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd885800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd88006e7d0 0x7fd88014e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:34.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd885800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd88015dd50 0x7fd8801584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:34.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd885800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd880162560 0x7fd8800cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:34.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd885800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd88016c3c0 con 0x7fd88015dd50 2024-09-10T23:39:34.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd885800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd88016c240 con 0x7fd880162560 2024-09-10T23:39:34.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd885800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd88016c540 con 0x7fd88006e7d0 2024-09-10T23:39:34.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd87e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd88015dd50 0x7fd8801584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:34.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd87e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd88015dd50 0x7fd8801584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55026/0 (socket says 172.21.15.94:55026) 2024-09-10T23:39:34.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd87e000640 1 -- 172.21.15.94:0/3837974669 learned_addr learned my addr 172.21.15.94:0/3837974669 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:34.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd87ea00640 1 --2- 172.21.15.94:0/3837974669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd88006e7d0 0x7fd88014e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:34.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd87f400640 1 --2- 172.21.15.94:0/3837974669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd880162560 0x7fd8800cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:34.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd87e000640 1 -- 172.21.15.94:0/3837974669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd88006e7d0 msgr2=0x7fd88014e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:34.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd87e000640 1 --2- 172.21.15.94:0/3837974669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd88006e7d0 0x7fd88014e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:34.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd87e000640 1 -- 172.21.15.94:0/3837974669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd880162560 msgr2=0x7fd8800cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:34.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd87e000640 1 --2- 172.21.15.94:0/3837974669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd880162560 0x7fd8800cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:34.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd87e000640 1 -- 172.21.15.94:0/3837974669 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd880159400 con 0x7fd88015dd50 2024-09-10T23:39:34.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd87e000640 1 --2- 172.21.15.94:0/3837974669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd88015dd50 0x7fd8801584a0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009850 tx=0x7fd86c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:34.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd876a00640 1 -- 172.21.15.94:0/3837974669 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0b2040 con 0x7fd88015dd50 2024-09-10T23:39:34.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd876a00640 1 -- 172.21.15.94:0/3837974669 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd86c003600 con 0x7fd88015dd50 2024-09-10T23:39:34.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd885800640 1 -- 172.21.15.94:0/3837974669 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8801596e0 con 0x7fd88015dd50 2024-09-10T23:39:34.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd876a00640 1 -- 172.21.15.94:0/3837974669 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c003a10 con 0x7fd88015dd50 2024-09-10T23:39:34.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd885800640 1 -- 172.21.15.94:0/3837974669 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd880152270 con 0x7fd88015dd50 2024-09-10T23:39:34.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd885800640 1 -- 172.21.15.94:0/3837974669 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd88006e290 con 0x7fd88015dd50 2024-09-10T23:39:34.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.828+0000 7fd876a00640 1 -- 172.21.15.94:0/3837974669 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7fd86c003bb0 con 0x7fd88015dd50 2024-09-10T23:39:34.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.832+0000 7fd876a00640 1 --2- 172.21.15.94:0/3837974669 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd860072ca0 0x7fd860075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:34.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.832+0000 7fd876a00640 1 -- 172.21.15.94:0/3837974669 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7fd86c12d6c0 con 0x7fd88015dd50 2024-09-10T23:39:34.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.832+0000 7fd87ea00640 1 --2- 172.21.15.94:0/3837974669 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd860072ca0 0x7fd860075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:34.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.832+0000 7fd87ea00640 1 --2- 172.21.15.94:0/3837974669 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd860072ca0 0x7fd860075160 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd870007fd0 tx=0x7fd870009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:34.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:34.836+0000 7fd876a00640 1 -- 172.21.15.94:0/3837974669 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd86c0164d0 con 0x7fd88015dd50 2024-09-10T23:39:35.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.036+0000 7fd885800640 1 -- 172.21.15.94:0/3837974669 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fd8800d29c0 con 0x7fd88015dd50 2024-09-10T23:39:35.414 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:39:35.418 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:39:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.820+0000 7fd876a00640 1 -- 172.21.15.94:0/3837974669 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v98) ==== 136+0+0 (secure 0 0 0) 0x7fd86c0f7ed0 con 0x7fd88015dd50 2024-09-10T23:39:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.824+0000 7fd885800640 1 -- 172.21.15.94:0/3837974669 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd860072ca0 msgr2=0x7fd860075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.824+0000 7fd885800640 1 --2- 172.21.15.94:0/3837974669 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd860072ca0 0x7fd860075160 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd870007fd0 tx=0x7fd870009040 comp rx=0 tx=0).stop 2024-09-10T23:39:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.824+0000 7fd885800640 1 -- 172.21.15.94:0/3837974669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd88015dd50 msgr2=0x7fd8801584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.824+0000 7fd885800640 1 --2- 172.21.15.94:0/3837974669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd88015dd50 0x7fd8801584a0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009850 tx=0x7fd86c0040c0 comp rx=0 tx=0).stop 2024-09-10T23:39:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.824+0000 7fd885800640 1 -- 172.21.15.94:0/3837974669 shutdown_connections 2024-09-10T23:39:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.824+0000 7fd885800640 1 --2- 172.21.15.94:0/3837974669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd880162560 0x7fd8800cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.824+0000 7fd885800640 1 --2- 172.21.15.94:0/3837974669 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd860072ca0 0x7fd860075160 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.824+0000 7fd885800640 1 --2- 172.21.15.94:0/3837974669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd88015dd50 0x7fd8801584a0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.824+0000 7fd885800640 1 --2- 172.21.15.94:0/3837974669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd88006e7d0 0x7fd88014e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.824+0000 7fd885800640 1 -- 172.21.15.94:0/3837974669 >> 172.21.15.94:0/3837974669 conn(0x7fd88009aa70 msgr2=0x7fd8800cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.824+0000 7fd885800640 1 -- 172.21.15.94:0/3837974669 shutdown_connections 2024-09-10T23:39:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.824+0000 7fd885800640 1 -- 172.21.15.94:0/3837974669 wait complete. 2024-09-10T23:39:35.847 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T23:39:35.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.988+0000 7fc03fc00640 1 -- 172.21.15.94:0/1234555553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc03806fc00 msgr2=0x7fc0380c49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:35.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.988+0000 7fc03fc00640 1 --2- 172.21.15.94:0/1234555553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc03806fc00 0x7fc0380c49b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc020009840 tx=0x7fc02001c820 comp rx=0 tx=0).stop 2024-09-10T23:39:35.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.988+0000 7fc03fc00640 1 -- 172.21.15.94:0/1234555553 shutdown_connections 2024-09-10T23:39:35.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.988+0000 7fc03fc00640 1 --2- 172.21.15.94:0/1234555553 >> v2:172.21.15.181:3300/0 conn(0x7fc0380cb4d0 0x7fc03815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:35.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.988+0000 7fc03fc00640 1 --2- 172.21.15.94:0/1234555553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc03806fc00 0x7fc0380c49b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:35.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.988+0000 7fc03fc00640 1 -- 172.21.15.94:0/1234555553 >> 172.21.15.94:0/1234555553 conn(0x7fc0380cfaf0 msgr2=0x7fc0380cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:35.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.988+0000 7fc03fc00640 1 -- 172.21.15.94:0/1234555553 shutdown_connections 2024-09-10T23:39:35.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.988+0000 7fc03fc00640 1 -- 172.21.15.94:0/1234555553 wait complete. 2024-09-10T23:39:35.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc03fc00640 1 Processor -- start 2024-09-10T23:39:35.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc03fc00640 1 -- start start 2024-09-10T23:39:35.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc03fc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc03806fc00 0x7fc038154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:35.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc03fc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0380cb4d0 0x7fc0380b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:35.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc03fc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc038158790 0x7fc038155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:35.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc03fc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc0381680d0 con 0x7fc0380cb4d0 2024-09-10T23:39:35.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc03fc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc038167f50 con 0x7fc038158790 2024-09-10T23:39:35.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc03fc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc038168250 con 0x7fc03806fc00 2024-09-10T23:39:35.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc02fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0380cb4d0 0x7fc0380b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:35.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc02fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0380cb4d0 0x7fc0380b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55030/0 (socket says 172.21.15.94:55030) 2024-09-10T23:39:35.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc02fe00640 1 -- 172.21.15.94:0/221180440 learned_addr learned my addr 172.21.15.94:0/221180440 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:35.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc03dc00640 1 --2- 172.21.15.94:0/221180440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc038158790 0x7fc038155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:35.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc03d200640 1 --2- 172.21.15.94:0/221180440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc03806fc00 0x7fc038154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:35.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc02fe00640 1 -- 172.21.15.94:0/221180440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc03806fc00 msgr2=0x7fc038154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:35.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc02fe00640 1 --2- 172.21.15.94:0/221180440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc03806fc00 0x7fc038154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:35.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc02fe00640 1 -- 172.21.15.94:0/221180440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc038158790 msgr2=0x7fc038155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:35.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc02fe00640 1 --2- 172.21.15.94:0/221180440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc038158790 0x7fc038155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:35.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc02fe00640 1 -- 172.21.15.94:0/221180440 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0381fb780 con 0x7fc0380cb4d0 2024-09-10T23:39:35.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc02fe00640 1 --2- 172.21.15.94:0/221180440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0380cb4d0 0x7fc0380b5280 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fc020009790 tx=0x7fc0200a55f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:35.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc035600640 1 -- 172.21.15.94:0/221180440 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0200b2040 con 0x7fc0380cb4d0 2024-09-10T23:39:35.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc035600640 1 -- 172.21.15.94:0/221180440 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0200043b0 con 0x7fc0380cb4d0 2024-09-10T23:39:35.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc035600640 1 -- 172.21.15.94:0/221180440 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020005280 con 0x7fc0380cb4d0 2024-09-10T23:39:35.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc03fc00640 1 -- 172.21.15.94:0/221180440 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0381fba60 con 0x7fc0380cb4d0 2024-09-10T23:39:35.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.992+0000 7fc03fc00640 1 -- 172.21.15.94:0/221180440 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0381fc020 con 0x7fc0380cb4d0 2024-09-10T23:39:35.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.996+0000 7fc03fc00640 1 -- 172.21.15.94:0/221180440 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0380c49b0 con 0x7fc0380cb4d0 2024-09-10T23:39:35.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.996+0000 7fc035600640 1 -- 172.21.15.94:0/221180440 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7fc0200b5070 con 0x7fc0380cb4d0 2024-09-10T23:39:36.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.996+0000 7fc035600640 1 --2- 172.21.15.94:0/221180440 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc014072cd0 0x7fc014075190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:36.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.996+0000 7fc035600640 1 -- 172.21.15.94:0/221180440 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7fc02012d5f0 con 0x7fc0380cb4d0 2024-09-10T23:39:36.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.996+0000 7fc03d200640 1 --2- 172.21.15.94:0/221180440 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc014072cd0 0x7fc014075190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:36.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:35.996+0000 7fc03d200640 1 --2- 172.21.15.94:0/221180440 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc014072cd0 0x7fc014075190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc0380cf6b0 tx=0x7fc0280013a0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.000+0000 7fc035600640 1 -- 172.21.15.94:0/221180440 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc0200f2de0 con 0x7fc0380cb4d0 2024-09-10T23:39:36.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.188+0000 7fc03fc00640 1 -- 172.21.15.94:0/221180440 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fc0380d0850 con 0x7fc014072cd0 2024-09-10T23:39:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.188+0000 7fc035600640 1 -- 172.21.15.94:0/221180440 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fc0380d0850 con 0x7fc014072cd0 2024-09-10T23:39:36.191 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:39:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephfs" 2024-09-10T23:39:36.191 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:36.191 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:39:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.188+0000 7fc03fc00640 1 -- 172.21.15.94:0/221180440 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc014072cd0 msgr2=0x7fc014075190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.188+0000 7fc03fc00640 1 --2- 172.21.15.94:0/221180440 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc014072cd0 0x7fc014075190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc0380cf6b0 tx=0x7fc0280013a0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.188+0000 7fc03fc00640 1 -- 172.21.15.94:0/221180440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0380cb4d0 msgr2=0x7fc0380b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.188+0000 7fc03fc00640 1 --2- 172.21.15.94:0/221180440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0380cb4d0 0x7fc0380b5280 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fc020009790 tx=0x7fc0200a55f0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.192+0000 7fc03fc00640 1 -- 172.21.15.94:0/221180440 shutdown_connections 2024-09-10T23:39:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.192+0000 7fc03fc00640 1 --2- 172.21.15.94:0/221180440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc038158790 0x7fc038155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.192+0000 7fc03fc00640 1 --2- 172.21.15.94:0/221180440 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc014072cd0 0x7fc014075190 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.192+0000 7fc03fc00640 1 --2- 172.21.15.94:0/221180440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0380cb4d0 0x7fc0380b5280 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.192+0000 7fc03fc00640 1 --2- 172.21.15.94:0/221180440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc03806fc00 0x7fc038154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.192+0000 7fc03fc00640 1 -- 172.21.15.94:0/221180440 >> 172.21.15.94:0/221180440 conn(0x7fc0380cfaf0 msgr2=0x7fc0380b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.192+0000 7fc03fc00640 1 -- 172.21.15.94:0/221180440 shutdown_connections 2024-09-10T23:39:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.192+0000 7fc03fc00640 1 -- 172.21.15.94:0/221180440 wait complete. 2024-09-10T23:39:36.213 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:39:36.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa862800640 1 -- 172.21.15.94:0/21859724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85c0c49b0 msgr2=0x7fa85c0cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:36.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa862800640 1 --2- 172.21.15.94:0/21859724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85c0c49b0 0x7fa85c0cadc0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa844009900 tx=0x7fa84401c8e0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa862800640 1 -- 172.21.15.94:0/21859724 shutdown_connections 2024-09-10T23:39:36.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa862800640 1 --2- 172.21.15.94:0/21859724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85c0c49b0 0x7fa85c0cadc0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa862800640 1 --2- 172.21.15.94:0/21859724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa85c158790 0x7fa85c06fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa862800640 1 -- 172.21.15.94:0/21859724 >> 172.21.15.94:0/21859724 conn(0x7fa85c0cfaf0 msgr2=0x7fa85c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:36.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa862800640 1 -- 172.21.15.94:0/21859724 shutdown_connections 2024-09-10T23:39:36.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa862800640 1 -- 172.21.15.94:0/21859724 wait complete. 2024-09-10T23:39:36.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa862800640 1 Processor -- start 2024-09-10T23:39:36.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa862800640 1 -- start start 2024-09-10T23:39:36.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa862800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa85c0c49b0 0x7fa85c153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:36.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa862800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85c0cb300 0x7fa85c1fb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:36.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa862800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa85c158790 0x7fa85c1559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:36.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa862800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c168290 con 0x7fa85c0cb300 2024-09-10T23:39:36.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa862800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c168110 con 0x7fa85c158790 2024-09-10T23:39:36.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa862800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa85c168410 con 0x7fa85c0c49b0 2024-09-10T23:39:36.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa852a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85c0cb300 0x7fa85c1fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:36.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa852a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85c0cb300 0x7fa85c1fb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55046/0 (socket says 172.21.15.94:55046) 2024-09-10T23:39:36.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa852a00640 1 -- 172.21.15.94:0/38843460 learned_addr learned my addr 172.21.15.94:0/38843460 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:36.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa85b400640 1 --2- 172.21.15.94:0/38843460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa85c0c49b0 0x7fa85c153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:36.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa85be00640 1 --2- 172.21.15.94:0/38843460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa85c158790 0x7fa85c1559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:36.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa852a00640 1 -- 172.21.15.94:0/38843460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa85c0c49b0 msgr2=0x7fa85c153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:36.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa852a00640 1 --2- 172.21.15.94:0/38843460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa85c0c49b0 0x7fa85c153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa852a00640 1 -- 172.21.15.94:0/38843460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa85c158790 msgr2=0x7fa85c1559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:36.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa852a00640 1 --2- 172.21.15.94:0/38843460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa85c158790 0x7fa85c1559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa852a00640 1 -- 172.21.15.94:0/38843460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c0d01a0 con 0x7fa85c0cb300 2024-09-10T23:39:36.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa85b400640 1 --2- 172.21.15.94:0/38843460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa85c0c49b0 0x7fa85c153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:36.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa85be00640 1 --2- 172.21.15.94:0/38843460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa85c158790 0x7fa85c1559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:36.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.356+0000 7fa852a00640 1 --2- 172.21.15.94:0/38843460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85c0cb300 0x7fa85c1fb960 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa844009850 tx=0x7fa8440a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:36.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.360+0000 7fa853e00640 1 -- 172.21.15.94:0/38843460 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8440b2040 con 0x7fa85c0cb300 2024-09-10T23:39:36.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.360+0000 7fa853e00640 1 -- 172.21.15.94:0/38843460 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8440a6d50 con 0x7fa85c0cb300 2024-09-10T23:39:36.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.360+0000 7fa862800640 1 -- 172.21.15.94:0/38843460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85c154200 con 0x7fa85c0cb300 2024-09-10T23:39:36.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.360+0000 7fa853e00640 1 -- 172.21.15.94:0/38843460 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844005280 con 0x7fa85c0cb300 2024-09-10T23:39:36.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.360+0000 7fa862800640 1 -- 172.21.15.94:0/38843460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85c0af2e0 con 0x7fa85c0cb300 2024-09-10T23:39:36.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.360+0000 7fa862800640 1 -- 172.21.15.94:0/38843460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85c15a2c0 con 0x7fa85c0cb300 2024-09-10T23:39:36.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.360+0000 7fa853e00640 1 -- 172.21.15.94:0/38843460 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7fa844005420 con 0x7fa85c0cb300 2024-09-10T23:39:36.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.360+0000 7fa853e00640 1 --2- 172.21.15.94:0/38843460 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa838072ca0 0x7fa838075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:36.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.360+0000 7fa853e00640 1 -- 172.21.15.94:0/38843460 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7fa84412d670 con 0x7fa85c0cb300 2024-09-10T23:39:36.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.360+0000 7fa85b400640 1 --2- 172.21.15.94:0/38843460 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa838072ca0 0x7fa838075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:36.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.364+0000 7fa85b400640 1 --2- 172.21.15.94:0/38843460 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa838072ca0 0x7fa838075160 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa84c005fd0 tx=0x7fa84c005d00 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:36.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.364+0000 7fa853e00640 1 -- 172.21.15.94:0/38843460 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa85c15a2c0 con 0x7fa85c0cb300 2024-09-10T23:39:36.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.532+0000 7fa862800640 1 -- 172.21.15.94:0/38843460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa85c0cadc0 con 0x7fa85c0cb300 2024-09-10T23:39:36.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.532+0000 7fa853e00640 1 -- 172.21.15.94:0/38843460 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7fa8440164d0 con 0x7fa85c0cb300 2024-09-10T23:39:36.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.532+0000 7fa862800640 1 -- 172.21.15.94:0/38843460 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa838072ca0 msgr2=0x7fa838075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:36.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.532+0000 7fa862800640 1 --2- 172.21.15.94:0/38843460 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa838072ca0 0x7fa838075160 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa84c005fd0 tx=0x7fa84c005d00 comp rx=0 tx=0).stop 2024-09-10T23:39:36.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.532+0000 7fa862800640 1 -- 172.21.15.94:0/38843460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85c0cb300 msgr2=0x7fa85c1fb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:36.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.532+0000 7fa862800640 1 --2- 172.21.15.94:0/38843460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85c0cb300 0x7fa85c1fb960 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa844009850 tx=0x7fa8440a56b0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.532+0000 7fa862800640 1 -- 172.21.15.94:0/38843460 shutdown_connections 2024-09-10T23:39:36.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.532+0000 7fa862800640 1 --2- 172.21.15.94:0/38843460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa85c158790 0x7fa85c1559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.532+0000 7fa862800640 1 --2- 172.21.15.94:0/38843460 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa838072ca0 0x7fa838075160 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.532+0000 7fa862800640 1 --2- 172.21.15.94:0/38843460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa85c0cb300 0x7fa85c1fb960 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.532+0000 7fa862800640 1 --2- 172.21.15.94:0/38843460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa85c0c49b0 0x7fa85c153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.532+0000 7fa862800640 1 -- 172.21.15.94:0/38843460 >> 172.21.15.94:0/38843460 conn(0x7fa85c0cfaf0 msgr2=0x7fa85c0afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:36.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.532+0000 7fa862800640 1 -- 172.21.15.94:0/38843460 shutdown_connections 2024-09-10T23:39:36.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.532+0000 7fa862800640 1 -- 172.21.15.94:0/38843460 wait complete. 2024-09-10T23:39:36.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:36.555 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol74910) 2024-09-10T23:39:36.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:36.587 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp85160/subvol37321) 2024-09-10T23:39:36.659 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol74910 2024-09-10T23:39:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.848+0000 7f3d59200640 1 -- 172.21.15.94:0/1104915404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d5415f560 msgr2=0x7f3d540cfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.848+0000 7f3d59200640 1 --2- 172.21.15.94:0/1104915404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d5415f560 0x7f3d540cfa90 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009900 tx=0x7f3d3c01c8a0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.848+0000 7f3d59200640 1 -- 172.21.15.94:0/1104915404 shutdown_connections 2024-09-10T23:39:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.848+0000 7f3d59200640 1 --2- 172.21.15.94:0/1104915404 >> v2:172.21.15.181:3300/0 conn(0x7f3d540cffd0 0x7f3d540d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.848+0000 7f3d59200640 1 --2- 172.21.15.94:0/1104915404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d5415f560 0x7f3d540cfa90 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.848+0000 7f3d59200640 1 --2- 172.21.15.94:0/1104915404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5415f180 0x7f3d540cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.848+0000 7f3d59200640 1 -- 172.21.15.94:0/1104915404 >> 172.21.15.94:0/1104915404 conn(0x7f3d540d1cb0 msgr2=0x7f3d540d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.848+0000 7f3d59200640 1 -- 172.21.15.94:0/1104915404 shutdown_connections 2024-09-10T23:39:36.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.848+0000 7f3d59200640 1 -- 172.21.15.94:0/1104915404 wait complete. 2024-09-10T23:39:36.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.848+0000 7f3d59200640 1 Processor -- start 2024-09-10T23:39:36.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.848+0000 7f3d59200640 1 -- start start 2024-09-10T23:39:36.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.848+0000 7f3d59200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d540cffd0 0x7f3d5415ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:36.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d59200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5415f180 0x7f3d54163e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:36.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d59200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5415f560 0x7f3d5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:36.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d59200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d540b1870 con 0x7f3d540cffd0 2024-09-10T23:39:36.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d59200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3d540b16f0 con 0x7f3d5415f180 2024-09-10T23:39:36.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d59200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3d540b19f0 con 0x7f3d5415f560 2024-09-10T23:39:36.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d51e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d540cffd0 0x7f3d5415ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d51e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d540cffd0 0x7f3d5415ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55058/0 (socket says 172.21.15.94:55058) 2024-09-10T23:39:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d51e00640 1 -- 172.21.15.94:0/701077529 learned_addr learned my addr 172.21.15.94:0/701077529 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d49400640 1 --2- 172.21.15.94:0/701077529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5415f180 0x7f3d54163e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d52800640 1 --2- 172.21.15.94:0/701077529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5415f560 0x7f3d5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d51e00640 1 -- 172.21.15.94:0/701077529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5415f560 msgr2=0x7f3d5406e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d51e00640 1 --2- 172.21.15.94:0/701077529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5415f560 0x7f3d5406e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d51e00640 1 -- 172.21.15.94:0/701077529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5415f180 msgr2=0x7f3d54163e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d51e00640 1 --2- 172.21.15.94:0/701077529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5415f180 0x7f3d54163e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d51e00640 1 -- 172.21.15.94:0/701077529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d54165920 con 0x7f3d540cffd0 2024-09-10T23:39:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d51e00640 1 --2- 172.21.15.94:0/701077529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d540cffd0 0x7f3d5415ad80 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400ead0 tx=0x7f3d4400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d4aa00640 1 -- 172.21.15.94:0/701077529 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4400ce40 con 0x7f3d540cffd0 2024-09-10T23:39:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d4aa00640 1 -- 172.21.15.94:0/701077529 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d44004c20 con 0x7f3d540cffd0 2024-09-10T23:39:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d4aa00640 1 -- 172.21.15.94:0/701077529 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d44005280 con 0x7f3d540cffd0 2024-09-10T23:39:36.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d59200640 1 -- 172.21.15.94:0/701077529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d54165bc0 con 0x7f3d540cffd0 2024-09-10T23:39:36.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d59200640 1 -- 172.21.15.94:0/701077529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d54166180 con 0x7f3d540cffd0 2024-09-10T23:39:36.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d59200640 1 -- 172.21.15.94:0/701077529 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d540cd0a0 con 0x7f3d540cffd0 2024-09-10T23:39:36.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.852+0000 7f3d4aa00640 1 -- 172.21.15.94:0/701077529 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f3d4400f040 con 0x7f3d540cffd0 2024-09-10T23:39:36.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.856+0000 7f3d4aa00640 1 --2- 172.21.15.94:0/701077529 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3d30072ca0 0x7f3d30075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:36.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.856+0000 7f3d4aa00640 1 -- 172.21.15.94:0/701077529 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7f3d440945f0 con 0x7f3d540cffd0 2024-09-10T23:39:36.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.856+0000 7f3d49400640 1 --2- 172.21.15.94:0/701077529 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3d30072ca0 0x7f3d30075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:36.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.856+0000 7f3d49400640 1 --2- 172.21.15.94:0/701077529 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3d30072ca0 0x7f3d30075160 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0af000 tx=0x7f3d3c01b0e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:36.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:36.860+0000 7f3d4aa00640 1 -- 172.21.15.94:0/701077529 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3d44016030 con 0x7f3d540cffd0 2024-09-10T23:39:37.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.048+0000 7f3d59200640 1 -- 172.21.15.94:0/701077529 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol74910", "target": ["mon-mgr", ""]}) -- 0x7f3d540cc7e0 con 0x7f3d30072ca0 2024-09-10T23:39:37.057 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:37.056+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:37.057 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:37.056+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:37.057 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:37.056+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:37.057 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:37.056+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:37.057 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:37.056+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:37.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.084+0000 7f3d4aa00640 1 -- 172.21.15.94:0/701077529 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f3d540cc7e0 con 0x7f3d30072ca0 2024-09-10T23:39:37.088 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/_nogroup/subvol74910 2024-09-10T23:39:37.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.088+0000 7f3d59200640 1 -- 172.21.15.94:0/701077529 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3d30072ca0 msgr2=0x7f3d30075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:37.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.088+0000 7f3d59200640 1 --2- 172.21.15.94:0/701077529 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3d30072ca0 0x7f3d30075160 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0af000 tx=0x7f3d3c01b0e0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.088+0000 7f3d59200640 1 -- 172.21.15.94:0/701077529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d540cffd0 msgr2=0x7f3d5415ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:37.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.088+0000 7f3d59200640 1 --2- 172.21.15.94:0/701077529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d540cffd0 0x7f3d5415ad80 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400ead0 tx=0x7f3d4400ef90 comp rx=0 tx=0).stop 2024-09-10T23:39:37.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.088+0000 7f3d59200640 1 -- 172.21.15.94:0/701077529 shutdown_connections 2024-09-10T23:39:37.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.088+0000 7f3d59200640 1 --2- 172.21.15.94:0/701077529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5415f180 0x7f3d54163e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.088+0000 7f3d59200640 1 --2- 172.21.15.94:0/701077529 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3d30072ca0 0x7f3d30075160 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.088+0000 7f3d59200640 1 --2- 172.21.15.94:0/701077529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5415f560 0x7f3d5406e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.088+0000 7f3d59200640 1 --2- 172.21.15.94:0/701077529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d540cffd0 0x7f3d5415ad80 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.088+0000 7f3d59200640 1 -- 172.21.15.94:0/701077529 >> 172.21.15.94:0/701077529 conn(0x7f3d540d1cb0 msgr2=0x7f3d540aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:37.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.088+0000 7f3d59200640 1 -- 172.21.15.94:0/701077529 shutdown_connections 2024-09-10T23:39:37.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.088+0000 7f3d59200640 1 -- 172.21.15.94:0/701077529 wait complete. 2024-09-10T23:39:37.109 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol37321 subvol_grp85160 2024-09-10T23:39:37.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.252+0000 7f3a6ec00640 1 -- 172.21.15.94:0/2917415297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a680b3f30 msgr2=0x7f3a680b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:37.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.252+0000 7f3a6ec00640 1 --2- 172.21.15.94:0/2917415297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a680b3f30 0x7f3a680b43c0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009bc0 tx=0x7f3a6001cb80 comp rx=0 tx=0).stop 2024-09-10T23:39:37.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.252+0000 7f3a6ec00640 1 -- 172.21.15.94:0/2917415297 shutdown_connections 2024-09-10T23:39:37.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.252+0000 7f3a6ec00640 1 --2- 172.21.15.94:0/2917415297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a680b3f30 0x7f3a680b43c0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.252+0000 7f3a6ec00640 1 --2- 172.21.15.94:0/2917415297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a680d1f00 0x7f3a680ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.252+0000 7f3a6ec00640 1 -- 172.21.15.94:0/2917415297 >> 172.21.15.94:0/2917415297 conn(0x7f3a6809d1c0 msgr2=0x7f3a680d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:37.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.252+0000 7f3a6ec00640 1 -- 172.21.15.94:0/2917415297 shutdown_connections 2024-09-10T23:39:37.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.252+0000 7f3a6ec00640 1 -- 172.21.15.94:0/2917415297 wait complete. 2024-09-10T23:39:37.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.252+0000 7f3a6ec00640 1 Processor -- start 2024-09-10T23:39:37.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.252+0000 7f3a6ec00640 1 -- start start 2024-09-10T23:39:37.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ec00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a680b3f30 0x7f3a6815f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:37.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ec00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a680cec30 0x7f3a68153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:37.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ec00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a680d1f00 0x7f3a6815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:37.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ec00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3a68162970 con 0x7f3a680cec30 2024-09-10T23:39:37.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ec00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3a681627f0 con 0x7f3a680b3f30 2024-09-10T23:39:37.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ec00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6816aec0 con 0x7f3a680d1f00 2024-09-10T23:39:37.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a680cec30 0x7f3a68153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:37.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a680cec30 0x7f3a68153040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55088/0 (socket says 172.21.15.94:55088) 2024-09-10T23:39:37.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ce00640 1 -- 172.21.15.94:0/428611092 learned_addr learned my addr 172.21.15.94:0/428611092 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:37.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6e200640 1 --2- 172.21.15.94:0/428611092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a680d1f00 0x7f3a6815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:37.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6d800640 1 --2- 172.21.15.94:0/428611092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a680b3f30 0x7f3a6815f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:37.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ce00640 1 -- 172.21.15.94:0/428611092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a680d1f00 msgr2=0x7f3a6815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:37.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ce00640 1 --2- 172.21.15.94:0/428611092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a680d1f00 0x7f3a6815b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ce00640 1 -- 172.21.15.94:0/428611092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a680b3f30 msgr2=0x7f3a6815f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:37.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ce00640 1 --2- 172.21.15.94:0/428611092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a680b3f30 0x7f3a6815f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ce00640 1 -- 172.21.15.94:0/428611092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6809d750 con 0x7f3a680cec30 2024-09-10T23:39:37.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6d800640 1 --2- 172.21.15.94:0/428611092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a680b3f30 0x7f3a6815f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:37.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ce00640 1 --2- 172.21.15.94:0/428611092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a680cec30 0x7f3a68153040 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400c450 tx=0x7f3a5400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:37.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a65600640 1 -- 172.21.15.94:0/428611092 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54017070 con 0x7f3a680cec30 2024-09-10T23:39:37.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a65600640 1 -- 172.21.15.94:0/428611092 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a54003600 con 0x7f3a680cec30 2024-09-10T23:39:37.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a65600640 1 -- 172.21.15.94:0/428611092 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54003a50 con 0x7f3a680cec30 2024-09-10T23:39:37.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ec00640 1 -- 172.21.15.94:0/428611092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6815fe90 con 0x7f3a680cec30 2024-09-10T23:39:37.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ec00640 1 -- 172.21.15.94:0/428611092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a68160450 con 0x7f3a680cec30 2024-09-10T23:39:37.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.256+0000 7f3a6ec00640 1 -- 172.21.15.94:0/428611092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a680b43c0 con 0x7f3a680cec30 2024-09-10T23:39:37.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.260+0000 7f3a65600640 1 -- 172.21.15.94:0/428611092 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7f3a54003bf0 con 0x7f3a680cec30 2024-09-10T23:39:37.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.260+0000 7f3a65600640 1 --2- 172.21.15.94:0/428611092 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3a48072ca0 0x7f3a48075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:37.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.260+0000 7f3a65600640 1 -- 172.21.15.94:0/428611092 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7f3a54093600 con 0x7f3a680cec30 2024-09-10T23:39:37.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.260+0000 7f3a6d800640 1 --2- 172.21.15.94:0/428611092 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3a48072ca0 0x7f3a48075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:37.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.260+0000 7f3a6d800640 1 --2- 172.21.15.94:0/428611092 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3a48072ca0 0x7f3a48075160 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0058d0 tx=0x7f3a5c005e30 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:37.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.264+0000 7f3a65600640 1 -- 172.21.15.94:0/428611092 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3a5405dbc0 con 0x7f3a680cec30 2024-09-10T23:39:37.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.456+0000 7f3a6ec00640 1 -- 172.21.15.94:0/428611092 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol37321", "group_name": "subvol_grp85160", "target": ["mon-mgr", ""]}) -- 0x7f3a680b2880 con 0x7f3a48072ca0 2024-09-10T23:39:37.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.476+0000 7f3a65600640 1 -- 172.21.15.94:0/428611092 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3a680b2880 con 0x7f3a48072ca0 2024-09-10T23:39:37.481 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/subvol_grp85160/subvol37321 2024-09-10T23:39:37.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.480+0000 7f3a6ec00640 1 -- 172.21.15.94:0/428611092 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3a48072ca0 msgr2=0x7f3a48075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:37.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.480+0000 7f3a6ec00640 1 --2- 172.21.15.94:0/428611092 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3a48072ca0 0x7f3a48075160 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0058d0 tx=0x7f3a5c005e30 comp rx=0 tx=0).stop 2024-09-10T23:39:37.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.480+0000 7f3a6ec00640 1 -- 172.21.15.94:0/428611092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a680cec30 msgr2=0x7f3a68153040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:37.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.480+0000 7f3a6ec00640 1 --2- 172.21.15.94:0/428611092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a680cec30 0x7f3a68153040 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400c450 tx=0x7f3a5400c910 comp rx=0 tx=0).stop 2024-09-10T23:39:37.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.480+0000 7f3a6ec00640 1 -- 172.21.15.94:0/428611092 shutdown_connections 2024-09-10T23:39:37.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.480+0000 7f3a6ec00640 1 --2- 172.21.15.94:0/428611092 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3a48072ca0 0x7f3a48075160 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.480+0000 7f3a6ec00640 1 --2- 172.21.15.94:0/428611092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a680d1f00 0x7f3a6815b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.480+0000 7f3a6ec00640 1 --2- 172.21.15.94:0/428611092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a680cec30 0x7f3a68153040 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.480+0000 7f3a6ec00640 1 --2- 172.21.15.94:0/428611092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a680b3f30 0x7f3a6815f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.480+0000 7f3a6ec00640 1 -- 172.21.15.94:0/428611092 >> 172.21.15.94:0/428611092 conn(0x7f3a6809d1c0 msgr2=0x7f3a6815e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:37.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.480+0000 7f3a6ec00640 1 -- 172.21.15.94:0/428611092 shutdown_connections 2024-09-10T23:39:37.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.480+0000 7f3a6ec00640 1 -- 172.21.15.94:0/428611092 wait complete. 2024-09-10T23:39:37.503 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol74910 2024-09-10T23:39:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.644+0000 7fe6e9800640 1 -- 172.21.15.94:0/2820280814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e40ce1e0 msgr2=0x7fe6e40cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.644+0000 7fe6e9800640 1 --2- 172.21.15.94:0/2820280814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e40ce1e0 0x7fe6e40cd710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009870 tx=0x7fe6cc01c7d0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.644+0000 7fe6e9800640 1 -- 172.21.15.94:0/2820280814 shutdown_connections 2024-09-10T23:39:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.644+0000 7fe6e9800640 1 --2- 172.21.15.94:0/2820280814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e40ce1e0 0x7fe6e40cd710 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.644+0000 7fe6e9800640 1 -- 172.21.15.94:0/2820280814 >> 172.21.15.94:0/2820280814 conn(0x7fe6e40d1c60 msgr2=0x7fe6e40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.644+0000 7fe6e9800640 1 -- 172.21.15.94:0/2820280814 shutdown_connections 2024-09-10T23:39:37.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.644+0000 7fe6e9800640 1 -- 172.21.15.94:0/2820280814 wait complete. 2024-09-10T23:39:37.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6e9800640 1 Processor -- start 2024-09-10T23:39:37.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6e9800640 1 -- start start 2024-09-10T23:39:37.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6e9800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6e40b1100 0x7fe6e40c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:37.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6e9800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6e40ce1e0 0x7fe6e41557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:37.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6e9800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e40ce890 0x7fe6e4155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:37.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6e9800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e416a300 con 0x7fe6e40ce1e0 2024-09-10T23:39:37.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6e9800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e416a180 con 0x7fe6e40b1100 2024-09-10T23:39:37.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6e9800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e416a480 con 0x7fe6e40ce890 2024-09-10T23:39:37.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6dbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6e40ce1e0 0x7fe6e41557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:37.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6dbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6e40ce1e0 0x7fe6e41557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55092/0 (socket says 172.21.15.94:55092) 2024-09-10T23:39:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6dbe00640 1 -- 172.21.15.94:0/1868858201 learned_addr learned my addr 172.21.15.94:0/1868858201 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6e2a00640 1 --2- 172.21.15.94:0/1868858201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6e40b1100 0x7fe6e40c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6e3400640 1 --2- 172.21.15.94:0/1868858201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e40ce890 0x7fe6e4155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6dbe00640 1 -- 172.21.15.94:0/1868858201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e40ce890 msgr2=0x7fe6e4155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6dbe00640 1 --2- 172.21.15.94:0/1868858201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e40ce890 0x7fe6e4155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6dbe00640 1 -- 172.21.15.94:0/1868858201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6e40b1100 msgr2=0x7fe6e40c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6dbe00640 1 --2- 172.21.15.94:0/1868858201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6e40b1100 0x7fe6e40c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6dbe00640 1 -- 172.21.15.94:0/1868858201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e40b73b0 con 0x7fe6e40ce1e0 2024-09-10T23:39:37.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6dbe00640 1 --2- 172.21.15.94:0/1868858201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6e40ce1e0 0x7fe6e41557f0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc01ccb0 tx=0x7fe6cc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6daa00640 1 -- 172.21.15.94:0/1868858201 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc0be070 con 0x7fe6e40ce1e0 2024-09-10T23:39:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6daa00640 1 -- 172.21.15.94:0/1868858201 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6cc005250 con 0x7fe6e40ce1e0 2024-09-10T23:39:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6e9800640 1 -- 172.21.15.94:0/1868858201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e40b7690 con 0x7fe6e40ce1e0 2024-09-10T23:39:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6daa00640 1 -- 172.21.15.94:0/1868858201 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc005660 con 0x7fe6e40ce1e0 2024-09-10T23:39:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.648+0000 7fe6e9800640 1 -- 172.21.15.94:0/1868858201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e40cb4f0 con 0x7fe6e40ce1e0 2024-09-10T23:39:37.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.652+0000 7fe6e9800640 1 -- 172.21.15.94:0/1868858201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e4157ff0 con 0x7fe6e40ce1e0 2024-09-10T23:39:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.652+0000 7fe6daa00640 1 -- 172.21.15.94:0/1868858201 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 22) ==== 97006+0+0 (secure 0 0 0) 0x7fe6cc0a66e0 con 0x7fe6e40ce1e0 2024-09-10T23:39:37.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.652+0000 7fe6daa00640 1 --2- 172.21.15.94:0/1868858201 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe6c0072ca0 0x7fe6c0075160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:37.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.652+0000 7fe6daa00640 1 -- 172.21.15.94:0/1868858201 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5737+0+0 (secure 0 0 0) 0x7fe6cc12dc10 con 0x7fe6e40ce1e0 2024-09-10T23:39:37.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.652+0000 7fe6e2a00640 1 --2- 172.21.15.94:0/1868858201 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe6c0072ca0 0x7fe6c0075160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:37.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.652+0000 7fe6e2a00640 1 --2- 172.21.15.94:0/1868858201 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe6c0072ca0 0x7fe6c0075160 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4007fd0 tx=0x7fe6d4009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:37.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.656+0000 7fe6daa00640 1 -- 172.21.15.94:0/1868858201 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe6cc0ac030 con 0x7fe6e40ce1e0 2024-09-10T23:39:37.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.852+0000 7fe6e9800640 1 -- 172.21.15.94:0/1868858201 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol74910", "target": ["mon-mgr", ""]}) -- 0x7fe6e40cd710 con 0x7fe6c0072ca0 2024-09-10T23:39:37.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.856+0000 7fe6daa00640 1 -- 172.21.15.94:0/1868858201 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7fe6e40cd710 con 0x7fe6c0072ca0 2024-09-10T23:39:37.860 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/_nogroup/subvol74910 2024-09-10T23:39:37.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.860+0000 7fe6e9800640 1 -- 172.21.15.94:0/1868858201 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe6c0072ca0 msgr2=0x7fe6c0075160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:37.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.860+0000 7fe6e9800640 1 --2- 172.21.15.94:0/1868858201 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe6c0072ca0 0x7fe6c0075160 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4007fd0 tx=0x7fe6d4009040 comp rx=0 tx=0).stop 2024-09-10T23:39:37.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.860+0000 7fe6e9800640 1 -- 172.21.15.94:0/1868858201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6e40ce1e0 msgr2=0x7fe6e41557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:37.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.860+0000 7fe6e9800640 1 --2- 172.21.15.94:0/1868858201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6e40ce1e0 0x7fe6e41557f0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc01ccb0 tx=0x7fe6cc004610 comp rx=0 tx=0).stop 2024-09-10T23:39:37.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.860+0000 7fe6e9800640 1 -- 172.21.15.94:0/1868858201 shutdown_connections 2024-09-10T23:39:37.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.860+0000 7fe6e9800640 1 --2- 172.21.15.94:0/1868858201 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe6c0072ca0 0x7fe6c0075160 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.860+0000 7fe6e9800640 1 --2- 172.21.15.94:0/1868858201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e40ce890 0x7fe6e4155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.860+0000 7fe6e9800640 1 --2- 172.21.15.94:0/1868858201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6e40ce1e0 0x7fe6e41557f0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.860+0000 7fe6e9800640 1 --2- 172.21.15.94:0/1868858201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6e40b1100 0x7fe6e40c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:37.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.860+0000 7fe6e9800640 1 -- 172.21.15.94:0/1868858201 >> 172.21.15.94:0/1868858201 conn(0x7fe6e40d1c60 msgr2=0x7fe6e40ae870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:37.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.860+0000 7fe6e9800640 1 -- 172.21.15.94:0/1868858201 shutdown_connections 2024-09-10T23:39:37.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:37.860+0000 7fe6e9800640 1 -- 172.21.15.94:0/1868858201 wait complete. 2024-09-10T23:39:37.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:37.882 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/9b39e40f8f12e672df575ba9cfefcacd.meta) 2024-09-10T23:39:41.125 INFO:teuthology.orchestra.run.smithi094.stdout:[GLOBAL] 2024-09-10T23:39:41.125 INFO:teuthology.orchestra.run.smithi094.stdout:version = 1 2024-09-10T23:39:41.125 INFO:teuthology.orchestra.run.smithi094.stdout:type = subvolume 2024-09-10T23:39:41.125 INFO:teuthology.orchestra.run.smithi094.stdout:path = /volumes/_nogroup/subvol74910 2024-09-10T23:39:41.126 INFO:teuthology.orchestra.run.smithi094.stdout:state = complete 2024-09-10T23:39:41.126 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:41.128 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol37321 subvol_grp85160 2024-09-10T23:39:41.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.280+0000 7f4242600640 1 -- 172.21.15.94:0/3079694054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f423c161b00 msgr2=0x7f423c161f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:41.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.280+0000 7f4242600640 1 --2- 172.21.15.94:0/3079694054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f423c161b00 0x7f423c161f40 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f4228009900 tx=0x7f422801c8e0 comp rx=0 tx=0).stop 2024-09-10T23:39:41.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.280+0000 7f4242600640 1 -- 172.21.15.94:0/3079694054 shutdown_connections 2024-09-10T23:39:41.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.280+0000 7f4242600640 1 --2- 172.21.15.94:0/3079694054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f423c161b00 0x7f423c161f40 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:41.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.280+0000 7f4242600640 1 --2- 172.21.15.94:0/3079694054 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f423c15e120 0x7f423c0d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:41.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.280+0000 7f4242600640 1 -- 172.21.15.94:0/3079694054 >> 172.21.15.94:0/3079694054 conn(0x7f423c0d3e90 msgr2=0x7f423c0d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:41.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.280+0000 7f4242600640 1 -- 172.21.15.94:0/3079694054 shutdown_connections 2024-09-10T23:39:41.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f4242600640 1 -- 172.21.15.94:0/3079694054 wait complete. 2024-09-10T23:39:41.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f4242600640 1 Processor -- start 2024-09-10T23:39:41.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f4242600640 1 -- start start 2024-09-10T23:39:41.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f4242600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f423c161b00 0x7f423c152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:41.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f4242600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f423c162480 0x7f423c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:41.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f4242600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f423c0b8fa0 0x7f423c15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:41.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f4242600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f423c16b130 con 0x7f423c162480 2024-09-10T23:39:41.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f4242600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f423c16afb0 con 0x7f423c0b8fa0 2024-09-10T23:39:41.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f4242600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f423c16b2b0 con 0x7f423c161b00 2024-09-10T23:39:41.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f423aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f423c162480 0x7f423c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:41.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f423aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f423c162480 0x7f423c0b8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55110/0 (socket says 172.21.15.94:55110) 2024-09-10T23:39:41.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f423aa00640 1 -- 172.21.15.94:0/3008981356 learned_addr learned my addr 172.21.15.94:0/3008981356 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:41.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f423b400640 1 --2- 172.21.15.94:0/3008981356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f423c161b00 0x7f423c152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:41.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f423be00640 1 --2- 172.21.15.94:0/3008981356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f423c0b8fa0 0x7f423c15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:41.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f423aa00640 1 -- 172.21.15.94:0/3008981356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f423c161b00 msgr2=0x7f423c152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:41.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f423aa00640 1 --2- 172.21.15.94:0/3008981356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f423c161b00 0x7f423c152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:41.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f423aa00640 1 -- 172.21.15.94:0/3008981356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f423c0b8fa0 msgr2=0x7f423c15a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:41.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f423aa00640 1 --2- 172.21.15.94:0/3008981356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f423c0b8fa0 0x7f423c15a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:41.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f423aa00640 1 -- 172.21.15.94:0/3008981356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423c1fc020 con 0x7f423c162480 2024-09-10T23:39:41.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f423aa00640 1 --2- 172.21.15.94:0/3008981356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f423c162480 0x7f423c0b8a60 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f42280b0040 tx=0x7f4228004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:41.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f4233400640 1 -- 172.21.15.94:0/3008981356 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42280b2040 con 0x7f423c162480 2024-09-10T23:39:41.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f4233400640 1 -- 172.21.15.94:0/3008981356 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4228005120 con 0x7f423c162480 2024-09-10T23:39:41.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f4242600640 1 -- 172.21.15.94:0/3008981356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423c15b2b0 con 0x7f423c162480 2024-09-10T23:39:41.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f4233400640 1 -- 172.21.15.94:0/3008981356 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228005570 con 0x7f423c162480 2024-09-10T23:39:41.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.284+0000 7f4242600640 1 -- 172.21.15.94:0/3008981356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423c1fb540 con 0x7f423c162480 2024-09-10T23:39:41.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.288+0000 7f4242600640 1 -- 172.21.15.94:0/3008981356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f423c0b93b0 con 0x7f423c162480 2024-09-10T23:39:41.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.288+0000 7f4233400640 1 -- 172.21.15.94:0/3008981356 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 97057+0+0 (secure 0 0 0) 0x7f42280046d0 con 0x7f423c162480 2024-09-10T23:39:41.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.288+0000 7f4233400640 1 --2- 172.21.15.94:0/3008981356 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f421c072d30 0x7f421c0751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:41.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.288+0000 7f4233400640 1 -- 172.21.15.94:0/3008981356 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5824+0+0 (secure 0 0 0) 0x7f422812de20 con 0x7f423c162480 2024-09-10T23:39:41.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.288+0000 7f423b400640 1 --2- 172.21.15.94:0/3008981356 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f421c072d30 0x7f421c0751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:41.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.288+0000 7f423b400640 1 --2- 172.21.15.94:0/3008981356 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f421c072d30 0x7f421c0751f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f422c00abe0 tx=0x7f422c00a2d0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:41.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.292+0000 7f4233400640 1 -- 172.21.15.94:0/3008981356 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f423c0b93b0 con 0x7f423c162480 2024-09-10T23:39:41.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.484+0000 7f4242600640 1 -- 172.21.15.94:0/3008981356 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol37321", "group_name": "subvol_grp85160", "target": ["mon-mgr", ""]}) -- 0x7f423c1616c0 con 0x7f421c072d30 2024-09-10T23:39:41.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.488+0000 7f4233400640 1 -- 172.21.15.94:0/3008981356 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f423c1616c0 con 0x7f421c072d30 2024-09-10T23:39:41.492 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/subvol_grp85160/subvol37321 2024-09-10T23:39:41.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.492+0000 7f4242600640 1 -- 172.21.15.94:0/3008981356 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f421c072d30 msgr2=0x7f421c0751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:41.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.492+0000 7f4242600640 1 --2- 172.21.15.94:0/3008981356 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f421c072d30 0x7f421c0751f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f422c00abe0 tx=0x7f422c00a2d0 comp rx=0 tx=0).stop 2024-09-10T23:39:41.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.492+0000 7f4242600640 1 -- 172.21.15.94:0/3008981356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f423c162480 msgr2=0x7f423c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:41.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.492+0000 7f4242600640 1 --2- 172.21.15.94:0/3008981356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f423c162480 0x7f423c0b8a60 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f42280b0040 tx=0x7f4228004220 comp rx=0 tx=0).stop 2024-09-10T23:39:41.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.492+0000 7f4242600640 1 -- 172.21.15.94:0/3008981356 shutdown_connections 2024-09-10T23:39:41.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.492+0000 7f4242600640 1 --2- 172.21.15.94:0/3008981356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f423c0b8fa0 0x7f423c15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:41.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.492+0000 7f4242600640 1 --2- 172.21.15.94:0/3008981356 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f421c072d30 0x7f421c0751f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:41.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.492+0000 7f4242600640 1 --2- 172.21.15.94:0/3008981356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f423c162480 0x7f423c0b8a60 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:41.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.492+0000 7f4242600640 1 --2- 172.21.15.94:0/3008981356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f423c161b00 0x7f423c152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:41.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.492+0000 7f4242600640 1 -- 172.21.15.94:0/3008981356 >> 172.21.15.94:0/3008981356 conn(0x7f423c0d3e90 msgr2=0x7f423c15bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:41.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.492+0000 7f4242600640 1 -- 172.21.15.94:0/3008981356 shutdown_connections 2024-09-10T23:39:41.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:41.492+0000 7f4242600640 1 -- 172.21.15.94:0/3008981356 wait complete. 2024-09-10T23:39:41.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:41.513 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/26ffb692303e89354b017166ef6694af.meta) 2024-09-10T23:39:46.125 INFO:teuthology.orchestra.run.smithi094.stdout:[GLOBAL] 2024-09-10T23:39:46.125 INFO:teuthology.orchestra.run.smithi094.stdout:version = 1 2024-09-10T23:39:46.125 INFO:teuthology.orchestra.run.smithi094.stdout:type = subvolume 2024-09-10T23:39:46.125 INFO:teuthology.orchestra.run.smithi094.stdout:path = /volumes/subvol_grp85160/subvol37321 2024-09-10T23:39:46.125 INFO:teuthology.orchestra.run.smithi094.stdout:state = complete 2024-09-10T23:39:46.126 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:46.128 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol74910 2024-09-10T23:39:46.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.279+0000 7fb3a1000640 1 -- 172.21.15.94:0/1363309252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb380003f80 msgr2=0x7fb380004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:46.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.279+0000 7fb3a1000640 1 --2- 172.21.15.94:0/1363309252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb380003f80 0x7fb380004810 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fb384001740 tx=0x7fb38401efb0 comp rx=0 tx=0).stop 2024-09-10T23:39:46.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.279+0000 7fb39a800640 1 -- 172.21.15.94:0/1363309252 reap_dead start 2024-09-10T23:39:46.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.279+0000 7fb3a1000640 1 -- 172.21.15.94:0/1363309252 shutdown_connections 2024-09-10T23:39:46.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.279+0000 7fb3a1000640 1 -- 172.21.15.94:0/1363309252 >> 172.21.15.94:0/1363309252 conn(0x7fb39c0cfaf0 msgr2=0x7fb39c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:46.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.279+0000 7fb3a1000640 1 -- 172.21.15.94:0/1363309252 shutdown_connections 2024-09-10T23:39:46.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.279+0000 7fb3a1000640 1 -- 172.21.15.94:0/1363309252 wait complete. 2024-09-10T23:39:46.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.279+0000 7fb3a1000640 1 Processor -- start 2024-09-10T23:39:46.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.279+0000 7fb3a1000640 1 -- start start 2024-09-10T23:39:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.279+0000 7fb3a1000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb380003f80 0x7fb39c14d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.279+0000 7fb3a1000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb39c0b2b60 0x7fb39c1412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.279+0000 7fb3a1000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c0c7e50 0x7fb39c0c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.279+0000 7fb3a1000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c163c80 con 0x7fb39c0c7e50 2024-09-10T23:39:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.279+0000 7fb3a1000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c163b00 con 0x7fb380003f80 2024-09-10T23:39:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.279+0000 7fb3a1000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb39c163e00 con 0x7fb39c0b2b60 2024-09-10T23:39:46.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb39a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c0c7e50 0x7fb39c0c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:46.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb39a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c0c7e50 0x7fb39c0c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52892/0 (socket says 172.21.15.94:52892) 2024-09-10T23:39:46.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb39a800640 1 -- 172.21.15.94:0/161700826 learned_addr learned my addr 172.21.15.94:0/161700826 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:46.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb39a800640 1 -- 172.21.15.94:0/161700826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb39c0b2b60 msgr2=0x7fb39c1412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:46.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb39a800640 1 --2- 172.21.15.94:0/161700826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb39c0b2b60 0x7fb39c1412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:46.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb399e00640 1 --2- 172.21.15.94:0/161700826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb380003f80 0x7fb39c14d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:46.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb39a800640 1 -- 172.21.15.94:0/161700826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb380003f80 msgr2=0x7fb39c14d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:46.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb39a800640 1 --2- 172.21.15.94:0/161700826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb380003f80 0x7fb39c14d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:46.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb39a800640 1 -- 172.21.15.94:0/161700826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39c0c79b0 con 0x7fb39c0c7e50 2024-09-10T23:39:46.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb399e00640 1 --2- 172.21.15.94:0/161700826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb380003f80 0x7fb39c14d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:46.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb39a800640 1 --2- 172.21.15.94:0/161700826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c0c7e50 0x7fb39c0c82d0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fb394011b20 tx=0x7fb394009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:46.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb392a00640 1 -- 172.21.15.94:0/161700826 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39400fce0 con 0x7fb39c0c7e50 2024-09-10T23:39:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb392a00640 1 -- 172.21.15.94:0/161700826 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb39400fe80 con 0x7fb39c0c7e50 2024-09-10T23:39:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb392a00640 1 -- 172.21.15.94:0/161700826 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394004d70 con 0x7fb39c0c7e50 2024-09-10T23:39:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb3a1000640 1 -- 172.21.15.94:0/161700826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39c0b2fe0 con 0x7fb39c0c7e50 2024-09-10T23:39:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb3a1000640 1 -- 172.21.15.94:0/161700826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb39c0b57d0 con 0x7fb39c0c7e50 2024-09-10T23:39:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb3a1000640 1 -- 172.21.15.94:0/161700826 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb39c0cadc0 con 0x7fb39c0c7e50 2024-09-10T23:39:46.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.283+0000 7fb392a00640 1 -- 172.21.15.94:0/161700826 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 97057+0+0 (secure 0 0 0) 0x7fb394019410 con 0x7fb39c0c7e50 2024-09-10T23:39:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.287+0000 7fb392a00640 1 --2- 172.21.15.94:0/161700826 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb378072d30 0x7fb3780751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.287+0000 7fb392a00640 1 -- 172.21.15.94:0/161700826 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5824+0+0 (secure 0 0 0) 0x7fb394094f70 con 0x7fb39c0c7e50 2024-09-10T23:39:46.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.287+0000 7fb399e00640 1 --2- 172.21.15.94:0/161700826 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb378072d30 0x7fb3780751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:46.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.287+0000 7fb399e00640 1 --2- 172.21.15.94:0/161700826 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb378072d30 0x7fb3780751f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb38c001890 tx=0x7fb38c000f00 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:46.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.291+0000 7fb392a00640 1 -- 172.21.15.94:0/161700826 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb39c0b2fe0 con 0x7fb39c0c7e50 2024-09-10T23:39:46.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.479+0000 7fb3a1000640 1 -- 172.21.15.94:0/161700826 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol74910", "target": ["mon-mgr", ""]}) -- 0x7fb39c0d0850 con 0x7fb378072d30 2024-09-10T23:39:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.495+0000 7fb392a00640 1 -- 172.21.15.94:0/161700826 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb39c0d0850 con 0x7fb378072d30 2024-09-10T23:39:46.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.499+0000 7fb3a1000640 1 -- 172.21.15.94:0/161700826 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb378072d30 msgr2=0x7fb3780751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:46.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.499+0000 7fb3a1000640 1 --2- 172.21.15.94:0/161700826 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb378072d30 0x7fb3780751f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb38c001890 tx=0x7fb38c000f00 comp rx=0 tx=0).stop 2024-09-10T23:39:46.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.499+0000 7fb3a1000640 1 -- 172.21.15.94:0/161700826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c0c7e50 msgr2=0x7fb39c0c82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:46.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.499+0000 7fb3a1000640 1 --2- 172.21.15.94:0/161700826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c0c7e50 0x7fb39c0c82d0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fb394011b20 tx=0x7fb394009200 comp rx=0 tx=0).stop 2024-09-10T23:39:46.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.499+0000 7fb3a1000640 1 -- 172.21.15.94:0/161700826 shutdown_connections 2024-09-10T23:39:46.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.499+0000 7fb3a1000640 1 --2- 172.21.15.94:0/161700826 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb378072d30 0x7fb3780751f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:46.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.499+0000 7fb3a1000640 1 --2- 172.21.15.94:0/161700826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb39c0c7e50 0x7fb39c0c82d0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:46.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.499+0000 7fb3a1000640 1 --2- 172.21.15.94:0/161700826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb39c0b2b60 0x7fb39c1412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:46.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.499+0000 7fb3a1000640 1 --2- 172.21.15.94:0/161700826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb380003f80 0x7fb39c14d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:46.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.499+0000 7fb3a1000640 1 -- 172.21.15.94:0/161700826 >> 172.21.15.94:0/161700826 conn(0x7fb39c0cfaf0 msgr2=0x7fb39c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:46.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.499+0000 7fb3a1000640 1 -- 172.21.15.94:0/161700826 shutdown_connections 2024-09-10T23:39:46.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.499+0000 7fb3a1000640 1 -- 172.21.15.94:0/161700826 wait complete. 2024-09-10T23:39:46.503 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:46.499+0000 7f9ea3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:46.503 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:46.499+0000 7f9ea3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:46.503 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:46.499+0000 7f9ea3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:46.503 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:46.499+0000 7f9ea3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:46.503 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:46.499+0000 7f9ea3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:46.519 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:46.515+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:46.519 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:46.515+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:46.519 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:46.515+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:46.519 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:46.515+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:46.519 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:39:46.515+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:39:46.526 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol37321 subvol_grp85160 2024-09-10T23:39:46.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed10600640 1 -- 172.21.15.94:0/1990873122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed08070150 msgr2=0x7fed08072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed10600640 1 --2- 172.21.15.94:0/1990873122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed08070150 0x7fed08072610 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fed0000acd0 tx=0x7fed0001de60 comp rx=0 tx=0).stop 2024-09-10T23:39:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed10600640 1 -- 172.21.15.94:0/1990873122 shutdown_connections 2024-09-10T23:39:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed10600640 1 --2- 172.21.15.94:0/1990873122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed08070150 0x7fed08072610 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed10600640 1 -- 172.21.15.94:0/1990873122 >> 172.21.15.94:0/1990873122 conn(0x7fed0806cdf0 msgr2=0x7fed0806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed10600640 1 -- 172.21.15.94:0/1990873122 shutdown_connections 2024-09-10T23:39:46.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed10600640 1 -- 172.21.15.94:0/1990873122 wait complete. 2024-09-10T23:39:46.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed10600640 1 Processor -- start 2024-09-10T23:39:46.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed10600640 1 -- start start 2024-09-10T23:39:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed10600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed0806f600 0x7fed08165810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed10600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08070150 0x7fed08159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed10600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed08157c60 0x7fed081fb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed10600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fed08168380 con 0x7fed0806f600 2024-09-10T23:39:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed10600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fed08168200 con 0x7fed08157c60 2024-09-10T23:39:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed10600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fed08168500 con 0x7fed08070150 2024-09-10T23:39:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed0dc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed0806f600 0x7fed08165810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed0dc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed0806f600 0x7fed08165810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52954/0 (socket says 172.21.15.94:52954) 2024-09-10T23:39:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed0dc00640 1 -- 172.21.15.94:0/568245140 learned_addr learned my addr 172.21.15.94:0/568245140 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed0e600640 1 --2- 172.21.15.94:0/568245140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed08157c60 0x7fed081fb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed0d200640 1 --2- 172.21.15.94:0/568245140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08070150 0x7fed08159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:46.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed0dc00640 1 -- 172.21.15.94:0/568245140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08070150 msgr2=0x7fed08159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed0dc00640 1 --2- 172.21.15.94:0/568245140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08070150 0x7fed08159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed0dc00640 1 -- 172.21.15.94:0/568245140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed08157c60 msgr2=0x7fed081fb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed0dc00640 1 --2- 172.21.15.94:0/568245140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed08157c60 0x7fed081fb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed0dc00640 1 -- 172.21.15.94:0/568245140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed080c4940 con 0x7fed0806f600 2024-09-10T23:39:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.671+0000 7fed0dc00640 1 --2- 172.21.15.94:0/568245140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed0806f600 0x7fed08165810 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c400 tx=0x7fecfc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:46.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.675+0000 7fed05600640 1 -- 172.21.15.94:0/568245140 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc017070 con 0x7fed0806f600 2024-09-10T23:39:46.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.675+0000 7fed05600640 1 -- 172.21.15.94:0/568245140 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecfc003600 con 0x7fed0806f600 2024-09-10T23:39:46.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.675+0000 7fed05600640 1 -- 172.21.15.94:0/568245140 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc003a50 con 0x7fed0806f600 2024-09-10T23:39:46.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.675+0000 7fed10600640 1 -- 172.21.15.94:0/568245140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed080c4c00 con 0x7fed0806f600 2024-09-10T23:39:46.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.675+0000 7fed10600640 1 -- 172.21.15.94:0/568245140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed08159140 con 0x7fed0806f600 2024-09-10T23:39:46.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.675+0000 7fed10600640 1 -- 172.21.15.94:0/568245140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed08072610 con 0x7fed0806f600 2024-09-10T23:39:46.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.675+0000 7fed05600640 1 -- 172.21.15.94:0/568245140 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 97057+0+0 (secure 0 0 0) 0x7fecfc003bf0 con 0x7fed0806f600 2024-09-10T23:39:46.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.675+0000 7fed05600640 1 --2- 172.21.15.94:0/568245140 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fece8072d30 0x7fece80751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:46.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.679+0000 7fed05600640 1 -- 172.21.15.94:0/568245140 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5824+0+0 (secure 0 0 0) 0x7fecfc093a20 con 0x7fed0806f600 2024-09-10T23:39:46.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.679+0000 7fed0d200640 1 --2- 172.21.15.94:0/568245140 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fece8072d30 0x7fece80751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:46.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.679+0000 7fed0d200640 1 --2- 172.21.15.94:0/568245140 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fece8072d30 0x7fece80751f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fed08156de0 tx=0x7fecf4009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:46.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.679+0000 7fed05600640 1 -- 172.21.15.94:0/568245140 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fecfc05dac0 con 0x7fed0806f600 2024-09-10T23:39:46.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.879+0000 7fed10600640 1 -- 172.21.15.94:0/568245140 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol37321", "group_name": "subvol_grp85160", "target": ["mon-mgr", ""]}) -- 0x7fed081557e0 con 0x7fece8072d30 2024-09-10T23:39:46.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.895+0000 7fed05600640 1 -- 172.21.15.94:0/568245140 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fed081557e0 con 0x7fece8072d30 2024-09-10T23:39:46.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.895+0000 7fed10600640 1 -- 172.21.15.94:0/568245140 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fece8072d30 msgr2=0x7fece80751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:46.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.895+0000 7fed10600640 1 --2- 172.21.15.94:0/568245140 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fece8072d30 0x7fece80751f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fed08156de0 tx=0x7fecf4009040 comp rx=0 tx=0).stop 2024-09-10T23:39:46.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.895+0000 7fed10600640 1 -- 172.21.15.94:0/568245140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed0806f600 msgr2=0x7fed08165810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:46.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.895+0000 7fed10600640 1 --2- 172.21.15.94:0/568245140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed0806f600 0x7fed08165810 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c400 tx=0x7fecfc00c8c0 comp rx=0 tx=0).stop 2024-09-10T23:39:46.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.899+0000 7fed10600640 1 -- 172.21.15.94:0/568245140 shutdown_connections 2024-09-10T23:39:46.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.899+0000 7fed10600640 1 --2- 172.21.15.94:0/568245140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed08157c60 0x7fed081fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:46.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.899+0000 7fed10600640 1 --2- 172.21.15.94:0/568245140 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fece8072d30 0x7fece80751f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:46.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.899+0000 7fed10600640 1 --2- 172.21.15.94:0/568245140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed08070150 0x7fed08159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:46.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.899+0000 7fed10600640 1 --2- 172.21.15.94:0/568245140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed0806f600 0x7fed08165810 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:46.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.899+0000 7fed10600640 1 -- 172.21.15.94:0/568245140 >> 172.21.15.94:0/568245140 conn(0x7fed0806cdf0 msgr2=0x7fed08156390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:46.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.899+0000 7fed10600640 1 -- 172.21.15.94:0/568245140 shutdown_connections 2024-09-10T23:39:46.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46.899+0000 7fed10600640 1 -- 172.21.15.94:0/568245140 wait complete. 2024-09-10T23:39:46.923 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T23:39:46.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T23:39:46.976 INFO:teuthology.orchestra.run.smithi094.stdout:2 2024-09-10T23:39:46.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:39:46.982 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp85160 2024-09-10T23:39:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.123+0000 7f8dae000640 1 -- 172.21.15.94:0/2913334637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da80af150 msgr2=0x7f8da80af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.123+0000 7f8dae000640 1 --2- 172.21.15.94:0/2913334637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da80af150 0x7f8da80af5e0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f8da8072eb0 tx=0x7f8da001cd50 comp rx=0 tx=0).stop 2024-09-10T23:39:47.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.123+0000 7f8dae000640 1 -- 172.21.15.94:0/2913334637 shutdown_connections 2024-09-10T23:39:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.123+0000 7f8dae000640 1 --2- 172.21.15.94:0/2913334637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da80af150 0x7f8da80af5e0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.123+0000 7f8dae000640 1 --2- 172.21.15.94:0/2913334637 >> v2:172.21.15.181:3300/0 conn(0x7f8da80b2ac0 0x7f8da80b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.123+0000 7f8dae000640 1 -- 172.21.15.94:0/2913334637 >> 172.21.15.94:0/2913334637 conn(0x7f8da80d1b00 msgr2=0x7f8da80d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.123+0000 7f8dae000640 1 -- 172.21.15.94:0/2913334637 shutdown_connections 2024-09-10T23:39:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.123+0000 7f8dae000640 1 -- 172.21.15.94:0/2913334637 wait complete. 2024-09-10T23:39:47.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.123+0000 7f8dae000640 1 Processor -- start 2024-09-10T23:39:47.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.123+0000 7f8dae000640 1 -- start start 2024-09-10T23:39:47.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dae000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8da80af150 0x7f8da814d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:47.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dae000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8da80b2ac0 0x7f8da806eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:47.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dae000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da815f140 0x7f8da81581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:47.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dae000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8168ad0 con 0x7f8da815f140 2024-09-10T23:39:47.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dae000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8168950 con 0x7f8da80af150 2024-09-10T23:39:47.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dae000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8da8168c50 con 0x7f8da80b2ac0 2024-09-10T23:39:47.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dad600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da815f140 0x7f8da81581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dad600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da815f140 0x7f8da81581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52972/0 (socket says 172.21.15.94:52972) 2024-09-10T23:39:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dad600640 1 -- 172.21.15.94:0/3689925246 learned_addr learned my addr 172.21.15.94:0/3689925246 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dacc00640 1 --2- 172.21.15.94:0/3689925246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8da80af150 0x7f8da814d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8da7e00640 1 --2- 172.21.15.94:0/3689925246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8da80b2ac0 0x7f8da806eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dad600640 1 -- 172.21.15.94:0/3689925246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8da80b2ac0 msgr2=0x7f8da806eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dad600640 1 --2- 172.21.15.94:0/3689925246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8da80b2ac0 0x7f8da806eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dad600640 1 -- 172.21.15.94:0/3689925246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8da80af150 msgr2=0x7f8da814d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dad600640 1 --2- 172.21.15.94:0/3689925246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8da80af150 0x7f8da814d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:47.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dad600640 1 -- 172.21.15.94:0/3689925246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da814e1f0 con 0x7f8da815f140 2024-09-10T23:39:47.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dad600640 1 --2- 172.21.15.94:0/3689925246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da815f140 0x7f8da81581b0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f8da001b400 tx=0x7f8da0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:47.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8da4c00640 1 -- 172.21.15.94:0/3689925246 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da00bb070 con 0x7f8da815f140 2024-09-10T23:39:47.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8da4c00640 1 -- 172.21.15.94:0/3689925246 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8da0004e00 con 0x7f8da815f140 2024-09-10T23:39:47.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8da4c00640 1 -- 172.21.15.94:0/3689925246 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da00b2040 con 0x7f8da815f140 2024-09-10T23:39:47.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dae000640 1 -- 172.21.15.94:0/3689925246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da8156c50 con 0x7f8da815f140 2024-09-10T23:39:47.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dae000640 1 -- 172.21.15.94:0/3689925246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da8157210 con 0x7f8da815f140 2024-09-10T23:39:47.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.127+0000 7f8dae000640 1 -- 172.21.15.94:0/3689925246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da80af5e0 con 0x7f8da815f140 2024-09-10T23:39:47.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.131+0000 7f8da4c00640 1 -- 172.21.15.94:0/3689925246 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 97057+0+0 (secure 0 0 0) 0x7f8da00b5030 con 0x7f8da815f140 2024-09-10T23:39:47.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.131+0000 7f8da4c00640 1 --2- 172.21.15.94:0/3689925246 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8d88072d30 0x7f8d880751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:47.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.131+0000 7f8da4c00640 1 -- 172.21.15.94:0/3689925246 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5824+0+0 (secure 0 0 0) 0x7f8da012df10 con 0x7f8da815f140 2024-09-10T23:39:47.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.131+0000 7f8dacc00640 1 --2- 172.21.15.94:0/3689925246 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8d88072d30 0x7f8d880751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:47.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.131+0000 7f8dacc00640 1 --2- 172.21.15.94:0/3689925246 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8d88072d30 0x7f8d880751f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c00b810 tx=0x7f8d9c008040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:47.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.135+0000 7f8da4c00640 1 -- 172.21.15.94:0/3689925246 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8da814e1f0 con 0x7f8da815f140 2024-09-10T23:39:47.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.319+0000 7f8dae000640 1 -- 172.21.15.94:0/3689925246 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp85160", "target": ["mon-mgr", ""]}) -- 0x7f8da815e8b0 con 0x7f8d88072d30 2024-09-10T23:39:47.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.327+0000 7f8da4c00640 1 -- 172.21.15.94:0/3689925246 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8da815e8b0 con 0x7f8d88072d30 2024-09-10T23:39:47.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.327+0000 7f8dae000640 1 -- 172.21.15.94:0/3689925246 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8d88072d30 msgr2=0x7f8d880751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.327+0000 7f8dae000640 1 --2- 172.21.15.94:0/3689925246 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8d88072d30 0x7f8d880751f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c00b810 tx=0x7f8d9c008040 comp rx=0 tx=0).stop 2024-09-10T23:39:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.327+0000 7f8dae000640 1 -- 172.21.15.94:0/3689925246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da815f140 msgr2=0x7f8da81581b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.327+0000 7f8dae000640 1 --2- 172.21.15.94:0/3689925246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da815f140 0x7f8da81581b0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f8da001b400 tx=0x7f8da0003f60 comp rx=0 tx=0).stop 2024-09-10T23:39:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.327+0000 7f8dae000640 1 -- 172.21.15.94:0/3689925246 shutdown_connections 2024-09-10T23:39:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.327+0000 7f8dae000640 1 --2- 172.21.15.94:0/3689925246 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8d88072d30 0x7f8d880751f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.327+0000 7f8dae000640 1 --2- 172.21.15.94:0/3689925246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da815f140 0x7f8da81581b0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.327+0000 7f8dae000640 1 --2- 172.21.15.94:0/3689925246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8da80b2ac0 0x7f8da806eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.327+0000 7f8dae000640 1 --2- 172.21.15.94:0/3689925246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8da80af150 0x7f8da814d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.327+0000 7f8dae000640 1 -- 172.21.15.94:0/3689925246 >> 172.21.15.94:0/3689925246 conn(0x7f8da80d1b00 msgr2=0x7f8da80cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.327+0000 7f8dae000640 1 -- 172.21.15.94:0/3689925246 shutdown_connections 2024-09-10T23:39:47.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.327+0000 7f8dae000640 1 -- 172.21.15.94:0/3689925246 wait complete. 2024-09-10T23:39:47.351 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:39:47.351 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:39:47.431 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:39:47.539 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:39:47.539 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:39:47.539 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:39:47.539 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:47.548 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:39:47.548 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:47.597 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:39:47.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.779+0000 7f0c9be00640 1 -- 172.21.15.94:0/724803518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c80004010 msgr2=0x7f0c80004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:47.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.779+0000 7f0c9be00640 1 --2- 172.21.15.94:0/724803518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c80004010 0x7f0c80004810 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f0c84009b30 tx=0x7f0c8401d230 comp rx=0 tx=0).stop 2024-09-10T23:39:47.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.779+0000 7f0c9b400640 1 -- 172.21.15.94:0/724803518 reap_dead start 2024-09-10T23:39:47.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.779+0000 7f0c9be00640 1 -- 172.21.15.94:0/724803518 shutdown_connections 2024-09-10T23:39:47.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.779+0000 7f0c9be00640 1 -- 172.21.15.94:0/724803518 >> 172.21.15.94:0/724803518 conn(0x7f0c9c0b2490 msgr2=0x7f0c9c0b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:47.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.779+0000 7f0c9be00640 1 -- 172.21.15.94:0/724803518 shutdown_connections 2024-09-10T23:39:47.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9be00640 1 -- 172.21.15.94:0/724803518 wait complete. 2024-09-10T23:39:47.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9be00640 1 Processor -- start 2024-09-10T23:39:47.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9be00640 1 -- start start 2024-09-10T23:39:47.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c80004010 0x7f0c9c0c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:47.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9be00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c9c15cde0 0x7f0c9c0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:47.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c9c0cdb40 0x7f0c9c0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:47.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c06cda0 con 0x7f0c9c0cdb40 2024-09-10T23:39:47.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9be00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c06cc20 con 0x7f0c80004010 2024-09-10T23:39:47.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9be00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0c9c06cf20 con 0x7f0c9c15cde0 2024-09-10T23:39:47.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c9c0cdb40 0x7f0c9c0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:47.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c80004010 0x7f0c9c0c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:47.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c92000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c9c15cde0 0x7f0c9c0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:47.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c9c0cdb40 0x7f0c9c0b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52992/0 (socket says 172.21.15.94:52992) 2024-09-10T23:39:47.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9b400640 1 -- 172.21.15.94:0/505244751 learned_addr learned my addr 172.21.15.94:0/505244751 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:47.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9b400640 1 -- 172.21.15.94:0/505244751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c9c15cde0 msgr2=0x7f0c9c0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:47.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9b400640 1 --2- 172.21.15.94:0/505244751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c9c15cde0 0x7f0c9c0cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:47.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9b400640 1 -- 172.21.15.94:0/505244751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c80004010 msgr2=0x7f0c9c0c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:47.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9b400640 1 --2- 172.21.15.94:0/505244751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c80004010 0x7f0c9c0c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:47.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9b400640 1 -- 172.21.15.94:0/505244751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9c0c7ab0 con 0x7f0c9c0cdb40 2024-09-10T23:39:47.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9b400640 1 --2- 172.21.15.94:0/505244751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c9c0cdb40 0x7f0c9c0b47e0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f0c940013e0 tx=0x7f0c94011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:47.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c93400640 1 -- 172.21.15.94:0/505244751 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c94017070 con 0x7f0c9c0cdb40 2024-09-10T23:39:47.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c93400640 1 -- 172.21.15.94:0/505244751 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c94013ad0 con 0x7f0c9c0cdb40 2024-09-10T23:39:47.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c93400640 1 -- 172.21.15.94:0/505244751 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c940123e0 con 0x7f0c9c0cdb40 2024-09-10T23:39:47.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.783+0000 7f0c9be00640 1 -- 172.21.15.94:0/505244751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c9c0c7d90 con 0x7f0c9c0cdb40 2024-09-10T23:39:47.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.787+0000 7f0c9be00640 1 -- 172.21.15.94:0/505244751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c9c0c8350 con 0x7f0c9c0cdb40 2024-09-10T23:39:47.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.787+0000 7f0c9be00640 1 -- 172.21.15.94:0/505244751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c9c0cba20 con 0x7f0c9c0cdb40 2024-09-10T23:39:47.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.787+0000 7f0c93400640 1 -- 172.21.15.94:0/505244751 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 97057+0+0 (secure 0 0 0) 0x7f0c94010040 con 0x7f0c9c0cdb40 2024-09-10T23:39:47.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.787+0000 7f0c93400640 1 --2- 172.21.15.94:0/505244751 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0c78072d30 0x7f0c780751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:47.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.787+0000 7f0c93400640 1 -- 172.21.15.94:0/505244751 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5824+0+0 (secure 0 0 0) 0x7f0c94097b80 con 0x7f0c9c0cdb40 2024-09-10T23:39:47.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.787+0000 7f0c9aa00640 1 --2- 172.21.15.94:0/505244751 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0c78072d30 0x7f0c780751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:47.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.787+0000 7f0c9aa00640 1 --2- 172.21.15.94:0/505244751 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0c78072d30 0x7f0c780751f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c000c10 tx=0x7f0c8c006b80 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:47.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.791+0000 7f0c93400640 1 -- 172.21.15.94:0/505244751 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0c94061ff0 con 0x7f0c9c0cdb40 2024-09-10T23:39:47.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.971+0000 7f0c9be00640 1 -- 172.21.15.94:0/505244751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0c9c0b68c0 con 0x7f0c9c0cdb40 2024-09-10T23:39:47.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.971+0000 7f0c93400640 1 -- 172.21.15.94:0/505244751 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v140) ==== 80+0+0 (secure 0 0 0) 0x7f0c94066a20 con 0x7f0c9c0cdb40 2024-09-10T23:39:47.974 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:39:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.971+0000 7f0c9be00640 1 -- 172.21.15.94:0/505244751 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0c78072d30 msgr2=0x7f0c780751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.971+0000 7f0c9be00640 1 --2- 172.21.15.94:0/505244751 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0c78072d30 0x7f0c780751f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c000c10 tx=0x7f0c8c006b80 comp rx=0 tx=0).stop 2024-09-10T23:39:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.971+0000 7f0c9be00640 1 -- 172.21.15.94:0/505244751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c9c0cdb40 msgr2=0x7f0c9c0b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.971+0000 7f0c9be00640 1 --2- 172.21.15.94:0/505244751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c9c0cdb40 0x7f0c9c0b47e0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f0c940013e0 tx=0x7f0c94011a30 comp rx=0 tx=0).stop 2024-09-10T23:39:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.971+0000 7f0c9be00640 1 -- 172.21.15.94:0/505244751 shutdown_connections 2024-09-10T23:39:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.971+0000 7f0c9be00640 1 --2- 172.21.15.94:0/505244751 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0c78072d30 0x7f0c780751f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.971+0000 7f0c9be00640 1 --2- 172.21.15.94:0/505244751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c9c0cdb40 0x7f0c9c0b47e0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:47.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.971+0000 7f0c9be00640 1 --2- 172.21.15.94:0/505244751 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c9c15cde0 0x7f0c9c0cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.971+0000 7f0c9be00640 1 --2- 172.21.15.94:0/505244751 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c80004010 0x7f0c9c0c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.971+0000 7f0c9be00640 1 -- 172.21.15.94:0/505244751 >> 172.21.15.94:0/505244751 conn(0x7f0c9c0b2490 msgr2=0x7f0c9c0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.971+0000 7f0c9be00640 1 -- 172.21.15.94:0/505244751 shutdown_connections 2024-09-10T23:39:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:47.971+0000 7f0c9be00640 1 -- 172.21.15.94:0/505244751 wait complete. 2024-09-10T23:39:47.995 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-09-10T23:39:47.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:47.996 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T23:39:48.012 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[50535]: fuse finished with error 0 and tester_r 0 2024-09-10T23:39:48.015 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:39:48.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:48.015 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T23:39:48.096 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:39:48.096 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:39:48.097 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:48.106 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:39:48.107 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:48.161 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:39:48.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.299+0000 7f2893c00640 1 -- 172.21.15.94:0/554193429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2870003a10 msgr2=0x7f2870003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:48.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.299+0000 7f2893c00640 1 --2- 172.21.15.94:0/554193429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2870003a10 0x7f2870003e30 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f287c009b60 tx=0x7f287c01d240 comp rx=0 tx=0).stop 2024-09-10T23:39:48.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.299+0000 7f2891c00640 1 -- 172.21.15.94:0/554193429 reap_dead start 2024-09-10T23:39:48.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.299+0000 7f2893c00640 1 -- 172.21.15.94:0/554193429 shutdown_connections 2024-09-10T23:39:48.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.299+0000 7f2893c00640 1 -- 172.21.15.94:0/554193429 >> 172.21.15.94:0/554193429 conn(0x7f288c0cfaf0 msgr2=0x7f288c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:48.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.299+0000 7f2893c00640 1 -- 172.21.15.94:0/554193429 shutdown_connections 2024-09-10T23:39:48.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.299+0000 7f2893c00640 1 -- 172.21.15.94:0/554193429 wait complete. 2024-09-10T23:39:48.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2893c00640 1 Processor -- start 2024-09-10T23:39:48.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2893c00640 1 -- start start 2024-09-10T23:39:48.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2893c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2870003a10 0x7f288c06f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:48.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2893c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f288c15ed70 0x7f288c15c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:48.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2893c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f288c15cb40 0x7f288c0af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:48.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2893c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f288c13ec30 con 0x7f2870003a10 2024-09-10T23:39:48.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2893c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f288c15e420 con 0x7f288c15cb40 2024-09-10T23:39:48.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2893c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f288c13edb0 con 0x7f288c15ed70 2024-09-10T23:39:48.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2891200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2870003a10 0x7f288c06f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:48.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2891200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2870003a10 0x7f288c06f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53006/0 (socket says 172.21.15.94:53006) 2024-09-10T23:39:48.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2891200640 1 -- 172.21.15.94:0/3213646921 learned_addr learned my addr 172.21.15.94:0/3213646921 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:48.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2891c00640 1 --2- 172.21.15.94:0/3213646921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f288c15cb40 0x7f288c0af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:48.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2883e00640 1 --2- 172.21.15.94:0/3213646921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f288c15ed70 0x7f288c15c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:48.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2891200640 1 -- 172.21.15.94:0/3213646921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f288c15ed70 msgr2=0x7f288c15c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:48.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2891200640 1 --2- 172.21.15.94:0/3213646921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f288c15ed70 0x7f288c15c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:48.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2891200640 1 -- 172.21.15.94:0/3213646921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f288c15cb40 msgr2=0x7f288c0af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:48.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2891200640 1 --2- 172.21.15.94:0/3213646921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f288c15cb40 0x7f288c0af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:48.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2891200640 1 -- 172.21.15.94:0/3213646921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288c06fae0 con 0x7f2870003a10 2024-09-10T23:39:48.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2883e00640 1 --2- 172.21.15.94:0/3213646921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f288c15ed70 0x7f288c15c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:48.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2891c00640 1 --2- 172.21.15.94:0/3213646921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f288c15cb40 0x7f288c0af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:48.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2891200640 1 --2- 172.21.15.94:0/3213646921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2870003a10 0x7f288c06f5a0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f287c0af040 tx=0x7f287c005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:48.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2889600640 1 -- 172.21.15.94:0/3213646921 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c0ba070 con 0x7f2870003a10 2024-09-10T23:39:48.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2889600640 1 -- 172.21.15.94:0/3213646921 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f287c0ac8a0 con 0x7f2870003a10 2024-09-10T23:39:48.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2893c00640 1 -- 172.21.15.94:0/3213646921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f288c159b60 con 0x7f2870003a10 2024-09-10T23:39:48.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2889600640 1 -- 172.21.15.94:0/3213646921 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c0b1040 con 0x7f2870003a10 2024-09-10T23:39:48.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.303+0000 7f2893c00640 1 -- 172.21.15.94:0/3213646921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f288c06e9f0 con 0x7f2870003a10 2024-09-10T23:39:48.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.307+0000 7f2893c00640 1 -- 172.21.15.94:0/3213646921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f288c15cf50 con 0x7f2870003a10 2024-09-10T23:39:48.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.307+0000 7f2889600640 1 -- 172.21.15.94:0/3213646921 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 97057+0+0 (secure 0 0 0) 0x7f287c0046d0 con 0x7f2870003a10 2024-09-10T23:39:48.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.307+0000 7f2889600640 1 --2- 172.21.15.94:0/3213646921 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2868072d30 0x7f28680751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:48.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.307+0000 7f2889600640 1 -- 172.21.15.94:0/3213646921 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5824+0+0 (secure 0 0 0) 0x7f287c12ef40 con 0x7f2870003a10 2024-09-10T23:39:48.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.307+0000 7f2883e00640 1 --2- 172.21.15.94:0/3213646921 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2868072d30 0x7f28680751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:48.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.307+0000 7f2883e00640 1 --2- 172.21.15.94:0/3213646921 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2868072d30 0x7f28680751f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f288c15f2b0 tx=0x7f28740091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:48.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.311+0000 7f2889600640 1 -- 172.21.15.94:0/3213646921 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f288c15cf50 con 0x7f2870003a10 2024-09-10T23:39:48.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.487+0000 7f2893c00640 1 -- 172.21.15.94:0/3213646921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f288c13fad0 con 0x7f2870003a10 2024-09-10T23:39:48.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.487+0000 7f2889600640 1 -- 172.21.15.94:0/3213646921 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v140) ==== 80+0+0 (secure 0 0 0) 0x7f287c0c4020 con 0x7f2870003a10 2024-09-10T23:39:48.491 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:39:48.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.487+0000 7f2893c00640 1 -- 172.21.15.94:0/3213646921 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2868072d30 msgr2=0x7f28680751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:48.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.487+0000 7f2893c00640 1 --2- 172.21.15.94:0/3213646921 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2868072d30 0x7f28680751f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f288c15f2b0 tx=0x7f28740091e0 comp rx=0 tx=0).stop 2024-09-10T23:39:48.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.487+0000 7f2893c00640 1 -- 172.21.15.94:0/3213646921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2870003a10 msgr2=0x7f288c06f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:48.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.487+0000 7f2893c00640 1 --2- 172.21.15.94:0/3213646921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2870003a10 0x7f288c06f5a0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f287c0af040 tx=0x7f287c005620 comp rx=0 tx=0).stop 2024-09-10T23:39:48.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.491+0000 7f2893c00640 1 -- 172.21.15.94:0/3213646921 shutdown_connections 2024-09-10T23:39:48.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.491+0000 7f2893c00640 1 --2- 172.21.15.94:0/3213646921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f288c15cb40 0x7f288c0af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:48.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.491+0000 7f2893c00640 1 --2- 172.21.15.94:0/3213646921 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2868072d30 0x7f28680751f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:48.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.491+0000 7f2893c00640 1 --2- 172.21.15.94:0/3213646921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f288c15ed70 0x7f288c15c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:48.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.491+0000 7f2893c00640 1 --2- 172.21.15.94:0/3213646921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2870003a10 0x7f288c06f5a0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:48.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.491+0000 7f2893c00640 1 -- 172.21.15.94:0/3213646921 >> 172.21.15.94:0/3213646921 conn(0x7f288c0cfaf0 msgr2=0x7f288c0cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:48.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.491+0000 7f2893c00640 1 -- 172.21.15.94:0/3213646921 shutdown_connections 2024-09-10T23:39:48.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.491+0000 7f2893c00640 1 -- 172.21.15.94:0/3213646921 wait complete. 2024-09-10T23:39:48.512 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi181.front.sepia.ceph.com... 2024-09-10T23:39:48.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:48.513 DEBUG:teuthology.orchestra.run.smithi181:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T23:39:48.532 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[36046]: fuse finished with error 0 and tester_r 0 2024-09-10T23:39:48.535 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:39:48.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:39:48.536 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T23:39:48.673 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:39:48.674 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:39:48.674 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:48.682 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:39:48.682 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:39:48.683 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:39:48.683 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:48.692 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:39:48.693 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:39:48.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.863+0000 7f689c600640 1 -- 172.21.15.94:0/759832793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6894070150 msgr2=0x7f6894072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:48.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.863+0000 7f689c600640 1 --2- 172.21.15.94:0/759832793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6894070150 0x7f6894072610 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f68940cc090 tx=0x7f688801c830 comp rx=0 tx=0).stop 2024-09-10T23:39:48.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.863+0000 7f689c600640 1 -- 172.21.15.94:0/759832793 shutdown_connections 2024-09-10T23:39:48.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.863+0000 7f689c600640 1 --2- 172.21.15.94:0/759832793 >> v2:172.21.15.181:3300/0 conn(0x7f6894157c60 0x7f68940b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:48.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.863+0000 7f689c600640 1 --2- 172.21.15.94:0/759832793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6894070150 0x7f6894072610 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:48.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.863+0000 7f689c600640 1 -- 172.21.15.94:0/759832793 >> 172.21.15.94:0/759832793 conn(0x7f689406cdf0 msgr2=0x7f689406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:48.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.863+0000 7f689c600640 1 -- 172.21.15.94:0/759832793 shutdown_connections 2024-09-10T23:39:48.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.863+0000 7f689c600640 1 -- 172.21.15.94:0/759832793 wait complete. 2024-09-10T23:39:48.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f689c600640 1 Processor -- start 2024-09-10T23:39:48.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f689c600640 1 -- start start 2024-09-10T23:39:48.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f689c600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6894070150 0x7f6894150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:48.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f689c600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68940cfd30 0x7f689413c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:48.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f689c600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6894157c60 0x7f689414aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:48.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f689c600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f689415b3e0 con 0x7f6894070150 2024-09-10T23:39:48.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f689c600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f689415b260 con 0x7f68940cfd30 2024-09-10T23:39:48.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f689c600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f689415b560 con 0x7f6894157c60 2024-09-10T23:39:48.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f6899c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6894070150 0x7f6894150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:48.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f6899c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6894070150 0x7f6894150b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53028/0 (socket says 172.21.15.94:53028) 2024-09-10T23:39:48.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f6899c00640 1 -- 172.21.15.94:0/1269356920 learned_addr learned my addr 172.21.15.94:0/1269356920 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:48.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f689a600640 1 --2- 172.21.15.94:0/1269356920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6894157c60 0x7f689414aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:48.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f6899200640 1 --2- 172.21.15.94:0/1269356920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68940cfd30 0x7f689413c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:48.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f6899c00640 1 -- 172.21.15.94:0/1269356920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6894157c60 msgr2=0x7f689414aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:48.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f6899c00640 1 --2- 172.21.15.94:0/1269356920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6894157c60 0x7f689414aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:48.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f6899c00640 1 -- 172.21.15.94:0/1269356920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68940cfd30 msgr2=0x7f689413c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:48.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f6899c00640 1 --2- 172.21.15.94:0/1269356920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68940cfd30 0x7f689413c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:48.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f6899c00640 1 -- 172.21.15.94:0/1269356920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689414afc0 con 0x7f6894070150 2024-09-10T23:39:48.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.867+0000 7f6899c00640 1 --2- 172.21.15.94:0/1269356920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6894070150 0x7f6894150b40 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f68880b0040 tx=0x7f68880a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.871+0000 7f6891600640 1 -- 172.21.15.94:0/1269356920 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68880b2040 con 0x7f6894070150 2024-09-10T23:39:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.871+0000 7f6891600640 1 -- 172.21.15.94:0/1269356920 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6888005020 con 0x7f6894070150 2024-09-10T23:39:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.871+0000 7f6891600640 1 -- 172.21.15.94:0/1269356920 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888005470 con 0x7f6894070150 2024-09-10T23:39:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.871+0000 7f689c600640 1 -- 172.21.15.94:0/1269356920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689414dfc0 con 0x7f6894070150 2024-09-10T23:39:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.871+0000 7f689c600640 1 -- 172.21.15.94:0/1269356920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6894150340 con 0x7f6894070150 2024-09-10T23:39:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.871+0000 7f689c600640 1 -- 172.21.15.94:0/1269356920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6894072610 con 0x7f6894070150 2024-09-10T23:39:48.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.875+0000 7f6891600640 1 -- 172.21.15.94:0/1269356920 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) ==== 97057+0+0 (secure 0 0 0) 0x7f68880046d0 con 0x7f6894070150 2024-09-10T23:39:48.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.875+0000 7f6891600640 1 --2- 172.21.15.94:0/1269356920 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6874072d30 0x7f68740751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:48.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.875+0000 7f6891600640 1 -- 172.21.15.94:0/1269356920 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5824+0+0 (secure 0 0 0) 0x7f688812dc30 con 0x7f6894070150 2024-09-10T23:39:48.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.875+0000 7f6899200640 1 --2- 172.21.15.94:0/1269356920 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6874072d30 0x7f68740751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:48.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.875+0000 7f6899200640 1 --2- 172.21.15.94:0/1269356920 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6874072d30 0x7f68740751f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f68800058d0 tx=0x7f6880005860 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:48.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.879+0000 7f6891600640 1 -- 172.21.15.94:0/1269356920 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f68880f41d0 con 0x7f6894070150 2024-09-10T23:39:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:48.887+0000 7f6891600640 1 -- 172.21.15.94:0/1269356920 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f68880164b0 con 0x7f6894070150 2024-09-10T23:39:49.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.083+0000 7f689c600640 1 -- 172.21.15.94:0/1269356920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f689406e7e0 con 0x7f6894070150 2024-09-10T23:39:49.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.083+0000 7f6891600640 1 -- 172.21.15.94:0/1269356920 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5231 (secure 0 0 0) 0x7f68880f8660 con 0x7f6894070150 2024-09-10T23:39:49.088 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:49.088 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":98,"btime":"2024-09-10T23:39:35:824099+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7864,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/188024700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":188024700},{"type":"v1","addr":"172.21.15.181:6835","nonce":188024700}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8376,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/35853213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":35853213},{"type":"v1","addr":"172.21.15.94:6834","nonce":35853213}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/800902310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":800902310},{"type":"v1","addr":"172.21.15.94:6835","nonce":800902310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:39:24.990806+0000","modified":"2024-09-10T23:39:27.995870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196874},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196874":{"gid":196874,"name":"b","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6801/2882789540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2882789540},{"type":"v1","addr":"172.21.15.181:6801","nonce":2882789540}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196874,"qdb_cluster":[196874]},"id":11}]} 2024-09-10T23:39:49.088 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 98 2024-09-10T23:39:49.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.087+0000 7f689c600640 1 -- 172.21.15.94:0/1269356920 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6874072d30 msgr2=0x7f68740751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:49.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.087+0000 7f689c600640 1 --2- 172.21.15.94:0/1269356920 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6874072d30 0x7f68740751f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f68800058d0 tx=0x7f6880005860 comp rx=0 tx=0).stop 2024-09-10T23:39:49.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.087+0000 7f689c600640 1 -- 172.21.15.94:0/1269356920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6894070150 msgr2=0x7f6894150b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:49.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.087+0000 7f689c600640 1 --2- 172.21.15.94:0/1269356920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6894070150 0x7f6894150b40 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f68880b0040 tx=0x7f68880a5710 comp rx=0 tx=0).stop 2024-09-10T23:39:49.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.087+0000 7f689c600640 1 -- 172.21.15.94:0/1269356920 shutdown_connections 2024-09-10T23:39:49.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.087+0000 7f689c600640 1 --2- 172.21.15.94:0/1269356920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68940cfd30 0x7f689413c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.087+0000 7f689c600640 1 --2- 172.21.15.94:0/1269356920 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6874072d30 0x7f68740751f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.087+0000 7f689c600640 1 --2- 172.21.15.94:0/1269356920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6894157c60 0x7f689414aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.087+0000 7f689c600640 1 --2- 172.21.15.94:0/1269356920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6894070150 0x7f6894150b40 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.087+0000 7f689c600640 1 -- 172.21.15.94:0/1269356920 >> 172.21.15.94:0/1269356920 conn(0x7f689406cdf0 msgr2=0x7f68940ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.087+0000 7f689c600640 1 -- 172.21.15.94:0/1269356920 shutdown_connections 2024-09-10T23:39:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.087+0000 7f689c600640 1 -- 172.21.15.94:0/1269356920 wait complete. 2024-09-10T23:39:49.110 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:39:49.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.247+0000 7fda72e00640 1 -- 172.21.15.94:0/444823274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda54003ab0 msgr2=0x7fda54003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:49.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.247+0000 7fda72e00640 1 --2- 172.21.15.94:0/444823274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda54003ab0 0x7fda54003ef0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fda60006f90 tx=0x7fda6001cdf0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.247+0000 7fda72e00640 1 -- 172.21.15.94:0/444823274 shutdown_connections 2024-09-10T23:39:49.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.247+0000 7fda72e00640 1 --2- 172.21.15.94:0/444823274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda54003ab0 0x7fda54003ef0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.247+0000 7fda72e00640 1 -- 172.21.15.94:0/444823274 >> 172.21.15.94:0/444823274 conn(0x7fda6c09aa70 msgr2=0x7fda6c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:49.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.247+0000 7fda72e00640 1 -- 172.21.15.94:0/444823274 shutdown_connections 2024-09-10T23:39:49.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.247+0000 7fda72e00640 1 -- 172.21.15.94:0/444823274 wait complete. 2024-09-10T23:39:49.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda72e00640 1 Processor -- start 2024-09-10T23:39:49.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda72e00640 1 -- start start 2024-09-10T23:39:49.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda72e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda54003ab0 0x7fda6c0c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:49.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda72e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda6c1fb0b0 0x7fda6c1485f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:49.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda72e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda6c14b7b0 0x7fda6c1eb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:49.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda72e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c0b0960 con 0x7fda54003ab0 2024-09-10T23:39:49.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda72e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c0b07e0 con 0x7fda6c1fb0b0 2024-09-10T23:39:49.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda72e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fda6c0b0ae0 con 0x7fda6c14b7b0 2024-09-10T23:39:49.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda71a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda54003ab0 0x7fda6c0c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:49.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda71a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda54003ab0 0x7fda6c0c5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53044/0 (socket says 172.21.15.94:53044) 2024-09-10T23:39:49.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda71a00640 1 -- 172.21.15.94:0/961536586 learned_addr learned my addr 172.21.15.94:0/961536586 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:49.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda71000640 1 --2- 172.21.15.94:0/961536586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda6c1fb0b0 0x7fda6c1485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:49.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda72400640 1 --2- 172.21.15.94:0/961536586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda6c14b7b0 0x7fda6c1eb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:49.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda71a00640 1 -- 172.21.15.94:0/961536586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda6c14b7b0 msgr2=0x7fda6c1eb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:49.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda71a00640 1 --2- 172.21.15.94:0/961536586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda6c14b7b0 0x7fda6c1eb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda71a00640 1 -- 172.21.15.94:0/961536586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda6c1fb0b0 msgr2=0x7fda6c1485f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:49.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda71a00640 1 --2- 172.21.15.94:0/961536586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda6c1fb0b0 0x7fda6c1485f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda71a00640 1 -- 172.21.15.94:0/961536586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6c1fb8a0 con 0x7fda54003ab0 2024-09-10T23:39:49.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda71000640 1 --2- 172.21.15.94:0/961536586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda6c1fb0b0 0x7fda6c1485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:49.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda71a00640 1 --2- 172.21.15.94:0/961536586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda54003ab0 0x7fda6c0c5ac0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fda6001b4f0 tx=0x7fda60006c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:49.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda69600640 1 -- 172.21.15.94:0/961536586 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda600014c0 con 0x7fda54003ab0 2024-09-10T23:39:49.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda69600640 1 -- 172.21.15.94:0/961536586 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda60001660 con 0x7fda54003ab0 2024-09-10T23:39:49.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda72e00640 1 -- 172.21.15.94:0/961536586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6c1fbb80 con 0x7fda54003ab0 2024-09-10T23:39:49.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda69600640 1 -- 172.21.15.94:0/961536586 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60006330 con 0x7fda54003ab0 2024-09-10T23:39:49.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda72e00640 1 -- 172.21.15.94:0/961536586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda6c1fb530 con 0x7fda54003ab0 2024-09-10T23:39:49.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.251+0000 7fda72e00640 1 -- 172.21.15.94:0/961536586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda6c0b1ae0 con 0x7fda54003ab0 2024-09-10T23:39:49.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.255+0000 7fda69600640 1 -- 172.21.15.94:0/961536586 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fda60003600 con 0x7fda54003ab0 2024-09-10T23:39:49.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.255+0000 7fda69600640 1 --2- 172.21.15.94:0/961536586 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fda4c072e80 0x7fda4c075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:49.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.255+0000 7fda69600640 1 -- 172.21.15.94:0/961536586 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5824+0+0 (secure 0 0 0) 0x7fda6012b3f0 con 0x7fda54003ab0 2024-09-10T23:39:49.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.255+0000 7fda71000640 1 --2- 172.21.15.94:0/961536586 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fda4c072e80 0x7fda4c075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:49.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.255+0000 7fda71000640 1 --2- 172.21.15.94:0/961536586 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fda4c072e80 0x7fda4c075340 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0b2840 tx=0x7fda580091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:49.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.259+0000 7fda69600640 1 -- 172.21.15.94:0/961536586 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fda600be060 con 0x7fda54003ab0 2024-09-10T23:39:49.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.459+0000 7fda72e00640 1 -- 172.21.15.94:0/961536586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda6c0c53a0 con 0x7fda54003ab0 2024-09-10T23:39:49.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.459+0000 7fda69600640 1 -- 172.21.15.94:0/961536586 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5231 (secure 0 0 0) 0x7fda60016bd0 con 0x7fda54003ab0 2024-09-10T23:39:49.462 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:49.463 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":98,"btime":"2024-09-10T23:39:35:824099+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7864,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/188024700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":188024700},{"type":"v1","addr":"172.21.15.181:6835","nonce":188024700}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8376,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/35853213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":35853213},{"type":"v1","addr":"172.21.15.94:6834","nonce":35853213}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/800902310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":800902310},{"type":"v1","addr":"172.21.15.94:6835","nonce":800902310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:39:24.990806+0000","modified":"2024-09-10T23:39:27.995870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196874},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196874":{"gid":196874,"name":"b","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6801/2882789540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2882789540},{"type":"v1","addr":"172.21.15.181:6801","nonce":2882789540}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196874,"qdb_cluster":[196874]},"id":11}]} 2024-09-10T23:39:49.463 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 98 2024-09-10T23:39:49.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.459+0000 7fda72e00640 1 -- 172.21.15.94:0/961536586 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fda4c072e80 msgr2=0x7fda4c075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:49.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.459+0000 7fda72e00640 1 --2- 172.21.15.94:0/961536586 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fda4c072e80 0x7fda4c075340 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0b2840 tx=0x7fda580091e0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.459+0000 7fda72e00640 1 -- 172.21.15.94:0/961536586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda54003ab0 msgr2=0x7fda6c0c5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:49.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.459+0000 7fda72e00640 1 --2- 172.21.15.94:0/961536586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda54003ab0 0x7fda6c0c5ac0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fda6001b4f0 tx=0x7fda60006c50 comp rx=0 tx=0).stop 2024-09-10T23:39:49.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.459+0000 7fda72e00640 1 -- 172.21.15.94:0/961536586 shutdown_connections 2024-09-10T23:39:49.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.459+0000 7fda72e00640 1 --2- 172.21.15.94:0/961536586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda6c1fb0b0 0x7fda6c1485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.459+0000 7fda72e00640 1 --2- 172.21.15.94:0/961536586 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fda4c072e80 0x7fda4c075340 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.459+0000 7fda72e00640 1 --2- 172.21.15.94:0/961536586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda6c14b7b0 0x7fda6c1eb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.459+0000 7fda72e00640 1 --2- 172.21.15.94:0/961536586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda54003ab0 0x7fda6c0c5ac0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.459+0000 7fda72e00640 1 -- 172.21.15.94:0/961536586 >> 172.21.15.94:0/961536586 conn(0x7fda6c09aa70 msgr2=0x7fda6c0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:49.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.459+0000 7fda72e00640 1 -- 172.21.15.94:0/961536586 shutdown_connections 2024-09-10T23:39:49.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.459+0000 7fda72e00640 1 -- 172.21.15.94:0/961536586 wait complete. 2024-09-10T23:39:49.484 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:39:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.623+0000 7f8d1dc00640 1 -- 172.21.15.94:0/3662574528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d180d01e0 msgr2=0x7f8d180af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.623+0000 7f8d1dc00640 1 --2- 172.21.15.94:0/3662574528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d180d01e0 0x7f8d180af540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009bf0 tx=0x7f8d1001cbb0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.623+0000 7f8d1dc00640 1 -- 172.21.15.94:0/3662574528 shutdown_connections 2024-09-10T23:39:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.623+0000 7f8d1dc00640 1 --2- 172.21.15.94:0/3662574528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d180d01e0 0x7f8d180af540 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.623+0000 7f8d1dc00640 1 -- 172.21.15.94:0/3662574528 >> 172.21.15.94:0/3662574528 conn(0x7f8d1809aa70 msgr2=0x7f8d1809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:49.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.623+0000 7f8d1dc00640 1 -- 172.21.15.94:0/3662574528 shutdown_connections 2024-09-10T23:39:49.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.623+0000 7f8d1dc00640 1 -- 172.21.15.94:0/3662574528 wait complete. 2024-09-10T23:39:49.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d1dc00640 1 Processor -- start 2024-09-10T23:39:49.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d1dc00640 1 -- start start 2024-09-10T23:39:49.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d1dc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d180cfda0 0x7f8d180ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:49.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d1dc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d180d01e0 0x7f8d18154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:49.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d1dc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d1815ba30 0x7f8d180ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:49.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d1dc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181698d0 con 0x7f8d180cfda0 2024-09-10T23:39:49.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d1dc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8d18169a50 con 0x7f8d180d01e0 2024-09-10T23:39:49.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d1dc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8d18169bd0 con 0x7f8d1815ba30 2024-09-10T23:39:49.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d17e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d180cfda0 0x7f8d180ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:49.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d17e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d180cfda0 0x7f8d180ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53048/0 (socket says 172.21.15.94:53048) 2024-09-10T23:39:49.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d17e00640 1 -- 172.21.15.94:0/1678368216 learned_addr learned my addr 172.21.15.94:0/1678368216 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:49.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d17400640 1 --2- 172.21.15.94:0/1678368216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d180d01e0 0x7f8d18154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:49.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d1d200640 1 --2- 172.21.15.94:0/1678368216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d1815ba30 0x7f8d180ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d17e00640 1 -- 172.21.15.94:0/1678368216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d1815ba30 msgr2=0x7f8d180ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d17e00640 1 --2- 172.21.15.94:0/1678368216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d1815ba30 0x7f8d180ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d17e00640 1 -- 172.21.15.94:0/1678368216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d180d01e0 msgr2=0x7f8d18154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d17e00640 1 --2- 172.21.15.94:0/1678368216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d180d01e0 0x7f8d18154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d17e00640 1 -- 172.21.15.94:0/1678368216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d180ca870 con 0x7f8d180cfda0 2024-09-10T23:39:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d17e00640 1 --2- 172.21.15.94:0/1678368216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d180cfda0 0x7f8d180ce0f0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800c430 tx=0x7f8d0800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d0fe00640 1 -- 172.21.15.94:0/1678368216 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08011850 con 0x7f8d180cfda0 2024-09-10T23:39:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d0fe00640 1 -- 172.21.15.94:0/1678368216 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d08017070 con 0x7f8d180cfda0 2024-09-10T23:39:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d0fe00640 1 -- 172.21.15.94:0/1678368216 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08011da0 con 0x7f8d180cfda0 2024-09-10T23:39:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d1dc00640 1 -- 172.21.15.94:0/1678368216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d180caaf0 con 0x7f8d180cfda0 2024-09-10T23:39:49.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.627+0000 7f8d1dc00640 1 -- 172.21.15.94:0/1678368216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d180cb0b0 con 0x7f8d180cfda0 2024-09-10T23:39:49.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.631+0000 7f8d1dc00640 1 -- 172.21.15.94:0/1678368216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d18158dd0 con 0x7f8d180cfda0 2024-09-10T23:39:49.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.631+0000 7f8d0fe00640 1 -- 172.21.15.94:0/1678368216 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f8d0801c460 con 0x7f8d180cfda0 2024-09-10T23:39:49.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.631+0000 7f8d0fe00640 1 --2- 172.21.15.94:0/1678368216 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8cf8072e80 0x7f8cf8075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:49.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.631+0000 7f8d0fe00640 1 -- 172.21.15.94:0/1678368216 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5824+0+0 (secure 0 0 0) 0x7f8d080916f0 con 0x7f8d180cfda0 2024-09-10T23:39:49.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.631+0000 7f8d17400640 1 --2- 172.21.15.94:0/1678368216 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8cf8072e80 0x7f8cf8075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:49.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.631+0000 7f8d17400640 1 --2- 172.21.15.94:0/1678368216 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8cf8072e80 0x7f8cf8075340 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8d04000c10 tx=0x7f8d04009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:49.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.635+0000 7f8d0fe00640 1 -- 172.21.15.94:0/1678368216 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8d180ca870 con 0x7f8d180cfda0 2024-09-10T23:39:49.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.803+0000 7f8d1dc00640 1 -- 172.21.15.94:0/1678368216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d180b20c0 con 0x7f8d180cfda0 2024-09-10T23:39:49.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.803+0000 7f8d0fe00640 1 -- 172.21.15.94:0/1678368216 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+16916 (secure 0 0 0) 0x7f8d0805e230 con 0x7f8d180cfda0 2024-09-10T23:39:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:49.809 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":140,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:39:38.844662+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:39:22.673102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T23:39:23.946666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.d","mappings":[{"from":5,"to":3}]},{"pgid":"23.1a","mappings":[{"from":5,"to":7}]},{"pgid":"23.3c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:39:49.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.807+0000 7f8d1dc00640 1 -- 172.21.15.94:0/1678368216 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8cf8072e80 msgr2=0x7f8cf8075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:49.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.807+0000 7f8d1dc00640 1 --2- 172.21.15.94:0/1678368216 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8cf8072e80 0x7f8cf8075340 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8d04000c10 tx=0x7f8d04009040 comp rx=0 tx=0).stop 2024-09-10T23:39:49.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.807+0000 7f8d1dc00640 1 -- 172.21.15.94:0/1678368216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d180cfda0 msgr2=0x7f8d180ce0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:49.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.807+0000 7f8d1dc00640 1 --2- 172.21.15.94:0/1678368216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d180cfda0 0x7f8d180ce0f0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800c430 tx=0x7f8d0800c8f0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.807+0000 7f8d1dc00640 1 -- 172.21.15.94:0/1678368216 shutdown_connections 2024-09-10T23:39:49.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.807+0000 7f8d1dc00640 1 --2- 172.21.15.94:0/1678368216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d180d01e0 0x7f8d18154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.807+0000 7f8d1dc00640 1 --2- 172.21.15.94:0/1678368216 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8cf8072e80 0x7f8cf8075340 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.807+0000 7f8d1dc00640 1 --2- 172.21.15.94:0/1678368216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d1815ba30 0x7f8d180ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.807+0000 7f8d1dc00640 1 --2- 172.21.15.94:0/1678368216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d180cfda0 0x7f8d180ce0f0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.807+0000 7f8d1dc00640 1 -- 172.21.15.94:0/1678368216 >> 172.21.15.94:0/1678368216 conn(0x7f8d1809aa70 msgr2=0x7f8d1806f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:49.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.807+0000 7f8d1dc00640 1 -- 172.21.15.94:0/1678368216 shutdown_connections 2024-09-10T23:39:49.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.807+0000 7f8d1dc00640 1 -- 172.21.15.94:0/1678368216 wait complete. 2024-09-10T23:39:49.831 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T23:39:49.831 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:39:49.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d89000640 1 -- 172.21.15.94:0/1863110001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d840d0010 msgr2=0x7f6d840ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:49.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d89000640 1 --2- 172.21.15.94:0/1863110001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d840d0010 0x7f6d840ceb10 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009c80 tx=0x7f6d7c01f5a0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d89000640 1 -- 172.21.15.94:0/1863110001 shutdown_connections 2024-09-10T23:39:49.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d89000640 1 --2- 172.21.15.94:0/1863110001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d840d0010 0x7f6d840ceb10 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d89000640 1 --2- 172.21.15.94:0/1863110001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d840cfc30 0x7f6d840ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d89000640 1 -- 172.21.15.94:0/1863110001 >> 172.21.15.94:0/1863110001 conn(0x7f6d840d1bf0 msgr2=0x7f6d840d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:49.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d89000640 1 -- 172.21.15.94:0/1863110001 shutdown_connections 2024-09-10T23:39:49.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d89000640 1 -- 172.21.15.94:0/1863110001 wait complete. 2024-09-10T23:39:49.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d89000640 1 Processor -- start 2024-09-10T23:39:49.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d89000640 1 -- start start 2024-09-10T23:39:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d89000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d840cd4e0 0x7f6d84159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d89000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d840cfc30 0x7f6d841f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:49.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d89000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d840d0010 0x7f6d841590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d89000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6d840b1c10 con 0x7f6d840cd4e0 2024-09-10T23:39:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d89000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6d840b1a90 con 0x7f6d840cfc30 2024-09-10T23:39:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d89000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6d840b1d90 con 0x7f6d840d0010 2024-09-10T23:39:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d81e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d840cd4e0 0x7f6d84159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d81e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d840cd4e0 0x7f6d84159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53072/0 (socket says 172.21.15.94:53072) 2024-09-10T23:39:49.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.971+0000 7f6d81e00640 1 -- 172.21.15.94:0/2654816518 learned_addr learned my addr 172.21.15.94:0/2654816518 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d81400640 1 --2- 172.21.15.94:0/2654816518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d840cfc30 0x7f6d841f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d82800640 1 --2- 172.21.15.94:0/2654816518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d840d0010 0x7f6d841590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d81e00640 1 -- 172.21.15.94:0/2654816518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d840d0010 msgr2=0x7f6d841590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d81e00640 1 --2- 172.21.15.94:0/2654816518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d840d0010 0x7f6d841590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d81e00640 1 -- 172.21.15.94:0/2654816518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d840cfc30 msgr2=0x7f6d841f66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d81e00640 1 --2- 172.21.15.94:0/2654816518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d840cfc30 0x7f6d841f66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:49.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d81e00640 1 -- 172.21.15.94:0/2654816518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d841f6c20 con 0x7f6d840cd4e0 2024-09-10T23:39:49.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d82800640 1 --2- 172.21.15.94:0/2654816518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d840d0010 0x7f6d841590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:49.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d81e00640 1 --2- 172.21.15.94:0/2654816518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d840cd4e0 0x7f6d84159fe0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400d9b0 tx=0x7f6d7400de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:49.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d7a000640 1 -- 172.21.15.94:0/2654816518 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d74017070 con 0x7f6d840cd4e0 2024-09-10T23:39:49.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d7a000640 1 -- 172.21.15.94:0/2654816518 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d74009cc0 con 0x7f6d840cd4e0 2024-09-10T23:39:49.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d89000640 1 -- 172.21.15.94:0/2654816518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d841f2af0 con 0x7f6d840cd4e0 2024-09-10T23:39:49.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d7a000640 1 -- 172.21.15.94:0/2654816518 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d74005280 con 0x7f6d840cd4e0 2024-09-10T23:39:49.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d89000640 1 -- 172.21.15.94:0/2654816518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d841f30b0 con 0x7f6d840cd4e0 2024-09-10T23:39:49.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d89000640 1 -- 172.21.15.94:0/2654816518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d840ccfa0 con 0x7f6d840cd4e0 2024-09-10T23:39:49.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d7a000640 1 -- 172.21.15.94:0/2654816518 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f6d74005420 con 0x7f6d840cd4e0 2024-09-10T23:39:49.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.975+0000 7f6d7a000640 1 --2- 172.21.15.94:0/2654816518 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6d64072e50 0x7f6d64075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:49.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.979+0000 7f6d7a000640 1 -- 172.21.15.94:0/2654816518 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5824+0+0 (secure 0 0 0) 0x7f6d740946a0 con 0x7f6d840cd4e0 2024-09-10T23:39:49.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.979+0000 7f6d81400640 1 --2- 172.21.15.94:0/2654816518 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6d64072e50 0x7f6d64075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:49.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.979+0000 7f6d81400640 1 --2- 172.21.15.94:0/2654816518 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6d64072e50 0x7f6d64075310 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6d8409f740 tx=0x7f6d70009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:49.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:49.979+0000 7f6d7a000640 1 -- 172.21.15.94:0/2654816518 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6d841f6c20 con 0x7f6d840cd4e0 2024-09-10T23:39:50.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.179+0000 7f6d89000640 1 -- 172.21.15.94:0/2654816518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d840b7850 con 0x7f6d840cd4e0 2024-09-10T23:39:50.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.179+0000 7f6d7a000640 1 -- 172.21.15.94:0/2654816518 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5231 (secure 0 0 0) 0x7f6d7405e7e0 con 0x7f6d840cd4e0 2024-09-10T23:39:50.183 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:50.183 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":98,"btime":"2024-09-10T23:39:35:824099+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7864,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/188024700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":188024700},{"type":"v1","addr":"172.21.15.181:6835","nonce":188024700}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8376,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/35853213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":35853213},{"type":"v1","addr":"172.21.15.94:6834","nonce":35853213}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/800902310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":800902310},{"type":"v1","addr":"172.21.15.94:6835","nonce":800902310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:39:24.990806+0000","modified":"2024-09-10T23:39:27.995870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196874},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196874":{"gid":196874,"name":"b","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6801/2882789540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2882789540},{"type":"v1","addr":"172.21.15.181:6801","nonce":2882789540}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196874,"qdb_cluster":[196874]},"id":11}]} 2024-09-10T23:39:50.183 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 98 2024-09-10T23:39:50.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.179+0000 7f6d89000640 1 -- 172.21.15.94:0/2654816518 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6d64072e50 msgr2=0x7f6d64075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:50.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.179+0000 7f6d89000640 1 --2- 172.21.15.94:0/2654816518 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6d64072e50 0x7f6d64075310 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6d8409f740 tx=0x7f6d70009040 comp rx=0 tx=0).stop 2024-09-10T23:39:50.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.179+0000 7f6d89000640 1 -- 172.21.15.94:0/2654816518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d840cd4e0 msgr2=0x7f6d84159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:50.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.179+0000 7f6d89000640 1 --2- 172.21.15.94:0/2654816518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d840cd4e0 0x7f6d84159fe0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400d9b0 tx=0x7f6d7400de70 comp rx=0 tx=0).stop 2024-09-10T23:39:50.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.179+0000 7f6d89000640 1 -- 172.21.15.94:0/2654816518 shutdown_connections 2024-09-10T23:39:50.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.179+0000 7f6d89000640 1 --2- 172.21.15.94:0/2654816518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d840cfc30 0x7f6d841f66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.179+0000 7f6d89000640 1 --2- 172.21.15.94:0/2654816518 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6d64072e50 0x7f6d64075310 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.179+0000 7f6d89000640 1 --2- 172.21.15.94:0/2654816518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d840d0010 0x7f6d841590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.179+0000 7f6d89000640 1 --2- 172.21.15.94:0/2654816518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d840cd4e0 0x7f6d84159fe0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.179+0000 7f6d89000640 1 -- 172.21.15.94:0/2654816518 >> 172.21.15.94:0/2654816518 conn(0x7f6d840d1bf0 msgr2=0x7f6d840af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.179+0000 7f6d89000640 1 -- 172.21.15.94:0/2654816518 shutdown_connections 2024-09-10T23:39:50.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.179+0000 7f6d89000640 1 -- 172.21.15.94:0/2654816518 wait complete. 2024-09-10T23:39:50.203 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:39:50.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.339+0000 7f64ee400640 1 -- 172.21.15.94:0/2787910398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64cc004810 msgr2=0x7f64cc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:50.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.339+0000 7f64ee400640 1 --2- 172.21.15.94:0/2787910398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64cc004810 0x7f64cc004c90 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f64e808e340 tx=0x7f64d401cd40 comp rx=0 tx=0).stop 2024-09-10T23:39:50.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.339+0000 7f64e7e00640 1 -- 172.21.15.94:0/2787910398 reap_dead start 2024-09-10T23:39:50.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64ee400640 1 -- 172.21.15.94:0/2787910398 shutdown_connections 2024-09-10T23:39:50.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64ee400640 1 -- 172.21.15.94:0/2787910398 >> 172.21.15.94:0/2787910398 conn(0x7f64e80d3de0 msgr2=0x7f64e80d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:50.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64ee400640 1 -- 172.21.15.94:0/2787910398 shutdown_connections 2024-09-10T23:39:50.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64ee400640 1 -- 172.21.15.94:0/2787910398 wait complete. 2024-09-10T23:39:50.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64ee400640 1 Processor -- start 2024-09-10T23:39:50.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64ee400640 1 -- start start 2024-09-10T23:39:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64ee400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64cc004810 0x7f64e81f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64ee400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64e8159250 0x7f64e815bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64ee400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64e81596d0 0x7f64e815c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64ee400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f64e80b74c0 con 0x7f64e8159250 2024-09-10T23:39:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64ee400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f64e80b7340 con 0x7f64cc004810 2024-09-10T23:39:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64ee400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f64e80b7640 con 0x7f64e81596d0 2024-09-10T23:39:50.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64e6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64e8159250 0x7f64e815bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64e6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64e8159250 0x7f64e815bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53080/0 (socket says 172.21.15.94:53080) 2024-09-10T23:39:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64e6a00640 1 -- 172.21.15.94:0/1093610416 learned_addr learned my addr 172.21.15.94:0/1093610416 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64e7400640 1 --2- 172.21.15.94:0/1093610416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64cc004810 0x7f64e81f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64e6a00640 1 -- 172.21.15.94:0/1093610416 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64e81596d0 msgr2=0x7f64e815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64e7e00640 1 --2- 172.21.15.94:0/1093610416 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64e81596d0 0x7f64e815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64e6a00640 1 --2- 172.21.15.94:0/1093610416 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64e81596d0 0x7f64e815c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64e6a00640 1 -- 172.21.15.94:0/1093610416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64cc004810 msgr2=0x7f64e81f45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64e6a00640 1 --2- 172.21.15.94:0/1093610416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64cc004810 0x7f64e81f45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64e6a00640 1 -- 172.21.15.94:0/1093610416 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e80cc690 con 0x7f64e8159250 2024-09-10T23:39:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64e7400640 1 --2- 172.21.15.94:0/1093610416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64cc004810 0x7f64e81f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64e6a00640 1 --2- 172.21.15.94:0/1093610416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64e8159250 0x7f64e815bce0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f64d4000c10 tx=0x7f64d4005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64df400640 1 -- 172.21.15.94:0/1093610416 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d40ab040 con 0x7f64e8159250 2024-09-10T23:39:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64df400640 1 -- 172.21.15.94:0/1093610416 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64d40a5b50 con 0x7f64e8159250 2024-09-10T23:39:50.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64ee400640 1 -- 172.21.15.94:0/1093610416 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e80cc970 con 0x7f64e8159250 2024-09-10T23:39:50.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64df400640 1 -- 172.21.15.94:0/1093610416 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4004b20 con 0x7f64e8159250 2024-09-10T23:39:50.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.343+0000 7f64ee400640 1 -- 172.21.15.94:0/1093610416 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e80ccf30 con 0x7f64e8159250 2024-09-10T23:39:50.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.347+0000 7f64ee400640 1 -- 172.21.15.94:0/1093610416 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e80b6c30 con 0x7f64e8159250 2024-09-10T23:39:50.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.347+0000 7f64df400640 1 -- 172.21.15.94:0/1093610416 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f64d40b8070 con 0x7f64e8159250 2024-09-10T23:39:50.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.347+0000 7f64df400640 1 --2- 172.21.15.94:0/1093610416 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64c8072e50 0x7f64c8075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:50.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.347+0000 7f64df400640 1 -- 172.21.15.94:0/1093610416 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5824+0+0 (secure 0 0 0) 0x7f64d412e6e0 con 0x7f64e8159250 2024-09-10T23:39:50.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.347+0000 7f64e7400640 1 --2- 172.21.15.94:0/1093610416 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64c8072e50 0x7f64c8075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:50.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.351+0000 7f64e7400640 1 --2- 172.21.15.94:0/1093610416 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64c8072e50 0x7f64c8075310 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f64d8001b00 tx=0x7f64d8006b30 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:50.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.351+0000 7f64df400640 1 -- 172.21.15.94:0/1093610416 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f64d40c7060 con 0x7f64e8159250 2024-09-10T23:39:50.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.555+0000 7f64ee400640 1 -- 172.21.15.94:0/1093610416 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64e80d1390 con 0x7f64e8159250 2024-09-10T23:39:50.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.555+0000 7f64df400640 1 -- 172.21.15.94:0/1093610416 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5231 (secure 0 0 0) 0x7f64d4016a50 con 0x7f64e8159250 2024-09-10T23:39:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:50.559 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":98,"btime":"2024-09-10T23:39:35:824099+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7864,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/188024700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":188024700},{"type":"v1","addr":"172.21.15.181:6835","nonce":188024700}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8376,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/35853213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":35853213},{"type":"v1","addr":"172.21.15.94:6834","nonce":35853213}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/800902310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":800902310},{"type":"v1","addr":"172.21.15.94:6835","nonce":800902310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:39:24.990806+0000","modified":"2024-09-10T23:39:27.995870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":196874},"failed":[],"damaged":[],"stopped":[],"info":{"gid_196874":{"gid":196874,"name":"b","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6801/2882789540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2882789540},{"type":"v1","addr":"172.21.15.181:6801","nonce":2882789540}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":196874,"qdb_cluster":[196874]},"id":11}]} 2024-09-10T23:39:50.559 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 98 2024-09-10T23:39:50.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.555+0000 7f64ee400640 1 -- 172.21.15.94:0/1093610416 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64c8072e50 msgr2=0x7f64c8075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:50.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.555+0000 7f64ee400640 1 --2- 172.21.15.94:0/1093610416 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64c8072e50 0x7f64c8075310 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f64d8001b00 tx=0x7f64d8006b30 comp rx=0 tx=0).stop 2024-09-10T23:39:50.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.555+0000 7f64ee400640 1 -- 172.21.15.94:0/1093610416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64e8159250 msgr2=0x7f64e815bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:50.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.555+0000 7f64ee400640 1 --2- 172.21.15.94:0/1093610416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64e8159250 0x7f64e815bce0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f64d4000c10 tx=0x7f64d4005680 comp rx=0 tx=0).stop 2024-09-10T23:39:50.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.555+0000 7f64ee400640 1 -- 172.21.15.94:0/1093610416 shutdown_connections 2024-09-10T23:39:50.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.555+0000 7f64ee400640 1 --2- 172.21.15.94:0/1093610416 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64c8072e50 0x7f64c8075310 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.555+0000 7f64ee400640 1 --2- 172.21.15.94:0/1093610416 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64e81596d0 0x7f64e815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.555+0000 7f64ee400640 1 --2- 172.21.15.94:0/1093610416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64e8159250 0x7f64e815bce0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.555+0000 7f64ee400640 1 --2- 172.21.15.94:0/1093610416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64cc004810 0x7f64e81f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.555+0000 7f64ee400640 1 -- 172.21.15.94:0/1093610416 >> 172.21.15.94:0/1093610416 conn(0x7f64e80d3de0 msgr2=0x7f64e80b8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:50.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.555+0000 7f64ee400640 1 -- 172.21.15.94:0/1093610416 shutdown_connections 2024-09-10T23:39:50.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.555+0000 7f64ee400640 1 -- 172.21.15.94:0/1093610416 wait complete. 2024-09-10T23:39:50.581 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:39:50.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.719+0000 7ff0e4000640 1 -- 172.21.15.94:0/2222975718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0dc15bcc0 msgr2=0x7ff0dc0ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:50.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.719+0000 7ff0e4000640 1 --2- 172.21.15.94:0/2222975718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0dc15bcc0 0x7ff0dc0ad060 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0cf380 tx=0x7ff0d001cb10 comp rx=0 tx=0).stop 2024-09-10T23:39:50.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.719+0000 7ff0e4000640 1 -- 172.21.15.94:0/2222975718 shutdown_connections 2024-09-10T23:39:50.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.719+0000 7ff0e4000640 1 --2- 172.21.15.94:0/2222975718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0dc15bcc0 0x7ff0dc0ad060 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.719+0000 7ff0e4000640 1 -- 172.21.15.94:0/2222975718 >> 172.21.15.94:0/2222975718 conn(0x7ff0dc0d02d0 msgr2=0x7ff0dc0d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:50.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.719+0000 7ff0e4000640 1 -- 172.21.15.94:0/2222975718 shutdown_connections 2024-09-10T23:39:50.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.719+0000 7ff0e4000640 1 -- 172.21.15.94:0/2222975718 wait complete. 2024-09-10T23:39:50.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.719+0000 7ff0e4000640 1 Processor -- start 2024-09-10T23:39:50.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.719+0000 7ff0e4000640 1 -- start start 2024-09-10T23:39:50.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e4000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0dc068c30 0x7ff0dc0b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:50.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e4000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0cfda0 0x7ff0dc152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:50.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e4000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0dc15bcc0 0x7ff0dc152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:50.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e4000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc16a520 con 0x7ff0dc0cfda0 2024-09-10T23:39:50.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e4000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc16a3a0 con 0x7ff0dc068c30 2024-09-10T23:39:50.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e4000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff0dc16a6a0 con 0x7ff0dc15bcc0 2024-09-10T23:39:50.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0cfda0 0x7ff0dc152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:50.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0cfda0 0x7ff0dc152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53086/0 (socket says 172.21.15.94:53086) 2024-09-10T23:39:50.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e0c00640 1 -- 172.21.15.94:0/637900341 learned_addr learned my addr 172.21.15.94:0/637900341 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:50.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e1600640 1 --2- 172.21.15.94:0/637900341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0dc068c30 0x7ff0dc0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:50.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e2000640 1 --2- 172.21.15.94:0/637900341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0dc15bcc0 0x7ff0dc152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:50.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e0c00640 1 -- 172.21.15.94:0/637900341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0dc15bcc0 msgr2=0x7ff0dc152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:50.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e0c00640 1 --2- 172.21.15.94:0/637900341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0dc15bcc0 0x7ff0dc152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e0c00640 1 -- 172.21.15.94:0/637900341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0dc068c30 msgr2=0x7ff0dc0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:50.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e0c00640 1 --2- 172.21.15.94:0/637900341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0dc068c30 0x7ff0dc0b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e0c00640 1 -- 172.21.15.94:0/637900341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0dc0ce390 con 0x7ff0dc0cfda0 2024-09-10T23:39:50.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e2000640 1 --2- 172.21.15.94:0/637900341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0dc15bcc0 0x7ff0dc152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:50.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e0c00640 1 --2- 172.21.15.94:0/637900341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0cfda0 0x7ff0dc152570 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800c4e0 tx=0x7ff0c800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:50.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0d9600640 1 -- 172.21.15.94:0/637900341 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c80052d0 con 0x7ff0dc0cfda0 2024-09-10T23:39:50.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0d9600640 1 -- 172.21.15.94:0/637900341 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0c8005470 con 0x7ff0dc0cfda0 2024-09-10T23:39:50.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0d9600640 1 -- 172.21.15.94:0/637900341 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8012890 con 0x7ff0dc0cfda0 2024-09-10T23:39:50.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e4000640 1 -- 172.21.15.94:0/637900341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0dc0b2bb0 con 0x7ff0dc0cfda0 2024-09-10T23:39:50.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e4000640 1 -- 172.21.15.94:0/637900341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0dc0c31f0 con 0x7ff0dc0cfda0 2024-09-10T23:39:50.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0e4000640 1 -- 172.21.15.94:0/637900341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0dc0ad060 con 0x7ff0dc0cfda0 2024-09-10T23:39:50.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.723+0000 7ff0d9600640 1 -- 172.21.15.94:0/637900341 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7ff0c8003600 con 0x7ff0dc0cfda0 2024-09-10T23:39:50.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.727+0000 7ff0d9600640 1 --2- 172.21.15.94:0/637900341 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff0bc072e80 0x7ff0bc075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:50.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.727+0000 7ff0d9600640 1 -- 172.21.15.94:0/637900341 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5824+0+0 (secure 0 0 0) 0x7ff0c80915e0 con 0x7ff0dc0cfda0 2024-09-10T23:39:50.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.727+0000 7ff0e1600640 1 --2- 172.21.15.94:0/637900341 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff0bc072e80 0x7ff0bc075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:50.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.727+0000 7ff0e1600640 1 --2- 172.21.15.94:0/637900341 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff0bc072e80 0x7ff0bc075340 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00af000 tx=0x7ff0d001b350 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:50.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.731+0000 7ff0d9600640 1 -- 172.21.15.94:0/637900341 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff0c805e730 con 0x7ff0dc0cfda0 2024-09-10T23:39:50.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.895+0000 7ff0e4000640 1 -- 172.21.15.94:0/637900341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0dc0afb10 con 0x7ff0dc0cfda0 2024-09-10T23:39:50.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.899+0000 7ff0d9600640 1 -- 172.21.15.94:0/637900341 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+16916 (secure 0 0 0) 0x7ff0c805e030 con 0x7ff0dc0cfda0 2024-09-10T23:39:50.902 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:50.902 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":140,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:39:38.844662+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:39:22.673102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T23:39:23.946666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.d","mappings":[{"from":5,"to":3}]},{"pgid":"23.1a","mappings":[{"from":5,"to":7}]},{"pgid":"23.3c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:39:50.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.899+0000 7ff0e4000640 1 -- 172.21.15.94:0/637900341 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff0bc072e80 msgr2=0x7ff0bc075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:50.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.899+0000 7ff0e4000640 1 --2- 172.21.15.94:0/637900341 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff0bc072e80 0x7ff0bc075340 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00af000 tx=0x7ff0d001b350 comp rx=0 tx=0).stop 2024-09-10T23:39:50.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.899+0000 7ff0e4000640 1 -- 172.21.15.94:0/637900341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0cfda0 msgr2=0x7ff0dc152570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:50.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.899+0000 7ff0e4000640 1 --2- 172.21.15.94:0/637900341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0cfda0 0x7ff0dc152570 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800c4e0 tx=0x7ff0c800c9a0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.899+0000 7ff0e4000640 1 -- 172.21.15.94:0/637900341 shutdown_connections 2024-09-10T23:39:50.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.899+0000 7ff0e4000640 1 --2- 172.21.15.94:0/637900341 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff0bc072e80 0x7ff0bc075340 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.899+0000 7ff0e4000640 1 --2- 172.21.15.94:0/637900341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff0dc15bcc0 0x7ff0dc152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.899+0000 7ff0e4000640 1 --2- 172.21.15.94:0/637900341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0dc0cfda0 0x7ff0dc152570 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.899+0000 7ff0e4000640 1 --2- 172.21.15.94:0/637900341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff0dc068c30 0x7ff0dc0b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:50.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.899+0000 7ff0e4000640 1 -- 172.21.15.94:0/637900341 >> 172.21.15.94:0/637900341 conn(0x7ff0dc0d02d0 msgr2=0x7ff0dc0995b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:50.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.899+0000 7ff0e4000640 1 -- 172.21.15.94:0/637900341 shutdown_connections 2024-09-10T23:39:50.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:50.899+0000 7ff0e4000640 1 -- 172.21.15.94:0/637900341 wait complete. 2024-09-10T23:39:50.924 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T23:39:51.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.063+0000 7fe5ec400640 1 -- 172.21.15.94:0/2140911977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e40cfce0 msgr2=0x7fe5e40d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:51.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.063+0000 7fe5ec400640 1 --2- 172.21.15.94:0/2140911977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e40cfce0 0x7fe5e40d0120 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009930 tx=0x7fe5d001c850 comp rx=0 tx=0).stop 2024-09-10T23:39:51.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.063+0000 7fe5ec400640 1 -- 172.21.15.94:0/2140911977 shutdown_connections 2024-09-10T23:39:51.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.063+0000 7fe5ec400640 1 --2- 172.21.15.94:0/2140911977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e40cfce0 0x7fe5e40d0120 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:51.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.063+0000 7fe5ec400640 1 --2- 172.21.15.94:0/2140911977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e415bf60 0x7fe5e40d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:51.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.063+0000 7fe5ec400640 1 -- 172.21.15.94:0/2140911977 >> 172.21.15.94:0/2140911977 conn(0x7fe5e40d1c60 msgr2=0x7fe5e40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:51.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.063+0000 7fe5ec400640 1 -- 172.21.15.94:0/2140911977 shutdown_connections 2024-09-10T23:39:51.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.063+0000 7fe5ec400640 1 -- 172.21.15.94:0/2140911977 wait complete. 2024-09-10T23:39:51.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.063+0000 7fe5ec400640 1 Processor -- start 2024-09-10T23:39:51.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.063+0000 7fe5ec400640 1 -- start start 2024-09-10T23:39:51.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5ec400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e40cfce0 0x7fe5e4159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:51.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5ec400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40d0660 0x7fe5e40b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:51.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5ec400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e415bf60 0x7fe5e40b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:51.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5ec400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4168e20 con 0x7fe5e40cfce0 2024-09-10T23:39:51.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5ec400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4168ca0 con 0x7fe5e40d0660 2024-09-10T23:39:51.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5ec400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe5e4168fa0 con 0x7fe5e415bf60 2024-09-10T23:39:51.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5e9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e40cfce0 0x7fe5e4159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:51.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5e9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e40cfce0 0x7fe5e4159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53100/0 (socket says 172.21.15.94:53100) 2024-09-10T23:39:51.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5e9a00640 1 -- 172.21.15.94:0/2861118547 learned_addr learned my addr 172.21.15.94:0/2861118547 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:51.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5ea400640 1 --2- 172.21.15.94:0/2861118547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e415bf60 0x7fe5e40b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:51.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5e9000640 1 --2- 172.21.15.94:0/2861118547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40d0660 0x7fe5e40b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:51.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5e9a00640 1 -- 172.21.15.94:0/2861118547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e415bf60 msgr2=0x7fe5e40b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:51.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5e9a00640 1 --2- 172.21.15.94:0/2861118547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e415bf60 0x7fe5e40b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:51.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5e9a00640 1 -- 172.21.15.94:0/2861118547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40d0660 msgr2=0x7fe5e40b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:51.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5e9a00640 1 --2- 172.21.15.94:0/2861118547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40d0660 0x7fe5e40b74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:51.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5e9a00640 1 -- 172.21.15.94:0/2861118547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e415c7b0 con 0x7fe5e40cfce0 2024-09-10T23:39:51.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5e9000640 1 --2- 172.21.15.94:0/2861118547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40d0660 0x7fe5e40b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:51.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5e9a00640 1 --2- 172.21.15.94:0/2861118547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e40cfce0 0x7fe5e4159930 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800d750 tx=0x7fe5d800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:51.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5e1600640 1 -- 172.21.15.94:0/2861118547 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8017070 con 0x7fe5e40cfce0 2024-09-10T23:39:51.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5e1600640 1 -- 172.21.15.94:0/2861118547 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5d8009d20 con 0x7fe5e40cfce0 2024-09-10T23:39:51.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5e1600640 1 -- 172.21.15.94:0/2861118547 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8005280 con 0x7fe5e40cfce0 2024-09-10T23:39:51.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5ec400640 1 -- 172.21.15.94:0/2861118547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e4158e50 con 0x7fe5e40cfce0 2024-09-10T23:39:51.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5ec400640 1 -- 172.21.15.94:0/2861118547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e4159410 con 0x7fe5e40cfce0 2024-09-10T23:39:51.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5ec400640 1 -- 172.21.15.94:0/2861118547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e40d0120 con 0x7fe5e40cfce0 2024-09-10T23:39:51.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.067+0000 7fe5e1600640 1 -- 172.21.15.94:0/2861118547 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fe5d8005420 con 0x7fe5e40cfce0 2024-09-10T23:39:51.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.071+0000 7fe5e1600640 1 --2- 172.21.15.94:0/2861118547 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5c4072e50 0x7fe5c4075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:51.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.071+0000 7fe5e1600640 1 -- 172.21.15.94:0/2861118547 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5824+0+0 (secure 0 0 0) 0x7fe5d80946e0 con 0x7fe5e40cfce0 2024-09-10T23:39:51.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.071+0000 7fe5e9000640 1 --2- 172.21.15.94:0/2861118547 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5c4072e50 0x7fe5c4075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:51.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.071+0000 7fe5e9000640 1 --2- 172.21.15.94:0/2861118547 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5c4072e50 0x7fe5c4075310 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00af000 tx=0x7fe5d001b090 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:51.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.075+0000 7fe5e1600640 1 -- 172.21.15.94:0/2861118547 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe5d800de60 con 0x7fe5e40cfce0 2024-09-10T23:39:51.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.271+0000 7fe5ec400640 1 -- 172.21.15.94:0/2861118547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe5e415af50 con 0x7fe5e40cfce0 2024-09-10T23:39:51.278 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-10T23:39:51.275+0000 7f4045a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:39:51.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.275+0000 7fe5e1600640 1 -- 172.21.15.94:0/2861118547 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v99) ==== 161+0+0 (secure 0 0 0) 0x7fe5d805ed60 con 0x7fe5e40cfce0 2024-09-10T23:39:51.280 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T23:39:51.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.279+0000 7fe5ec400640 1 -- 172.21.15.94:0/2861118547 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5c4072e50 msgr2=0x7fe5c4075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:51.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.279+0000 7fe5ec400640 1 --2- 172.21.15.94:0/2861118547 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5c4072e50 0x7fe5c4075310 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00af000 tx=0x7fe5d001b090 comp rx=0 tx=0).stop 2024-09-10T23:39:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.279+0000 7fe5ec400640 1 -- 172.21.15.94:0/2861118547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e40cfce0 msgr2=0x7fe5e4159930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.279+0000 7fe5ec400640 1 --2- 172.21.15.94:0/2861118547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e40cfce0 0x7fe5e4159930 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800d750 tx=0x7fe5d800dc10 comp rx=0 tx=0).stop 2024-09-10T23:39:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.279+0000 7fe5ec400640 1 -- 172.21.15.94:0/2861118547 shutdown_connections 2024-09-10T23:39:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.279+0000 7fe5ec400640 1 --2- 172.21.15.94:0/2861118547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40d0660 0x7fe5e40b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.279+0000 7fe5ec400640 1 --2- 172.21.15.94:0/2861118547 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe5c4072e50 0x7fe5c4075310 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.279+0000 7fe5ec400640 1 --2- 172.21.15.94:0/2861118547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e415bf60 0x7fe5e40b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.279+0000 7fe5ec400640 1 --2- 172.21.15.94:0/2861118547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5e40cfce0 0x7fe5e4159930 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.279+0000 7fe5ec400640 1 -- 172.21.15.94:0/2861118547 >> 172.21.15.94:0/2861118547 conn(0x7fe5e40d1c60 msgr2=0x7fe5e4071c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.279+0000 7fe5ec400640 1 -- 172.21.15.94:0/2861118547 shutdown_connections 2024-09-10T23:39:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.279+0000 7fe5ec400640 1 -- 172.21.15.94:0/2861118547 wait complete. 2024-09-10T23:39:51.303 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T23:39:51.349 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:39:51.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.443+0000 7f6e52600640 1 -- 172.21.15.94:0/1003544765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e30004d50 msgr2=0x7f6e300051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:51.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.443+0000 7f6e52600640 1 --2- 172.21.15.94:0/1003544765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e30004d50 0x7f6e300051d0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400c660 tx=0x7f6e4401fa40 comp rx=0 tx=0).stop 2024-09-10T23:39:51.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.443+0000 7f6e4be00640 1 -- 172.21.15.94:0/1003544765 reap_dead start 2024-09-10T23:39:51.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e52600640 1 -- 172.21.15.94:0/1003544765 shutdown_connections 2024-09-10T23:39:51.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e52600640 1 -- 172.21.15.94:0/1003544765 >> 172.21.15.94:0/1003544765 conn(0x7f6e4c0d1c60 msgr2=0x7f6e4c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:51.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e52600640 1 -- 172.21.15.94:0/1003544765 shutdown_connections 2024-09-10T23:39:51.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e52600640 1 -- 172.21.15.94:0/1003544765 wait complete. 2024-09-10T23:39:51.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e52600640 1 Processor -- start 2024-09-10T23:39:51.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e52600640 1 -- start start 2024-09-10T23:39:51.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e52600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e30004d50 0x7f6e4c1508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:51.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e52600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e4c0c5460 0x7f6e4c0b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:51.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e52600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e4c1530a0 0x7f6e4c153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:51.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e52600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c0cdaa0 con 0x7f6e30004d50 2024-09-10T23:39:51.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e52600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c0cd920 con 0x7f6e4c0c5460 2024-09-10T23:39:51.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e52600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6e4c1610d0 con 0x7f6e4c1530a0 2024-09-10T23:39:51.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e4b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e30004d50 0x7f6e4c1508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:51.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e4b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e30004d50 0x7f6e4c1508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53130/0 (socket says 172.21.15.94:53130) 2024-09-10T23:39:51.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e4b400640 1 -- 172.21.15.94:0/3590796065 learned_addr learned my addr 172.21.15.94:0/3590796065 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:51.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e43e00640 1 --2- 172.21.15.94:0/3590796065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e4c0c5460 0x7f6e4c0b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:51.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e4be00640 1 --2- 172.21.15.94:0/3590796065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e4c1530a0 0x7f6e4c153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:51.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e4b400640 1 -- 172.21.15.94:0/3590796065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e4c1530a0 msgr2=0x7f6e4c153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:51.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e4b400640 1 --2- 172.21.15.94:0/3590796065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e4c1530a0 0x7f6e4c153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:51.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e4b400640 1 -- 172.21.15.94:0/3590796065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e4c0c5460 msgr2=0x7f6e4c0b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:51.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e4b400640 1 --2- 172.21.15.94:0/3590796065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e4c0c5460 0x7f6e4c0b6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:51.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e4b400640 1 -- 172.21.15.94:0/3590796065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4c0b7530 con 0x7f6e30004d50 2024-09-10T23:39:51.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e4be00640 1 --2- 172.21.15.94:0/3590796065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e4c1530a0 0x7f6e4c153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:51.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e4b400640 1 --2- 172.21.15.94:0/3590796065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e30004d50 0x7f6e4c1508d0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c011b10 tx=0x7f6e3c009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:51.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e43400640 1 -- 172.21.15.94:0/3590796065 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c01e070 con 0x7f6e30004d50 2024-09-10T23:39:51.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.447+0000 7f6e43400640 1 -- 172.21.15.94:0/3590796065 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e3c004050 con 0x7f6e30004d50 2024-09-10T23:39:51.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.451+0000 7f6e43400640 1 -- 172.21.15.94:0/3590796065 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c00fea0 con 0x7f6e30004d50 2024-09-10T23:39:51.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.451+0000 7f6e52600640 1 -- 172.21.15.94:0/3590796065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e4c0c58e0 con 0x7f6e30004d50 2024-09-10T23:39:51.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.451+0000 7f6e52600640 1 -- 172.21.15.94:0/3590796065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e4c0b6230 con 0x7f6e30004d50 2024-09-10T23:39:51.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.451+0000 7f6e52600640 1 -- 172.21.15.94:0/3590796065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e4c0b15b0 con 0x7f6e30004d50 2024-09-10T23:39:51.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.451+0000 7f6e43400640 1 -- 172.21.15.94:0/3590796065 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f6e3c003600 con 0x7f6e30004d50 2024-09-10T23:39:51.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.451+0000 7f6e43400640 1 --2- 172.21.15.94:0/3590796065 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6e28072e80 0x7f6e28075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:51.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.451+0000 7f6e43400640 1 -- 172.21.15.94:0/3590796065 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5910+0+0 (secure 0 0 0) 0x7f6e3c0952b0 con 0x7f6e30004d50 2024-09-10T23:39:51.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.451+0000 7f6e43e00640 1 --2- 172.21.15.94:0/3590796065 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6e28072e80 0x7f6e28075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:51.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.451+0000 7f6e43e00640 1 --2- 172.21.15.94:0/3590796065 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6e28072e80 0x7f6e28075340 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0b7c10 tx=0x7f6e34006b80 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:51.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.455+0000 7f6e43400640 1 -- 172.21.15.94:0/3590796065 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6e3c05f390 con 0x7f6e30004d50 2024-09-10T23:39:51.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:51.655+0000 7f6e52600640 1 -- 172.21.15.94:0/3590796065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6e4c071030 con 0x7f6e30004d50 2024-09-10T23:39:52.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.279+0000 7f6e43400640 1 -- 172.21.15.94:0/3590796065 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v100) ==== 104+0+0 (secure 0 0 0) 0x7f6e3c068100 con 0x7f6e30004d50 2024-09-10T23:39:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.279+0000 7f6e52600640 1 -- 172.21.15.94:0/3590796065 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6e28072e80 msgr2=0x7f6e28075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.279+0000 7f6e52600640 1 --2- 172.21.15.94:0/3590796065 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6e28072e80 0x7f6e28075340 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0b7c10 tx=0x7f6e34006b80 comp rx=0 tx=0).stop 2024-09-10T23:39:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.279+0000 7f6e52600640 1 -- 172.21.15.94:0/3590796065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e30004d50 msgr2=0x7f6e4c1508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.279+0000 7f6e52600640 1 --2- 172.21.15.94:0/3590796065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e30004d50 0x7f6e4c1508d0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c011b10 tx=0x7f6e3c009ea0 comp rx=0 tx=0).stop 2024-09-10T23:39:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.279+0000 7f6e52600640 1 -- 172.21.15.94:0/3590796065 shutdown_connections 2024-09-10T23:39:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.279+0000 7f6e52600640 1 --2- 172.21.15.94:0/3590796065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e4c0c5460 0x7f6e4c0b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.279+0000 7f6e52600640 1 --2- 172.21.15.94:0/3590796065 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6e28072e80 0x7f6e28075340 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.279+0000 7f6e52600640 1 --2- 172.21.15.94:0/3590796065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e4c1530a0 0x7f6e4c153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.279+0000 7f6e52600640 1 --2- 172.21.15.94:0/3590796065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e30004d50 0x7f6e4c1508d0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.279+0000 7f6e52600640 1 -- 172.21.15.94:0/3590796065 >> 172.21.15.94:0/3590796065 conn(0x7f6e4c0d1c60 msgr2=0x7f6e4c15f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:52.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.279+0000 7f6e52600640 1 -- 172.21.15.94:0/3590796065 shutdown_connections 2024-09-10T23:39:52.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.279+0000 7f6e52600640 1 -- 172.21.15.94:0/3590796065 wait complete. 2024-09-10T23:39:52.304 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T23:39:52.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee27400640 1 -- 172.21.15.94:0/13911913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee200cebd0 msgr2=0x7fee200ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:52.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee27400640 1 --2- 172.21.15.94:0/13911913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee200cebd0 0x7fee200ccbf0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fee18009c80 tx=0x7fee1801cda0 comp rx=0 tx=0).stop 2024-09-10T23:39:52.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee27400640 1 -- 172.21.15.94:0/13911913 shutdown_connections 2024-09-10T23:39:52.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee27400640 1 --2- 172.21.15.94:0/13911913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee200cebd0 0x7fee200ccbf0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:52.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee27400640 1 --2- 172.21.15.94:0/13911913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee2015f370 0x7fee200ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:52.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee27400640 1 -- 172.21.15.94:0/13911913 >> 172.21.15.94:0/13911913 conn(0x7fee200d1c80 msgr2=0x7fee200d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:52.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee27400640 1 -- 172.21.15.94:0/13911913 shutdown_connections 2024-09-10T23:39:52.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee27400640 1 -- 172.21.15.94:0/13911913 wait complete. 2024-09-10T23:39:52.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee27400640 1 Processor -- start 2024-09-10T23:39:52.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee27400640 1 -- start start 2024-09-10T23:39:52.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee27400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee200b0ec0 0x7fee20150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:52.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee27400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee200cebd0 0x7fee201571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:52.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee27400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee2015f370 0x7fee201f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:52.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee27400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fee20071db0 con 0x7fee2015f370 2024-09-10T23:39:52.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee27400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fee201691e0 con 0x7fee200b0ec0 2024-09-10T23:39:52.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee27400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fee20169360 con 0x7fee200cebd0 2024-09-10T23:39:52.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee25400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee2015f370 0x7fee201f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:52.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee24a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee200b0ec0 0x7fee20150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:52.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee25400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee2015f370 0x7fee201f5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53156/0 (socket says 172.21.15.94:53156) 2024-09-10T23:39:52.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee25400640 1 -- 172.21.15.94:0/557325740 learned_addr learned my addr 172.21.15.94:0/557325740 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:52.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee17e00640 1 --2- 172.21.15.94:0/557325740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee200cebd0 0x7fee201571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:52.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee25400640 1 -- 172.21.15.94:0/557325740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee200cebd0 msgr2=0x7fee201571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:52.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee25400640 1 --2- 172.21.15.94:0/557325740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee200cebd0 0x7fee201571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:52.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee25400640 1 -- 172.21.15.94:0/557325740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee200b0ec0 msgr2=0x7fee20150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:52.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee25400640 1 --2- 172.21.15.94:0/557325740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee200b0ec0 0x7fee20150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:52.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee25400640 1 -- 172.21.15.94:0/557325740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee201f5ad0 con 0x7fee2015f370 2024-09-10T23:39:52.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.447+0000 7fee17e00640 1 --2- 172.21.15.94:0/557325740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee200cebd0 0x7fee201571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:52.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.451+0000 7fee25400640 1 --2- 172.21.15.94:0/557325740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee2015f370 0x7fee201f5fb0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fee18009bd0 tx=0x7fee18009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:52.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.451+0000 7fee1d600640 1 -- 172.21.15.94:0/557325740 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180bb070 con 0x7fee2015f370 2024-09-10T23:39:52.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.451+0000 7fee1d600640 1 -- 172.21.15.94:0/557325740 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee18004d10 con 0x7fee2015f370 2024-09-10T23:39:52.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.451+0000 7fee1d600640 1 -- 172.21.15.94:0/557325740 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180b2040 con 0x7fee2015f370 2024-09-10T23:39:52.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.451+0000 7fee27400640 1 -- 172.21.15.94:0/557325740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee20157dd0 con 0x7fee2015f370 2024-09-10T23:39:52.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.451+0000 7fee27400640 1 -- 172.21.15.94:0/557325740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee20150d60 con 0x7fee2015f370 2024-09-10T23:39:52.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.451+0000 7fee27400640 1 -- 172.21.15.94:0/557325740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee201f5ca0 con 0x7fee2015f370 2024-09-10T23:39:52.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.451+0000 7fee1d600640 1 -- 172.21.15.94:0/557325740 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fee180b5030 con 0x7fee2015f370 2024-09-10T23:39:52.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.451+0000 7fee1d600640 1 --2- 172.21.15.94:0/557325740 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fedfc072e50 0x7fedfc075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:52.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.451+0000 7fee1d600640 1 -- 172.21.15.94:0/557325740 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5910+0+0 (secure 0 0 0) 0x7fee1812dbc0 con 0x7fee2015f370 2024-09-10T23:39:52.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.451+0000 7fee24a00640 1 --2- 172.21.15.94:0/557325740 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fedfc072e50 0x7fedfc075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:52.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.455+0000 7fee24a00640 1 --2- 172.21.15.94:0/557325740 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fedfc072e50 0x7fedfc075310 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fee10004630 tx=0x7fee1000b2e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:52.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.455+0000 7fee1d600640 1 -- 172.21.15.94:0/557325740 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fee18016990 con 0x7fee2015f370 2024-09-10T23:39:52.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:52.623+0000 7fee27400640 1 -- 172.21.15.94:0/557325740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fee201f58c0 con 0x7fee2015f370 2024-09-10T23:39:53.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.287+0000 7fee1d600640 1 -- 172.21.15.94:0/557325740 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v142) ==== 181+0+0 (secure 0 0 0) 0x7fee180f7ff0 con 0x7fee2015f370 2024-09-10T23:39:53.291 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-09-10T23:39:53.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.291+0000 7fee27400640 1 -- 172.21.15.94:0/557325740 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fedfc072e50 msgr2=0x7fedfc075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:53.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.291+0000 7fee27400640 1 --2- 172.21.15.94:0/557325740 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fedfc072e50 0x7fedfc075310 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fee10004630 tx=0x7fee1000b2e0 comp rx=0 tx=0).stop 2024-09-10T23:39:53.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.291+0000 7fee27400640 1 -- 172.21.15.94:0/557325740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee2015f370 msgr2=0x7fee201f5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:53.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.291+0000 7fee27400640 1 --2- 172.21.15.94:0/557325740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee2015f370 0x7fee201f5fb0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fee18009bd0 tx=0x7fee18009390 comp rx=0 tx=0).stop 2024-09-10T23:39:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.291+0000 7fee27400640 1 -- 172.21.15.94:0/557325740 shutdown_connections 2024-09-10T23:39:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.291+0000 7fee27400640 1 --2- 172.21.15.94:0/557325740 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fedfc072e50 0x7fedfc075310 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.291+0000 7fee27400640 1 --2- 172.21.15.94:0/557325740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee2015f370 0x7fee201f5fb0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.291+0000 7fee27400640 1 --2- 172.21.15.94:0/557325740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee200cebd0 0x7fee201571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.291+0000 7fee27400640 1 --2- 172.21.15.94:0/557325740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee200b0ec0 0x7fee20150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.291+0000 7fee27400640 1 -- 172.21.15.94:0/557325740 >> 172.21.15.94:0/557325740 conn(0x7fee200d1c80 msgr2=0x7fee200cc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:53.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.291+0000 7fee27400640 1 -- 172.21.15.94:0/557325740 shutdown_connections 2024-09-10T23:39:53.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.291+0000 7fee27400640 1 -- 172.21.15.94:0/557325740 wait complete. 2024-09-10T23:39:53.316 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T23:39:53.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.459+0000 7f150d800640 1 -- 172.21.15.94:0/23510786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14ec0041d0 msgr2=0x7f14ec004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:53.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.459+0000 7f150d800640 1 --2- 172.21.15.94:0/23510786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14ec0041d0 0x7f14ec004810 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f14f4001830 tx=0x7f14f401f010 comp rx=0 tx=0).stop 2024-09-10T23:39:53.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.459+0000 7f1507400640 1 -- 172.21.15.94:0/23510786 reap_dead start 2024-09-10T23:39:53.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.459+0000 7f150d800640 1 -- 172.21.15.94:0/23510786 shutdown_connections 2024-09-10T23:39:53.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.459+0000 7f150d800640 1 -- 172.21.15.94:0/23510786 >> 172.21.15.94:0/23510786 conn(0x7f150809aa70 msgr2=0x7f150809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:53.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.459+0000 7f150d800640 1 -- 172.21.15.94:0/23510786 shutdown_connections 2024-09-10T23:39:53.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.459+0000 7f150d800640 1 -- 172.21.15.94:0/23510786 wait complete. 2024-09-10T23:39:53.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f150d800640 1 Processor -- start 2024-09-10T23:39:53.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f150d800640 1 -- start start 2024-09-10T23:39:53.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f150d800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14ec0041d0 0x7f150815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:53.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f150d800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15080c7e30 0x7f15080ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:53.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f150d800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1508159f40 0x7f150815a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:53.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f150d800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f150815c4a0 con 0x7f15080c7e30 2024-09-10T23:39:53.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f150d800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f150815c320 con 0x7f1508159f40 2024-09-10T23:39:53.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f150d800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f150815c620 con 0x7f14ec0041d0 2024-09-10T23:39:53.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f1506000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15080c7e30 0x7f15080ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:53.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f1506000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15080c7e30 0x7f15080ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53182/0 (socket says 172.21.15.94:53182) 2024-09-10T23:39:53.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f1506000640 1 -- 172.21.15.94:0/970907362 learned_addr learned my addr 172.21.15.94:0/970907362 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:53.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f1507400640 1 --2- 172.21.15.94:0/970907362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1508159f40 0x7f150815a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:53.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f1506a00640 1 --2- 172.21.15.94:0/970907362 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14ec0041d0 0x7f150815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:53.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f1506000640 1 -- 172.21.15.94:0/970907362 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14ec0041d0 msgr2=0x7f150815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:53.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f1506000640 1 --2- 172.21.15.94:0/970907362 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14ec0041d0 0x7f150815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:53.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f1506000640 1 -- 172.21.15.94:0/970907362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1508159f40 msgr2=0x7f150815a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:53.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f1506000640 1 --2- 172.21.15.94:0/970907362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1508159f40 0x7f150815a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:53.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f1506000640 1 -- 172.21.15.94:0/970907362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15080ceb60 con 0x7f15080c7e30 2024-09-10T23:39:53.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f1507400640 1 --2- 172.21.15.94:0/970907362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1508159f40 0x7f150815a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:53.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f1506a00640 1 --2- 172.21.15.94:0/970907362 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14ec0041d0 0x7f150815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:53.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f1506000640 1 --2- 172.21.15.94:0/970907362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15080c7e30 0x7f15080ce620 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f14f4005040 tx=0x7f14f4001480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:53.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f14fea00640 1 -- 172.21.15.94:0/970907362 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4003600 con 0x7f15080c7e30 2024-09-10T23:39:53.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f14fea00640 1 -- 172.21.15.94:0/970907362 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14f40b7070 con 0x7f15080c7e30 2024-09-10T23:39:53.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f150d800640 1 -- 172.21.15.94:0/970907362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15080cee20 con 0x7f15080c7e30 2024-09-10T23:39:53.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f14fea00640 1 -- 172.21.15.94:0/970907362 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4003ad0 con 0x7f15080c7e30 2024-09-10T23:39:53.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.463+0000 7f150d800640 1 -- 172.21.15.94:0/970907362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15080b4cf0 con 0x7f15080c7e30 2024-09-10T23:39:53.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.467+0000 7f150d800640 1 -- 172.21.15.94:0/970907362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15080d1b80 con 0x7f15080c7e30 2024-09-10T23:39:53.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.467+0000 7f14fea00640 1 -- 172.21.15.94:0/970907362 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f14f40c4030 con 0x7f15080c7e30 2024-09-10T23:39:53.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.467+0000 7f14fea00640 1 --2- 172.21.15.94:0/970907362 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f14e8072e80 0x7f14e8075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:53.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.467+0000 7f14fea00640 1 -- 172.21.15.94:0/970907362 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5472+0+0 (secure 0 0 0) 0x7f14f4131590 con 0x7f15080c7e30 2024-09-10T23:39:53.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.467+0000 7f1506a00640 1 --2- 172.21.15.94:0/970907362 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f14e8072e80 0x7f14e8075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:53.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.467+0000 7f1506a00640 1 --2- 172.21.15.94:0/970907362 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f14e8072e80 0x7f14e8075340 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009b10 tx=0x7f14f80091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:53.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.471+0000 7f14fea00640 1 -- 172.21.15.94:0/970907362 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f14f4018dc0 con 0x7f15080c7e30 2024-09-10T23:39:53.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:53.639+0000 7f150d800640 1 -- 172.21.15.94:0/970907362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f15080d0710 con 0x7f15080c7e30 2024-09-10T23:39:54.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.291+0000 7f14fea00640 1 -- 172.21.15.94:0/970907362 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v143) ==== 169+0+0 (secure 0 0 0) 0x7f14f40fc320 con 0x7f15080c7e30 2024-09-10T23:39:54.293 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-09-10T23:39:54.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.291+0000 7f150d800640 1 -- 172.21.15.94:0/970907362 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f14e8072e80 msgr2=0x7f14e8075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:54.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.291+0000 7f150d800640 1 --2- 172.21.15.94:0/970907362 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f14e8072e80 0x7f14e8075340 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009b10 tx=0x7f14f80091e0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.291+0000 7f150d800640 1 -- 172.21.15.94:0/970907362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15080c7e30 msgr2=0x7f15080ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:54.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.291+0000 7f150d800640 1 --2- 172.21.15.94:0/970907362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15080c7e30 0x7f15080ce620 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f14f4005040 tx=0x7f14f4001480 comp rx=0 tx=0).stop 2024-09-10T23:39:54.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.291+0000 7f150d800640 1 -- 172.21.15.94:0/970907362 shutdown_connections 2024-09-10T23:39:54.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.291+0000 7f150d800640 1 --2- 172.21.15.94:0/970907362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1508159f40 0x7f150815a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.291+0000 7f150d800640 1 --2- 172.21.15.94:0/970907362 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f14e8072e80 0x7f14e8075340 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.291+0000 7f150d800640 1 --2- 172.21.15.94:0/970907362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15080c7e30 0x7f15080ce620 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.291+0000 7f150d800640 1 --2- 172.21.15.94:0/970907362 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14ec0041d0 0x7f150815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.291+0000 7f150d800640 1 -- 172.21.15.94:0/970907362 >> 172.21.15.94:0/970907362 conn(0x7f150809aa70 msgr2=0x7f1508158430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:54.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.295+0000 7f150d800640 1 -- 172.21.15.94:0/970907362 shutdown_connections 2024-09-10T23:39:54.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.295+0000 7f150d800640 1 -- 172.21.15.94:0/970907362 wait complete. 2024-09-10T23:39:54.318 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T23:39:54.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.455+0000 7fa5cee00640 1 -- 172.21.15.94:0/2087069367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c80b2ac0 msgr2=0x7fa5c80b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:54.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.455+0000 7fa5cee00640 1 --2- 172.21.15.94:0/2087069367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c80b2ac0 0x7fa5c80b16b0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8072ee0 tx=0x7fa5b401c890 comp rx=0 tx=0).stop 2024-09-10T23:39:54.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cee00640 1 -- 172.21.15.94:0/2087069367 shutdown_connections 2024-09-10T23:39:54.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cee00640 1 --2- 172.21.15.94:0/2087069367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c80b2ac0 0x7fa5c80b16b0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cee00640 1 --2- 172.21.15.94:0/2087069367 >> v2:172.21.15.181:3300/0 conn(0x7fa5c815f140 0x7fa5c80b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cee00640 1 -- 172.21.15.94:0/2087069367 >> 172.21.15.94:0/2087069367 conn(0x7fa5c80d1b00 msgr2=0x7fa5c80d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:54.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cee00640 1 -- 172.21.15.94:0/2087069367 shutdown_connections 2024-09-10T23:39:54.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cee00640 1 -- 172.21.15.94:0/2087069367 wait complete. 2024-09-10T23:39:54.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cee00640 1 Processor -- start 2024-09-10T23:39:54.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cee00640 1 -- start start 2024-09-10T23:39:54.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cee00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c80b1bf0 0x7fa5c80aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:54.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cee00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5c80b2ac0 0x7fa5c8149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:54.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cee00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5c815f140 0x7fa5c80707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:54.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cee00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8168ad0 con 0x7fa5c80b1bf0 2024-09-10T23:39:54.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cee00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8168950 con 0x7fa5c80b2ac0 2024-09-10T23:39:54.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cee00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c8168c50 con 0x7fa5c815f140 2024-09-10T23:39:54.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cda00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c80b1bf0 0x7fa5c80aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:54.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cda00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c80b1bf0 0x7fa5c80aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39644/0 (socket says 172.21.15.94:39644) 2024-09-10T23:39:54.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cda00640 1 -- 172.21.15.94:0/2542106328 learned_addr learned my addr 172.21.15.94:0/2542106328 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:54.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cda00640 1 -- 172.21.15.94:0/2542106328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5c815f140 msgr2=0x7fa5c80707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:54.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cda00640 1 --2- 172.21.15.94:0/2542106328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5c815f140 0x7fa5c80707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cda00640 1 -- 172.21.15.94:0/2542106328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5c80b2ac0 msgr2=0x7fa5c8149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:54.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cda00640 1 --2- 172.21.15.94:0/2542106328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5c80b2ac0 0x7fa5c8149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cda00640 1 -- 172.21.15.94:0/2542106328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c80af360 con 0x7fa5c80b1bf0 2024-09-10T23:39:54.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.459+0000 7fa5cda00640 1 --2- 172.21.15.94:0/2542106328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c80b1bf0 0x7fa5c80aec10 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0058d0 tx=0x7fa5bc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:54.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.463+0000 7fa5c5600640 1 -- 172.21.15.94:0/2542106328 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc017070 con 0x7fa5c80b1bf0 2024-09-10T23:39:54.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.463+0000 7fa5c5600640 1 -- 172.21.15.94:0/2542106328 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5bc009d20 con 0x7fa5c80b1bf0 2024-09-10T23:39:54.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.463+0000 7fa5c5600640 1 -- 172.21.15.94:0/2542106328 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc0106c0 con 0x7fa5c80b1bf0 2024-09-10T23:39:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.463+0000 7fa5cee00640 1 -- 172.21.15.94:0/2542106328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c80c56d0 con 0x7fa5c80b1bf0 2024-09-10T23:39:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.463+0000 7fa5cee00640 1 -- 172.21.15.94:0/2542106328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c80c5c90 con 0x7fa5c80b1bf0 2024-09-10T23:39:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.463+0000 7fa5cee00640 1 -- 172.21.15.94:0/2542106328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c80cfa00 con 0x7fa5c80b1bf0 2024-09-10T23:39:54.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.463+0000 7fa5c5600640 1 -- 172.21.15.94:0/2542106328 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fa5bc010860 con 0x7fa5c80b1bf0 2024-09-10T23:39:54.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.463+0000 7fa5c5600640 1 --2- 172.21.15.94:0/2542106328 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa5a8072e50 0x7fa5a8075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:54.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.463+0000 7fa5c5600640 1 -- 172.21.15.94:0/2542106328 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4971+0+0 (secure 0 0 0) 0x7fa5bc091f60 con 0x7fa5c80b1bf0 2024-09-10T23:39:54.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.463+0000 7fa5cd000640 1 --2- 172.21.15.94:0/2542106328 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa5a8072e50 0x7fa5a8075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:54.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.467+0000 7fa5cd000640 1 --2- 172.21.15.94:0/2542106328 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa5a8072e50 0x7fa5a8075310 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40af000 tx=0x7fa5b4019630 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:54.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.467+0000 7fa5c5600640 1 -- 172.21.15.94:0/2542106328 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa5bc05f0f0 con 0x7fa5c80b1bf0 2024-09-10T23:39:54.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.643+0000 7fa5cee00640 1 -- 172.21.15.94:0/2542106328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fa5c8154680 con 0x7fa5c80b1bf0 2024-09-10T23:39:54.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.647+0000 7fa5c5600640 1 -- 172.21.15.94:0/2542106328 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v45) ==== 71+0+0 (secure 0 0 0) 0x7fa5bc05e9f0 con 0x7fa5c80b1bf0 2024-09-10T23:39:54.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.651+0000 7fa5cee00640 1 -- 172.21.15.94:0/2542106328 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa5a8072e50 msgr2=0x7fa5a8075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:54.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.651+0000 7fa5cee00640 1 --2- 172.21.15.94:0/2542106328 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa5a8072e50 0x7fa5a8075310 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40af000 tx=0x7fa5b4019630 comp rx=0 tx=0).stop 2024-09-10T23:39:54.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.651+0000 7fa5cee00640 1 -- 172.21.15.94:0/2542106328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c80b1bf0 msgr2=0x7fa5c80aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:54.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.651+0000 7fa5cee00640 1 --2- 172.21.15.94:0/2542106328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c80b1bf0 0x7fa5c80aec10 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0058d0 tx=0x7fa5bc00dde0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.651+0000 7fa5cee00640 1 -- 172.21.15.94:0/2542106328 shutdown_connections 2024-09-10T23:39:54.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.651+0000 7fa5cee00640 1 --2- 172.21.15.94:0/2542106328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5c80b2ac0 0x7fa5c8149f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.651+0000 7fa5cee00640 1 --2- 172.21.15.94:0/2542106328 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa5a8072e50 0x7fa5a8075310 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.651+0000 7fa5cee00640 1 --2- 172.21.15.94:0/2542106328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5c815f140 0x7fa5c80707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.651+0000 7fa5cee00640 1 --2- 172.21.15.94:0/2542106328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5c80b1bf0 0x7fa5c80aec10 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.651+0000 7fa5cee00640 1 -- 172.21.15.94:0/2542106328 >> 172.21.15.94:0/2542106328 conn(0x7fa5c80d1b00 msgr2=0x7fa5c80d1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:54.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.651+0000 7fa5cee00640 1 -- 172.21.15.94:0/2542106328 shutdown_connections 2024-09-10T23:39:54.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.651+0000 7fa5cee00640 1 -- 172.21.15.94:0/2542106328 wait complete. 2024-09-10T23:39:54.673 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-10T23:39:54.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.811+0000 7fac0cc00640 1 -- 172.21.15.94:0/2591051580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabec004810 msgr2=0x7fabec004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:54.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.811+0000 7fac0cc00640 1 --2- 172.21.15.94:0/2591051580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabec004810 0x7fabec004c90 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fabf000c310 tx=0x7fabf001f6c0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.811+0000 7fac0cc00640 1 -- 172.21.15.94:0/2591051580 shutdown_connections 2024-09-10T23:39:54.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.811+0000 7fac0cc00640 1 --2- 172.21.15.94:0/2591051580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabec004810 0x7fabec004c90 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.811+0000 7fac0cc00640 1 -- 172.21.15.94:0/2591051580 >> 172.21.15.94:0/2591051580 conn(0x7fac0806cdf0 msgr2=0x7fac0806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:54.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.811+0000 7fac0cc00640 1 -- 172.21.15.94:0/2591051580 shutdown_connections 2024-09-10T23:39:54.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.811+0000 7fac0cc00640 1 -- 172.21.15.94:0/2591051580 wait complete. 2024-09-10T23:39:54.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac0cc00640 1 Processor -- start 2024-09-10T23:39:54.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac0cc00640 1 -- start start 2024-09-10T23:39:54.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac0cc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabec004810 0x7fac080cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:54.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac0cc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac080ae0c0 0x7fac080c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:54.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac0cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac080c5650 0x7fac08072610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:54.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac0cc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fac080cff20 con 0x7fac080c5650 2024-09-10T23:39:54.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac0cc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fac080cfda0 con 0x7fabec004810 2024-09-10T23:39:54.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac0cc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fac080d00a0 con 0x7fac080ae0c0 2024-09-10T23:39:54.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac06800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac080c5650 0x7fac08072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:54.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac06800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac080c5650 0x7fac08072610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39662/0 (socket says 172.21.15.94:39662) 2024-09-10T23:39:54.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac06800640 1 -- 172.21.15.94:0/899020987 learned_addr learned my addr 172.21.15.94:0/899020987 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:54.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fabffe00640 1 --2- 172.21.15.94:0/899020987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac080ae0c0 0x7fac080c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:54.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac05e00640 1 --2- 172.21.15.94:0/899020987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabec004810 0x7fac080cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:54.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac06800640 1 -- 172.21.15.94:0/899020987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac080ae0c0 msgr2=0x7fac080c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:54.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac06800640 1 --2- 172.21.15.94:0/899020987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac080ae0c0 0x7fac080c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac06800640 1 -- 172.21.15.94:0/899020987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabec004810 msgr2=0x7fac080cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:54.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac06800640 1 --2- 172.21.15.94:0/899020987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabec004810 0x7fac080cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:54.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac06800640 1 -- 172.21.15.94:0/899020987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac080ae550 con 0x7fac080c5650 2024-09-10T23:39:54.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac05e00640 1 --2- 172.21.15.94:0/899020987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabec004810 0x7fac080cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:54.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac06800640 1 --2- 172.21.15.94:0/899020987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac080c5650 0x7fac08072610 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fac080cc5c0 tx=0x7fac000019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:54.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fabfe000640 1 -- 172.21.15.94:0/899020987 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00015020 con 0x7fac080c5650 2024-09-10T23:39:54.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fabfe000640 1 -- 172.21.15.94:0/899020987 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac00011960 con 0x7fac080c5650 2024-09-10T23:39:54.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fabfe000640 1 -- 172.21.15.94:0/899020987 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00011db0 con 0x7fac080c5650 2024-09-10T23:39:54.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac0cc00640 1 -- 172.21.15.94:0/899020987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac080ae830 con 0x7fac080c5650 2024-09-10T23:39:54.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.815+0000 7fac0cc00640 1 -- 172.21.15.94:0/899020987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac080cd5d0 con 0x7fac080c5650 2024-09-10T23:39:54.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.819+0000 7fac0cc00640 1 -- 172.21.15.94:0/899020987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac0815eaf0 con 0x7fac080c5650 2024-09-10T23:39:54.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.819+0000 7fabfe000640 1 -- 172.21.15.94:0/899020987 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fac000103c0 con 0x7fac080c5650 2024-09-10T23:39:54.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.819+0000 7fabfe000640 1 --2- 172.21.15.94:0/899020987 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fabe4072e50 0x7fabe4075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:54.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.819+0000 7fabfe000640 1 -- 172.21.15.94:0/899020987 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4971+0+0 (secure 0 0 0) 0x7fac00094ba0 con 0x7fac080c5650 2024-09-10T23:39:54.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.819+0000 7fac05e00640 1 --2- 172.21.15.94:0/899020987 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fabe4072e50 0x7fabe4075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:54.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.819+0000 7fac05e00640 1 --2- 172.21.15.94:0/899020987 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fabe4072e50 0x7fabe4075310 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fac0806b7e0 tx=0x7fabf80091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:54.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:54.823+0000 7fabfe000640 1 -- 172.21.15.94:0/899020987 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fac0001f030 con 0x7fac080c5650 2024-09-10T23:39:55.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.043+0000 7fac0cc00640 1 -- 172.21.15.94:0/899020987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7fac080af200 con 0x7fac080c5650 2024-09-10T23:39:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.291+0000 7fabfe000640 1 -- 172.21.15.94:0/899020987 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v405) ==== 149+0+0 (secure 0 0 0) 0x7fac0005f5c0 con 0x7fac080c5650 2024-09-10T23:39:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.291+0000 7fac0cc00640 1 -- 172.21.15.94:0/899020987 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fabe4072e50 msgr2=0x7fabe4075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.291+0000 7fac0cc00640 1 --2- 172.21.15.94:0/899020987 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fabe4072e50 0x7fabe4075310 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fac0806b7e0 tx=0x7fabf80091e0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.291+0000 7fac0cc00640 1 -- 172.21.15.94:0/899020987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac080c5650 msgr2=0x7fac08072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:55.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.291+0000 7fac0cc00640 1 --2- 172.21.15.94:0/899020987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac080c5650 0x7fac08072610 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fac080cc5c0 tx=0x7fac000019c0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.295+0000 7fac0cc00640 1 -- 172.21.15.94:0/899020987 shutdown_connections 2024-09-10T23:39:55.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.295+0000 7fac0cc00640 1 --2- 172.21.15.94:0/899020987 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fabe4072e50 0x7fabe4075310 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.295+0000 7fac0cc00640 1 --2- 172.21.15.94:0/899020987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac080c5650 0x7fac08072610 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.295+0000 7fac0cc00640 1 --2- 172.21.15.94:0/899020987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac080ae0c0 0x7fac080c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.295+0000 7fac0cc00640 1 --2- 172.21.15.94:0/899020987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabec004810 0x7fac080cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.295+0000 7fac0cc00640 1 -- 172.21.15.94:0/899020987 >> 172.21.15.94:0/899020987 conn(0x7fac0806cdf0 msgr2=0x7fac080af640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.295+0000 7fac0cc00640 1 -- 172.21.15.94:0/899020987 shutdown_connections 2024-09-10T23:39:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.295+0000 7fac0cc00640 1 -- 172.21.15.94:0/899020987 wait complete. 2024-09-10T23:39:55.318 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- rather than going through a full upgrade cycle, ... ok 2024-09-10T23:39:55.318 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-10T23:39:55.318 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T23:39:55.319 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:39:55.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd3200640 1 -- 172.21.15.94:0/3119810628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdb8004810 msgr2=0x7efdb8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:55.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd3200640 1 --2- 172.21.15.94:0/3119810628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdb8004810 0x7efdb8004c90 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009b90 tx=0x7efdc001d270 comp rx=0 tx=0).stop 2024-09-10T23:39:55.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd2800640 1 -- 172.21.15.94:0/3119810628 reap_dead start 2024-09-10T23:39:55.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd3200640 1 -- 172.21.15.94:0/3119810628 shutdown_connections 2024-09-10T23:39:55.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd3200640 1 -- 172.21.15.94:0/3119810628 >> 172.21.15.94:0/3119810628 conn(0x7efdd409aa70 msgr2=0x7efdd409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:55.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd3200640 1 -- 172.21.15.94:0/3119810628 shutdown_connections 2024-09-10T23:39:55.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd3200640 1 -- 172.21.15.94:0/3119810628 wait complete. 2024-09-10T23:39:55.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd3200640 1 Processor -- start 2024-09-10T23:39:55.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd3200640 1 -- start start 2024-09-10T23:39:55.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd3200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdb8004810 0x7efdd40c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:55.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd3200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdd40c57d0 0x7efdd41f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:55.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd3200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdd40cd8b0 0x7efdd415a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:55.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd3200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efdd40b34d0 con 0x7efdb8004810 2024-09-10T23:39:55.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd3200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efdd40b3350 con 0x7efdd40cd8b0 2024-09-10T23:39:55.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd3200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7efdd40d01e0 con 0x7efdd40c57d0 2024-09-10T23:39:55.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdb8004810 0x7efdd40c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdb8004810 0x7efdd40c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39672/0 (socket says 172.21.15.94:39672) 2024-09-10T23:39:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd1e00640 1 -- 172.21.15.94:0/3152917673 learned_addr learned my addr 172.21.15.94:0/3152917673 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.459+0000 7efdd2800640 1 --2- 172.21.15.94:0/3152917673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdd40cd8b0 0x7efdd415a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.463+0000 7efdd1e00640 1 -- 172.21.15.94:0/3152917673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdd40c57d0 msgr2=0x7efdd41f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.463+0000 7efdd1e00640 1 --2- 172.21.15.94:0/3152917673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdd40c57d0 0x7efdd41f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.463+0000 7efdd1e00640 1 -- 172.21.15.94:0/3152917673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdd40cd8b0 msgr2=0x7efdd415a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.463+0000 7efdd1e00640 1 --2- 172.21.15.94:0/3152917673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdd40cd8b0 0x7efdd415a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.463+0000 7efdd1e00640 1 -- 172.21.15.94:0/3152917673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd40c86a0 con 0x7efdb8004810 2024-09-10T23:39:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.463+0000 7efdd1e00640 1 --2- 172.21.15.94:0/3152917673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdb8004810 0x7efdd40c7fc0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7efdc40119a0 tx=0x7efdc4011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.463+0000 7efdca000640 1 -- 172.21.15.94:0/3152917673 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4017070 con 0x7efdb8004810 2024-09-10T23:39:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.463+0000 7efdca000640 1 -- 172.21.15.94:0/3152917673 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdc400fe10 con 0x7efdb8004810 2024-09-10T23:39:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.463+0000 7efdd3200640 1 -- 172.21.15.94:0/3152917673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd40c8980 con 0x7efdb8004810 2024-09-10T23:39:55.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.463+0000 7efdca000640 1 -- 172.21.15.94:0/3152917673 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc40136b0 con 0x7efdb8004810 2024-09-10T23:39:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.463+0000 7efdd3200640 1 -- 172.21.15.94:0/3152917673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd41f59e0 con 0x7efdb8004810 2024-09-10T23:39:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.463+0000 7efdd3200640 1 -- 172.21.15.94:0/3152917673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd40cfa40 con 0x7efdb8004810 2024-09-10T23:39:55.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.463+0000 7efdca000640 1 -- 172.21.15.94:0/3152917673 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7efdc4010040 con 0x7efdb8004810 2024-09-10T23:39:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.463+0000 7efdca000640 1 --2- 172.21.15.94:0/3152917673 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efdb4072e50 0x7efdb4075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.463+0000 7efdca000640 1 -- 172.21.15.94:0/3152917673 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4971+0+0 (secure 0 0 0) 0x7efdc4097c00 con 0x7efdb8004810 2024-09-10T23:39:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.467+0000 7efdd1400640 1 --2- 172.21.15.94:0/3152917673 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efdb4072e50 0x7efdb4075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:55.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.467+0000 7efdd1400640 1 --2- 172.21.15.94:0/3152917673 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efdb4072e50 0x7efdb4075310 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efdc00ae000 tx=0x7efdb8005840 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:55.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.467+0000 7efdca000640 1 -- 172.21.15.94:0/3152917673 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efdc4099e20 con 0x7efdb8004810 2024-09-10T23:39:55.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.635+0000 7efdd3200640 1 -- 172.21.15.94:0/3152917673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efdd40b1b50 con 0x7efdb8004810 2024-09-10T23:39:55.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.635+0000 7efdca000640 1 -- 172.21.15.94:0/3152917673 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) ==== 74+0+13076 (secure 0 0 0) 0x7efdc4062350 con 0x7efdb8004810 2024-09-10T23:39:55.639 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:55.639 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":143,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:39:54.293171+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6801/2882789540":"2024-09-11T23:39:51.279572+0000","172.21.15.181:6800/2882789540":"2024-09-11T23:39:51.279572+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:39:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.635+0000 7efdd3200640 1 -- 172.21.15.94:0/3152917673 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efdb4072e50 msgr2=0x7efdb4075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.635+0000 7efdd3200640 1 --2- 172.21.15.94:0/3152917673 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efdb4072e50 0x7efdb4075310 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efdc00ae000 tx=0x7efdb8005840 comp rx=0 tx=0).stop 2024-09-10T23:39:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.635+0000 7efdd3200640 1 -- 172.21.15.94:0/3152917673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdb8004810 msgr2=0x7efdd40c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.635+0000 7efdd3200640 1 --2- 172.21.15.94:0/3152917673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdb8004810 0x7efdd40c7fc0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7efdc40119a0 tx=0x7efdc4011e60 comp rx=0 tx=0).stop 2024-09-10T23:39:55.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.635+0000 7efdd3200640 1 -- 172.21.15.94:0/3152917673 shutdown_connections 2024-09-10T23:39:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.635+0000 7efdd3200640 1 --2- 172.21.15.94:0/3152917673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdd40cd8b0 0x7efdd415a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.635+0000 7efdd3200640 1 --2- 172.21.15.94:0/3152917673 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7efdb4072e50 0x7efdb4075310 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.635+0000 7efdd3200640 1 --2- 172.21.15.94:0/3152917673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdd40c57d0 0x7efdd41f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.635+0000 7efdd3200640 1 --2- 172.21.15.94:0/3152917673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efdb8004810 0x7efdd40c7fc0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.635+0000 7efdd3200640 1 -- 172.21.15.94:0/3152917673 >> 172.21.15.94:0/3152917673 conn(0x7efdd409aa70 msgr2=0x7efdd40cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.639+0000 7efdd3200640 1 -- 172.21.15.94:0/3152917673 shutdown_connections 2024-09-10T23:39:55.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.639+0000 7efdd3200640 1 -- 172.21.15.94:0/3152917673 wait complete. 2024-09-10T23:39:55.662 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T23:32:31.372521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T23:39:55.662 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T23:39:55.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe673a00640 1 -- 172.21.15.94:0/317815037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe66c0b1100 msgr2=0x7fe66c0b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:55.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe673a00640 1 --2- 172.21.15.94:0/317815037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe66c0b1100 0x7fe66c0b1590 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fe66c073990 tx=0x7fe66401c830 comp rx=0 tx=0).stop 2024-09-10T23:39:55.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe673a00640 1 -- 172.21.15.94:0/317815037 shutdown_connections 2024-09-10T23:39:55.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe673a00640 1 --2- 172.21.15.94:0/317815037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe66c0b1100 0x7fe66c0b1590 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe673a00640 1 --2- 172.21.15.94:0/317815037 >> v2:172.21.15.181:3300/0 conn(0x7fe66c0ce890 0x7fe66c0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe673a00640 1 -- 172.21.15.94:0/317815037 >> 172.21.15.94:0/317815037 conn(0x7fe66c0d1c60 msgr2=0x7fe66c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:55.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe673a00640 1 -- 172.21.15.94:0/317815037 shutdown_connections 2024-09-10T23:39:55.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe673a00640 1 -- 172.21.15.94:0/317815037 wait complete. 2024-09-10T23:39:55.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe673a00640 1 Processor -- start 2024-09-10T23:39:55.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe673a00640 1 -- start start 2024-09-10T23:39:55.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe673a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe66c0b1100 0x7fe66c0b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:55.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe673a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe66c0ce1e0 0x7fe66c0c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe673a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe66c0ce890 0x7fe66c165850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe673a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c169110 con 0x7fe66c0b1100 2024-09-10T23:39:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe673a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c0d0a30 con 0x7fe66c0ce890 2024-09-10T23:39:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe673a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c169290 con 0x7fe66c0ce1e0 2024-09-10T23:39:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe671000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe66c0b1100 0x7fe66c0b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe671000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe66c0b1100 0x7fe66c0b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39696/0 (socket says 172.21.15.94:39696) 2024-09-10T23:39:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe671000640 1 -- 172.21.15.94:0/1974827604 learned_addr learned my addr 172.21.15.94:0/1974827604 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe671a00640 1 --2- 172.21.15.94:0/1974827604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe66c0ce890 0x7fe66c165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe671000640 1 -- 172.21.15.94:0/1974827604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe66c0ce1e0 msgr2=0x7fe66c0c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe66be00640 1 --2- 172.21.15.94:0/1974827604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe66c0ce1e0 0x7fe66c0c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:55.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe671000640 1 --2- 172.21.15.94:0/1974827604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe66c0ce1e0 0x7fe66c0c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe671000640 1 -- 172.21.15.94:0/1974827604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe66c0ce890 msgr2=0x7fe66c165850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:55.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe671000640 1 --2- 172.21.15.94:0/1974827604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe66c0ce890 0x7fe66c165850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe671000640 1 -- 172.21.15.94:0/1974827604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c1f7680 con 0x7fe66c0b1100 2024-09-10T23:39:55.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe671a00640 1 --2- 172.21.15.94:0/1974827604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe66c0ce890 0x7fe66c165850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:55.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.803+0000 7fe66be00640 1 --2- 172.21.15.94:0/1974827604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe66c0ce1e0 0x7fe66c0c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:55.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.807+0000 7fe671000640 1 --2- 172.21.15.94:0/1974827604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe66c0b1100 0x7fe66c0b0910 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00e9f0 tx=0x7fe65c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:55.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.807+0000 7fe668c00640 1 -- 172.21.15.94:0/1974827604 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c00cd80 con 0x7fe66c0b1100 2024-09-10T23:39:55.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.807+0000 7fe668c00640 1 -- 172.21.15.94:0/1974827604 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe65c004c20 con 0x7fe66c0b1100 2024-09-10T23:39:55.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.807+0000 7fe668c00640 1 -- 172.21.15.94:0/1974827604 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c005020 con 0x7fe66c0b1100 2024-09-10T23:39:55.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.807+0000 7fe673a00640 1 -- 172.21.15.94:0/1974827604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66c1f7920 con 0x7fe66c0b1100 2024-09-10T23:39:55.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.807+0000 7fe673a00640 1 -- 172.21.15.94:0/1974827604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe66c1f7ee0 con 0x7fe66c0b1100 2024-09-10T23:39:55.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.807+0000 7fe673a00640 1 -- 172.21.15.94:0/1974827604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe66c0b1590 con 0x7fe66c0b1100 2024-09-10T23:39:55.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.807+0000 7fe668c00640 1 -- 172.21.15.94:0/1974827604 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fe65c00f040 con 0x7fe66c0b1100 2024-09-10T23:39:55.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.811+0000 7fe668c00640 1 --2- 172.21.15.94:0/1974827604 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe64c072e50 0x7fe64c075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:55.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.811+0000 7fe668c00640 1 -- 172.21.15.94:0/1974827604 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4971+0+0 (secure 0 0 0) 0x7fe65c094820 con 0x7fe66c0b1100 2024-09-10T23:39:55.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.811+0000 7fe66be00640 1 --2- 172.21.15.94:0/1974827604 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe64c072e50 0x7fe64c075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:55.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.811+0000 7fe66be00640 1 --2- 172.21.15.94:0/1974827604 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe64c072e50 0x7fe64c075310 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe658003e40 tx=0x7fe658009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:55.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.815+0000 7fe668c00640 1 -- 172.21.15.94:0/1974827604 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe65c016030 con 0x7fe66c0b1100 2024-09-10T23:39:55.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.979+0000 7fe673a00640 1 -- 172.21.15.94:0/1974827604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe66c1f83e0 con 0x7fe66c0b1100 2024-09-10T23:39:55.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.979+0000 7fe668c00640 1 -- 172.21.15.94:0/1974827604 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) ==== 93+0+10 (secure 0 0 0) 0x7fe65c05f240 con 0x7fe66c0b1100 2024-09-10T23:39:55.982 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-09-10T23:39:55.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.979+0000 7fe673a00640 1 -- 172.21.15.94:0/1974827604 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe64c072e50 msgr2=0x7fe64c075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:55.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.979+0000 7fe673a00640 1 --2- 172.21.15.94:0/1974827604 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe64c072e50 0x7fe64c075310 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe658003e40 tx=0x7fe658009040 comp rx=0 tx=0).stop 2024-09-10T23:39:55.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.979+0000 7fe673a00640 1 -- 172.21.15.94:0/1974827604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe66c0b1100 msgr2=0x7fe66c0b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:55.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.979+0000 7fe673a00640 1 --2- 172.21.15.94:0/1974827604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe66c0b1100 0x7fe66c0b0910 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00e9f0 tx=0x7fe65c00eeb0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.979+0000 7fe673a00640 1 -- 172.21.15.94:0/1974827604 shutdown_connections 2024-09-10T23:39:55.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.979+0000 7fe673a00640 1 --2- 172.21.15.94:0/1974827604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe66c0ce890 0x7fe66c165850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.979+0000 7fe673a00640 1 --2- 172.21.15.94:0/1974827604 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe64c072e50 0x7fe64c075310 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.979+0000 7fe673a00640 1 --2- 172.21.15.94:0/1974827604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe66c0ce1e0 0x7fe66c0c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.979+0000 7fe673a00640 1 --2- 172.21.15.94:0/1974827604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe66c0b1100 0x7fe66c0b0910 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:55.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.979+0000 7fe673a00640 1 -- 172.21.15.94:0/1974827604 >> 172.21.15.94:0/1974827604 conn(0x7fe66c0d1c60 msgr2=0x7fe66c15e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:55.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.979+0000 7fe673a00640 1 -- 172.21.15.94:0/1974827604 shutdown_connections 2024-09-10T23:39:55.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:55.979+0000 7fe673a00640 1 -- 172.21.15.94:0/1974827604 wait complete. 2024-09-10T23:39:56.004 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-10T23:39:56.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.143+0000 7fad91800640 1 -- 172.21.15.94:0/819147566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad70003a60 msgr2=0x7fad70003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:56.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.143+0000 7fad91800640 1 --2- 172.21.15.94:0/819147566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad70003a60 0x7fad70003e80 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009b60 tx=0x7fad7c01d270 comp rx=0 tx=0).stop 2024-09-10T23:39:56.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.143+0000 7fad8b400640 1 -- 172.21.15.94:0/819147566 reap_dead start 2024-09-10T23:39:56.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.143+0000 7fad91800640 1 -- 172.21.15.94:0/819147566 shutdown_connections 2024-09-10T23:39:56.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.143+0000 7fad91800640 1 -- 172.21.15.94:0/819147566 >> 172.21.15.94:0/819147566 conn(0x7fad8c0d0b70 msgr2=0x7fad8c0d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:56.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.143+0000 7fad91800640 1 -- 172.21.15.94:0/819147566 shutdown_connections 2024-09-10T23:39:56.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.143+0000 7fad91800640 1 -- 172.21.15.94:0/819147566 wait complete. 2024-09-10T23:39:56.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.143+0000 7fad91800640 1 Processor -- start 2024-09-10T23:39:56.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.143+0000 7fad91800640 1 -- start start 2024-09-10T23:39:56.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad91800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad70003a60 0x7fad8c165150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:56.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad91800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad8c14b9c0 0x7fad8c1fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:56.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad91800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c1f8e80 0x7fad8c14f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:56.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad91800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c166f70 con 0x7fad8c14b9c0 2024-09-10T23:39:56.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad91800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c166df0 con 0x7fad70003a60 2024-09-10T23:39:56.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad91800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fad8c1670f0 con 0x7fad8c1f8e80 2024-09-10T23:39:56.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad8a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad8c14b9c0 0x7fad8c1fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:56.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad8aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad70003a60 0x7fad8c165150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:56.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad8a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad8c14b9c0 0x7fad8c1fc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39708/0 (socket says 172.21.15.94:39708) 2024-09-10T23:39:56.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad8aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad70003a60 0x7fad8c165150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:45528/0 (socket says 172.21.15.94:45528) 2024-09-10T23:39:56.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad8a000640 1 -- 172.21.15.94:0/2049199200 learned_addr learned my addr 172.21.15.94:0/2049199200 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:56.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad8b400640 1 --2- 172.21.15.94:0/2049199200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c1f8e80 0x7fad8c14f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:56.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad8a000640 1 -- 172.21.15.94:0/2049199200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c1f8e80 msgr2=0x7fad8c14f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:56.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad8a000640 1 --2- 172.21.15.94:0/2049199200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c1f8e80 0x7fad8c14f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:56.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad8a000640 1 -- 172.21.15.94:0/2049199200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad70003a60 msgr2=0x7fad8c165150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:56.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad8a000640 1 --2- 172.21.15.94:0/2049199200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad70003a60 0x7fad8c165150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:56.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad8a000640 1 -- 172.21.15.94:0/2049199200 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8c1f86c0 con 0x7fad8c14b9c0 2024-09-10T23:39:56.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad8a000640 1 --2- 172.21.15.94:0/2049199200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad8c14b9c0 0x7fad8c1fc4d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fad78011a50 tx=0x7fad78011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:56.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad82a00640 1 -- 172.21.15.94:0/2049199200 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7801f070 con 0x7fad8c14b9c0 2024-09-10T23:39:56.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad82a00640 1 -- 172.21.15.94:0/2049199200 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad78012450 con 0x7fad8c14b9c0 2024-09-10T23:39:56.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad82a00640 1 -- 172.21.15.94:0/2049199200 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78012860 con 0x7fad8c14b9c0 2024-09-10T23:39:56.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad91800640 1 -- 172.21.15.94:0/2049199200 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8c163610 con 0x7fad8c14b9c0 2024-09-10T23:39:56.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad91800640 1 -- 172.21.15.94:0/2049199200 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad8c163bd0 con 0x7fad8c14b9c0 2024-09-10T23:39:56.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad91800640 1 -- 172.21.15.94:0/2049199200 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad8c06cc20 con 0x7fad8c14b9c0 2024-09-10T23:39:56.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.147+0000 7fad82a00640 1 -- 172.21.15.94:0/2049199200 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fad78012a00 con 0x7fad8c14b9c0 2024-09-10T23:39:56.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.151+0000 7fad82a00640 1 --2- 172.21.15.94:0/2049199200 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fad6c072e80 0x7fad6c075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:56.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.151+0000 7fad82a00640 1 -- 172.21.15.94:0/2049199200 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4971+0+0 (secure 0 0 0) 0x7fad78094ae0 con 0x7fad8c14b9c0 2024-09-10T23:39:56.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.151+0000 7fad8aa00640 1 --2- 172.21.15.94:0/2049199200 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fad6c072e80 0x7fad6c075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:56.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.151+0000 7fad8aa00640 1 --2- 172.21.15.94:0/2049199200 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fad6c072e80 0x7fad6c075340 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0ae000 tx=0x7fad7c01a010 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:56.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.155+0000 7fad82a00640 1 -- 172.21.15.94:0/2049199200 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fad7805f2c0 con 0x7fad8c14b9c0 2024-09-10T23:39:56.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:56.379+0000 7fad91800640 1 -- 172.21.15.94:0/2049199200 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7fad8c0b5fa0 con 0x7fad8c14b9c0 2024-09-10T23:39:57.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.299+0000 7fad82a00640 1 -- 172.21.15.94:0/2049199200 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v407) ==== 148+0+0 (secure 0 0 0) 0x7fad78063cf0 con 0x7fad8c14b9c0 2024-09-10T23:39:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.303+0000 7fad91800640 1 -- 172.21.15.94:0/2049199200 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fad6c072e80 msgr2=0x7fad6c075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.303+0000 7fad91800640 1 --2- 172.21.15.94:0/2049199200 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fad6c072e80 0x7fad6c075340 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0ae000 tx=0x7fad7c01a010 comp rx=0 tx=0).stop 2024-09-10T23:39:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.303+0000 7fad91800640 1 -- 172.21.15.94:0/2049199200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad8c14b9c0 msgr2=0x7fad8c1fc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.303+0000 7fad91800640 1 --2- 172.21.15.94:0/2049199200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad8c14b9c0 0x7fad8c1fc4d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fad78011a50 tx=0x7fad78011f10 comp rx=0 tx=0).stop 2024-09-10T23:39:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.303+0000 7fad91800640 1 -- 172.21.15.94:0/2049199200 shutdown_connections 2024-09-10T23:39:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.303+0000 7fad91800640 1 --2- 172.21.15.94:0/2049199200 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fad6c072e80 0x7fad6c075340 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.303+0000 7fad91800640 1 --2- 172.21.15.94:0/2049199200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c1f8e80 0x7fad8c14f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.303+0000 7fad91800640 1 --2- 172.21.15.94:0/2049199200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad8c14b9c0 0x7fad8c1fc4d0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.303+0000 7fad91800640 1 --2- 172.21.15.94:0/2049199200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad70003a60 0x7fad8c165150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.303+0000 7fad91800640 1 -- 172.21.15.94:0/2049199200 >> 172.21.15.94:0/2049199200 conn(0x7fad8c0d0b70 msgr2=0x7fad8c15b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.303+0000 7fad91800640 1 -- 172.21.15.94:0/2049199200 shutdown_connections 2024-09-10T23:39:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.303+0000 7fad91800640 1 -- 172.21.15.94:0/2049199200 wait complete. 2024-09-10T23:39:57.331 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T23:39:57.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.471+0000 7f3bef200640 1 -- 172.21.15.94:0/2884761791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bcc004d50 msgr2=0x7f3bcc0051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:57.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.471+0000 7f3bef200640 1 --2- 172.21.15.94:0/2884761791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bcc004d50 0x7f3bcc0051b0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f3be0007570 tx=0x7f3be001d7a0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.471+0000 7f3bed200640 1 -- 172.21.15.94:0/2884761791 reap_dead start 2024-09-10T23:39:57.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.471+0000 7f3bef200640 1 -- 172.21.15.94:0/2884761791 shutdown_connections 2024-09-10T23:39:57.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bef200640 1 -- 172.21.15.94:0/2884761791 >> 172.21.15.94:0/2884761791 conn(0x7f3be806cdf0 msgr2=0x7f3be806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:57.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bef200640 1 -- 172.21.15.94:0/2884761791 shutdown_connections 2024-09-10T23:39:57.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bef200640 1 -- 172.21.15.94:0/2884761791 wait complete. 2024-09-10T23:39:57.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bef200640 1 Processor -- start 2024-09-10T23:39:57.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bef200640 1 -- start start 2024-09-10T23:39:57.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bef200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bcc004d50 0x7f3be81f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:57.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bef200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3be80aeee0 0x7f3be80b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:57.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bef200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3be80af640 0x7f3be815c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:57.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bef200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3be815e290 con 0x7f3be80aeee0 2024-09-10T23:39:57.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bef200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3be8070910 con 0x7f3be80af640 2024-09-10T23:39:57.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bef200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3be815e410 con 0x7f3bcc004d50 2024-09-10T23:39:57.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bdf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3be80aeee0 0x7f3be80b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:57.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bdf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3be80aeee0 0x7f3be80b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39722/0 (socket says 172.21.15.94:39722) 2024-09-10T23:39:57.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bdf400640 1 -- 172.21.15.94:0/2647455564 learned_addr learned my addr 172.21.15.94:0/2647455564 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:57.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3be7e00640 1 --2- 172.21.15.94:0/2647455564 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bcc004d50 0x7f3be81f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:57.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bed200640 1 --2- 172.21.15.94:0/2647455564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3be80af640 0x7f3be815c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:57.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bdf400640 1 -- 172.21.15.94:0/2647455564 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bcc004d50 msgr2=0x7f3be81f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:57.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bdf400640 1 --2- 172.21.15.94:0/2647455564 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bcc004d50 0x7f3be81f7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bdf400640 1 -- 172.21.15.94:0/2647455564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3be80af640 msgr2=0x7f3be815c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:57.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bdf400640 1 --2- 172.21.15.94:0/2647455564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3be80af640 0x7f3be815c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bdf400640 1 -- 172.21.15.94:0/2647455564 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be81f4820 con 0x7f3be80aeee0 2024-09-10T23:39:57.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bed200640 1 --2- 172.21.15.94:0/2647455564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3be80af640 0x7f3be815c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:57.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bdf400640 1 --2- 172.21.15.94:0/2647455564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3be80aeee0 0x7f3be80b03d0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0003ea0 tx=0x7f3bd0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:57.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3be4c00640 1 -- 172.21.15.94:0/2647455564 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0015070 con 0x7f3be80aeee0 2024-09-10T23:39:57.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3be4c00640 1 -- 172.21.15.94:0/2647455564 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bd0011840 con 0x7f3be80aeee0 2024-09-10T23:39:57.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3bef200640 1 -- 172.21.15.94:0/2647455564 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be81f82e0 con 0x7f3be80aeee0 2024-09-10T23:39:57.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.475+0000 7f3be4c00640 1 -- 172.21.15.94:0/2647455564 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0011c90 con 0x7f3be80aeee0 2024-09-10T23:39:57.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.479+0000 7f3bef200640 1 -- 172.21.15.94:0/2647455564 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be81f88a0 con 0x7f3be80aeee0 2024-09-10T23:39:57.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.479+0000 7f3bef200640 1 -- 172.21.15.94:0/2647455564 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be80cfe90 con 0x7f3be80aeee0 2024-09-10T23:39:57.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.479+0000 7f3be4c00640 1 -- 172.21.15.94:0/2647455564 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f3bd00103c0 con 0x7f3be80aeee0 2024-09-10T23:39:57.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.479+0000 7f3be4c00640 1 --2- 172.21.15.94:0/2647455564 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3bc4072e80 0x7f3bc4075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:57.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.479+0000 7f3be4c00640 1 -- 172.21.15.94:0/2647455564 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4971+0+0 (secure 0 0 0) 0x7f3bd0094c10 con 0x7f3be80aeee0 2024-09-10T23:39:57.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.479+0000 7f3be7e00640 1 --2- 172.21.15.94:0/2647455564 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3bc4072e80 0x7f3bc4075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:57.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.479+0000 7f3be7e00640 1 --2- 172.21.15.94:0/2647455564 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3bc4072e80 0x7f3bc4075340 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8002240 tx=0x7f3bd8006ab0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:57.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.483+0000 7f3be4c00640 1 -- 172.21.15.94:0/2647455564 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3bd005ef50 con 0x7f3be80aeee0 2024-09-10T23:39:57.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.651+0000 7f3bef200640 1 -- 172.21.15.94:0/2647455564 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f3be80b2c80 con 0x7f3be80aeee0 2024-09-10T23:39:57.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.651+0000 7f3be4c00640 1 -- 172.21.15.94:0/2647455564 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v45) ==== 59+0+1998 (secure 0 0 0) 0x7f3bd0063980 con 0x7f3be80aeee0 2024-09-10T23:39:57.656 INFO:teuthology.orchestra.run.smithi094.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T23:39:57.656 INFO:teuthology.orchestra.run.smithi094.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T23:39:57.656 INFO:teuthology.orchestra.run.smithi094.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T23:39:57.656 INFO:teuthology.orchestra.run.smithi094.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T23:39:57.656 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3944.074506 2024-09-10T23:39:57.656 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3865.464527 2024-09-10T23:39:57.656 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3898.861024 2024-09-10T23:39:57.656 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4010.048529 2024-09-10T23:39:57.656 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4650.562436 2024-09-10T23:39:57.656 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4724.018776 2024-09-10T23:39:57.657 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5255.753663 2024-09-10T23:39:57.657 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4718.009443 2024-09-10T23:39:57.657 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds 20/20 2024-09-10T23:39:57.657 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T23:39:57.657 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_ms 2/2 2024-09-10T23:39:57.657 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_frag true 2024-09-10T23:39:57.657 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_scatterstat true 2024-09-10T23:39:57.657 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T23:39:57.657 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_verify_scatter true 2024-09-10T23:39:57.657 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T23:39:57.657 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T23:39:57.657 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T23:39:57.657 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced client_mount_timeout 600 2024-09-10T23:39:57.657 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_client 20/20 2024-09-10T23:39:57.658 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_ms 1/1 2024-09-10T23:39:57.658 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T23:39:57.658 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T23:39:57.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.655+0000 7f3bef200640 1 -- 172.21.15.94:0/2647455564 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3bc4072e80 msgr2=0x7f3bc4075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:57.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.655+0000 7f3bef200640 1 --2- 172.21.15.94:0/2647455564 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3bc4072e80 0x7f3bc4075340 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8002240 tx=0x7f3bd8006ab0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.655+0000 7f3bef200640 1 -- 172.21.15.94:0/2647455564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3be80aeee0 msgr2=0x7f3be80b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:57.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.655+0000 7f3bef200640 1 --2- 172.21.15.94:0/2647455564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3be80aeee0 0x7f3be80b03d0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0003ea0 tx=0x7f3bd0003f90 comp rx=0 tx=0).stop 2024-09-10T23:39:57.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.655+0000 7f3bef200640 1 -- 172.21.15.94:0/2647455564 shutdown_connections 2024-09-10T23:39:57.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.655+0000 7f3bef200640 1 --2- 172.21.15.94:0/2647455564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3be80af640 0x7f3be815c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.655+0000 7f3bef200640 1 --2- 172.21.15.94:0/2647455564 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3bc4072e80 0x7f3bc4075340 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.655+0000 7f3bef200640 1 --2- 172.21.15.94:0/2647455564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3be80aeee0 0x7f3be80b03d0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.655+0000 7f3bef200640 1 --2- 172.21.15.94:0/2647455564 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bcc004d50 0x7f3be81f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.655+0000 7f3bef200640 1 -- 172.21.15.94:0/2647455564 >> 172.21.15.94:0/2647455564 conn(0x7f3be806cdf0 msgr2=0x7f3be806e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:57.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.655+0000 7f3bef200640 1 -- 172.21.15.94:0/2647455564 shutdown_connections 2024-09-10T23:39:57.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.655+0000 7f3bef200640 1 -- 172.21.15.94:0/2647455564 wait complete. 2024-09-10T23:39:57.677 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:39:57.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.819+0000 7fe12e600640 1 -- 172.21.15.94:0/2499323251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe12806e7c0 msgr2=0x7fe12806ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:57.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.819+0000 7fe12e600640 1 --2- 172.21.15.94:0/2499323251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe12806e7c0 0x7fe12806ec30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe120009900 tx=0x7fe12001c8e0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.819+0000 7fe12e600640 1 -- 172.21.15.94:0/2499323251 shutdown_connections 2024-09-10T23:39:57.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.819+0000 7fe12e600640 1 --2- 172.21.15.94:0/2499323251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe12806e7c0 0x7fe12806ec30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.819+0000 7fe12e600640 1 --2- 172.21.15.94:0/2499323251 >> v2:172.21.15.181:3300/0 conn(0x7fe1280c64f0 0x7fe1280cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.819+0000 7fe12e600640 1 -- 172.21.15.94:0/2499323251 >> 172.21.15.94:0/2499323251 conn(0x7fe1280cfaf0 msgr2=0x7fe1280cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:57.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.819+0000 7fe12e600640 1 -- 172.21.15.94:0/2499323251 shutdown_connections 2024-09-10T23:39:57.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.819+0000 7fe12e600640 1 -- 172.21.15.94:0/2499323251 wait complete. 2024-09-10T23:39:57.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.819+0000 7fe12e600640 1 Processor -- start 2024-09-10T23:39:57.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.819+0000 7fe12e600640 1 -- start start 2024-09-10T23:39:57.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe12e600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe12806e7c0 0x7fe1281479a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:57.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe12e600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280c64f0 0x7fe1280b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:57.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe12e600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1280ce280 0x7fe1280b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:57.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe12e600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe12815b5c0 con 0x7fe12806e7c0 2024-09-10T23:39:57.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe12e600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe12815b440 con 0x7fe1280c64f0 2024-09-10T23:39:57.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe12e600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe12815b740 con 0x7fe1280ce280 2024-09-10T23:39:57.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe127400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe12806e7c0 0x7fe1281479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:57.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe127400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe12806e7c0 0x7fe1281479a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39726/0 (socket says 172.21.15.94:39726) 2024-09-10T23:39:57.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe127400640 1 -- 172.21.15.94:0/2307636398 learned_addr learned my addr 172.21.15.94:0/2307636398 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:57.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe127e00640 1 --2- 172.21.15.94:0/2307636398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1280ce280 0x7fe1280b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe11ea00640 1 --2- 172.21.15.94:0/2307636398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280c64f0 0x7fe1280b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe127400640 1 -- 172.21.15.94:0/2307636398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1280ce280 msgr2=0x7fe1280b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe127400640 1 --2- 172.21.15.94:0/2307636398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1280ce280 0x7fe1280b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe127400640 1 -- 172.21.15.94:0/2307636398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280c64f0 msgr2=0x7fe1280b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe127400640 1 --2- 172.21.15.94:0/2307636398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280c64f0 0x7fe1280b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe127400640 1 -- 172.21.15.94:0/2307636398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe128147ee0 con 0x7fe12806e7c0 2024-09-10T23:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe127e00640 1 --2- 172.21.15.94:0/2307636398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1280ce280 0x7fe1280b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe11ea00640 1 --2- 172.21.15.94:0/2307636398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280c64f0 0x7fe1280b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe127400640 1 --2- 172.21.15.94:0/2307636398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe12806e7c0 0x7fe1281479a0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fe11800c750 tx=0x7fe11800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:57.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe11fe00640 1 -- 172.21.15.94:0/2307636398 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118017070 con 0x7fe12806e7c0 2024-09-10T23:39:57.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe12e600640 1 -- 172.21.15.94:0/2307636398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12815a460 con 0x7fe12806e7c0 2024-09-10T23:39:57.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe11fe00640 1 -- 172.21.15.94:0/2307636398 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe118005070 con 0x7fe12806e7c0 2024-09-10T23:39:57.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe11fe00640 1 -- 172.21.15.94:0/2307636398 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11800ce60 con 0x7fe12806e7c0 2024-09-10T23:39:57.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe12e600640 1 -- 172.21.15.94:0/2307636398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1281fbc30 con 0x7fe12806e7c0 2024-09-10T23:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe12e600640 1 -- 172.21.15.94:0/2307636398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe12806ec30 con 0x7fe12806e7c0 2024-09-10T23:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.823+0000 7fe11fe00640 1 -- 172.21.15.94:0/2307636398 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fe1180054c0 con 0x7fe12806e7c0 2024-09-10T23:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.827+0000 7fe11fe00640 1 --2- 172.21.15.94:0/2307636398 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe104072e50 0x7fe104075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.827+0000 7fe11fe00640 1 -- 172.21.15.94:0/2307636398 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4971+0+0 (secure 0 0 0) 0x7fe118093cd0 con 0x7fe12806e7c0 2024-09-10T23:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.827+0000 7fe11ea00640 1 --2- 172.21.15.94:0/2307636398 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe104072e50 0x7fe104075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:57.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.827+0000 7fe11ea00640 1 --2- 172.21.15.94:0/2307636398 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe104072e50 0x7fe104075310 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe1280af4b0 tx=0x7fe1100091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:57.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.831+0000 7fe11fe00640 1 -- 172.21.15.94:0/2307636398 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe1180057e0 con 0x7fe12806e7c0 2024-09-10T23:39:58.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:57.999+0000 7fe12e600640 1 -- 172.21.15.94:0/2307636398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe1280d0850 con 0x7fe12806e7c0 2024-09-10T23:39:58.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.003+0000 7fe11fe00640 1 -- 172.21.15.94:0/2307636398 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7fe11805e550 con 0x7fe12806e7c0 2024-09-10T23:39:58.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.003+0000 7fe12e600640 1 -- 172.21.15.94:0/2307636398 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe104072e50 msgr2=0x7fe104075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:58.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.003+0000 7fe12e600640 1 --2- 172.21.15.94:0/2307636398 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe104072e50 0x7fe104075310 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe1280af4b0 tx=0x7fe1100091e0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.003+0000 7fe12e600640 1 -- 172.21.15.94:0/2307636398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe12806e7c0 msgr2=0x7fe1281479a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:58.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.003+0000 7fe12e600640 1 --2- 172.21.15.94:0/2307636398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe12806e7c0 0x7fe1281479a0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fe11800c750 tx=0x7fe11800cc10 comp rx=0 tx=0).stop 2024-09-10T23:39:58.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.003+0000 7fe12e600640 1 -- 172.21.15.94:0/2307636398 shutdown_connections 2024-09-10T23:39:58.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.003+0000 7fe12e600640 1 --2- 172.21.15.94:0/2307636398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1280c64f0 0x7fe1280b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.003+0000 7fe12e600640 1 --2- 172.21.15.94:0/2307636398 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe104072e50 0x7fe104075310 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.003+0000 7fe12e600640 1 --2- 172.21.15.94:0/2307636398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1280ce280 0x7fe1280b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.003+0000 7fe12e600640 1 --2- 172.21.15.94:0/2307636398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe12806e7c0 0x7fe1281479a0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.003+0000 7fe12e600640 1 -- 172.21.15.94:0/2307636398 >> 172.21.15.94:0/2307636398 conn(0x7fe1280cfaf0 msgr2=0x7fe1281556e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:58.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.003+0000 7fe12e600640 1 -- 172.21.15.94:0/2307636398 shutdown_connections 2024-09-10T23:39:58.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.003+0000 7fe12e600640 1 -- 172.21.15.94:0/2307636398 wait complete. 2024-09-10T23:39:58.028 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:39:58.029 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:39:58.123 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:39:58.215 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:39:58.215 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:39:58.215 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:58.224 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:39:58.224 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:39:58.224 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:58.233 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:39:58.233 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:39:58.233 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:58.272 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:39:58.273 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:39:58.273 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:39:58.284 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:39:58.284 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T23:39:58.284 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:39:58.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.447+0000 7ff124c00640 1 -- 172.21.15.94:0/824386148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12015bd30 msgr2=0x7ff12015c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:58.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.447+0000 7ff124c00640 1 --2- 172.21.15.94:0/824386148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12015bd30 0x7ff12015c110 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7ff1200b7d20 tx=0x7ff11001cd80 comp rx=0 tx=0).stop 2024-09-10T23:39:58.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.447+0000 7ff124c00640 1 -- 172.21.15.94:0/824386148 shutdown_connections 2024-09-10T23:39:58.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.447+0000 7ff124c00640 1 --2- 172.21.15.94:0/824386148 >> v2:172.21.15.181:3300/0 conn(0x7ff1200d2290 0x7ff1200b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.447+0000 7ff124c00640 1 --2- 172.21.15.94:0/824386148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12015bd30 0x7ff12015c110 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.447+0000 7ff124c00640 1 -- 172.21.15.94:0/824386148 >> 172.21.15.94:0/824386148 conn(0x7ff12009aa70 msgr2=0x7ff12009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:58.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.447+0000 7ff124c00640 1 -- 172.21.15.94:0/824386148 shutdown_connections 2024-09-10T23:39:58.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.447+0000 7ff124c00640 1 -- 172.21.15.94:0/824386148 wait complete. 2024-09-10T23:39:58.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff124c00640 1 Processor -- start 2024-09-10T23:39:58.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff124c00640 1 -- start start 2024-09-10T23:39:58.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff124c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1200ae980 0x7ff1201f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff124c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1200d2290 0x7ff1200b6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff124c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12015bd30 0x7ff12006f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff124c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1201694b0 con 0x7ff12015bd30 2024-09-10T23:39:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff124c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff120169330 con 0x7ff1200ae980 2024-09-10T23:39:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff124c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff120169630 con 0x7ff1200d2290 2024-09-10T23:39:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff11e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12015bd30 0x7ff12006f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff11e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12015bd30 0x7ff12006f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39732/0 (socket says 172.21.15.94:39732) 2024-09-10T23:39:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff11e800640 1 -- 172.21.15.94:0/369827711 learned_addr learned my addr 172.21.15.94:0/369827711 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff11de00640 1 --2- 172.21.15.94:0/369827711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1200ae980 0x7ff1201f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff11d400640 1 --2- 172.21.15.94:0/369827711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1200d2290 0x7ff1200b6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff11e800640 1 -- 172.21.15.94:0/369827711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1200d2290 msgr2=0x7ff1200b6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff11e800640 1 --2- 172.21.15.94:0/369827711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1200d2290 0x7ff1200b6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff11e800640 1 -- 172.21.15.94:0/369827711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1200ae980 msgr2=0x7ff1201f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff11e800640 1 --2- 172.21.15.94:0/369827711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1200ae980 0x7ff1201f8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff11e800640 1 -- 172.21.15.94:0/369827711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12006e0a0 con 0x7ff12015bd30 2024-09-10T23:39:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff11de00640 1 --2- 172.21.15.94:0/369827711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1200ae980 0x7ff1201f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:39:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff11e800640 1 --2- 172.21.15.94:0/369827711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12015bd30 0x7ff12006f140 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7ff118005d10 tx=0x7ff11800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff116000640 1 -- 172.21.15.94:0/369827711 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118017070 con 0x7ff12015bd30 2024-09-10T23:39:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff116000640 1 -- 172.21.15.94:0/369827711 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff118005310 con 0x7ff12015bd30 2024-09-10T23:39:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff124c00640 1 -- 172.21.15.94:0/369827711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12006e380 con 0x7ff12015bd30 2024-09-10T23:39:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff116000640 1 -- 172.21.15.94:0/369827711 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118009820 con 0x7ff12015bd30 2024-09-10T23:39:58.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.451+0000 7ff124c00640 1 -- 172.21.15.94:0/369827711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12006e940 con 0x7ff12015bd30 2024-09-10T23:39:58.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.455+0000 7ff124c00640 1 -- 172.21.15.94:0/369827711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1201f9390 con 0x7ff12015bd30 2024-09-10T23:39:58.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.455+0000 7ff116000640 1 -- 172.21.15.94:0/369827711 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7ff118009bd0 con 0x7ff12015bd30 2024-09-10T23:39:58.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.455+0000 7ff116000640 1 --2- 172.21.15.94:0/369827711 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff100072e50 0x7ff100075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:58.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.455+0000 7ff116000640 1 -- 172.21.15.94:0/369827711 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4971+0+0 (secure 0 0 0) 0x7ff118094580 con 0x7ff12015bd30 2024-09-10T23:39:58.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.455+0000 7ff11de00640 1 --2- 172.21.15.94:0/369827711 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff100072e50 0x7ff100075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:58.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.455+0000 7ff11de00640 1 --2- 172.21.15.94:0/369827711 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff100072e50 0x7ff100075310 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff1100af000 tx=0x7ff11001b5c0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:58.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.459+0000 7ff116000640 1 -- 172.21.15.94:0/369827711 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff1201f9390 con 0x7ff12015bd30 2024-09-10T23:39:58.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.659+0000 7ff124c00640 1 -- 172.21.15.94:0/369827711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff12006ee20 con 0x7ff12015bd30 2024-09-10T23:39:58.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.659+0000 7ff116000640 1 -- 172.21.15.94:0/369827711 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+3907 (secure 0 0 0) 0x7ff12006ee20 con 0x7ff12015bd30 2024-09-10T23:39:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:58.663 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":100,"btime":"2024-09-10T23:39:52:281764+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7864,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/188024700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":188024700},{"type":"v1","addr":"172.21.15.181:6835","nonce":188024700}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8083,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/1498839710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1498839710},{"type":"v1","addr":"172.21.15.181:6801","nonce":1498839710}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":8376,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/35853213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":35853213},{"type":"v1","addr":"172.21.15.94:6834","nonce":35853213}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/800902310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":800902310},{"type":"v1","addr":"172.21.15.94:6835","nonce":800902310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[]} 2024-09-10T23:39:58.663 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 100 2024-09-10T23:39:58.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.659+0000 7ff124c00640 1 -- 172.21.15.94:0/369827711 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff100072e50 msgr2=0x7ff100075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:58.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.659+0000 7ff124c00640 1 --2- 172.21.15.94:0/369827711 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff100072e50 0x7ff100075310 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff1100af000 tx=0x7ff11001b5c0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.659+0000 7ff124c00640 1 -- 172.21.15.94:0/369827711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12015bd30 msgr2=0x7ff12006f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.659+0000 7ff124c00640 1 --2- 172.21.15.94:0/369827711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12015bd30 0x7ff12006f140 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7ff118005d10 tx=0x7ff11800db90 comp rx=0 tx=0).stop 2024-09-10T23:39:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.659+0000 7ff124c00640 1 -- 172.21.15.94:0/369827711 shutdown_connections 2024-09-10T23:39:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.659+0000 7ff124c00640 1 --2- 172.21.15.94:0/369827711 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff100072e50 0x7ff100075310 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.659+0000 7ff124c00640 1 --2- 172.21.15.94:0/369827711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff12015bd30 0x7ff12006f140 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.659+0000 7ff124c00640 1 --2- 172.21.15.94:0/369827711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1200d2290 0x7ff1200b6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.659+0000 7ff124c00640 1 --2- 172.21.15.94:0/369827711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1200ae980 0x7ff1201f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.659+0000 7ff124c00640 1 -- 172.21.15.94:0/369827711 >> 172.21.15.94:0/369827711 conn(0x7ff12009aa70 msgr2=0x7ff120070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.659+0000 7ff124c00640 1 -- 172.21.15.94:0/369827711 shutdown_connections 2024-09-10T23:39:58.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.659+0000 7ff124c00640 1 -- 172.21.15.94:0/369827711 wait complete. 2024-09-10T23:39:58.684 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T23:39:58.684 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T23:39:58.684 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T23:39:58.684 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:39:58.685 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T23:39:58.685 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T23:39:58.685 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T23:39:58.685 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:39:58.685 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T23:39:58.685 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T23:39:58.685 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T23:39:58.685 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:39:58.686 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T23:39:58.686 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T23:39:58.686 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T23:39:58.686 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:39:58.697 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T23:39:58.697 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T23:39:58.698 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T23:39:58.698 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T23:39:58.700 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T23:39:58.700 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T23:39:58.701 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T23:39:58.701 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T23:39:58.743 INFO:tasks.ceph.mds.a:Started 2024-09-10T23:39:58.743 DEBUG:teuthology.parallel:result is None 2024-09-10T23:39:58.745 INFO:tasks.ceph.mds.c:Started 2024-09-10T23:39:58.745 DEBUG:teuthology.parallel:result is None 2024-09-10T23:39:58.750 INFO:tasks.ceph.mds.d:Started 2024-09-10T23:39:58.750 DEBUG:teuthology.parallel:result is None 2024-09-10T23:39:58.752 INFO:tasks.ceph.mds.b:Started 2024-09-10T23:39:58.752 DEBUG:teuthology.parallel:result is None 2024-09-10T23:39:58.752 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T23:39:58.829 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-09-10T23:39:58.829 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-09-10T23:39:58.840 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2024-09-10T23:39:58.840 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:39:58.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc58000640 1 -- 172.21.15.94:0/4068885983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc500c5e50 msgr2=0x7fcc500c62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:58.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc58000640 1 --2- 172.21.15.94:0/4068885983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc500c5e50 0x7fcc500c62e0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fcc500731c0 tx=0x7fcc4801ccf0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc58000640 1 -- 172.21.15.94:0/4068885983 shutdown_connections 2024-09-10T23:39:58.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc58000640 1 --2- 172.21.15.94:0/4068885983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc500c5e50 0x7fcc500c62e0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc58000640 1 --2- 172.21.15.94:0/4068885983 >> v2:172.21.15.181:3300/0 conn(0x7fcc500d2780 0x7fcc500d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc58000640 1 -- 172.21.15.94:0/4068885983 >> 172.21.15.94:0/4068885983 conn(0x7fcc500d1c60 msgr2=0x7fcc500d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:58.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc58000640 1 -- 172.21.15.94:0/4068885983 shutdown_connections 2024-09-10T23:39:58.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc58000640 1 -- 172.21.15.94:0/4068885983 wait complete. 2024-09-10T23:39:58.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc58000640 1 Processor -- start 2024-09-10T23:39:58.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc58000640 1 -- start start 2024-09-10T23:39:58.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc58000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc500c5e50 0x7fcc5015c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:58.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc58000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc500d2780 0x7fcc50143e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc58000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc5015bf60 0x7fcc5015deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc58000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50168f20 con 0x7fcc500c5e50 2024-09-10T23:39:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc58000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50168da0 con 0x7fcc500d2780 2024-09-10T23:39:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc58000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fcc501690a0 con 0x7fcc5015bf60 2024-09-10T23:39:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc55600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc500c5e50 0x7fcc5015c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc55600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc500c5e50 0x7fcc5015c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39796/0 (socket says 172.21.15.94:39796) 2024-09-10T23:39:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc55600640 1 -- 172.21.15.94:0/2297559120 learned_addr learned my addr 172.21.15.94:0/2297559120 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc55600640 1 -- 172.21.15.94:0/2297559120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc5015bf60 msgr2=0x7fcc5015deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc55600640 1 --2- 172.21.15.94:0/2297559120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc5015bf60 0x7fcc5015deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc55600640 1 -- 172.21.15.94:0/2297559120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc500d2780 msgr2=0x7fcc50143e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc55600640 1 --2- 172.21.15.94:0/2297559120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc500d2780 0x7fcc50143e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc55600640 1 -- 172.21.15.94:0/2297559120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5015c970 con 0x7fcc500c5e50 2024-09-10T23:39:58.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc55600640 1 --2- 172.21.15.94:0/2297559120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc500c5e50 0x7fcc5015c430 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400c3c0 tx=0x7fcc4400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:58.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc4d600640 1 -- 172.21.15.94:0/2297559120 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44017070 con 0x7fcc500c5e50 2024-09-10T23:39:58.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.899+0000 7fcc4d600640 1 -- 172.21.15.94:0/2297559120 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc440053c0 con 0x7fcc500c5e50 2024-09-10T23:39:58.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.903+0000 7fcc4d600640 1 -- 172.21.15.94:0/2297559120 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44003860 con 0x7fcc500c5e50 2024-09-10T23:39:58.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.903+0000 7fcc58000640 1 -- 172.21.15.94:0/2297559120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5015ee50 con 0x7fcc500c5e50 2024-09-10T23:39:58.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.903+0000 7fcc58000640 1 -- 172.21.15.94:0/2297559120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc500b4de0 con 0x7fcc500c5e50 2024-09-10T23:39:58.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.903+0000 7fcc58000640 1 -- 172.21.15.94:0/2297559120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc500c62e0 con 0x7fcc500c5e50 2024-09-10T23:39:58.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.903+0000 7fcc4d600640 1 -- 172.21.15.94:0/2297559120 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fcc44003a00 con 0x7fcc500c5e50 2024-09-10T23:39:58.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.903+0000 7fcc4d600640 1 --2- 172.21.15.94:0/2297559120 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcc30072e50 0x7fcc30075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:58.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.903+0000 7fcc4d600640 1 -- 172.21.15.94:0/2297559120 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4971+0+0 (secure 0 0 0) 0x7fcc44093800 con 0x7fcc500c5e50 2024-09-10T23:39:58.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.903+0000 7fcc54c00640 1 --2- 172.21.15.94:0/2297559120 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcc30072e50 0x7fcc30075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:58.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.907+0000 7fcc54c00640 1 --2- 172.21.15.94:0/2297559120 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcc30072e50 0x7fcc30075310 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0058d0 tx=0x7fcc3c005860 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:58.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:58.907+0000 7fcc4d600640 1 -- 172.21.15.94:0/2297559120 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcc4405e220 con 0x7fcc500c5e50 2024-09-10T23:39:59.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.083+0000 7fcc58000640 1 -- 172.21.15.94:0/2297559120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fcc5015b220 con 0x7fcc500c5e50 2024-09-10T23:39:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.311+0000 7fcc4d600640 1 -- 172.21.15.94:0/2297559120 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v145) ==== 97+0+0 (secure 0 0 0) 0x7fcc4405db20 con 0x7fcc500c5e50 2024-09-10T23:39:59.382 INFO:teuthology.orchestra.run.smithi094.stderr: removed all blocklist entries 2024-09-10T23:39:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.315+0000 7fcc58000640 1 -- 172.21.15.94:0/2297559120 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcc30072e50 msgr2=0x7fcc30075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.315+0000 7fcc58000640 1 --2- 172.21.15.94:0/2297559120 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcc30072e50 0x7fcc30075310 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0058d0 tx=0x7fcc3c005860 comp rx=0 tx=0).stop 2024-09-10T23:39:59.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.315+0000 7fcc58000640 1 -- 172.21.15.94:0/2297559120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc500c5e50 msgr2=0x7fcc5015c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:59.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.315+0000 7fcc58000640 1 --2- 172.21.15.94:0/2297559120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc500c5e50 0x7fcc5015c430 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400c3c0 tx=0x7fcc4400c880 comp rx=0 tx=0).stop 2024-09-10T23:39:59.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.315+0000 7fcc58000640 1 -- 172.21.15.94:0/2297559120 shutdown_connections 2024-09-10T23:39:59.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.315+0000 7fcc58000640 1 --2- 172.21.15.94:0/2297559120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc500d2780 0x7fcc50143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.315+0000 7fcc58000640 1 --2- 172.21.15.94:0/2297559120 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcc30072e50 0x7fcc30075310 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.315+0000 7fcc58000640 1 --2- 172.21.15.94:0/2297559120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc5015bf60 0x7fcc5015deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.315+0000 7fcc58000640 1 --2- 172.21.15.94:0/2297559120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc500c5e50 0x7fcc5015c430 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.315+0000 7fcc58000640 1 -- 172.21.15.94:0/2297559120 >> 172.21.15.94:0/2297559120 conn(0x7fcc500d1c60 msgr2=0x7fcc500c7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:59.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.315+0000 7fcc58000640 1 -- 172.21.15.94:0/2297559120 shutdown_connections 2024-09-10T23:39:59.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.315+0000 7fcc58000640 1 -- 172.21.15.94:0/2297559120 wait complete. 2024-09-10T23:39:59.383 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T23:39:59.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b39200640 1 -- 172.21.15.94:0/1102310065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b34070150 msgr2=0x7f2b34072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:59.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b39200640 1 --2- 172.21.15.94:0/1102310065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b34070150 0x7f2b34072610 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2b24009bc0 tx=0x7f2b2401cb10 comp rx=0 tx=0).stop 2024-09-10T23:39:59.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b39200640 1 -- 172.21.15.94:0/1102310065 shutdown_connections 2024-09-10T23:39:59.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b39200640 1 --2- 172.21.15.94:0/1102310065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b340b35c0 0x7f2b3406eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b39200640 1 --2- 172.21.15.94:0/1102310065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b34070150 0x7f2b34072610 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b39200640 1 -- 172.21.15.94:0/1102310065 >> 172.21.15.94:0/1102310065 conn(0x7f2b3406cdf0 msgr2=0x7f2b3406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:59.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b39200640 1 -- 172.21.15.94:0/1102310065 shutdown_connections 2024-09-10T23:39:59.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b39200640 1 -- 172.21.15.94:0/1102310065 wait complete. 2024-09-10T23:39:59.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b39200640 1 Processor -- start 2024-09-10T23:39:59.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b39200640 1 -- start start 2024-09-10T23:39:59.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b39200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b34070150 0x7f2b34151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:59.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b39200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b340b35c0 0x7f2b34141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:59.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b39200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b34157c60 0x7f2b341f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:59.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b39200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3415b6d0 con 0x7f2b34070150 2024-09-10T23:39:59.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b39200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3415b550 con 0x7f2b340b35c0 2024-09-10T23:39:59.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b39200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2b34167f90 con 0x7f2b34157c60 2024-09-10T23:39:59.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b31e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b34070150 0x7f2b34151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:59.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b31e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b34070150 0x7f2b34151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39798/0 (socket says 172.21.15.94:39798) 2024-09-10T23:39:59.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b31e00640 1 -- 172.21.15.94:0/1098426026 learned_addr learned my addr 172.21.15.94:0/1098426026 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:59.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b32800640 1 --2- 172.21.15.94:0/1098426026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b34157c60 0x7f2b341f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:59.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b29400640 1 --2- 172.21.15.94:0/1098426026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b340b35c0 0x7f2b34141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:59.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b31e00640 1 -- 172.21.15.94:0/1098426026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b34157c60 msgr2=0x7f2b341f8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:59.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b31e00640 1 --2- 172.21.15.94:0/1098426026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b34157c60 0x7f2b341f8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b31e00640 1 -- 172.21.15.94:0/1098426026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b340b35c0 msgr2=0x7f2b34141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:59.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b31e00640 1 --2- 172.21.15.94:0/1098426026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b340b35c0 0x7f2b34141440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.523+0000 7f2b31e00640 1 -- 172.21.15.94:0/1098426026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b341f9690 con 0x7f2b34070150 2024-09-10T23:39:59.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.527+0000 7f2b31e00640 1 --2- 172.21.15.94:0/1098426026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b34070150 0x7f2b34151c20 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f2b2401c580 tx=0x7f2b24009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:59.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.527+0000 7f2b2aa00640 1 -- 172.21.15.94:0/1098426026 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b240ba020 con 0x7f2b34070150 2024-09-10T23:39:59.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.527+0000 7f2b2aa00640 1 -- 172.21.15.94:0/1098426026 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b24004d40 con 0x7f2b34070150 2024-09-10T23:39:59.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.527+0000 7f2b2aa00640 1 -- 172.21.15.94:0/1098426026 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b240b2040 con 0x7f2b34070150 2024-09-10T23:39:59.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.527+0000 7f2b39200640 1 -- 172.21.15.94:0/1098426026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b341f9970 con 0x7f2b34070150 2024-09-10T23:39:59.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.527+0000 7f2b39200640 1 -- 172.21.15.94:0/1098426026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b34152160 con 0x7f2b34070150 2024-09-10T23:39:59.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.527+0000 7f2b39200640 1 -- 172.21.15.94:0/1098426026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b34072610 con 0x7f2b34070150 2024-09-10T23:39:59.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.527+0000 7f2b2aa00640 1 -- 172.21.15.94:0/1098426026 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f2b240b5070 con 0x7f2b34070150 2024-09-10T23:39:59.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.527+0000 7f2b2aa00640 1 --2- 172.21.15.94:0/1098426026 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2b10072e50 0x7f2b10075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:59.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.531+0000 7f2b2aa00640 1 -- 172.21.15.94:0/1098426026 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 4885+0+0 (secure 0 0 0) 0x7f2b2412d710 con 0x7f2b34070150 2024-09-10T23:39:59.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.531+0000 7f2b29400640 1 --2- 172.21.15.94:0/1098426026 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2b10072e50 0x7f2b10075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:59.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.531+0000 7f2b29400640 1 --2- 172.21.15.94:0/1098426026 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2b10072e50 0x7f2b10075310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2b341569a0 tx=0x7f2b1c009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:59.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.531+0000 7f2b2aa00640 1 -- 172.21.15.94:0/1098426026 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2b24016790 con 0x7f2b34070150 2024-09-10T23:39:59.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.755+0000 7f2b39200640 1 -- 172.21.15.94:0/1098426026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f2b3406e7e0 con 0x7f2b34070150 2024-09-10T23:39:59.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.755+0000 7f2b2aa00640 1 -- 172.21.15.94:0/1098426026 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v38) ==== 80+0+5304 (secure 0 0 0) 0x7f2b240f8590 con 0x7f2b34070150 2024-09-10T23:39:59.760 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:39:59.760 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:39:59.761 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_dump": [ 2024-09-10T23:39:59.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.761 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.a", 2024-09-10T23:39:59.761 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/2m3LxAAr0ErpLJg+0+I2r74zQya4w==", 2024-09-10T23:39:59.761 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.761 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:39:59.761 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:39:59.761 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:39:59.761 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:59.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.762 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.b", 2024-09-10T23:39:59.762 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmc8pYABAAw90zu8D+9IwK15h4HIlGBA==", 2024-09-10T23:39:59.762 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.762 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:39:59.762 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:39:59.762 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:39:59.762 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:59.763 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.763 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.c", 2024-09-10T23:39:59.763 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmk98KNxAA44e9N4VHbw/94Lh1V9zugQ==", 2024-09-10T23:39:59.763 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.763 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:39:59.763 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:39:59.763 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:39:59.764 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:59.764 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.764 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.d", 2024-09-10T23:39:59.764 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmVAKzBxAAJehOw6aphTF/g3ucoQfw9A==", 2024-09-10T23:39:59.764 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.764 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:39:59.764 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:39:59.765 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:39:59.765 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:59.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.765 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.0", 2024-09-10T23:39:59.765 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDa1uBmSAD3KBAAJTF1OEC44Ml599aL7iEuDw==", 2024-09-10T23:39:59.765 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.766 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:39:59.766 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:39:59.766 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:59.766 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.766 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.1", 2024-09-10T23:39:59.766 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDe1uBma1L9ABAAfEjW1/Hb4fh4Kwv/ZUgFAQ==", 2024-09-10T23:39:59.766 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.767 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:39:59.767 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:39:59.767 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:59.767 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.767 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.2", 2024-09-10T23:39:59.767 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDh1uBmtLUQFhAAweeFHj6BRQVC9BX+VLiWEQ==", 2024-09-10T23:39:59.767 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.767 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:39:59.767 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:39:59.768 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:59.768 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.768 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.3", 2024-09-10T23:39:59.768 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDk1uBm1KFCKRAAev28iYcrEYOViX95uo6Kbw==", 2024-09-10T23:39:59.768 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.768 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:39:59.768 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:39:59.768 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:59.768 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.768 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.4", 2024-09-10T23:39:59.769 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDq1uBm/Hg9LhAAvB2D0rj4DeRPxTrGmXYIsA==", 2024-09-10T23:39:59.769 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.769 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:39:59.769 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:39:59.769 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:59.769 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.769 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.5", 2024-09-10T23:39:59.769 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDu1uBm1fAKCBAAFHMjnGFAItLdaACF/UjE/w==", 2024-09-10T23:39:59.769 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.769 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:39:59.769 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:39:59.770 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:59.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.770 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.6", 2024-09-10T23:39:59.770 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDx1uBmXgYfHhAA/6wwtM4mDRXsIpoTIHKJ0w==", 2024-09-10T23:39:59.770 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.770 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:39:59.770 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:39:59.770 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:59.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.771 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.7", 2024-09-10T23:39:59.771 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQD01uBmw6hQMxAAk7oGAg+iIplNK5KrokGwNA==", 2024-09-10T23:39:59.771 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.771 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:39:59.771 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:39:59.771 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:59.771 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.771 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.0", 2024-09-10T23:39:59.771 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g==", 2024-09-10T23:39:59.771 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.771 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:39:59.771 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow r", 2024-09-10T23:39:59.772 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T23:39:59.772 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.772 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.1", 2024-09-10T23:39:59.772 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmuSkhFRAA1R6WNXLImkGfhW9/XJzEAQ==", 2024-09-10T23:39:59.772 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.772 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:39:59.772 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:39:59.772 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:39:59.772 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:39:59.772 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.773 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.2", 2024-09-10T23:39:59.773 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmfvaqERAASpL0KtglFcAuRf4a/0z5JQ==", 2024-09-10T23:39:59.773 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.773 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:39:59.773 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:39:59.773 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:39:59.773 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:39:59.773 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.773 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.3", 2024-09-10T23:39:59.773 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBm8xUtGxAARc0ywfAhMxBNd57TzBoV3g==", 2024-09-10T23:39:59.773 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.773 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:39:59.774 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:39:59.774 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:39:59.774 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:39:59.774 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.774 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.admin", 2024-09-10T23:39:59.774 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmOpg1FBAA9NV6Jqt6doWMuAfcxztlOg==", 2024-09-10T23:39:59.774 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.774 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:39:59.774 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:39:59.774 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:39:59.774 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:59.775 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.775 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.x", 2024-09-10T23:39:59.775 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/EPVKxAA/Myv8Zjf7BejbzlkxyCC1w==", 2024-09-10T23:39:59.775 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.775 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:39:59.775 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:39:59.775 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:59.775 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:39:59.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:39:59.775 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.y", 2024-09-10T23:39:59.775 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmhHvtJxAACe+79nYhv2ODwtmNhA45FA==", 2024-09-10T23:39:59.775 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:39:59.775 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:39:59.776 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:39:59.776 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:39:59.776 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.776 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:39:59.776 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:39:59.776 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:39:59.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.759+0000 7f2b39200640 1 -- 172.21.15.94:0/1098426026 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2b10072e50 msgr2=0x7f2b10075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:59.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.759+0000 7f2b39200640 1 --2- 172.21.15.94:0/1098426026 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2b10072e50 0x7f2b10075310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2b341569a0 tx=0x7f2b1c009040 comp rx=0 tx=0).stop 2024-09-10T23:39:59.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.759+0000 7f2b39200640 1 -- 172.21.15.94:0/1098426026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b34070150 msgr2=0x7f2b34151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:59.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.759+0000 7f2b39200640 1 --2- 172.21.15.94:0/1098426026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b34070150 0x7f2b34151c20 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f2b2401c580 tx=0x7f2b24009200 comp rx=0 tx=0).stop 2024-09-10T23:39:59.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.759+0000 7f2b39200640 1 -- 172.21.15.94:0/1098426026 shutdown_connections 2024-09-10T23:39:59.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.759+0000 7f2b39200640 1 --2- 172.21.15.94:0/1098426026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b340b35c0 0x7f2b34141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.759+0000 7f2b39200640 1 --2- 172.21.15.94:0/1098426026 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2b10072e50 0x7f2b10075310 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.759+0000 7f2b39200640 1 --2- 172.21.15.94:0/1098426026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b34157c60 0x7f2b341f8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.759+0000 7f2b39200640 1 --2- 172.21.15.94:0/1098426026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b34070150 0x7f2b34151c20 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.759+0000 7f2b39200640 1 -- 172.21.15.94:0/1098426026 >> 172.21.15.94:0/1098426026 conn(0x7f2b3406cdf0 msgr2=0x7f2b3406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:59.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.759+0000 7f2b39200640 1 -- 172.21.15.94:0/1098426026 shutdown_connections 2024-09-10T23:39:59.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.759+0000 7f2b39200640 1 -- 172.21.15.94:0/1098426026 wait complete. 2024-09-10T23:39:59.782 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T23:39:59.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.919+0000 7f80e0e00640 1 -- 172.21.15.94:0/1273335104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc0cfcb0 msgr2=0x7f80dc0d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:59.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.919+0000 7f80e0e00640 1 --2- 172.21.15.94:0/1273335104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc0cfcb0 0x7f80dc0d00d0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009900 tx=0x7f80c801c8b0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.919+0000 7f80e0e00640 1 -- 172.21.15.94:0/1273335104 shutdown_connections 2024-09-10T23:39:59.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.919+0000 7f80e0e00640 1 --2- 172.21.15.94:0/1273335104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80dc0cddb0 0x7f80dc0ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.919+0000 7f80e0e00640 1 --2- 172.21.15.94:0/1273335104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc0cfcb0 0x7f80dc0d00d0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.919+0000 7f80e0e00640 1 -- 172.21.15.94:0/1273335104 >> 172.21.15.94:0/1273335104 conn(0x7f80dc06cdf0 msgr2=0x7f80dc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:39:59.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.919+0000 7f80e0e00640 1 -- 172.21.15.94:0/1273335104 shutdown_connections 2024-09-10T23:39:59.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.919+0000 7f80e0e00640 1 -- 172.21.15.94:0/1273335104 wait complete. 2024-09-10T23:39:59.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.919+0000 7f80e0e00640 1 Processor -- start 2024-09-10T23:39:59.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.919+0000 7f80e0e00640 1 -- start start 2024-09-10T23:39:59.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80e0e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80dc0cddb0 0x7f80dc154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:59.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80e0e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80dc0cfcb0 0x7f80dc15e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:59.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80e0e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc15ba50 0x7f80dc1490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:59.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80e0e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc168be0 con 0x7f80dc15ba50 2024-09-10T23:39:59.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80e0e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc168a60 con 0x7f80dc0cfcb0 2024-09-10T23:39:59.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80e0e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f80dc168d60 con 0x7f80dc0cddb0 2024-09-10T23:39:59.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80d9400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80dc0cfcb0 0x7f80dc15e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:59.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80da800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc15ba50 0x7f80dc1490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:59.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80d9e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80dc0cddb0 0x7f80dc154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:59.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80d9400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80dc0cfcb0 0x7f80dc15e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:45656/0 (socket says 172.21.15.94:45656) 2024-09-10T23:39:59.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80d9400640 1 -- 172.21.15.94:0/2643399659 learned_addr learned my addr 172.21.15.94:0/2643399659 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:39:59.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80d9400640 1 -- 172.21.15.94:0/2643399659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80dc0cddb0 msgr2=0x7f80dc154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:59.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80d9400640 1 --2- 172.21.15.94:0/2643399659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80dc0cddb0 0x7f80dc154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80d9400640 1 -- 172.21.15.94:0/2643399659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc15ba50 msgr2=0x7f80dc1490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:39:59.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80d9400640 1 --2- 172.21.15.94:0/2643399659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc15ba50 0x7f80dc1490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:39:59.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80d9400640 1 -- 172.21.15.94:0/2643399659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80dc1fab00 con 0x7f80dc0cfcb0 2024-09-10T23:39:59.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80d9e00640 1 --2- 172.21.15.94:0/2643399659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80dc0cddb0 0x7f80dc154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:39:59.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80d9400640 1 --2- 172.21.15.94:0/2643399659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80dc0cfcb0 0x7f80dc15e390 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009850 tx=0x7f80c80a5880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:59.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80d2000640 1 -- 172.21.15.94:0/2643399659 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c80b2040 con 0x7f80dc0cfcb0 2024-09-10T23:39:59.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80d2000640 1 -- 172.21.15.94:0/2643399659 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80c8005320 con 0x7f80dc0cfcb0 2024-09-10T23:39:59.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80e0e00640 1 -- 172.21.15.94:0/2643399659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80dc1fade0 con 0x7f80dc0cfcb0 2024-09-10T23:39:59.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80d2000640 1 -- 172.21.15.94:0/2643399659 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8003700 con 0x7f80dc0cfcb0 2024-09-10T23:39:59.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80e0e00640 1 -- 172.21.15.94:0/2643399659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80dc1fb3a0 con 0x7f80dc0cfcb0 2024-09-10T23:39:59.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.923+0000 7f80e0e00640 1 -- 172.21.15.94:0/2643399659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80dc0ce220 con 0x7f80dc0cfcb0 2024-09-10T23:39:59.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.927+0000 7f80d2000640 1 -- 172.21.15.94:0/2643399659 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f80c80c0030 con 0x7f80dc0cfcb0 2024-09-10T23:39:59.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.927+0000 7f80d2000640 1 --2- 172.21.15.94:0/2643399659 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f80bc072e50 0x7f80bc075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:39:59.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.927+0000 7f80d2000640 1 -- 172.21.15.94:0/2643399659 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 4885+0+0 (secure 0 0 0) 0x7f80c812df50 con 0x7f80dc0cfcb0 2024-09-10T23:39:59.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.927+0000 7f80d9e00640 1 --2- 172.21.15.94:0/2643399659 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f80bc072e50 0x7f80bc075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:39:59.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.927+0000 7f80d9e00640 1 --2- 172.21.15.94:0/2643399659 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f80bc072e50 0x7f80bc075310 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f80cc007fd0 tx=0x7f80cc009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:39:59.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:39:59.931+0000 7f80d2000640 1 -- 172.21.15.94:0/2643399659 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f80c80164d0 con 0x7f80dc0cfcb0 2024-09-10T23:40:00.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.099+0000 7f80e0e00640 1 -- 172.21.15.94:0/2643399659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f80dc0008e0 con 0x7f80dc0cfcb0 2024-09-10T23:40:00.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.099+0000 7f80d2000640 1 -- 172.21.15.94:0/2643399659 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v147) ==== 84+0+65 (secure 0 0 0) 0x7f80c80f8780 con 0x7f80dc0cfcb0 2024-09-10T23:40:00.102 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:00.103 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:40:00.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:00.103 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-09-10T23:40:00.103 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-09-10T23:40:00.103 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:00.103 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:40:00.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.099+0000 7f80e0e00640 1 -- 172.21.15.94:0/2643399659 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f80bc072e50 msgr2=0x7f80bc075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:00.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.099+0000 7f80e0e00640 1 --2- 172.21.15.94:0/2643399659 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f80bc072e50 0x7f80bc075310 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f80cc007fd0 tx=0x7f80cc009040 comp rx=0 tx=0).stop 2024-09-10T23:40:00.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.099+0000 7f80e0e00640 1 -- 172.21.15.94:0/2643399659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80dc0cfcb0 msgr2=0x7f80dc15e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:00.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.099+0000 7f80e0e00640 1 --2- 172.21.15.94:0/2643399659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80dc0cfcb0 0x7f80dc15e390 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009850 tx=0x7f80c80a5880 comp rx=0 tx=0).stop 2024-09-10T23:40:00.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.099+0000 7f80e0e00640 1 -- 172.21.15.94:0/2643399659 shutdown_connections 2024-09-10T23:40:00.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.099+0000 7f80e0e00640 1 --2- 172.21.15.94:0/2643399659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80dc0cfcb0 0x7f80dc15e390 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:00.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.099+0000 7f80e0e00640 1 --2- 172.21.15.94:0/2643399659 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f80bc072e50 0x7f80bc075310 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:00.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.099+0000 7f80e0e00640 1 --2- 172.21.15.94:0/2643399659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc15ba50 0x7f80dc1490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:00.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.099+0000 7f80e0e00640 1 --2- 172.21.15.94:0/2643399659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80dc0cddb0 0x7f80dc154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:00.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.099+0000 7f80e0e00640 1 -- 172.21.15.94:0/2643399659 >> 172.21.15.94:0/2643399659 conn(0x7f80dc06cdf0 msgr2=0x7f80dc159bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:00.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.099+0000 7f80e0e00640 1 -- 172.21.15.94:0/2643399659 shutdown_connections 2024-09-10T23:40:00.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.099+0000 7f80e0e00640 1 -- 172.21.15.94:0/2643399659 wait complete. 2024-09-10T23:40:00.124 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T23:40:00.124 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T23:40:00.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0937a00640 1 -- 172.21.15.94:0/3546943924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09300d2290 msgr2=0x7f0930166b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:00.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0937a00640 1 --2- 172.21.15.94:0/3546943924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09300d2290 0x7f0930166b70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0928009bf0 tx=0x7f092801cbb0 comp rx=0 tx=0).stop 2024-09-10T23:40:00.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0937a00640 1 -- 172.21.15.94:0/3546943924 shutdown_connections 2024-09-10T23:40:00.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0937a00640 1 --2- 172.21.15.94:0/3546943924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09300d2290 0x7f0930166b70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:00.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0937a00640 1 -- 172.21.15.94:0/3546943924 >> 172.21.15.94:0/3546943924 conn(0x7f093009aa70 msgr2=0x7f093009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:00.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0937a00640 1 -- 172.21.15.94:0/3546943924 shutdown_connections 2024-09-10T23:40:00.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0937a00640 1 -- 172.21.15.94:0/3546943924 wait complete. 2024-09-10T23:40:00.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0937a00640 1 Processor -- start 2024-09-10T23:40:00.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0937a00640 1 -- start start 2024-09-10T23:40:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0937a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09300ae980 0x7f0930155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0937a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09300d2290 0x7f093015a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0937a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f093015bd30 0x7f093015ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0937a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f093016a430 con 0x7f09300ae980 2024-09-10T23:40:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0937a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f093016a2b0 con 0x7f09300d2290 2024-09-10T23:40:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0937a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f093016a5b0 con 0x7f093015bd30 2024-09-10T23:40:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0935000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09300ae980 0x7f0930155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0935000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09300ae980 0x7f0930155890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39820/0 (socket says 172.21.15.94:39820) 2024-09-10T23:40:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0935000640 1 -- 172.21.15.94:0/2127135820 learned_addr learned my addr 172.21.15.94:0/2127135820 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0935a00640 1 --2- 172.21.15.94:0/2127135820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f093015bd30 0x7f093015ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0927e00640 1 --2- 172.21.15.94:0/2127135820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09300d2290 0x7f093015a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:00.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0935000640 1 -- 172.21.15.94:0/2127135820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f093015bd30 msgr2=0x7f093015ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:00.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0935000640 1 --2- 172.21.15.94:0/2127135820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f093015bd30 0x7f093015ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:00.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0935000640 1 -- 172.21.15.94:0/2127135820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09300d2290 msgr2=0x7f093015a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:00.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0935000640 1 --2- 172.21.15.94:0/2127135820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09300d2290 0x7f093015a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:00.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.267+0000 7f0935000640 1 -- 172.21.15.94:0/2127135820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093015b610 con 0x7f09300ae980 2024-09-10T23:40:00.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.271+0000 7f0935000640 1 --2- 172.21.15.94:0/2127135820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09300ae980 0x7f0930155890 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f092000c400 tx=0x7f092000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:00.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.271+0000 7f092d600640 1 -- 172.21.15.94:0/2127135820 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920017070 con 0x7f09300ae980 2024-09-10T23:40:00.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.271+0000 7f092d600640 1 -- 172.21.15.94:0/2127135820 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0920005400 con 0x7f09300ae980 2024-09-10T23:40:00.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.271+0000 7f092d600640 1 -- 172.21.15.94:0/2127135820 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920003860 con 0x7f09300ae980 2024-09-10T23:40:00.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.271+0000 7f0937a00640 1 -- 172.21.15.94:0/2127135820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093015b330 con 0x7f09300ae980 2024-09-10T23:40:00.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.271+0000 7f0937a00640 1 -- 172.21.15.94:0/2127135820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09300cd1a0 con 0x7f09300ae980 2024-09-10T23:40:00.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.271+0000 7f0937a00640 1 -- 172.21.15.94:0/2127135820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09300d1b80 con 0x7f09300ae980 2024-09-10T23:40:00.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.271+0000 7f092d600640 1 -- 172.21.15.94:0/2127135820 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f0920003a00 con 0x7f09300ae980 2024-09-10T23:40:00.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.271+0000 7f092d600640 1 --2- 172.21.15.94:0/2127135820 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f090c072e50 0x7f090c075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:00.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.271+0000 7f092d600640 1 -- 172.21.15.94:0/2127135820 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 4885+0+0 (secure 0 0 0) 0x7f09200910c0 con 0x7f09300ae980 2024-09-10T23:40:00.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.271+0000 7f0927e00640 1 --2- 172.21.15.94:0/2127135820 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f090c072e50 0x7f090c075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:00.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.275+0000 7f0927e00640 1 --2- 172.21.15.94:0/2127135820 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f090c072e50 0x7f090c075310 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0918007f30 tx=0x7f0918007ec0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:00.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.275+0000 7f092d600640 1 -- 172.21.15.94:0/2127135820 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f092005db80 con 0x7f09300ae980 2024-09-10T23:40:00.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:00.447+0000 7f0937a00640 1 -- 172.21.15.94:0/2127135820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f09300d0710 con 0x7f09300ae980 2024-09-10T23:40:01.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.319+0000 7f092d600640 1 -- 172.21.15.94:0/2127135820 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v148) ==== 138+0+0 (secure 0 0 0) 0x7f09200625b0 con 0x7f09300ae980 2024-09-10T23:40:01.323 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-09-10T23:40:01.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.319+0000 7f0937a00640 1 -- 172.21.15.94:0/2127135820 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f090c072e50 msgr2=0x7f090c075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:01.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.319+0000 7f0937a00640 1 --2- 172.21.15.94:0/2127135820 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f090c072e50 0x7f090c075310 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0918007f30 tx=0x7f0918007ec0 comp rx=0 tx=0).stop 2024-09-10T23:40:01.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.319+0000 7f0937a00640 1 -- 172.21.15.94:0/2127135820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09300ae980 msgr2=0x7f0930155890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:01.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.319+0000 7f0937a00640 1 --2- 172.21.15.94:0/2127135820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09300ae980 0x7f0930155890 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f092000c400 tx=0x7f092000c8c0 comp rx=0 tx=0).stop 2024-09-10T23:40:01.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.319+0000 7f0937a00640 1 -- 172.21.15.94:0/2127135820 shutdown_connections 2024-09-10T23:40:01.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.319+0000 7f0937a00640 1 --2- 172.21.15.94:0/2127135820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09300d2290 0x7f093015a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:01.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.319+0000 7f0937a00640 1 --2- 172.21.15.94:0/2127135820 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f090c072e50 0x7f090c075310 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:01.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.319+0000 7f0937a00640 1 --2- 172.21.15.94:0/2127135820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f093015bd30 0x7f093015ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:01.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.319+0000 7f0937a00640 1 --2- 172.21.15.94:0/2127135820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09300ae980 0x7f0930155890 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:01.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.319+0000 7f0937a00640 1 -- 172.21.15.94:0/2127135820 >> 172.21.15.94:0/2127135820 conn(0x7f093009aa70 msgr2=0x7f093015e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:01.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.319+0000 7f0937a00640 1 -- 172.21.15.94:0/2127135820 shutdown_connections 2024-09-10T23:40:01.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.323+0000 7f0937a00640 1 -- 172.21.15.94:0/2127135820 wait complete. 2024-09-10T23:40:01.351 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T23:40:01.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f176aa00640 1 -- 172.21.15.94:0/2565655509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1748003c40 msgr2=0x7f1748004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:01.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f176aa00640 1 --2- 172.21.15.94:0/2565655509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1748003c40 0x7f1748004040 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f1754009b60 tx=0x7f175401d270 comp rx=0 tx=0).stop 2024-09-10T23:40:01.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f176aa00640 1 -- 172.21.15.94:0/2565655509 shutdown_connections 2024-09-10T23:40:01.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f176aa00640 1 --2- 172.21.15.94:0/2565655509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1748003c40 0x7f1748004040 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:01.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f176aa00640 1 -- 172.21.15.94:0/2565655509 >> 172.21.15.94:0/2565655509 conn(0x7f17640d1c60 msgr2=0x7f17640d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:01.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f176aa00640 1 -- 172.21.15.94:0/2565655509 shutdown_connections 2024-09-10T23:40:01.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f176aa00640 1 -- 172.21.15.94:0/2565655509 wait complete. 2024-09-10T23:40:01.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f176aa00640 1 Processor -- start 2024-09-10T23:40:01.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f176aa00640 1 -- start start 2024-09-10T23:40:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f176aa00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1748003c40 0x7f17641f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f176aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17641f8ae0 0x7f17640c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f176aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17640c7270 0x7f17640b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f176aa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f17640b1ab0 con 0x7f17640c7270 2024-09-10T23:40:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f176aa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f17640b1930 con 0x7f17641f8ae0 2024-09-10T23:40:01.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f176aa00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1764160610 con 0x7f1748003c40 2024-09-10T23:40:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f1768a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17640c7270 0x7f17640b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f1768a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17640c7270 0x7f17640b6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39848/0 (socket says 172.21.15.94:39848) 2024-09-10T23:40:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f1768a00640 1 -- 172.21.15.94:0/4168120728 learned_addr learned my addr 172.21.15.94:0/4168120728 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f1763e00640 1 --2- 172.21.15.94:0/4168120728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1748003c40 0x7f17641f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f175be00640 1 --2- 172.21.15.94:0/4168120728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17641f8ae0 0x7f17640c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f1768a00640 1 -- 172.21.15.94:0/4168120728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1748003c40 msgr2=0x7f17641f85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f1768a00640 1 --2- 172.21.15.94:0/4168120728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1748003c40 0x7f17641f85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f1768a00640 1 -- 172.21.15.94:0/4168120728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17641f8ae0 msgr2=0x7f17640c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f1768a00640 1 --2- 172.21.15.94:0/4168120728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17641f8ae0 0x7f17640c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f1768a00640 1 -- 172.21.15.94:0/4168120728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17640c78f0 con 0x7f17640c7270 2024-09-10T23:40:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.499+0000 7f1768a00640 1 --2- 172.21.15.94:0/4168120728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17640c7270 0x7f17640b6fb0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f17640d1880 tx=0x7f175c002200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.503+0000 7f1760c00640 1 -- 172.21.15.94:0/4168120728 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c011900 con 0x7f17640c7270 2024-09-10T23:40:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.503+0000 7f1760c00640 1 -- 172.21.15.94:0/4168120728 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f175c018070 con 0x7f17640c7270 2024-09-10T23:40:01.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.503+0000 7f1760c00640 1 -- 172.21.15.94:0/4168120728 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c011d60 con 0x7f17640c7270 2024-09-10T23:40:01.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.503+0000 7f176aa00640 1 -- 172.21.15.94:0/4168120728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17640c54a0 con 0x7f17640c7270 2024-09-10T23:40:01.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.503+0000 7f176aa00640 1 -- 172.21.15.94:0/4168120728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17640c5a60 con 0x7f17640c7270 2024-09-10T23:40:01.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.503+0000 7f176aa00640 1 -- 172.21.15.94:0/4168120728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1764160b00 con 0x7f17640c7270 2024-09-10T23:40:01.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.503+0000 7f1760c00640 1 -- 172.21.15.94:0/4168120728 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f175c01d460 con 0x7f17640c7270 2024-09-10T23:40:01.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.507+0000 7f1760c00640 1 --2- 172.21.15.94:0/4168120728 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1740072e50 0x7f1740075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:01.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.507+0000 7f1760c00640 1 -- 172.21.15.94:0/4168120728 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5255+0+0 (secure 0 0 0) 0x7f175c095490 con 0x7f17640c7270 2024-09-10T23:40:01.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.507+0000 7f1763e00640 1 --2- 172.21.15.94:0/4168120728 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1740072e50 0x7f1740075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:01.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.507+0000 7f1763e00640 1 --2- 172.21.15.94:0/4168120728 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1740072e50 0x7f1740075310 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f17540ae000 tx=0x7f175401a010 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:01.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.507+0000 7f1760c00640 1 -- 172.21.15.94:0/4168120728 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f175c05f730 con 0x7f17640c7270 2024-09-10T23:40:01.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:01.675+0000 7f176aa00640 1 -- 172.21.15.94:0/4168120728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f17640c8af0 con 0x7f17640c7270 2024-09-10T23:40:02.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.327+0000 7f1760c00640 1 -- 172.21.15.94:0/4168120728 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v149) ==== 170+0+0 (secure 0 0 0) 0x7f175c064160 con 0x7f17640c7270 2024-09-10T23:40:02.333 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-09-10T23:40:02.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.331+0000 7f176aa00640 1 -- 172.21.15.94:0/4168120728 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1740072e50 msgr2=0x7f1740075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:02.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.331+0000 7f176aa00640 1 --2- 172.21.15.94:0/4168120728 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1740072e50 0x7f1740075310 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f17540ae000 tx=0x7f175401a010 comp rx=0 tx=0).stop 2024-09-10T23:40:02.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.331+0000 7f176aa00640 1 -- 172.21.15.94:0/4168120728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17640c7270 msgr2=0x7f17640b6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:02.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.331+0000 7f176aa00640 1 --2- 172.21.15.94:0/4168120728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17640c7270 0x7f17640b6fb0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f17640d1880 tx=0x7f175c002200 comp rx=0 tx=0).stop 2024-09-10T23:40:02.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.331+0000 7f176aa00640 1 -- 172.21.15.94:0/4168120728 shutdown_connections 2024-09-10T23:40:02.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.331+0000 7f176aa00640 1 --2- 172.21.15.94:0/4168120728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17641f8ae0 0x7f17640c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:02.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.331+0000 7f176aa00640 1 --2- 172.21.15.94:0/4168120728 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1740072e50 0x7f1740075310 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:02.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.331+0000 7f176aa00640 1 --2- 172.21.15.94:0/4168120728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17640c7270 0x7f17640b6fb0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:02.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.331+0000 7f176aa00640 1 --2- 172.21.15.94:0/4168120728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1748003c40 0x7f17641f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:02.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.331+0000 7f176aa00640 1 -- 172.21.15.94:0/4168120728 >> 172.21.15.94:0/4168120728 conn(0x7f17640d1c60 msgr2=0x7f17640cf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:02.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.331+0000 7f176aa00640 1 -- 172.21.15.94:0/4168120728 shutdown_connections 2024-09-10T23:40:02.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.331+0000 7f176aa00640 1 -- 172.21.15.94:0/4168120728 wait complete. 2024-09-10T23:40:02.371 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T23:40:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.511+0000 7f99a4a00640 1 -- 172.21.15.94:0/2201528424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99a00ce280 msgr2=0x7f99a00cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.511+0000 7f99a4a00640 1 --2- 172.21.15.94:0/2201528424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99a00ce280 0x7f99a00cf280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f999000a950 tx=0x7f999001db70 comp rx=0 tx=0).stop 2024-09-10T23:40:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.511+0000 7f99a4a00640 1 -- 172.21.15.94:0/2201528424 shutdown_connections 2024-09-10T23:40:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.511+0000 7f99a4a00640 1 --2- 172.21.15.94:0/2201528424 >> v2:172.21.15.181:3300/0 conn(0x7f99a00c64f0 0x7f99a00ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.511+0000 7f99a4a00640 1 --2- 172.21.15.94:0/2201528424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99a00ce280 0x7f99a00cf280 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.511+0000 7f99a4a00640 1 -- 172.21.15.94:0/2201528424 >> 172.21.15.94:0/2201528424 conn(0x7f99a00cfaf0 msgr2=0x7f99a00cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.511+0000 7f99a4a00640 1 -- 172.21.15.94:0/2201528424 shutdown_connections 2024-09-10T23:40:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.511+0000 7f99a4a00640 1 -- 172.21.15.94:0/2201528424 wait complete. 2024-09-10T23:40:02.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.511+0000 7f99a4a00640 1 Processor -- start 2024-09-10T23:40:02.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.511+0000 7f99a4a00640 1 -- start start 2024-09-10T23:40:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f99a4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99a006e7c0 0x7f99a0159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f99a4a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99a00c64f0 0x7f99a00aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f99a4a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99a00ce280 0x7f99a00b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f99a4a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f99a0167680 con 0x7f99a006e7c0 2024-09-10T23:40:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f99a4a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f99a0167500 con 0x7f99a00ce280 2024-09-10T23:40:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f99a4a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f99a0167800 con 0x7f99a00c64f0 2024-09-10T23:40:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f999de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99a006e7c0 0x7f99a0159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f999de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99a006e7c0 0x7f99a0159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39860/0 (socket says 172.21.15.94:39860) 2024-09-10T23:40:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f999de00640 1 -- 172.21.15.94:0/3873982958 learned_addr learned my addr 172.21.15.94:0/3873982958 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f999d400640 1 --2- 172.21.15.94:0/3873982958 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99a00c64f0 0x7f99a00aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f999e800640 1 --2- 172.21.15.94:0/3873982958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99a00ce280 0x7f99a00b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f999de00640 1 -- 172.21.15.94:0/3873982958 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99a00c64f0 msgr2=0x7f99a00aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f999de00640 1 --2- 172.21.15.94:0/3873982958 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99a00c64f0 0x7f99a00aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f999de00640 1 -- 172.21.15.94:0/3873982958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99a00ce280 msgr2=0x7f99a00b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f999de00640 1 --2- 172.21.15.94:0/3873982958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99a00ce280 0x7f99a00b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f999de00640 1 -- 172.21.15.94:0/3873982958 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a006f540 con 0x7f99a006e7c0 2024-09-10T23:40:02.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f999de00640 1 --2- 172.21.15.94:0/3873982958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99a006e7c0 0x7f99a0159c90 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f999000a8a0 tx=0x7f9990004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:02.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f9996000640 1 -- 172.21.15.94:0/3873982958 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99900bb020 con 0x7f99a006e7c0 2024-09-10T23:40:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f9996000640 1 -- 172.21.15.94:0/3873982958 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9990005320 con 0x7f99a006e7c0 2024-09-10T23:40:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f9996000640 1 -- 172.21.15.94:0/3873982958 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99900b3040 con 0x7f99a006e7c0 2024-09-10T23:40:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f99a4a00640 1 -- 172.21.15.94:0/3873982958 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a0158bf0 con 0x7f99a006e7c0 2024-09-10T23:40:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f99a4a00640 1 -- 172.21.15.94:0/3873982958 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a01591b0 con 0x7f99a006e7c0 2024-09-10T23:40:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.515+0000 7f99a4a00640 1 -- 172.21.15.94:0/3873982958 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a00ce660 con 0x7f99a006e7c0 2024-09-10T23:40:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.519+0000 7f9996000640 1 -- 172.21.15.94:0/3873982958 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f999000b040 con 0x7f99a006e7c0 2024-09-10T23:40:02.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.519+0000 7f9996000640 1 --2- 172.21.15.94:0/3873982958 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9980072e50 0x7f9980075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:02.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.519+0000 7f9996000640 1 -- 172.21.15.94:0/3873982958 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5637+0+0 (secure 0 0 0) 0x7f999012bcd0 con 0x7f99a006e7c0 2024-09-10T23:40:02.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.519+0000 7f999d400640 1 --2- 172.21.15.94:0/3873982958 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9980072e50 0x7f9980075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:02.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.519+0000 7f999d400640 1 --2- 172.21.15.94:0/3873982958 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9980072e50 0x7f9980075310 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f998c009600 tx=0x7f998c0091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:02.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.523+0000 7f9996000640 1 -- 172.21.15.94:0/3873982958 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f99900c5060 con 0x7f99a006e7c0 2024-09-10T23:40:02.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:02.719+0000 7f99a4a00640 1 -- 172.21.15.94:0/3873982958 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f99a014bbd0 con 0x7f99a006e7c0 2024-09-10T23:40:03.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.351+0000 7f9996000640 1 -- 172.21.15.94:0/3873982958 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v103) ==== 174+0+0 (secure 0 0 0) 0x7f9990017760 con 0x7f99a006e7c0 2024-09-10T23:40:03.357 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 24 and data pool 25 2024-09-10T23:40:03.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.359+0000 7f99a4a00640 1 -- 172.21.15.94:0/3873982958 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9980072e50 msgr2=0x7f9980075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:03.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.359+0000 7f99a4a00640 1 --2- 172.21.15.94:0/3873982958 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9980072e50 0x7f9980075310 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f998c009600 tx=0x7f998c0091e0 comp rx=0 tx=0).stop 2024-09-10T23:40:03.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.359+0000 7f99a4a00640 1 -- 172.21.15.94:0/3873982958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99a006e7c0 msgr2=0x7f99a0159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:03.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.359+0000 7f99a4a00640 1 --2- 172.21.15.94:0/3873982958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99a006e7c0 0x7f99a0159c90 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f999000a8a0 tx=0x7f9990004620 comp rx=0 tx=0).stop 2024-09-10T23:40:03.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.359+0000 7f99a4a00640 1 -- 172.21.15.94:0/3873982958 shutdown_connections 2024-09-10T23:40:03.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.359+0000 7f99a4a00640 1 --2- 172.21.15.94:0/3873982958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99a00ce280 0x7f99a00b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:03.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.359+0000 7f99a4a00640 1 --2- 172.21.15.94:0/3873982958 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9980072e50 0x7f9980075310 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:03.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.359+0000 7f99a4a00640 1 --2- 172.21.15.94:0/3873982958 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99a00c64f0 0x7f99a00aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:03.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.359+0000 7f99a4a00640 1 --2- 172.21.15.94:0/3873982958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f99a006e7c0 0x7f99a0159c90 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:03.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.359+0000 7f99a4a00640 1 -- 172.21.15.94:0/3873982958 >> 172.21.15.94:0/3873982958 conn(0x7f99a00cfaf0 msgr2=0x7f99a014d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:03.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.359+0000 7f99a4a00640 1 -- 172.21.15.94:0/3873982958 shutdown_connections 2024-09-10T23:40:03.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.359+0000 7f99a4a00640 1 -- 172.21.15.94:0/3873982958 wait complete. 2024-09-10T23:40:03.391 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:40:03.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.531+0000 7fb376800640 1 -- 172.21.15.94:0/3884802608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb358003a10 msgr2=0x7fb358003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.531+0000 7fb376800640 1 --2- 172.21.15.94:0/3884802608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb358003a10 0x7fb358003e30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb364009b60 tx=0x7fb36401d2b0 comp rx=0 tx=0).stop 2024-09-10T23:40:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.531+0000 7fb375e00640 1 -- 172.21.15.94:0/3884802608 reap_dead start 2024-09-10T23:40:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.531+0000 7fb376800640 1 -- 172.21.15.94:0/3884802608 shutdown_connections 2024-09-10T23:40:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.531+0000 7fb376800640 1 -- 172.21.15.94:0/3884802608 >> 172.21.15.94:0/3884802608 conn(0x7fb37009aa70 msgr2=0x7fb37009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.531+0000 7fb376800640 1 -- 172.21.15.94:0/3884802608 shutdown_connections 2024-09-10T23:40:03.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.531+0000 7fb376800640 1 -- 172.21.15.94:0/3884802608 wait complete. 2024-09-10T23:40:03.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.531+0000 7fb376800640 1 Processor -- start 2024-09-10T23:40:03.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb376800640 1 -- start start 2024-09-10T23:40:03.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb376800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb358003a10 0x7fb3701f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:03.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb376800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3700b4b60 0x7fb3700cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:03.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb376800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3700cd990 0x7fb3701f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:03.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb376800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3700cfc70 con 0x7fb3700cd990 2024-09-10T23:40:03.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb376800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb37006ede0 con 0x7fb3700b4b60 2024-09-10T23:40:03.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb376800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb3700cfdf0 con 0x7fb358003a10 2024-09-10T23:40:03.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb375e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3700cd990 0x7fb3701f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:03.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb375e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3700cd990 0x7fb3701f81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39886/0 (socket says 172.21.15.94:39886) 2024-09-10T23:40:03.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb375e00640 1 -- 172.21.15.94:0/2083675084 learned_addr learned my addr 172.21.15.94:0/2083675084 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:03.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb374a00640 1 --2- 172.21.15.94:0/2083675084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3700b4b60 0x7fb3700cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:03.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb375400640 1 --2- 172.21.15.94:0/2083675084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb358003a10 0x7fb3701f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:03.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb375e00640 1 -- 172.21.15.94:0/2083675084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb358003a10 msgr2=0x7fb3701f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:03.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb375e00640 1 --2- 172.21.15.94:0/2083675084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb358003a10 0x7fb3701f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:03.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb375e00640 1 -- 172.21.15.94:0/2083675084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3700b4b60 msgr2=0x7fb3700cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:03.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb375e00640 1 --2- 172.21.15.94:0/2083675084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3700b4b60 0x7fb3700cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:03.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb375e00640 1 -- 172.21.15.94:0/2083675084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3701f4830 con 0x7fb3700cd990 2024-09-10T23:40:03.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb375e00640 1 --2- 172.21.15.94:0/2083675084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3700cd990 0x7fb3701f81a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fb3700cf700 tx=0x7fb36800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb36d600640 1 -- 172.21.15.94:0/2083675084 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36801e070 con 0x7fb3700cd990 2024-09-10T23:40:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb36d600640 1 -- 172.21.15.94:0/2083675084 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb36800e7c0 con 0x7fb3700cd990 2024-09-10T23:40:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb376800640 1 -- 172.21.15.94:0/2083675084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3700b74b0 con 0x7fb3700cd990 2024-09-10T23:40:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb36d600640 1 -- 172.21.15.94:0/2083675084 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368009ab0 con 0x7fb3700cd990 2024-09-10T23:40:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb376800640 1 -- 172.21.15.94:0/2083675084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3701f4ad0 con 0x7fb3700cd990 2024-09-10T23:40:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.535+0000 7fb376800640 1 -- 172.21.15.94:0/2083675084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3700cde10 con 0x7fb3700cd990 2024-09-10T23:40:03.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.539+0000 7fb36d600640 1 -- 172.21.15.94:0/2083675084 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fb36800ec10 con 0x7fb3700cd990 2024-09-10T23:40:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.539+0000 7fb36d600640 1 --2- 172.21.15.94:0/2083675084 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb350072e80 0x7fb350075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.539+0000 7fb36d600640 1 -- 172.21.15.94:0/2083675084 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5737+0+0 (secure 0 0 0) 0x7fb368095510 con 0x7fb3700cd990 2024-09-10T23:40:03.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.539+0000 7fb375400640 1 --2- 172.21.15.94:0/2083675084 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb350072e80 0x7fb350075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:03.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.539+0000 7fb375400640 1 --2- 172.21.15.94:0/2083675084 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb350072e80 0x7fb350075340 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb3640af000 tx=0x7fb36401a050 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:03.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.543+0000 7fb36d600640 1 -- 172.21.15.94:0/2083675084 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb36805f550 con 0x7fb3700cd990 2024-09-10T23:40:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.711+0000 7fb376800640 1 -- 172.21.15.94:0/2083675084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3700c81d0 con 0x7fb3700cd990 2024-09-10T23:40:03.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.711+0000 7fb36d600640 1 -- 172.21.15.94:0/2083675084 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) ==== 74+0+16728 (secure 0 0 0) 0x7fb3700c81d0 con 0x7fb3700cd990 2024-09-10T23:40:03.715 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:03.715 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":150,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:40:02.727072+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:40:00.461748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-10T23:40:01.690823+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:40:03.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.711+0000 7fb376800640 1 -- 172.21.15.94:0/2083675084 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb350072e80 msgr2=0x7fb350075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:03.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.711+0000 7fb376800640 1 --2- 172.21.15.94:0/2083675084 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb350072e80 0x7fb350075340 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb3640af000 tx=0x7fb36401a050 comp rx=0 tx=0).stop 2024-09-10T23:40:03.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.711+0000 7fb376800640 1 -- 172.21.15.94:0/2083675084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3700cd990 msgr2=0x7fb3701f81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:03.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.711+0000 7fb376800640 1 --2- 172.21.15.94:0/2083675084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3700cd990 0x7fb3701f81a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fb3700cf700 tx=0x7fb36800f790 comp rx=0 tx=0).stop 2024-09-10T23:40:03.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.711+0000 7fb376800640 1 -- 172.21.15.94:0/2083675084 shutdown_connections 2024-09-10T23:40:03.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.711+0000 7fb376800640 1 --2- 172.21.15.94:0/2083675084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3700b4b60 0x7fb3700cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:03.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.711+0000 7fb376800640 1 --2- 172.21.15.94:0/2083675084 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb350072e80 0x7fb350075340 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:03.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.711+0000 7fb376800640 1 --2- 172.21.15.94:0/2083675084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3700cd990 0x7fb3701f81a0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:03.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.711+0000 7fb376800640 1 --2- 172.21.15.94:0/2083675084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb358003a10 0x7fb3701f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:03.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.711+0000 7fb376800640 1 -- 172.21.15.94:0/2083675084 >> 172.21.15.94:0/2083675084 conn(0x7fb37009aa70 msgr2=0x7fb3700b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:03.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.711+0000 7fb376800640 1 -- 172.21.15.94:0/2083675084 shutdown_connections 2024-09-10T23:40:03.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.711+0000 7fb376800640 1 -- 172.21.15.94:0/2083675084 wait complete. 2024-09-10T23:40:03.737 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:40:03.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad6ca00640 1 -- 172.21.15.94:0/2009011288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad680b3e30 msgr2=0x7fad680b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:03.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad6ca00640 1 --2- 172.21.15.94:0/2009011288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad680b3e30 0x7fad680b4260 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fad68076740 tx=0x7fad6001c830 comp rx=0 tx=0).stop 2024-09-10T23:40:03.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad6ca00640 1 -- 172.21.15.94:0/2009011288 shutdown_connections 2024-09-10T23:40:03.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad6ca00640 1 --2- 172.21.15.94:0/2009011288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad680b3e30 0x7fad680b4260 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:03.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad6ca00640 1 --2- 172.21.15.94:0/2009011288 >> v2:172.21.15.181:3300/0 conn(0x7fad68070860 0x7fad68070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:03.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad6ca00640 1 -- 172.21.15.94:0/2009011288 >> 172.21.15.94:0/2009011288 conn(0x7fad680d0b70 msgr2=0x7fad680d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:03.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad6ca00640 1 -- 172.21.15.94:0/2009011288 shutdown_connections 2024-09-10T23:40:03.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad6ca00640 1 -- 172.21.15.94:0/2009011288 wait complete. 2024-09-10T23:40:03.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad6ca00640 1 Processor -- start 2024-09-10T23:40:03.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad6ca00640 1 -- start start 2024-09-10T23:40:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad6ca00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad68070860 0x7fad681633a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad6ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad680b3e30 0x7fad6815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad6ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad680d0770 0x7fad6815d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad6ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad68167c60 con 0x7fad680d0770 2024-09-10T23:40:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad6ca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fad68167ae0 con 0x7fad680b3e30 2024-09-10T23:40:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad6ca00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fad68167de0 con 0x7fad68070860 2024-09-10T23:40:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad66800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad680d0770 0x7fad6815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad66800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad680d0770 0x7fad6815d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34496/0 (socket says 172.21.15.94:34496) 2024-09-10T23:40:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.875+0000 7fad66800640 1 -- 172.21.15.94:0/2686588268 learned_addr learned my addr 172.21.15.94:0/2686588268 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad66800640 1 -- 172.21.15.94:0/2686588268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad68070860 msgr2=0x7fad681633a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad65e00640 1 --2- 172.21.15.94:0/2686588268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad68070860 0x7fad681633a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad5d400640 1 --2- 172.21.15.94:0/2686588268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad680b3e30 0x7fad6815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad66800640 1 --2- 172.21.15.94:0/2686588268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad68070860 0x7fad681633a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad66800640 1 -- 172.21.15.94:0/2686588268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad680b3e30 msgr2=0x7fad6815a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad66800640 1 --2- 172.21.15.94:0/2686588268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad680b3e30 0x7fad6815a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad66800640 1 -- 172.21.15.94:0/2686588268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad680b2410 con 0x7fad680d0770 2024-09-10T23:40:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad65e00640 1 --2- 172.21.15.94:0/2686588268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad68070860 0x7fad681633a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad5d400640 1 --2- 172.21.15.94:0/2686588268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad680b3e30 0x7fad6815a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad66800640 1 --2- 172.21.15.94:0/2686588268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad680d0770 0x7fad6815d820 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fad60019440 tx=0x7fad600a5670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad5ea00640 1 -- 172.21.15.94:0/2686588268 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad600bd070 con 0x7fad680d0770 2024-09-10T23:40:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad5ea00640 1 -- 172.21.15.94:0/2686588268 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad600043b0 con 0x7fad680d0770 2024-09-10T23:40:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad5ea00640 1 -- 172.21.15.94:0/2686588268 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad600b2040 con 0x7fad680d0770 2024-09-10T23:40:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad6ca00640 1 -- 172.21.15.94:0/2686588268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad68164330 con 0x7fad680d0770 2024-09-10T23:40:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad6ca00640 1 -- 172.21.15.94:0/2686588268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad681648f0 con 0x7fad680d0770 2024-09-10T23:40:03.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad6ca00640 1 -- 172.21.15.94:0/2686588268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad68070cb0 con 0x7fad680d0770 2024-09-10T23:40:03.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad5ea00640 1 -- 172.21.15.94:0/2686588268 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fad60003860 con 0x7fad680d0770 2024-09-10T23:40:03.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.879+0000 7fad5ea00640 1 --2- 172.21.15.94:0/2686588268 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fad44072e50 0x7fad44075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:03.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.883+0000 7fad5ea00640 1 -- 172.21.15.94:0/2686588268 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5737+0+0 (secure 0 0 0) 0x7fad6012bc00 con 0x7fad680d0770 2024-09-10T23:40:03.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.883+0000 7fad65e00640 1 --2- 172.21.15.94:0/2686588268 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fad44072e50 0x7fad44075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:03.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.883+0000 7fad65e00640 1 --2- 172.21.15.94:0/2686588268 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fad44072e50 0x7fad44075310 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fad58007fd0 tx=0x7fad58009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:03.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:03.887+0000 7fad5ea00640 1 -- 172.21.15.94:0/2686588268 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fad600164b0 con 0x7fad680d0770 2024-09-10T23:40:04.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.051+0000 7fad6ca00640 1 -- 172.21.15.94:0/2686588268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad68164ac0 con 0x7fad680d0770 2024-09-10T23:40:04.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.051+0000 7fad5ea00640 1 -- 172.21.15.94:0/2686588268 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) ==== 74+0+16728 (secure 0 0 0) 0x7fad600f8d70 con 0x7fad680d0770 2024-09-10T23:40:04.056 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:04.056 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":150,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:40:02.727072+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:40:00.461748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-10T23:40:01.690823+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:40:04.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.055+0000 7fad6ca00640 1 -- 172.21.15.94:0/2686588268 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fad44072e50 msgr2=0x7fad44075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:04.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.055+0000 7fad6ca00640 1 --2- 172.21.15.94:0/2686588268 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fad44072e50 0x7fad44075310 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fad58007fd0 tx=0x7fad58009040 comp rx=0 tx=0).stop 2024-09-10T23:40:04.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.055+0000 7fad6ca00640 1 -- 172.21.15.94:0/2686588268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad680d0770 msgr2=0x7fad6815d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:04.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.055+0000 7fad6ca00640 1 --2- 172.21.15.94:0/2686588268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad680d0770 0x7fad6815d820 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fad60019440 tx=0x7fad600a5670 comp rx=0 tx=0).stop 2024-09-10T23:40:04.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.055+0000 7fad6ca00640 1 -- 172.21.15.94:0/2686588268 shutdown_connections 2024-09-10T23:40:04.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.055+0000 7fad6ca00640 1 --2- 172.21.15.94:0/2686588268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad680b3e30 0x7fad6815a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:04.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.055+0000 7fad6ca00640 1 --2- 172.21.15.94:0/2686588268 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fad44072e50 0x7fad44075310 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:04.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.055+0000 7fad6ca00640 1 --2- 172.21.15.94:0/2686588268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad680d0770 0x7fad6815d820 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:04.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.055+0000 7fad6ca00640 1 --2- 172.21.15.94:0/2686588268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad68070860 0x7fad681633a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:04.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.055+0000 7fad6ca00640 1 -- 172.21.15.94:0/2686588268 >> 172.21.15.94:0/2686588268 conn(0x7fad680d0b70 msgr2=0x7fad681540c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:04.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.055+0000 7fad6ca00640 1 -- 172.21.15.94:0/2686588268 shutdown_connections 2024-09-10T23:40:04.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.055+0000 7fad6ca00640 1 -- 172.21.15.94:0/2686588268 wait complete. 2024-09-10T23:40:04.077 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T23:40:04.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf8e00640 1 -- 172.21.15.94:0/3350564563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdaf40c64f0 msgr2=0x7fdaf40cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:04.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf8e00640 1 --2- 172.21.15.94:0/3350564563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdaf40c64f0 0x7fdaf40cf280 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009900 tx=0x7fdadc01c8e0 comp rx=0 tx=0).stop 2024-09-10T23:40:04.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf8e00640 1 -- 172.21.15.94:0/3350564563 shutdown_connections 2024-09-10T23:40:04.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf8e00640 1 --2- 172.21.15.94:0/3350564563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdaf40c64f0 0x7fdaf40cf280 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:04.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf8e00640 1 -- 172.21.15.94:0/3350564563 >> 172.21.15.94:0/3350564563 conn(0x7fdaf40cfaf0 msgr2=0x7fdaf40cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:04.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf8e00640 1 -- 172.21.15.94:0/3350564563 shutdown_connections 2024-09-10T23:40:04.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf8e00640 1 -- 172.21.15.94:0/3350564563 wait complete. 2024-09-10T23:40:04.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf8e00640 1 Processor -- start 2024-09-10T23:40:04.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf8e00640 1 -- start start 2024-09-10T23:40:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdaf406e7c0 0x7fdaf4156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf8e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf40c64f0 0x7fdaf4150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf8e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf40ce280 0x7fdaf41506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf8e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41690b0 con 0x7fdaf406e7c0 2024-09-10T23:40:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf8e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4168f30 con 0x7fdaf40c64f0 2024-09-10T23:40:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf8e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf4169230 con 0x7fdaf40ce280 2024-09-10T23:40:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdaf406e7c0 0x7fdaf4156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdaf406e7c0 0x7fdaf4156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34516/0 (socket says 172.21.15.94:34516) 2024-09-10T23:40:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf1e00640 1 -- 172.21.15.94:0/4019312972 learned_addr learned my addr 172.21.15.94:0/4019312972 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdae9400640 1 --2- 172.21.15.94:0/4019312972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf40c64f0 0x7fdaf4150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.219+0000 7fdaf2800640 1 --2- 172.21.15.94:0/4019312972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf40ce280 0x7fdaf41506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdaf1e00640 1 -- 172.21.15.94:0/4019312972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf40ce280 msgr2=0x7fdaf41506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdaf1e00640 1 --2- 172.21.15.94:0/4019312972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf40ce280 0x7fdaf41506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdaf1e00640 1 -- 172.21.15.94:0/4019312972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf40c64f0 msgr2=0x7fdaf4150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdaf1e00640 1 --2- 172.21.15.94:0/4019312972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf40c64f0 0x7fdaf4150160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdaf1e00640 1 -- 172.21.15.94:0/4019312972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf4150d50 con 0x7fdaf406e7c0 2024-09-10T23:40:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdaf2800640 1 --2- 172.21.15.94:0/4019312972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf40ce280 0x7fdaf41506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:40:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdaf1e00640 1 --2- 172.21.15.94:0/4019312972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdaf406e7c0 0x7fdaf4156900 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fdae400b410 tx=0x7fdae400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdaeaa00640 1 -- 172.21.15.94:0/4019312972 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4015030 con 0x7fdaf406e7c0 2024-09-10T23:40:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdaf8e00640 1 -- 172.21.15.94:0/4019312972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf40aeee0 con 0x7fdaf406e7c0 2024-09-10T23:40:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdaeaa00640 1 -- 172.21.15.94:0/4019312972 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdae4004020 con 0x7fdaf406e7c0 2024-09-10T23:40:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdaeaa00640 1 -- 172.21.15.94:0/4019312972 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae40044b0 con 0x7fdaf406e7c0 2024-09-10T23:40:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdaf8e00640 1 -- 172.21.15.94:0/4019312972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf4151870 con 0x7fdaf406e7c0 2024-09-10T23:40:04.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdaf8e00640 1 -- 172.21.15.94:0/4019312972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf40730f0 con 0x7fdaf406e7c0 2024-09-10T23:40:04.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdaeaa00640 1 -- 172.21.15.94:0/4019312972 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fdae4022060 con 0x7fdaf406e7c0 2024-09-10T23:40:04.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdaeaa00640 1 --2- 172.21.15.94:0/4019312972 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdad0072e50 0x7fdad0075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:04.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdaeaa00640 1 -- 172.21.15.94:0/4019312972 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5737+0+0 (secure 0 0 0) 0x7fdae4094080 con 0x7fdaf406e7c0 2024-09-10T23:40:04.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.223+0000 7fdae9400640 1 --2- 172.21.15.94:0/4019312972 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdad0072e50 0x7fdad0075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:04.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.227+0000 7fdae9400640 1 --2- 172.21.15.94:0/4019312972 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdad0072e50 0x7fdad0075310 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0af000 tx=0x7fdadc01b120 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:04.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.227+0000 7fdaeaa00640 1 -- 172.21.15.94:0/4019312972 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdae405e720 con 0x7fdaf406e7c0 2024-09-10T23:40:04.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:04.431+0000 7fdaf8e00640 1 -- 172.21.15.94:0/4019312972 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdaf40cf200 con 0x7fdaf406e7c0 2024-09-10T23:40:05.323 INFO:tasks.ceph.mds.b.smithi181.stderr:2024-09-10T23:40:05.323+0000 7f66e4e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:40:05.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.355+0000 7fdaeaa00640 1 -- 172.21.15.94:0/4019312972 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v106) ==== 118+0+0 (secure 0 0 0) 0x7fdae405e020 con 0x7fdaf406e7c0 2024-09-10T23:40:05.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.359+0000 7fdaf8e00640 1 -- 172.21.15.94:0/4019312972 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdad0072e50 msgr2=0x7fdad0075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:05.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.359+0000 7fdaf8e00640 1 --2- 172.21.15.94:0/4019312972 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdad0072e50 0x7fdad0075310 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0af000 tx=0x7fdadc01b120 comp rx=0 tx=0).stop 2024-09-10T23:40:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.359+0000 7fdaf8e00640 1 -- 172.21.15.94:0/4019312972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdaf406e7c0 msgr2=0x7fdaf4156900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.359+0000 7fdaf8e00640 1 --2- 172.21.15.94:0/4019312972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdaf406e7c0 0x7fdaf4156900 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fdae400b410 tx=0x7fdae400b8d0 comp rx=0 tx=0).stop 2024-09-10T23:40:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.359+0000 7fdaf8e00640 1 -- 172.21.15.94:0/4019312972 shutdown_connections 2024-09-10T23:40:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.359+0000 7fdaf8e00640 1 --2- 172.21.15.94:0/4019312972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf40c64f0 0x7fdaf4150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.359+0000 7fdaf8e00640 1 --2- 172.21.15.94:0/4019312972 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdad0072e50 0x7fdad0075310 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.359+0000 7fdaf8e00640 1 --2- 172.21.15.94:0/4019312972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf40ce280 0x7fdaf41506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.359+0000 7fdaf8e00640 1 --2- 172.21.15.94:0/4019312972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdaf406e7c0 0x7fdaf4156900 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.359+0000 7fdaf8e00640 1 -- 172.21.15.94:0/4019312972 >> 172.21.15.94:0/4019312972 conn(0x7fdaf40cfaf0 msgr2=0x7fdaf40cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.359+0000 7fdaf8e00640 1 -- 172.21.15.94:0/4019312972 shutdown_connections 2024-09-10T23:40:05.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.359+0000 7fdaf8e00640 1 -- 172.21.15.94:0/4019312972 wait complete. 2024-09-10T23:40:05.386 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T23:40:05.386 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T23:40:05.470 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:40:05.474 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:40:05.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.531+0000 7f42af800640 1 -- 172.21.15.94:0/688308520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a80c49b0 msgr2=0x7f42a80cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:05.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.531+0000 7f42af800640 1 --2- 172.21.15.94:0/688308520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a80c49b0 0x7f42a80cadc0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f42a80c6b50 tx=0x7f429401c8c0 comp rx=0 tx=0).stop 2024-09-10T23:40:05.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.531+0000 7f42af800640 1 -- 172.21.15.94:0/688308520 shutdown_connections 2024-09-10T23:40:05.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.531+0000 7f42af800640 1 --2- 172.21.15.94:0/688308520 >> v2:172.21.15.181:3300/0 conn(0x7f42a80cb300 0x7f42a815e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:05.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.531+0000 7f42af800640 1 --2- 172.21.15.94:0/688308520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a80c49b0 0x7f42a80cadc0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:05.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.531+0000 7f42af800640 1 --2- 172.21.15.94:0/688308520 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42a8158790 0x7f42a806fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:05.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.531+0000 7f42af800640 1 -- 172.21.15.94:0/688308520 >> 172.21.15.94:0/688308520 conn(0x7f42a80cfaf0 msgr2=0x7f42a80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:05.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.531+0000 7f42af800640 1 -- 172.21.15.94:0/688308520 shutdown_connections 2024-09-10T23:40:05.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.531+0000 7f42af800640 1 -- 172.21.15.94:0/688308520 wait complete. 2024-09-10T23:40:05.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.531+0000 7f42af800640 1 Processor -- start 2024-09-10T23:40:05.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.531+0000 7f42af800640 1 -- start start 2024-09-10T23:40:05.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42af800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42a80c49b0 0x7f42a81fc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:05.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42af800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42a80cb300 0x7f42a80728e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:05.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42af800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a8158790 0x7f42a815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:05.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42af800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f42a815b830 con 0x7f42a8158790 2024-09-10T23:40:05.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42af800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f42a815b6b0 con 0x7f42a80cb300 2024-09-10T23:40:05.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42af800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f42a815b9b0 con 0x7f42a80c49b0 2024-09-10T23:40:05.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42ad800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a8158790 0x7f42a815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:05.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42ad800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a8158790 0x7f42a815cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34530/0 (socket says 172.21.15.94:34530) 2024-09-10T23:40:05.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42ad800640 1 -- 172.21.15.94:0/2724257284 learned_addr learned my addr 172.21.15.94:0/2724257284 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:05.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42ace00640 1 --2- 172.21.15.94:0/2724257284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42a80c49b0 0x7f42a81fc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:05.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42a7e00640 1 --2- 172.21.15.94:0/2724257284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42a80cb300 0x7f42a80728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:05.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42ad800640 1 -- 172.21.15.94:0/2724257284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42a80c49b0 msgr2=0x7f42a81fc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:05.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42ad800640 1 --2- 172.21.15.94:0/2724257284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42a80c49b0 0x7f42a81fc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:05.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42ad800640 1 -- 172.21.15.94:0/2724257284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42a80cb300 msgr2=0x7f42a80728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:05.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42ad800640 1 --2- 172.21.15.94:0/2724257284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42a80cb300 0x7f42a80728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:05.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42ad800640 1 -- 172.21.15.94:0/2724257284 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a8072e20 con 0x7f42a8158790 2024-09-10T23:40:05.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42ad800640 1 --2- 172.21.15.94:0/2724257284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a8158790 0x7f42a815cfd0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f42a000dac0 tx=0x7f42a000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:05.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42a4c00640 1 -- 172.21.15.94:0/2724257284 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a000bdd0 con 0x7f42a8158790 2024-09-10T23:40:05.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42a4c00640 1 -- 172.21.15.94:0/2724257284 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42a0003980 con 0x7f42a8158790 2024-09-10T23:40:05.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42a4c00640 1 -- 172.21.15.94:0/2724257284 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a0003dd0 con 0x7f42a8158790 2024-09-10T23:40:05.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42af800640 1 -- 172.21.15.94:0/2724257284 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a815d800 con 0x7f42a8158790 2024-09-10T23:40:05.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42af800640 1 -- 172.21.15.94:0/2724257284 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a8153d10 con 0x7f42a8158790 2024-09-10T23:40:05.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.535+0000 7f42af800640 1 -- 172.21.15.94:0/2724257284 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a815bf10 con 0x7f42a8158790 2024-09-10T23:40:05.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.539+0000 7f42a4c00640 1 -- 172.21.15.94:0/2724257284 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f42a0003f70 con 0x7f42a8158790 2024-09-10T23:40:05.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.539+0000 7f42a4c00640 1 --2- 172.21.15.94:0/2724257284 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4288072e80 0x7f4288075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:05.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.539+0000 7f42a4c00640 1 -- 172.21.15.94:0/2724257284 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f42a00921d0 con 0x7f42a8158790 2024-09-10T23:40:05.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.539+0000 7f42ace00640 1 --2- 172.21.15.94:0/2724257284 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4288072e80 0x7f4288075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:05.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.539+0000 7f42ace00640 1 --2- 172.21.15.94:0/2724257284 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4288072e80 0x7f4288075340 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f429c00bb50 tx=0x7f429c00b2e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:05.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.543+0000 7f42a4c00640 1 -- 172.21.15.94:0/2724257284 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f42a0020030 con 0x7f42a8158790 2024-09-10T23:40:05.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:05.743+0000 7f42af800640 1 -- 172.21.15.94:0/2724257284 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f42a80d0850 con 0x7f42a8158790 2024-09-10T23:40:06.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.359+0000 7f42a4c00640 1 -- 172.21.15.94:0/2724257284 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v107) ==== 122+0+0 (secure 0 0 0) 0x7f42a0017030 con 0x7f42a8158790 2024-09-10T23:40:06.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.363+0000 7f42af800640 1 -- 172.21.15.94:0/2724257284 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4288072e80 msgr2=0x7f4288075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:06.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.363+0000 7f42af800640 1 --2- 172.21.15.94:0/2724257284 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4288072e80 0x7f4288075340 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f429c00bb50 tx=0x7f429c00b2e0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.363+0000 7f42af800640 1 -- 172.21.15.94:0/2724257284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a8158790 msgr2=0x7f42a815cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:06.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.363+0000 7f42af800640 1 --2- 172.21.15.94:0/2724257284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a8158790 0x7f42a815cfd0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f42a000dac0 tx=0x7f42a000df80 comp rx=0 tx=0).stop 2024-09-10T23:40:06.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.363+0000 7f42af800640 1 -- 172.21.15.94:0/2724257284 shutdown_connections 2024-09-10T23:40:06.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.363+0000 7f42af800640 1 --2- 172.21.15.94:0/2724257284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42a80cb300 0x7f42a80728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.363+0000 7f42af800640 1 --2- 172.21.15.94:0/2724257284 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f4288072e80 0x7f4288075340 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.363+0000 7f42af800640 1 --2- 172.21.15.94:0/2724257284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42a8158790 0x7f42a815cfd0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.363+0000 7f42af800640 1 --2- 172.21.15.94:0/2724257284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42a80c49b0 0x7f42a81fc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.363+0000 7f42af800640 1 -- 172.21.15.94:0/2724257284 >> 172.21.15.94:0/2724257284 conn(0x7f42a80cfaf0 msgr2=0x7f42a8156540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:06.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.363+0000 7f42af800640 1 -- 172.21.15.94:0/2724257284 shutdown_connections 2024-09-10T23:40:06.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.363+0000 7f42af800640 1 -- 172.21.15.94:0/2724257284 wait complete. 2024-09-10T23:40:06.388 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T23:40:06.388 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:40:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.571+0000 7f0936c00640 1 -- 172.21.15.94:0/1929699325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09300cddb0 msgr2=0x7f09300ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:06.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.571+0000 7f0936c00640 1 --2- 172.21.15.94:0/1929699325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09300cddb0 0x7f09300ce220 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0928009900 tx=0x7f092801c8b0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.571+0000 7f0936c00640 1 -- 172.21.15.94:0/1929699325 shutdown_connections 2024-09-10T23:40:06.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.571+0000 7f0936c00640 1 --2- 172.21.15.94:0/1929699325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09300cddb0 0x7f09300ce220 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.571+0000 7f0936c00640 1 --2- 172.21.15.94:0/1929699325 >> v2:172.21.15.181:3300/0 conn(0x7f09300cfcb0 0x7f09300d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.571+0000 7f0936c00640 1 -- 172.21.15.94:0/1929699325 >> 172.21.15.94:0/1929699325 conn(0x7f093006cdf0 msgr2=0x7f093006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:06.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.571+0000 7f0936c00640 1 -- 172.21.15.94:0/1929699325 shutdown_connections 2024-09-10T23:40:06.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.571+0000 7f0936c00640 1 -- 172.21.15.94:0/1929699325 wait complete. 2024-09-10T23:40:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.571+0000 7f0936c00640 1 Processor -- start 2024-09-10T23:40:06.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.571+0000 7f0936c00640 1 -- start start 2024-09-10T23:40:06.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.571+0000 7f0936c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09300cddb0 0x7f0930152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:06.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.571+0000 7f0936c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09300cfcb0 0x7f09301492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:06.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.571+0000 7f0936c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f093015ba50 0x7f0930149830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:06.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.571+0000 7f0936c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0930168be0 con 0x7f093015ba50 2024-09-10T23:40:06.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.571+0000 7f0936c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0930168a60 con 0x7f09300cfcb0 2024-09-10T23:40:06.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.571+0000 7f0936c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0930168d60 con 0x7f09300cddb0 2024-09-10T23:40:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f0934c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f093015ba50 0x7f0930149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f0934c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f093015ba50 0x7f0930149830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34540/0 (socket says 172.21.15.94:34540) 2024-09-10T23:40:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f0934c00640 1 -- 172.21.15.94:0/3066899846 learned_addr learned my addr 172.21.15.94:0/3066899846 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f092fe00640 1 --2- 172.21.15.94:0/3066899846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09300cddb0 0x7f0930152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f092f400640 1 --2- 172.21.15.94:0/3066899846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09300cfcb0 0x7f09301492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f0934c00640 1 -- 172.21.15.94:0/3066899846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09300cddb0 msgr2=0x7f0930152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f0934c00640 1 --2- 172.21.15.94:0/3066899846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09300cddb0 0x7f0930152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f0934c00640 1 -- 172.21.15.94:0/3066899846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09300cfcb0 msgr2=0x7f09301492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f0934c00640 1 --2- 172.21.15.94:0/3066899846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09300cfcb0 0x7f09301492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f0934c00640 1 -- 172.21.15.94:0/3066899846 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09301f7c00 con 0x7f093015ba50 2024-09-10T23:40:06.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f0934c00640 1 --2- 172.21.15.94:0/3066899846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f093015ba50 0x7f0930149830 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f09280b0040 tx=0x7f0928005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:06.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f0927e00640 1 -- 172.21.15.94:0/3066899846 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09280b2040 con 0x7f093015ba50 2024-09-10T23:40:06.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f0927e00640 1 -- 172.21.15.94:0/3066899846 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0928005320 con 0x7f093015ba50 2024-09-10T23:40:06.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f0936c00640 1 -- 172.21.15.94:0/3066899846 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09301f7ee0 con 0x7f093015ba50 2024-09-10T23:40:06.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f0927e00640 1 -- 172.21.15.94:0/3066899846 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09280046d0 con 0x7f093015ba50 2024-09-10T23:40:06.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f0936c00640 1 -- 172.21.15.94:0/3066899846 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09301f84a0 con 0x7f093015ba50 2024-09-10T23:40:06.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f0936c00640 1 -- 172.21.15.94:0/3066899846 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0930153fb0 con 0x7f093015ba50 2024-09-10T23:40:06.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.575+0000 7f0927e00640 1 -- 172.21.15.94:0/3066899846 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f0928004890 con 0x7f093015ba50 2024-09-10T23:40:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.579+0000 7f0927e00640 1 --2- 172.21.15.94:0/3066899846 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0910072e50 0x7f0910075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.579+0000 7f0927e00640 1 -- 172.21.15.94:0/3066899846 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f092812e840 con 0x7f093015ba50 2024-09-10T23:40:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.579+0000 7f092fe00640 1 --2- 172.21.15.94:0/3066899846 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0910072e50 0x7f0910075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:06.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.579+0000 7f092fe00640 1 --2- 172.21.15.94:0/3066899846 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0910072e50 0x7f0910075310 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0920001cf0 tx=0x7f0920009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:06.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.583+0000 7f0927e00640 1 -- 172.21.15.94:0/3066899846 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0930153fb0 con 0x7f093015ba50 2024-09-10T23:40:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.783+0000 7f0936c00640 1 -- 172.21.15.94:0/3066899846 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09300ce220 con 0x7f093015ba50 2024-09-10T23:40:06.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.783+0000 7f0927e00640 1 -- 172.21.15.94:0/3066899846 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5249 (secure 0 0 0) 0x7f09280164d0 con 0x7f093015ba50 2024-09-10T23:40:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:06.788 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":107,"btime":"2024-09-10T23:40:06:362943+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8158,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1859150794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":1859150794},{"type":"v1","addr":"172.21.15.94:6835","nonce":1859150794}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8173,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/3891720008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3891720008},{"type":"v1","addr":"172.21.15.181:6801","nonce":3891720008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8787,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/4040262257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":4040262257},{"type":"v1","addr":"172.21.15.94:6834","nonce":4040262257}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:40:02.727036+0000","modified":"2024-09-10T23:40:05.749707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":197120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_197120":{"gid":197120,"name":"b","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/2758853868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2758853868},{"type":"v1","addr":"172.21.15.181:6835","nonce":2758853868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":197120,"qdb_cluster":[197120]},"id":12}]} 2024-09-10T23:40:06.788 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 107 2024-09-10T23:40:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.787+0000 7f0936c00640 1 -- 172.21.15.94:0/3066899846 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0910072e50 msgr2=0x7f0910075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.787+0000 7f0936c00640 1 --2- 172.21.15.94:0/3066899846 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0910072e50 0x7f0910075310 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0920001cf0 tx=0x7f0920009040 comp rx=0 tx=0).stop 2024-09-10T23:40:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.787+0000 7f0936c00640 1 -- 172.21.15.94:0/3066899846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f093015ba50 msgr2=0x7f0930149830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.787+0000 7f0936c00640 1 --2- 172.21.15.94:0/3066899846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f093015ba50 0x7f0930149830 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f09280b0040 tx=0x7f0928005de0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.787+0000 7f0936c00640 1 -- 172.21.15.94:0/3066899846 shutdown_connections 2024-09-10T23:40:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.787+0000 7f0936c00640 1 --2- 172.21.15.94:0/3066899846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09300cfcb0 0x7f09301492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.787+0000 7f0936c00640 1 --2- 172.21.15.94:0/3066899846 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0910072e50 0x7f0910075310 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.787+0000 7f0936c00640 1 --2- 172.21.15.94:0/3066899846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f093015ba50 0x7f0930149830 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.787+0000 7f0936c00640 1 --2- 172.21.15.94:0/3066899846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09300cddb0 0x7f0930152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.787+0000 7f0936c00640 1 -- 172.21.15.94:0/3066899846 >> 172.21.15.94:0/3066899846 conn(0x7f093006cdf0 msgr2=0x7f093014a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.787+0000 7f0936c00640 1 -- 172.21.15.94:0/3066899846 shutdown_connections 2024-09-10T23:40:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.787+0000 7f0936c00640 1 -- 172.21.15.94:0/3066899846 wait complete. 2024-09-10T23:40:06.810 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:40:06.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e1ca00640 1 -- 172.21.15.94:0/3984830527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dfc0053a0 msgr2=0x7f9dfc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:06.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e1ca00640 1 --2- 172.21.15.94:0/3984830527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dfc0053a0 0x7f9dfc00b9d0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000c610 tx=0x7f9e1001d280 comp rx=0 tx=0).stop 2024-09-10T23:40:06.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e1ca00640 1 -- 172.21.15.94:0/3984830527 shutdown_connections 2024-09-10T23:40:06.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e1ca00640 1 --2- 172.21.15.94:0/3984830527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dfc0053a0 0x7f9dfc00b9d0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e1ca00640 1 -- 172.21.15.94:0/3984830527 >> 172.21.15.94:0/3984830527 conn(0x7f9e180d1c60 msgr2=0x7f9e180d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:06.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e1ca00640 1 -- 172.21.15.94:0/3984830527 shutdown_connections 2024-09-10T23:40:06.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e1ca00640 1 -- 172.21.15.94:0/3984830527 wait complete. 2024-09-10T23:40:06.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e1ca00640 1 Processor -- start 2024-09-10T23:40:06.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e1ca00640 1 -- start start 2024-09-10T23:40:06.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e1ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dfc0053a0 0x7f9e18140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:06.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e1ca00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e180cb070 0x7f9e180cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:06.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e1ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e1814e160 0x7f9e180cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:06.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e1ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9e180cfce0 con 0x7f9e1814e160 2024-09-10T23:40:06.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e1ca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9e180cfb60 con 0x7f9dfc0053a0 2024-09-10T23:40:06.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e1ca00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9e180cfe60 con 0x7f9e180cb070 2024-09-10T23:40:06.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e16800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e1814e160 0x7f9e180cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:06.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e16800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e1814e160 0x7f9e180cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34564/0 (socket says 172.21.15.94:34564) 2024-09-10T23:40:06.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e16800640 1 -- 172.21.15.94:0/4251208218 learned_addr learned my addr 172.21.15.94:0/4251208218 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:06.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e15400640 1 --2- 172.21.15.94:0/4251208218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e180cb070 0x7f9e180cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:06.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e15e00640 1 --2- 172.21.15.94:0/4251208218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dfc0053a0 0x7f9e18140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:06.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e16800640 1 -- 172.21.15.94:0/4251208218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e180cb070 msgr2=0x7f9e180cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:06.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e16800640 1 --2- 172.21.15.94:0/4251208218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e180cb070 0x7f9e180cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e16800640 1 -- 172.21.15.94:0/4251208218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dfc0053a0 msgr2=0x7f9e18140ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:06.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e16800640 1 --2- 172.21.15.94:0/4251208218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dfc0053a0 0x7f9e18140ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:06.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e16800640 1 -- 172.21.15.94:0/4251208218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e180b7410 con 0x7f9e1814e160 2024-09-10T23:40:06.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.951+0000 7f9e16800640 1 --2- 172.21.15.94:0/4251208218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e1814e160 0x7f9e180cdae0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f9e10000f90 tx=0x7f9e10004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:06.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.955+0000 7f9e0e000640 1 -- 172.21.15.94:0/4251208218 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e10001a80 con 0x7f9e1814e160 2024-09-10T23:40:06.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.955+0000 7f9e0e000640 1 -- 172.21.15.94:0/4251208218 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e10001450 con 0x7f9e1814e160 2024-09-10T23:40:06.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.955+0000 7f9e0e000640 1 -- 172.21.15.94:0/4251208218 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e10003be0 con 0x7f9e1814e160 2024-09-10T23:40:06.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.955+0000 7f9e1ca00640 1 -- 172.21.15.94:0/4251208218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e18141420 con 0x7f9e1814e160 2024-09-10T23:40:06.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.955+0000 7f9e1ca00640 1 -- 172.21.15.94:0/4251208218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e180ce930 con 0x7f9e1814e160 2024-09-10T23:40:06.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.955+0000 7f9e1ca00640 1 -- 172.21.15.94:0/4251208218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e180d03c0 con 0x7f9e1814e160 2024-09-10T23:40:06.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.955+0000 7f9e0e000640 1 -- 172.21.15.94:0/4251208218 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f9e100060c0 con 0x7f9e1814e160 2024-09-10T23:40:06.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.955+0000 7f9e0e000640 1 --2- 172.21.15.94:0/4251208218 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9df8072e50 0x7f9df8075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:06.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.959+0000 7f9e0e000640 1 -- 172.21.15.94:0/4251208218 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f9e1012d5a0 con 0x7f9e1814e160 2024-09-10T23:40:06.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.959+0000 7f9e15e00640 1 --2- 172.21.15.94:0/4251208218 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9df8072e50 0x7f9df8075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:06.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.959+0000 7f9e15e00640 1 --2- 172.21.15.94:0/4251208218 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9df8072e50 0x7f9df8075310 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9e180cbf40 tx=0x7f9e080091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:06.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:06.963+0000 7f9e0e000640 1 -- 172.21.15.94:0/4251208218 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9e100bf070 con 0x7f9e1814e160 2024-09-10T23:40:07.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.127+0000 7f9e1ca00640 1 -- 172.21.15.94:0/4251208218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e180c8dc0 con 0x7f9e1814e160 2024-09-10T23:40:07.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.131+0000 7f9e0e000640 1 -- 172.21.15.94:0/4251208218 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) ==== 74+0+16715 (secure 0 0 0) 0x7f9e10017030 con 0x7f9e1814e160 2024-09-10T23:40:07.134 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:07.134 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":151,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:40:04.337069+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:40:00.461748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-10T23:40:01.690823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:40:07.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.131+0000 7f9e1ca00640 1 -- 172.21.15.94:0/4251208218 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9df8072e50 msgr2=0x7f9df8075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:07.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.131+0000 7f9e1ca00640 1 --2- 172.21.15.94:0/4251208218 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9df8072e50 0x7f9df8075310 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9e180cbf40 tx=0x7f9e080091e0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.131+0000 7f9e1ca00640 1 -- 172.21.15.94:0/4251208218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e1814e160 msgr2=0x7f9e180cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:07.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.131+0000 7f9e1ca00640 1 --2- 172.21.15.94:0/4251208218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e1814e160 0x7f9e180cdae0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f9e10000f90 tx=0x7f9e10004f70 comp rx=0 tx=0).stop 2024-09-10T23:40:07.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.131+0000 7f9e1ca00640 1 -- 172.21.15.94:0/4251208218 shutdown_connections 2024-09-10T23:40:07.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.131+0000 7f9e1ca00640 1 --2- 172.21.15.94:0/4251208218 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9df8072e50 0x7f9df8075310 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.131+0000 7f9e1ca00640 1 --2- 172.21.15.94:0/4251208218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e1814e160 0x7f9e180cdae0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.131+0000 7f9e1ca00640 1 --2- 172.21.15.94:0/4251208218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e180cb070 0x7f9e180cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.131+0000 7f9e1ca00640 1 --2- 172.21.15.94:0/4251208218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9dfc0053a0 0x7f9e18140ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.131+0000 7f9e1ca00640 1 -- 172.21.15.94:0/4251208218 >> 172.21.15.94:0/4251208218 conn(0x7f9e180d1c60 msgr2=0x7f9e180b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:07.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.131+0000 7f9e1ca00640 1 -- 172.21.15.94:0/4251208218 shutdown_connections 2024-09-10T23:40:07.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.131+0000 7f9e1ca00640 1 -- 172.21.15.94:0/4251208218 wait complete. 2024-09-10T23:40:07.157 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T23:40:07.157 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:40:07.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.299+0000 7fecfae00640 1 -- 172.21.15.94:0/1983031931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecf40cfcb0 msgr2=0x7fecf40d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:07.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.299+0000 7fecfae00640 1 --2- 172.21.15.94:0/1983031931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecf40cfcb0 0x7fecf40d00d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fece0009840 tx=0x7fece001c610 comp rx=0 tx=0).stop 2024-09-10T23:40:07.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.299+0000 7fecfae00640 1 -- 172.21.15.94:0/1983031931 shutdown_connections 2024-09-10T23:40:07.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.299+0000 7fecfae00640 1 --2- 172.21.15.94:0/1983031931 >> v2:172.21.15.181:3300/0 conn(0x7fecf40cddb0 0x7fecf40ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.299+0000 7fecfae00640 1 --2- 172.21.15.94:0/1983031931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecf40cfcb0 0x7fecf40d00d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.299+0000 7fecfae00640 1 -- 172.21.15.94:0/1983031931 >> 172.21.15.94:0/1983031931 conn(0x7fecf406cdf0 msgr2=0x7fecf406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:07.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.299+0000 7fecfae00640 1 -- 172.21.15.94:0/1983031931 shutdown_connections 2024-09-10T23:40:07.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.299+0000 7fecfae00640 1 -- 172.21.15.94:0/1983031931 wait complete. 2024-09-10T23:40:07.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.299+0000 7fecfae00640 1 Processor -- start 2024-09-10T23:40:07.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.299+0000 7fecfae00640 1 -- start start 2024-09-10T23:40:07.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.299+0000 7fecfae00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecf40cddb0 0x7fecf4148140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:07.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.299+0000 7fecfae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecf40cfcb0 0x7fecf406f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:07.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.299+0000 7fecfae00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecf415ba50 0x7fecf414ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:07.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.299+0000 7fecfae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fecf4168be0 con 0x7fecf40cfcb0 2024-09-10T23:40:07.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.299+0000 7fecfae00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fecf4168a60 con 0x7fecf415ba50 2024-09-10T23:40:07.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.299+0000 7fecfae00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fecf4168d60 con 0x7fecf40cddb0 2024-09-10T23:40:07.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecf3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecf40cfcb0 0x7fecf406f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:07.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecf3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecf40cfcb0 0x7fecf406f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34574/0 (socket says 172.21.15.94:34574) 2024-09-10T23:40:07.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecf3400640 1 -- 172.21.15.94:0/1792365781 learned_addr learned my addr 172.21.15.94:0/1792365781 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:07.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecf3e00640 1 --2- 172.21.15.94:0/1792365781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecf40cddb0 0x7fecf4148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:07.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecf8e00640 1 --2- 172.21.15.94:0/1792365781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecf415ba50 0x7fecf414ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:07.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecf3400640 1 -- 172.21.15.94:0/1792365781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecf40cddb0 msgr2=0x7fecf4148140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:07.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecf3400640 1 --2- 172.21.15.94:0/1792365781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecf40cddb0 0x7fecf4148140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecf3400640 1 -- 172.21.15.94:0/1792365781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecf415ba50 msgr2=0x7fecf414ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:07.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecf3400640 1 --2- 172.21.15.94:0/1792365781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecf415ba50 0x7fecf414ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecf3400640 1 -- 172.21.15.94:0/1792365781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf41fb0c0 con 0x7fecf40cfcb0 2024-09-10T23:40:07.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecf3400640 1 --2- 172.21.15.94:0/1792365781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecf40cfcb0 0x7fecf406f610 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fece0009790 tx=0x7fece00a65f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:07.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecebe00640 1 -- 172.21.15.94:0/1792365781 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece00bd020 con 0x7fecf40cfcb0 2024-09-10T23:40:07.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecebe00640 1 -- 172.21.15.94:0/1792365781 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fece0004320 con 0x7fecf40cfcb0 2024-09-10T23:40:07.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecebe00640 1 -- 172.21.15.94:0/1792365781 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece0005280 con 0x7fecf40cfcb0 2024-09-10T23:40:07.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecfae00640 1 -- 172.21.15.94:0/1792365781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf41fb3a0 con 0x7fecf40cfcb0 2024-09-10T23:40:07.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecfae00640 1 -- 172.21.15.94:0/1792365781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf41fb960 con 0x7fecf40cfcb0 2024-09-10T23:40:07.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecfae00640 1 -- 172.21.15.94:0/1792365781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf40ce260 con 0x7fecf40cfcb0 2024-09-10T23:40:07.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.303+0000 7fecebe00640 1 -- 172.21.15.94:0/1792365781 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fece00046d0 con 0x7fecf40cfcb0 2024-09-10T23:40:07.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.307+0000 7fecebe00640 1 --2- 172.21.15.94:0/1792365781 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fecd4072e50 0x7fecd4075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:07.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.307+0000 7fecebe00640 1 -- 172.21.15.94:0/1792365781 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7fece012e5b0 con 0x7fecf40cfcb0 2024-09-10T23:40:07.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.307+0000 7fecf3e00640 1 --2- 172.21.15.94:0/1792365781 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fecd4072e50 0x7fecd4075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:07.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.307+0000 7fecf3e00640 1 --2- 172.21.15.94:0/1792365781 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fecd4072e50 0x7fecd4075310 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fece40017b0 tx=0x7fece4001740 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:07.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.311+0000 7fecebe00640 1 -- 172.21.15.94:0/1792365781 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fecf40cefb0 con 0x7fecf40cfcb0 2024-09-10T23:40:07.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.475+0000 7fecfae00640 1 -- 172.21.15.94:0/1792365781 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fecf4076000 con 0x7fecd4072e50 2024-09-10T23:40:07.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.479+0000 7fecebe00640 1 -- 172.21.15.94:0/1792365781 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247114 (secure 0 0 0) 0x7fecf4076000 con 0x7fecd4072e50 2024-09-10T23:40:07.487 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:07.488 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:40:07.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.483+0000 7fecfae00640 1 -- 172.21.15.94:0/1792365781 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fecd4072e50 msgr2=0x7fecd4075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:07.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.483+0000 7fecfae00640 1 --2- 172.21.15.94:0/1792365781 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fecd4072e50 0x7fecd4075310 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fece40017b0 tx=0x7fece4001740 comp rx=0 tx=0).stop 2024-09-10T23:40:07.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.483+0000 7fecfae00640 1 -- 172.21.15.94:0/1792365781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecf40cfcb0 msgr2=0x7fecf406f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:07.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.483+0000 7fecfae00640 1 --2- 172.21.15.94:0/1792365781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecf40cfcb0 0x7fecf406f610 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fece0009790 tx=0x7fece00a65f0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.483+0000 7fecfae00640 1 -- 172.21.15.94:0/1792365781 shutdown_connections 2024-09-10T23:40:07.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.483+0000 7fecfae00640 1 --2- 172.21.15.94:0/1792365781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecf415ba50 0x7fecf414ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.483+0000 7fecfae00640 1 --2- 172.21.15.94:0/1792365781 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fecd4072e50 0x7fecd4075310 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.483+0000 7fecfae00640 1 --2- 172.21.15.94:0/1792365781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecf40cfcb0 0x7fecf406f610 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.483+0000 7fecfae00640 1 --2- 172.21.15.94:0/1792365781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecf40cddb0 0x7fecf4148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.483+0000 7fecfae00640 1 -- 172.21.15.94:0/1792365781 >> 172.21.15.94:0/1792365781 conn(0x7fecf406cdf0 msgr2=0x7fecf415e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:07.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.483+0000 7fecfae00640 1 -- 172.21.15.94:0/1792365781 shutdown_connections 2024-09-10T23:40:07.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.483+0000 7fecfae00640 1 -- 172.21.15.94:0/1792365781 wait complete. 2024-09-10T23:40:07.508 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-10T23:40:06.778391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":277720,"kb_used_data":15724,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754697000,"statfs":{"total":773094113280,"available":772809728000,"internally_reserved":0,"allocated":16101376,"data_stored":7735437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.453634"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342043+0000","last_change":"2024-09-10T23:40:02.342043+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342043+0000","last_undegraded":"2024-09-10T23:40:02.342043+0000","last_fullsized":"2024-09-10T23:40:02.342043+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344703+0000","last_change":"2024-09-10T23:40:02.344703+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344703+0000","last_undegraded":"2024-09-10T23:40:02.344703+0000","last_fullsized":"2024-09-10T23:40:02.344703+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.353593+0000","last_change":"2024-09-10T23:40:02.353593+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.353593+0000","last_undegraded":"2024-09-10T23:40:02.353593+0000","last_fullsized":"2024-09-10T23:40:02.353593+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.343368+0000","last_change":"2024-09-10T23:40:02.343368+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.343368+0000","last_undegraded":"2024-09-10T23:40:02.343368+0000","last_fullsized":"2024-09-10T23:40:02.343368+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.351849+0000","last_change":"2024-09-10T23:40:02.351849+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.351849+0000","last_undegraded":"2024-09-10T23:40:02.351849+0000","last_fullsized":"2024-09-10T23:40:02.351849+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.345840+0000","last_change":"2024-09-10T23:40:02.345840+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.345840+0000","last_undegraded":"2024-09-10T23:40:02.345840+0000","last_fullsized":"2024-09-10T23:40:02.345840+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.339971+0000","last_change":"2024-09-10T23:40:02.339971+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.339971+0000","last_undegraded":"2024-09-10T23:40:02.339971+0000","last_fullsized":"2024-09-10T23:40:02.339971+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350386+0000","last_change":"2024-09-10T23:40:02.350386+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350386+0000","last_undegraded":"2024-09-10T23:40:02.350386+0000","last_fullsized":"2024-09-10T23:40:02.350386+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348677+0000","last_change":"2024-09-10T23:40:02.348677+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348677+0000","last_undegraded":"2024-09-10T23:40:02.348677+0000","last_fullsized":"2024-09-10T23:40:02.348677+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348260+0000","last_change":"2024-09-10T23:40:02.348260+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348260+0000","last_undegraded":"2024-09-10T23:40:02.348260+0000","last_fullsized":"2024-09-10T23:40:02.348260+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342470+0000","last_change":"2024-09-10T23:40:02.342470+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342470+0000","last_undegraded":"2024-09-10T23:40:02.342470+0000","last_fullsized":"2024-09-10T23:40:02.342470+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344421+0000","last_change":"2024-09-10T23:40:02.344421+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344421+0000","last_undegraded":"2024-09-10T23:40:02.344421+0000","last_fullsized":"2024-09-10T23:40:02.344421+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.341773+0000","last_change":"2024-09-10T23:40:02.341773+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.341773+0000","last_undegraded":"2024-09-10T23:40:02.341773+0000","last_fullsized":"2024-09-10T23:40:02.341773+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.343249+0000","last_change":"2024-09-10T23:40:02.343249+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.343249+0000","last_undegraded":"2024-09-10T23:40:02.343249+0000","last_fullsized":"2024-09-10T23:40:02.343249+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.343783+0000","last_change":"2024-09-10T23:40:02.343783+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.343783+0000","last_undegraded":"2024-09-10T23:40:02.343783+0000","last_fullsized":"2024-09-10T23:40:02.343783+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350869+0000","last_change":"2024-09-10T23:40:02.350869+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350869+0000","last_undegraded":"2024-09-10T23:40:02.350869+0000","last_fullsized":"2024-09-10T23:40:02.350869+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.347466+0000","last_change":"2024-09-10T23:40:02.347466+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.347466+0000","last_undegraded":"2024-09-10T23:40:02.347466+0000","last_fullsized":"2024-09-10T23:40:02.347466+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.340509+0000","last_change":"2024-09-10T23:40:02.340509+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.340509+0000","last_undegraded":"2024-09-10T23:40:02.340509+0000","last_fullsized":"2024-09-10T23:40:02.340509+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.353158+0000","last_change":"2024-09-10T23:40:02.353158+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.353158+0000","last_undegraded":"2024-09-10T23:40:02.353158+0000","last_fullsized":"2024-09-10T23:40:02.353158+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348678+0000","last_change":"2024-09-10T23:40:02.348678+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348678+0000","last_undegraded":"2024-09-10T23:40:02.348678+0000","last_fullsized":"2024-09-10T23:40:02.348678+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.351641+0000","last_change":"2024-09-10T23:40:02.351641+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.351641+0000","last_undegraded":"2024-09-10T23:40:02.351641+0000","last_fullsized":"2024-09-10T23:40:02.351641+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350410+0000","last_change":"2024-09-10T23:40:02.350410+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350410+0000","last_undegraded":"2024-09-10T23:40:02.350410+0000","last_fullsized":"2024-09-10T23:40:02.350410+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.345990+0000","last_change":"2024-09-10T23:40:02.345990+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.345990+0000","last_undegraded":"2024-09-10T23:40:02.345990+0000","last_fullsized":"2024-09-10T23:40:02.345990+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.349962+0000","last_change":"2024-09-10T23:40:02.349962+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.349962+0000","last_undegraded":"2024-09-10T23:40:02.349962+0000","last_fullsized":"2024-09-10T23:40:02.349962+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348253+0000","last_change":"2024-09-10T23:40:02.348253+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348253+0000","last_undegraded":"2024-09-10T23:40:02.348253+0000","last_fullsized":"2024-09-10T23:40:02.348253+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348873+0000","last_change":"2024-09-10T23:40:02.348873+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348873+0000","last_undegraded":"2024-09-10T23:40:02.348873+0000","last_fullsized":"2024-09-10T23:40:02.348873+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.352668+0000","last_change":"2024-09-10T23:40:02.352668+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.352668+0000","last_undegraded":"2024-09-10T23:40:02.352668+0000","last_fullsized":"2024-09-10T23:40:02.352668+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344673+0000","last_change":"2024-09-10T23:40:02.344673+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344673+0000","last_undegraded":"2024-09-10T23:40:02.344673+0000","last_fullsized":"2024-09-10T23:40:02.344673+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.343596+0000","last_change":"2024-09-10T23:40:02.343596+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.343596+0000","last_undegraded":"2024-09-10T23:40:02.343596+0000","last_fullsized":"2024-09-10T23:40:02.343596+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.346547+0000","last_change":"2024-09-10T23:40:02.346547+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.346547+0000","last_undegraded":"2024-09-10T23:40:02.346547+0000","last_fullsized":"2024-09-10T23:40:02.346547+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350824+0000","last_change":"2024-09-10T23:40:02.350824+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350824+0000","last_undegraded":"2024-09-10T23:40:02.350824+0000","last_fullsized":"2024-09-10T23:40:02.350824+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348261+0000","last_change":"2024-09-10T23:40:02.348261+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348261+0000","last_undegraded":"2024-09-10T23:40:02.348261+0000","last_fullsized":"2024-09-10T23:40:02.348261+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.349611+0000","last_change":"2024-09-10T23:40:02.349611+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.349611+0000","last_undegraded":"2024-09-10T23:40:02.349611+0000","last_fullsized":"2024-09-10T23:40:02.349611+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359443+0000","last_change":"2024-09-10T23:40:02.357339+0000","last_active":"2024-09-10T23:40:02.359443+0000","last_peered":"2024-09-10T23:40:02.359443+0000","last_clean":"2024-09-10T23:40:02.359443+0000","last_became_active":"2024-09-10T23:40:02.357054+0000","last_became_peered":"2024-09-10T23:40:02.357054+0000","last_unstale":"2024-09-10T23:40:02.359443+0000","last_undegraded":"2024-09-10T23:40:02.359443+0000","last_fullsized":"2024-09-10T23:40:02.359443+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:21.460711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.345207+0000","last_change":"2024-09-10T23:40:02.345207+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.345207+0000","last_undegraded":"2024-09-10T23:40:02.345207+0000","last_fullsized":"2024-09-10T23:40:02.345207+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.358117+0000","last_change":"2024-09-10T23:40:02.354357+0000","last_active":"2024-09-10T23:40:02.358117+0000","last_peered":"2024-09-10T23:40:02.358117+0000","last_clean":"2024-09-10T23:40:02.358117+0000","last_became_active":"2024-09-10T23:40:02.354105+0000","last_became_peered":"2024-09-10T23:40:02.354105+0000","last_unstale":"2024-09-10T23:40:02.358117+0000","last_undegraded":"2024-09-10T23:40:02.358117+0000","last_fullsized":"2024-09-10T23:40:02.358117+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:34.016734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342932+0000","last_change":"2024-09-10T23:40:02.342932+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342932+0000","last_undegraded":"2024-09-10T23:40:02.342932+0000","last_fullsized":"2024-09-10T23:40:02.342932+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.369058+0000","last_change":"2024-09-10T23:40:02.363918+0000","last_active":"2024-09-10T23:40:02.369058+0000","last_peered":"2024-09-10T23:40:02.369058+0000","last_clean":"2024-09-10T23:40:02.369058+0000","last_became_active":"2024-09-10T23:40:02.363627+0000","last_became_peered":"2024-09-10T23:40:02.363627+0000","last_unstale":"2024-09-10T23:40:02.369058+0000","last_undegraded":"2024-09-10T23:40:02.369058+0000","last_fullsized":"2024-09-10T23:40:02.369058+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:14.649948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342179+0000","last_change":"2024-09-10T23:40:02.342179+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342179+0000","last_undegraded":"2024-09-10T23:40:02.342179+0000","last_fullsized":"2024-09-10T23:40:02.342179+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359942+0000","last_change":"2024-09-10T23:40:02.354874+0000","last_active":"2024-09-10T23:40:02.359942+0000","last_peered":"2024-09-10T23:40:02.359942+0000","last_clean":"2024-09-10T23:40:02.359942+0000","last_became_active":"2024-09-10T23:40:02.354568+0000","last_became_peered":"2024-09-10T23:40:02.354568+0000","last_unstale":"2024-09-10T23:40:02.359942+0000","last_undegraded":"2024-09-10T23:40:02.359942+0000","last_fullsized":"2024-09-10T23:40:02.359942+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:19.346732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.345401+0000","last_change":"2024-09-10T23:40:02.345401+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.345401+0000","last_undegraded":"2024-09-10T23:40:02.345401+0000","last_fullsized":"2024-09-10T23:40:02.345401+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.369026+0000","last_change":"2024-09-10T23:40:02.363569+0000","last_active":"2024-09-10T23:40:02.369026+0000","last_peered":"2024-09-10T23:40:02.369026+0000","last_clean":"2024-09-10T23:40:02.369026+0000","last_became_active":"2024-09-10T23:40:02.362386+0000","last_became_peered":"2024-09-10T23:40:02.362386+0000","last_unstale":"2024-09-10T23:40:02.369026+0000","last_undegraded":"2024-09-10T23:40:02.369026+0000","last_fullsized":"2024-09-10T23:40:02.369026+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:44.359631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.349639+0000","last_change":"2024-09-10T23:40:02.349639+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.349639+0000","last_undegraded":"2024-09-10T23:40:02.349639+0000","last_fullsized":"2024-09-10T23:40:02.349639+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.369067+0000","last_change":"2024-09-10T23:40:02.363592+0000","last_active":"2024-09-10T23:40:02.369067+0000","last_peered":"2024-09-10T23:40:02.369067+0000","last_clean":"2024-09-10T23:40:02.369067+0000","last_became_active":"2024-09-10T23:40:02.363294+0000","last_became_peered":"2024-09-10T23:40:02.363294+0000","last_unstale":"2024-09-10T23:40:02.369067+0000","last_undegraded":"2024-09-10T23:40:02.369067+0000","last_fullsized":"2024-09-10T23:40:02.369067+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:23.166700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.340809+0000","last_change":"2024-09-10T23:40:02.340809+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.340809+0000","last_undegraded":"2024-09-10T23:40:02.340809+0000","last_fullsized":"2024-09-10T23:40:02.340809+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359180+0000","last_change":"2024-09-10T23:40:02.353255+0000","last_active":"2024-09-10T23:40:02.359180+0000","last_peered":"2024-09-10T23:40:02.359180+0000","last_clean":"2024-09-10T23:40:02.359180+0000","last_became_active":"2024-09-10T23:40:02.352992+0000","last_became_peered":"2024-09-10T23:40:02.352992+0000","last_unstale":"2024-09-10T23:40:02.359180+0000","last_undegraded":"2024-09-10T23:40:02.359180+0000","last_fullsized":"2024-09-10T23:40:02.359180+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:19.346732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.353296+0000","last_change":"2024-09-10T23:40:02.353296+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.353296+0000","last_undegraded":"2024-09-10T23:40:02.353296+0000","last_fullsized":"2024-09-10T23:40:02.353296+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.360297+0000","last_change":"2024-09-10T23:40:02.359051+0000","last_active":"2024-09-10T23:40:02.360297+0000","last_peered":"2024-09-10T23:40:02.360297+0000","last_clean":"2024-09-10T23:40:02.360297+0000","last_became_active":"2024-09-10T23:40:02.357335+0000","last_became_peered":"2024-09-10T23:40:02.357335+0000","last_unstale":"2024-09-10T23:40:02.360297+0000","last_undegraded":"2024-09-10T23:40:02.360297+0000","last_fullsized":"2024-09-10T23:40:02.360297+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:07.378540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344844+0000","last_change":"2024-09-10T23:40:02.344844+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344844+0000","last_undegraded":"2024-09-10T23:40:02.344844+0000","last_fullsized":"2024-09-10T23:40:02.344844+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.362075+0000","last_change":"2024-09-10T23:40:02.359834+0000","last_active":"2024-09-10T23:40:02.362075+0000","last_peered":"2024-09-10T23:40:02.362075+0000","last_clean":"2024-09-10T23:40:02.362075+0000","last_became_active":"2024-09-10T23:40:02.359528+0000","last_became_peered":"2024-09-10T23:40:02.359528+0000","last_unstale":"2024-09-10T23:40:02.362075+0000","last_undegraded":"2024-09-10T23:40:02.362075+0000","last_fullsized":"2024-09-10T23:40:02.362075+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:53.213052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344044+0000","last_change":"2024-09-10T23:40:02.344044+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344044+0000","last_undegraded":"2024-09-10T23:40:02.344044+0000","last_fullsized":"2024-09-10T23:40:02.344044+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.369087+0000","last_change":"2024-09-10T23:40:02.364953+0000","last_active":"2024-09-10T23:40:02.369087+0000","last_peered":"2024-09-10T23:40:02.369087+0000","last_clean":"2024-09-10T23:40:02.369087+0000","last_became_active":"2024-09-10T23:40:02.364637+0000","last_became_peered":"2024-09-10T23:40:02.364637+0000","last_unstale":"2024-09-10T23:40:02.369087+0000","last_undegraded":"2024-09-10T23:40:02.369087+0000","last_fullsized":"2024-09-10T23:40:02.369087+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:22.601089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344602+0000","last_change":"2024-09-10T23:40:02.344602+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344602+0000","last_undegraded":"2024-09-10T23:40:02.344602+0000","last_fullsized":"2024-09-10T23:40:02.344602+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.366412+0000","last_change":"2024-09-10T23:40:02.361550+0000","last_active":"2024-09-10T23:40:02.366412+0000","last_peered":"2024-09-10T23:40:02.366412+0000","last_clean":"2024-09-10T23:40:02.366412+0000","last_became_active":"2024-09-10T23:40:02.361204+0000","last_became_peered":"2024-09-10T23:40:02.361204+0000","last_unstale":"2024-09-10T23:40:02.366412+0000","last_undegraded":"2024-09-10T23:40:02.366412+0000","last_fullsized":"2024-09-10T23:40:02.366412+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:30.158982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344061+0000","last_change":"2024-09-10T23:40:02.344061+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344061+0000","last_undegraded":"2024-09-10T23:40:02.344061+0000","last_fullsized":"2024-09-10T23:40:02.344061+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359981+0000","last_change":"2024-09-10T23:40:02.354529+0000","last_active":"2024-09-10T23:40:02.359981+0000","last_peered":"2024-09-10T23:40:02.359981+0000","last_clean":"2024-09-10T23:40:02.359981+0000","last_became_active":"2024-09-10T23:40:02.354273+0000","last_became_peered":"2024-09-10T23:40:02.354273+0000","last_unstale":"2024-09-10T23:40:02.359981+0000","last_undegraded":"2024-09-10T23:40:02.359981+0000","last_fullsized":"2024-09-10T23:40:02.359981+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:23:14.459079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.340447+0000","last_change":"2024-09-10T23:40:02.340447+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.340447+0000","last_undegraded":"2024-09-10T23:40:02.340447+0000","last_fullsized":"2024-09-10T23:40:02.340447+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.360107+0000","last_change":"2024-09-10T23:40:02.356793+0000","last_active":"2024-09-10T23:40:02.360107+0000","last_peered":"2024-09-10T23:40:02.360107+0000","last_clean":"2024-09-10T23:40:02.360107+0000","last_became_active":"2024-09-10T23:40:02.356539+0000","last_became_peered":"2024-09-10T23:40:02.356539+0000","last_unstale":"2024-09-10T23:40:02.360107+0000","last_undegraded":"2024-09-10T23:40:02.360107+0000","last_fullsized":"2024-09-10T23:40:02.360107+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:34.484400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342891+0000","last_change":"2024-09-10T23:40:02.342891+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342891+0000","last_undegraded":"2024-09-10T23:40:02.342891+0000","last_fullsized":"2024-09-10T23:40:02.342891+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.366476+0000","last_change":"2024-09-10T23:40:02.363352+0000","last_active":"2024-09-10T23:40:02.366476+0000","last_peered":"2024-09-10T23:40:02.366476+0000","last_clean":"2024-09-10T23:40:02.366476+0000","last_became_active":"2024-09-10T23:40:02.362332+0000","last_became_peered":"2024-09-10T23:40:02.362332+0000","last_unstale":"2024-09-10T23:40:02.366476+0000","last_undegraded":"2024-09-10T23:40:02.366476+0000","last_fullsized":"2024-09-10T23:40:02.366476+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:25.087218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348260+0000","last_change":"2024-09-10T23:40:02.348260+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348260+0000","last_undegraded":"2024-09-10T23:40:02.348260+0000","last_fullsized":"2024-09-10T23:40:02.348260+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359825+0000","last_change":"2024-09-10T23:40:02.357720+0000","last_active":"2024-09-10T23:40:02.359825+0000","last_peered":"2024-09-10T23:40:02.359825+0000","last_clean":"2024-09-10T23:40:02.359825+0000","last_became_active":"2024-09-10T23:40:02.357265+0000","last_became_peered":"2024-09-10T23:40:02.357265+0000","last_unstale":"2024-09-10T23:40:02.359825+0000","last_undegraded":"2024-09-10T23:40:02.359825+0000","last_fullsized":"2024-09-10T23:40:02.359825+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:23:14.459079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344468+0000","last_change":"2024-09-10T23:40:02.344468+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344468+0000","last_undegraded":"2024-09-10T23:40:02.344468+0000","last_fullsized":"2024-09-10T23:40:02.344468+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.369328+0000","last_change":"2024-09-10T23:40:02.367006+0000","last_active":"2024-09-10T23:40:02.369328+0000","last_peered":"2024-09-10T23:40:02.369328+0000","last_clean":"2024-09-10T23:40:02.369328+0000","last_became_active":"2024-09-10T23:40:02.366761+0000","last_became_peered":"2024-09-10T23:40:02.366761+0000","last_unstale":"2024-09-10T23:40:02.369328+0000","last_undegraded":"2024-09-10T23:40:02.369328+0000","last_fullsized":"2024-09-10T23:40:02.369328+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:29.028851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.343271+0000","last_change":"2024-09-10T23:40:02.343271+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.343271+0000","last_undegraded":"2024-09-10T23:40:02.343271+0000","last_fullsized":"2024-09-10T23:40:02.343271+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.352123+0000","last_change":"2024-09-10T23:40:02.350625+0000","last_active":"2024-09-10T23:40:02.352123+0000","last_peered":"2024-09-10T23:40:02.352123+0000","last_clean":"2024-09-10T23:40:02.352123+0000","last_became_active":"2024-09-10T23:40:02.349954+0000","last_became_peered":"2024-09-10T23:40:02.349954+0000","last_unstale":"2024-09-10T23:40:02.352123+0000","last_undegraded":"2024-09-10T23:40:02.352123+0000","last_fullsized":"2024-09-10T23:40:02.352123+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:06.867213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342438+0000","last_change":"2024-09-10T23:40:02.342438+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342438+0000","last_undegraded":"2024-09-10T23:40:02.342438+0000","last_fullsized":"2024-09-10T23:40:02.342438+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359215+0000","last_change":"2024-09-10T23:40:02.355908+0000","last_active":"2024-09-10T23:40:02.359215+0000","last_peered":"2024-09-10T23:40:02.359215+0000","last_clean":"2024-09-10T23:40:02.359215+0000","last_became_active":"2024-09-10T23:40:02.355653+0000","last_became_peered":"2024-09-10T23:40:02.355653+0000","last_unstale":"2024-09-10T23:40:02.359215+0000","last_undegraded":"2024-09-10T23:40:02.359215+0000","last_fullsized":"2024-09-10T23:40:02.359215+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:09.680725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344708+0000","last_change":"2024-09-10T23:40:02.344708+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344708+0000","last_undegraded":"2024-09-10T23:40:02.344708+0000","last_fullsized":"2024-09-10T23:40:02.344708+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.353839+0000","last_change":"2024-09-10T23:40:02.352194+0000","last_active":"2024-09-10T23:40:02.353839+0000","last_peered":"2024-09-10T23:40:02.353839+0000","last_clean":"2024-09-10T23:40:02.353839+0000","last_became_active":"2024-09-10T23:40:02.351903+0000","last_became_peered":"2024-09-10T23:40:02.351903+0000","last_unstale":"2024-09-10T23:40:02.353839+0000","last_undegraded":"2024-09-10T23:40:02.353839+0000","last_fullsized":"2024-09-10T23:40:02.353839+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:29.620254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344780+0000","last_change":"2024-09-10T23:40:02.344780+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344780+0000","last_undegraded":"2024-09-10T23:40:02.344780+0000","last_fullsized":"2024-09-10T23:40:02.344780+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.363830+0000","last_change":"2024-09-10T23:40:02.360470+0000","last_active":"2024-09-10T23:40:02.363830+0000","last_peered":"2024-09-10T23:40:02.363830+0000","last_clean":"2024-09-10T23:40:02.363830+0000","last_became_active":"2024-09-10T23:40:02.360193+0000","last_became_peered":"2024-09-10T23:40:02.360193+0000","last_unstale":"2024-09-10T23:40:02.363830+0000","last_undegraded":"2024-09-10T23:40:02.363830+0000","last_fullsized":"2024-09-10T23:40:02.363830+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:37.483769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.340014+0000","last_change":"2024-09-10T23:40:02.340014+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.340014+0000","last_undegraded":"2024-09-10T23:40:02.340014+0000","last_fullsized":"2024-09-10T23:40:02.340014+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.354285+0000","last_change":"2024-09-10T23:40:02.353010+0000","last_active":"2024-09-10T23:40:02.354285+0000","last_peered":"2024-09-10T23:40:02.354285+0000","last_clean":"2024-09-10T23:40:02.354285+0000","last_became_active":"2024-09-10T23:40:02.352736+0000","last_became_peered":"2024-09-10T23:40:02.352736+0000","last_unstale":"2024-09-10T23:40:02.354285+0000","last_undegraded":"2024-09-10T23:40:02.354285+0000","last_fullsized":"2024-09-10T23:40:02.354285+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:04:05.247046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.341152+0000","last_change":"2024-09-10T23:40:02.341152+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.341152+0000","last_undegraded":"2024-09-10T23:40:02.341152+0000","last_fullsized":"2024-09-10T23:40:02.341152+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.357525+0000","last_change":"2024-09-10T23:40:02.356421+0000","last_active":"2024-09-10T23:40:02.357525+0000","last_peered":"2024-09-10T23:40:02.357525+0000","last_clean":"2024-09-10T23:40:02.357525+0000","last_became_active":"2024-09-10T23:40:02.355527+0000","last_became_peered":"2024-09-10T23:40:02.355527+0000","last_unstale":"2024-09-10T23:40:02.357525+0000","last_undegraded":"2024-09-10T23:40:02.357525+0000","last_fullsized":"2024-09-10T23:40:02.357525+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:30.479559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.345583+0000","last_change":"2024-09-10T23:40:02.345583+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.345583+0000","last_undegraded":"2024-09-10T23:40:02.345583+0000","last_fullsized":"2024-09-10T23:40:02.345583+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.358394+0000","last_change":"2024-09-10T23:40:02.355475+0000","last_active":"2024-09-10T23:40:02.358394+0000","last_peered":"2024-09-10T23:40:02.358394+0000","last_clean":"2024-09-10T23:40:02.358394+0000","last_became_active":"2024-09-10T23:40:02.355204+0000","last_became_peered":"2024-09-10T23:40:02.355204+0000","last_unstale":"2024-09-10T23:40:02.358394+0000","last_undegraded":"2024-09-10T23:40:02.358394+0000","last_fullsized":"2024-09-10T23:40:02.358394+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:07.607518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350633+0000","last_change":"2024-09-10T23:40:02.350633+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350633+0000","last_undegraded":"2024-09-10T23:40:02.350633+0000","last_fullsized":"2024-09-10T23:40:02.350633+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.354125+0000","last_change":"2024-09-10T23:40:02.352054+0000","last_active":"2024-09-10T23:40:02.354125+0000","last_peered":"2024-09-10T23:40:02.354125+0000","last_clean":"2024-09-10T23:40:02.354125+0000","last_became_active":"2024-09-10T23:40:02.351786+0000","last_became_peered":"2024-09-10T23:40:02.351786+0000","last_unstale":"2024-09-10T23:40:02.354125+0000","last_undegraded":"2024-09-10T23:40:02.354125+0000","last_fullsized":"2024-09-10T23:40:02.354125+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:32.578739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350743+0000","last_change":"2024-09-10T23:40:02.350743+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350743+0000","last_undegraded":"2024-09-10T23:40:02.350743+0000","last_fullsized":"2024-09-10T23:40:02.350743+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.366263+0000","last_change":"2024-09-10T23:40:02.360781+0000","last_active":"2024-09-10T23:40:02.366263+0000","last_peered":"2024-09-10T23:40:02.366263+0000","last_clean":"2024-09-10T23:40:02.366263+0000","last_became_active":"2024-09-10T23:40:02.360156+0000","last_became_peered":"2024-09-10T23:40:02.360156+0000","last_unstale":"2024-09-10T23:40:02.366263+0000","last_undegraded":"2024-09-10T23:40:02.366263+0000","last_fullsized":"2024-09-10T23:40:02.366263+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:34.016734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.352067+0000","last_change":"2024-09-10T23:40:02.352067+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.352067+0000","last_undegraded":"2024-09-10T23:40:02.352067+0000","last_fullsized":"2024-09-10T23:40:02.352067+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.363490+0000","last_change":"2024-09-10T23:40:02.359868+0000","last_active":"2024-09-10T23:40:02.363490+0000","last_peered":"2024-09-10T23:40:02.363490+0000","last_clean":"2024-09-10T23:40:02.363490+0000","last_became_active":"2024-09-10T23:40:02.359600+0000","last_became_peered":"2024-09-10T23:40:02.359600+0000","last_unstale":"2024-09-10T23:40:02.363490+0000","last_undegraded":"2024-09-10T23:40:02.363490+0000","last_fullsized":"2024-09-10T23:40:02.363490+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:08.479640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.349167+0000","last_change":"2024-09-10T23:40:02.349167+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.349167+0000","last_undegraded":"2024-09-10T23:40:02.349167+0000","last_fullsized":"2024-09-10T23:40:02.349167+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.366436+0000","last_change":"2024-09-10T23:40:02.362452+0000","last_active":"2024-09-10T23:40:02.366436+0000","last_peered":"2024-09-10T23:40:02.366436+0000","last_clean":"2024-09-10T23:40:02.366436+0000","last_became_active":"2024-09-10T23:40:02.362039+0000","last_became_peered":"2024-09-10T23:40:02.362039+0000","last_unstale":"2024-09-10T23:40:02.366436+0000","last_undegraded":"2024-09-10T23:40:02.366436+0000","last_fullsized":"2024-09-10T23:40:02.366436+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:44.161075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342827+0000","last_change":"2024-09-10T23:40:02.342827+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342827+0000","last_undegraded":"2024-09-10T23:40:02.342827+0000","last_fullsized":"2024-09-10T23:40:02.342827+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359699+0000","last_change":"2024-09-10T23:40:02.356946+0000","last_active":"2024-09-10T23:40:02.359699+0000","last_peered":"2024-09-10T23:40:02.359699+0000","last_clean":"2024-09-10T23:40:02.359699+0000","last_became_active":"2024-09-10T23:40:02.356690+0000","last_became_peered":"2024-09-10T23:40:02.356690+0000","last_unstale":"2024-09-10T23:40:02.359699+0000","last_undegraded":"2024-09-10T23:40:02.359699+0000","last_fullsized":"2024-09-10T23:40:02.359699+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:48.949690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.341411+0000","last_change":"2024-09-10T23:40:02.341411+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.341411+0000","last_undegraded":"2024-09-10T23:40:02.341411+0000","last_fullsized":"2024-09-10T23:40:02.341411+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.355292+0000","last_change":"2024-09-10T23:40:02.349369+0000","last_active":"2024-09-10T23:40:02.355292+0000","last_peered":"2024-09-10T23:40:02.355292+0000","last_clean":"2024-09-10T23:40:02.355292+0000","last_became_active":"2024-09-10T23:40:02.349059+0000","last_became_peered":"2024-09-10T23:40:02.349059+0000","last_unstale":"2024-09-10T23:40:02.355292+0000","last_undegraded":"2024-09-10T23:40:02.355292+0000","last_fullsized":"2024-09-10T23:40:02.355292+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:30.313719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.341593+0000","last_change":"2024-09-10T23:40:02.341593+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.341593+0000","last_undegraded":"2024-09-10T23:40:02.341593+0000","last_fullsized":"2024-09-10T23:40:02.341593+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.354805+0000","last_change":"2024-09-10T23:40:02.349452+0000","last_active":"2024-09-10T23:40:02.354805+0000","last_peered":"2024-09-10T23:40:02.354805+0000","last_clean":"2024-09-10T23:40:02.354805+0000","last_became_active":"2024-09-10T23:40:02.349132+0000","last_became_peered":"2024-09-10T23:40:02.349132+0000","last_unstale":"2024-09-10T23:40:02.354805+0000","last_undegraded":"2024-09-10T23:40:02.354805+0000","last_fullsized":"2024-09-10T23:40:02.354805+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:51:55.558547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.347814+0000","last_change":"2024-09-10T23:40:02.347814+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.347814+0000","last_undegraded":"2024-09-10T23:40:02.347814+0000","last_fullsized":"2024-09-10T23:40:02.347814+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.362251+0000","last_change":"2024-09-10T23:40:02.360113+0000","last_active":"2024-09-10T23:40:02.362251+0000","last_peered":"2024-09-10T23:40:02.362251+0000","last_clean":"2024-09-10T23:40:02.362251+0000","last_became_active":"2024-09-10T23:40:02.359801+0000","last_became_peered":"2024-09-10T23:40:02.359801+0000","last_unstale":"2024-09-10T23:40:02.362251+0000","last_undegraded":"2024-09-10T23:40:02.362251+0000","last_fullsized":"2024-09-10T23:40:02.362251+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:19.215661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.346140+0000","last_change":"2024-09-10T23:40:02.346140+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.346140+0000","last_undegraded":"2024-09-10T23:40:02.346140+0000","last_fullsized":"2024-09-10T23:40:02.346140+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.360040+0000","last_change":"2024-09-10T23:40:02.354814+0000","last_active":"2024-09-10T23:40:02.360040+0000","last_peered":"2024-09-10T23:40:02.360040+0000","last_clean":"2024-09-10T23:40:02.360040+0000","last_became_active":"2024-09-10T23:40:02.354579+0000","last_became_peered":"2024-09-10T23:40:02.354579+0000","last_unstale":"2024-09-10T23:40:02.360040+0000","last_undegraded":"2024-09-10T23:40:02.360040+0000","last_fullsized":"2024-09-10T23:40:02.360040+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:04:11.336551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":371,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.352223+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:40:02.352223+0000","last_peered":"2024-09-10T23:40:02.352223+0000","last_clean":"2024-09-10T23:40:02.352223+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:40:02.352223+0000","last_undegraded":"2024-09-10T23:40:02.352223+0000","last_fullsized":"2024-09-10T23:40:02.352223+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509536,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33004,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338836,"statfs":{"total":96636764160,"available":96602968064,"internally_reserved":0,"allocated":1859584,"data_stored":816974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542239,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33004,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338836,"statfs":{"total":96636764160,"available":96602968064,"internally_reserved":0,"allocated":1859584,"data_stored":816974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509535,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33584,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338256,"statfs":{"total":96636764160,"available":96602374144,"internally_reserved":0,"allocated":2453504,"data_stored":1407342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509533,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1871872,"data_stored":823277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509534,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1871872,"data_stored":823277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509535,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37164,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334676,"statfs":{"total":96636764160,"available":96598708224,"internally_reserved":0,"allocated":1859584,"data_stored":816974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509534,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37744,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334096,"statfs":{"total":96636764160,"available":96598114304,"internally_reserved":0,"allocated":2453504,"data_stored":1407342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509534,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37180,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334660,"statfs":{"total":96636764160,"available":96598691840,"internally_reserved":0,"allocated":1871872,"data_stored":823277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:40:07.513 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:40:07.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea97800640 1 -- 172.21.15.94:0/134069560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea74004d50 msgr2=0x7fea740051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:07.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea97800640 1 --2- 172.21.15.94:0/134069560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea74004d50 0x7fea740051d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fea8800c640 tx=0x7fea8801fa60 comp rx=0 tx=0).stop 2024-09-10T23:40:07.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea95800640 1 -- 172.21.15.94:0/134069560 reap_dead start 2024-09-10T23:40:07.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea97800640 1 -- 172.21.15.94:0/134069560 shutdown_connections 2024-09-10T23:40:07.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea97800640 1 -- 172.21.15.94:0/134069560 >> 172.21.15.94:0/134069560 conn(0x7fea9006cdf0 msgr2=0x7fea9006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:07.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea97800640 1 -- 172.21.15.94:0/134069560 shutdown_connections 2024-09-10T23:40:07.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea97800640 1 -- 172.21.15.94:0/134069560 wait complete. 2024-09-10T23:40:07.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea97800640 1 Processor -- start 2024-09-10T23:40:07.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea97800640 1 -- start start 2024-09-10T23:40:07.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea97800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea74004d50 0x7fea9014f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea97800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea90150270 0x7fea901f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea97800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea901506f0 0x7fea901f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea97800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea900cef40 con 0x7fea901506f0 2024-09-10T23:40:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea97800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fea900cedc0 con 0x7fea90150270 2024-09-10T23:40:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea97800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fea900b4700 con 0x7fea74004d50 2024-09-10T23:40:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea95800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea901506f0 0x7fea901f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea95800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea901506f0 0x7fea901f7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34594/0 (socket says 172.21.15.94:34594) 2024-09-10T23:40:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea95800640 1 -- 172.21.15.94:0/552927975 learned_addr learned my addr 172.21.15.94:0/552927975 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea94e00640 1 --2- 172.21.15.94:0/552927975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea74004d50 0x7fea9014f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea95800640 1 -- 172.21.15.94:0/552927975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea74004d50 msgr2=0x7fea9014f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea8fe00640 1 --2- 172.21.15.94:0/552927975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea90150270 0x7fea901f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea95800640 1 --2- 172.21.15.94:0/552927975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea74004d50 0x7fea9014f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea95800640 1 -- 172.21.15.94:0/552927975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea90150270 msgr2=0x7fea901f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.687+0000 7fea95800640 1 --2- 172.21.15.94:0/552927975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea90150270 0x7fea901f79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.691+0000 7fea95800640 1 -- 172.21.15.94:0/552927975 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea90151560 con 0x7fea901506f0 2024-09-10T23:40:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.691+0000 7fea94e00640 1 --2- 172.21.15.94:0/552927975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea74004d50 0x7fea9014f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.691+0000 7fea8fe00640 1 --2- 172.21.15.94:0/552927975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea90150270 0x7fea901f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.691+0000 7fea95800640 1 --2- 172.21.15.94:0/552927975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea901506f0 0x7fea901f7f20 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fea88021140 tx=0x7fea8801fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.691+0000 7fea8cc00640 1 -- 172.21.15.94:0/552927975 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea880011d0 con 0x7fea901506f0 2024-09-10T23:40:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.691+0000 7fea8cc00640 1 -- 172.21.15.94:0/552927975 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea88001a40 con 0x7fea901506f0 2024-09-10T23:40:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.691+0000 7fea8cc00640 1 -- 172.21.15.94:0/552927975 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88002040 con 0x7fea901506f0 2024-09-10T23:40:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.691+0000 7fea97800640 1 -- 172.21.15.94:0/552927975 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea90150cc0 con 0x7fea901506f0 2024-09-10T23:40:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.691+0000 7fea97800640 1 -- 172.21.15.94:0/552927975 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea90151280 con 0x7fea901506f0 2024-09-10T23:40:07.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.691+0000 7fea97800640 1 -- 172.21.15.94:0/552927975 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea900cfcb0 con 0x7fea901506f0 2024-09-10T23:40:07.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.691+0000 7fea8cc00640 1 -- 172.21.15.94:0/552927975 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fea88003ba0 con 0x7fea901506f0 2024-09-10T23:40:07.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.691+0000 7fea8cc00640 1 --2- 172.21.15.94:0/552927975 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea70072e50 0x7fea70075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:07.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.691+0000 7fea8cc00640 1 -- 172.21.15.94:0/552927975 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7fea880b9020 con 0x7fea901506f0 2024-09-10T23:40:07.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.691+0000 7fea94e00640 1 --2- 172.21.15.94:0/552927975 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea70072e50 0x7fea70075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:07.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.695+0000 7fea94e00640 1 --2- 172.21.15.94:0/552927975 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea70072e50 0x7fea70075310 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fea80009b50 tx=0x7fea800091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:07.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.695+0000 7fea8cc00640 1 -- 172.21.15.94:0/552927975 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fea88019810 con 0x7fea901506f0 2024-09-10T23:40:07.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.863+0000 7fea97800640 1 -- 172.21.15.94:0/552927975 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fea9014f680 con 0x7fea70072e50 2024-09-10T23:40:07.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.867+0000 7fea8cc00640 1 -- 172.21.15.94:0/552927975 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247114 (secure 0 0 0) 0x7fea80011470 con 0x7fea70072e50 2024-09-10T23:40:07.873 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:07.874 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:40:07.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.871+0000 7fea97800640 1 -- 172.21.15.94:0/552927975 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea70072e50 msgr2=0x7fea70075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:07.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.871+0000 7fea97800640 1 --2- 172.21.15.94:0/552927975 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea70072e50 0x7fea70075310 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fea80009b50 tx=0x7fea800091e0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.871+0000 7fea97800640 1 -- 172.21.15.94:0/552927975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea901506f0 msgr2=0x7fea901f7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:07.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.871+0000 7fea97800640 1 --2- 172.21.15.94:0/552927975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea901506f0 0x7fea901f7f20 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fea88021140 tx=0x7fea8801fcb0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.871+0000 7fea97800640 1 -- 172.21.15.94:0/552927975 shutdown_connections 2024-09-10T23:40:07.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.871+0000 7fea97800640 1 --2- 172.21.15.94:0/552927975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea90150270 0x7fea901f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.871+0000 7fea97800640 1 --2- 172.21.15.94:0/552927975 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fea70072e50 0x7fea70075310 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.871+0000 7fea97800640 1 --2- 172.21.15.94:0/552927975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea901506f0 0x7fea901f7f20 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.871+0000 7fea97800640 1 --2- 172.21.15.94:0/552927975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea74004d50 0x7fea9014f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:07.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.871+0000 7fea97800640 1 -- 172.21.15.94:0/552927975 >> 172.21.15.94:0/552927975 conn(0x7fea9006cdf0 msgr2=0x7fea900b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:07.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.871+0000 7fea97800640 1 -- 172.21.15.94:0/552927975 shutdown_connections 2024-09-10T23:40:07.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:07.871+0000 7fea97800640 1 -- 172.21.15.94:0/552927975 wait complete. 2024-09-10T23:40:07.893 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-10T23:40:06.778391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":277720,"kb_used_data":15724,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754697000,"statfs":{"total":773094113280,"available":772809728000,"internally_reserved":0,"allocated":16101376,"data_stored":7735437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.453634"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342043+0000","last_change":"2024-09-10T23:40:02.342043+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342043+0000","last_undegraded":"2024-09-10T23:40:02.342043+0000","last_fullsized":"2024-09-10T23:40:02.342043+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344703+0000","last_change":"2024-09-10T23:40:02.344703+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344703+0000","last_undegraded":"2024-09-10T23:40:02.344703+0000","last_fullsized":"2024-09-10T23:40:02.344703+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.353593+0000","last_change":"2024-09-10T23:40:02.353593+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.353593+0000","last_undegraded":"2024-09-10T23:40:02.353593+0000","last_fullsized":"2024-09-10T23:40:02.353593+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.343368+0000","last_change":"2024-09-10T23:40:02.343368+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.343368+0000","last_undegraded":"2024-09-10T23:40:02.343368+0000","last_fullsized":"2024-09-10T23:40:02.343368+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.351849+0000","last_change":"2024-09-10T23:40:02.351849+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.351849+0000","last_undegraded":"2024-09-10T23:40:02.351849+0000","last_fullsized":"2024-09-10T23:40:02.351849+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.345840+0000","last_change":"2024-09-10T23:40:02.345840+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.345840+0000","last_undegraded":"2024-09-10T23:40:02.345840+0000","last_fullsized":"2024-09-10T23:40:02.345840+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.339971+0000","last_change":"2024-09-10T23:40:02.339971+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.339971+0000","last_undegraded":"2024-09-10T23:40:02.339971+0000","last_fullsized":"2024-09-10T23:40:02.339971+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350386+0000","last_change":"2024-09-10T23:40:02.350386+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350386+0000","last_undegraded":"2024-09-10T23:40:02.350386+0000","last_fullsized":"2024-09-10T23:40:02.350386+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348677+0000","last_change":"2024-09-10T23:40:02.348677+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348677+0000","last_undegraded":"2024-09-10T23:40:02.348677+0000","last_fullsized":"2024-09-10T23:40:02.348677+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348260+0000","last_change":"2024-09-10T23:40:02.348260+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348260+0000","last_undegraded":"2024-09-10T23:40:02.348260+0000","last_fullsized":"2024-09-10T23:40:02.348260+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342470+0000","last_change":"2024-09-10T23:40:02.342470+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342470+0000","last_undegraded":"2024-09-10T23:40:02.342470+0000","last_fullsized":"2024-09-10T23:40:02.342470+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344421+0000","last_change":"2024-09-10T23:40:02.344421+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344421+0000","last_undegraded":"2024-09-10T23:40:02.344421+0000","last_fullsized":"2024-09-10T23:40:02.344421+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.341773+0000","last_change":"2024-09-10T23:40:02.341773+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.341773+0000","last_undegraded":"2024-09-10T23:40:02.341773+0000","last_fullsized":"2024-09-10T23:40:02.341773+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.343249+0000","last_change":"2024-09-10T23:40:02.343249+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.343249+0000","last_undegraded":"2024-09-10T23:40:02.343249+0000","last_fullsized":"2024-09-10T23:40:02.343249+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.343783+0000","last_change":"2024-09-10T23:40:02.343783+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.343783+0000","last_undegraded":"2024-09-10T23:40:02.343783+0000","last_fullsized":"2024-09-10T23:40:02.343783+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350869+0000","last_change":"2024-09-10T23:40:02.350869+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350869+0000","last_undegraded":"2024-09-10T23:40:02.350869+0000","last_fullsized":"2024-09-10T23:40:02.350869+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.347466+0000","last_change":"2024-09-10T23:40:02.347466+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.347466+0000","last_undegraded":"2024-09-10T23:40:02.347466+0000","last_fullsized":"2024-09-10T23:40:02.347466+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.340509+0000","last_change":"2024-09-10T23:40:02.340509+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.340509+0000","last_undegraded":"2024-09-10T23:40:02.340509+0000","last_fullsized":"2024-09-10T23:40:02.340509+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.353158+0000","last_change":"2024-09-10T23:40:02.353158+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.353158+0000","last_undegraded":"2024-09-10T23:40:02.353158+0000","last_fullsized":"2024-09-10T23:40:02.353158+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348678+0000","last_change":"2024-09-10T23:40:02.348678+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348678+0000","last_undegraded":"2024-09-10T23:40:02.348678+0000","last_fullsized":"2024-09-10T23:40:02.348678+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.351641+0000","last_change":"2024-09-10T23:40:02.351641+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.351641+0000","last_undegraded":"2024-09-10T23:40:02.351641+0000","last_fullsized":"2024-09-10T23:40:02.351641+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350410+0000","last_change":"2024-09-10T23:40:02.350410+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350410+0000","last_undegraded":"2024-09-10T23:40:02.350410+0000","last_fullsized":"2024-09-10T23:40:02.350410+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.345990+0000","last_change":"2024-09-10T23:40:02.345990+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.345990+0000","last_undegraded":"2024-09-10T23:40:02.345990+0000","last_fullsized":"2024-09-10T23:40:02.345990+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.349962+0000","last_change":"2024-09-10T23:40:02.349962+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.349962+0000","last_undegraded":"2024-09-10T23:40:02.349962+0000","last_fullsized":"2024-09-10T23:40:02.349962+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348253+0000","last_change":"2024-09-10T23:40:02.348253+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348253+0000","last_undegraded":"2024-09-10T23:40:02.348253+0000","last_fullsized":"2024-09-10T23:40:02.348253+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348873+0000","last_change":"2024-09-10T23:40:02.348873+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348873+0000","last_undegraded":"2024-09-10T23:40:02.348873+0000","last_fullsized":"2024-09-10T23:40:02.348873+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.352668+0000","last_change":"2024-09-10T23:40:02.352668+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.352668+0000","last_undegraded":"2024-09-10T23:40:02.352668+0000","last_fullsized":"2024-09-10T23:40:02.352668+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344673+0000","last_change":"2024-09-10T23:40:02.344673+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344673+0000","last_undegraded":"2024-09-10T23:40:02.344673+0000","last_fullsized":"2024-09-10T23:40:02.344673+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.343596+0000","last_change":"2024-09-10T23:40:02.343596+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.343596+0000","last_undegraded":"2024-09-10T23:40:02.343596+0000","last_fullsized":"2024-09-10T23:40:02.343596+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.346547+0000","last_change":"2024-09-10T23:40:02.346547+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.346547+0000","last_undegraded":"2024-09-10T23:40:02.346547+0000","last_fullsized":"2024-09-10T23:40:02.346547+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350824+0000","last_change":"2024-09-10T23:40:02.350824+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350824+0000","last_undegraded":"2024-09-10T23:40:02.350824+0000","last_fullsized":"2024-09-10T23:40:02.350824+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348261+0000","last_change":"2024-09-10T23:40:02.348261+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348261+0000","last_undegraded":"2024-09-10T23:40:02.348261+0000","last_fullsized":"2024-09-10T23:40:02.348261+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.349611+0000","last_change":"2024-09-10T23:40:02.349611+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.349611+0000","last_undegraded":"2024-09-10T23:40:02.349611+0000","last_fullsized":"2024-09-10T23:40:02.349611+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359443+0000","last_change":"2024-09-10T23:40:02.357339+0000","last_active":"2024-09-10T23:40:02.359443+0000","last_peered":"2024-09-10T23:40:02.359443+0000","last_clean":"2024-09-10T23:40:02.359443+0000","last_became_active":"2024-09-10T23:40:02.357054+0000","last_became_peered":"2024-09-10T23:40:02.357054+0000","last_unstale":"2024-09-10T23:40:02.359443+0000","last_undegraded":"2024-09-10T23:40:02.359443+0000","last_fullsized":"2024-09-10T23:40:02.359443+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:21.460711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.345207+0000","last_change":"2024-09-10T23:40:02.345207+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.345207+0000","last_undegraded":"2024-09-10T23:40:02.345207+0000","last_fullsized":"2024-09-10T23:40:02.345207+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.358117+0000","last_change":"2024-09-10T23:40:02.354357+0000","last_active":"2024-09-10T23:40:02.358117+0000","last_peered":"2024-09-10T23:40:02.358117+0000","last_clean":"2024-09-10T23:40:02.358117+0000","last_became_active":"2024-09-10T23:40:02.354105+0000","last_became_peered":"2024-09-10T23:40:02.354105+0000","last_unstale":"2024-09-10T23:40:02.358117+0000","last_undegraded":"2024-09-10T23:40:02.358117+0000","last_fullsized":"2024-09-10T23:40:02.358117+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:34.016734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342932+0000","last_change":"2024-09-10T23:40:02.342932+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342932+0000","last_undegraded":"2024-09-10T23:40:02.342932+0000","last_fullsized":"2024-09-10T23:40:02.342932+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.369058+0000","last_change":"2024-09-10T23:40:02.363918+0000","last_active":"2024-09-10T23:40:02.369058+0000","last_peered":"2024-09-10T23:40:02.369058+0000","last_clean":"2024-09-10T23:40:02.369058+0000","last_became_active":"2024-09-10T23:40:02.363627+0000","last_became_peered":"2024-09-10T23:40:02.363627+0000","last_unstale":"2024-09-10T23:40:02.369058+0000","last_undegraded":"2024-09-10T23:40:02.369058+0000","last_fullsized":"2024-09-10T23:40:02.369058+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:14.649948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342179+0000","last_change":"2024-09-10T23:40:02.342179+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342179+0000","last_undegraded":"2024-09-10T23:40:02.342179+0000","last_fullsized":"2024-09-10T23:40:02.342179+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359942+0000","last_change":"2024-09-10T23:40:02.354874+0000","last_active":"2024-09-10T23:40:02.359942+0000","last_peered":"2024-09-10T23:40:02.359942+0000","last_clean":"2024-09-10T23:40:02.359942+0000","last_became_active":"2024-09-10T23:40:02.354568+0000","last_became_peered":"2024-09-10T23:40:02.354568+0000","last_unstale":"2024-09-10T23:40:02.359942+0000","last_undegraded":"2024-09-10T23:40:02.359942+0000","last_fullsized":"2024-09-10T23:40:02.359942+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:19.346732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.345401+0000","last_change":"2024-09-10T23:40:02.345401+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.345401+0000","last_undegraded":"2024-09-10T23:40:02.345401+0000","last_fullsized":"2024-09-10T23:40:02.345401+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.369026+0000","last_change":"2024-09-10T23:40:02.363569+0000","last_active":"2024-09-10T23:40:02.369026+0000","last_peered":"2024-09-10T23:40:02.369026+0000","last_clean":"2024-09-10T23:40:02.369026+0000","last_became_active":"2024-09-10T23:40:02.362386+0000","last_became_peered":"2024-09-10T23:40:02.362386+0000","last_unstale":"2024-09-10T23:40:02.369026+0000","last_undegraded":"2024-09-10T23:40:02.369026+0000","last_fullsized":"2024-09-10T23:40:02.369026+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:44.359631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.349639+0000","last_change":"2024-09-10T23:40:02.349639+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.349639+0000","last_undegraded":"2024-09-10T23:40:02.349639+0000","last_fullsized":"2024-09-10T23:40:02.349639+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.369067+0000","last_change":"2024-09-10T23:40:02.363592+0000","last_active":"2024-09-10T23:40:02.369067+0000","last_peered":"2024-09-10T23:40:02.369067+0000","last_clean":"2024-09-10T23:40:02.369067+0000","last_became_active":"2024-09-10T23:40:02.363294+0000","last_became_peered":"2024-09-10T23:40:02.363294+0000","last_unstale":"2024-09-10T23:40:02.369067+0000","last_undegraded":"2024-09-10T23:40:02.369067+0000","last_fullsized":"2024-09-10T23:40:02.369067+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:23.166700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.340809+0000","last_change":"2024-09-10T23:40:02.340809+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.340809+0000","last_undegraded":"2024-09-10T23:40:02.340809+0000","last_fullsized":"2024-09-10T23:40:02.340809+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359180+0000","last_change":"2024-09-10T23:40:02.353255+0000","last_active":"2024-09-10T23:40:02.359180+0000","last_peered":"2024-09-10T23:40:02.359180+0000","last_clean":"2024-09-10T23:40:02.359180+0000","last_became_active":"2024-09-10T23:40:02.352992+0000","last_became_peered":"2024-09-10T23:40:02.352992+0000","last_unstale":"2024-09-10T23:40:02.359180+0000","last_undegraded":"2024-09-10T23:40:02.359180+0000","last_fullsized":"2024-09-10T23:40:02.359180+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:19.346732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.353296+0000","last_change":"2024-09-10T23:40:02.353296+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.353296+0000","last_undegraded":"2024-09-10T23:40:02.353296+0000","last_fullsized":"2024-09-10T23:40:02.353296+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.360297+0000","last_change":"2024-09-10T23:40:02.359051+0000","last_active":"2024-09-10T23:40:02.360297+0000","last_peered":"2024-09-10T23:40:02.360297+0000","last_clean":"2024-09-10T23:40:02.360297+0000","last_became_active":"2024-09-10T23:40:02.357335+0000","last_became_peered":"2024-09-10T23:40:02.357335+0000","last_unstale":"2024-09-10T23:40:02.360297+0000","last_undegraded":"2024-09-10T23:40:02.360297+0000","last_fullsized":"2024-09-10T23:40:02.360297+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:07.378540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344844+0000","last_change":"2024-09-10T23:40:02.344844+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344844+0000","last_undegraded":"2024-09-10T23:40:02.344844+0000","last_fullsized":"2024-09-10T23:40:02.344844+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.362075+0000","last_change":"2024-09-10T23:40:02.359834+0000","last_active":"2024-09-10T23:40:02.362075+0000","last_peered":"2024-09-10T23:40:02.362075+0000","last_clean":"2024-09-10T23:40:02.362075+0000","last_became_active":"2024-09-10T23:40:02.359528+0000","last_became_peered":"2024-09-10T23:40:02.359528+0000","last_unstale":"2024-09-10T23:40:02.362075+0000","last_undegraded":"2024-09-10T23:40:02.362075+0000","last_fullsized":"2024-09-10T23:40:02.362075+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:53.213052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344044+0000","last_change":"2024-09-10T23:40:02.344044+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344044+0000","last_undegraded":"2024-09-10T23:40:02.344044+0000","last_fullsized":"2024-09-10T23:40:02.344044+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.369087+0000","last_change":"2024-09-10T23:40:02.364953+0000","last_active":"2024-09-10T23:40:02.369087+0000","last_peered":"2024-09-10T23:40:02.369087+0000","last_clean":"2024-09-10T23:40:02.369087+0000","last_became_active":"2024-09-10T23:40:02.364637+0000","last_became_peered":"2024-09-10T23:40:02.364637+0000","last_unstale":"2024-09-10T23:40:02.369087+0000","last_undegraded":"2024-09-10T23:40:02.369087+0000","last_fullsized":"2024-09-10T23:40:02.369087+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:22.601089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344602+0000","last_change":"2024-09-10T23:40:02.344602+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344602+0000","last_undegraded":"2024-09-10T23:40:02.344602+0000","last_fullsized":"2024-09-10T23:40:02.344602+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.366412+0000","last_change":"2024-09-10T23:40:02.361550+0000","last_active":"2024-09-10T23:40:02.366412+0000","last_peered":"2024-09-10T23:40:02.366412+0000","last_clean":"2024-09-10T23:40:02.366412+0000","last_became_active":"2024-09-10T23:40:02.361204+0000","last_became_peered":"2024-09-10T23:40:02.361204+0000","last_unstale":"2024-09-10T23:40:02.366412+0000","last_undegraded":"2024-09-10T23:40:02.366412+0000","last_fullsized":"2024-09-10T23:40:02.366412+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:30.158982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344061+0000","last_change":"2024-09-10T23:40:02.344061+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344061+0000","last_undegraded":"2024-09-10T23:40:02.344061+0000","last_fullsized":"2024-09-10T23:40:02.344061+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359981+0000","last_change":"2024-09-10T23:40:02.354529+0000","last_active":"2024-09-10T23:40:02.359981+0000","last_peered":"2024-09-10T23:40:02.359981+0000","last_clean":"2024-09-10T23:40:02.359981+0000","last_became_active":"2024-09-10T23:40:02.354273+0000","last_became_peered":"2024-09-10T23:40:02.354273+0000","last_unstale":"2024-09-10T23:40:02.359981+0000","last_undegraded":"2024-09-10T23:40:02.359981+0000","last_fullsized":"2024-09-10T23:40:02.359981+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:23:14.459079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.340447+0000","last_change":"2024-09-10T23:40:02.340447+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.340447+0000","last_undegraded":"2024-09-10T23:40:02.340447+0000","last_fullsized":"2024-09-10T23:40:02.340447+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.360107+0000","last_change":"2024-09-10T23:40:02.356793+0000","last_active":"2024-09-10T23:40:02.360107+0000","last_peered":"2024-09-10T23:40:02.360107+0000","last_clean":"2024-09-10T23:40:02.360107+0000","last_became_active":"2024-09-10T23:40:02.356539+0000","last_became_peered":"2024-09-10T23:40:02.356539+0000","last_unstale":"2024-09-10T23:40:02.360107+0000","last_undegraded":"2024-09-10T23:40:02.360107+0000","last_fullsized":"2024-09-10T23:40:02.360107+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:34.484400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342891+0000","last_change":"2024-09-10T23:40:02.342891+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342891+0000","last_undegraded":"2024-09-10T23:40:02.342891+0000","last_fullsized":"2024-09-10T23:40:02.342891+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.366476+0000","last_change":"2024-09-10T23:40:02.363352+0000","last_active":"2024-09-10T23:40:02.366476+0000","last_peered":"2024-09-10T23:40:02.366476+0000","last_clean":"2024-09-10T23:40:02.366476+0000","last_became_active":"2024-09-10T23:40:02.362332+0000","last_became_peered":"2024-09-10T23:40:02.362332+0000","last_unstale":"2024-09-10T23:40:02.366476+0000","last_undegraded":"2024-09-10T23:40:02.366476+0000","last_fullsized":"2024-09-10T23:40:02.366476+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:25.087218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348260+0000","last_change":"2024-09-10T23:40:02.348260+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348260+0000","last_undegraded":"2024-09-10T23:40:02.348260+0000","last_fullsized":"2024-09-10T23:40:02.348260+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359825+0000","last_change":"2024-09-10T23:40:02.357720+0000","last_active":"2024-09-10T23:40:02.359825+0000","last_peered":"2024-09-10T23:40:02.359825+0000","last_clean":"2024-09-10T23:40:02.359825+0000","last_became_active":"2024-09-10T23:40:02.357265+0000","last_became_peered":"2024-09-10T23:40:02.357265+0000","last_unstale":"2024-09-10T23:40:02.359825+0000","last_undegraded":"2024-09-10T23:40:02.359825+0000","last_fullsized":"2024-09-10T23:40:02.359825+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:23:14.459079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344468+0000","last_change":"2024-09-10T23:40:02.344468+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344468+0000","last_undegraded":"2024-09-10T23:40:02.344468+0000","last_fullsized":"2024-09-10T23:40:02.344468+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.369328+0000","last_change":"2024-09-10T23:40:02.367006+0000","last_active":"2024-09-10T23:40:02.369328+0000","last_peered":"2024-09-10T23:40:02.369328+0000","last_clean":"2024-09-10T23:40:02.369328+0000","last_became_active":"2024-09-10T23:40:02.366761+0000","last_became_peered":"2024-09-10T23:40:02.366761+0000","last_unstale":"2024-09-10T23:40:02.369328+0000","last_undegraded":"2024-09-10T23:40:02.369328+0000","last_fullsized":"2024-09-10T23:40:02.369328+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:29.028851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.343271+0000","last_change":"2024-09-10T23:40:02.343271+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.343271+0000","last_undegraded":"2024-09-10T23:40:02.343271+0000","last_fullsized":"2024-09-10T23:40:02.343271+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.352123+0000","last_change":"2024-09-10T23:40:02.350625+0000","last_active":"2024-09-10T23:40:02.352123+0000","last_peered":"2024-09-10T23:40:02.352123+0000","last_clean":"2024-09-10T23:40:02.352123+0000","last_became_active":"2024-09-10T23:40:02.349954+0000","last_became_peered":"2024-09-10T23:40:02.349954+0000","last_unstale":"2024-09-10T23:40:02.352123+0000","last_undegraded":"2024-09-10T23:40:02.352123+0000","last_fullsized":"2024-09-10T23:40:02.352123+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:06.867213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342438+0000","last_change":"2024-09-10T23:40:02.342438+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342438+0000","last_undegraded":"2024-09-10T23:40:02.342438+0000","last_fullsized":"2024-09-10T23:40:02.342438+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359215+0000","last_change":"2024-09-10T23:40:02.355908+0000","last_active":"2024-09-10T23:40:02.359215+0000","last_peered":"2024-09-10T23:40:02.359215+0000","last_clean":"2024-09-10T23:40:02.359215+0000","last_became_active":"2024-09-10T23:40:02.355653+0000","last_became_peered":"2024-09-10T23:40:02.355653+0000","last_unstale":"2024-09-10T23:40:02.359215+0000","last_undegraded":"2024-09-10T23:40:02.359215+0000","last_fullsized":"2024-09-10T23:40:02.359215+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:09.680725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344708+0000","last_change":"2024-09-10T23:40:02.344708+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344708+0000","last_undegraded":"2024-09-10T23:40:02.344708+0000","last_fullsized":"2024-09-10T23:40:02.344708+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.353839+0000","last_change":"2024-09-10T23:40:02.352194+0000","last_active":"2024-09-10T23:40:02.353839+0000","last_peered":"2024-09-10T23:40:02.353839+0000","last_clean":"2024-09-10T23:40:02.353839+0000","last_became_active":"2024-09-10T23:40:02.351903+0000","last_became_peered":"2024-09-10T23:40:02.351903+0000","last_unstale":"2024-09-10T23:40:02.353839+0000","last_undegraded":"2024-09-10T23:40:02.353839+0000","last_fullsized":"2024-09-10T23:40:02.353839+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:29.620254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344780+0000","last_change":"2024-09-10T23:40:02.344780+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344780+0000","last_undegraded":"2024-09-10T23:40:02.344780+0000","last_fullsized":"2024-09-10T23:40:02.344780+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.363830+0000","last_change":"2024-09-10T23:40:02.360470+0000","last_active":"2024-09-10T23:40:02.363830+0000","last_peered":"2024-09-10T23:40:02.363830+0000","last_clean":"2024-09-10T23:40:02.363830+0000","last_became_active":"2024-09-10T23:40:02.360193+0000","last_became_peered":"2024-09-10T23:40:02.360193+0000","last_unstale":"2024-09-10T23:40:02.363830+0000","last_undegraded":"2024-09-10T23:40:02.363830+0000","last_fullsized":"2024-09-10T23:40:02.363830+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:37.483769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.340014+0000","last_change":"2024-09-10T23:40:02.340014+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.340014+0000","last_undegraded":"2024-09-10T23:40:02.340014+0000","last_fullsized":"2024-09-10T23:40:02.340014+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.354285+0000","last_change":"2024-09-10T23:40:02.353010+0000","last_active":"2024-09-10T23:40:02.354285+0000","last_peered":"2024-09-10T23:40:02.354285+0000","last_clean":"2024-09-10T23:40:02.354285+0000","last_became_active":"2024-09-10T23:40:02.352736+0000","last_became_peered":"2024-09-10T23:40:02.352736+0000","last_unstale":"2024-09-10T23:40:02.354285+0000","last_undegraded":"2024-09-10T23:40:02.354285+0000","last_fullsized":"2024-09-10T23:40:02.354285+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:04:05.247046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.341152+0000","last_change":"2024-09-10T23:40:02.341152+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.341152+0000","last_undegraded":"2024-09-10T23:40:02.341152+0000","last_fullsized":"2024-09-10T23:40:02.341152+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.357525+0000","last_change":"2024-09-10T23:40:02.356421+0000","last_active":"2024-09-10T23:40:02.357525+0000","last_peered":"2024-09-10T23:40:02.357525+0000","last_clean":"2024-09-10T23:40:02.357525+0000","last_became_active":"2024-09-10T23:40:02.355527+0000","last_became_peered":"2024-09-10T23:40:02.355527+0000","last_unstale":"2024-09-10T23:40:02.357525+0000","last_undegraded":"2024-09-10T23:40:02.357525+0000","last_fullsized":"2024-09-10T23:40:02.357525+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:30.479559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.345583+0000","last_change":"2024-09-10T23:40:02.345583+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.345583+0000","last_undegraded":"2024-09-10T23:40:02.345583+0000","last_fullsized":"2024-09-10T23:40:02.345583+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.358394+0000","last_change":"2024-09-10T23:40:02.355475+0000","last_active":"2024-09-10T23:40:02.358394+0000","last_peered":"2024-09-10T23:40:02.358394+0000","last_clean":"2024-09-10T23:40:02.358394+0000","last_became_active":"2024-09-10T23:40:02.355204+0000","last_became_peered":"2024-09-10T23:40:02.355204+0000","last_unstale":"2024-09-10T23:40:02.358394+0000","last_undegraded":"2024-09-10T23:40:02.358394+0000","last_fullsized":"2024-09-10T23:40:02.358394+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:07.607518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350633+0000","last_change":"2024-09-10T23:40:02.350633+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350633+0000","last_undegraded":"2024-09-10T23:40:02.350633+0000","last_fullsized":"2024-09-10T23:40:02.350633+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.354125+0000","last_change":"2024-09-10T23:40:02.352054+0000","last_active":"2024-09-10T23:40:02.354125+0000","last_peered":"2024-09-10T23:40:02.354125+0000","last_clean":"2024-09-10T23:40:02.354125+0000","last_became_active":"2024-09-10T23:40:02.351786+0000","last_became_peered":"2024-09-10T23:40:02.351786+0000","last_unstale":"2024-09-10T23:40:02.354125+0000","last_undegraded":"2024-09-10T23:40:02.354125+0000","last_fullsized":"2024-09-10T23:40:02.354125+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:32.578739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350743+0000","last_change":"2024-09-10T23:40:02.350743+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350743+0000","last_undegraded":"2024-09-10T23:40:02.350743+0000","last_fullsized":"2024-09-10T23:40:02.350743+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.366263+0000","last_change":"2024-09-10T23:40:02.360781+0000","last_active":"2024-09-10T23:40:02.366263+0000","last_peered":"2024-09-10T23:40:02.366263+0000","last_clean":"2024-09-10T23:40:02.366263+0000","last_became_active":"2024-09-10T23:40:02.360156+0000","last_became_peered":"2024-09-10T23:40:02.360156+0000","last_unstale":"2024-09-10T23:40:02.366263+0000","last_undegraded":"2024-09-10T23:40:02.366263+0000","last_fullsized":"2024-09-10T23:40:02.366263+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:34.016734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.352067+0000","last_change":"2024-09-10T23:40:02.352067+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.352067+0000","last_undegraded":"2024-09-10T23:40:02.352067+0000","last_fullsized":"2024-09-10T23:40:02.352067+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.363490+0000","last_change":"2024-09-10T23:40:02.359868+0000","last_active":"2024-09-10T23:40:02.363490+0000","last_peered":"2024-09-10T23:40:02.363490+0000","last_clean":"2024-09-10T23:40:02.363490+0000","last_became_active":"2024-09-10T23:40:02.359600+0000","last_became_peered":"2024-09-10T23:40:02.359600+0000","last_unstale":"2024-09-10T23:40:02.363490+0000","last_undegraded":"2024-09-10T23:40:02.363490+0000","last_fullsized":"2024-09-10T23:40:02.363490+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:08.479640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.349167+0000","last_change":"2024-09-10T23:40:02.349167+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.349167+0000","last_undegraded":"2024-09-10T23:40:02.349167+0000","last_fullsized":"2024-09-10T23:40:02.349167+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.366436+0000","last_change":"2024-09-10T23:40:02.362452+0000","last_active":"2024-09-10T23:40:02.366436+0000","last_peered":"2024-09-10T23:40:02.366436+0000","last_clean":"2024-09-10T23:40:02.366436+0000","last_became_active":"2024-09-10T23:40:02.362039+0000","last_became_peered":"2024-09-10T23:40:02.362039+0000","last_unstale":"2024-09-10T23:40:02.366436+0000","last_undegraded":"2024-09-10T23:40:02.366436+0000","last_fullsized":"2024-09-10T23:40:02.366436+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:44.161075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342827+0000","last_change":"2024-09-10T23:40:02.342827+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342827+0000","last_undegraded":"2024-09-10T23:40:02.342827+0000","last_fullsized":"2024-09-10T23:40:02.342827+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359699+0000","last_change":"2024-09-10T23:40:02.356946+0000","last_active":"2024-09-10T23:40:02.359699+0000","last_peered":"2024-09-10T23:40:02.359699+0000","last_clean":"2024-09-10T23:40:02.359699+0000","last_became_active":"2024-09-10T23:40:02.356690+0000","last_became_peered":"2024-09-10T23:40:02.356690+0000","last_unstale":"2024-09-10T23:40:02.359699+0000","last_undegraded":"2024-09-10T23:40:02.359699+0000","last_fullsized":"2024-09-10T23:40:02.359699+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:48.949690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.341411+0000","last_change":"2024-09-10T23:40:02.341411+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.341411+0000","last_undegraded":"2024-09-10T23:40:02.341411+0000","last_fullsized":"2024-09-10T23:40:02.341411+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.355292+0000","last_change":"2024-09-10T23:40:02.349369+0000","last_active":"2024-09-10T23:40:02.355292+0000","last_peered":"2024-09-10T23:40:02.355292+0000","last_clean":"2024-09-10T23:40:02.355292+0000","last_became_active":"2024-09-10T23:40:02.349059+0000","last_became_peered":"2024-09-10T23:40:02.349059+0000","last_unstale":"2024-09-10T23:40:02.355292+0000","last_undegraded":"2024-09-10T23:40:02.355292+0000","last_fullsized":"2024-09-10T23:40:02.355292+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:30.313719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.341593+0000","last_change":"2024-09-10T23:40:02.341593+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.341593+0000","last_undegraded":"2024-09-10T23:40:02.341593+0000","last_fullsized":"2024-09-10T23:40:02.341593+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.354805+0000","last_change":"2024-09-10T23:40:02.349452+0000","last_active":"2024-09-10T23:40:02.354805+0000","last_peered":"2024-09-10T23:40:02.354805+0000","last_clean":"2024-09-10T23:40:02.354805+0000","last_became_active":"2024-09-10T23:40:02.349132+0000","last_became_peered":"2024-09-10T23:40:02.349132+0000","last_unstale":"2024-09-10T23:40:02.354805+0000","last_undegraded":"2024-09-10T23:40:02.354805+0000","last_fullsized":"2024-09-10T23:40:02.354805+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:51:55.558547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.347814+0000","last_change":"2024-09-10T23:40:02.347814+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.347814+0000","last_undegraded":"2024-09-10T23:40:02.347814+0000","last_fullsized":"2024-09-10T23:40:02.347814+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.362251+0000","last_change":"2024-09-10T23:40:02.360113+0000","last_active":"2024-09-10T23:40:02.362251+0000","last_peered":"2024-09-10T23:40:02.362251+0000","last_clean":"2024-09-10T23:40:02.362251+0000","last_became_active":"2024-09-10T23:40:02.359801+0000","last_became_peered":"2024-09-10T23:40:02.359801+0000","last_unstale":"2024-09-10T23:40:02.362251+0000","last_undegraded":"2024-09-10T23:40:02.362251+0000","last_fullsized":"2024-09-10T23:40:02.362251+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:19.215661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.346140+0000","last_change":"2024-09-10T23:40:02.346140+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.346140+0000","last_undegraded":"2024-09-10T23:40:02.346140+0000","last_fullsized":"2024-09-10T23:40:02.346140+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.360040+0000","last_change":"2024-09-10T23:40:02.354814+0000","last_active":"2024-09-10T23:40:02.360040+0000","last_peered":"2024-09-10T23:40:02.360040+0000","last_clean":"2024-09-10T23:40:02.360040+0000","last_became_active":"2024-09-10T23:40:02.354579+0000","last_became_peered":"2024-09-10T23:40:02.354579+0000","last_unstale":"2024-09-10T23:40:02.360040+0000","last_undegraded":"2024-09-10T23:40:02.360040+0000","last_fullsized":"2024-09-10T23:40:02.360040+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:04:11.336551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":371,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.352223+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:40:02.352223+0000","last_peered":"2024-09-10T23:40:02.352223+0000","last_clean":"2024-09-10T23:40:02.352223+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:40:02.352223+0000","last_undegraded":"2024-09-10T23:40:02.352223+0000","last_fullsized":"2024-09-10T23:40:02.352223+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509536,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33004,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338836,"statfs":{"total":96636764160,"available":96602968064,"internally_reserved":0,"allocated":1859584,"data_stored":816974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542239,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33004,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338836,"statfs":{"total":96636764160,"available":96602968064,"internally_reserved":0,"allocated":1859584,"data_stored":816974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509535,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33584,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338256,"statfs":{"total":96636764160,"available":96602374144,"internally_reserved":0,"allocated":2453504,"data_stored":1407342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509533,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1871872,"data_stored":823277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509534,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1871872,"data_stored":823277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509535,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37164,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334676,"statfs":{"total":96636764160,"available":96598708224,"internally_reserved":0,"allocated":1859584,"data_stored":816974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509534,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37744,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334096,"statfs":{"total":96636764160,"available":96598114304,"internally_reserved":0,"allocated":2453504,"data_stored":1407342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509534,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37180,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334660,"statfs":{"total":96636764160,"available":96598691840,"internally_reserved":0,"allocated":1871872,"data_stored":823277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:40:07.897 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:40:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc9a00640 1 -- 172.21.15.94:0/1454999394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc406f600 msgr2=0x7f3bc4072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc9a00640 1 --2- 172.21.15.94:0/1454999394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc406f600 0x7f3bc4072610 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f3bc406ff10 tx=0x7f3bac01ccf0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc9a00640 1 -- 172.21.15.94:0/1454999394 shutdown_connections 2024-09-10T23:40:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc9a00640 1 --2- 172.21.15.94:0/1454999394 >> v2:172.21.15.181:3300/0 conn(0x7f3bc4070150 0x7f3bc40b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc9a00640 1 --2- 172.21.15.94:0/1454999394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc406f600 0x7f3bc4072610 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc9a00640 1 -- 172.21.15.94:0/1454999394 >> 172.21.15.94:0/1454999394 conn(0x7f3bc406cdf0 msgr2=0x7f3bc406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:08.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc9a00640 1 -- 172.21.15.94:0/1454999394 shutdown_connections 2024-09-10T23:40:08.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc9a00640 1 -- 172.21.15.94:0/1454999394 wait complete. 2024-09-10T23:40:08.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc9a00640 1 Processor -- start 2024-09-10T23:40:08.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc9a00640 1 -- start start 2024-09-10T23:40:08.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc406f600 0x7f3bc41f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc9a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bc4070150 0x7f3bc41670e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc9a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc4157c60 0x7f3bc413dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc9a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc40cfeb0 con 0x7f3bc406f600 2024-09-10T23:40:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc9a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc40cfd30 con 0x7f3bc4157c60 2024-09-10T23:40:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc9a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc40d0030 con 0x7f3bc4070150 2024-09-10T23:40:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc406f600 0x7f3bc41f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc406f600 0x7f3bc41f5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34614/0 (socket says 172.21.15.94:34614) 2024-09-10T23:40:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc2a00640 1 -- 172.21.15.94:0/1399700332 learned_addr learned my addr 172.21.15.94:0/1399700332 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc3400640 1 --2- 172.21.15.94:0/1399700332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc4157c60 0x7f3bc413dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bba000640 1 --2- 172.21.15.94:0/1399700332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bc4070150 0x7f3bc41670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc2a00640 1 -- 172.21.15.94:0/1399700332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bc4070150 msgr2=0x7f3bc41670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc2a00640 1 --2- 172.21.15.94:0/1399700332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bc4070150 0x7f3bc41670e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc2a00640 1 -- 172.21.15.94:0/1399700332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc4157c60 msgr2=0x7f3bc413dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc2a00640 1 --2- 172.21.15.94:0/1399700332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc4157c60 0x7f3bc413dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bc2a00640 1 -- 172.21.15.94:0/1399700332 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc4156370 con 0x7f3bc406f600 2024-09-10T23:40:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.075+0000 7f3bba000640 1 --2- 172.21.15.94:0/1399700332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bc4070150 0x7f3bc41670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:40:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.079+0000 7f3bc2a00640 1 --2- 172.21.15.94:0/1399700332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc406f600 0x7f3bc41f5e70 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400c3c0 tx=0x7f3bb400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.079+0000 7f3bbb400640 1 -- 172.21.15.94:0/1399700332 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4017070 con 0x7f3bc406f600 2024-09-10T23:40:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.079+0000 7f3bbb400640 1 -- 172.21.15.94:0/1399700332 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bb40053c0 con 0x7f3bc406f600 2024-09-10T23:40:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.079+0000 7f3bc9a00640 1 -- 172.21.15.94:0/1399700332 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc4140e90 con 0x7f3bc406f600 2024-09-10T23:40:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.079+0000 7f3bbb400640 1 -- 172.21.15.94:0/1399700332 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4003860 con 0x7f3bc406f600 2024-09-10T23:40:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.079+0000 7f3bc9a00640 1 -- 172.21.15.94:0/1399700332 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc4141450 con 0x7f3bc406f600 2024-09-10T23:40:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.079+0000 7f3bc9a00640 1 -- 172.21.15.94:0/1399700332 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc4072610 con 0x7f3bc406f600 2024-09-10T23:40:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.079+0000 7f3bbb400640 1 -- 172.21.15.94:0/1399700332 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f3bb4003a00 con 0x7f3bc406f600 2024-09-10T23:40:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.079+0000 7f3bbb400640 1 --2- 172.21.15.94:0/1399700332 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3ba0072e50 0x7f3ba0075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.079+0000 7f3bbb400640 1 -- 172.21.15.94:0/1399700332 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f3bb4093b50 con 0x7f3bc406f600 2024-09-10T23:40:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.079+0000 7f3bba000640 1 --2- 172.21.15.94:0/1399700332 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3ba0072e50 0x7f3ba0075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.083+0000 7f3bba000640 1 --2- 172.21.15.94:0/1399700332 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3ba0072e50 0x7f3ba0075310 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0af000 tx=0x7f3bac01b530 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:08.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.083+0000 7f3bbb400640 1 -- 172.21.15.94:0/1399700332 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3bc4156370 con 0x7f3bc406f600 2024-09-10T23:40:08.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.247+0000 7f3bc9a00640 1 -- 172.21.15.94:0/1399700332 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3bc4167b40 con 0x7f3ba0072e50 2024-09-10T23:40:08.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.251+0000 7f3bbb400640 1 -- 172.21.15.94:0/1399700332 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247114 (secure 0 0 0) 0x7f3bac0f9560 con 0x7f3ba0072e50 2024-09-10T23:40:08.258 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:08.259 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:40:08.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.255+0000 7f3bc9a00640 1 -- 172.21.15.94:0/1399700332 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3ba0072e50 msgr2=0x7f3ba0075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:08.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.255+0000 7f3bc9a00640 1 --2- 172.21.15.94:0/1399700332 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3ba0072e50 0x7f3ba0075310 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0af000 tx=0x7f3bac01b530 comp rx=0 tx=0).stop 2024-09-10T23:40:08.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.255+0000 7f3bc9a00640 1 -- 172.21.15.94:0/1399700332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc406f600 msgr2=0x7f3bc41f5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:08.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.255+0000 7f3bc9a00640 1 --2- 172.21.15.94:0/1399700332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc406f600 0x7f3bc41f5e70 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400c3c0 tx=0x7f3bb400c7c0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.255+0000 7f3bc9a00640 1 -- 172.21.15.94:0/1399700332 shutdown_connections 2024-09-10T23:40:08.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.255+0000 7f3bc9a00640 1 --2- 172.21.15.94:0/1399700332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc4157c60 0x7f3bc413dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.255+0000 7f3bc9a00640 1 --2- 172.21.15.94:0/1399700332 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3ba0072e50 0x7f3ba0075310 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.255+0000 7f3bc9a00640 1 --2- 172.21.15.94:0/1399700332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bc4070150 0x7f3bc41670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.255+0000 7f3bc9a00640 1 --2- 172.21.15.94:0/1399700332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc406f600 0x7f3bc41f5e70 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.255+0000 7f3bc9a00640 1 -- 172.21.15.94:0/1399700332 >> 172.21.15.94:0/1399700332 conn(0x7f3bc406cdf0 msgr2=0x7f3bc4156ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:08.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.255+0000 7f3bc9a00640 1 -- 172.21.15.94:0/1399700332 shutdown_connections 2024-09-10T23:40:08.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.255+0000 7f3bc9a00640 1 -- 172.21.15.94:0/1399700332 wait complete. 2024-09-10T23:40:08.279 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-10T23:40:06.778391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":277720,"kb_used_data":15724,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754697000,"statfs":{"total":773094113280,"available":772809728000,"internally_reserved":0,"allocated":16101376,"data_stored":7735437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.453634"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342043+0000","last_change":"2024-09-10T23:40:02.342043+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342043+0000","last_undegraded":"2024-09-10T23:40:02.342043+0000","last_fullsized":"2024-09-10T23:40:02.342043+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344703+0000","last_change":"2024-09-10T23:40:02.344703+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344703+0000","last_undegraded":"2024-09-10T23:40:02.344703+0000","last_fullsized":"2024-09-10T23:40:02.344703+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.353593+0000","last_change":"2024-09-10T23:40:02.353593+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.353593+0000","last_undegraded":"2024-09-10T23:40:02.353593+0000","last_fullsized":"2024-09-10T23:40:02.353593+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.343368+0000","last_change":"2024-09-10T23:40:02.343368+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.343368+0000","last_undegraded":"2024-09-10T23:40:02.343368+0000","last_fullsized":"2024-09-10T23:40:02.343368+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.351849+0000","last_change":"2024-09-10T23:40:02.351849+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.351849+0000","last_undegraded":"2024-09-10T23:40:02.351849+0000","last_fullsized":"2024-09-10T23:40:02.351849+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.345840+0000","last_change":"2024-09-10T23:40:02.345840+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.345840+0000","last_undegraded":"2024-09-10T23:40:02.345840+0000","last_fullsized":"2024-09-10T23:40:02.345840+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.339971+0000","last_change":"2024-09-10T23:40:02.339971+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.339971+0000","last_undegraded":"2024-09-10T23:40:02.339971+0000","last_fullsized":"2024-09-10T23:40:02.339971+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350386+0000","last_change":"2024-09-10T23:40:02.350386+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350386+0000","last_undegraded":"2024-09-10T23:40:02.350386+0000","last_fullsized":"2024-09-10T23:40:02.350386+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348677+0000","last_change":"2024-09-10T23:40:02.348677+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348677+0000","last_undegraded":"2024-09-10T23:40:02.348677+0000","last_fullsized":"2024-09-10T23:40:02.348677+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348260+0000","last_change":"2024-09-10T23:40:02.348260+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348260+0000","last_undegraded":"2024-09-10T23:40:02.348260+0000","last_fullsized":"2024-09-10T23:40:02.348260+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342470+0000","last_change":"2024-09-10T23:40:02.342470+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342470+0000","last_undegraded":"2024-09-10T23:40:02.342470+0000","last_fullsized":"2024-09-10T23:40:02.342470+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344421+0000","last_change":"2024-09-10T23:40:02.344421+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344421+0000","last_undegraded":"2024-09-10T23:40:02.344421+0000","last_fullsized":"2024-09-10T23:40:02.344421+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.341773+0000","last_change":"2024-09-10T23:40:02.341773+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.341773+0000","last_undegraded":"2024-09-10T23:40:02.341773+0000","last_fullsized":"2024-09-10T23:40:02.341773+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.343249+0000","last_change":"2024-09-10T23:40:02.343249+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.343249+0000","last_undegraded":"2024-09-10T23:40:02.343249+0000","last_fullsized":"2024-09-10T23:40:02.343249+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.343783+0000","last_change":"2024-09-10T23:40:02.343783+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.343783+0000","last_undegraded":"2024-09-10T23:40:02.343783+0000","last_fullsized":"2024-09-10T23:40:02.343783+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350869+0000","last_change":"2024-09-10T23:40:02.350869+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350869+0000","last_undegraded":"2024-09-10T23:40:02.350869+0000","last_fullsized":"2024-09-10T23:40:02.350869+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.347466+0000","last_change":"2024-09-10T23:40:02.347466+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.347466+0000","last_undegraded":"2024-09-10T23:40:02.347466+0000","last_fullsized":"2024-09-10T23:40:02.347466+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.340509+0000","last_change":"2024-09-10T23:40:02.340509+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.340509+0000","last_undegraded":"2024-09-10T23:40:02.340509+0000","last_fullsized":"2024-09-10T23:40:02.340509+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.353158+0000","last_change":"2024-09-10T23:40:02.353158+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.353158+0000","last_undegraded":"2024-09-10T23:40:02.353158+0000","last_fullsized":"2024-09-10T23:40:02.353158+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348678+0000","last_change":"2024-09-10T23:40:02.348678+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348678+0000","last_undegraded":"2024-09-10T23:40:02.348678+0000","last_fullsized":"2024-09-10T23:40:02.348678+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.351641+0000","last_change":"2024-09-10T23:40:02.351641+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.351641+0000","last_undegraded":"2024-09-10T23:40:02.351641+0000","last_fullsized":"2024-09-10T23:40:02.351641+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350410+0000","last_change":"2024-09-10T23:40:02.350410+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350410+0000","last_undegraded":"2024-09-10T23:40:02.350410+0000","last_fullsized":"2024-09-10T23:40:02.350410+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.345990+0000","last_change":"2024-09-10T23:40:02.345990+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.345990+0000","last_undegraded":"2024-09-10T23:40:02.345990+0000","last_fullsized":"2024-09-10T23:40:02.345990+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.349962+0000","last_change":"2024-09-10T23:40:02.349962+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.349962+0000","last_undegraded":"2024-09-10T23:40:02.349962+0000","last_fullsized":"2024-09-10T23:40:02.349962+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348253+0000","last_change":"2024-09-10T23:40:02.348253+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348253+0000","last_undegraded":"2024-09-10T23:40:02.348253+0000","last_fullsized":"2024-09-10T23:40:02.348253+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348873+0000","last_change":"2024-09-10T23:40:02.348873+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348873+0000","last_undegraded":"2024-09-10T23:40:02.348873+0000","last_fullsized":"2024-09-10T23:40:02.348873+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.352668+0000","last_change":"2024-09-10T23:40:02.352668+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.352668+0000","last_undegraded":"2024-09-10T23:40:02.352668+0000","last_fullsized":"2024-09-10T23:40:02.352668+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344673+0000","last_change":"2024-09-10T23:40:02.344673+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344673+0000","last_undegraded":"2024-09-10T23:40:02.344673+0000","last_fullsized":"2024-09-10T23:40:02.344673+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.343596+0000","last_change":"2024-09-10T23:40:02.343596+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.343596+0000","last_undegraded":"2024-09-10T23:40:02.343596+0000","last_fullsized":"2024-09-10T23:40:02.343596+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.346547+0000","last_change":"2024-09-10T23:40:02.346547+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.346547+0000","last_undegraded":"2024-09-10T23:40:02.346547+0000","last_fullsized":"2024-09-10T23:40:02.346547+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350824+0000","last_change":"2024-09-10T23:40:02.350824+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350824+0000","last_undegraded":"2024-09-10T23:40:02.350824+0000","last_fullsized":"2024-09-10T23:40:02.350824+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348261+0000","last_change":"2024-09-10T23:40:02.348261+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348261+0000","last_undegraded":"2024-09-10T23:40:02.348261+0000","last_fullsized":"2024-09-10T23:40:02.348261+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.349611+0000","last_change":"2024-09-10T23:40:02.349611+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.349611+0000","last_undegraded":"2024-09-10T23:40:02.349611+0000","last_fullsized":"2024-09-10T23:40:02.349611+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359443+0000","last_change":"2024-09-10T23:40:02.357339+0000","last_active":"2024-09-10T23:40:02.359443+0000","last_peered":"2024-09-10T23:40:02.359443+0000","last_clean":"2024-09-10T23:40:02.359443+0000","last_became_active":"2024-09-10T23:40:02.357054+0000","last_became_peered":"2024-09-10T23:40:02.357054+0000","last_unstale":"2024-09-10T23:40:02.359443+0000","last_undegraded":"2024-09-10T23:40:02.359443+0000","last_fullsized":"2024-09-10T23:40:02.359443+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:21.460711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.345207+0000","last_change":"2024-09-10T23:40:02.345207+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.345207+0000","last_undegraded":"2024-09-10T23:40:02.345207+0000","last_fullsized":"2024-09-10T23:40:02.345207+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.358117+0000","last_change":"2024-09-10T23:40:02.354357+0000","last_active":"2024-09-10T23:40:02.358117+0000","last_peered":"2024-09-10T23:40:02.358117+0000","last_clean":"2024-09-10T23:40:02.358117+0000","last_became_active":"2024-09-10T23:40:02.354105+0000","last_became_peered":"2024-09-10T23:40:02.354105+0000","last_unstale":"2024-09-10T23:40:02.358117+0000","last_undegraded":"2024-09-10T23:40:02.358117+0000","last_fullsized":"2024-09-10T23:40:02.358117+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:34.016734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342932+0000","last_change":"2024-09-10T23:40:02.342932+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342932+0000","last_undegraded":"2024-09-10T23:40:02.342932+0000","last_fullsized":"2024-09-10T23:40:02.342932+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.369058+0000","last_change":"2024-09-10T23:40:02.363918+0000","last_active":"2024-09-10T23:40:02.369058+0000","last_peered":"2024-09-10T23:40:02.369058+0000","last_clean":"2024-09-10T23:40:02.369058+0000","last_became_active":"2024-09-10T23:40:02.363627+0000","last_became_peered":"2024-09-10T23:40:02.363627+0000","last_unstale":"2024-09-10T23:40:02.369058+0000","last_undegraded":"2024-09-10T23:40:02.369058+0000","last_fullsized":"2024-09-10T23:40:02.369058+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:14.649948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342179+0000","last_change":"2024-09-10T23:40:02.342179+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342179+0000","last_undegraded":"2024-09-10T23:40:02.342179+0000","last_fullsized":"2024-09-10T23:40:02.342179+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359942+0000","last_change":"2024-09-10T23:40:02.354874+0000","last_active":"2024-09-10T23:40:02.359942+0000","last_peered":"2024-09-10T23:40:02.359942+0000","last_clean":"2024-09-10T23:40:02.359942+0000","last_became_active":"2024-09-10T23:40:02.354568+0000","last_became_peered":"2024-09-10T23:40:02.354568+0000","last_unstale":"2024-09-10T23:40:02.359942+0000","last_undegraded":"2024-09-10T23:40:02.359942+0000","last_fullsized":"2024-09-10T23:40:02.359942+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:19.346732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.345401+0000","last_change":"2024-09-10T23:40:02.345401+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.345401+0000","last_undegraded":"2024-09-10T23:40:02.345401+0000","last_fullsized":"2024-09-10T23:40:02.345401+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.369026+0000","last_change":"2024-09-10T23:40:02.363569+0000","last_active":"2024-09-10T23:40:02.369026+0000","last_peered":"2024-09-10T23:40:02.369026+0000","last_clean":"2024-09-10T23:40:02.369026+0000","last_became_active":"2024-09-10T23:40:02.362386+0000","last_became_peered":"2024-09-10T23:40:02.362386+0000","last_unstale":"2024-09-10T23:40:02.369026+0000","last_undegraded":"2024-09-10T23:40:02.369026+0000","last_fullsized":"2024-09-10T23:40:02.369026+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:44.359631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.349639+0000","last_change":"2024-09-10T23:40:02.349639+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.349639+0000","last_undegraded":"2024-09-10T23:40:02.349639+0000","last_fullsized":"2024-09-10T23:40:02.349639+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.369067+0000","last_change":"2024-09-10T23:40:02.363592+0000","last_active":"2024-09-10T23:40:02.369067+0000","last_peered":"2024-09-10T23:40:02.369067+0000","last_clean":"2024-09-10T23:40:02.369067+0000","last_became_active":"2024-09-10T23:40:02.363294+0000","last_became_peered":"2024-09-10T23:40:02.363294+0000","last_unstale":"2024-09-10T23:40:02.369067+0000","last_undegraded":"2024-09-10T23:40:02.369067+0000","last_fullsized":"2024-09-10T23:40:02.369067+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:23.166700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.340809+0000","last_change":"2024-09-10T23:40:02.340809+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.340809+0000","last_undegraded":"2024-09-10T23:40:02.340809+0000","last_fullsized":"2024-09-10T23:40:02.340809+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359180+0000","last_change":"2024-09-10T23:40:02.353255+0000","last_active":"2024-09-10T23:40:02.359180+0000","last_peered":"2024-09-10T23:40:02.359180+0000","last_clean":"2024-09-10T23:40:02.359180+0000","last_became_active":"2024-09-10T23:40:02.352992+0000","last_became_peered":"2024-09-10T23:40:02.352992+0000","last_unstale":"2024-09-10T23:40:02.359180+0000","last_undegraded":"2024-09-10T23:40:02.359180+0000","last_fullsized":"2024-09-10T23:40:02.359180+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:19.346732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.353296+0000","last_change":"2024-09-10T23:40:02.353296+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.353296+0000","last_undegraded":"2024-09-10T23:40:02.353296+0000","last_fullsized":"2024-09-10T23:40:02.353296+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.360297+0000","last_change":"2024-09-10T23:40:02.359051+0000","last_active":"2024-09-10T23:40:02.360297+0000","last_peered":"2024-09-10T23:40:02.360297+0000","last_clean":"2024-09-10T23:40:02.360297+0000","last_became_active":"2024-09-10T23:40:02.357335+0000","last_became_peered":"2024-09-10T23:40:02.357335+0000","last_unstale":"2024-09-10T23:40:02.360297+0000","last_undegraded":"2024-09-10T23:40:02.360297+0000","last_fullsized":"2024-09-10T23:40:02.360297+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:07.378540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344844+0000","last_change":"2024-09-10T23:40:02.344844+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344844+0000","last_undegraded":"2024-09-10T23:40:02.344844+0000","last_fullsized":"2024-09-10T23:40:02.344844+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.362075+0000","last_change":"2024-09-10T23:40:02.359834+0000","last_active":"2024-09-10T23:40:02.362075+0000","last_peered":"2024-09-10T23:40:02.362075+0000","last_clean":"2024-09-10T23:40:02.362075+0000","last_became_active":"2024-09-10T23:40:02.359528+0000","last_became_peered":"2024-09-10T23:40:02.359528+0000","last_unstale":"2024-09-10T23:40:02.362075+0000","last_undegraded":"2024-09-10T23:40:02.362075+0000","last_fullsized":"2024-09-10T23:40:02.362075+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:53.213052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344044+0000","last_change":"2024-09-10T23:40:02.344044+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344044+0000","last_undegraded":"2024-09-10T23:40:02.344044+0000","last_fullsized":"2024-09-10T23:40:02.344044+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.369087+0000","last_change":"2024-09-10T23:40:02.364953+0000","last_active":"2024-09-10T23:40:02.369087+0000","last_peered":"2024-09-10T23:40:02.369087+0000","last_clean":"2024-09-10T23:40:02.369087+0000","last_became_active":"2024-09-10T23:40:02.364637+0000","last_became_peered":"2024-09-10T23:40:02.364637+0000","last_unstale":"2024-09-10T23:40:02.369087+0000","last_undegraded":"2024-09-10T23:40:02.369087+0000","last_fullsized":"2024-09-10T23:40:02.369087+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:22.601089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344602+0000","last_change":"2024-09-10T23:40:02.344602+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344602+0000","last_undegraded":"2024-09-10T23:40:02.344602+0000","last_fullsized":"2024-09-10T23:40:02.344602+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.366412+0000","last_change":"2024-09-10T23:40:02.361550+0000","last_active":"2024-09-10T23:40:02.366412+0000","last_peered":"2024-09-10T23:40:02.366412+0000","last_clean":"2024-09-10T23:40:02.366412+0000","last_became_active":"2024-09-10T23:40:02.361204+0000","last_became_peered":"2024-09-10T23:40:02.361204+0000","last_unstale":"2024-09-10T23:40:02.366412+0000","last_undegraded":"2024-09-10T23:40:02.366412+0000","last_fullsized":"2024-09-10T23:40:02.366412+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:30.158982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344061+0000","last_change":"2024-09-10T23:40:02.344061+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344061+0000","last_undegraded":"2024-09-10T23:40:02.344061+0000","last_fullsized":"2024-09-10T23:40:02.344061+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359981+0000","last_change":"2024-09-10T23:40:02.354529+0000","last_active":"2024-09-10T23:40:02.359981+0000","last_peered":"2024-09-10T23:40:02.359981+0000","last_clean":"2024-09-10T23:40:02.359981+0000","last_became_active":"2024-09-10T23:40:02.354273+0000","last_became_peered":"2024-09-10T23:40:02.354273+0000","last_unstale":"2024-09-10T23:40:02.359981+0000","last_undegraded":"2024-09-10T23:40:02.359981+0000","last_fullsized":"2024-09-10T23:40:02.359981+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:23:14.459079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.340447+0000","last_change":"2024-09-10T23:40:02.340447+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.340447+0000","last_undegraded":"2024-09-10T23:40:02.340447+0000","last_fullsized":"2024-09-10T23:40:02.340447+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.360107+0000","last_change":"2024-09-10T23:40:02.356793+0000","last_active":"2024-09-10T23:40:02.360107+0000","last_peered":"2024-09-10T23:40:02.360107+0000","last_clean":"2024-09-10T23:40:02.360107+0000","last_became_active":"2024-09-10T23:40:02.356539+0000","last_became_peered":"2024-09-10T23:40:02.356539+0000","last_unstale":"2024-09-10T23:40:02.360107+0000","last_undegraded":"2024-09-10T23:40:02.360107+0000","last_fullsized":"2024-09-10T23:40:02.360107+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:34.484400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342891+0000","last_change":"2024-09-10T23:40:02.342891+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342891+0000","last_undegraded":"2024-09-10T23:40:02.342891+0000","last_fullsized":"2024-09-10T23:40:02.342891+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.366476+0000","last_change":"2024-09-10T23:40:02.363352+0000","last_active":"2024-09-10T23:40:02.366476+0000","last_peered":"2024-09-10T23:40:02.366476+0000","last_clean":"2024-09-10T23:40:02.366476+0000","last_became_active":"2024-09-10T23:40:02.362332+0000","last_became_peered":"2024-09-10T23:40:02.362332+0000","last_unstale":"2024-09-10T23:40:02.366476+0000","last_undegraded":"2024-09-10T23:40:02.366476+0000","last_fullsized":"2024-09-10T23:40:02.366476+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:25.087218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.348260+0000","last_change":"2024-09-10T23:40:02.348260+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.348260+0000","last_undegraded":"2024-09-10T23:40:02.348260+0000","last_fullsized":"2024-09-10T23:40:02.348260+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359825+0000","last_change":"2024-09-10T23:40:02.357720+0000","last_active":"2024-09-10T23:40:02.359825+0000","last_peered":"2024-09-10T23:40:02.359825+0000","last_clean":"2024-09-10T23:40:02.359825+0000","last_became_active":"2024-09-10T23:40:02.357265+0000","last_became_peered":"2024-09-10T23:40:02.357265+0000","last_unstale":"2024-09-10T23:40:02.359825+0000","last_undegraded":"2024-09-10T23:40:02.359825+0000","last_fullsized":"2024-09-10T23:40:02.359825+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:23:14.459079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344468+0000","last_change":"2024-09-10T23:40:02.344468+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344468+0000","last_undegraded":"2024-09-10T23:40:02.344468+0000","last_fullsized":"2024-09-10T23:40:02.344468+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.369328+0000","last_change":"2024-09-10T23:40:02.367006+0000","last_active":"2024-09-10T23:40:02.369328+0000","last_peered":"2024-09-10T23:40:02.369328+0000","last_clean":"2024-09-10T23:40:02.369328+0000","last_became_active":"2024-09-10T23:40:02.366761+0000","last_became_peered":"2024-09-10T23:40:02.366761+0000","last_unstale":"2024-09-10T23:40:02.369328+0000","last_undegraded":"2024-09-10T23:40:02.369328+0000","last_fullsized":"2024-09-10T23:40:02.369328+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:29.028851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.343271+0000","last_change":"2024-09-10T23:40:02.343271+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.343271+0000","last_undegraded":"2024-09-10T23:40:02.343271+0000","last_fullsized":"2024-09-10T23:40:02.343271+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.352123+0000","last_change":"2024-09-10T23:40:02.350625+0000","last_active":"2024-09-10T23:40:02.352123+0000","last_peered":"2024-09-10T23:40:02.352123+0000","last_clean":"2024-09-10T23:40:02.352123+0000","last_became_active":"2024-09-10T23:40:02.349954+0000","last_became_peered":"2024-09-10T23:40:02.349954+0000","last_unstale":"2024-09-10T23:40:02.352123+0000","last_undegraded":"2024-09-10T23:40:02.352123+0000","last_fullsized":"2024-09-10T23:40:02.352123+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:06.867213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342438+0000","last_change":"2024-09-10T23:40:02.342438+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342438+0000","last_undegraded":"2024-09-10T23:40:02.342438+0000","last_fullsized":"2024-09-10T23:40:02.342438+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359215+0000","last_change":"2024-09-10T23:40:02.355908+0000","last_active":"2024-09-10T23:40:02.359215+0000","last_peered":"2024-09-10T23:40:02.359215+0000","last_clean":"2024-09-10T23:40:02.359215+0000","last_became_active":"2024-09-10T23:40:02.355653+0000","last_became_peered":"2024-09-10T23:40:02.355653+0000","last_unstale":"2024-09-10T23:40:02.359215+0000","last_undegraded":"2024-09-10T23:40:02.359215+0000","last_fullsized":"2024-09-10T23:40:02.359215+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:09.680725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344708+0000","last_change":"2024-09-10T23:40:02.344708+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344708+0000","last_undegraded":"2024-09-10T23:40:02.344708+0000","last_fullsized":"2024-09-10T23:40:02.344708+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.353839+0000","last_change":"2024-09-10T23:40:02.352194+0000","last_active":"2024-09-10T23:40:02.353839+0000","last_peered":"2024-09-10T23:40:02.353839+0000","last_clean":"2024-09-10T23:40:02.353839+0000","last_became_active":"2024-09-10T23:40:02.351903+0000","last_became_peered":"2024-09-10T23:40:02.351903+0000","last_unstale":"2024-09-10T23:40:02.353839+0000","last_undegraded":"2024-09-10T23:40:02.353839+0000","last_fullsized":"2024-09-10T23:40:02.353839+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:29.620254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.344780+0000","last_change":"2024-09-10T23:40:02.344780+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.344780+0000","last_undegraded":"2024-09-10T23:40:02.344780+0000","last_fullsized":"2024-09-10T23:40:02.344780+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.363830+0000","last_change":"2024-09-10T23:40:02.360470+0000","last_active":"2024-09-10T23:40:02.363830+0000","last_peered":"2024-09-10T23:40:02.363830+0000","last_clean":"2024-09-10T23:40:02.363830+0000","last_became_active":"2024-09-10T23:40:02.360193+0000","last_became_peered":"2024-09-10T23:40:02.360193+0000","last_unstale":"2024-09-10T23:40:02.363830+0000","last_undegraded":"2024-09-10T23:40:02.363830+0000","last_fullsized":"2024-09-10T23:40:02.363830+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:37.483769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.340014+0000","last_change":"2024-09-10T23:40:02.340014+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.340014+0000","last_undegraded":"2024-09-10T23:40:02.340014+0000","last_fullsized":"2024-09-10T23:40:02.340014+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.354285+0000","last_change":"2024-09-10T23:40:02.353010+0000","last_active":"2024-09-10T23:40:02.354285+0000","last_peered":"2024-09-10T23:40:02.354285+0000","last_clean":"2024-09-10T23:40:02.354285+0000","last_became_active":"2024-09-10T23:40:02.352736+0000","last_became_peered":"2024-09-10T23:40:02.352736+0000","last_unstale":"2024-09-10T23:40:02.354285+0000","last_undegraded":"2024-09-10T23:40:02.354285+0000","last_fullsized":"2024-09-10T23:40:02.354285+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:04:05.247046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.341152+0000","last_change":"2024-09-10T23:40:02.341152+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.341152+0000","last_undegraded":"2024-09-10T23:40:02.341152+0000","last_fullsized":"2024-09-10T23:40:02.341152+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.357525+0000","last_change":"2024-09-10T23:40:02.356421+0000","last_active":"2024-09-10T23:40:02.357525+0000","last_peered":"2024-09-10T23:40:02.357525+0000","last_clean":"2024-09-10T23:40:02.357525+0000","last_became_active":"2024-09-10T23:40:02.355527+0000","last_became_peered":"2024-09-10T23:40:02.355527+0000","last_unstale":"2024-09-10T23:40:02.357525+0000","last_undegraded":"2024-09-10T23:40:02.357525+0000","last_fullsized":"2024-09-10T23:40:02.357525+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:30.479559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.345583+0000","last_change":"2024-09-10T23:40:02.345583+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.345583+0000","last_undegraded":"2024-09-10T23:40:02.345583+0000","last_fullsized":"2024-09-10T23:40:02.345583+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.358394+0000","last_change":"2024-09-10T23:40:02.355475+0000","last_active":"2024-09-10T23:40:02.358394+0000","last_peered":"2024-09-10T23:40:02.358394+0000","last_clean":"2024-09-10T23:40:02.358394+0000","last_became_active":"2024-09-10T23:40:02.355204+0000","last_became_peered":"2024-09-10T23:40:02.355204+0000","last_unstale":"2024-09-10T23:40:02.358394+0000","last_undegraded":"2024-09-10T23:40:02.358394+0000","last_fullsized":"2024-09-10T23:40:02.358394+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:07.607518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350633+0000","last_change":"2024-09-10T23:40:02.350633+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350633+0000","last_undegraded":"2024-09-10T23:40:02.350633+0000","last_fullsized":"2024-09-10T23:40:02.350633+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.354125+0000","last_change":"2024-09-10T23:40:02.352054+0000","last_active":"2024-09-10T23:40:02.354125+0000","last_peered":"2024-09-10T23:40:02.354125+0000","last_clean":"2024-09-10T23:40:02.354125+0000","last_became_active":"2024-09-10T23:40:02.351786+0000","last_became_peered":"2024-09-10T23:40:02.351786+0000","last_unstale":"2024-09-10T23:40:02.354125+0000","last_undegraded":"2024-09-10T23:40:02.354125+0000","last_fullsized":"2024-09-10T23:40:02.354125+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:32.578739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.350743+0000","last_change":"2024-09-10T23:40:02.350743+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.350743+0000","last_undegraded":"2024-09-10T23:40:02.350743+0000","last_fullsized":"2024-09-10T23:40:02.350743+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.366263+0000","last_change":"2024-09-10T23:40:02.360781+0000","last_active":"2024-09-10T23:40:02.366263+0000","last_peered":"2024-09-10T23:40:02.366263+0000","last_clean":"2024-09-10T23:40:02.366263+0000","last_became_active":"2024-09-10T23:40:02.360156+0000","last_became_peered":"2024-09-10T23:40:02.360156+0000","last_unstale":"2024-09-10T23:40:02.366263+0000","last_undegraded":"2024-09-10T23:40:02.366263+0000","last_fullsized":"2024-09-10T23:40:02.366263+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:34.016734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.352067+0000","last_change":"2024-09-10T23:40:02.352067+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.352067+0000","last_undegraded":"2024-09-10T23:40:02.352067+0000","last_fullsized":"2024-09-10T23:40:02.352067+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.363490+0000","last_change":"2024-09-10T23:40:02.359868+0000","last_active":"2024-09-10T23:40:02.363490+0000","last_peered":"2024-09-10T23:40:02.363490+0000","last_clean":"2024-09-10T23:40:02.363490+0000","last_became_active":"2024-09-10T23:40:02.359600+0000","last_became_peered":"2024-09-10T23:40:02.359600+0000","last_unstale":"2024-09-10T23:40:02.363490+0000","last_undegraded":"2024-09-10T23:40:02.363490+0000","last_fullsized":"2024-09-10T23:40:02.363490+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:08.479640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.349167+0000","last_change":"2024-09-10T23:40:02.349167+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.349167+0000","last_undegraded":"2024-09-10T23:40:02.349167+0000","last_fullsized":"2024-09-10T23:40:02.349167+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.366436+0000","last_change":"2024-09-10T23:40:02.362452+0000","last_active":"2024-09-10T23:40:02.366436+0000","last_peered":"2024-09-10T23:40:02.366436+0000","last_clean":"2024-09-10T23:40:02.366436+0000","last_became_active":"2024-09-10T23:40:02.362039+0000","last_became_peered":"2024-09-10T23:40:02.362039+0000","last_unstale":"2024-09-10T23:40:02.366436+0000","last_undegraded":"2024-09-10T23:40:02.366436+0000","last_fullsized":"2024-09-10T23:40:02.366436+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:44.161075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.342827+0000","last_change":"2024-09-10T23:40:02.342827+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.342827+0000","last_undegraded":"2024-09-10T23:40:02.342827+0000","last_fullsized":"2024-09-10T23:40:02.342827+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.359699+0000","last_change":"2024-09-10T23:40:02.356946+0000","last_active":"2024-09-10T23:40:02.359699+0000","last_peered":"2024-09-10T23:40:02.359699+0000","last_clean":"2024-09-10T23:40:02.359699+0000","last_became_active":"2024-09-10T23:40:02.356690+0000","last_became_peered":"2024-09-10T23:40:02.356690+0000","last_unstale":"2024-09-10T23:40:02.359699+0000","last_undegraded":"2024-09-10T23:40:02.359699+0000","last_fullsized":"2024-09-10T23:40:02.359699+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:48.949690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.341411+0000","last_change":"2024-09-10T23:40:02.341411+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.341411+0000","last_undegraded":"2024-09-10T23:40:02.341411+0000","last_fullsized":"2024-09-10T23:40:02.341411+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.355292+0000","last_change":"2024-09-10T23:40:02.349369+0000","last_active":"2024-09-10T23:40:02.355292+0000","last_peered":"2024-09-10T23:40:02.355292+0000","last_clean":"2024-09-10T23:40:02.355292+0000","last_became_active":"2024-09-10T23:40:02.349059+0000","last_became_peered":"2024-09-10T23:40:02.349059+0000","last_unstale":"2024-09-10T23:40:02.355292+0000","last_undegraded":"2024-09-10T23:40:02.355292+0000","last_fullsized":"2024-09-10T23:40:02.355292+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:30.313719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.341593+0000","last_change":"2024-09-10T23:40:02.341593+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.341593+0000","last_undegraded":"2024-09-10T23:40:02.341593+0000","last_fullsized":"2024-09-10T23:40:02.341593+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.354805+0000","last_change":"2024-09-10T23:40:02.349452+0000","last_active":"2024-09-10T23:40:02.354805+0000","last_peered":"2024-09-10T23:40:02.354805+0000","last_clean":"2024-09-10T23:40:02.354805+0000","last_became_active":"2024-09-10T23:40:02.349132+0000","last_became_peered":"2024-09-10T23:40:02.349132+0000","last_unstale":"2024-09-10T23:40:02.354805+0000","last_undegraded":"2024-09-10T23:40:02.354805+0000","last_fullsized":"2024-09-10T23:40:02.354805+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:51:55.558547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.347814+0000","last_change":"2024-09-10T23:40:02.347814+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.347814+0000","last_undegraded":"2024-09-10T23:40:02.347814+0000","last_fullsized":"2024-09-10T23:40:02.347814+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.362251+0000","last_change":"2024-09-10T23:40:02.360113+0000","last_active":"2024-09-10T23:40:02.362251+0000","last_peered":"2024-09-10T23:40:02.362251+0000","last_clean":"2024-09-10T23:40:02.362251+0000","last_became_active":"2024-09-10T23:40:02.359801+0000","last_became_peered":"2024-09-10T23:40:02.359801+0000","last_unstale":"2024-09-10T23:40:02.362251+0000","last_undegraded":"2024-09-10T23:40:02.362251+0000","last_fullsized":"2024-09-10T23:40:02.362251+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:19.215661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-10T23:40:02.346140+0000","last_change":"2024-09-10T23:40:02.346140+0000","last_active":"2024-09-10T23:40:02.324338+0000","last_peered":"2024-09-10T23:40:02.324338+0000","last_clean":"2024-09-10T23:40:02.324338+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T23:40:02.346140+0000","last_undegraded":"2024-09-10T23:40:02.346140+0000","last_fullsized":"2024-09-10T23:40:02.346140+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.360040+0000","last_change":"2024-09-10T23:40:02.354814+0000","last_active":"2024-09-10T23:40:02.360040+0000","last_peered":"2024-09-10T23:40:02.360040+0000","last_clean":"2024-09-10T23:40:02.360040+0000","last_became_active":"2024-09-10T23:40:02.354579+0000","last_became_peered":"2024-09-10T23:40:02.354579+0000","last_unstale":"2024-09-10T23:40:02.360040+0000","last_undegraded":"2024-09-10T23:40:02.360040+0000","last_fullsized":"2024-09-10T23:40:02.360040+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:04:11.336551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":371,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-10T23:40:02.352223+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:40:02.352223+0000","last_peered":"2024-09-10T23:40:02.352223+0000","last_clean":"2024-09-10T23:40:02.352223+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:40:02.352223+0000","last_undegraded":"2024-09-10T23:40:02.352223+0000","last_fullsized":"2024-09-10T23:40:02.352223+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509536,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33004,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338836,"statfs":{"total":96636764160,"available":96602968064,"internally_reserved":0,"allocated":1859584,"data_stored":816974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542239,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33004,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338836,"statfs":{"total":96636764160,"available":96602968064,"internally_reserved":0,"allocated":1859584,"data_stored":816974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509535,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33584,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338256,"statfs":{"total":96636764160,"available":96602374144,"internally_reserved":0,"allocated":2453504,"data_stored":1407342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509533,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1871872,"data_stored":823277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509534,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1871872,"data_stored":823277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509535,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37164,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334676,"statfs":{"total":96636764160,"available":96598708224,"internally_reserved":0,"allocated":1859584,"data_stored":816974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509534,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37744,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334096,"statfs":{"total":96636764160,"available":96598114304,"internally_reserved":0,"allocated":2453504,"data_stored":1407342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509534,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37180,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334660,"statfs":{"total":96636764160,"available":96598691840,"internally_reserved":0,"allocated":1871872,"data_stored":823277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:40:08.283 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3f is not active+clean 2024-09-10T23:40:08.283 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.342043+0000', 'last_change': '2024-09-10T23:40:02.342043+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.342043+0000', 'last_undegraded': '2024-09-10T23:40:02.342043+0000', 'last_fullsized': '2024-09-10T23:40:02.342043+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T23:40:08.283 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3e is not active+clean 2024-09-10T23:40:08.283 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.344703+0000', 'last_change': '2024-09-10T23:40:02.344703+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.344703+0000', 'last_undegraded': '2024-09-10T23:40:02.344703+0000', 'last_fullsized': '2024-09-10T23:40:02.344703+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T23:40:08.283 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3d is not active+clean 2024-09-10T23:40:08.283 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.353593+0000', 'last_change': '2024-09-10T23:40:02.353593+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.353593+0000', 'last_undegraded': '2024-09-10T23:40:02.353593+0000', 'last_fullsized': '2024-09-10T23:40:02.353593+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:40:08.283 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3c is not active+clean 2024-09-10T23:40:08.283 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.343368+0000', 'last_change': '2024-09-10T23:40:02.343368+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.343368+0000', 'last_undegraded': '2024-09-10T23:40:02.343368+0000', 'last_fullsized': '2024-09-10T23:40:02.343368+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T23:40:08.283 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3b is not active+clean 2024-09-10T23:40:08.283 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.351849+0000', 'last_change': '2024-09-10T23:40:02.351849+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.351849+0000', 'last_undegraded': '2024-09-10T23:40:02.351849+0000', 'last_fullsized': '2024-09-10T23:40:02.351849+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:40:08.284 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3a is not active+clean 2024-09-10T23:40:08.284 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.345840+0000', 'last_change': '2024-09-10T23:40:02.345840+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.345840+0000', 'last_undegraded': '2024-09-10T23:40:02.345840+0000', 'last_fullsized': '2024-09-10T23:40:02.345840+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T23:40:08.284 INFO:tasks.ceph.ceph_manager.ceph:PG 25.39 is not active+clean 2024-09-10T23:40:08.284 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.339971+0000', 'last_change': '2024-09-10T23:40:02.339971+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.339971+0000', 'last_undegraded': '2024-09-10T23:40:02.339971+0000', 'last_fullsized': '2024-09-10T23:40:02.339971+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T23:40:08.284 INFO:tasks.ceph.ceph_manager.ceph:PG 25.38 is not active+clean 2024-09-10T23:40:08.284 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.350386+0000', 'last_change': '2024-09-10T23:40:02.350386+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.350386+0000', 'last_undegraded': '2024-09-10T23:40:02.350386+0000', 'last_fullsized': '2024-09-10T23:40:02.350386+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:40:08.284 INFO:tasks.ceph.ceph_manager.ceph:PG 25.37 is not active+clean 2024-09-10T23:40:08.284 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.348677+0000', 'last_change': '2024-09-10T23:40:02.348677+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.348677+0000', 'last_undegraded': '2024-09-10T23:40:02.348677+0000', 'last_fullsized': '2024-09-10T23:40:02.348677+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T23:40:08.284 INFO:tasks.ceph.ceph_manager.ceph:PG 25.36 is not active+clean 2024-09-10T23:40:08.284 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.348260+0000', 'last_change': '2024-09-10T23:40:02.348260+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.348260+0000', 'last_undegraded': '2024-09-10T23:40:02.348260+0000', 'last_fullsized': '2024-09-10T23:40:02.348260+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T23:40:08.284 INFO:tasks.ceph.ceph_manager.ceph:PG 25.35 is not active+clean 2024-09-10T23:40:08.285 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.342470+0000', 'last_change': '2024-09-10T23:40:02.342470+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.342470+0000', 'last_undegraded': '2024-09-10T23:40:02.342470+0000', 'last_fullsized': '2024-09-10T23:40:02.342470+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T23:40:08.285 INFO:tasks.ceph.ceph_manager.ceph:PG 25.34 is not active+clean 2024-09-10T23:40:08.285 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.344421+0000', 'last_change': '2024-09-10T23:40:02.344421+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.344421+0000', 'last_undegraded': '2024-09-10T23:40:02.344421+0000', 'last_fullsized': '2024-09-10T23:40:02.344421+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T23:40:08.285 INFO:tasks.ceph.ceph_manager.ceph:PG 25.33 is not active+clean 2024-09-10T23:40:08.285 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.341773+0000', 'last_change': '2024-09-10T23:40:02.341773+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.341773+0000', 'last_undegraded': '2024-09-10T23:40:02.341773+0000', 'last_fullsized': '2024-09-10T23:40:02.341773+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T23:40:08.285 INFO:tasks.ceph.ceph_manager.ceph:PG 25.32 is not active+clean 2024-09-10T23:40:08.285 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.343249+0000', 'last_change': '2024-09-10T23:40:02.343249+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.343249+0000', 'last_undegraded': '2024-09-10T23:40:02.343249+0000', 'last_fullsized': '2024-09-10T23:40:02.343249+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T23:40:08.285 INFO:tasks.ceph.ceph_manager.ceph:PG 25.31 is not active+clean 2024-09-10T23:40:08.285 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.343783+0000', 'last_change': '2024-09-10T23:40:02.343783+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.343783+0000', 'last_undegraded': '2024-09-10T23:40:02.343783+0000', 'last_fullsized': '2024-09-10T23:40:02.343783+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T23:40:08.285 INFO:tasks.ceph.ceph_manager.ceph:PG 25.30 is not active+clean 2024-09-10T23:40:08.285 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.350869+0000', 'last_change': '2024-09-10T23:40:02.350869+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.350869+0000', 'last_undegraded': '2024-09-10T23:40:02.350869+0000', 'last_fullsized': '2024-09-10T23:40:02.350869+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:40:08.286 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2f is not active+clean 2024-09-10T23:40:08.286 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.347466+0000', 'last_change': '2024-09-10T23:40:02.347466+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.347466+0000', 'last_undegraded': '2024-09-10T23:40:02.347466+0000', 'last_fullsized': '2024-09-10T23:40:02.347466+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T23:40:08.286 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2e is not active+clean 2024-09-10T23:40:08.286 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.340509+0000', 'last_change': '2024-09-10T23:40:02.340509+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.340509+0000', 'last_undegraded': '2024-09-10T23:40:02.340509+0000', 'last_fullsized': '2024-09-10T23:40:02.340509+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T23:40:08.286 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2d is not active+clean 2024-09-10T23:40:08.286 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.353158+0000', 'last_change': '2024-09-10T23:40:02.353158+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.353158+0000', 'last_undegraded': '2024-09-10T23:40:02.353158+0000', 'last_fullsized': '2024-09-10T23:40:02.353158+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:40:08.286 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2c is not active+clean 2024-09-10T23:40:08.286 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.348678+0000', 'last_change': '2024-09-10T23:40:02.348678+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.348678+0000', 'last_undegraded': '2024-09-10T23:40:02.348678+0000', 'last_fullsized': '2024-09-10T23:40:02.348678+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:40:08.286 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2b is not active+clean 2024-09-10T23:40:08.286 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.351641+0000', 'last_change': '2024-09-10T23:40:02.351641+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.351641+0000', 'last_undegraded': '2024-09-10T23:40:02.351641+0000', 'last_fullsized': '2024-09-10T23:40:02.351641+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:40:08.286 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2a is not active+clean 2024-09-10T23:40:08.286 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.350410+0000', 'last_change': '2024-09-10T23:40:02.350410+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.350410+0000', 'last_undegraded': '2024-09-10T23:40:02.350410+0000', 'last_fullsized': '2024-09-10T23:40:02.350410+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:40:08.287 INFO:tasks.ceph.ceph_manager.ceph:PG 25.29 is not active+clean 2024-09-10T23:40:08.287 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.345990+0000', 'last_change': '2024-09-10T23:40:02.345990+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.345990+0000', 'last_undegraded': '2024-09-10T23:40:02.345990+0000', 'last_fullsized': '2024-09-10T23:40:02.345990+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T23:40:08.287 INFO:tasks.ceph.ceph_manager.ceph:PG 25.28 is not active+clean 2024-09-10T23:40:08.287 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.349962+0000', 'last_change': '2024-09-10T23:40:02.349962+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.349962+0000', 'last_undegraded': '2024-09-10T23:40:02.349962+0000', 'last_fullsized': '2024-09-10T23:40:02.349962+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:40:08.287 INFO:tasks.ceph.ceph_manager.ceph:PG 25.27 is not active+clean 2024-09-10T23:40:08.287 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.348253+0000', 'last_change': '2024-09-10T23:40:02.348253+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.348253+0000', 'last_undegraded': '2024-09-10T23:40:02.348253+0000', 'last_fullsized': '2024-09-10T23:40:02.348253+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T23:40:08.287 INFO:tasks.ceph.ceph_manager.ceph:PG 25.26 is not active+clean 2024-09-10T23:40:08.287 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.348873+0000', 'last_change': '2024-09-10T23:40:02.348873+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.348873+0000', 'last_undegraded': '2024-09-10T23:40:02.348873+0000', 'last_fullsized': '2024-09-10T23:40:02.348873+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T23:40:08.287 INFO:tasks.ceph.ceph_manager.ceph:PG 25.25 is not active+clean 2024-09-10T23:40:08.287 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.352668+0000', 'last_change': '2024-09-10T23:40:02.352668+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.352668+0000', 'last_undegraded': '2024-09-10T23:40:02.352668+0000', 'last_fullsized': '2024-09-10T23:40:02.352668+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:40:08.287 INFO:tasks.ceph.ceph_manager.ceph:PG 25.24 is not active+clean 2024-09-10T23:40:08.287 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.344673+0000', 'last_change': '2024-09-10T23:40:02.344673+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.344673+0000', 'last_undegraded': '2024-09-10T23:40:02.344673+0000', 'last_fullsized': '2024-09-10T23:40:02.344673+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T23:40:08.288 INFO:tasks.ceph.ceph_manager.ceph:PG 25.23 is not active+clean 2024-09-10T23:40:08.288 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.343596+0000', 'last_change': '2024-09-10T23:40:02.343596+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.343596+0000', 'last_undegraded': '2024-09-10T23:40:02.343596+0000', 'last_fullsized': '2024-09-10T23:40:02.343596+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T23:40:08.288 INFO:tasks.ceph.ceph_manager.ceph:PG 25.22 is not active+clean 2024-09-10T23:40:08.288 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.346547+0000', 'last_change': '2024-09-10T23:40:02.346547+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.346547+0000', 'last_undegraded': '2024-09-10T23:40:02.346547+0000', 'last_fullsized': '2024-09-10T23:40:02.346547+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T23:40:08.288 INFO:tasks.ceph.ceph_manager.ceph:PG 25.21 is not active+clean 2024-09-10T23:40:08.288 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.350824+0000', 'last_change': '2024-09-10T23:40:02.350824+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.350824+0000', 'last_undegraded': '2024-09-10T23:40:02.350824+0000', 'last_fullsized': '2024-09-10T23:40:02.350824+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:40:08.288 INFO:tasks.ceph.ceph_manager.ceph:PG 25.20 is not active+clean 2024-09-10T23:40:08.288 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.348261+0000', 'last_change': '2024-09-10T23:40:02.348261+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.348261+0000', 'last_undegraded': '2024-09-10T23:40:02.348261+0000', 'last_fullsized': '2024-09-10T23:40:02.348261+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:40:08.288 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1e is not active+clean 2024-09-10T23:40:08.288 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.349611+0000', 'last_change': '2024-09-10T23:40:02.349611+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.349611+0000', 'last_undegraded': '2024-09-10T23:40:02.349611+0000', 'last_fullsized': '2024-09-10T23:40:02.349611+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:40:08.289 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1f is not active+clean 2024-09-10T23:40:08.289 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.345207+0000', 'last_change': '2024-09-10T23:40:02.345207+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.345207+0000', 'last_undegraded': '2024-09-10T23:40:02.345207+0000', 'last_fullsized': '2024-09-10T23:40:02.345207+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T23:40:08.289 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1c is not active+clean 2024-09-10T23:40:08.289 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.342932+0000', 'last_change': '2024-09-10T23:40:02.342932+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.342932+0000', 'last_undegraded': '2024-09-10T23:40:02.342932+0000', 'last_fullsized': '2024-09-10T23:40:02.342932+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T23:40:08.289 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1d is not active+clean 2024-09-10T23:40:08.289 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.342179+0000', 'last_change': '2024-09-10T23:40:02.342179+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.342179+0000', 'last_undegraded': '2024-09-10T23:40:02.342179+0000', 'last_fullsized': '2024-09-10T23:40:02.342179+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T23:40:08.289 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1a is not active+clean 2024-09-10T23:40:08.289 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.345401+0000', 'last_change': '2024-09-10T23:40:02.345401+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.345401+0000', 'last_undegraded': '2024-09-10T23:40:02.345401+0000', 'last_fullsized': '2024-09-10T23:40:02.345401+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T23:40:08.289 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1b is not active+clean 2024-09-10T23:40:08.289 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.349639+0000', 'last_change': '2024-09-10T23:40:02.349639+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.349639+0000', 'last_undegraded': '2024-09-10T23:40:02.349639+0000', 'last_fullsized': '2024-09-10T23:40:02.349639+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:40:08.290 INFO:tasks.ceph.ceph_manager.ceph:PG 25.19 is not active+clean 2024-09-10T23:40:08.290 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.340809+0000', 'last_change': '2024-09-10T23:40:02.340809+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.340809+0000', 'last_undegraded': '2024-09-10T23:40:02.340809+0000', 'last_fullsized': '2024-09-10T23:40:02.340809+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T23:40:08.290 INFO:tasks.ceph.ceph_manager.ceph:PG 25.16 is not active+clean 2024-09-10T23:40:08.290 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.353296+0000', 'last_change': '2024-09-10T23:40:02.353296+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.353296+0000', 'last_undegraded': '2024-09-10T23:40:02.353296+0000', 'last_fullsized': '2024-09-10T23:40:02.353296+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:40:08.290 INFO:tasks.ceph.ceph_manager.ceph:PG 25.17 is not active+clean 2024-09-10T23:40:08.290 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.344844+0000', 'last_change': '2024-09-10T23:40:02.344844+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.344844+0000', 'last_undegraded': '2024-09-10T23:40:02.344844+0000', 'last_fullsized': '2024-09-10T23:40:02.344844+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T23:40:08.290 INFO:tasks.ceph.ceph_manager.ceph:PG 25.14 is not active+clean 2024-09-10T23:40:08.290 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.344044+0000', 'last_change': '2024-09-10T23:40:02.344044+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.344044+0000', 'last_undegraded': '2024-09-10T23:40:02.344044+0000', 'last_fullsized': '2024-09-10T23:40:02.344044+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T23:40:08.290 INFO:tasks.ceph.ceph_manager.ceph:PG 25.15 is not active+clean 2024-09-10T23:40:08.290 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.344602+0000', 'last_change': '2024-09-10T23:40:02.344602+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.344602+0000', 'last_undegraded': '2024-09-10T23:40:02.344602+0000', 'last_fullsized': '2024-09-10T23:40:02.344602+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T23:40:08.290 INFO:tasks.ceph.ceph_manager.ceph:PG 25.12 is not active+clean 2024-09-10T23:40:08.290 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.344061+0000', 'last_change': '2024-09-10T23:40:02.344061+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.344061+0000', 'last_undegraded': '2024-09-10T23:40:02.344061+0000', 'last_fullsized': '2024-09-10T23:40:02.344061+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T23:40:08.291 INFO:tasks.ceph.ceph_manager.ceph:PG 25.13 is not active+clean 2024-09-10T23:40:08.291 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.340447+0000', 'last_change': '2024-09-10T23:40:02.340447+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.340447+0000', 'last_undegraded': '2024-09-10T23:40:02.340447+0000', 'last_fullsized': '2024-09-10T23:40:02.340447+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T23:40:08.291 INFO:tasks.ceph.ceph_manager.ceph:PG 25.10 is not active+clean 2024-09-10T23:40:08.291 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.342891+0000', 'last_change': '2024-09-10T23:40:02.342891+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.342891+0000', 'last_undegraded': '2024-09-10T23:40:02.342891+0000', 'last_fullsized': '2024-09-10T23:40:02.342891+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T23:40:08.291 INFO:tasks.ceph.ceph_manager.ceph:PG 25.11 is not active+clean 2024-09-10T23:40:08.291 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.348260+0000', 'last_change': '2024-09-10T23:40:02.348260+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.348260+0000', 'last_undegraded': '2024-09-10T23:40:02.348260+0000', 'last_fullsized': '2024-09-10T23:40:02.348260+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:40:08.291 INFO:tasks.ceph.ceph_manager.ceph:PG 25.e is not active+clean 2024-09-10T23:40:08.291 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.344468+0000', 'last_change': '2024-09-10T23:40:02.344468+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.344468+0000', 'last_undegraded': '2024-09-10T23:40:02.344468+0000', 'last_fullsized': '2024-09-10T23:40:02.344468+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T23:40:08.291 INFO:tasks.ceph.ceph_manager.ceph:PG 25.f is not active+clean 2024-09-10T23:40:08.291 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.343271+0000', 'last_change': '2024-09-10T23:40:02.343271+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.343271+0000', 'last_undegraded': '2024-09-10T23:40:02.343271+0000', 'last_fullsized': '2024-09-10T23:40:02.343271+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T23:40:08.291 INFO:tasks.ceph.ceph_manager.ceph:PG 25.c is not active+clean 2024-09-10T23:40:08.292 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.342438+0000', 'last_change': '2024-09-10T23:40:02.342438+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.342438+0000', 'last_undegraded': '2024-09-10T23:40:02.342438+0000', 'last_fullsized': '2024-09-10T23:40:02.342438+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T23:40:08.292 INFO:tasks.ceph.ceph_manager.ceph:PG 25.d is not active+clean 2024-09-10T23:40:08.292 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.344708+0000', 'last_change': '2024-09-10T23:40:02.344708+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.344708+0000', 'last_undegraded': '2024-09-10T23:40:02.344708+0000', 'last_fullsized': '2024-09-10T23:40:02.344708+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T23:40:08.292 INFO:tasks.ceph.ceph_manager.ceph:PG 25.a is not active+clean 2024-09-10T23:40:08.292 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.344780+0000', 'last_change': '2024-09-10T23:40:02.344780+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.344780+0000', 'last_undegraded': '2024-09-10T23:40:02.344780+0000', 'last_fullsized': '2024-09-10T23:40:02.344780+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T23:40:08.292 INFO:tasks.ceph.ceph_manager.ceph:PG 25.b is not active+clean 2024-09-10T23:40:08.292 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.340014+0000', 'last_change': '2024-09-10T23:40:02.340014+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.340014+0000', 'last_undegraded': '2024-09-10T23:40:02.340014+0000', 'last_fullsized': '2024-09-10T23:40:02.340014+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T23:40:08.292 INFO:tasks.ceph.ceph_manager.ceph:PG 25.8 is not active+clean 2024-09-10T23:40:08.292 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.341152+0000', 'last_change': '2024-09-10T23:40:02.341152+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.341152+0000', 'last_undegraded': '2024-09-10T23:40:02.341152+0000', 'last_fullsized': '2024-09-10T23:40:02.341152+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T23:40:08.292 INFO:tasks.ceph.ceph_manager.ceph:PG 25.9 is not active+clean 2024-09-10T23:40:08.292 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.345583+0000', 'last_change': '2024-09-10T23:40:02.345583+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.345583+0000', 'last_undegraded': '2024-09-10T23:40:02.345583+0000', 'last_fullsized': '2024-09-10T23:40:02.345583+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T23:40:08.293 INFO:tasks.ceph.ceph_manager.ceph:PG 25.6 is not active+clean 2024-09-10T23:40:08.293 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.350633+0000', 'last_change': '2024-09-10T23:40:02.350633+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.350633+0000', 'last_undegraded': '2024-09-10T23:40:02.350633+0000', 'last_fullsized': '2024-09-10T23:40:02.350633+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:40:08.293 INFO:tasks.ceph.ceph_manager.ceph:PG 25.7 is not active+clean 2024-09-10T23:40:08.293 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.350743+0000', 'last_change': '2024-09-10T23:40:02.350743+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.350743+0000', 'last_undegraded': '2024-09-10T23:40:02.350743+0000', 'last_fullsized': '2024-09-10T23:40:02.350743+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:40:08.293 INFO:tasks.ceph.ceph_manager.ceph:PG 25.4 is not active+clean 2024-09-10T23:40:08.293 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.352067+0000', 'last_change': '2024-09-10T23:40:02.352067+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.352067+0000', 'last_undegraded': '2024-09-10T23:40:02.352067+0000', 'last_fullsized': '2024-09-10T23:40:02.352067+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T23:40:08.293 INFO:tasks.ceph.ceph_manager.ceph:PG 25.5 is not active+clean 2024-09-10T23:40:08.293 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.349167+0000', 'last_change': '2024-09-10T23:40:02.349167+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.349167+0000', 'last_undegraded': '2024-09-10T23:40:02.349167+0000', 'last_fullsized': '2024-09-10T23:40:02.349167+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:40:08.293 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2 is not active+clean 2024-09-10T23:40:08.293 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.342827+0000', 'last_change': '2024-09-10T23:40:02.342827+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.342827+0000', 'last_undegraded': '2024-09-10T23:40:02.342827+0000', 'last_fullsized': '2024-09-10T23:40:02.342827+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T23:40:08.293 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3 is not active+clean 2024-09-10T23:40:08.294 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.341411+0000', 'last_change': '2024-09-10T23:40:02.341411+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.341411+0000', 'last_undegraded': '2024-09-10T23:40:02.341411+0000', 'last_fullsized': '2024-09-10T23:40:02.341411+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T23:40:08.294 INFO:tasks.ceph.ceph_manager.ceph:PG 25.0 is not active+clean 2024-09-10T23:40:08.294 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.341593+0000', 'last_change': '2024-09-10T23:40:02.341593+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.341593+0000', 'last_undegraded': '2024-09-10T23:40:02.341593+0000', 'last_fullsized': '2024-09-10T23:40:02.341593+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T23:40:08.294 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1 is not active+clean 2024-09-10T23:40:08.294 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.347814+0000', 'last_change': '2024-09-10T23:40:02.347814+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.347814+0000', 'last_undegraded': '2024-09-10T23:40:02.347814+0000', 'last_fullsized': '2024-09-10T23:40:02.347814+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T23:40:08.294 INFO:tasks.ceph.ceph_manager.ceph:PG 25.18 is not active+clean 2024-09-10T23:40:08.294 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-10T23:40:02.346140+0000', 'last_change': '2024-09-10T23:40:02.346140+0000', 'last_active': '2024-09-10T23:40:02.324338+0000', 'last_peered': '2024-09-10T23:40:02.324338+0000', 'last_clean': '2024-09-10T23:40:02.324338+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T23:40:02.346140+0000', 'last_undegraded': '2024-09-10T23:40:02.346140+0000', 'last_fullsized': '2024-09-10T23:40:02.346140+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'last_clean_scrub_stamp': '2024-09-10T23:40:02.324338+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T23:40:08.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T23:40:08.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.459+0000 7f6c0a600640 1 -- 172.21.15.94:0/1848903809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c040b30e0 msgr2=0x7f6c040d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:08.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.459+0000 7f6c0a600640 1 --2- 172.21.15.94:0/1848903809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c040b30e0 0x7f6c040d1fa0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009900 tx=0x7f6bf001c8b0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.459+0000 7f6c0a600640 1 -- 172.21.15.94:0/1848903809 shutdown_connections 2024-09-10T23:40:08.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.459+0000 7f6c0a600640 1 --2- 172.21.15.94:0/1848903809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c040b30e0 0x7f6c040d1fa0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.459+0000 7f6c0a600640 1 --2- 172.21.15.94:0/1848903809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c0415bcd0 0x7f6c040d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.459+0000 7f6c0a600640 1 -- 172.21.15.94:0/1848903809 >> 172.21.15.94:0/1848903809 conn(0x7f6c0409aa70 msgr2=0x7f6c0409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:08.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.459+0000 7f6c0a600640 1 -- 172.21.15.94:0/1848903809 shutdown_connections 2024-09-10T23:40:08.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.459+0000 7f6c0a600640 1 -- 172.21.15.94:0/1848903809 wait complete. 2024-09-10T23:40:08.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.459+0000 7f6c0a600640 1 Processor -- start 2024-09-10T23:40:08.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.459+0000 7f6c0a600640 1 -- start start 2024-09-10T23:40:08.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.459+0000 7f6c0a600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c040afad0 0x7f6c040b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:08.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.459+0000 7f6c0a600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c040b30e0 0x7f6c040c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:08.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.459+0000 7f6c0a600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c0415bcd0 0x7f6c04147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:08.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.459+0000 7f6c0a600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c04168db0 con 0x7f6c040afad0 2024-09-10T23:40:08.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.459+0000 7f6c0a600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6c04168c30 con 0x7f6c0415bcd0 2024-09-10T23:40:08.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.459+0000 7f6c0a600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6c04168f30 con 0x7f6c040b30e0 2024-09-10T23:40:08.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c09200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c040afad0 0x7f6c040b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:08.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c09200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c040afad0 0x7f6c040b69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34628/0 (socket says 172.21.15.94:34628) 2024-09-10T23:40:08.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c09200640 1 -- 172.21.15.94:0/1087960905 learned_addr learned my addr 172.21.15.94:0/1087960905 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:08.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c03e00640 1 --2- 172.21.15.94:0/1087960905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c040b30e0 0x7f6c040c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:08.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c09c00640 1 --2- 172.21.15.94:0/1087960905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c0415bcd0 0x7f6c04147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:08.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c09200640 1 -- 172.21.15.94:0/1087960905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c040b30e0 msgr2=0x7f6c040c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:08.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c09200640 1 --2- 172.21.15.94:0/1087960905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c040b30e0 0x7f6c040c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c09200640 1 -- 172.21.15.94:0/1087960905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c0415bcd0 msgr2=0x7f6c04147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:08.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c09200640 1 --2- 172.21.15.94:0/1087960905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c0415bcd0 0x7f6c04147650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c09200640 1 -- 172.21.15.94:0/1087960905 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c040b6f30 con 0x7f6c040afad0 2024-09-10T23:40:08.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c09200640 1 --2- 172.21.15.94:0/1087960905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c040afad0 0x7f6c040b69f0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800d750 tx=0x7f6bf800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:08.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c00c00640 1 -- 172.21.15.94:0/1087960905 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8017070 con 0x7f6c040afad0 2024-09-10T23:40:08.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c00c00640 1 -- 172.21.15.94:0/1087960905 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6bf8009d20 con 0x7f6c040afad0 2024-09-10T23:40:08.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c00c00640 1 -- 172.21.15.94:0/1087960905 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8005280 con 0x7f6c040afad0 2024-09-10T23:40:08.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c0a600640 1 -- 172.21.15.94:0/1087960905 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c040c5790 con 0x7f6c040afad0 2024-09-10T23:40:08.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c0a600640 1 -- 172.21.15.94:0/1087960905 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c040b7490 con 0x7f6c040afad0 2024-09-10T23:40:08.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c0a600640 1 -- 172.21.15.94:0/1087960905 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c040aff80 con 0x7f6c040afad0 2024-09-10T23:40:08.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.463+0000 7f6c00c00640 1 -- 172.21.15.94:0/1087960905 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f6bf8005420 con 0x7f6c040afad0 2024-09-10T23:40:08.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.467+0000 7f6c00c00640 1 --2- 172.21.15.94:0/1087960905 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6be4072e80 0x7f6be4075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:08.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.467+0000 7f6c00c00640 1 -- 172.21.15.94:0/1087960905 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f6bf80945c0 con 0x7f6c040afad0 2024-09-10T23:40:08.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.467+0000 7f6c03e00640 1 --2- 172.21.15.94:0/1087960905 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6be4072e80 0x7f6be4075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:08.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.467+0000 7f6c03e00640 1 --2- 172.21.15.94:0/1087960905 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6be4072e80 0x7f6be4075340 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00af000 tx=0x7f6bf001b0f0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:08.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.471+0000 7f6c00c00640 1 -- 172.21.15.94:0/1087960905 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6c040b6f30 con 0x7f6c040afad0 2024-09-10T23:40:08.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.699+0000 7f6c0a600640 1 -- 172.21.15.94:0/1087960905 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f6c040c53a0 con 0x7f6c040afad0 2024-09-10T23:40:08.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.699+0000 7f6c00c00640 1 -- 172.21.15.94:0/1087960905 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1631 (secure 0 0 0) 0x7f6bf800de60 con 0x7f6c040afad0 2024-09-10T23:40:08.703 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:08.703 INFO:teuthology.orchestra.run.smithi094.stdout:{"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":471,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":151,"num_osds":8,"num_up_osds":8,"osd_up_since":1726011151,"num_in_osds":8,"osd_in_since":1726011140,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":284385280,"bytes_avail":772809728000,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":107,"btime":"2024-09-10T23:40:06:362943+0000","id":12,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":12,"rank":0,"name":"b","status":"up:active","gid":197120}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-10T23:37:42.748205+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T23:40:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.699+0000 7f6c0a600640 1 -- 172.21.15.94:0/1087960905 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6be4072e80 msgr2=0x7f6be4075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.699+0000 7f6c0a600640 1 --2- 172.21.15.94:0/1087960905 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6be4072e80 0x7f6be4075340 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00af000 tx=0x7f6bf001b0f0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.699+0000 7f6c0a600640 1 -- 172.21.15.94:0/1087960905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c040afad0 msgr2=0x7f6c040b69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.699+0000 7f6c0a600640 1 --2- 172.21.15.94:0/1087960905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c040afad0 0x7f6c040b69f0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800d750 tx=0x7f6bf800dc10 comp rx=0 tx=0).stop 2024-09-10T23:40:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.699+0000 7f6c0a600640 1 -- 172.21.15.94:0/1087960905 shutdown_connections 2024-09-10T23:40:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.699+0000 7f6c0a600640 1 --2- 172.21.15.94:0/1087960905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c0415bcd0 0x7f6c04147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.699+0000 7f6c0a600640 1 --2- 172.21.15.94:0/1087960905 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6be4072e80 0x7f6be4075340 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.699+0000 7f6c0a600640 1 --2- 172.21.15.94:0/1087960905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c040b30e0 0x7f6c040c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.699+0000 7f6c0a600640 1 --2- 172.21.15.94:0/1087960905 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c040afad0 0x7f6c040b69f0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.699+0000 7f6c0a600640 1 -- 172.21.15.94:0/1087960905 >> 172.21.15.94:0/1087960905 conn(0x7f6c0409aa70 msgr2=0x7f6c04159fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:08.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.703+0000 7f6c0a600640 1 -- 172.21.15.94:0/1087960905 shutdown_connections 2024-09-10T23:40:08.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.703+0000 7f6c0a600640 1 -- 172.21.15.94:0/1087960905 wait complete. 2024-09-10T23:40:08.724 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T23:40:08.725 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:40:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.863+0000 7f0004c00640 1 -- 172.21.15.94:0/726439093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00000703e0 msgr2=0x7f00000b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.863+0000 7f0004c00640 1 --2- 172.21.15.94:0/726439093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00000703e0 0x7f00000b45e0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7effe8009900 tx=0x7effe801c820 comp rx=0 tx=0).stop 2024-09-10T23:40:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.863+0000 7f0004c00640 1 -- 172.21.15.94:0/726439093 shutdown_connections 2024-09-10T23:40:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.863+0000 7f0004c00640 1 --2- 172.21.15.94:0/726439093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f000015eba0 0x7f000006e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.863+0000 7f0004c00640 1 --2- 172.21.15.94:0/726439093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00000703e0 0x7f00000b45e0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.863+0000 7f0004c00640 1 -- 172.21.15.94:0/726439093 >> 172.21.15.94:0/726439093 conn(0x7f00000b1840 msgr2=0x7f00000b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.863+0000 7f0004c00640 1 -- 172.21.15.94:0/726439093 shutdown_connections 2024-09-10T23:40:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.863+0000 7f0004c00640 1 -- 172.21.15.94:0/726439093 wait complete. 2024-09-10T23:40:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.863+0000 7f0004c00640 1 Processor -- start 2024-09-10T23:40:08.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.863+0000 7f0004c00640 1 -- start start 2024-09-10T23:40:08.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7f0004c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0000070000 0x7f000015aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7f0004c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00000703e0 0x7f0000163450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7f0004c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f000015eba0 0x7f00001504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7f0004c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0000167c10 con 0x7f0000070000 2024-09-10T23:40:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7f0004c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0000167a90 con 0x7f00000703e0 2024-09-10T23:40:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7f0004c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0000167d90 con 0x7f000015eba0 2024-09-10T23:40:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7effff400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0000070000 0x7f000015aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7effff400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0000070000 0x7f000015aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34648/0 (socket says 172.21.15.94:34648) 2024-09-10T23:40:08.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7effff400640 1 -- 172.21.15.94:0/1717638579 learned_addr learned my addr 172.21.15.94:0/1717638579 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:08.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7effff400640 1 -- 172.21.15.94:0/1717638579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f000015eba0 msgr2=0x7f00001504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:08.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7effff400640 1 --2- 172.21.15.94:0/1717638579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f000015eba0 0x7f00001504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7effff400640 1 -- 172.21.15.94:0/1717638579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00000703e0 msgr2=0x7f0000163450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7effff400640 1 --2- 172.21.15.94:0/1717638579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00000703e0 0x7f0000163450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7effff400640 1 -- 172.21.15.94:0/1717638579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0000161670 con 0x7f0000070000 2024-09-10T23:40:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7effff400640 1 --2- 172.21.15.94:0/1717638579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0000070000 0x7f000015aeb0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7efff000c500 tx=0x7efff000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7efff7e00640 1 -- 172.21.15.94:0/1717638579 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0017070 con 0x7f0000070000 2024-09-10T23:40:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7efff7e00640 1 -- 172.21.15.94:0/1717638579 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efff00053c0 con 0x7f0000070000 2024-09-10T23:40:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7f0004c00640 1 -- 172.21.15.94:0/1717638579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00001f5fe0 con 0x7f0000070000 2024-09-10T23:40:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7efff7e00640 1 -- 172.21.15.94:0/1717638579 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0003860 con 0x7f0000070000 2024-09-10T23:40:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7f0004c00640 1 -- 172.21.15.94:0/1717638579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000015b3f0 con 0x7f0000070000 2024-09-10T23:40:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7f0004c00640 1 -- 172.21.15.94:0/1717638579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00000b45e0 con 0x7f0000070000 2024-09-10T23:40:08.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.867+0000 7efff7e00640 1 -- 172.21.15.94:0/1717638579 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7efff0003a00 con 0x7f0000070000 2024-09-10T23:40:08.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.871+0000 7efff7e00640 1 --2- 172.21.15.94:0/1717638579 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7effdc072e50 0x7effdc075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:08.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.871+0000 7efff7e00640 1 -- 172.21.15.94:0/1717638579 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7efff0093c20 con 0x7f0000070000 2024-09-10T23:40:08.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.871+0000 7efff6a00640 1 --2- 172.21.15.94:0/1717638579 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7effdc072e50 0x7effdc075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:08.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.871+0000 7efff6a00640 1 --2- 172.21.15.94:0/1717638579 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7effdc072e50 0x7effdc075310 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7effe80af000 tx=0x7effe80195c0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:08.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:08.875+0000 7efff7e00640 1 -- 172.21.15.94:0/1717638579 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f00000731e0 con 0x7f0000070000 2024-09-10T23:40:09.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:09.035+0000 7f0004c00640 1 -- 172.21.15.94:0/1717638579 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f00000b32b0 con 0x7effdc072e50 2024-09-10T23:40:09.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:09.039+0000 7efff7e00640 1 -- 172.21.15.94:0/1717638579 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7effe80f7a60 con 0x7effdc072e50 2024-09-10T23:40:09.047 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:40:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:09.043+0000 7f0004c00640 1 -- 172.21.15.94:0/1717638579 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7effdc072e50 msgr2=0x7effdc075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:09.043+0000 7f0004c00640 1 --2- 172.21.15.94:0/1717638579 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7effdc072e50 0x7effdc075310 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7effe80af000 tx=0x7effe80195c0 comp rx=0 tx=0).stop 2024-09-10T23:40:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:09.043+0000 7f0004c00640 1 -- 172.21.15.94:0/1717638579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0000070000 msgr2=0x7f000015aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:09.043+0000 7f0004c00640 1 --2- 172.21.15.94:0/1717638579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0000070000 0x7f000015aeb0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7efff000c500 tx=0x7efff000c9c0 comp rx=0 tx=0).stop 2024-09-10T23:40:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:09.043+0000 7f0004c00640 1 -- 172.21.15.94:0/1717638579 shutdown_connections 2024-09-10T23:40:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:09.043+0000 7f0004c00640 1 --2- 172.21.15.94:0/1717638579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00000703e0 0x7f0000163450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:09.043+0000 7f0004c00640 1 --2- 172.21.15.94:0/1717638579 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7effdc072e50 0x7effdc075310 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:09.043+0000 7f0004c00640 1 --2- 172.21.15.94:0/1717638579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f000015eba0 0x7f00001504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:09.043+0000 7f0004c00640 1 --2- 172.21.15.94:0/1717638579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0000070000 0x7f000015aeb0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:09.043+0000 7f0004c00640 1 -- 172.21.15.94:0/1717638579 >> 172.21.15.94:0/1717638579 conn(0x7f00000b1840 msgr2=0x7f00000b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:09.043+0000 7f0004c00640 1 -- 172.21.15.94:0/1717638579 shutdown_connections 2024-09-10T23:40:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:09.043+0000 7f0004c00640 1 -- 172.21.15.94:0/1717638579 wait complete. 2024-09-10T23:40:09.067 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-10T23:40:08.778703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282196,"kb_used_data":16040,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692524,"statfs":{"total":773094113280,"available":772805144576,"internally_reserved":0,"allocated":16424960,"data_stored":7876820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.443332"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341333+0000","last_change":"2024-09-10T23:40:03.347799+0000","last_active":"2024-09-10T23:40:04.341333+0000","last_peered":"2024-09-10T23:40:04.341333+0000","last_clean":"2024-09-10T23:40:04.341333+0000","last_became_active":"2024-09-10T23:40:03.347210+0000","last_became_peered":"2024-09-10T23:40:03.347210+0000","last_unstale":"2024-09-10T23:40:04.341333+0000","last_undegraded":"2024-09-10T23:40:04.341333+0000","last_fullsized":"2024-09-10T23:40:04.341333+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:36:23.105511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.983204+0000","last_change":"2024-09-10T23:40:03.355190+0000","last_active":"2024-09-10T23:40:04.983204+0000","last_peered":"2024-09-10T23:40:04.983204+0000","last_clean":"2024-09-10T23:40:04.983204+0000","last_became_active":"2024-09-10T23:40:03.354908+0000","last_became_peered":"2024-09-10T23:40:03.354908+0000","last_unstale":"2024-09-10T23:40:04.983204+0000","last_undegraded":"2024-09-10T23:40:04.983204+0000","last_fullsized":"2024-09-10T23:40:04.983204+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:54.609589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346251+0000","last_change":"2024-09-10T23:40:03.356963+0000","last_active":"2024-09-10T23:40:04.346251+0000","last_peered":"2024-09-10T23:40:04.346251+0000","last_clean":"2024-09-10T23:40:04.346251+0000","last_became_active":"2024-09-10T23:40:03.356657+0000","last_became_peered":"2024-09-10T23:40:03.356657+0000","last_unstale":"2024-09-10T23:40:04.346251+0000","last_undegraded":"2024-09-10T23:40:04.346251+0000","last_fullsized":"2024-09-10T23:40:04.346251+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:06.383906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982892+0000","last_change":"2024-09-10T23:40:03.358164+0000","last_active":"2024-09-10T23:40:04.982892+0000","last_peered":"2024-09-10T23:40:04.982892+0000","last_clean":"2024-09-10T23:40:04.982892+0000","last_became_active":"2024-09-10T23:40:03.357896+0000","last_became_peered":"2024-09-10T23:40:03.357896+0000","last_unstale":"2024-09-10T23:40:04.982892+0000","last_undegraded":"2024-09-10T23:40:04.982892+0000","last_fullsized":"2024-09-10T23:40:04.982892+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:30.035176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066627+0000","last_change":"2024-09-10T23:40:03.364653+0000","last_active":"2024-09-10T23:40:05.066627+0000","last_peered":"2024-09-10T23:40:05.066627+0000","last_clean":"2024-09-10T23:40:05.066627+0000","last_became_active":"2024-09-10T23:40:03.362003+0000","last_became_peered":"2024-09-10T23:40:03.362003+0000","last_unstale":"2024-09-10T23:40:05.066627+0000","last_undegraded":"2024-09-10T23:40:05.066627+0000","last_fullsized":"2024-09-10T23:40:05.066627+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:54.442244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734485+0000","last_change":"2024-09-10T23:40:03.370116+0000","last_active":"2024-09-10T23:40:05.734485+0000","last_peered":"2024-09-10T23:40:05.734485+0000","last_clean":"2024-09-10T23:40:05.734485+0000","last_became_active":"2024-09-10T23:40:03.369558+0000","last_became_peered":"2024-09-10T23:40:03.369558+0000","last_unstale":"2024-09-10T23:40:05.734485+0000","last_undegraded":"2024-09-10T23:40:05.734485+0000","last_fullsized":"2024-09-10T23:40:05.734485+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:01.665066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.966555+0000","last_change":"2024-09-10T23:40:03.359832+0000","last_active":"2024-09-10T23:40:04.966555+0000","last_peered":"2024-09-10T23:40:04.966555+0000","last_clean":"2024-09-10T23:40:04.966555+0000","last_became_active":"2024-09-10T23:40:03.359524+0000","last_became_peered":"2024-09-10T23:40:03.359524+0000","last_unstale":"2024-09-10T23:40:04.966555+0000","last_undegraded":"2024-09-10T23:40:04.966555+0000","last_fullsized":"2024-09-10T23:40:04.966555+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:55:51.627429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066411+0000","last_change":"2024-09-10T23:40:03.367253+0000","last_active":"2024-09-10T23:40:05.066411+0000","last_peered":"2024-09-10T23:40:05.066411+0000","last_clean":"2024-09-10T23:40:05.066411+0000","last_became_active":"2024-09-10T23:40:03.366683+0000","last_became_peered":"2024-09-10T23:40:03.366683+0000","last_unstale":"2024-09-10T23:40:05.066411+0000","last_undegraded":"2024-09-10T23:40:05.066411+0000","last_fullsized":"2024-09-10T23:40:05.066411+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:28.572221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.983109+0000","last_change":"2024-09-10T23:40:03.359302+0000","last_active":"2024-09-10T23:40:04.983109+0000","last_peered":"2024-09-10T23:40:04.983109+0000","last_clean":"2024-09-10T23:40:04.983109+0000","last_became_active":"2024-09-10T23:40:03.358968+0000","last_became_peered":"2024-09-10T23:40:03.358968+0000","last_unstale":"2024-09-10T23:40:04.983109+0000","last_undegraded":"2024-09-10T23:40:04.983109+0000","last_fullsized":"2024-09-10T23:40:04.983109+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:54.663834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345613+0000","last_change":"2024-09-10T23:40:03.354046+0000","last_active":"2024-09-10T23:40:04.345613+0000","last_peered":"2024-09-10T23:40:04.345613+0000","last_clean":"2024-09-10T23:40:04.345613+0000","last_became_active":"2024-09-10T23:40:03.352447+0000","last_became_peered":"2024-09-10T23:40:03.352447+0000","last_unstale":"2024-09-10T23:40:04.345613+0000","last_undegraded":"2024-09-10T23:40:04.345613+0000","last_fullsized":"2024-09-10T23:40:04.345613+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:02.595009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341254+0000","last_change":"2024-09-10T23:40:03.354408+0000","last_active":"2024-09-10T23:40:04.341254+0000","last_peered":"2024-09-10T23:40:04.341254+0000","last_clean":"2024-09-10T23:40:04.341254+0000","last_became_active":"2024-09-10T23:40:03.354148+0000","last_became_peered":"2024-09-10T23:40:03.354148+0000","last_unstale":"2024-09-10T23:40:04.341254+0000","last_undegraded":"2024-09-10T23:40:04.341254+0000","last_fullsized":"2024-09-10T23:40:04.341254+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:54.112024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727703+0000","last_change":"2024-09-10T23:40:03.353432+0000","last_active":"2024-09-10T23:40:05.727703+0000","last_peered":"2024-09-10T23:40:05.727703+0000","last_clean":"2024-09-10T23:40:05.727703+0000","last_became_active":"2024-09-10T23:40:03.353187+0000","last_became_peered":"2024-09-10T23:40:03.353187+0000","last_unstale":"2024-09-10T23:40:05.727703+0000","last_undegraded":"2024-09-10T23:40:05.727703+0000","last_fullsized":"2024-09-10T23:40:05.727703+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:13.692433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341418+0000","last_change":"2024-09-10T23:40:03.354377+0000","last_active":"2024-09-10T23:40:04.341418+0000","last_peered":"2024-09-10T23:40:04.341418+0000","last_clean":"2024-09-10T23:40:04.341418+0000","last_became_active":"2024-09-10T23:40:03.354110+0000","last_became_peered":"2024-09-10T23:40:03.354110+0000","last_unstale":"2024-09-10T23:40:04.341418+0000","last_undegraded":"2024-09-10T23:40:04.341418+0000","last_fullsized":"2024-09-10T23:40:04.341418+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:10.175646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982798+0000","last_change":"2024-09-10T23:40:03.357792+0000","last_active":"2024-09-10T23:40:04.982798+0000","last_peered":"2024-09-10T23:40:04.982798+0000","last_clean":"2024-09-10T23:40:04.982798+0000","last_became_active":"2024-09-10T23:40:03.357534+0000","last_became_peered":"2024-09-10T23:40:03.357534+0000","last_unstale":"2024-09-10T23:40:04.982798+0000","last_undegraded":"2024-09-10T23:40:04.982798+0000","last_fullsized":"2024-09-10T23:40:04.982798+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:23.607415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341522+0000","last_change":"2024-09-10T23:40:03.354924+0000","last_active":"2024-09-10T23:40:04.341522+0000","last_peered":"2024-09-10T23:40:04.341522+0000","last_clean":"2024-09-10T23:40:04.341522+0000","last_became_active":"2024-09-10T23:40:03.354666+0000","last_became_peered":"2024-09-10T23:40:03.354666+0000","last_unstale":"2024-09-10T23:40:04.341522+0000","last_undegraded":"2024-09-10T23:40:04.341522+0000","last_fullsized":"2024-09-10T23:40:04.341522+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:08.034276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345605+0000","last_change":"2024-09-10T23:40:03.358303+0000","last_active":"2024-09-10T23:40:04.345605+0000","last_peered":"2024-09-10T23:40:04.345605+0000","last_clean":"2024-09-10T23:40:04.345605+0000","last_became_active":"2024-09-10T23:40:03.358059+0000","last_became_peered":"2024-09-10T23:40:03.358059+0000","last_unstale":"2024-09-10T23:40:04.345605+0000","last_undegraded":"2024-09-10T23:40:04.345605+0000","last_fullsized":"2024-09-10T23:40:04.345605+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:24.847923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734570+0000","last_change":"2024-09-10T23:40:03.368848+0000","last_active":"2024-09-10T23:40:05.734570+0000","last_peered":"2024-09-10T23:40:05.734570+0000","last_clean":"2024-09-10T23:40:05.734570+0000","last_became_active":"2024-09-10T23:40:03.368537+0000","last_became_peered":"2024-09-10T23:40:03.368537+0000","last_unstale":"2024-09-10T23:40:05.734570+0000","last_undegraded":"2024-09-10T23:40:05.734570+0000","last_fullsized":"2024-09-10T23:40:05.734570+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:19.047231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.966765+0000","last_change":"2024-09-10T23:40:03.352069+0000","last_active":"2024-09-10T23:40:04.966765+0000","last_peered":"2024-09-10T23:40:04.966765+0000","last_clean":"2024-09-10T23:40:04.966765+0000","last_became_active":"2024-09-10T23:40:03.351712+0000","last_became_peered":"2024-09-10T23:40:03.351712+0000","last_unstale":"2024-09-10T23:40:04.966765+0000","last_undegraded":"2024-09-10T23:40:04.966765+0000","last_fullsized":"2024-09-10T23:40:04.966765+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:48.871207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346447+0000","last_change":"2024-09-10T23:40:03.365598+0000","last_active":"2024-09-10T23:40:04.346447+0000","last_peered":"2024-09-10T23:40:04.346447+0000","last_clean":"2024-09-10T23:40:04.346447+0000","last_became_active":"2024-09-10T23:40:03.365072+0000","last_became_peered":"2024-09-10T23:40:03.365072+0000","last_unstale":"2024-09-10T23:40:04.346447+0000","last_undegraded":"2024-09-10T23:40:04.346447+0000","last_fullsized":"2024-09-10T23:40:04.346447+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:13.692433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066888+0000","last_change":"2024-09-10T23:40:03.363503+0000","last_active":"2024-09-10T23:40:05.066888+0000","last_peered":"2024-09-10T23:40:05.066888+0000","last_clean":"2024-09-10T23:40:05.066888+0000","last_became_active":"2024-09-10T23:40:03.363267+0000","last_became_peered":"2024-09-10T23:40:03.363267+0000","last_unstale":"2024-09-10T23:40:05.066888+0000","last_undegraded":"2024-09-10T23:40:05.066888+0000","last_fullsized":"2024-09-10T23:40:05.066888+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:47.625167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345357+0000","last_change":"2024-09-10T23:40:03.351549+0000","last_active":"2024-09-10T23:40:04.345357+0000","last_peered":"2024-09-10T23:40:04.345357+0000","last_clean":"2024-09-10T23:40:04.345357+0000","last_became_active":"2024-09-10T23:40:03.351231+0000","last_became_peered":"2024-09-10T23:40:03.351231+0000","last_unstale":"2024-09-10T23:40:04.345357+0000","last_undegraded":"2024-09-10T23:40:04.345357+0000","last_fullsized":"2024-09-10T23:40:04.345357+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:48.871207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.067192+0000","last_change":"2024-09-10T23:40:03.366474+0000","last_active":"2024-09-10T23:40:05.067192+0000","last_peered":"2024-09-10T23:40:05.067192+0000","last_clean":"2024-09-10T23:40:05.067192+0000","last_became_active":"2024-09-10T23:40:03.360637+0000","last_became_peered":"2024-09-10T23:40:03.360637+0000","last_unstale":"2024-09-10T23:40:05.067192+0000","last_undegraded":"2024-09-10T23:40:05.067192+0000","last_fullsized":"2024-09-10T23:40:05.067192+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:17.315952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.344845+0000","last_change":"2024-09-10T23:40:03.372722+0000","last_active":"2024-09-10T23:40:04.344845+0000","last_peered":"2024-09-10T23:40:04.344845+0000","last_clean":"2024-09-10T23:40:04.344845+0000","last_became_active":"2024-09-10T23:40:03.372467+0000","last_became_peered":"2024-09-10T23:40:03.372467+0000","last_unstale":"2024-09-10T23:40:04.344845+0000","last_undegraded":"2024-09-10T23:40:04.344845+0000","last_fullsized":"2024-09-10T23:40:04.344845+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:43.564546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066410+0000","last_change":"2024-09-10T23:40:03.360620+0000","last_active":"2024-09-10T23:40:05.066410+0000","last_peered":"2024-09-10T23:40:05.066410+0000","last_clean":"2024-09-10T23:40:05.066410+0000","last_became_active":"2024-09-10T23:40:03.360326+0000","last_became_peered":"2024-09-10T23:40:03.360326+0000","last_unstale":"2024-09-10T23:40:05.066410+0000","last_undegraded":"2024-09-10T23:40:05.066410+0000","last_fullsized":"2024-09-10T23:40:05.066410+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:04.666223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.983117+0000","last_change":"2024-09-10T23:40:03.356548+0000","last_active":"2024-09-10T23:40:04.983117+0000","last_peered":"2024-09-10T23:40:04.983117+0000","last_clean":"2024-09-10T23:40:04.983117+0000","last_became_active":"2024-09-10T23:40:03.356305+0000","last_became_peered":"2024-09-10T23:40:03.356305+0000","last_unstale":"2024-09-10T23:40:04.983117+0000","last_undegraded":"2024-09-10T23:40:04.983117+0000","last_fullsized":"2024-09-10T23:40:04.983117+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:39.794032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727911+0000","last_change":"2024-09-10T23:40:03.358680+0000","last_active":"2024-09-10T23:40:05.727911+0000","last_peered":"2024-09-10T23:40:05.727911+0000","last_clean":"2024-09-10T23:40:05.727911+0000","last_became_active":"2024-09-10T23:40:03.355890+0000","last_became_peered":"2024-09-10T23:40:03.355890+0000","last_unstale":"2024-09-10T23:40:05.727911+0000","last_undegraded":"2024-09-10T23:40:05.727911+0000","last_fullsized":"2024-09-10T23:40:05.727911+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:33.993062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346569+0000","last_change":"2024-09-10T23:40:03.366128+0000","last_active":"2024-09-10T23:40:04.346569+0000","last_peered":"2024-09-10T23:40:04.346569+0000","last_clean":"2024-09-10T23:40:04.346569+0000","last_became_active":"2024-09-10T23:40:03.364713+0000","last_became_peered":"2024-09-10T23:40:03.364713+0000","last_unstale":"2024-09-10T23:40:04.346569+0000","last_undegraded":"2024-09-10T23:40:04.346569+0000","last_fullsized":"2024-09-10T23:40:04.346569+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:47.309787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346002+0000","last_change":"2024-09-10T23:40:03.358246+0000","last_active":"2024-09-10T23:40:04.346002+0000","last_peered":"2024-09-10T23:40:04.346002+0000","last_clean":"2024-09-10T23:40:04.346002+0000","last_became_active":"2024-09-10T23:40:03.357993+0000","last_became_peered":"2024-09-10T23:40:03.357993+0000","last_unstale":"2024-09-10T23:40:04.346002+0000","last_undegraded":"2024-09-10T23:40:04.346002+0000","last_fullsized":"2024-09-10T23:40:04.346002+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:42.026210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727632+0000","last_change":"2024-09-10T23:40:03.356696+0000","last_active":"2024-09-10T23:40:05.727632+0000","last_peered":"2024-09-10T23:40:05.727632+0000","last_clean":"2024-09-10T23:40:05.727632+0000","last_became_active":"2024-09-10T23:40:03.356295+0000","last_became_peered":"2024-09-10T23:40:03.356295+0000","last_unstale":"2024-09-10T23:40:05.727632+0000","last_undegraded":"2024-09-10T23:40:05.727632+0000","last_fullsized":"2024-09-10T23:40:05.727632+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:06.383906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727604+0000","last_change":"2024-09-10T23:40:03.360053+0000","last_active":"2024-09-10T23:40:05.727604+0000","last_peered":"2024-09-10T23:40:05.727604+0000","last_clean":"2024-09-10T23:40:05.727604+0000","last_became_active":"2024-09-10T23:40:03.359780+0000","last_became_peered":"2024-09-10T23:40:03.359780+0000","last_unstale":"2024-09-10T23:40:05.727604+0000","last_undegraded":"2024-09-10T23:40:05.727604+0000","last_fullsized":"2024-09-10T23:40:05.727604+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:12.470964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066446+0000","last_change":"2024-09-10T23:40:03.369580+0000","last_active":"2024-09-10T23:40:05.066446+0000","last_peered":"2024-09-10T23:40:05.066446+0000","last_clean":"2024-09-10T23:40:05.066446+0000","last_became_active":"2024-09-10T23:40:03.368920+0000","last_became_peered":"2024-09-10T23:40:03.368920+0000","last_unstale":"2024-09-10T23:40:05.066446+0000","last_undegraded":"2024-09-10T23:40:05.066446+0000","last_fullsized":"2024-09-10T23:40:05.066446+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:51.802426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066721+0000","last_change":"2024-09-10T23:40:03.369593+0000","last_active":"2024-09-10T23:40:05.066721+0000","last_peered":"2024-09-10T23:40:05.066721+0000","last_clean":"2024-09-10T23:40:05.066721+0000","last_became_active":"2024-09-10T23:40:03.367008+0000","last_became_peered":"2024-09-10T23:40:03.367008+0000","last_unstale":"2024-09-10T23:40:05.066721+0000","last_undegraded":"2024-09-10T23:40:05.066721+0000","last_fullsized":"2024-09-10T23:40:05.066721+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:35:56.962775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066768+0000","last_change":"2024-09-10T23:40:03.366844+0000","last_active":"2024-09-10T23:40:05.066768+0000","last_peered":"2024-09-10T23:40:05.066768+0000","last_clean":"2024-09-10T23:40:05.066768+0000","last_became_active":"2024-09-10T23:40:03.366592+0000","last_became_peered":"2024-09-10T23:40:03.366592+0000","last_unstale":"2024-09-10T23:40:05.066768+0000","last_undegraded":"2024-09-10T23:40:05.066768+0000","last_fullsized":"2024-09-10T23:40:05.066768+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:52.149201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"150'4","reported_seq":19,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346836+0000","last_change":"2024-09-10T23:40:02.357339+0000","last_active":"2024-09-10T23:40:04.346836+0000","last_peered":"2024-09-10T23:40:04.346836+0000","last_clean":"2024-09-10T23:40:04.346836+0000","last_became_active":"2024-09-10T23:40:02.357054+0000","last_became_peered":"2024-09-10T23:40:02.357054+0000","last_unstale":"2024-09-10T23:40:04.346836+0000","last_undegraded":"2024-09-10T23:40:04.346836+0000","last_fullsized":"2024-09-10T23:40:04.346836+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:52.567685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346458+0000","last_change":"2024-09-10T23:40:03.365971+0000","last_active":"2024-09-10T23:40:04.346458+0000","last_peered":"2024-09-10T23:40:04.346458+0000","last_clean":"2024-09-10T23:40:04.346458+0000","last_became_active":"2024-09-10T23:40:03.365725+0000","last_became_peered":"2024-09-10T23:40:03.365725+0000","last_unstale":"2024-09-10T23:40:04.346458+0000","last_undegraded":"2024-09-10T23:40:04.346458+0000","last_fullsized":"2024-09-10T23:40:04.346458+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:44.231823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727889+0000","last_change":"2024-09-10T23:40:02.354357+0000","last_active":"2024-09-10T23:40:05.727889+0000","last_peered":"2024-09-10T23:40:05.727889+0000","last_clean":"2024-09-10T23:40:05.727889+0000","last_became_active":"2024-09-10T23:40:02.354105+0000","last_became_peered":"2024-09-10T23:40:02.354105+0000","last_unstale":"2024-09-10T23:40:05.727889+0000","last_undegraded":"2024-09-10T23:40:05.727889+0000","last_fullsized":"2024-09-10T23:40:05.727889+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:57:49.719773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.983027+0000","last_change":"2024-09-10T23:40:03.354808+0000","last_active":"2024-09-10T23:40:04.983027+0000","last_peered":"2024-09-10T23:40:04.983027+0000","last_clean":"2024-09-10T23:40:04.983027+0000","last_became_active":"2024-09-10T23:40:03.354539+0000","last_became_peered":"2024-09-10T23:40:03.354539+0000","last_unstale":"2024-09-10T23:40:04.983027+0000","last_undegraded":"2024-09-10T23:40:04.983027+0000","last_fullsized":"2024-09-10T23:40:04.983027+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:08.122357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066946+0000","last_change":"2024-09-10T23:40:02.363918+0000","last_active":"2024-09-10T23:40:05.066946+0000","last_peered":"2024-09-10T23:40:05.066946+0000","last_clean":"2024-09-10T23:40:05.066946+0000","last_became_active":"2024-09-10T23:40:02.363627+0000","last_became_peered":"2024-09-10T23:40:02.363627+0000","last_unstale":"2024-09-10T23:40:05.066946+0000","last_undegraded":"2024-09-10T23:40:05.066946+0000","last_fullsized":"2024-09-10T23:40:05.066946+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:53.657508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982919+0000","last_change":"2024-09-10T23:40:03.356438+0000","last_active":"2024-09-10T23:40:04.982919+0000","last_peered":"2024-09-10T23:40:04.982919+0000","last_clean":"2024-09-10T23:40:04.982919+0000","last_became_active":"2024-09-10T23:40:03.356144+0000","last_became_peered":"2024-09-10T23:40:03.356144+0000","last_unstale":"2024-09-10T23:40:04.982919+0000","last_undegraded":"2024-09-10T23:40:04.982919+0000","last_fullsized":"2024-09-10T23:40:04.982919+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:04:47.070469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727868+0000","last_change":"2024-09-10T23:40:02.354874+0000","last_active":"2024-09-10T23:40:05.727868+0000","last_peered":"2024-09-10T23:40:05.727868+0000","last_clean":"2024-09-10T23:40:05.727868+0000","last_became_active":"2024-09-10T23:40:02.354568+0000","last_became_peered":"2024-09-10T23:40:02.354568+0000","last_unstale":"2024-09-10T23:40:05.727868+0000","last_undegraded":"2024-09-10T23:40:05.727868+0000","last_fullsized":"2024-09-10T23:40:05.727868+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:07.668909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734487+0000","last_change":"2024-09-10T23:40:03.369829+0000","last_active":"2024-09-10T23:40:05.734487+0000","last_peered":"2024-09-10T23:40:05.734487+0000","last_clean":"2024-09-10T23:40:05.734487+0000","last_became_active":"2024-09-10T23:40:03.369106+0000","last_became_peered":"2024-09-10T23:40:03.369106+0000","last_unstale":"2024-09-10T23:40:05.734487+0000","last_undegraded":"2024-09-10T23:40:05.734487+0000","last_fullsized":"2024-09-10T23:40:05.734487+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:59.420683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.067206+0000","last_change":"2024-09-10T23:40:02.363569+0000","last_active":"2024-09-10T23:40:05.067206+0000","last_peered":"2024-09-10T23:40:05.067206+0000","last_clean":"2024-09-10T23:40:05.067206+0000","last_became_active":"2024-09-10T23:40:02.362386+0000","last_became_peered":"2024-09-10T23:40:02.362386+0000","last_unstale":"2024-09-10T23:40:05.067206+0000","last_undegraded":"2024-09-10T23:40:05.067206+0000","last_fullsized":"2024-09-10T23:40:05.067206+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:38.787706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066980+0000","last_change":"2024-09-10T23:40:03.362243+0000","last_active":"2024-09-10T23:40:05.066980+0000","last_peered":"2024-09-10T23:40:05.066980+0000","last_clean":"2024-09-10T23:40:05.066980+0000","last_became_active":"2024-09-10T23:40:03.361700+0000","last_became_peered":"2024-09-10T23:40:03.361700+0000","last_unstale":"2024-09-10T23:40:05.066980+0000","last_undegraded":"2024-09-10T23:40:05.066980+0000","last_fullsized":"2024-09-10T23:40:05.066980+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:57.759800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.067416+0000","last_change":"2024-09-10T23:40:02.363592+0000","last_active":"2024-09-10T23:40:05.067416+0000","last_peered":"2024-09-10T23:40:05.067416+0000","last_clean":"2024-09-10T23:40:05.067416+0000","last_became_active":"2024-09-10T23:40:02.363294+0000","last_became_peered":"2024-09-10T23:40:02.363294+0000","last_unstale":"2024-09-10T23:40:05.067416+0000","last_undegraded":"2024-09-10T23:40:05.067416+0000","last_fullsized":"2024-09-10T23:40:05.067416+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:14.181789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341693+0000","last_change":"2024-09-10T23:40:03.363981+0000","last_active":"2024-09-10T23:40:04.341693+0000","last_peered":"2024-09-10T23:40:04.341693+0000","last_clean":"2024-09-10T23:40:04.341693+0000","last_became_active":"2024-09-10T23:40:03.363696+0000","last_became_peered":"2024-09-10T23:40:03.363696+0000","last_unstale":"2024-09-10T23:40:04.341693+0000","last_undegraded":"2024-09-10T23:40:04.341693+0000","last_fullsized":"2024-09-10T23:40:04.341693+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:27.074805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.966387+0000","last_change":"2024-09-10T23:40:02.353255+0000","last_active":"2024-09-10T23:40:04.966387+0000","last_peered":"2024-09-10T23:40:04.966387+0000","last_clean":"2024-09-10T23:40:04.966387+0000","last_became_active":"2024-09-10T23:40:02.352992+0000","last_became_peered":"2024-09-10T23:40:02.352992+0000","last_unstale":"2024-09-10T23:40:04.966387+0000","last_undegraded":"2024-09-10T23:40:04.966387+0000","last_fullsized":"2024-09-10T23:40:04.966387+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:17.250644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345574+0000","last_change":"2024-09-10T23:40:03.358205+0000","last_active":"2024-09-10T23:40:04.345574+0000","last_peered":"2024-09-10T23:40:04.345574+0000","last_clean":"2024-09-10T23:40:04.345574+0000","last_became_active":"2024-09-10T23:40:03.357946+0000","last_became_peered":"2024-09-10T23:40:03.357946+0000","last_unstale":"2024-09-10T23:40:04.345574+0000","last_undegraded":"2024-09-10T23:40:04.345574+0000","last_fullsized":"2024-09-10T23:40:04.345574+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:04.124383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341296+0000","last_change":"2024-09-10T23:40:02.359051+0000","last_active":"2024-09-10T23:40:04.341296+0000","last_peered":"2024-09-10T23:40:04.341296+0000","last_clean":"2024-09-10T23:40:04.341296+0000","last_became_active":"2024-09-10T23:40:02.357335+0000","last_became_peered":"2024-09-10T23:40:02.357335+0000","last_unstale":"2024-09-10T23:40:04.341296+0000","last_undegraded":"2024-09-10T23:40:04.341296+0000","last_fullsized":"2024-09-10T23:40:04.341296+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:36.624712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346668+0000","last_change":"2024-09-10T23:40:03.354222+0000","last_active":"2024-09-10T23:40:04.346668+0000","last_peered":"2024-09-10T23:40:04.346668+0000","last_clean":"2024-09-10T23:40:04.346668+0000","last_became_active":"2024-09-10T23:40:03.353981+0000","last_became_peered":"2024-09-10T23:40:03.353981+0000","last_unstale":"2024-09-10T23:40:04.346668+0000","last_undegraded":"2024-09-10T23:40:04.346668+0000","last_fullsized":"2024-09-10T23:40:04.346668+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:48.915081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346031+0000","last_change":"2024-09-10T23:40:02.359834+0000","last_active":"2024-09-10T23:40:04.346031+0000","last_peered":"2024-09-10T23:40:04.346031+0000","last_clean":"2024-09-10T23:40:04.346031+0000","last_became_active":"2024-09-10T23:40:02.359528+0000","last_became_peered":"2024-09-10T23:40:02.359528+0000","last_unstale":"2024-09-10T23:40:04.346031+0000","last_undegraded":"2024-09-10T23:40:04.346031+0000","last_fullsized":"2024-09-10T23:40:04.346031+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:39.104094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727775+0000","last_change":"2024-09-10T23:40:03.358068+0000","last_active":"2024-09-10T23:40:05.727775+0000","last_peered":"2024-09-10T23:40:05.727775+0000","last_clean":"2024-09-10T23:40:05.727775+0000","last_became_active":"2024-09-10T23:40:03.357794+0000","last_became_peered":"2024-09-10T23:40:03.357794+0000","last_unstale":"2024-09-10T23:40:05.727775+0000","last_undegraded":"2024-09-10T23:40:05.727775+0000","last_fullsized":"2024-09-10T23:40:05.727775+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:47.309787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"150'3","reported_seq":18,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.067082+0000","last_change":"2024-09-10T23:40:02.364953+0000","last_active":"2024-09-10T23:40:05.067082+0000","last_peered":"2024-09-10T23:40:05.067082+0000","last_clean":"2024-09-10T23:40:05.067082+0000","last_became_active":"2024-09-10T23:40:02.364637+0000","last_became_peered":"2024-09-10T23:40:02.364637+0000","last_unstale":"2024-09-10T23:40:05.067082+0000","last_undegraded":"2024-09-10T23:40:05.067082+0000","last_fullsized":"2024-09-10T23:40:05.067082+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:53.213242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346348+0000","last_change":"2024-09-10T23:40:03.353089+0000","last_active":"2024-09-10T23:40:04.346348+0000","last_peered":"2024-09-10T23:40:04.346348+0000","last_clean":"2024-09-10T23:40:04.346348+0000","last_became_active":"2024-09-10T23:40:03.352834+0000","last_became_peered":"2024-09-10T23:40:03.352834+0000","last_unstale":"2024-09-10T23:40:04.346348+0000","last_undegraded":"2024-09-10T23:40:04.346348+0000","last_fullsized":"2024-09-10T23:40:04.346348+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:27.499094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"150'6","reported_seq":21,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066826+0000","last_change":"2024-09-10T23:40:02.361550+0000","last_active":"2024-09-10T23:40:05.066826+0000","last_peered":"2024-09-10T23:40:05.066826+0000","last_clean":"2024-09-10T23:40:05.066826+0000","last_became_active":"2024-09-10T23:40:02.361204+0000","last_became_peered":"2024-09-10T23:40:02.361204+0000","last_unstale":"2024-09-10T23:40:05.066826+0000","last_undegraded":"2024-09-10T23:40:05.066826+0000","last_fullsized":"2024-09-10T23:40:05.066826+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:04:46.064144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346116+0000","last_change":"2024-09-10T23:40:03.365358+0000","last_active":"2024-09-10T23:40:04.346116+0000","last_peered":"2024-09-10T23:40:04.346116+0000","last_clean":"2024-09-10T23:40:04.346116+0000","last_became_active":"2024-09-10T23:40:03.365061+0000","last_became_peered":"2024-09-10T23:40:03.365061+0000","last_unstale":"2024-09-10T23:40:04.346116+0000","last_undegraded":"2024-09-10T23:40:04.346116+0000","last_fullsized":"2024-09-10T23:40:04.346116+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:30.588432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.13","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727720+0000","last_change":"2024-09-10T23:40:02.354529+0000","last_active":"2024-09-10T23:40:05.727720+0000","last_peered":"2024-09-10T23:40:05.727720+0000","last_clean":"2024-09-10T23:40:05.727720+0000","last_became_active":"2024-09-10T23:40:02.354273+0000","last_became_peered":"2024-09-10T23:40:02.354273+0000","last_unstale":"2024-09-10T23:40:05.727720+0000","last_undegraded":"2024-09-10T23:40:05.727720+0000","last_fullsized":"2024-09-10T23:40:05.727720+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:44.907161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.966622+0000","last_change":"2024-09-10T23:40:03.356503+0000","last_active":"2024-09-10T23:40:04.966622+0000","last_peered":"2024-09-10T23:40:04.966622+0000","last_clean":"2024-09-10T23:40:04.966622+0000","last_became_active":"2024-09-10T23:40:03.356243+0000","last_became_peered":"2024-09-10T23:40:03.356243+0000","last_unstale":"2024-09-10T23:40:04.966622+0000","last_undegraded":"2024-09-10T23:40:04.966622+0000","last_fullsized":"2024-09-10T23:40:04.966622+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:41.299187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345762+0000","last_change":"2024-09-10T23:40:02.356793+0000","last_active":"2024-09-10T23:40:04.345762+0000","last_peered":"2024-09-10T23:40:04.345762+0000","last_clean":"2024-09-10T23:40:04.345762+0000","last_became_active":"2024-09-10T23:40:02.356539+0000","last_became_peered":"2024-09-10T23:40:02.356539+0000","last_unstale":"2024-09-10T23:40:04.345762+0000","last_undegraded":"2024-09-10T23:40:04.345762+0000","last_fullsized":"2024-09-10T23:40:04.345762+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:35.356292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727496+0000","last_change":"2024-09-10T23:40:03.358716+0000","last_active":"2024-09-10T23:40:05.727496+0000","last_peered":"2024-09-10T23:40:05.727496+0000","last_clean":"2024-09-10T23:40:05.727496+0000","last_became_active":"2024-09-10T23:40:03.358444+0000","last_became_peered":"2024-09-10T23:40:03.358444+0000","last_unstale":"2024-09-10T23:40:05.727496+0000","last_undegraded":"2024-09-10T23:40:05.727496+0000","last_fullsized":"2024-09-10T23:40:05.727496+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:47.625167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066910+0000","last_change":"2024-09-10T23:40:02.363352+0000","last_active":"2024-09-10T23:40:05.066910+0000","last_peered":"2024-09-10T23:40:05.066910+0000","last_clean":"2024-09-10T23:40:05.066910+0000","last_became_active":"2024-09-10T23:40:02.362332+0000","last_became_peered":"2024-09-10T23:40:02.362332+0000","last_unstale":"2024-09-10T23:40:05.066910+0000","last_undegraded":"2024-09-10T23:40:05.066910+0000","last_fullsized":"2024-09-10T23:40:05.066910+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:53.719176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066564+0000","last_change":"2024-09-10T23:40:03.369222+0000","last_active":"2024-09-10T23:40:05.066564+0000","last_peered":"2024-09-10T23:40:05.066564+0000","last_clean":"2024-09-10T23:40:05.066564+0000","last_became_active":"2024-09-10T23:40:03.368598+0000","last_became_peered":"2024-09-10T23:40:03.368598+0000","last_unstale":"2024-09-10T23:40:05.066564+0000","last_undegraded":"2024-09-10T23:40:05.066564+0000","last_fullsized":"2024-09-10T23:40:05.066564+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:28.728488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"150'8","reported_seq":23,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.966452+0000","last_change":"2024-09-10T23:40:02.357720+0000","last_active":"2024-09-10T23:40:04.966452+0000","last_peered":"2024-09-10T23:40:04.966452+0000","last_clean":"2024-09-10T23:40:04.966452+0000","last_became_active":"2024-09-10T23:40:02.357265+0000","last_became_peered":"2024-09-10T23:40:02.357265+0000","last_unstale":"2024-09-10T23:40:04.966452+0000","last_undegraded":"2024-09-10T23:40:04.966452+0000","last_fullsized":"2024-09-10T23:40:04.966452+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:44.907161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345859+0000","last_change":"2024-09-10T23:40:03.347613+0000","last_active":"2024-09-10T23:40:04.345859+0000","last_peered":"2024-09-10T23:40:04.345859+0000","last_clean":"2024-09-10T23:40:04.345859+0000","last_became_active":"2024-09-10T23:40:03.347314+0000","last_became_peered":"2024-09-10T23:40:03.347314+0000","last_unstale":"2024-09-10T23:40:04.345859+0000","last_undegraded":"2024-09-10T23:40:04.345859+0000","last_fullsized":"2024-09-10T23:40:04.345859+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:24.543980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.f","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.067281+0000","last_change":"2024-09-10T23:40:02.367006+0000","last_active":"2024-09-10T23:40:05.067281+0000","last_peered":"2024-09-10T23:40:05.067281+0000","last_clean":"2024-09-10T23:40:05.067281+0000","last_became_active":"2024-09-10T23:40:02.366761+0000","last_became_peered":"2024-09-10T23:40:02.366761+0000","last_unstale":"2024-09-10T23:40:05.067281+0000","last_undegraded":"2024-09-10T23:40:05.067281+0000","last_fullsized":"2024-09-10T23:40:05.067281+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:44.949937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.728027+0000","last_change":"2024-09-10T23:40:03.367960+0000","last_active":"2024-09-10T23:40:05.728027+0000","last_peered":"2024-09-10T23:40:05.728027+0000","last_clean":"2024-09-10T23:40:05.728027+0000","last_became_active":"2024-09-10T23:40:03.367643+0000","last_became_peered":"2024-09-10T23:40:03.367643+0000","last_unstale":"2024-09-10T23:40:05.728027+0000","last_undegraded":"2024-09-10T23:40:05.728027+0000","last_fullsized":"2024-09-10T23:40:05.728027+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:37.078737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734569+0000","last_change":"2024-09-10T23:40:02.350625+0000","last_active":"2024-09-10T23:40:05.734569+0000","last_peered":"2024-09-10T23:40:05.734569+0000","last_clean":"2024-09-10T23:40:05.734569+0000","last_became_active":"2024-09-10T23:40:02.349954+0000","last_became_peered":"2024-09-10T23:40:02.349954+0000","last_unstale":"2024-09-10T23:40:05.734569+0000","last_undegraded":"2024-09-10T23:40:05.734569+0000","last_fullsized":"2024-09-10T23:40:05.734569+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:14.193750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.983122+0000","last_change":"2024-09-10T23:40:03.362463+0000","last_active":"2024-09-10T23:40:04.983122+0000","last_peered":"2024-09-10T23:40:04.983122+0000","last_clean":"2024-09-10T23:40:04.983122+0000","last_became_active":"2024-09-10T23:40:03.362183+0000","last_became_peered":"2024-09-10T23:40:03.362183+0000","last_unstale":"2024-09-10T23:40:04.983122+0000","last_undegraded":"2024-09-10T23:40:04.983122+0000","last_fullsized":"2024-09-10T23:40:04.983122+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:54.725502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341143+0000","last_change":"2024-09-10T23:40:02.355908+0000","last_active":"2024-09-10T23:40:04.341143+0000","last_peered":"2024-09-10T23:40:04.341143+0000","last_clean":"2024-09-10T23:40:04.341143+0000","last_became_active":"2024-09-10T23:40:02.355653+0000","last_became_peered":"2024-09-10T23:40:02.355653+0000","last_unstale":"2024-09-10T23:40:04.341143+0000","last_undegraded":"2024-09-10T23:40:04.341143+0000","last_fullsized":"2024-09-10T23:40:04.341143+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:21.969940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727849+0000","last_change":"2024-09-10T23:40:03.353933+0000","last_active":"2024-09-10T23:40:05.727849+0000","last_peered":"2024-09-10T23:40:05.727849+0000","last_clean":"2024-09-10T23:40:05.727849+0000","last_became_active":"2024-09-10T23:40:03.353680+0000","last_became_peered":"2024-09-10T23:40:03.353680+0000","last_unstale":"2024-09-10T23:40:05.727849+0000","last_undegraded":"2024-09-10T23:40:05.727849+0000","last_fullsized":"2024-09-10T23:40:05.727849+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:05.717713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341266+0000","last_change":"2024-09-10T23:40:02.352194+0000","last_active":"2024-09-10T23:40:04.341266+0000","last_peered":"2024-09-10T23:40:04.341266+0000","last_clean":"2024-09-10T23:40:04.341266+0000","last_became_active":"2024-09-10T23:40:02.351903+0000","last_became_peered":"2024-09-10T23:40:02.351903+0000","last_unstale":"2024-09-10T23:40:04.341266+0000","last_undegraded":"2024-09-10T23:40:04.341266+0000","last_fullsized":"2024-09-10T23:40:04.341266+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:03.963558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734783+0000","last_change":"2024-09-10T23:40:03.356347+0000","last_active":"2024-09-10T23:40:05.734783+0000","last_peered":"2024-09-10T23:40:05.734783+0000","last_clean":"2024-09-10T23:40:05.734783+0000","last_became_active":"2024-09-10T23:40:03.356044+0000","last_became_peered":"2024-09-10T23:40:03.356044+0000","last_unstale":"2024-09-10T23:40:05.734783+0000","last_undegraded":"2024-09-10T23:40:05.734783+0000","last_fullsized":"2024-09-10T23:40:05.734783+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:37.631038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.067259+0000","last_change":"2024-09-10T23:40:02.360470+0000","last_active":"2024-09-10T23:40:05.067259+0000","last_peered":"2024-09-10T23:40:05.067259+0000","last_clean":"2024-09-10T23:40:05.067259+0000","last_became_active":"2024-09-10T23:40:02.360193+0000","last_became_peered":"2024-09-10T23:40:02.360193+0000","last_unstale":"2024-09-10T23:40:05.067259+0000","last_undegraded":"2024-09-10T23:40:05.067259+0000","last_fullsized":"2024-09-10T23:40:05.067259+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:55.532229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.966555+0000","last_change":"2024-09-10T23:40:03.354495+0000","last_active":"2024-09-10T23:40:04.966555+0000","last_peered":"2024-09-10T23:40:04.966555+0000","last_clean":"2024-09-10T23:40:04.966555+0000","last_became_active":"2024-09-10T23:40:03.354248+0000","last_became_peered":"2024-09-10T23:40:03.354248+0000","last_unstale":"2024-09-10T23:40:04.966555+0000","last_undegraded":"2024-09-10T23:40:04.966555+0000","last_fullsized":"2024-09-10T23:40:04.966555+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:04.666223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734888+0000","last_change":"2024-09-10T23:40:02.353010+0000","last_active":"2024-09-10T23:40:05.734888+0000","last_peered":"2024-09-10T23:40:05.734888+0000","last_clean":"2024-09-10T23:40:05.734888+0000","last_became_active":"2024-09-10T23:40:02.352736+0000","last_became_peered":"2024-09-10T23:40:02.352736+0000","last_unstale":"2024-09-10T23:40:05.734888+0000","last_undegraded":"2024-09-10T23:40:05.734888+0000","last_fullsized":"2024-09-10T23:40:05.734888+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:32.633723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341145+0000","last_change":"2024-09-10T23:40:03.350265+0000","last_active":"2024-09-10T23:40:04.341145+0000","last_peered":"2024-09-10T23:40:04.341145+0000","last_clean":"2024-09-10T23:40:04.341145+0000","last_became_active":"2024-09-10T23:40:03.349982+0000","last_became_peered":"2024-09-10T23:40:03.349982+0000","last_unstale":"2024-09-10T23:40:04.341145+0000","last_undegraded":"2024-09-10T23:40:04.341145+0000","last_fullsized":"2024-09-10T23:40:04.341145+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:58.791371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734280+0000","last_change":"2024-09-10T23:40:02.356421+0000","last_active":"2024-09-10T23:40:05.734280+0000","last_peered":"2024-09-10T23:40:05.734280+0000","last_clean":"2024-09-10T23:40:05.734280+0000","last_became_active":"2024-09-10T23:40:02.355527+0000","last_became_peered":"2024-09-10T23:40:02.355527+0000","last_unstale":"2024-09-10T23:40:05.734280+0000","last_undegraded":"2024-09-10T23:40:05.734280+0000","last_fullsized":"2024-09-10T23:40:05.734280+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:25.830166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345052+0000","last_change":"2024-09-10T23:40:03.351934+0000","last_active":"2024-09-10T23:40:04.345052+0000","last_peered":"2024-09-10T23:40:04.345052+0000","last_clean":"2024-09-10T23:40:04.345052+0000","last_became_active":"2024-09-10T23:40:03.351636+0000","last_became_peered":"2024-09-10T23:40:03.351636+0000","last_unstale":"2024-09-10T23:40:04.345052+0000","last_undegraded":"2024-09-10T23:40:04.345052+0000","last_fullsized":"2024-09-10T23:40:04.345052+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:24.962965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341061+0000","last_change":"2024-09-10T23:40:02.355475+0000","last_active":"2024-09-10T23:40:04.341061+0000","last_peered":"2024-09-10T23:40:04.341061+0000","last_clean":"2024-09-10T23:40:04.341061+0000","last_became_active":"2024-09-10T23:40:02.355204+0000","last_became_peered":"2024-09-10T23:40:02.355204+0000","last_unstale":"2024-09-10T23:40:04.341061+0000","last_undegraded":"2024-09-10T23:40:04.341061+0000","last_fullsized":"2024-09-10T23:40:04.341061+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:40.101644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345844+0000","last_change":"2024-09-10T23:40:03.352942+0000","last_active":"2024-09-10T23:40:04.345844+0000","last_peered":"2024-09-10T23:40:04.345844+0000","last_clean":"2024-09-10T23:40:04.345844+0000","last_became_active":"2024-09-10T23:40:03.352670+0000","last_became_peered":"2024-09-10T23:40:03.352670+0000","last_unstale":"2024-09-10T23:40:04.345844+0000","last_undegraded":"2024-09-10T23:40:04.345844+0000","last_fullsized":"2024-09-10T23:40:04.345844+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:47.625167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341417+0000","last_change":"2024-09-10T23:40:02.352054+0000","last_active":"2024-09-10T23:40:04.341417+0000","last_peered":"2024-09-10T23:40:04.341417+0000","last_clean":"2024-09-10T23:40:04.341417+0000","last_became_active":"2024-09-10T23:40:02.351786+0000","last_became_peered":"2024-09-10T23:40:02.351786+0000","last_unstale":"2024-09-10T23:40:04.341417+0000","last_undegraded":"2024-09-10T23:40:04.341417+0000","last_fullsized":"2024-09-10T23:40:04.341417+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:17.235272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346887+0000","last_change":"2024-09-10T23:40:03.355026+0000","last_active":"2024-09-10T23:40:04.346887+0000","last_peered":"2024-09-10T23:40:04.346887+0000","last_clean":"2024-09-10T23:40:04.346887+0000","last_became_active":"2024-09-10T23:40:03.354682+0000","last_became_peered":"2024-09-10T23:40:03.354682+0000","last_unstale":"2024-09-10T23:40:04.346887+0000","last_undegraded":"2024-09-10T23:40:04.346887+0000","last_fullsized":"2024-09-10T23:40:04.346887+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:05.717713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"151'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346079+0000","last_change":"2024-09-10T23:40:02.360781+0000","last_active":"2024-09-10T23:40:04.346079+0000","last_peered":"2024-09-10T23:40:04.346079+0000","last_clean":"2024-09-10T23:40:04.346079+0000","last_became_active":"2024-09-10T23:40:02.360156+0000","last_became_peered":"2024-09-10T23:40:02.360156+0000","last_unstale":"2024-09-10T23:40:04.346079+0000","last_undegraded":"2024-09-10T23:40:04.346079+0000","last_fullsized":"2024-09-10T23:40:04.346079+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:29.712948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345900+0000","last_change":"2024-09-10T23:40:03.370366+0000","last_active":"2024-09-10T23:40:04.345900+0000","last_peered":"2024-09-10T23:40:04.345900+0000","last_clean":"2024-09-10T23:40:04.345900+0000","last_became_active":"2024-09-10T23:40:03.370110+0000","last_became_peered":"2024-09-10T23:40:03.370110+0000","last_unstale":"2024-09-10T23:40:04.345900+0000","last_undegraded":"2024-09-10T23:40:04.345900+0000","last_fullsized":"2024-09-10T23:40:04.345900+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:12.470964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346613+0000","last_change":"2024-09-10T23:40:02.359868+0000","last_active":"2024-09-10T23:40:04.346613+0000","last_peered":"2024-09-10T23:40:04.346613+0000","last_clean":"2024-09-10T23:40:04.346613+0000","last_became_active":"2024-09-10T23:40:02.359600+0000","last_became_peered":"2024-09-10T23:40:02.359600+0000","last_unstale":"2024-09-10T23:40:04.346613+0000","last_undegraded":"2024-09-10T23:40:04.346613+0000","last_fullsized":"2024-09-10T23:40:04.346613+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:11.452645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066882+0000","last_change":"2024-09-10T23:40:03.368479+0000","last_active":"2024-09-10T23:40:05.066882+0000","last_peered":"2024-09-10T23:40:05.066882+0000","last_clean":"2024-09-10T23:40:05.066882+0000","last_became_active":"2024-09-10T23:40:03.368202+0000","last_became_peered":"2024-09-10T23:40:03.368202+0000","last_unstale":"2024-09-10T23:40:05.066882+0000","last_undegraded":"2024-09-10T23:40:05.066882+0000","last_fullsized":"2024-09-10T23:40:05.066882+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:03.720617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.067081+0000","last_change":"2024-09-10T23:40:02.362452+0000","last_active":"2024-09-10T23:40:05.067081+0000","last_peered":"2024-09-10T23:40:05.067081+0000","last_clean":"2024-09-10T23:40:05.067081+0000","last_became_active":"2024-09-10T23:40:02.362039+0000","last_became_peered":"2024-09-10T23:40:02.362039+0000","last_unstale":"2024-09-10T23:40:05.067081+0000","last_undegraded":"2024-09-10T23:40:05.067081+0000","last_fullsized":"2024-09-10T23:40:05.067081+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:07.116031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982677+0000","last_change":"2024-09-10T23:40:03.361292+0000","last_active":"2024-09-10T23:40:04.982677+0000","last_peered":"2024-09-10T23:40:04.982677+0000","last_clean":"2024-09-10T23:40:04.982677+0000","last_became_active":"2024-09-10T23:40:03.360859+0000","last_became_peered":"2024-09-10T23:40:03.360859+0000","last_unstale":"2024-09-10T23:40:04.982677+0000","last_undegraded":"2024-09-10T23:40:04.982677+0000","last_fullsized":"2024-09-10T23:40:04.982677+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:15.656273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982929+0000","last_change":"2024-09-10T23:40:02.356946+0000","last_active":"2024-09-10T23:40:04.982929+0000","last_peered":"2024-09-10T23:40:04.982929+0000","last_clean":"2024-09-10T23:40:04.982929+0000","last_became_active":"2024-09-10T23:40:02.356690+0000","last_became_peered":"2024-09-10T23:40:02.356690+0000","last_unstale":"2024-09-10T23:40:04.982929+0000","last_undegraded":"2024-09-10T23:40:04.982929+0000","last_fullsized":"2024-09-10T23:40:04.982929+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:09:45.872598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341401+0000","last_change":"2024-09-10T23:40:03.354787+0000","last_active":"2024-09-10T23:40:04.341401+0000","last_peered":"2024-09-10T23:40:04.341401+0000","last_clean":"2024-09-10T23:40:04.341401+0000","last_became_active":"2024-09-10T23:40:03.352966+0000","last_became_peered":"2024-09-10T23:40:03.352966+0000","last_unstale":"2024-09-10T23:40:04.341401+0000","last_undegraded":"2024-09-10T23:40:04.341401+0000","last_fullsized":"2024-09-10T23:40:04.341401+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:26.409688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982841+0000","last_change":"2024-09-10T23:40:02.349369+0000","last_active":"2024-09-10T23:40:04.982841+0000","last_peered":"2024-09-10T23:40:04.982841+0000","last_clean":"2024-09-10T23:40:04.982841+0000","last_became_active":"2024-09-10T23:40:02.349059+0000","last_became_peered":"2024-09-10T23:40:02.349059+0000","last_unstale":"2024-09-10T23:40:04.982841+0000","last_undegraded":"2024-09-10T23:40:04.982841+0000","last_fullsized":"2024-09-10T23:40:04.982841+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:50.367930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982700+0000","last_change":"2024-09-10T23:40:03.354960+0000","last_active":"2024-09-10T23:40:04.982700+0000","last_peered":"2024-09-10T23:40:04.982700+0000","last_clean":"2024-09-10T23:40:04.982700+0000","last_became_active":"2024-09-10T23:40:03.354675+0000","last_became_peered":"2024-09-10T23:40:03.354675+0000","last_unstale":"2024-09-10T23:40:04.982700+0000","last_undegraded":"2024-09-10T23:40:04.982700+0000","last_fullsized":"2024-09-10T23:40:04.982700+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:24.173026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982726+0000","last_change":"2024-09-10T23:40:02.349452+0000","last_active":"2024-09-10T23:40:04.982726+0000","last_peered":"2024-09-10T23:40:04.982726+0000","last_clean":"2024-09-10T23:40:04.982726+0000","last_became_active":"2024-09-10T23:40:02.349132+0000","last_became_peered":"2024-09-10T23:40:02.349132+0000","last_unstale":"2024-09-10T23:40:04.982726+0000","last_undegraded":"2024-09-10T23:40:04.982726+0000","last_fullsized":"2024-09-10T23:40:04.982726+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:54.008345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066783+0000","last_change":"2024-09-10T23:40:03.362881+0000","last_active":"2024-09-10T23:40:05.066783+0000","last_peered":"2024-09-10T23:40:05.066783+0000","last_clean":"2024-09-10T23:40:05.066783+0000","last_became_active":"2024-09-10T23:40:03.362635+0000","last_became_peered":"2024-09-10T23:40:03.362635+0000","last_unstale":"2024-09-10T23:40:05.066783+0000","last_undegraded":"2024-09-10T23:40:05.066783+0000","last_fullsized":"2024-09-10T23:40:05.066783+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:04.124383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734506+0000","last_change":"2024-09-10T23:40:02.360113+0000","last_active":"2024-09-10T23:40:05.734506+0000","last_peered":"2024-09-10T23:40:05.734506+0000","last_clean":"2024-09-10T23:40:05.734506+0000","last_became_active":"2024-09-10T23:40:02.359801+0000","last_became_peered":"2024-09-10T23:40:02.359801+0000","last_unstale":"2024-09-10T23:40:05.734506+0000","last_undegraded":"2024-09-10T23:40:05.734506+0000","last_fullsized":"2024-09-10T23:40:05.734506+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:32.744092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734537+0000","last_change":"2024-09-10T23:40:03.359682+0000","last_active":"2024-09-10T23:40:05.734537+0000","last_peered":"2024-09-10T23:40:05.734537+0000","last_clean":"2024-09-10T23:40:05.734537+0000","last_became_active":"2024-09-10T23:40:03.359397+0000","last_became_peered":"2024-09-10T23:40:03.359397+0000","last_unstale":"2024-09-10T23:40:05.734537+0000","last_undegraded":"2024-09-10T23:40:05.734537+0000","last_fullsized":"2024-09-10T23:40:05.734537+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:08.469491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727539+0000","last_change":"2024-09-10T23:40:02.354814+0000","last_active":"2024-09-10T23:40:05.727539+0000","last_peered":"2024-09-10T23:40:05.727539+0000","last_clean":"2024-09-10T23:40:05.727539+0000","last_became_active":"2024-09-10T23:40:02.354579+0000","last_became_peered":"2024-09-10T23:40:02.354579+0000","last_unstale":"2024-09-10T23:40:05.727539+0000","last_undegraded":"2024-09-10T23:40:05.727539+0000","last_fullsized":"2024-09-10T23:40:05.727539+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:17.250644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":375,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982735+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:40:04.982735+0000","last_peered":"2024-09-10T23:40:04.982735+0000","last_clean":"2024-09-10T23:40:04.982735+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:40:04.982735+0000","last_undegraded":"2024-09-10T23:40:04.982735+0000","last_fullsized":"2024-09-10T23:40:04.982735+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509537,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33060,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1916928,"data_stored":838132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542240,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1900544,"data_stored":836498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509536,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33632,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338208,"statfs":{"total":96636764160,"available":96602324992,"internally_reserved":0,"allocated":2498560,"data_stored":1427380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509534,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1900544,"data_stored":836442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509535,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":838090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509536,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37204,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334636,"statfs":{"total":96636764160,"available":96598667264,"internally_reserved":0,"allocated":1900544,"data_stored":836478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509535,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37784,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334056,"statfs":{"total":96636764160,"available":96598073344,"internally_reserved":0,"allocated":2490368,"data_stored":1426776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509535,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37208,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334632,"statfs":{"total":96636764160,"available":96598663168,"internally_reserved":0,"allocated":1904640,"data_stored":837024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:40:12.072 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:40:12.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.219+0000 7f9027a00640 1 -- 172.21.15.94:0/1915654046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902006f600 msgr2=0x7f9020072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:12.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.219+0000 7f9027a00640 1 --2- 172.21.15.94:0/1915654046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902006f600 0x7f9020072610 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f902006ff10 tx=0x7f900c01c830 comp rx=0 tx=0).stop 2024-09-10T23:40:12.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.219+0000 7f9027a00640 1 -- 172.21.15.94:0/1915654046 shutdown_connections 2024-09-10T23:40:12.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.219+0000 7f9027a00640 1 --2- 172.21.15.94:0/1915654046 >> v2:172.21.15.181:3300/0 conn(0x7f9020070150 0x7f90200b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.219+0000 7f9027a00640 1 --2- 172.21.15.94:0/1915654046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f902006f600 0x7f9020072610 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.219+0000 7f9027a00640 1 -- 172.21.15.94:0/1915654046 >> 172.21.15.94:0/1915654046 conn(0x7f902006cdf0 msgr2=0x7f902006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:12.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.219+0000 7f9027a00640 1 -- 172.21.15.94:0/1915654046 shutdown_connections 2024-09-10T23:40:12.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.219+0000 7f9027a00640 1 -- 172.21.15.94:0/1915654046 wait complete. 2024-09-10T23:40:12.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f9027a00640 1 Processor -- start 2024-09-10T23:40:12.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f9027a00640 1 -- start start 2024-09-10T23:40:12.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f9027a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f902006f600 0x7f90200c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:12.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f9027a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9020070150 0x7f9020152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:12.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f9027a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9020157c60 0x7f9020150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:12.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f9027a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f90200d00c0 con 0x7f9020070150 2024-09-10T23:40:12.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f9027a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f90200cff40 con 0x7f9020157c60 2024-09-10T23:40:12.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f9027a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f90200d0240 con 0x7f902006f600 2024-09-10T23:40:12.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f901fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9020070150 0x7f9020152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:12.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f901fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9020070150 0x7f9020152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34666/0 (socket says 172.21.15.94:34666) 2024-09-10T23:40:12.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f901fe00640 1 -- 172.21.15.94:0/661358770 learned_addr learned my addr 172.21.15.94:0/661358770 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:12.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f9025000640 1 --2- 172.21.15.94:0/661358770 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f902006f600 0x7f90200c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:12.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f9025a00640 1 --2- 172.21.15.94:0/661358770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9020157c60 0x7f9020150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:12.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f901fe00640 1 -- 172.21.15.94:0/661358770 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f902006f600 msgr2=0x7f90200c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:12.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f901fe00640 1 --2- 172.21.15.94:0/661358770 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f902006f600 0x7f90200c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f901fe00640 1 -- 172.21.15.94:0/661358770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9020157c60 msgr2=0x7f9020150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:12.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f901fe00640 1 --2- 172.21.15.94:0/661358770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9020157c60 0x7f9020150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f901fe00640 1 -- 172.21.15.94:0/661358770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90200c4e80 con 0x7f9020070150 2024-09-10T23:40:12.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f9025a00640 1 --2- 172.21.15.94:0/661358770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9020157c60 0x7f9020150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:12.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f901fe00640 1 --2- 172.21.15.94:0/661358770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9020070150 0x7f9020152520 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f900c0b0040 tx=0x7f900c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:12.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f901cc00640 1 -- 172.21.15.94:0/661358770 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900c0b2040 con 0x7f9020070150 2024-09-10T23:40:12.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f901cc00640 1 -- 172.21.15.94:0/661358770 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f900c005020 con 0x7f9020070150 2024-09-10T23:40:12.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f901cc00640 1 -- 172.21.15.94:0/661358770 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900c005470 con 0x7f9020070150 2024-09-10T23:40:12.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f9027a00640 1 -- 172.21.15.94:0/661358770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9020151d40 con 0x7f9020070150 2024-09-10T23:40:12.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.223+0000 7f9027a00640 1 -- 172.21.15.94:0/661358770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90200c5640 con 0x7f9020070150 2024-09-10T23:40:12.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.227+0000 7f9027a00640 1 -- 172.21.15.94:0/661358770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9020072610 con 0x7f9020070150 2024-09-10T23:40:12.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.227+0000 7f901cc00640 1 -- 172.21.15.94:0/661358770 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f900c0046d0 con 0x7f9020070150 2024-09-10T23:40:12.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.227+0000 7f901cc00640 1 --2- 172.21.15.94:0/661358770 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9000072e50 0x7f9000075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:12.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.227+0000 7f901cc00640 1 -- 172.21.15.94:0/661358770 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f900c12dda0 con 0x7f9020070150 2024-09-10T23:40:12.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.227+0000 7f9025000640 1 --2- 172.21.15.94:0/661358770 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9000072e50 0x7f9000075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:12.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.231+0000 7f9025000640 1 --2- 172.21.15.94:0/661358770 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9000072e50 0x7f9000075310 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9010001cf0 tx=0x7f9010009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:12.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.231+0000 7f901cc00640 1 -- 172.21.15.94:0/661358770 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f900c0164b0 con 0x7f9020070150 2024-09-10T23:40:12.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.395+0000 7f9027a00640 1 -- 172.21.15.94:0/661358770 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9020154d60 con 0x7f9000072e50 2024-09-10T23:40:12.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.399+0000 7f901cc00640 1 -- 172.21.15.94:0/661358770 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7f9020154d60 con 0x7f9000072e50 2024-09-10T23:40:12.407 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:40:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.403+0000 7f9027a00640 1 -- 172.21.15.94:0/661358770 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9000072e50 msgr2=0x7f9000075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.403+0000 7f9027a00640 1 --2- 172.21.15.94:0/661358770 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9000072e50 0x7f9000075310 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9010001cf0 tx=0x7f9010009040 comp rx=0 tx=0).stop 2024-09-10T23:40:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.403+0000 7f9027a00640 1 -- 172.21.15.94:0/661358770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9020070150 msgr2=0x7f9020152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.403+0000 7f9027a00640 1 --2- 172.21.15.94:0/661358770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9020070150 0x7f9020152520 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f900c0b0040 tx=0x7f900c004040 comp rx=0 tx=0).stop 2024-09-10T23:40:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.403+0000 7f9027a00640 1 -- 172.21.15.94:0/661358770 shutdown_connections 2024-09-10T23:40:12.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.403+0000 7f9027a00640 1 --2- 172.21.15.94:0/661358770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9020157c60 0x7f9020150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.403+0000 7f9027a00640 1 --2- 172.21.15.94:0/661358770 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9000072e50 0x7f9000075310 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.403+0000 7f9027a00640 1 --2- 172.21.15.94:0/661358770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9020070150 0x7f9020152520 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.403+0000 7f9027a00640 1 --2- 172.21.15.94:0/661358770 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f902006f600 0x7f90200c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.403+0000 7f9027a00640 1 -- 172.21.15.94:0/661358770 >> 172.21.15.94:0/661358770 conn(0x7f902006cdf0 msgr2=0x7f902006eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:12.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.403+0000 7f9027a00640 1 -- 172.21.15.94:0/661358770 shutdown_connections 2024-09-10T23:40:12.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.403+0000 7f9027a00640 1 -- 172.21.15.94:0/661358770 wait complete. 2024-09-10T23:40:12.428 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-10T23:40:10.779278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282196,"kb_used_data":16040,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692524,"statfs":{"total":773094113280,"available":772805144576,"internally_reserved":0,"allocated":16424960,"data_stored":7876820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002018"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341333+0000","last_change":"2024-09-10T23:40:03.347799+0000","last_active":"2024-09-10T23:40:04.341333+0000","last_peered":"2024-09-10T23:40:04.341333+0000","last_clean":"2024-09-10T23:40:04.341333+0000","last_became_active":"2024-09-10T23:40:03.347210+0000","last_became_peered":"2024-09-10T23:40:03.347210+0000","last_unstale":"2024-09-10T23:40:04.341333+0000","last_undegraded":"2024-09-10T23:40:04.341333+0000","last_fullsized":"2024-09-10T23:40:04.341333+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:36:23.105511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.983204+0000","last_change":"2024-09-10T23:40:03.355190+0000","last_active":"2024-09-10T23:40:04.983204+0000","last_peered":"2024-09-10T23:40:04.983204+0000","last_clean":"2024-09-10T23:40:04.983204+0000","last_became_active":"2024-09-10T23:40:03.354908+0000","last_became_peered":"2024-09-10T23:40:03.354908+0000","last_unstale":"2024-09-10T23:40:04.983204+0000","last_undegraded":"2024-09-10T23:40:04.983204+0000","last_fullsized":"2024-09-10T23:40:04.983204+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:54.609589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346251+0000","last_change":"2024-09-10T23:40:03.356963+0000","last_active":"2024-09-10T23:40:04.346251+0000","last_peered":"2024-09-10T23:40:04.346251+0000","last_clean":"2024-09-10T23:40:04.346251+0000","last_became_active":"2024-09-10T23:40:03.356657+0000","last_became_peered":"2024-09-10T23:40:03.356657+0000","last_unstale":"2024-09-10T23:40:04.346251+0000","last_undegraded":"2024-09-10T23:40:04.346251+0000","last_fullsized":"2024-09-10T23:40:04.346251+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:06.383906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982892+0000","last_change":"2024-09-10T23:40:03.358164+0000","last_active":"2024-09-10T23:40:04.982892+0000","last_peered":"2024-09-10T23:40:04.982892+0000","last_clean":"2024-09-10T23:40:04.982892+0000","last_became_active":"2024-09-10T23:40:03.357896+0000","last_became_peered":"2024-09-10T23:40:03.357896+0000","last_unstale":"2024-09-10T23:40:04.982892+0000","last_undegraded":"2024-09-10T23:40:04.982892+0000","last_fullsized":"2024-09-10T23:40:04.982892+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:30.035176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066627+0000","last_change":"2024-09-10T23:40:03.364653+0000","last_active":"2024-09-10T23:40:05.066627+0000","last_peered":"2024-09-10T23:40:05.066627+0000","last_clean":"2024-09-10T23:40:05.066627+0000","last_became_active":"2024-09-10T23:40:03.362003+0000","last_became_peered":"2024-09-10T23:40:03.362003+0000","last_unstale":"2024-09-10T23:40:05.066627+0000","last_undegraded":"2024-09-10T23:40:05.066627+0000","last_fullsized":"2024-09-10T23:40:05.066627+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:54.442244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734485+0000","last_change":"2024-09-10T23:40:03.370116+0000","last_active":"2024-09-10T23:40:05.734485+0000","last_peered":"2024-09-10T23:40:05.734485+0000","last_clean":"2024-09-10T23:40:05.734485+0000","last_became_active":"2024-09-10T23:40:03.369558+0000","last_became_peered":"2024-09-10T23:40:03.369558+0000","last_unstale":"2024-09-10T23:40:05.734485+0000","last_undegraded":"2024-09-10T23:40:05.734485+0000","last_fullsized":"2024-09-10T23:40:05.734485+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:01.665066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.966555+0000","last_change":"2024-09-10T23:40:03.359832+0000","last_active":"2024-09-10T23:40:04.966555+0000","last_peered":"2024-09-10T23:40:04.966555+0000","last_clean":"2024-09-10T23:40:04.966555+0000","last_became_active":"2024-09-10T23:40:03.359524+0000","last_became_peered":"2024-09-10T23:40:03.359524+0000","last_unstale":"2024-09-10T23:40:04.966555+0000","last_undegraded":"2024-09-10T23:40:04.966555+0000","last_fullsized":"2024-09-10T23:40:04.966555+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:55:51.627429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066411+0000","last_change":"2024-09-10T23:40:03.367253+0000","last_active":"2024-09-10T23:40:05.066411+0000","last_peered":"2024-09-10T23:40:05.066411+0000","last_clean":"2024-09-10T23:40:05.066411+0000","last_became_active":"2024-09-10T23:40:03.366683+0000","last_became_peered":"2024-09-10T23:40:03.366683+0000","last_unstale":"2024-09-10T23:40:05.066411+0000","last_undegraded":"2024-09-10T23:40:05.066411+0000","last_fullsized":"2024-09-10T23:40:05.066411+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:28.572221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.983109+0000","last_change":"2024-09-10T23:40:03.359302+0000","last_active":"2024-09-10T23:40:04.983109+0000","last_peered":"2024-09-10T23:40:04.983109+0000","last_clean":"2024-09-10T23:40:04.983109+0000","last_became_active":"2024-09-10T23:40:03.358968+0000","last_became_peered":"2024-09-10T23:40:03.358968+0000","last_unstale":"2024-09-10T23:40:04.983109+0000","last_undegraded":"2024-09-10T23:40:04.983109+0000","last_fullsized":"2024-09-10T23:40:04.983109+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:54.663834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345613+0000","last_change":"2024-09-10T23:40:03.354046+0000","last_active":"2024-09-10T23:40:04.345613+0000","last_peered":"2024-09-10T23:40:04.345613+0000","last_clean":"2024-09-10T23:40:04.345613+0000","last_became_active":"2024-09-10T23:40:03.352447+0000","last_became_peered":"2024-09-10T23:40:03.352447+0000","last_unstale":"2024-09-10T23:40:04.345613+0000","last_undegraded":"2024-09-10T23:40:04.345613+0000","last_fullsized":"2024-09-10T23:40:04.345613+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:02.595009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341254+0000","last_change":"2024-09-10T23:40:03.354408+0000","last_active":"2024-09-10T23:40:04.341254+0000","last_peered":"2024-09-10T23:40:04.341254+0000","last_clean":"2024-09-10T23:40:04.341254+0000","last_became_active":"2024-09-10T23:40:03.354148+0000","last_became_peered":"2024-09-10T23:40:03.354148+0000","last_unstale":"2024-09-10T23:40:04.341254+0000","last_undegraded":"2024-09-10T23:40:04.341254+0000","last_fullsized":"2024-09-10T23:40:04.341254+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:54.112024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727703+0000","last_change":"2024-09-10T23:40:03.353432+0000","last_active":"2024-09-10T23:40:05.727703+0000","last_peered":"2024-09-10T23:40:05.727703+0000","last_clean":"2024-09-10T23:40:05.727703+0000","last_became_active":"2024-09-10T23:40:03.353187+0000","last_became_peered":"2024-09-10T23:40:03.353187+0000","last_unstale":"2024-09-10T23:40:05.727703+0000","last_undegraded":"2024-09-10T23:40:05.727703+0000","last_fullsized":"2024-09-10T23:40:05.727703+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:13.692433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341418+0000","last_change":"2024-09-10T23:40:03.354377+0000","last_active":"2024-09-10T23:40:04.341418+0000","last_peered":"2024-09-10T23:40:04.341418+0000","last_clean":"2024-09-10T23:40:04.341418+0000","last_became_active":"2024-09-10T23:40:03.354110+0000","last_became_peered":"2024-09-10T23:40:03.354110+0000","last_unstale":"2024-09-10T23:40:04.341418+0000","last_undegraded":"2024-09-10T23:40:04.341418+0000","last_fullsized":"2024-09-10T23:40:04.341418+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:10.175646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982798+0000","last_change":"2024-09-10T23:40:03.357792+0000","last_active":"2024-09-10T23:40:04.982798+0000","last_peered":"2024-09-10T23:40:04.982798+0000","last_clean":"2024-09-10T23:40:04.982798+0000","last_became_active":"2024-09-10T23:40:03.357534+0000","last_became_peered":"2024-09-10T23:40:03.357534+0000","last_unstale":"2024-09-10T23:40:04.982798+0000","last_undegraded":"2024-09-10T23:40:04.982798+0000","last_fullsized":"2024-09-10T23:40:04.982798+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:23.607415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341522+0000","last_change":"2024-09-10T23:40:03.354924+0000","last_active":"2024-09-10T23:40:04.341522+0000","last_peered":"2024-09-10T23:40:04.341522+0000","last_clean":"2024-09-10T23:40:04.341522+0000","last_became_active":"2024-09-10T23:40:03.354666+0000","last_became_peered":"2024-09-10T23:40:03.354666+0000","last_unstale":"2024-09-10T23:40:04.341522+0000","last_undegraded":"2024-09-10T23:40:04.341522+0000","last_fullsized":"2024-09-10T23:40:04.341522+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:08.034276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345605+0000","last_change":"2024-09-10T23:40:03.358303+0000","last_active":"2024-09-10T23:40:04.345605+0000","last_peered":"2024-09-10T23:40:04.345605+0000","last_clean":"2024-09-10T23:40:04.345605+0000","last_became_active":"2024-09-10T23:40:03.358059+0000","last_became_peered":"2024-09-10T23:40:03.358059+0000","last_unstale":"2024-09-10T23:40:04.345605+0000","last_undegraded":"2024-09-10T23:40:04.345605+0000","last_fullsized":"2024-09-10T23:40:04.345605+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:24.847923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734570+0000","last_change":"2024-09-10T23:40:03.368848+0000","last_active":"2024-09-10T23:40:05.734570+0000","last_peered":"2024-09-10T23:40:05.734570+0000","last_clean":"2024-09-10T23:40:05.734570+0000","last_became_active":"2024-09-10T23:40:03.368537+0000","last_became_peered":"2024-09-10T23:40:03.368537+0000","last_unstale":"2024-09-10T23:40:05.734570+0000","last_undegraded":"2024-09-10T23:40:05.734570+0000","last_fullsized":"2024-09-10T23:40:05.734570+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:19.047231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.966765+0000","last_change":"2024-09-10T23:40:03.352069+0000","last_active":"2024-09-10T23:40:04.966765+0000","last_peered":"2024-09-10T23:40:04.966765+0000","last_clean":"2024-09-10T23:40:04.966765+0000","last_became_active":"2024-09-10T23:40:03.351712+0000","last_became_peered":"2024-09-10T23:40:03.351712+0000","last_unstale":"2024-09-10T23:40:04.966765+0000","last_undegraded":"2024-09-10T23:40:04.966765+0000","last_fullsized":"2024-09-10T23:40:04.966765+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:48.871207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346447+0000","last_change":"2024-09-10T23:40:03.365598+0000","last_active":"2024-09-10T23:40:04.346447+0000","last_peered":"2024-09-10T23:40:04.346447+0000","last_clean":"2024-09-10T23:40:04.346447+0000","last_became_active":"2024-09-10T23:40:03.365072+0000","last_became_peered":"2024-09-10T23:40:03.365072+0000","last_unstale":"2024-09-10T23:40:04.346447+0000","last_undegraded":"2024-09-10T23:40:04.346447+0000","last_fullsized":"2024-09-10T23:40:04.346447+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:13.692433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066888+0000","last_change":"2024-09-10T23:40:03.363503+0000","last_active":"2024-09-10T23:40:05.066888+0000","last_peered":"2024-09-10T23:40:05.066888+0000","last_clean":"2024-09-10T23:40:05.066888+0000","last_became_active":"2024-09-10T23:40:03.363267+0000","last_became_peered":"2024-09-10T23:40:03.363267+0000","last_unstale":"2024-09-10T23:40:05.066888+0000","last_undegraded":"2024-09-10T23:40:05.066888+0000","last_fullsized":"2024-09-10T23:40:05.066888+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:47.625167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345357+0000","last_change":"2024-09-10T23:40:03.351549+0000","last_active":"2024-09-10T23:40:04.345357+0000","last_peered":"2024-09-10T23:40:04.345357+0000","last_clean":"2024-09-10T23:40:04.345357+0000","last_became_active":"2024-09-10T23:40:03.351231+0000","last_became_peered":"2024-09-10T23:40:03.351231+0000","last_unstale":"2024-09-10T23:40:04.345357+0000","last_undegraded":"2024-09-10T23:40:04.345357+0000","last_fullsized":"2024-09-10T23:40:04.345357+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:48.871207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.067192+0000","last_change":"2024-09-10T23:40:03.366474+0000","last_active":"2024-09-10T23:40:05.067192+0000","last_peered":"2024-09-10T23:40:05.067192+0000","last_clean":"2024-09-10T23:40:05.067192+0000","last_became_active":"2024-09-10T23:40:03.360637+0000","last_became_peered":"2024-09-10T23:40:03.360637+0000","last_unstale":"2024-09-10T23:40:05.067192+0000","last_undegraded":"2024-09-10T23:40:05.067192+0000","last_fullsized":"2024-09-10T23:40:05.067192+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:17.315952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.344845+0000","last_change":"2024-09-10T23:40:03.372722+0000","last_active":"2024-09-10T23:40:04.344845+0000","last_peered":"2024-09-10T23:40:04.344845+0000","last_clean":"2024-09-10T23:40:04.344845+0000","last_became_active":"2024-09-10T23:40:03.372467+0000","last_became_peered":"2024-09-10T23:40:03.372467+0000","last_unstale":"2024-09-10T23:40:04.344845+0000","last_undegraded":"2024-09-10T23:40:04.344845+0000","last_fullsized":"2024-09-10T23:40:04.344845+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:43.564546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066410+0000","last_change":"2024-09-10T23:40:03.360620+0000","last_active":"2024-09-10T23:40:05.066410+0000","last_peered":"2024-09-10T23:40:05.066410+0000","last_clean":"2024-09-10T23:40:05.066410+0000","last_became_active":"2024-09-10T23:40:03.360326+0000","last_became_peered":"2024-09-10T23:40:03.360326+0000","last_unstale":"2024-09-10T23:40:05.066410+0000","last_undegraded":"2024-09-10T23:40:05.066410+0000","last_fullsized":"2024-09-10T23:40:05.066410+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:04.666223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.983117+0000","last_change":"2024-09-10T23:40:03.356548+0000","last_active":"2024-09-10T23:40:04.983117+0000","last_peered":"2024-09-10T23:40:04.983117+0000","last_clean":"2024-09-10T23:40:04.983117+0000","last_became_active":"2024-09-10T23:40:03.356305+0000","last_became_peered":"2024-09-10T23:40:03.356305+0000","last_unstale":"2024-09-10T23:40:04.983117+0000","last_undegraded":"2024-09-10T23:40:04.983117+0000","last_fullsized":"2024-09-10T23:40:04.983117+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:39.794032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727911+0000","last_change":"2024-09-10T23:40:03.358680+0000","last_active":"2024-09-10T23:40:05.727911+0000","last_peered":"2024-09-10T23:40:05.727911+0000","last_clean":"2024-09-10T23:40:05.727911+0000","last_became_active":"2024-09-10T23:40:03.355890+0000","last_became_peered":"2024-09-10T23:40:03.355890+0000","last_unstale":"2024-09-10T23:40:05.727911+0000","last_undegraded":"2024-09-10T23:40:05.727911+0000","last_fullsized":"2024-09-10T23:40:05.727911+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:33.993062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346569+0000","last_change":"2024-09-10T23:40:03.366128+0000","last_active":"2024-09-10T23:40:04.346569+0000","last_peered":"2024-09-10T23:40:04.346569+0000","last_clean":"2024-09-10T23:40:04.346569+0000","last_became_active":"2024-09-10T23:40:03.364713+0000","last_became_peered":"2024-09-10T23:40:03.364713+0000","last_unstale":"2024-09-10T23:40:04.346569+0000","last_undegraded":"2024-09-10T23:40:04.346569+0000","last_fullsized":"2024-09-10T23:40:04.346569+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:47.309787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346002+0000","last_change":"2024-09-10T23:40:03.358246+0000","last_active":"2024-09-10T23:40:04.346002+0000","last_peered":"2024-09-10T23:40:04.346002+0000","last_clean":"2024-09-10T23:40:04.346002+0000","last_became_active":"2024-09-10T23:40:03.357993+0000","last_became_peered":"2024-09-10T23:40:03.357993+0000","last_unstale":"2024-09-10T23:40:04.346002+0000","last_undegraded":"2024-09-10T23:40:04.346002+0000","last_fullsized":"2024-09-10T23:40:04.346002+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:42.026210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727632+0000","last_change":"2024-09-10T23:40:03.356696+0000","last_active":"2024-09-10T23:40:05.727632+0000","last_peered":"2024-09-10T23:40:05.727632+0000","last_clean":"2024-09-10T23:40:05.727632+0000","last_became_active":"2024-09-10T23:40:03.356295+0000","last_became_peered":"2024-09-10T23:40:03.356295+0000","last_unstale":"2024-09-10T23:40:05.727632+0000","last_undegraded":"2024-09-10T23:40:05.727632+0000","last_fullsized":"2024-09-10T23:40:05.727632+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:06.383906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727604+0000","last_change":"2024-09-10T23:40:03.360053+0000","last_active":"2024-09-10T23:40:05.727604+0000","last_peered":"2024-09-10T23:40:05.727604+0000","last_clean":"2024-09-10T23:40:05.727604+0000","last_became_active":"2024-09-10T23:40:03.359780+0000","last_became_peered":"2024-09-10T23:40:03.359780+0000","last_unstale":"2024-09-10T23:40:05.727604+0000","last_undegraded":"2024-09-10T23:40:05.727604+0000","last_fullsized":"2024-09-10T23:40:05.727604+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:12.470964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066446+0000","last_change":"2024-09-10T23:40:03.369580+0000","last_active":"2024-09-10T23:40:05.066446+0000","last_peered":"2024-09-10T23:40:05.066446+0000","last_clean":"2024-09-10T23:40:05.066446+0000","last_became_active":"2024-09-10T23:40:03.368920+0000","last_became_peered":"2024-09-10T23:40:03.368920+0000","last_unstale":"2024-09-10T23:40:05.066446+0000","last_undegraded":"2024-09-10T23:40:05.066446+0000","last_fullsized":"2024-09-10T23:40:05.066446+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:51.802426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066721+0000","last_change":"2024-09-10T23:40:03.369593+0000","last_active":"2024-09-10T23:40:05.066721+0000","last_peered":"2024-09-10T23:40:05.066721+0000","last_clean":"2024-09-10T23:40:05.066721+0000","last_became_active":"2024-09-10T23:40:03.367008+0000","last_became_peered":"2024-09-10T23:40:03.367008+0000","last_unstale":"2024-09-10T23:40:05.066721+0000","last_undegraded":"2024-09-10T23:40:05.066721+0000","last_fullsized":"2024-09-10T23:40:05.066721+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:35:56.962775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066768+0000","last_change":"2024-09-10T23:40:03.366844+0000","last_active":"2024-09-10T23:40:05.066768+0000","last_peered":"2024-09-10T23:40:05.066768+0000","last_clean":"2024-09-10T23:40:05.066768+0000","last_became_active":"2024-09-10T23:40:03.366592+0000","last_became_peered":"2024-09-10T23:40:03.366592+0000","last_unstale":"2024-09-10T23:40:05.066768+0000","last_undegraded":"2024-09-10T23:40:05.066768+0000","last_fullsized":"2024-09-10T23:40:05.066768+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:52.149201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"150'4","reported_seq":19,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346836+0000","last_change":"2024-09-10T23:40:02.357339+0000","last_active":"2024-09-10T23:40:04.346836+0000","last_peered":"2024-09-10T23:40:04.346836+0000","last_clean":"2024-09-10T23:40:04.346836+0000","last_became_active":"2024-09-10T23:40:02.357054+0000","last_became_peered":"2024-09-10T23:40:02.357054+0000","last_unstale":"2024-09-10T23:40:04.346836+0000","last_undegraded":"2024-09-10T23:40:04.346836+0000","last_fullsized":"2024-09-10T23:40:04.346836+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:52.567685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346458+0000","last_change":"2024-09-10T23:40:03.365971+0000","last_active":"2024-09-10T23:40:04.346458+0000","last_peered":"2024-09-10T23:40:04.346458+0000","last_clean":"2024-09-10T23:40:04.346458+0000","last_became_active":"2024-09-10T23:40:03.365725+0000","last_became_peered":"2024-09-10T23:40:03.365725+0000","last_unstale":"2024-09-10T23:40:04.346458+0000","last_undegraded":"2024-09-10T23:40:04.346458+0000","last_fullsized":"2024-09-10T23:40:04.346458+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:44.231823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727889+0000","last_change":"2024-09-10T23:40:02.354357+0000","last_active":"2024-09-10T23:40:05.727889+0000","last_peered":"2024-09-10T23:40:05.727889+0000","last_clean":"2024-09-10T23:40:05.727889+0000","last_became_active":"2024-09-10T23:40:02.354105+0000","last_became_peered":"2024-09-10T23:40:02.354105+0000","last_unstale":"2024-09-10T23:40:05.727889+0000","last_undegraded":"2024-09-10T23:40:05.727889+0000","last_fullsized":"2024-09-10T23:40:05.727889+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:57:49.719773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.983027+0000","last_change":"2024-09-10T23:40:03.354808+0000","last_active":"2024-09-10T23:40:04.983027+0000","last_peered":"2024-09-10T23:40:04.983027+0000","last_clean":"2024-09-10T23:40:04.983027+0000","last_became_active":"2024-09-10T23:40:03.354539+0000","last_became_peered":"2024-09-10T23:40:03.354539+0000","last_unstale":"2024-09-10T23:40:04.983027+0000","last_undegraded":"2024-09-10T23:40:04.983027+0000","last_fullsized":"2024-09-10T23:40:04.983027+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:08.122357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066946+0000","last_change":"2024-09-10T23:40:02.363918+0000","last_active":"2024-09-10T23:40:05.066946+0000","last_peered":"2024-09-10T23:40:05.066946+0000","last_clean":"2024-09-10T23:40:05.066946+0000","last_became_active":"2024-09-10T23:40:02.363627+0000","last_became_peered":"2024-09-10T23:40:02.363627+0000","last_unstale":"2024-09-10T23:40:05.066946+0000","last_undegraded":"2024-09-10T23:40:05.066946+0000","last_fullsized":"2024-09-10T23:40:05.066946+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:53.657508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982919+0000","last_change":"2024-09-10T23:40:03.356438+0000","last_active":"2024-09-10T23:40:04.982919+0000","last_peered":"2024-09-10T23:40:04.982919+0000","last_clean":"2024-09-10T23:40:04.982919+0000","last_became_active":"2024-09-10T23:40:03.356144+0000","last_became_peered":"2024-09-10T23:40:03.356144+0000","last_unstale":"2024-09-10T23:40:04.982919+0000","last_undegraded":"2024-09-10T23:40:04.982919+0000","last_fullsized":"2024-09-10T23:40:04.982919+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:04:47.070469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727868+0000","last_change":"2024-09-10T23:40:02.354874+0000","last_active":"2024-09-10T23:40:05.727868+0000","last_peered":"2024-09-10T23:40:05.727868+0000","last_clean":"2024-09-10T23:40:05.727868+0000","last_became_active":"2024-09-10T23:40:02.354568+0000","last_became_peered":"2024-09-10T23:40:02.354568+0000","last_unstale":"2024-09-10T23:40:05.727868+0000","last_undegraded":"2024-09-10T23:40:05.727868+0000","last_fullsized":"2024-09-10T23:40:05.727868+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:07.668909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734487+0000","last_change":"2024-09-10T23:40:03.369829+0000","last_active":"2024-09-10T23:40:05.734487+0000","last_peered":"2024-09-10T23:40:05.734487+0000","last_clean":"2024-09-10T23:40:05.734487+0000","last_became_active":"2024-09-10T23:40:03.369106+0000","last_became_peered":"2024-09-10T23:40:03.369106+0000","last_unstale":"2024-09-10T23:40:05.734487+0000","last_undegraded":"2024-09-10T23:40:05.734487+0000","last_fullsized":"2024-09-10T23:40:05.734487+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:59.420683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.067206+0000","last_change":"2024-09-10T23:40:02.363569+0000","last_active":"2024-09-10T23:40:05.067206+0000","last_peered":"2024-09-10T23:40:05.067206+0000","last_clean":"2024-09-10T23:40:05.067206+0000","last_became_active":"2024-09-10T23:40:02.362386+0000","last_became_peered":"2024-09-10T23:40:02.362386+0000","last_unstale":"2024-09-10T23:40:05.067206+0000","last_undegraded":"2024-09-10T23:40:05.067206+0000","last_fullsized":"2024-09-10T23:40:05.067206+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:38.787706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066980+0000","last_change":"2024-09-10T23:40:03.362243+0000","last_active":"2024-09-10T23:40:05.066980+0000","last_peered":"2024-09-10T23:40:05.066980+0000","last_clean":"2024-09-10T23:40:05.066980+0000","last_became_active":"2024-09-10T23:40:03.361700+0000","last_became_peered":"2024-09-10T23:40:03.361700+0000","last_unstale":"2024-09-10T23:40:05.066980+0000","last_undegraded":"2024-09-10T23:40:05.066980+0000","last_fullsized":"2024-09-10T23:40:05.066980+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:57.759800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.067416+0000","last_change":"2024-09-10T23:40:02.363592+0000","last_active":"2024-09-10T23:40:05.067416+0000","last_peered":"2024-09-10T23:40:05.067416+0000","last_clean":"2024-09-10T23:40:05.067416+0000","last_became_active":"2024-09-10T23:40:02.363294+0000","last_became_peered":"2024-09-10T23:40:02.363294+0000","last_unstale":"2024-09-10T23:40:05.067416+0000","last_undegraded":"2024-09-10T23:40:05.067416+0000","last_fullsized":"2024-09-10T23:40:05.067416+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:14.181789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341693+0000","last_change":"2024-09-10T23:40:03.363981+0000","last_active":"2024-09-10T23:40:04.341693+0000","last_peered":"2024-09-10T23:40:04.341693+0000","last_clean":"2024-09-10T23:40:04.341693+0000","last_became_active":"2024-09-10T23:40:03.363696+0000","last_became_peered":"2024-09-10T23:40:03.363696+0000","last_unstale":"2024-09-10T23:40:04.341693+0000","last_undegraded":"2024-09-10T23:40:04.341693+0000","last_fullsized":"2024-09-10T23:40:04.341693+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:27.074805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.966387+0000","last_change":"2024-09-10T23:40:02.353255+0000","last_active":"2024-09-10T23:40:04.966387+0000","last_peered":"2024-09-10T23:40:04.966387+0000","last_clean":"2024-09-10T23:40:04.966387+0000","last_became_active":"2024-09-10T23:40:02.352992+0000","last_became_peered":"2024-09-10T23:40:02.352992+0000","last_unstale":"2024-09-10T23:40:04.966387+0000","last_undegraded":"2024-09-10T23:40:04.966387+0000","last_fullsized":"2024-09-10T23:40:04.966387+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:17.250644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345574+0000","last_change":"2024-09-10T23:40:03.358205+0000","last_active":"2024-09-10T23:40:04.345574+0000","last_peered":"2024-09-10T23:40:04.345574+0000","last_clean":"2024-09-10T23:40:04.345574+0000","last_became_active":"2024-09-10T23:40:03.357946+0000","last_became_peered":"2024-09-10T23:40:03.357946+0000","last_unstale":"2024-09-10T23:40:04.345574+0000","last_undegraded":"2024-09-10T23:40:04.345574+0000","last_fullsized":"2024-09-10T23:40:04.345574+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:04.124383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341296+0000","last_change":"2024-09-10T23:40:02.359051+0000","last_active":"2024-09-10T23:40:04.341296+0000","last_peered":"2024-09-10T23:40:04.341296+0000","last_clean":"2024-09-10T23:40:04.341296+0000","last_became_active":"2024-09-10T23:40:02.357335+0000","last_became_peered":"2024-09-10T23:40:02.357335+0000","last_unstale":"2024-09-10T23:40:04.341296+0000","last_undegraded":"2024-09-10T23:40:04.341296+0000","last_fullsized":"2024-09-10T23:40:04.341296+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:36.624712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346668+0000","last_change":"2024-09-10T23:40:03.354222+0000","last_active":"2024-09-10T23:40:04.346668+0000","last_peered":"2024-09-10T23:40:04.346668+0000","last_clean":"2024-09-10T23:40:04.346668+0000","last_became_active":"2024-09-10T23:40:03.353981+0000","last_became_peered":"2024-09-10T23:40:03.353981+0000","last_unstale":"2024-09-10T23:40:04.346668+0000","last_undegraded":"2024-09-10T23:40:04.346668+0000","last_fullsized":"2024-09-10T23:40:04.346668+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:48.915081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346031+0000","last_change":"2024-09-10T23:40:02.359834+0000","last_active":"2024-09-10T23:40:04.346031+0000","last_peered":"2024-09-10T23:40:04.346031+0000","last_clean":"2024-09-10T23:40:04.346031+0000","last_became_active":"2024-09-10T23:40:02.359528+0000","last_became_peered":"2024-09-10T23:40:02.359528+0000","last_unstale":"2024-09-10T23:40:04.346031+0000","last_undegraded":"2024-09-10T23:40:04.346031+0000","last_fullsized":"2024-09-10T23:40:04.346031+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:39.104094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727775+0000","last_change":"2024-09-10T23:40:03.358068+0000","last_active":"2024-09-10T23:40:05.727775+0000","last_peered":"2024-09-10T23:40:05.727775+0000","last_clean":"2024-09-10T23:40:05.727775+0000","last_became_active":"2024-09-10T23:40:03.357794+0000","last_became_peered":"2024-09-10T23:40:03.357794+0000","last_unstale":"2024-09-10T23:40:05.727775+0000","last_undegraded":"2024-09-10T23:40:05.727775+0000","last_fullsized":"2024-09-10T23:40:05.727775+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:47.309787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"150'3","reported_seq":18,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.067082+0000","last_change":"2024-09-10T23:40:02.364953+0000","last_active":"2024-09-10T23:40:05.067082+0000","last_peered":"2024-09-10T23:40:05.067082+0000","last_clean":"2024-09-10T23:40:05.067082+0000","last_became_active":"2024-09-10T23:40:02.364637+0000","last_became_peered":"2024-09-10T23:40:02.364637+0000","last_unstale":"2024-09-10T23:40:05.067082+0000","last_undegraded":"2024-09-10T23:40:05.067082+0000","last_fullsized":"2024-09-10T23:40:05.067082+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:53.213242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346348+0000","last_change":"2024-09-10T23:40:03.353089+0000","last_active":"2024-09-10T23:40:04.346348+0000","last_peered":"2024-09-10T23:40:04.346348+0000","last_clean":"2024-09-10T23:40:04.346348+0000","last_became_active":"2024-09-10T23:40:03.352834+0000","last_became_peered":"2024-09-10T23:40:03.352834+0000","last_unstale":"2024-09-10T23:40:04.346348+0000","last_undegraded":"2024-09-10T23:40:04.346348+0000","last_fullsized":"2024-09-10T23:40:04.346348+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:27.499094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"150'6","reported_seq":21,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066826+0000","last_change":"2024-09-10T23:40:02.361550+0000","last_active":"2024-09-10T23:40:05.066826+0000","last_peered":"2024-09-10T23:40:05.066826+0000","last_clean":"2024-09-10T23:40:05.066826+0000","last_became_active":"2024-09-10T23:40:02.361204+0000","last_became_peered":"2024-09-10T23:40:02.361204+0000","last_unstale":"2024-09-10T23:40:05.066826+0000","last_undegraded":"2024-09-10T23:40:05.066826+0000","last_fullsized":"2024-09-10T23:40:05.066826+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:04:46.064144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346116+0000","last_change":"2024-09-10T23:40:03.365358+0000","last_active":"2024-09-10T23:40:04.346116+0000","last_peered":"2024-09-10T23:40:04.346116+0000","last_clean":"2024-09-10T23:40:04.346116+0000","last_became_active":"2024-09-10T23:40:03.365061+0000","last_became_peered":"2024-09-10T23:40:03.365061+0000","last_unstale":"2024-09-10T23:40:04.346116+0000","last_undegraded":"2024-09-10T23:40:04.346116+0000","last_fullsized":"2024-09-10T23:40:04.346116+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:30.588432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.13","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727720+0000","last_change":"2024-09-10T23:40:02.354529+0000","last_active":"2024-09-10T23:40:05.727720+0000","last_peered":"2024-09-10T23:40:05.727720+0000","last_clean":"2024-09-10T23:40:05.727720+0000","last_became_active":"2024-09-10T23:40:02.354273+0000","last_became_peered":"2024-09-10T23:40:02.354273+0000","last_unstale":"2024-09-10T23:40:05.727720+0000","last_undegraded":"2024-09-10T23:40:05.727720+0000","last_fullsized":"2024-09-10T23:40:05.727720+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:44.907161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.966622+0000","last_change":"2024-09-10T23:40:03.356503+0000","last_active":"2024-09-10T23:40:04.966622+0000","last_peered":"2024-09-10T23:40:04.966622+0000","last_clean":"2024-09-10T23:40:04.966622+0000","last_became_active":"2024-09-10T23:40:03.356243+0000","last_became_peered":"2024-09-10T23:40:03.356243+0000","last_unstale":"2024-09-10T23:40:04.966622+0000","last_undegraded":"2024-09-10T23:40:04.966622+0000","last_fullsized":"2024-09-10T23:40:04.966622+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:41.299187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345762+0000","last_change":"2024-09-10T23:40:02.356793+0000","last_active":"2024-09-10T23:40:04.345762+0000","last_peered":"2024-09-10T23:40:04.345762+0000","last_clean":"2024-09-10T23:40:04.345762+0000","last_became_active":"2024-09-10T23:40:02.356539+0000","last_became_peered":"2024-09-10T23:40:02.356539+0000","last_unstale":"2024-09-10T23:40:04.345762+0000","last_undegraded":"2024-09-10T23:40:04.345762+0000","last_fullsized":"2024-09-10T23:40:04.345762+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:35.356292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727496+0000","last_change":"2024-09-10T23:40:03.358716+0000","last_active":"2024-09-10T23:40:05.727496+0000","last_peered":"2024-09-10T23:40:05.727496+0000","last_clean":"2024-09-10T23:40:05.727496+0000","last_became_active":"2024-09-10T23:40:03.358444+0000","last_became_peered":"2024-09-10T23:40:03.358444+0000","last_unstale":"2024-09-10T23:40:05.727496+0000","last_undegraded":"2024-09-10T23:40:05.727496+0000","last_fullsized":"2024-09-10T23:40:05.727496+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:47.625167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066910+0000","last_change":"2024-09-10T23:40:02.363352+0000","last_active":"2024-09-10T23:40:05.066910+0000","last_peered":"2024-09-10T23:40:05.066910+0000","last_clean":"2024-09-10T23:40:05.066910+0000","last_became_active":"2024-09-10T23:40:02.362332+0000","last_became_peered":"2024-09-10T23:40:02.362332+0000","last_unstale":"2024-09-10T23:40:05.066910+0000","last_undegraded":"2024-09-10T23:40:05.066910+0000","last_fullsized":"2024-09-10T23:40:05.066910+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:53.719176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066564+0000","last_change":"2024-09-10T23:40:03.369222+0000","last_active":"2024-09-10T23:40:05.066564+0000","last_peered":"2024-09-10T23:40:05.066564+0000","last_clean":"2024-09-10T23:40:05.066564+0000","last_became_active":"2024-09-10T23:40:03.368598+0000","last_became_peered":"2024-09-10T23:40:03.368598+0000","last_unstale":"2024-09-10T23:40:05.066564+0000","last_undegraded":"2024-09-10T23:40:05.066564+0000","last_fullsized":"2024-09-10T23:40:05.066564+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:28.728488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"150'8","reported_seq":23,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.966452+0000","last_change":"2024-09-10T23:40:02.357720+0000","last_active":"2024-09-10T23:40:04.966452+0000","last_peered":"2024-09-10T23:40:04.966452+0000","last_clean":"2024-09-10T23:40:04.966452+0000","last_became_active":"2024-09-10T23:40:02.357265+0000","last_became_peered":"2024-09-10T23:40:02.357265+0000","last_unstale":"2024-09-10T23:40:04.966452+0000","last_undegraded":"2024-09-10T23:40:04.966452+0000","last_fullsized":"2024-09-10T23:40:04.966452+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:44.907161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345859+0000","last_change":"2024-09-10T23:40:03.347613+0000","last_active":"2024-09-10T23:40:04.345859+0000","last_peered":"2024-09-10T23:40:04.345859+0000","last_clean":"2024-09-10T23:40:04.345859+0000","last_became_active":"2024-09-10T23:40:03.347314+0000","last_became_peered":"2024-09-10T23:40:03.347314+0000","last_unstale":"2024-09-10T23:40:04.345859+0000","last_undegraded":"2024-09-10T23:40:04.345859+0000","last_fullsized":"2024-09-10T23:40:04.345859+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:24.543980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.f","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.067281+0000","last_change":"2024-09-10T23:40:02.367006+0000","last_active":"2024-09-10T23:40:05.067281+0000","last_peered":"2024-09-10T23:40:05.067281+0000","last_clean":"2024-09-10T23:40:05.067281+0000","last_became_active":"2024-09-10T23:40:02.366761+0000","last_became_peered":"2024-09-10T23:40:02.366761+0000","last_unstale":"2024-09-10T23:40:05.067281+0000","last_undegraded":"2024-09-10T23:40:05.067281+0000","last_fullsized":"2024-09-10T23:40:05.067281+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:44.949937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.728027+0000","last_change":"2024-09-10T23:40:03.367960+0000","last_active":"2024-09-10T23:40:05.728027+0000","last_peered":"2024-09-10T23:40:05.728027+0000","last_clean":"2024-09-10T23:40:05.728027+0000","last_became_active":"2024-09-10T23:40:03.367643+0000","last_became_peered":"2024-09-10T23:40:03.367643+0000","last_unstale":"2024-09-10T23:40:05.728027+0000","last_undegraded":"2024-09-10T23:40:05.728027+0000","last_fullsized":"2024-09-10T23:40:05.728027+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:37.078737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734569+0000","last_change":"2024-09-10T23:40:02.350625+0000","last_active":"2024-09-10T23:40:05.734569+0000","last_peered":"2024-09-10T23:40:05.734569+0000","last_clean":"2024-09-10T23:40:05.734569+0000","last_became_active":"2024-09-10T23:40:02.349954+0000","last_became_peered":"2024-09-10T23:40:02.349954+0000","last_unstale":"2024-09-10T23:40:05.734569+0000","last_undegraded":"2024-09-10T23:40:05.734569+0000","last_fullsized":"2024-09-10T23:40:05.734569+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:14.193750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.983122+0000","last_change":"2024-09-10T23:40:03.362463+0000","last_active":"2024-09-10T23:40:04.983122+0000","last_peered":"2024-09-10T23:40:04.983122+0000","last_clean":"2024-09-10T23:40:04.983122+0000","last_became_active":"2024-09-10T23:40:03.362183+0000","last_became_peered":"2024-09-10T23:40:03.362183+0000","last_unstale":"2024-09-10T23:40:04.983122+0000","last_undegraded":"2024-09-10T23:40:04.983122+0000","last_fullsized":"2024-09-10T23:40:04.983122+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:54.725502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341143+0000","last_change":"2024-09-10T23:40:02.355908+0000","last_active":"2024-09-10T23:40:04.341143+0000","last_peered":"2024-09-10T23:40:04.341143+0000","last_clean":"2024-09-10T23:40:04.341143+0000","last_became_active":"2024-09-10T23:40:02.355653+0000","last_became_peered":"2024-09-10T23:40:02.355653+0000","last_unstale":"2024-09-10T23:40:04.341143+0000","last_undegraded":"2024-09-10T23:40:04.341143+0000","last_fullsized":"2024-09-10T23:40:04.341143+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:21.969940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727849+0000","last_change":"2024-09-10T23:40:03.353933+0000","last_active":"2024-09-10T23:40:05.727849+0000","last_peered":"2024-09-10T23:40:05.727849+0000","last_clean":"2024-09-10T23:40:05.727849+0000","last_became_active":"2024-09-10T23:40:03.353680+0000","last_became_peered":"2024-09-10T23:40:03.353680+0000","last_unstale":"2024-09-10T23:40:05.727849+0000","last_undegraded":"2024-09-10T23:40:05.727849+0000","last_fullsized":"2024-09-10T23:40:05.727849+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:05.717713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341266+0000","last_change":"2024-09-10T23:40:02.352194+0000","last_active":"2024-09-10T23:40:04.341266+0000","last_peered":"2024-09-10T23:40:04.341266+0000","last_clean":"2024-09-10T23:40:04.341266+0000","last_became_active":"2024-09-10T23:40:02.351903+0000","last_became_peered":"2024-09-10T23:40:02.351903+0000","last_unstale":"2024-09-10T23:40:04.341266+0000","last_undegraded":"2024-09-10T23:40:04.341266+0000","last_fullsized":"2024-09-10T23:40:04.341266+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:03.963558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734783+0000","last_change":"2024-09-10T23:40:03.356347+0000","last_active":"2024-09-10T23:40:05.734783+0000","last_peered":"2024-09-10T23:40:05.734783+0000","last_clean":"2024-09-10T23:40:05.734783+0000","last_became_active":"2024-09-10T23:40:03.356044+0000","last_became_peered":"2024-09-10T23:40:03.356044+0000","last_unstale":"2024-09-10T23:40:05.734783+0000","last_undegraded":"2024-09-10T23:40:05.734783+0000","last_fullsized":"2024-09-10T23:40:05.734783+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:37.631038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.067259+0000","last_change":"2024-09-10T23:40:02.360470+0000","last_active":"2024-09-10T23:40:05.067259+0000","last_peered":"2024-09-10T23:40:05.067259+0000","last_clean":"2024-09-10T23:40:05.067259+0000","last_became_active":"2024-09-10T23:40:02.360193+0000","last_became_peered":"2024-09-10T23:40:02.360193+0000","last_unstale":"2024-09-10T23:40:05.067259+0000","last_undegraded":"2024-09-10T23:40:05.067259+0000","last_fullsized":"2024-09-10T23:40:05.067259+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:55.532229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.966555+0000","last_change":"2024-09-10T23:40:03.354495+0000","last_active":"2024-09-10T23:40:04.966555+0000","last_peered":"2024-09-10T23:40:04.966555+0000","last_clean":"2024-09-10T23:40:04.966555+0000","last_became_active":"2024-09-10T23:40:03.354248+0000","last_became_peered":"2024-09-10T23:40:03.354248+0000","last_unstale":"2024-09-10T23:40:04.966555+0000","last_undegraded":"2024-09-10T23:40:04.966555+0000","last_fullsized":"2024-09-10T23:40:04.966555+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:04.666223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734888+0000","last_change":"2024-09-10T23:40:02.353010+0000","last_active":"2024-09-10T23:40:05.734888+0000","last_peered":"2024-09-10T23:40:05.734888+0000","last_clean":"2024-09-10T23:40:05.734888+0000","last_became_active":"2024-09-10T23:40:02.352736+0000","last_became_peered":"2024-09-10T23:40:02.352736+0000","last_unstale":"2024-09-10T23:40:05.734888+0000","last_undegraded":"2024-09-10T23:40:05.734888+0000","last_fullsized":"2024-09-10T23:40:05.734888+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:32.633723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341145+0000","last_change":"2024-09-10T23:40:03.350265+0000","last_active":"2024-09-10T23:40:04.341145+0000","last_peered":"2024-09-10T23:40:04.341145+0000","last_clean":"2024-09-10T23:40:04.341145+0000","last_became_active":"2024-09-10T23:40:03.349982+0000","last_became_peered":"2024-09-10T23:40:03.349982+0000","last_unstale":"2024-09-10T23:40:04.341145+0000","last_undegraded":"2024-09-10T23:40:04.341145+0000","last_fullsized":"2024-09-10T23:40:04.341145+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:58.791371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734280+0000","last_change":"2024-09-10T23:40:02.356421+0000","last_active":"2024-09-10T23:40:05.734280+0000","last_peered":"2024-09-10T23:40:05.734280+0000","last_clean":"2024-09-10T23:40:05.734280+0000","last_became_active":"2024-09-10T23:40:02.355527+0000","last_became_peered":"2024-09-10T23:40:02.355527+0000","last_unstale":"2024-09-10T23:40:05.734280+0000","last_undegraded":"2024-09-10T23:40:05.734280+0000","last_fullsized":"2024-09-10T23:40:05.734280+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:25.830166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345052+0000","last_change":"2024-09-10T23:40:03.351934+0000","last_active":"2024-09-10T23:40:04.345052+0000","last_peered":"2024-09-10T23:40:04.345052+0000","last_clean":"2024-09-10T23:40:04.345052+0000","last_became_active":"2024-09-10T23:40:03.351636+0000","last_became_peered":"2024-09-10T23:40:03.351636+0000","last_unstale":"2024-09-10T23:40:04.345052+0000","last_undegraded":"2024-09-10T23:40:04.345052+0000","last_fullsized":"2024-09-10T23:40:04.345052+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:24.962965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341061+0000","last_change":"2024-09-10T23:40:02.355475+0000","last_active":"2024-09-10T23:40:04.341061+0000","last_peered":"2024-09-10T23:40:04.341061+0000","last_clean":"2024-09-10T23:40:04.341061+0000","last_became_active":"2024-09-10T23:40:02.355204+0000","last_became_peered":"2024-09-10T23:40:02.355204+0000","last_unstale":"2024-09-10T23:40:04.341061+0000","last_undegraded":"2024-09-10T23:40:04.341061+0000","last_fullsized":"2024-09-10T23:40:04.341061+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:40.101644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345844+0000","last_change":"2024-09-10T23:40:03.352942+0000","last_active":"2024-09-10T23:40:04.345844+0000","last_peered":"2024-09-10T23:40:04.345844+0000","last_clean":"2024-09-10T23:40:04.345844+0000","last_became_active":"2024-09-10T23:40:03.352670+0000","last_became_peered":"2024-09-10T23:40:03.352670+0000","last_unstale":"2024-09-10T23:40:04.345844+0000","last_undegraded":"2024-09-10T23:40:04.345844+0000","last_fullsized":"2024-09-10T23:40:04.345844+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:47.625167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341417+0000","last_change":"2024-09-10T23:40:02.352054+0000","last_active":"2024-09-10T23:40:04.341417+0000","last_peered":"2024-09-10T23:40:04.341417+0000","last_clean":"2024-09-10T23:40:04.341417+0000","last_became_active":"2024-09-10T23:40:02.351786+0000","last_became_peered":"2024-09-10T23:40:02.351786+0000","last_unstale":"2024-09-10T23:40:04.341417+0000","last_undegraded":"2024-09-10T23:40:04.341417+0000","last_fullsized":"2024-09-10T23:40:04.341417+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:17.235272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346887+0000","last_change":"2024-09-10T23:40:03.355026+0000","last_active":"2024-09-10T23:40:04.346887+0000","last_peered":"2024-09-10T23:40:04.346887+0000","last_clean":"2024-09-10T23:40:04.346887+0000","last_became_active":"2024-09-10T23:40:03.354682+0000","last_became_peered":"2024-09-10T23:40:03.354682+0000","last_unstale":"2024-09-10T23:40:04.346887+0000","last_undegraded":"2024-09-10T23:40:04.346887+0000","last_fullsized":"2024-09-10T23:40:04.346887+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:05.717713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"151'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346079+0000","last_change":"2024-09-10T23:40:02.360781+0000","last_active":"2024-09-10T23:40:04.346079+0000","last_peered":"2024-09-10T23:40:04.346079+0000","last_clean":"2024-09-10T23:40:04.346079+0000","last_became_active":"2024-09-10T23:40:02.360156+0000","last_became_peered":"2024-09-10T23:40:02.360156+0000","last_unstale":"2024-09-10T23:40:04.346079+0000","last_undegraded":"2024-09-10T23:40:04.346079+0000","last_fullsized":"2024-09-10T23:40:04.346079+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:29.712948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.345900+0000","last_change":"2024-09-10T23:40:03.370366+0000","last_active":"2024-09-10T23:40:04.345900+0000","last_peered":"2024-09-10T23:40:04.345900+0000","last_clean":"2024-09-10T23:40:04.345900+0000","last_became_active":"2024-09-10T23:40:03.370110+0000","last_became_peered":"2024-09-10T23:40:03.370110+0000","last_unstale":"2024-09-10T23:40:04.345900+0000","last_undegraded":"2024-09-10T23:40:04.345900+0000","last_fullsized":"2024-09-10T23:40:04.345900+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:12.470964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.346613+0000","last_change":"2024-09-10T23:40:02.359868+0000","last_active":"2024-09-10T23:40:04.346613+0000","last_peered":"2024-09-10T23:40:04.346613+0000","last_clean":"2024-09-10T23:40:04.346613+0000","last_became_active":"2024-09-10T23:40:02.359600+0000","last_became_peered":"2024-09-10T23:40:02.359600+0000","last_unstale":"2024-09-10T23:40:04.346613+0000","last_undegraded":"2024-09-10T23:40:04.346613+0000","last_fullsized":"2024-09-10T23:40:04.346613+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:11.452645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066882+0000","last_change":"2024-09-10T23:40:03.368479+0000","last_active":"2024-09-10T23:40:05.066882+0000","last_peered":"2024-09-10T23:40:05.066882+0000","last_clean":"2024-09-10T23:40:05.066882+0000","last_became_active":"2024-09-10T23:40:03.368202+0000","last_became_peered":"2024-09-10T23:40:03.368202+0000","last_unstale":"2024-09-10T23:40:05.066882+0000","last_undegraded":"2024-09-10T23:40:05.066882+0000","last_fullsized":"2024-09-10T23:40:05.066882+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:03.720617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.067081+0000","last_change":"2024-09-10T23:40:02.362452+0000","last_active":"2024-09-10T23:40:05.067081+0000","last_peered":"2024-09-10T23:40:05.067081+0000","last_clean":"2024-09-10T23:40:05.067081+0000","last_became_active":"2024-09-10T23:40:02.362039+0000","last_became_peered":"2024-09-10T23:40:02.362039+0000","last_unstale":"2024-09-10T23:40:05.067081+0000","last_undegraded":"2024-09-10T23:40:05.067081+0000","last_fullsized":"2024-09-10T23:40:05.067081+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:07.116031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982677+0000","last_change":"2024-09-10T23:40:03.361292+0000","last_active":"2024-09-10T23:40:04.982677+0000","last_peered":"2024-09-10T23:40:04.982677+0000","last_clean":"2024-09-10T23:40:04.982677+0000","last_became_active":"2024-09-10T23:40:03.360859+0000","last_became_peered":"2024-09-10T23:40:03.360859+0000","last_unstale":"2024-09-10T23:40:04.982677+0000","last_undegraded":"2024-09-10T23:40:04.982677+0000","last_fullsized":"2024-09-10T23:40:04.982677+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:03:15.656273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982929+0000","last_change":"2024-09-10T23:40:02.356946+0000","last_active":"2024-09-10T23:40:04.982929+0000","last_peered":"2024-09-10T23:40:04.982929+0000","last_clean":"2024-09-10T23:40:04.982929+0000","last_became_active":"2024-09-10T23:40:02.356690+0000","last_became_peered":"2024-09-10T23:40:02.356690+0000","last_unstale":"2024-09-10T23:40:04.982929+0000","last_undegraded":"2024-09-10T23:40:04.982929+0000","last_fullsized":"2024-09-10T23:40:04.982929+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:09:45.872598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.341401+0000","last_change":"2024-09-10T23:40:03.354787+0000","last_active":"2024-09-10T23:40:04.341401+0000","last_peered":"2024-09-10T23:40:04.341401+0000","last_clean":"2024-09-10T23:40:04.341401+0000","last_became_active":"2024-09-10T23:40:03.352966+0000","last_became_peered":"2024-09-10T23:40:03.352966+0000","last_unstale":"2024-09-10T23:40:04.341401+0000","last_undegraded":"2024-09-10T23:40:04.341401+0000","last_fullsized":"2024-09-10T23:40:04.341401+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:26.409688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982841+0000","last_change":"2024-09-10T23:40:02.349369+0000","last_active":"2024-09-10T23:40:04.982841+0000","last_peered":"2024-09-10T23:40:04.982841+0000","last_clean":"2024-09-10T23:40:04.982841+0000","last_became_active":"2024-09-10T23:40:02.349059+0000","last_became_peered":"2024-09-10T23:40:02.349059+0000","last_unstale":"2024-09-10T23:40:04.982841+0000","last_undegraded":"2024-09-10T23:40:04.982841+0000","last_fullsized":"2024-09-10T23:40:04.982841+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:50.367930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982700+0000","last_change":"2024-09-10T23:40:03.354960+0000","last_active":"2024-09-10T23:40:04.982700+0000","last_peered":"2024-09-10T23:40:04.982700+0000","last_clean":"2024-09-10T23:40:04.982700+0000","last_became_active":"2024-09-10T23:40:03.354675+0000","last_became_peered":"2024-09-10T23:40:03.354675+0000","last_unstale":"2024-09-10T23:40:04.982700+0000","last_undegraded":"2024-09-10T23:40:04.982700+0000","last_fullsized":"2024-09-10T23:40:04.982700+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:24.173026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982726+0000","last_change":"2024-09-10T23:40:02.349452+0000","last_active":"2024-09-10T23:40:04.982726+0000","last_peered":"2024-09-10T23:40:04.982726+0000","last_clean":"2024-09-10T23:40:04.982726+0000","last_became_active":"2024-09-10T23:40:02.349132+0000","last_became_peered":"2024-09-10T23:40:02.349132+0000","last_unstale":"2024-09-10T23:40:04.982726+0000","last_undegraded":"2024-09-10T23:40:04.982726+0000","last_fullsized":"2024-09-10T23:40:04.982726+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:54.008345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.066783+0000","last_change":"2024-09-10T23:40:03.362881+0000","last_active":"2024-09-10T23:40:05.066783+0000","last_peered":"2024-09-10T23:40:05.066783+0000","last_clean":"2024-09-10T23:40:05.066783+0000","last_became_active":"2024-09-10T23:40:03.362635+0000","last_became_peered":"2024-09-10T23:40:03.362635+0000","last_unstale":"2024-09-10T23:40:05.066783+0000","last_undegraded":"2024-09-10T23:40:05.066783+0000","last_fullsized":"2024-09-10T23:40:05.066783+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:04.124383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734506+0000","last_change":"2024-09-10T23:40:02.360113+0000","last_active":"2024-09-10T23:40:05.734506+0000","last_peered":"2024-09-10T23:40:05.734506+0000","last_clean":"2024-09-10T23:40:05.734506+0000","last_became_active":"2024-09-10T23:40:02.359801+0000","last_became_peered":"2024-09-10T23:40:02.359801+0000","last_unstale":"2024-09-10T23:40:05.734506+0000","last_undegraded":"2024-09-10T23:40:05.734506+0000","last_fullsized":"2024-09-10T23:40:05.734506+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:32.744092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.734537+0000","last_change":"2024-09-10T23:40:03.359682+0000","last_active":"2024-09-10T23:40:05.734537+0000","last_peered":"2024-09-10T23:40:05.734537+0000","last_clean":"2024-09-10T23:40:05.734537+0000","last_became_active":"2024-09-10T23:40:03.359397+0000","last_became_peered":"2024-09-10T23:40:03.359397+0000","last_unstale":"2024-09-10T23:40:05.734537+0000","last_undegraded":"2024-09-10T23:40:05.734537+0000","last_fullsized":"2024-09-10T23:40:05.734537+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:02.324338+0000","last_clean_scrub_stamp":"2024-09-10T23:40:02.324338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:08.469491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:05.727539+0000","last_change":"2024-09-10T23:40:02.354814+0000","last_active":"2024-09-10T23:40:05.727539+0000","last_peered":"2024-09-10T23:40:05.727539+0000","last_clean":"2024-09-10T23:40:05.727539+0000","last_became_active":"2024-09-10T23:40:02.354579+0000","last_became_peered":"2024-09-10T23:40:02.354579+0000","last_unstale":"2024-09-10T23:40:05.727539+0000","last_undegraded":"2024-09-10T23:40:05.727539+0000","last_fullsized":"2024-09-10T23:40:05.727539+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:01.318012+0000","last_clean_scrub_stamp":"2024-09-10T23:40:01.318012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:17.250644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":375,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-10T23:40:04.982735+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:40:04.982735+0000","last_peered":"2024-09-10T23:40:04.982735+0000","last_clean":"2024-09-10T23:40:04.982735+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:40:04.982735+0000","last_undegraded":"2024-09-10T23:40:04.982735+0000","last_fullsized":"2024-09-10T23:40:04.982735+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509537,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33060,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1916928,"data_stored":838132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542240,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1900544,"data_stored":836498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509536,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33632,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338208,"statfs":{"total":96636764160,"available":96602324992,"internally_reserved":0,"allocated":2498560,"data_stored":1427380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509534,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1900544,"data_stored":836442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509535,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":838090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509536,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37204,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334636,"statfs":{"total":96636764160,"available":96598667264,"internally_reserved":0,"allocated":1900544,"data_stored":836478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509535,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37784,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334056,"statfs":{"total":96636764160,"available":96598073344,"internally_reserved":0,"allocated":2490368,"data_stored":1426776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509535,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37208,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334632,"statfs":{"total":96636764160,"available":96598663168,"internally_reserved":0,"allocated":1904640,"data_stored":837024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:40:12.431 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T23:40:12.432 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T23:40:12.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.615+0000 7f8457c00640 1 -- 172.21.15.94:0/3514500541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84500d0830 msgr2=0x7f84500c5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:12.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.615+0000 7f8457c00640 1 --2- 172.21.15.94:0/3514500541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84500d0830 0x7f84500c5e50 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f8448009c80 tx=0x7f844801ccd0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.615+0000 7f8457c00640 1 -- 172.21.15.94:0/3514500541 shutdown_connections 2024-09-10T23:40:12.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.615+0000 7f8457c00640 1 --2- 172.21.15.94:0/3514500541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84500d0830 0x7f84500c5e50 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.615+0000 7f8457c00640 1 --2- 172.21.15.94:0/3514500541 >> v2:172.21.15.181:3300/0 conn(0x7f845015bf60 0x7f84500d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.615+0000 7f8457c00640 1 -- 172.21.15.94:0/3514500541 >> 172.21.15.94:0/3514500541 conn(0x7f84500d1c60 msgr2=0x7f84500d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:12.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.615+0000 7f8457c00640 1 -- 172.21.15.94:0/3514500541 shutdown_connections 2024-09-10T23:40:12.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.615+0000 7f8457c00640 1 -- 172.21.15.94:0/3514500541 wait complete. 2024-09-10T23:40:12.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8457c00640 1 Processor -- start 2024-09-10T23:40:12.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8457c00640 1 -- start start 2024-09-10T23:40:12.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8457c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84500cfce0 0x7f8450154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:12.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8457c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84500d0830 0x7f8450157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:12.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8457c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f845015bf60 0x7f8450157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:12.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8457c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f84500c6fe0 con 0x7f84500d0830 2024-09-10T23:40:12.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8457c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f84500c6e60 con 0x7f84500cfce0 2024-09-10T23:40:12.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8457c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f84500c7160 con 0x7f845015bf60 2024-09-10T23:40:12.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8447e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84500d0830 0x7f8450157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:12.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8447e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84500d0830 0x7f8450157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34676/0 (socket says 172.21.15.94:34676) 2024-09-10T23:40:12.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8447e00640 1 -- 172.21.15.94:0/2417450145 learned_addr learned my addr 172.21.15.94:0/2417450145 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:12.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8455200640 1 --2- 172.21.15.94:0/2417450145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84500cfce0 0x7f8450154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:12.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8455c00640 1 --2- 172.21.15.94:0/2417450145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f845015bf60 0x7f8450157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:12.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8447e00640 1 -- 172.21.15.94:0/2417450145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f845015bf60 msgr2=0x7f8450157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:12.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8447e00640 1 --2- 172.21.15.94:0/2417450145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f845015bf60 0x7f8450157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8447e00640 1 -- 172.21.15.94:0/2417450145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84500cfce0 msgr2=0x7f8450154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:12.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8447e00640 1 --2- 172.21.15.94:0/2417450145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84500cfce0 0x7f8450154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8447e00640 1 -- 172.21.15.94:0/2417450145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8450152d10 con 0x7f84500d0830 2024-09-10T23:40:12.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8447e00640 1 --2- 172.21.15.94:0/2417450145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84500d0830 0x7f8450157520 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f843800c4e0 tx=0x7f843800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:12.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f844d600640 1 -- 172.21.15.94:0/2417450145 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84380052d0 con 0x7f84500d0830 2024-09-10T23:40:12.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f844d600640 1 -- 172.21.15.94:0/2417450145 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8438005470 con 0x7f84500d0830 2024-09-10T23:40:12.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f844d600640 1 -- 172.21.15.94:0/2417450145 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438012890 con 0x7f84500d0830 2024-09-10T23:40:12.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8457c00640 1 -- 172.21.15.94:0/2417450145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8450152ff0 con 0x7f84500d0830 2024-09-10T23:40:12.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.619+0000 7f8457c00640 1 -- 172.21.15.94:0/2417450145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84501535b0 con 0x7f84500d0830 2024-09-10T23:40:12.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.623+0000 7f8457c00640 1 -- 172.21.15.94:0/2417450145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84500c5e50 con 0x7f84500d0830 2024-09-10T23:40:12.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.623+0000 7f844d600640 1 -- 172.21.15.94:0/2417450145 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f8438003600 con 0x7f84500d0830 2024-09-10T23:40:12.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.623+0000 7f844d600640 1 --2- 172.21.15.94:0/2417450145 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f842c072e80 0x7f842c075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:12.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.623+0000 7f844d600640 1 -- 172.21.15.94:0/2417450145 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f8438093f90 con 0x7f84500d0830 2024-09-10T23:40:12.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.623+0000 7f8455200640 1 --2- 172.21.15.94:0/2417450145 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f842c072e80 0x7f842c075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:12.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.623+0000 7f8455200640 1 --2- 172.21.15.94:0/2417450145 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f842c072e80 0x7f842c075340 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8450158570 tx=0x7f844000b2e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:12.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.627+0000 7f844d600640 1 -- 172.21.15.94:0/2417450145 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8438097030 con 0x7f84500d0830 2024-09-10T23:40:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.855+0000 7f8457c00640 1 -- 172.21.15.94:0/2417450145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f8450152670 con 0x7f84500d0830 2024-09-10T23:40:12.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.859+0000 7f844d600640 1 -- 172.21.15.94:0/2417450145 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v39) ==== 190+0+147 (secure 0 0 0) 0x7f843805dfd0 con 0x7f84500d0830 2024-09-10T23:40:12.862 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-10T23:40:12.862 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g== 2024-09-10T23:40:12.862 INFO:teuthology.orchestra.run.smithi094.stdout: caps mds = "allow" 2024-09-10T23:40:12.862 INFO:teuthology.orchestra.run.smithi094.stdout: caps mon = "allow r" 2024-09-10T23:40:12.863 INFO:teuthology.orchestra.run.smithi094.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T23:40:12.863 INFO:teuthology.orchestra.run.smithi094.stderr:updated caps for client.0 2024-09-10T23:40:12.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.859+0000 7f8457c00640 1 -- 172.21.15.94:0/2417450145 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f842c072e80 msgr2=0x7f842c075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:12.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.859+0000 7f8457c00640 1 --2- 172.21.15.94:0/2417450145 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f842c072e80 0x7f842c075340 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8450158570 tx=0x7f844000b2e0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.859+0000 7f8457c00640 1 -- 172.21.15.94:0/2417450145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84500d0830 msgr2=0x7f8450157520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.859+0000 7f8457c00640 1 --2- 172.21.15.94:0/2417450145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84500d0830 0x7f8450157520 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f843800c4e0 tx=0x7f843800c9a0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.859+0000 7f8457c00640 1 -- 172.21.15.94:0/2417450145 shutdown_connections 2024-09-10T23:40:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.859+0000 7f8457c00640 1 --2- 172.21.15.94:0/2417450145 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f842c072e80 0x7f842c075340 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.859+0000 7f8457c00640 1 --2- 172.21.15.94:0/2417450145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f845015bf60 0x7f8450157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.859+0000 7f8457c00640 1 --2- 172.21.15.94:0/2417450145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84500d0830 0x7f8450157520 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.859+0000 7f8457c00640 1 --2- 172.21.15.94:0/2417450145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84500cfce0 0x7f8450154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.859+0000 7f8457c00640 1 -- 172.21.15.94:0/2417450145 >> 172.21.15.94:0/2417450145 conn(0x7f84500d1c60 msgr2=0x7f84500c77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.859+0000 7f8457c00640 1 -- 172.21.15.94:0/2417450145 shutdown_connections 2024-09-10T23:40:12.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:12.859+0000 7f8457c00640 1 -- 172.21.15.94:0/2417450145 wait complete. 2024-09-10T23:40:12.884 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:40:13.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 -- 172.21.15.94:0/2600257929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f202c0ae980 msgr2=0x7f202c0d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:13.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 --2- 172.21.15.94:0/2600257929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f202c0ae980 0x7f202c0d1b80 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f202c0d2850 tx=0x7f201401c830 comp rx=0 tx=0).stop 2024-09-10T23:40:13.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 -- 172.21.15.94:0/2600257929 shutdown_connections 2024-09-10T23:40:13.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 --2- 172.21.15.94:0/2600257929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f202c0b2ec0 0x7f202c0d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 --2- 172.21.15.94:0/2600257929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f202c0ae980 0x7f202c0d1b80 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 --2- 172.21.15.94:0/2600257929 >> v2:172.21.15.181:3300/0 conn(0x7f202c15bd30 0x7f202c15c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 -- 172.21.15.94:0/2600257929 >> 172.21.15.94:0/2600257929 conn(0x7f202c09aa70 msgr2=0x7f202c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:13.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 -- 172.21.15.94:0/2600257929 shutdown_connections 2024-09-10T23:40:13.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 -- 172.21.15.94:0/2600257929 wait complete. 2024-09-10T23:40:13.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 Processor -- start 2024-09-10T23:40:13.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 -- start start 2024-09-10T23:40:13.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f202c0ae980 0x7f202c159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f202c0b2ec0 0x7f202c156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f202c15bd30 0x7f202c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f202c0b1ff0 con 0x7f202c0b2ec0 2024-09-10T23:40:13.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f202c0b1e70 con 0x7f202c0ae980 2024-09-10T23:40:13.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2032200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f202c0b2170 con 0x7f202c15bd30 2024-09-10T23:40:13.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2022a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f202c0b2ec0 0x7f202c156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2022a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f202c0b2ec0 0x7f202c156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34694/0 (socket says 172.21.15.94:34694) 2024-09-10T23:40:13.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f2022a00640 1 -- 172.21.15.94:0/4172555537 learned_addr learned my addr 172.21.15.94:0/4172555537 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:13.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f202b400640 1 --2- 172.21.15.94:0/4172555537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f202c0ae980 0x7f202c159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.023+0000 7f202be00640 1 --2- 172.21.15.94:0/4172555537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f202c15bd30 0x7f202c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.027+0000 7f2022a00640 1 -- 172.21.15.94:0/4172555537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f202c15bd30 msgr2=0x7f202c06eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:13.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.027+0000 7f2022a00640 1 --2- 172.21.15.94:0/4172555537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f202c15bd30 0x7f202c06eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.027+0000 7f2022a00640 1 -- 172.21.15.94:0/4172555537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f202c0ae980 msgr2=0x7f202c159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:13.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.027+0000 7f2022a00640 1 --2- 172.21.15.94:0/4172555537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f202c0ae980 0x7f202c159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.027+0000 7f2022a00640 1 -- 172.21.15.94:0/4172555537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202c0b56c0 con 0x7f202c0b2ec0 2024-09-10T23:40:13.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.027+0000 7f2022a00640 1 --2- 172.21.15.94:0/4172555537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f202c0b2ec0 0x7f202c156440 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f20140b0040 tx=0x7f2014004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:13.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.027+0000 7f2023e00640 1 -- 172.21.15.94:0/4172555537 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20140b2040 con 0x7f202c0b2ec0 2024-09-10T23:40:13.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.027+0000 7f2023e00640 1 -- 172.21.15.94:0/4172555537 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2014005030 con 0x7f202c0b2ec0 2024-09-10T23:40:13.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.027+0000 7f2023e00640 1 -- 172.21.15.94:0/4172555537 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014005480 con 0x7f202c0b2ec0 2024-09-10T23:40:13.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.027+0000 7f2032200640 1 -- 172.21.15.94:0/4172555537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202c15a500 con 0x7f202c0b2ec0 2024-09-10T23:40:13.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.027+0000 7f2032200640 1 -- 172.21.15.94:0/4172555537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f202c0b5960 con 0x7f202c0b2ec0 2024-09-10T23:40:13.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.027+0000 7f2032200640 1 -- 172.21.15.94:0/4172555537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f202c15c110 con 0x7f202c0b2ec0 2024-09-10T23:40:13.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.027+0000 7f2023e00640 1 -- 172.21.15.94:0/4172555537 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f20140046d0 con 0x7f202c0b2ec0 2024-09-10T23:40:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.031+0000 7f2023e00640 1 --2- 172.21.15.94:0/4172555537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2008072e50 0x7f2008075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.031+0000 7f2023e00640 1 -- 172.21.15.94:0/4172555537 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f201412d6a0 con 0x7f202c0b2ec0 2024-09-10T23:40:13.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.031+0000 7f202b400640 1 --2- 172.21.15.94:0/4172555537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2008072e50 0x7f2008075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.031+0000 7f202b400640 1 --2- 172.21.15.94:0/4172555537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2008072e50 0x7f2008075310 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f202c0cc1b0 tx=0x7f201c00b2e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:13.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.035+0000 7f2023e00640 1 -- 172.21.15.94:0/4172555537 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f20140164b0 con 0x7f202c0b2ec0 2024-09-10T23:40:13.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.235+0000 7f2032200640 1 -- 172.21.15.94:0/4172555537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f202c0b0080 con 0x7f202c0b2ec0 2024-09-10T23:40:13.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.235+0000 7f2023e00640 1 -- 172.21.15.94:0/4172555537 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5249 (secure 0 0 0) 0x7f20140f7cc0 con 0x7f202c0b2ec0 2024-09-10T23:40:13.238 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:13.238 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":107,"btime":"2024-09-10T23:40:06:362943+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8158,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1859150794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":1859150794},{"type":"v1","addr":"172.21.15.94:6835","nonce":1859150794}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8173,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/3891720008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3891720008},{"type":"v1","addr":"172.21.15.181:6801","nonce":3891720008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8787,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/4040262257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":4040262257},{"type":"v1","addr":"172.21.15.94:6834","nonce":4040262257}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:40:02.727036+0000","modified":"2024-09-10T23:40:05.749707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":197120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_197120":{"gid":197120,"name":"b","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/2758853868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2758853868},{"type":"v1","addr":"172.21.15.181:6835","nonce":2758853868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":197120,"qdb_cluster":[197120]},"id":12}]} 2024-09-10T23:40:13.238 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 107 2024-09-10T23:40:13.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.235+0000 7f2032200640 1 -- 172.21.15.94:0/4172555537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2008072e50 msgr2=0x7f2008075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:13.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.235+0000 7f2032200640 1 --2- 172.21.15.94:0/4172555537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2008072e50 0x7f2008075310 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f202c0cc1b0 tx=0x7f201c00b2e0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.235+0000 7f2032200640 1 -- 172.21.15.94:0/4172555537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f202c0b2ec0 msgr2=0x7f202c156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:13.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.235+0000 7f2032200640 1 --2- 172.21.15.94:0/4172555537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f202c0b2ec0 0x7f202c156440 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f20140b0040 tx=0x7f2014004040 comp rx=0 tx=0).stop 2024-09-10T23:40:13.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.235+0000 7f2032200640 1 -- 172.21.15.94:0/4172555537 shutdown_connections 2024-09-10T23:40:13.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.235+0000 7f2032200640 1 --2- 172.21.15.94:0/4172555537 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2008072e50 0x7f2008075310 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.235+0000 7f2032200640 1 --2- 172.21.15.94:0/4172555537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f202c15bd30 0x7f202c06eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.235+0000 7f2032200640 1 --2- 172.21.15.94:0/4172555537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f202c0b2ec0 0x7f202c156440 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.235+0000 7f2032200640 1 --2- 172.21.15.94:0/4172555537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f202c0ae980 0x7f202c159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.235+0000 7f2032200640 1 -- 172.21.15.94:0/4172555537 >> 172.21.15.94:0/4172555537 conn(0x7f202c09aa70 msgr2=0x7f202c071200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:13.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.235+0000 7f2032200640 1 -- 172.21.15.94:0/4172555537 shutdown_connections 2024-09-10T23:40:13.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.235+0000 7f2032200640 1 -- 172.21.15.94:0/4172555537 wait complete. 2024-09-10T23:40:13.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 107, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T23:40:02.727036+0000', 'modified': '2024-09-10T23:40:05.749707+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 197120}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_197120': {'gid': 197120, 'name': 'b', 'rank': 0, 'incarnation': 103, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6835/2758853868', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 2758853868}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 2758853868}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 197120, 'qdb_cluster': [197120]} 2024-09-10T23:40:13.662 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T23:40:13.662 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T23:40:13.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.803+0000 7ff8ce000640 1 -- 172.21.15.94:0/1292252899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c813d530 msgr2=0x7ff8c815dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:13.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.803+0000 7ff8ce000640 1 --2- 172.21.15.94:0/1292252899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c813d530 0x7ff8c815dfb0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8069670 tx=0x7ff8c001ca30 comp rx=0 tx=0).stop 2024-09-10T23:40:13.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.803+0000 7ff8ce000640 1 -- 172.21.15.94:0/1292252899 shutdown_connections 2024-09-10T23:40:13.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.803+0000 7ff8ce000640 1 --2- 172.21.15.94:0/1292252899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c813d530 0x7ff8c815dfb0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.803+0000 7ff8ce000640 1 --2- 172.21.15.94:0/1292252899 >> v2:172.21.15.181:3300/0 conn(0x7ff8c813cc00 0x7ff8c815da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.803+0000 7ff8ce000640 1 -- 172.21.15.94:0/1292252899 >> 172.21.15.94:0/1292252899 conn(0x7ff8c80cfaf0 msgr2=0x7ff8c80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:13.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.803+0000 7ff8ce000640 1 -- 172.21.15.94:0/1292252899 shutdown_connections 2024-09-10T23:40:13.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.803+0000 7ff8ce000640 1 -- 172.21.15.94:0/1292252899 wait complete. 2024-09-10T23:40:13.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.803+0000 7ff8ce000640 1 Processor -- start 2024-09-10T23:40:13.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.803+0000 7ff8ce000640 1 -- start start 2024-09-10T23:40:13.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8ce000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c813cc00 0x7ff8c812f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8ce000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8c813cfe0 0x7ff8c81ecb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8ce000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c813d530 0x7ff8c80b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8ce000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c813f4a0 con 0x7ff8c813cc00 2024-09-10T23:40:13.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8ce000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c813f320 con 0x7ff8c813cfe0 2024-09-10T23:40:13.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8ce000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c813f620 con 0x7ff8c813d530 2024-09-10T23:40:13.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8c7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c813cc00 0x7ff8c812f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8c7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c813cc00 0x7ff8c812f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36058/0 (socket says 172.21.15.94:36058) 2024-09-10T23:40:13.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8c7400640 1 -- 172.21.15.94:0/1278075080 learned_addr learned my addr 172.21.15.94:0/1278075080 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:13.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8c6a00640 1 --2- 172.21.15.94:0/1278075080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8c813cfe0 0x7ff8c81ecb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8c7e00640 1 --2- 172.21.15.94:0/1278075080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c813d530 0x7ff8c80b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8c7400640 1 -- 172.21.15.94:0/1278075080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c813d530 msgr2=0x7ff8c80b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:13.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8c7400640 1 --2- 172.21.15.94:0/1278075080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c813d530 0x7ff8c80b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8c7400640 1 -- 172.21.15.94:0/1278075080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8c813cfe0 msgr2=0x7ff8c81ecb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:13.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8c7400640 1 --2- 172.21.15.94:0/1278075080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8c813cfe0 0x7ff8c81ecb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8c7400640 1 -- 172.21.15.94:0/1278075080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c81ed510 con 0x7ff8c813cc00 2024-09-10T23:40:13.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8c7e00640 1 --2- 172.21.15.94:0/1278075080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c813d530 0x7ff8c80b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:13.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8c6a00640 1 --2- 172.21.15.94:0/1278075080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8c813cfe0 0x7ff8c81ecb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:13.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8c7400640 1 --2- 172.21.15.94:0/1278075080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c813cc00 0x7ff8c812f1d0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800dbf0 tx=0x7ff8b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:13.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8bf400640 1 -- 172.21.15.94:0/1278075080 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8004220 con 0x7ff8c813cc00 2024-09-10T23:40:13.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8bf400640 1 -- 172.21.15.94:0/1278075080 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8b800ae50 con 0x7ff8c813cc00 2024-09-10T23:40:13.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8bf400640 1 -- 172.21.15.94:0/1278075080 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8003860 con 0x7ff8c813cc00 2024-09-10T23:40:13.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8ce000640 1 -- 172.21.15.94:0/1278075080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c81ed7f0 con 0x7ff8c813cc00 2024-09-10T23:40:13.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.807+0000 7ff8ce000640 1 -- 172.21.15.94:0/1278075080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c812f710 con 0x7ff8c813cc00 2024-09-10T23:40:13.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8bf400640 1 -- 172.21.15.94:0/1278075080 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7ff8b800b040 con 0x7ff8c813cc00 2024-09-10T23:40:13.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8bf400640 1 --2- 172.21.15.94:0/1278075080 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff8a8072e50 0x7ff8a8075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8d141b000 1 Processor -- start 2024-09-10T23:40:13.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8bf400640 1 -- 172.21.15.94:0/1278075080 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7ff8b8094450 con 0x7ff8c813cc00 2024-09-10T23:40:13.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8d141b000 1 -- start start 2024-09-10T23:40:13.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8c6a00640 1 --2- 172.21.15.94:0/1278075080 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff8a8072e50 0x7ff8a8075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8c6a00640 1 --2- 172.21.15.94:0/1278075080 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff8a8072e50 0x7ff8a8075310 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8165cd0 tx=0x7ff8b4009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:13.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8d141b000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602426e29d0 0x5602426e2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8d141b000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5602426e32f0 0x5602426968d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8d141b000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560242696e10 0x5602426971f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8d141b000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5602425b73f0 con 0x5602426e29d0 2024-09-10T23:40:13.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8d141b000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5602424f4b20 con 0x560242696e10 2024-09-10T23:40:13.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8d141b000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x5602424e3f50 con 0x5602426e32f0 2024-09-10T23:40:13.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8c7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602426e29d0 0x5602426e2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8c7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602426e29d0 0x5602426e2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36068/0 (socket says 172.21.15.94:36068) 2024-09-10T23:40:13.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8c7400640 1 -- 172.21.15.94:0/3488073149 learned_addr learned my addr 172.21.15.94:0/3488073149 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:13.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8c6a00640 1 --2- 172.21.15.94:0/3488073149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5602426e32f0 0x5602426968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8c7e00640 1 --2- 172.21.15.94:0/3488073149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560242696e10 0x5602426971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8c7400640 1 -- 172.21.15.94:0/3488073149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5602426e32f0 msgr2=0x5602426968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:40:13.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8c7400640 1 --2- 172.21.15.94:0/3488073149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5602426e32f0 0x5602426968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8c7400640 1 -- 172.21.15.94:0/3488073149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560242696e10 msgr2=0x5602426971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:40:13.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8c7400640 1 --2- 172.21.15.94:0/3488073149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560242696e10 0x5602426971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8c7400640 1 -- 172.21.15.94:0/3488073149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602424ab590 con 0x5602426e29d0 2024-09-10T23:40:13.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8c7e00640 1 --2- 172.21.15.94:0/3488073149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560242696e10 0x5602426971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:40:13.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8c7400640 1 --2- 172.21.15.94:0/3488073149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602426e29d0 0x5602426e2db0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7ff8b805e580 tx=0x7ff8b805e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de88793ba9803419 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:13.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8b3e00640 1 -- 172.21.15.94:0/3488073149 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b805e390 con 0x5602426e29d0 2024-09-10T23:40:13.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8b3e00640 1 -- 172.21.15.94:0/3488073149 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8b805ef80 con 0x5602426e29d0 2024-09-10T23:40:13.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.811+0000 7ff8b3e00640 1 -- 172.21.15.94:0/3488073149 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b805e880 con 0x5602426e29d0 2024-09-10T23:40:13.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 -- 172.21.15.94:0/3488073149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602426e29d0 msgr2=0x5602426e2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:13.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 --2- 172.21.15.94:0/3488073149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602426e29d0 0x5602426e2db0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7ff8b805e580 tx=0x7ff8b805e650 comp rx=0 tx=0).stop 2024-09-10T23:40:13.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 -- 172.21.15.94:0/3488073149 shutdown_connections 2024-09-10T23:40:13.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 --2- 172.21.15.94:0/3488073149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560242696e10 0x5602426971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 --2- 172.21.15.94:0/3488073149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5602426e32f0 0x5602426968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 --2- 172.21.15.94:0/3488073149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602426e29d0 0x5602426e2db0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 -- 172.21.15.94:0/3488073149 >> 172.21.15.94:0/3488073149 conn(0x560242580130 msgr2=0x5602426c5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:13.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 -- 172.21.15.94:0/3488073149 shutdown_connections 2024-09-10T23:40:13.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 -- 172.21.15.94:0/3488073149 wait complete. 2024-09-10T23:40:13.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 -- start start 2024-09-10T23:40:13.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 Processor -- start 2024-09-10T23:40:13.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560242696e10 0x5602426c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602426e29d0 0x5602426c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e32f0 0x5602426c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56024247e5d0 con 0x5602426e29d0 2024-09-10T23:40:13.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5602424e7d60 con 0x5602426e32f0 2024-09-10T23:40:13.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x56024248a500 con 0x560242696e10 2024-09-10T23:40:13.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 10 client.0 _ll_register_callbacks cb 0x56024248e810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:40:13.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8c6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602426e29d0 0x5602426c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8c6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602426e29d0 0x5602426c3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36084/0 (socket says 172.21.15.94:36084) 2024-09-10T23:40:13.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8c6a00640 1 -- 172.21.15.94:0/2745194068 learned_addr learned my addr 172.21.15.94:0/2745194068 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:13.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8c7e00640 1 --2- 172.21.15.94:0/2745194068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e32f0 0x5602426c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8c7400640 1 --2- 172.21.15.94:0/2745194068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560242696e10 0x5602426c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8c6a00640 1 -- 172.21.15.94:0/2745194068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560242696e10 msgr2=0x5602426c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:40:13.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8c6a00640 1 --2- 172.21.15.94:0/2745194068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560242696e10 0x5602426c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8c6a00640 1 -- 172.21.15.94:0/2745194068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e32f0 msgr2=0x5602426c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:40:13.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8c6a00640 1 --2- 172.21.15.94:0/2745194068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e32f0 0x5602426c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8c6a00640 1 -- 172.21.15.94:0/2745194068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560242482170 con 0x5602426e29d0 2024-09-10T23:40:13.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8c7e00640 1 --2- 172.21.15.94:0/2745194068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e32f0 0x5602426c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:40:13.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8c7400640 1 --2- 172.21.15.94:0/2745194068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560242696e10 0x5602426c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:40:13.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8c6a00640 1 --2- 172.21.15.94:0/2745194068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602426e29d0 0x5602426c3e70 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400ddb0 tx=0x7ff8b4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948b1e4e20bb1075 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:13.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:40:13.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8b0c00640 1 -- 172.21.15.94:0/2745194068 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4007050 con 0x5602426e29d0 2024-09-10T23:40:13.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 -- 172.21.15.94:0/2745194068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56024247c0f0 con 0x5602426e29d0 2024-09-10T23:40:13.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8b0c00640 1 -- 172.21.15.94:0/2745194068 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7ff8b40071f0 con 0x5602426e29d0 2024-09-10T23:40:13.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8b0c00640 1 -- 172.21.15.94:0/2745194068 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7ff8b402d050 con 0x5602426e29d0 2024-09-10T23:40:13.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 10 client.8928 fetch_fsmap learned FSMap version 107 2024-09-10T23:40:13.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8d141b000 1 -- 172.21.15.94:0/2745194068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x560242568ef0 con 0x5602426e29d0 2024-09-10T23:40:13.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.815+0000 7ff8b0c00640 1 -- 172.21.15.94:0/2745194068 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56024247c0f0 con 0x5602426e29d0 2024-09-10T23:40:13.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8b0c00640 1 -- 172.21.15.94:0/2745194068 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 107) ==== 3502+0+0 (secure 0 0 0) 0x7ff8b4030020 con 0x5602426e29d0 2024-09-10T23:40:13.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 10 client.8928 fetch_fsmap finished waiting for FSMap version 107 2024-09-10T23:40:13.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 10 client.8928 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T23:40:13.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 20 client.8928 populate_metadata read hostname 'smithi094' 2024-09-10T23:40:13.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 1 --2- 172.21.15.94:0/2745194068 >> [v2:172.21.15.181:6834/2758853868,v1:172.21.15.181:6835/2758853868] conn(0x5602426c50f0 0x5602426f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 4 client.8928 mds_command: new command op to 197120 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:40:13.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 1 -- 172.21.15.94:0/2745194068 --> [v2:172.21.15.181:6834/2758853868,v1:172.21.15.181:6835/2758853868] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602424f6a70 con 0x5602426c50f0 2024-09-10T23:40:13.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8c7400640 1 --2- 172.21.15.94:0/2745194068 >> [v2:172.21.15.181:6834/2758853868,v1:172.21.15.181:6835/2758853868] conn(0x5602426c50f0 0x5602426f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8c7400640 1 --2- 172.21.15.94:0/2745194068 >> [v2:172.21.15.181:6834/2758853868,v1:172.21.15.181:6835/2758853868] conn(0x5602426c50f0 0x5602426f4c90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ffaf9b36c53cd50 server_cookie=dd6072d5d8a8276d in_seq=0 out_seq=0 2024-09-10T23:40:13.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8b0c00640 10 client.8928 ms_handle_connect on v2:172.21.15.181:6834/2758853868 2024-09-10T23:40:13.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8b0c00640 1 -- 172.21.15.94:0/2745194068 <== mds.0 v2:172.21.15.181:6834/2758853868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5602424f6a70 con 0x5602426c50f0 2024-09-10T23:40:13.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8b0c00640 10 client.8928 handle_command_reply: tid=0 2024-09-10T23:40:13.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 1 client.8928 shutdown 2024-09-10T23:40:13.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 1 -- 172.21.15.94:0/2745194068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602426e29d0 msgr2=0x5602426c3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:13.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 1 --2- 172.21.15.94:0/2745194068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602426e29d0 0x5602426c3e70 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400ddb0 tx=0x7ff8b4000ee0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 1 -- 172.21.15.94:0/2745194068 shutdown_connections 2024-09-10T23:40:13.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 1 --2- 172.21.15.94:0/2745194068 >> [v2:172.21.15.181:6834/2758853868,v1:172.21.15.181:6835/2758853868] conn(0x5602426c50f0 0x5602426f4c90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 1 --2- 172.21.15.94:0/2745194068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e32f0 0x5602426c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 1 --2- 172.21.15.94:0/2745194068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602426e29d0 0x5602426c3e70 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8b0c00640 0 client.8928 ms_handle_reset on v2:172.21.15.181:6834/2758853868 2024-09-10T23:40:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 1 --2- 172.21.15.94:0/2745194068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x560242696e10 0x5602426c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 1 -- 172.21.15.94:0/2745194068 >> 172.21.15.94:0/2745194068 conn(0x560242580130 msgr2=0x560242695fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 1 -- 172.21.15.94:0/2745194068 shutdown_connections 2024-09-10T23:40:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 1 -- 172.21.15.94:0/2745194068 wait complete. 2024-09-10T23:40:13.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.819+0000 7ff8d141b000 20 client.8928 trim_cache size 0 max 16384 2024-09-10T23:40:13.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8d141b000 1 Processor -- start 2024-09-10T23:40:13.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8d141b000 1 -- start start 2024-09-10T23:40:13.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8d141b000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560242696e10 0x560242754bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8d141b000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5602426e29d0 0x560242755100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8d141b000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e32f0 0x560242755640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8d141b000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81fbf70 con 0x560242696e10 2024-09-10T23:40:13.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8d141b000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c815d130 con 0x5602426e32f0 2024-09-10T23:40:13.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8d141b000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c80b5000 con 0x5602426e29d0 2024-09-10T23:40:13.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8c7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560242696e10 0x560242754bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8c7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560242696e10 0x560242754bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36090/0 (socket says 172.21.15.94:36090) 2024-09-10T23:40:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8c7400640 1 -- 172.21.15.94:0/2468207754 learned_addr learned my addr 172.21.15.94:0/2468207754 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8c6a00640 1 --2- 172.21.15.94:0/2468207754 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5602426e29d0 0x560242755100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8c7e00640 1 --2- 172.21.15.94:0/2468207754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e32f0 0x560242755640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8c7400640 1 -- 172.21.15.94:0/2468207754 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5602426e29d0 msgr2=0x560242755100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:40:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8c7400640 1 --2- 172.21.15.94:0/2468207754 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5602426e29d0 0x560242755100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8c7400640 1 -- 172.21.15.94:0/2468207754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e32f0 msgr2=0x560242755640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:40:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8c7400640 1 --2- 172.21.15.94:0/2468207754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e32f0 0x560242755640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8c7400640 1 -- 172.21.15.94:0/2468207754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56024253fd90 con 0x560242696e10 2024-09-10T23:40:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8c7e00640 1 --2- 172.21.15.94:0/2468207754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e32f0 0x560242755640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.839+0000 7ff8c7400640 1 --2- 172.21.15.94:0/2468207754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560242696e10 0x560242754bc0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7ff8b805baf0 tx=0x7ff8b8021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af41dc1b5cb3d61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:13.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8b3400640 1 -- 172.21.15.94:0/2468207754 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8063360 con 0x560242696e10 2024-09-10T23:40:13.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8b3400640 1 -- 172.21.15.94:0/2468207754 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8b8067d80 con 0x560242696e10 2024-09-10T23:40:13.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8b3400640 1 -- 172.21.15.94:0/2468207754 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b805e880 con 0x560242696e10 2024-09-10T23:40:13.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 -- 172.21.15.94:0/2468207754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560242696e10 msgr2=0x560242754bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:13.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 --2- 172.21.15.94:0/2468207754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560242696e10 0x560242754bc0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7ff8b805baf0 tx=0x7ff8b8021040 comp rx=0 tx=0).stop 2024-09-10T23:40:13.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 -- 172.21.15.94:0/2468207754 shutdown_connections 2024-09-10T23:40:13.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 --2- 172.21.15.94:0/2468207754 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e32f0 0x560242755640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 --2- 172.21.15.94:0/2468207754 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5602426e29d0 0x560242755100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 --2- 172.21.15.94:0/2468207754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560242696e10 0x560242754bc0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 -- 172.21.15.94:0/2468207754 >> 172.21.15.94:0/2468207754 conn(0x560242580130 msgr2=0x5602426effb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:13.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 -- 172.21.15.94:0/2468207754 shutdown_connections 2024-09-10T23:40:13.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 -- 172.21.15.94:0/2468207754 wait complete. 2024-09-10T23:40:13.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 -- start start 2024-09-10T23:40:13.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 Processor -- start 2024-09-10T23:40:13.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560242696e10 0x560242758c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e29d0 0x560242759140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5602426e32f0 0x560242759680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56024269f300 con 0x560242696e10 2024-09-10T23:40:13.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5602426f9ce0 con 0x5602426e29d0 2024-09-10T23:40:13.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x5602424ec390 con 0x5602426e32f0 2024-09-10T23:40:13.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 10 client.0 _ll_register_callbacks cb 0x7ff8b4001d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:40:13.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8c7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560242696e10 0x560242758c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8c7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560242696e10 0x560242758c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36104/0 (socket says 172.21.15.94:36104) 2024-09-10T23:40:13.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8c7400640 1 -- 172.21.15.94:0/3993403156 learned_addr learned my addr 172.21.15.94:0/3993403156 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:13.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8c6a00640 1 --2- 172.21.15.94:0/3993403156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e29d0 0x560242759140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8c7e00640 1 --2- 172.21.15.94:0/3993403156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5602426e32f0 0x560242759680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8c7400640 1 -- 172.21.15.94:0/3993403156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5602426e32f0 msgr2=0x560242759680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:40:13.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8c7400640 1 --2- 172.21.15.94:0/3993403156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5602426e32f0 0x560242759680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8c7400640 1 -- 172.21.15.94:0/3993403156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e29d0 msgr2=0x560242759140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:40:13.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8c7400640 1 --2- 172.21.15.94:0/3993403156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e29d0 0x560242759140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8c7400640 1 -- 172.21.15.94:0/3993403156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602424aaa90 con 0x560242696e10 2024-09-10T23:40:13.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8c7400640 1 --2- 172.21.15.94:0/3993403156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560242696e10 0x560242758c00 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800dbc0 tx=0x7ff8b8004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8a1c50a26a0051 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:13.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:40:13.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8b0c00640 1 -- 172.21.15.94:0/3993403156 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b805e880 con 0x560242696e10 2024-09-10T23:40:13.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8b0c00640 1 -- 172.21.15.94:0/3993403156 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7ff8b80678f0 con 0x560242696e10 2024-09-10T23:40:13.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 -- 172.21.15.94:0/3993403156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8b8067d80 con 0x560242696e10 2024-09-10T23:40:13.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8b0c00640 1 -- 172.21.15.94:0/3993403156 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7ff8b8063360 con 0x560242696e10 2024-09-10T23:40:13.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 10 client.8934 fetch_fsmap learned FSMap version 107 2024-09-10T23:40:13.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.843+0000 7ff8d141b000 1 -- 172.21.15.94:0/3993403156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x560242489050 con 0x560242696e10 2024-09-10T23:40:13.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8b0c00640 1 -- 172.21.15.94:0/3993403156 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8b8067d80 con 0x560242696e10 2024-09-10T23:40:13.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8b0c00640 1 -- 172.21.15.94:0/3993403156 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 107) ==== 3502+0+0 (secure 0 0 0) 0x7ff8b805fbb0 con 0x560242696e10 2024-09-10T23:40:13.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 10 client.8934 fetch_fsmap finished waiting for FSMap version 107 2024-09-10T23:40:13.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 10 client.8934 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T23:40:13.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 20 client.8934 populate_metadata read hostname 'smithi094' 2024-09-10T23:40:13.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 1 --2- 172.21.15.94:0/3993403156 >> [v2:172.21.15.181:6834/2758853868,v1:172.21.15.181:6835/2758853868] conn(0x5602426f6b20 0x5602426f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:13.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 4 client.8934 mds_command: new command op to 197120 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T23:40:13.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 1 -- 172.21.15.94:0/3993403156 --> [v2:172.21.15.181:6834/2758853868,v1:172.21.15.181:6835/2758853868] -- command(tid 0: {"prefix": "status"}) -- 0x5602424f2520 con 0x5602426f6b20 2024-09-10T23:40:13.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8c6a00640 1 --2- 172.21.15.94:0/3993403156 >> [v2:172.21.15.181:6834/2758853868,v1:172.21.15.181:6835/2758853868] conn(0x5602426f6b20 0x5602426f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8c6a00640 1 --2- 172.21.15.94:0/3993403156 >> [v2:172.21.15.181:6834/2758853868,v1:172.21.15.181:6835/2758853868] conn(0x5602426f6b20 0x5602426f6f00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ecb734ae2d112b2 server_cookie=55bf809ac82bdc5c in_seq=0 out_seq=0 2024-09-10T23:40:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8b0c00640 10 client.8934 ms_handle_connect on v2:172.21.15.181:6834/2758853868 2024-09-10T23:40:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8b0c00640 1 -- 172.21.15.94:0/3993403156 <== mds.0 v2:172.21.15.181:6834/2758853868 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5602424f2520 con 0x5602426f6b20 2024-09-10T23:40:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8b0c00640 10 client.8934 handle_command_reply: tid=0 2024-09-10T23:40:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 1 client.8934 shutdown 2024-09-10T23:40:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 1 -- 172.21.15.94:0/3993403156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560242696e10 msgr2=0x560242758c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 1 --2- 172.21.15.94:0/3993403156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560242696e10 0x560242758c00 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800dbc0 tx=0x7ff8b8004570 comp rx=0 tx=0).stop 2024-09-10T23:40:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 1 -- 172.21.15.94:0/3993403156 shutdown_connections 2024-09-10T23:40:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 1 --2- 172.21.15.94:0/3993403156 >> [v2:172.21.15.181:6834/2758853868,v1:172.21.15.181:6835/2758853868] conn(0x5602426f6b20 0x5602426f6f00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 1 --2- 172.21.15.94:0/3993403156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5602426e29d0 0x560242759140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 1 --2- 172.21.15.94:0/3993403156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5602426e32f0 0x560242759680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 1 --2- 172.21.15.94:0/3993403156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560242696e10 0x560242758c00 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8b0c00640 0 client.8934 ms_handle_reset on v2:172.21.15.181:6834/2758853868 2024-09-10T23:40:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 1 -- 172.21.15.94:0/3993403156 >> 172.21.15.94:0/3993403156 conn(0x560242580130 msgr2=0x56024269eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 1 -- 172.21.15.94:0/3993403156 shutdown_connections 2024-09-10T23:40:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 1 -- 172.21.15.94:0/3993403156 wait complete. 2024-09-10T23:40:13.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8d141b000 20 client.8934 trim_cache size 0 max 16384 2024-09-10T23:40:13.857 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:40:13.857 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "1798cdb9-9937-4918-a569-86b41a92817e", 2024-09-10T23:40:13.857 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-09-10T23:40:13.858 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 197120, 2024-09-10T23:40:13.858 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-10T23:40:13.858 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-10T23:40:13.858 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-10T23:40:13.858 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 10.496214568999999, 2024-09-10T23:40:13.858 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 107, 2024-09-10T23:40:13.858 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 151, 2024-09-10T23:40:13.858 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 151, 2024-09-10T23:40:13.858 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 15.007038407 2024-09-10T23:40:13.858 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:40:13.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8ce000640 1 -- 172.21.15.94:0/1278075080 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff8a8072e50 msgr2=0x7ff8a8075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:13.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8ce000640 1 --2- 172.21.15.94:0/1278075080 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff8a8072e50 0x7ff8a8075310 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8165cd0 tx=0x7ff8b4009040 comp rx=0 tx=0).stop 2024-09-10T23:40:13.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8ce000640 1 -- 172.21.15.94:0/1278075080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c813cc00 msgr2=0x7ff8c812f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:13.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8ce000640 1 --2- 172.21.15.94:0/1278075080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c813cc00 0x7ff8c812f1d0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800dbf0 tx=0x7ff8b800a620 comp rx=0 tx=0).stop 2024-09-10T23:40:13.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8ce000640 1 -- 172.21.15.94:0/1278075080 shutdown_connections 2024-09-10T23:40:13.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8ce000640 1 --2- 172.21.15.94:0/1278075080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8c813cfe0 0x7ff8c81ecb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8ce000640 1 --2- 172.21.15.94:0/1278075080 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff8a8072e50 0x7ff8a8075310 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8ce000640 1 --2- 172.21.15.94:0/1278075080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c813d530 0x7ff8c80b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8ce000640 1 --2- 172.21.15.94:0/1278075080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8c813cc00 0x7ff8c812f1d0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:13.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8ce000640 1 -- 172.21.15.94:0/1278075080 >> 172.21.15.94:0/1278075080 conn(0x7ff8c80cfaf0 msgr2=0x7ff8c80cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:13.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8ce000640 1 -- 172.21.15.94:0/1278075080 shutdown_connections 2024-09-10T23:40:13.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:13.847+0000 7ff8ce000640 1 -- 172.21.15.94:0/1278075080 wait complete. 2024-09-10T23:40:13.870 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:40:14.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.007+0000 7f8a5e600640 1 -- 172.21.15.94:0/112588356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a5806fb20 msgr2=0x7f8a580b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:14.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.007+0000 7f8a5e600640 1 --2- 172.21.15.94:0/112588356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a5806fb20 0x7f8a580b3f50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009870 tx=0x7f8a580b53b0 comp rx=0 tx=0).stop 2024-09-10T23:40:14.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.007+0000 7f8a5e600640 1 -- 172.21.15.94:0/112588356 shutdown_connections 2024-09-10T23:40:14.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.007+0000 7f8a5e600640 1 --2- 172.21.15.94:0/112588356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a5806fb20 0x7f8a580b3f50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:14.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.007+0000 7f8a5e600640 1 -- 172.21.15.94:0/112588356 >> 172.21.15.94:0/112588356 conn(0x7f8a580d0b70 msgr2=0x7f8a580d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:14.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.007+0000 7f8a5e600640 1 -- 172.21.15.94:0/112588356 shutdown_connections 2024-09-10T23:40:14.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.007+0000 7f8a5e600640 1 -- 172.21.15.94:0/112588356 wait complete. 2024-09-10T23:40:14.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a5e600640 1 Processor -- start 2024-09-10T23:40:14.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a5e600640 1 -- start start 2024-09-10T23:40:14.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a5e600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a5806fb20 0x7f8a58151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:14.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a5e600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a581586e0 0x7f8a58158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:14.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a5e600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a5815b010 0x7f8a580b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:14.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a5e600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58169100 con 0x7f8a581586e0 2024-09-10T23:40:14.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a5e600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58168f80 con 0x7f8a5806fb20 2024-09-10T23:40:14.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a5e600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8a58169280 con 0x7f8a5815b010 2024-09-10T23:40:14.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a4ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a581586e0 0x7f8a58158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:14.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a4ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a581586e0 0x7f8a58158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36110/0 (socket says 172.21.15.94:36110) 2024-09-10T23:40:14.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a4ea00640 1 -- 172.21.15.94:0/3124021526 learned_addr learned my addr 172.21.15.94:0/3124021526 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:14.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a57400640 1 --2- 172.21.15.94:0/3124021526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a5806fb20 0x7f8a58151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:14.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a57e00640 1 --2- 172.21.15.94:0/3124021526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a5815b010 0x7f8a580b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:14.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a4ea00640 1 -- 172.21.15.94:0/3124021526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a5815b010 msgr2=0x7f8a580b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:14.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a4ea00640 1 --2- 172.21.15.94:0/3124021526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a5815b010 0x7f8a580b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:14.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a4ea00640 1 -- 172.21.15.94:0/3124021526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a5806fb20 msgr2=0x7f8a58151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:14.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a4ea00640 1 --2- 172.21.15.94:0/3124021526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a5806fb20 0x7f8a58151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:14.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a4ea00640 1 -- 172.21.15.94:0/3124021526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a58151d00 con 0x7f8a581586e0 2024-09-10T23:40:14.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a57400640 1 --2- 172.21.15.94:0/3124021526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a5806fb20 0x7f8a58151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:14.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a57e00640 1 --2- 172.21.15.94:0/3124021526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a5815b010 0x7f8a580b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:14.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a4ea00640 1 --2- 172.21.15.94:0/3124021526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a581586e0 0x7f8a58158bc0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f8a40098160 tx=0x7f8a400056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:14.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a4fe00640 1 -- 172.21.15.94:0/3124021526 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a400a86d0 con 0x7f8a581586e0 2024-09-10T23:40:14.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a5e600640 1 -- 172.21.15.94:0/3124021526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a581f8900 con 0x7f8a581586e0 2024-09-10T23:40:14.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a4fe00640 1 -- 172.21.15.94:0/3124021526 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a400a8870 con 0x7f8a581586e0 2024-09-10T23:40:14.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a4fe00640 1 -- 172.21.15.94:0/3124021526 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a400a8c80 con 0x7f8a581586e0 2024-09-10T23:40:14.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a5e600640 1 -- 172.21.15.94:0/3124021526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a58163b70 con 0x7f8a581586e0 2024-09-10T23:40:14.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.011+0000 7f8a5e600640 1 -- 172.21.15.94:0/3124021526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a5806f410 con 0x7f8a581586e0 2024-09-10T23:40:14.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.015+0000 7f8a4fe00640 1 -- 172.21.15.94:0/3124021526 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f8a400046d0 con 0x7f8a581586e0 2024-09-10T23:40:14.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.015+0000 7f8a4fe00640 1 --2- 172.21.15.94:0/3124021526 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8a34072e50 0x7f8a34075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:14.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.015+0000 7f8a4fe00640 1 -- 172.21.15.94:0/3124021526 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f8a4012e620 con 0x7f8a581586e0 2024-09-10T23:40:14.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.015+0000 7f8a57400640 1 --2- 172.21.15.94:0/3124021526 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8a34072e50 0x7f8a34075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:14.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.015+0000 7f8a57400640 1 --2- 172.21.15.94:0/3124021526 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8a34072e50 0x7f8a34075310 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8a48000fe0 tx=0x7f8a48009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:14.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.019+0000 7f8a4fe00640 1 -- 172.21.15.94:0/3124021526 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8a400ac030 con 0x7f8a581586e0 2024-09-10T23:40:14.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.219+0000 7f8a5e600640 1 -- 172.21.15.94:0/3124021526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8a5815c7a0 con 0x7f8a581586e0 2024-09-10T23:40:14.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.219+0000 7f8a4fe00640 1 -- 172.21.15.94:0/3124021526 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v107) ==== 53+0+73 (secure 0 0 0) 0x7f8a40016370 con 0x7f8a581586e0 2024-09-10T23:40:14.224 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:40:14.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.223+0000 7f8a5e600640 1 -- 172.21.15.94:0/3124021526 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8a34072e50 msgr2=0x7f8a34075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:14.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.223+0000 7f8a5e600640 1 --2- 172.21.15.94:0/3124021526 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8a34072e50 0x7f8a34075310 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8a48000fe0 tx=0x7f8a48009040 comp rx=0 tx=0).stop 2024-09-10T23:40:14.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.223+0000 7f8a5e600640 1 -- 172.21.15.94:0/3124021526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a581586e0 msgr2=0x7f8a58158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:14.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.223+0000 7f8a5e600640 1 --2- 172.21.15.94:0/3124021526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a581586e0 0x7f8a58158bc0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f8a40098160 tx=0x7f8a400056d0 comp rx=0 tx=0).stop 2024-09-10T23:40:14.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.223+0000 7f8a5e600640 1 -- 172.21.15.94:0/3124021526 shutdown_connections 2024-09-10T23:40:14.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.223+0000 7f8a5e600640 1 --2- 172.21.15.94:0/3124021526 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8a34072e50 0x7f8a34075310 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:14.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.223+0000 7f8a5e600640 1 --2- 172.21.15.94:0/3124021526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a5815b010 0x7f8a580b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:14.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.223+0000 7f8a5e600640 1 --2- 172.21.15.94:0/3124021526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a581586e0 0x7f8a58158bc0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:14.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.223+0000 7f8a5e600640 1 --2- 172.21.15.94:0/3124021526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a5806fb20 0x7f8a58151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:14.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.223+0000 7f8a5e600640 1 -- 172.21.15.94:0/3124021526 >> 172.21.15.94:0/3124021526 conn(0x7f8a580d0b70 msgr2=0x7f8a581544a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:14.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.223+0000 7f8a5e600640 1 -- 172.21.15.94:0/3124021526 shutdown_connections 2024-09-10T23:40:14.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14.223+0000 7f8a5e600640 1 -- 172.21.15.94:0/3124021526 wait complete. 2024-09-10T23:40:14.245 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:40:14.245 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:40:14.245 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T23:40:14.245 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T23:40:14.245 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T23:40:14.245 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:40:14.245 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:40:14.245 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:40:14.245 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:40:14.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:14.246 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T23:40:14.254 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:40:14.254 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:40:14.254 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:40:14.254 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:14.254 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T23:40:14.254 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:14.255 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:40:14.255 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:14.255 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T23:40:14.255 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:40:14.255 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:14.255 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T23:40:14.255 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:40:14.255 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:14.255 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:40:14.255 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:14.255 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:40:14.255 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 41997sec preferred_lft 41997sec 2024-09-10T23:40:14.255 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-09-10T23:40:14.255 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:14.256 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:40:14.256 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:ed:7d:46:db brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:14.256 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:40:14.256 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:14.256 INFO:teuthology.orchestra.run.smithi094.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:40:14.256 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:14.256 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:40:14.256 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:14.256 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:40:14.256 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:14.256 INFO:teuthology.orchestra.run.smithi094.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:40:14.256 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:40:14.256 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:40:14.256 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:14.257 INFO:teuthology.orchestra.run.smithi094.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:40:14.257 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:40:14.257 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:40:14.257 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:14.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:14.258 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:40:14.318 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T23:40:14.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:14.320 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T23:40:14.376 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T23:40:14.377 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:40:14.377 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:40:14.377 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:40:14.377 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:40:14.377 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:40:14.377 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:40:14.377 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:40:14.377 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:40:14.377 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:40:14.377 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:40:14.378 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:40:14.378 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:40:14.378 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:40:14.378 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:40:14.378 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:40:14.378 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:40:14.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:40:14.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:40:14.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:14.517 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T23:40:14.526 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T23:40:14.526 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T23:40:14.527 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:40:14.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:14.527 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-09-10T23:40:14.592 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T23:40:14.592 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:40:14.592 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T23:40:14.644 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:40:14.644 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:40:14.645 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:40:14.696 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:40:14.756 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:40:14.814 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:40:14.814 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:40:14.814 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:40:14.814 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:40:14.814 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:40:14.814 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:40:14.814 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:40:14.814 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:40:14.814 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:40:14.814 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:40:14.815 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:40:14.815 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:40:14.815 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:40:14.815 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:40:14.815 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:40:14.815 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:40:14.815 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:40:14.815 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:40:14.815 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:40:14.815 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:40:14.815 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:40:14.815 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:40:14.815 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:40:14.816 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:40:14.816 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:40:14.816 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:40:14.816 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:40:14.816 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:40:14.816 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:40:14.816 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:40:14.816 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:40:14.816 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:40:14.816 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:40:14.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:14.817 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:40:14.868 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:40:14.869 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T23:40:14.919 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:40:14.939 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:40:14.990 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:40:14.991 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:40:14.991 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:40:14.991 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:40:14.991 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:40:14.991 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:40:14.991 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:40:14.991 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:40:14.991 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:40:14.991 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:40:14.992 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:40:14.992 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:40:14.992 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:40:14.992 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:40:14.992 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:40:14.992 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:40:14.992 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:40:14.992 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:40:14.992 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:40:14.992 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:40:14.992 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:40:14.992 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:40:14.992 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:40:14.993 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:40:14.993 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:40:14.993 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:40:14.993 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:40:14.993 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:40:14.993 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:40:14.993 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:40:14.993 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:40:14.993 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:40:14.994 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:40:14.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:14.994 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:40:15.015 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:2024-09-10T23:40:15.011+0000 7f64afb9a580 -1 init, newargv = 0x561011753f30 newargc=15 2024-09-10T23:40:15.016 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[52141]: starting ceph client 2024-09-10T23:40:15.024 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[52141]: starting fuse 2024-09-10T23:40:16.125 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:40:16.148 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:40:16.199 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:40:16.199 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:40:16.199 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:40:16.199 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:40:16.199 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:40:16.199 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:40:16.199 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:40:16.199 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:40:16.199 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:40:16.199 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:40:16.199 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:40:16.199 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:40:16.200 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:40:16.200 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:40:16.200 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:40:16.200 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:40:16.200 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:40:16.200 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:40:16.200 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:40:16.200 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:40:16.200 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:40:16.200 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:40:16.200 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:40:16.200 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:40:16.200 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:40:16.200 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:40:16.201 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:40:16.201 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:40:16.201 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:40:16.201 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:40:16.201 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:40:16.201 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:40:16.201 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:40:16.201 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-10T23:40:16.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:16.202 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:40:16.253 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T23:40:16.253 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:40:16.253 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:40:16.253 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-09-10T23:40:16.253 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:40:16.254 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:40:16.255 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-09-10T23:40:16.255 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-09-10T23:40:16.255 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-09-10T23:40:16.255 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:40:16.255 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:40:16.255 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.0")) 2024-09-10T23:40:16.255 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-09-10T23:40:16.390 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.0.52141.asok 2024-09-10T23:40:16.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:40:16.404 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52141.asok 2024-09-10T23:40:16.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:16.404 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52141.asok status 2024-09-10T23:40:16.581 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:40:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-09-10T23:40:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:40:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:40:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "0", 2024-09-10T23:40:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-09-10T23:40:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T23:40:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "52141", 2024-09-10T23:40:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-09-10T23:40:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-09-10T23:40:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-09-10T23:40:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 8946, 2024-09-10T23:40:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-09-10T23:40:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-09-10T23:40:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-09-10T23:40:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 8946 2024-09-10T23:40:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:40:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:40:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-09-10T23:40:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 740863163 2024-09-10T23:40:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:40:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:40:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-09-10T23:40:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 740863163 2024-09-10T23:40:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.8946 172.21.15.94:0/740863163", 2024-09-10T23:40:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "172.21.15.94:0/740863163", 2024-09-10T23:40:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-09-10T23:40:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 107, 2024-09-10T23:40:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 151, 2024-09-10T23:40:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:40:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-09-10T23:40:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-09-10T23:40:16.584 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:40:16.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:16.596 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-09-10T23:40:16.597 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T23:40:16.609 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-09-10T23:40:16.609 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T23:40:16.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:16.609 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:40:16.676 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:40:16.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f9834e00640 1 -- 172.21.15.94:0/2819706497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300af310 msgr2=0x7f98300af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:16.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f9834e00640 1 --2- 172.21.15.94:0/2819706497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300af310 0x7f98300af7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9828009bc0 tx=0x7f982801ca80 comp rx=0 tx=0).stop 2024-09-10T23:40:16.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f9834e00640 1 -- 172.21.15.94:0/2819706497 shutdown_connections 2024-09-10T23:40:16.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f9834e00640 1 --2- 172.21.15.94:0/2819706497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300af310 0x7f98300af7a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:16.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f9834e00640 1 --2- 172.21.15.94:0/2819706497 >> v2:172.21.15.181:3300/0 conn(0x7f983015f5c0 0x7f98300cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:16.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f9834e00640 1 -- 172.21.15.94:0/2819706497 >> 172.21.15.94:0/2819706497 conn(0x7f98300d1c60 msgr2=0x7f98300d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:16.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f9834e00640 1 -- 172.21.15.94:0/2819706497 shutdown_connections 2024-09-10T23:40:16.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f9834e00640 1 -- 172.21.15.94:0/2819706497 wait complete. 2024-09-10T23:40:16.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f9834e00640 1 Processor -- start 2024-09-10T23:40:16.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f9834e00640 1 -- start start 2024-09-10T23:40:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f9834e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98300af310 0x7f983006f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f9834e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300b2bb0 0x7f9830147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f9834e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f983015f5c0 0x7f9830146c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f9834e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9830169050 con 0x7f983015f5c0 2024-09-10T23:40:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f9834e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9830168ed0 con 0x7f98300af310 2024-09-10T23:40:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f9834e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f98301691d0 con 0x7f98300b2bb0 2024-09-10T23:40:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f982e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f983015f5c0 0x7f9830146c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f982e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f983015f5c0 0x7f9830146c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36124/0 (socket says 172.21.15.94:36124) 2024-09-10T23:40:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f982de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98300af310 0x7f983006f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f982e800640 1 -- 172.21.15.94:0/1251207691 learned_addr learned my addr 172.21.15.94:0/1251207691 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.855+0000 7f982d400640 1 --2- 172.21.15.94:0/1251207691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300b2bb0 0x7f9830147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.859+0000 7f982e800640 1 -- 172.21.15.94:0/1251207691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300b2bb0 msgr2=0x7f9830147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.859+0000 7f982e800640 1 --2- 172.21.15.94:0/1251207691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300b2bb0 0x7f9830147320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.859+0000 7f982e800640 1 -- 172.21.15.94:0/1251207691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98300af310 msgr2=0x7f983006f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.859+0000 7f982e800640 1 --2- 172.21.15.94:0/1251207691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98300af310 0x7f983006f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.859+0000 7f982e800640 1 -- 172.21.15.94:0/1251207691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9830071240 con 0x7f983015f5c0 2024-09-10T23:40:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.859+0000 7f982e800640 1 --2- 172.21.15.94:0/1251207691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f983015f5c0 0x7f9830146c50 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f9828009b10 tx=0x7f9828003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.859+0000 7f9826000640 1 -- 172.21.15.94:0/1251207691 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98280ba020 con 0x7f983015f5c0 2024-09-10T23:40:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.859+0000 7f9826000640 1 -- 172.21.15.94:0/1251207691 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9828004d40 con 0x7f983015f5c0 2024-09-10T23:40:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.859+0000 7f9826000640 1 -- 172.21.15.94:0/1251207691 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98280b2040 con 0x7f983015f5c0 2024-09-10T23:40:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.859+0000 7f9834e00640 1 -- 172.21.15.94:0/1251207691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98301f8c70 con 0x7f983015f5c0 2024-09-10T23:40:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.859+0000 7f9834e00640 1 -- 172.21.15.94:0/1251207691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9830150230 con 0x7f983015f5c0 2024-09-10T23:40:16.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.859+0000 7f9834e00640 1 -- 172.21.15.94:0/1251207691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9830150060 con 0x7f983015f5c0 2024-09-10T23:40:16.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.859+0000 7f9826000640 1 -- 172.21.15.94:0/1251207691 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f98280b5070 con 0x7f983015f5c0 2024-09-10T23:40:16.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.859+0000 7f9826000640 1 --2- 172.21.15.94:0/1251207691 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9810072e80 0x7f9810075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:16.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.863+0000 7f9826000640 1 -- 172.21.15.94:0/1251207691 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f982812e2c0 con 0x7f983015f5c0 2024-09-10T23:40:16.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.863+0000 7f982de00640 1 --2- 172.21.15.94:0/1251207691 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9810072e80 0x7f9810075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:16.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.863+0000 7f982de00640 1 --2- 172.21.15.94:0/1251207691 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9810072e80 0x7f9810075340 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9830154720 tx=0x7f98200091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:16.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:16.863+0000 7f9826000640 1 -- 172.21.15.94:0/1251207691 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f98280c4020 con 0x7f983015f5c0 2024-09-10T23:40:17.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:17.063+0000 7f9834e00640 1 -- 172.21.15.94:0/1251207691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f98300af7a0 con 0x7f983015f5c0 2024-09-10T23:40:17.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:17.063+0000 7f9826000640 1 -- 172.21.15.94:0/1251207691 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v107) ==== 53+0+73 (secure 0 0 0) 0x7f9828016790 con 0x7f983015f5c0 2024-09-10T23:40:17.069 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:40:17.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:17.067+0000 7f9834e00640 1 -- 172.21.15.94:0/1251207691 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9810072e80 msgr2=0x7f9810075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:17.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:17.067+0000 7f9834e00640 1 --2- 172.21.15.94:0/1251207691 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9810072e80 0x7f9810075340 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9830154720 tx=0x7f98200091e0 comp rx=0 tx=0).stop 2024-09-10T23:40:17.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:17.067+0000 7f9834e00640 1 -- 172.21.15.94:0/1251207691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f983015f5c0 msgr2=0x7f9830146c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:17.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:17.067+0000 7f9834e00640 1 --2- 172.21.15.94:0/1251207691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f983015f5c0 0x7f9830146c50 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f9828009b10 tx=0x7f9828003f60 comp rx=0 tx=0).stop 2024-09-10T23:40:17.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:17.067+0000 7f9834e00640 1 -- 172.21.15.94:0/1251207691 shutdown_connections 2024-09-10T23:40:17.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:17.067+0000 7f9834e00640 1 --2- 172.21.15.94:0/1251207691 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9810072e80 0x7f9810075340 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:17.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:17.067+0000 7f9834e00640 1 --2- 172.21.15.94:0/1251207691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f983015f5c0 0x7f9830146c50 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:17.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:17.067+0000 7f9834e00640 1 --2- 172.21.15.94:0/1251207691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98300b2bb0 0x7f9830147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:17.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:17.067+0000 7f9834e00640 1 --2- 172.21.15.94:0/1251207691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98300af310 0x7f983006f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:17.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:17.067+0000 7f9834e00640 1 -- 172.21.15.94:0/1251207691 >> 172.21.15.94:0/1251207691 conn(0x7f98300d1c60 msgr2=0x7f9830153d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:17.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:17.067+0000 7f9834e00640 1 -- 172.21.15.94:0/1251207691 shutdown_connections 2024-09-10T23:40:17.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:17.067+0000 7f9834e00640 1 -- 172.21.15.94:0/1251207691 wait complete. 2024-09-10T23:40:17.090 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:40:17.090 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:40:17.090 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-09-10T23:40:17.090 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T23:40:17.091 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T23:40:17.091 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:40:17.091 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:40:17.091 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:40:17.091 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:40:17.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:17.091 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-09-10T23:40:17.101 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:40:17.101 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:40:17.101 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:40:17.102 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:17.102 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-09-10T23:40:17.102 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:17.102 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:40:17.102 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:17.102 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-09-10T23:40:17.102 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:40:17.102 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:17.102 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-09-10T23:40:17.102 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:40:17.102 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:17.102 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:40:17.102 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:17.102 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:40:17.103 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 41998sec preferred_lft 41998sec 2024-09-10T23:40:17.103 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-09-10T23:40:17.103 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:17.103 INFO:teuthology.orchestra.run.smithi181.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:40:17.103 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 02:42:3b:2e:09:2d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:17.103 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:40:17.103 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:17.103 INFO:teuthology.orchestra.run.smithi181.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:40:17.103 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:17.103 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:40:17.103 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:17.103 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:40:17.103 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:17.103 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:40:17.104 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:40:17.104 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:40:17.104 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:17.104 INFO:teuthology.orchestra.run.smithi181.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:40:17.104 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:40:17.104 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:40:17.104 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:17.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:17.105 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:40:17.167 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-09-10T23:40:17.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:17.170 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-09-10T23:40:17.225 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-09-10T23:40:17.225 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:40:17.225 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:40:17.225 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:40:17.225 INFO:teuthology.orchestra.run.smithi181.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:40:17.226 INFO:teuthology.orchestra.run.smithi181.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:40:17.226 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:40:17.226 INFO:teuthology.orchestra.run.smithi181.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:40:17.226 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:40:17.226 INFO:teuthology.orchestra.run.smithi181.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:40:17.226 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:40:17.226 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:40:17.227 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:40:17.227 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:40:17.227 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:40:17.227 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:40:17.227 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:40:17.320 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:40:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:40:17.364 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:40:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:40:17.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:17.371 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-09-10T23:40:17.381 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T23:40:17.381 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T23:40:17.382 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:40:17.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:17.382 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-09-10T23:40:17.447 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T23:40:17.448 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:40:17.448 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T23:40:17.496 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:40:17.496 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:40:17.496 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:40:17.552 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:40:17.612 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:40:17.669 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:40:17.669 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:40:17.670 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:40:17.670 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:40:17.670 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:40:17.670 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:40:17.670 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:40:17.670 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:40:17.670 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:40:17.670 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:40:17.670 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:40:17.670 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:40:17.670 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:40:17.670 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:40:17.670 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:40:17.670 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:40:17.671 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:40:17.671 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:40:17.671 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:40:17.671 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:40:17.671 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:40:17.671 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:40:17.671 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:40:17.671 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:40:17.671 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:40:17.671 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:40:17.671 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:40:17.671 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:40:17.671 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:40:17.671 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:40:17.671 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:40:17.672 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:40:17.672 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:40:17.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:17.673 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:40:17.723 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:40:17.723 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T23:40:17.766 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:40:17.787 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:40:17.837 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:40:17.837 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:40:17.837 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:40:17.838 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:40:17.838 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:40:17.838 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:40:17.838 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:40:17.838 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:40:17.838 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:40:17.838 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:40:17.838 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:40:17.838 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:40:17.838 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:40:17.838 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:40:17.838 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:40:17.838 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:40:17.839 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:40:17.839 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:40:17.839 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:40:17.839 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:40:17.839 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:40:17.839 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:40:17.839 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:40:17.839 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:40:17.839 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:40:17.839 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:40:17.839 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:40:17.840 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:40:17.840 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:40:17.840 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:40:17.840 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:40:17.840 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:40:17.840 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:40:17.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:17.841 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:40:17.861 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:2024-09-10T23:40:17.859+0000 7f3d29a4b580 -1 init, newargv = 0x5647f527cdc0 newargc=15 2024-09-10T23:40:17.861 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[36226]: starting ceph client 2024-09-10T23:40:17.869 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[36226]: starting fuse 2024-09-10T23:40:17.869 INFO:teuthology.orchestra.run.smithi181.stdout:44 2024-09-10T23:40:17.870 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:40:17.870 DEBUG:teuthology.orchestra.run.smithi181:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:40:17.870 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2024-09-10T23:40:17.870 DEBUG:teuthology.orchestra.run.smithi181:> import re 2024-09-10T23:40:17.870 DEBUG:teuthology.orchestra.run.smithi181:> import os 2024-09-10T23:40:17.870 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2024-09-10T23:40:17.870 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:40:17.870 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2024-09-10T23:40:17.870 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T23:40:17.870 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2024-09-10T23:40:17.870 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T23:40:17.870 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:40:17.871 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2024-09-10T23:40:17.871 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2024-09-10T23:40:17.871 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2024-09-10T23:40:17.871 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2024-09-10T23:40:17.871 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:40:17.871 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2024-09-10T23:40:17.871 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:40:17.871 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:40:17.871 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:40:17.871 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2024-09-10T23:40:17.871 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2024-09-10T23:40:17.871 DEBUG:teuthology.orchestra.run.smithi181:> return f 2024-09-10T23:40:17.871 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:40:17.871 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:40:17.871 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.1")) 2024-09-10T23:40:17.872 DEBUG:teuthology.orchestra.run.smithi181:> ' 2024-09-10T23:40:17.991 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.1.36226.asok 2024-09-10T23:40:17.995 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:40:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:40:18.002 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36226.asok 2024-09-10T23:40:18.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:18.003 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36226.asok status 2024-09-10T23:40:18.174 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T23:40:18.174 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2024-09-10T23:40:18.174 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:40:18.174 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:40:18.175 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "1", 2024-09-10T23:40:18.175 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2024-09-10T23:40:18.175 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T23:40:18.175 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "36226", 2024-09-10T23:40:18.175 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2024-09-10T23:40:18.175 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:40:18.175 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2024-09-10T23:40:18.176 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2024-09-10T23:40:18.176 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 8326, 2024-09-10T23:40:18.176 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2024-09-10T23:40:18.176 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2024-09-10T23:40:18.176 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2024-09-10T23:40:18.176 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 8326 2024-09-10T23:40:18.176 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:40:18.176 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:40:18.177 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:40:18.177 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:40:18.177 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 3318168378 2024-09-10T23:40:18.177 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T23:40:18.177 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:40:18.177 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:40:18.177 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:40:18.177 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:40:18.177 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 3318168378 2024-09-10T23:40:18.177 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:40:18.178 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.8326 192.168.0.1:0/3318168378", 2024-09-10T23:40:18.178 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.1:0/3318168378", 2024-09-10T23:40:18.178 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2024-09-10T23:40:18.178 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 107, 2024-09-10T23:40:18.178 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 151, 2024-09-10T23:40:18.178 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:40:18.178 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2024-09-10T23:40:18.178 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2024-09-10T23:40:18.178 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T23:40:18.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:18.189 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2024-09-10T23:40:18.189 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T23:40:18.198 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2024-09-10T23:40:18.198 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T23:40:18.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:18.198 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:40:18.263 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T23:40:18.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.407+0000 7f585f000640 1 -- 172.21.15.94:0/2808859130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58580d04b0 msgr2=0x7f585815e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:18.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.407+0000 7f585f000640 1 --2- 172.21.15.94:0/2808859130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58580d04b0 0x7f585815e770 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f5844009900 tx=0x7f584401c8b0 comp rx=0 tx=0).stop 2024-09-10T23:40:18.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.407+0000 7f585f000640 1 -- 172.21.15.94:0/2808859130 shutdown_connections 2024-09-10T23:40:18.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.407+0000 7f585f000640 1 --2- 172.21.15.94:0/2808859130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58580d04b0 0x7f585815e770 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:18.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.407+0000 7f585f000640 1 -- 172.21.15.94:0/2808859130 >> 172.21.15.94:0/2808859130 conn(0x7f585806cdf0 msgr2=0x7f585806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:18.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.407+0000 7f585f000640 1 -- 172.21.15.94:0/2808859130 shutdown_connections 2024-09-10T23:40:18.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.407+0000 7f585f000640 1 -- 172.21.15.94:0/2808859130 wait complete. 2024-09-10T23:40:18.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.407+0000 7f585f000640 1 Processor -- start 2024-09-10T23:40:18.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.407+0000 7f585f000640 1 -- start start 2024-09-10T23:40:18.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f585f000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5858073ff0 0x7f585814bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:18.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f585f000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58580d04b0 0x7f58580b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:18.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f585f000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f585815ecb0 0x7f5858151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:18.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f585f000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5858168d70 con 0x7f5858073ff0 2024-09-10T23:40:18.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f585f000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5858168bf0 con 0x7f585815ecb0 2024-09-10T23:40:18.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f585f000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5858168ef0 con 0x7f58580d04b0 2024-09-10T23:40:18.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f5857e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5858073ff0 0x7f585814bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:18.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f5857e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5858073ff0 0x7f585814bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36142/0 (socket says 172.21.15.94:36142) 2024-09-10T23:40:18.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f5857e00640 1 -- 172.21.15.94:0/3987548267 learned_addr learned my addr 172.21.15.94:0/3987548267 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:18.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f5857400640 1 --2- 172.21.15.94:0/3987548267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58580d04b0 0x7f58580b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:18.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f585d000640 1 --2- 172.21.15.94:0/3987548267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f585815ecb0 0x7f5858151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:18.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f5857e00640 1 -- 172.21.15.94:0/3987548267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58580d04b0 msgr2=0x7f58580b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:18.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f5857e00640 1 --2- 172.21.15.94:0/3987548267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58580d04b0 0x7f58580b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:18.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f5857e00640 1 -- 172.21.15.94:0/3987548267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f585815ecb0 msgr2=0x7f5858151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:18.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f5857e00640 1 --2- 172.21.15.94:0/3987548267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f585815ecb0 0x7f5858151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:18.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f5857e00640 1 -- 172.21.15.94:0/3987548267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58580b1130 con 0x7f5858073ff0 2024-09-10T23:40:18.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f5857e00640 1 --2- 172.21.15.94:0/3987548267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5858073ff0 0x7f585814bb10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f584800c450 tx=0x7f584800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:18.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f584fe00640 1 -- 172.21.15.94:0/3987548267 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848012830 con 0x7f5858073ff0 2024-09-10T23:40:18.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f584fe00640 1 -- 172.21.15.94:0/3987548267 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5848017070 con 0x7f5858073ff0 2024-09-10T23:40:18.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f584fe00640 1 -- 172.21.15.94:0/3987548267 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848012d80 con 0x7f5858073ff0 2024-09-10T23:40:18.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f585f000640 1 -- 172.21.15.94:0/3987548267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58580b1410 con 0x7f5858073ff0 2024-09-10T23:40:18.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f585f000640 1 -- 172.21.15.94:0/3987548267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58580b19d0 con 0x7f5858073ff0 2024-09-10T23:40:18.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.411+0000 7f585f000640 1 -- 172.21.15.94:0/3987548267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f585815e770 con 0x7f5858073ff0 2024-09-10T23:40:18.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.415+0000 7f584fe00640 1 -- 172.21.15.94:0/3987548267 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f584801c460 con 0x7f5858073ff0 2024-09-10T23:40:18.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.415+0000 7f584fe00640 1 --2- 172.21.15.94:0/3987548267 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5838072e80 0x7f5838075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:18.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.415+0000 7f584fe00640 1 -- 172.21.15.94:0/3987548267 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f5848094150 con 0x7f5858073ff0 2024-09-10T23:40:18.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.415+0000 7f5857400640 1 --2- 172.21.15.94:0/3987548267 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5838072e80 0x7f5838075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:18.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.415+0000 7f5857400640 1 --2- 172.21.15.94:0/3987548267 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5838072e80 0x7f5838075340 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f58440af000 tx=0x7f584401b0f0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:18.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.419+0000 7f584fe00640 1 -- 172.21.15.94:0/3987548267 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f584805e220 con 0x7f5858073ff0 2024-09-10T23:40:18.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.619+0000 7f585f000640 1 -- 172.21.15.94:0/3987548267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f585806d270 con 0x7f5858073ff0 2024-09-10T23:40:18.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.843+0000 7f584fe00640 1 -- 172.21.15.94:0/3987548267 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v108) ==== 136+0+0 (secure 0 0 0) 0x7f5848062c50 con 0x7f5858073ff0 2024-09-10T23:40:18.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.847+0000 7f585f000640 1 -- 172.21.15.94:0/3987548267 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5838072e80 msgr2=0x7f5838075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:18.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.847+0000 7f585f000640 1 --2- 172.21.15.94:0/3987548267 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5838072e80 0x7f5838075340 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f58440af000 tx=0x7f584401b0f0 comp rx=0 tx=0).stop 2024-09-10T23:40:18.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.847+0000 7f585f000640 1 -- 172.21.15.94:0/3987548267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5858073ff0 msgr2=0x7f585814bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:18.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.847+0000 7f585f000640 1 --2- 172.21.15.94:0/3987548267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5858073ff0 0x7f585814bb10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f584800c450 tx=0x7f584800c910 comp rx=0 tx=0).stop 2024-09-10T23:40:18.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.847+0000 7f585f000640 1 -- 172.21.15.94:0/3987548267 shutdown_connections 2024-09-10T23:40:18.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.847+0000 7f585f000640 1 --2- 172.21.15.94:0/3987548267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f585815ecb0 0x7f5858151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:18.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.847+0000 7f585f000640 1 --2- 172.21.15.94:0/3987548267 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5838072e80 0x7f5838075340 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:18.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.847+0000 7f585f000640 1 --2- 172.21.15.94:0/3987548267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58580d04b0 0x7f58580b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:18.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.847+0000 7f585f000640 1 --2- 172.21.15.94:0/3987548267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5858073ff0 0x7f585814bb10 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:18.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.847+0000 7f585f000640 1 -- 172.21.15.94:0/3987548267 >> 172.21.15.94:0/3987548267 conn(0x7f585806cdf0 msgr2=0x7f585815cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:18.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.847+0000 7f585f000640 1 -- 172.21.15.94:0/3987548267 shutdown_connections 2024-09-10T23:40:18.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:18.847+0000 7f585f000640 1 -- 172.21.15.94:0/3987548267 wait complete. 2024-09-10T23:40:18.871 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T23:40:19.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.011+0000 7eff4c400640 1 -- 172.21.15.94:0/1795565638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff440c64f0 msgr2=0x7eff440ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:19.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.011+0000 7eff4c400640 1 --2- 172.21.15.94:0/1795565638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff440c64f0 0x7eff440ce660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7eff3000a950 tx=0x7eff3001da90 comp rx=0 tx=0).stop 2024-09-10T23:40:19.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.011+0000 7eff4c400640 1 -- 172.21.15.94:0/1795565638 shutdown_connections 2024-09-10T23:40:19.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.011+0000 7eff4c400640 1 --2- 172.21.15.94:0/1795565638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff440c64f0 0x7eff440ce660 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.011+0000 7eff4c400640 1 --2- 172.21.15.94:0/1795565638 >> v2:172.21.15.181:3300/0 conn(0x7eff440ce280 0x7eff440cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.011+0000 7eff4c400640 1 -- 172.21.15.94:0/1795565638 >> 172.21.15.94:0/1795565638 conn(0x7eff440cfaf0 msgr2=0x7eff440cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:19.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.011+0000 7eff4c400640 1 -- 172.21.15.94:0/1795565638 shutdown_connections 2024-09-10T23:40:19.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.011+0000 7eff4c400640 1 -- 172.21.15.94:0/1795565638 wait complete. 2024-09-10T23:40:19.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.011+0000 7eff4c400640 1 Processor -- start 2024-09-10T23:40:19.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.011+0000 7eff4c400640 1 -- start start 2024-09-10T23:40:19.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4c400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff4406e7c0 0x7eff4414c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:19.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4c400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff440c64f0 0x7eff441f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:19.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4c400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff440ce280 0x7eff440adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:19.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4c400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff44167680 con 0x7eff440ce280 2024-09-10T23:40:19.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4c400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7eff44167500 con 0x7eff440c64f0 2024-09-10T23:40:19.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4c400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7eff44167800 con 0x7eff4406e7c0 2024-09-10T23:40:19.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4a400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff440ce280 0x7eff440adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:19.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4a400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff440ce280 0x7eff440adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36150/0 (socket says 172.21.15.94:36150) 2024-09-10T23:40:19.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4a400640 1 -- 172.21.15.94:0/3961776386 learned_addr learned my addr 172.21.15.94:0/3961776386 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:19.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff49000640 1 --2- 172.21.15.94:0/3961776386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff440c64f0 0x7eff441f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:19.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff49a00640 1 --2- 172.21.15.94:0/3961776386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff4406e7c0 0x7eff4414c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:19.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4a400640 1 -- 172.21.15.94:0/3961776386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff4406e7c0 msgr2=0x7eff4414c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:19.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4a400640 1 --2- 172.21.15.94:0/3961776386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff4406e7c0 0x7eff4414c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4a400640 1 -- 172.21.15.94:0/3961776386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff440c64f0 msgr2=0x7eff441f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:19.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4a400640 1 --2- 172.21.15.94:0/3961776386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff440c64f0 0x7eff441f8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4a400640 1 -- 172.21.15.94:0/3961776386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff441f8a80 con 0x7eff440ce280 2024-09-10T23:40:19.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff49000640 1 --2- 172.21.15.94:0/3961776386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff440c64f0 0x7eff441f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:19.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff49a00640 1 --2- 172.21.15.94:0/3961776386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff4406e7c0 0x7eff4414c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:19.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4a400640 1 --2- 172.21.15.94:0/3961776386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff440ce280 0x7eff440adfa0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00cc20 tx=0x7eff3c00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:19.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff41600640 1 -- 172.21.15.94:0/3961776386 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c016020 con 0x7eff440ce280 2024-09-10T23:40:19.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff41600640 1 -- 172.21.15.94:0/3961776386 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff3c00ab60 con 0x7eff440ce280 2024-09-10T23:40:19.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff41600640 1 -- 172.21.15.94:0/3961776386 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c012420 con 0x7eff440ce280 2024-09-10T23:40:19.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4c400640 1 -- 172.21.15.94:0/3961776386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff441f8d60 con 0x7eff440ce280 2024-09-10T23:40:19.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4c400640 1 -- 172.21.15.94:0/3961776386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff440ada80 con 0x7eff440ce280 2024-09-10T23:40:19.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff4c400640 1 -- 172.21.15.94:0/3961776386 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff441f9280 con 0x7eff440ce280 2024-09-10T23:40:19.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.015+0000 7eff41600640 1 -- 172.21.15.94:0/3961776386 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7eff3c01b400 con 0x7eff440ce280 2024-09-10T23:40:19.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.019+0000 7eff41600640 1 --2- 172.21.15.94:0/3961776386 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7eff24072e50 0x7eff24075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:19.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.019+0000 7eff41600640 1 -- 172.21.15.94:0/3961776386 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7eff3c094d00 con 0x7eff440ce280 2024-09-10T23:40:19.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.019+0000 7eff49a00640 1 --2- 172.21.15.94:0/3961776386 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7eff24072e50 0x7eff24075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:19.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.019+0000 7eff49a00640 1 --2- 172.21.15.94:0/3961776386 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7eff24072e50 0x7eff24075310 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7eff440af350 tx=0x7eff380091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:19.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.023+0000 7eff41600640 1 -- 172.21.15.94:0/3961776386 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7eff441f9280 con 0x7eff440ce280 2024-09-10T23:40:19.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.207+0000 7eff4c400640 1 -- 172.21.15.94:0/3961776386 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7eff440ce660 con 0x7eff24072e50 2024-09-10T23:40:19.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.207+0000 7eff41600640 1 -- 172.21.15.94:0/3961776386 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7eff440ce660 con 0x7eff24072e50 2024-09-10T23:40:19.212 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:40:19.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:19.212 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephfs" 2024-09-10T23:40:19.213 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:19.213 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:40:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.211+0000 7eff4c400640 1 -- 172.21.15.94:0/3961776386 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7eff24072e50 msgr2=0x7eff24075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.211+0000 7eff4c400640 1 --2- 172.21.15.94:0/3961776386 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7eff24072e50 0x7eff24075310 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7eff440af350 tx=0x7eff380091e0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.211+0000 7eff4c400640 1 -- 172.21.15.94:0/3961776386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff440ce280 msgr2=0x7eff440adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.211+0000 7eff4c400640 1 --2- 172.21.15.94:0/3961776386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff440ce280 0x7eff440adfa0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00cc20 tx=0x7eff3c00cd10 comp rx=0 tx=0).stop 2024-09-10T23:40:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.211+0000 7eff4c400640 1 -- 172.21.15.94:0/3961776386 shutdown_connections 2024-09-10T23:40:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.211+0000 7eff4c400640 1 --2- 172.21.15.94:0/3961776386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff440c64f0 0x7eff441f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.211+0000 7eff4c400640 1 --2- 172.21.15.94:0/3961776386 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7eff24072e50 0x7eff24075310 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.211+0000 7eff4c400640 1 --2- 172.21.15.94:0/3961776386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff440ce280 0x7eff440adfa0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.211+0000 7eff4c400640 1 --2- 172.21.15.94:0/3961776386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff4406e7c0 0x7eff4414c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.211+0000 7eff4c400640 1 -- 172.21.15.94:0/3961776386 >> 172.21.15.94:0/3961776386 conn(0x7eff440cfaf0 msgr2=0x7eff440afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:19.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.211+0000 7eff4c400640 1 -- 172.21.15.94:0/3961776386 shutdown_connections 2024-09-10T23:40:19.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.211+0000 7eff4c400640 1 -- 172.21.15.94:0/3961776386 wait complete. 2024-09-10T23:40:19.233 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:40:19.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.375+0000 7fc9dda00640 1 -- 172.21.15.94:0/17944949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d815f1e0 msgr2=0x7fc9d80cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:19.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.375+0000 7fc9dda00640 1 --2- 172.21.15.94:0/17944949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d815f1e0 0x7fc9d80cf8f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009840 tx=0x7fc9c801c790 comp rx=0 tx=0).stop 2024-09-10T23:40:19.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.375+0000 7fc9dda00640 1 -- 172.21.15.94:0/17944949 shutdown_connections 2024-09-10T23:40:19.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.375+0000 7fc9dda00640 1 --2- 172.21.15.94:0/17944949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d815f1e0 0x7fc9d80cf8f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.375+0000 7fc9dda00640 1 -- 172.21.15.94:0/17944949 >> 172.21.15.94:0/17944949 conn(0x7fc9d80d1b10 msgr2=0x7fc9d80d1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:19.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.375+0000 7fc9dda00640 1 -- 172.21.15.94:0/17944949 shutdown_connections 2024-09-10T23:40:19.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.375+0000 7fc9dda00640 1 -- 172.21.15.94:0/17944949 wait complete. 2024-09-10T23:40:19.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9dda00640 1 Processor -- start 2024-09-10T23:40:19.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9dda00640 1 -- start start 2024-09-10T23:40:19.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9dda00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80af680 0x7fc9d80cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:19.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9dda00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d80d2630 0x7fc9d8150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:19.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9dda00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d815f1e0 0x7fc9d81f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:19.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9dda00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d8169c00 con 0x7fc9d80d2630 2024-09-10T23:40:19.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9dda00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d8169a80 con 0x7fc9d815f1e0 2024-09-10T23:40:19.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9dda00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc9d8169d80 con 0x7fc9d80af680 2024-09-10T23:40:19.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9cf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d80d2630 0x7fc9d8150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:19.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9cf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d80d2630 0x7fc9d8150b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36166/0 (socket says 172.21.15.94:36166) 2024-09-10T23:40:19.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9cf400640 1 -- 172.21.15.94:0/317933359 learned_addr learned my addr 172.21.15.94:0/317933359 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:19.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9d7e00640 1 --2- 172.21.15.94:0/317933359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80af680 0x7fc9d80cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:19.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9dd000640 1 --2- 172.21.15.94:0/317933359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d815f1e0 0x7fc9d81f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:19.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9cf400640 1 -- 172.21.15.94:0/317933359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80af680 msgr2=0x7fc9d80cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:19.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9cf400640 1 --2- 172.21.15.94:0/317933359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80af680 0x7fc9d80cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9cf400640 1 -- 172.21.15.94:0/317933359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d815f1e0 msgr2=0x7fc9d81f77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:19.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9cf400640 1 --2- 172.21.15.94:0/317933359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d815f1e0 0x7fc9d81f77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9cf400640 1 -- 172.21.15.94:0/317933359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d80ccbe0 con 0x7fc9d80d2630 2024-09-10T23:40:19.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9cf400640 1 --2- 172.21.15.94:0/317933359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d80d2630 0x7fc9d8150b40 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000c480 tx=0x7fc9c000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:19.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9d4c00640 1 -- 172.21.15.94:0/317933359 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00052d0 con 0x7fc9d80d2630 2024-09-10T23:40:19.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9d4c00640 1 -- 172.21.15.94:0/317933359 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9c0005470 con 0x7fc9d80d2630 2024-09-10T23:40:19.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9d4c00640 1 -- 172.21.15.94:0/317933359 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0012870 con 0x7fc9d80d2630 2024-09-10T23:40:19.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9dda00640 1 -- 172.21.15.94:0/317933359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d80ccec0 con 0x7fc9d80d2630 2024-09-10T23:40:19.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.379+0000 7fc9dda00640 1 -- 172.21.15.94:0/317933359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d80cd480 con 0x7fc9d80d2630 2024-09-10T23:40:19.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.383+0000 7fc9dda00640 1 -- 172.21.15.94:0/317933359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d80afb30 con 0x7fc9d80d2630 2024-09-10T23:40:19.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.383+0000 7fc9d4c00640 1 -- 172.21.15.94:0/317933359 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fc9c0003600 con 0x7fc9d80d2630 2024-09-10T23:40:19.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.383+0000 7fc9d4c00640 1 --2- 172.21.15.94:0/317933359 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc9b4072e50 0x7fc9b4075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:19.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.383+0000 7fc9d4c00640 1 -- 172.21.15.94:0/317933359 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7fc9c0093f90 con 0x7fc9d80d2630 2024-09-10T23:40:19.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.383+0000 7fc9d7e00640 1 --2- 172.21.15.94:0/317933359 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc9b4072e50 0x7fc9b4075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:19.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.383+0000 7fc9d7e00640 1 --2- 172.21.15.94:0/317933359 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc9b4072e50 0x7fc9b4075310 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80af000 tx=0x7fc9c801afd0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:19.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.387+0000 7fc9d4c00640 1 -- 172.21.15.94:0/317933359 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc9c0097030 con 0x7fc9d80d2630 2024-09-10T23:40:19.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.551+0000 7fc9dda00640 1 -- 172.21.15.94:0/317933359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc9d806cff0 con 0x7fc9d80d2630 2024-09-10T23:40:19.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.555+0000 7fc9d4c00640 1 -- 172.21.15.94:0/317933359 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7fc9c005e060 con 0x7fc9d80d2630 2024-09-10T23:40:19.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.555+0000 7fc9dda00640 1 -- 172.21.15.94:0/317933359 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc9b4072e50 msgr2=0x7fc9b4075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:19.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.555+0000 7fc9dda00640 1 --2- 172.21.15.94:0/317933359 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc9b4072e50 0x7fc9b4075310 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80af000 tx=0x7fc9c801afd0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.555+0000 7fc9dda00640 1 -- 172.21.15.94:0/317933359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d80d2630 msgr2=0x7fc9d8150b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:19.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.555+0000 7fc9dda00640 1 --2- 172.21.15.94:0/317933359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d80d2630 0x7fc9d8150b40 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000c480 tx=0x7fc9c000c940 comp rx=0 tx=0).stop 2024-09-10T23:40:19.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.555+0000 7fc9dda00640 1 -- 172.21.15.94:0/317933359 shutdown_connections 2024-09-10T23:40:19.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.555+0000 7fc9dda00640 1 --2- 172.21.15.94:0/317933359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9d815f1e0 0x7fc9d81f77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.555+0000 7fc9dda00640 1 --2- 172.21.15.94:0/317933359 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc9b4072e50 0x7fc9b4075310 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.555+0000 7fc9dda00640 1 --2- 172.21.15.94:0/317933359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d80d2630 0x7fc9d8150b40 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.555+0000 7fc9dda00640 1 --2- 172.21.15.94:0/317933359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80af680 0x7fc9d80cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:19.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.555+0000 7fc9dda00640 1 -- 172.21.15.94:0/317933359 >> 172.21.15.94:0/317933359 conn(0x7fc9d80d1b10 msgr2=0x7fc9d815b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:19.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.555+0000 7fc9dda00640 1 -- 172.21.15.94:0/317933359 shutdown_connections 2024-09-10T23:40:19.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:19.555+0000 7fc9dda00640 1 -- 172.21.15.94:0/317933359 wait complete. 2024-09-10T23:40:19.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:19.579 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol95790/9b464252-103c-4bc8-b7c1-b8532b673f4b) 2024-09-10T23:40:19.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:19.613 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T23:40:19.669 INFO:teuthology.orchestra.run.smithi094.stdout:cephfs_data 2024-09-10T23:40:19.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:19.669 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol95790/9b464252-103c-4bc8-b7c1-b8532b673f4b) 2024-09-10T23:40:19.739 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:40:19.739 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol95790/.meta 2024-09-10T23:40:19.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:19.808 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp77380/subvol65011/df2d7e23-0c1e-4255-b75b-67d6509c4cc1) 2024-09-10T23:40:19.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:19.879 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T23:40:19.936 INFO:teuthology.orchestra.run.smithi094.stdout:cephfs_data 2024-09-10T23:40:19.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:19.937 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp77380/subvol65011/df2d7e23-0c1e-4255-b75b-67d6509c4cc1) 2024-09-10T23:40:20.004 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:40:20.004 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp77380/subvol65011/.meta 2024-09-10T23:40:20.071 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol95790 2024-09-10T23:40:20.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19c6600640 1 -- 172.21.15.94:0/1049540628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19a4003a90 msgr2=0x7f19a4003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:20.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19c6600640 1 --2- 172.21.15.94:0/1049540628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19a4003a90 0x7f19a4003eb0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f19b00016c0 tx=0x7f19b001c730 comp rx=0 tx=0).stop 2024-09-10T23:40:20.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19c6600640 1 -- 172.21.15.94:0/1049540628 shutdown_connections 2024-09-10T23:40:20.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19c6600640 1 --2- 172.21.15.94:0/1049540628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19a4003a90 0x7f19a4003eb0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:20.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19c6600640 1 -- 172.21.15.94:0/1049540628 >> 172.21.15.94:0/1049540628 conn(0x7f19c00d1c60 msgr2=0x7f19c00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:20.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19c6600640 1 -- 172.21.15.94:0/1049540628 shutdown_connections 2024-09-10T23:40:20.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19c6600640 1 -- 172.21.15.94:0/1049540628 wait complete. 2024-09-10T23:40:20.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19c6600640 1 Processor -- start 2024-09-10T23:40:20.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19c6600640 1 -- start start 2024-09-10T23:40:20.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19c6600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19a4003a90 0x7f19c015f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:20.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19c6600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c009c530 0x7f19c00caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:20.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19c6600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19c009c9b0 0x7f19c00cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:20.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19c6600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f19c00ce3f0 con 0x7f19c009c9b0 2024-09-10T23:40:20.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19c6600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f19c00ce570 con 0x7f19a4003a90 2024-09-10T23:40:20.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19c6600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f19c00ce6f0 con 0x7f19c009c530 2024-09-10T23:40:20.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19bfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19c009c9b0 0x7f19c00cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:20.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19bfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19c009c9b0 0x7f19c00cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36184/0 (socket says 172.21.15.94:36184) 2024-09-10T23:40:20.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19bfe00640 1 -- 172.21.15.94:0/236011785 learned_addr learned my addr 172.21.15.94:0/236011785 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:20.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.259+0000 7f19bf400640 1 --2- 172.21.15.94:0/236011785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19a4003a90 0x7f19c015f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:20.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.263+0000 7f19b6a00640 1 --2- 172.21.15.94:0/236011785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c009c530 0x7f19c00caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:20.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.263+0000 7f19bfe00640 1 -- 172.21.15.94:0/236011785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c009c530 msgr2=0x7f19c00caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:20.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.263+0000 7f19bfe00640 1 --2- 172.21.15.94:0/236011785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c009c530 0x7f19c00caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:20.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.263+0000 7f19bfe00640 1 -- 172.21.15.94:0/236011785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19a4003a90 msgr2=0x7f19c015f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:20.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.263+0000 7f19bfe00640 1 --2- 172.21.15.94:0/236011785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19a4003a90 0x7f19c015f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:20.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.263+0000 7f19bfe00640 1 -- 172.21.15.94:0/236011785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c009cf30 con 0x7f19c009c9b0 2024-09-10T23:40:20.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.263+0000 7f19bfe00640 1 --2- 172.21.15.94:0/236011785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19c009c9b0 0x7f19c00cb440 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f19b8009fd0 tx=0x7f19b8010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:20.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.263+0000 7f19b7e00640 1 -- 172.21.15.94:0/236011785 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b800eea0 con 0x7f19c009c9b0 2024-09-10T23:40:20.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.263+0000 7f19b7e00640 1 -- 172.21.15.94:0/236011785 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19b8015070 con 0x7f19c009c9b0 2024-09-10T23:40:20.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.263+0000 7f19b7e00640 1 -- 172.21.15.94:0/236011785 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8003980 con 0x7f19c009c9b0 2024-09-10T23:40:20.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.263+0000 7f19c6600640 1 -- 172.21.15.94:0/236011785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c015c3f0 con 0x7f19c009c9b0 2024-09-10T23:40:20.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.263+0000 7f19c6600640 1 -- 172.21.15.94:0/236011785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c015c9b0 con 0x7f19c009c9b0 2024-09-10T23:40:20.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.263+0000 7f19c6600640 1 -- 172.21.15.94:0/236011785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c00cec50 con 0x7f19c009c9b0 2024-09-10T23:40:20.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.263+0000 7f19b7e00640 1 -- 172.21.15.94:0/236011785 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f19b8003b20 con 0x7f19c009c9b0 2024-09-10T23:40:20.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.263+0000 7f19b7e00640 1 --2- 172.21.15.94:0/236011785 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f199c072e50 0x7f199c075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:20.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.267+0000 7f19b7e00640 1 -- 172.21.15.94:0/236011785 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f19b8094bc0 con 0x7f19c009c9b0 2024-09-10T23:40:20.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.267+0000 7f19bf400640 1 --2- 172.21.15.94:0/236011785 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f199c072e50 0x7f199c075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:20.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.267+0000 7f19bf400640 1 --2- 172.21.15.94:0/236011785 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f199c072e50 0x7f199c075310 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f19a40057f0 tx=0x7f19b0019520 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:20.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.267+0000 7f19b7e00640 1 -- 172.21.15.94:0/236011785 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f19b8020020 con 0x7f19c009c9b0 2024-09-10T23:40:20.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.459+0000 7f19c6600640 1 -- 172.21.15.94:0/236011785 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol95790", "target": ["mon-mgr", ""]}) -- 0x7f19c015aa10 con 0x7f199c072e50 2024-09-10T23:40:20.469 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:20.467+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:20.469 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:20.467+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:20.469 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:20.467+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:20.469 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:20.467+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:20.469 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:20.467+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:20.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.511+0000 7f19b7e00640 1 -- 172.21.15.94:0/236011785 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f19b00ae040 con 0x7f199c072e50 2024-09-10T23:40:20.517 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/_nogroup/subvol95790/9b464252-103c-4bc8-b7c1-b8532b673f4b 2024-09-10T23:40:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.515+0000 7f19c6600640 1 -- 172.21.15.94:0/236011785 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f199c072e50 msgr2=0x7f199c075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.515+0000 7f19c6600640 1 --2- 172.21.15.94:0/236011785 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f199c072e50 0x7f199c075310 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f19a40057f0 tx=0x7f19b0019520 comp rx=0 tx=0).stop 2024-09-10T23:40:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.515+0000 7f19c6600640 1 -- 172.21.15.94:0/236011785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19c009c9b0 msgr2=0x7f19c00cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.515+0000 7f19c6600640 1 --2- 172.21.15.94:0/236011785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19c009c9b0 0x7f19c00cb440 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f19b8009fd0 tx=0x7f19b8010f40 comp rx=0 tx=0).stop 2024-09-10T23:40:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.515+0000 7f19c6600640 1 -- 172.21.15.94:0/236011785 shutdown_connections 2024-09-10T23:40:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.515+0000 7f19c6600640 1 --2- 172.21.15.94:0/236011785 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f199c072e50 0x7f199c075310 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.515+0000 7f19c6600640 1 --2- 172.21.15.94:0/236011785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19c009c9b0 0x7f19c00cb440 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.515+0000 7f19c6600640 1 --2- 172.21.15.94:0/236011785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c009c530 0x7f19c00caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.515+0000 7f19c6600640 1 --2- 172.21.15.94:0/236011785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19a4003a90 0x7f19c015f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.515+0000 7f19c6600640 1 -- 172.21.15.94:0/236011785 >> 172.21.15.94:0/236011785 conn(0x7f19c00d1c60 msgr2=0x7f19c00b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.515+0000 7f19c6600640 1 -- 172.21.15.94:0/236011785 shutdown_connections 2024-09-10T23:40:20.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.515+0000 7f19c6600640 1 -- 172.21.15.94:0/236011785 wait complete. 2024-09-10T23:40:20.537 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol65011 subvol_grp77380 2024-09-10T23:40:20.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.679+0000 7fb402600640 1 -- 172.21.15.94:0/3228430158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3fc157c60 msgr2=0x7fb3fc072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:20.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.679+0000 7fb402600640 1 --2- 172.21.15.94:0/3228430158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3fc157c60 0x7fb3fc072610 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc06fee0 tx=0x7fb3ec01c770 comp rx=0 tx=0).stop 2024-09-10T23:40:20.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.679+0000 7fb402600640 1 -- 172.21.15.94:0/3228430158 shutdown_connections 2024-09-10T23:40:20.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.679+0000 7fb402600640 1 --2- 172.21.15.94:0/3228430158 >> v2:172.21.15.181:3300/0 conn(0x7fb3fc06f600 0x7fb3fc0b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:20.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.679+0000 7fb402600640 1 --2- 172.21.15.94:0/3228430158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3fc157c60 0x7fb3fc072610 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:20.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.679+0000 7fb402600640 1 -- 172.21.15.94:0/3228430158 >> 172.21.15.94:0/3228430158 conn(0x7fb3fc06cdf0 msgr2=0x7fb3fc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:20.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.679+0000 7fb402600640 1 -- 172.21.15.94:0/3228430158 shutdown_connections 2024-09-10T23:40:20.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.679+0000 7fb402600640 1 -- 172.21.15.94:0/3228430158 wait complete. 2024-09-10T23:40:20.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.679+0000 7fb402600640 1 Processor -- start 2024-09-10T23:40:20.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.679+0000 7fb402600640 1 -- start start 2024-09-10T23:40:20.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb402600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3fc06f600 0x7fb3fc0cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:20.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb402600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3fc070150 0x7fb3fc14e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:20.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb402600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3fc157c60 0x7fb3fc163230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:20.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb402600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc15b270 con 0x7fb3fc070150 2024-09-10T23:40:20.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb402600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc15b0f0 con 0x7fb3fc06f600 2024-09-10T23:40:20.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb402600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb3fc15b3f0 con 0x7fb3fc157c60 2024-09-10T23:40:20.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb3f3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3fc070150 0x7fb3fc14e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:20.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb3f3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3fc070150 0x7fb3fc14e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36226/0 (socket says 172.21.15.94:36226) 2024-09-10T23:40:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb3f3e00640 1 -- 172.21.15.94:0/3120156444 learned_addr learned my addr 172.21.15.94:0/3120156444 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb3fb400640 1 --2- 172.21.15.94:0/3120156444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3fc06f600 0x7fb3fc0cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb3fbe00640 1 --2- 172.21.15.94:0/3120156444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3fc157c60 0x7fb3fc163230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb3f3e00640 1 -- 172.21.15.94:0/3120156444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3fc157c60 msgr2=0x7fb3fc163230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb3f3e00640 1 --2- 172.21.15.94:0/3120156444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3fc157c60 0x7fb3fc163230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb3f3e00640 1 -- 172.21.15.94:0/3120156444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3fc06f600 msgr2=0x7fb3fc0cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb3f3e00640 1 --2- 172.21.15.94:0/3120156444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3fc06f600 0x7fb3fc0cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb3f3e00640 1 -- 172.21.15.94:0/3120156444 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3fc15bfc0 con 0x7fb3fc070150 2024-09-10T23:40:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb3f3e00640 1 --2- 172.21.15.94:0/3120156444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3fc070150 0x7fb3fc14e130 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400c490 tx=0x7fb3e400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:20.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb3f3400640 1 -- 172.21.15.94:0/3120156444 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4016020 con 0x7fb3fc070150 2024-09-10T23:40:20.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb3f3400640 1 -- 172.21.15.94:0/3120156444 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3e4012940 con 0x7fb3fc070150 2024-09-10T23:40:20.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb402600640 1 -- 172.21.15.94:0/3120156444 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3fc15c2a0 con 0x7fb3fc070150 2024-09-10T23:40:20.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb3f3400640 1 -- 172.21.15.94:0/3120156444 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4012d90 con 0x7fb3fc070150 2024-09-10T23:40:20.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb402600640 1 -- 172.21.15.94:0/3120156444 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3fc15c860 con 0x7fb3fc070150 2024-09-10T23:40:20.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb402600640 1 -- 172.21.15.94:0/3120156444 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3fc0b35c0 con 0x7fb3fc070150 2024-09-10T23:40:20.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.683+0000 7fb3f3400640 1 -- 172.21.15.94:0/3120156444 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fb3e401b400 con 0x7fb3fc070150 2024-09-10T23:40:20.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.687+0000 7fb3f3400640 1 --2- 172.21.15.94:0/3120156444 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb3d8072e80 0x7fb3d8075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:20.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.687+0000 7fb3f3400640 1 -- 172.21.15.94:0/3120156444 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7fb3e4094670 con 0x7fb3fc070150 2024-09-10T23:40:20.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.687+0000 7fb3fb400640 1 --2- 172.21.15.94:0/3120156444 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb3d8072e80 0x7fb3d8075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:20.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.687+0000 7fb3fb400640 1 --2- 172.21.15.94:0/3120156444 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb3d8072e80 0x7fb3d8075340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0af000 tx=0x7fb3ec01afb0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:20.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.691+0000 7fb3f3400640 1 -- 172.21.15.94:0/3120156444 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb3e4020020 con 0x7fb3fc070150 2024-09-10T23:40:20.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.883+0000 7fb402600640 1 -- 172.21.15.94:0/3120156444 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol65011", "group_name": "subvol_grp77380", "target": ["mon-mgr", ""]}) -- 0x7fb3fc14e670 con 0x7fb3d8072e80 2024-09-10T23:40:20.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.927+0000 7fb3f3400640 1 -- 172.21.15.94:0/3120156444 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7fb3fc14e670 con 0x7fb3d8072e80 2024-09-10T23:40:20.930 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/subvol_grp77380/subvol65011/df2d7e23-0c1e-4255-b75b-67d6509c4cc1 2024-09-10T23:40:20.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.927+0000 7fb402600640 1 -- 172.21.15.94:0/3120156444 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb3d8072e80 msgr2=0x7fb3d8075340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:20.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.927+0000 7fb402600640 1 --2- 172.21.15.94:0/3120156444 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb3d8072e80 0x7fb3d8075340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0af000 tx=0x7fb3ec01afb0 comp rx=0 tx=0).stop 2024-09-10T23:40:20.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.927+0000 7fb402600640 1 -- 172.21.15.94:0/3120156444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3fc070150 msgr2=0x7fb3fc14e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:20.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.927+0000 7fb402600640 1 --2- 172.21.15.94:0/3120156444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3fc070150 0x7fb3fc14e130 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400c490 tx=0x7fb3e400c950 comp rx=0 tx=0).stop 2024-09-10T23:40:20.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.927+0000 7fb402600640 1 -- 172.21.15.94:0/3120156444 shutdown_connections 2024-09-10T23:40:20.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.927+0000 7fb402600640 1 --2- 172.21.15.94:0/3120156444 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb3d8072e80 0x7fb3d8075340 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:20.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.927+0000 7fb402600640 1 --2- 172.21.15.94:0/3120156444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3fc157c60 0x7fb3fc163230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:20.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.927+0000 7fb402600640 1 --2- 172.21.15.94:0/3120156444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3fc070150 0x7fb3fc14e130 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:20.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.927+0000 7fb402600640 1 --2- 172.21.15.94:0/3120156444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3fc06f600 0x7fb3fc0cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.927+0000 7fb402600640 1 -- 172.21.15.94:0/3120156444 >> 172.21.15.94:0/3120156444 conn(0x7fb3fc06cdf0 msgr2=0x7fb3fc06eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.927+0000 7fb402600640 1 -- 172.21.15.94:0/3120156444 shutdown_connections 2024-09-10T23:40:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:20.927+0000 7fb402600640 1 -- 172.21.15.94:0/3120156444 wait complete. 2024-09-10T23:40:20.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:20.952 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol95790/.meta) 2024-09-10T23:40:20.976 INFO:teuthology.orchestra.run.smithi094.stdout:[GLOBAL] 2024-09-10T23:40:20.976 INFO:teuthology.orchestra.run.smithi094.stdout:version = 2 2024-09-10T23:40:20.976 INFO:teuthology.orchestra.run.smithi094.stdout:type = subvolume 2024-09-10T23:40:20.976 INFO:teuthology.orchestra.run.smithi094.stdout:path = /volumes/_nogroup/subvol95790/9b464252-103c-4bc8-b7c1-b8532b673f4b 2024-09-10T23:40:20.976 INFO:teuthology.orchestra.run.smithi094.stdout:state = complete 2024-09-10T23:40:20.976 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:20.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:20.979 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp77380/subvol65011/.meta) 2024-09-10T23:40:21.045 INFO:teuthology.orchestra.run.smithi094.stdout:[GLOBAL] 2024-09-10T23:40:21.045 INFO:teuthology.orchestra.run.smithi094.stdout:version = 2 2024-09-10T23:40:21.045 INFO:teuthology.orchestra.run.smithi094.stdout:type = subvolume 2024-09-10T23:40:21.045 INFO:teuthology.orchestra.run.smithi094.stdout:path = /volumes/subvol_grp77380/subvol65011/df2d7e23-0c1e-4255-b75b-67d6509c4cc1 2024-09-10T23:40:21.045 INFO:teuthology.orchestra.run.smithi094.stdout:state = complete 2024-09-10T23:40:21.045 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:21.047 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol95790 2024-09-10T23:40:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.227+0000 7fcb5b400640 1 -- 172.21.15.94:0/623403682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5406e7c0 msgr2=0x7fcb5406ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.227+0000 7fcb5b400640 1 --2- 172.21.15.94:0/623403682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5406e7c0 0x7fcb5406ec00 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009900 tx=0x7fcb3c01c880 comp rx=0 tx=0).stop 2024-09-10T23:40:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.227+0000 7fcb5b400640 1 -- 172.21.15.94:0/623403682 shutdown_connections 2024-09-10T23:40:21.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.227+0000 7fcb5b400640 1 --2- 172.21.15.94:0/623403682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5406e7c0 0x7fcb5406ec00 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:21.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.227+0000 7fcb5b400640 1 --2- 172.21.15.94:0/623403682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb540c64f0 0x7fcb540cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:21.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.227+0000 7fcb5b400640 1 -- 172.21.15.94:0/623403682 >> 172.21.15.94:0/623403682 conn(0x7fcb540cfaf0 msgr2=0x7fcb540cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:21.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb5b400640 1 -- 172.21.15.94:0/623403682 shutdown_connections 2024-09-10T23:40:21.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb5b400640 1 -- 172.21.15.94:0/623403682 wait complete. 2024-09-10T23:40:21.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb5b400640 1 Processor -- start 2024-09-10T23:40:21.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb5b400640 1 -- start start 2024-09-10T23:40:21.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb5b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5406e7c0 0x7fcb54156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:21.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb5b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb540c64f0 0x7fcb54145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:21.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb5b400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb5415af90 0x7fcb54146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:21.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb5b400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcb54168400 con 0x7fcb5406e7c0 2024-09-10T23:40:21.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb5b400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcb540cec10 con 0x7fcb540c64f0 2024-09-10T23:40:21.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb5b400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fcb54168580 con 0x7fcb5415af90 2024-09-10T23:40:21.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb58a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5406e7c0 0x7fcb54156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:21.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb58a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5406e7c0 0x7fcb54156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36248/0 (socket says 172.21.15.94:36248) 2024-09-10T23:40:21.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb58a00640 1 -- 172.21.15.94:0/894868311 learned_addr learned my addr 172.21.15.94:0/894868311 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:21.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb4be00640 1 --2- 172.21.15.94:0/894868311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb540c64f0 0x7fcb54145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:21.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb59400640 1 --2- 172.21.15.94:0/894868311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb5415af90 0x7fcb54146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:21.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb58a00640 1 -- 172.21.15.94:0/894868311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb5415af90 msgr2=0x7fcb54146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:21.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb58a00640 1 --2- 172.21.15.94:0/894868311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb5415af90 0x7fcb54146520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:21.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb58a00640 1 -- 172.21.15.94:0/894868311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb540c64f0 msgr2=0x7fcb54145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:21.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb58a00640 1 --2- 172.21.15.94:0/894868311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb540c64f0 0x7fcb54145fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:21.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb58a00640 1 -- 172.21.15.94:0/894868311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb540b5900 con 0x7fcb5406e7c0 2024-09-10T23:40:21.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb58a00640 1 --2- 172.21.15.94:0/894868311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5406e7c0 0x7fcb54156410 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400ea20 tx=0x7fcb4400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:21.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.231+0000 7fcb51600640 1 -- 172.21.15.94:0/894868311 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400cd80 con 0x7fcb5406e7c0 2024-09-10T23:40:21.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.235+0000 7fcb51600640 1 -- 172.21.15.94:0/894868311 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb44010430 con 0x7fcb5406e7c0 2024-09-10T23:40:21.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.235+0000 7fcb5b400640 1 -- 172.21.15.94:0/894868311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb540c6bc0 con 0x7fcb5406e7c0 2024-09-10T23:40:21.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.235+0000 7fcb51600640 1 -- 172.21.15.94:0/894868311 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44010860 con 0x7fcb5406e7c0 2024-09-10T23:40:21.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.235+0000 7fcb5b400640 1 -- 172.21.15.94:0/894868311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb540c7180 con 0x7fcb5406e7c0 2024-09-10T23:40:21.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.235+0000 7fcb5b400640 1 -- 172.21.15.94:0/894868311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb5406ec00 con 0x7fcb5406e7c0 2024-09-10T23:40:21.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.235+0000 7fcb51600640 1 -- 172.21.15.94:0/894868311 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7fcb44010a00 con 0x7fcb5406e7c0 2024-09-10T23:40:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.235+0000 7fcb51600640 1 --2- 172.21.15.94:0/894868311 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb30072e50 0x7fcb30075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.235+0000 7fcb51600640 1 -- 172.21.15.94:0/894868311 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7fcb44095150 con 0x7fcb5406e7c0 2024-09-10T23:40:21.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.235+0000 7fcb4be00640 1 --2- 172.21.15.94:0/894868311 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb30072e50 0x7fcb30075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:21.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.239+0000 7fcb4be00640 1 --2- 172.21.15.94:0/894868311 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb30072e50 0x7fcb30075310 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0af000 tx=0x7fcb3c01b120 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:21.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.239+0000 7fcb51600640 1 -- 172.21.15.94:0/894868311 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcb4405f110 con 0x7fcb5406e7c0 2024-09-10T23:40:21.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.431+0000 7fcb5b400640 1 -- 172.21.15.94:0/894868311 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol95790", "target": ["mon-mgr", ""]}) -- 0x7fcb540d0850 con 0x7fcb30072e50 2024-09-10T23:40:21.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.443+0000 7fcb51600640 1 -- 172.21.15.94:0/894868311 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcb540d0850 con 0x7fcb30072e50 2024-09-10T23:40:21.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.447+0000 7fcb5b400640 1 -- 172.21.15.94:0/894868311 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb30072e50 msgr2=0x7fcb30075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:21.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.447+0000 7fcb5b400640 1 --2- 172.21.15.94:0/894868311 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb30072e50 0x7fcb30075310 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0af000 tx=0x7fcb3c01b120 comp rx=0 tx=0).stop 2024-09-10T23:40:21.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.447+0000 7fcb5b400640 1 -- 172.21.15.94:0/894868311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5406e7c0 msgr2=0x7fcb54156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:21.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.447+0000 7fcb5b400640 1 --2- 172.21.15.94:0/894868311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5406e7c0 0x7fcb54156410 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400ea20 tx=0x7fcb4400eee0 comp rx=0 tx=0).stop 2024-09-10T23:40:21.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.447+0000 7fcb5b400640 1 -- 172.21.15.94:0/894868311 shutdown_connections 2024-09-10T23:40:21.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.447+0000 7fcb5b400640 1 --2- 172.21.15.94:0/894868311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb540c64f0 0x7fcb54145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:21.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.447+0000 7fcb5b400640 1 --2- 172.21.15.94:0/894868311 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb30072e50 0x7fcb30075310 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:21.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.447+0000 7fcb5b400640 1 --2- 172.21.15.94:0/894868311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb5415af90 0x7fcb54146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:21.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.447+0000 7fcb5b400640 1 --2- 172.21.15.94:0/894868311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb5406e7c0 0x7fcb54156410 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:21.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.447+0000 7fcb5b400640 1 -- 172.21.15.94:0/894868311 >> 172.21.15.94:0/894868311 conn(0x7fcb540cfaf0 msgr2=0x7fcb541575f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:21.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.447+0000 7fcb5b400640 1 -- 172.21.15.94:0/894868311 shutdown_connections 2024-09-10T23:40:21.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.447+0000 7fcb5b400640 1 -- 172.21.15.94:0/894868311 wait complete. 2024-09-10T23:40:21.453 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:21.451+0000 7f9ea3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:21.454 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:21.451+0000 7f9ea3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:21.454 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:21.451+0000 7f9ea3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:21.454 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:21.451+0000 7f9ea3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:21.454 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:21.451+0000 7f9ea3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:21.469 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:21.467+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:21.469 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:21.467+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:21.469 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:21.467+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:21.469 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:21.467+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:21.469 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:21.467+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:21.474 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol65011 subvol_grp77380 2024-09-10T23:40:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.615+0000 7f7ac0000640 1 -- 172.21.15.94:0/317106906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ab815db80 msgr2=0x7f7ab815dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.615+0000 7f7ac0000640 1 --2- 172.21.15.94:0/317106906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ab815db80 0x7f7ab815dfb0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80af5e0 tx=0x7f7ab001cb80 comp rx=0 tx=0).stop 2024-09-10T23:40:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.615+0000 7f7ac0000640 1 -- 172.21.15.94:0/317106906 shutdown_connections 2024-09-10T23:40:21.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.615+0000 7f7ac0000640 1 --2- 172.21.15.94:0/317106906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ab815db80 0x7f7ab815dfb0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:21.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.615+0000 7f7ac0000640 1 --2- 172.21.15.94:0/317106906 >> v2:172.21.15.181:3300/0 conn(0x7f7ab80cfe90 0x7f7ab80d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:21.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.615+0000 7f7ac0000640 1 -- 172.21.15.94:0/317106906 >> 172.21.15.94:0/317106906 conn(0x7f7ab806cdf0 msgr2=0x7f7ab806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:21.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7ac0000640 1 -- 172.21.15.94:0/317106906 shutdown_connections 2024-09-10T23:40:21.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7ac0000640 1 -- 172.21.15.94:0/317106906 wait complete. 2024-09-10T23:40:21.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7ac0000640 1 Processor -- start 2024-09-10T23:40:21.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7ac0000640 1 -- start start 2024-09-10T23:40:21.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7ac0000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ab8070700 0x7f7ab8153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:21.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7ac0000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ab80cfe90 0x7f7ab813b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:21.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7ac0000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ab815db80 0x7f7ab813be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7ac0000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab8167a80 con 0x7f7ab80cfe90 2024-09-10T23:40:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7ac0000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab8074420 con 0x7f7ab8070700 2024-09-10T23:40:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7ac0000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab8167c00 con 0x7f7ab815db80 2024-09-10T23:40:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7abcc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ab80cfe90 0x7f7ab813b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7abcc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ab80cfe90 0x7f7ab813b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36288/0 (socket says 172.21.15.94:36288) 2024-09-10T23:40:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7abcc00640 1 -- 172.21.15.94:0/475210146 learned_addr learned my addr 172.21.15.94:0/475210146 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7abe000640 1 --2- 172.21.15.94:0/475210146 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ab815db80 0x7f7ab813be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7abd600640 1 --2- 172.21.15.94:0/475210146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ab8070700 0x7f7ab8153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7abcc00640 1 -- 172.21.15.94:0/475210146 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ab815db80 msgr2=0x7f7ab813be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7abcc00640 1 --2- 172.21.15.94:0/475210146 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ab815db80 0x7f7ab813be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7abcc00640 1 -- 172.21.15.94:0/475210146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ab8070700 msgr2=0x7f7ab8153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7abcc00640 1 --2- 172.21.15.94:0/475210146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ab8070700 0x7f7ab8153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:21.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7abcc00640 1 -- 172.21.15.94:0/475210146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab80ccc50 con 0x7f7ab80cfe90 2024-09-10T23:40:21.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7abd600640 1 --2- 172.21.15.94:0/475210146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ab8070700 0x7f7ab8153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:40:21.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7abcc00640 1 --2- 172.21.15.94:0/475210146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ab80cfe90 0x7f7ab813b6e0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400ea20 tx=0x7f7aa400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:21.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.619+0000 7f7ab5600640 1 -- 172.21.15.94:0/475210146 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa400cd80 con 0x7f7ab80cfe90 2024-09-10T23:40:21.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.623+0000 7f7ab5600640 1 -- 172.21.15.94:0/475210146 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7aa4010430 con 0x7f7ab80cfe90 2024-09-10T23:40:21.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.623+0000 7f7ac0000640 1 -- 172.21.15.94:0/475210146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab8153b30 con 0x7f7ab80cfe90 2024-09-10T23:40:21.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.623+0000 7f7ab5600640 1 -- 172.21.15.94:0/475210146 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4010860 con 0x7f7ab80cfe90 2024-09-10T23:40:21.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.623+0000 7f7ac0000640 1 -- 172.21.15.94:0/475210146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab80af770 con 0x7f7ab80cfe90 2024-09-10T23:40:21.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.623+0000 7f7ac0000640 1 -- 172.21.15.94:0/475210146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab80d02e0 con 0x7f7ab80cfe90 2024-09-10T23:40:21.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.623+0000 7f7ab5600640 1 -- 172.21.15.94:0/475210146 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 97159+0+0 (secure 0 0 0) 0x7f7aa4010a00 con 0x7f7ab80cfe90 2024-09-10T23:40:21.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.623+0000 7f7ab5600640 1 --2- 172.21.15.94:0/475210146 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7a98072e50 0x7f7a98075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:21.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.623+0000 7f7ab5600640 1 -- 172.21.15.94:0/475210146 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f7aa40950a0 con 0x7f7ab80cfe90 2024-09-10T23:40:21.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.623+0000 7f7abd600640 1 --2- 172.21.15.94:0/475210146 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7a98072e50 0x7f7a98075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:21.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.627+0000 7f7abd600640 1 --2- 172.21.15.94:0/475210146 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7a98072e50 0x7f7a98075310 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7aac000f70 tx=0x7f7aac009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:21.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.627+0000 7f7ab5600640 1 -- 172.21.15.94:0/475210146 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7ab80ccc50 con 0x7f7ab80cfe90 2024-09-10T23:40:21.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:21.819+0000 7f7ac0000640 1 -- 172.21.15.94:0/475210146 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol65011", "group_name": "subvol_grp77380", "target": ["mon-mgr", ""]}) -- 0x7f7ab80b2c80 con 0x7f7a98072e50 2024-09-10T23:40:22.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:22.967+0000 7f7ab5600640 1 -- 172.21.15.94:0/475210146 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f7aa405b3e0 con 0x7f7ab80cfe90 2024-09-10T23:40:28.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:28.855+0000 7f7ab5600640 1 -- 172.21.15.94:0/475210146 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7ab80b2c80 con 0x7f7a98072e50 2024-09-10T23:40:28.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:28.859+0000 7f7ac0000640 1 -- 172.21.15.94:0/475210146 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7a98072e50 msgr2=0x7f7a98075310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:28.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:28.859+0000 7f7ac0000640 1 --2- 172.21.15.94:0/475210146 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7a98072e50 0x7f7a98075310 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7aac000f70 tx=0x7f7aac009040 comp rx=0 tx=0).stop 2024-09-10T23:40:28.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:28.859+0000 7f7ac0000640 1 -- 172.21.15.94:0/475210146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ab80cfe90 msgr2=0x7f7ab813b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:28.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:28.859+0000 7f7ac0000640 1 --2- 172.21.15.94:0/475210146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ab80cfe90 0x7f7ab813b6e0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400ea20 tx=0x7f7aa400eee0 comp rx=0 tx=0).stop 2024-09-10T23:40:28.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:28.859+0000 7f7ac0000640 1 -- 172.21.15.94:0/475210146 shutdown_connections 2024-09-10T23:40:28.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:28.859+0000 7f7ac0000640 1 --2- 172.21.15.94:0/475210146 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7a98072e50 0x7f7a98075310 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:28.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:28.859+0000 7f7ac0000640 1 --2- 172.21.15.94:0/475210146 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ab815db80 0x7f7ab813be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:28.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:28.859+0000 7f7ac0000640 1 --2- 172.21.15.94:0/475210146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ab80cfe90 0x7f7ab813b6e0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:28.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:28.859+0000 7f7ac0000640 1 --2- 172.21.15.94:0/475210146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ab8070700 0x7f7ab8153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:28.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:28.859+0000 7f7ac0000640 1 -- 172.21.15.94:0/475210146 >> 172.21.15.94:0/475210146 conn(0x7f7ab806cdf0 msgr2=0x7f7ab806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:28.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:28.859+0000 7f7ac0000640 1 -- 172.21.15.94:0/475210146 shutdown_connections 2024-09-10T23:40:28.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:28.859+0000 7f7ac0000640 1 -- 172.21.15.94:0/475210146 wait complete. 2024-09-10T23:40:28.889 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T23:40:28.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T23:40:28.949 INFO:teuthology.orchestra.run.smithi094.stdout:2 2024-09-10T23:40:28.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:40:28.956 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:40:28.956 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:40:29.048 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:40:29.155 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:40:29.155 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:40:29.155 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:40:29.155 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:40:29.164 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:40:29.164 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:40:29.212 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:40:29.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.391+0000 7fdc92c00640 1 -- 172.21.15.94:0/40118543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc8c0c49b0 msgr2=0x7fdc8c0cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:29.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.391+0000 7fdc92c00640 1 --2- 172.21.15.94:0/40118543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc8c0c49b0 0x7fdc8c0cadc0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009900 tx=0x7fdc7801c8e0 comp rx=0 tx=0).stop 2024-09-10T23:40:29.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.391+0000 7fdc92c00640 1 -- 172.21.15.94:0/40118543 shutdown_connections 2024-09-10T23:40:29.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.391+0000 7fdc92c00640 1 --2- 172.21.15.94:0/40118543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc8c0c49b0 0x7fdc8c0cadc0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:29.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.391+0000 7fdc92c00640 1 --2- 172.21.15.94:0/40118543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c158790 0x7fdc8c06fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:29.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.391+0000 7fdc92c00640 1 -- 172.21.15.94:0/40118543 >> 172.21.15.94:0/40118543 conn(0x7fdc8c0cfaf0 msgr2=0x7fdc8c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:29.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.391+0000 7fdc92c00640 1 -- 172.21.15.94:0/40118543 shutdown_connections 2024-09-10T23:40:29.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.391+0000 7fdc92c00640 1 -- 172.21.15.94:0/40118543 wait complete. 2024-09-10T23:40:29.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc92c00640 1 Processor -- start 2024-09-10T23:40:29.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc92c00640 1 -- start start 2024-09-10T23:40:29.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc92c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c0c49b0 0x7fdc8c15da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:29.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc92c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc8c0cb300 0x7fdc8c14ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:29.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc92c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc8c158790 0x7fdc8c14c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:29.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc92c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c0cea20 con 0x7fdc8c0cb300 2024-09-10T23:40:29.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc92c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c0ce8a0 con 0x7fdc8c158790 2024-09-10T23:40:29.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc92c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fdc8c0ceba0 con 0x7fdc8c0c49b0 2024-09-10T23:40:29.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc8b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc8c0cb300 0x7fdc8c14ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:29.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc8b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc8c0cb300 0x7fdc8c14ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46480/0 (socket says 172.21.15.94:46480) 2024-09-10T23:40:29.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc8b400640 1 -- 172.21.15.94:0/4285172515 learned_addr learned my addr 172.21.15.94:0/4285172515 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:29.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc8be00640 1 --2- 172.21.15.94:0/4285172515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c0c49b0 0x7fdc8c15da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:29.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc90c00640 1 --2- 172.21.15.94:0/4285172515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc8c158790 0x7fdc8c14c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:29.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc8b400640 1 -- 172.21.15.94:0/4285172515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c0c49b0 msgr2=0x7fdc8c15da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:29.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc8b400640 1 --2- 172.21.15.94:0/4285172515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c0c49b0 0x7fdc8c15da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:29.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc8b400640 1 -- 172.21.15.94:0/4285172515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc8c158790 msgr2=0x7fdc8c14c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:29.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc8b400640 1 --2- 172.21.15.94:0/4285172515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc8c158790 0x7fdc8c14c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:29.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc8b400640 1 -- 172.21.15.94:0/4285172515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8c14cc10 con 0x7fdc8c0cb300 2024-09-10T23:40:29.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc8b400640 1 --2- 172.21.15.94:0/4285172515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc8c0cb300 0x7fdc8c14ce00 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009850 tx=0x7fdc780a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:29.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc83e00640 1 -- 172.21.15.94:0/4285172515 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc780b2040 con 0x7fdc8c0cb300 2024-09-10T23:40:29.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc83e00640 1 -- 172.21.15.94:0/4285172515 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc780a6ca0 con 0x7fdc8c0cb300 2024-09-10T23:40:29.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc83e00640 1 -- 172.21.15.94:0/4285172515 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78005280 con 0x7fdc8c0cb300 2024-09-10T23:40:29.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc92c00640 1 -- 172.21.15.94:0/4285172515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8c15e330 con 0x7fdc8c0cb300 2024-09-10T23:40:29.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc92c00640 1 -- 172.21.15.94:0/4285172515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8c0744f0 con 0x7fdc8c0cb300 2024-09-10T23:40:29.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.395+0000 7fdc92c00640 1 -- 172.21.15.94:0/4285172515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc8c0cf0c0 con 0x7fdc8c0cb300 2024-09-10T23:40:29.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.399+0000 7fdc83e00640 1 -- 172.21.15.94:0/4285172515 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7fdc78005420 con 0x7fdc8c0cb300 2024-09-10T23:40:29.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.399+0000 7fdc83e00640 1 --2- 172.21.15.94:0/4285172515 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdc6c073000 0x7fdc6c0754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:29.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.399+0000 7fdc83e00640 1 -- 172.21.15.94:0/4285172515 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7fdc7812b7f0 con 0x7fdc8c0cb300 2024-09-10T23:40:29.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.399+0000 7fdc8be00640 1 --2- 172.21.15.94:0/4285172515 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdc6c073000 0x7fdc6c0754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:29.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.399+0000 7fdc8be00640 1 --2- 172.21.15.94:0/4285172515 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdc6c073000 0x7fdc6c0754c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c151f10 tx=0x7fdc7c00b2e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:29.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.403+0000 7fdc83e00640 1 -- 172.21.15.94:0/4285172515 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdc8c15e330 con 0x7fdc8c0cb300 2024-09-10T23:40:29.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.579+0000 7fdc92c00640 1 -- 172.21.15.94:0/4285172515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdc8c15c1f0 con 0x7fdc8c0cb300 2024-09-10T23:40:29.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.579+0000 7fdc83e00640 1 -- 172.21.15.94:0/4285172515 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v151) ==== 80+0+0 (secure 0 0 0) 0x7fdc780164d0 con 0x7fdc8c0cb300 2024-09-10T23:40:29.584 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:40:29.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.583+0000 7fdc92c00640 1 -- 172.21.15.94:0/4285172515 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdc6c073000 msgr2=0x7fdc6c0754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:29.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.583+0000 7fdc92c00640 1 --2- 172.21.15.94:0/4285172515 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdc6c073000 0x7fdc6c0754c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c151f10 tx=0x7fdc7c00b2e0 comp rx=0 tx=0).stop 2024-09-10T23:40:29.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.583+0000 7fdc92c00640 1 -- 172.21.15.94:0/4285172515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc8c0cb300 msgr2=0x7fdc8c14ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:29.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.583+0000 7fdc92c00640 1 --2- 172.21.15.94:0/4285172515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc8c0cb300 0x7fdc8c14ce00 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009850 tx=0x7fdc780a55a0 comp rx=0 tx=0).stop 2024-09-10T23:40:29.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.583+0000 7fdc92c00640 1 -- 172.21.15.94:0/4285172515 shutdown_connections 2024-09-10T23:40:29.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.583+0000 7fdc92c00640 1 --2- 172.21.15.94:0/4285172515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc8c158790 0x7fdc8c14c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:29.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.583+0000 7fdc92c00640 1 --2- 172.21.15.94:0/4285172515 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdc6c073000 0x7fdc6c0754c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:29.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.583+0000 7fdc92c00640 1 --2- 172.21.15.94:0/4285172515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc8c0cb300 0x7fdc8c14ce00 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:29.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.583+0000 7fdc92c00640 1 --2- 172.21.15.94:0/4285172515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c0c49b0 0x7fdc8c15da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:29.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.583+0000 7fdc92c00640 1 -- 172.21.15.94:0/4285172515 >> 172.21.15.94:0/4285172515 conn(0x7fdc8c0cfaf0 msgr2=0x7fdc8c1516c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:29.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.583+0000 7fdc92c00640 1 -- 172.21.15.94:0/4285172515 shutdown_connections 2024-09-10T23:40:29.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.583+0000 7fdc92c00640 1 -- 172.21.15.94:0/4285172515 wait complete. 2024-09-10T23:40:29.605 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-09-10T23:40:29.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:29.606 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T23:40:29.624 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[52141]: fuse finished with error 0 and tester_r 0 2024-09-10T23:40:29.626 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:40:29.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:29.626 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T23:40:29.793 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:40:29.793 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:40:29.793 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:40:29.802 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:40:29.802 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:40:29.856 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:40:30.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.995+0000 7fd7bfa00640 1 -- 172.21.15.94:0/2776918238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7b80ce280 msgr2=0x7fd7b80cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:30.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.995+0000 7fd7bfa00640 1 --2- 172.21.15.94:0/2776918238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7b80ce280 0x7fd7b80cf280 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800a950 tx=0x7fd7a801db70 comp rx=0 tx=0).stop 2024-09-10T23:40:30.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.995+0000 7fd7bfa00640 1 -- 172.21.15.94:0/2776918238 shutdown_connections 2024-09-10T23:40:30.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.995+0000 7fd7bfa00640 1 --2- 172.21.15.94:0/2776918238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7b80ce280 0x7fd7b80cf280 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.995+0000 7fd7bfa00640 1 -- 172.21.15.94:0/2776918238 >> 172.21.15.94:0/2776918238 conn(0x7fd7b80cfaf0 msgr2=0x7fd7b80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:30.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.995+0000 7fd7bfa00640 1 -- 172.21.15.94:0/2776918238 shutdown_connections 2024-09-10T23:40:30.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.995+0000 7fd7bfa00640 1 -- 172.21.15.94:0/2776918238 wait complete. 2024-09-10T23:40:30.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bfa00640 1 Processor -- start 2024-09-10T23:40:30.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bfa00640 1 -- start start 2024-09-10T23:40:30.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bfa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7b806e7c0 0x7fd7b81f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:30.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bfa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7b80c64f0 0x7fd7b814b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:30.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bfa00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7b80ce280 0x7fd7b8151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:30.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bfa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b81687a0 con 0x7fd7b806e7c0 2024-09-10T23:40:30.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bfa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b8168620 con 0x7fd7b80c64f0 2024-09-10T23:40:30.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bfa00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd7b8168920 con 0x7fd7b80ce280 2024-09-10T23:40:30.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bd000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7b806e7c0 0x7fd7b81f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:30.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bd000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7b806e7c0 0x7fd7b81f82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46498/0 (socket says 172.21.15.94:46498) 2024-09-10T23:40:30.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bd000640 1 -- 172.21.15.94:0/3578648359 learned_addr learned my addr 172.21.15.94:0/3578648359 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:30.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bd000640 1 -- 172.21.15.94:0/3578648359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7b80ce280 msgr2=0x7fd7b8151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:30.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bda00640 1 --2- 172.21.15.94:0/3578648359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7b80ce280 0x7fd7b8151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:30.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7afe00640 1 --2- 172.21.15.94:0/3578648359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7b80c64f0 0x7fd7b814b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:30.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bd000640 1 --2- 172.21.15.94:0/3578648359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7b80ce280 0x7fd7b8151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bd000640 1 -- 172.21.15.94:0/3578648359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7b80c64f0 msgr2=0x7fd7b814b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bd000640 1 --2- 172.21.15.94:0/3578648359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7b80c64f0 0x7fd7b814b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bd000640 1 -- 172.21.15.94:0/3578648359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b8050ff0 con 0x7fd7b806e7c0 2024-09-10T23:40:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bda00640 1 --2- 172.21.15.94:0/3578648359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7b80ce280 0x7fd7b8151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7afe00640 1 --2- 172.21.15.94:0/3578648359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7b80c64f0 0x7fd7b814b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bd000640 1 --2- 172.21.15.94:0/3578648359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7b806e7c0 0x7fd7b81f82e0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800a8a0 tx=0x7fd7a80a6940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7b5600640 1 -- 172.21.15.94:0/3578648359 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a80b3040 con 0x7fd7b806e7c0 2024-09-10T23:40:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7b5600640 1 -- 172.21.15.94:0/3578648359 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7a8003970 con 0x7fd7b806e7c0 2024-09-10T23:40:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bfa00640 1 -- 172.21.15.94:0/3578648359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b80b00a0 con 0x7fd7b806e7c0 2024-09-10T23:40:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7b5600640 1 -- 172.21.15.94:0/3578648359 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8003d60 con 0x7fd7b806e7c0 2024-09-10T23:40:30.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bfa00640 1 -- 172.21.15.94:0/3578648359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b80b0660 con 0x7fd7b806e7c0 2024-09-10T23:40:30.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:29.999+0000 7fd7bfa00640 1 -- 172.21.15.94:0/3578648359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b80d0850 con 0x7fd7b806e7c0 2024-09-10T23:40:30.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.003+0000 7fd7b5600640 1 -- 172.21.15.94:0/3578648359 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7fd7a800b040 con 0x7fd7b806e7c0 2024-09-10T23:40:30.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.003+0000 7fd7b5600640 1 --2- 172.21.15.94:0/3578648359 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd794073000 0x7fd7940754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:30.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.003+0000 7fd7b5600640 1 -- 172.21.15.94:0/3578648359 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7fd7a812e6a0 con 0x7fd7b806e7c0 2024-09-10T23:40:30.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.003+0000 7fd7afe00640 1 --2- 172.21.15.94:0/3578648359 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd794073000 0x7fd7940754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:30.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.003+0000 7fd7afe00640 1 --2- 172.21.15.94:0/3578648359 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd794073000 0x7fd7940754c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0000f70 tx=0x7fd7a0009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:30.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.007+0000 7fd7b5600640 1 -- 172.21.15.94:0/3578648359 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd7a80bf060 con 0x7fd7b806e7c0 2024-09-10T23:40:30.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.183+0000 7fd7bfa00640 1 -- 172.21.15.94:0/3578648359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd7b8157540 con 0x7fd7b806e7c0 2024-09-10T23:40:30.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.183+0000 7fd7b5600640 1 -- 172.21.15.94:0/3578648359 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v151) ==== 80+0+0 (secure 0 0 0) 0x7fd7a8017760 con 0x7fd7b806e7c0 2024-09-10T23:40:30.187 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:40:30.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.183+0000 7fd7bfa00640 1 -- 172.21.15.94:0/3578648359 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd794073000 msgr2=0x7fd7940754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:30.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.183+0000 7fd7bfa00640 1 --2- 172.21.15.94:0/3578648359 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd794073000 0x7fd7940754c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0000f70 tx=0x7fd7a0009040 comp rx=0 tx=0).stop 2024-09-10T23:40:30.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.183+0000 7fd7bfa00640 1 -- 172.21.15.94:0/3578648359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7b806e7c0 msgr2=0x7fd7b81f82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:30.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.183+0000 7fd7bfa00640 1 --2- 172.21.15.94:0/3578648359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7b806e7c0 0x7fd7b81f82e0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800a8a0 tx=0x7fd7a80a6940 comp rx=0 tx=0).stop 2024-09-10T23:40:30.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.183+0000 7fd7bfa00640 1 -- 172.21.15.94:0/3578648359 shutdown_connections 2024-09-10T23:40:30.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.183+0000 7fd7bfa00640 1 --2- 172.21.15.94:0/3578648359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7b80c64f0 0x7fd7b814b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.183+0000 7fd7bfa00640 1 --2- 172.21.15.94:0/3578648359 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd794073000 0x7fd7940754c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.183+0000 7fd7bfa00640 1 --2- 172.21.15.94:0/3578648359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7b80ce280 0x7fd7b8151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.183+0000 7fd7bfa00640 1 --2- 172.21.15.94:0/3578648359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7b806e7c0 0x7fd7b81f82e0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.183+0000 7fd7bfa00640 1 -- 172.21.15.94:0/3578648359 >> 172.21.15.94:0/3578648359 conn(0x7fd7b80cfaf0 msgr2=0x7fd7b8073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:30.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.183+0000 7fd7bfa00640 1 -- 172.21.15.94:0/3578648359 shutdown_connections 2024-09-10T23:40:30.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.187+0000 7fd7bfa00640 1 -- 172.21.15.94:0/3578648359 wait complete. 2024-09-10T23:40:30.208 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi181.front.sepia.ceph.com... 2024-09-10T23:40:30.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:30.208 DEBUG:teuthology.orchestra.run.smithi181:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T23:40:30.228 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[36226]: fuse finished with error 0 and tester_r 0 2024-09-10T23:40:30.231 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:40:30.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:30.231 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T23:40:30.316 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:40:30.316 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:40:30.316 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:40:30.329 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:40:30.329 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:40:30.329 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:40:30.329 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:40:30.368 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:40:30.369 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:40:30.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.507+0000 7fb6a6200640 1 -- 172.21.15.94:0/3809210851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a0160120 msgr2=0x7fb6a0160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:30.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.507+0000 7fb6a6200640 1 --2- 172.21.15.94:0/3809210851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a0160120 0x7fb6a0160590 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00b7c70 tx=0x7fb69801f580 comp rx=0 tx=0).stop 2024-09-10T23:40:30.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.507+0000 7fb6a6200640 1 -- 172.21.15.94:0/3809210851 shutdown_connections 2024-09-10T23:40:30.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.507+0000 7fb6a6200640 1 --2- 172.21.15.94:0/3809210851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a0160120 0x7fb6a0160590 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.507+0000 7fb6a6200640 1 --2- 172.21.15.94:0/3809210851 >> v2:172.21.15.181:3300/0 conn(0x7fb6a00d0010 0x7fb6a015fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.507+0000 7fb6a6200640 1 -- 172.21.15.94:0/3809210851 >> 172.21.15.94:0/3809210851 conn(0x7fb6a00d1bf0 msgr2=0x7fb6a00d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:30.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb6a6200640 1 -- 172.21.15.94:0/3809210851 shutdown_connections 2024-09-10T23:40:30.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb6a6200640 1 -- 172.21.15.94:0/3809210851 wait complete. 2024-09-10T23:40:30.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb6a6200640 1 Processor -- start 2024-09-10T23:40:30.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb6a6200640 1 -- start start 2024-09-10T23:40:30.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb6a6200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a00cfc30 0x7fb6a015d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:30.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb6a6200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a00d0010 0x7fb6a00b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:30.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb6a6200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6a0160120 0x7fb6a00c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:30.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb6a6200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a00b1c10 con 0x7fb6a00cfc30 2024-09-10T23:40:30.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb6a6200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a00b1a90 con 0x7fb6a0160120 2024-09-10T23:40:30.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb6a6200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a00b1d90 con 0x7fb6a00d0010 2024-09-10T23:40:30.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb69f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a00cfc30 0x7fb6a015d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:30.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb69f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a00cfc30 0x7fb6a015d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46524/0 (socket says 172.21.15.94:46524) 2024-09-10T23:40:30.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb69f400640 1 -- 172.21.15.94:0/2339542677 learned_addr learned my addr 172.21.15.94:0/2339542677 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:30.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb69fe00640 1 --2- 172.21.15.94:0/2339542677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6a0160120 0x7fb6a00c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:30.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb69ea00640 1 --2- 172.21.15.94:0/2339542677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a00d0010 0x7fb6a00b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:30.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb69f400640 1 -- 172.21.15.94:0/2339542677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a00d0010 msgr2=0x7fb6a00b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:30.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb69f400640 1 --2- 172.21.15.94:0/2339542677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a00d0010 0x7fb6a00b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb69f400640 1 -- 172.21.15.94:0/2339542677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6a0160120 msgr2=0x7fb6a00c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:30.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb69f400640 1 --2- 172.21.15.94:0/2339542677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6a0160120 0x7fb6a00c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb69f400640 1 -- 172.21.15.94:0/2339542677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a00b62c0 con 0x7fb6a00cfc30 2024-09-10T23:40:30.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb69f400640 1 --2- 172.21.15.94:0/2339542677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a00cfc30 0x7fb6a015d350 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb69000c480 tx=0x7fb69000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:30.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb697400640 1 -- 172.21.15.94:0/2339542677 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690012830 con 0x7fb6a00cfc30 2024-09-10T23:40:30.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb697400640 1 -- 172.21.15.94:0/2339542677 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb690017070 con 0x7fb6a00cfc30 2024-09-10T23:40:30.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb6a6200640 1 -- 172.21.15.94:0/2339542677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a00b65a0 con 0x7fb6a00cfc30 2024-09-10T23:40:30.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb697400640 1 -- 172.21.15.94:0/2339542677 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690012d80 con 0x7fb6a00cfc30 2024-09-10T23:40:30.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.511+0000 7fb6a6200640 1 -- 172.21.15.94:0/2339542677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a00b5fb0 con 0x7fb6a00cfc30 2024-09-10T23:40:30.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.515+0000 7fb6a6200640 1 -- 172.21.15.94:0/2339542677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a015fbe0 con 0x7fb6a00cfc30 2024-09-10T23:40:30.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.515+0000 7fb697400640 1 -- 172.21.15.94:0/2339542677 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7fb69001c460 con 0x7fb6a00cfc30 2024-09-10T23:40:30.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.515+0000 7fb697400640 1 --2- 172.21.15.94:0/2339542677 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb680073000 0x7fb6800754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:30.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.515+0000 7fb697400640 1 -- 172.21.15.94:0/2339542677 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7fb6900946b0 con 0x7fb6a00cfc30 2024-09-10T23:40:30.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.515+0000 7fb69ea00640 1 --2- 172.21.15.94:0/2339542677 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb680073000 0x7fb6800754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:30.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.515+0000 7fb69ea00640 1 --2- 172.21.15.94:0/2339542677 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb680073000 0x7fb6800754c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb68c005fd0 tx=0x7fb68c005db0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:30.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.519+0000 7fb697400640 1 -- 172.21.15.94:0/2339542677 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb69005ea70 con 0x7fb6a00cfc30 2024-09-10T23:40:30.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.719+0000 7fb6a6200640 1 -- 172.21.15.94:0/2339542677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6a0071750 con 0x7fb6a00cfc30 2024-09-10T23:40:30.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.719+0000 7fb697400640 1 -- 172.21.15.94:0/2339542677 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5249 (secure 0 0 0) 0x7fb69005e370 con 0x7fb6a00cfc30 2024-09-10T23:40:30.722 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:30.722 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":108,"btime":"2024-09-10T23:40:18:845040+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8158,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1859150794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":1859150794},{"type":"v1","addr":"172.21.15.94:6835","nonce":1859150794}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8173,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/3891720008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3891720008},{"type":"v1","addr":"172.21.15.181:6801","nonce":3891720008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8787,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/4040262257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":4040262257},{"type":"v1","addr":"172.21.15.94:6834","nonce":4040262257}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:40:02.727036+0000","modified":"2024-09-10T23:40:05.749707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":197120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_197120":{"gid":197120,"name":"b","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/2758853868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2758853868},{"type":"v1","addr":"172.21.15.181:6835","nonce":2758853868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":197120,"qdb_cluster":[197120]},"id":12}]} 2024-09-10T23:40:30.722 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 108 2024-09-10T23:40:30.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.719+0000 7fb6a6200640 1 -- 172.21.15.94:0/2339542677 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb680073000 msgr2=0x7fb6800754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:30.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.719+0000 7fb6a6200640 1 --2- 172.21.15.94:0/2339542677 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb680073000 0x7fb6800754c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb68c005fd0 tx=0x7fb68c005db0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.719+0000 7fb6a6200640 1 -- 172.21.15.94:0/2339542677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a00cfc30 msgr2=0x7fb6a015d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:30.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.719+0000 7fb6a6200640 1 --2- 172.21.15.94:0/2339542677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a00cfc30 0x7fb6a015d350 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb69000c480 tx=0x7fb69000c940 comp rx=0 tx=0).stop 2024-09-10T23:40:30.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.719+0000 7fb6a6200640 1 -- 172.21.15.94:0/2339542677 shutdown_connections 2024-09-10T23:40:30.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.719+0000 7fb6a6200640 1 --2- 172.21.15.94:0/2339542677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6a0160120 0x7fb6a00c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.719+0000 7fb6a6200640 1 --2- 172.21.15.94:0/2339542677 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb680073000 0x7fb6800754c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.719+0000 7fb6a6200640 1 --2- 172.21.15.94:0/2339542677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6a00d0010 0x7fb6a00b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.719+0000 7fb6a6200640 1 --2- 172.21.15.94:0/2339542677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6a00cfc30 0x7fb6a015d350 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.719+0000 7fb6a6200640 1 -- 172.21.15.94:0/2339542677 >> 172.21.15.94:0/2339542677 conn(0x7fb6a00d1bf0 msgr2=0x7fb6a00b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:30.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.719+0000 7fb6a6200640 1 -- 172.21.15.94:0/2339542677 shutdown_connections 2024-09-10T23:40:30.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.719+0000 7fb6a6200640 1 -- 172.21.15.94:0/2339542677 wait complete. 2024-09-10T23:40:30.745 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:40:30.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.883+0000 7f7e14c00640 1 -- 172.21.15.94:0/1876379411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e1015ead0 msgr2=0x7f7e10073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:30.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.883+0000 7f7e14c00640 1 --2- 172.21.15.94:0/1876379411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e1015ead0 0x7f7e10073720 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f7e100d0a80 tx=0x7f7e0801c540 comp rx=0 tx=0).stop 2024-09-10T23:40:30.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.883+0000 7f7e14c00640 1 -- 172.21.15.94:0/1876379411 shutdown_connections 2024-09-10T23:40:30.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.883+0000 7f7e14c00640 1 --2- 172.21.15.94:0/1876379411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e1015ead0 0x7f7e10073720 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.883+0000 7f7e14c00640 1 --2- 172.21.15.94:0/1876379411 >> v2:172.21.15.181:3300/0 conn(0x7f7e100d01f0 0x7f7e1015e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.883+0000 7f7e14c00640 1 --2- 172.21.15.94:0/1876379411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e100cf3c0 0x7f7e100cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.883+0000 7f7e14c00640 1 -- 172.21.15.94:0/1876379411 >> 172.21.15.94:0/1876379411 conn(0x7f7e1006cdf0 msgr2=0x7f7e1006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:30.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.883+0000 7f7e14c00640 1 -- 172.21.15.94:0/1876379411 shutdown_connections 2024-09-10T23:40:30.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.883+0000 7f7e14c00640 1 -- 172.21.15.94:0/1876379411 wait complete. 2024-09-10T23:40:30.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.883+0000 7f7e14c00640 1 Processor -- start 2024-09-10T23:40:30.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.883+0000 7f7e14c00640 1 -- start start 2024-09-10T23:40:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e14c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e100cf3c0 0x7f7e1014b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e14c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e100d01f0 0x7f7e101ec430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e14c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e1015ead0 0x7f7e1014c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e14c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1006fb20 con 0x7f7e1015ead0 2024-09-10T23:40:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e14c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1006f9a0 con 0x7f7e100d01f0 2024-09-10T23:40:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e14c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f7e1006fca0 con 0x7f7e100cf3c0 2024-09-10T23:40:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e0e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e1015ead0 0x7f7e1014c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e0e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e1015ead0 0x7f7e1014c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46546/0 (socket says 172.21.15.94:46546) 2024-09-10T23:40:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e0e800640 1 -- 172.21.15.94:0/3019732633 learned_addr learned my addr 172.21.15.94:0/3019732633 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e0de00640 1 --2- 172.21.15.94:0/3019732633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e100cf3c0 0x7f7e1014b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:30.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e0d400640 1 --2- 172.21.15.94:0/3019732633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e100d01f0 0x7f7e101ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:30.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e0e800640 1 -- 172.21.15.94:0/3019732633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e100cf3c0 msgr2=0x7f7e1014b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:30.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e0e800640 1 --2- 172.21.15.94:0/3019732633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e100cf3c0 0x7f7e1014b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e0e800640 1 -- 172.21.15.94:0/3019732633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e100d01f0 msgr2=0x7f7e101ec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:30.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e0e800640 1 --2- 172.21.15.94:0/3019732633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e100d01f0 0x7f7e101ec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:30.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e0e800640 1 -- 172.21.15.94:0/3019732633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1014e7c0 con 0x7f7e1015ead0 2024-09-10T23:40:30.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e0e800640 1 --2- 172.21.15.94:0/3019732633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e1015ead0 0x7f7e1014c010 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f7e0801abf0 tx=0x7f7e080a8590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:30.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e06000640 1 -- 172.21.15.94:0/3019732633 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08004240 con 0x7f7e1015ead0 2024-09-10T23:40:30.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e06000640 1 -- 172.21.15.94:0/3019732633 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e080044d0 con 0x7f7e1015ead0 2024-09-10T23:40:30.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e14c00640 1 -- 172.21.15.94:0/3019732633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1014eaa0 con 0x7f7e1015ead0 2024-09-10T23:40:30.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e06000640 1 -- 172.21.15.94:0/3019732633 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e080052d0 con 0x7f7e1015ead0 2024-09-10T23:40:30.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e14c00640 1 -- 172.21.15.94:0/3019732633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e1014f060 con 0x7f7e1015ead0 2024-09-10T23:40:30.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e14c00640 1 -- 172.21.15.94:0/3019732633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e1015e590 con 0x7f7e1015ead0 2024-09-10T23:40:30.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.887+0000 7f7e06000640 1 -- 172.21.15.94:0/3019732633 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f7e080c1030 con 0x7f7e1015ead0 2024-09-10T23:40:30.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.891+0000 7f7e06000640 1 --2- 172.21.15.94:0/3019732633 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7df0073000 0x7f7df00754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:30.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.891+0000 7f7e06000640 1 -- 172.21.15.94:0/3019732633 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f7e0812bdd0 con 0x7f7e1015ead0 2024-09-10T23:40:30.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.891+0000 7f7e0de00640 1 --2- 172.21.15.94:0/3019732633 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7df0073000 0x7f7df00754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:30.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.891+0000 7f7e0de00640 1 --2- 172.21.15.94:0/3019732633 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7df0073000 0x7f7df00754c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7e1014ca60 tx=0x7f7e00009650 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:30.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:30.895+0000 7f7e06000640 1 -- 172.21.15.94:0/3019732633 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7e080ac030 con 0x7f7e1015ead0 2024-09-10T23:40:31.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.099+0000 7f7e14c00640 1 -- 172.21.15.94:0/3019732633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e10069850 con 0x7f7e1015ead0 2024-09-10T23:40:31.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.099+0000 7f7e06000640 1 -- 172.21.15.94:0/3019732633 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5249 (secure 0 0 0) 0x7f7e080164b0 con 0x7f7e1015ead0 2024-09-10T23:40:31.102 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:31.102 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":108,"btime":"2024-09-10T23:40:18:845040+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8158,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1859150794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":1859150794},{"type":"v1","addr":"172.21.15.94:6835","nonce":1859150794}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8173,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/3891720008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3891720008},{"type":"v1","addr":"172.21.15.181:6801","nonce":3891720008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8787,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/4040262257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":4040262257},{"type":"v1","addr":"172.21.15.94:6834","nonce":4040262257}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:40:02.727036+0000","modified":"2024-09-10T23:40:05.749707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":197120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_197120":{"gid":197120,"name":"b","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/2758853868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2758853868},{"type":"v1","addr":"172.21.15.181:6835","nonce":2758853868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":197120,"qdb_cluster":[197120]},"id":12}]} 2024-09-10T23:40:31.102 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 108 2024-09-10T23:40:31.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.099+0000 7f7e14c00640 1 -- 172.21.15.94:0/3019732633 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7df0073000 msgr2=0x7f7df00754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:31.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.099+0000 7f7e14c00640 1 --2- 172.21.15.94:0/3019732633 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7df0073000 0x7f7df00754c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7e1014ca60 tx=0x7f7e00009650 comp rx=0 tx=0).stop 2024-09-10T23:40:31.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.099+0000 7f7e14c00640 1 -- 172.21.15.94:0/3019732633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e1015ead0 msgr2=0x7f7e1014c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:31.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.099+0000 7f7e14c00640 1 --2- 172.21.15.94:0/3019732633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e1015ead0 0x7f7e1014c010 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f7e0801abf0 tx=0x7f7e080a8590 comp rx=0 tx=0).stop 2024-09-10T23:40:31.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.099+0000 7f7e14c00640 1 -- 172.21.15.94:0/3019732633 shutdown_connections 2024-09-10T23:40:31.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.099+0000 7f7e14c00640 1 --2- 172.21.15.94:0/3019732633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e100d01f0 0x7f7e101ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.099+0000 7f7e14c00640 1 --2- 172.21.15.94:0/3019732633 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7df0073000 0x7f7df00754c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.099+0000 7f7e14c00640 1 --2- 172.21.15.94:0/3019732633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e1015ead0 0x7f7e1014c010 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.099+0000 7f7e14c00640 1 --2- 172.21.15.94:0/3019732633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e100cf3c0 0x7f7e1014b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.099+0000 7f7e14c00640 1 -- 172.21.15.94:0/3019732633 >> 172.21.15.94:0/3019732633 conn(0x7f7e1006cdf0 msgr2=0x7f7e10073400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:31.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.099+0000 7f7e14c00640 1 -- 172.21.15.94:0/3019732633 shutdown_connections 2024-09-10T23:40:31.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.099+0000 7f7e14c00640 1 -- 172.21.15.94:0/3019732633 wait complete. 2024-09-10T23:40:31.124 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:40:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.263+0000 7f1b13800640 1 -- 172.21.15.94:0/328253604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af0004d50 msgr2=0x7f1af00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.263+0000 7f1b13800640 1 --2- 172.21.15.94:0/328253604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af0004d50 0x7f1af00051d0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0d26a0 tx=0x7f1b0401d590 comp rx=0 tx=0).stop 2024-09-10T23:40:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.263+0000 7f1b11800640 1 -- 172.21.15.94:0/328253604 reap_dead start 2024-09-10T23:40:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.263+0000 7f1b13800640 1 -- 172.21.15.94:0/328253604 shutdown_connections 2024-09-10T23:40:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.263+0000 7f1b13800640 1 -- 172.21.15.94:0/328253604 >> 172.21.15.94:0/328253604 conn(0x7f1b0c09aa70 msgr2=0x7f1b0c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.263+0000 7f1b13800640 1 -- 172.21.15.94:0/328253604 shutdown_connections 2024-09-10T23:40:31.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.263+0000 7f1b13800640 1 -- 172.21.15.94:0/328253604 wait complete. 2024-09-10T23:40:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.263+0000 7f1b13800640 1 Processor -- start 2024-09-10T23:40:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.263+0000 7f1b13800640 1 -- start start 2024-09-10T23:40:31.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b13800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af0004d50 0x7f1b0c152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:31.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b13800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b0c0b5490 0x7f1b0c154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:31.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b13800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b0c1550e0 0x7f1b0c1f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:31.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b13800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c164ce0 con 0x7f1af0004d50 2024-09-10T23:40:31.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b13800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c164b60 con 0x7f1b0c1550e0 2024-09-10T23:40:31.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b13800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1b0c164e60 con 0x7f1b0c0b5490 2024-09-10T23:40:31.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b10e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af0004d50 0x7f1b0c152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:31.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b10e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af0004d50 0x7f1b0c152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46560/0 (socket says 172.21.15.94:46560) 2024-09-10T23:40:31.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b10e00640 1 -- 172.21.15.94:0/835868632 learned_addr learned my addr 172.21.15.94:0/835868632 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:31.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b0be00640 1 --2- 172.21.15.94:0/835868632 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b0c0b5490 0x7f1b0c154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:31.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b11800640 1 --2- 172.21.15.94:0/835868632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b0c1550e0 0x7f1b0c1f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:31.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b10e00640 1 -- 172.21.15.94:0/835868632 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b0c0b5490 msgr2=0x7f1b0c154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:31.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b10e00640 1 --2- 172.21.15.94:0/835868632 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b0c0b5490 0x7f1b0c154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b10e00640 1 -- 172.21.15.94:0/835868632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b0c1550e0 msgr2=0x7f1b0c1f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:31.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b10e00640 1 --2- 172.21.15.94:0/835868632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b0c1550e0 0x7f1b0c1f5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b10e00640 1 -- 172.21.15.94:0/835868632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0c0b0e50 con 0x7f1af0004d50 2024-09-10T23:40:31.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b11800640 1 --2- 172.21.15.94:0/835868632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b0c1550e0 0x7f1b0c1f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:40:31.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b10e00640 1 --2- 172.21.15.94:0/835868632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af0004d50 0x7f1b0c152ed0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f1afc003fc0 tx=0x7f1afc001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:31.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b08c00640 1 -- 172.21.15.94:0/835868632 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc015070 con 0x7f1af0004d50 2024-09-10T23:40:31.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b08c00640 1 -- 172.21.15.94:0/835868632 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1afc011900 con 0x7f1af0004d50 2024-09-10T23:40:31.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b13800640 1 -- 172.21.15.94:0/835868632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0c153410 con 0x7f1af0004d50 2024-09-10T23:40:31.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b08c00640 1 -- 172.21.15.94:0/835868632 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc011d50 con 0x7f1af0004d50 2024-09-10T23:40:31.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b13800640 1 -- 172.21.15.94:0/835868632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b0c0cc710 con 0x7f1af0004d50 2024-09-10T23:40:31.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b13800640 1 -- 172.21.15.94:0/835868632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b0c15c9d0 con 0x7f1af0004d50 2024-09-10T23:40:31.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.267+0000 7f1b08c00640 1 -- 172.21.15.94:0/835868632 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f1afc010420 con 0x7f1af0004d50 2024-09-10T23:40:31.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.271+0000 7f1b08c00640 1 --2- 172.21.15.94:0/835868632 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1aec073030 0x7f1aec0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:31.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.271+0000 7f1b08c00640 1 -- 172.21.15.94:0/835868632 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f1afc0956c0 con 0x7f1af0004d50 2024-09-10T23:40:31.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.271+0000 7f1b0be00640 1 --2- 172.21.15.94:0/835868632 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1aec073030 0x7f1aec0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:31.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.271+0000 7f1b0be00640 1 --2- 172.21.15.94:0/835868632 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1aec073030 0x7f1aec0754f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c1f5e20 tx=0x7f1af8006730 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:31.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.275+0000 7f1b08c00640 1 -- 172.21.15.94:0/835868632 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1afc05f9b0 con 0x7f1af0004d50 2024-09-10T23:40:31.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.443+0000 7f1b13800640 1 -- 172.21.15.94:0/835868632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b0c1554c0 con 0x7f1af0004d50 2024-09-10T23:40:31.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.443+0000 7f1b08c00640 1 -- 172.21.15.94:0/835868632 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) ==== 74+0+16715 (secure 0 0 0) 0x7f1afc05f2b0 con 0x7f1af0004d50 2024-09-10T23:40:31.446 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:31.447 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":151,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:40:04.337069+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:40:00.461748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-10T23:40:01.690823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:40:31.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.443+0000 7f1b13800640 1 -- 172.21.15.94:0/835868632 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1aec073030 msgr2=0x7f1aec0754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:31.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.443+0000 7f1b13800640 1 --2- 172.21.15.94:0/835868632 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1aec073030 0x7f1aec0754f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c1f5e20 tx=0x7f1af8006730 comp rx=0 tx=0).stop 2024-09-10T23:40:31.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.443+0000 7f1b13800640 1 -- 172.21.15.94:0/835868632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af0004d50 msgr2=0x7f1b0c152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:31.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.443+0000 7f1b13800640 1 --2- 172.21.15.94:0/835868632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af0004d50 0x7f1b0c152ed0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f1afc003fc0 tx=0x7f1afc001800 comp rx=0 tx=0).stop 2024-09-10T23:40:31.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.443+0000 7f1b13800640 1 -- 172.21.15.94:0/835868632 shutdown_connections 2024-09-10T23:40:31.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.443+0000 7f1b13800640 1 --2- 172.21.15.94:0/835868632 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b0c1550e0 0x7f1b0c1f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.443+0000 7f1b13800640 1 --2- 172.21.15.94:0/835868632 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1aec073030 0x7f1aec0754f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.443+0000 7f1b13800640 1 --2- 172.21.15.94:0/835868632 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b0c0b5490 0x7f1b0c154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.443+0000 7f1b13800640 1 --2- 172.21.15.94:0/835868632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af0004d50 0x7f1b0c152ed0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.443+0000 7f1b13800640 1 -- 172.21.15.94:0/835868632 >> 172.21.15.94:0/835868632 conn(0x7f1b0c09aa70 msgr2=0x7f1b0c0af420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:31.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.443+0000 7f1b13800640 1 -- 172.21.15.94:0/835868632 shutdown_connections 2024-09-10T23:40:31.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.443+0000 7f1b13800640 1 -- 172.21.15.94:0/835868632 wait complete. 2024-09-10T23:40:31.468 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T23:40:31.468 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:40:31.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.603+0000 7fb78ac00640 1 -- 172.21.15.94:0/1491107640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb784160120 msgr2=0x7fb784160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:31.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.603+0000 7fb78ac00640 1 --2- 172.21.15.94:0/1491107640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb784160120 0x7fb784160590 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fb7840b7c70 tx=0x7fb77c01f580 comp rx=0 tx=0).stop 2024-09-10T23:40:31.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.603+0000 7fb78ac00640 1 -- 172.21.15.94:0/1491107640 shutdown_connections 2024-09-10T23:40:31.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.603+0000 7fb78ac00640 1 --2- 172.21.15.94:0/1491107640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb784160120 0x7fb784160590 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.603+0000 7fb78ac00640 1 --2- 172.21.15.94:0/1491107640 >> v2:172.21.15.181:3300/0 conn(0x7fb7840d0010 0x7fb78415fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.603+0000 7fb78ac00640 1 -- 172.21.15.94:0/1491107640 >> 172.21.15.94:0/1491107640 conn(0x7fb7840d1bf0 msgr2=0x7fb7840d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:31.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.603+0000 7fb78ac00640 1 -- 172.21.15.94:0/1491107640 shutdown_connections 2024-09-10T23:40:31.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.603+0000 7fb78ac00640 1 -- 172.21.15.94:0/1491107640 wait complete. 2024-09-10T23:40:31.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb78ac00640 1 Processor -- start 2024-09-10T23:40:31.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb78ac00640 1 -- start start 2024-09-10T23:40:31.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb78ac00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7840cfc30 0x7fb78414c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:31.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb78ac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7840d0010 0x7fb7840b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:31.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb78ac00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb784160120 0x7fb7841f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:31.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb78ac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb7840b1c10 con 0x7fb7840d0010 2024-09-10T23:40:31.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb78ac00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb7840b1a90 con 0x7fb7840cfc30 2024-09-10T23:40:31.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb78ac00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb7840b1d90 con 0x7fb784160120 2024-09-10T23:40:31.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb783400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7840d0010 0x7fb7840b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:31.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb783400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7840d0010 0x7fb7840b6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46572/0 (socket says 172.21.15.94:46572) 2024-09-10T23:40:31.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb783400640 1 -- 172.21.15.94:0/1990301660 learned_addr learned my addr 172.21.15.94:0/1990301660 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:31.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb788c00640 1 --2- 172.21.15.94:0/1990301660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb784160120 0x7fb7841f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:31.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb783e00640 1 --2- 172.21.15.94:0/1990301660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7840cfc30 0x7fb78414c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb783400640 1 -- 172.21.15.94:0/1990301660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb784160120 msgr2=0x7fb7841f7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb783400640 1 --2- 172.21.15.94:0/1990301660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb784160120 0x7fb7841f7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb783400640 1 -- 172.21.15.94:0/1990301660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7840cfc30 msgr2=0x7fb78414c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:31.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb783400640 1 --2- 172.21.15.94:0/1990301660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7840cfc30 0x7fb78414c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb783400640 1 -- 172.21.15.94:0/1990301660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7840b6f00 con 0x7fb7840d0010 2024-09-10T23:40:31.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb783400640 1 --2- 172.21.15.94:0/1990301660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7840d0010 0x7fb7840b6480 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fb77000b550 tx=0x7fb77000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:31.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb77be00640 1 -- 172.21.15.94:0/1990301660 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770014070 con 0x7fb7840d0010 2024-09-10T23:40:31.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb77be00640 1 -- 172.21.15.94:0/1990301660 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb770010980 con 0x7fb7840d0010 2024-09-10T23:40:31.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb77be00640 1 -- 172.21.15.94:0/1990301660 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770010dd0 con 0x7fb7840d0010 2024-09-10T23:40:31.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb78ac00640 1 -- 172.21.15.94:0/1990301660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7840b71e0 con 0x7fb7840d0010 2024-09-10T23:40:31.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.607+0000 7fb78ac00640 1 -- 172.21.15.94:0/1990301660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb78414c560 con 0x7fb7840d0010 2024-09-10T23:40:31.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.611+0000 7fb78ac00640 1 -- 172.21.15.94:0/1990301660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb78415fbe0 con 0x7fb7840d0010 2024-09-10T23:40:31.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.611+0000 7fb77be00640 1 -- 172.21.15.94:0/1990301660 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7fb77000f3c0 con 0x7fb7840d0010 2024-09-10T23:40:31.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.611+0000 7fb77be00640 1 --2- 172.21.15.94:0/1990301660 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb764073000 0x7fb7640754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:31.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.611+0000 7fb77be00640 1 -- 172.21.15.94:0/1990301660 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7fb770091e30 con 0x7fb7840d0010 2024-09-10T23:40:31.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.611+0000 7fb783e00640 1 --2- 172.21.15.94:0/1990301660 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb764073000 0x7fb7640754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:31.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.611+0000 7fb783e00640 1 --2- 172.21.15.94:0/1990301660 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb764073000 0x7fb7640754c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb774007fd0 tx=0x7fb774009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.615+0000 7fb77be00640 1 -- 172.21.15.94:0/1990301660 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb77005e6f0 con 0x7fb7840d0010 2024-09-10T23:40:31.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.815+0000 7fb78ac00640 1 -- 172.21.15.94:0/1990301660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb78415f7b0 con 0x7fb7840d0010 2024-09-10T23:40:31.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.815+0000 7fb77be00640 1 -- 172.21.15.94:0/1990301660 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5249 (secure 0 0 0) 0x7fb770063120 con 0x7fb7840d0010 2024-09-10T23:40:31.819 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:31.819 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":108,"btime":"2024-09-10T23:40:18:845040+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8158,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1859150794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":1859150794},{"type":"v1","addr":"172.21.15.94:6835","nonce":1859150794}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8173,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/3891720008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3891720008},{"type":"v1","addr":"172.21.15.181:6801","nonce":3891720008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8787,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/4040262257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":4040262257},{"type":"v1","addr":"172.21.15.94:6834","nonce":4040262257}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:40:02.727036+0000","modified":"2024-09-10T23:40:05.749707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":197120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_197120":{"gid":197120,"name":"b","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/2758853868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2758853868},{"type":"v1","addr":"172.21.15.181:6835","nonce":2758853868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":197120,"qdb_cluster":[197120]},"id":12}]} 2024-09-10T23:40:31.819 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 108 2024-09-10T23:40:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.819+0000 7fb78ac00640 1 -- 172.21.15.94:0/1990301660 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb764073000 msgr2=0x7fb7640754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.819+0000 7fb78ac00640 1 --2- 172.21.15.94:0/1990301660 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb764073000 0x7fb7640754c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb774007fd0 tx=0x7fb774009040 comp rx=0 tx=0).stop 2024-09-10T23:40:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.819+0000 7fb78ac00640 1 -- 172.21.15.94:0/1990301660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7840d0010 msgr2=0x7fb7840b6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.819+0000 7fb78ac00640 1 --2- 172.21.15.94:0/1990301660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7840d0010 0x7fb7840b6480 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fb77000b550 tx=0x7fb77000ba10 comp rx=0 tx=0).stop 2024-09-10T23:40:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.819+0000 7fb78ac00640 1 -- 172.21.15.94:0/1990301660 shutdown_connections 2024-09-10T23:40:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.819+0000 7fb78ac00640 1 --2- 172.21.15.94:0/1990301660 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb764073000 0x7fb7640754c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.819+0000 7fb78ac00640 1 --2- 172.21.15.94:0/1990301660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb784160120 0x7fb7841f7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.819+0000 7fb78ac00640 1 --2- 172.21.15.94:0/1990301660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7840d0010 0x7fb7840b6480 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.819+0000 7fb78ac00640 1 --2- 172.21.15.94:0/1990301660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7840cfc30 0x7fb78414c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.819+0000 7fb78ac00640 1 -- 172.21.15.94:0/1990301660 >> 172.21.15.94:0/1990301660 conn(0x7fb7840d1bf0 msgr2=0x7fb78406de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.819+0000 7fb78ac00640 1 -- 172.21.15.94:0/1990301660 shutdown_connections 2024-09-10T23:40:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.819+0000 7fb78ac00640 1 -- 172.21.15.94:0/1990301660 wait complete. 2024-09-10T23:40:31.842 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:40:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.979+0000 7f157ba00640 1 -- 172.21.15.94:0/2760443492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15740d03c0 msgr2=0x7f15740cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:31.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.979+0000 7f157ba00640 1 --2- 172.21.15.94:0/2760443492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15740d03c0 0x7f15740cddb0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f156c009c80 tx=0x7f156c01cce0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.979+0000 7f157ba00640 1 -- 172.21.15.94:0/2760443492 shutdown_connections 2024-09-10T23:40:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.979+0000 7f157ba00640 1 --2- 172.21.15.94:0/2760443492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15740d03c0 0x7f15740cddb0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.979+0000 7f157ba00640 1 --2- 172.21.15.94:0/2760443492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f157415ba50 0x7f15740b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.979+0000 7f157ba00640 1 -- 172.21.15.94:0/2760443492 >> 172.21.15.94:0/2760443492 conn(0x7f157406cdf0 msgr2=0x7f157406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.979+0000 7f157ba00640 1 -- 172.21.15.94:0/2760443492 shutdown_connections 2024-09-10T23:40:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.979+0000 7f157ba00640 1 -- 172.21.15.94:0/2760443492 wait complete. 2024-09-10T23:40:31.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.979+0000 7f157ba00640 1 Processor -- start 2024-09-10T23:40:31.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.979+0000 7f157ba00640 1 -- start start 2024-09-10T23:40:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f157ba00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15740b4c40 0x7f1574140510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f157ba00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15740d03c0 0x7f157414ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f157ba00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157415ba50 0x7f157414d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f157ba00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1574168be0 con 0x7f15740d03c0 2024-09-10T23:40:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f157ba00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1574168a60 con 0x7f157415ba50 2024-09-10T23:40:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f157ba00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1574168d60 con 0x7f15740b4c40 2024-09-10T23:40:31.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f1573e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15740d03c0 0x7f157414ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f1573e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15740d03c0 0x7f157414ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46594/0 (socket says 172.21.15.94:46594) 2024-09-10T23:40:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f1573e00640 1 -- 172.21.15.94:0/3999009868 learned_addr learned my addr 172.21.15.94:0/3999009868 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f1579000640 1 --2- 172.21.15.94:0/3999009868 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15740b4c40 0x7f1574140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f1579a00640 1 --2- 172.21.15.94:0/3999009868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157415ba50 0x7f157414d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f1573e00640 1 -- 172.21.15.94:0/3999009868 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15740b4c40 msgr2=0x7f1574140510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f1573e00640 1 --2- 172.21.15.94:0/3999009868 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15740b4c40 0x7f1574140510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f1573e00640 1 -- 172.21.15.94:0/3999009868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157415ba50 msgr2=0x7f157414d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f1573e00640 1 --2- 172.21.15.94:0/3999009868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157415ba50 0x7f157414d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f1573e00640 1 -- 172.21.15.94:0/3999009868 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1574140a50 con 0x7f15740d03c0 2024-09-10T23:40:31.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f1573e00640 1 --2- 172.21.15.94:0/3999009868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15740d03c0 0x7f157414ce90 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f156000c410 tx=0x7f156000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:31.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f1570c00640 1 -- 172.21.15.94:0/3999009868 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560017070 con 0x7f15740d03c0 2024-09-10T23:40:31.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f1570c00640 1 -- 172.21.15.94:0/3999009868 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15600053c0 con 0x7f15740d03c0 2024-09-10T23:40:31.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f157ba00640 1 -- 172.21.15.94:0/3999009868 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1574163790 con 0x7f15740d03c0 2024-09-10T23:40:31.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f1570c00640 1 -- 172.21.15.94:0/3999009868 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560003880 con 0x7f15740d03c0 2024-09-10T23:40:31.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.983+0000 7f157ba00640 1 -- 172.21.15.94:0/3999009868 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1574163d50 con 0x7f15740d03c0 2024-09-10T23:40:31.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.987+0000 7f157ba00640 1 -- 172.21.15.94:0/3999009868 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15740cddb0 con 0x7f15740d03c0 2024-09-10T23:40:31.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.987+0000 7f1570c00640 1 -- 172.21.15.94:0/3999009868 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f1560003a20 con 0x7f15740d03c0 2024-09-10T23:40:31.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.987+0000 7f1570c00640 1 --2- 172.21.15.94:0/3999009868 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1554073000 0x7f15540754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:31.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.987+0000 7f1570c00640 1 -- 172.21.15.94:0/3999009868 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f1560091360 con 0x7f15740d03c0 2024-09-10T23:40:31.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.987+0000 7f1579000640 1 --2- 172.21.15.94:0/3999009868 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1554073000 0x7f15540754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:31.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.987+0000 7f1579000640 1 --2- 172.21.15.94:0/3999009868 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1554073000 0x7f15540754c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1574144dc0 tx=0x7f15680091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:31.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:31.991+0000 7f1570c00640 1 -- 172.21.15.94:0/3999009868 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f156005dc20 con 0x7f15740d03c0 2024-09-10T23:40:32.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.191+0000 7f157ba00640 1 -- 172.21.15.94:0/3999009868 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f157406f770 con 0x7f15740d03c0 2024-09-10T23:40:32.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.191+0000 7f1570c00640 1 -- 172.21.15.94:0/3999009868 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5249 (secure 0 0 0) 0x7f1560062650 con 0x7f15740d03c0 2024-09-10T23:40:32.194 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:32.194 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":108,"btime":"2024-09-10T23:40:18:845040+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8158,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1859150794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":1859150794},{"type":"v1","addr":"172.21.15.94:6835","nonce":1859150794}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8173,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/3891720008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3891720008},{"type":"v1","addr":"172.21.15.181:6801","nonce":3891720008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8787,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/4040262257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":4040262257},{"type":"v1","addr":"172.21.15.94:6834","nonce":4040262257}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:40:02.727036+0000","modified":"2024-09-10T23:40:05.749707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":197120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_197120":{"gid":197120,"name":"b","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/2758853868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2758853868},{"type":"v1","addr":"172.21.15.181:6835","nonce":2758853868}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":197120,"qdb_cluster":[197120]},"id":12}]} 2024-09-10T23:40:32.195 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 108 2024-09-10T23:40:32.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.191+0000 7f157ba00640 1 -- 172.21.15.94:0/3999009868 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1554073000 msgr2=0x7f15540754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:32.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.191+0000 7f157ba00640 1 --2- 172.21.15.94:0/3999009868 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1554073000 0x7f15540754c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1574144dc0 tx=0x7f15680091e0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.191+0000 7f157ba00640 1 -- 172.21.15.94:0/3999009868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15740d03c0 msgr2=0x7f157414ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:32.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.191+0000 7f157ba00640 1 --2- 172.21.15.94:0/3999009868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15740d03c0 0x7f157414ce90 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f156000c410 tx=0x7f156000c8d0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.191+0000 7f157ba00640 1 -- 172.21.15.94:0/3999009868 shutdown_connections 2024-09-10T23:40:32.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.191+0000 7f157ba00640 1 --2- 172.21.15.94:0/3999009868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157415ba50 0x7f157414d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.191+0000 7f157ba00640 1 --2- 172.21.15.94:0/3999009868 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1554073000 0x7f15540754c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.191+0000 7f157ba00640 1 --2- 172.21.15.94:0/3999009868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15740d03c0 0x7f157414ce90 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.191+0000 7f157ba00640 1 --2- 172.21.15.94:0/3999009868 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15740b4c40 0x7f1574140510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.191+0000 7f157ba00640 1 -- 172.21.15.94:0/3999009868 >> 172.21.15.94:0/3999009868 conn(0x7f157406cdf0 msgr2=0x7f157406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:32.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.191+0000 7f157ba00640 1 -- 172.21.15.94:0/3999009868 shutdown_connections 2024-09-10T23:40:32.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.191+0000 7f157ba00640 1 -- 172.21.15.94:0/3999009868 wait complete. 2024-09-10T23:40:32.215 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:40:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.355+0000 7f8b62200640 1 -- 172.21.15.94:0/2634717991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b440051d0 msgr2=0x7f8b4400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.355+0000 7f8b62200640 1 --2- 172.21.15.94:0/2634717991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b440051d0 0x7f8b4400b9d0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400e400 tx=0x7f8b5401d7d0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.355+0000 7f8b61800640 1 -- 172.21.15.94:0/2634717991 reap_dead start 2024-09-10T23:40:32.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.355+0000 7f8b62200640 1 -- 172.21.15.94:0/2634717991 shutdown_connections 2024-09-10T23:40:32.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.355+0000 7f8b62200640 1 -- 172.21.15.94:0/2634717991 >> 172.21.15.94:0/2634717991 conn(0x7f8b5c09aa70 msgr2=0x7f8b5c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:32.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.355+0000 7f8b62200640 1 -- 172.21.15.94:0/2634717991 shutdown_connections 2024-09-10T23:40:32.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.355+0000 7f8b62200640 1 -- 172.21.15.94:0/2634717991 wait complete. 2024-09-10T23:40:32.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.355+0000 7f8b62200640 1 Processor -- start 2024-09-10T23:40:32.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.355+0000 7f8b62200640 1 -- start start 2024-09-10T23:40:32.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b62200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b440051d0 0x7f8b5c0b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:32.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b62200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b5c0b5850 0x7f8b5c151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:32.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b62200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5c0c4700 0x7f8b5c0c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:32.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b62200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c0b34d0 con 0x7f8b5c0b5850 2024-09-10T23:40:32.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b62200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c0b3350 con 0x7f8b5c0c4700 2024-09-10T23:40:32.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b62200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8b5c0d01e0 con 0x7f8b440051d0 2024-09-10T23:40:32.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b5be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b5c0b5850 0x7f8b5c151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:32.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b5be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b5c0b5850 0x7f8b5c151a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46612/0 (socket says 172.21.15.94:46612) 2024-09-10T23:40:32.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b5be00640 1 -- 172.21.15.94:0/4137239848 learned_addr learned my addr 172.21.15.94:0/4137239848 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:32.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b60e00640 1 --2- 172.21.15.94:0/4137239848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b440051d0 0x7f8b5c0b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:32.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b61800640 1 --2- 172.21.15.94:0/4137239848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5c0c4700 0x7f8b5c0c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:32.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b5be00640 1 -- 172.21.15.94:0/4137239848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b440051d0 msgr2=0x7f8b5c0b6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:32.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b5be00640 1 --2- 172.21.15.94:0/4137239848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b440051d0 0x7f8b5c0b6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b5be00640 1 -- 172.21.15.94:0/4137239848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5c0c4700 msgr2=0x7f8b5c0c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:32.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b5be00640 1 --2- 172.21.15.94:0/4137239848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5c0c4700 0x7f8b5c0c4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b5be00640 1 -- 172.21.15.94:0/4137239848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5c151fc0 con 0x7f8b5c0b5850 2024-09-10T23:40:32.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b5be00640 1 --2- 172.21.15.94:0/4137239848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b5c0b5850 0x7f8b5c151a80 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f8b48011ab0 tx=0x7f8b48011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:32.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b58c00640 1 -- 172.21.15.94:0/4137239848 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4801e070 con 0x7f8b5c0b5850 2024-09-10T23:40:32.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b58c00640 1 -- 172.21.15.94:0/4137239848 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b4800fe20 con 0x7f8b5c0b5850 2024-09-10T23:40:32.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b58c00640 1 -- 172.21.15.94:0/4137239848 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48019650 con 0x7f8b5c0b5850 2024-09-10T23:40:32.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b62200640 1 -- 172.21.15.94:0/4137239848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5c1f7660 con 0x7f8b5c0b5850 2024-09-10T23:40:32.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b62200640 1 -- 172.21.15.94:0/4137239848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5c1f7c20 con 0x7f8b5c0b5850 2024-09-10T23:40:32.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b62200640 1 -- 172.21.15.94:0/4137239848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b5c0c50c0 con 0x7f8b5c0b5850 2024-09-10T23:40:32.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.359+0000 7f8b58c00640 1 -- 172.21.15.94:0/4137239848 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f8b480197f0 con 0x7f8b5c0b5850 2024-09-10T23:40:32.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.363+0000 7f8b58c00640 1 --2- 172.21.15.94:0/4137239848 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8b3c073030 0x7f8b3c0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:32.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.363+0000 7f8b58c00640 1 -- 172.21.15.94:0/4137239848 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f8b48095660 con 0x7f8b5c0b5850 2024-09-10T23:40:32.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.363+0000 7f8b60e00640 1 --2- 172.21.15.94:0/4137239848 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8b3c073030 0x7f8b3c0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:32.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.363+0000 7f8b60e00640 1 --2- 172.21.15.94:0/4137239848 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8b3c073030 0x7f8b3c0754f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8b50001250 tx=0x7f8b50006040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:32.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.367+0000 7f8b58c00640 1 -- 172.21.15.94:0/4137239848 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8b4805f390 con 0x7f8b5c0b5850 2024-09-10T23:40:32.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.535+0000 7f8b62200640 1 -- 172.21.15.94:0/4137239848 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b5c09b200 con 0x7f8b5c0b5850 2024-09-10T23:40:32.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.535+0000 7f8b58c00640 1 -- 172.21.15.94:0/4137239848 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) ==== 74+0+16715 (secure 0 0 0) 0x7f8b48063dc0 con 0x7f8b5c0b5850 2024-09-10T23:40:32.540 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:32.540 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":151,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:40:04.337069+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:40:00.461748+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-10T23:40:01.690823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:40:32.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.539+0000 7f8b62200640 1 -- 172.21.15.94:0/4137239848 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8b3c073030 msgr2=0x7f8b3c0754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:32.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.539+0000 7f8b62200640 1 --2- 172.21.15.94:0/4137239848 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8b3c073030 0x7f8b3c0754f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8b50001250 tx=0x7f8b50006040 comp rx=0 tx=0).stop 2024-09-10T23:40:32.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.539+0000 7f8b62200640 1 -- 172.21.15.94:0/4137239848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b5c0b5850 msgr2=0x7f8b5c151a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:32.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.539+0000 7f8b62200640 1 --2- 172.21.15.94:0/4137239848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b5c0b5850 0x7f8b5c151a80 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f8b48011ab0 tx=0x7f8b48011f70 comp rx=0 tx=0).stop 2024-09-10T23:40:32.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.539+0000 7f8b62200640 1 -- 172.21.15.94:0/4137239848 shutdown_connections 2024-09-10T23:40:32.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.539+0000 7f8b62200640 1 --2- 172.21.15.94:0/4137239848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b5c0c4700 0x7f8b5c0c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.539+0000 7f8b62200640 1 --2- 172.21.15.94:0/4137239848 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8b3c073030 0x7f8b3c0754f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.539+0000 7f8b62200640 1 --2- 172.21.15.94:0/4137239848 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b5c0b5850 0x7f8b5c151a80 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.539+0000 7f8b62200640 1 --2- 172.21.15.94:0/4137239848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b440051d0 0x7f8b5c0b6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.539+0000 7f8b62200640 1 -- 172.21.15.94:0/4137239848 >> 172.21.15.94:0/4137239848 conn(0x7f8b5c09aa70 msgr2=0x7f8b5c15a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:32.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.539+0000 7f8b62200640 1 -- 172.21.15.94:0/4137239848 shutdown_connections 2024-09-10T23:40:32.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.539+0000 7f8b62200640 1 -- 172.21.15.94:0/4137239848 wait complete. 2024-09-10T23:40:32.562 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T23:40:32.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fb6e00640 1 -- 172.21.15.94:0/4184203329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f94004d50 msgr2=0x7f0f940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:32.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fb6e00640 1 --2- 172.21.15.94:0/4184203329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f94004d50 0x7f0f940051d0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f0fa800af30 tx=0x7f0fa801d0d0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fb4e00640 1 -- 172.21.15.94:0/4184203329 reap_dead start 2024-09-10T23:40:32.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fb6e00640 1 -- 172.21.15.94:0/4184203329 shutdown_connections 2024-09-10T23:40:32.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fb6e00640 1 -- 172.21.15.94:0/4184203329 >> 172.21.15.94:0/4184203329 conn(0x7f0fb00cfaf0 msgr2=0x7f0fb00cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:32.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fb6e00640 1 -- 172.21.15.94:0/4184203329 shutdown_connections 2024-09-10T23:40:32.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fb6e00640 1 -- 172.21.15.94:0/4184203329 wait complete. 2024-09-10T23:40:32.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fb6e00640 1 Processor -- start 2024-09-10T23:40:32.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fb6e00640 1 -- start start 2024-09-10T23:40:32.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fb6e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f94004d50 0x7f0fb01f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:32.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fb6e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb014dbd0 0x7f0fb014b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:32.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fb6e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fb00ae010 0x7f0fb00ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:32.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fb6e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb006edb0 con 0x7f0fb014dbd0 2024-09-10T23:40:32.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fb6e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb006ec30 con 0x7f0fb00ae010 2024-09-10T23:40:32.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fb6e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb006ef30 con 0x7f0f94004d50 2024-09-10T23:40:32.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fa7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb014dbd0 0x7f0fb014b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:32.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fa7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb014dbd0 0x7f0fb014b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46632/0 (socket says 172.21.15.94:46632) 2024-09-10T23:40:32.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fa7400640 1 -- 172.21.15.94:0/487300712 learned_addr learned my addr 172.21.15.94:0/487300712 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:32.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fafe00640 1 --2- 172.21.15.94:0/487300712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f94004d50 0x7f0fb01f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:32.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.703+0000 7f0fb4e00640 1 --2- 172.21.15.94:0/487300712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fb00ae010 0x7f0fb00ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:32.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.707+0000 7f0fa7400640 1 -- 172.21.15.94:0/487300712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f94004d50 msgr2=0x7f0fb01f54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:32.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.707+0000 7f0fa7400640 1 --2- 172.21.15.94:0/487300712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f94004d50 0x7f0fb01f54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.707+0000 7f0fa7400640 1 -- 172.21.15.94:0/487300712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fb00ae010 msgr2=0x7f0fb00ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:32.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.707+0000 7f0fa7400640 1 --2- 172.21.15.94:0/487300712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fb00ae010 0x7f0fb00ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.707+0000 7f0fa7400640 1 -- 172.21.15.94:0/487300712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb014e3b0 con 0x7f0fb014dbd0 2024-09-10T23:40:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.707+0000 7f0fa7400640 1 --2- 172.21.15.94:0/487300712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb014dbd0 0x7f0fb014b290 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f0f98011a50 tx=0x7f0f98011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.707+0000 7f0facc00640 1 -- 172.21.15.94:0/487300712 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9801f070 con 0x7f0fb014dbd0 2024-09-10T23:40:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.707+0000 7f0facc00640 1 -- 172.21.15.94:0/487300712 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f9800fd60 con 0x7f0fb014dbd0 2024-09-10T23:40:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.707+0000 7f0fb6e00640 1 -- 172.21.15.94:0/487300712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb014e690 con 0x7f0fb014dbd0 2024-09-10T23:40:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.707+0000 7f0facc00640 1 -- 172.21.15.94:0/487300712 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f980126b0 con 0x7f0fb014dbd0 2024-09-10T23:40:32.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.707+0000 7f0fb6e00640 1 -- 172.21.15.94:0/487300712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb00adaf0 con 0x7f0fb014dbd0 2024-09-10T23:40:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.707+0000 7f0fb6e00640 1 -- 172.21.15.94:0/487300712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb00ad8e0 con 0x7f0fb014dbd0 2024-09-10T23:40:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.707+0000 7f0facc00640 1 -- 172.21.15.94:0/487300712 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f0f98012850 con 0x7f0fb014dbd0 2024-09-10T23:40:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.707+0000 7f0facc00640 1 --2- 172.21.15.94:0/487300712 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0f8c073030 0x7f0f8c0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.707+0000 7f0facc00640 1 -- 172.21.15.94:0/487300712 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5737+0+0 (secure 0 0 0) 0x7f0f980928f0 con 0x7f0fb014dbd0 2024-09-10T23:40:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.707+0000 7f0fafe00640 1 --2- 172.21.15.94:0/487300712 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0f8c073030 0x7f0f8c0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:32.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.711+0000 7f0fafe00640 1 --2- 172.21.15.94:0/487300712 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0f8c073030 0x7f0f8c0754f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00ca2c0 tx=0x7f0fa0001ba0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:32.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.711+0000 7f0facc00640 1 -- 172.21.15.94:0/487300712 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0f9805f1b0 con 0x7f0fb014dbd0 2024-09-10T23:40:32.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.915+0000 7f0fb6e00640 1 -- 172.21.15.94:0/487300712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0fb00d0850 con 0x7f0fb014dbd0 2024-09-10T23:40:32.918 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-10T23:40:32.915+0000 7f4045a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:40:32.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.915+0000 7f0facc00640 1 -- 172.21.15.94:0/487300712 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v109) ==== 161+0+0 (secure 0 0 0) 0x7f0f98063be0 con 0x7f0fb014dbd0 2024-09-10T23:40:32.920 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T23:40:32.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.919+0000 7f0fb6e00640 1 -- 172.21.15.94:0/487300712 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0f8c073030 msgr2=0x7f0f8c0754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:32.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.919+0000 7f0fb6e00640 1 --2- 172.21.15.94:0/487300712 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0f8c073030 0x7f0f8c0754f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00ca2c0 tx=0x7f0fa0001ba0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.919+0000 7f0fb6e00640 1 -- 172.21.15.94:0/487300712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb014dbd0 msgr2=0x7f0fb014b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:32.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.919+0000 7f0fb6e00640 1 --2- 172.21.15.94:0/487300712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb014dbd0 0x7f0fb014b290 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f0f98011a50 tx=0x7f0f98011e50 comp rx=0 tx=0).stop 2024-09-10T23:40:32.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.919+0000 7f0fb6e00640 1 -- 172.21.15.94:0/487300712 shutdown_connections 2024-09-10T23:40:32.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.919+0000 7f0fb6e00640 1 --2- 172.21.15.94:0/487300712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fb00ae010 0x7f0fb00ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.919+0000 7f0fb6e00640 1 --2- 172.21.15.94:0/487300712 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0f8c073030 0x7f0f8c0754f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.919+0000 7f0fb6e00640 1 --2- 172.21.15.94:0/487300712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fb014dbd0 0x7f0fb014b290 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.919+0000 7f0fb6e00640 1 --2- 172.21.15.94:0/487300712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f94004d50 0x7f0fb01f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:32.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.919+0000 7f0fb6e00640 1 -- 172.21.15.94:0/487300712 >> 172.21.15.94:0/487300712 conn(0x7f0fb00cfaf0 msgr2=0x7f0fb00afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:32.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.919+0000 7f0fb6e00640 1 -- 172.21.15.94:0/487300712 shutdown_connections 2024-09-10T23:40:32.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:32.919+0000 7f0fb6e00640 1 -- 172.21.15.94:0/487300712 wait complete. 2024-09-10T23:40:32.943 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T23:40:32.993 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:40:33.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.083+0000 7fb885800640 1 -- 172.21.15.94:0/868239005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb88015f140 msgr2=0x7fb8800b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:33.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.083+0000 7fb885800640 1 --2- 172.21.15.94:0/868239005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb88015f140 0x7fb8800b1170 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fb88006d940 tx=0x7fb87001c890 comp rx=0 tx=0).stop 2024-09-10T23:40:33.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.083+0000 7fb885800640 1 -- 172.21.15.94:0/868239005 shutdown_connections 2024-09-10T23:40:33.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.083+0000 7fb885800640 1 --2- 172.21.15.94:0/868239005 >> v2:172.21.15.181:3300/0 conn(0x7fb8800b1bf0 0x7fb8800cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:33.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.083+0000 7fb885800640 1 --2- 172.21.15.94:0/868239005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8800b2ac0 0x7fb8800b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:33.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.083+0000 7fb885800640 1 --2- 172.21.15.94:0/868239005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb88015f140 0x7fb8800b1170 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:33.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.083+0000 7fb885800640 1 -- 172.21.15.94:0/868239005 >> 172.21.15.94:0/868239005 conn(0x7fb8800d1b00 msgr2=0x7fb8800d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:33.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.083+0000 7fb885800640 1 -- 172.21.15.94:0/868239005 shutdown_connections 2024-09-10T23:40:33.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.083+0000 7fb885800640 1 -- 172.21.15.94:0/868239005 wait complete. 2024-09-10T23:40:33.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb885800640 1 Processor -- start 2024-09-10T23:40:33.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb885800640 1 -- start start 2024-09-10T23:40:33.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb885800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8800b1bf0 0x7fb880153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:33.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb885800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8800b2ac0 0x7fb8801f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:33.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb885800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb88015f140 0x7fb8801fb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:33.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb885800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb8800af4e0 con 0x7fb88015f140 2024-09-10T23:40:33.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb885800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb8800af360 con 0x7fb8800b2ac0 2024-09-10T23:40:33.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb885800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb8800af660 con 0x7fb8800b1bf0 2024-09-10T23:40:33.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb884e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb88015f140 0x7fb8801fb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:33.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb87fe00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8800b1bf0 0x7fb880153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:33.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb884e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb88015f140 0x7fb8801fb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46658/0 (socket says 172.21.15.94:46658) 2024-09-10T23:40:33.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb884e00640 1 -- 172.21.15.94:0/676182837 learned_addr learned my addr 172.21.15.94:0/676182837 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:33.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb877400640 1 --2- 172.21.15.94:0/676182837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8800b2ac0 0x7fb8801f78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:33.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb87fe00640 1 -- 172.21.15.94:0/676182837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8800b2ac0 msgr2=0x7fb8801f78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:33.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb87fe00640 1 --2- 172.21.15.94:0/676182837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8800b2ac0 0x7fb8801f78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:33.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb87fe00640 1 -- 172.21.15.94:0/676182837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb88015f140 msgr2=0x7fb8801fb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:33.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb87fe00640 1 --2- 172.21.15.94:0/676182837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb88015f140 0x7fb8801fb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:33.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb87fe00640 1 -- 172.21.15.94:0/676182837 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8800ce3a0 con 0x7fb8800b1bf0 2024-09-10T23:40:33.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb884e00640 1 --2- 172.21.15.94:0/676182837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb88015f140 0x7fb8801fb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:40:33.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb87fe00640 1 --2- 172.21.15.94:0/676182837 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8800b1bf0 0x7fb880153580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb880076620 tx=0x7fb8700a5730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:33.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.087+0000 7fb87cc00640 1 -- 172.21.15.94:0/676182837 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8700bc070 con 0x7fb8800b1bf0 2024-09-10T23:40:33.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.091+0000 7fb87cc00640 1 -- 172.21.15.94:0/676182837 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb870004450 con 0x7fb8800b1bf0 2024-09-10T23:40:33.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.091+0000 7fb87cc00640 1 -- 172.21.15.94:0/676182837 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8700b2040 con 0x7fb8800b1bf0 2024-09-10T23:40:33.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.091+0000 7fb885800640 1 -- 172.21.15.94:0/676182837 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8800ce680 con 0x7fb8800b1bf0 2024-09-10T23:40:33.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.091+0000 7fb885800640 1 -- 172.21.15.94:0/676182837 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb880153ac0 con 0x7fb8800b1bf0 2024-09-10T23:40:33.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.091+0000 7fb885800640 1 -- 172.21.15.94:0/676182837 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8800cfa00 con 0x7fb8800b1bf0 2024-09-10T23:40:33.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.091+0000 7fb87cc00640 1 -- 172.21.15.94:0/676182837 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7fb870003860 con 0x7fb8800b1bf0 2024-09-10T23:40:33.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.091+0000 7fb87cc00640 1 --2- 172.21.15.94:0/676182837 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb85c073000 0x7fb85c0754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.091+0000 7fb87cc00640 1 -- 172.21.15.94:0/676182837 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(152..152 src has 1..152) ==== 5823+0+0 (secure 0 0 0) 0x7fb87012b5a0 con 0x7fb8800b1bf0 2024-09-10T23:40:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.091+0000 7fb877400640 1 --2- 172.21.15.94:0/676182837 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb85c073000 0x7fb85c0754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:33.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.095+0000 7fb877400640 1 --2- 172.21.15.94:0/676182837 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb85c073000 0x7fb85c0754c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb8800ceea0 tx=0x7fb86800b390 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:33.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.095+0000 7fb87cc00640 1 -- 172.21.15.94:0/676182837 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb8700164b0 con 0x7fb8800b1bf0 2024-09-10T23:40:33.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.295+0000 7fb885800640 1 -- 172.21.15.94:0/676182837 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb88015c390 con 0x7fb8800b1bf0 2024-09-10T23:40:33.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.991+0000 7fb87cc00640 1 -- 172.21.15.94:0/676182837 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v110) ==== 104+0+0 (secure 0 0 0) 0x7fb8700f7e60 con 0x7fb8800b1bf0 2024-09-10T23:40:34.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.995+0000 7fb885800640 1 -- 172.21.15.94:0/676182837 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb85c073000 msgr2=0x7fb85c0754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:34.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.995+0000 7fb885800640 1 --2- 172.21.15.94:0/676182837 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb85c073000 0x7fb85c0754c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb8800ceea0 tx=0x7fb86800b390 comp rx=0 tx=0).stop 2024-09-10T23:40:34.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.995+0000 7fb885800640 1 -- 172.21.15.94:0/676182837 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8800b1bf0 msgr2=0x7fb880153580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:34.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.995+0000 7fb885800640 1 --2- 172.21.15.94:0/676182837 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8800b1bf0 0x7fb880153580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb880076620 tx=0x7fb8700a5730 comp rx=0 tx=0).stop 2024-09-10T23:40:34.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.995+0000 7fb885800640 1 -- 172.21.15.94:0/676182837 shutdown_connections 2024-09-10T23:40:34.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.995+0000 7fb885800640 1 --2- 172.21.15.94:0/676182837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8800b2ac0 0x7fb8801f78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:34.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.995+0000 7fb885800640 1 --2- 172.21.15.94:0/676182837 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb85c073000 0x7fb85c0754c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:34.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.995+0000 7fb885800640 1 --2- 172.21.15.94:0/676182837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb88015f140 0x7fb8801fb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:34.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.995+0000 7fb885800640 1 --2- 172.21.15.94:0/676182837 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8800b1bf0 0x7fb880153580 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:34.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.995+0000 7fb885800640 1 -- 172.21.15.94:0/676182837 >> 172.21.15.94:0/676182837 conn(0x7fb8800d1b00 msgr2=0x7fb8800b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:34.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.995+0000 7fb885800640 1 -- 172.21.15.94:0/676182837 shutdown_connections 2024-09-10T23:40:34.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:33.995+0000 7fb885800640 1 -- 172.21.15.94:0/676182837 wait complete. 2024-09-10T23:40:34.020 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T23:40:34.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.163+0000 7f3a01800640 1 -- 172.21.15.94:0/3058830094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39fc15ba50 msgr2=0x7f39fc0b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:34.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.163+0000 7f3a01800640 1 --2- 172.21.15.94:0/3058830094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39fc15ba50 0x7f39fc0b4700 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f39ec009900 tx=0x7f39ec01c820 comp rx=0 tx=0).stop 2024-09-10T23:40:34.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.163+0000 7f3a01800640 1 -- 172.21.15.94:0/3058830094 shutdown_connections 2024-09-10T23:40:34.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.163+0000 7f3a01800640 1 --2- 172.21.15.94:0/3058830094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39fc0d03c0 0x7f39fc0cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:34.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.163+0000 7f3a01800640 1 --2- 172.21.15.94:0/3058830094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39fc15ba50 0x7f39fc0b4700 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:34.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.163+0000 7f3a01800640 1 -- 172.21.15.94:0/3058830094 >> 172.21.15.94:0/3058830094 conn(0x7f39fc06cdf0 msgr2=0x7f39fc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:34.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.163+0000 7f3a01800640 1 -- 172.21.15.94:0/3058830094 shutdown_connections 2024-09-10T23:40:34.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.163+0000 7f3a01800640 1 -- 172.21.15.94:0/3058830094 wait complete. 2024-09-10T23:40:34.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.163+0000 7f3a01800640 1 Processor -- start 2024-09-10T23:40:34.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f3a01800640 1 -- start start 2024-09-10T23:40:34.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f3a01800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39fc0b4c40 0x7f39fc142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:34.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f3a01800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39fc0d03c0 0x7f39fc15cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:34.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f3a01800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39fc15ba50 0x7f39fc14e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:34.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f3a01800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc168a60 con 0x7f39fc15ba50 2024-09-10T23:40:34.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f3a01800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc0cefc0 con 0x7f39fc0b4c40 2024-09-10T23:40:34.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f3a01800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f39fc168be0 con 0x7f39fc0d03c0 2024-09-10T23:40:34.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f39fb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39fc15ba50 0x7f39fc14e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:34.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f39faa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39fc0b4c40 0x7f39fc142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:34.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f39fb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39fc15ba50 0x7f39fc14e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46698/0 (socket says 172.21.15.94:46698) 2024-09-10T23:40:34.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f39fb400640 1 -- 172.21.15.94:0/3638508360 learned_addr learned my addr 172.21.15.94:0/3638508360 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:34.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f39f2000640 1 --2- 172.21.15.94:0/3638508360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39fc0d03c0 0x7f39fc15cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:34.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f39fb400640 1 -- 172.21.15.94:0/3638508360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39fc0d03c0 msgr2=0x7f39fc15cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:34.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f39fb400640 1 --2- 172.21.15.94:0/3638508360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39fc0d03c0 0x7f39fc15cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:34.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f39fb400640 1 -- 172.21.15.94:0/3638508360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39fc0b4c40 msgr2=0x7f39fc142070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:34.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f39fb400640 1 --2- 172.21.15.94:0/3638508360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39fc0b4c40 0x7f39fc142070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:34.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f39fb400640 1 -- 172.21.15.94:0/3638508360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39fc15d9b0 con 0x7f39fc15ba50 2024-09-10T23:40:34.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f39fb400640 1 --2- 172.21.15.94:0/3638508360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39fc15ba50 0x7f39fc14e160 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f39f400efd0 tx=0x7f39f400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:34.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f39f3400640 1 -- 172.21.15.94:0/3638508360 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4003b80 con 0x7f39fc15ba50 2024-09-10T23:40:34.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f39f3400640 1 -- 172.21.15.94:0/3638508360 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39f4003e10 con 0x7f39fc15ba50 2024-09-10T23:40:34.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f3a01800640 1 -- 172.21.15.94:0/3638508360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39fc15dc90 con 0x7f39fc15ba50 2024-09-10T23:40:34.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f39f3400640 1 -- 172.21.15.94:0/3638508360 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4004280 con 0x7f39fc15ba50 2024-09-10T23:40:34.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f3a01800640 1 -- 172.21.15.94:0/3638508360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39fc15e250 con 0x7f39fc15ba50 2024-09-10T23:40:34.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.167+0000 7f3a01800640 1 -- 172.21.15.94:0/3638508360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39fc0cddb0 con 0x7f39fc15ba50 2024-09-10T23:40:34.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.171+0000 7f39f3400640 1 -- 172.21.15.94:0/3638508360 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f39f4004420 con 0x7f39fc15ba50 2024-09-10T23:40:34.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.171+0000 7f39f3400640 1 --2- 172.21.15.94:0/3638508360 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f39d8073000 0x7f39d80754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:34.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.171+0000 7f39f3400640 1 -- 172.21.15.94:0/3638508360 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5823+0+0 (secure 0 0 0) 0x7f39f4092bd0 con 0x7f39fc15ba50 2024-09-10T23:40:34.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.171+0000 7f39faa00640 1 --2- 172.21.15.94:0/3638508360 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f39d8073000 0x7f39d80754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:34.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.171+0000 7f39faa00640 1 --2- 172.21.15.94:0/3638508360 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f39d8073000 0x7f39d80754c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0af000 tx=0x7f39ec01b060 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:34.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.175+0000 7f39f3400640 1 -- 172.21.15.94:0/3638508360 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f39f405f490 con 0x7f39fc15ba50 2024-09-10T23:40:34.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.343+0000 7f3a01800640 1 -- 172.21.15.94:0/3638508360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f39fc157f80 con 0x7f39fc15ba50 2024-09-10T23:40:35.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:34.999+0000 7f39f3400640 1 -- 172.21.15.94:0/3638508360 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v153) ==== 181+0+0 (secure 0 0 0) 0x7f39f4068030 con 0x7f39fc15ba50 2024-09-10T23:40:35.004 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-09-10T23:40:35.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.003+0000 7f3a01800640 1 -- 172.21.15.94:0/3638508360 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f39d8073000 msgr2=0x7f39d80754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:35.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.003+0000 7f3a01800640 1 --2- 172.21.15.94:0/3638508360 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f39d8073000 0x7f39d80754c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f39ec0af000 tx=0x7f39ec01b060 comp rx=0 tx=0).stop 2024-09-10T23:40:35.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.003+0000 7f3a01800640 1 -- 172.21.15.94:0/3638508360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39fc15ba50 msgr2=0x7f39fc14e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:35.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.003+0000 7f3a01800640 1 --2- 172.21.15.94:0/3638508360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39fc15ba50 0x7f39fc14e160 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f39f400efd0 tx=0x7f39f400c6a0 comp rx=0 tx=0).stop 2024-09-10T23:40:35.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.003+0000 7f3a01800640 1 -- 172.21.15.94:0/3638508360 shutdown_connections 2024-09-10T23:40:35.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.003+0000 7f3a01800640 1 --2- 172.21.15.94:0/3638508360 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f39d8073000 0x7f39d80754c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:35.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.003+0000 7f3a01800640 1 --2- 172.21.15.94:0/3638508360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39fc15ba50 0x7f39fc14e160 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:35.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.003+0000 7f3a01800640 1 --2- 172.21.15.94:0/3638508360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39fc0d03c0 0x7f39fc15cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:35.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.003+0000 7f3a01800640 1 --2- 172.21.15.94:0/3638508360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39fc0b4c40 0x7f39fc142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:35.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.003+0000 7f3a01800640 1 -- 172.21.15.94:0/3638508360 >> 172.21.15.94:0/3638508360 conn(0x7f39fc06cdf0 msgr2=0x7f39fc15ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:35.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.003+0000 7f3a01800640 1 -- 172.21.15.94:0/3638508360 shutdown_connections 2024-09-10T23:40:35.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.003+0000 7f3a01800640 1 -- 172.21.15.94:0/3638508360 wait complete. 2024-09-10T23:40:35.029 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T23:40:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.171+0000 7f68b3200640 1 -- 172.21.15.94:0/2046936267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b40d2020 msgr2=0x7f68b41688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.171+0000 7f68b3200640 1 --2- 172.21.15.94:0/2046936267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b40d2020 0x7f68b41688d0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009930 tx=0x7f68ac01c820 comp rx=0 tx=0).stop 2024-09-10T23:40:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.171+0000 7f68b3200640 1 -- 172.21.15.94:0/2046936267 shutdown_connections 2024-09-10T23:40:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.171+0000 7f68b3200640 1 --2- 172.21.15.94:0/2046936267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b40d2020 0x7f68b41688d0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:35.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.171+0000 7f68b3200640 1 -- 172.21.15.94:0/2046936267 >> 172.21.15.94:0/2046936267 conn(0x7f68b409aa70 msgr2=0x7f68b409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.171+0000 7f68b3200640 1 -- 172.21.15.94:0/2046936267 shutdown_connections 2024-09-10T23:40:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.171+0000 7f68b3200640 1 -- 172.21.15.94:0/2046936267 wait complete. 2024-09-10T23:40:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b3200640 1 Processor -- start 2024-09-10T23:40:35.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b3200640 1 -- start start 2024-09-10T23:40:35.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b3200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b40b25c0 0x7f68b40c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:35.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b3200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68b40d2020 0x7f68b40c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:35.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b3200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b415de30 0x7f68b40c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:35.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b3200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68b416bfb0 con 0x7f68b415de30 2024-09-10T23:40:35.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b3200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f68b416be30 con 0x7f68b40b25c0 2024-09-10T23:40:35.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b3200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f68b416c130 con 0x7f68b40d2020 2024-09-10T23:40:35.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b415de30 0x7f68b40c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:35.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b415de30 0x7f68b40c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46716/0 (socket says 172.21.15.94:46716) 2024-09-10T23:40:35.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b2800640 1 -- 172.21.15.94:0/12029403 learned_addr learned my addr 172.21.15.94:0/12029403 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:35.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b1e00640 1 --2- 172.21.15.94:0/12029403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b40b25c0 0x7f68b40c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:35.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b1400640 1 --2- 172.21.15.94:0/12029403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68b40d2020 0x7f68b40c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:35.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b2800640 1 -- 172.21.15.94:0/12029403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68b40d2020 msgr2=0x7f68b40c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:35.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b2800640 1 --2- 172.21.15.94:0/12029403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68b40d2020 0x7f68b40c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:35.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b2800640 1 -- 172.21.15.94:0/12029403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b40b25c0 msgr2=0x7f68b40c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:35.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b2800640 1 --2- 172.21.15.94:0/12029403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b40b25c0 0x7f68b40c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:35.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b2800640 1 -- 172.21.15.94:0/12029403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b40c5e40 con 0x7f68b415de30 2024-09-10T23:40:35.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b1400640 1 --2- 172.21.15.94:0/12029403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68b40d2020 0x7f68b40c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:40:35.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b2800640 1 --2- 172.21.15.94:0/12029403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b415de30 0x7f68b40c8e50 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009850 tx=0x7f68ac0041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:35.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68aa000640 1 -- 172.21.15.94:0/12029403 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0ba020 con 0x7f68b415de30 2024-09-10T23:40:35.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68aa000640 1 -- 172.21.15.94:0/12029403 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68ac005030 con 0x7f68b415de30 2024-09-10T23:40:35.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68aa000640 1 -- 172.21.15.94:0/12029403 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0b2040 con 0x7f68b415de30 2024-09-10T23:40:35.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b3200640 1 -- 172.21.15.94:0/12029403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b40c6120 con 0x7f68b415de30 2024-09-10T23:40:35.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.175+0000 7f68b3200640 1 -- 172.21.15.94:0/12029403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b40c66e0 con 0x7f68b415de30 2024-09-10T23:40:35.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.179+0000 7f68b3200640 1 -- 172.21.15.94:0/12029403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b40d1910 con 0x7f68b415de30 2024-09-10T23:40:35.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.179+0000 7f68aa000640 1 -- 172.21.15.94:0/12029403 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f68ac0046d0 con 0x7f68b415de30 2024-09-10T23:40:35.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.179+0000 7f68aa000640 1 --2- 172.21.15.94:0/12029403 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6894073000 0x7f68940754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:35.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.179+0000 7f68aa000640 1 -- 172.21.15.94:0/12029403 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5385+0+0 (secure 0 0 0) 0x7f68ac12be00 con 0x7f68b415de30 2024-09-10T23:40:35.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.179+0000 7f68b1e00640 1 --2- 172.21.15.94:0/12029403 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6894073000 0x7f68940754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:35.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.179+0000 7f68b1e00640 1 --2- 172.21.15.94:0/12029403 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6894073000 0x7f68940754c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f68b40cc200 tx=0x7f68a4009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:35.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.183+0000 7f68aa000640 1 -- 172.21.15.94:0/12029403 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f68ac0c4060 con 0x7f68b415de30 2024-09-10T23:40:35.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:35.351+0000 7f68b3200640 1 -- 172.21.15.94:0/12029403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f68b40cc520 con 0x7f68b415de30 2024-09-10T23:40:35.526 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:40:35.530 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:40:36.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.007+0000 7f68aa000640 1 -- 172.21.15.94:0/12029403 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v154) ==== 169+0+0 (secure 0 0 0) 0x7f68ac0164d0 con 0x7f68b415de30 2024-09-10T23:40:36.011 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-09-10T23:40:36.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.011+0000 7f68b3200640 1 -- 172.21.15.94:0/12029403 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6894073000 msgr2=0x7f68940754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:36.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.011+0000 7f68b3200640 1 --2- 172.21.15.94:0/12029403 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6894073000 0x7f68940754c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f68b40cc200 tx=0x7f68a4009040 comp rx=0 tx=0).stop 2024-09-10T23:40:36.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.011+0000 7f68b3200640 1 -- 172.21.15.94:0/12029403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b415de30 msgr2=0x7f68b40c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:36.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.011+0000 7f68b3200640 1 --2- 172.21.15.94:0/12029403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b415de30 0x7f68b40c8e50 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009850 tx=0x7f68ac0041a0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.011+0000 7f68b3200640 1 -- 172.21.15.94:0/12029403 shutdown_connections 2024-09-10T23:40:36.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.011+0000 7f68b3200640 1 --2- 172.21.15.94:0/12029403 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6894073000 0x7f68940754c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.011+0000 7f68b3200640 1 --2- 172.21.15.94:0/12029403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b415de30 0x7f68b40c8e50 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.011+0000 7f68b3200640 1 --2- 172.21.15.94:0/12029403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68b40d2020 0x7f68b40c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.011+0000 7f68b3200640 1 --2- 172.21.15.94:0/12029403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b40b25c0 0x7f68b40c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.011+0000 7f68b3200640 1 -- 172.21.15.94:0/12029403 >> 172.21.15.94:0/12029403 conn(0x7f68b409aa70 msgr2=0x7f68b4070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:36.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.011+0000 7f68b3200640 1 -- 172.21.15.94:0/12029403 shutdown_connections 2024-09-10T23:40:36.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.011+0000 7f68b3200640 1 -- 172.21.15.94:0/12029403 wait complete. 2024-09-10T23:40:36.038 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T23:40:36.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.183+0000 7f038a600640 1 -- 172.21.15.94:0/2121049273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f038415bf60 msgr2=0x7f03840d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:36.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.183+0000 7f038a600640 1 --2- 172.21.15.94:0/2121049273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f038415bf60 0x7f03840d2780 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f03840731c0 tx=0x7f037401c570 comp rx=0 tx=0).stop 2024-09-10T23:40:36.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.183+0000 7f038a600640 1 -- 172.21.15.94:0/2121049273 shutdown_connections 2024-09-10T23:40:36.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.183+0000 7f038a600640 1 --2- 172.21.15.94:0/2121049273 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03840d0660 0x7f03840ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.183+0000 7f038a600640 1 --2- 172.21.15.94:0/2121049273 >> v2:172.21.15.181:3300/0 conn(0x7f03840cfce0 0x7f03840d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.183+0000 7f038a600640 1 --2- 172.21.15.94:0/2121049273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f038415bf60 0x7f03840d2780 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.183+0000 7f038a600640 1 -- 172.21.15.94:0/2121049273 >> 172.21.15.94:0/2121049273 conn(0x7f03840d1c60 msgr2=0x7f03840d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:36.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.183+0000 7f038a600640 1 -- 172.21.15.94:0/2121049273 shutdown_connections 2024-09-10T23:40:36.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.183+0000 7f038a600640 1 -- 172.21.15.94:0/2121049273 wait complete. 2024-09-10T23:40:36.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f038a600640 1 Processor -- start 2024-09-10T23:40:36.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f038a600640 1 -- start start 2024-09-10T23:40:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f038a600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03840cfce0 0x7f0384165950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f038a600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03840d0660 0x7f03840d2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:36.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f038a600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f038415bf60 0x7f03840d2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f038a600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f03840c6a50 con 0x7f03840d0660 2024-09-10T23:40:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f038a600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f03840c68d0 con 0x7f03840cfce0 2024-09-10T23:40:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f038a600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f03840c6bd0 con 0x7f038415bf60 2024-09-10T23:40:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f037aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03840d0660 0x7f03840d2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f037aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03840d0660 0x7f03840d2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46746/0 (socket says 172.21.15.94:46746) 2024-09-10T23:40:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f037aa00640 1 -- 172.21.15.94:0/1569510581 learned_addr learned my addr 172.21.15.94:0/1569510581 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f0383400640 1 --2- 172.21.15.94:0/1569510581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03840cfce0 0x7f0384165950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f0383e00640 1 --2- 172.21.15.94:0/1569510581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f038415bf60 0x7f03840d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f037aa00640 1 -- 172.21.15.94:0/1569510581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f038415bf60 msgr2=0x7f03840d2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f037aa00640 1 --2- 172.21.15.94:0/1569510581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f038415bf60 0x7f03840d2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f037aa00640 1 -- 172.21.15.94:0/1569510581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03840cfce0 msgr2=0x7f0384165950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f037aa00640 1 --2- 172.21.15.94:0/1569510581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03840cfce0 0x7f0384165950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f037aa00640 1 -- 172.21.15.94:0/1569510581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03841f8ec0 con 0x7f03840d0660 2024-09-10T23:40:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f037aa00640 1 --2- 172.21.15.94:0/1569510581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03840d0660 0x7f03840d2040 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f036c00d7a0 tx=0x7f036c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f037be00640 1 -- 172.21.15.94:0/1569510581 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c017070 con 0x7f03840d0660 2024-09-10T23:40:36.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f037be00640 1 -- 172.21.15.94:0/1569510581 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f036c009d20 con 0x7f03840d0660 2024-09-10T23:40:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f037be00640 1 -- 172.21.15.94:0/1569510581 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c0106c0 con 0x7f03840d0660 2024-09-10T23:40:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f038a600640 1 -- 172.21.15.94:0/1569510581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03841f94a0 con 0x7f03840d0660 2024-09-10T23:40:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.187+0000 7f038a600640 1 -- 172.21.15.94:0/1569510581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03841f9a60 con 0x7f03840d0660 2024-09-10T23:40:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.191+0000 7f038a600640 1 -- 172.21.15.94:0/1569510581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03840ce3f0 con 0x7f03840d0660 2024-09-10T23:40:36.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.191+0000 7f037be00640 1 -- 172.21.15.94:0/1569510581 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f036c010860 con 0x7f03840d0660 2024-09-10T23:40:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.191+0000 7f037be00640 1 --2- 172.21.15.94:0/1569510581 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0360073000 0x7f03600754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:36.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.191+0000 7f037be00640 1 -- 172.21.15.94:0/1569510581 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4971+0+0 (secure 0 0 0) 0x7f036c092130 con 0x7f03840d0660 2024-09-10T23:40:36.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.191+0000 7f0383400640 1 --2- 172.21.15.94:0/1569510581 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0360073000 0x7f03600754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:36.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.191+0000 7f0383400640 1 --2- 172.21.15.94:0/1569510581 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0360073000 0x7f03600754c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f03740ae000 tx=0x7f037401adb0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:36.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.195+0000 7f037be00640 1 -- 172.21.15.94:0/1569510581 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f036c05ea50 con 0x7f03840d0660 2024-09-10T23:40:36.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.363+0000 7f038a600640 1 -- 172.21.15.94:0/1569510581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f038406f2f0 con 0x7f03840d0660 2024-09-10T23:40:36.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.367+0000 7f037be00640 1 -- 172.21.15.94:0/1569510581 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v47) ==== 71+0+0 (secure 0 0 0) 0x7f036c063480 con 0x7f03840d0660 2024-09-10T23:40:36.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.371+0000 7f038a600640 1 -- 172.21.15.94:0/1569510581 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0360073000 msgr2=0x7f03600754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:36.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.371+0000 7f038a600640 1 --2- 172.21.15.94:0/1569510581 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0360073000 0x7f03600754c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f03740ae000 tx=0x7f037401adb0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.371+0000 7f038a600640 1 -- 172.21.15.94:0/1569510581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03840d0660 msgr2=0x7f03840d2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:36.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.371+0000 7f038a600640 1 --2- 172.21.15.94:0/1569510581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03840d0660 0x7f03840d2040 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f036c00d7a0 tx=0x7f036c00dc60 comp rx=0 tx=0).stop 2024-09-10T23:40:36.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.371+0000 7f038a600640 1 -- 172.21.15.94:0/1569510581 shutdown_connections 2024-09-10T23:40:36.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.371+0000 7f038a600640 1 --2- 172.21.15.94:0/1569510581 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0360073000 0x7f03600754c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.371+0000 7f038a600640 1 --2- 172.21.15.94:0/1569510581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f038415bf60 0x7f03840d2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.371+0000 7f038a600640 1 --2- 172.21.15.94:0/1569510581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03840d0660 0x7f03840d2040 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.371+0000 7f038a600640 1 --2- 172.21.15.94:0/1569510581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03840cfce0 0x7f0384165950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.371+0000 7f038a600640 1 -- 172.21.15.94:0/1569510581 >> 172.21.15.94:0/1569510581 conn(0x7f03840d1c60 msgr2=0x7f038406f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:36.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.371+0000 7f038a600640 1 -- 172.21.15.94:0/1569510581 shutdown_connections 2024-09-10T23:40:36.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.371+0000 7f038a600640 1 -- 172.21.15.94:0/1569510581 wait complete. 2024-09-10T23:40:36.392 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-10T23:40:36.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.531+0000 7f9402c00640 1 -- 172.21.15.94:0/2010602224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc070860 msgr2=0x7f93fc070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:36.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.531+0000 7f9402c00640 1 --2- 172.21.15.94:0/2010602224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc070860 0x7f93fc070c80 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009840 tx=0x7f93e401c730 comp rx=0 tx=0).stop 2024-09-10T23:40:36.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.531+0000 7f9402c00640 1 -- 172.21.15.94:0/2010602224 shutdown_connections 2024-09-10T23:40:36.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.531+0000 7f9402c00640 1 --2- 172.21.15.94:0/2010602224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc070860 0x7f93fc070c80 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.531+0000 7f9402c00640 1 --2- 172.21.15.94:0/2010602224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93fc0d0770 0x7f93fc15ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.531+0000 7f9402c00640 1 -- 172.21.15.94:0/2010602224 >> 172.21.15.94:0/2010602224 conn(0x7f93fc0d0b70 msgr2=0x7f93fc0d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:36.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.531+0000 7f9402c00640 1 -- 172.21.15.94:0/2010602224 shutdown_connections 2024-09-10T23:40:36.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.531+0000 7f9402c00640 1 -- 172.21.15.94:0/2010602224 wait complete. 2024-09-10T23:40:36.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.531+0000 7f9402c00640 1 Processor -- start 2024-09-10T23:40:36.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.531+0000 7f9402c00640 1 -- start start 2024-09-10T23:40:36.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9402c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93fc06f410 0x7f93fc1f6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:36.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9402c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93fc070860 0x7f93fc146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:36.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9402c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0d0770 0x7f93fc14c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:36.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9402c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1682b0 con 0x7f93fc0d0770 2024-09-10T23:40:36.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9402c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc168130 con 0x7f93fc06f410 2024-09-10T23:40:36.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9402c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f93fc168430 con 0x7f93fc070860 2024-09-10T23:40:36.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9400c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0d0770 0x7f93fc14c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:36.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9400c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0d0770 0x7f93fc14c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46768/0 (socket says 172.21.15.94:46768) 2024-09-10T23:40:36.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9400c00640 1 -- 172.21.15.94:0/3732194380 learned_addr learned my addr 172.21.15.94:0/3732194380 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:36.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f93f3400640 1 --2- 172.21.15.94:0/3732194380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93fc070860 0x7f93fc146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:36.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f93fbe00640 1 --2- 172.21.15.94:0/3732194380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93fc06f410 0x7f93fc1f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:36.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9400c00640 1 -- 172.21.15.94:0/3732194380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93fc070860 msgr2=0x7f93fc146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:36.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9400c00640 1 --2- 172.21.15.94:0/3732194380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93fc070860 0x7f93fc146de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9400c00640 1 -- 172.21.15.94:0/3732194380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93fc06f410 msgr2=0x7f93fc1f6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:36.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9400c00640 1 --2- 172.21.15.94:0/3732194380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93fc06f410 0x7f93fc1f6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:36.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9400c00640 1 -- 172.21.15.94:0/3732194380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93fc147320 con 0x7f93fc0d0770 2024-09-10T23:40:36.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9400c00640 1 --2- 172.21.15.94:0/3732194380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0d0770 0x7f93fc14c720 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c3a0 tx=0x7f93f400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:36.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f93f8c00640 1 -- 172.21.15.94:0/3732194380 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4017070 con 0x7f93fc0d0770 2024-09-10T23:40:36.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f93f8c00640 1 -- 172.21.15.94:0/3732194380 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93f4003600 con 0x7f93fc0d0770 2024-09-10T23:40:36.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9402c00640 1 -- 172.21.15.94:0/3732194380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93fc14b680 con 0x7f93fc0d0770 2024-09-10T23:40:36.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f93f8c00640 1 -- 172.21.15.94:0/3732194380 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4003a50 con 0x7f93fc0d0770 2024-09-10T23:40:36.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9402c00640 1 -- 172.21.15.94:0/3732194380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93fc14bc40 con 0x7f93fc0d0770 2024-09-10T23:40:36.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f9402c00640 1 -- 172.21.15.94:0/3732194380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93fc15ef90 con 0x7f93fc0d0770 2024-09-10T23:40:36.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.535+0000 7f93f8c00640 1 -- 172.21.15.94:0/3732194380 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f93f4003bf0 con 0x7f93fc0d0770 2024-09-10T23:40:36.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.539+0000 7f93f8c00640 1 --2- 172.21.15.94:0/3732194380 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f93d8073000 0x7f93d80754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:36.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.539+0000 7f93f8c00640 1 -- 172.21.15.94:0/3732194380 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4971+0+0 (secure 0 0 0) 0x7f93f4091290 con 0x7f93fc0d0770 2024-09-10T23:40:36.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.539+0000 7f93fbe00640 1 --2- 172.21.15.94:0/3732194380 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f93d8073000 0x7f93d80754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:36.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.539+0000 7f93fbe00640 1 --2- 172.21.15.94:0/3732194380 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f93d8073000 0x7f93d80754c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009e50 tx=0x7f93ec0095e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:36.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.543+0000 7f93f8c00640 1 -- 172.21.15.94:0/3732194380 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f93f405db50 con 0x7f93fc0d0770 2024-09-10T23:40:36.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:36.763+0000 7f9402c00640 1 -- 172.21.15.94:0/3732194380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7f93fc074410 con 0x7f93fc0d0770 2024-09-10T23:40:37.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.007+0000 7f93f8c00640 1 -- 172.21.15.94:0/3732194380 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v441) ==== 145+0+0 (secure 0 0 0) 0x7f93f4062580 con 0x7f93fc0d0770 2024-09-10T23:40:37.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.011+0000 7f9402c00640 1 -- 172.21.15.94:0/3732194380 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f93d8073000 msgr2=0x7f93d80754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:37.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.011+0000 7f9402c00640 1 --2- 172.21.15.94:0/3732194380 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f93d8073000 0x7f93d80754c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009e50 tx=0x7f93ec0095e0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.011+0000 7f9402c00640 1 -- 172.21.15.94:0/3732194380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0d0770 msgr2=0x7f93fc14c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:37.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.011+0000 7f9402c00640 1 --2- 172.21.15.94:0/3732194380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0d0770 0x7f93fc14c720 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f93f400c3a0 tx=0x7f93f400c860 comp rx=0 tx=0).stop 2024-09-10T23:40:37.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.011+0000 7f9402c00640 1 -- 172.21.15.94:0/3732194380 shutdown_connections 2024-09-10T23:40:37.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.011+0000 7f9402c00640 1 --2- 172.21.15.94:0/3732194380 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f93d8073000 0x7f93d80754c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.011+0000 7f9402c00640 1 --2- 172.21.15.94:0/3732194380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0d0770 0x7f93fc14c720 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.011+0000 7f9402c00640 1 --2- 172.21.15.94:0/3732194380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93fc070860 0x7f93fc146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.011+0000 7f9402c00640 1 --2- 172.21.15.94:0/3732194380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93fc06f410 0x7f93fc1f6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.011+0000 7f9402c00640 1 -- 172.21.15.94:0/3732194380 >> 172.21.15.94:0/3732194380 conn(0x7f93fc0d0b70 msgr2=0x7f93fc0cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:37.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.011+0000 7f9402c00640 1 -- 172.21.15.94:0/3732194380 shutdown_connections 2024-09-10T23:40:37.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.011+0000 7f9402c00640 1 -- 172.21.15.94:0/3732194380 wait complete. 2024-09-10T23:40:37.033 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-10T23:40:37.034 INFO:tasks.cephfs_test_runner:Starting test: test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate unparseable text .meta file on legacy subvol root doesn't break the system 2024-09-10T23:40:37.034 INFO:tasks.cephfs_test_runner:test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-10T23:40:37.034 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:40:37.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.171+0000 7f3ac4e00640 1 -- 172.21.15.94:0/64032391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ac00b2bb0 msgr2=0x7f3ac00d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:37.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.171+0000 7f3ac4e00640 1 --2- 172.21.15.94:0/64032391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ac00b2bb0 0x7f3ac00d0270 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f3ac015f9a0 tx=0x7f3aa801c7a0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.171+0000 7f3ac4e00640 1 -- 172.21.15.94:0/64032391 shutdown_connections 2024-09-10T23:40:37.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.171+0000 7f3ac4e00640 1 --2- 172.21.15.94:0/64032391 >> v2:172.21.15.181:3300/0 conn(0x7f3ac00af310 0x7f3ac00af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.171+0000 7f3ac4e00640 1 --2- 172.21.15.94:0/64032391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ac00b2bb0 0x7f3ac00d0270 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.171+0000 7f3ac4e00640 1 -- 172.21.15.94:0/64032391 >> 172.21.15.94:0/64032391 conn(0x7f3ac00d1c60 msgr2=0x7f3ac00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:37.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.171+0000 7f3ac4e00640 1 -- 172.21.15.94:0/64032391 shutdown_connections 2024-09-10T23:40:37.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.171+0000 7f3ac4e00640 1 -- 172.21.15.94:0/64032391 wait complete. 2024-09-10T23:40:37.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ac4e00640 1 Processor -- start 2024-09-10T23:40:37.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ac4e00640 1 -- start start 2024-09-10T23:40:37.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ac4e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ac00af310 0x7f3ac00c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:37.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ac4e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ac00b2bb0 0x7f3ac00c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:37.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ac4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac015f5c0 0x7f3ac015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:37.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ac4e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac0169050 con 0x7f3ac00b2bb0 2024-09-10T23:40:37.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ac4e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac0168ed0 con 0x7f3ac015f5c0 2024-09-10T23:40:37.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ac4e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f3ac01691d0 con 0x7f3ac00af310 2024-09-10T23:40:37.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ab5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ac00b2bb0 0x7f3ac00c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:37.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ab5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ac00b2bb0 0x7f3ac00c4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46780/0 (socket says 172.21.15.94:46780) 2024-09-10T23:40:37.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ab5400640 1 -- 172.21.15.94:0/1930247238 learned_addr learned my addr 172.21.15.94:0/1930247238 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:37.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3abde00640 1 --2- 172.21.15.94:0/1930247238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ac00af310 0x7f3ac00c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:37.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3abe800640 1 --2- 172.21.15.94:0/1930247238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac015f5c0 0x7f3ac015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:37.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ab5400640 1 -- 172.21.15.94:0/1930247238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ac00af310 msgr2=0x7f3ac00c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:37.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ab5400640 1 --2- 172.21.15.94:0/1930247238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ac00af310 0x7f3ac00c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ab5400640 1 -- 172.21.15.94:0/1930247238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac015f5c0 msgr2=0x7f3ac015a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:37.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ab5400640 1 --2- 172.21.15.94:0/1930247238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac015f5c0 0x7f3ac015a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ab5400640 1 -- 172.21.15.94:0/1930247238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac00b4ce0 con 0x7f3ac00b2bb0 2024-09-10T23:40:37.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ab5400640 1 --2- 172.21.15.94:0/1930247238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ac00b2bb0 0x7f3ac00c4e50 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f3aa801ae50 tx=0x7f3aa80a55b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:37.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ab6a00640 1 -- 172.21.15.94:0/1930247238 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80bb070 con 0x7f3ac00b2bb0 2024-09-10T23:40:37.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ab6a00640 1 -- 172.21.15.94:0/1930247238 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3aa8004470 con 0x7f3ac00b2bb0 2024-09-10T23:40:37.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ac4e00640 1 -- 172.21.15.94:0/1930247238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac0151fc0 con 0x7f3ac00b2bb0 2024-09-10T23:40:37.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ab6a00640 1 -- 172.21.15.94:0/1930247238 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80b2040 con 0x7f3ac00b2bb0 2024-09-10T23:40:37.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ac4e00640 1 -- 172.21.15.94:0/1930247238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac00b4fc0 con 0x7f3ac00b2bb0 2024-09-10T23:40:37.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.175+0000 7f3ac4e00640 1 -- 172.21.15.94:0/1930247238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ac00af7a0 con 0x7f3ac00b2bb0 2024-09-10T23:40:37.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.179+0000 7f3ab6a00640 1 -- 172.21.15.94:0/1930247238 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f3aa80052d0 con 0x7f3ac00b2bb0 2024-09-10T23:40:37.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.179+0000 7f3ab6a00640 1 --2- 172.21.15.94:0/1930247238 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3a9c073000 0x7f3a9c0754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:37.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.179+0000 7f3ab6a00640 1 -- 172.21.15.94:0/1930247238 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4971+0+0 (secure 0 0 0) 0x7f3aa812ba50 con 0x7f3ac00b2bb0 2024-09-10T23:40:37.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.179+0000 7f3abde00640 1 --2- 172.21.15.94:0/1930247238 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3a9c073000 0x7f3a9c0754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:37.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.179+0000 7f3abde00640 1 --2- 172.21.15.94:0/1930247238 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3a9c073000 0x7f3a9c0754c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009f20 tx=0x7f3ab00094d0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:37.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.183+0000 7f3ab6a00640 1 -- 172.21.15.94:0/1930247238 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3aa80164b0 con 0x7f3ac00b2bb0 2024-09-10T23:40:37.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.351+0000 7f3ac4e00640 1 -- 172.21.15.94:0/1930247238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ac014f8c0 con 0x7f3ac00b2bb0 2024-09-10T23:40:37.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.351+0000 7f3ab6a00640 1 -- 172.21.15.94:0/1930247238 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) ==== 74+0+13076 (secure 0 0 0) 0x7f3aa80f88e0 con 0x7f3ac00b2bb0 2024-09-10T23:40:37.354 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:37.354 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":154,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:40:36.006070+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6835/2758853868":"2024-09-11T23:40:32.920607+0000","172.21.15.181:6834/2758853868":"2024-09-11T23:40:32.920607+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:40:37.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.351+0000 7f3ac4e00640 1 -- 172.21.15.94:0/1930247238 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3a9c073000 msgr2=0x7f3a9c0754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:37.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.351+0000 7f3ac4e00640 1 --2- 172.21.15.94:0/1930247238 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3a9c073000 0x7f3a9c0754c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009f20 tx=0x7f3ab00094d0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.351+0000 7f3ac4e00640 1 -- 172.21.15.94:0/1930247238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ac00b2bb0 msgr2=0x7f3ac00c4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:37.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.351+0000 7f3ac4e00640 1 --2- 172.21.15.94:0/1930247238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ac00b2bb0 0x7f3ac00c4e50 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f3aa801ae50 tx=0x7f3aa80a55b0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.351+0000 7f3ac4e00640 1 -- 172.21.15.94:0/1930247238 shutdown_connections 2024-09-10T23:40:37.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.351+0000 7f3ac4e00640 1 --2- 172.21.15.94:0/1930247238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ac015f5c0 0x7f3ac015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.351+0000 7f3ac4e00640 1 --2- 172.21.15.94:0/1930247238 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3a9c073000 0x7f3a9c0754c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.351+0000 7f3ac4e00640 1 --2- 172.21.15.94:0/1930247238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ac00b2bb0 0x7f3ac00c4e50 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.351+0000 7f3ac4e00640 1 --2- 172.21.15.94:0/1930247238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ac00af310 0x7f3ac00c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.351+0000 7f3ac4e00640 1 -- 172.21.15.94:0/1930247238 >> 172.21.15.94:0/1930247238 conn(0x7f3ac00d1c60 msgr2=0x7f3ac014fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:37.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.351+0000 7f3ac4e00640 1 -- 172.21.15.94:0/1930247238 shutdown_connections 2024-09-10T23:40:37.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.351+0000 7f3ac4e00640 1 -- 172.21.15.94:0/1930247238 wait complete. 2024-09-10T23:40:37.377 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T23:32:31.372521+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T23:40:37.377 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T23:40:37.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa64e000640 1 -- 172.21.15.94:0/3913480024 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa64815bf60 msgr2=0x7fa6480d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:37.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa64e000640 1 --2- 172.21.15.94:0/3913480024 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa64815bf60 0x7fa6480d2780 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa638009870 tx=0x7fa63801c790 comp rx=0 tx=0).stop 2024-09-10T23:40:37.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa64e000640 1 -- 172.21.15.94:0/3913480024 shutdown_connections 2024-09-10T23:40:37.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa64e000640 1 --2- 172.21.15.94:0/3913480024 >> v2:172.21.15.181:3300/0 conn(0x7fa6480cfce0 0x7fa6480d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa64e000640 1 --2- 172.21.15.94:0/3913480024 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa64815bf60 0x7fa6480d2780 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa64e000640 1 -- 172.21.15.94:0/3913480024 >> 172.21.15.94:0/3913480024 conn(0x7fa6480d1c60 msgr2=0x7fa6480d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:37.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa64e000640 1 -- 172.21.15.94:0/3913480024 shutdown_connections 2024-09-10T23:40:37.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa64e000640 1 -- 172.21.15.94:0/3913480024 wait complete. 2024-09-10T23:40:37.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa64e000640 1 Processor -- start 2024-09-10T23:40:37.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa64e000640 1 -- start start 2024-09-10T23:40:37.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa64e000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6480cfce0 0x7fa648152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:37.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa64e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6480d0660 0x7fa64815d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:37.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa64e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64815bf60 0x7fa64815de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:37.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa64e000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa648168d10 con 0x7fa64815bf60 2024-09-10T23:40:37.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa64e000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa648168b90 con 0x7fa6480d0660 2024-09-10T23:40:37.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa64e000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa648168e90 con 0x7fa6480cfce0 2024-09-10T23:40:37.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa647e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64815bf60 0x7fa64815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:37.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa647e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64815bf60 0x7fa64815de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46796/0 (socket says 172.21.15.94:46796) 2024-09-10T23:40:37.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa647e00640 1 -- 172.21.15.94:0/272060037 learned_addr learned my addr 172.21.15.94:0/272060037 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:37.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa647400640 1 --2- 172.21.15.94:0/272060037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6480cfce0 0x7fa648152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:37.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.519+0000 7fa646a00640 1 --2- 172.21.15.94:0/272060037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6480d0660 0x7fa64815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:37.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.523+0000 7fa647e00640 1 -- 172.21.15.94:0/272060037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6480cfce0 msgr2=0x7fa648152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:37.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.523+0000 7fa647e00640 1 --2- 172.21.15.94:0/272060037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6480cfce0 0x7fa648152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.523+0000 7fa647e00640 1 -- 172.21.15.94:0/272060037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6480d0660 msgr2=0x7fa64815d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:37.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.523+0000 7fa647e00640 1 --2- 172.21.15.94:0/272060037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6480d0660 0x7fa64815d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.523+0000 7fa647e00640 1 -- 172.21.15.94:0/272060037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6481562e0 con 0x7fa64815bf60 2024-09-10T23:40:37.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.523+0000 7fa647e00640 1 --2- 172.21.15.94:0/272060037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64815bf60 0x7fa64815de60 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fa64000cbf0 tx=0x7fa64000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:37.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.523+0000 7fa63f400640 1 -- 172.21.15.94:0/272060037 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640016020 con 0x7fa64815bf60 2024-09-10T23:40:37.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.523+0000 7fa63f400640 1 -- 172.21.15.94:0/272060037 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa64000ab40 con 0x7fa64815bf60 2024-09-10T23:40:37.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.523+0000 7fa63f400640 1 -- 172.21.15.94:0/272060037 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640012420 con 0x7fa64815bf60 2024-09-10T23:40:37.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.523+0000 7fa64e000640 1 -- 172.21.15.94:0/272060037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa648157060 con 0x7fa64815bf60 2024-09-10T23:40:37.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.523+0000 7fa64e000640 1 -- 172.21.15.94:0/272060037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa648156860 con 0x7fa64815bf60 2024-09-10T23:40:37.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.523+0000 7fa64e000640 1 -- 172.21.15.94:0/272060037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6480d0120 con 0x7fa64815bf60 2024-09-10T23:40:37.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.523+0000 7fa63f400640 1 -- 172.21.15.94:0/272060037 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7fa64001b400 con 0x7fa64815bf60 2024-09-10T23:40:37.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.523+0000 7fa63f400640 1 --2- 172.21.15.94:0/272060037 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa628073000 0x7fa6280754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:37.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.523+0000 7fa63f400640 1 -- 172.21.15.94:0/272060037 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4971+0+0 (secure 0 0 0) 0x7fa640092290 con 0x7fa64815bf60 2024-09-10T23:40:37.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.523+0000 7fa647400640 1 --2- 172.21.15.94:0/272060037 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa628073000 0x7fa6280754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:37.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.527+0000 7fa647400640 1 --2- 172.21.15.94:0/272060037 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa628073000 0x7fa6280754c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa6380af000 tx=0x7fa63801afd0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:37.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.527+0000 7fa63f400640 1 -- 172.21.15.94:0/272060037 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa640020020 con 0x7fa64815bf60 2024-09-10T23:40:37.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.695+0000 7fa64e000640 1 -- 172.21.15.94:0/272060037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa648155500 con 0x7fa64815bf60 2024-09-10T23:40:37.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.695+0000 7fa63f400640 1 -- 172.21.15.94:0/272060037 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v154) ==== 93+0+10 (secure 0 0 0) 0x7fa6480d0b10 con 0x7fa64815bf60 2024-09-10T23:40:37.698 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-09-10T23:40:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.695+0000 7fa64e000640 1 -- 172.21.15.94:0/272060037 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa628073000 msgr2=0x7fa6280754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.695+0000 7fa64e000640 1 --2- 172.21.15.94:0/272060037 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa628073000 0x7fa6280754c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa6380af000 tx=0x7fa63801afd0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.695+0000 7fa64e000640 1 -- 172.21.15.94:0/272060037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64815bf60 msgr2=0x7fa64815de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.695+0000 7fa64e000640 1 --2- 172.21.15.94:0/272060037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64815bf60 0x7fa64815de60 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fa64000cbf0 tx=0x7fa64000cce0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.695+0000 7fa64e000640 1 -- 172.21.15.94:0/272060037 shutdown_connections 2024-09-10T23:40:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.695+0000 7fa64e000640 1 --2- 172.21.15.94:0/272060037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6480d0660 0x7fa64815d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.695+0000 7fa64e000640 1 --2- 172.21.15.94:0/272060037 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa628073000 0x7fa6280754c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.695+0000 7fa64e000640 1 --2- 172.21.15.94:0/272060037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa64815bf60 0x7fa64815de60 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.695+0000 7fa64e000640 1 --2- 172.21.15.94:0/272060037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6480cfce0 0x7fa648152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.695+0000 7fa64e000640 1 -- 172.21.15.94:0/272060037 >> 172.21.15.94:0/272060037 conn(0x7fa6480d1c60 msgr2=0x7fa6480ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.695+0000 7fa64e000640 1 -- 172.21.15.94:0/272060037 shutdown_connections 2024-09-10T23:40:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.695+0000 7fa64e000640 1 -- 172.21.15.94:0/272060037 wait complete. 2024-09-10T23:40:37.719 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-10T23:40:37.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.859+0000 7fdb13600640 1 -- 172.21.15.94:0/1228675002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdaf00077e0 msgr2=0x7fdaf0009c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:37.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.859+0000 7fdb13600640 1 --2- 172.21.15.94:0/1228675002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdaf00077e0 0x7fdaf0009c70 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400c310 tx=0x7fdaf401f680 comp rx=0 tx=0).stop 2024-09-10T23:40:37.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.859+0000 7fdb13600640 1 -- 172.21.15.94:0/1228675002 shutdown_connections 2024-09-10T23:40:37.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.859+0000 7fdb13600640 1 --2- 172.21.15.94:0/1228675002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdaf00077e0 0x7fdaf0009c70 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.859+0000 7fdb13600640 1 -- 172.21.15.94:0/1228675002 >> 172.21.15.94:0/1228675002 conn(0x7fdb0c06cdf0 msgr2=0x7fdb0c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:37.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.859+0000 7fdb13600640 1 -- 172.21.15.94:0/1228675002 shutdown_connections 2024-09-10T23:40:37.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.859+0000 7fdb13600640 1 -- 172.21.15.94:0/1228675002 wait complete. 2024-09-10T23:40:37.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.859+0000 7fdb13600640 1 Processor -- start 2024-09-10T23:40:37.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.859+0000 7fdb13600640 1 -- start start 2024-09-10T23:40:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.859+0000 7fdb13600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf00077e0 0x7fdb0c156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.859+0000 7fdb13600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb0c1efbe0 0x7fdb0c1f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.859+0000 7fdb13600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb0c1f6290 0x7fdb0c1f2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.859+0000 7fdb13600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c0cfe90 con 0x7fdb0c1f6290 2024-09-10T23:40:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.859+0000 7fdb13600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c0d0010 con 0x7fdb0c1efbe0 2024-09-10T23:40:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.859+0000 7fdb13600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0c0d0190 con 0x7fdaf00077e0 2024-09-10T23:40:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb11600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb0c1f6290 0x7fdb0c1f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb11600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb0c1f6290 0x7fdb0c1f2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46812/0 (socket says 172.21.15.94:46812) 2024-09-10T23:40:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb11600640 1 -- 172.21.15.94:0/3224281205 learned_addr learned my addr 172.21.15.94:0/3224281205 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb10c00640 1 --2- 172.21.15.94:0/3224281205 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf00077e0 0x7fdb0c156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb03e00640 1 --2- 172.21.15.94:0/3224281205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb0c1efbe0 0x7fdb0c1f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb11600640 1 -- 172.21.15.94:0/3224281205 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf00077e0 msgr2=0x7fdb0c156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb11600640 1 --2- 172.21.15.94:0/3224281205 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf00077e0 0x7fdb0c156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb11600640 1 -- 172.21.15.94:0/3224281205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb0c1efbe0 msgr2=0x7fdb0c1f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb11600640 1 --2- 172.21.15.94:0/3224281205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb0c1efbe0 0x7fdb0c1f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb11600640 1 -- 172.21.15.94:0/3224281205 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0c1565b0 con 0x7fdb0c1f6290 2024-09-10T23:40:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb03e00640 1 --2- 172.21.15.94:0/3224281205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb0c1efbe0 0x7fdb0c1f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:40:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb11600640 1 --2- 172.21.15.94:0/3224281205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb0c1f6290 0x7fdb0c1f2760 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c1404d0 tx=0x7fdb0400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb09600640 1 -- 172.21.15.94:0/3224281205 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0400eea0 con 0x7fdb0c1f6290 2024-09-10T23:40:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb09600640 1 -- 172.21.15.94:0/3224281205 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb04015070 con 0x7fdb0c1f6290 2024-09-10T23:40:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb09600640 1 -- 172.21.15.94:0/3224281205 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04004030 con 0x7fdb0c1f6290 2024-09-10T23:40:37.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb13600640 1 -- 172.21.15.94:0/3224281205 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0c1f3490 con 0x7fdb0c1f6290 2024-09-10T23:40:37.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb13600640 1 -- 172.21.15.94:0/3224281205 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb0c152850 con 0x7fdb0c1f6290 2024-09-10T23:40:37.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb13600640 1 -- 172.21.15.94:0/3224281205 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb0c15dcf0 con 0x7fdb0c1f6290 2024-09-10T23:40:37.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.863+0000 7fdb09600640 1 -- 172.21.15.94:0/3224281205 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7fdb040041d0 con 0x7fdb0c1f6290 2024-09-10T23:40:37.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.867+0000 7fdb09600640 1 --2- 172.21.15.94:0/3224281205 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdae8073030 0x7fdae80754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:37.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.867+0000 7fdb09600640 1 -- 172.21.15.94:0/3224281205 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4971+0+0 (secure 0 0 0) 0x7fdb040929b0 con 0x7fdb0c1f6290 2024-09-10T23:40:37.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.867+0000 7fdb10c00640 1 --2- 172.21.15.94:0/3224281205 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdae8073030 0x7fdae80754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:37.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.867+0000 7fdb10c00640 1 --2- 172.21.15.94:0/3224281205 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdae8073030 0x7fdae80754f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c140470 tx=0x7fdafc006cb0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:37.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:37.871+0000 7fdb09600640 1 -- 172.21.15.94:0/3224281205 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdb0405f270 con 0x7fdb0c1f6290 2024-09-10T23:40:38.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:38.095+0000 7fdb13600640 1 -- 172.21.15.94:0/3224281205 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fdb0c0cf860 con 0x7fdb0c1f6290 2024-09-10T23:40:39.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.015+0000 7fdb09600640 1 -- 172.21.15.94:0/3224281205 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v443) ==== 166+0+0 (secure 0 0 0) 0x7fdb04063ca0 con 0x7fdb0c1f6290 2024-09-10T23:40:39.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.019+0000 7fdb13600640 1 -- 172.21.15.94:0/3224281205 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdae8073030 msgr2=0x7fdae80754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:39.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.019+0000 7fdb13600640 1 --2- 172.21.15.94:0/3224281205 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdae8073030 0x7fdae80754f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c140470 tx=0x7fdafc006cb0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.019+0000 7fdb13600640 1 -- 172.21.15.94:0/3224281205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb0c1f6290 msgr2=0x7fdb0c1f2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:39.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.019+0000 7fdb13600640 1 --2- 172.21.15.94:0/3224281205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb0c1f6290 0x7fdb0c1f2760 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c1404d0 tx=0x7fdb0400e720 comp rx=0 tx=0).stop 2024-09-10T23:40:39.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.019+0000 7fdb13600640 1 -- 172.21.15.94:0/3224281205 shutdown_connections 2024-09-10T23:40:39.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.019+0000 7fdb13600640 1 --2- 172.21.15.94:0/3224281205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb0c1efbe0 0x7fdb0c1f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.019+0000 7fdb13600640 1 --2- 172.21.15.94:0/3224281205 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdae8073030 0x7fdae80754f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.019+0000 7fdb13600640 1 --2- 172.21.15.94:0/3224281205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb0c1f6290 0x7fdb0c1f2760 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.019+0000 7fdb13600640 1 --2- 172.21.15.94:0/3224281205 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdaf00077e0 0x7fdb0c156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.019+0000 7fdb13600640 1 -- 172.21.15.94:0/3224281205 >> 172.21.15.94:0/3224281205 conn(0x7fdb0c06cdf0 msgr2=0x7fdb0c0b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:39.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.019+0000 7fdb13600640 1 -- 172.21.15.94:0/3224281205 shutdown_connections 2024-09-10T23:40:39.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.019+0000 7fdb13600640 1 -- 172.21.15.94:0/3224281205 wait complete. 2024-09-10T23:40:39.046 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T23:40:39.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.187+0000 7f8f2e600640 1 -- 172.21.15.94:0/4066866873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f2815ba50 msgr2=0x7f8f280b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:39.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.187+0000 7f8f2e600640 1 --2- 172.21.15.94:0/4066866873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f2815ba50 0x7f8f280b4700 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009870 tx=0x7f8f1801c6c0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.187+0000 7f8f2e600640 1 -- 172.21.15.94:0/4066866873 shutdown_connections 2024-09-10T23:40:39.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.187+0000 7f8f2e600640 1 --2- 172.21.15.94:0/4066866873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f2815ba50 0x7f8f280b4700 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.187+0000 7f8f2e600640 1 -- 172.21.15.94:0/4066866873 >> 172.21.15.94:0/4066866873 conn(0x7f8f2806cdf0 msgr2=0x7f8f2806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:39.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.187+0000 7f8f2e600640 1 -- 172.21.15.94:0/4066866873 shutdown_connections 2024-09-10T23:40:39.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.187+0000 7f8f2e600640 1 -- 172.21.15.94:0/4066866873 wait complete. 2024-09-10T23:40:39.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f2e600640 1 Processor -- start 2024-09-10T23:40:39.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f2e600640 1 -- start start 2024-09-10T23:40:39.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f2e600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f280b4c40 0x7f8f281462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:39.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f2e600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f280d03c0 0x7f8f2814a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:39.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f2e600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f2815ba50 0x7f8f2814ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:39.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f2e600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8f28169c40 con 0x7f8f280d03c0 2024-09-10T23:40:39.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f2e600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8f28169ac0 con 0x7f8f280b4c40 2024-09-10T23:40:39.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f2e600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8f28169dc0 con 0x7f8f2815ba50 2024-09-10T23:40:39.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f1ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f280d03c0 0x7f8f2814a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:39.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f27400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f280b4c40 0x7f8f281462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:39.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f1ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f280d03c0 0x7f8f2814a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46816/0 (socket says 172.21.15.94:46816) 2024-09-10T23:40:39.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f1ea00640 1 -- 172.21.15.94:0/1734709643 learned_addr learned my addr 172.21.15.94:0/1734709643 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:39.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f27e00640 1 --2- 172.21.15.94:0/1734709643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f2815ba50 0x7f8f2814ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:39.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f27400640 1 -- 172.21.15.94:0/1734709643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f2815ba50 msgr2=0x7f8f2814ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:39.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f27400640 1 --2- 172.21.15.94:0/1734709643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f2815ba50 0x7f8f2814ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f27400640 1 -- 172.21.15.94:0/1734709643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f280d03c0 msgr2=0x7f8f2814a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:39.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f27400640 1 --2- 172.21.15.94:0/1734709643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f280d03c0 0x7f8f2814a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f27400640 1 -- 172.21.15.94:0/1734709643 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2814d2a0 con 0x7f8f280b4c40 2024-09-10T23:40:39.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f1ea00640 1 --2- 172.21.15.94:0/1734709643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f280d03c0 0x7f8f2814a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:40:39.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f27400640 1 --2- 172.21.15.94:0/1734709643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f280b4c40 0x7f8f281462d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8f180ac000 tx=0x7f8f18005d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:39.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f1fe00640 1 -- 172.21.15.94:0/1734709643 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18005220 con 0x7f8f280b4c40 2024-09-10T23:40:39.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f1fe00640 1 -- 172.21.15.94:0/1734709643 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f180054b0 con 0x7f8f280b4c40 2024-09-10T23:40:39.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f1fe00640 1 -- 172.21.15.94:0/1734709643 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f180a7b20 con 0x7f8f280b4c40 2024-09-10T23:40:39.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f2e600640 1 -- 172.21.15.94:0/1734709643 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f281f59b0 con 0x7f8f280b4c40 2024-09-10T23:40:39.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.191+0000 7f8f2e600640 1 -- 172.21.15.94:0/1734709643 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f281f5f70 con 0x7f8f280b4c40 2024-09-10T23:40:39.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.195+0000 7f8f2e600640 1 -- 172.21.15.94:0/1734709643 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f280b4700 con 0x7f8f280b4c40 2024-09-10T23:40:39.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.195+0000 7f8f1fe00640 1 -- 172.21.15.94:0/1734709643 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f8f180a7cc0 con 0x7f8f280b4c40 2024-09-10T23:40:39.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.195+0000 7f8f1fe00640 1 --2- 172.21.15.94:0/1734709643 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8f04073000 0x7f8f040754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:39.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.195+0000 7f8f1fe00640 1 -- 172.21.15.94:0/1734709643 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4971+0+0 (secure 0 0 0) 0x7f8f1812c350 con 0x7f8f280b4c40 2024-09-10T23:40:39.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.195+0000 7f8f1ea00640 1 --2- 172.21.15.94:0/1734709643 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8f04073000 0x7f8f040754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:39.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.195+0000 7f8f1ea00640 1 --2- 172.21.15.94:0/1734709643 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8f04073000 0x7f8f040754c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8f2814bd30 tx=0x7f8f10009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:39.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.199+0000 7f8f1fe00640 1 -- 172.21.15.94:0/1734709643 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8f18016370 con 0x7f8f280b4c40 2024-09-10T23:40:39.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.363+0000 7f8f2e600640 1 -- 172.21.15.94:0/1734709643 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f8f280cf840 con 0x7f8f280b4c40 2024-09-10T23:40:39.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.367+0000 7f8f1fe00640 1 -- 172.21.15.94:0/1734709643 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v47) ==== 59+0+1998 (secure 0 0 0) 0x7f8f180f8c10 con 0x7f8f280b4c40 2024-09-10T23:40:39.370 INFO:teuthology.orchestra.run.smithi094.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T23:40:39.370 INFO:teuthology.orchestra.run.smithi094.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T23:40:39.370 INFO:teuthology.orchestra.run.smithi094.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T23:40:39.370 INFO:teuthology.orchestra.run.smithi094.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T23:40:39.370 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3944.074506 2024-09-10T23:40:39.370 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3865.464527 2024-09-10T23:40:39.370 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3898.861024 2024-09-10T23:40:39.370 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4010.048529 2024-09-10T23:40:39.370 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4650.562436 2024-09-10T23:40:39.370 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4724.018776 2024-09-10T23:40:39.370 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5255.753663 2024-09-10T23:40:39.371 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4718.009443 2024-09-10T23:40:39.371 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds 20/20 2024-09-10T23:40:39.371 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T23:40:39.371 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced debug_ms 2/2 2024-09-10T23:40:39.371 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_frag true 2024-09-10T23:40:39.371 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_debug_scatterstat true 2024-09-10T23:40:39.371 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T23:40:39.371 INFO:teuthology.orchestra.run.smithi094.stdout:mds dev mds_verify_scatter true 2024-09-10T23:40:39.371 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T23:40:39.371 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T23:40:39.371 INFO:teuthology.orchestra.run.smithi094.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T23:40:39.371 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced client_mount_timeout 600 2024-09-10T23:40:39.371 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_client 20/20 2024-09-10T23:40:39.372 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced debug_ms 1/1 2024-09-10T23:40:39.372 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T23:40:39.372 INFO:teuthology.orchestra.run.smithi094.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T23:40:39.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.367+0000 7f8f2e600640 1 -- 172.21.15.94:0/1734709643 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8f04073000 msgr2=0x7f8f040754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:39.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.367+0000 7f8f2e600640 1 --2- 172.21.15.94:0/1734709643 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8f04073000 0x7f8f040754c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8f2814bd30 tx=0x7f8f10009040 comp rx=0 tx=0).stop 2024-09-10T23:40:39.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.367+0000 7f8f2e600640 1 -- 172.21.15.94:0/1734709643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f280b4c40 msgr2=0x7f8f281462d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:39.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.367+0000 7f8f2e600640 1 --2- 172.21.15.94:0/1734709643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f280b4c40 0x7f8f281462d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8f180ac000 tx=0x7f8f18005d20 comp rx=0 tx=0).stop 2024-09-10T23:40:39.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.367+0000 7f8f2e600640 1 -- 172.21.15.94:0/1734709643 shutdown_connections 2024-09-10T23:40:39.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.367+0000 7f8f2e600640 1 --2- 172.21.15.94:0/1734709643 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8f04073000 0x7f8f040754c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.367+0000 7f8f2e600640 1 --2- 172.21.15.94:0/1734709643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f2815ba50 0x7f8f2814ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.367+0000 7f8f2e600640 1 --2- 172.21.15.94:0/1734709643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f280d03c0 0x7f8f2814a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.367+0000 7f8f2e600640 1 --2- 172.21.15.94:0/1734709643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f280b4c40 0x7f8f281462d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.367+0000 7f8f2e600640 1 -- 172.21.15.94:0/1734709643 >> 172.21.15.94:0/1734709643 conn(0x7f8f2806cdf0 msgr2=0x7f8f280b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:39.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.367+0000 7f8f2e600640 1 -- 172.21.15.94:0/1734709643 shutdown_connections 2024-09-10T23:40:39.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.367+0000 7f8f2e600640 1 -- 172.21.15.94:0/1734709643 wait complete. 2024-09-10T23:40:39.391 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:40:39.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.531+0000 7fe15b200640 1 -- 172.21.15.94:0/2425930960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe15c0b2ac0 msgr2=0x7fe15c0b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:39.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.531+0000 7fe15b200640 1 --2- 172.21.15.94:0/2425930960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe15c0b2ac0 0x7fe15c0b1880 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fe148009900 tx=0x7fe14801c8b0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.531+0000 7fe15b200640 1 -- 172.21.15.94:0/2425930960 shutdown_connections 2024-09-10T23:40:39.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.531+0000 7fe15b200640 1 --2- 172.21.15.94:0/2425930960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe15c0af150 0x7fe15c0af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.531+0000 7fe15b200640 1 --2- 172.21.15.94:0/2425930960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe15c0b2ac0 0x7fe15c0b1880 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.531+0000 7fe15b200640 1 -- 172.21.15.94:0/2425930960 >> 172.21.15.94:0/2425930960 conn(0x7fe15c0d1b00 msgr2=0x7fe15c0d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:39.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.531+0000 7fe15b200640 1 -- 172.21.15.94:0/2425930960 shutdown_connections 2024-09-10T23:40:39.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.531+0000 7fe15b200640 1 -- 172.21.15.94:0/2425930960 wait complete. 2024-09-10T23:40:39.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.531+0000 7fe15b200640 1 Processor -- start 2024-09-10T23:40:39.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe15b200640 1 -- start start 2024-09-10T23:40:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe15b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe15c0af150 0x7fe15c14f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe15b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe15c0b2ac0 0x7fe15c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe15b200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe15c15f140 0x7fe15c1551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe15b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c168ad0 con 0x7fe15c0b2ac0 2024-09-10T23:40:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe15b200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c168950 con 0x7fe15c0af150 2024-09-10T23:40:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe15b200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe15c168c50 con 0x7fe15c15f140 2024-09-10T23:40:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe159400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe15c0b2ac0 0x7fe15c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe159400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe15c0b2ac0 0x7fe15c06e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46832/0 (socket says 172.21.15.94:46832) 2024-09-10T23:40:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe159400640 1 -- 172.21.15.94:0/2309042621 learned_addr learned my addr 172.21.15.94:0/2309042621 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe159e00640 1 --2- 172.21.15.94:0/2309042621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe15c0af150 0x7fe15c14f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe15a800640 1 --2- 172.21.15.94:0/2309042621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe15c15f140 0x7fe15c1551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe159400640 1 -- 172.21.15.94:0/2309042621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe15c15f140 msgr2=0x7fe15c1551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe159400640 1 --2- 172.21.15.94:0/2309042621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe15c15f140 0x7fe15c1551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe159400640 1 -- 172.21.15.94:0/2309042621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe15c0af150 msgr2=0x7fe15c14f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe159400640 1 --2- 172.21.15.94:0/2309042621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe15c0af150 0x7fe15c14f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe159400640 1 -- 172.21.15.94:0/2309042621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15c1f7960 con 0x7fe15c0b2ac0 2024-09-10T23:40:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe15a800640 1 --2- 172.21.15.94:0/2309042621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe15c15f140 0x7fe15c1551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe159400640 1 --2- 172.21.15.94:0/2309042621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe15c0b2ac0 0x7fe15c06e410 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fe148009850 tx=0x7fe1480a5880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe152000640 1 -- 172.21.15.94:0/2309042621 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1480b2040 con 0x7fe15c0b2ac0 2024-09-10T23:40:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe152000640 1 -- 172.21.15.94:0/2309042621 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe148005320 con 0x7fe15c0b2ac0 2024-09-10T23:40:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe15b200640 1 -- 172.21.15.94:0/2309042621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15c1f7c40 con 0x7fe15c0b2ac0 2024-09-10T23:40:39.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe152000640 1 -- 172.21.15.94:0/2309042621 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148003700 con 0x7fe15c0b2ac0 2024-09-10T23:40:39.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe15b200640 1 -- 172.21.15.94:0/2309042621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe15c1f8200 con 0x7fe15c0b2ac0 2024-09-10T23:40:39.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.535+0000 7fe15b200640 1 -- 172.21.15.94:0/2309042621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe15c0af5e0 con 0x7fe15c0b2ac0 2024-09-10T23:40:39.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.539+0000 7fe152000640 1 -- 172.21.15.94:0/2309042621 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7fe1480038c0 con 0x7fe15c0b2ac0 2024-09-10T23:40:39.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.539+0000 7fe152000640 1 --2- 172.21.15.94:0/2309042621 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe13c073000 0x7fe13c0754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:39.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.539+0000 7fe152000640 1 -- 172.21.15.94:0/2309042621 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4971+0+0 (secure 0 0 0) 0x7fe14812c3f0 con 0x7fe15c0b2ac0 2024-09-10T23:40:39.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.539+0000 7fe159e00640 1 --2- 172.21.15.94:0/2309042621 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe13c073000 0x7fe13c0754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:39.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.539+0000 7fe159e00640 1 --2- 172.21.15.94:0/2309042621 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe13c073000 0x7fe13c0754c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe14c000f70 tx=0x7fe14c009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:39.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.543+0000 7fe152000640 1 -- 172.21.15.94:0/2309042621 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe1480164d0 con 0x7fe15c0b2ac0 2024-09-10T23:40:39.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.707+0000 7fe15b200640 1 -- 172.21.15.94:0/2309042621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe15c15d930 con 0x7fe15c0b2ac0 2024-09-10T23:40:39.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.711+0000 7fe152000640 1 -- 172.21.15.94:0/2309042621 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7fe1480f9280 con 0x7fe15c0b2ac0 2024-09-10T23:40:39.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.715+0000 7fe15b200640 1 -- 172.21.15.94:0/2309042621 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe13c073000 msgr2=0x7fe13c0754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:39.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.715+0000 7fe15b200640 1 --2- 172.21.15.94:0/2309042621 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe13c073000 0x7fe13c0754c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe14c000f70 tx=0x7fe14c009040 comp rx=0 tx=0).stop 2024-09-10T23:40:39.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.715+0000 7fe15b200640 1 -- 172.21.15.94:0/2309042621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe15c0b2ac0 msgr2=0x7fe15c06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:39.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.715+0000 7fe15b200640 1 --2- 172.21.15.94:0/2309042621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe15c0b2ac0 0x7fe15c06e410 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fe148009850 tx=0x7fe1480a5880 comp rx=0 tx=0).stop 2024-09-10T23:40:39.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.715+0000 7fe15b200640 1 -- 172.21.15.94:0/2309042621 shutdown_connections 2024-09-10T23:40:39.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.715+0000 7fe15b200640 1 --2- 172.21.15.94:0/2309042621 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe13c073000 0x7fe13c0754c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.715+0000 7fe15b200640 1 --2- 172.21.15.94:0/2309042621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe15c15f140 0x7fe15c1551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.715+0000 7fe15b200640 1 --2- 172.21.15.94:0/2309042621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe15c0b2ac0 0x7fe15c06e410 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.715+0000 7fe15b200640 1 --2- 172.21.15.94:0/2309042621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe15c0af150 0x7fe15c14f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:39.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.715+0000 7fe15b200640 1 -- 172.21.15.94:0/2309042621 >> 172.21.15.94:0/2309042621 conn(0x7fe15c0d1b00 msgr2=0x7fe15c070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:39.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.715+0000 7fe15b200640 1 -- 172.21.15.94:0/2309042621 shutdown_connections 2024-09-10T23:40:39.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:39.715+0000 7fe15b200640 1 -- 172.21.15.94:0/2309042621 wait complete. 2024-09-10T23:40:39.738 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:40:39.739 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:40:39.843 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:40:39.955 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:40:39.955 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:40:39.955 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:40:39.964 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:40:39.964 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:40:39.964 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:40:39.973 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:40:39.973 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:40:39.973 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:40:40.016 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:40:40.016 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:40:40.016 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:40:40.026 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:40:40.026 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T23:40:40.026 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:40:40.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0577200640 1 -- 172.21.15.94:0/2117025946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f057015cbc0 msgr2=0x7f05700cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:40.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0577200640 1 --2- 172.21.15.94:0/2117025946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f057015cbc0 0x7f05700cde20 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f05700cd3d0 tx=0x7f055801c8c0 comp rx=0 tx=0).stop 2024-09-10T23:40:40.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0577200640 1 -- 172.21.15.94:0/2117025946 shutdown_connections 2024-09-10T23:40:40.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0577200640 1 --2- 172.21.15.94:0/2117025946 >> v2:172.21.15.181:3300/0 conn(0x7f05700cfd50 0x7f05700d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:40.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0577200640 1 --2- 172.21.15.94:0/2117025946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f057015cbc0 0x7f05700cde20 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:40.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0577200640 1 -- 172.21.15.94:0/2117025946 >> 172.21.15.94:0/2117025946 conn(0x7f057006cdf0 msgr2=0x7f057006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:40.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0577200640 1 -- 172.21.15.94:0/2117025946 shutdown_connections 2024-09-10T23:40:40.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0577200640 1 -- 172.21.15.94:0/2117025946 wait complete. 2024-09-10T23:40:40.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0577200640 1 Processor -- start 2024-09-10T23:40:40.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0577200640 1 -- start start 2024-09-10T23:40:40.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0577200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05700cfd50 0x7f05700c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:40.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0577200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570152c10 0x7f0570151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:40.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0577200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f057015cbc0 0x7f0570151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:40.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0577200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0570168c00 con 0x7f05700cfd50 2024-09-10T23:40:40.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0577200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0570168a80 con 0x7f0570152c10 2024-09-10T23:40:40.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0577200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0570168d80 con 0x7f057015cbc0 2024-09-10T23:40:40.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f056fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05700cfd50 0x7f05700c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:40.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f056fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05700cfd50 0x7f05700c7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46858/0 (socket says 172.21.15.94:46858) 2024-09-10T23:40:40.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f056fe00640 1 -- 172.21.15.94:0/3841409252 learned_addr learned my addr 172.21.15.94:0/3841409252 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:40.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0575200640 1 --2- 172.21.15.94:0/3841409252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f057015cbc0 0x7f0570151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:40.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f056fe00640 1 -- 172.21.15.94:0/3841409252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f057015cbc0 msgr2=0x7f0570151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:40.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0567400640 1 --2- 172.21.15.94:0/3841409252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570152c10 0x7f0570151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:40.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f056fe00640 1 --2- 172.21.15.94:0/3841409252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f057015cbc0 0x7f0570151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:40.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f056fe00640 1 -- 172.21.15.94:0/3841409252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570152c10 msgr2=0x7f0570151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:40.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f056fe00640 1 --2- 172.21.15.94:0/3841409252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570152c10 0x7f0570151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:40.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f056fe00640 1 -- 172.21.15.94:0/3841409252 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0570145fd0 con 0x7f05700cfd50 2024-09-10T23:40:40.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0567400640 1 --2- 172.21.15.94:0/3841409252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570152c10 0x7f0570151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:40.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.195+0000 7f0575200640 1 --2- 172.21.15.94:0/3841409252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f057015cbc0 0x7f0570151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:40.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.199+0000 7f056fe00640 1 --2- 172.21.15.94:0/3841409252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05700cfd50 0x7f05700c7bf0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f056000c7c0 tx=0x7f056000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:40.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.199+0000 7f056cc00640 1 -- 172.21.15.94:0/3841409252 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560016020 con 0x7f05700cfd50 2024-09-10T23:40:40.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.199+0000 7f056cc00640 1 -- 172.21.15.94:0/3841409252 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f056000ab40 con 0x7f05700cfd50 2024-09-10T23:40:40.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.199+0000 7f056cc00640 1 -- 172.21.15.94:0/3841409252 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560012420 con 0x7f05700cfd50 2024-09-10T23:40:40.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.199+0000 7f0577200640 1 -- 172.21.15.94:0/3841409252 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05701462b0 con 0x7f05700cfd50 2024-09-10T23:40:40.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.199+0000 7f0577200640 1 -- 172.21.15.94:0/3841409252 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05700c9a80 con 0x7f05700cfd50 2024-09-10T23:40:40.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.199+0000 7f0577200640 1 -- 172.21.15.94:0/3841409252 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05700d0200 con 0x7f05700cfd50 2024-09-10T23:40:40.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.199+0000 7f056cc00640 1 -- 172.21.15.94:0/3841409252 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f056001b400 con 0x7f05700cfd50 2024-09-10T23:40:40.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.199+0000 7f056cc00640 1 --2- 172.21.15.94:0/3841409252 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f054c073000 0x7f054c0754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:40.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.199+0000 7f056cc00640 1 -- 172.21.15.94:0/3841409252 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4971+0+0 (secure 0 0 0) 0x7f0560092170 con 0x7f05700cfd50 2024-09-10T23:40:40.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.203+0000 7f0567400640 1 --2- 172.21.15.94:0/3841409252 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f054c073000 0x7f054c0754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:40.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.203+0000 7f0567400640 1 --2- 172.21.15.94:0/3841409252 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f054c073000 0x7f054c0754c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f05580af000 tx=0x7f055801b100 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:40.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.203+0000 7f056cc00640 1 -- 172.21.15.94:0/3841409252 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0560020030 con 0x7f05700cfd50 2024-09-10T23:40:40.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.403+0000 7f0577200640 1 -- 172.21.15.94:0/3841409252 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05700b46c0 con 0x7f05700cfd50 2024-09-10T23:40:40.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.403+0000 7f056cc00640 1 -- 172.21.15.94:0/3841409252 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+3922 (secure 0 0 0) 0x7f056005f080 con 0x7f05700cfd50 2024-09-10T23:40:40.407 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:40.407 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":110,"btime":"2024-09-10T23:40:33:994191+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8158,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1859150794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":1859150794},{"type":"v1","addr":"172.21.15.94:6835","nonce":1859150794}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8173,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/3891720008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3891720008},{"type":"v1","addr":"172.21.15.181:6801","nonce":3891720008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8431,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1579703004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1579703004},{"type":"v1","addr":"172.21.15.181:6835","nonce":1579703004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":8787,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/4040262257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":4040262257},{"type":"v1","addr":"172.21.15.94:6834","nonce":4040262257}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101}],"filesystems":[]} 2024-09-10T23:40:40.407 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 110 2024-09-10T23:40:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.403+0000 7f0577200640 1 -- 172.21.15.94:0/3841409252 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f054c073000 msgr2=0x7f054c0754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.403+0000 7f0577200640 1 --2- 172.21.15.94:0/3841409252 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f054c073000 0x7f054c0754c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f05580af000 tx=0x7f055801b100 comp rx=0 tx=0).stop 2024-09-10T23:40:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.403+0000 7f0577200640 1 -- 172.21.15.94:0/3841409252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05700cfd50 msgr2=0x7f05700c7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.403+0000 7f0577200640 1 --2- 172.21.15.94:0/3841409252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05700cfd50 0x7f05700c7bf0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f056000c7c0 tx=0x7f056000cc80 comp rx=0 tx=0).stop 2024-09-10T23:40:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.403+0000 7f0577200640 1 -- 172.21.15.94:0/3841409252 shutdown_connections 2024-09-10T23:40:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.403+0000 7f0577200640 1 --2- 172.21.15.94:0/3841409252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0570152c10 0x7f0570151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:40.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.407+0000 7f0577200640 1 --2- 172.21.15.94:0/3841409252 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f054c073000 0x7f054c0754c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.407+0000 7f0577200640 1 --2- 172.21.15.94:0/3841409252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f057015cbc0 0x7f0570151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.407+0000 7f0577200640 1 --2- 172.21.15.94:0/3841409252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05700cfd50 0x7f05700c7bf0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.407+0000 7f0577200640 1 -- 172.21.15.94:0/3841409252 >> 172.21.15.94:0/3841409252 conn(0x7f057006cdf0 msgr2=0x7f05700b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.407+0000 7f0577200640 1 -- 172.21.15.94:0/3841409252 shutdown_connections 2024-09-10T23:40:40.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.407+0000 7f0577200640 1 -- 172.21.15.94:0/3841409252 wait complete. 2024-09-10T23:40:40.428 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T23:40:40.428 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T23:40:40.428 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T23:40:40.428 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:40:40.428 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T23:40:40.429 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T23:40:40.429 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T23:40:40.429 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:40:40.429 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T23:40:40.429 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T23:40:40.429 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T23:40:40.429 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:40:40.429 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T23:40:40.430 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T23:40:40.430 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T23:40:40.430 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:40:40.441 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T23:40:40.441 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T23:40:40.442 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T23:40:40.442 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T23:40:40.443 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T23:40:40.444 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T23:40:40.445 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T23:40:40.445 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T23:40:40.486 INFO:tasks.ceph.mds.a:Started 2024-09-10T23:40:40.487 DEBUG:teuthology.parallel:result is None 2024-09-10T23:40:40.488 INFO:tasks.ceph.mds.c:Started 2024-09-10T23:40:40.489 DEBUG:teuthology.parallel:result is None 2024-09-10T23:40:40.494 INFO:tasks.ceph.mds.b:Started 2024-09-10T23:40:40.494 DEBUG:teuthology.parallel:result is None 2024-09-10T23:40:40.496 INFO:tasks.ceph.mds.d:Started 2024-09-10T23:40:40.496 DEBUG:teuthology.parallel:result is None 2024-09-10T23:40:40.496 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T23:40:40.574 INFO:tasks.ceph.mds.c.smithi094.stdout:starting mds.c at 2024-09-10T23:40:40.575 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-09-10T23:40:40.585 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2024-09-10T23:40:40.585 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2024-09-10T23:40:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.643+0000 7f1775400640 1 -- 172.21.15.94:0/4214967493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17701455d0 msgr2=0x7f17700c6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.643+0000 7f1775400640 1 --2- 172.21.15.94:0/4214967493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17701455d0 0x7f17700c6460 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f17700d0b70 tx=0x7f176001c8c0 comp rx=0 tx=0).stop 2024-09-10T23:40:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.643+0000 7f1775400640 1 -- 172.21.15.94:0/4214967493 shutdown_connections 2024-09-10T23:40:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.643+0000 7f1775400640 1 --2- 172.21.15.94:0/4214967493 >> v2:172.21.15.181:3300/0 conn(0x7f17700cd9f0 0x7f17700cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:40.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.643+0000 7f1775400640 1 --2- 172.21.15.94:0/4214967493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17701455d0 0x7f17700c6460 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:40.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.643+0000 7f1775400640 1 -- 172.21.15.94:0/4214967493 >> 172.21.15.94:0/4214967493 conn(0x7f17700cfaf0 msgr2=0x7f17700cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:40.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.643+0000 7f1775400640 1 -- 172.21.15.94:0/4214967493 shutdown_connections 2024-09-10T23:40:40.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.643+0000 7f1775400640 1 -- 172.21.15.94:0/4214967493 wait complete. 2024-09-10T23:40:40.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.643+0000 7f1775400640 1 Processor -- start 2024-09-10T23:40:40.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.643+0000 7f1775400640 1 -- start start 2024-09-10T23:40:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f1775400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17700cd9f0 0x7f1770159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f1775400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17700d0710 0x7f17700ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f1775400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17701455d0 0x7f17700c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f1775400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f17701690e0 con 0x7f17700d0710 2024-09-10T23:40:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f1775400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1770168f60 con 0x7f17700cd9f0 2024-09-10T23:40:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f1775400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1770169260 con 0x7f17701455d0 2024-09-10T23:40:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f176de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17700d0710 0x7f17700ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f176de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17700d0710 0x7f17700ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46894/0 (socket says 172.21.15.94:46894) 2024-09-10T23:40:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f176de00640 1 -- 172.21.15.94:0/549276122 learned_addr learned my addr 172.21.15.94:0/549276122 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f176e800640 1 --2- 172.21.15.94:0/549276122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17700cd9f0 0x7f1770159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f176f200640 1 --2- 172.21.15.94:0/549276122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17701455d0 0x7f17700c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f176de00640 1 -- 172.21.15.94:0/549276122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17701455d0 msgr2=0x7f17700c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f176de00640 1 --2- 172.21.15.94:0/549276122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17701455d0 0x7f17700c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f176de00640 1 -- 172.21.15.94:0/549276122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17700cd9f0 msgr2=0x7f1770159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f176de00640 1 --2- 172.21.15.94:0/549276122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17700cd9f0 0x7f1770159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f176de00640 1 -- 172.21.15.94:0/549276122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17700cb770 con 0x7f17700d0710 2024-09-10T23:40:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f176de00640 1 --2- 172.21.15.94:0/549276122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17700d0710 0x7f17700ca140 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f175c00c4e0 tx=0x7f175c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f1766a00640 1 -- 172.21.15.94:0/549276122 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c0052d0 con 0x7f17700d0710 2024-09-10T23:40:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f1766a00640 1 -- 172.21.15.94:0/549276122 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f175c005470 con 0x7f17700d0710 2024-09-10T23:40:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f1775400640 1 -- 172.21.15.94:0/549276122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17700cba50 con 0x7f17700d0710 2024-09-10T23:40:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f1766a00640 1 -- 172.21.15.94:0/549276122 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c012890 con 0x7f17700d0710 2024-09-10T23:40:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f1775400640 1 -- 172.21.15.94:0/549276122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17700cc010 con 0x7f17700d0710 2024-09-10T23:40:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.647+0000 7f1775400640 1 -- 172.21.15.94:0/549276122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17700cde30 con 0x7f17700d0710 2024-09-10T23:40:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.651+0000 7f1766a00640 1 -- 172.21.15.94:0/549276122 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f175c003600 con 0x7f17700d0710 2024-09-10T23:40:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.651+0000 7f1766a00640 1 --2- 172.21.15.94:0/549276122 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1750073000 0x7f17500754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.651+0000 7f1766a00640 1 -- 172.21.15.94:0/549276122 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4971+0+0 (secure 0 0 0) 0x7f175c091830 con 0x7f17700d0710 2024-09-10T23:40:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.651+0000 7f176e800640 1 --2- 172.21.15.94:0/549276122 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1750073000 0x7f17500754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.651+0000 7f176e800640 1 --2- 172.21.15.94:0/549276122 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1750073000 0x7f17500754c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f17600af000 tx=0x7f17600a7040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:40.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.655+0000 7f1766a00640 1 -- 172.21.15.94:0/549276122 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f17700701c0 con 0x7f17700d0710 2024-09-10T23:40:40.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:40.831+0000 7f1775400640 1 -- 172.21.15.94:0/549276122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f17700cf6e0 con 0x7f17700d0710 2024-09-10T23:40:41.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.031+0000 7f1766a00640 1 -- 172.21.15.94:0/549276122 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v156) ==== 97+0+0 (secure 0 0 0) 0x7f175c05e0f0 con 0x7f17700d0710 2024-09-10T23:40:41.035 INFO:teuthology.orchestra.run.smithi094.stderr: removed all blocklist entries 2024-09-10T23:40:41.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.035+0000 7f1775400640 1 -- 172.21.15.94:0/549276122 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1750073000 msgr2=0x7f17500754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:41.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.035+0000 7f1775400640 1 --2- 172.21.15.94:0/549276122 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1750073000 0x7f17500754c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f17600af000 tx=0x7f17600a7040 comp rx=0 tx=0).stop 2024-09-10T23:40:41.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.035+0000 7f1775400640 1 -- 172.21.15.94:0/549276122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17700d0710 msgr2=0x7f17700ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:41.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.035+0000 7f1775400640 1 --2- 172.21.15.94:0/549276122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17700d0710 0x7f17700ca140 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f175c00c4e0 tx=0x7f175c00c9a0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.035+0000 7f1775400640 1 -- 172.21.15.94:0/549276122 shutdown_connections 2024-09-10T23:40:41.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.035+0000 7f1775400640 1 --2- 172.21.15.94:0/549276122 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1750073000 0x7f17500754c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.035+0000 7f1775400640 1 --2- 172.21.15.94:0/549276122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17701455d0 0x7f17700c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.035+0000 7f1775400640 1 --2- 172.21.15.94:0/549276122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17700d0710 0x7f17700ca140 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.035+0000 7f1775400640 1 --2- 172.21.15.94:0/549276122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17700cd9f0 0x7f1770159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.035+0000 7f1775400640 1 -- 172.21.15.94:0/549276122 >> 172.21.15.94:0/549276122 conn(0x7f17700cfaf0 msgr2=0x7f177015b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:41.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.035+0000 7f1775400640 1 -- 172.21.15.94:0/549276122 shutdown_connections 2024-09-10T23:40:41.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.035+0000 7f1775400640 1 -- 172.21.15.94:0/549276122 wait complete. 2024-09-10T23:40:41.062 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T23:40:41.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.199+0000 7f55a7a00640 1 -- 172.21.15.94:0/2857676052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55a00b2bb0 msgr2=0x7f55a00d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:41.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.199+0000 7f55a7a00640 1 --2- 172.21.15.94:0/2857676052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55a00b2bb0 0x7f55a00d00a0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f55a0073820 tx=0x7f558801c4e0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.199+0000 7f55a7a00640 1 -- 172.21.15.94:0/2857676052 shutdown_connections 2024-09-10T23:40:41.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.199+0000 7f55a7a00640 1 --2- 172.21.15.94:0/2857676052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55a00d05e0 0x7f55a00b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.199+0000 7f55a7a00640 1 --2- 172.21.15.94:0/2857676052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55a00b2bb0 0x7f55a00d00a0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.199+0000 7f55a7a00640 1 --2- 172.21.15.94:0/2857676052 >> v2:172.21.15.181:3300/0 conn(0x7f55a015f5c0 0x7f55a00cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.199+0000 7f55a7a00640 1 -- 172.21.15.94:0/2857676052 >> 172.21.15.94:0/2857676052 conn(0x7f55a00d1c60 msgr2=0x7f55a00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:41.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.199+0000 7f55a7a00640 1 -- 172.21.15.94:0/2857676052 shutdown_connections 2024-09-10T23:40:41.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.199+0000 7f55a7a00640 1 -- 172.21.15.94:0/2857676052 wait complete. 2024-09-10T23:40:41.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a7a00640 1 Processor -- start 2024-09-10T23:40:41.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a7a00640 1 -- start start 2024-09-10T23:40:41.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a7a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55a00b2bb0 0x7f55a0152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a7a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55a00d05e0 0x7f55a01fbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a7a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55a015f5c0 0x7f55a0153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a7a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55a00afa90 con 0x7f55a00b2bb0 2024-09-10T23:40:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a7a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f55a00af910 con 0x7f55a015f5c0 2024-09-10T23:40:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a7a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f55a00afc10 con 0x7f55a00d05e0 2024-09-10T23:40:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a5000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55a00b2bb0 0x7f55a0152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a5000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55a00b2bb0 0x7f55a0152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46908/0 (socket says 172.21.15.94:46908) 2024-09-10T23:40:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a5000640 1 -- 172.21.15.94:0/1050720547 learned_addr learned my addr 172.21.15.94:0/1050720547 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a5a00640 1 --2- 172.21.15.94:0/1050720547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55a015f5c0 0x7f55a0153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f5597e00640 1 --2- 172.21.15.94:0/1050720547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55a00d05e0 0x7f55a01fbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a5000640 1 -- 172.21.15.94:0/1050720547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55a00d05e0 msgr2=0x7f55a01fbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:41.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a5000640 1 --2- 172.21.15.94:0/1050720547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55a00d05e0 0x7f55a01fbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a5000640 1 -- 172.21.15.94:0/1050720547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55a015f5c0 msgr2=0x7f55a0153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:41.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a5000640 1 --2- 172.21.15.94:0/1050720547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55a015f5c0 0x7f55a0153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a5000640 1 -- 172.21.15.94:0/1050720547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a01fc1a0 con 0x7f55a00b2bb0 2024-09-10T23:40:41.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a5000640 1 --2- 172.21.15.94:0/1050720547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55a00b2bb0 0x7f55a0152310 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f559000c9f0 tx=0x7f559000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:41.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f559d600640 1 -- 172.21.15.94:0/1050720547 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590007bb0 con 0x7f55a00b2bb0 2024-09-10T23:40:41.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f559d600640 1 -- 172.21.15.94:0/1050720547 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5590007e40 con 0x7f55a00b2bb0 2024-09-10T23:40:41.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f559d600640 1 -- 172.21.15.94:0/1050720547 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559000f690 con 0x7f55a00b2bb0 2024-09-10T23:40:41.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a7a00640 1 -- 172.21.15.94:0/1050720547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a00cb520 con 0x7f55a00b2bb0 2024-09-10T23:40:41.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a7a00640 1 -- 172.21.15.94:0/1050720547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a00b6170 con 0x7f55a00b2bb0 2024-09-10T23:40:41.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.203+0000 7f55a7a00640 1 -- 172.21.15.94:0/1050720547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a00b15c0 con 0x7f55a00b2bb0 2024-09-10T23:40:41.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.207+0000 7f559d600640 1 -- 172.21.15.94:0/1050720547 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f559000f830 con 0x7f55a00b2bb0 2024-09-10T23:40:41.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.207+0000 7f559d600640 1 --2- 172.21.15.94:0/1050720547 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f557c073000 0x7f557c0754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:41.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.207+0000 7f559d600640 1 -- 172.21.15.94:0/1050720547 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4885+0+0 (secure 0 0 0) 0x7f5590092450 con 0x7f55a00b2bb0 2024-09-10T23:40:41.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.207+0000 7f5597e00640 1 --2- 172.21.15.94:0/1050720547 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f557c073000 0x7f557c0754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:41.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.207+0000 7f5597e00640 1 --2- 172.21.15.94:0/1050720547 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f557c073000 0x7f557c0754c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f55880ae000 tx=0x7f558801ad20 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:41.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.211+0000 7f559d600640 1 -- 172.21.15.94:0/1050720547 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f55a01fc1a0 con 0x7f55a00b2bb0 2024-09-10T23:40:41.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.431+0000 7f55a7a00640 1 -- 172.21.15.94:0/1050720547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f55a0070f70 con 0x7f55a00b2bb0 2024-09-10T23:40:41.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.435+0000 7f559d600640 1 -- 172.21.15.94:0/1050720547 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v40) ==== 80+0+5304 (secure 0 0 0) 0x7f5590014030 con 0x7f55a00b2bb0 2024-09-10T23:40:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:41.437 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:40:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_dump": [ 2024-09-10T23:40:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.437 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.a", 2024-09-10T23:40:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/2m3LxAAr0ErpLJg+0+I2r74zQya4w==", 2024-09-10T23:40:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:40:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:40:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:40:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:40:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.b", 2024-09-10T23:40:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmc8pYABAAw90zu8D+9IwK15h4HIlGBA==", 2024-09-10T23:40:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:40:41.438 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:40:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:40:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:40:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.c", 2024-09-10T23:40:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmk98KNxAA44e9N4VHbw/94Lh1V9zugQ==", 2024-09-10T23:40:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:40:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:40:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:40:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:40:41.439 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mds.d", 2024-09-10T23:40:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmVAKzBxAAJehOw6aphTF/g3ucoQfw9A==", 2024-09-10T23:40:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:40:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:40:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:40:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:40:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.0", 2024-09-10T23:40:41.440 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDa1uBmSAD3KBAAJTF1OEC44Ml599aL7iEuDw==", 2024-09-10T23:40:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:40:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:40:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:40:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.1", 2024-09-10T23:40:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDe1uBma1L9ABAAfEjW1/Hb4fh4Kwv/ZUgFAQ==", 2024-09-10T23:40:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:40:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:40:41.441 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:40:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.2", 2024-09-10T23:40:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDh1uBmtLUQFhAAweeFHj6BRQVC9BX+VLiWEQ==", 2024-09-10T23:40:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:40:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:40:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:40:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.3", 2024-09-10T23:40:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDk1uBm1KFCKRAAev28iYcrEYOViX95uo6Kbw==", 2024-09-10T23:40:41.442 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:40:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:40:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:40:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.4", 2024-09-10T23:40:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDq1uBm/Hg9LhAAvB2D0rj4DeRPxTrGmXYIsA==", 2024-09-10T23:40:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:40:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:40:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:40:41.443 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.5", 2024-09-10T23:40:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDu1uBm1fAKCBAAFHMjnGFAItLdaACF/UjE/w==", 2024-09-10T23:40:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:40:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:40:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:40:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.6", 2024-09-10T23:40:41.444 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDx1uBmXgYfHhAA/6wwtM4mDRXsIpoTIHKJ0w==", 2024-09-10T23:40:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:40:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:40:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:40:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "osd.7", 2024-09-10T23:40:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQD01uBmw6hQMxAAk7oGAg+iIplNK5KrokGwNA==", 2024-09-10T23:40:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow profile osd", 2024-09-10T23:40:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile osd", 2024-09-10T23:40:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:40:41.445 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.0", 2024-09-10T23:40:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g==", 2024-09-10T23:40:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:40:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow r", 2024-09-10T23:40:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T23:40:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.1", 2024-09-10T23:40:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmuSkhFRAA1R6WNXLImkGfhW9/XJzEAQ==", 2024-09-10T23:40:41.446 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.447 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:40:41.447 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:40:41.447 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:40:41.447 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:40:41.447 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.447 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.2", 2024-09-10T23:40:41.447 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBmfvaqERAASpL0KtglFcAuRf4a/0z5JQ==", 2024-09-10T23:40:41.447 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.447 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:40:41.447 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:40:41.447 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:40:41.447 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:40:41.448 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.448 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.3", 2024-09-10T23:40:41.448 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDZ1uBm8xUtGxAARc0ywfAhMxBNd57TzBoV3g==", 2024-09-10T23:40:41.448 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.448 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow", 2024-09-10T23:40:41.448 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow r", 2024-09-10T23:40:41.448 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow rw", 2024-09-10T23:40:41.448 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow rwx" 2024-09-10T23:40:41.448 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.449 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "client.admin", 2024-09-10T23:40:41.449 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmOpg1FBAA9NV6Jqt6doWMuAfcxztlOg==", 2024-09-10T23:40:41.449 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.449 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:40:41.449 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": "allow *", 2024-09-10T23:40:41.449 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow *", 2024-09-10T23:40:41.449 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:40:41.449 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.449 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.x", 2024-09-10T23:40:41.449 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBm/EPVKxAA/Myv8Zjf7BejbzlkxyCC1w==", 2024-09-10T23:40:41.449 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.450 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:40:41.450 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:40:41.450 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:40:41.450 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:41.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.450 INFO:teuthology.orchestra.run.smithi094.stdout: "entity": "mgr.y", 2024-09-10T23:40:41.450 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "AQDY1uBmhHvtJxAACe+79nYhv2ODwtmNhA45FA==", 2024-09-10T23:40:41.450 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": { 2024-09-10T23:40:41.450 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": "allow *", 2024-09-10T23:40:41.450 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": "allow profile mgr", 2024-09-10T23:40:41.450 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": "allow *" 2024-09-10T23:40:41.450 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.450 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.451 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-10T23:40:41.451 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:40:41.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.435+0000 7f55a7a00640 1 -- 172.21.15.94:0/1050720547 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f557c073000 msgr2=0x7f557c0754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:41.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.435+0000 7f55a7a00640 1 --2- 172.21.15.94:0/1050720547 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f557c073000 0x7f557c0754c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f55880ae000 tx=0x7f558801ad20 comp rx=0 tx=0).stop 2024-09-10T23:40:41.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.435+0000 7f55a7a00640 1 -- 172.21.15.94:0/1050720547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55a00b2bb0 msgr2=0x7f55a0152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:41.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.435+0000 7f55a7a00640 1 --2- 172.21.15.94:0/1050720547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55a00b2bb0 0x7f55a0152310 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f559000c9f0 tx=0x7f559000ceb0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.435+0000 7f55a7a00640 1 -- 172.21.15.94:0/1050720547 shutdown_connections 2024-09-10T23:40:41.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.435+0000 7f55a7a00640 1 --2- 172.21.15.94:0/1050720547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55a015f5c0 0x7f55a0153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.435+0000 7f55a7a00640 1 --2- 172.21.15.94:0/1050720547 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f557c073000 0x7f557c0754c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.435+0000 7f55a7a00640 1 --2- 172.21.15.94:0/1050720547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55a00d05e0 0x7f55a01fbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.435+0000 7f55a7a00640 1 --2- 172.21.15.94:0/1050720547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f55a00b2bb0 0x7f55a0152310 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.435+0000 7f55a7a00640 1 -- 172.21.15.94:0/1050720547 >> 172.21.15.94:0/1050720547 conn(0x7f55a00d1c60 msgr2=0x7f55a00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:41.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.435+0000 7f55a7a00640 1 -- 172.21.15.94:0/1050720547 shutdown_connections 2024-09-10T23:40:41.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.435+0000 7f55a7a00640 1 -- 172.21.15.94:0/1050720547 wait complete. 2024-09-10T23:40:41.459 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T23:40:41.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.595+0000 7fece7a00640 1 -- 172.21.15.94:0/3965066454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fece015dd50 msgr2=0x7fece00b1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:41.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.595+0000 7fece7a00640 1 --2- 172.21.15.94:0/3965066454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fece015dd50 0x7fece00b1b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009870 tx=0x7fecd401c830 comp rx=0 tx=0).stop 2024-09-10T23:40:41.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.595+0000 7fece7a00640 1 -- 172.21.15.94:0/3965066454 shutdown_connections 2024-09-10T23:40:41.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.595+0000 7fece7a00640 1 --2- 172.21.15.94:0/3965066454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fece015dd50 0x7fece00b1b80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.595+0000 7fece7a00640 1 -- 172.21.15.94:0/3965066454 >> 172.21.15.94:0/3965066454 conn(0x7fece009aa70 msgr2=0x7fece009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:41.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.595+0000 7fece7a00640 1 -- 172.21.15.94:0/3965066454 shutdown_connections 2024-09-10T23:40:41.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.595+0000 7fece7a00640 1 -- 172.21.15.94:0/3965066454 wait complete. 2024-09-10T23:40:41.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece7a00640 1 Processor -- start 2024-09-10T23:40:41.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece7a00640 1 -- start start 2024-09-10T23:40:41.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece7a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece006e9a0 0x7fece00b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:41.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece7a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fece015dd50 0x7fece00cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:41.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece7a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fece0162560 0x7fece01fb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:41.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece7a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fece016c410 con 0x7fece0162560 2024-09-10T23:40:41.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece7a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fece016c290 con 0x7fece006e9a0 2024-09-10T23:40:41.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece7a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fece016c590 con 0x7fece015dd50 2024-09-10T23:40:41.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece5a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fece0162560 0x7fece01fb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:41.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece5a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fece0162560 0x7fece01fb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46918/0 (socket says 172.21.15.94:46918) 2024-09-10T23:40:41.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece5a00640 1 -- 172.21.15.94:0/354996368 learned_addr learned my addr 172.21.15.94:0/354996368 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:41.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece5000640 1 --2- 172.21.15.94:0/354996368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece006e9a0 0x7fece00b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:41.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fecdfe00640 1 --2- 172.21.15.94:0/354996368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fece015dd50 0x7fece00cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:41.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece5a00640 1 -- 172.21.15.94:0/354996368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fece015dd50 msgr2=0x7fece00cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:41.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece5a00640 1 --2- 172.21.15.94:0/354996368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fece015dd50 0x7fece00cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece5a00640 1 -- 172.21.15.94:0/354996368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece006e9a0 msgr2=0x7fece00b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:41.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece5a00640 1 --2- 172.21.15.94:0/354996368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece006e9a0 0x7fece00b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece5a00640 1 -- 172.21.15.94:0/354996368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece00b1870 con 0x7fece0162560 2024-09-10T23:40:41.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fecdfe00640 1 --2- 172.21.15.94:0/354996368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fece015dd50 0x7fece00cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:40:41.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece5a00640 1 --2- 172.21.15.94:0/354996368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fece0162560 0x7fece01fb3f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fecd800cba0 tx=0x7fecd800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:41.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fecdcc00640 1 -- 172.21.15.94:0/354996368 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8016020 con 0x7fece0162560 2024-09-10T23:40:41.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fecdcc00640 1 -- 172.21.15.94:0/354996368 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecd800ab00 con 0x7fece0162560 2024-09-10T23:40:41.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fecdcc00640 1 -- 172.21.15.94:0/354996368 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8012420 con 0x7fece0162560 2024-09-10T23:40:41.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece7a00640 1 -- 172.21.15.94:0/354996368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece015cac0 con 0x7fece0162560 2024-09-10T23:40:41.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.599+0000 7fece7a00640 1 -- 172.21.15.94:0/354996368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece015d080 con 0x7fece0162560 2024-09-10T23:40:41.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.603+0000 7fece7a00640 1 -- 172.21.15.94:0/354996368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece00b1b80 con 0x7fece0162560 2024-09-10T23:40:41.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.603+0000 7fecdcc00640 1 -- 172.21.15.94:0/354996368 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7fecd80125c0 con 0x7fece0162560 2024-09-10T23:40:41.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.603+0000 7fecdcc00640 1 --2- 172.21.15.94:0/354996368 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fecc0073000 0x7fecc00754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:41.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.603+0000 7fecdcc00640 1 -- 172.21.15.94:0/354996368 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4885+0+0 (secure 0 0 0) 0x7fecd8092350 con 0x7fece0162560 2024-09-10T23:40:41.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.603+0000 7fece5000640 1 --2- 172.21.15.94:0/354996368 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fecc0073000 0x7fecc00754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:41.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.603+0000 7fece5000640 1 --2- 172.21.15.94:0/354996368 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fecc0073000 0x7fecc00754c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fecd40af000 tx=0x7fecd401b070 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:41.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.607+0000 7fecdcc00640 1 -- 172.21.15.94:0/354996368 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fecd8020020 con 0x7fece0162560 2024-09-10T23:40:41.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.775+0000 7fece7a00640 1 -- 172.21.15.94:0/354996368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fece01598e0 con 0x7fece0162560 2024-09-10T23:40:41.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.775+0000 7fecdcc00640 1 -- 172.21.15.94:0/354996368 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v158) ==== 84+0+65 (secure 0 0 0) 0x7fecd805f1e0 con 0x7fece0162560 2024-09-10T23:40:41.778 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:41.778 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:40:41.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:41.778 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-09-10T23:40:41.778 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-09-10T23:40:41.778 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:41.778 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:40:41.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.775+0000 7fece7a00640 1 -- 172.21.15.94:0/354996368 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fecc0073000 msgr2=0x7fecc00754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:41.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.775+0000 7fece7a00640 1 --2- 172.21.15.94:0/354996368 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fecc0073000 0x7fecc00754c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fecd40af000 tx=0x7fecd401b070 comp rx=0 tx=0).stop 2024-09-10T23:40:41.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.775+0000 7fece7a00640 1 -- 172.21.15.94:0/354996368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fece0162560 msgr2=0x7fece01fb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:41.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.775+0000 7fece7a00640 1 --2- 172.21.15.94:0/354996368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fece0162560 0x7fece01fb3f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fecd800cba0 tx=0x7fecd800cc90 comp rx=0 tx=0).stop 2024-09-10T23:40:41.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.775+0000 7fece7a00640 1 -- 172.21.15.94:0/354996368 shutdown_connections 2024-09-10T23:40:41.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.775+0000 7fece7a00640 1 --2- 172.21.15.94:0/354996368 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fecc0073000 0x7fecc00754c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.775+0000 7fece7a00640 1 --2- 172.21.15.94:0/354996368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fece0162560 0x7fece01fb3f0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.775+0000 7fece7a00640 1 --2- 172.21.15.94:0/354996368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fece015dd50 0x7fece00cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.775+0000 7fece7a00640 1 --2- 172.21.15.94:0/354996368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fece006e9a0 0x7fece00b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.775+0000 7fece7a00640 1 -- 172.21.15.94:0/354996368 >> 172.21.15.94:0/354996368 conn(0x7fece009aa70 msgr2=0x7fece00cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.775+0000 7fece7a00640 1 -- 172.21.15.94:0/354996368 shutdown_connections 2024-09-10T23:40:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.775+0000 7fece7a00640 1 -- 172.21.15.94:0/354996368 wait complete. 2024-09-10T23:40:41.801 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T23:40:41.802 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T23:40:41.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.943+0000 7f025e800640 1 -- 172.21.15.94:0/62422412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f023c003c40 msgr2=0x7f023c004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:41.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.943+0000 7f025e800640 1 --2- 172.21.15.94:0/62422412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f023c003c40 0x7f023c004040 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f0248001860 tx=0x7f024801c880 comp rx=0 tx=0).stop 2024-09-10T23:40:41.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.943+0000 7f025e800640 1 -- 172.21.15.94:0/62422412 shutdown_connections 2024-09-10T23:40:41.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.943+0000 7f025e800640 1 --2- 172.21.15.94:0/62422412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f023c003c40 0x7f023c004040 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.943+0000 7f025e800640 1 -- 172.21.15.94:0/62422412 >> 172.21.15.94:0/62422412 conn(0x7f02580d0b70 msgr2=0x7f02580d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:41.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.943+0000 7f025e800640 1 -- 172.21.15.94:0/62422412 shutdown_connections 2024-09-10T23:40:41.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.943+0000 7f025e800640 1 -- 172.21.15.94:0/62422412 wait complete. 2024-09-10T23:40:41.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f025e800640 1 Processor -- start 2024-09-10T23:40:41.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f025e800640 1 -- start start 2024-09-10T23:40:41.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f025e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f023c003c40 0x7f02581502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:41.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f025e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025814e8b0 0x7f025814e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:41.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f025e800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025815a880 0x7f025815ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:41.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f025e800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0258161150 con 0x7f025814e8b0 2024-09-10T23:40:41.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f025e800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02580b49b0 con 0x7f023c003c40 2024-09-10T23:40:41.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f025e800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f02581612d0 con 0x7f025815a880 2024-09-10T23:40:41.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f024ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025814e8b0 0x7f025814e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:41.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f024ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025814e8b0 0x7f025814e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46948/0 (socket says 172.21.15.94:46948) 2024-09-10T23:40:41.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f024ea00640 1 -- 172.21.15.94:0/2095612882 learned_addr learned my addr 172.21.15.94:0/2095612882 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:41.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f0257400640 1 --2- 172.21.15.94:0/2095612882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f023c003c40 0x7f02581502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:41.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f0257e00640 1 --2- 172.21.15.94:0/2095612882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025815a880 0x7f025815ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:41.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f024ea00640 1 -- 172.21.15.94:0/2095612882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025815a880 msgr2=0x7f025815ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:41.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f024ea00640 1 --2- 172.21.15.94:0/2095612882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025815a880 0x7f025815ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f024ea00640 1 -- 172.21.15.94:0/2095612882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f023c003c40 msgr2=0x7f02581502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:41.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f024ea00640 1 --2- 172.21.15.94:0/2095612882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f023c003c40 0x7f02581502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:41.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f024ea00640 1 -- 172.21.15.94:0/2095612882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02581f7f40 con 0x7f025814e8b0 2024-09-10T23:40:41.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f0257400640 1 --2- 172.21.15.94:0/2095612882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f023c003c40 0x7f02581502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:40:41.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f024ea00640 1 --2- 172.21.15.94:0/2095612882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025814e8b0 0x7f025814e210 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f024000e590 tx=0x7f024000ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:41.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f024fe00640 1 -- 172.21.15.94:0/2095612882 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240015070 con 0x7f025814e8b0 2024-09-10T23:40:41.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f024fe00640 1 -- 172.21.15.94:0/2095612882 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0240010980 con 0x7f025814e8b0 2024-09-10T23:40:41.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f024fe00640 1 -- 172.21.15.94:0/2095612882 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240010db0 con 0x7f025814e8b0 2024-09-10T23:40:41.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f025e800640 1 -- 172.21.15.94:0/2095612882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02581f8220 con 0x7f025814e8b0 2024-09-10T23:40:41.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.947+0000 7f025e800640 1 -- 172.21.15.94:0/2095612882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02581f87e0 con 0x7f025814e8b0 2024-09-10T23:40:41.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.951+0000 7f025e800640 1 -- 172.21.15.94:0/2095612882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02581f89e0 con 0x7f025814e8b0 2024-09-10T23:40:41.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.951+0000 7f024fe00640 1 -- 172.21.15.94:0/2095612882 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f0240011460 con 0x7f025814e8b0 2024-09-10T23:40:41.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.951+0000 7f024fe00640 1 --2- 172.21.15.94:0/2095612882 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0234073000 0x7f02340754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:41.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.951+0000 7f024fe00640 1 -- 172.21.15.94:0/2095612882 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4885+0+0 (secure 0 0 0) 0x7f02400923d0 con 0x7f025814e8b0 2024-09-10T23:40:41.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.951+0000 7f0257400640 1 --2- 172.21.15.94:0/2095612882 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0234073000 0x7f02340754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:41.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.951+0000 7f0257400640 1 --2- 172.21.15.94:0/2095612882 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0234073000 0x7f02340754c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0248001830 tx=0x7f02480196c0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:41.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:41.955+0000 7f024fe00640 1 -- 172.21.15.94:0/2095612882 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f02581f89e0 con 0x7f025814e8b0 2024-09-10T23:40:42.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:42.127+0000 7f025e800640 1 -- 172.21.15.94:0/2095612882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0258161830 con 0x7f025814e8b0 2024-09-10T23:40:43.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.047+0000 7f024fe00640 1 -- 172.21.15.94:0/2095612882 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v159) ==== 138+0+0 (secure 0 0 0) 0x7f024005f320 con 0x7f025814e8b0 2024-09-10T23:40:43.048 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-09-10T23:40:43.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.051+0000 7f025e800640 1 -- 172.21.15.94:0/2095612882 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0234073000 msgr2=0x7f02340754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:43.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.051+0000 7f025e800640 1 --2- 172.21.15.94:0/2095612882 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0234073000 0x7f02340754c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0248001830 tx=0x7f02480196c0 comp rx=0 tx=0).stop 2024-09-10T23:40:43.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.051+0000 7f025e800640 1 -- 172.21.15.94:0/2095612882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025814e8b0 msgr2=0x7f025814e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.051+0000 7f025e800640 1 --2- 172.21.15.94:0/2095612882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025814e8b0 0x7f025814e210 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f024000e590 tx=0x7f024000ea50 comp rx=0 tx=0).stop 2024-09-10T23:40:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.051+0000 7f025e800640 1 -- 172.21.15.94:0/2095612882 shutdown_connections 2024-09-10T23:40:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.051+0000 7f025e800640 1 --2- 172.21.15.94:0/2095612882 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0234073000 0x7f02340754c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.051+0000 7f025e800640 1 --2- 172.21.15.94:0/2095612882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025815a880 0x7f025815ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.051+0000 7f025e800640 1 --2- 172.21.15.94:0/2095612882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025814e8b0 0x7f025814e210 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.051+0000 7f025e800640 1 --2- 172.21.15.94:0/2095612882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f023c003c40 0x7f02581502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.051+0000 7f025e800640 1 -- 172.21.15.94:0/2095612882 >> 172.21.15.94:0/2095612882 conn(0x7f02580d0b70 msgr2=0x7f025814f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.051+0000 7f025e800640 1 -- 172.21.15.94:0/2095612882 shutdown_connections 2024-09-10T23:40:43.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.051+0000 7f025e800640 1 -- 172.21.15.94:0/2095612882 wait complete. 2024-09-10T23:40:43.080 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T23:40:43.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.231+0000 7f9063000640 1 -- 172.21.15.94:0/921018730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9040004d50 msgr2=0x7f90400051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.231+0000 7f9063000640 1 --2- 172.21.15.94:0/921018730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9040004d50 0x7f90400051d0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f9054009eb0 tx=0x7f905401d5b0 comp rx=0 tx=0).stop 2024-09-10T23:40:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.231+0000 7f9061000640 1 -- 172.21.15.94:0/921018730 reap_dead start 2024-09-10T23:40:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.231+0000 7f9063000640 1 -- 172.21.15.94:0/921018730 shutdown_connections 2024-09-10T23:40:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.231+0000 7f9063000640 1 -- 172.21.15.94:0/921018730 >> 172.21.15.94:0/921018730 conn(0x7f905c0cfaf0 msgr2=0x7f905c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.231+0000 7f9063000640 1 -- 172.21.15.94:0/921018730 shutdown_connections 2024-09-10T23:40:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.231+0000 7f9063000640 1 -- 172.21.15.94:0/921018730 wait complete. 2024-09-10T23:40:43.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9063000640 1 Processor -- start 2024-09-10T23:40:43.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9063000640 1 -- start start 2024-09-10T23:40:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9063000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9040004d50 0x7f905c0c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9063000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f905c0b4fb0 0x7f905c0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9063000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f905c1404e0 0x7f905c140960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9063000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f905c0ce210 con 0x7f905c1404e0 2024-09-10T23:40:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9063000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f905c0ce090 con 0x7f9040004d50 2024-09-10T23:40:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9063000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f905c145a60 con 0x7f905c0b4fb0 2024-09-10T23:40:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9061000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f905c1404e0 0x7f905c140960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9061000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f905c1404e0 0x7f905c140960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46968/0 (socket says 172.21.15.94:46968) 2024-09-10T23:40:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9061000640 1 -- 172.21.15.94:0/3689590962 learned_addr learned my addr 172.21.15.94:0/3689590962 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9053e00640 1 --2- 172.21.15.94:0/3689590962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f905c0b4fb0 0x7f905c0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f905be00640 1 --2- 172.21.15.94:0/3689590962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9040004d50 0x7f905c0c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9061000640 1 -- 172.21.15.94:0/3689590962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f905c0b4fb0 msgr2=0x7f905c0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9061000640 1 --2- 172.21.15.94:0/3689590962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f905c0b4fb0 0x7f905c0b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:43.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9061000640 1 -- 172.21.15.94:0/3689590962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9040004d50 msgr2=0x7f905c0c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:43.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9061000640 1 --2- 172.21.15.94:0/3689590962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9040004d50 0x7f905c0c7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:43.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9061000640 1 -- 172.21.15.94:0/3689590962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905c0c6d80 con 0x7f905c1404e0 2024-09-10T23:40:43.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9061000640 1 --2- 172.21.15.94:0/3689590962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f905c1404e0 0x7f905c140960 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f9054009eb0 tx=0x7f90540abb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:43.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9058c00640 1 -- 172.21.15.94:0/3689590962 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054009ab0 con 0x7f905c1404e0 2024-09-10T23:40:43.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9063000640 1 -- 172.21.15.94:0/3689590962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905c1f81d0 con 0x7f905c1404e0 2024-09-10T23:40:43.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9058c00640 1 -- 172.21.15.94:0/3689590962 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90540ba020 con 0x7f905c1404e0 2024-09-10T23:40:43.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9058c00640 1 -- 172.21.15.94:0/3689590962 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054009200 con 0x7f905c1404e0 2024-09-10T23:40:43.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.235+0000 7f9063000640 1 -- 172.21.15.94:0/3689590962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f905c1f8790 con 0x7f905c1404e0 2024-09-10T23:40:43.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.239+0000 7f9063000640 1 -- 172.21.15.94:0/3689590962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f905c0703a0 con 0x7f905c1404e0 2024-09-10T23:40:43.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.239+0000 7f9058c00640 1 -- 172.21.15.94:0/3689590962 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f9054004a50 con 0x7f905c1404e0 2024-09-10T23:40:43.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.239+0000 7f9058c00640 1 --2- 172.21.15.94:0/3689590962 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9038073000 0x7f90380754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:43.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.239+0000 7f9058c00640 1 -- 172.21.15.94:0/3689590962 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5255+0+0 (secure 0 0 0) 0x7f905412d970 con 0x7f905c1404e0 2024-09-10T23:40:43.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.239+0000 7f905be00640 1 --2- 172.21.15.94:0/3689590962 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9038073000 0x7f90380754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:43.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.239+0000 7f905be00640 1 --2- 172.21.15.94:0/3689590962 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9038073000 0x7f90380754c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f904c000c10 tx=0x7f904c006b20 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:43.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.243+0000 7f9058c00640 1 -- 172.21.15.94:0/3689590962 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f90540c4030 con 0x7f905c1404e0 2024-09-10T23:40:43.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:43.411+0000 7f9063000640 1 -- 172.21.15.94:0/3689590962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f905c0cf6e0 con 0x7f905c1404e0 2024-09-10T23:40:44.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.055+0000 7f9058c00640 1 -- 172.21.15.94:0/3689590962 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v160) ==== 170+0+0 (secure 0 0 0) 0x7f90540172c0 con 0x7f905c1404e0 2024-09-10T23:40:44.057 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-09-10T23:40:44.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.059+0000 7f9063000640 1 -- 172.21.15.94:0/3689590962 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9038073000 msgr2=0x7f90380754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:44.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.059+0000 7f9063000640 1 --2- 172.21.15.94:0/3689590962 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9038073000 0x7f90380754c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f904c000c10 tx=0x7f904c006b20 comp rx=0 tx=0).stop 2024-09-10T23:40:44.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.059+0000 7f9063000640 1 -- 172.21.15.94:0/3689590962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f905c1404e0 msgr2=0x7f905c140960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:44.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.059+0000 7f9063000640 1 --2- 172.21.15.94:0/3689590962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f905c1404e0 0x7f905c140960 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f9054009eb0 tx=0x7f90540abb20 comp rx=0 tx=0).stop 2024-09-10T23:40:44.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.059+0000 7f9063000640 1 -- 172.21.15.94:0/3689590962 shutdown_connections 2024-09-10T23:40:44.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.059+0000 7f9063000640 1 --2- 172.21.15.94:0/3689590962 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9038073000 0x7f90380754c0 secure :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f904c000c10 tx=0x7f904c006b20 comp rx=0 tx=0).stop 2024-09-10T23:40:44.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.059+0000 7f9063000640 1 --2- 172.21.15.94:0/3689590962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f905c1404e0 0x7f905c140960 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:44.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.059+0000 7f9063000640 1 --2- 172.21.15.94:0/3689590962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f905c0b4fb0 0x7f905c0b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:44.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.059+0000 7f9063000640 1 --2- 172.21.15.94:0/3689590962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9040004d50 0x7f905c0c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:44.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.059+0000 7f9063000640 1 -- 172.21.15.94:0/3689590962 >> 172.21.15.94:0/3689590962 conn(0x7f905c0cfaf0 msgr2=0x7f905c15d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:44.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.059+0000 7f9063000640 1 -- 172.21.15.94:0/3689590962 shutdown_connections 2024-09-10T23:40:44.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.059+0000 7f9063000640 1 -- 172.21.15.94:0/3689590962 wait complete. 2024-09-10T23:40:44.096 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T23:40:44.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.239+0000 7f1cbfc00640 1 -- 172.21.15.94:0/2028831423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb80c64f0 msgr2=0x7f1cb80ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:44.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.239+0000 7f1cbfc00640 1 --2- 172.21.15.94:0/2028831423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb80c64f0 0x7f1cb80ce660 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009840 tx=0x7f1ca401c820 comp rx=0 tx=0).stop 2024-09-10T23:40:44.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.239+0000 7f1cbfc00640 1 -- 172.21.15.94:0/2028831423 shutdown_connections 2024-09-10T23:40:44.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.239+0000 7f1cbfc00640 1 --2- 172.21.15.94:0/2028831423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb80c64f0 0x7f1cb80ce660 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:44.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.239+0000 7f1cbfc00640 1 --2- 172.21.15.94:0/2028831423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1cb80ce280 0x7f1cb80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:44.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.239+0000 7f1cbfc00640 1 -- 172.21.15.94:0/2028831423 >> 172.21.15.94:0/2028831423 conn(0x7f1cb80cfaf0 msgr2=0x7f1cb80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:44.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.239+0000 7f1cbfc00640 1 -- 172.21.15.94:0/2028831423 shutdown_connections 2024-09-10T23:40:44.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.239+0000 7f1cbfc00640 1 -- 172.21.15.94:0/2028831423 wait complete. 2024-09-10T23:40:44.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cbfc00640 1 Processor -- start 2024-09-10T23:40:44.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cbfc00640 1 -- start start 2024-09-10T23:40:44.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cbfc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb806e7c0 0x7f1cb81522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:44.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cbfc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb80c64f0 0x7f1cb80c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:44.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cbfc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1cb80ce280 0x7f1cb80c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:44.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cbfc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8167500 con 0x7f1cb80c64f0 2024-09-10T23:40:44.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cbfc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb81583e0 con 0x7f1cb806e7c0 2024-09-10T23:40:44.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cbfc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb8167680 con 0x7f1cb80ce280 2024-09-10T23:40:44.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cb7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb80c64f0 0x7f1cb80c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:44.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cb7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb80c64f0 0x7f1cb80c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59258/0 (socket says 172.21.15.94:59258) 2024-09-10T23:40:44.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cb7e00640 1 -- 172.21.15.94:0/3744490370 learned_addr learned my addr 172.21.15.94:0/3744490370 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:44.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cbdc00640 1 --2- 172.21.15.94:0/3744490370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1cb80ce280 0x7f1cb80c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:44.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cbd200640 1 --2- 172.21.15.94:0/3744490370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb806e7c0 0x7f1cb81522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:44.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cb7e00640 1 -- 172.21.15.94:0/3744490370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1cb80ce280 msgr2=0x7f1cb80c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:44.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cb7e00640 1 --2- 172.21.15.94:0/3744490370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1cb80ce280 0x7f1cb80c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:44.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cb7e00640 1 -- 172.21.15.94:0/3744490370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb806e7c0 msgr2=0x7f1cb81522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:44.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cb7e00640 1 --2- 172.21.15.94:0/3744490370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb806e7c0 0x7f1cb81522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:44.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cb7e00640 1 -- 172.21.15.94:0/3744490370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb8152820 con 0x7f1cb80c64f0 2024-09-10T23:40:44.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cb7e00640 1 --2- 172.21.15.94:0/3744490370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb80c64f0 0x7f1cb80c82c0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009790 tx=0x7f1ca4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:44.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cb4c00640 1 -- 172.21.15.94:0/3744490370 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40bc070 con 0x7f1cb80c64f0 2024-09-10T23:40:44.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cb4c00640 1 -- 172.21.15.94:0/3744490370 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ca4003600 con 0x7f1cb80c64f0 2024-09-10T23:40:44.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cb4c00640 1 -- 172.21.15.94:0/3744490370 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40b2040 con 0x7f1cb80c64f0 2024-09-10T23:40:44.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cbfc00640 1 -- 172.21.15.94:0/3744490370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb80c6bc0 con 0x7f1cb80c64f0 2024-09-10T23:40:44.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.243+0000 7f1cbfc00640 1 -- 172.21.15.94:0/3744490370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb80c7180 con 0x7f1cb80c64f0 2024-09-10T23:40:44.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.247+0000 7f1cbfc00640 1 -- 172.21.15.94:0/3744490370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb80ce660 con 0x7f1cb80c64f0 2024-09-10T23:40:44.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.247+0000 7f1cb4c00640 1 -- 172.21.15.94:0/3744490370 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f1ca40b5070 con 0x7f1cb80c64f0 2024-09-10T23:40:44.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.247+0000 7f1cb4c00640 1 --2- 172.21.15.94:0/3744490370 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1c98073000 0x7f1c980754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:44.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.251+0000 7f1cb4c00640 1 -- 172.21.15.94:0/3744490370 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5637+0+0 (secure 0 0 0) 0x7f1ca412b6d0 con 0x7f1cb80c64f0 2024-09-10T23:40:44.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.251+0000 7f1cbd200640 1 --2- 172.21.15.94:0/3744490370 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1c98073000 0x7f1c980754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:44.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.251+0000 7f1cbd200640 1 --2- 172.21.15.94:0/3744490370 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1c98073000 0x7f1c980754c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0058d0 tx=0x7f1cac005860 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:44.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.251+0000 7f1cb4c00640 1 -- 172.21.15.94:0/3744490370 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1ca40c5080 con 0x7f1cb80c64f0 2024-09-10T23:40:44.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:44.451+0000 7f1cbfc00640 1 -- 172.21.15.94:0/3744490370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1cb80d0850 con 0x7f1cb80c64f0 2024-09-10T23:40:45.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.071+0000 7f1cb4c00640 1 -- 172.21.15.94:0/3744490370 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v113) ==== 174+0+0 (secure 0 0 0) 0x7f1ca4016410 con 0x7f1cb80c64f0 2024-09-10T23:40:45.076 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 26 and data pool 27 2024-09-10T23:40:45.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.079+0000 7f1cbfc00640 1 -- 172.21.15.94:0/3744490370 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1c98073000 msgr2=0x7f1c980754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:45.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.079+0000 7f1cbfc00640 1 --2- 172.21.15.94:0/3744490370 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1c98073000 0x7f1c980754c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0058d0 tx=0x7f1cac005860 comp rx=0 tx=0).stop 2024-09-10T23:40:45.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.079+0000 7f1cbfc00640 1 -- 172.21.15.94:0/3744490370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb80c64f0 msgr2=0x7f1cb80c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:45.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.079+0000 7f1cbfc00640 1 --2- 172.21.15.94:0/3744490370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb80c64f0 0x7f1cb80c82c0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009790 tx=0x7f1ca4004040 comp rx=0 tx=0).stop 2024-09-10T23:40:45.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.079+0000 7f1cbfc00640 1 -- 172.21.15.94:0/3744490370 shutdown_connections 2024-09-10T23:40:45.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.079+0000 7f1cbfc00640 1 --2- 172.21.15.94:0/3744490370 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f1c98073000 0x7f1c980754c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.079+0000 7f1cbfc00640 1 --2- 172.21.15.94:0/3744490370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1cb80ce280 0x7f1cb80c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.079+0000 7f1cbfc00640 1 --2- 172.21.15.94:0/3744490370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1cb80c64f0 0x7f1cb80c82c0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.079+0000 7f1cbfc00640 1 --2- 172.21.15.94:0/3744490370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb806e7c0 0x7f1cb81522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.079+0000 7f1cbfc00640 1 -- 172.21.15.94:0/3744490370 >> 172.21.15.94:0/3744490370 conn(0x7f1cb80cfaf0 msgr2=0x7f1cb80af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:45.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.083+0000 7f1cbfc00640 1 -- 172.21.15.94:0/3744490370 shutdown_connections 2024-09-10T23:40:45.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.083+0000 7f1cbfc00640 1 -- 172.21.15.94:0/3744490370 wait complete. 2024-09-10T23:40:45.118 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:40:45.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.263+0000 7f02a9e00640 1 -- 172.21.15.94:0/3904193431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0288003c90 msgr2=0x7f0288004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:45.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.263+0000 7f02a9e00640 1 --2- 172.21.15.94:0/3904193431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0288003c90 0x7f0288004090 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f0294009b60 tx=0x7f029401d240 comp rx=0 tx=0).stop 2024-09-10T23:40:45.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.263+0000 7f02a9e00640 1 -- 172.21.15.94:0/3904193431 shutdown_connections 2024-09-10T23:40:45.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.263+0000 7f02a9e00640 1 --2- 172.21.15.94:0/3904193431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0288003c90 0x7f0288004090 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.263+0000 7f02a9e00640 1 -- 172.21.15.94:0/3904193431 >> 172.21.15.94:0/3904193431 conn(0x7f02a406cdf0 msgr2=0x7f02a406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:45.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.263+0000 7f02a9e00640 1 -- 172.21.15.94:0/3904193431 shutdown_connections 2024-09-10T23:40:45.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.263+0000 7f02a9e00640 1 -- 172.21.15.94:0/3904193431 wait complete. 2024-09-10T23:40:45.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.263+0000 7f02a9e00640 1 Processor -- start 2024-09-10T23:40:45.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.263+0000 7f02a9e00640 1 -- start start 2024-09-10T23:40:45.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.263+0000 7f02a9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0288003c90 0x7f02a4145520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:45.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.263+0000 7f02a9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a4145ae0 0x7f02a4144de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:45.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.263+0000 7f02a9e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02a41567f0 0x7f02a4156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:45.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.263+0000 7f02a9e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f02a4160260 con 0x7f02a4145ae0 2024-09-10T23:40:45.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.263+0000 7f02a9e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02a41600e0 con 0x7f0288003c90 2024-09-10T23:40:45.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.263+0000 7f02a9e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f02a41603e0 con 0x7f02a41567f0 2024-09-10T23:40:45.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f029a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a4145ae0 0x7f02a4144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:45.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f029a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a4145ae0 0x7f02a4144de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59270/0 (socket says 172.21.15.94:59270) 2024-09-10T23:40:45.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f029a000640 1 -- 172.21.15.94:0/639187018 learned_addr learned my addr 172.21.15.94:0/639187018 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:45.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f02a2a00640 1 --2- 172.21.15.94:0/639187018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0288003c90 0x7f02a4145520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:45.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f02a3400640 1 --2- 172.21.15.94:0/639187018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02a41567f0 0x7f02a4156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:45.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f029a000640 1 -- 172.21.15.94:0/639187018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02a41567f0 msgr2=0x7f02a4156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:45.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f029a000640 1 --2- 172.21.15.94:0/639187018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02a41567f0 0x7f02a4156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f029a000640 1 -- 172.21.15.94:0/639187018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0288003c90 msgr2=0x7f02a4145520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:45.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f029a000640 1 --2- 172.21.15.94:0/639187018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0288003c90 0x7f02a4145520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f029a000640 1 -- 172.21.15.94:0/639187018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a4145320 con 0x7f02a4145ae0 2024-09-10T23:40:45.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f029a000640 1 --2- 172.21.15.94:0/639187018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a4145ae0 0x7f02a4144de0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f028c004f90 tx=0x7f028c0064e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:45.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f029be00640 1 -- 172.21.15.94:0/639187018 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c015070 con 0x7f02a4145ae0 2024-09-10T23:40:45.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f029be00640 1 -- 172.21.15.94:0/639187018 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f028c00ea00 con 0x7f02a4145ae0 2024-09-10T23:40:45.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f029be00640 1 -- 172.21.15.94:0/639187018 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c00ee50 con 0x7f02a4145ae0 2024-09-10T23:40:45.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f02a9e00640 1 -- 172.21.15.94:0/639187018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a40b1ba0 con 0x7f02a4145ae0 2024-09-10T23:40:45.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f02a9e00640 1 -- 172.21.15.94:0/639187018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a4148660 con 0x7f02a4145ae0 2024-09-10T23:40:45.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f02a9e00640 1 -- 172.21.15.94:0/639187018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a40ce060 con 0x7f02a4145ae0 2024-09-10T23:40:45.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.267+0000 7f029be00640 1 -- 172.21.15.94:0/639187018 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f028c011460 con 0x7f02a4145ae0 2024-09-10T23:40:45.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.271+0000 7f029be00640 1 --2- 172.21.15.94:0/639187018 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0280073030 0x7f02800754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:45.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.271+0000 7f029be00640 1 -- 172.21.15.94:0/639187018 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f028c0927e0 con 0x7f02a4145ae0 2024-09-10T23:40:45.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.271+0000 7f02a2a00640 1 --2- 172.21.15.94:0/639187018 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0280073030 0x7f02800754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:45.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.271+0000 7f02a2a00640 1 --2- 172.21.15.94:0/639187018 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0280073030 0x7f02800754f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f02940ae000 tx=0x7f0294019fe0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:45.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.275+0000 7f029be00640 1 -- 172.21.15.94:0/639187018 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f02a4145320 con 0x7f02a4145ae0 2024-09-10T23:40:45.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.439+0000 7f02a9e00640 1 -- 172.21.15.94:0/639187018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02a414db30 con 0x7f02a4145ae0 2024-09-10T23:40:45.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.443+0000 7f029be00640 1 -- 172.21.15.94:0/639187018 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) ==== 74+0+16720 (secure 0 0 0) 0x7f028c05f0a0 con 0x7f02a4145ae0 2024-09-10T23:40:45.442 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:45.442 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":161,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:40:44.453104+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:40:42.141193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-10T23:40:43.422826+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:40:45.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.443+0000 7f02a9e00640 1 -- 172.21.15.94:0/639187018 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0280073030 msgr2=0x7f02800754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:45.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.443+0000 7f02a9e00640 1 --2- 172.21.15.94:0/639187018 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0280073030 0x7f02800754f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f02940ae000 tx=0x7f0294019fe0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.443+0000 7f02a9e00640 1 -- 172.21.15.94:0/639187018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a4145ae0 msgr2=0x7f02a4144de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:45.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.443+0000 7f02a9e00640 1 --2- 172.21.15.94:0/639187018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a4145ae0 0x7f02a4144de0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f028c004f90 tx=0x7f028c0064e0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.443+0000 7f02a9e00640 1 -- 172.21.15.94:0/639187018 shutdown_connections 2024-09-10T23:40:45.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.443+0000 7f02a9e00640 1 --2- 172.21.15.94:0/639187018 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0280073030 0x7f02800754f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.443+0000 7f02a9e00640 1 --2- 172.21.15.94:0/639187018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02a41567f0 0x7f02a4156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.443+0000 7f02a9e00640 1 --2- 172.21.15.94:0/639187018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02a4145ae0 0x7f02a4144de0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.443+0000 7f02a9e00640 1 --2- 172.21.15.94:0/639187018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0288003c90 0x7f02a4145520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.443+0000 7f02a9e00640 1 -- 172.21.15.94:0/639187018 >> 172.21.15.94:0/639187018 conn(0x7f02a406cdf0 msgr2=0x7f02a4146100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:45.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.443+0000 7f02a9e00640 1 -- 172.21.15.94:0/639187018 shutdown_connections 2024-09-10T23:40:45.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.443+0000 7f02a9e00640 1 -- 172.21.15.94:0/639187018 wait complete. 2024-09-10T23:40:45.464 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:40:45.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.607+0000 7f66b6e00640 1 -- 172.21.15.94:0/4163564611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6698004d50 msgr2=0x7f66980051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:45.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.607+0000 7f66b6e00640 1 --2- 172.21.15.94:0/4163564611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6698004d50 0x7f66980051d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f66a8002240 tx=0x7f66a801d2a0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.607+0000 7f66b6400640 1 -- 172.21.15.94:0/4163564611 reap_dead start 2024-09-10T23:40:45.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.607+0000 7f66b6e00640 1 -- 172.21.15.94:0/4163564611 shutdown_connections 2024-09-10T23:40:45.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.607+0000 7f66b6e00640 1 -- 172.21.15.94:0/4163564611 >> 172.21.15.94:0/4163564611 conn(0x7f66b00cd6b0 msgr2=0x7f66b00cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:45.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.607+0000 7f66b6e00640 1 -- 172.21.15.94:0/4163564611 shutdown_connections 2024-09-10T23:40:45.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.607+0000 7f66b6e00640 1 -- 172.21.15.94:0/4163564611 wait complete. 2024-09-10T23:40:45.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.607+0000 7f66b6e00640 1 Processor -- start 2024-09-10T23:40:45.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.607+0000 7f66b6e00640 1 -- start start 2024-09-10T23:40:45.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6698004d50 0x7f66b0154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:45.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66b01edb70 0x7f66b015c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:45.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b01edff0 0x7f66b015c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:45.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0166f80 con 0x7f66b01edff0 2024-09-10T23:40:45.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0166e00 con 0x7f6698004d50 2024-09-10T23:40:45.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f66b0167100 con 0x7f66b01edb70 2024-09-10T23:40:45.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b01edff0 0x7f66b015c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:45.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b01edff0 0x7f66b015c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59286/0 (socket says 172.21.15.94:59286) 2024-09-10T23:40:45.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6400640 1 -- 172.21.15.94:0/2282420025 learned_addr learned my addr 172.21.15.94:0/2282420025 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:45.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b5a00640 1 --2- 172.21.15.94:0/2282420025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6698004d50 0x7f66b0154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:45.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b5000640 1 --2- 172.21.15.94:0/2282420025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66b01edb70 0x7f66b015c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:45.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6400640 1 -- 172.21.15.94:0/2282420025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66b01edb70 msgr2=0x7f66b015c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:45.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6400640 1 --2- 172.21.15.94:0/2282420025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66b01edb70 0x7f66b015c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6400640 1 -- 172.21.15.94:0/2282420025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6698004d50 msgr2=0x7f66b0154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:45.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6400640 1 --2- 172.21.15.94:0/2282420025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6698004d50 0x7f66b0154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6400640 1 -- 172.21.15.94:0/2282420025 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b0153410 con 0x7f66b01edff0 2024-09-10T23:40:45.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6400640 1 --2- 172.21.15.94:0/2282420025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b01edff0 0x7f66b015c600 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f66a801cd60 tx=0x7f66a8006620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:45.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66ad600640 1 -- 172.21.15.94:0/2282420025 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c001390 con 0x7f66b01edff0 2024-09-10T23:40:45.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66ad600640 1 -- 172.21.15.94:0/2282420025 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66a8004e40 con 0x7f66b01edff0 2024-09-10T23:40:45.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66ad600640 1 -- 172.21.15.94:0/2282420025 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8002070 con 0x7f66b01edff0 2024-09-10T23:40:45.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6e00640 1 -- 172.21.15.94:0/2282420025 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b01536f0 con 0x7f66b01edff0 2024-09-10T23:40:45.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6e00640 1 -- 172.21.15.94:0/2282420025 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b0153cb0 con 0x7f66b01edff0 2024-09-10T23:40:45.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.611+0000 7f66b6e00640 1 -- 172.21.15.94:0/2282420025 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b015b3f0 con 0x7f66b01edff0 2024-09-10T23:40:45.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.615+0000 7f66ad600640 1 -- 172.21.15.94:0/2282420025 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f66a8003980 con 0x7f66b01edff0 2024-09-10T23:40:45.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.615+0000 7f66ad600640 1 --2- 172.21.15.94:0/2282420025 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6690073000 0x7f66900754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:45.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.615+0000 7f66ad600640 1 -- 172.21.15.94:0/2282420025 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f66a812c1d0 con 0x7f66b01edff0 2024-09-10T23:40:45.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.615+0000 7f66b5a00640 1 --2- 172.21.15.94:0/2282420025 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6690073000 0x7f66900754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:45.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.615+0000 7f66b5a00640 1 --2- 172.21.15.94:0/2282420025 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6690073000 0x7f66900754c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f66b015f230 tx=0x7f66a40091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:45.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.619+0000 7f66ad600640 1 -- 172.21.15.94:0/2282420025 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f66a8017050 con 0x7f66b01edff0 2024-09-10T23:40:45.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.787+0000 7f66b6e00640 1 -- 172.21.15.94:0/2282420025 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66b00ce1d0 con 0x7f66b01edff0 2024-09-10T23:40:45.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.787+0000 7f66ad600640 1 -- 172.21.15.94:0/2282420025 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) ==== 74+0+16720 (secure 0 0 0) 0x7f66a80f91e0 con 0x7f66b01edff0 2024-09-10T23:40:45.787 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:45.787 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":161,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:40:44.453104+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:40:42.141193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-10T23:40:43.422826+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:40:45.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.787+0000 7f66b6e00640 1 -- 172.21.15.94:0/2282420025 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6690073000 msgr2=0x7f66900754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:45.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.787+0000 7f66b6e00640 1 --2- 172.21.15.94:0/2282420025 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6690073000 0x7f66900754c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f66b015f230 tx=0x7f66a40091e0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.787+0000 7f66b6e00640 1 -- 172.21.15.94:0/2282420025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b01edff0 msgr2=0x7f66b015c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:45.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.787+0000 7f66b6e00640 1 --2- 172.21.15.94:0/2282420025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b01edff0 0x7f66b015c600 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f66a801cd60 tx=0x7f66a8006620 comp rx=0 tx=0).stop 2024-09-10T23:40:45.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.791+0000 7f66b6e00640 1 -- 172.21.15.94:0/2282420025 shutdown_connections 2024-09-10T23:40:45.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.791+0000 7f66b6e00640 1 --2- 172.21.15.94:0/2282420025 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6690073000 0x7f66900754c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.791+0000 7f66b6e00640 1 --2- 172.21.15.94:0/2282420025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66b01edff0 0x7f66b015c600 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.791+0000 7f66b6e00640 1 --2- 172.21.15.94:0/2282420025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66b01edb70 0x7f66b015c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.791+0000 7f66b6e00640 1 --2- 172.21.15.94:0/2282420025 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6698004d50 0x7f66b0154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.791+0000 7f66b6e00640 1 -- 172.21.15.94:0/2282420025 >> 172.21.15.94:0/2282420025 conn(0x7f66b00cd6b0 msgr2=0x7f66b015a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:45.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.791+0000 7f66b6e00640 1 -- 172.21.15.94:0/2282420025 shutdown_connections 2024-09-10T23:40:45.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.791+0000 7f66b6e00640 1 -- 172.21.15.94:0/2282420025 wait complete. 2024-09-10T23:40:45.808 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T23:40:45.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.951+0000 7fc500200640 1 -- 172.21.15.94:0/1153890317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4f80ce1e0 msgr2=0x7fc4f80cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:45.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.951+0000 7fc500200640 1 --2- 172.21.15.94:0/1153890317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4f80ce1e0 0x7fc4f80cec70 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8073990 tx=0x7fc4e401c800 comp rx=0 tx=0).stop 2024-09-10T23:40:45.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.951+0000 7fc500200640 1 -- 172.21.15.94:0/1153890317 shutdown_connections 2024-09-10T23:40:45.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.951+0000 7fc500200640 1 --2- 172.21.15.94:0/1153890317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4f80ce1e0 0x7fc4f80cec70 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.951+0000 7fc500200640 1 --2- 172.21.15.94:0/1153890317 >> v2:172.21.15.181:3300/0 conn(0x7fc4f80ce890 0x7fc4f80cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.951+0000 7fc500200640 1 -- 172.21.15.94:0/1153890317 >> 172.21.15.94:0/1153890317 conn(0x7fc4f80d1c60 msgr2=0x7fc4f80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:45.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.951+0000 7fc500200640 1 -- 172.21.15.94:0/1153890317 shutdown_connections 2024-09-10T23:40:45.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.951+0000 7fc500200640 1 -- 172.21.15.94:0/1153890317 wait complete. 2024-09-10T23:40:45.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc500200640 1 Processor -- start 2024-09-10T23:40:45.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc500200640 1 -- start start 2024-09-10T23:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc500200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f80b1100 0x7fc4f81f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc500200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4f80ce1e0 0x7fc4f814fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc500200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f80ce890 0x7fc4f80b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc500200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f80d0980 con 0x7fc4f80ce1e0 2024-09-10T23:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc500200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f80d0800 con 0x7fc4f80b1100 2024-09-10T23:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc500200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f80d0b00 con 0x7fc4f80ce890 2024-09-10T23:40:45.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc4fce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4f80ce1e0 0x7fc4f814fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc4fce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4f80ce1e0 0x7fc4f814fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59304/0 (socket says 172.21.15.94:59304) 2024-09-10T23:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc4fce00640 1 -- 172.21.15.94:0/4148175709 learned_addr learned my addr 172.21.15.94:0/4148175709 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc4fd800640 1 --2- 172.21.15.94:0/4148175709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f80b1100 0x7fc4f81f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc4fce00640 1 -- 172.21.15.94:0/4148175709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f80ce890 msgr2=0x7fc4f80b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc4fe200640 1 --2- 172.21.15.94:0/4148175709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f80ce890 0x7fc4f80b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc4fce00640 1 --2- 172.21.15.94:0/4148175709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f80ce890 0x7fc4f80b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc4fce00640 1 -- 172.21.15.94:0/4148175709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f80b1100 msgr2=0x7fc4f81f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc4fce00640 1 --2- 172.21.15.94:0/4148175709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f80b1100 0x7fc4f81f8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc4fce00640 1 -- 172.21.15.94:0/4148175709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f8150250 con 0x7fc4f80ce1e0 2024-09-10T23:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc4fe200640 1 --2- 172.21.15.94:0/4148175709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f80ce890 0x7fc4f80b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:45.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc4fce00640 1 --2- 172.21.15.94:0/4148175709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4f80ce1e0 0x7fc4f814fd10 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40b0040 tx=0x7fc4e40a5700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:45.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc4f5600640 1 -- 172.21.15.94:0/4148175709 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40b2040 con 0x7fc4f80ce1e0 2024-09-10T23:40:45.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc4f5600640 1 -- 172.21.15.94:0/4148175709 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4e40043e0 con 0x7fc4f80ce1e0 2024-09-10T23:40:45.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc4f5600640 1 -- 172.21.15.94:0/4148175709 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4005290 con 0x7fc4f80ce1e0 2024-09-10T23:40:45.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc500200640 1 -- 172.21.15.94:0/4148175709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f81f75a0 con 0x7fc4f80ce1e0 2024-09-10T23:40:45.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc500200640 1 -- 172.21.15.94:0/4148175709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f81f7b60 con 0x7fc4f80ce1e0 2024-09-10T23:40:45.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.955+0000 7fc500200640 1 -- 172.21.15.94:0/4148175709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f80cec70 con 0x7fc4f80ce1e0 2024-09-10T23:40:45.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.959+0000 7fc4f5600640 1 -- 172.21.15.94:0/4148175709 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7fc4e4005430 con 0x7fc4f80ce1e0 2024-09-10T23:40:45.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.959+0000 7fc4f5600640 1 --2- 172.21.15.94:0/4148175709 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc4d8073000 0x7fc4d80754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:45.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.959+0000 7fc4f5600640 1 -- 172.21.15.94:0/4148175709 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7fc4e412b960 con 0x7fc4f80ce1e0 2024-09-10T23:40:45.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.959+0000 7fc4fd800640 1 --2- 172.21.15.94:0/4148175709 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc4d8073000 0x7fc4d80754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:45.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.959+0000 7fc4fd800640 1 --2- 172.21.15.94:0/4148175709 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc4d8073000 0x7fc4d80754c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec006fd0 tx=0x7fc4ec008040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:45.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:45.963+0000 7fc4f5600640 1 -- 172.21.15.94:0/4148175709 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc4e40164b0 con 0x7fc4f80ce1e0 2024-09-10T23:40:46.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:46.167+0000 7fc500200640 1 -- 172.21.15.94:0/4148175709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc4f815da30 con 0x7fc4f80ce1e0 2024-09-10T23:40:47.045 INFO:tasks.ceph.mds.a.smithi094.stderr:2024-09-10T23:40:47.047+0000 7fe6be000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T23:40:47.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.087+0000 7fc4f5600640 1 -- 172.21.15.94:0/4148175709 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v115) ==== 118+0+0 (secure 0 0 0) 0x7fc4e40f8870 con 0x7fc4f80ce1e0 2024-09-10T23:40:47.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.091+0000 7fc500200640 1 -- 172.21.15.94:0/4148175709 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc4d8073000 msgr2=0x7fc4d80754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:47.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.091+0000 7fc500200640 1 --2- 172.21.15.94:0/4148175709 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc4d8073000 0x7fc4d80754c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec006fd0 tx=0x7fc4ec008040 comp rx=0 tx=0).stop 2024-09-10T23:40:47.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.091+0000 7fc500200640 1 -- 172.21.15.94:0/4148175709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4f80ce1e0 msgr2=0x7fc4f814fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:47.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.091+0000 7fc500200640 1 --2- 172.21.15.94:0/4148175709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4f80ce1e0 0x7fc4f814fd10 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40b0040 tx=0x7fc4e40a5700 comp rx=0 tx=0).stop 2024-09-10T23:40:47.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.091+0000 7fc500200640 1 -- 172.21.15.94:0/4148175709 shutdown_connections 2024-09-10T23:40:47.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.091+0000 7fc500200640 1 --2- 172.21.15.94:0/4148175709 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc4d8073000 0x7fc4d80754c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:47.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.091+0000 7fc500200640 1 --2- 172.21.15.94:0/4148175709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4f80ce890 0x7fc4f80b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:47.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.091+0000 7fc500200640 1 --2- 172.21.15.94:0/4148175709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4f80ce1e0 0x7fc4f814fd10 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:47.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.091+0000 7fc500200640 1 --2- 172.21.15.94:0/4148175709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4f80b1100 0x7fc4f81f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:47.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.091+0000 7fc500200640 1 -- 172.21.15.94:0/4148175709 >> 172.21.15.94:0/4148175709 conn(0x7fc4f80d1c60 msgr2=0x7fc4f80ae870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:47.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.091+0000 7fc500200640 1 -- 172.21.15.94:0/4148175709 shutdown_connections 2024-09-10T23:40:47.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.091+0000 7fc500200640 1 -- 172.21.15.94:0/4148175709 wait complete. 2024-09-10T23:40:47.112 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T23:40:47.113 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T23:40:47.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d65600640 1 -- 172.21.15.94:0/2838284384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d600d0710 msgr2=0x7f0d600c5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:47.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d65600640 1 --2- 172.21.15.94:0/2838284384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d600d0710 0x7f0d600c5530 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009840 tx=0x7f0d4c01c820 comp rx=0 tx=0).stop 2024-09-10T23:40:47.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d65600640 1 -- 172.21.15.94:0/2838284384 shutdown_connections 2024-09-10T23:40:47.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d65600640 1 --2- 172.21.15.94:0/2838284384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d600d0710 0x7f0d600c5530 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:47.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d65600640 1 -- 172.21.15.94:0/2838284384 >> 172.21.15.94:0/2838284384 conn(0x7f0d600cfaf0 msgr2=0x7f0d600cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:47.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d65600640 1 -- 172.21.15.94:0/2838284384 shutdown_connections 2024-09-10T23:40:47.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d65600640 1 -- 172.21.15.94:0/2838284384 wait complete. 2024-09-10T23:40:47.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d65600640 1 Processor -- start 2024-09-10T23:40:47.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d65600640 1 -- start start 2024-09-10T23:40:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d65600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d600c5a70 0x7f0d600ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d65600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d600d0710 0x7f0d600cea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d65600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d601455d0 0x7f0d601534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d65600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6016a820 con 0x7f0d600d0710 2024-09-10T23:40:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d65600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6016a6a0 con 0x7f0d600c5a70 2024-09-10T23:40:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d65600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0d6016a9a0 con 0x7f0d601455d0 2024-09-10T23:40:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d5e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d600d0710 0x7f0d600cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d5e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d600d0710 0x7f0d600cea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59322/0 (socket says 172.21.15.94:59322) 2024-09-10T23:40:47.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d5e000640 1 -- 172.21.15.94:0/304497877 learned_addr learned my addr 172.21.15.94:0/304497877 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:47.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d5e000640 1 -- 172.21.15.94:0/304497877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d601455d0 msgr2=0x7f0d601534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d5e000640 1 --2- 172.21.15.94:0/304497877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d601455d0 0x7f0d601534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d5e000640 1 -- 172.21.15.94:0/304497877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d600c5a70 msgr2=0x7f0d600ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d5e000640 1 --2- 172.21.15.94:0/304497877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d600c5a70 0x7f0d600ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d5e000640 1 -- 172.21.15.94:0/304497877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d60143f20 con 0x7f0d600d0710 2024-09-10T23:40:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d5e000640 1 --2- 172.21.15.94:0/304497877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d600d0710 0x7f0d600cea50 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c01a450 tx=0x7f0d4c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.263+0000 7f0d56a00640 1 -- 172.21.15.94:0/304497877 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c0bb070 con 0x7f0d600d0710 2024-09-10T23:40:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.267+0000 7f0d56a00640 1 -- 172.21.15.94:0/304497877 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d4c005030 con 0x7f0d600d0710 2024-09-10T23:40:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.267+0000 7f0d56a00640 1 -- 172.21.15.94:0/304497877 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c0b2040 con 0x7f0d600d0710 2024-09-10T23:40:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.267+0000 7f0d65600640 1 -- 172.21.15.94:0/304497877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d601441c0 con 0x7f0d600d0710 2024-09-10T23:40:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.267+0000 7f0d65600640 1 -- 172.21.15.94:0/304497877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d60144780 con 0x7f0d600d0710 2024-09-10T23:40:47.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.267+0000 7f0d65600640 1 -- 172.21.15.94:0/304497877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d600c5530 con 0x7f0d600d0710 2024-09-10T23:40:47.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.267+0000 7f0d56a00640 1 -- 172.21.15.94:0/304497877 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f0d4c0b5030 con 0x7f0d600d0710 2024-09-10T23:40:47.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.267+0000 7f0d56a00640 1 --2- 172.21.15.94:0/304497877 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0d40073000 0x7f0d400754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:47.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.267+0000 7f0d56a00640 1 -- 172.21.15.94:0/304497877 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f0d4c12b4b0 con 0x7f0d600d0710 2024-09-10T23:40:47.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.267+0000 7f0d5ea00640 1 --2- 172.21.15.94:0/304497877 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0d40073000 0x7f0d400754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:47.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.271+0000 7f0d5ea00640 1 --2- 172.21.15.94:0/304497877 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0d40073000 0x7f0d400754c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0d50002050 tx=0x7f0d50001660 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:47.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.271+0000 7f0d56a00640 1 -- 172.21.15.94:0/304497877 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0d4c016410 con 0x7f0d600d0710 2024-09-10T23:40:47.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:47.471+0000 7f0d65600640 1 -- 172.21.15.94:0/304497877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0d600cf6e0 con 0x7f0d600d0710 2024-09-10T23:40:48.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.095+0000 7f0d56a00640 1 -- 172.21.15.94:0/304497877 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v116) ==== 122+0+0 (secure 0 0 0) 0x7f0d4c0f83c0 con 0x7f0d600d0710 2024-09-10T23:40:48.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.099+0000 7f0d65600640 1 -- 172.21.15.94:0/304497877 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0d40073000 msgr2=0x7f0d400754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:48.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.099+0000 7f0d65600640 1 --2- 172.21.15.94:0/304497877 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0d40073000 0x7f0d400754c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0d50002050 tx=0x7f0d50001660 comp rx=0 tx=0).stop 2024-09-10T23:40:48.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.099+0000 7f0d65600640 1 -- 172.21.15.94:0/304497877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d600d0710 msgr2=0x7f0d600cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:48.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.099+0000 7f0d65600640 1 --2- 172.21.15.94:0/304497877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d600d0710 0x7f0d600cea50 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c01a450 tx=0x7f0d4c0040c0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.099+0000 7f0d65600640 1 -- 172.21.15.94:0/304497877 shutdown_connections 2024-09-10T23:40:48.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.099+0000 7f0d65600640 1 --2- 172.21.15.94:0/304497877 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0d40073000 0x7f0d400754c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.099+0000 7f0d65600640 1 --2- 172.21.15.94:0/304497877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d601455d0 0x7f0d601534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.099+0000 7f0d65600640 1 --2- 172.21.15.94:0/304497877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d600d0710 0x7f0d600cea50 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.099+0000 7f0d65600640 1 --2- 172.21.15.94:0/304497877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d600c5a70 0x7f0d600ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.099+0000 7f0d65600640 1 -- 172.21.15.94:0/304497877 >> 172.21.15.94:0/304497877 conn(0x7f0d600cfaf0 msgr2=0x7f0d60158340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:48.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.099+0000 7f0d65600640 1 -- 172.21.15.94:0/304497877 shutdown_connections 2024-09-10T23:40:48.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.099+0000 7f0d65600640 1 -- 172.21.15.94:0/304497877 wait complete. 2024-09-10T23:40:48.120 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T23:40:48.120 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:40:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.263+0000 7fd947c00640 1 -- 172.21.15.94:0/1251056467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd94015f5c0 msgr2=0x7fd9400cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.263+0000 7fd947c00640 1 --2- 172.21.15.94:0/1251056467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd94015f5c0 0x7fd9400cfb60 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fd930009900 tx=0x7fd93001c7c0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.263+0000 7fd947c00640 1 -- 172.21.15.94:0/1251056467 shutdown_connections 2024-09-10T23:40:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.263+0000 7fd947c00640 1 --2- 172.21.15.94:0/1251056467 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9400af310 0x7fd9400af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.263+0000 7fd947c00640 1 --2- 172.21.15.94:0/1251056467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd94015f5c0 0x7fd9400cfb60 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.263+0000 7fd947c00640 1 -- 172.21.15.94:0/1251056467 >> 172.21.15.94:0/1251056467 conn(0x7fd9400d1c60 msgr2=0x7fd9400d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.263+0000 7fd947c00640 1 -- 172.21.15.94:0/1251056467 shutdown_connections 2024-09-10T23:40:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.263+0000 7fd947c00640 1 -- 172.21.15.94:0/1251056467 wait complete. 2024-09-10T23:40:48.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd947c00640 1 Processor -- start 2024-09-10T23:40:48.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd947c00640 1 -- start start 2024-09-10T23:40:48.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd947c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9400af310 0x7fd9400b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:48.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd947c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9400b2bb0 0x7fd9401f9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:48.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd947c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd94015f5c0 0x7fd940153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:48.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd947c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd940169000 con 0x7fd94015f5c0 2024-09-10T23:40:48.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd947c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd940168e80 con 0x7fd9400b2bb0 2024-09-10T23:40:48.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd947c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd940169180 con 0x7fd9400af310 2024-09-10T23:40:48.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd945c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd94015f5c0 0x7fd940153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:48.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd945c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd94015f5c0 0x7fd940153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59340/0 (socket says 172.21.15.94:59340) 2024-09-10T23:40:48.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd945c00640 1 -- 172.21.15.94:0/18996139 learned_addr learned my addr 172.21.15.94:0/18996139 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:48.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd945200640 1 --2- 172.21.15.94:0/18996139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9400af310 0x7fd9400b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd937e00640 1 --2- 172.21.15.94:0/18996139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9400b2bb0 0x7fd9401f9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd945c00640 1 -- 172.21.15.94:0/18996139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9400af310 msgr2=0x7fd9400b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd945c00640 1 --2- 172.21.15.94:0/18996139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9400af310 0x7fd9400b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd945c00640 1 -- 172.21.15.94:0/18996139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9400b2bb0 msgr2=0x7fd9401f9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd945c00640 1 --2- 172.21.15.94:0/18996139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9400b2bb0 0x7fd9401f9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd945c00640 1 -- 172.21.15.94:0/18996139 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9401fa200 con 0x7fd94015f5c0 2024-09-10T23:40:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd945c00640 1 --2- 172.21.15.94:0/18996139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd94015f5c0 0x7fd940153c80 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fd93800ea20 tx=0x7fd93800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd93d600640 1 -- 172.21.15.94:0/18996139 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93800cd80 con 0x7fd94015f5c0 2024-09-10T23:40:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd93d600640 1 -- 172.21.15.94:0/18996139 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd938004c40 con 0x7fd94015f5c0 2024-09-10T23:40:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd947c00640 1 -- 172.21.15.94:0/18996139 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9401544d0 con 0x7fd94015f5c0 2024-09-10T23:40:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd93d600640 1 -- 172.21.15.94:0/18996139 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938005280 con 0x7fd94015f5c0 2024-09-10T23:40:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd947c00640 1 -- 172.21.15.94:0/18996139 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9401f94c0 con 0x7fd94015f5c0 2024-09-10T23:40:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.267+0000 7fd947c00640 1 -- 172.21.15.94:0/18996139 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9400af7a0 con 0x7fd94015f5c0 2024-09-10T23:40:48.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.271+0000 7fd93d600640 1 -- 172.21.15.94:0/18996139 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7fd93800f040 con 0x7fd94015f5c0 2024-09-10T23:40:48.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.271+0000 7fd93d600640 1 --2- 172.21.15.94:0/18996139 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd91c073000 0x7fd91c0754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:48.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.271+0000 7fd93d600640 1 -- 172.21.15.94:0/18996139 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7fd9380922f0 con 0x7fd94015f5c0 2024-09-10T23:40:48.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.271+0000 7fd945200640 1 --2- 172.21.15.94:0/18996139 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd91c073000 0x7fd91c0754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:48.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.271+0000 7fd945200640 1 --2- 172.21.15.94:0/18996139 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd91c073000 0x7fd91c0754c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd9300af000 tx=0x7fd93001b000 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:48.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.275+0000 7fd93d600640 1 -- 172.21.15.94:0/18996139 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd938016030 con 0x7fd94015f5c0 2024-09-10T23:40:48.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.471+0000 7fd947c00640 1 -- 172.21.15.94:0/18996139 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9400749c0 con 0x7fd94015f5c0 2024-09-10T23:40:48.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.475+0000 7fd93d600640 1 -- 172.21.15.94:0/18996139 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5239 (secure 0 0 0) 0x7fd93805ebb0 con 0x7fd94015f5c0 2024-09-10T23:40:48.473 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:48.473 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":116,"btime":"2024-09-10T23:40:48:088811+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/2653759764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2653759764},{"type":"v1","addr":"172.21.15.181:6801","nonce":2653759764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":8518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2240199080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2240199080},{"type":"v1","addr":"172.21.15.181:6835","nonce":2240199080}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1675026105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1675026105},{"type":"v1","addr":"172.21.15.94:6834","nonce":1675026105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:40:44.453069+0000","modified":"2024-09-10T23:40:47.471636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9171":{"gid":9171,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/3607539278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3607539278},{"type":"v1","addr":"172.21.15.94:6835","nonce":3607539278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9171,"qdb_cluster":[9171]},"id":13}]} 2024-09-10T23:40:48.473 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 116 2024-09-10T23:40:48.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.475+0000 7fd947c00640 1 -- 172.21.15.94:0/18996139 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd91c073000 msgr2=0x7fd91c0754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:48.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.475+0000 7fd947c00640 1 --2- 172.21.15.94:0/18996139 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd91c073000 0x7fd91c0754c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd9300af000 tx=0x7fd93001b000 comp rx=0 tx=0).stop 2024-09-10T23:40:48.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.475+0000 7fd947c00640 1 -- 172.21.15.94:0/18996139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd94015f5c0 msgr2=0x7fd940153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:48.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.475+0000 7fd947c00640 1 --2- 172.21.15.94:0/18996139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd94015f5c0 0x7fd940153c80 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fd93800ea20 tx=0x7fd93800eee0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.475+0000 7fd947c00640 1 -- 172.21.15.94:0/18996139 shutdown_connections 2024-09-10T23:40:48.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.475+0000 7fd947c00640 1 --2- 172.21.15.94:0/18996139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9400b2bb0 0x7fd9401f9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.475+0000 7fd947c00640 1 --2- 172.21.15.94:0/18996139 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd91c073000 0x7fd91c0754c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.475+0000 7fd947c00640 1 --2- 172.21.15.94:0/18996139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd94015f5c0 0x7fd940153c80 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.475+0000 7fd947c00640 1 --2- 172.21.15.94:0/18996139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9400af310 0x7fd9400b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.475+0000 7fd947c00640 1 -- 172.21.15.94:0/18996139 >> 172.21.15.94:0/18996139 conn(0x7fd9400d1c60 msgr2=0x7fd9401562d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:48.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.475+0000 7fd947c00640 1 -- 172.21.15.94:0/18996139 shutdown_connections 2024-09-10T23:40:48.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.475+0000 7fd947c00640 1 -- 172.21.15.94:0/18996139 wait complete. 2024-09-10T23:40:48.494 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:40:48.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.635+0000 7ff909400640 1 -- 172.21.15.94:0/182365628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9040cfcb0 msgr2=0x7ff9040d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:48.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff909400640 1 --2- 172.21.15.94:0/182365628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9040cfcb0 0x7ff9040d00d0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009840 tx=0x7ff8ec01c7f0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff909400640 1 -- 172.21.15.94:0/182365628 shutdown_connections 2024-09-10T23:40:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff909400640 1 --2- 172.21.15.94:0/182365628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9040cfcb0 0x7ff9040d00d0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff909400640 1 -- 172.21.15.94:0/182365628 >> 172.21.15.94:0/182365628 conn(0x7ff90406cdf0 msgr2=0x7ff90406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff909400640 1 -- 172.21.15.94:0/182365628 shutdown_connections 2024-09-10T23:40:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff909400640 1 -- 172.21.15.94:0/182365628 wait complete. 2024-09-10T23:40:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff909400640 1 Processor -- start 2024-09-10T23:40:48.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff909400640 1 -- start start 2024-09-10T23:40:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff909400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9040cddb0 0x7ff9041417d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff909400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9040cfcb0 0x7ff90414a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff909400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff90415ba50 0x7ff90414d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff909400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff904169bf0 con 0x7ff90415ba50 2024-09-10T23:40:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff909400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff904169a70 con 0x7ff9040cddb0 2024-09-10T23:40:48.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff909400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff904169d70 con 0x7ff9040cfcb0 2024-09-10T23:40:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff903200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff90415ba50 0x7ff90414d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff903200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff90415ba50 0x7ff90414d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59350/0 (socket says 172.21.15.94:59350) 2024-09-10T23:40:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff903200640 1 -- 172.21.15.94:0/1221075623 learned_addr learned my addr 172.21.15.94:0/1221075623 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff8f9e00640 1 --2- 172.21.15.94:0/1221075623 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9040cfcb0 0x7ff90414a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff902800640 1 --2- 172.21.15.94:0/1221075623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9040cddb0 0x7ff9041417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff903200640 1 -- 172.21.15.94:0/1221075623 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9040cfcb0 msgr2=0x7ff90414a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff903200640 1 --2- 172.21.15.94:0/1221075623 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9040cfcb0 0x7ff90414a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff903200640 1 -- 172.21.15.94:0/1221075623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9040cddb0 msgr2=0x7ff9041417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff903200640 1 --2- 172.21.15.94:0/1221075623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9040cddb0 0x7ff9041417d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff903200640 1 -- 172.21.15.94:0/1221075623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90414bf10 con 0x7ff90415ba50 2024-09-10T23:40:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.639+0000 7ff903200640 1 --2- 172.21.15.94:0/1221075623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff90415ba50 0x7ff90414d270 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00cbc0 tx=0x7ff8fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.643+0000 7ff8fb400640 1 -- 172.21.15.94:0/1221075623 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc016020 con 0x7ff90415ba50 2024-09-10T23:40:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.643+0000 7ff8fb400640 1 -- 172.21.15.94:0/1221075623 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8fc00ab00 con 0x7ff90415ba50 2024-09-10T23:40:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.643+0000 7ff8fb400640 1 -- 172.21.15.94:0/1221075623 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc012420 con 0x7ff90415ba50 2024-09-10T23:40:48.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.643+0000 7ff909400640 1 -- 172.21.15.94:0/1221075623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff90414c1f0 con 0x7ff90415ba50 2024-09-10T23:40:48.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.643+0000 7ff909400640 1 -- 172.21.15.94:0/1221075623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff90414c7b0 con 0x7ff90415ba50 2024-09-10T23:40:48.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.643+0000 7ff909400640 1 -- 172.21.15.94:0/1221075623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9040ce200 con 0x7ff90415ba50 2024-09-10T23:40:48.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.643+0000 7ff8fb400640 1 -- 172.21.15.94:0/1221075623 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7ff8fc0125c0 con 0x7ff90415ba50 2024-09-10T23:40:48.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.643+0000 7ff8fb400640 1 --2- 172.21.15.94:0/1221075623 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff8e0073030 0x7ff8e00754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:48.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.643+0000 7ff8fb400640 1 -- 172.21.15.94:0/1221075623 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7ff8fc092260 con 0x7ff90415ba50 2024-09-10T23:40:48.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.643+0000 7ff902800640 1 --2- 172.21.15.94:0/1221075623 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff8e0073030 0x7ff8e00754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:48.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.647+0000 7ff902800640 1 --2- 172.21.15.94:0/1221075623 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff8e0073030 0x7ff8e00754f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4000c10 tx=0x7ff8f4009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:48.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.647+0000 7ff8fb400640 1 -- 172.21.15.94:0/1221075623 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff8fc020020 con 0x7ff90415ba50 2024-09-10T23:40:48.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.815+0000 7ff909400640 1 -- 172.21.15.94:0/1221075623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff90415ef50 con 0x7ff90415ba50 2024-09-10T23:40:48.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.815+0000 7ff8fb400640 1 -- 172.21.15.94:0/1221075623 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) ==== 74+0+16707 (secure 0 0 0) 0x7ff8fc05eb20 con 0x7ff90415ba50 2024-09-10T23:40:48.816 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:48.816 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":162,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:40:46.071176+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:40:42.141193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-10T23:40:43.422826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:40:48.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.819+0000 7ff909400640 1 -- 172.21.15.94:0/1221075623 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff8e0073030 msgr2=0x7ff8e00754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.819+0000 7ff909400640 1 --2- 172.21.15.94:0/1221075623 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff8e0073030 0x7ff8e00754f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4000c10 tx=0x7ff8f4009040 comp rx=0 tx=0).stop 2024-09-10T23:40:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.819+0000 7ff909400640 1 -- 172.21.15.94:0/1221075623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff90415ba50 msgr2=0x7ff90414d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.819+0000 7ff909400640 1 --2- 172.21.15.94:0/1221075623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff90415ba50 0x7ff90414d270 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00cbc0 tx=0x7ff8fc00ccb0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.819+0000 7ff909400640 1 -- 172.21.15.94:0/1221075623 shutdown_connections 2024-09-10T23:40:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.819+0000 7ff909400640 1 --2- 172.21.15.94:0/1221075623 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff8e0073030 0x7ff8e00754f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.819+0000 7ff909400640 1 --2- 172.21.15.94:0/1221075623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff90415ba50 0x7ff90414d270 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.819+0000 7ff909400640 1 --2- 172.21.15.94:0/1221075623 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9040cfcb0 0x7ff90414a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.819+0000 7ff909400640 1 --2- 172.21.15.94:0/1221075623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9040cddb0 0x7ff9041417d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.819+0000 7ff909400640 1 -- 172.21.15.94:0/1221075623 >> 172.21.15.94:0/1221075623 conn(0x7ff90406cdf0 msgr2=0x7ff904156a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.819+0000 7ff909400640 1 -- 172.21.15.94:0/1221075623 shutdown_connections 2024-09-10T23:40:48.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.819+0000 7ff909400640 1 -- 172.21.15.94:0/1221075623 wait complete. 2024-09-10T23:40:48.839 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T23:40:48.839 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:40:48.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.979+0000 7fc6ae000640 1 -- 172.21.15.94:0/2886239461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a80b4c40 msgr2=0x7fc6a80cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:48.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.979+0000 7fc6ae000640 1 --2- 172.21.15.94:0/2886239461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a80b4c40 0x7fc6a80cfcb0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fc6a80d0890 tx=0x7fc69001c740 comp rx=0 tx=0).stop 2024-09-10T23:40:48.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.979+0000 7fc6ae000640 1 -- 172.21.15.94:0/2886239461 shutdown_connections 2024-09-10T23:40:48.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.979+0000 7fc6ae000640 1 --2- 172.21.15.94:0/2886239461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a80b4c40 0x7fc6a80cfcb0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.979+0000 7fc6ae000640 1 --2- 172.21.15.94:0/2886239461 >> v2:172.21.15.181:3300/0 conn(0x7fc6a815ba50 0x7fc6a80b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.979+0000 7fc6ae000640 1 -- 172.21.15.94:0/2886239461 >> 172.21.15.94:0/2886239461 conn(0x7fc6a806cdf0 msgr2=0x7fc6a806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:48.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.979+0000 7fc6ae000640 1 -- 172.21.15.94:0/2886239461 shutdown_connections 2024-09-10T23:40:48.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.979+0000 7fc6ae000640 1 -- 172.21.15.94:0/2886239461 wait complete. 2024-09-10T23:40:48.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6ae000640 1 Processor -- start 2024-09-10T23:40:48.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6ae000640 1 -- start start 2024-09-10T23:40:48.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6ae000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6a80b4c40 0x7fc6a81546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:48.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6ae000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6a80d01f0 0x7fc6a814af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:48.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6ae000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a815ba50 0x7fc6a814b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:48.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6ae000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8168a60 con 0x7fc6a815ba50 2024-09-10T23:40:48.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6ae000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a80cefb0 con 0x7fc6a80d01f0 2024-09-10T23:40:48.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6ae000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fc6a8168be0 con 0x7fc6a80b4c40 2024-09-10T23:40:48.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6a7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a815ba50 0x7fc6a814b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:48.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6a7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a815ba50 0x7fc6a814b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59366/0 (socket says 172.21.15.94:59366) 2024-09-10T23:40:48.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6a7e00640 1 -- 172.21.15.94:0/3327022317 learned_addr learned my addr 172.21.15.94:0/3327022317 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:48.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6a7400640 1 --2- 172.21.15.94:0/3327022317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6a80b4c40 0x7fc6a81546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:48.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc69ea00640 1 --2- 172.21.15.94:0/3327022317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6a80d01f0 0x7fc6a814af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:48.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6a7e00640 1 -- 172.21.15.94:0/3327022317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6a80b4c40 msgr2=0x7fc6a81546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:48.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6a7e00640 1 --2- 172.21.15.94:0/3327022317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6a80b4c40 0x7fc6a81546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6a7e00640 1 -- 172.21.15.94:0/3327022317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6a80d01f0 msgr2=0x7fc6a814af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:48.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6a7e00640 1 --2- 172.21.15.94:0/3327022317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6a80d01f0 0x7fc6a814af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:48.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6a7e00640 1 -- 172.21.15.94:0/3327022317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a8152200 con 0x7fc6a815ba50 2024-09-10T23:40:48.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6a7e00640 1 --2- 172.21.15.94:0/3327022317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a815ba50 0x7fc6a814b450 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000c3d0 tx=0x7fc6a000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:48.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc69fe00640 1 -- 172.21.15.94:0/3327022317 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0017070 con 0x7fc6a815ba50 2024-09-10T23:40:48.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc69fe00640 1 -- 172.21.15.94:0/3327022317 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6a0003600 con 0x7fc6a815ba50 2024-09-10T23:40:48.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc69fe00640 1 -- 172.21.15.94:0/3327022317 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0003a50 con 0x7fc6a815ba50 2024-09-10T23:40:48.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6ae000640 1 -- 172.21.15.94:0/3327022317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a81f8be0 con 0x7fc6a815ba50 2024-09-10T23:40:48.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.983+0000 7fc6ae000640 1 -- 172.21.15.94:0/3327022317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a81f91a0 con 0x7fc6a815ba50 2024-09-10T23:40:48.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.987+0000 7fc6ae000640 1 -- 172.21.15.94:0/3327022317 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a80cfcb0 con 0x7fc6a815ba50 2024-09-10T23:40:48.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.987+0000 7fc69fe00640 1 -- 172.21.15.94:0/3327022317 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7fc6a0003bf0 con 0x7fc6a815ba50 2024-09-10T23:40:48.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.987+0000 7fc69fe00640 1 --2- 172.21.15.94:0/3327022317 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc684073000 0x7fc6840754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:48.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.987+0000 7fc69fe00640 1 -- 172.21.15.94:0/3327022317 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7fc6a0091340 con 0x7fc6a815ba50 2024-09-10T23:40:48.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.987+0000 7fc6a7400640 1 --2- 172.21.15.94:0/3327022317 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc684073000 0x7fc6840754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:48.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.991+0000 7fc6a7400640 1 --2- 172.21.15.94:0/3327022317 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc684073000 0x7fc6840754c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc698009eb0 tx=0x7fc698009500 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:48.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:48.991+0000 7fc69fe00640 1 -- 172.21.15.94:0/3327022317 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc6a005e250 con 0x7fc6a815ba50 2024-09-10T23:40:49.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.159+0000 7fc6ae000640 1 -- 172.21.15.94:0/3327022317 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc6a814ba80 con 0x7fc684073000 2024-09-10T23:40:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.163+0000 7fc69fe00640 1 -- 172.21.15.94:0/3327022317 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7fc6a814ba80 con 0x7fc684073000 2024-09-10T23:40:49.164 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:49.165 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:40:49.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.167+0000 7fc6ae000640 1 -- 172.21.15.94:0/3327022317 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc684073000 msgr2=0x7fc6840754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:49.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.167+0000 7fc6ae000640 1 --2- 172.21.15.94:0/3327022317 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc684073000 0x7fc6840754c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc698009eb0 tx=0x7fc698009500 comp rx=0 tx=0).stop 2024-09-10T23:40:49.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.167+0000 7fc6ae000640 1 -- 172.21.15.94:0/3327022317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a815ba50 msgr2=0x7fc6a814b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:49.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.167+0000 7fc6ae000640 1 --2- 172.21.15.94:0/3327022317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a815ba50 0x7fc6a814b450 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000c3d0 tx=0x7fc6a000c890 comp rx=0 tx=0).stop 2024-09-10T23:40:49.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.167+0000 7fc6ae000640 1 -- 172.21.15.94:0/3327022317 shutdown_connections 2024-09-10T23:40:49.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.167+0000 7fc6ae000640 1 --2- 172.21.15.94:0/3327022317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6a80d01f0 0x7fc6a814af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:49.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.167+0000 7fc6ae000640 1 --2- 172.21.15.94:0/3327022317 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fc684073000 0x7fc6840754c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:49.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.167+0000 7fc6ae000640 1 --2- 172.21.15.94:0/3327022317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a815ba50 0x7fc6a814b450 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:49.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.167+0000 7fc6ae000640 1 --2- 172.21.15.94:0/3327022317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6a80b4c40 0x7fc6a81546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:49.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.167+0000 7fc6ae000640 1 -- 172.21.15.94:0/3327022317 >> 172.21.15.94:0/3327022317 conn(0x7fc6a806cdf0 msgr2=0x7fc6a815c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:49.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.167+0000 7fc6ae000640 1 -- 172.21.15.94:0/3327022317 shutdown_connections 2024-09-10T23:40:49.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.167+0000 7fc6ae000640 1 -- 172.21.15.94:0/3327022317 wait complete. 2024-09-10T23:40:49.185 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-10T23:40:48.787864+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283324,"kb_used_data":17160,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754691396,"statfs":{"total":773094113280,"available":772803989504,"internally_reserved":0,"allocated":17571840,"data_stored":8384884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.737031"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089786+0000","last_change":"2024-09-10T23:40:45.089655+0000","last_active":"2024-09-10T23:40:46.089786+0000","last_peered":"2024-09-10T23:40:46.089786+0000","last_clean":"2024-09-10T23:40:46.089786+0000","last_became_active":"2024-09-10T23:40:45.089311+0000","last_became_peered":"2024-09-10T23:40:45.089311+0000","last_unstale":"2024-09-10T23:40:46.089786+0000","last_undegraded":"2024-09-10T23:40:46.089786+0000","last_fullsized":"2024-09-10T23:40:46.089786+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:10.781519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.083167+0000","last_change":"2024-09-10T23:40:45.091720+0000","last_active":"2024-09-10T23:40:46.083167+0000","last_peered":"2024-09-10T23:40:46.083167+0000","last_clean":"2024-09-10T23:40:46.083167+0000","last_became_active":"2024-09-10T23:40:45.091354+0000","last_became_peered":"2024-09-10T23:40:45.091354+0000","last_unstale":"2024-09-10T23:40:46.083167+0000","last_undegraded":"2024-09-10T23:40:46.083167+0000","last_fullsized":"2024-09-10T23:40:46.083167+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:07.822794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181933+0000","last_change":"2024-09-10T23:40:45.084475+0000","last_active":"2024-09-10T23:40:46.181933+0000","last_peered":"2024-09-10T23:40:46.181933+0000","last_clean":"2024-09-10T23:40:46.181933+0000","last_became_active":"2024-09-10T23:40:45.084116+0000","last_became_peered":"2024-09-10T23:40:45.084116+0000","last_unstale":"2024-09-10T23:40:46.181933+0000","last_undegraded":"2024-09-10T23:40:46.181933+0000","last_fullsized":"2024-09-10T23:40:46.181933+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:59.689357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181043+0000","last_change":"2024-09-10T23:40:45.080097+0000","last_active":"2024-09-10T23:40:46.181043+0000","last_peered":"2024-09-10T23:40:46.181043+0000","last_clean":"2024-09-10T23:40:46.181043+0000","last_became_active":"2024-09-10T23:40:45.079590+0000","last_became_peered":"2024-09-10T23:40:45.079590+0000","last_unstale":"2024-09-10T23:40:46.181043+0000","last_undegraded":"2024-09-10T23:40:46.181043+0000","last_fullsized":"2024-09-10T23:40:46.181043+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:49.599844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182232+0000","last_change":"2024-09-10T23:40:45.092375+0000","last_active":"2024-09-10T23:40:46.182232+0000","last_peered":"2024-09-10T23:40:46.182232+0000","last_clean":"2024-09-10T23:40:46.182232+0000","last_became_active":"2024-09-10T23:40:45.092080+0000","last_became_peered":"2024-09-10T23:40:45.092080+0000","last_unstale":"2024-09-10T23:40:46.182232+0000","last_undegraded":"2024-09-10T23:40:46.182232+0000","last_fullsized":"2024-09-10T23:40:46.182232+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:24.625098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089480+0000","last_change":"2024-09-10T23:40:45.086618+0000","last_active":"2024-09-10T23:40:46.089480+0000","last_peered":"2024-09-10T23:40:46.089480+0000","last_clean":"2024-09-10T23:40:46.089480+0000","last_became_active":"2024-09-10T23:40:45.086363+0000","last_became_peered":"2024-09-10T23:40:45.086363+0000","last_unstale":"2024-09-10T23:40:46.089480+0000","last_undegraded":"2024-09-10T23:40:46.089480+0000","last_fullsized":"2024-09-10T23:40:46.089480+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:58.771287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089252+0000","last_change":"2024-09-10T23:40:45.087904+0000","last_active":"2024-09-10T23:40:46.089252+0000","last_peered":"2024-09-10T23:40:46.089252+0000","last_clean":"2024-09-10T23:40:46.089252+0000","last_became_active":"2024-09-10T23:40:45.087629+0000","last_became_peered":"2024-09-10T23:40:45.087629+0000","last_unstale":"2024-09-10T23:40:46.089252+0000","last_undegraded":"2024-09-10T23:40:46.089252+0000","last_fullsized":"2024-09-10T23:40:46.089252+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:27.475616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180775+0000","last_change":"2024-09-10T23:40:45.078731+0000","last_active":"2024-09-10T23:40:46.180775+0000","last_peered":"2024-09-10T23:40:46.180775+0000","last_clean":"2024-09-10T23:40:46.180775+0000","last_became_active":"2024-09-10T23:40:45.078285+0000","last_became_peered":"2024-09-10T23:40:45.078285+0000","last_unstale":"2024-09-10T23:40:46.180775+0000","last_undegraded":"2024-09-10T23:40:46.180775+0000","last_fullsized":"2024-09-10T23:40:46.180775+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:43.466953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.172015+0000","last_change":"2024-09-10T23:40:45.087453+0000","last_active":"2024-09-10T23:40:46.172015+0000","last_peered":"2024-09-10T23:40:46.172015+0000","last_clean":"2024-09-10T23:40:46.172015+0000","last_became_active":"2024-09-10T23:40:45.087096+0000","last_became_peered":"2024-09-10T23:40:45.087096+0000","last_unstale":"2024-09-10T23:40:46.172015+0000","last_undegraded":"2024-09-10T23:40:46.172015+0000","last_fullsized":"2024-09-10T23:40:46.172015+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:16.873022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182339+0000","last_change":"2024-09-10T23:40:45.085815+0000","last_active":"2024-09-10T23:40:46.182339+0000","last_peered":"2024-09-10T23:40:46.182339+0000","last_clean":"2024-09-10T23:40:46.182339+0000","last_became_active":"2024-09-10T23:40:45.083284+0000","last_became_peered":"2024-09-10T23:40:45.083284+0000","last_unstale":"2024-09-10T23:40:46.182339+0000","last_undegraded":"2024-09-10T23:40:46.182339+0000","last_fullsized":"2024-09-10T23:40:46.182339+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:47:35.838329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089891+0000","last_change":"2024-09-10T23:40:45.087685+0000","last_active":"2024-09-10T23:40:46.089891+0000","last_peered":"2024-09-10T23:40:46.089891+0000","last_clean":"2024-09-10T23:40:46.089891+0000","last_became_active":"2024-09-10T23:40:45.085525+0000","last_became_peered":"2024-09-10T23:40:45.085525+0000","last_unstale":"2024-09-10T23:40:46.089891+0000","last_undegraded":"2024-09-10T23:40:46.089891+0000","last_fullsized":"2024-09-10T23:40:46.089891+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:12.553879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180708+0000","last_change":"2024-09-10T23:40:45.070594+0000","last_active":"2024-09-10T23:40:46.180708+0000","last_peered":"2024-09-10T23:40:46.180708+0000","last_clean":"2024-09-10T23:40:46.180708+0000","last_became_active":"2024-09-10T23:40:45.070051+0000","last_became_peered":"2024-09-10T23:40:45.070051+0000","last_unstale":"2024-09-10T23:40:46.180708+0000","last_undegraded":"2024-09-10T23:40:46.180708+0000","last_fullsized":"2024-09-10T23:40:46.180708+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:41.146987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180567+0000","last_change":"2024-09-10T23:40:45.085757+0000","last_active":"2024-09-10T23:40:46.180567+0000","last_peered":"2024-09-10T23:40:46.180567+0000","last_clean":"2024-09-10T23:40:46.180567+0000","last_became_active":"2024-09-10T23:40:45.085484+0000","last_became_peered":"2024-09-10T23:40:45.085484+0000","last_unstale":"2024-09-10T23:40:46.180567+0000","last_undegraded":"2024-09-10T23:40:46.180567+0000","last_fullsized":"2024-09-10T23:40:46.180567+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:57.777941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.088350+0000","last_change":"2024-09-10T23:40:45.078342+0000","last_active":"2024-09-10T23:40:46.088350+0000","last_peered":"2024-09-10T23:40:46.088350+0000","last_clean":"2024-09-10T23:40:46.088350+0000","last_became_active":"2024-09-10T23:40:45.078067+0000","last_became_peered":"2024-09-10T23:40:45.078067+0000","last_unstale":"2024-09-10T23:40:46.088350+0000","last_undegraded":"2024-09-10T23:40:46.088350+0000","last_fullsized":"2024-09-10T23:40:46.088350+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:55.912272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089694+0000","last_change":"2024-09-10T23:40:45.084613+0000","last_active":"2024-09-10T23:40:46.089694+0000","last_peered":"2024-09-10T23:40:46.089694+0000","last_clean":"2024-09-10T23:40:46.089694+0000","last_became_active":"2024-09-10T23:40:45.084308+0000","last_became_peered":"2024-09-10T23:40:45.084308+0000","last_unstale":"2024-09-10T23:40:46.089694+0000","last_undegraded":"2024-09-10T23:40:46.089694+0000","last_fullsized":"2024-09-10T23:40:46.089694+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:43.615447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089103+0000","last_change":"2024-09-10T23:40:45.090696+0000","last_active":"2024-09-10T23:40:46.089103+0000","last_peered":"2024-09-10T23:40:46.089103+0000","last_clean":"2024-09-10T23:40:46.089103+0000","last_became_active":"2024-09-10T23:40:45.090380+0000","last_became_peered":"2024-09-10T23:40:45.090380+0000","last_unstale":"2024-09-10T23:40:46.089103+0000","last_undegraded":"2024-09-10T23:40:46.089103+0000","last_fullsized":"2024-09-10T23:40:46.089103+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:56.322876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.090060+0000","last_change":"2024-09-10T23:40:45.103193+0000","last_active":"2024-09-10T23:40:46.090060+0000","last_peered":"2024-09-10T23:40:46.090060+0000","last_clean":"2024-09-10T23:40:46.090060+0000","last_became_active":"2024-09-10T23:40:45.102814+0000","last_became_peered":"2024-09-10T23:40:45.102814+0000","last_unstale":"2024-09-10T23:40:46.090060+0000","last_undegraded":"2024-09-10T23:40:46.090060+0000","last_fullsized":"2024-09-10T23:40:46.090060+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:24.718349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182531+0000","last_change":"2024-09-10T23:40:45.092610+0000","last_active":"2024-09-10T23:40:46.182531+0000","last_peered":"2024-09-10T23:40:46.182531+0000","last_clean":"2024-09-10T23:40:46.182531+0000","last_became_active":"2024-09-10T23:40:45.092353+0000","last_became_peered":"2024-09-10T23:40:45.092353+0000","last_unstale":"2024-09-10T23:40:46.182531+0000","last_undegraded":"2024-09-10T23:40:46.182531+0000","last_fullsized":"2024-09-10T23:40:46.182531+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:49.760581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182333+0000","last_change":"2024-09-10T23:40:45.085629+0000","last_active":"2024-09-10T23:40:46.182333+0000","last_peered":"2024-09-10T23:40:46.182333+0000","last_clean":"2024-09-10T23:40:46.182333+0000","last_became_active":"2024-09-10T23:40:45.085359+0000","last_became_peered":"2024-09-10T23:40:45.085359+0000","last_unstale":"2024-09-10T23:40:46.182333+0000","last_undegraded":"2024-09-10T23:40:46.182333+0000","last_fullsized":"2024-09-10T23:40:46.182333+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:37:04.831815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181944+0000","last_change":"2024-09-10T23:40:45.088469+0000","last_active":"2024-09-10T23:40:46.181944+0000","last_peered":"2024-09-10T23:40:46.181944+0000","last_clean":"2024-09-10T23:40:46.181944+0000","last_became_active":"2024-09-10T23:40:45.086877+0000","last_became_peered":"2024-09-10T23:40:45.086877+0000","last_unstale":"2024-09-10T23:40:46.181944+0000","last_undegraded":"2024-09-10T23:40:46.181944+0000","last_fullsized":"2024-09-10T23:40:46.181944+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:49.527141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180434+0000","last_change":"2024-09-10T23:40:45.074233+0000","last_active":"2024-09-10T23:40:46.180434+0000","last_peered":"2024-09-10T23:40:46.180434+0000","last_clean":"2024-09-10T23:40:46.180434+0000","last_became_active":"2024-09-10T23:40:45.073965+0000","last_became_peered":"2024-09-10T23:40:45.073965+0000","last_unstale":"2024-09-10T23:40:46.180434+0000","last_undegraded":"2024-09-10T23:40:46.180434+0000","last_fullsized":"2024-09-10T23:40:46.180434+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:50.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.083331+0000","last_change":"2024-09-10T23:40:45.076285+0000","last_active":"2024-09-10T23:40:46.083331+0000","last_peered":"2024-09-10T23:40:46.083331+0000","last_clean":"2024-09-10T23:40:46.083331+0000","last_became_active":"2024-09-10T23:40:45.076015+0000","last_became_peered":"2024-09-10T23:40:45.076015+0000","last_unstale":"2024-09-10T23:40:46.083331+0000","last_undegraded":"2024-09-10T23:40:46.083331+0000","last_fullsized":"2024-09-10T23:40:46.083331+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:29.781657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180601+0000","last_change":"2024-09-10T23:40:45.080722+0000","last_active":"2024-09-10T23:40:46.180601+0000","last_peered":"2024-09-10T23:40:46.180601+0000","last_clean":"2024-09-10T23:40:46.180601+0000","last_became_active":"2024-09-10T23:40:45.078520+0000","last_became_peered":"2024-09-10T23:40:45.078520+0000","last_unstale":"2024-09-10T23:40:46.180601+0000","last_undegraded":"2024-09-10T23:40:46.180601+0000","last_fullsized":"2024-09-10T23:40:46.180601+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:49.894589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.171848+0000","last_change":"2024-09-10T23:40:45.079335+0000","last_active":"2024-09-10T23:40:46.171848+0000","last_peered":"2024-09-10T23:40:46.171848+0000","last_clean":"2024-09-10T23:40:46.171848+0000","last_became_active":"2024-09-10T23:40:45.078994+0000","last_became_peered":"2024-09-10T23:40:45.078994+0000","last_unstale":"2024-09-10T23:40:46.171848+0000","last_undegraded":"2024-09-10T23:40:46.171848+0000","last_fullsized":"2024-09-10T23:40:46.171848+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:52.866792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181289+0000","last_change":"2024-09-10T23:40:45.091347+0000","last_active":"2024-09-10T23:40:46.181289+0000","last_peered":"2024-09-10T23:40:46.181289+0000","last_clean":"2024-09-10T23:40:46.181289+0000","last_became_active":"2024-09-10T23:40:45.091053+0000","last_became_peered":"2024-09-10T23:40:45.091053+0000","last_unstale":"2024-09-10T23:40:46.181289+0000","last_undegraded":"2024-09-10T23:40:46.181289+0000","last_fullsized":"2024-09-10T23:40:46.181289+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:04:47.979677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181620+0000","last_change":"2024-09-10T23:40:45.074492+0000","last_active":"2024-09-10T23:40:46.181620+0000","last_peered":"2024-09-10T23:40:46.181620+0000","last_clean":"2024-09-10T23:40:46.181620+0000","last_became_active":"2024-09-10T23:40:45.074126+0000","last_became_peered":"2024-09-10T23:40:45.074126+0000","last_unstale":"2024-09-10T23:40:46.181620+0000","last_undegraded":"2024-09-10T23:40:46.181620+0000","last_fullsized":"2024-09-10T23:40:46.181620+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:28.996896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180964+0000","last_change":"2024-09-10T23:40:45.088346+0000","last_active":"2024-09-10T23:40:46.180964+0000","last_peered":"2024-09-10T23:40:46.180964+0000","last_clean":"2024-09-10T23:40:46.180964+0000","last_became_active":"2024-09-10T23:40:45.088100+0000","last_became_peered":"2024-09-10T23:40:45.088100+0000","last_unstale":"2024-09-10T23:40:46.180964+0000","last_undegraded":"2024-09-10T23:40:46.180964+0000","last_fullsized":"2024-09-10T23:40:46.180964+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:00.773536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181164+0000","last_change":"2024-09-10T23:40:45.081358+0000","last_active":"2024-09-10T23:40:46.181164+0000","last_peered":"2024-09-10T23:40:46.181164+0000","last_clean":"2024-09-10T23:40:46.181164+0000","last_became_active":"2024-09-10T23:40:45.079006+0000","last_became_peered":"2024-09-10T23:40:45.079006+0000","last_unstale":"2024-09-10T23:40:46.181164+0000","last_undegraded":"2024-09-10T23:40:46.181164+0000","last_fullsized":"2024-09-10T23:40:46.181164+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:40.336638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.082960+0000","last_change":"2024-09-10T23:40:45.077105+0000","last_active":"2024-09-10T23:40:46.082960+0000","last_peered":"2024-09-10T23:40:46.082960+0000","last_clean":"2024-09-10T23:40:46.082960+0000","last_became_active":"2024-09-10T23:40:45.076711+0000","last_became_peered":"2024-09-10T23:40:45.076711+0000","last_unstale":"2024-09-10T23:40:46.082960+0000","last_undegraded":"2024-09-10T23:40:46.082960+0000","last_fullsized":"2024-09-10T23:40:46.082960+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.061601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180576+0000","last_change":"2024-09-10T23:40:45.097534+0000","last_active":"2024-09-10T23:40:46.180576+0000","last_peered":"2024-09-10T23:40:46.180576+0000","last_clean":"2024-09-10T23:40:46.180576+0000","last_became_active":"2024-09-10T23:40:45.097278+0000","last_became_peered":"2024-09-10T23:40:45.097278+0000","last_unstale":"2024-09-10T23:40:46.180576+0000","last_undegraded":"2024-09-10T23:40:46.180576+0000","last_fullsized":"2024-09-10T23:40:46.180576+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:19.477684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181744+0000","last_change":"2024-09-10T23:40:45.082465+0000","last_active":"2024-09-10T23:40:46.181744+0000","last_peered":"2024-09-10T23:40:46.181744+0000","last_clean":"2024-09-10T23:40:46.181744+0000","last_became_active":"2024-09-10T23:40:45.082143+0000","last_became_peered":"2024-09-10T23:40:45.082143+0000","last_unstale":"2024-09-10T23:40:46.181744+0000","last_undegraded":"2024-09-10T23:40:46.181744+0000","last_fullsized":"2024-09-10T23:40:46.181744+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:12.062896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.088525+0000","last_change":"2024-09-10T23:40:45.084090+0000","last_active":"2024-09-10T23:40:46.088525+0000","last_peered":"2024-09-10T23:40:46.088525+0000","last_clean":"2024-09-10T23:40:46.088525+0000","last_became_active":"2024-09-10T23:40:45.083222+0000","last_became_peered":"2024-09-10T23:40:45.083222+0000","last_unstale":"2024-09-10T23:40:46.088525+0000","last_undegraded":"2024-09-10T23:40:46.088525+0000","last_fullsized":"2024-09-10T23:40:46.088525+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:06.028814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181279+0000","last_change":"2024-09-10T23:40:45.084956+0000","last_active":"2024-09-10T23:40:46.181279+0000","last_peered":"2024-09-10T23:40:46.181279+0000","last_clean":"2024-09-10T23:40:46.181279+0000","last_became_active":"2024-09-10T23:40:45.084695+0000","last_became_peered":"2024-09-10T23:40:45.084695+0000","last_unstale":"2024-09-10T23:40:46.181279+0000","last_undegraded":"2024-09-10T23:40:46.181279+0000","last_fullsized":"2024-09-10T23:40:46.181279+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:42.070233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"161'4","reported_seq":19,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181245+0000","last_change":"2024-09-10T23:40:44.080123+0000","last_active":"2024-09-10T23:40:46.181245+0000","last_peered":"2024-09-10T23:40:46.181245+0000","last_clean":"2024-09-10T23:40:46.181245+0000","last_became_active":"2024-09-10T23:40:44.079862+0000","last_became_peered":"2024-09-10T23:40:44.079862+0000","last_unstale":"2024-09-10T23:40:46.181245+0000","last_undegraded":"2024-09-10T23:40:46.181245+0000","last_fullsized":"2024-09-10T23:40:46.181245+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:14.466933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181292+0000","last_change":"2024-09-10T23:40:45.081330+0000","last_active":"2024-09-10T23:40:46.181292+0000","last_peered":"2024-09-10T23:40:46.181292+0000","last_clean":"2024-09-10T23:40:46.181292+0000","last_became_active":"2024-09-10T23:40:45.080777+0000","last_became_peered":"2024-09-10T23:40:45.080777+0000","last_unstale":"2024-09-10T23:40:46.181292+0000","last_undegraded":"2024-09-10T23:40:46.181292+0000","last_fullsized":"2024-09-10T23:40:46.181292+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:13.212190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.083421+0000","last_change":"2024-09-10T23:40:44.083068+0000","last_active":"2024-09-10T23:40:46.083421+0000","last_peered":"2024-09-10T23:40:46.083421+0000","last_clean":"2024-09-10T23:40:46.083421+0000","last_became_active":"2024-09-10T23:40:44.081744+0000","last_became_peered":"2024-09-10T23:40:44.081744+0000","last_unstale":"2024-09-10T23:40:46.083421+0000","last_undegraded":"2024-09-10T23:40:46.083421+0000","last_fullsized":"2024-09-10T23:40:46.083421+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:09.367656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182251+0000","last_change":"2024-09-10T23:40:45.093812+0000","last_active":"2024-09-10T23:40:46.182251+0000","last_peered":"2024-09-10T23:40:46.182251+0000","last_clean":"2024-09-10T23:40:46.182251+0000","last_became_active":"2024-09-10T23:40:45.092777+0000","last_became_peered":"2024-09-10T23:40:45.092777+0000","last_unstale":"2024-09-10T23:40:46.182251+0000","last_undegraded":"2024-09-10T23:40:46.182251+0000","last_fullsized":"2024-09-10T23:40:46.182251+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:40.517676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181303+0000","last_change":"2024-09-10T23:40:44.076217+0000","last_active":"2024-09-10T23:40:46.181303+0000","last_peered":"2024-09-10T23:40:46.181303+0000","last_clean":"2024-09-10T23:40:46.181303+0000","last_became_active":"2024-09-10T23:40:44.075964+0000","last_became_peered":"2024-09-10T23:40:44.075964+0000","last_unstale":"2024-09-10T23:40:46.181303+0000","last_undegraded":"2024-09-10T23:40:46.181303+0000","last_fullsized":"2024-09-10T23:40:46.181303+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:35.744447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182402+0000","last_change":"2024-09-10T23:40:45.084782+0000","last_active":"2024-09-10T23:40:46.182402+0000","last_peered":"2024-09-10T23:40:46.182402+0000","last_clean":"2024-09-10T23:40:46.182402+0000","last_became_active":"2024-09-10T23:40:45.084203+0000","last_became_peered":"2024-09-10T23:40:45.084203+0000","last_unstale":"2024-09-10T23:40:46.182402+0000","last_undegraded":"2024-09-10T23:40:46.182402+0000","last_fullsized":"2024-09-10T23:40:46.182402+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:51.901951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1c","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181207+0000","last_change":"2024-09-10T23:40:44.081505+0000","last_active":"2024-09-10T23:40:46.181207+0000","last_peered":"2024-09-10T23:40:46.181207+0000","last_clean":"2024-09-10T23:40:46.181207+0000","last_became_active":"2024-09-10T23:40:44.081254+0000","last_became_peered":"2024-09-10T23:40:44.081254+0000","last_unstale":"2024-09-10T23:40:46.181207+0000","last_undegraded":"2024-09-10T23:40:46.181207+0000","last_fullsized":"2024-09-10T23:40:46.181207+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:26.630002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180811+0000","last_change":"2024-09-10T23:40:45.082104+0000","last_active":"2024-09-10T23:40:46.180811+0000","last_peered":"2024-09-10T23:40:46.180811+0000","last_clean":"2024-09-10T23:40:46.180811+0000","last_became_active":"2024-09-10T23:40:45.081841+0000","last_became_peered":"2024-09-10T23:40:45.081841+0000","last_unstale":"2024-09-10T23:40:46.180811+0000","last_undegraded":"2024-09-10T23:40:46.180811+0000","last_fullsized":"2024-09-10T23:40:46.180811+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:35.028658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089901+0000","last_change":"2024-09-10T23:40:44.074983+0000","last_active":"2024-09-10T23:40:46.089901+0000","last_peered":"2024-09-10T23:40:46.089901+0000","last_clean":"2024-09-10T23:40:46.089901+0000","last_became_active":"2024-09-10T23:40:44.074679+0000","last_became_peered":"2024-09-10T23:40:44.074679+0000","last_unstale":"2024-09-10T23:40:46.089901+0000","last_undegraded":"2024-09-10T23:40:46.089901+0000","last_fullsized":"2024-09-10T23:40:46.089901+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:14.301580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089182+0000","last_change":"2024-09-10T23:40:45.087279+0000","last_active":"2024-09-10T23:40:46.089182+0000","last_peered":"2024-09-10T23:40:46.089182+0000","last_clean":"2024-09-10T23:40:46.089182+0000","last_became_active":"2024-09-10T23:40:45.086944+0000","last_became_peered":"2024-09-10T23:40:45.086944+0000","last_unstale":"2024-09-10T23:40:46.089182+0000","last_undegraded":"2024-09-10T23:40:46.089182+0000","last_fullsized":"2024-09-10T23:40:46.089182+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:34.228170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180685+0000","last_change":"2024-09-10T23:40:44.079045+0000","last_active":"2024-09-10T23:40:46.180685+0000","last_peered":"2024-09-10T23:40:46.180685+0000","last_clean":"2024-09-10T23:40:46.180685+0000","last_became_active":"2024-09-10T23:40:44.078710+0000","last_became_peered":"2024-09-10T23:40:44.078710+0000","last_unstale":"2024-09-10T23:40:46.180685+0000","last_undegraded":"2024-09-10T23:40:46.180685+0000","last_fullsized":"2024-09-10T23:40:46.180685+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:24.057491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.171760+0000","last_change":"2024-09-10T23:40:45.093006+0000","last_active":"2024-09-10T23:40:46.171760+0000","last_peered":"2024-09-10T23:40:46.171760+0000","last_clean":"2024-09-10T23:40:46.171760+0000","last_became_active":"2024-09-10T23:40:45.092679+0000","last_became_peered":"2024-09-10T23:40:45.092679+0000","last_unstale":"2024-09-10T23:40:46.171760+0000","last_undegraded":"2024-09-10T23:40:46.171760+0000","last_fullsized":"2024-09-10T23:40:46.171760+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:05.330116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180334+0000","last_change":"2024-09-10T23:40:44.080346+0000","last_active":"2024-09-10T23:40:46.180334+0000","last_peered":"2024-09-10T23:40:46.180334+0000","last_clean":"2024-09-10T23:40:46.180334+0000","last_became_active":"2024-09-10T23:40:44.079671+0000","last_became_peered":"2024-09-10T23:40:44.079671+0000","last_unstale":"2024-09-10T23:40:46.180334+0000","last_undegraded":"2024-09-10T23:40:46.180334+0000","last_fullsized":"2024-09-10T23:40:46.180334+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:23:56.181920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180946+0000","last_change":"2024-09-10T23:40:45.085153+0000","last_active":"2024-09-10T23:40:46.180946+0000","last_peered":"2024-09-10T23:40:46.180946+0000","last_clean":"2024-09-10T23:40:46.180946+0000","last_became_active":"2024-09-10T23:40:45.084638+0000","last_became_peered":"2024-09-10T23:40:45.084638+0000","last_unstale":"2024-09-10T23:40:46.180946+0000","last_undegraded":"2024-09-10T23:40:46.180946+0000","last_fullsized":"2024-09-10T23:40:46.180946+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:16.773746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182345+0000","last_change":"2024-09-10T23:40:44.084811+0000","last_active":"2024-09-10T23:40:46.182345+0000","last_peered":"2024-09-10T23:40:46.182345+0000","last_clean":"2024-09-10T23:40:46.182345+0000","last_became_active":"2024-09-10T23:40:44.084437+0000","last_became_peered":"2024-09-10T23:40:44.084437+0000","last_unstale":"2024-09-10T23:40:46.182345+0000","last_undegraded":"2024-09-10T23:40:46.182345+0000","last_fullsized":"2024-09-10T23:40:46.182345+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:07.553007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.088613+0000","last_change":"2024-09-10T23:40:45.073784+0000","last_active":"2024-09-10T23:40:46.088613+0000","last_peered":"2024-09-10T23:40:46.088613+0000","last_clean":"2024-09-10T23:40:46.088613+0000","last_became_active":"2024-09-10T23:40:45.073492+0000","last_became_peered":"2024-09-10T23:40:45.073492+0000","last_unstale":"2024-09-10T23:40:46.088613+0000","last_undegraded":"2024-09-10T23:40:46.088613+0000","last_fullsized":"2024-09-10T23:40:46.088613+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:37.402798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181038+0000","last_change":"2024-09-10T23:40:44.080664+0000","last_active":"2024-09-10T23:40:46.181038+0000","last_peered":"2024-09-10T23:40:46.181038+0000","last_clean":"2024-09-10T23:40:46.181038+0000","last_became_active":"2024-09-10T23:40:44.080165+0000","last_became_peered":"2024-09-10T23:40:44.080165+0000","last_unstale":"2024-09-10T23:40:46.181038+0000","last_undegraded":"2024-09-10T23:40:46.181038+0000","last_fullsized":"2024-09-10T23:40:46.181038+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:49.391750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.172010+0000","last_change":"2024-09-10T23:40:45.077463+0000","last_active":"2024-09-10T23:40:46.172010+0000","last_peered":"2024-09-10T23:40:46.172010+0000","last_clean":"2024-09-10T23:40:46.172010+0000","last_became_active":"2024-09-10T23:40:45.077173+0000","last_became_peered":"2024-09-10T23:40:45.077173+0000","last_unstale":"2024-09-10T23:40:46.172010+0000","last_undegraded":"2024-09-10T23:40:46.172010+0000","last_fullsized":"2024-09-10T23:40:46.172010+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:03.998259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"161'3","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181342+0000","last_change":"2024-09-10T23:40:44.082263+0000","last_active":"2024-09-10T23:40:46.181342+0000","last_peered":"2024-09-10T23:40:46.181342+0000","last_clean":"2024-09-10T23:40:46.181342+0000","last_became_active":"2024-09-10T23:40:44.082008+0000","last_became_peered":"2024-09-10T23:40:44.082008+0000","last_unstale":"2024-09-10T23:40:46.181342+0000","last_undegraded":"2024-09-10T23:40:46.181342+0000","last_fullsized":"2024-09-10T23:40:46.181342+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:53:08.264800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181158+0000","last_change":"2024-09-10T23:40:45.078110+0000","last_active":"2024-09-10T23:40:46.181158+0000","last_peered":"2024-09-10T23:40:46.181158+0000","last_clean":"2024-09-10T23:40:46.181158+0000","last_became_active":"2024-09-10T23:40:45.077829+0000","last_became_peered":"2024-09-10T23:40:45.077829+0000","last_unstale":"2024-09-10T23:40:46.181158+0000","last_undegraded":"2024-09-10T23:40:46.181158+0000","last_fullsized":"2024-09-10T23:40:46.181158+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:38.343003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"161'6","reported_seq":21,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.171831+0000","last_change":"2024-09-10T23:40:44.076821+0000","last_active":"2024-09-10T23:40:46.171831+0000","last_peered":"2024-09-10T23:40:46.171831+0000","last_clean":"2024-09-10T23:40:46.171831+0000","last_became_active":"2024-09-10T23:40:44.076552+0000","last_became_peered":"2024-09-10T23:40:44.076552+0000","last_unstale":"2024-09-10T23:40:46.171831+0000","last_undegraded":"2024-09-10T23:40:46.171831+0000","last_fullsized":"2024-09-10T23:40:46.171831+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:14.955232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.088356+0000","last_change":"2024-09-10T23:40:45.085846+0000","last_active":"2024-09-10T23:40:46.088356+0000","last_peered":"2024-09-10T23:40:46.088356+0000","last_clean":"2024-09-10T23:40:46.088356+0000","last_became_active":"2024-09-10T23:40:45.085566+0000","last_became_peered":"2024-09-10T23:40:45.085566+0000","last_unstale":"2024-09-10T23:40:46.088356+0000","last_undegraded":"2024-09-10T23:40:46.088356+0000","last_fullsized":"2024-09-10T23:40:46.088356+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:43.635344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.13","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.171690+0000","last_change":"2024-09-10T23:40:44.084413+0000","last_active":"2024-09-10T23:40:46.171690+0000","last_peered":"2024-09-10T23:40:46.171690+0000","last_clean":"2024-09-10T23:40:46.171690+0000","last_became_active":"2024-09-10T23:40:44.084132+0000","last_became_peered":"2024-09-10T23:40:44.084132+0000","last_unstale":"2024-09-10T23:40:46.171690+0000","last_undegraded":"2024-09-10T23:40:46.171690+0000","last_fullsized":"2024-09-10T23:40:46.171690+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:44.715798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182545+0000","last_change":"2024-09-10T23:40:45.101237+0000","last_active":"2024-09-10T23:40:46.182545+0000","last_peered":"2024-09-10T23:40:46.182545+0000","last_clean":"2024-09-10T23:40:46.182545+0000","last_became_active":"2024-09-10T23:40:45.100916+0000","last_became_peered":"2024-09-10T23:40:45.100916+0000","last_unstale":"2024-09-10T23:40:46.182545+0000","last_undegraded":"2024-09-10T23:40:46.182545+0000","last_fullsized":"2024-09-10T23:40:46.182545+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:08.801109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.082866+0000","last_change":"2024-09-10T23:40:44.078360+0000","last_active":"2024-09-10T23:40:46.082866+0000","last_peered":"2024-09-10T23:40:46.082866+0000","last_clean":"2024-09-10T23:40:46.082866+0000","last_became_active":"2024-09-10T23:40:44.078103+0000","last_became_peered":"2024-09-10T23:40:44.078103+0000","last_unstale":"2024-09-10T23:40:46.082866+0000","last_undegraded":"2024-09-10T23:40:46.082866+0000","last_fullsized":"2024-09-10T23:40:46.082866+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:04.092914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181296+0000","last_change":"2024-09-10T23:40:45.088219+0000","last_active":"2024-09-10T23:40:46.181296+0000","last_peered":"2024-09-10T23:40:46.181296+0000","last_clean":"2024-09-10T23:40:46.181296+0000","last_became_active":"2024-09-10T23:40:45.087895+0000","last_became_peered":"2024-09-10T23:40:45.087895+0000","last_unstale":"2024-09-10T23:40:46.181296+0000","last_undegraded":"2024-09-10T23:40:46.181296+0000","last_fullsized":"2024-09-10T23:40:46.181296+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:08.426049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180711+0000","last_change":"2024-09-10T23:40:44.077071+0000","last_active":"2024-09-10T23:40:46.180711+0000","last_peered":"2024-09-10T23:40:46.180711+0000","last_clean":"2024-09-10T23:40:46.180711+0000","last_became_active":"2024-09-10T23:40:44.076812+0000","last_became_peered":"2024-09-10T23:40:44.076812+0000","last_unstale":"2024-09-10T23:40:46.180711+0000","last_undegraded":"2024-09-10T23:40:46.180711+0000","last_fullsized":"2024-09-10T23:40:46.180711+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:08.132953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089544+0000","last_change":"2024-09-10T23:40:45.090469+0000","last_active":"2024-09-10T23:40:46.089544+0000","last_peered":"2024-09-10T23:40:46.089544+0000","last_clean":"2024-09-10T23:40:46.089544+0000","last_became_active":"2024-09-10T23:40:45.090130+0000","last_became_peered":"2024-09-10T23:40:45.090130+0000","last_unstale":"2024-09-10T23:40:46.089544+0000","last_undegraded":"2024-09-10T23:40:46.089544+0000","last_fullsized":"2024-09-10T23:40:46.089544+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:11.039668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.10","version":"161'8","reported_seq":23,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181344+0000","last_change":"2024-09-10T23:40:44.079722+0000","last_active":"2024-09-10T23:40:46.181344+0000","last_peered":"2024-09-10T23:40:46.181344+0000","last_clean":"2024-09-10T23:40:46.181344+0000","last_became_active":"2024-09-10T23:40:44.074591+0000","last_became_peered":"2024-09-10T23:40:44.074591+0000","last_unstale":"2024-09-10T23:40:46.181344+0000","last_undegraded":"2024-09-10T23:40:46.181344+0000","last_fullsized":"2024-09-10T23:40:46.181344+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:23.708559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181323+0000","last_change":"2024-09-10T23:40:45.084770+0000","last_active":"2024-09-10T23:40:46.181323+0000","last_peered":"2024-09-10T23:40:46.181323+0000","last_clean":"2024-09-10T23:40:46.181323+0000","last_became_active":"2024-09-10T23:40:45.083550+0000","last_became_peered":"2024-09-10T23:40:45.083550+0000","last_unstale":"2024-09-10T23:40:46.181323+0000","last_undegraded":"2024-09-10T23:40:46.181323+0000","last_fullsized":"2024-09-10T23:40:46.181323+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:43.913895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.f","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.090237+0000","last_change":"2024-09-10T23:40:44.080468+0000","last_active":"2024-09-10T23:40:46.090237+0000","last_peered":"2024-09-10T23:40:46.090237+0000","last_clean":"2024-09-10T23:40:46.090237+0000","last_became_active":"2024-09-10T23:40:44.080151+0000","last_became_peered":"2024-09-10T23:40:44.080151+0000","last_unstale":"2024-09-10T23:40:46.090237+0000","last_undegraded":"2024-09-10T23:40:46.090237+0000","last_fullsized":"2024-09-10T23:40:46.090237+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:11.343095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181401+0000","last_change":"2024-09-10T23:40:45.083171+0000","last_active":"2024-09-10T23:40:46.181401+0000","last_peered":"2024-09-10T23:40:46.181401+0000","last_clean":"2024-09-10T23:40:46.181401+0000","last_became_active":"2024-09-10T23:40:45.082895+0000","last_became_peered":"2024-09-10T23:40:45.082895+0000","last_unstale":"2024-09-10T23:40:46.181401+0000","last_undegraded":"2024-09-10T23:40:46.181401+0000","last_fullsized":"2024-09-10T23:40:46.181401+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:01.948292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.171906+0000","last_change":"2024-09-10T23:40:44.077485+0000","last_active":"2024-09-10T23:40:46.171906+0000","last_peered":"2024-09-10T23:40:46.171906+0000","last_clean":"2024-09-10T23:40:46.171906+0000","last_became_active":"2024-09-10T23:40:44.077214+0000","last_became_peered":"2024-09-10T23:40:44.077214+0000","last_unstale":"2024-09-10T23:40:46.171906+0000","last_undegraded":"2024-09-10T23:40:46.171906+0000","last_fullsized":"2024-09-10T23:40:46.171906+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:17.795252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182367+0000","last_change":"2024-09-10T23:40:45.093289+0000","last_active":"2024-09-10T23:40:46.182367+0000","last_peered":"2024-09-10T23:40:46.182367+0000","last_clean":"2024-09-10T23:40:46.182367+0000","last_became_active":"2024-09-10T23:40:45.091960+0000","last_became_peered":"2024-09-10T23:40:45.091960+0000","last_unstale":"2024-09-10T23:40:46.182367+0000","last_undegraded":"2024-09-10T23:40:46.182367+0000","last_fullsized":"2024-09-10T23:40:46.182367+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:08.135993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.d","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.088713+0000","last_change":"2024-09-10T23:40:44.074809+0000","last_active":"2024-09-10T23:40:46.088713+0000","last_peered":"2024-09-10T23:40:46.088713+0000","last_clean":"2024-09-10T23:40:46.088713+0000","last_became_active":"2024-09-10T23:40:44.074511+0000","last_became_peered":"2024-09-10T23:40:44.074511+0000","last_unstale":"2024-09-10T23:40:46.088713+0000","last_undegraded":"2024-09-10T23:40:46.088713+0000","last_fullsized":"2024-09-10T23:40:46.088713+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:24.979379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089908+0000","last_change":"2024-09-10T23:40:45.089732+0000","last_active":"2024-09-10T23:40:46.089908+0000","last_peered":"2024-09-10T23:40:46.089908+0000","last_clean":"2024-09-10T23:40:46.089908+0000","last_became_active":"2024-09-10T23:40:45.089250+0000","last_became_peered":"2024-09-10T23:40:45.089250+0000","last_unstale":"2024-09-10T23:40:46.089908+0000","last_undegraded":"2024-09-10T23:40:46.089908+0000","last_fullsized":"2024-09-10T23:40:46.089908+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:25.067281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089565+0000","last_change":"2024-09-10T23:40:44.080523+0000","last_active":"2024-09-10T23:40:46.089565+0000","last_peered":"2024-09-10T23:40:46.089565+0000","last_clean":"2024-09-10T23:40:46.089565+0000","last_became_active":"2024-09-10T23:40:44.078290+0000","last_became_peered":"2024-09-10T23:40:44.078290+0000","last_unstale":"2024-09-10T23:40:46.089565+0000","last_undegraded":"2024-09-10T23:40:46.089565+0000","last_fullsized":"2024-09-10T23:40:46.089565+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:55.916591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182073+0000","last_change":"2024-09-10T23:40:45.087475+0000","last_active":"2024-09-10T23:40:46.182073+0000","last_peered":"2024-09-10T23:40:46.182073+0000","last_clean":"2024-09-10T23:40:46.182073+0000","last_became_active":"2024-09-10T23:40:45.086602+0000","last_became_peered":"2024-09-10T23:40:45.086602+0000","last_unstale":"2024-09-10T23:40:46.182073+0000","last_undegraded":"2024-09-10T23:40:46.182073+0000","last_fullsized":"2024-09-10T23:40:46.182073+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:20.787848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.b","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.088336+0000","last_change":"2024-09-10T23:40:44.073522+0000","last_active":"2024-09-10T23:40:46.088336+0000","last_peered":"2024-09-10T23:40:46.088336+0000","last_clean":"2024-09-10T23:40:46.088336+0000","last_became_active":"2024-09-10T23:40:44.073222+0000","last_became_peered":"2024-09-10T23:40:44.073222+0000","last_unstale":"2024-09-10T23:40:46.088336+0000","last_undegraded":"2024-09-10T23:40:46.088336+0000","last_fullsized":"2024-09-10T23:40:46.088336+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:29.109728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.090072+0000","last_change":"2024-09-10T23:40:45.085432+0000","last_active":"2024-09-10T23:40:46.090072+0000","last_peered":"2024-09-10T23:40:46.090072+0000","last_clean":"2024-09-10T23:40:46.090072+0000","last_became_active":"2024-09-10T23:40:45.085070+0000","last_became_peered":"2024-09-10T23:40:45.085070+0000","last_unstale":"2024-09-10T23:40:46.090072+0000","last_undegraded":"2024-09-10T23:40:46.090072+0000","last_fullsized":"2024-09-10T23:40:46.090072+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:36.792518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181230+0000","last_change":"2024-09-10T23:40:44.079389+0000","last_active":"2024-09-10T23:40:46.181230+0000","last_peered":"2024-09-10T23:40:46.181230+0000","last_clean":"2024-09-10T23:40:46.181230+0000","last_became_active":"2024-09-10T23:40:44.079085+0000","last_became_peered":"2024-09-10T23:40:44.079085+0000","last_unstale":"2024-09-10T23:40:46.181230+0000","last_undegraded":"2024-09-10T23:40:46.181230+0000","last_fullsized":"2024-09-10T23:40:46.181230+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:35.782729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180074+0000","last_change":"2024-09-10T23:40:45.079781+0000","last_active":"2024-09-10T23:40:46.180074+0000","last_peered":"2024-09-10T23:40:46.180074+0000","last_clean":"2024-09-10T23:40:46.180074+0000","last_became_active":"2024-09-10T23:40:45.079516+0000","last_became_peered":"2024-09-10T23:40:45.079516+0000","last_unstale":"2024-09-10T23:40:46.180074+0000","last_undegraded":"2024-09-10T23:40:46.180074+0000","last_fullsized":"2024-09-10T23:40:46.180074+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:59.042257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181687+0000","last_change":"2024-09-10T23:40:44.082435+0000","last_active":"2024-09-10T23:40:46.181687+0000","last_peered":"2024-09-10T23:40:46.181687+0000","last_clean":"2024-09-10T23:40:46.181687+0000","last_became_active":"2024-09-10T23:40:44.082147+0000","last_became_peered":"2024-09-10T23:40:44.082147+0000","last_unstale":"2024-09-10T23:40:46.181687+0000","last_undegraded":"2024-09-10T23:40:46.181687+0000","last_fullsized":"2024-09-10T23:40:46.181687+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:12.202400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180533+0000","last_change":"2024-09-10T23:40:45.090690+0000","last_active":"2024-09-10T23:40:46.180533+0000","last_peered":"2024-09-10T23:40:46.180533+0000","last_clean":"2024-09-10T23:40:46.180533+0000","last_became_active":"2024-09-10T23:40:45.090276+0000","last_became_peered":"2024-09-10T23:40:45.090276+0000","last_unstale":"2024-09-10T23:40:46.180533+0000","last_undegraded":"2024-09-10T23:40:46.180533+0000","last_fullsized":"2024-09-10T23:40:46.180533+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:09.368470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181114+0000","last_change":"2024-09-10T23:40:44.082739+0000","last_active":"2024-09-10T23:40:46.181114+0000","last_peered":"2024-09-10T23:40:46.181114+0000","last_clean":"2024-09-10T23:40:46.181114+0000","last_became_active":"2024-09-10T23:40:44.080910+0000","last_became_peered":"2024-09-10T23:40:44.080910+0000","last_unstale":"2024-09-10T23:40:46.181114+0000","last_undegraded":"2024-09-10T23:40:46.181114+0000","last_fullsized":"2024-09-10T23:40:46.181114+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:48.655919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181414+0000","last_change":"2024-09-10T23:40:45.095933+0000","last_active":"2024-09-10T23:40:46.181414+0000","last_peered":"2024-09-10T23:40:46.181414+0000","last_clean":"2024-09-10T23:40:46.181414+0000","last_became_active":"2024-09-10T23:40:45.094366+0000","last_became_peered":"2024-09-10T23:40:45.094366+0000","last_unstale":"2024-09-10T23:40:46.181414+0000","last_undegraded":"2024-09-10T23:40:46.181414+0000","last_fullsized":"2024-09-10T23:40:46.181414+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:40.636817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.7","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181518+0000","last_change":"2024-09-10T23:40:44.079487+0000","last_active":"2024-09-10T23:40:46.181518+0000","last_peered":"2024-09-10T23:40:46.181518+0000","last_clean":"2024-09-10T23:40:46.181518+0000","last_became_active":"2024-09-10T23:40:44.075501+0000","last_became_peered":"2024-09-10T23:40:44.075501+0000","last_unstale":"2024-09-10T23:40:46.181518+0000","last_undegraded":"2024-09-10T23:40:46.181518+0000","last_fullsized":"2024-09-10T23:40:46.181518+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:26.051116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.090075+0000","last_change":"2024-09-10T23:40:45.090499+0000","last_active":"2024-09-10T23:40:46.090075+0000","last_peered":"2024-09-10T23:40:46.090075+0000","last_clean":"2024-09-10T23:40:46.090075+0000","last_became_active":"2024-09-10T23:40:45.090112+0000","last_became_peered":"2024-09-10T23:40:45.090112+0000","last_unstale":"2024-09-10T23:40:46.090075+0000","last_undegraded":"2024-09-10T23:40:46.090075+0000","last_fullsized":"2024-09-10T23:40:46.090075+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:49.451778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.6","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.083324+0000","last_change":"2024-09-10T23:40:44.076657+0000","last_active":"2024-09-10T23:40:46.083324+0000","last_peered":"2024-09-10T23:40:46.083324+0000","last_clean":"2024-09-10T23:40:46.083324+0000","last_became_active":"2024-09-10T23:40:44.076402+0000","last_became_peered":"2024-09-10T23:40:44.076402+0000","last_unstale":"2024-09-10T23:40:46.083324+0000","last_undegraded":"2024-09-10T23:40:46.083324+0000","last_fullsized":"2024-09-10T23:40:46.083324+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:42.457163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.083120+0000","last_change":"2024-09-10T23:40:45.075558+0000","last_active":"2024-09-10T23:40:46.083120+0000","last_peered":"2024-09-10T23:40:46.083120+0000","last_clean":"2024-09-10T23:40:46.083120+0000","last_became_active":"2024-09-10T23:40:45.074348+0000","last_became_peered":"2024-09-10T23:40:45.074348+0000","last_unstale":"2024-09-10T23:40:46.083120+0000","last_undegraded":"2024-09-10T23:40:46.083120+0000","last_fullsized":"2024-09-10T23:40:46.083120+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:55.908829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.090012+0000","last_change":"2024-09-10T23:40:44.077621+0000","last_active":"2024-09-10T23:40:46.090012+0000","last_peered":"2024-09-10T23:40:46.090012+0000","last_clean":"2024-09-10T23:40:46.090012+0000","last_became_active":"2024-09-10T23:40:44.077160+0000","last_became_peered":"2024-09-10T23:40:44.077160+0000","last_unstale":"2024-09-10T23:40:46.090012+0000","last_undegraded":"2024-09-10T23:40:46.090012+0000","last_fullsized":"2024-09-10T23:40:46.090012+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:00.848681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180980+0000","last_change":"2024-09-10T23:40:45.095623+0000","last_active":"2024-09-10T23:40:46.180980+0000","last_peered":"2024-09-10T23:40:46.180980+0000","last_clean":"2024-09-10T23:40:46.180980+0000","last_became_active":"2024-09-10T23:40:45.094804+0000","last_became_peered":"2024-09-10T23:40:45.094804+0000","last_unstale":"2024-09-10T23:40:46.180980+0000","last_undegraded":"2024-09-10T23:40:46.180980+0000","last_fullsized":"2024-09-10T23:40:46.180980+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:50.195796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089522+0000","last_change":"2024-09-10T23:40:44.081193+0000","last_active":"2024-09-10T23:40:46.089522+0000","last_peered":"2024-09-10T23:40:46.089522+0000","last_clean":"2024-09-10T23:40:46.089522+0000","last_became_active":"2024-09-10T23:40:44.080941+0000","last_became_peered":"2024-09-10T23:40:44.080941+0000","last_unstale":"2024-09-10T23:40:46.089522+0000","last_undegraded":"2024-09-10T23:40:46.089522+0000","last_fullsized":"2024-09-10T23:40:46.089522+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:14.356564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089589+0000","last_change":"2024-09-10T23:40:45.086167+0000","last_active":"2024-09-10T23:40:46.089589+0000","last_peered":"2024-09-10T23:40:46.089589+0000","last_clean":"2024-09-10T23:40:46.089589+0000","last_became_active":"2024-09-10T23:40:45.085618+0000","last_became_peered":"2024-09-10T23:40:45.085618+0000","last_unstale":"2024-09-10T23:40:46.089589+0000","last_undegraded":"2024-09-10T23:40:46.089589+0000","last_fullsized":"2024-09-10T23:40:46.089589+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:35.179057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180725+0000","last_change":"2024-09-10T23:40:44.079412+0000","last_active":"2024-09-10T23:40:46.180725+0000","last_peered":"2024-09-10T23:40:46.180725+0000","last_clean":"2024-09-10T23:40:46.180725+0000","last_became_active":"2024-09-10T23:40:44.079111+0000","last_became_peered":"2024-09-10T23:40:44.079111+0000","last_unstale":"2024-09-10T23:40:46.180725+0000","last_undegraded":"2024-09-10T23:40:46.180725+0000","last_fullsized":"2024-09-10T23:40:46.180725+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:31.442614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.083177+0000","last_change":"2024-09-10T23:40:45.081514+0000","last_active":"2024-09-10T23:40:46.083177+0000","last_peered":"2024-09-10T23:40:46.083177+0000","last_clean":"2024-09-10T23:40:46.083177+0000","last_became_active":"2024-09-10T23:40:45.081246+0000","last_became_peered":"2024-09-10T23:40:45.081246+0000","last_unstale":"2024-09-10T23:40:46.083177+0000","last_undegraded":"2024-09-10T23:40:46.083177+0000","last_fullsized":"2024-09-10T23:40:46.083177+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:21.423350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182115+0000","last_change":"2024-09-10T23:40:44.081308+0000","last_active":"2024-09-10T23:40:46.182115+0000","last_peered":"2024-09-10T23:40:46.182115+0000","last_clean":"2024-09-10T23:40:46.182115+0000","last_became_active":"2024-09-10T23:40:44.080898+0000","last_became_peered":"2024-09-10T23:40:44.080898+0000","last_unstale":"2024-09-10T23:40:46.182115+0000","last_undegraded":"2024-09-10T23:40:46.182115+0000","last_fullsized":"2024-09-10T23:40:46.182115+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:00.938502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180827+0000","last_change":"2024-09-10T23:40:45.076548+0000","last_active":"2024-09-10T23:40:46.180827+0000","last_peered":"2024-09-10T23:40:46.180827+0000","last_clean":"2024-09-10T23:40:46.180827+0000","last_became_active":"2024-09-10T23:40:45.076264+0000","last_became_peered":"2024-09-10T23:40:45.076264+0000","last_unstale":"2024-09-10T23:40:46.180827+0000","last_undegraded":"2024-09-10T23:40:46.180827+0000","last_fullsized":"2024-09-10T23:40:46.180827+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:10.377446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180246+0000","last_change":"2024-09-10T23:40:44.074775+0000","last_active":"2024-09-10T23:40:46.180246+0000","last_peered":"2024-09-10T23:40:46.180246+0000","last_clean":"2024-09-10T23:40:46.180246+0000","last_became_active":"2024-09-10T23:40:44.074206+0000","last_became_peered":"2024-09-10T23:40:44.074206+0000","last_unstale":"2024-09-10T23:40:46.180246+0000","last_undegraded":"2024-09-10T23:40:46.180246+0000","last_fullsized":"2024-09-10T23:40:46.180246+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:07.553007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180730+0000","last_change":"2024-09-10T23:40:45.090693+0000","last_active":"2024-09-10T23:40:46.180730+0000","last_peered":"2024-09-10T23:40:46.180730+0000","last_clean":"2024-09-10T23:40:46.180730+0000","last_became_active":"2024-09-10T23:40:45.090206+0000","last_became_peered":"2024-09-10T23:40:45.090206+0000","last_unstale":"2024-09-10T23:40:46.180730+0000","last_undegraded":"2024-09-10T23:40:46.180730+0000","last_fullsized":"2024-09-10T23:40:46.180730+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:11:08.283129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180283+0000","last_change":"2024-09-10T23:40:44.080507+0000","last_active":"2024-09-10T23:40:46.180283+0000","last_peered":"2024-09-10T23:40:46.180283+0000","last_clean":"2024-09-10T23:40:46.180283+0000","last_became_active":"2024-09-10T23:40:44.080069+0000","last_became_peered":"2024-09-10T23:40:44.080069+0000","last_unstale":"2024-09-10T23:40:46.180283+0000","last_undegraded":"2024-09-10T23:40:46.180283+0000","last_fullsized":"2024-09-10T23:40:46.180283+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:58.973485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181897+0000","last_change":"2024-09-10T23:40:45.091633+0000","last_active":"2024-09-10T23:40:46.181897+0000","last_peered":"2024-09-10T23:40:46.181897+0000","last_clean":"2024-09-10T23:40:46.181897+0000","last_became_active":"2024-09-10T23:40:45.091354+0000","last_became_peered":"2024-09-10T23:40:45.091354+0000","last_unstale":"2024-09-10T23:40:46.181897+0000","last_undegraded":"2024-09-10T23:40:46.181897+0000","last_fullsized":"2024-09-10T23:40:46.181897+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:21.659769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.083305+0000","last_change":"2024-09-10T23:40:44.074854+0000","last_active":"2024-09-10T23:40:46.083305+0000","last_peered":"2024-09-10T23:40:46.083305+0000","last_clean":"2024-09-10T23:40:46.083305+0000","last_became_active":"2024-09-10T23:40:44.074588+0000","last_became_peered":"2024-09-10T23:40:44.074588+0000","last_unstale":"2024-09-10T23:40:46.083305+0000","last_undegraded":"2024-09-10T23:40:46.083305+0000","last_fullsized":"2024-09-10T23:40:46.083305+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:45.851870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":396,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180368+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:40:46.180368+0000","last_peered":"2024-09-10T23:40:46.180368+0000","last_clean":"2024-09-10T23:40:46.180368+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:40:46.180368+0000","last_undegraded":"2024-09-10T23:40:46.180368+0000","last_fullsized":"2024-09-10T23:40:46.180368+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509545,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2043904,"data_stored":899938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542248,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2043904,"data_stored":899986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509544,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33780,"kb_used_data":2588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338060,"statfs":{"total":96636764160,"available":96602173440,"internally_reserved":0,"allocated":2650112,"data_stored":1491012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33192,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338648,"statfs":{"total":96636764160,"available":96602775552,"internally_reserved":0,"allocated":2043904,"data_stored":900006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509543,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37352,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334488,"statfs":{"total":96636764160,"available":96598515712,"internally_reserved":0,"allocated":2052096,"data_stored":901528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509544,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37360,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334480,"statfs":{"total":96636764160,"available":96598507520,"internally_reserved":0,"allocated":2060288,"data_stored":901632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509543,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37924,"kb_used_data":2572,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333916,"statfs":{"total":96636764160,"available":96597929984,"internally_reserved":0,"allocated":2633728,"data_stored":1490284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509543,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37344,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334496,"statfs":{"total":96636764160,"available":96598523904,"internally_reserved":0,"allocated":2043904,"data_stored":900498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:40:49.190 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:40:49.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.375+0000 7f548de00640 1 -- 172.21.15.94:0/669843621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f546c004810 msgr2=0x7f546c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:49.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.375+0000 7f548de00640 1 --2- 172.21.15.94:0/669843621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f546c004810 0x7f546c004c90 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f548809b7d0 tx=0x7f547401f660 comp rx=0 tx=0).stop 2024-09-10T23:40:49.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.375+0000 7f548de00640 1 -- 172.21.15.94:0/669843621 shutdown_connections 2024-09-10T23:40:49.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.375+0000 7f548de00640 1 --2- 172.21.15.94:0/669843621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f546c004810 0x7f546c004c90 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:49.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.375+0000 7f548de00640 1 -- 172.21.15.94:0/669843621 >> 172.21.15.94:0/669843621 conn(0x7f548809aa70 msgr2=0x7f548809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:49.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.375+0000 7f548de00640 1 -- 172.21.15.94:0/669843621 shutdown_connections 2024-09-10T23:40:49.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.375+0000 7f548de00640 1 -- 172.21.15.94:0/669843621 wait complete. 2024-09-10T23:40:49.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.375+0000 7f548de00640 1 Processor -- start 2024-09-10T23:40:49.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.375+0000 7f548de00640 1 -- start start 2024-09-10T23:40:49.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.375+0000 7f548de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f546c004810 0x7f54880b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:49.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.375+0000 7f548de00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54881f5990 0x7f54881f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:49.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.375+0000 7f548de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5488071590 0x7f5488071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:49.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.375+0000 7f548de00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f54880d1d00 con 0x7f5488071590 2024-09-10T23:40:49.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.375+0000 7f548de00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f54880d1b80 con 0x7f546c004810 2024-09-10T23:40:49.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.375+0000 7f548de00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f54880d1e80 con 0x7f54881f5990 2024-09-10T23:40:49.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f5486a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f546c004810 0x7f54880b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:49.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f5487400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5488071590 0x7f5488071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:49.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f5486000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54881f5990 0x7f54881f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:49.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f5486a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f546c004810 0x7f54880b7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:45148/0 (socket says 172.21.15.94:45148) 2024-09-10T23:40:49.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f5486a00640 1 -- 172.21.15.94:0/570253184 learned_addr learned my addr 172.21.15.94:0/570253184 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:49.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f5487400640 1 -- 172.21.15.94:0/570253184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54881f5990 msgr2=0x7f54881f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:49.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f5487400640 1 --2- 172.21.15.94:0/570253184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54881f5990 0x7f54881f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:49.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f5487400640 1 -- 172.21.15.94:0/570253184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f546c004810 msgr2=0x7f54880b7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:49.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f5487400640 1 --2- 172.21.15.94:0/570253184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f546c004810 0x7f54880b7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:49.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f5487400640 1 -- 172.21.15.94:0/570253184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54880b7920 con 0x7f5488071590 2024-09-10T23:40:49.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f5487400640 1 --2- 172.21.15.94:0/570253184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5488071590 0x7f5488071a10 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f54880744b0 tx=0x7f5480010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:49.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f547f400640 1 -- 172.21.15.94:0/570253184 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548000ebd0 con 0x7f5488071590 2024-09-10T23:40:49.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f547f400640 1 -- 172.21.15.94:0/570253184 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f548000ee60 con 0x7f5488071590 2024-09-10T23:40:49.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f547f400640 1 -- 172.21.15.94:0/570253184 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54800126b0 con 0x7f5488071590 2024-09-10T23:40:49.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f548de00640 1 -- 172.21.15.94:0/570253184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54881f9390 con 0x7f5488071590 2024-09-10T23:40:49.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f548de00640 1 -- 172.21.15.94:0/570253184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54881f88b0 con 0x7f5488071590 2024-09-10T23:40:49.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f548de00640 1 -- 172.21.15.94:0/570253184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f548815c1f0 con 0x7f5488071590 2024-09-10T23:40:49.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.379+0000 7f547f400640 1 -- 172.21.15.94:0/570253184 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f548000f040 con 0x7f5488071590 2024-09-10T23:40:49.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.383+0000 7f547f400640 1 --2- 172.21.15.94:0/570253184 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5468073030 0x7f54680754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:49.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.383+0000 7f547f400640 1 -- 172.21.15.94:0/570253184 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f5480092390 con 0x7f5488071590 2024-09-10T23:40:49.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.383+0000 7f5486a00640 1 --2- 172.21.15.94:0/570253184 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5468073030 0x7f54680754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:49.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.383+0000 7f5486a00640 1 --2- 172.21.15.94:0/570253184 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5468073030 0x7f54680754f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5478006670 tx=0x7f5478006dc0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:49.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.387+0000 7f547f400640 1 -- 172.21.15.94:0/570253184 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5480023030 con 0x7f5488071590 2024-09-10T23:40:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.547+0000 7f548de00640 1 -- 172.21.15.94:0/570253184 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f54880d0710 con 0x7f5468073030 2024-09-10T23:40:49.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.551+0000 7f547f400640 1 -- 172.21.15.94:0/570253184 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7f54880d0710 con 0x7f5468073030 2024-09-10T23:40:49.553 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:49.555 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:40:49.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.555+0000 7f548de00640 1 -- 172.21.15.94:0/570253184 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5468073030 msgr2=0x7f54680754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:49.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.555+0000 7f548de00640 1 --2- 172.21.15.94:0/570253184 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5468073030 0x7f54680754f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5478006670 tx=0x7f5478006dc0 comp rx=0 tx=0).stop 2024-09-10T23:40:49.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.555+0000 7f548de00640 1 -- 172.21.15.94:0/570253184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5488071590 msgr2=0x7f5488071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:49.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.555+0000 7f548de00640 1 --2- 172.21.15.94:0/570253184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5488071590 0x7f5488071a10 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f54880744b0 tx=0x7f5480010da0 comp rx=0 tx=0).stop 2024-09-10T23:40:49.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.555+0000 7f548de00640 1 -- 172.21.15.94:0/570253184 shutdown_connections 2024-09-10T23:40:49.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.555+0000 7f548de00640 1 --2- 172.21.15.94:0/570253184 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5468073030 0x7f54680754f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:49.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.555+0000 7f548de00640 1 --2- 172.21.15.94:0/570253184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5488071590 0x7f5488071a10 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:49.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.555+0000 7f548de00640 1 --2- 172.21.15.94:0/570253184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54881f5990 0x7f54881f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:49.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.555+0000 7f548de00640 1 --2- 172.21.15.94:0/570253184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f546c004810 0x7f54880b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:49.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.555+0000 7f548de00640 1 -- 172.21.15.94:0/570253184 >> 172.21.15.94:0/570253184 conn(0x7f548809aa70 msgr2=0x7f54880afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:49.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.555+0000 7f548de00640 1 -- 172.21.15.94:0/570253184 shutdown_connections 2024-09-10T23:40:49.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.555+0000 7f548de00640 1 -- 172.21.15.94:0/570253184 wait complete. 2024-09-10T23:40:49.576 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-10T23:40:48.787864+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283324,"kb_used_data":17160,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754691396,"statfs":{"total":773094113280,"available":772803989504,"internally_reserved":0,"allocated":17571840,"data_stored":8384884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.737031"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089786+0000","last_change":"2024-09-10T23:40:45.089655+0000","last_active":"2024-09-10T23:40:46.089786+0000","last_peered":"2024-09-10T23:40:46.089786+0000","last_clean":"2024-09-10T23:40:46.089786+0000","last_became_active":"2024-09-10T23:40:45.089311+0000","last_became_peered":"2024-09-10T23:40:45.089311+0000","last_unstale":"2024-09-10T23:40:46.089786+0000","last_undegraded":"2024-09-10T23:40:46.089786+0000","last_fullsized":"2024-09-10T23:40:46.089786+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:10.781519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.083167+0000","last_change":"2024-09-10T23:40:45.091720+0000","last_active":"2024-09-10T23:40:46.083167+0000","last_peered":"2024-09-10T23:40:46.083167+0000","last_clean":"2024-09-10T23:40:46.083167+0000","last_became_active":"2024-09-10T23:40:45.091354+0000","last_became_peered":"2024-09-10T23:40:45.091354+0000","last_unstale":"2024-09-10T23:40:46.083167+0000","last_undegraded":"2024-09-10T23:40:46.083167+0000","last_fullsized":"2024-09-10T23:40:46.083167+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:07.822794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181933+0000","last_change":"2024-09-10T23:40:45.084475+0000","last_active":"2024-09-10T23:40:46.181933+0000","last_peered":"2024-09-10T23:40:46.181933+0000","last_clean":"2024-09-10T23:40:46.181933+0000","last_became_active":"2024-09-10T23:40:45.084116+0000","last_became_peered":"2024-09-10T23:40:45.084116+0000","last_unstale":"2024-09-10T23:40:46.181933+0000","last_undegraded":"2024-09-10T23:40:46.181933+0000","last_fullsized":"2024-09-10T23:40:46.181933+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:59.689357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181043+0000","last_change":"2024-09-10T23:40:45.080097+0000","last_active":"2024-09-10T23:40:46.181043+0000","last_peered":"2024-09-10T23:40:46.181043+0000","last_clean":"2024-09-10T23:40:46.181043+0000","last_became_active":"2024-09-10T23:40:45.079590+0000","last_became_peered":"2024-09-10T23:40:45.079590+0000","last_unstale":"2024-09-10T23:40:46.181043+0000","last_undegraded":"2024-09-10T23:40:46.181043+0000","last_fullsized":"2024-09-10T23:40:46.181043+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:49.599844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182232+0000","last_change":"2024-09-10T23:40:45.092375+0000","last_active":"2024-09-10T23:40:46.182232+0000","last_peered":"2024-09-10T23:40:46.182232+0000","last_clean":"2024-09-10T23:40:46.182232+0000","last_became_active":"2024-09-10T23:40:45.092080+0000","last_became_peered":"2024-09-10T23:40:45.092080+0000","last_unstale":"2024-09-10T23:40:46.182232+0000","last_undegraded":"2024-09-10T23:40:46.182232+0000","last_fullsized":"2024-09-10T23:40:46.182232+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:24.625098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089480+0000","last_change":"2024-09-10T23:40:45.086618+0000","last_active":"2024-09-10T23:40:46.089480+0000","last_peered":"2024-09-10T23:40:46.089480+0000","last_clean":"2024-09-10T23:40:46.089480+0000","last_became_active":"2024-09-10T23:40:45.086363+0000","last_became_peered":"2024-09-10T23:40:45.086363+0000","last_unstale":"2024-09-10T23:40:46.089480+0000","last_undegraded":"2024-09-10T23:40:46.089480+0000","last_fullsized":"2024-09-10T23:40:46.089480+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:58.771287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089252+0000","last_change":"2024-09-10T23:40:45.087904+0000","last_active":"2024-09-10T23:40:46.089252+0000","last_peered":"2024-09-10T23:40:46.089252+0000","last_clean":"2024-09-10T23:40:46.089252+0000","last_became_active":"2024-09-10T23:40:45.087629+0000","last_became_peered":"2024-09-10T23:40:45.087629+0000","last_unstale":"2024-09-10T23:40:46.089252+0000","last_undegraded":"2024-09-10T23:40:46.089252+0000","last_fullsized":"2024-09-10T23:40:46.089252+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:27.475616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180775+0000","last_change":"2024-09-10T23:40:45.078731+0000","last_active":"2024-09-10T23:40:46.180775+0000","last_peered":"2024-09-10T23:40:46.180775+0000","last_clean":"2024-09-10T23:40:46.180775+0000","last_became_active":"2024-09-10T23:40:45.078285+0000","last_became_peered":"2024-09-10T23:40:45.078285+0000","last_unstale":"2024-09-10T23:40:46.180775+0000","last_undegraded":"2024-09-10T23:40:46.180775+0000","last_fullsized":"2024-09-10T23:40:46.180775+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:43.466953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.172015+0000","last_change":"2024-09-10T23:40:45.087453+0000","last_active":"2024-09-10T23:40:46.172015+0000","last_peered":"2024-09-10T23:40:46.172015+0000","last_clean":"2024-09-10T23:40:46.172015+0000","last_became_active":"2024-09-10T23:40:45.087096+0000","last_became_peered":"2024-09-10T23:40:45.087096+0000","last_unstale":"2024-09-10T23:40:46.172015+0000","last_undegraded":"2024-09-10T23:40:46.172015+0000","last_fullsized":"2024-09-10T23:40:46.172015+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:16.873022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182339+0000","last_change":"2024-09-10T23:40:45.085815+0000","last_active":"2024-09-10T23:40:46.182339+0000","last_peered":"2024-09-10T23:40:46.182339+0000","last_clean":"2024-09-10T23:40:46.182339+0000","last_became_active":"2024-09-10T23:40:45.083284+0000","last_became_peered":"2024-09-10T23:40:45.083284+0000","last_unstale":"2024-09-10T23:40:46.182339+0000","last_undegraded":"2024-09-10T23:40:46.182339+0000","last_fullsized":"2024-09-10T23:40:46.182339+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:47:35.838329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089891+0000","last_change":"2024-09-10T23:40:45.087685+0000","last_active":"2024-09-10T23:40:46.089891+0000","last_peered":"2024-09-10T23:40:46.089891+0000","last_clean":"2024-09-10T23:40:46.089891+0000","last_became_active":"2024-09-10T23:40:45.085525+0000","last_became_peered":"2024-09-10T23:40:45.085525+0000","last_unstale":"2024-09-10T23:40:46.089891+0000","last_undegraded":"2024-09-10T23:40:46.089891+0000","last_fullsized":"2024-09-10T23:40:46.089891+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:12.553879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180708+0000","last_change":"2024-09-10T23:40:45.070594+0000","last_active":"2024-09-10T23:40:46.180708+0000","last_peered":"2024-09-10T23:40:46.180708+0000","last_clean":"2024-09-10T23:40:46.180708+0000","last_became_active":"2024-09-10T23:40:45.070051+0000","last_became_peered":"2024-09-10T23:40:45.070051+0000","last_unstale":"2024-09-10T23:40:46.180708+0000","last_undegraded":"2024-09-10T23:40:46.180708+0000","last_fullsized":"2024-09-10T23:40:46.180708+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:41.146987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180567+0000","last_change":"2024-09-10T23:40:45.085757+0000","last_active":"2024-09-10T23:40:46.180567+0000","last_peered":"2024-09-10T23:40:46.180567+0000","last_clean":"2024-09-10T23:40:46.180567+0000","last_became_active":"2024-09-10T23:40:45.085484+0000","last_became_peered":"2024-09-10T23:40:45.085484+0000","last_unstale":"2024-09-10T23:40:46.180567+0000","last_undegraded":"2024-09-10T23:40:46.180567+0000","last_fullsized":"2024-09-10T23:40:46.180567+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:57.777941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.088350+0000","last_change":"2024-09-10T23:40:45.078342+0000","last_active":"2024-09-10T23:40:46.088350+0000","last_peered":"2024-09-10T23:40:46.088350+0000","last_clean":"2024-09-10T23:40:46.088350+0000","last_became_active":"2024-09-10T23:40:45.078067+0000","last_became_peered":"2024-09-10T23:40:45.078067+0000","last_unstale":"2024-09-10T23:40:46.088350+0000","last_undegraded":"2024-09-10T23:40:46.088350+0000","last_fullsized":"2024-09-10T23:40:46.088350+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:55.912272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089694+0000","last_change":"2024-09-10T23:40:45.084613+0000","last_active":"2024-09-10T23:40:46.089694+0000","last_peered":"2024-09-10T23:40:46.089694+0000","last_clean":"2024-09-10T23:40:46.089694+0000","last_became_active":"2024-09-10T23:40:45.084308+0000","last_became_peered":"2024-09-10T23:40:45.084308+0000","last_unstale":"2024-09-10T23:40:46.089694+0000","last_undegraded":"2024-09-10T23:40:46.089694+0000","last_fullsized":"2024-09-10T23:40:46.089694+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:43.615447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089103+0000","last_change":"2024-09-10T23:40:45.090696+0000","last_active":"2024-09-10T23:40:46.089103+0000","last_peered":"2024-09-10T23:40:46.089103+0000","last_clean":"2024-09-10T23:40:46.089103+0000","last_became_active":"2024-09-10T23:40:45.090380+0000","last_became_peered":"2024-09-10T23:40:45.090380+0000","last_unstale":"2024-09-10T23:40:46.089103+0000","last_undegraded":"2024-09-10T23:40:46.089103+0000","last_fullsized":"2024-09-10T23:40:46.089103+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:56.322876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.090060+0000","last_change":"2024-09-10T23:40:45.103193+0000","last_active":"2024-09-10T23:40:46.090060+0000","last_peered":"2024-09-10T23:40:46.090060+0000","last_clean":"2024-09-10T23:40:46.090060+0000","last_became_active":"2024-09-10T23:40:45.102814+0000","last_became_peered":"2024-09-10T23:40:45.102814+0000","last_unstale":"2024-09-10T23:40:46.090060+0000","last_undegraded":"2024-09-10T23:40:46.090060+0000","last_fullsized":"2024-09-10T23:40:46.090060+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:24.718349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182531+0000","last_change":"2024-09-10T23:40:45.092610+0000","last_active":"2024-09-10T23:40:46.182531+0000","last_peered":"2024-09-10T23:40:46.182531+0000","last_clean":"2024-09-10T23:40:46.182531+0000","last_became_active":"2024-09-10T23:40:45.092353+0000","last_became_peered":"2024-09-10T23:40:45.092353+0000","last_unstale":"2024-09-10T23:40:46.182531+0000","last_undegraded":"2024-09-10T23:40:46.182531+0000","last_fullsized":"2024-09-10T23:40:46.182531+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:49.760581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182333+0000","last_change":"2024-09-10T23:40:45.085629+0000","last_active":"2024-09-10T23:40:46.182333+0000","last_peered":"2024-09-10T23:40:46.182333+0000","last_clean":"2024-09-10T23:40:46.182333+0000","last_became_active":"2024-09-10T23:40:45.085359+0000","last_became_peered":"2024-09-10T23:40:45.085359+0000","last_unstale":"2024-09-10T23:40:46.182333+0000","last_undegraded":"2024-09-10T23:40:46.182333+0000","last_fullsized":"2024-09-10T23:40:46.182333+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:37:04.831815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181944+0000","last_change":"2024-09-10T23:40:45.088469+0000","last_active":"2024-09-10T23:40:46.181944+0000","last_peered":"2024-09-10T23:40:46.181944+0000","last_clean":"2024-09-10T23:40:46.181944+0000","last_became_active":"2024-09-10T23:40:45.086877+0000","last_became_peered":"2024-09-10T23:40:45.086877+0000","last_unstale":"2024-09-10T23:40:46.181944+0000","last_undegraded":"2024-09-10T23:40:46.181944+0000","last_fullsized":"2024-09-10T23:40:46.181944+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:49.527141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180434+0000","last_change":"2024-09-10T23:40:45.074233+0000","last_active":"2024-09-10T23:40:46.180434+0000","last_peered":"2024-09-10T23:40:46.180434+0000","last_clean":"2024-09-10T23:40:46.180434+0000","last_became_active":"2024-09-10T23:40:45.073965+0000","last_became_peered":"2024-09-10T23:40:45.073965+0000","last_unstale":"2024-09-10T23:40:46.180434+0000","last_undegraded":"2024-09-10T23:40:46.180434+0000","last_fullsized":"2024-09-10T23:40:46.180434+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:50.111171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.083331+0000","last_change":"2024-09-10T23:40:45.076285+0000","last_active":"2024-09-10T23:40:46.083331+0000","last_peered":"2024-09-10T23:40:46.083331+0000","last_clean":"2024-09-10T23:40:46.083331+0000","last_became_active":"2024-09-10T23:40:45.076015+0000","last_became_peered":"2024-09-10T23:40:45.076015+0000","last_unstale":"2024-09-10T23:40:46.083331+0000","last_undegraded":"2024-09-10T23:40:46.083331+0000","last_fullsized":"2024-09-10T23:40:46.083331+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:29.781657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180601+0000","last_change":"2024-09-10T23:40:45.080722+0000","last_active":"2024-09-10T23:40:46.180601+0000","last_peered":"2024-09-10T23:40:46.180601+0000","last_clean":"2024-09-10T23:40:46.180601+0000","last_became_active":"2024-09-10T23:40:45.078520+0000","last_became_peered":"2024-09-10T23:40:45.078520+0000","last_unstale":"2024-09-10T23:40:46.180601+0000","last_undegraded":"2024-09-10T23:40:46.180601+0000","last_fullsized":"2024-09-10T23:40:46.180601+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:49.894589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.171848+0000","last_change":"2024-09-10T23:40:45.079335+0000","last_active":"2024-09-10T23:40:46.171848+0000","last_peered":"2024-09-10T23:40:46.171848+0000","last_clean":"2024-09-10T23:40:46.171848+0000","last_became_active":"2024-09-10T23:40:45.078994+0000","last_became_peered":"2024-09-10T23:40:45.078994+0000","last_unstale":"2024-09-10T23:40:46.171848+0000","last_undegraded":"2024-09-10T23:40:46.171848+0000","last_fullsized":"2024-09-10T23:40:46.171848+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:52.866792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181289+0000","last_change":"2024-09-10T23:40:45.091347+0000","last_active":"2024-09-10T23:40:46.181289+0000","last_peered":"2024-09-10T23:40:46.181289+0000","last_clean":"2024-09-10T23:40:46.181289+0000","last_became_active":"2024-09-10T23:40:45.091053+0000","last_became_peered":"2024-09-10T23:40:45.091053+0000","last_unstale":"2024-09-10T23:40:46.181289+0000","last_undegraded":"2024-09-10T23:40:46.181289+0000","last_fullsized":"2024-09-10T23:40:46.181289+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:04:47.979677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181620+0000","last_change":"2024-09-10T23:40:45.074492+0000","last_active":"2024-09-10T23:40:46.181620+0000","last_peered":"2024-09-10T23:40:46.181620+0000","last_clean":"2024-09-10T23:40:46.181620+0000","last_became_active":"2024-09-10T23:40:45.074126+0000","last_became_peered":"2024-09-10T23:40:45.074126+0000","last_unstale":"2024-09-10T23:40:46.181620+0000","last_undegraded":"2024-09-10T23:40:46.181620+0000","last_fullsized":"2024-09-10T23:40:46.181620+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:28.996896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180964+0000","last_change":"2024-09-10T23:40:45.088346+0000","last_active":"2024-09-10T23:40:46.180964+0000","last_peered":"2024-09-10T23:40:46.180964+0000","last_clean":"2024-09-10T23:40:46.180964+0000","last_became_active":"2024-09-10T23:40:45.088100+0000","last_became_peered":"2024-09-10T23:40:45.088100+0000","last_unstale":"2024-09-10T23:40:46.180964+0000","last_undegraded":"2024-09-10T23:40:46.180964+0000","last_fullsized":"2024-09-10T23:40:46.180964+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:00.773536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181164+0000","last_change":"2024-09-10T23:40:45.081358+0000","last_active":"2024-09-10T23:40:46.181164+0000","last_peered":"2024-09-10T23:40:46.181164+0000","last_clean":"2024-09-10T23:40:46.181164+0000","last_became_active":"2024-09-10T23:40:45.079006+0000","last_became_peered":"2024-09-10T23:40:45.079006+0000","last_unstale":"2024-09-10T23:40:46.181164+0000","last_undegraded":"2024-09-10T23:40:46.181164+0000","last_fullsized":"2024-09-10T23:40:46.181164+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:40.336638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.082960+0000","last_change":"2024-09-10T23:40:45.077105+0000","last_active":"2024-09-10T23:40:46.082960+0000","last_peered":"2024-09-10T23:40:46.082960+0000","last_clean":"2024-09-10T23:40:46.082960+0000","last_became_active":"2024-09-10T23:40:45.076711+0000","last_became_peered":"2024-09-10T23:40:45.076711+0000","last_unstale":"2024-09-10T23:40:46.082960+0000","last_undegraded":"2024-09-10T23:40:46.082960+0000","last_fullsized":"2024-09-10T23:40:46.082960+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.061601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180576+0000","last_change":"2024-09-10T23:40:45.097534+0000","last_active":"2024-09-10T23:40:46.180576+0000","last_peered":"2024-09-10T23:40:46.180576+0000","last_clean":"2024-09-10T23:40:46.180576+0000","last_became_active":"2024-09-10T23:40:45.097278+0000","last_became_peered":"2024-09-10T23:40:45.097278+0000","last_unstale":"2024-09-10T23:40:46.180576+0000","last_undegraded":"2024-09-10T23:40:46.180576+0000","last_fullsized":"2024-09-10T23:40:46.180576+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:19.477684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181744+0000","last_change":"2024-09-10T23:40:45.082465+0000","last_active":"2024-09-10T23:40:46.181744+0000","last_peered":"2024-09-10T23:40:46.181744+0000","last_clean":"2024-09-10T23:40:46.181744+0000","last_became_active":"2024-09-10T23:40:45.082143+0000","last_became_peered":"2024-09-10T23:40:45.082143+0000","last_unstale":"2024-09-10T23:40:46.181744+0000","last_undegraded":"2024-09-10T23:40:46.181744+0000","last_fullsized":"2024-09-10T23:40:46.181744+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:39:12.062896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.088525+0000","last_change":"2024-09-10T23:40:45.084090+0000","last_active":"2024-09-10T23:40:46.088525+0000","last_peered":"2024-09-10T23:40:46.088525+0000","last_clean":"2024-09-10T23:40:46.088525+0000","last_became_active":"2024-09-10T23:40:45.083222+0000","last_became_peered":"2024-09-10T23:40:45.083222+0000","last_unstale":"2024-09-10T23:40:46.088525+0000","last_undegraded":"2024-09-10T23:40:46.088525+0000","last_fullsized":"2024-09-10T23:40:46.088525+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:06.028814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181279+0000","last_change":"2024-09-10T23:40:45.084956+0000","last_active":"2024-09-10T23:40:46.181279+0000","last_peered":"2024-09-10T23:40:46.181279+0000","last_clean":"2024-09-10T23:40:46.181279+0000","last_became_active":"2024-09-10T23:40:45.084695+0000","last_became_peered":"2024-09-10T23:40:45.084695+0000","last_unstale":"2024-09-10T23:40:46.181279+0000","last_undegraded":"2024-09-10T23:40:46.181279+0000","last_fullsized":"2024-09-10T23:40:46.181279+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:42.070233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"161'4","reported_seq":19,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181245+0000","last_change":"2024-09-10T23:40:44.080123+0000","last_active":"2024-09-10T23:40:46.181245+0000","last_peered":"2024-09-10T23:40:46.181245+0000","last_clean":"2024-09-10T23:40:46.181245+0000","last_became_active":"2024-09-10T23:40:44.079862+0000","last_became_peered":"2024-09-10T23:40:44.079862+0000","last_unstale":"2024-09-10T23:40:46.181245+0000","last_undegraded":"2024-09-10T23:40:46.181245+0000","last_fullsized":"2024-09-10T23:40:46.181245+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:14.466933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181292+0000","last_change":"2024-09-10T23:40:45.081330+0000","last_active":"2024-09-10T23:40:46.181292+0000","last_peered":"2024-09-10T23:40:46.181292+0000","last_clean":"2024-09-10T23:40:46.181292+0000","last_became_active":"2024-09-10T23:40:45.080777+0000","last_became_peered":"2024-09-10T23:40:45.080777+0000","last_unstale":"2024-09-10T23:40:46.181292+0000","last_undegraded":"2024-09-10T23:40:46.181292+0000","last_fullsized":"2024-09-10T23:40:46.181292+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:13.212190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.083421+0000","last_change":"2024-09-10T23:40:44.083068+0000","last_active":"2024-09-10T23:40:46.083421+0000","last_peered":"2024-09-10T23:40:46.083421+0000","last_clean":"2024-09-10T23:40:46.083421+0000","last_became_active":"2024-09-10T23:40:44.081744+0000","last_became_peered":"2024-09-10T23:40:44.081744+0000","last_unstale":"2024-09-10T23:40:46.083421+0000","last_undegraded":"2024-09-10T23:40:46.083421+0000","last_fullsized":"2024-09-10T23:40:46.083421+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:09.367656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182251+0000","last_change":"2024-09-10T23:40:45.093812+0000","last_active":"2024-09-10T23:40:46.182251+0000","last_peered":"2024-09-10T23:40:46.182251+0000","last_clean":"2024-09-10T23:40:46.182251+0000","last_became_active":"2024-09-10T23:40:45.092777+0000","last_became_peered":"2024-09-10T23:40:45.092777+0000","last_unstale":"2024-09-10T23:40:46.182251+0000","last_undegraded":"2024-09-10T23:40:46.182251+0000","last_fullsized":"2024-09-10T23:40:46.182251+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:40.517676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181303+0000","last_change":"2024-09-10T23:40:44.076217+0000","last_active":"2024-09-10T23:40:46.181303+0000","last_peered":"2024-09-10T23:40:46.181303+0000","last_clean":"2024-09-10T23:40:46.181303+0000","last_became_active":"2024-09-10T23:40:44.075964+0000","last_became_peered":"2024-09-10T23:40:44.075964+0000","last_unstale":"2024-09-10T23:40:46.181303+0000","last_undegraded":"2024-09-10T23:40:46.181303+0000","last_fullsized":"2024-09-10T23:40:46.181303+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:35.744447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182402+0000","last_change":"2024-09-10T23:40:45.084782+0000","last_active":"2024-09-10T23:40:46.182402+0000","last_peered":"2024-09-10T23:40:46.182402+0000","last_clean":"2024-09-10T23:40:46.182402+0000","last_became_active":"2024-09-10T23:40:45.084203+0000","last_became_peered":"2024-09-10T23:40:45.084203+0000","last_unstale":"2024-09-10T23:40:46.182402+0000","last_undegraded":"2024-09-10T23:40:46.182402+0000","last_fullsized":"2024-09-10T23:40:46.182402+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:51.901951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1c","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181207+0000","last_change":"2024-09-10T23:40:44.081505+0000","last_active":"2024-09-10T23:40:46.181207+0000","last_peered":"2024-09-10T23:40:46.181207+0000","last_clean":"2024-09-10T23:40:46.181207+0000","last_became_active":"2024-09-10T23:40:44.081254+0000","last_became_peered":"2024-09-10T23:40:44.081254+0000","last_unstale":"2024-09-10T23:40:46.181207+0000","last_undegraded":"2024-09-10T23:40:46.181207+0000","last_fullsized":"2024-09-10T23:40:46.181207+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:26.630002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180811+0000","last_change":"2024-09-10T23:40:45.082104+0000","last_active":"2024-09-10T23:40:46.180811+0000","last_peered":"2024-09-10T23:40:46.180811+0000","last_clean":"2024-09-10T23:40:46.180811+0000","last_became_active":"2024-09-10T23:40:45.081841+0000","last_became_peered":"2024-09-10T23:40:45.081841+0000","last_unstale":"2024-09-10T23:40:46.180811+0000","last_undegraded":"2024-09-10T23:40:46.180811+0000","last_fullsized":"2024-09-10T23:40:46.180811+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:35.028658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089901+0000","last_change":"2024-09-10T23:40:44.074983+0000","last_active":"2024-09-10T23:40:46.089901+0000","last_peered":"2024-09-10T23:40:46.089901+0000","last_clean":"2024-09-10T23:40:46.089901+0000","last_became_active":"2024-09-10T23:40:44.074679+0000","last_became_peered":"2024-09-10T23:40:44.074679+0000","last_unstale":"2024-09-10T23:40:46.089901+0000","last_undegraded":"2024-09-10T23:40:46.089901+0000","last_fullsized":"2024-09-10T23:40:46.089901+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:14.301580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089182+0000","last_change":"2024-09-10T23:40:45.087279+0000","last_active":"2024-09-10T23:40:46.089182+0000","last_peered":"2024-09-10T23:40:46.089182+0000","last_clean":"2024-09-10T23:40:46.089182+0000","last_became_active":"2024-09-10T23:40:45.086944+0000","last_became_peered":"2024-09-10T23:40:45.086944+0000","last_unstale":"2024-09-10T23:40:46.089182+0000","last_undegraded":"2024-09-10T23:40:46.089182+0000","last_fullsized":"2024-09-10T23:40:46.089182+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:34.228170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180685+0000","last_change":"2024-09-10T23:40:44.079045+0000","last_active":"2024-09-10T23:40:46.180685+0000","last_peered":"2024-09-10T23:40:46.180685+0000","last_clean":"2024-09-10T23:40:46.180685+0000","last_became_active":"2024-09-10T23:40:44.078710+0000","last_became_peered":"2024-09-10T23:40:44.078710+0000","last_unstale":"2024-09-10T23:40:46.180685+0000","last_undegraded":"2024-09-10T23:40:46.180685+0000","last_fullsized":"2024-09-10T23:40:46.180685+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:24.057491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.171760+0000","last_change":"2024-09-10T23:40:45.093006+0000","last_active":"2024-09-10T23:40:46.171760+0000","last_peered":"2024-09-10T23:40:46.171760+0000","last_clean":"2024-09-10T23:40:46.171760+0000","last_became_active":"2024-09-10T23:40:45.092679+0000","last_became_peered":"2024-09-10T23:40:45.092679+0000","last_unstale":"2024-09-10T23:40:46.171760+0000","last_undegraded":"2024-09-10T23:40:46.171760+0000","last_fullsized":"2024-09-10T23:40:46.171760+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:05.330116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180334+0000","last_change":"2024-09-10T23:40:44.080346+0000","last_active":"2024-09-10T23:40:46.180334+0000","last_peered":"2024-09-10T23:40:46.180334+0000","last_clean":"2024-09-10T23:40:46.180334+0000","last_became_active":"2024-09-10T23:40:44.079671+0000","last_became_peered":"2024-09-10T23:40:44.079671+0000","last_unstale":"2024-09-10T23:40:46.180334+0000","last_undegraded":"2024-09-10T23:40:46.180334+0000","last_fullsized":"2024-09-10T23:40:46.180334+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:23:56.181920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180946+0000","last_change":"2024-09-10T23:40:45.085153+0000","last_active":"2024-09-10T23:40:46.180946+0000","last_peered":"2024-09-10T23:40:46.180946+0000","last_clean":"2024-09-10T23:40:46.180946+0000","last_became_active":"2024-09-10T23:40:45.084638+0000","last_became_peered":"2024-09-10T23:40:45.084638+0000","last_unstale":"2024-09-10T23:40:46.180946+0000","last_undegraded":"2024-09-10T23:40:46.180946+0000","last_fullsized":"2024-09-10T23:40:46.180946+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:16.773746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182345+0000","last_change":"2024-09-10T23:40:44.084811+0000","last_active":"2024-09-10T23:40:46.182345+0000","last_peered":"2024-09-10T23:40:46.182345+0000","last_clean":"2024-09-10T23:40:46.182345+0000","last_became_active":"2024-09-10T23:40:44.084437+0000","last_became_peered":"2024-09-10T23:40:44.084437+0000","last_unstale":"2024-09-10T23:40:46.182345+0000","last_undegraded":"2024-09-10T23:40:46.182345+0000","last_fullsized":"2024-09-10T23:40:46.182345+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:07.553007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.088613+0000","last_change":"2024-09-10T23:40:45.073784+0000","last_active":"2024-09-10T23:40:46.088613+0000","last_peered":"2024-09-10T23:40:46.088613+0000","last_clean":"2024-09-10T23:40:46.088613+0000","last_became_active":"2024-09-10T23:40:45.073492+0000","last_became_peered":"2024-09-10T23:40:45.073492+0000","last_unstale":"2024-09-10T23:40:46.088613+0000","last_undegraded":"2024-09-10T23:40:46.088613+0000","last_fullsized":"2024-09-10T23:40:46.088613+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:10:37.402798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181038+0000","last_change":"2024-09-10T23:40:44.080664+0000","last_active":"2024-09-10T23:40:46.181038+0000","last_peered":"2024-09-10T23:40:46.181038+0000","last_clean":"2024-09-10T23:40:46.181038+0000","last_became_active":"2024-09-10T23:40:44.080165+0000","last_became_peered":"2024-09-10T23:40:44.080165+0000","last_unstale":"2024-09-10T23:40:46.181038+0000","last_undegraded":"2024-09-10T23:40:46.181038+0000","last_fullsized":"2024-09-10T23:40:46.181038+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:49.391750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.172010+0000","last_change":"2024-09-10T23:40:45.077463+0000","last_active":"2024-09-10T23:40:46.172010+0000","last_peered":"2024-09-10T23:40:46.172010+0000","last_clean":"2024-09-10T23:40:46.172010+0000","last_became_active":"2024-09-10T23:40:45.077173+0000","last_became_peered":"2024-09-10T23:40:45.077173+0000","last_unstale":"2024-09-10T23:40:46.172010+0000","last_undegraded":"2024-09-10T23:40:46.172010+0000","last_fullsized":"2024-09-10T23:40:46.172010+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:03.998259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"161'3","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181342+0000","last_change":"2024-09-10T23:40:44.082263+0000","last_active":"2024-09-10T23:40:46.181342+0000","last_peered":"2024-09-10T23:40:46.181342+0000","last_clean":"2024-09-10T23:40:46.181342+0000","last_became_active":"2024-09-10T23:40:44.082008+0000","last_became_peered":"2024-09-10T23:40:44.082008+0000","last_unstale":"2024-09-10T23:40:46.181342+0000","last_undegraded":"2024-09-10T23:40:46.181342+0000","last_fullsized":"2024-09-10T23:40:46.181342+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:53:08.264800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181158+0000","last_change":"2024-09-10T23:40:45.078110+0000","last_active":"2024-09-10T23:40:46.181158+0000","last_peered":"2024-09-10T23:40:46.181158+0000","last_clean":"2024-09-10T23:40:46.181158+0000","last_became_active":"2024-09-10T23:40:45.077829+0000","last_became_peered":"2024-09-10T23:40:45.077829+0000","last_unstale":"2024-09-10T23:40:46.181158+0000","last_undegraded":"2024-09-10T23:40:46.181158+0000","last_fullsized":"2024-09-10T23:40:46.181158+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:38.343003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"161'6","reported_seq":21,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.171831+0000","last_change":"2024-09-10T23:40:44.076821+0000","last_active":"2024-09-10T23:40:46.171831+0000","last_peered":"2024-09-10T23:40:46.171831+0000","last_clean":"2024-09-10T23:40:46.171831+0000","last_became_active":"2024-09-10T23:40:44.076552+0000","last_became_peered":"2024-09-10T23:40:44.076552+0000","last_unstale":"2024-09-10T23:40:46.171831+0000","last_undegraded":"2024-09-10T23:40:46.171831+0000","last_fullsized":"2024-09-10T23:40:46.171831+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:14.955232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.088356+0000","last_change":"2024-09-10T23:40:45.085846+0000","last_active":"2024-09-10T23:40:46.088356+0000","last_peered":"2024-09-10T23:40:46.088356+0000","last_clean":"2024-09-10T23:40:46.088356+0000","last_became_active":"2024-09-10T23:40:45.085566+0000","last_became_peered":"2024-09-10T23:40:45.085566+0000","last_unstale":"2024-09-10T23:40:46.088356+0000","last_undegraded":"2024-09-10T23:40:46.088356+0000","last_fullsized":"2024-09-10T23:40:46.088356+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:43.635344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.13","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.171690+0000","last_change":"2024-09-10T23:40:44.084413+0000","last_active":"2024-09-10T23:40:46.171690+0000","last_peered":"2024-09-10T23:40:46.171690+0000","last_clean":"2024-09-10T23:40:46.171690+0000","last_became_active":"2024-09-10T23:40:44.084132+0000","last_became_peered":"2024-09-10T23:40:44.084132+0000","last_unstale":"2024-09-10T23:40:46.171690+0000","last_undegraded":"2024-09-10T23:40:46.171690+0000","last_fullsized":"2024-09-10T23:40:46.171690+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:44.715798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182545+0000","last_change":"2024-09-10T23:40:45.101237+0000","last_active":"2024-09-10T23:40:46.182545+0000","last_peered":"2024-09-10T23:40:46.182545+0000","last_clean":"2024-09-10T23:40:46.182545+0000","last_became_active":"2024-09-10T23:40:45.100916+0000","last_became_peered":"2024-09-10T23:40:45.100916+0000","last_unstale":"2024-09-10T23:40:46.182545+0000","last_undegraded":"2024-09-10T23:40:46.182545+0000","last_fullsized":"2024-09-10T23:40:46.182545+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:28:08.801109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.082866+0000","last_change":"2024-09-10T23:40:44.078360+0000","last_active":"2024-09-10T23:40:46.082866+0000","last_peered":"2024-09-10T23:40:46.082866+0000","last_clean":"2024-09-10T23:40:46.082866+0000","last_became_active":"2024-09-10T23:40:44.078103+0000","last_became_peered":"2024-09-10T23:40:44.078103+0000","last_unstale":"2024-09-10T23:40:46.082866+0000","last_undegraded":"2024-09-10T23:40:46.082866+0000","last_fullsized":"2024-09-10T23:40:46.082866+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:04.092914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181296+0000","last_change":"2024-09-10T23:40:45.088219+0000","last_active":"2024-09-10T23:40:46.181296+0000","last_peered":"2024-09-10T23:40:46.181296+0000","last_clean":"2024-09-10T23:40:46.181296+0000","last_became_active":"2024-09-10T23:40:45.087895+0000","last_became_peered":"2024-09-10T23:40:45.087895+0000","last_unstale":"2024-09-10T23:40:46.181296+0000","last_undegraded":"2024-09-10T23:40:46.181296+0000","last_fullsized":"2024-09-10T23:40:46.181296+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:08.426049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180711+0000","last_change":"2024-09-10T23:40:44.077071+0000","last_active":"2024-09-10T23:40:46.180711+0000","last_peered":"2024-09-10T23:40:46.180711+0000","last_clean":"2024-09-10T23:40:46.180711+0000","last_became_active":"2024-09-10T23:40:44.076812+0000","last_became_peered":"2024-09-10T23:40:44.076812+0000","last_unstale":"2024-09-10T23:40:46.180711+0000","last_undegraded":"2024-09-10T23:40:46.180711+0000","last_fullsized":"2024-09-10T23:40:46.180711+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:08.132953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089544+0000","last_change":"2024-09-10T23:40:45.090469+0000","last_active":"2024-09-10T23:40:46.089544+0000","last_peered":"2024-09-10T23:40:46.089544+0000","last_clean":"2024-09-10T23:40:46.089544+0000","last_became_active":"2024-09-10T23:40:45.090130+0000","last_became_peered":"2024-09-10T23:40:45.090130+0000","last_unstale":"2024-09-10T23:40:46.089544+0000","last_undegraded":"2024-09-10T23:40:46.089544+0000","last_fullsized":"2024-09-10T23:40:46.089544+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:11.039668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.10","version":"161'8","reported_seq":23,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181344+0000","last_change":"2024-09-10T23:40:44.079722+0000","last_active":"2024-09-10T23:40:46.181344+0000","last_peered":"2024-09-10T23:40:46.181344+0000","last_clean":"2024-09-10T23:40:46.181344+0000","last_became_active":"2024-09-10T23:40:44.074591+0000","last_became_peered":"2024-09-10T23:40:44.074591+0000","last_unstale":"2024-09-10T23:40:46.181344+0000","last_undegraded":"2024-09-10T23:40:46.181344+0000","last_fullsized":"2024-09-10T23:40:46.181344+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:23.708559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181323+0000","last_change":"2024-09-10T23:40:45.084770+0000","last_active":"2024-09-10T23:40:46.181323+0000","last_peered":"2024-09-10T23:40:46.181323+0000","last_clean":"2024-09-10T23:40:46.181323+0000","last_became_active":"2024-09-10T23:40:45.083550+0000","last_became_peered":"2024-09-10T23:40:45.083550+0000","last_unstale":"2024-09-10T23:40:46.181323+0000","last_undegraded":"2024-09-10T23:40:46.181323+0000","last_fullsized":"2024-09-10T23:40:46.181323+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:43.913895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.f","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.090237+0000","last_change":"2024-09-10T23:40:44.080468+0000","last_active":"2024-09-10T23:40:46.090237+0000","last_peered":"2024-09-10T23:40:46.090237+0000","last_clean":"2024-09-10T23:40:46.090237+0000","last_became_active":"2024-09-10T23:40:44.080151+0000","last_became_peered":"2024-09-10T23:40:44.080151+0000","last_unstale":"2024-09-10T23:40:46.090237+0000","last_undegraded":"2024-09-10T23:40:46.090237+0000","last_fullsized":"2024-09-10T23:40:46.090237+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:11.343095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181401+0000","last_change":"2024-09-10T23:40:45.083171+0000","last_active":"2024-09-10T23:40:46.181401+0000","last_peered":"2024-09-10T23:40:46.181401+0000","last_clean":"2024-09-10T23:40:46.181401+0000","last_became_active":"2024-09-10T23:40:45.082895+0000","last_became_peered":"2024-09-10T23:40:45.082895+0000","last_unstale":"2024-09-10T23:40:46.181401+0000","last_undegraded":"2024-09-10T23:40:46.181401+0000","last_fullsized":"2024-09-10T23:40:46.181401+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:01.948292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.171906+0000","last_change":"2024-09-10T23:40:44.077485+0000","last_active":"2024-09-10T23:40:46.171906+0000","last_peered":"2024-09-10T23:40:46.171906+0000","last_clean":"2024-09-10T23:40:46.171906+0000","last_became_active":"2024-09-10T23:40:44.077214+0000","last_became_peered":"2024-09-10T23:40:44.077214+0000","last_unstale":"2024-09-10T23:40:46.171906+0000","last_undegraded":"2024-09-10T23:40:46.171906+0000","last_fullsized":"2024-09-10T23:40:46.171906+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:17.795252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182367+0000","last_change":"2024-09-10T23:40:45.093289+0000","last_active":"2024-09-10T23:40:46.182367+0000","last_peered":"2024-09-10T23:40:46.182367+0000","last_clean":"2024-09-10T23:40:46.182367+0000","last_became_active":"2024-09-10T23:40:45.091960+0000","last_became_peered":"2024-09-10T23:40:45.091960+0000","last_unstale":"2024-09-10T23:40:46.182367+0000","last_undegraded":"2024-09-10T23:40:46.182367+0000","last_fullsized":"2024-09-10T23:40:46.182367+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:40:08.135993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.d","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.088713+0000","last_change":"2024-09-10T23:40:44.074809+0000","last_active":"2024-09-10T23:40:46.088713+0000","last_peered":"2024-09-10T23:40:46.088713+0000","last_clean":"2024-09-10T23:40:46.088713+0000","last_became_active":"2024-09-10T23:40:44.074511+0000","last_became_peered":"2024-09-10T23:40:44.074511+0000","last_unstale":"2024-09-10T23:40:46.088713+0000","last_undegraded":"2024-09-10T23:40:46.088713+0000","last_fullsized":"2024-09-10T23:40:46.088713+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:24.979379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089908+0000","last_change":"2024-09-10T23:40:45.089732+0000","last_active":"2024-09-10T23:40:46.089908+0000","last_peered":"2024-09-10T23:40:46.089908+0000","last_clean":"2024-09-10T23:40:46.089908+0000","last_became_active":"2024-09-10T23:40:45.089250+0000","last_became_peered":"2024-09-10T23:40:45.089250+0000","last_unstale":"2024-09-10T23:40:46.089908+0000","last_undegraded":"2024-09-10T23:40:46.089908+0000","last_fullsized":"2024-09-10T23:40:46.089908+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:25.067281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089565+0000","last_change":"2024-09-10T23:40:44.080523+0000","last_active":"2024-09-10T23:40:46.089565+0000","last_peered":"2024-09-10T23:40:46.089565+0000","last_clean":"2024-09-10T23:40:46.089565+0000","last_became_active":"2024-09-10T23:40:44.078290+0000","last_became_peered":"2024-09-10T23:40:44.078290+0000","last_unstale":"2024-09-10T23:40:46.089565+0000","last_undegraded":"2024-09-10T23:40:46.089565+0000","last_fullsized":"2024-09-10T23:40:46.089565+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:55.916591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182073+0000","last_change":"2024-09-10T23:40:45.087475+0000","last_active":"2024-09-10T23:40:46.182073+0000","last_peered":"2024-09-10T23:40:46.182073+0000","last_clean":"2024-09-10T23:40:46.182073+0000","last_became_active":"2024-09-10T23:40:45.086602+0000","last_became_peered":"2024-09-10T23:40:45.086602+0000","last_unstale":"2024-09-10T23:40:46.182073+0000","last_undegraded":"2024-09-10T23:40:46.182073+0000","last_fullsized":"2024-09-10T23:40:46.182073+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:20.787848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.b","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.088336+0000","last_change":"2024-09-10T23:40:44.073522+0000","last_active":"2024-09-10T23:40:46.088336+0000","last_peered":"2024-09-10T23:40:46.088336+0000","last_clean":"2024-09-10T23:40:46.088336+0000","last_became_active":"2024-09-10T23:40:44.073222+0000","last_became_peered":"2024-09-10T23:40:44.073222+0000","last_unstale":"2024-09-10T23:40:46.088336+0000","last_undegraded":"2024-09-10T23:40:46.088336+0000","last_fullsized":"2024-09-10T23:40:46.088336+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:29.109728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.090072+0000","last_change":"2024-09-10T23:40:45.085432+0000","last_active":"2024-09-10T23:40:46.090072+0000","last_peered":"2024-09-10T23:40:46.090072+0000","last_clean":"2024-09-10T23:40:46.090072+0000","last_became_active":"2024-09-10T23:40:45.085070+0000","last_became_peered":"2024-09-10T23:40:45.085070+0000","last_unstale":"2024-09-10T23:40:46.090072+0000","last_undegraded":"2024-09-10T23:40:46.090072+0000","last_fullsized":"2024-09-10T23:40:46.090072+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:36.792518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181230+0000","last_change":"2024-09-10T23:40:44.079389+0000","last_active":"2024-09-10T23:40:46.181230+0000","last_peered":"2024-09-10T23:40:46.181230+0000","last_clean":"2024-09-10T23:40:46.181230+0000","last_became_active":"2024-09-10T23:40:44.079085+0000","last_became_peered":"2024-09-10T23:40:44.079085+0000","last_unstale":"2024-09-10T23:40:46.181230+0000","last_undegraded":"2024-09-10T23:40:46.181230+0000","last_fullsized":"2024-09-10T23:40:46.181230+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:35.782729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180074+0000","last_change":"2024-09-10T23:40:45.079781+0000","last_active":"2024-09-10T23:40:46.180074+0000","last_peered":"2024-09-10T23:40:46.180074+0000","last_clean":"2024-09-10T23:40:46.180074+0000","last_became_active":"2024-09-10T23:40:45.079516+0000","last_became_peered":"2024-09-10T23:40:45.079516+0000","last_unstale":"2024-09-10T23:40:46.180074+0000","last_undegraded":"2024-09-10T23:40:46.180074+0000","last_fullsized":"2024-09-10T23:40:46.180074+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:59.042257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181687+0000","last_change":"2024-09-10T23:40:44.082435+0000","last_active":"2024-09-10T23:40:46.181687+0000","last_peered":"2024-09-10T23:40:46.181687+0000","last_clean":"2024-09-10T23:40:46.181687+0000","last_became_active":"2024-09-10T23:40:44.082147+0000","last_became_peered":"2024-09-10T23:40:44.082147+0000","last_unstale":"2024-09-10T23:40:46.181687+0000","last_undegraded":"2024-09-10T23:40:46.181687+0000","last_fullsized":"2024-09-10T23:40:46.181687+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:12.202400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180533+0000","last_change":"2024-09-10T23:40:45.090690+0000","last_active":"2024-09-10T23:40:46.180533+0000","last_peered":"2024-09-10T23:40:46.180533+0000","last_clean":"2024-09-10T23:40:46.180533+0000","last_became_active":"2024-09-10T23:40:45.090276+0000","last_became_peered":"2024-09-10T23:40:45.090276+0000","last_unstale":"2024-09-10T23:40:46.180533+0000","last_undegraded":"2024-09-10T23:40:46.180533+0000","last_fullsized":"2024-09-10T23:40:46.180533+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:09.368470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181114+0000","last_change":"2024-09-10T23:40:44.082739+0000","last_active":"2024-09-10T23:40:46.181114+0000","last_peered":"2024-09-10T23:40:46.181114+0000","last_clean":"2024-09-10T23:40:46.181114+0000","last_became_active":"2024-09-10T23:40:44.080910+0000","last_became_peered":"2024-09-10T23:40:44.080910+0000","last_unstale":"2024-09-10T23:40:46.181114+0000","last_undegraded":"2024-09-10T23:40:46.181114+0000","last_fullsized":"2024-09-10T23:40:46.181114+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:48.655919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181414+0000","last_change":"2024-09-10T23:40:45.095933+0000","last_active":"2024-09-10T23:40:46.181414+0000","last_peered":"2024-09-10T23:40:46.181414+0000","last_clean":"2024-09-10T23:40:46.181414+0000","last_became_active":"2024-09-10T23:40:45.094366+0000","last_became_peered":"2024-09-10T23:40:45.094366+0000","last_unstale":"2024-09-10T23:40:46.181414+0000","last_undegraded":"2024-09-10T23:40:46.181414+0000","last_fullsized":"2024-09-10T23:40:46.181414+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:40.636817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.7","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181518+0000","last_change":"2024-09-10T23:40:44.079487+0000","last_active":"2024-09-10T23:40:46.181518+0000","last_peered":"2024-09-10T23:40:46.181518+0000","last_clean":"2024-09-10T23:40:46.181518+0000","last_became_active":"2024-09-10T23:40:44.075501+0000","last_became_peered":"2024-09-10T23:40:44.075501+0000","last_unstale":"2024-09-10T23:40:46.181518+0000","last_undegraded":"2024-09-10T23:40:46.181518+0000","last_fullsized":"2024-09-10T23:40:46.181518+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:26.051116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.090075+0000","last_change":"2024-09-10T23:40:45.090499+0000","last_active":"2024-09-10T23:40:46.090075+0000","last_peered":"2024-09-10T23:40:46.090075+0000","last_clean":"2024-09-10T23:40:46.090075+0000","last_became_active":"2024-09-10T23:40:45.090112+0000","last_became_peered":"2024-09-10T23:40:45.090112+0000","last_unstale":"2024-09-10T23:40:46.090075+0000","last_undegraded":"2024-09-10T23:40:46.090075+0000","last_fullsized":"2024-09-10T23:40:46.090075+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:49.451778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.6","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.083324+0000","last_change":"2024-09-10T23:40:44.076657+0000","last_active":"2024-09-10T23:40:46.083324+0000","last_peered":"2024-09-10T23:40:46.083324+0000","last_clean":"2024-09-10T23:40:46.083324+0000","last_became_active":"2024-09-10T23:40:44.076402+0000","last_became_peered":"2024-09-10T23:40:44.076402+0000","last_unstale":"2024-09-10T23:40:46.083324+0000","last_undegraded":"2024-09-10T23:40:46.083324+0000","last_fullsized":"2024-09-10T23:40:46.083324+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:42.457163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.083120+0000","last_change":"2024-09-10T23:40:45.075558+0000","last_active":"2024-09-10T23:40:46.083120+0000","last_peered":"2024-09-10T23:40:46.083120+0000","last_clean":"2024-09-10T23:40:46.083120+0000","last_became_active":"2024-09-10T23:40:45.074348+0000","last_became_peered":"2024-09-10T23:40:45.074348+0000","last_unstale":"2024-09-10T23:40:46.083120+0000","last_undegraded":"2024-09-10T23:40:46.083120+0000","last_fullsized":"2024-09-10T23:40:46.083120+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:55.908829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.090012+0000","last_change":"2024-09-10T23:40:44.077621+0000","last_active":"2024-09-10T23:40:46.090012+0000","last_peered":"2024-09-10T23:40:46.090012+0000","last_clean":"2024-09-10T23:40:46.090012+0000","last_became_active":"2024-09-10T23:40:44.077160+0000","last_became_peered":"2024-09-10T23:40:44.077160+0000","last_unstale":"2024-09-10T23:40:46.090012+0000","last_undegraded":"2024-09-10T23:40:46.090012+0000","last_fullsized":"2024-09-10T23:40:46.090012+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:00.848681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180980+0000","last_change":"2024-09-10T23:40:45.095623+0000","last_active":"2024-09-10T23:40:46.180980+0000","last_peered":"2024-09-10T23:40:46.180980+0000","last_clean":"2024-09-10T23:40:46.180980+0000","last_became_active":"2024-09-10T23:40:45.094804+0000","last_became_peered":"2024-09-10T23:40:45.094804+0000","last_unstale":"2024-09-10T23:40:46.180980+0000","last_undegraded":"2024-09-10T23:40:46.180980+0000","last_fullsized":"2024-09-10T23:40:46.180980+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:50.195796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089522+0000","last_change":"2024-09-10T23:40:44.081193+0000","last_active":"2024-09-10T23:40:46.089522+0000","last_peered":"2024-09-10T23:40:46.089522+0000","last_clean":"2024-09-10T23:40:46.089522+0000","last_became_active":"2024-09-10T23:40:44.080941+0000","last_became_peered":"2024-09-10T23:40:44.080941+0000","last_unstale":"2024-09-10T23:40:46.089522+0000","last_undegraded":"2024-09-10T23:40:46.089522+0000","last_fullsized":"2024-09-10T23:40:46.089522+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:37:14.356564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.089589+0000","last_change":"2024-09-10T23:40:45.086167+0000","last_active":"2024-09-10T23:40:46.089589+0000","last_peered":"2024-09-10T23:40:46.089589+0000","last_clean":"2024-09-10T23:40:46.089589+0000","last_became_active":"2024-09-10T23:40:45.085618+0000","last_became_peered":"2024-09-10T23:40:45.085618+0000","last_unstale":"2024-09-10T23:40:46.089589+0000","last_undegraded":"2024-09-10T23:40:46.089589+0000","last_fullsized":"2024-09-10T23:40:46.089589+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:35.179057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180725+0000","last_change":"2024-09-10T23:40:44.079412+0000","last_active":"2024-09-10T23:40:46.180725+0000","last_peered":"2024-09-10T23:40:46.180725+0000","last_clean":"2024-09-10T23:40:46.180725+0000","last_became_active":"2024-09-10T23:40:44.079111+0000","last_became_peered":"2024-09-10T23:40:44.079111+0000","last_unstale":"2024-09-10T23:40:46.180725+0000","last_undegraded":"2024-09-10T23:40:46.180725+0000","last_fullsized":"2024-09-10T23:40:46.180725+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:31.442614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.083177+0000","last_change":"2024-09-10T23:40:45.081514+0000","last_active":"2024-09-10T23:40:46.083177+0000","last_peered":"2024-09-10T23:40:46.083177+0000","last_clean":"2024-09-10T23:40:46.083177+0000","last_became_active":"2024-09-10T23:40:45.081246+0000","last_became_peered":"2024-09-10T23:40:45.081246+0000","last_unstale":"2024-09-10T23:40:46.083177+0000","last_undegraded":"2024-09-10T23:40:46.083177+0000","last_fullsized":"2024-09-10T23:40:46.083177+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:21.423350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.182115+0000","last_change":"2024-09-10T23:40:44.081308+0000","last_active":"2024-09-10T23:40:46.182115+0000","last_peered":"2024-09-10T23:40:46.182115+0000","last_clean":"2024-09-10T23:40:46.182115+0000","last_became_active":"2024-09-10T23:40:44.080898+0000","last_became_peered":"2024-09-10T23:40:44.080898+0000","last_unstale":"2024-09-10T23:40:46.182115+0000","last_undegraded":"2024-09-10T23:40:46.182115+0000","last_fullsized":"2024-09-10T23:40:46.182115+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:00.938502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180827+0000","last_change":"2024-09-10T23:40:45.076548+0000","last_active":"2024-09-10T23:40:46.180827+0000","last_peered":"2024-09-10T23:40:46.180827+0000","last_clean":"2024-09-10T23:40:46.180827+0000","last_became_active":"2024-09-10T23:40:45.076264+0000","last_became_peered":"2024-09-10T23:40:45.076264+0000","last_unstale":"2024-09-10T23:40:46.180827+0000","last_undegraded":"2024-09-10T23:40:46.180827+0000","last_fullsized":"2024-09-10T23:40:46.180827+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:10.377446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180246+0000","last_change":"2024-09-10T23:40:44.074775+0000","last_active":"2024-09-10T23:40:46.180246+0000","last_peered":"2024-09-10T23:40:46.180246+0000","last_clean":"2024-09-10T23:40:46.180246+0000","last_became_active":"2024-09-10T23:40:44.074206+0000","last_became_peered":"2024-09-10T23:40:44.074206+0000","last_unstale":"2024-09-10T23:40:46.180246+0000","last_undegraded":"2024-09-10T23:40:46.180246+0000","last_fullsized":"2024-09-10T23:40:46.180246+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:07.553007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180730+0000","last_change":"2024-09-10T23:40:45.090693+0000","last_active":"2024-09-10T23:40:46.180730+0000","last_peered":"2024-09-10T23:40:46.180730+0000","last_clean":"2024-09-10T23:40:46.180730+0000","last_became_active":"2024-09-10T23:40:45.090206+0000","last_became_peered":"2024-09-10T23:40:45.090206+0000","last_unstale":"2024-09-10T23:40:46.180730+0000","last_undegraded":"2024-09-10T23:40:46.180730+0000","last_fullsized":"2024-09-10T23:40:46.180730+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:11:08.283129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180283+0000","last_change":"2024-09-10T23:40:44.080507+0000","last_active":"2024-09-10T23:40:46.180283+0000","last_peered":"2024-09-10T23:40:46.180283+0000","last_clean":"2024-09-10T23:40:46.180283+0000","last_became_active":"2024-09-10T23:40:44.080069+0000","last_became_peered":"2024-09-10T23:40:44.080069+0000","last_unstale":"2024-09-10T23:40:46.180283+0000","last_undegraded":"2024-09-10T23:40:46.180283+0000","last_fullsized":"2024-09-10T23:40:46.180283+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:58.973485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.181897+0000","last_change":"2024-09-10T23:40:45.091633+0000","last_active":"2024-09-10T23:40:46.181897+0000","last_peered":"2024-09-10T23:40:46.181897+0000","last_clean":"2024-09-10T23:40:46.181897+0000","last_became_active":"2024-09-10T23:40:45.091354+0000","last_became_peered":"2024-09-10T23:40:45.091354+0000","last_unstale":"2024-09-10T23:40:46.181897+0000","last_undegraded":"2024-09-10T23:40:46.181897+0000","last_fullsized":"2024-09-10T23:40:46.181897+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:44.050643+0000","last_clean_scrub_stamp":"2024-09-10T23:40:44.050643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:21.659769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.083305+0000","last_change":"2024-09-10T23:40:44.074854+0000","last_active":"2024-09-10T23:40:46.083305+0000","last_peered":"2024-09-10T23:40:46.083305+0000","last_clean":"2024-09-10T23:40:46.083305+0000","last_became_active":"2024-09-10T23:40:44.074588+0000","last_became_peered":"2024-09-10T23:40:44.074588+0000","last_unstale":"2024-09-10T23:40:46.083305+0000","last_undegraded":"2024-09-10T23:40:46.083305+0000","last_fullsized":"2024-09-10T23:40:46.083305+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:40:43.040853+0000","last_clean_scrub_stamp":"2024-09-10T23:40:43.040853+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:45.851870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"70'79","reported_seq":396,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-10T23:40:46.180368+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:40:46.180368+0000","last_peered":"2024-09-10T23:40:46.180368+0000","last_clean":"2024-09-10T23:40:46.180368+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:40:46.180368+0000","last_undegraded":"2024-09-10T23:40:46.180368+0000","last_fullsized":"2024-09-10T23:40:46.180368+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509545,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2043904,"data_stored":899938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542248,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2043904,"data_stored":899986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509544,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33780,"kb_used_data":2588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338060,"statfs":{"total":96636764160,"available":96602173440,"internally_reserved":0,"allocated":2650112,"data_stored":1491012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33192,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338648,"statfs":{"total":96636764160,"available":96602775552,"internally_reserved":0,"allocated":2043904,"data_stored":900006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509543,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37352,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334488,"statfs":{"total":96636764160,"available":96598515712,"internally_reserved":0,"allocated":2052096,"data_stored":901528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509544,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37360,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334480,"statfs":{"total":96636764160,"available":96598507520,"internally_reserved":0,"allocated":2060288,"data_stored":901632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509543,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37924,"kb_used_data":2572,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333916,"statfs":{"total":96636764160,"available":96597929984,"internally_reserved":0,"allocated":2633728,"data_stored":1490284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509543,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37344,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334496,"statfs":{"total":96636764160,"available":96598523904,"internally_reserved":0,"allocated":2043904,"data_stored":900498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:40:49.580 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T23:40:49.580 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T23:40:49.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.767+0000 7f7c98800640 1 -- 172.21.15.94:0/735848181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c78004d50 msgr2=0x7f7c780051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:49.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.767+0000 7f7c98800640 1 --2- 172.21.15.94:0/735848181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c78004d50 0x7f7c780051d0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009eb0 tx=0x7f7c8c01d5b0 comp rx=0 tx=0).stop 2024-09-10T23:40:49.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.767+0000 7f7c96800640 1 -- 172.21.15.94:0/735848181 reap_dead start 2024-09-10T23:40:49.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.767+0000 7f7c98800640 1 -- 172.21.15.94:0/735848181 shutdown_connections 2024-09-10T23:40:49.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.767+0000 7f7c98800640 1 -- 172.21.15.94:0/735848181 >> 172.21.15.94:0/735848181 conn(0x7f7c900d1c60 msgr2=0x7f7c900d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:49.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.767+0000 7f7c98800640 1 -- 172.21.15.94:0/735848181 shutdown_connections 2024-09-10T23:40:49.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.767+0000 7f7c98800640 1 -- 172.21.15.94:0/735848181 wait complete. 2024-09-10T23:40:49.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c98800640 1 Processor -- start 2024-09-10T23:40:49.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c98800640 1 -- start start 2024-09-10T23:40:49.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c98800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c9014f2d0 0x7f7c90153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:49.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c98800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c90153540 0x7f7c90155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:49.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c98800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c90155640 0x7f7c901f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:49.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c98800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c90164ae0 con 0x7f7c90153540 2024-09-10T23:40:49.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c98800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7c90164960 con 0x7f7c9014f2d0 2024-09-10T23:40:49.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c98800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f7c90164c60 con 0x7f7c90155640 2024-09-10T23:40:49.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c95400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c90153540 0x7f7c90155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:49.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c95400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c90153540 0x7f7c90155100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59392/0 (socket says 172.21.15.94:59392) 2024-09-10T23:40:49.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c95400640 1 -- 172.21.15.94:0/1244414350 learned_addr learned my addr 172.21.15.94:0/1244414350 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:49.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c96800640 1 --2- 172.21.15.94:0/1244414350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c90155640 0x7f7c901f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:49.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c95e00640 1 --2- 172.21.15.94:0/1244414350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c9014f2d0 0x7f7c90153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:49.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c95400640 1 -- 172.21.15.94:0/1244414350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c90155640 msgr2=0x7f7c901f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:49.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c95400640 1 --2- 172.21.15.94:0/1244414350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c90155640 0x7f7c901f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:49.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c95400640 1 -- 172.21.15.94:0/1244414350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c9014f2d0 msgr2=0x7f7c90153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:49.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c95400640 1 --2- 172.21.15.94:0/1244414350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c9014f2d0 0x7f7c90153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:49.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c95400640 1 -- 172.21.15.94:0/1244414350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c90155e30 con 0x7f7c90153540 2024-09-10T23:40:49.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c95e00640 1 --2- 172.21.15.94:0/1244414350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c9014f2d0 0x7f7c90153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:40:49.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c95400640 1 --2- 172.21.15.94:0/1244414350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c90153540 0x7f7c90155100 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005f50 tx=0x7f7c80011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:49.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c8a000640 1 -- 172.21.15.94:0/1244414350 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80005320 con 0x7f7c90153540 2024-09-10T23:40:49.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c8a000640 1 -- 172.21.15.94:0/1244414350 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c80018020 con 0x7f7c90153540 2024-09-10T23:40:49.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c98800640 1 -- 172.21.15.94:0/1244414350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c90156110 con 0x7f7c90153540 2024-09-10T23:40:49.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c8a000640 1 -- 172.21.15.94:0/1244414350 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80003860 con 0x7f7c90153540 2024-09-10T23:40:49.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.771+0000 7f7c98800640 1 -- 172.21.15.94:0/1244414350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c90155af0 con 0x7f7c90153540 2024-09-10T23:40:49.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.775+0000 7f7c98800640 1 -- 172.21.15.94:0/1244414350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c90156350 con 0x7f7c90153540 2024-09-10T23:40:49.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.775+0000 7f7c8a000640 1 -- 172.21.15.94:0/1244414350 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f7c80010040 con 0x7f7c90153540 2024-09-10T23:40:49.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.775+0000 7f7c8a000640 1 --2- 172.21.15.94:0/1244414350 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7c74073030 0x7f7c740754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:49.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.775+0000 7f7c8a000640 1 -- 172.21.15.94:0/1244414350 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f7c80095210 con 0x7f7c90153540 2024-09-10T23:40:49.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.775+0000 7f7c95e00640 1 --2- 172.21.15.94:0/1244414350 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7c74073030 0x7f7c740754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:49.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.775+0000 7f7c95e00640 1 --2- 172.21.15.94:0/1244414350 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7c74073030 0x7f7c740754f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7c84001ec0 tx=0x7f7c84001dd0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:49.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:49.779+0000 7f7c8a000640 1 -- 172.21.15.94:0/1244414350 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7c90156350 con 0x7f7c90153540 2024-09-10T23:40:50.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.003+0000 7f7c98800640 1 -- 172.21.15.94:0/1244414350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7c900cfce0 con 0x7f7c90153540 2024-09-10T23:40:50.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.007+0000 7f7c8a000640 1 -- 172.21.15.94:0/1244414350 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v41) ==== 190+0+147 (secure 0 0 0) 0x7f7c80022020 con 0x7f7c90153540 2024-09-10T23:40:50.006 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-10T23:40:50.006 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDZ1uBmacixCxAAdOtkve12A0VB2FrscMN73g== 2024-09-10T23:40:50.006 INFO:teuthology.orchestra.run.smithi094.stdout: caps mds = "allow" 2024-09-10T23:40:50.006 INFO:teuthology.orchestra.run.smithi094.stdout: caps mon = "allow r" 2024-09-10T23:40:50.007 INFO:teuthology.orchestra.run.smithi094.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T23:40:50.007 INFO:teuthology.orchestra.run.smithi094.stderr:updated caps for client.0 2024-09-10T23:40:50.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.007+0000 7f7c98800640 1 -- 172.21.15.94:0/1244414350 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7c74073030 msgr2=0x7f7c740754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:50.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.007+0000 7f7c98800640 1 --2- 172.21.15.94:0/1244414350 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7c74073030 0x7f7c740754f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7c84001ec0 tx=0x7f7c84001dd0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.007+0000 7f7c98800640 1 -- 172.21.15.94:0/1244414350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c90153540 msgr2=0x7f7c90155100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:50.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.007+0000 7f7c98800640 1 --2- 172.21.15.94:0/1244414350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c90153540 0x7f7c90155100 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005f50 tx=0x7f7c80011800 comp rx=0 tx=0).stop 2024-09-10T23:40:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.007+0000 7f7c98800640 1 -- 172.21.15.94:0/1244414350 shutdown_connections 2024-09-10T23:40:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.007+0000 7f7c98800640 1 --2- 172.21.15.94:0/1244414350 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f7c74073030 0x7f7c740754f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.007+0000 7f7c98800640 1 --2- 172.21.15.94:0/1244414350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c90155640 0x7f7c901f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.007+0000 7f7c98800640 1 --2- 172.21.15.94:0/1244414350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c90153540 0x7f7c90155100 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.007+0000 7f7c98800640 1 --2- 172.21.15.94:0/1244414350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c9014f2d0 0x7f7c90153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.007+0000 7f7c98800640 1 -- 172.21.15.94:0/1244414350 >> 172.21.15.94:0/1244414350 conn(0x7f7c900d1c60 msgr2=0x7f7c900709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.007+0000 7f7c98800640 1 -- 172.21.15.94:0/1244414350 shutdown_connections 2024-09-10T23:40:50.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.007+0000 7f7c98800640 1 -- 172.21.15.94:0/1244414350 wait complete. 2024-09-10T23:40:50.027 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:40:50.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.171+0000 7f9bd3800640 1 -- 172.21.15.94:0/537895649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bb0003a10 msgr2=0x7f9bb0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:50.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.171+0000 7f9bd3800640 1 --2- 172.21.15.94:0/537895649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bb0003a10 0x7f9bb0003e30 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00c290 tx=0x7f9bbc01cf70 comp rx=0 tx=0).stop 2024-09-10T23:40:50.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.171+0000 7f9bd1800640 1 -- 172.21.15.94:0/537895649 reap_dead start 2024-09-10T23:40:50.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.171+0000 7f9bd3800640 1 -- 172.21.15.94:0/537895649 shutdown_connections 2024-09-10T23:40:50.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.171+0000 7f9bd3800640 1 -- 172.21.15.94:0/537895649 >> 172.21.15.94:0/537895649 conn(0x7f9bcc06cdf0 msgr2=0x7f9bcc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:50.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.171+0000 7f9bd3800640 1 -- 172.21.15.94:0/537895649 shutdown_connections 2024-09-10T23:40:50.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.171+0000 7f9bd3800640 1 -- 172.21.15.94:0/537895649 wait complete. 2024-09-10T23:40:50.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.171+0000 7f9bd3800640 1 Processor -- start 2024-09-10T23:40:50.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.171+0000 7f9bd3800640 1 -- start start 2024-09-10T23:40:50.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bd3800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bb0003a10 0x7f9bcc13fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bd3800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc1fbca0 0x7f9bcc1fb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bd3800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bcc0b1e60 0x7f9bcc13f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bd3800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9bcc1657a0 con 0x7f9bcc1fbca0 2024-09-10T23:40:50.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bd3800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9bcc0704c0 con 0x7f9bb0003a10 2024-09-10T23:40:50.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bd3800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9bcc165920 con 0x7f9bcc0b1e60 2024-09-10T23:40:50.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bc3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc1fbca0 0x7f9bcc1fb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bc3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc1fbca0 0x7f9bcc1fb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59416/0 (socket says 172.21.15.94:59416) 2024-09-10T23:40:50.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bc3e00640 1 -- 172.21.15.94:0/298179631 learned_addr learned my addr 172.21.15.94:0/298179631 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:50.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bc3e00640 1 -- 172.21.15.94:0/298179631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bcc0b1e60 msgr2=0x7f9bcc13f0e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T23:40:50.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bc3e00640 1 --2- 172.21.15.94:0/298179631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bcc0b1e60 0x7f9bcc13f0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bc3e00640 1 -- 172.21.15.94:0/298179631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bb0003a10 msgr2=0x7f9bcc13fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:50.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bc3e00640 1 --2- 172.21.15.94:0/298179631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bb0003a10 0x7f9bcc13fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bc3e00640 1 -- 172.21.15.94:0/298179631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bcc06a160 con 0x7f9bcc1fbca0 2024-09-10T23:40:50.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bc3e00640 1 --2- 172.21.15.94:0/298179631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc1fbca0 0x7f9bcc1fb280 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4005e10 tx=0x7f9bb400fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:50.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bc9600640 1 -- 172.21.15.94:0/298179631 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb400b9f0 con 0x7f9bcc1fbca0 2024-09-10T23:40:50.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bc9600640 1 -- 172.21.15.94:0/298179631 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bb400bc80 con 0x7f9bcc1fbca0 2024-09-10T23:40:50.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bc9600640 1 -- 172.21.15.94:0/298179631 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4011440 con 0x7f9bcc1fbca0 2024-09-10T23:40:50.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bd3800640 1 -- 172.21.15.94:0/298179631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bcc15c200 con 0x7f9bcc1fbca0 2024-09-10T23:40:50.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bd3800640 1 -- 172.21.15.94:0/298179631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bcc15ca30 con 0x7f9bcc1fbca0 2024-09-10T23:40:50.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bd3800640 1 -- 172.21.15.94:0/298179631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bcc06f4b0 con 0x7f9bcc1fbca0 2024-09-10T23:40:50.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.175+0000 7f9bc9600640 1 -- 172.21.15.94:0/298179631 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f9bb40115e0 con 0x7f9bcc1fbca0 2024-09-10T23:40:50.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.179+0000 7f9bc9600640 1 --2- 172.21.15.94:0/298179631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9ba8073000 0x7f9ba80754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.179+0000 7f9bc9600640 1 -- 172.21.15.94:0/298179631 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f9bb4092900 con 0x7f9bcc1fbca0 2024-09-10T23:40:50.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.179+0000 7f9bd0e00640 1 --2- 172.21.15.94:0/298179631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9ba8073000 0x7f9ba80754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.179+0000 7f9bd0e00640 1 --2- 172.21.15.94:0/298179631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9ba8073000 0x7f9ba80754c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0ac000 tx=0x7f9bb0005840 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:50.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.183+0000 7f9bc9600640 1 -- 172.21.15.94:0/298179631 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9bb405f810 con 0x7f9bcc1fbca0 2024-09-10T23:40:50.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.383+0000 7f9bd3800640 1 -- 172.21.15.94:0/298179631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bcc1647a0 con 0x7f9bcc1fbca0 2024-09-10T23:40:50.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.383+0000 7f9bc9600640 1 -- 172.21.15.94:0/298179631 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5239 (secure 0 0 0) 0x7f9bb405f110 con 0x7f9bcc1fbca0 2024-09-10T23:40:50.383 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:40:50.383 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":116,"btime":"2024-09-10T23:40:48:088811+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/2653759764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2653759764},{"type":"v1","addr":"172.21.15.181:6801","nonce":2653759764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":8518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2240199080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2240199080},{"type":"v1","addr":"172.21.15.181:6835","nonce":2240199080}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1675026105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1675026105},{"type":"v1","addr":"172.21.15.94:6834","nonce":1675026105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:40:44.453069+0000","modified":"2024-09-10T23:40:47.471636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9171":{"gid":9171,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/3607539278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3607539278},{"type":"v1","addr":"172.21.15.94:6835","nonce":3607539278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9171,"qdb_cluster":[9171]},"id":13}]} 2024-09-10T23:40:50.384 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 116 2024-09-10T23:40:50.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.387+0000 7f9bd3800640 1 -- 172.21.15.94:0/298179631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9ba8073000 msgr2=0x7f9ba80754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:50.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.387+0000 7f9bd3800640 1 --2- 172.21.15.94:0/298179631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9ba8073000 0x7f9ba80754c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0ac000 tx=0x7f9bb0005840 comp rx=0 tx=0).stop 2024-09-10T23:40:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.387+0000 7f9bd3800640 1 -- 172.21.15.94:0/298179631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc1fbca0 msgr2=0x7f9bcc1fb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.387+0000 7f9bd3800640 1 --2- 172.21.15.94:0/298179631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc1fbca0 0x7f9bcc1fb280 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4005e10 tx=0x7f9bb400fbe0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.387+0000 7f9bd3800640 1 -- 172.21.15.94:0/298179631 shutdown_connections 2024-09-10T23:40:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.387+0000 7f9bd3800640 1 --2- 172.21.15.94:0/298179631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9ba8073000 0x7f9ba80754c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.387+0000 7f9bd3800640 1 --2- 172.21.15.94:0/298179631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bcc0b1e60 0x7f9bcc13f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.387+0000 7f9bd3800640 1 --2- 172.21.15.94:0/298179631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc1fbca0 0x7f9bcc1fb280 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.387+0000 7f9bd3800640 1 --2- 172.21.15.94:0/298179631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bb0003a10 0x7f9bcc13fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.387+0000 7f9bd3800640 1 -- 172.21.15.94:0/298179631 >> 172.21.15.94:0/298179631 conn(0x7f9bcc06cdf0 msgr2=0x7f9bcc15f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.387+0000 7f9bd3800640 1 -- 172.21.15.94:0/298179631 shutdown_connections 2024-09-10T23:40:50.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.387+0000 7f9bd3800640 1 -- 172.21.15.94:0/298179631 wait complete. 2024-09-10T23:40:50.807 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 116, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T23:40:44.453069+0000', 'modified': '2024-09-10T23:40:47.471636+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9171}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9171': {'gid': 9171, 'name': 'a', 'rank': 0, 'incarnation': 113, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.94:6835/3607539278', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6833', 'nonce': 3607539278}, {'type': 'v1', 'addr': '172.21.15.94:6835', 'nonce': 3607539278}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9171, 'qdb_cluster': [9171]} 2024-09-10T23:40:50.808 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T23:40:50.808 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T23:40:50.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.951+0000 7f0dc7400640 1 -- 172.21.15.94:0/676914296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dc0158790 msgr2=0x7f0dc006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:50.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.951+0000 7f0dc7400640 1 --2- 172.21.15.94:0/676914296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dc0158790 0x7f0dc006fc00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00caab0 tx=0x7f0db001c820 comp rx=0 tx=0).stop 2024-09-10T23:40:50.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.951+0000 7f0dc7400640 1 -- 172.21.15.94:0/676914296 shutdown_connections 2024-09-10T23:40:50.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.951+0000 7f0dc7400640 1 --2- 172.21.15.94:0/676914296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dc0158790 0x7f0dc006fc00 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.951+0000 7f0dc7400640 1 -- 172.21.15.94:0/676914296 >> 172.21.15.94:0/676914296 conn(0x7f0dc00cfaf0 msgr2=0x7f0dc00cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:50.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.951+0000 7f0dc7400640 1 -- 172.21.15.94:0/676914296 shutdown_connections 2024-09-10T23:40:50.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.951+0000 7f0dc7400640 1 -- 172.21.15.94:0/676914296 wait complete. 2024-09-10T23:40:50.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.951+0000 7f0dc7400640 1 Processor -- start 2024-09-10T23:40:50.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.951+0000 7f0dc7400640 1 -- start start 2024-09-10T23:40:50.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dc00c49b0 0x7f0dc015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dc00cb4d0 0x7f0dc015e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc7400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dc0158790 0x7f0dc015c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc7400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc0168f20 con 0x7f0dc00c49b0 2024-09-10T23:40:50.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc7400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc0168da0 con 0x7f0dc00cb4d0 2024-09-10T23:40:50.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc7400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc01690a0 con 0x7f0dc0158790 2024-09-10T23:40:50.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dc00c49b0 0x7f0dc015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dc00c49b0 0x7f0dc015dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59422/0 (socket says 172.21.15.94:59422) 2024-09-10T23:40:50.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc4a00640 1 -- 172.21.15.94:0/368187945 learned_addr learned my addr 172.21.15.94:0/368187945 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:50.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc5400640 1 --2- 172.21.15.94:0/368187945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dc0158790 0x7f0dc015c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0db7e00640 1 --2- 172.21.15.94:0/368187945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dc00cb4d0 0x7f0dc015e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc4a00640 1 -- 172.21.15.94:0/368187945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dc0158790 msgr2=0x7f0dc015c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:50.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc4a00640 1 --2- 172.21.15.94:0/368187945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dc0158790 0x7f0dc015c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc4a00640 1 -- 172.21.15.94:0/368187945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dc00cb4d0 msgr2=0x7f0dc015e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:50.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc4a00640 1 --2- 172.21.15.94:0/368187945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dc00cb4d0 0x7f0dc015e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc4a00640 1 -- 172.21.15.94:0/368187945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc01f88a0 con 0x7f0dc00c49b0 2024-09-10T23:40:50.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0db7e00640 1 --2- 172.21.15.94:0/368187945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dc00cb4d0 0x7f0dc015e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:50.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc5400640 1 --2- 172.21.15.94:0/368187945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dc0158790 0x7f0dc015c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:50.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc4a00640 1 --2- 172.21.15.94:0/368187945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dc00c49b0 0x7f0dc015dba0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f0db00b0040 tx=0x7f0db0005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:50.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dbd600640 1 -- 172.21.15.94:0/368187945 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db00bb070 con 0x7f0dc00c49b0 2024-09-10T23:40:50.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dbd600640 1 -- 172.21.15.94:0/368187945 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0db0004320 con 0x7f0dc00c49b0 2024-09-10T23:40:50.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dbd600640 1 -- 172.21.15.94:0/368187945 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db00b2040 con 0x7f0dc00c49b0 2024-09-10T23:40:50.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc7400640 1 -- 172.21.15.94:0/368187945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc01f8b80 con 0x7f0dc00c49b0 2024-09-10T23:40:50.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.955+0000 7f0dc7400640 1 -- 172.21.15.94:0/368187945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc01f9140 con 0x7f0dc00c49b0 2024-09-10T23:40:50.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0dbd600640 1 -- 172.21.15.94:0/368187945 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f0db00b5030 con 0x7f0dc00c49b0 2024-09-10T23:40:50.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0dbd600640 1 --2- 172.21.15.94:0/368187945 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0d9c073030 0x7f0d9c0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0dca75c000 1 Processor -- start 2024-09-10T23:40:50.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0dbd600640 1 -- 172.21.15.94:0/368187945 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f0db012b1e0 con 0x7f0dc00c49b0 2024-09-10T23:40:50.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0dca75c000 1 -- start start 2024-09-10T23:40:50.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0db7e00640 1 --2- 172.21.15.94:0/368187945 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0d9c073030 0x7f0d9c0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0db7e00640 1 --2- 172.21.15.94:0/368187945 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0d9c073030 0x7f0d9c0754f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0da8001230 tx=0x7f0da8009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:50.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0dca75c000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 0x56521cc36db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0dca75c000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cc372f0 0x56521cbea8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0dca75c000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cbeae10 0x56521cbeb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0dca75c000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56521cb0b3f0 con 0x56521cc372f0 2024-09-10T23:40:50.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0dca75c000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56521ca48b20 con 0x56521cc369d0 2024-09-10T23:40:50.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0dca75c000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x56521ca37f50 con 0x56521cbeae10 2024-09-10T23:40:50.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0db7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cc372f0 0x56521cbea8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0db7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cc372f0 0x56521cbea8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59434/0 (socket says 172.21.15.94:59434) 2024-09-10T23:40:50.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0db7e00640 1 -- 172.21.15.94:0/2749876290 learned_addr learned my addr 172.21.15.94:0/2749876290 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:50.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0dc5400640 1 --2- 172.21.15.94:0/2749876290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cbeae10 0x56521cbeb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0dc4a00640 1 --2- 172.21.15.94:0/2749876290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 0x56521cc36db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0db7e00640 1 -- 172.21.15.94:0/2749876290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cbeae10 msgr2=0x56521cbeb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:40:50.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0db7e00640 1 --2- 172.21.15.94:0/2749876290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cbeae10 0x56521cbeb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0db7e00640 1 -- 172.21.15.94:0/2749876290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 msgr2=0x56521cc36db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:40:50.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0db7e00640 1 --2- 172.21.15.94:0/2749876290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 0x56521cc36db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0db7e00640 1 -- 172.21.15.94:0/2749876290 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56521c9ff590 con 0x56521cc372f0 2024-09-10T23:40:50.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0db7e00640 1 --2- 172.21.15.94:0/2749876290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cc372f0 0x56521cbea8d0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f0da800d9f0 tx=0x7f0da8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b67b01a9b29df6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:50.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0db5600640 1 -- 172.21.15.94:0/2749876290 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8024070 con 0x56521cc372f0 2024-09-10T23:40:50.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0db5600640 1 -- 172.21.15.94:0/2749876290 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0da8014db0 con 0x56521cc372f0 2024-09-10T23:40:50.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.959+0000 7f0db5600640 1 -- 172.21.15.94:0/2749876290 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da801f660 con 0x56521cc372f0 2024-09-10T23:40:50.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 -- 172.21.15.94:0/2749876290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cc372f0 msgr2=0x56521cbea8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:50.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 --2- 172.21.15.94:0/2749876290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cc372f0 0x56521cbea8d0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f0da800d9f0 tx=0x7f0da8010000 comp rx=0 tx=0).stop 2024-09-10T23:40:50.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 -- 172.21.15.94:0/2749876290 shutdown_connections 2024-09-10T23:40:50.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 --2- 172.21.15.94:0/2749876290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cbeae10 0x56521cbeb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 --2- 172.21.15.94:0/2749876290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cc372f0 0x56521cbea8d0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 --2- 172.21.15.94:0/2749876290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 0x56521cc36db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 -- 172.21.15.94:0/2749876290 >> 172.21.15.94:0/2749876290 conn(0x56521cad4130 msgr2=0x56521cc19e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:50.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 -- 172.21.15.94:0/2749876290 shutdown_connections 2024-09-10T23:40:50.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 -- 172.21.15.94:0/2749876290 wait complete. 2024-09-10T23:40:50.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 -- start start 2024-09-10T23:40:50.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 Processor -- start 2024-09-10T23:40:50.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cbec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 0x56521cbec780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cc372f0 0x56521cbeccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56521c9d25d0 con 0x56521cbeae10 2024-09-10T23:40:50.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56521ca3bd60 con 0x56521cc369d0 2024-09-10T23:40:50.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x56521c9de500 con 0x56521cc372f0 2024-09-10T23:40:50.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 10 client.0 _ll_register_callbacks cb 0x56521c9e2810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:40:50.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dc4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cbec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dc4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cbec240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59450/0 (socket says 172.21.15.94:59450) 2024-09-10T23:40:50.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dc4a00640 1 -- 172.21.15.94:0/1172267099 learned_addr learned my addr 172.21.15.94:0/1172267099 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:50.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dc5400640 1 --2- 172.21.15.94:0/1172267099 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cc372f0 0x56521cbeccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0db7e00640 1 --2- 172.21.15.94:0/1172267099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 0x56521cbec780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dc4a00640 1 -- 172.21.15.94:0/1172267099 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cc372f0 msgr2=0x56521cbeccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:40:50.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dc4a00640 1 --2- 172.21.15.94:0/1172267099 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cc372f0 0x56521cbeccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dc4a00640 1 -- 172.21.15.94:0/1172267099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 msgr2=0x56521cbec780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:40:50.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dc4a00640 1 --2- 172.21.15.94:0/1172267099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 0x56521cbec780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dc4a00640 1 -- 172.21.15.94:0/1172267099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56521c9d6170 con 0x56521cbeae10 2024-09-10T23:40:50.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dc4a00640 1 --2- 172.21.15.94:0/1172267099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cbec240 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f0db00f76f0 tx=0x7f0db00f71d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30499c23a06109b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:50.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dae000640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:40:50.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dae000640 1 -- 172.21.15.94:0/1172267099 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0100c60 con 0x56521cbeae10 2024-09-10T23:40:50.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dae000640 1 -- 172.21.15.94:0/1172267099 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f0db00f6c50 con 0x56521cbeae10 2024-09-10T23:40:50.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 -- 172.21.15.94:0/1172267099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56521c9d00f0 con 0x56521cbeae10 2024-09-10T23:40:50.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dae000640 1 -- 172.21.15.94:0/1172267099 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7f0db00f79f0 con 0x56521cbeae10 2024-09-10T23:40:50.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 10 client.9303 fetch_fsmap learned FSMap version 116 2024-09-10T23:40:50.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.963+0000 7f0dca75c000 1 -- 172.21.15.94:0/1172267099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) -- 0x56521cabcef0 con 0x56521cbeae10 2024-09-10T23:40:50.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dae000640 1 -- 172.21.15.94:0/1172267099 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56521c9d00f0 con 0x56521cbeae10 2024-09-10T23:40:50.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dae000640 1 -- 172.21.15.94:0/1172267099 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 116) ==== 3502+0+0 (secure 0 0 0) 0x7f0db00f3340 con 0x56521cbeae10 2024-09-10T23:40:50.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 10 client.9303 fetch_fsmap finished waiting for FSMap version 116 2024-09-10T23:40:50.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 10 client.9303 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T23:40:50.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 20 client.9303 populate_metadata read hostname 'smithi094' 2024-09-10T23:40:50.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 1 --2- 172.21.15.94:0/1172267099 >> [v2:172.21.15.94:6833/3607539278,v1:172.21.15.94:6835/3607539278] conn(0x56521cc45c70 0x56521cc48060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 4 client.9303 mds_command: new command op to 9171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T23:40:50.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 1 -- 172.21.15.94:0/1172267099 --> [v2:172.21.15.94:6833/3607539278,v1:172.21.15.94:6835/3607539278] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56521ca4aa70 con 0x56521cc45c70 2024-09-10T23:40:50.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0db7e00640 1 --2- 172.21.15.94:0/1172267099 >> [v2:172.21.15.94:6833/3607539278,v1:172.21.15.94:6835/3607539278] conn(0x56521cc45c70 0x56521cc48060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0db7e00640 1 --2- 172.21.15.94:0/1172267099 >> [v2:172.21.15.94:6833/3607539278,v1:172.21.15.94:6835/3607539278] conn(0x56521cc45c70 0x56521cc48060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fcca74b2ac4c311 server_cookie=3f88bfdcd7b6515a in_seq=0 out_seq=0 2024-09-10T23:40:50.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dae000640 10 client.9303 ms_handle_connect on v2:172.21.15.94:6833/3607539278 2024-09-10T23:40:50.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dae000640 1 -- 172.21.15.94:0/1172267099 <== mds.0 v2:172.21.15.94:6833/3607539278 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56521ca4aa70 con 0x56521cc45c70 2024-09-10T23:40:50.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dae000640 10 client.9303 handle_command_reply: tid=0 2024-09-10T23:40:50.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 1 client.9303 shutdown 2024-09-10T23:40:50.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 1 -- 172.21.15.94:0/1172267099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 msgr2=0x56521cbec240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:50.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 1 --2- 172.21.15.94:0/1172267099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cbec240 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f0db00f76f0 tx=0x7f0db00f71d0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 1 -- 172.21.15.94:0/1172267099 shutdown_connections 2024-09-10T23:40:50.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 1 --2- 172.21.15.94:0/1172267099 >> [v2:172.21.15.94:6833/3607539278,v1:172.21.15.94:6835/3607539278] conn(0x56521cc45c70 0x56521cc48060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 1 --2- 172.21.15.94:0/1172267099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 0x56521cbec780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 1 --2- 172.21.15.94:0/1172267099 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cc372f0 0x56521cbeccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dae000640 0 client.9303 ms_handle_reset on v2:172.21.15.94:6833/3607539278 2024-09-10T23:40:50.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 1 --2- 172.21.15.94:0/1172267099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cbec240 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 1 -- 172.21.15.94:0/1172267099 >> 172.21.15.94:0/1172267099 conn(0x56521cad4130 msgr2=0x56521cbe9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:50.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 1 -- 172.21.15.94:0/1172267099 shutdown_connections 2024-09-10T23:40:50.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 1 -- 172.21.15.94:0/1172267099 wait complete. 2024-09-10T23:40:50.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.967+0000 7f0dca75c000 20 client.9303 trim_cache size 0 max 16384 2024-09-10T23:40:50.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 Processor -- start 2024-09-10T23:40:50.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 -- start start 2024-09-10T23:40:50.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cca8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 0x56521cca9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cc372f0 0x56521cca9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01f9f70 con 0x56521cbeae10 2024-09-10T23:40:50.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01fa5c0 con 0x56521cc369d0 2024-09-10T23:40:50.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x56521ca40390 con 0x56521cc372f0 2024-09-10T23:40:50.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dc4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cca8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dc4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cca8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59452/0 (socket says 172.21.15.94:59452) 2024-09-10T23:40:50.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dc4a00640 1 -- 172.21.15.94:0/1657247377 learned_addr learned my addr 172.21.15.94:0/1657247377 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:50.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dc5400640 1 --2- 172.21.15.94:0/1657247377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cc372f0 0x56521cca9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0db7e00640 1 --2- 172.21.15.94:0/1657247377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 0x56521cca9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dc4a00640 1 -- 172.21.15.94:0/1657247377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cc372f0 msgr2=0x56521cca9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:40:50.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dc4a00640 1 --2- 172.21.15.94:0/1657247377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cc372f0 0x56521cca9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dc4a00640 1 -- 172.21.15.94:0/1657247377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 msgr2=0x56521cca9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T23:40:50.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dc4a00640 1 --2- 172.21.15.94:0/1657247377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 0x56521cca9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dc4a00640 1 -- 172.21.15.94:0/1657247377 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56521ca93d90 con 0x56521cbeae10 2024-09-10T23:40:50.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dc4a00640 1 --2- 172.21.15.94:0/1657247377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cca8c00 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f0db001c650 tx=0x7f0db0005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555f9f369c1e476c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:50.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0db4c00640 1 -- 172.21.15.94:0/1657247377 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0122070 con 0x56521cbeae10 2024-09-10T23:40:50.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0db4c00640 1 -- 172.21.15.94:0/1657247377 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0db00044d0 con 0x56521cbeae10 2024-09-10T23:40:50.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0db4c00640 1 -- 172.21.15.94:0/1657247377 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db00f7340 con 0x56521cbeae10 2024-09-10T23:40:50.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 -- 172.21.15.94:0/1657247377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 msgr2=0x56521cca8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:50.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 --2- 172.21.15.94:0/1657247377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cca8c00 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f0db001c650 tx=0x7f0db0005f20 comp rx=0 tx=0).stop 2024-09-10T23:40:50.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 -- 172.21.15.94:0/1657247377 shutdown_connections 2024-09-10T23:40:50.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 --2- 172.21.15.94:0/1657247377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 0x56521cca9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 --2- 172.21.15.94:0/1657247377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cc372f0 0x56521cca9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 --2- 172.21.15.94:0/1657247377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cca8c00 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 -- 172.21.15.94:0/1657247377 >> 172.21.15.94:0/1657247377 conn(0x56521cad4130 msgr2=0x56521cc4de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:50.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 -- 172.21.15.94:0/1657247377 shutdown_connections 2024-09-10T23:40:50.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.987+0000 7f0dca75c000 1 -- 172.21.15.94:0/1657247377 wait complete. 2024-09-10T23:40:50.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 1 -- start start 2024-09-10T23:40:50.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 1 Processor -- start 2024-09-10T23:40:50.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cc4d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 0x56521cc91a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cc372f0 0x56521cc91f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56521ca3b070 con 0x56521cbeae10 2024-09-10T23:40:50.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56521cabca60 con 0x56521cc369d0 2024-09-10T23:40:50.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x56521ca478f0 con 0x56521cc372f0 2024-09-10T23:40:50.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 10 client.0 _ll_register_callbacks cb 0x7f0db00f77c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T23:40:50.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dc4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cc4d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dc4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cc4d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59464/0 (socket says 172.21.15.94:59464) 2024-09-10T23:40:50.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dc4a00640 1 -- 172.21.15.94:0/1072353211 learned_addr learned my addr 172.21.15.94:0/1072353211 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:50.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dc4a00640 1 -- 172.21.15.94:0/1072353211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cc372f0 msgr2=0x56521cc91f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:40:50.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dc4a00640 1 --2- 172.21.15.94:0/1072353211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cc372f0 0x56521cc91f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dc4a00640 1 -- 172.21.15.94:0/1072353211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 msgr2=0x56521cc91a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T23:40:50.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dc4a00640 1 --2- 172.21.15.94:0/1072353211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 0x56521cc91a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dc4a00640 1 -- 172.21.15.94:0/1072353211 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56521c9fea90 con 0x56521cbeae10 2024-09-10T23:40:50.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dc4a00640 1 --2- 172.21.15.94:0/1072353211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cc4d160 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f0db00f76f0 tx=0x7f0db0100c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f879452be2158d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:50.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dae000640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-09-10T23:40:50.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dae000640 1 -- 172.21.15.94:0/1072353211 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0122050 con 0x56521cbeae10 2024-09-10T23:40:50.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dae000640 1 -- 172.21.15.94:0/1072353211 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f0db010ebb0 con 0x56521cbeae10 2024-09-10T23:40:50.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 1 -- 172.21.15.94:0/1072353211 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56521ca46520 con 0x56521cbeae10 2024-09-10T23:40:50.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dae000640 1 -- 172.21.15.94:0/1072353211 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7f0db00f3890 con 0x56521cbeae10 2024-09-10T23:40:50.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 10 client.9309 fetch_fsmap learned FSMap version 116 2024-09-10T23:40:50.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 1 -- 172.21.15.94:0/1072353211 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) -- 0x56521c9dd050 con 0x56521cbeae10 2024-09-10T23:40:50.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dae000640 1 -- 172.21.15.94:0/1072353211 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56521ca46520 con 0x56521cbeae10 2024-09-10T23:40:50.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dae000640 1 -- 172.21.15.94:0/1072353211 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 116) ==== 3502+0+0 (secure 0 0 0) 0x7f0db00f66c0 con 0x56521cbeae10 2024-09-10T23:40:50.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 10 client.9309 fetch_fsmap finished waiting for FSMap version 116 2024-09-10T23:40:50.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 10 client.9309 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T23:40:50.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 20 client.9309 populate_metadata read hostname 'smithi094' 2024-09-10T23:40:50.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 1 --2- 172.21.15.94:0/1072353211 >> [v2:172.21.15.94:6833/3607539278,v1:172.21.15.94:6835/3607539278] conn(0x56521cc924a0 0x56521ccace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:50.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 4 client.9309 mds_command: new command op to 9171 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T23:40:50.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 1 -- 172.21.15.94:0/1072353211 --> [v2:172.21.15.94:6833/3607539278,v1:172.21.15.94:6835/3607539278] -- command(tid 0: {"prefix": "status"}) -- 0x56521ca43950 con 0x56521cc924a0 2024-09-10T23:40:50.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0db7e00640 1 --2- 172.21.15.94:0/1072353211 >> [v2:172.21.15.94:6833/3607539278,v1:172.21.15.94:6835/3607539278] conn(0x56521cc924a0 0x56521ccace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:50.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dae000640 10 client.9309 ms_handle_connect on v2:172.21.15.94:6833/3607539278 2024-09-10T23:40:50.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0db7e00640 1 --2- 172.21.15.94:0/1072353211 >> [v2:172.21.15.94:6833/3607539278,v1:172.21.15.94:6835/3607539278] conn(0x56521cc924a0 0x56521ccace50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cc5c00de2720365 server_cookie=5387865ffef15407 in_seq=0 out_seq=0 2024-09-10T23:40:50.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dae000640 1 -- 172.21.15.94:0/1072353211 <== mds.0 v2:172.21.15.94:6833/3607539278 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x56521ca43950 con 0x56521cc924a0 2024-09-10T23:40:50.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dae000640 10 client.9309 handle_command_reply: tid=0 2024-09-10T23:40:50.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 1 client.9309 shutdown 2024-09-10T23:40:50.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 1 -- 172.21.15.94:0/1072353211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 msgr2=0x56521cc4d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:50.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.991+0000 7f0dca75c000 1 --2- 172.21.15.94:0/1072353211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cc4d160 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f0db00f76f0 tx=0x7f0db0100c90 comp rx=0 tx=0).stop 2024-09-10T23:40:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dca75c000 1 -- 172.21.15.94:0/1072353211 shutdown_connections 2024-09-10T23:40:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dca75c000 1 --2- 172.21.15.94:0/1072353211 >> [v2:172.21.15.94:6833/3607539278,v1:172.21.15.94:6835/3607539278] conn(0x56521cc924a0 0x56521ccace50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dca75c000 1 --2- 172.21.15.94:0/1072353211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56521cc369d0 0x56521cc91a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dca75c000 1 --2- 172.21.15.94:0/1072353211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56521cc372f0 0x56521cc91f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dae000640 0 client.9309 ms_handle_reset on v2:172.21.15.94:6833/3607539278 2024-09-10T23:40:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dca75c000 1 --2- 172.21.15.94:0/1072353211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56521cbeae10 0x56521cc4d160 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dca75c000 1 -- 172.21.15.94:0/1072353211 >> 172.21.15.94:0/1072353211 conn(0x56521cad4130 msgr2=0x56521cc48060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dca75c000 1 -- 172.21.15.94:0/1072353211 shutdown_connections 2024-09-10T23:40:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dca75c000 1 -- 172.21.15.94:0/1072353211 wait complete. 2024-09-10T23:40:50.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dca75c000 20 client.9309 trim_cache size 0 max 16384 2024-09-10T23:40:50.998 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:40:50.998 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "1798cdb9-9937-4918-a569-86b41a92817e", 2024-09-10T23:40:50.998 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-09-10T23:40:50.998 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 9171, 2024-09-10T23:40:50.998 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-09-10T23:40:50.998 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-09-10T23:40:50.998 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-09-10T23:40:50.998 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 5.9211116869999998, 2024-09-10T23:40:50.998 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 116, 2024-09-10T23:40:50.998 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 162, 2024-09-10T23:40:50.998 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 162, 2024-09-10T23:40:50.998 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 10.414879244 2024-09-10T23:40:50.998 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:40:50.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dc7400640 1 -- 172.21.15.94:0/368187945 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0d9c073030 msgr2=0x7f0d9c0754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:50.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dc7400640 1 --2- 172.21.15.94:0/368187945 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0d9c073030 0x7f0d9c0754f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0da8001230 tx=0x7f0da8009040 comp rx=0 tx=0).stop 2024-09-10T23:40:50.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dc7400640 1 -- 172.21.15.94:0/368187945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dc00c49b0 msgr2=0x7f0dc015dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:50.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dc7400640 1 --2- 172.21.15.94:0/368187945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dc00c49b0 0x7f0dc015dba0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f0db00b0040 tx=0x7f0db0005db0 comp rx=0 tx=0).stop 2024-09-10T23:40:50.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dc7400640 1 -- 172.21.15.94:0/368187945 shutdown_connections 2024-09-10T23:40:51.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dc7400640 1 --2- 172.21.15.94:0/368187945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dc00cb4d0 0x7f0dc015e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:51.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dc7400640 1 --2- 172.21.15.94:0/368187945 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f0d9c073030 0x7f0d9c0754f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:51.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dc7400640 1 --2- 172.21.15.94:0/368187945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dc0158790 0x7f0dc015c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:51.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dc7400640 1 --2- 172.21.15.94:0/368187945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0dc00c49b0 0x7f0dc015dba0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:51.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dc7400640 1 -- 172.21.15.94:0/368187945 >> 172.21.15.94:0/368187945 conn(0x7f0dc00cfaf0 msgr2=0x7f0dc00aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:51.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dc7400640 1 -- 172.21.15.94:0/368187945 shutdown_connections 2024-09-10T23:40:51.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:50.995+0000 7f0dc7400640 1 -- 172.21.15.94:0/368187945 wait complete. 2024-09-10T23:40:51.013 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:40:51.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.159+0000 7f6461e00640 1 -- 172.21.15.94:0/1755972373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f645c0d2160 msgr2=0x7f645c166850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:51.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.159+0000 7f6461e00640 1 --2- 172.21.15.94:0/1755972373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f645c0d2160 0x7f645c166850 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6454009bc0 tx=0x7f645401cac0 comp rx=0 tx=0).stop 2024-09-10T23:40:51.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.159+0000 7f6461e00640 1 -- 172.21.15.94:0/1755972373 shutdown_connections 2024-09-10T23:40:51.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.159+0000 7f6461e00640 1 --2- 172.21.15.94:0/1755972373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f645c0d2160 0x7f645c166850 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:51.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.159+0000 7f6461e00640 1 -- 172.21.15.94:0/1755972373 >> 172.21.15.94:0/1755972373 conn(0x7f645c0749c0 msgr2=0x7f645c09aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:51.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.159+0000 7f6461e00640 1 -- 172.21.15.94:0/1755972373 shutdown_connections 2024-09-10T23:40:51.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.159+0000 7f6461e00640 1 -- 172.21.15.94:0/1755972373 wait complete. 2024-09-10T23:40:51.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6461e00640 1 Processor -- start 2024-09-10T23:40:51.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6461e00640 1 -- start start 2024-09-10T23:40:51.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6461e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f645c0b1040 0x7f645c0b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:51.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6461e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c0d2160 0x7f645c1fbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:51.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6461e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f645c15bc20 0x7f645c159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:51.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6461e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f645c16a480 con 0x7f645c0b1040 2024-09-10T23:40:51.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6461e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f645c16a300 con 0x7f645c15bc20 2024-09-10T23:40:51.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6461e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f645c16a600 con 0x7f645c0d2160 2024-09-10T23:40:51.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6460a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f645c0b1040 0x7f645c0b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:51.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6460a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f645c0b1040 0x7f645c0b50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59472/0 (socket says 172.21.15.94:59472) 2024-09-10T23:40:51.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6460a00640 1 -- 172.21.15.94:0/713312970 learned_addr learned my addr 172.21.15.94:0/713312970 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:51.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6453e00640 1 --2- 172.21.15.94:0/713312970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c0d2160 0x7f645c1fbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:51.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6461400640 1 --2- 172.21.15.94:0/713312970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f645c15bc20 0x7f645c159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:51.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6460a00640 1 -- 172.21.15.94:0/713312970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c0d2160 msgr2=0x7f645c1fbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:51.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6460a00640 1 --2- 172.21.15.94:0/713312970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c0d2160 0x7f645c1fbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:51.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6460a00640 1 -- 172.21.15.94:0/713312970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f645c15bc20 msgr2=0x7f645c159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:51.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6460a00640 1 --2- 172.21.15.94:0/713312970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f645c15bc20 0x7f645c159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:51.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6460a00640 1 -- 172.21.15.94:0/713312970 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c0cba80 con 0x7f645c0b1040 2024-09-10T23:40:51.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6460a00640 1 --2- 172.21.15.94:0/713312970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f645c0b1040 0x7f645c0b50f0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f644c00c400 tx=0x7f644c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:51.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6459600640 1 -- 172.21.15.94:0/713312970 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c017070 con 0x7f645c0b1040 2024-09-10T23:40:51.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6459600640 1 -- 172.21.15.94:0/713312970 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f644c005400 con 0x7f645c0b1040 2024-09-10T23:40:51.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6461e00640 1 -- 172.21.15.94:0/713312970 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c0affb0 con 0x7f645c0b1040 2024-09-10T23:40:51.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6459600640 1 -- 172.21.15.94:0/713312970 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c003860 con 0x7f645c0b1040 2024-09-10T23:40:51.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6461e00640 1 -- 172.21.15.94:0/713312970 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c0b0570 con 0x7f645c0b1040 2024-09-10T23:40:51.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.163+0000 7f6461e00640 1 -- 172.21.15.94:0/713312970 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645c0d1a50 con 0x7f645c0b1040 2024-09-10T23:40:51.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.167+0000 7f6459600640 1 -- 172.21.15.94:0/713312970 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f644c003a00 con 0x7f645c0b1040 2024-09-10T23:40:51.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.167+0000 7f6459600640 1 --2- 172.21.15.94:0/713312970 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6438073000 0x7f64380754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:51.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.167+0000 7f6459600640 1 -- 172.21.15.94:0/713312970 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f644c0913d0 con 0x7f645c0b1040 2024-09-10T23:40:51.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.167+0000 7f6453e00640 1 --2- 172.21.15.94:0/713312970 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6438073000 0x7f64380754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:51.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.167+0000 7f6453e00640 1 --2- 172.21.15.94:0/713312970 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6438073000 0x7f64380754c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6444000c10 tx=0x7f6444009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:51.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.171+0000 7f6459600640 1 -- 172.21.15.94:0/713312970 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f644c098070 con 0x7f645c0b1040 2024-09-10T23:40:51.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.371+0000 7f6461e00640 1 -- 172.21.15.94:0/713312970 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f645c15b570 con 0x7f645c0b1040 2024-09-10T23:40:51.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.371+0000 7f6459600640 1 -- 172.21.15.94:0/713312970 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v116) ==== 53+0+73 (secure 0 0 0) 0x7f644c05dc10 con 0x7f645c0b1040 2024-09-10T23:40:51.371 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:40:51.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.375+0000 7f6461e00640 1 -- 172.21.15.94:0/713312970 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6438073000 msgr2=0x7f64380754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:51.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.375+0000 7f6461e00640 1 --2- 172.21.15.94:0/713312970 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6438073000 0x7f64380754c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6444000c10 tx=0x7f6444009040 comp rx=0 tx=0).stop 2024-09-10T23:40:51.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.375+0000 7f6461e00640 1 -- 172.21.15.94:0/713312970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f645c0b1040 msgr2=0x7f645c0b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:51.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.375+0000 7f6461e00640 1 --2- 172.21.15.94:0/713312970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f645c0b1040 0x7f645c0b50f0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f644c00c400 tx=0x7f644c00c8c0 comp rx=0 tx=0).stop 2024-09-10T23:40:51.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.375+0000 7f6461e00640 1 -- 172.21.15.94:0/713312970 shutdown_connections 2024-09-10T23:40:51.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.375+0000 7f6461e00640 1 --2- 172.21.15.94:0/713312970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f645c15bc20 0x7f645c159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:51.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.375+0000 7f6461e00640 1 --2- 172.21.15.94:0/713312970 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6438073000 0x7f64380754c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:51.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.375+0000 7f6461e00640 1 --2- 172.21.15.94:0/713312970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f645c0d2160 0x7f645c1fbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:51.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.375+0000 7f6461e00640 1 --2- 172.21.15.94:0/713312970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f645c0b1040 0x7f645c0b50f0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:51.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.375+0000 7f6461e00640 1 -- 172.21.15.94:0/713312970 >> 172.21.15.94:0/713312970 conn(0x7f645c0749c0 msgr2=0x7f645c0ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:51.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.375+0000 7f6461e00640 1 -- 172.21.15.94:0/713312970 shutdown_connections 2024-09-10T23:40:51.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51.375+0000 7f6461e00640 1 -- 172.21.15.94:0/713312970 wait complete. 2024-09-10T23:40:51.392 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:40:51.392 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:40:51.392 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T23:40:51.392 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T23:40:51.392 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T23:40:51.392 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:40:51.392 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:40:51.392 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:40:51.392 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:40:51.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:51.393 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T23:40:51.402 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:40:51.402 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:40:51.402 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:40:51.402 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:51.402 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T23:40:51.402 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:51.402 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:40:51.402 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:51.402 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T23:40:51.402 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:40:51.402 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:51.402 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T23:40:51.403 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:40:51.403 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:51.403 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:40:51.403 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:51.403 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:40:51.403 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 41960sec preferred_lft 41960sec 2024-09-10T23:40:51.403 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-09-10T23:40:51.403 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:51.403 INFO:teuthology.orchestra.run.smithi094.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:40:51.403 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 02:42:ed:7d:46:db brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:51.403 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:40:51.403 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:51.403 INFO:teuthology.orchestra.run.smithi094.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:40:51.403 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:51.403 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:40:51.404 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:51.404 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:40:51.404 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:51.404 INFO:teuthology.orchestra.run.smithi094.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:40:51.404 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:40:51.404 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:40:51.404 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:51.404 INFO:teuthology.orchestra.run.smithi094.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:40:51.404 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:40:51.404 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:40:51.404 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:51.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:51.405 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:40:51.466 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T23:40:51.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:51.468 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T23:40:51.529 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T23:40:51.529 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:40:51.529 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:40:51.529 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:40:51.530 INFO:teuthology.orchestra.run.smithi094.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:40:51.530 INFO:teuthology.orchestra.run.smithi094.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:40:51.530 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:40:51.530 INFO:teuthology.orchestra.run.smithi094.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:40:51.530 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:40:51.530 INFO:teuthology.orchestra.run.smithi094.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:40:51.530 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:40:51.530 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:40:51.530 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T23:40:51.530 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:40:51.530 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:40:51.530 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:40:51.531 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T23:40:51.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:40:51.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:40:51.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:51.669 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T23:40:51.678 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T23:40:51.678 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T23:40:51.679 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:40:51.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:51.679 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-09-10T23:40:51.740 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T23:40:51.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:40:51.740 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T23:40:51.789 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T23:40:51.789 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:40:51.789 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:40:51.842 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:40:51.904 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:40:51.954 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:40:51.954 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:40:51.954 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:40:51.954 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:40:51.954 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:40:51.954 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:40:51.954 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:40:51.954 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:40:51.954 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:40:51.954 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:40:51.955 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:40:51.955 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:40:51.955 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:40:51.955 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:40:51.955 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:40:51.955 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:40:51.955 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:40:51.955 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:40:51.955 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:40:51.955 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:40:51.955 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:40:51.956 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:40:51.956 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:40:51.956 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:40:51.956 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:40:51.956 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:40:51.956 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:40:51.956 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:40:51.956 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:40:51.956 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:40:51.956 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:40:51.956 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:40:51.956 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:40:51.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:51.957 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:40:52.012 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:40:52.013 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T23:40:52.059 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-09-10T23:40:52.081 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:40:52.138 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-09-10T23:40:52.138 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-09-10T23:40:52.138 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-09-10T23:40:52.138 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-09-10T23:40:52.138 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-09-10T23:40:52.138 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-09-10T23:40:52.138 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-09-10T23:40:52.138 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-09-10T23:40:52.138 INFO:teuthology.orchestra.run.smithi094.stdout:/run/lock 2024-09-10T23:40:52.138 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-09-10T23:40:52.138 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-09-10T23:40:52.139 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-09-10T23:40:52.139 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:40:52.139 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-09-10T23:40:52.139 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-09-10T23:40:52.139 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-09-10T23:40:52.139 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-09-10T23:40:52.139 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/fs/nfsd 2024-09-10T23:40:52.139 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-09-10T23:40:52.139 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-09-10T23:40:52.139 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:40:52.139 INFO:teuthology.orchestra.run.smithi094.stdout:/run/rpc_pipefs 2024-09-10T23:40:52.139 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-09-10T23:40:52.139 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-09-10T23:40:52.139 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T23:40:52.140 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T23:40:52.140 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T23:40:52.140 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T23:40:52.140 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-09-10T23:40:52.140 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:40:52.140 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:40:52.140 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:40:52.140 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:40:52.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:52.141 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-09-10T23:40:52.155 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:2024-09-10T23:40:52.155+0000 7f9ec9a4c580 -1 init, newargv = 0x562766fd1fc0 newargc=15 2024-09-10T23:40:52.155 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[53600]: starting ceph client 2024-09-10T23:40:52.162 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[53600]: starting fuse 2024-09-10T23:40:52.164 INFO:teuthology.orchestra.run.smithi094.stdout:44 2024-09-10T23:40:52.164 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:40:52.164 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:40:52.164 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-09-10T23:40:52.165 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-09-10T23:40:52.165 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-09-10T23:40:52.165 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-09-10T23:40:52.165 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:40:52.165 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-09-10T23:40:52.165 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T23:40:52.165 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-09-10T23:40:52.165 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T23:40:52.165 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:40:52.165 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-09-10T23:40:52.165 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-09-10T23:40:52.165 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-09-10T23:40:52.165 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-09-10T23:40:52.165 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:40:52.165 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-09-10T23:40:52.166 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:40:52.166 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:40:52.166 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:40:52.166 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-09-10T23:40:52.166 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-09-10T23:40:52.166 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-09-10T23:40:52.166 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:40:52.166 DEBUG:teuthology.orchestra.run.smithi094:> 2024-09-10T23:40:52.166 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.0")) 2024-09-10T23:40:52.166 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-09-10T23:40:52.288 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.0.53600.asok 2024-09-10T23:40:52.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:40:52.300 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53600.asok 2024-09-10T23:40:52.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:52.300 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53600.asok status 2024-09-10T23:40:52.468 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:40:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-09-10T23:40:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:40:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:40:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "0", 2024-09-10T23:40:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-09-10T23:40:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T23:40:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "53600", 2024-09-10T23:40:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-09-10T23:40:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-09-10T23:40:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-09-10T23:40:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 9321, 2024-09-10T23:40:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-09-10T23:40:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-09-10T23:40:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-09-10T23:40:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 9321 2024-09-10T23:40:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:40:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:40:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:40:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 525808977 2024-09-10T23:40:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-09-10T23:40:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-10T23:40:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:40:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 525808977 2024-09-10T23:40:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-10T23:40:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.9321 192.168.0.1:0/525808977", 2024-09-10T23:40:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "192.168.0.1:0/525808977", 2024-09-10T23:40:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-09-10T23:40:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 116, 2024-09-10T23:40:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 162, 2024-09-10T23:40:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:40:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-09-10T23:40:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-09-10T23:40:52.471 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:40:52.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:52.483 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-09-10T23:40:52.483 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T23:40:52.491 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-09-10T23:40:52.492 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T23:40:52.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:52.492 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T23:40:52.556 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T23:40:52.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d14e00640 1 -- 172.21.15.94:0/2263715266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d100d2290 msgr2=0x7f5d10166b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:52.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d14e00640 1 --2- 172.21.15.94:0/2263715266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d100d2290 0x7f5d10166b70 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f5d1015c9a0 tx=0x7f5d0801c8b0 comp rx=0 tx=0).stop 2024-09-10T23:40:52.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d14e00640 1 -- 172.21.15.94:0/2263715266 shutdown_connections 2024-09-10T23:40:52.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d14e00640 1 --2- 172.21.15.94:0/2263715266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d100d2290 0x7f5d10166b70 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:52.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d14e00640 1 -- 172.21.15.94:0/2263715266 >> 172.21.15.94:0/2263715266 conn(0x7f5d1009aa70 msgr2=0x7f5d1009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:52.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d14e00640 1 -- 172.21.15.94:0/2263715266 shutdown_connections 2024-09-10T23:40:52.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d14e00640 1 -- 172.21.15.94:0/2263715266 wait complete. 2024-09-10T23:40:52.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d14e00640 1 Processor -- start 2024-09-10T23:40:52.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d14e00640 1 -- start start 2024-09-10T23:40:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d14e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d100ae980 0x7f5d10157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d14e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d100d2290 0x7f5d10157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:52.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d14e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d1015bd30 0x7f5d1015e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d14e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1016a360 con 0x7f5d1015bd30 2024-09-10T23:40:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d14e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1016a1e0 con 0x7f5d100d2290 2024-09-10T23:40:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d14e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5d1016a4e0 con 0x7f5d100ae980 2024-09-10T23:40:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d0e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d1015bd30 0x7f5d1015e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d0e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d1015bd30 0x7f5d1015e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59482/0 (socket says 172.21.15.94:59482) 2024-09-10T23:40:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.739+0000 7f5d0e800640 1 -- 172.21.15.94:0/3510079402 learned_addr learned my addr 172.21.15.94:0/3510079402 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d0e800640 1 -- 172.21.15.94:0/3510079402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d100ae980 msgr2=0x7f5d10157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d0d400640 1 --2- 172.21.15.94:0/3510079402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d100d2290 0x7f5d10157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d0de00640 1 --2- 172.21.15.94:0/3510079402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d100ae980 0x7f5d10157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d0e800640 1 --2- 172.21.15.94:0/3510079402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d100ae980 0x7f5d10157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d0e800640 1 -- 172.21.15.94:0/3510079402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d100d2290 msgr2=0x7f5d10157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d0e800640 1 --2- 172.21.15.94:0/3510079402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d100d2290 0x7f5d10157e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d0e800640 1 -- 172.21.15.94:0/3510079402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d100b78a0 con 0x7f5d1015bd30 2024-09-10T23:40:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d0de00640 1 --2- 172.21.15.94:0/3510079402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d100ae980 0x7f5d10157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d0d400640 1 --2- 172.21.15.94:0/3510079402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d100d2290 0x7f5d10157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d0e800640 1 --2- 172.21.15.94:0/3510079402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d1015bd30 0x7f5d1015e150 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009850 tx=0x7f5d080a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d06000640 1 -- 172.21.15.94:0/3510079402 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d080b2040 con 0x7f5d1015bd30 2024-09-10T23:40:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d06000640 1 -- 172.21.15.94:0/3510079402 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d08003970 con 0x7f5d1015bd30 2024-09-10T23:40:52.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d06000640 1 -- 172.21.15.94:0/3510079402 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08003dc0 con 0x7f5d1015bd30 2024-09-10T23:40:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d14e00640 1 -- 172.21.15.94:0/3510079402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d101fbd20 con 0x7f5d1015bd30 2024-09-10T23:40:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d14e00640 1 -- 172.21.15.94:0/3510079402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d101fb240 con 0x7f5d1015bd30 2024-09-10T23:40:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d14e00640 1 -- 172.21.15.94:0/3510079402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d100d1b80 con 0x7f5d1015bd30 2024-09-10T23:40:52.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.743+0000 7f5d06000640 1 -- 172.21.15.94:0/3510079402 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f5d080ad440 con 0x7f5d1015bd30 2024-09-10T23:40:52.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.747+0000 7f5d06000640 1 --2- 172.21.15.94:0/3510079402 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5cf0073000 0x7f5cf00754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:52.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.747+0000 7f5d06000640 1 -- 172.21.15.94:0/3510079402 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f5d0812c180 con 0x7f5d1015bd30 2024-09-10T23:40:52.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.747+0000 7f5d0de00640 1 --2- 172.21.15.94:0/3510079402 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5cf0073000 0x7f5cf00754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:52.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.747+0000 7f5d0de00640 1 --2- 172.21.15.94:0/3510079402 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5cf0073000 0x7f5cf00754c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5d00007eb0 tx=0x7f5d000073c0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:52.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.751+0000 7f5d06000640 1 -- 172.21.15.94:0/3510079402 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5d080164d0 con 0x7f5d1015bd30 2024-09-10T23:40:52.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.955+0000 7f5d14e00640 1 -- 172.21.15.94:0/3510079402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5d100d28a0 con 0x7f5d1015bd30 2024-09-10T23:40:52.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.955+0000 7f5d06000640 1 -- 172.21.15.94:0/3510079402 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v116) ==== 53+0+73 (secure 0 0 0) 0x7f5d080f9010 con 0x7f5d1015bd30 2024-09-10T23:40:52.954 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T23:40:52.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.955+0000 7f5d14e00640 1 -- 172.21.15.94:0/3510079402 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5cf0073000 msgr2=0x7f5cf00754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.955+0000 7f5d14e00640 1 --2- 172.21.15.94:0/3510079402 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5cf0073000 0x7f5cf00754c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5d00007eb0 tx=0x7f5d000073c0 comp rx=0 tx=0).stop 2024-09-10T23:40:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.955+0000 7f5d14e00640 1 -- 172.21.15.94:0/3510079402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d1015bd30 msgr2=0x7f5d1015e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.955+0000 7f5d14e00640 1 --2- 172.21.15.94:0/3510079402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d1015bd30 0x7f5d1015e150 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009850 tx=0x7f5d080a5790 comp rx=0 tx=0).stop 2024-09-10T23:40:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.955+0000 7f5d14e00640 1 -- 172.21.15.94:0/3510079402 shutdown_connections 2024-09-10T23:40:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.955+0000 7f5d14e00640 1 --2- 172.21.15.94:0/3510079402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d100d2290 0x7f5d10157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.955+0000 7f5d14e00640 1 --2- 172.21.15.94:0/3510079402 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5cf0073000 0x7f5cf00754c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.955+0000 7f5d14e00640 1 --2- 172.21.15.94:0/3510079402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d1015bd30 0x7f5d1015e150 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.955+0000 7f5d14e00640 1 --2- 172.21.15.94:0/3510079402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d100ae980 0x7f5d10157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.955+0000 7f5d14e00640 1 -- 172.21.15.94:0/3510079402 >> 172.21.15.94:0/3510079402 conn(0x7f5d1009aa70 msgr2=0x7f5d10165b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.955+0000 7f5d14e00640 1 -- 172.21.15.94:0/3510079402 shutdown_connections 2024-09-10T23:40:52.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:52.955+0000 7f5d14e00640 1 -- 172.21.15.94:0/3510079402 wait complete. 2024-09-10T23:40:52.976 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T23:40:52.976 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T23:40:52.976 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2024-09-10T23:40:52.976 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T23:40:52.976 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T23:40:52.977 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T23:40:52.977 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T23:40:52.977 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T23:40:52.977 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:40:52.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:52.977 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2024-09-10T23:40:52.987 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T23:40:52.987 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T23:40:52.987 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T23:40:52.987 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:52.988 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2024-09-10T23:40:52.988 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:52.988 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:40:52.988 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:52.988 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2024-09-10T23:40:52.988 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:40:52.988 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:52.988 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2024-09-10T23:40:52.988 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T23:40:52.988 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:52.988 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T23:40:52.988 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:52.988 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-10T23:40:52.989 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 41962sec preferred_lft 41962sec 2024-09-10T23:40:52.989 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2024-09-10T23:40:52.989 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:52.989 INFO:teuthology.orchestra.run.smithi181.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T23:40:52.989 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 02:42:3b:2e:09:2d brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:52.989 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T23:40:52.989 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:52.989 INFO:teuthology.orchestra.run.smithi181.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T23:40:52.989 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-10T23:40:52.989 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T23:40:52.989 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:52.989 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-10T23:40:52.989 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:52.989 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:40:52.990 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:40:52.990 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-10T23:40:52.990 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:52.990 INFO:teuthology.orchestra.run.smithi181.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T23:40:52.990 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:40:52.990 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-10T23:40:52.990 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2024-09-10T23:40:52.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:52.991 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T23:40:53.056 INFO:teuthology.orchestra.run.smithi181.stdout:1 2024-09-10T23:40:53.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:53.058 DEBUG:teuthology.orchestra.run.smithi181:> route 2024-09-10T23:40:53.112 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2024-09-10T23:40:53.112 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T23:40:53.112 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-10T23:40:53.112 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T23:40:53.112 INFO:teuthology.orchestra.run.smithi181.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-10T23:40:53.113 INFO:teuthology.orchestra.run.smithi181.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T23:40:53.113 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T23:40:53.113 INFO:teuthology.orchestra.run.smithi181.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:40:53.113 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:40:53.113 INFO:teuthology.orchestra.run.smithi181.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-10T23:40:53.113 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T23:40:53.113 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T23:40:53.113 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2024-09-10T23:40:53.113 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T23:40:53.113 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T23:40:53.114 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T23:40:53.114 DEBUG:teuthology.orchestra.run.smithi181:> ') 2024-09-10T23:40:53.209 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:40:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T23:40:53.252 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:40:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:40:53.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:53.257 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-09-10T23:40:53.266 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T23:40:53.266 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T23:40:53.267 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:40:53.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:53.267 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2024-09-10T23:40:53.327 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T23:40:53.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:40:53.328 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T23:40:53.375 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T23:40:53.376 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T23:40:53.376 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:40:53.427 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:40:53.487 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:40:53.537 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:40:53.537 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:40:53.537 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:40:53.537 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:40:53.537 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:40:53.537 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:40:53.537 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:40:53.537 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:40:53.537 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:40:53.537 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:40:53.537 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:40:53.538 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:40:53.538 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:40:53.538 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:40:53.538 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:40:53.538 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:40:53.538 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:40:53.538 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:40:53.538 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:40:53.538 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:40:53.538 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:40:53.538 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:40:53.538 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:40:53.538 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:40:53.539 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:40:53.539 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:40:53.539 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:40:53.539 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:40:53.539 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:40:53.539 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:40:53.539 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:40:53.539 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:40:53.539 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:40:53.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:53.540 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:40:53.591 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T23:40:53.591 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T23:40:53.634 DEBUG:teuthology.orchestra.run.smithi181:> sudo modprobe fuse 2024-09-10T23:40:53.654 DEBUG:teuthology.orchestra.run.smithi181:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T23:40:53.710 INFO:teuthology.orchestra.run.smithi181.stdout:/sys 2024-09-10T23:40:53.710 INFO:teuthology.orchestra.run.smithi181.stdout:/proc 2024-09-10T23:40:53.710 INFO:teuthology.orchestra.run.smithi181.stdout:/dev 2024-09-10T23:40:53.710 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/pts 2024-09-10T23:40:53.710 INFO:teuthology.orchestra.run.smithi181.stdout:/run 2024-09-10T23:40:53.710 INFO:teuthology.orchestra.run.smithi181.stdout:/ 2024-09-10T23:40:53.710 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/security 2024-09-10T23:40:53.710 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/shm 2024-09-10T23:40:53.710 INFO:teuthology.orchestra.run.smithi181.stdout:/run/lock 2024-09-10T23:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/cgroup 2024-09-10T23:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/pstore 2024-09-10T23:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/bpf 2024-09-10T23:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T23:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/hugepages 2024-09-10T23:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout:/dev/mqueue 2024-09-10T23:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/debug 2024-09-10T23:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/tracing 2024-09-10T23:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout:/proc/fs/nfsd 2024-09-10T23:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/fs/fuse/connections 2024-09-10T23:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout:/sys/kernel/config 2024-09-10T23:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T23:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout:/run/rpc_pipefs 2024-09-10T23:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout:/run/user/1000 2024-09-10T23:40:53.712 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph 2024-09-10T23:40:53.712 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T23:40:53.712 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T23:40:53.712 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T23:40:53.712 INFO:teuthology.orchestra.run.smithi181.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T23:40:53.712 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns 2024-09-10T23:40:53.712 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:40:53.712 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:40:53.712 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:40:53.712 INFO:teuthology.orchestra.run.smithi181.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:40:53.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:53.713 DEBUG:teuthology.orchestra.run.smithi181:> ls /sys/fs/fuse/connections 2024-09-10T23:40:53.730 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:2024-09-10T23:40:53.727+0000 7fb1aa1fc580 -1 init, newargv = 0x55755fe1ee10 newargc=15 2024-09-10T23:40:53.730 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[36394]: starting ceph client 2024-09-10T23:40:53.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[36394]: starting fuse 2024-09-10T23:40:53.738 INFO:teuthology.orchestra.run.smithi181.stdout:44 2024-09-10T23:40:53.738 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T23:40:53.738 DEBUG:teuthology.orchestra.run.smithi181:> sudo stdin-killer -- python3 -c ' 2024-09-10T23:40:53.739 DEBUG:teuthology.orchestra.run.smithi181:> import glob 2024-09-10T23:40:53.739 DEBUG:teuthology.orchestra.run.smithi181:> import re 2024-09-10T23:40:53.739 DEBUG:teuthology.orchestra.run.smithi181:> import os 2024-09-10T23:40:53.739 DEBUG:teuthology.orchestra.run.smithi181:> import subprocess 2024-09-10T23:40:53.739 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:40:53.739 DEBUG:teuthology.orchestra.run.smithi181:> def _find_admin_socket(client_name): 2024-09-10T23:40:53.739 DEBUG:teuthology.orchestra.run.smithi181:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T23:40:53.739 DEBUG:teuthology.orchestra.run.smithi181:> files = glob.glob(asok_path) 2024-09-10T23:40:53.739 DEBUG:teuthology.orchestra.run.smithi181:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T23:40:53.739 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:40:53.739 DEBUG:teuthology.orchestra.run.smithi181:> # Given a non-glob path, it better be there 2024-09-10T23:40:53.739 DEBUG:teuthology.orchestra.run.smithi181:> if "*" not in asok_path: 2024-09-10T23:40:53.739 DEBUG:teuthology.orchestra.run.smithi181:> assert(len(files) == 1) 2024-09-10T23:40:53.739 DEBUG:teuthology.orchestra.run.smithi181:> return files[0] 2024-09-10T23:40:53.740 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:40:53.740 DEBUG:teuthology.orchestra.run.smithi181:> for f in files: 2024-09-10T23:40:53.740 DEBUG:teuthology.orchestra.run.smithi181:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T23:40:53.740 DEBUG:teuthology.orchestra.run.smithi181:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T23:40:53.740 DEBUG:teuthology.orchestra.run.smithi181:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T23:40:53.740 DEBUG:teuthology.orchestra.run.smithi181:> contents = proc_f.read() 2024-09-10T23:40:53.740 DEBUG:teuthology.orchestra.run.smithi181:> if mountpoint in contents: 2024-09-10T23:40:53.740 DEBUG:teuthology.orchestra.run.smithi181:> return f 2024-09-10T23:40:53.740 DEBUG:teuthology.orchestra.run.smithi181:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T23:40:53.740 DEBUG:teuthology.orchestra.run.smithi181:> 2024-09-10T23:40:53.740 DEBUG:teuthology.orchestra.run.smithi181:> print(_find_admin_socket("client.1")) 2024-09-10T23:40:53.740 DEBUG:teuthology.orchestra.run.smithi181:> ' 2024-09-10T23:40:53.859 INFO:teuthology.orchestra.run.smithi181.stdout:/var/run/ceph/ceph-client.1.36394.asok 2024-09-10T23:40:53.864 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-10T23:40:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:40:53.870 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36394.asok 2024-09-10T23:40:53.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:53.871 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36394.asok status 2024-09-10T23:40:54.042 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-10T23:40:54.042 INFO:teuthology.orchestra.run.smithi181.stdout: "metadata": { 2024-09-10T23:40:54.043 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T23:40:54.043 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:40:54.043 INFO:teuthology.orchestra.run.smithi181.stdout: "entity_id": "1", 2024-09-10T23:40:54.043 INFO:teuthology.orchestra.run.smithi181.stdout: "hostname": "smithi181", 2024-09-10T23:40:54.043 INFO:teuthology.orchestra.run.smithi181.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T23:40:54.043 INFO:teuthology.orchestra.run.smithi181.stdout: "pid": "36394", 2024-09-10T23:40:54.043 INFO:teuthology.orchestra.run.smithi181.stdout: "root": "/" 2024-09-10T23:40:54.043 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:40:54.043 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_count": 0, 2024-09-10T23:40:54.043 INFO:teuthology.orchestra.run.smithi181.stdout: "dentry_pinned_count": 0, 2024-09-10T23:40:54.043 INFO:teuthology.orchestra.run.smithi181.stdout: "id": 197462, 2024-09-10T23:40:54.043 INFO:teuthology.orchestra.run.smithi181.stdout: "inst": { 2024-09-10T23:40:54.043 INFO:teuthology.orchestra.run.smithi181.stdout: "name": { 2024-09-10T23:40:54.043 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "client", 2024-09-10T23:40:54.044 INFO:teuthology.orchestra.run.smithi181.stdout: "num": 197462 2024-09-10T23:40:54.044 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:40:54.044 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:40:54.044 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:40:54.044 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:40:54.044 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 2963342291 2024-09-10T23:40:54.044 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-10T23:40:54.044 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:40:54.044 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": { 2024-09-10T23:40:54.044 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "v1", 2024-09-10T23:40:54.044 INFO:teuthology.orchestra.run.smithi181.stdout: "addr": "192.168.0.1:0", 2024-09-10T23:40:54.044 INFO:teuthology.orchestra.run.smithi181.stdout: "nonce": 2963342291 2024-09-10T23:40:54.044 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-10T23:40:54.044 INFO:teuthology.orchestra.run.smithi181.stdout: "inst_str": "client.197462 192.168.0.1:0/2963342291", 2024-09-10T23:40:54.045 INFO:teuthology.orchestra.run.smithi181.stdout: "addr_str": "192.168.0.1:0/2963342291", 2024-09-10T23:40:54.045 INFO:teuthology.orchestra.run.smithi181.stdout: "inode_count": 1, 2024-09-10T23:40:54.045 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_epoch": 116, 2024-09-10T23:40:54.045 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch": 162, 2024-09-10T23:40:54.045 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_epoch_barrier": 0, 2024-09-10T23:40:54.045 INFO:teuthology.orchestra.run.smithi181.stdout: "blocklisted": false, 2024-09-10T23:40:54.045 INFO:teuthology.orchestra.run.smithi181.stdout: "fs_name": "cephfs" 2024-09-10T23:40:54.045 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-10T23:40:54.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:54.057 DEBUG:teuthology.orchestra.run.smithi181:> stat --file-system '--printf=%T 2024-09-10T23:40:54.057 DEBUG:teuthology.orchestra.run.smithi181:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T23:40:54.066 INFO:teuthology.orchestra.run.smithi181.stdout:fuseblk 2024-09-10T23:40:54.066 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T23:40:54.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:54.067 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T23:40:54.126 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-10T23:40:54.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.287+0000 7f38a7200640 1 -- 172.21.15.94:0/2953288715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38a80cfda0 msgr2=0x7f38a80d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:54.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.287+0000 7f38a7200640 1 --2- 172.21.15.94:0/2953288715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38a80cfda0 0x7f38a80d01e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3894009870 tx=0x7f389401c830 comp rx=0 tx=0).stop 2024-09-10T23:40:54.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.287+0000 7f38a7200640 1 -- 172.21.15.94:0/2953288715 shutdown_connections 2024-09-10T23:40:54.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.287+0000 7f38a7200640 1 --2- 172.21.15.94:0/2953288715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38a80cfda0 0x7f38a80d01e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:54.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.287+0000 7f38a7200640 1 -- 172.21.15.94:0/2953288715 >> 172.21.15.94:0/2953288715 conn(0x7f38a809aa70 msgr2=0x7f38a809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:54.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.287+0000 7f38a7200640 1 -- 172.21.15.94:0/2953288715 shutdown_connections 2024-09-10T23:40:54.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.287+0000 7f38a7200640 1 -- 172.21.15.94:0/2953288715 wait complete. 2024-09-10T23:40:54.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a7200640 1 Processor -- start 2024-09-10T23:40:54.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a7200640 1 -- start start 2024-09-10T23:40:54.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a7200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a80aee30 0x7f38a8151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:54.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a7200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38a80cfda0 0x7f38a815a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:54.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a7200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38a815ba30 0x7f38a815a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:54.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a7200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8169a50 con 0x7f38a80aee30 2024-09-10T23:40:54.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a7200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f38a81698d0 con 0x7f38a80cfda0 2024-09-10T23:40:54.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a7200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f38a8169bd0 con 0x7f38a815ba30 2024-09-10T23:40:54.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a80aee30 0x7f38a8151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:54.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a80aee30 0x7f38a8151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34156/0 (socket says 172.21.15.94:34156) 2024-09-10T23:40:54.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a5e00640 1 -- 172.21.15.94:0/1644359787 learned_addr learned my addr 172.21.15.94:0/1644359787 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:54.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a5400640 1 --2- 172.21.15.94:0/1644359787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38a80cfda0 0x7f38a815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:54.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a6800640 1 --2- 172.21.15.94:0/1644359787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38a815ba30 0x7f38a815a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:54.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a5400640 1 -- 172.21.15.94:0/1644359787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38a815ba30 msgr2=0x7f38a815a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:54.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a5400640 1 --2- 172.21.15.94:0/1644359787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38a815ba30 0x7f38a815a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:54.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a5400640 1 -- 172.21.15.94:0/1644359787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a80aee30 msgr2=0x7f38a8151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:54.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a5400640 1 --2- 172.21.15.94:0/1644359787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a80aee30 0x7f38a8151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:54.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a5400640 1 -- 172.21.15.94:0/1644359787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a81539d0 con 0x7f38a80cfda0 2024-09-10T23:40:54.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a5e00640 1 --2- 172.21.15.94:0/1644359787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a80aee30 0x7f38a8151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:40:54.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a5400640 1 --2- 172.21.15.94:0/1644359787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38a80cfda0 0x7f38a815a490 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f38940ac000 tx=0x7f38940a5770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:54.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f389e000640 1 -- 172.21.15.94:0/1644359787 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38940b2040 con 0x7f38a80cfda0 2024-09-10T23:40:54.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f389e000640 1 -- 172.21.15.94:0/1644359787 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38940043b0 con 0x7f38a80cfda0 2024-09-10T23:40:54.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f389e000640 1 -- 172.21.15.94:0/1644359787 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894005280 con 0x7f38a80cfda0 2024-09-10T23:40:54.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a7200640 1 -- 172.21.15.94:0/1644359787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a8153cb0 con 0x7f38a80cfda0 2024-09-10T23:40:54.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a7200640 1 -- 172.21.15.94:0/1644359787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a8154270 con 0x7f38a80cfda0 2024-09-10T23:40:54.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.291+0000 7f38a7200640 1 -- 172.21.15.94:0/1644359787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a80d01e0 con 0x7f38a80cfda0 2024-09-10T23:40:54.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.295+0000 7f389e000640 1 -- 172.21.15.94:0/1644359787 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f38940b5070 con 0x7f38a80cfda0 2024-09-10T23:40:54.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.295+0000 7f389e000640 1 --2- 172.21.15.94:0/1644359787 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3888073000 0x7f38880754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:54.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.295+0000 7f389e000640 1 -- 172.21.15.94:0/1644359787 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f389412b700 con 0x7f38a80cfda0 2024-09-10T23:40:54.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.295+0000 7f38a5e00640 1 --2- 172.21.15.94:0/1644359787 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3888073000 0x7f38880754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:54.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.295+0000 7f38a5e00640 1 --2- 172.21.15.94:0/1644359787 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3888073000 0x7f38880754c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3898007b40 tx=0x7f3898007a90 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:54.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.299+0000 7f389e000640 1 -- 172.21.15.94:0/1644359787 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3894016450 con 0x7f38a80cfda0 2024-09-10T23:40:54.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.495+0000 7f38a7200640 1 -- 172.21.15.94:0/1644359787 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f38a809aff0 con 0x7f38a80cfda0 2024-09-10T23:40:54.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.555+0000 7f389e000640 1 -- 172.21.15.94:0/1644359787 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v117) ==== 136+0+0 (secure 0 0 0) 0x7f38940f7fc0 con 0x7f38a80cfda0 2024-09-10T23:40:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.555+0000 7f38a7200640 1 -- 172.21.15.94:0/1644359787 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3888073000 msgr2=0x7f38880754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.555+0000 7f38a7200640 1 --2- 172.21.15.94:0/1644359787 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3888073000 0x7f38880754c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3898007b40 tx=0x7f3898007a90 comp rx=0 tx=0).stop 2024-09-10T23:40:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.555+0000 7f38a7200640 1 -- 172.21.15.94:0/1644359787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38a80cfda0 msgr2=0x7f38a815a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.555+0000 7f38a7200640 1 --2- 172.21.15.94:0/1644359787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38a80cfda0 0x7f38a815a490 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f38940ac000 tx=0x7f38940a5770 comp rx=0 tx=0).stop 2024-09-10T23:40:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.555+0000 7f38a7200640 1 -- 172.21.15.94:0/1644359787 shutdown_connections 2024-09-10T23:40:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.555+0000 7f38a7200640 1 --2- 172.21.15.94:0/1644359787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38a80cfda0 0x7f38a815a490 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.555+0000 7f38a7200640 1 --2- 172.21.15.94:0/1644359787 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f3888073000 0x7f38880754c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.555+0000 7f38a7200640 1 --2- 172.21.15.94:0/1644359787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38a815ba30 0x7f38a815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.555+0000 7f38a7200640 1 --2- 172.21.15.94:0/1644359787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38a80aee30 0x7f38a8151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.555+0000 7f38a7200640 1 -- 172.21.15.94:0/1644359787 >> 172.21.15.94:0/1644359787 conn(0x7f38a809aa70 msgr2=0x7f38a8166cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.555+0000 7f38a7200640 1 -- 172.21.15.94:0/1644359787 shutdown_connections 2024-09-10T23:40:54.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.555+0000 7f38a7200640 1 -- 172.21.15.94:0/1644359787 wait complete. 2024-09-10T23:40:54.576 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-10T23:40:54.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.719+0000 7f5a48200640 1 -- 172.21.15.94:0/4179702434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a4006ce80 msgr2=0x7f5a4015f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:54.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.719+0000 7f5a48200640 1 --2- 172.21.15.94:0/4179702434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a4006ce80 0x7f5a4015f820 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009840 tx=0x7f5a3401c710 comp rx=0 tx=0).stop 2024-09-10T23:40:54.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.719+0000 7f5a48200640 1 -- 172.21.15.94:0/4179702434 shutdown_connections 2024-09-10T23:40:54.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.719+0000 7f5a48200640 1 --2- 172.21.15.94:0/4179702434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a4006ce80 0x7f5a4015f820 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:54.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.719+0000 7f5a48200640 1 -- 172.21.15.94:0/4179702434 >> 172.21.15.94:0/4179702434 conn(0x7f5a400d0d80 msgr2=0x7f5a400d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:54.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.719+0000 7f5a48200640 1 -- 172.21.15.94:0/4179702434 shutdown_connections 2024-09-10T23:40:54.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.719+0000 7f5a48200640 1 -- 172.21.15.94:0/4179702434 wait complete. 2024-09-10T23:40:54.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.719+0000 7f5a48200640 1 Processor -- start 2024-09-10T23:40:54.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.719+0000 7f5a48200640 1 -- start start 2024-09-10T23:40:54.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a48200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a4006ce80 0x7f5a4014f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:54.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a48200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a4006d260 0x7f5a40135080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:54.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a48200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a40073920 0x7f5a4015ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:54.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a48200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40169f60 con 0x7f5a4006d260 2024-09-10T23:40:54.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a48200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40169de0 con 0x7f5a4006ce80 2024-09-10T23:40:54.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a48200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5a4016a0e0 con 0x7f5a40073920 2024-09-10T23:40:54.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a44e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a4006d260 0x7f5a40135080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:54.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a44e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a4006d260 0x7f5a40135080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34172/0 (socket says 172.21.15.94:34172) 2024-09-10T23:40:54.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a44e00640 1 -- 172.21.15.94:0/2742534254 learned_addr learned my addr 172.21.15.94:0/2742534254 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:54.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a45800640 1 --2- 172.21.15.94:0/2742534254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a4006ce80 0x7f5a4014f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:54.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a46200640 1 --2- 172.21.15.94:0/2742534254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a40073920 0x7f5a4015ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:54.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a44e00640 1 -- 172.21.15.94:0/2742534254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a40073920 msgr2=0x7f5a4015ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:54.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a44e00640 1 --2- 172.21.15.94:0/2742534254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a40073920 0x7f5a4015ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:54.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a44e00640 1 -- 172.21.15.94:0/2742534254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a4006ce80 msgr2=0x7f5a4014f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:54.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a44e00640 1 --2- 172.21.15.94:0/2742534254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a4006ce80 0x7f5a4014f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:54.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a44e00640 1 -- 172.21.15.94:0/2742534254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a40152840 con 0x7f5a4006d260 2024-09-10T23:40:54.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a45800640 1 --2- 172.21.15.94:0/2742534254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a4006ce80 0x7f5a4014f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:54.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a46200640 1 --2- 172.21.15.94:0/2742534254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a40073920 0x7f5a4015ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:54.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a44e00640 1 --2- 172.21.15.94:0/2742534254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a4006d260 0x7f5a40135080 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00caa0 tx=0x7f5a2c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:54.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a3d600640 1 -- 172.21.15.94:0/2742534254 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c016020 con 0x7f5a4006d260 2024-09-10T23:40:54.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a3d600640 1 -- 172.21.15.94:0/2742534254 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a2c00ab50 con 0x7f5a4006d260 2024-09-10T23:40:54.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a3d600640 1 -- 172.21.15.94:0/2742534254 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c012420 con 0x7f5a4006d260 2024-09-10T23:40:54.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a48200640 1 -- 172.21.15.94:0/2742534254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a40152b20 con 0x7f5a4006d260 2024-09-10T23:40:54.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a48200640 1 -- 172.21.15.94:0/2742534254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a40137be0 con 0x7f5a4006d260 2024-09-10T23:40:54.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a48200640 1 -- 172.21.15.94:0/2742534254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a401376f0 con 0x7f5a4006d260 2024-09-10T23:40:54.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.723+0000 7f5a3d600640 1 -- 172.21.15.94:0/2742534254 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f5a2c01a3b0 con 0x7f5a4006d260 2024-09-10T23:40:54.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.727+0000 7f5a3d600640 1 --2- 172.21.15.94:0/2742534254 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5a20073000 0x7f5a200754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:54.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.727+0000 7f5a3d600640 1 -- 172.21.15.94:0/2742534254 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f5a2c092140 con 0x7f5a4006d260 2024-09-10T23:40:54.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.727+0000 7f5a45800640 1 --2- 172.21.15.94:0/2742534254 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5a20073000 0x7f5a200754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:54.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.727+0000 7f5a45800640 1 --2- 172.21.15.94:0/2742534254 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5a20073000 0x7f5a200754c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5a340af000 tx=0x7f5a3401af50 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:54.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.731+0000 7f5a3d600640 1 -- 172.21.15.94:0/2742534254 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5a2c020030 con 0x7f5a4006d260 2024-09-10T23:40:54.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.915+0000 7f5a48200640 1 -- 172.21.15.94:0/2742534254 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f5a4015f820 con 0x7f5a20073000 2024-09-10T23:40:54.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.919+0000 7f5a3d600640 1 -- 172.21.15.94:0/2742534254 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f5a4015f820 con 0x7f5a20073000 2024-09-10T23:40:54.918 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-09-10T23:40:54.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-10T23:40:54.918 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephfs" 2024-09-10T23:40:54.918 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-10T23:40:54.918 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-09-10T23:40:54.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.919+0000 7f5a48200640 1 -- 172.21.15.94:0/2742534254 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5a20073000 msgr2=0x7f5a200754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:54.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.919+0000 7f5a48200640 1 --2- 172.21.15.94:0/2742534254 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5a20073000 0x7f5a200754c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5a340af000 tx=0x7f5a3401af50 comp rx=0 tx=0).stop 2024-09-10T23:40:54.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.919+0000 7f5a48200640 1 -- 172.21.15.94:0/2742534254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a4006d260 msgr2=0x7f5a40135080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:54.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.919+0000 7f5a48200640 1 --2- 172.21.15.94:0/2742534254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a4006d260 0x7f5a40135080 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00caa0 tx=0x7f5a2c00cf60 comp rx=0 tx=0).stop 2024-09-10T23:40:54.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.919+0000 7f5a48200640 1 -- 172.21.15.94:0/2742534254 shutdown_connections 2024-09-10T23:40:54.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.919+0000 7f5a48200640 1 --2- 172.21.15.94:0/2742534254 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5a20073000 0x7f5a200754c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:54.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.919+0000 7f5a48200640 1 --2- 172.21.15.94:0/2742534254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a40073920 0x7f5a4015ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:54.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.919+0000 7f5a48200640 1 --2- 172.21.15.94:0/2742534254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a4006d260 0x7f5a40135080 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:54.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.919+0000 7f5a48200640 1 --2- 172.21.15.94:0/2742534254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a4006ce80 0x7f5a4014f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:54.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.919+0000 7f5a48200640 1 -- 172.21.15.94:0/2742534254 >> 172.21.15.94:0/2742534254 conn(0x7f5a400d0d80 msgr2=0x7f5a40073d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:54.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.919+0000 7f5a48200640 1 -- 172.21.15.94:0/2742534254 shutdown_connections 2024-09-10T23:40:54.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:54.919+0000 7f5a48200640 1 -- 172.21.15.94:0/2742534254 wait complete. 2024-09-10T23:40:54.940 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T23:40:55.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.083+0000 7f8affa00640 1 -- 172.21.15.94:0/132304722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af8158790 msgr2=0x7f8af806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:55.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.083+0000 7f8affa00640 1 --2- 172.21.15.94:0/132304722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af8158790 0x7f8af806fc00 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f8af815f840 tx=0x7f8aec01c8c0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.083+0000 7f8affa00640 1 -- 172.21.15.94:0/132304722 shutdown_connections 2024-09-10T23:40:55.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.083+0000 7f8affa00640 1 --2- 172.21.15.94:0/132304722 >> v2:172.21.15.181:3300/0 conn(0x7f8af80cb4d0 0x7f8af815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.083+0000 7f8affa00640 1 --2- 172.21.15.94:0/132304722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af8158790 0x7f8af806fc00 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.083+0000 7f8affa00640 1 -- 172.21.15.94:0/132304722 >> 172.21.15.94:0/132304722 conn(0x7f8af80cfaf0 msgr2=0x7f8af80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:55.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.083+0000 7f8affa00640 1 -- 172.21.15.94:0/132304722 shutdown_connections 2024-09-10T23:40:55.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.083+0000 7f8affa00640 1 -- 172.21.15.94:0/132304722 wait complete. 2024-09-10T23:40:55.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8affa00640 1 Processor -- start 2024-09-10T23:40:55.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8affa00640 1 -- start start 2024-09-10T23:40:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8affa00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8af80c49b0 0x7f8af80aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8affa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af80cb4d0 0x7f8af81fb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8affa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8af8158790 0x7f8af80c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8affa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8af8167f10 con 0x7f8af80cb4d0 2024-09-10T23:40:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8affa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8af8167d90 con 0x7f8af8158790 2024-09-10T23:40:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8affa00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8af8168090 con 0x7f8af80c49b0 2024-09-10T23:40:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8af7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af80cb4d0 0x7f8af81fb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8af7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af80cb4d0 0x7f8af81fb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34198/0 (socket says 172.21.15.94:34198) 2024-09-10T23:40:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8af7e00640 1 -- 172.21.15.94:0/1253107525 learned_addr learned my addr 172.21.15.94:0/1253107525 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8afd000640 1 --2- 172.21.15.94:0/1253107525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8af80c49b0 0x7f8af80aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8afda00640 1 --2- 172.21.15.94:0/1253107525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8af8158790 0x7f8af80c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8af7e00640 1 -- 172.21.15.94:0/1253107525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8af80c49b0 msgr2=0x7f8af80aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8af7e00640 1 --2- 172.21.15.94:0/1253107525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8af80c49b0 0x7f8af80aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8af7e00640 1 -- 172.21.15.94:0/1253107525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8af8158790 msgr2=0x7f8af80c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8af7e00640 1 --2- 172.21.15.94:0/1253107525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8af8158790 0x7f8af80c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8af7e00640 1 -- 172.21.15.94:0/1253107525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af81f8080 con 0x7f8af80cb4d0 2024-09-10T23:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8af7e00640 1 --2- 172.21.15.94:0/1253107525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af80cb4d0 0x7f8af81fb480 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c500 tx=0x7f8ae400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8af4c00640 1 -- 172.21.15.94:0/1253107525 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4017070 con 0x7f8af80cb4d0 2024-09-10T23:40:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8af4c00640 1 -- 172.21.15.94:0/1253107525 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ae40053c0 con 0x7f8af80cb4d0 2024-09-10T23:40:55.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8af4c00640 1 -- 172.21.15.94:0/1253107525 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4012890 con 0x7f8af80cb4d0 2024-09-10T23:40:55.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8affa00640 1 -- 172.21.15.94:0/1253107525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af80c85e0 con 0x7f8af80cb4d0 2024-09-10T23:40:55.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.087+0000 7f8affa00640 1 -- 172.21.15.94:0/1253107525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af81fbed0 con 0x7f8af80cb4d0 2024-09-10T23:40:55.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.091+0000 7f8affa00640 1 -- 172.21.15.94:0/1253107525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af815b190 con 0x7f8af80cb4d0 2024-09-10T23:40:55.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.091+0000 7f8af4c00640 1 -- 172.21.15.94:0/1253107525 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7f8ae4003600 con 0x7f8af80cb4d0 2024-09-10T23:40:55.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.091+0000 7f8af4c00640 1 --2- 172.21.15.94:0/1253107525 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ad8073000 0x7f8ad80754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:55.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.091+0000 7f8af4c00640 1 -- 172.21.15.94:0/1253107525 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f8ae40913b0 con 0x7f8af80cb4d0 2024-09-10T23:40:55.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.091+0000 7f8afd000640 1 --2- 172.21.15.94:0/1253107525 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ad8073000 0x7f8ad80754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:55.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.091+0000 7f8afd000640 1 --2- 172.21.15.94:0/1253107525 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ad8073000 0x7f8ad80754c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0af000 tx=0x7f8aec0a7040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:55.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.095+0000 7f8af4c00640 1 -- 172.21.15.94:0/1253107525 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8ae4098070 con 0x7f8af80cb4d0 2024-09-10T23:40:55.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.263+0000 7f8affa00640 1 -- 172.21.15.94:0/1253107525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8af81f8250 con 0x7f8af80cb4d0 2024-09-10T23:40:55.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.263+0000 7f8af4c00640 1 -- 172.21.15.94:0/1253107525 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f8ae405dbf0 con 0x7f8af80cb4d0 2024-09-10T23:40:55.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.263+0000 7f8affa00640 1 -- 172.21.15.94:0/1253107525 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ad8073000 msgr2=0x7f8ad80754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:55.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.263+0000 7f8affa00640 1 --2- 172.21.15.94:0/1253107525 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ad8073000 0x7f8ad80754c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0af000 tx=0x7f8aec0a7040 comp rx=0 tx=0).stop 2024-09-10T23:40:55.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.263+0000 7f8affa00640 1 -- 172.21.15.94:0/1253107525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af80cb4d0 msgr2=0x7f8af81fb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:55.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.263+0000 7f8affa00640 1 --2- 172.21.15.94:0/1253107525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af80cb4d0 0x7f8af81fb480 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c500 tx=0x7f8ae400c9c0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.263+0000 7f8affa00640 1 -- 172.21.15.94:0/1253107525 shutdown_connections 2024-09-10T23:40:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.263+0000 7f8affa00640 1 --2- 172.21.15.94:0/1253107525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8af8158790 0x7f8af80c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.263+0000 7f8affa00640 1 --2- 172.21.15.94:0/1253107525 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8ad8073000 0x7f8ad80754c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.263+0000 7f8affa00640 1 --2- 172.21.15.94:0/1253107525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8af80cb4d0 0x7f8af81fb480 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.263+0000 7f8affa00640 1 --2- 172.21.15.94:0/1253107525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8af80c49b0 0x7f8af80aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.263+0000 7f8affa00640 1 -- 172.21.15.94:0/1253107525 >> 172.21.15.94:0/1253107525 conn(0x7f8af80cfaf0 msgr2=0x7f8af8156880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.263+0000 7f8affa00640 1 -- 172.21.15.94:0/1253107525 shutdown_connections 2024-09-10T23:40:55.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.263+0000 7f8affa00640 1 -- 172.21.15.94:0/1253107525 wait complete. 2024-09-10T23:40:55.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:55.284 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp99170/subvol83700) 2024-09-10T23:40:55.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:55.313 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-10T23:40:55.367 INFO:teuthology.orchestra.run.smithi094.stdout:cephfs_data 2024-09-10T23:40:55.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:55.368 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp99170/subvol83700) 2024-09-10T23:40:55.434 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:40:55.434 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp99170/subvol83700/.meta 2024-09-10T23:40:55.501 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol83700 subvol_grp99170 2024-09-10T23:40:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe513200640 1 -- 172.21.15.94:0/4070770310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe50c06f600 msgr2=0x7fe50c0b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe513200640 1 --2- 172.21.15.94:0/4070770310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe50c06f600 0x7fe50c0b35c0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0b2760 tx=0x7fe50401caf0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe513200640 1 -- 172.21.15.94:0/4070770310 shutdown_connections 2024-09-10T23:40:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe513200640 1 --2- 172.21.15.94:0/4070770310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe50c06f600 0x7fe50c0b35c0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe513200640 1 --2- 172.21.15.94:0/4070770310 >> v2:172.21.15.181:3300/0 conn(0x7fe50c157c60 0x7fe50c072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe513200640 1 -- 172.21.15.94:0/4070770310 >> 172.21.15.94:0/4070770310 conn(0x7fe50c06cdf0 msgr2=0x7fe50c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe513200640 1 -- 172.21.15.94:0/4070770310 shutdown_connections 2024-09-10T23:40:55.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe513200640 1 -- 172.21.15.94:0/4070770310 wait complete. 2024-09-10T23:40:55.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe513200640 1 Processor -- start 2024-09-10T23:40:55.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe513200640 1 -- start start 2024-09-10T23:40:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe513200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe50c06f600 0x7fe50c1535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe513200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe50c070150 0x7fe50c14f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe513200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe50c157c60 0x7fe50c164020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe513200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c15b310 con 0x7fe50c157c60 2024-09-10T23:40:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe513200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c15b190 con 0x7fe50c06f600 2024-09-10T23:40:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe513200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fe50c15b490 con 0x7fe50c070150 2024-09-10T23:40:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe511200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe50c157c60 0x7fe50c164020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe511200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe50c157c60 0x7fe50c164020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34206/0 (socket says 172.21.15.94:34206) 2024-09-10T23:40:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe511200640 1 -- 172.21.15.94:0/3440830217 learned_addr learned my addr 172.21.15.94:0/3440830217 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe50be00640 1 --2- 172.21.15.94:0/3440830217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe50c06f600 0x7fe50c1535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe503400640 1 --2- 172.21.15.94:0/3440830217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe50c070150 0x7fe50c14f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe511200640 1 -- 172.21.15.94:0/3440830217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe50c070150 msgr2=0x7fe50c14f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe511200640 1 --2- 172.21.15.94:0/3440830217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe50c070150 0x7fe50c14f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe511200640 1 -- 172.21.15.94:0/3440830217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe50c06f600 msgr2=0x7fe50c1535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe511200640 1 --2- 172.21.15.94:0/3440830217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe50c06f600 0x7fe50c1535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe511200640 1 -- 172.21.15.94:0/3440830217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50c150860 con 0x7fe50c157c60 2024-09-10T23:40:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.695+0000 7fe503400640 1 --2- 172.21.15.94:0/3440830217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe50c070150 0x7fe50c14f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:40:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.699+0000 7fe511200640 1 --2- 172.21.15.94:0/3440830217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe50c157c60 0x7fe50c164020 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe504009b10 tx=0x7fe5040a5870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.699+0000 7fe508c00640 1 -- 172.21.15.94:0/3440830217 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5040bb070 con 0x7fe50c157c60 2024-09-10T23:40:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.699+0000 7fe508c00640 1 -- 172.21.15.94:0/3440830217 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe504004d40 con 0x7fe50c157c60 2024-09-10T23:40:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.699+0000 7fe508c00640 1 -- 172.21.15.94:0/3440830217 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5040b2040 con 0x7fe50c157c60 2024-09-10T23:40:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.699+0000 7fe513200640 1 -- 172.21.15.94:0/3440830217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50c160f00 con 0x7fe50c157c60 2024-09-10T23:40:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.699+0000 7fe513200640 1 -- 172.21.15.94:0/3440830217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe50c1614c0 con 0x7fe50c157c60 2024-09-10T23:40:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.699+0000 7fe513200640 1 -- 172.21.15.94:0/3440830217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50c0b35c0 con 0x7fe50c157c60 2024-09-10T23:40:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.699+0000 7fe508c00640 1 -- 172.21.15.94:0/3440830217 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7fe5040b5030 con 0x7fe50c157c60 2024-09-10T23:40:55.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.699+0000 7fe508c00640 1 --2- 172.21.15.94:0/3440830217 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe4e8073000 0x7fe4e80754c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:55.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.699+0000 7fe508c00640 1 -- 172.21.15.94:0/3440830217 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7fe50412b9a0 con 0x7fe50c157c60 2024-09-10T23:40:55.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.699+0000 7fe50be00640 1 --2- 172.21.15.94:0/3440830217 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe4e8073000 0x7fe4e80754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:55.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.703+0000 7fe50be00640 1 --2- 172.21.15.94:0/3440830217 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe4e8073000 0x7fe4e80754c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc005fd0 tx=0x7fe4fc005f20 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:55.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.703+0000 7fe508c00640 1 -- 172.21.15.94:0/3440830217 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe504016770 con 0x7fe50c157c60 2024-09-10T23:40:55.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.895+0000 7fe513200640 1 -- 172.21.15.94:0/3440830217 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol83700", "group_name": "subvol_grp99170", "target": ["mon-mgr", ""]}) -- 0x7fe50c06e7e0 con 0x7fe4e8073000 2024-09-10T23:40:55.900 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:55.899+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:55.900 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:55.899+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:55.900 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:55.899+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:55.900 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:55.899+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:55.901 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:40:55.899+0000 7f9e9fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:40:55.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.931+0000 7fe508c00640 1 -- 172.21.15.94:0/3440830217 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe50c06e7e0 con 0x7fe4e8073000 2024-09-10T23:40:55.933 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/subvol_grp99170/subvol83700 2024-09-10T23:40:55.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.935+0000 7fe513200640 1 -- 172.21.15.94:0/3440830217 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe4e8073000 msgr2=0x7fe4e80754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:55.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.935+0000 7fe513200640 1 --2- 172.21.15.94:0/3440830217 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe4e8073000 0x7fe4e80754c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc005fd0 tx=0x7fe4fc005f20 comp rx=0 tx=0).stop 2024-09-10T23:40:55.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.935+0000 7fe513200640 1 -- 172.21.15.94:0/3440830217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe50c157c60 msgr2=0x7fe50c164020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:55.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.935+0000 7fe513200640 1 --2- 172.21.15.94:0/3440830217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe50c157c60 0x7fe50c164020 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe504009b10 tx=0x7fe5040a5870 comp rx=0 tx=0).stop 2024-09-10T23:40:55.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.935+0000 7fe513200640 1 -- 172.21.15.94:0/3440830217 shutdown_connections 2024-09-10T23:40:55.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.935+0000 7fe513200640 1 --2- 172.21.15.94:0/3440830217 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fe4e8073000 0x7fe4e80754c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.935+0000 7fe513200640 1 --2- 172.21.15.94:0/3440830217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe50c157c60 0x7fe50c164020 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.935+0000 7fe513200640 1 --2- 172.21.15.94:0/3440830217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe50c070150 0x7fe50c14f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.935+0000 7fe513200640 1 --2- 172.21.15.94:0/3440830217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe50c06f600 0x7fe50c1535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:55.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.935+0000 7fe513200640 1 -- 172.21.15.94:0/3440830217 >> 172.21.15.94:0/3440830217 conn(0x7fe50c06cdf0 msgr2=0x7fe50c06eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:55.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.935+0000 7fe513200640 1 -- 172.21.15.94:0/3440830217 shutdown_connections 2024-09-10T23:40:55.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:55.935+0000 7fe513200640 1 -- 172.21.15.94:0/3440830217 wait complete. 2024-09-10T23:40:55.954 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol83700 subvol_grp99170 2024-09-10T23:40:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.099+0000 7fa355400640 1 -- 172.21.15.94:0/3192179956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa35015f3f0 msgr2=0x7fa350068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.099+0000 7fa355400640 1 --2- 172.21.15.94:0/3192179956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa35015f3f0 0x7fa350068990 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fa340009840 tx=0x7fa34001c790 comp rx=0 tx=0).stop 2024-09-10T23:40:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.099+0000 7fa355400640 1 -- 172.21.15.94:0/3192179956 shutdown_connections 2024-09-10T23:40:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.099+0000 7fa355400640 1 --2- 172.21.15.94:0/3192179956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa35015f3f0 0x7fa350068990 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.099+0000 7fa355400640 1 -- 172.21.15.94:0/3192179956 >> 172.21.15.94:0/3192179956 conn(0x7fa3500d02d0 msgr2=0x7fa3500d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.099+0000 7fa355400640 1 -- 172.21.15.94:0/3192179956 shutdown_connections 2024-09-10T23:40:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.099+0000 7fa355400640 1 -- 172.21.15.94:0/3192179956 wait complete. 2024-09-10T23:40:56.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa355400640 1 Processor -- start 2024-09-10T23:40:56.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa355400640 1 -- start start 2024-09-10T23:40:56.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa355400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3500cc2f0 0x7fa350165e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:56.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa355400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3500cdea0 0x7fa350098480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:56.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa355400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa35015f3f0 0x7fa3500989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:56.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa355400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa35006da30 con 0x7fa35015f3f0 2024-09-10T23:40:56.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa355400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa35006d8b0 con 0x7fa3500cdea0 2024-09-10T23:40:56.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa355400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa3501697f0 con 0x7fa3500cc2f0 2024-09-10T23:40:56.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa34f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa35015f3f0 0x7fa3500989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:56.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa34f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa35015f3f0 0x7fa3500989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34238/0 (socket says 172.21.15.94:34238) 2024-09-10T23:40:56.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa34f200640 1 -- 172.21.15.94:0/290953335 learned_addr learned my addr 172.21.15.94:0/290953335 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:40:56.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa345e00640 1 --2- 172.21.15.94:0/290953335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3500cdea0 0x7fa350098480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:56.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa34e800640 1 --2- 172.21.15.94:0/290953335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3500cc2f0 0x7fa350165e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:56.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa34f200640 1 -- 172.21.15.94:0/290953335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3500cc2f0 msgr2=0x7fa350165e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:56.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa34f200640 1 --2- 172.21.15.94:0/290953335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3500cc2f0 0x7fa350165e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:56.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa34f200640 1 -- 172.21.15.94:0/290953335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3500cdea0 msgr2=0x7fa350098480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:56.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa34f200640 1 --2- 172.21.15.94:0/290953335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3500cdea0 0x7fa350098480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:56.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa34f200640 1 -- 172.21.15.94:0/290953335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3501f9210 con 0x7fa35015f3f0 2024-09-10T23:40:56.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa345e00640 1 --2- 172.21.15.94:0/290953335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3500cdea0 0x7fa350098480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:40:56.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa34f200640 1 --2- 172.21.15.94:0/290953335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa35015f3f0 0x7fa3500989c0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fa34800c7c0 tx=0x7fa34800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:56.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa347400640 1 -- 172.21.15.94:0/290953335 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348003b80 con 0x7fa35015f3f0 2024-09-10T23:40:56.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa347400640 1 -- 172.21.15.94:0/290953335 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa348003d20 con 0x7fa35015f3f0 2024-09-10T23:40:56.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa347400640 1 -- 172.21.15.94:0/290953335 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34800aa50 con 0x7fa35015f3f0 2024-09-10T23:40:56.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa355400640 1 -- 172.21.15.94:0/290953335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3500b27f0 con 0x7fa35015f3f0 2024-09-10T23:40:56.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa355400640 1 -- 172.21.15.94:0/290953335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3500b24e0 con 0x7fa35015f3f0 2024-09-10T23:40:56.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.103+0000 7fa355400640 1 -- 172.21.15.94:0/290953335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa350068990 con 0x7fa35015f3f0 2024-09-10T23:40:56.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.107+0000 7fa347400640 1 -- 172.21.15.94:0/290953335 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 97312+0+0 (secure 0 0 0) 0x7fa34800abf0 con 0x7fa35015f3f0 2024-09-10T23:40:56.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.107+0000 7fa347400640 1 --2- 172.21.15.94:0/290953335 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa32c073030 0x7fa32c0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:40:56.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.107+0000 7fa347400640 1 -- 172.21.15.94:0/290953335 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7fa3480946a0 con 0x7fa35015f3f0 2024-09-10T23:40:56.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.107+0000 7fa34e800640 1 --2- 172.21.15.94:0/290953335 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa32c073030 0x7fa32c0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:40:56.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.107+0000 7fa34e800640 1 --2- 172.21.15.94:0/290953335 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa32c073030 0x7fa32c0754f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa3400af000 tx=0x7fa34001afd0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:40:56.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.111+0000 7fa347400640 1 -- 172.21.15.94:0/290953335 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa34805eaa0 con 0x7fa35015f3f0 2024-09-10T23:40:56.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.303+0000 7fa355400640 1 -- 172.21.15.94:0/290953335 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol83700", "group_name": "subvol_grp99170", "target": ["mon-mgr", ""]}) -- 0x7fa3500ae620 con 0x7fa32c073030 2024-09-10T23:40:56.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.307+0000 7fa347400640 1 -- 172.21.15.94:0/290953335 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa3500ae620 con 0x7fa32c073030 2024-09-10T23:40:56.307 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/subvol_grp99170/subvol83700 2024-09-10T23:40:56.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.311+0000 7fa355400640 1 -- 172.21.15.94:0/290953335 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa32c073030 msgr2=0x7fa32c0754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:56.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.311+0000 7fa355400640 1 --2- 172.21.15.94:0/290953335 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa32c073030 0x7fa32c0754f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa3400af000 tx=0x7fa34001afd0 comp rx=0 tx=0).stop 2024-09-10T23:40:56.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.311+0000 7fa355400640 1 -- 172.21.15.94:0/290953335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa35015f3f0 msgr2=0x7fa3500989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:40:56.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.311+0000 7fa355400640 1 --2- 172.21.15.94:0/290953335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa35015f3f0 0x7fa3500989c0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fa34800c7c0 tx=0x7fa34800cc80 comp rx=0 tx=0).stop 2024-09-10T23:40:56.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.311+0000 7fa355400640 1 -- 172.21.15.94:0/290953335 shutdown_connections 2024-09-10T23:40:56.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.311+0000 7fa355400640 1 --2- 172.21.15.94:0/290953335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3500cdea0 0x7fa350098480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:56.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.311+0000 7fa355400640 1 --2- 172.21.15.94:0/290953335 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa32c073030 0x7fa32c0754f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:56.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.311+0000 7fa355400640 1 --2- 172.21.15.94:0/290953335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa35015f3f0 0x7fa3500989c0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:56.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.311+0000 7fa355400640 1 --2- 172.21.15.94:0/290953335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3500cc2f0 0x7fa350165e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:40:56.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.311+0000 7fa355400640 1 -- 172.21.15.94:0/290953335 >> 172.21.15.94:0/290953335 conn(0x7fa3500d02d0 msgr2=0x7fa3500ad220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:40:56.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.311+0000 7fa355400640 1 -- 172.21.15.94:0/290953335 shutdown_connections 2024-09-10T23:40:56.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:40:56.311+0000 7fa355400640 1 -- 172.21.15.94:0/290953335 wait complete. 2024-09-10T23:40:56.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:40:56.329 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/c7cf4af1c6bd70a0599eed595edd55d1.meta) 2024-09-10T23:41:00.590 INFO:teuthology.orchestra.run.smithi094.stdout:[GLOBAL] 2024-09-10T23:41:00.590 INFO:teuthology.orchestra.run.smithi094.stdout:version = 1 2024-09-10T23:41:00.590 INFO:teuthology.orchestra.run.smithi094.stdout:type = subvolume 2024-09-10T23:41:00.590 INFO:teuthology.orchestra.run.smithi094.stdout:path = /volumes/subvol_grp99170/subvol83700 2024-09-10T23:41:00.590 INFO:teuthology.orchestra.run.smithi094.stdout:state = complete 2024-09-10T23:41:00.590 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:41:00.593 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol83700 subvol_grp99170 2024-09-10T23:41:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.746+0000 7fcb76800640 1 -- 172.21.15.94:0/1418900053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb58003c70 msgr2=0x7fcb58004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.746+0000 7fcb76800640 1 --2- 172.21.15.94:0/1418900053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb58003c70 0x7fcb58004070 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400c640 tx=0x7fcb6401d250 comp rx=0 tx=0).stop 2024-09-10T23:41:00.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.746+0000 7fcb75e00640 1 -- 172.21.15.94:0/1418900053 reap_dead start 2024-09-10T23:41:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.746+0000 7fcb76800640 1 -- 172.21.15.94:0/1418900053 shutdown_connections 2024-09-10T23:41:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.746+0000 7fcb76800640 1 -- 172.21.15.94:0/1418900053 >> 172.21.15.94:0/1418900053 conn(0x7fcb7009aa70 msgr2=0x7fcb7009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.746+0000 7fcb76800640 1 -- 172.21.15.94:0/1418900053 shutdown_connections 2024-09-10T23:41:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.746+0000 7fcb76800640 1 -- 172.21.15.94:0/1418900053 wait complete. 2024-09-10T23:41:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb76800640 1 Processor -- start 2024-09-10T23:41:00.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb76800640 1 -- start start 2024-09-10T23:41:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb76800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb58003c70 0x7fcb7006ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb76800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb70152ac0 0x7fcb701fa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb76800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb701faf20 0x7fcb701fb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb76800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcb70167a40 con 0x7fcb58003c70 2024-09-10T23:41:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb76800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcb701678c0 con 0x7fcb701faf20 2024-09-10T23:41:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb76800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fcb70167bc0 con 0x7fcb70152ac0 2024-09-10T23:41:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb75400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb58003c70 0x7fcb7006ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb75400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb58003c70 0x7fcb7006ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34246/0 (socket says 172.21.15.94:34246) 2024-09-10T23:41:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb75400640 1 -- 172.21.15.94:0/2573553167 learned_addr learned my addr 172.21.15.94:0/2573553167 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:00.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb74a00640 1 --2- 172.21.15.94:0/2573553167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb70152ac0 0x7fcb701fa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:00.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb75e00640 1 --2- 172.21.15.94:0/2573553167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb701faf20 0x7fcb701fb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:00.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb75400640 1 -- 172.21.15.94:0/2573553167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb70152ac0 msgr2=0x7fcb701fa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:00.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb75400640 1 --2- 172.21.15.94:0/2573553167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb70152ac0 0x7fcb701fa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:00.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb75400640 1 -- 172.21.15.94:0/2573553167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb701faf20 msgr2=0x7fcb701fb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:00.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb75400640 1 --2- 172.21.15.94:0/2573553167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb701faf20 0x7fcb701fb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:00.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb75400640 1 -- 172.21.15.94:0/2573553167 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb701fb8e0 con 0x7fcb58003c70 2024-09-10T23:41:00.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb75e00640 1 --2- 172.21.15.94:0/2573553167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb701faf20 0x7fcb701fb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:41:00.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb75400640 1 --2- 172.21.15.94:0/2573553167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb58003c70 0x7fcb7006ea80 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400c640 tx=0x7fcb640014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:00.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb6d600640 1 -- 172.21.15.94:0/2573553167 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64004e60 con 0x7fcb58003c70 2024-09-10T23:41:00.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb6d600640 1 -- 172.21.15.94:0/2573553167 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb64003c70 con 0x7fcb58003c70 2024-09-10T23:41:00.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb6d600640 1 -- 172.21.15.94:0/2573553167 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64006370 con 0x7fcb58003c70 2024-09-10T23:41:00.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb76800640 1 -- 172.21.15.94:0/2573553167 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb701616b0 con 0x7fcb58003c70 2024-09-10T23:41:00.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.750+0000 7fcb76800640 1 -- 172.21.15.94:0/2573553167 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb70161c70 con 0x7fcb58003c70 2024-09-10T23:41:00.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.754+0000 7fcb76800640 1 -- 172.21.15.94:0/2573553167 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb700af2c0 con 0x7fcb58003c70 2024-09-10T23:41:00.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.754+0000 7fcb6d600640 1 -- 172.21.15.94:0/2573553167 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 97363+0+0 (secure 0 0 0) 0x7fcb64006510 con 0x7fcb58003c70 2024-09-10T23:41:00.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.754+0000 7fcb6d600640 1 --2- 172.21.15.94:0/2573553167 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb500730c0 0x7fcb50075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:00.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.754+0000 7fcb6d600640 1 -- 172.21.15.94:0/2573553167 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7fcb6412c070 con 0x7fcb58003c70 2024-09-10T23:41:00.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.754+0000 7fcb74a00640 1 --2- 172.21.15.94:0/2573553167 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb500730c0 0x7fcb50075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:00.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.754+0000 7fcb74a00640 1 --2- 172.21.15.94:0/2573553167 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb500730c0 0x7fcb50075580 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcb7006ff10 tx=0x7fcb5c0091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:00.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.758+0000 7fcb6d600640 1 -- 172.21.15.94:0/2573553167 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcb64017030 con 0x7fcb58003c70 2024-09-10T23:41:00.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.950+0000 7fcb76800640 1 -- 172.21.15.94:0/2573553167 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol83700", "group_name": "subvol_grp99170", "target": ["mon-mgr", ""]}) -- 0x7fcb70070f70 con 0x7fcb500730c0 2024-09-10T23:41:00.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.962+0000 7fcb6d600640 1 -- 172.21.15.94:0/2573553167 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcb70070f70 con 0x7fcb500730c0 2024-09-10T23:41:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.966+0000 7fcb76800640 1 -- 172.21.15.94:0/2573553167 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb500730c0 msgr2=0x7fcb50075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.966+0000 7fcb76800640 1 --2- 172.21.15.94:0/2573553167 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb500730c0 0x7fcb50075580 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcb7006ff10 tx=0x7fcb5c0091e0 comp rx=0 tx=0).stop 2024-09-10T23:41:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.966+0000 7fcb76800640 1 -- 172.21.15.94:0/2573553167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb58003c70 msgr2=0x7fcb7006ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.966+0000 7fcb76800640 1 --2- 172.21.15.94:0/2573553167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb58003c70 0x7fcb7006ea80 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400c640 tx=0x7fcb640014a0 comp rx=0 tx=0).stop 2024-09-10T23:41:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.966+0000 7fcb76800640 1 -- 172.21.15.94:0/2573553167 shutdown_connections 2024-09-10T23:41:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.966+0000 7fcb76800640 1 --2- 172.21.15.94:0/2573553167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb701faf20 0x7fcb701fb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.966+0000 7fcb76800640 1 --2- 172.21.15.94:0/2573553167 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fcb500730c0 0x7fcb50075580 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.966+0000 7fcb76800640 1 --2- 172.21.15.94:0/2573553167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb70152ac0 0x7fcb701fa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.966+0000 7fcb76800640 1 --2- 172.21.15.94:0/2573553167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb58003c70 0x7fcb7006ea80 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.966+0000 7fcb76800640 1 -- 172.21.15.94:0/2573553167 >> 172.21.15.94:0/2573553167 conn(0x7fcb7009aa70 msgr2=0x7fcb70156fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.966+0000 7fcb76800640 1 -- 172.21.15.94:0/2573553167 shutdown_connections 2024-09-10T23:41:00.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:00.966+0000 7fcb76800640 1 -- 172.21.15.94:0/2573553167 wait complete. 2024-09-10T23:41:00.971 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:41:00.970+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:41:00.971 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:41:00.970+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:41:00.971 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:41:00.970+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:41:00.971 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:41:00.970+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:41:00.971 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:41:00.970+0000 7f9ea1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:41:00.990 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:41:00.990+0000 7f9ea3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:41:00.990 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:41:00.990+0000 7f9ea3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:41:00.990 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:41:00.990+0000 7f9ea3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:41:00.990 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:41:00.990+0000 7f9ea3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:41:00.990 INFO:tasks.ceph.mgr.y.smithi094.stderr:2024-09-10T23:41:00.990+0000 7f9ea3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T23:41:00.994 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-10T23:41:01.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T23:41:01.051 INFO:teuthology.orchestra.run.smithi094.stdout:2 2024-09-10T23:41:01.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T23:41:01.058 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp99170 2024-09-10T23:41:01.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.206+0000 7fd545200640 1 -- 172.21.15.94:0/1078115228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd524003a10 msgr2=0x7fd524003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:01.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.206+0000 7fd545200640 1 --2- 172.21.15.94:0/1078115228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd524003a10 0x7fd524003e30 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fd530009b60 tx=0x7fd53001d2b0 comp rx=0 tx=0).stop 2024-09-10T23:41:01.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.206+0000 7fd53e800640 1 -- 172.21.15.94:0/1078115228 reap_dead start 2024-09-10T23:41:01.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.206+0000 7fd545200640 1 -- 172.21.15.94:0/1078115228 shutdown_connections 2024-09-10T23:41:01.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.206+0000 7fd545200640 1 -- 172.21.15.94:0/1078115228 >> 172.21.15.94:0/1078115228 conn(0x7fd5400d1c60 msgr2=0x7fd5400d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:01.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.206+0000 7fd545200640 1 -- 172.21.15.94:0/1078115228 shutdown_connections 2024-09-10T23:41:01.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.206+0000 7fd545200640 1 -- 172.21.15.94:0/1078115228 wait complete. 2024-09-10T23:41:01.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.206+0000 7fd545200640 1 Processor -- start 2024-09-10T23:41:01.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.206+0000 7fd545200640 1 -- start start 2024-09-10T23:41:01.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd545200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd524003a10 0x7fd54015d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:01.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd545200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5400b4fd0 0x7fd5401f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:01.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd545200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5401f3420 0x7fd5401f9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:01.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd545200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd540166db0 con 0x7fd5400b4fd0 2024-09-10T23:41:01.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd545200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd540166c30 con 0x7fd524003a10 2024-09-10T23:41:01.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd545200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fd540166f30 con 0x7fd5401f3420 2024-09-10T23:41:01.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd53d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5400b4fd0 0x7fd5401f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:01.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd53d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5400b4fd0 0x7fd5401f6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34278/0 (socket says 172.21.15.94:34278) 2024-09-10T23:41:01.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd53d400640 1 -- 172.21.15.94:0/1514193006 learned_addr learned my addr 172.21.15.94:0/1514193006 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:01.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd53e800640 1 --2- 172.21.15.94:0/1514193006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5401f3420 0x7fd5401f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:01.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd53de00640 1 --2- 172.21.15.94:0/1514193006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd524003a10 0x7fd54015d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:01.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd53d400640 1 -- 172.21.15.94:0/1514193006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5401f3420 msgr2=0x7fd5401f9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:01.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd53d400640 1 --2- 172.21.15.94:0/1514193006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5401f3420 0x7fd5401f9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:01.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd53d400640 1 -- 172.21.15.94:0/1514193006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd524003a10 msgr2=0x7fd54015d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:01.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd53d400640 1 --2- 172.21.15.94:0/1514193006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd524003a10 0x7fd54015d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:01.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd53d400640 1 -- 172.21.15.94:0/1514193006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5400cab90 con 0x7fd5400b4fd0 2024-09-10T23:41:01.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd53e800640 1 --2- 172.21.15.94:0/1514193006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5401f3420 0x7fd5401f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:41:01.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd53de00640 1 --2- 172.21.15.94:0/1514193006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd524003a10 0x7fd54015d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:41:01.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd53d400640 1 --2- 172.21.15.94:0/1514193006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5400b4fd0 0x7fd5401f6b00 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0118a0 tx=0x7fd52c011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:01.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd536000640 1 -- 172.21.15.94:0/1514193006 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c00fbb0 con 0x7fd5400b4fd0 2024-09-10T23:41:01.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd536000640 1 -- 172.21.15.94:0/1514193006 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd52c00fd50 con 0x7fd5400b4fd0 2024-09-10T23:41:01.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd545200640 1 -- 172.21.15.94:0/1514193006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5401f7040 con 0x7fd5400b4fd0 2024-09-10T23:41:01.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd536000640 1 -- 172.21.15.94:0/1514193006 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c019670 con 0x7fd5400b4fd0 2024-09-10T23:41:01.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd545200640 1 -- 172.21.15.94:0/1514193006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5400c9140 con 0x7fd5400b4fd0 2024-09-10T23:41:01.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.210+0000 7fd545200640 1 -- 172.21.15.94:0/1514193006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5400d2780 con 0x7fd5400b4fd0 2024-09-10T23:41:01.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.214+0000 7fd536000640 1 -- 172.21.15.94:0/1514193006 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 97363+0+0 (secure 0 0 0) 0x7fd52c019810 con 0x7fd5400b4fd0 2024-09-10T23:41:01.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.214+0000 7fd536000640 1 --2- 172.21.15.94:0/1514193006 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd5200730c0 0x7fd520075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:01.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.214+0000 7fd536000640 1 -- 172.21.15.94:0/1514193006 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7fd52c092860 con 0x7fd5400b4fd0 2024-09-10T23:41:01.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.214+0000 7fd53de00640 1 --2- 172.21.15.94:0/1514193006 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd5200730c0 0x7fd520075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:01.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.214+0000 7fd53de00640 1 --2- 172.21.15.94:0/1514193006 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd5200730c0 0x7fd520075580 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd5300af000 tx=0x7fd53001a050 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:01.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.218+0000 7fd536000640 1 -- 172.21.15.94:0/1514193006 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd5401f7040 con 0x7fd5400b4fd0 2024-09-10T23:41:01.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.406+0000 7fd545200640 1 -- 172.21.15.94:0/1514193006 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp99170", "target": ["mon-mgr", ""]}) -- 0x7fd5400c54a0 con 0x7fd5200730c0 2024-09-10T23:41:01.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.410+0000 7fd536000640 1 -- 172.21.15.94:0/1514193006 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd5400c54a0 con 0x7fd5200730c0 2024-09-10T23:41:01.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.414+0000 7fd545200640 1 -- 172.21.15.94:0/1514193006 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd5200730c0 msgr2=0x7fd520075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:01.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.414+0000 7fd545200640 1 --2- 172.21.15.94:0/1514193006 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd5200730c0 0x7fd520075580 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd5300af000 tx=0x7fd53001a050 comp rx=0 tx=0).stop 2024-09-10T23:41:01.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.414+0000 7fd545200640 1 -- 172.21.15.94:0/1514193006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5400b4fd0 msgr2=0x7fd5401f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:01.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.414+0000 7fd545200640 1 --2- 172.21.15.94:0/1514193006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5400b4fd0 0x7fd5401f6b00 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0118a0 tx=0x7fd52c011d60 comp rx=0 tx=0).stop 2024-09-10T23:41:01.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.414+0000 7fd545200640 1 -- 172.21.15.94:0/1514193006 shutdown_connections 2024-09-10T23:41:01.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.414+0000 7fd545200640 1 --2- 172.21.15.94:0/1514193006 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fd5200730c0 0x7fd520075580 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:01.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.414+0000 7fd545200640 1 --2- 172.21.15.94:0/1514193006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5401f3420 0x7fd5401f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:01.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.414+0000 7fd545200640 1 --2- 172.21.15.94:0/1514193006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5400b4fd0 0x7fd5401f6b00 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:01.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.414+0000 7fd545200640 1 --2- 172.21.15.94:0/1514193006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd524003a10 0x7fd54015d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:01.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.414+0000 7fd545200640 1 -- 172.21.15.94:0/1514193006 >> 172.21.15.94:0/1514193006 conn(0x7fd5400d1c60 msgr2=0x7fd540161fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:01.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.414+0000 7fd545200640 1 -- 172.21.15.94:0/1514193006 shutdown_connections 2024-09-10T23:41:01.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.414+0000 7fd545200640 1 -- 172.21.15.94:0/1514193006 wait complete. 2024-09-10T23:41:01.464 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T23:41:01.464 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:41:01.575 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T23:41:01.679 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T23:41:01.679 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:41:01.679 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:41:01.679 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:41:01.688 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:41:01.688 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:41:01.737 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:41:01.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.922+0000 7f4485600640 1 -- 172.21.15.94:0/1574364568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4464004010 msgr2=0x7f4464004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:01.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.922+0000 7f4485600640 1 --2- 172.21.15.94:0/1574364568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4464004010 0x7f4464004810 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f446c006fd0 tx=0x7f446c01cf40 comp rx=0 tx=0).stop 2024-09-10T23:41:01.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.922+0000 7f447f400640 1 -- 172.21.15.94:0/1574364568 reap_dead start 2024-09-10T23:41:01.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.922+0000 7f4485600640 1 -- 172.21.15.94:0/1574364568 shutdown_connections 2024-09-10T23:41:01.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.922+0000 7f4485600640 1 -- 172.21.15.94:0/1574364568 >> 172.21.15.94:0/1574364568 conn(0x7f44800d1c60 msgr2=0x7f44800d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:01.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.922+0000 7f4485600640 1 -- 172.21.15.94:0/1574364568 shutdown_connections 2024-09-10T23:41:01.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f4485600640 1 -- 172.21.15.94:0/1574364568 wait complete. 2024-09-10T23:41:01.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f4485600640 1 Processor -- start 2024-09-10T23:41:01.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f4485600640 1 -- start start 2024-09-10T23:41:01.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f4485600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4464004010 0x7f44800af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:01.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f4485600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44800b03a0 0x7f4480155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:01.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f4485600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4480163590 0x7f4480163a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:01.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f4485600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4480168260 con 0x7f4464004010 2024-09-10T23:41:01.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f4485600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f44801680e0 con 0x7f4480163590 2024-09-10T23:41:01.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f4485600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f44801683e0 con 0x7f44800b03a0 2024-09-10T23:41:01.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f447ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4464004010 0x7f44800af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:01.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f447ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4464004010 0x7f44800af560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34308/0 (socket says 172.21.15.94:34308) 2024-09-10T23:41:01.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f447ea00640 1 -- 172.21.15.94:0/3959084904 learned_addr learned my addr 172.21.15.94:0/3959084904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:01.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f447f400640 1 --2- 172.21.15.94:0/3959084904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4480163590 0x7f4480163a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:01.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f447e000640 1 --2- 172.21.15.94:0/3959084904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44800b03a0 0x7f4480155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:01.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f447ea00640 1 -- 172.21.15.94:0/3959084904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44800b03a0 msgr2=0x7f4480155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:01.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f447ea00640 1 --2- 172.21.15.94:0/3959084904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44800b03a0 0x7f4480155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:01.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f447ea00640 1 -- 172.21.15.94:0/3959084904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4480163590 msgr2=0x7f4480163a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:01.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f447ea00640 1 --2- 172.21.15.94:0/3959084904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4480163590 0x7f4480163a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:01.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f447ea00640 1 -- 172.21.15.94:0/3959084904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44800afaa0 con 0x7f4464004010 2024-09-10T23:41:01.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f447f400640 1 --2- 172.21.15.94:0/3959084904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4480163590 0x7f4480163a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:41:01.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f447ea00640 1 --2- 172.21.15.94:0/3959084904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4464004010 0x7f44800af560 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f4470011ae0 tx=0x7f4470009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:01.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f4476a00640 1 -- 172.21.15.94:0/3959084904 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447001e070 con 0x7f4464004010 2024-09-10T23:41:01.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f4476a00640 1 -- 172.21.15.94:0/3959084904 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f447000fe40 con 0x7f4464004010 2024-09-10T23:41:01.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f4476a00640 1 -- 172.21.15.94:0/3959084904 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44700042b0 con 0x7f4464004010 2024-09-10T23:41:01.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.926+0000 7f4485600640 1 -- 172.21.15.94:0/3959084904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44800b0940 con 0x7f4464004010 2024-09-10T23:41:01.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.930+0000 7f4485600640 1 -- 172.21.15.94:0/3959084904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44800c6ad0 con 0x7f4464004010 2024-09-10T23:41:01.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.930+0000 7f4485600640 1 -- 172.21.15.94:0/3959084904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44800b15b0 con 0x7f4464004010 2024-09-10T23:41:01.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.930+0000 7f4476a00640 1 -- 172.21.15.94:0/3959084904 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 97363+0+0 (secure 0 0 0) 0x7f4470003600 con 0x7f4464004010 2024-09-10T23:41:01.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.930+0000 7f4476a00640 1 --2- 172.21.15.94:0/3959084904 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44600730c0 0x7f4460075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:01.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.930+0000 7f4476a00640 1 -- 172.21.15.94:0/3959084904 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f44700956c0 con 0x7f4464004010 2024-09-10T23:41:01.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.930+0000 7f447e000640 1 --2- 172.21.15.94:0/3959084904 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44600730c0 0x7f4460075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:01.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.930+0000 7f447e000640 1 --2- 172.21.15.94:0/3959084904 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44600730c0 0x7f4460075580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f446c0ab000 tx=0x7f446c019d80 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:01.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:01.934+0000 7f4476a00640 1 -- 172.21.15.94:0/3959084904 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f447005f460 con 0x7f4464004010 2024-09-10T23:41:02.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.114+0000 7f4485600640 1 -- 172.21.15.94:0/3959084904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f44800cf130 con 0x7f4464004010 2024-09-10T23:41:02.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.114+0000 7f4476a00640 1 -- 172.21.15.94:0/3959084904 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v162) ==== 80+0+0 (secure 0 0 0) 0x7f4470063e90 con 0x7f4464004010 2024-09-10T23:41:02.115 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:41:02.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.114+0000 7f4485600640 1 -- 172.21.15.94:0/3959084904 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44600730c0 msgr2=0x7f4460075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:02.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.114+0000 7f4485600640 1 --2- 172.21.15.94:0/3959084904 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44600730c0 0x7f4460075580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f446c0ab000 tx=0x7f446c019d80 comp rx=0 tx=0).stop 2024-09-10T23:41:02.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.114+0000 7f4485600640 1 -- 172.21.15.94:0/3959084904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4464004010 msgr2=0x7f44800af560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:02.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.114+0000 7f4485600640 1 --2- 172.21.15.94:0/3959084904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4464004010 0x7f44800af560 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f4470011ae0 tx=0x7f4470009ea0 comp rx=0 tx=0).stop 2024-09-10T23:41:02.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.118+0000 7f4485600640 1 -- 172.21.15.94:0/3959084904 shutdown_connections 2024-09-10T23:41:02.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.118+0000 7f4485600640 1 --2- 172.21.15.94:0/3959084904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4480163590 0x7f4480163a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:02.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.118+0000 7f4485600640 1 --2- 172.21.15.94:0/3959084904 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f44600730c0 0x7f4460075580 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:02.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.118+0000 7f4485600640 1 --2- 172.21.15.94:0/3959084904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44800b03a0 0x7f4480155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:02.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.118+0000 7f4485600640 1 --2- 172.21.15.94:0/3959084904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4464004010 0x7f44800af560 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:02.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.118+0000 7f4485600640 1 -- 172.21.15.94:0/3959084904 >> 172.21.15.94:0/3959084904 conn(0x7f44800d1c60 msgr2=0x7f448015a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:02.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.118+0000 7f4485600640 1 -- 172.21.15.94:0/3959084904 shutdown_connections 2024-09-10T23:41:02.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.118+0000 7f4485600640 1 -- 172.21.15.94:0/3959084904 wait complete. 2024-09-10T23:41:02.137 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-09-10T23:41:02.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:41:02.137 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T23:41:02.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi094.stderr:ceph-fuse[53600]: fuse finished with error 0 and tester_r 0 2024-09-10T23:41:02.158 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:41:02.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:41:02.158 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T23:41:02.326 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:41:02.326 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:41:02.326 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:41:02.335 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:41:02.335 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:41:02.388 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T23:41:02.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e6c600640 1 -- 172.21.15.94:0/2252844205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e640d0710 msgr2=0x7f5e640c5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:02.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e6c600640 1 --2- 172.21.15.94:0/2252844205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e640d0710 0x7f5e640c5530 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f5e640b2780 tx=0x7f5e5001c7d0 comp rx=0 tx=0).stop 2024-09-10T23:41:02.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e6c600640 1 -- 172.21.15.94:0/2252844205 shutdown_connections 2024-09-10T23:41:02.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e6c600640 1 --2- 172.21.15.94:0/2252844205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e640d0710 0x7f5e640c5530 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:02.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e6c600640 1 --2- 172.21.15.94:0/2252844205 >> v2:172.21.15.181:3300/0 conn(0x7f5e641455d0 0x7f5e640c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:02.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e6c600640 1 -- 172.21.15.94:0/2252844205 >> 172.21.15.94:0/2252844205 conn(0x7f5e640cfaf0 msgr2=0x7f5e640cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:02.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e6c600640 1 -- 172.21.15.94:0/2252844205 shutdown_connections 2024-09-10T23:41:02.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e6c600640 1 -- 172.21.15.94:0/2252844205 wait complete. 2024-09-10T23:41:02.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e6c600640 1 Processor -- start 2024-09-10T23:41:02.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e6c600640 1 -- start start 2024-09-10T23:41:02.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e6c600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e640c5a70 0x7f5e641fc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:02.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e6c600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e640d0710 0x7f5e64158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:02.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e6c600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e641455d0 0x7f5e640ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:02.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e6c600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e64169450 con 0x7f5e640c5a70 2024-09-10T23:41:02.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e6c600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641692d0 con 0x7f5e641455d0 2024-09-10T23:41:02.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e6c600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5e641695d0 con 0x7f5e640d0710 2024-09-10T23:41:02.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e69c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e640c5a70 0x7f5e641fc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:02.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e69c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e640c5a70 0x7f5e641fc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34332/0 (socket says 172.21.15.94:34332) 2024-09-10T23:41:02.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e69c00640 1 -- 172.21.15.94:0/2281873063 learned_addr learned my addr 172.21.15.94:0/2281873063 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:02.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e69c00640 1 -- 172.21.15.94:0/2281873063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e640d0710 msgr2=0x7f5e64158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:02.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e69c00640 1 --2- 172.21.15.94:0/2281873063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e640d0710 0x7f5e64158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:02.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e69c00640 1 -- 172.21.15.94:0/2281873063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e641455d0 msgr2=0x7f5e640ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:02.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e69c00640 1 --2- 172.21.15.94:0/2281873063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e641455d0 0x7f5e640ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:02.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.530+0000 7f5e69c00640 1 -- 172.21.15.94:0/2281873063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e640ca5d0 con 0x7f5e640c5a70 2024-09-10T23:41:02.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.534+0000 7f5e69c00640 1 --2- 172.21.15.94:0/2281873063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e640c5a70 0x7f5e641fc3c0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800e7d0 tx=0x7f5e5800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:02.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.534+0000 7f5e61600640 1 -- 172.21.15.94:0/2281873063 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800ccc0 con 0x7f5e640c5a70 2024-09-10T23:41:02.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.534+0000 7f5e61600640 1 -- 172.21.15.94:0/2281873063 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e58010430 con 0x7f5e640c5a70 2024-09-10T23:41:02.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.534+0000 7f5e61600640 1 -- 172.21.15.94:0/2281873063 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58010860 con 0x7f5e640c5a70 2024-09-10T23:41:02.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.534+0000 7f5e6c600640 1 -- 172.21.15.94:0/2281873063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e64073f50 con 0x7f5e640c5a70 2024-09-10T23:41:02.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.534+0000 7f5e6c600640 1 -- 172.21.15.94:0/2281873063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e64158ca0 con 0x7f5e640c5a70 2024-09-10T23:41:02.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.534+0000 7f5e6c600640 1 -- 172.21.15.94:0/2281873063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e64073d40 con 0x7f5e640c5a70 2024-09-10T23:41:02.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.534+0000 7f5e61600640 1 -- 172.21.15.94:0/2281873063 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 97363+0+0 (secure 0 0 0) 0x7f5e58010a00 con 0x7f5e640c5a70 2024-09-10T23:41:02.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.534+0000 7f5e61600640 1 --2- 172.21.15.94:0/2281873063 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5e440730c0 0x7f5e44075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:02.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.534+0000 7f5e61600640 1 -- 172.21.15.94:0/2281873063 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f5e580927e0 con 0x7f5e640c5a70 2024-09-10T23:41:02.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.534+0000 7f5e69200640 1 --2- 172.21.15.94:0/2281873063 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5e440730c0 0x7f5e44075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:02.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.538+0000 7f5e69200640 1 --2- 172.21.15.94:0/2281873063 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5e440730c0 0x7f5e44075580 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5e500af000 tx=0x7f5e50019570 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.538+0000 7f5e61600640 1 -- 172.21.15.94:0/2281873063 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5e5805f6e0 con 0x7f5e640c5a70 2024-09-10T23:41:02.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.714+0000 7f5e6c600640 1 -- 172.21.15.94:0/2281873063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5e640c5530 con 0x7f5e640c5a70 2024-09-10T23:41:02.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.718+0000 7f5e61600640 1 -- 172.21.15.94:0/2281873063 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v162) ==== 80+0+0 (secure 0 0 0) 0x7f5e5805efe0 con 0x7f5e640c5a70 2024-09-10T23:41:02.717 INFO:teuthology.orchestra.run.smithi094.stderr:listed 0 entries 2024-09-10T23:41:02.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.718+0000 7f5e6c600640 1 -- 172.21.15.94:0/2281873063 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5e440730c0 msgr2=0x7f5e44075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:02.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.718+0000 7f5e6c600640 1 --2- 172.21.15.94:0/2281873063 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5e440730c0 0x7f5e44075580 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5e500af000 tx=0x7f5e50019570 comp rx=0 tx=0).stop 2024-09-10T23:41:02.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.718+0000 7f5e6c600640 1 -- 172.21.15.94:0/2281873063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e640c5a70 msgr2=0x7f5e641fc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:02.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.718+0000 7f5e6c600640 1 --2- 172.21.15.94:0/2281873063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e640c5a70 0x7f5e641fc3c0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800e7d0 tx=0x7f5e5800ec90 comp rx=0 tx=0).stop 2024-09-10T23:41:02.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.718+0000 7f5e6c600640 1 -- 172.21.15.94:0/2281873063 shutdown_connections 2024-09-10T23:41:02.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.718+0000 7f5e6c600640 1 --2- 172.21.15.94:0/2281873063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e641455d0 0x7f5e640ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:02.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.718+0000 7f5e6c600640 1 --2- 172.21.15.94:0/2281873063 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5e440730c0 0x7f5e44075580 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:02.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.718+0000 7f5e6c600640 1 --2- 172.21.15.94:0/2281873063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e640d0710 0x7f5e64158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:02.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.718+0000 7f5e6c600640 1 --2- 172.21.15.94:0/2281873063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e640c5a70 0x7f5e641fc3c0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:02.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.718+0000 7f5e6c600640 1 -- 172.21.15.94:0/2281873063 >> 172.21.15.94:0/2281873063 conn(0x7f5e640cfaf0 msgr2=0x7f5e6415b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:02.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.718+0000 7f5e6c600640 1 -- 172.21.15.94:0/2281873063 shutdown_connections 2024-09-10T23:41:02.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:02.718+0000 7f5e6c600640 1 -- 172.21.15.94:0/2281873063 wait complete. 2024-09-10T23:41:02.737 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi181.front.sepia.ceph.com... 2024-09-10T23:41:02.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:41:02.738 DEBUG:teuthology.orchestra.run.smithi181:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T23:41:02.757 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi181.stderr:ceph-fuse[36394]: fuse finished with error 0 and tester_r 0 2024-09-10T23:41:02.759 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:41:02.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:41:02.760 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T23:41:02.925 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:41:02.925 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:41:02.925 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:41:02.934 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:41:02.934 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T23:41:02.934 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:41:02.934 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:41:02.981 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:41:02.982 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:41:03.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ddc00640 1 -- 172.21.15.94:0/4005069339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d80d1690 msgr2=0x7f53d80cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:03.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ddc00640 1 --2- 172.21.15.94:0/4005069339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d80d1690 0x7f53d80cf8d0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f53d8076280 tx=0x7f53d001c770 comp rx=0 tx=0).stop 2024-09-10T23:41:03.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ddc00640 1 -- 172.21.15.94:0/4005069339 shutdown_connections 2024-09-10T23:41:03.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ddc00640 1 --2- 172.21.15.94:0/4005069339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d80d1690 0x7f53d80cf8d0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ddc00640 1 --2- 172.21.15.94:0/4005069339 >> v2:172.21.15.181:3300/0 conn(0x7f53d8071a20 0x7f53d80cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ddc00640 1 -- 172.21.15.94:0/4005069339 >> 172.21.15.94:0/4005069339 conn(0x7f53d80d0b70 msgr2=0x7f53d80d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:03.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ddc00640 1 -- 172.21.15.94:0/4005069339 shutdown_connections 2024-09-10T23:41:03.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ddc00640 1 -- 172.21.15.94:0/4005069339 wait complete. 2024-09-10T23:41:03.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ddc00640 1 Processor -- start 2024-09-10T23:41:03.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ddc00640 1 -- start start 2024-09-10T23:41:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ddc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53d806cc20 0x7f53d8141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ddc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d8071a20 0x7f53d80b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ddc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d80d1690 0x7f53d8158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ddc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f53d8168eb0 con 0x7f53d8071a20 2024-09-10T23:41:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ddc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f53d8168d30 con 0x7f53d80d1690 2024-09-10T23:41:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ddc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f53d8169030 con 0x7f53d806cc20 2024-09-10T23:41:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ce000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d8071a20 0x7f53d80b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ce000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d8071a20 0x7f53d80b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34350/0 (socket says 172.21.15.94:34350) 2024-09-10T23:41:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53ce000640 1 -- 172.21.15.94:0/469995281 learned_addr learned my addr 172.21.15.94:0/469995281 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53d7400640 1 --2- 172.21.15.94:0/469995281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d80d1690 0x7f53d8158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.126+0000 7f53d6a00640 1 --2- 172.21.15.94:0/469995281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53d806cc20 0x7f53d8141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.130+0000 7f53ce000640 1 -- 172.21.15.94:0/469995281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53d806cc20 msgr2=0x7f53d8141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.130+0000 7f53ce000640 1 --2- 172.21.15.94:0/469995281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53d806cc20 0x7f53d8141440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.130+0000 7f53ce000640 1 -- 172.21.15.94:0/469995281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d80d1690 msgr2=0x7f53d8158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.130+0000 7f53ce000640 1 --2- 172.21.15.94:0/469995281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d80d1690 0x7f53d8158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.130+0000 7f53ce000640 1 -- 172.21.15.94:0/469995281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d80b45a0 con 0x7f53d8071a20 2024-09-10T23:41:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.130+0000 7f53d7400640 1 --2- 172.21.15.94:0/469995281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d80d1690 0x7f53d8158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:41:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.130+0000 7f53ce000640 1 --2- 172.21.15.94:0/469995281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d8071a20 0x7f53d80b3bc0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f53c000ea20 tx=0x7f53c000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.130+0000 7f53cf400640 1 -- 172.21.15.94:0/469995281 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c000cd80 con 0x7f53d8071a20 2024-09-10T23:41:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.130+0000 7f53cf400640 1 -- 172.21.15.94:0/469995281 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53c0010430 con 0x7f53d8071a20 2024-09-10T23:41:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.130+0000 7f53cf400640 1 -- 172.21.15.94:0/469995281 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0010860 con 0x7f53d8071a20 2024-09-10T23:41:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.130+0000 7f53ddc00640 1 -- 172.21.15.94:0/469995281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d80b4880 con 0x7f53d8071a20 2024-09-10T23:41:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.130+0000 7f53ddc00640 1 -- 172.21.15.94:0/469995281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d80b4e40 con 0x7f53d8071a20 2024-09-10T23:41:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.130+0000 7f53ddc00640 1 -- 172.21.15.94:0/469995281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d806d000 con 0x7f53d8071a20 2024-09-10T23:41:03.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.130+0000 7f53cf400640 1 -- 172.21.15.94:0/469995281 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f53c0010a00 con 0x7f53d8071a20 2024-09-10T23:41:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.130+0000 7f53cf400640 1 --2- 172.21.15.94:0/469995281 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f53b40731b0 0x7f53b4075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.130+0000 7f53cf400640 1 -- 172.21.15.94:0/469995281 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f53c0092780 con 0x7f53d8071a20 2024-09-10T23:41:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.134+0000 7f53d6a00640 1 --2- 172.21.15.94:0/469995281 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f53b40731b0 0x7f53b4075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.134+0000 7f53d6a00640 1 --2- 172.21.15.94:0/469995281 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f53b40731b0 0x7f53b4075670 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f53c80021a0 tx=0x7f53c8009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.134+0000 7f53cf400640 1 -- 172.21.15.94:0/469995281 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f53c005f910 con 0x7f53d8071a20 2024-09-10T23:41:03.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.334+0000 7f53ddc00640 1 -- 172.21.15.94:0/469995281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53d8071f00 con 0x7f53d8071a20 2024-09-10T23:41:03.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.334+0000 7f53cf400640 1 -- 172.21.15.94:0/469995281 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+5239 (secure 0 0 0) 0x7f53c005f210 con 0x7f53d8071a20 2024-09-10T23:41:03.336 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:41:03.336 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":117,"btime":"2024-09-10T23:40:54:547608+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/2653759764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2653759764},{"type":"v1","addr":"172.21.15.181:6801","nonce":2653759764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":8518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2240199080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2240199080},{"type":"v1","addr":"172.21.15.181:6835","nonce":2240199080}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1675026105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1675026105},{"type":"v1","addr":"172.21.15.94:6834","nonce":1675026105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:40:44.453069+0000","modified":"2024-09-10T23:40:47.471636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9171":{"gid":9171,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/3607539278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3607539278},{"type":"v1","addr":"172.21.15.94:6835","nonce":3607539278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9171,"qdb_cluster":[9171]},"id":13}]} 2024-09-10T23:41:03.336 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 117 2024-09-10T23:41:03.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.338+0000 7f53ddc00640 1 -- 172.21.15.94:0/469995281 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f53b40731b0 msgr2=0x7f53b4075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:03.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.338+0000 7f53ddc00640 1 --2- 172.21.15.94:0/469995281 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f53b40731b0 0x7f53b4075670 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f53c80021a0 tx=0x7f53c8009040 comp rx=0 tx=0).stop 2024-09-10T23:41:03.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.338+0000 7f53ddc00640 1 -- 172.21.15.94:0/469995281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d8071a20 msgr2=0x7f53d80b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:03.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.338+0000 7f53ddc00640 1 --2- 172.21.15.94:0/469995281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d8071a20 0x7f53d80b3bc0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f53c000ea20 tx=0x7f53c000eee0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.338+0000 7f53ddc00640 1 -- 172.21.15.94:0/469995281 shutdown_connections 2024-09-10T23:41:03.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.338+0000 7f53ddc00640 1 --2- 172.21.15.94:0/469995281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d80d1690 0x7f53d8158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.338+0000 7f53ddc00640 1 --2- 172.21.15.94:0/469995281 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f53b40731b0 0x7f53b4075670 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.338+0000 7f53ddc00640 1 --2- 172.21.15.94:0/469995281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53d8071a20 0x7f53d80b3bc0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.338+0000 7f53ddc00640 1 --2- 172.21.15.94:0/469995281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53d806cc20 0x7f53d8141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.338+0000 7f53ddc00640 1 -- 172.21.15.94:0/469995281 >> 172.21.15.94:0/469995281 conn(0x7f53d80d0b70 msgr2=0x7f53d815d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:03.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.338+0000 7f53ddc00640 1 -- 172.21.15.94:0/469995281 shutdown_connections 2024-09-10T23:41:03.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.338+0000 7f53ddc00640 1 -- 172.21.15.94:0/469995281 wait complete. 2024-09-10T23:41:03.358 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:41:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.498+0000 7ff540200640 1 -- 172.21.15.94:0/1073900398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5380ae7e0 msgr2=0x7ff5380b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.498+0000 7ff540200640 1 --2- 172.21.15.94:0/1073900398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5380ae7e0 0x7ff5380b44d0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7ff524009900 tx=0x7ff52401c790 comp rx=0 tx=0).stop 2024-09-10T23:41:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.498+0000 7ff540200640 1 -- 172.21.15.94:0/1073900398 shutdown_connections 2024-09-10T23:41:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.498+0000 7ff540200640 1 --2- 172.21.15.94:0/1073900398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5380ae7e0 0x7ff5380b44d0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.498+0000 7ff540200640 1 --2- 172.21.15.94:0/1073900398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff53806ce80 0x7ff5380d1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.498+0000 7ff540200640 1 -- 172.21.15.94:0/1073900398 >> 172.21.15.94:0/1073900398 conn(0x7ff5380d0b70 msgr2=0x7ff5380d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.498+0000 7ff540200640 1 -- 172.21.15.94:0/1073900398 shutdown_connections 2024-09-10T23:41:03.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.498+0000 7ff540200640 1 -- 172.21.15.94:0/1073900398 wait complete. 2024-09-10T23:41:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.498+0000 7ff540200640 1 Processor -- start 2024-09-10T23:41:03.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.498+0000 7ff540200640 1 -- start start 2024-09-10T23:41:03.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff540200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff53806ce80 0x7ff53814afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:03.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff540200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5380afec0 0x7ff538149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:03.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff540200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff538073690 0x7ff538073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:03.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff540200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff53816b570 con 0x7ff538073690 2024-09-10T23:41:03.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff540200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff53816b3f0 con 0x7ff53806ce80 2024-09-10T23:41:03.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff540200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff53816b6f0 con 0x7ff5380afec0 2024-09-10T23:41:03.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff53e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff538073690 0x7ff538073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:03.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff53e200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff538073690 0x7ff538073b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34374/0 (socket says 172.21.15.94:34374) 2024-09-10T23:41:03.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff53e200640 1 -- 172.21.15.94:0/2081659950 learned_addr learned my addr 172.21.15.94:0/2081659950 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:03.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff53d800640 1 --2- 172.21.15.94:0/2081659950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff53806ce80 0x7ff53814afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:03.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff53ce00640 1 --2- 172.21.15.94:0/2081659950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5380afec0 0x7ff538149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:03.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff53e200640 1 -- 172.21.15.94:0/2081659950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5380afec0 msgr2=0x7ff538149a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:03.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff53e200640 1 --2- 172.21.15.94:0/2081659950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5380afec0 0x7ff538149a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff53e200640 1 -- 172.21.15.94:0/2081659950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff53806ce80 msgr2=0x7ff53814afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:03.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff53e200640 1 --2- 172.21.15.94:0/2081659950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff53806ce80 0x7ff53814afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff53e200640 1 -- 172.21.15.94:0/2081659950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53814b500 con 0x7ff538073690 2024-09-10T23:41:03.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff53e200640 1 --2- 172.21.15.94:0/2081659950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff538073690 0x7ff538073b20 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7ff53000b930 tx=0x7ff53000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:03.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff535600640 1 -- 172.21.15.94:0/2081659950 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff530019070 con 0x7ff538073690 2024-09-10T23:41:03.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff535600640 1 -- 172.21.15.94:0/2081659950 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff53000cc50 con 0x7ff538073690 2024-09-10T23:41:03.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff540200640 1 -- 172.21.15.94:0/2081659950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5381fbe80 con 0x7ff538073690 2024-09-10T23:41:03.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff535600640 1 -- 172.21.15.94:0/2081659950 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5300146a0 con 0x7ff538073690 2024-09-10T23:41:03.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff540200640 1 -- 172.21.15.94:0/2081659950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff53814a4e0 con 0x7ff538073690 2024-09-10T23:41:03.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff540200640 1 -- 172.21.15.94:0/2081659950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff538149fa0 con 0x7ff538073690 2024-09-10T23:41:03.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.502+0000 7ff535600640 1 -- 172.21.15.94:0/2081659950 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7ff530014840 con 0x7ff538073690 2024-09-10T23:41:03.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.506+0000 7ff535600640 1 --2- 172.21.15.94:0/2081659950 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff5180731e0 0x7ff5180756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:03.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.506+0000 7ff535600640 1 -- 172.21.15.94:0/2081659950 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7ff530091cc0 con 0x7ff538073690 2024-09-10T23:41:03.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.506+0000 7ff53d800640 1 --2- 172.21.15.94:0/2081659950 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff5180731e0 0x7ff5180756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:03.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.506+0000 7ff53d800640 1 --2- 172.21.15.94:0/2081659950 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff5180731e0 0x7ff5180756a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009990 tx=0x7ff52c009110 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:03.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.510+0000 7ff535600640 1 -- 172.21.15.94:0/2081659950 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff53005e540 con 0x7ff538073690 2024-09-10T23:41:03.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.706+0000 7ff540200640 1 -- 172.21.15.94:0/2081659950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5380d1690 con 0x7ff538073690 2024-09-10T23:41:03.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.706+0000 7ff535600640 1 -- 172.21.15.94:0/2081659950 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+5239 (secure 0 0 0) 0x7ff530062f70 con 0x7ff538073690 2024-09-10T23:41:03.707 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:41:03.707 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":117,"btime":"2024-09-10T23:40:54:547608+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/2653759764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2653759764},{"type":"v1","addr":"172.21.15.181:6801","nonce":2653759764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":8518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2240199080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2240199080},{"type":"v1","addr":"172.21.15.181:6835","nonce":2240199080}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1675026105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1675026105},{"type":"v1","addr":"172.21.15.94:6834","nonce":1675026105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:40:44.453069+0000","modified":"2024-09-10T23:40:47.471636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9171":{"gid":9171,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/3607539278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3607539278},{"type":"v1","addr":"172.21.15.94:6835","nonce":3607539278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9171,"qdb_cluster":[9171]},"id":13}]} 2024-09-10T23:41:03.708 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 117 2024-09-10T23:41:03.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.710+0000 7ff540200640 1 -- 172.21.15.94:0/2081659950 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff5180731e0 msgr2=0x7ff5180756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:03.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.710+0000 7ff540200640 1 --2- 172.21.15.94:0/2081659950 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff5180731e0 0x7ff5180756a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009990 tx=0x7ff52c009110 comp rx=0 tx=0).stop 2024-09-10T23:41:03.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.710+0000 7ff540200640 1 -- 172.21.15.94:0/2081659950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff538073690 msgr2=0x7ff538073b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:03.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.710+0000 7ff540200640 1 --2- 172.21.15.94:0/2081659950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff538073690 0x7ff538073b20 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7ff53000b930 tx=0x7ff53000bdf0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.710+0000 7ff540200640 1 -- 172.21.15.94:0/2081659950 shutdown_connections 2024-09-10T23:41:03.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.710+0000 7ff540200640 1 --2- 172.21.15.94:0/2081659950 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff5180731e0 0x7ff5180756a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.710+0000 7ff540200640 1 --2- 172.21.15.94:0/2081659950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff538073690 0x7ff538073b20 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.710+0000 7ff540200640 1 --2- 172.21.15.94:0/2081659950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5380afec0 0x7ff538149a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.710+0000 7ff540200640 1 --2- 172.21.15.94:0/2081659950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff53806ce80 0x7ff53814afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.710+0000 7ff540200640 1 -- 172.21.15.94:0/2081659950 >> 172.21.15.94:0/2081659950 conn(0x7ff5380d0b70 msgr2=0x7ff538161910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:03.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.710+0000 7ff540200640 1 -- 172.21.15.94:0/2081659950 shutdown_connections 2024-09-10T23:41:03.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.710+0000 7ff540200640 1 -- 172.21.15.94:0/2081659950 wait complete. 2024-09-10T23:41:03.729 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:41:03.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.870+0000 7fbfdca00640 1 -- 172.21.15.94:0/2087167950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfbc003a60 msgr2=0x7fbfbc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:03.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.870+0000 7fbfdca00640 1 --2- 172.21.15.94:0/2087167950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfbc003a60 0x7fbfbc003e80 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8006fd0 tx=0x7fbfc801cf00 comp rx=0 tx=0).stop 2024-09-10T23:41:03.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.870+0000 7fbfdca00640 1 -- 172.21.15.94:0/2087167950 shutdown_connections 2024-09-10T23:41:03.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.870+0000 7fbfdca00640 1 --2- 172.21.15.94:0/2087167950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfbc003a60 0x7fbfbc003e80 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.870+0000 7fbfdca00640 1 -- 172.21.15.94:0/2087167950 >> 172.21.15.94:0/2087167950 conn(0x7fbfd80d1c60 msgr2=0x7fbfd80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:03.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.870+0000 7fbfdca00640 1 -- 172.21.15.94:0/2087167950 shutdown_connections 2024-09-10T23:41:03.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.870+0000 7fbfdca00640 1 -- 172.21.15.94:0/2087167950 wait complete. 2024-09-10T23:41:03.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfdca00640 1 Processor -- start 2024-09-10T23:41:03.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfdca00640 1 -- start start 2024-09-10T23:41:03.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfdca00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfbc003a60 0x7fbfd81fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:03.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfdca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd80b3490 0x7fbfd80b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfdca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfd80c8d30 0x7fbfd80c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfdca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd81622c0 con 0x7fbfd80c8d30 2024-09-10T23:41:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfdca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd8162140 con 0x7fbfd80b3490 2024-09-10T23:41:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfdca00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd8162440 con 0x7fbfbc003a60 2024-09-10T23:41:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfd6800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfd80c8d30 0x7fbfd80c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfd6800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfd80c8d30 0x7fbfd80c91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43158/0 (socket says 172.21.15.94:43158) 2024-09-10T23:41:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfd6800640 1 -- 172.21.15.94:0/3916336631 learned_addr learned my addr 172.21.15.94:0/3916336631 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfcd400640 1 --2- 172.21.15.94:0/3916336631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd80b3490 0x7fbfd80b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfd5e00640 1 --2- 172.21.15.94:0/3916336631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfbc003a60 0x7fbfd81fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfd6800640 1 -- 172.21.15.94:0/3916336631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfbc003a60 msgr2=0x7fbfd81fb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfd6800640 1 --2- 172.21.15.94:0/3916336631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfbc003a60 0x7fbfd81fb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfd6800640 1 -- 172.21.15.94:0/3916336631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd80b3490 msgr2=0x7fbfd80b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:03.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfd6800640 1 --2- 172.21.15.94:0/3916336631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd80b3490 0x7fbfd80b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:03.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfd6800640 1 -- 172.21.15.94:0/3916336631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd80ca680 con 0x7fbfd80c8d30 2024-09-10T23:41:03.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfd5e00640 1 --2- 172.21.15.94:0/3916336631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfbc003a60 0x7fbfd81fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:41:03.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfd6800640 1 --2- 172.21.15.94:0/3916336631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfd80c8d30 0x7fbfd80c91e0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8069940 tx=0x7fbfd000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:03.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfcea00640 1 -- 172.21.15.94:0/3916336631 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0004b30 con 0x7fbfd80c8d30 2024-09-10T23:41:03.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfcea00640 1 -- 172.21.15.94:0/3916336631 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfd0004cd0 con 0x7fbfd80c8d30 2024-09-10T23:41:03.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfcea00640 1 -- 172.21.15.94:0/3916336631 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0003be0 con 0x7fbfd80c8d30 2024-09-10T23:41:03.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfdca00640 1 -- 172.21.15.94:0/3916336631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd81fc140 con 0x7fbfd80c8d30 2024-09-10T23:41:03.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfdca00640 1 -- 172.21.15.94:0/3916336631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd81fc700 con 0x7fbfd80c8d30 2024-09-10T23:41:03.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.874+0000 7fbfdca00640 1 -- 172.21.15.94:0/3916336631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd80cf150 con 0x7fbfd80c8d30 2024-09-10T23:41:03.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.878+0000 7fbfcea00640 1 -- 172.21.15.94:0/3916336631 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7fbfd0018070 con 0x7fbfd80c8d30 2024-09-10T23:41:03.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.878+0000 7fbfcea00640 1 --2- 172.21.15.94:0/3916336631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbfb40731e0 0x7fbfb40756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:03.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.878+0000 7fbfcea00640 1 -- 172.21.15.94:0/3916336631 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7fbfd00927e0 con 0x7fbfd80c8d30 2024-09-10T23:41:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.878+0000 7fbfd5e00640 1 --2- 172.21.15.94:0/3916336631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbfb40731e0 0x7fbfb40756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.878+0000 7fbfd5e00640 1 --2- 172.21.15.94:0/3916336631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbfb40731e0 0x7fbfb40756a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80ab000 tx=0x7fbfc8019d40 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:03.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:03.882+0000 7fbfcea00640 1 -- 172.21.15.94:0/3916336631 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbfd005f710 con 0x7fbfd80c8d30 2024-09-10T23:41:04.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.050+0000 7fbfdca00640 1 -- 172.21.15.94:0/3916336631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbfd80749c0 con 0x7fbfd80c8d30 2024-09-10T23:41:04.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.054+0000 7fbfcea00640 1 -- 172.21.15.94:0/3916336631 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) ==== 74+0+16707 (secure 0 0 0) 0x7fbfd005f010 con 0x7fbfd80c8d30 2024-09-10T23:41:04.053 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:41:04.054 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":162,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:40:46.071176+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:40:42.141193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-10T23:40:43.422826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:41:04.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.054+0000 7fbfdca00640 1 -- 172.21.15.94:0/3916336631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbfb40731e0 msgr2=0x7fbfb40756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:04.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.054+0000 7fbfdca00640 1 --2- 172.21.15.94:0/3916336631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbfb40731e0 0x7fbfb40756a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80ab000 tx=0x7fbfc8019d40 comp rx=0 tx=0).stop 2024-09-10T23:41:04.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.054+0000 7fbfdca00640 1 -- 172.21.15.94:0/3916336631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfd80c8d30 msgr2=0x7fbfd80c91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:04.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.054+0000 7fbfdca00640 1 --2- 172.21.15.94:0/3916336631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfd80c8d30 0x7fbfd80c91e0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8069940 tx=0x7fbfd000f3c0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.054+0000 7fbfdca00640 1 -- 172.21.15.94:0/3916336631 shutdown_connections 2024-09-10T23:41:04.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.054+0000 7fbfdca00640 1 --2- 172.21.15.94:0/3916336631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfd80b3490 0x7fbfd80b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.054+0000 7fbfdca00640 1 --2- 172.21.15.94:0/3916336631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbfb40731e0 0x7fbfb40756a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.054+0000 7fbfdca00640 1 --2- 172.21.15.94:0/3916336631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfd80c8d30 0x7fbfd80c91e0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.054+0000 7fbfdca00640 1 --2- 172.21.15.94:0/3916336631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfbc003a60 0x7fbfd81fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.054+0000 7fbfdca00640 1 -- 172.21.15.94:0/3916336631 >> 172.21.15.94:0/3916336631 conn(0x7fbfd80d1c60 msgr2=0x7fbfd80afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:04.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.054+0000 7fbfdca00640 1 -- 172.21.15.94:0/3916336631 shutdown_connections 2024-09-10T23:41:04.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.054+0000 7fbfdca00640 1 -- 172.21.15.94:0/3916336631 wait complete. 2024-09-10T23:41:04.075 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T23:41:04.075 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:41:04.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.218+0000 7f6479400640 1 -- 172.21.15.94:0/573663056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6458004810 msgr2=0x7f6458004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.218+0000 7f6479400640 1 --2- 172.21.15.94:0/573663056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6458004810 0x7f6458004c90 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f6460009b90 tx=0x7f646001d270 comp rx=0 tx=0).stop 2024-09-10T23:41:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.218+0000 7f6478a00640 1 -- 172.21.15.94:0/573663056 reap_dead start 2024-09-10T23:41:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.218+0000 7f6479400640 1 -- 172.21.15.94:0/573663056 shutdown_connections 2024-09-10T23:41:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.218+0000 7f6479400640 1 -- 172.21.15.94:0/573663056 >> 172.21.15.94:0/573663056 conn(0x7f647409aa70 msgr2=0x7f647409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.218+0000 7f6479400640 1 -- 172.21.15.94:0/573663056 shutdown_connections 2024-09-10T23:41:04.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.218+0000 7f6479400640 1 -- 172.21.15.94:0/573663056 wait complete. 2024-09-10T23:41:04.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6479400640 1 Processor -- start 2024-09-10T23:41:04.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6479400640 1 -- start start 2024-09-10T23:41:04.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6479400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6458004810 0x7f6474157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:04.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6479400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6474148680 0x7f647415aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:04.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6479400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64741f22a0 0x7f64741f1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:04.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6479400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f64740b34d0 con 0x7f6458004810 2024-09-10T23:41:04.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6479400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f64740b3350 con 0x7f6474148680 2024-09-10T23:41:04.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6479400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f64740d01e0 con 0x7f64741f22a0 2024-09-10T23:41:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6473e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6458004810 0x7f6474157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6473e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6458004810 0x7f6474157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43186/0 (socket says 172.21.15.94:43186) 2024-09-10T23:41:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6473e00640 1 -- 172.21.15.94:0/3231237194 learned_addr learned my addr 172.21.15.94:0/3231237194 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6478a00640 1 --2- 172.21.15.94:0/3231237194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64741f22a0 0x7f64741f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6473400640 1 --2- 172.21.15.94:0/3231237194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6474148680 0x7f647415aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6473e00640 1 -- 172.21.15.94:0/3231237194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64741f22a0 msgr2=0x7f64741f1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6473e00640 1 --2- 172.21.15.94:0/3231237194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64741f22a0 0x7f64741f1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6473e00640 1 -- 172.21.15.94:0/3231237194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6474148680 msgr2=0x7f647415aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6473e00640 1 --2- 172.21.15.94:0/3231237194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6474148680 0x7f647415aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6473e00640 1 -- 172.21.15.94:0/3231237194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6474159a30 con 0x7f6458004810 2024-09-10T23:41:04.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6473e00640 1 --2- 172.21.15.94:0/3231237194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6458004810 0x7f6474157fe0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f6464001750 tx=0x7f646400e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f646be00640 1 -- 172.21.15.94:0/3231237194 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6464012850 con 0x7f6458004810 2024-09-10T23:41:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f646be00640 1 -- 172.21.15.94:0/3231237194 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6464012ae0 con 0x7f6458004810 2024-09-10T23:41:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f646be00640 1 -- 172.21.15.94:0/3231237194 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646401a400 con 0x7f6458004810 2024-09-10T23:41:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6479400640 1 -- 172.21.15.94:0/3231237194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6474159cd0 con 0x7f6458004810 2024-09-10T23:41:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6479400640 1 -- 172.21.15.94:0/3231237194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f647415a290 con 0x7f6458004810 2024-09-10T23:41:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.222+0000 7f6479400640 1 -- 172.21.15.94:0/3231237194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64740cd6d0 con 0x7f6458004810 2024-09-10T23:41:04.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.226+0000 7f646be00640 1 -- 172.21.15.94:0/3231237194 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f64640113d0 con 0x7f6458004810 2024-09-10T23:41:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.226+0000 7f646be00640 1 --2- 172.21.15.94:0/3231237194 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64540731b0 0x7f6454075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:04.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.226+0000 7f646be00640 1 -- 172.21.15.94:0/3231237194 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f6464095250 con 0x7f6458004810 2024-09-10T23:41:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.226+0000 7f6473400640 1 --2- 172.21.15.94:0/3231237194 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64540731b0 0x7f6454075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:04.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.226+0000 7f6473400640 1 --2- 172.21.15.94:0/3231237194 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64540731b0 0x7f6454075670 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f64600ae000 tx=0x7f6458005840 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:04.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.230+0000 7f646be00640 1 -- 172.21.15.94:0/3231237194 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f646409b030 con 0x7f6458004810 2024-09-10T23:41:04.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.426+0000 7f6479400640 1 -- 172.21.15.94:0/3231237194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f647415a660 con 0x7f6458004810 2024-09-10T23:41:04.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.430+0000 7f646be00640 1 -- 172.21.15.94:0/3231237194 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+5239 (secure 0 0 0) 0x7f6464016030 con 0x7f6458004810 2024-09-10T23:41:04.429 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:41:04.430 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":117,"btime":"2024-09-10T23:40:54:547608+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/2653759764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2653759764},{"type":"v1","addr":"172.21.15.181:6801","nonce":2653759764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":8518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2240199080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2240199080},{"type":"v1","addr":"172.21.15.181:6835","nonce":2240199080}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1675026105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1675026105},{"type":"v1","addr":"172.21.15.94:6834","nonce":1675026105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:40:44.453069+0000","modified":"2024-09-10T23:40:47.471636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9171":{"gid":9171,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/3607539278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3607539278},{"type":"v1","addr":"172.21.15.94:6835","nonce":3607539278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9171,"qdb_cluster":[9171]},"id":13}]} 2024-09-10T23:41:04.430 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 117 2024-09-10T23:41:04.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.430+0000 7f6479400640 1 -- 172.21.15.94:0/3231237194 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64540731b0 msgr2=0x7f6454075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.430+0000 7f6479400640 1 --2- 172.21.15.94:0/3231237194 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64540731b0 0x7f6454075670 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f64600ae000 tx=0x7f6458005840 comp rx=0 tx=0).stop 2024-09-10T23:41:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.430+0000 7f6479400640 1 -- 172.21.15.94:0/3231237194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6458004810 msgr2=0x7f6474157fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.430+0000 7f6479400640 1 --2- 172.21.15.94:0/3231237194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6458004810 0x7f6474157fe0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f6464001750 tx=0x7f646400e6b0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.430+0000 7f6479400640 1 -- 172.21.15.94:0/3231237194 shutdown_connections 2024-09-10T23:41:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.430+0000 7f6479400640 1 --2- 172.21.15.94:0/3231237194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6474148680 0x7f647415aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.430+0000 7f6479400640 1 --2- 172.21.15.94:0/3231237194 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f64540731b0 0x7f6454075670 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.430+0000 7f6479400640 1 --2- 172.21.15.94:0/3231237194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64741f22a0 0x7f64741f1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.430+0000 7f6479400640 1 --2- 172.21.15.94:0/3231237194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6458004810 0x7f6474157fe0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.430+0000 7f6479400640 1 -- 172.21.15.94:0/3231237194 >> 172.21.15.94:0/3231237194 conn(0x7f647409aa70 msgr2=0x7f64740b0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:04.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.430+0000 7f6479400640 1 -- 172.21.15.94:0/3231237194 shutdown_connections 2024-09-10T23:41:04.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.430+0000 7f6479400640 1 -- 172.21.15.94:0/3231237194 wait complete. 2024-09-10T23:41:04.450 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:41:04.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.590+0000 7f1591200640 1 -- 172.21.15.94:0/2949821698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f158c0b2bb0 msgr2=0x7f158c0d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:04.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.590+0000 7f1591200640 1 --2- 172.21.15.94:0/2949821698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f158c0b2bb0 0x7f158c0d0270 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f158c15f9a0 tx=0x7f157401c7a0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.590+0000 7f1591200640 1 -- 172.21.15.94:0/2949821698 shutdown_connections 2024-09-10T23:41:04.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.590+0000 7f1591200640 1 --2- 172.21.15.94:0/2949821698 >> v2:172.21.15.181:3300/0 conn(0x7f158c0af310 0x7f158c0af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.590+0000 7f1591200640 1 --2- 172.21.15.94:0/2949821698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f158c0b2bb0 0x7f158c0d0270 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.590+0000 7f1591200640 1 -- 172.21.15.94:0/2949821698 >> 172.21.15.94:0/2949821698 conn(0x7f158c0d1c60 msgr2=0x7f158c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:04.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.590+0000 7f1591200640 1 -- 172.21.15.94:0/2949821698 shutdown_connections 2024-09-10T23:41:04.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.590+0000 7f1591200640 1 -- 172.21.15.94:0/2949821698 wait complete. 2024-09-10T23:41:04.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1591200640 1 Processor -- start 2024-09-10T23:41:04.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1591200640 1 -- start start 2024-09-10T23:41:04.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1591200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f158c0af310 0x7f158c0c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:04.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1591200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f158c0b2bb0 0x7f158c0711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:04.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1591200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f158c15f5c0 0x7f158c0716e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:04.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1591200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1690b0 con 0x7f158c15f5c0 2024-09-10T23:41:04.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1591200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f158c168f30 con 0x7f158c0b2bb0 2024-09-10T23:41:04.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1591200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f158c169230 con 0x7f158c0af310 2024-09-10T23:41:04.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f158a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f158c15f5c0 0x7f158c0716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:04.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f158a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f158c15f5c0 0x7f158c0716e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43208/0 (socket says 172.21.15.94:43208) 2024-09-10T23:41:04.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f158a800640 1 -- 172.21.15.94:0/528179506 learned_addr learned my addr 172.21.15.94:0/528179506 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:04.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1583e00640 1 --2- 172.21.15.94:0/528179506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f158c0b2bb0 0x7f158c0711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:04.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1589e00640 1 --2- 172.21.15.94:0/528179506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f158c0af310 0x7f158c0c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:04.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f158a800640 1 -- 172.21.15.94:0/528179506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f158c0af310 msgr2=0x7f158c0c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:04.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f158a800640 1 --2- 172.21.15.94:0/528179506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f158c0af310 0x7f158c0c7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f158a800640 1 -- 172.21.15.94:0/528179506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f158c0b2bb0 msgr2=0x7f158c0711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:04.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f158a800640 1 --2- 172.21.15.94:0/528179506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f158c0b2bb0 0x7f158c0711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f158a800640 1 -- 172.21.15.94:0/528179506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158c0c84a0 con 0x7f158c15f5c0 2024-09-10T23:41:04.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1589e00640 1 --2- 172.21.15.94:0/528179506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f158c0af310 0x7f158c0c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:41:04.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f158a800640 1 --2- 172.21.15.94:0/528179506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f158c15f5c0 0x7f158c0716e0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f158400b9a0 tx=0x7f158400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:04.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1582000640 1 -- 172.21.15.94:0/528179506 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584019070 con 0x7f158c15f5c0 2024-09-10T23:41:04.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1582000640 1 -- 172.21.15.94:0/528179506 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f158400ccb0 con 0x7f158c15f5c0 2024-09-10T23:41:04.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1591200640 1 -- 172.21.15.94:0/528179506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158c14cbe0 con 0x7f158c15f5c0 2024-09-10T23:41:04.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1582000640 1 -- 172.21.15.94:0/528179506 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15840146a0 con 0x7f158c15f5c0 2024-09-10T23:41:04.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1591200640 1 -- 172.21.15.94:0/528179506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158c14c8d0 con 0x7f158c15f5c0 2024-09-10T23:41:04.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.594+0000 7f1591200640 1 -- 172.21.15.94:0/528179506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f158c0af7a0 con 0x7f158c15f5c0 2024-09-10T23:41:04.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.598+0000 7f1582000640 1 -- 172.21.15.94:0/528179506 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f158401d3b0 con 0x7f158c15f5c0 2024-09-10T23:41:04.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.598+0000 7f1582000640 1 --2- 172.21.15.94:0/528179506 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f15680731e0 0x7f15680756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:04.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.598+0000 7f1582000640 1 -- 172.21.15.94:0/528179506 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f1584091e70 con 0x7f158c15f5c0 2024-09-10T23:41:04.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.598+0000 7f1589e00640 1 --2- 172.21.15.94:0/528179506 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f15680731e0 0x7f15680756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:04.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.598+0000 7f1589e00640 1 --2- 172.21.15.94:0/528179506 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f15680731e0 0x7f15680756a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f158c0b7be0 tx=0x7f157c008040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:04.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.602+0000 7f1582000640 1 -- 172.21.15.94:0/528179506 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f158405ed40 con 0x7f158c15f5c0 2024-09-10T23:41:04.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.798+0000 7f1591200640 1 -- 172.21.15.94:0/528179506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f158c0749c0 con 0x7f158c15f5c0 2024-09-10T23:41:04.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.798+0000 7f1582000640 1 -- 172.21.15.94:0/528179506 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+5239 (secure 0 0 0) 0x7f158405e640 con 0x7f158c15f5c0 2024-09-10T23:41:04.801 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:41:04.801 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":117,"btime":"2024-09-10T23:40:54:547608+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/2653759764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2653759764},{"type":"v1","addr":"172.21.15.181:6801","nonce":2653759764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":8518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2240199080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2240199080},{"type":"v1","addr":"172.21.15.181:6835","nonce":2240199080}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1675026105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1675026105},{"type":"v1","addr":"172.21.15.94:6834","nonce":1675026105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T23:40:44.453069+0000","modified":"2024-09-10T23:40:47.471636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9171":{"gid":9171,"name":"a","rank":0,"incarnation":113,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6835/3607539278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":3607539278},{"type":"v1","addr":"172.21.15.94:6835","nonce":3607539278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9171,"qdb_cluster":[9171]},"id":13}]} 2024-09-10T23:41:04.801 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 117 2024-09-10T23:41:04.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.802+0000 7f1591200640 1 -- 172.21.15.94:0/528179506 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f15680731e0 msgr2=0x7f15680756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:04.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.802+0000 7f1591200640 1 --2- 172.21.15.94:0/528179506 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f15680731e0 0x7f15680756a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f158c0b7be0 tx=0x7f157c008040 comp rx=0 tx=0).stop 2024-09-10T23:41:04.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.802+0000 7f1591200640 1 -- 172.21.15.94:0/528179506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f158c15f5c0 msgr2=0x7f158c0716e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:04.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.802+0000 7f1591200640 1 --2- 172.21.15.94:0/528179506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f158c15f5c0 0x7f158c0716e0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f158400b9a0 tx=0x7f158400be60 comp rx=0 tx=0).stop 2024-09-10T23:41:04.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.802+0000 7f1591200640 1 -- 172.21.15.94:0/528179506 shutdown_connections 2024-09-10T23:41:04.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.802+0000 7f1591200640 1 --2- 172.21.15.94:0/528179506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f158c0b2bb0 0x7f158c0711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.802+0000 7f1591200640 1 --2- 172.21.15.94:0/528179506 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f15680731e0 0x7f15680756a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.802+0000 7f1591200640 1 --2- 172.21.15.94:0/528179506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f158c15f5c0 0x7f158c0716e0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.802+0000 7f1591200640 1 --2- 172.21.15.94:0/528179506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f158c0af310 0x7f158c0c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.802+0000 7f1591200640 1 -- 172.21.15.94:0/528179506 >> 172.21.15.94:0/528179506 conn(0x7f158c0d1c60 msgr2=0x7f158c15d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:04.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.802+0000 7f1591200640 1 -- 172.21.15.94:0/528179506 shutdown_connections 2024-09-10T23:41:04.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.802+0000 7f1591200640 1 -- 172.21.15.94:0/528179506 wait complete. 2024-09-10T23:41:04.821 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:41:04.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.962+0000 7f25a2000640 1 -- 172.21.15.94:0/2174238380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f259c15f5c0 msgr2=0x7f259c0cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:04.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.962+0000 7f25a2000640 1 --2- 172.21.15.94:0/2174238380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f259c15f5c0 0x7f259c0cfb60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f258c009840 tx=0x7f258c01c700 comp rx=0 tx=0).stop 2024-09-10T23:41:04.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.962+0000 7f25a2000640 1 -- 172.21.15.94:0/2174238380 shutdown_connections 2024-09-10T23:41:04.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.962+0000 7f25a2000640 1 --2- 172.21.15.94:0/2174238380 >> v2:172.21.15.181:3300/0 conn(0x7f259c0b2bb0 0x7f259c0d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.962+0000 7f25a2000640 1 --2- 172.21.15.94:0/2174238380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f259c15f5c0 0x7f259c0cfb60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.962+0000 7f25a2000640 1 -- 172.21.15.94:0/2174238380 >> 172.21.15.94:0/2174238380 conn(0x7f259c0d1c60 msgr2=0x7f259c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:04.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.962+0000 7f25a2000640 1 -- 172.21.15.94:0/2174238380 shutdown_connections 2024-09-10T23:41:04.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.962+0000 7f25a2000640 1 -- 172.21.15.94:0/2174238380 wait complete. 2024-09-10T23:41:04.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f25a2000640 1 Processor -- start 2024-09-10T23:41:04.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f25a2000640 1 -- start start 2024-09-10T23:41:04.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f25a2000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f259c0b2bb0 0x7f259c06f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:04.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f25a2000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f259c0d05e0 0x7f259c1471d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:04.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f25a2000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f259c15f5c0 0x7f259c0c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:04.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f25a2000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f259c169250 con 0x7f259c15f5c0 2024-09-10T23:41:04.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f25a2000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f259c1690d0 con 0x7f259c0b2bb0 2024-09-10T23:41:04.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f25a2000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f259c1693d0 con 0x7f259c0d05e0 2024-09-10T23:41:04.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f259be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f259c15f5c0 0x7f259c0c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:04.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f259be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f259c15f5c0 0x7f259c0c5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43216/0 (socket says 172.21.15.94:43216) 2024-09-10T23:41:04.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f259be00640 1 -- 172.21.15.94:0/2298306961 learned_addr learned my addr 172.21.15.94:0/2298306961 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:04.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f259aa00640 1 --2- 172.21.15.94:0/2298306961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f259c0d05e0 0x7f259c1471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:04.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f259b400640 1 --2- 172.21.15.94:0/2298306961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f259c0b2bb0 0x7f259c06f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:04.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f259be00640 1 -- 172.21.15.94:0/2298306961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f259c0d05e0 msgr2=0x7f259c1471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:04.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f259be00640 1 --2- 172.21.15.94:0/2298306961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f259c0d05e0 0x7f259c1471d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f259be00640 1 -- 172.21.15.94:0/2298306961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f259c0b2bb0 msgr2=0x7f259c06f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:04.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f259be00640 1 --2- 172.21.15.94:0/2298306961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f259c0b2bb0 0x7f259c06f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:04.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f259be00640 1 -- 172.21.15.94:0/2298306961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259c0c5cd0 con 0x7f259c15f5c0 2024-09-10T23:41:04.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f259be00640 1 --2- 172.21.15.94:0/2298306961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f259c15f5c0 0x7f259c0c5790 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f259400c760 tx=0x7f259400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:04.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f2593400640 1 -- 172.21.15.94:0/2298306961 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2594003db0 con 0x7f259c15f5c0 2024-09-10T23:41:04.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f2593400640 1 -- 172.21.15.94:0/2298306961 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2594005070 con 0x7f259c15f5c0 2024-09-10T23:41:04.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f25a2000640 1 -- 172.21.15.94:0/2298306961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259c1547e0 con 0x7f259c15f5c0 2024-09-10T23:41:04.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f2593400640 1 -- 172.21.15.94:0/2298306961 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25940054c0 con 0x7f259c15f5c0 2024-09-10T23:41:04.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f25a2000640 1 -- 172.21.15.94:0/2298306961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f259c150060 con 0x7f259c15f5c0 2024-09-10T23:41:04.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.966+0000 7f25a2000640 1 -- 172.21.15.94:0/2298306961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f259c0b15c0 con 0x7f259c15f5c0 2024-09-10T23:41:04.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.970+0000 7f2593400640 1 -- 172.21.15.94:0/2298306961 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f2594005660 con 0x7f259c15f5c0 2024-09-10T23:41:04.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.970+0000 7f2593400640 1 --2- 172.21.15.94:0/2298306961 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f257c0731b0 0x7f257c075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:04.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.970+0000 7f2593400640 1 -- 172.21.15.94:0/2298306961 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f2594091bc0 con 0x7f259c15f5c0 2024-09-10T23:41:04.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.970+0000 7f259b400640 1 --2- 172.21.15.94:0/2298306961 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f257c0731b0 0x7f257c075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:04.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.970+0000 7f259b400640 1 --2- 172.21.15.94:0/2298306961 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f257c0731b0 0x7f257c075670 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f258c0af000 tx=0x7f258c01af40 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:04.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:04.974+0000 7f2593400640 1 -- 172.21.15.94:0/2298306961 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f259405e440 con 0x7f259c15f5c0 2024-09-10T23:41:05.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.142+0000 7f25a2000640 1 -- 172.21.15.94:0/2298306961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f259c0749c0 con 0x7f259c15f5c0 2024-09-10T23:41:05.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.142+0000 7f2593400640 1 -- 172.21.15.94:0/2298306961 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) ==== 74+0+16707 (secure 0 0 0) 0x7f25940671b0 con 0x7f259c15f5c0 2024-09-10T23:41:05.145 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:41:05.145 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":162,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:40:46.071176+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-10T23:40:42.141193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-10T23:40:43.422826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:41:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.146+0000 7f25a2000640 1 -- 172.21.15.94:0/2298306961 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f257c0731b0 msgr2=0x7f257c075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.146+0000 7f25a2000640 1 --2- 172.21.15.94:0/2298306961 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f257c0731b0 0x7f257c075670 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f258c0af000 tx=0x7f258c01af40 comp rx=0 tx=0).stop 2024-09-10T23:41:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.146+0000 7f25a2000640 1 -- 172.21.15.94:0/2298306961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f259c15f5c0 msgr2=0x7f259c0c5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.146+0000 7f25a2000640 1 --2- 172.21.15.94:0/2298306961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f259c15f5c0 0x7f259c0c5790 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f259400c760 tx=0x7f259400cc20 comp rx=0 tx=0).stop 2024-09-10T23:41:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.146+0000 7f25a2000640 1 -- 172.21.15.94:0/2298306961 shutdown_connections 2024-09-10T23:41:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.146+0000 7f25a2000640 1 --2- 172.21.15.94:0/2298306961 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f257c0731b0 0x7f257c075670 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.146+0000 7f25a2000640 1 --2- 172.21.15.94:0/2298306961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f259c15f5c0 0x7f259c0c5790 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:05.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.146+0000 7f25a2000640 1 --2- 172.21.15.94:0/2298306961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f259c0d05e0 0x7f259c1471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:05.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.146+0000 7f25a2000640 1 --2- 172.21.15.94:0/2298306961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f259c0b2bb0 0x7f259c06f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:05.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.146+0000 7f25a2000640 1 -- 172.21.15.94:0/2298306961 >> 172.21.15.94:0/2298306961 conn(0x7f259c0d1c60 msgr2=0x7f259c156490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:05.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.146+0000 7f25a2000640 1 -- 172.21.15.94:0/2298306961 shutdown_connections 2024-09-10T23:41:05.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.146+0000 7f25a2000640 1 -- 172.21.15.94:0/2298306961 wait complete. 2024-09-10T23:41:05.167 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T23:41:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.310+0000 7f5f73e00640 1 -- 172.21.15.94:0/1113528800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f54004810 msgr2=0x7f5f54004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.310+0000 7f5f73e00640 1 --2- 172.21.15.94:0/1113528800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f54004810 0x7f5f54004c90 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f5f58001770 tx=0x7f5f5801c820 comp rx=0 tx=0).stop 2024-09-10T23:41:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.310+0000 7f5f73e00640 1 -- 172.21.15.94:0/1113528800 shutdown_connections 2024-09-10T23:41:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.310+0000 7f5f73e00640 1 --2- 172.21.15.94:0/1113528800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f54004810 0x7f5f54004c90 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.310+0000 7f5f73e00640 1 -- 172.21.15.94:0/1113528800 >> 172.21.15.94:0/1113528800 conn(0x7f5f6c0d1c90 msgr2=0x7f5f6c0d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.310+0000 7f5f73e00640 1 -- 172.21.15.94:0/1113528800 shutdown_connections 2024-09-10T23:41:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.310+0000 7f5f73e00640 1 -- 172.21.15.94:0/1113528800 wait complete. 2024-09-10T23:41:05.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.310+0000 7f5f73e00640 1 Processor -- start 2024-09-10T23:41:05.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.310+0000 7f5f73e00640 1 -- start start 2024-09-10T23:41:05.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f73e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f54004810 0x7f5f6c0c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:05.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f73e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f6c0ced90 0x7f5f6c14e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:05.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f73e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c0c60e0 0x7f5f6c0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:05.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f73e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c15f6b0 con 0x7f5f6c0c60e0 2024-09-10T23:41:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f73e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c0b25b0 con 0x7f5f6c0ced90 2024-09-10T23:41:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f73e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6c15f830 con 0x7f5f54004810 2024-09-10T23:41:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f71e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c0c60e0 0x7f5f6c0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f71e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c0c60e0 0x7f5f6c0cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43228/0 (socket says 172.21.15.94:43228) 2024-09-10T23:41:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f71e00640 1 -- 172.21.15.94:0/1878145243 learned_addr learned my addr 172.21.15.94:0/1878145243 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f70a00640 1 --2- 172.21.15.94:0/1878145243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f6c0ced90 0x7f5f6c14e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f71400640 1 --2- 172.21.15.94:0/1878145243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f54004810 0x7f5f6c0c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:05.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f71e00640 1 -- 172.21.15.94:0/1878145243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f54004810 msgr2=0x7f5f6c0c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:05.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f71e00640 1 --2- 172.21.15.94:0/1878145243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f54004810 0x7f5f6c0c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:05.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f71e00640 1 -- 172.21.15.94:0/1878145243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f6c0ced90 msgr2=0x7f5f6c14e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:05.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f71e00640 1 --2- 172.21.15.94:0/1878145243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f6c0ced90 0x7f5f6c14e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:05.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f71e00640 1 -- 172.21.15.94:0/1878145243 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6c0c9000 con 0x7f5f6c0c60e0 2024-09-10T23:41:05.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f71e00640 1 --2- 172.21.15.94:0/1878145243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c0c60e0 0x7f5f6c0cadc0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c073790 tx=0x7f5f64012aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:05.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f69600640 1 -- 172.21.15.94:0/1878145243 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64013920 con 0x7f5f6c0c60e0 2024-09-10T23:41:05.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f69600640 1 -- 172.21.15.94:0/1878145243 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f64013b00 con 0x7f5f6c0c60e0 2024-09-10T23:41:05.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f69600640 1 -- 172.21.15.94:0/1878145243 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64010480 con 0x7f5f6c0c60e0 2024-09-10T23:41:05.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f73e00640 1 -- 172.21.15.94:0/1878145243 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6c0c6590 con 0x7f5f6c0c60e0 2024-09-10T23:41:05.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f73e00640 1 -- 172.21.15.94:0/1878145243 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f6c0cca40 con 0x7f5f6c0c60e0 2024-09-10T23:41:05.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f73e00640 1 -- 172.21.15.94:0/1878145243 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f6c0b1870 con 0x7f5f6c0c60e0 2024-09-10T23:41:05.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.314+0000 7f5f69600640 1 -- 172.21.15.94:0/1878145243 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f5f64010620 con 0x7f5f6c0c60e0 2024-09-10T23:41:05.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.318+0000 7f5f69600640 1 --2- 172.21.15.94:0/1878145243 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f4c0731b0 0x7f5f4c075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:05.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.318+0000 7f5f69600640 1 -- 172.21.15.94:0/1878145243 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5737+0+0 (secure 0 0 0) 0x7f5f64095bd0 con 0x7f5f6c0c60e0 2024-09-10T23:41:05.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.318+0000 7f5f71400640 1 --2- 172.21.15.94:0/1878145243 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f4c0731b0 0x7f5f4c075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:05.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.318+0000 7f5f71400640 1 --2- 172.21.15.94:0/1878145243 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f4c0731b0 0x7f5f4c075670 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c15a120 tx=0x7f5f60009500 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:05.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.322+0000 7f5f69600640 1 -- 172.21.15.94:0/1878145243 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5f64062450 con 0x7f5f6c0c60e0 2024-09-10T23:41:05.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.518+0000 7f5f73e00640 1 -- 172.21.15.94:0/1878145243 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5f6c071760 con 0x7f5f6c0c60e0 2024-09-10T23:41:05.519 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-10T23:41:05.518+0000 7f4045a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T23:41:05.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.522+0000 7f5f69600640 1 -- 172.21.15.94:0/1878145243 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v118) ==== 161+0+0 (secure 0 0 0) 0x7f5f64066e80 con 0x7f5f6c0c60e0 2024-09-10T23:41:05.521 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T23:41:05.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.522+0000 7f5f73e00640 1 -- 172.21.15.94:0/1878145243 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f4c0731b0 msgr2=0x7f5f4c075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:05.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.522+0000 7f5f73e00640 1 --2- 172.21.15.94:0/1878145243 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f4c0731b0 0x7f5f4c075670 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c15a120 tx=0x7f5f60009500 comp rx=0 tx=0).stop 2024-09-10T23:41:05.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.522+0000 7f5f73e00640 1 -- 172.21.15.94:0/1878145243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c0c60e0 msgr2=0x7f5f6c0cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:05.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.522+0000 7f5f73e00640 1 --2- 172.21.15.94:0/1878145243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c0c60e0 0x7f5f6c0cadc0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c073790 tx=0x7f5f64012aa0 comp rx=0 tx=0).stop 2024-09-10T23:41:05.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.522+0000 7f5f73e00640 1 -- 172.21.15.94:0/1878145243 shutdown_connections 2024-09-10T23:41:05.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.522+0000 7f5f73e00640 1 --2- 172.21.15.94:0/1878145243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f6c0ced90 0x7f5f6c14e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:05.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.522+0000 7f5f73e00640 1 --2- 172.21.15.94:0/1878145243 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f5f4c0731b0 0x7f5f4c075670 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:05.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.522+0000 7f5f73e00640 1 --2- 172.21.15.94:0/1878145243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f6c0c60e0 0x7f5f6c0cadc0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:05.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.522+0000 7f5f73e00640 1 --2- 172.21.15.94:0/1878145243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f54004810 0x7f5f6c0c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:05.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.522+0000 7f5f73e00640 1 -- 172.21.15.94:0/1878145243 >> 172.21.15.94:0/1878145243 conn(0x7f5f6c0d1c90 msgr2=0x7f5f6c0b7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:05.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.526+0000 7f5f73e00640 1 -- 172.21.15.94:0/1878145243 shutdown_connections 2024-09-10T23:41:05.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.526+0000 7f5f73e00640 1 -- 172.21.15.94:0/1878145243 wait complete. 2024-09-10T23:41:05.545 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T23:41:05.590 INFO:tasks.ceph.mds.a.smithi094.stdout:starting mds.a at 2024-09-10T23:41:05.591 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:41:05.595 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:41:05.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.690+0000 7f4203000640 1 -- 172.21.15.94:0/3966907559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41fc06e7c0 msgr2=0x7f41fc06ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:05.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.690+0000 7f4203000640 1 --2- 172.21.15.94:0/3966907559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41fc06e7c0 0x7f41fc06ebf0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f41fc069670 tx=0x7f41f401db30 comp rx=0 tx=0).stop 2024-09-10T23:41:05.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.690+0000 7f4203000640 1 -- 172.21.15.94:0/3966907559 shutdown_connections 2024-09-10T23:41:05.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.690+0000 7f4203000640 1 --2- 172.21.15.94:0/3966907559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41fc06e7c0 0x7f41fc06ebf0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:05.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.690+0000 7f4203000640 1 --2- 172.21.15.94:0/3966907559 >> v2:172.21.15.181:3300/0 conn(0x7f41fc0ce280 0x7f41fc0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:05.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.690+0000 7f4203000640 1 -- 172.21.15.94:0/3966907559 >> 172.21.15.94:0/3966907559 conn(0x7f41fc0cfaf0 msgr2=0x7f41fc0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:05.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.690+0000 7f4203000640 1 -- 172.21.15.94:0/3966907559 shutdown_connections 2024-09-10T23:41:05.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.690+0000 7f4203000640 1 -- 172.21.15.94:0/3966907559 wait complete. 2024-09-10T23:41:05.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f4203000640 1 Processor -- start 2024-09-10T23:41:05.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f4203000640 1 -- start start 2024-09-10T23:41:05.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f4203000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41fc06e7c0 0x7f41fc0b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:05.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f4203000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41fc0c64f0 0x7f41fc142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:05.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f4203000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41fc0ce280 0x7f41fc145740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:05.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f4203000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc15b5c0 con 0x7f41fc0c64f0 2024-09-10T23:41:05.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f4203000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc15b440 con 0x7f41fc06e7c0 2024-09-10T23:41:05.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f4203000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f41fc15b740 con 0x7f41fc0ce280 2024-09-10T23:41:05.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f41fb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41fc0c64f0 0x7f41fc142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:05.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f41fb400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41fc0c64f0 0x7f41fc142c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43278/0 (socket says 172.21.15.94:43278) 2024-09-10T23:41:05.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f41fb400640 1 -- 172.21.15.94:0/458888250 learned_addr learned my addr 172.21.15.94:0/458888250 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:05.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f41fb400640 1 -- 172.21.15.94:0/458888250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41fc0ce280 msgr2=0x7f41fc145740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:05.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f41fb400640 1 --2- 172.21.15.94:0/458888250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41fc0ce280 0x7f41fc145740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:05.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f41fb400640 1 -- 172.21.15.94:0/458888250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41fc06e7c0 msgr2=0x7f41fc0b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:05.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f41fb400640 1 --2- 172.21.15.94:0/458888250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41fc06e7c0 0x7f41fc0b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:05.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f41fb400640 1 -- 172.21.15.94:0/458888250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc0b5070 con 0x7f41fc0c64f0 2024-09-10T23:41:05.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f41fb400640 1 --2- 172.21.15.94:0/458888250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41fc0c64f0 0x7f41fc142c60 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f41e800c420 tx=0x7f41e800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:05.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f41f3e00640 1 -- 172.21.15.94:0/458888250 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8017070 con 0x7f41fc0c64f0 2024-09-10T23:41:05.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f41f3e00640 1 -- 172.21.15.94:0/458888250 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41e8003600 con 0x7f41fc0c64f0 2024-09-10T23:41:05.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f41f3e00640 1 -- 172.21.15.94:0/458888250 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8003a90 con 0x7f41fc0c64f0 2024-09-10T23:41:05.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f4203000640 1 -- 172.21.15.94:0/458888250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41fc0b5350 con 0x7f41fc0c64f0 2024-09-10T23:41:05.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.694+0000 7f4203000640 1 -- 172.21.15.94:0/458888250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41fc0b5910 con 0x7f41fc0c64f0 2024-09-10T23:41:05.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.698+0000 7f4203000640 1 -- 172.21.15.94:0/458888250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41fc1ec290 con 0x7f41fc0c64f0 2024-09-10T23:41:05.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.698+0000 7f41f3e00640 1 -- 172.21.15.94:0/458888250 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f41e8003c30 con 0x7f41fc0c64f0 2024-09-10T23:41:05.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.698+0000 7f41f3e00640 1 --2- 172.21.15.94:0/458888250 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f41dc0731b0 0x7f41dc075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:05.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.698+0000 7f41f3e00640 1 -- 172.21.15.94:0/458888250 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 5823+0+0 (secure 0 0 0) 0x7f41e8091490 con 0x7f41fc0c64f0 2024-09-10T23:41:05.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.698+0000 7f41fbe00640 1 --2- 172.21.15.94:0/458888250 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f41dc0731b0 0x7f41dc075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:05.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.698+0000 7f41fbe00640 1 --2- 172.21.15.94:0/458888250 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f41dc0731b0 0x7f41dc075670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f41ec005c50 tx=0x7f41ec005be0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:05.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.702+0000 7f41f3e00640 1 -- 172.21.15.94:0/458888250 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f41fc1ec290 con 0x7f41fc0c64f0 2024-09-10T23:41:05.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:05.902+0000 7f4203000640 1 -- 172.21.15.94:0/458888250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f41fc06ebf0 con 0x7f41fc0c64f0 2024-09-10T23:41:06.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.522+0000 7f41f3e00640 1 -- 172.21.15.94:0/458888250 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v119) ==== 104+0+0 (secure 0 0 0) 0x7f41e805e360 con 0x7f41fc0c64f0 2024-09-10T23:41:06.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.526+0000 7f4203000640 1 -- 172.21.15.94:0/458888250 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f41dc0731b0 msgr2=0x7f41dc075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:06.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.526+0000 7f4203000640 1 --2- 172.21.15.94:0/458888250 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f41dc0731b0 0x7f41dc075670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f41ec005c50 tx=0x7f41ec005be0 comp rx=0 tx=0).stop 2024-09-10T23:41:06.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.526+0000 7f4203000640 1 -- 172.21.15.94:0/458888250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41fc0c64f0 msgr2=0x7f41fc142c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:06.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.526+0000 7f4203000640 1 --2- 172.21.15.94:0/458888250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41fc0c64f0 0x7f41fc142c60 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f41e800c420 tx=0x7f41e800c8e0 comp rx=0 tx=0).stop 2024-09-10T23:41:06.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.526+0000 7f4203000640 1 -- 172.21.15.94:0/458888250 shutdown_connections 2024-09-10T23:41:06.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.526+0000 7f4203000640 1 --2- 172.21.15.94:0/458888250 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f41dc0731b0 0x7f41dc075670 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:06.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.526+0000 7f4203000640 1 --2- 172.21.15.94:0/458888250 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41fc0ce280 0x7f41fc145740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:06.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.526+0000 7f4203000640 1 --2- 172.21.15.94:0/458888250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41fc0c64f0 0x7f41fc142c60 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:06.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.526+0000 7f4203000640 1 --2- 172.21.15.94:0/458888250 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41fc06e7c0 0x7f41fc0b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:06.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.526+0000 7f4203000640 1 -- 172.21.15.94:0/458888250 >> 172.21.15.94:0/458888250 conn(0x7f41fc0cfaf0 msgr2=0x7f41fc0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:06.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.526+0000 7f4203000640 1 -- 172.21.15.94:0/458888250 shutdown_connections 2024-09-10T23:41:06.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.526+0000 7f4203000640 1 -- 172.21.15.94:0/458888250 wait complete. 2024-09-10T23:41:06.548 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T23:41:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a30e00640 1 -- 172.21.15.94:0/1525758234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a2c0d03c0 msgr2=0x7f9a2c0cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a30e00640 1 --2- 172.21.15.94:0/1525758234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a2c0d03c0 0x7f9a2c0cddb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9a24009bc0 tx=0x7f9a2401cae0 comp rx=0 tx=0).stop 2024-09-10T23:41:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a30e00640 1 -- 172.21.15.94:0/1525758234 shutdown_connections 2024-09-10T23:41:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a30e00640 1 --2- 172.21.15.94:0/1525758234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a2c0d03c0 0x7f9a2c0cddb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a30e00640 1 --2- 172.21.15.94:0/1525758234 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a2c15ba50 0x7f9a2c0b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a30e00640 1 -- 172.21.15.94:0/1525758234 >> 172.21.15.94:0/1525758234 conn(0x7f9a2c06cdf0 msgr2=0x7f9a2c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a30e00640 1 -- 172.21.15.94:0/1525758234 shutdown_connections 2024-09-10T23:41:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a30e00640 1 -- 172.21.15.94:0/1525758234 wait complete. 2024-09-10T23:41:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a30e00640 1 Processor -- start 2024-09-10T23:41:06.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a30e00640 1 -- start start 2024-09-10T23:41:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a30e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a2c0b4c40 0x7f9a2c164af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:06.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a30e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a2c0d03c0 0x7f9a2c147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a30e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a2c15ba50 0x7f9a2c14e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a30e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c168be0 con 0x7f9a2c0b4c40 2024-09-10T23:41:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a30e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c168a60 con 0x7f9a2c15ba50 2024-09-10T23:41:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a30e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9a2c168d60 con 0x7f9a2c0d03c0 2024-09-10T23:41:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a29e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a2c0b4c40 0x7f9a2c164af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a29e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a2c0b4c40 0x7f9a2c164af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43288/0 (socket says 172.21.15.94:43288) 2024-09-10T23:41:06.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.694+0000 7f9a29e00640 1 -- 172.21.15.94:0/1463802939 learned_addr learned my addr 172.21.15.94:0/1463802939 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:06.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.698+0000 7f9a2a800640 1 --2- 172.21.15.94:0/1463802939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a2c15ba50 0x7f9a2c14e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:06.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.698+0000 7f9a29400640 1 --2- 172.21.15.94:0/1463802939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a2c0d03c0 0x7f9a2c147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:06.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.698+0000 7f9a29e00640 1 -- 172.21.15.94:0/1463802939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a2c0d03c0 msgr2=0x7f9a2c147510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:06.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.698+0000 7f9a29e00640 1 --2- 172.21.15.94:0/1463802939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a2c0d03c0 0x7f9a2c147510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:06.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.698+0000 7f9a29e00640 1 -- 172.21.15.94:0/1463802939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a2c15ba50 msgr2=0x7f9a2c14e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:06.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.698+0000 7f9a29e00640 1 --2- 172.21.15.94:0/1463802939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a2c15ba50 0x7f9a2c14e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:06.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.698+0000 7f9a29e00640 1 -- 172.21.15.94:0/1463802939 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c165030 con 0x7f9a2c0b4c40 2024-09-10T23:41:06.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.698+0000 7f9a29e00640 1 --2- 172.21.15.94:0/1463802939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a2c0b4c40 0x7f9a2c164af0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00e970 tx=0x7f9a1c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:06.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.698+0000 7f9a22000640 1 -- 172.21.15.94:0/1463802939 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c00cd80 con 0x7f9a2c0b4c40 2024-09-10T23:41:06.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.698+0000 7f9a22000640 1 -- 172.21.15.94:0/1463802939 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a1c004020 con 0x7f9a2c0b4c40 2024-09-10T23:41:06.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.698+0000 7f9a30e00640 1 -- 172.21.15.94:0/1463802939 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2c1652d0 con 0x7f9a2c0b4c40 2024-09-10T23:41:06.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.698+0000 7f9a22000640 1 -- 172.21.15.94:0/1463802939 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c004470 con 0x7f9a2c0b4c40 2024-09-10T23:41:06.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.698+0000 7f9a30e00640 1 -- 172.21.15.94:0/1463802939 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2c165890 con 0x7f9a2c0b4c40 2024-09-10T23:41:06.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.698+0000 7f9a30e00640 1 -- 172.21.15.94:0/1463802939 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a2c0cddb0 con 0x7f9a2c0b4c40 2024-09-10T23:41:06.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.698+0000 7f9a22000640 1 -- 172.21.15.94:0/1463802939 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f9a1c003600 con 0x7f9a2c0b4c40 2024-09-10T23:41:06.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.702+0000 7f9a22000640 1 --2- 172.21.15.94:0/1463802939 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9a0c0731b0 0x7f9a0c075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:06.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.702+0000 7f9a22000640 1 -- 172.21.15.94:0/1463802939 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 5823+0+0 (secure 0 0 0) 0x7f9a1c092590 con 0x7f9a2c0b4c40 2024-09-10T23:41:06.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.702+0000 7f9a29400640 1 --2- 172.21.15.94:0/1463802939 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9a0c0731b0 0x7f9a0c075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:06.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.702+0000 7f9a29400640 1 --2- 172.21.15.94:0/1463802939 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9a0c0731b0 0x7f9a0c075670 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0b2710 tx=0x7f9a18009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:06.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.706+0000 7f9a22000640 1 -- 172.21.15.94:0/1463802939 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9a1c05f6c0 con 0x7f9a2c0b4c40 2024-09-10T23:41:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:06.874+0000 7f9a30e00640 1 -- 172.21.15.94:0/1463802939 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9a2c155d20 con 0x7f9a2c0b4c40 2024-09-10T23:41:07.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.526+0000 7f9a22000640 1 -- 172.21.15.94:0/1463802939 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v164) ==== 181+0+0 (secure 0 0 0) 0x7f9a1c05efc0 con 0x7f9a2c0b4c40 2024-09-10T23:41:07.529 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-09-10T23:41:07.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.530+0000 7f9a30e00640 1 -- 172.21.15.94:0/1463802939 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9a0c0731b0 msgr2=0x7f9a0c075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:07.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.530+0000 7f9a30e00640 1 --2- 172.21.15.94:0/1463802939 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9a0c0731b0 0x7f9a0c075670 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0b2710 tx=0x7f9a18009040 comp rx=0 tx=0).stop 2024-09-10T23:41:07.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.530+0000 7f9a30e00640 1 -- 172.21.15.94:0/1463802939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a2c0b4c40 msgr2=0x7f9a2c164af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:07.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.530+0000 7f9a30e00640 1 --2- 172.21.15.94:0/1463802939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a2c0b4c40 0x7f9a2c164af0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00e970 tx=0x7f9a1c00ee30 comp rx=0 tx=0).stop 2024-09-10T23:41:07.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.530+0000 7f9a30e00640 1 -- 172.21.15.94:0/1463802939 shutdown_connections 2024-09-10T23:41:07.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.530+0000 7f9a30e00640 1 --2- 172.21.15.94:0/1463802939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a2c15ba50 0x7f9a2c14e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:07.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.530+0000 7f9a30e00640 1 --2- 172.21.15.94:0/1463802939 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9a0c0731b0 0x7f9a0c075670 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:07.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.530+0000 7f9a30e00640 1 --2- 172.21.15.94:0/1463802939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a2c0d03c0 0x7f9a2c147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:07.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.530+0000 7f9a30e00640 1 --2- 172.21.15.94:0/1463802939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a2c0b4c40 0x7f9a2c164af0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:07.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.530+0000 7f9a30e00640 1 -- 172.21.15.94:0/1463802939 >> 172.21.15.94:0/1463802939 conn(0x7f9a2c06cdf0 msgr2=0x7f9a2c14a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:07.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.530+0000 7f9a30e00640 1 -- 172.21.15.94:0/1463802939 shutdown_connections 2024-09-10T23:41:07.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.530+0000 7f9a30e00640 1 -- 172.21.15.94:0/1463802939 wait complete. 2024-09-10T23:41:07.552 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T23:41:07.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e42600640 1 -- 172.21.15.94:0/3084007087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e200052f0 msgr2=0x7f8e2000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:07.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e42600640 1 --2- 172.21.15.94:0/3084007087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e200052f0 0x7f8e2000b9d0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400c610 tx=0x7f8e3401d260 comp rx=0 tx=0).stop 2024-09-10T23:41:07.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e42600640 1 -- 172.21.15.94:0/3084007087 shutdown_connections 2024-09-10T23:41:07.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e42600640 1 --2- 172.21.15.94:0/3084007087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e200052f0 0x7f8e2000b9d0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:07.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e42600640 1 -- 172.21.15.94:0/3084007087 >> 172.21.15.94:0/3084007087 conn(0x7f8e3c06cdf0 msgr2=0x7f8e3c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:07.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e42600640 1 -- 172.21.15.94:0/3084007087 shutdown_connections 2024-09-10T23:41:07.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e42600640 1 -- 172.21.15.94:0/3084007087 wait complete. 2024-09-10T23:41:07.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e42600640 1 Processor -- start 2024-09-10T23:41:07.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e42600640 1 -- start start 2024-09-10T23:41:07.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e42600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e200052f0 0x7f8e3c153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:07.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e42600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e3c154500 0x7f8e3c1f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:07.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e42600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e3c1f5f60 0x7f8e3c159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:07.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e42600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c0cff20 con 0x7f8e200052f0 2024-09-10T23:41:07.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e42600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c0cfda0 con 0x7f8e3c1f5f60 2024-09-10T23:41:07.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e42600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f8e3c0d00a0 con 0x7f8e3c154500 2024-09-10T23:41:07.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e3b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e200052f0 0x7f8e3c153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:07.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e3b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e200052f0 0x7f8e3c153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43306/0 (socket says 172.21.15.94:43306) 2024-09-10T23:41:07.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e3b400640 1 -- 172.21.15.94:0/2770943331 learned_addr learned my addr 172.21.15.94:0/2770943331 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:07.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e3b400640 1 -- 172.21.15.94:0/2770943331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e3c154500 msgr2=0x7f8e3c1f5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:07.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e3b400640 1 --2- 172.21.15.94:0/2770943331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e3c154500 0x7f8e3c1f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:07.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e3b400640 1 -- 172.21.15.94:0/2770943331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e3c1f5f60 msgr2=0x7f8e3c159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:07.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e3b400640 1 --2- 172.21.15.94:0/2770943331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e3c1f5f60 0x7f8e3c159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:07.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.702+0000 7f8e3b400640 1 -- 172.21.15.94:0/2770943331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3c1554b0 con 0x7f8e200052f0 2024-09-10T23:41:07.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.706+0000 7f8e3b400640 1 --2- 172.21.15.94:0/2770943331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e200052f0 0x7f8e3c153fc0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00f800 tx=0x7f8e2c00fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:07.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.706+0000 7f8e33e00640 1 -- 172.21.15.94:0/2770943331 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c00bcb0 con 0x7f8e200052f0 2024-09-10T23:41:07.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.706+0000 7f8e33e00640 1 -- 172.21.15.94:0/2770943331 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e2c011440 con 0x7f8e200052f0 2024-09-10T23:41:07.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.706+0000 7f8e42600640 1 -- 172.21.15.94:0/2770943331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3c1f5490 con 0x7f8e200052f0 2024-09-10T23:41:07.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.706+0000 7f8e33e00640 1 -- 172.21.15.94:0/2770943331 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c011870 con 0x7f8e200052f0 2024-09-10T23:41:07.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.706+0000 7f8e42600640 1 -- 172.21.15.94:0/2770943331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e3c1537c0 con 0x7f8e200052f0 2024-09-10T23:41:07.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.706+0000 7f8e42600640 1 -- 172.21.15.94:0/2770943331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e3c15e980 con 0x7f8e200052f0 2024-09-10T23:41:07.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.706+0000 7f8e33e00640 1 -- 172.21.15.94:0/2770943331 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f8e2c023030 con 0x7f8e200052f0 2024-09-10T23:41:07.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.706+0000 7f8e33e00640 1 --2- 172.21.15.94:0/2770943331 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8e180731b0 0x7f8e18075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:07.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.706+0000 7f8e33e00640 1 -- 172.21.15.94:0/2770943331 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 5385+0+0 (secure 0 0 0) 0x7f8e2c092400 con 0x7f8e200052f0 2024-09-10T23:41:07.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.706+0000 7f8e32a00640 1 --2- 172.21.15.94:0/2770943331 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8e180731b0 0x7f8e18075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:07.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.710+0000 7f8e32a00640 1 --2- 172.21.15.94:0/2770943331 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8e180731b0 0x7f8e18075670 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c15dfe0 tx=0x7f8e24006a60 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:07.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.710+0000 7f8e33e00640 1 -- 172.21.15.94:0/2770943331 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8e2c05f530 con 0x7f8e200052f0 2024-09-10T23:41:07.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:07.882+0000 7f8e42600640 1 -- 172.21.15.94:0/2770943331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8e3c15a9b0 con 0x7f8e200052f0 2024-09-10T23:41:08.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.530+0000 7f8e33e00640 1 -- 172.21.15.94:0/2770943331 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v165) ==== 169+0+0 (secure 0 0 0) 0x7f8e3c15a9b0 con 0x7f8e200052f0 2024-09-10T23:41:08.531 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-09-10T23:41:08.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.534+0000 7f8e42600640 1 -- 172.21.15.94:0/2770943331 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8e180731b0 msgr2=0x7f8e18075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:08.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.534+0000 7f8e42600640 1 --2- 172.21.15.94:0/2770943331 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8e180731b0 0x7f8e18075670 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c15dfe0 tx=0x7f8e24006a60 comp rx=0 tx=0).stop 2024-09-10T23:41:08.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.534+0000 7f8e42600640 1 -- 172.21.15.94:0/2770943331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e200052f0 msgr2=0x7f8e3c153fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:08.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.534+0000 7f8e42600640 1 --2- 172.21.15.94:0/2770943331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e200052f0 0x7f8e3c153fc0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00f800 tx=0x7f8e2c00fcc0 comp rx=0 tx=0).stop 2024-09-10T23:41:08.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.534+0000 7f8e42600640 1 -- 172.21.15.94:0/2770943331 shutdown_connections 2024-09-10T23:41:08.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.534+0000 7f8e42600640 1 --2- 172.21.15.94:0/2770943331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e3c1f5f60 0x7f8e3c159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:08.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.534+0000 7f8e42600640 1 --2- 172.21.15.94:0/2770943331 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f8e180731b0 0x7f8e18075670 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:08.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.534+0000 7f8e42600640 1 --2- 172.21.15.94:0/2770943331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e3c154500 0x7f8e3c1f5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:08.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.534+0000 7f8e42600640 1 --2- 172.21.15.94:0/2770943331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e200052f0 0x7f8e3c153fc0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.534+0000 7f8e42600640 1 -- 172.21.15.94:0/2770943331 >> 172.21.15.94:0/2770943331 conn(0x7f8e3c06cdf0 msgr2=0x7f8e3c1e7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.534+0000 7f8e42600640 1 -- 172.21.15.94:0/2770943331 shutdown_connections 2024-09-10T23:41:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.534+0000 7f8e42600640 1 -- 172.21.15.94:0/2770943331 wait complete. 2024-09-10T23:41:08.557 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T23:41:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.698+0000 7ff19b200640 1 -- 172.21.15.94:0/3028084183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff19406d000 msgr2=0x7ff194073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.698+0000 7ff19b200640 1 --2- 172.21.15.94:0/3028084183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff19406d000 0x7ff194073fd0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7ff1940caf40 tx=0x7ff18401c890 comp rx=0 tx=0).stop 2024-09-10T23:41:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.698+0000 7ff19b200640 1 -- 172.21.15.94:0/3028084183 shutdown_connections 2024-09-10T23:41:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.698+0000 7ff19b200640 1 --2- 172.21.15.94:0/3028084183 >> v2:172.21.15.181:3300/0 conn(0x7ff19406fb10 0x7ff19406ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.698+0000 7ff19b200640 1 --2- 172.21.15.94:0/3028084183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff19406d000 0x7ff194073fd0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.698+0000 7ff19b200640 1 -- 172.21.15.94:0/3028084183 >> 172.21.15.94:0/3028084183 conn(0x7ff19406e610 msgr2=0x7ff1940b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.698+0000 7ff19b200640 1 -- 172.21.15.94:0/3028084183 shutdown_connections 2024-09-10T23:41:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.698+0000 7ff19b200640 1 -- 172.21.15.94:0/3028084183 wait complete. 2024-09-10T23:41:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff19b200640 1 Processor -- start 2024-09-10T23:41:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff19b200640 1 -- start start 2024-09-10T23:41:08.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff19b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff19406cc20 0x7ff1940cef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:08.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff19b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff19406d000 0x7ff19414f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:08.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff19b200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff19406fb10 0x7ff1940c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:08.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff19b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff194169230 con 0x7ff19406d000 2024-09-10T23:41:08.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff19b200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff1941690b0 con 0x7ff19406cc20 2024-09-10T23:41:08.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff19b200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff1941693b0 con 0x7ff19406fb10 2024-09-10T23:41:08.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff199400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff19406d000 0x7ff19414f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:08.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff199400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff19406d000 0x7ff19414f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43312/0 (socket says 172.21.15.94:43312) 2024-09-10T23:41:08.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff199400640 1 -- 172.21.15.94:0/2471591437 learned_addr learned my addr 172.21.15.94:0/2471591437 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:08.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff199e00640 1 --2- 172.21.15.94:0/2471591437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff19406cc20 0x7ff1940cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:08.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff19a800640 1 --2- 172.21.15.94:0/2471591437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff19406fb10 0x7ff1940c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff199400640 1 -- 172.21.15.94:0/2471591437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff19406fb10 msgr2=0x7ff1940c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff199400640 1 --2- 172.21.15.94:0/2471591437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff19406fb10 0x7ff1940c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff199400640 1 -- 172.21.15.94:0/2471591437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff19406cc20 msgr2=0x7ff1940cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff199400640 1 --2- 172.21.15.94:0/2471591437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff19406cc20 0x7ff1940cef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff199400640 1 -- 172.21.15.94:0/2471591437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1940adb00 con 0x7ff19406d000 2024-09-10T23:41:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff199400640 1 --2- 172.21.15.94:0/2471591437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff19406d000 0x7ff19414f900 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7ff1840b0040 tx=0x7ff1840a5890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff18e000640 1 -- 172.21.15.94:0/2471591437 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1840b2040 con 0x7ff19406d000 2024-09-10T23:41:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff18e000640 1 -- 172.21.15.94:0/2471591437 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff184005030 con 0x7ff19406d000 2024-09-10T23:41:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff18e000640 1 -- 172.21.15.94:0/2471591437 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184005480 con 0x7ff19406d000 2024-09-10T23:41:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff19b200640 1 -- 172.21.15.94:0/2471591437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1940adde0 con 0x7ff19406d000 2024-09-10T23:41:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff19b200640 1 -- 172.21.15.94:0/2471591437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1940ae3a0 con 0x7ff19406d000 2024-09-10T23:41:08.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.702+0000 7ff19b200640 1 -- 172.21.15.94:0/2471591437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff194073fd0 con 0x7ff19406d000 2024-09-10T23:41:08.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.706+0000 7ff18e000640 1 -- 172.21.15.94:0/2471591437 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7ff1840046d0 con 0x7ff19406d000 2024-09-10T23:41:08.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.706+0000 7ff18e000640 1 --2- 172.21.15.94:0/2471591437 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff1780731b0 0x7ff178075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:08.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.706+0000 7ff18e000640 1 -- 172.21.15.94:0/2471591437 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7ff18412b6a0 con 0x7ff19406d000 2024-09-10T23:41:08.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.706+0000 7ff199e00640 1 --2- 172.21.15.94:0/2471591437 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff1780731b0 0x7ff178075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:08.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.706+0000 7ff199e00640 1 --2- 172.21.15.94:0/2471591437 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff1780731b0 0x7ff178075670 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff1940d1460 tx=0x7ff188009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:08.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.710+0000 7ff18e000640 1 -- 172.21.15.94:0/2471591437 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff1840bc060 con 0x7ff19406d000 2024-09-10T23:41:08.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.874+0000 7ff19b200640 1 -- 172.21.15.94:0/2471591437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7ff19414fe40 con 0x7ff19406d000 2024-09-10T23:41:08.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.878+0000 7ff18e000640 1 -- 172.21.15.94:0/2471591437 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v49) ==== 71+0+0 (secure 0 0 0) 0x7ff1840164b0 con 0x7ff19406d000 2024-09-10T23:41:08.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.882+0000 7ff19b200640 1 -- 172.21.15.94:0/2471591437 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff1780731b0 msgr2=0x7ff178075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:08.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.882+0000 7ff19b200640 1 --2- 172.21.15.94:0/2471591437 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff1780731b0 0x7ff178075670 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff1940d1460 tx=0x7ff188009040 comp rx=0 tx=0).stop 2024-09-10T23:41:08.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.882+0000 7ff19b200640 1 -- 172.21.15.94:0/2471591437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff19406d000 msgr2=0x7ff19414f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:08.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.882+0000 7ff19b200640 1 --2- 172.21.15.94:0/2471591437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff19406d000 0x7ff19414f900 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7ff1840b0040 tx=0x7ff1840a5890 comp rx=0 tx=0).stop 2024-09-10T23:41:08.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.882+0000 7ff19b200640 1 -- 172.21.15.94:0/2471591437 shutdown_connections 2024-09-10T23:41:08.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.882+0000 7ff19b200640 1 --2- 172.21.15.94:0/2471591437 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff1780731b0 0x7ff178075670 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:08.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.882+0000 7ff19b200640 1 --2- 172.21.15.94:0/2471591437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff19406fb10 0x7ff1940c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:08.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.882+0000 7ff19b200640 1 --2- 172.21.15.94:0/2471591437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff19406d000 0x7ff19414f900 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:08.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.882+0000 7ff19b200640 1 --2- 172.21.15.94:0/2471591437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff19406cc20 0x7ff1940cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:08.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.882+0000 7ff19b200640 1 -- 172.21.15.94:0/2471591437 >> 172.21.15.94:0/2471591437 conn(0x7ff19406e610 msgr2=0x7ff19415e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:08.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.882+0000 7ff19b200640 1 -- 172.21.15.94:0/2471591437 shutdown_connections 2024-09-10T23:41:08.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:08.882+0000 7ff19b200640 1 -- 172.21.15.94:0/2471591437 wait complete. 2024-09-10T23:41:08.902 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-10T23:41:09.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f4000640 1 -- 172.21.15.94:0/264221701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1d40053d0 msgr2=0x7fb1d400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:09.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f4000640 1 --2- 172.21.15.94:0/264221701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1d40053d0 0x7fb1d400b9d0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4001dc0 tx=0x7fb1e401cb30 comp rx=0 tx=0).stop 2024-09-10T23:41:09.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f4000640 1 -- 172.21.15.94:0/264221701 shutdown_connections 2024-09-10T23:41:09.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f4000640 1 --2- 172.21.15.94:0/264221701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1d40053d0 0x7fb1d400b9d0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:09.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f4000640 1 -- 172.21.15.94:0/264221701 >> 172.21.15.94:0/264221701 conn(0x7fb1ec0d1c60 msgr2=0x7fb1ec0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:09.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f4000640 1 -- 172.21.15.94:0/264221701 shutdown_connections 2024-09-10T23:41:09.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f4000640 1 -- 172.21.15.94:0/264221701 wait complete. 2024-09-10T23:41:09.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f4000640 1 Processor -- start 2024-09-10T23:41:09.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f4000640 1 -- start start 2024-09-10T23:41:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f4000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1d40053d0 0x7fb1ec0c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f4000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1ec159960 0x7fb1ec152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f4000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1ec159eb0 0x7fb1ec0c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f4000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec0cfce0 con 0x7fb1d40053d0 2024-09-10T23:41:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f4000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec0cfb60 con 0x7fb1ec159eb0 2024-09-10T23:41:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f4000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fb1ec0cfe60 con 0x7fb1ec159960 2024-09-10T23:41:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f1600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1d40053d0 0x7fb1ec0c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f1600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1d40053d0 0x7fb1ec0c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43330/0 (socket says 172.21.15.94:43330) 2024-09-10T23:41:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f1600640 1 -- 172.21.15.94:0/1582691428 learned_addr learned my addr 172.21.15.94:0/1582691428 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f0c00640 1 --2- 172.21.15.94:0/1582691428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1ec159960 0x7fb1ec152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f2000640 1 --2- 172.21.15.94:0/1582691428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1ec159eb0 0x7fb1ec0c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f1600640 1 -- 172.21.15.94:0/1582691428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1ec159960 msgr2=0x7fb1ec152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f1600640 1 --2- 172.21.15.94:0/1582691428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1ec159960 0x7fb1ec152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:09.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f1600640 1 -- 172.21.15.94:0/1582691428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1ec159eb0 msgr2=0x7fb1ec0c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:09.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f1600640 1 --2- 172.21.15.94:0/1582691428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1ec159eb0 0x7fb1ec0c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:09.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.046+0000 7fb1f1600640 1 -- 172.21.15.94:0/1582691428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ec0b73d0 con 0x7fb1d40053d0 2024-09-10T23:41:09.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.050+0000 7fb1f1600640 1 --2- 172.21.15.94:0/1582691428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1d40053d0 0x7fb1ec0c8140 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0011ae0 tx=0x7fb1e0009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:09.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.050+0000 7fb1e9600640 1 -- 172.21.15.94:0/1582691428 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e001e070 con 0x7fb1d40053d0 2024-09-10T23:41:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.050+0000 7fb1e9600640 1 -- 172.21.15.94:0/1582691428 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1e0004460 con 0x7fb1d40053d0 2024-09-10T23:41:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.050+0000 7fb1e9600640 1 -- 172.21.15.94:0/1582691428 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e000fc40 con 0x7fb1d40053d0 2024-09-10T23:41:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.050+0000 7fb1f4000640 1 -- 172.21.15.94:0/1582691428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ec0b76b0 con 0x7fb1d40053d0 2024-09-10T23:41:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.050+0000 7fb1f4000640 1 -- 172.21.15.94:0/1582691428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ec0c78d0 con 0x7fb1d40053d0 2024-09-10T23:41:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.050+0000 7fb1f4000640 1 -- 172.21.15.94:0/1582691428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1ec0d03c0 con 0x7fb1d40053d0 2024-09-10T23:41:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.050+0000 7fb1e9600640 1 -- 172.21.15.94:0/1582691428 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7fb1e0003600 con 0x7fb1d40053d0 2024-09-10T23:41:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.050+0000 7fb1e9600640 1 --2- 172.21.15.94:0/1582691428 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb1cc0731e0 0x7fb1cc0756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.050+0000 7fb1e9600640 1 -- 172.21.15.94:0/1582691428 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7fb1e0092b80 con 0x7fb1d40053d0 2024-09-10T23:41:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.050+0000 7fb1f0c00640 1 --2- 172.21.15.94:0/1582691428 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb1cc0731e0 0x7fb1cc0756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.054+0000 7fb1f0c00640 1 --2- 172.21.15.94:0/1582691428 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb1cc0731e0 0x7fb1cc0756a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0c87b0 tx=0x7fb1d80067b0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:09.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.054+0000 7fb1e9600640 1 -- 172.21.15.94:0/1582691428 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb1e005f400 con 0x7fb1d40053d0 2024-09-10T23:41:09.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.282+0000 7fb1f4000640 1 -- 172.21.15.94:0/1582691428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fb1ec070c00 con 0x7fb1d40053d0 2024-09-10T23:41:09.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.534+0000 7fb1e9600640 1 -- 172.21.15.94:0/1582691428 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v470) ==== 163+0+0 (secure 0 0 0) 0x7fb1e0063e30 con 0x7fb1d40053d0 2024-09-10T23:41:09.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.538+0000 7fb1f4000640 1 -- 172.21.15.94:0/1582691428 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb1cc0731e0 msgr2=0x7fb1cc0756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:09.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.538+0000 7fb1f4000640 1 --2- 172.21.15.94:0/1582691428 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb1cc0731e0 0x7fb1cc0756a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0c87b0 tx=0x7fb1d80067b0 comp rx=0 tx=0).stop 2024-09-10T23:41:09.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.538+0000 7fb1f4000640 1 -- 172.21.15.94:0/1582691428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1d40053d0 msgr2=0x7fb1ec0c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:09.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.538+0000 7fb1f4000640 1 --2- 172.21.15.94:0/1582691428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1d40053d0 0x7fb1ec0c8140 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0011ae0 tx=0x7fb1e0009e50 comp rx=0 tx=0).stop 2024-09-10T23:41:09.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.538+0000 7fb1f4000640 1 -- 172.21.15.94:0/1582691428 shutdown_connections 2024-09-10T23:41:09.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.538+0000 7fb1f4000640 1 --2- 172.21.15.94:0/1582691428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1ec159eb0 0x7fb1ec0c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:09.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.538+0000 7fb1f4000640 1 --2- 172.21.15.94:0/1582691428 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fb1cc0731e0 0x7fb1cc0756a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:09.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.538+0000 7fb1f4000640 1 --2- 172.21.15.94:0/1582691428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1ec159960 0x7fb1ec152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:09.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.538+0000 7fb1f4000640 1 --2- 172.21.15.94:0/1582691428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1d40053d0 0x7fb1ec0c8140 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:09.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.538+0000 7fb1f4000640 1 -- 172.21.15.94:0/1582691428 >> 172.21.15.94:0/1582691428 conn(0x7fb1ec0d1c60 msgr2=0x7fb1ec15c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:09.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.538+0000 7fb1f4000640 1 -- 172.21.15.94:0/1582691428 shutdown_connections 2024-09-10T23:41:09.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:09.538+0000 7fb1f4000640 1 -- 172.21.15.94:0/1582691428 wait complete. 2024-09-10T23:41:09.559 INFO:tasks.cephfs_test_runner:Validate unparseable text .meta file on legacy subvol root doesn't break the system ... ok 2024-09-10T23:41:09.560 INFO:tasks.cephfs_test_runner: 2024-09-10T23:41:09.560 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-10T23:41:09.560 INFO:tasks.cephfs_test_runner:Ran 11 tests in 479.487s 2024-09-10T23:41:09.560 INFO:tasks.cephfs_test_runner: 2024-09-10T23:41:09.560 INFO:tasks.cephfs_test_runner:OK 2024-09-10T23:41:09.560 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-10T23:41:09.568 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-10T23:41:09.584 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-10T23:41:09.584 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:41:09.584 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:41:09.593 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:41:09.593 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T23:41:09.596 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:41:09.644 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350200k,nr_inodes=4087550,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4183 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274972k,nr_inodes=818743,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274976k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T23:41:09.644 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:41:09.645 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:41:09.654 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:41:09.654 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-10T23:41:09.654 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T23:41:09.704 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350152k,nr_inodes=4087538,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4648 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274964k,nr_inodes=818741,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274968k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T23:41:09.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:41:09.705 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T23:41:09.714 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T23:41:09.714 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T23:41:09.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:41:09.715 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T23:41:09.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:41:09.782 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T23:41:10.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:41:10.234 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-09-10T23:41:10.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:41:10.351 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2024-09-10T23:41:10.361 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T23:41:10.361 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T23:41:10.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:41:10.362 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T23:41:10.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:41:10.427 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T23:41:10.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T23:41:10.883 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2024-09-10T23:41:11.012 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-10T23:41:11.024 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T23:41:11.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.166+0000 7f2ae4a00640 1 -- 172.21.15.94:0/2794007856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae015bcb0 msgr2=0x7f2ae00d2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.166+0000 7f2ae4a00640 1 --2- 172.21.15.94:0/2794007856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae015bcb0 0x7f2ae00d2560 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0069890 tx=0x7f2ad001f010 comp rx=0 tx=0).stop 2024-09-10T23:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.166+0000 7f2ae4a00640 1 -- 172.21.15.94:0/2794007856 shutdown_connections 2024-09-10T23:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.166+0000 7f2ae4a00640 1 --2- 172.21.15.94:0/2794007856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae015bcb0 0x7f2ae00d2560 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.166+0000 7f2ae4a00640 1 -- 172.21.15.94:0/2794007856 >> 172.21.15.94:0/2794007856 conn(0x7f2ae00d1a40 msgr2=0x7f2ae00d1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.166+0000 7f2ae4a00640 1 -- 172.21.15.94:0/2794007856 shutdown_connections 2024-09-10T23:41:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.166+0000 7f2ae4a00640 1 -- 172.21.15.94:0/2794007856 wait complete. 2024-09-10T23:41:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.166+0000 7f2ae4a00640 1 Processor -- start 2024-09-10T23:41:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.166+0000 7f2ae4a00640 1 -- start start 2024-09-10T23:41:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2ae4a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ae00b0b10 0x7f2ae01f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2ae4a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ae00d0690 0x7f2ae01f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2ae4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae00c44e0 0x7f2ae00b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2ae4a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae00710b0 con 0x7f2ae00c44e0 2024-09-10T23:41:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2ae4a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae0070f30 con 0x7f2ae00b0b10 2024-09-10T23:41:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2ae4a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae0071230 con 0x7f2ae00d0690 2024-09-10T23:41:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2adfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae00c44e0 0x7f2ae00b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2adfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae00c44e0 0x7f2ae00b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43350/0 (socket says 172.21.15.94:43350) 2024-09-10T23:41:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2adfe00640 1 -- 172.21.15.94:0/1534811014 learned_addr learned my addr 172.21.15.94:0/1534811014 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2adf400640 1 --2- 172.21.15.94:0/1534811014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ae00b0b10 0x7f2ae01f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2ad6a00640 1 --2- 172.21.15.94:0/1534811014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ae00d0690 0x7f2ae01f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2adfe00640 1 -- 172.21.15.94:0/1534811014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ae00d0690 msgr2=0x7f2ae01f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2adfe00640 1 --2- 172.21.15.94:0/1534811014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ae00d0690 0x7f2ae01f4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2adfe00640 1 -- 172.21.15.94:0/1534811014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ae00b0b10 msgr2=0x7f2ae01f8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2adfe00640 1 --2- 172.21.15.94:0/1534811014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ae00b0b10 0x7f2ae01f8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2adfe00640 1 -- 172.21.15.94:0/1534811014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae01f4bb0 con 0x7f2ae00c44e0 2024-09-10T23:41:11.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2adfe00640 1 --2- 172.21.15.94:0/1534811014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae00c44e0 0x7f2ae00b27c0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800c780 tx=0x7f2ad800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:11.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2ad7e00640 1 -- 172.21.15.94:0/1534811014 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8016020 con 0x7f2ae00c44e0 2024-09-10T23:41:11.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2ad7e00640 1 -- 172.21.15.94:0/1534811014 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ad800aae0 con 0x7f2ae00c44e0 2024-09-10T23:41:11.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2ad7e00640 1 -- 172.21.15.94:0/1534811014 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8012420 con 0x7f2ae00c44e0 2024-09-10T23:41:11.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2ae4a00640 1 -- 172.21.15.94:0/1534811014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae01f4e50 con 0x7f2ae00c44e0 2024-09-10T23:41:11.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2ae4a00640 1 -- 172.21.15.94:0/1534811014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae01f5410 con 0x7f2ae00c44e0 2024-09-10T23:41:11.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2ae4a00640 1 -- 172.21.15.94:0/1534811014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae00d2560 con 0x7f2ae00c44e0 2024-09-10T23:41:11.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.170+0000 7f2ad7e00640 1 -- 172.21.15.94:0/1534811014 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f2ad801a3b0 con 0x7f2ae00c44e0 2024-09-10T23:41:11.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.174+0000 7f2ad7e00640 1 --2- 172.21.15.94:0/1534811014 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2abc0731b0 0x7f2abc075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:11.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.174+0000 7f2ad7e00640 1 -- 172.21.15.94:0/1534811014 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7f2ad8092010 con 0x7f2ae00c44e0 2024-09-10T23:41:11.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.174+0000 7f2adf400640 1 --2- 172.21.15.94:0/1534811014 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2abc0731b0 0x7f2abc075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:11.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.174+0000 7f2adf400640 1 --2- 172.21.15.94:0/1534811014 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2abc0731b0 0x7f2abc075670 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00b2000 tx=0x7f2ad001d850 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.178+0000 7f2ad7e00640 1 -- 172.21.15.94:0/1534811014 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2ad8020030 con 0x7f2ae00c44e0 2024-09-10T23:41:11.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.342+0000 7f2ae4a00640 1 -- 172.21.15.94:0/1534811014 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2ae00d02e0 con 0x7f2abc0731b0 2024-09-10T23:41:11.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.342+0000 7f2ad7e00640 1 -- 172.21.15.94:0/1534811014 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13503 (secure 0 0 0) 0x7f2ae00d02e0 con 0x7f2abc0731b0 2024-09-10T23:41:11.343 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:41:11.343 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:41:11.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.346+0000 7f2ae4a00640 1 -- 172.21.15.94:0/1534811014 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2abc0731b0 msgr2=0x7f2abc075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:11.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.346+0000 7f2ae4a00640 1 --2- 172.21.15.94:0/1534811014 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2abc0731b0 0x7f2abc075670 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00b2000 tx=0x7f2ad001d850 comp rx=0 tx=0).stop 2024-09-10T23:41:11.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.346+0000 7f2ae4a00640 1 -- 172.21.15.94:0/1534811014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae00c44e0 msgr2=0x7f2ae00b27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:11.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.346+0000 7f2ae4a00640 1 --2- 172.21.15.94:0/1534811014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae00c44e0 0x7f2ae00b27c0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800c780 tx=0x7f2ad800cc40 comp rx=0 tx=0).stop 2024-09-10T23:41:11.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.346+0000 7f2ae4a00640 1 -- 172.21.15.94:0/1534811014 shutdown_connections 2024-09-10T23:41:11.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.346+0000 7f2ae4a00640 1 --2- 172.21.15.94:0/1534811014 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2abc0731b0 0x7f2abc075670 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.346+0000 7f2ae4a00640 1 --2- 172.21.15.94:0/1534811014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ae00c44e0 0x7f2ae00b27c0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.346+0000 7f2ae4a00640 1 --2- 172.21.15.94:0/1534811014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ae00d0690 0x7f2ae01f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.346+0000 7f2ae4a00640 1 --2- 172.21.15.94:0/1534811014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ae00b0b10 0x7f2ae01f8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.346+0000 7f2ae4a00640 1 -- 172.21.15.94:0/1534811014 >> 172.21.15.94:0/1534811014 conn(0x7f2ae00d1a40 msgr2=0x7f2ae015ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:11.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.346+0000 7f2ae4a00640 1 -- 172.21.15.94:0/1534811014 shutdown_connections 2024-09-10T23:41:11.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.346+0000 7f2ae4a00640 1 -- 172.21.15.94:0/1534811014 wait complete. 2024-09-10T23:41:11.363 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-10T23:41:10.792428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283532,"kb_used_data":17388,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754691188,"statfs":{"total":773094113280,"available":772803776512,"internally_reserved":0,"allocated":17805312,"data_stored":8556819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-60683,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-78,"num_write_kb":-88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001108"},"pg_stats":[{"pgid":"1.0","version":"70'79","reported_seq":400,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-10T23:41:07.533067+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:41:07.533067+0000","last_peered":"2024-09-10T23:41:07.533067+0000","last_clean":"2024-09-10T23:41:07.533067+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:41:07.533067+0000","last_undegraded":"2024-09-10T23:41:07.533067+0000","last_fullsized":"2024-09-10T23:41:07.533067+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509549,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33196,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338644,"statfs":{"total":96636764160,"available":96602771456,"internally_reserved":0,"allocated":2056192,"data_stored":906047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542252,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33196,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338644,"statfs":{"total":96636764160,"available":96602771456,"internally_reserved":0,"allocated":2056192,"data_stored":906095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509548,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33788,"kb_used_data":2600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338052,"statfs":{"total":96636764160,"available":96602165248,"internally_reserved":0,"allocated":2662400,"data_stored":1497121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509546,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33208,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338632,"statfs":{"total":96636764160,"available":96602759168,"internally_reserved":0,"allocated":2068480,"data_stored":912056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509547,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37420,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334420,"statfs":{"total":96636764160,"available":96598446080,"internally_reserved":0,"allocated":2121728,"data_stored":965895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509548,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37428,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2129920,"data_stored":965999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509547,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37936,"kb_used_data":2588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333904,"statfs":{"total":96636764160,"available":96597917696,"internally_reserved":0,"allocated":2650112,"data_stored":1496696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509547,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37360,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334480,"statfs":{"total":96636764160,"available":96598507520,"internally_reserved":0,"allocated":2060288,"data_stored":906910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:41:11.364 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:41:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.506+0000 7ff46bc00640 1 -- 172.21.15.94:0/2805451609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4640d1690 msgr2=0x7ff46415fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.506+0000 7ff46bc00640 1 --2- 172.21.15.94:0/2805451609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4640d1690 0x7ff46415fb50 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7ff46406ef50 tx=0x7ff45c01ccd0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.506+0000 7ff46bc00640 1 -- 172.21.15.94:0/2805451609 shutdown_connections 2024-09-10T23:41:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.506+0000 7ff46bc00640 1 --2- 172.21.15.94:0/2805451609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4640d1690 0x7ff46415fb50 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.506+0000 7ff46bc00640 1 --2- 172.21.15.94:0/2805451609 >> v2:172.21.15.181:3300/0 conn(0x7ff464071a20 0x7ff46415f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.506+0000 7ff46bc00640 1 --2- 172.21.15.94:0/2805451609 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff46406cc20 0x7ff46406d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.506+0000 7ff46bc00640 1 -- 172.21.15.94:0/2805451609 >> 172.21.15.94:0/2805451609 conn(0x7ff4640d0b70 msgr2=0x7ff4640d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.506+0000 7ff46bc00640 1 -- 172.21.15.94:0/2805451609 shutdown_connections 2024-09-10T23:41:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.506+0000 7ff46bc00640 1 -- 172.21.15.94:0/2805451609 wait complete. 2024-09-10T23:41:11.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.506+0000 7ff46bc00640 1 Processor -- start 2024-09-10T23:41:11.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.506+0000 7ff46bc00640 1 -- start start 2024-09-10T23:41:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff46bc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff46406cc20 0x7ff46414b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff46bc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff464071a20 0x7ff46414ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff46bc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4640d1690 0x7ff464158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff46bc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4640cffe0 con 0x7ff4640d1690 2024-09-10T23:41:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff46bc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff4640cfe60 con 0x7ff464071a20 2024-09-10T23:41:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff46bc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7ff4640d0160 con 0x7ff46406cc20 2024-09-10T23:41:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff469c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4640d1690 0x7ff464158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff469c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4640d1690 0x7ff464158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43368/0 (socket says 172.21.15.94:43368) 2024-09-10T23:41:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff469c00640 1 -- 172.21.15.94:0/1404891140 learned_addr learned my addr 172.21.15.94:0/1404891140 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff469200640 1 --2- 172.21.15.94:0/1404891140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff46406cc20 0x7ff46414b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff45be00640 1 --2- 172.21.15.94:0/1404891140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff464071a20 0x7ff46414ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff469c00640 1 -- 172.21.15.94:0/1404891140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff46406cc20 msgr2=0x7ff46414b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff469c00640 1 --2- 172.21.15.94:0/1404891140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff46406cc20 0x7ff46414b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff469c00640 1 -- 172.21.15.94:0/1404891140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff464071a20 msgr2=0x7ff46414ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff469c00640 1 --2- 172.21.15.94:0/1404891140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff464071a20 0x7ff46414ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff469c00640 1 -- 172.21.15.94:0/1404891140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4641fae60 con 0x7ff4640d1690 2024-09-10T23:41:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff45be00640 1 --2- 172.21.15.94:0/1404891140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff464071a20 0x7ff46414ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:41:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff469c00640 1 --2- 172.21.15.94:0/1404891140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4640d1690 0x7ff464158d40 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7ff45c01b380 tx=0x7ff45c0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff461600640 1 -- 172.21.15.94:0/1404891140 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c0bb070 con 0x7ff4640d1690 2024-09-10T23:41:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff461600640 1 -- 172.21.15.94:0/1404891140 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff45c004a50 con 0x7ff4640d1690 2024-09-10T23:41:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff461600640 1 -- 172.21.15.94:0/1404891140 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c0b2040 con 0x7ff4640d1690 2024-09-10T23:41:11.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff46bc00640 1 -- 172.21.15.94:0/1404891140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4641fb140 con 0x7ff4640d1690 2024-09-10T23:41:11.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff46bc00640 1 -- 172.21.15.94:0/1404891140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4641fb700 con 0x7ff4640d1690 2024-09-10T23:41:11.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff46bc00640 1 -- 172.21.15.94:0/1404891140 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff46406d000 con 0x7ff4640d1690 2024-09-10T23:41:11.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.510+0000 7ff461600640 1 -- 172.21.15.94:0/1404891140 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7ff45c0b5070 con 0x7ff4640d1690 2024-09-10T23:41:11.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.514+0000 7ff461600640 1 --2- 172.21.15.94:0/1404891140 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff4400731b0 0x7ff440075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:11.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.514+0000 7ff461600640 1 -- 172.21.15.94:0/1404891140 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7ff45c12b7b0 con 0x7ff4640d1690 2024-09-10T23:41:11.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.514+0000 7ff469200640 1 --2- 172.21.15.94:0/1404891140 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff4400731b0 0x7ff440075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:11.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.514+0000 7ff469200640 1 --2- 172.21.15.94:0/1404891140 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff4400731b0 0x7ff440075670 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff4540058d0 tx=0x7ff454005860 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:11.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.518+0000 7ff461600640 1 -- 172.21.15.94:0/1404891140 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff45c016950 con 0x7ff4640d1690 2024-09-10T23:41:11.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.686+0000 7ff46bc00640 1 -- 172.21.15.94:0/1404891140 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff46414dd70 con 0x7ff4400731b0 2024-09-10T23:41:11.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.686+0000 7ff461600640 1 -- 172.21.15.94:0/1404891140 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13503 (secure 0 0 0) 0x7ff46414dd70 con 0x7ff4400731b0 2024-09-10T23:41:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:41:11.687 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:41:11.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.686+0000 7ff46bc00640 1 -- 172.21.15.94:0/1404891140 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff4400731b0 msgr2=0x7ff440075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:11.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.686+0000 7ff46bc00640 1 --2- 172.21.15.94:0/1404891140 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff4400731b0 0x7ff440075670 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff4540058d0 tx=0x7ff454005860 comp rx=0 tx=0).stop 2024-09-10T23:41:11.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.686+0000 7ff46bc00640 1 -- 172.21.15.94:0/1404891140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4640d1690 msgr2=0x7ff464158d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:11.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.686+0000 7ff46bc00640 1 --2- 172.21.15.94:0/1404891140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4640d1690 0x7ff464158d40 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7ff45c01b380 tx=0x7ff45c0092c0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.686+0000 7ff46bc00640 1 -- 172.21.15.94:0/1404891140 shutdown_connections 2024-09-10T23:41:11.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.686+0000 7ff46bc00640 1 --2- 172.21.15.94:0/1404891140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff464071a20 0x7ff46414ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.686+0000 7ff46bc00640 1 --2- 172.21.15.94:0/1404891140 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7ff4400731b0 0x7ff440075670 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.686+0000 7ff46bc00640 1 --2- 172.21.15.94:0/1404891140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4640d1690 0x7ff464158d40 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.686+0000 7ff46bc00640 1 --2- 172.21.15.94:0/1404891140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff46406cc20 0x7ff46414b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.686+0000 7ff46bc00640 1 -- 172.21.15.94:0/1404891140 >> 172.21.15.94:0/1404891140 conn(0x7ff4640d0b70 msgr2=0x7ff46414e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:11.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.686+0000 7ff46bc00640 1 -- 172.21.15.94:0/1404891140 shutdown_connections 2024-09-10T23:41:11.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.686+0000 7ff46bc00640 1 -- 172.21.15.94:0/1404891140 wait complete. 2024-09-10T23:41:11.708 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-10T23:41:10.792428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283532,"kb_used_data":17388,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754691188,"statfs":{"total":773094113280,"available":772803776512,"internally_reserved":0,"allocated":17805312,"data_stored":8556819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-60683,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-78,"num_write_kb":-88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001108"},"pg_stats":[{"pgid":"1.0","version":"70'79","reported_seq":400,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-10T23:41:07.533067+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:41:07.533067+0000","last_peered":"2024-09-10T23:41:07.533067+0000","last_clean":"2024-09-10T23:41:07.533067+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:41:07.533067+0000","last_undegraded":"2024-09-10T23:41:07.533067+0000","last_fullsized":"2024-09-10T23:41:07.533067+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509549,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33196,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338644,"statfs":{"total":96636764160,"available":96602771456,"internally_reserved":0,"allocated":2056192,"data_stored":906047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542252,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33196,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338644,"statfs":{"total":96636764160,"available":96602771456,"internally_reserved":0,"allocated":2056192,"data_stored":906095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509548,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33788,"kb_used_data":2600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338052,"statfs":{"total":96636764160,"available":96602165248,"internally_reserved":0,"allocated":2662400,"data_stored":1497121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509546,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33208,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338632,"statfs":{"total":96636764160,"available":96602759168,"internally_reserved":0,"allocated":2068480,"data_stored":912056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509547,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37420,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334420,"statfs":{"total":96636764160,"available":96598446080,"internally_reserved":0,"allocated":2121728,"data_stored":965895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509548,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37428,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2129920,"data_stored":965999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509547,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37936,"kb_used_data":2588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333904,"statfs":{"total":96636764160,"available":96597917696,"internally_reserved":0,"allocated":2650112,"data_stored":1496696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509547,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37360,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334480,"statfs":{"total":96636764160,"available":96598507520,"internally_reserved":0,"allocated":2060288,"data_stored":906910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:41:11.709 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T23:41:11.709 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:41:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.850+0000 7f1605800640 1 -- 172.21.15.94:0/2819411183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f160015ba50 msgr2=0x7f16000b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.850+0000 7f1605800640 1 --2- 172.21.15.94:0/2819411183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f160015ba50 0x7f16000b4700 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009bc0 tx=0x7f15f001cae0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.850+0000 7f1605800640 1 -- 172.21.15.94:0/2819411183 shutdown_connections 2024-09-10T23:41:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.850+0000 7f1605800640 1 --2- 172.21.15.94:0/2819411183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f160015ba50 0x7f16000b4700 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.850+0000 7f1605800640 1 -- 172.21.15.94:0/2819411183 >> 172.21.15.94:0/2819411183 conn(0x7f160006cdf0 msgr2=0x7f160006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.850+0000 7f1605800640 1 -- 172.21.15.94:0/2819411183 shutdown_connections 2024-09-10T23:41:11.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.850+0000 7f1605800640 1 -- 172.21.15.94:0/2819411183 wait complete. 2024-09-10T23:41:11.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.850+0000 7f1605800640 1 Processor -- start 2024-09-10T23:41:11.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.850+0000 7f1605800640 1 -- start start 2024-09-10T23:41:11.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f1605800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16000b4c40 0x7f16001fb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:11.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f1605800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16000d03c0 0x7f160015a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:11.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f1605800640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f160015ba50 0x7f16000b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f1605800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1600169bf0 con 0x7f16000d03c0 2024-09-10T23:41:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f1605800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1600169a70 con 0x7f16000b4c40 2024-09-10T23:41:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f1605800640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1600169d70 con 0x7f160015ba50 2024-09-10T23:41:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f15f6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16000d03c0 0x7f160015a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f15f6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16000d03c0 0x7f160015a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43392/0 (socket says 172.21.15.94:43392) 2024-09-10T23:41:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f15f6000640 1 -- 172.21.15.94:0/1931194384 learned_addr learned my addr 172.21.15.94:0/1931194384 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f15fea00640 1 --2- 172.21.15.94:0/1931194384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16000b4c40 0x7f16001fb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f15ff400640 1 --2- 172.21.15.94:0/1931194384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f160015ba50 0x7f16000b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f15f6000640 1 -- 172.21.15.94:0/1931194384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f160015ba50 msgr2=0x7f16000b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f15f6000640 1 --2- 172.21.15.94:0/1931194384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f160015ba50 0x7f16000b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f15f6000640 1 -- 172.21.15.94:0/1931194384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16000b4c40 msgr2=0x7f16001fb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:11.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f15f6000640 1 --2- 172.21.15.94:0/1931194384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16000b4c40 0x7f16001fb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:11.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f15f6000640 1 -- 172.21.15.94:0/1931194384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1600150410 con 0x7f16000d03c0 2024-09-10T23:41:11.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f15fea00640 1 --2- 172.21.15.94:0/1931194384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16000b4c40 0x7f16001fb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:41:11.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f15f6000640 1 --2- 172.21.15.94:0/1931194384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16000d03c0 0x7f160015a2b0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f15e800c430 tx=0x7f15e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:11.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f15f7400640 1 -- 172.21.15.94:0/1931194384 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8011790 con 0x7f16000d03c0 2024-09-10T23:41:11.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f15f7400640 1 -- 172.21.15.94:0/1931194384 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15e8017070 con 0x7f16000d03c0 2024-09-10T23:41:11.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f1605800640 1 -- 172.21.15.94:0/1931194384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16001506f0 con 0x7f16000d03c0 2024-09-10T23:41:11.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f15f7400640 1 -- 172.21.15.94:0/1931194384 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8011ce0 con 0x7f16000d03c0 2024-09-10T23:41:11.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f1605800640 1 -- 172.21.15.94:0/1931194384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16000b0d70 con 0x7f16000d03c0 2024-09-10T23:41:11.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f1605800640 1 -- 172.21.15.94:0/1931194384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16000b4700 con 0x7f16000d03c0 2024-09-10T23:41:11.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.854+0000 7f15f7400640 1 -- 172.21.15.94:0/1931194384 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f15e801c460 con 0x7f16000d03c0 2024-09-10T23:41:11.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.858+0000 7f15f7400640 1 --2- 172.21.15.94:0/1931194384 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f15dc0731b0 0x7f15dc075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:11.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.858+0000 7f15f7400640 1 -- 172.21.15.94:0/1931194384 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7f15e8091cb0 con 0x7f16000d03c0 2024-09-10T23:41:11.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.858+0000 7f15fea00640 1 --2- 172.21.15.94:0/1931194384 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f15dc0731b0 0x7f15dc075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:11.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.858+0000 7f15fea00640 1 --2- 172.21.15.94:0/1931194384 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f15dc0731b0 0x7f15dc075670 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f15f00af000 tx=0x7f15f001b320 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:11.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:11.862+0000 7f15f7400640 1 -- 172.21.15.94:0/1931194384 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f15e805eb80 con 0x7f16000d03c0 2024-09-10T23:41:12.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.022+0000 7f1605800640 1 -- 172.21.15.94:0/1931194384 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1600150e50 con 0x7f15dc0731b0 2024-09-10T23:41:12.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.026+0000 7f15f7400640 1 -- 172.21.15.94:0/1931194384 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13503 (secure 0 0 0) 0x7f1600150e50 con 0x7f15dc0731b0 2024-09-10T23:41:12.026 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:41:12.026 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:41:12.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.026+0000 7f1605800640 1 -- 172.21.15.94:0/1931194384 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f15dc0731b0 msgr2=0x7f15dc075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:12.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.026+0000 7f1605800640 1 --2- 172.21.15.94:0/1931194384 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f15dc0731b0 0x7f15dc075670 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f15f00af000 tx=0x7f15f001b320 comp rx=0 tx=0).stop 2024-09-10T23:41:12.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.026+0000 7f1605800640 1 -- 172.21.15.94:0/1931194384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16000d03c0 msgr2=0x7f160015a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:12.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.026+0000 7f1605800640 1 --2- 172.21.15.94:0/1931194384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16000d03c0 0x7f160015a2b0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f15e800c430 tx=0x7f15e800c8f0 comp rx=0 tx=0).stop 2024-09-10T23:41:12.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.026+0000 7f1605800640 1 -- 172.21.15.94:0/1931194384 shutdown_connections 2024-09-10T23:41:12.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.026+0000 7f1605800640 1 --2- 172.21.15.94:0/1931194384 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f15dc0731b0 0x7f15dc075670 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:12.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.026+0000 7f1605800640 1 --2- 172.21.15.94:0/1931194384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f160015ba50 0x7f16000b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:12.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.026+0000 7f1605800640 1 --2- 172.21.15.94:0/1931194384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16000d03c0 0x7f160015a2b0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:12.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.026+0000 7f1605800640 1 --2- 172.21.15.94:0/1931194384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16000b4c40 0x7f16001fb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:12.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.026+0000 7f1605800640 1 -- 172.21.15.94:0/1931194384 >> 172.21.15.94:0/1931194384 conn(0x7f160006cdf0 msgr2=0x7f160006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:12.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.026+0000 7f1605800640 1 -- 172.21.15.94:0/1931194384 shutdown_connections 2024-09-10T23:41:12.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.026+0000 7f1605800640 1 -- 172.21.15.94:0/1931194384 wait complete. 2024-09-10T23:41:12.050 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-10T23:41:10.792428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283532,"kb_used_data":17388,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754691188,"statfs":{"total":773094113280,"available":772803776512,"internally_reserved":0,"allocated":17805312,"data_stored":8556819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-60683,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-78,"num_write_kb":-88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001108"},"pg_stats":[{"pgid":"1.0","version":"70'79","reported_seq":400,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-10T23:41:07.533067+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:41:07.533067+0000","last_peered":"2024-09-10T23:41:07.533067+0000","last_clean":"2024-09-10T23:41:07.533067+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:41:07.533067+0000","last_undegraded":"2024-09-10T23:41:07.533067+0000","last_fullsized":"2024-09-10T23:41:07.533067+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509549,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33196,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338644,"statfs":{"total":96636764160,"available":96602771456,"internally_reserved":0,"allocated":2056192,"data_stored":906047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542252,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33196,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338644,"statfs":{"total":96636764160,"available":96602771456,"internally_reserved":0,"allocated":2056192,"data_stored":906095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509548,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33788,"kb_used_data":2600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338052,"statfs":{"total":96636764160,"available":96602165248,"internally_reserved":0,"allocated":2662400,"data_stored":1497121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509546,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33208,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338632,"statfs":{"total":96636764160,"available":96602759168,"internally_reserved":0,"allocated":2068480,"data_stored":912056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509547,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37420,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334420,"statfs":{"total":96636764160,"available":96598446080,"internally_reserved":0,"allocated":2121728,"data_stored":965895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509548,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37428,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2129920,"data_stored":965999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509547,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37936,"kb_used_data":2588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333904,"statfs":{"total":96636764160,"available":96597917696,"internally_reserved":0,"allocated":2650112,"data_stored":1496696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509547,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37360,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334480,"statfs":{"total":96636764160,"available":96598507520,"internally_reserved":0,"allocated":2060288,"data_stored":906910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:41:12.050 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T23:41:12.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.190+0000 7f290ae00640 1 -- 172.21.15.94:0/623971968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f290415f5c0 msgr2=0x7f29040cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:12.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.190+0000 7f290ae00640 1 --2- 172.21.15.94:0/623971968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f290415f5c0 0x7f29040cfb60 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f290415f9a0 tx=0x7f28f401c6e0 comp rx=0 tx=0).stop 2024-09-10T23:41:12.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.190+0000 7f290ae00640 1 -- 172.21.15.94:0/623971968 shutdown_connections 2024-09-10T23:41:12.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.190+0000 7f290ae00640 1 --2- 172.21.15.94:0/623971968 >> v2:172.21.15.181:3300/0 conn(0x7f29040af310 0x7f29040af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:12.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.190+0000 7f290ae00640 1 --2- 172.21.15.94:0/623971968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f290415f5c0 0x7f29040cfb60 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:12.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.190+0000 7f290ae00640 1 -- 172.21.15.94:0/623971968 >> 172.21.15.94:0/623971968 conn(0x7f29040d1c60 msgr2=0x7f29040d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:12.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.190+0000 7f290ae00640 1 -- 172.21.15.94:0/623971968 shutdown_connections 2024-09-10T23:41:12.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.190+0000 7f290ae00640 1 -- 172.21.15.94:0/623971968 wait complete. 2024-09-10T23:41:12.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.190+0000 7f290ae00640 1 Processor -- start 2024-09-10T23:41:12.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.190+0000 7f290ae00640 1 -- start start 2024-09-10T23:41:12.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f290ae00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29040af310 0x7f29040ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:12.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f290ae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29040b2bb0 0x7f290414f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:12.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f290ae00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f290415f5c0 0x7f2904152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:12.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f290ae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2904169000 con 0x7f29040b2bb0 2024-09-10T23:41:12.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f290ae00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2904168e80 con 0x7f290415f5c0 2024-09-10T23:41:12.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f290ae00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2904169180 con 0x7f29040af310 2024-09-10T23:41:12.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f2903400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29040b2bb0 0x7f290414f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:12.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f2903400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29040b2bb0 0x7f290414f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43418/0 (socket says 172.21.15.94:43418) 2024-09-10T23:41:12.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f2903400640 1 -- 172.21.15.94:0/2358881454 learned_addr learned my addr 172.21.15.94:0/2358881454 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:12.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f2903e00640 1 --2- 172.21.15.94:0/2358881454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29040af310 0x7f29040ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:12.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f2908e00640 1 --2- 172.21.15.94:0/2358881454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f290415f5c0 0x7f2904152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:12.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f2903400640 1 -- 172.21.15.94:0/2358881454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29040af310 msgr2=0x7f29040ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:12.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f2903400640 1 --2- 172.21.15.94:0/2358881454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29040af310 0x7f29040ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:12.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f2903400640 1 -- 172.21.15.94:0/2358881454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f290415f5c0 msgr2=0x7f2904152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:12.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f2903400640 1 --2- 172.21.15.94:0/2358881454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f290415f5c0 0x7f2904152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:12.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f2903400640 1 -- 172.21.15.94:0/2358881454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29040cab70 con 0x7f29040b2bb0 2024-09-10T23:41:12.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f2903e00640 1 --2- 172.21.15.94:0/2358881454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29040af310 0x7f29040ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:41:12.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f2903400640 1 --2- 172.21.15.94:0/2358881454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29040b2bb0 0x7f290414f060 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f28f000c4c0 tx=0x7f28f000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:12.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f28fbe00640 1 -- 172.21.15.94:0/2358881454 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0016020 con 0x7f29040b2bb0 2024-09-10T23:41:12.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f28fbe00640 1 -- 172.21.15.94:0/2358881454 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28f0012940 con 0x7f29040b2bb0 2024-09-10T23:41:12.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f28fbe00640 1 -- 172.21.15.94:0/2358881454 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0012d90 con 0x7f29040b2bb0 2024-09-10T23:41:12.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f290ae00640 1 -- 172.21.15.94:0/2358881454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29040cae10 con 0x7f29040b2bb0 2024-09-10T23:41:12.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f290ae00640 1 -- 172.21.15.94:0/2358881454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f290414e860 con 0x7f29040b2bb0 2024-09-10T23:41:12.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f290ae00640 1 -- 172.21.15.94:0/2358881454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29040af7a0 con 0x7f29040b2bb0 2024-09-10T23:41:12.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.194+0000 7f28fbe00640 1 -- 172.21.15.94:0/2358881454 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f28f001b400 con 0x7f29040b2bb0 2024-09-10T23:41:12.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.198+0000 7f28fbe00640 1 --2- 172.21.15.94:0/2358881454 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f28e40731b0 0x7f28e4075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:12.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.198+0000 7f28fbe00640 1 -- 172.21.15.94:0/2358881454 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7f28f00921b0 con 0x7f29040b2bb0 2024-09-10T23:41:12.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.198+0000 7f2903e00640 1 --2- 172.21.15.94:0/2358881454 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f28e40731b0 0x7f28e4075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:12.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.198+0000 7f2903e00640 1 --2- 172.21.15.94:0/2358881454 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f28e40731b0 0x7f28e4075670 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f28f40af000 tx=0x7f28f401af20 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:12.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.202+0000 7f28fbe00640 1 -- 172.21.15.94:0/2358881454 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f28f0020020 con 0x7f29040b2bb0 2024-09-10T23:41:12.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.370+0000 7f290ae00640 1 -- 172.21.15.94:0/2358881454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29040749c0 con 0x7f29040b2bb0 2024-09-10T23:41:12.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.370+0000 7f28fbe00640 1 -- 172.21.15.94:0/2358881454 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) ==== 74+0+13074 (secure 0 0 0) 0x7f28f005f140 con 0x7f29040b2bb0 2024-09-10T23:41:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:41:12.370 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":165,"fsid":"1798cdb9-9937-4918-a569-86b41a92817e","created":"2024-09-10T23:32:16.095397+0000","modified":"2024-09-10T23:41:08.530350+0000","last_up_change":"2024-09-10T23:32:31.318135+0000","last_in_change":"2024-09-10T23:32:20.614769+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:32:31.372521+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5aecd0fb-3049-4af5-a6a9-1ba3dfe62cea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6801","nonce":576048946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6804","nonce":576048946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6812","nonce":576048946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":576048946},{"type":"v1","addr":"172.21.15.94:6808","nonce":576048946}]},"public_addr":"172.21.15.94:6801/576048946","cluster_addr":"172.21.15.94:6804/576048946","heartbeat_back_addr":"172.21.15.94:6812/576048946","heartbeat_front_addr":"172.21.15.94:6808/576048946","state":["exists","up"]},{"osd":1,"uuid":"870fb073-c560-4844-b8e7-594f8b27b3db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6803","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6805","nonce":3936794679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6807","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6809","nonce":3936794679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6815","nonce":3936794679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6811","nonce":3936794679},{"type":"v1","addr":"172.21.15.94:6813","nonce":3936794679}]},"public_addr":"172.21.15.94:6805/3936794679","cluster_addr":"172.21.15.94:6809/3936794679","heartbeat_back_addr":"172.21.15.94:6815/3936794679","heartbeat_front_addr":"172.21.15.94:6813/3936794679","state":["exists","up"]},{"osd":2,"uuid":"55862624-0f40-4cf4-8969-dc385ce085b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6817","nonce":163947633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6819","nonce":163947633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6823","nonce":163947633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":163947633},{"type":"v1","addr":"172.21.15.94:6821","nonce":163947633}]},"public_addr":"172.21.15.94:6817/163947633","cluster_addr":"172.21.15.94:6819/163947633","heartbeat_back_addr":"172.21.15.94:6823/163947633","heartbeat_front_addr":"172.21.15.94:6821/163947633","state":["exists","up"]},{"osd":3,"uuid":"2a01b817-57fc-4068-ad96-19def0495f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6825","nonce":3187639367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6827","nonce":3187639367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6831","nonce":3187639367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3187639367},{"type":"v1","addr":"172.21.15.94:6829","nonce":3187639367}]},"public_addr":"172.21.15.94:6825/3187639367","cluster_addr":"172.21.15.94:6827/3187639367","heartbeat_back_addr":"172.21.15.94:6831/3187639367","heartbeat_front_addr":"172.21.15.94:6829/3187639367","state":["exists","up"]},{"osd":4,"uuid":"6362ed7b-8d59-474a-9e4c-6459c133b664","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6812","nonce":955716519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6816","nonce":955716519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6824","nonce":955716519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":955716519},{"type":"v1","addr":"172.21.15.181:6820","nonce":955716519}]},"public_addr":"172.21.15.181:6812/955716519","cluster_addr":"172.21.15.181:6816/955716519","heartbeat_back_addr":"172.21.15.181:6824/955716519","heartbeat_front_addr":"172.21.15.181:6820/955716519","state":["exists","up"]},{"osd":5,"uuid":"bee62996-8fff-4730-9568-8f33784497c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6827","nonce":1536008029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6829","nonce":1536008029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6833","nonce":1536008029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1536008029},{"type":"v1","addr":"172.21.15.181:6831","nonce":1536008029}]},"public_addr":"172.21.15.181:6827/1536008029","cluster_addr":"172.21.15.181:6829/1536008029","heartbeat_back_addr":"172.21.15.181:6833/1536008029","heartbeat_front_addr":"172.21.15.181:6831/1536008029","state":["exists","up"]},{"osd":6,"uuid":"59d2093f-f326-45e0-b9a5-947eaf9ded46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6803","nonce":1977708519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6805","nonce":1977708519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6809","nonce":1977708519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1977708519},{"type":"v1","addr":"172.21.15.181:6807","nonce":1977708519}]},"public_addr":"172.21.15.181:6803/1977708519","cluster_addr":"172.21.15.181:6805/1977708519","heartbeat_back_addr":"172.21.15.181:6809/1977708519","heartbeat_front_addr":"172.21.15.181:6807/1977708519","state":["exists","up"]},{"osd":7,"uuid":"20c823a4-fe17-4261-8074-e8274fdb9274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6813","nonce":3568562877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6817","nonce":3568562877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6825","nonce":3568562877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6819","nonce":3568562877},{"type":"v1","addr":"172.21.15.181:6821","nonce":3568562877}]},"public_addr":"172.21.15.181:6813/3568562877","cluster_addr":"172.21.15.181:6817/3568562877","heartbeat_back_addr":"172.21.15.181:6825/3568562877","heartbeat_front_addr":"172.21.15.181:6821/3568562877","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.559509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.622129+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.787432+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.296011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.764068+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:28.430560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.510289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:32:27.564233+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6835/3607539278":"2024-09-11T23:41:05.521431+0000","172.21.15.94:6833/3607539278":"2024-09-11T23:41:05.521431+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T23:41:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.370+0000 7f290ae00640 1 -- 172.21.15.94:0/2358881454 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f28e40731b0 msgr2=0x7f28e4075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.370+0000 7f290ae00640 1 --2- 172.21.15.94:0/2358881454 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f28e40731b0 0x7f28e4075670 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f28f40af000 tx=0x7f28f401af20 comp rx=0 tx=0).stop 2024-09-10T23:41:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.370+0000 7f290ae00640 1 -- 172.21.15.94:0/2358881454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29040b2bb0 msgr2=0x7f290414f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.370+0000 7f290ae00640 1 --2- 172.21.15.94:0/2358881454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29040b2bb0 0x7f290414f060 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f28f000c4c0 tx=0x7f28f000c980 comp rx=0 tx=0).stop 2024-09-10T23:41:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.370+0000 7f290ae00640 1 -- 172.21.15.94:0/2358881454 shutdown_connections 2024-09-10T23:41:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.370+0000 7f290ae00640 1 --2- 172.21.15.94:0/2358881454 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f290415f5c0 0x7f2904152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.370+0000 7f290ae00640 1 --2- 172.21.15.94:0/2358881454 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f28e40731b0 0x7f28e4075670 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:12.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.370+0000 7f290ae00640 1 --2- 172.21.15.94:0/2358881454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29040b2bb0 0x7f290414f060 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:12.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.370+0000 7f290ae00640 1 --2- 172.21.15.94:0/2358881454 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29040af310 0x7f29040ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:12.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.370+0000 7f290ae00640 1 -- 172.21.15.94:0/2358881454 >> 172.21.15.94:0/2358881454 conn(0x7f29040d1c60 msgr2=0x7f29040b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:12.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.370+0000 7f290ae00640 1 -- 172.21.15.94:0/2358881454 shutdown_connections 2024-09-10T23:41:12.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:12.370+0000 7f290ae00640 1 -- 172.21.15.94:0/2358881454 wait complete. 2024-09-10T23:41:13.393 INFO:tasks.ceph:Scrubbing osd.0 2024-09-10T23:41:13.393 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-10T23:41:13.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.538+0000 7fa616a00640 1 -- 172.21.15.94:0/1901477731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6100d2290 msgr2=0x7fa6100b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:13.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.538+0000 7fa616a00640 1 --2- 172.21.15.94:0/1901477731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6100d2290 0x7fa6100b1e70 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fa6100d2850 tx=0x7fa60801c8c0 comp rx=0 tx=0).stop 2024-09-10T23:41:13.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.538+0000 7fa616a00640 1 -- 172.21.15.94:0/1901477731 shutdown_connections 2024-09-10T23:41:13.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.538+0000 7fa616a00640 1 --2- 172.21.15.94:0/1901477731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6100d2290 0x7fa6100b1e70 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:13.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.538+0000 7fa616a00640 1 --2- 172.21.15.94:0/1901477731 >> v2:172.21.15.181:3300/0 conn(0x7fa61015bd30 0x7fa61015c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:13.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.538+0000 7fa616a00640 1 -- 172.21.15.94:0/1901477731 >> 172.21.15.94:0/1901477731 conn(0x7fa61009aa70 msgr2=0x7fa61009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:13.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.538+0000 7fa616a00640 1 -- 172.21.15.94:0/1901477731 shutdown_connections 2024-09-10T23:41:13.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.538+0000 7fa616a00640 1 -- 172.21.15.94:0/1901477731 wait complete. 2024-09-10T23:41:13.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa616a00640 1 Processor -- start 2024-09-10T23:41:13.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa616a00640 1 -- start start 2024-09-10T23:41:13.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa616a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6100ae980 0x7fa610166200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:13.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa616a00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6100d2290 0x7fa610156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:13.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa616a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa61015bd30 0x7fa6100b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:13.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa616a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa610169550 con 0x7fa61015bd30 2024-09-10T23:41:13.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa616a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101693d0 con 0x7fa6100ae980 2024-09-10T23:41:13.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa616a00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa6101696d0 con 0x7fa6100d2290 2024-09-10T23:41:13.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa614a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa61015bd30 0x7fa6100b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:13.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa60fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6100ae980 0x7fa610166200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:13.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa60fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6100ae980 0x7fa610166200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.94:40018/0 (socket says 172.21.15.94:40018) 2024-09-10T23:41:13.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa60fe00640 1 -- 172.21.15.94:0/2817755722 learned_addr learned my addr 172.21.15.94:0/2817755722 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:13.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa607400640 1 --2- 172.21.15.94:0/2817755722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6100d2290 0x7fa610156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:13.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa614a00640 1 -- 172.21.15.94:0/2817755722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6100d2290 msgr2=0x7fa610156900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa614a00640 1 --2- 172.21.15.94:0/2817755722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6100d2290 0x7fa610156900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa614a00640 1 -- 172.21.15.94:0/2817755722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6100ae980 msgr2=0x7fa610166200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa614a00640 1 --2- 172.21.15.94:0/2817755722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6100ae980 0x7fa610166200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa614a00640 1 -- 172.21.15.94:0/2817755722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61015a720 con 0x7fa61015bd30 2024-09-10T23:41:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa614a00640 1 --2- 172.21.15.94:0/2817755722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa61015bd30 0x7fa6100b4ac0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fa6080ac000 tx=0x7fa6080a5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa60cc00640 1 -- 172.21.15.94:0/2817755722 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6080b2040 con 0x7fa61015bd30 2024-09-10T23:41:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa60cc00640 1 -- 172.21.15.94:0/2817755722 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6080043e0 con 0x7fa61015bd30 2024-09-10T23:41:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa60cc00640 1 -- 172.21.15.94:0/2817755722 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608005290 con 0x7fa61015bd30 2024-09-10T23:41:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa616a00640 1 -- 172.21.15.94:0/2817755722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61015aa00 con 0x7fa61015bd30 2024-09-10T23:41:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.542+0000 7fa616a00640 1 -- 172.21.15.94:0/2817755722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa610166740 con 0x7fa61015bd30 2024-09-10T23:41:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.546+0000 7fa616a00640 1 -- 172.21.15.94:0/2817755722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa6100b1e70 con 0x7fa61015bd30 2024-09-10T23:41:13.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.546+0000 7fa60cc00640 1 -- 172.21.15.94:0/2817755722 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7fa608005430 con 0x7fa61015bd30 2024-09-10T23:41:13.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.546+0000 7fa60cc00640 1 --2- 172.21.15.94:0/2817755722 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa5ec0731e0 0x7fa5ec0756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:13.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.546+0000 7fa60cc00640 1 -- 172.21.15.94:0/2817755722 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7fa60812c290 con 0x7fa61015bd30 2024-09-10T23:41:13.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.546+0000 7fa60fe00640 1 --2- 172.21.15.94:0/2817755722 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa5ec0731e0 0x7fa5ec0756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:13.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.546+0000 7fa60cc00640 1 --2- 172.21.15.94:0/2817755722 >> [v2:172.21.15.94:6800/576048946,v1:172.21.15.94:6801/576048946] conn(0x7fa5ec0790a0 0x7fa5ec07b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:13.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.546+0000 7fa60cc00640 1 -- 172.21.15.94:0/2817755722 --> [v2:172.21.15.94:6800/576048946,v1:172.21.15.94:6801/576048946] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa6080ade50 con 0x7fa5ec0790a0 2024-09-10T23:41:13.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.546+0000 7fa60cc00640 1 -- 172.21.15.94:0/2817755722 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) ==== 24+0+0 (secure 0 0 0) 0x7fa6080f5160 con 0x7fa61015bd30 2024-09-10T23:41:13.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.546+0000 7fa607400640 1 --2- 172.21.15.94:0/2817755722 >> [v2:172.21.15.94:6800/576048946,v1:172.21.15.94:6801/576048946] conn(0x7fa5ec0790a0 0x7fa5ec07b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:13.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.546+0000 7fa60fe00640 1 --2- 172.21.15.94:0/2817755722 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa5ec0731e0 0x7fa5ec0756a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa600009fc0 tx=0x7fa600009500 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:13.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.550+0000 7fa607400640 1 --2- 172.21.15.94:0/2817755722 >> [v2:172.21.15.94:6800/576048946,v1:172.21.15.94:6801/576048946] conn(0x7fa5ec0790a0 0x7fa5ec07b4c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:13.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.550+0000 7fa60cc00640 1 -- 172.21.15.94:0/2817755722 <== osd.0 v2:172.21.15.94:6800/576048946 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa6080ade50 con 0x7fa5ec0790a0 2024-09-10T23:41:13.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.562+0000 7fa616a00640 1 -- 172.21.15.94:0/2817755722 --> [v2:172.21.15.94:6800/576048946,v1:172.21.15.94:6801/576048946] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa6100d28a0 con 0x7fa5ec0790a0 2024-09-10T23:41:13.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.562+0000 7fa60cc00640 1 -- 172.21.15.94:0/2817755722 <== osd.0 v2:172.21.15.94:6800/576048946 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa6100d28a0 con 0x7fa5ec0790a0 2024-09-10T23:41:13.563 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:41:13.563 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T23:41:13.563 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:41:13.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.562+0000 7fa616a00640 1 -- 172.21.15.94:0/2817755722 >> [v2:172.21.15.94:6800/576048946,v1:172.21.15.94:6801/576048946] conn(0x7fa5ec0790a0 msgr2=0x7fa5ec07b4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.562+0000 7fa616a00640 1 --2- 172.21.15.94:0/2817755722 >> [v2:172.21.15.94:6800/576048946,v1:172.21.15.94:6801/576048946] conn(0x7fa5ec0790a0 0x7fa5ec07b4c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.562+0000 7fa616a00640 1 -- 172.21.15.94:0/2817755722 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa5ec0731e0 msgr2=0x7fa5ec0756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.562+0000 7fa616a00640 1 --2- 172.21.15.94:0/2817755722 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa5ec0731e0 0x7fa5ec0756a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa600009fc0 tx=0x7fa600009500 comp rx=0 tx=0).stop 2024-09-10T23:41:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.562+0000 7fa616a00640 1 -- 172.21.15.94:0/2817755722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa61015bd30 msgr2=0x7fa6100b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.562+0000 7fa616a00640 1 --2- 172.21.15.94:0/2817755722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa61015bd30 0x7fa6100b4ac0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fa6080ac000 tx=0x7fa6080a5920 comp rx=0 tx=0).stop 2024-09-10T23:41:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.562+0000 7fa614a00640 1 -- 172.21.15.94:0/2817755722 reap_dead start 2024-09-10T23:41:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.562+0000 7fa616a00640 1 -- 172.21.15.94:0/2817755722 shutdown_connections 2024-09-10T23:41:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.562+0000 7fa616a00640 1 -- 172.21.15.94:0/2817755722 >> 172.21.15.94:0/2817755722 conn(0x7fa61009aa70 msgr2=0x7fa61015b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.562+0000 7fa616a00640 1 -- 172.21.15.94:0/2817755722 shutdown_connections 2024-09-10T23:41:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.562+0000 7fa616a00640 1 -- 172.21.15.94:0/2817755722 wait complete. 2024-09-10T23:41:13.581 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-10T23:41:13.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd12000640 1 -- 172.21.15.94:0/3606788881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcf0003a10 msgr2=0x7fdcf0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:13.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd12000640 1 --2- 172.21.15.94:0/3606788881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcf0003a10 0x7fdcf0003e30 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc009f10 tx=0x7fdcfc01d5a0 comp rx=0 tx=0).stop 2024-09-10T23:41:13.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd0be00640 1 -- 172.21.15.94:0/3606788881 reap_dead start 2024-09-10T23:41:13.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd12000640 1 -- 172.21.15.94:0/3606788881 shutdown_connections 2024-09-10T23:41:13.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd12000640 1 -- 172.21.15.94:0/3606788881 >> 172.21.15.94:0/3606788881 conn(0x7fdd0c0cfaf0 msgr2=0x7fdd0c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:13.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd12000640 1 -- 172.21.15.94:0/3606788881 shutdown_connections 2024-09-10T23:41:13.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd12000640 1 -- 172.21.15.94:0/3606788881 wait complete. 2024-09-10T23:41:13.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd12000640 1 Processor -- start 2024-09-10T23:41:13.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd12000640 1 -- start start 2024-09-10T23:41:13.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd12000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdcf0003a10 0x7fdd0c0aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:13.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd12000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd0c0cb690 0x7fdd0c143c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:13.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd12000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd0c144160 0x7fdd0c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:13.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd12000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c167010 con 0x7fdd0c144160 2024-09-10T23:41:13.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd12000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c166e90 con 0x7fdd0c0cb690 2024-09-10T23:41:13.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd12000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fdd0c167190 con 0x7fdcf0003a10 2024-09-10T23:41:13.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd0be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd0c144160 0x7fdd0c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:13.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd0be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd0c144160 0x7fdd0c0ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43448/0 (socket says 172.21.15.94:43448) 2024-09-10T23:41:13.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd0be00640 1 -- 172.21.15.94:0/427047637 learned_addr learned my addr 172.21.15.94:0/427047637 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:13.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd02a00640 1 --2- 172.21.15.94:0/427047637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd0c0cb690 0x7fdd0c143c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:13.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.726+0000 7fdd0b400640 1 --2- 172.21.15.94:0/427047637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdcf0003a10 0x7fdd0c0aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:13.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.730+0000 7fdd0be00640 1 -- 172.21.15.94:0/427047637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdcf0003a10 msgr2=0x7fdd0c0aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:13.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.730+0000 7fdd0be00640 1 --2- 172.21.15.94:0/427047637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdcf0003a10 0x7fdd0c0aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:13.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.730+0000 7fdd0be00640 1 -- 172.21.15.94:0/427047637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd0c0cb690 msgr2=0x7fdd0c143c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:13.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.730+0000 7fdd0be00640 1 --2- 172.21.15.94:0/427047637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd0c0cb690 0x7fdd0c143c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:13.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.730+0000 7fdd0be00640 1 -- 172.21.15.94:0/427047637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c144600 con 0x7fdd0c144160 2024-09-10T23:41:13.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.730+0000 7fdd0be00640 1 --2- 172.21.15.94:0/427047637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd0c144160 0x7fdd0c0ce510 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0c5e70 tx=0x7fdd04006ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:13.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.730+0000 7fdd03e00640 1 -- 172.21.15.94:0/427047637 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd040107d0 con 0x7fdd0c144160 2024-09-10T23:41:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.730+0000 7fdd03e00640 1 -- 172.21.15.94:0/427047637 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd04016070 con 0x7fdd0c144160 2024-09-10T23:41:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.730+0000 7fdd03e00640 1 -- 172.21.15.94:0/427047637 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04010d00 con 0x7fdd0c144160 2024-09-10T23:41:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.730+0000 7fdd12000640 1 -- 172.21.15.94:0/427047637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0c1448e0 con 0x7fdd0c144160 2024-09-10T23:41:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.730+0000 7fdd12000640 1 -- 172.21.15.94:0/427047637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0c0aea40 con 0x7fdd0c144160 2024-09-10T23:41:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.730+0000 7fdd12000640 1 -- 172.21.15.94:0/427047637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd0c0c5530 con 0x7fdd0c144160 2024-09-10T23:41:13.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.730+0000 7fdd03e00640 1 -- 172.21.15.94:0/427047637 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7fdd0401b460 con 0x7fdd0c144160 2024-09-10T23:41:13.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.730+0000 7fdd03e00640 1 --2- 172.21.15.94:0/427047637 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdce80731b0 0x7fdce8075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:13.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.730+0000 7fdd03e00640 1 -- 172.21.15.94:0/427047637 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7fdd040926c0 con 0x7fdd0c144160 2024-09-10T23:41:13.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.730+0000 7fdd0b400640 1 --2- 172.21.15.94:0/427047637 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdce80731b0 0x7fdce8075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:13.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.734+0000 7fdd0b400640 1 --2- 172.21.15.94:0/427047637 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdce80731b0 0x7fdce8075670 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0af000 tx=0x7fdcfc01a340 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:13.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.734+0000 7fdd03e00640 1 -- 172.21.15.94:0/427047637 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdd0c0701c0 con 0x7fdd0c144160 2024-09-10T23:41:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.902+0000 7fdd12000640 1 -- 172.21.15.94:0/427047637 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fdd0c0aec10 con 0x7fdce80731b0 2024-09-10T23:41:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.902+0000 7fdd03e00640 1 -- 172.21.15.94:0/427047637 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdd0c0aec10 con 0x7fdce80731b0 2024-09-10T23:41:13.902 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 0 to deep-scrub 2024-09-10T23:41:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.902+0000 7fdd12000640 1 -- 172.21.15.94:0/427047637 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdce80731b0 msgr2=0x7fdce8075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.902+0000 7fdd12000640 1 --2- 172.21.15.94:0/427047637 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdce80731b0 0x7fdce8075670 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0af000 tx=0x7fdcfc01a340 comp rx=0 tx=0).stop 2024-09-10T23:41:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.902+0000 7fdd12000640 1 -- 172.21.15.94:0/427047637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd0c144160 msgr2=0x7fdd0c0ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.902+0000 7fdd12000640 1 --2- 172.21.15.94:0/427047637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd0c144160 0x7fdd0c0ce510 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0c5e70 tx=0x7fdd04006ef0 comp rx=0 tx=0).stop 2024-09-10T23:41:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.902+0000 7fdd12000640 1 -- 172.21.15.94:0/427047637 shutdown_connections 2024-09-10T23:41:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.902+0000 7fdd12000640 1 --2- 172.21.15.94:0/427047637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd0c0cb690 0x7fdd0c143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.902+0000 7fdd12000640 1 --2- 172.21.15.94:0/427047637 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fdce80731b0 0x7fdce8075670 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.902+0000 7fdd12000640 1 --2- 172.21.15.94:0/427047637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd0c144160 0x7fdd0c0ce510 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:13.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.902+0000 7fdd12000640 1 --2- 172.21.15.94:0/427047637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdcf0003a10 0x7fdd0c0aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:13.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.902+0000 7fdd12000640 1 -- 172.21.15.94:0/427047637 >> 172.21.15.94:0/427047637 conn(0x7fdd0c0cfaf0 msgr2=0x7fdd0c0b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:13.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.902+0000 7fdd12000640 1 -- 172.21.15.94:0/427047637 shutdown_connections 2024-09-10T23:41:13.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:13.902+0000 7fdd12000640 1 -- 172.21.15.94:0/427047637 wait complete. 2024-09-10T23:41:13.924 INFO:tasks.ceph:Scrubbing osd.1 2024-09-10T23:41:13.925 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-10T23:41:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.070+0000 7f01b9200640 1 -- 172.21.15.94:0/115834226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01980051d0 msgr2=0x7f019800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.070+0000 7f01b9200640 1 --2- 172.21.15.94:0/115834226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01980051d0 0x7f019800b9d0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f01ac001dc0 tx=0x7f01ac01cb30 comp rx=0 tx=0).stop 2024-09-10T23:41:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.070+0000 7f01b9200640 1 -- 172.21.15.94:0/115834226 shutdown_connections 2024-09-10T23:41:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.070+0000 7f01b9200640 1 --2- 172.21.15.94:0/115834226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01980051d0 0x7f019800b9d0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.070+0000 7f01b9200640 1 -- 172.21.15.94:0/115834226 >> 172.21.15.94:0/115834226 conn(0x7f01b406cdf0 msgr2=0x7f01b406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.070+0000 7f01b9200640 1 -- 172.21.15.94:0/115834226 shutdown_connections 2024-09-10T23:41:14.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.070+0000 7f01b9200640 1 -- 172.21.15.94:0/115834226 wait complete. 2024-09-10T23:41:14.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b9200640 1 Processor -- start 2024-09-10T23:41:14.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b9200640 1 -- start start 2024-09-10T23:41:14.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b9200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01980051d0 0x7f01b40aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b9200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01b4152b30 0x7f01b4154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b9200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01b4154fb0 0x7f01b40b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b9200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f01b40cff20 con 0x7f01980051d0 2024-09-10T23:41:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b9200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f01b40cfda0 con 0x7f01b4154fb0 2024-09-10T23:41:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b9200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f01b40d00a0 con 0x7f01b4152b30 2024-09-10T23:41:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01980051d0 0x7f01b40aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01980051d0 0x7f01b40aec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56846/0 (socket says 172.21.15.94:56846) 2024-09-10T23:41:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b1e00640 1 -- 172.21.15.94:0/1412663660 learned_addr learned my addr 172.21.15.94:0/1412663660 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b1400640 1 --2- 172.21.15.94:0/1412663660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01b4152b30 0x7f01b4154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b2800640 1 --2- 172.21.15.94:0/1412663660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01b4154fb0 0x7f01b40b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b1e00640 1 -- 172.21.15.94:0/1412663660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01b4152b30 msgr2=0x7f01b4154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b1e00640 1 --2- 172.21.15.94:0/1412663660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01b4152b30 0x7f01b4154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b1e00640 1 -- 172.21.15.94:0/1412663660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01b4154fb0 msgr2=0x7f01b40b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b1e00640 1 --2- 172.21.15.94:0/1412663660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01b4154fb0 0x7f01b40b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b1e00640 1 -- 172.21.15.94:0/1412663660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b41539d0 con 0x7f01980051d0 2024-09-10T23:41:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b1e00640 1 --2- 172.21.15.94:0/1412663660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01980051d0 0x7f01b40aec70 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f01a4011750 tx=0x7f01a4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:14.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01aa000640 1 -- 172.21.15.94:0/1412663660 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a400fd50 con 0x7f01980051d0 2024-09-10T23:41:14.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01aa000640 1 -- 172.21.15.94:0/1412663660 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01a4016070 con 0x7f01980051d0 2024-09-10T23:41:14.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01aa000640 1 -- 172.21.15.94:0/1412663660 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4012690 con 0x7f01980051d0 2024-09-10T23:41:14.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b9200640 1 -- 172.21.15.94:0/1412663660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b4153cb0 con 0x7f01980051d0 2024-09-10T23:41:14.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b9200640 1 -- 172.21.15.94:0/1412663660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b4154270 con 0x7f01980051d0 2024-09-10T23:41:14.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.074+0000 7f01b9200640 1 -- 172.21.15.94:0/1412663660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f01b415e980 con 0x7f01980051d0 2024-09-10T23:41:14.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.078+0000 7f01aa000640 1 -- 172.21.15.94:0/1412663660 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f01a4012830 con 0x7f01980051d0 2024-09-10T23:41:14.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.078+0000 7f01aa000640 1 --2- 172.21.15.94:0/1412663660 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f01940731b0 0x7f0194075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.078+0000 7f01aa000640 1 -- 172.21.15.94:0/1412663660 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7f01a40933e0 con 0x7f01980051d0 2024-09-10T23:41:14.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.078+0000 7f01b1400640 1 --2- 172.21.15.94:0/1412663660 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f01940731b0 0x7f0194075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.078+0000 7f01aa000640 1 --2- 172.21.15.94:0/1412663660 >> [v2:172.21.15.94:6803/3936794679,v1:172.21.15.94:6805/3936794679] conn(0x7f0194079070 0x7f019407b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.078+0000 7f01aa000640 1 -- 172.21.15.94:0/1412663660 --> [v2:172.21.15.94:6803/3936794679,v1:172.21.15.94:6805/3936794679] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f019407bb60 con 0x7f0194079070 2024-09-10T23:41:14.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.078+0000 7f01aa000640 1 -- 172.21.15.94:0/1412663660 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) ==== 24+0+0 (secure 0 0 0) 0x7f01a4095c40 con 0x7f01980051d0 2024-09-10T23:41:14.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.078+0000 7f01b2800640 1 --2- 172.21.15.94:0/1412663660 >> [v2:172.21.15.94:6803/3936794679,v1:172.21.15.94:6805/3936794679] conn(0x7f0194079070 0x7f019407b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.078+0000 7f01b1400640 1 --2- 172.21.15.94:0/1412663660 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f01940731b0 0x7f0194075670 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f01b415ae00 tx=0x7f01a0006040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:14.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.078+0000 7f01b2800640 1 --2- 172.21.15.94:0/1412663660 >> [v2:172.21.15.94:6803/3936794679,v1:172.21.15.94:6805/3936794679] conn(0x7f0194079070 0x7f019407b490 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:14.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.082+0000 7f01aa000640 1 -- 172.21.15.94:0/1412663660 <== osd.1 v2:172.21.15.94:6803/3936794679 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f019407bb60 con 0x7f0194079070 2024-09-10T23:41:14.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.094+0000 7f01b9200640 1 -- 172.21.15.94:0/1412663660 --> [v2:172.21.15.94:6803/3936794679,v1:172.21.15.94:6805/3936794679] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f01b41e4da0 con 0x7f0194079070 2024-09-10T23:41:14.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.094+0000 7f01aa000640 1 -- 172.21.15.94:0/1412663660 <== osd.1 v2:172.21.15.94:6803/3936794679 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f01b41e4da0 con 0x7f0194079070 2024-09-10T23:41:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:41:14.094 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T23:41:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:41:14.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.094+0000 7f01b9200640 1 -- 172.21.15.94:0/1412663660 >> [v2:172.21.15.94:6803/3936794679,v1:172.21.15.94:6805/3936794679] conn(0x7f0194079070 msgr2=0x7f019407b490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.094+0000 7f01b9200640 1 --2- 172.21.15.94:0/1412663660 >> [v2:172.21.15.94:6803/3936794679,v1:172.21.15.94:6805/3936794679] conn(0x7f0194079070 0x7f019407b490 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.094+0000 7f01b9200640 1 -- 172.21.15.94:0/1412663660 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f01940731b0 msgr2=0x7f0194075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.094+0000 7f01b9200640 1 --2- 172.21.15.94:0/1412663660 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f01940731b0 0x7f0194075670 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f01b415ae00 tx=0x7f01a0006040 comp rx=0 tx=0).stop 2024-09-10T23:41:14.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.094+0000 7f01b9200640 1 -- 172.21.15.94:0/1412663660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01980051d0 msgr2=0x7f01b40aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.094+0000 7f01b9200640 1 --2- 172.21.15.94:0/1412663660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01980051d0 0x7f01b40aec70 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f01a4011750 tx=0x7f01a4011c10 comp rx=0 tx=0).stop 2024-09-10T23:41:14.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.094+0000 7f01b2800640 1 -- 172.21.15.94:0/1412663660 reap_dead start 2024-09-10T23:41:14.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.094+0000 7f01b9200640 1 -- 172.21.15.94:0/1412663660 shutdown_connections 2024-09-10T23:41:14.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.094+0000 7f01b9200640 1 -- 172.21.15.94:0/1412663660 >> 172.21.15.94:0/1412663660 conn(0x7f01b406cdf0 msgr2=0x7f01b415dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:14.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.094+0000 7f01b9200640 1 -- 172.21.15.94:0/1412663660 shutdown_connections 2024-09-10T23:41:14.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.094+0000 7f01b9200640 1 -- 172.21.15.94:0/1412663660 wait complete. 2024-09-10T23:41:14.113 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-10T23:41:14.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.254+0000 7f95cfa00640 1 -- 172.21.15.94:0/2534172074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c80d2780 msgr2=0x7f95c80d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.254+0000 7f95cfa00640 1 --2- 172.21.15.94:0/2534172074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c80d2780 0x7f95c80d03f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009840 tx=0x7f95b001c790 comp rx=0 tx=0).stop 2024-09-10T23:41:14.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.254+0000 7f95cfa00640 1 -- 172.21.15.94:0/2534172074 shutdown_connections 2024-09-10T23:41:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.254+0000 7f95cfa00640 1 --2- 172.21.15.94:0/2534172074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c80d2780 0x7f95c80d03f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.254+0000 7f95cfa00640 1 -- 172.21.15.94:0/2534172074 >> 172.21.15.94:0/2534172074 conn(0x7f95c80d1c60 msgr2=0x7f95c80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.254+0000 7f95cfa00640 1 -- 172.21.15.94:0/2534172074 shutdown_connections 2024-09-10T23:41:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.254+0000 7f95cfa00640 1 -- 172.21.15.94:0/2534172074 wait complete. 2024-09-10T23:41:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.254+0000 7f95cfa00640 1 Processor -- start 2024-09-10T23:41:14.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.254+0000 7f95cfa00640 1 -- start start 2024-09-10T23:41:14.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.254+0000 7f95cfa00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95c80c5e50 0x7f95c814ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.254+0000 7f95cfa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c80d2780 0x7f95c8158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.254+0000 7f95cfa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c815bf60 0x7f95c8157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.254+0000 7f95cfa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95c8169f90 con 0x7f95c80d2780 2024-09-10T23:41:14.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.254+0000 7f95cfa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f95c8169e10 con 0x7f95c815bf60 2024-09-10T23:41:14.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.254+0000 7f95cfa00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f95c816a110 con 0x7f95c80c5e50 2024-09-10T23:41:14.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95bfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c80d2780 0x7f95c8158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95bfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c80d2780 0x7f95c8158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56850/0 (socket says 172.21.15.94:56850) 2024-09-10T23:41:14.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95bfe00640 1 -- 172.21.15.94:0/2444578897 learned_addr learned my addr 172.21.15.94:0/2444578897 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:14.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95cd000640 1 --2- 172.21.15.94:0/2444578897 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95c80c5e50 0x7f95c814ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95cda00640 1 --2- 172.21.15.94:0/2444578897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c815bf60 0x7f95c8157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95bfe00640 1 -- 172.21.15.94:0/2444578897 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95c80c5e50 msgr2=0x7f95c814ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95bfe00640 1 --2- 172.21.15.94:0/2444578897 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95c80c5e50 0x7f95c814ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95bfe00640 1 -- 172.21.15.94:0/2444578897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c815bf60 msgr2=0x7f95c8157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95bfe00640 1 --2- 172.21.15.94:0/2444578897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c815bf60 0x7f95c8157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95bfe00640 1 -- 172.21.15.94:0/2444578897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c815dd90 con 0x7f95c80d2780 2024-09-10T23:41:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95cda00640 1 --2- 172.21.15.94:0/2444578897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c815bf60 0x7f95c8157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:41:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95bfe00640 1 --2- 172.21.15.94:0/2444578897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c80d2780 0x7f95c8158a20 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f95b00193a0 tx=0x7f95b0004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95c5600640 1 -- 172.21.15.94:0/2444578897 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b00b2040 con 0x7f95c80d2780 2024-09-10T23:41:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95c5600640 1 -- 172.21.15.94:0/2444578897 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95b00a5c50 con 0x7f95c80d2780 2024-09-10T23:41:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95c5600640 1 -- 172.21.15.94:0/2444578897 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0005020 con 0x7f95c80d2780 2024-09-10T23:41:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95cfa00640 1 -- 172.21.15.94:0/2444578897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c815e070 con 0x7f95c80d2780 2024-09-10T23:41:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95cfa00640 1 -- 172.21.15.94:0/2444578897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c81564d0 con 0x7f95c80d2780 2024-09-10T23:41:14.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95cfa00640 1 -- 172.21.15.94:0/2444578897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c80c62e0 con 0x7f95c80d2780 2024-09-10T23:41:14.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.258+0000 7f95c5600640 1 -- 172.21.15.94:0/2444578897 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f95b00b5070 con 0x7f95c80d2780 2024-09-10T23:41:14.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.262+0000 7f95c5600640 1 --2- 172.21.15.94:0/2444578897 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95a40731b0 0x7f95a4075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.262+0000 7f95c5600640 1 -- 172.21.15.94:0/2444578897 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7f95b012b330 con 0x7f95c80d2780 2024-09-10T23:41:14.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.262+0000 7f95cd000640 1 --2- 172.21.15.94:0/2444578897 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95a40731b0 0x7f95a4075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.262+0000 7f95cd000640 1 --2- 172.21.15.94:0/2444578897 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95a40731b0 0x7f95a4075670 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f95c809fdd0 tx=0x7f95b8007c90 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.266+0000 7f95c5600640 1 -- 172.21.15.94:0/2444578897 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f95b0016410 con 0x7f95c80d2780 2024-09-10T23:41:14.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.430+0000 7f95cfa00640 1 -- 172.21.15.94:0/2444578897 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f95c80d0b40 con 0x7f95a40731b0 2024-09-10T23:41:14.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.430+0000 7f95c5600640 1 -- 172.21.15.94:0/2444578897 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f95c80d0b40 con 0x7f95a40731b0 2024-09-10T23:41:14.429 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 1 to deep-scrub 2024-09-10T23:41:14.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.430+0000 7f95cfa00640 1 -- 172.21.15.94:0/2444578897 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95a40731b0 msgr2=0x7f95a4075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.430+0000 7f95cfa00640 1 --2- 172.21.15.94:0/2444578897 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95a40731b0 0x7f95a4075670 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f95c809fdd0 tx=0x7f95b8007c90 comp rx=0 tx=0).stop 2024-09-10T23:41:14.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.430+0000 7f95cfa00640 1 -- 172.21.15.94:0/2444578897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c80d2780 msgr2=0x7f95c8158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.430+0000 7f95cfa00640 1 --2- 172.21.15.94:0/2444578897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c80d2780 0x7f95c8158a20 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f95b00193a0 tx=0x7f95b0004620 comp rx=0 tx=0).stop 2024-09-10T23:41:14.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.430+0000 7f95cfa00640 1 -- 172.21.15.94:0/2444578897 shutdown_connections 2024-09-10T23:41:14.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.430+0000 7f95cfa00640 1 --2- 172.21.15.94:0/2444578897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95c815bf60 0x7f95c8157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.430+0000 7f95cfa00640 1 --2- 172.21.15.94:0/2444578897 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f95a40731b0 0x7f95a4075670 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.430+0000 7f95cfa00640 1 --2- 172.21.15.94:0/2444578897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95c80d2780 0x7f95c8158a20 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.430+0000 7f95cfa00640 1 --2- 172.21.15.94:0/2444578897 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95c80c5e50 0x7f95c814ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.430+0000 7f95cfa00640 1 -- 172.21.15.94:0/2444578897 >> 172.21.15.94:0/2444578897 conn(0x7f95c80d1c60 msgr2=0x7f95c80b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:14.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.430+0000 7f95cfa00640 1 -- 172.21.15.94:0/2444578897 shutdown_connections 2024-09-10T23:41:14.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.430+0000 7f95cfa00640 1 -- 172.21.15.94:0/2444578897 wait complete. 2024-09-10T23:41:14.451 INFO:tasks.ceph:Scrubbing osd.2 2024-09-10T23:41:14.451 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-10T23:41:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.598+0000 7f739f000640 1 -- 172.21.15.94:0/3477465056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7398071a20 msgr2=0x7f73980cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.598+0000 7f739f000640 1 --2- 172.21.15.94:0/3477465056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7398071a20 0x7f73980cf390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7380009840 tx=0x7f738001c820 comp rx=0 tx=0).stop 2024-09-10T23:41:14.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.598+0000 7f739f000640 1 -- 172.21.15.94:0/3477465056 shutdown_connections 2024-09-10T23:41:14.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.598+0000 7f739f000640 1 --2- 172.21.15.94:0/3477465056 >> v2:172.21.15.181:3300/0 conn(0x7f73980d1690 0x7f73980cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.598+0000 7f739f000640 1 --2- 172.21.15.94:0/3477465056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7398071a20 0x7f73980cf390 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.598+0000 7f739f000640 1 -- 172.21.15.94:0/3477465056 >> 172.21.15.94:0/3477465056 conn(0x7f73980d0b70 msgr2=0x7f73980d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:14.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.598+0000 7f739f000640 1 -- 172.21.15.94:0/3477465056 shutdown_connections 2024-09-10T23:41:14.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.598+0000 7f739f000640 1 -- 172.21.15.94:0/3477465056 wait complete. 2024-09-10T23:41:14.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.598+0000 7f739f000640 1 Processor -- start 2024-09-10T23:41:14.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.598+0000 7f739f000640 1 -- start start 2024-09-10T23:41:14.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.598+0000 7f739f000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f739806cc20 0x7f7398141290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.598+0000 7f739f000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7398071a20 0x7f7398140b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.598+0000 7f739f000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73980d1690 0x7f7398159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.598+0000 7f739f000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7398168e60 con 0x7f739806cc20 2024-09-10T23:41:14.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.598+0000 7f739f000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7398168ce0 con 0x7f7398071a20 2024-09-10T23:41:14.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.598+0000 7f739f000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f7398168fe0 con 0x7f73980d1690 2024-09-10T23:41:14.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f7397e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f739806cc20 0x7f7398141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f7397e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f739806cc20 0x7f7398141290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56852/0 (socket says 172.21.15.94:56852) 2024-09-10T23:41:14.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f7397e00640 1 -- 172.21.15.94:0/3600339683 learned_addr learned my addr 172.21.15.94:0/3600339683 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:14.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f739d000640 1 --2- 172.21.15.94:0/3600339683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73980d1690 0x7f7398159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f738f400640 1 --2- 172.21.15.94:0/3600339683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7398071a20 0x7f7398140b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f7397e00640 1 -- 172.21.15.94:0/3600339683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73980d1690 msgr2=0x7f7398159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f7397e00640 1 --2- 172.21.15.94:0/3600339683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73980d1690 0x7f7398159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f7397e00640 1 -- 172.21.15.94:0/3600339683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7398071a20 msgr2=0x7f7398140b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f7397e00640 1 --2- 172.21.15.94:0/3600339683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7398071a20 0x7f7398140b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f7397e00640 1 -- 172.21.15.94:0/3600339683 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739815c480 con 0x7f739806cc20 2024-09-10T23:41:14.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f7397e00640 1 --2- 172.21.15.94:0/3600339683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f739806cc20 0x7f7398141290 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f738800c490 tx=0x7f738800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:14.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f7394c00640 1 -- 172.21.15.94:0/3600339683 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388011790 con 0x7f739806cc20 2024-09-10T23:41:14.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f7394c00640 1 -- 172.21.15.94:0/3600339683 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7388017070 con 0x7f739806cc20 2024-09-10T23:41:14.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f739f000640 1 -- 172.21.15.94:0/3600339683 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7398141080 con 0x7f739806cc20 2024-09-10T23:41:14.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f7394c00640 1 -- 172.21.15.94:0/3600339683 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388011ce0 con 0x7f739806cc20 2024-09-10T23:41:14.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f739f000640 1 -- 172.21.15.94:0/3600339683 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7398147580 con 0x7f739806cc20 2024-09-10T23:41:14.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f739f000640 1 -- 172.21.15.94:0/3600339683 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f739806d000 con 0x7f739806cc20 2024-09-10T23:41:14.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.602+0000 7f7394c00640 1 -- 172.21.15.94:0/3600339683 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f738801c460 con 0x7f739806cc20 2024-09-10T23:41:14.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.606+0000 7f7394c00640 1 --2- 172.21.15.94:0/3600339683 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73740731e0 0x7f73740756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.606+0000 7f7394c00640 1 -- 172.21.15.94:0/3600339683 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7f73880925f0 con 0x7f739806cc20 2024-09-10T23:41:14.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.606+0000 7f738f400640 1 --2- 172.21.15.94:0/3600339683 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73740731e0 0x7f73740756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.606+0000 7f7394c00640 1 --2- 172.21.15.94:0/3600339683 >> [v2:172.21.15.94:6816/163947633,v1:172.21.15.94:6817/163947633] conn(0x7f73740790a0 0x7f737407b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.606+0000 7f7394c00640 1 -- 172.21.15.94:0/3600339683 --> [v2:172.21.15.94:6816/163947633,v1:172.21.15.94:6817/163947633] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7388014e70 con 0x7f73740790a0 2024-09-10T23:41:14.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.606+0000 7f7394c00640 1 -- 172.21.15.94:0/3600339683 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) ==== 24+0+0 (secure 0 0 0) 0x7f73880952f0 con 0x7f739806cc20 2024-09-10T23:41:14.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.606+0000 7f739d000640 1 --2- 172.21.15.94:0/3600339683 >> [v2:172.21.15.94:6816/163947633,v1:172.21.15.94:6817/163947633] conn(0x7f73740790a0 0x7f737407b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.606+0000 7f738f400640 1 --2- 172.21.15.94:0/3600339683 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73740731e0 0x7f73740756a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f73800af000 tx=0x7f738001b060 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:14.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.606+0000 7f739d000640 1 --2- 172.21.15.94:0/3600339683 >> [v2:172.21.15.94:6816/163947633,v1:172.21.15.94:6817/163947633] conn(0x7f73740790a0 0x7f737407b4c0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:14.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.606+0000 7f7394c00640 1 -- 172.21.15.94:0/3600339683 <== osd.2 v2:172.21.15.94:6816/163947633 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7388014e70 con 0x7f73740790a0 2024-09-10T23:41:14.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.618+0000 7f739f000640 1 -- 172.21.15.94:0/3600339683 --> [v2:172.21.15.94:6816/163947633,v1:172.21.15.94:6817/163947633] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f739815e480 con 0x7f73740790a0 2024-09-10T23:41:14.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.618+0000 7f7394c00640 1 -- 172.21.15.94:0/3600339683 <== osd.2 v2:172.21.15.94:6816/163947633 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f739815e480 con 0x7f73740790a0 2024-09-10T23:41:14.621 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:41:14.622 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T23:41:14.622 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:41:14.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.618+0000 7f739f000640 1 -- 172.21.15.94:0/3600339683 >> [v2:172.21.15.94:6816/163947633,v1:172.21.15.94:6817/163947633] conn(0x7f73740790a0 msgr2=0x7f737407b4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.618+0000 7f739f000640 1 --2- 172.21.15.94:0/3600339683 >> [v2:172.21.15.94:6816/163947633,v1:172.21.15.94:6817/163947633] conn(0x7f73740790a0 0x7f737407b4c0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.618+0000 7f739f000640 1 -- 172.21.15.94:0/3600339683 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73740731e0 msgr2=0x7f73740756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.618+0000 7f739f000640 1 --2- 172.21.15.94:0/3600339683 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f73740731e0 0x7f73740756a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f73800af000 tx=0x7f738001b060 comp rx=0 tx=0).stop 2024-09-10T23:41:14.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.622+0000 7f739f000640 1 -- 172.21.15.94:0/3600339683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f739806cc20 msgr2=0x7f7398141290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.622+0000 7f739f000640 1 --2- 172.21.15.94:0/3600339683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f739806cc20 0x7f7398141290 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f738800c490 tx=0x7f738800c890 comp rx=0 tx=0).stop 2024-09-10T23:41:14.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.622+0000 7f739d000640 1 -- 172.21.15.94:0/3600339683 reap_dead start 2024-09-10T23:41:14.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.622+0000 7f739f000640 1 -- 172.21.15.94:0/3600339683 shutdown_connections 2024-09-10T23:41:14.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.622+0000 7f739f000640 1 -- 172.21.15.94:0/3600339683 >> 172.21.15.94:0/3600339683 conn(0x7f73980d0b70 msgr2=0x7f739814c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:14.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.622+0000 7f739f000640 1 -- 172.21.15.94:0/3600339683 shutdown_connections 2024-09-10T23:41:14.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.622+0000 7f739f000640 1 -- 172.21.15.94:0/3600339683 wait complete. 2024-09-10T23:41:14.638 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-10T23:41:14.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e52000640 1 -- 172.21.15.94:0/2647489776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e4c06e7c0 msgr2=0x7f2e4c06ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e52000640 1 --2- 172.21.15.94:0/2647489776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e4c06e7c0 0x7f2e4c06ec50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2e4400ad90 tx=0x7f2e4401dcb0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e52000640 1 -- 172.21.15.94:0/2647489776 shutdown_connections 2024-09-10T23:41:14.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e52000640 1 --2- 172.21.15.94:0/2647489776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e4c06e7c0 0x7f2e4c06ec50 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e52000640 1 --2- 172.21.15.94:0/2647489776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e4c0c64f0 0x7f2e4c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e52000640 1 -- 172.21.15.94:0/2647489776 >> 172.21.15.94:0/2647489776 conn(0x7f2e4c0cfaf0 msgr2=0x7f2e4c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:14.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e52000640 1 -- 172.21.15.94:0/2647489776 shutdown_connections 2024-09-10T23:41:14.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e52000640 1 -- 172.21.15.94:0/2647489776 wait complete. 2024-09-10T23:41:14.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e52000640 1 Processor -- start 2024-09-10T23:41:14.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e52000640 1 -- start start 2024-09-10T23:41:14.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e52000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e4c06e7c0 0x7f2e4c0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e52000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e4c0c64f0 0x7f2e4c0c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e52000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e4c0ce280 0x7f2e4c1fafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e52000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c15b5d0 con 0x7f2e4c06e7c0 2024-09-10T23:41:14.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e52000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c15b450 con 0x7f2e4c0ce280 2024-09-10T23:41:14.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e52000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4c15b750 con 0x7f2e4c0c64f0 2024-09-10T23:41:14.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e4b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e4c06e7c0 0x7f2e4c0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e4b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e4c06e7c0 0x7f2e4c0b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56868/0 (socket says 172.21.15.94:56868) 2024-09-10T23:41:14.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e4b400640 1 -- 172.21.15.94:0/2009444217 learned_addr learned my addr 172.21.15.94:0/2009444217 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:14.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e4be00640 1 --2- 172.21.15.94:0/2009444217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e4c0ce280 0x7f2e4c1fafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e4aa00640 1 --2- 172.21.15.94:0/2009444217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e4c0c64f0 0x7f2e4c0c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e4b400640 1 -- 172.21.15.94:0/2009444217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e4c0c64f0 msgr2=0x7f2e4c0c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e4b400640 1 --2- 172.21.15.94:0/2009444217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e4c0c64f0 0x7f2e4c0c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e4b400640 1 -- 172.21.15.94:0/2009444217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e4c0ce280 msgr2=0x7f2e4c1fafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e4b400640 1 --2- 172.21.15.94:0/2009444217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e4c0ce280 0x7f2e4c1fafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e4b400640 1 -- 172.21.15.94:0/2009444217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c1f8c40 con 0x7f2e4c06e7c0 2024-09-10T23:41:14.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.782+0000 7f2e4be00640 1 --2- 172.21.15.94:0/2009444217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e4c0ce280 0x7f2e4c1fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:41:14.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.786+0000 7f2e4b400640 1 --2- 172.21.15.94:0/2009444217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e4c06e7c0 0x7f2e4c0b5070 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00c3c0 tx=0x7f2e3c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:14.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.786+0000 7f2e43400640 1 -- 172.21.15.94:0/2009444217 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c017070 con 0x7f2e4c06e7c0 2024-09-10T23:41:14.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.786+0000 7f2e43400640 1 -- 172.21.15.94:0/2009444217 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e3c0053c0 con 0x7f2e4c06e7c0 2024-09-10T23:41:14.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.786+0000 7f2e52000640 1 -- 172.21.15.94:0/2009444217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4c0c8020 con 0x7f2e4c06e7c0 2024-09-10T23:41:14.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.786+0000 7f2e43400640 1 -- 172.21.15.94:0/2009444217 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c003860 con 0x7f2e4c06e7c0 2024-09-10T23:41:14.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.786+0000 7f2e52000640 1 -- 172.21.15.94:0/2009444217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4c0c85e0 con 0x7f2e4c06e7c0 2024-09-10T23:41:14.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.786+0000 7f2e52000640 1 -- 172.21.15.94:0/2009444217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e4c1f8e10 con 0x7f2e4c06e7c0 2024-09-10T23:41:14.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.786+0000 7f2e43400640 1 -- 172.21.15.94:0/2009444217 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f2e3c003a00 con 0x7f2e4c06e7c0 2024-09-10T23:41:14.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.790+0000 7f2e43400640 1 --2- 172.21.15.94:0/2009444217 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2e2c0731e0 0x7f2e2c0756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:14.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.790+0000 7f2e43400640 1 -- 172.21.15.94:0/2009444217 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7f2e3c091420 con 0x7f2e4c06e7c0 2024-09-10T23:41:14.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.790+0000 7f2e4aa00640 1 --2- 172.21.15.94:0/2009444217 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2e2c0731e0 0x7f2e2c0756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:14.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.790+0000 7f2e4aa00640 1 --2- 172.21.15.94:0/2009444217 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2e2c0731e0 0x7f2e2c0756a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0ca070 tx=0x7f2e380091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:14.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.794+0000 7f2e43400640 1 -- 172.21.15.94:0/2009444217 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2e3c05dc80 con 0x7f2e4c06e7c0 2024-09-10T23:41:14.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.958+0000 7f2e52000640 1 -- 172.21.15.94:0/2009444217 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f2e4c0d0850 con 0x7f2e2c0731e0 2024-09-10T23:41:14.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.958+0000 7f2e43400640 1 -- 172.21.15.94:0/2009444217 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2e4c0d0850 con 0x7f2e2c0731e0 2024-09-10T23:41:14.959 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 2 to deep-scrub 2024-09-10T23:41:14.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.958+0000 7f2e52000640 1 -- 172.21.15.94:0/2009444217 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2e2c0731e0 msgr2=0x7f2e2c0756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.958+0000 7f2e52000640 1 --2- 172.21.15.94:0/2009444217 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2e2c0731e0 0x7f2e2c0756a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0ca070 tx=0x7f2e380091e0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.958+0000 7f2e52000640 1 -- 172.21.15.94:0/2009444217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e4c06e7c0 msgr2=0x7f2e4c0b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:14.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.958+0000 7f2e52000640 1 --2- 172.21.15.94:0/2009444217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e4c06e7c0 0x7f2e4c0b5070 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00c3c0 tx=0x7f2e3c00c880 comp rx=0 tx=0).stop 2024-09-10T23:41:14.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.962+0000 7f2e52000640 1 -- 172.21.15.94:0/2009444217 shutdown_connections 2024-09-10T23:41:14.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.962+0000 7f2e52000640 1 --2- 172.21.15.94:0/2009444217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e4c0ce280 0x7f2e4c1fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.962+0000 7f2e52000640 1 --2- 172.21.15.94:0/2009444217 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2e2c0731e0 0x7f2e2c0756a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.962+0000 7f2e52000640 1 --2- 172.21.15.94:0/2009444217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e4c0c64f0 0x7f2e4c0c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.962+0000 7f2e52000640 1 --2- 172.21.15.94:0/2009444217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e4c06e7c0 0x7f2e4c0b5070 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:14.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.962+0000 7f2e52000640 1 -- 172.21.15.94:0/2009444217 >> 172.21.15.94:0/2009444217 conn(0x7f2e4c0cfaf0 msgr2=0x7f2e4c157610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:14.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.962+0000 7f2e52000640 1 -- 172.21.15.94:0/2009444217 shutdown_connections 2024-09-10T23:41:14.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:14.962+0000 7f2e52000640 1 -- 172.21.15.94:0/2009444217 wait complete. 2024-09-10T23:41:14.979 INFO:tasks.ceph:Scrubbing osd.3 2024-09-10T23:41:14.979 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-10T23:41:15.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.126+0000 7f85e1400640 1 -- 172.21.15.94:0/3076540465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85dc15af30 msgr2=0x7f85dc06f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.126+0000 7f85e1400640 1 --2- 172.21.15.94:0/3076540465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85dc15af30 0x7f85dc06f3b0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f85dc075310 tx=0x7f85c401c7d0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.126+0000 7f85e1400640 1 -- 172.21.15.94:0/3076540465 shutdown_connections 2024-09-10T23:41:15.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.126+0000 7f85e1400640 1 --2- 172.21.15.94:0/3076540465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85dc15af30 0x7f85dc06f3b0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.126+0000 7f85e1400640 1 --2- 172.21.15.94:0/3076540465 >> v2:172.21.15.181:3300/0 conn(0x7f85dc073630 0x7f85dc073a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.126+0000 7f85e1400640 1 -- 172.21.15.94:0/3076540465 >> 172.21.15.94:0/3076540465 conn(0x7f85dc0cd6b0 msgr2=0x7f85dc0cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:15.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.126+0000 7f85e1400640 1 -- 172.21.15.94:0/3076540465 shutdown_connections 2024-09-10T23:41:15.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.126+0000 7f85e1400640 1 -- 172.21.15.94:0/3076540465 wait complete. 2024-09-10T23:41:15.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.126+0000 7f85e1400640 1 Processor -- start 2024-09-10T23:41:15.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.126+0000 7f85e1400640 1 -- start start 2024-09-10T23:41:15.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.126+0000 7f85e1400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85dc073630 0x7f85dc152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.126+0000 7f85e1400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85dc15af30 0x7f85dc14e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.126+0000 7f85e1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85dc15e580 0x7f85dc0c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.126+0000 7f85e1400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc1684c0 con 0x7f85dc15e580 2024-09-10T23:41:15.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.126+0000 7f85e1400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc168340 con 0x7f85dc15af30 2024-09-10T23:41:15.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.126+0000 7f85e1400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f85dc168640 con 0x7f85dc073630 2024-09-10T23:41:15.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85e0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85dc15e580 0x7f85dc0c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:15.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85e0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85dc15e580 0x7f85dc0c9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56892/0 (socket says 172.21.15.94:56892) 2024-09-10T23:41:15.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85e0a00640 1 -- 172.21.15.94:0/4275046816 learned_addr learned my addr 172.21.15.94:0/4275046816 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:15.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85dbe00640 1 --2- 172.21.15.94:0/4275046816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85dc073630 0x7f85dc152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:15.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85d3400640 1 --2- 172.21.15.94:0/4275046816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85dc15af30 0x7f85dc14e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:15.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85e0a00640 1 -- 172.21.15.94:0/4275046816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85dc073630 msgr2=0x7f85dc152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85e0a00640 1 --2- 172.21.15.94:0/4275046816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85dc073630 0x7f85dc152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85e0a00640 1 -- 172.21.15.94:0/4275046816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85dc15af30 msgr2=0x7f85dc14e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85e0a00640 1 --2- 172.21.15.94:0/4275046816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85dc15af30 0x7f85dc14e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85e0a00640 1 -- 172.21.15.94:0/4275046816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85dc14e760 con 0x7f85dc15e580 2024-09-10T23:41:15.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85d3400640 1 --2- 172.21.15.94:0/4275046816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85dc15af30 0x7f85dc14e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:41:15.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85dbe00640 1 --2- 172.21.15.94:0/4275046816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85dc073630 0x7f85dc152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:41:15.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85e0a00640 1 --2- 172.21.15.94:0/4275046816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85dc15e580 0x7f85dc0c9ac0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f85d400c3c0 tx=0x7f85d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:15.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85d8c00640 1 -- 172.21.15.94:0/4275046816 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4011790 con 0x7f85dc15e580 2024-09-10T23:41:15.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85d8c00640 1 -- 172.21.15.94:0/4275046816 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85d4017070 con 0x7f85dc15e580 2024-09-10T23:41:15.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85e1400640 1 -- 172.21.15.94:0/4275046816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85dc1fb5b0 con 0x7f85dc15e580 2024-09-10T23:41:15.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85d8c00640 1 -- 172.21.15.94:0/4275046816 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4011ce0 con 0x7f85dc15e580 2024-09-10T23:41:15.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85e1400640 1 -- 172.21.15.94:0/4275046816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85dc1fbb70 con 0x7f85dc15e580 2024-09-10T23:41:15.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85e1400640 1 -- 172.21.15.94:0/4275046816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f85dc1531e0 con 0x7f85dc15e580 2024-09-10T23:41:15.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.130+0000 7f85d8c00640 1 -- 172.21.15.94:0/4275046816 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f85d401c460 con 0x7f85dc15e580 2024-09-10T23:41:15.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.134+0000 7f85d8c00640 1 --2- 172.21.15.94:0/4275046816 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f85b80731b0 0x7f85b8075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.134+0000 7f85d8c00640 1 -- 172.21.15.94:0/4275046816 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7f85d4092660 con 0x7f85dc15e580 2024-09-10T23:41:15.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.134+0000 7f85dbe00640 1 --2- 172.21.15.94:0/4275046816 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f85b80731b0 0x7f85b8075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:15.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.134+0000 7f85d8c00640 1 --2- 172.21.15.94:0/4275046816 >> [v2:172.21.15.94:6824/3187639367,v1:172.21.15.94:6825/3187639367] conn(0x7f85b8079070 0x7f85b807b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.134+0000 7f85d8c00640 1 -- 172.21.15.94:0/4275046816 --> [v2:172.21.15.94:6824/3187639367,v1:172.21.15.94:6825/3187639367] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85d4024e50 con 0x7f85b8079070 2024-09-10T23:41:15.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.134+0000 7f85d8c00640 1 -- 172.21.15.94:0/4275046816 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) ==== 24+0+0 (secure 0 0 0) 0x7f85d4094f50 con 0x7f85dc15e580 2024-09-10T23:41:15.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.134+0000 7f85d3400640 1 --2- 172.21.15.94:0/4275046816 >> [v2:172.21.15.94:6824/3187639367,v1:172.21.15.94:6825/3187639367] conn(0x7f85b8079070 0x7f85b807b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:15.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.134+0000 7f85dbe00640 1 --2- 172.21.15.94:0/4275046816 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f85b80731b0 0x7f85b8075670 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009f40 tx=0x7f85cc009500 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:15.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.134+0000 7f85d3400640 1 --2- 172.21.15.94:0/4275046816 >> [v2:172.21.15.94:6824/3187639367,v1:172.21.15.94:6825/3187639367] conn(0x7f85b8079070 0x7f85b807b490 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:15.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.134+0000 7f85d8c00640 1 -- 172.21.15.94:0/4275046816 <== osd.3 v2:172.21.15.94:6824/3187639367 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f85d4024e50 con 0x7f85b8079070 2024-09-10T23:41:15.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.146+0000 7f85e1400640 1 -- 172.21.15.94:0/4275046816 --> [v2:172.21.15.94:6824/3187639367,v1:172.21.15.94:6825/3187639367] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f85dc06f3b0 con 0x7f85b8079070 2024-09-10T23:41:15.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.146+0000 7f85d8c00640 1 -- 172.21.15.94:0/4275046816 <== osd.3 v2:172.21.15.94:6824/3187639367 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f85dc06f3b0 con 0x7f85b8079070 2024-09-10T23:41:15.148 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:41:15.148 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T23:41:15.148 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:41:15.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.146+0000 7f85e1400640 1 -- 172.21.15.94:0/4275046816 >> [v2:172.21.15.94:6824/3187639367,v1:172.21.15.94:6825/3187639367] conn(0x7f85b8079070 msgr2=0x7f85b807b490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.146+0000 7f85e1400640 1 --2- 172.21.15.94:0/4275046816 >> [v2:172.21.15.94:6824/3187639367,v1:172.21.15.94:6825/3187639367] conn(0x7f85b8079070 0x7f85b807b490 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.146+0000 7f85e1400640 1 -- 172.21.15.94:0/4275046816 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f85b80731b0 msgr2=0x7f85b8075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.146+0000 7f85e1400640 1 --2- 172.21.15.94:0/4275046816 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f85b80731b0 0x7f85b8075670 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009f40 tx=0x7f85cc009500 comp rx=0 tx=0).stop 2024-09-10T23:41:15.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.146+0000 7f85e1400640 1 -- 172.21.15.94:0/4275046816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85dc15e580 msgr2=0x7f85dc0c9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.146+0000 7f85e1400640 1 --2- 172.21.15.94:0/4275046816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85dc15e580 0x7f85dc0c9ac0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f85d400c3c0 tx=0x7f85d400c880 comp rx=0 tx=0).stop 2024-09-10T23:41:15.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.146+0000 7f85e0a00640 1 -- 172.21.15.94:0/4275046816 reap_dead start 2024-09-10T23:41:15.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.146+0000 7f85e1400640 1 -- 172.21.15.94:0/4275046816 shutdown_connections 2024-09-10T23:41:15.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.146+0000 7f85e1400640 1 -- 172.21.15.94:0/4275046816 >> 172.21.15.94:0/4275046816 conn(0x7f85dc0cd6b0 msgr2=0x7f85dc15ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:15.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.146+0000 7f85e1400640 1 -- 172.21.15.94:0/4275046816 shutdown_connections 2024-09-10T23:41:15.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.146+0000 7f85e1400640 1 -- 172.21.15.94:0/4275046816 wait complete. 2024-09-10T23:41:15.166 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-10T23:41:15.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.310+0000 7f9aab000640 1 -- 172.21.15.94:0/3792765193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a88003a40 msgr2=0x7f9a88003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.310+0000 7f9aab000640 1 --2- 172.21.15.94:0/3792765193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a88003a40 0x7f9a88003e60 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f9a94006fd0 tx=0x7f9a9401ca90 comp rx=0 tx=0).stop 2024-09-10T23:41:15.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.310+0000 7f9aa9000640 1 -- 172.21.15.94:0/3792765193 reap_dead start 2024-09-10T23:41:15.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.310+0000 7f9aab000640 1 -- 172.21.15.94:0/3792765193 shutdown_connections 2024-09-10T23:41:15.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.310+0000 7f9aab000640 1 -- 172.21.15.94:0/3792765193 >> 172.21.15.94:0/3792765193 conn(0x7f9aa40cfaf0 msgr2=0x7f9aa40cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:15.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.310+0000 7f9aab000640 1 -- 172.21.15.94:0/3792765193 shutdown_connections 2024-09-10T23:41:15.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.310+0000 7f9aab000640 1 -- 172.21.15.94:0/3792765193 wait complete. 2024-09-10T23:41:15.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.310+0000 7f9aab000640 1 Processor -- start 2024-09-10T23:41:15.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.310+0000 7f9aab000640 1 -- start start 2024-09-10T23:41:15.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.310+0000 7f9aab000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a88003a40 0x7f9aa41f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.310+0000 7f9aab000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9aa40b3430 0x7f9aa41554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.310+0000 7f9aab000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9aa4073850 0x7f9aa4073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.310+0000 7f9aab000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa415b320 con 0x7f9aa40b3430 2024-09-10T23:41:15.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.310+0000 7f9aab000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa415b1a0 con 0x7f9aa4073850 2024-09-10T23:41:15.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.310+0000 7f9aab000640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa415b4a0 con 0x7f9a88003a40 2024-09-10T23:41:15.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.314+0000 7f9aa3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9aa40b3430 0x7f9aa41554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.314+0000 7f9aa3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9aa40b3430 0x7f9aa41554a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56918/0 (socket says 172.21.15.94:56918) 2024-09-10T23:41:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.314+0000 7f9aa3400640 1 -- 172.21.15.94:0/2958448911 learned_addr learned my addr 172.21.15.94:0/2958448911 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.314+0000 7f9aa3400640 1 -- 172.21.15.94:0/2958448911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a88003a40 msgr2=0x7f9aa41f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.314+0000 7f9aa3400640 1 --2- 172.21.15.94:0/2958448911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a88003a40 0x7f9aa41f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.314+0000 7f9aa3400640 1 -- 172.21.15.94:0/2958448911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9aa4073850 msgr2=0x7f9aa4073cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.314+0000 7f9aa3400640 1 --2- 172.21.15.94:0/2958448911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9aa4073850 0x7f9aa4073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.314+0000 7f9aa3400640 1 -- 172.21.15.94:0/2958448911 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa40b3090 con 0x7f9aa40b3430 2024-09-10T23:41:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.314+0000 7f9aa3400640 1 --2- 172.21.15.94:0/2958448911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9aa40b3430 0x7f9aa41554a0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f9a90006660 tx=0x7f9a900039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.314+0000 7f9a9be00640 1 -- 172.21.15.94:0/2958448911 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90015070 con 0x7f9aa40b3430 2024-09-10T23:41:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.314+0000 7f9a9be00640 1 -- 172.21.15.94:0/2958448911 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a900119e0 con 0x7f9aa40b3430 2024-09-10T23:41:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.314+0000 7f9a9be00640 1 -- 172.21.15.94:0/2958448911 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90011e30 con 0x7f9aa40b3430 2024-09-10T23:41:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.314+0000 7f9aab000640 1 -- 172.21.15.94:0/2958448911 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa40b2db0 con 0x7f9aa40b3430 2024-09-10T23:41:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.314+0000 7f9aab000640 1 -- 172.21.15.94:0/2958448911 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa41f7810 con 0x7f9aa40b3430 2024-09-10T23:41:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.314+0000 7f9aab000640 1 -- 172.21.15.94:0/2958448911 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa406e7c0 con 0x7f9aa40b3430 2024-09-10T23:41:15.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.314+0000 7f9a9be00640 1 -- 172.21.15.94:0/2958448911 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f9a900103c0 con 0x7f9aa40b3430 2024-09-10T23:41:15.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.318+0000 7f9a9be00640 1 --2- 172.21.15.94:0/2958448911 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9a840731e0 0x7f9a840756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.318+0000 7f9a9be00640 1 -- 172.21.15.94:0/2958448911 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7f9a90092ec0 con 0x7f9aa40b3430 2024-09-10T23:41:15.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.318+0000 7f9aa3e00640 1 --2- 172.21.15.94:0/2958448911 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9a840731e0 0x7f9a840756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:15.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.318+0000 7f9aa3e00640 1 --2- 172.21.15.94:0/2958448911 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9a840731e0 0x7f9a840756a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9a940aa000 tx=0x7f9a88005840 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:15.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.322+0000 7f9a9be00640 1 -- 172.21.15.94:0/2958448911 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9a9005fd90 con 0x7f9aa40b3430 2024-09-10T23:41:15.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.486+0000 7f9aab000640 1 -- 172.21.15.94:0/2958448911 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f9aa40d0850 con 0x7f9a840731e0 2024-09-10T23:41:15.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.486+0000 7f9a9be00640 1 -- 172.21.15.94:0/2958448911 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9aa40d0850 con 0x7f9a840731e0 2024-09-10T23:41:15.488 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 3 to deep-scrub 2024-09-10T23:41:15.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.490+0000 7f9aab000640 1 -- 172.21.15.94:0/2958448911 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9a840731e0 msgr2=0x7f9a840756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.490+0000 7f9aab000640 1 --2- 172.21.15.94:0/2958448911 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9a840731e0 0x7f9a840756a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9a940aa000 tx=0x7f9a88005840 comp rx=0 tx=0).stop 2024-09-10T23:41:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.490+0000 7f9aab000640 1 -- 172.21.15.94:0/2958448911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9aa40b3430 msgr2=0x7f9aa41554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.490+0000 7f9aab000640 1 --2- 172.21.15.94:0/2958448911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9aa40b3430 0x7f9aa41554a0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f9a90006660 tx=0x7f9a900039d0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.490+0000 7f9aab000640 1 -- 172.21.15.94:0/2958448911 shutdown_connections 2024-09-10T23:41:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.490+0000 7f9aab000640 1 --2- 172.21.15.94:0/2958448911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9aa4073850 0x7f9aa4073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.490+0000 7f9aab000640 1 --2- 172.21.15.94:0/2958448911 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9a840731e0 0x7f9a840756a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.490+0000 7f9aab000640 1 --2- 172.21.15.94:0/2958448911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9aa40b3430 0x7f9aa41554a0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.490+0000 7f9aab000640 1 --2- 172.21.15.94:0/2958448911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a88003a40 0x7f9aa41f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.490+0000 7f9aab000640 1 -- 172.21.15.94:0/2958448911 >> 172.21.15.94:0/2958448911 conn(0x7f9aa40cfaf0 msgr2=0x7f9aa40aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.490+0000 7f9aab000640 1 -- 172.21.15.94:0/2958448911 shutdown_connections 2024-09-10T23:41:15.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.490+0000 7f9aab000640 1 -- 172.21.15.94:0/2958448911 wait complete. 2024-09-10T23:41:15.510 INFO:tasks.ceph:Scrubbing osd.4 2024-09-10T23:41:15.510 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-10T23:41:15.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.658+0000 7f27d9600640 1 -- 172.21.15.94:0/4105189264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d40b0b10 msgr2=0x7f27d40cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.658+0000 7f27d9600640 1 --2- 172.21.15.94:0/4105189264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d40b0b10 0x7f27d40cff80 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009900 tx=0x7f27bc01f0d0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.658+0000 7f27d9600640 1 -- 172.21.15.94:0/4105189264 shutdown_connections 2024-09-10T23:41:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.658+0000 7f27d9600640 1 --2- 172.21.15.94:0/4105189264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d40b0b10 0x7f27d40cff80 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.658+0000 7f27d9600640 1 --2- 172.21.15.94:0/4105189264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27d415bcb0 0x7f27d40d2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.658+0000 7f27d9600640 1 -- 172.21.15.94:0/4105189264 >> 172.21.15.94:0/4105189264 conn(0x7f27d40d1a40 msgr2=0x7f27d40d1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.658+0000 7f27d9600640 1 -- 172.21.15.94:0/4105189264 shutdown_connections 2024-09-10T23:41:15.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.658+0000 7f27d9600640 1 -- 172.21.15.94:0/4105189264 wait complete. 2024-09-10T23:41:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d9600640 1 Processor -- start 2024-09-10T23:41:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d9600640 1 -- start start 2024-09-10T23:41:15.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d9600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27d40b0b10 0x7f27d40c5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d9600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d40d04c0 0x7f27d40c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d9600640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27d40c8590 0x7f27d4150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d9600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f27d40ce9f0 con 0x7f27d40b0b10 2024-09-10T23:41:15.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d9600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f27d40ce870 con 0x7f27d40d04c0 2024-09-10T23:41:15.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d9600640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f27d40ceb70 con 0x7f27d40c8590 2024-09-10T23:41:15.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27d40b0b10 0x7f27d40c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:15.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27d40b0b10 0x7f27d40c5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56934/0 (socket says 172.21.15.94:56934) 2024-09-10T23:41:15.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d3e00640 1 -- 172.21.15.94:0/921372393 learned_addr learned my addr 172.21.15.94:0/921372393 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:15.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27cbe00640 1 --2- 172.21.15.94:0/921372393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d40d04c0 0x7f27d40c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:15.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d3e00640 1 -- 172.21.15.94:0/921372393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27d40c8590 msgr2=0x7f27d4150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d8c00640 1 --2- 172.21.15.94:0/921372393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27d40c8590 0x7f27d4150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:15.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d3e00640 1 --2- 172.21.15.94:0/921372393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27d40c8590 0x7f27d4150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d3e00640 1 -- 172.21.15.94:0/921372393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d40d04c0 msgr2=0x7f27d40c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d3e00640 1 --2- 172.21.15.94:0/921372393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d40d04c0 0x7f27d40c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d3e00640 1 -- 172.21.15.94:0/921372393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d40c5b60 con 0x7f27d40b0b10 2024-09-10T23:41:15.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27cbe00640 1 --2- 172.21.15.94:0/921372393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d40d04c0 0x7f27d40c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:41:15.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d8c00640 1 --2- 172.21.15.94:0/921372393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27d40c8590 0x7f27d4150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:41:15.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d3e00640 1 --2- 172.21.15.94:0/921372393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27d40b0b10 0x7f27d40c5510 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f27c400e970 tx=0x7f27c400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:15.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d0c00640 1 -- 172.21.15.94:0/921372393 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c400cd80 con 0x7f27d40b0b10 2024-09-10T23:41:15.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d0c00640 1 -- 172.21.15.94:0/921372393 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27c4004020 con 0x7f27d40b0b10 2024-09-10T23:41:15.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d9600640 1 -- 172.21.15.94:0/921372393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d40c7290 con 0x7f27d40b0b10 2024-09-10T23:41:15.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d0c00640 1 -- 172.21.15.94:0/921372393 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c4004470 con 0x7f27d40b0b10 2024-09-10T23:41:15.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d9600640 1 -- 172.21.15.94:0/921372393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d40c7850 con 0x7f27d40b0b10 2024-09-10T23:41:15.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.662+0000 7f27d9600640 1 -- 172.21.15.94:0/921372393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f27d40cff80 con 0x7f27d40b0b10 2024-09-10T23:41:15.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.666+0000 7f27d0c00640 1 -- 172.21.15.94:0/921372393 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f27c4003600 con 0x7f27d40b0b10 2024-09-10T23:41:15.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.666+0000 7f27d0c00640 1 --2- 172.21.15.94:0/921372393 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f27b00731e0 0x7f27b00756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.666+0000 7f27d0c00640 1 -- 172.21.15.94:0/921372393 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7f27c4092860 con 0x7f27d40b0b10 2024-09-10T23:41:15.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.666+0000 7f27cbe00640 1 --2- 172.21.15.94:0/921372393 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f27b00731e0 0x7f27b00756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:15.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.666+0000 7f27d0c00640 1 --2- 172.21.15.94:0/921372393 >> [v2:172.21.15.181:6810/955716519,v1:172.21.15.181:6812/955716519] conn(0x7f27b00790a0 0x7f27b007b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.666+0000 7f27d0c00640 1 -- 172.21.15.94:0/921372393 --> [v2:172.21.15.181:6810/955716519,v1:172.21.15.181:6812/955716519] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f27b007bb90 con 0x7f27b00790a0 2024-09-10T23:41:15.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.666+0000 7f27d0c00640 1 -- 172.21.15.94:0/921372393 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) ==== 24+0+0 (secure 0 0 0) 0x7f27c40950c0 con 0x7f27d40b0b10 2024-09-10T23:41:15.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.666+0000 7f27cbe00640 1 --2- 172.21.15.94:0/921372393 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f27b00731e0 0x7f27b00756a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0b2000 tx=0x7f27bc01d910 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:15.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.666+0000 7f27d8c00640 1 --2- 172.21.15.94:0/921372393 >> [v2:172.21.15.181:6810/955716519,v1:172.21.15.181:6812/955716519] conn(0x7f27b00790a0 0x7f27b007b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:15.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.666+0000 7f27d8c00640 1 --2- 172.21.15.94:0/921372393 >> [v2:172.21.15.181:6810/955716519,v1:172.21.15.181:6812/955716519] conn(0x7f27b00790a0 0x7f27b007b4c0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:15.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.670+0000 7f27d0c00640 1 -- 172.21.15.94:0/921372393 <== osd.4 v2:172.21.15.181:6810/955716519 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27b007bb90 con 0x7f27b00790a0 2024-09-10T23:41:15.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.682+0000 7f27d9600640 1 -- 172.21.15.94:0/921372393 --> [v2:172.21.15.181:6810/955716519,v1:172.21.15.181:6812/955716519] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f27d40d0200 con 0x7f27b00790a0 2024-09-10T23:41:15.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.682+0000 7f27d0c00640 1 -- 172.21.15.94:0/921372393 <== osd.4 v2:172.21.15.181:6810/955716519 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f27d40d0200 con 0x7f27b00790a0 2024-09-10T23:41:15.682 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:41:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T23:41:15.683 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:41:15.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.682+0000 7f27d9600640 1 -- 172.21.15.94:0/921372393 >> [v2:172.21.15.181:6810/955716519,v1:172.21.15.181:6812/955716519] conn(0x7f27b00790a0 msgr2=0x7f27b007b4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.682+0000 7f27d9600640 1 --2- 172.21.15.94:0/921372393 >> [v2:172.21.15.181:6810/955716519,v1:172.21.15.181:6812/955716519] conn(0x7f27b00790a0 0x7f27b007b4c0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.682+0000 7f27d9600640 1 -- 172.21.15.94:0/921372393 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f27b00731e0 msgr2=0x7f27b00756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.682+0000 7f27d9600640 1 --2- 172.21.15.94:0/921372393 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f27b00731e0 0x7f27b00756a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0b2000 tx=0x7f27bc01d910 comp rx=0 tx=0).stop 2024-09-10T23:41:15.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.682+0000 7f27d9600640 1 -- 172.21.15.94:0/921372393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27d40b0b10 msgr2=0x7f27d40c5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.682+0000 7f27d9600640 1 --2- 172.21.15.94:0/921372393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27d40b0b10 0x7f27d40c5510 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f27c400e970 tx=0x7f27c400ee30 comp rx=0 tx=0).stop 2024-09-10T23:41:15.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.682+0000 7f27d8c00640 1 -- 172.21.15.94:0/921372393 reap_dead start 2024-09-10T23:41:15.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.682+0000 7f27d9600640 1 -- 172.21.15.94:0/921372393 shutdown_connections 2024-09-10T23:41:15.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.682+0000 7f27d9600640 1 -- 172.21.15.94:0/921372393 >> 172.21.15.94:0/921372393 conn(0x7f27d40d1a40 msgr2=0x7f27d40b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:15.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.682+0000 7f27d9600640 1 -- 172.21.15.94:0/921372393 shutdown_connections 2024-09-10T23:41:15.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.682+0000 7f27d9600640 1 -- 172.21.15.94:0/921372393 wait complete. 2024-09-10T23:41:15.701 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-10T23:41:15.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.842+0000 7fbdbe200640 1 -- 172.21.15.94:0/2411772646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb80c64f0 msgr2=0x7fbdb80cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.842+0000 7fbdbe200640 1 --2- 172.21.15.94:0/2411772646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb80c64f0 0x7fbdb80cf280 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fbda000a950 tx=0x7fbda001db70 comp rx=0 tx=0).stop 2024-09-10T23:41:15.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.842+0000 7fbdbe200640 1 -- 172.21.15.94:0/2411772646 shutdown_connections 2024-09-10T23:41:15.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.842+0000 7fbdbe200640 1 --2- 172.21.15.94:0/2411772646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb80c64f0 0x7fbdb80cf280 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.842+0000 7fbdbe200640 1 -- 172.21.15.94:0/2411772646 >> 172.21.15.94:0/2411772646 conn(0x7fbdb80cfaf0 msgr2=0x7fbdb80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:15.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.842+0000 7fbdbe200640 1 -- 172.21.15.94:0/2411772646 shutdown_connections 2024-09-10T23:41:15.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.842+0000 7fbdbe200640 1 -- 172.21.15.94:0/2411772646 wait complete. 2024-09-10T23:41:15.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.842+0000 7fbdbe200640 1 Processor -- start 2024-09-10T23:41:15.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdbe200640 1 -- start start 2024-09-10T23:41:15.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdbe200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb806e7c0 0x7fbdb8151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdbe200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdb80c64f0 0x7fbdb81fa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdbe200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdb80ce280 0x7fbdb81414e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdbe200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb8168850 con 0x7fbdb806e7c0 2024-09-10T23:41:15.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdbe200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb81686d0 con 0x7fbdb80c64f0 2024-09-10T23:41:15.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdbe200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb81689d0 con 0x7fbdb80ce280 2024-09-10T23:41:15.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdb7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb806e7c0 0x7fbdb8151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:15.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdb7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb806e7c0 0x7fbdb8151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56944/0 (socket says 172.21.15.94:56944) 2024-09-10T23:41:15.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdb7400640 1 -- 172.21.15.94:0/2180163936 learned_addr learned my addr 172.21.15.94:0/2180163936 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:15.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdaea00640 1 --2- 172.21.15.94:0/2180163936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdb80c64f0 0x7fbdb81fa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:15.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdb7e00640 1 --2- 172.21.15.94:0/2180163936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdb80ce280 0x7fbdb81414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:15.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdb7400640 1 -- 172.21.15.94:0/2180163936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdb80ce280 msgr2=0x7fbdb81414e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdb7400640 1 --2- 172.21.15.94:0/2180163936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdb80ce280 0x7fbdb81414e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdb7400640 1 -- 172.21.15.94:0/2180163936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdb80c64f0 msgr2=0x7fbdb81fa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:15.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdb7400640 1 --2- 172.21.15.94:0/2180163936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdb80c64f0 0x7fbdb81fa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:15.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdb7400640 1 -- 172.21.15.94:0/2180163936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb81fa5c0 con 0x7fbdb806e7c0 2024-09-10T23:41:15.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdb7e00640 1 --2- 172.21.15.94:0/2180163936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdb80ce280 0x7fbdb81414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:41:15.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdb7400640 1 --2- 172.21.15.94:0/2180163936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb806e7c0 0x7fbdb8151fd0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fbda800c750 tx=0x7fbda800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:15.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdafe00640 1 -- 172.21.15.94:0/2180163936 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8017070 con 0x7fbdb806e7c0 2024-09-10T23:41:15.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdafe00640 1 -- 172.21.15.94:0/2180163936 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbda8005070 con 0x7fbdb806e7c0 2024-09-10T23:41:15.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdbe200640 1 -- 172.21.15.94:0/2180163936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb8141a20 con 0x7fbdb806e7c0 2024-09-10T23:41:15.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdafe00640 1 -- 172.21.15.94:0/2180163936 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda800ce60 con 0x7fbdb806e7c0 2024-09-10T23:41:15.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdbe200640 1 -- 172.21.15.94:0/2180163936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb8151b70 con 0x7fbdb806e7c0 2024-09-10T23:41:15.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.846+0000 7fbdbe200640 1 -- 172.21.15.94:0/2180163936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb80cd420 con 0x7fbdb806e7c0 2024-09-10T23:41:15.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.850+0000 7fbdafe00640 1 -- 172.21.15.94:0/2180163936 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7fbda80054c0 con 0x7fbdb806e7c0 2024-09-10T23:41:15.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.850+0000 7fbdafe00640 1 --2- 172.21.15.94:0/2180163936 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbd940731b0 0x7fbd94075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:15.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.850+0000 7fbdafe00640 1 -- 172.21.15.94:0/2180163936 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7fbda8091780 con 0x7fbdb806e7c0 2024-09-10T23:41:15.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.850+0000 7fbdaea00640 1 --2- 172.21.15.94:0/2180163936 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbd940731b0 0x7fbd94075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:15.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.850+0000 7fbdaea00640 1 --2- 172.21.15.94:0/2180163936 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbd940731b0 0x7fbd94075670 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbda00b0000 tx=0x7fbda001c3b0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:15.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:15.854+0000 7fbdafe00640 1 -- 172.21.15.94:0/2180163936 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbda80057e0 con 0x7fbdb806e7c0 2024-09-10T23:41:16.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.022+0000 7fbdbe200640 1 -- 172.21.15.94:0/2180163936 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fbdb8151640 con 0x7fbd940731b0 2024-09-10T23:41:16.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.026+0000 7fbdafe00640 1 -- 172.21.15.94:0/2180163936 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbdb8151640 con 0x7fbd940731b0 2024-09-10T23:41:16.025 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 4 to deep-scrub 2024-09-10T23:41:16.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.026+0000 7fbdbe200640 1 -- 172.21.15.94:0/2180163936 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbd940731b0 msgr2=0x7fbd94075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.026+0000 7fbdbe200640 1 --2- 172.21.15.94:0/2180163936 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbd940731b0 0x7fbd94075670 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbda00b0000 tx=0x7fbda001c3b0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.026+0000 7fbdbe200640 1 -- 172.21.15.94:0/2180163936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb806e7c0 msgr2=0x7fbdb8151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.026+0000 7fbdbe200640 1 --2- 172.21.15.94:0/2180163936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb806e7c0 0x7fbdb8151fd0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fbda800c750 tx=0x7fbda800cc10 comp rx=0 tx=0).stop 2024-09-10T23:41:16.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.026+0000 7fbdbe200640 1 -- 172.21.15.94:0/2180163936 shutdown_connections 2024-09-10T23:41:16.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.026+0000 7fbdbe200640 1 --2- 172.21.15.94:0/2180163936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdb80c64f0 0x7fbdb81fa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.026+0000 7fbdbe200640 1 --2- 172.21.15.94:0/2180163936 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbd940731b0 0x7fbd94075670 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.026+0000 7fbdbe200640 1 --2- 172.21.15.94:0/2180163936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdb80ce280 0x7fbdb81414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.026+0000 7fbdbe200640 1 --2- 172.21.15.94:0/2180163936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdb806e7c0 0x7fbdb8151fd0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.026+0000 7fbdbe200640 1 -- 172.21.15.94:0/2180163936 >> 172.21.15.94:0/2180163936 conn(0x7fbdb80cfaf0 msgr2=0x7fbdb8152740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:16.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.026+0000 7fbdbe200640 1 -- 172.21.15.94:0/2180163936 shutdown_connections 2024-09-10T23:41:16.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.026+0000 7fbdbe200640 1 -- 172.21.15.94:0/2180163936 wait complete. 2024-09-10T23:41:16.047 INFO:tasks.ceph:Scrubbing osd.5 2024-09-10T23:41:16.047 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-10T23:41:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1bdc00640 1 -- 172.21.15.94:0/1832411164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1b8152c10 msgr2=0x7fa1b80b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1bdc00640 1 --2- 172.21.15.94:0/1832411164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1b8152c10 0x7fa1b80b29d0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fa1b815c040 tx=0x7fa1a801c7d0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1bdc00640 1 -- 172.21.15.94:0/1832411164 shutdown_connections 2024-09-10T23:41:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1bdc00640 1 --2- 172.21.15.94:0/1832411164 >> v2:172.21.15.181:3300/0 conn(0x7fa1b815cbc0 0x7fa1b80cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1bdc00640 1 --2- 172.21.15.94:0/1832411164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1b8152c10 0x7fa1b80b29d0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1bdc00640 1 -- 172.21.15.94:0/1832411164 >> 172.21.15.94:0/1832411164 conn(0x7fa1b806cdf0 msgr2=0x7fa1b806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1bdc00640 1 -- 172.21.15.94:0/1832411164 shutdown_connections 2024-09-10T23:41:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1bdc00640 1 -- 172.21.15.94:0/1832411164 wait complete. 2024-09-10T23:41:16.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1bdc00640 1 Processor -- start 2024-09-10T23:41:16.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1bdc00640 1 -- start start 2024-09-10T23:41:16.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1bdc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1b80cfd50 0x7fa1b813f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1bdc00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1b8152c10 0x7fa1b814c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1bdc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b815cbc0 0x7fa1b8140250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1bdc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8169470 con 0x7fa1b80cfd50 2024-09-10T23:41:16.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1bdc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81692f0 con 0x7fa1b815cbc0 2024-09-10T23:41:16.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1bdc00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b81695f0 con 0x7fa1b8152c10 2024-09-10T23:41:16.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1b6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1b80cfd50 0x7fa1b813f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:16.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1b6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1b80cfd50 0x7fa1b813f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56962/0 (socket says 172.21.15.94:56962) 2024-09-10T23:41:16.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1b6a00640 1 -- 172.21.15.94:0/836490761 learned_addr learned my addr 172.21.15.94:0/836490761 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:16.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1ae000640 1 --2- 172.21.15.94:0/836490761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1b8152c10 0x7fa1b814c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:16.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.194+0000 7fa1b7400640 1 --2- 172.21.15.94:0/836490761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b815cbc0 0x7fa1b8140250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:16.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1b6a00640 1 -- 172.21.15.94:0/836490761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1b8152c10 msgr2=0x7fa1b814c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1b6a00640 1 --2- 172.21.15.94:0/836490761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1b8152c10 0x7fa1b814c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1b6a00640 1 -- 172.21.15.94:0/836490761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b815cbc0 msgr2=0x7fa1b8140250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1b6a00640 1 --2- 172.21.15.94:0/836490761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1b815cbc0 0x7fa1b8140250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1b6a00640 1 -- 172.21.15.94:0/836490761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b814b0d0 con 0x7fa1b80cfd50 2024-09-10T23:41:16.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1ae000640 1 --2- 172.21.15.94:0/836490761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1b8152c10 0x7fa1b814c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:41:16.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1b6a00640 1 --2- 172.21.15.94:0/836490761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1b80cfd50 0x7fa1b813f030 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8000c10 tx=0x7fa1a8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:16.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1af400640 1 -- 172.21.15.94:0/836490761 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a80bc070 con 0x7fa1b80cfd50 2024-09-10T23:41:16.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1af400640 1 -- 172.21.15.94:0/836490761 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1a8003600 con 0x7fa1b80cfd50 2024-09-10T23:41:16.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1bdc00640 1 -- 172.21.15.94:0/836490761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b814b370 con 0x7fa1b80cfd50 2024-09-10T23:41:16.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1af400640 1 -- 172.21.15.94:0/836490761 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a80b2040 con 0x7fa1b80cfd50 2024-09-10T23:41:16.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1bdc00640 1 -- 172.21.15.94:0/836490761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b814b930 con 0x7fa1b80cfd50 2024-09-10T23:41:16.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1bdc00640 1 -- 172.21.15.94:0/836490761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa1b80cde20 con 0x7fa1b80cfd50 2024-09-10T23:41:16.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1af400640 1 -- 172.21.15.94:0/836490761 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7fa1a80b5070 con 0x7fa1b80cfd50 2024-09-10T23:41:16.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1af400640 1 --2- 172.21.15.94:0/836490761 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa1940731e0 0x7fa1940756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1af400640 1 -- 172.21.15.94:0/836490761 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7fa1a812b8f0 con 0x7fa1b80cfd50 2024-09-10T23:41:16.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.198+0000 7fa1ae000640 1 --2- 172.21.15.94:0/836490761 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa1940731e0 0x7fa1940756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:16.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.202+0000 7fa1af400640 1 --2- 172.21.15.94:0/836490761 >> [v2:172.21.15.181:6826/1536008029,v1:172.21.15.181:6827/1536008029] conn(0x7fa1940790a0 0x7fa19407b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.202+0000 7fa1af400640 1 -- 172.21.15.94:0/836490761 --> [v2:172.21.15.181:6826/1536008029,v1:172.21.15.181:6827/1536008029] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa1a80c6ea0 con 0x7fa1940790a0 2024-09-10T23:41:16.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.202+0000 7fa1af400640 1 -- 172.21.15.94:0/836490761 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) ==== 24+0+0 (secure 0 0 0) 0x7fa1a812e150 con 0x7fa1b80cfd50 2024-09-10T23:41:16.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.202+0000 7fa1ae000640 1 --2- 172.21.15.94:0/836490761 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa1940731e0 0x7fa1940756a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8151dc0 tx=0x7fa1a0009500 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:16.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.202+0000 7fa1b7400640 1 --2- 172.21.15.94:0/836490761 >> [v2:172.21.15.181:6826/1536008029,v1:172.21.15.181:6827/1536008029] conn(0x7fa1940790a0 0x7fa19407b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:16.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.202+0000 7fa1b7400640 1 --2- 172.21.15.94:0/836490761 >> [v2:172.21.15.181:6826/1536008029,v1:172.21.15.181:6827/1536008029] conn(0x7fa1940790a0 0x7fa19407b4c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:16.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.202+0000 7fa1af400640 1 -- 172.21.15.94:0/836490761 <== osd.5 v2:172.21.15.181:6826/1536008029 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa1a80c6ea0 con 0x7fa1940790a0 2024-09-10T23:41:16.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.214+0000 7fa1bdc00640 1 -- 172.21.15.94:0/836490761 --> [v2:172.21.15.181:6826/1536008029,v1:172.21.15.181:6827/1536008029] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa1b814bd60 con 0x7fa1940790a0 2024-09-10T23:41:16.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.214+0000 7fa1af400640 1 -- 172.21.15.94:0/836490761 <== osd.5 v2:172.21.15.181:6826/1536008029 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa1b814bd60 con 0x7fa1940790a0 2024-09-10T23:41:16.216 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:41:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T23:41:16.217 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:41:16.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.214+0000 7fa1bdc00640 1 -- 172.21.15.94:0/836490761 >> [v2:172.21.15.181:6826/1536008029,v1:172.21.15.181:6827/1536008029] conn(0x7fa1940790a0 msgr2=0x7fa19407b4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.214+0000 7fa1bdc00640 1 --2- 172.21.15.94:0/836490761 >> [v2:172.21.15.181:6826/1536008029,v1:172.21.15.181:6827/1536008029] conn(0x7fa1940790a0 0x7fa19407b4c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.218+0000 7fa1bdc00640 1 -- 172.21.15.94:0/836490761 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa1940731e0 msgr2=0x7fa1940756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.218+0000 7fa1bdc00640 1 --2- 172.21.15.94:0/836490761 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa1940731e0 0x7fa1940756a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8151dc0 tx=0x7fa1a0009500 comp rx=0 tx=0).stop 2024-09-10T23:41:16.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.218+0000 7fa1bdc00640 1 -- 172.21.15.94:0/836490761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1b80cfd50 msgr2=0x7fa1b813f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.218+0000 7fa1bdc00640 1 --2- 172.21.15.94:0/836490761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1b80cfd50 0x7fa1b813f030 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8000c10 tx=0x7fa1a8004040 comp rx=0 tx=0).stop 2024-09-10T23:41:16.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.218+0000 7fa1b7400640 1 -- 172.21.15.94:0/836490761 reap_dead start 2024-09-10T23:41:16.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.218+0000 7fa1bdc00640 1 -- 172.21.15.94:0/836490761 shutdown_connections 2024-09-10T23:41:16.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.218+0000 7fa1bdc00640 1 -- 172.21.15.94:0/836490761 >> 172.21.15.94:0/836490761 conn(0x7fa1b806cdf0 msgr2=0x7fa1b814f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:16.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.218+0000 7fa1bdc00640 1 -- 172.21.15.94:0/836490761 shutdown_connections 2024-09-10T23:41:16.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.218+0000 7fa1bdc00640 1 -- 172.21.15.94:0/836490761 wait complete. 2024-09-10T23:41:16.234 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-10T23:41:16.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1392400640 1 -- 172.21.15.94:0/3445966215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1370004d50 msgr2=0x7f13700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1392400640 1 --2- 172.21.15.94:0/3445966215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1370004d50 0x7f13700051d0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f1384009b10 tx=0x7f138401d250 comp rx=0 tx=0).stop 2024-09-10T23:41:16.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f138be00640 1 -- 172.21.15.94:0/3445966215 reap_dead start 2024-09-10T23:41:16.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1392400640 1 -- 172.21.15.94:0/3445966215 shutdown_connections 2024-09-10T23:41:16.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1392400640 1 -- 172.21.15.94:0/3445966215 >> 172.21.15.94:0/3445966215 conn(0x7f138c0d1c60 msgr2=0x7f138c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:16.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1392400640 1 -- 172.21.15.94:0/3445966215 shutdown_connections 2024-09-10T23:41:16.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1392400640 1 -- 172.21.15.94:0/3445966215 wait complete. 2024-09-10T23:41:16.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1392400640 1 Processor -- start 2024-09-10T23:41:16.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1392400640 1 -- start start 2024-09-10T23:41:16.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1392400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1370004d50 0x7f138c1f8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1392400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f138c0cb560 0x7f138c0719c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1392400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f138c0b6c40 0x7f138c071f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1392400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f138c164ae0 con 0x7f138c0cb560 2024-09-10T23:41:16.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1392400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f138c164960 con 0x7f1370004d50 2024-09-10T23:41:16.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1392400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f138c164c60 con 0x7f138c0b6c40 2024-09-10T23:41:16.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1382a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f138c0cb560 0x7f138c0719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:16.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1382a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f138c0cb560 0x7f138c0719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56984/0 (socket says 172.21.15.94:56984) 2024-09-10T23:41:16.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1382a00640 1 -- 172.21.15.94:0/2294777388 learned_addr learned my addr 172.21.15.94:0/2294777388 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:16.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f138b400640 1 --2- 172.21.15.94:0/2294777388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1370004d50 0x7f138c1f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:16.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f138be00640 1 --2- 172.21.15.94:0/2294777388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f138c0b6c40 0x7f138c071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:16.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1382a00640 1 -- 172.21.15.94:0/2294777388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f138c0b6c40 msgr2=0x7f138c071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1382a00640 1 --2- 172.21.15.94:0/2294777388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f138c0b6c40 0x7f138c071f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1382a00640 1 -- 172.21.15.94:0/2294777388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1370004d50 msgr2=0x7f138c1f8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1382a00640 1 --2- 172.21.15.94:0/2294777388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1370004d50 0x7f138c1f8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.378+0000 7f1382a00640 1 -- 172.21.15.94:0/2294777388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138c0ae910 con 0x7f138c0cb560 2024-09-10T23:41:16.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.382+0000 7f1382a00640 1 --2- 172.21.15.94:0/2294777388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f138c0cb560 0x7f138c0719c0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f1374011990 tx=0x7f1374011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:16.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.382+0000 7f1383e00640 1 -- 172.21.15.94:0/2294777388 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374017070 con 0x7f138c0cb560 2024-09-10T23:41:16.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.382+0000 7f1383e00640 1 -- 172.21.15.94:0/2294777388 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f137400fd70 con 0x7f138c0cb560 2024-09-10T23:41:16.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.382+0000 7f1392400640 1 -- 172.21.15.94:0/2294777388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138c0aebf0 con 0x7f138c0cb560 2024-09-10T23:41:16.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.382+0000 7f1383e00640 1 -- 172.21.15.94:0/2294777388 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13740136b0 con 0x7f138c0cb560 2024-09-10T23:41:16.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.382+0000 7f1392400640 1 -- 172.21.15.94:0/2294777388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f138c0af1b0 con 0x7f138c0cb560 2024-09-10T23:41:16.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.382+0000 7f1392400640 1 -- 172.21.15.94:0/2294777388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f138c0cfce0 con 0x7f138c0cb560 2024-09-10T23:41:16.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.382+0000 7f1383e00640 1 -- 172.21.15.94:0/2294777388 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f1374010040 con 0x7f138c0cb560 2024-09-10T23:41:16.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.382+0000 7f1383e00640 1 --2- 172.21.15.94:0/2294777388 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f13680731b0 0x7f1368075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.382+0000 7f1383e00640 1 -- 172.21.15.94:0/2294777388 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7f1374095c20 con 0x7f138c0cb560 2024-09-10T23:41:16.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.382+0000 7f138b400640 1 --2- 172.21.15.94:0/2294777388 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f13680731b0 0x7f1368075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:16.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.386+0000 7f138b400640 1 --2- 172.21.15.94:0/2294777388 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f13680731b0 0x7f1368075670 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f138c0b73f0 tx=0x7f137c006d70 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:16.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.386+0000 7f1383e00640 1 -- 172.21.15.94:0/2294777388 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f13740624a0 con 0x7f138c0cb560 2024-09-10T23:41:16.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.554+0000 7f1392400640 1 -- 172.21.15.94:0/2294777388 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f138c0b2790 con 0x7f13680731b0 2024-09-10T23:41:16.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.554+0000 7f1383e00640 1 -- 172.21.15.94:0/2294777388 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f138c0b2790 con 0x7f13680731b0 2024-09-10T23:41:16.554 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 5 to deep-scrub 2024-09-10T23:41:16.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.554+0000 7f1392400640 1 -- 172.21.15.94:0/2294777388 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f13680731b0 msgr2=0x7f1368075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.554+0000 7f1392400640 1 --2- 172.21.15.94:0/2294777388 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f13680731b0 0x7f1368075670 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f138c0b73f0 tx=0x7f137c006d70 comp rx=0 tx=0).stop 2024-09-10T23:41:16.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.554+0000 7f1392400640 1 -- 172.21.15.94:0/2294777388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f138c0cb560 msgr2=0x7f138c0719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.554+0000 7f1392400640 1 --2- 172.21.15.94:0/2294777388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f138c0cb560 0x7f138c0719c0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f1374011990 tx=0x7f1374011e50 comp rx=0 tx=0).stop 2024-09-10T23:41:16.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.554+0000 7f1392400640 1 -- 172.21.15.94:0/2294777388 shutdown_connections 2024-09-10T23:41:16.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.554+0000 7f1392400640 1 --2- 172.21.15.94:0/2294777388 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f13680731b0 0x7f1368075670 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.554+0000 7f1392400640 1 --2- 172.21.15.94:0/2294777388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f138c0b6c40 0x7f138c071f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.554+0000 7f1392400640 1 --2- 172.21.15.94:0/2294777388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f138c0cb560 0x7f138c0719c0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.554+0000 7f1392400640 1 --2- 172.21.15.94:0/2294777388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1370004d50 0x7f138c1f8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.554+0000 7f1392400640 1 -- 172.21.15.94:0/2294777388 >> 172.21.15.94:0/2294777388 conn(0x7f138c0d1c60 msgr2=0x7f138c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:16.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.554+0000 7f1392400640 1 -- 172.21.15.94:0/2294777388 shutdown_connections 2024-09-10T23:41:16.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.554+0000 7f1392400640 1 -- 172.21.15.94:0/2294777388 wait complete. 2024-09-10T23:41:16.577 INFO:tasks.ceph:Scrubbing osd.6 2024-09-10T23:41:16.577 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-10T23:41:16.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.722+0000 7f2f90400640 1 -- 172.21.15.94:0/181649162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f880d02e0 msgr2=0x7f2f8815e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.722+0000 7f2f90400640 1 --2- 172.21.15.94:0/181649162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f880d02e0 0x7f2f8815e770 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009900 tx=0x7f2f7401c8b0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.722+0000 7f2f90400640 1 -- 172.21.15.94:0/181649162 shutdown_connections 2024-09-10T23:41:16.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.722+0000 7f2f90400640 1 --2- 172.21.15.94:0/181649162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f880d02e0 0x7f2f8815e770 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.722+0000 7f2f90400640 1 --2- 172.21.15.94:0/181649162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f88073ff0 0x7f2f880cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.722+0000 7f2f90400640 1 -- 172.21.15.94:0/181649162 >> 172.21.15.94:0/181649162 conn(0x7f2f8806cdf0 msgr2=0x7f2f8806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:16.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.722+0000 7f2f90400640 1 -- 172.21.15.94:0/181649162 shutdown_connections 2024-09-10T23:41:16.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.722+0000 7f2f90400640 1 -- 172.21.15.94:0/181649162 wait complete. 2024-09-10T23:41:16.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.722+0000 7f2f90400640 1 Processor -- start 2024-09-10T23:41:16.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f90400640 1 -- start start 2024-09-10T23:41:16.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f90400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f88073ff0 0x7f2f88156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f90400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f880d02e0 0x7f2f8815a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f90400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f8815ecb0 0x7f2f880b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f90400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2f88167b40 con 0x7f2f8815ecb0 2024-09-10T23:41:16.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f90400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2f881679c0 con 0x7f2f88073ff0 2024-09-10T23:41:16.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f90400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f2f88167cc0 con 0x7f2f880d02e0 2024-09-10T23:41:16.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f8e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f8815ecb0 0x7f2f880b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:16.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f8d000640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f880d02e0 0x7f2f8815a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:16.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f8e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f8815ecb0 0x7f2f880b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57000/0 (socket says 172.21.15.94:57000) 2024-09-10T23:41:16.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f8e400640 1 -- 172.21.15.94:0/2772404627 learned_addr learned my addr 172.21.15.94:0/2772404627 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:16.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f8da00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f88073ff0 0x7f2f88156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:16.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f8e400640 1 -- 172.21.15.94:0/2772404627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f880d02e0 msgr2=0x7f2f8815a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f8e400640 1 --2- 172.21.15.94:0/2772404627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f880d02e0 0x7f2f8815a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f8e400640 1 -- 172.21.15.94:0/2772404627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f88073ff0 msgr2=0x7f2f88156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f8e400640 1 --2- 172.21.15.94:0/2772404627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f88073ff0 0x7f2f88156270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f8e400640 1 -- 172.21.15.94:0/2772404627 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f88150ca0 con 0x7f2f8815ecb0 2024-09-10T23:41:16.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f8e400640 1 --2- 172.21.15.94:0/2772404627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f8815ecb0 0x7f2f880b1ab0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000b9a0 tx=0x7f2f8000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:16.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f85600640 1 -- 172.21.15.94:0/2772404627 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80019070 con 0x7f2f8815ecb0 2024-09-10T23:41:16.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f85600640 1 -- 172.21.15.94:0/2772404627 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f8000cc30 con 0x7f2f8815ecb0 2024-09-10T23:41:16.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f90400640 1 -- 172.21.15.94:0/2772404627 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f881f9560 con 0x7f2f8815ecb0 2024-09-10T23:41:16.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f85600640 1 -- 172.21.15.94:0/2772404627 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f800146a0 con 0x7f2f8815ecb0 2024-09-10T23:41:16.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f90400640 1 -- 172.21.15.94:0/2772404627 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f88155a70 con 0x7f2f8815ecb0 2024-09-10T23:41:16.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.726+0000 7f2f90400640 1 -- 172.21.15.94:0/2772404627 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2f880cfda0 con 0x7f2f8815ecb0 2024-09-10T23:41:16.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.730+0000 7f2f85600640 1 -- 172.21.15.94:0/2772404627 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f2f8001e440 con 0x7f2f8815ecb0 2024-09-10T23:41:16.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.730+0000 7f2f85600640 1 --2- 172.21.15.94:0/2772404627 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2f680731b0 0x7f2f68075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.730+0000 7f2f85600640 1 -- 172.21.15.94:0/2772404627 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7f2f800922c0 con 0x7f2f8815ecb0 2024-09-10T23:41:16.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.730+0000 7f2f8da00640 1 --2- 172.21.15.94:0/2772404627 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2f680731b0 0x7f2f68075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:16.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.730+0000 7f2f85600640 1 --2- 172.21.15.94:0/2772404627 >> [v2:172.21.15.181:6802/1977708519,v1:172.21.15.181:6803/1977708519] conn(0x7f2f68079070 0x7f2f6807b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.730+0000 7f2f85600640 1 -- 172.21.15.94:0/2772404627 --> [v2:172.21.15.181:6802/1977708519,v1:172.21.15.181:6803/1977708519] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f80020e50 con 0x7f2f68079070 2024-09-10T23:41:16.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.730+0000 7f2f85600640 1 -- 172.21.15.94:0/2772404627 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) ==== 24+0+0 (secure 0 0 0) 0x7f2f80094fc0 con 0x7f2f8815ecb0 2024-09-10T23:41:16.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.730+0000 7f2f8d000640 1 --2- 172.21.15.94:0/2772404627 >> [v2:172.21.15.181:6802/1977708519,v1:172.21.15.181:6803/1977708519] conn(0x7f2f68079070 0x7f2f6807b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:16.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.730+0000 7f2f8da00640 1 --2- 172.21.15.94:0/2772404627 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2f680731b0 0x7f2f68075670 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c004630 tx=0x7f2f7c00b390 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:16.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.730+0000 7f2f8d000640 1 --2- 172.21.15.94:0/2772404627 >> [v2:172.21.15.181:6802/1977708519,v1:172.21.15.181:6803/1977708519] conn(0x7f2f68079070 0x7f2f6807b490 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:16.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.730+0000 7f2f85600640 1 -- 172.21.15.94:0/2772404627 <== osd.6 v2:172.21.15.181:6802/1977708519 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2f80020e50 con 0x7f2f68079070 2024-09-10T23:41:16.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.746+0000 7f2f90400640 1 -- 172.21.15.94:0/2772404627 --> [v2:172.21.15.181:6802/1977708519,v1:172.21.15.181:6803/1977708519] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2f880aff40 con 0x7f2f68079070 2024-09-10T23:41:16.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.746+0000 7f2f85600640 1 -- 172.21.15.94:0/2772404627 <== osd.6 v2:172.21.15.181:6802/1977708519 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2f880aff40 con 0x7f2f68079070 2024-09-10T23:41:16.746 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:41:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T23:41:16.746 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:41:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.746+0000 7f2f90400640 1 -- 172.21.15.94:0/2772404627 >> [v2:172.21.15.181:6802/1977708519,v1:172.21.15.181:6803/1977708519] conn(0x7f2f68079070 msgr2=0x7f2f6807b490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.746+0000 7f2f90400640 1 --2- 172.21.15.94:0/2772404627 >> [v2:172.21.15.181:6802/1977708519,v1:172.21.15.181:6803/1977708519] conn(0x7f2f68079070 0x7f2f6807b490 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.746+0000 7f2f90400640 1 -- 172.21.15.94:0/2772404627 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2f680731b0 msgr2=0x7f2f68075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.746+0000 7f2f90400640 1 --2- 172.21.15.94:0/2772404627 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f2f680731b0 0x7f2f68075670 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c004630 tx=0x7f2f7c00b390 comp rx=0 tx=0).stop 2024-09-10T23:41:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.746+0000 7f2f90400640 1 -- 172.21.15.94:0/2772404627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f8815ecb0 msgr2=0x7f2f880b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.746+0000 7f2f90400640 1 --2- 172.21.15.94:0/2772404627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f8815ecb0 0x7f2f880b1ab0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000b9a0 tx=0x7f2f8000be60 comp rx=0 tx=0).stop 2024-09-10T23:41:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.746+0000 7f2f8e400640 1 -- 172.21.15.94:0/2772404627 reap_dead start 2024-09-10T23:41:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.746+0000 7f2f90400640 1 -- 172.21.15.94:0/2772404627 shutdown_connections 2024-09-10T23:41:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.746+0000 7f2f90400640 1 -- 172.21.15.94:0/2772404627 >> 172.21.15.94:0/2772404627 conn(0x7f2f8806cdf0 msgr2=0x7f2f88151290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.746+0000 7f2f90400640 1 -- 172.21.15.94:0/2772404627 shutdown_connections 2024-09-10T23:41:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.746+0000 7f2f90400640 1 -- 172.21.15.94:0/2772404627 wait complete. 2024-09-10T23:41:16.764 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-10T23:41:16.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.906+0000 7f9df6c00640 1 -- 172.21.15.94:0/3977285815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df00c64f0 msgr2=0x7f9df00ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.906+0000 7f9df6c00640 1 --2- 172.21.15.94:0/3977285815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df00c64f0 0x7f9df00ce660 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f9df00ca500 tx=0x7f9dd801c880 comp rx=0 tx=0).stop 2024-09-10T23:41:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.906+0000 7f9df6c00640 1 -- 172.21.15.94:0/3977285815 shutdown_connections 2024-09-10T23:41:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.906+0000 7f9df6c00640 1 --2- 172.21.15.94:0/3977285815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df00c64f0 0x7f9df00ce660 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.906+0000 7f9df6c00640 1 --2- 172.21.15.94:0/3977285815 >> v2:172.21.15.181:3300/0 conn(0x7f9df00ce280 0x7f9df00cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.906+0000 7f9df6c00640 1 -- 172.21.15.94:0/3977285815 >> 172.21.15.94:0/3977285815 conn(0x7f9df00cfaf0 msgr2=0x7f9df00cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.906+0000 7f9df6c00640 1 -- 172.21.15.94:0/3977285815 shutdown_connections 2024-09-10T23:41:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.906+0000 7f9df6c00640 1 -- 172.21.15.94:0/3977285815 wait complete. 2024-09-10T23:41:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.906+0000 7f9df6c00640 1 Processor -- start 2024-09-10T23:41:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.906+0000 7f9df6c00640 1 -- start start 2024-09-10T23:41:16.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.906+0000 7f9df6c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9df006e7c0 0x7f9df00c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.906+0000 7f9df6c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df00c64f0 0x7f9df014cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.906+0000 7f9df6c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9df00ce280 0x7f9df00b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9df6c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9df0167500 con 0x7f9df00c64f0 2024-09-10T23:41:16.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9df6c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9df01583d0 con 0x7f9df00ce280 2024-09-10T23:41:16.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9df6c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9df0167680 con 0x7f9df006e7c0 2024-09-10T23:41:16.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9de7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df00c64f0 0x7f9df014cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9de7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df00c64f0 0x7f9df014cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57020/0 (socket says 172.21.15.94:57020) 2024-09-10T23:41:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9de7400640 1 -- 172.21.15.94:0/3039920015 learned_addr learned my addr 172.21.15.94:0/3039920015 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9de7400640 1 -- 172.21.15.94:0/3039920015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9df006e7c0 msgr2=0x7f9df00c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9de7400640 1 --2- 172.21.15.94:0/3039920015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9df006e7c0 0x7f9df00c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9de7400640 1 -- 172.21.15.94:0/3039920015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9df00ce280 msgr2=0x7f9df00b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9de7400640 1 --2- 172.21.15.94:0/3039920015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9df00ce280 0x7f9df00b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:16.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9de7400640 1 -- 172.21.15.94:0/3039920015 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df00c7b90 con 0x7f9df00c64f0 2024-09-10T23:41:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9de7400640 1 --2- 172.21.15.94:0/3039920015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df00c64f0 0x7f9df014cfb0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80b0040 tx=0x7f9dd80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9decc00640 1 -- 172.21.15.94:0/3039920015 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd80b2040 con 0x7f9df00c64f0 2024-09-10T23:41:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9decc00640 1 -- 172.21.15.94:0/3039920015 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dd8005030 con 0x7f9df00c64f0 2024-09-10T23:41:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9df6c00640 1 -- 172.21.15.94:0/3039920015 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df014b6b0 con 0x7f9df00c64f0 2024-09-10T23:41:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9decc00640 1 -- 172.21.15.94:0/3039920015 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd80054c0 con 0x7f9df00c64f0 2024-09-10T23:41:16.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9df6c00640 1 -- 172.21.15.94:0/3039920015 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df014bc70 con 0x7f9df00c64f0 2024-09-10T23:41:16.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9df6c00640 1 -- 172.21.15.94:0/3039920015 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df00ce660 con 0x7f9df00c64f0 2024-09-10T23:41:16.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.910+0000 7f9decc00640 1 -- 172.21.15.94:0/3039920015 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f9dd80046d0 con 0x7f9df00c64f0 2024-09-10T23:41:16.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.914+0000 7f9decc00640 1 --2- 172.21.15.94:0/3039920015 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9dcc0731b0 0x7f9dcc075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:16.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.914+0000 7f9decc00640 1 -- 172.21.15.94:0/3039920015 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7f9dd812c120 con 0x7f9df00c64f0 2024-09-10T23:41:16.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.914+0000 7f9defe00640 1 --2- 172.21.15.94:0/3039920015 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9dcc0731b0 0x7f9dcc075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:16.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.914+0000 7f9defe00640 1 --2- 172.21.15.94:0/3039920015 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9dcc0731b0 0x7f9dcc075670 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9de00058d0 tx=0x7f9de0005860 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:16.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:16.914+0000 7f9decc00640 1 -- 172.21.15.94:0/3039920015 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9dd8016470 con 0x7f9df00c64f0 2024-09-10T23:41:17.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.082+0000 7f9df6c00640 1 -- 172.21.15.94:0/3039920015 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f9df0069e20 con 0x7f9dcc0731b0 2024-09-10T23:41:17.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.082+0000 7f9decc00640 1 -- 172.21.15.94:0/3039920015 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9df0069e20 con 0x7f9dcc0731b0 2024-09-10T23:41:17.083 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 6 to deep-scrub 2024-09-10T23:41:17.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.082+0000 7f9df6c00640 1 -- 172.21.15.94:0/3039920015 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9dcc0731b0 msgr2=0x7f9dcc075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.082+0000 7f9df6c00640 1 --2- 172.21.15.94:0/3039920015 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9dcc0731b0 0x7f9dcc075670 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9de00058d0 tx=0x7f9de0005860 comp rx=0 tx=0).stop 2024-09-10T23:41:17.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.082+0000 7f9df6c00640 1 -- 172.21.15.94:0/3039920015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df00c64f0 msgr2=0x7f9df014cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.082+0000 7f9df6c00640 1 --2- 172.21.15.94:0/3039920015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df00c64f0 0x7f9df014cfb0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80b0040 tx=0x7f9dd80040c0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.082+0000 7f9df6c00640 1 -- 172.21.15.94:0/3039920015 shutdown_connections 2024-09-10T23:41:17.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.082+0000 7f9df6c00640 1 --2- 172.21.15.94:0/3039920015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9df00ce280 0x7f9df00b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.082+0000 7f9df6c00640 1 --2- 172.21.15.94:0/3039920015 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f9dcc0731b0 0x7f9dcc075670 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.082+0000 7f9df6c00640 1 --2- 172.21.15.94:0/3039920015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9df00c64f0 0x7f9df014cfb0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.082+0000 7f9df6c00640 1 --2- 172.21.15.94:0/3039920015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9df006e7c0 0x7f9df00c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.082+0000 7f9df6c00640 1 -- 172.21.15.94:0/3039920015 >> 172.21.15.94:0/3039920015 conn(0x7f9df00cfaf0 msgr2=0x7f9df0072560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:17.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.086+0000 7f9df6c00640 1 -- 172.21.15.94:0/3039920015 shutdown_connections 2024-09-10T23:41:17.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.086+0000 7f9df6c00640 1 -- 172.21.15.94:0/3039920015 wait complete. 2024-09-10T23:41:17.105 INFO:tasks.ceph:Scrubbing osd.7 2024-09-10T23:41:17.105 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-10T23:41:17.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.250+0000 7fa725400640 1 -- 172.21.15.94:0/892883492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa704004030 msgr2=0x7fa704004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.250+0000 7fa725400640 1 --2- 172.21.15.94:0/892883492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa704004030 0x7fa704004810 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fa720073130 tx=0x7fa70801d210 comp rx=0 tx=0).stop 2024-09-10T23:41:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.250+0000 7fa724a00640 1 -- 172.21.15.94:0/892883492 reap_dead start 2024-09-10T23:41:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.250+0000 7fa725400640 1 -- 172.21.15.94:0/892883492 shutdown_connections 2024-09-10T23:41:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.250+0000 7fa725400640 1 -- 172.21.15.94:0/892883492 >> 172.21.15.94:0/892883492 conn(0x7fa72009aa70 msgr2=0x7fa72009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.250+0000 7fa725400640 1 -- 172.21.15.94:0/892883492 shutdown_connections 2024-09-10T23:41:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.250+0000 7fa725400640 1 -- 172.21.15.94:0/892883492 wait complete. 2024-09-10T23:41:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa725400640 1 Processor -- start 2024-09-10T23:41:17.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa725400640 1 -- start start 2024-09-10T23:41:17.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa725400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa704004030 0x7fa72015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:17.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa725400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa720156b00 0x7fa7201f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa725400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7201f4a30 0x7fa7201f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa725400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa7200affc0 con 0x7fa7201f4a30 2024-09-10T23:41:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa725400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa7200afe40 con 0x7fa720156b00 2024-09-10T23:41:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa725400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fa7200b0140 con 0x7fa704004030 2024-09-10T23:41:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa724a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7201f4a30 0x7fa7201f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa724a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7201f4a30 0x7fa7201f4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57050/0 (socket says 172.21.15.94:57050) 2024-09-10T23:41:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa71fe00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa704004030 0x7fa72015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa724a00640 1 -- 172.21.15.94:0/2360585218 learned_addr learned my addr 172.21.15.94:0/2360585218 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa717400640 1 --2- 172.21.15.94:0/2360585218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa720156b00 0x7fa7201f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa724a00640 1 -- 172.21.15.94:0/2360585218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa704004030 msgr2=0x7fa72015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa724a00640 1 --2- 172.21.15.94:0/2360585218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa704004030 0x7fa72015a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa724a00640 1 -- 172.21.15.94:0/2360585218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa720156b00 msgr2=0x7fa7201f44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa724a00640 1 --2- 172.21.15.94:0/2360585218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa720156b00 0x7fa7201f44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa724a00640 1 -- 172.21.15.94:0/2360585218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7201591b0 con 0x7fa7201f4a30 2024-09-10T23:41:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa724a00640 1 --2- 172.21.15.94:0/2360585218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7201f4a30 0x7fa7201f4eb0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fa720072c20 tx=0x7fa718011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa71cc00640 1 -- 172.21.15.94:0/2360585218 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718013920 con 0x7fa7201f4a30 2024-09-10T23:41:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa71cc00640 1 -- 172.21.15.94:0/2360585218 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa71801a070 con 0x7fa7201f4a30 2024-09-10T23:41:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa71cc00640 1 -- 172.21.15.94:0/2360585218 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718013e30 con 0x7fa7201f4a30 2024-09-10T23:41:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa725400640 1 -- 172.21.15.94:0/2360585218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7201575f0 con 0x7fa7201f4a30 2024-09-10T23:41:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.254+0000 7fa725400640 1 -- 172.21.15.94:0/2360585218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa720158910 con 0x7fa7201f4a30 2024-09-10T23:41:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.258+0000 7fa725400640 1 -- 172.21.15.94:0/2360585218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa7200cfda0 con 0x7fa7201f4a30 2024-09-10T23:41:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.258+0000 7fa71cc00640 1 -- 172.21.15.94:0/2360585218 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7fa718010040 con 0x7fa7201f4a30 2024-09-10T23:41:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.258+0000 7fa71cc00640 1 --2- 172.21.15.94:0/2360585218 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa6fc0731e0 0x7fa6fc0756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.258+0000 7fa71cc00640 1 -- 172.21.15.94:0/2360585218 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7fa718095e10 con 0x7fa7201f4a30 2024-09-10T23:41:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.258+0000 7fa71fe00640 1 --2- 172.21.15.94:0/2360585218 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa6fc0731e0 0x7fa6fc0756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.258+0000 7fa71cc00640 1 --2- 172.21.15.94:0/2360585218 >> [v2:172.21.15.181:6811/3568562877,v1:172.21.15.181:6813/3568562877] conn(0x7fa6fc0790a0 0x7fa6fc07b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.258+0000 7fa71cc00640 1 -- 172.21.15.94:0/2360585218 --> [v2:172.21.15.181:6811/3568562877,v1:172.21.15.181:6813/3568562877] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa718027e50 con 0x7fa6fc0790a0 2024-09-10T23:41:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.258+0000 7fa71cc00640 1 -- 172.21.15.94:0/2360585218 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) ==== 24+0+0 (secure 0 0 0) 0x7fa718098540 con 0x7fa7201f4a30 2024-09-10T23:41:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.258+0000 7fa717400640 1 --2- 172.21.15.94:0/2360585218 >> [v2:172.21.15.181:6811/3568562877,v1:172.21.15.181:6813/3568562877] conn(0x7fa6fc0790a0 0x7fa6fc07b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.258+0000 7fa71fe00640 1 --2- 172.21.15.94:0/2360585218 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa6fc0731e0 0x7fa6fc0756a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa710009a80 tx=0x7fa7100091e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.258+0000 7fa717400640 1 --2- 172.21.15.94:0/2360585218 >> [v2:172.21.15.181:6811/3568562877,v1:172.21.15.181:6813/3568562877] conn(0x7fa6fc0790a0 0x7fa6fc07b4c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.262+0000 7fa71cc00640 1 -- 172.21.15.94:0/2360585218 <== osd.7 v2:172.21.15.181:6811/3568562877 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa718027e50 con 0x7fa6fc0790a0 2024-09-10T23:41:17.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.274+0000 7fa725400640 1 -- 172.21.15.94:0/2360585218 --> [v2:172.21.15.181:6811/3568562877,v1:172.21.15.181:6813/3568562877] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa7201e7a20 con 0x7fa6fc0790a0 2024-09-10T23:41:17.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.274+0000 7fa71cc00640 1 -- 172.21.15.94:0/2360585218 <== osd.7 v2:172.21.15.181:6811/3568562877 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa7201e7a20 con 0x7fa6fc0790a0 2024-09-10T23:41:17.275 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-10T23:41:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T23:41:17.275 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-10T23:41:17.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.274+0000 7fa725400640 1 -- 172.21.15.94:0/2360585218 >> [v2:172.21.15.181:6811/3568562877,v1:172.21.15.181:6813/3568562877] conn(0x7fa6fc0790a0 msgr2=0x7fa6fc07b4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.274+0000 7fa725400640 1 --2- 172.21.15.94:0/2360585218 >> [v2:172.21.15.181:6811/3568562877,v1:172.21.15.181:6813/3568562877] conn(0x7fa6fc0790a0 0x7fa6fc07b4c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.274+0000 7fa725400640 1 -- 172.21.15.94:0/2360585218 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa6fc0731e0 msgr2=0x7fa6fc0756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.274+0000 7fa725400640 1 --2- 172.21.15.94:0/2360585218 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fa6fc0731e0 0x7fa6fc0756a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa710009a80 tx=0x7fa7100091e0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.274+0000 7fa725400640 1 -- 172.21.15.94:0/2360585218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7201f4a30 msgr2=0x7fa7201f4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.274+0000 7fa725400640 1 --2- 172.21.15.94:0/2360585218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7201f4a30 0x7fa7201f4eb0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fa720072c20 tx=0x7fa718011a10 comp rx=0 tx=0).stop 2024-09-10T23:41:17.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.274+0000 7fa724a00640 1 -- 172.21.15.94:0/2360585218 reap_dead start 2024-09-10T23:41:17.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.274+0000 7fa725400640 1 -- 172.21.15.94:0/2360585218 shutdown_connections 2024-09-10T23:41:17.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.274+0000 7fa725400640 1 -- 172.21.15.94:0/2360585218 >> 172.21.15.94:0/2360585218 conn(0x7fa72009aa70 msgr2=0x7fa72006f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:17.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.274+0000 7fa725400640 1 -- 172.21.15.94:0/2360585218 shutdown_connections 2024-09-10T23:41:17.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.274+0000 7fa725400640 1 -- 172.21.15.94:0/2360585218 wait complete. 2024-09-10T23:41:17.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-10T23:41:17.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6acaa00640 1 -- 172.21.15.94:0/1148764539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac415b6a0 msgr2=0x7f6ac406ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6acaa00640 1 --2- 172.21.15.94:0/1148764539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac415b6a0 0x7f6ac406ec00 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f6abc009c80 tx=0x7f6abc01cda0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6acaa00640 1 -- 172.21.15.94:0/1148764539 shutdown_connections 2024-09-10T23:41:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6acaa00640 1 --2- 172.21.15.94:0/1148764539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac415b6a0 0x7f6ac406ec00 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6acaa00640 1 --2- 172.21.15.94:0/1148764539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ac40c64f0 0x7f6ac40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6acaa00640 1 -- 172.21.15.94:0/1148764539 >> 172.21.15.94:0/1148764539 conn(0x7f6ac40cfaf0 msgr2=0x7f6ac40cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6acaa00640 1 -- 172.21.15.94:0/1148764539 shutdown_connections 2024-09-10T23:41:17.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6acaa00640 1 -- 172.21.15.94:0/1148764539 wait complete. 2024-09-10T23:41:17.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6acaa00640 1 Processor -- start 2024-09-10T23:41:17.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6acaa00640 1 -- start start 2024-09-10T23:41:17.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6acaa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac406e7c0 0x7f6ac40b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:17.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6acaa00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ac40c64f0 0x7f6ac414f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:17.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6acaa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac415b6a0 0x7f6ac41fbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:17.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6acaa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac40ce830 con 0x7f6ac406e7c0 2024-09-10T23:41:17.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6acaa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac40ce6b0 con 0x7f6ac415b6a0 2024-09-10T23:41:17.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6acaa00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f6ac4168400 con 0x7f6ac40c64f0 2024-09-10T23:41:17.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6ac3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac406e7c0 0x7f6ac40b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:17.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6ac3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac406e7c0 0x7f6ac40b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57064/0 (socket says 172.21.15.94:57064) 2024-09-10T23:41:17.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6ac3e00640 1 -- 172.21.15.94:0/2375359945 learned_addr learned my addr 172.21.15.94:0/2375359945 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:17.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6abb400640 1 --2- 172.21.15.94:0/2375359945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ac40c64f0 0x7f6ac414f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:17.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6ac8a00640 1 --2- 172.21.15.94:0/2375359945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac415b6a0 0x7f6ac41fbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:17.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6ac3e00640 1 -- 172.21.15.94:0/2375359945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ac40c64f0 msgr2=0x7f6ac414f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6ac3e00640 1 --2- 172.21.15.94:0/2375359945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ac40c64f0 0x7f6ac414f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6ac3e00640 1 -- 172.21.15.94:0/2375359945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac415b6a0 msgr2=0x7f6ac41fbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6ac3e00640 1 --2- 172.21.15.94:0/2375359945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac415b6a0 0x7f6ac41fbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.438+0000 7f6ac3e00640 1 -- 172.21.15.94:0/2375359945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac40b41a0 con 0x7f6ac406e7c0 2024-09-10T23:41:17.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.442+0000 7f6ac3e00640 1 --2- 172.21.15.94:0/2375359945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac406e7c0 0x7f6ac40b3c60 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400c970 tx=0x7f6ab400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:17.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.442+0000 7f6ac0c00640 1 -- 172.21.15.94:0/2375359945 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4007c70 con 0x7f6ac406e7c0 2024-09-10T23:41:17.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.442+0000 7f6ac0c00640 1 -- 172.21.15.94:0/2375359945 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ab400f450 con 0x7f6ac406e7c0 2024-09-10T23:41:17.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.442+0000 7f6ac0c00640 1 -- 172.21.15.94:0/2375359945 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab400f8a0 con 0x7f6ac406e7c0 2024-09-10T23:41:17.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.442+0000 7f6acaa00640 1 -- 172.21.15.94:0/2375359945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac4155130 con 0x7f6ac406e7c0 2024-09-10T23:41:17.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.442+0000 7f6acaa00640 1 -- 172.21.15.94:0/2375359945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac41556f0 con 0x7f6ac406e7c0 2024-09-10T23:41:17.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.442+0000 7f6acaa00640 1 -- 172.21.15.94:0/2375359945 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac40d0850 con 0x7f6ac406e7c0 2024-09-10T23:41:17.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.442+0000 7f6ac0c00640 1 -- 172.21.15.94:0/2375359945 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f6ab400fa40 con 0x7f6ac406e7c0 2024-09-10T23:41:17.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.442+0000 7f6ac0c00640 1 --2- 172.21.15.94:0/2375359945 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6aa00731e0 0x7f6aa00756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:17.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.442+0000 7f6ac0c00640 1 -- 172.21.15.94:0/2375359945 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7f6ab4092420 con 0x7f6ac406e7c0 2024-09-10T23:41:17.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.446+0000 7f6abb400640 1 --2- 172.21.15.94:0/2375359945 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6aa00731e0 0x7f6aa00756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:17.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.446+0000 7f6abb400640 1 --2- 172.21.15.94:0/2375359945 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6aa00731e0 0x7f6aa00756a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4074d40 tx=0x7f6aac009040 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:17.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.446+0000 7f6ac0c00640 1 -- 172.21.15.94:0/2375359945 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6ab4014030 con 0x7f6ac406e7c0 2024-09-10T23:41:17.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.614+0000 7f6acaa00640 1 -- 172.21.15.94:0/2375359945 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f6ac40737b0 con 0x7f6aa00731e0 2024-09-10T23:41:17.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.614+0000 7f6ac0c00640 1 -- 172.21.15.94:0/2375359945 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6ac40737b0 con 0x7f6aa00731e0 2024-09-10T23:41:17.616 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 7 to deep-scrub 2024-09-10T23:41:17.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.618+0000 7f6acaa00640 1 -- 172.21.15.94:0/2375359945 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6aa00731e0 msgr2=0x7f6aa00756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.618+0000 7f6acaa00640 1 --2- 172.21.15.94:0/2375359945 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6aa00731e0 0x7f6aa00756a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4074d40 tx=0x7f6aac009040 comp rx=0 tx=0).stop 2024-09-10T23:41:17.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.618+0000 7f6acaa00640 1 -- 172.21.15.94:0/2375359945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac406e7c0 msgr2=0x7f6ac40b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.618+0000 7f6acaa00640 1 --2- 172.21.15.94:0/2375359945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac406e7c0 0x7f6ac40b3c60 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400c970 tx=0x7f6ab400ce30 comp rx=0 tx=0).stop 2024-09-10T23:41:17.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.618+0000 7f6acaa00640 1 -- 172.21.15.94:0/2375359945 shutdown_connections 2024-09-10T23:41:17.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.618+0000 7f6acaa00640 1 --2- 172.21.15.94:0/2375359945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac415b6a0 0x7f6ac41fbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.618+0000 7f6acaa00640 1 --2- 172.21.15.94:0/2375359945 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f6aa00731e0 0x7f6aa00756a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.618+0000 7f6acaa00640 1 --2- 172.21.15.94:0/2375359945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ac40c64f0 0x7f6ac414f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.618+0000 7f6acaa00640 1 --2- 172.21.15.94:0/2375359945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ac406e7c0 0x7f6ac40b3c60 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.618+0000 7f6acaa00640 1 -- 172.21.15.94:0/2375359945 >> 172.21.15.94:0/2375359945 conn(0x7f6ac40cfaf0 msgr2=0x7f6ac4073cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:17.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.618+0000 7f6acaa00640 1 -- 172.21.15.94:0/2375359945 shutdown_connections 2024-09-10T23:41:17.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.618+0000 7f6acaa00640 1 -- 172.21.15.94:0/2375359945 wait complete. 2024-09-10T23:41:17.637 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:41:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.778+0000 7f187be00640 1 -- 172.21.15.94:0/756371031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c004030 msgr2=0x7f185c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.778+0000 7f187be00640 1 --2- 172.21.15.94:0/756371031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c004030 0x7f185c004810 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f1860009b60 tx=0x7f186001d270 comp rx=0 tx=0).stop 2024-09-10T23:41:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.778+0000 7f1879e00640 1 -- 172.21.15.94:0/756371031 reap_dead start 2024-09-10T23:41:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.778+0000 7f187be00640 1 -- 172.21.15.94:0/756371031 shutdown_connections 2024-09-10T23:41:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.778+0000 7f187be00640 1 -- 172.21.15.94:0/756371031 >> 172.21.15.94:0/756371031 conn(0x7f187406cdf0 msgr2=0x7f187406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.778+0000 7f187be00640 1 -- 172.21.15.94:0/756371031 shutdown_connections 2024-09-10T23:41:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.778+0000 7f187be00640 1 -- 172.21.15.94:0/756371031 wait complete. 2024-09-10T23:41:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.778+0000 7f187be00640 1 Processor -- start 2024-09-10T23:41:17.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.778+0000 7f187be00640 1 -- start start 2024-09-10T23:41:17.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f187be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c004030 0x7f18741e4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f187be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f187413e350 0x7f18741e59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f187be00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f187413e7d0 0x7f18741e5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f187be00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1874072bd0 con 0x7f185c004030 2024-09-10T23:41:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f187be00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f18740ce370 con 0x7f187413e350 2024-09-10T23:41:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f187be00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f1874072d50 con 0x7f187413e7d0 2024-09-10T23:41:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f1879400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c004030 0x7f18741e4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f1879400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c004030 0x7f18741e4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57070/0 (socket says 172.21.15.94:57070) 2024-09-10T23:41:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f1879400640 1 -- 172.21.15.94:0/2718898814 learned_addr learned my addr 172.21.15.94:0/2718898814 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f1878a00640 1 --2- 172.21.15.94:0/2718898814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f187413e350 0x7f18741e59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f1879e00640 1 --2- 172.21.15.94:0/2718898814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f187413e7d0 0x7f18741e5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f1879400640 1 -- 172.21.15.94:0/2718898814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f187413e7d0 msgr2=0x7f18741e5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f1879400640 1 --2- 172.21.15.94:0/2718898814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f187413e7d0 0x7f18741e5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f1879400640 1 -- 172.21.15.94:0/2718898814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f187413e350 msgr2=0x7f18741e59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f1879400640 1 --2- 172.21.15.94:0/2718898814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f187413e350 0x7f18741e59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f1879400640 1 -- 172.21.15.94:0/2718898814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18741f8c10 con 0x7f185c004030 2024-09-10T23:41:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f1879400640 1 --2- 172.21.15.94:0/2718898814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c004030 0x7f18741e4260 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f186800e310 tx=0x7f186800e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f1871600640 1 -- 172.21.15.94:0/2718898814 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868012850 con 0x7f185c004030 2024-09-10T23:41:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f1871600640 1 -- 172.21.15.94:0/2718898814 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1868012ae0 con 0x7f185c004030 2024-09-10T23:41:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f187be00640 1 -- 172.21.15.94:0/2718898814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18741f7cb0 con 0x7f185c004030 2024-09-10T23:41:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f1871600640 1 -- 172.21.15.94:0/2718898814 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186801a400 con 0x7f185c004030 2024-09-10T23:41:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f187be00640 1 -- 172.21.15.94:0/2718898814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18741f8270 con 0x7f185c004030 2024-09-10T23:41:17.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f187be00640 1 -- 172.21.15.94:0/2718898814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18740732e0 con 0x7f185c004030 2024-09-10T23:41:17.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.782+0000 7f1871600640 1 -- 172.21.15.94:0/2718898814 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f1868022030 con 0x7f185c004030 2024-09-10T23:41:17.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.786+0000 7f1871600640 1 --2- 172.21.15.94:0/2718898814 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f18540731b0 0x7f1854075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:17.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.786+0000 7f1871600640 1 -- 172.21.15.94:0/2718898814 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7f1868095120 con 0x7f185c004030 2024-09-10T23:41:17.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.786+0000 7f1878a00640 1 --2- 172.21.15.94:0/2718898814 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f18540731b0 0x7f1854075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:17.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.786+0000 7f1878a00640 1 --2- 172.21.15.94:0/2718898814 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f18540731b0 0x7f1854075670 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f18600ae000 tx=0x7f186001a010 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:17.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.790+0000 7f1871600640 1 -- 172.21.15.94:0/2718898814 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1868016030 con 0x7f185c004030 2024-09-10T23:41:17.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.950+0000 7f187be00640 1 -- 172.21.15.94:0/2718898814 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f187414eb90 con 0x7f18540731b0 2024-09-10T23:41:17.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.954+0000 7f1871600640 1 -- 172.21.15.94:0/2718898814 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13484 (secure 0 0 0) 0x7f187414eb90 con 0x7f18540731b0 2024-09-10T23:41:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:41:17.953 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:41:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.954+0000 7f187be00640 1 -- 172.21.15.94:0/2718898814 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f18540731b0 msgr2=0x7f1854075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.954+0000 7f187be00640 1 --2- 172.21.15.94:0/2718898814 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f18540731b0 0x7f1854075670 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f18600ae000 tx=0x7f186001a010 comp rx=0 tx=0).stop 2024-09-10T23:41:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.954+0000 7f187be00640 1 -- 172.21.15.94:0/2718898814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c004030 msgr2=0x7f18741e4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.954+0000 7f187be00640 1 --2- 172.21.15.94:0/2718898814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c004030 0x7f18741e4260 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f186800e310 tx=0x7f186800e7d0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.954+0000 7f187be00640 1 -- 172.21.15.94:0/2718898814 shutdown_connections 2024-09-10T23:41:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.954+0000 7f187be00640 1 --2- 172.21.15.94:0/2718898814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f187413e350 0x7f18741e59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.954+0000 7f187be00640 1 --2- 172.21.15.94:0/2718898814 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f18540731b0 0x7f1854075670 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.954+0000 7f187be00640 1 --2- 172.21.15.94:0/2718898814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f187413e7d0 0x7f18741e5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.954+0000 7f187be00640 1 --2- 172.21.15.94:0/2718898814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c004030 0x7f18741e4260 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:17.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.954+0000 7f187be00640 1 -- 172.21.15.94:0/2718898814 >> 172.21.15.94:0/2718898814 conn(0x7f187406cdf0 msgr2=0x7f187415e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:17.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.954+0000 7f187be00640 1 -- 172.21.15.94:0/2718898814 shutdown_connections 2024-09-10T23:41:17.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:17.954+0000 7f187be00640 1 -- 172.21.15.94:0/2718898814 wait complete. 2024-09-10T23:41:17.974 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-10T23:41:16.793514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":287776,"kb_used_data":17448,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754686944,"statfs":{"total":773094113280,"available":772799430656,"internally_reserved":0,"allocated":17866752,"data_stored":8521928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-156,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-2,"num_write_kb":-2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.262701"},"pg_stats":[{"pgid":"1.0","version":"70'79","reported_seq":402,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T23:41:10.044729+0000","last_change":"2024-09-10T23:32:33.334294+0000","last_active":"2024-09-10T23:41:10.044729+0000","last_peered":"2024-09-10T23:41:10.044729+0000","last_clean":"2024-09-10T23:41:10.044729+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:41:10.044729+0000","last_undegraded":"2024-09-10T23:41:10.044729+0000","last_fullsized":"2024-09-10T23:41:10.044729+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:32:32.321671+0000","last_clean_scrub_stamp":"2024-09-10T23:32:32.321671+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.545496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33232,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338608,"statfs":{"total":96636764160,"available":96602734592,"internally_reserved":0,"allocated":2084864,"data_stored":917649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542253,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33228,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2084864,"data_stored":917649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509549,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33804,"kb_used_data":2616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338036,"statfs":{"total":96636764160,"available":96602148864,"internally_reserved":0,"allocated":2678784,"data_stored":1508017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37388,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2084864,"data_stored":917649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37384,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334456,"statfs":{"total":96636764160,"available":96598482944,"internally_reserved":0,"allocated":2084864,"data_stored":917649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509549,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37388,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2084864,"data_stored":917649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509548,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37964,"kb_used_data":2616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333876,"statfs":{"total":96636764160,"available":96597889024,"internally_reserved":0,"allocated":2678784,"data_stored":1508017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37388,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2084864,"data_stored":917649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:41:17.978 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-10T23:32:32.321671+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=23, tm_min=32, tm_sec=32, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=23, tm_min=41, tm_sec=12, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T23:41:17.978 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-10T23:41:35.778 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:41:35.782 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:41:37.979 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T23:41:38.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.130+0000 7f9988200640 1 -- 172.21.15.94:0/2224542855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9980157c60 msgr2=0x7f9980072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:38.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.130+0000 7f9988200640 1 --2- 172.21.15.94:0/2224542855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9980157c60 0x7f9980072610 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f997400acd0 tx=0x7f997401de60 comp rx=0 tx=0).stop 2024-09-10T23:41:38.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.130+0000 7f9988200640 1 -- 172.21.15.94:0/2224542855 shutdown_connections 2024-09-10T23:41:38.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.130+0000 7f9988200640 1 --2- 172.21.15.94:0/2224542855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9980157c60 0x7f9980072610 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.130+0000 7f9988200640 1 -- 172.21.15.94:0/2224542855 >> 172.21.15.94:0/2224542855 conn(0x7f998006cdf0 msgr2=0x7f998006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:38.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.130+0000 7f9988200640 1 -- 172.21.15.94:0/2224542855 shutdown_connections 2024-09-10T23:41:38.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.130+0000 7f9988200640 1 -- 172.21.15.94:0/2224542855 wait complete. 2024-09-10T23:41:38.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9988200640 1 Processor -- start 2024-09-10T23:41:38.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9988200640 1 -- start start 2024-09-10T23:41:38.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9988200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f998006f600 0x7f99801f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:38.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9988200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9980070150 0x7f99800c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:38.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9988200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9980157c60 0x7f99800cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:38.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9988200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9980168380 con 0x7f9980070150 2024-09-10T23:41:38.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9988200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9980168200 con 0x7f998006f600 2024-09-10T23:41:38.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9988200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7f9980168500 con 0x7f9980157c60 2024-09-10T23:41:38.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9984e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9980070150 0x7f99800c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:38.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9984e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9980070150 0x7f99800c4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47062/0 (socket says 172.21.15.94:47062) 2024-09-10T23:41:38.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9984e00640 1 -- 172.21.15.94:0/858342405 learned_addr learned my addr 172.21.15.94:0/858342405 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:38.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9985800640 1 --2- 172.21.15.94:0/858342405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f998006f600 0x7f99801f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:38.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9986200640 1 --2- 172.21.15.94:0/858342405 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9980157c60 0x7f99800cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:38.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9984e00640 1 -- 172.21.15.94:0/858342405 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9980157c60 msgr2=0x7f99800cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:38.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9984e00640 1 --2- 172.21.15.94:0/858342405 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9980157c60 0x7f99800cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9984e00640 1 -- 172.21.15.94:0/858342405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f998006f600 msgr2=0x7f99801f7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:38.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9984e00640 1 --2- 172.21.15.94:0/858342405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f998006f600 0x7f99801f7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9984e00640 1 -- 172.21.15.94:0/858342405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99800c4e80 con 0x7f9980070150 2024-09-10T23:41:38.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9984e00640 1 --2- 172.21.15.94:0/858342405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9980070150 0x7f99800c4940 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f996c00c500 tx=0x7f996c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:38.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f997d600640 1 -- 172.21.15.94:0/858342405 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c017070 con 0x7f9980070150 2024-09-10T23:41:38.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f997d600640 1 -- 172.21.15.94:0/858342405 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f996c003600 con 0x7f9980070150 2024-09-10T23:41:38.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9988200640 1 -- 172.21.15.94:0/858342405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99801f84c0 con 0x7f9980070150 2024-09-10T23:41:38.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f997d600640 1 -- 172.21.15.94:0/858342405 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c003a90 con 0x7f9980070150 2024-09-10T23:41:38.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.134+0000 7f9988200640 1 -- 172.21.15.94:0/858342405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99801f8d40 con 0x7f9980070150 2024-09-10T23:41:38.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.138+0000 7f9988200640 1 -- 172.21.15.94:0/858342405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9980072610 con 0x7f9980070150 2024-09-10T23:41:38.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.138+0000 7f997d600640 1 -- 172.21.15.94:0/858342405 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7f996c003c30 con 0x7f9980070150 2024-09-10T23:41:38.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.138+0000 7f997d600640 1 --2- 172.21.15.94:0/858342405 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f99600731b0 0x7f9960075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:38.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.138+0000 7f997d600640 1 -- 172.21.15.94:0/858342405 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7f996c0916a0 con 0x7f9980070150 2024-09-10T23:41:38.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.138+0000 7f9985800640 1 --2- 172.21.15.94:0/858342405 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f99600731b0 0x7f9960075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:38.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.138+0000 7f9985800640 1 --2- 172.21.15.94:0/858342405 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f99600731b0 0x7f9960075670 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f99740b0000 tx=0x7f997401c6a0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:38.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.142+0000 7f997d600640 1 -- 172.21.15.94:0/858342405 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f996c05e630 con 0x7f9980070150 2024-09-10T23:41:38.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.306+0000 7f9988200640 1 -- 172.21.15.94:0/858342405 --> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f998006e7e0 con 0x7f99600731b0 2024-09-10T23:41:38.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.310+0000 7f997d600640 1 -- 172.21.15.94:0/858342405 <== mgr.4146 v2:172.21.15.94:6836/2927908830 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13370 (secure 0 0 0) 0x7f998006e7e0 con 0x7f99600731b0 2024-09-10T23:41:38.309 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:41:38.309 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-10T23:41:38.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.310+0000 7f9988200640 1 -- 172.21.15.94:0/858342405 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f99600731b0 msgr2=0x7f9960075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:38.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.310+0000 7f9988200640 1 --2- 172.21.15.94:0/858342405 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f99600731b0 0x7f9960075670 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f99740b0000 tx=0x7f997401c6a0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.310+0000 7f9988200640 1 -- 172.21.15.94:0/858342405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9980070150 msgr2=0x7f99800c4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:38.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.310+0000 7f9988200640 1 --2- 172.21.15.94:0/858342405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9980070150 0x7f99800c4940 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f996c00c500 tx=0x7f996c00c9c0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.310+0000 7f9988200640 1 -- 172.21.15.94:0/858342405 shutdown_connections 2024-09-10T23:41:38.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.310+0000 7f9988200640 1 --2- 172.21.15.94:0/858342405 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7f99600731b0 0x7f9960075670 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.310+0000 7f9988200640 1 --2- 172.21.15.94:0/858342405 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9980157c60 0x7f99800cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.310+0000 7f9988200640 1 --2- 172.21.15.94:0/858342405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9980070150 0x7f99800c4940 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.310+0000 7f9988200640 1 --2- 172.21.15.94:0/858342405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f998006f600 0x7f99801f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.310+0000 7f9988200640 1 -- 172.21.15.94:0/858342405 >> 172.21.15.94:0/858342405 conn(0x7f998006cdf0 msgr2=0x7f9980153560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:38.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.310+0000 7f9988200640 1 -- 172.21.15.94:0/858342405 shutdown_connections 2024-09-10T23:41:38.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.310+0000 7f9988200640 1 -- 172.21.15.94:0/858342405 wait complete. 2024-09-10T23:41:38.331 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-09-10T23:41:36.798200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":287768,"kb_used_data":17448,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754686952,"statfs":{"total":773094113280,"available":772799438848,"internally_reserved":0,"allocated":17866752,"data_stored":8521928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002876"},"pg_stats":[{"pgid":"1.0","version":"165'81","reported_seq":412,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-10T23:41:15.186888+0000","last_change":"2024-09-10T23:41:15.186888+0000","last_active":"2024-09-10T23:41:15.186888+0000","last_peered":"2024-09-10T23:41:15.186888+0000","last_clean":"2024-09-10T23:41:15.186888+0000","last_became_active":"2024-09-10T23:32:33.333923+0000","last_became_peered":"2024-09-10T23:32:33.333923+0000","last_unstale":"2024-09-10T23:41:15.186888+0000","last_undegraded":"2024-09-10T23:41:15.186888+0000","last_fullsized":"2024-09-10T23:41:15.186888+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"165'81","last_scrub_stamp":"2024-09-10T23:41:15.186773+0000","last_deep_scrub":"165'81","last_deep_scrub_stamp":"2024-09-10T23:41:15.186773+0000","last_clean_scrub_stamp":"2024-09-10T23:41:15.186773+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:46.447500+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509554,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33232,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338608,"statfs":{"total":96636764160,"available":96602734592,"internally_reserved":0,"allocated":2084864,"data_stored":917649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542257,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33228,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2084864,"data_stored":917649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509553,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33800,"kb_used_data":2616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338040,"statfs":{"total":96636764160,"available":96602152960,"internally_reserved":0,"allocated":2678784,"data_stored":1508017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37388,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2084864,"data_stored":917649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37384,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334456,"statfs":{"total":96636764160,"available":96598482944,"internally_reserved":0,"allocated":2084864,"data_stored":917649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37388,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2084864,"data_stored":917649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509552,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37960,"kb_used_data":2616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333880,"statfs":{"total":96636764160,"available":96597893120,"internally_reserved":0,"allocated":2678784,"data_stored":1508017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37388,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2084864,"data_stored":917649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T23:41:38.332 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-10T23:41:38.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.470+0000 7fbc95e00640 1 -- 172.21.15.94:0/3964565399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc900b1100 msgr2=0x7fbc900b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:38.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.470+0000 7fbc95e00640 1 --2- 172.21.15.94:0/3964565399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc900b1100 0x7fbc900b1590 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fbc900cc2b0 tx=0x7fbc8801ccf0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.470+0000 7fbc95e00640 1 -- 172.21.15.94:0/3964565399 shutdown_connections 2024-09-10T23:41:38.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.470+0000 7fbc95e00640 1 --2- 172.21.15.94:0/3964565399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc900b1100 0x7fbc900b1590 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.470+0000 7fbc95e00640 1 --2- 172.21.15.94:0/3964565399 >> v2:172.21.15.181:3300/0 conn(0x7fbc900ce1e0 0x7fbc900cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.470+0000 7fbc95e00640 1 -- 172.21.15.94:0/3964565399 >> 172.21.15.94:0/3964565399 conn(0x7fbc900d1c60 msgr2=0x7fbc900d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:38.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.470+0000 7fbc95e00640 1 -- 172.21.15.94:0/3964565399 shutdown_connections 2024-09-10T23:41:38.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.470+0000 7fbc95e00640 1 -- 172.21.15.94:0/3964565399 wait complete. 2024-09-10T23:41:38.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc95e00640 1 Processor -- start 2024-09-10T23:41:38.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc95e00640 1 -- start start 2024-09-10T23:41:38.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc95e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc900b1100 0x7fbc901fadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:38.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc95e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc900ce1e0 0x7fbc900af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:38.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc95e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc900ce890 0x7fbc900af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:38.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc95e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc90169110 con 0x7fbc900b1100 2024-09-10T23:41:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc95e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbc900d0a30 con 0x7fbc900ce890 2024-09-10T23:41:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc95e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fbc90169290 con 0x7fbc900ce1e0 2024-09-10T23:41:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc8ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc900b1100 0x7fbc901fadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc8ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc900b1100 0x7fbc901fadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47080/0 (socket says 172.21.15.94:47080) 2024-09-10T23:41:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc8ea00640 1 -- 172.21.15.94:0/1256257120 learned_addr learned my addr 172.21.15.94:0/1256257120 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc8e000640 1 --2- 172.21.15.94:0/1256257120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc900ce1e0 0x7fbc900af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc8f400640 1 --2- 172.21.15.94:0/1256257120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc900ce890 0x7fbc900af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc8ea00640 1 -- 172.21.15.94:0/1256257120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc900ce1e0 msgr2=0x7fbc900af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc8ea00640 1 --2- 172.21.15.94:0/1256257120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc900ce1e0 0x7fbc900af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc8ea00640 1 -- 172.21.15.94:0/1256257120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc900ce890 msgr2=0x7fbc900af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc8ea00640 1 --2- 172.21.15.94:0/1256257120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc900ce890 0x7fbc900af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc8ea00640 1 -- 172.21.15.94:0/1256257120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc90156cd0 con 0x7fbc900b1100 2024-09-10T23:41:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc8ea00640 1 --2- 172.21.15.94:0/1256257120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc900b1100 0x7fbc901fadd0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000b410 tx=0x7fbc8000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:38.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc87400640 1 -- 172.21.15.94:0/1256257120 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc80015030 con 0x7fbc900b1100 2024-09-10T23:41:38.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc87400640 1 -- 172.21.15.94:0/1256257120 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc80004020 con 0x7fbc900b1100 2024-09-10T23:41:38.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc87400640 1 -- 172.21.15.94:0/1256257120 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc800044b0 con 0x7fbc900b1100 2024-09-10T23:41:38.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc95e00640 1 -- 172.21.15.94:0/1256257120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc90155530 con 0x7fbc900b1100 2024-09-10T23:41:38.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.474+0000 7fbc95e00640 1 -- 172.21.15.94:0/1256257120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc90157250 con 0x7fbc900b1100 2024-09-10T23:41:38.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.478+0000 7fbc95e00640 1 -- 172.21.15.94:0/1256257120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc900b1590 con 0x7fbc900b1100 2024-09-10T23:41:38.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.478+0000 7fbc87400640 1 -- 172.21.15.94:0/1256257120 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7fbc80022060 con 0x7fbc900b1100 2024-09-10T23:41:38.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.478+0000 7fbc87400640 1 --2- 172.21.15.94:0/1256257120 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbc700731b0 0x7fbc70075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:38.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.478+0000 7fbc87400640 1 -- 172.21.15.94:0/1256257120 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7fbc800943a0 con 0x7fbc900b1100 2024-09-10T23:41:38.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.478+0000 7fbc8e000640 1 --2- 172.21.15.94:0/1256257120 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbc700731b0 0x7fbc70075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:38.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.478+0000 7fbc8e000640 1 --2- 172.21.15.94:0/1256257120 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbc700731b0 0x7fbc70075670 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbc90071020 tx=0x7fbc7c00b390 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:38.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.482+0000 7fbc87400640 1 -- 172.21.15.94:0/1256257120 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbc8005e140 con 0x7fbc900b1100 2024-09-10T23:41:38.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.650+0000 7fbc95e00640 1 -- 172.21.15.94:0/1256257120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fbc900b06f0 con 0x7fbc900b1100 2024-09-10T23:41:38.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.654+0000 7fbc87400640 1 -- 172.21.15.94:0/1256257120 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v50)=0 v50) ==== 123+0+0 (secure 0 0 0) 0x7fbc80062b70 con 0x7fbc900b1100 2024-09-10T23:41:38.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.654+0000 7fbc87400640 1 -- 172.21.15.94:0/1256257120 <== mon.0 v2:172.21.15.94:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fbc8005dc50 con 0x7fbc900b1100 2024-09-10T23:41:38.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.658+0000 7fbc95e00640 1 -- 172.21.15.94:0/1256257120 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbc700731b0 msgr2=0x7fbc70075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:38.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.658+0000 7fbc95e00640 1 --2- 172.21.15.94:0/1256257120 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbc700731b0 0x7fbc70075670 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbc90071020 tx=0x7fbc7c00b390 comp rx=0 tx=0).stop 2024-09-10T23:41:38.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.658+0000 7fbc95e00640 1 -- 172.21.15.94:0/1256257120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc900b1100 msgr2=0x7fbc901fadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:38.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.658+0000 7fbc95e00640 1 --2- 172.21.15.94:0/1256257120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc900b1100 0x7fbc901fadd0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000b410 tx=0x7fbc8000b8d0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.658+0000 7fbc95e00640 1 -- 172.21.15.94:0/1256257120 shutdown_connections 2024-09-10T23:41:38.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.658+0000 7fbc95e00640 1 --2- 172.21.15.94:0/1256257120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc900ce890 0x7fbc900af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.658+0000 7fbc95e00640 1 --2- 172.21.15.94:0/1256257120 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbc700731b0 0x7fbc70075670 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.658+0000 7fbc95e00640 1 --2- 172.21.15.94:0/1256257120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc900ce1e0 0x7fbc900af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.658+0000 7fbc95e00640 1 --2- 172.21.15.94:0/1256257120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc900b1100 0x7fbc901fadd0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.658+0000 7fbc95e00640 1 -- 172.21.15.94:0/1256257120 >> 172.21.15.94:0/1256257120 conn(0x7fbc900d1c60 msgr2=0x7fbc900d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:38.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.658+0000 7fbc95e00640 1 -- 172.21.15.94:0/1256257120 shutdown_connections 2024-09-10T23:41:38.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.658+0000 7fbc95e00640 1 -- 172.21.15.94:0/1256257120 wait complete. 2024-09-10T23:41:38.678 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T23:41:38.678 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T23:41:38.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.818+0000 7fbf36200640 1 -- 172.21.15.94:0/1921488374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf300b2bb0 msgr2=0x7fbf300d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:38.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.818+0000 7fbf36200640 1 --2- 172.21.15.94:0/1921488374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf300b2bb0 0x7fbf300d00a0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009900 tx=0x7fbf1801c7c0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.818+0000 7fbf36200640 1 -- 172.21.15.94:0/1921488374 shutdown_connections 2024-09-10T23:41:38.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.818+0000 7fbf36200640 1 --2- 172.21.15.94:0/1921488374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf300b2bb0 0x7fbf300d00a0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.818+0000 7fbf36200640 1 --2- 172.21.15.94:0/1921488374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf3015f5c0 0x7fbf300cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.818+0000 7fbf36200640 1 -- 172.21.15.94:0/1921488374 >> 172.21.15.94:0/1921488374 conn(0x7fbf300d1c60 msgr2=0x7fbf300d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:38.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.818+0000 7fbf36200640 1 -- 172.21.15.94:0/1921488374 shutdown_connections 2024-09-10T23:41:38.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.818+0000 7fbf36200640 1 -- 172.21.15.94:0/1921488374 wait complete. 2024-09-10T23:41:38.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf36200640 1 Processor -- start 2024-09-10T23:41:38.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf36200640 1 -- start start 2024-09-10T23:41:38.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf36200640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf300b2bb0 0x7fbf301489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:38.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf36200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf300d05e0 0x7fbf3015ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:38.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf36200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf3015f5c0 0x7fbf30151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:38.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf36200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbf301692b0 con 0x7fbf300d05e0 2024-09-10T23:41:38.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf36200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbf30169130 con 0x7fbf3015f5c0 2024-09-10T23:41:38.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf36200640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 -- 0x7fbf30169430 con 0x7fbf300b2bb0 2024-09-10T23:41:38.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf27e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf300d05e0 0x7fbf3015ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:38.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf27e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf300d05e0 0x7fbf3015ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47100/0 (socket says 172.21.15.94:47100) 2024-09-10T23:41:38.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf27e00640 1 -- 172.21.15.94:0/2951130631 learned_addr learned my addr 172.21.15.94:0/2951130631 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T23:41:38.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf2f400640 1 --2- 172.21.15.94:0/2951130631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf300b2bb0 0x7fbf301489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:38.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf2fe00640 1 --2- 172.21.15.94:0/2951130631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf3015f5c0 0x7fbf30151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:38.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf27e00640 1 -- 172.21.15.94:0/2951130631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf300b2bb0 msgr2=0x7fbf301489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:38.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf27e00640 1 --2- 172.21.15.94:0/2951130631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf300b2bb0 0x7fbf301489e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf27e00640 1 -- 172.21.15.94:0/2951130631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf3015f5c0 msgr2=0x7fbf30151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:38.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf27e00640 1 --2- 172.21.15.94:0/2951130631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf3015f5c0 0x7fbf30151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:38.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf27e00640 1 -- 172.21.15.94:0/2951130631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf301f8020 con 0x7fbf300d05e0 2024-09-10T23:41:38.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf27e00640 1 --2- 172.21.15.94:0/2951130631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf300d05e0 0x7fbf3015ba30 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fbf1801a3f0 tx=0x7fbf180a6510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:38.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf27400640 1 -- 172.21.15.94:0/2951130631 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf180ac070 con 0x7fbf300d05e0 2024-09-10T23:41:38.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf27400640 1 -- 172.21.15.94:0/2951130631 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fbf18004330 con 0x7fbf300d05e0 2024-09-10T23:41:38.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf27400640 1 -- 172.21.15.94:0/2951130631 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18005320 con 0x7fbf300d05e0 2024-09-10T23:41:38.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf36200640 1 -- 172.21.15.94:0/2951130631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3015c4d0 con 0x7fbf300d05e0 2024-09-10T23:41:38.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.822+0000 7fbf36200640 1 -- 172.21.15.94:0/2951130631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf301f85a0 con 0x7fbf300d05e0 2024-09-10T23:41:38.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.826+0000 7fbf36200640 1 -- 172.21.15.94:0/2951130631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf300b15c0 con 0x7fbf300d05e0 2024-09-10T23:41:38.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.826+0000 7fbf27400640 1 -- 172.21.15.94:0/2951130631 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) ==== 97465+0+0 (secure 0 0 0) 0x7fbf180c2060 con 0x7fbf300d05e0 2024-09-10T23:41:38.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.826+0000 7fbf27400640 1 --2- 172.21.15.94:0/2951130631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbf0c0731e0 0x7fbf0c0756a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:41:38.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.826+0000 7fbf27400640 1 -- 172.21.15.94:0/2951130631 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4971+0+0 (secure 0 0 0) 0x7fbf1812c050 con 0x7fbf300d05e0 2024-09-10T23:41:38.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.826+0000 7fbf2f400640 1 --2- 172.21.15.94:0/2951130631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbf0c0731e0 0x7fbf0c0756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:41:38.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.826+0000 7fbf2f400640 1 --2- 172.21.15.94:0/2951130631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbf0c0731e0 0x7fbf0c0756a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbf20004630 tx=0x7fbf2000b2e0 comp rx=0 tx=0).ready entity=mgr.4146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:41:38.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:38.830+0000 7fbf27400640 1 -- 172.21.15.94:0/2951130631 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbf180b6070 con 0x7fbf300d05e0 2024-09-10T23:41:39.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:39.030+0000 7fbf36200640 1 -- 172.21.15.94:0/2951130631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf300749c0 con 0x7fbf300d05e0 2024-09-10T23:41:39.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:39.030+0000 7fbf27400640 1 -- 172.21.15.94:0/2951130631 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) ==== 95+0+3922 (secure 0 0 0) 0x7fbf180164d0 con 0x7fbf300d05e0 2024-09-10T23:41:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T23:41:39.031 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":119,"btime":"2024-09-10T23:41:06:523530+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6801/2653759764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2653759764},{"type":"v1","addr":"172.21.15.181:6801","nonce":2653759764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":8518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2240199080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2240199080},{"type":"v1","addr":"172.21.15.181:6835","nonce":2240199080}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":9168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6834/1675026105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1675026105},{"type":"v1","addr":"172.21.15.94:6834","nonce":1675026105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":9465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1503769243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6833","nonce":1503769243},{"type":"v1","addr":"172.21.15.94:6835","nonce":1503769243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":119}],"filesystems":[]} 2024-09-10T23:41:39.031 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 119 2024-09-10T23:41:39.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:39.030+0000 7fbf36200640 1 -- 172.21.15.94:0/2951130631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbf0c0731e0 msgr2=0x7fbf0c0756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:39.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:39.030+0000 7fbf36200640 1 --2- 172.21.15.94:0/2951130631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbf0c0731e0 0x7fbf0c0756a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbf20004630 tx=0x7fbf2000b2e0 comp rx=0 tx=0).stop 2024-09-10T23:41:39.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:39.030+0000 7fbf36200640 1 -- 172.21.15.94:0/2951130631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf300d05e0 msgr2=0x7fbf3015ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:41:39.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:39.030+0000 7fbf36200640 1 --2- 172.21.15.94:0/2951130631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf300d05e0 0x7fbf3015ba30 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fbf1801a3f0 tx=0x7fbf180a6510 comp rx=0 tx=0).stop 2024-09-10T23:41:39.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:39.034+0000 7fbf36200640 1 -- 172.21.15.94:0/2951130631 shutdown_connections 2024-09-10T23:41:39.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:39.034+0000 7fbf36200640 1 --2- 172.21.15.94:0/2951130631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf3015f5c0 0x7fbf30151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:39.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:39.034+0000 7fbf36200640 1 --2- 172.21.15.94:0/2951130631 >> [v2:172.21.15.94:6836/2927908830,v1:172.21.15.94:6837/2927908830] conn(0x7fbf0c0731e0 0x7fbf0c0756a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:39.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:39.034+0000 7fbf36200640 1 --2- 172.21.15.94:0/2951130631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf300d05e0 0x7fbf3015ba30 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:39.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:39.034+0000 7fbf36200640 1 --2- 172.21.15.94:0/2951130631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf300b2bb0 0x7fbf301489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:41:39.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:39.034+0000 7fbf36200640 1 -- 172.21.15.94:0/2951130631 >> 172.21.15.94:0/2951130631 conn(0x7fbf300d1c60 msgr2=0x7fbf30070c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:41:39.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:39.034+0000 7fbf36200640 1 -- 172.21.15.94:0/2951130631 shutdown_connections 2024-09-10T23:41:39.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T23:41:39.034+0000 7fbf36200640 1 -- 172.21.15.94:0/2951130631 wait complete. 2024-09-10T23:41:39.052 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-10T23:41:39.052 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-10T23:41:39.052 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T23:41:39.053 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:39.064 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T23:41:39.064 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T23:41:39.064 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:39.119 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T23:41:39.119 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T23:41:39.119 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:39.134 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T23:41:39.134 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T23:41:39.134 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:39.190 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T23:41:39.190 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-10T23:41:39.190 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T23:41:39.190 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:42.750 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-10T23:41:45.292 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T23:41:45.293 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T23:41:45.293 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:45.374 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T23:41:45.374 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T23:41:45.374 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:48.758 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-10T23:41:48.758 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-10T23:41:51.475 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T23:41:51.476 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-10T23:41:51.476 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:51.544 INFO:tasks.ceph.osd.3:Stopped 2024-09-10T23:41:51.544 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-10T23:41:51.544 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:51.637 INFO:tasks.ceph.osd.4:Stopped 2024-09-10T23:41:51.637 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-10T23:41:51.637 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:51.738 INFO:tasks.ceph.osd.5:Stopped 2024-09-10T23:41:51.738 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-10T23:41:51.738 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:54.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-10T23:41:54.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-10T23:41:57.840 INFO:tasks.ceph.osd.6:Stopped 2024-09-10T23:41:57.840 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-10T23:41:57.840 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:57.898 INFO:tasks.ceph.osd.7:Stopped 2024-09-10T23:41:57.898 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-10T23:41:57.899 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-10T23:41:57.899 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:57.967 INFO:tasks.ceph.mgr.y:Stopped 2024-09-10T23:41:57.967 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-10T23:41:57.967 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:58.029 INFO:tasks.ceph.mgr.x:Stopped 2024-09-10T23:41:58.029 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-10T23:41:58.029 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T23:41:58.029 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:58.049 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T23:41:58.049 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T23:41:58.050 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:58.066 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T23:41:58.066 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T23:41:58.066 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T23:41:58.129 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T23:41:58.129 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-10T23:41:58.129 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-09-10T23:41:58.169 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:41:58.169 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-10T23:41:58.449 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:41:58.449 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-10T23:41:58.675 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:41:58.675 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-10T23:41:58.899 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:41:58.899 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-10T23:41:59.091 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:41:59.091 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-10T23:41:59.341 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:41:59.341 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-10T23:41:59.364 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-10T23:41:59.587 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:41:59.587 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-10T23:41:59.774 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:41:59.774 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-10T23:41:59.987 INFO:tasks.ceph:Archiving mon data... 2024-09-10T23:41:59.988 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899790/data/mon.a.tgz 2024-09-10T23:41:59.988 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-09-10T23:41:59.997 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.cXEfnuzHkr 2024-09-10T23:41:59.998 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.cXEfnuzHkr 2024-09-10T23:42:00.234 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.cXEfnuzHkr 2024-09-10T23:42:00.312 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.cXEfnuzHkr is 1MB 2024-09-10T23:42:00.383 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.cXEfnuzHkr 2024-09-10T23:42:00.392 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899790/data/mon.b.tgz 2024-09-10T23:42:00.392 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2024-09-10T23:42:00.400 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.nH0u4KBSEs 2024-09-10T23:42:00.400 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.nH0u4KBSEs 2024-09-10T23:42:00.636 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.nH0u4KBSEs 2024-09-10T23:42:00.715 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.nH0u4KBSEs is 1MB 2024-09-10T23:42:00.786 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.nH0u4KBSEs 2024-09-10T23:42:00.794 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899790/data/mon.c.tgz 2024-09-10T23:42:00.794 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2024-09-10T23:42:00.843 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.WaRF1mfwQk 2024-09-10T23:42:00.843 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.WaRF1mfwQk 2024-09-10T23:42:01.079 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.WaRF1mfwQk 2024-09-10T23:42:01.151 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.WaRF1mfwQk is 1MB 2024-09-10T23:42:01.230 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.WaRF1mfwQk 2024-09-10T23:42:01.239 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-10T23:42:01.239 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T23:42:01.242 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T23:42:01.312 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T23:42:01.347 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T23:42:01.347 INFO:tasks.ceph:Archiving crash dumps... 2024-09-10T23:42:01.349 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899790/remote/smithi094/crash 2024-09-10T23:42:01.350 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T23:42:01.376 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899790/remote/smithi181/crash 2024-09-10T23:42:01.377 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T23:42:01.408 INFO:tasks.ceph:Shutting down logrotate 2024-09-10T23:42:01.408 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:42:01.423 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T23:42:01.494 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T23:42:01.514 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T23:42:01.553 INFO:tasks.ceph:Compressing logs... 2024-09-10T23:42:01.553 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T23:42:01.563 DEBUG:teuthology.orchestra.run.smithi181:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T23:42:01.589 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53416.log 2024-09-10T23:42:01.590 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46123.log 2024-09-10T23:42:01.591 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51014.log 2024-09-10T23:42:01.591 INFO:teuthology.orchestra.run.smithi094.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53416.log.gz 2024-09-10T23:42:01.592 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43357.log 2024-09-10T23:42:01.592 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46123.log.gz 2024-09-10T23:42:01.593 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45330.log 2024-09-10T23:42:01.593 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51014.log.gz 2024-09-10T23:42:01.593 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39067.log 2024-09-10T23:42:01.594 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43357.log.gz 2024-09-10T23:42:01.595 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36942.log 2024-09-10T23:42:01.595 INFO:teuthology.orchestra.run.smithi094.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45330.log.gz 2024-09-10T23:42:01.595 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47346.log 2024-09-10T23:42:01.595 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39067.log.gz 2024-09-10T23:42:01.596 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45254.log 2024-09-10T23:42:01.596 INFO:teuthology.orchestra.run.smithi094.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36942.log.gz 2024-09-10T23:42:01.597 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47891.log 2024-09-10T23:42:01.597 INFO:teuthology.orchestra.run.smithi094.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.47346.log.gz 2024-09-10T23:42:01.598 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54203.log 2024-09-10T23:42:01.598 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45254.log.gz 2024-09-10T23:42:01.599 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34385.log 2024-09-10T23:42:01.599 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47891.log.gz 2024-09-10T23:42:01.599 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51801.log 2024-09-10T23:42:01.599 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54203.log.gz 2024-09-10T23:42:01.600 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51378.log 2024-09-10T23:42:01.600 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34385.log.gz 2024-09-10T23:42:01.601 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42923.log 2024-09-10T23:42:01.601 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51801.log.gz 2024-09-10T23:42:01.601 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43972.log 2024-09-10T23:42:01.602 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51378.log.gz 2024-09-10T23:42:01.602 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41363.log 2024-09-10T23:42:01.602 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42923.log.gz 2024-09-10T23:42:01.603 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38804.log 2024-09-10T23:42:01.603 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43972.log.gz 2024-09-10T23:42:01.604 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38882.log 2024-09-10T23:42:01.604 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41363.log.gz 2024-09-10T23:42:01.605 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38804.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38804.log.gz 2024-09-10T23:42:01.605 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34099.log 2024-09-10T23:42:01.606 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36812.log 2024-09-10T23:42:01.606 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38882.log.gz 2024-09-10T23:42:01.606 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51066.log 2024-09-10T23:42:01.607 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34099.log.gz 2024-09-10T23:42:01.607 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48178.log 2024-09-10T23:42:01.608 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36812.log.gz 2024-09-10T23:42:01.608 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52035.log 2024-09-10T23:42:01.608 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51066.log.gz 2024-09-10T23:42:01.609 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33965.log 2024-09-10T23:42:01.609 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48178.log.gz 2024-09-10T23:42:01.609 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54281.log 2024-09-10T23:42:01.610 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33965.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52035.log.gz 2024-09-10T23:42:01.610 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52826.log 2024-09-10T23:42:01.611 INFO:teuthology.orchestra.run.smithi094.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33965.log.gz 2024-09-10T23:42:01.611 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49371.log 2024-09-10T23:42:01.611 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54281.log.gz 2024-09-10T23:42:01.612 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36429.log 2024-09-10T23:42:01.613 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52826.log.gz 2024-09-10T23:42:01.613 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53700.log 2024-09-10T23:42:01.613 INFO:teuthology.orchestra.run.smithi094.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.49371.log.gz 2024-09-10T23:42:01.614 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36429.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-10T23:42:01.614 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36429.log.gz 2024-09-10T23:42:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43305.log 2024-09-10T23:42:01.615 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53700.log.gz 2024-09-10T23:42:01.615 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37648.log 2024-09-10T23:42:01.616 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41298.log 2024-09-10T23:42:01.616 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43305.log.gz 2024-09-10T23:42:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44810.log 2024-09-10T23:42:01.617 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37648.log.gz 2024-09-10T23:42:01.618 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37518.log 2024-09-10T23:42:01.618 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41298.log.gz 2024-09-10T23:42:01.619 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50403.log 2024-09-10T23:42:01.620 INFO:teuthology.orchestra.run.smithi094.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44810.log.gz 2024-09-10T23:42:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36573.log 2024-09-10T23:42:01.620 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37518.log.gz 2024-09-10T23:42:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46383.log 2024-09-10T23:42:01.621 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50403.log.gz 2024-09-10T23:42:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52640.log 87.7% 2024-09-10T23:42:01.621 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36573.log.gz 2024-09-10T23:42:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39125.log 2024-09-10T23:42:01.623 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46383.log.gz 2024-09-10T23:42:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46201.log 2024-09-10T23:42:01.623 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52640.log.gz 2024-09-10T23:42:01.624 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37941.log 2024-09-10T23:42:01.624 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39125.log.gz 2024-09-10T23:42:01.625 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46201.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46201.log.gz -5 2024-09-10T23:42:01.625 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43589.log 2024-09-10T23:42:01.626 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43069.log 2024-09-10T23:42:01.626 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37941.log.gz 2024-09-10T23:42:01.626 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36732.log 2024-09-10T23:42:01.627 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43589.log.gz 2024-09-10T23:42:01.627 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50760.log 2024-09-10T23:42:01.628 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43069.log.gz 2024-09-10T23:42:01.628 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40221.log 2024-09-10T23:42:01.629 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36732.log.gz 2024-09-10T23:42:01.629 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41442.log 2024-09-10T23:42:01.629 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50760.log.gz 2024-09-10T23:42:01.630 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39870.log 2024-09-10T23:42:01.630 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40221.log.gz 2024-09-10T23:42:01.631 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40928.log 2024-09-10T23:42:01.631 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41442.log.gz 2024-09-10T23:42:01.632 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52390.log 2024-09-10T23:42:01.632 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39870.log.gz 2024-09-10T23:42:01.633 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36081.log 2024-09-10T23:42:01.633 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40928.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40928.log.gz 2024-09-10T23:42:01.634 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54665.log 2024-09-10T23:42:01.634 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52390.log.gz 2024-09-10T23:42:01.634 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51669.log 2024-09-10T23:42:01.635 INFO:teuthology.orchestra.run.smithi094.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36081.log.gz 2024-09-10T23:42:01.635 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40195.log 2024-09-10T23:42:01.635 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54665.log.gz 2024-09-10T23:42:01.636 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46435.log 2024-09-10T23:42:01.636 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51669.log.gz 2024-09-10T23:42:01.637 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46908.log 2024-09-10T23:42:01.637 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40195.log.gz 2024-09-10T23:42:01.638 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40273.log 2024-09-10T23:42:01.638 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46435.log.gz 2024-09-10T23:42:01.640 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43383.log 2024-09-10T23:42:01.640 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46908.log.gz 2024-09-10T23:42:01.640 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39601.log 2024-09-10T23:42:01.641 INFO:teuthology.orchestra.run.smithi094.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40273.log.gz 2024-09-10T23:42:01.641 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47631.log 2024-09-10T23:42:01.641 INFO:teuthology.orchestra.run.smithi094.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43383.log.gz 2024-09-10T23:42:01.642 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36293.log 2024-09-10T23:42:01.642 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39601.log.gz 2024-09-10T23:42:01.642 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49271.log 2024-09-10T23:42:01.643 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47631.log.gz 2024-09-10T23:42:01.644 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36293.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36293.log.gz 2024-09-10T23:42:01.644 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49512.log 2024-09-10T23:42:01.644 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39418.log 2024-09-10T23:42:01.645 INFO:teuthology.orchestra.run.smithi094.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.49271.log.gz 2024-09-10T23:42:01.645 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50220.log 2024-09-10T23:42:01.645 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49512.log.gz 2024-09-10T23:42:01.646 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54561.log 2024-09-10T23:42:01.646 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39418.log.gz 2024-09-10T23:42:01.647 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45818.log 2024-09-10T23:42:01.647 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50220.log.gz 2024-09-10T23:42:01.647 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50663.log 2024-09-10T23:42:01.648 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54561.log.gz 2024-09-10T23:42:01.648 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47943.log 2024-09-10T23:42:01.649 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36046.log 2024-09-10T23:42:01.650 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45818.log.gz 2024-09-10T23:42:01.650 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42186.log 2024-09-10T23:42:01.650 INFO:teuthology.orchestra.run.smithi094.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50663.log.gz 2024-09-10T23:42:01.650 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46934.log 2024-09-10T23:42:01.654 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47943.log.gz 2024-09-10T23:42:01.654 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37372.log 2024-09-10T23:42:01.654 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42186.log.gz 2024-09-10T23:42:01.655 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46934.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-10T23:42:01.655 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46934.log.gz 2024-09-10T23:42:01.655 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37465.log 2024-09-10T23:42:01.655 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37372.log.gz 2024-09-10T23:42:01.655 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48470.log 2024-09-10T23:42:01.655 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28969.log 2024-09-10T23:42:01.655 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37465.log.gz 2024-09-10T23:42:01.657 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35684.log 2024-09-10T23:42:01.657 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.36046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34643.log 2024-09-10T23:42:01.657 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.35684.log: 89.8% -- replaced with /var/log/ceph/ceph-client.1.36046.log.gz 2024-09-10T23:42:01.657 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34244.log 2024-09-10T23:42:01.657 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.34643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36226.log 2024-09-10T23:42:01.657 INFO:teuthology.orchestra.run.smithi181.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.1.35684.log.gz 2024-09-10T23:42:01.657 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.3.34244.log: 87.7% -- replaced with /var/log/ceph/ceph-client.1.34643.log.gz 2024-09-10T23:42:01.657 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-10T23:42:01.657 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.3.34244.log.gz 2024-09-10T23:42:01.658 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.36226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35501.log 2024-09-10T23:42:01.658 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.7.log: 89.5% -- replaced with /var/log/ceph/ceph-client.1.36226.log.gz 2024-09-10T23:42:01.658 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-10T23:42:01.658 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.35501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29109.log 2024-09-10T23:42:01.658 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28964.log 2024-09-10T23:42:01.658 INFO:teuthology.orchestra.run.smithi181.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.1.35501.log.gz 2024-09-10T23:42:01.658 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.29109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29085.log 2024-09-10T23:42:01.658 INFO:teuthology.orchestra.run.smithi181.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29109.log.gz 2024-09-10T23:42:01.663 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.28964.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.28964.log.gz 2024-09-10T23:42:01.663 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-10T23:42:01.663 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.29085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35085.log 2024-09-10T23:42:01.663 INFO:teuthology.orchestra.run.smithi181.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29085.log.gz 2024-09-10T23:42:01.663 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-10T23:42:01.663 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.35085.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T23:42:01.663 INFO:teuthology.orchestra.run.smithi181.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.1.35085.log.gz 2024-09-10T23:42:01.663 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35866.log 2024-09-10T23:42:01.663 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29036.log 2024-09-10T23:42:01.667 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.35866.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-10T23:42:01.667 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.29036.log: 89.5% -- replaced with /var/log/ceph/ceph-client.1.35866.log.gz 2024-09-10T23:42:01.667 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34131.log 2024-09-10T23:42:01.667 INFO:teuthology.orchestra.run.smithi181.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29036.log.gz 2024-09-10T23:42:01.667 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mon.c.log: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T23:42:01.667 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T23:42:01.667 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36394.log 2024-09-10T23:42:01.670 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43433.log 2024-09-10T23:42:01.670 INFO:teuthology.orchestra.run.smithi094.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.48470.log.gz 2024-09-10T23:42:01.670 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.28969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49590.log 2024-09-10T23:42:01.670 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28969.log.gz 2024-09-10T23:42:01.670 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34712.log 2024-09-10T23:42:01.670 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43433.log.gz 2024-09-10T23:42:01.670 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44024.log 2024-09-10T23:42:01.670 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49590.log.gz 2024-09-10T23:42:01.670 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53883.log 2024-09-10T23:42:01.670 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34712.log.gz 2024-09-10T23:42:01.671 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38173.log 2024-09-10T23:42:01.671 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44024.log.gz 2024-09-10T23:42:01.671 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48522.log 2024-09-10T23:42:01.671 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53883.log.gz 2024-09-10T23:42:01.671 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36371.log 2024-09-10T23:42:01.671 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38173.log.gz 2024-09-10T23:42:01.671 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48522.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48522.log.gz 2024-09-10T23:42:01.671 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51118.log 2024-09-10T23:42:01.671 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49772.log 2024-09-10T23:42:01.671 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36371.log.gz 2024-09-10T23:42:01.671 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51118.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51118.log.gz 2024-09-10T23:42:01.671 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51827.log 2024-09-10T23:42:01.671 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44992.log 2024-09-10T23:42:01.671 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49772.log.gz 2024-09-10T23:42:01.672 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44495.log 2024-09-10T23:42:01.672 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51827.log.gz 2024-09-10T23:42:01.672 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38277.log 2024-09-10T23:42:01.672 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44992.log.gz 2024-09-10T23:42:01.672 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44495.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44495.log.gz 2024-09-10T23:42:01.672 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46804.log 2024-09-10T23:42:01.672 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49057.log 2024-09-10T23:42:01.672 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38277.log.gz 2024-09-10T23:42:01.672 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52588.log 2024-09-10T23:42:01.672 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46804.log.gz 2024-09-10T23:42:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.34131.log: /var/log/ceph/ceph.audit.log: 87.8% -- replaced with /var/log/ceph/ceph-client.1.34131.log.gz 2024-09-10T23:42:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.36394.log: 88.9% -- replaced with /var/log/ceph/ceph-client.1.36394.log.gz 2024-09-10T23:42:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-10T23:42:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34889.log 2024-09-10T23:42:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.1.34889.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T23:42:01.675 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28988.log 2024-09-10T23:42:01.675 INFO:teuthology.orchestra.run.smithi181.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.1.34889.log.gz 2024-09-10T23:42:01.677 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38038.log 2024-09-10T23:42:01.678 INFO:teuthology.orchestra.run.smithi094.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.49057.log.gz 2024-09-10T23:42:01.678 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52614.log 2024-09-10T23:42:01.678 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52588.log.gz 2024-09-10T23:42:01.678 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48204.log 2024-09-10T23:42:01.678 INFO:teuthology.orchestra.run.smithi094.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38038.log.gz 2024-09-10T23:42:01.678 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52562.log 2024-09-10T23:42:01.678 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52614.log.gz 2024-09-10T23:42:01.678 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28921.log 2024-09-10T23:42:01.678 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48204.log.gz 2024-09-10T23:42:01.678 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41654.log 2024-09-10T23:42:01.678 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52562.log.gz 2024-09-10T23:42:01.678 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.28921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34913.log 2024-09-10T23:42:01.678 INFO:teuthology.orchestra.run.smithi094.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.28921.log.gz 2024-09-10T23:42:01.679 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44349.log 2024-09-10T23:42:01.679 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41654.log.gz 2024-09-10T23:42:01.681 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28933.log 2024-09-10T23:42:01.681 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.28988.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28988.log.gz 2024-09-10T23:42:01.681 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.28933.log: -- /var/log/ceph/ceph-mds.b.log 2024-09-10T23:42:01.681 INFO:teuthology.orchestra.run.smithi181.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.28933.log.gz 2024-09-10T23:42:01.681 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-10T23:42:01.683 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33939.log 2024-09-10T23:42:01.683 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34913.log.gz 2024-09-10T23:42:01.683 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35243.log 2024-09-10T23:42:01.683 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44349.log.gz 2024-09-10T23:42:01.683 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35651.log 2024-09-10T23:42:01.683 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33939.log.gz 2024-09-10T23:42:01.683 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46778.log 2024-09-10T23:42:01.683 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35243.log.gz 2024-09-10T23:42:01.683 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.2.35651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40902.log 2024-09-10T23:42:01.683 INFO:teuthology.orchestra.run.smithi094.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.2.35651.log.gz 2024-09-10T23:42:01.684 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-10T23:42:01.685 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46500.log 2024-09-10T23:42:01.685 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46778.log.gz 2024-09-10T23:42:01.685 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40902.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40902.log.gz 2024-09-10T23:42:01.685 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43201.log 2024-09-10T23:42:01.686 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47709.log 2024-09-10T23:42:01.686 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46500.log.gz 2024-09-10T23:42:01.687 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34425.log 2024-09-10T23:42:01.687 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39523.log 2024-09-10T23:42:01.687 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43201.log.gz 2024-09-10T23:42:01.688 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49007.log 2024-09-10T23:42:01.688 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47709.log.gz 2024-09-10T23:42:01.690 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29012.log 2024-09-10T23:42:01.690 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42431.log 2024-09-10T23:42:01.690 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39523.log.gz 2024-09-10T23:42:01.690 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34463.log 2024-09-10T23:42:01.690 INFO:teuthology.orchestra.run.smithi094.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.49007.log.gz 2024-09-10T23:42:01.690 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42431.log: gzip -5 --verbose -- 87.6% /var/log/ceph/ceph-client.admin.51560.log -- replaced with /var/log/ceph/ceph-client.admin.42431.log.gz 2024-09-10T23:42:01.690 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-10T23:42:01.692 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54717.log 2024-09-10T23:42:01.692 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34463.log.gz 2024-09-10T23:42:01.692 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48982.log 2024-09-10T23:42:01.692 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51560.log.gz 2024-09-10T23:42:01.693 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44076.log 2024-09-10T23:42:01.693 INFO:teuthology.orchestra.run.smithi094.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54717.log.gz 2024-09-10T23:42:01.694 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48982.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.48982.log.gz 2024-09-10T23:42:01.694 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49903.log 2024-09-10T23:42:01.694 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.34425.log: /var/log/ceph/ceph-client.admin.29012.log: 88.3% -- replaced with /var/log/ceph/ceph-client.1.34425.log.gz 2024-09-10T23:42:01.695 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54795.log 2024-09-10T23:42:01.695 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44076.log.gz 2024-09-10T23:42:01.697 INFO:teuthology.orchestra.run.smithi181.stderr: 86.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35320.log 2024-09-10T23:42:01.697 INFO:teuthology.orchestra.run.smithi181.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29012.log.gz 2024-09-10T23:42:01.698 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50377.log 2024-09-10T23:42:01.698 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49903.log.gz 2024-09-10T23:42:01.698 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36189.log 2024-09-10T23:42:01.698 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54795.log.gz 2024-09-10T23:42:01.698 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48728.log 2024-09-10T23:42:01.698 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50377.log.gz 2024-09-10T23:42:01.699 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44784.log 2024-09-10T23:42:01.699 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36189.log.gz 2024-09-10T23:42:01.699 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48728.log: gzip 87.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.48728.log.gz --verbose 2024-09-10T23:42:01.700 INFO:teuthology.orchestra.run.smithi094.stderr: -- /var/log/ceph/ceph-client.admin.42832.log 2024-09-10T23:42:01.700 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29060.log 2024-09-10T23:42:01.701 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54431.log 2024-09-10T23:42:01.701 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44784.log.gz 2024-09-10T23:42:01.701 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46409.log 2024-09-10T23:42:01.702 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42832.log.gz 2024-09-10T23:42:01.702 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.35320.log: 89.3% -- replaced with /var/log/ceph/ceph-client.1.35320.log.gz 2024-09-10T23:42:01.703 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39177.log 2024-09-10T23:42:01.703 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54431.log.gz 2024-09-10T23:42:01.703 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36994.log 2024-09-10T23:42:01.704 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46409.log.gz 2024-09-10T23:42:01.704 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39177.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39177.log.gz 2024-09-10T23:42:01.704 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41207.log 2024-09-10T23:42:01.705 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48314.log 2024-09-10T23:42:01.706 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36994.log.gz 2024-09-10T23:42:01.706 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.29060.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29060.log.gz 2024-09-10T23:42:01.707 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34905.log 2024-09-10T23:42:01.707 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41207.log.gz 2024-09-10T23:42:01.707 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52930.log 2024-09-10T23:42:01.708 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48314.log.gz 2024-09-10T23:42:01.708 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34905.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34905.log.gz 2024-09-10T23:42:01.708 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45381.log 2024-09-10T23:42:01.709 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47657.log 2024-09-10T23:42:01.709 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52930.log.gz 2024-09-10T23:42:01.710 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35165.log 2024-09-10T23:42:01.710 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45381.log.gz 2024-09-10T23:42:01.711 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47787.log 2024-09-10T23:42:01.711 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47657.log.gz 2024-09-10T23:42:01.712 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54078.log 2024-09-10T23:42:01.712 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35165.log.gz 2024-09-10T23:42:01.713 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38303.log 2024-09-10T23:42:01.713 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47787.log.gz 2024-09-10T23:42:01.714 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52141.log 2024-09-10T23:42:01.714 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54078.log.gz 2024-09-10T23:42:01.715 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47256.log 2024-09-10T23:42:01.715 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38303.log.gz 2024-09-10T23:42:01.715 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.0.52141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49798.log 2024-09-10T23:42:01.717 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47256.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47256.log.gz 2024-09-10T23:42:01.717 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29017.log 2024-09-10T23:42:01.718 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35139.log 2024-09-10T23:42:01.718 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49798.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49798.log.gz 2024-09-10T23:42:01.719 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38694.log 2024-09-10T23:42:01.719 INFO:teuthology.orchestra.run.smithi094.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29017.log.gz 2024-09-10T23:42:01.720 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53195.log 2024-09-10T23:42:01.720 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35139.log.gz 2024-09-10T23:42:01.721 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35942.log 2024-09-10T23:42:01.721 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38694.log.gz 2024-09-10T23:42:01.722 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36345.log 2024-09-10T23:42:01.722 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53195.log.gz 2024-09-10T23:42:01.723 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46331.log 2024-09-10T23:42:01.723 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35942.log.gz 2024-09-10T23:42:01.724 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53468.log 2024-09-10T23:42:01.724 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36345.log.gz 2024-09-10T23:42:01.725 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41891.log 2024-09-10T23:42:01.725 INFO:teuthology.orchestra.run.smithi094.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.0.52141.log.gz 2024-09-10T23:42:01.726 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46331.log.gz 2024-09-10T23:42:01.726 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37544.log 2024-09-10T23:42:01.727 INFO:teuthology.orchestra.run.smithi094.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.53468.log.gz 2024-09-10T23:42:01.727 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.0.41891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53312.log 2024-09-10T23:42:01.727 INFO:teuthology.orchestra.run.smithi094.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.0.41891.log.gz 2024-09-10T23:42:01.728 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52440.log 2024-09-10T23:42:01.728 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37544.log.gz 2024-09-10T23:42:01.729 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37700.log 2024-09-10T23:42:01.729 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53312.log.gz 2024-09-10T23:42:01.729 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38199.log 2024-09-10T23:42:01.729 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52440.log.gz 2024-09-10T23:42:01.730 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54052.log 2024-09-10T23:42:01.730 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37700.log.gz 2024-09-10T23:42:01.731 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37434.log 2024-09-10T23:42:01.731 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38199.log.gz 2024-09-10T23:42:01.732 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37022.log 2024-09-10T23:42:01.732 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54052.log.gz 2024-09-10T23:42:01.733 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44414.log 2024-09-10T23:42:01.733 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37434.log.gz 2024-09-10T23:42:01.734 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37022.log: gzip 87.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.37022.log.gz --verbose 2024-09-10T23:42:01.734 INFO:teuthology.orchestra.run.smithi094.stderr: -- /var/log/ceph/ceph-client.admin.43641.log 2024-09-10T23:42:01.735 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44414.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.24033.log 2024-09-10T23:42:01.735 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44414.log.gz 2024-09-10T23:42:01.735 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37317.log 2024-09-10T23:42:01.735 INFO:teuthology.orchestra.run.smithi094.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.43641.log.gz 2024-09-10T23:42:01.736 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.tmp-client.admin.24033.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.24033.log.gz 2024-09-10T23:42:01.736 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-10T23:42:01.737 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35321.log 2024-09-10T23:42:01.737 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37317.log.gz 2024-09-10T23:42:01.738 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38856.log 2024-09-10T23:42:01.739 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49538.log 2024-09-10T23:42:01.740 INFO:teuthology.orchestra.run.smithi094.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.35321.log.gz 2024-09-10T23:42:01.740 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48340.log 2024-09-10T23:42:01.740 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38856.log.gz 2024-09-10T23:42:01.741 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35753.log 2024-09-10T23:42:01.741 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49538.log.gz 2024-09-10T23:42:01.742 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34928.log 2024-09-10T23:42:01.742 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48340.log.gz 2024-09-10T23:42:01.743 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53086.log 2024-09-10T23:42:01.743 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35753.log.gz 2024-09-10T23:42:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38830.log 2024-09-10T23:42:01.744 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34928.log.gz 2024-09-10T23:42:01.745 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50637.log 2024-09-10T23:42:01.745 INFO:teuthology.orchestra.run.smithi094.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53086.log.gz 2024-09-10T23:42:01.746 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40039.log 2024-09-10T23:42:01.746 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38830.log.gz 2024-09-10T23:42:01.747 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44206.log 2024-09-10T23:42:01.747 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50637.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50637.log.gz/var/log/ceph/ceph-client.admin.40039.log: 2024-09-10T23:42:01.748 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39015.log 2024-09-10T23:42:01.748 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40039.log.gz 2024-09-10T23:42:01.749 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39660.log 2024-09-10T23:42:01.749 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44206.log.gz 2024-09-10T23:42:01.750 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50194.log 2024-09-10T23:42:01.750 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39015.log.gz 2024-09-10T23:42:01.751 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47761.log 2024-09-10T23:42:01.751 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39660.log.gz 2024-09-10T23:42:01.752 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50059.log 2024-09-10T23:42:01.752 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50194.log.gz 2024-09-10T23:42:01.753 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49428.log 2024-09-10T23:42:01.753 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47761.log.gz 2024-09-10T23:42:01.754 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36599.log 2024-09-10T23:42:01.754 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50059.log.gz 2024-09-10T23:42:01.755 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49428.log: gzip 87.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.49428.log.gz --verbose 2024-09-10T23:42:01.755 INFO:teuthology.orchestra.run.smithi094.stderr: -- /var/log/ceph/ceph-client.admin.40247.log 2024-09-10T23:42:01.756 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34509.log 2024-09-10T23:42:01.756 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36599.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36599.log.gz 2024-09-10T23:42:01.757 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43665.log 2024-09-10T23:42:01.757 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40247.log.gz 2024-09-10T23:42:01.758 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49032.log 2024-09-10T23:42:01.758 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34509.log.gz 2024-09-10T23:42:01.759 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42266.log 2024-09-10T23:42:01.759 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43665.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.43665.log.gz 2024-09-10T23:42:01.760 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40540.log 2024-09-10T23:42:01.760 INFO:teuthology.orchestra.run.smithi094.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.49032.log.gz 2024-09-10T23:42:01.761 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28899.log 2024-09-10T23:42:01.761 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42266.log.gz 2024-09-10T23:42:01.762 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-10T23:42:01.762 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40540.log: /var/log/ceph/ceph-client.admin.28899.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40540.log.gz 2024-09-10T23:42:01.763 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 77.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.28899.log.gz -- 2024-09-10T23:42:01.763 INFO:teuthology.orchestra.run.smithi094.stderr: /var/log/ceph/ceph-client.admin.48418.log 2024-09-10T23:42:01.764 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48882.log 2024-09-10T23:42:01.765 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42754.log 2024-09-10T23:42:01.765 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48418.log.gz 2024-09-10T23:42:01.766 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44758.log 2024-09-10T23:42:01.766 INFO:teuthology.orchestra.run.smithi094.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.48882.log.gz 2024-09-10T23:42:01.767 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46017.log 2024-09-10T23:42:01.767 INFO:teuthology.orchestra.run.smithi094.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42754.log.gz 2024-09-10T23:42:01.769 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44758.log: gzip 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44758.log.gz 2024-09-10T23:42:01.769 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54769.log 2024-09-10T23:42:01.770 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46017.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46017.log.gz 2024-09-10T23:42:01.770 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54613.log 2024-09-10T23:42:01.771 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51170.log 2024-09-10T23:42:01.772 INFO:teuthology.orchestra.run.smithi094.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.54769.log.gz 2024-09-10T23:42:01.772 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42728.log 2024-09-10T23:42:01.773 INFO:teuthology.orchestra.run.smithi094.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.54613.log.gz 2024-09-10T23:42:01.773 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39575.log 2024-09-10T23:42:01.774 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51170.log.gz 2024-09-10T23:42:01.775 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43754.log 2024-09-10T23:42:01.775 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42728.log.gz 2024-09-10T23:42:01.776 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43615.log 2024-09-10T23:42:01.776 INFO:teuthology.orchestra.run.smithi094.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.39575.log.gz 2024-09-10T23:42:01.777 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35695.log 2024-09-10T23:42:01.777 INFO:teuthology.orchestra.run.smithi094.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43754.log.gz 2024-09-10T23:42:01.778 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50988.log 2024-09-10T23:42:01.778 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43615.log.gz 2024-09-10T23:42:01.779 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38541.log 2024-09-10T23:42:01.779 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35695.log.gz 2024-09-10T23:42:01.780 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54877.log 2024-09-10T23:42:01.780 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50988.log.gz 2024-09-10T23:42:01.781 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.0.38541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47404.log 2024-09-10T23:42:01.785 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54877.log: 88.6% -- replaced with /var/log/ceph/ceph-client.0.38541.log.gz 2024-09-10T23:42:01.785 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38090.log 2024-09-10T23:42:01.785 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54877.log.gz 2024-09-10T23:42:01.785 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29089.log 2024-09-10T23:42:01.785 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47404.log.gz 2024-09-10T23:42:01.785 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48806.log 2024-09-10T23:42:01.785 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38090.log.gz 2024-09-10T23:42:01.786 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47115.log 2024-09-10T23:42:01.786 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29089.log.gz 2024-09-10T23:42:01.787 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41680.log 2024-09-10T23:42:01.787 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48806.log.gz 2024-09-10T23:42:01.788 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37401.log 2024-09-10T23:42:01.788 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47115.log.gz 2024-09-10T23:42:01.789 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34307.log 2024-09-10T23:42:01.789 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41680.log.gz 2024-09-10T23:42:01.790 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47067.log 2024-09-10T23:42:01.790 INFO:teuthology.orchestra.run.smithi094.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.37401.log.gz 2024-09-10T23:42:01.791 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36652.log 2024-09-10T23:42:01.791 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34307.log.gz 2024-09-10T23:42:01.792 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.0.47067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45176.log 2024-09-10T23:42:01.794 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41550.log 2024-09-10T23:42:01.794 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36652.log.gz 2024-09-10T23:42:01.794 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45176.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45176.log.gz 2024-09-10T23:42:01.795 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50114.log 2024-09-10T23:42:01.796 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41550.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41550.log.gz 2024-09-10T23:42:01.796 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44258.log 2024-09-10T23:42:01.797 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38485.log 2024-09-10T23:42:01.797 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50114.log.gz 2024-09-10T23:42:01.798 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34502.log 2024-09-10T23:42:01.799 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44258.log.gz 2024-09-10T23:42:01.800 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38485.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38485.log.gz 2024-09-10T23:42:01.801 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34502.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34502.log.gz 2024-09-10T23:42:01.801 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42806.log 2024-09-10T23:42:01.802 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45480.log 2024-09-10T23:42:01.804 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38329.log 2024-09-10T23:42:01.804 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42806.log.gz 2024-09-10T23:42:01.805 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45480.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45480.log.gz 2024-09-10T23:42:01.806 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38329.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38329.log.gz 2024-09-10T23:42:01.806 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49486.log 2024-09-10T23:42:01.807 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51879.log 2024-09-10T23:42:01.809 INFO:teuthology.orchestra.run.smithi094.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.0.47067.log.gz 2024-09-10T23:42:01.809 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42310.log 2024-09-10T23:42:01.809 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49486.log.gz 2024-09-10T23:42:01.810 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37973.log 2024-09-10T23:42:01.810 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51879.log.gz 2024-09-10T23:42:01.811 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38119.log 2024-09-10T23:42:01.811 INFO:teuthology.orchestra.run.smithi094.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.42310.log.gz 2024-09-10T23:42:01.812 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42483.log 2024-09-10T23:42:01.812 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37973.log.gz 2024-09-10T23:42:01.813 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41602.log 2024-09-10T23:42:01.813 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38119.log.gz 2024-09-10T23:42:01.814 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39627.log 2024-09-10T23:42:01.815 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42483.log.gz 2024-09-10T23:42:01.816 INFO:teuthology.orchestra.run.smithi181.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-10T23:42:01.816 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50609.log 2024-09-10T23:42:01.816 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41602.log.gz 2024-09-10T23:42:01.816 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47480.log 2024-09-10T23:42:01.817 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39627.log.gz 2024-09-10T23:42:01.818 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49321.log 2024-09-10T23:42:01.818 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50609.log.gz 2024-09-10T23:42:01.819 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34511.log 2024-09-10T23:42:01.819 INFO:teuthology.orchestra.run.smithi094.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.47480.log.gz 2024-09-10T23:42:01.820 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54457.log 2024-09-10T23:42:01.820 INFO:teuthology.orchestra.run.smithi094.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.49321.log.gz 2024-09-10T23:42:01.821 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40954.log 2024-09-10T23:42:01.821 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34511.log.gz 2024-09-10T23:42:01.822 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34437.log 2024-09-10T23:42:01.822 INFO:teuthology.orchestra.run.smithi094.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54457.log.gz 2024-09-10T23:42:01.823 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34255.log 2024-09-10T23:42:01.823 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40954.log.gz 2024-09-10T23:42:01.824 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48624.log 2024-09-10T23:42:01.825 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34437.log.gz 2024-09-10T23:42:01.825 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34047.log 2024-09-10T23:42:01.825 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34255.log.gz 2024-09-10T23:42:01.826 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43894.log 2024-09-10T23:42:01.826 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48624.log.gz 2024-09-10T23:42:01.827 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37830.log 2024-09-10T23:42:01.827 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34047.log.gz 2024-09-10T23:42:01.829 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37048.log 2024-09-10T23:42:01.829 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43894.log.gz 2024-09-10T23:42:01.830 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44050.log 2024-09-10T23:42:01.830 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37830.log.gz 2024-09-10T23:42:01.830 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51437.log 2024-09-10T23:42:01.832 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44050.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.37048.log.gz 2024-09-10T23:42:01.832 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37254.log 2024-09-10T23:42:01.832 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44050.log.gz 2024-09-10T23:42:01.833 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52273.log 2024-09-10T23:42:01.833 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51437.log.gz 2024-09-10T23:42:01.834 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45084.log 2024-09-10T23:42:01.834 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37254.log.gz 2024-09-10T23:42:01.835 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36968.log 2024-09-10T23:42:01.835 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52273.log.gz 2024-09-10T23:42:01.836 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54847.log 2024-09-10T23:42:01.836 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45084.log.gz 2024-09-10T23:42:01.837 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44550.log 2024-09-10T23:42:01.837 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36968.log.gz 2024-09-10T23:42:01.838 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37908.log 2024-09-10T23:42:01.839 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54847.log.gz 2024-09-10T23:42:01.839 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53544.log 2024-09-10T23:42:01.840 INFO:teuthology.orchestra.run.smithi094.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44550.log.gz 2024-09-10T23:42:01.840 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43149.log 2024-09-10T23:42:01.841 INFO:teuthology.orchestra.run.smithi094.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37908.log.gz 2024-09-10T23:42:01.841 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34701.log 2024-09-10T23:42:01.842 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53544.log.gz 2024-09-10T23:42:01.842 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54353.log 2024-09-10T23:42:01.843 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43149.log.gz 2024-09-10T23:42:01.844 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47010.log 2024-09-10T23:42:01.844 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34701.log.gz 2024-09-10T23:42:01.845 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46617.log 2024-09-10T23:42:01.845 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54353.log.gz 2024-09-10T23:42:01.846 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39366.log 2024-09-10T23:42:01.846 INFO:teuthology.orchestra.run.smithi094.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47010.log.gz 2024-09-10T23:42:01.847 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44440.log 2024-09-10T23:42:01.847 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46617.log.gz 2024-09-10T23:42:01.848 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53600.log 2024-09-10T23:42:01.848 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39366.log.gz 2024-09-10T23:42:01.849 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33991.log 2024-09-10T23:42:01.849 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44440.log.gz 2024-09-10T23:42:01.850 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.0.53600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40702.log 2024-09-10T23:42:01.851 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46279.log 2024-09-10T23:42:01.851 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33991.log.gz 2024-09-10T23:42:01.853 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51248.log 2024-09-10T23:42:01.853 INFO:teuthology.orchestra.run.smithi094.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.40702.log.gz 2024-09-10T23:42:01.854 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46279.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46279.log.gz 2024-09-10T23:42:01.855 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51248.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51248.log.gz 2024-09-10T23:42:01.855 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42108.log 2024-09-10T23:42:01.856 INFO:teuthology.orchestra.run.smithi094.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.0.53600.log.gz 2024-09-10T23:42:01.856 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34912.log 2024-09-10T23:42:01.857 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34713.log 2024-09-10T23:42:01.858 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42108.log.gz 2024-09-10T23:42:01.859 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37674.log 2024-09-10T23:42:01.859 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34912.log.gz 2024-09-10T23:42:01.860 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49694.log 2024-09-10T23:42:01.860 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34713.log.gz 2024-09-10T23:42:01.861 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37674.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37674.log.gz 2024-09-10T23:42:01.861 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37154.log 2024-09-10T23:42:01.862 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33276.log 2024-09-10T23:42:01.863 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49694.log.gz 2024-09-10T23:42:01.863 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.0.37154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37622.log 2024-09-10T23:42:01.865 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45740.log 2024-09-10T23:42:01.865 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33276.log.gz 2024-09-10T23:42:01.866 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37622.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37622.log.gz 2024-09-10T23:42:01.866 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35857.log 2024-09-10T23:42:01.867 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45740.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45740.log.gz 2024-09-10T23:42:01.867 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40379.log 2024-09-10T23:42:01.868 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45685.log 2024-09-10T23:42:01.868 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35857.log.gz 2024-09-10T23:42:01.870 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.0.40379.log: 92.5% -- replaced with /var/log/ceph/ceph-client.0.37154.log.gz 2024-09-10T23:42:01.871 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45044.log 2024-09-10T23:42:01.871 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45685.log.gz 2024-09-10T23:42:01.871 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45124.log 2024-09-10T23:42:01.873 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35805.log 2024-09-10T23:42:01.873 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45044.log.gz 2024-09-10T23:42:01.874 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45124.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45124.log.gz 2024-09-10T23:42:01.875 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35805.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35805.log.gz 2024-09-10T23:42:01.875 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41025.log 2024-09-10T23:42:01.876 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36678.log 2024-09-10T23:42:01.877 INFO:teuthology.orchestra.run.smithi094.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.0.40379.log.gz 2024-09-10T23:42:01.878 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53282.log 2024-09-10T23:42:01.878 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41025.log.gz 2024-09-10T23:42:01.879 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54483.log 2024-09-10T23:42:01.879 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36678.log.gz 2024-09-10T23:42:01.880 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44154.log 2024-09-10T23:42:01.880 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53282.log.gz 2024-09-10T23:42:01.881 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54483.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-10T23:42:01.881 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54483.log.gz 2024-09-10T23:42:01.882 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54104.log 2024-09-10T23:42:01.883 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44154.log.gz 2024-09-10T23:42:01.883 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40427.log 2024-09-10T23:42:01.884 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38667.log 2024-09-10T23:42:01.886 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54104.log.gz 2024-09-10T23:42:01.886 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40427.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40427.log.gz 2024-09-10T23:42:01.887 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38667.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38667.log.gz 2024-09-10T23:42:01.887 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50818.log 2024-09-10T23:42:01.888 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50325.log 2024-09-10T23:42:01.889 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53974.log 2024-09-10T23:42:01.890 INFO:teuthology.orchestra.run.smithi094.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.50818.log.gz 2024-09-10T23:42:01.890 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50325.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50325.log.gz 2024-09-10T23:42:01.892 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53974.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53974.log.gz 2024-09-10T23:42:01.892 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39236.log 2024-09-10T23:42:01.893 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35269.log 2024-09-10T23:42:01.895 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45923.log 2024-09-10T23:42:01.895 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39236.log.gz 2024-09-10T23:42:01.896 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35269.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35269.log.gz 2024-09-10T23:42:01.897 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45923.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45923.log.gz 2024-09-10T23:42:01.897 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36241.log 2024-09-10T23:42:01.898 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54691.log 2024-09-10T23:42:01.900 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46253.log 2024-09-10T23:42:01.900 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36241.log.gz 2024-09-10T23:42:01.902 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54691.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54691.log.gz 2024-09-10T23:42:01.902 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46253.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46253.log.gz 2024-09-10T23:42:01.903 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37492.log 2024-09-10T23:42:01.904 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36494.log 2024-09-10T23:42:01.906 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50868.log 2024-09-10T23:42:01.906 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37492.log.gz 2024-09-10T23:42:01.906 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36494.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36494.log.gz 2024-09-10T23:42:01.907 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50868.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50868.log.gz 2024-09-10T23:42:01.908 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51931.log 2024-09-10T23:42:01.909 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44317.log 2024-09-10T23:42:01.911 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40798.log 2024-09-10T23:42:01.911 INFO:teuthology.orchestra.run.smithi094.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.51931.log.gz 2024-09-10T23:42:01.911 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44317.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44317.log.gz 2024-09-10T23:42:01.912 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40798.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40798.log.gz 2024-09-10T23:42:01.913 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40505.log 2024-09-10T23:42:01.914 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35883.log 2024-09-10T23:42:01.915 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49746.log 2024-09-10T23:42:01.916 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40505.log.gz 2024-09-10T23:42:01.916 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35883.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35883.log.gz 2024-09-10T23:42:01.917 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49746.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49746.log.gz 2024-09-10T23:42:01.918 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39957.log 2024-09-10T23:42:01.919 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50535.log 2024-09-10T23:42:01.921 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39392.log 2024-09-10T23:42:01.921 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39957.log.gz 2024-09-10T23:42:01.923 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.0.50535.log: /var/log/ceph/ceph-client.admin.39392.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39392.log.gz 2024-09-10T23:42:01.923 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53167.log 2024-09-10T23:42:01.925 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39760.log 2024-09-10T23:42:01.926 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53167.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53167.log.gz 2024-09-10T23:42:01.927 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39760.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39760.log.gz 2024-09-10T23:42:01.927 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48957.log 2024-09-10T23:42:01.928 INFO:teuthology.orchestra.run.smithi094.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.0.50535.log.gz 2024-09-10T23:42:01.928 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41129.log 2024-09-10T23:42:01.930 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51274.log 2024-09-10T23:42:01.930 INFO:teuthology.orchestra.run.smithi094.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.48957.log.gz 2024-09-10T23:42:01.931 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41129.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41129.log.gz 2024-09-10T23:42:01.932 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51274.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51274.log.gz 2024-09-10T23:42:01.933 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46646.log 2024-09-10T23:42:01.933 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39497.log 2024-09-10T23:42:01.935 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52195.log 2024-09-10T23:42:01.935 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46646.log.gz 2024-09-10T23:42:01.936 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39497.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39497.log.gz 2024-09-10T23:42:01.937 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52195.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52195.log.gz 2024-09-10T23:42:01.937 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54901.log 2024-09-10T23:42:01.938 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38963.log 2024-09-10T23:42:01.940 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44916.log 2024-09-10T23:42:01.940 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54901.log.gz 2024-09-10T23:42:01.941 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38963.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38963.log.gz 2024-09-10T23:42:01.944 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.0.44916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34333.log 2024-09-10T23:42:01.945 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47141.log 2024-09-10T23:42:01.947 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48366.log 2024-09-10T23:42:01.947 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34333.log.gz 2024-09-10T23:42:01.948 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47372.log 2024-09-10T23:42:01.948 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47141.log.gz 2024-09-10T23:42:01.950 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51196.log 2024-09-10T23:42:01.950 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48366.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48366.log.gz 2024-09-10T23:42:01.950 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34281.log 2024-09-10T23:42:01.952 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47372.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.47372.log.gz 2024-09-10T23:42:01.953 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34281.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34281.log.gz 2024-09-10T23:42:01.953 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49564.log 2024-09-10T23:42:01.954 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51196.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51196.log.gz 2024-09-10T23:42:01.954 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45766.log 2024-09-10T23:42:01.957 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50351.log 2024-09-10T23:42:01.957 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49564.log: /var/log/ceph/ceph-client.admin.45766.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49564.log.gz 2024-09-10T23:42:01.957 INFO:teuthology.orchestra.run.smithi094.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45766.log.gz 2024-09-10T23:42:01.959 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50351.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.50351.log.gz 2024-09-10T23:42:01.959 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53112.log 2024-09-10T23:42:01.960 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42780.log 2024-09-10T23:42:01.962 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53112.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53112.log.gz 2024-09-10T23:42:01.963 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42780.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42780.log.gz 2024-09-10T23:42:01.963 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51144.log 2024-09-10T23:42:01.964 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34513.log 2024-09-10T23:42:01.966 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51144.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51144.log.gz 2024-09-10T23:42:01.967 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34513.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34513.log.gz 2024-09-10T23:42:01.967 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50479.log 2024-09-10T23:42:01.968 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53916.log 2024-09-10T23:42:01.970 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50479.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50479.log.gz 2024-09-10T23:42:01.971 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53916.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53916.log.gz 2024-09-10T23:42:01.971 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51615.log 2024-09-10T23:42:01.973 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44102.log 2024-09-10T23:42:01.974 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51615.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51615.log.gz 2024-09-10T23:42:01.975 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44102.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44102.log.gz 2024-09-10T23:42:01.975 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44602.log 2024-09-10T23:42:01.977 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53364.log 2024-09-10T23:42:01.978 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44602.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44602.log.gz 2024-09-10T23:42:01.979 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53364.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53364.log.gz 2024-09-10T23:42:01.979 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34707.log 2024-09-10T23:42:01.981 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38355.log 2024-09-10T23:42:01.982 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34707.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34707.log.gz 2024-09-10T23:42:01.983 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38355.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38355.log.gz 2024-09-10T23:42:01.983 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51695.log 2024-09-10T23:42:01.985 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44284.log 2024-09-10T23:42:01.987 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51695.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51695.log.gz 2024-09-10T23:42:01.987 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44284.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44284.log.gz 2024-09-10T23:42:01.988 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36397.log 2024-09-10T23:42:01.989 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45792.log 2024-09-10T23:42:01.992 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36397.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36397.log.gz 2024-09-10T23:42:01.992 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45792.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45792.log.gz 2024-09-10T23:42:01.992 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45150.log 2024-09-10T23:42:01.993 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35721.log 2024-09-10T23:42:01.995 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45150.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.45150.log.gz 2024-09-10T23:42:01.995 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35721.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35721.log.gz 2024-09-10T23:42:01.996 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45897.log 2024-09-10T23:42:01.997 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38908.log 2024-09-10T23:42:01.999 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45897.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45897.log.gz 2024-09-10T23:42:01.999 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38908.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38908.log.gz 2024-09-10T23:42:02.000 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50961.log 2024-09-10T23:42:02.001 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34506.log 2024-09-10T23:42:02.003 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50961.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50961.log.gz 2024-09-10T23:42:02.003 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34506.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34506.log.gz 2024-09-10T23:42:02.004 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47454.log 2024-09-10T23:42:02.005 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-10T23:42:02.006 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47454.log: 93.2% -- replaced with /var/log/ceph/ceph-client.0.44916.log.gz 2024-09-10T23:42:02.006 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47454.log.gz 2024-09-10T23:42:02.008 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52720.log 2024-09-10T23:42:02.009 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34721.log 2024-09-10T23:42:02.011 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53442.log 2024-09-10T23:42:02.011 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52720.log.gz 2024-09-10T23:42:02.012 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51586.log 2024-09-10T23:42:02.012 INFO:teuthology.orchestra.run.smithi094.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.34721.log.gz 2024-09-10T23:42:02.014 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53442.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53442.log.gz 2024-09-10T23:42:02.015 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40453.log 2024-09-10T23:42:02.016 INFO:teuthology.orchestra.run.smithi181.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-10T23:42:02.018 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48260.log 2024-09-10T23:42:02.018 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51586.log: /var/log/ceph/ceph-client.admin.40453.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51586.log.gz 2024-09-10T23:42:02.018 INFO:teuthology.orchestra.run.smithi094.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.40453.log.gz 2024-09-10T23:42:02.019 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48260.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48260.log.gz 2024-09-10T23:42:02.020 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38721.log 2024-09-10T23:42:02.021 INFO:teuthology.orchestra.run.smithi094.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-10T23:42:02.021 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45424.log 2024-09-10T23:42:02.023 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35376.log 2024-09-10T23:42:02.023 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38721.log.gz 2024-09-10T23:42:02.023 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45424.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45424.log.gz 2024-09-10T23:42:02.024 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35376.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35376.log.gz 2024-09-10T23:42:02.025 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50033.log 2024-09-10T23:42:02.026 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54587.log 2024-09-10T23:42:02.028 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51222.log 2024-09-10T23:42:02.028 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50033.log.gz 2024-09-10T23:42:02.028 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41051.log 2024-09-10T23:42:02.029 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54587.log.gz 2024-09-10T23:42:02.030 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51222.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51222.log.gz 2024-09-10T23:42:02.030 INFO:teuthology.orchestra.run.smithi094.stderr:gzip/var/log/ceph/ceph-client.admin.41051.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43706.log 2024-09-10T23:42:02.031 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41051.log.gz 2024-09-10T23:42:02.031 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51853.log 2024-09-10T23:42:02.033 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36520.log 2024-09-10T23:42:02.033 INFO:teuthology.orchestra.run.smithi094.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.43706.log.gz 2024-09-10T23:42:02.034 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51853.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51853.log.gz 2024-09-10T23:42:02.035 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36520.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36520.log.gz 2024-09-10T23:42:02.036 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36319.log 2024-09-10T23:42:02.037 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34177.log 2024-09-10T23:42:02.038 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44523.log 2024-09-10T23:42:02.038 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36319.log.gz 2024-09-10T23:42:02.039 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48392.log 2024-09-10T23:42:02.039 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34177.log.gz 2024-09-10T23:42:02.040 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44523.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44523.log.gz 2024-09-10T23:42:02.041 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39097.log 2024-09-10T23:42:02.041 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48392.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48392.log.gz 2024-09-10T23:42:02.042 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50930.log 2024-09-10T23:42:02.043 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40091.log 2024-09-10T23:42:02.044 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39097.log.gz 2024-09-10T23:42:02.044 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50930.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50930.log.gz 2024-09-10T23:42:02.045 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38778.log 2024-09-10T23:42:02.046 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40091.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40091.log.gz 2024-09-10T23:42:02.046 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41576.log 2024-09-10T23:42:02.047 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38778.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38778.log.gz 2024-09-10T23:42:02.048 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46097.log 2024-09-10T23:42:02.049 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35113.log 2024-09-10T23:42:02.049 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41576.log.gz 2024-09-10T23:42:02.050 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46097.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46097.log.gz 2024-09-10T23:42:02.051 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35113.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35113.log.gz 2024-09-10T23:42:02.052 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39445.log 2024-09-10T23:42:02.053 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39549.log 2024-09-10T23:42:02.054 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38989.log 2024-09-10T23:42:02.054 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39445.log.gz 2024-09-10T23:42:02.055 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29407.log 2024-09-10T23:42:02.055 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39549.log.gz 2024-09-10T23:42:02.057 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38989.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38989.log.gz 2024-09-10T23:42:02.057 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41784.log 2024-09-10T23:42:02.057 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29407.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29407.log.gz 2024-09-10T23:42:02.058 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40738.log 2024-09-10T23:42:02.059 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47320.log 2024-09-10T23:42:02.060 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40738.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41784.log.gz 2024-09-10T23:42:02.061 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40738.log.gz 2024-09-10T23:42:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47320.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.47320.log.gz 2024-09-10T23:42:02.062 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38435.log 2024-09-10T23:42:02.063 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47683.log 2024-09-10T23:42:02.065 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38383.log 2024-09-10T23:42:02.065 INFO:teuthology.orchestra.run.smithi094.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38435.log.gz 2024-09-10T23:42:02.066 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47683.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47683.log.gz 2024-09-10T23:42:02.066 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38383.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38383.log.gz 2024-09-10T23:42:02.067 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42509.log 2024-09-10T23:42:02.068 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52956.log 2024-09-10T23:42:02.070 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49459.log 2024-09-10T23:42:02.070 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42509.log.gz 2024-09-10T23:42:02.071 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39694.log 2024-09-10T23:42:02.071 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52956.log.gz 2024-09-10T23:42:02.071 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49459.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49459.log.gz 2024-09-10T23:42:02.072 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47917.log 2024-09-10T23:42:02.073 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39694.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39694.log.gz 2024-09-10T23:42:02.073 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28858.log 2024-09-10T23:42:02.075 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46227.log 2024-09-10T23:42:02.075 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47917.log.gz 2024-09-10T23:42:02.075 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.28858.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28858.log.gz 2024-09-10T23:42:02.077 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46227.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46227.log.gz 2024-09-10T23:42:02.077 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53390.log 2024-09-10T23:42:02.078 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43730.log 2024-09-10T23:42:02.080 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48444.log 2024-09-10T23:42:02.080 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53390.log.gz 2024-09-10T23:42:02.081 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34725.log 2024-09-10T23:42:02.081 INFO:teuthology.orchestra.run.smithi094.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.43730.log.gz 2024-09-10T23:42:02.082 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48444.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48444.log.gz 2024-09-10T23:42:02.082 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54000.log 2024-09-10T23:42:02.084 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34725.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34725.log.gz 2024-09-10T23:42:02.084 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29493.log 2024-09-10T23:42:02.085 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34151.log 2024-09-10T23:42:02.085 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54000.log.gz 2024-09-10T23:42:02.086 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29493.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29493.log.gz 2024-09-10T23:42:02.087 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34151.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34151.log.gz 2024-09-10T23:42:02.088 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38225.log 2024-09-10T23:42:02.089 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40323.log 2024-09-10T23:42:02.090 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41155.log 2024-09-10T23:42:02.090 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38225.log.gz 2024-09-10T23:42:02.091 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40323.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40323.log.gz 2024-09-10T23:42:02.092 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41155.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41155.log.gz 2024-09-10T23:42:02.093 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54026.log 2024-09-10T23:42:02.094 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46565.log 2024-09-10T23:42:02.095 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46882.log 2024-09-10T23:42:02.096 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54026.log.gz 2024-09-10T23:42:02.096 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37596.log 2024-09-10T23:42:02.096 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46565.log.gz 2024-09-10T23:42:02.098 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46882.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46882.log.gz 2024-09-10T23:42:02.098 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51721.log 2024-09-10T23:42:02.098 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37596.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37596.log.gz 2024-09-10T23:42:02.099 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54821.log 2024-09-10T23:42:02.101 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43014.log 2024-09-10T23:42:02.101 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51721.log.gz 2024-09-10T23:42:02.101 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54821.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54821.log.gz 2024-09-10T23:42:02.102 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40850.log 2024-09-10T23:42:02.103 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51040.log 2024-09-10T23:42:02.103 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43014.log.gz 2024-09-10T23:42:02.104 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40850.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40850.log.gz 2024-09-10T23:42:02.105 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51040.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51040.log.gz 2024-09-10T23:42:02.106 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40769.log 2024-09-10T23:42:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29077.log 2024-09-10T23:42:02.108 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42240.log 2024-09-10T23:42:02.108 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40769.log.gz 2024-09-10T23:42:02.109 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40595.log 2024-09-10T23:42:02.109 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29077.log.gz 2024-09-10T23:42:02.110 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36163.log 2024-09-10T23:42:02.110 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42240.log.gz 2024-09-10T23:42:02.111 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34927.log 2024-09-10T23:42:02.111 INFO:teuthology.orchestra.run.smithi094.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.40595.log.gz 2024-09-10T23:42:02.113 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36163.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36163.log.gz 2024-09-10T23:42:02.114 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34927.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34927.log.gz 2024-09-10T23:42:02.114 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46071.log 2024-09-10T23:42:02.115 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49346.log 2024-09-10T23:42:02.117 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52878.log 2024-09-10T23:42:02.117 INFO:teuthology.orchestra.run.smithi094.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46071.log.gz 2024-09-10T23:42:02.118 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49346.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.49346.log.gz 2024-09-10T23:42:02.119 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52878.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52878.log.gz 2024-09-10T23:42:02.119 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44128.log 2024-09-10T23:42:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42676.log 2024-09-10T23:42:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41997.log 2024-09-10T23:42:02.122 INFO:teuthology.orchestra.run.smithi094.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44128.log.gz 2024-09-10T23:42:02.123 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42676.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42676.log.gz 2024-09-10T23:42:02.124 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41997.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41997.log.gz 2024-09-10T23:42:02.124 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38147.log 2024-09-10T23:42:02.125 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40569.log 2024-09-10T23:42:02.127 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52247.log 2024-09-10T23:42:02.127 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38147.log.gz 2024-09-10T23:42:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51643.log 2024-09-10T23:42:02.128 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40569.log.gz 2024-09-10T23:42:02.129 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52247.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52247.log.gz 2024-09-10T23:42:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42457.log 2024-09-10T23:42:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51643.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51643.log.gz 2024-09-10T23:42:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45711.log 2024-09-10T23:42:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46700.log 2024-09-10T23:42:02.132 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42457.log.gz 2024-09-10T23:42:02.133 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41233.log 2024-09-10T23:42:02.133 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45711.log.gz 2024-09-10T23:42:02.134 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37570.log 2024-09-10T23:42:02.134 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46700.log.gz 2024-09-10T23:42:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41233.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41233.log.gz 2024-09-10T23:42:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37882.log 2024-09-10T23:42:02.137 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37570.log.gz 2024-09-10T23:42:02.137 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44732.log 2024-09-10T23:42:02.139 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48574.log 2024-09-10T23:42:02.139 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37882.log.gz 2024-09-10T23:42:02.140 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44732.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44732.log.gz 2024-09-10T23:42:02.141 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48574.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T23:42:02.142 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36838.log 2024-09-10T23:42:02.143 INFO:teuthology.orchestra.run.smithi094.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48574.log.gz 2024-09-10T23:42:02.143 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41758.log 2024-09-10T23:42:02.145 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36838.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36838.log.gz 2024-09-10T23:42:02.146 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41758.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41758.log.gz 2024-09-10T23:42:02.146 INFO:teuthology.orchestra.run.smithi094.stderr: 86.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T23:42:02.146 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42160.log 2024-09-10T23:42:02.147 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47813.log 2024-09-10T23:42:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37752.log 2024-09-10T23:42:02.149 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42160.log.gz 2024-09-10T23:42:02.150 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47813.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47813.log.gz 2024-09-10T23:42:02.150 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37752.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37752.log.gz 2024-09-10T23:42:02.151 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40624.log 2024-09-10T23:42:02.152 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39288.log 2024-09-10T23:42:02.154 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50299.log 2024-09-10T23:42:02.154 INFO:teuthology.orchestra.run.smithi094.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.40624.log.gz 2024-09-10T23:42:02.155 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51983.log 2024-09-10T23:42:02.155 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39288.log.gz 2024-09-10T23:42:02.156 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50299.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50299.log.gz 2024-09-10T23:42:02.156 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52746.log 2024-09-10T23:42:02.157 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51983.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51983.log.gz 2024-09-10T23:42:02.157 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47735.log 2024-09-10T23:42:02.159 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51534.log 2024-09-10T23:42:02.159 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52746.log.gz 2024-09-10T23:42:02.160 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47735.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47735.log.gz 2024-09-10T23:42:02.161 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51534.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51534.log.gz 2024-09-10T23:42:02.161 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34499.log 2024-09-10T23:42:02.162 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53338.log 2024-09-10T23:42:02.164 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49877.log 2024-09-10T23:42:02.164 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34499.log.gz 2024-09-10T23:42:02.165 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53338.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53338.log.gz 2024-09-10T23:42:02.166 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49877.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49877.log.gz 2024-09-10T23:42:02.166 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54509.log 2024-09-10T23:42:02.167 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42050.log 2024-09-10T23:42:02.169 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34943.log 2024-09-10T23:42:02.169 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54509.log.gz 2024-09-10T23:42:02.170 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42050.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42050.log.gz 2024-09-10T23:42:02.171 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34943.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34943.log.gz 2024-09-10T23:42:02.171 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46175.log 2024-09-10T23:42:02.173 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36026.log 2024-09-10T23:42:02.174 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42212.log 2024-09-10T23:42:02.174 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46175.log.gz 2024-09-10T23:42:02.175 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36026.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36026.log.gz 2024-09-10T23:42:02.176 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42212.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42212.log.gz 2024-09-10T23:42:02.176 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36625.log 2024-09-10T23:42:02.177 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49196.log 2024-09-10T23:42:02.179 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51326.log 2024-09-10T23:42:02.179 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36625.log.gz 2024-09-10T23:42:02.180 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36267.log 2024-09-10T23:42:02.180 INFO:teuthology.orchestra.run.smithi094.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.49196.log.gz 2024-09-10T23:42:02.181 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51326.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51326.log.gz 2024-09-10T23:42:02.182 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.36267.log: -- /var/log/ceph/ceph-client.admin.51092.log 2024-09-10T23:42:02.183 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36267.log.gz 2024-09-10T23:42:02.183 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48907.log 2024-09-10T23:42:02.184 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53763.log 2024-09-10T23:42:02.185 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51092.log.gz 2024-09-10T23:42:02.186 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48907.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.48907.log.gz 2024-09-10T23:42:02.187 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53763.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53763.log.gz 2024-09-10T23:42:02.187 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54535.log 2024-09-10T23:42:02.188 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54405.log 2024-09-10T23:42:02.190 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34073.log 2024-09-10T23:42:02.190 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54535.log.gz 2024-09-10T23:42:02.191 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47294.log 2024-09-10T23:42:02.191 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54405.log.gz 2024-09-10T23:42:02.192 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34073.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34073.log.gz 2024-09-10T23:42:02.193 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37726.log 2024-09-10T23:42:02.193 INFO:teuthology.orchestra.run.smithi181.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-10T23:42:02.193 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47294.log.gz 2024-09-10T23:42:02.196 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49850.log 2024-09-10T23:42:02.196 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42546.log 2024-09-10T23:42:02.196 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37726.log.gz 2024-09-10T23:42:02.196 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49850.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49850.log.gz 2024-09-10T23:42:02.197 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42546.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42546.log.gz 2024-09-10T23:42:02.198 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37343.log 2024-09-10T23:42:02.199 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37778.log 2024-09-10T23:42:02.200 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46830.log 2024-09-10T23:42:02.200 INFO:teuthology.orchestra.run.smithi094.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37343.log.gz 2024-09-10T23:42:02.201 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49642.log 2024-09-10T23:42:02.201 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37778.log.gz 2024-09-10T23:42:02.203 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46830.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46830.log.gz 2024-09-10T23:42:02.203 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36000.log 2024-09-10T23:42:02.204 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49642.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49642.log.gz 2024-09-10T23:42:02.204 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43040.log 2024-09-10T23:42:02.206 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43868.log 2024-09-10T23:42:02.206 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36000.log.gz 2024-09-10T23:42:02.206 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43040.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43040.log.gz 2024-09-10T23:42:02.208 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43868.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43868.log.gz 2024-09-10T23:42:02.208 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50583.log 2024-09-10T23:42:02.209 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41834.log 2024-09-10T23:42:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48496.log 2024-09-10T23:42:02.211 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50583.log.gz 2024-09-10T23:42:02.212 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52989.log 2024-09-10T23:42:02.212 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41834.log.gz 2024-09-10T23:42:02.213 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48496.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48496.log.gz 2024-09-10T23:42:02.213 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40169.log 2024-09-10T23:42:02.214 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52989.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52989.log.gz 2024-09-10T23:42:02.215 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52852.log 2024-09-10T23:42:02.216 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36916.log 2024-09-10T23:42:02.216 INFO:teuthology.orchestra.run.smithi094.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.40169.log.gz 2024-09-10T23:42:02.217 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52852.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52852.log.gz 2024-09-10T23:42:02.218 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36916.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36916.log.gz 2024-09-10T23:42:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44576.log 2024-09-10T23:42:02.220 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48932.log 2024-09-10T23:42:02.221 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40980.log 2024-09-10T23:42:02.221 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44576.log.gz 2024-09-10T23:42:02.222 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48932.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.48932.log.gz 2024-09-10T23:42:02.223 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40980.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40980.log.gz 2024-09-10T23:42:02.224 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38064.log 2024-09-10T23:42:02.224 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43946.log 2024-09-10T23:42:02.226 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46726.log 2024-09-10T23:42:02.226 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38064.log.gz 2024-09-10T23:42:02.227 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43946.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43946.log.gz 2024-09-10T23:42:02.228 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46726.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46726.log.gz 2024-09-10T23:42:02.228 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39041.log 2024-09-10T23:42:02.229 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41732.log 2024-09-10T23:42:02.231 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36864.log 2024-09-10T23:42:02.231 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39041.log.gz 2024-09-10T23:42:02.233 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41732.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41732.log.gz 2024-09-10T23:42:02.233 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29041.log 2024-09-10T23:42:02.234 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41266.log 2024-09-10T23:42:02.234 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36864.log.gz 2024-09-10T23:42:02.235 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39987.log 2024-09-10T23:42:02.235 INFO:teuthology.orchestra.run.smithi094.stderr: 76.7% -- replaced with /var/log/ceph/ceph-client.admin.29041.log.gz 2024-09-10T23:42:02.237 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41266.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41266.log.gz 2024-09-10T23:42:02.237 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39987.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39987.log.gz 2024-09-10T23:42:02.238 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50142.log 2024-09-10T23:42:02.239 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47571.log 2024-09-10T23:42:02.241 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52509.log 2024-09-10T23:42:02.241 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50142.log.gz 2024-09-10T23:42:02.241 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47571.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47571.log.gz 2024-09-10T23:42:02.242 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52509.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52509.log.gz 2024-09-10T23:42:02.243 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43331.log 2024-09-10T23:42:02.244 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47865.log 2024-09-10T23:42:02.246 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45228.log 2024-09-10T23:42:02.246 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43331.log.gz 2024-09-10T23:42:02.246 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47865.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47865.log.gz 2024-09-10T23:42:02.247 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45228.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45228.log.gz 2024-09-10T23:42:02.248 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34515.log 2024-09-10T23:42:02.249 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34720.log 2024-09-10T23:42:02.250 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52800.log 2024-09-10T23:42:02.250 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34515.log.gz 2024-09-10T23:42:02.252 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34720.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34720.log.gz 2024-09-10T23:42:02.253 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52800.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52800.log.gz 2024-09-10T23:42:02.253 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46149.log 2024-09-10T23:42:02.254 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37856.log 2024-09-10T23:42:02.256 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51300.log 2024-09-10T23:42:02.256 INFO:teuthology.orchestra.run.smithi094.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46149.log.gz 2024-09-10T23:42:02.256 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37856.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37856.log.gz 2024-09-10T23:42:02.258 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51300.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51300.log.gz 2024-09-10T23:42:02.258 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38251.log 2024-09-10T23:42:02.259 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52009.log 2024-09-10T23:42:02.261 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40676.log 2024-09-10T23:42:02.261 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38251.log.gz 2024-09-10T23:42:02.262 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52009.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52009.log.gz 2024-09-10T23:42:02.263 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40676.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40676.log.gz 2024-09-10T23:42:02.263 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53801.log 2024-09-10T23:42:02.265 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50789.log 2024-09-10T23:42:02.266 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46674.log 2024-09-10T23:42:02.266 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53801.log.gz 2024-09-10T23:42:02.267 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46357.log 2024-09-10T23:42:02.267 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50789.log.gz 2024-09-10T23:42:02.269 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46674.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46674.log.gz 2024-09-10T23:42:02.269 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52478.log 2024-09-10T23:42:02.269 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46357.log.gz 2024-09-10T23:42:02.270 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44232.log 2024-09-10T23:42:02.271 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28945.log 2024-09-10T23:42:02.272 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52478.log.gz 2024-09-10T23:42:02.273 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44232.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44232.log.gz 2024-09-10T23:42:02.273 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.28945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28945.log.gz 2024-09-10T23:42:02.274 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35191.log 2024-09-10T23:42:02.275 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42988.log 2024-09-10T23:42:02.277 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45304.log 2024-09-10T23:42:02.277 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35191.log.gz 2024-09-10T23:42:02.277 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42988.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42988.log.gz 2024-09-10T23:42:02.278 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45304.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45304.log.gz 2024-09-10T23:42:02.279 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52904.log 2024-09-10T23:42:02.280 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41181.log 2024-09-10T23:42:02.282 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47534.log 2024-09-10T23:42:02.282 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52904.log.gz 2024-09-10T23:42:02.282 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41181.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41181.log.gz 2024-09-10T23:42:02.283 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47534.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47534.log.gz 2024-09-10T23:42:02.284 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43253.log 2024-09-10T23:42:02.285 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29065.log 2024-09-10T23:42:02.286 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50429.log 2024-09-10T23:42:02.287 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43253.log.gz 2024-09-10T23:42:02.287 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29065.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29065.log.gz 2024-09-10T23:42:02.289 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50429.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50429.log.gz 2024-09-10T23:42:02.289 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50168.log 2024-09-10T23:42:02.290 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47969.log 2024-09-10T23:42:02.292 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38589.log 2024-09-10T23:42:02.292 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50168.log.gz 2024-09-10T23:42:02.293 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47969.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47969.log.gz 2024-09-10T23:42:02.294 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38589.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38589.log.gz 2024-09-10T23:42:02.294 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48680.log 2024-09-10T23:42:02.296 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38615.log 2024-09-10T23:42:02.297 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.0.48680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36111.log 2024-09-10T23:42:02.298 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38615.log: 90.1% -- replaced with /var/log/ceph/ceph-client.0.48680.log.gz 2024-09-10T23:42:02.298 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38615.log.gz 2024-09-10T23:42:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36111.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36111.log.gz 2024-09-10T23:42:02.299 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54743.log 2024-09-10T23:42:02.301 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34125.log 2024-09-10T23:42:02.302 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49159.log 2024-09-10T23:42:02.302 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54743.log.gz 2024-09-10T23:42:02.303 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50247.log 2024-09-10T23:42:02.303 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34125.log.gz 2024-09-10T23:42:02.304 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49159.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.49159.log.gz 2024-09-10T23:42:02.305 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52358.log 2024-09-10T23:42:02.305 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50247.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50247.log.gz 2024-09-10T23:42:02.306 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40479.log 2024-09-10T23:42:02.307 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36786.log 2024-09-10T23:42:02.307 INFO:teuthology.orchestra.run.smithi094.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52358.log.gz 2024-09-10T23:42:02.308 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40479.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40479.log.gz 2024-09-10T23:42:02.309 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36786.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36786.log.gz 2024-09-10T23:42:02.310 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T23:42:02.311 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52536.log 2024-09-10T23:42:02.312 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-10T23:42:02.313 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52536.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52536.log.gz 2024-09-10T23:42:02.316 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mds.c.log: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T23:42:02.316 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36890.log 2024-09-10T23:42:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39786.log 2024-09-10T23:42:02.319 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40876.log 2024-09-10T23:42:02.319 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36890.log.gz 2024-09-10T23:42:02.320 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36760.log 2024-09-10T23:42:02.320 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39786.log.gz 2024-09-10T23:42:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41103.log 2024-09-10T23:42:02.322 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54927.log 2024-09-10T23:42:02.322 INFO:teuthology.orchestra.run.smithi094.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40876.log.gz 2024-09-10T23:42:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47508.log 2024-09-10T23:42:02.323 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36760.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36760.log.gz 2024-09-10T23:42:02.325 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54927.log: /var/log/ceph/ceph-client.admin.41103.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54927.log.gz 2024-09-10T23:42:02.326 INFO:teuthology.orchestra.run.smithi094.stderr:gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41103.log.gz 2024-09-10T23:42:02.326 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50696.log 2024-09-10T23:42:02.327 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47508.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47508.log.gz 2024-09-10T23:42:02.328 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50696.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50696.log.gz 2024-09-10T23:42:02.329 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34411.log 2024-09-10T23:42:02.330 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35295.log 2024-09-10T23:42:02.332 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34411.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34411.log.gz 2024-09-10T23:42:02.332 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35295.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35295.log.gz 2024-09-10T23:42:02.333 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-10T23:42:02.333 INFO:teuthology.orchestra.run.smithi181.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-10T23:42:02.334 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49082.log 2024-09-10T23:42:02.337 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.49082.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.49082.log.gz 2024-09-10T23:42:02.337 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47604.log 2024-09-10T23:42:02.337 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48288.log 2024-09-10T23:42:02.340 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47604.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47604.log.gz 2024-09-10T23:42:02.341 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35504.log 2024-09-10T23:42:02.346 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48288.log: /var/log/ceph/ceph-client.0.35504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49824.log 2024-09-10T23:42:02.346 INFO:teuthology.orchestra.run.smithi094.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.0.35504.log.gz 2024-09-10T23:42:02.346 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48288.log.gz 2024-09-10T23:42:02.348 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34229.log 2024-09-10T23:42:02.349 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49824.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49824.log.gz 2024-09-10T23:42:02.349 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53726.log 2024-09-10T23:42:02.353 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34229.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34229.log.gz 2024-09-10T23:42:02.354 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53726.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53726.log.gz 2024-09-10T23:42:02.354 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50897.log 2024-09-10T23:42:02.357 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50897.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50897.log.gz 2024-09-10T23:42:02.357 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42023.log 2024-09-10T23:42:02.358 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34019.log 2024-09-10T23:42:02.363 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34019.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34019.log.gz 2024-09-10T23:42:02.363 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42023.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42023.log.gz 2024-09-10T23:42:02.363 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35909.log 2024-09-10T23:42:02.365 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41524.log 2024-09-10T23:42:02.366 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35909.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35909.log.gz 2024-09-10T23:42:02.369 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41524.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41524.log.gz 2024-09-10T23:42:02.369 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37098.log 2024-09-10T23:42:02.372 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41971.log 2024-09-10T23:42:02.372 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37098.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37098.log.gz 2024-09-10T23:42:02.375 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41971.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41971.log.gz 2024-09-10T23:42:02.375 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28901.log 2024-09-10T23:42:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.28901.log: 76.5% -- replaced with /var/log/ceph/ceph-client.admin.28901.log.gz 2024-09-10T23:42:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43537.log 2024-09-10T23:42:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43537.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43537.log.gz 2024-09-10T23:42:02.381 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39151.log 2024-09-10T23:42:02.384 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39151.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39151.log.gz 2024-09-10T23:42:02.384 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48857.log 2024-09-10T23:42:02.386 INFO:teuthology.orchestra.run.smithi181.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-10T23:42:02.387 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45658.log 2024-09-10T23:42:02.387 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48857.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.48857.log.gz 2024-09-10T23:42:02.390 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45658.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45658.log.gz 2024-09-10T23:42:02.390 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39842.log 2024-09-10T23:42:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39842.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39842.log.gz 2024-09-10T23:42:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52320.log 2024-09-10T23:42:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52320.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52320.log.gz 2024-09-10T23:42:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46468.log 2024-09-10T23:42:02.400 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46468.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46468.log.gz 2024-09-10T23:42:02.400 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34702.log 2024-09-10T23:42:02.400 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54639.log 2024-09-10T23:42:02.402 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34702.log: /var/log/ceph/ceph-client.admin.54639.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34702.log.gz 2024-09-10T23:42:02.402 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47839.log 2024-09-10T23:42:02.402 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54639.log.gz 2024-09-10T23:42:02.405 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47839.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47839.log.gz 2024-09-10T23:42:02.405 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47995.log 2024-09-10T23:42:02.408 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47995.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47995.log.gz 2024-09-10T23:42:02.408 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45847.log 2024-09-10T23:42:02.410 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45847.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45847.log.gz 2024-09-10T23:42:02.411 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43227.log 2024-09-10T23:42:02.414 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43227.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43227.log.gz 2024-09-10T23:42:02.414 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53230.log 2024-09-10T23:42:02.417 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53230.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53230.log.gz 2024-09-10T23:42:02.417 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39931.log 2024-09-10T23:42:02.419 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43175.log 2024-09-10T23:42:02.420 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39931.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39931.log.gz 2024-09-10T23:42:02.422 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43175.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43175.log.gz 2024-09-10T23:42:02.422 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48754.log 2024-09-10T23:42:02.423 INFO:teuthology.orchestra.run.smithi181.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-10T23:42:02.425 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39262.log 2024-09-10T23:42:02.426 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48754.log.gz 2024-09-10T23:42:02.427 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52772.log 2024-09-10T23:42:02.429 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39262.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39262.log.gz 2024-09-10T23:42:02.431 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52772.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52772.log.gz 2024-09-10T23:42:02.431 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39210.log 2024-09-10T23:42:02.434 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39210.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39210.log.gz 2024-09-10T23:42:02.434 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49720.log 2024-09-10T23:42:02.437 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44628.log 2024-09-10T23:42:02.437 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49720.log.gz 2024-09-10T23:42:02.439 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36546.log 2024-09-10T23:42:02.443 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36546.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36546.log.gz 2024-09-10T23:42:02.443 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44628.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44628.log.gz 2024-09-10T23:42:02.443 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39314.log 2024-09-10T23:42:02.446 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39314.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39314.log.gz 2024-09-10T23:42:02.446 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28981.log 2024-09-10T23:42:02.449 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.28981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35217.log 2024-09-10T23:42:02.449 INFO:teuthology.orchestra.run.smithi094.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.28981.log.gz 2024-09-10T23:42:02.450 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42702.log 2024-09-10T23:42:02.452 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35217.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35217.log.gz 2024-09-10T23:42:02.454 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42702.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42702.log.gz 2024-09-10T23:42:02.455 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51957.log 2024-09-10T23:42:02.456 INFO:teuthology.orchestra.run.smithi094.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-10T23:42:02.457 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43841.log 2024-09-10T23:42:02.457 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51957.log.gz 2024-09-10T23:42:02.458 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35969.log 2024-09-10T23:42:02.460 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43841.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43841.log.gz 2024-09-10T23:42:02.461 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35969.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35969.log.gz 2024-09-10T23:42:02.461 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46960.log 2024-09-10T23:42:02.463 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53256.log 2024-09-10T23:42:02.464 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46960.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.46960.log.gz 2024-09-10T23:42:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53256.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53256.log.gz 2024-09-10T23:42:02.465 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43097.log 2024-09-10T23:42:02.467 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54229.log 2024-09-10T23:42:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43097.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43097.log.gz 2024-09-10T23:42:02.468 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52668.log 2024-09-10T23:42:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54229.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54229.log.gz 2024-09-10T23:42:02.470 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49936.log 2024-09-10T23:42:02.471 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52668.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52668.log.gz 2024-09-10T23:42:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49936.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49936.log.gz 2024-09-10T23:42:02.473 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44680.log 2024-09-10T23:42:02.474 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48832.log 2024-09-10T23:42:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44680.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44680.log.gz 2024-09-10T23:42:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48832.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.48832.log.gz 2024-09-10T23:42:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37280.log 2024-09-10T23:42:02.478 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41945.log 2024-09-10T23:42:02.480 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37280.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37280.log.gz 2024-09-10T23:42:02.480 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41945.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41945.log.gz 2024-09-10T23:42:02.481 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28933.log 2024-09-10T23:42:02.482 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54130.log 2024-09-10T23:42:02.483 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.28933.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28933.log.gz 2024-09-10T23:42:02.485 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54130.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54130.log.gz 2024-09-10T23:42:02.485 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29353.log 2024-09-10T23:42:02.486 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33873.log 2024-09-10T23:42:02.488 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29353.log: 93.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-10T23:42:02.488 INFO:teuthology.orchestra.run.smithi094.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29353.log.gz 2024-09-10T23:42:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33873.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33873.log.gz 2024-09-10T23:42:02.489 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41077.log 2024-09-10T23:42:02.490 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47167.log 2024-09-10T23:42:02.491 INFO:teuthology.orchestra.run.smithi181.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-10T23:42:02.491 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44706.log 2024-09-10T23:42:02.492 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41077.log.gz 2024-09-10T23:42:02.493 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51775.log 2024-09-10T23:42:02.493 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47167.log.gz 2024-09-10T23:42:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44706.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44706.log.gz 2024-09-10T23:42:02.494 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44964.log 2024-09-10T23:42:02.495 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51775.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51775.log.gz 2024-09-10T23:42:02.495 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37228.log 2024-09-10T23:42:02.497 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34929.log 2024-09-10T23:42:02.497 INFO:teuthology.orchestra.run.smithi094.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44964.log.gz 2024-09-10T23:42:02.498 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37228.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37228.log.gz 2024-09-10T23:42:02.499 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34929.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34929.log.gz 2024-09-10T23:42:02.500 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42598.log 2024-09-10T23:42:02.501 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28993.log 2024-09-10T23:42:02.502 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45202.log 2024-09-10T23:42:02.502 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42598.log.gz 2024-09-10T23:42:02.503 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.28993.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28993.log.gz 2024-09-10T23:42:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45202.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45202.log.gz 2024-09-10T23:42:02.504 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43810.log 2024-09-10T23:42:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48230.log 2024-09-10T23:42:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42891.log 2024-09-10T23:42:02.507 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43810.log.gz 2024-09-10T23:42:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43279.log 2024-09-10T23:42:02.508 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48230.log.gz 2024-09-10T23:42:02.509 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42891.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42891.log.gz 2024-09-10T23:42:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42858.log 2024-09-10T23:42:02.510 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43279.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43279.log.gz 2024-09-10T23:42:02.511 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40117.log 2024-09-10T23:42:02.512 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53947.log 2024-09-10T23:42:02.512 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42858.log.gz 2024-09-10T23:42:02.513 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40117.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40117.log.gz 2024-09-10T23:42:02.514 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53947.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53947.log.gz 2024-09-10T23:42:02.515 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46856.log 2024-09-10T23:42:02.516 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34517.log 2024-09-10T23:42:02.517 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44860.log 2024-09-10T23:42:02.517 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46856.log.gz 2024-09-10T23:42:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34517.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34517.log.gz 2024-09-10T23:42:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44860.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44860.log.gz 2024-09-10T23:42:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40650.log 2024-09-10T23:42:02.521 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49107.log 2024-09-10T23:42:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40143.log 2024-09-10T23:42:02.522 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40650.log.gz 2024-09-10T23:42:02.523 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49107.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.49107.log.gz 2024-09-10T23:42:02.524 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40143.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40143.log.gz 2024-09-10T23:42:02.525 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52694.log 2024-09-10T23:42:02.526 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28957.log 2024-09-10T23:42:02.527 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36054.log 2024-09-10T23:42:02.527 INFO:teuthology.orchestra.run.smithi094.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52694.log.gz 2024-09-10T23:42:02.528 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.28957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41470.log 2024-09-10T23:42:02.528 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28957.log.gz 2024-09-10T23:42:02.529 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36054.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36054.log.gz 2024-09-10T23:42:02.529 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39812.log 2024-09-10T23:42:02.531 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41470.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41470.log.gz 2024-09-10T23:42:02.531 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43998.log 2024-09-10T23:42:02.532 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39812.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39812.log.gz 2024-09-10T23:42:02.532 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53648.log 2024-09-10T23:42:02.533 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40065.log 2024-09-10T23:42:02.533 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43998.log.gz 2024-09-10T23:42:02.534 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53648.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53648.log.gz 2024-09-10T23:42:02.535 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43563.log 2024-09-10T23:42:02.535 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40065.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40065.log.gz 2024-09-10T23:42:02.536 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49968.log 2024-09-10T23:42:02.537 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41628.log 2024-09-10T23:42:02.537 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43563.log.gz 2024-09-10T23:42:02.539 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49968.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49968.log.gz 2024-09-10T23:42:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41628.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41628.log.gz 2024-09-10T23:42:02.540 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45959.log 2024-09-10T23:42:02.541 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29053.log 2024-09-10T23:42:02.543 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39471.log 2024-09-10T23:42:02.543 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45959.log.gz 2024-09-10T23:42:02.543 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29053.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29053.log.gz -5 2024-09-10T23:42:02.544 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43123.log 2024-09-10T23:42:02.545 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39471.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39471.log.gz 2024-09-10T23:42:02.545 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44654.log 2024-09-10T23:42:02.546 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43123.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43123.log.gz 2024-09-10T23:42:02.546 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36704.log 2024-09-10T23:42:02.548 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53021.log 2024-09-10T23:42:02.548 INFO:teuthology.orchestra.run.smithi094.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44654.log.gz 2024-09-10T23:42:02.549 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48021.log 2024-09-10T23:42:02.549 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36704.log.gz 2024-09-10T23:42:02.549 INFO:teuthology.orchestra.run.smithi094.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-10T23:42:02.550 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53021.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53021.log.gz 2024-09-10T23:42:02.550 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47193.log 2024-09-10T23:42:02.551 INFO:teuthology.orchestra.run.smithi181.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-10T23:42:02.551 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34203.log 2024-09-10T23:42:02.552 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48021.log.gz 2024-09-10T23:42:02.553 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38409.log 2024-09-10T23:42:02.553 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47193.log.gz 2024-09-10T23:42:02.554 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40013.log 2024-09-10T23:42:02.554 INFO:teuthology.orchestra.run.smithi094.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34203.log.gz 2024-09-10T23:42:02.555 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42650.log 2024-09-10T23:42:02.555 INFO:teuthology.orchestra.run.smithi094.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38409.log.gz 2024-09-10T23:42:02.556 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53494.log 2024-09-10T23:42:02.556 INFO:teuthology.orchestra.run.smithi094.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.40013.log.gz 2024-09-10T23:42:02.557 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46305.log 2024-09-10T23:42:02.557 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42650.log.gz 2024-09-10T23:42:02.558 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41389.log 2024-09-10T23:42:02.559 INFO:teuthology.orchestra.run.smithi094.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53494.log.gz 2024-09-10T23:42:02.559 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38935.log 2024-09-10T23:42:02.559 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46305.log.gz 2024-09-10T23:42:02.560 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45018.log 2024-09-10T23:42:02.560 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41389.log.gz 2024-09-10T23:42:02.561 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50734.log 2024-09-10T23:42:02.561 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38935.log.gz 2024-09-10T23:42:02.562 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41498.log 2024-09-10T23:42:02.563 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45018.log.gz 2024-09-10T23:42:02.564 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36215.log 2024-09-10T23:42:02.564 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50734.log.gz 2024-09-10T23:42:02.565 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49396.log 2024-09-10T23:42:02.565 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41498.log.gz 2024-09-10T23:42:02.566 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44180.log 2024-09-10T23:42:02.566 INFO:teuthology.orchestra.run.smithi094.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36215.log.gz 2024-09-10T23:42:02.567 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38752.log 2024-09-10T23:42:02.567 INFO:teuthology.orchestra.run.smithi094.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.49396.log.gz 2024-09-10T23:42:02.568 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53138.log 2024-09-10T23:42:02.568 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44180.log.gz 2024-09-10T23:42:02.569 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29005.log 2024-09-10T23:42:02.569 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38752.log.gz 2024-09-10T23:42:02.570 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44466.log 2024-09-10T23:42:02.570 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53138.log.gz 2024-09-10T23:42:02.571 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54327.log 2024-09-10T23:42:02.571 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29005.log.gz 2024-09-10T23:42:02.572 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46043.log 2024-09-10T23:42:02.573 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44466.log.gz 2024-09-10T23:42:02.574 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50273.log 2024-09-10T23:42:02.574 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54327.log.gz 2024-09-10T23:42:02.575 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51404.log 2024-09-10T23:42:02.575 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46043.log.gz 2024-09-10T23:42:02.576 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51905.log 2024-09-10T23:42:02.576 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50273.log.gz 2024-09-10T23:42:02.577 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35554.log 2024-09-10T23:42:02.577 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51404.log.gz 2024-09-10T23:42:02.578 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42077.log 2024-09-10T23:42:02.578 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51905.log.gz 2024-09-10T23:42:02.579 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53830.log 2024-09-10T23:42:02.579 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35554.log.gz 2024-09-10T23:42:02.580 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49296.log 2024-09-10T23:42:02.581 INFO:teuthology.orchestra.run.smithi094.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42077.log.gz 2024-09-10T23:42:02.581 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48047.log 2024-09-10T23:42:02.581 INFO:teuthology.orchestra.run.smithi094.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53830.log.gz 2024-09-10T23:42:02.582 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33840.log 2024-09-10T23:42:02.583 INFO:teuthology.orchestra.run.smithi094.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.49296.log.gz 2024-09-10T23:42:02.584 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48112.log 2024-09-10T23:42:02.584 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48047.log.gz 2024-09-10T23:42:02.585 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50085.log 2024-09-10T23:42:02.585 INFO:teuthology.orchestra.run.smithi094.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-10T23:42:02.585 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33840.log.gz 2024-09-10T23:42:02.586 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45450.log 2024-09-10T23:42:02.586 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48112.log.gz 2024-09-10T23:42:02.586 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50085.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50085.log.gz 2024-09-10T23:42:02.587 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38641.log 2024-09-10T23:42:02.587 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49616.log 2024-09-10T23:42:02.588 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45450.log: /var/log/ceph/ceph-client.admin.38641.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45450.log.gz 2024-09-10T23:42:02.589 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38641.log.gz 2024-09-10T23:42:02.589 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39340.log 2024-09-10T23:42:02.589 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37804.log 2024-09-10T23:42:02.590 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49616.log: 87.8%/var/log/ceph/ceph-client.admin.39340.log: -- replaced with /var/log/ceph/ceph-client.admin.49616.log.gz 2024-09-10T23:42:02.590 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34921.log 2024-09-10T23:42:02.590 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39340.log.gz 2024-09-10T23:42:02.591 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48548.log 2024-09-10T23:42:02.591 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37804.log.gz 2024-09-10T23:42:02.592 INFO:teuthology.orchestra.run.smithi094.stderr:gzip/var/log/ceph/ceph-client.admin.34921.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48780.log 2024-09-10T23:42:02.593 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34921.log.gz 2024-09-10T23:42:02.594 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42624.log 2024-09-10T23:42:02.594 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48548.log.gz 2024-09-10T23:42:02.595 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53674.log 2024-09-10T23:42:02.595 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48780.log.gz 2024-09-10T23:42:02.595 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36137.log 2024-09-10T23:42:02.596 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42624.log: /var/log/ceph/ceph-client.admin.53674.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42624.log.gz 2024-09-10T23:42:02.597 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52085.log 2024-09-10T23:42:02.597 INFO:teuthology.orchestra.run.smithi094.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53674.log.gz 2024-09-10T23:42:02.598 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51469.log 2024-09-10T23:42:02.598 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36137.log.gz 2024-09-10T23:42:02.598 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51352.log 2024-09-10T23:42:02.599 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52085.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52085.log.gz 2024-09-10T23:42:02.600 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39905.log 2024-09-10T23:42:02.600 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51469.log.gz 2024-09-10T23:42:02.601 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43489.log 2024-09-10T23:42:02.601 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51352.log.gz 2024-09-10T23:42:02.606 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49668.log 2024-09-10T23:42:02.606 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39905.log: /var/log/ceph/ceph-client.0.43489.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39905.log.gz 2024-09-10T23:42:02.606 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34359.log 2024-09-10T23:42:02.606 INFO:teuthology.orchestra.run.smithi094.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.0.43489.log.gz 2024-09-10T23:42:02.606 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49221.log 2024-09-10T23:42:02.606 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49668.log.gz 2024-09-10T23:42:02.606 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.34359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51747.log 2024-09-10T23:42:02.606 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34359.log.gz 2024-09-10T23:42:02.607 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37202.log 2024-09-10T23:42:02.607 INFO:teuthology.orchestra.run.smithi094.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.49221.log.gz 2024-09-10T23:42:02.607 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51747.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51747.log.gz 2024-09-10T23:42:02.608 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45990.log 2024-09-10T23:42:02.608 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41706.log 2024-09-10T23:42:02.608 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37202.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37202.log.gz 2024-09-10T23:42:02.609 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45990.log: gzip -5 --verbose -- 87.4% /var/log/ceph/ceph-client.admin.42134.log 2024-09-10T23:42:02.609 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45990.log.gz 2024-09-10T23:42:02.610 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54156.log 2024-09-10T23:42:02.610 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41706.log.gz 2024-09-10T23:42:02.611 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42134.log: -- /var/log/ceph/ceph-client.admin.46752.log 2024-09-10T23:42:02.611 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42134.log.gz 2024-09-10T23:42:02.612 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54156.log: gzip -5 --verbose 87.1% -- -- replaced with /var/log/ceph/ceph-client.admin.54156.log.gz /var/log/ceph/ceph-client.admin.49246.log 2024-09-10T23:42:02.612 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-10T23:42:02.613 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41415.log 2024-09-10T23:42:02.613 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46752.log.gz 2024-09-10T23:42:02.614 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35831.log 2024-09-10T23:42:02.614 INFO:teuthology.orchestra.run.smithi094.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.49246.log.gz 2024-09-10T23:42:02.615 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49134.log 2024-09-10T23:42:02.616 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41415.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41415.log.gz 2024-09-10T23:42:02.616 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52221.log 2024-09-10T23:42:02.616 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35831.log.gz 2024-09-10T23:42:02.617 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46591.log 2024-09-10T23:42:02.617 INFO:teuthology.orchestra.run.smithi094.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.49134.log.gz 2024-09-10T23:42:02.618 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54379.log 2024-09-10T23:42:02.619 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52221.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52221.log.gz 2024-09-10T23:42:02.619 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43920.log 2024-09-10T23:42:02.619 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46591.log.gz 2024-09-10T23:42:02.620 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40824.log 2024-09-10T23:42:02.620 INFO:teuthology.orchestra.run.smithi094.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54379.log.gz 2024-09-10T23:42:02.621 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48080.log 2024-09-10T23:42:02.622 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43920.log: /var/log/ceph/ceph-client.admin.40824.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43920.log.gz 2024-09-10T23:42:02.622 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54255.log 2024-09-10T23:42:02.622 INFO:teuthology.orchestra.run.smithi094.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40824.log.gz 2024-09-10T23:42:02.623 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42572.log 2024-09-10T23:42:02.624 INFO:teuthology.orchestra.run.smithi094.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48080.log.gz 2024-09-10T23:42:02.624 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35779.log 2024-09-10T23:42:02.625 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54255.log: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-10T23:42:02.625 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54255.log.gz 2024-09-10T23:42:02.625 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29029.log 2024-09-10T23:42:02.626 INFO:teuthology.orchestra.run.smithi094.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42572.log.gz 2024-09-10T23:42:02.626 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.35779.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35779.log.gz 2024-09-10T23:42:02.627 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.29029.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29029.log.gz 2024-09-10T23:42:02.635 INFO:teuthology.orchestra.run.smithi181.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-10T23:42:02.637 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-09-10T23:42:02.638 INFO:teuthology.orchestra.run.smithi181.stderr:real 0m1.036s 2024-09-10T23:42:02.638 INFO:teuthology.orchestra.run.smithi181.stderr:user 0m5.261s 2024-09-10T23:42:02.638 INFO:teuthology.orchestra.run.smithi181.stderr:sys 0m0.534s 2024-09-10T23:42:02.744 INFO:teuthology.orchestra.run.smithi094.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-10T23:42:02.788 INFO:teuthology.orchestra.run.smithi094.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-10T23:42:02.790 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-10T23:42:02.790 INFO:teuthology.orchestra.run.smithi094.stderr:real 0m1.224s 2024-09-10T23:42:02.790 INFO:teuthology.orchestra.run.smithi094.stderr:user 0m5.467s 2024-09-10T23:42:02.790 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m1.617s 2024-09-10T23:42:02.791 INFO:tasks.ceph:Archiving logs... 2024-09-10T23:42:02.791 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899790/remote/smithi094/log 2024-09-10T23:42:02.792 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T23:42:03.867 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899790/remote/smithi181/log 2024-09-10T23:42:03.868 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T23:42:04.415 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T23:42:04.421 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T23:42:04.422 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T23:42:04.425 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T23:42:04.526 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T23:42:04.535 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T23:42:04.535 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T23:42:04.539 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T23:42:04.551 INFO:teuthology.orchestra.run.smithi094.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T23:42:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:============================================================================== 2024-09-10T23:42:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:*hv01.front.sepi 67.205.162.81 3 u 12 64 377 0.089 -1.451 0.769 2024-09-10T23:42:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:+hv02.front.sepi 63.231.80.2 3 u 19 64 377 0.076 -2.252 0.835 2024-09-10T23:42:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:+hv03.front.sepi 74.6.168.72 3 u 8 64 377 0.079 -0.545 1.276 2024-09-10T23:42:04.552 INFO:teuthology.orchestra.run.smithi094.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-10T23:42:04.554 INFO:teuthology.orchestra.run.smithi181.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T23:42:04.554 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================== 2024-09-10T23:42:04.554 INFO:teuthology.orchestra.run.smithi181.stdout:+hv01.front.sepi 67.205.162.81 3 u 8 64 377 0.091 -2.080 1.206 2024-09-10T23:42:04.555 INFO:teuthology.orchestra.run.smithi181.stdout:+hv02.front.sepi 63.231.80.2 3 u 2 64 377 0.069 -0.850 1.780 2024-09-10T23:42:04.555 INFO:teuthology.orchestra.run.smithi181.stdout:*hv03.front.sepi 74.6.168.72 3 u 12 64 377 0.096 -2.059 1.211 2024-09-10T23:42:04.555 INFO:teuthology.orchestra.run.smithi181.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-10T23:42:04.555 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T23:42:04.600 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T23:42:04.600 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T23:42:04.608 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T23:42:04.617 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T23:42:04.625 INFO:teuthology.task.internal:Duration was 1302.346547 seconds 2024-09-10T23:42:04.625 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T23:42:04.634 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T23:42:04.634 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T23:42:04.637 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T23:42:04.745 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T23:42:04.745 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T23:42:04.746 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T23:42:04.766 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2024-09-10T23:42:04.766 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T23:42:04.786 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T23:42:04.786 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T23:42:04.815 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T23:42:04.866 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T23:42:04.867 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T23:42:04.899 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T23:42:04.987 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T23:42:04.997 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T23:42:04.997 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T23:42:05.017 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T23:42:05.037 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T23:42:05.046 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:42:05.067 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:42:05.081 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-09-10T23:42:05.095 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2024-09-10T23:42:05.113 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:42:05.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:42:05.151 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T23:42:05.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:42:05.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T23:42:05.175 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T23:42:05.176 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899790/remote/smithi094 2024-09-10T23:42:05.176 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T23:42:05.222 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899790/remote/smithi181 2024-09-10T23:42:05.223 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T23:42:05.248 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T23:42:05.248 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T23:42:05.263 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T23:42:05.295 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T23:42:05.305 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T23:42:05.305 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T23:42:05.315 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T23:42:05.315 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T23:42:05.318 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T23:42:05.323 INFO:teuthology.orchestra.run.smithi094.stdout: 658141 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 23:42 /home/ubuntu/cephtest 2024-09-10T23:42:05.343 INFO:teuthology.orchestra.run.smithi181.stdout: 658141 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 23:42 /home/ubuntu/cephtest 2024-09-10T23:42:05.345 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T23:42:05.355 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T23:42:05.377 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} duration: 1302.3465473651886 flavor: default mon.a-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a mon.b-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a owner: scheduled_hyelloji@teuthology success: true 2024-09-10T23:42:05.377 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T23:42:05.470 INFO:teuthology.run:pass